Prescient ingests the full Kalshi and Polymarket tape alongside a deep crypto and macro firehose, estimates the true probability of events better than the displayed price, and exploits the mispricings that survive honest statistical gating. It is a complete, integration-validated platform: 22 collectors feeding a three-tier data substrate, a nine-layer data model designed so that every class of prediction-market edge is testable, a four-core market-making engine, a counterfactual crypto engine, and a self-proving governance loop that promotes strategies from gate → shadow → canary → live entirely on evidence. Every capability below cites a commit and a test suite.
Venues: Kalshi (elections, sports, crypto event binaries, 15-minute and strike series), Polymarket (5m/15m up/down + long-horizon). Reference feeds: sharp-book odds (de-vigged Pinnacle + 23-book consensus), FRED 60+ macro series, Alpaca (13,900+ US equities + news), CDE futures, 19-symbol crypto spot breadth.
§1System map
Five layers. Data flows down; nothing reaches execution without clearing the governance layer. Prescient runs on its own isolated infrastructure (own Postgres, Redis, Redpanda) and reads Chimera's crypto firehose through a strictly read-only, one-directional seam; a federated two-system design.
L1Ingestion spine
22 collectors: Kalshi REST + WebSocket + user-fill streams (194 booked tickers, exchange-timestamped), Polymarket REST/WS, dedicated short-horizon binary book recorders (11 series across both venues, ~72–120 samples/min), sharp-book odds (de-vigged Pinnacle fair values), Alpaca equities + news, FRED economics, CDE futures, 19-symbol crypto spot breadth, sports ratings. All fan into a Redpanda/Kafka spine — one produce, three tiers.
L2Data platform (3-tier)
HOT: Redis (sub-second live state, TTL-guarded) · WARM: TimescaleDB, compressed hypertables + continuous aggregates (1m/5m/1h rollups, 4–7.5× compression, 30-day window) · COLD: zstd Parquet lake on dedicated NVMe, Hive-partitioned by date/venue/symbol, queried via DuckDB at columnar speed. Beneath it, the nine-layer data model: raw event lake → canonical contract ontology (fees, strikes, resolution rules) → structural graph (cross-venue equivalence, brackets) → derived microstructure (logit fair value, jump/vol surface) → cross-asset reference (the Chimera firehose seam) → settlement & oracle truth → trader/flow intelligence (point-in-time wallet skill) → external signals (sharp lines, news) → the feature→model→ledger spine.
L3Analytics & models
90+ module domains: an edge catalog spanning structural, flow, microstructure, and cross-venue families · truth calibrator + structural-residual models · swarm fusion over 28 models · unified model-serving across five prediction strata · point-in-time wallet-skill scoring · queue-accurate replay engine (FIFO price-time priority, order-removal typing, 200 µs resolution) · walk-forward evaluation harness · counterfactual crypto engine: off-policy evaluation + reward/regret tracking + contextual Thompson-sampling bandit, running live on a 17-pair corpus.
L4Execution & MM engine
Four-core market maker: live local order book → logit-space fair-value model → inventory- and toxicity-aware quoting (Avellaneda–Stoikov family) → event-factor hedge engine. 426 tests. RSA-PSS venue auth · capacity-aware sizing · canary harness with hard-gated order submission (three independent safety gates between shadow and live) · verified venue-truth reconciliation — exchange state is canonical, never derived P&L.
L5Governance & ops
The self-proving loop: 9-point statistical gate (deflated Sharpe with a campaign-wide trial ledger, null controls, walk-forward, tape-first phantom-fill blocking) → shadow daemon accruing out-of-sample evidence → gate-calibration harness (gate-predicted vs realized, with miscalibration alarms) → automated confirm/decay classifier → DB-independent alerting → canary. Control tower: live operator boards, shadow-forward, verdict-watch, feed health, candidate verdicts. Self-healing: feed-freshness watchdog with Docker-level restart (16 feeds), disk-monitor auto-prune, Postgres-independent watchdog — the platform detects and repairs its own failures, validated live in the 20/20 integration suite.
Isolation invariants: Prescient writes only to its own infrastructure; all Chimera access is SELECT-only. One wallet-touching instance at a time, platform-wide. Every file edit is backed up first.
§2Validation engine & the current book
Truth over strategy. Prescient's core design premise is that the venue's displayed price is a hypothesis, not a fact — and so is every strategy claim. Fees are verified against the venue's own schedule per fee-type (including the maker-fee distinctions most participants miss); fills are simulated through a queue-accurate engine with real order-removal semantics; and every candidate must pass the 9-point gate where the deflated-Sharpe penalty is computed against the campaign-wide trial count, so best-of-N selection cannot masquerade as skill. The platform has gate-tested the entire published edge catalog for prediction markets — structural arbitrage, flow toxicity, smart-money copy, catalyst momentum, cross-asset lead-lag, and more — producing a complete, reproducible efficiency map of two venues that tells capital exactly where to concentrate.
Candidates in forward validation
| Candidate | Evidence | Status |
|---|---|---|
| Candidate A, latency-structured microstructure edge | Net-positive across 4 independent assets, CI>0; beats a random-timing null control | Forward validation, gated |
| Candidate B, maker/liquidity edge on a slow book | Gate CLEAR (deflated Sharpe, n=3,210); shadow tracking gate prediction | Shadow, confirming |
| Candidate C, cross-venue latency edge | Gate CLEAR (n=11,290) | Shadow, monitored |
| Candidate D, settlement-pricing yield | Directionally consistent with mispricing; Wilson-CI framework guards survivorship | Accruing resolutions |
The distinctive asset: proprietary aligned tape. Prescient records what no vendor sells — full order books for short-horizon event binaries across both venues, time-aligned to the 1-second crypto firehose, plus a growing settlement-truth corpus (priced resolutions) and per-trade flow intelligence. Cross-domain edges — spot tape leading prediction prices, sharp odds leading sports books — are exactly the edges only a federated two-domain platform can see.
§3Method: the self-proving discipline
- Gate → shadow → canary → live. A candidate must clear the 9-point gate, then track its gate-predicted economics in live shadow over a fixed out-of-sample window, then earn an automated confirmed verdict, then receive an explicit operator go — only then does the canary's hard-blocked
submit()unlock, at capped size. - The gate is itself audited. A calibration harness compares what the gate predicted to what shadow delivered, per candidate, with miscalibration alarms — the validator is validated.
- Venue truth is canonical. All P&L reconciles to exchange state; fees are verified per fee-type against the venue's own API; fills come from recorded books, never assumptions.
- Every claim cites proof. Each capability carries a commit hash and test count; the full platform passes a 20-scenario end-to-end integration suite, including a live self-healing drill.
§4Economics & outlook
Prediction markets reward two profiles Prescient is built to capture: latency-structured edges (repricing lags around fast underlyings, bounded dollars per day on minimal collateral, harvested while conditions hold) and maker/liquidity edges (fair-value quoting on slow books where the platform's queue modeling and toxicity filters are the moat). Both ride the same promotion ladder; the operator's capital is never exposed to a strategy that hasn't proven itself forward, in shadow, against its own pre-registered prediction.
What makes it distinctive. A nine-layer data architecture designed backward from the full catalog of testable edges; a governance loop rigorous enough to audit itself; proprietary time-aligned tape across two prediction venues and a crypto firehose; and a codebase where every "done" is a commit hash plus a green test suite — 500+ deterministic tests and a 20/20 integration validation.
Road ahead. Shadow clocks mature the current candidates toward confirmed verdicts and capped live canaries; the resolution corpus grows toward powering the settlement-truth and favorite-pricing families; and the venue set expands as new short-horizon markets list. The platform is complete — from here, evidence accrues and capital follows it.
Prescient is built and operated by Daniel Roberts. Public edition; operational specifics redacted. Companion research: Chimera (autonomous trading research & execution) and Insta-Apply (autonomous career operations).
Live operations: daniel-roberts.com/live · daniel-roberts.com