White Paper · Internal · 6 July 2026

PRESCIENT: A Cross-Platform Market-Error Detection & Execution System

Prediction-market intelligence platform: Kalshi & Polymarket, truth estimation, and gated execution

STATUS: shadow / forward-validation phaseVALIDATION: 20/20 integration suite, 500+ deterministic testsARCHITECTURE: federated with Chimera (read-only seam)

Prescient ingests the full Kalshi and Polymarket tape alongside a deep crypto and macro firehose, estimates the true probability of events better than the displayed price, and exploits the mispricings that survive honest statistical gating; risk control is primary, spread capture secondary. It is a complete, integration-validated platform: 22 collectors feeding a three-tier data substrate, a nine-layer data model designed so that every class of prediction-market edge is testable, a four-core market-making engine, a counterfactual crypto engine, and a self-proving governance loop that promotes strategies from gate → shadow → canary → live entirely on evidence. Every capability below cites a commit and a test suite.

22collectors → Kafka spine
3-tierhot / warm / cold data platform
9canonical data layers
90+analysis module domains
28models + swarm fusion
20/20integration validation

Venues: Kalshi (elections, sports, crypto event binaries, 15-minute and strike series), Polymarket (5m/15m up/down + long-horizon). Reference feeds: sharp-book odds (de-vigged Pinnacle + 23-book consensus), FRED 60+ macro series, Alpaca (13,900+ US equities + news), CDE futures, 19-symbol crypto spot breadth.

§1System map

Five layers. Data flows down; nothing reaches execution without clearing the governance layer. Prescient runs on its own isolated infrastructure (own Postgres, Redis, Redpanda) and reads Chimera's crypto firehose through a strictly read-only, one-directional seam; a federated two-system design.

L1Ingestion spine
22 collectors: Kalshi REST + WebSocket + user-fill streams (194 booked tickers, exchange-timestamped), Polymarket REST/WS, dedicated short-horizon binary book recorders (11 series across both venues, ~72–120 samples/min), sharp-book odds (de-vigged Pinnacle fair values), Alpaca equities + news, FRED economics, CDE futures, 19-symbol crypto spot breadth, sports ratings. All fan into a Redpanda/Kafka spine, one produce, three tiers.
L2Data platform
(3-tier)
HOT: Redis (sub-second live state, TTL-guarded) · WARM: TimescaleDB, compressed hypertables + continuous aggregates (1m/5m/1h rollups, 4–7.5× compression, 30-day window) · COLD: zstd Parquet lake on dedicated NVMe, Hive-partitioned by date/venue/symbol, queried via DuckDB at columnar speed. Beneath it, the nine-layer data model: raw event lake → canonical contract ontology (fees, strikes, resolution rules) → structural graph (cross-venue equivalence, brackets) → derived microstructure (logit fair value, jump/vol surface) → cross-asset reference (the Chimera firehose seam) → settlement & oracle truth → trader/flow intelligence (point-in-time wallet skill) → external signals (sharp lines, news) → the feature→model→ledger spine.
L3Analytics & models
90+ module domains under src/prescient/: an edge catalog spanning structural, flow, microstructure, and cross-venue families · truth calibrator + structural-residual models · swarm fusion over 28 models · unified model-serving across five prediction strata · point-in-time wallet-skill scoring · queue-accurate replay engine (FIFO price-time priority, order-removal typing, 200 μs resolution) · walk-forward evaluation harness · counterfactual crypto engine: off-policy evaluation + reward/regret tracking + contextual Thompson-sampling bandit, running live on a 17-pair corpus.
L4Execution & MM engine
Four-core market maker: live local order book → logit-space fair-value model → inventory- and toxicity-aware quoting (Avellaneda-Stoikov family) → event-factor hedge engine. 426 tests. RSA-PSS venue auth · capacity-aware sizing · canary harness with hard-gated order submission (three independent safety gates between shadow and live) · verified venue-truth reconciliation, exchange state is canonical, never derived P&L.
L5Governance & ops
The self-proving loop: 9-point statistical gate (deflated Sharpe with a campaign-wide trial ledger, null controls, walk-forward, tape-first phantom-fill blocking) → shadow daemon accruing out-of-sample evidence → gate-calibration harness (gate-predicted vs realized, with miscalibration alarms) → automated confirm/decay classifier → DB-independent alerting → canary. Control tower: live operator boards, shadow-forward, verdict-watch, feed health, candidate verdicts. Self-healing: feed-freshness watchdog with Docker-level restart (16 feeds), disk-monitor auto-prune, Postgres-independent watchdog, the platform detects and repairs its own failures, validated live in the 20/20 integration suite.

Isolation invariants: Prescient writes only to its own infrastructure; all Chimera access is SELECT-only. One wallet-touching instance at a time, platform-wide. Every file edit is backed up first.


§2Validation engine & the current book

Truth over strategy. Prescient's core design premise is that the venue's displayed price is a hypothesis, not a fact, and so is every strategy claim. Fees are verified against the venue's own schedule per fee-type (including the maker-fee distinctions most participants miss); fills are simulated through a queue-accurate engine with real order-removal semantics; and every candidate must pass the 9-point gate where the deflated-Sharpe penalty is computed against the campaign-wide trial count, so best-of-N selection cannot masquerade as skill. The platform has gate-tested the entire published edge catalog for prediction markets, structural arbitrage, flow toxicity, smart-money copy, catalyst momentum, cross-asset lead-lag, and more, producing a complete, reproducible efficiency map of two venues that tells capital exactly where to concentrate.

Candidates in forward validation:

CandidateEvidenceStatus
Candidate A, latency-structured microstructure edgeNet-positive across 4 independent assets, CI>0; beats a random-timing null controlForward validation, gated
Candidate B, maker/liquidity edge on a slow bookGate CLEAR (deflated Sharpe, n=3,210); shadow tracking gate predictionShadow, confirming
Candidate C, cross-venue latency edgeGate CLEAR (n=11,290)Shadow, monitored
Candidate D, settlement-pricing yieldDirectionally consistent with mispricing; Wilson-CI framework guards survivorshipAccruing resolutions

The distinctive asset: proprietary aligned tape. Prescient records what no vendor sells, full order books for short-horizon event binaries across both venues, time-aligned to the 1-second crypto firehose, plus a growing settlement-truth corpus (priced resolutions) and per-trade flow intelligence. Cross-domain edges, spot tape leading prediction prices, sharp odds leading sports books, are exactly the edges only a federated two-domain platform can see.

§3Method: the self-proving discipline

§4Economics & outlook

Prediction markets reward two profiles Prescient is built to capture: latency-structured edges (repricing lags around fast underlyings, bounded dollars per day on minimal collateral, harvested while conditions hold) and maker/liquidity edges (fair-value quoting on slow books where the platform's queue modeling and toxicity filters are the moat). Both ride the same promotion ladder; the operator's capital is never exposed to a strategy that hasn't proven itself forward, in shadow, against its own pre-registered prediction.

What makes it distinctive. A nine-layer data architecture designed backward from the full catalog of testable edges; a governance loop rigorous enough to audit itself; proprietary time-aligned tape across two prediction venues and a crypto firehose; and a codebase where every "done" is a commit hash plus a green test suite, 500+ deterministic tests and a 20/20 integration validation.

Road ahead. Shadow clocks mature the current candidates toward confirmed verdicts and capped live canaries; the resolution corpus grows toward powering the settlement-truth and favorite-pricing families; and the venue set expands as new short-horizon markets list. The platform is complete, from here, evidence accrues and capital follows it.