From 3e4d765497633c8051b51ceb8f9bcfab436534d6 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 24 Jul 2026 16:41:46 -0400 Subject: [PATCH] V16: Rules 22-25 (trust calibration, error-feedback, attribution gate, backtest integrity) + land prior uncommitted Rule 15 two-phase exit rewrite --- cron-contracts/alpaca-trading-system.prose.md | 198 ++++++++++++++++-- 1 file changed, 179 insertions(+), 19 deletions(-) diff --git a/cron-contracts/alpaca-trading-system.prose.md b/cron-contracts/alpaca-trading-system.prose.md index 260450b..e13d8cc 100644 --- a/cron-contracts/alpaca-trading-system.prose.md +++ b/cron-contracts/alpaca-trading-system.prose.md @@ -483,20 +483,29 @@ correct but structurally insufficient — exits are not happening fast enough to sustain the reset. **Fix:** Do NOT wait for another 4h cycle. Classify as INTERNAL structural -deadlock. Generate exit orders for the 2 highest-beta positions (sourced -from portfolio_beta.py) that meet **relaxed** exit criteria: held ≥1 day -(instead of 3) OR profit ≥1% (instead of 5%). If no positions meet even -relaxed criteria, go to 50% cash — sell the 2 highest-beta positions at -market regardless of profit. The breaker is self-correct; the exits are not. +deadlock. Generate exit orders in two phases: -**Verify:** At least 2 exit orders submitted with status `accepted` or -`filled`. Re-compute position count — < 10/10. +*Phase 1 (Relaxed exits):* Select up to 2 positions that meet **relaxed** +exit criteria: held ≥1 day (instead of the adaptive STALE_EXIT_DAYS, +default 5) OR profit ≥1% (instead of the adaptive PROFIT_TAKE_PCT, default +5%). Among qualifying positions, exit the 2 with the **largest market +value** first — this maximizes deadlock-breaking impact by freeing the most +capacity. -**Escalate:** After 1 cycle — breaker cycling is a structural deadlock. The -contract has exhausted its self-resolution surface. Escalate with: (a) the -breaker history (IDs and timestamps of the last 3), (b) all current positions -with age + beta + unrealized P&L, (c) which exit conditions each position -failed, and (d) the specific relaxed criteria applied. +*Phase 2 (Emergency — 50% cash):* If no positions meet even relaxed +criteria, go to 50% cash — sell the 2 **highest-beta** positions (sourced +from `compute_position_beta()` in `scripts/portfolio_adaptation.py`) at +market regardless of profit, with market value as tiebreaker. The breaker +is self-correct; the exits are not. + +**Verify:** At least 2 exit orders executed with status `executed`. +Re-compute position count — must be < MAX_EQUITY_POSITIONS (default 5). + +**Escalate:** Immediately — breaker cycling is a structural deadlock and +the contract has exhausted its self-resolution surface. Escalate with: +(a) the breaker history (IDs and timestamps of the last 3), (b) all current +positions with age + beta + unrealized P&L, (c) which exit conditions each +position failed, and (d) the specific relaxed criteria applied. #### Rule 16: Data Pipeline Integrity — Stale Local Tables **Detect:** Any OSS sub-component or performance-summary field returns 0, None, @@ -635,6 +644,39 @@ coverage gap counts trend downward over rolling 90-day window. cycles — the system's data universe is shrinking, not expanding. Escalate with per-source coverage breadth trend. +#### Rule 20: Duplicate-Symbol Consensus Prevention (Position-Aware Sophie) + +**Detect:** Sophie's signal aggregation produces `action=execute` BUY signals +for tickers that are already held in the live portfolio. Ross Gate 6 will +invariably block these as duplicate-symbol — creating a perpetual cycle of: +generate BUY → block → gap analysis noise. This is structurally wasteful +and generates false-positive "GAP" verdicts that obscure genuine execution +failures. + +**Prevention (Sophie pre-filter, `sophie.py:_get_held_symbols()`):** +Before processing any signals, Sophie queries live broker adapters +(RobinhoodCrypto.get_positions() for crypto, AlpacaLiveAdapter.get_positions() +for equity) and builds a set of currently-held ticker symbols. For every +incoming bullish signal where the ticker is already held, Sophie SKIPs the +signal at the aggregation layer — the consensus never reaches Ross. +Bearish/sell signals on held positions are unaffected (valid exits). + +**Detection (gap_analysis.py, passive):** +`gap_analysis.py` now cross-references `consensus.last_skip_reason` and +`broker_orders` to distinguish "blocked-valid" from "genuinely missed." +Consensus with `last_skip_reason` matching `duplicate-symbol`, `Already holding`, +`no_position`, `circuit_breaker`, or `asset_not_found` is classified as +`blocked_valid` — NOT a gap. Consensus with any `broker_orders` row is also +excluded from gap counting. + +**Verify:** Gap analysis returns `Verdict: CLEAN` when all unexecuted consensus +are blocked-valid. Sophie logs `⏭️ ALREADY-HELD` for skipped bullish signals. +Zero duplicate-symbol blocks appear in Ross output for Sophie-generated consensus. + +**Escalate:** Never — blocked-valid is expected behavior, not a failure. +The pre-filter eliminates the cycle at source; gap_analysis correctly +classifies any remaining blocked-valid consensus. + ### Invariants - **P-001 — Paper trading is dead code.** The paper trading adapter @@ -898,17 +940,20 @@ with per-source coverage breadth trend. cycle. **Tiered, fail-closed health — defined relative to what is achievable.** The - adaptation controller SHALL read a health tier derived from clean-snapshot - availability, where the clean-snapshot floor `MIN_CLEAN` ramps with system - age (`clamp(age_days × 0.5, 3, 8)`) so a young or freshly-healed system is + adaptation controller SHALL read a health tier derived from the count of + DISTINCT CALENDAR DAYS carrying at least one clean snapshot (not raw clean + row count — multiple same-day snapshots SHALL NOT inflate the tier or + false-promote the controller to HEALTHY on a single day of data), where the + clean-day floor `MIN_CLEAN` ramps with system age + (`clamp(age_days × 0.5, 3, 8)`) so a young or freshly-healed system is never starved into permanent inaction: - - **HEALTHY** (clean ≥ `MIN_CLEAN`): full loop — propose AND commit to + - **HEALTHY** (clean_days ≥ `MIN_CLEAN`): full loop — propose AND commit to `.env.prod`. - - **DEGRADED** (1 ≤ clean < `MIN_CLEAN`): evaluate + auto-unwind run + - **DEGRADED** (1 ≤ clean_days < `MIN_CLEAN`): evaluate + auto-unwind run (bookkeeping / immune system), the proposal is computed and REPORTED, but SHALL NOT be committed to `.env.prod`. - - **CRITICAL** (clean = 0): hold all adaptation; only bookkeeping runs; emit - a loud alert. + - **CRITICAL** (clean_days = 0): hold all adaptation; only bookkeeping runs; + emit a loud alert. **Self-healing by natural refill.** Because poisoned history is quarantined rather than resurrected, the OSS window refills with genuine post-inception @@ -1226,3 +1271,118 @@ the system-health contract detects, adapts, and self-heals. - `cli:python3`: required for running auditor_query.py and track scripts - `cli:sqlite3`: required for direct database queries when auditor is unavailable - `cli:curl`: required for DNS reachability tests and API health checks +--- + +## V17.2 Additions — Council-Verified (2026-07-15) + +### Pre-Sophie Completeness Gate (H6) + +The pipeline SHALL verify that all 9 signal tracks have produced data (via +`track_status.last_run_at` timestamps) before running Sophie consensus. If any +track is stale, the pipeline SHALL emit a WARNING but SHALL NOT block — partial +data is better than no consensus. This gate prevents Sophie from running on +empty or incomplete signal data after a cascade of track failures. + +**Implementation:** `consensus/run_pipeline.py:213-245` + +### Market Holiday Calendar (L2) + +The system SHALL maintain a zero-dependency static calendar of NYSE market +holidays (`consensus/market_holidays.json`) covering the current and next two +calendar years. All signal tracks SHALL check `is_market_open()` before +initiating yfinance API calls on weekdays. The holiday data SHALL be refreshed +quarterly by a council-verified cron job with explicit stale-holiday detection. + +**Implementation:** `consensus/market_holidays.json`, `consensus/market_calendar.py` +**Cron:** `b53521c1e0cd` (quarterly validation) + +### Cron Health Monitor (C2 + W1) + +The system SHALL maintain a standalone heartbeat cron job that checks ALL +trading pipeline, watchdog, and infrastructure jobs for staleness every 15 +minutes. If the health monitor itself stops reporting, its absence SHALL be +detectable by the config hash watchdog. The monitor covers 22 jobs across three +categories: trading (11), watchdog (3), and infrastructure (8). + +**Implementation:** `~/.hermes/scripts/cron_health_check.py` +**Cron:** `d9adf1bf25a8` (every 15 min) + +### Validation Promotion Pipeline (C1) + +The validation buffer SHALL permanently promote the Rico RSI track to +PRE_PROMOTED status (bypassing the validation gate), reflecting its proven +signal quality. Luna and Iris SHALL require 5 and 3 consecutive time-based +cycles respectively before promotion. This prevents validation starvation where +new tracks accumulate zero-signal cycles and never exit the validation phase. + +**Implementation:** `consensus/validation_buffer.py:33-41` + +### Discovery Deadlock Skip (C3) + +When Ross discovery encounters 3+ consecutive timeout cycles against the Alpaca +API, the pipeline SHALL skip discovery for that cycle and use the frozen ticker +universe. This prevents the pipeline from stalling indefinitely when the Alpaca +discovery endpoint is unavailable. The timeout SHALL be 180 seconds. + +**Implementation:** `consensus/run_pipeline.py:89-150` + +### Ticker Denylist (C5) + +The system SHALL maintain a shared denylist of non-tradable ticker symbols +(`PORTFOLIO`, `REGIME`, `MACRO`) in the data validation module. All signal +tracks that fetch per-ticker OHLCV data SHALL filter their ticker universe +against this denylist before making yfinance API calls. This prevents spurious +OHLCV validation failures from contaminating track status. + +**Implementation:** `consensus/data_validation.py:23` + +### Infrastructure Monitoring (F7) + +The cron health monitor SHALL be expanded to include EOD Digest, Adaptation +Cycle, Reconcile Watchdog, Health Monitor, DB Trigger Watchdog, Jobs.json +Watchdog, Config Hash Watchdog, and Market Holiday Calendar. Per-job staleness +thresholds SHALL account for weekend gaps and trading-hours-only schedules. + +**Implementation:** `~/.hermes/scripts/cron_health_check.py:45-56` + +#### Rule 22: Calibrated Trust in Consensus (R3/R4) +**Detect:** A track's rolling 60-day Brier score exceeds 0.25 (worse than coin-flip +calibration) with n_obs ≥ 20. +**Fix:** Trust weight auto-discounts via `track_trust` (mapping in +scripts/calibrate_tracks.py). Consensus uses weighted_score ≥ 1.6 instead of raw +track count. Trust floor 0.30 — tracks are discounted, never silenced. +**Verify:** `SELECT track, trust, brier, n_obs FROM track_trust` shows fresh rows +(computed_at within 24h on trading days). +**Escalate:** If ALL tracks sit at trust floor for 5 consecutive cycles — the +signal population itself is degrading; escalate with per-track Brier history. + +#### Rule 23: Error-Feedback Suppression (R5) +**Detect:** A track produces 2+ consecutive wrong outcomes on the same +ticker+direction within 10 days (signal_outcomes ledger). +**Fix:** Sophie drops that track from that ticker's group for the current cycle +(bypass tracks janet/eddie exempt). Suppression lifts automatically when the +streak breaks. +**Verify:** Pipeline logs show `MEM-SUPPRESS` lines only when ledger streaks ≥ 2. +**Escalate:** If one track is suppressed on >50% of its signals for 5 cycles — +the track is systematically miscalibrated; escalate for track review. + +#### Rule 24: Edge Claims Require Attribution (R1/R8) +**Detect:** Any report (OSS, health monitor, EOD digest) claiming outperformance +vs SPY. +**Fix:** The claim must cite scripts/attribution.py output and is only valid when +`selection_alpha_significant=true` (n_days ≥ 60 AND |t-stat| ≥ 2). Otherwise the +report must read "performance attribution insufficient — return may be beta/noise." +**Verify:** Health Monitor output includes the attribution verdict line verbatim. +**Escalate:** If attribution shows significant NEGATIVE alpha (t ≤ −2) for 20+ +trading days — the strategy destroys value vs beta; escalate with full +attribution JSON. + +#### Rule 25: Backtest Integrity (R1/R2) +**Detect:** Any parameter change justified by backtest results. +**Fix:** The justification must name the walk-forward windows used (train vs test +from scripts/backtest.py::walk_forward_windows). Same-window tuning is invalid +and must be rejected on review. Frictions of 30 bps equity / 50 bps crypto per +side must have been charged. +**Verify:** Backtest JSON contains `frictions_bps` and the report cites +train/test window pairs. +**Escalate:** Never to automation — integrity violations go to human review.