From ca4f18e53974e2cd9ecd7b58da71989e721c57af Mon Sep 17 00:00:00 2001 From: Hermes Agent Date: Wed, 1 Jul 2026 12:05:52 -0400 Subject: [PATCH] =?UTF-8?q?alpaca:=20Full=20remediation=20(Gates=200-4=20c?= =?UTF-8?q?omplete)=20=E2=80=94=20drawdown-first=20objective,=20P-001=20th?= =?UTF-8?q?rough=20P-011=20invariants?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../CHECKPOINT-alpaca-remediation.md | 391 ++++++ cron-contracts/alpaca-trading-system.prose.md | 1161 +++++++++++++++++ 2 files changed, 1552 insertions(+) create mode 100644 cron-contracts/CHECKPOINT-alpaca-remediation.md create mode 100644 cron-contracts/alpaca-trading-system.prose.md diff --git a/cron-contracts/CHECKPOINT-alpaca-remediation.md b/cron-contracts/CHECKPOINT-alpaca-remediation.md new file mode 100644 index 0000000..5866bf8 --- /dev/null +++ b/cron-contracts/CHECKPOINT-alpaca-remediation.md @@ -0,0 +1,391 @@ +# CHECKPOINT — Alpaca Trading System Remediation Plan +**Status:** GATE 0 COMPLETE · **Created:** 2026-07-01 · **Resumed:** 2026-07-01T14:35Z +**Pick-up instruction for a new session:** Read this file top-to-bottom. Gate 0 is done; resume at Gate 1. + +--- + +## ✅ GATE 0 — COMPLETE (2026-07-01T14:46Z) + +### OSS Before/After + +| Metric | Before (broken) | After (fixed) | Delta | +|--------|-----------------|---------------|-------| +| `oss` | 1.094 (above target!) | **-0.605** (below floor) | −1.699 | +| `oss_trend` | "improving" | **"degrading"** | ✓ fixed | +| `calmar` | 2.174 (sign destroyed) | **-2.174** | sign restored | +| `spy_30d_return` | 0.58 (hardcoded) | **-1.44** (live) | real benchmark | +| `rolling_alpha` | -1.01 (wrong) | **1.01** (portfolio beat SPY) | flipped | +| `portfolio_beta` | -0.74 (wrong) | **0.30** (correct) | sign fixed | +| `win_rate_adaptations` | 0.5 (dead code) | **removed** | dead code excised| +| `benchmark_error` | — | **null** | fetch succeeded | + +### Changes Made + +| Gate | File | Changes | +|------|------|---------| +| 0.1 | `scripts/compute_oss.py:65-66` | `abs(cagr)` → `cagr`; floor `0.0` → `-5.0` | +| 0.2 | `scripts/compute_oss.py:344-362` | Removed dead `_compute_adapt_win_rate()` + call site | +| 0.3 | `scripts/compute_oss.py:45-61` | Added `_fetch_spy_30d_return()` (yfinance via ^GSPC) | +| 0.3 | `scripts/compute_oss.py:293-301` | Replaced hardcoded 0.58 with live fetch; alpha=None on failure | +| 0.3 | `scripts/compute_oss.py:334` | `_compute_beta` now accepts live spy_return param | +| 0.3 | `scripts/compute_oss.py:328-329` | JSON output now includes `spy_30d_return` + `benchmark_error` | +| 0.4 | `tests/test_oss_monotonicity.py` | NEW — 3 property tests (Calmar sign, monotonicity, known values) | +| 0.4 | `tests/test_benchmark_integrity.py` | NEW — 3 tests (no 0.58 literal, fetch signature, alpha sign mock) | +| 0.4 | `~/.hermes/scripts/contract_invariant_watchdog.py` | Added 2 new test files to TEST_FILES list (now 7 files) | + +### Verification + +- `compute_oss.py` runs clean: OSS=-0.605, trend=degrading, live SPY=-1.44% +- `tests/`: 36 passed (30 original + 6 new) +- Watchdog: "✅ All contract invariants hold (7 test files passed)" + +### Next: Gate 1 — Make the optimization loop load-bearing + +--- + +## ✅ GATE 1 — COMPLETE (2026-07-01T15:00Z) + +### Changes Made + +| Sub-gate | File | Changes | +|----------|------|---------| +| 1.1 | `scripts/migrate_gate1_schema.py` | NEW — creates `adaptations` + `parameter_state` tables, seeds 5 params | +| 1.1 | `trade_journal.db` | `parameter_state` (5 params) + `adaptations` (0 rows, clean) | +| 1.2 | `scripts/apply_adaptations.py` | NEW — reads OSS+regime, proposes bounded changes, writes DB | +| 1.3 | `scripts/apply_adaptations.py:auto_unwind()` | Auto-reverts params after 3 consecutive non-positive forward_oss_delta | +| 1.4 | `scripts/compute_oss.py:W_ADAPT=0.10` | Re-weighted: W_CALMAR=0.35, W_WIN_RATE=0.10, W_ADAPT=0.10 | +| 1.4 | `scripts/compute_oss.py:_compute_adapt_win_rate()` | Queries `adaptations.result='improved'` (not dead `analysis_runs`) | +| 1.5 | `tests/test_adaptation_persistence.py` | NEW — 3 tests (seeded params, write+unwind, migration idempotent) | +| 1.5 | `~/.hermes/scripts/contract_invariant_watchdog.py` | Added test_adaptation_persistence.py (now 8 test files) | + +### Adaptation Logic + +| OSS Range | Trend | Regime | Action | +|-----------|-------|--------|--------| +| < 0.5 | degrading | bearish | **Tighten**: lower STOP_LOSS_PCT, raise CONFIDENCE_FLOOR, lower HEDGE_CAP | +| > 0.8 | improving | bullish | **Loosen**: raise STOP_LOSS_PCT, lower CONFIDENCE_FLOOR, raise HEDGE_CAP | +| Any | — | neutral | **Hold** — no change needed | + +- Step size: 10% of (max−min) range +- Bounds enforced from `parameter_state` +- Auto-unwind after 3 consecutive non-positive forward_oss_delta + +### Verification + +- `apply_adaptations.py` runs clean: proposes STOP_LOSS_PCT adaptation in bearish regime +- Auto-unwind test: 3 failed adaptations → reversion to original value +- `compute_oss.py` shows `AdaptWR=0.5` (neutral default, wired to real data) +- **39 tests pass** (30 original + 6 Gate 0 + 3 Gate 1) +- **Watchdog: "✅ All contract invariants hold (8 test files passed)"** +- LLM cron `2ca03134ea62` is now reporter, not actor — calls `apply_adaptations.py` + +### Next: Gate 2 — Make hedging capable + +--- + +## 0. ORIENTATION (read first) + +**Repo:** `/opt/alpaca-trading-agent` · **venv:** `.venv/bin/python` +**Contract (source of truth):** `/opt/openprose-contracts/cron-contracts/alpaca-trading-system.prose.md` +**Full audit narrative:** `/opt/openprose-contracts/cron-contracts/alpaca-adversarial-audit-2026-07-01.md` +**Watchdog:** `~/.hermes/scripts/contract_invariant_watchdog.py` (cron job `85522a081e55`, daily 08:00, no_agent, silent on pass) +**Adaptation cron (LLM-mode):** job `2ca03134ea62` (DeepSeek-v4-pro, pinned) + +**DBs:** +- `trade_journal.db` — perf, orders, trades, `analysis_runs`, `config_store`. **This is the DB `compute_oss.py` reads** (`DB_PATH`, line 18). +- `consensus/signals.db` — `signals`, `consensus`. + +**Governing design principle (Theo, non-negotiable):** the contract must be **declarative** — declare desired *outcome states* + expose *levers* + define *feedback signals*, IaC-style. NOT prescriptive step lists. Every fix below converts a frozen constant into a tuned lever or adds a missing feedback signal. Wherever you must hardcode, justify it as an irreducible exception. + +**[fv] discipline for this work:** internal claims (code lines, DB columns) are verified by direct inspection (stronger than search — it's ground truth). External claims (financial definitions, SOTA findings) require independent `web_search` per claim. Label every assertion CONFIRMED / DISPUTED / UNVERIFIED. Add a property test that would have caught each defect before shipping the fix. + +--- + +## 1. VERIFIED CURRENT STATE (the evidence base) + +### Live OSS output (2026-07-01T14:19Z) — the smoking gun +``` +oss=1.094 (target 1.0, floor 0.5) oss_trend="improving" +calmar=2.174 profit_factor=0.679 win_rate=0.5 +total_return_pct=-0.43% sharpe=-1.38 realized_pnl_30d=-$1.67 over 24 trades +``` +A losing, negative-Sharpe, PF<1 portfolio scores ABOVE its aspirational target and reads "improving." +**VERIFY:** `cd /opt/alpaca-trading-agent && .venv/bin/python scripts/compute_oss.py | python3 -m json.tool` + +### Confirmed defects (all [fv] CONFIRMED by direct inspection) + +| ID | Defect | Exact location | Evidence | +|----|--------|---------------|----------| +| D1 | Calmar uses `abs(cagr)` — sign destroyed | `scripts/compute_oss.py:65` | `compute_calmar(-2.35,-0.43,30)=2.174` vs `(+0.43)=2.28` | +| D1b | Calmar clamp floor `max(calmar,0.0)` erases negatives even if sign fixed | `scripts/compute_oss.py:66` | `return round(min(max(calmar,0.0),5.0),3)` | +| D2 | Benchmark hardcoded | `scripts/compute_oss.py:274` (`portfolio_return - 0.58`), `:319` (`spy_30d_return = 0.58`) | never fetched; alpha+beta are self-referential | +| D3 | Win-rate query references 2 non-existent columns | `scripts/compute_oss.py:352` (`analysis_type`), `:349` (`result`) | `PRAGMA table_info(analysis_runs)` has NEITHER → bare `except:pass` at :360 → returns 0.5 forever | +| D4 | Hedge structurally incapable | `.env.prod` + `consensus/ross/config.py:84-86` | `MAX_HEDGE_POSITIONS=1`, `HEDGE_POSITION_SIZE_PCT=0.01` → max ~1% of equity; contract Rule 8 assumes 25% (`PARKER_HEDGE_CAP_PCT=0.25`, config.py:113) | +| D5 | P-004/005/006 unimplemented | grep entire tree | `hedge_strategy`, `discovery_lens`, `source_provenance` = 0 hits in *.py, 0 DB columns | +| D6 | No adaptation persistence | DB introspection | tables `parameter_state`/`adaptations`/`regime_state` do not exist; `config_store` untouched since 2026-05-25 | +| D7 | Discovery is static | `config_store.tickers` | 7 fixed symbols, last write 2026-05-22; frank=1, janet=1 distinct tickers ever | +| D8 | Position size flat, `POSITION_SIZE_PCT` dead | `consensus/ross/sizing.py:7`, `config.py:138` | `calculate_position_size` returns `cfg.MAX_POSITION_NOTIONAL` which `= MIN_NOTIONAL` ($5); the 0.025 pct is never used | +| D9 | Watchdog covers only P-002/P-003 | `contract_invariant_watchdog.py:19-25` | 5 test files = dust filter + equity source + pipeline; P-004/5/6 never tested | +| D10 | "Consensus" is single-track | `consensus/sophie.py` + signals.db | all executed decisions score=1; universes disjoint by design | + +### Confirmed ASSETS (things that already exist — reuse, don't rebuild) + +| Asset | Location | Use for | +|-------|----------|---------| +| Live market fetch (VIX/SPX/NDX/BTC via yfinance) | `consensus/track_parker.py:76-101` (`_fetch_market_data`) | D2 fix — reuse `spx.history(period="1mo")` for real SPY 30d return | +| Bounded tunable levers (already documented w/ min-max) | `config.py:99-113`: STOP_LOSS_PCT[-0.03,-0.07], PROFIT_TAKE_PCT[0.03,0.10], STALE_EXIT_DAYS[3,7], CONFIDENCE_FLOOR[0.65,0.85], PARKER_HEDGE_CAP_PCT[0.10,0.30] | D6 — levers EXIST; they need a *writer*, not a redesign | +| Property-test harness (hypothesis, temp-DB, module-reload) | `tests/test_perf_equity_property_invariants.py` | template for all new invariant tests | +| Regime detection (VIX-trend + SPX-vs-SMA20) | `track_parker.py:152+` (`_detect_regime`) | D4 — regime signal exists to drive dynamic hedge sizing | +| slippage_bps / intended_price columns | broker_orders schema | P-008 cost-aware gate (data already captured, unused) | +| Risk-gate framework (Gates 0-7) | `consensus/ross/gates.py:14` (`check_risk_gates`) | insert cost-aware gate here | + +--- + +## 2. EXTERNAL EVIDENCE (July 2026 SOTA) — [fv] CONFIRMED against sources + +- **Calmar = signed CAGR ÷ |maxDD|** — negative return MUST yield negative Calmar. (Investopedia, Wikipedia, QuantifiedStrategies — CONFIRMED, ≥3 sources.) → D1/D1b are unambiguous bugs. +- **FINSABER (arXiv:2505.07078v5, KDD'26):** LLM edge deteriorates over long horizons/broad universe; agents "overly conservative in bull, overly aggressive in bear." Prescription: **prioritize trend detection + regime-aware risk controls over framework complexity.** CONFIRMED (abstract). +- **StockBench (arXiv:2510.02209):** most LLM agents don't beat buy&hold over 4mo, but ALL limit drawdown (-11/-14% vs -15.2%). → realistic capturable edge = **drawdown control, not alpha.** CONFIRMED. +- **Agent Market Arena (arXiv:2510.11695):** performance driven by **architecture (memory/risk/sizing), not LLM choice.** CONFIRMED. +- **CLQT (arXiv:2606.29771):** return leaderboards mislead; apparent alpha dissolves once look-ahead controlled → "mostly passive factor exposure." Evaluate capability, not single score. CONFIRMED. → OSS is exactly the single-number leaderboard being warned against. +- **Signal half-life/capacity (OpenAlgo):** turnover set by half-life; every trade pays half-spread+impact; net-of-cost is the only honest test. CONFIRMED. +- **Leveraged inverse decay (StockTitan 2026):** daily-reset -3x (SQQQ/SPXU) decays even when directionally right → prefer 1x inverse (SH/PSQ) for multi-day holds. CONFIRMED. Contract Rule 18 already says this; `MAX_HEDGE_HOLD_DAYS=3` + HEDGE_TICKERS still lists SQQQ/SPXU fight it. + +**Synthesis implication:** the honest declared outcome state is **"match benchmark net of costs while bounding drawdown,"** NOT "maximize growth." Re-weight the objective accordingly (P-011). + +--- + +## 3. THE PLAN (dependency-ordered gates; LLM-paced, no human timelines) + +> Each task: **DO** (exact edit) → **VERIFY** (command + expected result) → **TEST** (property/regression test to add). No task is "done" until its VERIFY passes and its TEST is committed. + +### GATE 0 — Restore metric truth (BLOCKING; everything downstream is meaningless until this lands) + +**0.1 — Fix Calmar sign + clamp** +- DO: `scripts/compute_oss.py:65` → `calmar = cagr / (abs(max_drawdown_pct) / 100.0)` (drop `abs(` on cagr). +- DO: `:66` → allow negatives: `return round(min(max(calmar, -5.0), 5.0), 3)` (floor -5.0, not 0.0). +- CONSIDER: the OSS aggregate weight math (`:278-280`) assumes non-negative components — after this, re-derive so a negative Calmar drags OSS below floor. Verify OSS can now go below 0.5 on a losing month. +- VERIFY: `.venv/bin/python -c "import sys;sys.path.insert(0,'scripts');from compute_oss import compute_calmar as c;print(c(-2.35,-0.43,30), c(-2.35,0.43,30))"` → first value must be **negative**, second positive. +- VERIFY: `.venv/bin/python scripts/compute_oss.py` → with current -0.43% return, `oss` must now be **< 0.5** and `oss_trend` must NOT be "improving". +- TEST: add `tests/test_oss_monotonicity.py` (hypothesis): `∀ dd, days, r1<0 50 bps + filled < intended×0.995 | +| test | `tests/test_hedge_sizing.py` | **NEW** — 6 tests (monotonicity, leverage tiers, bucket bounds, non-hedge passthrough) | +| wdog | `contract_invariant_watchdog.py` | Added `test_hedge_sizing.py` (now 9 test files) | + +### Hedge Sizing Buckets + +| Bearish Score | Bucket | Max Positions | Size/Position | 1x Hold Days | 3x Hold Days | +|--------------|--------|--------------|---------------|-------------|-------------| +| < 0.20 | no hedge | 0 | 0% | — | — | +| 0.20–0.39 | moderate | 1 | 1.0% | 5 | 2 | +| 0.40–0.69 | heavy | 2 | 1.5% | 5 | 2 | +| ≥ 0.70 | extreme | 3 | 2.0% | 5 | 2 | + +- Capped at `PARKER_HEDGE_CAP_PCT` from `parameter_state` (current: 0.25) +- 1x inverses (SH, BITI, PSQ): 5-day max hold (Rule 18 + StockTitan evidence) +- 3x inverses (SQQQ, SPXU): 2-day max hold (crisis-only, high decay) + +### New Gates + +| Gate | Scope | Enforcement | +|------|-------|------------| +| **Gate 8** | Hedge position count + notional cap | Hard block: returns False if at limit | +| **Gate 2.3** | Cost-aware execution (slippage_bps) | Advisory warning (non-blocking — data is retrospective) | + +### Verification + +- Sizing: SH (hedge, bearish=0.8) → $4 on $200 equity; SH (bearish=0.3) → $2; AAPL → $5 flat +- Hold days: SH (1x) → 5 days, SQQQ (3x) → 2 days ✓ +- **45 tests pass** (30 + 6 Gate 0 + 3 Gate 1 + 6 Gate 2) +- **Watchdog: "✅ All contract invariants hold (9 test files passed)"** +- OSS = −0.475 (unchanged — Gate 2 doesn't affect OSS computation) + +--- + +## ✅ GATE 3 — COMPLETE (2026-07-01T15:40Z) + +### Changes Made + +| Sub-gate | File | Changes | +|----------|------|---------| +| 3.1a | `consensus/ross/discovery.py` | **NEW** — DiscoveryLens framework: 3 built-in lenses (ParkerTrend, VolumeSurge, SectorRotation), registry, Source Health Score, correlation-admission gate, probationary sizing | +| 3.1b | `consensus/signals.db:signals` | Added `discovery_lens TEXT` + `source_provenance TEXT` columns | +| 3.1b | `trade_journal.db:broker_orders` | Added `discovery_lens TEXT` + `source_provenance TEXT` columns | +| 3.1c | `consensus/ross/sizing.py` | Probationary sizing: 50% notional for tickers with <3 trades or PF < 1.0 | +| 3.1c | `consensus/ross/executor.py` | Passes `db_path` to sizing for probationary check | +| 3.2 | `scripts/update_ticker_universe.py` | **NEW** — Replaces static config_store.tickers with lens-proposed, correlation-gated, aggregated universe (≥2 lenses → priority) | +| 3.3 | `scripts/update_ticker_universe.py` | Coverage-gap streaks + auto-broaden lens criteria after 3 consecutive gaps (Rule 19) | +| test | `tests/test_discovery_lens.py` | **NEW** — 9 tests (≥3 lens invariant, SHS empty-DB, correlation gate edge cases, probationary sizing, script idempotency, gap tracking) | +| wdog | `contract_invariant_watchdog.py` | Added `test_discovery_lens.py` (now 10 test files) | + +### Lens Architecture + +| Lens | Source | Max Tickers | Criteria | +|------|--------|-------------|----------| +| `parker_trend` | Parker macro regime signals (signals.db) | 10 | min 3 active tickers | +| `volume_surge` | Volume anomaly detection | 15 | min 3 active tickers | +| `sector_rotation` | Maya gold miners / critical minerals | 12 | min 3 active tickers | + +- **Source Health Score**: yield_rate × avg_PF × 100 × decay (0.85^months) — per lens, 90-day lookback +- **Correlation gate**: rejects proposed tickers with r ≥ 0.80 vs existing positions (20-day min history) +- **Probationary sizing**: 50% notional for tickers with <3 trades or PF < 1.0 +- **≥3 lens invariant**: universe update aborts if <3 lenses active +- **Rule 19 hindsight**: coverage-gap streaks tracked; criteria auto-broaden (1.3× factor) after ≥3 gaps + +### Dynamic Universe Pipeline + +``` +Active Lenses → propose_candidates() → correlation-gate → aggregate votes +(≥2 lenses = priority) → merge with existing proven tickers → write config_store +``` + +### Verification + +- Lens registry: 3 active lenses (`parker_trend`, `volume_surge`, `sector_rotation`) ✓ +- SHS on empty DB: 0.0 (no crash) ✓ +- Pearson: 1.0 perfect, -1.0 perfect negative ✓ +- Probationary: SH/untraded tickers → True ✓ +- Dynamic universe script: idempotent, preserves existing tickers ✓ +- Gap tracking: streaks initialized ✓ +- **54 tests pass** (30 + 6 Gate 0 + 3 Gate 1 + 6 Gate 2 + 9 Gate 3) +- **Watchdog: "✅ All contract invariants hold (10 test files passed)"** +- OSS = −0.477 (unchanged — Gate 3 doesn't affect OSS computation) + +--- + +## ✅ GATE 4 — COMPLETE (2026-07-01T15:50Z) + +### Contract Edits (human-approved) + +| Section | Change | +|---------|--------| +| §Goal "Return Maximization" → "Return Optimization" | Primary outcome = match benchmark net of costs, bound drawdown ≤25%; outperformance is secondary | +| OSS weights | Calmar 35→**40%**, Alpha 20→**15%**, Win Rate 10%, Adapt WR 10% (unchanged) | +| `parameter-state` | Added `oss_target` (0.5), `oss_floor` (0.3), `oss_weights` (per-weight bounds), `hedge_regime_buckets` (3 tiers) | +| Invariant list | Added **P-007** (benchmark integrity), **P-008** (cost-aware execution), **P-009** (metric monotonicity), **P-010** (adaptation reversibility), **P-011** (drawdown-first objective) | + +### Code Changes + +| File | Change | +|------|--------| +| `scripts/compute_oss.py` | W_CALMAR 0.35→**0.40**, W_ALPHA 0.20→**0.15** | +| `alpaca-trading-system.prose.md` | Full contract update: 3 sections edited + 5 invariants added | + +### OSS Impact + +| Metric | Before (Gate 3) | After (Gate 4) | +|--------|----------------|----------------| +| OSS | −0.477 | **−0.590** | +| Trend | degrading | degrading | +| Calmar weight | 35% | **40%** | +| Alpha weight | 20% | **15%** | + +The shift to drawdown-first weighting amplifies the honest signal: Calmar at −2.174 now drives the composite more heavily, producing a truer picture. + +### Verification + +- Contract updated per Theo's approval ✓ +- **54 tests pass** (unchanged — contract edits don't affect test count) +- **Watchdog: "✅ All contract invariants hold (10 test files passed)"** +- OSS −0.590 | Calmar −2.174 | SPY −1.1% | Beta 0.39 + +### Next: Gate 5 — Wire the loop end-to-end (cron + adaptation pipeline integration) + + +### GATE 3 — Real field of view (parallelizable with Gate 2) + +**2.1 — Dynamic hedge sizing.** Replace frozen `MAX_HEDGE_POSITIONS=1`/`HEDGE_POSITION_SIZE_PCT=0.01` with regime-driven sizing bounded by `PARKER_HEDGE_CAP_PCT` (up to the 0.25 already authorized). Sizing reads `parameter_state`. Reconcile `MAX_HEDGE_HOLD_DAYS=3` with Rule 18 (allow longer 1x-inverse holds). +**2.2 — P-004 hedge attribution.** Add `hedge_strategy` column to orders/trades + tagging; compute per-strategy profit factor; retire strategies <0.5. Demote SQQQ/SPXU (decay) to crisis-only; default SH/PSQ (1x) per Rule 18 + StockTitan evidence. +**2.3 — P-008 cost-aware gate.** New gate in `gates.py` using existing `slippage_bps`/`intended_price`: reject trades where expected edge < modeled round-trip cost (half-spread+commission+slippage). Directly addresses small-account cost drag. +- TEST: hedge sizing scales with regime severity; cost gate blocks a sub-cost-edge trade; P-004 attribution sums correctly. + +### GATE 3 — Real field of view (parallelizable with Gate 2) + +**3.1 — P-005/P-006.** Add `discovery_lens` + `source_provenance` columns + tagging; per-lens yield + Source Health Score; correlation-admission gate; probationary sizing for new tickers. +**3.2 — Dynamic universe.** Replace static `config_store.tickers` with lens-proposed candidates each cycle (declare: "≥3 active lenses, correlation-gated"; expose lens criteria as lever). +**3.3 — Rule 19 hindsight.** Write coverage-gap counts; broaden lens criteria when gaps detected. +- TEST: universe changes across cycles; correlation gate rejects a highly-correlated add; ≥3 lenses active invariant (P-005). + +### GATE 4 — Re-baseline objective to evidence (contract edit; depends on Gates 0-2) + +**4.1 — Rewrite objective declaratively (P-011).** Primary outcome = **match benchmark net of costs + bounded drawdown** (per StockBench/FINSABER), not "maximize growth." Re-weight OSS toward downside/drawdown metrics. Make hedge cap a regime-driven lever. +**4.2 — Convert remaining prescriptive constants** in the contract to `#### parameter-state` blocks: each = declared target + lever + bounds + feedback signal. The system tunes them; the contract stops fixing them. +**4.3 — Add P-007/008/009/010/011** to the invariant set + watchdog. + +--- + +## 4. NEW CONTRACT PROVISIONS (declarative form — to add in Gate 4) + +- **P-007 Benchmark integrity:** every alpha/beta computed vs a live point-in-time benchmark fetched this cycle; alpha sign flips when portfolio crosses benchmark. (Kills 0.58 literal.) +- **P-008 Cost-aware execution:** no trade admitted whose expected edge < modeled round-trip cost; feedback = realized slippage_bps vs intended_price. +- **P-009 Metric monotonicity:** OSS and every sub-metric sign-correct; a losing period can never score ≥ a winning one. (Property-tested.) +- **P-010 Adaptation persistence & reversibility:** every param change writes before/after + regime; auto-reverts if forward OSS doesn't improve over N evals. +- **P-011 Drawdown-first objective:** declared outcome = benchmark-match net of costs with bounded drawdown; hedge capacity sized to actually bound it. + +--- + +## 5. EXECUTION ORDER & GUARDRAILS +1. **Gate 0 first, always.** Until OSS sees losses, every adaptation optimizes against a lie. +2. Human gate before contract edits (Gate 4) per Theo's 2-agent + 1-human policy. Contract source of truth is Gitea `SyslogSolution/prose-contracts` — mirror any contract edit there, not just `/opt/openprose-contracts/`. +3. Every code change ships with a property/regression test that would have caught the defect it fixes. +4. Pin any new cron jobs (unpinned LLM crons break on provider switch). +5. After each gate: run `compute_oss.py` + watchdog; record OSS before/after as evidence. +6. [fv] label every claim in progress reports: CONFIRMED / DISPUTED / UNVERIFIED. + +## 6. OPEN ITEMS TO RE-VERIFY AT PICK-UP (state may drift) +- Re-run the live OSS command — confirm it still shows the inverted "improving" before you fix it (proves you're fixing a live bug, not a stale one). +- Confirm `PRAGMA table_info(analysis_runs)` still lacks `analysis_type` + `result`. +- Confirm `.env.prod` hedge caps unchanged (`MAX_HEDGE_POSITIONS`, `HEDGE_POSITION_SIZE_PCT`). +- Re-grep `hedge_strategy|discovery_lens|source_provenance` — confirm still 0 hits before building P-004/5/6. +- Skill hygiene: `verification-protocol` has TWO colliding copies (`productivity/` + `ra-h-os-sync/`) causing `skill_view` ambiguity errors. Load by full path or dedupe. diff --git a/cron-contracts/alpaca-trading-system.prose.md b/cron-contracts/alpaca-trading-system.prose.md new file mode 100644 index 0000000..47c401b --- /dev/null +++ b/cron-contracts/alpaca-trading-system.prose.md @@ -0,0 +1,1161 @@ +--- +kind: responsibility +name: alpaca-trading-system-health +description: > + Standing responsibility that monitors and optimizes the Alpaca multi-track + trading system. Two mandates: (1) Prime Directive — valid consensus-backed + trades SHALL reach the broker, with external dependency failures as the only + acceptable blocks. (2) Return Maximization — the system actively adapts + positions, tickers, and allocation to market regime to maximize risk-adjusted + returns, measured by a composite Optimization Success Score (Calmar Ratio + + Profit Factor + Rolling Alpha). All adaptations are reversible and bounded. +--- + +### Goal + +**Prime Directive:** Valid trades are not blocked by the Alpaca trading system. +Every block that prevents a Sophie-consensus `execute` decision from reaching +the broker is either self-healed within one cycle or escalated with a clear root +cause. External dependency failures are classified, never confused with internal +issues, and not counted against system health. + +**Return Optimization:** The primary objective is to **match the S&P 500 benchmark +net of costs while bounding peak-to-trough drawdown to ≤25%.** Risk-adjusted +outperformance is a secondary goal, pursued only when market-regime detection +confirms the system has an edge. The system measures its Optimization Success +Score (OSS) on a rolling 30-day window, adapts positions, tickers, and allocation +to market regime within bounded, reversible guardrails, and never chases returns +the market does not offer. The target OSS is ≥ 0.5 (benchmark-matching +institutional-grade); ≥ 1.0 is aspirational in favorable regimes. The floor is +0.3 (below which adaptation and hedging are required). + +### Requires + +- `kojo-audit`: a current structured audit of open orders, signal freshness + per track, DNS reachability, and reconcile drift — sourced from the Kojo + auditor cron job (a4476b04abab, runs 07:00 and 20:00 daily) +- `sophie-consensus`: the latest consensus decisions from Sophie (execute / + watch / skip per ticker) — sourced from the consensus table in signals.db + written by Sophie+Ross pipeline jobs (43ad4780620a, 8d027ddff088) +- `execution-results`: the latest Ross trade execution outcomes — sourced + from broker_orders in trade_journal.db + +### Maintains + +The current system health truth. Postcondition: every block is classified as +`internal` (actionable, must self-heal) or `external` (documented, escalated). +Postcondition: no internal block persists across two consecutive audit cycles +without escalation. Postcondition: the system never reports "0 issues" while +valid trades are blocked. + +#### system-health +Overall health: `healthy` (all gates pass, no blocks), `degraded` (non-fatal +blocks found, some trades may be affected), or `blocked` (trades are being +prevented from reaching the broker). Material: the status and a summary of +every active block. + +#### active-blocks +Every currently active block preventing trade execution. Each block carries: +`source` (gate/component), `ticker` (affected symbol), `reason` (machine-readable +classification), `classified` (internal | external), `first_seen` (timestamp), +`last_seen` (timestamp), `self_heal_attempts` (count). Material: the block set +and each block's classified/reason fields. + +#### stale-data +Every data source that is stale beyond its freshness threshold. Each entry +carries: `track` (eddie/maggie/frank/maya/janet/parker), `last_signal` +(timestamp), `hours_since`, `last_run_at` (from track_status), `threshold_hours`, +`blocking` (whether the staleness is currently preventing trades). Material: +the track name, hours_since, and blocking fields. + +#### failure-log +Rolling log of the last 50 failure events. Each event carries: `timestamp`, +`component`, `classification` (internal | external), `resolved` (boolean), +`resolution` (self-healed | escalated | transient-cleared), `duration_s` +(seconds until resolution). Immaterial: raw stack traces, full HTTP response +bodies. + +#### return-optimization +The composite Optimization Success Score and its components, computed on a +rolling 30-day window from trade_journal.db. OSS weights: Calmar Ratio 40%, +Profit Factor 25%, Rolling Alpha over SPY 15%, Win Rate 10%, Adaptation +Win Rate 10% (per P-010). + +- `oss`: the current OSS (0.0–5.0+) +- `oss_target`: 1.0 (institutional-grade), `oss_floor`: 0.5 (adaptation required) +- `calmar`: rolling CAGR ÷ max drawdown (absolute) +- `profit_factor`: gross profit ÷ gross loss (all closed trades, 30-day rolling) +- `rolling_alpha`: portfolio excess return over SPY benchmark (annualized %) +- `portfolio_beta`: rolling beta to SPY — measures how much of our return is + just tracking the market +- `win_rate_adaptations`: % of rotation/hedge adjustments that improved 7-day + forward return +- `last_computed`: timestamp of most recent OSS calculation +- `trend`: improving | flat | degrading — direction of OSS over last 3 readings + +Material: all numeric fields. Immaterial: individual trade P&L lines, raw +price data. + +Postcondition: OSS is computed at least once per trading day. Postcondition: +when OSS falls below 0.5, at least one adaptation rule fires within 2 cycles. +Postcondition: when portfolio_beta > 0.6 AND market_regime is bearish, a +hedge or rotation rule fires within 1 cycle. + +#### market-regime +The current detected market regime, refreshed each cycle from yfinance +(VIX, SPX 20-day SMA, Nasdaq 20-day SMA, BTC trend). Regimes are: + +- `bull`: VIX < 18 AND SPX above 20-SMA AND Nasdaq above 20-SMA +- `neutral`: VIX 18–25 OR mixed signals (SPX and Nasdaq diverge) +- `bear`: VIX > 25 OR (SPX below 20-SMA AND Nasdaq below 20-SMA) +- `crisis`: VIX > 35 AND SPX drawdown > 10% from 30-day high + +- `vix_close`: latest VIX close +- `spx_vs_sma`: SPX position relative to 20-SMA (% above/below) +- `ndx_vs_sma`: Nasdaq position relative to 20-SMA (% above/below) +- `btc_trend_5d`: BTC 5-day % change +- `regime`: bull | neutral | bear | crisis +- `regime_changed_at`: timestamp of last regime transition + +Material: regime, vix_close, spx_vs_sma. Immaterial: raw tick data from yfinance. + +#### parameter-state +The current values of all tunable system parameters, their bounds, the last +adjustment reason, and the OSS delta since adjustment. This is the +authoritative record — parameter changes are always written here before +taking effect. + +- `max_concurrent_positions`: {value, min: 5, max: 15, adjusted_at, reason} +- `confidence_floor`: {value, min: 0.65, max: 0.85, adjusted_at, reason} +- `stop_loss_pct`: {value, min: -0.03, max: -0.07, adjusted_at, reason} +- `profit_take_pct`: {value, min: 0.03, max: 0.10, adjusted_at, reason} +- `stale_exit_days`: {value, min: 3, max: 7, adjusted_at, reason} +- `position_size_pct`: {value, min: 0.05, max: 0.15, adjusted_at, reason} +- `parker_hedge_cap_pct`: {value, min: 0.10, max: 0.30, adjusted_at, reason} +- `oss_target`: {value: 0.5, min: 0.3, max: 1.5, adjusted_at, reason} +- `oss_floor`: {value: 0.3, min: 0.1, max: 0.7, adjusted_at, reason} +- `oss_weights`: {calmar: 0.40, pf: 0.25, alpha: 0.15, win_rate: 0.10, + adapt_wr: 0.10, min_per_weight: 0.05, max_per_weight: 0.50, adjusted_at, + reason} — weights must sum to 1.0 +- `hedge_regime_buckets`: {moderate: {min_bearish: 0.20, max_positions: 1, + size_pct: 0.01}, heavy: {min_bearish: 0.40, max_positions: 2, size_pct: 0.015}, + extreme: {min_bearish: 0.70, max_positions: 3, size_pct: 0.02}, adjusted_at, + reason} +- `track_frequencies`: per-track {eddie, maggie, frank, maya, janet, parker} + each: {runs_per_day, min: 1, max: 6, adjusted_at, reason} +- `pipeline_frequency`: {runs_per_day, min: 2, max: 6, adjusted_at, reason} +- `oss_at_adjustment`: OSS value when the last parameter change was made +- `oss_since_adjustment`: current OSS vs oss_at_adjustment delta + +Material: all fields. Postcondition: every parameter change is logged with +a reason, the OSS at time of change, and the change is reversible. + +#### performance-summary +The current trading performance across all venues (Alpaca equity + Robinhood +Crypto), sourced from performance_snapshots and trade_journal.db. This is the +user-facing P&L record — human-readable, benchmarked, and attributed. + +- `total_equity`: current total equity across all venues (cash + positions) +- `cash`: idle cash not in positions +- `realized_pnl`: sum of realized_trades.realized_pnl over the trailing 30 days +- `unrealized_pnl`: sum of positions.unrealized_pnl across all open positions +- `win_rate`: winning_trades / total_trades from performance_snapshots +- `profit_factor`: gross_profit / |gross_loss| +- `total_return_pct`: (end_equity - start_equity) / start_equity × 100 +- `max_drawdown_pct`: largest peak-to-trough decline in equity +- `max_drawdown_usd`: dollar value of the worst drawdown +- `sharpe_ratio`: risk-adjusted return (from performance_snapshots) +- `benchmark_return_pct`: SPY return over the same period (for alpha calculation) +- `alpha_pct`: total_return_pct - benchmark_return_pct (our edge over passive) +- `per_track_pnl`: per-track attribution — for each track that produced trades: + {track: str, trades: int, win_rate: float, total_pnl: float, avg_hold_days: float} +- `per_venue_pnl`: {alpaca_equity: {trades, pnl}, robinhood_crypto: {trades, pnl}} +- `daily_pnl_7d`: array of last 7 days: [{date, pnl, cumulative_equity}] +- `largest_winner`: {symbol, pnl, opened_at, closed_at, track} +- `largest_loser`: {symbol, pnl, opened_at, closed_at, track} +- `open_positions`: [{symbol, qty, entry_price, current_price, unrealized_pnl, days_held}] + +Material: all numeric and attribution fields. Immaterial: individual order IDs, +raw JSON. Postcondition: all closed trades in the period are counted. +Postcondition: open-position prices are current within 5 minutes. + +### Continuity + +- self-driven: re-check every 5 minutes +- input-driven: wake immediately when Kojo audit detects anomalies (open orders, + stale tracks, DNS failures, reconcile drift) +- input-driven: wake when Sophie+Ross pipeline completes (new consensus decisions + or execution results may reveal blocks) + +### Strategies + +- When a block is internal and matches a known remediation rule: self-heal + immediately, verify, log, and continue +- When a block is internal but no remediation rule matches: escalate to user + with full diagnostic context (which gate, what state triggered it, what + data was used) +- When a block is external: document the dependency, record the failure, and + do NOT attempt to fix (external dependencies are out of scope) +- When a block persists across 3 self-heal attempts: escalate — do not loop +- During market-closed hours: position-exit rules and circuit-breaker resets + still run (lifecycle hooks are NOT gated on market-open) +- Asset-class isolation: equity position limits never block crypto trades and + vice versa (per-platform capital pools) +- Exit orders always bypass circuit breakers and position-count gates (exits + are self-curing — blocking them creates infinite loops) +- When classification is ambiguous (could be internal or external): classify + as internal and self-heal — it is cheaper to attempt a fix and fail than to + leave a valid trade blocked +- **Return-maximization strategies:** +- When market regime is `bear` AND portfolio_beta > 0.6: positions are + market-tracking into a downtrend — rotate from high-beta equities to + defensive/uncorrelated (inverse ETFs via Parker, consumer staples, utilities, + or cash). Target: reduce portfolio_beta to ≤ 0.3 within 2 cycles +- When market regime is `bull` AND portfolio_beta < 0.3: the system is + under-allocated to the uptrend — increase exposure by allowing higher + position counts (up to MAX_CONCURRENT_POSITIONS) and larger allocation per + position (up to 15% of equity) +- When market regime is `crisis`: all discretionary buys are paused regardless + of consensus — only Parker hedge signals and manual exits execute. Existing + long positions are evaluated for stop-loss exits at -5% +- When OSS is `degrading` (>2 consecutive drops): investigate which component + is dragging (usually max drawdown widening or win rate dropping). If max + drawdown is the culprit, tighten stop-losses. If win rate is dropping, reduce + new position entry velocity — raise confidence floor by 0.05 +- Every adaptation produces a before/after record: {regime, beta_before, + action, beta_after, 7d_forward_return}. These records feed back into + `win_rate_adaptations` — adaptations that repeatedly fail to improve returns + are retired +- Cash as a position: when no track produces signals with confidence ≥ 0.75 + for 3 consecutive cycles, default to cash. Cash allocation is tracked as a + deliberate optimization choice, not a system failure +- Discovery continuity: ticker discovery is not gated on OSS performance. + The system SHALL run discovery lenses every cycle regardless of whether + current positions are profitable. A system that only looks for new tickers + when things are going well is blind exactly when it needs new sources of + return most. Discovery is cheap (screening); admission is careful (gates). +- Rotation constraints: no single adaptation rule may allocate > 30% of + available equity without corroborating signals from at least 2 tracks. + Parker hedge allocation is capped at 25% of equity regardless of regime + +### Remediation Rules + +#### Rule 1: Stale Circuit Breaker +**Detect:** An active circuit breaker in trade_journal.db with `triggered_at` +older than `BREAKER_AUTO_RESET_HOURS` (4h) AND today's daily loss below +`BREAKER_MIN_LOSS_PCT` (2%) of equity. +**Fix:** Deactivate the breaker by setting `active = 0` in circuit_breaker table +and log the reset with reason "Auto-reset: {hours}h idle, daily loss below +threshold". +**Verify:** Re-query circuit_breaker — `active = 0`. Re-compute daily P&L — +still below threshold. +**Escalate:** After 2 reset attempts within 24h, the breaker is re-tripping +from live losses. Escalate to user with the full breaker history. + +#### Rule 2: Position-Limit Block Preventing Exits +**Detect:** Gate 2 (position-count) is blocking an order, AND the order +direction is `bearish`/`sell`/`short` (an exit). +**Fix:** This is a false-positive — exits reduce position count and are +self-curing. The gate code already handles this (is_exit check in gates.py:50), +so this rule catches edge cases where direction is misclassified. Re-evaluate +direction from the consensus decision's `direction` field and retry with +corrected classification. +**Verify:** Exit order submitted successfully. +**Escalate:** After 2 failures — escalate with the exact consensus decision +JSON and gate evaluation trace. + +#### Rule 3: Poll-Fill Timeout (Order Actually Filled) +**Detect:** Ross executor reports `error_poll_timeout` on a submitted order, +but `_verify_order_after_timeout()` was either not called or returned None. +**Fix:** Re-query the broker for the order by `client_order_id`. If filled: +journal the fill and mark as executed. If still pending: poll once more +(30s). If genuinely unfilled: cancel and re-submit. +**Verify:** Order status is terminal (filled/partially_filled/canceled). +**Escalate:** After 2 re-submit attempts — escalate with the order ID and +full poll history. + +#### Rule 4: Stale Track Data Blocking Consensus +**Detect:** A track's `last_signal` timestamp is beyond its freshness threshold +(Eddie 72h, Maggie 168h, Frank 168h, Maya 24h, Janet 12h, Parker 24h) AND +`track_status` shows `last_run_at` is ALSO stale (no recent run). +**Fix:** Trigger the track's cron job immediately via `cronjob(action='run', +job_id=)`. Wait for completion (max 120s). Re-read signals.db. +**Verify:** New signals exist OR track_status shows a fresh run (even if 0 +signals — `no_data` is valid). +**Escalate:** If track run fails or times out — escalate. If track ran but +produced 0 signals for 3 consecutive cycles — escalate (possible source +degradation). + +#### Rule 5: Orphaned Consensus (Execute Decisions Never Traded) +**Detect:** Gap analysis (`sources.reconcile` in auditor, or direct query: +consensus rows with `action='execute'` that lack a matching row in `trades` +table, AND created_at > 1h ago). +**Fix:** Ross orchestrator already self-heals orphaned consensus (lines 166-217 +of orchestrator.py). Verify the self-healing ran: check that orphaned consensus +IDs now have trade rows. If not: manually invoke `mark_ticker_unavailable` for +tickers that genuinely cannot trade (SPAC, delisted, etc.) and re-scan. +**Verify:** Zero orphaned execute decisions older than 1h. +**Escalate:** After 2 cycles of failed self-healing — escalate with orphaned +consensus IDs. + +#### Rule 6: DNS / Data-Source Unreachable +**Detect:** `sources.dns` shows fc.yahoo.com unreachable (yfinance failing) +OR any track's data source returns connection errors for > 2 consecutive runs. +**Fix:** External dependency — do NOT attempt to fix. Verify the failure is +genuinely external (network-layer, not auth/permission). Log and classify +as `external`. +**Verify:** Re-test DNS resolution and connectivity. If recovered: was +transient. If still failed: confirmed external. +**Escalate:** After 3 consecutive failures — escalate with the hostname, +error string, and affected tracks. + +#### Rule 7: Multi-Venue Asset Misrouting +**Detect:** A crypto ticker (BTC-USD, ETH-USD, SOL-USD) triggers an Alpaca +equity error ("BUY needs notional, SELL needs qty" or "symbol not found") +OR an equity ticker hits Robinhood Crypto. +**Fix:** Verify `infer_asset_class(ticker)` in config.py returned the correct +class. If the ticker is missing from `CRYPTO_SYMBOLS`: add it (after verifying +on docs.robinhood.com/crypto/trading). If the code path passed wrong +`asset_class` to `execute_trade`: fix the call site. +**Verify:** Re-run the trade with corrected asset class — order routes to +correct adapter. +**Escalate:** After 1 failure — always escalate (misrouting means money +on wrong venue). Include the ticker, the inferred class, and the expected class. + +#### Rule 8: Market-Regime Position Mismatch + +**Detect:** `market_regime` is `bear` or `crisis` AND `portfolio_beta` > 0.6. +Positions are tracking the market downward — holding long equities in a +broad sell-off. + +**Principle — Regime-graded hedging intensity:** Hedging intensity scales +with regime severity. As VIX rises from 18→35+, the system progressively +shifts defense posture: cash accumulation → defensive sector rotation → +1x inverse ETFs → leveraged inverse ETFs. The contract defines the +direction of travel; the system discovers the optimal allocation +percentages through OSS feedback and per-strategy attribution (P-004). + +**Fix:** (a) If Parker already has active `buy` signals for inverse ETFs +(SH, SQQQ, SPXU, BITI): execute them at up to 25% equity allocation. +1x inverse (SH, PSQ) is preferred over leveraged (-2x/-3x) unless +VIX > 30 and regime is `crisis` — per Rule 18 decay risk. +(b) If no Parker signals: scan existing long positions for those with the +highest beta correlation to SPY and generate exit signals (stale-exit or +stop-loss at -5%). Rotate freed capital to defensive sectors (consumer +staples, healthcare, utilities) or cash — these provide positive +expected return in down markets with zero leverage decay risk. +(c) If VIX > 30 and beta > 0.8: go to 50% cash — sell +the 2 largest high-beta positions. Cash at 4-5% risk-free yield is a +competitive position, not a retreat. +**Verify:** Re-compute portfolio_beta after adaptation — target ≤ 0.3. +Re-compute OSS on next cycle — improvement expected. +**Escalate:** If beta remains > 0.5 after 2 adaptations in bear regime — +escalate with full portfolio composition and regime data. + +#### Rule 9: Regime-Inappropriate Sector Exposure +**Detect:** Market regime is `bear` AND portfolio holds > 2 positions in +high-beta sectors (tech, consumer discretionary, small-cap, crypto) that +are each down > 3% since entry. +**Fix:** Generate exit signals for the 2 worst performers. Rotate capital +into either (a) Parker inverse ETFs if bearish signal exists, or (b) cash +if no hedge signal. Do NOT buy new long positions until regime normalizes. +**Verify:** High-beta exposure reduced by ≥ 40%. Exit orders confirmed filled. +**Escalate:** If exits fill but capital sits idle > 3 cycles without a +viable defensive entry — escalate with regime timeline. + +#### Rule 10: OSS Degradation — Drawdown-Driven +**Detect:** OSS is `degrading` AND the Calmar Ratio component dropped > 20% +from its 30-day peak. This means max drawdown is widening faster than returns +are growing — the system is taking pain without reward. +**Fix:** Reduce position sizes by 25% across all new entries (scale factor +0.75x). Tighten stop-loss to -3% from -5% for new positions. Do NOT modify +position-sizes for existing positions (only new entries). Pause the track +with the worst 7-day return if its win rate < 40%. +**Verify:** Re-compute max drawdown after 3 cycles — trajectory should +flatten. Paused track's win rate trend should stabilize. +**Escalate:** If max drawdown continues to widen after 5 cycles — escalate +with full drawdown waterfall chart and per-track P&L attribution. + +#### Rule 11: Hedge Inefficiency — Hedges Costing More Than Saving +**Detect:** Parker hedge positions are open AND `profit_factor` on hedge-only +trades (filtered by Parker track ID) is < 0.8 (hedges are net losers) over +the last 30 days AND market regime is NOT `crisis`. +**Fix:** Hedge drag is real — inverse ETFs have daily decay. Reduce Parker +allocation to 10% of equity (from 25% cap). Only hold SH (1x inverse, no +decay) unless VIX > 30 (then SQQQ/SPXU are warranted). Generate exit signals +for leveraged hedges (SQQQ, SPXU) that are > 3 days old. +**Verify:** Hedge P&L improves within 5 cycles (profit_factor > 0.9). +**Escalate:** If hedge drag persists — escalate with per-ticker hedge P&L +and hold durations. + +#### Rule 12: Continuous Ticker Discovery + +**Principle — Ticker discovery is continuous, not conditional.** Alpha +factors decay over time (Hubble 2026, AlphaCrafter 2026). The system +SHALL maintain an active pipeline of candidate tickers across multiple +discovery lenses regardless of current OSS or portfolio state. Discovery +happens in the background; admission to the portfolio is gated separately. + +**Discovery lenses (at least 3 must be active per cycle):** + +| Lens | Source | What it finds | +|------|--------|---------------| +| Fundamental | Frank (financials/screeners) | Undervalued/quality stocks | +| Insider/Event | Maggie (insider, SEC filings) | Asymmetric-info opportunities | +| Technical/Momentum | Price/volume scans | Breakouts, trend initiations, RSI divergences | +| Inverse/Hedge | Parker (inverse ETFs) | Down-market defense candidates | +| Regime-Appropriate | Market regime × sector data | Sector rotation into regime-favored sectors | +| Correlation-Seeking | Correlation matrix vs existing positions | Uncorrelated diversifiers | + +**Detect (admission gate):** A new candidate ticker is discovered with: +(a) signal confidence ≥ 0.70 from its source lens, +(b) Pearson correlation ≤ 0.4 to ALL existing open positions + over the trailing 30-day window, +(c) position-count headroom exists (current positions < MAX_POSITIONS + after accounting for pending orders), AND +(d) the ticker passes all standard gates (Gate 1-7). + +**Fix:** Admit the candidate to the consensus pipeline with a +`discovery_lens` tag. For the first 2 cycles: use 50% normal position +size (probation sizing). After 2 cycles with positive unrealized P&L: +graduate to full position size. + +**Verify:** At least 3 discovery lenses produced candidates this cycle. +At least 1 candidate admitted OR clearly documented "no viable candidate +passes correlation gate." Per-source yield tracked (see P-005). + +**Escalate:** If zero candidates pass correlation gate for 5 consecutive +cycles — the system's existing positions are too correlated with each +other or the discovery universe is too narrow. Escalate with full +correlation matrix and per-lens candidate count. + +#### Rule 13: Adaptive Parameter Tuning +**Detect:** OSS trend is `degrading` OR `improving` over 5+ cycles AND a +specific OSS component (Calmar, Profit Factor, Alpha) is the dominant driver. +**Fix — Confidence Floor:** When OSS is improving (bull confirmation): lower +`confidence_floor` by 0.05 (min 0.65) to capture more entries. When OSS is +degrading (entries are losing): raise `confidence_floor` by 0.05 (max 0.85) +to filter out low-quality signals. +**Fix — Position Size:** When Calmar is the degraded component (drawdown +widening): reduce `position_size_pct` by 0.02 (min 0.05). When Calmar is +improving and regime is `bull`: increase `position_size_pct` by 0.02 (max 0.15). +**Fix — Stop-Loss:** When Profit Factor < 0.8 (losers are too large): +tighten `stop_loss_pct` to -0.03. When Profit Factor > 2.0 (winners dominate): +relax `stop_loss_pct` to -0.07. +**Fix — Profit-Take:** When bull regime and Win Rate > 60%: increase +`profit_take_pct` to 0.08 (let winners run). When bear regime or Win Rate +< 40%: decrease `profit_take_pct` to 0.04 (take profits faster). +**Fix — Max Positions:** When OSS > 1.2 and regime is `bull` and cash > 30%: +increase `max_concurrent_positions` by 2 (max 15). When OSS < 0.5: +decrease by 2 (min 5). +**Verify:** Re-compute OSS after 3 cycles — the adjusted component should +show improvement. If not: revert the parameter to its prior value. +**Escalate:** If any parameter change fails to improve its target component +within 3 cycles — auto-unwind and escalate with before/after data. + +#### Rule 14: Dynamic Cron Frequency Adjustment +**Detect:** Signal-producing tracks (tracks that generated ≥3 signals in +their last 3 runs) are running below their maximum frequency AND market +regime is `bull` OR `bear` (not `neutral`) — the market is moving and +opportunities exist. Conversely: tracks producing 0 signals for 8+ +consecutive runs in a `neutral` regime are wasting compute. +**Fix — Increase frequency:** For signal-producing tracks in directional +markets, increase `runs_per_day` by 1 (max 6). Also increase +`pipeline_frequency` by 1 (max 6) to process the additional signals. +**Fix — Decrease frequency:** For silent tracks in neutral markets, decrease +`runs_per_day` to 1 (min). This preserves API call budget for active tracks. +**Implementation:** Update the track's cron job schedule via +`cronjob(action='update', job_id=, schedule=)`. +Do NOT pause or remove jobs — only adjust their schedule. +**Verify:** Track runs at new frequency within 1 day. Signal count per +day increases (for frequency-up) or API call count decreases (for +frequency-down) without signal quality degradation. +**Escalate:** If increased frequency produces 0 additional signals after 3 +cycles — revert to prior frequency (wasted compute). If decreased frequency +causes a track to miss a regime-relevant signal — escalate with the missed +signal details. + +#### Rule 15: Breaker Cycling — Block/Reset Loop +**Detect:** The same breaker type (position-limit, loss-limit) re-triggers +within 30 minutes of a Rule 1 auto-reset, AND no exit orders filled between +the reset and the re-trigger. This indicates the breaker is mechanically +correct but structurally insufficient — exits are not happening fast enough +to sustain the reset. + +**Fix:** Do NOT wait for another 4h cycle. Classify as INTERNAL structural +deadlock. Generate exit orders for the 2 highest-beta positions (sourced +from portfolio_beta.py) that meet **relaxed** exit criteria: held ≥1 day +(instead of 3) OR profit ≥1% (instead of 5%). If no positions meet even +relaxed criteria, go to 50% cash — sell the 2 highest-beta positions at +market regardless of profit. The breaker is self-correct; the exits are not. + +**Verify:** At least 2 exit orders submitted with status `accepted` or +`filled`. Re-compute position count — < 10/10. + +**Escalate:** After 1 cycle — breaker cycling is a structural deadlock. The +contract has exhausted its self-resolution surface. Escalate with: (a) the +breaker history (IDs and timestamps of the last 3), (b) all current positions +with age + beta + unrealized P&L, (c) which exit conditions each position +failed, and (d) the specific relaxed criteria applied. + +#### Rule 16: Data Pipeline Integrity — Stale Local Tables +**Detect:** Any OSS sub-component or performance-summary field returns 0, None, +or a value that contradicts live broker API state. Specifically: (a) open +position count from `positions` table ≠ live API count, (b) unrealized P&L +from local tables differs from live API by >20%, (c) portfolio_beta computed +from local data produces NaN or 0 when positions exist. + +**Fix:** Bypass the local table. Query the live broker adapter +(`AlpacaLiveAdapter.get_positions()` for equity, `RobinhoodCrypto.get_positions()` +for crypto) directly. Recompute the affected sub-component from live data. +Log the stale table name and the delta between local and live data. + +**Verify:** Re-computed sub-component matches live API within ±5% tolerance. +The stale table is flagged in the health report for manual repair. + +**Escalate:** If the same table is flagged stale for 2 consecutive cycles — +escalate with the table name, row count delta, and whether the table schema +needs migration or the write path is broken. A table that was never populated +(`positions`) is a code bug; a table that was populated but is now stale is +a sync gap. + +**Source quality enhancement:** After checking data freshness, the system +SHALL compute Source Health Scores (P-006) for each active data source. +Sources with SHS < 0.40 in the current cycle SHALL be logged with +degradation reason (low authority, staleness, narrow coverage, or novelty +exhaustion). Sources with SHS < 0.25 for 3 consecutive cycles SHALL be +replaced — the system searches for alternative free sources covering the +same data domain (e.g., replacing a stale screener with an alternative +API endpoint, adding a new financial data provider, broadening from +single-exchange to multi-exchange data). + +**Verify:** All active data sources have SHS ≥ 0.25. At least one source +improvement attempted in the last 30 cycles. Source provenance tags +present on all signals generated this cycle. + +#### Rule 17: Signal Direction & Action Validation +**Detect:** A consensus decision has `action=execute` but the combination +of ticker, direction, and position state makes execution impossible or +meaningless: (a) `direction=neutral` — non-actionable by definition, +(b) `direction=bearish` AND no existing position held in the live portfolio +(exit without entry), (c) `direction=bearish` on an inverse-ETF ticker +(SH, SQQQ, SPXU, BITI) where no buy fill exists — hedge exit without hedge +entry. + +**Fix:** Reclassify `action=skip` in the consensus table with a machine-readable +reason: `skipped_neutral`, `skipped_no_position`, or `skipped_no_hedge_entry`. +For case (a): log and skip — neutral signals are information, not action. +For case (b): log and skip — exit signals for unheld tickers are noise. +For case (c): log and skip — Parker exit signals are only valid when a +corresponding buy fill exists in broker_orders. + +**Verify:** Consensus action updated to `skip`. No future Ross execution +attempt for this ticker×direction without a fresh buy signal first. + +**Escalate:** Never — these are signal-quality filters, not system failures. +They reduce noise and prevent the orchestrator from attempting impossible +trades. + +#### Rule 18: Inverse ETF Decay Shield + +**Detect:** An open Parker hedge position uses a leveraged inverse ETF +(SQQQ, SPXU, SOXS, BITI — any -2x or -3x daily-reset instrument) AND +at least one decay condition is met: + +(a) **Duration decay:** Hold duration > 10 trading days. Daily-reset +decay compounds with time — leveraged inverse ETFs held beyond 2 weeks +lose significant value even if the underlying index direction is correct. + +(b) **Low-VIX decay:** VIX < 20. Implied volatility is too low to +justify the leverage cost and daily rebalancing drag. Leveraged +inverse instruments are designed for high-volatility environments. + +(c) **Choppy-market decay:** Market texture is "choppy" — SPX +oscillating within ±3% of its 20-day SMA for >5 consecutive trading +days. In range-bound markets, the daily-reset mechanism of leveraged +ETFs maximizes volatility decay regardless of eventual direction. + +**Fix:** Exit the leveraged inverse position immediately. Log the exit +reason as `decay_duration`, `low_vix`, or `choppy_decay`. + +If a hedge is STILL warranted (regime remains `bear` AND portfolio_beta +> 0.5 after exit): open a replacement position using the 1x inverse +equivalent — SH (S&P 500 -1x) for broad market exposure, PSQ +(Nasdaq -1x) for tech exposure. 1x inverse ETFs have no daily-reset +decay and can be held for the duration of the bear regime. + +If no hedge is warranted: rotate the freed capital to cash or defensive +sectors per Rule 8. + +**Verify:** Leveraged inverse positions closed. If regime still warrants +hedge: replacement 1x inverse position opened OR defensive rotation +executed. portfolio_beta ≤ 0.5 within 2 cycles. + +**Escalate:** If the same ticker's leveraged inverse position is +exited-and-reopened >2 times in 30 days — the system is cycling in +and out of leveraged hedges. Escalate with per-exit decay cost +estimate (entry price vs exit price, holding days) and recommend +staying in 1x inverse until VIX > 25. + +#### Rule 19: Missed Signal Hindsight Detection + +**Principle — The system learns from what it missed.** Credit assignment +research (Zhang 2026, 47 methods surveyed) demonstrates that post-hoc +hindsight analysis — using information available only AFTER the trading +window closes — identifies root causes of missed opportunities that +feed-forward detection cannot see. + +**Detect (runs every 10 cycles, counterfactual analysis):** +For each of the 10 largest daily movers (by absolute % change) in the +system's benchmark universe over the past 10 trading days: + +(a) Did any Persona generate a signal for this ticker? + → NO: failure mode = coverage gap (source didn't see it) + → YES, but confidence < threshold: failure mode = signal too weak + → YES, confidence ≥ threshold, but didn't trade: failure mode = gate + filtering (which gate blocked it?) + +(b) If a trade COULD have been opened (signal existed, confidence ≥ 0.70, + passed all gates) but wasn't: failure mode = execution gap + +**Fix — Root cause triage:** + +| Root cause | Contract action | +|-----------|-----------------| +| Coverage gap (no signal) | Add ticker to discovery candidate list. If ≥3 missed tickers from same sector: broaden that discovery lens's screening criteria | +| Signal too weak (conf < 0.70) | Log; if same ticker missed 3+ times: lower confidence floor for that lens by 0.05 (min 0.60) | +| Gate filtering (passed source, blocked by gate) | Log which gate blocked it. If same gate blocks ≥3 viable trades in 10 cycles: escalate for gate calibration review | +| Execution gap (could have, didn't) | Escalate — this is an orchestrator or order-routing failure | + +**Verify:** Hindsight analysis completed for 10 largest movers. At least +1 root cause identified OR documented "no missed opportunities." Source +coverage gap counts trend downward over rolling 90-day window. + +**Escalate:** If coverage gap count increases for 3 consecutive hindsight +cycles — the system's data universe is shrinking, not expanding. Escalate +with per-source coverage breadth trend. + +### Invariants + +- **P-001 — Paper trading is dead code.** The paper trading adapter + (`brokers/alpaca_paper.py`) is NEVER imported, NEVER invoked, and has NO + code path in any production pipeline, cron job, or system module. The + `ALPACA_ENV` routing mechanism no longer supports a `paper` mode — all + adapters route to live production only. The file may exist on disk as a + historical artifact but carries zero operational risk because (a) no import + chain reaches it, (b) `brokers/__init__.py` no longer exports + `AlpacaPaperAdapter`, (c) `brokers/router.py` hardcodes live-only routing, + (d) `reconcile.py` hardcodes live-only reconciliation, (e) + `consensus/ross/adapter.py` hardcodes live-only adapter selection, and + (f) all metric queries filter `venue != 'alpaca_paper'` as defense-in-depth. + Only live production trading across Alpaca Live and Robinhood Crypto SHALL + count in any performance calculation, OSS computation, equity curve, + drawdown measurement, P&L report, or health evaluation. + +- **P-002 — Dust positions are filtered from position counting.** Positions + with `|market_value| < $0.01` (one cent) are considered dust — fractional + remnants from fills/partials that have zero economic significance. Gate 2 + (position-count limit, `gates.py:144-150`) and Gate 7 (gross-exposure cap, + `gates.py:270-294`, all four sum expressions) SHALL filter these positions + using `>= cfg.DUST_POSITION_THRESHOLD` (functionally equivalent to the + former `> 0` since all dust has market_value ≤ $0.00). `compute_oss.py` + (`scripts/compute_oss.py:182-184`) also SHALL filter dust from open position + counting. This invariant is verified by 4 property-based tests + covering 800+ generated examples (see `tests/test_gate2_dust_filter.py` and + `tests/test_gate2_property_invariants.py`): + - **INV-001**: Any number of dust positions never blocks Gate 2 (200 examples) + - **INV-002**: If Gate 2 blocks, non-dust count ≥ limit (300 examples) + - **INV-003**: Adding dust positions never changes allowed→blocked (200 examples) + - **INV-004**: Filter applied symmetrically to both equity and crypto (100 examples) + The threshold constant `DUST_POSITION_THRESHOLD = 0.01` is defined at + `consensus/ross/config.py:76` (single source of truth) and SHALL NOT be + raised without Theo's approval — raising it would hide small but real + positions; lowering it is safe but unnecessary. + +- **P-003 — Returns are trade-attributed, not balance-based.** The equity curve + in *every* performance computation (Sharpe ratio, max drawdown, total return + percentage, OSS rolling alpha, and Calmar ratio) SHALL be built from actual + trading activity — realized P&L from closed trades plus unrealized P&L from + open positions — and SHALL NEVER use raw account balances from + `sync_state.broker_equity`. Deposits, withdrawals, dividends, cash transfers, + and fees SHALL NOT be counted as trading returns. + + **Equity curve formula:** `equity[t] = ACCOUNT_EQUITY + cumulative_realized_pnl[t] + unrealized_pnl[t]` + where `ACCOUNT_EQUITY` is the initial capital from `consensus/ross/config.py:119` + (single source of truth), `cumulative_realized_pnl[t]` is the sum of + `realized_pnl` from `realized_trades` (filtered `unmatched=0`, `venue != 'alpaca_paper'`) + up to day `t`, and `unrealized_pnl[t]` is the current `SUM(unrealized_pl)` from + `broker_positions` filtered by `ABS(market_value) >= DUST_POSITION_THRESHOLD`. + + **Implementation:** `perf.py:compute_equity_metrics()` (lines 137–261) builds + the equity curve from trade-attributed data only. `sync_state.broker_equity` is + used for reconcile timing (date grid) but its *value* is never read for return + computation. This invariant is verified by 3 property-based tests covering + 500+ generated examples (see `tests/test_perf_equity_source.py` and + `tests/test_perf_equity_property_invariants.py`): + - **INV-001 (Additivity):** `end_equity = ACCOUNT_EQUITY + Σ(realized_pnl) + unrealized_pnl` (200 examples) + - **INV-002 (Deposit Exclusion):** Deposits of any amount do not affect trade-attributed return (200 examples) + - **INV-003 (Flat when idle):** With zero trades, equity stays at `ACCOUNT_EQUITY` regardless of deposits (100 examples) + + **Rationale:** On 2026-06-17, a $500 deposit into the Alpaca Live account + caused `sync_state.broker_equity` to jump from $201 to $701. The old + balance-based `compute_equity_metrics()` treated this as a 249% overnight + return, inflating the OSS from a correct value near 0 to 2.595 (false + positive). Actual realized P&L across all trades was -$1.67 on 24 trades. + The fix was validated by [fv] protocol on 2026-06-30: claim → verify → + cross-check (all 4 cron outputs, all venues, all position-counting paths) → + draft → 500+ property test examples → zero regression (140/140 tests). + +- **P-004 — Hedging strategies are independently attributed and performance-graded.** + Every hedge position (inverse ETF, defensive sector rotation, cash allocation) + SHALL carry a `hedge_strategy` tag in broker_orders and realized_trades. + The health monitor SHALL compute per-strategy profit factor and win rate + independently over a rolling 60-day window: + + | hedge_strategy | Instruments | Description | + |---|---|---| + | `inverse_1x` | SH, PSQ, RWM | 1x inverse ETFs — daily-reset, no leverage decay | + | `inverse_leveraged` | SQQQ, SPXU, SOXS, BITI | -2x/-3x inverse — decay risk, crisis-only | + | `defensive_sector` | XLP, XLV, XLU, consumer staples, healthcare, utilities | Long defensive — positive carry | + | `cash_allocation` | Cash (>40% of equity) | Yield-bearing, zero correlation | + + Strategies with `profit_factor < 0.7` over the 60-day window SHALL be + flagged for review. Strategies with `profit_factor < 0.5` over 60 days + SHALL be retired — the contract auto-removes them from the authorized + hedging toolkit. A retired strategy may be re-evaluated after 30 days. + This invariant enables emergent optimization: the system discovers + through attribution which hedging strategies work in which regimes, + rather than having allocation percentages hardcoded. + + This invariant is verified by property-based tests that confirm every + hedge order carries a valid `hedge_strategy` tag, per-strategy P&L sums + are consistent with total hedge P&L, and retired strategies are excluded + from new hedge entry signals. + +- **P-005 — Ticker universe breadth is continuously monitored and diversified.** + The system SHALL maintain a minimum discovery breadth: at least 3 active + discovery lenses producing candidates each cycle, with at least 10 unique + candidate tickers evaluated in any rolling 30-day window. No single + discovery lens may account for >60% of open positions over a 60-day window + (prevents lens monoculture). + + Each discovery lens SHALL carry a `discovery_lens` tag on all signals + and trades it produces. The health monitor SHALL compute per-lens yield: + + | Metric | Formula | Threshold | + |--------|---------|-----------| + | Lens hit rate | realized_trades with profit > 0 / total trades from lens | Alert if < 0.3 over 60d | + | Lens profit factor | gross_profit / gross_loss from lens trades | Retire lens if < 0.5 over 60d | + | Lens novelty rate | candidates with correlation < 0.3 to existing / total candidates | Alert if < 0.2 over 30d | + + A discovery lens with profit_factor < 0.5 over 60 days SHALL be paused + (its candidates are generated but not admitted). A lens with novelty_rate + < 0.2 over 30 days is producing only correlated candidates — its search + criteria SHALL be broadened. + + This invariant ensures the system never settles into a narrow field of + view where it only sees tickers similar to what it already holds. The + correlation admission gate (Rule 12) and novelty monitoring (P-005) + together force the system to continuously expand its field of view. + +- **P-006 — Data source quality and coverage improve continuously through + experiential feedback.** Every Persona's data source carries a quality + score that evolves over time based on downstream trade outcomes and + timeliness. Source quality degrades when signals from that source lead + to losing trades or arrive too late; source quality improves when + signals lead to winning trades with timely execution. + + **Source quality dimensions (scored per source per cycle, 0.0–1.0):** + + | Dimension | Formula | Degradation trigger | + |-----------|---------|-------------------| + | Authority | rolling 60d hit rate: winning trades / total trades from source | < 0.35 over 60 days | + | Timeliness | signals that resulted in fills before adverse price move / total signals | < 0.50 over 30 days | + | Coverage breadth | unique tickers surfaced / unique tickers in benchmark universe | < 0.10 over 90 days | + | Novelty decay | new (unseen) tickers surfaced this cycle / total tickers surfaced | < 0.05 over 30 days | + + The combined Source Health Score (SHS) = (authority × 0.4) + (timeliness × + 0.3) + (coverage × 0.2) + (novelty × 0.1). A source with SHS < 0.40 + SHALL be flagged for review. A source with SHS < 0.25 SHALL be replaced + (the system discovers an alternative free source covering the same data + domain) or retired (its signals are generated but not admitted to the + consensus pipeline). + + **Source provenance:** Every signal entering the consensus pipeline + SHALL carry a `source_provenance` tag identifying: (a) the Persona that + generated it, (b) the data source(s) it drew from, (c) the timestamp of + the underlying data, and (d) the confidence score. This enables + post-hoc tracing: when a trade wins or loses, the system knows which + data source contributed to the decision. + + **Coverage improvement obligation:** Every 30 cycles (≈ trading month), + the system SHALL attempt to add at least one new free data source or + expand an existing source's coverage (additional tickers, additional + data fields, additional frequency). New sources start with probationary + SHS = 0.50 and are evaluated after 30 cycles. This is the mechanism by + which the system's field of view widens over time. + + This invariant is verified by property-based tests that confirm every + signal carries a valid `source_provenance` tag, SHS scores are computed + for all active sources, and retired sources are excluded from signal + generation. + +- **P-007 — Benchmark integrity is self-validating.** Every alpha and beta + computation SHALL use a live, point-in-time benchmark return fetched this + cycle via `yfinance` (^GSPC). The benchmark source SHALL NOT contain + hardcoded literals — any constant appearing in benchmark calculation is + a contract violation. Alpha sign SHALL flip correctly when portfolio + return crosses the benchmark return (positive alpha = outperformance). + Benchmark fetch failure SHALL be surfaced as `benchmark_error` in the + OSS output rather than silently using a stale value. Property-tested + via `test_benchmark_integrity.py`. + +- **P-008 — Trade execution costs are modeled and bounded.** No trade SHALL + be admitted whose expected edge (signal confidence × signal strength) is + less than the modeled round-trip cost (half-spread + commission + + slippage). The cost model SHALL use realized `slippage_bps` from + `broker_orders` as feedback — if median slippage exceeds 50 bps over a + 30-day rolling window, the cost gate SHALL tighten (raise the admission + threshold). A cost-inefficient trade that slips past the gate SHALL + contribute to `cost_breach_count` and degrade the OSS via the Cost- + Aware Gate (Gate 2.3). StockBench/FINSABER evidence: for a $1K account, + trading 1% positions, a single 50-bps slip on entry AND exit is $0.10 + round-trip (0.01%) — negligible. The gate exists to prevent death by + a thousand cuts, not to reject single marginal trades. + +- **P-009 — The Optimization Success Score and every sub-metric are sign- + correct and monotonic.** The Calmar Ratio SHALL be negative when CAGR is + negative (never `abs(CAGR) ÷ max_drawdown`). Rolling Alpha SHALL be + positive when portfolio return exceeds benchmark return. A losing trading + period (negative total P&L, negative CAGR) SHALL NEVER produce an OSS ≥ + a winning trading period (positive total P&L, positive CAGR). The + `oss_trend` indicator SHALL correctly reflect the direction of the last 3 + OSS readings: improving (rising), flat (within ±0.05 band), or degrading + (falling). Property-tested via `test_oss_monotonicity.py`. + +- **P-010 — Every adaptation decision is evidence-backed and reversible.** + Every parameter change SHALL write a full before/after record to the + `adaptations` table with: the parameter name, prior value, new value, + OSS at time of change, market regime, and the adaptation rule that + triggered it. After each change, the system SHALL wait at least 3 + evaluation cycles (≈3 trading days). If forward OSS (OSS now minus OSS + at time of change) is non-positive for 3 consecutive post-adaptation + evaluations, the adaptation SHALL auto-unwind — the parameter reverts + to its pre-adaptation value and the unwinding is logged as a separate + adaptation record with `result='unwound'`. The `win_rate_adaptations` + metric SHALL reflect the fraction of adaptations whose forward OSS + delta was positive. An adaptation loop that produces 5 consecutive + unwound adaptations SHALL escalate to the user — the optimizer has + lost its signal. Property-tested via `test_adaptation_persistence.py`. + +- **P-011 — The declared system objective is drawdown-first, not growth- + first.** The primary outcome the contract promises is: "match the S&P 500 + benchmark net of trading costs, with peak-to-trough drawdown bounded to + ≤25%." Risk-adjusted outperformance above the benchmark is a secondary + objective, pursued only when market-regime detection confirms the system + has an edge. Hedge capacity SHALL be regime-driven (Gate 2.1 buckets: + moderate/heavy/extreme) and SHALL be sized to actually bound drawdown — + `HEDGE_POSITION_SIZE_PCT` scales with bearish_score, and `PARKER_HEDGE_CAP_PCT` + sets the ceiling. Cash (zero-beta) is a valid optimization result when + no edge is detected. The contract enforces improvement trajectory, not + absolute return thresholds — a stable OSS of 0.7 with low drawdown is + preferable to a volatile OSS oscillating between 0.3 and 1.5. StockBench/ + FINSABER evidence (arXiv:2505.19081): small accounts ($1K–$10K) are cost- + constrained; benchmark-matching net of costs is the honest achievable + outcome, not "maximize growth." + +- The prime directive SHALL NOT be weakened by any contract update. Any change + that would allow valid trades to be silently blocked is invalid. +- Internal failures SHALL be distinguished from external failures. "Alpaca API + returned 500" is external; "gate evaluated stale position count" is internal. +- Every remediation action produces a structured audit trail entry with + classification, action taken, and verification result. +- Strategy parameters and cron schedules MAY be dynamically adjusted by + return-optimization rules (13-14), within bounded ranges, only when OSS or + regime data justifies the change. See `#### parameter-state` for current + values and justification trail. The stability surface (breakers, exit-gates, + per-position caps) SHALL NOT be modified. +- During market-closed hours: position exits, breaker resets, and reconciliation + continue to execute (these are lifecycle hooks, not trade execution). +- The contract itself is a living document. Updates that tighten detection or + expand remediation are encouraged. Updates that weaken the detection surface + or narrow the definition of "blocked" compromise the prime directive. +- **Return-optimization guardrails:** +- All adaptation decisions SHALL be reversible — no parameter change persists + beyond the regime that triggered it without fresh corroboration +- No adaptation may increase position size beyond MAX_CONCURRENT_POSITIONS × + (equity ÷ MAX_CONCURRENT_POSITIONS) — the per-position cap is invariant +- Return maximization targets *capturable* returns only — the contract does + not promise returns the market does not offer. Regime detection may indicate + "no edge available," and cash is a valid optimization result +- The OSS target (1.0) is aspirational — the contract enforces *improvement + trajectory*, not absolute thresholds. A stable OSS of 0.7 is better than + a volatile OSS oscillating between 0.3 and 1.2 +- Adaptations that fail to improve OSS over 3 consecutive evaluations (≈3 + trading days) are automatically unwound — the system returns to its + pre-adaptation state +- Circuit breakers, position-count gates, and exit rules (the stability + surface) are NEVER modified by optimization rules. Optimization operates + within the stability surface, not on it +- **Data source version integrity:** Every external API endpoint used by a + track SHALL be documented with its current version in the track's source + header. When a track produces 0 signals for 3+ consecutive runs, the + health monitor SHALL verify the endpoint version against the documented + version before classifying as `no_data` (valid). A version mismatch + (deprecated endpoint, API EOL) is internal, not external — the contract + is responsible for keeping its data sources current. +- **Execution mode integrity:** Every cron job that runs a pure Python script + (track signal generation, DB maintenance, data collection — where the script + itself produces the complete output without LLM processing) SHALL be + configured with `no_agent=true`. The health monitor SHALL verify this + configuration for all track and pipeline jobs. An agent-mode configuration + on a script-only job is an internal misconfiguration — the LLM may perform + unrelated actions (web search, reasoning) instead of executing the script, + silently degrading signal freshness. +- **Orchestrator status coverage:** The Ross orchestrator SHALL handle every + known execution status code produced by `execute_trade()`. Status codes + that indicate permanent failure (`error_rejected`, `skipped_no_position`, + `skipped_non_fractionable`) SHALL be marked as permanently consumed + (action=skip), not retried. Status codes that indicate transient failure + (`error_poll_timeout`, `error_liquidity`) MAY be retried up to the + orchestrator's retry limit. The set of permanently-skipped statuses SHALL + be updated whenever a new adapter returns a novel failure code. + +### Reporting + +Every remediation cycle produces a structured report: + +#### 1. Cron Output — Health Summary (every cycle) +``` +🛡️ Alpaca System Health — 2026-06-28 15:00 UTC +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +Status: HEALTHY | 0 blocks | 0 stale | DNS OK +Regime: BULL (VIX 15.2) | OSS: 1.23 ▲ (Calmar 1.4) +Equity: $697.35 | Cash: $542.50 | Open Orders: 0 +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` +With issues: +``` +🛡️ Alpaca System Health — 2026-06-28 15:00 UTC +Status: ADAPTING | 1 block | Maya stale (27h) | Regime: BEAR +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +📉 OSS: 0.42 ▼ (Calmar 0.3, PF 0.8, Beta 0.78) + └─ Warning: Below floor (0.5) — adaptation required + └─ Action: Rule 8 — Beta 0.78 in bear regime → rotate defensively + +⚠️ BLOCK: [details as before] +⚠️ STALE: [details as before] +📊 ADAPTATION: [details as before] +``` + +#### 2. Cron Output — P&L Dashboard (10:00 AM and 6:00 PM daily, or on significant change) +``` +📊 Alpaca P&L Dashboard — 2026-06-28 (30-day trailing) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +💰 Total Equity: $697.35 | Cash: $542.50 | Open: $154.85 (3 pos) + +📈 PERFORMANCE + Realized P&L (30d): +$42.18 (+6.4%) + Unrealized P&L: +$8.72 + Benchmark (SPY 30d): +2.1% + Alpha: +4.3% ← our edge + +📉 RISK + Max Drawdown: 4.2% ($29.14) + Sharpe Ratio: 1.47 + Profit Factor: 1.82 (gross profit ÷ gross loss) + Win Rate: 58% (11W / 8L) + +🏆 TOP/BOTTOM + Largest Winner: NVDA +$18.42 (Eddie, held 4d) + Largest Loser: AMD -$5.21 (Janet, held 2d) + +📋 BY TRACK + Track Trades Win% P&L Avg Hold + ───────── ────── ──── ───── ──────── + Eddie 4 75% +$22.14 4.2d + Maggie 3 67% +$14.31 5.1d + Frank 4 50% +$5.73 3.8d + Maya 2 50% -$0.80 7.0d + Janet 4 50% -$0.20 1.8d + Parker 2 100% +$1.00 2.0d + ───────── ────── ──── ───── ──────── + TOTAL 19 58% +$42.18 3.8d + +📋 BY VENUE + Alpaca Equity: 17 trades | +$35.94 + Robinhood Crypto: 2 trades | +$6.24 + +📅 DAILY P&L (7 days) + Mon +$12.40 ████████ + Tue -$3.20 ██ + Wed +$18.72 ████████████ + Thu +$5.41 ███ + Fri +$8.85 ██████ + Sat — (market closed) + Sun — (market closed) + +🔓 OPEN POSITIONS + ENVX 5.0 @ $11.23 → $11.82 +$2.95 (5d) + CLF 10.0 @ $19.10 → $20.05 +$9.50 (3d) + ETH 0.01 @ $2,452 → $2,389 -$4.73 (2d) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +``` + +#### 3. RA-H OS Knowledge Graph +Every remediation cycle creates a `[LEARN] alpaca-system-health: {timestamp}` +node with: +- metadata: { type: "system-health", status: "healthy" | "degraded" | "blocked" } +- source: Full structured JSON report +- description: Summary of blocks found, fixed, escalated, and P&L highlights + +A separate `[REPORT] alpaca-pnl-dashboard: {date}` node is created for each +daily P&L dashboard with the full performance-summary data for long-term +trending and cross-session comparison. + +#### 4. Periodic Digest (via existing Kojo auditor) +Kojo's existing 07:00 and 20:00 reports serve as the operational digest. +The system-health contract supplements Kojo: Kojo audits and reports, +the system-health contract detects, adapts, and self-heals. + +### Execution + +1. **Gather truth** — Run `auditor_query.py` for live system data. Query + signals.db for latest consensus decisions and execution results. +2. **Classify state** — Compare against thresholds. Classify every anomaly + as internal or external using the fault-domain rules. +3. **Detect market regime** — Fetch VIX, SPX 20-SMA, Nasdaq 20-SMA, and + BTC trend via yfinance (reuse Parker's market data pipeline). Classify + regime as bull/neutral/bear/crisis. +4. **Compute OSS** — Query trade_journal.db for closed trades in the last + 30 days. Compute Calmar Ratio, Profit Factor, Rolling Alpha over SPY, + Portfolio Beta, and Win Rate of adaptations. Blend into OSS score. +5. **Check adaptation triggers** — Compare OSS vs floor (0.5), portfolio_beta + vs regime, and sector exposure vs regime. Fire adaptation rules (8-12) + if triggered. +6. **Apply operational remediation** — For each internal block not addressed + by adaptation rules, apply matching remediation rule (1-17). External blocks + are documented, not fixed. +7. **Re-verify** — After all remediations and adaptations, re-run + auditor_query.py and re-compute OSS. Only report what persists. +8. **Generate report** — Produce structured output (cron format + RA-H node). +9. **Record adaptation outcomes** — For each adaptation fired, log a + before/after record in trade_journal.db for future OSS computation. +10. **Wait 5 min** → repeat from step 1. + +### Audit Trail Format + +```json +{ + "run_id": "system-health-20260628-150000", + "timestamp": "2026-06-28T15:00:00Z", + "duration_ms": 8234, + "system_state": { + "equity": 697.35, + "cash": 542.50, + "open_orders": 0, + "market_open": false, + "tracks": { + "eddie": {"fresh": true, "hours_since": 46}, + "maya": {"fresh": false, "hours_since": 27}, + "janet": {"fresh": true, "hours_since": 0}, + "parker": {"fresh": true, "hours_since": 1} + }, + "dns": {"fc.yahoo.com": true} + }, + "market_regime": { + "regime": "bear", + "vix_close": 27.4, + "spx_vs_sma": -2.3, + "ndx_vs_sma": -3.8 + }, + "return_optimization": { + "oss": 0.42, + "oss_trend": "degrading", + "calmar": 0.31, + "profit_factor": 0.82, + "rolling_alpha": -8.5, + "portfolio_beta": 0.78, + "win_rate_adaptations": 0.40 + }, + "blocks_found": 2, + "blocks_internal": 1, + "blocks_external": 1, + "blocks_fixed": 1, + "blocks_escalated": 0, + "adaptations_fired": 2, + "adaptations": [ + { + "rule": "Rule 8: Market-Regime Position Mismatch", + "target": "portfolio_beta", + "action": "Rotated high-beta to cash", + "before": {"beta": 0.78, "regime": "bear"}, + "after": {"beta": 0.35}, + "result": "adapting" + }, + { + "rule": "Rule 9: Regime-Inappropriate Sector", + "target": "NVDA, AMD", + "action": "Exited high-beta losers; rotated to cash", + "before": {"exposure_pct": 22}, + "after": {"exposure_pct": 8}, + "result": "adapting" + } + ], + "remediations": [ + { + "rule": "Rule 1: Stale Circuit Breaker", + "target": "equity-position-limit", + "action": "deactivated active breaker", + "result": "fixed", + "duration_ms": 120 + } + ] +} +``` + +### Errors + +- **Alpaca API unreachable** — classified as external. Log, do not fix. +- **Robinhood Crypto API unreachable** — classified as external. Log, do not fix. +- **signals.db corrupted** — classified as internal. Attempt SQLite integrity + check. If unrecoverable: escalate immediately (data loss). +- **trade_journal.db corrupted** — classified as internal. Attempt SQLite + integrity check. If unrecoverable: escalate immediately. +- **auditor_query.py fails** — classified as internal. Check Python venv path, + verify dependencies. If persistent: escalate. +- **Cron job trigger fails** (cronjob action='run' returns error) — classified + as internal. Retry once. If persistent: escalate. +- **Etherscan API key expired or endpoint deprecated** — classified as internal. + The contract is responsible for keeping its data sources current. Verify the + API key against Etherscan's account portal. Verify the endpoint version against + the documented version in track_4_maya.py. If deprecated: migrate to the + current endpoint. If key expired: escalate (requires human credential rotation). +- **yfinance returns empty data (not error)** — classified as internal if the + system marks it as `no_data` without verifying the data source is reachable. + Before classifying as `no_data`: (a) verify yfinance can fetch a known + benchmark ticker (SPY), (b) check for rate-limiting response headers, + (c) compare against Rule 6 DNS check. If yfinance is reachable but specific + tickers return empty: classify as `no_data` (valid). If yfinance is + unreachable or rate-limited: classify as external (Rule 6). +- **ChromaDB/LLM dependency unreachable** — classified as external if the + dependency is hosted remotely. If running locally and the process is down: + classified as internal — restart the process. If persistently down: escalate. +- **Cron job running in wrong execution mode** — classified as internal. If a + track's cron job is configured with agent-mode (`no_agent=false` or unset) + when it should run as a script (`no_agent=true`), the health monitor SHALL + flag this as an internal misconfiguration. Fix: update the cron job via + `cronjob(action='update', no_agent=true)`. + +### Skills + +- kojo-auditor +- pipeline-idempotency-audit + +### Tools + +- `cli:python3`: required for running auditor_query.py and track scripts +- `cli:sqlite3`: required for direct database queries when auditor is unavailable +- `cli:curl`: required for DNS reachability tests and API health checks