1162 lines
62 KiB
Markdown
1162 lines
62 KiB
Markdown
---
|
||
kind: responsibility
|
||
name: alpaca-trading-system-health
|
||
description: >
|
||
Standing responsibility that monitors and optimizes the Alpaca multi-track
|
||
trading system. Two mandates: (1) Prime Directive — valid consensus-backed
|
||
trades SHALL reach the broker, with external dependency failures as the only
|
||
acceptable blocks. (2) Return Maximization — the system actively adapts
|
||
positions, tickers, and allocation to market regime to maximize risk-adjusted
|
||
returns, measured by a composite Optimization Success Score (Calmar Ratio +
|
||
Profit Factor + Rolling Alpha). All adaptations are reversible and bounded.
|
||
---
|
||
|
||
### Goal
|
||
|
||
**Prime Directive:** Valid trades are not blocked by the Alpaca trading system.
|
||
Every block that prevents a Sophie-consensus `execute` decision from reaching
|
||
the broker is either self-healed within one cycle or escalated with a clear root
|
||
cause. External dependency failures are classified, never confused with internal
|
||
issues, and not counted against system health.
|
||
|
||
**Return Optimization:** The primary objective is to **match the S&P 500 benchmark
|
||
net of costs while bounding peak-to-trough drawdown to ≤25%.** Risk-adjusted
|
||
outperformance is a secondary goal, pursued only when market-regime detection
|
||
confirms the system has an edge. The system measures its Optimization Success
|
||
Score (OSS) on a rolling 30-day window, adapts positions, tickers, and allocation
|
||
to market regime within bounded, reversible guardrails, and never chases returns
|
||
the market does not offer. The target OSS is ≥ 0.5 (benchmark-matching
|
||
institutional-grade); ≥ 1.0 is aspirational in favorable regimes. The floor is
|
||
0.3 (below which adaptation and hedging are required).
|
||
|
||
### Requires
|
||
|
||
- `kojo-audit`: a current structured audit of open orders, signal freshness
|
||
per track, DNS reachability, and reconcile drift — sourced from the Kojo
|
||
auditor cron job (a4476b04abab, runs 07:00 and 20:00 daily)
|
||
- `sophie-consensus`: the latest consensus decisions from Sophie (execute /
|
||
watch / skip per ticker) — sourced from the consensus table in signals.db
|
||
written by Sophie+Ross pipeline jobs (43ad4780620a, 8d027ddff088)
|
||
- `execution-results`: the latest Ross trade execution outcomes — sourced
|
||
from broker_orders in trade_journal.db
|
||
|
||
### Maintains
|
||
|
||
The current system health truth. Postcondition: every block is classified as
|
||
`internal` (actionable, must self-heal) or `external` (documented, escalated).
|
||
Postcondition: no internal block persists across two consecutive audit cycles
|
||
without escalation. Postcondition: the system never reports "0 issues" while
|
||
valid trades are blocked.
|
||
|
||
#### system-health
|
||
Overall health: `healthy` (all gates pass, no blocks), `degraded` (non-fatal
|
||
blocks found, some trades may be affected), or `blocked` (trades are being
|
||
prevented from reaching the broker). Material: the status and a summary of
|
||
every active block.
|
||
|
||
#### active-blocks
|
||
Every currently active block preventing trade execution. Each block carries:
|
||
`source` (gate/component), `ticker` (affected symbol), `reason` (machine-readable
|
||
classification), `classified` (internal | external), `first_seen` (timestamp),
|
||
`last_seen` (timestamp), `self_heal_attempts` (count). Material: the block set
|
||
and each block's classified/reason fields.
|
||
|
||
#### stale-data
|
||
Every data source that is stale beyond its freshness threshold. Each entry
|
||
carries: `track` (eddie/maggie/frank/maya/janet/parker), `last_signal`
|
||
(timestamp), `hours_since`, `last_run_at` (from track_status), `threshold_hours`,
|
||
`blocking` (whether the staleness is currently preventing trades). Material:
|
||
the track name, hours_since, and blocking fields.
|
||
|
||
#### failure-log
|
||
Rolling log of the last 50 failure events. Each event carries: `timestamp`,
|
||
`component`, `classification` (internal | external), `resolved` (boolean),
|
||
`resolution` (self-healed | escalated | transient-cleared), `duration_s`
|
||
(seconds until resolution). Immaterial: raw stack traces, full HTTP response
|
||
bodies.
|
||
|
||
#### return-optimization
|
||
The composite Optimization Success Score and its components, computed on a
|
||
rolling 30-day window from trade_journal.db. OSS weights: Calmar Ratio 40%,
|
||
Profit Factor 25%, Rolling Alpha over SPY 15%, Win Rate 10%, Adaptation
|
||
Win Rate 10% (per P-010).
|
||
|
||
- `oss`: the current OSS (0.0–5.0+)
|
||
- `oss_target`: 1.0 (institutional-grade), `oss_floor`: 0.5 (adaptation required)
|
||
- `calmar`: rolling CAGR ÷ max drawdown (absolute)
|
||
- `profit_factor`: gross profit ÷ gross loss (all closed trades, 30-day rolling)
|
||
- `rolling_alpha`: portfolio excess return over SPY benchmark (annualized %)
|
||
- `portfolio_beta`: rolling beta to SPY — measures how much of our return is
|
||
just tracking the market
|
||
- `win_rate_adaptations`: % of rotation/hedge adjustments that improved 7-day
|
||
forward return
|
||
- `last_computed`: timestamp of most recent OSS calculation
|
||
- `trend`: improving | flat | degrading — direction of OSS over last 3 readings
|
||
|
||
Material: all numeric fields. Immaterial: individual trade P&L lines, raw
|
||
price data.
|
||
|
||
Postcondition: OSS is computed at least once per trading day. Postcondition:
|
||
when OSS falls below 0.5, at least one adaptation rule fires within 2 cycles.
|
||
Postcondition: when portfolio_beta > 0.6 AND market_regime is bearish, a
|
||
hedge or rotation rule fires within 1 cycle.
|
||
|
||
#### market-regime
|
||
The current detected market regime, refreshed each cycle from yfinance
|
||
(VIX, SPX 20-day SMA, Nasdaq 20-day SMA, BTC trend). Regimes are:
|
||
|
||
- `bull`: VIX < 18 AND SPX above 20-SMA AND Nasdaq above 20-SMA
|
||
- `neutral`: VIX 18–25 OR mixed signals (SPX and Nasdaq diverge)
|
||
- `bear`: VIX > 25 OR (SPX below 20-SMA AND Nasdaq below 20-SMA)
|
||
- `crisis`: VIX > 35 AND SPX drawdown > 10% from 30-day high
|
||
|
||
- `vix_close`: latest VIX close
|
||
- `spx_vs_sma`: SPX position relative to 20-SMA (% above/below)
|
||
- `ndx_vs_sma`: Nasdaq position relative to 20-SMA (% above/below)
|
||
- `btc_trend_5d`: BTC 5-day % change
|
||
- `regime`: bull | neutral | bear | crisis
|
||
- `regime_changed_at`: timestamp of last regime transition
|
||
|
||
Material: regime, vix_close, spx_vs_sma. Immaterial: raw tick data from yfinance.
|
||
|
||
#### parameter-state
|
||
The current values of all tunable system parameters, their bounds, the last
|
||
adjustment reason, and the OSS delta since adjustment. This is the
|
||
authoritative record — parameter changes are always written here before
|
||
taking effect.
|
||
|
||
- `max_concurrent_positions`: {value, min: 5, max: 15, adjusted_at, reason}
|
||
- `confidence_floor`: {value, min: 0.65, max: 0.85, adjusted_at, reason}
|
||
- `stop_loss_pct`: {value, min: -0.03, max: -0.07, adjusted_at, reason}
|
||
- `profit_take_pct`: {value, min: 0.03, max: 0.10, adjusted_at, reason}
|
||
- `stale_exit_days`: {value, min: 3, max: 7, adjusted_at, reason}
|
||
- `position_size_pct`: {value, min: 0.05, max: 0.15, adjusted_at, reason}
|
||
- `parker_hedge_cap_pct`: {value, min: 0.10, max: 0.30, adjusted_at, reason}
|
||
- `oss_target`: {value: 0.5, min: 0.3, max: 1.5, adjusted_at, reason}
|
||
- `oss_floor`: {value: 0.3, min: 0.1, max: 0.7, adjusted_at, reason}
|
||
- `oss_weights`: {calmar: 0.40, pf: 0.25, alpha: 0.15, win_rate: 0.10,
|
||
adapt_wr: 0.10, min_per_weight: 0.05, max_per_weight: 0.50, adjusted_at,
|
||
reason} — weights must sum to 1.0
|
||
- `hedge_regime_buckets`: {moderate: {min_bearish: 0.20, max_positions: 1,
|
||
size_pct: 0.01}, heavy: {min_bearish: 0.40, max_positions: 2, size_pct: 0.015},
|
||
extreme: {min_bearish: 0.70, max_positions: 3, size_pct: 0.02}, adjusted_at,
|
||
reason}
|
||
- `track_frequencies`: per-track {eddie, maggie, frank, maya, janet, parker}
|
||
each: {runs_per_day, min: 1, max: 6, adjusted_at, reason}
|
||
- `pipeline_frequency`: {runs_per_day, min: 2, max: 6, adjusted_at, reason}
|
||
- `oss_at_adjustment`: OSS value when the last parameter change was made
|
||
- `oss_since_adjustment`: current OSS vs oss_at_adjustment delta
|
||
|
||
Material: all fields. Postcondition: every parameter change is logged with
|
||
a reason, the OSS at time of change, and the change is reversible.
|
||
|
||
#### performance-summary
|
||
The current trading performance across all venues (Alpaca equity + Robinhood
|
||
Crypto), sourced from performance_snapshots and trade_journal.db. This is the
|
||
user-facing P&L record — human-readable, benchmarked, and attributed.
|
||
|
||
- `total_equity`: current total equity across all venues (cash + positions)
|
||
- `cash`: idle cash not in positions
|
||
- `realized_pnl`: sum of realized_trades.realized_pnl over the trailing 30 days
|
||
- `unrealized_pnl`: sum of positions.unrealized_pnl across all open positions
|
||
- `win_rate`: winning_trades / total_trades from performance_snapshots
|
||
- `profit_factor`: gross_profit / |gross_loss|
|
||
- `total_return_pct`: (end_equity - start_equity) / start_equity × 100
|
||
- `max_drawdown_pct`: largest peak-to-trough decline in equity
|
||
- `max_drawdown_usd`: dollar value of the worst drawdown
|
||
- `sharpe_ratio`: risk-adjusted return (from performance_snapshots)
|
||
- `benchmark_return_pct`: SPY return over the same period (for alpha calculation)
|
||
- `alpha_pct`: total_return_pct - benchmark_return_pct (our edge over passive)
|
||
- `per_track_pnl`: per-track attribution — for each track that produced trades:
|
||
{track: str, trades: int, win_rate: float, total_pnl: float, avg_hold_days: float}
|
||
- `per_venue_pnl`: {alpaca_equity: {trades, pnl}, robinhood_crypto: {trades, pnl}}
|
||
- `daily_pnl_7d`: array of last 7 days: [{date, pnl, cumulative_equity}]
|
||
- `largest_winner`: {symbol, pnl, opened_at, closed_at, track}
|
||
- `largest_loser`: {symbol, pnl, opened_at, closed_at, track}
|
||
- `open_positions`: [{symbol, qty, entry_price, current_price, unrealized_pnl, days_held}]
|
||
|
||
Material: all numeric and attribution fields. Immaterial: individual order IDs,
|
||
raw JSON. Postcondition: all closed trades in the period are counted.
|
||
Postcondition: open-position prices are current within 5 minutes.
|
||
|
||
### Continuity
|
||
|
||
- self-driven: re-check every 5 minutes
|
||
- input-driven: wake immediately when Kojo audit detects anomalies (open orders,
|
||
stale tracks, DNS failures, reconcile drift)
|
||
- input-driven: wake when Sophie+Ross pipeline completes (new consensus decisions
|
||
or execution results may reveal blocks)
|
||
|
||
### Strategies
|
||
|
||
- When a block is internal and matches a known remediation rule: self-heal
|
||
immediately, verify, log, and continue
|
||
- When a block is internal but no remediation rule matches: escalate to user
|
||
with full diagnostic context (which gate, what state triggered it, what
|
||
data was used)
|
||
- When a block is external: document the dependency, record the failure, and
|
||
do NOT attempt to fix (external dependencies are out of scope)
|
||
- When a block persists across 3 self-heal attempts: escalate — do not loop
|
||
- During market-closed hours: position-exit rules and circuit-breaker resets
|
||
still run (lifecycle hooks are NOT gated on market-open)
|
||
- Asset-class isolation: equity position limits never block crypto trades and
|
||
vice versa (per-platform capital pools)
|
||
- Exit orders always bypass circuit breakers and position-count gates (exits
|
||
are self-curing — blocking them creates infinite loops)
|
||
- When classification is ambiguous (could be internal or external): classify
|
||
as internal and self-heal — it is cheaper to attempt a fix and fail than to
|
||
leave a valid trade blocked
|
||
- **Return-maximization strategies:**
|
||
- When market regime is `bear` AND portfolio_beta > 0.6: positions are
|
||
market-tracking into a downtrend — rotate from high-beta equities to
|
||
defensive/uncorrelated (inverse ETFs via Parker, consumer staples, utilities,
|
||
or cash). Target: reduce portfolio_beta to ≤ 0.3 within 2 cycles
|
||
- When market regime is `bull` AND portfolio_beta < 0.3: the system is
|
||
under-allocated to the uptrend — increase exposure by allowing higher
|
||
position counts (up to MAX_CONCURRENT_POSITIONS) and larger allocation per
|
||
position (up to 15% of equity)
|
||
- When market regime is `crisis`: all discretionary buys are paused regardless
|
||
of consensus — only Parker hedge signals and manual exits execute. Existing
|
||
long positions are evaluated for stop-loss exits at -5%
|
||
- When OSS is `degrading` (>2 consecutive drops): investigate which component
|
||
is dragging (usually max drawdown widening or win rate dropping). If max
|
||
drawdown is the culprit, tighten stop-losses. If win rate is dropping, reduce
|
||
new position entry velocity — raise confidence floor by 0.05
|
||
- Every adaptation produces a before/after record: {regime, beta_before,
|
||
action, beta_after, 7d_forward_return}. These records feed back into
|
||
`win_rate_adaptations` — adaptations that repeatedly fail to improve returns
|
||
are retired
|
||
- Cash as a position: when no track produces signals with confidence ≥ 0.75
|
||
for 3 consecutive cycles, default to cash. Cash allocation is tracked as a
|
||
deliberate optimization choice, not a system failure
|
||
- Discovery continuity: ticker discovery is not gated on OSS performance.
|
||
The system SHALL run discovery lenses every cycle regardless of whether
|
||
current positions are profitable. A system that only looks for new tickers
|
||
when things are going well is blind exactly when it needs new sources of
|
||
return most. Discovery is cheap (screening); admission is careful (gates).
|
||
- Rotation constraints: no single adaptation rule may allocate > 30% of
|
||
available equity without corroborating signals from at least 2 tracks.
|
||
Parker hedge allocation is capped at 25% of equity regardless of regime
|
||
|
||
### Remediation Rules
|
||
|
||
#### Rule 1: Stale Circuit Breaker
|
||
**Detect:** An active circuit breaker in trade_journal.db with `triggered_at`
|
||
older than `BREAKER_AUTO_RESET_HOURS` (4h) AND today's daily loss below
|
||
`BREAKER_MIN_LOSS_PCT` (2%) of equity.
|
||
**Fix:** Deactivate the breaker by setting `active = 0` in circuit_breaker table
|
||
and log the reset with reason "Auto-reset: {hours}h idle, daily loss below
|
||
threshold".
|
||
**Verify:** Re-query circuit_breaker — `active = 0`. Re-compute daily P&L —
|
||
still below threshold.
|
||
**Escalate:** After 2 reset attempts within 24h, the breaker is re-tripping
|
||
from live losses. Escalate to user with the full breaker history.
|
||
|
||
#### Rule 2: Position-Limit Block Preventing Exits
|
||
**Detect:** Gate 2 (position-count) is blocking an order, AND the order
|
||
direction is `bearish`/`sell`/`short` (an exit).
|
||
**Fix:** This is a false-positive — exits reduce position count and are
|
||
self-curing. The gate code already handles this (is_exit check in gates.py:50),
|
||
so this rule catches edge cases where direction is misclassified. Re-evaluate
|
||
direction from the consensus decision's `direction` field and retry with
|
||
corrected classification.
|
||
**Verify:** Exit order submitted successfully.
|
||
**Escalate:** After 2 failures — escalate with the exact consensus decision
|
||
JSON and gate evaluation trace.
|
||
|
||
#### Rule 3: Poll-Fill Timeout (Order Actually Filled)
|
||
**Detect:** Ross executor reports `error_poll_timeout` on a submitted order,
|
||
but `_verify_order_after_timeout()` was either not called or returned None.
|
||
**Fix:** Re-query the broker for the order by `client_order_id`. If filled:
|
||
journal the fill and mark as executed. If still pending: poll once more
|
||
(30s). If genuinely unfilled: cancel and re-submit.
|
||
**Verify:** Order status is terminal (filled/partially_filled/canceled).
|
||
**Escalate:** After 2 re-submit attempts — escalate with the order ID and
|
||
full poll history.
|
||
|
||
#### Rule 4: Stale Track Data Blocking Consensus
|
||
**Detect:** A track's `last_signal` timestamp is beyond its freshness threshold
|
||
(Eddie 72h, Maggie 168h, Frank 168h, Maya 24h, Janet 12h, Parker 24h) AND
|
||
`track_status` shows `last_run_at` is ALSO stale (no recent run).
|
||
**Fix:** Trigger the track's cron job immediately via `cronjob(action='run',
|
||
job_id=<track_job_id>)`. Wait for completion (max 120s). Re-read signals.db.
|
||
**Verify:** New signals exist OR track_status shows a fresh run (even if 0
|
||
signals — `no_data` is valid).
|
||
**Escalate:** If track run fails or times out — escalate. If track ran but
|
||
produced 0 signals for 3 consecutive cycles — escalate (possible source
|
||
degradation).
|
||
|
||
#### Rule 5: Orphaned Consensus (Execute Decisions Never Traded)
|
||
**Detect:** Gap analysis (`sources.reconcile` in auditor, or direct query:
|
||
consensus rows with `action='execute'` that lack a matching row in `trades`
|
||
table, AND created_at > 1h ago).
|
||
**Fix:** Ross orchestrator already self-heals orphaned consensus (lines 166-217
|
||
of orchestrator.py). Verify the self-healing ran: check that orphaned consensus
|
||
IDs now have trade rows. If not: manually invoke `mark_ticker_unavailable` for
|
||
tickers that genuinely cannot trade (SPAC, delisted, etc.) and re-scan.
|
||
**Verify:** Zero orphaned execute decisions older than 1h.
|
||
**Escalate:** After 2 cycles of failed self-healing — escalate with orphaned
|
||
consensus IDs.
|
||
|
||
#### Rule 6: DNS / Data-Source Unreachable
|
||
**Detect:** `sources.dns` shows fc.yahoo.com unreachable (yfinance failing)
|
||
OR any track's data source returns connection errors for > 2 consecutive runs.
|
||
**Fix:** External dependency — do NOT attempt to fix. Verify the failure is
|
||
genuinely external (network-layer, not auth/permission). Log and classify
|
||
as `external`.
|
||
**Verify:** Re-test DNS resolution and connectivity. If recovered: was
|
||
transient. If still failed: confirmed external.
|
||
**Escalate:** After 3 consecutive failures — escalate with the hostname,
|
||
error string, and affected tracks.
|
||
|
||
#### Rule 7: Multi-Venue Asset Misrouting
|
||
**Detect:** A crypto ticker (BTC-USD, ETH-USD, SOL-USD) triggers an Alpaca
|
||
equity error ("BUY needs notional, SELL needs qty" or "symbol not found")
|
||
OR an equity ticker hits Robinhood Crypto.
|
||
**Fix:** Verify `infer_asset_class(ticker)` in config.py returned the correct
|
||
class. If the ticker is missing from `CRYPTO_SYMBOLS`: add it (after verifying
|
||
on docs.robinhood.com/crypto/trading). If the code path passed wrong
|
||
`asset_class` to `execute_trade`: fix the call site.
|
||
**Verify:** Re-run the trade with corrected asset class — order routes to
|
||
correct adapter.
|
||
**Escalate:** After 1 failure — always escalate (misrouting means money
|
||
on wrong venue). Include the ticker, the inferred class, and the expected class.
|
||
|
||
#### Rule 8: Market-Regime Position Mismatch
|
||
|
||
**Detect:** `market_regime` is `bear` or `crisis` AND `portfolio_beta` > 0.6.
|
||
Positions are tracking the market downward — holding long equities in a
|
||
broad sell-off.
|
||
|
||
**Principle — Regime-graded hedging intensity:** Hedging intensity scales
|
||
with regime severity. As VIX rises from 18→35+, the system progressively
|
||
shifts defense posture: cash accumulation → defensive sector rotation →
|
||
1x inverse ETFs → leveraged inverse ETFs. The contract defines the
|
||
direction of travel; the system discovers the optimal allocation
|
||
percentages through OSS feedback and per-strategy attribution (P-004).
|
||
|
||
**Fix:** (a) If Parker already has active `buy` signals for inverse ETFs
|
||
(SH, SQQQ, SPXU, BITI): execute them at up to 25% equity allocation.
|
||
1x inverse (SH, PSQ) is preferred over leveraged (-2x/-3x) unless
|
||
VIX > 30 and regime is `crisis` — per Rule 18 decay risk.
|
||
(b) If no Parker signals: scan existing long positions for those with the
|
||
highest beta correlation to SPY and generate exit signals (stale-exit or
|
||
stop-loss at -5%). Rotate freed capital to defensive sectors (consumer
|
||
staples, healthcare, utilities) or cash — these provide positive
|
||
expected return in down markets with zero leverage decay risk.
|
||
(c) If VIX > 30 and beta > 0.8: go to 50% cash — sell
|
||
the 2 largest high-beta positions. Cash at 4-5% risk-free yield is a
|
||
competitive position, not a retreat.
|
||
**Verify:** Re-compute portfolio_beta after adaptation — target ≤ 0.3.
|
||
Re-compute OSS on next cycle — improvement expected.
|
||
**Escalate:** If beta remains > 0.5 after 2 adaptations in bear regime —
|
||
escalate with full portfolio composition and regime data.
|
||
|
||
#### Rule 9: Regime-Inappropriate Sector Exposure
|
||
**Detect:** Market regime is `bear` AND portfolio holds > 2 positions in
|
||
high-beta sectors (tech, consumer discretionary, small-cap, crypto) that
|
||
are each down > 3% since entry.
|
||
**Fix:** Generate exit signals for the 2 worst performers. Rotate capital
|
||
into either (a) Parker inverse ETFs if bearish signal exists, or (b) cash
|
||
if no hedge signal. Do NOT buy new long positions until regime normalizes.
|
||
**Verify:** High-beta exposure reduced by ≥ 40%. Exit orders confirmed filled.
|
||
**Escalate:** If exits fill but capital sits idle > 3 cycles without a
|
||
viable defensive entry — escalate with regime timeline.
|
||
|
||
#### Rule 10: OSS Degradation — Drawdown-Driven
|
||
**Detect:** OSS is `degrading` AND the Calmar Ratio component dropped > 20%
|
||
from its 30-day peak. This means max drawdown is widening faster than returns
|
||
are growing — the system is taking pain without reward.
|
||
**Fix:** Reduce position sizes by 25% across all new entries (scale factor
|
||
0.75x). Tighten stop-loss to -3% from -5% for new positions. Do NOT modify
|
||
position-sizes for existing positions (only new entries). Pause the track
|
||
with the worst 7-day return if its win rate < 40%.
|
||
**Verify:** Re-compute max drawdown after 3 cycles — trajectory should
|
||
flatten. Paused track's win rate trend should stabilize.
|
||
**Escalate:** If max drawdown continues to widen after 5 cycles — escalate
|
||
with full drawdown waterfall chart and per-track P&L attribution.
|
||
|
||
#### Rule 11: Hedge Inefficiency — Hedges Costing More Than Saving
|
||
**Detect:** Parker hedge positions are open AND `profit_factor` on hedge-only
|
||
trades (filtered by Parker track ID) is < 0.8 (hedges are net losers) over
|
||
the last 30 days AND market regime is NOT `crisis`.
|
||
**Fix:** Hedge drag is real — inverse ETFs have daily decay. Reduce Parker
|
||
allocation to 10% of equity (from 25% cap). Only hold SH (1x inverse, no
|
||
decay) unless VIX > 30 (then SQQQ/SPXU are warranted). Generate exit signals
|
||
for leveraged hedges (SQQQ, SPXU) that are > 3 days old.
|
||
**Verify:** Hedge P&L improves within 5 cycles (profit_factor > 0.9).
|
||
**Escalate:** If hedge drag persists — escalate with per-ticker hedge P&L
|
||
and hold durations.
|
||
|
||
#### Rule 12: Continuous Ticker Discovery
|
||
|
||
**Principle — Ticker discovery is continuous, not conditional.** Alpha
|
||
factors decay over time (Hubble 2026, AlphaCrafter 2026). The system
|
||
SHALL maintain an active pipeline of candidate tickers across multiple
|
||
discovery lenses regardless of current OSS or portfolio state. Discovery
|
||
happens in the background; admission to the portfolio is gated separately.
|
||
|
||
**Discovery lenses (at least 3 must be active per cycle):**
|
||
|
||
| Lens | Source | What it finds |
|
||
|------|--------|---------------|
|
||
| Fundamental | Frank (financials/screeners) | Undervalued/quality stocks |
|
||
| Insider/Event | Maggie (insider, SEC filings) | Asymmetric-info opportunities |
|
||
| Technical/Momentum | Price/volume scans | Breakouts, trend initiations, RSI divergences |
|
||
| Inverse/Hedge | Parker (inverse ETFs) | Down-market defense candidates |
|
||
| Regime-Appropriate | Market regime × sector data | Sector rotation into regime-favored sectors |
|
||
| Correlation-Seeking | Correlation matrix vs existing positions | Uncorrelated diversifiers |
|
||
|
||
**Detect (admission gate):** A new candidate ticker is discovered with:
|
||
(a) signal confidence ≥ 0.70 from its source lens,
|
||
(b) Pearson correlation ≤ 0.4 to ALL existing open positions
|
||
over the trailing 30-day window,
|
||
(c) position-count headroom exists (current positions < MAX_POSITIONS
|
||
after accounting for pending orders), AND
|
||
(d) the ticker passes all standard gates (Gate 1-7).
|
||
|
||
**Fix:** Admit the candidate to the consensus pipeline with a
|
||
`discovery_lens` tag. For the first 2 cycles: use 50% normal position
|
||
size (probation sizing). After 2 cycles with positive unrealized P&L:
|
||
graduate to full position size.
|
||
|
||
**Verify:** At least 3 discovery lenses produced candidates this cycle.
|
||
At least 1 candidate admitted OR clearly documented "no viable candidate
|
||
passes correlation gate." Per-source yield tracked (see P-005).
|
||
|
||
**Escalate:** If zero candidates pass correlation gate for 5 consecutive
|
||
cycles — the system's existing positions are too correlated with each
|
||
other or the discovery universe is too narrow. Escalate with full
|
||
correlation matrix and per-lens candidate count.
|
||
|
||
#### Rule 13: Adaptive Parameter Tuning
|
||
**Detect:** OSS trend is `degrading` OR `improving` over 5+ cycles AND a
|
||
specific OSS component (Calmar, Profit Factor, Alpha) is the dominant driver.
|
||
**Fix — Confidence Floor:** When OSS is improving (bull confirmation): lower
|
||
`confidence_floor` by 0.05 (min 0.65) to capture more entries. When OSS is
|
||
degrading (entries are losing): raise `confidence_floor` by 0.05 (max 0.85)
|
||
to filter out low-quality signals.
|
||
**Fix — Position Size:** When Calmar is the degraded component (drawdown
|
||
widening): reduce `position_size_pct` by 0.02 (min 0.05). When Calmar is
|
||
improving and regime is `bull`: increase `position_size_pct` by 0.02 (max 0.15).
|
||
**Fix — Stop-Loss:** When Profit Factor < 0.8 (losers are too large):
|
||
tighten `stop_loss_pct` to -0.03. When Profit Factor > 2.0 (winners dominate):
|
||
relax `stop_loss_pct` to -0.07.
|
||
**Fix — Profit-Take:** When bull regime and Win Rate > 60%: increase
|
||
`profit_take_pct` to 0.08 (let winners run). When bear regime or Win Rate
|
||
< 40%: decrease `profit_take_pct` to 0.04 (take profits faster).
|
||
**Fix — Max Positions:** When OSS > 1.2 and regime is `bull` and cash > 30%:
|
||
increase `max_concurrent_positions` by 2 (max 15). When OSS < 0.5:
|
||
decrease by 2 (min 5).
|
||
**Verify:** Re-compute OSS after 3 cycles — the adjusted component should
|
||
show improvement. If not: revert the parameter to its prior value.
|
||
**Escalate:** If any parameter change fails to improve its target component
|
||
within 3 cycles — auto-unwind and escalate with before/after data.
|
||
|
||
#### Rule 14: Dynamic Cron Frequency Adjustment
|
||
**Detect:** Signal-producing tracks (tracks that generated ≥3 signals in
|
||
their last 3 runs) are running below their maximum frequency AND market
|
||
regime is `bull` OR `bear` (not `neutral`) — the market is moving and
|
||
opportunities exist. Conversely: tracks producing 0 signals for 8+
|
||
consecutive runs in a `neutral` regime are wasting compute.
|
||
**Fix — Increase frequency:** For signal-producing tracks in directional
|
||
markets, increase `runs_per_day` by 1 (max 6). Also increase
|
||
`pipeline_frequency` by 1 (max 6) to process the additional signals.
|
||
**Fix — Decrease frequency:** For silent tracks in neutral markets, decrease
|
||
`runs_per_day` to 1 (min). This preserves API call budget for active tracks.
|
||
**Implementation:** Update the track's cron job schedule via
|
||
`cronjob(action='update', job_id=<track_job_id>, schedule=<new_cron_expr>)`.
|
||
Do NOT pause or remove jobs — only adjust their schedule.
|
||
**Verify:** Track runs at new frequency within 1 day. Signal count per
|
||
day increases (for frequency-up) or API call count decreases (for
|
||
frequency-down) without signal quality degradation.
|
||
**Escalate:** If increased frequency produces 0 additional signals after 3
|
||
cycles — revert to prior frequency (wasted compute). If decreased frequency
|
||
causes a track to miss a regime-relevant signal — escalate with the missed
|
||
signal details.
|
||
|
||
#### Rule 15: Breaker Cycling — Block/Reset Loop
|
||
**Detect:** The same breaker type (position-limit, loss-limit) re-triggers
|
||
within 30 minutes of a Rule 1 auto-reset, AND no exit orders filled between
|
||
the reset and the re-trigger. This indicates the breaker is mechanically
|
||
correct but structurally insufficient — exits are not happening fast enough
|
||
to sustain the reset.
|
||
|
||
**Fix:** Do NOT wait for another 4h cycle. Classify as INTERNAL structural
|
||
deadlock. Generate exit orders for the 2 highest-beta positions (sourced
|
||
from portfolio_beta.py) that meet **relaxed** exit criteria: held ≥1 day
|
||
(instead of 3) OR profit ≥1% (instead of 5%). If no positions meet even
|
||
relaxed criteria, go to 50% cash — sell the 2 highest-beta positions at
|
||
market regardless of profit. The breaker is self-correct; the exits are not.
|
||
|
||
**Verify:** At least 2 exit orders submitted with status `accepted` or
|
||
`filled`. Re-compute position count — < 10/10.
|
||
|
||
**Escalate:** After 1 cycle — breaker cycling is a structural deadlock. The
|
||
contract has exhausted its self-resolution surface. Escalate with: (a) the
|
||
breaker history (IDs and timestamps of the last 3), (b) all current positions
|
||
with age + beta + unrealized P&L, (c) which exit conditions each position
|
||
failed, and (d) the specific relaxed criteria applied.
|
||
|
||
#### Rule 16: Data Pipeline Integrity — Stale Local Tables
|
||
**Detect:** Any OSS sub-component or performance-summary field returns 0, None,
|
||
or a value that contradicts live broker API state. Specifically: (a) open
|
||
position count from `positions` table ≠ live API count, (b) unrealized P&L
|
||
from local tables differs from live API by >20%, (c) portfolio_beta computed
|
||
from local data produces NaN or 0 when positions exist.
|
||
|
||
**Fix:** Bypass the local table. Query the live broker adapter
|
||
(`AlpacaLiveAdapter.get_positions()` for equity, `RobinhoodCrypto.get_positions()`
|
||
for crypto) directly. Recompute the affected sub-component from live data.
|
||
Log the stale table name and the delta between local and live data.
|
||
|
||
**Verify:** Re-computed sub-component matches live API within ±5% tolerance.
|
||
The stale table is flagged in the health report for manual repair.
|
||
|
||
**Escalate:** If the same table is flagged stale for 2 consecutive cycles —
|
||
escalate with the table name, row count delta, and whether the table schema
|
||
needs migration or the write path is broken. A table that was never populated
|
||
(`positions`) is a code bug; a table that was populated but is now stale is
|
||
a sync gap.
|
||
|
||
**Source quality enhancement:** After checking data freshness, the system
|
||
SHALL compute Source Health Scores (P-006) for each active data source.
|
||
Sources with SHS < 0.40 in the current cycle SHALL be logged with
|
||
degradation reason (low authority, staleness, narrow coverage, or novelty
|
||
exhaustion). Sources with SHS < 0.25 for 3 consecutive cycles SHALL be
|
||
replaced — the system searches for alternative free sources covering the
|
||
same data domain (e.g., replacing a stale screener with an alternative
|
||
API endpoint, adding a new financial data provider, broadening from
|
||
single-exchange to multi-exchange data).
|
||
|
||
**Verify:** All active data sources have SHS ≥ 0.25. At least one source
|
||
improvement attempted in the last 30 cycles. Source provenance tags
|
||
present on all signals generated this cycle.
|
||
|
||
#### Rule 17: Signal Direction & Action Validation
|
||
**Detect:** A consensus decision has `action=execute` but the combination
|
||
of ticker, direction, and position state makes execution impossible or
|
||
meaningless: (a) `direction=neutral` — non-actionable by definition,
|
||
(b) `direction=bearish` AND no existing position held in the live portfolio
|
||
(exit without entry), (c) `direction=bearish` on an inverse-ETF ticker
|
||
(SH, SQQQ, SPXU, BITI) where no buy fill exists — hedge exit without hedge
|
||
entry.
|
||
|
||
**Fix:** Reclassify `action=skip` in the consensus table with a machine-readable
|
||
reason: `skipped_neutral`, `skipped_no_position`, or `skipped_no_hedge_entry`.
|
||
For case (a): log and skip — neutral signals are information, not action.
|
||
For case (b): log and skip — exit signals for unheld tickers are noise.
|
||
For case (c): log and skip — Parker exit signals are only valid when a
|
||
corresponding buy fill exists in broker_orders.
|
||
|
||
**Verify:** Consensus action updated to `skip`. No future Ross execution
|
||
attempt for this ticker×direction without a fresh buy signal first.
|
||
|
||
**Escalate:** Never — these are signal-quality filters, not system failures.
|
||
They reduce noise and prevent the orchestrator from attempting impossible
|
||
trades.
|
||
|
||
#### Rule 18: Inverse ETF Decay Shield
|
||
|
||
**Detect:** An open Parker hedge position uses a leveraged inverse ETF
|
||
(SQQQ, SPXU, SOXS, BITI — any -2x or -3x daily-reset instrument) AND
|
||
at least one decay condition is met:
|
||
|
||
(a) **Duration decay:** Hold duration > 10 trading days. Daily-reset
|
||
decay compounds with time — leveraged inverse ETFs held beyond 2 weeks
|
||
lose significant value even if the underlying index direction is correct.
|
||
|
||
(b) **Low-VIX decay:** VIX < 20. Implied volatility is too low to
|
||
justify the leverage cost and daily rebalancing drag. Leveraged
|
||
inverse instruments are designed for high-volatility environments.
|
||
|
||
(c) **Choppy-market decay:** Market texture is "choppy" — SPX
|
||
oscillating within ±3% of its 20-day SMA for >5 consecutive trading
|
||
days. In range-bound markets, the daily-reset mechanism of leveraged
|
||
ETFs maximizes volatility decay regardless of eventual direction.
|
||
|
||
**Fix:** Exit the leveraged inverse position immediately. Log the exit
|
||
reason as `decay_duration`, `low_vix`, or `choppy_decay`.
|
||
|
||
If a hedge is STILL warranted (regime remains `bear` AND portfolio_beta
|
||
> 0.5 after exit): open a replacement position using the 1x inverse
|
||
equivalent — SH (S&P 500 -1x) for broad market exposure, PSQ
|
||
(Nasdaq -1x) for tech exposure. 1x inverse ETFs have no daily-reset
|
||
decay and can be held for the duration of the bear regime.
|
||
|
||
If no hedge is warranted: rotate the freed capital to cash or defensive
|
||
sectors per Rule 8.
|
||
|
||
**Verify:** Leveraged inverse positions closed. If regime still warrants
|
||
hedge: replacement 1x inverse position opened OR defensive rotation
|
||
executed. portfolio_beta ≤ 0.5 within 2 cycles.
|
||
|
||
**Escalate:** If the same ticker's leveraged inverse position is
|
||
exited-and-reopened >2 times in 30 days — the system is cycling in
|
||
and out of leveraged hedges. Escalate with per-exit decay cost
|
||
estimate (entry price vs exit price, holding days) and recommend
|
||
staying in 1x inverse until VIX > 25.
|
||
|
||
#### Rule 19: Missed Signal Hindsight Detection
|
||
|
||
**Principle — The system learns from what it missed.** Credit assignment
|
||
research (Zhang 2026, 47 methods surveyed) demonstrates that post-hoc
|
||
hindsight analysis — using information available only AFTER the trading
|
||
window closes — identifies root causes of missed opportunities that
|
||
feed-forward detection cannot see.
|
||
|
||
**Detect (runs every 10 cycles, counterfactual analysis):**
|
||
For each of the 10 largest daily movers (by absolute % change) in the
|
||
system's benchmark universe over the past 10 trading days:
|
||
|
||
(a) Did any Persona generate a signal for this ticker?
|
||
→ NO: failure mode = coverage gap (source didn't see it)
|
||
→ YES, but confidence < threshold: failure mode = signal too weak
|
||
→ YES, confidence ≥ threshold, but didn't trade: failure mode = gate
|
||
filtering (which gate blocked it?)
|
||
|
||
(b) If a trade COULD have been opened (signal existed, confidence ≥ 0.70,
|
||
passed all gates) but wasn't: failure mode = execution gap
|
||
|
||
**Fix — Root cause triage:**
|
||
|
||
| Root cause | Contract action |
|
||
|-----------|-----------------|
|
||
| Coverage gap (no signal) | Add ticker to discovery candidate list. If ≥3 missed tickers from same sector: broaden that discovery lens's screening criteria |
|
||
| Signal too weak (conf < 0.70) | Log; if same ticker missed 3+ times: lower confidence floor for that lens by 0.05 (min 0.60) |
|
||
| Gate filtering (passed source, blocked by gate) | Log which gate blocked it. If same gate blocks ≥3 viable trades in 10 cycles: escalate for gate calibration review |
|
||
| Execution gap (could have, didn't) | Escalate — this is an orchestrator or order-routing failure |
|
||
|
||
**Verify:** Hindsight analysis completed for 10 largest movers. At least
|
||
1 root cause identified OR documented "no missed opportunities." Source
|
||
coverage gap counts trend downward over rolling 90-day window.
|
||
|
||
**Escalate:** If coverage gap count increases for 3 consecutive hindsight
|
||
cycles — the system's data universe is shrinking, not expanding. Escalate
|
||
with per-source coverage breadth trend.
|
||
|
||
### Invariants
|
||
|
||
- **P-001 — Paper trading is dead code.** The paper trading adapter
|
||
(`brokers/alpaca_paper.py`) is NEVER imported, NEVER invoked, and has NO
|
||
code path in any production pipeline, cron job, or system module. The
|
||
`ALPACA_ENV` routing mechanism no longer supports a `paper` mode — all
|
||
adapters route to live production only. The file may exist on disk as a
|
||
historical artifact but carries zero operational risk because (a) no import
|
||
chain reaches it, (b) `brokers/__init__.py` no longer exports
|
||
`AlpacaPaperAdapter`, (c) `brokers/router.py` hardcodes live-only routing,
|
||
(d) `reconcile.py` hardcodes live-only reconciliation, (e)
|
||
`consensus/ross/adapter.py` hardcodes live-only adapter selection, and
|
||
(f) all metric queries filter `venue != 'alpaca_paper'` as defense-in-depth.
|
||
Only live production trading across Alpaca Live and Robinhood Crypto SHALL
|
||
count in any performance calculation, OSS computation, equity curve,
|
||
drawdown measurement, P&L report, or health evaluation.
|
||
|
||
- **P-002 — Dust positions are filtered from position counting.** Positions
|
||
with `|market_value| < $0.01` (one cent) are considered dust — fractional
|
||
remnants from fills/partials that have zero economic significance. Gate 2
|
||
(position-count limit, `gates.py:144-150`) and Gate 7 (gross-exposure cap,
|
||
`gates.py:270-294`, all four sum expressions) SHALL filter these positions
|
||
using `>= cfg.DUST_POSITION_THRESHOLD` (functionally equivalent to the
|
||
former `> 0` since all dust has market_value ≤ $0.00). `compute_oss.py`
|
||
(`scripts/compute_oss.py:182-184`) also SHALL filter dust from open position
|
||
counting. This invariant is verified by 4 property-based tests
|
||
covering 800+ generated examples (see `tests/test_gate2_dust_filter.py` and
|
||
`tests/test_gate2_property_invariants.py`):
|
||
- **INV-001**: Any number of dust positions never blocks Gate 2 (200 examples)
|
||
- **INV-002**: If Gate 2 blocks, non-dust count ≥ limit (300 examples)
|
||
- **INV-003**: Adding dust positions never changes allowed→blocked (200 examples)
|
||
- **INV-004**: Filter applied symmetrically to both equity and crypto (100 examples)
|
||
The threshold constant `DUST_POSITION_THRESHOLD = 0.01` is defined at
|
||
`consensus/ross/config.py:76` (single source of truth) and SHALL NOT be
|
||
raised without Theo's approval — raising it would hide small but real
|
||
positions; lowering it is safe but unnecessary.
|
||
|
||
- **P-003 — Returns are trade-attributed, not balance-based.** The equity curve
|
||
in *every* performance computation (Sharpe ratio, max drawdown, total return
|
||
percentage, OSS rolling alpha, and Calmar ratio) SHALL be built from actual
|
||
trading activity — realized P&L from closed trades plus unrealized P&L from
|
||
open positions — and SHALL NEVER use raw account balances from
|
||
`sync_state.broker_equity`. Deposits, withdrawals, dividends, cash transfers,
|
||
and fees SHALL NOT be counted as trading returns.
|
||
|
||
**Equity curve formula:** `equity[t] = ACCOUNT_EQUITY + cumulative_realized_pnl[t] + unrealized_pnl[t]`
|
||
where `ACCOUNT_EQUITY` is the initial capital from `consensus/ross/config.py:119`
|
||
(single source of truth), `cumulative_realized_pnl[t]` is the sum of
|
||
`realized_pnl` from `realized_trades` (filtered `unmatched=0`, `venue != 'alpaca_paper'`)
|
||
up to day `t`, and `unrealized_pnl[t]` is the current `SUM(unrealized_pl)` from
|
||
`broker_positions` filtered by `ABS(market_value) >= DUST_POSITION_THRESHOLD`.
|
||
|
||
**Implementation:** `perf.py:compute_equity_metrics()` (lines 137–261) builds
|
||
the equity curve from trade-attributed data only. `sync_state.broker_equity` is
|
||
used for reconcile timing (date grid) but its *value* is never read for return
|
||
computation. This invariant is verified by 3 property-based tests covering
|
||
500+ generated examples (see `tests/test_perf_equity_source.py` and
|
||
`tests/test_perf_equity_property_invariants.py`):
|
||
- **INV-001 (Additivity):** `end_equity = ACCOUNT_EQUITY + Σ(realized_pnl) + unrealized_pnl` (200 examples)
|
||
- **INV-002 (Deposit Exclusion):** Deposits of any amount do not affect trade-attributed return (200 examples)
|
||
- **INV-003 (Flat when idle):** With zero trades, equity stays at `ACCOUNT_EQUITY` regardless of deposits (100 examples)
|
||
|
||
**Rationale:** On 2026-06-17, a $500 deposit into the Alpaca Live account
|
||
caused `sync_state.broker_equity` to jump from $201 to $701. The old
|
||
balance-based `compute_equity_metrics()` treated this as a 249% overnight
|
||
return, inflating the OSS from a correct value near 0 to 2.595 (false
|
||
positive). Actual realized P&L across all trades was -$1.67 on 24 trades.
|
||
The fix was validated by [fv] protocol on 2026-06-30: claim → verify →
|
||
cross-check (all 4 cron outputs, all venues, all position-counting paths) →
|
||
draft → 500+ property test examples → zero regression (140/140 tests).
|
||
|
||
- **P-004 — Hedging strategies are independently attributed and performance-graded.**
|
||
Every hedge position (inverse ETF, defensive sector rotation, cash allocation)
|
||
SHALL carry a `hedge_strategy` tag in broker_orders and realized_trades.
|
||
The health monitor SHALL compute per-strategy profit factor and win rate
|
||
independently over a rolling 60-day window:
|
||
|
||
| hedge_strategy | Instruments | Description |
|
||
|---|---|---|
|
||
| `inverse_1x` | SH, PSQ, RWM | 1x inverse ETFs — daily-reset, no leverage decay |
|
||
| `inverse_leveraged` | SQQQ, SPXU, SOXS, BITI | -2x/-3x inverse — decay risk, crisis-only |
|
||
| `defensive_sector` | XLP, XLV, XLU, consumer staples, healthcare, utilities | Long defensive — positive carry |
|
||
| `cash_allocation` | Cash (>40% of equity) | Yield-bearing, zero correlation |
|
||
|
||
Strategies with `profit_factor < 0.7` over the 60-day window SHALL be
|
||
flagged for review. Strategies with `profit_factor < 0.5` over 60 days
|
||
SHALL be retired — the contract auto-removes them from the authorized
|
||
hedging toolkit. A retired strategy may be re-evaluated after 30 days.
|
||
This invariant enables emergent optimization: the system discovers
|
||
through attribution which hedging strategies work in which regimes,
|
||
rather than having allocation percentages hardcoded.
|
||
|
||
This invariant is verified by property-based tests that confirm every
|
||
hedge order carries a valid `hedge_strategy` tag, per-strategy P&L sums
|
||
are consistent with total hedge P&L, and retired strategies are excluded
|
||
from new hedge entry signals.
|
||
|
||
- **P-005 — Ticker universe breadth is continuously monitored and diversified.**
|
||
The system SHALL maintain a minimum discovery breadth: at least 3 active
|
||
discovery lenses producing candidates each cycle, with at least 10 unique
|
||
candidate tickers evaluated in any rolling 30-day window. No single
|
||
discovery lens may account for >60% of open positions over a 60-day window
|
||
(prevents lens monoculture).
|
||
|
||
Each discovery lens SHALL carry a `discovery_lens` tag on all signals
|
||
and trades it produces. The health monitor SHALL compute per-lens yield:
|
||
|
||
| Metric | Formula | Threshold |
|
||
|--------|---------|-----------|
|
||
| Lens hit rate | realized_trades with profit > 0 / total trades from lens | Alert if < 0.3 over 60d |
|
||
| Lens profit factor | gross_profit / gross_loss from lens trades | Retire lens if < 0.5 over 60d |
|
||
| Lens novelty rate | candidates with correlation < 0.3 to existing / total candidates | Alert if < 0.2 over 30d |
|
||
|
||
A discovery lens with profit_factor < 0.5 over 60 days SHALL be paused
|
||
(its candidates are generated but not admitted). A lens with novelty_rate
|
||
< 0.2 over 30 days is producing only correlated candidates — its search
|
||
criteria SHALL be broadened.
|
||
|
||
This invariant ensures the system never settles into a narrow field of
|
||
view where it only sees tickers similar to what it already holds. The
|
||
correlation admission gate (Rule 12) and novelty monitoring (P-005)
|
||
together force the system to continuously expand its field of view.
|
||
|
||
- **P-006 — Data source quality and coverage improve continuously through
|
||
experiential feedback.** Every Persona's data source carries a quality
|
||
score that evolves over time based on downstream trade outcomes and
|
||
timeliness. Source quality degrades when signals from that source lead
|
||
to losing trades or arrive too late; source quality improves when
|
||
signals lead to winning trades with timely execution.
|
||
|
||
**Source quality dimensions (scored per source per cycle, 0.0–1.0):**
|
||
|
||
| Dimension | Formula | Degradation trigger |
|
||
|-----------|---------|-------------------|
|
||
| Authority | rolling 60d hit rate: winning trades / total trades from source | < 0.35 over 60 days |
|
||
| Timeliness | signals that resulted in fills before adverse price move / total signals | < 0.50 over 30 days |
|
||
| Coverage breadth | unique tickers surfaced / unique tickers in benchmark universe | < 0.10 over 90 days |
|
||
| Novelty decay | new (unseen) tickers surfaced this cycle / total tickers surfaced | < 0.05 over 30 days |
|
||
|
||
The combined Source Health Score (SHS) = (authority × 0.4) + (timeliness ×
|
||
0.3) + (coverage × 0.2) + (novelty × 0.1). A source with SHS < 0.40
|
||
SHALL be flagged for review. A source with SHS < 0.25 SHALL be replaced
|
||
(the system discovers an alternative free source covering the same data
|
||
domain) or retired (its signals are generated but not admitted to the
|
||
consensus pipeline).
|
||
|
||
**Source provenance:** Every signal entering the consensus pipeline
|
||
SHALL carry a `source_provenance` tag identifying: (a) the Persona that
|
||
generated it, (b) the data source(s) it drew from, (c) the timestamp of
|
||
the underlying data, and (d) the confidence score. This enables
|
||
post-hoc tracing: when a trade wins or loses, the system knows which
|
||
data source contributed to the decision.
|
||
|
||
**Coverage improvement obligation:** Every 30 cycles (≈ trading month),
|
||
the system SHALL attempt to add at least one new free data source or
|
||
expand an existing source's coverage (additional tickers, additional
|
||
data fields, additional frequency). New sources start with probationary
|
||
SHS = 0.50 and are evaluated after 30 cycles. This is the mechanism by
|
||
which the system's field of view widens over time.
|
||
|
||
This invariant is verified by property-based tests that confirm every
|
||
signal carries a valid `source_provenance` tag, SHS scores are computed
|
||
for all active sources, and retired sources are excluded from signal
|
||
generation.
|
||
|
||
- **P-007 — Benchmark integrity is self-validating.** Every alpha and beta
|
||
computation SHALL use a live, point-in-time benchmark return fetched this
|
||
cycle via `yfinance` (^GSPC). The benchmark source SHALL NOT contain
|
||
hardcoded literals — any constant appearing in benchmark calculation is
|
||
a contract violation. Alpha sign SHALL flip correctly when portfolio
|
||
return crosses the benchmark return (positive alpha = outperformance).
|
||
Benchmark fetch failure SHALL be surfaced as `benchmark_error` in the
|
||
OSS output rather than silently using a stale value. Property-tested
|
||
via `test_benchmark_integrity.py`.
|
||
|
||
- **P-008 — Trade execution costs are modeled and bounded.** No trade SHALL
|
||
be admitted whose expected edge (signal confidence × signal strength) is
|
||
less than the modeled round-trip cost (half-spread + commission +
|
||
slippage). The cost model SHALL use realized `slippage_bps` from
|
||
`broker_orders` as feedback — if median slippage exceeds 50 bps over a
|
||
30-day rolling window, the cost gate SHALL tighten (raise the admission
|
||
threshold). A cost-inefficient trade that slips past the gate SHALL
|
||
contribute to `cost_breach_count` and degrade the OSS via the Cost-
|
||
Aware Gate (Gate 2.3). StockBench/FINSABER evidence: for a $1K account,
|
||
trading 1% positions, a single 50-bps slip on entry AND exit is $0.10
|
||
round-trip (0.01%) — negligible. The gate exists to prevent death by
|
||
a thousand cuts, not to reject single marginal trades.
|
||
|
||
- **P-009 — The Optimization Success Score and every sub-metric are sign-
|
||
correct and monotonic.** The Calmar Ratio SHALL be negative when CAGR is
|
||
negative (never `abs(CAGR) ÷ max_drawdown`). Rolling Alpha SHALL be
|
||
positive when portfolio return exceeds benchmark return. A losing trading
|
||
period (negative total P&L, negative CAGR) SHALL NEVER produce an OSS ≥
|
||
a winning trading period (positive total P&L, positive CAGR). The
|
||
`oss_trend` indicator SHALL correctly reflect the direction of the last 3
|
||
OSS readings: improving (rising), flat (within ±0.05 band), or degrading
|
||
(falling). Property-tested via `test_oss_monotonicity.py`.
|
||
|
||
- **P-010 — Every adaptation decision is evidence-backed and reversible.**
|
||
Every parameter change SHALL write a full before/after record to the
|
||
`adaptations` table with: the parameter name, prior value, new value,
|
||
OSS at time of change, market regime, and the adaptation rule that
|
||
triggered it. After each change, the system SHALL wait at least 3
|
||
evaluation cycles (≈3 trading days). If forward OSS (OSS now minus OSS
|
||
at time of change) is non-positive for 3 consecutive post-adaptation
|
||
evaluations, the adaptation SHALL auto-unwind — the parameter reverts
|
||
to its pre-adaptation value and the unwinding is logged as a separate
|
||
adaptation record with `result='unwound'`. The `win_rate_adaptations`
|
||
metric SHALL reflect the fraction of adaptations whose forward OSS
|
||
delta was positive. An adaptation loop that produces 5 consecutive
|
||
unwound adaptations SHALL escalate to the user — the optimizer has
|
||
lost its signal. Property-tested via `test_adaptation_persistence.py`.
|
||
|
||
- **P-011 — The declared system objective is drawdown-first, not growth-
|
||
first.** The primary outcome the contract promises is: "match the S&P 500
|
||
benchmark net of trading costs, with peak-to-trough drawdown bounded to
|
||
≤25%." Risk-adjusted outperformance above the benchmark is a secondary
|
||
objective, pursued only when market-regime detection confirms the system
|
||
has an edge. Hedge capacity SHALL be regime-driven (Gate 2.1 buckets:
|
||
moderate/heavy/extreme) and SHALL be sized to actually bound drawdown —
|
||
`HEDGE_POSITION_SIZE_PCT` scales with bearish_score, and `PARKER_HEDGE_CAP_PCT`
|
||
sets the ceiling. Cash (zero-beta) is a valid optimization result when
|
||
no edge is detected. The contract enforces improvement trajectory, not
|
||
absolute return thresholds — a stable OSS of 0.7 with low drawdown is
|
||
preferable to a volatile OSS oscillating between 0.3 and 1.5. StockBench/
|
||
FINSABER evidence (arXiv:2505.19081): small accounts ($1K–$10K) are cost-
|
||
constrained; benchmark-matching net of costs is the honest achievable
|
||
outcome, not "maximize growth."
|
||
|
||
- The prime directive SHALL NOT be weakened by any contract update. Any change
|
||
that would allow valid trades to be silently blocked is invalid.
|
||
- Internal failures SHALL be distinguished from external failures. "Alpaca API
|
||
returned 500" is external; "gate evaluated stale position count" is internal.
|
||
- Every remediation action produces a structured audit trail entry with
|
||
classification, action taken, and verification result.
|
||
- Strategy parameters and cron schedules MAY be dynamically adjusted by
|
||
return-optimization rules (13-14), within bounded ranges, only when OSS or
|
||
regime data justifies the change. See `#### parameter-state` for current
|
||
values and justification trail. The stability surface (breakers, exit-gates,
|
||
per-position caps) SHALL NOT be modified.
|
||
- During market-closed hours: position exits, breaker resets, and reconciliation
|
||
continue to execute (these are lifecycle hooks, not trade execution).
|
||
- The contract itself is a living document. Updates that tighten detection or
|
||
expand remediation are encouraged. Updates that weaken the detection surface
|
||
or narrow the definition of "blocked" compromise the prime directive.
|
||
- **Return-optimization guardrails:**
|
||
- All adaptation decisions SHALL be reversible — no parameter change persists
|
||
beyond the regime that triggered it without fresh corroboration
|
||
- No adaptation may increase position size beyond MAX_CONCURRENT_POSITIONS ×
|
||
(equity ÷ MAX_CONCURRENT_POSITIONS) — the per-position cap is invariant
|
||
- Return maximization targets *capturable* returns only — the contract does
|
||
not promise returns the market does not offer. Regime detection may indicate
|
||
"no edge available," and cash is a valid optimization result
|
||
- The OSS target (1.0) is aspirational — the contract enforces *improvement
|
||
trajectory*, not absolute thresholds. A stable OSS of 0.7 is better than
|
||
a volatile OSS oscillating between 0.3 and 1.2
|
||
- Adaptations that fail to improve OSS over 3 consecutive evaluations (≈3
|
||
trading days) are automatically unwound — the system returns to its
|
||
pre-adaptation state
|
||
- Circuit breakers, position-count gates, and exit rules (the stability
|
||
surface) are NEVER modified by optimization rules. Optimization operates
|
||
within the stability surface, not on it
|
||
- **Data source version integrity:** Every external API endpoint used by a
|
||
track SHALL be documented with its current version in the track's source
|
||
header. When a track produces 0 signals for 3+ consecutive runs, the
|
||
health monitor SHALL verify the endpoint version against the documented
|
||
version before classifying as `no_data` (valid). A version mismatch
|
||
(deprecated endpoint, API EOL) is internal, not external — the contract
|
||
is responsible for keeping its data sources current.
|
||
- **Execution mode integrity:** Every cron job that runs a pure Python script
|
||
(track signal generation, DB maintenance, data collection — where the script
|
||
itself produces the complete output without LLM processing) SHALL be
|
||
configured with `no_agent=true`. The health monitor SHALL verify this
|
||
configuration for all track and pipeline jobs. An agent-mode configuration
|
||
on a script-only job is an internal misconfiguration — the LLM may perform
|
||
unrelated actions (web search, reasoning) instead of executing the script,
|
||
silently degrading signal freshness.
|
||
- **Orchestrator status coverage:** The Ross orchestrator SHALL handle every
|
||
known execution status code produced by `execute_trade()`. Status codes
|
||
that indicate permanent failure (`error_rejected`, `skipped_no_position`,
|
||
`skipped_non_fractionable`) SHALL be marked as permanently consumed
|
||
(action=skip), not retried. Status codes that indicate transient failure
|
||
(`error_poll_timeout`, `error_liquidity`) MAY be retried up to the
|
||
orchestrator's retry limit. The set of permanently-skipped statuses SHALL
|
||
be updated whenever a new adapter returns a novel failure code.
|
||
|
||
### Reporting
|
||
|
||
Every remediation cycle produces a structured report:
|
||
|
||
#### 1. Cron Output — Health Summary (every cycle)
|
||
```
|
||
🛡️ Alpaca System Health — 2026-06-28 15:00 UTC
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
Status: HEALTHY | 0 blocks | 0 stale | DNS OK
|
||
Regime: BULL (VIX 15.2) | OSS: 1.23 ▲ (Calmar 1.4)
|
||
Equity: $697.35 | Cash: $542.50 | Open Orders: 0
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
```
|
||
With issues:
|
||
```
|
||
🛡️ Alpaca System Health — 2026-06-28 15:00 UTC
|
||
Status: ADAPTING | 1 block | Maya stale (27h) | Regime: BEAR
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
|
||
📉 OSS: 0.42 ▼ (Calmar 0.3, PF 0.8, Beta 0.78)
|
||
└─ Warning: Below floor (0.5) — adaptation required
|
||
└─ Action: Rule 8 — Beta 0.78 in bear regime → rotate defensively
|
||
|
||
⚠️ BLOCK: [details as before]
|
||
⚠️ STALE: [details as before]
|
||
📊 ADAPTATION: [details as before]
|
||
```
|
||
|
||
#### 2. Cron Output — P&L Dashboard (10:00 AM and 6:00 PM daily, or on significant change)
|
||
```
|
||
📊 Alpaca P&L Dashboard — 2026-06-28 (30-day trailing)
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
|
||
💰 Total Equity: $697.35 | Cash: $542.50 | Open: $154.85 (3 pos)
|
||
|
||
📈 PERFORMANCE
|
||
Realized P&L (30d): +$42.18 (+6.4%)
|
||
Unrealized P&L: +$8.72
|
||
Benchmark (SPY 30d): +2.1%
|
||
Alpha: +4.3% ← our edge
|
||
|
||
📉 RISK
|
||
Max Drawdown: 4.2% ($29.14)
|
||
Sharpe Ratio: 1.47
|
||
Profit Factor: 1.82 (gross profit ÷ gross loss)
|
||
Win Rate: 58% (11W / 8L)
|
||
|
||
🏆 TOP/BOTTOM
|
||
Largest Winner: NVDA +$18.42 (Eddie, held 4d)
|
||
Largest Loser: AMD -$5.21 (Janet, held 2d)
|
||
|
||
📋 BY TRACK
|
||
Track Trades Win% P&L Avg Hold
|
||
───────── ────── ──── ───── ────────
|
||
Eddie 4 75% +$22.14 4.2d
|
||
Maggie 3 67% +$14.31 5.1d
|
||
Frank 4 50% +$5.73 3.8d
|
||
Maya 2 50% -$0.80 7.0d
|
||
Janet 4 50% -$0.20 1.8d
|
||
Parker 2 100% +$1.00 2.0d
|
||
───────── ────── ──── ───── ────────
|
||
TOTAL 19 58% +$42.18 3.8d
|
||
|
||
📋 BY VENUE
|
||
Alpaca Equity: 17 trades | +$35.94
|
||
Robinhood Crypto: 2 trades | +$6.24
|
||
|
||
📅 DAILY P&L (7 days)
|
||
Mon +$12.40 ████████
|
||
Tue -$3.20 ██
|
||
Wed +$18.72 ████████████
|
||
Thu +$5.41 ███
|
||
Fri +$8.85 ██████
|
||
Sat — (market closed)
|
||
Sun — (market closed)
|
||
|
||
🔓 OPEN POSITIONS
|
||
ENVX 5.0 @ $11.23 → $11.82 +$2.95 (5d)
|
||
CLF 10.0 @ $19.10 → $20.05 +$9.50 (3d)
|
||
ETH 0.01 @ $2,452 → $2,389 -$4.73 (2d)
|
||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||
```
|
||
|
||
#### 3. RA-H OS Knowledge Graph
|
||
Every remediation cycle creates a `[LEARN] alpaca-system-health: {timestamp}`
|
||
node with:
|
||
- metadata: { type: "system-health", status: "healthy" | "degraded" | "blocked" }
|
||
- source: Full structured JSON report
|
||
- description: Summary of blocks found, fixed, escalated, and P&L highlights
|
||
|
||
A separate `[REPORT] alpaca-pnl-dashboard: {date}` node is created for each
|
||
daily P&L dashboard with the full performance-summary data for long-term
|
||
trending and cross-session comparison.
|
||
|
||
#### 4. Periodic Digest (via existing Kojo auditor)
|
||
Kojo's existing 07:00 and 20:00 reports serve as the operational digest.
|
||
The system-health contract supplements Kojo: Kojo audits and reports,
|
||
the system-health contract detects, adapts, and self-heals.
|
||
|
||
### Execution
|
||
|
||
1. **Gather truth** — Run `auditor_query.py` for live system data. Query
|
||
signals.db for latest consensus decisions and execution results.
|
||
2. **Classify state** — Compare against thresholds. Classify every anomaly
|
||
as internal or external using the fault-domain rules.
|
||
3. **Detect market regime** — Fetch VIX, SPX 20-SMA, Nasdaq 20-SMA, and
|
||
BTC trend via yfinance (reuse Parker's market data pipeline). Classify
|
||
regime as bull/neutral/bear/crisis.
|
||
4. **Compute OSS** — Query trade_journal.db for closed trades in the last
|
||
30 days. Compute Calmar Ratio, Profit Factor, Rolling Alpha over SPY,
|
||
Portfolio Beta, and Win Rate of adaptations. Blend into OSS score.
|
||
5. **Check adaptation triggers** — Compare OSS vs floor (0.5), portfolio_beta
|
||
vs regime, and sector exposure vs regime. Fire adaptation rules (8-12)
|
||
if triggered.
|
||
6. **Apply operational remediation** — For each internal block not addressed
|
||
by adaptation rules, apply matching remediation rule (1-17). External blocks
|
||
are documented, not fixed.
|
||
7. **Re-verify** — After all remediations and adaptations, re-run
|
||
auditor_query.py and re-compute OSS. Only report what persists.
|
||
8. **Generate report** — Produce structured output (cron format + RA-H node).
|
||
9. **Record adaptation outcomes** — For each adaptation fired, log a
|
||
before/after record in trade_journal.db for future OSS computation.
|
||
10. **Wait 5 min** → repeat from step 1.
|
||
|
||
### Audit Trail Format
|
||
|
||
```json
|
||
{
|
||
"run_id": "system-health-20260628-150000",
|
||
"timestamp": "2026-06-28T15:00:00Z",
|
||
"duration_ms": 8234,
|
||
"system_state": {
|
||
"equity": 697.35,
|
||
"cash": 542.50,
|
||
"open_orders": 0,
|
||
"market_open": false,
|
||
"tracks": {
|
||
"eddie": {"fresh": true, "hours_since": 46},
|
||
"maya": {"fresh": false, "hours_since": 27},
|
||
"janet": {"fresh": true, "hours_since": 0},
|
||
"parker": {"fresh": true, "hours_since": 1}
|
||
},
|
||
"dns": {"fc.yahoo.com": true}
|
||
},
|
||
"market_regime": {
|
||
"regime": "bear",
|
||
"vix_close": 27.4,
|
||
"spx_vs_sma": -2.3,
|
||
"ndx_vs_sma": -3.8
|
||
},
|
||
"return_optimization": {
|
||
"oss": 0.42,
|
||
"oss_trend": "degrading",
|
||
"calmar": 0.31,
|
||
"profit_factor": 0.82,
|
||
"rolling_alpha": -8.5,
|
||
"portfolio_beta": 0.78,
|
||
"win_rate_adaptations": 0.40
|
||
},
|
||
"blocks_found": 2,
|
||
"blocks_internal": 1,
|
||
"blocks_external": 1,
|
||
"blocks_fixed": 1,
|
||
"blocks_escalated": 0,
|
||
"adaptations_fired": 2,
|
||
"adaptations": [
|
||
{
|
||
"rule": "Rule 8: Market-Regime Position Mismatch",
|
||
"target": "portfolio_beta",
|
||
"action": "Rotated high-beta to cash",
|
||
"before": {"beta": 0.78, "regime": "bear"},
|
||
"after": {"beta": 0.35},
|
||
"result": "adapting"
|
||
},
|
||
{
|
||
"rule": "Rule 9: Regime-Inappropriate Sector",
|
||
"target": "NVDA, AMD",
|
||
"action": "Exited high-beta losers; rotated to cash",
|
||
"before": {"exposure_pct": 22},
|
||
"after": {"exposure_pct": 8},
|
||
"result": "adapting"
|
||
}
|
||
],
|
||
"remediations": [
|
||
{
|
||
"rule": "Rule 1: Stale Circuit Breaker",
|
||
"target": "equity-position-limit",
|
||
"action": "deactivated active breaker",
|
||
"result": "fixed",
|
||
"duration_ms": 120
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
### Errors
|
||
|
||
- **Alpaca API unreachable** — classified as external. Log, do not fix.
|
||
- **Robinhood Crypto API unreachable** — classified as external. Log, do not fix.
|
||
- **signals.db corrupted** — classified as internal. Attempt SQLite integrity
|
||
check. If unrecoverable: escalate immediately (data loss).
|
||
- **trade_journal.db corrupted** — classified as internal. Attempt SQLite
|
||
integrity check. If unrecoverable: escalate immediately.
|
||
- **auditor_query.py fails** — classified as internal. Check Python venv path,
|
||
verify dependencies. If persistent: escalate.
|
||
- **Cron job trigger fails** (cronjob action='run' returns error) — classified
|
||
as internal. Retry once. If persistent: escalate.
|
||
- **Etherscan API key expired or endpoint deprecated** — classified as internal.
|
||
The contract is responsible for keeping its data sources current. Verify the
|
||
API key against Etherscan's account portal. Verify the endpoint version against
|
||
the documented version in track_4_maya.py. If deprecated: migrate to the
|
||
current endpoint. If key expired: escalate (requires human credential rotation).
|
||
- **yfinance returns empty data (not error)** — classified as internal if the
|
||
system marks it as `no_data` without verifying the data source is reachable.
|
||
Before classifying as `no_data`: (a) verify yfinance can fetch a known
|
||
benchmark ticker (SPY), (b) check for rate-limiting response headers,
|
||
(c) compare against Rule 6 DNS check. If yfinance is reachable but specific
|
||
tickers return empty: classify as `no_data` (valid). If yfinance is
|
||
unreachable or rate-limited: classify as external (Rule 6).
|
||
- **ChromaDB/LLM dependency unreachable** — classified as external if the
|
||
dependency is hosted remotely. If running locally and the process is down:
|
||
classified as internal — restart the process. If persistently down: escalate.
|
||
- **Cron job running in wrong execution mode** — classified as internal. If a
|
||
track's cron job is configured with agent-mode (`no_agent=false` or unset)
|
||
when it should run as a script (`no_agent=true`), the health monitor SHALL
|
||
flag this as an internal misconfiguration. Fix: update the cron job via
|
||
`cronjob(action='update', no_agent=true)`.
|
||
|
||
### Skills
|
||
|
||
- kojo-auditor
|
||
- pipeline-idempotency-audit
|
||
|
||
### Tools
|
||
|
||
- `cli:python3`: required for running auditor_query.py and track scripts
|
||
- `cli:sqlite3`: required for direct database queries when auditor is unavailable
|
||
- `cli:curl`: required for DNS reachability tests and API health checks
|