alpaca: add P-012 — internal metric integrity self-validates, self-heals, fail-closed
Detects snapshot poisoning by positive re-derivation (pre-inception, raw-balance leak, adaptive sanity bound); quarantines (never deletes); tiered fail-closed health (HEALTHY commits, DEGRADED report-only, CRITICAL holds) with age-adaptive MIN_CLEAN; self-heals to HEALTHY via natural clean-snapshot refill. Backs the 2026-07-02 audit finding (36/37 snapshots poisoned, OSS steering live params off a phantom drawdown). Impl: scripts/metric_integrity.py, scripts/migrate_p010_metric_integrity.py, compute_oss.py clean-filter+tier, apply_adaptations.py tier gate. Tests: tests/test_p010_metric_integrity.py (10).
This commit is contained in:
@@ -864,6 +864,73 @@ with per-source coverage breadth trend.
|
||||
constrained; benchmark-matching net of costs is the honest achievable
|
||||
outcome, not "maximize growth."
|
||||
|
||||
- **P-012 — Internal performance metrics are self-validating, self-healing,
|
||||
and fail-closed against poisoning.** The `performance_snapshots` table feeds
|
||||
the Optimization Success Score (P-009), which drives live parameter
|
||||
adaptation (P-010). A poisoned snapshot therefore does not merely produce a
|
||||
wrong number — it steers the system's parameter-adaptation loop AWAY from the
|
||||
optimal state. The contract's North Star is that the system regresses to its
|
||||
mean HEALTHY state on its own and stays on mission (capturing risk-adjusted
|
||||
returns) without user intervention; this invariant is the mechanism that
|
||||
guarantees it.
|
||||
|
||||
**Detection by positive re-derivation (not blocklist).** Every snapshot SHALL
|
||||
be validated by re-deriving it against ground truth AT READ TIME, never
|
||||
trusted merely because it was written or because it lacks a known-bad shape.
|
||||
A snapshot is INVALID if any hold: (a) it is dated before the first realized
|
||||
trade (`realized_trades.closed_at`, `unmatched=0`, `venue != 'alpaca_paper'`)
|
||||
— such rows are pre-inception non-data and SHALL NEVER be reconstructed by
|
||||
fabricating equity (zero-assumptions doctrine); (b) `start_equity` does not
|
||||
equal the trade-attributed equity-curve inception anchor (the `sys_start`
|
||||
deposit from the P-003 deposit schedule) within $1 — this catches raw-balance
|
||||
leaks (the $100,383 and flat-$200 poisonings); (c) `|total_return_pct|`
|
||||
exceeds the adaptive sanity bound, the looser of 40% absolute or 5× the
|
||||
rolling standard deviation of clean returns — this catches the +249% deposit-
|
||||
inflation and −99.8% raw-balance poisonings without false-positiving a
|
||||
legitimately volatile week. Because validation is a pure function of the row
|
||||
plus the current capital schedule, the SAME validator detects both current
|
||||
and historical poisoning and self-corrects as the P-003 formula improves.
|
||||
|
||||
**Containment by quarantine, never deletion.** Invalid rows SHALL be flagged
|
||||
(`quarantined=1`, `quarantine_reason`), preserved as an audit trail of the
|
||||
bug, and EXCLUDED from every OSS input window (`compute_oss_from_snapshot`,
|
||||
beta, trend). OSS SHALL be computed only from rows that pass validation this
|
||||
cycle.
|
||||
|
||||
**Tiered, fail-closed health — defined relative to what is achievable.** The
|
||||
adaptation controller SHALL read a health tier derived from clean-snapshot
|
||||
availability, where the clean-snapshot floor `MIN_CLEAN` ramps with system
|
||||
age (`clamp(age_days × 0.5, 3, 8)`) so a young or freshly-healed system is
|
||||
never starved into permanent inaction:
|
||||
- **HEALTHY** (clean ≥ `MIN_CLEAN`): full loop — propose AND commit to
|
||||
`.env.prod`.
|
||||
- **DEGRADED** (1 ≤ clean < `MIN_CLEAN`): evaluate + auto-unwind run
|
||||
(bookkeeping / immune system), the proposal is computed and REPORTED, but
|
||||
SHALL NOT be committed to `.env.prod`.
|
||||
- **CRITICAL** (clean = 0): hold all adaptation; only bookkeeping runs; emit
|
||||
a loud alert.
|
||||
|
||||
**Self-healing by natural refill.** Because poisoned history is quarantined
|
||||
rather than resurrected, the OSS window refills with genuine post-inception
|
||||
snapshots as they accrue (~1/day). A system that starts DEGRADED regresses to
|
||||
HEALTHY unaided as clean rows accumulate — no user intervention required to
|
||||
unblock adaptation. The evaluate/auto-unwind half of P-010 ALWAYS runs
|
||||
regardless of tier, because reverting a proven-bad change is pure risk
|
||||
reduction. Only the new-risk action (committing a new adaptation) is gated.
|
||||
|
||||
**Rationale:** On 2026-07-02, an audit found that 36 of 37 snapshots were
|
||||
poisoned — 29 with a raw ~$100K paper-balance leak producing −99.8% returns,
|
||||
and pre-inception rows dated before the first trade — while OSS was silently
|
||||
computed from a mix of real and fabricated rows, driving live
|
||||
`POSITION_SIZE_PCT` tightening against a phantom −2.35% drawdown. Quarantine
|
||||
restored a truthful (if thin) window; the tier gate held the controller
|
||||
report-only until real clean data accrues. Property-tested via
|
||||
`tests/test_p010_metric_integrity.py` (detector correctness, tier gating,
|
||||
regress-to-HEALTHY, quarantine-never-deletes). Detector:
|
||||
`scripts/metric_integrity.py`; migration:
|
||||
`scripts/migrate_p010_metric_integrity.py`; controller gate:
|
||||
`scripts/apply_adaptations.py:main`.
|
||||
|
||||
- The prime directive SHALL NOT be weakened by any contract update. Any change
|
||||
that would allow valid trades to be silently blocked is invalid.
|
||||
- Internal failures SHALL be distinguished from external failures. "Alpaca API
|
||||
|
||||
Reference in New Issue
Block a user