alpaca: add P-012 — internal metric integrity self-validates, self-heals, fail-closed

Detects snapshot poisoning by positive re-derivation (pre-inception, raw-balance
leak, adaptive sanity bound); quarantines (never deletes); tiered fail-closed
health (HEALTHY commits, DEGRADED report-only, CRITICAL holds) with age-adaptive
MIN_CLEAN; self-heals to HEALTHY via natural clean-snapshot refill. Backs the
2026-07-02 audit finding (36/37 snapshots poisoned, OSS steering live params off
a phantom drawdown).

Impl: scripts/metric_integrity.py, scripts/migrate_p010_metric_integrity.py,
compute_oss.py clean-filter+tier, apply_adaptations.py tier gate.
Tests: tests/test_p010_metric_integrity.py (10).
This commit is contained in:
root
2026-07-24 16:46:15 -04:00
committed by root
parent ca4f18e539
commit 661c95e39d
@@ -864,6 +864,73 @@ with per-source coverage breadth trend.
constrained; benchmark-matching net of costs is the honest achievable
outcome, not "maximize growth."
- **P-012 — Internal performance metrics are self-validating, self-healing,
and fail-closed against poisoning.** The `performance_snapshots` table feeds
the Optimization Success Score (P-009), which drives live parameter
adaptation (P-010). A poisoned snapshot therefore does not merely produce a
wrong number — it steers the system's parameter-adaptation loop AWAY from the
optimal state. The contract's North Star is that the system regresses to its
mean HEALTHY state on its own and stays on mission (capturing risk-adjusted
returns) without user intervention; this invariant is the mechanism that
guarantees it.
**Detection by positive re-derivation (not blocklist).** Every snapshot SHALL
be validated by re-deriving it against ground truth AT READ TIME, never
trusted merely because it was written or because it lacks a known-bad shape.
A snapshot is INVALID if any hold: (a) it is dated before the first realized
trade (`realized_trades.closed_at`, `unmatched=0`, `venue != 'alpaca_paper'`)
— such rows are pre-inception non-data and SHALL NEVER be reconstructed by
fabricating equity (zero-assumptions doctrine); (b) `start_equity` does not
equal the trade-attributed equity-curve inception anchor (the `sys_start`
deposit from the P-003 deposit schedule) within $1 — this catches raw-balance
leaks (the $100,383 and flat-$200 poisonings); (c) `|total_return_pct|`
exceeds the adaptive sanity bound, the looser of 40% absolute or 5× the
rolling standard deviation of clean returns — this catches the +249% deposit-
inflation and 99.8% raw-balance poisonings without false-positiving a
legitimately volatile week. Because validation is a pure function of the row
plus the current capital schedule, the SAME validator detects both current
and historical poisoning and self-corrects as the P-003 formula improves.
**Containment by quarantine, never deletion.** Invalid rows SHALL be flagged
(`quarantined=1`, `quarantine_reason`), preserved as an audit trail of the
bug, and EXCLUDED from every OSS input window (`compute_oss_from_snapshot`,
beta, trend). OSS SHALL be computed only from rows that pass validation this
cycle.
**Tiered, fail-closed health — defined relative to what is achievable.** The
adaptation controller SHALL read a health tier derived from clean-snapshot
availability, where the clean-snapshot floor `MIN_CLEAN` ramps with system
age (`clamp(age_days × 0.5, 3, 8)`) so a young or freshly-healed system is
never starved into permanent inaction:
- **HEALTHY** (clean ≥ `MIN_CLEAN`): full loop — propose AND commit to
`.env.prod`.
- **DEGRADED** (1 ≤ clean < `MIN_CLEAN`): evaluate + auto-unwind run
(bookkeeping / immune system), the proposal is computed and REPORTED, but
SHALL NOT be committed to `.env.prod`.
- **CRITICAL** (clean = 0): hold all adaptation; only bookkeeping runs; emit
a loud alert.
**Self-healing by natural refill.** Because poisoned history is quarantined
rather than resurrected, the OSS window refills with genuine post-inception
snapshots as they accrue (~1/day). A system that starts DEGRADED regresses to
HEALTHY unaided as clean rows accumulate — no user intervention required to
unblock adaptation. The evaluate/auto-unwind half of P-010 ALWAYS runs
regardless of tier, because reverting a proven-bad change is pure risk
reduction. Only the new-risk action (committing a new adaptation) is gated.
**Rationale:** On 2026-07-02, an audit found that 36 of 37 snapshots were
poisoned — 29 with a raw ~$100K paper-balance leak producing 99.8% returns,
and pre-inception rows dated before the first trade — while OSS was silently
computed from a mix of real and fabricated rows, driving live
`POSITION_SIZE_PCT` tightening against a phantom 2.35% drawdown. Quarantine
restored a truthful (if thin) window; the tier gate held the controller
report-only until real clean data accrues. Property-tested via
`tests/test_p010_metric_integrity.py` (detector correctness, tier gating,
regress-to-HEALTHY, quarantine-never-deletes). Detector:
`scripts/metric_integrity.py`; migration:
`scripts/migrate_p010_metric_integrity.py`; controller gate:
`scripts/apply_adaptations.py:main`.
- The prime directive SHALL NOT be weakened by any contract update. Any change
that would allow valid trades to be silently blocked is invalid.
- Internal failures SHALL be distinguished from external failures. "Alpaca API