alpaca: add P-012 — internal metric integrity self-validates, self-heals, fail-closed

Detects snapshot poisoning by positive re-derivation (pre-inception, raw-balance
leak, adaptive sanity bound); quarantines (never deletes); tiered fail-closed
health (HEALTHY commits, DEGRADED report-only, CRITICAL holds) with age-adaptive
MIN_CLEAN; self-heals to HEALTHY via natural clean-snapshot refill. Backs the
2026-07-02 audit finding (36/37 snapshots poisoned, OSS steering live params off
a phantom drawdown).

Impl: scripts/metric_integrity.py, scripts/migrate_p010_metric_integrity.py,
compute_oss.py clean-filter+tier, apply_adaptations.py tier gate.
Tests: tests/test_p010_metric_integrity.py (10).
This commit is contained in:
root
2026-07-24 16:46:15 -04:00
committed by root
parent ca4f18e539
commit 661c95e39d
@@ -864,6 +864,73 @@ with per-source coverage breadth trend.
constrained; benchmark-matching net of costs is the honest achievable constrained; benchmark-matching net of costs is the honest achievable
outcome, not "maximize growth." outcome, not "maximize growth."
- **P-012 — Internal performance metrics are self-validating, self-healing,
and fail-closed against poisoning.** The `performance_snapshots` table feeds
the Optimization Success Score (P-009), which drives live parameter
adaptation (P-010). A poisoned snapshot therefore does not merely produce a
wrong number — it steers the system's parameter-adaptation loop AWAY from the
optimal state. The contract's North Star is that the system regresses to its
mean HEALTHY state on its own and stays on mission (capturing risk-adjusted
returns) without user intervention; this invariant is the mechanism that
guarantees it.
**Detection by positive re-derivation (not blocklist).** Every snapshot SHALL
be validated by re-deriving it against ground truth AT READ TIME, never
trusted merely because it was written or because it lacks a known-bad shape.
A snapshot is INVALID if any hold: (a) it is dated before the first realized
trade (`realized_trades.closed_at`, `unmatched=0`, `venue != 'alpaca_paper'`)
— such rows are pre-inception non-data and SHALL NEVER be reconstructed by
fabricating equity (zero-assumptions doctrine); (b) `start_equity` does not
equal the trade-attributed equity-curve inception anchor (the `sys_start`
deposit from the P-003 deposit schedule) within $1 — this catches raw-balance
leaks (the $100,383 and flat-$200 poisonings); (c) `|total_return_pct|`
exceeds the adaptive sanity bound, the looser of 40% absolute or 5× the
rolling standard deviation of clean returns — this catches the +249% deposit-
inflation and 99.8% raw-balance poisonings without false-positiving a
legitimately volatile week. Because validation is a pure function of the row
plus the current capital schedule, the SAME validator detects both current
and historical poisoning and self-corrects as the P-003 formula improves.
**Containment by quarantine, never deletion.** Invalid rows SHALL be flagged
(`quarantined=1`, `quarantine_reason`), preserved as an audit trail of the
bug, and EXCLUDED from every OSS input window (`compute_oss_from_snapshot`,
beta, trend). OSS SHALL be computed only from rows that pass validation this
cycle.
**Tiered, fail-closed health — defined relative to what is achievable.** The
adaptation controller SHALL read a health tier derived from clean-snapshot
availability, where the clean-snapshot floor `MIN_CLEAN` ramps with system
age (`clamp(age_days × 0.5, 3, 8)`) so a young or freshly-healed system is
never starved into permanent inaction:
- **HEALTHY** (clean ≥ `MIN_CLEAN`): full loop — propose AND commit to
`.env.prod`.
- **DEGRADED** (1 ≤ clean < `MIN_CLEAN`): evaluate + auto-unwind run
(bookkeeping / immune system), the proposal is computed and REPORTED, but
SHALL NOT be committed to `.env.prod`.
- **CRITICAL** (clean = 0): hold all adaptation; only bookkeeping runs; emit
a loud alert.
**Self-healing by natural refill.** Because poisoned history is quarantined
rather than resurrected, the OSS window refills with genuine post-inception
snapshots as they accrue (~1/day). A system that starts DEGRADED regresses to
HEALTHY unaided as clean rows accumulate — no user intervention required to
unblock adaptation. The evaluate/auto-unwind half of P-010 ALWAYS runs
regardless of tier, because reverting a proven-bad change is pure risk
reduction. Only the new-risk action (committing a new adaptation) is gated.
**Rationale:** On 2026-07-02, an audit found that 36 of 37 snapshots were
poisoned — 29 with a raw ~$100K paper-balance leak producing 99.8% returns,
and pre-inception rows dated before the first trade — while OSS was silently
computed from a mix of real and fabricated rows, driving live
`POSITION_SIZE_PCT` tightening against a phantom 2.35% drawdown. Quarantine
restored a truthful (if thin) window; the tier gate held the controller
report-only until real clean data accrues. Property-tested via
`tests/test_p010_metric_integrity.py` (detector correctness, tier gating,
regress-to-HEALTHY, quarantine-never-deletes). Detector:
`scripts/metric_integrity.py`; migration:
`scripts/migrate_p010_metric_integrity.py`; controller gate:
`scripts/apply_adaptations.py:main`.
- The prime directive SHALL NOT be weakened by any contract update. Any change - The prime directive SHALL NOT be weakened by any contract update. Any change
that would allow valid trades to be silently blocked is invalid. that would allow valid trades to be silently blocked is invalid.
- Internal failures SHALL be distinguished from external failures. "Alpaca API - Internal failures SHALL be distinguished from external failures. "Alpaca API