fix(revision-preflight): default to warn, name the refusal class, bound the fetch
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 10s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 12s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Failing after 6s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 5s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Skipped

Bounded correction round on PR #134 after a PASS-WITH-FINDINGS review whose
Finding 4 is High. The guard's purpose and its fail-closed fix stand; the
problem was that with 'enforce' as the default it gates EVERY scheduled
contract, and three legitimate states produce a refusal - a clone legitimately
ahead of origin/master mid-review, a detached HEAD, and an offline or failed
fetch - so any of them would turn the fleet's monitoring into withheld
verdicts. That risk outweighs the staleness the guard catches.

1. DEFAULT IS NOW 'warn'. 'enforce' remains available and documented. The
   criteria for flipping the default later are written into the doc as a
   decision with evidence - a sustained window (30 days / 200+ runs) with zero
   mismatch:* and zero cannot-verify:* refusals, no fetch blips, and a pinned
   clone demonstrably kept current - explicitly as its own change, not a silent
   flip.

2. 'COULD NOT CHECK' IS NOW DISTINGUISHABLE FROM 'THIS COPY IS WRONG'. Every
   non-zero exit prints a machine-readable REASON=<class> line:
     cannot-verify:fetch-failed | cannot-verify:ref-unresolvable   (exit 2)
     mismatch:path-absent | mismatch:content
     mismatch:detached-head | mismatch:clone-ahead                (exit 1)
   detached-head and clone-ahead are named separately because they are
   legitimate states, far less alarming than a hand-edited file. clone-ahead
   requires HEAD to be STRICTLY ahead; an uncommitted edit on a commit that IS
   the ref is a plain content mismatch (my own first cut got this wrong and the
   new test 7d caught it).

3. THE DEFAULT FETCH IS BOUNDED: --fetch-timeout, default 20s, 0 = unbounded,
   and a missing 'timeout' binary is itself a cannot-verify rather than an
   unbounded fetch inside a scheduled contract.

4. TEST COVERAGE ADDED for every new class: fetch failure, fetch timeout
   (asserted to return promptly under a 1s bound), unresolvable ref, detached
   HEAD, clone-ahead, genuine content mismatch, and the contract-run.sh default.
   The pre-fix draft fixture comparisons are kept: 31 passed, 0 failed.

5. MERGE-TIME SEQUENCE documented: fast-forward /opt/contract-runner, confirm
   clean, prove a contract runs and reports. Baseline recorded as of today -
   firstmate has already fast-forwarded it to 9faffe4 - with the note that an
   untracked file blocks a fast-forward even when byte-identical.

Live behaviour re-verified on the real runner path:
  default: REASON=mismatch:clone-ahead -> 'continuing because ...=warn' -> VERDICT: PASS, exit 0
  enforce: REASON=mismatch:clone-ahead -> 'VERDICT WITHHELD: mismatch:clone-ahead', exit 2

MANDATORY CHECKS (master went red once from a credential-SHAPED string, so
these are now run on every shippable branch):
  bash scripts/prose-lint.sh        -> LINT PASSED (18 warning(s))
  secret scan                       -> secret scan clean (tree; 34 allowlisted,
                                       24 inert value(s) ignored); No committed credentials
  shellcheck revision-preflight.sh  -> clean
  shellcheck test_revision_preflight.sh -> clean
  shellcheck contract-run.sh        -> SC2034 x1, SC2086 x2 - byte-identical on
                                       master, i.e. pre-existing, none introduced

tests/test_probe_drift.py::test_prose_lint_accepts_report_format_with_provenance
fails both before and after this branch (it runs prose-lint from a temp CWD and
cannot find its sibling secret-scan.sh). Pre-existing, unrelated, not fixed here.
This commit is contained in:
root
2026-09-25 11:29:15 +00:00
parent 574cb99d76
commit c5a6dcd42a
4 changed files with 341 additions and 67 deletions
+104 -4
View File
@@ -100,21 +100,65 @@ else
fi
rm -rf "$(dirname "$C")"
# ── 5. unresolvable ref → must fail ──────────────────────────────────────────
reason_of() { grep -m1 '^REASON=' <<<"$1" | cut -d= -f2-; }
# ── 5. unresolvable ref → CANNOT VERIFY (exit 2) ─────────────────────────────
echo "5. unresolvable ref"
C=$(make_clone)
out=$("$GUARD" --no-fetch --ref origin/nope "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
if [[ $rc -eq 1 ]]; then pass "unresolvable ref exits 1"; else fail "unresolvable ref must exit 1 (got $rc)"; fi
if [[ $rc -eq 2 ]]; then pass "unresolvable ref exits 2 (cannot verify)"; else fail "unresolvable ref must exit 2 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "cannot-verify:ref-unresolvable" ]]; then
pass "unresolvable ref names cannot-verify:ref-unresolvable"
else
fail "unresolvable ref should name its class (got: $(reason_of "$out"))"
fi
rm -rf "$(dirname "$C")"
# ── 6. missing script → must fail ────────────────────────────────────────────
# ── 5b. fetch failure → CANNOT VERIFY, and it names that ─────────────────────
echo "5b. fetch failed"
C=$(make_clone)
( cd "$C" && git remote set-url origin /nonexistent/definitely-not-a-repo )
out=$("$GUARD" "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
if [[ $rc -eq 2 ]]; then pass "fetch failure exits 2 (cannot verify)"; else fail "fetch failure must exit 2 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "cannot-verify:fetch-failed" ]]; then
pass "fetch failure names cannot-verify:fetch-failed"
else
fail "fetch failure should name its class (got: $(reason_of "$out"))"
fi
if [[ "$out" == *"bound:"* ]]; then pass "fetch failure reports the bound"; else fail "fetch failure should report the bound"; fi
rm -rf "$(dirname "$C")"
# ── 5c. fetch timeout → CANNOT VERIFY, bounded (never hangs) ─────────────────
echo "5c. fetch timeout is bounded"
C=$(make_clone)
# a remote that will never answer: a fifo-backed git daemon is overkill, so use
# a black-hole address with a 1s bound and assert we return promptly.
( cd "$C" && git remote set-url origin http://10.255.255.1:9/never.git )
start=$(date +%s)
out=$("$GUARD" --fetch-timeout 1 "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
elapsed=$(( $(date +%s) - start ))
if [[ $rc -eq 2 ]]; then pass "timeout exits 2 (cannot verify)"; else fail "timeout must exit 2 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "cannot-verify:fetch-failed" ]]; then
pass "timeout names cannot-verify:fetch-failed"
else
fail "timeout should name its class (got: $(reason_of "$out"))"
fi
if [[ $elapsed -le 10 ]]; then pass "timeout returned promptly (${elapsed}s, bound 1s)"; else fail "timeout did not bound the fetch (${elapsed}s)"; fi
rm -rf "$(dirname "$C")"
# ── 6. missing script → MISMATCH:path-absent (exit 1) ────────────────────────
echo "6. missing script"
C=$(make_clone)
out=$("$GUARD" "$C/scripts/nope.sh" "$C" 2>&1); rc=$?
if [[ $rc -eq 1 ]]; then pass "missing script exits 1"; else fail "missing script must exit 1 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "mismatch:path-absent" ]]; then
pass "missing script names mismatch:path-absent"
else
fail "missing script should name its class (got: $(reason_of "$out"))"
fi
rm -rf "$(dirname "$C")"
# ── 7. script outside the clone → must fail ──────────────────────────────────
# ── 7. script outside the clone → MISMATCH (exit 1) ──────────────────────────
echo "7. script outside the clone"
C=$(make_clone)
OUTSIDE=$(mktemp)
@@ -123,6 +167,48 @@ out=$("$GUARD" "$OUTSIDE" "$C" 2>&1); rc=$?
if [[ $rc -eq 1 ]]; then pass "outside script exits 1"; else fail "outside script must exit 1 (got $rc)"; fi
rm -f "$OUTSIDE"; rm -rf "$(dirname "$C")"
# ── 7b. detached HEAD is named, not reported as a raw content mismatch ───────
echo "7b. detached HEAD"
C=$(make_clone)
( cd "$C" && printf '#!/bin/bash\necho TAMPERED\n' > scripts/demo.sh \
&& git add scripts/demo.sh && git commit -qm tamper && git checkout -q --detach HEAD )
out=$("$GUARD" "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
if [[ $rc -eq 1 ]]; then pass "detached HEAD exits 1"; else fail "detached HEAD must exit 1 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "mismatch:detached-head" ]]; then
pass "detached HEAD names mismatch:detached-head"
else
fail "detached HEAD should name its class (got: $(reason_of "$out"))"
fi
rm -rf "$(dirname "$C")"
# ── 7c. a branch ahead of the ref is named as such, not as a raw mismatch ────
echo "7c. clone ahead of the ref"
C=$(make_clone)
( cd "$C" && git checkout -q -b feature \
&& printf '#!/bin/bash\necho FEATURE\n' > scripts/demo.sh \
&& git add scripts/demo.sh && git commit -qm feature )
out=$("$GUARD" "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
if [[ $rc -eq 1 ]]; then pass "clone ahead exits 1"; else fail "clone ahead must exit 1 (got $rc)"; fi
if [[ "$(reason_of "$out")" == "mismatch:clone-ahead" ]]; then
pass "clone ahead names mismatch:clone-ahead"
else
fail "clone ahead should name its class (got: $(reason_of "$out"))"
fi
if [[ "$out" == *"mid-review"* ]]; then pass "clone ahead explains it is a legitimate state"; else fail "clone ahead should explain the state"; fi
rm -rf "$(dirname "$C")"
# ── 7d. a genuine content mismatch is named as content ──────────────────────
echo "7d. genuine content mismatch"
C=$(make_clone)
printf '#!/bin/bash\necho TAMPERED\n' > "$C/scripts/demo.sh"
out=$("$GUARD" "$C/scripts/demo.sh" "$C" 2>&1); rc=$?
if [[ "$(reason_of "$out")" == "mismatch:content" ]]; then
pass "hand-edit names mismatch:content"
else
fail "hand-edit should name mismatch:content (got: $(reason_of "$out"))"
fi
rm -rf "$(dirname "$C")"
# ── 8. --no-fetch states the freshness assumption ────────────────────────────
echo "8. --no-fetch states its assumption"
C=$(make_clone)
@@ -134,6 +220,20 @@ else
fi
rm -rf "$(dirname "$C")"
# ── 8b. contract-run.sh defaults to warn, not enforce ───────────────────────
echo "8b. contract-run.sh default mode"
DEFAULT=$(grep -m1 'CONTRACT_REVISION_PREFLIGHT:-' "$REPO/scripts/contract-run.sh" | sed 's/.*:-//; s/}.*//')
if [[ "$DEFAULT" == "warn" ]]; then
pass "contract-run.sh defaults to warn"
else
fail "contract-run.sh default must be warn (found: '$DEFAULT')"
fi
if grep -q 'CONTRACT_REVISION_PREFLIGHT=enforce' "$REPO/scripts/contract-run.sh"; then
pass "enforce remains available and documented"
else
fail "enforce must remain documented"
fi
# ── 9. the pre-fix guard must FAIL these same cases (proves the tests bite) ──
echo "9. pre-fix draft fails the same cases (bite proof)"
if [[ ! -f "$PREFIX_GUARD" ]]; then