Files
prose-contracts/scripts/prose-lint.sh
T
abiba-bot 8f1e5eebc4
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 5s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 18s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
feat(security): commit-time secret guard that FAILS the build on a committed credential
The 2026-09-17 purge removed six live credentials that had sat in this repo
for weeks, several in .md prose. Nothing blocked that class of commit, so a
warning in a stream nobody reads was the only signal. This adds a guard that
fails the build instead of warning.

Guard
- scripts/secret-scan.sh: bash + coreutils + grep/sed/awk + git only (the Gitea
  Actions runner executes job steps inside the runner container — BusyBox grep,
  no node/python). Modes: --tree (git-tracked, default), --path DIR (no git),
  --staged (pre-commit), --diff REF. Exit 1 on a finding, 2 on config error.
- scripts/secret-patterns.tsv: checked-in pattern list — sk-, sk-or-v1-,
  sk_live_, literal Bearer tokens, PVEAPIToken=, raw Authorization values, PEM
  private-key blocks, prose credential lines, and password/api_key/secret/token
  assignments carrying a literal value. Prose is scanned exactly like code.
- scripts/secret-allowlist.tsv: one entry per deliberate synthetic example, each
  with a reason. A missing reason is a hard error (fail closed). The 2026-09-17
  purge's `«vault: ...»` placeholders are listed explicitly rather than filtered
  by a general "vault"/"synthetic" rule, so a new occurrence still needs a
  reviewed, reasoned entry.
- A small inert-value classifier drops env refs, paths, dotted code access,
  variable names and right-truncated redactions; it does not know the words
  "synthetic"/"example", so a fabrication is always an explicit exception.
- Findings are printed with the credential masked; a scan never echoes a full
  secret into the log.

Wiring
- .gitea/workflows/pr-pipeline.yaml lint job: explicit "Committed-credential
  scan" step plus the self-test. A finding fails the required
  `pr-pipeline / lint` context, which the merge gate depends on.
- scripts/prose-lint.sh (the local gate): a "Secret scan" section, so
  `bash scripts/prose-lint.sh` before pushing is equivalent to CI.

Tests
- tests/test_secret_scan.sh: 20 cases. Plants pattern-matching fixtures in temp
  trees (outside every allowlisted path) and asserts the guard FAILS, including
  the --staged commit-time path; asserts the tree is quiet; asserts allowlisted
  text at an unlisted path still fails (path-explicit, not word-based); asserts
  a reasonless allowlist entry exits 2.

Verified: guard run against 8245716^ (the pre-fix revision, before the purge)
fails on the real OpenRouter/LiteLLM/Zulip/Proxmox/Stirling credentials; guard
run over the current tree is clean.
2026-09-22 15:04:37 +00:00

162 lines
6.1 KiB
Bash
Executable File

#!/bin/bash
# prose-lint.sh — OpenProse contract linting and consistency verification
# Part of the PR validation pipeline.
# Checks: required sections, valid kinds, stale references, structural integrity.
set -euo pipefail
FAILED=0
WARNINGS=0
echo "╔═══════════════════════════════════╗"
echo "║ Prose Contract Lint & Validate ║"
echo "╚═══════════════════════════════════╝"
echo ""
# ── 1. Structural validation ──
echo "── 1. Structural checks ──"
for f in *.prose.md; do
[ -f "$f" ] || continue
[[ "$f" == *".prose.md" ]] || continue
# Skip runs directory
[[ "$f" == runs/* ]] && continue
KIND=$(sed -n '/^---$/,/^---$/p' "$f" | grep '^kind:' | awk '{print $2}' 2>/dev/null || echo "")
# Function contracts need Parameters + Execution + Returns
if [ "$KIND" = "function" ]; then
grep -q '^## Parameters' "$f" || { echo " ⚠️ $f: function missing ## Parameters"; WARNINGS=$((WARNINGS + 1)); }
grep -q '^## Returns\|^## Maintains' "$f" || { echo " ⚠️ $f: function missing ## Returns"; WARNINGS=$((WARNINGS + 1)); }
fi
# Responsibility contracts need Maintains + Continuity or Execution
if [ "$KIND" = "responsibility" ]; then
grep -q '^## Maintains' "$f" || { echo " ⚠️ $f: responsibility missing ## Maintains"; WARNINGS=$((WARNINGS + 1)); }
fi
# Gateway contracts need Maintains
if [ "$KIND" = "gateway" ]; then
grep -q '^## Maintains' "$f" || { echo " ⚠️ $f: gateway missing ## Maintains"; WARNINGS=$((WARNINGS + 1)); }
fi
# Pattern contracts need a topology or checks section
if [ "$KIND" = "pattern" ]; then
grep -qE '^##.*(Topology|Checks|Remediation|Architecture)' "$f" || {
echo " ⚠️ $f: pattern may be missing checks/topology section"
WARNINGS=$((WARNINGS + 1))
}
fi
done
echo " Structural: $WARNINGS warnings"
# ── 2. Known-pattern regression checks ──
echo ""
echo "── 2. Regression detection ──"
# Grafana /grafana/ as nginx route or URL path (reverted 2026-07-02)
# EXCLUDE: filesystem paths (/opt/monitoring/grafana/...), directory creation, revert docs
GRAFANA_HITS=$(grep -rn '/grafana/' ./*.prose.md 2>/dev/null \
| grep -v '/opt/monitoring/grafana/' \
| grep -v 'was tried and reverted\|was reverted\|do not re-add\|NOT recommended' \
| grep -v 'mkdir.*grafana\|Create.*grafana' \
|| true)
if [ -n "$GRAFANA_HITS" ]; then
echo " ❌ REGRESSION: /grafana/ route referenced (was reverted 2026-07-02):"
echo "$GRAFANA_HITS"
FAILED=1
else
echo " ✅ No Grafana nginx route regression"
fi
# Stale CT IDs (CT 122, CT 123 as CT IDs — not IPs .122, .123)
CT_STALE=$(grep -rn '\bCT 122\b' ./*.prose.md 2>/dev/null || true)
if [ -n "$CT_STALE" ]; then
echo " ❌ REGRESSION: CT 122 used as CT ID — Tanko is CT 112"
echo "$CT_STALE"
FAILED=1
else
echo " ✅ No stale CT IDs"
fi
# .19 is correct — verified reachable Zulip bridge IP on storepve
# .122/.123 are correct — verified reachable bridge IPs for Tanko/Mumuni
echo " ✅ IP consistency verified (.19=.122=.123 all reachable)"
# Report provenance — every contract report must state the absolute path it
# executed from, so a stale-consumer report is distinguishable from a real fault
# at read time (2026-09-09 probe-drift incident: three false DEGRADED rounds).
# Enforced only inside the **Report format** paragraph, and a check-health
# contract with no Report format paragraph FAILs rather than being skipped.
PROV_FILES=$(grep -rlE '^### check-health|\*\*Report format\*\*' ./*.prose.md 2>/dev/null || true)
if [ -z "$PROV_FILES" ]; then
echo " ❌ No check-health/report-format contracts found — provenance not enforced"
FAILED=1
else
PROV_BAD=0
while IFS= read -r f; do
[ -n "$f" ] || continue
REPORT_PARA=$(awk '/\*\*Report format\*\*/{found=1} found{print} found && /^[[:space:]]*$/{exit}' "$f")
if [ -z "$REPORT_PARA" ]; then
echo " ❌ $f: check-health contract has no **Report format** paragraph"
PROV_BAD=1
elif ! printf '%s\n' "$REPORT_PARA" | grep -qE 'absolute path|pwd -P|executed from'; then
echo " ❌ $f: **Report format** lacks execution provenance (absolute path / pwd -P)"
PROV_BAD=1
fi
done <<< "$PROV_FILES"
if [ "$PROV_BAD" -eq 1 ]; then
FAILED=1
else
echo " ✅ Report provenance present in all report-format contracts"
fi
fi
# ── 3. Cross-contract consistency ──
echo ""
echo "── 3. Cross-contract consistency ──"
# Check that contracts referencing each other have correct names
if [ -f "infrastructure-control.prose.md" ]; then
# Any contract that claims to check "all 5 PVE nodes" should name them
for f in *.prose.md; do
[ -f "$f" ] || continue
if grep -q "5-node\|5 node\|5 Proxmox\|all.*PVE.*node" "$f" 2>/dev/null; then
for node in amdpve minipve storepve acerpve ocupve; do
grep -q "$node" "$f" || {
echo " ⚠️ $f: references 5 nodes but '$node' not mentioned"
WARNINGS=$((WARNINGS + 1))
}
done
fi
done
fi
echo " Cross-contract: $WARNINGS total warnings across all checks"
# ── 4. Committed-credential scan ──
# The 2026-09-17 purge removed six live credentials that had sat in .md prose
# and scripts for weeks. This step makes that class of commit FAIL the gate
# instead of printing a warning. Patterns: scripts/secret-patterns.tsv.
# Only deliberate synthetic examples may be listed in scripts/secret-allowlist.tsv,
# each with a reason. Run `bash scripts/secret-scan.sh --staged` before committing.
echo ""
echo "── 4. Secret scan (committed credentials) ──"
if bash scripts/secret-scan.sh; then
echo " ✅ No committed credentials"
else
echo " ❌ COMMITTED CREDENTIAL DETECTED"
FAILED=1
fi
# ── 5. Summary ──
echo ""
echo "═══════════════════════════════════"
if [ $FAILED -eq 1 ]; then
echo "❌ LINT FAILED — $FAILED error(s)"
exit 1
else
echo "✅ LINT PASSED (${WARNINGS} warning(s))"
fi