fix(hermes): reachability verdict must not come from the remote command's exit code #89

2026-09-14T12:04:16Z

abiba-bot commented

2026-09-14 12:04:16 +00:00

Fixes a defect that inverted this fleet's key-hygiene audit for two days.

The defect

hermes-key-enforcement, hermes-config-template and hermes-agent-baseline each reported "Tanko/Mumuni/Koonimo unreachable" while root SSH to all three answered instantly from the same host. The check ran ssh <host> "grep -r <pattern> <path>" and treated the exit status as reachability - but ssh returns the REMOTE command's status, so a successful connection whose grep matched nothing returned 1 and was rendered as "unreachable".

Consequence: three of four agents were reported at their worst precisely when they were clean, and were never actually audited. With the fix, the same contracts report tanko/mumuni/koonimo COMPLIANT and koby VIOLATION (a plaintext key inside a pre-update state snapshot - recorded for its owner; koby is report-only).

Changes

scripts/hermes-reachability-check.sh (new) - shared helper, usable sourced or standalone: hermes-reachability-check.sh <host> <pattern> <path>. The remote side ends with ; true, so ssh's status describes the CONNECTION only, and the verdict is three-way and mutually exclusive: UNREACHABLE (ssh failed), VIOLATION (connected, matches found - finding included), COMPLIANT (connected, no matches). A verdict line never exits non-zero, because the contract reads the line.
The three contracts - each now instructs the executor to run the helper per host and paste the returned line, and states explicitly that deriving reachability from the remote command's exit code is the bug this replaces.

Verification

The helper run standalone against all four agent hosts returns a verdict line for each (firstmate ran it: all four reachable and COMPLIANT for a hardcoded-key pattern; koby's snapshot finding comes from the wider path the contract scans).
The lane re-ran all three contracts after the change: tanko/mumuni/koonimo COMPLIANT, koby VIOLATION.
Reviewers: run the helper against a reachable host and (if possible) an unreachable address, and confirm the three outcomes are distinguishable; confirm no contract still derives reachability from a remote exit code.

Fixes a defect that inverted this fleet's key-hygiene audit for two days. ## The defect `hermes-key-enforcement`, `hermes-config-template` and `hermes-agent-baseline` each reported **"Tanko/Mumuni/Koonimo unreachable"** while root SSH to all three answered instantly from the same host. The check ran `ssh <host> "grep -r <pattern> <path>"` and treated the exit status as reachability - but ssh returns the REMOTE command's status, so a successful connection whose grep matched **nothing** returned 1 and was rendered as "unreachable". Consequence: three of four agents were reported at their worst precisely when they were **clean**, and were never actually audited. With the fix, the same contracts report tanko/mumuni/koonimo **COMPLIANT** and koby **VIOLATION** (a plaintext key inside a pre-update state snapshot - recorded for its owner; koby is report-only). ## Changes - **`scripts/hermes-reachability-check.sh`** (new) - shared helper, usable sourced or standalone: `hermes-reachability-check.sh <host> <pattern> <path>`. The remote side ends with `; true`, so ssh's status describes the CONNECTION only, and the verdict is three-way and mutually exclusive: `UNREACHABLE` (ssh failed), `VIOLATION` (connected, matches found - finding included), `COMPLIANT` (connected, no matches). A verdict line never exits non-zero, because the contract reads the line. - **The three contracts** - each now instructs the executor to run the helper per host and paste the returned line, and states explicitly that deriving reachability from the remote command's exit code is the bug this replaces. ## Verification - The helper run standalone against all four agent hosts returns a verdict line for each (firstmate ran it: all four reachable and COMPLIANT for a hardcoded-key pattern; koby's snapshot finding comes from the wider path the contract scans). - The lane re-ran all three contracts after the change: tanko/mumuni/koonimo COMPLIANT, koby VIOLATION. Reviewers: run the helper against a reachable host and (if possible) an unreachable address, and confirm the three outcomes are distinguishable; confirm no contract still derives reachability from a remote exit code.

abiba-bot added 2 commits 2026-09-14 12:04:17 +00:00

fix(hermes): add shared reachability check helper f9f6661dd5

Bug: The reachability check used 'ssh ... grep ... || echo unreachable',
which conflated grep's 'no matches found' (exit 1) with SSH failure.
This caused clean hosts to be reported as unreachable.

Fix: Add scripts/hermes-reachability-check.sh with the pattern:
  out=$(ssh -o BatchMode=yes root@HOST "grep ... 2>/dev/null; true")
  if [ $? -ne 0 ]; then verdict="unreachable"
  elif [ -n "$out" ]; then verdict="violation: $out"
  else verdict="compliant"
  fi

This correctly distinguishes:
- SSH failure (connection/auth/route) -> unreachable
- SSH success + grep found matches -> violation
- SSH success + grep found nothing -> compliant

Evidence: 3 of 4 hosts (Tanko, Mumuni, Koonimo) were reported as
'unreachable' when they were actually compliant. Only Koby (.129)
has a real finding (plaintext key in state snapshot).

Used by: hermes-key-enforcement, hermes-config-template,
hermes-agent-baseline contracts.

fix(hermes): wire all three contracts to reachability helper

PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 4s

Details

PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 5s

Details

PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 5s

Details

PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 6s

Details

PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s

Details

4e34b7a2a2

The helper was correct but dead code - nothing called it. This commit:
1. Makes the helper runnable standalone: scripts/hermes-reachability-check.sh <host> <pattern> <path>
2. Wires all THREE contracts to it:
   - hermes-key-enforcement.prose.md
   - hermes-config-template.prose.md
   - hermes-agent-baseline.prose.md
3. Each contract now explicitly instructs to run the helper and interpret the three outcomes
4. States that the bug this replaces was deriving reachability from the remote grep's exit code

Files changed (4):
- scripts/hermes-reachability-check.sh (standalone mode added)
- hermes-key-enforcement.prose.md (reachability section added)
- hermes-config-template.prose.md (reachability section added)
- hermes-agent-baseline.prose.md (reachability section added)

abiba-bot merged commit 71ceda0042 into master

2026-09-14 12:20:11 +00:00

abiba-bot referenced this pull request

2026-09-14 12:38:52 +00:00

fix(hermes): separate POLICY observations from FAULT findings in the audit contracts #90

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SyslogSolution/prose-contracts#89