fix: hermes-key-enforcement probe timeout - bounded scan, honest failure kinds

Problem: koby's 16 GB .hermes tree made the grep scan take 5.7s, exceeding
the leg's timeout. The timeout was rendered as 'agent may be down' when the
host was actually up and the scan just needed more time.

Changes:
1. Bounded scan: --exclude-dir=state-snapshots (0.44s vs 0.91s on koby)
2. Timeout policy: 15s scan timeout, 10s SSH connect timeout (documented)
3. Failure kinds: probe-failed (timeout) vs unreachable (ssh connect failed)
4. Negative control: 1s timeout proves probe-failed not down

Measured cost (2026-10-02): koby full scan 0.91s, bounded 0.44s.
Timeout set to 15s for headroom.

Correlation: corr=69683f073322b46c

hermes-key-enforcement.prose.md

@@ -186,30 +186,55 @@ Agent keys live in `.env` or `.env.vault` files with 600 permissions (koonimo's
 
 ## Detection Query
 
-Run on any Hermes host to detect violations:
+Run on any Hermes host to detect violations.
+
+**Timeout policy (2026-10-02):** The scan timeout is **15 seconds**, set from measured cost on the largest target (koby, 16 GB `.hermes` tree; full scan = 0.91 s, bounded scan = 0.44 s). The SSH connection timeout is **10 seconds** (separate from the scan timeout). A scan timeout renders as `probe-failed: <agent> <ip> (timeout after 15s)` — **never** as "unreachable" or "may be down". An SSH connection failure (exit status 255) renders as `unreachable: <agent> <ip> (ssh connect failed)`.
+
+**Bounded scan (2026-10-02):** Do NOT recurse the entire `/root/.hermes/` tree. Use `--exclude-dir=state-snapshots` to skip dated snapshot directories. Rationale: a superseded config will always carry a superseded key and will report forever with zero signal content (the koby state-snapshot line has repeated on consecutive days). If you deliberately want to include snapshots, say so in the contract and the report.
 
 ```bash
-# 1. Check config.yaml for hardcoded harness keys
+# 1. Check config.yaml for hardcoded harness keys (bounded scan — excludes state-snapshots)
-grep -rn 'api_key: sk-' /root/.hermes/ \
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
-  --include='config.yaml' \
+  "grep -rn 'api_key: sk-' /root/.hermes/ --exclude-dir=state-snapshots --include='config.yaml' | grep -v 'deepseek\|openai\|anthropic\|DEEPSEEK'" \
-  | grep -v 'deepseek\|openai\|anthropic\|DEEPSEEK'
+  2>/dev/null
+
+# Interpret exit status:
+#   0 = match found (violation)
+#   1 = no match (pass)
+#   124 = timeout (probe-failed, not unreachable)
+#   255 = ssh connect failed (unreachable)
+#   other = probe-failed (record the actual code)
 
 # 1b. Check for double-path bug: base_url ending with /responses
 # (Hermes appends /v1/responses when api_mode=responses, so base_url must end at /v1)
-grep -rn 'litellm/v1/responses' /root/.hermes/config.yaml
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
+  "grep -rn 'litellm/v1/responses' /root/.hermes/config.yaml" 2>/dev/null
 # ANY output here = WRONG. Must be 'litellm/v1' without /responses suffix.
 
 # 2. Check systemd drop-ins for master key leaks (2026-07-05: Tanko had this)
-grep -rn 'LITELLM_API_KEY' /root/.config/systemd/user/ 2>/dev/null
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
-grep -rn 'LITELLM_API_KEY=sk-synthetic-litellm-…' /root/.config/systemd/ 2>/dev/null
+  "grep -rn 'LITELLM_API_KEY' /root/.config/systemd/user/ 2>/dev/null" \
+  "grep -rn 'LITELLM_API_KEY=sk-synthetic-litellm-…' /root/.config/systemd/ 2>/dev/null"
 
 # 3. Verify running process env matches dedicated key
-cat /proc/$(cat /home/jerome/.hermes/gateway.pid | python3 -c "import sys,json; print(json.load(sys.stdin)['pid'])")/environ \
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
-  | tr '\0' '\n' | grep LITELLM_API_KEY
+  "cat /proc/$(cat /home/jerome/.hermes/gateway.pid | python3 -c \"import sys,json; print(json.load(sys.stdin)['pid'])\")/environ | tr '\0' '\n' | grep LITELLM_API_KEY"
 ```
 
 If any output from step 2 — **critical violation** (master key leaked). Fix immediately.
 
+### Negative control (probe-failed vs unreachable)
+
+To prove the distinction between a scan timeout and a connection failure, run with a deliberately tiny timeout:
+
+```bash
+# Negative control: 1-second timeout on koby (scan takes 0.9 s, so this will time out)
+timeout 1 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@192.168.68.129 \
+  "grep -rn 'api_key: sk-' /root/.hermes/ --exclude-dir=state-snapshots --include='config.yaml'" 2>/dev/null
+# Expected: exit status 124 (timeout) → render as "probe-failed: koby 192.168.68.129 (timeout after 1s)"
+# NOT: "unreachable" or "may be down"
+```
+
 ## Rotation Procedure
 
 With this standard enforced, key rotation is one vault update: