hermes-key-enforcement.prose.md

@@ -186,30 +186,55 @@ Agent keys live in `.env` or `.env.vault` files with 600 permissions (koonimo's
 
 ## Detection Query
 
-Run on any Hermes host to detect violations:
+Run on any Hermes host to detect violations.
+
+**Timeout policy (2026-10-02):** The scan timeout is **15 seconds**, set from measured cost on the largest target (koby, 16 GB `.hermes` tree; full scan = 0.91 s, bounded scan = 0.44 s). The SSH connection timeout is **10 seconds** (separate from the scan timeout). A scan timeout renders as `probe-failed: <agent> <ip> (timeout after 15s)` — **never** as "unreachable" or "may be down". An SSH connection failure (exit status 255) renders as `unreachable: <agent> <ip> (ssh connect failed)`.
+
+**Bounded scan (2026-10-02):** Do NOT recurse the entire `/root/.hermes/` tree. Use `--exclude-dir=state-snapshots` to skip dated snapshot directories. Rationale: a superseded config will always carry a superseded key and will report forever with zero signal content (the koby state-snapshot line has repeated on consecutive days). If you deliberately want to include snapshots, say so in the contract and the report.
 
 ```bash
-# 1. Check config.yaml for hardcoded harness keys
-grep -rn 'api_key: sk-' /root/.hermes/ \
-  --include='config.yaml' \
-  | grep -v 'deepseek\|openai\|anthropic\|DEEPSEEK'
+# 1. Check config.yaml for hardcoded harness keys (bounded scan — excludes state-snapshots)
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
+  "grep -rn 'api_key: sk-' /root/.hermes/ --exclude-dir=state-snapshots --include='config.yaml' | grep -v 'deepseek\|openai\|anthropic\|DEEPSEEK'" \
+  2>/dev/null
+
+# Interpret exit status:
+#   0 = match found (violation)
+#   1 = no match (pass)
+#   124 = timeout (probe-failed, not unreachable)
+#   255 = ssh connect failed (unreachable)
+#   other = probe-failed (record the actual code)
 
 # 1b. Check for double-path bug: base_url ending with /responses
 # (Hermes appends /v1/responses when api_mode=responses, so base_url must end at /v1)
-grep -rn 'litellm/v1/responses' /root/.hermes/config.yaml
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
+  "grep -rn 'litellm/v1/responses' /root/.hermes/config.yaml" 2>/dev/null
 # ANY output here = WRONG. Must be 'litellm/v1' without /responses suffix.
 
 # 2. Check systemd drop-ins for master key leaks (2026-07-05: Tanko had this)
-grep -rn 'LITELLM_API_KEY' /root/.config/systemd/user/ 2>/dev/null
-grep -rn 'LITELLM_API_KEY=sk-synthetic-litellm-…' /root/.config/systemd/ 2>/dev/null
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
+  "grep -rn 'LITELLM_API_KEY' /root/.config/systemd/user/ 2>/dev/null" \
+  "grep -rn 'LITELLM_API_KEY=sk-synthetic-litellm-…' /root/.config/systemd/ 2>/dev/null"
 
 # 3. Verify running process env matches dedicated key
-cat /proc/$(cat /home/jerome/.hermes/gateway.pid | python3 -c "import sys,json; print(json.load(sys.stdin)['pid'])")/environ \
-  | tr '\0' '\n' | grep LITELLM_API_KEY
+timeout 15 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@<ip> \
+  "cat /proc/$(cat /home/jerome/.hermes/gateway.pid | python3 -c \"import sys,json; print(json.load(sys.stdin)['pid'])\")/environ | tr '\0' '\n' | grep LITELLM_API_KEY"
 ```
 
 If any output from step 2 — **critical violation** (master key leaked). Fix immediately.
 
+### Negative control (probe-failed vs unreachable)
+
+To prove the distinction between a scan timeout and a connection failure, run with a deliberately tiny timeout:
+
+```bash
+# Negative control: 1-second timeout on koby (scan takes 0.9 s, so this will time out)
+timeout 1 ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=no root@192.168.68.129 \
+  "grep -rn 'api_key: sk-' /root/.hermes/ --exclude-dir=state-snapshots --include='config.yaml'" 2>/dev/null
+# Expected: exit status 124 (timeout) → render as "probe-failed: koby 192.168.68.129 (timeout after 1s)"
+# NOT: "unreachable" or "may be down"
+```
+
 ## Rotation Procedure
 
 With this standard enforced, key rotation is one vault update: