fix(audit-hermes): handle fallback_providers as list or dict #141

2026-09-27T11:20:09Z

abiba-bot commented

2026-09-27 11:20:09 +00:00

Why

audit-hermes-config.py assumed fallback_providers is a dict. Two live agents carry it as a
list, so the key-hygiene audit died instead of evaluating them:

File "audit-hermes-config.py", line 211, in audit
    fb.get("provider") == "deepseek",
AttributeError: 'list' object has no attribute 'get'

Reproduced against the real configs of koby (192.168.68.129) and koonimo (192.168.68.114). The
audit therefore had no coverage for two of the four agents while appearing to run - and the
runner reported success daily for six days without executing at all, which this crash is very
likely connected to.

What changed

fallback_providers is now normalized to accept either a dict (single provider) or a list
of entries, applying the existing checks to each entry.
A malformed entry (a list element that is not a mapping) is reported as a VIOLATION naming the
offending entry rather than raising. A checker must never die on the input it audits.
Behaviour for the dict shape is unchanged; no existing rule was relaxed or re-scoped.

Evidence

Regression test added using the real failing shape, shown to fail against the pre-fix revision
with the AttributeError and pass after the fix.
All 14 existing tests in tests/test_audit_hermes_config_alias.py stay green.
Audit re-run against all four real agent configs: the two that previously crashed now return
verdicts (koby 16 violations, koonimo 10) instead of dying.

Not in scope

The violations found in agent configs (mumuni 7, tanko 21, koby 16, koonimo 10) are separate
findings and are deliberately NOT fixed here. Tanko has migrated to DSH, so its Hermes config is
vestigial - noted rather than acted on. The retired alias list (gpu-light, gemma-4-12b) still
fails as before.

## Why `audit-hermes-config.py` assumed `fallback_providers` is a dict. Two live agents carry it as a **list**, so the key-hygiene audit died instead of evaluating them: ``` File "audit-hermes-config.py", line 211, in audit fb.get("provider") == "deepseek", AttributeError: 'list' object has no attribute 'get' ``` Reproduced against the real configs of koby (192.168.68.129) and koonimo (192.168.68.114). The audit therefore had **no coverage for two of the four agents** while appearing to run - and the runner reported success daily for six days without executing at all, which this crash is very likely connected to. ## What changed - `fallback_providers` is now normalized to accept **either** a dict (single provider) **or** a list of entries, applying the existing checks to each entry. - A malformed entry (a list element that is not a mapping) is reported as a **VIOLATION naming the offending entry** rather than raising. A checker must never die on the input it audits. - Behaviour for the dict shape is unchanged; no existing rule was relaxed or re-scoped. ## Evidence - Regression test added using the real failing shape, shown to **fail against the pre-fix revision** with the AttributeError and pass after the fix. - All 14 existing tests in `tests/test_audit_hermes_config_alias.py` stay green. - Audit re-run against all four real agent configs: the two that previously crashed now return verdicts (koby 16 violations, koonimo 10) instead of dying. ## Not in scope The violations found in agent configs (mumuni 7, tanko 21, koby 16, koonimo 10) are separate findings and are deliberately NOT fixed here. Tanko has migrated to DSH, so its Hermes config is vestigial - noted rather than acted on. The retired alias list (`gpu-light`, `gemma-4-12b`) still fails as before.

abiba-bot added 1 commit 2026-09-27 11:20:10 +00:00

fix(audit-hermes): handle fallback_providers as list or dict

PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 5s

Details

PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 7s

Details

PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 17s

Details

PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s

Details

PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s

Details

e42b970dec

The audit assumed fallback_providers was always a dict (single provider).
Two live agents (koby, koonimo) carry it as a LIST of dicts (one entry per
fallback), so the script crashed with:

    File "audit-hermes-config.py", line 211, in audit
        fb.get("provider") == "deepseek",
    AttributeError: 'list' object has no attribute 'get'

Both are REAL agent configs, so this is not a malformed-input case — the
script simply could not audit two of the four agents it exists to audit.

Fix:
- Normalize fallback_providers to a list of entries (dict → [dict], list → list)
- Apply the existing checks to each entry
- A malformed entry (not a mapping) produces a reported VIOLATION naming the
  offending entry, NOT an uncaught exception

Adds regression test using the real failing shape (list of dicts) and proves
it bites against the pre-fix revision.

Real audit results after fix:
- mumuni: FAIL — 7 violations
- tanko: FAIL — 21 violations
- koby: FAIL — 16 violations (previously crashed)
- koonimo: FAIL — 10 violations (previously crashed)

No agent configs were changed. No existing rules were relaxed.

abiba-bot merged commit fb7f351a2b into master

2026-09-27 11:35:08 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SyslogSolution/prose-contracts#141