Files
zulip-platform-plugins/hermes-enhancement/CHECKLIST-POC.md
T
rootandAbiba (pi) aa305ce431 Add hermes-enhancement/: pi-style behavioral enhancement for Hermes agents
Self-service enhancement package for Hermes agents to adopt pi-style
conduct quality. Contains:
- prompts/behavioral-core.md: Distilled Three Pillars (~800 tokens)
- config/compression.yaml: 256K model optimization (80% threshold)
- config/mcp-servers.yaml: Tool parity (Context7, GitHub, Firecrawl, etc.)
- skills/: On-demand skills for conduct, verification, self-healing
- CHECKLIST-POC.md: Tanko POC verification checklist

POC pilot: Tanko
2026-06-27 19:03:43 +00:00

2.3 KiB

POC Checklist — Tanko Enhancement Verification

Pilot: Tanko Date: 2026-06-24 Status: Not started

Pre-Checks

  • Read this entire repo (README.md, INSTALL.md, all prompts and configs)
  • Verify current Hermes version
  • Backup current config.yaml, SOUL.md, and skills directory
  • Note baseline context usage (prompt tokens before any changes)

Installation

  • Merge prompts/behavioral-core.md into SOUL.md behavioral section
  • Add MCP servers from config/mcp-servers.yaml to config.yaml
  • Verify MCP tools appear (expect: ra-h-os, context7, github, firecrawl, searxng)
  • Copy skills/ to skills directory
  • Update compression config from config/compression.yaml
  • Update external_dirs in config.yaml for skills loading
  • Restart Hermes session

Behavioral Verification

  • Agent demonstrates Proactive behavior — suggests improvements unprompted
  • Agent demonstrates Persistent behavior — recovers from compaction without missing a beat
  • Agent demonstrates Self-Improving behavior — diagnoses errors, logs learnings
  • Agent verifies CLI flags and API params before answering (typescript-like discipline)
  • Agent is direct and decisive — no option-listing, no unnecessary politeness

Tool Parity Verification

  • Can query RA-H OS knowledge graph (ra-h-os-queryNodes)
  • Can verify docs via Context7 (context7-resolve-library-id + context7-query-docs)
  • Can search code via GitHub MCP (github-search_code)
  • Can scrape web pages via Firecrawl (firecrawl-scrape)
  • Can search web via SearXNG (searxng-searxng_web_search)

Performance Verification

  • Measure first-token latency before/after
  • Measure total system prompt tokens before/after (target: 50-60% reduction)
  • Verify compression triggers at correct threshold (80% of 256K = 204,800 tokens)
  • Verify no functionality lost — can still accomplish all prior tasks

Rollback

  • If enhancement causes regression, restore from backup
  • Report what failed to SyslogSolution/hermes-pi-enhancement issues

Report

After completing, Tanko should create a relay message or issue with:

  • Before/after measurements (prompt tokens, latency)
  • What felt different (qualitative)
  • Any tools that didn't work
  • Recommendations for other agents