prose-contracts/search-stack-visibility.prose.md

kind, name, description, version

kind	name	description	version
function	search-stack-visibility	Makes the shared search stack observable. Every agent reaches one SearXNG instance (http://192.168.68.7:8888) and one extraction service (Firecrawl, http://192.168.68.7:3002). Before this check the stack could degrade to a single engine, or an enabled engine could return nothing at all, without any error surfacing anywhere. This contract runs scripts/search-stack-check.py, which: * runs two fixed queries against SearXNG and FAILS when fewer than two engines contribute, printing the contributing engines and every unresponsive_engines entry; * checks extraction by scraping a known page through Firecrawl and FAILS when the returned markdown is empty or the request fails; * reports every silent-zero engine explicitly (enabled, not in unresponsive_engines, contributed no results). Multi-engine post-fix state (2026-09-25): bing, google cse, brave, yandex contribute on every query; duckduckgo is best-effort via a VPS egress network and is expected to regress when DuckDuckGo flags the datacenter IP. SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15, via scripts/contract-run.sh search-stack-visibility. Logs land in /var/log/contract-runs/. A failure also raises a firstmate inbox note.	1.0.0

Purpose

The fleet has exactly one search endpoint and one extraction endpoint. If either degrades, every agent silently loses capability at the same moment. The failure mode this contract exists to close is silent degradation: a query that still returns a page of results while all but one engine have stopped contributing, or an enabled engine that answers with zero results and raises no error.

Execution model

The contract is a host-scheduled check, not an agent workflow. It is driven by scripts/contract-run.sh search-stack-visibility from /etc/cron.d/contract-runner on CT 100. contract-run.sh resolves the mapping to scripts/search-stack-check.py, runs it under a timeout, writes a timestamped log to /var/log/contract-runs/, and on non-zero exit raises a firstmate inbox note through bin/fm-inbox.sh.

What passing looks like

$ bash scripts/contract-run.sh search-stack-visibility
Expected engines, enabled (4): ['bing', 'brave', 'google cse', 'yandex']
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
                                    unresponsive: (none)
         'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
                                    unresponsive: (none)
EXTRACTION: 71016 chars of markdown returned
VERDICT: PASS -- multiple engines contributing, extraction healthy

What failing looks like

• A query whose results come from fewer than SEARCH_CHECK_MIN_ENGINES engines (default 2) fails and names the engines that did contribute.
• An extraction request that errors or returns empty markdown fails.
• An enabled, expected engine that contributed nothing without reporting an error is printed under SILENT-ZERO ENGINES REPORTED.

Configuration

Environment overrides (see the script docstring for the full list):

Variable	Default	Meaning
SEARXNG_URL	http://192.168.68.7:8888	SearXNG base URL
FIRECRAWL_URL	http://192.168.68.7:3002	Firecrawl base URL
SEARCH_CHECK_QUERIES	proxmox backup server,python asyncio tutorial	fixed queries
SEARCH_CHECK_MIN_ENGINES	2	minimum contributing engines per query
SEARCH_CHECK_ENGINES	bing,brave,google cse,yandex,duckduckgo	engines a silent zero is reported for
SEARCH_CHECK_EXTRACT_URL	Wikipedia Proxmox article	page used for the extraction leg

Known residual risk

DuckDuckGo is reached through a forward proxy on the VPS (10.10.10.1:3128, WireGuard) because the house egress IP is CAPTCHA'd. The VPS is a datacenter address and DuckDuckGo may flag it, so DuckDuckGo is coverage, never a required engine. When it regresses it appears either in unresponsive_engines (an explicit error) or under the silent-zero report; neither fails the check, but both are visible.