docs+check: correct DDG status, explain silent-zero semantics, credit google cse
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s

Follow-up corrections after review:

1. DuckDuckGo is NOT fixed. The VPS fallback egress has since been flagged by
   DuckDuckGo too (HTTP 202 + challenge markers), so it reports CAPTCHA on both
   paths. The contract and script docstring now say so instead of claiming a
   fix that had already expired. It stays enabled as best-effort coverage so a
   recovery shows up as a contribution.

2. The relationship between silent zeros and the verdict is now explicit in
   both the script output and the contract: an enabled expected engine that
   contributes zero with no error is REPORTED, not fatal. Only the
   <SEARCH_CHECK_MIN_ENGINES> floor and the extraction leg fail the run. This is
   deliberate - de-duplication and query-shape make a zero non-probative.

3. Recorded that 'google cse' uses a THIRD PARTY's public search-engine id
   hardcoded in the SearXNG build, not a key we own; its quota and availability
   are outside our control, and our own free key would need a wrapper (not
   built).

VPS forward proxy is now a real service: /opt/fwd-proxy docker compose with
restart: unless-stopped, a healthy healthcheck, and Docker enabled at boot.
This commit is contained in:
root
2026-09-25 01:19:27 +00:00
parent 040fecef3e
commit 8b2eba4f7a
2 changed files with 59 additions and 16 deletions
+9 -1
View File
@@ -174,7 +174,10 @@ def main() -> int:
)
if silent:
silent_zero_all[query] = silent
print(f" SILENT ZERO (enabled, no error, no results): {silent}")
print(
" SILENT ZERO (enabled, no error, no results -- reported, "
f"not fatal): {silent}"
)
print("=" * 72)
print("Engine contribution across all queries:")
@@ -187,6 +190,11 @@ def main() -> int:
print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):")
for query, names in silent_zero_all.items():
print(f" {query!r}: {names}")
print(" NOTE: a silent zero is REPORTED, not counted as a failure. These")
print(" engines are expected to answer a general query, but contributing")
print(" nothing to one query can be legitimate (result de-duplication, or")
print(" an engine that only fires on certain query shapes). Only the")
print(f" <{MIN_ENGINES}-contributing-engine floor and the extraction leg fail the run.")
print("-" * 72)
print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape")
+50 -15
View File
@@ -17,14 +17,19 @@ description: >
* reports every silent-zero engine explicitly (enabled, not in
unresponsive_engines, contributed no results).
Multi-engine post-fix state (2026-09-25): bing, google cse, brave, yandex
contribute on every query; duckduckgo is best-effort via a VPS egress
network and is expected to regress when DuckDuckGo flags the datacenter IP.
Multi-engine state (2026-09-25): bing, google cse, brave and yandex
contribute on every query. duckduckgo is NOT working: the house egress IP
and the VPS fallback egress are both flagged by DuckDuckGo and it reports
CAPTCHA. It is left enabled as best-effort coverage so that a recovery shows
up as a contribution.
google cse is a third party's public search-engine id hardcoded in the
SearXNG build. Quota and availability are outside our control.
SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15,
via scripts/contract-run.sh search-stack-visibility. Logs land in
/var/log/contract-runs/. A failure also raises a firstmate inbox note.
version: 1.0.0
version: 1.1.0
---
## Purpose
@@ -49,11 +54,10 @@ firstmate inbox note through `bin/fm-inbox.sh`.
```
$ bash scripts/contract-run.sh search-stack-visibility
Expected engines, enabled (4): ['bing', 'brave', 'google cse', 'yandex']
Expected engines, enabled (5): ['bing', 'brave', 'duckduckgo', 'google cse', 'yandex']
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
unresponsive: (none)
unresponsive: duckduckgo=CAPTCHA
'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
unresponsive: (none)
EXTRACTION: 71016 chars of markdown returned
VERDICT: PASS -- multiple engines contributing, extraction healthy
```
@@ -63,8 +67,34 @@ VERDICT: PASS -- multiple engines contributing, extraction healthy
* A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES`
engines (default 2) fails and names the engines that did contribute.
* An extraction request that errors or returns empty markdown fails.
* An enabled, expected engine that contributed nothing without reporting an
error is printed under `SILENT-ZERO ENGINES REPORTED`.
## Silent zeros are reported, not fatal
An enabled, expected engine that contributed nothing **without reporting an
error** is printed under `SILENT-ZERO ENGINES REPORTED`, and each occurrence is
annotated `reported, not fatal`. This is deliberate:
* a general query can legitimately draw zero results from an engine that only
fires on certain query shapes, and results are de-duplicated across engines,
so a zero does not by itself prove the engine is broken;
* the run therefore fails only on the two conditions that do prove loss of
capability -- fewer than two contributing engines, and a broken extraction
leg.
A run can consequently print `VERDICT: PASS` while still listing a silent
zero. That is the intended relationship: the zero is *visible*, not *fatal*.
An engine that fails with an error (for example DuckDuckGo returning CAPTCHA)
appears in `unresponsive_engines` instead.
## Google coverage is third-party, not ours
The free Google-derived results come from the SearXNG build's built-in
`google cse` engine. It uses **a third party's public search-engine id
hardcoded in the build** (`google_cse.py`, `CX = "partner-pub-8993..."`,
blackle.com), not a key or id we own. Its quota and availability are outside
our control and it can be rate-limited or withdrawn without notice. No engine
in this build accepts our own Google Custom Search key; using our own free key
would require a small wrapper service, which is deliberately **not** built.
## Configuration
@@ -81,9 +111,14 @@ Environment overrides (see the script docstring for the full list):
## Known residual risk
DuckDuckGo is reached through a forward proxy on the VPS
(`10.10.10.1:3128`, WireGuard) because the house egress IP is CAPTCHA'd. The
VPS is a datacenter address and DuckDuckGo may flag it, so DuckDuckGo is
coverage, never a required engine. When it regresses it appears either in
`unresponsive_engines` (an explicit error) or under the silent-zero report;
neither fails the check, but both are visible.
DuckDuckGo is **not** working. The house egress IP is CAPTCHA'd by
DuckDuckGo, and a forward proxy on the VPS (`10.10.10.1:3128`, WireGuard) was
built as a second egress -- but DuckDuckGo has since flagged the VPS address
too (HTTP 202 with challenge markers), so DuckDuckGo now reports CAPTCHA on
both paths. It is left enabled as best-effort coverage: if DuckDuckGo
unflags either address it will show up as a contribution, and until then it is
visible in `unresponsive_engines` every run. It is never a required engine.
The VPS forward proxy remains a real service (`/opt/fwd-proxy`,
`restart: unless-stopped`, healthy healthcheck, Docker enabled at boot) so the
second egress path is available for any engine that benefits from it in future.