docs+check: correct DDG status, explain silent-zero semantics, credit google cse
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s

Follow-up corrections after review:

1. DuckDuckGo is NOT fixed. The VPS fallback egress has since been flagged by
   DuckDuckGo too (HTTP 202 + challenge markers), so it reports CAPTCHA on both
   paths. The contract and script docstring now say so instead of claiming a
   fix that had already expired. It stays enabled as best-effort coverage so a
   recovery shows up as a contribution.

2. The relationship between silent zeros and the verdict is now explicit in
   both the script output and the contract: an enabled expected engine that
   contributes zero with no error is REPORTED, not fatal. Only the
   <SEARCH_CHECK_MIN_ENGINES> floor and the extraction leg fail the run. This is
   deliberate - de-duplication and query-shape make a zero non-probative.

3. Recorded that 'google cse' uses a THIRD PARTY's public search-engine id
   hardcoded in the SearXNG build, not a key we own; its quota and availability
   are outside our control, and our own free key would need a wrapper (not
   built).

VPS forward proxy is now a real service: /opt/fwd-proxy docker compose with
restart: unless-stopped, a healthy healthcheck, and Docker enabled at boot.
This commit is contained in:
root
2026-09-25 01:19:27 +00:00
parent 040fecef3e
commit 8b2eba4f7a
2 changed files with 59 additions and 16 deletions
+9 -1
View File
@@ -174,7 +174,10 @@ def main() -> int:
) )
if silent: if silent:
silent_zero_all[query] = silent silent_zero_all[query] = silent
print(f" SILENT ZERO (enabled, no error, no results): {silent}") print(
" SILENT ZERO (enabled, no error, no results -- reported, "
f"not fatal): {silent}"
)
print("=" * 72) print("=" * 72)
print("Engine contribution across all queries:") print("Engine contribution across all queries:")
@@ -187,6 +190,11 @@ def main() -> int:
print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):") print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):")
for query, names in silent_zero_all.items(): for query, names in silent_zero_all.items():
print(f" {query!r}: {names}") print(f" {query!r}: {names}")
print(" NOTE: a silent zero is REPORTED, not counted as a failure. These")
print(" engines are expected to answer a general query, but contributing")
print(" nothing to one query can be legitimate (result de-duplication, or")
print(" an engine that only fires on certain query shapes). Only the")
print(f" <{MIN_ENGINES}-contributing-engine floor and the extraction leg fail the run.")
print("-" * 72) print("-" * 72)
print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape") print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape")
+50 -15
View File
@@ -17,14 +17,19 @@ description: >
* reports every silent-zero engine explicitly (enabled, not in * reports every silent-zero engine explicitly (enabled, not in
unresponsive_engines, contributed no results). unresponsive_engines, contributed no results).
Multi-engine post-fix state (2026-09-25): bing, google cse, brave, yandex Multi-engine state (2026-09-25): bing, google cse, brave and yandex
contribute on every query; duckduckgo is best-effort via a VPS egress contribute on every query. duckduckgo is NOT working: the house egress IP
network and is expected to regress when DuckDuckGo flags the datacenter IP. and the VPS fallback egress are both flagged by DuckDuckGo and it reports
CAPTCHA. It is left enabled as best-effort coverage so that a recovery shows
up as a contribution.
google cse is a third party's public search-engine id hardcoded in the
SearXNG build. Quota and availability are outside our control.
SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15, SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15,
via scripts/contract-run.sh search-stack-visibility. Logs land in via scripts/contract-run.sh search-stack-visibility. Logs land in
/var/log/contract-runs/. A failure also raises a firstmate inbox note. /var/log/contract-runs/. A failure also raises a firstmate inbox note.
version: 1.0.0 version: 1.1.0
--- ---
## Purpose ## Purpose
@@ -49,11 +54,10 @@ firstmate inbox note through `bin/fm-inbox.sh`.
``` ```
$ bash scripts/contract-run.sh search-stack-visibility $ bash scripts/contract-run.sh search-stack-visibility
Expected engines, enabled (4): ['bing', 'brave', 'google cse', 'yandex'] Expected engines, enabled (5): ['bing', 'brave', 'duckduckgo', 'google cse', 'yandex']
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
unresponsive: (none) unresponsive: duckduckgo=CAPTCHA
'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex 'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
unresponsive: (none)
EXTRACTION: 71016 chars of markdown returned EXTRACTION: 71016 chars of markdown returned
VERDICT: PASS -- multiple engines contributing, extraction healthy VERDICT: PASS -- multiple engines contributing, extraction healthy
``` ```
@@ -63,8 +67,34 @@ VERDICT: PASS -- multiple engines contributing, extraction healthy
* A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES` * A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES`
engines (default 2) fails and names the engines that did contribute. engines (default 2) fails and names the engines that did contribute.
* An extraction request that errors or returns empty markdown fails. * An extraction request that errors or returns empty markdown fails.
* An enabled, expected engine that contributed nothing without reporting an
error is printed under `SILENT-ZERO ENGINES REPORTED`. ## Silent zeros are reported, not fatal
An enabled, expected engine that contributed nothing **without reporting an
error** is printed under `SILENT-ZERO ENGINES REPORTED`, and each occurrence is
annotated `reported, not fatal`. This is deliberate:
* a general query can legitimately draw zero results from an engine that only
fires on certain query shapes, and results are de-duplicated across engines,
so a zero does not by itself prove the engine is broken;
* the run therefore fails only on the two conditions that do prove loss of
capability -- fewer than two contributing engines, and a broken extraction
leg.
A run can consequently print `VERDICT: PASS` while still listing a silent
zero. That is the intended relationship: the zero is *visible*, not *fatal*.
An engine that fails with an error (for example DuckDuckGo returning CAPTCHA)
appears in `unresponsive_engines` instead.
## Google coverage is third-party, not ours
The free Google-derived results come from the SearXNG build's built-in
`google cse` engine. It uses **a third party's public search-engine id
hardcoded in the build** (`google_cse.py`, `CX = "partner-pub-8993..."`,
blackle.com), not a key or id we own. Its quota and availability are outside
our control and it can be rate-limited or withdrawn without notice. No engine
in this build accepts our own Google Custom Search key; using our own free key
would require a small wrapper service, which is deliberately **not** built.
## Configuration ## Configuration
@@ -81,9 +111,14 @@ Environment overrides (see the script docstring for the full list):
## Known residual risk ## Known residual risk
DuckDuckGo is reached through a forward proxy on the VPS DuckDuckGo is **not** working. The house egress IP is CAPTCHA'd by
(`10.10.10.1:3128`, WireGuard) because the house egress IP is CAPTCHA'd. The DuckDuckGo, and a forward proxy on the VPS (`10.10.10.1:3128`, WireGuard) was
VPS is a datacenter address and DuckDuckGo may flag it, so DuckDuckGo is built as a second egress -- but DuckDuckGo has since flagged the VPS address
coverage, never a required engine. When it regresses it appears either in too (HTTP 202 with challenge markers), so DuckDuckGo now reports CAPTCHA on
`unresponsive_engines` (an explicit error) or under the silent-zero report; both paths. It is left enabled as best-effort coverage: if DuckDuckGo
neither fails the check, but both are visible. unflags either address it will show up as a contribution, and until then it is
visible in `unresponsive_engines` every run. It is never a required engine.
The VPS forward proxy remains a real service (`/opt/fwd-proxy`,
`restart: unless-stopped`, healthy healthcheck, Docker enabled at boot) so the
second egress path is available for any engine that benefits from it in future.