docs+check: correct DDG status, explain silent-zero semantics, credit google cse
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
Follow-up corrections after review: 1. DuckDuckGo is NOT fixed. The VPS fallback egress has since been flagged by DuckDuckGo too (HTTP 202 + challenge markers), so it reports CAPTCHA on both paths. The contract and script docstring now say so instead of claiming a fix that had already expired. It stays enabled as best-effort coverage so a recovery shows up as a contribution. 2. The relationship between silent zeros and the verdict is now explicit in both the script output and the contract: an enabled expected engine that contributes zero with no error is REPORTED, not fatal. Only the <SEARCH_CHECK_MIN_ENGINES> floor and the extraction leg fail the run. This is deliberate - de-duplication and query-shape make a zero non-probative. 3. Recorded that 'google cse' uses a THIRD PARTY's public search-engine id hardcoded in the SearXNG build, not a key we own; its quota and availability are outside our control, and our own free key would need a wrapper (not built). VPS forward proxy is now a real service: /opt/fwd-proxy docker compose with restart: unless-stopped, a healthy healthcheck, and Docker enabled at boot.
This commit is contained in:
@@ -174,7 +174,10 @@ def main() -> int:
|
||||
)
|
||||
if silent:
|
||||
silent_zero_all[query] = silent
|
||||
print(f" SILENT ZERO (enabled, no error, no results): {silent}")
|
||||
print(
|
||||
" SILENT ZERO (enabled, no error, no results -- reported, "
|
||||
f"not fatal): {silent}"
|
||||
)
|
||||
|
||||
print("=" * 72)
|
||||
print("Engine contribution across all queries:")
|
||||
@@ -187,6 +190,11 @@ def main() -> int:
|
||||
print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):")
|
||||
for query, names in silent_zero_all.items():
|
||||
print(f" {query!r}: {names}")
|
||||
print(" NOTE: a silent zero is REPORTED, not counted as a failure. These")
|
||||
print(" engines are expected to answer a general query, but contributing")
|
||||
print(" nothing to one query can be legitimate (result de-duplication, or")
|
||||
print(" an engine that only fires on certain query shapes). Only the")
|
||||
print(f" <{MIN_ENGINES}-contributing-engine floor and the extraction leg fail the run.")
|
||||
|
||||
print("-" * 72)
|
||||
print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape")
|
||||
|
||||
@@ -17,14 +17,19 @@ description: >
|
||||
* reports every silent-zero engine explicitly (enabled, not in
|
||||
unresponsive_engines, contributed no results).
|
||||
|
||||
Multi-engine post-fix state (2026-09-25): bing, google cse, brave, yandex
|
||||
contribute on every query; duckduckgo is best-effort via a VPS egress
|
||||
network and is expected to regress when DuckDuckGo flags the datacenter IP.
|
||||
Multi-engine state (2026-09-25): bing, google cse, brave and yandex
|
||||
contribute on every query. duckduckgo is NOT working: the house egress IP
|
||||
and the VPS fallback egress are both flagged by DuckDuckGo and it reports
|
||||
CAPTCHA. It is left enabled as best-effort coverage so that a recovery shows
|
||||
up as a contribution.
|
||||
|
||||
google cse is a third party's public search-engine id hardcoded in the
|
||||
SearXNG build. Quota and availability are outside our control.
|
||||
|
||||
SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15,
|
||||
via scripts/contract-run.sh search-stack-visibility. Logs land in
|
||||
/var/log/contract-runs/. A failure also raises a firstmate inbox note.
|
||||
version: 1.0.0
|
||||
version: 1.1.0
|
||||
---
|
||||
|
||||
## Purpose
|
||||
@@ -49,11 +54,10 @@ firstmate inbox note through `bin/fm-inbox.sh`.
|
||||
|
||||
```
|
||||
$ bash scripts/contract-run.sh search-stack-visibility
|
||||
Expected engines, enabled (4): ['bing', 'brave', 'google cse', 'yandex']
|
||||
Expected engines, enabled (5): ['bing', 'brave', 'duckduckgo', 'google cse', 'yandex']
|
||||
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
|
||||
unresponsive: (none)
|
||||
unresponsive: duckduckgo=CAPTCHA
|
||||
'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
|
||||
unresponsive: (none)
|
||||
EXTRACTION: 71016 chars of markdown returned
|
||||
VERDICT: PASS -- multiple engines contributing, extraction healthy
|
||||
```
|
||||
@@ -63,8 +67,34 @@ VERDICT: PASS -- multiple engines contributing, extraction healthy
|
||||
* A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES`
|
||||
engines (default 2) fails and names the engines that did contribute.
|
||||
* An extraction request that errors or returns empty markdown fails.
|
||||
* An enabled, expected engine that contributed nothing without reporting an
|
||||
error is printed under `SILENT-ZERO ENGINES REPORTED`.
|
||||
|
||||
## Silent zeros are reported, not fatal
|
||||
|
||||
An enabled, expected engine that contributed nothing **without reporting an
|
||||
error** is printed under `SILENT-ZERO ENGINES REPORTED`, and each occurrence is
|
||||
annotated `reported, not fatal`. This is deliberate:
|
||||
|
||||
* a general query can legitimately draw zero results from an engine that only
|
||||
fires on certain query shapes, and results are de-duplicated across engines,
|
||||
so a zero does not by itself prove the engine is broken;
|
||||
* the run therefore fails only on the two conditions that do prove loss of
|
||||
capability -- fewer than two contributing engines, and a broken extraction
|
||||
leg.
|
||||
|
||||
A run can consequently print `VERDICT: PASS` while still listing a silent
|
||||
zero. That is the intended relationship: the zero is *visible*, not *fatal*.
|
||||
An engine that fails with an error (for example DuckDuckGo returning CAPTCHA)
|
||||
appears in `unresponsive_engines` instead.
|
||||
|
||||
## Google coverage is third-party, not ours
|
||||
|
||||
The free Google-derived results come from the SearXNG build's built-in
|
||||
`google cse` engine. It uses **a third party's public search-engine id
|
||||
hardcoded in the build** (`google_cse.py`, `CX = "partner-pub-8993..."`,
|
||||
blackle.com), not a key or id we own. Its quota and availability are outside
|
||||
our control and it can be rate-limited or withdrawn without notice. No engine
|
||||
in this build accepts our own Google Custom Search key; using our own free key
|
||||
would require a small wrapper service, which is deliberately **not** built.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -81,9 +111,14 @@ Environment overrides (see the script docstring for the full list):
|
||||
|
||||
## Known residual risk
|
||||
|
||||
DuckDuckGo is reached through a forward proxy on the VPS
|
||||
(`10.10.10.1:3128`, WireGuard) because the house egress IP is CAPTCHA'd. The
|
||||
VPS is a datacenter address and DuckDuckGo may flag it, so DuckDuckGo is
|
||||
coverage, never a required engine. When it regresses it appears either in
|
||||
`unresponsive_engines` (an explicit error) or under the silent-zero report;
|
||||
neither fails the check, but both are visible.
|
||||
DuckDuckGo is **not** working. The house egress IP is CAPTCHA'd by
|
||||
DuckDuckGo, and a forward proxy on the VPS (`10.10.10.1:3128`, WireGuard) was
|
||||
built as a second egress -- but DuckDuckGo has since flagged the VPS address
|
||||
too (HTTP 202 with challenge markers), so DuckDuckGo now reports CAPTCHA on
|
||||
both paths. It is left enabled as best-effort coverage: if DuckDuckGo
|
||||
unflags either address it will show up as a contribution, and until then it is
|
||||
visible in `unresponsive_engines` every run. It is never a required engine.
|
||||
|
||||
The VPS forward proxy remains a real service (`/opt/fwd-proxy`,
|
||||
`restart: unless-stopped`, healthy healthcheck, Docker enabled at boot) so the
|
||||
second egress path is available for any engine that benefits from it in future.
|
||||
|
||||
Reference in New Issue
Block a user