docs+check: correct DDG status, explain silent-zero semantics, credit google cse
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 11s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 8s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
Follow-up corrections after review: 1. DuckDuckGo is NOT fixed. The VPS fallback egress has since been flagged by DuckDuckGo too (HTTP 202 + challenge markers), so it reports CAPTCHA on both paths. The contract and script docstring now say so instead of claiming a fix that had already expired. It stays enabled as best-effort coverage so a recovery shows up as a contribution. 2. The relationship between silent zeros and the verdict is now explicit in both the script output and the contract: an enabled expected engine that contributes zero with no error is REPORTED, not fatal. Only the <SEARCH_CHECK_MIN_ENGINES> floor and the extraction leg fail the run. This is deliberate - de-duplication and query-shape make a zero non-probative. 3. Recorded that 'google cse' uses a THIRD PARTY's public search-engine id hardcoded in the SearXNG build, not a key we own; its quota and availability are outside our control, and our own free key would need a wrapper (not built). VPS forward proxy is now a real service: /opt/fwd-proxy docker compose with restart: unless-stopped, a healthy healthcheck, and Docker enabled at boot.
This commit is contained in:
@@ -174,7 +174,10 @@ def main() -> int:
|
|||||||
)
|
)
|
||||||
if silent:
|
if silent:
|
||||||
silent_zero_all[query] = silent
|
silent_zero_all[query] = silent
|
||||||
print(f" SILENT ZERO (enabled, no error, no results): {silent}")
|
print(
|
||||||
|
" SILENT ZERO (enabled, no error, no results -- reported, "
|
||||||
|
f"not fatal): {silent}"
|
||||||
|
)
|
||||||
|
|
||||||
print("=" * 72)
|
print("=" * 72)
|
||||||
print("Engine contribution across all queries:")
|
print("Engine contribution across all queries:")
|
||||||
@@ -187,6 +190,11 @@ def main() -> int:
|
|||||||
print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):")
|
print("SILENT-ZERO ENGINES REPORTED (no error raised, no results returned):")
|
||||||
for query, names in silent_zero_all.items():
|
for query, names in silent_zero_all.items():
|
||||||
print(f" {query!r}: {names}")
|
print(f" {query!r}: {names}")
|
||||||
|
print(" NOTE: a silent zero is REPORTED, not counted as a failure. These")
|
||||||
|
print(" engines are expected to answer a general query, but contributing")
|
||||||
|
print(" nothing to one query can be legitimate (result de-duplication, or")
|
||||||
|
print(" an engine that only fires on certain query shapes). Only the")
|
||||||
|
print(f" <{MIN_ENGINES}-contributing-engine floor and the extraction leg fail the run.")
|
||||||
|
|
||||||
print("-" * 72)
|
print("-" * 72)
|
||||||
print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape")
|
print(f"EXTRACTION: scraping {EXTRACT_URL} via {FIRECRAWL_URL}/v1/scrape")
|
||||||
|
|||||||
@@ -17,14 +17,19 @@ description: >
|
|||||||
* reports every silent-zero engine explicitly (enabled, not in
|
* reports every silent-zero engine explicitly (enabled, not in
|
||||||
unresponsive_engines, contributed no results).
|
unresponsive_engines, contributed no results).
|
||||||
|
|
||||||
Multi-engine post-fix state (2026-09-25): bing, google cse, brave, yandex
|
Multi-engine state (2026-09-25): bing, google cse, brave and yandex
|
||||||
contribute on every query; duckduckgo is best-effort via a VPS egress
|
contribute on every query. duckduckgo is NOT working: the house egress IP
|
||||||
network and is expected to regress when DuckDuckGo flags the datacenter IP.
|
and the VPS fallback egress are both flagged by DuckDuckGo and it reports
|
||||||
|
CAPTCHA. It is left enabled as best-effort coverage so that a recovery shows
|
||||||
|
up as a contribution.
|
||||||
|
|
||||||
|
google cse is a third party's public search-engine id hardcoded in the
|
||||||
|
SearXNG build. Quota and availability are outside our control.
|
||||||
|
|
||||||
SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15,
|
SCHEDULED: /etc/cron.d/contract-runner on CT 100 (abiba), hourly at :15,
|
||||||
via scripts/contract-run.sh search-stack-visibility. Logs land in
|
via scripts/contract-run.sh search-stack-visibility. Logs land in
|
||||||
/var/log/contract-runs/. A failure also raises a firstmate inbox note.
|
/var/log/contract-runs/. A failure also raises a firstmate inbox note.
|
||||||
version: 1.0.0
|
version: 1.1.0
|
||||||
---
|
---
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
@@ -49,11 +54,10 @@ firstmate inbox note through `bin/fm-inbox.sh`.
|
|||||||
|
|
||||||
```
|
```
|
||||||
$ bash scripts/contract-run.sh search-stack-visibility
|
$ bash scripts/contract-run.sh search-stack-visibility
|
||||||
Expected engines, enabled (4): ['bing', 'brave', 'google cse', 'yandex']
|
Expected engines, enabled (5): ['bing', 'brave', 'duckduckgo', 'google cse', 'yandex']
|
||||||
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
|
queries: 'proxmox backup server' -> contributing: bing, brave, google cse, yandex
|
||||||
unresponsive: (none)
|
unresponsive: duckduckgo=CAPTCHA
|
||||||
'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
|
'python asyncio tutorial' -> contributing: bing, brave, google cse, yandex
|
||||||
unresponsive: (none)
|
|
||||||
EXTRACTION: 71016 chars of markdown returned
|
EXTRACTION: 71016 chars of markdown returned
|
||||||
VERDICT: PASS -- multiple engines contributing, extraction healthy
|
VERDICT: PASS -- multiple engines contributing, extraction healthy
|
||||||
```
|
```
|
||||||
@@ -63,8 +67,34 @@ VERDICT: PASS -- multiple engines contributing, extraction healthy
|
|||||||
* A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES`
|
* A query whose results come from fewer than `SEARCH_CHECK_MIN_ENGINES`
|
||||||
engines (default 2) fails and names the engines that did contribute.
|
engines (default 2) fails and names the engines that did contribute.
|
||||||
* An extraction request that errors or returns empty markdown fails.
|
* An extraction request that errors or returns empty markdown fails.
|
||||||
* An enabled, expected engine that contributed nothing without reporting an
|
|
||||||
error is printed under `SILENT-ZERO ENGINES REPORTED`.
|
## Silent zeros are reported, not fatal
|
||||||
|
|
||||||
|
An enabled, expected engine that contributed nothing **without reporting an
|
||||||
|
error** is printed under `SILENT-ZERO ENGINES REPORTED`, and each occurrence is
|
||||||
|
annotated `reported, not fatal`. This is deliberate:
|
||||||
|
|
||||||
|
* a general query can legitimately draw zero results from an engine that only
|
||||||
|
fires on certain query shapes, and results are de-duplicated across engines,
|
||||||
|
so a zero does not by itself prove the engine is broken;
|
||||||
|
* the run therefore fails only on the two conditions that do prove loss of
|
||||||
|
capability -- fewer than two contributing engines, and a broken extraction
|
||||||
|
leg.
|
||||||
|
|
||||||
|
A run can consequently print `VERDICT: PASS` while still listing a silent
|
||||||
|
zero. That is the intended relationship: the zero is *visible*, not *fatal*.
|
||||||
|
An engine that fails with an error (for example DuckDuckGo returning CAPTCHA)
|
||||||
|
appears in `unresponsive_engines` instead.
|
||||||
|
|
||||||
|
## Google coverage is third-party, not ours
|
||||||
|
|
||||||
|
The free Google-derived results come from the SearXNG build's built-in
|
||||||
|
`google cse` engine. It uses **a third party's public search-engine id
|
||||||
|
hardcoded in the build** (`google_cse.py`, `CX = "partner-pub-8993..."`,
|
||||||
|
blackle.com), not a key or id we own. Its quota and availability are outside
|
||||||
|
our control and it can be rate-limited or withdrawn without notice. No engine
|
||||||
|
in this build accepts our own Google Custom Search key; using our own free key
|
||||||
|
would require a small wrapper service, which is deliberately **not** built.
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -81,9 +111,14 @@ Environment overrides (see the script docstring for the full list):
|
|||||||
|
|
||||||
## Known residual risk
|
## Known residual risk
|
||||||
|
|
||||||
DuckDuckGo is reached through a forward proxy on the VPS
|
DuckDuckGo is **not** working. The house egress IP is CAPTCHA'd by
|
||||||
(`10.10.10.1:3128`, WireGuard) because the house egress IP is CAPTCHA'd. The
|
DuckDuckGo, and a forward proxy on the VPS (`10.10.10.1:3128`, WireGuard) was
|
||||||
VPS is a datacenter address and DuckDuckGo may flag it, so DuckDuckGo is
|
built as a second egress -- but DuckDuckGo has since flagged the VPS address
|
||||||
coverage, never a required engine. When it regresses it appears either in
|
too (HTTP 202 with challenge markers), so DuckDuckGo now reports CAPTCHA on
|
||||||
`unresponsive_engines` (an explicit error) or under the silent-zero report;
|
both paths. It is left enabled as best-effort coverage: if DuckDuckGo
|
||||||
neither fails the check, but both are visible.
|
unflags either address it will show up as a contribution, and until then it is
|
||||||
|
visible in `unresponsive_engines` every run. It is never a required engine.
|
||||||
|
|
||||||
|
The VPS forward proxy remains a real service (`/opt/fwd-proxy`,
|
||||||
|
`restart: unless-stopped`, healthy healthcheck, Docker enabled at boot) so the
|
||||||
|
second egress path is available for any engine that benefits from it in future.
|
||||||
|
|||||||
Reference in New Issue
Block a user