SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(disk-gc): host filesystem bands, named volumes, report-only, and state-change alerts #105

Merged
abiba-bot merged 2 commits from fix/host-filesystem-thresholds-20260915 into master 2026-09-15 14:40:22 +00:00
Conversation 0 Commits 2 Files Changed 1
+44 -1
2 Commits
Author SHA1 Message Date
root b9b1712ac6 fix: make host escalations state-change driven
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
Details
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s
Details
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 4s
Details
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 4s
Details
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
Details 
A volume alerts ONCE when it enters a higher band (GREEN->WARN, WARN->AMBER,
AMBER->RED) and ONCE when it drops back down (recovery notice). While it stays
in the same band, it is reported in the scan output only — no DM, no channel
alert. This stops the same 96% easystore2 from re-DMing the owner on every 6h
scan.

State lives in a small JSON state file (state/host-disk-bands.json), keyed by
host/volume -> last-seen band. The scanner reads the prior band, compares to the
current band, and DMs only on a transition; the state file is written after
every scan. Chosen over a periodic digest because the scan already runs every
6h and a transition is genuinely new, actionable state.

First-run behavior: when the state file does not yet exist, the current band of
every volume is recorded as baseline WITHOUT alerting — a first run would
otherwise DM every already-elevated volume at once.

Report-only restriction and volume-naming output kept exactly as-is.
 2026-09-15 14:25:49 +00:00
root 6fb411613e fix: add host filesystem thresholds to disk-gc contract 
Add separate threat bands for HOST filesystems (distinct from guest bands):
- HOST-WARN at 85%: name volume + % + absolute free space in scan output
- HOST-AMBER at 90%: flag for owner attention, Zulip DM
- HOST-RED at 95%: flag for immediate owner attention, Zulip DM + channel alert

Volume naming rule: every host line MUST name the volume and what lives on it.
Action classes by volume type:
- host-root: near full = real risk (backup staging, thin-pool metadata)
- media (/media/*): near full = capacity decision for owner, never auto-delete
- pbs-datastore (tank): near full = breaks Proxmox Backup Server

Report-only restriction: no automatic deletion of media or datastore content ever.

Justification (measured 2026-09-15): storepve /media/easystore2 at 96% was
reported but never banded or acted on. Two incidents this weekend showed the
host filesystem is the thing that breaks, not the guest's.

Added HOST-WARN/AMBER/RED alert templates.
Added report-only execution rule for host filesystems.
 2026-09-15 14:15:53 +00:00