Compare commits

...
Author SHA1 Message Date
root 27bc6bafba Fix zulip-monitor.sh LOG variable scope bug
Move LOG, TIMESTAMP, and ISSUES declarations from inside notify() to
script scope so they are available throughout the script.
2026-08-29 22:09:46 +00:00
root 97dd261123 Fix zulip-monitor.sh email spam: Replace Gmail SMTP with Zulip stream post to #agent-hub on topic 'zulip-health' 2026-08-29 21:30:54 +00:00
2 changed files with 12 additions and 26 deletions
+11 -26
View File
@@ -9,10 +9,11 @@ ZULIP_EMAIL="abiba-bot@chat.sysloggh.net"
ZULIP_KEY="cKTDMZAPW08dk3zl05sStzO7HRztzyn8"
OWNER_ZULIP_ID="9"
# Email config
GMAIL_USER="jtabiri@gmail.com"
GMAIL_PASS="rgbuomwcydxwbszd"
EMAIL_TO="jerome@sysloggh.com"
LOG="/root/zulip-health-monitor.log"
TIMESTAMP=$(date -u '+%Y-%m-%d %H:%M UTC')
ISSUES=0
echo "=== Zulip Health Check — $TIMESTAMP ===" >> "$LOG"
notify() {
local severity="$1" msg="$2"
@@ -24,30 +25,14 @@ notify() {
curl -sf -X POST "${ZULIP_SITE}/api/v1/messages" \
-u "${ZULIP_EMAIL}:${ZULIP_KEY}" \
-d "${form}" > /dev/null 2>&1 || true
# Email alert
local subject="${severity} Zulip Monitor Alert"
python3 -c "
import smtplib
from email.mime.text import MIMEText
m = MIMEText('''${msg}''')
m['From'] = 'abiba@sysloggh.com'
m['To'] = '${EMAIL_TO}'
m['Subject'] = '${subject}'
s = smtplib.SMTP('smtp.gmail.com', 587)
s.starttls()
s.login('${GMAIL_USER}', '${GMAIL_PASS}')
s.sendmail('abiba@sysloggh.com', ['${EMAIL_TO}'], m.as_string())
s.quit()
" 2>/dev/null || true
# Zulip stream post to #agent-hub on topic 'zulip-health'
local stream_content="${severity} Zulip Monitor: ${msg}"
curl -sf -X POST "${ZULIP_SITE}/api/v1/messages" \
-u "${ZULIP_EMAIL}:${ZULIP_KEY}" \
-d "type=stream\&to=%5B7%5D\&topic=zulip-health\&content=$(python3 -c "import urllib.parse; print(urllib.parse.quote(str()))")" \
> /dev/null 2>&1 || true
}
TIMESTAMP=$(date -u '+%Y-%m-%d %H:%M UTC')
ISSUES=0
LOG="/root/zulip-health-monitor.log"
echo "=== Zulip Health Check — $TIMESTAMP ===" >> "$LOG"
# ── Global: Zulip Server ──
SERVER_CODE=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 10 \
https://chat.sysloggh.net/api/v1/server_settings \
+1
View File
@@ -324,6 +324,7 @@ Track via `/tmp/zulip-monitor-debounce` (unix timestamp of last restart).
- **Persistent failure threshold**: Only restart after 3 consecutive failed checks (45 min) AND no self-healing in progress
- **Debounce gate**: Skip all checks entirely if recently restarted
4. **Zulip stream delivery** — Alerts posted to #agent-hub on topic 'zulip-health' (stream_id 7) instead of email, keeping DM to captain for critical (🔴) alerts
### Gen 4 (2026-06-29) — Response Pipeline Fix
**Root Cause**: LLM hangs due to GPU saturation (503 QUEUE_TIMEOUT) → pi's `agent_end` never fires → pending Zulip replies accumulate with no timeout. Health endpoint showed `connected: true, stuck: false` while user experienced complete silence.