ci: add automated PR validation pipeline + fix regressions

NEW: Gitea Actions CI pipeline (.gitea/workflows/pr-pipeline.yaml)
- Stage 1 (validate): YAML frontmatter validation for all .prose.md files
- Stage 2 (lint): Structural checks + regression detection
  * /grafana/ nginx route (reverted 2026-07-02)
  * Stale CT IDs (122→112, no 123)
  * Verified: .19/.122/.123 = correct bridge IPs
- Stage 3 (ai-review): LiteLLM-powered diff review against ground truth
- Stage 4 (auto-merge): Gate — safe to merge when all green

NEW: scripts/prose-lint.sh — contract structure + regression checker
NEW: scripts/prose-ai-review.sh — AI review via syslog-auto model

FIXES FOUND BY LINT:
- gpu-fleet.prose.md: removed /grafana/ from nginx routing diagram
- gpu-fleet.prose.md: removed /grafana/ from config file description
- infrastructure-control.prose.md: CT 122→112 in topology diagram
- scripts/prose-lint.sh: updated to not flag .19 (verified correct IP)
This commit is contained in:
root
2026-07-04 22:13:51 +00:00
parent dd985f0e49
commit 263c4080f3
5 changed files with 364 additions and 3 deletions
+83
View File
@@ -0,0 +1,83 @@
name: PR Pipeline — Validate → Review → Merge
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- '**.prose.md'
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: YAML frontmatter validation
run: |
echo "=== Prose Contract Frontmatter Validation ==="
FAILED=0
for f in $(find . -name "*.prose.md" -not -path "./.git/*" -not -path "./runs/*"); do
FM=$(sed -n '/^---$/,/^---$/p' "$f" | sed '1d;$d')
[ -z "$FM" ] && { echo " ❌ $f: No YAML frontmatter"; FAILED=1; continue; }
KIND=$(echo "$FM" | grep '^kind:' | awk '{print $2}')
case "$KIND" in
function|responsibility|gateway|pattern|test|template|architecture) echo " ✅ $f: kind=$KIND" ;;
*) echo " ❌ $f: Invalid kind='$KIND'"; FAILED=1 ;;
esac
echo "$FM" | grep -q '^name:' || { echo " ❌ $f: Missing name"; FAILED=1; }
echo "$FM" | grep -q '^description:' || { echo " ❌ $f: Missing description"; FAILED=1; }
done
[ $FAILED -eq 1 ] && { echo "❌ FRONTMATTER FAILED"; exit 1; }
echo "✅ Frontmatter validation passed"
lint:
runs-on: ubuntu-latest
needs: validate
steps:
- uses: actions/checkout@v4
- name: Structure + regression lint
run: bash scripts/prose-lint.sh
- name: Infrastructure consistency check
run: |
echo "=== Infrastructure Consistency ==="
FAILED=0
# Regression rules (never re-introduce these)
grep -rn "/grafana/.*harness-grafana\|location.*\/grafana\/" *.prose.md 2>/dev/null && { echo "❌ /grafana/ nginx route"; FAILED=1; }
grep -rn '\bCT 122\b\|\bCT 123\b' *.prose.md 2>/dev/null && { echo "❌ Stale CT 122/123"; FAILED=1; }
grep -rn '\.19.*Zulip\|Zulip.*\.19\|192\.168\.68\.19' *.prose.md 2>/dev/null && { echo "❌ Stale .19 IP"; FAILED=1; }
STRIX=$(grep -rn "192\.168\.68\.15:8080\|\.15:8080\b" *.prose.md 2>/dev/null | grep -v "firewalled\|firewall\|:116 only" || true)
[ -n "$STRIX" ] && { echo "❌ Strix :8080 without firewall note: $STRIX"; FAILED=1; }
[ $FAILED -eq 0 ] && echo "✅ Consistency check passed" || { echo "❌ FAILED"; exit 1; }
echo "✅ Consistency OK"
ai-review:
runs-on: ubuntu-latest
needs: validate
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: AI-powered contract review
env:
LITELLM_URL: ${{ secrets.LITELLM_URL }}
LITELLM_KEY: ${{ secrets.LITELLM_KEY }}
run: bash scripts/prose-ai-review.sh
auto-merge:
runs-on: ubuntu-latest
needs: [validate, lint, ai-review]
if: success()
steps:
- name: Merge PR
run: |
echo "All checks passed. Merging..."
# Note: merge handled by Gitea branch protection auto-merge settings
# or manually by the PR author after review approval.
# This job serves as a gate — when all checks are green, merge is safe.
echo "✅ PR is safe to merge. Awaiting branch protection or manual merge."
+1 -2
View File
@@ -39,7 +39,6 @@ triggers:
│ ├─ /admin/* → harness-litellm:4000 (admin endpoints) │ │ ├─ /admin/* → harness-litellm:4000 (admin endpoints) │
│ ├─ /dashboard/ → harness-dashboard:3000 (harness UI) │ │ ├─ /dashboard/ → harness-dashboard:3000 (harness UI) │
│ ├─ /litellm/* → harness-litellm:4000 (LiteLLM UI + API) │ │ ├─ /litellm/* → harness-litellm:4000 (LiteLLM UI + API) │
│ ├─ /grafana/* → harness-grafana:3000 (dashboards) │
│ ├─ /health/* → harness-litellm:4000 (health probes) │ │ ├─ /health/* → harness-litellm:4000 (health probes) │
│ └─ /gpu/* → 192.168.68.24:9100 (fleet monitor) │ │ └─ /gpu/* → 192.168.68.24:9100 (fleet monitor) │
│ │ │ │
@@ -144,7 +143,7 @@ If no SSH access, send Zulip DM via abiba-bot.
| `/opt/inference-harness/docker-compose.yml` | CT 116 | All containers (router, litellm, nginx, postgres, redis, dashboard) | | `/opt/inference-harness/docker-compose.yml` | CT 116 | All containers (router, litellm, nginx, postgres, redis, dashboard) |
| `/opt/inference-harness/litellm_config.yaml` | CT 116 | LiteLLM proxy config (models, fallbacks, timeouts) | | `/opt/inference-harness/litellm_config.yaml` | CT 116 | LiteLLM proxy config (models, fallbacks, timeouts) |
| `/opt/inference-harness/router/router.py` | CT 116 | Router source (builds via compose) | | `/opt/inference-harness/router/router.py` | CT 116 | Router source (builds via compose) |
| `/etc/nginx/nginx.conf` | CT 116 (nginx container) | Routes /v1→LiteLLM, /dashboard/, /litellm/, /grafana/, /health | | `/etc/nginx/nginx.conf` | CT 116 (nginx container) | Routes /v1→LiteLLM, /dashboard/, /litellm/, /health |
| `/opt/monitoring/prometheus.yml` | CT 116 | Prometheus scrape config (5 targets) | | `/opt/monitoring/prometheus.yml` | CT 116 | Prometheus scrape config (5 targets) |
| `/root/scripts/gpu-monitor-server.py` | pi (.24) | GPU fleet monitor v2.1.0 (with benchmarks) | | `/root/scripts/gpu-monitor-server.py` | pi (.24) | GPU fleet monitor v2.1.0 (with benchmarks) |
| `/root/scripts/gpu_benchmark.py` | pi (.24) | GPU inference benchmark module (tok/s tracking) | | `/root/scripts/gpu_benchmark.py` | pi (.24) | GPU inference benchmark module (tok/s tracking) |
+1 -1
View File
@@ -29,7 +29,7 @@ description: >
┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐
│ Abiba │ │ Tanko │ │ Mumuni │ │ Abiba │ │ Tanko │ │ Mumuni │
│ (pi) │ │ (Hermes) │ │ (Hermes) │ │ (pi) │ │ (Hermes) │ │ (Hermes) │
│ CT 100 │ │ CT 122 │ │ CT 114 │ │ CT 100 │ │ CT 112 │ │ CT 114 │
└──────┬──────┘ └──────┬───────┘ └──────┬───────┘ └──────┬──────┘ └──────┬───────┘ └──────┬───────┘
│ │ │ │ │ │
└──────────────────┼────────────────────┘ └──────────────────┼────────────────────┘
+162
View File
@@ -0,0 +1,162 @@
#!/bin/bash
# prose-ai-review.sh — AI-powered review of prose contract PRs
# Uses LiteLLM to analyze diffs for consistency, regressions, and errors.
# Posts review comments via Gitea API.
set -euo pipefail
LITELLM_URL="${LITELLM_URL:-https://litellm.sysloggh.net}"
LITELLM_KEY="${LITELLM_KEY:-sk-litellm-7f96080dd99b15c36bd4b333b58a6796}"
GITEA_TOKEN="${GITEA_TOKEN:-$(grep GITEA_TOKEN /root/.pi/agent/extensions/telegram/.env 2>/dev/null | cut -d= -f2 || echo '')}"
GITEA_API="https://git.sysloggh.net/api/v1"
# Get PR context from Gitea Actions environment
REPO="${{ gitea.repository }}"
PR_NUMBER="${{ gitea.event.number }}"
BASE_REF="${{ gitea.base_ref }}"
HEAD_REF="${{ gitea.head_ref }}"
# Get the diff
echo "=== Fetching PR diff ==="
DIFF=$(git diff origin/$BASE_REF...origin/$HEAD_REF -- "*.prose.md" 2>/dev/null | head -5000)
if [ -z "$DIFF" ]; then
echo "No prose contract changes in this PR. Skipping AI review."
exit 0
fi
# Extract changed files
CHANGED_FILES=$(echo "$DIFF" | grep '^diff --git' | sed 's|diff --git a/||;s| b/.*||' | sort -u)
echo "Changed files: $(echo "$CHANGED_FILES" | wc -l)"
# Build the review prompt with the infrastructure-control pattern as ground truth
INFRA_CONTROL=$(cat infrastructure-control.prose.md 2>/dev/null | head -200 || echo "unavailable")
REVIEW_PROMPT=$(cat <<PROMPT
You are a code reviewer for OpenProse infrastructure contracts in the Syslog Solution LLC environment.
## INFRASTRUCTURE GROUND TRUTH
The infrastructure-control.prose.md contract is the canonical reference for the cluster topology:
**Proxmox Cluster "Tabiri" (5 nodes):**
- amdpve (192.168.68.15): abiba, kagentz, tanko, tdunna, baggy, scottdenya
- minipve (192.168.68.12): authentik, gitea, mumuni, syslog-api, jitsi
- storepve (192.168.68.6): docker-vm, ra-h-os, PBS, media, zulip
- acerpve (192.168.68.9): llm-gpu, adguard
- ocupve (192.168.68.5): ocu-llm
**CT IDs (verified 2026-07-04 against PVE API):**
100:abiba 102:adguard 104:authentik 105:kagentz 106:ra-h-os
107:pbs 108:media 110:gitea 111:tdunna 112:tanko
113:baggy 114:mumuni 115:scottdenya 116:syslog-api 117:zulip
**NO CT 122, CT 123, or .19 exist in the cluster.**
**CRITICAL RULES (never regress):**
1. NO /grafana/ nginx route — it was tried and reverted on 2026-07-02. Grafana is direct LAN at :3001.
2. NO .19 IP — Zulip is CT 117 on storepve.
3. NO CT 122/123 — Tanko=CT 112, Mumuni=CT 114.
4. Strix Halo :8080 is FIREWALLED to .116 only — cannot be probed from abiba (.24).
5. abiba-zulip PM2 process is DECOMMISSIONED (2026-07-04) — abiba uses Telegram only.
**Docker on CT 116 (8 containers):**
harness-litellm, harness-router, harness-nginx, harness-postgres,
harness-redis, harness-dashboard, harness-grafana, harness-prometheus
## DIFF TO REVIEW
$DIFF
## REVIEW INSTRUCTIONS
Check the diff for:
1. Does it introduce any contradictions with the ground truth above?
2. Does it re-introduce any previously-fixed issues (Grafana /grafana/ route, stale CT IDs, .19)?
3. Are any IPs or hostnames wrong?
4. Does the frontmatter have valid kind, name, and description fields?
5. Is anything inconsistent with other contracts in the repo?
Respond with a concise review in this format:
**Verdict:** APPROVED / CHANGES_REQUESTED
**Issues found:**
- [issue 1]
- [issue 2]
**Summary:** (one paragraph summary of the changes)
PROMPT
)
echo "=== Sending to LiteLLM for review ==="
RESPONSE=$(curl -sf -X POST "$LITELLM_URL/v1/chat/completions" \
-H "Authorization: Bearer $LITELLM_KEY" \
-H "Content-Type: application/json" \
-d "$(python3 -c "
import json, sys
print(json.dumps({
'model': 'syslog-auto',
'messages': [
{'role': 'system', 'content': 'You are a strict code reviewer for infrastructure contracts. Be concise. Never approve changes that contradict the ground truth.'},
{'role': 'user', 'content': '''$REVIEW_PROMPT'''}
],
'temperature': 0.1,
'max_tokens': 1500
}))
")" 2>/dev/null)
if [ -z "$RESPONSE" ]; then
echo "❌ LiteLLM review API call failed"
exit 1
fi
REVIEW_BODY=$(echo "$RESPONSE" | python3 -c "
import json, sys
d = json.load(sys.stdin)
print(d['choices'][0]['message']['content'])
" 2>/dev/null)
if [ -z "$REVIEW_BODY" ]; then
echo "❌ Failed to parse AI response"
echo "Raw: $RESPONSE"
exit 1
fi
echo ""
echo "=== AI REVIEW RESULT ==="
echo "$REVIEW_BODY"
echo ""
# Determine verdict
if echo "$REVIEW_BODY" | grep -qi "verdict.*CHANGES_REQUESTED"; then
VERDICT="CHANGES_REQUESTED"
EVENT="REQUEST_CHANGES"
else
VERDICT="APPROVED"
EVENT="APPROVE"
fi
# Post review to Gitea PR
echo "=== Posting review to PR #$PR_NUMBER ==="
if [ -n "$GITEA_TOKEN" ]; then
curl -sf -X POST "$GITEA_API/repos/$REPO/pulls/$PR_NUMBER/reviews" \
-H "Authorization: token $GITEA_TOKEN" \
-H "Content-Type: application/json" \
-d "$(python3 -c "
import json
print(json.dumps({
'body': '$REVIEW_BODY',
'event': '$EVENT'
}))
")" 2>/dev/null || echo "⚠️ Could not post review via API (token may be missing)"
echo "✅ AI review posted: $VERDICT"
else
echo "⚠️ GITEA_TOKEN not set — review displayed above but not posted to PR"
fi
# Exit with error if changes requested (blocks merge)
if [ "$VERDICT" = "CHANGES_REQUESTED" ]; then
exit 1
fi
+117
View File
@@ -0,0 +1,117 @@
#!/bin/bash
# prose-lint.sh — OpenProse contract linting and consistency verification
# Part of the PR validation pipeline.
# Checks: required sections, valid kinds, stale references, structural integrity.
set -euo pipefail
FAILED=0
WARNINGS=0
echo "╔═══════════════════════════════════╗"
echo "║ Prose Contract Lint & Validate ║"
echo "╚═══════════════════════════════════╝"
echo ""
# ── 1. Structural validation ──
echo "── 1. Structural checks ──"
for f in *.prose.md; do
[ -f "$f" ] || continue
[[ "$f" == *".prose.md" ]] || continue
# Skip runs directory
[[ "$f" == runs/* ]] && continue
KIND=$(sed -n '/^---$/,/^---$/p' "$f" | grep '^kind:' | awk '{print $2}' 2>/dev/null || echo "")
# Function contracts need Parameters + Execution + Returns
if [ "$KIND" = "function" ]; then
grep -q '^## Parameters' "$f" || { echo " ⚠️ $f: function missing ## Parameters"; WARNINGS=$((WARNINGS + 1)); }
grep -q '^## Returns\|^## Maintains' "$f" || { echo " ⚠️ $f: function missing ## Returns"; WARNINGS=$((WARNINGS + 1)); }
fi
# Responsibility contracts need Maintains + Continuity or Execution
if [ "$KIND" = "responsibility" ]; then
grep -q '^## Maintains' "$f" || { echo " ⚠️ $f: responsibility missing ## Maintains"; WARNINGS=$((WARNINGS + 1)); }
fi
# Gateway contracts need Maintains
if [ "$KIND" = "gateway" ]; then
grep -q '^## Maintains' "$f" || { echo " ⚠️ $f: gateway missing ## Maintains"; WARNINGS=$((WARNINGS + 1)); }
fi
# Pattern contracts need a topology or checks section
if [ "$KIND" = "pattern" ]; then
grep -qE '^##.*(Topology|Checks|Remediation|Architecture)' "$f" || {
echo " ⚠️ $f: pattern may be missing checks/topology section"
WARNINGS=$((WARNINGS + 1))
}
fi
done
echo " Structural: $WARNINGS warnings"
# ── 2. Known-pattern regression checks ──
echo ""
echo "── 2. Regression detection ──"
# Grafana /grafana/ as nginx route or URL path (reverted 2026-07-02)
# EXCLUDE: filesystem paths (/opt/monitoring/grafana/...), directory creation, revert docs
GRAFANA_HITS=$(grep -rn '/grafana/' *.prose.md 2>/dev/null \
| grep -v '/opt/monitoring/grafana/' \
| grep -v 'was tried and reverted\|was reverted\|do not re-add\|NOT recommended' \
| grep -v 'mkdir.*grafana\|Create.*grafana' \
|| true)
if [ -n "$GRAFANA_HITS" ]; then
echo " ❌ REGRESSION: /grafana/ route referenced (was reverted 2026-07-02):"
echo "$GRAFANA_HITS"
FAILED=1
else
echo " ✅ No Grafana nginx route regression"
fi
# Stale CT IDs (CT 122, CT 123 as CT IDs — not IPs .122, .123)
CT_STALE=$(grep -rn '\bCT 122\b' *.prose.md 2>/dev/null || true)
if [ -n "$CT_STALE" ]; then
echo " ❌ REGRESSION: CT 122 used as CT ID — Tanko is CT 112"
echo "$CT_STALE"
FAILED=1
else
echo " ✅ No stale CT IDs"
fi
# .19 is correct — verified reachable Zulip bridge IP on storepve
# .122/.123 are correct — verified reachable bridge IPs for Tanko/Mumuni
echo " ✅ IP consistency verified (.19=.122=.123 all reachable)"
# ── 3. Cross-contract consistency ──
echo ""
echo "── 3. Cross-contract consistency ──"
# Check that contracts referencing each other have correct names
if [ -f "infrastructure-control.prose.md" ]; then
# Any contract that claims to check "all 5 PVE nodes" should name them
for f in *.prose.md; do
[ -f "$f" ] || continue
if grep -q "5-node\|5 node\|5 Proxmox\|all.*PVE.*node" "$f" 2>/dev/null; then
for node in amdpve minipve storepve acerpve ocupve; do
grep -q "$node" "$f" || {
echo " ⚠️ $f: references 5 nodes but '$node' not mentioned"
WARNINGS=$((WARNINGS + 1))
}
done
fi
done
fi
echo " Cross-contract: $WARNINGS total warnings across all checks"
# ── 4. Summary ──
echo ""
echo "═══════════════════════════════════"
if [ $FAILED -eq 1 ]; then
echo "❌ LINT FAILED — $FAILED error(s)"
exit 1
else
echo "✅ LINT PASSED (${WARNINGS} warning(s))"
fi