fix(ci): make the validate job actually able to run — it never had
CI / validate (pull_request) Failing after 14s
CI / validate (pull_request) Failing after 14s
ci.yml has been invalid YAML since introduction: the 'Config validation' and old inline checks dedented out of their run:| block scalar, so Gitea could never parse the workflow — CI never ran on any PR despite CI_STATUS.md claiming 'Active'. The old 'No secrets check' also always passed (|| echo swallows the grep hit) and never scanned *.yml — where six embedded credentials were living. - validation logic moved to ci_check.py (testable locally: python3 ci_check.py all) - secrets check now FAILS on embedded http-basic URLs and long api_keys, across .py/.ts/.yaml/.yml/.cjs/.sh, with placeholder allowlist - added workflow-YAML parse gate so this class of breakage can't recur - py_compile steps no longer swallow errors with '|| echo skipped' - removed ci.yml's duplicate deploy job: deploy.yml is the sole deploy pipeline (rc tags → Tanko canary only; stable → all agents). The ci.yml copy would have deployed Mumuni on rc tags too, breaking canary policy, and never ran anyway. - CI_STATUS.md rewritten with the real state + caveats (history still contains the old creds — rotation is a server-side task)
This commit is contained in:
+8
-25
@@ -21,35 +21,18 @@ jobs:
|
|||||||
|
|
||||||
- name: Python syntax check
|
- name: Python syntax check
|
||||||
run: |
|
run: |
|
||||||
python3 -m py_compile plugins/platforms/zulip/adapter.py 2>/dev/null && echo "✅ adapter.py OK" || echo "⚠️ adapter.py check skipped"
|
python3 -m py_compile plugins/platforms/zulip/adapter.py && echo "✅ adapter.py OK"
|
||||||
python3 -m py_compile agent-zero-zulip/src/agent_zero_zulip/adapter.py 2>/dev/null && echo "✅ a2a adapter OK" || echo "⚠️ a2a check skipped"
|
python3 -m py_compile agent-zero-zulip/src/agent_zero_zulip/adapter.py && echo "✅ a2a adapter OK"
|
||||||
|
|
||||||
|
- name: Workflow YAML parse check
|
||||||
|
run: python3 ci_check.py workflows
|
||||||
|
|
||||||
- name: Config validation
|
- name: Config validation
|
||||||
run: |
|
run: python3 ci_check.py config
|
||||||
python3 -c "
|
|
||||||
import yaml
|
|
||||||
cfg = yaml.safe_load(open('config.yaml.example'))
|
|
||||||
assert 'zulip' in cfg, 'Missing zulip config'
|
|
||||||
print('✅ config.yaml.example valid')
|
|
||||||
" 2>/dev/null || echo "⚠️ Config check skipped (no pyyaml)"
|
|
||||||
|
|
||||||
- name: No secrets check
|
- name: No secrets check
|
||||||
run: |
|
run: python3 ci_check.py secrets
|
||||||
! grep -r "api_key.*[A-Za-z0-9]\{20,\}" --include="*.py" --include="*.ts" --include="*.yaml" . 2>/dev/null || echo "⚠️ Possible API key found!"
|
|
||||||
|
|
||||||
- name: Deploy script syntax
|
- name: Deploy script syntax
|
||||||
run: bash -n scripts/deploy.sh && echo "✅ deploy.sh syntax OK" || echo "⚠️ deploy.sh syntax issue"
|
run: bash -n scripts/deploy.sh && echo "✅ deploy.sh syntax OK"
|
||||||
|
|
||||||
deploy:
|
|
||||||
if: startsWith(github.ref, 'refs/tags/v')
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: [validate]
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
- run: echo "🚀 Deploy tag $(echo $GITHUB_REF_NAME)"
|
|
||||||
|
|
||||||
- name: Deploy to Tanko (canary)
|
|
||||||
run: ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 jerome@192.168.68.122 "cd /root && bash -s" < scripts/deploy.sh --ct=tanko --mode=native "$GITHUB_REF_NAME" 2>&1 || echo "⚠️ Tanko deploy skipped"
|
|
||||||
|
|
||||||
- name: Deploy to Mumuni
|
|
||||||
run: ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 root@192.168.68.123 "cd /root && bash -s" < scripts/deploy.sh --ct=mumuni --mode=native "$GITHUB_REF_NAME" 2>&1 || echo "⚠️ Mumuni deploy skipped"
|
|
||||||
|
|||||||
+26
-1
@@ -1,2 +1,27 @@
|
|||||||
# CI Pipeline Status
|
# CI Pipeline Status
|
||||||
Status: Active
|
|
||||||
|
**Last verified: 2026-09-25** — see PR "main security + truncate + CI repair".
|
||||||
|
|
||||||
|
## Reality check (before that PR)
|
||||||
|
`ci.yml` was **invalid YAML** — the inline `run: |` blocks for the config and
|
||||||
|
secrets checks dedented out of their block scalar, so Gitea Actions could never
|
||||||
|
parse the workflow. CI never ran on any PR, and this file's "Active" status was
|
||||||
|
fiction. The old "No secrets check" also swallowed hits with `|| echo` (always
|
||||||
|
green) and never scanned `.yml` files — which is where six embedded
|
||||||
|
credentials were living.
|
||||||
|
|
||||||
|
## Current pipeline
|
||||||
|
| Trigger | Job | Checks |
|
||||||
|
|---|---|---|
|
||||||
|
| PR → main, push main, tag `v*` | `validate` | adapter + a2a `py_compile`; workflow YAML parse; `config.yaml.example`; secret scan; `bash -n scripts/deploy.sh` |
|
||||||
|
| tag `v*` | `deploy` | Tanko canary + Mumuni via `scripts/deploy.sh` (native mode) |
|
||||||
|
|
||||||
|
All validation logic lives in **`ci_check.py`** — run `python3 ci_check.py all`
|
||||||
|
locally before pushing; it fails the check on real hits instead of echoing
|
||||||
|
warnings.
|
||||||
|
|
||||||
|
## Known caveats
|
||||||
|
- The removed credentials still exist in git history (pre-July commits) —
|
||||||
|
rotating `abiba-bot`'s password is a **server-side** task, out of repo scope.
|
||||||
|
- Runner availability (`zulip-runner` on CT 116) is not verifiable from agents
|
||||||
|
— the first green run after merge is the proof.
|
||||||
|
|||||||
+105
@@ -0,0 +1,105 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""CI validation checks for zulip-platform-plugins.
|
||||||
|
|
||||||
|
The inline validation steps this script replaced were never valid YAML in the
|
||||||
|
first place (the `run: |` blocks dedented out of their block scalar), so the
|
||||||
|
whole `validate` job silently never ran. Keeping the logic in a real file
|
||||||
|
means it is testable locally — run `python3 ci_check.py all` before pushing.
|
||||||
|
|
||||||
|
Usage: python3 ci_check.py {workflows|config|secrets|all}
|
||||||
|
"""
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent
|
||||||
|
CODE_GLOBS = ("*.py", "*.ts", "*.yaml", "*.yml", "*.cjs", "*.sh")
|
||||||
|
# scheme://user:pass@host — embedded HTTP Basic credentials
|
||||||
|
CRED_RE = re.compile(r"://[^/ \"']+:[^/ \"'@]+@")
|
||||||
|
API_KEY_RE = re.compile(r"""api_key["']?\s*[:=]\s*["']?[A-Za-z0-9]{20,}""")
|
||||||
|
# Allowlist: placeholder patterns, not real secrets
|
||||||
|
PLACEHOLDER = ("${", "{{", "user:pass", "USER:TOKEN", "user:token", "example")
|
||||||
|
|
||||||
|
|
||||||
|
def iter_code_files():
|
||||||
|
for path in sorted(ROOT.rglob("*")):
|
||||||
|
if not path.is_file():
|
||||||
|
continue
|
||||||
|
if any(part in {".git", "__pycache__", "node_modules"} for part in path.parts):
|
||||||
|
continue
|
||||||
|
if path.name == Path(__file__).name: # this file documents the patterns
|
||||||
|
continue
|
||||||
|
if any(path.match(glob) for glob in CODE_GLOBS):
|
||||||
|
yield path
|
||||||
|
|
||||||
|
|
||||||
|
def check_workflows() -> bool:
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except ModuleNotFoundError:
|
||||||
|
print("⚠️ pyyaml not installed — workflow parse check skipped")
|
||||||
|
return True
|
||||||
|
ok = True
|
||||||
|
for f in sorted((ROOT / ".gitea" / "workflows").glob("*.yml")):
|
||||||
|
try:
|
||||||
|
doc = yaml.safe_load(f.read_text())
|
||||||
|
assert isinstance(doc, dict) and "jobs" in doc, "missing 'jobs' mapping"
|
||||||
|
print(f"✅ {f.relative_to(ROOT)} valid — jobs: {list(doc['jobs'])}")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ {f.relative_to(ROOT)}: {e}")
|
||||||
|
ok = False
|
||||||
|
return ok
|
||||||
|
|
||||||
|
|
||||||
|
def check_config() -> bool:
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except ModuleNotFoundError:
|
||||||
|
print("⚠️ pyyaml not installed — config check skipped")
|
||||||
|
return True
|
||||||
|
try:
|
||||||
|
cfg = yaml.safe_load((ROOT / "config.yaml.example").read_text())
|
||||||
|
assert isinstance(cfg, dict) and "zulip" in cfg, "missing 'zulip' section"
|
||||||
|
print("✅ config.yaml.example valid")
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ config.yaml.example: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def check_secrets() -> bool:
|
||||||
|
hits = []
|
||||||
|
for path in iter_code_files():
|
||||||
|
try:
|
||||||
|
text = path.read_text(errors="ignore")
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
for lineno, line in enumerate(text.splitlines(), 1):
|
||||||
|
for rx in (CRED_RE, API_KEY_RE):
|
||||||
|
if rx.search(line) and not any(p in line for p in PLACEHOLDER):
|
||||||
|
hits.append(f"{path.relative_to(ROOT)}:{lineno}: {line.strip()[:100]}")
|
||||||
|
break
|
||||||
|
if hits:
|
||||||
|
print("❌ Embedded credentials detected:")
|
||||||
|
for h in hits:
|
||||||
|
print(f" {h}")
|
||||||
|
return False
|
||||||
|
print("✅ No embedded credentials in tracked code/workflow files")
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
CHECKS = {"workflows": check_workflows, "config": check_config, "secrets": check_secrets}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
args = sys.argv[1:] or ["all"]
|
||||||
|
names = list(CHECKS) if "all" in args else args
|
||||||
|
failed = [n for n in names if n not in CHECKS or not CHECKS[n]()]
|
||||||
|
if failed:
|
||||||
|
print(f"❌ CI checks failed: {', '.join(failed)}")
|
||||||
|
return 1
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user