@techwavedev/agi-agent-kit 1.5.3 → 1.6.1

package/templates/base/tests/agents/scenario_02_parallel_subagents.md

@@ -0,0 +1,70 @@
+# Scenario 02: Parallel Sub-Agents on Independent Domains
+
+**Pattern:** `dispatching-parallel-agents`  
+**Team used:** `documentation_team` (dispatched twice, different domains)  
+**Sub-agents:** Two independent `doc-writer` instances
+
+---
+
+## Goal
+
+Validate that the framework supports dispatching multiple agents on independent problem domains simultaneously — each with its own context, run ID, and memory entry.
+
+---
+
+## What This Tests
+
+| Test Point | Description |
+|---|---|
+| Independent dispatch | Two dispatches of `documentation_team` with different payloads |
+| Unique run IDs | Each dispatch gets a unique `run_id` (no collision) |
+| Context isolation | Each manifest carries only its own payload |
+| No shared state conflict | Both can coexist without interfering |
+
+---
+
+## Run
+
+```bash
+python3 execution/run_test_scenario.py --scenario 2
+```
+
+---
+
+## Expected Output
+
+```json
+{
+  "scenario": "scenario_02_parallel_subagents",
+  "pattern": "dispatching-parallel-agents",
+  "status": "pass",
+  "steps": [
+    { "step": "dispatch domain_1", "exit_code": 0 },
+    { "step": "dispatch domain_2", "exit_code": 0 },
+    { "step": "validate parallel dispatches", "run_ids_unique": true }
+  ]
+}
+```
+
+---
+
+## Manual Walkthrough (for human-in-the-loop testing)
+
+1. Identify two independent code changes (e.g., `session_boot.py` and `memory_manager.py`)
+2. Dispatch two documentation teams concurrently:
+
+```bash
+# Domain 1
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{"changed_files": ["execution/session_boot.py"], "commit_msg": "feat: add auto-fix", "change_type": "feat"}'
+
+# Domain 2 (independent)
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{"changed_files": ["execution/memory_manager.py"], "commit_msg": "fix: empty result", "change_type": "fix"}'
+```
+
+3. Verify both manifests have unique `run_id` values
+4. Confirm neither manifest contains the other's `changed_files`
+5. Invoke both doc-writer sub-agents (they work on different files — no conflict)

package/templates/base/tests/agents/scenario_03_doc_team_on_code.md

@@ -0,0 +1,107 @@
+# Scenario 03: Documentation Team Triggered on Code Change
+
+**Pattern:** `doc-team-on-code` ← **Primary use case**  
+**Team used:** `documentation_team`  
+**Sub-agents:** `doc-writer` → `doc-reviewer` → `changelog-updater`
+
+---
+
+## Goal
+
+Validate the core pattern: every coding action that changes `execution/`, `skills/`, `templates/`, or `directives/` must automatically trigger the documentation team. This ensures docs never go stale.
+
+---
+
+## What This Tests
+
+| Test Point | Description |
+|---|---|
+| Team trigger on code change | `documentation_team` dispatched when code files change |
+| All 3 sub-agents present | `doc-writer`, `doc-reviewer`, `changelog-updater` in manifest |
+| Sub-agent directives exist | All directive files found on disk |
+| Correct execution order | Sequential: writer → reviewer → changelog |
+| Payload carries context | `changed_files`, `commit_msg`, `change_type` passed through |
+
+---
+
+## Run
+
+```bash
+python3 execution/run_test_scenario.py --scenario 3
+```
+
+---
+
+## Expected Output
+
+```json
+{
+  "scenario": "scenario_03_doc_team_on_code",
+  "pattern": "doc-team-on-code",
+  "status": "pass",
+  "steps": [
+    { "step": "trigger documentation_team on code change", "exit_code": 0 },
+    {
+      "step": "validate team has all 3 documentation sub-agents",
+      "expected": ["doc-writer", "doc-reviewer", "changelog-updater"],
+      "found": ["doc-writer", "doc-reviewer", "changelog-updater"],
+      "pass": true
+    },
+    {
+      "step": "validate all sub-agent directives exist",
+      "pass": true
+    }
+  ]
+}
+```
+
+---
+
+## Manual Walkthrough (the full pattern)
+
+This is the scenario to run whenever you make a code change to validate the framework is working:
+
+```bash
+# 1. Make a code change (example: edit session_boot.py)
+
+# 2. Trigger doc team — this is what should happen automatically per agent_team_rules.md
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{
+    "changed_files": ["execution/session_boot.py"],
+    "commit_msg": "feat: add --auto-fix flag to session boot",
+    "change_type": "feat"
+  }'
+
+# 3. Read the manifest — invoke doc-writer:
+#    Read directives/subagents/doc_writer.md
+#    → Read execution/session_boot.py
+#    → Update or create docs/execution/session_boot.md
+
+# 4. Invoke doc-reviewer:
+#    Read directives/subagents/doc_reviewer.md
+#    → Compare doc vs code → output ✅ or ❌ with gaps
+
+# 5. If gaps found → doc-writer fixes → doc-reviewer re-checks (max 2 loops)
+
+# 6. Once ✅, invoke changelog-updater:
+#    Read directives/subagents/changelog_updater.md
+#    → Append to CHANGELOG.md under [Unreleased] > Added
+
+# 7. Check results:
+python3 execution/agent_team_result.py --team documentation_team
+```
+
+---
+
+## Why This Matters
+
+Without this pattern:
+- Docs drift from code silently
+- CHANGELOG goes stale
+- Next developer (or agent) has wrong context
+
+With this pattern:
+- Documentation is **a side effect of coding** — automatic, not optional
+- Every commit is self-documenting
+- Memory system stays accurate for future agent sessions

package/templates/base/tests/agents/scenario_04_full_pipeline.md

@@ -0,0 +1,114 @@
+# Scenario 04: Full Team Agent Pipeline
+
+**Pattern:** `full-team-pipeline`  
+**Teams used:** `code_review_team` → `documentation_team` → `qa_team` (sequential)  
+**Sub-agents:** Spec-reviewer, quality-reviewer, doc-writer, doc-reviewer, changelog-updater, test-generator, test-verifier
+
+---
+
+## Goal
+
+Validate that all three agent teams can be chained sequentially in a complete pipeline, where each team's completion triggers the next. This represents the most complete quality assurance flow in the framework.
+
+---
+
+## Pipeline Flow
+
+```
+Code Change
+    │
+    ▼
+code_review_team
+    ├── spec-reviewer ✅
+    └── quality-reviewer ✅
+    │
+    ▼
+documentation_team
+    ├── doc-writer ✅
+    ├── doc-reviewer ✅
+    └── changelog-updater ✅
+    │
+    ▼
+qa_team
+    ├── test-generator ✅
+    └── test-verifier ✅
+    │
+    ▼
+Task Complete ✅
+```
+
+---
+
+## What This Tests
+
+| Test Point | Description |
+|---|---|
+| Sequential team dispatch | All 3 teams dispatched in correct order |
+| Context threading | Each team receives the same base payload |
+| All teams dispatch cleanly | Each returns exit code 0 with a valid manifest |
+| Run ID uniqueness | Each team dispatch has a unique run ID |
+
+---
+
+## Run
+
+```bash
+python3 execution/run_test_scenario.py --scenario 4
+```
+
+---
+
+## Expected Output
+
+```json
+{
+  "scenario": "scenario_04_full_pipeline",
+  "pattern": "full-team-pipeline",
+  "status": "pass",
+  "steps": [
+    { "step": "dispatch code_review_team", "exit_code": 0, "run_id": "abc12345" },
+    { "step": "dispatch documentation_team", "exit_code": 0, "run_id": "def67890" },
+    { "step": "dispatch qa_team", "exit_code": 0, "run_id": "ghi11223" },
+    {
+      "step": "validate full pipeline completed",
+      "teams_dispatched": ["code_review_team", "documentation_team", "qa_team"],
+      "all_passed": true
+    }
+  ]
+}
+```
+
+---
+
+## Manual Walkthrough
+
+```bash
+# After implementing a feature:
+
+# Step 1: Code review
+python3 execution/dispatch_agent_team.py \
+  --team code_review_team \
+  --payload '{"task_spec": "Add --dry-run flag", "changed_files": ["execution/dispatch_agent_team.py"], "git_shas": ["abc1234"], "task_id": "feat-001"}'
+
+# → Invoke spec-reviewer, then quality-reviewer
+# → Both must ✅ before proceeding
+
+# Step 2: Documentation
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{"changed_files": ["execution/dispatch_agent_team.py"], "commit_msg": "feat: dry-run flag", "change_type": "feat"}'
+
+# → doc-writer, doc-reviewer, changelog-updater
+
+# Step 3: QA
+python3 execution/dispatch_agent_team.py \
+  --team qa_team \
+  --payload '{"changed_files": ["execution/dispatch_agent_team.py"], "task_spec": "Add --dry-run flag", "test_runner": "pytest"}'
+
+# → test-generator writes tests, test-verifier runs them
+
+# Check full pipeline result
+python3 execution/agent_team_result.py --team code_review_team
+python3 execution/agent_team_result.py --team documentation_team
+python3 execution/agent_team_result.py --team qa_team
+```

package/templates/base/tests/agents/scenario_05_failure_recovery.md

@@ -0,0 +1,105 @@
+# Scenario 05: Failure Recovery
+
+**Pattern:** `failure-recovery`  
+**Tests:** Invalid team ID, malformed payload, then successful re-dispatch
+
+---
+
+## Goal
+
+Validate that the framework handles errors gracefully and recovers — it never crashes silently, always returns structured errors, and allows a re-dispatch after failure without leaving the system in a broken state.
+
+---
+
+## What This Tests
+
+| Test Point | Exit Code | Description |
+|---|---|---|
+| Invalid team ID | 2 | `dispatch_agent_team.py` returns structured error, exits 2 |
+| Malformed JSON payload | 3 | Script returns structured error, exits 3 |
+| Recovery re-dispatch | 0 | Valid dispatch after failures succeeds normally |
+
+---
+
+## Run
+
+```bash
+python3 execution/run_test_scenario.py --scenario 5
+```
+
+---
+
+## Expected Output
+
+```json
+{
+  "scenario": "scenario_05_failure_recovery",
+  "pattern": "failure-recovery",
+  "status": "pass",
+  "steps": [
+    {
+      "step": "dispatch invalid team (expect fail)",
+      "expected_exit_code": 2,
+      "actual_exit_code": 2,
+      "pass": true
+    },
+    {
+      "step": "dispatch invalid payload JSON (expect exit 3)",
+      "expected_exit_code": 3,
+      "actual_exit_code": 3,
+      "pass": true
+    },
+    {
+      "step": "recovery: re-dispatch valid request (expect pass)",
+      "expected_exit_code": 0,
+      "actual_exit_code": 0,
+      "pass": true
+    }
+  ]
+}
+```
+
+---
+
+## Manual Walkthrough
+
+```bash
+# Simulate failure: bad team name
+python3 execution/dispatch_agent_team.py \
+  --team not_a_real_team \
+  --payload '{"changed_files": []}'
+# Expected: exit 2, stderr shows structured JSON error with hint
+
+# Simulate failure: broken JSON
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{this is not valid json'
+# Expected: exit 3, stderr shows JSON parse error with hint
+
+# Recovery: valid dispatch after failures
+python3 execution/dispatch_agent_team.py \
+  --team documentation_team \
+  --payload '{"changed_files": ["README.md"], "commit_msg": "docs: fix typo", "change_type": "docs"}'
+# Expected: exit 0, manifest printed to stdout
+
+# Confirm no state corruption from previous failures
+python3 execution/agent_team_result.py --team documentation_team
+# Expected: result for the successful (recovery) dispatch only
+```
+
+---
+
+## Self-Annealing Principle
+
+This scenario validates the **self-annealing pattern** from `AGENTS.md`:
+
+> Errors are learning opportunities, not failures. When something breaks:
+> 1. Read error message and stack trace
+> 2. Diagnose root cause
+> 3. Fix and retry
+> 4. Update directive with what was learned
+
+A well-designed sub-agent or team script should:
+- **Never panic** on bad input
+- **Always** return structured JSON errors (not raw Python tracebacks)
+- **Always** use semantic exit codes so the orchestrator can respond appropriately

package/templates/base/tests/test_agent_events.py

@@ -0,0 +1,249 @@
+#!/usr/bin/env python3
+"""
+Test: Agent Events (Apache Pulsar)
+==================================
+
+Tests for the real-time agent event bus:
+1. Health check (graceful when Pulsar not running)
+2. Event publishing (graceful degradation)
+3. Topic listing
+4. Integration with memory_manager.py
+
+MODE 1 — With Pulsar running (full integration):
+    docker compose -f docker-compose.pulsar.yml up -d
+    python3 templates/base/tests/test_agent_events.py
+
+MODE 2 — Without Pulsar (offline/graceful degradation):
+    python3 templates/base/tests/test_agent_events.py --offline
+"""
+
+import json
+import os
+import sys
+
+# Resolve paths
+_test_dir = os.path.dirname(os.path.abspath(__file__))
+_base_dir = os.path.dirname(_test_dir)
+sys.path.insert(0, os.path.join(_base_dir, "execution"))
+
+from agent_events import (
+    health_check, publish_event, subscribe_events,
+    list_topics, PulsarClient, EVENT_TYPES
+)
+
+OFFLINE = "--offline" in sys.argv
+
+
+class TestResults:
+    def __init__(self):
+        self.passed = 0
+        self.failed = 0
+        self.skipped = 0
+        self.results = []
+
+    def check(self, name, condition, detail=""):
+        status = "PASS" if condition else "FAIL"
+        self.results.append({"test": name, "status": status, "detail": detail})
+        if condition:
+            self.passed += 1
+        else:
+            self.failed += 1
+        icon = "✅" if condition else "❌"
+        print(f"  {icon} {name}" + (f" — {detail}" if detail else ""))
+
+    def skip(self, name, reason=""):
+        self.skipped += 1
+        self.results.append({"test": name, "status": "SKIP", "detail": reason})
+        print(f"  ⏭️  {name} — {reason}")
+
+
+# ═══════════════════════════════════════════════════════════════
+# OFFLINE Tests (no Pulsar needed)
+# ═══════════════════════════════════════════════════════════════
+
+def test_event_types(t: TestResults):
+    """Event types are well-defined."""
+    t.check("event types list exists", len(EVENT_TYPES) > 0,
+            f"{len(EVENT_TYPES)} types defined")
+    t.check("memory_stored in event types", "memory_stored" in EVENT_TYPES)
+    t.check("decision_made in event types", "decision_made" in EVENT_TYPES)
+    t.check("context_request in event types", "context_request" in EVENT_TYPES)
+
+
+def test_graceful_degradation(t: TestResults):
+    """When Pulsar is unreachable, all operations degrade gracefully."""
+    bad_client = PulsarClient(http_url="http://localhost:19999")
+    t.check("bad client reports unavailable", bad_client.is_available() is False)
+
+    # publish_event should return unavailable, not crash
+    result = publish_event("test-project", "memory_stored", "test content")
+    # This uses the global client — may or may not be available
+    t.check("publish returns structured response",
+            isinstance(result, dict) and "status" in result)
+
+    # subscribe should return unavailable, not crash
+    result = subscribe_events("test-project")
+    t.check("subscribe returns structured response",
+            isinstance(result, dict) and "status" in result)
+
+
+def test_topic_naming(t: TestResults):
+    """Topic names are correctly formatted."""
+    client = PulsarClient()
+    topic = client.topic_name("my-app")
+    t.check("topic has persistent prefix", topic.startswith("persistent://"))
+    t.check("topic contains project name", "my-app" in topic)
+
+    # Special characters get sanitized
+    topic2 = client.topic_name("My App/v2")
+    t.check("special chars sanitized", "/" not in topic2.split("//", 1)[1].split("/", 2)[-1])
+
+
+def test_health_check_structure(t: TestResults):
+    """Health check returns proper structure."""
+    h = health_check()
+    t.check("health has agent field", "agent" in h)
+    t.check("health has status field", "status" in h)
+    t.check("health has optional flag", h.get("optional") is True)
+
+
+# ═══════════════════════════════════════════════════════════════
+# ONLINE Tests (require Pulsar running)
+# ═══════════════════════════════════════════════════════════════
+
+def test_pulsar_health(t: TestResults):
+    """Pulsar is running and healthy."""
+    h = health_check()
+    t.check("Pulsar is healthy", h["status"] == "healthy",
+            f"topics: {h.get('topics', 0)}")
+
+
+def test_publish_event(t: TestResults):
+    """Publish events to a project topic."""
+    import uuid
+    project = f"test-{uuid.uuid4().hex[:6]}"
+
+    result = publish_event(
+        project=project,
+        event_type="decision_made",
+        content="Test decision: use Qdrant for distributed auth",
+        developer_id="test@example.com"
+    )
+    t.check("event published successfully", result["status"] == "published",
+            f"method: {result.get('method', 'unknown')}")
+
+
+def test_list_topics(t: TestResults):
+    """List topics after publishing."""
+    result = list_topics()
+    t.check("topics response has count", "count" in result)
+    t.check("topics list is not empty", result.get("count", 0) >= 0,
+            f"found {result.get('count', 0)} topics")
+
+
+def test_memory_manager_integration(t: TestResults):
+    """memory_manager.py store auto-publishes events when Pulsar is running."""
+    import subprocess
+    import uuid
+    project = f"evt-test-{uuid.uuid4().hex[:6]}"
+
+    # Suppress pulsar-client INFO logs (they write to stdout too)
+    env = {**os.environ, "MEMORY_MODE": "team", "PULSAR_LOG_LEVEL": "ERROR"}
+    result = subprocess.run(
+        ["python3", os.path.join(_base_dir, "execution", "memory_manager.py"),
+         "store", "--content", "Event integration test decision",
+         "--type", "decision", "--project", project],
+        capture_output=True, text=True, timeout=30, env=env
+    )
+    t.check("memory store succeeds", result.returncode == 0)
+
+    if result.returncode == 0:
+        # Extract JSON: find matching braces to handle interleaved log lines
+        stdout = result.stdout.strip()
+        # Find the first { and last } to extract the JSON object
+        json_start = stdout.find("{")
+        json_end = stdout.rfind("}")
+        if json_start >= 0 and json_end > json_start:
+            # Filter out non-JSON lines between braces
+            raw = stdout[json_start:json_end + 1]
+            lines = [l for l in raw.split("\n") if not l.strip().startswith(("2026-", "20"))]
+            clean_json = "\n".join(lines)
+            try:
+                output = json.loads(clean_json)
+                has_event = "event" in output
+                t.check("store output includes event field", has_event,
+                        f"event status: {output.get('event', {}).get('status', 'missing')}")
+            except json.JSONDecodeError:
+                t.check("store output includes event field", "event" in stdout,
+                        "parsed from raw output")
+        else:
+            t.skip("event field check", "no JSON in output")
+    else:
+        t.skip("event field check", f"store failed: {result.stderr[:200]}")
+
+
+# ═══════════════════════════════════════════════════════════════
+# Main Runner
+# ═══════════════════════════════════════════════════════════════
+
+def main():
+    print("=" * 60)
+    print("📡 Agent Events Integration Tests")
+    print(f"   Mode: {'OFFLINE (no Pulsar)' if OFFLINE else 'ONLINE (Pulsar required)'}")
+    print("=" * 60)
+
+    t = TestResults()
+
+    # Always run offline tests
+    print("\n📋 Offline: Event Types")
+    test_event_types(t)
+
+    print("\n📋 Offline: Graceful Degradation")
+    test_graceful_degradation(t)
+
+    print("\n📋 Offline: Topic Naming")
+    test_topic_naming(t)
+
+    print("\n📋 Offline: Health Check Structure")
+    test_health_check_structure(t)
+
+    if not OFFLINE:
+        h = health_check()
+        if h["status"] != "healthy":
+            print(f"\n⚠️  Pulsar not available ({h['status']}). Skipping online tests.")
+            t.skip("online tests", "Pulsar not running")
+        else:
+            print("\n📋 Online: Pulsar Health")
+            test_pulsar_health(t)
+
+            print("\n📋 Online: Publish Event")
+            test_publish_event(t)
+
+            print("\n📋 Online: List Topics")
+            test_list_topics(t)
+
+            print("\n📋 Online: memory_manager.py Integration")
+            test_memory_manager_integration(t)
+
+    # Summary
+    print("\n" + "=" * 60)
+    total = t.passed + t.failed
+    print(f"📊 Results: {t.passed}/{total} passed, {t.failed} failed, {t.skipped} skipped")
+    print("=" * 60)
+
+    report = {
+        "suite": "agent_events",
+        "mode": "offline" if OFFLINE else "online",
+        "passed": t.passed,
+        "failed": t.failed,
+        "skipped": t.skipped,
+        "total": total,
+        "status": "pass" if t.failed == 0 else "fail",
+        "tests": t.results
+    }
+    print(f"\n{json.dumps(report, indent=2)}")
+    sys.exit(0 if t.failed == 0 else 1)
+
+
+if __name__ == "__main__":
+    main()