@techwavedev/agi-agent-kit 1.6.0 → 1.6.1

package/CHANGELOG.md

@@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.6.1] - 2026-03-14
+
+### Added
+
+- **Distributed Agent Authentication** — Replaced local SQLite auth storage with shared Qdrant `agent_auth` collection. Identities, access grants, content hashes, and audit trails are now distributed across all agents sharing a Qdrant instance. 8 payload indexes for efficient filtering. Deterministic UUIDs for idempotent operations. (36/36 tests)
+- **Hyperledger Aries Integration** — Replaced deprecated MultiChain with Hyperledger Aries ACA-Py 1.5.0 (OpenWallet Foundation). W3C DID identity, Ed25519 signing, official Docker image (`ghcr.io/openwallet-foundation/acapy-agent:1.5.0`). Optional add-on — HMAC-SHA256 works without Aries.
+- **Memory Mode Tiers** — Three operational modes (`MEMORY_MODE`): Solo (single user), Team (multi-tenancy with developer isolation), Pro (blockchain auth + access control). All backward-compatible, no data migration needed.
+- **Real-Time Agent Events** — Apache Pulsar event bus for push notifications between agents. Auto-publishes on `memory_manager.py store` (team/pro modes). 9 event types. Project-scoped topics (`persistent://agi/memory/<project>`). Graceful degradation when Pulsar unavailable. (19/19 tests)
+- **BM25 Auto-Sync on Boot** — `session_boot.py` now automatically syncs the local BM25 keyword index from shared Qdrant data, ensuring every machine has consistent hybrid search results.
+- **Docker Compose for Pulsar** — `docker-compose.pulsar.yml` with `apachepulsar/pulsar:4.1.3`, lightweight standalone mode (256-512MB heap), health checks.
+- **Docker Compose for Aries** — `docker-compose.aries.yml` with official ACA-Py 1.5.0 image, wallet configuration.
+
+### Changed
+
+- **`blockchain_auth.py`** — Complete rewrite: `QdrantAuthStore` replaces SQLite, `AriesClient` replaces MultiChain. All auth data stored in shared Qdrant collection.
+- **`session_boot.py`** — Now reports Pulsar status, Aries status, and BM25 sync results. Auto-syncs BM25 from Qdrant on every boot.
+- **`memory_manager.py`** — Auto-publishes Pulsar events on store (team/pro). Health check includes events status.
+- **`session_init.py`** — Creates `agent_auth` Qdrant collection with 4-dimensional vectors and 8 payload indexes.
+
+### Documentation
+
+- **`docs/memory-modes.md`** — Complete rewrite. Why each mode, when to use, scenario comparisons, data storage table. Every feature verified with test evidence.
+- **`docs/blockchain-auth.md`** — Complete rewrite. Technology comparison (MultiChain vs Ethereum vs Aries), Qdrant vs SQLite rationale, trust model diagram, all CLI commands verified.
+- **`docs/agent-events.md`** — Complete rewrite. Pulsar vs Redis/RabbitMQ/Kafka comparison, verified capabilities table, corrected dependency requirements.
+
+### Removed
+
+- **MultiChain** — Deprecated (last commit 2023, no maintained Docker image). Replaced by Hyperledger Aries.
+- **SQLite auth storage** — Replaced by shared Qdrant `agent_auth` collection for distributed consistency.
+
 ## [1.6.0] - 2026-03-12
 
 ### Added

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@techwavedev/agi-agent-kit",
-  "version": "1.6.0",
+  "version": "1.6.1",
   "description": "Enterprise-Grade Agentic Framework - Modular skill-based AI assistant toolkit with deterministic execution, semantic memory, and platform-adaptive orchestration.",
   "bin": {
     "agi-agent-kit": "./bin/init.js"

package/templates/base/.env.example

@@ -69,3 +69,31 @@ SNYK_TOKEN=
 SENTRY_DSN=
 # Datadog API Key
 DD_API_KEY=
+
+# --- Memory Mode (solo / team / pro) ---
+# solo  = Ollama + Qdrant only (single developer, no identity tagging)
+# team  = + Multi-tenancy (developer/agent identity, shared memory, isolation)
+# pro   = + Blockchain auth (signed writes, hash anchoring, access control)
+MEMORY_MODE=team
+
+# Developer identity (optional — auto-detected from git config user.email)
+# AGI_DEVELOPER_ID=your@email.com
+# AGI_AGENT_ID=my-agent
+
+# --- Hyperledger Aries — Blockchain Auth (pro mode only, optional add-on) ---
+# Only needed if MEMORY_MODE=pro. Solo and team modes ignore these entirely.
+# Start with: docker compose -f docker-compose.aries.yml up -d
+ARIES_ADMIN_URL=http://localhost:8031
+ARIES_ADMIN_KEY=changeme_use_a_strong_api_key
+ARIES_WALLET_KEY=changeme_use_a_strong_wallet_key
+ARIES_HTTP_PORT=8030
+ARIES_ADMIN_PORT=8031
+
+# --- Apache Pulsar — Real-Time Events (team/pro, optional add-on) ---
+# Enables real-time event push between agents (auto-publish on memory store).
+# Without Pulsar, agents fall back to polling Qdrant (which always works).
+# Start with: docker compose -f docker-compose.pulsar.yml up -d
+PULSAR_URL=pulsar://localhost:6650
+PULSAR_HTTP_URL=http://localhost:8080
+PULSAR_PORT=6650
+PULSAR_HTTP_PORT=8080

package/templates/base/docker-compose.aries.yml

@@ -0,0 +1,47 @@
+# Hyperledger Aries (ACA-Py) — Blockchain-Authenticated Agent Memory
+# Usage: docker compose -f docker-compose.aries.yml up -d
+# Credentials: set ARIES_ADMIN_KEY in your .env file
+
+services:
+  aries-agent:
+    image: ghcr.io/openwallet-foundation/acapy-agent:1.5.0
+    container_name: agi-aries-agent
+    ports:
+      - "${ARIES_HTTP_PORT:-8030}:8030"       # DIDComm endpoint
+      - "${ARIES_ADMIN_PORT:-8031}:8031"      # Admin REST API
+    environment:
+      ACAPY_ADMIN: "[0.0.0.0,8031]"
+      ACAPY_ADMIN_API_KEY: "${ARIES_ADMIN_KEY:-changeme_set_in_dotenv}"
+      ACAPY_ENDPOINT: "http://localhost:${ARIES_HTTP_PORT:-8030}"
+      ACAPY_LABEL: "AGI Memory Agent"
+      ACAPY_LOG_LEVEL: "info"
+      ACAPY_AUTO_PROVISION: "true"
+      ACAPY_WALLET_TYPE: "askar"
+      ACAPY_WALLET_NAME: "agi-memory-wallet"
+      ACAPY_WALLET_KEY: "${ARIES_WALLET_KEY:-changeme_wallet_key}"
+    command: >
+      start
+        --inbound-transport http 0.0.0.0 8030
+        --outbound-transport http
+        --admin 0.0.0.0 8031
+        --admin-api-key ${ARIES_ADMIN_KEY:-changeme_set_in_dotenv}
+        --auto-provision
+        --wallet-type askar
+        --wallet-name agi-memory-wallet
+        --wallet-key ${ARIES_WALLET_KEY:-changeme_wallet_key}
+        --label "AGI Memory Agent"
+        --endpoint http://localhost:${ARIES_HTTP_PORT:-8030}
+        --log-level info
+    volumes:
+      - aries_wallet:/home/aries/.aries_cloudagent
+    env_file:
+      - .env
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "-H", "X-API-Key: ${ARIES_ADMIN_KEY:-changeme_set_in_dotenv}", "http://localhost:8031/status"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+volumes:
+  aries_wallet:

package/templates/base/docker-compose.pulsar.yml

@@ -0,0 +1,29 @@
+# Apache Pulsar — Real-Time Agent Event Bus
+# Usage: docker compose -f docker-compose.pulsar.yml up -d
+# Only needed for real-time push between agents (optional)
+
+services:
+  pulsar:
+    image: apachepulsar/pulsar:4.1.3
+    container_name: agi-pulsar
+    ports:
+      - "${PULSAR_PORT:-6650}:6650"         # Binary protocol
+      - "${PULSAR_HTTP_PORT:-8080}:8080"     # HTTP admin API
+    volumes:
+      - pulsar_data:/pulsar/data
+      - pulsar_conf:/pulsar/conf
+    command: bin/pulsar standalone
+    environment:
+      PULSAR_MEM: "-Xms256m -Xmx512m"
+    env_file:
+      - .env
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8080/admin/v2/brokers/healthcheck"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+volumes:
+  pulsar_data:
+  pulsar_conf:

package/templates/base/execution/agent_events.py

@@ -0,0 +1,349 @@
+#!/usr/bin/env python3
+"""
+Script: agent_events.py
+Purpose: Real-time agent event bus via Apache Pulsar.
+
+Enables agents on different machines to:
+  - Publish events when they store memories, make decisions, or complete tasks
+  - Subscribe to events from other agents in their project/team
+  - Coordinate work in real-time (not just poll Qdrant)
+
+Topics follow the pattern: persistent://agi/memory/<project>
+
+Graceful degradation: if Pulsar is not running, events are silently dropped
+and agents fall back to polling Qdrant (which always works).
+
+Usage:
+    python3 agent_events.py health
+    python3 agent_events.py publish --project myapp --event-type decision --content "Use PostgreSQL"
+    python3 agent_events.py subscribe --project myapp --callback print
+    python3 agent_events.py list-topics
+
+Exit Codes:
+    0 - Success
+    1 - Error
+"""
+
+import argparse
+import json
+import os
+import sys
+import time
+import threading
+from datetime import datetime, timezone
+from urllib.request import Request, urlopen
+from urllib.error import URLError, HTTPError
+
+
+# ═══════════════════════════════════════════════════════════════
+# Configuration
+# ═══════════════════════════════════════════════════════════════
+
+PULSAR_URL = os.environ.get("PULSAR_URL", "pulsar://localhost:6650")
+PULSAR_HTTP_URL = os.environ.get("PULSAR_HTTP_URL", "http://localhost:8080")
+PULSAR_TENANT = os.environ.get("PULSAR_TENANT", "agi")
+PULSAR_NAMESPACE = os.environ.get("PULSAR_NAMESPACE", "memory")
+
+# Event types emitted by the memory system
+EVENT_TYPES = [
+    "memory_stored",       # Agent stored a new memory
+    "decision_made",       # Agent made a decision
+    "code_written",        # Agent wrote/modified code
+    "error_resolved",      # Agent resolved an error
+    "task_completed",      # Agent completed a task
+    "access_granted",      # Access was granted to an entity
+    "identity_registered", # New agent/developer registered
+    "context_request",     # Agent is requesting context from peers
+    "context_response",    # Agent is responding to a context request
+]
+
+
+# ═══════════════════════════════════════════════════════════════
+# Pulsar Client (HTTP Admin API + binary protocol)
+# ═══════════════════════════════════════════════════════════════
+
+class PulsarClient:
+    """Client for Apache Pulsar — HTTP admin API for management,
+    pulsar-client library for pub/sub if available, HTTP producer/consumer as fallback."""
+
+    def __init__(self, http_url=None, binary_url=None):
+        self.http_url = (http_url or PULSAR_HTTP_URL).rstrip("/")
+        self.binary_url = binary_url or PULSAR_URL
+        self._available = None
+        self._pulsar_lib = None
+        self._producers = {}
+        self._try_import_pulsar()
+
+    def _try_import_pulsar(self):
+        """Try to import the pulsar-client library (optional)."""
+        try:
+            import pulsar
+            self._pulsar_lib = pulsar
+        except ImportError:
+            self._pulsar_lib = None
+
+    def _http_request(self, method: str, path: str, data: dict = None) -> dict:
+        """Make HTTP request to Pulsar admin API."""
+        url = f"{self.http_url}{path}"
+        body = json.dumps(data).encode() if data else None
+        headers = {"Content-Type": "application/json"}
+        req = Request(url, data=body, headers=headers, method=method)
+        with urlopen(req, timeout=10) as resp:
+            content = resp.read().decode()
+            return json.loads(content) if content else {}
+
+    def is_available(self) -> bool:
+        """Check if Pulsar broker is reachable."""
+        if self._available is not None:
+            return self._available
+        try:
+            req = Request(
+                f"{self.http_url}/admin/v2/brokers/healthcheck",
+                method="GET"
+            )
+            with urlopen(req, timeout=5) as resp:
+                self._available = resp.status == 200
+        except Exception:
+            self._available = False
+        return self._available
+
+    def get_status(self) -> dict:
+        """Get broker status."""
+        try:
+            # Get cluster info
+            clusters = self._http_request("GET", "/admin/v2/clusters")
+            return {"status": "healthy", "clusters": clusters}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+
+    def ensure_tenant_namespace(self):
+        """Create tenant and namespace if they don't exist."""
+        # Create tenant
+        try:
+            self._http_request("GET", f"/admin/v2/tenants/{PULSAR_TENANT}")
+        except HTTPError as e:
+            if e.code == 404:
+                self._http_request("PUT", f"/admin/v2/tenants/{PULSAR_TENANT}", {
+                    "allowedClusters": ["standalone"]
+                })
+
+        # Create namespace
+        try:
+            self._http_request("GET", f"/admin/v2/namespaces/{PULSAR_TENANT}/{PULSAR_NAMESPACE}")
+        except HTTPError as e:
+            if e.code == 404:
+                self._http_request("PUT", f"/admin/v2/namespaces/{PULSAR_TENANT}/{PULSAR_NAMESPACE}")
+
+    def topic_name(self, project: str) -> str:
+        """Build a full topic name for a project."""
+        safe_project = project.replace("/", "-").replace(" ", "-").lower()
+        return f"persistent://{PULSAR_TENANT}/{PULSAR_NAMESPACE}/{safe_project}"
+
+    def list_topics(self) -> list:
+        """List all topics in the namespace."""
+        try:
+            return self._http_request(
+                "GET",
+                f"/admin/v2/persistent/{PULSAR_TENANT}/{PULSAR_NAMESPACE}"
+            )
+        except Exception:
+            return []
+
+    def publish(self, project: str, event: dict) -> dict:
+        """Publish an event to a project topic."""
+        if not self.is_available():
+            return {"status": "unavailable", "reason": "Pulsar not running"}
+
+        topic = self.topic_name(project)
+
+        # Try native pulsar-client first (faster, binary protocol)
+        if self._pulsar_lib:
+            return self._publish_native(topic, event)
+
+        # Fallback: HTTP producer
+        return self._publish_http(topic, event)
+
+    def _publish_native(self, topic: str, event: dict) -> dict:
+        """Publish using native pulsar-client library."""
+        try:
+            if topic not in self._producers:
+                client = self._pulsar_lib.Client(self.binary_url)
+                self._producers[topic] = client.create_producer(topic)
+            msg_id = self._producers[topic].send(
+                json.dumps(event).encode("utf-8")
+            )
+            return {"status": "published", "topic": topic, "method": "native", "msg_id": str(msg_id)}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+
+    def _publish_http(self, topic: str, event: dict) -> dict:
+        """Publish using HTTP producer API (no library needed)."""
+        try:
+            import base64
+            encoded = base64.b64encode(json.dumps(event).encode()).decode()
+            # Pulsar REST produce endpoint
+            safe_topic = topic.replace("persistent://", "persistent/")
+            url = f"{self.http_url}/topics/{safe_topic}"
+            req = Request(
+                url,
+                data=json.dumps({"payload": encoded}).encode(),
+                headers={"Content-Type": "application/json"},
+                method="POST"
+            )
+            with urlopen(req, timeout=10) as resp:
+                return {"status": "published", "topic": topic, "method": "http"}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+
+    def subscribe(self, project: str, subscription: str, callback, timeout_sec: int = 0):
+        """Subscribe to a project topic (blocking or background).
+        callback receives (event_dict) for each message.
+        timeout_sec=0 means run forever in background thread."""
+        if not self.is_available():
+            return {"status": "unavailable", "reason": "Pulsar not running"}
+
+        topic = self.topic_name(project)
+
+        if self._pulsar_lib:
+            return self._subscribe_native(topic, subscription, callback, timeout_sec)
+
+        return {"status": "error", "message": "pulsar-client library required for subscription (pip install pulsar-client)"}
+
+    def _subscribe_native(self, topic: str, subscription: str, callback, timeout_sec: int):
+        """Subscribe using native pulsar-client library."""
+        try:
+            client = self._pulsar_lib.Client(self.binary_url)
+            consumer = client.subscribe(
+                topic,
+                subscription,
+                consumer_type=self._pulsar_lib.ConsumerType.Shared
+            )
+
+            def _consume():
+                deadline = time.time() + timeout_sec if timeout_sec > 0 else float('inf')
+                while time.time() < deadline:
+                    try:
+                        msg = consumer.receive(timeout_millis=1000)
+                        event = json.loads(msg.data().decode())
+                        callback(event)
+                        consumer.acknowledge(msg)
+                    except Exception:
+                        continue
+
+            if timeout_sec > 0:
+                _consume()
+            else:
+                t = threading.Thread(target=_consume, daemon=True)
+                t.start()
+
+            return {"status": "subscribed", "topic": topic, "subscription": subscription}
+        except Exception as e:
+            return {"status": "error", "message": str(e)}
+
+
+# Global client
+_client = PulsarClient()
+
+
+# ═══════════════════════════════════════════════════════════════
+# Public API
+# ═══════════════════════════════════════════════════════════════
+
+def health_check() -> dict:
+    """Check Pulsar broker health."""
+    result = {"agent": "Apache Pulsar", "status": "not_running", "optional": True}
+    if _client.is_available():
+        status = _client.get_status()
+        result["status"] = status.get("status", "unknown")
+        result["topics"] = len(_client.list_topics())
+        result["native_client"] = _client._pulsar_lib is not None
+    return result
+
+
+def publish_event(project: str, event_type: str, content: str,
+                  developer_id: str = None, agent_id: str = None,
+                  metadata: dict = None) -> dict:
+    """Publish a memory event to project topic."""
+    if not _client.is_available():
+        return {"status": "unavailable", "reason": "Pulsar not running (events silently dropped)"}
+
+    _client.ensure_tenant_namespace()
+
+    event = {
+        "event_type": event_type,
+        "content": content,
+        "developer_id": developer_id or os.environ.get("AGI_DEVELOPER_ID", "unknown"),
+        "agent_id": agent_id or os.environ.get("AGI_AGENT_ID", "primary"),
+        "project": project,
+        "timestamp": datetime.now(timezone.utc).isoformat(),
+        "metadata": metadata or {},
+    }
+
+    return _client.publish(project, event)
+
+
+def subscribe_events(project: str, subscription: str = None, callback=None,
+                     timeout_sec: int = 0) -> dict:
+    """Subscribe to events from a project."""
+    if not _client.is_available():
+        return {"status": "unavailable", "reason": "Pulsar not running"}
+
+    sub_name = subscription or f"agi-{os.environ.get('AGI_DEVELOPER_ID', 'default')}"
+    cb = callback or (lambda e: print(json.dumps(e, indent=2)))
+
+    return _client.subscribe(project, sub_name, cb, timeout_sec)
+
+
+def list_topics() -> dict:
+    """List all active project topics."""
+    if not _client.is_available():
+        return {"topics": [], "status": "unavailable"}
+    topics = _client.list_topics()
+    return {"topics": topics, "count": len(topics)}
+
+
+# ═══════════════════════════════════════════════════════════════
+# CLI
+# ═══════════════════════════════════════════════════════════════
+
+def main():
+    parser = argparse.ArgumentParser(description="Agent event bus (Apache Pulsar)")
+    sub = parser.add_subparsers(dest="command")
+
+    sub.add_parser("health", help="Check Pulsar health")
+
+    pub = sub.add_parser("publish", help="Publish an event")
+    pub.add_argument("--project", required=True)
+    pub.add_argument("--event-type", required=True, choices=EVENT_TYPES)
+    pub.add_argument("--content", required=True)
+
+    sub_cmd = sub.add_parser("subscribe", help="Subscribe to events")
+    sub_cmd.add_argument("--project", required=True)
+    sub_cmd.add_argument("--subscription", default=None)
+    sub_cmd.add_argument("--timeout", type=int, default=30, help="Seconds to listen (0=forever)")
+
+    sub.add_parser("list-topics", help="List all active topics")
+
+    args = parser.parse_args()
+
+    if not args.command:
+        parser.print_help()
+        sys.exit(1)
+
+    if args.command == "health":
+        print(json.dumps(health_check(), indent=2))
+    elif args.command == "publish":
+        print(json.dumps(publish_event(args.project, args.event_type, args.content), indent=2))
+    elif args.command == "subscribe":
+        result = subscribe_events(args.project, args.subscription, timeout_sec=args.timeout)
+        if result.get("status") != "subscribed":
+            print(json.dumps(result, indent=2))
+            sys.exit(1)
+        print(f"Listening on {args.project} for {args.timeout}s...")
+        time.sleep(args.timeout)
+    elif args.command == "list-topics":
+        print(json.dumps(list_topics(), indent=2))
+
+
+if __name__ == "__main__":
+    main()