@techwavedev/agi-agent-kit 1.6.4 → 1.6.5

package/CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [1.6.5] - 2026-03-19
+
+### Added
+
+- **Skill Self-Improvement (Karpathy Loop)** — Autonomous test → improve → commit/reset cycle for continuous skill quality improvement. Inspired by Andrej Karpathy's "auto-research" concept:
+  - `execution/run_skill_eval.py` — Binary assertion runner with 18 assertion types (`contains`, `regex_match`, `max_words`, `has_yaml_frontmatter`, `no_trailing_whitespace`, etc.)
+  - `execution/karpathy_loop.py` — Autonomous loop orchestrator with git commit/reset integration, dry-run mode, and status reporting
+  - `eval/evals.json` standard — Structured binary assertion format for objective skill quality measurement
+- **Skill Creator: Step 8** — Added "Self-Improvement Loop (Karpathy Loop)" step to `SKILL_skillcreator.md` with methodology, assertion examples, and auto-generation guidance
+- **Eval Directory Scaffolding** — `init_skill.py` now creates `eval/evals.json` with 4 starter assertions (frontmatter, description quality, structure, no-placeholders) for every new skill
+- **Example evals** — `skills/qdrant-memory/eval/evals.json` added as reference implementation (17/17 passing)
+
+### Documentation
+
+- **`AGENTS.md`** — Added "Skill Self-Improvement (Karpathy Loop)" section with quick start commands, assertion types reference, and key rules. Updated directory structure to show `eval/`. Added eval scripts to Key Scripts section.
 
 ## [1.6.4] - 2026-03-14
 

package/README.md

@@ -95,6 +95,7 @@ python3 skills/plugin-discovery/scripts/platform_setup.py --project-dir .
 | **TDD Enforcement**           | Iron-law RED-GREEN-REFACTOR cycle — no production code without failing test                   |
 | **Verification Gates**        | Evidence before claims — no completion without fresh verification output                      |
 | **Self-Healing Workflows**    | Agents read error logs, patch scripts, and update directives automatically                    |
+| **Skill Self-Improvement**    | Karpathy Loop: autonomous test → improve → commit/reset cycle with 18 binary assertion types  |
 | **One-Shot Setup**            | Platform detection + project stack scan + auto-configuration in one command                   |
 
 ---
@@ -608,6 +609,7 @@ If these community skills help you, consider [starring the original repo](https:
 | **MCP Compatibility**               | ✅ Shipped    | Memory + cross-agent coordination exposed as MCP tools via `execution/mcp_server.py` (13 tools) and `skills/qdrant-memory/mcp_server.py` (6 tools). Pure chat clients (Claude Desktop) get full memory access. ([docs](./docs/mcp-compatibility.md)) |
 | **Platform-Adaptive Orchestration** | ✅ Shipped    | 10 platforms share one `AGENTS.md` via symlinks (Claude Code, Gemini CLI, Codex CLI, Cursor, Copilot, OpenCode, AdaL, Antigravity, OpenClaw, Kiro). Each uses its native orchestration strategy automatically.           |
 | **Workflow Engine**                 | ✅ Shipped    | `execution/workflow_engine.py` executes `data/workflows.json` playbooks as guided multi-skill sequences with progress tracking, skip/abort, and state persistence in `.tmp/playbook_state.json`.                         |
+| **Skill Self-Improvement**          | ✅ Shipped    | Karpathy Loop: `run_skill_eval.py` (18 binary assertion types) + `karpathy_loop.py` (autonomous test/improve/commit/reset). Skills include `eval/evals.json` for objective quality measurement.                           |
 | **Control Tower Orchestrator**      | 🚧 Active    | Basic dispatcher for agent registration and heartbeat via Qdrant (`control_tower.py`). Needs dedicated docs, test coverage, and integration with `session_boot`.                                                          |
 | **Secrets Management (Vault)**      | 🔬 Design    | HashiCorp Vault integration for secure secret sharing. Agents authenticate via Ed25519 keypair, access tenant-scoped secrets. Zero long-lived credentials.                                                                |
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@techwavedev/agi-agent-kit",
-  "version": "1.6.4",
+  "version": "1.6.5",
   "description": "Enterprise-Grade Agentic Framework - Modular skill-based AI assistant toolkit with deterministic execution, semantic memory, and platform-adaptive orchestration.",
   "bin": {
     "agi-agent-kit": "./bin/init.js"

package/templates/skills/core/qdrant-memory/scripts/embedding_utils.py

@@ -36,6 +36,15 @@ from typing import List
 from urllib.request import Request, urlopen
 from urllib.error import URLError
 
+# Optional Langfuse Observability for token tracking
+try:
+    from langfuse.decorators import observe
+except ImportError:
+    def observe(*args, **kwargs):
+        def decorator(func):
+            return func
+        return decorator
+
 # Configuration with Ollama as default (local/private)
 EMBEDDING_PROVIDER = os.environ.get("EMBEDDING_PROVIDER", "ollama")
 OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://localhost:11434")
@@ -190,6 +199,7 @@ def get_embedding_bedrock(text: str) -> List[float]:
     return result.get("embedding", result.get("embeddings", [[]])[0])
 
 
+@observe(name="generate_embedding", as_type="generation")
 def get_embedding(text: str) -> List[float]:
     """
     Generate embedding using the configured provider.