macro-agent 0.1.0 → 0.1.2

package/references/skill-tree/CLAUDE.md

@@ -0,0 +1,150 @@
+# CLAUDE.md
+
+This file provides guidance for Claude Code when working with this repository.
+
+## Project Overview
+
+**skill-tree** is a TypeScript library for managing agent skill versions and evolution. It extracts, iterates, and adapts skills from agent trajectories, enabling AI agents to build and maintain reusable skill libraries.
+
+## Key Architecture
+
+```
+src/
+├── index.ts              # Main exports
+├── skill-bank.ts         # Main orchestrator (SkillBank class)
+├── types.ts              # Core type definitions
+├── semver.ts             # Semantic versioning utilities
+├── adapters/             # Session format adapters (Claude Code, OpenAI, etc.)
+├── extraction/           # Manual and automatic skill extraction
+├── storage/              # Storage adapters (memory, filesystem, SQLite)
+├── matching/             # Semantic skill matching with embeddings
+├── versioning/           # Lineage tracking and version management
+├── learning/             # Learning providers for skill extraction
+│   ├── types.ts          # LearningProvider interface, candidate types
+│   └── providers/        # Provider implementations
+│       ├── naive.ts      # Simple heuristic-based extraction
+│       └── cognitive-core.ts  # Advanced batch learning via cognitive-core
+├── validation/           # Skill validation and testing
+├── metrics/              # Usage tracking and analytics
+├── hooks/                # Event hooks and activation system
+├── agents/               # AGENTS.md generation and parsing
+├── composition/          # Skill composition and merging
+└── cli/                  # Command-line interface
+```
+
+## Core Concepts
+
+### Skill
+A reusable piece of knowledge with:
+- `problem`: What it solves
+- `triggerConditions`: When to apply
+- `solution`: Step-by-step guidance
+- `verification`: How to verify success
+- Semantic versioning and lineage tracking
+
+### Trajectory
+An agent session parsed into structured turns (user, assistant, tool interactions) with metadata and outcome.
+
+### LearningProvider
+Interface for extracting candidates from trajectories. Two implementations:
+- `NaiveLearningProvider`: Simple, synchronous extraction
+- `CognitiveCoreProvider`: Advanced batch learning with dependency injection
+
+### LearningCandidate
+Extracted knowledge before promotion to a versioned skill. Has kinds: `skill`, `strategy`, `pattern`, `error-fix`.
+
+## Development Commands
+
+```bash
+# Build
+npm run build
+
+# Run all tests
+npm test
+
+# Run specific test file
+npx tsx test/learning.test.ts
+
+# Type check
+npx tsc --noEmit
+```
+
+## Testing Patterns
+
+Tests use a simple test harness in `test/run-all.ts`. Each test file exports tests via `describe()` and `it()` functions.
+
+```typescript
+describe('My Feature', () => {
+  it('does something', () => {
+    // assertions
+  });
+});
+```
+
+Mock factories are used for dependency injection testing, especially for `CognitiveCoreProvider`:
+
+```typescript
+function createMockCognitiveCoreFactory(): CognitiveCoreFactory & MockInspection {
+  // Returns mock implementations with inspection helpers
+}
+```
+
+## CognitiveCoreProvider
+
+This provider integrates with [cognitive-core](./references/cognitive-core/) (Atlas) for advanced learning. Key design decisions:
+
+1. **Dependency Injection**: Uses `CognitiveCoreFactory` interface instead of direct imports to avoid `tsconfig.json` `rootDir` issues and maintain clean separation.
+
+2. **Batch Accumulation**: Accumulates trajectories until `minTrajectories` threshold, then runs batch extraction for cross-trajectory pattern detection.
+
+3. **Trajectory Conversion**: Converts skill-tree `Turn[]` to cognitive-core's ReAct `Step[]` format when `convertToReAct: true`.
+
+4. **Kind Inference**: Automatically determines `LearningKind` from playbook content (error triggers → `error-fix`, strategy keywords → `strategy`, etc.).
+
+5. **Outcome Feedback**: Records success/failure to refine playbook confidence over time.
+
+See `src/learning/providers/README.md` for detailed documentation.
+
+## Important Files
+
+- `src/skill-bank.ts`: Main API entry point
+- `src/learning/types.ts`: Learning interface definitions
+- `src/learning/providers/cognitive-core.ts`: Cognitive-core adapter
+- `test/learning.test.ts`: Comprehensive learning provider tests
+
+## Common Tasks
+
+### Adding a new learning provider
+
+1. Implement `LearningProvider` interface in `src/learning/providers/`
+2. Export from `src/learning/providers/index.ts`
+3. Re-export from `src/learning/index.ts`
+4. Add tests in `test/learning.test.ts`
+
+### Extending CognitiveCoreProvider
+
+The provider uses interface types (`CCPlaybook`, `CCTrajectory`, etc.) that mirror cognitive-core. If cognitive-core adds new fields, update the interfaces in `cognitive-core.ts`.
+
+### Running a subset of tests
+
+```bash
+# Run only learning tests
+npx tsx test/learning.test.ts
+
+# Run with verbose output
+DEBUG=1 npx tsx test/learning.test.ts
+```
+
+## Code Style
+
+- TypeScript with strict mode
+- ES modules (`.js` extensions in imports)
+- JSDoc comments for public APIs
+- Interface types preferred over type aliases for extensibility
+- Dependency injection for external integrations
+
+## References
+
+- [cognitive-core (Atlas)](./references/cognitive-core/README.md): External learning engine
+- [OpenSkills](https://github.com/numman-ali/openskills): Skill format inspiration
+- [Claudeception](https://github.com/blader/Claudeception): Quality gates concept

package/references/skill-tree/README.md

@@ -0,0 +1,324 @@
+# skill-tree
+
+A TypeScript library for managing agent skill versions and evolution. Extract, iterate, and adapt skills from agent trajectories.
+
+## Overview
+
+skill-tree helps you build and maintain a library of reusable skills for AI agents by:
+
+- **Extracting skills** from agent sessions/trajectories (manual or automatic)
+- **Storing skills** in a versioned, searchable format (OpenSkills-compatible)
+- **Evolving skills** through versioning, forking, and merging
+- **Quality gates** to ensure extracted skills are reusable and non-trivial
+
+Inspired by research on skill libraries ([arXiv:2512.17102](https://arxiv.org/abs/2512.17102)), [Claudeception](https://github.com/blader/Claudeception), and [OpenSkills](https://github.com/numman-ali/openskills).
+
+## Installation
+
+```bash
+npm install skill-tree
+```
+
+## Quick Start
+
+```typescript
+import { createSkillBank } from 'skill-tree';
+
+// Create a skill bank with filesystem storage
+const skillBank = createSkillBank({
+  storage: {
+    type: 'filesystem',
+    basePath: './skills',
+    openSkillsCompatible: true,
+  },
+});
+
+await skillBank.initialize();
+
+// Parse a Claude Code session
+const trajectory = await skillBank.parseSession(sessionContent);
+
+// Extract a skill manually
+const result = await skillBank.extractManual(trajectory, {
+  suggestedName: 'typescript-import-fix',
+  description: 'Fix TypeScript ES module import errors',
+  tags: ['typescript', 'imports'],
+});
+
+if (result.success) {
+  console.log('Extracted skill:', result.skill.name);
+}
+
+// Search for skills
+const skills = await skillBank.searchSkills('typescript error');
+
+// Create a new version
+const updated = await skillBank.createVersion('typescript-import-fix', {
+  solution: 'Updated solution...',
+}, { bumpType: 'minor' });
+```
+
+## Core Concepts
+
+### Skills
+
+A skill represents a reusable piece of knowledge extracted from agent interactions:
+
+```typescript
+interface Skill {
+  id: string;                    // Unique identifier
+  name: string;                  // Human-readable name
+  version: string;               // Semantic version
+  description: string;           // For semantic matching
+  problem: string;               // What problem this solves
+  triggerConditions: TriggerCondition[];  // When to apply
+  solution: string;              // Step-by-step solution
+  verification: string;          // How to verify it worked
+  examples: SkillExample[];      // Usage examples
+  // ... metadata, metrics, lineage
+}
+```
+
+### Trajectories
+
+Agent sessions are parsed into trajectories:
+
+```typescript
+interface Trajectory {
+  sessionId: string;
+  turns: Turn[];       // User/assistant/tool interactions
+  metadata: TrajectoryMetadata;
+  outcome?: TrajectoryOutcome;
+}
+```
+
+### Quality Gates
+
+Extraction includes quality gates (inspired by Claudeception) to ensure skills are:
+
+- **Reusable**: Applicable across multiple future tasks
+- **Non-trivial**: Involves discovery beyond documentation lookup
+- **Specific**: Has clear trigger conditions
+- **Verified**: Solution demonstrably works
+
+## Features
+
+### Session Adapters
+
+Parse different agent session formats:
+
+```typescript
+// Built-in: Claude Code JSONL
+const trajectory = await skillBank.parseSession(jsonlContent);
+
+// Register custom adapters
+skillBank.registerAdapter(myCustomAdapter);
+```
+
+### Extraction Modes
+
+**Manual extraction** with user guidance:
+
+```typescript
+const result = await skillBank.extractManual(trajectory, {
+  turnRange: [5, 15],           // Extract specific turns
+  suggestedName: 'my-skill',
+  tags: ['debugging'],
+});
+```
+
+**Automatic extraction** using LLM:
+
+```typescript
+skillBank.setLLMProvider(myLLMProvider);
+
+const results = await skillBank.extractAutomatic(trajectory, {
+  minConfidence: 0.7,
+});
+```
+
+### Storage
+
+**Filesystem storage** (OpenSkills-compatible):
+
+```typescript
+const skillBank = createSkillBank({
+  storage: {
+    type: 'filesystem',
+    basePath: '~/.skills',
+    openSkillsCompatible: true,  // YAML frontmatter + Markdown
+  },
+});
+```
+
+**Memory storage** (for testing):
+
+```typescript
+const skillBank = createSkillBank({
+  storage: { type: 'memory' },
+});
+```
+
+Skills are stored in the OpenSkills format:
+
+```
+skills/
+├── my-skill/
+│   ├── SKILL.md           # Skill content (YAML + Markdown)
+│   └── .skilltree.json    # Metadata and lineage
+└── .versions/
+    └── my-skill/
+        ├── 1.0.0.json     # Version snapshots
+        └── 1.1.0.json
+```
+
+### Versioning
+
+Semantic versioning with full lineage tracking:
+
+```typescript
+// Create new version
+const v2 = await skillBank.createVersion('my-skill', updates, {
+  bumpType: 'minor',
+  changelog: 'Added alternative solution',
+});
+
+// Get version history
+const history = await skillBank.getVersionHistory('my-skill');
+
+// Rollback to previous version
+const restored = await skillBank.rollbackSkill('my-skill', '1.0.0');
+
+// Compare versions
+const diff = await skillBank.compareVersions('my-skill', '1.0.0', '2.0.0');
+```
+
+### Skill Evolution
+
+Fork and merge skills:
+
+```typescript
+// Fork for a specialized use case
+const forked = await skillBank.forkSkill('my-skill', {
+  newId: 'my-skill-react',
+  newName: 'My Skill (React variant)',
+  reason: 'Specialized for React projects',
+});
+
+// Merge improvements back
+const merged = await skillBank.lineageTracker.mergeSkill(
+  'my-skill',
+  'my-skill-react',
+  { fields: ['solution', 'examples'] }
+);
+```
+
+### Events
+
+Subscribe to skill bank events:
+
+```typescript
+const unsubscribe = skillBank.on((event) => {
+  switch (event.type) {
+    case 'skill:created':
+      console.log('New skill:', event.skill.name);
+      break;
+    case 'extraction:completed':
+      console.log('Extraction confidence:', event.result.confidence);
+      break;
+  }
+});
+```
+
+## API Reference
+
+### SkillBank
+
+Main orchestrator class.
+
+| Method | Description |
+|--------|-------------|
+| `initialize()` | Initialize storage (required before use) |
+| `parseSession(content, adapter?)` | Parse session into trajectory |
+| `extractManual(trajectory, options?)` | Extract skill with guidance |
+| `extractAutomatic(trajectory, options?)` | Extract skills using LLM |
+| `getSkill(id, version?)` | Get skill by ID |
+| `listSkills(filter?)` | List skills with optional filter |
+| `searchSkills(query)` | Search skills by text |
+| `saveSkill(skill)` | Save or update a skill |
+| `deleteSkill(id, version?)` | Delete a skill |
+| `createVersion(id, updates, options?)` | Create new version |
+| `forkSkill(id, options)` | Fork a skill |
+| `getVersionHistory(id)` | Get version history |
+| `rollbackSkill(id, version)` | Rollback to version |
+| `on(handler)` | Subscribe to events |
+
+### Versioning Utilities
+
+```typescript
+import {
+  parseVersion,
+  compareVersions,
+  bumpVersion,
+  satisfiesRange,
+} from 'skill-tree';
+
+bumpVersion('1.2.3', 'minor');  // '1.3.0'
+satisfiesRange('1.5.0', '^1.2.0');  // true
+```
+
+## Skill Format
+
+Skills use YAML frontmatter + Markdown (OpenSkills-compatible):
+
+```markdown
+---
+name: typescript-esm-import-fix
+description: |
+  Fix TypeScript ES module import errors by adding .js extension
+version: 1.0.0
+author: extracted
+status: active
+date: 2024-01-15
+tags:
+  - typescript
+  - esm
+  - imports
+---
+
+## Problem
+
+TypeScript with ES modules requires explicit .js extensions in imports.
+
+## Trigger Conditions
+
+- **error**: `Cannot find module './utils'`
+- **pattern**: `import .* from '\./[^']+(?<!\.js)'`
+
+## Solution
+
+1. Add `.js` extension to relative imports
+2. Even for `.ts` files, use `.js` in the import path
+
+## Verification
+
+Run `tsc` and verify no module resolution errors.
+
+## Examples
+
+### Basic import fix
+
+**Before:**
+```typescript
+import { helper } from './utils'
+```
+
+**After:**
+```typescript
+import { helper } from './utils.js'
+```
+```
+
+## License
+
+MIT

package/references/skill-tree/docs/GAPS_v1.md

@@ -0,0 +1,221 @@
+# Integration Gaps Tracking - v1
+
+This document tracks the remaining work to complete the skill indexer integration as defined in [INTEGRATION_PLAN.md](./INTEGRATION_PLAN.md).
+
+## Status Overview
+
+| Phase | Status | Completion |
+|-------|--------|------------|
+| Phase 1: Import Pipeline | ✅ Complete | 100% |
+| Phase 2: CLI Unification | ✅ Complete | 100% |
+| Phase 3: Unified Storage | ✅ Complete | 100% |
+| Phase 4: Direct Integration | ✅ Complete | 100% |
+| Phase 5: Bidirectional Sync | ✅ Complete | 100% |
+
+---
+
+## High Priority Items
+
+### 1. Unified Configuration System
+
+**Status**: ✅ Complete
+
+**Goal**: Single config file for both skill-tree and indexer settings.
+
+**Location**: `~/.skill-tree/config.yaml`
+
+**Tasks**:
+- [x] Create `src/config/` module
+- [x] Define `SkillTreeConfig` interface
+- [x] Implement config file loading with defaults
+- [x] Support environment variable substitution
+- [x] Integrate with CLI global options
+- [x] Integrate with IndexerService
+
+**Files Created**:
+- `src/config/types.ts` - Configuration type definitions
+- `src/config/loader.ts` - Config loading and merging logic
+- `src/config/index.ts` - Module exports
+- `src/cli/commands/config.ts` - CLI config command
+
+**Config Schema**:
+```yaml
+storage:
+  type: sqlite | filesystem | memory
+  path: ~/.skill-tree/skills.db
+
+indexer:
+  github_token: ${GITHUB_TOKEN}
+  anthropic_key: ${ANTHROPIC_API_KEY}
+  sources:
+    - https://github.com/VoltAgent/awesome-agent-skills
+  batch_size: 10
+  min_confidence: 0.7
+
+matching:
+  embedding_model: text-embedding-3-small
+  similarity_threshold: 0.8
+
+sync:
+  auto_check: false
+  check_interval_hours: 24
+  conflict_resolution: prompt
+
+cli:
+  output_format: table
+  color: true
+  quiet: false
+```
+
+---
+
+### 2. IndexerService Direct Execution
+
+**Status**: ✅ Complete
+
+**Goal**: CLI commands execute directly instead of delegating to subprocess.
+
+**Tasks**:
+- [x] Update `IndexerService.initialize()` to properly load scraper modules
+- [x] Implement `scrape()` method with actual scraper integration
+- [x] Implement `classify()` method with actual indexer integration
+- [x] Implement `detectRelationships()` method
+- [x] Implement `getTaxonomyTree()` using storage adapter
+- [x] Implement `getStats()` method
+- [x] Update CLI commands to use IndexerService directly
+- [x] Add fallback to subprocess if modules unavailable
+
+**Files Modified**:
+- `src/services/indexer.ts` - Full IndexerService implementation
+- `src/cli/commands/indexer/scrape.ts` - Direct execution with fallback
+- `src/cli/commands/indexer/classify.ts` - Direct execution with fallback
+- `src/cli/commands/indexer/taxonomy.ts` - Direct execution with fallback
+- `src/cli/commands/indexer/relationships.ts` - Direct execution with fallback
+- `src/cli/commands/indexer/stats.ts` - Direct execution with fallback
+
+---
+
+## Medium Priority Items
+
+### 3. Direct SkillBank Integration
+
+**Status**: ✅ Complete
+
+**Goal**: IndexerService writes directly to SkillBank, no export/import step.
+
+**Tasks**:
+- [x] Implement `IndexerService.scrapeAndIndex()` method
+- [x] Convert scraped skills on-the-fly using converter
+- [x] Store directly via SkillBank.addSkill()
+- [x] Auto-place in taxonomy after save
+- [x] Update `skill-tree index scrape` to use direct integration
+
+**Files Modified**:
+- `src/services/indexer.ts` - Added `scrapeAndIndex()` method
+- `src/skill-bank.ts` - Added `addSkill()` method with taxonomy placement
+- `src/cli/commands/indexer/scrape.ts` - Added `--import` flag for auto-import
+
+**Workflow After**:
+```bash
+# One command does everything
+skill-tree index scrape https://github.com/org/repo
+# Skills immediately available in skill-tree
+skill-tree list
+```
+
+---
+
+### 4. SkillBank Event System
+
+**Status**: ✅ Complete
+
+**Goal**: Event-driven hooks for skill lifecycle.
+
+**Tasks**:
+- [x] Add EventEmitter to SkillBank (already existed via `on()`)
+- [x] Define event types: `skill:created`, `skill:updated`, `skill:deleted`
+- [x] Emit events from `addSkill()`, `updateSkill()`, `deleteSkill()`
+- [x] Hook relationship detection to `skill:created` for imported skills
+- [x] Add `on()` and `off()` methods to SkillBank API
+- [x] Add `onEvent()` for typed event subscriptions
+- [x] Add `getStorage()` for advanced integration
+
+**Files Modified**:
+- `src/skill-bank.ts` - Enhanced event system with `off()`, `onEvent()`, `getStorage()`
+
+**Event Interface**:
+```typescript
+interface SkillEvent {
+  type: 'skill:created' | 'skill:updated' | 'skill:deleted' | 'skill:deprecated';
+  skill: Skill;
+  timestamp: Date;
+  source?: 'manual' | 'imported' | 'extracted';
+}
+```
+
+---
+
+## Lower Priority Items
+
+### 5. Bidirectional Sync
+
+**Status**: ✅ Complete
+
+**Goal**: Keep skills in sync with remote sources.
+
+**Tasks**:
+- [x] Create `src/services/sync.ts`
+- [x] Define `SkillSyncState` type
+- [x] Implement `checkForUpdates()` - compare ETags/versions
+- [x] Implement `pullRemoteChanges()` - fetch and merge
+- [x] Implement `detectConflicts()` - local vs remote changes
+- [x] Add `skill-tree index sync` command with options:
+  - [x] `--status` - show sync state
+  - [x] `--pull` - pull remote changes
+  - [x] `--dry-run` - preview changes
+
+**Files Created**:
+- `src/services/sync.ts` - Full SyncService implementation
+
+**Files Modified**:
+- `src/cli/commands/indexer/sync.ts` - Updated with sync status and pull commands
+
+**Note**: SQLite sync_state table not yet added to schema (sync states stored in memory for now). This can be added in a future iteration.
+
+---
+
+## Open Questions (Decisions Made)
+
+| Question | Status | Decision |
+|----------|--------|----------|
+| Taxonomy ownership | ✅ Resolved | Preserve original, allow re-classification on demand |
+| Deduplication strategy | ✅ Resolved | Use `externalSource.url` as dedup key |
+| Version conflict resolution | ✅ Resolved | Default to manual, with `--resolve` flag for automation |
+| Namespace separation | ✅ Resolved | Use `source.type` field, no namespace prefix |
+
+---
+
+## Success Criteria Checklist
+
+- [x] Single CLI (`skill-tree`) for all operations
+- [x] Single database with skills from all sources
+- [x] Taxonomy browsing works for all skills
+- [x] Relationships detected across indexed and extracted skills
+- [x] Clear provenance tracking (source: extracted vs indexed)
+- [x] Seamless workflow from discovery to usage
+
+---
+
+## Change Log
+
+| Date | Change |
+|------|--------|
+| 2025-01-XX | Initial gap analysis created |
+| 2025-01-XX | Phases 1-3 implemented |
+| 2025-01-XX | Tests added (56 tests) |
+| 2025-02-01 | Unified configuration system implemented |
+| 2025-02-01 | IndexerService direct execution implemented |
+| 2025-02-01 | Direct SkillBank integration implemented |
+| 2025-02-01 | SkillBank event system enhanced |
+| 2025-02-01 | Bidirectional sync service implemented |
+| 2025-02-01 | All phases complete |