claude-code-swarm 0.3.3 → 0.3.5

package/references/skill-tree/docs/design/federated-skill-trees.md

@@ -0,0 +1,524 @@
+# Federated Skill Trees Design Spec
+
+## Status
+**Draft** | Last Updated: 2026-02-04
+
+## Summary
+
+Enable users to maintain personal skill repositories while selectively syncing with team and community skill repositories. This extends SkillBank with optional remote capabilities, allowing skills to flow between repositories through import/export operations with upstream tracking.
+
+---
+
+## Background & Motivation
+
+### The Problem
+
+Agents accumulate skills through learning and extraction. Currently, skills live in a single storage location. In practice, users want:
+
+1. **Personal skill libraries** - Private skills they develop and customize
+2. **Team skill sharing** - Contribute to and benefit from team knowledge
+3. **Community skills** - Access public skill repositories (read-only)
+
+These are distinct concerns with different access patterns:
+
+| Repository | Ownership | Access | Typical Operations |
+|------------|-----------|--------|-------------------|
+| Personal | Individual | Full control | Create, modify, delete freely |
+| Team | Shared | Read + contribute | Import skills, submit contributions |
+| Community | Public | Read-only | Browse, import, track updates |
+
+### Why Not Just Use Git?
+
+Git handles version control, but doesn't address:
+- **Skill-level operations**: Import one skill, not the whole repo
+- **Upstream tracking**: Know when an imported skill has updates
+- **Selective sync**: Choose which skills to import/share
+- **Identity preservation**: Skills maintain provenance across repos
+
+### Design Goals
+
+1. **Progressive enhancement** - Simple usage stays simple; add remotes only when needed
+2. **Familiar API** - Remote operations feel like natural extensions of SkillBank
+3. **Explicit over implicit** - User controls what syncs, when, and how
+4. **Offline-first** - Local operations work without network; sync when ready
+
+---
+
+## Prior Art & Alternatives Considered
+
+### Alternative 1: Separate FederatedSkillBank Class
+
+```typescript
+const federation = new FederatedSkillBank({
+  sources: {
+    personal: { bank: myBank, writable: true },
+    team: { bank: teamBank, writable: false },
+  },
+});
+```
+
+**Rejected because:**
+- Requires users to learn a new class
+- Awkward migration from simple SkillBank usage
+- Two mental models instead of one
+
+### Alternative 2: HierarchicalSyncAdapter (Implemented in Phase 1)
+
+Manages tiers (personal/team/global) as directories within a single base path, with promotion/demotion between tiers.
+
+**Limitations:**
+- Tiers share a base directory, not truly independent repos
+- Designed for scope-based organization, not repo federation
+- Promotion/demotion is about visibility, not selective sync
+
+**Conclusion:** HierarchicalSyncAdapter is useful for organizing skills by scope within a single system. Federation extends this to connect independent repositories.
+
+### Alternative 3: CompositeSkillBank Wrapper
+
+```typescript
+const composite = new CompositeSkillBank([bank1, bank2, bank3]);
+```
+
+**Rejected because:**
+- Hides which bank skills come from
+- Unclear write semantics (which bank receives saves?)
+- Doesn't address upstream tracking
+
+### Chosen Approach: Extend SkillBank with Remote Capabilities
+
+Add optional `addRemote()` method that enables remote operations on an existing SkillBank. Remote methods are only available after adding remotes.
+
+**Benefits:**
+- No new classes to learn
+- Gradual adoption - start simple, add remotes later
+- Clear ownership - your SkillBank is the source of truth
+- Remotes are explicit connections, not merged views
+
+---
+
+## Detailed Design
+
+### Core Concepts
+
+#### 1. Remotes
+
+A remote is a connection to another skill repository. Remotes are:
+- Named (e.g., `'team'`, `'community'`)
+- Configured with URL and access level
+- Cached locally for offline access
+
+```typescript
+bank.addRemote('team', {
+  url: 'git@github.com:org/team-skills.git',
+  access: 'read-write',
+});
+```
+
+#### 2. Import Modes: Link vs Fork
+
+When importing a skill from a remote, users choose:
+
+| Mode | Relationship | Use Case |
+|------|--------------|----------|
+| **Link** | Tracks upstream | "I want this skill and future updates" |
+| **Fork** | Independent copy | "I want to customize this my way" |
+
+**Linked skills:**
+- ID prefixed: `@team/skill-name`
+- Track upstream version in `skill.upstream`
+- Can check for updates via `checkUpstream()`
+- Can pull updates via `pullUpstream()`
+- Can unlink to convert to fork
+
+**Forked skills:**
+- Custom ID or prefixed
+- Record origin in `skill.derivedFrom`
+- No ongoing sync relationship
+- Fully independent
+
+#### 3. Skill Identity
+
+Imported skills use namespaced IDs to prevent collisions:
+
+```
+@{remote}/{original-id}
+```
+
+Examples:
+- `@team/connection-pool`
+- `@community/rate-limiter`
+
+Users can override with custom IDs when importing.
+
+#### 4. Upstream Tracking
+
+Linked skills store upstream metadata:
+
+```typescript
+interface Skill {
+  // ... existing fields ...
+
+  upstream?: {
+    remote: string;      // Remote name
+    skillId: string;     // ID in remote
+    version: string;     // Version when last synced
+    syncedAt: Date;      // When last synced
+  };
+}
+```
+
+### API Design
+
+#### Adding Remotes
+
+```typescript
+// Add a remote repository
+bank.addRemote(name: string, config: RemoteConfig): void
+
+interface RemoteConfig {
+  url: string;                          // Git URL or local path
+  access: 'read-only' | 'read-write';   // Access level
+  localCache?: string;                  // Cache location (default: .remotes/{name})
+}
+
+// Remove a remote
+bank.removeRemote(name: string): void
+
+// List configured remotes
+bank.listRemotes(): string[]
+
+// Check if a remote exists
+bank.hasRemote(name: string): boolean
+```
+
+#### Browsing Remotes
+
+```typescript
+// List skills from a remote (doesn't import)
+bank.browse(remote: string, filter?: SkillFilter): Promise<Skill[]>
+
+// Get a specific skill from remote
+bank.browseSkill(remote: string, skillId: string): Promise<Skill | null>
+```
+
+#### Importing Skills
+
+```typescript
+// Import a skill from remote to local
+bank.import(
+  remote: string,
+  skillId: string,
+  options?: ImportOptions
+): Promise<Skill>
+
+interface ImportOptions {
+  mode?: 'link' | 'fork';  // Default: 'link'
+  as?: string;             // Local ID (default: @remote/skillId)
+}
+```
+
+#### Sharing Skills
+
+```typescript
+// Share a local skill to a remote
+bank.share(
+  localId: string,
+  remote: string,
+  options?: ShareOptions
+): Promise<ShareResult>
+
+interface ShareOptions {
+  as?: string;        // Remote ID (default: strip @prefix from localId)
+  message?: string;   // Commit message
+}
+
+interface ShareResult {
+  success: boolean;
+  remoteId: string;
+  commitHash?: string;
+}
+```
+
+#### Upstream Sync (for Linked Skills)
+
+```typescript
+// Check for updates to linked skills
+bank.checkUpstream(): Promise<UpstreamUpdate[]>
+
+interface UpstreamUpdate {
+  localId: string;
+  remote: string;
+  remoteId: string;
+  localVersion: string;
+  remoteVersion: string;
+  behind: number;  // Commits behind
+}
+
+// Pull upstream changes for a linked skill
+bank.pullUpstream(
+  localId: string,
+  options?: PullOptions
+): Promise<Skill>
+
+interface PullOptions {
+  strategy?: 'merge' | 'overwrite';  // Default: 'merge'
+}
+
+// Convert a link to a fork (stop tracking)
+bank.unlink(localId: string): Promise<void>
+```
+
+#### Unified Search (Optional)
+
+```typescript
+// Search across local + remotes
+bank.search(query: string, options?: SearchOptions): Promise<SearchResult[]>
+
+interface SearchOptions {
+  includeRemotes?: boolean;  // Default: false
+  remotes?: string[];        // Specific remotes to search
+}
+
+interface SearchResult {
+  skill: Skill;
+  source: string;  // 'local' or remote name
+  score: number;
+}
+```
+
+### Storage & Caching
+
+#### Directory Structure
+
+```
+my-skills/                    # User's skill bank
+├── skills/                   # Local skills
+│   ├── my-pattern/
+│   ├── @team/               # Imported from team (linked)
+│   │   └── connection-pool/
+│   └── forked-util/         # Forked from community
+├── .remotes/                # Cached remote repos
+│   ├── team/                # Git clone of team repo
+│   └── community/           # Git clone of community repo
+└── .skillbank/
+    └── remotes.json         # Remote configurations
+```
+
+#### Remote Caching Strategy
+
+1. **On first access**: Clone remote repository to `.remotes/{name}/`
+2. **On browse/import**: Fetch latest from remote
+3. **Offline mode**: Use cached data if fetch fails
+4. **Manual refresh**: `bank.refreshRemote(name)`
+
+### Conflict Handling
+
+#### Import Conflicts
+
+If local skill ID conflicts with import:
+
+```typescript
+// This would fail
+await bank.import('team', 'my-skill');
+// Error: Skill 'my-skill' already exists locally
+
+// Solutions:
+await bank.import('team', 'my-skill', { as: '@team/my-skill' });
+await bank.import('team', 'my-skill', { as: 'team-my-skill' });
+```
+
+#### Upstream Merge Conflicts
+
+When pulling updates to a locally-modified linked skill:
+
+```typescript
+const result = await bank.pullUpstream('@team/skill', {
+  strategy: 'merge'
+});
+
+if (result.conflicts) {
+  // Manual resolution needed
+  console.log(result.conflicts);
+  // [{ field: 'solution', local: '...', remote: '...', base: '...' }]
+
+  // Resolve using existing SkillMerger
+  const resolved = await bank.resolveConflicts('@team/skill', {
+    solution: 'merged content',
+  });
+}
+```
+
+---
+
+## Usage Examples
+
+### Example 1: Solo Developer Adding Team Repo
+
+```typescript
+// Start with personal skills
+const bank = createSkillBank({
+  storage: { type: 'filesystem', basePath: './my-skills' },
+});
+await bank.initialize();
+
+// Later, connect to team
+bank.addRemote('team', {
+  url: 'git@github.com:org/team-skills.git',
+  access: 'read-write',
+});
+
+// Browse what's available
+const teamSkills = await bank.browse('team');
+console.log(`Team has ${teamSkills.length} skills`);
+
+// Import useful patterns
+await bank.import('team', 'error-handling');
+await bank.import('team', 'api-patterns');
+
+// These are now local as @team/error-handling, @team/api-patterns
+```
+
+### Example 2: Contributing Back to Team
+
+```typescript
+// Create a new skill locally
+await bank.saveSkill({
+  id: 'my-discovery',
+  name: 'Database Migration Pattern',
+  // ...
+});
+
+// Share with team
+const result = await bank.share('my-discovery', 'team', {
+  message: 'Add database migration pattern',
+});
+
+console.log(`Shared as ${result.remoteId}`);
+```
+
+### Example 3: Keeping Skills Updated
+
+```typescript
+// Check what's changed upstream
+const updates = await bank.checkUpstream();
+
+for (const update of updates) {
+  console.log(`${update.localId}: ${update.localVersion} → ${update.remoteVersion}`);
+}
+
+// Pull updates for specific skill
+await bank.pullUpstream('@team/api-patterns');
+
+// Or pull all updates
+for (const update of updates) {
+  await bank.pullUpstream(update.localId);
+}
+```
+
+### Example 4: Forking for Customization
+
+```typescript
+// Fork instead of link
+await bank.import('community', 'rate-limiter', {
+  mode: 'fork',
+  as: 'my-rate-limiter',
+});
+
+// Customize freely - no upstream tracking
+const skill = await bank.getSkill('my-rate-limiter');
+skill.solution = 'My improved version...';
+await bank.saveSkill(skill);
+```
+
+### Example 5: Converting Link to Fork
+
+```typescript
+// Initially linked
+await bank.import('team', 'auth-pattern');
+
+// Decide to diverge
+await bank.unlink('@team/auth-pattern');
+
+// Now it's a fork - skill.upstream removed, origin recorded in derivedFrom
+```
+
+---
+
+## Implementation Plan
+
+### Phase 1: Foundation (Completed)
+- [x] Add namespace types (SkillScope, SkillVisibility, SkillNamespace)
+- [x] Implement namespace-aware storage filtering
+- [x] Create HierarchicalSyncAdapter for tier-based organization
+- [x] Add namespace support to SkillBank
+
+### Phase 2: Remote Infrastructure
+- [ ] Add `upstream` field to Skill type
+- [ ] Implement remote configuration storage (`.skillbank/remotes.json`)
+- [ ] Implement remote caching (git clone/fetch to `.remotes/`)
+- [ ] Add `addRemote()`, `removeRemote()`, `listRemotes()` methods
+
+### Phase 3: Browse & Import
+- [ ] Implement `browse()` - list skills from cached remote
+- [ ] Implement `import()` with link/fork modes
+- [ ] Handle ID namespacing (@remote/skillId)
+- [ ] Add import conflict detection
+
+### Phase 4: Upstream Sync
+- [ ] Implement `checkUpstream()` - compare local vs remote versions
+- [ ] Implement `pullUpstream()` - fetch and merge updates
+- [ ] Implement `unlink()` - convert link to fork
+- [ ] Integrate with existing SkillMerger for conflicts
+
+### Phase 5: Sharing
+- [ ] Implement `share()` - push skill to writable remote
+- [ ] Handle remote conflicts (skill already exists)
+- [ ] Add commit message support
+
+### Phase 6: Advanced Features
+- [ ] Unified search across local + remotes
+- [ ] Bulk operations (import multiple, sync all)
+- [ ] Remote health/status reporting
+- [ ] PR workflow support for share operations
+
+---
+
+## Open Questions
+
+1. **Nested imports**: If team imports from community, and you import from team, should you see the chain?
+
+2. **Circular dependencies**: Team imports your skill, you import team's version. How to handle?
+
+3. **Version pinning**: Should links track specific versions or always latest?
+
+4. **Partial sync**: Subscribe to specific tags/categories from a remote instead of browsing all?
+
+5. **Authentication**: How to handle private repos requiring auth?
+
+---
+
+## Appendix: Relationship to Existing Features
+
+### HierarchicalSyncAdapter
+
+The existing HierarchicalSyncAdapter manages **tiers within a single system**:
+- Personal, team, global as directories under one base path
+- Skills promoted/demoted between tiers
+- Useful for visibility/scope organization
+
+Federation manages **connections between independent systems**:
+- Each repo is independent with its own storage
+- Skills imported/shared between repos
+- Useful for cross-organization collaboration
+
+These are complementary:
+- Use HierarchicalSyncAdapter to organize your personal repo by scope
+- Use federation to connect to team/community repos
+
+### GitSyncAdapter
+
+GitSyncAdapter handles git operations for a single repository (pull, push, conflict resolution). Federation will use GitSyncAdapter internally for each remote's cache.
+
+### Namespace & Visibility
+
+The namespace types (SkillScope, SkillVisibility) apply to local organization. Federation adds a layer above this - imported skills can have any namespace, but their origin is tracked via `upstream` or `derivedFrom`.