specweave 0.18.1 → 0.20.1

package/plugins/specweave-confluent/skills/confluent-schema-registry/SKILL.md

@@ -0,0 +1,316 @@
+---
+name: confluent-schema-registry
+description: Schema Registry expert for Avro, Protobuf, and JSON Schema management. Covers schema evolution strategies, compatibility modes, validation, and best practices for managing schemas in Confluent Cloud and self-hosted Schema Registry. Activates for schema registry, avro, protobuf, json schema, schema evolution, compatibility modes, schema validation.
+---
+
+# Confluent Schema Registry Skill
+
+Expert knowledge of Confluent Schema Registry for managing Avro, Protobuf, and JSON Schema schemas in Kafka ecosystems.
+
+## What I Know
+
+### Schema Formats
+
+**Avro** (Most Popular):
+- Binary serialization format
+- Schema evolution support
+- Smaller message size vs JSON
+- Self-describing with schema ID in header
+- Best for: High-throughput applications, data warehousing
+
+**Protobuf** (Google Protocol Buffers):
+- Binary serialization
+- Strong typing with .proto files
+- Language-agnostic (Java, Python, Go, C++, etc.)
+- Efficient encoding
+- Best for: Polyglot environments, gRPC integration
+
+**JSON Schema**:
+- Human-readable text format
+- Easy debugging
+- Widely supported
+- Larger message size
+- Best for: Development, debugging, REST APIs
+
+### Compatibility Modes
+
+| Mode | Producer Can | Consumer Can | Use Case |
+|------|-------------|-------------|----------|
+| **BACKWARD** | Remove fields, add optional fields | Read old data with new schema | Most common, safe for consumers |
+| **FORWARD** | Add fields, remove optional fields | Read new data with old schema | Safe for producers |
+| **FULL** | Add/remove optional fields only | Bi-directional compatibility | Both producers and consumers upgrade independently |
+| **NONE** | Any change | Must coordinate upgrades | Development only, NOT production |
+| **BACKWARD_TRANSITIVE** | BACKWARD across all versions | Read any old data | Strictest backward compatibility |
+| **FORWARD_TRANSITIVE** | FORWARD across all versions | Read any new data | Strictest forward compatibility |
+| **FULL_TRANSITIVE** | FULL across all versions | Complete bi-directional | Strictest overall |
+
+**Default**: `BACKWARD` (recommended for production)
+
+### Schema Evolution Strategies
+
+**Adding Fields**:
+```avro
+// V1
+{
+  "type": "record",
+  "name": "User",
+  "fields": [
+    {"name": "id", "type": "long"},
+    {"name": "name", "type": "string"}
+  ]
+}
+
+// V2 - BACKWARD compatible (added optional field with default)
+{
+  "type": "record",
+  "name": "User",
+  "fields": [
+    {"name": "id", "type": "long"},
+    {"name": "name", "type": "string"},
+    {"name": "email", "type": ["null", "string"], "default": null}
+  ]
+}
+```
+
+**Removing Fields** (BACKWARD compatible):
+```avro
+// V1
+{"name": "address", "type": "string"}
+
+// V2 - Remove field (old consumers will ignore it)
+// Field removed from schema
+```
+
+**Changing Field Types** (Breaking Change!):
+```avro
+// ❌ BREAKING - Cannot change string to int
+{"name": "age", "type": "string"} → {"name": "age", "type": "int"}
+
+// ✅ SAFE - Use union types
+{"name": "age", "type": ["string", "int"], "default": "unknown"}
+```
+
+## When to Use This Skill
+
+Activate me when you need help with:
+- Schema evolution strategies ("How do I evolve my Avro schema?")
+- Compatibility mode selection ("Which compatibility mode for production?")
+- Schema validation ("Validate my Avro schema")
+- Best practices ("Schema Registry best practices")
+- Schema registration ("Register Avro schema with Schema Registry")
+- Debugging schema issues ("Schema compatibility error")
+- Format comparison ("Avro vs Protobuf vs JSON Schema")
+
+## Best Practices
+
+### 1. Always Use Compatible Evolution
+
+✅ **DO**:
+- Add optional fields with defaults
+- Remove optional fields
+- Use union types for flexibility
+- Test schema changes in staging first
+
+❌ **DON'T**:
+- Change field types
+- Remove required fields
+- Rename fields (add new + deprecate old)
+- Use `NONE` compatibility in production
+
+### 2. Schema Naming Conventions
+
+**Hierarchical Namespaces**:
+```
+com.company.domain.EntityName
+com.acme.ecommerce.Order
+com.acme.ecommerce.OrderLineItem
+```
+
+**Subject Naming** (Kafka topics):
+- `<topic-name>-value` - For record values
+- `<topic-name>-key` - For record keys
+- Example: `orders-value`, `orders-key`
+
+### 3. Schema Registry Configuration
+
+**Producer** (with Avro):
+```javascript
+const { Kafka } = require('kafkajs');
+const { SchemaRegistry } = require('@kafkajs/confluent-schema-registry');
+
+const registry = new SchemaRegistry({
+  host: 'https://schema-registry:8081',
+  auth: {
+    username: 'SR_API_KEY',
+    password: 'SR_API_SECRET'
+  }
+});
+
+// Register schema
+const schema = `
+{
+  "type": "record",
+  "name": "User",
+  "fields": [
+    {"name": "id", "type": "long"},
+    {"name": "name", "type": "string"}
+  ]
+}
+`;
+
+const { id } = await registry.register({
+  type: SchemaType.AVRO,
+  schema
+});
+
+// Encode message with schema
+const payload = await registry.encode(id, {
+  id: 1,
+  name: 'John Doe'
+});
+
+await producer.send({
+  topic: 'users',
+  messages: [{ value: payload }]
+});
+```
+
+**Consumer** (with Avro):
+```javascript
+const consumer = kafka.consumer({ groupId: 'user-processor' });
+
+await consumer.subscribe({ topic: 'users' });
+
+await consumer.run({
+  eachMessage: async ({ message }) => {
+    // Decode message (schema ID is in header)
+    const decodedMessage = await registry.decode(message.value);
+    console.log(decodedMessage); // { id: 1, name: 'John Doe' }
+  }
+});
+```
+
+### 4. Schema Validation Workflow
+
+**Before Registering**:
+1. Validate schema syntax (Avro JSON, .proto, JSON Schema)
+2. Check compatibility with existing versions
+3. Test with sample data
+4. Register in dev/staging first
+5. Deploy to production after validation
+
+**CLI Validation**:
+```bash
+# Check compatibility (before registering)
+curl -X POST http://localhost:8081/compatibility/subjects/users-value/versions/latest \
+  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
+  -d '{"schema": "{...}"}'
+
+# Register schema
+curl -X POST http://localhost:8081/subjects/users-value/versions \
+  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
+  -d '{"schema": "{...}"}'
+```
+
+## Common Issues & Solutions
+
+### Issue 1: Schema Compatibility Error
+
+**Error**:
+```
+Schema being registered is incompatible with an earlier schema
+```
+
+**Root Cause**: Violates compatibility mode (e.g., removed required field with BACKWARD mode)
+
+**Solution**:
+1. Check current compatibility mode:
+   ```bash
+   curl http://localhost:8081/config/users-value
+   ```
+2. Fix schema to be compatible OR change mode (carefully!)
+3. Validate before registering:
+   ```bash
+   curl -X POST http://localhost:8081/compatibility/subjects/users-value/versions/latest \
+     -d '{"schema": "{...}"}'
+   ```
+
+### Issue 2: Schema Not Found
+
+**Error**:
+```
+Subject 'users-value' not found
+```
+
+**Root Cause**: Schema not registered yet OR wrong subject name
+
+**Solution**:
+1. List all subjects:
+   ```bash
+   curl http://localhost:8081/subjects
+   ```
+2. Register schema if missing
+3. Check subject naming convention (`<topic>-key` or `<topic>-value`)
+
+### Issue 3: Message Deserialization Failed
+
+**Error**:
+```
+Unknown magic byte!
+```
+
+**Root Cause**: Message not encoded with Schema Registry (missing magic byte + schema ID)
+
+**Solution**:
+1. Ensure producer uses Schema Registry encoder
+2. Check message format: [magic_byte(1) + schema_id(4) + payload]
+3. Use `@kafkajs/confluent-schema-registry` library
+
+## Schema Evolution Decision Tree
+
+```
+Need to change schema?
+├─ Adding new field?
+│  ├─ Required field? → Add with default value (BACKWARD)
+│  └─ Optional field? → Add with default null (BACKWARD)
+│
+├─ Removing field?
+│  ├─ Required field? → ❌ BREAKING CHANGE (coordinate upgrade)
+│  └─ Optional field? → ✅ BACKWARD compatible
+│
+├─ Changing field type?
+│  ├─ Compatible types (e.g., int → long)? → Use union types
+│  └─ Incompatible types? → ❌ BREAKING CHANGE (add new field, deprecate old)
+│
+└─ Renaming field?
+   └─ ❌ BREAKING CHANGE → Add new field + mark old as deprecated
+```
+
+## Avro vs Protobuf vs JSON Schema Comparison
+
+| Feature | Avro | Protobuf | JSON Schema |
+|---------|------|----------|-------------|
+| **Encoding** | Binary | Binary | Text (JSON) |
+| **Message Size** | Small (90% smaller) | Small (80% smaller) | Large (baseline) |
+| **Human Readable** | No | No | Yes |
+| **Schema Evolution** | Excellent | Good | Fair |
+| **Language Support** | Java, Python, C++ | 20+ languages | Universal |
+| **Performance** | Very Fast | Very Fast | Slower |
+| **Debugging** | Harder | Harder | Easy |
+| **Best For** | Data warehousing, ETL | Polyglot, gRPC | REST APIs, dev |
+
+**Recommendation**:
+- **Production**: Avro (best balance)
+- **Polyglot teams**: Protobuf
+- **Development/Debugging**: JSON Schema
+
+## References
+
+- Schema Registry REST API: https://docs.confluent.io/platform/current/schema-registry/develop/api.html
+- Avro Specification: https://avro.apache.org/docs/current/spec.html
+- Protobuf Guide: https://developers.google.com/protocol-buffers
+- JSON Schema Spec: https://json-schema.org/
+
+---
+
+**Invoke me when you need schema management, evolution strategies, or compatibility guidance!**

package/plugins/specweave-github/agents/github-task-splitter/AGENT.md

@@ -139,9 +139,9 @@ Total Tasks: 2
 For each repository, I create a tracking issue:
 
 ```markdown
-# [INC-0015] Shopping Cart - Frontend Tasks
+# [FS-25-11-14] Shopping Cart - Frontend Tasks
 
-Part of increment 0015-shopping-cart
+Part of increment 0015-shopping-cart (Feature: FS-25-11-14)
 
 ## Tasks (5)
 - [ ] T-001: Create CartItem component

package/plugins/specweave-github/agents/user-story-updater/AGENT.md

@@ -0,0 +1,148 @@
+---
+name: user-story-updater
+description: Updates GitHub issues for user stories with proper ACs and tasks. Activates for update user story issue, fix GitHub issue format, add checkable ACs, refresh user story issue, sync user story to GitHub.
+---
+
+# User Story Updater Agent
+
+**Role**: Updates GitHub issues for individual user stories to include checkable acceptance criteria and task connections.
+
+**Activates For**:
+- "Update user story issue"
+- "Fix GitHub issue format for US-004"
+- "Add checkable ACs to GitHub issue"
+- "Refresh user story issue #501"
+- "Sync user story to GitHub"
+
+**Capabilities**:
+
+1. **Parse User Story Files** (`us-*.md` from living docs)
+   - Extract frontmatter (id, epic, status, priority)
+   - Parse user story description (As a... I want... So that...)
+   - Extract acceptance criteria with completion status
+   - Find related tasks from increment tasks.md
+
+2. **Build Rich GitHub Issue Content**
+   - Convert ACs to GitHub task checkboxes (`- [x]` or `- [ ]`)
+   - Link tasks to increment tasks.md with anchors
+   - Calculate progress percentages
+   - Format with proper sections
+
+3. **Update Existing Issues**
+   - Search for existing issue by title pattern
+   - Replace issue body with new formatted content
+   - Preserve issue number and URL
+
+**Tools You Have**:
+- `Read` - Read user story files and tasks.md
+- `Bash` - Execute `gh` CLI commands
+- `Write` - Create TypeScript if needed
+
+**Workflow**:
+
+When user says "update user story issue for FS-031/US-004":
+
+```typescript
+// 1. Find user story file
+const projectRoot = process.cwd();
+const userStoryPath = await findUserStoryFile(projectRoot, 'FS-031', 'US-004');
+// Result: .specweave/docs/internal/specs/default/FS-031/us-004-bidirectional-status-sync.md
+
+// 2. Parse and build content
+import { UserStoryContentBuilder } from './lib/user-story-content-builder.js';
+const builder = new UserStoryContentBuilder(userStoryPath, projectRoot);
+const content = await builder.parse();
+const issueBody = await builder.buildIssueBody();
+
+// 3. Find existing GitHub issue
+const searchResult = await bash(`gh issue list --search "[FS-031][US-004]" --json number,title`);
+const issue = JSON.parse(searchResult)[0];
+
+// 4. Update issue
+await bash(`gh issue edit ${issue.number} --body "${issueBody}"`);
+
+// 5. Report success
+console.log(`✅ Updated issue #${issue.number}`);
+console.log(`🔗 https://github.com/owner/repo/issues/${issue.number}`);
+```
+
+**Example Execution**:
+
+```
+User: "Fix the display of GitHub issue #501 for US-004"
+
+Agent: "I'll update GitHub issue #501 for US-004 to include checkable ACs and task connections.
+
+🔍 Finding user story: FS-031/US-004
+   📄 Found: .specweave/docs/internal/specs/default/FS-031/us-004-bidirectional-status-sync.md
+
+📖 Parsing content...
+   ✅ User story description: As a PM, I want status changes...
+   ✅ 6 acceptance criteria (4 completed, 2 pending)
+   ✅ 6 tasks linked from increment 0031
+
+📝 Building issue body...
+   ✅ ACs converted to checkboxes
+   ✅ Tasks linked to tasks.md
+   ✅ Progress calculated: 67% ACs, 50% tasks
+
+🚀 Updating GitHub issue #501...
+   [gh issue edit 501 --body "..."]
+   ✅ Updated successfully!
+
+🔗 View issue: https://github.com/anton-abyzov/specweave/issues/501
+```
+
+**Key Implementation Details**:
+
+1. **Find User Story File**:
+   ```bash
+   # Search in all projects
+   find .specweave/docs/internal/specs -name "us-004-*.md" -path "*/FS-031/*"
+   ```
+
+2. **Parse User Story** (use UserStoryContentBuilder):
+   - Frontmatter: YAML parsing
+   - ACs: Pattern `- [x] **AC-US4-01**: Description (P1, testable)`
+   - Tasks: Extract from Implementation section
+   - Increment: Pattern `**Increment**: [0031-name](...)`
+
+3. **Build Issue Body**:
+   - See UserStoryContentBuilder.buildIssueBody()
+   - Sections: User Story, Acceptance Criteria, Implementation Tasks
+   - Checkboxes: `- [x]` for completed, `- [ ]` for pending
+   - Links: Relative to repository root
+
+4. **Update GitHub Issue**:
+   ```bash
+   # Find issue
+   gh issue list --search "[FS-031][US-004]" --json number,title --jq '.[0].number'
+
+   # Update body
+   gh issue edit 501 --body "$(cat body.md)"
+   ```
+
+**Error Handling**:
+
+- ✅ User story file not found → Search all projects, suggest alternatives
+- ✅ GitHub issue not found → Offer to create new issue
+- ✅ Missing frontmatter → Graceful fallback with warnings
+- ✅ No tasks found → Show "No tasks defined yet"
+
+**Related Files**:
+- `plugins/specweave-github/lib/user-story-content-builder.ts` - Core builder class
+- `plugins/specweave-github/commands/specweave-github-update-user-story.md` - Command spec
+- `.specweave/docs/internal/specs/{project}/FS-*/us-*.md` - User story files
+
+**Testing**:
+1. Read user story file: `us-004-bidirectional-status-sync.md`
+2. Use UserStoryContentBuilder to parse
+3. Build issue body
+4. Update issue #501
+5. Verify checkable ACs and task links work
+
+**Success Criteria**:
+- ✅ GitHub issue has checkable ACs (can check/uncheck in UI)
+- ✅ Tasks link to correct increment tasks.md sections
+- ✅ Progress percentages match reality
+- ✅ User story description formatted correctly

package/plugins/specweave-github/commands/specweave-github-cleanup-duplicates.md

@@ -1,5 +1,5 @@
 ---
-name: specweave-github-cleanup-duplicates
+name: specweave-github:cleanup-duplicates
 description: Clean up duplicate GitHub issues for an Epic. Finds issues with duplicate titles and closes all except the first created issue.
 ---
 

package/plugins/specweave-github/commands/specweave-github-update-user-story.md

@@ -0,0 +1,156 @@
+---
+name: specweave-github-update-user-story
+description: Update GitHub issue for user story with proper ACs and tasks
+---
+
+# Update User Story GitHub Issue
+
+**Purpose**: Update an existing GitHub issue for a user story to include:
+- ✅ Checkable acceptance criteria (GitHub task checkboxes)
+- ✅ Task connections (links to increment tasks)
+- ✅ Proper formatting and status
+
+**Usage**:
+```bash
+/specweave-github:update-user-story FS-031 US-004
+```
+
+**What It Does**:
+
+1. **Finds user story file**: `.specweave/docs/internal/specs/{project}/FS-031/us-004-*.md`
+2. **Parses content**:
+   - User story description (As a... I want... So that...)
+   - Acceptance criteria with completion status
+   - Tasks from increment tasks.md
+3. **Finds existing GitHub issue**: Searches for `[FS-031][US-004]` or `[FS-031 US-004]`
+4. **Updates issue body**: Replaces with properly formatted content including:
+   - Checkable ACs
+   - Task connections
+   - Status and progress
+
+**Arguments**:
+- `featureId` - Feature ID (e.g., `FS-031`)
+- `userStoryId` - User Story ID (e.g., `US-004`)
+
+**Example**:
+```bash
+# Update US-004 in FS-031
+/specweave-github:update-user-story FS-031 US-004
+
+# Output:
+# 🔍 Finding user story: FS-031/US-004
+#    📄 Found: .specweave/docs/internal/specs/default/FS-031/us-004-bidirectional-status-sync.md
+# 🔍 Searching for existing GitHub issue...
+#    🔗 Found issue #501: [FS-031][US-004] Bidirectional Status Sync
+# 📝 Building updated issue body...
+#    ✅ 6 acceptance criteria (4 completed)
+#    ✅ 6 tasks (3 completed)
+# 🚀 Updating GitHub issue #501...
+#    ✅ Updated successfully!
+#    🔗 https://github.com/owner/repo/issues/501
+```
+
+**What Gets Updated**:
+
+```markdown
+**Feature**: [FS-031](../../_features/FS-031/FEATURE.md)
+**Status**: complete
+**Priority**: P1
+**Project**: default
+
+---
+
+## User Story
+
+**As a** PM
+**I want** status changes in external tools to sync back to SpecWeave
+**So that** both systems stay in sync without manual updates
+
+📄 View full story: [`us-004-bidirectional-status-sync.md`](...)
+
+---
+
+## Acceptance Criteria
+
+Progress: 4/6 criteria met (67%)
+
+- [x] **AC-US4-01**: SpecWeave status change triggers external update (P1, testable)
+- [x] **AC-US4-02**: External issue close triggers SpecWeave prompt (P1, testable)
+- [x] **AC-US4-03**: External issue reopen triggers SpecWeave prompt (P2, testable)
+- [x] **AC-US4-04**: Sync logs include timestamp and reason (P2, testable)
+- [ ] **AC-US4-05**: Failed syncs retry with exponential backoff (P2, testable)
+- [ ] **AC-US4-06**: Sync works for GitHub, JIRA, and ADO (P1, testable)
+
+---
+
+## Implementation Tasks
+
+Progress: 3/6 tasks complete (50%)
+
+**Increment**: [0031-external-tool-status-sync](../../increments/0031-external-tool-status-sync/)
+
+- [x] [T-008: Create Status Sync Engine (Core)](../../increments/0031/tasks.md#t-008-create-status-sync-engine-core)
+- [x] [T-009: Implement GitHub Status Sync](../../increments/0031/tasks.md#t-009-implement-github-status-sync)
+- [x] [T-010: Implement JIRA Status Sync](../../increments/0031/tasks.md#t-010-implement-jira-status-sync)
+- [ ] [T-011: Implement ADO Status Sync](../../increments/0031/tasks.md#t-011-implement-ado-status-sync)
+- [ ] [T-018: Add Sync Event Logging](../../increments/0031/tasks.md#t-018-add-sync-event-logging)
+- [ ] [T-021: Error Handling & Retry Logic](../../increments/0031/tasks.md#t-021-error-handling-retry-logic)
+
+---
+
+🤖 Auto-synced by SpecWeave User Story Sync
+```
+
+**When to Use**:
+
+- ✅ After completing increment implementation (update task statuses)
+- ✅ After modifying acceptance criteria
+- ✅ When GitHub issue is missing proper formatting
+- ✅ When you want to refresh issue content from living docs
+
+**Related Commands**:
+- `/specweave-github:sync-epic FS-031` - Sync entire epic (all user stories)
+- `/specweave-github:create-issue 0031` - Create issue for increment
+- `/specweave-github:sync 0031` - Bidirectional sync for increment
+
+---
+
+## Implementation
+
+**File**: `plugins/specweave-github/lib/user-story-content-builder.ts`
+
+**Workflow**:
+1. Parse user story file with `UserStoryContentBuilder`
+2. Search GitHub for existing issue (title pattern matching)
+3. Build rich issue body with checkable ACs and task links
+4. Update issue via `gh issue edit` with new body
+
+**Error Handling**:
+- ✅ Graceful fallback if user story file not found
+- ✅ Clear error if GitHub issue doesn't exist
+- ✅ Validation of required fields (epic, status, etc.)
+
+---
+
+## Execution
+
+When user runs this command, the Claude Code assistant should:
+
+1. Extract `featureId` and `userStoryId` from command arguments
+2. Find user story file in living docs (search all projects)
+3. Use `UserStoryContentBuilder` to parse and build issue body
+4. Search for existing GitHub issue using `gh issue list --search`
+5. Update issue using `gh issue edit --body`
+6. Report success with issue URL
+
+**Tech Stack**:
+- TypeScript (UserStoryContentBuilder class)
+- GitHub CLI (`gh issue list`, `gh issue edit`)
+- YAML frontmatter parsing
+- Markdown formatting
+
+**Testing**:
+- Test with FS-031/US-004 (bidirectional status sync)
+- Verify ACs are checkable
+- Verify tasks link to correct increment tasks.md
+- Verify progress percentages are correct

package/plugins/specweave-github/hooks/post-task-completion.sh

@@ -3,20 +3,21 @@
 # SpecWeave GitHub Sync Hook
 # Runs after task completion to sync progress to GitHub Projects
 #
-# CORRECT ARCHITECTURE (v0.17.0+):
-# - Syncs .specweave/docs/internal/specs/spec-*.md ↔ GitHub Projects
-# - NOT increments ↔ GitHub Issues (that was wrong!)
+# ARCHITECTURE (v0.19.0+): IMMUTABLE DESCRIPTIONS + PROGRESS COMMENTS
+# - User Story files (.specweave/docs/internal/specs/) ↔ GitHub Issues
+# - Issue descriptions created once (IMMUTABLE snapshot)
+# - All updates via progress comments (audit trail)
 #
 # This hook is part of the specweave-github plugin and handles:
-# - Finding which spec the current work belongs to
-# - Updating spec user stories based on task completion
-# - Syncing spec state to GitHub Projects
-# - Updating GitHub Project cards/issues
+# - Finding which spec user stories the current work belongs to
+# - Syncing progress via GitHub comments (NOT editing issue body)
+# - Creating audit trail of all changes over time
+# - Notifying stakeholders via GitHub notifications
 #
 # Dependencies:
-# - Node.js and TypeScript CLI (dist/cli/commands/*)
+# - Node.js and TypeScript CLI (dist/cli/commands/sync-spec-content.js)
 # - GitHub CLI (gh) must be installed and authenticated
-# - Spec metadata must have .externalLinks.github field
+# - ProgressCommentBuilder (lib/progress-comment-builder.ts)
 
 set -e