@memnexus-ai/cli 0.1.0

package/docs/code-generation-strategy.md

@@ -0,0 +1,560 @@
+# CLI Code Generation Strategy
+
+## The Core Problem
+
+**Question**: When we add/remove/change APIs, what updates the actual CLI code?
+
+**Current Gap**: While `@memnexus-ai/contracts` auto-generates TypeScript types from OpenAPI, **the CLI command implementations must be written manually**.
+
+### What's Automated Today ✅
+
+```typescript
+// When you add: POST /api/artifacts/export to OpenAPI spec
+// This happens automatically:
+// 1. Types are generated in @memnexus-ai/contracts
+export interface ExportArtifactRequest {
+  format: 'pdf' | 'json' | 'csv';
+  artifactIds: string[];
+}
+
+// 2. Client method is created
+class CoreApiClient {
+  artifacts = {
+    export: (data: ExportArtifactRequest) => Promise<Blob>
+  }
+}
+```
+
+### What's Manual Today ❌
+
+```typescript
+// Someone must manually write:
+// 1. CLI command in src/commands/artifacts.ts
+program
+  .command('artifacts export')
+  .option('--format <format>', 'Export format')
+  .option('--output <file>', 'Output file path')
+  .action(async (options) => {
+    const client = getApiClient();
+    const result = await client.artifacts.export({...});
+    // Format output, handle errors, etc.
+  });
+
+// 2. Help text
+// 3. Input validation
+// 4. Output formatting
+// 5. Error handling
+```
+
+## Evaluated Solutions
+
+### Option 1: Full Auto-Generation (Not Recommended)
+
+**Tools Investigated**:
+- **openapi-cli-generator** - Generates Go CLIs only
+- **Restish** - Runtime API client (Go), not code generator
+- **OpenAPI Generator** - No TypeScript/Commander.js templates
+
+**Why Not Recommended**:
+- ❌ No mature tools for TypeScript/Commander.js generation
+- ❌ Generated code is often rigid and hard to customize
+- ❌ Loses the "developer experience" polish (help text, examples, validation)
+- ❌ Difficult to add custom logic (formatters, interactive prompts)
+- ❌ Breaking changes harder to manage with generated code
+
+**Verdict**: Full auto-generation trades flexibility for convenience, which isn't worth it for a developer-facing CLI.
+
+### Option 2: Hybrid Approach (Recommended)
+
+**Philosophy**: Auto-generate scaffolding, manually implement business logic.
+
+**What to Auto-Generate**:
+1. Command structure skeleton
+2. Type definitions (already done via contracts)
+3. Basic parameter parsing
+4. API method invocations
+
+**What to Keep Manual**:
+1. Help text and examples
+2. Output formatting logic
+3. Error messages and handling
+4. Interactive prompts
+5. Complex validation
+6. Edge case handling
+
+### Option 3: Convention-Based Generation (Best for MemNexus)
+
+**Philosophy**: Use conventions + metadata to minimize manual work while keeping flexibility.
+
+## Recommended Approach: Smart Scaffolding
+
+### 1. Command Scaffold Generator
+
+Create a custom generator that creates **starter templates** from OpenAPI spec:
+
+```bash
+# Developer runs this when new API endpoint added:
+npm run generate:command -- --operation exportArtifact
+
+# Generates: src/commands/artifacts-export.ts with skeleton
+```
+
+**Generated Template**:
+```typescript
+// AUTO-GENERATED SKELETON - DO NOT EDIT HEADER
+// Generated from: POST /api/artifacts/export
+// Last updated: 2025-11-13T10:00:00Z
+
+import { Command } from 'commander';
+import { getApiClient } from '../lib/api-client';
+import type { ExportArtifactRequest } from '@memnexus-ai/contracts';
+
+export const exportArtifactCommand = new Command('export')
+  .description('Export artifacts to various formats')
+  // TODO: Add detailed help text and examples
+  .option('--format <format>', 'Export format (pdf, json, csv)')
+  .option('--artifact-ids <ids...>', 'Artifact IDs to export')
+  .option('--output <file>', 'Output file path')
+  .action(async (options) => {
+    try {
+      const client = getApiClient();
+
+      // TODO: Add input validation
+      const request: ExportArtifactRequest = {
+        format: options.format,
+        artifactIds: options.artifactIds,
+      };
+
+      // TODO: Add progress indicator
+      const result = await client.artifacts.export(request);
+
+      // TODO: Implement output formatting
+      console.log('Export completed:', result);
+    } catch (error) {
+      // TODO: Improve error handling
+      console.error('Export failed:', error);
+      process.exit(1);
+    }
+  });
+```
+
+### 2. Implementation Workflow
+
+**Step 1: API Change**
+```bash
+cd mx-core-api
+# Add new endpoint
+# Update OpenAPI spec
+```
+
+**Step 2: Contracts Update**
+```bash
+cd mx-api-gateway/contracts
+npm run generate:sdk  # Auto-generates types
+npm run build
+npm run test
+# Publish via GitHub Actions
+```
+
+**Step 3: CLI Scaffold Generation**
+```bash
+cd mx-cli
+npm install @memnexus-ai/contracts@latest
+
+# Run custom scaffold generator
+npm run generate:command -- --spec ../mx-api-gateway/contracts/openapi.yaml
+
+# Or interactive mode:
+npm run generate:command
+# ? Select operation to implement: POST /api/artifacts/export
+# ? Command name: artifacts export
+# ✓ Generated src/commands/artifacts-export.ts
+# ✓ Updated src/commands/index.ts
+# TODO: Implement business logic in src/commands/artifacts-export.ts
+```
+
+**Step 4: Developer Implements**
+```typescript
+// Developer fills in TODOs:
+.description('Export artifacts to PDF, JSON, or CSV formats')
+.addHelpText('after', `
+Examples:
+  $ mx artifacts export --format pdf --artifact-ids art_123 art_456 --output export.pdf
+  $ mx artifacts export --format json --artifact-ids art_123 --output export.json
+`)
+.option('--format <format>', 'Export format', (val) => {
+  if (!['pdf', 'json', 'csv'].includes(val)) {
+    throw new Error('Format must be pdf, json, or csv');
+  }
+  return val;
+})
+```
+
+**Step 5: Test & Publish**
+```bash
+npm run build
+npm test
+# Push to main → GitHub Actions publishes
+```
+
+### 3. Scaffold Generator Implementation
+
+**Tool**: Custom Node.js script using `openapi-typescript` AST
+
+**Location**: `mx-cli/scripts/generate-command.ts`
+
+**Key Features**:
+1. **Parse OpenAPI Spec** - Read operation definitions
+2. **Extract Metadata**:
+   - HTTP method & path
+   - Parameters (path, query, body)
+   - Request/response schemas
+   - Description & tags
+3. **Generate Command File** - Template with TODOs
+4. **Update Index** - Auto-register command
+5. **Generate Test Stub** - Basic test template
+
+**Usage**:
+```bash
+# Interactive mode
+npm run generate:command
+
+# CLI mode
+npm run generate:command -- \
+  --spec ../mx-api-gateway/contracts/openapi.yaml \
+  --operation exportArtifact \
+  --command-path artifacts.export
+```
+
+**Example Generator Script**:
+```typescript
+// scripts/generate-command.ts
+import fs from 'fs';
+import path from 'path';
+import { parse } from 'yaml';
+
+interface GenerateOptions {
+  spec: string;
+  operation: string;
+  commandPath: string;
+}
+
+function generateCommand(options: GenerateOptions): void {
+  // 1. Parse OpenAPI spec
+  const spec = parse(fs.readFileSync(options.spec, 'utf8'));
+
+  // 2. Find operation
+  const operation = findOperation(spec, options.operation);
+  if (!operation) {
+    throw new Error(`Operation ${options.operation} not found`);
+  }
+
+  // 3. Extract metadata
+  const metadata = extractMetadata(operation);
+
+  // 4. Generate command file
+  const template = generateTemplate(metadata);
+  const outputPath = path.join(
+    'src/commands',
+    `${options.commandPath.replace('.', '-')}.ts`
+  );
+  fs.writeFileSync(outputPath, template);
+
+  // 5. Update index
+  updateCommandIndex(options.commandPath, outputPath);
+
+  console.log(`✓ Generated ${outputPath}`);
+  console.log('TODO: Implement business logic and polish the command');
+}
+```
+
+### 4. Detection of Missing Commands
+
+**Problem**: How to know when API changes require CLI updates?
+
+**Solution**: Automated comparison tool
+
+**Tool**: `mx-cli/scripts/check-coverage.ts`
+
+```typescript
+// Checks which API operations don't have CLI commands
+npm run check:coverage
+
+// Output:
+// ✓ GET /api/memories - Implemented (src/commands/memories.ts)
+// ✓ POST /api/memories - Implemented (src/commands/memories.ts)
+// ✗ POST /api/artifacts/export - NOT IMPLEMENTED
+// ✗ GET /api/patterns/recommendations - NOT IMPLEMENTED
+//
+// Coverage: 45/47 operations (95.7%)
+// Run 'npm run generate:command' to scaffold missing commands
+```
+
+**Implementation**:
+```typescript
+// scripts/check-coverage.ts
+import { parse } from 'yaml';
+import { getAllCommands } from '../src/commands';
+
+function checkCoverage(): void {
+  const spec = loadOpenAPISpec();
+  const operations = extractAllOperations(spec);
+  const commands = getAllCommands();
+
+  const missing: string[] = [];
+
+  for (const op of operations) {
+    const commandExists = commands.some(cmd =>
+      cmd.operationId === op.operationId
+    );
+
+    if (commandExists) {
+      console.log(`✓ ${op.method} ${op.path} - Implemented`);
+    } else {
+      console.log(`✗ ${op.method} ${op.path} - NOT IMPLEMENTED`);
+      missing.push(op.operationId);
+    }
+  }
+
+  if (missing.length > 0) {
+    console.log(`\n⚠️  ${missing.length} operations not implemented`);
+    console.log(`Run: npm run generate:command --operation ${missing[0]}`);
+    process.exit(1); // Fail CI/CD
+  }
+}
+```
+
+**Integration with CI/CD**:
+```yaml
+# .github/workflows/publish.yml
+jobs:
+  validate:
+    steps:
+      - name: Check API coverage
+        run: |
+          npm run check:coverage
+          # Fails if operations are missing
+```
+
+### 5. Metadata-Driven Commands
+
+**Philosophy**: Store command metadata alongside OpenAPI extensions
+
+**OpenAPI Extensions**:
+```yaml
+# openapi.yaml
+paths:
+  /api/artifacts/export:
+    post:
+      operationId: exportArtifact
+      x-cli-command: artifacts export
+      x-cli-examples:
+        - description: Export single artifact as PDF
+          command: mx artifacts export --format pdf --artifact-ids art_123 --output export.pdf
+        - description: Export multiple artifacts as JSON
+          command: mx artifacts export --format json --artifact-ids art_123 art_456 --output export.json
+      x-cli-options:
+        format:
+          help: Output format (pdf, json, csv)
+          required: true
+        output:
+          help: Output file path
+          required: true
+      summary: Export artifacts to various formats
+      description: |
+        Export one or more artifacts to PDF, JSON, or CSV format.
+        The export is generated asynchronously and streamed to the output file.
+```
+
+**Generated Command Uses Metadata**:
+```typescript
+// Auto-generated from x-cli-* extensions
+export const exportArtifactCommand = new Command('export')
+  .description('Export artifacts to various formats')
+  .addHelpText('after', `
+Examples:
+  $ mx artifacts export --format pdf --artifact-ids art_123 --output export.pdf
+  $ mx artifacts export --format json --artifact-ids art_123 art_456 --output export.json
+`)
+  .option('--format <format>', 'Output format (pdf, json, csv)', { required: true })
+  .option('--output <file>', 'Output file path', { required: true })
+  .action(async (options) => {
+    // Implementation...
+  });
+```
+
+### 6. Update Notification System
+
+**Problem**: Developers need to know when CLI is out of sync
+
+**Solution**: Automated PR creation
+
+```yaml
+# .github/workflows/check-cli-sync.yml
+name: Check CLI Sync
+
+on:
+  workflow_run:
+    workflows: ["Publish Contracts Package"]
+    types: [completed]
+
+jobs:
+  check-sync:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install contracts package
+        run: |
+          cd mx-cli
+          npm install @memnexus-ai/contracts@latest
+
+      - name: Check coverage
+        id: coverage
+        run: |
+          cd mx-cli
+          npm run check:coverage || echo "missing=true" >> $GITHUB_OUTPUT
+
+      - name: Create PR if out of sync
+        if: steps.coverage.outputs.missing == 'true'
+        uses: peter-evans/create-pull-request@v5
+        with:
+          title: "🔄 Update CLI for new API endpoints"
+          body: |
+            ## New API Endpoints Detected
+
+            The `@memnexus-ai/contracts` package has been updated with new API operations.
+            The following CLI commands need to be implemented:
+
+            ${{ steps.coverage.outputs.missing_operations }}
+
+            ### Next Steps
+            1. Run `npm run generate:command` to scaffold new commands
+            2. Implement business logic for each command
+            3. Add tests
+            4. Update documentation
+
+            ### Related
+            - Contracts version: `${{ steps.coverage.outputs.contracts_version }}`
+            - PR will auto-close when all operations are implemented
+          labels: enhancement, auto-generated
+          branch: cli-sync-update
+```
+
+## Comparison: Manual vs. Hybrid
+
+### Fully Manual (Current State)
+
+**Time per Endpoint**: ~2-4 hours
+- Write command boilerplate (30 min)
+- Implement business logic (1-2 hours)
+- Write tests (30-60 min)
+- Write documentation (30 min)
+
+**Pros**:
+- ✅ Maximum flexibility
+- ✅ Consistent code style
+
+**Cons**:
+- ❌ Repetitive boilerplate
+- ❌ Easy to miss new endpoints
+- ❌ Slow to keep pace with API changes
+
+### Hybrid with Scaffolding (Recommended)
+
+**Time per Endpoint**: ~1-2 hours (50% faster)
+- Run generator (1 min)
+- Implement business logic (1-1.5 hours)
+- Tests auto-generated, just fill in (15-30 min)
+- Documentation from OpenAPI extensions (5 min)
+
+**Pros**:
+- ✅ Faster development
+- ✅ Consistent structure
+- ✅ Never miss endpoints (automated checks)
+- ✅ Keeps flexibility where needed
+
+**Cons**:
+- ❌ Initial generator development (~1 week)
+- ❌ Maintain generator as patterns evolve
+
+## Implementation Roadmap
+
+### Phase 1: Manual with Conventions (Week 1-4)
+**Status**: Current approach for v1.0
+
+- [x] Use `@memnexus-ai/contracts` for types
+- [ ] Establish command structure conventions
+- [ ] Document command implementation guide
+- [ ] Build first set of commands manually
+
+**Goal**: Ship v1.0, learn patterns
+
+### Phase 2: Coverage Detection (Week 5-6)
+**Status**: Can be done in parallel with v1.0 development
+
+- [ ] Build `check-coverage` script
+- [ ] Add to CI/CD
+- [ ] Create GitHub Action for notifications
+
+**Goal**: Never miss API changes
+
+### Phase 3: Command Scaffolding (Month 2)
+**Status**: After v1.0 patterns are established
+
+- [ ] Build `generate-command` script
+- [ ] Support OpenAPI `x-cli-*` extensions
+- [ ] Generate tests alongside commands
+- [ ] Update generator as patterns evolve
+
+**Goal**: 50% faster development for new endpoints
+
+### Phase 4: Metadata-Driven (Month 3+)
+**Status**: Future enhancement
+
+- [ ] Add `x-cli-*` extensions to OpenAPI spec
+- [ ] Generate help text from extensions
+- [ ] Auto-update CLI when contracts change
+- [ ] Consider runtime command loading
+
+**Goal**: Near-automatic CLI updates
+
+## Recommendation
+
+**For MemNexus v1.0**:
+1. ✅ Start manual (Phases 1) - Fastest path to v1.0
+2. ✅ Add coverage detection (Phase 2) - Prevents missing APIs
+3. ⏸️ Hold on scaffolding (Phase 3) - Wait until patterns stabilize
+4. ⏸️ Defer metadata-driven (Phase 4) - v2.0 feature
+
+**Rationale**:
+- Manual is fine for ~50 endpoints in v1.0
+- Learn patterns before automating
+- Coverage detection gives 80% of benefit with 20% of effort
+- Generator can be built once patterns are proven
+
+## Decision Matrix
+
+| Approach | Dev Time | Flexibility | Consistency | Maintenance | Recommendation |
+|----------|----------|-------------|-------------|-------------|----------------|
+| **Fully Manual** | 🔴 Slow | 🟢 High | 🟡 Medium | 🟢 Low | ⏸️ V1.0 only |
+| **Hybrid (Coverage + Manual)** | 🟡 Medium | 🟢 High | 🟢 High | 🟡 Medium | ✅ **Recommended** |
+| **Hybrid (Scaffold + Manual)** | 🟢 Fast | 🟢 High | 🟢 High | 🟡 Medium | ⏸️ Post-v1.0 |
+| **Fully Generated** | 🟢 Fastest | 🔴 Low | 🟢 High | 🔴 High | ❌ Not viable |
+
+## Key Takeaways
+
+1. **No magic bullet** - Full automation loses flexibility for developer CLIs
+2. **Hybrid is best** - Generate structure, implement logic manually
+3. **Coverage detection first** - Know when you're out of sync (easy win)
+4. **Scaffolding later** - Build generator after patterns stabilize
+5. **Manual is okay** - For v1.0 with ~50 endpoints, manual is acceptable
+6. **Metadata helps** - OpenAPI extensions reduce duplication
+
+---
+
+**Document Version**: 1.0
+**Last Updated**: 2025-11-13
+**Author**: Claude Code
+**Status**: Proposed Strategy