coaiajs 0.3.0 → 0.3.2

package/articles/technical/commander-cli-framework.md

@@ -1,262 +0,0 @@
-# Commander.js CLI Framework: Technical Assessment for CoAiA.js
-
-> Package selection brief — CLI framework for coaiajs's multi-subcommand interface (tash, fetch, fuse, pipeline, env, gh, narrative, pde, planning)
-
-## Summary & Recommendation
-
-**Use `commander` v12.x** for the coaiajs CLI. Commander's Git-style subcommand model maps directly to our 9+ subcommand structure. It's the most widely adopted Node.js CLI framework (weekly downloads dwarf alternatives), has zero dependencies, generates help automatically, and every team member already knows it. Clipanion's class-based approach is elegant but niche; yargs is powerful but over-engineered for our tree.
-
-**Pin:** `"commander": "^12.1.0"`
-
-## What We're Replacing
-
-**Coaiapy** uses Python's `argparse` with subparsers:
-
-```python
-# coaiapy/coaiacli.py — argparse subcommand pattern
-import argparse
-parser = argparse.ArgumentParser(description='CoAiA CLI')
-subparsers = parser.add_subparsers(dest='command')
-parser_fuse = subparsers.add_parser('fuse', help='Manage Langfuse integrations.')
-parser_gh = subparsers.add_parser('gh', help='GitHub operations.')
-# ... 9+ subcommands
-```
-
-**coaia-narrative** uses `minimist` — bare-bones argv parsing with no help generation:
-
-```typescript
-// coaia-narrative/src/cli.ts — minimist pattern
-import minimist from 'minimist';
-const args = minimist(process.argv.slice(2));
-// Manual flag handling: --memory-path, -M, --json, --no-color, --interactive
-```
-
-**coaia-planning** also uses `minimist` for `--plans-dir` and `--output-dir`.
-
-## Options Compared
-
-| Feature | Commander v12 | Yargs v17 | Clipanion v4 | minimist v1.2 |
-|---------|--------------|-----------|-------------|--------------|
-| Weekly downloads | ~125M | ~80M | ~4M | ~50M |
-| Dependencies | 0 | 5+ | 0 | 0 |
-| Subcommand model | Git-style `.command()` | Builder callbacks | Class-based `paths` | None (manual) |
-| Auto-help | ✅ Excellent | ✅ Excellent | ✅ Good | ❌ None |
-| TypeScript | Good (DefinitelyTyped) | Good | Excellent (first-class) | Minimal |
-| Learning curve | Low | Medium | Medium-high | Very low |
-| Argument parsing | Options, args, variadic | All types + coercion | All types + validation | Raw key-value |
-| Completion (bash/zsh) | Plugin available | Built-in | Not built-in | ❌ |
-| Version handling | Built-in `.version()` | Built-in | Manual | ❌ |
-| Nested subcommands | `.command()` nesting | Recursive builders | Arbitrary `paths` depth | ❌ |
-| Exit override | `.exitOverride()` for testing | `fail()` handler | Exception-based | N/A |
-
-## API Overview
-
-### Main CLI Entry Point
-
-```typescript
-// src/cli.ts
-import { Command } from 'commander';
-import { version } from '../package.json';
-
-const program = new Command()
-  .name('coaia')
-  .version(version)
-  .description('CoAiA.js — Structural Tension Agent Framework');
-
-// Register subcommands
-program.addCommand(createTashCommand());
-program.addCommand(createFetchCommand());
-program.addCommand(createFuseCommand());
-program.addCommand(createPipelineCommand());
-program.addCommand(createEnvCommand());
-program.addCommand(createGhCommand());
-program.addCommand(createNarrativeCommand());
-program.addCommand(createPdeCommand());
-program.addCommand(createPlanningCommand());
-
-program.parse();
-```
-
-### Subcommand Definition Pattern
-
-```typescript
-// src/commands/fuse.ts
-import { Command } from 'commander';
-
-export function createFuseCommand(): Command {
-  const fuse = new Command('fuse')
-    .description('Manage Langfuse integrations');
-
-  fuse
-    .command('trace')
-    .description('Create or list traces')
-    .option('-s, --session <id>', 'Session ID')
-    .option('--list', 'List recent traces')
-    .option('--json', 'Output as JSON')
-    .action(async (opts) => {
-      if (opts.list) {
-        const traces = await listTraces(opts);
-        console.log(opts.json ? JSON.stringify(traces) : formatTable(traces));
-      } else {
-        await createTrace(opts);
-      }
-    });
-
-  fuse
-    .command('score')
-    .description('Score a trace')
-    .requiredOption('-t, --trace-id <id>', 'Trace ID to score')
-    .requiredOption('-n, --name <name>', 'Score name')
-    .requiredOption('-v, --value <number>', 'Score value', parseFloat)
-    .action(async (opts) => {
-      await scoreTrace(opts.traceId, opts.name, opts.value);
-    });
-
-  return fuse;
-}
-```
-
-### GitHub Subcommand (replacing cogh.py)
-
-```typescript
-// src/commands/gh.ts
-import { Command } from 'commander';
-
-export function createGhCommand(): Command {
-  const gh = new Command('gh')
-    .description('GitHub operations');
-
-  gh.command('issues')
-    .description('List repository issues')
-    .argument('<owner>', 'Repository owner')
-    .argument('<repo>', 'Repository name')
-    .option('--state <state>', 'Filter by state', 'open')
-    .option('--labels <labels...>', 'Filter by labels')
-    .option('--json', 'JSON output')
-    .action(async (owner, repo, opts) => {
-      const issues = await listIssues(owner, repo, opts);
-      console.log(opts.json ? JSON.stringify(issues) : formatIssuesTable(issues));
-    });
-
-  gh.command('issue <owner> <repo> <number>')
-    .description('Get a specific issue')
-    .action(async (owner, repo, number) => {
-      const issue = await getIssue(owner, repo, parseInt(number));
-      console.log(JSON.stringify(issue, null, 2));
-    });
-
-  return gh;
-}
-```
-
-### Pipeline Subcommand (with YAML template loading)
-
-```typescript
-// src/commands/pipeline.ts
-import { Command } from 'commander';
-
-export function createPipelineCommand(): Command {
-  const pipeline = new Command('pipeline')
-    .description('Execute pipeline templates');
-
-  pipeline
-    .command('run <template>')
-    .description('Run a pipeline template')
-    .option('-v, --var <key=value...>', 'Template variables')
-    .option('--dry-run', 'Show execution plan without running')
-    .option('--template-dir <dir>', 'Template search directory')
-    .action(async (template, opts) => {
-      const vars = parseVars(opts.var ?? []);
-      await runPipeline(template, vars, opts);
-    });
-
-  pipeline
-    .command('list')
-    .description('List available templates')
-    .action(async () => {
-      const templates = await listTemplates();
-      templates.forEach(t => console.log(`  ${t.name} (v${t.version})`));
-    });
-
-  return pipeline;
-}
-```
-
-### Global Options Pattern
-
-```typescript
-// Shared options across all subcommands
-program
-  .option('--json', 'Output as JSON')
-  .option('--no-color', 'Disable colored output')
-  .option('--verbose', 'Verbose logging')
-  .option('-M, --memory-path <path>', 'Memory file path')
-  .option('--env <file>', 'Custom env file');
-
-// Access in any subcommand action
-function getGlobalOpts(cmd: Command) {
-  const root = cmd.parent ?? cmd;
-  return root.opts() as { json?: boolean; color?: boolean; verbose?: boolean; memoryPath?: string };
-}
-```
-
-## Integration Plan
-
-1. **Entry point:** `src/cli.ts` — Commander program with `.parse()`
-2. **Command directory:** `src/commands/` — one file per subcommand
-   - `tash.ts` — Task management (structural tension)
-   - `fetch.ts` — Data fetching operations
-   - `fuse.ts` — Langfuse integration
-   - `pipeline.ts` — Pipeline template execution
-   - `env.ts` — Environment management
-   - `gh.ts` — GitHub operations
-   - `narrative.ts` — Narrative/chart operations
-   - `pde.ts` — Prompt decomposition
-   - `planning.ts` — Plan-to-STC operations
-3. **Bin entry:** `package.json` → `"bin": { "coaia": "./dist/src/cli.js" }`
-4. **Testing:** Commander's `.exitOverride()` + `.configureOutput()` for unit tests
-
-### Migration Path from minimist
-
-```typescript
-// BEFORE (coaia-narrative minimist pattern)
-const args = minimist(process.argv.slice(2));
-const memoryPath = args['memory-path'] || args.M || process.env.COAIA_MEMORY_PATH;
-
-// AFTER (Commander)
-program.option('-M, --memory-path <path>', 'Memory file path', process.env.COAIA_MEMORY_PATH);
-// memoryPath is now typed and auto-documented in --help
-```
-
-## Version & Ecosystem
-
-| Metric | Value |
-|--------|-------|
-| Current version | 12.1.0 |
-| Weekly downloads | ~125M |
-| Dependencies | 0 |
-| TypeScript | `@types/commander` (DefinitelyTyped) |
-| GitHub stars | ~27,000 |
-| Used by | npm CLI, Vue CLI, create-react-app |
-| Node.js compat | ≥16 (we target ≥20) |
-| License | MIT |
-
-## Why Not Yargs?
-
-Yargs is powerful but overkill. Our subcommand tree is wide (9 commands) but not deeply nested. Commander's chainable API makes each command file self-contained and readable. Yargs' middleware system and callback-based builders add complexity we don't need.
-
-## Why Not Clipanion?
-
-Clipanion's TypeScript-first class-based approach is architecturally beautiful, but:
-1. ~4M weekly downloads vs Commander's ~125M = smaller help ecosystem
-2. Class-based commands are less familiar to contributors
-3. Yarn is its primary consumer; documentation coverage is thin for edge cases
-4. Commander is already in our `package.json`
-
-## References
-
-- npm: https://www.npmjs.com/package/commander
-- GitHub: https://github.com/tj/commander.js
-- Guide: https://betterstack.com/community/guides/scaling-nodejs/commander-explained/
-- Comparison: https://npm-compare.com/commander,yargs
-- Clipanion: https://github.com/arcanis/clipanion

package/articles/technical/dotenv-config-patterns.md

@@ -1,360 +0,0 @@
-# Config Management in Node.js: Technical Assessment for CoAiA.js
-
-> Package selection brief — Configuration cascade replacing coaiapy's multi-file config (env vars > .env > coaia.json > defaults)
-
-## Summary & Recommendation
-
-**Use `dotenv` v16.x + `cosmiconfig` v9.x + `zod` for validation.** This triple replaces coaiapy's hand-rolled config cascade with a standard, validated pattern:
-- `dotenv` loads `.env` files (matching coaiapy's `.coaia-env`)
-- `cosmiconfig` discovers `coaia.json`, `coaia.yaml`, `.coaiarc`, or `coaia.config.js` (matching coaiapy's multi-format loading)
-- `zod` validates the merged config at startup (what coaiapy lacks entirely)
-
-**Pin:**
-```json
-"dotenv": "^16.4.0",
-"cosmiconfig": "^9.0.0",
-"zod": "^4.0.0"
-```
-
-## What We're Replacing
-
-Coaiapy has a complex config cascade across multiple modules:
-
-### environment.py (EnvironmentManager)
-```python
-@dataclass
-class EnvironmentConfig:
-    project_env_file: str = ".coaia-env"
-    global_env_file: str = "~/.coaia/global.env"
-    supported_formats: List[str] = ["json", "env"]
-
-class EnvironmentManager:
-    def _read_env_file(self, file_path: Path) -> Dict[str, Any]:
-        if content.startswith('{'):
-            return json.loads(content)
-        # Parse KEY=value format
-```
-
-### coaiamodule.py (read_config)
-```python
-def read_config():
-    # Priority: env vars > .env > coaia.json > defaults
-    # Searches: cwd, parent dirs, ~/.coaia/
-    # Supports: JSON, .env, shell env vars
-```
-
-### coaia-narrative CLI (minimist + dotenv)
-```typescript
-// Priority order:
-// 1. Command-line flags (--memory-path, -M)
-// 2. Custom env file (--env)
-// 3. .env in cwd
-// 4. _env.sh (fallback)
-// 5. Environment variables
-// 6. Defaults
-```
-
-The Python approach has no validation — missing keys fail at runtime deep in execution. The TypeScript approach in coaia-narrative works but is ad-hoc per module.
-
-## Options Compared
-
-| Feature | dotenv + cosmiconfig + zod | dotenv only | convict | nconf | node-config |
-|---------|--------------------------|-------------|---------|-------|-------------|
-| .env loading | ✅ dotenv | ✅ | ⚠️ Manual | ✅ | ❌ |
-| Multi-format (JSON/YAML/JS) | ✅ cosmiconfig | ❌ | ✅ JSON | ✅ | ✅ |
-| Parent dir search | ✅ cosmiconfig | ❌ | ❌ | ❌ | ❌ |
-| Schema validation | ✅ zod | ❌ | ✅ convict | ❌ | ❌ |
-| Type inference | ✅ zod infer | ❌ | ⚠️ Manual | ❌ | ❌ |
-| Variable expansion | ✅ dotenv-expand | ✅ dotenv-expand | ❌ | ❌ | ❌ |
-| Env override | ✅ process.env priority | ✅ | ✅ | ✅ | ✅ |
-| Weekly downloads | ~93M + ~30M + ~100M | ~93M | ~900K | ~900K | ~1.5M |
-| Bundle overhead | Minimal | Tiny | Medium | Medium | Medium |
-| TypeScript | All have types | ✅ | ⚠️ | ⚠️ | ⚠️ |
-
-## API Overview
-
-### Config Schema (Zod)
-
-```typescript
-// src/config/schema.ts
-import { z } from 'zod';
-
-export const CoaiaConfigSchema = z.object({
-  // Redis
-  redisUrl: z.string().url().default('redis://localhost:6379'),
-  
-  // Langfuse
-  langfusePublicKey: z.string().optional(),
-  langfuseSecretKey: z.string().optional(),
-  langfuseBaseUrl: z.string().url().default('https://cloud.langfuse.com'),
-  
-  // OpenAI
-  openaiApiKey: z.string().optional(),
-  openaiBaseUrl: z.string().url().optional(),
-  openaiModel: z.string().default('gpt-5.4'),
-  
-  // AWS Polly
-  awsRegion: z.string().default('us-east-1'),
-  pollyKey: z.string().optional(),
-  pollySecret: z.string().optional(),
-  
-  // GitHub
-  githubToken: z.string().optional(),
-  
-  // Paths
-  memoryPath: z.string().default('.coaia/memory.jsonl'),
-  plansDir: z.string().default('.coaia/plans'),
-  outputDir: z.string().default('.coaia/output'),
-  templateDir: z.string().default('.coaia/templates'),
-  
-  // Behavior
-  jsonOutput: z.boolean().default(false),
-  noColor: z.boolean().default(false),
-  verbose: z.boolean().default(false),
-});
-
-export type CoaiaConfig = z.infer<typeof CoaiaConfigSchema>;
-```
-
-### Config Loader (cosmiconfig + dotenv)
-
-```typescript
-// src/config/loader.ts
-import { cosmiconfig } from 'cosmiconfig';
-import dotenv from 'dotenv';
-import { CoaiaConfigSchema, type CoaiaConfig } from './schema.js';
-import { existsSync } from 'fs';
-import { resolve } from 'path';
-
-// cosmiconfig explorer — searches for config files
-const explorer = cosmiconfig('coaia', {
-  searchPlaces: [
-    'coaia.config.js',
-    'coaia.config.ts',
-    'coaia.json',
-    'coaia.yaml',
-    'coaia.yml',
-    '.coaiarc',
-    '.coaiarc.json',
-    '.coaiarc.yaml',
-    'package.json',  // "coaia" key in package.json
-  ],
-});
-
-let _config: CoaiaConfig | null = null;
-
-export async function loadConfig(overrides?: Partial<CoaiaConfig>): Promise<CoaiaConfig> {
-  if (_config && !overrides) return _config;
-
-  // Step 1: Load .env files (matches coaiapy's .coaia-env pattern)
-  const envFiles = [
-    '.env',
-    '.coaia-env',
-    resolve(process.env.HOME ?? '', '.coaia', 'global.env'),
-  ];
-  
-  for (const envFile of envFiles) {
-    if (existsSync(envFile)) {
-      dotenv.config({ path: envFile });
-    }
-  }
-
-  // Step 2: Load config file via cosmiconfig (searches parent dirs)
-  const result = await explorer.search();
-  const fileConfig = result?.config ?? {};
-
-  // Step 3: Merge sources (priority: CLI overrides > env vars > config file > defaults)
-  const merged = {
-    ...fileConfig,
-    // Env var overrides (matching coaiapy's env var names)
-    ...(process.env.REDIS_URL && { redisUrl: process.env.REDIS_URL }),
-    ...(process.env.UPSTASH_REDIS_URL && { redisUrl: process.env.UPSTASH_REDIS_URL }),
-    ...(process.env.LANGFUSE_PUBLIC_KEY && { langfusePublicKey: process.env.LANGFUSE_PUBLIC_KEY }),
-    ...(process.env.LANGFUSE_SECRET_KEY && { langfuseSecretKey: process.env.LANGFUSE_SECRET_KEY }),
-    ...(process.env.LANGFUSE_BASE_URL && { langfuseBaseUrl: process.env.LANGFUSE_BASE_URL }),
-    ...(process.env.OPENAI_API_KEY && { openaiApiKey: process.env.OPENAI_API_KEY }),
-    ...(process.env.OPENAI_BASE_URL && { openaiBaseUrl: process.env.OPENAI_BASE_URL }),
-    ...(process.env.AWS_REGION && { awsRegion: process.env.AWS_REGION }),
-    ...(process.env.GH_TOKEN && { githubToken: process.env.GH_TOKEN }),
-    ...(process.env.GITHUB_TOKEN && { githubToken: process.env.GITHUB_TOKEN }),
-    ...(process.env.COAIA_MEMORY_PATH && { memoryPath: process.env.COAIA_MEMORY_PATH }),
-    ...(process.env.COAIA_PLANS_DIR && { plansDir: process.env.COAIA_PLANS_DIR }),
-    ...(process.env.COAIA_OUTPUT_DIR && { outputDir: process.env.COAIA_OUTPUT_DIR }),
-    // CLI overrides (highest priority)
-    ...overrides,
-  };
-
-  // Step 4: Validate with Zod
-  _config = CoaiaConfigSchema.parse(merged);
-  return _config;
-}
-
-// Synchronous getter after initial load
-export function getConfig(): CoaiaConfig {
-  if (!_config) throw new Error('Config not loaded. Call loadConfig() first.');
-  return _config;
-}
-```
-
-### CLI Integration with Commander
-
-```typescript
-// src/cli.ts — CLI flags feed into config overrides
-import { Command } from 'commander';
-import { loadConfig } from './config/loader.js';
-
-const program = new Command()
-  .option('-M, --memory-path <path>', 'Memory file path')
-  .option('--env <file>', 'Custom env file path')
-  .option('--json', 'Output as JSON')
-  .option('--no-color', 'Disable colored output')
-  .option('--verbose', 'Verbose logging');
-
-program.hook('preAction', async (thisCommand) => {
-  const opts = thisCommand.opts();
-  
-  // Load custom env file if specified
-  if (opts.env) {
-    dotenv.config({ path: opts.env });
-  }
-  
-  // Initialize config with CLI overrides
-  await loadConfig({
-    ...(opts.memoryPath && { memoryPath: opts.memoryPath }),
-    ...(opts.json && { jsonOutput: true }),
-    ...(opts.color === false && { noColor: true }),
-    ...(opts.verbose && { verbose: true }),
-  });
-});
-```
-
-### Variable Expansion (dotenv-expand)
-
-```typescript
-// For .env files that reference other variables
-// .env:
-// BASE_DIR=/opt/coaia
-// MEMORY_PATH=${BASE_DIR}/memory.jsonl
-// PLANS_DIR=${BASE_DIR}/plans
-
-import dotenv from 'dotenv';
-import { expand } from 'dotenv-expand';
-
-const env = dotenv.config({ path: '.env' });
-expand(env);  // Resolves ${BASE_DIR} references
-```
-
-### Config File Examples
-
-```yaml
-# coaia.yaml — project-level config
-redis:
-  url: redis://localhost:6379
-
-langfuse:
-  publicKey: pk-lf-xxx
-  secretKey: sk-lf-xxx
-  baseUrl: https://cloud.langfuse.com
-
-openai:
-  apiKey: sk-xxx
-  model: gpt-5.4
-
-paths:
-  memory: .coaia/memory.jsonl
-  plans: .coaia/plans
-  templates: .coaia/templates
-```
-
-```json
-// coaia.json — alternative JSON format
-{
-  "redisUrl": "redis://localhost:6379",
-  "langfusePublicKey": "pk-lf-xxx",
-  "openaiModel": "gpt-5.4",
-  "memoryPath": ".coaia/memory.jsonl"
-}
-```
-
-```ini
-# .coaia-env — environment variable format (coaiapy compatible)
-REDIS_URL=redis://localhost:6379
-LANGFUSE_PUBLIC_KEY=pk-lf-xxx
-LANGFUSE_SECRET_KEY=sk-lf-xxx
-OPENAI_API_KEY=sk-xxx
-GH_TOKEN=ghp_xxx
-AWS_REGION=us-east-1
-COAIA_MEMORY_PATH=.coaia/memory.jsonl
-```
-
-### Validation Error Messages
-
-```typescript
-// On startup with invalid config:
-try {
-  await loadConfig();
-} catch (error) {
-  if (error instanceof z.ZodError) {
-    console.error('Configuration errors:');
-    error.issues.forEach(issue => {
-      console.error(`  ${issue.path.join('.')}: ${issue.message}`);
-    });
-    process.exit(1);
-  }
-  throw error;
-}
-
-// Output:
-// Configuration errors:
-//   redisUrl: Invalid url
-//   langfuseBaseUrl: Invalid url
-```
-
-## Integration Plan
-
-1. **Schema:** `src/config/schema.ts` — Zod schema for all config values
-2. **Loader:** `src/config/loader.ts` — dotenv + cosmiconfig + Zod validation
-3. **Export:** `src/config/index.ts` — `loadConfig()` and `getConfig()`
-4. **CLI hook:** Commander `preAction` hook loads config with CLI overrides
-5. **Testing:** Config loader is mockable via overrides parameter
-6. **Migration:** Support both `coaia.json` and `.coaia-env` formats (backward compat)
-
-### Config Priority (matches coaiapy's cascade)
-
-```
-1. CLI flags              (--memory-path, --json)     ← Highest
-2. Environment variables  (REDIS_URL, GH_TOKEN)
-3. .env / .coaia-env      (project root)
-4. ~/.coaia/global.env    (global user config)
-5. coaia.json / coaia.yaml (cosmiconfig search)
-6. Zod defaults           (schema defaults)            ← Lowest
-```
-
-## Version & Ecosystem
-
-| Package | Version | Weekly Downloads | Purpose |
-|---------|---------|-----------------|---------|
-| dotenv | 16.4.x | ~93M | .env file loading |
-| dotenv-expand | 12.0.x | ~30M | Variable expansion in .env |
-| cosmiconfig | 9.0.x | ~30M | Multi-format config discovery |
-| zod | 4.3.x | ~100M+ | Schema validation + type inference |
-
-All packages: MIT licensed, zero or minimal dependencies, TypeScript supported.
-
-## Why Not Convict?
-
-Convict (from Mozilla) offers built-in schema validation, but:
-1. ~900K downloads vs dotenv's ~93M — smaller ecosystem
-2. Doesn't support YAML config files natively
-3. Doesn't search parent directories (cosmiconfig does)
-4. We already have Zod for validation (no need for convict's schema DSL)
-5. Convict's API is less TypeScript-friendly than Zod's type inference
-
-## References
-
-- dotenv npm: https://www.npmjs.com/package/dotenv
-- cosmiconfig npm: https://www.npmjs.com/package/cosmiconfig
-- cosmiconfig GitHub: https://github.com/cosmiconfig/cosmiconfig
-- dotenv security: https://www.johal.in/secrets-dotenv-secure-config-management-in-production-2026/
-- Config comparison: https://npm-compare.com/config,convict,dotenv,nconf
-- dotenv usage: https://oneuptime.com/blog/post/2026-01-25-dotenv-configuration-nodejs/view