rapidkit 0.37.0 → 0.38.0

package/docs/OPTIMIZATION_GUIDE.md

@@ -0,0 +1,504 @@
+# Optimization Guide
+
+Optimization ideas for the RapidKit npm CLI codebase.
+
+**Users:** [../README.md](../README.md) · [OPEN_SOURCE_USER_SCENARIOS.md](./OPEN_SOURCE_USER_SCENARIOS.md) · [Documentation index](./README.md)
+
+## 1. Performance Optimizations
+
+### 1.1 Reduce Bundle Size
+```json
+// package.json - Add these
+{
+  "sideEffects": false,
+  "exports": {
+    ".": {
+      "import": "./dist/index.js",
+      "types": "./dist/index.d.ts"
+    }
+  }
+}
+```
+
+### 1.2 Better Tree-shaking
+- Use dynamic imports for heavy libraries (inquirer, ora)
+- Lazy loading for template files
+
+### 1.3 Result Caching
+```typescript
+// Add caching for version checks and template loading
+import { createHash } from 'crypto';
+import os from 'os';
+
+const CACHE_DIR = path.join(os.homedir(), '.rapidkit', 'cache');
+const CACHE_TTL = 24 * 60 * 60 * 1000; // 24 hours
+
+async function getCachedOrFetch(key: string, fetcher: () => Promise<any>) {
+  const cachePath = path.join(CACHE_DIR, `${createHash('md5').update(key).digest('hex')}.json`);
+  
+  if (await fsExtra.pathExists(cachePath)) {
+    const cached = await fsExtra.readJson(cachePath);
+    if (Date.now() - cached.timestamp < CACHE_TTL) {
+      return cached.data;
+    }
+  }
+  
+  const data = await fetcher();
+  await fsExtra.ensureDir(CACHE_DIR);
+  await fsExtra.writeJson(cachePath, { data, timestamp: Date.now() });
+  return data;
+}
+```
+
+## 2. Code Quality Optimizations
+
+### 2.1 Add ESLint
+```bash
+npm install -D @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint
+```
+
+```javascript
+// .eslintrc.cjs
+module.exports = {
+  parser: '@typescript-eslint/parser',
+  extends: [
+    'eslint:recommended',
+    'plugin:@typescript-eslint/recommended',
+  ],
+  parserOptions: {
+    ecmaVersion: 2022,
+    sourceType: 'module',
+  },
+  rules: {
+    '@typescript-eslint/no-explicit-any': 'warn',
+    '@typescript-eslint/no-unused-vars': ['error', { argsIgnorePattern: '^_' }],
+  },
+};
+```
+
+### 2.2 Add Prettier
+```json
+// .prettierrc
+{
+  "semi": true,
+  "trailingComma": "es5",
+  "singleQuote": true,
+  "printWidth": 100,
+  "tabWidth": 2
+}
+```
+
+### 2.3 Pre-commit Hooks
+```bash
+npm install -D husky lint-staged
+```
+
+```json
+// package.json
+{
+  "lint-staged": {
+    "*.ts": ["eslint --fix", "prettier --write"]
+  }
+}
+```
+
+## 3. User Experience (UX) Optimizations
+
+### 3.1 Better Progress Indicators
+```typescript
+// Use multi-step progress bar
+import cliProgress from 'cli-progress';
+
+const multibar = new cliProgress.MultiBar({
+  clearOnComplete: false,
+  hideCursor: true,
+  format: '{bar} | {filename} | {value}/{total}',
+});
+
+const installBar = multibar.create(100, 0, { filename: 'Installing dependencies' });
+const templateBar = multibar.create(100, 0, { filename: 'Copying templates' });
+```
+
+### 3.2 Better Error Messages with Suggestions
+```typescript
+export class PythonNotFoundError extends RapidKitError {
+  constructor() {
+    super(
+      'Python not found',
+      'PYTHON_NOT_FOUND',
+      'Python 3.10+ is required but not found in PATH.\n\n' +
+      '💡 Quick fixes:\n' +
+      '  • macOS: brew install python@3.11\n' +
+      '  • Ubuntu/Debian: sudo apt install python3.11\n' +
+      '  • Windows: Download from python.org\n' +
+      '  • Or use pyenv: pyenv install 3.11.0'
+    );
+  }
+}
+```
+
+### 3.3 Interactive Mode Improvements
+```typescript
+// Add fuzzy search for template selection
+import inquirerPrompt from 'inquirer-autocomplete-prompt';
+
+inquirer.registerPrompt('autocomplete', inquirerPrompt);
+
+await inquirer.prompt([{
+  type: 'autocomplete',
+  name: 'template',
+  message: 'Choose a template:',
+  source: async (_, input) => {
+    return templates.filter(t => 
+      t.name.includes(input?.toLowerCase() || '')
+    );
+  },
+}]);
+```
+
+## 4. Security Optimizations
+
+### 4.1 Better Input Validation
+```typescript
+import validator from 'validator';
+
+export function validateProjectName(name: string): void {
+  // Check for path traversal
+  if (name.includes('..') || name.includes('/') || name.includes('\\')) {
+    throw new ValidationError('Project name cannot contain path separators');
+  }
+  
+  // Check for dangerous characters
+  if (!validator.isAlphanumeric(name.replace(/[-_]/g, ''))) {
+    throw new ValidationError('Project name can only contain alphanumeric, dash, and underscore');
+  }
+  
+  // Check length
+  if (name.length > 100) {
+    throw new ValidationError('Project name is too long (max 100 characters)');
+  }
+}
+```
+
+### 4.2 Dependency Security
+```bash
+# Add npm audit to CI/CD
+npm audit --audit-level=moderate
+```
+
+### 4.3 Template Sanitization
+```typescript
+// Use secure template rendering
+import nunjucks from 'nunjucks';
+
+const env = new nunjucks.Environment(null, { 
+  autoescape: true,
+  throwOnUndefined: true,
+});
+```
+
+## 5. Architecture Optimizations
+
+### 5.1 Plugin System
+```typescript
+// src/plugins/plugin-manager.ts
+interface Plugin {
+  name: string;
+  version: string;
+  hooks: {
+    beforeCreate?: (context: CreateContext) => Promise<void>;
+    afterCreate?: (context: CreateContext) => Promise<void>;
+    beforeInstall?: (context: InstallContext) => Promise<void>;
+    afterInstall?: (context: InstallContext) => Promise<void>;
+  };
+}
+
+export class PluginManager {
+  private plugins: Plugin[] = [];
+  
+  register(plugin: Plugin) {
+    this.plugins.push(plugin);
+  }
+  
+  async runHook(hookName: string, context: any) {
+    for (const plugin of this.plugins) {
+      const hook = plugin.hooks[hookName];
+      if (hook) {
+        await hook(context);
+      }
+    }
+  }
+}
+```
+
+### 5.2 Modular Template System
+```typescript
+// src/templates/template-registry.ts
+export class TemplateRegistry {
+  private templates = new Map<string, Template>();
+  
+  register(id: string, template: Template) {
+    this.templates.set(id, template);
+  }
+  
+  get(id: string): Template | undefined {
+    return this.templates.get(id);
+  }
+  
+  list(): Template[] {
+    return Array.from(this.templates.values());
+  }
+}
+```
+
+### 5.3 State Machine for Installation Flow
+```typescript
+// src/core/installation-state-machine.ts
+enum State {
+  IDLE = 'idle',
+  VALIDATING = 'validating',
+  CREATING_DIR = 'creating_dir',
+  INSTALLING_DEPS = 'installing_deps',
+  COPYING_TEMPLATES = 'copying_templates',
+  INITIALIZING_GIT = 'initializing_git',
+  COMPLETE = 'complete',
+  ERROR = 'error',
+}
+
+export class InstallationStateMachine {
+  private state: State = State.IDLE;
+  
+  async transition(to: State) {
+    logger.debug(`State transition: ${this.state} -> ${to}`);
+    this.state = to;
+  }
+  
+  canTransition(to: State): boolean {
+    // Define valid transitions
+    const validTransitions = {
+      [State.IDLE]: [State.VALIDATING],
+      [State.VALIDATING]: [State.CREATING_DIR, State.ERROR],
+      // ...
+    };
+    return validTransitions[this.state]?.includes(to) ?? false;
+  }
+}
+```
+
+## 6. Testing Optimizations
+
+### 6.1 Integration Tests
+```typescript
+// src/__tests__/integration/full-flow.test.ts
+describe('Full installation flow', () => {
+  it('should create demo workspace successfully', async () => {
+    const tmpDir = await fsExtra.mkdtemp(path.join(os.tmpdir(), 'rapidkit-test-'));
+    
+    try {
+      await createProject('test-workspace', {
+        demoMode: true,
+        skipGit: true,
+      });
+      
+      expect(await fsExtra.pathExists(path.join(tmpDir, 'test-workspace'))).toBe(true);
+      expect(await fsExtra.pathExists(path.join(tmpDir, 'test-workspace', 'generate-demo.js'))).toBe(true);
+    } finally {
+      await fsExtra.remove(tmpDir);
+    }
+  });
+});
+```
+
+### 6.2 Snapshot Tests for Generated Files
+```typescript
+// src/__tests__/snapshots/templates.test.ts
+it('should generate correct pyproject.toml', async () => {
+  const result = await generateTemplate('pyproject.toml', {
+    project_name: 'test_project',
+    author: 'Test Author',
+  });
+  
+  expect(result).toMatchSnapshot();
+});
+```
+
+### 6.3 Performance Benchmarks
+```typescript
+// src/__tests__/benchmarks/creation-speed.bench.ts
+import { bench, describe } from 'vitest';
+
+describe('Creation performance', () => {
+  bench('create demo workspace', async () => {
+    await createProject('bench-test', { demoMode: true, dryRun: true });
+  });
+});
+```
+
+## 7. Documentation Optimizations
+
+### 7.1 API Documentation
+```bash
+npm install -D typedoc
+```
+
+```json
+// package.json
+{
+  "scripts": {
+    "docs": "typedoc --out docs src/index.ts"
+  }
+}
+```
+
+### 7.2 Interactive Examples
+```typescript
+// examples/programmatic-usage.ts
+import { createProject } from 'rapidkit';
+
+async function example() {
+  await createProject('my-workspace', {
+    demoMode: true,
+    skipGit: false,
+  });
+}
+```
+
+## 8. CI/CD Optimizations
+
+**Note**: CI/CD is optional. If you don't need automated workflows, you can skip this section.
+
+### 8.1 GitHub Actions Workflow
+```yaml
+# .github/workflows/ci.yml
+name: CI
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        node-version: [18, 20, 22]
+    
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          node-version: ${{ matrix.node-version }}
+      - run: npm ci
+      - run: npm test
+      - run: npm run build
+```
+
+### 8.2 Automated Releases
+```yaml
+# .github/workflows/release.yml
+name: Release
+
+on:
+  push:
+    tags:
+      - 'v*'
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+      - run: npm ci
+      - run: npm run build
+      - run: npm publish
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+```
+
+## 9. Monitoring & Analytics Optimizations
+
+### 9.1 Telemetry (Optional with User Consent)
+```typescript
+// src/telemetry.ts
+import os from 'os';
+import { getVersion } from './update-checker.js';
+
+interface TelemetryData {
+  version: string;
+  command: string;
+  options: string[];
+  platform: string;
+  nodeVersion: string;
+  success: boolean;
+  duration: number;
+}
+
+export async function sendTelemetry(data: TelemetryData) {
+  // Only if user opted-in via config
+  const config = await loadUserConfig();
+  if (!config.telemetry) return;
+  
+  try {
+    await fetch('https://telemetry.getrapidkit.com/api/events', {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({
+        ...data,
+        timestamp: new Date().toISOString(),
+      }),
+    });
+  } catch {
+    // Fail silently
+  }
+}
+```
+
+## 10. Bundle Size Optimizations
+
+### 10.1 Analyze Bundle
+```bash
+npm install -D webpack-bundle-analyzer
+```
+
+### 10.2 Replace Heavy Dependencies
+- `inquirer` → `prompts` (lighter alternative)
+- `chalk` → `picocolors` (much smaller)
+- `fs-extra` → native `fs/promises` where possible
+
+### 10.3 Code Splitting
+```typescript
+// Dynamic imports for heavy operations
+const createDemoWorkspace = async () => {
+  const { generateDemoKit } = await import('./demo-kit.js');
+  return generateDemoKit(...);
+};
+```
+
+## Implementation Priority
+
+### 🔴 High Priority (Week 1)
+1. ESLint + Prettier setup ✅
+2. Better error messages with suggestions
+3. Input validation improvements
+4. Bundle size optimization
+
+### 🟡 Medium Priority (Weeks 2-3)
+1. Plugin system
+2. Integration tests
+3. Performance benchmarks
+4. CI/CD workflows (optional)
+
+### 🟢 Low Priority (Month 2+)
+1. Telemetry system
+2. Advanced caching
+3. Multi-language support
+4. Interactive documentation
+
+## Summary
+
+These optimizations can achieve:
+- **Performance**: ~40% faster installation
+- **Bundle Size**: ~30% reduction
+- **User Experience**: Significant improvements in error handling and progress tracking
+- **Code Quality**: Higher coverage and better maintainability
+- **Security**: Reduced attack surface and better validation

package/docs/PACKAGE_MANAGER_POLICY.md

@@ -0,0 +1,25 @@
+# Package Manager Policy
+
+## Official Policy
+
+This repository is **npm-only** for development and CI workflows.
+
+## Why
+
+- CI workflows use `npm ci` as the canonical install path.
+- A single lockfile strategy reduces dependency drift and release variance.
+- Release and E2E scripts are standardized on npm commands.
+
+## Rules
+
+- Use `npm install` (or `npm ci` in CI/local clean installs).
+- Do not use `yarn` or `pnpm` for contributor workflows in this repository.
+- Keep `package-lock.json` as the canonical lockfile.
+
+## Enforcement
+
+A `preinstall` guard blocks non-npm package managers during local install.
+
+## Notes
+
+This policy applies to the `rapidkit-npm` repository itself. Generated projects may use their own package-manager conventions depending on framework and template outputs.

package/docs/README.md

@@ -0,0 +1,121 @@
+# RapidKit NPM — Documentation Index
+
+Hub for user and contributor documentation. Start with the [main README](../README.md) for install and quickstarts.
+
+## Table of contents
+
+- [User documentation](#user-documentation)
+- [Operations & security](#operations--security)
+- [AI module recommendations](#ai-module-recommendations)
+- [Technical contracts](#technical-contracts)
+- [Contributor documentation](#contributor-documentation)
+- [Validation commands](#validation-commands)
+
+## User documentation
+
+| Document | Description |
+| --- | --- |
+| [commands-reference.md](./commands-reference.md) | Full CLI syntax, profiles, and policy keys |
+| [workspace-operations.md](./workspace-operations.md) | Import, adopt, snapshots, archives, contracts, infra |
+| [workspace-run.md](./workspace-run.md) | Polyglot fleet orchestration (`workspace run`) |
+| [create-planner-capabilities.md](./create-planner-capabilities.md) | Native create, external-create-adopt, and adopt-only lanes |
+| [from-code-to-shared-understanding.md](./from-code-to-shared-understanding.md) | GitHub-rendered Workspace Intelligence diagram |
+| [OPEN_SOURCE_USER_SCENARIOS.md](./OPEN_SOURCE_USER_SCENARIOS.md) | Role-based workflows (junior → enterprise) |
+| [doctor-command.md](./doctor-command.md) | Doctor scopes, CI exit codes, JSON evidence |
+| [config-file-guide.md](./config-file-guide.md) | User config file (`~/.rapidkitrc.json`, `rapidkit.config.*`) |
+| [WORKSPACE_MARKER_SPEC.md](./WORKSPACE_MARKER_SPEC.md) | Workspace marker format |
+| [PACKAGE_MANAGER_POLICY.md](./PACKAGE_MANAGER_POLICY.md) | npm-only policy for this repository |
+
+**Common tasks**
+
+- Adopt an existing repo: [workspace-operations.md#import-and-adoption](./workspace-operations.md#import-and-adoption)
+- Scaffold a frontend app: [commands-reference.md](./commands-reference.md) (`create frontend`)
+- CI release gate: [commands-reference.md](./commands-reference.md) (`pipeline`, `readiness`)
+- Agent context: `workspace model` / `workspace context` — schemas in [contracts/workspace-intelligence/](../contracts/workspace-intelligence/)
+
+## Operations & security
+
+| Document | Description |
+| --- | --- |
+| [SECURITY.md](./SECURITY.md) | Vulnerability reporting and supported versions |
+| [policies.workspace.example.yml](./policies.workspace.example.yml) | Workspace policy template |
+| [governance-policy.enterprise.example.json](./governance-policy.enterprise.example.json) | Sigstore governance allowlist template |
+| [mirror-config.enterprise.example.json](./mirror-config.enterprise.example.json) | Mirror + evidence export template |
+
+## AI module recommendations
+
+FastAPI/NestJS module suggestions via OpenAI embeddings (optional).
+
+| Document | Description |
+| --- | --- |
+| [AI_QUICKSTART.md](./AI_QUICKSTART.md) | 60-second setup |
+| [AI_FEATURES.md](./AI_FEATURES.md) | Complete feature reference |
+| [AI_EXAMPLES.md](./AI_EXAMPLES.md) | Use-case examples |
+| [AI_DYNAMIC_INTEGRATION.md](./AI_DYNAMIC_INTEGRATION.md) | Integration architecture |
+
+## Technical contracts
+
+JSON schemas and ownership rules for tooling parity.
+
+| Location | Description |
+| --- | --- |
+| [contracts/README.md](./contracts/README.md) | Core CLI JSON contracts + generator scripts |
+| [contracts/COMMAND_OWNERSHIP_MATRIX.md](./contracts/COMMAND_OWNERSHIP_MATRIX.md) | npm wrapper vs Core command ownership |
+| [contracts/RUNTIME_SUPPORT_MATRIX.md](./contracts/RUNTIME_SUPPORT_MATRIX.md) | Scaffold/import/lifecycle support tiers |
+| [contracts/RUNTIME_ACCEPTANCE_MATRIX.md](./contracts/RUNTIME_ACCEPTANCE_MATRIX.md) | Runtime acceptance test expectations |
+| [../contracts/](../contracts/) | Canonical JSON schemas (published in npm tarball) |
+
+Regenerate and verify:
+
+```bash
+npm run generate:contracts
+npm run check:generated-contracts
+npm run validate:contracts
+```
+
+## Contributor documentation
+
+| Document | Description |
+| --- | --- |
+| [DEVELOPMENT.md](./DEVELOPMENT.md) | Local dev, testing, debugging |
+| [SETUP.md](./SETUP.md) | Build gates, smoke flows, release hygiene |
+| [ci-workflows.md](./ci-workflows.md) | GitHub Actions workflow map |
+| [OPTIMIZATION_GUIDE.md](./OPTIMIZATION_GUIDE.md) | Performance and improvement notes |
+| [UTILITIES.md](./UTILITIES.md) | Internal cache and metrics helpers |
+
+Also see [../CONTRIBUTING.md](../CONTRIBUTING.md) and [../CHANGELOG.md](../CHANGELOG.md).
+
+## Validation commands
+
+```bash
+npm run validate:docs          # links + drift guard + examples + README smoke
+npm run check:markdown-links   # local markdown link integrity
+npm run validate:docs-examples # example JSON/YAML in docs
+npm run smoke:readme           # CLI help smoke for documented commands
+```
+
+## Repository layout
+
+```text
+rapidkit-npm/
+├── README.md                 # User hub (install, quickstarts, doc links)
+├── CHANGELOG.md
+├── RELEASE_NOTES.md
+├── releases/                 # Per-version release notes
+└── docs/
+    ├── README.md             # This index
+    ├── commands-reference.md
+    ├── workspace-operations.md
+    ├── workspace-run.md
+    ├── ci-workflows.md
+    ├── doctor-command.md
+    ├── OPEN_SOURCE_USER_SCENARIOS.md
+    ├── config-file-guide.md
+    ├── SECURITY.md
+    ├── SETUP.md
+    ├── DEVELOPMENT.md
+    ├── contracts/            # Contract docs (mirrors + matrices)
+    └── …                     # AI guides, policies, examples
+```
+
+Enterprise governance runbooks are maintained outside this OSS docs tree.

package/docs/SECURITY.md

@@ -0,0 +1,63 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 0.35.x (latest minor) | :white_check_mark: |
+| < 0.35.0 | :x: |
+
+During the `0.x` phase, only the latest minor line receives security fixes.
+
+## Known Security Considerations
+
+### Development Dependencies
+
+Our CI/CD pipeline may report moderate severity vulnerabilities in development dependencies (vitest, vite, esbuild). These packages are:
+
+- ✅ **Only used during development and testing**
+- ✅ **Not included in the published npm package**
+- ✅ **Not shipped to end users**
+- ✅ **Do not affect runtime security**
+
+The published package only includes runtime dependencies required for workspace creation.
+
+### Production Dependencies
+
+We actively monitor and address any security vulnerabilities in production dependencies that are shipped with the package.
+
+## Reporting a Vulnerability
+
+If you discover a security vulnerability in rapidkit, please report it by emailing **security@getrapidkit.com** or opening a private security advisory on GitHub.
+
+Please include:
+- Description of the vulnerability
+- Steps to reproduce
+- Potential impact
+- Suggested fix (if available)
+
+We will respond within 48 hours and work to address critical issues as quickly as possible.
+
+## Security Best Practices
+
+When using rapidkit:
+
+1. **Keep dependencies updated**: Run `npm update` regularly
+2. **Review generated code**: Always review the workspace structure before deployment
+3. **Use official releases**: Install from npm registry, not from git directly
+4. **Verify package integrity**: Use `npm audit` on your generated project
+
+## Security Scanning
+
+We use:
+- GitHub Security Advisories
+- npm audit (production dependencies)
+- Dependabot for automated updates
+- Regular manual security reviews
+
+## Updates
+
+Security updates are released as patch versions on the latest `0.x` minor line and announced in:
+- GitHub Releases
+- CHANGELOG.md
+- npm package updates