@latentforce/latentgraph 1.0.0

package/README.md

@@ -0,0 +1,320 @@
+# @latentforce/latentgraph
+
+AI-powered code intelligence CLI and MCP server for Claude.
+
+Shift indexes your codebase, builds a dependency relationship graph (DRG), and provides AI-powered insights via MCP tools — enabling Claude to understand your project's structure, dependencies, and blast radius of changes.
+
+## Installation
+
+```bash
+npm install -g @latentforce/latentgraph
+```
+
+## Getting an API Key
+
+1. Go to **https://dev-shift-lite.latentforce.ai/auth**
+2. **Sign up** for an account
+3. **Sign in** to your dashboard
+4. Copy your **API key** from the dashboard
+
+From the dashboard you can also **create a project** and **select a migration template** — or you can do both directly from the CLI (see below).
+
+## Quick Start
+
+All commands **must be run** from your project's root directory:
+
+```bash
+cd /path/to/your/project
+```
+
+### Option A: Interactive (recommended)
+
+```bash
+lgraph start    # Step 1: Configure API key and project, launch daemon
+lgraph init     # Step 2: Scan and index project files
+lgraph status   # Step 3: Verify everything is connected
+```
+
+When you run `lgraph start` interactively, it will:
+1. **Ask for your API key** — paste the key from your dashboard (or choose guest mode)
+2. **Ask to create or select a project** — you can either:
+   - **Select an existing project** from your account (if you created one on the dashboard)
+   - **Create a new project** from the CLI — it will prompt for a name (defaults to your directory name) and let you select a migration template
+
+### Option B: Non-interactive (CI/CD friendly)
+
+```bash
+lgraph init --api-key YOUR_KEY --project-name "My App"
+```
+
+If a project named "My App" already exists in your account, it will be reused. Otherwise a new one is created with the specified template.
+
+
+## CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `lgraph start` | Start the daemon and configure the project |
+| `lgraph init` | Scan and index project files |
+| `lgraph status` | Show current status |
+| `lgraph update-drg` | Update the dependency relationship graph |
+| `lgraph stop` | Stop the daemon |
+| `lgraph add <tool>` | Add Shift MCP server to an AI coding tool |
+| `lgraph config` | Manage configuration |
+
+### `lgraph start`
+
+Resolves authentication, configures the project, and launches a background daemon that maintains a WebSocket connection to the Shift backend.
+
+**When run interactively (no flags):**
+1. Prompts you to enter your API key or choose guest mode
+2. Prompts you to **select an existing project** or **create a new one**
+   - If creating: asks for a project name and lets you pick a migration template
+3. Saves config to `.shift/config.json`
+4. Launches the background daemon
+
+If you already created a project on the dashboard (https://dev-shift-lite.latentforce.ai), you can select it from the list. Otherwise, create one directly from the CLI.
+
+| Flag | Description |
+|------|-------------|
+| `--guest` | Use guest authentication (auto-creates a temporary project) |
+| `--api-key <key>` | Provide API key directly (skips the key prompt) |
+| `--project-name <name>` | Create new project or match existing by name (skips project prompt) |
+| `--project-id <id>` | Use existing project UUID (skips project prompt) |
+| `--template <id>` | Migration template ID for project creation |
+
+```bash
+lgraph start                                     # Interactive setup (prompts for key + project)
+lgraph start --guest                              # Quick start without API key
+lgraph start --api-key <key> --project-name "My App"  # Non-interactive
+```
+
+### `lgraph init`
+
+Performs a full project scan — collects the file tree, categorizes files (source, config, assets), gathers git info, and sends everything to the Shift backend for indexing. Automatically starts the daemon if not running.
+
+If the project is already indexed, you will be prompted to re-index. Use `--force` to skip the prompt.
+
+| Flag | Description |
+|------|-------------|
+| `--guest` | Use guest authentication (auto-creates a temporary project) |
+| `--api-key <key>` | Provide API key directly |
+| `--project-name <name>` | Create new project or match existing by name |
+| `--project-id <id>` | Use existing project UUID |
+| `--template <id>` | Migration template ID for project creation |
+| `-f, --force` | Force re-indexing even if already indexed |
+
+```bash
+lgraph init                     # Interactive initialization
+lgraph init --force             # Force re-index without prompt
+```
+
+### `lgraph status`
+
+Displays comprehensive information about the current Shift setup — API key status, project details, backend indexing state, and daemon health.
+
+```bash
+lgraph status
+```
+
+### `lgraph update-drg`
+
+Scans source files across all supported languages, detects git changes, and sends file contents to the backend for dependency analysis.
+
+**Supported languages and extensions:**
+
+| Language | Extensions |
+|----------|------------|
+| JavaScript / TypeScript | `.js`, `.jsx`, `.ts`, `.tsx`, `.mjs`, `.cjs` |
+| Python | `.py` |
+| C# | `.cs` |
+| C/C++ | `.cpp`, `.cc`, `.cxx`, `.h`, `.hpp`, `.c` |
+
+| Flag | Description |
+|------|-------------|
+| `-m, --mode <mode>` | `baseline` (all files) or `incremental` (git-changed only, default) |
+
+**Modes:**
+- **incremental** (default) — sends only git-changed files for faster updates
+- **baseline** — sends all source files for a full re-analysis
+
+```bash
+lgraph update-drg                       # Incremental update (default)
+lgraph update-drg -m baseline           # Full re-analysis
+```
+
+### `lgraph stop`
+
+Stops the background daemon process.
+
+```bash
+lgraph stop
+```
+
+### `lgraph config`
+
+Manage Shift configuration (URLs, API key).
+
+| Action | Description |
+|--------|-------------|
+| `show` | Display current configuration (default) |
+| `set <key> <value>` | Set a configuration value |
+| `clear [key]` | Clear a specific key or all configuration |
+
+**Configurable key:** `api-key`
+
+```bash
+lgraph config                                      # Show config
+lgraph config set api-key sk-abc123                 # Set API key
+lgraph config clear api-key                         # Clear API key
+lgraph config clear                                 # Clear all config
+```
+## MCP Integration
+
+After initializing your project, use `lgraph add` to configure Shift's MCP server in your AI coding tool:
+
+```bash
+lgraph add <tool>
+```
+
+### Supported tools
+
+| Tool | Command | Config method |
+|------|---------|---------------|
+| Shiftcode | `lgraph add shiftcode` | Writes `shiftcode.json` |
+| Claude Code | `lgraph add claude-code` | Runs `claude mcp add-json` |
+| Opencode | `lgraph add opencode` | Writes `opencode.json` |
+| Codex | `lgraph add codex` | Runs `codex mcp add` |
+| GitHub Copilot | `lgraph add copilot` | Writes `.vscode/mcp.json` |
+| Factory Droid | `lgraph add droid` | Runs `droid mcp add` |
+
+For CLI-based tools, if the CLI is not installed, the command will print manual setup instructions.
+
+### Claude Desktop (manual)
+
+Add to your config file (`%APPDATA%\Claude\claude_desktop_config.json` on Windows, `~/Library/Application Support/Claude/claude_desktop_config.json` on macOS):
+
+```json
+{
+  "mcpServers": {
+    "shift": {
+      "command": "lgraph",
+      "args": ["mcp"],
+      "env": {
+        "SHIFT_PROJECT_ID": "YOUR_PROJECT_ID"
+      }
+    }
+  }
+}
+```
+## `.shift/scan_target.json`
+
+Shift uses a `scan_target.json` file inside the `.shift/` directory to let you specify which parts of your codebase to scan and in which language. This is especially useful for monorepos or projects with multiple languages.
+
+A default template is **automatically created** the first time you run `lgraph init` or `lgraph start`. By default, it is unconfigured and the server will auto-detect scan targets. Edit the file to manually specify targets.
+
+### Valid Languages
+
+```
+javascript, typescript, python, cpp, csharp
+```
+
+### Format
+
+The file is a JSON array of objects, each with:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `language` | `string \| null` | One of the valid languages above, or `null` for auto-detect |
+| `path` | `string` | Relative path from project root (empty string `""` for root) |
+
+### Default Template (auto-generated)
+
+```json
+[
+  {
+    "language": null,
+    "path": ""
+  }
+]
+```
+
+When left as-is (single entry with `language: null` and `path: ""`), the server auto-detects scan targets.
+
+### Examples
+
+**Single-language project (TypeScript at root):**
+
+```json
+[
+  {
+    "language": "typescript",
+    "path": ""
+  }
+]
+```
+
+**Monorepo with multiple languages:**
+
+```json
+[
+  {
+    "language": "typescript",
+    "path": "frontend"
+  },
+  {
+    "language": "python",
+    "path": "backend"
+  },
+  {
+    "language": "csharp",
+    "path": "services/auth"
+  }
+]
+```
+
+**C++ project in a subdirectory:**
+
+```json
+[
+  {
+    "language": "cpp",
+    "path": "engine/src"
+  }
+]
+```
+
+**Mixed JavaScript and Python at root:**
+
+```json
+[
+  {
+    "language": "javascript",
+    "path": ""
+  },
+  {
+    "language": "python",
+    "path": ""
+  }
+]
+```
+
+## MCP Tools
+
+| Tool | Description |
+|------|-------------|
+| `project_overview` | Get the highest-level context for the project: architecture, tech stack, entry points, and top-level modules |
+| `file_summary` | Get a summary of a file with optional parent directory context |
+| `module_summary` | Get full documentation for the module that owns a given file, including DRG cluster metadata and parent/child module relationships |
+| `dependencies` | Get all dependencies for a file with relationship summaries |
+| `blast_radius` | Analyze what files would be affected if a file is modified or deleted |
+
+Each tool accepts an optional `project_id` parameter. If not provided, it falls back to the `SHIFT_PROJECT_ID` environment variable.
+
+## Configuration Files
+
+| File | Location | Description |
+|------|----------|-------------|
+| Global config | `~/.shift/config.json` | API key and server URLs |
+| Project config | `.shift/config.json` | Project ID, name, and agent info |
+| Scan targets | `.shift/scan_target.json` | Language and path targets for scanning (auto-created) |

package/build/cli/commands/add.js

@@ -0,0 +1,350 @@
+import { exec } from 'child_process';
+import { promisify } from 'util';
+import * as fs from 'fs';
+import * as path from 'path';
+import { getProjectId } from '../../utils/config.js';
+const execAsync = promisify(exec);
+const SUPPORTED_TOOLS = ['shiftcode', 'claude-code', 'opencode', 'codex', 'copilot', 'droid'];
+function isNotFound(e) {
+    return e.code === 'ENOENT' || /ENOENT|not found|not recognized/.test(e.message);
+}
+function getStderr(e) {
+    return (e.stderr || e.message || '').trim();
+}
+function shellEscape(arg) {
+    if (/^[a-zA-Z0-9._\-=/]+$/.test(arg))
+        return arg;
+    return `"${arg.replace(/"/g, '\\"')}"`;
+}
+function buildCommand(bin, args) {
+    return [bin, ...args.map(shellEscape)].join(' ');
+}
+function getMcpConfig(projectId) {
+    return {
+        command: 'lgraph',
+        args: ['mcp'],
+        env: { SHIFT_PROJECT_ID: projectId },
+    };
+}
+const CLAUDE_MD_SECTION_MARKER = '<!-- shift-mcp-instructions -->';
+const CLAUDE_MD_CONTENT = `${CLAUDE_MD_SECTION_MARKER}
+## Shift MCP Tools — MANDATORY USAGE RULES
+
+This project has the **Shift MCP server** (\`shift\`) configured with a pre-built knowledge graph of the entire codebase. You MUST use its tools as the primary way to understand code. Do NOT rely solely on reading raw files.
+
+### Tools
+
+| Tool | Purpose |
+|------|---------|
+| \`file_summary\` | Get an summary of any file |
+| \`dependencies\` | List every file that a given file imports/depends on |
+| \`blast_radius\` | List every file that would be affected if a given file changed |
+
+### Supported file types
+
+The knowledge graph is built **only** from source files. The tools will return an error for any other file type.
+
+**Indexed (use MCP tools):**
+\`.js\` \`.jsx\` \`.ts\` \`.tsx\` \`.py\` \`.java\` \`.cpp\` \`.cs\` \`.go\` \`.c\` \`.h\` \`.css\` \`.scss\` \`.html\`
+
+**NOT indexed (read directly with normal file tools):**
+\`.json\` \`.yaml\` \`.yml\` \`.toml\` \`.env\` \`.md\` \`.txt\` \`.pdf\` \`.png\` \`.lock\` \`.xml\` \`.csv\` and any other non-source format.
+
+Do NOT call \`file_summary\`, \`dependencies\`, or \`blast_radius\` on non-indexed files — it will fail. Read those files directly instead.
+
+---
+
+### RULE 1 — Codebase exploration (e.g. "explain the codebase", "how does this project work")
+
+You MUST call \`file_summary\` on **multiple key files** to build a complete picture. Do not stop after one call.
+
+Follow this protocol:
+1. Identify the likely entry points (e.g. \`main.ts\`, \`index.ts\`, \`app.py\`, \`server.ts\`, \`main.py\`, route files, CLI entry files).
+2. Call \`file_summary\` on each entry point (run calls in parallel where possible).
+3. Call \`dependencies\` on the most central files to discover the module graph.
+4. Call \`file_summary\` on the key modules discovered in step 3.
+5. Synthesise all results into a coherent explanation.
+
+Never answer a "explain the whole project" question from a single \`file_summary\` call.
+
+---
+
+### RULE 2 — Before reading any file
+
+ALWAYS call \`file_summary\` first. Only open the raw file if you need implementation details that the summary does not cover.
+
+---
+
+### RULE 3 — Before editing any file
+
+ALWAYS call \`blast_radius\` on that file first. List the affected files in your plan before making any changes.
+
+---
+
+### RULE 4 — When tracing a bug or data flow
+
+Use \`dependencies\` to follow the chain rather than grepping manually. Start from the file where the symptom appears and walk the dependency graph.
+
+---
+
+### Usage reference
+
+\`\`\`
+# Understand a file
+file_summary(file_path="src/server.ts")
+
+# Understand with folder context (level = number of parent dirs to include)
+file_summary(file_path="src/server.ts", level=1)
+
+# See what a file imports
+dependencies(file_path="src/server.ts")
+
+# See what breaks if this file changes
+blast_radius(file_path="src/server.ts")
+\`\`\`
+
+> Run \`lgraph update-drg\` after significant code changes to keep the knowledge graph current.
+<!-- end-shift-mcp-instructions -->
+`;
+const CLAUDE_MD_END_MARKER = '<!-- end-shift-mcp-instructions -->';
+function createClaudeMd(projectRoot) {
+    const claudeMdPath = path.join(projectRoot, 'CLAUDE.md');
+    if (fs.existsSync(claudeMdPath)) {
+        const existing = fs.readFileSync(claudeMdPath, 'utf-8');
+        if (existing.includes(CLAUDE_MD_SECTION_MARKER)) {
+            // Replace the existing section (start marker to end marker inclusive)
+            const start = existing.indexOf(CLAUDE_MD_SECTION_MARKER);
+            const end = existing.indexOf(CLAUDE_MD_END_MARKER);
+            if (end !== -1) {
+                const before = existing.slice(0, start).trimEnd();
+                const after = existing.slice(end + CLAUDE_MD_END_MARKER.length).trimStart();
+                const updated = (before ? before + '\n\n' : '') + CLAUDE_MD_CONTENT + (after ? '\n\n' + after : '');
+                fs.writeFileSync(claudeMdPath, updated);
+            }
+            else {
+                // Malformed — just overwrite from the marker onwards
+                const before = existing.slice(0, existing.indexOf(CLAUDE_MD_SECTION_MARKER)).trimEnd();
+                fs.writeFileSync(claudeMdPath, (before ? before + '\n\n' : '') + CLAUDE_MD_CONTENT);
+            }
+            console.log('  Updated Shift MCP instructions in CLAUDE.md.');
+            return;
+        }
+        // Append section to existing file
+        fs.writeFileSync(claudeMdPath, existing.trimEnd() + '\n\n' + CLAUDE_MD_CONTENT);
+        console.log('  Updated CLAUDE.md with Shift MCP instructions.');
+    }
+    else {
+        fs.writeFileSync(claudeMdPath, CLAUDE_MD_CONTENT);
+        console.log('  Created CLAUDE.md with Shift MCP instructions.');
+    }
+}
+async function addClaudeCode(projectId, projectRoot) {
+    const jsonPayload = JSON.stringify({
+        type: 'stdio',
+        command: 'lgraph',
+        args: ['mcp'],
+        env: { SHIFT_PROJECT_ID: projectId },
+    });
+    try {
+        await execAsync(buildCommand('claude', ['mcp', 'add-json', 'shift', jsonPayload]));
+        console.log('  Added Shift MCP server to Claude Code.');
+        console.log('\n  Verify with: claude mcp list');
+    }
+    catch (e) {
+        const err = e;
+        const stderr = getStderr(err);
+        if (/already exists/.test(stderr)) {
+            console.log('  Shift MCP server is already configured in Claude Code.');
+            console.log('  To update, remove it first: claude mcp remove shift');
+        }
+        else if (isNotFound(err)) {
+            console.log('  "claude" CLI not found. Add manually:\n');
+            console.log(`  claude mcp add-json shift '${jsonPayload}'`);
+        }
+        else {
+            console.log(`  Failed: ${stderr}`);
+            console.log('\n  Try adding manually:\n');
+            console.log(`  claude mcp add-json shift '${jsonPayload}'`);
+        }
+    }
+    createClaudeMd(projectRoot);
+}
+async function addCodex(projectId) {
+    const config = getMcpConfig(projectId);
+    const args = ['mcp', 'add', 'shift'];
+    for (const [k, v] of Object.entries(config.env)) {
+        args.push('--env', `${k}=${v}`);
+    }
+    args.push('--', config.command, ...config.args);
+    const manualCmd = `codex ${args.join(' ')}`;
+    try {
+        await execAsync(buildCommand('codex', args));
+        console.log('  Added Shift MCP server to Codex.');
+        console.log('\n  Verify with: codex mcp list');
+    }
+    catch (e) {
+        const err = e;
+        const stderr = getStderr(err);
+        if (isNotFound(err)) {
+            console.log('  "codex" CLI not found. Make sure Codex CLI is installed and on your PATH.\n');
+            console.log('  Once installed, add manually:\n');
+        }
+        else if (/already exists/.test(stderr)) {
+            console.log('  Shift MCP server is already configured in Codex.');
+            console.log('  To update, remove it first: codex mcp remove shift');
+            return;
+        }
+        else {
+            console.log(`  Failed: ${stderr}`);
+            console.log('\n  Try adding manually:\n');
+        }
+        console.log(`  ${manualCmd}`);
+    }
+}
+async function addFactoryDroid(projectId) {
+    const config = getMcpConfig(projectId);
+    const args = ['mcp', 'add', 'shift', config.command];
+    for (const [k, v] of Object.entries(config.env)) {
+        args.push('--env', `${k}=${v}`);
+    }
+    const manualCmd = `droid ${args.join(' ')}`;
+    try {
+        await execAsync(buildCommand('droid', args));
+        console.log('  Added Shift MCP server to Factory Droid.');
+        console.log('\n  Verify: type /mcp within droid to see configured servers');
+    }
+    catch (e) {
+        const err = e;
+        console.log(`  Failed: ${getStderr(err)}`);
+        console.log('\n  Try adding manually:\n');
+        console.log(`  ${manualCmd}`);
+    }
+}
+function mergeJsonConfig(filePath, serverKey, serverValue) {
+    let config = {};
+    if (fs.existsSync(filePath)) {
+        try {
+            config = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
+        }
+        catch {
+            console.log(`  Warning: Could not parse existing ${filePath}, creating new file.`);
+            config = {};
+        }
+    }
+    // Navigate/create nested keys
+    let obj = config;
+    for (let i = 0; i < serverKey.length - 1; i++) {
+        const key = serverKey[i];
+        if (typeof obj[key] !== 'object' || obj[key] === null) {
+            obj[key] = {};
+        }
+        obj = obj[key];
+    }
+    const finalKey = serverKey[serverKey.length - 1];
+    obj[finalKey] = serverValue;
+    // Ensure parent directory exists
+    const dir = path.dirname(filePath);
+    if (!fs.existsSync(dir)) {
+        fs.mkdirSync(dir, { recursive: true });
+    }
+    fs.writeFileSync(filePath, JSON.stringify(config, null, 2) + '\n');
+}
+function addOpencode(projectId, projectRoot) {
+    const filePath = path.join(projectRoot, 'opencode.json');
+    let existing = {};
+    if (fs.existsSync(filePath)) {
+        try {
+            existing = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
+        }
+        catch { /* will be recreated */ }
+    }
+    if (!existing['$schema']) {
+        existing['$schema'] = 'https://opencode.ai/config.json';
+        const dir = path.dirname(filePath);
+        if (!fs.existsSync(dir))
+            fs.mkdirSync(dir, { recursive: true });
+        fs.writeFileSync(filePath, JSON.stringify(existing, null, 2) + '\n');
+    }
+    mergeJsonConfig(filePath, ['mcp', 'shift'], {
+        type: 'local',
+        command: ["lgraph", "mcp"],
+        enabled: true,
+        environment: { "SHIFT_PROJECT_ID": projectId },
+    });
+    console.log(`  Updated ${filePath}`);
+    console.log('\n  Verify: check "mcp.shift" in opencode.json');
+}
+function addShiftcode(projectId, projectRoot) {
+    const filePath = path.join(projectRoot, 'shiftcode.json');
+    let existing = {};
+    if (fs.existsSync(filePath)) {
+        try {
+            existing = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
+        }
+        catch { /* will be recreated */ }
+    }
+    if (!existing['$schema']) {
+        existing['$schema'] = 'https://opencode.ai/config.json';
+        const dir = path.dirname(filePath);
+        if (!fs.existsSync(dir))
+            fs.mkdirSync(dir, { recursive: true });
+        fs.writeFileSync(filePath, JSON.stringify(existing, null, 2) + '\n');
+    }
+    mergeJsonConfig(filePath, ['mcp', 'shift'], {
+        type: 'local',
+        command: ["lgraph", "mcp"],
+        enabled: true,
+        environment: { "SHIFT_PROJECT_ID": projectId },
+    });
+    console.log(`  Updated ${filePath}`);
+    console.log('\n  Verify: check "mcp.shift" in shiftcode.json');
+}
+function addCopilot(projectId, projectRoot) {
+    const config = getMcpConfig(projectId);
+    const filePath = path.join(projectRoot, '.vscode', 'mcp.json');
+    mergeJsonConfig(filePath, ['servers', 'shift'], {
+        command: config.command,
+        args: config.args,
+        env: config.env,
+    });
+    console.log(`  Updated ${filePath}`);
+    console.log('\n  Verify: check "servers.shift" in .vscode/mcp.json');
+}
+export async function addCommand(tool) {
+    if (!SUPPORTED_TOOLS.includes(tool)) {
+        console.error(`\n  Unknown tool: "${tool}"\n`);
+        console.error('  Supported tools:');
+        for (const t of SUPPORTED_TOOLS) {
+            console.error(`    - ${t}`);
+        }
+        process.exit(1);
+    }
+    const projectRoot = process.cwd();
+    const projectId = getProjectId(projectRoot);
+    if (!projectId) {
+        console.error('\n  No project configured. Run "lgraph init" first to set up your project.\n');
+        process.exit(1);
+    }
+    console.log(`\n  Configuring Shift MCP for ${tool}...\n`);
+    switch (tool) {
+        case 'shiftcode':
+            addShiftcode(projectId, projectRoot);
+            break;
+        case 'claude-code':
+            await addClaudeCode(projectId, projectRoot);
+            break;
+        case 'opencode':
+            addOpencode(projectId, projectRoot);
+            break;
+        case 'codex':
+            await addCodex(projectId);
+            break;
+        case 'copilot':
+            addCopilot(projectId, projectRoot);
+            break;
+        case 'droid':
+            await addFactoryDroid(projectId);
+            break;
+    }
+    console.log('');
+}