@deltrace/cli 0.1.2 → 0.1.4

package/README.md

@@ -1,33 +1,100 @@
-# @deltrace/cli
+# compression-server
 
-Deltrace compresses your codebase into a high-density context block and injects it into every Claude Code or Codex request automatically — no manual file sharing, no wasted context.
+The chunking engine, AI proxy, and user API for Deltrace. Parses source files with tree-sitter, stores semantic code chunks in RDS via S3, and injects compressed codebase context into every Claude or Codex request.
 
-## Install
+---
+
+## How it works
+
+Before every prompt, a `UserPromptSubmit` hook runs `sync.ts`. It walks the current working directory, hashes each file, and POSTs changed files to the **chunker**. The chunker parses each file with tree-sitter into semantic `CodeChunk[]` objects and stores them in RDS (keyed by content hash — unchanged files are skipped). When a Claude Code or Codex request arrives at the **proxy**, it creates a job in the `jobs` table, pushes it to SQS, and a **Minions inference worker** (Qwen2.5-Coder-14B on a GPU box) pulls the message, runs the MinionS protocol against all workspace chunks, and writes a compressed codebase summary back. The proxy polls the row, injects the summary as `<codebase_context>` into the system prompt, and forwards to `api.anthropic.com` or `api.openai.com` with the user's own API key. If the worker doesn't finish in time, the proxy degrades gracefully and forwards without context.
+
+---
+
+## Processes
+
+Three PM2 processes, each on its own port. nginx terminates TLS on 443 and routes by path prefix — ports 3000/3001/3002 are never exposed to the internet.
+
+| Process | Port | Handles |
+|---|---|---|
+| `proxy` | 3000 | `/ws/:id/v1/*` — AI request forwarding, pure I/O |
+| `chunker` | 3001 | `/upload`, `/auth/activate` — file ingestion, tree-sitter |
+| `api` | 3002 | `/api/*` — user auth, API keys, repos, usage stats |
+
+---
+
+## Codebase structure
 
-```bash
-npm install -g @deltrace/cli
 ```
+src/
+├── proxy.ts          — Port 3000. AI proxy routes.
+├── chunker-server.ts — Port 3001. File ingestion.
+├── api-server.ts     — Port 3002. User-facing API server.
+├── userApi.ts        — Route handlers for api-server (me, keys, repos, usage).
+├── store.ts          — All PostgreSQL access. Fetches DB credentials from AWS
+│                       Secrets Manager at startup — no DATABASE_URL needed in env.
+├── session.ts        — Redis session store. Falls back to in-memory Map in dev.
+├── s3.ts             — Chunk blob storage helpers.
+├── sqs.ts            — SQS helpers for Minions job dispatch.
+├── chunker.ts        — AST-based semantic chunking via tree-sitter.
+├── treeSitter.ts     — WASM tree-sitter init and language loading.
+├── languageMap.ts    — File extension → language + WASM mapping.
+├── tokenizer.ts      — Token estimation and overlap splitting.
+├── types.ts          — Shared types (CodeChunk, CompressedChunk, etc.)
+├── sync.ts           — Pre-prompt hook: walks cwd, uploads changed files.
+├── login.ts          — OAuth setup for Claude Code.
+├── login-codex.ts    — OAuth setup for Codex.
+└── cli.ts            — deltrace help entry point.
+```
+
+---
+
+## Infrastructure
+
+| Component | Details |
+|---|---|
+| EC2 | t3.medium — proxy + chunker + api as PM2 processes |
+| Database | RDS PostgreSQL — credentials auto-rotated via Secrets Manager |
+| Credentials | EC2 IAM role — no API keys in env |
+| Session state | ElastiCache Redis (in-memory fallback for dev) |
+| Chunk storage | S3 |
+| Job queue | SQS → Minions GPU worker |
+| TLS | nginx + Let's Encrypt |
+| Process manager | PM2 |
+
+---
 
-## Setup
+## Commands
 
-**Claude Code:**
 ```bash
-deltrace-claude
+npm run dev          # run with ts-node
+npm run build        # compile TypeScript → dist/
+npm start            # run compiled output
+
+npm run compressor:claude  # OAuth setup for Claude Code
+npm run compressor:codex   # OAuth setup for Codex
+npm run teardown:claude    # undo Claude Code setup
+npm run teardown:codex     # undo Codex setup
+npm run sync               # manually upload cwd to server
 ```
 
-**Codex:**
+## Deploy (EC2)
+
 ```bash
-deltrace-codex
+git pull && npm run build && pm2 restart all
 ```
 
-Both commands open a browser, sign you in, and configure your agent automatically.
+## Migrations
 
-## Uninstall
+Run once against RDS when deploying schema changes:
 
 ```bash
-deltrace-teardown
+psql $DATABASE_URL -f migrations/001_compression_tables.sql
+psql $DATABASE_URL -f migrations/002_invite.sql
 ```
 
-## How it works
+## Publishing the CLI
 
-A hook runs before every prompt. It walks your working directory, uploads changed files to the Deltrace service, and the proxy injects a compressed codebase summary into the system prompt before forwarding to Anthropic or OpenAI. Your API keys are forwarded directly and never stored.
+```bash
+npm version patch --no-git-tag-version
+npm publish --access public
+```

package/dist/cli.js

@@ -0,0 +1,39 @@
+#!/usr/bin/env node
+"use strict";
+const [cmd, sub] = process.argv.slice(2);
+const help = `
+  Deltrace — codebase context compression for AI coding agents
+
+  Commands:
+    deltrace setup claude   Set up Claude Code (writes ~/.claude/settings.json)
+    deltrace setup codex    Set up Codex (writes ~/.codex/config.toml)
+    deltrace teardown       Remove Deltrace config and hooks from this machine
+
+  Sync runs automatically before every prompt once set up.
+  Your API keys are forwarded directly and never stored.
+
+  docs: deltrace.com
+`;
+if (!cmd || cmd === 'help' || cmd === '--help' || cmd === '-h') {
+    console.log(help);
+    process.exit(0);
+}
+if (cmd === 'setup') {
+    if (sub === 'claude') {
+        require('./login');
+        process.exit(0);
+    }
+    if (sub === 'codex') {
+        require('./login-codex');
+        process.exit(0);
+    }
+    console.error(`  Unknown agent: ${sub ?? '(none)'}\n  Usage: deltrace setup claude|codex`);
+    process.exit(1);
+}
+if (cmd === 'teardown') {
+    require('./teardown');
+    require('./teardown-codex');
+    process.exit(0);
+}
+console.error(`  Unknown command: ${cmd}\n${help}`);
+process.exit(1);

package/package.json

@@ -1,8 +1,9 @@
 {
   "name": "@deltrace/cli",
-  "version": "0.1.2",
+  "version": "0.1.4",
   "description": "Codebase context compression for AI coding agents",
   "files": [
+    "dist/cli.js",
     "dist/login.js",
     "dist/login-codex.js",
     "dist/sync.js",
@@ -10,10 +11,8 @@
     "dist/teardown-codex.js"
   ],
   "bin": {
-    "deltrace-claude": "dist/login.js",
-    "deltrace-codex": "dist/login-codex.js",
-    "deltrace-sync": "dist/sync.js",
-    "deltrace-teardown": "dist/teardown.js"
+    "deltrace": "dist/cli.js",
+    "deltrace-sync": "dist/sync.js"
   },
   "scripts": {
     "dev": "ts-node src/index.ts",
@@ -32,6 +31,7 @@
   "dependencies": {
     "@aws-sdk/client-s3": "^3.1042.0",
     "@aws-sdk/client-secrets-manager": "^3.1043.0",
+    "@aws-sdk/client-sesv2": "^3.1044.0",
     "@aws-sdk/client-sqs": "^3.0.0",
     "aws-jwt-verify": "^5.1.1",
     "ioredis": "^5.3.0",