@c4a/extract 0.5.29-beta.18

package/README.md

@@ -0,0 +1,154 @@
+# @c4a/extract
+
+Code extraction framework: plugin protocol, registry, digest generation, and shared parsing infrastructure.
+
+## Role in the monorepo
+
+Defines the standard extraction protocol (`ExtractionPlugin` interface) and provides:
+- Plugin registry for language-specific extraction plugins
+- ExtractionResult v2 schema (SymbolInfo + RelationInfo)
+- Digest generation (exported/internal symbol partitioning) and symbol diff
+- Shared Tree-sitter parsing and file scanning utilities
+
+**Depends on:** `core`
+**Depended on by:** `extract-ts`, `daemon`, `api`, `e2e`
+
+## Key exports
+
+### Plugin protocol
+
+- `ExtractionPlugin` — interface that language plugins implement
+- `ExtractionPluginRegistry` — register/resolve plugins by source manifest
+- `FileSystem` — abstraction for file access (plugins don't touch disk directly)
+- `SourceInfo` / `ManifestInfo` — source metadata passed to `canHandle`/`detectEntries`
+- `EntryDetectionResult` — package info + entry files (supports monorepo recursion via `subPackages`)
+- `PatternDetectionResult` — framework pattern detection (endpoints, implicit deps)
+
+### Types (ExtractionResult v2)
+
+- `ExtractionResult` — standard output format: `{ version: "2", meta, package, files, symbols, relations, stats }`
+- `SymbolInfo` — extracted symbol: name, kind, visibility, file, line, members, params, returnType, etc.
+- `RelationInfo` — extracted relation: type, from, to, grounding, confidence, source
+
+### Digest & diff
+
+- `generateDigest(result)` — ExtractionResult -> DigestData (exported/internal symbol partitioning)
+- `generateSymbolDiff(current, previous)` — compare two digests, output `{ added, removed }`
+- `DigestData` / `SymbolDiff` types
+
+### Shared utilities
+
+- `initParser()` / `parseFile()` — Tree-sitter parser initialization and per-file AST parsing
+- `scanSourceFiles()` / `detectModules()` / `detectModuleAt()` — file system scanning and module detection
+- `getGitCommitHash()` — reads the current git commit hash
+- `detectTechStack()` — identifies languages and frameworks in a repository
+
+## Writing a language plugin
+
+To add support for a new language (e.g., Python), create a new package `@c4a/extract-python` that implements `ExtractionPlugin`:
+
+```typescript
+import type {
+  ExtractionPlugin,
+  ExtractionResult,
+  EntryDetectionResult,
+  FileSystem,
+  ManifestInfo,
+  SourceInfo,
+  EntryFile,
+} from "@c4a/extract";
+
+export class PythonPlugin implements ExtractionPlugin {
+  readonly id = "c4a-extract-python";
+  readonly languages = ["python"];
+  readonly packageManagers = ["pip"];
+
+  canHandle(source: SourceInfo): boolean {
+    // Return true if this source has a pyproject.toml or setup.py
+    return source.manifests.some(
+      (m) => m.type === "pyproject.toml",
+    );
+  }
+
+  async detectEntries(
+    manifest: ManifestInfo,
+    fs: FileSystem,
+  ): Promise<EntryDetectionResult> {
+    // Parse pyproject.toml to find package name, kind, entry points
+    // Return: { package: { name, kind, language }, entries: [...] }
+  }
+
+  async extractSymbols(
+    entries: EntryFile[],
+    fs: FileSystem,
+  ): Promise<ExtractionResult> {
+    // Use Tree-sitter with Python grammar to parse AST
+    // Extract SymbolInfo[] (functions, classes, etc.)
+    // Extract RelationInfo[] (imports, inheritance, etc.)
+    // Return ExtractionResult v2
+  }
+}
+```
+
+### Plugin interface
+
+| Method | When called | Returns |
+|--------|------------|---------|
+| `canHandle(source)` | Registry resolves which plugin handles a source | `boolean` |
+| `detectEntries(manifest, fs)` | Layer 0: identify package info + entry files | `EntryDetectionResult` |
+| `extractSymbols(entries, fs)` | Layer 1: AST parsing + symbol extraction | `ExtractionResult` |
+| `detectPatterns?(fs)` | Layer 2 (optional): framework-specific patterns | `PatternDetectionResult` |
+
+### Registration
+
+Plugins are currently registered statically in the daemon:
+
+```typescript
+import { ExtractionPluginRegistry } from "@c4a/extract";
+import { TypeScriptPlugin } from "@c4a/extract-ts";
+
+const registry = new ExtractionPluginRegistry();
+registry.register(new TypeScriptPlugin());
+
+const plugin = registry.resolve(sourceInfo);
+if (plugin) {
+  const entries = await plugin.detectEntries(manifest, fs);
+  const result = await plugin.extractSymbols(entries.entries, fs);
+}
+```
+
+### FileSystem abstraction
+
+Plugins access files through the `FileSystem` interface, not directly via `node:fs`. This enables future API-based file loading without changing plugin code.
+
+```typescript
+interface FileSystem {
+  readFile(path: string): Promise<string>;
+  readdir(path: string): Promise<string[]>;
+  exists(path: string): Promise<boolean>;
+  readJson<T = unknown>(path: string): Promise<T>;
+}
+```
+
+### ExtractionResult v2 format
+
+```typescript
+{
+  version: "2",
+  meta: { extractedAt, pluginId, commitHash, language },
+  package: { name, kind, language },
+  files: [{ path, language, lines }],
+  symbols: SymbolInfo[],   // exported + internal
+  relations: RelationInfo[], // imports, calls, type refs, etc.
+  stats: { files, lines, exportedSymbols, internalSymbols, relations },
+}
+```
+
+## Development
+
+```bash
+bun run --filter @c4a/extract build
+bun run --filter @c4a/extract typecheck
+bun run --filter @c4a/extract test
+bun run --filter @c4a/extract lint
+```