@typicalday/firegraph 0.1.0 → 0.3.0

package/README.md

@@ -2,7 +2,7 @@
 
 > **Warning:** This library is experimental. APIs may change without notice between releases.
 
-A typed graph data layer for Firebase Cloud Firestore. Store nodes and edges in a single collection with smart query planning, sharded document IDs, optional schema validation, and multi-hop traversal.
+A typed graph data layer for Firebase Cloud Firestore. Store nodes and edges as triples in a Firestore collection with smart query planning, sharded document IDs, optional schema validation, multi-hop traversal, and nested subgraphs.
 
 ## Install
 
@@ -57,7 +57,7 @@ const departures = await g.findEdges({ aUid: tourId, axbType: 'hasDeparture' });
 
 ### Graph Model
 
-Firegraph stores everything as **triples** in a single Firestore collection:
+Firegraph stores everything as **triples** in a Firestore collection (with optional nested subcollections for [subgraphs](#subgraphs)):
 
 ```
 (aType, aUid) -[axbType]-> (bType, bUid)
@@ -193,7 +193,7 @@ await batch.commit();
 
 ### Graph Traversal
 
-Multi-hop traversal with budget enforcement, concurrency control, and in-memory filtering:
+Multi-hop traversal with budget enforcement, concurrency control, in-memory filtering, and cross-graph hops:
 
 ```typescript
 import { createTraversal } from 'firegraph';
@@ -213,6 +213,8 @@ result.totalReads; // number — Firestore reads consumed
 result.truncated;  // boolean — true if budget was hit
 ```
 
+`createTraversal` accepts a `GraphClient` or `GraphReader`. When passed a `GraphClient`, cross-graph hops via `targetGraph` are supported (see [Cross-Graph Edges](#cross-graph-edges)).
+
 #### Reverse Traversal
 
 Walk edges backwards to find parents:
@@ -249,6 +251,7 @@ await g.runTransaction(async (tx) => {
 | `limit` | `number` | `10` | Max edges per source node |
 | `orderBy` | `{ field, direction? }` | — | Firestore-level ordering |
 | `filter` | `(edge) => boolean` | — | In-memory post-filter |
+| `targetGraph` | `string` | — | Subgraph to cross into (forward only). See [Cross-Graph Edges](#cross-graph-edges) |
 
 #### Run Options
 
@@ -341,6 +344,241 @@ Key behaviors:
 
 Dynamic registry returns a `DynamicGraphClient` which extends `GraphClient` with `defineNodeType()`, `defineEdgeType()`, and `reloadRegistry()`. Transactions and batches also validate against the compiled dynamic registry.
 
+### Subgraphs
+
+Create isolated graph namespaces inside a parent node's Firestore document as subcollections. Each subgraph is a full `GraphClient` scoped to its own collection path.
+
+```typescript
+const agentId = generateId();
+await g.putNode('agent', agentId, { name: 'ResearchBot' });
+
+// Create a subgraph under the agent's document
+const memories = g.subgraph(agentId, 'memories');
+
+// CRUD works exactly like the parent client
+const memId = generateId();
+await memories.putNode('memory', memId, { text: 'The sky is blue' });
+const mem = await memories.getNode(memId);
+
+// Subgraph data is isolated — parent can't see it
+const parentNodes = await g.findNodes({ aType: 'memory', allowCollectionScan: true });
+// → [] (empty — memories live in the subcollection)
+```
+
+#### Nested Subgraphs
+
+Subgraphs can be nested to any depth:
+
+```typescript
+const workspace = g.subgraph(agentId, 'workspace');
+const taskId = generateId();
+await workspace.putNode('task', taskId, { name: 'Analyze data' });
+
+// Nest further
+const subtasks = workspace.subgraph(taskId, 'subtasks');
+await subtasks.putNode('subtask', generateId(), { name: 'Parse CSV' });
+```
+
+#### Scope Constraints (`allowedIn`)
+
+Registry entries support `allowedIn` patterns that restrict where a type can be used:
+
+```typescript
+const registry = createRegistry([
+  { aType: 'agent', axbType: 'is', bType: 'agent', allowedIn: ['root'] },
+  { aType: 'memory', axbType: 'is', bType: 'memory', allowedIn: ['**/memories'] },
+  { aType: 'task', axbType: 'is', bType: 'task', allowedIn: ['workspace', '**/workspace'] },
+]);
+
+const g = createGraphClient(db, 'graph', { registry });
+
+// Agent only at root
+await g.putNode('agent', agentId, {}); // OK
+await memories.putNode('agent', generateId(), {}); // throws RegistryScopeError
+
+// Memory only in 'memories' subgraphs
+await memories.putNode('memory', generateId(), {}); // OK
+await g.putNode('memory', generateId(), {}); // throws RegistryScopeError
+```
+
+**Pattern syntax:**
+
+| Pattern | Matches |
+|---------|---------|
+| `root` | Top-level collection only |
+| `memories` | Exact subgraph name |
+| `workspace/tasks` | Exact path |
+| `*/memories` | `*` matches one segment |
+| `**/memories` | `**` matches zero or more segments |
+| `**` | Everything including root |
+
+Omitting `allowedIn` (or passing an empty array) means the type is allowed everywhere.
+
+#### Transactions & Batches in Subgraphs
+
+```typescript
+const sub = g.subgraph(agentId, 'memories');
+
+// Transaction
+await sub.runTransaction(async (tx) => {
+  const node = await tx.getNode(memId);
+  await tx.putNode('memory', memId, { text: 'updated' });
+});
+
+// Batch
+const batch = sub.batch();
+await batch.putNode('memory', generateId(), { text: 'first' });
+await batch.putNode('memory', generateId(), { text: 'second' });
+await batch.commit();
+```
+
+#### Cascade Delete
+
+`removeNodeCascade` recursively deletes subcollections by default:
+
+```typescript
+// Deletes the agent node, all its edges, and all subgraph data
+await g.removeNodeCascade(agentId);
+
+// To preserve subgraph data:
+await g.removeNodeCascade(agentId, { deleteSubcollections: false });
+```
+
+#### Firestore Path Layout
+
+```
+graph/                          ← root collection
+  {agentId}                     ← agent node document
+  {agentId}/memories/           ← subgraph subcollection
+    {memId}                     ← memory node document
+    {shard:aUid:rel:bUid}       ← edge document
+  {agentId}/workspace/          ← another subgraph
+    {taskId}                    ← task node document
+    {taskId}/subtasks/          ← nested subgraph
+      {subtaskId}               ← subtask node document
+```
+
+### Cross-Graph Edges
+
+Edges that connect nodes across different subgraphs. The key rule: **edges live with the target node**. A cross-graph edge is stored in the same collection as its target (bUid), while the source (aUid) may be a parent node in an ancestor graph.
+
+```typescript
+import { createGraphClient, createRegistry, createTraversal, generateId } from 'firegraph';
+
+// Registry declares that 'assignedTo' edges live in the 'workflow' subgraph
+const registry = createRegistry([
+  { aType: 'task', axbType: 'is', bType: 'task', jsonSchema: taskSchema },
+  { aType: 'agent', axbType: 'is', bType: 'agent', jsonSchema: agentSchema },
+  { aType: 'task', axbType: 'assignedTo', bType: 'agent', targetGraph: 'workflow' },
+]);
+
+const g = createGraphClient(db, 'graph', { registry });
+
+// Create a task in the root graph
+const taskId = generateId();
+await g.putNode('task', taskId, { title: 'Build API' });
+
+// Create agents in a workflow subgraph under the task
+const workflow = g.subgraph(taskId, 'workflow');
+const agentId = generateId();
+await workflow.putNode('agent', agentId, { name: 'Backend Dev' });
+
+// Create the cross-graph edge in the workflow subgraph
+// The edge lives alongside the target (agent), source (task) is an ancestor
+await workflow.putEdge('task', taskId, 'assignedTo', 'agent', agentId, { role: 'lead' });
+
+// Forward traversal: task → agents (automatically crosses into workflow subgraph)
+const result = await createTraversal(g, taskId, registry)
+  .follow('assignedTo')
+  .run();
+// result.nodes contains the agent edges from the workflow subgraph
+```
+
+#### How It Works
+
+1. **Writing**: You explicitly call `putEdge` on the subgraph client where the target node lives. The caller decides where the edge goes.
+
+2. **Reverse traversal is free**: Since the edge lives with the target, querying from the agent's perspective (`findEdges({ bUid: agentId })` on the workflow client) finds it locally.
+
+3. **Forward traversal uses `targetGraph`**: When traversing from the task, the engine sees `targetGraph: 'workflow'` on the registry entry and queries `g.subgraph(taskId, 'workflow')` automatically.
+
+4. **Path-scanning resolution**: To determine if an edge's `aUid` is an ancestor node, firegraph parses the Firestore collection path. The path `graph/taskId/workflow` reveals that `taskId` is a document in the `graph` collection.
+
+#### Registry `targetGraph`
+
+The `targetGraph` field on a `RegistryEntry` tells forward traversal which subgraph to query under each source node:
+
+```typescript
+createRegistry([
+  // When traversing forward from a task along 'assignedTo',
+  // look in the 'workflow' subgraph under each task
+  { aType: 'task', axbType: 'assignedTo', bType: 'agent', targetGraph: 'workflow' },
+]);
+```
+
+`targetGraph` must be a single segment (no `/`). It can also be set in entity discovery via `edge.json`:
+
+```json
+{ "from": "task", "to": "agent", "targetGraph": "workflow" }
+```
+
+#### Explicit Hop Override
+
+You can override the registry's `targetGraph` on a per-hop basis:
+
+```typescript
+// Use 'team' subgraph instead of registry's default
+const result = await createTraversal(g, taskId)
+  .follow('assignedTo', { targetGraph: 'team' })
+  .run();
+```
+
+Resolution priority: explicit hop `targetGraph` > registry `targetGraph` > no cross-graph.
+
+#### `findEdgesGlobal` — Collection Group Queries
+
+For cross-cutting reads across all subgraphs, use `findEdgesGlobal`:
+
+```typescript
+// Find all 'assignedTo' edges across all 'workflow' subgraphs in the database
+const allAssignments = await g.findEdgesGlobal(
+  { axbType: 'assignedTo', allowCollectionScan: true },
+  'workflow',  // collection name to query across
+);
+```
+
+This uses Firestore collection group queries and requires collection group indexes. The collection name defaults to the last segment of the client's collection path if omitted.
+
+#### Multi-Hop Limitation
+
+Each hop resolves its reader from the root client. If hop 1 crosses into a subgraph, hop 2 does **not** stay in that subgraph — it reverts to the root. To chain hops within a subgraph, create a separate traversal from the subgraph client:
+
+```typescript
+// This traversal finds agents in the workflow subgraph
+const agents = await createTraversal(g, taskId, registry)
+  .follow('assignedTo')
+  .run();
+
+// To continue traversing within the workflow subgraph,
+// create a new traversal from the subgraph client
+const workflow = g.subgraph(taskId, 'workflow');
+for (const agent of agents.nodes) {
+  const mentees = await createTraversal(workflow, agent.bUid)
+    .follow('mentors')
+    .run();
+}
+```
+
+#### Firestore Path Layout
+
+```
+graph/                              <- root collection
+  {taskId}                          <- task node
+  {taskId}/workflow/                <- workflow subgraph
+    {agentId}                       <- agent node
+    {shard:taskId:assignedTo:agentId} <- cross-graph edge
+```
+
 ### ID Generation
 
 ```typescript
@@ -360,8 +598,10 @@ All errors extend `FiregraphError` with a `code` property:
 | `EdgeNotFoundError` | `EDGE_NOT_FOUND` | Edge lookup fails |
 | `ValidationError` | `VALIDATION_ERROR` | Schema validation fails (registry + Zod) |
 | `RegistryViolationError` | `REGISTRY_VIOLATION` | Triple not registered |
+| `RegistryScopeError` | `REGISTRY_SCOPE` | Type not allowed at this subgraph scope |
 | `DynamicRegistryError` | `DYNAMIC_REGISTRY_ERROR` | Dynamic registry misconfiguration or misuse |
 | `InvalidQueryError` | `INVALID_QUERY` | `findEdges` called with no filters |
+| `QuerySafetyError` | `QUERY_SAFETY` | Query would cause a full collection scan |
 | `TraversalError` | `TRAVERSAL_ERROR` | `run()` called with zero hops |
 
 ```typescript
@@ -403,8 +643,9 @@ import type {
   GraphClientOptions,
 
   // Registry
-  RegistryEntry,
-  GraphRegistry,
+  RegistryEntry,       // includes targetGraph, allowedIn
+  GraphRegistry,       // includes lookupByAxbType
+  EdgeTopology,        // includes targetGraph
 
   // Dynamic Registry
   DynamicGraphClient,
@@ -413,11 +654,15 @@ import type {
   EdgeTypeData,
 
   // Traversal
-  HopDefinition,
+  HopDefinition,       // includes targetGraph
   TraversalOptions,
   HopResult,
   TraversalResult,
   TraversalBuilder,
+
+  // Entity Discovery
+  DiscoveredEntity,
+  DiscoveryResult,
 } from 'firegraph';
 ```
 
@@ -449,6 +694,8 @@ When you call `findEdges`, the query planner decides the strategy:
 
 1. Start with `sourceUids = [startUid]`
 2. For each hop in sequence:
+   - Resolve `targetGraph`: check hop override, then registry, then none
+   - If cross-graph (forward + `targetGraph` + `GraphClient` reader): create a subgraph reader via `reader.subgraph(sourceUid, targetGraph)` for each source
    - Fan out: query edges for each source UID (parallel, bounded by semaphore)
    - Each `findEdges` call counts as 1 read against the budget
    - Apply in-memory `filter` if specified, then apply `limit`

package/bin/firegraph.mjs

@@ -86,6 +86,45 @@ if (subcommand === 'editor') {
     console.error(`Error: ${err.message}`);
     process.exit(1);
   }
+} else if (subcommand === 'indexes') {
+  const args = parseArgs(process.argv.slice(3));
+  const entitiesDir = args.entities ? path.resolve(args.entities) : null;
+  const collection = args.collection || 'graph';
+  const outPath = args.out || null;
+
+  const distIndex = path.join(__dirname, '..', 'dist', 'index.js');
+  const { generateIndexConfig, discoverEntities } = await import(distIndex);
+
+  try {
+    let entities = undefined;
+    if (entitiesDir) {
+      const { result, warnings } = discoverEntities(entitiesDir);
+      for (const w of warnings) {
+        console.warn(`  warning: ${w.message}`);
+      }
+      entities = result;
+      const nodeCount = result.nodes.size;
+      const edgeCount = result.edges.size;
+      if (nodeCount > 0 || edgeCount > 0) {
+        console.error(`Discovered ${nodeCount} node type(s) + ${edgeCount} edge type(s)`);
+      }
+    }
+
+    const config = generateIndexConfig(collection, entities);
+    const output = JSON.stringify(config, null, 2) + '\n';
+
+    if (outPath) {
+      const resolved = path.resolve(outPath);
+      fs.mkdirSync(path.dirname(resolved), { recursive: true });
+      fs.writeFileSync(resolved, output, 'utf-8');
+      console.log(`Generated ${config.indexes.length} index(es) → ${resolved}`);
+    } else {
+      process.stdout.write(output);
+    }
+  } catch (err) {
+    console.error(`Error: ${err.message}`);
+    process.exit(1);
+  }
 } else if (subcommand === '--help' || subcommand === '-h' || !subcommand) {
   console.log('');
   console.log('  Usage: firegraph <command> [options]');
@@ -94,6 +133,7 @@ if (subcommand === 'editor') {
   console.log('    editor         Launch the Firegraph Editor UI');
   console.log('    query          Query the graph via the editor API');
   console.log('    codegen        Generate TypeScript types from entity schemas');
+  console.log('    indexes        Generate recommended Firestore index definitions');
   console.log('');
   console.log('  Editor options:');
   console.log('    --config <path>        Path to firegraph.config.ts (default: auto-discover in cwd)');
@@ -111,6 +151,11 @@ if (subcommand === 'editor') {
   console.log('    --entities <path>      Path to entities directory (default: ./entities)');
   console.log('    --out <path>           Output file path (default: stdout)');
   console.log('');
+  console.log('  Indexes options:');
+  console.log('    --entities <path>      Path to entities directory (adds per-entity data field indexes)');
+  console.log('    --collection <name>    Firestore collection name (default: graph)');
+  console.log('    --out <path>           Output file path (default: stdout)');
+  console.log('');
   console.log('  Config file:');
   console.log('    Create a firegraph.config.ts in your project root to avoid passing');
   console.log('    flags every time. CLI flags override config file values.');
@@ -121,6 +166,8 @@ if (subcommand === 'editor') {
   console.log('    npx firegraph editor --entities ./entities            # per-entity convention');
   console.log('    npx firegraph codegen --entities ./entities           # types to stdout');
   console.log('    npx firegraph codegen --entities ./entities --out src/generated/types.ts');
+  console.log('    npx firegraph indexes                                  # 4 base indexes to stdout');
+  console.log('    npx firegraph indexes --entities ./entities --out firestore.indexes.json');
   console.log('');
 } else {
   console.error(`Unknown command: ${subcommand}`);

package/dist/codegen/index.d.cts

@@ -1,2 +1,2 @@
-export { k as CodegenOptions, I as generateTypes } from '../index-CG3R68Hu.cjs';
+export { l as CodegenOptions, J as generateTypes } from '../index-CQkofEC_.cjs';
 import '@google-cloud/firestore';

package/dist/codegen/index.d.ts

@@ -1,2 +1,2 @@
-export { k as CodegenOptions, I as generateTypes } from '../index-CG3R68Hu.js';
+export { l as CodegenOptions, J as generateTypes } from '../index-CQkofEC_.js';
 import '@google-cloud/firestore';