@deepagents/text2sql 0.39.0 → 2.1.0

package/README.md

@@ -26,24 +26,15 @@ npm install mysql2                   # MySQL / MariaDB
 npm install @google-cloud/bigquery   # BigQuery
 ```
 
-Requires Node.js LTS (20+).
+Requires Node.js LTS
 
 ## Quick Start
 
 ```typescript
 import { groq } from '@ai-sdk/groq';
-import { InMemoryFs } from 'just-bash';
 import pg from 'pg';
 
-import {
-  ContextEngine,
-  InMemoryContextStore,
-  createBashTool,
-  createRoutingSandbox,
-  createVirtualSandbox,
-  user,
-} from '@deepagents/context';
-import { Text2Sql, sqlSandboxExtension } from '@deepagents/text2sql';
+import { Text2Sql } from '@deepagents/text2sql';
 import {
   Postgres,
   columnValues,
@@ -59,12 +50,6 @@ const pool = new pg.Pool({
   connectionString: process.env.DATABASE_URL,
 });
 
-const store = new InMemoryContextStore();
-const engine = new ContextEngine({
-  store,
-  chatId: 'chat-123',
-  userId: 'user-456',
-});
 const adapter = new Postgres({
   execute: async (sql) => {
     const result = await pool.query(sql);
@@ -80,19 +65,11 @@ const adapter = new Postgres({
     columnValues(),
   ],
 });
-const sandbox = await createBashTool({
-  sandbox: await createRoutingSandbox({
-    backend: await createVirtualSandbox({ fs: new InMemoryFs() }),
-    hostExtensions: [sqlSandboxExtension({ main: adapter })],
-  }),
-});
 
 const text2sql = new Text2Sql({
   version: 'v1',
   model: groq('openai/gpt-oss-20b'),
-  sandbox,
   adapters: { main: adapter },
-  context: engine,
 });
 
 // Generate SQL
@@ -105,32 +82,131 @@ console.log(sql);
 
 The adapter-map key (`main` here) is the adapter name. Reuse that same key in
 `text2sql.toSql(..., 'main')` and in any `sql validate <db> "..."` /
-`sql run <db> "..."` sandbox calls.
+`sql run <db> "..."` calls from a sandbox where the package CLI is installed.
 
 Adapter names must match `/^[A-Za-z_][A-Za-z0-9_]*$/`. If you build adapter
 maps dynamically, use `isValidAdapterName(name)` to check one key or
 `validateAdapterNames(names)` to fail fast before constructing `Text2Sql` or
-`sqlSandboxExtension`.
+the sandbox-side adapter module used by the `sql` CLI.
 
 ## AI Model Providers
 
 Text2SQL works with any model provider supported by the [Vercel AI SDK](https://sdk.vercel.ai/docs), including OpenAI, Anthropic, Google, Groq, and more.
 
-## Advanced: Composing Sandbox Extensions
+## Building a Conversational Agent
+
+`Text2Sql` owns SQL generation and schema indexing, but you own the chat loop.
+Build the agent by composing a `ContextEngine`, a sandbox, and `agent` +
+`chat` from `@deepagents/context`:
+
+```typescript
+import {
+  ContextEngine,
+  type ContextFragment,
+  InMemoryContextStore,
+  agent,
+  chat,
+  createContainerTool,
+  errorRecoveryGuardrail,
+  npm,
+  user,
+} from '@deepagents/context';
+import { createSqlCommandHooks, instructions } from '@deepagents/text2sql';
+
+const store = new InMemoryContextStore();
+const context = new ContextEngine({
+  store,
+  chatId: 'chat-123',
+  userId: 'user-456',
+});
+
+const sandbox = await createContainerTool({
+  installers: [npm('@deepagents/text2sql', { ensureRuntime: true })],
+  volumes: [
+    {
+      type: 'bind',
+      hostPath: process.cwd(),
+      containerPath: '/workspace',
+      readOnly: true,
+    },
+  ],
+  env: {
+    TEXT2SQL_ADAPTERS: '/workspace/text2sql-adapters.ts',
+  },
+  ...createSqlCommandHooks({ adapters: { main: adapter } }),
+});
+
+const indexResult = await sandbox.sandbox.executeCommand('sql index');
+if (indexResult.exitCode !== 0) throw new Error(indexResult.stderr);
+const manifest = JSON.parse(indexResult.stdout) as { fragmentsPath: string };
+const fragments = JSON.parse(
+  await sandbox.sandbox.readFile(manifest.fragmentsPath),
+) as ContextFragment[];
+context.set(...instructions(), ...fragments);
+
+const ai = agent({
+  name: 'sql-assistant',
+  sandbox,
+  model: groq('openai/gpt-oss-20b'),
+  context,
+  guardrails: [errorRecoveryGuardrail],
+  maxGuardrailRetries: 3,
+});
+
+await context.continue(user('Show me the top 10 customers by revenue'));
+const stream = await chat(ai);
+
+for await (const chunk of stream) {
+  // handle streaming response
+}
+```
+
+The `/workspace/text2sql-adapters.ts` module must exist in the sandbox and
+default-export your adapter map. Mount your project into `/workspace` (as above)
+or upload/write that module before you call `sql index` or `chat()`.
+
+`instructions()` returns the SQL-flavored system fragments (policies, workflows,
+clarifications, style guides) — spread them into `context.set()` alongside the
+schema fragments returned by `sql index`. Add or replace them with your
+own domain fragments as needed.
+
+## Advanced: SQL CLI in Sandboxes
 
-`Text2Sql` takes a caller-owned `sandbox`. `sqlSandboxExtension(adaptersMap)`
-returns a `SandboxExtension` that bundles the `sql` subcommand, transform
-plugins, and arg-repair hook. In the simplest case, that map is
-`{ main: adapter }`. Compose it (alongside any of your own extensions) via
-`createBashTool` + `createRoutingSandbox` + `createVirtualSandbox`.
-See [Caller-Owned Sandbox](https://januarylabs.github.io/deepagents/docs/text2sql/sqlv3)
-for composition patterns.
+`sql validate <db> "..."` and `sql run <db> "..."` are real commands from the
+`@deepagents/text2sql` package. `sql index` writes schema fragments plus
+progress events for chat setup, indexing all configured adapters by default
+(same as `--all`) unless adapter names are provided.
+
+Install the package inside the sandbox and set `TEXT2SQL_ADAPTERS` to a module
+whose default export is `Record<string, Adapter>`. Missing `sql` means the
+sandbox was not prepared correctly.
+
+`sql index` output details:
+
+- `stdout`: JSON manifest with `fragmentsPath`, `eventsPath`, adapters, and
+  fragment count.
+- `--verbose pretty` or `--verbose json`: mirrors progress events to `stderr`
+  while keeping `stdout` as the manifest.
+- `--out-dir <path>`: writes artifacts under that path (default:
+  `$TEXT2SQL_OUT_DIR` or `./sql`).
+
+Set `TEXT2SQL_INDEX_VERSION` to manage cache invalidation across runs. Cache
+keys are `index-<version>-<adapter>`, so bump the version when schema changes.
+
+Spread `createSqlCommandHooks({ adapters })` into `createBashTool()` or
+`createContainerTool()` for model-driven bash calls. The before hook preserves
+the old virtual-command tolerance for common LLM quote mistakes, rewrites SQL
+identifier backticks so bash does not run them as command substitutions, and
+blocks raw database access so read-only and scope checks stay behind
+`sql validate` / `sql run`. The after hook restores hidden `formattedSql`
+metadata from the host adapter map without putting that concern into the real
+CLI.
 
 ## Fragments
 
-Inject domain knowledge by setting fragments on the `ContextEngine` you pass to
-`Text2Sql`. Those fragments affect `chat()` sessions. If you need direct SQL
-generation with extra fragments, use the lower-level `toSql({ fragments })`
+Inject domain knowledge by setting fragments on the `ContextEngine` you build
+for the agent. Those fragments affect every `chat()` turn. If you need direct
+SQL generation with extra fragments, use the lower-level `toSql({ fragments })`
 helper exported from `@deepagents/text2sql`.
 
 ```typescript
@@ -144,13 +220,13 @@ import {
 } from '@deepagents/context';
 
 const store = new InMemoryContextStore();
-const engine = new ContextEngine({
+const context = new ContextEngine({
   store,
   chatId: 'chat-123',
   userId: 'user-456',
 });
 
-engine.set(
+context.set(
   term('MRR', 'monthly recurring revenue'),
   hint('Always exclude test accounts with email ending in @test.com'),
   guardrail({
@@ -188,62 +264,69 @@ Control what schema metadata the AI receives:
 
 ## Conversations
 
-`chat()` persists history through the `ContextEngine` you pass to `Text2Sql`.
-Reuse the same store, `chatId`, and `userId` to continue the same thread.
-Before each turn, append only the new incoming message to the engine; do not
-replay earlier turns that are already stored in the context store:
+`chat()` (from `@deepagents/context`) persists history through the
+`ContextEngine` you build for the agent. Reuse the same store, `chatId`, and
+`userId` to continue the same thread. Before each turn, append only the new
+incoming message to the engine; do not replay earlier turns that are already
+stored in the context store:
 
 ```typescript
-await engine.continue(user('Show me orders from last month'));
-const stream = await text2sql.chat();
+await context.continue(user('Show me orders from last month'));
+const stream = await chat(ai);
 
 for await (const chunk of stream) {
   // handle streaming response
 }
 
 // Continue the same conversation
-await engine.continue(user('Now filter to only completed ones'));
-const followUp = await text2sql.chat();
+await context.continue(user('Now filter to only completed ones'));
+const followUp = await chat(ai);
 for await (const chunk of followUp) {
   // handle streaming response
 }
 ```
 
-While `chat()` builds or reads cached schema context, it emits progress chunks
-before assistant text starts:
+## Streaming Index Progress
+
+`AdapterIndexer#index({ onProgress })` emits progress events while it warms or reads
+the schema cache for host-side indexing. The sandbox CLI writes the same event
+shape to the `eventsPath` file returned by `sql index`:
 
 ```typescript
+import { createUIMessageStream } from 'ai';
+
 import {
+  AdapterIndexer,
   TEXT2SQL_INDEX_PROGRESS_CHUNK,
   type Text2SqlIndexProgressEvent,
 } from '@deepagents/text2sql';
 
-await engine.continue(user('Show me top 10 customers'));
-const stream = await text2sql.chat();
-const startedAt = new Map<string, number>();
+const indexer = new AdapterIndexer({ adapters, version: 'v1' });
 
-for await (const chunk of stream) {
-  if (chunk.type === TEXT2SQL_INDEX_PROGRESS_CHUNK) {
-    const event = chunk.data as Text2SqlIndexProgressEvent;
-    const scope =
-      [event.adapter, event.phase].filter(Boolean).join(':') || 'index';
+await context.continue(user('Show me top 10 customers'));
 
-    if (event.type.endsWith(':start') || event.type === 'index:start') {
-      startedAt.set(scope, event.timestampMs ?? Date.now());
+const stream = createUIMessageStream({
+  execute: async ({ writer }) => {
+    const head = await context.headMessage();
+    if (!head || head.name !== 'assistant') {
+      throw new Error(
+        'expected head to be an assistant message — call context.continue() before chat()',
+      );
     }
-
-    if (event.type.endsWith(':end') || event.type.endsWith(':error')) {
-      const durationMs =
-        (event.timestampMs ?? Date.now()) -
-        (startedAt.get(scope) ?? event.timestampMs ?? Date.now());
-      console.log(event.message, durationMs);
-    }
-
-    continue;
-  }
-
-  // handle assistant stream chunks
-}
+    writer.write({ type: 'start', messageId: head.id });
+
+    const fragments = await indexer.index({
+      onProgress: (event) =>
+        writer.write({
+          type: TEXT2SQL_INDEX_PROGRESS_CHUNK,
+          data: event,
+        }),
+    });
+
+    context.set(...instructions(), ...fragments);
+    writer.merge(await chat(ai));
+  },
+});
 ```
 
 Each progress event includes a generic Unix epoch `timestampMs` so clients can
@@ -281,7 +364,7 @@ Full documentation available at [januarylabs.github.io/deepagents](https://janua
 
 - [Getting Started](https://januarylabs.github.io/deepagents/docs/text2sql/getting-started)
 - [Generate SQL](https://januarylabs.github.io/deepagents/docs/text2sql/to-sql)
-- [Caller-Owned Sandbox](https://januarylabs.github.io/deepagents/docs/text2sql/sqlv3)
+- [Agent + SQLite Store Recipe](https://januarylabs.github.io/deepagents/docs/text2sql/recipes/agent-sqlite-store)
 - [Teach the System](https://januarylabs.github.io/deepagents/docs/text2sql/teach-the-system)
 - [Build Conversations](https://januarylabs.github.io/deepagents/docs/text2sql/build-conversations)
 - [Grounding](https://januarylabs.github.io/deepagents/docs/text2sql/grounding)