@deepagents/text2sql 0.38.1 → 1.0.0

package/README.md

@@ -32,17 +32,9 @@ Requires Node.js LTS (20+).
 
 ```typescript
 import { groq } from '@ai-sdk/groq';
-import { InMemoryFs } from 'just-bash';
 import pg from 'pg';
 
-import {
-  ContextEngine,
-  InMemoryContextStore,
-  createBashTool,
-  createRoutingSandbox,
-  createVirtualSandbox,
-} from '@deepagents/context';
-import { Text2Sql, sqlSandboxExtension } from '@deepagents/text2sql';
+import { Text2Sql } from '@deepagents/text2sql';
 import {
   Postgres,
   columnValues,
@@ -58,7 +50,6 @@ const pool = new pg.Pool({
   connectionString: process.env.DATABASE_URL,
 });
 
-const store = new InMemoryContextStore();
 const adapter = new Postgres({
   execute: async (sql) => {
     const result = await pool.query(sql);
@@ -74,27 +65,11 @@ const adapter = new Postgres({
     columnValues(),
   ],
 });
-const sandbox = await createBashTool({
-  sandbox: await createRoutingSandbox({
-    backend: await createVirtualSandbox({ fs: new InMemoryFs() }),
-    hostExtensions: [sqlSandboxExtension({ main: adapter })],
-  }),
-});
 
 const text2sql = new Text2Sql({
   version: 'v1',
   model: groq('openai/gpt-oss-20b'),
-  sandbox,
   adapters: { main: adapter },
-  context: (...fragments) => {
-    const engine = new ContextEngine({
-      store,
-      chatId: 'chat-123',
-      userId: 'user-456',
-    });
-    engine.set(...fragments);
-    return engine;
-  },
 });
 
 // Generate SQL
@@ -107,33 +82,107 @@ console.log(sql);
 
 The adapter-map key (`main` here) is the adapter name. Reuse that same key in
 `text2sql.toSql(..., 'main')` and in any `sql validate <db> "..."` /
-`sql run <db> "..."` sandbox calls.
+`sql run <db> "..."` calls from a sandbox where the package CLI is installed.
 
 Adapter names must match `/^[A-Za-z_][A-Za-z0-9_]*$/`. If you build adapter
 maps dynamically, use `isValidAdapterName(name)` to check one key or
 `validateAdapterNames(names)` to fail fast before constructing `Text2Sql` or
-`sqlSandboxExtension`.
+the sandbox-side adapter module used by the `sql` CLI.
 
 ## AI Model Providers
 
 Text2SQL works with any model provider supported by the [Vercel AI SDK](https://sdk.vercel.ai/docs), including OpenAI, Anthropic, Google, Groq, and more.
 
-## Advanced: Composing Sandbox Extensions
+## Building a Conversational Agent
+
+`Text2Sql` ships only the SQL-aware primitives: `toSql`, `toPairs`,
+`AdapterIndexer`, and `instructions()`. You build the chat agent yourself by
+composing a `ContextEngine`, a sandbox, and `agent` + `chat` from
+`@deepagents/context`:
+
+```typescript
+import {
+  ContextEngine,
+  type ContextFragment,
+  InMemoryContextStore,
+  agent,
+  chat,
+  createContainerTool,
+  errorRecoveryGuardrail,
+  npm,
+  user,
+} from '@deepagents/context';
+import { createSqlCommandHooks, instructions } from '@deepagents/text2sql';
+
+const store = new InMemoryContextStore();
+const context = new ContextEngine({
+  store,
+  chatId: 'chat-123',
+  userId: 'user-456',
+});
+
+const sandbox = await createContainerTool({
+  installers: [npm('@deepagents/text2sql', { ensureRuntime: true })],
+  env: {
+    TEXT2SQL_ADAPTERS: '/workspace/text2sql-adapters.ts',
+  },
+  ...createSqlCommandHooks({ adapters: { main: adapter } }),
+});
+
+const indexResult = await sandbox.sandbox.executeCommand('sql index');
+if (indexResult.exitCode !== 0) throw new Error(indexResult.stderr);
+const manifest = JSON.parse(indexResult.stdout) as { fragmentsPath: string };
+const fragments = JSON.parse(
+  await sandbox.sandbox.readFile(manifest.fragmentsPath),
+) as ContextFragment[];
+context.set(...instructions(), ...fragments);
+
+const ai = agent({
+  name: 'sql-assistant',
+  sandbox,
+  model: groq('openai/gpt-oss-20b'),
+  context,
+  guardrails: [errorRecoveryGuardrail],
+  maxGuardrailRetries: 3,
+});
+
+await context.continue(user('Show me the top 10 customers by revenue'));
+const stream = await chat(ai);
 
-`Text2Sql` takes a caller-owned `sandbox`. `sqlSandboxExtension(adaptersMap)`
-returns a `SandboxExtension` that bundles the `sql` subcommand, transform
-plugins, and arg-repair hook. In the simplest case, that map is
-`{ main: adapter }`. Compose it (alongside any of your own extensions) via
-`createBashTool` + `createRoutingSandbox` + `createVirtualSandbox`.
-See [Caller-Owned Sandbox](https://januarylabs.github.io/deepagents/docs/text2sql/sqlv3)
-for composition patterns.
+for await (const chunk of stream) {
+  // handle streaming response
+}
+```
+
+`instructions()` returns the SQL-flavored system fragments (policies, workflows,
+clarifications, style guides) — spread them into `context.set()` alongside the
+schema fragments returned by `sql index`. Add or replace them with your
+own domain fragments as needed.
+
+## Advanced: SQL CLI in Sandboxes
+
+`sql validate <db> "..."` and `sql run <db> "..."` are real commands from the
+`@deepagents/text2sql` package. `sql index [adapter ...]` writes schema
+fragments plus progress events for chat setup. Install the package inside the
+sandbox and set `TEXT2SQL_ADAPTERS` to a module whose default export is
+`Record<string, Adapter>`. Missing `sql` means the sandbox was not prepared
+correctly.
+
+Spread `createSqlCommandHooks({ adapters })` into `createBashTool()` or
+`createContainerTool()` for model-driven bash calls. The before hook preserves
+the old virtual-command tolerance for common LLM quote mistakes, rewrites SQL
+identifier backticks so bash does not run them as command substitutions, and
+blocks raw database access so read-only and scope checks stay behind
+`sql validate` / `sql run`. The after hook restores hidden `formattedSql`
+metadata from the host adapter map without putting that concern into the real
+CLI.
 
 ## Fragments
 
-Inject domain knowledge using fragments from `@deepagents/context` inside the
-`context` factory you pass to `Text2Sql`. Those fragments affect `chat()`
-sessions. If you need direct SQL generation with extra fragments, use the
-lower-level `toSql({ fragments })` helper exported from `@deepagents/text2sql`.
+Inject domain knowledge by setting fragments on the `ContextEngine` you build
+for the agent. Those fragments affect every `chat()` turn. If you need direct
+SQL generation with extra fragments, use the lower-level `toSql({ fragments })`
+helper exported from `@deepagents/text2sql`.
 
 ```typescript
 import {
@@ -146,31 +195,25 @@ import {
 } from '@deepagents/context';
 
 const store = new InMemoryContextStore();
+const context = new ContextEngine({
+  store,
+  chatId: 'chat-123',
+  userId: 'user-456',
+});
 
-const context = (...fragments) => {
-  const engine = new ContextEngine({
-    store,
-    chatId: 'chat-123',
-    userId: 'user-456',
-  });
-
-  engine.set(
-    term('MRR', 'monthly recurring revenue'),
-    hint('Always exclude test accounts with email ending in @test.com'),
-    guardrail({
-      rule: 'Never expose individual salaries',
-      reason: 'Confidential HR data',
-      action: 'Aggregate by department instead',
-    }),
-    example({
-      question: 'show me churned customers',
-      answer: `SELECT * FROM customers WHERE status = 'churned' ORDER BY churned_at DESC`,
-    }),
-    ...fragments,
-  );
-
-  return engine;
-};
+context.set(
+  term('MRR', 'monthly recurring revenue'),
+  hint('Always exclude test accounts with email ending in @test.com'),
+  guardrail({
+    rule: 'Never expose individual salaries',
+    reason: 'Confidential HR data',
+    action: 'Aggregate by department instead',
+  }),
+  example({
+    question: 'show me churned customers',
+    answer: `SELECT * FROM customers WHERE status = 'churned' ORDER BY churned_at DESC`,
+  }),
+);
 ```
 
 **Domain fragments** (11 types): `term`, `hint`, `guardrail`, `example`, `explain`, `clarification`, `workflow`, `quirk`, `styleGuide`, `analogy`, `glossary`.
@@ -196,23 +239,75 @@ Control what schema metadata the AI receives:
 
 ## Conversations
 
-`chat()` persists history through the `ContextEngine` returned by your `context` factory. Reuse the same store, `chatId`, and `userId` to continue the same thread. Pass only the new incoming message(s) for the current turn; do not replay earlier turns that are already stored in the context store:
+`chat()` (from `@deepagents/context`) persists history through the
+`ContextEngine` you build for the agent. Reuse the same store, `chatId`, and
+`userId` to continue the same thread. Before each turn, append only the new
+incoming message to the engine; do not replay earlier turns that are already
+stored in the context store:
 
 ```typescript
-const stream = await text2sql.chat([
-  { role: 'user', content: 'Show me orders from last month' },
-]);
+await context.continue(user('Show me orders from last month'));
+const stream = await chat(ai);
 
 for await (const chunk of stream) {
   // handle streaming response
 }
 
 // Continue the same conversation
-const followUp = await text2sql.chat([
-  { role: 'user', content: 'Now filter to only completed ones' },
-]);
+await context.continue(user('Now filter to only completed ones'));
+const followUp = await chat(ai);
+for await (const chunk of followUp) {
+  // handle streaming response
+}
 ```
 
+## Streaming Index Progress
+
+`AdapterIndexer#index({ onProgress })` emits progress events while it warms or reads
+the schema cache for host-side indexing. The sandbox CLI writes the same event
+shape to the `eventsPath` file returned by `sql index`:
+
+```typescript
+import { createUIMessageStream } from 'ai';
+
+import {
+  AdapterIndexer,
+  TEXT2SQL_INDEX_PROGRESS_CHUNK,
+  type Text2SqlIndexProgressEvent,
+} from '@deepagents/text2sql';
+
+const indexer = new AdapterIndexer({ adapters, version: 'v1' });
+
+await context.continue(user('Show me top 10 customers'));
+
+const stream = createUIMessageStream({
+  execute: async ({ writer }) => {
+    const head = await context.headMessage();
+    if (!head || head.name !== 'assistant') {
+      throw new Error(
+        'expected head to be an assistant message — call context.continue() before chat()',
+      );
+    }
+    writer.write({ type: 'start', messageId: head.id });
+
+    const fragments = await indexer.index({
+      onProgress: (event) =>
+        writer.write({
+          type: TEXT2SQL_INDEX_PROGRESS_CHUNK,
+          data: event,
+        }),
+    });
+
+    context.set(...instructions(), ...fragments);
+    writer.merge(await chat(ai));
+  },
+});
+```
+
+Each progress event includes a generic Unix epoch `timestampMs` so clients can
+calculate per-index, per-adapter, or per-phase durations without Text2SQL
+imposing one duration policy.
+
 `toSql()` is stateless. It reads the current schema fragments, but it does not
 write messages, titles, or usage into your context store.
 
@@ -244,7 +339,6 @@ Full documentation available at [januarylabs.github.io/deepagents](https://janua
 
 - [Getting Started](https://januarylabs.github.io/deepagents/docs/text2sql/getting-started)
 - [Generate SQL](https://januarylabs.github.io/deepagents/docs/text2sql/to-sql)
-- [Caller-Owned Sandbox](https://januarylabs.github.io/deepagents/docs/text2sql/sqlv3)
 - [Teach the System](https://januarylabs.github.io/deepagents/docs/text2sql/teach-the-system)
 - [Build Conversations](https://januarylabs.github.io/deepagents/docs/text2sql/build-conversations)
 - [Grounding](https://januarylabs.github.io/deepagents/docs/text2sql/grounding)