@brainbox-labs/cli 0.0.0-stage → 0.14.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +264 -2
  2. package/dist/cli.js +3922 -0
  3. package/package.json +59 -4
package/README.md CHANGED
@@ -1,3 +1,265 @@
1
- # Temporary Holding Version
1
+ # brainbox CLI
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Introspect your Postgres or Trino database and submit a semantic layer to BrainBox for AI-powered annotation and curation.
4
+
5
+ ## Installation
6
+
7
+ ```bash
8
+ npm install -g @brainbox-labs/cli
9
+ ```
10
+
11
+ Or run without installing:
12
+
13
+ ```bash
14
+ npx -y @brainbox-labs/cli@latest --help
15
+ ```
16
+
17
+ ## Usage
18
+
19
+ ```bash
20
+ # Set once in your shell profile:
21
+ export BRAINBOX_API_KEY=bb_xxxxxxxxxxxx
22
+ # export BRAINBOX_API_URL=https://ctx.brainbox-ai.app # optional, this is the default
23
+
24
+ brainbox connect \
25
+ --db postgres://user:password@localhost:5432/mydb \
26
+ --org "Acme Corp" \
27
+ --project "Finance Analytics" \
28
+ --queries ./sql/
29
+ ```
30
+
31
+ ### Options
32
+
33
+ | Flag | Required | Default | Description |
34
+ |------|----------|---------|-------------|
35
+ | `--db <url>` | ✅ | — | Postgres connection string |
36
+ | `--org <name>` | ✅ | — | Organization name (created if it doesn't exist) |
37
+ | `--project <name>` | ✅ | — | Project name (created if it doesn't exist) |
38
+ | `--api-url <url>` | — | `$BRAINBOX_API_URL` or `https://ctx.brainbox-ai.app` | BrainBox API base URL |
39
+ | `--queries <dir>` | — | — | Directory of `.sql` files to include |
40
+ | `--query-file <file>` | — | — | Single `.sql` file to include |
41
+ | `--api-key <key>` | — | `$BRAINBOX_API_KEY` | API key for authentication |
42
+ | `--ui-url <url>` | — | `$BRAINBOX_API_URL` or `https://ctx.brainbox-ai.app` | Web UI URL printed in the success message |
43
+
44
+ Both `BRAINBOX_API_KEY` and `BRAINBOX_API_URL` can be set as environment variables so you don't have to pass them on every invocation.
45
+
46
+ ## What it does
47
+
48
+ 1. **Introspects** your database — tables, columns, primary keys, foreign keys, indexes, and constraints are all extracted locally. Your connection string is never sent to BrainBox. (Trino exposes tables and columns only — it has no enforced keys or indexes.)
49
+ 2. **Parses** any `.sql` files you provide — queries are analysed for referenced tables, columns, join patterns, and named parameters.
50
+ 3. **Submits** the schema + queries to the BrainBox API as a draft (status: `pending`). No LLM fires yet.
51
+ 4. **Triggers generation** via the Context API — LLM annotation runs async; the CLI polls until the context is `ready`.
52
+ 5. **Activates** the context, promoting it to a versioned semantic layer.
53
+ 6. **Prints** a link to the BrainBox UI where you can review the generated semantic layer.
54
+
55
+ ## Commands
56
+
57
+ ### `brainbox meta` — stage data sources locally
58
+
59
+ | Command | Description |
60
+ |---------|-------------|
61
+ | `meta ingest db <url>` | Introspect a Postgres (`postgres://`) or Trino (`trino://user@host:port/catalog[/schema]`) database and stage the schema |
62
+ | `meta ingest mcp <url>` | Introspect an MCP server and stage its tools |
63
+ | `meta ingest doc <files...>` | Stage documents — md, txt, html, pdf, docx (pdf/docx auto-converted to markdown) |
64
+ | `meta ingest nosql <file>` | Stage a NoSQL schema from a JSON file (`{ collections: [...] }`) |
65
+ | `meta ingest dbt --artifacts <dir>` | Upload dbt artifacts (manifest.json + optional catalog / semantic_manifest) directly to the API |
66
+
67
+ #### `meta ingest db` flags
68
+
69
+ | Flag | Default | Description |
70
+ |------|---------|-------------|
71
+ | `--sql <dir>` | — | Directory of `.sql` query files |
72
+ | `--sql-file <file>` | — | Single `.sql` file |
73
+ | `--tables <names>` | all | Comma-separated table filter |
74
+ | `--mapping <path>` | — | ETL/column-mapping document describing how this schema was produced. Repeatable; globs allowed. Sent with the schema so the server can derive column lineage |
75
+ | `--dialect <name>` | auto-detected | SQL dialect tag sent to the server (`postgres`, `mysql`, `snowflake`, `bigquery`, `duckdb`, `trino`, `athena`, `presto`) |
76
+ | `--profile-data [mode]` | — | Profile column data. Omit value for cardinality-only; use `full` for enum values + JSON shapes |
77
+ | `--skip-profile-columns <list>` | — | Comma-separated `table.column` pairs to exclude from profiling |
78
+ | `--profile-sample-rows <n>` | 10000 | Max rows sampled per table during profiling |
79
+
80
+ #### Column lineage via `--mapping`
81
+
82
+ BrainBox ingests your **canonical** schema. If an ETL maps your own source
83
+ columns into that schema, pass the mapping document so the server can explain
84
+ that `amount_cents` came from `txn_amt`:
85
+
86
+ ```bash
87
+ brainbox meta ingest db postgres://... --mapping etl/orders-mapping.yaml
88
+ brainbox meta ingest db postgres://... --mapping 'etl/*-mapping.yaml' --mapping docs/transforms.md
89
+ ```
90
+
91
+ The file is read and sent verbatim — the CLI never parses or validates its
92
+ contents, so YAML, dbt models, SQL transforms, and prose all work. Mappings
93
+ travel inside the same SQL connector draft as the schema (not as a separate
94
+ docs connector), so lineage never depends on upload order. A path that matches
95
+ no file is an error rather than a silent skip, as is a basename collision
96
+ between two different mapping files — mappings are identified by filename, so
97
+ rename one rather than lose its lineage.
98
+
99
+ #### `meta ingest doc` flags
100
+
101
+ | Flag | Default | Description |
102
+ |------|---------|-------------|
103
+ | `--analysis-depth <depth>` | server falls back to `shallow` | How deeply to analyse these documents: `none`, `shallow`, or `deep` |
104
+
105
+ ### `brainbox collect` — acquire a locked source into a local markdown bundle
106
+
107
+ Deterministic, local acquisition — **no LLM, no upload**. Output goes to
108
+ `.brainbox/collected/<collector>/<run-id>/` (per-item markdown plus a
109
+ `manifest.json` with source identifiers, fetch timestamps, and the collector
110
+ version). The customer's own agent reads these files off disk while following a
111
+ context recipe.
112
+
113
+ | Command | Description |
114
+ |---------|-------------|
115
+ | `collect web <url>` | Crawl same-origin pages and convert each to markdown. Honours `robots.txt` |
116
+ | `collect openapi <file\|url>` | Normalize an OpenAPI/Swagger doc (JSON or YAML) into one markdown file per operation |
117
+ | `collect proto <file\|dir>` | Normalize `.proto` files into a markdown message / enum / service list |
118
+ | `collect files <paths...>` | Convert PDF/DOCX/HTML/MD files to markdown (same conversion as `meta ingest doc`) |
119
+
120
+ #### `collect web` flags
121
+
122
+ | Flag | Default | Description |
123
+ |------|---------|-------------|
124
+ | `--depth <n>` | `0` | Link-follow depth from the start URL |
125
+ | `--max-pages <n>` | `50` | Hard cap on pages fetched |
126
+ | `--allow <glob>` | — | Extra URL glob to crawl beyond the start origin. Repeatable |
127
+
128
+ ### `brainbox recipe` — context recipes
129
+
130
+ A **context recipe** is a procedure plus a machine-checkable output schema,
131
+ published by the BrainBox server and installed into your project; your agent
132
+ executes it locally.
133
+
134
+ | Command | Description |
135
+ |---------|-------------|
136
+ | `recipe list` | List the server's recipe catalog — slug, description, and the collectors each recipe consumes |
137
+ | `recipe install <slug>` | Install a recipe into the resolved project. Checks the recipe's `requires.cli` range first; prints the skill name your agent passes to `get_skill`. Re-running prints "already installed" and exits zero |
138
+ | `recipe check <file>` | Advisory readiness report on a drafted markdown file. Posts the content to the server's `check_context_draft` tool and renders per-requirement pass/fail with fixes. **Exits zero even when checks fail** — non-zero is reserved for network/auth/file errors. Recipe slug comes from the file's `recipe:` frontmatter key, or `--recipe <slug>` |
139
+
140
+ ### `brainbox profile` — preview column profiles locally
141
+
142
+ | Command | Description |
143
+ |---------|-------------|
144
+ | `profile preview <url>` | Compute a column profile locally and print what would be uploaded — no API call made |
145
+
146
+ ### `brainbox repo` — version management
147
+
148
+ | Command | Description |
149
+ |---------|-------------|
150
+ | `repo init` | Create `.brainbox/` config in the current directory |
151
+ | `repo clone <project>` | Fetch project config from the API into `.brainbox/` |
152
+ | `repo commit [-m <message>]` | Snapshot staged sources as a local commit |
153
+ | `repo push [--default-analysis-depth <depth>]` | Upload the latest commit to the server (data sync only — no LLM). The depth flag (`none`/`shallow`/`deep`) applies to any staged docs that didn't set their own `--analysis-depth` |
154
+ | `repo pull [--out <file>]` | Download the active layer locally |
155
+ | `repo log` | Show local commit history and remote version history |
156
+ | `repo meta create [-m <message>]` | Trigger LLM generation from the pushed draft (PR equivalent) |
157
+ | `repo meta approve [<contextId>]` | Approve the generated draft → publish as new versioned layer |
158
+
159
+ ### `brainbox dev` — local MCP server
160
+
161
+ | Command | Description |
162
+ |---------|-------------|
163
+ | `dev start [--port <port>]` | Serve the active layer as a local MCP server (default: port 3100) |
164
+ | `dev stop` | Stop the running dev server |
165
+
166
+ ### Typical workflow
167
+
168
+ ```bash
169
+ # 1. Set up a project
170
+ brainbox repo init --org "Acme Corp" --project "Finance Analytics"
171
+
172
+ # 2. Stage your data sources
173
+ brainbox meta ingest db postgres://user:pass@localhost:5432/mydb --sql ./sql/
174
+ # …or a Trino catalog (TLS is inferred when a password is present; ?ssl=false to opt out)
175
+ brainbox meta ingest db trino://user:pass@trino.example.com:443/hive/sales
176
+ brainbox meta ingest doc ./docs/*.md
177
+
178
+ # 3. Commit and push
179
+ brainbox repo commit -m "Q2 schema + runbooks"
180
+ brainbox repo push
181
+
182
+ # 4. Trigger LLM generation and review on web
183
+ brainbox repo meta create -m "Q2 context layer"
184
+ # → opens review link; edit on web if needed
185
+
186
+ # 5. Approve → publish as new version
187
+ brainbox repo meta approve
188
+
189
+ # 6. Use locally
190
+ brainbox repo pull
191
+ brainbox dev start
192
+ ```
193
+
194
+ ---
195
+
196
+ ### Legacy commands (deprecated — use the commands above for new projects)
197
+
198
+ | Command | Description |
199
+ |---------|-------------|
200
+ | `connect` | Introspect a Postgres database, generate a context layer via LLM, and activate it |
201
+ | `push` | Submit a semantic layer from a JSON file (bypasses LLM pipeline) |
202
+ | `pull` | Download the active semantic layer to a local JSON file |
203
+ | `versions` | List all semantic layer versions for a project |
204
+ | `activate` | Set a specific semantic layer version as active |
205
+ | `kb-submit` | Submit documents/text files to the Knowledge Base and generate a context layer |
206
+ | `kb-status` | Check the status and results of a Knowledge Base draft by ID |
207
+ | `kb-layers` | List published Knowledge Base layers for a project |
208
+ | `extract` | Extract structured data from a document using a known template |
209
+ | `extract-templates` | List available extraction templates |
210
+ | `extract-approve` | Approve an extraction and add it to the dataset |
211
+ | `extract-reject` | Reject an extraction |
212
+
213
+ ### Knowledge Base workflow
214
+
215
+ ```bash
216
+ # 1. Submit documents — LLM generation + activation happens automatically
217
+ brainbox kb-submit ./docs/*.md --org "Acme" --project "Finance"
218
+
219
+ # 2. List published layers
220
+ brainbox kb-layers --org "Acme" --project "Finance"
221
+ ```
222
+
223
+ `--project` and `--tenant` are mutually exclusive ways to target where documents land:
224
+
225
+ - `--project <name>` — any org project, auto-creates it if it doesn't exist yet.
226
+ - `--tenant <name>` — an existing tenant overlay, resolved by exact tenant name. Never
227
+ auto-creates — if the tenant doesn't exist yet, the command fails loudly instead of
228
+ silently ingesting into an orphan project. Use this when ingesting into a customer's
229
+ tenant overlay (internally named `"<tenant name> — Tenant Overlay"`) so you don't have
230
+ to know or type that internal naming convention:
231
+
232
+ ```bash
233
+ brainbox kb-submit ./docs/*.md --org "Acme" --tenant "Acme Customer Co"
234
+ ```
235
+
236
+ `BRAINBOX_TENANT` is supported alongside `BRAINBOX_PROJECT` as an environment variable.
237
+
238
+ ## Development
239
+
240
+ ```bash
241
+ # Install dependencies
242
+ npm install
243
+
244
+ # Run from source (no build needed)
245
+ npm run dev -- connect --db postgres://... --org test --project test --api-url http://localhost:3001
246
+
247
+ # Build
248
+ npm run build
249
+
250
+ # Type-check
251
+ npm run lint
252
+ ```
253
+
254
+ ### Install globally from local source
255
+
256
+ The package is not yet published to npm. To install the `brainbox` command globally from this repo:
257
+
258
+ ```bash
259
+ # From the repo root
260
+ npm install
261
+ npm run build -w packages/cli
262
+ npm install -g ./packages/cli
263
+ ```
264
+
265
+ > **Note:** Use `./packages/cli` (with the `./` prefix) — omitting it causes npm to treat the path as a git shorthand and fail.