@brainbox-labs/cli 0.0.0-stage → 0.14.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +264 -2
- package/dist/cli.js +3922 -0
- package/package.json +59 -4
package/README.md
CHANGED
|
@@ -1,3 +1,265 @@
|
|
|
1
|
-
#
|
|
1
|
+
# brainbox CLI
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Introspect your Postgres or Trino database and submit a semantic layer to BrainBox for AI-powered annotation and curation.
|
|
4
|
+
|
|
5
|
+
## Installation
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
npm install -g @brainbox-labs/cli
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Or run without installing:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
npx -y @brainbox-labs/cli@latest --help
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Usage
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
# Set once in your shell profile:
|
|
21
|
+
export BRAINBOX_API_KEY=bb_xxxxxxxxxxxx
|
|
22
|
+
# export BRAINBOX_API_URL=https://ctx.brainbox-ai.app # optional, this is the default
|
|
23
|
+
|
|
24
|
+
brainbox connect \
|
|
25
|
+
--db postgres://user:password@localhost:5432/mydb \
|
|
26
|
+
--org "Acme Corp" \
|
|
27
|
+
--project "Finance Analytics" \
|
|
28
|
+
--queries ./sql/
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### Options
|
|
32
|
+
|
|
33
|
+
| Flag | Required | Default | Description |
|
|
34
|
+
|------|----------|---------|-------------|
|
|
35
|
+
| `--db <url>` | ✅ | — | Postgres connection string |
|
|
36
|
+
| `--org <name>` | ✅ | — | Organization name (created if it doesn't exist) |
|
|
37
|
+
| `--project <name>` | ✅ | — | Project name (created if it doesn't exist) |
|
|
38
|
+
| `--api-url <url>` | — | `$BRAINBOX_API_URL` or `https://ctx.brainbox-ai.app` | BrainBox API base URL |
|
|
39
|
+
| `--queries <dir>` | — | — | Directory of `.sql` files to include |
|
|
40
|
+
| `--query-file <file>` | — | — | Single `.sql` file to include |
|
|
41
|
+
| `--api-key <key>` | — | `$BRAINBOX_API_KEY` | API key for authentication |
|
|
42
|
+
| `--ui-url <url>` | — | `$BRAINBOX_API_URL` or `https://ctx.brainbox-ai.app` | Web UI URL printed in the success message |
|
|
43
|
+
|
|
44
|
+
Both `BRAINBOX_API_KEY` and `BRAINBOX_API_URL` can be set as environment variables so you don't have to pass them on every invocation.
|
|
45
|
+
|
|
46
|
+
## What it does
|
|
47
|
+
|
|
48
|
+
1. **Introspects** your database — tables, columns, primary keys, foreign keys, indexes, and constraints are all extracted locally. Your connection string is never sent to BrainBox. (Trino exposes tables and columns only — it has no enforced keys or indexes.)
|
|
49
|
+
2. **Parses** any `.sql` files you provide — queries are analysed for referenced tables, columns, join patterns, and named parameters.
|
|
50
|
+
3. **Submits** the schema + queries to the BrainBox API as a draft (status: `pending`). No LLM fires yet.
|
|
51
|
+
4. **Triggers generation** via the Context API — LLM annotation runs async; the CLI polls until the context is `ready`.
|
|
52
|
+
5. **Activates** the context, promoting it to a versioned semantic layer.
|
|
53
|
+
6. **Prints** a link to the BrainBox UI where you can review the generated semantic layer.
|
|
54
|
+
|
|
55
|
+
## Commands
|
|
56
|
+
|
|
57
|
+
### `brainbox meta` — stage data sources locally
|
|
58
|
+
|
|
59
|
+
| Command | Description |
|
|
60
|
+
|---------|-------------|
|
|
61
|
+
| `meta ingest db <url>` | Introspect a Postgres (`postgres://`) or Trino (`trino://user@host:port/catalog[/schema]`) database and stage the schema |
|
|
62
|
+
| `meta ingest mcp <url>` | Introspect an MCP server and stage its tools |
|
|
63
|
+
| `meta ingest doc <files...>` | Stage documents — md, txt, html, pdf, docx (pdf/docx auto-converted to markdown) |
|
|
64
|
+
| `meta ingest nosql <file>` | Stage a NoSQL schema from a JSON file (`{ collections: [...] }`) |
|
|
65
|
+
| `meta ingest dbt --artifacts <dir>` | Upload dbt artifacts (manifest.json + optional catalog / semantic_manifest) directly to the API |
|
|
66
|
+
|
|
67
|
+
#### `meta ingest db` flags
|
|
68
|
+
|
|
69
|
+
| Flag | Default | Description |
|
|
70
|
+
|------|---------|-------------|
|
|
71
|
+
| `--sql <dir>` | — | Directory of `.sql` query files |
|
|
72
|
+
| `--sql-file <file>` | — | Single `.sql` file |
|
|
73
|
+
| `--tables <names>` | all | Comma-separated table filter |
|
|
74
|
+
| `--mapping <path>` | — | ETL/column-mapping document describing how this schema was produced. Repeatable; globs allowed. Sent with the schema so the server can derive column lineage |
|
|
75
|
+
| `--dialect <name>` | auto-detected | SQL dialect tag sent to the server (`postgres`, `mysql`, `snowflake`, `bigquery`, `duckdb`, `trino`, `athena`, `presto`) |
|
|
76
|
+
| `--profile-data [mode]` | — | Profile column data. Omit value for cardinality-only; use `full` for enum values + JSON shapes |
|
|
77
|
+
| `--skip-profile-columns <list>` | — | Comma-separated `table.column` pairs to exclude from profiling |
|
|
78
|
+
| `--profile-sample-rows <n>` | 10000 | Max rows sampled per table during profiling |
|
|
79
|
+
|
|
80
|
+
#### Column lineage via `--mapping`
|
|
81
|
+
|
|
82
|
+
BrainBox ingests your **canonical** schema. If an ETL maps your own source
|
|
83
|
+
columns into that schema, pass the mapping document so the server can explain
|
|
84
|
+
that `amount_cents` came from `txn_amt`:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
brainbox meta ingest db postgres://... --mapping etl/orders-mapping.yaml
|
|
88
|
+
brainbox meta ingest db postgres://... --mapping 'etl/*-mapping.yaml' --mapping docs/transforms.md
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
The file is read and sent verbatim — the CLI never parses or validates its
|
|
92
|
+
contents, so YAML, dbt models, SQL transforms, and prose all work. Mappings
|
|
93
|
+
travel inside the same SQL connector draft as the schema (not as a separate
|
|
94
|
+
docs connector), so lineage never depends on upload order. A path that matches
|
|
95
|
+
no file is an error rather than a silent skip, as is a basename collision
|
|
96
|
+
between two different mapping files — mappings are identified by filename, so
|
|
97
|
+
rename one rather than lose its lineage.
|
|
98
|
+
|
|
99
|
+
#### `meta ingest doc` flags
|
|
100
|
+
|
|
101
|
+
| Flag | Default | Description |
|
|
102
|
+
|------|---------|-------------|
|
|
103
|
+
| `--analysis-depth <depth>` | server falls back to `shallow` | How deeply to analyse these documents: `none`, `shallow`, or `deep` |
|
|
104
|
+
|
|
105
|
+
### `brainbox collect` — acquire a locked source into a local markdown bundle
|
|
106
|
+
|
|
107
|
+
Deterministic, local acquisition — **no LLM, no upload**. Output goes to
|
|
108
|
+
`.brainbox/collected/<collector>/<run-id>/` (per-item markdown plus a
|
|
109
|
+
`manifest.json` with source identifiers, fetch timestamps, and the collector
|
|
110
|
+
version). The customer's own agent reads these files off disk while following a
|
|
111
|
+
context recipe.
|
|
112
|
+
|
|
113
|
+
| Command | Description |
|
|
114
|
+
|---------|-------------|
|
|
115
|
+
| `collect web <url>` | Crawl same-origin pages and convert each to markdown. Honours `robots.txt` |
|
|
116
|
+
| `collect openapi <file\|url>` | Normalize an OpenAPI/Swagger doc (JSON or YAML) into one markdown file per operation |
|
|
117
|
+
| `collect proto <file\|dir>` | Normalize `.proto` files into a markdown message / enum / service list |
|
|
118
|
+
| `collect files <paths...>` | Convert PDF/DOCX/HTML/MD files to markdown (same conversion as `meta ingest doc`) |
|
|
119
|
+
|
|
120
|
+
#### `collect web` flags
|
|
121
|
+
|
|
122
|
+
| Flag | Default | Description |
|
|
123
|
+
|------|---------|-------------|
|
|
124
|
+
| `--depth <n>` | `0` | Link-follow depth from the start URL |
|
|
125
|
+
| `--max-pages <n>` | `50` | Hard cap on pages fetched |
|
|
126
|
+
| `--allow <glob>` | — | Extra URL glob to crawl beyond the start origin. Repeatable |
|
|
127
|
+
|
|
128
|
+
### `brainbox recipe` — context recipes
|
|
129
|
+
|
|
130
|
+
A **context recipe** is a procedure plus a machine-checkable output schema,
|
|
131
|
+
published by the BrainBox server and installed into your project; your agent
|
|
132
|
+
executes it locally.
|
|
133
|
+
|
|
134
|
+
| Command | Description |
|
|
135
|
+
|---------|-------------|
|
|
136
|
+
| `recipe list` | List the server's recipe catalog — slug, description, and the collectors each recipe consumes |
|
|
137
|
+
| `recipe install <slug>` | Install a recipe into the resolved project. Checks the recipe's `requires.cli` range first; prints the skill name your agent passes to `get_skill`. Re-running prints "already installed" and exits zero |
|
|
138
|
+
| `recipe check <file>` | Advisory readiness report on a drafted markdown file. Posts the content to the server's `check_context_draft` tool and renders per-requirement pass/fail with fixes. **Exits zero even when checks fail** — non-zero is reserved for network/auth/file errors. Recipe slug comes from the file's `recipe:` frontmatter key, or `--recipe <slug>` |
|
|
139
|
+
|
|
140
|
+
### `brainbox profile` — preview column profiles locally
|
|
141
|
+
|
|
142
|
+
| Command | Description |
|
|
143
|
+
|---------|-------------|
|
|
144
|
+
| `profile preview <url>` | Compute a column profile locally and print what would be uploaded — no API call made |
|
|
145
|
+
|
|
146
|
+
### `brainbox repo` — version management
|
|
147
|
+
|
|
148
|
+
| Command | Description |
|
|
149
|
+
|---------|-------------|
|
|
150
|
+
| `repo init` | Create `.brainbox/` config in the current directory |
|
|
151
|
+
| `repo clone <project>` | Fetch project config from the API into `.brainbox/` |
|
|
152
|
+
| `repo commit [-m <message>]` | Snapshot staged sources as a local commit |
|
|
153
|
+
| `repo push [--default-analysis-depth <depth>]` | Upload the latest commit to the server (data sync only — no LLM). The depth flag (`none`/`shallow`/`deep`) applies to any staged docs that didn't set their own `--analysis-depth` |
|
|
154
|
+
| `repo pull [--out <file>]` | Download the active layer locally |
|
|
155
|
+
| `repo log` | Show local commit history and remote version history |
|
|
156
|
+
| `repo meta create [-m <message>]` | Trigger LLM generation from the pushed draft (PR equivalent) |
|
|
157
|
+
| `repo meta approve [<contextId>]` | Approve the generated draft → publish as new versioned layer |
|
|
158
|
+
|
|
159
|
+
### `brainbox dev` — local MCP server
|
|
160
|
+
|
|
161
|
+
| Command | Description |
|
|
162
|
+
|---------|-------------|
|
|
163
|
+
| `dev start [--port <port>]` | Serve the active layer as a local MCP server (default: port 3100) |
|
|
164
|
+
| `dev stop` | Stop the running dev server |
|
|
165
|
+
|
|
166
|
+
### Typical workflow
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
# 1. Set up a project
|
|
170
|
+
brainbox repo init --org "Acme Corp" --project "Finance Analytics"
|
|
171
|
+
|
|
172
|
+
# 2. Stage your data sources
|
|
173
|
+
brainbox meta ingest db postgres://user:pass@localhost:5432/mydb --sql ./sql/
|
|
174
|
+
# …or a Trino catalog (TLS is inferred when a password is present; ?ssl=false to opt out)
|
|
175
|
+
brainbox meta ingest db trino://user:pass@trino.example.com:443/hive/sales
|
|
176
|
+
brainbox meta ingest doc ./docs/*.md
|
|
177
|
+
|
|
178
|
+
# 3. Commit and push
|
|
179
|
+
brainbox repo commit -m "Q2 schema + runbooks"
|
|
180
|
+
brainbox repo push
|
|
181
|
+
|
|
182
|
+
# 4. Trigger LLM generation and review on web
|
|
183
|
+
brainbox repo meta create -m "Q2 context layer"
|
|
184
|
+
# → opens review link; edit on web if needed
|
|
185
|
+
|
|
186
|
+
# 5. Approve → publish as new version
|
|
187
|
+
brainbox repo meta approve
|
|
188
|
+
|
|
189
|
+
# 6. Use locally
|
|
190
|
+
brainbox repo pull
|
|
191
|
+
brainbox dev start
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
---
|
|
195
|
+
|
|
196
|
+
### Legacy commands (deprecated — use the commands above for new projects)
|
|
197
|
+
|
|
198
|
+
| Command | Description |
|
|
199
|
+
|---------|-------------|
|
|
200
|
+
| `connect` | Introspect a Postgres database, generate a context layer via LLM, and activate it |
|
|
201
|
+
| `push` | Submit a semantic layer from a JSON file (bypasses LLM pipeline) |
|
|
202
|
+
| `pull` | Download the active semantic layer to a local JSON file |
|
|
203
|
+
| `versions` | List all semantic layer versions for a project |
|
|
204
|
+
| `activate` | Set a specific semantic layer version as active |
|
|
205
|
+
| `kb-submit` | Submit documents/text files to the Knowledge Base and generate a context layer |
|
|
206
|
+
| `kb-status` | Check the status and results of a Knowledge Base draft by ID |
|
|
207
|
+
| `kb-layers` | List published Knowledge Base layers for a project |
|
|
208
|
+
| `extract` | Extract structured data from a document using a known template |
|
|
209
|
+
| `extract-templates` | List available extraction templates |
|
|
210
|
+
| `extract-approve` | Approve an extraction and add it to the dataset |
|
|
211
|
+
| `extract-reject` | Reject an extraction |
|
|
212
|
+
|
|
213
|
+
### Knowledge Base workflow
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
# 1. Submit documents — LLM generation + activation happens automatically
|
|
217
|
+
brainbox kb-submit ./docs/*.md --org "Acme" --project "Finance"
|
|
218
|
+
|
|
219
|
+
# 2. List published layers
|
|
220
|
+
brainbox kb-layers --org "Acme" --project "Finance"
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
`--project` and `--tenant` are mutually exclusive ways to target where documents land:
|
|
224
|
+
|
|
225
|
+
- `--project <name>` — any org project, auto-creates it if it doesn't exist yet.
|
|
226
|
+
- `--tenant <name>` — an existing tenant overlay, resolved by exact tenant name. Never
|
|
227
|
+
auto-creates — if the tenant doesn't exist yet, the command fails loudly instead of
|
|
228
|
+
silently ingesting into an orphan project. Use this when ingesting into a customer's
|
|
229
|
+
tenant overlay (internally named `"<tenant name> — Tenant Overlay"`) so you don't have
|
|
230
|
+
to know or type that internal naming convention:
|
|
231
|
+
|
|
232
|
+
```bash
|
|
233
|
+
brainbox kb-submit ./docs/*.md --org "Acme" --tenant "Acme Customer Co"
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
`BRAINBOX_TENANT` is supported alongside `BRAINBOX_PROJECT` as an environment variable.
|
|
237
|
+
|
|
238
|
+
## Development
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
# Install dependencies
|
|
242
|
+
npm install
|
|
243
|
+
|
|
244
|
+
# Run from source (no build needed)
|
|
245
|
+
npm run dev -- connect --db postgres://... --org test --project test --api-url http://localhost:3001
|
|
246
|
+
|
|
247
|
+
# Build
|
|
248
|
+
npm run build
|
|
249
|
+
|
|
250
|
+
# Type-check
|
|
251
|
+
npm run lint
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
### Install globally from local source
|
|
255
|
+
|
|
256
|
+
The package is not yet published to npm. To install the `brainbox` command globally from this repo:
|
|
257
|
+
|
|
258
|
+
```bash
|
|
259
|
+
# From the repo root
|
|
260
|
+
npm install
|
|
261
|
+
npm run build -w packages/cli
|
|
262
|
+
npm install -g ./packages/cli
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
> **Note:** Use `./packages/cli` (with the `./` prefix) — omitting it causes npm to treat the path as a git shorthand and fail.
|