bc-cli 0.2.0__tar.gz → 0.3.0__tar.gz

{bc_cli-0.2.0 → bc_cli-0.3.0}/.gitignore

@@ -33,3 +33,6 @@ PRD-*.md
 TODOS.md
 bcapi_cli_prd.md
 .planning/
+
+# Codex CLI session metadata — local-only, not part of the repo
+.context/

{bc_cli-0.2.0 → bc_cli-0.3.0}/CHANGELOG.md

@@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- **Clean SIGPIPE handling for piped output** — `bcli <cmd> | head`,
+  `| grep -m 1`, and similar pipe-truncating consumers now terminate
+  the CLI silently, matching `cat` and `grep` conventions, instead of
+  emitting a `BrokenPipeError: [Errno 32] Broken pipe` traceback at
+  interpreter shutdown. Implemented as a new `bcli_cli.app:main`
+  console-script entry point that installs `SIGPIPE -> SIG_DFL` on
+  POSIX with a `BrokenPipeError` safety net for Windows.
+- **Hyphenated saved-query param names** — the workflow template
+  resolver now accepts hyphens in identifiers, so references like
+  `${{ params.vendor-no }}` substitute correctly. Previously the regex
+  matched only `[\w.]`, silently leaving the literal `${{ … }}` token
+  in the rendered filter (BC then 400'd or, worse, returned mismatched
+  rows). Affects both `bcli q` saved queries and `bcli batch`
+  workflows.
+
 ## [0.2.0] — 2026-05-06
 
 ### Added

{bc_cli-0.2.0 → bc_cli-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bc-cli
-Version: 0.2.0
+Version: 0.3.0
 Summary: Python SDK and CLI for Microsoft Dynamics 365 Business Central APIs
 Project-URL: Homepage, https://github.com/igor-ctrl/bcli
 Project-URL: Repository, https://github.com/igor-ctrl/bcli
@@ -32,14 +32,27 @@ Requires-Dist: tomlkit>=0.13
 Requires-Dist: typer>=0.12
 Provides-Extra: cli
 Provides-Extra: dev
+Requires-Dist: anthropic>=0.40; extra == 'dev'
 Requires-Dist: dlt[filesystem,parquet,s3]>=1.0; extra == 'dev'
 Requires-Dist: mcp>=1.0; extra == 'dev'
+Requires-Dist: openai>=1.50; extra == 'dev'
+Requires-Dist: pypdf>=4.0; extra == 'dev'
 Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
 Requires-Dist: pytest-httpx>=0.30; extra == 'dev'
 Requires-Dist: pytest>=8.0; extra == 'dev'
 Requires-Dist: ruff>=0.5; extra == 'dev'
 Provides-Extra: etl
 Requires-Dist: dlt[filesystem,parquet,s3]>=1.0; extra == 'etl'
+Provides-Extra: extract
+Requires-Dist: anthropic>=0.40; extra == 'extract'
+Requires-Dist: openai>=1.50; extra == 'extract'
+Requires-Dist: pypdf>=4.0; extra == 'extract'
+Provides-Extra: extract-claude
+Requires-Dist: anthropic>=0.40; extra == 'extract-claude'
+Requires-Dist: pypdf>=4.0; extra == 'extract-claude'
+Provides-Extra: extract-openai
+Requires-Dist: openai>=1.50; extra == 'extract-openai'
+Requires-Dist: pypdf>=4.0; extra == 'extract-openai'
 Provides-Extra: mcp
 Requires-Dist: mcp>=1.0; extra == 'mcp'
 Provides-Extra: polaris

bc_cli-0.3.0/docs/extraction.md

@@ -0,0 +1,236 @@
+# PDF Extraction (`bcli extract`)
+
+Extract structured records from PDFs (scans, forms, tabular reports — vendor
+invoices, packing slips, statements, anything tabular) via an AI vision
+backend, then promote the result to Business Central through the existing
+batch runner.
+
+```
+PDF + YAML schema
+      │
+      ▼
+bcli extract  ──►  <pdf>.batch.yaml      ◄── operator reviews
+                   <pdf>.extracted.json     against the source PDF
+      │
+      ▼
+bcli batch run … --profile sandbox --dry-run
+bcli batch run … --profile sandbox
+      │ (verify in BC sandbox UI)
+      ▼
+bcli batch run … --profile production
+```
+
+The extraction layer never writes to BC. It produces files; humans
+review; the existing `bcli batch run` machinery (with `disable_writes`,
+production confirmation, audit log) handles the actual mutation. This
+review step matters most when the extracted data has high blast radius
+(regulated records, financial postings) — emitting batch.yaml + sidecar
+instead of writing directly gives a deterministic, auditable review seam.
+
+## Install
+
+Pick a backend:
+
+```bash
+# Claude (Anthropic)
+uv pip install -e ".[extract-claude]"
+export ANTHROPIC_API_KEY=sk-ant-…
+
+# OpenAI
+uv pip install -e ".[extract-openai]"
+export OPENAI_API_KEY=sk-…
+
+# Both
+uv pip install -e ".[extract]"
+```
+
+Then enable the backend in `~/.config/bcli/config.toml`. With the
+defaults shown, just setting `backend` is enough — each backend fills
+in its own model + env-var name:
+
+```toml
+[extract]
+backend = "claude"     # or "openai"
+
+# Optional overrides — leave blank to use backend-appropriate defaults:
+#   claude:  model = "claude-sonnet-4-6", api_key_env = "ANTHROPIC_API_KEY"
+#   openai:  model = "gpt-5",             api_key_env = "OPENAI_API_KEY"
+# model = "claude-sonnet-4-6"
+# api_key_env = "ANTHROPIC_API_KEY"
+# schemas_dir = "~/.config/bcli/extract/schemas"
+# max_pdf_bytes = 33554432    # 32 MiB
+# max_pdf_pages = 100
+# max_output_tokens = 8000
+# openai_base_url = ""        # e.g. an Azure OpenAI / proxy endpoint
+# openai_organization = ""    # OpenAI org id (optional)
+```
+
+Switching between backends is a one-line config change — schemas and
+generated batch.yamls are backend-agnostic, so iterating on a
+schema with one provider and running production with another is fine.
+
+## Schemas
+
+A schema is a YAML file that tells the backend *what* to extract and
+*how* to map the result onto a BC endpoint. Drop one into
+`~/.config/bcli/extract/schemas/` and `bcli extract list-schemas` picks
+it up.
+
+Minimal example:
+
+```yaml
+name: "Purchase invoice line items"
+description: "One PDF = one invoice with many line items."
+prompt: |
+  Extract one record per line item in this vendor invoice. Skip the
+  header row, subtotal, tax, and grand-total rows. If a row is
+  illegible, OMIT it — do not guess.
+
+list: true     # one PDF → many records
+
+fields:
+  item_no:
+    type: string
+    description: "Item number / SKU."
+    required: true
+  description:
+    type: string
+    description: "Item description."
+    required: true
+  quantity:
+    type: number
+    description: "Quantity ordered."
+    required: true
+  unit_price:
+    type: number
+    description: "Unit price."
+
+output:
+  endpoint: purchaseLines
+  action: post
+  parent_field: documentNo
+  parent_param: invoice_no
+  field_map:
+    "no": item_no
+    description: description
+    quantity: quantity
+    directUnitCost: unit_price
+  constants:
+    documentType: "Invoice"
+    type: "Item"
+```
+
+`parent_param` + `parent_field` emit a `${{ params.invoice_no }}`
+placeholder in the generated `batch.yaml`. The operator fills it in
+before `batch run` (or passes `--set invoice_no=…`). This is the
+intentional human-in-the-loop seam: extraction can't know which BC
+record the rows belong to, so it asks.
+
+See `examples/extract/purchase_invoice_lines.yaml` for the fully-worked
+schema. Author your own under `~/.config/bcli/extract/schemas/` — one
+YAML per document type.
+
+## Use
+
+```bash
+# Drop the schema in the well-known location, or pass a path.
+bcli extract list-schemas
+
+# Run extraction. Emits two files next to the PDF.
+bcli extract run ./invoice-acme-1234.pdf --schema purchase_invoice_lines
+
+# Output:
+#   invoice-acme-1234.batch.yaml        ← workflow to run
+#   invoice-acme-1234.extracted.json    ← traceability sidecar
+
+# Promote to sandbox (dry-run first, then real).
+bcli batch run invoice-acme-1234.batch.yaml \
+    --set invoice_no=<bc-invoice-number> \
+    --profile sandbox --dry-run
+
+bcli batch run invoice-acme-1234.batch.yaml \
+    --set invoice_no=<bc-invoice-number> \
+    --profile sandbox
+
+# Eyeball in the BC sandbox UI, then production.
+bcli batch run invoice-acme-1234.batch.yaml \
+    --set invoice_no=<bc-invoice-number> \
+    --profile production
+```
+
+## Traceability sidecar
+
+Every `bcli extract` run drops `<pdf>.extracted.json` next to the
+batch.yaml. It contains:
+
+- The schema name + endpoint.
+- The Claude model + token usage.
+- Every record, including the raw model output and the 1-indexed PDF
+  pages the values were read from.
+- Any warnings (e.g. `list: false` schema with multiple records).
+
+Reviewers open this side by side with the PDF to verify each value
+before the batch runs. The sidecar is the deterministic, auditable
+artifact your reviewer signs off on — never run the batch without
+it for high-blast-radius data (regulated records, financial postings,
+anything where a wrong identifier has real-world consequences).
+
+## PDF size limits
+
+Anthropic caps each document block at **32 MB** and **100 pages**.
+`bcli extract` checks both before sending. If your PDF is too big:
+
+```bash
+# Split with qpdf (homebrew: brew install qpdf)
+qpdf --split-pages=50 big_report.pdf split-%d.pdf
+
+# Extract each split, then concatenate batch.yamls (or run them serially).
+for f in split-*.pdf; do
+    bcli extract run "$f" --schema <your_schema_slug>
+done
+```
+
+Chunked-extract orchestration is a follow-up; the primitive is shipped
+first.
+
+## Pluggable backends
+
+`[extract] backend` accepts:
+
+- `"null"`   — no extraction (default). Returns an empty result with a warning.
+- `"claude"` — Anthropic Claude (built-in, `[extract-claude]`).
+- `"openai"` — OpenAI Responses API + Files API (built-in, `[extract-openai]`).
+- `"my_pkg.module:MyExtractor"` — any class implementing
+  `bcli.extract.ExtractorBackend`. The class needs `is_active`,
+  `extract(pdf_path, schema)`, and a `from_config(cls, config)`
+  classmethod. AWS Textract, Firecrawl, OpenDataLoader, a Vertex AI
+  Gemini wrapper, or a self-hosted vision model all fit this shape.
+
+Custom-backend failures fall back to `NullExtractor` with a one-shot
+warning — extraction never crashes the CLI on a config mistake.
+
+### Backend choice tips
+
+- Both built-ins accept the same schema. Switching is a one-line
+  config change; you can iterate a schema cheaply on one provider and
+  promote with the other.
+- Aviation/regulated data: pick the provider with the residency /
+  compliance posture your org accepts. Neither built-in routes through
+  Beautech infrastructure — your API key, your traffic.
+- Cost: at time of writing, both providers price PDF input in the same
+  ballpark for short documents. Long tabular reports tend to favor
+  whichever provider has the cheaper input-token rate.
+
+## Safety / regulated-data note
+
+If the extracted data has real-world consequences (regulated records,
+financial postings, anything where a wrong identifier matters), the
+design enforces four reviews before bytes hit production:
+
+1. **Sidecar review** — `extracted.json` against the source PDF, by a human.
+2. **Sandbox dry-run** — `bcli batch run … --dry-run` against a non-prod profile.
+3. **Sandbox write + UI verification** — `bcli batch run … --profile sandbox` then eyeball.
+4. **Production** — only after the first three pass.
+
+Skipping any of these defeats the design. The CLI doesn't enforce the
+sequence (yet); the schema-author and the operator do.

bc_cli-0.3.0/docs/plans/team-deployment.md

@@ -0,0 +1,309 @@
+# Beautech Team Deployment Plan
+
+Status: draft. **Beautech-internal bootstrap document, not part of the OSS bcli roadmap.** The bcli OSS tool ships independently of this plan; the work below describes how Beautech rolls bundles + diagnostics to its own finance and technical teams on top of the upstream substrate.
+
+Target: finance team (~10 users) + engine technical team (~10 users) on the existing scoped-profile substrate.
+
+## Why this plan exists
+
+bcli is moving from solo developer to a real team rollout. The current substrate already supports it (sandboxed profiles, curated registries, scoped saved queries, device-code auth, read-only-by-permission-set), but three workflow gaps will dominate first-month support load:
+
+1. **No team-wide registry/query distribution.** Whoever last updated the JSON wins. New hires re-discover everything.
+2. **No diagnostic surface.** Users with broken setups can't self-rescue. "Wrong profile / stale bundle / not authenticated / wrong company" will be every other support ticket.
+3. **No discoverability for the saved-query library.** Finance ops will not learn YAML. They will ask "is there a query for overdue intercompany invoices?" via email.
+
+This plan ships three boring high-leverage things to address (1)–(3), explicitly defers Redis and response caching until telemetry justifies them, and locks in trigger conditions so the deferral doesn't become indefinite.
+
+## Phase 0 — pre-flight (this sprint)
+
+- Decide bundle storage backend: pick whichever of `S3`, `Azure Blob`, or `GitHub Releases` the org already authenticates to cleanly. Decision lives in `docs/plans/team-deployment.md` once made.
+- Identify two bundle owners: one finance, one technical. They are the publish path.
+- Land a minimal telemetry sink config so phase 4 has data when it's time. The pluggable `[telemetry]` substrate already exists at `src/bcli/telemetry/`; pick `console` for dev, set up Azure Monitor or a custom HTTP sink for prod. Capture `bcli.command`, `bcli.query`, `bcli.error` at minimum. **Do not** capture filter text or UPN unless privacy review approves.
+
+## Phase 1 — `bcli doctor` (ships first)
+
+Self-rescue command for non-technical users. This alone will eliminate most week-one tickets.
+
+### Surface
+
+```
+bcli doctor [--profile <name>] [--json]
+```
+
+Default output: human-readable, color-coded (green/yellow/red per check), with a one-line verdict at the bottom (`OK`, `WARN`, or `FAIL`). `--json` for scripting. Non-zero exit on `FAIL`.
+
+### Checks
+
+| Check | Source | Fail condition |
+|---|---|---|
+| Active profile | `CLIState` resolved profile | Missing or unknown profile name |
+| Bundle version | `manifest.json` `version` field (phase 2) | Missing manifest, or `last_refresh > 30d` |
+| Signature verified | bundle signature check (phase 2) | Signature missing or invalid |
+| Last refresh time | bundle metadata | > 7d warn, > 30d fail |
+| Registry endpoint count | `EndpointRegistry.list_all()` | 0 endpoints in scoped profile |
+| Saved query count | `queries/<profile>.yaml` | File missing in scoped profile |
+| Field-list coverage | count of endpoints with `field_names` populated | Warn under 50% for scoped profiles |
+| Auth mode | profile config | Unknown mode |
+| Auth status | non-blocking probe of cached token | Expired and no refresh path |
+| Company | `--company` resolution | No default and no override available |
+| Environment | profile config | Missing |
+| Tenant ID | profile config | Missing |
+| BC connectivity | one-shot `GET companies` with 5s timeout | Non-2xx |
+| Local overlay present | overlay file exists (phase 2) | Informational only |
+
+### Output sketch
+
+```
+bcli doctor — profile: finance
+
+  ✓ Active profile        finance
+  ✓ Bundle version        2026.05.07-1 (signed by ops-bcli-bot)
+  ✓ Last refresh          2 days ago
+  ✓ Registry              42 endpoints, 38 with field lists (90%)
+  ✓ Saved queries         17 queries
+  ✓ Auth                  device_code, token valid for 47 min
+  ✓ Tenant                contoso.onmicrosoft.com
+  ✓ Environment           production
+  ✓ Company               BTALI (default)
+  ✓ BC connectivity       reachable, 312 ms
+  ⚠ Field coverage        2 endpoints below 80% (run `bcli endpoint fields ...`)
+
+  Verdict: OK
+```
+
+### Files to add/modify
+
+- New: `src/bcli_cli/commands/doctor_cmd.py`
+- New: `src/bcli/diagnostics/_checks.py` (testable check primitives, returns `CheckResult` with status + message)
+- Modify: `src/bcli_cli/app.py` to register the command group
+- New: `tests/test_diagnostics/test_checks.py`
+
+### Done when
+
+- `bcli doctor` runs in under 3 seconds for a healthy install.
+- Each check is independently unit-tested with parametrized fail cases.
+- Engine-tech and finance both ran it on their own laptop and the output made sense without explanation.
+
+## Phase 2 — signed bundle distribution
+
+### Bundle layout
+
+```
+finance-2026.05.07-1.tar.gz
+  manifest.json
+  registry.json          # mirrors current ~/.config/bcli/registries/<profile>.json
+  queries.yaml           # mirrors current ~/.config/bcli/queries/<profile>.yaml
+  field_lists.json       # pre-warmed field discovery (avoids first-touch tax)
+  README.md              # human notes for this bundle version
+```
+
+`manifest.json`:
+
+```json
+{
+  "schema_version": 1,
+  "profile": "finance",
+  "version": "2026.05.07-1",
+  "published_at": "2026-05-07T14:32:00Z",
+  "publisher": "ops-bcli-bot",
+  "checksum_sha256": "…",
+  "signature": "…",
+  "min_bcli_version": "0.3.0",
+  "previous_version": "2026.04.30-2",
+  "release_notes": "Added overdue-ic and posted-invoice-by-id queries"
+}
+```
+
+### Storage + transport
+
+- Backend: signed HTTPS, single source of truth per profile. Bundles published to versioned object keys, e.g. `https://bundles.example.com/bcli/finance/2026.05.07-1.tar.gz` with a `latest.json` pointer for resolution. The org-specific URL lives in `~/.config/bcli/config.toml` as `[bundle.finance] url = "..."` so different teams can self-host.
+- Signing: detached signature (`minisign` or `cosign`, decision pending) with the public key shipped in the user's profile config. Refresh fails closed if signature does not verify.
+- **Not** `git pull`. Maintainers can author bundles in a private GitHub repo and ship via Release assets, but the user-facing transport is a signed tarball over HTTPS.
+
+### `bcli config refresh` UX
+
+```
+bcli config refresh                       # refresh active profile
+bcli config refresh --profile technical # explicit profile
+bcli config refresh --dry-run             # show diff vs local, no writes
+bcli config refresh --rollback            # restore previous version
+bcli config refresh --check               # exit code only, no output (cron-friendly)
+```
+
+Non-interactive, atomic. Output:
+
+```
+Refreshing finance from https://bundles.example.com/bcli/finance/latest.json
+  Current:  2026.04.30-2
+  Latest:   2026.05.07-1 (published 2 days ago by ops-bcli-bot)
+  Verifying signature… ok
+  Diff:
+    + 2 endpoints (postedSalesInvoices, salesInvoiceLines)
+    + 3 saved queries (overdue-ic, posted-by-id, customer-aging)
+    ~ 1 query updated (open-pos: added customer parameter)
+  Applied. Previous version retained at ~/.config/bcli/registries/finance.2026.04.30-2.json
+```
+
+### Overlay semantics
+
+- Team bundle is **authoritative** for scoped profiles. `bcli config refresh` overwrites `registry.json` and `queries.yaml` atomically (write-temp + rename). Previous version retained for one rollback.
+- Local overlay file `~/.config/bcli/overlays/<profile>.yaml` exists *only if* the profile config has `allow_local_overrides = true`. Off by default for sandboxed domain profiles.
+- Effective view: team bundle merged with overlay, **team wins on name conflicts**. No interactive merge prompts ever. Non-technical users never see a conflict.
+- `bcli endpoint fields` discovery for sandboxed profiles writes to overlay if enabled, otherwise informs the user to send the discovered fields to their bundle owner.
+
+### Files to add/modify
+
+- New: `src/bcli/bundle/__init__.py` (manifest schema, signature verification, atomic apply)
+- New: `src/bcli/bundle/_fetch.py`, `_verify.py`, `_apply.py`, `_rollback.py`
+- New: `src/bcli_cli/commands/refresh_cmd.py` (registered as `bcli config refresh`)
+- Modify: `src/bcli/config/_loader.py` to compose registry from team bundle + optional overlay
+- Modify: `src/bcli/registry/_registry.py` to load from the new layered path
+- New: `examples/bundles/sample-bundle.tar.gz` + a `make-bundle` script for admins
+- New: `docs/team-bundles.md` covering the publish workflow
+
+### Done when
+
+- An admin can produce a signed bundle with one command and publish it.
+- A finance user can run `bcli config refresh` cold and have a working setup in under 30 seconds.
+- `--rollback` restores the previous version verifiably.
+- Tampered bundle is rejected with a clear error, not silently applied.
+
+## Phase 3 — query metadata extension
+
+Saved queries get richer descriptive metadata so substring + tag search beats the discoverability problem without embeddings.
+
+### YAML schema additions
+
+```yaml
+queries:
+  overdue-ic:
+    description: Overdue intercompany invoices for a vendor
+    aliases: [overdue-intercompany, ic-overdue, ar-overdue-ic]
+    tags: [period-close, ap, intercompany]
+    owner: finance-ops
+    freshness: live  # one of: live, daily, reference
+    examples:
+      - bcli q overdue-ic vendor=ACME-IC
+      - bcli q overdue-ic vendor=ACME-IC days=30
+    related: [open-invoices, vendor-aging]
+    params:
+      vendor: { required: true, hint: "BC Vendor No." }
+      days:   { default: 30, hint: "Days overdue" }
+    # Query body lives at the top level (matches the runtime — there is
+    # no `odata:` wrapper). The metadata block above is purely for
+    # discoverability; nothing in it changes how the query executes.
+    endpoint: vendorLedgerEntries
+    filter: "vendorNumber eq '${{ params.vendor }}' and dueDate lt now sub '${{ params.days }}d' and remainingAmount gt 0"
+    orderby: dueDate
+```
+
+### Search surface
+
+```
+bcli q list                                 # all queries, table
+bcli q list --tag period-close              # filter by tag
+bcli q list --owner finance-ops             # filter by owner
+bcli q search "overdue invoices"            # substring + alias + description match
+bcli q info overdue-ic                      # full metadata view
+```
+
+`bcli q search` ranks by: exact name > alias hit > tag hit > description substring > example substring. No embeddings. Honest "no match, did you mean X" output when the score floor isn't met.
+
+### Files to add/modify
+
+- Modify: `src/bcli/workflow/` query schema (extend Pydantic model)
+- Modify: `src/bcli_cli/commands/query_cmd.py` to add `list`, `search`, `info` subcommands
+- New: `tests/test_workflow/test_query_metadata.py`
+- Update: `docs/saved-queries.md` with the schema additions
+- Migration: existing queries without the new fields keep working (all new fields optional)
+
+### Done when
+
+- Existing queries still run unchanged.
+- `bcli q search` finds an existing query when the user types a plausible NL phrase.
+- Finance ops can browse the catalog by tag without reading YAML.
+
+## Phase 4 — response caching (deferred, telemetry-gated)
+
+**Do not ship until all three triggers are met:**
+
+1. Telemetry shows P95 latency for posted-invoice / open-PO endpoints exceeding 2s under finance close-week load.
+2. Telemetry shows ≥ 5% of GETs returning 429 / 503 during close week.
+3. The two bundle owners agree the workflow pain is real, not theoretical.
+
+If shipped:
+
+- Backend: `hishel`-backed disk cache around `httpx`, **not** Redis. Single-process. Lives at `~/.config/bcli/cache/`.
+- Cache key composition: `tenant_id + environment + company + profile + resolved_url + sorted(query_params) + select_hash`. Never less.
+- TTL ceilings (max — actual values configurable per endpoint, can be lower):
+  - Vendor balances: no cache by default; 5-15s if forced, output labeled `(cached 8s ago)`
+  - Open POs / open invoices: 15-60s
+  - Inventory / utilization / preservation status: 10-60s
+  - Posted invoice **list** queries: 60-300s only when filtered to a closed period
+  - Posted invoice / journal entry **by exact record ID**: 1-24h (immutable post-posting)
+- Never cache `--all`. Never cache write-adjacent commands. Cache hits are visible in output and structured logs.
+- Opt-in per profile via `[cache] enabled = true`.
+
+## Phase 5 — Redis-for-AI (deferred, condition-gated)
+
+**Do not ship until at least one of:**
+
+1. A centralized `bcli-mcp` service is running for multiple agents/users (cross-process state stops being free).
+2. Saved-query library exceeds 200 entries with measurable search misses in telemetry.
+3. Field-list "did you mean" produces wrong suggestions ≥ 5% of attempts measurably.
+
+If shipped, the integration points are:
+
+- Vector "did you mean" over discovered field names (replaces substring fuzzy in `src/bcli/client/_async.py:469`).
+- Vector search over saved queries by NL intent. Caches **query plans**, never query results.
+- Shared field-discovery cache for the centralized MCP path.
+
+Backend: pluggable `cache_backend` with `redis` extra, mirroring the existing `[telemetry]` pattern. NullCache default. Redis is optional infrastructure.
+
+## Out of scope
+
+- Semantic caching of OData result data. Stale balances / inventory / posted-document state silently returned would destroy trust in bcli as a BC truth source. If we want a low-risk reference subset later, it lands as an explicit feature with `cached as of` labels in output, not a silent layer.
+- Token sharing across users. Each user authenticates as themselves. The BC permission set is the security boundary, not the bcli flag.
+- Replacing the existing per-profile registry JSON layout. Bundles are a publish-and-distribute layer on top, not a replacement.
+
+## Risks and open questions
+
+- **Beautech rollout gate: publisher signing.** The current
+  `Sha256Verifier` only proves internal consistency: each file matches
+  its declared hash, and the manifest's roll-up matches the contents
+  map. It does NOT authenticate the publisher. A compromised CDN can
+  mint a malicious `registry.json`, recompute the hashes, and pass
+  verification. Before Beautech rolls bundles to finance / technical,
+  either ship a real cryptographic signer (`minisign` / `cosign` /
+  ed25519 + pinned key) at the `bcli.bundle.Verifier` seam, or restrict
+  bundle distribution to private blob storage with org-level auth and
+  treat HTTPS+auth as the trust boundary. Document the choice
+  internally; do not enable `bcli config refresh` for finance/engine-
+  tech without one of the two. Note: this is a Beautech deployment
+  gate, not an OSS bcli release gate — the upstream tool can ship the
+  bundle infra without dictating how operators use it.
+- **Signing key custody.** Who owns the bundle signing key, and how is it rotated when an owner leaves? Decide before phase 2 ships.
+- **Bundle URL discovery.** First-time install needs to know where to refresh from. Likely `bcli config init --scoped --bundle-url <url>` extends the existing wizard. Verify this fits the wizard's current shape.
+- **Field discovery in scoped profiles.** Today `bcli endpoint fields` writes back to the local registry. With overlay-off-by-default, sandboxed users can't improve their own setup. The plan: those discoveries get logged to a "candidate fields" file the user can email to their bundle owner. Better mechanism welcome.
+- **Bundle drift between teams.** If finance and technical import the same standard endpoint into both bundles and they diverge, which wins? Today: each profile is isolated. Keep it that way.
+- **Telemetry privacy.** Phase 0 telemetry must not capture filter text or UPN by default. Confirm with the legal/privacy reviewer.
+
+## Validation gates per phase
+
+| Phase | Gate |
+|---|---|
+| 1 | `bcli doctor` runs on technical and finance laptops cold; output makes sense to a non-developer |
+| 2 | Bundle round-trip works: admin publishes, user runs `refresh`, signature verifies, rollback works |
+| 3 | Existing queries unchanged; `q search "overdue invoices"` finds `overdue-ic` |
+| 4 | Telemetry triggers met before any code is written |
+| 5 | At least one of three triggers met before any code is written |
+
+## Sequencing summary
+
+1. **Now:** phase 0 (pick backend + telemetry sink) and phase 1 (`bcli doctor`). Two weeks.
+2. **Next:** phase 2 (bundle distribution + `config refresh`). Three weeks.
+3. **After phase 2 ships and bakes for two weeks:** phase 3 (query metadata + search). One week.
+4. **On telemetry:** phase 4. Maybe never.
+5. **On condition trigger:** phase 5. Maybe never.
+
+The honest read: phases 1–3 are most of the user-facing value. Phases 4–5 are the interesting features but only earn their cost under conditions we haven't observed yet.

bc_cli-0.3.0/examples/extract/purchase_invoice_lines.yaml

@@ -0,0 +1,61 @@
+name: "Purchase invoice line items"
+description: |
+  Illustrative schema for extracting line items from a vendor PDF
+  invoice. One PDF typically contains many line items as a single
+  table; this schema captures the common columns.
+
+  Adjust the field map to match your tenant's purchaseLines field
+  names. Treat this file as a starting point, not production-ready —
+  every supplier formats invoices differently, and the prompt below
+  may need sharpening for your specific vendor templates.
+
+prompt: |
+  This PDF is a vendor purchase invoice. Extract one record per
+  line item in the line-items table. Skip the header row, subtotal
+  rows, tax rows, and the grand total — those are not line items.
+
+  Expected columns (names vary by vendor; match by meaning):
+    - Item / SKU / Part number   → item_no
+    - Description / Item name    → description
+    - Quantity / Qty             → quantity
+    - Unit price / Rate          → unit_price
+    - Line total / Amount        → line_total
+
+  If a row's item_no or quantity is illegible, OMIT the row rather
+  than guessing. Record the 1-indexed PDF page in source_pages.
+
+list: true
+
+fields:
+  item_no:
+    type: string
+    description: "Item number / SKU / part number from the leftmost identifier column."
+    required: true
+  description:
+    type: string
+    description: "Item description / name."
+    required: true
+  quantity:
+    type: number
+    description: "Quantity ordered. Numeric only."
+    required: true
+  unit_price:
+    type: number
+    description: "Unit price / unit cost. Numeric only, no currency symbol."
+  line_total:
+    type: number
+    description: "Line total (quantity × unit_price). Numeric only."
+
+output:
+  endpoint: purchaseLines
+  action: post
+  parent_field: documentNo
+  parent_param: invoice_no
+  field_map:
+    "no": item_no
+    description: description
+    quantity: quantity
+    directUnitCost: unit_price
+  constants:
+    documentType: "Invoice"
+    type: "Item"