@jaypie/mcp 0.8.162 → 0.8.163

package/dist/assets.js

@@ -55,11 +55,11 @@ function getMcpAssetPaths() {
  */
 function getMcpBuildInfo() {
     return {
-        commit: "09240f3e"
+        commit: "d5f7052e"
             ,
-        version: "0.8.162"
+        version: "0.8.163"
             ,
-        versionString: "@jaypie/mcp@0.8.162#09240f3e"
+        versionString: "@jaypie/mcp@0.8.163#d5f7052e"
             ,
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jaypie/mcp",
-  "version": "0.8.162",
+  "version": "0.8.163",
   "description": "Jaypie MCP",
   "repository": {
     "type": "git",

package/release-notes/jaypie/1.2.103.md

@@ -0,0 +1,11 @@
+---
+version: 1.2.103
+date: 2026-09-19
+summary: Pulls @jaypie/llm 1.3.44
+---
+
+## Changes
+
+- **`@jaypie/llm` 1.3.44**: `Llm.ocr` extracts documents to per-page markdown
+  over Mistral OCR or the new `llamacloud` provider (LlamaParse), behind one
+  request and response shape with fallback across engines.

package/release-notes/llm/1.3.44.md

@@ -0,0 +1,120 @@
+---
+version: 1.3.44
+date: 2026-09-19
+summary: "Llm.ocr extracts documents to per-page markdown over Mistral OCR or LlamaParse behind one shape"
+---
+
+## Changes
+
+### New Features
+
+- `Llm.ocr(document, options?)` is a fifth entry point: a document in,
+  per-page markdown out. Static and instance forms mirror `question`,
+  including `model` array sugar, `fallback`, `retry`, and `signal`. The
+  document (an `https://` URL, a `data:` URI, an S3 key, a local path, or the
+  `{ file }` / `{ image }` objects `operate()` accepts) is resolved once before
+  the fallback chain runs.
+- New provider `llamacloud` serves LlamaParse over the LlamaCloud Parse API
+  v2 with `LLAMA_CLOUD_API_KEY` and match words `llamacloud`, `llamaindex`,
+  and `llamaparse`. A parse is a job: submitted by `source_url` or multipart
+  upload, polled from one second backing off to five, then fetched. It
+  implements `ocr` only; `send` throws `NotImplementedError`. The client is
+  `fetch` against four routes; no SDK dependency is taken.
+- The LlamaParse **tier is the model id**: `MODEL.LLAMAPARSE.FAST`,
+  `COST_EFFECTIVE`, `AGENTIC` (the provider default), and `AGENTIC_PLUS`.
+  A chain such as `[MODEL.MISTRAL.OCR, MODEL.LLAMAPARSE.COST_EFFECTIVE]`
+  therefore reads as a plain model list and routes through
+  `determineModelProvider`.
+- The API version is pinned per tier in `PROVIDER.LLAMACLOUD.VERSION`; each
+  tier accepts its own dated versions and rejects the others (verified live
+  2026-09-19). `providerOptions.version` overrides it per call, and the
+  response `model` is `<tier id>@<version>`.
+- Common shape: `markdown`, 1-indexed `pages[]` with `header`, `footer`,
+  `images`, `success`/`error`, and the vendor page at `raw`; `images[]`;
+  `usage` with `pages`, LlamaParse `credits`, and `cost`; `responses[]`; and
+  the `provider`/`model`/`fallbackAttempts`/`fallbackUsed` fields `operate`
+  carries. Options: `pages` (`[1, 3]` or `"1,3,5-10"`), `tables`
+  (`"markdown"` or `"html"`), `images` (download extracted images as `data:`
+  URIs), `timeout` (asynchronous job ceiling, default ten minutes), and
+  `providerOptions` spread last onto the vendor request.
+- `PAGE_COST` prices document-extraction engines in USD per 1,000 pages,
+  keyed by literal id: `mistral-ocr-4-1` at $4 and the four LlamaParse tiers
+  at $1.25, $3.75, $12.50, and $56.25 (credits at
+  `PROVIDER.LLAMACLOUD.CREDIT_COST`, $1.25 per 1,000). A LlamaParse job's
+  recorded credits win over the per-page estimate.
+- `DEFAULT.OCR` names the engine `Llm.ocr` uses when neither a provider nor a
+  model is given: Mistral OCR, one synchronous call and the cheapest markdown
+  tier on offer.
+- **OCR emulation on chat models**, as `question` emulates: a provider with
+  `operate` but no `ocr` transcribes through `src/ocr/OcrEmulator.ts`, so a
+  chain such as `[MODEL.MISTRAL.OCR, MODEL.HAIKU]` falls from a native engine
+  to any model that accepts files. Each page rides alone in one structured
+  `operate()` call (a PDF is trimmed to one page per request, so the model
+  never counts or numbers pages), under a fixed `{ confidence, markdown,
+notes }` schema and a verbatim-transcription prompt with `[UNCLEAR: …]` and
+  `[ELEMENT: …]` markers. A page scored under 0.4 is transcribed once more
+  and the higher score wins. `concurrency` (default 5) bounds pages in
+  flight; the first failure aborts the rest and rethrows so the next engine
+  starts clean, and unreadable content throws `InternalError` to the same
+  end. The response carries `emulated: true`, `pages[].confidence`,
+  `usage.tokens`, and `usage.cost` priced from `COST` through the new
+  `tokenCost` helper. `user` forwards to the calls. A provider with neither
+  (`typesafe`) fails its attempt and the chain moves on.
+- `LlmOcrResponse.emulated` is present on every response (`false` from a
+  native engine), matching `LlmQuestionResponse`.
+
+### Changed
+
+- `MistralProvider.ocr()` takes the common `(document, options)` signature
+  and returns the common shape. The provider-specific `{ markdown, model,
+pages, raw }` return shipped in 1.3.43 and is gone: `raw` is
+  `responses[0]`, and the 0-indexed `pages[].index` is the 1-indexed
+  `pages[].page` with the vendor page at `pages[].raw`. `MistralOcrPage` and
+  `MistralOcrResponse` are no longer exported. `OcrRequest` widens to the full
+  Mistral `OCRRequest` field set.
+- Client failures on Mistral OCR classify through the chat adapter into the
+  typed `LlmError` family, and both engines share one retry loop
+  (`src/ocr/runOcrAttempts.ts`), so `retry` and `signal` mean the same thing
+  on each. A LlamaParse job that reports `FAILED` throws
+  `LlmUnrecoverableError` with the API's message, `CANCELLED` throws
+  `LlmAbortError`, and a job that outlives `timeout` throws
+  `LlmTransientError` so a chain can try the next engine.
+- `loadFileData` and `getExtensionForMimeType` are exported from the upload
+  helpers for reuse; `LlmProvider` gains an optional `ocr`.
+- `extractPdfPages` throws `BadRequestError` on an out-of-range page where it
+  threw a vanilla `Error`.
+
+### Internal
+
+- `MODEL.LLAMAPARSE.*` join `MODEL.MISTRAL.OCR` in `MATRIX_EXCLUDE`: a
+  document-extraction engine implements neither `operate()` nor `send()`, so
+  every capability cell would fail by construction. They are exercised by
+  `tsx test/ocr.ts` (`npm run test:llm:ocr`) and the provider hot specs.
+- Both CI workflows pass `LLAMA_CLOUD_API_KEY` from the
+  `CICD_LLAMA_CLOUD_API_KEY` org secret; the LlamaCloud hot spec skips until
+  the secret exists.
+
+## Testing
+
+- Live, 2026-09-19 (`npm run test:llm:ocr`): `page.pdf` and `page.png`
+  through Mistral OCR and all four LlamaParse tiers, every run reading the
+  fixture text, plus a chain falling from an unknown tier to Mistral. 11 of 11
+  passed. Mistral answered in about one second at $0.004 per page; LlamaParse
+  completed in 3 to 5 polls (6 to 15 seconds) at 1, 3, 10, and 45 credits per
+  page by tier.
+- Live, 2026-09-19, with emulation: the same fixtures through Haiku and
+  Luna, plus a chain falling from an unknown tier into Haiku. 15 of 15
+  passed. Haiku transcribed a page in 1.5 to 2.5 seconds for $0.0015 to
+  $0.0026 (1,200 to 2,200 tokens) at 0.95 confidence; Luna in 1 to 3 seconds
+  for $0.0002 to $0.0003 at 0.98 to 0.99. Both marked the fixture's rule as
+  `---` or `[ELEMENT: decorative horizontal line]`; Luna read the PDF's
+  heading last, where every other engine read it first.
+- New unit coverage for page-selection parsing, document resolution (URL,
+  data URI, S3, disk, inline data, PDF trimming), the shared retry loop, the
+  LlamaCloud client's routes and status-to-error mapping, the provider's
+  submit/poll/fetch cycle (including fast-tier text, failed pages, image
+  download, per-tier version, timeout, and abort), the reworked Mistral OCR
+  mapping, `Llm.ocr` routing and fallback (native to emulated and back),
+  the emulator (page splitting, selection, image and URL input, low-confidence
+  re-transcription, concurrency cap, first-failure abort, caller abort, and
+  fetch classification), `tokenCost`, and provider routing for every new id.

package/release-notes/mcp/0.8.163.md

@@ -0,0 +1,17 @@
+---
+version: 0.8.163
+date: 2026-09-19
+summary: Document Llm.ocr and the LlamaCloud provider in the llm skill
+---
+
+## Changes
+
+### Documentation
+
+- `skills/llm.md` gains an "OCR" section: the document forms `Llm.ocr`
+  accepts, the page and usage shape, `pages`/`tables`/`images`/`timeout`/
+  `providerOptions`, per-tier LlamaParse versions, and fallback across
+  engines.
+- The providers table carries a `llamacloud` row, the model constants name
+  `MODEL.LLAMAPARSE.*` and `PAGE_COST`, and the environment block carries
+  `LLAMA_CLOUD_API_KEY`.

package/release-notes/testkit/1.2.76.md

@@ -0,0 +1,11 @@
+---
+version: 1.2.76
+date: 2026-09-19
+summary: Mock Llm.ocr and the LlamaCloud provider
+---
+
+## Changes
+
+- `Llm.ocr` (static, instance, and on the inner provider) resolves a mock OCR
+  response with one page of `_MOCK_OCR_MARKDOWN`.
+- New mocks: `LlamaCloudProvider` and `LlamaCloudClient`.

package/skills/llm.md

@@ -42,6 +42,7 @@ the alias rather than the id it currently resolves to. Read the id off
 | Meta | "muse" (plus the exact name `meta` and a `meta:` prefix) | `MODEL.MUSE_SPARK` |
 | Bedrock | "amazon.nova", "anthropic.claude", "meta.llama", "deepseek.", "google.gemma", "moonshotai.", "openai.gpt-oss", … | `MODEL.NOVA_PRO` |
 | TypeSafe | "jev", "typesafe" | `MODEL.JEV` |
+| LlamaCloud | "llamacloud", "llamaindex", "llamaparse" | `MODEL.LLAMAPARSE.AGENTIC` |
 
 The provider name for Gemini models is `"google"` — `"gemini"` is accepted as a deprecated alias.
 
@@ -51,10 +52,12 @@ Mistral's family names mostly do not contain the substring "mistral" (`ministral
 
 TypeSafe serves System One models (Jev), which answer typed questions and generate no text. The provider implements `question()` only: `operate()`, `send()`, and `stream()` throw. See [Questions](#questions-jev-shape).
 
+LlamaCloud serves LlamaParse, which extracts documents and generates no text. The provider implements `ocr()` only. The parse tier is the model id (`MODEL.LLAMAPARSE.FAST`, `COST_EFFECTIVE`, `AGENTIC`, `AGENTIC_PLUS`). See [OCR](#ocr).
+
 ### Model Constants
 
 - **`PROVIDER.<name>.DEFAULT`** — the single default model per provider (above), used when no `model` is given.
-- **`LLM.MODEL.*`** — the named model catalog (e.g. `MODEL.SONNET`, `MODEL.ASTRA`, `MODEL.SOL`, `MODEL.GEMINI_FLASH`, `MODEL.GROK`, `MODEL.MUSE_SPARK`, `MODEL.MUSE_SPARK_CONTRIBUTOR`, `MODEL.NOVA_PRO`, `MODEL.NOVA_LITE`), plus three nested subtrees: `MODEL.FIREWORKS.*` for Fireworks serverless models (`DEEPSEEK`, `GLM`, `GPT_OSS`, `INKLING`, `KIMI`, `MINIMAX`, `NEMOTRON`, `QWEN`), `MODEL.MISTRAL.*` (`LARGE`, `OCR`, `SMALL`), and `MODEL.OPENROUTER.*` for provider-prefixed routes (`GLM`, `LUNA`, `SONNET`). Pick specific models from here. `MODEL.MISTRAL.OCR` serves `POST /v1/ocr` rather than chat completions: it is priced per page, carries no `COST` entry, and is reached through `MistralProvider.ocr()`. Amazon's Nova models are first-class ids served over Bedrock; Bedrock's third-party routes are not catalogued — pass the literal id (e.g. `us.anthropic.claude-sonnet-4-6`) and `determineModelProvider` resolves it to `bedrock`.
+- **`LLM.MODEL.*`** — the named model catalog (e.g. `MODEL.SONNET`, `MODEL.ASTRA`, `MODEL.SOL`, `MODEL.GEMINI_FLASH`, `MODEL.GROK`, `MODEL.MUSE_SPARK`, `MODEL.MUSE_SPARK_CONTRIBUTOR`, `MODEL.NOVA_PRO`, `MODEL.NOVA_LITE`), plus three nested subtrees: `MODEL.FIREWORKS.*` for Fireworks serverless models (`DEEPSEEK`, `GLM`, `GPT_OSS`, `INKLING`, `KIMI`, `MINIMAX`, `NEMOTRON`, `QWEN`), `MODEL.MISTRAL.*` (`LARGE`, `OCR`, `SMALL`), `MODEL.LLAMAPARSE.*` for LlamaParse tiers (`FAST`, `COST_EFFECTIVE`, `AGENTIC`, `AGENTIC_PLUS`), and `MODEL.OPENROUTER.*` for provider-prefixed routes (`GLM`, `LUNA`, `SONNET`). Pick specific models from here. `MODEL.MISTRAL.OCR` and `MODEL.LLAMAPARSE.*` are document-extraction engines reached through `Llm.ocr`, not chat completions: they are priced per page in `LLM.PAGE_COST` (USD per 1,000 pages, keyed by literal id) and carry no `COST` entry. Amazon's Nova models are first-class ids served over Bedrock; Bedrock's third-party routes are not catalogued — pass the literal id (e.g. `us.anthropic.claude-sonnet-4-6`) and `determineModelProvider` resolves it to `bedrock`.
 - `MODEL.MUSE_SPARK` and `MODEL.MUSE_SPARK_CONTRIBUTOR` are the same Meta model at two tiers. The contributor tier is roughly a tenth of the price because prompts and completions may train Meta models; it is limited to 100 RPM and does not accept `reasoning.effort: "max"`. It is an explicit opt-in: `PROVIDER.META.DEFAULT` is `MODEL.MUSE_SPARK`.
 - The catalog is the **single source of truth for CI coverage**: `packages/llm/test/models.ts` derives the live capability matrix from `MODEL.*` plus each `PROVIDER.*.DEFAULT`, and the workflow shards it by provider. Adding a model to `MODEL.*` puts it under test; no id list exists anywhere else.
 - **Deprecated:** the size-tier map `PROVIDER.<name>.MODEL.{DEFAULT,LARGE,SMALL,TINY}`, the `DEFAULT.MODEL` bundle, and `ALL` are `@deprecated` and retired in 2.0 — use `PROVIDER.*.DEFAULT` for defaults and `MODEL.*` for named models.
@@ -485,6 +488,70 @@ an attempt with somewhere left to go skips the rate-limit wait. A provider
 that can neither `question` nor `operate` counts as a failed attempt and the
 chain moves on.
 
+## OCR
+
+`Llm.ocr(document, options?)` turns a document into per-page markdown. Two
+engines answer natively behind one request and response shape: Mistral OCR
+(`MODEL.MISTRAL.OCR`, one synchronous call, the default when no provider or
+model is named) and LlamaParse over LlamaCloud (`MODEL.LLAMAPARSE.*`, a job
+that is submitted, polled, and fetched; `LLAMA_CLOUD_API_KEY`). Every other
+provider answers through emulation, as `Llm.question` does: one structured
+`operate()` call per page. A `model` array is therefore a fallback chain
+across native engines and chat models alike.
+
+```typescript
+import { Llm, LLM } from "@jaypie/llm";
+
+const { emulated, markdown, pages, usage } = await Llm.ocr(
+  "./scans/invoice.pdf",
+  {
+    model: [
+      LLM.MODEL.MISTRAL.OCR,
+      LLM.MODEL.LLAMAPARSE.COST_EFFECTIVE,
+      LLM.MODEL.HAIKU, // emulated: any provider that accepts files
+    ],
+    pages: "1,3-5", // 1-indexed; [1, 3, 4, 5] also works
+    tables: "markdown", // or "html"
+  },
+);
+
+pages[0].page; // 1
+pages[0].markdown;
+pages[0].confidence; // emulated only: the model's self-reported 0..1
+emulated; // true when a chat model transcribed the pages
+usage.pages; // pages billed
+usage.cost; // USD from LLM.PAGE_COST (LlamaParse: from recorded credits; emulated: from COST tokens)
+usage.tokens; // emulated only: every operate() call's usage
+```
+
+Emulation sends each page alone (a PDF is trimmed to one page per call, so
+the model never numbers pages), asks for verbatim Markdown with
+`[UNCLEAR: best guess]` and `[ELEMENT: description]` markers, transcribes a
+page scored under 0.4 once more and keeps the higher score, and runs
+`concurrency` pages at a time (default 5); the first failure stops the rest
+so the next engine starts clean. Unreadable content fails the attempt so the
+chain moves on. A provider with native `ocr` (Mistral) always answers
+natively; a provider with neither `ocr` nor `operate` (`typesafe`) fails its
+attempt and the chain moves on.
+
+`document` is an `https://` URL (fetched by the vendor), a `data:` URI, an S3
+key (when `CDK_ENV_BUCKET` is set), a local path, or the `{ file, bucket?,
+data?, pages? }` / `{ image, ... }` objects `operate()` accepts. Set
+`images: true` to download extracted images as `data:` URIs onto
+`images[]` and `pages[].images`. `providerOptions` spreads last onto the
+vendor request: Mistral `OCRRequest` fields (`document_annotation_format`,
+`include_blocks`, ...) or LlamaParse `ParseRequestConfiguration` fields
+(`version`, `agentic_options.custom_prompt`, `processing_options`,
+`output_options`, `disable_cache`, ...). `timeout` (default ten minutes)
+bounds a LlamaParse job; expiry throws `LlmTransientError` so a chain can try
+the next engine. `retry` and `signal` behave as on `operate()`.
+
+LlamaParse's API version is pinned per tier in `PROVIDER.LLAMACLOUD.VERSION`
+(each tier accepts its own dated versions); the response `model` reads
+`llamaparse-agentic@2026-09-13` so usage records name the configuration. The
+fast tier returns text only, copied into `markdown`. Failed pages arrive with
+`success: false` and `error` rather than failing the call.
+
 ## Conversation History
 
 Continue conversations across calls:
@@ -734,6 +801,7 @@ OPENROUTER_API_KEY  # Required for OpenRouter
 XAI_API_KEY         # Required for xAI (Grok)
 META_API_KEY        # Required for Meta (Muse Spark); MODEL_API_KEY is read as a fallback
 TYPESAFE_API_KEY    # Required for TypeSafe (Jev); emulated questions use the answering provider's key
+LLAMA_CLOUD_API_KEY # Required for LlamaParse (Llm.ocr); Mistral OCR uses MISTRAL_API_KEY
 ```
 
 Keys are resolved via `getEnvSecret()` which supports AWS Secrets Manager.