openzoo 0.48.49 → 0.48.51

package/README.md

@@ -93,7 +93,7 @@ Builds a ~965k-token document with one planted fact, shows that buying direct re
 
 ```
 bound once in 14.8s: 3.7MB → context ctx_01KZZY8YQE…
-quote for the ask: $0.000480 · pricing=markup (3× a tiny body — the 3.7MB corpus is not re-priced)
+quote for the ask: $0.000480 · pricing=markup (a tiny body — the 3.7MB corpus is not re-priced)
 ```
 
 If the wallet is funded with USDC or TOKEN it pays (capped at `OPENZOO_DEMO_MAX_USD`, default $0.01) and prints the answer, the tokens the model actually read, and the receipt. If not, it prints exactly what to fund. Long waits (the one-time upload, pricing, payment, the answer) show a live progress line with stage + elapsed seconds.
@@ -111,7 +111,7 @@ If the wallet is funded with USDC or TOKEN it pays (capped at `OPENZOO_DEMO_MAX_
 The zoo keeps your corpus in leCore holographic memory; the shim keeps a manifest at `~/.openzoo/contexts.json` (chmod 600) mapping `sha256(corpus)` → the zoo's `context_id`, scoped per API base. When a request carries a corpus the manifest already knows:
 
 - **nothing big is uploaded** — the ask ships alone with an `X-HRR-Context` header,
-- **the 402 prices the tiny ask** (markup basis, honestly labeled `pricing=markup`), typically a few hundredths of a cent instead of re-pricing megabytes,
+- **the 402 prices the tiny ask** (honestly labeled `pricing=markup`), typically a few hundredths of a cent instead of re-pricing megabytes,
 - **the answer still comes from your corpus** — the zoo recalls the relevant slices server-side.
 
 This works in all three fronts: the **proxy** (a big pasted body in the last message is split at its last blank line, bound once, and reused on every later call — even with a different question), the **MCP** `zoo_ask` `corpus` parameter, and the **demo**. If the zoo ever forgets a context (sidecar wipe), the gateway answers 404 *before* any payment and the shim transparently re-binds once and retries — a stale manifest never fails a call.
@@ -175,9 +175,9 @@ Pin the key with `OPENZOO_TUNNEL_TOKEN` if your IDE stores it. Keys never leave
 ## Honest pricing note
 
 Two bases, reported per call in the 402 (`extra.pricing`):
-- **Short prompts price at a markup** (3× provider cost) — there's nothing to spill, you're paying for passthrough.
+- **Short prompts price at cost.** There's nothing to spill, so there's no saving to share — you pay what the call cost us, reconciled against the provider's own metered figure after it completes.
 - **Big bodies price at a counterfactual discount** (~10× cheaper than buying the same call direct) — the zoo's leCore memory means it never forwards your whole body upstream, and passes the savings on. Measured numbers at [benches.openzoo.fun](https://benches.openzoo.fun).
-- **Asks against a bound corpus price on the markup basis** (3× a tiny body — the receipt says `pricing=markup`). That is not a discount trick: 3× of a few hundred tokens is normally far below even the counterfactual price of shipping the corpus, which is the whole point of binding once.
+- **Asks against a bound corpus price on the small forwarded body** (the receipt says `pricing=markup`). That is not a discount trick: a few hundred tokens at cost is far below the counterfactual price of shipping the corpus, which is the whole point of binding once.
 
 The receipt names which base you got; `extra.directUsd` / `extra.savesVsDirect` let you check the math.
 

package/lib/anthropic.js

@@ -177,3 +177,109 @@ export function writeAnthropicSse(res, message, upstream) {
   ev('message_stop', {});
   res.end();
 }
+
+/**
+ * Translate an OpenAI SSE stream into an Anthropic one, INCREMENTALLY.
+ *
+ * The buffered path above exists because the gateway used to answer with a
+ * finished JSON body. It streams for real now, and piping those frames straight
+ * through gives an Anthropic client a 200 whose body it cannot parse — observed
+ * as "API returned an empty or malformed response (HTTP 200)". The stopgap was
+ * to ask the gateway not to stream and keep buffering, which is correct and
+ * silent: Claude Code sends max_tokens=32000 against a 600-turn transcript, so a
+ * turn is MINUTES of zero bytes and reads as a hang.
+ *
+ * So: same grammar, emitted as it arrives.
+ *
+ * BLOCK INDICES ARE NOT THE OPENAI ONES. Anthropic numbers content blocks
+ * sequentially across the whole message, while OpenAI numbers tool_calls in
+ * their own space starting at 0 — which collides with the text block. Tool
+ * indices are therefore remapped on first sight and remembered.
+ *
+ * `onReceipt` takes the gateway's trailing `: x402 {...}` SSE comment, which is
+ * swallowed here rather than forwarded: it is ours, and the Anthropic grammar
+ * has no room for it.
+ */
+export function streamOpenAIToAnthropic(res, upstream, requestedModel, onReceipt) {
+  const headers = {
+    'content-type': 'text/event-stream; charset=utf-8',
+    'cache-control': 'no-cache, no-transform',
+    'x-accel-buffering': 'no',
+    connection: 'keep-alive',
+  };
+  const settle = upstream?.headers?.get?.('x-payment-response');
+  if (settle) headers['x-payment-response'] = settle;
+  res.writeHead(200, headers);
+  const ev = (type, data) => res.write(`event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`);
+
+  let started = false;
+  let textIndex = -1;      // Anthropic index of the text block, -1 until opened
+  let nextIndex = 0;       // next free Anthropic block index
+  const toolIndex = new Map();   // OpenAI tool_calls[].index -> Anthropic index
+  const open = new Set();
+  let stopReason = 'end_turn';
+  let usage = null;
+  let msgId = null;
+
+  const start = () => {
+    if (started) return;
+    started = true;
+    ev('message_start', {
+      message: {
+        id: msgId ?? `msg_${Date.now()}`, type: 'message', role: 'assistant',
+        model: requestedModel, content: [], stop_reason: null, stop_sequence: null,
+        usage: { input_tokens: 0, output_tokens: 0 },
+      },
+    });
+  };
+
+  const onChunk = (d) => {
+    if (d.id && !msgId) msgId = d.id;
+    if (d.usage) usage = d.usage;
+    const ch = (d.choices ?? [])[0];
+    if (!ch) return;
+    start();
+    const delta = ch.delta ?? {};
+    if (typeof delta.content === 'string' && delta.content.length) {
+      if (textIndex < 0) {
+        textIndex = nextIndex++;
+        ev('content_block_start', { index: textIndex, content_block: { type: 'text', text: '' } });
+        open.add(textIndex);
+      }
+      ev('content_block_delta', { index: textIndex, delta: { type: 'text_delta', text: delta.content } });
+    }
+    for (const t of delta.tool_calls ?? []) {
+      const k = t.index ?? 0;
+      if (!toolIndex.has(k)) {
+        const idx = nextIndex++;
+        toolIndex.set(k, idx);
+        ev('content_block_start', {
+          index: idx,
+          content_block: { type: 'tool_use', id: t.id ?? `toolu_${idx}`, name: t.function?.name ?? '', input: {} },
+        });
+        open.add(idx);
+      }
+      const frag = t.function?.arguments;
+      if (typeof frag === 'string' && frag.length) {
+        ev('content_block_delta', {
+          index: toolIndex.get(k),
+          delta: { type: 'input_json_delta', partial_json: frag },
+        });
+      }
+    }
+    if (ch.finish_reason) stopReason = STOP_REASON[ch.finish_reason] ?? 'end_turn';
+  };
+
+  const finish = () => {
+    start();   // a stream that produced nothing still owes the client a message
+    for (const i of open) ev('content_block_stop', { index: i });
+    ev('message_delta', {
+      delta: { stop_reason: stopReason, stop_sequence: null },
+      usage: { output_tokens: usage?.completion_tokens ?? 0 },
+    });
+    ev('message_stop', {});
+    res.end();
+  };
+
+  return { onChunk, finish, receipt: onReceipt, usageOf: () => usage };
+}

package/lib/proxy.js

@@ -15,7 +15,7 @@ import { maybeRewriteModel, rewritablePath, augmentModelList, ALIAS_IDS } from '
 import { forgetContext } from './contexts.js';
 import { injectBrief } from './brief.js';
 import { withNamespace } from './namespace.js';
-import { anthropicToOpenAI, openAIToAnthropic, writeAnthropicSse } from './anthropic.js';
+import { anthropicToOpenAI, openAIToAnthropic, streamOpenAIToAnthropic, writeAnthropicSse } from './anthropic.js';
 import { responsesToChat, chatToResponses, writeResponsesSse } from './responses.js';
 
 const HOP_BY_HOP = new Set([
@@ -391,6 +391,29 @@ async function spillTranscript(body, log) {
   }
   if (tailStart > cut) cut = tailStart;
 
+  // NEVER SPILL THE CURRENT ASK.
+  //
+  // The tail budget walks BACKWARD accumulating bytes, and in an agent loop the
+  // last few messages are tool results — file reads, greps, build output. Those
+  // alone blow through 6,000 chars, so the cut lands AFTER the user's actual
+  // instruction and the instruction goes into the bound corpus instead of the
+  // forwarded window. It then only comes back if top-k recall happens to surface
+  // it against its own text, which is exactly the query it is least likely to
+  // match.
+  //
+  // OBSERVED: "sending 2/578 turns", and the model replying "I don't have a
+  // specific request to act on — your message came through empty", then
+  // re-reading the plan doc to work out where it was, every single turn. A loop
+  // that looks like amnesia and is actually us deleting the question.
+  //
+  // Retrieval is for CONTEXT. The ask itself is never context, and must survive
+  // any budget.
+  let lastUser = -1;
+  for (let i = msgs.length - 1; i > firstSpillable; i--) {
+    if (msgs[i].role === 'user' && msgText(msgs[i]).trim()) { lastUser = i; break; }
+  }
+  if (lastUser > firstSpillable && cut > lastUser) cut = lastUser;
+
   const head = msgs.slice(0, firstSpillable);     // system block, always kept
   const corpus = msgs.slice(firstSpillable, cut).map(msgText).filter(Boolean).join('\n\n');
   if (corpus.length <= BIND_MIN_CHARS) return null;
@@ -932,7 +955,15 @@ export async function startProxy({ silent = false, requireToken = null, sessionM
         // translator exists; a readable answer late beats an unreadable one now.
         const converted = anthropicToOpenAI(inbound);
         clientWantsStream = converted.stream === true || inbound.stream === true;
-        converted.stream = false;
+        // STREAM AGAIN. This forced `stream:false` for a few hours because
+        // relay() piped the gateway's OpenAI frames straight to a client that
+        // speaks message_start / content_block_delta, producing a 200 nobody
+        // could parse. Buffering fixed the parse and cost the whole point:
+        // Claude Code sends max_tokens=32000 against a 600-turn transcript, so
+        // every turn became minutes of zero bytes and read as a hang.
+        // streamOpenAIToAnthropic() translates the grammar frame by frame, so
+        // the lane can be fast AND readable.
+        converted.stream = clientWantsStream;
         bodyBuf = Buffer.from(JSON.stringify(converted));
         anthropicMode = true;
         req.url = '/v1/chat/completions';
@@ -1259,7 +1290,7 @@ export async function startProxy({ silent = false, requireToken = null, sessionM
       // A STREAMED call is metered from the gateway's trailing SSE comment —
       // same figures the JSON path reads out of `data.x402`, same counters, so
       // the status line does not care which transport served the answer.
-      await relay(res, response, (x) => {
+      const meterStreamed = (x) => {
         if (paid || typeof x?.billedUsd !== 'number') return;
         sessionSpent += x.billedUsd;
         sessionCogs += typeof x.cogsUsd === 'number' ? x.cogsUsd : x.billedUsd / MARKUP;
@@ -1274,7 +1305,42 @@ export async function startProxy({ silent = false, requireToken = null, sessionM
         paidCalls += 1;
         if (viaTunnel) tunnelSpent += x.billedUsd;
         say(`credit -> $${x.billedUsd.toFixed(6)}  ·  session $${sessionSpent.toFixed(6)}`);
-      });
+      };
+
+      // ANTHROPIC CLIENTS GET THE SAME STREAM, IN THEIR OWN GRAMMAR.
+      // Translated frame by frame rather than buffered, so Claude Code sees
+      // tokens as they are produced instead of nothing until the turn ends.
+      if (anthropicMode && wantsStream
+          && (response.headers.get('content-type') || '').includes('text/event-stream')
+          && response.ok && response.body) {
+        const tr = streamOpenAIToAnthropic(res, response, anthropicModel, meterStreamed);
+        let pending = '';
+        const body = Readable.fromWeb(response.body);
+        res.on('close', () => body.destroy());
+        try {
+          for await (const c of body) {
+            pending += c.toString('utf8');
+            const lines = pending.split('\n');
+            pending = lines.pop() ?? '';
+            for (const line of lines) {
+              if (line.startsWith(': x402 ')) {
+                try { meterStreamed(JSON.parse(line.slice(7))); } catch { /* not ours */ }
+                continue;
+              }
+              if (!line.startsWith('data:')) continue;
+              const payload = line.slice(5).trim();
+              if (!payload || payload === '[DONE]') continue;
+              try { tr.onChunk(JSON.parse(payload)); } catch { /* frame split across chunks */ }
+            }
+          }
+        } catch (e) {
+          log(`anthropic stream aborted: ${e.message}`);
+        }
+        tr.finish();
+        return;
+      }
+
+      await relay(res, response, meterStreamed);
     } catch (err) {
       if (err instanceof QuoteTooHighError) {
         log(err.message);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "openzoo",
-  "version": "0.48.49",
+  "version": "0.48.51",
   "description": "Local x402-paying proxy + MCP server for openzoo.fun — point any OpenAI-compatible harness (Cursor, Claude Code, aider, SDKs) at localhost and it pays per call from a local burner wallet. Solana and Base rails live; Robinhood experimental.",
   "license": "MIT",
   "type": "module",