@pi-in-go/pigpen-pi-typesafe 0.1.0

package/port/PORT.md

@@ -0,0 +1,98 @@
+# Port record: pi-typesafe
+
+| Input | Identity |
+|---|---|
+| Original | https://github.com/DevMortimer/pi-typesafe, commit `ed439f834665ad6fc652787f5ba7c23852566d49` (0.8.0), MIT, Ryan Gapac; vendored unmodified in `port/oracle/` (no `.github`, `package-lock.json` or preview image). Baseline: `npm test` 134/134 in that directory. |
+| Oracle | Pi 0.87.1 (the npm release), Node 24.19.0; the extension loaded from `port/oracle/src/extension.ts` with `npm install` dependencies (`@typesafe-ai/sdk` 0.6.0, `typebox`) |
+| Target | PiG 0.3.0+0.87.1 (source commit `63c6ba456`), go1.27.1, shared client `components/typesafe` (port of `@typesafe-ai/sdk` 0.6.0 and system-one-adapter) |
+| Original's tests | 134 cases in 12 files; the ledger is `port/upstream-tests.json` |
+
+Two Packages carry the port: `components/pi-typesafe` (extension, this record) and `components/pi-typesafe-api` (the typed API).
+
+## File mapping
+
+| Original (`port/oracle/`) | Go |
+|---|---|
+| `src/index.ts` (exports) | the exported API of `pi-typesafe-api` (package `pitypesafe`) |
+| `src/client.ts` (`createTypeSafe`, budget, ledger, model mapping, backend transport) | `client.go` (`New`, `Evaluate`, `EvaluateRaw`, `ListModels`, `backendDoer`); the SDK calls are the shared client's |
+| `src/backends.ts` | `backends.go` (+ the own-model backend, `hostmodel/`) |
+| `src/schema.ts` (typebox schema, admission) | `schema.go`, `evaluation_schema.json` (the schema TypeBox serializes to, generated from the original), `json.go` (ordered JSON) |
+| `src/errors.ts` | `errors.go` |
+| `src/credentials.ts`, `src/auth.ts`, `src/usage.ts` | `credentials.go`, `auth.go`, `usage.go` |
+| `src/batch.ts`, `src/ask.ts`, `src/calibrate.ts` | `batch.go`, `ask.go`, `calibrate.go` |
+| `src/login.ts`, `src/key-prompt.ts`, `src/ui.ts` | `ui/login.go`, `ui/keyprompt.go` |
+| `src/extension.ts` | `extensions/pi-typesafe/{extension,tool,command,format}.go` |
+| `extensions/index.js`, `examples/decision-extension.ts` | the `pi.extensions` entry in `package.json`; the example is not ported (its content is in the README) |
+| `tests/*.test.ts` | Go twins: `pi-typesafe-api/*_test.go`, `pi-typesafe-api/ui/ui_test.go`, `extensions/pi-typesafe/extension_test.go` |
+
+## Twins and scenarios
+
+- **134 exact twins, 0 skipped** (`pigeq twins check`: 111 in `pi-typesafe-api`, 7 in `pi-typesafe-api/ui`, 16 in the extension).
+  Sub-cases with no Go counterpart are named inside their twin: JavaScript getters, `Date`, `BigInt`, `NaN` options, `null` (Go nil is
+  the default), a value of the wrong type in a struct field (covered through map-form endpoints).
+- **Layer 2: 12 scenarios** (`port/scenarios`), golden traces recorded from the original under Pi 0.87.1 and each also identical under
+  PiG's Node runtime; the Go port reproduces every one (`pigeq check`: 14/14 with the gap and exec pseudo-scenarios). The scenarios
+  cover what needs no network: status, setup, login refusal, logout, consent accepted and declined, the disabled tool called by a
+  model, the playground's three no-send paths, and actions followed by extra words (`trailing-words`, added by the review). The harness reserves `PI_*` variables, so `PI_TYPESAFE_ENABLED` scenarios
+  cannot be scripted; those (headless opt-in, callouts) are layer-1 cases with the fake host. **Network paths** (a successful
+  evaluation, HTTP errors, model lists) run against a fake TypeSafe HTTP server in Go; the original cannot be pointed at one (it
+  deliberately ignores `TYPESAFE_BASE_URL`), so those are proven by the twins, not by traces.
+- **Mutations** (`port/mutations.json`, `pigeq mutate --unit`): see Results.
+
+## Deliberate differences (each named, none silent)
+
+1. **PiG's directories.** State lives in `<PiG agent dir>/pi-typesafe/` (`PIG_CODING_AGENT_DIR`, else `$PIG_HOME/agent`, else `~/.pig/agent`;
+   Pi's `PI_CODING_AGENT_DIR`/`~/.pi/agent` with `PIG_USE_PI_DIRS=1`) instead of Pi's fixed directory (PiG divergence D2). Twin
+   "credentials live under Pi's agent directory" asserts PiG's directory.
+2. **Own-model backend** (owner requirement): `ownmodel` in the registry, `PI_TYPESAFE_BACKEND`, and `/typesafe backend [typesafe|ownmodel]`
+   (completed, not in the usage text). Unknown-backend text therefore lists `ownmodel`. Disclosure and consent follow the destination.
+3. **Zero values mean "default"** in `Options` (timeout, model, maxRequests, ...); negatives are rejected. The original's `timeoutMs: 0`, `NaN`
+   and `model: ""` rejections have no Go counterpart.
+4. **Key order in tool arguments.** PiG's Go SDK decodes tool arguments into maps, so question ids and Choice options arrive sorted, where JavaScript
+   keeps insertion order. Typed (`typesafe.SystemOneRequest`) and JSON-text (`ParseJSON`, the playground) paths keep order end to end.
+   The tool's `details` carry an extra `order` array so the renderer shows answers in question order. Reported as an SDK gap.
+5. **Validation wording.** Request-level failures (missing properties, counts, non-objects) use TypeBox's words (`testdata/typebox-messages.json`
+   compares them with the original); deep union-branch failures name the offending path with different words. The host validates tool
+   arguments against the same schema first, so the difference is reachable only from the playground and the library.
+6. **Model list** entries are read leniently as in the original; the Go client's typed decoder is bypassed (`ListRaw`).
+7. **Answer order** in `Evaluation` comes from the request's questions (`Order`), since answers are a map.
+8. **Scheduling.** Two requests started together race for the last allowed request; JavaScript ran the first to its cap check first. The batch twin "a daily cap stops the batch with the cap named" therefore asserts that exactly one of the two is refused, naming the cap.
+9. **Consent across a concurrent backend switch** (review). The original runs on one thread, so its consent check and its send cannot be
+   separated; here a `/typesafe backend` command can run while a tool call or the `/typesafe test` dialog is in progress. A switch
+   counts as a new destination: a call or dialog admitted before it stops with "The judgment backend changed after this request was
+   admitted; nothing was sent." (`clientFor`, `review_test.go`). A call already sending is not cancelled, as with `/typesafe disable`.
+10. **Loopback http: hosts** (review). The 127.0.0.0/8 rule also requires the host to parse as an IP address. WHATWG `URL` rejects
+   `127.999.0.1`, and Go's `url.Parse` accepts it as a name that DNS would resolve. The Go rule is stricter than the original's for forms
+   WHATWG normalizes (`127.1`, `0x7f.0.0.1`, `127.0.0.01`): they are refused, not rewritten.
+11. **Own-model errors** (review, PiG-only backend). Setup failures (no model selected, no Context) are configuration errors, so their
+   reason reaches the operator, and the callout names the own-model backend instead of a key.
+
+SDK gap check (`pigeq gaps`): no blocking gaps. PARTIAL stand-ins: entry renderer, tool call/result renderers and prompt snippet render
+lines instead of pi-tui Components (host limit shared by every Go extension).
+
+## Findings about the hosts and the harness
+
+1. A Package with an extension nested under a library root (`go.mod` at the Package root) breaks PiG's packed runner (it derives the
+   import path from the parent module) and `pigeq mutate` (the parent is copied as a sibling). Split the library into its own Package.
+2. `pigeq env`, `pigeq mutate --unit` and `scripts/go-modules.mjs` wrote a go.work in which the SDK is a `replace`; a library module that
+   requires another workspace module at `v0.0.0` then fails with "unknown revision". Fixed here (versioned replace for every used module,
+   test first) and reported to the porter driver.
+3. The Go SDK has no raw access to tool arguments (see difference 4).
+
+## Results (this revision)
+
+- `go test -race` in `pi-typesafe-api` (root, `ui`) and the extension: pass. `GOOS=windows|darwin|linux go vet`: see the report.
+- `pigeq check` against the Pi-recorded goldens: 12 of 12 scenarios plus `port-gaps` and `exec-coverage`.
+- **Mutations (extension)**: `pigeq mutate --unit`, 50 mutants (4 added by the review), **50 killed** (33 by the fake-host tests alone; the scenarios kill the rest or share the kill).
+  The first run left 13 unproven; the added `branches_test.go` cases (callout channel and level, 403, logout's auth record, unusable stored
+  key, consent titles, entries, renderers, tool content) killed them, and three mutants that did not compile were rewritten.
+- **Mutations (library)**: `port/library-mutations.py`, 22 applied mutants over client, backends, credentials, auth, usage, schema, errors, batch,
+  calibrate and ask: all killed after `gaps_test.go` was added, except mutant 5, which is equivalent (a model containing `/` cannot match either
+  OpenRouter rewrite rule) and was dropped.
+
+## Re-verified on PiG 0.4.1 (porter-verify)
+
+The golden traces were recorded again from the original under Pi 1.0.1 (host check: identical under PiG's Node runtime), with PiG 0.4.1 content (`5f948f86a`, `pig --version`
+`0.3.1+1.0.1`) and normalizer v3, because Pi 1.0.x changed the trace format: a `prompt` response now carries
+`data.disposition` (Pi 0.99.0, #9098). An event-by-event diff against the previous traces shows no other
+difference, and `pigeq check` passes on the Go port. Details: `docs/plan/progress/porter-verify.md`.

package/port/accepted-gaps.json

@@ -0,0 +1,3 @@
+{
+  "exec:pi": "scripts/dev-pi.mjs is the upstream repository's development launcher: it is not in package.json `files` and Pi never loads it; the extension starts no process"
+}

package/port/golden/enable-confirm.jsonl

@@ -0,0 +1,11 @@
+{"kind":"header","scenario":"enable-confirm","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-enable","ch":"ui","data":{"message":"Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"confirm","title":"Enable TypeSafe for this session?"}}
+{"step":"01-enable","ch":"ui","data":{"message":"TypeSafe enabled. Up to 20 attempts in this session; /typesafe disable stops future agent calls.","method":"notify","notifyType":"info"}}
+{"step":"01-enable","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-status","ch":"ui","data":{"message":"TypeSafe: enabled. TypeSafe key: TYPESAFE_API_KEY (not verified yet — the first request proves it). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"02-status","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"03-disable","ch":"ui","data":{"message":"TypeSafe disabled for future agent calls. In-flight requests are not cancelled.","method":"notify","notifyType":"info"}}
+{"step":"03-disable","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"04-status-after","ch":"ui","data":{"message":"TypeSafe: disabled. TypeSafe key: TYPESAFE_API_KEY (not verified yet — the first request proves it). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"04-status-after","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/enable-decline.jsonl

@@ -0,0 +1,20 @@
+{"kind":"header","scenario":"enable-decline","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-enable","ch":"ui","data":{"message":"Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"confirm","title":"Enable TypeSafe for this session?"}}
+{"step":"01-enable","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-call","ch":"response","data":{"command":"prompt","data":{"disposition":"started"},"success":true}}
+{"step":"02-call","ch":"host","data":{"type":"agent_start"}}
+{"step":"02-call","ch":"host","data":{"type":"turn_start"}}
+{"step":"02-call","ch":"host","data":{"message":{"content":"","role":"system"},"type":"message_end"}}
+{"step":"02-call","ch":"host","data":{"message":{"content":[{"text":"judge it","type":"text"}],"role":"user"},"type":"message_end"}}
+{"step":"02-call","ch":"llm","data":{"messages":[{"role":"user","text":"judge it"}],"system":{"inserted":"- typesafe_evaluate: Ask batched structured questions with TypeSafe (external service; operator opt-in required)\n\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites.\n- Request shape, all three question kinds in one call: {\"state\":{\"message\":\"I was charged twice for my subscription. Please help today.\"},\"questions\":{\"category\":{\"type\":\"choice\",\"instructions\":\"Which team should handle this message?\",\"criteria\":{\"billing\":\"Charges and payments\",\"technical\":\"Software failures\",\"other\":\"None of these\"}},\"urgent\":{\"type\":\"noul\",\"instructions\":\"Does the sender request help today?\"},\"frustration\":{\"type\":\"score\",\"instructions\":\"How frustrated does the sender sound?\",\"criteria\":[\"Neutral request\",\"Frustrated but civil\",\"Angry or threatening\"]}}}\n- Use typesafe_evaluate only for requested semantic judgments, not calculations or exact lookups; send only the relevant permitted data.\n- Batch independent typesafe_evaluate questions over the same state; use code or explicit permission rules for actions, never confidence as authorization.\n- When typesafe_evaluate judges several items, give each item a named state field and ask one question per item per dimension, naming the field in the instructions; one question over many items returns an unusable blend.\n- Report typesafe_evaluate answers as the model's judgments with their probabilities; do not replace them with your own guesses, and say when an answer is uncertain","removed":"\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites"},"tools":[{"description":"Read the contents of a file. Supports text files and images (jpg, png, gif, webp, bmp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.","name":"read","parameters":{"properties":{"limit":{"description":"Maximum number of lines to read","type":"number"},"offset":{"description":"Line number to start reading from (1-indexed)","type":"number"},"path":{"description":"Path to the file to read (relative or absolute)","type":"string"}},"required":["path"],"type":"object"}},{"description":"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.","name":"bash","parameters":{"properties":{"command":{"description":"Shell command to execute","type":"string"},"timeout":{"description":"Timeout in seconds (optional, no default timeout)","type":"number"}},"required":["command"],"type":"object"}},{"description":"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.","name":"edit","parameters":{"properties":{"edits":{"description":"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.","items":{"properties":{"newText":{"description":"Replacement text for this targeted edit.","type":"string"},"oldText":{"description":"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.","type":"string"}},"required":["oldText","newText"],"type":"object"},"type":"array"},"path":{"description":"Path to the file to edit (relative or absolute)","type":"string"}},"required":["path","edits"],"type":"object"}},{"description":"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.","name":"write","parameters":{"properties":{"content":{"description":"Content to write to the file","type":"string"},"path":{"description":"Path to the file to write (relative or absolute)","type":"string"}},"required":["path","content"],"type":"object"}},{"description":"Evaluate supplied state with independent Choice, Score, and Noul questions in one TypeSafe request. Each question judges the whole state, so when several items are involved, put each item in a named state field (e.g. `reports.r1`) and ask one question per item per dimension (e.g. `r1_owner`, `r2_owner`), naming the field in the instructions; never aggregate several items into one question. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization. Requires operator opt-in via /typesafe enable or PI_TYPESAFE_ENABLED=1. Limit: 32 questions, 64 KiB JSON, 20 attempts per session; no retries.","name":"typesafe_evaluate","parameters":{"additionalProperties":false,"properties":{"model":{"description":"Jev model id, e.g. jev-latest. Omit for the default.","maxLength":100,"minLength":1,"type":"string"},"questions":{"description":"Questions keyed by a short id, as an object map, not an array: { \"urgent\": { type: \"noul\", instructions: ... } }.","maxProperties":32,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"additionalProperties":false,"properties":{"criteria":{"anyOf":[{"type":"null"},{"additionalProperties":false,"properties":{"false":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"true":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"}],"description":"Optional: what counts as yes and what counts as no, { true, false }."},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"noul","description":"Yes or no: the probability the instructions hold.","type":"string"}},"required":["type"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"The options as a map from label to when it applies: { billing: \"Charges and payments\", other: null }. 1–64 entries.","maxProperties":64,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"choice","description":"Pick one criteria label.","type":"string"}},"required":["type","criteria"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"Ordered rubric levels, lowest first: [\"neutral\", \"angry\"]. 2–32 levels.","items":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"maxItems":32,"minItems":2,"type":"array"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"score","description":"Rate against the ordered criteria levels.","type":"string"}},"required":["type","criteria"],"type":"object"}]}},"type":"object"},"state":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"What to judge: text, or an object whose fields the questions name."}},"required":["state","questions"],"type":"object"}}]}}
+{"step":"02-call","ch":"host","data":{"message":{"content":[{"arguments":{"questions":{"yes":{"instructions":"?","type":"noul"}},"state":"s"},"name":"typesafe_evaluate","type":"toolCall"}],"role":"assistant","stopReason":"toolUse"},"type":"message_end"}}
+{"step":"02-call","ch":"host","data":{"args":{"questions":{"yes":{"instructions":"?","type":"noul"}},"state":"s"},"toolName":"typesafe_evaluate","type":"tool_execution_start"}}
+{"step":"02-call","ch":"host","data":{"isError":true,"result":{"content":[{"text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files.","type":"text"}],"details":{}},"toolName":"typesafe_evaluate","type":"tool_execution_end"}}
+{"step":"02-call","ch":"host","data":{"message":{"content":[{"text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files.","type":"text"}],"details":{},"isError":true,"role":"toolResult","toolName":"typesafe_evaluate"},"type":"message_end"}}
+{"step":"02-call","ch":"host","data":{"type":"turn_end"}}
+{"step":"02-call","ch":"host","data":{"type":"turn_start"}}
+{"step":"02-call","ch":"llm","data":{"messages":[{"role":"user","text":"judge it"},{"role":"assistant","text":"","toolCalls":[{"arguments":"{\"questions\":{\"yes\":{\"instructions\":\"?\",\"type\":\"noul\"}},\"state\":\"s\"}","name":"typesafe_evaluate"}]},{"role":"tool","text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files."}],"system":{"inserted":"- typesafe_evaluate: Ask batched structured questions with TypeSafe (external service; operator opt-in required)\n\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites.\n- Request shape, all three question kinds in one call: {\"state\":{\"message\":\"I was charged twice for my subscription. Please help today.\"},\"questions\":{\"category\":{\"type\":\"choice\",\"instructions\":\"Which team should handle this message?\",\"criteria\":{\"billing\":\"Charges and payments\",\"technical\":\"Software failures\",\"other\":\"None of these\"}},\"urgent\":{\"type\":\"noul\",\"instructions\":\"Does the sender request help today?\"},\"frustration\":{\"type\":\"score\",\"instructions\":\"How frustrated does the sender sound?\",\"criteria\":[\"Neutral request\",\"Frustrated but civil\",\"Angry or threatening\"]}}}\n- Use typesafe_evaluate only for requested semantic judgments, not calculations or exact lookups; send only the relevant permitted data.\n- Batch independent typesafe_evaluate questions over the same state; use code or explicit permission rules for actions, never confidence as authorization.\n- When typesafe_evaluate judges several items, give each item a named state field and ask one question per item per dimension, naming the field in the instructions; one question over many items returns an unusable blend.\n- Report typesafe_evaluate answers as the model's judgments with their probabilities; do not replace them with your own guesses, and say when an answer is uncertain","removed":"\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites"},"tools":[{"description":"Read the contents of a file. Supports text files and images (jpg, png, gif, webp, bmp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.","name":"read","parameters":{"properties":{"limit":{"description":"Maximum number of lines to read","type":"number"},"offset":{"description":"Line number to start reading from (1-indexed)","type":"number"},"path":{"description":"Path to the file to read (relative or absolute)","type":"string"}},"required":["path"],"type":"object"}},{"description":"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.","name":"bash","parameters":{"properties":{"command":{"description":"Shell command to execute","type":"string"},"timeout":{"description":"Timeout in seconds (optional, no default timeout)","type":"number"}},"required":["command"],"type":"object"}},{"description":"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.","name":"edit","parameters":{"properties":{"edits":{"description":"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.","items":{"properties":{"newText":{"description":"Replacement text for this targeted edit.","type":"string"},"oldText":{"description":"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.","type":"string"}},"required":["oldText","newText"],"type":"object"},"type":"array"},"path":{"description":"Path to the file to edit (relative or absolute)","type":"string"}},"required":["path","edits"],"type":"object"}},{"description":"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.","name":"write","parameters":{"properties":{"content":{"description":"Content to write to the file","type":"string"},"path":{"description":"Path to the file to write (relative or absolute)","type":"string"}},"required":["path","content"],"type":"object"}},{"description":"Evaluate supplied state with independent Choice, Score, and Noul questions in one TypeSafe request. Each question judges the whole state, so when several items are involved, put each item in a named state field (e.g. `reports.r1`) and ask one question per item per dimension (e.g. `r1_owner`, `r2_owner`), naming the field in the instructions; never aggregate several items into one question. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization. Requires operator opt-in via /typesafe enable or PI_TYPESAFE_ENABLED=1. Limit: 32 questions, 64 KiB JSON, 20 attempts per session; no retries.","name":"typesafe_evaluate","parameters":{"additionalProperties":false,"properties":{"model":{"description":"Jev model id, e.g. jev-latest. Omit for the default.","maxLength":100,"minLength":1,"type":"string"},"questions":{"description":"Questions keyed by a short id, as an object map, not an array: { \"urgent\": { type: \"noul\", instructions: ... } }.","maxProperties":32,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"additionalProperties":false,"properties":{"criteria":{"anyOf":[{"type":"null"},{"additionalProperties":false,"properties":{"false":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"true":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"}],"description":"Optional: what counts as yes and what counts as no, { true, false }."},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"noul","description":"Yes or no: the probability the instructions hold.","type":"string"}},"required":["type"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"The options as a map from label to when it applies: { billing: \"Charges and payments\", other: null }. 1–64 entries.","maxProperties":64,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"choice","description":"Pick one criteria label.","type":"string"}},"required":["type","criteria"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"Ordered rubric levels, lowest first: [\"neutral\", \"angry\"]. 2–32 levels.","items":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"maxItems":32,"minItems":2,"type":"array"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"score","description":"Rate against the ordered criteria levels.","type":"string"}},"required":["type","criteria"],"type":"object"}]}},"type":"object"},"state":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"What to judge: text, or an object whose fields the questions name."}},"required":["state","questions"],"type":"object"}}]}}
+{"step":"02-call","ch":"host","data":{"message":{"content":[{"text":"done","type":"text"}],"role":"assistant","stopReason":"stop"},"type":"message_end"}}
+{"step":"02-call","ch":"host","data":{"type":"turn_end"}}
+{"step":"02-call","ch":"host","data":{"type":"agent_end"}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/enable-missing-key.jsonl

@@ -0,0 +1,4 @@
+{"kind":"header","scenario":"enable-missing-key","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-enable","ch":"ui","data":{"message":"Run /typesafe login first: no API key is configured.","method":"notify","notifyType":"warning"}}
+{"step":"01-enable","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/login-shadow.jsonl

@@ -0,0 +1,4 @@
+{"kind":"header","scenario":"login-shadow","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-login","ch":"ui","data":{"message":"TYPESAFE_API_KEY is set in the environment and takes precedence over a stored key. Unset it before using /typesafe login.","method":"notify","notifyType":"warning"}}
+{"step":"01-login","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/logout-env-key.jsonl

@@ -0,0 +1,6 @@
+{"kind":"header","scenario":"logout-env-key","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-logout","ch":"ui","data":{"message":"No stored key to remove. TYPESAFE_API_KEY is still set in the environment.","method":"notify","notifyType":"info"}}
+{"step":"01-logout","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-status","ch":"ui","data":{"message":"TypeSafe: disabled. TypeSafe key: TYPESAFE_API_KEY (not verified yet — the first request proves it). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"02-status","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/playground-cancel.jsonl

@@ -0,0 +1,4 @@
+{"kind":"header","scenario":"playground-cancel","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-playground","ch":"ui","data":{"method":"editor","prefill":"{\n  \"state\": {\n    \"message\": \"I was charged twice for my subscription. Please help today.\"\n  },\n  \"questions\": {\n    \"category\": {\n      \"type\": \"choice\",\n      \"instructions\": \"Which team should handle this message?\",\n      \"criteria\": {\n        \"billing\": \"Charges and payments\",\n        \"technical\": \"Software failures\",\n        \"other\": \"None of these\"\n      }\n    },\n    \"urgent\": {\n      \"type\": \"noul\",\n      \"instructions\": \"Does the sender request help today?\"\n    },\n    \"frustration\": {\n      \"type\": \"score\",\n      \"instructions\": \"How frustrated does the sender sound?\",\n      \"criteria\": [\n        \"Neutral request\",\n        \"Frustrated but civil\",\n        \"Angry or threatening\"\n      ]\n    }\n  }\n}","title":"TypeSafe request JSON · edit state and questions"}}
+{"step":"01-playground","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/playground-invalid-json.jsonl

@@ -0,0 +1,5 @@
+{"kind":"header","scenario":"playground-invalid-json","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-playground","ch":"ui","data":{"method":"editor","prefill":"{\n  \"state\": {\n    \"message\": \"I was charged twice for my subscription. Please help today.\"\n  },\n  \"questions\": {\n    \"category\": {\n      \"type\": \"choice\",\n      \"instructions\": \"Which team should handle this message?\",\n      \"criteria\": {\n        \"billing\": \"Charges and payments\",\n        \"technical\": \"Software failures\",\n        \"other\": \"None of these\"\n      }\n    },\n    \"urgent\": {\n      \"type\": \"noul\",\n      \"instructions\": \"Does the sender request help today?\"\n    },\n    \"frustration\": {\n      \"type\": \"score\",\n      \"instructions\": \"How frustrated does the sender sound?\",\n      \"criteria\": [\n        \"Neutral request\",\n        \"Frustrated but civil\",\n        \"Angry or threatening\"\n      ]\n    }\n  }\n}","title":"TypeSafe request JSON · edit state and questions"}}
+{"step":"01-playground","ch":"ui","data":{"message":"Invalid JSON. Keep quoted strings on one line; nothing was sent.","method":"notify","notifyType":"error"}}
+{"step":"01-playground","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/playground-invalid-questions.jsonl

@@ -0,0 +1,5 @@
+{"kind":"header","scenario":"playground-invalid-questions","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-playground","ch":"ui","data":{"method":"editor","prefill":"{\n  \"state\": {\n    \"message\": \"I was charged twice for my subscription. Please help today.\"\n  },\n  \"questions\": {\n    \"category\": {\n      \"type\": \"choice\",\n      \"instructions\": \"Which team should handle this message?\",\n      \"criteria\": {\n        \"billing\": \"Charges and payments\",\n        \"technical\": \"Software failures\",\n        \"other\": \"None of these\"\n      }\n    },\n    \"urgent\": {\n      \"type\": \"noul\",\n      \"instructions\": \"Does the sender request help today?\"\n    },\n    \"frustration\": {\n      \"type\": \"score\",\n      \"instructions\": \"How frustrated does the sender sound?\",\n      \"criteria\": [\n        \"Neutral request\",\n        \"Frustrated but civil\",\n        \"Angry or threatening\"\n      ]\n    }\n  }\n}","title":"TypeSafe request JSON · edit state and questions"}}
+{"step":"01-playground","ch":"ui","data":{"message":"Invalid evaluation request at questions: must not have fewer than 1 properties. Expected { state, questions: { <id>: { type: \"choice\", instructions, criteria: { label: description|null } } | { type: \"score\", instructions, criteria: [level0, level1, ...] } | { type: \"noul\", instructions } } }; 1–32 questions, Choice 1–64 options, Score 2–32 levels.","method":"notify","notifyType":"error"}}
+{"step":"01-playground","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/status-env-key.jsonl

@@ -0,0 +1,6 @@
+{"kind":"header","scenario":"status-env-key","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-status","ch":"ui","data":{"message":"TypeSafe: disabled. TypeSafe key: TYPESAFE_API_KEY (not verified yet — the first request proves it). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"01-status","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-setup","ch":"ui","data":{"message":"Key configured via TYPESAFE_API_KEY. Run /typesafe test for one sample request or /typesafe enable to allow agent tool calls.","method":"notify","notifyType":"info"}}
+{"step":"02-setup","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/status-no-key.jsonl

@@ -0,0 +1,6 @@
+{"kind":"header","scenario":"status-no-key","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-status","ch":"ui","data":{"message":"TypeSafe: disabled. TypeSafe key: missing — every Jev judgment is skipped until a key is configured (/typesafe login or TYPESAFE_API_KEY). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"01-status","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-usage","ch":"ui","data":{"message":"Usage: /typesafe login | logout | setup | status | enable | disable | test | playground","method":"notify","notifyType":"warning"}}
+{"step":"02-usage","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/tool-disabled.jsonl

@@ -0,0 +1,18 @@
+{"kind":"header","scenario":"tool-disabled","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-call","ch":"response","data":{"command":"prompt","data":{"disposition":"started"},"success":true}}
+{"step":"01-call","ch":"host","data":{"type":"agent_start"}}
+{"step":"01-call","ch":"host","data":{"type":"turn_start"}}
+{"step":"01-call","ch":"host","data":{"message":{"content":"","role":"system"},"type":"message_end"}}
+{"step":"01-call","ch":"host","data":{"message":{"content":[{"text":"judge it","type":"text"}],"role":"user"},"type":"message_end"}}
+{"step":"01-call","ch":"llm","data":{"messages":[{"role":"user","text":"judge it"}],"system":{"inserted":"- typesafe_evaluate: Ask batched structured questions with TypeSafe (external service; operator opt-in required)\n\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites.\n- Request shape, all three question kinds in one call: {\"state\":{\"message\":\"I was charged twice for my subscription. Please help today.\"},\"questions\":{\"category\":{\"type\":\"choice\",\"instructions\":\"Which team should handle this message?\",\"criteria\":{\"billing\":\"Charges and payments\",\"technical\":\"Software failures\",\"other\":\"None of these\"}},\"urgent\":{\"type\":\"noul\",\"instructions\":\"Does the sender request help today?\"},\"frustration\":{\"type\":\"score\",\"instructions\":\"How frustrated does the sender sound?\",\"criteria\":[\"Neutral request\",\"Frustrated but civil\",\"Angry or threatening\"]}}}\n- Use typesafe_evaluate only for requested semantic judgments, not calculations or exact lookups; send only the relevant permitted data.\n- Batch independent typesafe_evaluate questions over the same state; use code or explicit permission rules for actions, never confidence as authorization.\n- When typesafe_evaluate judges several items, give each item a named state field and ask one question per item per dimension, naming the field in the instructions; one question over many items returns an unusable blend.\n- Report typesafe_evaluate answers as the model's judgments with their probabilities; do not replace them with your own guesses, and say when an answer is uncertain","removed":"\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites"},"tools":[{"description":"Read the contents of a file. Supports text files and images (jpg, png, gif, webp, bmp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.","name":"read","parameters":{"properties":{"limit":{"description":"Maximum number of lines to read","type":"number"},"offset":{"description":"Line number to start reading from (1-indexed)","type":"number"},"path":{"description":"Path to the file to read (relative or absolute)","type":"string"}},"required":["path"],"type":"object"}},{"description":"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.","name":"bash","parameters":{"properties":{"command":{"description":"Shell command to execute","type":"string"},"timeout":{"description":"Timeout in seconds (optional, no default timeout)","type":"number"}},"required":["command"],"type":"object"}},{"description":"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.","name":"edit","parameters":{"properties":{"edits":{"description":"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.","items":{"properties":{"newText":{"description":"Replacement text for this targeted edit.","type":"string"},"oldText":{"description":"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.","type":"string"}},"required":["oldText","newText"],"type":"object"},"type":"array"},"path":{"description":"Path to the file to edit (relative or absolute)","type":"string"}},"required":["path","edits"],"type":"object"}},{"description":"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.","name":"write","parameters":{"properties":{"content":{"description":"Content to write to the file","type":"string"},"path":{"description":"Path to the file to write (relative or absolute)","type":"string"}},"required":["path","content"],"type":"object"}},{"description":"Evaluate supplied state with independent Choice, Score, and Noul questions in one TypeSafe request. Each question judges the whole state, so when several items are involved, put each item in a named state field (e.g. `reports.r1`) and ask one question per item per dimension (e.g. `r1_owner`, `r2_owner`), naming the field in the instructions; never aggregate several items into one question. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization. Requires operator opt-in via /typesafe enable or PI_TYPESAFE_ENABLED=1. Limit: 32 questions, 64 KiB JSON, 20 attempts per session; no retries.","name":"typesafe_evaluate","parameters":{"additionalProperties":false,"properties":{"model":{"description":"Jev model id, e.g. jev-latest. Omit for the default.","maxLength":100,"minLength":1,"type":"string"},"questions":{"description":"Questions keyed by a short id, as an object map, not an array: { \"urgent\": { type: \"noul\", instructions: ... } }.","maxProperties":32,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"additionalProperties":false,"properties":{"criteria":{"anyOf":[{"type":"null"},{"additionalProperties":false,"properties":{"false":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"true":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"}],"description":"Optional: what counts as yes and what counts as no, { true, false }."},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"noul","description":"Yes or no: the probability the instructions hold.","type":"string"}},"required":["type"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"The options as a map from label to when it applies: { billing: \"Charges and payments\", other: null }. 1–64 entries.","maxProperties":64,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"choice","description":"Pick one criteria label.","type":"string"}},"required":["type","criteria"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"Ordered rubric levels, lowest first: [\"neutral\", \"angry\"]. 2–32 levels.","items":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"maxItems":32,"minItems":2,"type":"array"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"score","description":"Rate against the ordered criteria levels.","type":"string"}},"required":["type","criteria"],"type":"object"}]}},"type":"object"},"state":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"What to judge: text, or an object whose fields the questions name."}},"required":["state","questions"],"type":"object"}}]}}
+{"step":"01-call","ch":"host","data":{"message":{"content":[{"arguments":{"questions":{"yes":{"instructions":"?","type":"noul"}},"state":"s"},"name":"typesafe_evaluate","type":"toolCall"}],"role":"assistant","stopReason":"toolUse"},"type":"message_end"}}
+{"step":"01-call","ch":"host","data":{"args":{"questions":{"yes":{"instructions":"?","type":"noul"}},"state":"s"},"toolName":"typesafe_evaluate","type":"tool_execution_start"}}
+{"step":"01-call","ch":"host","data":{"isError":true,"result":{"content":[{"text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files.","type":"text"}],"details":{}},"toolName":"typesafe_evaluate","type":"tool_execution_end"}}
+{"step":"01-call","ch":"host","data":{"message":{"content":[{"text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files.","type":"text"}],"details":{},"isError":true,"role":"toolResult","toolName":"typesafe_evaluate"},"type":"message_end"}}
+{"step":"01-call","ch":"host","data":{"type":"turn_end"}}
+{"step":"01-call","ch":"host","data":{"type":"turn_start"}}
+{"step":"01-call","ch":"llm","data":{"messages":[{"role":"user","text":"judge it"},{"role":"assistant","text":"","toolCalls":[{"arguments":"{\"questions\":{\"yes\":{\"instructions\":\"?\",\"type\":\"noul\"}},\"state\":\"s\"}","name":"typesafe_evaluate"}]},{"role":"tool","text":"TypeSafe is disabled. Ask the operator to run /typesafe enable; do not enable it by editing configuration or environment files."}],"system":{"inserted":"- typesafe_evaluate: Ask batched structured questions with TypeSafe (external service; operator opt-in required)\n\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites.\n- Request shape, all three question kinds in one call: {\"state\":{\"message\":\"I was charged twice for my subscription. Please help today.\"},\"questions\":{\"category\":{\"type\":\"choice\",\"instructions\":\"Which team should handle this message?\",\"criteria\":{\"billing\":\"Charges and payments\",\"technical\":\"Software failures\",\"other\":\"None of these\"}},\"urgent\":{\"type\":\"noul\",\"instructions\":\"Does the sender request help today?\"},\"frustration\":{\"type\":\"score\",\"instructions\":\"How frustrated does the sender sound?\",\"criteria\":[\"Neutral request\",\"Frustrated but civil\",\"Angry or threatening\"]}}}\n- Use typesafe_evaluate only for requested semantic judgments, not calculations or exact lookups; send only the relevant permitted data.\n- Batch independent typesafe_evaluate questions over the same state; use code or explicit permission rules for actions, never confidence as authorization.\n- When typesafe_evaluate judges several items, give each item a named state field and ask one question per item per dimension, naming the field in the instructions; one question over many items returns an unusable blend.\n- Report typesafe_evaluate answers as the model's judgments with their probabilities; do not replace them with your own guesses, and say when an answer is uncertain","removed":"\nIn addition to the tools above, you may have access to other custom tools depending on the project.\n</tools>\n\n<rules>\n- Use bash for file operations like ls, rg, find\n- Use read to examine files instead of cat or sed.\n- You can inspect PI_* environment variables for current model and session details.\n- Use edit for precise changes (edits[].oldText must match exactly)\n- When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls\n- Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit.\n- Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions.\n- Use write only for new files or complete rewrites"},"tools":[{"description":"Read the contents of a file. Supports text files and images (jpg, png, gif, webp, bmp). Images are sent as attachments. For text files, output is truncated to 2000 lines or 50KB (whichever is hit first). Use offset/limit for large files. When you need the full file, continue with offset until complete.","name":"read","parameters":{"properties":{"limit":{"description":"Maximum number of lines to read","type":"number"},"offset":{"description":"Line number to start reading from (1-indexed)","type":"number"},"path":{"description":"Path to the file to read (relative or absolute)","type":"string"}},"required":["path"],"type":"object"}},{"description":"Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file. Optionally provide a timeout in seconds.","name":"bash","parameters":{"properties":{"command":{"description":"Shell command to execute","type":"string"},"timeout":{"description":"Timeout in seconds (optional, no default timeout)","type":"number"}},"required":["command"],"type":"object"}},{"description":"Edit a single file using exact text replacement. Every edits[].oldText must match a unique, non-overlapping region of the original file. If two changes affect the same block or nearby lines, merge them into one edit instead of emitting overlapping edits. Do not include large unchanged regions just to connect distant changes.","name":"edit","parameters":{"properties":{"edits":{"description":"One or more targeted replacements. Each edit is matched against the original file, not incrementally. Do not include overlapping or nested edits. If two changes touch the same block or nearby lines, merge them into one edit instead.","items":{"properties":{"newText":{"description":"Replacement text for this targeted edit.","type":"string"},"oldText":{"description":"Exact text for one targeted replacement. It must be unique in the original file and must not overlap with any other edits[].oldText in the same call.","type":"string"}},"required":["oldText","newText"],"type":"object"},"type":"array"},"path":{"description":"Path to the file to edit (relative or absolute)","type":"string"}},"required":["path","edits"],"type":"object"}},{"description":"Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Automatically creates parent directories.","name":"write","parameters":{"properties":{"content":{"description":"Content to write to the file","type":"string"},"path":{"description":"Path to the file to write (relative or absolute)","type":"string"}},"required":["path","content"],"type":"object"}},{"description":"Evaluate supplied state with independent Choice, Score, and Noul questions in one TypeSafe request. Each question judges the whole state, so when several items are involved, put each item in a named state field (e.g. `reports.r1`) and ask one question per item per dimension (e.g. `r1_owner`, `r2_owner`), naming the field in the instructions; never aggregate several items into one question. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization. Requires operator opt-in via /typesafe enable or PI_TYPESAFE_ENABLED=1. Limit: 32 questions, 64 KiB JSON, 20 attempts per session; no retries.","name":"typesafe_evaluate","parameters":{"additionalProperties":false,"properties":{"model":{"description":"Jev model id, e.g. jev-latest. Omit for the default.","maxLength":100,"minLength":1,"type":"string"},"questions":{"description":"Questions keyed by a short id, as an object map, not an array: { \"urgent\": { type: \"noul\", instructions: ... } }.","maxProperties":32,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"additionalProperties":false,"properties":{"criteria":{"anyOf":[{"type":"null"},{"additionalProperties":false,"properties":{"false":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"true":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"}],"description":"Optional: what counts as yes and what counts as no, { true, false }."},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"noul","description":"Yes or no: the probability the instructions hold.","type":"string"}},"required":["type"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"The options as a map from label to when it applies: { billing: \"Charges and payments\", other: null }. 1–64 entries.","maxProperties":64,"minProperties":1,"patternProperties":{"^.*$":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]}},"type":"object"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"choice","description":"Pick one criteria label.","type":"string"}},"required":["type","criteria"],"type":"object"},{"additionalProperties":false,"properties":{"criteria":{"description":"Ordered rubric levels, lowest first: [\"neutral\", \"angry\"]. 2–32 levels.","items":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}]},"maxItems":32,"minItems":2,"type":"array"},"instructions":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"One judgment about the whole state, phrased as a question or a statement."},"type":{"const":"score","description":"Rate against the ordered criteria levels.","type":"string"}},"required":["type","criteria"],"type":"object"}]}},"type":"object"},"state":{"anyOf":[{"type":"string"},{"type":"null"},{"items":{},"type":"array"},{"patternProperties":{"^.*$":{}},"type":"object"}],"description":"What to judge: text, or an object whose fields the questions name."}},"required":["state","questions"],"type":"object"}}]}}
+{"step":"01-call","ch":"host","data":{"message":{"content":[{"text":"done","type":"text"}],"role":"assistant","stopReason":"stop"},"type":"message_end"}}
+{"step":"01-call","ch":"host","data":{"type":"turn_end"}}
+{"step":"01-call","ch":"host","data":{"type":"agent_end"}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/golden/trailing-words.jsonl

@@ -0,0 +1,10 @@
+{"kind":"header","scenario":"trailing-words","lane":"pi-ts","host":"pi 1.0.1","extension":"sha256:2dbf2078e91fc7fface683f6ff536e10e8218b387b768a473d7d11f296272329","normalizer":"v3"}
+{"step":"01-status-now","ch":"ui","data":{"message":"Usage: /typesafe login | logout | setup | status | enable | disable | test | playground","method":"notify","notifyType":"warning"}}
+{"step":"01-status-now","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"02-enable-yes","ch":"ui","data":{"message":"Usage: /typesafe login | logout | setup | status | enable | disable | test | playground","method":"notify","notifyType":"warning"}}
+{"step":"02-enable-yes","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"03-logout-please","ch":"ui","data":{"message":"Usage: /typesafe login | logout | setup | status | enable | disable | test | playground","method":"notify","notifyType":"warning"}}
+{"step":"03-logout-please","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"04-status","ch":"ui","data":{"message":"TypeSafe: disabled. TypeSafe key: TYPESAFE_API_KEY (not verified yet — the first request proves it). Session 0/20 attempts; no client yet in this session.  Model: jev-latest. Session limits reset on session start/reload; daily counters persist and caps come from client options or PI_TYPESAFE_MAX_* environment variables. Submitted state and questions will be sent to api.typesafe.ai and may incur charges. Do not include secrets. The extension does not collect files or conversation history. Results are model judgments, not proof or authorization.","method":"notify","notifyType":"info"}}
+{"step":"04-status","ch":"response","data":{"command":"prompt","data":{"disposition":"handled"},"success":true}}
+{"step":"99-shutdown","ch":"exit","data":{"code":0}}

package/port/library-mutations.py

@@ -0,0 +1,44 @@
+# Library mutation check: applies each (file, find, replace) to a copy of components/pi-typesafe-api and runs its tests; expects every mutant killed.
+# Needs a go.work at /tmp/pts/go.work that uses the SDK, pi-typesafe-api and typesafe (see port/PORT.md). Mutant 5 is equivalent (removed); 13 and 18 no longer match the source.
+import subprocess,shutil,os,sys
+HERE=os.path.dirname(os.path.abspath(__file__))
+API=os.path.normpath(os.path.join(HERE,'..','..','pi-typesafe-api'))
+WORK=os.environ.get('LIBMUT_GOWORK','/tmp/pts/go.work')  # a go.work that uses the SDK dir, pi-typesafe-api and typesafe
+M=[
+("client.go","if c.usage.RequestsStarted >= c.maxRequests {\n\t\tc.mu.Unlock()","if c.usage.RequestsStarted > c.maxRequests {\n\t\tc.mu.Unlock()"),
+("client.go","c.usage.RequestsStarted++\n\tc.ledger.RecordStart()","c.ledger.RecordStart()"),
+("client.go","if ctx.Err() != nil {\n\t\treturn nil, newError(CodeAborted","if false {\n\t\treturn nil, newError(CodeAborted"),
+("client.go","if c.backend.ModelsVerifyKey {","if true {"),
+("client.go","transport = backendDoer{inner: opts.HTTPClient, backend: backend}","_ = backend"),
+("backends.go","if strings.Contains(model, \"/\") || backend != BackendOpenRouter {","if backend != BackendOpenRouter {"),
+("backends.go","strings.EqualFold(keyEnv, typesafeKeyEnv)","keyEnv == typesafeKeyEnv"),
+("backends.go","return !b.Local && (b.KeyEnv == \"\" || b.KeyEnv == typesafeKeyEnv)","return !b.Local"),
+("credentials.go","info.Mode().Perm()&0o077 != 0","info.Mode().Perm()&0o007 != 0"),
+("credentials.go","len(key) >= 16","len(key) >= 4"),
+("auth.go","(f.Status == 401 || f.Status == 403)","(f.Status == 401)"),
+("auth.go","f.Code == CodeHTTP &&","true &&"),
+("usage.go","if c.limit > 0 && c.used >= c.limit {","if c.limit > 0 && c.used > c.limit {"),
+("usage.go","return min(a, b)\n\t}\n\treturn SpendCaps","return max(a, b)\n\t}\n\treturn SpendCaps"),
+("schema.go","if qs.Len() < 1 || qs.Len() > DefaultMaxQuestions {","if qs.Len() < 1 || qs.Len() > DefaultMaxQuestions+1 {"),
+("schema.go","if len(text) > maxInputBytes","if len(text) >= maxInputBytes"),
+("schema.go",'item.Set("criteria", labels)','_ = labels'),
+("errors.go",'advice += " Retry after "','advice += " Wait "'),
+("errors.go",'if openrouter','if false'),
+("batch.go","if ctx.Err() != nil {\n\t\t\t\tstopped = true","if false {\n\t\t\t\tstopped = true"),
+("batch.go","return ok && (ie.Code == CodeBudget || ie.Code == CodeAborted)","return ok && ie.Code == CodeBudget"),
+("calibrate.go","case p == n:\n\t\t\t\twins += 0.5","case p == n:\n\t\t\t\twins += 1"),
+("ask.go","timeout = DefaultAskTimeout","timeout = 0"),
+]
+res=[]
+for i,(f,a,b) in enumerate(M):
+    d=f'/tmp/libmut/{i}'
+    shutil.rmtree(d,ignore_errors=True); shutil.copytree(API,d,ignore=shutil.ignore_patterns('.git'))
+    p=os.path.join(d,f); t=open(p).read()
+    if t.count(a)!=1: print('BAD',i,f,t.count(a)); continue
+    open(p,'w').write(t.replace(a,b))
+    env=dict(os.environ); 
+    work=open(WORK).read().replace(API,d)
+    open(d+'/go.work.tmp','w').write(work); env['GOWORK']=d+'/go.work.tmp'
+    r=subprocess.run(['go','test','-count=1','-timeout=120s','.'],cwd=d,env=env,capture_output=True,text=True)
+    status='KILLED' if r.returncode!=0 and ('FAIL' in r.stdout) else ('INVALID' if r.returncode!=0 else 'SURVIVED')
+    print(status,i,f,a[:40].replace('\n',' '))