@pikku/core 0.12.80 → 0.12.82

package/CHANGELOG.md

@@ -1,3 +1,315 @@
+## 0.12.82
+
+### Patch Changes
+
+- 063f43a: api-report.md reports public signatures, one member per line
+
+  The report was built from `declaration.getText()`, so a class arrived as its own
+  source — private fields and method bodies included — flattened onto a single
+  line. `PikkuWorkflowService` was 40,603 characters of one line.
+
+  That made the file unmergeable. One line is one conflict hunk, so two branches
+  touching different methods of the same class conflicted on a line neither had
+  meaningfully changed, and the repo paid for it every rebase.
+
+  Now each member is its own line and stops at its signature, with private members
+  dropped and inferred return types filled in from the checker. The file is a
+  quarter smaller, and every declaration in it parses as TypeScript — 42 of its 50
+  code fences previously did not, because collapsing a multi-line object type threw
+  away the newline that was serving as the member separator.
+
+  Also adds the report to `.prettierignore`: prettier pads the summary tables to
+  their widest cell, so one changed count rewrote all fifty rows, and it reflowed a
+  file that `api-report.test.ts` compares byte-for-byte.
+
+- ce66bf8: A channel message handler that returns nothing no longer tries to send it.
+
+  `local-channel-runner` sent the message handler's result unconditionally, so a
+  handler with nothing to say produced `send requires a non-empty message` on every
+  inbound message. The connect path directly above it has always guarded this; the
+  message path did not.
+
+  Gateway websockets hit it every time — `wireGateway` generates a message handler
+  that returns `undefined` by design — which showed up as a chat gateway accepting
+  a connection and then erroring on each message rather than delivering it.
+
+  Found while testing a webchat gateway in a template. Note that fixing this is
+  necessary but not sufficient for that case: codegen does not emit the channel a
+  websocket gateway registers, so route resolution finds no handler and falls back
+  to the empty inline `onMessage` the gateway wires as a placeholder. That gap is
+  still open.
+
+- d0307a8: Stop the runner overwriting a status the route set
+
+  A function that calls `response.redirect()` has nothing left to return, so it
+  returns `undefined` — and the HTTP runner read that as "no content" and
+  overwrote the 3xx with a 204. The `Location` header survived, but a browser
+  does not follow `Location` on a 204, so the redirect silently became a dead
+  end: the user sits on the page that sent them, waiting for a hop that never
+  comes. This is the whole OAuth/app-install callback shape, where the redirect
+  back to the app is the last step of the flow.
+
+  The same clobber applied to a body: a route that set `201` and returned a
+  value was answered `200`.
+
+  The runner's 204 and 200 are now defaults rather than overrides — they apply
+  only when the route left the status alone.
+
+- ce66bf8: MCP calls now carry the caller's HTTP request, so an MCP tool can require a session.
+
+  Every auth middleware opens with `if (!http?.request) return`. The MCP runner
+  never put an `http` on the wire, so all of them bailed on their first line and
+  an MCP call reached the function with no session — no cookie, no bearer token,
+  no API key, whatever the app had registered. A tool fronting a session-requiring
+  `pikkuFunc` could therefore only ever answer `Authentication required`, and a
+  tool fronting a sessionless one was callable by anyone who could reach the mount.
+
+  Almost nothing was missing. Global middleware already ran for MCP wirings, and
+  the runner already built a `PikkuSessionService` and the middleware session wire
+  props. Only the request was being dropped — twice: `RunMCPEndpointParams` had
+  nowhere to put one, and `createFetchHandler` received the caller's `Request` and
+  discarded it.
+
+  `RunMCPEndpointParams` gains an optional `http`, which the runner places on the
+  wire, and the fetch handler wraps the incoming `Request` in a
+  `PikkuFetchHTTPRequest` and threads it through tools, resources and prompts. The
+  request is cloned before wrapping, because the MCP transport reads the body and
+  both would otherwise compete for one single-use stream; only headers and cookies
+  are wanted, since a tool's input arrives in the JSON-RPC params.
+
+  Transports with no request to offer — stdio, and the long-lived stdio/SSE server
+  paths — pass nothing and stay anonymous. That is a property of those transports
+  rather than a default chosen here, and it is now visible in the type.
+
+  The generated auth middleware moves from `addHTTPMiddleware('*')` to
+  `addGlobalMiddleware`. Carrying the request is necessary but not sufficient:
+  session middleware registered as HTTP middleware runs for HTTP wirings only, so
+  an MCP call still met no middleware and still had no session. Both entries —
+  the Better Auth session and the console bearer token — resolve a session from
+  whatever request the call arrived on, which is not an HTTP routing concern.
+  Wirings with no request are unaffected, since each middleware returns
+  immediately without one.
+
+  That move also retires a hazard the old shape carried: the two entries had to
+  share a single `addHTTPMiddleware('*')` call because the inspector keys
+  route-middleware groups by pattern, so a second `'*'` registration from another
+  file would silently displace the first. Global middleware is an append-only
+  list.
+
+  **Regenerate the auth scaffold after upgrading** — an app still carrying the
+  `addHTTPMiddleware('*')` form keeps anonymous MCP calls.
+
+  Two consequences worth planning for:
+  - **A tool fronting a session-requiring function starts working.** It previously
+    could not run at all.
+  - **A tool fronting a sessionless function is unchanged and still anonymous.**
+    Scopes and permissions now apply to MCP calls exactly as they do elsewhere, so
+    audit any tool that mutates state and give it the scope its HTTP sibling has.
+
+  `PikkuHTTP` is now exported from `@pikku/core/http`; it is part of this contract
+  and was previously only reachable as a type on other exported shapes.
+
+- 3ad2131: Name models by what they are for, and switch them all in one place
+
+  A `models` table in pikku.config.json maps an alias to a provider-qualified
+  model, so a declaration can say `model: 'cheap'` and the project repoints every
+  use of that tier at once instead of editing each agent. A model containing `/`
+  is still concrete and used exactly as written, which is how an agent that needs
+  one specific model pins it — aliases are opt-in.
+
+  The table is baked into codegen rather than read at runtime, so it applies to
+  deployed units and not just local runs, and `pikku dev`/`pikku serve` take
+  `--model cheap:openai/gpt-5-nano` to repoint a tier for one run without editing
+  the config.
+
+  Because the inspector already holds every agent's model literal, a bare name
+  with no matching alias now fails the build (PKU146) naming the aliases that do
+  exist, rather than reaching a provider as an unknown model.
+
+  Aliases resolve for every modality, not just agents: image, speech,
+  transcription, embedding and reranking all reach a provider through the same
+  point in the Vercel runner.
+
+- b930dca: Remove the `secretBroker` escape hatch and scope addon secrets and credentials
+
+  `secretBroker` let three named console functions receive the real `SecretService`,
+  against the rule that a function never sees one. It is gone: the inspector allowlist,
+  the `FunctionRuntimeMeta` flag, the runner branches, and the `WiredSecretBrokerServices`
+  type. Console secret administration moved into the console addon, where a
+  `SecretAdminService` holds the `SecretService` and the functions hold none.
+
+  Addons are now scoped rather than trusted. The CLI emits each package's declared secret
+  keys, and the host wraps the `SecretService` in a `ScopedSecretService` and the
+  `CredentialService` in a new `ScopedCredentialService` before the addon's service factory
+  runs — so an addon reads only what it declared, cannot write secrets, and cannot enumerate
+  the app's users. `wireAddon({ globalSecrets, globalCredentials })` waives this, taking the
+  reason as its value; only the consuming app can grant it, and the deploy manifest reports
+  every grant under `unscopedSecretAddons` / `unscopedCredentialAddons`.
+
+- b95e77d: fix(core): persist working memory on streamed agent runs
+
+  Working memory was never persisted when an agent streamed. The working memory
+  middleware strips the `<working_memory>` block from the outgoing deltas, and the
+  channel that accumulates the reply sits downstream of that strip — so the text
+  later handed to `modifyOutput`, the only place that calls `saveWorkingMemory`,
+  had already had the block removed. It now persists from the stream hook, at the
+  step's `usage` (or `done`) event, where the raw text is still reachable.
+
+  With that dependency gone, `modifyOutput` no longer runs at all on a streamed
+  run: nothing on that path could act on what it returned, since the text has
+  already reached the client and each step is flushed to storage as it goes. A
+  middleware that rewrites in `modifyOutput` without a `modifyOutputStream` — a
+  redaction hook, typically — was silently ineffective while streaming, and is now
+  warned about once per agent.
+
+- fd9d834: Stop publishing internals that only their own package or file used. The declarations stay; only the entrypoint re-export is removed, so nothing that imported a name from where it is declared is affected.
+- 8978fbd: feat(workflow): let an approval gate declare who may answer it
+
+  `workflow.approval()` gains `approvers` (`'any' | 'owner' | 'not-initiator'`)
+  and `approverScope`, so a gate can require four-eyes sign-off, restrict itself
+  to the run's initiator, or require the decider to hold a named scope.
+
+  Both are enforced when the workflow replays the gate — the same place, and for
+  the same reason, the decision payload is validated: the policy is a value on
+  the workflow, and a decision can be recorded before the run has ever reached
+  the gate. A decision that fails the policy is discarded and the gate stays
+  closed. Where the run has already published its policy, the check also runs at
+  submission time so the caller gets a 403 rather than silence.
+
+  An answer is now recorded where it can be answered for later. The settled
+  decision carries `decidedBy` and `decidedAt` in its `ApprovalOutcome`, so who
+  signed reaches `workflowStep.result` and `workflowStepHistory` rather than
+  living only in mutable run state. Every answer — accepted, refused at the door,
+  or cleared on replay — is also written to the audit sink as
+  `workflow.approval.decided`, which outlives the run: `deleteRun` cascades to
+  steps and history, and a refused attempt never reaches a step at all. Projects
+  with no audit service wired are unaffected.
+
+  **This loosens the default.** `approveStep` previously refused anyone but the
+  run's initiator, unconditionally. A gate that declares no `approvers` now
+  accepts a decision from anyone the approve entrypoint admits — restore the old
+  behaviour per-gate with `approvers: 'owner'`, or gate the approve route with
+  `auth`/`permissions`. Ownership still governs _reads_ of a run unchanged.
+
+## 0.12.81
+
+### Patch Changes
+
+- e110c55: Give a finished agent run one finalization seam, and make a failed tool call
+  distinguishable from a tool that returned text saying "Error:".
+  - Tool results carry `error` as its own field, from the runner through the
+    stream event and the run's step record into persisted messages.
+  - `modifyOutput` receives the run's tool calls and may return a rewritten list,
+    which is redistributed back onto the steps it came from.
+  - Streamed runs accumulate their tool calls across steps, and every completion
+    path — streamed, non-streamed, and resumed after a tool approval — now
+    finalizes through `finalizeAgentRun`. A tool that fails after being approved
+    leaves a record on the run instead of vanishing.
+
+- e110c55: Add `scenario.expectScore` — grade a finished agent run with a declared scorer and assert on it.
+
+  An agent's answer cannot be matched against a fixed string, so a scenario grades
+  it instead. `expectScore(step, runId, scorer, { atLeast, atMost, reference })`
+  runs one declared scorer against the run the scenario just triggered and fails
+  with the reason the judge gave. The default bound is `atLeast: 0.5`, so an
+  unqualified assertion still fails a run graded zero.
+
+  Grading goes over the new `pikkuScenarioGradeRun` instrumentation RPC, which the
+  dev server registers alongside the coverage and stub RPCs — so it exists only in
+  processes that should have it, and never in a deployed bundle. It grades from
+  the snapshot the runtime already took when the run finished, which is what makes
+  a scenario's grade the same measurement production's sampler makes rather than
+  an approximation of it: a run's prompt, answer and tool calls are spread across
+  a thread's messages, where the boundary of one run is not recoverable.
+
+  Two things differ deliberately from live scoring. The sample rate is ignored — a
+  scorer grading 1% of traffic still grades every scenario run — and the grade is
+  returned rather than recorded, so a test's score never lands among the
+  production figures. `reference` supplies the answer key a `requiresReference`
+  judge grades against, which is the only way such a judge is reachable at all.
+
+- e110c55: Add runtime scoring for AI agents: `pikkuAIScorer` for heuristic grades and
+  `pikkuAIJudge` for LLM-judged ones, graded off the request path on two queue
+  lanes so a slow judge cannot starve the cheap checks. Grades are sampled
+  deterministically per `(run, scorer)` and persisted to `ai_run_score`.
+- acc8077: Enforce a CLI command's declared `auth`/`permissions`, and mask CLI channel
+  errors in production.
+
+  A command's declared `auth`/`permissions` were accepted by the types but dropped
+  in `registerCLICommands`: when the command wrapped a function-config object,
+  `unwrapFunc` kept only the inner func's fields, so a command-level access-control
+  declaration was a silent no-op. They are now merged into the config passed to
+  `addFunction` — command-level winning, falling back to the handler's — so the
+  function runner enforces them.
+
+  The CLI raw channel runner returned the raw exception message to the remote
+  client. It can carry internals (a stack, a DB error, a path), so it is now logged
+  server-side and replaced with a generic `Command failed` in production; dev keeps
+  the message inline.
+
+  CWE-862 / CWE-209.
+
+- 905f737: Restrict a graph workflow's `startNode` to its declared entry nodes.
+
+  The scaffolded public route `POST /workflow/:name/graph/:nodeId` passes `:nodeId`
+  straight through as `startNode`, and `validateGraphReferences` only checked that
+  the node exists. A caller could name any dependency-free node — one whose input
+  reads only `trigger` — and fire its RPC directly with chosen data, skipping every
+  upstream eligibility, validation or approval node. These node RPCs are internal,
+  so the public `/rpc` endpoint refuses them; this route was the only outside path
+  to them.
+
+  `startWorkflow` — the boundary the public route and triggers enter through — now
+  rejects a `startNode` that is not in the graph's `entryNodeIds`. Internal
+  resume/replay drives `runWorkflowGraph` directly and keeps full node targeting,
+  so the check sits at the trust boundary rather than in the low-level runner.
+
+  CWE-20 / CWE-863.
+
+- 3cc6428: Run a templated MCP resource's middleware for concrete request URIs.
+
+  `runMCPResource` resolves a templated resource's `pikkuFuncId` via the template
+  key, but `runMCPPikkuFunc` then re-looked-up the resource meta by the concrete
+  request URI (`resource://users/123`), which no meta is stored under — so meta was
+  `undefined` and the resource's merged middleware, including any tag-derived auth
+  gate, was silently dropped. A templated MCP resource was reachable with its gate
+  skipped.
+
+  The meta key — the template for a templated match, the URI otherwise — is now
+  carried through to the meta lookup, so the declared middleware runs.
+
+  CWE-863 / CWE-306.
+
+- c524adf: fix(cli,core): make scenario captures reachable, filed per scenario, and findable
+
+  `--screenshots` and `--video` were read by `scenario run` but never declared as
+  options, so both flags were rejected as unknown and silently ignored — capture
+  could not be switched on from the command line at all.
+
+  A provider's `beginScenario` was never called, so every capture in a run was
+  filed under one shared label instead of the scenario that produced it. It is now
+  part of `ScenarioBrowserProvider` and called after the per-scenario reset, once
+  the previous scenario's context is closed and its video finalised.
+
+  The run also never said where it wrote anything. It now reports `Captures → …`
+  after the browser closes, which is the point at which a video exists.
+
+- e110c55: fix(core): persist working memory on streamed agent runs
+
+  Working memory was never persisted when an agent streamed. The working memory
+  middleware strips the `<working_memory>` block from the outgoing deltas, and the
+  channel that accumulates the reply sits downstream of that strip — so the text
+  later handed to `modifyOutput`, the only place that calls `saveWorkingMemory`,
+  had already had the block removed. It now persists from the stream hook, at the
+  step's `usage` (or `done`) event, where the raw text is still reachable.
+
+  With that dependency gone, `modifyOutput` no longer runs at all on a streamed
+  run: nothing on that path could act on what it returned, since the text has
+  already reached the client and each step is flushed to storage as it goes. A
+  middleware that rewrites in `modifyOutput` without a `modifyOutputStream` — a
+  redaction hook, typically — was silently ineffective while streaming, and is now
+  warned about once per agent.
+
 ## 0.12.80
 
 ### Patch Changes

package/dist/errors/index.d.ts

@@ -1,2 +1,2 @@
 export * from './errors.js';
-export { PikkuError, addError, addErrors } from './error-handler.js';
+export { PikkuError, addError } from './error-handler.js';

package/dist/errors/index.js

@@ -1,2 +1,2 @@
 export * from './errors.js';
-export { PikkuError, addError, addErrors } from './error-handler.js';
+export { PikkuError, addError } from './error-handler.js';

package/dist/function/function-runner.js

@@ -84,7 +84,6 @@ export const runPikkuFunc = async (wireType, wireId, funcName, { singletonServic
         throw new Error(`Function meta not found: ${funcName}`);
     }
     const resolvedFunctionId = funcMeta.pikkuFuncId ?? funcName;
-    const keepsSecrets = funcMeta.secretBroker === true;
     const resolvedSingletonServices = packageName
         ? await getOrCreatePackageSingletonServices(packageName, singletonServices, addonInstance)
         : singletonServices;
@@ -201,9 +200,7 @@ export const runPikkuFunc = async (wireType, wireId, funcName, { singletonServic
         }
         await runPermissions({
             funcPermissions: funcConfig.permissions,
-            services: keepsSecrets
-                ? resolvedSingletonServices
-                : withoutSecrets(resolvedSingletonServices, 'a permission'),
+            services: withoutSecrets(resolvedSingletonServices, 'a permission'),
             // knowledge: decisions/security/a-permission-gets-a-wire-it-cannot-reply-on.md
             wire: invocationWire,
             data: actualData,
@@ -237,7 +234,7 @@ export const runPikkuFunc = async (wireType, wireId, funcName, { singletonServic
                 configurable: true,
                 enumerable: true,
             });
-            return await funcConfig.func(keepsSecrets ? services : withoutSecrets(services, 'a pikku function'), actualData, invocationWire);
+            return await funcConfig.func(withoutSecrets(services, 'a pikku function'), actualData, invocationWire);
         }
         finally {
             // Flush the runner-installed audit buffer before wire services close.

package/dist/function/index.d.ts

@@ -1,4 +1,4 @@
 export { addFunction, getAllFunctionNames } from './function-runner.js';
 export { pikkuAuth, pikkuPermission, pikkuPermissionFactory, pikkuApprovalDescription, } from './functions.types.js';
 export type { CorePikkuFunction, CorePikkuFunctionSessionless, CorePikkuFunctionConfig, CorePikkuSessionlessFunctionConfig, CorePikkuAuth, CorePikkuAuthConfig, CorePikkuPermission, } from './functions.types.js';
-export type { ListInput, ListOutput, Filter, LeafFilter, LeafValue, } from './list.types.js';
+export type { ListInput, ListOutput, Filter, } from './list.types.js';

package/dist/index.d.ts

@@ -1,18 +1,18 @@
 /**
  * @module @pikku/core
  */
-export type { AuthInstance, CommonWireMeta, CoreConfig, CorePikkuMiddleware, CorePikkuMiddlewareConfig, CorePikkuMiddlewareFactory, CorePikkuMiddlewareGroup, CoreServices, CoreSecretlessSingletonServices, CoreSingletonServices, CoreUserSession, SecretlessServices, CreateConfig, ServerLifecycle, FunctionMeta, FunctionRuntimeMeta, FunctionServicesMeta, FunctionWiresMeta, FunctionsMeta, FunctionsRuntimeMeta, JSONPrimitive, JSONValue, MakeRequired, MiddlewareMetadata, MiddlewarePriority, PermissionMetadata, PickOptional, PickRequired, PikkuAIMiddlewareHooks, PikkuWire, PikkuRawWire, PikkuWiringTypes, PostgresConfig, RequireAtLeastOne, SecurityAuditIssue, SecurityAuditReport, SecurityAuditSummary, SecurityAuditUpdate, SecuritySeverity, SecurityUpdateLevel, SerializedError, WireServices, } from './types/core.types.js';
+export type { AuthInstance, CommonWireMeta, CoreConfig, CorePikkuMiddleware, CorePikkuMiddlewareConfig, CorePikkuMiddlewareFactory, CorePikkuMiddlewareGroup, CoreServices, CoreSecretlessSingletonServices, CoreSingletonServices, CoreUserSession, SecretlessServices, CreateConfig, ServerLifecycle, FunctionMeta, FunctionServicesMeta, FunctionWiresMeta, FunctionsMeta, FunctionsRuntimeMeta, JSONValue, MakeRequired, MiddlewareMetadata, MiddlewarePriority, PermissionMetadata, PickOptional, PickRequired, PikkuAIMiddlewareHooks, PikkuWire, PikkuRawWire, PikkuWiringTypes, PostgresConfig, RequireAtLeastOne, SecurityAuditIssue, SecurityAuditReport, SecurityAuditSummary, SecurityAuditUpdate, SecuritySeverity, SecurityUpdateLevel, SerializedError, } from './types/core.types.js';
 export { pikkuAIMiddleware, pikkuChannelMiddleware, pikkuChannelMiddlewareFactory, pikkuMiddleware, pikkuMiddlewareFactory, } from './types/core.types.js';
-export type { CorePikkuAuth, CorePikkuAuthConfig, CorePikkuFunction, CorePikkuFunctionConfig, CorePikkuFunctionHook, CorePikkuPermission, CorePikkuPermissionConfig, CorePikkuPermissionFactory, CorePikkuApprovalDescription, CorePermissionGroup, } from './function/functions.types.js';
+export type { CorePikkuAuth, CorePikkuAuthConfig, CorePikkuFunction, CorePikkuFunctionConfig, CorePikkuPermission, CorePikkuPermissionConfig, CorePikkuPermissionFactory, CorePikkuApprovalDescription, CorePermissionGroup, } from './function/functions.types.js';
 export { pikkuAuth, pikkuPermission, pikkuPermissionFactory, pikkuApprovalDescription, } from './function/functions.types.js';
 export { getAllFunctionNames } from './function/index.js';
-export type { ListInput, ListOutput, Filter, LeafFilter, LeafValue, } from './function/list.types.js';
+export type { ListInput, ListOutput, Filter, } from './function/list.types.js';
 export { pikkuCLIRender } from './wirings/cli/cli-runner.js';
 export { PikkuRequest } from './pikku-request.js';
 export { getRelativeTimeOffsetFromNow, parseDurationString, } from './time-utils.js';
 export type { RelativeTimeInput } from './time-utils.js';
 export { formatVersionedId, isVersionedId, parseVersionedId, } from './version.js';
-export { AbandonedError, beginChanges, getAbortScope, runInAbortScope, type AbortScope, } from './function/abort-scope.js';
+export { AbandonedError, type AbortScope, } from './function/abort-scope.js';
 export { fetch } from './wirings/http/http-runner.js';
 export type { MCPToolResponse, MCPResourceResponse, MCPPromptResponse, } from './wirings/mcp/mcp.types.js';
 export { AIProviderAuthError, AIProviderNotConfiguredError, BadGatewayError, BadRequestError, ConflictError, ExpectationFailedError, ForbiddenError, GatewayTimeoutError, GoneError, HTTPVersionNotSupportedError, InternalServerError, InvalidMiddlewareWireError, InvalidOriginError, InvalidSessionError, LengthRequiredError, LocalEnvironmentOnlyError, LockedError, MaxComputeTimeReachedError, MethodNotAllowedError, MissingCredentialError, MissingSchemaError, MissingScopeError, MissingServiceError, MissingSessionError, NotAcceptableError, NotFoundError, NotImplementedError, PayloadTooLargeError, PaymentRequiredError, PikkuMissingMetaError, PreconditionFailedError, ProxyAuthenticationRequiredError, RangeNotSatisfiableError, ReadonlySessionError, RequestTimeoutError, ServiceUnavailableError, TooManyRequestsError, URITooLongError, UnauthorizedError, UnprocessableContentError, UnsupportedMediaTypeError, } from './errors/errors.js';
@@ -20,7 +20,7 @@ export { PikkuError, isExpectedError } from './errors/error-handler.js';
 export type { EventHubService } from './wirings/channel/eventhub-service.js';
 export type { QueueService } from './wirings/queue/queue.types.js';
 export type { JWTService } from './services/jwt-service.js';
-export type { EmailService, EmailTemplateReference, SendEmailInput, SendEmailResult, SendHTMLEmailInput, SendTemplateEmailInput, SendTextEmailInput, } from './services/email-service.js';
+export type { EmailService, SendEmailInput, SendEmailResult, SendHTMLEmailInput, SendTemplateEmailInput, SendTextEmailInput, } from './services/email-service.js';
 export type { SecretService } from './services/secret-service.js';
 export { SecretAccessDeniedError, withoutSecrets, } from './services/secretless.js';
 export { SecretHostNotAllowedError, assertSecretAllowedForHost, } from './services/secret-host-binding.js';
@@ -32,13 +32,13 @@ export type { GatewayService } from './services/gateway-service.js';
 export type { TriggerService } from './services/trigger-service.js';
 export type { SchemaService } from './services/schema-service.js';
 export type { SessionService } from './services/user-session-service.js';
-export { NoopAuditService, createInvocationAudit, resolveAuditConfig, resolveAuditUserIdentityFromWire, } from './services/audit-service.js';
-export type { AuditConfig, AuditDurability, AuditEvent, AuditEventBatch, AuditFacets, AuditLog, AuditLogWriteInput, AuditOutcome, AuditQuery, AuditQueryResult, AuditService, AuditSource, AuditUserIdentity, ResolvedAuditConfig, } from './services/audit-service.js';
-export type { AIAgentRunnerService, AIEmbedManyParams, AIEmbedManyResult, AIEmbedParams, AIEmbedResult, AIGenerateImageParams, AIGenerateImagePrompt, AIGenerateImageResult, AIGenerateSpeechParams, AIGenerateSpeechResult, AIProviderOptions, AIRerankParams, AIRerankResult, AITranscriptionParams, AITranscriptionResult, } from './services/ai-agent-runner-service.js';
+export { NoopAuditService, createInvocationAudit, } from './services/audit-service.js';
+export type { AuditConfig, AuditDurability, AuditEvent, AuditEventBatch, AuditFacets, AuditLog, AuditQuery, AuditQueryResult, AuditService, AuditUserIdentity, ResolvedAuditConfig, } from './services/audit-service.js';
+export type { AIAgentRunnerService, } from './services/ai-agent-runner-service.js';
 export type { AIEmbeddingService } from './services/ai-embedding-service.js';
 export type { AIRunStateService } from './services/ai-run-state-service.js';
 export type { AIStorageService } from './services/ai-storage-service.js';
-export type { EmailsMeta, EmailTemplateMeta, EmailTemplateLocaleMeta, EmailTemplateAssets, MetaService, } from './services/meta-service.js';
+export type { EmailsMeta, EmailTemplateMeta, MetaService, } from './services/meta-service.js';
 export type { HTTPMethod } from './wirings/http/http.types.js';
 export type { GraphNodeConfig } from './wirings/workflow/graph/workflow-graph.types.js';
 export { createGraph } from './wirings/workflow/graph/graph-node.js';
@@ -54,7 +54,7 @@ export { isSerializable, stopSingletonServices, pikkuServerLifecycle, } from './
 export { clearPikkuRuntimeState } from './test-utils.js';
 export { type ScheduledTaskInfo, type ScheduledTaskSummary, } from './services/scheduler-service.js';
 export { SchedulerService } from './services/scheduler-service.js';
-export type { Private, Pii, Secret, Classification, AnonymizeStrategy, ColumnClassification, ClassificationManifest, ColumnForm, WrappedValue, SealedValue, HashedValue, } from './data-classification.js';
-export { hashToken, unsafeAsWrapped, unsafeAsSealed, unsafeAsHashed, } from './column-form.js';
+export type { Private, Pii, Secret, Classification, AnonymizeStrategy, ClassificationManifest, ColumnForm, WrappedValue, SealedValue, HashedValue, } from './data-classification.js';
+export { hashToken, } from './column-form.js';
 export type { SecretValue, Safe } from './secret-value.js';
 export { createSecretValue, isSecretValue, SecretCoercionError, REDACTED, } from './secret-value.js';

package/dist/index.js

@@ -5,13 +5,13 @@ export { pikkuCLIRender } from './wirings/cli/cli-runner.js';
 export { PikkuRequest } from './pikku-request.js';
 export { getRelativeTimeOffsetFromNow, parseDurationString, } from './time-utils.js';
 export { formatVersionedId, isVersionedId, parseVersionedId, } from './version.js';
-export { AbandonedError, beginChanges, getAbortScope, runInAbortScope, } from './function/abort-scope.js';
+export { AbandonedError, } from './function/abort-scope.js';
 export { fetch } from './wirings/http/http-runner.js';
 export { AIProviderAuthError, AIProviderNotConfiguredError, BadGatewayError, BadRequestError, ConflictError, ExpectationFailedError, ForbiddenError, GatewayTimeoutError, GoneError, HTTPVersionNotSupportedError, InternalServerError, InvalidMiddlewareWireError, InvalidOriginError, InvalidSessionError, LengthRequiredError, LocalEnvironmentOnlyError, LockedError, MaxComputeTimeReachedError, MethodNotAllowedError, MissingCredentialError, MissingSchemaError, MissingScopeError, MissingServiceError, MissingSessionError, NotAcceptableError, NotFoundError, NotImplementedError, PayloadTooLargeError, PaymentRequiredError, PikkuMissingMetaError, PreconditionFailedError, ProxyAuthenticationRequiredError, RangeNotSatisfiableError, ReadonlySessionError, RequestTimeoutError, ServiceUnavailableError, TooManyRequestsError, URITooLongError, UnauthorizedError, UnprocessableContentError, UnsupportedMediaTypeError, } from './errors/errors.js';
 export { PikkuError, isExpectedError } from './errors/error-handler.js';
 export { SecretAccessDeniedError, withoutSecrets, } from './services/secretless.js';
 export { SecretHostNotAllowedError, assertSecretAllowedForHost, } from './services/secret-host-binding.js';
-export { NoopAuditService, createInvocationAudit, resolveAuditConfig, resolveAuditUserIdentityFromWire, } from './services/audit-service.js';
+export { NoopAuditService, createInvocationAudit, } from './services/audit-service.js';
 export { createGraph } from './wirings/workflow/graph/graph-node.js';
 export { wireAddon } from './wirings/rpc/wire-addon.js';
 export { wireRemoteAddon } from './wirings/rpc/wire-remote-addon.js';
@@ -21,5 +21,5 @@ export { hasScopes, verifyScopes } from './scopes.js';
 export { isSerializable, stopSingletonServices, pikkuServerLifecycle, } from './utils.js';
 export { clearPikkuRuntimeState } from './test-utils.js';
 export { SchedulerService } from './services/scheduler-service.js';
-export { hashToken, unsafeAsWrapped, unsafeAsSealed, unsafeAsHashed, } from './column-form.js';
+export { hashToken, } from './column-form.js';
 export { createSecretValue, isSecretValue, SecretCoercionError, REDACTED, } from './secret-value.js';

package/dist/pikku-state.js

@@ -84,6 +84,9 @@ const createEmptyPackageState = () => ({
     agent: {
         agents: new Map(),
         agentsMeta: {},
+        scorers: new Map(),
+        scorersMeta: {},
+        modelAliases: {},
     },
     gateway: {
         gateways: new Map(),
@@ -117,6 +120,7 @@ const createEmptyPackageState = () => ({
         authFactory: null,
         credentialsMeta: null,
         requiredParentServices: null,
+        declaredSecrets: null,
     },
 });
 export const initializePikkuState = (packageName) => {

package/dist/services/ai-agent-runner-service.d.ts

@@ -54,6 +54,13 @@ export type AIAgentStepResult = {
         toolCallId: string;
         toolName: string;
         result: unknown;
+        /**
+         * Set when the tool threw rather than returned. Carried as its own field
+         * because `result` is a rendered string by the time anything downstream
+         * reads it, and "did this tool fail" is not answerable by looking for an
+         * `Error:` prefix in text a tool could legitimately have returned.
+         */
+        error?: string;
     }[];
     usage: {
         inputTokens: number;

package/dist/services/ai-run-state-service.d.ts

@@ -1,5 +1,7 @@
 import type { AgentRunState, PendingApproval } from '../wirings/ai-agent/ai-agent.types.js';
+import type { AIRunScore } from '../wirings/ai-scorer/ai-scorer.types.js';
 export type CreateRunInput = Omit<AgentRunState, 'runId'>;
+export type SaveScoreInput = Omit<AIRunScore, 'createdAt'>;
 export interface AIRunStateService {
     createRun(run: CreateRunInput): Promise<string>;
     updateRun(runId: string, updates: Partial<AgentRunState>): Promise<void>;
@@ -16,4 +18,12 @@ export interface AIRunStateService {
         run: AgentRunState;
         approval: PendingApproval;
     } | null>;
+    /**
+     * Record one scorer's grade of a finished run. A run accumulates one row per
+     * scorer, and a scorer may grade the same run more than once — a retried job
+     * appends rather than replaces, so a re-grade never silently overwrites the
+     * grade that was acted on.
+     */
+    saveScore(score: SaveScoreInput): Promise<void>;
+    getScores(runId: string): Promise<AIRunScore[]>;
 }

package/dist/services/in-memory-ai-run-state-service.d.ts

@@ -1,7 +1,9 @@
-import type { AIRunStateService, CreateRunInput } from './ai-run-state-service.js';
+import type { AIRunStateService, CreateRunInput, SaveScoreInput } from './ai-run-state-service.js';
 import type { AgentRunState, PendingApproval } from '../wirings/ai-agent/ai-agent.types.js';
+import type { AIRunScore } from '../wirings/ai-scorer/ai-scorer.types.js';
 export declare class InMemoryAIRunStateService implements AIRunStateService {
     private runs;
+    private scores;
     private counter;
     createRun(run: CreateRunInput): Promise<string>;
     updateRun(runId: string, updates: Partial<AgentRunState>): Promise<void>;
@@ -12,4 +14,6 @@ export declare class InMemoryAIRunStateService implements AIRunStateService {
         run: AgentRunState;
         approval: PendingApproval;
     } | null>;
+    saveScore(score: SaveScoreInput): Promise<void>;
+    getScores(runId: string): Promise<AIRunScore[]>;
 }

package/dist/services/in-memory-ai-run-state-service.js

@@ -1,5 +1,6 @@
 export class InMemoryAIRunStateService {
     runs = new Map();
+    scores = new Map();
     counter = 0;
     async createRun(run) {
         const runId = `run-${++this.counter}-${Date.now()}`;
@@ -42,4 +43,12 @@ export class InMemoryAIRunStateService {
         }
         return null;
     }
+    async saveScore(score) {
+        const existing = this.scores.get(score.runId) ?? [];
+        existing.push({ ...score, createdAt: new Date() });
+        this.scores.set(score.runId, existing);
+    }
+    async getScores(runId) {
+        return [...(this.scores.get(runId) ?? [])];
+    }
 }

package/dist/services/index.d.ts

@@ -1,8 +1,9 @@
 export { LogLevel } from './logger.js';
 export { ScopedSecretService } from './scoped-secret-service.js';
-export { PikkuSessionService, createMiddlewareSessionWireProps, createFunctionSessionWireProps, } from './user-session-service.js';
+export { ScopedCredentialService } from './scoped-credential-service.js';
+export { PikkuSessionService, createMiddlewareSessionWireProps, } from './user-session-service.js';
 export { TypedSecretService } from './typed-secret-service.js';
-export { PikkuCredentialWireService, createMiddlewareCredentialWireProps, createWireServicesCredentialWireProps, } from './credential-wire-service.js';
+export { PikkuCredentialWireService, } from './credential-wire-service.js';
 export { TypedVariablesService } from './typed-variables-service.js';
 export { LocalSecretService } from './local-secrets.js';
 export { LocalEmailService } from './local-email-service.js';
@@ -16,11 +17,11 @@ export { InMemoryTriggerService } from './in-memory-trigger-service.js';
 export { InMemoryAIRunStateService } from './in-memory-ai-run-state-service.js';
 export { LocalGatewayService } from './local-gateway-service.js';
 export type { ContentService, SignContentKeyArgs, SignURLArgs, GetUploadURLArgs, UploadURLResult, BucketKeyArgs, WriteFileArgs, CopyFileArgs, } from './content-service.js';
-export type { ScenarioPersona, ResolvedPersona, ScenarioPersonaOf, ScenarioPersonas, ScenarioInvokeOptions, ScenarioRpcMap, ScenarioHttpResponse, } from './personas-service.js';
+export type { ScenarioPersona, ResolvedPersona, ScenarioPersonas, ScenarioHttpResponse, } from './personas-service.js';
 export type { HttpPersonasConfig } from './http-personas.js';
 export type { JWTService } from './jwt-service.js';
-export type { EmailService, EmailTemplateReference, SendEmailInput, SendEmailResult, SendHTMLEmailInput, SendTemplateEmailInput, SendTextEmailInput, } from './email-service.js';
-export { DEFAULT_WEBHOOK_RETRIES, DEFAULT_WEBHOOK_SIGNATURE_HEADER, PIKKU_OUTGOING_WEBHOOK_QUEUE_NAME, WebhookService, type SendWebhookInput, type SendWebhookResult, type WebhookAttemptRecord, type WebhookAttemptResult, type WebhookDeliveryRecord, type WebhookDeliveryWithAttempts, type WebhookJobData, type WebhookServiceConfig, } from './webhook-service.js';
+export type { EmailService, SendEmailInput, SendEmailResult, SendHTMLEmailInput, SendTemplateEmailInput, SendTextEmailInput, } from './email-service.js';
+export { DEFAULT_WEBHOOK_RETRIES, PIKKU_OUTGOING_WEBHOOK_QUEUE_NAME, WebhookService, type SendWebhookInput, type SendWebhookResult, type WebhookAttemptResult, type WebhookDeliveryRecord, type WebhookDeliveryWithAttempts, type WebhookJobData, type WebhookServiceConfig, } from './webhook-service.js';
 export type { Logger } from './logger.js';
 export type { SecretService, SecretValues } from './secret-service.js';
 export type { VariablesService } from './variables-service.js';
@@ -31,23 +32,22 @@ export type { TriggerService } from './trigger-service.js';
 export type { GatewayService } from './gateway-service.js';
 export type { DeploymentService, DeploymentConfig, DeploymentInfo, DeploymentServiceConfig, } from './deployment-service.js';
 export type { AIStorageService } from './ai-storage-service.js';
-export type { AIAgentRunnerParams, AIAgentRunnerResult, AIAgentStepResult, AIEmbedManyParams, AIEmbedManyResult, AIEmbedParams, AIEmbedResult, AIGenerateImageParams, AIGenerateImagePrompt, AIGenerateImageResult, AIGenerateSpeechParams, AIGenerateSpeechResult, AIProviderOptions, AIRerankParams, AIRerankResult, AITranscriptionParams, AITranscriptionResult, AIAgentRunnerService, } from './ai-agent-runner-service.js';
+export type { AIAgentRunnerParams, AIAgentRunnerResult, AIAgentStepResult, AIAgentRunnerService, } from './ai-agent-runner-service.js';
 export type { AIEmbeddingService } from './ai-embedding-service.js';
-export type { CreateRunInput, AIRunStateService, } from './ai-run-state-service.js';
-export type { CredentialStatus, CredentialMeta, } from './typed-secret-service.js';
+export type { CreateRunInput, SaveScoreInput, AIRunStateService, } from './ai-run-state-service.js';
+export type { CredentialMeta, } from './typed-secret-service.js';
 export type { CredentialService } from './credential-service.js';
 export { TypedCredentialService } from './typed-credential-service.js';
-export type { CredentialStatusInfo, CredentialMetaInfo, } from './typed-credential-service.js';
-export type { VariableStatus, VariableMeta } from './typed-variables-service.js';
+export type { CredentialMetaInfo, } from './typed-credential-service.js';
+export type { VariableMeta } from './typed-variables-service.js';
 export type { MetaService } from './meta-service.js';
 export type { SessionStore } from './session-store.js';
 export type { ScopeService, Role } from './scope-service.js';
-export { assertRoleIsMutable, assertRoleNameAvailable, roleLockReason, } from './system-role-guard.js';
 export type { IsSystemRole } from './system-role-guard.js';
-export { NoopAuditService, createInvocationAudit, resolveAuditConfig, resolveAuditUserIdentityFromWire, } from './audit-service.js';
-export type { AuditConfig, AuditDurability, AuditEvent, AuditEventBatch, AuditLog, AuditLogWriteInput, AuditOutcome, AuditService, AuditSource, AuditUserIdentity, ResolvedAuditConfig, } from './audit-service.js';
+export { NoopAuditService, createInvocationAudit, } from './audit-service.js';
+export type { AuditConfig, AuditDurability, AuditEvent, AuditEventBatch, AuditLog, AuditService, AuditUserIdentity, ResolvedAuditConfig, } from './audit-service.js';
 export { InMemorySessionStore } from './in-memory-session-store.js';
-export type { MCPMeta, RPCMetaRecord, ServiceMeta, ServicesMetaRecord, MiddlewareDefinitionMeta, MiddlewareInstanceMeta, GroupMeta, MiddlewareGroupsMeta, PermissionDefinitionMeta, PermissionsGroupsMeta, FunctionsMeta, FunctionMeta, MiddlewareMeta, PermissionMeta, AgentsMeta, AgentMeta, EmailsMeta, EmailTemplateMeta, EmailTemplateLocaleMeta, EmailTemplateAssets, } from './meta-service.js';
-export type { CoverageService, CoverageSnapshot, LineHits, ScriptCoverage, FunctionCoverage, CoverageRange, CoverageStatus, FunctionCoverageEntry, FunctionCoverageReport, CoverageFunctionMeta, } from './v8-coverage-service.js';
+export type { MCPMeta, RPCMetaRecord, ServiceMeta, ServicesMetaRecord, MiddlewareDefinitionMeta, MiddlewareInstanceMeta, GroupMeta, MiddlewareGroupsMeta, PermissionDefinitionMeta, PermissionsGroupsMeta, FunctionsMeta, FunctionMeta, MiddlewareMeta, PermissionMeta, AgentsMeta, AgentMeta, EmailsMeta, EmailTemplateMeta, } from './meta-service.js';
+export type { CoverageService, ScriptCoverage, FunctionCoverageEntry, FunctionCoverageReport, CoverageFunctionMeta, } from './v8-coverage-service.js';
 export { StubTracker, createStubProxy, getStubTracker, isTestRun, stub, spy, type StubCall, } from './stub-tracker.js';
 export { SecretHostNotAllowedError, assertSecretAllowedForHost, } from './secret-host-binding.js';