@pikku/core 0.12.98 → 0.12.100
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +75 -0
- package/dist/services/http-personas.d.ts +16 -5
- package/dist/services/http-personas.js +7 -3
- package/dist/services/index.d.ts +1 -0
- package/dist/services/index.js +1 -0
- package/dist/services/persona-actor-secret.d.ts +38 -0
- package/dist/services/persona-actor-secret.js +39 -0
- package/dist/services/persona-sign-in.d.ts +23 -1
- package/dist/services/persona-sign-in.js +25 -1
- package/dist/services/typed-secret-service.js +4 -1
- package/dist/wirings/agent-scorer/agent-scorer.d.ts +14 -0
- package/dist/wirings/gateway/gateway.types.d.ts +13 -0
- package/dist/wirings/persona/index.d.ts +2 -1
- package/dist/wirings/persona/index.js +2 -1
- package/dist/wirings/secret/secret.types.d.ts +8 -0
- package/knowledge/decisions/internals/a-virtual-user-cadence-is-a-row-not-a-timer.md +1 -1
- package/knowledge/decisions/internals/a-virtual-user-run-is-not-a-workflow-but-it-needs-a-trigger.md +65 -0
- package/knowledge/decisions/internals/index.md +1 -1
- package/knowledge/decisions/security/actor-sign-in-only-works-for-actor-flagged-users.md +19 -15
- package/knowledge/decisions/security/an-actor-credential-is-derived-per-persona.md +41 -0
- package/knowledge/decisions/security/index.md +2 -1
- package/package.json +4 -4
- package/src/public-surface.json +13 -0
- package/src/services/http-personas-converse.test.ts +3 -3
- package/src/services/http-personas.test.ts +92 -5
- package/src/services/http-personas.ts +27 -6
- package/src/services/index.ts +8 -0
- package/src/services/persona-actor-secret.test.ts +68 -0
- package/src/services/persona-actor-secret.ts +70 -0
- package/src/services/persona-sign-in.ts +36 -2
- package/src/services/typed-secret-service.test.ts +26 -1
- package/src/services/typed-secret-service.ts +4 -1
- package/src/wirings/agent-scorer/agent-scorer.ts +14 -0
- package/src/wirings/gateway/gateway.types.ts +20 -1
- package/src/wirings/persona/index.ts +10 -0
- package/src/wirings/secret/secret.types.ts +8 -0
- package/tsconfig.tsbuildinfo +1 -1
- package/knowledge/decisions/internals/a-virtual-user-run-is-not-a-workflow-and-not-a-queued-job.md +0 -53
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,78 @@
|
|
|
1
|
+
## 0.12.100
|
|
2
|
+
|
|
3
|
+
### Patch Changes
|
|
4
|
+
|
|
5
|
+
- a0ed1e8: Derive a persona's session and operator paths from the mount its sign-in path names.
|
|
6
|
+
|
|
7
|
+
Both `sessionRoles()` and the Fabric operator handshake asked for a fixed
|
|
8
|
+
`/auth/…` no matter where auth was mounted. An app serving better-auth under
|
|
9
|
+
`/api/auth` while keeping its RPCs at the root cannot put the mount in
|
|
10
|
+
`apiUrl`, so it moves `signInPath` — and the other two stayed behind.
|
|
11
|
+
|
|
12
|
+
For the session read that meant a 404, which returns `null`, which means "this
|
|
13
|
+
stage does not report roles": every `pikku persona run` on such an app warned
|
|
14
|
+
"running unverified" and lost the one thing that tells a permissions finding
|
|
15
|
+
from seed drift. For the operator handshake it was worse — `HttpPersona`
|
|
16
|
+
reused the _actor_ path verbatim, so an operator token was posted to the actor
|
|
17
|
+
endpoint and came back as a validation error about a missing email and secret,
|
|
18
|
+
which reads like a broken persona rather than a wrong URL. The browser provider
|
|
19
|
+
had the same fixed default.
|
|
20
|
+
|
|
21
|
+
All three now follow `signInPath`, and `environments[].sessionPath` in
|
|
22
|
+
pikku.config.json overrides the session read for a stage that reports it
|
|
23
|
+
elsewhere.
|
|
24
|
+
|
|
25
|
+
## 0.12.99
|
|
26
|
+
|
|
27
|
+
### Patch Changes
|
|
28
|
+
|
|
29
|
+
- ee9da9e: Reading an optional secret that is not set no longer makes `hasSecret` report it as set. `TypedSecretService` caches `undefined` to remember the absence, and the cache probe read that as a value.
|
|
30
|
+
- 7a15c9c: An actor credential is one persona's, not everyone's
|
|
31
|
+
|
|
32
|
+
`SCENARIO_ACTOR_SECRET` was a skeleton key. Anyone holding it could post any
|
|
33
|
+
`actor: true` address to `/auth/sign-in/actor` and get that persona's session —
|
|
34
|
+
including the `admin` persona, which provisioning grants real admin. The browser
|
|
35
|
+
switcher held it too, baked into the dev bundle as `VITE_SCENARIO_ACTOR_SECRET`,
|
|
36
|
+
so "the reviewer can sign in as each kind of user" and "the reviewer's bundle is
|
|
37
|
+
entitled to every persona" were the same fact.
|
|
38
|
+
|
|
39
|
+
It is now a root that credentials derive from, never one that is presented:
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
deriveActorSecret(root, email) // HKDF-expanded HMAC-SHA256 over the address
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
The endpoint re-derives the expected value for whichever address is signing in
|
|
46
|
+
and compares, so nothing is stored or looked up, a credential minted for one
|
|
47
|
+
persona is refused for every other, and rotating the root invalidates all of
|
|
48
|
+
them at once. The root itself is no longer a valid credential, and a root under
|
|
49
|
+
32 characters refuses the endpoint rather than deriving weak credentials from
|
|
50
|
+
it — the server log says why, the client is not told.
|
|
51
|
+
|
|
52
|
+
What that buys, in the places that used to need the whole key:
|
|
53
|
+
|
|
54
|
+
- **`pikku dev`** mints one credential per declared persona into
|
|
55
|
+
`VITE_DEV_ACTOR_SECRETS` and no longer writes `VITE_SCENARIO_ACTOR_SECRET` at
|
|
56
|
+
all. The root stays on the server.
|
|
57
|
+
- **`pikku persona secret <id>`** mints them for anything else, and a run given
|
|
58
|
+
`PIKKU_PERSONA_SECRETS=id=secret,…` can sign in as those personas and no
|
|
59
|
+
others — asking for one outside the list throws naming the persona instead of
|
|
60
|
+
falling back to the root.
|
|
61
|
+
|
|
62
|
+
`useDevActors()` and `<DevActorSwitcher />` take `secrets` (one per address)
|
|
63
|
+
where they took `secret`, and an actor with no credential is no longer offered
|
|
64
|
+
rather than rendering a row that 401s. `HttpPersonasConfig.secret` and the
|
|
65
|
+
Playwright provider's `secret` additionally accept a resolver, which is how a
|
|
66
|
+
partially-credentialled run is expressed.
|
|
67
|
+
|
|
68
|
+
- ee9da9e: the surface gate measures the surface it actually ships
|
|
69
|
+
|
|
70
|
+
The doc-quality gate went in with ceilings of 112, 823 and 10 beside a surface
|
|
71
|
+
that measured 160, 1210 and 15, so it never passed on any build. Re-baselined to
|
|
72
|
+
the real measurements, and the key-documentation floor earned its way from 76%
|
|
73
|
+
to 79% by documenting what a caller has to put in `defineSecret`, the gateway
|
|
74
|
+
message shapes, and the scorer and judge configs.
|
|
75
|
+
|
|
1
76
|
## 0.12.98
|
|
2
77
|
|
|
3
78
|
### Patch Changes
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { ScenarioPersona, ResolvedPersona, ScenarioPersonas, ScenarioInvokeOptions, ScenarioHttpResponse } from './personas-service.js';
|
|
2
2
|
import type { ConverseOptions, ActorFlowVerdict } from '../wirings/actor-flow/actor-flow.types.js';
|
|
3
|
-
import { type OperatorSignInOptions } from './persona-sign-in.js';
|
|
3
|
+
import { type ActorSecretResolver, type OperatorSignInOptions } from './persona-sign-in.js';
|
|
4
4
|
export interface HttpPersonasConfig {
|
|
5
5
|
/**
|
|
6
6
|
* Base API URL of the target app, INCLUDING the HTTP prefix — e.g.
|
|
@@ -10,13 +10,19 @@ export interface HttpPersonasConfig {
|
|
|
10
10
|
*/
|
|
11
11
|
apiUrl: string;
|
|
12
12
|
/**
|
|
13
|
-
* The
|
|
14
|
-
*
|
|
13
|
+
* The ROOT actor secret, from which each persona's own credential is derived
|
|
14
|
+
* and bound to their address. Sign-in only ever works for user rows flagged
|
|
15
|
+
* `actor: true`, and a derived credential only ever works for the one address
|
|
16
|
+
* it was derived for.
|
|
17
|
+
*
|
|
18
|
+
* Pass an {@link ActorSecretResolver} instead to drive personas whose
|
|
19
|
+
* credentials were minted elsewhere — a caller entitled to one persona then
|
|
20
|
+
* never holds the root.
|
|
15
21
|
*
|
|
16
22
|
* The local-development credential. A deployed stage has none, and passes
|
|
17
23
|
* {@link HttpPersonasConfig.operator} instead.
|
|
18
24
|
*/
|
|
19
|
-
secret?: string;
|
|
25
|
+
secret?: string | ActorSecretResolver;
|
|
20
26
|
/**
|
|
21
27
|
* Fabric operator credentials, for signing personas into a DEPLOYED stage.
|
|
22
28
|
*
|
|
@@ -34,7 +40,12 @@ export interface HttpPersonasConfig {
|
|
|
34
40
|
* {@link OperatorSignInOptions.signInPath} overrides it.
|
|
35
41
|
*/
|
|
36
42
|
signInPath?: string;
|
|
37
|
-
/**
|
|
43
|
+
/**
|
|
44
|
+
* Where the session (and its roles) is read back. Defaults to `get-session`
|
|
45
|
+
* under the same auth mount as {@link HttpPersonasConfig.signInPath}, so an
|
|
46
|
+
* app that moved auth under `/api` moves this with it and does not have to
|
|
47
|
+
* say so twice.
|
|
48
|
+
*/
|
|
38
49
|
sessionPath?: string;
|
|
39
50
|
/** Exposed-RPC path prefix under apiUrl. Default `/rpc`. */
|
|
40
51
|
rpcPath?: string;
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { readScenarioHttpResponse } from './personas-service.js';
|
|
2
2
|
import { runConversation } from '../wirings/actor-flow/run-conversation.js';
|
|
3
3
|
import { createCookieJar, } from '../wirings/workflow/scenario-cookie-jar.js';
|
|
4
|
-
import { ActorSignIn, OperatorSignIn, } from './persona-sign-in.js';
|
|
4
|
+
import { ActorSignIn, OperatorSignIn, authMount, } from './persona-sign-in.js';
|
|
5
5
|
import { getSingletonServices } from '../pikku-state.js';
|
|
6
6
|
import { AIProviderNotConfiguredError } from '../errors/errors.js';
|
|
7
7
|
/**
|
|
@@ -34,7 +34,10 @@ export class HttpPersona {
|
|
|
34
34
|
if (config.operator) {
|
|
35
35
|
this.signIn = new OperatorSignIn(config.apiUrl, {
|
|
36
36
|
...config.operator,
|
|
37
|
-
signInPath: config.operator.signInPath ??
|
|
37
|
+
signInPath: config.operator.signInPath ??
|
|
38
|
+
(authMount(config.signInPath)
|
|
39
|
+
? `${authMount(config.signInPath)}/sign-in/fabric`
|
|
40
|
+
: undefined),
|
|
38
41
|
});
|
|
39
42
|
}
|
|
40
43
|
else if (config.secret) {
|
|
@@ -115,7 +118,8 @@ export class HttpPersona {
|
|
|
115
118
|
if (!this.signedIn) {
|
|
116
119
|
await this.login();
|
|
117
120
|
}
|
|
118
|
-
const
|
|
121
|
+
const mount = authMount(this.config.operator?.signInPath ?? this.config.signInPath);
|
|
122
|
+
const sessionPath = this.config.sessionPath ?? `${mount ?? '/auth'}/get-session`;
|
|
119
123
|
const res = await this.jar.fetch(`${this.config.apiUrl}${sessionPath}`, {
|
|
120
124
|
headers: this.signIn.headers(),
|
|
121
125
|
});
|
package/dist/services/index.d.ts
CHANGED
|
@@ -23,6 +23,7 @@ export type { JWTService } from './jwt-service.js';
|
|
|
23
23
|
export type { EmailService, SendEmailInput, SendEmailResult, SendHTMLEmailInput, SendTemplateEmailInput, SendTextEmailInput, } from './email-service.js';
|
|
24
24
|
export { renderEmail, type EmailAssets, type EmailTemplateAssets, type EmailTemplateHashes, type RenderEmailRequest, type RenderedEmailResult, } from './email-template.js';
|
|
25
25
|
export { DEFAULT_WEBHOOK_RETRIES, PIKKU_OUTGOING_WEBHOOK_QUEUE_NAME, WebhookService, type SendWebhookInput, type SendWebhookResult, type WebhookAttemptResult, type WebhookDeliveryRecord, type WebhookDeliveryWithAttempts, type WebhookJobData, type WebhookServiceConfig, } from './webhook-service.js';
|
|
26
|
+
export { ACTOR_ROOT_SECRET_MIN_LENGTH, ACTOR_SECRET_INFO, ACTOR_SECRET_NAME, actorSecretSubject, deriveActorSecret, verifyActorSecret, } from './persona-actor-secret.js';
|
|
26
27
|
export type { Logger } from './logger.js';
|
|
27
28
|
export type { SecretService, SecretValues } from './secret-service.js';
|
|
28
29
|
export type { VariablesService } from './variables-service.js';
|
package/dist/services/index.js
CHANGED
|
@@ -19,6 +19,7 @@ export { LocalGatewayService } from './local-gateway-service.js';
|
|
|
19
19
|
export { FileScenarioRunStore, scenarioArtifactContentType, scenarioRunSummary, } from './file-scenario-run-store.js';
|
|
20
20
|
export { renderEmail, } from './email-template.js';
|
|
21
21
|
export { DEFAULT_WEBHOOK_RETRIES, PIKKU_OUTGOING_WEBHOOK_QUEUE_NAME, WebhookService, } from './webhook-service.js';
|
|
22
|
+
export { ACTOR_ROOT_SECRET_MIN_LENGTH, ACTOR_SECRET_INFO, ACTOR_SECRET_NAME, actorSecretSubject, deriveActorSecret, verifyActorSecret, } from './persona-actor-secret.js';
|
|
22
23
|
export { SchedulerService } from './scheduler-service.js';
|
|
23
24
|
export { TypedCredentialService } from './typed-credential-service.js';
|
|
24
25
|
export { NoopAuditService, createInvocationAudit } from './audit-service.js';
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
/** The name the root secret is held under, used only in error messages. */
|
|
2
|
+
export declare const ACTOR_SECRET_NAME = "SCENARIO_ACTOR_SECRET";
|
|
3
|
+
/**
|
|
4
|
+
* Namespaces the derivation so the same root secret used for anything else
|
|
5
|
+
* produces different values. See knowledge/crypto.md.
|
|
6
|
+
*/
|
|
7
|
+
export declare const ACTOR_SECRET_INFO = "pikku:actor-sign-in";
|
|
8
|
+
/** The root must be strong: every persona's credential is derived from it. */
|
|
9
|
+
export declare const ACTOR_ROOT_SECRET_MIN_LENGTH = 32;
|
|
10
|
+
/**
|
|
11
|
+
* What the derivation is bound to. Lowercased because the sign-in endpoint
|
|
12
|
+
* looks the user up by lowercased address, and a credential that verified
|
|
13
|
+
* against a different string than the row it opens is a credential for nothing.
|
|
14
|
+
*/
|
|
15
|
+
export declare const actorSecretSubject: (email: string) => string;
|
|
16
|
+
/**
|
|
17
|
+
* One persona's actor credential: `HMAC-SHA256(root, email)`, base64url.
|
|
18
|
+
*
|
|
19
|
+
* The root secret is not itself a valid credential and never travels: what a
|
|
20
|
+
* scenario run, a CI job or a virtual user is handed is the derived value for
|
|
21
|
+
* the one address it is entitled to. Presenting it for any other address fails,
|
|
22
|
+
* so a leaked credential is worth exactly one synthetic account rather than the
|
|
23
|
+
* whole actor population.
|
|
24
|
+
*
|
|
25
|
+
* Deterministic, so nothing is stored and nothing is provisioned — the target
|
|
26
|
+
* re-derives the expected value from the address being signed in as. Rotating
|
|
27
|
+
* the root invalidates every derived credential at once, which is the property
|
|
28
|
+
* a per-persona secret table would have to implement by hand.
|
|
29
|
+
*/
|
|
30
|
+
export declare const deriveActorSecret: (rootSecret: string, email: string) => Promise<string>;
|
|
31
|
+
/**
|
|
32
|
+
* Whether `presented` is the credential for `email` under `rootSecret`.
|
|
33
|
+
*
|
|
34
|
+
* False — never throws — for a malformed, truncated or mismatched value, and
|
|
35
|
+
* the comparison is WebCrypto's own HMAC verify, so it does not exit early on
|
|
36
|
+
* the first differing byte.
|
|
37
|
+
*/
|
|
38
|
+
export declare const verifyActorSecret: (rootSecret: string, email: string, presented: string) => Promise<boolean>;
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
import { MIN_KEY_MATERIAL_LENGTH, signWithKeyMaterial, verifyWithKeyMaterial, } from '../crypto-utils.js';
|
|
2
|
+
/** The name the root secret is held under, used only in error messages. */
|
|
3
|
+
export const ACTOR_SECRET_NAME = 'SCENARIO_ACTOR_SECRET';
|
|
4
|
+
/**
|
|
5
|
+
* Namespaces the derivation so the same root secret used for anything else
|
|
6
|
+
* produces different values. See knowledge/crypto.md.
|
|
7
|
+
*/
|
|
8
|
+
export const ACTOR_SECRET_INFO = 'pikku:actor-sign-in';
|
|
9
|
+
/** The root must be strong: every persona's credential is derived from it. */
|
|
10
|
+
export const ACTOR_ROOT_SECRET_MIN_LENGTH = MIN_KEY_MATERIAL_LENGTH;
|
|
11
|
+
/**
|
|
12
|
+
* What the derivation is bound to. Lowercased because the sign-in endpoint
|
|
13
|
+
* looks the user up by lowercased address, and a credential that verified
|
|
14
|
+
* against a different string than the row it opens is a credential for nothing.
|
|
15
|
+
*/
|
|
16
|
+
export const actorSecretSubject = (email) => email.trim().toLowerCase();
|
|
17
|
+
/**
|
|
18
|
+
* One persona's actor credential: `HMAC-SHA256(root, email)`, base64url.
|
|
19
|
+
*
|
|
20
|
+
* The root secret is not itself a valid credential and never travels: what a
|
|
21
|
+
* scenario run, a CI job or a virtual user is handed is the derived value for
|
|
22
|
+
* the one address it is entitled to. Presenting it for any other address fails,
|
|
23
|
+
* so a leaked credential is worth exactly one synthetic account rather than the
|
|
24
|
+
* whole actor population.
|
|
25
|
+
*
|
|
26
|
+
* Deterministic, so nothing is stored and nothing is provisioned — the target
|
|
27
|
+
* re-derives the expected value from the address being signed in as. Rotating
|
|
28
|
+
* the root invalidates every derived credential at once, which is the property
|
|
29
|
+
* a per-persona secret table would have to implement by hand.
|
|
30
|
+
*/
|
|
31
|
+
export const deriveActorSecret = async (rootSecret, email) => signWithKeyMaterial(ACTOR_SECRET_NAME, rootSecret, ACTOR_SECRET_INFO, actorSecretSubject(email));
|
|
32
|
+
/**
|
|
33
|
+
* Whether `presented` is the credential for `email` under `rootSecret`.
|
|
34
|
+
*
|
|
35
|
+
* False — never throws — for a malformed, truncated or mismatched value, and
|
|
36
|
+
* the comparison is WebCrypto's own HMAC verify, so it does not exit early on
|
|
37
|
+
* the first differing byte.
|
|
38
|
+
*/
|
|
39
|
+
export const verifyActorSecret = async (rootSecret, email, presented) => verifyWithKeyMaterial(ACTOR_SECRET_NAME, rootSecret, ACTOR_SECRET_INFO, actorSecretSubject(email), presented);
|
|
@@ -26,6 +26,11 @@ export interface PersonaSignIn {
|
|
|
26
26
|
/** Headers every request after `login` must carry. */
|
|
27
27
|
headers(): Record<string, string>;
|
|
28
28
|
}
|
|
29
|
+
/**
|
|
30
|
+
* Yields the credential for one persona, for a caller that holds that persona's
|
|
31
|
+
* derived secret and not the root it came from.
|
|
32
|
+
*/
|
|
33
|
+
export type ActorSecretResolver = (persona: ResolvedPersona) => string | Promise<string>;
|
|
29
34
|
/**
|
|
30
35
|
* Sign a persona in through the Better Auth actor plugin — the local-development
|
|
31
36
|
* path.
|
|
@@ -34,15 +39,32 @@ export interface PersonaSignIn {
|
|
|
34
39
|
* for it. Passwordless by design and refused for any row not carrying that flag,
|
|
35
40
|
* so the secret can never reach a real user's account; the plugin still declines
|
|
36
41
|
* to serve the endpoint at all outside `pikku dev`.
|
|
42
|
+
*
|
|
43
|
+
* What is presented is the persona's own credential, derived from the root and
|
|
44
|
+
* bound to their address. A run driving many personas holds the root and
|
|
45
|
+
* derives as it goes; a run entitled to one persona is handed that one value
|
|
46
|
+
* through a resolver and can sign in as nobody else.
|
|
37
47
|
*/
|
|
38
48
|
export declare class ActorSignIn implements PersonaSignIn {
|
|
39
49
|
private readonly apiUrl;
|
|
40
50
|
private readonly secret;
|
|
41
51
|
private readonly signInPath;
|
|
42
|
-
constructor(apiUrl: string, secret: string, signInPath: string);
|
|
52
|
+
constructor(apiUrl: string, secret: string | ActorSecretResolver, signInPath: string);
|
|
43
53
|
login(jar: ScenarioCookieJar, persona: ResolvedPersona): Promise<void>;
|
|
44
54
|
headers(): Record<string, string>;
|
|
45
55
|
}
|
|
56
|
+
/**
|
|
57
|
+
* The auth mount a configured sign-in path sits under, or `undefined` when it
|
|
58
|
+
* names nothing recognisable.
|
|
59
|
+
*
|
|
60
|
+
* better-auth serves sign-in, operator sign-in and `get-session` from one
|
|
61
|
+
* prefix, so an app that mounts it at `/api/auth` moves all three together and
|
|
62
|
+
* says so once through `signInPath`. Reading the other two from a hardcoded
|
|
63
|
+
* `/auth` on such an app 404s — and for `get-session` a 404 reads as "this
|
|
64
|
+
* stage does not report roles", which silently turns off the check that tells a
|
|
65
|
+
* permissions finding from seed drift.
|
|
66
|
+
*/
|
|
67
|
+
export declare const authMount: (signInPath?: string) => string | undefined;
|
|
46
68
|
export interface OperatorSignInOptions {
|
|
47
69
|
/**
|
|
48
70
|
* The short-lived RS256 operator token, or a function that mints one. Prefer
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { deriveActorSecret } from './persona-actor-secret.js';
|
|
1
2
|
/**
|
|
2
3
|
* The header `resolveImpersonatedSession` reads the target user id from.
|
|
3
4
|
*
|
|
@@ -18,6 +19,11 @@ const failed = async (what, personaId, res) => {
|
|
|
18
19
|
* for it. Passwordless by design and refused for any row not carrying that flag,
|
|
19
20
|
* so the secret can never reach a real user's account; the plugin still declines
|
|
20
21
|
* to serve the endpoint at all outside `pikku dev`.
|
|
22
|
+
*
|
|
23
|
+
* What is presented is the persona's own credential, derived from the root and
|
|
24
|
+
* bound to their address. A run driving many personas holds the root and
|
|
25
|
+
* derives as it goes; a run entitled to one persona is handed that one value
|
|
26
|
+
* through a resolver and can sign in as nobody else.
|
|
21
27
|
*/
|
|
22
28
|
export class ActorSignIn {
|
|
23
29
|
apiUrl;
|
|
@@ -29,13 +35,16 @@ export class ActorSignIn {
|
|
|
29
35
|
this.signInPath = signInPath;
|
|
30
36
|
}
|
|
31
37
|
async login(jar, persona) {
|
|
38
|
+
const secret = typeof this.secret === 'function'
|
|
39
|
+
? await this.secret(persona)
|
|
40
|
+
: await deriveActorSecret(this.secret, persona.email);
|
|
32
41
|
const res = await jar.fetch(`${this.apiUrl}${this.signInPath}`, {
|
|
33
42
|
method: 'POST',
|
|
34
43
|
headers: { 'content-type': 'application/json' },
|
|
35
44
|
body: JSON.stringify({
|
|
36
45
|
email: persona.email,
|
|
37
46
|
name: persona.name,
|
|
38
|
-
secret
|
|
47
|
+
secret,
|
|
39
48
|
}),
|
|
40
49
|
});
|
|
41
50
|
if (!res.ok) {
|
|
@@ -51,6 +60,21 @@ export class ActorSignIn {
|
|
|
51
60
|
return {};
|
|
52
61
|
}
|
|
53
62
|
}
|
|
63
|
+
/**
|
|
64
|
+
* The auth mount a configured sign-in path sits under, or `undefined` when it
|
|
65
|
+
* names nothing recognisable.
|
|
66
|
+
*
|
|
67
|
+
* better-auth serves sign-in, operator sign-in and `get-session` from one
|
|
68
|
+
* prefix, so an app that mounts it at `/api/auth` moves all three together and
|
|
69
|
+
* says so once through `signInPath`. Reading the other two from a hardcoded
|
|
70
|
+
* `/auth` on such an app 404s — and for `get-session` a 404 reads as "this
|
|
71
|
+
* stage does not report roles", which silently turns off the check that tells a
|
|
72
|
+
* permissions finding from seed drift.
|
|
73
|
+
*/
|
|
74
|
+
export const authMount = (signInPath) => {
|
|
75
|
+
const mount = signInPath ? signInPath.lastIndexOf('/sign-in/') : -1;
|
|
76
|
+
return !signInPath || mount === -1 ? undefined : signInPath.slice(0, mount);
|
|
77
|
+
};
|
|
54
78
|
/**
|
|
55
79
|
* Establish a Fabric operator session against `apiUrl` and resolve the target's
|
|
56
80
|
* own id for `persona`, which is what the impersonation header names.
|
|
@@ -21,8 +21,11 @@ export class TypedSecretService {
|
|
|
21
21
|
return value;
|
|
22
22
|
}
|
|
23
23
|
async hasSecret(key) {
|
|
24
|
+
// `undefined` is cached for an optional secret that resolved absent, so a
|
|
25
|
+
// cache hit means "already looked", not "there is a value". Reporting true
|
|
26
|
+
// for it would let a read of an optional secret assert its own presence.
|
|
24
27
|
if (this.cache.has(key)) {
|
|
25
|
-
return
|
|
28
|
+
return this.cache.get(key) !== undefined;
|
|
26
29
|
}
|
|
27
30
|
return this.secrets.hasSecret(key);
|
|
28
31
|
}
|
|
@@ -6,7 +6,9 @@ import type { JudgeToolCallDisclosure, PikkuAgentScorer, ScorerInput, ScorerOutp
|
|
|
6
6
|
* @example snippet: agentScorer
|
|
7
7
|
*/
|
|
8
8
|
export declare const pikkuAgentScorer: <Services = any>(config: {
|
|
9
|
+
/** Identifies the scorer in results and in the Console. Unique per project. */
|
|
9
10
|
name: string;
|
|
11
|
+
/** What this scorer grades, in one line, for whoever reads the score later. */
|
|
10
12
|
description: string;
|
|
11
13
|
/** 0..1 fraction of live runs to grade. Defaults to all of them. */
|
|
12
14
|
sampleRate?: number;
|
|
@@ -15,6 +17,10 @@ export declare const pikkuAgentScorer: <Services = any>(config: {
|
|
|
15
17
|
* traffic has no answer key, so the runtime never samples it.
|
|
16
18
|
*/
|
|
17
19
|
requiresReference?: boolean;
|
|
20
|
+
/**
|
|
21
|
+
* The grade itself: read the finished run and return `{ score, reason }`.
|
|
22
|
+
* Runs in-process, so it may use your own services.
|
|
23
|
+
*/
|
|
18
24
|
score: (input: ScorerInput, services: Services) => ScorerOutput | Promise<ScorerOutput>;
|
|
19
25
|
}) => PikkuAgentScorer<Services>;
|
|
20
26
|
/**
|
|
@@ -28,7 +34,9 @@ export declare const pikkuAgentScorer: <Services = any>(config: {
|
|
|
28
34
|
* @example snippet: agentJudge
|
|
29
35
|
*/
|
|
30
36
|
export declare const pikkuAgentJudge: <Services = any>(config: {
|
|
37
|
+
/** Identifies the judge in results and in the Console. Unique per project. */
|
|
31
38
|
name: string;
|
|
39
|
+
/** What this judge grades, in one line, for whoever reads the score later. */
|
|
32
40
|
description: string;
|
|
33
41
|
/** 0..1 fraction of live runs to grade. Defaults to all of them. */
|
|
34
42
|
sampleRate?: number;
|
|
@@ -37,7 +45,9 @@ export declare const pikkuAgentJudge: <Services = any>(config: {
|
|
|
37
45
|
* traffic has no answer key, so the runtime never samples it.
|
|
38
46
|
*/
|
|
39
47
|
requiresReference?: boolean;
|
|
48
|
+
/** The model that grades, e.g. `'claude-sonnet-4-5'`. Not the model under test. */
|
|
40
49
|
model: string;
|
|
50
|
+
/** The rubric: what a good answer looks like, phrased as the goal it should meet. */
|
|
41
51
|
goal: string;
|
|
42
52
|
/**
|
|
43
53
|
* How much of the run's trajectory to disclose to the judge. Defaults to
|
|
@@ -45,5 +55,9 @@ export declare const pikkuAgentJudge: <Services = any>(config: {
|
|
|
45
55
|
* sending a third-party model the rows the tools returned.
|
|
46
56
|
*/
|
|
47
57
|
toolCalls?: JudgeToolCallDisclosure;
|
|
58
|
+
/**
|
|
59
|
+
* Replaces the generated rubric prompt outright, for framing `goal` cannot
|
|
60
|
+
* express. The `{ score, reason }` response is still forced.
|
|
61
|
+
*/
|
|
48
62
|
prompt?: (input: ScorerInput) => string;
|
|
49
63
|
}) => PikkuAgentScorer<Services>;
|
|
@@ -16,9 +16,13 @@ export interface GatewayAttachment {
|
|
|
16
16
|
export interface GatewayInboundMessage {
|
|
17
17
|
/** Platform-specific: a phone number, a Slack user id, and so on. */
|
|
18
18
|
senderId: string;
|
|
19
|
+
/** What they said, as plain text, with the provider's markup stripped. */
|
|
19
20
|
text: string;
|
|
21
|
+
/** The provider's own event, untouched, for anything this shape drops. */
|
|
20
22
|
raw: unknown;
|
|
23
|
+
/** Files and media that came with the message. */
|
|
21
24
|
attachments?: GatewayAttachment[];
|
|
25
|
+
/** Anything else the adapter wants to carry through to the wiring. */
|
|
22
26
|
metadata?: Record<string, unknown>;
|
|
23
27
|
}
|
|
24
28
|
/**
|
|
@@ -26,8 +30,11 @@ export interface GatewayInboundMessage {
|
|
|
26
30
|
* own rich content.
|
|
27
31
|
*/
|
|
28
32
|
export interface GatewayOutboundMessage {
|
|
33
|
+
/** The reply as plain text. Every provider can render this. */
|
|
29
34
|
text?: string;
|
|
35
|
+
/** The provider's own rich payload, e.g. Slack blocks. Passed through as-is. */
|
|
30
36
|
richContent?: Record<string, unknown>;
|
|
37
|
+
/** Files and media to send alongside. */
|
|
31
38
|
attachments?: GatewayAttachment[];
|
|
32
39
|
}
|
|
33
40
|
/**
|
|
@@ -35,9 +42,12 @@ export interface GatewayOutboundMessage {
|
|
|
35
42
|
* provider expects back, or not.
|
|
36
43
|
*/
|
|
37
44
|
export type WebhookVerificationResult = {
|
|
45
|
+
/** True when the request really came from the provider. */
|
|
38
46
|
verified: true;
|
|
47
|
+
/** What to echo back, e.g. Meta's hub.challenge. */
|
|
39
48
|
response: unknown;
|
|
40
49
|
} | {
|
|
50
|
+
/** False when the signature or challenge did not check out. */
|
|
41
51
|
verified: false;
|
|
42
52
|
};
|
|
43
53
|
/**
|
|
@@ -45,12 +55,15 @@ export type WebhookVerificationResult = {
|
|
|
45
55
|
* message, send one back, and open and close the connection.
|
|
46
56
|
*/
|
|
47
57
|
export interface GatewayAdapter {
|
|
58
|
+
/** Identifies the gateway in wirings and logs, e.g. `'slack'`. */
|
|
48
59
|
name: string;
|
|
49
60
|
/** Return null to ignore the event, e.g. a delivery receipt. */
|
|
50
61
|
parse(data: unknown): GatewayInboundMessage | null;
|
|
62
|
+
/** Deliver a reply back to the sender the message came from. */
|
|
51
63
|
send(senderId: string, message: GatewayOutboundMessage): Promise<void>;
|
|
52
64
|
/** Called by GatewayService.start(); must call onMessage per incoming event. */
|
|
53
65
|
init(onMessage: (data: unknown) => Promise<void>): Promise<void>;
|
|
66
|
+
/** Called by GatewayService.stop(); release the connection init() opened. */
|
|
54
67
|
close(): Promise<void>;
|
|
55
68
|
/** Receives the GET query params, or the POST body when called from the POST handler. */
|
|
56
69
|
verifyWebhook?(data: unknown, request?: PikkuHTTPRequest): WebhookVerificationResult | Promise<WebhookVerificationResult>;
|
|
@@ -18,5 +18,6 @@ export type { CorePersona, CorePersonas, PersonaAccountMeta, PersonaDefinitions,
|
|
|
18
18
|
* Lambda deploy would load outright.
|
|
19
19
|
*/
|
|
20
20
|
export { HttpPersona, createHttpPersonas, type HttpPersonasConfig, } from '../../services/http-personas.js';
|
|
21
|
-
export { ActorSignIn, OperatorSignIn, establishOperatorSession, IMPERSONATE_USER_ID_HEADER, type PersonaSignIn, type OperatorSignInOptions, type OperatorSessionResult, } from '../../services/persona-sign-in.js';
|
|
21
|
+
export { ActorSignIn, OperatorSignIn, establishOperatorSession, IMPERSONATE_USER_ID_HEADER, type ActorSecretResolver, type PersonaSignIn, type OperatorSignInOptions, type OperatorSessionResult, authMount, } from '../../services/persona-sign-in.js';
|
|
22
|
+
export { ACTOR_ROOT_SECRET_MIN_LENGTH, ACTOR_SECRET_INFO, ACTOR_SECRET_NAME, actorSecretSubject, deriveActorSecret, verifyActorSecret, } from '../../services/persona-actor-secret.js';
|
|
22
23
|
export { postScenarioJson, readScenarioHttpResponse, } from '../../services/personas-service.js';
|
|
@@ -14,5 +14,6 @@ export { APP_SCOPE_ROOT, appScopeId, buildAppScopeDefinition, } from './persona-
|
|
|
14
14
|
* Lambda deploy would load outright.
|
|
15
15
|
*/
|
|
16
16
|
export { HttpPersona, createHttpPersonas, } from '../../services/http-personas.js';
|
|
17
|
-
export { ActorSignIn, OperatorSignIn, establishOperatorSession, IMPERSONATE_USER_ID_HEADER, } from '../../services/persona-sign-in.js';
|
|
17
|
+
export { ActorSignIn, OperatorSignIn, establishOperatorSession, IMPERSONATE_USER_ID_HEADER, authMount, } from '../../services/persona-sign-in.js';
|
|
18
|
+
export { ACTOR_ROOT_SECRET_MIN_LENGTH, ACTOR_SECRET_INFO, ACTOR_SECRET_NAME, actorSecretSubject, deriveActorSecret, verifyActorSecret, } from '../../services/persona-actor-secret.js';
|
|
18
19
|
export { postScenarioJson, readScenarioHttpResponse, } from '../../services/personas-service.js';
|
|
@@ -1,8 +1,16 @@
|
|
|
1
1
|
export type CoreSecret<T = unknown> = {
|
|
2
|
+
/** The key code reads it by: `secrets.getSecret('NAME')`. SCREAMING_SNAKE_CASE. */
|
|
2
3
|
name: string;
|
|
4
|
+
/** How the secret is labelled wherever a person is asked to supply it. */
|
|
3
5
|
displayName: string;
|
|
6
|
+
/** What this secret is for, shown beside the field someone has to fill in. */
|
|
4
7
|
description?: string;
|
|
8
|
+
/** The id under the backing store, which is where the value actually lives. */
|
|
5
9
|
secretId: string;
|
|
10
|
+
/**
|
|
11
|
+
* The shape of the value, as a schema. This is what types `getSecret`'s
|
|
12
|
+
* result — pass the schema itself, not an instance of it.
|
|
13
|
+
*/
|
|
6
14
|
schema: T;
|
|
7
15
|
/** Required by default: this says absence is a supported state, and `getSecret` resolves `undefined` rather than throwing. */
|
|
8
16
|
optional?: boolean;
|
|
@@ -52,7 +52,7 @@ instances:
|
|
|
52
52
|
different one, and every finding it produces is unreproducible.
|
|
53
53
|
- **A run still `running` after `STALE_RUN_AFTER_MS` is failed and the persona
|
|
54
54
|
runs again.** This is where the stranded-record cost of
|
|
55
|
-
[a virtual user run being
|
|
55
|
+
[a virtual user run not being a workflow](a-virtual-user-run-is-not-a-workflow-but-it-needs-a-trigger.md)
|
|
56
56
|
gets paid: without it, one restart mid-run would block that persona's schedule
|
|
57
57
|
permanently.
|
|
58
58
|
|
package/knowledge/decisions/internals/a-virtual-user-run-is-not-a-workflow-but-it-needs-a-trigger.md
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: decision
|
|
3
|
+
title: A virtual user run is not a workflow, but it needs a trigger
|
|
4
|
+
description: runVirtualUser writes its record and dispatches the run onto a queue at one attempt — an exploratory run has nothing to replay, but a deployment that puts each function in its own unit has nothing to fire it either
|
|
5
|
+
tags: virtual-user, storage, deploy
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# A virtual user run is not a workflow, but it needs a trigger
|
|
9
|
+
|
|
10
|
+
`runVirtualUser` — the RPC `scaffold.virtualUser` generates — does three things
|
|
11
|
+
in order: writes a `VirtualUserRunStore` record, dispatches
|
|
12
|
+
`executeVirtualUserRun`, and returns the `runId`. The request never waits for
|
|
13
|
+
the run; a run takes minutes and survives neither a rollout nor a proxy timeout.
|
|
14
|
+
|
|
15
|
+
**A workflow** is still the wrong shape, for the reason it always was. Its value
|
|
16
|
+
is that a run can be resumed at the step it died on, and that the same input
|
|
17
|
+
reaches the same step. A virtual user is the opposite by construction — it is an
|
|
18
|
+
LLM deciding what to try next, so no two attempts take the same steps, and there
|
|
19
|
+
is no step to resume _to_. Recording a run as a workflow puts entries in the
|
|
20
|
+
workflow store that can never be replayed, and gives every operator reading that
|
|
21
|
+
store a row that lies about what it is. The seed makes a run _reproducible_ —
|
|
22
|
+
run it again and it explores the same way — which is a different property from
|
|
23
|
+
resumable, and one the record already carries.
|
|
24
|
+
|
|
25
|
+
**A queue was rejected once, on durability, and that was the wrong question.**
|
|
26
|
+
The original reasoning weighed a broker dependency against a retry nobody wants,
|
|
27
|
+
and concluded the in-process dispatch was enough. It is enough in one process.
|
|
28
|
+
It is not a dispatch at all under a deployment that puts each function in its own
|
|
29
|
+
unit: there is no in-process promise to leave running, and `executeVirtualUserRun`
|
|
30
|
+
— sessionless, unexposed, wired to nothing — is not a function any unit can be
|
|
31
|
+
reached at. The RPC resolves to nothing, the rejection is swallowed by the
|
|
32
|
+
`catch` that exists to stop it taking the process down, and the run parks at
|
|
33
|
+
`running` with zero steps and no error anywhere. That is what a fabric stage did.
|
|
34
|
+
|
|
35
|
+
So the queue is not bought for durability. It is bought because **a trigger is
|
|
36
|
+
what makes a function deployable**: `wireQueueWorker` puts `executeVirtualUserRun`
|
|
37
|
+
in the manifest, which gives it a unit and gives the platform somewhere to
|
|
38
|
+
deliver to. The job is dispatched at `attempts: 1`, because a redelivery is a
|
|
39
|
+
second different outing writing into a record that already has an outcome — the
|
|
40
|
+
retry the queue offers is precisely the part that stays unused.
|
|
41
|
+
|
|
42
|
+
A project with no queue service keeps the in-process dispatch. That is not a
|
|
43
|
+
fallback that hides a failure: a project without a broker runs in one process,
|
|
44
|
+
where an unawaited promise is a real dispatch and the only correct one.
|
|
45
|
+
|
|
46
|
+
The record remains the run's only trace, and that is what `VirtualUserRunStore`
|
|
47
|
+
exists for. It is also why `fail()` is a method rather than an absence: a run
|
|
48
|
+
that crashed and a run that found nothing are different answers, and a record
|
|
49
|
+
left at `running` is neither.
|
|
50
|
+
|
|
51
|
+
The cost is smaller than it was but has not gone: **a restart mid-run strands a
|
|
52
|
+
record at `running` with nothing left to finish it**, since nothing retries. A
|
|
53
|
+
run older than its budget window and still `running` is dead, not working — a
|
|
54
|
+
read-side rule. A stranded run is started again, with its seed if the caller
|
|
55
|
+
wants the same exploration.
|
|
56
|
+
|
|
57
|
+
Where that rule is actually applied is
|
|
58
|
+
[the schedule tick](a-virtual-user-cadence-is-a-row-not-a-timer.md), which has
|
|
59
|
+
to: a record stuck at `running` would otherwise block its persona's cadence
|
|
60
|
+
forever.
|
|
61
|
+
|
|
62
|
+
**What this rules out:** dispatching the run through `startWorkflow`; awaiting
|
|
63
|
+
the engine inside the request; retrying a run that failed; storing the operator
|
|
64
|
+
token on the record rather than on the dispatch; and inferring `status` from
|
|
65
|
+
`finishedAt` being unset, which cannot separate a crash from a run still going.
|
|
@@ -17,7 +17,7 @@ caller is entitled to assume.
|
|
|
17
17
|
- [A secret that fails to decrypt fails the whole read](a-secret-that-fails-to-decrypt-fails-the-whole-read.md) — getSecrets throws naming the key and its key_version rather than omitting the row, because a silent omission surfaces as an unrelated failure much later
|
|
18
18
|
- [A virtual user decides whether to trust its notes once per turn, by one roll](a-virtual-user-decides-whether-to-trust-memory-once-per-turn.md) — The difference between the stale, newcomer and auditor dispositions is expressed as a single probability rather than as prose in each prompt
|
|
19
19
|
- [A virtual user cadence is a row, not a timer](a-virtual-user-cadence-is-a-row-not-a-timer.md) — how often a persona runs is stored as a due time per persona and acted on by a tick the project schedules — pikku never starts a timer, and a run never reschedules itself
|
|
20
|
-
- [A virtual user run is not a workflow
|
|
20
|
+
- [A virtual user run is not a workflow, but it needs a trigger](a-virtual-user-run-is-not-a-workflow-but-it-needs-a-trigger.md) — runVirtualUser writes its record and dispatches the run onto a queue at one attempt — an exploratory run has nothing to replay, but a deployment that puts each function in its own unit has nothing to fire it either
|
|
21
21
|
- [A wall-clock threshold is a load test in disguise](a-wall-clock-threshold-is-a-load-test-in-disguise.md) — The KEK derivation test asserted a fixed 50ms budget for work that took 10ms, which went red about one run in five once the suite was large enough to compete for the machine
|
|
22
22
|
- [A workflow's wire is built from the run record, not from the RPC service](a-workflow-wire-is-built-from-the-run-not-from-the-rpc-service.md) — The RPC service exposes no wire, so every rpcService.wire read was undefined; the run record is the only thing that carries the caller across a step boundary
|
|
23
23
|
- [An actor conversation starts from a seeded kickoff message](actor-flow-conversations-seed-a-hidden-kickoff-message.md) — The actor's first turn needs a non-empty message list because providers reject an empty prompt; the seed is an instruction and stays out of the transcript
|