@flow-engineer/flow 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -21
- package/README.md +190 -17
- package/coding-agents/auth-skill.md +137 -0
- package/coding-agents/context-skill.md +94 -0
- package/coding-agents/skill.md +245 -11
- package/coding-agents/test-skill.md +13 -11
- package/coding-agents/tools-skill.md +78 -1
- package/dist/cli/src/auth.d.ts +11 -0
- package/dist/cli/src/auth.js +147 -0
- package/dist/cli/src/commands.d.ts +119 -0
- package/dist/cli/src/commands.js +212 -0
- package/dist/cli/src/index.d.ts +2 -0
- package/dist/cli/src/index.js +5 -0
- package/dist/cli/src/run.d.ts +28 -0
- package/dist/cli/src/run.js +523 -0
- package/dist/cli/src/terminal.d.ts +3 -0
- package/dist/cli/src/terminal.js +135 -0
- package/dist/flow-messaging/src/client.d.ts +7 -1
- package/dist/flow-messaging/src/client.js +13 -1
- package/dist/flow-messaging/src/index.d.ts +38 -3
- package/dist/flow-messaging/src/index.js +242 -40
- package/dist/mcp/src/client.d.ts +54 -0
- package/dist/mcp/src/client.js +175 -0
- package/dist/mcp/src/describe.d.ts +41 -0
- package/dist/mcp/src/describe.js +110 -0
- package/dist/mcp/src/index.d.ts +4 -0
- package/dist/mcp/src/index.js +7 -0
- package/dist/mcp/src/server.d.ts +70 -0
- package/dist/mcp/src/server.js +252 -0
- package/dist/mcp/src/terminal.d.ts +24 -0
- package/dist/mcp/src/terminal.js +54 -0
- package/dist/simulators/src/index.d.ts +1 -1
- package/dist/simulators/src/page.html +267 -49
- package/dist/simulators/src/server.d.ts +27 -0
- package/dist/simulators/src/server.js +264 -40
- package/dist/src/agent.d.ts +69 -3
- package/dist/src/agent.js +162 -8
- package/dist/src/channels.d.ts +2 -0
- package/dist/src/conversation.d.ts +16 -1
- package/dist/src/conversation.js +140 -12
- package/dist/src/events.d.ts +1 -1
- package/dist/src/index.d.ts +2 -2
- package/dist/src/index.js +1 -1
- package/dist/src/model.d.ts +27 -0
- package/dist/src/model.js +85 -13
- package/dist/src/names.d.ts +5 -0
- package/dist/src/names.js +5 -0
- package/dist/src/skills.d.ts +4 -0
- package/dist/src/skills.js +23 -0
- package/dist/src/store.d.ts +2 -0
- package/dist/src/store.js +18 -0
- package/dist/src/tools.d.ts +69 -0
- package/dist/src/tools.js +17 -1
- package/dist/src/traces.js +32 -4
- package/dist/src/types.d.ts +2 -0
- package/dist/testing/src/account.d.ts +4 -0
- package/dist/testing/src/account.js +891 -0
- package/dist/testing/src/chat.d.ts +3 -0
- package/dist/testing/src/chat.js +125 -0
- package/dist/testing/src/check.d.ts +4 -0
- package/dist/testing/src/check.js +90 -0
- package/dist/testing/src/chinwag.js +1 -1
- package/dist/testing/src/cli.js +105 -18
- package/dist/testing/src/client.d.ts +2 -0
- package/dist/testing/src/client.js +4 -3
- package/dist/testing/src/flow-login.d.ts +176 -0
- package/dist/testing/src/flow-login.js +274 -0
- package/dist/testing/src/new.d.ts +15 -2
- package/dist/testing/src/new.js +80 -21
- package/dist/testing/src/plugin.d.ts +4 -0
- package/dist/testing/src/plugin.js +116 -0
- package/docs/agents.md +160 -0
- package/examples/cafe/src/agent.ts +6 -2
- package/examples/cafe/src/channels.ts +20 -0
- package/examples/cafe/src/cli.ts +24 -0
- package/examples/cafe/src/main.ts +17 -3
- package/examples/cafe/src/mcp.ts +22 -0
- package/examples/cafe/src/tools.ts +13 -0
- package/flow-messaging/README.md +66 -27
- package/package.json +30 -2
- package/spec/agent-endpoint.openapi.yaml +2 -2
- package/spec/nudges.openapi.yaml +2 -2
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: flow-framework-auth
|
|
3
|
+
description: Set up verified-phone sign-in, a brand’s OAuth server or its existing login, with one account across chat, MCP and the CLI and backend credentials kept in tool code.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Sign-in for a Flow Framework agent’s tools
|
|
7
|
+
|
|
8
|
+
Read `flow-framework-context` first: `context/access.yaml` describes the brand’s accounts and how its users sign in. Keep that account system. Every interface resolves to the same stable brand user id; a phone number, provider login and CLI connection are ways to reach it, not separate accounts.
|
|
9
|
+
|
|
10
|
+
## Token authority and custody: the app mints, everyone else carries
|
|
11
|
+
|
|
12
|
+
The brand (or its identity provider) issues the credential its API trusts, controls its lifetime and refresh, and can revoke it. The end client, such as the CLI or an MCP host, receives and maintains its own connection credentials and sends them with requests. Flow is the protocol handoff, not a separate authority that turns a user id into access to the brand's API.
|
|
13
|
+
|
|
14
|
+
For a brand with ordinary login, its backend functions validate the existing session and mint or exchange the brand-authorized credential for the intended client and resource. Flow can coordinate OAuth registration, redirects, PKCE and one-time code exchange, retaining the short-lived protocol state needed for that exchange. It must not become a durable vault for the end client's brand access/refresh credentials or replace them with an independent Flow-owned permission to act as that user. Refresh and revocation stay under the brand's authority.
|
|
15
|
+
|
|
16
|
+
A messaging agent can itself be a server-side client: its channel connection credentials belong behind the brand's integration boundary, not in WhatsApp/iMessage messages or model context. Distinguish that client from the OAuth handoff service. Verified-phone sign-in can authorize an operation directly when that is the brand's account policy; it does not require an extra OAuth token or login screen.
|
|
17
|
+
|
|
18
|
+
## Choose the entry point
|
|
19
|
+
|
|
20
|
+
1. **Phone is the brand’s primary identity.** A sender on a provider-verified phone channel (WhatsApp, phone-based iMessage, SMS) is signed in. Resolve that phone to the brand’s account, creating it only when the brand supports that. Give their normal full account access. Do not add an OTP, browser login, consent screen or permission picker on top of the verified channel.
|
|
21
|
+
2. **The brand already exposes an OAuth authorization server.** Use it for MCP and CLI connections. On chat channels that need account sign-in, send a connection link to its existing login. A Google sign-in button in an app is not by itself an OAuth authorization server for MCP clients.
|
|
22
|
+
3. **The brand has an ordinary login.** Use that login as the authentication step behind Flow’s OAuth layer (or a self-hosted OAuth server). Phone OTP, email, social login, company SSO and custom login can all finish the same connection. The brand confirms the signed-in account server-side and owns credential issuance and refresh; the OAuth layer coordinates client registration, one-time codes, PKCE and the brand-authorized token exchange. Do not add a Flow grant-selection screen by default. An existing provider can have its own required screens.
|
|
23
|
+
|
|
24
|
+
A phone-first brand uses case 1 on WhatsApp and case 3 for its CLI: `auth login` opens the brand’s phone login in a browser, phone verification signs in, and OAuth returns to the CLI. The user never has to create an email account just to use another interface.
|
|
25
|
+
|
|
26
|
+
## Verified phone → account
|
|
27
|
+
|
|
28
|
+
`resolve` receives `phone` only when the channel verifies phone identities. It is normalized; `address` remains the provider’s original address. A custom web address that looks like a phone does not get this evidence. Only mark a channel `phone: true` when its provider verifies the sender and your webhook authenticates that provider.
|
|
29
|
+
|
|
30
|
+
```ts
|
|
31
|
+
const resolve = async ({ phone }) => phone ? accounts.forVerifiedPhone(phone) : null;
|
|
32
|
+
const agent = createAgent({ ...options, resolve });
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Return the brand’s account id. MCP’s `auth.verify` returns that same id as `key`, so the terminal, web and phone channels share the person’s history. Without a brand resolver, phone channels still share one `phone:<E.164>` identity; `context.key` is null. Connect other login methods to an existing account only with verified account-linking proof, never a number or email supplied in model input.
|
|
36
|
+
|
|
37
|
+
## Tool code gets backend access
|
|
38
|
+
|
|
39
|
+
Supply the same `credentials` provider to `createAgent` and `mcpServer`. It runs lazily when tool code requests a named credential or backend client, with the framework’s verified identity. The brand decides how to obtain, refresh, authorize and revoke that access. This callback can support an API token, an authenticated client, a grant reference or the brand’s own authentication mechanism.
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
const credentials = async (name, context) => {
|
|
43
|
+
if (name !== "backend" || !context.key) throw new Error("Sign in required");
|
|
44
|
+
return backend.forAccount(context.key); // checks current account/session access
|
|
45
|
+
};
|
|
46
|
+
|
|
47
|
+
const tools = [tool({
|
|
48
|
+
name: "my_orders", description: "Your orders.", input: z.object({}),
|
|
49
|
+
run: async (_input, ctx) => {
|
|
50
|
+
const api = await ctx.credentials.get("backend");
|
|
51
|
+
return api.orders();
|
|
52
|
+
},
|
|
53
|
+
})];
|
|
54
|
+
|
|
55
|
+
const agent = createAgent({ ...options, tools, resolve, credentials });
|
|
56
|
+
const mcp = mcpServer({ ...serverOptions, tools, auth, credentials });
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
In TypeScript use `ctx.credentials!.get<YourBackendClient>("backend")`. Never take an account id from tool input as authorization. The credential accessor is non-enumerable and the framework sends neither its provider nor resolved credentials to the model, history or events. Tool authors must also keep credentials out of their results, errors and logs. Do not cache a user’s access in a process-global variable. A request-local token workaround is not necessary when backend access is resolved by verified account.
|
|
60
|
+
|
|
61
|
+
When backend access depends on the caller's actual credential, `auth.verify` may return `{ key, credentials }`. That request's credential provider overrides the default for MCP and the authenticated terminal; close over the verified token there. A verified account id alone is not permission to mint or fetch a backend token. The provider is server-side and non-enumerable; it never enters model inputs or traces.
|
|
62
|
+
|
|
63
|
+
The default is full normal user access, not global administrator access. Existing business ownership checks still apply. Optional scoped connections remain available for brands that require them; do not invent a permissions UI or restrict the brand’s normal experience by default.
|
|
64
|
+
|
|
65
|
+
## Authentication at a chat tool boundary
|
|
66
|
+
|
|
67
|
+
Configure `createAgent({ auth: { verify?, signIn } })`. Public conversation remains available while signed out. Tools use the same `signIn` metadata as MCP: `"none"` is public, `"optional"` runs with a null key when unsigned, and `"required"` (default) cannot execute until authenticated. Agents without `auth` retain their existing behavior.
|
|
68
|
+
|
|
69
|
+
`verify(context)` can recheck revocation or the current channel connection. Without it, a non-null key from trusted `resolve` is enough, including a provider-verified phone mapped to a brand account. `signIn(context)` returns `{ message, url? }`: the framework delivers this as a real channel message with the login link, records it in the transcript, and does not run the protected tool. Parallel protected calls produce one challenge per channel per turn. A custom in-chat challenge may omit the URL. Login functions receive server-issued channel/address and tool identity, never model-supplied account ids. Errors fail closed and credentials stay outside model context.
|
|
70
|
+
|
|
71
|
+
After login, resolve that channel to the canonical account key; the user can resume the request. Do not automatically replay an irreversible operation merely because login succeeded.
|
|
72
|
+
|
|
73
|
+
## OAuth on MCP and the CLI
|
|
74
|
+
|
|
75
|
+
The MCP server is a resource server, not the user’s login page. Its auth adapter verifies a token on every request and returns the brand’s user id:
|
|
76
|
+
|
|
77
|
+
```ts
|
|
78
|
+
const auth = {
|
|
79
|
+
issuer: "https://login.example.com",
|
|
80
|
+
resource: "https://agent.example.com/mcp",
|
|
81
|
+
async verify(token, request) {
|
|
82
|
+
const session = await accounts.verifyAgentAccess(token, request);
|
|
83
|
+
return session ? { key: session.accountId } : null;
|
|
84
|
+
},
|
|
85
|
+
};
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Verify signature (or introspect opaque tokens), issuer, expiry, intended resource and current revocation/account status. The token must be issued for this MCP resource. Native plugin endpoints advertise their own resource too: configure the authorization server and verifier for each enabled resource. Never accept an unrelated website session token and forward it unchanged to a downstream API. Backend access is separate and resolved through `credentials`, or tools run within the brand’s backend under its existing authorization.
|
|
89
|
+
|
|
90
|
+
- Missing sign-in on a required tool returns HTTP 401 with `WWW-Authenticate`; MCP hosts discover the authorization server from protected-resource metadata. Do not return “please sign in” as a tool result and expect a host to start OAuth.
|
|
91
|
+
- `signIn: "none"` tools are public, `"optional"` can run either way, and the default is `"required"`. With no scope mapping, being signed in is enough. Brands that need scopes may configure `auth.scopes` and return granted `scopes` from `verify`.
|
|
92
|
+
- The CLI already implements browser-based authorization-code sign-in with PKCE and a loopback redirect. It stores credentials per server and the SDK refreshes them. CI can supply `<NAME>_TOKEN`. Device authorization is a separate optional flow, not an existing CLI feature.
|
|
93
|
+
- The authorization server handles registered clients and redirect validation. A third-party client’s existing authorization requirements remain its own. Flow’s first-party experiences do not need an extra permissions picker.
|
|
94
|
+
|
|
95
|
+
## Hosting choices
|
|
96
|
+
|
|
97
|
+
| Deployment | Authorization server | Brand work |
|
|
98
|
+
| --- | --- | --- |
|
|
99
|
+
| Flow-hosted, existing OAuth/OIDC server | Existing provider (for example Cognito) | Register clients/redirects, authorize the agent resource, and configure token verification/backend access. Flow does not add a broker. |
|
|
100
|
+
| Flow-hosted, ordinary/custom login | Flow's protocol bridge | Keep normal login. Add brand-owned one-time login completion, credential issuance, refresh, verification and revocation functions. The bridge relays brand-issued client credentials. |
|
|
101
|
+
| Self-hosted | Existing provider or the team's own OAuth server | Publish OAuth discovery and resource metadata, configure PKCE and client/redirect validation, implement the same brand authority boundary. No Flow service is required. |
|
|
102
|
+
|
|
103
|
+
An existing OAuth server must authorize the actual agent resource and supported clients; a website's social sign-in token alone is insufficient. Discover its authorization and token endpoints rather than assuming `/auth` and `/token`. Providers without dynamic registration need registered client configuration. The CLI accepts a configured OAuth client ID, and custom surfaces remain replaceable.
|
|
104
|
+
|
|
105
|
+
## Connecting an ordinary brand login
|
|
106
|
+
|
|
107
|
+
Keep the OAuth protocol layer reusable and the account authority in the brand. The brand integration must:
|
|
108
|
+
|
|
109
|
+
1. **Finish its existing login.** A small server-side function verifies its normal session and returns a short-lived assertion or opaque one-time code identifying the account and the pending Flow connection. Bind it to the client, intended resource, callback and nonce; exchange it for a single-use brand authorization required by credential issuance; never accept a user id merely posted by the browser.
|
|
110
|
+
2. **Provide account access.** Its backend validates a credential issued for that API, or supplies an authenticated execution path. Use separate resource credentials for MCP and downstream APIs; a UID alone never authorizes an API call.
|
|
111
|
+
3. **End access.** Account disable/deletion and connection revocation stop new requests and refresh. Signing out of one client need not disconnect the others.
|
|
112
|
+
|
|
113
|
+
For Firebase the app’s function validates the signed-in user’s Firebase session, or the authenticated provider’s phone proof, and resolves its Firebase UID. Where its API uses Firebase credentials, the intended client can exchange a brand-minted custom token for its own agent session and maintain that session; Flow’s handoff service does not retain its refresh credential. Keep the Admin SDK and its service-account keys in the brand’s backend. The old local test-account bootstrap is test setup, not a production sign-in flow.
|
|
114
|
+
|
|
115
|
+
A custom login implements the same boundary. Do not make every brand adopt one identity provider or copy an OAuth server into its agent repository.
|
|
116
|
+
|
|
117
|
+
After a verified OAuth callback has saved the chat client's connection, call
|
|
118
|
+
`await agent.signInCompleted({ channel, address, key, idempotencyKey: "login:" + connectionId })`.
|
|
119
|
+
This links the verified identity and queues an internal prompt to the agent, not a fixed user-facing
|
|
120
|
+
reply. The agent continues a request blocked by sign-in, or acknowledges naturally if nothing is
|
|
121
|
+
waiting. It does not automatically replay executed tools or unrelated failures. An anonymous
|
|
122
|
+
request blocked at the auth boundary is carried forward as user text; anonymous history stays
|
|
123
|
+
separate. Pass the originating `conversationId` when the login challenge was created inside a
|
|
124
|
+
conversation; it must belong to that channel address. The prompt targets that address even if
|
|
125
|
+
another session on the same channel wrote later. Signing in preserves no-contact rules, including
|
|
126
|
+
STOP. Use a stable unique completion ID so retries and restarts do not queue another event.
|
|
127
|
+
For playground connections pass `via: "playground"`; replies stay in the simulator. Persist pending
|
|
128
|
+
completion events and retry queueing after restart instead of relying on the browser staying open.
|
|
129
|
+
`linkIdentity` and `recordNudge` remain available for custom completion behavior.
|
|
130
|
+
|
|
131
|
+
## Exercise sign-in in the playground
|
|
132
|
+
|
|
133
|
+
Pass `authentication` to `simulators` to add **New user · signed out** to the person picker. Existing declared test people retain their current identities. The hook receives the `channel` and an opaque `visitor` issued by the playground server, and returns `{ key?, address?, label?, signInUrl? }`. Resolve that visitor's connection with the application's normal auth adapter and return the canonical account `key` only after authentication. A phone-primary brand can resolve a simulated provider-verified phone immediately; other interfaces can return the application's OAuth connection URL. Never take the key or verified phone proof from request bodies, and keep credentials out of the result.
|
|
134
|
+
|
|
135
|
+
For a new user the playground checks this hook before history, messages, events and person-specific actions. While signed out, the visitor can chat and see only their own anonymous conversation; requesting a protected tool produces the normal login message. Signing in gives access to the verified account conversation. It calls `agent.simulateVisitor` with the server-verified identity, so the account's existing conversation is shared with other interfaces and replies stay in the simulator. A new user needs no declared phone or preassigned account. Each interface needs its own proof; signing in on web does not silently authorize another CLI session.
|
|
136
|
+
|
|
137
|
+
After login return to the mounted playground with `person=visitor`; its message draft stays available. **Clear history** resets the selected visitor or authenticated account's conversation and agent memory across channels, after confirmation, while keeping its sign-in connection. **New signed-out user** replaces the browser's connection identity without clearing account history or revoking other clients. An existing authorization-server session may still allow login without re-entering a password. The operator token remains required on a deployed playground, and declared people still use the developer identity shortcut. This is a developer simulator, not a customer-facing chat endpoint.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: flow-framework-context
|
|
3
|
+
description: Build the context pack (OpenAPI spec, overlay, workflows, business, data model, access rules, examples) from an organization's existing codebase, before writing a Flow Framework agent's tools. Load when starting an agent for a company that already has an app, API or database, when asked for an OpenAPI spec or "the spec", or when the tools need facts nobody has written down.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# The context pack
|
|
7
|
+
|
|
8
|
+
Tools are only as good as what you know when you write them. Before writing a Flow Framework agent's
|
|
9
|
+
tools (the `flow-framework-tools` skill), gather everything about the organization's systems into one
|
|
10
|
+
folder in the agent's repository, `context/`. It is the input for every later step: the tools,
|
|
11
|
+
the agent's skills, the sign-in (the `flow-framework-auth` skill), the tests and the CLI.
|
|
12
|
+
|
|
13
|
+
Work from the organization's own code and documents. Never invent: anything you cannot find goes
|
|
14
|
+
in `context/README.md` under "Questions for the team", phrased so someone can answer it in a line.
|
|
15
|
+
|
|
16
|
+
## What goes in it
|
|
17
|
+
|
|
18
|
+
| File | Format | What it holds |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| `context/README.md` | Markdown | The index: where each file came from (repo, path, commit, who confirmed it), environments, and the questions for the team |
|
|
21
|
+
| `context/openapi.yaml` | OpenAPI 3.1, **exactly as the team gives it** | Their API: operations, request and response schemas, errors, security schemes |
|
|
22
|
+
| `context/overlay.yaml` | [OpenAPI Overlay 1.0](https://spec.openapis.org/overlay/v1.0.0) | Everything you add to their spec, as changes applied to it: clearer descriptions, units ("amount is in paise"), which fields matter, gotchas, operations to avoid |
|
|
23
|
+
| `context/workflows.arazzo.yaml` | [Arazzo 1.0](https://spec.openapis.org/arazzo/v1.0) | How calls connect: the steps of each real task, how one step's output feeds the next, what counts as success, what to do when a step fails |
|
|
24
|
+
| `context/business.md` | Markdown | What the company does, who its customers are, the words they use, the policies a person would be told ("refunds within 30 days", "never promise delivery times") |
|
|
25
|
+
| `context/data.md` | Markdown | The entities and their ids, how they relate ("an order belongs to one store"), and which fields are personal data |
|
|
26
|
+
| `context/access.yaml` | YAML | Who may do what: for each task, whether it works signed out, needs the person signed in, needs a write, or needs the person's approval each time; customer or staff only |
|
|
27
|
+
| `context/examples/*.md` | Markdown | Real situations and what should happen, including the awkward ones (a refund after the window, an order that fails to pay) |
|
|
28
|
+
|
|
29
|
+
Never put keys, tokens, passwords or real customers' data in any of these files.
|
|
30
|
+
|
|
31
|
+
**Why their spec stays untouched:** the team will send new versions. Your knowledge lives in the
|
|
32
|
+
overlay, which is applied to each new version the same way, so nothing is lost and you can see
|
|
33
|
+
exactly which overlay entries, workflows and tools point at operations that changed.
|
|
34
|
+
|
|
35
|
+
## Find the API, by stack
|
|
36
|
+
|
|
37
|
+
Look at the repository's routes, its framework and its dependencies first, then pick the case:
|
|
38
|
+
|
|
39
|
+
- **A REST API with an OpenAPI spec.** Ask for the latest version, or find it in the repo
|
|
40
|
+
(`openapi.*`, `swagger.*`, a generator such as FastAPI, NestJS's Swagger module, springdoc,
|
|
41
|
+
drf-spectacular, rswag). Use theirs as is.
|
|
42
|
+
- **A REST API without a spec.** Draft one from the routes and handlers, with an `operationId` and a
|
|
43
|
+
plain description on every operation, request and response schemas from the code's own types,
|
|
44
|
+
and the error responses the handlers send. Put `x-chinwag-draft: true` at its top and list it in
|
|
45
|
+
the questions: the team confirms it, and from then on it is theirs.
|
|
46
|
+
- **GraphQL.** Keep the schema (`context/schema.graphql`) instead of OpenAPI, and describe the
|
|
47
|
+
queries and mutations the tasks use in the workflows file as plain steps.
|
|
48
|
+
- **Supabase.** PostgREST generates an OpenAPI spec from the database. It is no longer served with
|
|
49
|
+
the public anon key: the team exports it with a secret key or the Management API
|
|
50
|
+
(`GET /v1/projects/{ref}/database/openapi`), once, and gives you the file. Read the row-level
|
|
51
|
+
security policies (`supabase/migrations`, `pg_policies`) into `access.yaml`: they are the real
|
|
52
|
+
permission model.
|
|
53
|
+
- **Firebase (Firestore, Realtime Database, Storage).** There is no API: the database and its
|
|
54
|
+
security rules are the API. There is no OpenAPI; instead:
|
|
55
|
+
- `firestore.rules` (and `database.rules.json`, `storage.rules`) into `access.yaml`: who may read
|
|
56
|
+
and write what, condition by condition;
|
|
57
|
+
- `firestore.indexes.json`: which queries are possible;
|
|
58
|
+
- the app's models and the code that reads and writes each collection into `data.md`: the
|
|
59
|
+
collections, their fields and types, and the ids that link them;
|
|
60
|
+
- Callable or HTTP Cloud Functions are its only real endpoints: list them in `data.md` with their
|
|
61
|
+
input and output.
|
|
62
|
+
- **Several of these at once** (a REST API plus Firebase for chat, say): one section per system in
|
|
63
|
+
`data.md`, and say in the workflows which system each step uses.
|
|
64
|
+
|
|
65
|
+
## Fill the rest
|
|
66
|
+
|
|
67
|
+
- **Workflows:** one per task the agent or a host should be able to do in one call ("place an
|
|
68
|
+
order" = check stock, create the order, take payment; if payment fails, cancel the order). These
|
|
69
|
+
become the tools: a tool is a task, not an endpoint.
|
|
70
|
+
- **Business and data:** from the app's own copy (its website, help pages, emails, the UI's
|
|
71
|
+
strings), its terms and policies, and its models. Mark every personal field (names, phone
|
|
72
|
+
numbers, addresses, payment details) in `data.md`: tools return them only when a task needs them.
|
|
73
|
+
- **Access:** from the rules and middleware. For each task in the workflows, write one of
|
|
74
|
+
`signed_out`, `signed_in`, `signed_in_write` or `approval` (the person confirms each time), and
|
|
75
|
+
`staff` when only the team's staff may do it. This is what the tools' `signIn` and `hints` say.
|
|
76
|
+
- **Examples:** from support tickets, chat logs or the team's own words, with personal data
|
|
77
|
+
removed. They become test scenarios (the `flow-framework-tests` skill).
|
|
78
|
+
- **Environments:** a sandbox or test server and how to get test accounts, the rate limits, how
|
|
79
|
+
the API pages results, and whether writes take an idempotency key. In `README.md`.
|
|
80
|
+
|
|
81
|
+
## Questions for the team
|
|
82
|
+
|
|
83
|
+
Ask for what the code cannot tell you. The usual ones:
|
|
84
|
+
|
|
85
|
+
1. The latest OpenAPI spec (3.1), with an `operationId` and a description on every operation.
|
|
86
|
+
2. Request and response examples, including the errors.
|
|
87
|
+
3. The security schemes and the scopes each operation needs, and how their users sign in today.
|
|
88
|
+
4. How results are paged, the rate limits, and whether writes accept an idempotency key.
|
|
89
|
+
5. A sandbox or test server, and test accounts.
|
|
90
|
+
6. The webhooks or events they send (orders shipped, payments failed).
|
|
91
|
+
7. Which tasks customers may do themselves, which need their approval each time, and which are
|
|
92
|
+
staff only.
|
|
93
|
+
|
|
94
|
+
Then go on to the `flow-framework-auth` skill (sign-in) and the `flow-framework-tools` skill (the tools).
|
package/coding-agents/skill.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: flow-framework
|
|
3
|
-
description: Build or change a chat agent on Flow Framework (WhatsApp, SMS, iMessage, Telegram, web). Load when creating a Flow Framework agent, editing its prompt or skills, wiring it into the playground, or when code imports @flow-engineer/flow
|
|
3
|
+
description: Build or change a chat agent on Flow Framework (WhatsApp, SMS, iMessage, Telegram, web), and serve its tools as an MCP server for Claude and ChatGPT or as a CLI. Load when creating a Flow Framework agent, editing its prompt or skills, wiring it into the playground, adding MCP or a CLI, or when code imports @flow-engineer/flow. Also load the flow-framework-tools skill (tools-skill.md) before writing tools and flow-framework-tests (test-skill.md) before writing tests.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Building an agent on Flow Framework
|
|
@@ -10,17 +10,20 @@ reply, acknowledges people fast, runs tools in parallel and waits for them, movi
|
|
|
10
10
|
background only when the person writes again or it passes its limit.
|
|
11
11
|
You write the agent's prompt, skills and tools; Flow Framework handles the conversation.
|
|
12
12
|
|
|
13
|
-
|
|
14
|
-
tool,
|
|
13
|
+
More skills go with this one: `flow-framework-tools` (`tools-skill.md`) before writing or changing a
|
|
14
|
+
tool, `flow-framework-tests` (`test-skill.md`) before writing or changing a test, `flow-framework-context`
|
|
15
|
+
(`context-skill.md`) before writing tools for a company that already has an app, API or database,
|
|
16
|
+
and `flow-framework-auth` (`auth-skill.md`) when tools need to know who the person is.
|
|
15
17
|
|
|
16
18
|
## Start a new agent
|
|
17
19
|
|
|
18
20
|
`npx @flow-engineer/flow new <name>` (lowercase letters, digits and dashes) makes `./<name>/` from the café
|
|
19
21
|
example: a complete agent with tools and tests that depends on the `@flow-engineer/flow` package and imports
|
|
20
22
|
`@flow-engineer/flow` and `@flow-engineer/flow/simulators`, with these skills in `.claude/skills/`. Run `npm install` there.
|
|
21
|
-
Build in it, changing Pip's prompt, skills, tools and tests into yours.
|
|
22
|
-
|
|
23
|
-
|
|
23
|
+
Build in it, changing Pip's prompt, skills, tools and tests into yours. A provider key is optional:
|
|
24
|
+
`npx @flow-engineer/flow link` (or `setup`) writes `FLOW_KEY`, the agent's one Flow key, into `.env` (never
|
|
25
|
+
read or print it), and the agent calls models through Flow with it on $2 of Flow trial credit. A key of the owner's own wins over it, made
|
|
26
|
+
at the provider: `OPENROUTER_API_KEY` at https://openrouter.ai/keys,
|
|
24
27
|
`OPENAI_API_KEY` at https://platform.openai.com/api-keys, `ANTHROPIC_API_KEY` at
|
|
25
28
|
https://console.anthropic.com/settings/keys, or `GEMINI_API_KEY` at https://aistudio.google.com/apikey.
|
|
26
29
|
The first one with a value is used, and `CAFE_MODEL` sets the café's model id. Then `npm start`
|
|
@@ -41,6 +44,10 @@ my-agent/
|
|
|
41
44
|
src/tools.ts the agent's tools (see tools-skill.md)
|
|
42
45
|
src/agent.ts buildAgent(): the one place the agent is built
|
|
43
46
|
src/main.ts runs it in the playground
|
|
47
|
+
src/mcp.ts optional: the tools as an MCP server (below)
|
|
48
|
+
src/cli.ts optional: the CLI file, for the organization's command line (below)
|
|
49
|
+
context/ optional: the context pack the tools are written from (context-skill.md)
|
|
50
|
+
mcp.lock.json optional: the tools of other MCP servers the agent uses (tools-skill.md)
|
|
44
51
|
tests/tools/*.yaml how each tool behaves in tests (see tools-skill.md)
|
|
45
52
|
tests/scenarios/*.yaml the tests (see test-skill.md)
|
|
46
53
|
```
|
|
@@ -89,8 +96,10 @@ is how long a finished tool's result waits for the others of its response; `shar
|
|
|
89
96
|
guidance with a string.
|
|
90
97
|
|
|
91
98
|
Models are reached through any OpenAI-compatible API. `openAICompatible()` with no key reads the
|
|
92
|
-
first of `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY
|
|
93
|
-
|
|
99
|
+
first of `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` and
|
|
100
|
+
`FLOW_KEY` (the agent's Flow key, on its trial credit, from `setup`) that has a value, and uses that provider's
|
|
101
|
+
default model (`MODEL` overrides it). `flowModel()` uses Flow alone; when its credit is used up a
|
|
102
|
+
call fails with code `trial_credit_used` and a next step (tell the owner to add their own key). With none it stops at once and
|
|
94
103
|
names them. Pass `apiKey` and `baseURL` for anywhere else (`baseURL` alone for a local server that needs no key).
|
|
95
104
|
|
|
96
105
|
Which model: recommend Claude Haiku 5.5 (`anthropic/claude-haiku-5.5`, the default above) or OpenAI
|
|
@@ -294,9 +303,50 @@ To demo what your backend does (an order ships), send the nudge it would send fr
|
|
|
294
303
|
On a deployed agent, mount it in your own server with a token of at least 32 characters, from a
|
|
295
304
|
secret: `const playground = play.handler(agent, { token: process.env.PLAYGROUND_TOKEN!, base: "/playground" })`,
|
|
296
305
|
and hand it the requests under `/playground` (`if (url.pathname.startsWith("/playground")) return playground(req, res)`).
|
|
297
|
-
Open `/playground/?token=...` once (it becomes an HttpOnly cookie and leaves the URL) or call the API
|
|
298
|
-
with `Authorization: Bearer <token>`.
|
|
299
|
-
|
|
306
|
+
Open `/playground/?token=...` once (it becomes an HttpOnly, SameSite=Lax cookie and leaves the URL) or call the API
|
|
307
|
+
with `Authorization: Bearer <token>`. The cookie survives a top-level return from sign-in on another site;
|
|
308
|
+
the API still accepts only requests from the playground’s own origin. Wrong tokens are limited per address,
|
|
309
|
+
and every simulated message is logged with the person, channel, time and address.
|
|
310
|
+
|
|
311
|
+
## Put it on a phone
|
|
312
|
+
|
|
313
|
+
To chat with the agent from the owner's phone, with no tunnel and no key made by hand, run these
|
|
314
|
+
from the agent's folder (`npx @flow-engineer/flow init --display-name "<name>"` makes a starter in
|
|
315
|
+
an empty one: first ask the owner what people should call their agent, such as "FlatX agent"; the
|
|
316
|
+
phone's first message is `Hi FlatX agent (ref <code>)`, and `package.json`'s `displayName` keeps it).
|
|
317
|
+
Only `chat` reads stdin; `init` asks for the name only in a terminal, and nothing else asks for input.
|
|
318
|
+
|
|
319
|
+
```sh
|
|
320
|
+
npx @flow-engineer/flow link --json # signs in and runs setup when needed (Flow's trial model key too), then the channel's own link (t.me, wa.me, sms:); give its say_to_person to the owner
|
|
321
|
+
npm start # the agent: its playground link, and the owner texts it while it runs
|
|
322
|
+
npx @flow-engineer/flow status --json # what the folder has and lacks, at any time
|
|
323
|
+
printf 'hi\nA latte please\n' | npx @flow-engineer/flow chat # your own check: one message per line, replies printed
|
|
324
|
+
npx @flow-engineer/flow test --synthetic # your own check: the tests, on the key setup wrote
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
When the owner asks how to test it, recommend first the playground (`npm start`, then open the
|
|
328
|
+
link it prints: the test people and the channels side by side, with Debug), then their phone (the
|
|
329
|
+
`link`, while `npm start` runs). `chat` and `test --synthetic` are for you to check your own work;
|
|
330
|
+
do not offer them to the owner as ways to test.
|
|
331
|
+
|
|
332
|
+
`link` runs `login` first when not signed in: it opens the owner's browser itself and prints the
|
|
333
|
+
link and code at once (with `--json` the first line is `{"status":"pending","verification_uri":...,
|
|
334
|
+
"user_code":...,"next":...}`), waits at most 120 s (`--timeout <s>`), then exits 4 with the code
|
|
335
|
+
kept: give the owner the link and code, and run `link` again (it polls the same code). Then it runs
|
|
336
|
+
`setup`, which writes the test cloud, traces and Flow Messaging keys into `.env`, prints names and
|
|
337
|
+
last 4 characters only, keeps values already there (`--replace` takes the new ones), and is safe to
|
|
338
|
+
run again. `--channel telegram|whatsapp|imessage` picks the channel; a channel flow.engineer cannot
|
|
339
|
+
open yet exits 3 with its `next`. `login`, `setup` and `link` also run on their own.
|
|
340
|
+
|
|
341
|
+
In `src/main.ts`, put `fm.telegram()` from `@flow-engineer/flow/flow-messaging` in the agent's
|
|
342
|
+
channels and call `fm.start(agent, { cursorFile: ".flow/flow-messaging-cursor" })`: with no webhook
|
|
343
|
+
secret it opens Flow Messaging's event stream, an outbound connection, so the laptop needs no public
|
|
344
|
+
URL, and the cursor file lets messages sent while it was stopped arrive when it starts again. The
|
|
345
|
+
starter does this when `.env` has `FLOW_MESSAGING_KEY`. Each error is
|
|
346
|
+
`{"error":{"code","message","next"}}`: run its `next`. Exit codes: 0 done, 1 usage or not signed in,
|
|
347
|
+
2 network, 3 refused, 4 sign-in waiting. Never ask the owner to paste a key into the chat. Always
|
|
348
|
+
name the package (`npx @flow-engineer/flow ...`): `npx flow` on its own fetches another npm package.
|
|
349
|
+
CI never logs in: `setup --ci` says how to give it the cloud key.
|
|
300
350
|
|
|
301
351
|
## Send traces to a dashboard
|
|
302
352
|
|
|
@@ -319,3 +369,187 @@ touches a chat; call `await agent.flushTraces()` before the process exits. Tests
|
|
|
319
369
|
`node --test` never send from the environment. Under `flow test` the traces say they are a test's
|
|
320
370
|
(`chinwag.run_kind` is "test", else "production", and each turn names the test session as
|
|
321
371
|
`chinwag.test.session`), so the dashboard keeps them apart from real conversations.
|
|
372
|
+
|
|
373
|
+
## Serve the tools over MCP
|
|
374
|
+
|
|
375
|
+
The same tools and skills go to Claude, ChatGPT and any MCP host as an MCP server. Install the
|
|
376
|
+
SDK it runs on (`npm i @modelcontextprotocol/server@2.3.1`, the version Flow Framework pins) and add
|
|
377
|
+
`src/mcp.ts`:
|
|
378
|
+
|
|
379
|
+
```ts
|
|
380
|
+
import { createServer } from "node:http";
|
|
381
|
+
import { fileURLToPath } from "node:url";
|
|
382
|
+
import { mcpServer } from "@flow-engineer/flow/mcp";
|
|
383
|
+
import { tools } from "./tools.ts";
|
|
384
|
+
|
|
385
|
+
export const mcp = mcpServer({
|
|
386
|
+
name: "acme",
|
|
387
|
+
version: "1.0.0",
|
|
388
|
+
tools,
|
|
389
|
+
dir: fileURLToPath(new URL("../agent", import.meta.url)), // its skills; AGENTS.md is not sent
|
|
390
|
+
// auth: { issuer, resource, scopes, verify } // sign-in: the flow-framework-auth skill
|
|
391
|
+
});
|
|
392
|
+
createServer((req, res) => void mcp.node(req, res)).listen(8790); // Streamable HTTP at /mcp
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
- `mcp.fetch(request)` serves a web-standard `Request` instead (Cloudflare Workers, Deno, Bun), so
|
|
396
|
+
it mounts beside the chat agent in one process or runs on its own: it needs no model, channels
|
|
397
|
+
or store.
|
|
398
|
+
- Hosts get the skills as the server's instructions (each one's description) and read one with the
|
|
399
|
+
`read_skill` tool. A skill that tells the chat agent to use `send_*` or `share_updates` makes
|
|
400
|
+
no sense to a host: `flow check` warns about it.
|
|
401
|
+
- Every part can be changed: `instructions` replaces the instructions, `descriptor(tool, generated)`
|
|
402
|
+
changes any tool's MCP descriptor, `extend(server)` adds handlers of your own (resources,
|
|
403
|
+
prompts) to the SDK's server, `path` moves the endpoint, `allowedHosts` checks the Host header.
|
|
404
|
+
- Try it locally with `claude mcp add --transport http acme http://localhost:8790/mcp`.
|
|
405
|
+
- Run `npx flow check` from the agent's folder before serving: it refuses tools that cannot be
|
|
406
|
+
described exactly (the `flow-framework-tools` skill lists the rules).
|
|
407
|
+
|
|
408
|
+
## A CLI for the organization
|
|
409
|
+
|
|
410
|
+
The same tools as commands, over the organization's MCP server, with the same sign-in and gating.
|
|
411
|
+
It runs on Stricli (`@stricli/core`, the framework Sentry's CLI is built on). Every tool is already a
|
|
412
|
+
command under its own name (`place_order` is `acme place-order`); a CLI file adds what needs
|
|
413
|
+
judgement. Install `@modelcontextprotocol/client@2.3.1` and `@stricli/core@1.3.0` (the versions
|
|
414
|
+
Flow Framework pins) and write `src/cli.ts`:
|
|
415
|
+
|
|
416
|
+
```ts
|
|
417
|
+
#!/usr/bin/env node
|
|
418
|
+
import { fileURLToPath } from "node:url";
|
|
419
|
+
import { defineCli, runCli } from "@flow-engineer/flow/cli";
|
|
420
|
+
|
|
421
|
+
export const cli = defineCli({
|
|
422
|
+
name: "acme",
|
|
423
|
+
server: process.env.ACME_SERVER ?? "https://acme.example.com/mcp",
|
|
424
|
+
description: "Acme's orders and customers",
|
|
425
|
+
groups: { order: "Place and track orders" },
|
|
426
|
+
commands: {
|
|
427
|
+
"order list": { tool: "list_orders", columns: ["order_id", "status", "total"], alias: "orders" },
|
|
428
|
+
"order view": { tool: "get_order", args: ["order_id"], examples: ["acme order view BL-1042 --json"] },
|
|
429
|
+
"order place": { tool: "place_order" },
|
|
430
|
+
},
|
|
431
|
+
});
|
|
432
|
+
|
|
433
|
+
if (process.argv[1] === fileURLToPath(import.meta.url)) process.exit(await runCli(cli, process.argv.slice(2)));
|
|
434
|
+
```
|
|
435
|
+
|
|
436
|
+
- Words are groups then a verb: `order` is a **command group**, `list`, `view` and `place` its
|
|
437
|
+
subcommands. `acme order` and `acme order -h` list the group's subcommands, `acme order place -h`
|
|
438
|
+
that command's flags and examples, and `acme -h` the groups and top-level commands. A word is a
|
|
439
|
+
group or a command, never both. `groups` gives each group its line in help.
|
|
440
|
+
- `args` are the input fields taken as positional arguments, `columns` a list's table, `view` one
|
|
441
|
+
record's fields, `alias` another name, `examples` lines for help. Every other input field is a
|
|
442
|
+
flag (`order_id` is `--order-id`); a list is a repeatable or comma-separated flag; an object or a
|
|
443
|
+
list of objects takes JSON, or `@file.json`, and its help shows the shape
|
|
444
|
+
(`JSON list of {item, size, milk?}`).
|
|
445
|
+
- A command of the team's own is any Stricli command: `"report daily": { command: buildCommand({ ... }) }`
|
|
446
|
+
with `buildCommand` from `@stricli/core`; inside it, `this.cli.call(tool, input)` calls the server's
|
|
447
|
+
tools and `this.cli.print(value, { json })` prints like the rest.
|
|
448
|
+
- Installed, it is the organization's own command: `"bin": { "acme": "src/cli.ts" }` in its package
|
|
449
|
+
(node runs `.ts` files outside `node_modules`; publish a compiled `.js` bin to npm). While building
|
|
450
|
+
it, run it with `npm run cli -- order -h`: npm takes flags before `--` as its own.
|
|
451
|
+
- Every tool command takes `--json` and `--fields a,b`; one that changes things takes `--dry-run`; one
|
|
452
|
+
that may delete or overwrite asks first in a terminal and needs `--yes` without one. Exit codes: 0
|
|
453
|
+
done, 1 the tool failed, 2 wrong command or arguments (with the help to read), 3 sign-in needed, 4
|
|
454
|
+
needs `--yes`, 5 the server could not be reached. `--server <url>` and `--token <token>` work on
|
|
455
|
+
every command. `acme auth login` signs in on the organization's own login (tokens in
|
|
456
|
+
`~/.config/acme/`, mode 600), `ACME_TOKEN` is for CI, `acme tools` lists every tool and schema, and
|
|
457
|
+
`acme skill --write ~/.claude/skills` installs the CLI's SKILL.md, made from the tools and the CLI
|
|
458
|
+
file, for coding agents. For any MCP server without a CLI file, `npx flow cli --server <url> -h`.
|
|
459
|
+
- `npx flow check` checks the CLI file against the tools: a command naming a tool that does not
|
|
460
|
+
exist, an `args` field the input lacks, a `columns` or `view` field the output lacks, two commands
|
|
461
|
+
with one name, a word that is both a command and a group, or one of the CLI's own words (`auth`,
|
|
462
|
+
`tools`, `skill`) is an error; a tool no command names is a warning (it keeps its own name).
|
|
463
|
+
|
|
464
|
+
|
|
465
|
+
### Native plugins and standalone MCP
|
|
466
|
+
|
|
467
|
+
The standalone `/mcp` endpoint includes the skills list and `read_skill`. Native plugins
|
|
468
|
+
copy the agent's `agent/skills/` folder and connect to `/mcp/plugin`, which serves tools only,
|
|
469
|
+
so a host receives each skill once. Both paths share tools, auth and descriptor overrides.
|
|
470
|
+
`mcpServer({ ..., plugin: false })` disables the plugin path; `plugin: { path: "/native" }`
|
|
471
|
+
moves it. `handle.plugin` exposes its path and descriptors. Your `instructions` override
|
|
472
|
+
applies to both endpoints, and `extend` can add SDK handlers on both.
|
|
473
|
+
|
|
474
|
+
From the agent folder, run:
|
|
475
|
+
|
|
476
|
+
```sh
|
|
477
|
+
flow plugin --server https://agent.example.com/mcp --out plugins
|
|
478
|
+
flow check --plugin plugins
|
|
479
|
+
```
|
|
480
|
+
|
|
481
|
+
`plugins/claude/` uses Claude Code's `.claude-plugin/plugin.json` and `.mcp.json` with
|
|
482
|
+
`type: "http"`. `plugins/agent-plugins/` uses Agent Plugins 1.0.0 root `plugin.json` and
|
|
483
|
+
`mcp.json` with `type: "streamable-http"`; Codex supports this portable format.
|
|
484
|
+
Pass `--name` to override the package name. For a custom plugin route, pass its URL ending
|
|
485
|
+
in `/plugin`, or edit the generated MCP config. No token is written into either package.
|
|
486
|
+
Regenerate after changing a skill or its assets; `check --plugin` compares content hashes.
|
|
487
|
+
Generated files remain plain files in your repository: customize manifests and add components
|
|
488
|
+
alongside them. Regeneration rewrites manifests and source skill copies; keep hand-written
|
|
489
|
+
skills outside those source paths.
|
|
490
|
+
|
|
491
|
+
Formats checked 18 October 2026: [Agent Plugins 1.0.0](https://agent-plugins.org/specification),
|
|
492
|
+
[OpenAI plugin packaging](https://developers.openai.com/plugins/build/plugins), and
|
|
493
|
+
[Claude Code plugin reference](https://code.claude.com/docs/en/plugins-reference).
|
|
494
|
+
|
|
495
|
+
MCP calls emit `tool.start`/`tool.end` with surface (`mcp`, `mcp-plugin`, `cli`), client name
|
|
496
|
+
and version, session, person, duration, outcome and error. CLI calls send command and an
|
|
497
|
+
idempotency key in `_meta`; a caller can reuse its key on a retry. `defineCli`'s `version`
|
|
498
|
+
sets its client version (default `"1"`). Neither CLI arguments nor trace credentials go in `_meta`.
|
|
499
|
+
`mcpServer` uses `FLOW_FRAMEWORK_TRACES_*`, like chat agents, or explicit `traces` options;
|
|
500
|
+
`traces: false` disables export without disabling your event sinks. `await handle.flushTraces()`
|
|
501
|
+
flushes its bounded background queue. Each tool call is a root trace without a conversation;
|
|
502
|
+
arguments and results require `FLOW_FRAMEWORK_TRACES_CONTENT=1`. Legacy clients' initialization
|
|
503
|
+
metadata is retained for up to 24 hours in at most 4,096 sessions; bearer tokens are never cached.
|
|
504
|
+
|
|
505
|
+
### Local playground CLI
|
|
506
|
+
|
|
507
|
+
To exercise the real CLI on your computer, pass `localCli: { channel: "terminal", config: cli }`
|
|
508
|
+
to `play.listen(agent, port, options)` or a locally mounted `play.handler(agent, options)`.
|
|
509
|
+
`cli` is the team's existing `defineCli` configuration, with its server URL; the named channel
|
|
510
|
+
must exist. This opens an independent CLI session in that tab, with real stdout/stderr, exit
|
|
511
|
+
codes, interactive input and confirmation prompts. It does not use the selected test person's
|
|
512
|
+
identity. Run `auth login`, open the login link and return, then use commands or `chat` (`/exit`
|
|
513
|
+
leaves chat). Reload reconnects. New CLI session discards only this runner's temporary sign-in;
|
|
514
|
+
server restart or 30 minutes without browser activity also ends it. Clear output keeps sign-in.
|
|
515
|
+
An operation already accepted by the agent may still complete after its client session ends.
|
|
516
|
+
|
|
517
|
+
Enable this only in local startup code. The runner is off by default, rejects non-loopback or
|
|
518
|
+
forwarded connections, and is not part of the hosted test cloud. It calls the actual CLI runner
|
|
519
|
+
only locally. Hosted CLI support is deferred pending security review; do not add it to cloud startup
|
|
520
|
+
or deployment wiring. See the hosted CLI TODO in the README. The runner invokes the CLI
|
|
521
|
+
without a shell: quoting works, pipelines/expansion do not. The browser cannot override server
|
|
522
|
+
or token flags, or use `skill --write` to write onto the developer's computer.
|
|
523
|
+
For other embedded clients, `CliConfig.directory` overrides credential/conversation storage;
|
|
524
|
+
`CliIO.env` overrides the inherited process environment and `CliIO.signal` cancels sign-in and
|
|
525
|
+
terminal HTTP requests. Installed CLI defaults stay unchanged.
|
|
526
|
+
|
|
527
|
+
### Terminal conversations
|
|
528
|
+
|
|
529
|
+
`chat` talks directly to the agent, using the same bearer as MCP; it is not an MCP tool. `defineCli({ name: "acme", server, agent: "https://agent.example.com/agent/terminal" })`
|
|
530
|
+
overrides the terminal URL; the default is the MCP server's origin plus `/agent/terminal`.
|
|
531
|
+
`acme chat "Which orders are ready?" --json` prints one response; `acme chat` reads a TTY until
|
|
532
|
+
Ctrl-D or `/exit`. Without a TTY pass a message to `chat` (an interactive chat exits 2); sign-in failures exit 3.
|
|
533
|
+
Human chat shows a `You ›` input prompt, a waiting indicator and agent replies under the CLI's
|
|
534
|
+
name. `--json` keeps stdout as one JSON response per line, without those labels. Embedded CLIs
|
|
535
|
+
can supply `CliIO.lines(prompt?)` and their own output handlers to replace the presentation.
|
|
536
|
+
`chat.json` is mode 600 and scoped by terminal endpoint. `--new` forgets the terminal's saved
|
|
537
|
+
reference; the agent keeps its one-person history across channels. It does not erase a person's
|
|
538
|
+
WhatsApp history. Teams can replace `chat` with their own Stricli command in `commands`.
|
|
539
|
+
|
|
540
|
+
Mount the optional terminal handler beside MCP (no model or extra dependency in the CLI):
|
|
541
|
+
|
|
542
|
+
```ts
|
|
543
|
+
import { terminalServer } from "@flow-engineer/flow/mcp";
|
|
544
|
+
const terminal = terminalServer({ agent: () => agent, auth });
|
|
545
|
+
const agent = createAgent({ ...options, channels: [...channels, terminal.channel] });
|
|
546
|
+
// Route terminal.path to terminal.fetch(request), and MCP's paths to mcp.fetch(request).
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
`terminalServer` accepts `path` and `channel` overrides. It verifies auth on every POST,
|
|
550
|
+
serializes concurrent requests per person, and returns `{ conversationId, replies }` after the
|
|
551
|
+
turn settles. The authenticated key becomes the same person that `resolve` returns on other
|
|
552
|
+
channels. A body cannot choose its own person; a conversation reference from another person is
|
|
553
|
+
refused. Its custom UI uses Markdown and exports turns as channel `cli`.
|
|
554
|
+
For your own HTTP adapters, `agent.request({ channel, key, text, conversationId? })` exposes this
|
|
555
|
+
without the handler. Supply a verified key and serialize calls for that channel and person.
|
|
@@ -5,7 +5,9 @@ description: Write or change tests for a Flow Framework agent (tests/scenarios/*
|
|
|
5
5
|
|
|
6
6
|
# Writing tests for a Flow Framework agent
|
|
7
7
|
|
|
8
|
-
Read the `flow-framework` skill (`skill.md`) first if you haven't.
|
|
8
|
+
Read the `flow-framework` skill (`skill.md`) first if you haven't. Turn the situations in
|
|
9
|
+
`context/examples/` (the `flow-framework-context` skill) into scenarios, and run `npx flow check`
|
|
10
|
+
before the tests when the agent serves MCP or a CLI.
|
|
9
11
|
|
|
10
12
|
A test is a scenario: a situation to put the agent in, played through a platform simulator by a
|
|
11
13
|
model acting as the person, with the agent's tools played by a model from `tests/tools/*.yaml`
|
|
@@ -120,7 +122,7 @@ connects out to the cloud, which plays the person and the tools and judges.
|
|
|
120
122
|
1. Have a test cloud: the hosted one at https://flow.engineer/observe, the default (see "Get a test
|
|
121
123
|
cloud" below), or another with `--cloud <url>` or `FLOW_FRAMEWORK_CLOUD`.
|
|
122
124
|
2. From the agent's folder (it must export `buildAgent` and `tools` from `src/agent.ts`), which reads
|
|
123
|
-
the agent's `.env` itself: `npx flow test --synthetic [--runs 3]` (`npm test -- --synthetic`
|
|
125
|
+
the agent's `.env` itself: `npx @flow-engineer/flow test --synthetic [--runs 3]` (`npm test -- --synthetic`
|
|
124
126
|
in an agent `flow new` made).
|
|
125
127
|
3. It first reads every test file as the cloud does; a mistake stops it with
|
|
126
128
|
`tests/scenarios/<file>.yaml:<line>: <field>: <what is wrong>`, one line each, and nothing is sent.
|
|
@@ -151,17 +153,17 @@ connects out to the cloud, which plays the person and the tools and judges.
|
|
|
151
153
|
|
|
152
154
|
Work in this loop, and never finish on a partial run:
|
|
153
155
|
|
|
154
|
-
1. `npx flow test --synthetic`: the whole suite.
|
|
156
|
+
1. `npx @flow-engineer/flow test --synthetic`: the whole suite.
|
|
155
157
|
2. Fix what failed (the agent, or a test that asks for the wrong thing; say which).
|
|
156
|
-
3. `npx flow test --synthetic --failed`, again until every run in it passes (exit 0).
|
|
157
|
-
4. `npx flow test --synthetic`: the whole suite once more, since a fix can break another scenario.
|
|
158
|
+
3. `npx @flow-engineer/flow test --synthetic --failed`, again until every run in it passes (exit 0).
|
|
159
|
+
4. `npx @flow-engineer/flow test --synthetic`: the whole suite once more, since a fix can break another scenario.
|
|
158
160
|
|
|
159
161
|
### When a run fails
|
|
160
162
|
|
|
161
|
-
1. `npx flow test --synthetic --json > report.json`, and read the runs whose `verdict` is not
|
|
163
|
+
1. `npx @flow-engineer/flow test --synthetic --json > report.json`, and read the runs whose `verdict` is not
|
|
162
164
|
`passed` (`jq '.runs[] | select(.verdict != "passed")' report.json`).
|
|
163
|
-
2. `npx flow show last` (the first run that did not pass; it names the others) or
|
|
164
|
-
`npx flow show <runId>`: the checks with the judge's reasons, then the conversation in time
|
|
165
|
+
2. `npx @flow-engineer/flow show last` (the first run that did not pass; it names the others) or
|
|
166
|
+
`npx @flow-engineer/flow show <runId>`: the checks with the judge's reasons, then the conversation in time
|
|
165
167
|
order with every tool call the simulator played, its input, output and what it changed.
|
|
166
168
|
`--json` gives the whole record.
|
|
167
169
|
3. Decide which is wrong, and say which:
|
|
@@ -169,7 +171,7 @@ Work in this loop, and never finish on a partial run:
|
|
|
169
171
|
- the test: a check asks for the wrong thing, or the scenario cannot reach its goal; fix the YAML;
|
|
170
172
|
- the simulator: a tool output that could not happen, or a person who left the scenario; fix
|
|
171
173
|
`tests/tools/*.yaml` or the scenario's `world` and `person`.
|
|
172
|
-
4. `npx flow test --synthetic --failed`, then `--runs 3 --only <scenario>` on what you fixed, so
|
|
174
|
+
4. `npx @flow-engineer/flow test --synthetic --failed`, then `--runs 3 --only <scenario>` on what you fixed, so
|
|
173
175
|
a pass is not luck.
|
|
174
176
|
5. The whole suite once more.
|
|
175
177
|
|
|
@@ -181,7 +183,7 @@ The owner does these; never ask for the key in chat or print it:
|
|
|
181
183
|
2. Workspace → New agent.
|
|
182
184
|
3. On the agent's Developers page, "Make a CI key". It starts with `oci_` and is shown once.
|
|
183
185
|
4. Put it in the agent's `.env` as `FLOW_FRAMEWORK_CLOUD_KEY` (never on the command line).
|
|
184
|
-
5. `npx flow test --cloud https://flow.engineer/observe --synthetic` from the agent's folder (`--cloud` is that address by default).
|
|
186
|
+
5. `npx @flow-engineer/flow test --cloud https://flow.engineer/observe --synthetic` from the agent's folder (`--cloud` is that address by default).
|
|
185
187
|
|
|
186
188
|
Your agent runs on your machine with your own model key. The cloud's person simulator, tool simulator and judge are free up to $5 a day per workspace, with 2 sessions at once; test data only (`--synthetic`), and records are deleted after 30 days. `--synthetic` is the promise that the test files hold no real customer or member data;
|
|
187
189
|
the CLI refuses a hosted cloud without it and the key. `--environment staging` or `production` is
|
|
@@ -192,7 +194,7 @@ answers, with the session's own token. `--agent-version` (else `GITHUB_SHA`) and
|
|
|
192
194
|
GitHub's branch) are kept with the session. A CI step (GitHub Actions shown):
|
|
193
195
|
|
|
194
196
|
```yaml
|
|
195
|
-
- run: npx flow test --cloud https://flow.engineer/observe --synthetic --environment staging
|
|
197
|
+
- run: npx @flow-engineer/flow test --cloud https://flow.engineer/observe --synthetic --environment staging
|
|
196
198
|
env:
|
|
197
199
|
FLOW_FRAMEWORK_CLOUD_KEY: ${{ secrets.FLOW_FRAMEWORK_CLOUD_KEY }}
|
|
198
200
|
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|