norn-cli 2.9.8 → 3.1.0

package/demos/mcp-ticket-testing/README.md

@@ -0,0 +1,114 @@
+# Ticket testing with declared MCP servers
+
+An agent graph that **does** something: a ticket arrives, one agent tests the product through a
+browser, another tests it through the team's own API, a third writes the results up, and a
+`judge` rules on whether the expected test cases were actually covered.
+
+Everything it connects to is declared in [`agents.nornagent`](./agents.nornagent), beside the
+model. There is **no `norn.config.json` in this folder** — the servers, their transports, and
+their session lifetimes are all in the file that uses them.
+
+## What to look at
+
+```nornagent
+mcp Browser
+    transport stdio
+    command node ./servers/browser-server.js
+    session agent
+end mcp
+
+agent FrontendTester
+    mcp Browser                                    # the run of the browser
+    tools House.getFixtureUser, House.resetTenant  # only what it needs from ours
+end agent
+
+agent BackendTester
+    tools House.callEndpoint, House.readAuditLog, House.queryDb
+end agent
+```
+
+Three things this shows, and they are the point of the whole slice:
+
+- **Two agents, one custom server, different tools each.** `BackendTester` gets three named
+  tools from `House` and cannot touch the browser at all. The access boundary is visible per
+  agent, in one file, in git.
+- **`mcp Browser` and `tools House.callEndpoint` are visibly different statements.** Dropping
+  four characters must not silently widen access from one tool to twenty, so there is no
+  short form that means "all of it".
+- **`session agent` on the browser.** Each agent invocation gets its own browser, closed when
+  that hop ends. The default, `session run`, shares one browser across every agent in the
+  sequence — right for a continuous flow, and wrong the moment a second agent is granted the
+  same server: it would inherit the first one's login and could pass a test it never
+  authenticated for, which is a false green and worse than a false red. Only `FrontendTester`
+  reaches the browser in this graph, so this is the lifetime worth authoring by default rather
+  than a bleed this demo would actually suffer.
+
+## Run it
+
+The servers are deterministic and ship with the demo, so nothing needs installing or starting.
+Prove they are reachable **without spending anything on a model**:
+
+```bash
+npm run compile
+node ./dist/cli.js demos/mcp-ticket-testing/tickets.norn --sequence ServersUp
+```
+
+That resolves the same aliases the agents use, so a failure points at the server rather than at
+an agent. When you are building a server of your own, this is the loop to develop against.
+
+Then the real thing, which does need a model:
+
+```bash
+export OPENAI_API="your-key"
+node ./dist/cli.js demos/mcp-ticket-testing/tickets.norn --sequence TicketPROJ142
+```
+
+The variable name is not a convention Norn assumes — it is written in the model block as
+`apiKey {{$env.OPENAI_API}}`, so any name works as long as the file and the environment
+agree. A `.nornenv` value wins over the process environment; add `--env <name>` to select one.
+
+## The browser here is a stand-in
+
+[`servers/browser-server.js`](./servers/browser-server.js) is a deterministic scripted site, not
+a real browser, so this demo runs anywhere `node` does — no browser download, no server to keep
+alive. Its declaration is the **same shape** a real Playwright MCP block takes, and both real
+forms are written out in the comments at the top of
+[`agents.nornagent`](./agents.nornagent):
+
+```nornagent
+mcp Browser
+    transport stdio
+    command npx playwright-mcp --headless --isolated
+    session agent
+end mcp
+```
+
+For CI with more than one sequence, run Playwright MCP as a service and dial it instead — that
+is what `transport http` is for. Norn never starts an http server; it only connects to one, and
+a refused connection fails before any model spend, naming the URL it tried.
+
+## The judge, and the CI story
+
+The last line of `TicketPROJ142` is the part that makes this a test rather than a demo:
+
+```norn
+judge report with Reviewer expects file expectations/proj-142.md
+```
+
+Every expectation in [`expectations/proj-142.md`](./expectations/proj-142.md) must be met;
+testing *more* than the list is not a failure. The judge is never asked for an overall verdict —
+it rules on each expectation and quotes its evidence, and Norn computes the result.
+
+Note the comment lines in that file. A line of prose left uncommented there **is** an
+expectation, which is both Norn's comment rule and what makes Markdown headings work.
+
+A live run records itself under `.norn-cache/runs/`. Replay it with no servers running at all:
+
+```bash
+node ./dist/cli.js replay .norn-cache/runs/<recording>.json
+```
+
+That is the CI answer for any expensive server: run it live where the servers exist, and gate
+pull requests on the replay — no browser, no API, no model, and the graph and contracts still
+checked. The recording keeps the tool names each granted server advertised, so a server that
+quietly gains or loses a tool is reported the next time a live run sees it.

package/demos/mcp-ticket-testing/agents.nornagent

@@ -0,0 +1,77 @@
+# Ticket testing — declared MCP servers, granted per agent.
+#
+# Every server this graph uses is declared here, beside the model, so the file says exactly
+# what it connects to. Nothing is left to a norn.config.json the reader cannot see.
+
+model Workbench
+    provider openai
+    name gpt-4o
+    apiKey {{$env.OPENAI_API}}
+end model
+
+# The browser. This demo ships a deterministic stand-in so it runs anywhere `node` does —
+# the declaration is the same shape a real Playwright MCP server takes:
+#
+#   mcp Browser
+#       transport stdio
+#       command npx playwright-mcp --headless --isolated
+#       session agent
+#   end mcp
+#
+# ...or, for a long-lived server you started yourself (a CI job with many sequences):
+#
+#   mcp Browser
+#       transport http
+#       url {{$env.BROWSER_MCP_URL}}     # e.g. http://localhost:8931/mcp
+#       timeout 120000                   # browser work is slow; the default is not for this
+#   end mcp
+mcp Browser
+    transport stdio
+    command node ./servers/browser-server.js
+    # One browser per agent invocation, closed when that hop ends. The default, `session run`,
+    # would share one browser across every agent in the sequence — which is what you want for a
+    # continuous flow, and what you do not want the moment a second agent is granted this server:
+    # it would inherit the first one's login and could pass a test it never authenticated for.
+    # Only FrontendTester reaches the browser here, so this is the safe default to author rather
+    # than a bleed this graph would actually suffer.
+    session agent
+end mcp
+
+# The team's own server: fixtures, endpoints, the audit log, the database.
+mcp House
+    transport stdio
+    command node ./servers/house-server.js
+end mcp
+
+agent FrontendTester
+    model Workbench
+    # The run of the browser: this agent drives the site, and which tools that needs is not
+    # something the author should have to enumerate.
+    mcp Browser
+    # From our own server it gets only what it needs to set up — and cannot reach the
+    # database or the audit log at all.
+    tools House.getFixtureUser, House.resetTenant
+    returns contracts/test-run.schema.json
+    system file prompts/frontend-tester.md
+end agent
+
+agent BackendTester
+    model Workbench
+    # The same server, narrowed differently. Both grants are visible per agent, in one file.
+    tools House.callEndpoint, House.readAuditLog, House.queryDb
+    returns contracts/test-run.schema.json
+    system file prompts/backend-tester.md
+end agent
+
+agent Reporter
+    model Workbench
+    system file prompts/reporter.md
+end agent
+
+agent Reviewer
+    model Workbench
+    system "
+    You are reviewing a QA report against a checklist of expected test cases.
+    Rule on each expectation independently and quote the report as evidence.
+    "
+end agent

package/demos/mcp-ticket-testing/contracts/test-run.schema.json

@@ -0,0 +1,31 @@
+{
+    "type": "object",
+    "additionalProperties": false,
+    "properties": {
+        "surface": {
+            "type": "string",
+            "description": "Which side of the product was exercised: frontend or backend."
+        },
+        "cases": {
+            "type": "array",
+            "description": "One entry per test case actually carried out.",
+            "items": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                    "name": { "type": "string" },
+                    "steps": { "type": "array", "items": { "type": "string" } },
+                    "outcome": { "type": "string", "enum": ["passed", "failed"] },
+                    "evidence": { "type": "string" }
+                },
+                "required": ["name", "steps", "outcome", "evidence"]
+            }
+        },
+        "findings": {
+            "type": "array",
+            "description": "Anything that looked wrong. Empty when nothing did.",
+            "items": { "type": "string" }
+        }
+    },
+    "required": ["surface", "cases", "findings"]
+}

package/demos/mcp-ticket-testing/expectations/proj-142.md

@@ -0,0 +1,12 @@
+# Expected test cases for PROJ-142
+
+# Each line below is one expectation. All must be met; testing more than this is not a
+# failure. Lines starting with # are comments — prose left uncommented here would itself
+# become an expectation, which is both Norn's comment rule and what makes Markdown work.
+
+- A test covers that a signed-out visitor opening /orders is sent to the sign-in page
+- A test covers signing in and seeing the orders list
+- A test covers opening order A-1001 and reading its total
+- A test covers GET /orders/A-1001 returning 200 with the order
+- A test covers GET /orders/A-9999 returning 404
+- A test confirms endpoint calls reach the audit log

package/demos/mcp-ticket-testing/fixtures/proj-142.json

@@ -0,0 +1,13 @@
+{
+    "key": "PROJ-142",
+    "title": "Order detail page and order lookup endpoint",
+    "description": "A signed-in customer can open an order from the orders list and see its total. The API must return the order for a valid id and 404 for an unknown one.",
+    "acceptanceCriteria": [
+        "A signed-out visitor who opens /orders is sent to the sign-in page",
+        "After signing in, the orders list shows the customer's orders",
+        "Opening order A-1001 shows its total",
+        "GET /orders/A-1001 returns 200 with the order",
+        "GET /orders/A-9999 returns 404",
+        "Every endpoint call is written to the audit log"
+    ]
+}

package/demos/mcp-ticket-testing/prompts/backend-tester.md

@@ -0,0 +1,12 @@
+You test the backend of a web product through its team's own MCP server.
+
+Work only from the ticket you are given. For each acceptance criterion that is about an API,
+a database row, or an audit entry, exercise it with the tools you have and record what came
+back — never what you assume would come back.
+
+`callEndpoint` calls the API. `queryDb` reads seeded tables. `readAuditLog` returns what this
+run wrote. You have no browser, and you must not claim to have used one.
+
+Return the required JSON only: one case per criterion you exercised, the steps you took, the
+outcome, and a short quote of the evidence you saw. Put anything that looked wrong in
+`findings`. Set `surface` to "backend".

package/demos/mcp-ticket-testing/prompts/frontend-tester.md

@@ -0,0 +1,14 @@
+You test the front end of a web product by driving a browser through an MCP server.
+
+Work only from the ticket you are given. For each acceptance criterion that is about what a
+person sees or does in the browser, carry out the steps yourself with the browser tools and
+record what actually happened — never what you assume would happen.
+
+The browser starts signed out. `getFixtureUser` gives you the test account; `resetTenant`
+puts the tenant back to its seeded state before you start.
+
+Snapshots are text. Read the elements listed in a snapshot rather than guessing at a page.
+
+Return the required JSON only: one case per criterion you exercised, the steps you took, the
+outcome, and a short quote of the evidence you saw. Put anything that looked wrong in
+`findings`. Set `surface` to "frontend".

package/demos/mcp-ticket-testing/prompts/reporter.md

@@ -0,0 +1,10 @@
+You turn two test runs into one short QA report for a ticket.
+
+You are given the frontend run and the backend run. Write, in plain prose:
+
+1. Which test cases were carried out, naming each one.
+2. What the evidence was for each.
+3. Any findings, or a single sentence saying there were none.
+
+Report only what the runs contain. Do not invent a case that nobody ran, and do not soften a
+failure. Someone will check this report against the list of expected test cases.

package/demos/mcp-ticket-testing/servers/browser-server.js

@@ -0,0 +1,133 @@
+#!/usr/bin/env node
+
+/**
+ * A deterministic stand-in for Playwright MCP.
+ *
+ * The `mcp Browser` block in `agents.nornagent` is the *same shape* a real Playwright MCP
+ * declaration takes — see the commented block there — so this demo runs anywhere `node` does,
+ * with no browser download, while showing exactly what granting a browser looks like.
+ *
+ * It answers from a tiny scripted site, and its snapshots are text, which is what a browser
+ * agent should be working from anyway: image content never reaches a model's prompt.
+ */
+
+const { Server } = require('@modelcontextprotocol/sdk/server/index.js');
+const { StdioServerTransport } = require('@modelcontextprotocol/sdk/server/stdio.js');
+const { ListToolsRequestSchema, CallToolRequestSchema } = require('@modelcontextprotocol/sdk/types.js');
+
+const PAGES = {
+    '/login': {
+        title: 'Sign in',
+        elements: ['textbox "Email"', 'textbox "Password"', 'button "Sign in"']
+    },
+    '/orders': {
+        title: 'Your orders',
+        elements: ['link "A-1001"', 'link "A-1002"', 'button "New order"']
+    },
+    '/orders/A-1001': {
+        title: 'Order A-1001',
+        elements: ['heading "Order A-1001"', 'text "Total: 42.50"', 'button "Cancel order"']
+    }
+};
+
+const state = { url: '/login', signedIn: false, filled: {} };
+
+const TOOLS = [
+    {
+        name: 'browser_navigate',
+        description: 'Navigate to a path on the site under test.',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: { path: { type: 'string', description: 'Path to open, e.g. /orders.' } },
+            required: ['path']
+        }
+    },
+    {
+        name: 'browser_fill',
+        description: 'Fill a named field on the current page.',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: {
+                field: { type: 'string', description: 'Field label, e.g. Email.' },
+                value: { type: 'string', description: 'Value to type.' }
+            },
+            required: ['field', 'value']
+        }
+    },
+    {
+        name: 'browser_click',
+        description: 'Click a named control on the current page.',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: { control: { type: 'string', description: 'Control label, e.g. Sign in.' } },
+            required: ['control']
+        }
+    },
+    {
+        name: 'browser_snapshot',
+        description: 'Return a text accessibility snapshot of the current page.',
+        inputSchema: { type: 'object', additionalProperties: false, properties: {} }
+    }
+];
+
+function structured(value) {
+    return { content: [{ type: 'text', text: JSON.stringify(value) }], structuredContent: value };
+}
+
+function snapshot() {
+    const page = PAGES[state.url] || { title: 'Not found', elements: [] };
+    return {
+        url: state.url,
+        title: page.title,
+        signedIn: state.signedIn,
+        elements: page.elements
+    };
+}
+
+async function main() {
+    const server = new Server(
+        { name: 'norn-demo-browser', version: '1.0.0' },
+        { capabilities: { tools: {} } }
+    );
+
+    server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: TOOLS }));
+
+    server.setRequestHandler(CallToolRequestSchema, async request => {
+        const args = request.params.arguments || {};
+        switch (request.params.name) {
+            case 'browser_navigate':
+                state.url = String(args.path || '/');
+                if (state.url !== '/login' && !state.signedIn) {
+                    // The whole point of a session: an unauthenticated visit lands on login.
+                    state.url = '/login';
+                }
+                return structured(snapshot());
+            case 'browser_fill':
+                state.filled[String(args.field)] = String(args.value);
+                return structured({ filled: args.field, ...snapshot() });
+            case 'browser_click':
+                if (String(args.control) === 'Sign in' && state.filled.Email && state.filled.Password) {
+                    state.signedIn = true;
+                    state.url = '/orders';
+                }
+                return structured(snapshot());
+            case 'browser_snapshot':
+                return structured(snapshot());
+            default:
+                return {
+                    isError: true,
+                    content: [{ type: 'text', text: `Unknown tool: ${request.params.name}` }]
+                };
+        }
+    });
+
+    await server.connect(new StdioServerTransport());
+}
+
+main().catch(error => {
+    console.error(error instanceof Error ? error.message : String(error));
+    process.exit(1);
+});

package/demos/mcp-ticket-testing/servers/house-server.js

@@ -0,0 +1,125 @@
+#!/usr/bin/env node
+
+/**
+ * "House" — the custom MCP server a team writes for its own product.
+ *
+ * Deterministic on purpose: the demo is about declaring a server and granting it, not about
+ * what any particular tool does. Everything here is in-memory and reset per process.
+ */
+
+const { Server } = require('@modelcontextprotocol/sdk/server/index.js');
+const { StdioServerTransport } = require('@modelcontextprotocol/sdk/server/stdio.js');
+const { ListToolsRequestSchema, CallToolRequestSchema } = require('@modelcontextprotocol/sdk/types.js');
+
+const state = {
+    tenant: 'acme',
+    auditLog: [],
+    orders: [
+        { id: 'A-1001', tenant: 'acme', total: 42.5, status: 'placed' },
+        { id: 'A-1002', tenant: 'acme', total: 18.0, status: 'shipped' }
+    ]
+};
+
+const TOOLS = [
+    {
+        name: 'getFixtureUser',
+        description: 'Return the seeded test user for the current tenant.',
+        inputSchema: { type: 'object', additionalProperties: false, properties: {} }
+    },
+    {
+        name: 'resetTenant',
+        description: 'Reset the tenant to its seeded state before a test run.',
+        inputSchema: { type: 'object', additionalProperties: false, properties: {} }
+    },
+    {
+        name: 'callEndpoint',
+        description: 'Call a backend endpoint and return its status and body.',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: {
+                method: { type: 'string', description: 'HTTP method, e.g. GET or POST.' },
+                path: { type: 'string', description: 'Endpoint path, e.g. /orders/A-1001.' }
+            },
+            required: ['method', 'path']
+        }
+    },
+    {
+        name: 'readAuditLog',
+        description: 'Return the audit entries written during this run, oldest first.',
+        inputSchema: { type: 'object', additionalProperties: false, properties: {} }
+    },
+    {
+        name: 'queryDb',
+        description: 'Read rows from a seeded table by name.',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: { table: { type: 'string', description: 'Table to read, e.g. orders.' } },
+            required: ['table']
+        }
+    }
+];
+
+function structured(value) {
+    return { content: [{ type: 'text', text: JSON.stringify(value) }], structuredContent: value };
+}
+
+function callEndpoint(args) {
+    const path = String(args.path || '');
+    const method = String(args.method || 'GET').toUpperCase();
+    state.auditLog.push({ at: state.auditLog.length + 1, method, path });
+
+    const orderMatch = path.match(/^\/orders\/([A-Za-z0-9-]+)$/);
+    if (orderMatch) {
+        const order = state.orders.find(item => item.id === orderMatch[1]);
+        return order
+            ? { status: 200, body: order }
+            : { status: 404, body: { error: 'order not found' } };
+    }
+    if (path === '/orders') {
+        return { status: 200, body: { orders: state.orders } };
+    }
+    return { status: 404, body: { error: 'no such endpoint' } };
+}
+
+async function main() {
+    const server = new Server(
+        { name: 'norn-demo-house', version: '1.0.0' },
+        { capabilities: { tools: {} } }
+    );
+
+    server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: TOOLS }));
+
+    server.setRequestHandler(CallToolRequestSchema, async request => {
+        const args = request.params.arguments || {};
+        switch (request.params.name) {
+            case 'getFixtureUser':
+                return structured({ id: 'u-1', email: 'tester@example.com', tenant: state.tenant });
+            case 'resetTenant':
+                state.auditLog = [];
+                return structured({ tenant: state.tenant, reset: true });
+            case 'callEndpoint':
+                return structured(callEndpoint(args));
+            case 'readAuditLog':
+                return structured({ entries: state.auditLog });
+            case 'queryDb':
+                return structured({
+                    table: args.table,
+                    rows: args.table === 'orders' ? state.orders : []
+                });
+            default:
+                return {
+                    isError: true,
+                    content: [{ type: 'text', text: `Unknown tool: ${request.params.name}` }]
+                };
+        }
+    });
+
+    await server.connect(new StdioServerTransport());
+}
+
+main().catch(error => {
+    console.error(error instanceof Error ? error.message : String(error));
+    process.exit(1);
+});

package/demos/mcp-ticket-testing/tickets.norn

@@ -0,0 +1,32 @@
+import "./agents.nornagent"
+
+# Prove both servers are reachable without spending anything on a model.
+# These steps resolve the same aliases the agents use, so a failure here points at the server
+# rather than at an agent — this is the loop to develop a server against.
+test sequence ServersUp
+    var browserTools = run mcp list Browser
+    assert browserTools.length > 0
+
+    var houseTools = run mcp list House
+    assert houseTools.length > 0
+
+    var user = run mcp call House getFixtureUser()
+    assert user.structuredContent.email == "tester@example.com"
+end sequence
+
+# The application this whole slice was designed against: a ticket arrives, two agents test it
+# from different sides with different tools, a third writes it up, and a judge rules on whether
+# the expected test cases were actually covered.
+test sequence TicketPROJ142
+    var ticket = run readJson "./fixtures/proj-142.json"
+
+    var frontend = run FrontendTester ticket
+    var backend = run BackendTester ticket
+    # An agent's input is a variable or a string — there is no object-literal form — so the two
+    # runs are interpolated into one prompt.
+    var report = run Reporter "Frontend run:\n{{frontend.body}}\n\nBackend run:\n{{backend.body}}"
+
+    print "QA report" | "{{report.text}}"
+
+    judge report with Reviewer expects file expectations/proj-142.md
+end sequence