rigorrun 0.2.0 → 0.3.1

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2026 RigorRun
+Copyright (c) 2026 Erbol Tahirov
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,135 +1,128 @@
+<div align="center">
+
 # RigorRun
 
-**Acceptance testing for tool-using AI agents.**
+**Your agent said it worked. RigorRun checks what it actually did.**
+
+Show RigorRun a job once. It turns that into a repeatable acceptance suite and decides whether your
+agent is safe to ship by reading the system it changed — never by trusting what it says about itself.
+
+**Early Access · v0.3** — parts of it are honestly unfinished, and they are listed rather than hidden.
 
-Connect your system. Show RigorRun how one job is done. Connect your agent.
-RigorRun proves whether the agent can do that job safely — by reading the
-system it changed, never by trusting what it says about itself.
+[rigorrun.xyz](https://rigorrun.xyz) · [Documentation](https://docs.rigorrun.xyz) · [Evidence](https://rigorrun.xyz/evidence) · [What is and is not built](https://rigorrun.xyz/what-is-built)
+
+</div>
 
 ```bash
-npx rigorrun
+npx rigorrun demo    # a real recorded run, replayed offline in a second
+npx rigorrun         # your own system and agent, in the local interface
 ```
 
-Open the URL it prints. That is the whole install.
+`demo` replays a real model working a bundled support desk: what it said beside what the system held
+afterwards. The same run, case by case, is at [rigorrun.xyz/replay](https://rigorrun.xyz/replay).
 
-**Early Access · v0.2.** It does what this page says and it is young. The
-limits are written down and marked one by one, rather than left for you to
-find: [the v1 gap audit](https://github.com/Konuktor/rigorrun/blob/master/docs/V1_GAP_AUDIT.md).
+`npx rigorrun` opens a local interface. Everything runs on your machine: there is no account, and no
+hosted component to send your systems to.
 
-## Or start with a server you already use
+---
 
-An MCP server can annotate a tool `readOnlyHint: true`. Nothing checks that.
+## Why
+
+An agent that reports success and an agent that achieved it are indistinguishable from the
+transcript. They are trivially distinguishable from the database.
+
+Evaluation harnesses score what the model wrote. RigorRun compares system state before and after,
+through read operations you nominate, and asks the questions that have a fact behind them: does the
+refund exist, is the amount right, is the ticket attached, was approval required, and did anything
+change that should not have.
 
-```bash
-npx rigorrun verify npm:@modelcontextprotocol/server-memory@2026.8.31
+```
+most tools:   you write the tests   →  the tool runs them
+  RigorRun:   you do the job once   →  RigorRun writes the tests
 ```
 
-No project, no browser, no agent, nothing to configure first. It pins the server
-to the exact bytes the registry published, runs it in a container with no
-network and no access to your machine, calls each tool with arguments derived
-from its own schema, and reads the filesystem before and after to see what
-actually changed — then compares that against what the server declared.
+## How
 
-Exit `0` verified · `1` a declaration was contradicted · `2` it could not run ·
-`3` it ran and established too little to be worth much.
+1. **Connect your system** — an MCP server you already run, an OpenAPI document, or a web
+   application through a browser.
+2. **Do the job once.** RigorRun reads your system before and after and derives what the rules must
+   be.
+3. **Rule on what it worked out.** It shows the evidence behind each proposed rule. A rule you
+   reject cannot fail your agent.
+4. **Connect your agent** — unchanged, wherever it runs: RigorRun sends each case's work to a URL it
+   already serves and reads the result from your system (a black box). Or an HTTP endpoint, a local
+   command, or your own loop pulling work.
+5. **Run it.** A verdict, with how strongly each answer could be verified.
+6. **Gate the next change** in CI.
 
-**Needs Docker.** Only `npm:` and `dir:` references, and only stdio servers.
-Against four published servers nobody here wrote, it exercised **19 of 37
-tools**; the rest are named with reasons in every record it writes.
+Setting a project up happens in the local interface. Running, gating and comparing are also
+available from the command line, which is the half CI needs.
 
-## What it does
+```bash
+rigorrun gate --project <id>     # exit 1 stops the build
+```
 
-You have an agent that calls tools. You need to know whether it can do a real
-job in your real system without doing something unsafe — and you need to know
-again next week, after somebody changes a prompt.
+## Verify a published MCP server
 
-RigorRun watches a person do that job once, reads the system before and after,
-works out what the rules must be, asks about what it can only guess, and turns
-the answers into an executable acceptance suite. Then it runs your agent against
-it and reads your system to find out what actually happened.
+An MCP server can annotate a tool `readOnlyHint: true`. Nothing checks that, and agents use it to
+decide whether a tool may be called without asking.
 
+```bash
+rigorrun verify npm:@modelcontextprotocol/server-memory@2026.8.31
 ```
-   most tools:   you write the tests   →  the tool runs them
-     RigorRun:   you do the job once   →  RigorRun writes the tests
-```
-
-## How strongly was it verified?
 
-RigorRun tells you how strongly each result was verified, on every result:
+Pins the server to the exact bytes the registry published, runs it in a container with no network
+and no access to your machine, calls each tool with arguments derived from its own schema, and reads
+the filesystem before and after. Needs a container runtime; nothing else does.
 
-- **AUTHORITATIVE** — checked against direct, trusted state.
-- **PARTIAL** — verified through the reads your system exposes. A normal
-  connected MCP server, whose state RigorRun reads back through the tools you
-  nominated, is **PARTIAL** — the common, honest case, not a defect.
-- **OBSERVATIONAL** — actions were observed but the final state could not be
-  independently proven (e.g. a browser with nothing readable attached).
+Run against four published servers, it exercised **19 of 37** tools. The other 18 are named, each
+with the reason it was not reached — [see the records](https://rigorrun.xyz/evidence).
 
-`AUTHORITATIVE` is claimed only where RigorRun genuinely has authoritative state
-access. Against your own system the honest label is usually `PARTIAL`, and it is
-shown on the same line as the verdict.
+## What you should know before relying on it
 
-## What it can build depends on your system
+- **A verdict against a real system is `PARTIAL`, by design.** RigorRun reads back what your
+  nominated reads return and no more, and says so on every result.
+- **A browser cannot verify itself.** A page saying "done" is a claim by the system that would have
+  to be wrong for it not to be done, so a browser connection is `OBSERVATIONAL`. Attach a readable
+  API or MCP connection for the same system to strengthen it.
+- **Isolation is `DECLARED`, not `RESET`, when you nominate a reset tool** — RigorRun has not run it
+  twice and compared.
+- **Three of the six verification sources are not emitted yet.** Model-judged and human-review
+  evaluators exist in the schema and are unreachable in practice.
+- **No external team has used this yet.** There are no customers.
 
-RigorRun generates every case it can safely and reproducibly verify, and tells
-you what it could not test. A system it can **seed and reset** yields the
-richest suite (boundary and adversarial cases, repeated destructive checks). A
-system without a reset still works, but produces fewer cases, disables repeated
-mutating cases, reports isolation `NONE`, and verifies `PARTIAL`. Best results:
-a staging or scratch environment with read-back and a reset.
+The full list is at [rigorrun.xyz/what-is-built](https://rigorrun.xyz/what-is-built), and the
+version carrying how each line was checked is
+[docs/V1_GAP_AUDIT.md](https://github.com/Konuktor/rigorrun/blob/master/docs/V1_GAP_AUDIT.md).
 
-## Why it runs locally
+## Privacy
 
-Your MCP server, your internal API and your staging box are usually not
-reachable from the public internet, and a page served over `https` cannot fetch
-`http://127.0.0.1`. So RigorRun's interface is served by this process, on your
-machine. Your credentials, recordings and systems never touch anybody's
-infrastructure, because there is no path by which they could.
+Your systems, your credentials and your recordings stay on this machine. There is no account, and
+nothing is uploaded unless you ask for it.
 
-## What you need
+Three things can make a network request, all because you asked: `rigorrun verify` downloads the named
+package from the npm registry; connecting your own system sends requests to your own system; and an
+LLM-backed agent, which exists only if you set a provider key, talks to that provider. None of them
+reach RigorRun. [How it is built](https://rigorrun.xyz/security).
 
-- **Node 20.11 or newer.**
-- **A way in to the system you want to test**: an MCP server, or an OpenAPI
-  document and the address it is served from. Ideally staging or a scratch
-  instance, with a way to reset it.
-- **An agent.** If it speaks MCP it works unchanged; RigorRun hands it a URL —
-  whether your agent listens on an address or is a command RigorRun runs. If it
-  does not speak MCP, about ten lines of plain HTTP — there is no package to
-  install; the protocol is documented at
-  [docs/HTTP_AGENT.md](https://github.com/Konuktor/rigorrun/blob/master/docs/HTTP_AGENT.md).
+## Verifying what you installed
 
-## Commands
+Published by a GitHub Actions workflow that holds no npm token, using npm trusted publishing, with a
+SLSA build provenance attestation.
 
 ```bash
-npx rigorrun                              # start the runner and open the interface
-npx rigorrun verify <server-ref>          # what does this server's tools actually do?
-npx rigorrun doctor                       # check this machine and every project
-npx rigorrun projects                     # what is on this machine
-npx rigorrun run --project <id>           # run the suite
-npx rigorrun gate --project <id>          # run it, and exit non-zero if it misses the bar
-npx rigorrun compare-runs --project <id> <runId>
-npx rigorrun feedback export              # a sanitised bundle for a bug report
+npm audit signatures
+npm view rigorrun dist.integrity
 ```
 
-## This is early access
+## Requirements
 
-It connects to MCP servers, to HTTP APIs with an OpenAPI document, and to web
-applications through a browser — though a browser cannot verify itself, so a
-verdict from one is OBSERVATIONAL unless something readable is attached.
-Setting a project up needs the interface; running and gating it does not. It has
-been used successfully by the people who wrote it and is now looking for people
-who did not.
+Node 20.11 or newer. Docker only for `rigorrun verify`.
 
-Every capability is marked WORKING, PARTIAL, DEMO-ONLY, BROKEN or MISSING in
-[the v1 gap audit](https://github.com/Konuktor/rigorrun/blob/master/docs/V1_GAP_AUDIT.md),
-with how each one was checked. Read it before you rely on this for anything
-that matters.
-
-If it goes wrong, `npx rigorrun feedback export` produces a bundle that contains
-no credentials, no tool arguments and no results — only what is needed to work
-out where it broke.
-
-## Documentation
+```bash
+rigorrun doctor      # what this machine has, and what it does not
+```
 
-<https://github.com/Konuktor/rigorrun/tree/master/docs>
+---
 
-MIT licensed.
+MIT · Built by Erbol Tahirov · [Report a problem](https://github.com/Konuktor/rigorrun/issues)