api-tracer-kit 1.0.0 → 1.1.0

package/docs/console.md

@@ -0,0 +1,428 @@
+# The console
+
+A separate process that maps every endpoint in your source, receives your app's
+live traffic, and lets you fire any endpoint and read the response. It needs no
+framework, no build step and — in most projects — no configuration.
+
+```bash
+npx api-tracer start
+```
+
+Reads your source, writes a catalog, and opens at `http://127.0.0.1:4400`.
+
+## Commands
+
+| | |
+| --- | --- |
+| `api-tracer scan` | read the source, write the endpoint catalog |
+| `api-tracer serve` | run the console |
+| `api-tracer start` | scan, then serve |
+| `api-tracer report` | build the insight report |
+| `api-tracer init` | write a config file from what the scan guessed |
+
+### Options
+
+| | |
+| --- | --- |
+| `--root <dir>` | the project to read (default: cwd) |
+| `--data <dir>` | where captures are kept (default: `<root>/.api-tracer`) |
+| `--port <n>` | default `4400` |
+| `--host <addr>` | default `127.0.0.1` |
+| `--preset <name>` | force a scanner preset |
+| `--format md\|json` | report format (default `md`) |
+| `--out <file>` | write the report to a file (default: stdout) |
+| `--env <name>` | which base URL the report describes |
+| `--force` | let `init` overwrite an existing config |
+
+## The catalog
+
+`scan` reads your source into `<data>/endpoints.json`: one entry per endpoint,
+with its module, HTTP verb, path template, the `${...}` holes in it, whether it
+sends query params or a body, its auth quirk, and the file and line it came
+from.
+
+It is a plain JSON file, so a deployed console needs no source and no build —
+just the file that shipped with it. Re-running `scan` where there is no source
+keeps the catalog that is already there.
+
+The scan also counts how many files outside its own module reference each
+endpoint. Anything at zero is flagged **unused in app** and gets its own filter.
+
+> This is a substring count, so `import * as services` or a re-export would hide
+> a real usage. Confirm with a grep before deleting anything.
+
+See [Configuration](./configuration.md) for how endpoints are found and what to
+do when the presets do not fit your codebase.
+
+## Live recording
+
+The main way to use it. Run your app normally with `reportTo` pointed at the
+console, and every call it makes streams in as you click — no export, no import,
+no typing payloads.
+
+Each request lands on its endpoint in the tree with the real query params, the
+real payload and the real path ids already filled in. The response is scored
+too, so an endpoint your app just used successfully goes green without you
+running anything.
+
+Click **Live** in the header to pause or resume. The counter shows what has
+arrived.
+
+A call that no endpoint in the catalog explains is **not** dropped — it is
+adopted as an endpoint of its own, marked *not found in source*, and behaves
+like the rest. Recording only what could be matched would quietly hide real
+traffic. **Clear uncatalogued** removes them.
+
+## Reading pass and fail
+
+Some APIs return HTTP 200 for failures and put the real verdict in the envelope:
+
+```json
+{ "status": 801, "success": false, "message": "Authentication token header missing" }
+```
+
+So an endpoint counts as **passing** only when the HTTP status is 2xx **and**
+the body's status code is not at or above the failure threshold **and** the
+success flag is not `false`. The response viewer shows the HTTP code and the
+body code side by side.
+
+Configure the convention with `envelope` in the config file, or set it to
+`false` if your API uses the status line honestly.
+
+## Using one endpoint
+
+Pick it in the tree. The request pane is tabbed like Postman:
+
+- **Params** — query params, as JSON, stringified into one param, or as a raw
+  query string
+- **Headers** — extra headers, merged on top of the token and `Content-Type` the
+  server adds
+- **Body** — the type selector: `none` / `JSON` / `multipart/form-data` /
+  `x-www-form-urlencoded`. JSON gives a validated editor; the form types give
+  key/value rows. Switching type keeps your values.
+- **Capture** — rules that store response values as `{{variables}}`
+
+The body type is preselected from what your app actually sent, so a captured
+`FormData` call opens on **multipart/form-data** with its fields already in the
+table, and **Send** replays it that way. Replaying a form post as JSON would
+fail for reasons that have nothing to do with the endpoint.
+
+Also on the pane: the source location (`src/services/appointments.js:91`), the
+resolved URL updating live as you type, a field per path placeholder, and after
+**Send** (or Cmd/Ctrl+Enter) the status, body code, latency, size, response body
+and headers. **Copy as cURL** leaves the token as a placeholder.
+
+JSON editors are syntax highlighted with line numbers and Tab indents rather
+than leaving the field — a coloured layer behind a transparent textarea, so
+there is no editor library and no dependency.
+
+### Query params: three modes
+
+Switching carries what you typed across.
+
+- **JSON** — `{"page": 1, "ids": [3, 4], "filter": {"status": "new"}}`, serialized
+  the way axios does by default: `?page=1&ids[]=3&ids[]=4&filter[status]=new`
+- **stringified** — the whole object in one encoded param,
+  `?params=%7B%22page%22%3A1%7D`. The param name is editable.
+- **query string** — pasted straight from DevTools and sent byte-for-byte, so an
+  already-encoded `%2B` stays `%2B` instead of becoming `%252B`
+
+The URL bar shows the fully resolved URL in every mode, built by the same rules
+the server uses, so the preview never disagrees with what goes out.
+
+## Bulk runs and replay
+
+**Run every GET endpoint** and **Run all GETs in this module** check many at
+once, six in parallel.
+
+Two deliberate limits on bulk GET runs:
+
+- **Only GETs are ever run in bulk.** A bulk run that fired nineteen DELETEs
+  would be a very bad afternoon.
+- **Path placeholders are left blank**, so `/appointments/${data.id}.json` is
+  called as `/appointments/.json`. That proves the endpoint answers; it does not
+  prove it behaves correctly with a real id. Treat bulk results as a smoke test.
+
+For each placeholder a run takes the first of: the value captured from real
+traffic, a variable sharing the placeholder's name, then blank. The run reports
+how many endpoints went out blank, so a weak result never looks like a strong
+one, and every saved result records the URL that was actually called.
+
+### Replay every captured call
+
+Re-sends every endpoint that has a sample — GET and writes alike — with the
+payload, query params, path ids and body type your app used.
+
+It shows exactly what it will do before it does anything: a count per method,
+the environment and base URL, and what it is leaving out. Defaults, all
+changeable in that dialog:
+
+- **DELETEs excluded.** Opt in with a checkbox; the endpoints are named.
+- **Logout excluded**, since it can invalidate the token the rest depends on.
+- **Endpoints needing a signed token excluded.**
+- **Payloads containing a stripped secret are skipped**, because sending the
+  literal `<redacted>` would fail for a reason unrelated to the endpoint.
+- **Runs one at a time**, in order. Captured writes often depend on each other,
+  and a failure order you cannot reproduce is worse than a slow run.
+- **Refused entirely on a `prod` environment.**
+
+This is not a dry run. Every POST creates a new record each time.
+
+While it runs the dialog becomes a progress view: a pass/fail bar, a running
+count, the call in flight, and the last few results. **Stop** halts after the
+call in progress. Results survive navigating away and reloading, because the run
+is held on the server.
+
+A call that fails during a replay keeps its response — open that endpoint and
+the response pane shows what came back, labelled *from the last replay*, until
+you send it again yourself. Those endpoints are marked **failed when replayed**.
+
+Results are saved and survive a restart, so you can see what changed since
+yesterday. An endpoint that was passing and now fails is marked **broke since
+last run**. Switching environment clears them, since dev results would be
+misleading on stage.
+
+## Contract drift
+
+A status check cannot see the failure that matters most on an API that answers
+`200` for everything: a backend that quietly drops a field, or changes an id
+from a number to a string, still looks green.
+
+So every passing response is reduced to its **shape** — field paths and types,
+values ignored — and compared with the endpoint's baseline. The first good
+response becomes the baseline; after that a change is reported:
+
+```
+-data[].pinned, +data[].author, data[].id: number->string
+```
+
+Drifted endpoints get an amber dot in the tree, a panel on the endpoint, and
+their own filter. When the change is intended, **This change is expected, make
+it the baseline** promotes the new shape and clears the flag.
+
+Live traffic feeds this for free: your app's own calls set and check the
+baseline as you use it.
+
+## Coverage
+
+The home screen shows how much of the API your app has actually exercised — how
+many endpoints have ever been seen, which modules remain untouched, and how many
+have drifted. Module rows show `seen/total`.
+
+An endpoint nobody has triggered has not been proven either way, which is
+different from failing. The **never captured** filter is the to-do list.
+
+## Variables and chaining
+
+Use `{{name}}` in any path value, query param or payload. Set them by hand in
+**Variables**, or capture them from a response: a rule like
+`patient_id = data.0.id` stores that value after a passing call, for every later
+call. Dotted paths walk arrays. A path that finds nothing reports an error
+rather than silently storing a blank.
+
+Variables are not a place for tokens — those stay in the console's own token
+store.
+
+## Import
+
+For traffic captured earlier, or from someone else's session.
+
+1. Open your app in Chrome, DevTools → **Network**, tick **Preserve log**.
+2. Click through the screens you care about.
+3. Right-click the request list → **Save all as HAR with content**.
+4. Drop the file anywhere on the console page.
+
+Each request is matched back to the endpoint the scanner found, by method and
+path shape, so a real call to `POST /appointments/4821/accept` lands on the
+template `/appointments/${data.id}/accept`. From it the console keeps the path
+id, the query params and the request body, which become the endpoint's starting
+values.
+
+For a single request, **Copy as cURL** in DevTools and paste it into the import
+dialog.
+
+The import reports what it could not place — usually an endpoint the backend has
+but the frontend does not call yet.
+
+## Export
+
+Every module page has **Export HAR** and **Export cURL**; the home screen has
+**Export all captured**.
+
+| | |
+| --- | --- |
+| **HAR** | a valid HAR 1.2 file with request, query string, payload and recorded response body. Opens in Chrome DevTools, imports into Postman, and re-imports here. |
+| **cURL** | one runnable command per endpoint, labelled with its source location |
+| **Postman** | a Collection v2.1. Modules become folders; each request carries the real query params and body in the right mode, so a captured form post imports as form-data, not broken JSON. Host and token are left as `{{baseUrl}}` and `{{token}}`. |
+
+One entry per endpoint, not per call — the most recent capture wins, which keeps
+an export of a busy session readable.
+
+All exports carry `<paste your token>` in place of a real token, so an export is
+safe to attach to a ticket.
+
+## The report
+
+**Report** in the header builds an insight report from everything the console
+already holds. No extra capture, no new run.
+
+| Section | |
+| --- | --- |
+| **Coverage** | exercised vs total, per module, and which modules nobody has walked |
+| **Failing** | status, body code and the message pulled out of the envelope, with regressions and replay failures marked |
+| **Contract drift** | response shapes that changed, with the field-level diff |
+| **Risks** | writes never exercised, captured DELETEs, 2xx responses carrying an error code, payloads whose secrets were stripped |
+| **Hygiene** | endpoints referenced nowhere, duplicate routes, calls not found in the source |
+| **Auth surface** | which endpoints need which token, and which third-party ones deliberately get none |
+| **Latency** | median, p95 and the slowest endpoints |
+| **Inventory** | request params, body fields and response fields per captured endpoint, from real traffic rather than hand-written docs |
+| **Trend** | pass rate and coverage over time, one line appended per run |
+
+**Export Markdown** for a ticket, **Export JSON** for CI. Or from the command
+line:
+
+```bash
+npx api-tracer report --format json --out api-report.json
+```
+
+The latency numbers are one measurement per endpoint, from its most recent call
+— indicative, not a benchmark.
+
+## Signing in
+
+Click **Set token** and paste one. The token is held by the server, never sent
+back to the page, and written owner-only (`chmod 600`) to the data directory so
+a restart does not sign you out. **Sign out** deletes it.
+
+If your API has a multi-step sign-in worth automating — password, then a
+one-time code, then an organisation — describe it in the config file and the
+console will drive it, so nobody has to paste anything. See
+[sign-in flows](./configuration.md#sign-in-flows).
+
+Third-party endpoints (anything flagged absolute) never receive your token.
+
+## Why calls go through the server
+
+The page posts the composed request to the console and the server forwards it.
+Calling the API from the page directly would be blocked by CORS, and the token
+would have to sit in client-side JavaScript.
+
+## Getting around
+
+- **The logo** goes back to the overview from anywhere
+- **Breadcrumbs** at the top of every page, each part clickable
+- **Escape** backs out one level — endpoint, then module, then overview. Ignored
+  while you are typing or a dialog is open.
+- Filters: method, and pass/fail/not-run. Search matches function name, path and
+  module.
+
+Typed path values and payloads are kept in the browser's local storage, so a
+reload does not lose your work.
+
+## Environment variables
+
+| | |
+| --- | --- |
+| `API_TRACER_DATA` | where captures are kept (default `<project>/.api-tracer`) |
+| `PORT` / `HOST` | default `4400` / `127.0.0.1` |
+| `BASE_PATH` | serve under a path, e.g. `/api-console`, behind a proxy |
+| `ALLOWED_ORIGINS` | comma-separated origins allowed to post recordings; localhost always works |
+| `CONSOLE_USER` / `CONSOLE_PASS` | basic auth on everything except `/api/record` |
+| `LOCK_RECORDING=1` | recording cannot be paused from the UI |
+| `LOCK_ENV=<name>` | pins the environment |
+
+`HOST` stays on `127.0.0.1` by default, so a laptop instance is not exposed by
+accident.
+
+## Running one for a team
+
+Every developer whose app has the tracer posts to the console whether or not
+they ever open it. The captures accumulate: one person exercises appointments,
+another exercises rota, and the console ends up holding both.
+
+Two variables stop one person's click changing things for everyone:
+
+```bash
+LOCK_RECORDING=1 LOCK_ENV=dev npx api-tracer serve
+```
+
+- Without `LOCK_RECORDING`, one developer clicking **Live** silently stops
+  recording for the whole team.
+- Without `LOCK_ENV`, someone switching to stage redirects everyone's next
+  replay and clears the shared results.
+
+The console also shows **whose** session the shared token belongs to, since
+everyone replays as that user.
+
+`CONSOLE_USER` / `CONSOLE_PASS` turn on basic auth. **`/api/record` stays
+open**, because the app posts recordings from a browser that cannot send those
+credentials. The gate protects reading captures, replaying calls and the stored
+token; it does not stop someone who can reach the host from posting junk
+recordings. If that matters, put the whole thing behind VPN or SSO.
+
+It is single-user by design: one token, one environment, one set of results. Two
+people on a shared instance will overwrite each other's environment and results.
+
+### Behind a proxy
+
+The mount path is forwarded as-is, so tell the console where it lives:
+
+```bash
+BASE_PATH=/api-console HOST=0.0.0.0 ALLOWED_ORIGINS=https://app.example.com npx api-tracer serve
+```
+
+Then point the tracer at the same path:
+
+```ts
+apiTracer.init({ reportTo: '/api-console' });
+```
+
+Same origin means no CORS and no mixed content.
+
+> **Mixed content.** An HTTPS app cannot post to an `http://` console; the
+> browser blocks it silently. Terminate TLS in front of the console, or serve it
+> under a path on the app's own origin.
+
+## HTTP API
+
+Everything the UI does is available directly.
+
+| Route | |
+| --- | --- |
+| `GET /api/endpoints` | catalog, base URLs, auth info, results, samples, live state |
+| `POST /api/token` | store or clear a token |
+| `GET/POST /api/login` | the configured sign-in flow: the next step, and submitting it |
+| `POST /api/env` | switch environment |
+| `POST /api/call` | send one endpoint |
+| `POST /api/run` | bulk run; GETs only unless explicitly allowed |
+| `POST /api/run/preview` | what a run would do |
+| `GET /api/run/status`, `POST /api/run/cancel` | progress and cancellation |
+| `POST /api/record` | one live call from the running app |
+| `GET/POST/DELETE /api/live` | recorder state: counts, pause, reset |
+| `GET/POST/DELETE /api/contracts` | response-shape baselines; POST accepts a drift |
+| `GET /api/export` | `?module=&format=har\|curl\|postman` |
+| `POST /api/import` | import a HAR or a cURL command |
+| `DELETE /api/samples` | drop all captured samples |
+| `DELETE /api/uncatalogued` | drop adopted endpoints |
+| `GET/POST /api/variables` | read or set `{{variables}}` |
+| `DELETE /api/results` | clear saved results |
+| `GET /api/report`, `GET /api/report/export` | the report model, and `?format=md\|json` |
+
+## What lives in the data directory
+
+`<project>/.api-tracer/`, or wherever `--data` points.
+
+| | |
+| --- | --- |
+| `endpoints.json` | the catalog, written by `scan` |
+| `samples.json` | requests captured from real traffic |
+| `results.json` | the last result per endpoint |
+| `contracts.json` | response-shape baselines |
+| `uncatalogued.json` | endpoints adopted from traffic |
+| `variables.json` | `{{variables}}` |
+| `lastrun.json`, `runs.jsonl` | the last run, and one line per run for the trend |
+| `tokens.json` | the session token, written `chmod 600` |
+
+**Add `.api-tracer/` to your `.gitignore`.** It contains real request and
+response bodies from the environment you are watching, and a real token. See
+[Security](./security.md).

package/docs/frameworks.md

@@ -0,0 +1,192 @@
+# Frameworks
+
+The tracer only needs one thing: `init()` called once, at the entry point,
+before anything makes a request. No bundler, Babel, TypeScript or ESLint
+configuration is required anywhere.
+
+## Plain JavaScript
+
+```html
+<script type="module">
+  import { apiTracer } from 'https://esm.sh/api-tracer-kit';
+  apiTracer.init();
+</script>
+```
+
+or in a bundled project:
+
+```js
+import { apiTracer } from 'api-tracer-kit';
+apiTracer.init();
+```
+
+## React
+
+```tsx
+// src/main.tsx — before ReactDOM.render / createRoot
+import { apiTracer } from 'api-tracer-kit';
+
+apiTracer.init({ reportTo: import.meta.env.DEV });
+```
+
+Or from inside the tree, which is StrictMode-safe:
+
+```tsx
+import { useApiTracer } from 'api-tracer-kit/react';
+
+function App() {
+  useApiTracer({ reportTo: true });
+  return <Routes />;
+}
+```
+
+`useApiTracer` deliberately does **not** destroy on unmount: in StrictMode that
+would tear the interceptors down immediately after installing them.
+
+To render the traces:
+
+```tsx
+import { useApiTraces } from 'api-tracer-kit/react';
+
+function CallCount() {
+  const { traces, clear } = useApiTraces();
+  return <button onClick={clear}>{traces.length} calls</button>;
+}
+```
+
+Updates are batched on an animation frame, so a burst of calls on page load
+re-renders once rather than once per request.
+
+## Create React App
+
+CRA's `ModuleScopePlugin` allows imports only from `src/` and `node_modules/`.
+Always import by **package name** — never by a relative path to a checkout
+outside the project:
+
+```js
+import { apiTracer } from 'api-tracer-kit';                     // ✅
+import { apiTracer } from '../../../packages/api-tracer-kit';   // ❌ rejected
+```
+
+A `yarn add file:` dependency creates a **symlink**, and webpack resolves
+symlinks to their real path — which lands outside the project and is rejected
+the same way. Depend on a published version, or on a packed tarball:
+
+```bash
+npm pack --pack-destination ./vendor   # in the package
+```
+
+```json
+"api-tracer-kit": "file:vendor/api-tracer-kit-1.0.1.tgz"
+```
+
+Gate it on `REACT_APP_ENV`, not `NODE_ENV` — a dev or stage deploy is a
+production build, so `NODE_ENV` would compile the tracer out of exactly the
+environments it is meant for:
+
+```js
+// src/services/apiRecorder.js
+import axios from 'axios';
+import { apiTracer } from 'api-tracer-kit';
+
+const isEnabled =
+  process.env.REACT_APP_ENV === 'dev' || process.env.REACT_APP_ENV === 'stage';
+
+if (isEnabled) {
+  apiTracer.init({ reportTo: process.env.REACT_APP_API_CONSOLE_URL || 'http://localhost:4400' });
+  apiTracer.useAxios(axios);
+}
+```
+
+```js
+// src/index.js — one import, at the top
+import './services/apiRecorder';
+```
+
+Compare `process.env.REACT_APP_ENV` **directly**, not through a variable, so
+DefinePlugin substitutes the literal and the whole condition folds away in a
+production build.
+
+## Vite
+
+```ts
+// src/main.ts
+import { apiTracer } from 'api-tracer-kit';
+
+if (import.meta.env.DEV) {
+  apiTracer.init({ reportTo: true });
+}
+```
+
+`import.meta.env.DEV` is statically replaced, so the branch is removed from a
+production build.
+
+## Next.js
+
+Importing the package on the server is safe — it evaluates no browser global —
+but there is no point tracing a render, so guard the call:
+
+```tsx
+// app/providers.tsx
+'use client';
+
+import { useApiTracer } from 'api-tracer-kit/react';
+
+export function Providers({ children }) {
+  useApiTracer({ reportTo: process.env.NODE_ENV === 'development' });
+  return children;
+}
+```
+
+For the Pages Router, `pages/_app.tsx` works the same way.
+
+To trace server-side calls too, `init()` in `instrumentation.ts` — Node has
+`fetch`, so it works there as well:
+
+```ts
+export async function register() {
+  if (process.env.NEXT_RUNTIME === 'nodejs') {
+    const { apiTracer } = await import('api-tracer-kit');
+    apiTracer.init({ include: ['/api/'] });
+  }
+}
+```
+
+Server traces stay in that process's memory; they do not reach the browser
+panel.
+
+## Node.js
+
+```js
+import { apiTracer } from 'api-tracer-kit';
+
+apiTracer.init();
+await fetch('https://api.example.com/users');
+console.log(apiTracer.getTraces());
+```
+
+Node 18+ has global `fetch`, so it is traced. There is no `XMLHttpRequest`, and
+that transport is skipped rather than erroring. For axios on Node's `http`
+adapter, `useAxios()` is required — no global patch can reach it.
+
+## Testing
+
+Use an isolated instance rather than the shared one, so cases cannot leak into
+each other:
+
+```ts
+import { ApiTracer } from 'api-tracer-kit';
+
+let tracer;
+beforeEach(() => { tracer = new ApiTracer().init({ transports: ['fetch'] }); });
+afterEach(() => { tracer.destroy(); });
+```
+
+`destroy()` restores the original `fetch`, so a suite that traces is
+indistinguishable from one that does not.
+
+## Webpack, Rollup, esbuild, Parcel
+
+Nothing to configure. The package ships ESM and CJS with `sideEffects: false`,
+so an unused import is dropped entirely and the four entry points are bundled
+separately — importing the tracer never pulls in React.