api-tracer-kit 1.0.0 → 1.1.0

package/docs/architecture.md

@@ -0,0 +1,136 @@
+# Architecture
+
+```
+api-tracer-kit
+├── src/                    the runtime tracer (TypeScript)
+│   ├── core/
+│   │   ├── tracer.ts       the instance: options, storage, subscribers, adapters
+│   │   ├── types.ts        the public data model
+│   │   ├── env.ts          runtime detection and trace ids
+│   │   ├── global.ts       the cross-bundle registry
+│   │   ├── redact.ts       credential redaction
+│   │   ├── body.ts         reading bodies without breaking them
+│   │   ├── url.ts          absolute URLs, axios-style query serialization
+│   │   └── envelope.ts     reading a verdict out of a 2xx body
+│   ├── adapters/
+│   │   ├── fetch.ts        global fetch
+│   │   ├── xhr.ts          XMLHttpRequest prototype
+│   │   ├── axios.ts        an axios instance and everything it creates
+│   │   └── suppress.ts     stops axios-on-XHR being counted twice
+│   ├── storage/memory.ts   the capped in-memory default
+│   ├── transport/report.ts posting finished traces to a console
+│   ├── ui/index.tsx        the React panel
+│   ├── index.ts            api-tracer-kit
+│   ├── axios.ts            api-tracer-kit/axios
+│   └── react.ts            api-tracer-kit/react
+└── cli/                    the console (plain ESM, no build step)
+    ├── bin/api-tracer.mjs  the command
+    ├── presets.mjs         how to find endpoints in a codebase
+    ├── config.mjs          defaults, and guessing what was not configured
+    ├── scan.mjs            source -> endpoint catalog
+    ├── server.mjs          forwards calls, replays, exports, records
+    ├── import.mjs          HAR and cURL, matched back to the catalog
+    ├── shape.mjs           response shapes and drift diffing
+    ├── report.mjs          the report model and its Markdown export
+    └── web/                the dashboard: no framework, no build
+```
+
+## The two halves
+
+The **tracer** is framework-free and knows nothing about any application. It
+records what happened and hands it to whoever asked.
+
+The **console** knows nothing about any application either: the catalog supplies
+the endpoints, base URLs, auth header and envelope convention, and the config
+file supplies the sign-in flow. Everything project-specific lives in generated
+data or in your config, never in the code.
+
+They are useful separately. The tracer with no console is an in-app panel and a
+subscriber API. The console with no tracer still scans your source, still lets
+you fire endpoints, and still imports a HAR.
+
+## Design constraints
+
+Three rules shaped most of the code.
+
+### It must never change what the application sees
+
+Every adapter is written around this. Responses are read from a `clone()` after
+the original has been returned, never before. Request bodies that are streams
+are described rather than consumed. Errors are rethrown as they arrived. XHR is
+observed with `addEventListener`, never by taking `onload` — that property
+belongs to the caller. A subscriber that throws is caught, because a bad
+listener must not be able to break a request.
+
+Where a body cannot be read safely — a stream, a huge payload, a binary blob —
+it is recorded as omitted with a reason, rather than read anyway.
+
+### It must never grow without limit
+
+The default store is a capped ring. Bodies over a size limit are recorded as
+omitted rather than copied. The reporter gives up after six consecutive
+failures. A tracer that leaks is worse than no tracer.
+
+### It must be removable
+
+`destroy()` restores every patched global and every axios adapter, including
+instances created through the patched `create()`. `fetch` is only restored if
+nothing else has patched it since — clobbering another tool's interceptor on the
+way out would be worse than leaving ours in place.
+
+## Two problems worth explaining
+
+### The cross-bundle registry
+
+A package with several entry points is bundled once per entry. So
+`api-tracer-kit` and `api-tracer-kit/axios` would each get their own copy of the
+module graph — and their own "shared" tracer, which is not shared at all.
+`useAxios()` would attach to an instance `getTraces()` never reads from. Two
+versions of the package in one dependency tree cause the same thing.
+
+`src/core/global.ts` anchors the instance on
+`globalThis[Symbol.for('api-tracer-kit.registry')]`, the one place every copy can
+agree on.
+
+### Counting an axios call once
+
+In a browser axios rides on `XMLHttpRequest`, so a call would be seen by both
+adapters. axios builds and opens its XHR **synchronously** inside its adapter, so
+the axios adapter raises a counter around the adapter call: it is still up when
+`open` and `send` run, and down again before anything else can start a request.
+That makes the suppression exact rather than time-based.
+
+The axios adapter wraps the instance's *adapter* rather than only its
+interceptors, because the adapter is the one place both the request and the raw
+response are available — and it is where the XHR is created, which is what makes
+the suppression possible. The property is installed as an accessor, so an
+application that later assigns `axios.defaults.adapter` replaces what the wrapper
+calls rather than the wrapper itself.
+
+## The scanner
+
+Regex-based, per preset, and honest about it. A parser would need a dependency
+per language and per flavour of syntax and would still not understand a template
+literal assembled from three variables.
+
+Which preset applies is decided by **running every preset over a sample of the
+codebase and keeping whichever finds the most**. Guessing by looking for marker
+strings was tried first and was wrong often enough to matter — a project can
+import axios and still not use it for its API layer. Running the presets asks
+the only question that counts: which one actually reads this code?
+
+The sample is weighted towards paths that look like an API layer, so the guess
+is made on the code that matters rather than the first hundred components in
+alphabetical order.
+
+What the scanner misses, live recording covers: a call no endpoint explains is
+adopted as an endpoint of its own rather than dropped.
+
+## The dashboard
+
+Vanilla JavaScript, no framework, no build step — it is served as static files
+straight from the package. JSON editors are syntax highlighted by a coloured
+layer behind a transparent textarea, so even that needs no editor library.
+
+This is deliberate. A dev tool that needs its own toolchain to be maintained is
+a dev tool nobody maintains.

package/docs/configuration.md

@@ -0,0 +1,275 @@
+# Configuration
+
+There is no required configuration. `api-tracer scan` works in a project it has
+never seen: it runs every scanner preset over your codebase and keeps whichever
+actually reads it, then reads your base URLs and auth header out of the source.
+
+A config file exists for what guessing cannot cover — a bespoke service shape, a
+base URL that is not written down, a sign-in flow worth automating.
+
+## The file
+
+One of these in your project root, checked in order:
+
+```
+api-tracer.config.mjs
+api-tracer.config.js
+api-tracer.config.json
+```
+
+Use `.mjs` unless your project is ESM already — a `.js` file with
+`export default` will not load inside a CommonJS package.
+
+Generate one filled in with what the scan worked out:
+
+```bash
+npx api-tracer init
+```
+
+It prints what each preset found before writing, so you can see the evidence:
+
+```
+1,284 source files scanned
+  service-object   325
+  fetch-direct       1
+  axios-direct       0
+```
+
+### Every field
+
+```js
+export default {
+  // shown in the console header and in Postman exports
+  name: 'My API',
+
+  // which preset reads this codebase; null or absent means guess
+  preset: 'service-object',
+
+  // for the service-object preset: the object key holding the path
+  pathKey: 'subUrl',
+
+  // where to look for API calls, relative to the project root
+  sources: ['src', 'app', 'lib', 'services', 'api'],
+
+  // never walked
+  ignore: ['node_modules', 'dist', 'build', 'coverage', '.git', '.next', 'out', 'vendor'],
+
+  extensions: ['.js', '.jsx', '.ts', '.tsx', '.mjs', '.cjs'],
+
+  // environment -> base URL; guessed from the source when absent
+  baseUrls: {
+    dev: 'https://dev-api.example.com',
+    stage: 'https://stage-api.example.com',
+  },
+
+  // the header the API authenticates with
+  auth: { header: 'Authorization' },
+
+  // how to read pass/fail out of a 2xx body; false turns it off
+  envelope: { codeFields: ['status', 'code'], okField: 'success', failFrom: 400 },
+
+  // the module an endpoint belongs to; the filename by default
+  moduleOf: (file, root) => file.split('/').at(-2),
+
+  // your own scanner, when no preset fits
+  parse: (src, config) => [],
+
+  // an optional multi-step sign-in chain
+  login: null,
+};
+```
+
+## Scanner presets
+
+### `service-object`
+
+One exported function per endpoint, building a request object:
+
+```js
+export const getPatients = (params) => {
+  const request = { subUrl: `/v1/patients.json`, params };
+  return get(request);
+};
+```
+
+Recognised verbs: `get`, `get2`, `getRequest`, `post`, `postRequest`, `put`,
+`patch`, `delete`, `deletee`, `del`, `destroy`. The request argument may be
+named `request`, `config`, `options` or `req`. Change the path key with
+`pathKey` if yours is not `subUrl`.
+
+### `axios-direct`
+
+```js
+export const listUsers = () => axios.get('/api/users', { params: { page: 1 } });
+export const addUser = (body) => api.post(`/api/users/${body.id}`, body);
+```
+
+Any identifier works, not just `axios` — `api`, `http`, `client`.
+
+### `fetch-direct`
+
+```js
+export const load = () => fetch('/api/things');
+export const save = (b) => fetch('/api/things', { method: 'POST', body: JSON.stringify(b) });
+```
+
+### Path placeholders
+
+All three read `${expr}`, `:name` and `{name}` as holes, so Express- and
+OpenAPI-style paths work as well as template literals.
+
+### Why regexes and not a parser
+
+A parser would need a dependency per language and per flavour of syntax, and
+would still not understand a template literal assembled from three variables. A
+scanner that reads 90% of a codebase and says what it could not read is more
+useful than a parser that refuses to start. Live recording covers the rest: a
+call the scan cannot explain is adopted as an endpoint of its own.
+
+Commented-out lines are blanked before scanning, with line numbers preserved.
+Services routinely keep the previous URL directly above the live one, and
+reading the first match would silently take the dead path.
+
+### Your own scanner
+
+When no preset fits, supply `parse` rather than forking the tool. It is called
+once per file with the source (comments already stripped) and the config, and
+returns one object per endpoint:
+
+```js
+export default {
+  parse(src, config) {
+    return [...src.matchAll(/route\("(\w+)",\s*"([^"]+)"\)/g)].map((m) => ({
+      name: m[2].replace(/\W+/g, '_'),   // the endpoint's function name
+      method: m[1].toUpperCase(),        // GET, POST, ...
+      subUrl: m[2],                      // the path template
+      usesParams: m[1] === 'get',        // sends a query string
+      usesData: m[1] !== 'get',          // sends a body
+      absolute: /^https?:\/\//.test(m[2]),
+      chunk: src.slice(m.index, m.index + 300),  // scanned for auth quirks
+      index: m.index,                    // for the source line number
+    }));
+  },
+};
+```
+
+`parse` takes precedence over `preset`. Throwing on one file skips that file
+rather than failing the scan.
+
+## Base URLs and the auth header
+
+Both are read out of your source by default, so your environments stay in sync
+with the app rather than being copied into a config that drifts.
+
+Base URLs are found by looking for an object mapping environment names to URLs:
+
+```js
+const apiBaseUrls = {
+  dev: 'https://dev-api.example.com',
+  prod: 'https://api.example.com',
+};
+```
+
+Recognised environment names: `dev`, `develop`, `development`, `stage`,
+`staging`, `uat`, `qa`, `test`, `prod`, `production`, `local`. Failing that, the
+first API-looking URL becomes `dev`.
+
+The auth header is found by looking for a named constant — the strongest signal,
+because a codebase that uses the header in more than one place tends to write it
+that way:
+
+```js
+const authTokenKey = 'AUTH_TOKEN';
+```
+
+then for header names used as object keys or header lookups.
+
+Set either explicitly in the config to override. Consider commenting out a
+production base URL in a generated config, so nobody replays writes against live
+data by selecting the wrong environment.
+
+## Sign-in flows
+
+Plenty of APIs need more than one call to hand out a usable token. Rather than
+make everyone paste one, describe the chain and the console will drive it.
+
+Each step declares the fields to ask for, the call to make, and what to do with
+the answer:
+
+```js
+export default {
+  login: {
+    // optional: a short-lived signed JWT sent with every step
+    jwt: {
+      secret: process.env.API_SIGNING_KEY,
+      secretProd: process.env.API_SIGNING_KEY_PROD,
+      headerName: 'ACCESS_TOKEN',
+      timestampPrefix: 'MyApp-',
+      ttlSeconds: 600,
+      // or build the payload yourself
+      payload: ({ now, secretCode, env }) => ({ iat: now, exp: now + 600 }),
+    },
+
+    steps: [
+      {
+        name: 'credentials',
+        title: 'Sign in with your email and password.',
+        fields: [
+          { name: 'email', label: 'Email', type: 'email' },
+          { name: 'password', label: 'Password', type: 'password' },
+        ],
+        // the scanned endpoint id, so a path change in the app is picked up;
+        // `path` is the fallback when there is no catalog entry
+        endpointId: 'session.loginApi',
+        path: '/users/login.json',
+        body: (input) => ({ user: input }),
+        then: (data) => ({
+          state: { apiToken: data.api_token, phone: data.phone },
+          next: 'code',
+        }),
+      },
+      {
+        name: 'code',
+        title: 'Enter the code we sent you.',
+        fields: [{ name: 'otp', label: 'Code', inputmode: 'numeric' }],
+        path: '/users/verify_otp.json',
+        headers: (state) => ({ SECRET_TOKEN: state.apiToken }),
+        body: (input, state) => ({ otp: input.otp, phone: state.phone }),
+        then: (data) => ({
+          token: data.auth_token,
+          owner: { name: data.name, email: data.email },
+        }),
+      },
+    ],
+  },
+};
+```
+
+### The step contract
+
+| Field | |
+| --- | --- |
+| `name` | the step's id, used as `next` from the previous step |
+| `title` | shown above the fields |
+| `fields` | `{ name, label?, type?, inputmode?, placeholder?, options? }`. `options` renders a select. |
+| `endpointId` | a scanned endpoint whose path to use |
+| `path` | the path to POST to, when `endpointId` is absent or unmatched |
+| `headers(state)` | extra headers for this step |
+| `body(input, state)` | the request body from the submitted fields and the carried state |
+| `then(data, state, input)` | what to do with the response |
+
+`then` returns any of:
+
+| | |
+| --- | --- |
+| `state` | merged into the carried state for later steps |
+| `next` | the name of the step to show next |
+| `choices` | options for a select on the next step |
+| `token` | the auth token — ends the flow and stores it |
+| `owner` | `{ name, email }`, shown on a shared console so people know whose session they are replaying as |
+
+The password is forwarded once and is never stored, logged, or written to disk.
+Only the resulting token is kept.
+
+A project with no `login` config gets the paste-a-token form, which is all most
+APIs need.