@binary-balance/seshat 0.1.0 → 0.3.0

package/README.md

@@ -1,129 +1,155 @@
 # Seshat
 
-Seshat is a native CLI for TypeScript and TSX complexity analysis and mutation
-testing. `check` runs both assessments, `crap` runs complexity and coverage
-analysis, and `mutate` runs mutation testing from the original test baseline.
+> Seshat was the ancient Egyptian goddess of writing, wisdom, and knowledge. Seen as a scribe and record keeper, she became identified as the goddess of measurement, accounting, architecture, science, astronomy, mathematics, geometry, history and surveying.
 
-## Install
-
-Install the entry package in the project you want to assess:
-
-```sh
-npm install --save-dev @binary-balance/seshat
-```
+A native code-assurance tool for TypeScript and TSX, written in Rust. Seshat
+combines function-level [CRAP](https://testing.googleblog.com/2011/02/this-code-is-crap.html)
+analysis with [comparison-operator mutation testing](https://en.wikipedia.org/wiki/Mutation_testing) to find complex and/or
+insufficiently tested code.
 
-The package declares and verifies Node.js 24.20.0 exactly. It selects one
-native payload for the current platform from these five verified targets:
+## What Seshat does
 
-- Linux x64 (glibc; Debian 11 userspace, glibc 2.31)
-- Linux ARM64 (glibc; Ubuntu 22.04, glibc 2.35)
-- macOS x64 (macOS 15.0 minimum)
-- macOS ARM64 (macOS 15.0 minimum)
-- Windows x64 (Windows Server 2022 verification)
+- Calculates function complexity and CRAP scores from branch coverage.
+- Changes comparison operators one at a time and checks whether tests detect them.
+- Runs tests in isolated copies of the captured project.
+- Produces readable terminal output and versioned JSON, with optional CI thresholds.
+- Supports explicit source selection, multiple test setups and parallel mutation workers.
 
-These are the known verification floors; see the [native platform support
-matrix](https://github.com/Binary-Balance/seshat/blob/main/docs/platform-support.md)
-for the detailed target evidence.
-
-## Minimal mutation example
-
-For a small Node test setup, install TypeScript in the project first:
+## Install
 
 ```sh
-npm install --save-dev typescript
+npm install --save-dev @binary-balance/seshat
 ```
 
-Create `src/rules.mts`:
-
-```ts
-export function classify(value: number): 'positive' | 'negative' {
-  return value >= 0 ? 'positive' : 'negative';
-}
-```
+Seshat needs Node `>=24.20.0 <25`. npm installs a prebuilt binary for Linux
+x64/ARM64 (glibc), macOS x64/ARM64 or Windows x64, so Rust is not required.
+Alpine Linux, which uses musl instead of glibc, is not supported. See the
+[platform support matrix](https://github.com/Binary-Balance/seshat/blob/main/docs/platform-support.md) for details.
 
-Create `tests/rules.test.mjs`:
+The project being assessed must already have its own dependencies installed,
+including TypeScript, its test runner and a coverage tool.
 
-```js
-import assert from 'node:assert/strict';
-import {test} from 'node:test';
-import {classify} from '../src/rules.mts';
+## Supported test runners
 
-test('classifies zero and positive values', () => {
-  assert.equal(classify(0), 'positive');
-  assert.equal(classify(1), 'positive');
-});
-```
+| Runner | Supported versions |
+| --- | --- |
+| Node's built-in test runner | Node `>=24.20.0 <25` |
+| Vitest | `>=5.0.0 <6` |
+| Jest with Expo | Jest `29.7.0` with jest-expo `57.0.5` |
 
-Create `tsconfig.json`:
+Seshat checks the runner version on every run and stops if it is not
+supported. See the [runner compatibility guide](https://github.com/Binary-Balance/seshat/blob/main/docs/runner-compatibility.md).
 
-```json
-{
-  "compilerOptions": {
-    "strict": true,
-    "noEmit": true,
-    "skipLibCheck": true,
-    "target": "ES2022",
-    "module": "NodeNext",
-    "moduleResolution": "NodeNext",
-    "allowImportingTsExtensions": true
-  },
-  "include": ["src/**/*.mts"]
-}
-```
+## Configure
 
-Create `seshat.json`:
+Add a `seshat.json` to the project root. This one is for Node's built-in test
+runner:
 
 ```json
 {
-  "source": {"include": ["src/**/*.mts"]},
+  "source": {
+    "include": ["src/**/*.ts"]
+  },
   "capture": [
-    "package.json",
-    "package-lock.json",
-    "tsconfig.json",
-    "src",
-    "tests",
-    "node_modules"
+    "package.json", "package-lock.json", "tsconfig.json", "collect-node.mjs",
+    "src", "tests", "node_modules"
   ],
   "setups": [{
     "name": "node",
     "runner": "node",
     "cwd": ".",
-    "typecheck": [
-      "node",
-      "node_modules/typescript/bin/tsc",
-      "--project",
-      "tsconfig.json"
-    ],
+    "typecheck": ["node", "node_modules/typescript/bin/tsc", "--project", "tsconfig.json"],
     "test": [
-      "node",
-      "--test",
-      "--test-concurrency=1",
-      "--test-reporter={seshatReporter}",
-      "tests/rules.test.mjs"
+      "node", "--test", "--test-concurrency=1",
+      "--test-reporter={seshatReporter}", "tests/rules.test.mjs"
     ],
     "coverage": {
-      "command": ["node", "-e", "process.exit(0)"],
-      "report": "coverage/unused.json"
+      "command": ["node", "collect-node.mjs", "tests/rules.test.mjs"],
+      "report": "coverage/coverage-final.json"
     }
   }]
 }
 ```
 
-Run mutation testing and save its JSON report:
+- `source` selects the files to assess, using glob patterns.
+- `capture` lists the files and directories copied into each isolated run. It
+  must include the selected source and everything the commands need.
+- `setups` gives the typecheck, test and coverage commands. The coverage
+  command must write an Istanbul JSON report; `collect-node.mjs` is a small
+  adapter from the [Node example](https://github.com/Binary-Balance/seshat/tree/main/examples/node).
+
+The [examples](https://github.com/Binary-Balance/seshat/blob/main/examples/README.md) include complete Node, npm workspace, Vitest
+and Jest/Expo projects. The [configuration guide](https://github.com/Binary-Balance/seshat/blob/main/docs/configuration.md) covers
+every field, including `workers` for parallel mutation runs.
+
+## Use
 
 ```sh
-./node_modules/.bin/seshat mutate --config ./seshat.json \
-  --json --no-progress > seshat-report.json
+npx @binary-balance/seshat check    # CRAP and mutation testing
+npx @binary-balance/seshat crap     # coverage and CRAP only
+npx @binary-balance/seshat mutate   # mutation testing only
+```
+
+Use the full package name with `npx`. A bare `npx seshat` can fetch an
+unrelated package of that name.
+
+Each command reads `./seshat.json`; use `--config PATH` for another file. Add
+`--json` to write a report to stdout, as described in the
+[report format](https://github.com/Binary-Balance/seshat/blob/main/docs/report-format.md). `--help` lists all options.
+
+Run only trusted test commands. Seshat's copies protect the project checkout,
+but tests can still access external files, services, databases and credentials.
+
+## Understand the results
+
+CRAP uses complexity and branch coverage, a fraction from 0 to 1. Functions
+with no recorded branches use statement coverage instead:
+
+```text
+complexity^2 * (1 - coverage)^3 + complexity
+```
+
+For complexity 10, zero coverage gives `110`, 50% coverage gives `22.5` and
+full coverage gives `10`.
+
+Statement coverage counts code that ran; branch coverage counts the recorded
+outcomes of decisions. For example, testing only the member price in
+`if (member) total *= 0.9; return total;` can run every statement while
+leaving the non-member outcome untested. Its statement coverage is 100%,
+but its branch coverage is 50%. Both measurements appear in the report.
+
+The mutation score is the percentage of mutants that made a test fail. A mutant
+is one changed comparison, such as `>` to `>=`. If tests catch two of three
+mutants, the score is `66.67%`. A timeout or execution error leaves the run
+incomplete and withholds the score. Neither score proves correctness.
+
+## Use in CI
+
+Add optional thresholds to `seshat.json`:
+
+```json
+"thresholds": {
+  "maxCrap": 30,
+  "minMutationScore": 80
+}
 ```
 
-The `coverage` block is required by the configuration schema but is not run by
-`mutate`; the no-op command above keeps this example dependency-light. `check`
-and `crap` do run coverage and need a command that writes a fresh Istanbul JSON
-report. Use the complete [Node example](https://github.com/Binary-Balance/seshat/tree/main/examples/node),
-[Vitest example](https://github.com/Binary-Balance/seshat/tree/main/examples/vitest),
-or [Jest/Expo example](https://github.com/Binary-Balance/seshat/tree/main/examples/jest-expo)
-for those commands.
-
-See the [full configuration guide](https://github.com/Binary-Balance/seshat/blob/main/docs/configuration.md)
-for all fields and runner setup, and the [JSON report format](https://github.com/Binary-Balance/seshat/blob/main/docs/report-format.md)
-for the `--json` output contract.
+| Exit status | Meaning |
+| ---: | --- |
+| `0` | Complete run, thresholds met. |
+| `1` | Complete run, a threshold was not met. |
+| `2` | Invalid input, failed baseline or incomplete run. |
+| `130` / `143` | Cancelled by `SIGINT` / `SIGTERM` on Unix. |
+
+See [quality thresholds](https://github.com/Binary-Balance/seshat/blob/main/docs/configuration.md#quality-thresholds) for the full
+rules.
+
+## Development
+
+See the [architecture](https://github.com/Binary-Balance/seshat/blob/main/docs/architecture.md), the
+[build and regression guide](https://github.com/Binary-Balance/seshat/blob/main/benchmarks/proofs/README.md) and the
+[packaging guide](https://github.com/Binary-Balance/seshat/blob/main/packaging/README.md).
+
+## License
+
+[MIT](https://github.com/Binary-Balance/seshat/blob/main/LICENSE).

package/bin/seshat.mjs

@@ -2,6 +2,7 @@
 
 import {spawn} from 'node:child_process';
 import {createRequire} from 'node:module';
+import {constants} from 'node:os';
 import {existsSync, readFileSync, statSync} from 'node:fs';
 import {basename, dirname, join} from 'node:path';
 import {fileURLToPath} from 'node:url';
@@ -11,6 +12,8 @@ const packageRoot = dirname(dirname(fileURLToPath(import.meta.url)));
 const packageManifest = JSON.parse(readFileSync(join(packageRoot, 'package.json'), 'utf8'));
 const version = packageManifest.version;
 const args = process.argv.slice(2);
+// Match native argument errors: no -- separator or --flag=value syntax exists,
+// and path values cannot start with --. A literal --json always requests JSON.
 const jsonRequested = args.includes('--json');
 const commands = new Set(['check', 'crap', 'mutate']);
 const started = performance.now();
@@ -87,10 +90,6 @@ function payloadExecutable(target) {
   return executable;
 }
 
-function signalNumber(signal) {
-  return {SIGINT: 2, SIGTERM: 15}[signal] ?? 1;
-}
-
 const host = targetForHost();
 if (host.error) {
   fail(host.error);
@@ -102,37 +101,39 @@ if (host.error) {
     fail(error instanceof Error ? error.message : String(error));
   }
   if (executable) {
-    const child = spawn(executable, args, {
-      cwd: process.cwd(),
-      env: {...process.env},
-      shell: false,
-      windowsHide: false,
-      stdio: 'inherit',
-    });
-    let spawnFailed = false;
-    const forward = signal => {
-      if (child.exitCode === null) child.kill(signal);
-    };
-    // A CTRL_BREAK_EVENT is broadcast to the console process group. Keep the
-    // launcher alive on Windows so the native child can handle that event and
-    // return its cancellation report through inherited stdio.
-    const preserveConsole = () => {};
-    process.on('SIGINT', forward);
-    process.on('SIGTERM', forward);
-    if (process.platform === 'win32') process.on('SIGBREAK', preserveConsole);
-    const cleanup = () => {
-      process.removeListener('SIGINT', forward);
-      process.removeListener('SIGTERM', forward);
-      if (process.platform === 'win32') process.removeListener('SIGBREAK', preserveConsole);
-    };
-    child.once('error', error => {
-      spawnFailed = true;
-      cleanup();
+    let child;
+    try {
+      child = spawn(executable, args, {
+        cwd: process.cwd(),
+        env: {...process.env},
+        shell: false,
+        windowsHide: false,
+        stdio: 'inherit',
+      });
+    } catch (error) {
       fail(`cannot start native payload: ${error.message}`);
-    });
-    child.once('close', (code, signal) => {
-      cleanup();
-      if (!spawnFailed) process.exitCode = signal ? 128 + signalNumber(signal) : code ?? 1;
-    });
+    }
+    if (child) {
+      let spawnFailed = false;
+      const forward = signal => {
+        if (!spawnFailed && child.exitCode === null && child.signalCode === null) child.kill(signal);
+      };
+      // Windows delivers Ctrl+C/Break to the shared console. Forwarding SIGINT
+      // with child.kill() force-terminates the native child before it can clean up.
+      const preserveConsole = () => {};
+      const interrupt = process.platform === 'win32' ? preserveConsole : forward;
+      process.on('SIGINT', interrupt);
+      process.on('SIGTERM', forward);
+      if (process.platform === 'win32') process.on('SIGBREAK', preserveConsole);
+      // Keep handlers until launcher exit: a second console/group signal can arrive
+      // after child close and must not replace the child's already chosen exit code.
+      child.once('error', error => {
+        spawnFailed = true;
+        fail(`cannot start native payload: ${error.message}`);
+      });
+      child.once('close', (code, signal) => {
+        if (!spawnFailed) process.exitCode = signal ? 128 + (constants.signals[signal] ?? 1) : code ?? 1;
+      });
+    }
   }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@binary-balance/seshat",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "A native CLI for TypeScript and TSX complexity analysis and mutation testing.",
   "license": "MIT",
   "repository": {
@@ -9,7 +9,7 @@
   },
   "type": "module",
   "engines": {
-    "node": "24.20.0"
+    "node": ">=24.20.0 <25"
   },
   "bin": {
     "seshat": "bin/seshat.mjs"
@@ -20,10 +20,10 @@
     "LICENSE"
   ],
   "optionalDependencies": {
-    "@binary-balance/seshat-linux-x64": "0.1.0",
-    "@binary-balance/seshat-linux-arm64": "0.1.0",
-    "@binary-balance/seshat-darwin-x64": "0.1.0",
-    "@binary-balance/seshat-darwin-arm64": "0.1.0",
-    "@binary-balance/seshat-win32-x64": "0.1.0"
+    "@binary-balance/seshat-linux-x64": "0.3.0",
+    "@binary-balance/seshat-linux-arm64": "0.3.0",
+    "@binary-balance/seshat-darwin-x64": "0.3.0",
+    "@binary-balance/seshat-darwin-arm64": "0.3.0",
+    "@binary-balance/seshat-win32-x64": "0.3.0"
   }
 }