nano-benchmark 1.0.8 → 1.0.10

package/README.md

@@ -1,20 +1,19 @@
 # nano-benchmark [![NPM version][npm-img]][npm-url]
 
-[npm-img]:      https://img.shields.io/npm/v/nano-benchmark.svg
-[npm-url]:      https://npmjs.org/package/nano-benchmark
+[npm-img]: https://img.shields.io/npm/v/nano-benchmark.svg
+[npm-url]: https://npmjs.org/package/nano-benchmark
 
-`nano-benchmark` provides command-line utilities for benchmarking code and related statistical modules.
+`nano-benchmark` provides command-line utilities for micro-benchmarking code
+with nonparametric statistics and significance testing.
 
 Two utilities are available:
 
-* `nano-watch` — provides statistics in a streaming mode continuously running your code,
-  watching memory usage and updating the output.
-* `nano-bench` — runs benchmark tests on your code, calculating statistics and
-  statistical significance, and presenting them in a tabular format.
+- `nano-watch` — continuously benchmarks a single function, showing live statistics
+  and memory usage.
+- `nano-bench` — benchmarks and compares multiple functions, calculating confidence
+  intervals and statistical significance.
 
-The utilities are mostly used to measure performance of your code and compare it with other variants.
-It is geared toward benchmarking and performance tuning of small fast snippets of code, e.g.,
-used in tight loops.
+Designed for performance tuning of small, fast code snippets used in tight loops.
 
 ## Visual samples
 
@@ -29,19 +28,14 @@ used in tight loops.
 ## Installation
 
 ```bash
-npm install --save nano-benchmark
+npm install nano-benchmark
 ```
 
 ### Deno and Bun support
 
 Both [deno](https://deno.land/) and [bun](https://bun.sh/) are supported.
 
-If you want to run the benchmark in Deno, Bun, etc. you can specify `self` as the `file` argument
-or the `--self` option.
-In this case the utility will print out its file name to `stdout` and exit. It allows running
-the utility with alternative JavaScript interpreters.
-
-Examples with `bash`:
+Use `--self` to get the script path for running with alternative interpreters:
 
 ```bash
 npx nano-bench benchmark.js
@@ -51,9 +45,8 @@ deno run -A `npx nano-bench --self` benchmark.js
 node `npx nano-bench --self` benchmark.js
 ```
 
-Don't forget to specify the appropriate permissions for Deno to run the benchmark scripts:
-`--allow-read` (required) and `--allow-hrtime` (optional but recommended). Or consider using
-`-A` or `--allow-all` to allow all permissions (used it only in safe environments!).
+For Deno, `--allow-read` is required and `--allow-hrtime` is recommended.
+Use `-A` for convenience in safe environments.
 
 ## Documentation
 
@@ -64,10 +57,9 @@ your `package.json` file or from the command line by prefixing them with `npx`,
 
 Utilities are self-documented — run them with `--help` flag to learn about arguments.
 
-Both utilities import a module to benchmark using its (default) export.
-`nano-bench` assumes that it is an object with functional properties,
-which should be benchmarked and compared. `nano-watch` can use the same file format
-as `nano-bench` or it can use a single function.
+Both utilities import a module and benchmark its (default) export.
+`nano-bench` expects an object whose properties are the functions to compare.
+`nano-watch` accepts the same format or a single function.
 
 Example of a module for `nano-bench` called `bench-strings-concat.js`:
 
@@ -97,7 +89,7 @@ export default {
 };
 ```
 
-The way to use it:
+Usage:
 
 ```bash
 npx nano-bench bench-strings-concat.js
@@ -106,18 +98,32 @@ npx nano-watch bench-strings-concat.js backticks
 
 See [wiki](https://github.com/uhop/nano-bench/wiki) for more details.
 
+## AI agents and contributing
+
+If you are an AI agent or an AI-assisted developer working on this project, read
+[AGENTS.md](./AGENTS.md) first — it contains the project rules and conventions.
+
+Other useful files:
+
+- [ARCHITECTURE.md](./ARCHITECTURE.md) — module map, dependency graph, how benchmarking works.
+- [CONTRIBUTING.md](./CONTRIBUTING.md) — development workflow and coding conventions.
+- [llms.txt](./llms.txt) — project summary for LLMs.
+- [llms-full.txt](./llms-full.txt) — detailed CLI reference for LLMs.
+
 ## License
 
 BSD 3-Clause License
 
 ## Release history
 
-* 1.0.8: *Updated dependencies.*
-* 1.0.7: *Updated dependencies.*
-* 1.0.6: *Updated dependencies.*
-* 1.0.5: *Updated dependencies.*
-* 1.0.4: *Updated dependencies + added more tests.*
-* 1.0.3: *Updated dependencies.*
-* 1.0.2: *Added the `--self` option.*
-* 1.0.1: *Added "self" argument to utilities so it can be used with Deno, Bun, etc.*
-* 1.0.0: *Initial release.*
+- 1.0.10: _Added Prettier lint scripts, GitHub issue templates, Copilot instructions, and Windsurf workflows._
+- 1.0.9: _Updated dependencies._
+- 1.0.8: _Updated dependencies._
+- 1.0.7: _Updated dependencies._
+- 1.0.6: _Updated dependencies._
+- 1.0.5: _Updated dependencies._
+- 1.0.4: _Updated dependencies + added more tests._
+- 1.0.3: _Updated dependencies._
+- 1.0.2: _Added the `--self` option._
+- 1.0.1: _Added "self" argument to utilities so it can be used with Deno, Bun, etc._
+- 1.0.0: _Initial release._

package/bin/nano-bench.js

@@ -299,8 +299,6 @@ if (results.length > 1) {
   if (significance) {
     const sortedStats = stats.slice().sort((a, b) => a.median - b.median),
       tableData = [['  ', bold('#'), bold('name')]];
-    let rabbitIndex = -1,
-      turtleIndex = -1;
     for (let i = 0; i < names.length; ++i) {
       tableData[0].push({value: bold(formatInteger(i + 1)), align: 'c'});
       const row = [null, formatInteger(i + 1), bold(names[i])],

package/llms-full.txt

@@ -0,0 +1,208 @@
+# nano-benchmark
+
+> Command-line utilities for micro-benchmarking JavaScript code with nonparametric statistics and significance testing.
+
+- NPM: https://npmjs.org/package/nano-benchmark
+- GitHub: https://github.com/uhop/nano-bench
+- Wiki: https://github.com/uhop/nano-bench/wiki
+- License: BSD-3-Clause
+- Runtime: Node.js 20+, Bun, Deno
+- Module system: ESM only (`"type": "module"`)
+
+## Installation
+
+```bash
+npm install nano-benchmark
+```
+
+## CLI tool: nano-bench
+
+Benchmarks multiple functions, compares them with bootstrap confidence intervals and significance tests, outputs a styled table.
+
+### Usage
+
+```
+nano-bench [options] <file>
+```
+
+### Arguments
+
+- `file` — JavaScript module to benchmark. If `"self"`, prints its own file path and exits.
+
+### Options
+
+- `-m, --ms <ms>` — measurement time in milliseconds per sample (default: 50). The tool auto-discovers the batch size where one call takes at least this long.
+- `-i, --iterations <iterations>` — fixed iteration count per sample (overrides `--ms`).
+- `--min-iterations <n>` — minimum iterations per sample (default: 1).
+- `-s, --samples <samples>` — number of samples to collect (default: 100).
+- `-b, --bootstrap <bootstrap>` — number of bootstrap resamples for CI estimation (default: 1000).
+- `-a, --alpha <alpha>` — significance level for confidence interval and tests (default: 0.05 = 95% CI).
+- `-p, --parallel` — collect samples in parallel (useful for async benchmarks).
+- `-e, --export <name>` — name of the export to use from the file (default: `"default"`).
+- `--self` — print the script's file path to stdout and exit (for Deno/Bun usage).
+
+### Output
+
+A styled table with columns:
+
+| Column | Description |
+|--------|-------------|
+| name | Function name |
+| median | Median execution time |
+| + | Upper bound of confidence interval (median to high) |
+| − | Lower bound of confidence interval (median to low) |
+| op/s | Operations per second (1000 / median) |
+| batch | Iterations per sample (batch size) |
+
+If differences are statistically significant, a significance matrix is printed showing pairwise comparisons with percentage or ratio differences. The fastest function is marked with 🐇 and the slowest with 🐢.
+
+### How it works
+
+1. **Find level**: auto-discovers batch size `n` where `fn(n)` takes ≥ `--ms` milliseconds.
+2. **Collect samples**: runs `fn(n)` `--samples` times, collecting timing data.
+3. **Bootstrap CI**: uses bootstrap resampling to estimate median and confidence interval.
+4. **Significance test**: Mann-Whitney U (2 functions) or Kruskal-Wallis with post-hoc tests (3+ functions).
+
+### Example
+
+```bash
+npx nano-bench bench/bench-string-concat.js
+npx nano-bench -s 200 -b 2000 -a 0.01 bench/bench-string-concat.js
+```
+
+---
+
+## CLI tool: nano-watch
+
+Continuously benchmarks a single function in streaming mode, showing live statistics and memory usage. Runs indefinitely until stopped with Ctrl+C (or until `--iterations` is reached).
+
+### Usage
+
+```
+nano-watch [options] <file> [method]
+```
+
+### Arguments
+
+- `file` — JavaScript module to benchmark. If `"self"`, prints its own file path and exits.
+- `method` — optional method name if the export is an object of functions (same format as nano-bench).
+
+### Options
+
+- `-m, --ms <ms>` — milliseconds per measurement iteration (default: 500).
+- `-i, --iterations <number>` — number of iterations to run (default: Infinity).
+- `-e, --export <name>` — name of the export to use from the file (default: `"default"`).
+- `--self` — print the script's file path to stdout and exit.
+
+### Output
+
+A live-updating styled table with:
+
+| Row | Columns |
+|-----|---------|
+| Stats | #, time, mean, stdDev, median, skewness, kurtosis |
+| op/s | operations per second for time, mean, median |
+| memory | heapUsed, heapTotal, rss |
+
+All statistics are computed using online/streaming algorithms (constant memory):
+- **StatCounter**: streaming mean, variance, skewness, kurtosis (Welford's algorithm).
+- **MedianCounter**: approximate streaming median (median-of-medians).
+
+### Example
+
+```bash
+npx nano-watch bench/watch-sample.js
+npx nano-watch bench/bench-string-concat.js backticks
+npx nano-watch -i 50 bench/watch-sample.js
+```
+
+---
+
+## Benchmark file format
+
+Both tools import a JavaScript module. The module should export (default or named) either:
+
+### Object of functions (for nano-bench, or nano-watch with method argument)
+
+```js
+export default {
+  variant1: n => {
+    const a = 'a', b = 'b';
+    for (let i = 0; i < n; ++i) {
+      const x = a + '-' + b;
+    }
+  },
+  variant2: n => {
+    const a = 'a', b = 'b';
+    for (let i = 0; i < n; ++i) {
+      const x = `${a}-${b}`;
+    }
+  }
+};
+```
+
+### Single function (for nano-watch without method argument)
+
+```js
+export default n => {
+  const a = 'a', b = 'b';
+  for (let i = 0; i < n; ++i) {
+    const x = a + '-' + b;
+  }
+};
+```
+
+### Key design principle
+
+Each function takes `n` (iteration count) and runs the measured code in a `for` loop. This amortizes function-call overhead over `n` iterations, which is critical for micro-benchmarks where the measured code is faster than the overhead of calling a function.
+
+The batch size `n` is either specified via `--iterations` or auto-discovered by the tool so that one call takes at least `--ms` milliseconds.
+
+### Async functions
+
+Benchmark functions can return a Promise (be async). The tools detect thenables and measure the time until resolution.
+
+---
+
+## Deno and Bun support
+
+Use `--self` to get the script path, then run with the alternative interpreter:
+
+```bash
+# nano-bench
+bun `npx nano-bench --self` benchmark.js
+deno run -A `npx nano-bench --self` benchmark.js
+
+# nano-watch
+bun `npx nano-watch --self` benchmark.js methodName
+deno run -A `npx nano-watch --self` benchmark.js methodName
+```
+
+For Deno, `--allow-read` is required and `--allow-hrtime` is recommended. Use `-A` for convenience in safe environments.
+
+---
+
+## Statistical methods
+
+### Bootstrap resampling
+
+Used by nano-bench to estimate confidence intervals. Resamples the collected timing data `--bootstrap` times, computing the median of each resample, then takes the mean of those medians for the final estimate.
+
+### Mann-Whitney U test
+
+Nonparametric two-sample test used when comparing exactly 2 functions. Tests whether the two timing distributions are significantly different at the given `--alpha` level. Does not assume normal distribution.
+
+### Kruskal-Wallis test
+
+Nonparametric k-sample test used when comparing 3+ functions. Uses beta approximation for the critical value. If significant, performs post-hoc pairwise comparisons to identify which specific pairs differ.
+
+### Kolmogorov-Smirnov test
+
+Two-sample distribution comparison test. Available in the internal API (`src/significance/kstest.js`).
+
+### Online/streaming algorithms
+
+Used by nano-watch for indefinite monitoring with constant memory:
+
+- **StatCounter** — Welford's online algorithm for streaming mean, variance (M2), skewness (M3), and kurtosis (M4). Numerically stable single-pass computation.
+- **MedianCounter** — approximate streaming median using a hierarchical median-of-three structure. Provides O(1) memory approximate median without storing all values.

package/llms.txt

@@ -0,0 +1,80 @@
+# nano-benchmark
+
+> Command-line utilities for micro-benchmarking JavaScript code with nonparametric statistics and significance testing.
+
+## Overview
+
+nano-benchmark provides two CLI tools for benchmarking code. It uses nonparametric statistics (bootstrap resampling, quantile-based confidence intervals) and rank-based significance tests (Mann-Whitney U, Kruskal-Wallis) to produce statistically rigorous results. Designed for micro-benchmarks where function-call overhead matters.
+
+- NPM: https://npmjs.org/package/nano-benchmark
+- GitHub: https://github.com/uhop/nano-bench
+- Wiki: https://github.com/uhop/nano-bench/wiki
+- License: BSD-3-Clause
+- Runtime: Node.js 20+, Bun, Deno
+- Module system: ESM only (`"type": "module"`)
+
+## Installation
+
+```bash
+npm install nano-benchmark
+```
+
+## CLI tools
+
+### nano-bench
+
+Benchmarks multiple functions, compares them with bootstrap confidence intervals and significance tests, outputs a styled table.
+
+```bash
+npx nano-bench benchmark.js
+npx nano-bench -s 200 -b 2000 -a 0.01 benchmark.js
+```
+
+Options: `--ms` (measurement time, default 50), `--iterations` (overrides --ms), `--samples` (default 100), `--bootstrap` (default 1000), `--alpha` (significance level, default 0.05), `--parallel`, `--export` (default "default"), `--self`.
+
+### nano-watch
+
+Continuously benchmarks a single function in streaming mode, showing live stats (mean, stdDev, median, skewness, kurtosis, ops/sec) and memory usage.
+
+```bash
+npx nano-watch benchmark.js
+npx nano-watch benchmark.js methodName
+```
+
+Options: `--ms` (measurement time, default 500), `--iterations` (default Infinity), `--export` (default "default"), `--self`.
+
+## Benchmark file format
+
+Both tools import a module. `nano-bench` expects an object of functions; `nano-watch` can use a single function or an object with a method name argument. Each function takes `n` (iteration count) and runs the measured code in a loop:
+
+```js
+export default {
+  variant1: n => {
+    for (let i = 0; i < n; ++i) {
+      // measured code
+    }
+  },
+  variant2: n => {
+    for (let i = 0; i < n; ++i) {
+      // measured code
+    }
+  }
+};
+```
+
+## Deno and Bun support
+
+Use `--self` to get the script path for running with alternative interpreters:
+
+```bash
+bun `npx nano-bench --self` benchmark.js
+deno run -A `npx nano-bench --self` benchmark.js
+```
+
+## Statistical methods
+
+- **Bootstrap resampling** for confidence intervals (median, percentiles).
+- **Mann-Whitney U test** for comparing two samples.
+- **Kruskal-Wallis test** with post-hoc pairwise tests for comparing 3+ samples.
+- **Kolmogorov-Smirnov test** for two-sample distribution comparison.
+- **Online algorithms** (streaming mean, variance, skewness, kurtosis, approximate median) for continuous monitoring.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "nano-benchmark",
-  "version": "1.0.8",
-  "description": "Small utilities to benchmark code with Node.",
+  "version": "1.0.10",
+  "description": "CLI micro-benchmarking with nonparametric statistics and significance testing.",
   "type": "module",
   "main": "src/index.js",
   "exports": {
@@ -17,8 +17,13 @@
     "test:bun": "tape6-bun --flags FO",
     "test:deno": "tape6-deno --flags FO",
     "test:proc": "tape6-proc --flags FO",
-    "test:proc:bun": "bun run `npx tape6-proc --self` --flags FO",
-    "test:proc:deno": "deno run -A `npx tape6-proc --self` --flags FO -r -A"
+    "test:proc:bun": "bun run `tape6-proc --self` --flags FO",
+    "test:proc:deno": "deno run -A `tape6-proc --self` --flags FO -r -A",
+    "test:seq": "tape6-seq --flags FO",
+    "test:seq:bun": "bun run `tape6-seq --self` --flags FO",
+    "test:seq:deno": "deno run -A `tape6-seq --self` --flags FO",
+    "lint": "prettier --check .",
+    "lint:fix": "prettier --write ."
   },
   "repository": {
     "type": "git",
@@ -26,8 +31,16 @@
   },
   "keywords": [
     "benchmark",
+    "micro-benchmark",
     "performance",
-    "statistics"
+    "profiling",
+    "statistics",
+    "significance",
+    "bootstrap",
+    "mann-whitney",
+    "kruskal-wallis",
+    "cli",
+    "compare"
   ],
   "author": "Eugene Lazutkin <eugene.lazutkin@gmail.com> (https://www.lazutkin.com/)",
   "license": "BSD-3-Clause",
@@ -40,15 +53,19 @@
   },
   "homepage": "https://github.com/uhop/nano-bench#readme",
   "files": [
-    "src"
+    "src",
+    "bin",
+    "llms.txt",
+    "llms-full.txt"
   ],
   "devDependencies": {
-    "tape-six": "^1.5.1",
-    "tape-six-proc": "^1.2.1"
+    "prettier": "^3.8.1",
+    "tape-six": "^1.7.2",
+    "tape-six-proc": "^1.2.3"
   },
   "dependencies": {
-    "commander": "^14.0.2",
-    "console-toolkit": "^1.2.8"
+    "commander": "^14.0.3",
+    "console-toolkit": "^1.2.11"
   },
   "tape6": {
     "tests": [

package/src/bench/compare.js

@@ -22,7 +22,10 @@ const compare = async (inputs, options = {}, report) => {
   report?.('calculating-significance', {stats, options});
   let results;
   if (keys.length > 2) {
-    results = kwtest(stats.map(stat => stat.data), options.alpha);
+    results = kwtest(
+      stats.map(stat => stat.data),
+      options.alpha
+    );
   } else {
     results = mwtest(stats[0].data, stats[1].data, options.alpha);
   }

package/src/index.js

@@ -0,0 +1,27 @@
+export {
+  mean,
+  variance,
+  stdDev,
+  skewness,
+  kurtosis,
+  excessKurtosis,
+  bootstrap,
+  getWeightedValue
+} from './stats.js';
+export {median} from './median.js';
+export {StatCounter, streamStats} from './stream-stats.js';
+export {MedianCounter, streamMedian} from './stream-median.js';
+export {
+  findLevel,
+  benchmark,
+  benchmarkSeries,
+  benchmarkSeriesPar,
+  measure,
+  measurePar,
+  Stats,
+  wrapper
+} from './bench/runner.js';
+export {default as compare} from './bench/compare.js';
+export {default as mwtest} from './significance/mwtest.js';
+export {default as kwtest} from './significance/kwtest.js';
+export {default as kstest} from './significance/kstest.js';

package/src/significance/kwtest.js

@@ -37,7 +37,7 @@ export const rankData = groups => {
   for (let i = 0; i < t.length; ++i) {
     const x = t[i].rank - avgRank;
     denominator += x * x;
-    S2 = t[i].rank * t[i].rank - avgRankC;
+    S2 += t[i].rank * t[i].rank - avgRankC;
   }
 
   S2 /= N - 1;

package/src/stats/normal.js

@@ -1,8 +1,5 @@
 import erf from './erf.js';
 
-const LIMIT = 1000;
-const EPSILON = 1e-30;
-
 const SQRT_2 = Math.sqrt(2),
   SQRT_2_PI = Math.sqrt(2 * Math.PI);
 

package/src/stats/rank.js

@@ -33,7 +33,8 @@ export const rank = groups => {
     }
     i = ahead;
   }
-  const avgRank = (N + 1) / 2, avgGroupRank = groupRank.map((rank, i) => rank / groups[i].length);
+  const avgRank = (N + 1) / 2,
+    avgGroupRank = groupRank.map((rank, i) => rank / groups[i].length);
 
   return {ranked: t, N, k, avgRank, groupRank, avgGroupRank, groups};
 };

package/src/stats/z-ppf.js

@@ -2,7 +2,7 @@ import {zCdf, zPdf} from './z.js';
 import ppf from './ppf.js';
 
 // percent point function
-const zPpf = (z) => {
+const zPpf = z => {
   // find the lower bound
   let x = -6,
     p = zCdf(x);

package/src/stats/z.js

@@ -3,5 +3,5 @@ import erf from './erf.js';
 const SQRT_2 = Math.sqrt(2),
   SQRT_2_PI = Math.sqrt(2 * Math.PI);
 
-export const zCdf = (z) => 0.5 * (1 + erf((z) / SQRT_2));
-export const zPdf = (z) => Math.exp(-0.5 * z * z) / SQRT_2_PI;
+export const zCdf = z => 0.5 * (1 + erf(z / SQRT_2));
+export const zPdf = z => Math.exp(-0.5 * z * z) / SQRT_2_PI;

package/src/utils/rk.js

@@ -24,7 +24,7 @@ export const rk23 = (fn, {a = 0, b = 1, tolerance = 1e-6, initialValue = 0} = {}
 
     if (error < maxError) {
       ts.push((t += h));
-      us.push(u = uNew);
+      us.push((u = uNew));
       s1 = s4;
     }