@maverickcer/env-cap 0.7.4 → 0.7.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ADOPTION.md +106 -124
- package/AGENTS.md +94 -128
- package/CHANGELOG.md +42 -0
- package/CODE_REVIEW.md +69 -173
- package/CONTRIBUTING.md +57 -74
- package/GUIDE.md +405 -866
- package/README.md +112 -89
- package/RELEASING.md +81 -65
- package/SECURITY.md +76 -227
- package/SUPPORT.md +21 -45
- package/VERSIONING.md +69 -105
- package/benchmarks/READING-BENCHMARKS.md +103 -81
- package/benchmarks/README.md +12 -9
- package/benchmarks/WRITING-BENCHMARKS.md +90 -80
- package/benchmarks/performance-buildtime/BENCHMARKS.md +20 -20
- package/benchmarks/performance-runtime/BENCHMARKS.md +107 -107
- package/dist/.dts/_ipc/build-fs.d.ts +12 -17
- package/dist/.dts/_ipc/build-fs.d.ts.map +1 -1
- package/dist/.dts/_ipc/compiler-api.d.ts +6 -6
- package/dist/.dts/_ipc/display-path.d.ts +1 -1
- package/dist/.dts/_ipc/display-path.d.ts.map +1 -1
- package/dist/.dts/_ipc/generated-banner.d.ts +8 -8
- package/dist/.dts/_ipc/glob.d.ts +9 -8
- package/dist/.dts/_ipc/glob.d.ts.map +1 -1
- package/dist/.dts/_ipc/no-node-fs.d.ts +2 -2
- package/dist/.dts/_ipc/no-node-fs.d.ts.map +1 -1
- package/dist/.dts/_ipc/resolve-tsconfig-paths.d.ts +16 -16
- package/dist/.dts/_ipc/resolve-tsconfig-paths.d.ts.map +1 -1
- package/dist/.dts/_ipc/typescript.d.ts +4 -4
- package/dist/.dts/build/assemble-project.d.ts +14 -15
- package/dist/.dts/build/assemble-project.d.ts.map +1 -1
- package/dist/.dts/build/change-model.d.ts +14 -14
- package/dist/.dts/build/change-model.d.ts.map +1 -1
- package/dist/.dts/build/check-artifacts.d.ts +4 -4
- package/dist/.dts/build/check-artifacts.d.ts.map +1 -1
- package/dist/.dts/build/citation-verification.d.ts +13 -14
- package/dist/.dts/build/citation-verification.d.ts.map +1 -1
- package/dist/.dts/build/compatibility.d.ts +24 -24
- package/dist/.dts/build/compatibility.d.ts.map +1 -1
- package/dist/.dts/build/contract-model.d.ts +10 -10
- package/dist/.dts/build/contract-model.d.ts.map +1 -1
- package/dist/.dts/build/deep-equal.d.ts +2 -2
- package/dist/.dts/build/deep-freeze.d.ts +2 -2
- package/dist/.dts/build/dependency-graph.d.ts +48 -55
- package/dist/.dts/build/dependency-graph.d.ts.map +1 -1
- package/dist/.dts/build/dependency-model.d.ts +18 -18
- package/dist/.dts/build/dependency-model.d.ts.map +1 -1
- package/dist/.dts/build/discover.d.ts +11 -11
- package/dist/.dts/build/discover.d.ts.map +1 -1
- package/dist/.dts/build/docs.d.ts +34 -38
- package/dist/.dts/build/docs.d.ts.map +1 -1
- package/dist/.dts/build/env-example.d.ts +13 -13
- package/dist/.dts/build/env-example.d.ts.map +1 -1
- package/dist/.dts/build/errors.d.ts +8 -8
- package/dist/.dts/build/evidence-cache.d.ts +23 -23
- package/dist/.dts/build/evidence-cache.d.ts.map +1 -1
- package/dist/.dts/build/evidence-model.d.ts +3 -3
- package/dist/.dts/build/evidence-reference.d.ts +15 -15
- package/dist/.dts/build/evidence-reference.d.ts.map +1 -1
- package/dist/.dts/build/evidence-snapshot.d.ts +47 -53
- package/dist/.dts/build/evidence-snapshot.d.ts.map +1 -1
- package/dist/.dts/build/exclusive-group.d.ts +4 -4
- package/dist/.dts/build/exclusive-group.d.ts.map +1 -1
- package/dist/.dts/build/finding-model.d.ts +12 -12
- package/dist/.dts/build/finding-model.d.ts.map +1 -1
- package/dist/.dts/build/generate-documentation.d.ts +19 -19
- package/dist/.dts/build/generate-documentation.d.ts.map +1 -1
- package/dist/.dts/build/generate-env-artifacts.d.ts +43 -43
- package/dist/.dts/build/generate-env-artifacts.d.ts.map +1 -1
- package/dist/.dts/build/generate-evidence.d.ts +19 -19
- package/dist/.dts/build/generate-evidence.d.ts.map +1 -1
- package/dist/.dts/build/generate-manifest.d.ts +11 -12
- package/dist/.dts/build/generate-manifest.d.ts.map +1 -1
- package/dist/.dts/build/generate-usage.d.ts +23 -24
- package/dist/.dts/build/generate-usage.d.ts.map +1 -1
- package/dist/.dts/build/generated-banner.d.ts +5 -5
- package/dist/.dts/build/governance-fields.d.ts +7 -7
- package/dist/.dts/build/humanize-key.d.ts +1 -1
- package/dist/.dts/build/humanize-key.d.ts.map +1 -1
- package/dist/.dts/build/index.d.ts +20 -36
- package/dist/.dts/build/index.d.ts.map +1 -1
- package/dist/.dts/build/lifecycle-model.d.ts +90 -83
- package/dist/.dts/build/lifecycle-model.d.ts.map +1 -1
- package/dist/.dts/build/link.d.ts +38 -39
- package/dist/.dts/build/link.d.ts.map +1 -1
- package/dist/.dts/build/literal-eval.d.ts +2 -2
- package/dist/.dts/build/literal-eval.d.ts.map +1 -1
- package/dist/.dts/build/live-expirations.d.ts +14 -13
- package/dist/.dts/build/live-expirations.d.ts.map +1 -1
- package/dist/.dts/build/manifest.d.ts +6 -6
- package/dist/.dts/build/map-utils.d.ts +4 -4
- package/dist/.dts/build/merge-documentation.d.ts +9 -8
- package/dist/.dts/build/merge-documentation.d.ts.map +1 -1
- package/dist/.dts/build/ownership-model.d.ts +14 -15
- package/dist/.dts/build/ownership-model.d.ts.map +1 -1
- package/dist/.dts/build/parse.d.ts +20 -21
- package/dist/.dts/build/parse.d.ts.map +1 -1
- package/dist/.dts/build/reference-projections.d.ts +36 -38
- package/dist/.dts/build/reference-projections.d.ts.map +1 -1
- package/dist/.dts/build/resolution/resolve-import.d.ts +18 -14
- package/dist/.dts/build/resolution/resolve-import.d.ts.map +1 -1
- package/dist/.dts/build/resolution/resolve-package-schema.d.ts +26 -26
- package/dist/.dts/build/resolution/resolve-package-schema.d.ts.map +1 -1
- package/dist/.dts/build/resolution/resolve-within-root.d.ts +9 -9
- package/dist/.dts/build/sarif.d.ts +5 -5
- package/dist/.dts/build/sarif.d.ts.map +1 -1
- package/dist/.dts/build/scan-dependencies.d.ts +17 -16
- package/dist/.dts/build/scan-dependencies.d.ts.map +1 -1
- package/dist/.dts/build/sort-by-identity.d.ts +1 -1
- package/dist/.dts/build/source-position.d.ts +13 -13
- package/dist/.dts/build/tool-version.d.ts +1 -1
- package/dist/.dts/build/usage-report.d.ts +19 -19
- package/dist/.dts/build/usage-report.d.ts.map +1 -1
- package/dist/.dts/eslint-plugin/index.d.ts +16 -5
- package/dist/.dts/eslint-plugin/index.d.ts.map +1 -1
- package/dist/.dts/eslint-plugin/no-raw-process-env.d.ts +4 -4
- package/dist/.dts/evidence/define-projection.d.ts +15 -16
- package/dist/.dts/evidence/define-projection.d.ts.map +1 -1
- package/dist/.dts/evidence/index.d.ts +11 -15
- package/dist/.dts/evidence/index.d.ts.map +1 -1
- package/dist/.dts/helpers/index.d.ts +15 -18
- package/dist/.dts/helpers/index.d.ts.map +1 -1
- package/dist/.dts/helpers/validators.d.ts +2 -2
- package/dist/.dts/helpers/validators.d.ts.map +1 -1
- package/dist/.dts/node/index.d.ts +19 -0
- package/dist/.dts/node/index.d.ts.map +1 -1
- package/dist/.dts/runtime/create.d.ts +6 -3
- package/dist/.dts/runtime/create.d.ts.map +1 -1
- package/dist/.dts/runtime/document.d.ts +91 -97
- package/dist/.dts/runtime/document.d.ts.map +1 -1
- package/dist/.dts/runtime/errors.d.ts +5 -5
- package/dist/.dts/runtime/index.d.ts +11 -10
- package/dist/.dts/runtime/index.d.ts.map +1 -1
- package/dist/.dts/runtime/reset.d.ts +1 -1
- package/dist/.dts/runtime/types.d.ts +39 -24
- package/dist/.dts/runtime/types.d.ts.map +1 -1
- package/dist/.dts/runtime/validate.d.ts +15 -2
- package/dist/.dts/runtime/validate.d.ts.map +1 -1
- package/dist/.dts7/_ipc/build-fs.d.ts +48 -0
- package/dist/.dts7/_ipc/build-fs.d.ts.map +1 -0
- package/dist/.dts7/_ipc/compiler-api.d.ts +17 -0
- package/dist/.dts7/_ipc/compiler-api.d.ts.map +1 -0
- package/dist/.dts7/_ipc/display-path.d.ts +27 -0
- package/dist/.dts7/_ipc/display-path.d.ts.map +1 -0
- package/dist/.dts7/_ipc/generated-banner.d.ts +58 -0
- package/dist/.dts7/_ipc/generated-banner.d.ts.map +1 -0
- package/dist/.dts7/_ipc/glob.d.ts +23 -0
- package/dist/.dts7/_ipc/glob.d.ts.map +1 -0
- package/dist/.dts7/_ipc/no-node-fs.d.ts +21 -0
- package/dist/.dts7/_ipc/no-node-fs.d.ts.map +1 -0
- package/dist/.dts7/_ipc/resolve-tsconfig-paths.d.ts +134 -0
- package/dist/.dts7/_ipc/resolve-tsconfig-paths.d.ts.map +1 -0
- package/dist/.dts7/_ipc/typescript.d.ts +20 -0
- package/dist/.dts7/_ipc/typescript.d.ts.map +1 -0
- package/dist/.dts7/build/assemble-project.d.ts +46 -0
- package/dist/.dts7/build/assemble-project.d.ts.map +1 -0
- package/dist/.dts7/build/change-model.d.ts +63 -0
- package/dist/.dts7/build/change-model.d.ts.map +1 -0
- package/dist/.dts7/build/check-artifacts.d.ts +48 -0
- package/dist/.dts7/build/check-artifacts.d.ts.map +1 -0
- package/dist/.dts7/build/citation-verification.d.ts +61 -0
- package/dist/.dts7/build/citation-verification.d.ts.map +1 -0
- package/dist/.dts7/build/compatibility.d.ts +83 -0
- package/dist/.dts7/build/compatibility.d.ts.map +1 -0
- package/dist/.dts7/build/compiler-api.d.ts +12 -0
- package/dist/.dts7/build/compiler-api.d.ts.map +1 -0
- package/dist/.dts7/build/contract-model.d.ts +93 -0
- package/dist/.dts7/build/contract-model.d.ts.map +1 -0
- package/dist/.dts7/build/deep-equal.d.ts +9 -0
- package/dist/.dts7/build/deep-equal.d.ts.map +1 -0
- package/dist/.dts7/build/deep-freeze.d.ts +20 -0
- package/dist/.dts7/build/deep-freeze.d.ts.map +1 -0
- package/dist/.dts7/build/dependency-graph.d.ts +163 -0
- package/dist/.dts7/build/dependency-graph.d.ts.map +1 -0
- package/dist/.dts7/build/dependency-model.d.ts +81 -0
- package/dist/.dts7/build/dependency-model.d.ts.map +1 -0
- package/dist/.dts7/build/discover.d.ts +41 -0
- package/dist/.dts7/build/discover.d.ts.map +1 -0
- package/dist/.dts7/build/display-path.d.ts +6 -0
- package/dist/.dts7/build/display-path.d.ts.map +1 -0
- package/dist/.dts7/build/docs.d.ts +172 -0
- package/dist/.dts7/build/docs.d.ts.map +1 -0
- package/dist/.dts7/build/env-example.d.ts +85 -0
- package/dist/.dts7/build/env-example.d.ts.map +1 -0
- package/dist/.dts7/build/errors.d.ts +78 -0
- package/dist/.dts7/build/errors.d.ts.map +1 -0
- package/dist/.dts7/build/evidence-cache.d.ts +74 -0
- package/dist/.dts7/build/evidence-cache.d.ts.map +1 -0
- package/dist/.dts7/build/evidence-model.d.ts +50 -0
- package/dist/.dts7/build/evidence-model.d.ts.map +1 -0
- package/dist/.dts7/build/evidence-reference.d.ts +67 -0
- package/dist/.dts7/build/evidence-reference.d.ts.map +1 -0
- package/dist/.dts7/build/evidence-snapshot.d.ts +142 -0
- package/dist/.dts7/build/evidence-snapshot.d.ts.map +1 -0
- package/dist/.dts7/build/exclusive-group.d.ts +19 -0
- package/dist/.dts7/build/exclusive-group.d.ts.map +1 -0
- package/dist/.dts7/build/finding-model.d.ts +80 -0
- package/dist/.dts7/build/finding-model.d.ts.map +1 -0
- package/dist/.dts7/build/generate-documentation.d.ts +176 -0
- package/dist/.dts7/build/generate-documentation.d.ts.map +1 -0
- package/dist/.dts7/build/generate-env-artifacts.d.ts +191 -0
- package/dist/.dts7/build/generate-env-artifacts.d.ts.map +1 -0
- package/dist/.dts7/build/generate-evidence.d.ts +77 -0
- package/dist/.dts7/build/generate-evidence.d.ts.map +1 -0
- package/dist/.dts7/build/generate-manifest.d.ts +100 -0
- package/dist/.dts7/build/generate-manifest.d.ts.map +1 -0
- package/dist/.dts7/build/generate-usage.d.ts +94 -0
- package/dist/.dts7/build/generate-usage.d.ts.map +1 -0
- package/dist/.dts7/build/generated-banner.d.ts +43 -0
- package/dist/.dts7/build/generated-banner.d.ts.map +1 -0
- package/dist/.dts7/build/glob.d.ts +6 -0
- package/dist/.dts7/build/glob.d.ts.map +1 -0
- package/dist/.dts7/build/governance-fields.d.ts +34 -0
- package/dist/.dts7/build/governance-fields.d.ts.map +1 -0
- package/dist/.dts7/build/humanize-key.d.ts +14 -0
- package/dist/.dts7/build/humanize-key.d.ts.map +1 -0
- package/dist/.dts7/build/index.d.ts +89 -0
- package/dist/.dts7/build/index.d.ts.map +1 -0
- package/dist/.dts7/build/lifecycle-model.d.ts +181 -0
- package/dist/.dts7/build/lifecycle-model.d.ts.map +1 -0
- package/dist/.dts7/build/link.d.ts +222 -0
- package/dist/.dts7/build/link.d.ts.map +1 -0
- package/dist/.dts7/build/literal-eval.d.ts +24 -0
- package/dist/.dts7/build/literal-eval.d.ts.map +1 -0
- package/dist/.dts7/build/live-expirations.d.ts +48 -0
- package/dist/.dts7/build/live-expirations.d.ts.map +1 -0
- package/dist/.dts7/build/manifest.d.ts +31 -0
- package/dist/.dts7/build/manifest.d.ts.map +1 -0
- package/dist/.dts7/build/map-utils.d.ts +8 -0
- package/dist/.dts7/build/map-utils.d.ts.map +1 -0
- package/dist/.dts7/build/merge-documentation.d.ts +36 -0
- package/dist/.dts7/build/merge-documentation.d.ts.map +1 -0
- package/dist/.dts7/build/ownership-model.d.ts +58 -0
- package/dist/.dts7/build/ownership-model.d.ts.map +1 -0
- package/dist/.dts7/build/parse.d.ts +206 -0
- package/dist/.dts7/build/parse.d.ts.map +1 -0
- package/dist/.dts7/build/reference-projections.d.ts +168 -0
- package/dist/.dts7/build/reference-projections.d.ts.map +1 -0
- package/dist/.dts7/build/resolution/resolve-import.d.ts +62 -0
- package/dist/.dts7/build/resolution/resolve-import.d.ts.map +1 -0
- package/dist/.dts7/build/resolution/resolve-package-schema.d.ts +83 -0
- package/dist/.dts7/build/resolution/resolve-package-schema.d.ts.map +1 -0
- package/dist/.dts7/build/resolution/resolve-tsconfig-paths.d.ts +18 -0
- package/dist/.dts7/build/resolution/resolve-tsconfig-paths.d.ts.map +1 -0
- package/dist/.dts7/build/resolution/resolve-within-root.d.ts +35 -0
- package/dist/.dts7/build/resolution/resolve-within-root.d.ts.map +1 -0
- package/dist/.dts7/build/sarif.d.ts +66 -0
- package/dist/.dts7/build/sarif.d.ts.map +1 -0
- package/dist/.dts7/build/scan-dependencies.d.ts +71 -0
- package/dist/.dts7/build/scan-dependencies.d.ts.map +1 -0
- package/dist/.dts7/build/sort-by-identity.d.ts +14 -0
- package/dist/.dts7/build/sort-by-identity.d.ts.map +1 -0
- package/dist/.dts7/build/source-position.d.ts +69 -0
- package/dist/.dts7/build/source-position.d.ts.map +1 -0
- package/dist/.dts7/build/tool-version.d.ts +8 -0
- package/dist/.dts7/build/tool-version.d.ts.map +1 -0
- package/dist/.dts7/build/types.d.ts +7 -0
- package/dist/.dts7/build/types.d.ts.map +1 -0
- package/dist/.dts7/build/typescript.d.ts +6 -0
- package/dist/.dts7/build/typescript.d.ts.map +1 -0
- package/dist/.dts7/build/usage-report.d.ts +124 -0
- package/dist/.dts7/build/usage-report.d.ts.map +1 -0
- package/dist/.dts7/cli/filesystem.d.ts +7 -0
- package/dist/.dts7/cli/filesystem.d.ts.map +1 -0
- package/dist/.dts7/eslint-plugin/glob.d.ts +7 -0
- package/dist/.dts7/eslint-plugin/glob.d.ts.map +1 -0
- package/dist/.dts7/eslint-plugin/index.d.ts +38 -0
- package/dist/.dts7/eslint-plugin/index.d.ts.map +1 -0
- package/dist/.dts7/eslint-plugin/no-node-fs.d.ts +15 -0
- package/dist/.dts7/eslint-plugin/no-node-fs.d.ts.map +1 -0
- package/dist/.dts7/eslint-plugin/no-raw-process-env.d.ts +23 -0
- package/dist/.dts7/eslint-plugin/no-raw-process-env.d.ts.map +1 -0
- package/dist/.dts7/evidence/define-projection.d.ts +51 -0
- package/dist/.dts7/evidence/define-projection.d.ts.map +1 -0
- package/dist/.dts7/evidence/index.d.ts +19 -0
- package/dist/.dts7/evidence/index.d.ts.map +1 -0
- package/dist/.dts7/helpers/index.d.ts +132 -0
- package/dist/.dts7/helpers/index.d.ts.map +1 -0
- package/dist/.dts7/helpers/processors.d.ts +32 -0
- package/dist/.dts7/helpers/processors.d.ts.map +1 -0
- package/dist/.dts7/helpers/validators.d.ts +68 -0
- package/dist/.dts7/helpers/validators.d.ts.map +1 -0
- package/dist/.dts7/node/index.d.ts +21 -0
- package/dist/.dts7/node/index.d.ts.map +1 -0
- package/dist/.dts7/runtime/cache.d.ts +17 -0
- package/dist/.dts7/runtime/cache.d.ts.map +1 -0
- package/dist/.dts7/runtime/create.d.ts +23 -0
- package/dist/.dts7/runtime/create.d.ts.map +1 -0
- package/dist/.dts7/runtime/document.d.ts +214 -0
- package/dist/.dts7/runtime/document.d.ts.map +1 -0
- package/dist/.dts7/runtime/errors.d.ts +55 -0
- package/dist/.dts7/runtime/errors.d.ts.map +1 -0
- package/dist/.dts7/runtime/index.d.ts +25 -0
- package/dist/.dts7/runtime/index.d.ts.map +1 -0
- package/dist/.dts7/runtime/registry.d.ts +12 -0
- package/dist/.dts7/runtime/registry.d.ts.map +1 -0
- package/dist/.dts7/runtime/reset.d.ts +8 -0
- package/dist/.dts7/runtime/reset.d.ts.map +1 -0
- package/dist/.dts7/runtime/types.d.ts +190 -0
- package/dist/.dts7/runtime/types.d.ts.map +1 -0
- package/dist/.dts7/runtime/validate.d.ts +27 -0
- package/dist/.dts7/runtime/validate.d.ts.map +1 -0
- package/dist/{build-6XFHJFQK.js → build-UEENZEUX.js} +43 -42
- package/dist/build-UEENZEUX.js.map +1 -0
- package/dist/build.cjs +42 -41
- package/dist/build.cjs.map +1 -1
- package/dist/build.js +42 -41
- package/dist/build.js.map +1 -1
- package/dist/build.ts7.d.cts +1 -0
- package/dist/build.ts7.d.ts +1 -0
- package/dist/{chunk-RL5KFVUT.js → chunk-TBL3O7IG.js} +2 -2
- package/dist/cli/index.js +29 -28
- package/dist/cli/index.js.map +1 -1
- package/dist/eslint-plugin/index.cjs.map +1 -1
- package/dist/eslint-plugin/index.js.map +1 -1
- package/dist/eslint-plugin/index.ts7.d.cts +1 -0
- package/dist/eslint-plugin/index.ts7.d.ts +1 -0
- package/dist/evidence.cjs +1 -1
- package/dist/evidence.cjs.map +1 -1
- package/dist/evidence.js +1 -1
- package/dist/evidence.js.map +1 -1
- package/dist/evidence.ts7.d.cts +1 -0
- package/dist/evidence.ts7.d.ts +1 -0
- package/dist/helpers.cjs +4 -4
- package/dist/helpers.cjs.map +1 -1
- package/dist/helpers.js +4 -4
- package/dist/helpers.js.map +1 -1
- package/dist/helpers.ts7.d.cts +1 -0
- package/dist/helpers.ts7.d.ts +1 -0
- package/dist/index.cjs.map +1 -1
- package/dist/index.js.map +1 -1
- package/dist/index.ts7.d.cts +1 -0
- package/dist/index.ts7.d.ts +1 -0
- package/dist/node.ts7.d.cts +1 -0
- package/dist/node.ts7.d.ts +1 -0
- package/examples/README.md +46 -80
- package/package.json +43 -12
- package/schemas/contract-model.schema.json +7 -7
- package/schemas/env-cap-report.schema.json +88 -88
- package/skills/env-cap/SKILL.md +12 -13
- package/skills/env-cap/references/api.md +1 -1
- package/specs/architecture.md +147 -122
- package/specs/decisions/0001-runtime-documentation-separation.md +1 -1
- package/specs/decisions/0014-cross-package-schema-discovery.md +4 -1
- package/specs/decisions/0023-tsconfig-path-alias-resolution.md +4 -1
- package/specs/decisions/0049-typescript-7-scanner-uses-a-bundled-typescript-6.md +13 -8
- package/specs/generated-artifacts.md +91 -71
- package/specs/migrations/README.md +6 -6
- package/specs/migrations/from-centralized-schema.md +1 -1
- package/specs/migrations/from-t3-env.md +9 -9
- package/PROMPT.md +0 -89
- package/dist/build-6XFHJFQK.js.map +0 -1
- /package/dist/{chunk-RL5KFVUT.js.map → chunk-TBL3O7IG.js.map} +0 -0
|
@@ -1,26 +1,28 @@
|
|
|
1
1
|
# Writing benchmarks
|
|
2
2
|
|
|
3
3
|
This guide is for the person adding or changing a benchmark. It explains **what** to test, **how** to
|
|
4
|
-
test it, **why** each choice is made, and **how to document** the result
|
|
5
|
-
can read
|
|
6
|
-
explains how to interpret what you produce.
|
|
4
|
+
test it, **why** each choice is made, and **how to document** the result. The goal is a result a platform
|
|
5
|
+
engineer or CTO can read without knowing the code. Its companion,
|
|
6
|
+
[READING-BENCHMARKS.md](READING-BENCHMARKS.md), explains how to interpret what you produce.
|
|
7
7
|
|
|
8
8
|
The benchmarks exist to answer one question for the people who run this package in production:
|
|
9
|
-
|
|
10
|
-
|
|
9
|
+
|
|
10
|
+
**What does it cost me, in time and in money, and how does that grow as my workload grows?**
|
|
11
|
+
|
|
12
|
+
The rest of the repository makes the case for what the package gives you. The benchmarks are deliberately
|
|
11
13
|
honest about what it costs.
|
|
12
14
|
|
|
13
15
|
## The three views
|
|
14
16
|
|
|
15
17
|
Every suite measures the package from three angles, each building on the last.
|
|
16
18
|
|
|
17
|
-
1. **End-to-end: the total impact.** One operation done two ways, with empty or minimal functions
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
2. **Functions: each part on its own.** Every function the end-to-end run uses
|
|
22
|
-
|
|
23
|
-
|
|
19
|
+
1. **End-to-end: the total impact.** One operation is done two ways, with empty or minimal functions plugged
|
|
20
|
+
in. It runs once with no package (the _baseline_) and once routed through the package. The difference is
|
|
21
|
+
the package's own cost: the floor it imposes on every operation. Keep both sides trivial on purpose: if
|
|
22
|
+
you plug in real work, you are measuring that work.
|
|
23
|
+
2. **Functions: each part on its own.** Every function the end-to-end run uses is measured separately across
|
|
24
|
+
the full size ladder. So is every public function worth knowing about. That shows how each one scales and
|
|
25
|
+
whether it behaves the way we say it does.
|
|
24
26
|
3. **Contribution: where the cost lives.** Each function says how often one end-to-end operation calls
|
|
25
27
|
it (`inEndToEnd`). The report multiplies that by the function's measured cost and shows its share
|
|
26
28
|
of the overhead, so effort goes to the function that actually matters.
|
|
@@ -36,22 +38,22 @@ a function whose cost nobody can see.
|
|
|
36
38
|
- **Cold start**, if the package does work at import or startup (module evaluation, building
|
|
37
39
|
registries, reading configuration). Cold start is paid by every process, every deploy and every
|
|
38
40
|
serverless invocation that scales from zero.
|
|
39
|
-
- **Memory**,
|
|
41
|
+
- **Memory**, as well as time: what the operation allocates and retains. The report records the heap
|
|
40
42
|
delta of every sample.
|
|
41
|
-
- **Every variable that could change the cost.** This is the part that is
|
|
42
|
-
that matters most. Walk through each of these for every function and decide, explicitly,
|
|
43
|
-
is _swept_ (the size axis), a _variant_ (measured separately), or _fixed_ (held constant
|
|
44
|
-
purpose, with the value stated):
|
|
45
|
-
- **Input size
|
|
46
|
-
- **Input shape
|
|
47
|
-
|
|
48
|
-
- **State
|
|
49
|
-
- **Options and modes
|
|
50
|
-
- **Outcome
|
|
43
|
+
- **Every variable that could change the cost.** This is the part that is tempting to skip and the part
|
|
44
|
+
that matters most. Walk through each of these for every function and decide, explicitly, how it is
|
|
45
|
+
treated. Each is _swept_ (the size axis), a _variant_ (measured separately), or _fixed_ (held constant
|
|
46
|
+
on purpose, with the value stated):
|
|
47
|
+
- **Input size**: the size ladder below. Always swept.
|
|
48
|
+
- **Input shape**: flat vs deeply nested, many small items vs few large ones, wide vs narrow objects.
|
|
49
|
+
Also strings vs numbers, and sorted vs random vs adversarial order.
|
|
50
|
+
- **State**: empty vs populated, first call (cold) vs repeated call (warm), cache hit vs miss.
|
|
51
|
+
- **Options and modes**: every flag that changes the algorithm used.
|
|
52
|
+
- **Outcome**: the success path **and** the error path. Errors are often slower and often hit
|
|
51
53
|
under load, exactly when slowness hurts most.
|
|
52
|
-
- **Concurrency
|
|
54
|
+
- **Concurrency**: one caller vs many at once, if the function keeps shared state or does async
|
|
53
55
|
work.
|
|
54
|
-
- **Environment
|
|
56
|
+
- **Environment**: runtime and version, platform, available CPU and memory.
|
|
55
57
|
- **What you deliberately do not cover**, listed in `notCovered` with the reason. An honest gap is
|
|
56
58
|
documentation; a silent gap is a trap.
|
|
57
59
|
|
|
@@ -63,14 +65,14 @@ Every function is measured at **20, 40, 80, 160, 320, 640, 1280, 2560, 5120 and
|
|
|
63
65
|
The default ladder of ten doubling points is not arbitrary. Big-O is estimated by fitting a line through
|
|
64
66
|
`log(time)` against `log(size)`; its slope is the growth exponent. Three points cannot tell `O(n)`
|
|
65
67
|
from `O(n log n)` or `O(1)` from `O(log n)`. A ladder spanning more than two orders of magnitude, with
|
|
66
|
-
ten points, can. `typicalN` (default 640) marks the size the cost summary quotes
|
|
67
|
-
|
|
68
|
+
ten points, can. `typicalN` (default 640) marks the size the cost summary quotes. Choose the size a real
|
|
69
|
+
operation of this package typically handles, and say why in `workload.description`.
|
|
68
70
|
|
|
69
|
-
Never shorten the ladder for committed results, with one recorded exception
|
|
71
|
+
Never shorten the ladder for committed results, with one recorded exception. A function too slow to
|
|
70
72
|
measure at the largest sizes (it compiles a package, or spawns a process, per unit) may list its own
|
|
71
|
-
`tiers
|
|
72
|
-
prints beside its results. `--quick` exists for a smoke test and says so in its output; do not
|
|
73
|
-
what it produces.
|
|
73
|
+
`tiers`. These are at least four ascending sizes, given together with a `tiersReason` saying why, which
|
|
74
|
+
the report prints beside its results. `--quick` exists for a smoke test and says so in its output; do not
|
|
75
|
+
commit what it produces.
|
|
74
76
|
|
|
75
77
|
## How to test
|
|
76
78
|
|
|
@@ -80,52 +82,52 @@ what it produces.
|
|
|
80
82
|
whose result is thrown away can be optimized into nothing, and you will have benchmarked an empty
|
|
81
83
|
loop.
|
|
82
84
|
- **Fresh state when the work changes its own input.** If `run` mutates what `setup` built (committing
|
|
83
|
-
to a store, draining a queue), set `fresh: true
|
|
85
|
+
to a store, draining a queue), set `fresh: true`. Each sample then gets a new `setup`. Otherwise the
|
|
84
86
|
second sample measures a different, usually easier, problem.
|
|
85
87
|
- **Deterministic input.** No `Math.random()`, no `Date.now()` in input generation. Generate from the
|
|
86
88
|
index (`item i = { id: i, name: "item-" + i }`) so two runs measure the same thing. If you need
|
|
87
89
|
pseudo-randomness, use a seeded generator and record the seed in the suite.
|
|
88
90
|
- **No real I/O.** A network call or disk read measures the network or the disk. Simulate it with a
|
|
89
|
-
function that resolves from a prepared pool
|
|
91
|
+
function that resolves from a prepared pool. Keep the real code path (queues, deduplication, error
|
|
90
92
|
handling) intact, because that is what the package contributes.
|
|
91
93
|
- **Import the package the way a consumer does.** Its built entry point, never a source path. The
|
|
92
94
|
published `exports` map is part of the cost (and of the risk). A package whose costly code is internal
|
|
93
|
-
(an engine no entry point exposes on its own) may import that source instead
|
|
94
|
-
|
|
95
|
-
- **Let the engine handle fast operations.**
|
|
96
|
-
back to back inside one sample and the result divided
|
|
97
|
-
|
|
95
|
+
(an engine no entry point exposes on its own) may import that source instead. Its benchmarks README then
|
|
96
|
+
says so and why.
|
|
97
|
+
- **Let the engine handle fast operations.** Timers cannot resolve operations faster than about a
|
|
98
|
+
millisecond. Those are repeated back to back inside one sample, and the result is divided. You do
|
|
99
|
+
nothing; the report records the batch size.
|
|
98
100
|
- **Quiet machine.** Close heavy programs, plug in a laptop, and run on the same machine you compared
|
|
99
101
|
against last time. Never compare across machines (see the reading guide).
|
|
100
102
|
|
|
101
103
|
## How to document a benchmark
|
|
102
104
|
|
|
103
|
-
`defineSuite` refuses a suite with missing or placeholder documentation
|
|
104
|
-
|
|
105
|
-
|
|
105
|
+
`defineSuite` refuses a suite with missing or placeholder documentation. `npm run benchmark:check` runs
|
|
106
|
+
that validation without measuring anything, so it can run in CI on every change. Every function needs the
|
|
107
|
+
following, written for a reader who does not know the code.
|
|
106
108
|
|
|
107
|
-
### `why
|
|
109
|
+
### `why`: why we benchmark it
|
|
108
110
|
|
|
109
111
|
The reason this function earns a benchmark: where it runs, how often, and who feels it. Not what it
|
|
110
|
-
does
|
|
112
|
+
does, but why its cost matters.
|
|
111
113
|
|
|
112
|
-
> **Good:** "Runs once per getter call, on every request that reads fetched data
|
|
113
|
-
> is multiplied by the application's whole read traffic."
|
|
114
|
+
> **Good:** "Runs once per getter call, on every request that reads fetched data. Its per-call cost
|
|
115
|
+
> is therefore multiplied by the application's whole read traffic."
|
|
114
116
|
>
|
|
115
117
|
> **Bad:** "Tests the commit function."
|
|
116
118
|
|
|
117
|
-
### `poorPerformanceMeans
|
|
119
|
+
### `poorPerformanceMeans`: what slow would mean
|
|
118
120
|
|
|
119
121
|
Translate slowness into consequences for the people running the system: latency users notice, server
|
|
120
122
|
capacity, cloud spend, cold-start time, memory pressure.
|
|
121
123
|
|
|
122
|
-
> **Good:** "A slowdown adds directly to every response time and to compute cost per request
|
|
124
|
+
> **Good:** "A slowdown adds directly to every response time and to compute cost per request. At
|
|
123
125
|
> 10,000 items a quadratic regression would turn a 50 ms commit into several seconds and exhaust
|
|
124
126
|
> request timeouts."
|
|
125
127
|
>
|
|
126
128
|
> **Bad:** "Performance would be worse."
|
|
127
129
|
|
|
128
|
-
### `expectedComplexity` and `complexityReason
|
|
130
|
+
### `expectedComplexity` and `complexityReason`: the big-O and why
|
|
129
131
|
|
|
130
132
|
Pick the class you **expect from reading the code** before you run anything:
|
|
131
133
|
|
|
@@ -139,27 +141,27 @@ Pick the class you **expect from reading the code** before you run anything:
|
|
|
139
141
|
| `cubic-or-worse` | O(n³) or worse | Anything faster-growing than quadratic. |
|
|
140
142
|
|
|
141
143
|
The reason is one or two sentences that **name the mechanism**, so a reader can check it against the
|
|
142
|
-
code
|
|
143
|
-
run, the report compares the measured class with this one. **Agreement** is the goal
|
|
144
|
-
means the documentation or the code is wrong
|
|
145
|
-
(constant/logarithmic, linear/linearithmic) are reported as "close" because a bounded ladder
|
|
146
|
-
reliably separate them.
|
|
144
|
+
code. For example, "walks every newly arrived item once to freeze it" is a reason; "it is linear" is not.
|
|
145
|
+
After the run, the report compares the measured class with this one. **Agreement** is the goal. A
|
|
146
|
+
**difference** means the documentation or the code is wrong, and both are worth fixing. Neighbouring
|
|
147
|
+
classes (constant/logarithmic, linear/linearithmic) are reported as "close" because a bounded ladder
|
|
148
|
+
cannot reliably separate them.
|
|
147
149
|
|
|
148
|
-
### `variables
|
|
150
|
+
### `variables`: everything that could change the cost
|
|
149
151
|
|
|
150
152
|
One row per variable from the checklist above, each marked:
|
|
151
153
|
|
|
152
|
-
- `swept
|
|
153
|
-
- `variant
|
|
154
|
-
- `fixed
|
|
154
|
+
- `swept`: it is the size axis.
|
|
155
|
+
- `variant`: measured separately; describe each in `variants`.
|
|
156
|
+
- `fixed`: held constant on purpose; give the `value`, so a reader knows what was and was not
|
|
155
157
|
covered.
|
|
156
158
|
|
|
157
|
-
### `covers` and `notBenchmarked
|
|
159
|
+
### `covers` and `notBenchmarked`: every input of every public function
|
|
158
160
|
|
|
159
161
|
A public function with more than one input (several parameters, or an object with several keys) has
|
|
160
|
-
several things that can change its cost
|
|
161
|
-
|
|
162
|
-
|
|
162
|
+
several things that can change its cost. The `BenchmarkCoverage` check fails until each one is accounted
|
|
163
|
+
for. In the benchmark that exercises the function, list what you did with every parameter, and with every
|
|
164
|
+
key of an object parameter:
|
|
163
165
|
|
|
164
166
|
```js
|
|
165
167
|
covers: {
|
|
@@ -172,17 +174,23 @@ covers: {
|
|
|
172
174
|
},
|
|
173
175
|
```
|
|
174
176
|
|
|
175
|
-
Each is one of
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
177
|
+
Each is one of:
|
|
178
|
+
|
|
179
|
+
- `swept`: the suite's unit varies it;
|
|
180
|
+
- `sweep:<variable>`: one of the function's `sweeps`;
|
|
181
|
+
- `variant:<name>`: one of its `variants`;
|
|
182
|
+
- `fixed: <why>`: held at one value, and why;
|
|
183
|
+
- `not covered: <why>`: left out, and why.
|
|
184
|
+
|
|
185
|
+
A function that has no benchmark at all is listed in the suite's `notBenchmarked` with the reason. One
|
|
186
|
+
such reason: a benchmarked function that calls it already accounts for its cost.
|
|
179
187
|
|
|
180
|
-
### `sweeps
|
|
188
|
+
### `sweeps`: a second variable that scales
|
|
181
189
|
|
|
182
190
|
Some functions grow with more than one input: the number of records _and_ how many fields each record
|
|
183
191
|
has. One big-O for both would be wrong, because the two do not have to grow at the same rate. List each
|
|
184
|
-
other variable that scales under `sweeps
|
|
185
|
-
held at its `baseValue
|
|
192
|
+
other variable that scales under `sweeps`, and the suite measures it on its own. Every other variable is
|
|
193
|
+
held at its `baseValue`; the suite's own unit is held at `primaryValue`, by default the typical size:
|
|
186
194
|
|
|
187
195
|
```js
|
|
188
196
|
sweeps: [
|
|
@@ -197,28 +205,30 @@ sweeps: [
|
|
|
197
205
|
```
|
|
198
206
|
|
|
199
207
|
The variable must also be listed under `variables` as `swept`. `setup` (or `input`) receives a third
|
|
200
|
-
argument, `{ variable, values }
|
|
201
|
-
measurement, so one function builds its input for any of them. In a sweep of another variable, `n` is
|
|
202
|
-
|
|
203
|
-
|
|
208
|
+
argument, `{ variable, values }`. It holds the variable being swept and the value of every variable at
|
|
209
|
+
this measurement, so one function builds its input for any of them. In a sweep of another variable, `n` is
|
|
210
|
+
that variable's value, not the size of the unit. Always read every variable from `context.values`, which
|
|
211
|
+
is right for the primary sweep too. Each sweep is its own group (`fn:<id>#<variable>`), with its own
|
|
212
|
+
growth class and its own documented big-O. It also has its own row in the report, the history and the
|
|
213
|
+
pull-request summary.
|
|
204
214
|
|
|
205
|
-
### `variants
|
|
215
|
+
### `variants`: measured alternatives
|
|
206
216
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
chart in history, so a regression in one path cannot hide behind another.
|
|
217
|
+
Some variables change the algorithm: cache hit vs miss, flat vs nested, success vs error. Measure each
|
|
218
|
+
alternative as a named variant across the whole ladder. Each variant is its own row in the report and its
|
|
219
|
+
own chart in history, so a regression in one path cannot hide behind another.
|
|
210
220
|
|
|
211
|
-
### `notCovered
|
|
221
|
+
### `notCovered`: the honest gaps
|
|
212
222
|
|
|
213
223
|
Name what you chose not to measure and why ("concurrent callers: the function is synchronous and
|
|
214
224
|
keeps no shared state"). A reader should never wonder whether something was forgotten.
|
|
215
225
|
|
|
216
|
-
### `inEndToEnd
|
|
226
|
+
### `inEndToEnd`: where it fits in the whole
|
|
217
227
|
|
|
218
228
|
State how often one end-to-end operation calls this function (`callsPerOperation`, a number or a
|
|
219
229
|
function of `n`) and describe the call. This is what lets section 3 of the report attribute the
|
|
220
|
-
time of one operation to the functions that cause it. A function measured alone includes everything
|
|
221
|
-
|
|
230
|
+
time of one operation to the functions that cause it. A function measured alone includes everything it
|
|
231
|
+
calls. When one attributed function calls another, list the callee's id in `includes`. Its time is then
|
|
222
232
|
subtracted from the caller and counted once.
|
|
223
233
|
|
|
224
234
|
### The end-to-end block
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# env-cap: performance and cost report
|
|
2
2
|
|
|
3
|
-
What does adopting this package add to latency, CPU, memory and compute spend
|
|
3
|
+
What does adopting this package add to latency, CPU, memory and compute spend, and how does that grow with workload? Three views, from the whole to the part: **(1)** the end-to-end total, **(2)** every function on its own, **(3)** which functions make up the total. New to benchmarks? Read [READING-BENCHMARKS.md](../READING-BENCHMARKS.md) first.
|
|
4
4
|
|
|
5
5
|
## What adopting this package costs
|
|
6
6
|
|
|
7
7
|
For a typical workload of **640 variables** per operation, routing the work through `env-cap` adds **139 ms** per operation compared with a bare-minimum baseline (**20× the baseline**). Overall, it grows O(n) with workload size (measured exponent 0.90). At list prices that is on the order of **~$0.1 – $10 per million operations** of compute.
|
|
8
8
|
|
|
9
|
-
> The relative figure and the growth class are the steadier ones
|
|
9
|
+
> The relative figure and the growth class are the steadier ones. Both come from the same run, so they depend far less on the machine than a time does. Runtime behaviour and contention on the runner still move them. The growth class is an empirical fit to the sizes tested, not a proof of asymptotic complexity. Dollar figures are **order-of-magnitude** estimates from published list prices on shared hardware (see _Cost model_ below). They show scale, not a budget.
|
|
10
10
|
|
|
11
11
|
| Cost | Typical (640 variables) | Largest (5120 variables) |
|
|
12
12
|
| --- | --- | --- |
|
|
@@ -25,7 +25,7 @@ Shows what a pipeline run or a developer pays to regenerate every env-cap artifa
|
|
|
25
25
|
- **Baseline (no package):** Walk the project directory and read every schema file as text -- no env-cap.
|
|
26
26
|
- **With the package:** `generateEnvArtifacts` discovers, parses and links every schema once and writes the manifest, the documentation and the usage report.
|
|
27
27
|
|
|
28
|
-
Both sides use empty or minimal functions on purpose
|
|
28
|
+
Both sides use empty or minimal functions on purpose. The difference is then the package's own cost, not the cost of the work an application would plug into it. Real applications add their own work on top; this is the floor the package imposes.
|
|
29
29
|
|
|
30
30
|
**Variables that could change this result**
|
|
31
31
|
|
|
@@ -38,7 +38,7 @@ Both sides use empty or minimal functions on purpose, so the difference is the p
|
|
|
38
38
|
| artifacts requested | fixed at "manifest + docs + usage report" | Requesting fewer artifacts costs less; each standalone generator is measured below. |
|
|
39
39
|
| cache state | fixed at "no incremental cache" | Every run parses from scratch; the tooling keeps no build cache. |
|
|
40
40
|
|
|
41
|
-
The baseline is an empty or minimal function, so it costs almost nothing
|
|
41
|
+
The baseline is an empty or minimal function, so it costs almost nothing. The _relative_ overhead can then look enormous (shown as a multiple of the baseline). Read the absolute columns (time, CPU and dollars added): they are what a bill and a latency budget are made of.
|
|
42
42
|
|
|
43
43
|
| variables | Baseline | With package | Added | Added vs baseline | Added CPU | Est. $ / 1M ops |
|
|
44
44
|
| --- | --- | --- | --- | --- | --- | --- |
|
|
@@ -56,7 +56,7 @@ The baseline is an empty or minimal function, so it costs almost nothing and the
|
|
|
56
56
|
|
|
57
57
|
## 2. Function by function
|
|
58
58
|
|
|
59
|
-
Every function the package exposes is measured on its own across the full size ladder
|
|
59
|
+
Every function the package exposes is measured on its own across the full size ladder. Its measured growth rate is then compared with the big-O we documented for it. A function whose measured class **differs** from its documented one is the most useful thing in this report. Either the documentation or the code is wrong.
|
|
60
60
|
|
|
61
61
|
| Function | Varies | Documented | Measured | Agreement | At 640 | At 5120 |
|
|
62
62
|
| --- | --- | --- | --- | --- | --- | --- |
|
|
@@ -123,7 +123,7 @@ Every function the package exposes is measured on its own across the full size l
|
|
|
123
123
|
|
|
124
124
|
**In the end-to-end run:** Once at the start of every artifact generation.
|
|
125
125
|
|
|
126
|
-
**Measured: O(n)** (exponent 0.88, 9 sizes)
|
|
126
|
+
**Measured: O(n)** (exponent 0.88, 9 sizes): ✅ matches.
|
|
127
127
|
|
|
128
128
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
129
129
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -164,7 +164,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
164
164
|
|
|
165
165
|
**Deliberately not covered**
|
|
166
166
|
|
|
167
|
-
- **incompatible schemas
|
|
167
|
+
- **incompatible schemas**: Files the parser cannot handle take a warning path; the benchmark uses only supported schemas.
|
|
168
168
|
|
|
169
169
|
#### Variant `all-contracts`
|
|
170
170
|
|
|
@@ -172,7 +172,7 @@ The manifest covers every schema file in the project.
|
|
|
172
172
|
|
|
173
173
|
**Expected for this variant: O(n log n).** Parsing and rendering are linear in the source text, but the model sorts every variable and declaration by key (contract-model.ts, compatibility.ts) so the manifest and the compatibility findings are stable, and that sort adds an n log n term that shows once the whole project is covered.
|
|
174
174
|
|
|
175
|
-
**Measured: O(n)** (exponent 0.89, 9 sizes)
|
|
175
|
+
**Measured: O(n)** (exponent 0.89, 9 sizes): 🟡 close (neighbouring class).
|
|
176
176
|
|
|
177
177
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
178
178
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -198,7 +198,7 @@ An `include` selects one schema file; discovery still walks the tree but only th
|
|
|
198
198
|
|
|
199
199
|
**Expected for this variant: O(log n).** Only the one selected file is parsed and rendered, but the directory walk that still happens grows with the tree. Across the measured sizes it is a small, sub-linear share of the cost, so the curve rises slowly without being flat.
|
|
200
200
|
|
|
201
|
-
**Measured: O(log n)** (exponent 0.62, 9 sizes)
|
|
201
|
+
**Measured: O(log n)** (exponent 0.62, 9 sizes): ✅ matches.
|
|
202
202
|
|
|
203
203
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
204
204
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -243,7 +243,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
243
243
|
|
|
244
244
|
One short description per variable.
|
|
245
245
|
|
|
246
|
-
**Measured: O(n)** (exponent 0.73, 9 sizes)
|
|
246
|
+
**Measured: O(n)** (exponent 0.73, 9 sizes): ✅ matches.
|
|
247
247
|
|
|
248
248
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
249
249
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -267,7 +267,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
267
267
|
|
|
268
268
|
Long descriptions and extra documentation fields per variable (a heavily documented enterprise).
|
|
269
269
|
|
|
270
|
-
**Measured: O(n)** (exponent 0.76, 9 sizes)
|
|
270
|
+
**Measured: O(n)** (exponent 0.76, 9 sizes): ✅ matches.
|
|
271
271
|
|
|
272
272
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
273
273
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -308,7 +308,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
308
308
|
|
|
309
309
|
**In the end-to-end run:** Not called by the end-to-end operation: generateEnvArtifacts builds the usage report from its shared parse, while this standalone generator parses again.
|
|
310
310
|
|
|
311
|
-
**Measured: O(n)** (exponent 0.80, 9 sizes)
|
|
311
|
+
**Measured: O(n)** (exponent 0.80, 9 sizes): ✅ matches.
|
|
312
312
|
|
|
313
313
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
314
314
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -349,7 +349,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
349
349
|
|
|
350
350
|
**In the end-to-end run:** This is the end-to-end operation itself.
|
|
351
351
|
|
|
352
|
-
**Measured: O(n)** (exponent 0.87, 9 sizes)
|
|
352
|
+
**Measured: O(n)** (exponent 0.87, 9 sizes): ✅ matches.
|
|
353
353
|
|
|
354
354
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
355
355
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -440,7 +440,7 @@ How many variables each schema file declares (ten in the main sweep). With the t
|
|
|
440
440
|
| filesystem | fixed at "local SSD through the Node adapter" | A network or virtual filesystem adds latency per file; not covered. |
|
|
441
441
|
| runtime | fixed at "Node (V8)" | Measured on Node only. |
|
|
442
442
|
|
|
443
|
-
**Measured: O(n)** (exponent 0.77, 9 sizes)
|
|
443
|
+
**Measured: O(n)** (exponent 0.77, 9 sizes): ✅ matches.
|
|
444
444
|
|
|
445
445
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
446
446
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -527,7 +527,7 @@ No artifact exists: the model is rebuilt.
|
|
|
527
527
|
|
|
528
528
|
**Deliberately not covered**
|
|
529
529
|
|
|
530
|
-
- **aliased and package imports
|
|
530
|
+
- **aliased and package imports**: Following a tsconfig alias or an allowlisted package adds a resolution per import; the fixtures import nothing.
|
|
531
531
|
|
|
532
532
|
**In the end-to-end run:** Run once inside every generator; measured on its own.
|
|
533
533
|
|
|
@@ -836,7 +836,7 @@ No artifact exists: the model is rebuilt.
|
|
|
836
836
|
| projection shape | fixed at "five derived fields across all six fact models" | A projection that reads fewer fields still pays the clone; one that computes more adds proportional work. |
|
|
837
837
|
| runtime | fixed at "Node (V8)" | Measured on Node only. |
|
|
838
838
|
|
|
839
|
-
**Measured: O(n)** (exponent 1.07, 9 sizes)
|
|
839
|
+
**Measured: O(n)** (exponent 1.07, 9 sizes): ✅ matches.
|
|
840
840
|
|
|
841
841
|
| variables | Median | p95 | CPU (median) | Heap Δ | Ops/s |
|
|
842
842
|
| --- | --- | --- | --- | --- | --- |
|
|
@@ -852,7 +852,7 @@ No artifact exists: the model is rebuilt.
|
|
|
852
852
|
|
|
853
853
|
## 3. What makes up one end-to-end operation
|
|
854
854
|
|
|
855
|
-
Each function's measured cost is multiplied by how many times one end-to-end operation calls it. A function measured on its own includes everything it calls
|
|
855
|
+
Each function's measured cost is multiplied by how many times one end-to-end operation calls it. A function measured on its own includes everything it calls. So where one attributed function calls another, the nested time is subtracted from the caller (its _exclusive_ time). Every moment of the operation then belongs to at most one row. Shares are measured against the whole operation with the package, a figure measured directly. They are not measured against the _added_ time, which is the small difference of two noisy medians. They are estimates (each function was measured separately); the remainder is shown as _unattributed_.
|
|
856
856
|
|
|
857
857
|
**At 640 variables** (whole operation: 147 ms)
|
|
858
858
|
|
|
@@ -876,14 +876,14 @@ Each function's measured cost is multiplied by how many times one end-to-end ope
|
|
|
876
876
|
|
|
877
877
|
## Cost model
|
|
878
878
|
|
|
879
|
-
Estimates use two bracketing price shapes
|
|
879
|
+
Estimates use two bracketing price shapes. **Low** is CPU-priced compute ($0.040 per vCPU-hour, billed on CPU time). **High** is duration-and-memory-priced functions ($0.000017 per GB-second, billed on wall time, at least 128 MB reserved). Both are rounded list prices and change over time; override them in `benchmark.config.json` → `costRates` to match your platform and negotiated pricing.
|
|
880
880
|
|
|
881
881
|
## Environment and method
|
|
882
882
|
|
|
883
883
|
- Run: `2026-10-02T00:45:43.860Z` → `2026-10-02T00:46:55.879Z` (72 s), ci
|
|
884
884
|
- Machine: AMD EPYC 9V45 96-Core Processor, 4 logical core(s) (2 physical), 15990 MB RAM, linux/x64, Node v22.23.3, GitHub Actions
|
|
885
885
|
- Git: `77acd16d0d0b13e9c162061da6d6f7d739436547` on `chore/no-minify-no-dist-urls` (uncommitted changes)
|
|
886
|
-
- Sizes: 20, 40, 80, 160, 320, 640, 1280, 2560, 5120 variables
|
|
886
|
+
- Sizes: 20, 40, 80, 160, 320, 640, 1280, 2560, 5120 variables: One environment variable declared in a schema file. Schema files hold ten variables each, so 640 variables is 64 files -- a large monorepo's worth of configuration. The ladder stops at 5,120 variables (512 files) because `discoverSchemaFiles` refuses more than 1,000 files as a sanity limit, which 10,240 variables would exceed.
|
|
887
887
|
|
|
888
|
-
**Do not compare these numbers with another machine's, another day's, or another package's.** They
|
|
888
|
+
**Do not compare these numbers with another machine's, another day's, or another package's.** They show how _this_ package's cost changes between runs on comparable hardware, and how it scales with size. See [READING-BENCHMARKS.md](../READING-BENCHMARKS.md).
|
|
889
889
|
|