didrun 0.1.0__tar.gz

didrun-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Seth Wheeler
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

didrun-0.1.0/PKG-INFO

@@ -0,0 +1,207 @@
+Metadata-Version: 2.4
+Name: didrun
+Version: 0.1.0
+Summary: An exit code cannot tell you whether anything happened. Did it RUN, did it FAIL, and was the failure the RIGHT one.
+License: MIT
+Keywords: ci,exit-code,subprocess,evidence,no-tests,zero-tests,empty-glob,test-runner,smoke-test,guard,verification,false-positive,silent-failure,zero-dependency
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# `didrun`
+
+An exit code cannot tell you whether anything happened.
+
+`0` means **"I did not fail."** A suite of ten thousand assertions and a suite that
+collected nothing both report it, and no amount of reading the number harder will
+separate them. That is not a nuisance — it is how a check silently stops checking and
+nobody finds out for a year.
+
+```sh
+pip install didrun          # the Python half
+npm install -g didrun       # the JavaScript half
+
+didrun --expect-count "(\d+) passed" -- pytest tests/
+didrun --expect "^ok " -- go test ./...
+didrun --wrote coverage/lcov.info -- npm run coverage
+```
+
+Both halves ship the same command, the same flags and the same exit codes. A CI file
+should not have to ask which one is installed, and `python/tests/test_parity.py` asserts
+the vocabulary they share — the four state names, the two exit codes, and that both
+classify the same run identically.
+
+```
+$ go test ./...
+?   	x	[no test files]
+$ echo $?
+0                       # <- this is the whole problem
+
+$ didrun --expect "^ok " -- go test ./...
+?   	x	[no test files]
+
+[didrun] DID NOT RUN — there is no evidence this command did anything  (exit 0, 296ms)
+  --  output matches /^ok /: nothing in output matched /^ok /
+
+  It exited 0. That is the failure: a check that stopped checking reports exactly this.
+$ echo $?
+3
+```
+
+## The rule
+
+> A check must answer separately whether it **RAN**, whether it **FAILED**, and whether
+> the failure was the **RIGHT** one. Collapsing any two of those three is how every
+> defect in this family happens.
+
+So this returns four states rather than a number:
+
+| state | means | exit |
+|---|---|---|
+| `did-not-run` | no evidence the command did anything — **regardless of exit 0** | **3** |
+| `ran-and-passed` | evidence found, exit 0 | 0 |
+| `ran-and-failed` | evidence found, non-zero exit, failure looked right | the command's own code |
+| `ran-and-failed-wrongly` | it failed, but not the way you said it would | **4** |
+
+The fourth exists because *"it failed"* is not *"my check caught something"*. A suite
+that dies on a syntax error fails; so does one that caught your mutation. Scoring those
+alike is the difference between a harness that works and one that reports success for a
+file it never parsed.
+
+`did-not-run` gets its own exit code and never borrows the command's — including when
+the command itself failed. *"Your tests failed"* and *"you have no tests"* send you to
+different places.
+
+## Read this first: some runners already answer question one
+
+Let them. Checked, not assumed:
+
+| | answers "did it run"? |
+|---|---|
+| `pytest` | **yes** — exits 5 when it collects nothing |
+| `jest`, `vitest` | **yes** — fail by default when no test matches (`--passWithNoTests` is the decision this tool exists to argue with) |
+| `go test ./...` | **no** — prints `[no test files]` and exits **0**. Measured, not assumed |
+| linters given a glob that matched nothing | generally no |
+| any shell step in any CI file | no notion of the question at all |
+
+If your runner is in the first two rows, you may not need this. It is for the rest.
+
+## Evidence
+
+At least one predicate is required. **With none, `run()` throws and the CLI exits 2** —
+a tool that silently degrades into forwarding the exit code is the thing it is replacing.
+
+| flag | evidence |
+|---|---|
+| `--expect REGEX` | combined output matches |
+| `--expect-stdout` / `--expect-stderr` | one stream matches |
+| `--expect-count REGEX` | the first capture group is a count, `>= --min` (default 1) |
+| `--wrote PATH` | the file was actually written **during this run** |
+| `--took-at-least MS` | a duration floor (weak — prefer a count) |
+
+### `--expect-count` is the one that matters
+
+The thing that makes a green run meaningless is almost always a **zero**, not an
+absence: `0 passed`, `Ran 0 tests`, `0 files checked`. A pattern that only asks whether
+the *line* was printed is satisfied by exactly the run it was meant to catch, because
+the runner cheerfully prints its zero. Both behaviours are pinned by one test:
+
+```js
+// `0 passed in 0.01s`, exit 0
+count(/(\d+) passed/)   -> did-not-run      "reported 0"
+matches(/\d+ passed/)   -> ran-and-passed   // fooled, as advertised
+```
+
+### `--wrote` is not "the file exists"
+
+A `junit.xml` left over from yesterday exists, and a runner that never started leaves it
+exactly where it was. The file must be **created, changed, or rewritten during the run**
+— a byte-for-byte identical artefact is reported as the stale thing it is.
+
+## API
+
+```python
+from didrun import run, report, exit_code_for, evidence
+
+result = run(["pytest", "tests/"],
+             evidence=[evidence.count(r"(\d+) passed"), evidence.wrote("junit.xml")],
+             expect_failure=r"AssertionError",   # when it fails, it must fail THIS way
+             timeout=600)                        # killed and still classified
+
+result.state    # "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks   # every predicate, with what it looked for and what it found
+```
+
+```js
+import { run, report, exitCodeFor, evidence } from "didrun";
+
+const result = await run(["pytest", "tests/"], {
+  evidence: [evidence.count(/(\d+) passed/), evidence.wrote("junit.xml")],
+  expectFailure: /AssertionError/,   // when it fails, it must fail THIS way
+  timeout: 600_000,                  // killed and still classified
+});
+
+result.state   // "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks  // every predicate, with what it looked for and what it found
+process.exitCode = exitCodeFor(result);
+```
+
+Every predicate must hold — `every`, not `some`. The report prints what was looked for
+**and what was found, including when everything passed**, because a check whose output
+is only a verdict is one nobody can audit.
+
+A timeout is classified rather than swallowed: a killed check is a check that did not
+finish, never one that passed.
+
+## Prior art
+
+Swept across **both registries** on mechanism nouns — the first pass queried npm only,
+and npm-only sweeping is what nearly missed `crosshair` elsewhere in this line of work.
+
+On npm, `keywords:no-tests` returns one package (a CRA template) and `keywords:ci-guard`
+returns zero — dead tags, so wrong bucket names rather than open fields. `test-silence`
+inventories *skipped* tests from git history; `jest-fail-on-console` and
+`cypress-fail-fast` change what a runner does *while* it runs.
+
+On PyPI, searched by name across the full 881,198-entry index plus web search, three
+neighbours are real and none of them is this:
+
+- **`pytest-custom-exit-code`** changes *pytest's* exit code when nothing is collected.
+  That is one runner answering question one for itself — the thing this wraps, not a
+  replacement for it.
+- **`evidence-gate`** audits GitHub Actions **evidence bundles** after the fact: whether
+  an audit trail is complete and temporally bounded. A different question, downstream.
+- **`ranit`** reports which functions *in your diff* were executed by nothing, by
+  intersecting a coverage database with the git diff. Function granularity via coverage,
+  not an arbitrary command with a supplied predicate — and the closest thing in spirit
+  to this anywhere.
+
+Nothing found wraps an arbitrary command and asks whether it did anything.
+
+## Limits
+
+- **Evidence is only as good as the pattern you supply.** This does not know what your
+  command should print; it makes you say so, once, where the next person can read it.
+- `--took-at-least` cannot prove work happened. It is marked `[weak]` in the report and
+  it is here because a suite that takes a minute finishing in 9ms is a real signal.
+- It does not parse junit/TAP. `--wrote` plus `--expect-count` covers most of what that
+  would buy.
+- Zero dependencies in either half. Node ≥ 18, Python ≥ 3.9.
+
+## Tests
+
+```sh
+npm test                                        # 22
+python3 -m unittest discover -s python/tests    # 21, four of them the cross-half contract
+```
+
+22 JavaScript tests and 21 Python tests, no dependencies in either half. Six mutations to the source — never reporting `did-not-run`,
+treating it as success, accepting one predicate instead of all, allowing a run with no
+evidence, ignoring the count floor, and accepting a stale artefact — were each caught by
+the test that should catch them.

didrun-0.1.0/README.md

@@ -0,0 +1,191 @@
+# `didrun`
+
+An exit code cannot tell you whether anything happened.
+
+`0` means **"I did not fail."** A suite of ten thousand assertions and a suite that
+collected nothing both report it, and no amount of reading the number harder will
+separate them. That is not a nuisance — it is how a check silently stops checking and
+nobody finds out for a year.
+
+```sh
+pip install didrun          # the Python half
+npm install -g didrun       # the JavaScript half
+
+didrun --expect-count "(\d+) passed" -- pytest tests/
+didrun --expect "^ok " -- go test ./...
+didrun --wrote coverage/lcov.info -- npm run coverage
+```
+
+Both halves ship the same command, the same flags and the same exit codes. A CI file
+should not have to ask which one is installed, and `python/tests/test_parity.py` asserts
+the vocabulary they share — the four state names, the two exit codes, and that both
+classify the same run identically.
+
+```
+$ go test ./...
+?   	x	[no test files]
+$ echo $?
+0                       # <- this is the whole problem
+
+$ didrun --expect "^ok " -- go test ./...
+?   	x	[no test files]
+
+[didrun] DID NOT RUN — there is no evidence this command did anything  (exit 0, 296ms)
+  --  output matches /^ok /: nothing in output matched /^ok /
+
+  It exited 0. That is the failure: a check that stopped checking reports exactly this.
+$ echo $?
+3
+```
+
+## The rule
+
+> A check must answer separately whether it **RAN**, whether it **FAILED**, and whether
+> the failure was the **RIGHT** one. Collapsing any two of those three is how every
+> defect in this family happens.
+
+So this returns four states rather than a number:
+
+| state | means | exit |
+|---|---|---|
+| `did-not-run` | no evidence the command did anything — **regardless of exit 0** | **3** |
+| `ran-and-passed` | evidence found, exit 0 | 0 |
+| `ran-and-failed` | evidence found, non-zero exit, failure looked right | the command's own code |
+| `ran-and-failed-wrongly` | it failed, but not the way you said it would | **4** |
+
+The fourth exists because *"it failed"* is not *"my check caught something"*. A suite
+that dies on a syntax error fails; so does one that caught your mutation. Scoring those
+alike is the difference between a harness that works and one that reports success for a
+file it never parsed.
+
+`did-not-run` gets its own exit code and never borrows the command's — including when
+the command itself failed. *"Your tests failed"* and *"you have no tests"* send you to
+different places.
+
+## Read this first: some runners already answer question one
+
+Let them. Checked, not assumed:
+
+| | answers "did it run"? |
+|---|---|
+| `pytest` | **yes** — exits 5 when it collects nothing |
+| `jest`, `vitest` | **yes** — fail by default when no test matches (`--passWithNoTests` is the decision this tool exists to argue with) |
+| `go test ./...` | **no** — prints `[no test files]` and exits **0**. Measured, not assumed |
+| linters given a glob that matched nothing | generally no |
+| any shell step in any CI file | no notion of the question at all |
+
+If your runner is in the first two rows, you may not need this. It is for the rest.
+
+## Evidence
+
+At least one predicate is required. **With none, `run()` throws and the CLI exits 2** —
+a tool that silently degrades into forwarding the exit code is the thing it is replacing.
+
+| flag | evidence |
+|---|---|
+| `--expect REGEX` | combined output matches |
+| `--expect-stdout` / `--expect-stderr` | one stream matches |
+| `--expect-count REGEX` | the first capture group is a count, `>= --min` (default 1) |
+| `--wrote PATH` | the file was actually written **during this run** |
+| `--took-at-least MS` | a duration floor (weak — prefer a count) |
+
+### `--expect-count` is the one that matters
+
+The thing that makes a green run meaningless is almost always a **zero**, not an
+absence: `0 passed`, `Ran 0 tests`, `0 files checked`. A pattern that only asks whether
+the *line* was printed is satisfied by exactly the run it was meant to catch, because
+the runner cheerfully prints its zero. Both behaviours are pinned by one test:
+
+```js
+// `0 passed in 0.01s`, exit 0
+count(/(\d+) passed/)   -> did-not-run      "reported 0"
+matches(/\d+ passed/)   -> ran-and-passed   // fooled, as advertised
+```
+
+### `--wrote` is not "the file exists"
+
+A `junit.xml` left over from yesterday exists, and a runner that never started leaves it
+exactly where it was. The file must be **created, changed, or rewritten during the run**
+— a byte-for-byte identical artefact is reported as the stale thing it is.
+
+## API
+
+```python
+from didrun import run, report, exit_code_for, evidence
+
+result = run(["pytest", "tests/"],
+             evidence=[evidence.count(r"(\d+) passed"), evidence.wrote("junit.xml")],
+             expect_failure=r"AssertionError",   # when it fails, it must fail THIS way
+             timeout=600)                        # killed and still classified
+
+result.state    # "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks   # every predicate, with what it looked for and what it found
+```
+
+```js
+import { run, report, exitCodeFor, evidence } from "didrun";
+
+const result = await run(["pytest", "tests/"], {
+  evidence: [evidence.count(/(\d+) passed/), evidence.wrote("junit.xml")],
+  expectFailure: /AssertionError/,   // when it fails, it must fail THIS way
+  timeout: 600_000,                  // killed and still classified
+});
+
+result.state   // "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks  // every predicate, with what it looked for and what it found
+process.exitCode = exitCodeFor(result);
+```
+
+Every predicate must hold — `every`, not `some`. The report prints what was looked for
+**and what was found, including when everything passed**, because a check whose output
+is only a verdict is one nobody can audit.
+
+A timeout is classified rather than swallowed: a killed check is a check that did not
+finish, never one that passed.
+
+## Prior art
+
+Swept across **both registries** on mechanism nouns — the first pass queried npm only,
+and npm-only sweeping is what nearly missed `crosshair` elsewhere in this line of work.
+
+On npm, `keywords:no-tests` returns one package (a CRA template) and `keywords:ci-guard`
+returns zero — dead tags, so wrong bucket names rather than open fields. `test-silence`
+inventories *skipped* tests from git history; `jest-fail-on-console` and
+`cypress-fail-fast` change what a runner does *while* it runs.
+
+On PyPI, searched by name across the full 881,198-entry index plus web search, three
+neighbours are real and none of them is this:
+
+- **`pytest-custom-exit-code`** changes *pytest's* exit code when nothing is collected.
+  That is one runner answering question one for itself — the thing this wraps, not a
+  replacement for it.
+- **`evidence-gate`** audits GitHub Actions **evidence bundles** after the fact: whether
+  an audit trail is complete and temporally bounded. A different question, downstream.
+- **`ranit`** reports which functions *in your diff* were executed by nothing, by
+  intersecting a coverage database with the git diff. Function granularity via coverage,
+  not an arbitrary command with a supplied predicate — and the closest thing in spirit
+  to this anywhere.
+
+Nothing found wraps an arbitrary command and asks whether it did anything.
+
+## Limits
+
+- **Evidence is only as good as the pattern you supply.** This does not know what your
+  command should print; it makes you say so, once, where the next person can read it.
+- `--took-at-least` cannot prove work happened. It is marked `[weak]` in the report and
+  it is here because a suite that takes a minute finishing in 9ms is a real signal.
+- It does not parse junit/TAP. `--wrote` plus `--expect-count` covers most of what that
+  would buy.
+- Zero dependencies in either half. Node ≥ 18, Python ≥ 3.9.
+
+## Tests
+
+```sh
+npm test                                        # 22
+python3 -m unittest discover -s python/tests    # 21, four of them the cross-half contract
+```
+
+22 JavaScript tests and 21 Python tests, no dependencies in either half. Six mutations to the source — never reporting `did-not-run`,
+treating it as success, accepting one predicate instead of all, allowing a run with no
+evidence, ignoring the count floor, and accepting a stale artefact — were each caught by
+the test that should catch them.

didrun-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "didrun"
+version = "0.1.0"
+description = "An exit code cannot tell you whether anything happened. Did it RUN, did it FAIL, and was the failure the RIGHT one."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+keywords = [
+  "ci", "exit-code", "subprocess", "evidence", "no-tests", "zero-tests",
+  "empty-glob", "test-runner", "smoke-test", "guard", "verification",
+  "false-positive", "silent-failure", "zero-dependency",
+]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Topic :: Software Development :: Testing",
+]
+# LAYER 0. No in-network dependencies: this is a root, like assay-checks and lexindex,
+# and `tools/depgraph.py` in the reckoner tree asserts the graph stays acyclic.
+dependencies = []
+
+[project.scripts]
+didrun = "didrun.cli:main"
+
+[tool.setuptools]
+# The Python half lives in `python/` beside `js/`, and is still imported as `didrun`.
+package-dir = { "" = "python" }
+packages = ["didrun"]

didrun-0.1.0/python/didrun/__init__.py

@@ -0,0 +1,25 @@
+r"""didrun — an exit code cannot tell you whether anything happened.
+
+    from didrun import run, evidence
+    result = run(["pytest", "tests/"], evidence=[evidence.count(r"(\d+) passed")])
+    result.state   # "did-not-run" | "ran-and-passed" | "ran-and-failed" | ...
+"""
+
+from . import evidence
+from .core import (
+    DID_NOT_RUN,
+    EXIT_DID_NOT_RUN,
+    EXIT_WRONG_FAILURE,
+    RAN_AND_FAILED,
+    RAN_AND_FAILED_WRONGLY,
+    RAN_AND_PASSED,
+    Result,
+    exit_code_for,
+    report,
+    run,
+)
+
+__all__ = ["run", "report", "exit_code_for", "evidence", "Result",
+           "DID_NOT_RUN", "RAN_AND_PASSED", "RAN_AND_FAILED", "RAN_AND_FAILED_WRONGLY",
+           "EXIT_DID_NOT_RUN", "EXIT_WRONG_FAILURE"]
+__version__ = "0.1.0"

didrun-0.1.0/python/didrun/cli.py

@@ -0,0 +1,150 @@
+"""`didrun` — run a command and say whether it actually did anything.
+
+    didrun --expect-count "(\\d+) passed" -- pytest tests/
+    didrun --expect "^ok " -- go test ./...
+    didrun --wrote coverage.xml -- coverage run -m pytest
+
+Exit codes: 0 ran and passed · 3 DID NOT RUN · 4 failed the wrong way · otherwise the
+command's own status.
+
+THE FLAGS AND THE EXIT CODES ARE THE JAVASCRIPT HALF'S, deliberately. A CI file should
+not have to ask which half is installed, and `python/tests/test_parity.py` asserts the
+vocabulary the two share.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+
+from . import evidence as ev
+from .core import EXIT_DID_NOT_RUN, exit_code_for, report, run
+
+USAGE = """didrun — an exit code cannot tell you whether anything happened.
+
+  didrun [evidence...] [options] -- COMMAND...
+
+Evidence (at least one is required — without it there is nothing to add to the
+exit code, and this refuses rather than pretending):
+  --expect REGEX          combined output must match
+  --expect-stdout REGEX   stdout must match
+  --expect-stderr REGEX   stderr must match
+  --expect-count REGEX    first capture group is a count, and must be >= --min
+  --min N                 the floor for --expect-count (default 1)
+  --wrote PATH            the file must have been written during this run
+  --took-at-least MS      a floor on the duration (weak; prefer a count)
+
+Options:
+  --expect-failure REGEX  when it fails, the output must match this or the
+                          failure is scored as the WRONG one (exit 4)
+  --timeout SECONDS       kill the command and classify anyway
+  --quiet                 only print on a bad verdict
+  --json                  print the result as JSON
+  -h, --help
+
+Exit: 0 ran and passed · 3 did not run · 4 failed the wrong way ·
+      otherwise the command's own status.
+"""
+
+
+def main(argv=None):
+    argv = list(sys.argv[1:] if argv is None else argv)
+    if not argv or "-h" in argv or "--help" in argv:
+        sys.stderr.write(USAGE)
+        return 0 if argv else 2
+    if "--" not in argv:
+        sys.stderr.write("didrun: put the command after `--`\n")
+        return 2
+    split = argv.index("--")
+    flags, command = argv[:split], argv[split + 1:]
+    if not command:
+        sys.stderr.write("didrun: nothing to run after `--`\n")
+        return 2
+
+    predicates, counts = [], []
+    expect_failure = timeout = None
+    minimum, quiet, as_json = 1, False, False
+
+    i = 0
+    while i < len(flags):
+        flag = flags[i]
+
+        def value():
+            nonlocal i
+            i += 1
+            if i >= len(flags):
+                raise SystemExit(f"didrun: {flag} needs a value")
+            return flags[i]
+
+        if flag == "--expect":
+            predicates.append(ev.matches(value(), "output"))
+        elif flag == "--expect-stdout":
+            predicates.append(ev.matches(value(), "stdout"))
+        elif flag == "--expect-stderr":
+            predicates.append(ev.matches(value(), "stderr"))
+        elif flag == "--expect-count":
+            counts.append(value())
+        elif flag == "--min":
+            minimum = int(value())
+        elif flag == "--wrote":
+            predicates.append(ev.wrote(value()))
+        elif flag == "--took-at-least":
+            predicates.append(ev.took_at_least(int(value())))
+        elif flag == "--expect-failure":
+            expect_failure = value()
+        elif flag == "--timeout":
+            timeout = float(value())
+        elif flag == "--quiet":
+            quiet = True
+        elif flag == "--json":
+            as_json = True
+        else:
+            sys.stderr.write(f"didrun: unknown option {flag}\n")
+            return 2
+        i += 1
+
+    # `--min` is applied after the loop so it works however it was ordered on the
+    # command line. A flag that silently means different things depending on where you
+    # put it is a flag that will be wrong in somebody's CI file.
+    for pattern in counts:
+        predicates.append(ev.count(pattern, minimum=minimum))
+
+    if not predicates:
+        sys.stderr.write(
+            "didrun: give at least one piece of evidence (--expect, --expect-count,\n"
+            "        --wrote, --took-at-least).\n"
+            "        Without one, this can only report the exit code — which is the\n"
+            "        thing it exists to stop you trusting.\n"
+        )
+        return 2
+
+    try:
+        result = run(command, evidence=predicates, expect_failure=expect_failure,
+                     timeout=timeout)
+    except (OSError, FileNotFoundError) as exc:
+        sys.stderr.write(f"didrun: cannot run {command[0]!r} ({exc})\n")
+        return 2
+
+    if as_json:
+        json.dump({
+            "command": result.command,
+            "state": result.state,
+            "code": result.code,
+            "duration_ms": result.duration_ms,
+            "killed": result.killed,
+            "checks": [{"name": c.name, "satisfied": c.satisfied,
+                        "detail": c.detail, "weak": c.weak} for c in result.checks],
+        }, sys.stdout, indent=2, sort_keys=True)
+        sys.stdout.write("\n")
+    else:
+        # The command's own output first, then the verdict, so the verdict is the last
+        # thing on the screen.
+        sys.stdout.write(result.stdout)
+        sys.stderr.write(result.stderr)
+        if not quiet or not result.ok:
+            sys.stderr.write("\n[didrun] " + report(result) + "\n")
+    return exit_code_for(result)
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

didrun-0.1.0/python/didrun/core.py

@@ -0,0 +1,151 @@
+"""didrun — an exit code cannot tell you whether anything happened.
+
+`0` means "I did not fail". A suite of ten thousand assertions and a suite that
+collected nothing both report it, and no amount of reading the number harder will
+separate them. That is not a nuisance: it is how a check silently stops checking and
+nobody finds out for a year.
+
+The rule this implements is one sentence:
+
+    A check must answer separately whether it RAN, whether it FAILED, and whether the
+    failure was the RIGHT one. Collapsing any two of those three is how every defect in
+    this family happens.
+
+WHAT THIS IS NOT FOR. Several runners already answer the first question for themselves
+and you should let them: `pytest` exits 5 when it collects nothing, and `jest` and
+`vitest` fail by default when no test matches. Reach for this where nothing answers it:
+`go test ./...` prints `[no test files]` and exits 0 — verified, not assumed — as do most
+linters given a glob that matched nothing, and every shell step ever written.
+
+THE STATE NAMES AND EXIT CODES ARE SHARED WITH THE JAVASCRIPT HALF, deliberately. A CI
+file branches on these, and it should not have to ask which half is installed.
+"""
+
+from __future__ import annotations
+
+import re
+import subprocess
+import time
+from dataclasses import dataclass, field
+
+from . import evidence
+from .evidence import Check
+
+DID_NOT_RUN = "did-not-run"
+RAN_AND_PASSED = "ran-and-passed"
+RAN_AND_FAILED = "ran-and-failed"
+RAN_AND_FAILED_WRONGLY = "ran-and-failed-wrongly"
+
+# DID-NOT-RUN GETS ITS OWN EXIT CODE and never borrows the command's. Folding it into 1
+# would put "your tests failed" and "you have no tests" in the same bucket, which are the
+# two states this whole module exists to keep apart.
+EXIT_DID_NOT_RUN = 3
+EXIT_WRONG_FAILURE = 4
+
+
+@dataclass
+class Result:
+    command: list
+    stdout: str
+    stderr: str
+    code: int
+    duration_ms: int
+    killed: bool = False
+    checks: list = field(default_factory=list)
+    state: str = ""
+
+    @property
+    def ok(self):
+        return self.state == RAN_AND_PASSED
+
+
+def run(command, evidence=(), expect_failure=None, timeout=None, cwd=None, env=None):
+    """Run `command`, gather evidence, and classify.
+
+    `evidence` is a sequence of predicates from `didrun.evidence`; at least one is
+    required. REFUSING IS THE POINT: with none this could only forward the exit code,
+    which is the behaviour it is replacing, and a tool that silently degrades into the
+    thing it exists to fix is worse than no tool.
+    """
+    if not command:
+        raise TypeError("run() needs a command as a list of arguments")
+    predicates = list(evidence)
+    if not predicates:
+        raise TypeError(
+            "run() needs at least one evidence predicate; without one it can only "
+            "report the exit code, which is what it exists to stop you trusting"
+        )
+
+    before = [p.before() for p in predicates]
+    started = time.monotonic()
+    killed = False
+    try:
+        proc = subprocess.run(command, capture_output=True, text=True,
+                              timeout=timeout, cwd=cwd, env=env)
+        code, out, err = proc.returncode, proc.stdout or "", proc.stderr or ""
+    except subprocess.TimeoutExpired as expired:
+        # `subprocess.run` has already killed the child by the time this is caught. A
+        # timed-out check is a check that did not finish rather than one that passed.
+        killed = True
+        code = 124  # the conventional timeout status, as `timeout(1)` uses
+        out = (expired.stdout or b"").decode(errors="replace") if expired.stdout else ""
+        err = (expired.stderr or b"").decode(errors="replace") if expired.stderr else ""
+    duration = int((time.monotonic() - started) * 1000)
+
+    result = Result(command=list(command), stdout=out, stderr=err, code=code,
+                    duration_ms=duration, killed=killed)
+    result.checks = [p.check(result, b) for p, b in zip(predicates, before)]
+    ran = all(c.satisfied for c in result.checks)
+
+    if not ran:
+        result.state = DID_NOT_RUN
+    elif code == 0 and not killed:
+        result.state = RAN_AND_PASSED
+    elif expect_failure:
+        pattern = (expect_failure if isinstance(expect_failure, re.Pattern)
+                   else re.compile(expect_failure))
+        result.state = (RAN_AND_FAILED if pattern.search(out + err)
+                        else RAN_AND_FAILED_WRONGLY)
+    else:
+        result.state = RAN_AND_FAILED
+    return result
+
+
+def exit_code_for(result):
+    """The process exit code this result should produce."""
+    if result.state == DID_NOT_RUN:
+        return EXIT_DID_NOT_RUN
+    if result.state == RAN_AND_FAILED_WRONGLY:
+        return EXIT_WRONG_FAILURE
+    if result.state == RAN_AND_PASSED:
+        return 0
+    # A real failure keeps the command's own status, so an existing pipeline that reads
+    # it keeps working.
+    return 1 if result.code == 0 else result.code
+
+
+def report(result):
+    """What to print. It always says what was looked for, including when all passed."""
+    verdict = {
+        DID_NOT_RUN: "DID NOT RUN — there is no evidence this command did anything",
+        RAN_AND_PASSED: "ran, and passed",
+        RAN_AND_FAILED: "ran, and failed",
+        RAN_AND_FAILED_WRONGLY:
+            "FAILED THE WRONG WAY — it failed, but not for the reason you named",
+    }[result.state]
+    lines = [f"{verdict}  (exit {result.code}"
+             f"{', killed' if result.killed else ''}, {result.duration_ms}ms)"]
+    for c in result.checks:
+        mark = "  ok  " if c.satisfied else "  --  "
+        weak = "  [weak]" if c.weak and c.satisfied else ""
+        lines.append(f"{mark}{c.name}: {c.detail}{weak}")
+    if result.state == DID_NOT_RUN and result.code == 0:
+        lines.append("")
+        lines.append("  It exited 0. That is the failure: a check that stopped checking "
+                     "reports exactly this.")
+    return "\n".join(lines)
+
+
+__all__ = ["run", "report", "exit_code_for", "evidence", "Result", "Check",
+           "DID_NOT_RUN", "RAN_AND_PASSED", "RAN_AND_FAILED", "RAN_AND_FAILED_WRONGLY",
+           "EXIT_DID_NOT_RUN", "EXIT_WRONG_FAILURE"]

didrun-0.1.0/python/didrun/evidence.py

@@ -0,0 +1,202 @@
+"""The evidence predicates: how a command proves it did something.
+
+THIS IS THE HALF AN EXIT CODE CANNOT PROVIDE. `0` means "I did not fail", which is what
+a run of ten thousand assertions and a run of nothing both report. Separating them needs
+a second signal, and the only general one is something the command produced: a line of
+output, a count in that output, a file it wrote, or time it spent.
+
+Each predicate answers a `Check` rather than a boolean, because "no evidence" is a
+message somebody has to act on and `False` is not one. The detail is what gets printed,
+so it says what was looked for AND what was there.
+
+Kept deliberately in step with `js/src/evidence.js`: the names, the wording of the
+details and the semantics are one contract, so a CI file that moves between the two
+halves does not change meaning. `python/tests/test_parity.py` asserts the parts that can
+be asserted.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import os
+import re
+from dataclasses import dataclass
+
+
+@dataclass
+class Check:
+    """One predicate's answer: did it hold, and what was actually there."""
+
+    name: str
+    satisfied: bool
+    detail: str
+    weak: bool = False
+
+
+def _pick(result, where):
+    if where == "stdout":
+        return result.stdout
+    if where == "stderr":
+        return result.stderr
+    return result.stdout + result.stderr
+
+
+def _trim(text, limit=60):
+    one = " ".join(str(text).split())
+    return one[:limit] + "…" if len(one) > limit else one
+
+
+def _compile(pattern):
+    return pattern if isinstance(pattern, re.Pattern) else re.compile(pattern)
+
+
+class Predicate:
+    """A named question about a finished run. `before()` runs first when it needs to."""
+
+    weak = False
+
+    def before(self):
+        return None
+
+    def check(self, result, before):  # pragma: no cover - overridden
+        raise NotImplementedError
+
+
+class Matches(Predicate):
+    """A regex the output must match. The bluntest evidence, and often enough."""
+
+    def __init__(self, pattern, where="output"):
+        self.re = _compile(pattern)
+        self.where = where
+        self.name = f"{where} matches {self.re.pattern}"
+
+    def check(self, result, before):
+        text = _pick(result, self.where)
+        found = self.re.search(text)
+        if found:
+            return Check(self.name, True, f"matched {_trim(found.group(0))!r}")
+        empty = "" if text.strip() else f" — {self.where} was empty"
+        return Check(self.name, False,
+                     f"nothing in {self.where} matched {self.re.pattern}{empty}")
+
+
+class Count(Predicate):
+    """A regex whose first group is a COUNT, which must be at least `minimum`.
+
+    THE PREDICATE WORTH REACHING FOR. What makes a green run meaningless is almost
+    always a zero rather than an absence: `0 passed`, `Ran 0 tests`, `0 files checked`.
+    A pattern that only asked whether the LINE was printed is satisfied by exactly the
+    run it is meant to catch, because the runner cheerfully prints its zero.
+    """
+
+    def __init__(self, pattern, minimum=1, where="output"):
+        self.re = _compile(pattern)
+        self.minimum = minimum
+        self.where = where
+        self.name = f"{where} reports at least {minimum} via {self.re.pattern}"
+
+    def check(self, result, before):
+        text = _pick(result, self.where)
+        found = self.re.search(text)
+        if not found:
+            return Check(self.name, False,
+                         f"nothing in {self.where} matched {self.re.pattern}, so no "
+                         f"count was reported at all")
+        raw = found.group(1) if found.groups() else found.group(0)
+        digits = re.sub(r"[^\d-]", "", str(raw))
+        try:
+            n = int(digits)
+        except ValueError:
+            return Check(self.name, False,
+                         f"matched {_trim(found.group(0))!r} but no number could be "
+                         f"read from it")
+        if n >= self.minimum:
+            return Check(self.name, True, f"{n} (needed {self.minimum})")
+        return Check(self.name, False,
+                     f"reported {n}, which is below {self.minimum} — "
+                     f"{_trim(found.group(0))!r}")
+
+
+class Wrote(Predicate):
+    """A file the command must have written DURING the run.
+
+    "Exists" is not the check. A junit.xml left over from yesterday exists, and a runner
+    that never started leaves it exactly where it was.
+    """
+
+    def __init__(self, path):
+        self.path = path
+        self.name = f"{path} was written during the run"
+
+    def _snapshot(self):
+        try:
+            stat = os.stat(self.path)
+            with open(self.path, "rb") as fh:
+                data = fh.read()
+            return {"exists": True, "size": stat.st_size, "mtime": stat.st_mtime,
+                    "digest": hashlib.sha256(data).hexdigest()}
+        except OSError:
+            return {"exists": False, "size": 0, "mtime": 0, "digest": None}
+
+    def before(self):
+        return self._snapshot()
+
+    def check(self, result, before):
+        after = self._snapshot()
+        if not after["exists"]:
+            return Check(self.name, False,
+                         f"{self.path} existed before the run and is gone"
+                         if before["exists"] else f"{self.path} was never created")
+        if not before["exists"]:
+            return Check(self.name, True,
+                         f"{self.path} was created ({after['size']} bytes)")
+        if after["digest"] != before["digest"]:
+            return Check(self.name, True,
+                         f"{self.path} changed ({before['size']} -> {after['size']} bytes)")
+        if after["mtime"] > before["mtime"]:
+            return Check(self.name, True,
+                         f"{self.path} was rewritten with the same content")
+        return Check(self.name, False,
+                     f"{self.path} is byte for byte what it was before the run — a "
+                     f"stale artefact from an earlier run looks exactly like this")
+
+
+class TookAtLeast(Predicate):
+    """A floor on how long the command took.
+
+    THE WEAKEST PREDICATE HERE, and it is included with that said out loud. It cannot
+    prove work happened; it can only catch a suite that takes a minute finishing in nine
+    milliseconds. Prefer a count when the command will give you one.
+    """
+
+    weak = True
+
+    def __init__(self, ms):
+        self.ms = ms
+        self.name = f"the command took at least {ms}ms"
+
+    def check(self, result, before):
+        if result.duration_ms >= self.ms:
+            return Check(self.name, True, f"{result.duration_ms}ms", weak=True)
+        return Check(self.name, False,
+                     f"finished in {result.duration_ms}ms, under the {self.ms}ms floor",
+                     weak=True)
+
+
+class Exits(Predicate):
+    """The command's exit code must be one of `codes`."""
+
+    def __init__(self, codes):
+        self.codes = {codes} if isinstance(codes, int) else set(codes)
+        self.name = f"exit code is one of {', '.join(str(c) for c in sorted(self.codes))}"
+
+    def check(self, result, before):
+        return Check(self.name, result.code in self.codes, f"exited {result.code}")
+
+
+# Short constructors, so a caller writes `evidence.count(...)` in either language.
+matches = Matches
+count = Count
+wrote = Wrote
+took_at_least = TookAtLeast
+exits = Exits

didrun-0.1.0/python/didrun.egg-info/PKG-INFO

@@ -0,0 +1,207 @@
+Metadata-Version: 2.4
+Name: didrun
+Version: 0.1.0
+Summary: An exit code cannot tell you whether anything happened. Did it RUN, did it FAIL, and was the failure the RIGHT one.
+License: MIT
+Keywords: ci,exit-code,subprocess,evidence,no-tests,zero-tests,empty-glob,test-runner,smoke-test,guard,verification,false-positive,silent-failure,zero-dependency
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# `didrun`
+
+An exit code cannot tell you whether anything happened.
+
+`0` means **"I did not fail."** A suite of ten thousand assertions and a suite that
+collected nothing both report it, and no amount of reading the number harder will
+separate them. That is not a nuisance — it is how a check silently stops checking and
+nobody finds out for a year.
+
+```sh
+pip install didrun          # the Python half
+npm install -g didrun       # the JavaScript half
+
+didrun --expect-count "(\d+) passed" -- pytest tests/
+didrun --expect "^ok " -- go test ./...
+didrun --wrote coverage/lcov.info -- npm run coverage
+```
+
+Both halves ship the same command, the same flags and the same exit codes. A CI file
+should not have to ask which one is installed, and `python/tests/test_parity.py` asserts
+the vocabulary they share — the four state names, the two exit codes, and that both
+classify the same run identically.
+
+```
+$ go test ./...
+?   	x	[no test files]
+$ echo $?
+0                       # <- this is the whole problem
+
+$ didrun --expect "^ok " -- go test ./...
+?   	x	[no test files]
+
+[didrun] DID NOT RUN — there is no evidence this command did anything  (exit 0, 296ms)
+  --  output matches /^ok /: nothing in output matched /^ok /
+
+  It exited 0. That is the failure: a check that stopped checking reports exactly this.
+$ echo $?
+3
+```
+
+## The rule
+
+> A check must answer separately whether it **RAN**, whether it **FAILED**, and whether
+> the failure was the **RIGHT** one. Collapsing any two of those three is how every
+> defect in this family happens.
+
+So this returns four states rather than a number:
+
+| state | means | exit |
+|---|---|---|
+| `did-not-run` | no evidence the command did anything — **regardless of exit 0** | **3** |
+| `ran-and-passed` | evidence found, exit 0 | 0 |
+| `ran-and-failed` | evidence found, non-zero exit, failure looked right | the command's own code |
+| `ran-and-failed-wrongly` | it failed, but not the way you said it would | **4** |
+
+The fourth exists because *"it failed"* is not *"my check caught something"*. A suite
+that dies on a syntax error fails; so does one that caught your mutation. Scoring those
+alike is the difference between a harness that works and one that reports success for a
+file it never parsed.
+
+`did-not-run` gets its own exit code and never borrows the command's — including when
+the command itself failed. *"Your tests failed"* and *"you have no tests"* send you to
+different places.
+
+## Read this first: some runners already answer question one
+
+Let them. Checked, not assumed:
+
+| | answers "did it run"? |
+|---|---|
+| `pytest` | **yes** — exits 5 when it collects nothing |
+| `jest`, `vitest` | **yes** — fail by default when no test matches (`--passWithNoTests` is the decision this tool exists to argue with) |
+| `go test ./...` | **no** — prints `[no test files]` and exits **0**. Measured, not assumed |
+| linters given a glob that matched nothing | generally no |
+| any shell step in any CI file | no notion of the question at all |
+
+If your runner is in the first two rows, you may not need this. It is for the rest.
+
+## Evidence
+
+At least one predicate is required. **With none, `run()` throws and the CLI exits 2** —
+a tool that silently degrades into forwarding the exit code is the thing it is replacing.
+
+| flag | evidence |
+|---|---|
+| `--expect REGEX` | combined output matches |
+| `--expect-stdout` / `--expect-stderr` | one stream matches |
+| `--expect-count REGEX` | the first capture group is a count, `>= --min` (default 1) |
+| `--wrote PATH` | the file was actually written **during this run** |
+| `--took-at-least MS` | a duration floor (weak — prefer a count) |
+
+### `--expect-count` is the one that matters
+
+The thing that makes a green run meaningless is almost always a **zero**, not an
+absence: `0 passed`, `Ran 0 tests`, `0 files checked`. A pattern that only asks whether
+the *line* was printed is satisfied by exactly the run it was meant to catch, because
+the runner cheerfully prints its zero. Both behaviours are pinned by one test:
+
+```js
+// `0 passed in 0.01s`, exit 0
+count(/(\d+) passed/)   -> did-not-run      "reported 0"
+matches(/\d+ passed/)   -> ran-and-passed   // fooled, as advertised
+```
+
+### `--wrote` is not "the file exists"
+
+A `junit.xml` left over from yesterday exists, and a runner that never started leaves it
+exactly where it was. The file must be **created, changed, or rewritten during the run**
+— a byte-for-byte identical artefact is reported as the stale thing it is.
+
+## API
+
+```python
+from didrun import run, report, exit_code_for, evidence
+
+result = run(["pytest", "tests/"],
+             evidence=[evidence.count(r"(\d+) passed"), evidence.wrote("junit.xml")],
+             expect_failure=r"AssertionError",   # when it fails, it must fail THIS way
+             timeout=600)                        # killed and still classified
+
+result.state    # "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks   # every predicate, with what it looked for and what it found
+```
+
+```js
+import { run, report, exitCodeFor, evidence } from "didrun";
+
+const result = await run(["pytest", "tests/"], {
+  evidence: [evidence.count(/(\d+) passed/), evidence.wrote("junit.xml")],
+  expectFailure: /AssertionError/,   // when it fails, it must fail THIS way
+  timeout: 600_000,                  // killed and still classified
+});
+
+result.state   // "did-not-run" | "ran-and-passed" | "ran-and-failed" | "ran-and-failed-wrongly"
+result.checks  // every predicate, with what it looked for and what it found
+process.exitCode = exitCodeFor(result);
+```
+
+Every predicate must hold — `every`, not `some`. The report prints what was looked for
+**and what was found, including when everything passed**, because a check whose output
+is only a verdict is one nobody can audit.
+
+A timeout is classified rather than swallowed: a killed check is a check that did not
+finish, never one that passed.
+
+## Prior art
+
+Swept across **both registries** on mechanism nouns — the first pass queried npm only,
+and npm-only sweeping is what nearly missed `crosshair` elsewhere in this line of work.
+
+On npm, `keywords:no-tests` returns one package (a CRA template) and `keywords:ci-guard`
+returns zero — dead tags, so wrong bucket names rather than open fields. `test-silence`
+inventories *skipped* tests from git history; `jest-fail-on-console` and
+`cypress-fail-fast` change what a runner does *while* it runs.
+
+On PyPI, searched by name across the full 881,198-entry index plus web search, three
+neighbours are real and none of them is this:
+
+- **`pytest-custom-exit-code`** changes *pytest's* exit code when nothing is collected.
+  That is one runner answering question one for itself — the thing this wraps, not a
+  replacement for it.
+- **`evidence-gate`** audits GitHub Actions **evidence bundles** after the fact: whether
+  an audit trail is complete and temporally bounded. A different question, downstream.
+- **`ranit`** reports which functions *in your diff* were executed by nothing, by
+  intersecting a coverage database with the git diff. Function granularity via coverage,
+  not an arbitrary command with a supplied predicate — and the closest thing in spirit
+  to this anywhere.
+
+Nothing found wraps an arbitrary command and asks whether it did anything.
+
+## Limits
+
+- **Evidence is only as good as the pattern you supply.** This does not know what your
+  command should print; it makes you say so, once, where the next person can read it.
+- `--took-at-least` cannot prove work happened. It is marked `[weak]` in the report and
+  it is here because a suite that takes a minute finishing in 9ms is a real signal.
+- It does not parse junit/TAP. `--wrote` plus `--expect-count` covers most of what that
+  would buy.
+- Zero dependencies in either half. Node ≥ 18, Python ≥ 3.9.
+
+## Tests
+
+```sh
+npm test                                        # 22
+python3 -m unittest discover -s python/tests    # 21, four of them the cross-half contract
+```
+
+22 JavaScript tests and 21 Python tests, no dependencies in either half. Six mutations to the source — never reporting `did-not-run`,
+treating it as success, accepting one predicate instead of all, allowing a run with no
+evidence, ignoring the count floor, and accepting a stale artefact — were each caught by
+the test that should catch them.

didrun-0.1.0/python/didrun.egg-info/SOURCES.txt

@@ -0,0 +1,12 @@
+LICENSE
+README.md
+pyproject.toml
+python/didrun/__init__.py
+python/didrun/cli.py
+python/didrun/core.py
+python/didrun/evidence.py
+python/didrun.egg-info/PKG-INFO
+python/didrun.egg-info/SOURCES.txt
+python/didrun.egg-info/dependency_links.txt
+python/didrun.egg-info/entry_points.txt
+python/didrun.egg-info/top_level.txt

didrun-0.1.0/python/didrun.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

didrun-0.1.0/python/didrun.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+didrun = didrun.cli:main

didrun-0.1.0/python/didrun.egg-info/top_level.txt

@@ -0,0 +1 @@
+didrun

didrun-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+