testinghq 1.0.0__tar.gz

testinghq-1.0.0/CHANGELOG.md

@@ -0,0 +1,315 @@
+# Changelog
+
+All notable changes to TestingHQ are recorded here.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project uses [semantic versioning](https://semver.org/spec/v2.0.0.html).
+From 1.0.0 on, a change to a flag, an exit code, a config key or the
+`report --json` shape is a breaking change and needs a major version.
+
+## [Unreleased]
+
+## [1.0.0] - 2026-10-15
+
+The first published release.
+
+### Added
+
+**Tools**
+
+- `blast generate|fire|replay`: seeded inbound-email payloads, from clean to
+  degenerate, fired at an endpoint you control.
+- `barrage fire|replay`: load at a steady or ramped rate, open loop or closed
+  loop, with a knee detector.
+- `verify fire|check`: check what a pipeline actually stored, not what it
+  answered. A 200 is not evidence a message was parsed. `verify check` reads
+  back a run that already happened and refuses a blast artifact, which has no
+  tags.
+- `ledger fire`: account for every payload sent against every record produced:
+  missing, duplicated, misparsed and stray records.
+- `redeliver fire`: duplicate, slow-retry, reply-first and references delivery
+  scenarios, and whether the pipeline handles each.
+- `loop fire`: machine mail and auto-reply loop detection.
+- `steady fire`: label stability under meaning-preserving rewrites of the same
+  message.
+- `sigcheck fire`: webhook signature enforcement for Mailgun, SendGrid and
+  Postmark basic auth. Up to six variants (`valid`, `wrong-secret`,
+  `unsigned`, `stale-timestamp`, `tampered-body`, `replayed-token`); a variant
+  the scheme has nothing to vary is not sent. The secret is an `env:NAME`
+  reference and a literal secret is refused. Dry run by default. The dry run
+  refuses a target whose format the scheme cannot sign, as `--send` does.
+- `compare`: diff two blast runs payload by payload, or two barrage runs on
+  load metrics (see Compare below).
+- `report`: read any tool's artifact and summarise it as text, as JSON in one
+  stable shape, or with `--html OUT` as a single self-contained HTML page
+  (verdict, counts, findings, records, Barrage charts as inline SVG, config,
+  exit code; light and dark; no network).
+- `config validate`: load a config with the real loaders and print what they
+  resolved, including each target's `format`, `public` and public-target
+  resolution. Sends nothing and never prints a header value.
+
+**Compare**
+
+- Two barrage runs are diffed on error rate, hold-stage p99 latency and
+  achieved rps. A candidate regressed when the error or unsent rate rises by
+  more than 1 point, hold p99 rises by more than 20% and 5ms, or achieved rate
+  falls by more than 10% (judged only when mode and rate match).
+- It refuses a dry-run artifact, an artifact from another tool and a pair
+  that mixes a blast run with a barrage run. It says "nothing to compare" when
+  no blast records align.
+- `--fail-on-regression` exits 3 on a regression. Without it the exit is 0.
+  The artifact records `fail_on_regression` and the exit code.
+
+**Report**
+
+- `report --json` always carries `schema_version` and `aborted`, in one fixed
+  key set for every tool.
+- It reads the artifacts of every tool, including steady, sigcheck and
+  compare.
+- `tool` is `verify` for both `verify fire` and `verify check`; the text report
+  shows which.
+- A blast dry-run artifact reads as DRY RUN with exit code 2, and nothing in it
+  counts as sent.
+- A compare artifact reports the exit code the command gave, which depends on
+  `--fail-on-regression`.
+
+**Sending**
+
+- `--allow-public-target` on every send command. Barrage at a public host also
+  needs `public = true` on the target's config table.
+- Target hosts are resolved: a name that resolves only to private, loopback or
+  link-local addresses counts as private.
+- `[targets.<name>].format` selects the wire format, and every send path uses
+  it: `sendgrid`, `sendgrid-raw` (raw MIME in the `email` field), `mailgun`,
+  `postmark` and `mime`. `docs/FORMATS.md` documents each.
+- The `sendgrid` format carries the documented fields it was missing: `dkim`,
+  `SPF`, `sender_ip`, `spam_score`, `spam_report`, `content-ids`, `cc`,
+  `charsets.text` and `attachment-info[*].content-id`.
+- A header hook: `transport.post` and `transport.build_request` accept
+  `extra_headers`, a callable over the finished body bytes.
+
+**Blast**
+
+- `--mix clean=20,messy=40,...`, `--catalog [names]` and
+  `--attachments none|seeded|adversarial` (default `seeded`).
+- `blast fire --send --catalog` sends every case. The synthetic-address guard
+  skips a field that is deliberately empty, and refuses any other value that
+  holds no address.
+- `--fail-on-body <regex>`: a 2xx whose body matches is a flag.
+- Catalog cases `null_byte_in_body`, `rtl_override_in_filename`,
+  `missing_boundary_field`, `duplicate_form_field`, `malformed_envelope_json`
+  and `oversized_body_1mb`; Outlook and Apple Mail quote and forward shapes.
+- Realistic clean mail: consistent names, topics and weekdays, signatures,
+  Gmail and Outlook HTML wrappers, threading headers on replies, `Received`,
+  `Return-Path` and `Authentication-Results` on every message, RFC 2047
+  encoded non-ASCII subjects in `headers`.
+- Encoding sabotage puts real cp1252, latin-1 and shift_jis bytes on the wire
+  with a lying `charsets` entry, for `text` as well as subject and html.
+
+**Barrage**
+
+- Concurrent dispatch through a pool of `--concurrency` worker threads, capped
+  at 64 unless `--allow-high-rate` is passed. A pool too small for the schedule
+  shows as achieved throughput falling short, and the run says how many
+  requests waited for a free worker and for how long.
+- A real closed loop: N workers each send, wait, and take the next token. A
+  shortfall that `--concurrency` explains (workers / median latency is below
+  the planned rate) is not reported as an endpoint knee; the summary carries
+  `concurrency_limited` and says to raise `--concurrency`.
+- One keep-alive connection per worker (`--no-keepalive` opens one per
+  request), with connection setup time reported separately.
+- `--timeout` (default 10s), `--grace` (default 10s) and
+  `--max-queue-seconds` (default 5).
+- Per-request `records` in the artifact, so `compare` can diff two load runs.
+- Report: 429 counted and named "shed", a status-class histogram per bucket,
+  p50, p90 and p99 per bucket, service and offered latency series, and
+  percentiles overall and for the hold stage.
+
+**Readback**
+
+- Retry with backoff and jitter on 429, 5xx, timeouts and connection errors,
+  honouring `Retry-After` (`retry_max`, default 5), and a readback rate cap
+  (`--readback-rate`, default 4/s).
+- A preflight request before anything is sent: refused credentials (401 or
+  403), a response that is not JSON, a field map that finds no identifier or a
+  list url that 404s is a refusal. A 404 on the tag lookup is not, because a
+  missing record answers 404; a url that 404s every lookup is READBACK FAILED.
+- `query` templates, `method = "POST"` with a `body` template, and pagination
+  by `next_path` or `page_param`/`page_size_param` with `max_pages`.
+- `*` wildcards in field paths, `false` to unmap a default field, and senders
+  stored as objects.
+- Settling rule, set by `--quiet-window` (default 5s), `--max-wait` (default
+  60s) and `--poll-interval`. The readback is settled once the found tags,
+  their record counts and the checked field values have held still for
+  `--quiet-window`, counted from the start of the first of two identical
+  complete polls. A tag still missing then is lost. Holding still is also
+  what catches a duplicate that lands after the first sighting.
+- UNSETTLED (exit 3) only when `--max-wait` runs out while the store is still
+  changing, or before every tag could be polled twice. The headline names which
+  and gives the numbers, with the `--max-wait` or `--readback-rate` that would
+  fit. Findings already proven stay listed and counted.
+- Every readback tool's dry run warns when the plan cannot settle within a
+  given `--max-wait`: about 2 x tags / `--readback-rate` plus `--quiet-window`.
+- `--max-wait` not given grows with the plan: the larger of 60s and that settle
+  time plus 25%, so a plan above about 110 tags at the default
+  `--readback-rate` still settles. A value given is used as is. The dry run
+  prints the max-wait a real run would use, and the artifact records it with
+  whether it was the default or given.
+- `--max-wait`, `--poll-interval` and `--quiet-window` are checked by the
+  parser, before anything is sent: the first two must be above zero, the last
+  zero or more.
+- Every `--rate` and `--readback-rate` must be a positive number, refused at
+  the parser.
+- READBACK FAILED (exit 3) when every lookup was a 404. The readback url is
+  almost certainly wrong, so no message is reported as lost: ledger lists the
+  tags under `not_checked`, verify and redeliver say "not checked: every
+  lookup was 404", and `report` shows one readback-failed finding.
+- Readback fields `category` and `priority`, so a tool can read a classifier's
+  label.
+- Recipes for Zendesk, Freshdesk, a generic REST API, an outbound mail sink,
+  Mailpit, Chatwoot and a HandleHQ deployment in `examples/readback/`.
+- Per-run default tag prefix `hq-<yymmddHHMM>-<4 hex>`; `--tag-prefix`
+  overrides it and the artifact records it.
+- Message-IDs are unique per run, so a pipeline that drops a repeated
+  Message-ID does not drop a second run. Verify, ledger, redeliver and steady
+  derive each from the message's tag; sigcheck draws one per run and every
+  variant shares it. One explicit `--tag-prefix` gives the same Message-IDs on
+  every run.
+
+**Checks**
+
+- `--subject-match exact|contains|suffix` (default `contains`) and
+  `--attachment-match exact|loose`, honoured by verify, ledger and redeliver
+  and recorded in the artifact.
+- `--expect-route` honoured by verify, ledger and redeliver.
+- `verify check` re-checks with the match modes the artifact recorded and says
+  where each came from. A flag overrides one; `--no-expect-route` and
+  `--no-body-exact` turn those two off.
+- `[loop].ticket_policy` and `[steady].intents_path` config keys.
+- A key or top-level table the schema does not declare is refused (exit 1)
+  before anything is sent, naming the file, the table, the key and the closest
+  known key. `config validate` reports the same error and checks the key names
+  of `[loop]`, `[steady]` and `[sigcheck]`. The names inside `[readback.headers]`
+  and `[readback.fields]` stay open, and a custom adapter's table may carry its
+  own keys.
+
+**Artifacts and project**
+
+- Every artifact carries `schema_version: 1` and a top-level `tool`. Readers
+  treat a missing `schema_version` as 0 and refuse a newer one.
+- `docs/CONFIG.md`, generated from `testinghq/core/config_schema.py`, with a
+  test that fails when the two drift.
+- `docs/README.md`, an index of the docs, and `docs/DOGFOOD.md`, how to run
+  the suite against a real deployment.
+- `examples/pipeline_demo.py` runs verify, ledger, redeliver, loop and steady
+  over real sockets against a pipeline that imports nothing from TestingHQ and
+  owns its machine-mail check, tag regex, Message-ID dedup, threading and
+  classifier. Its defects are on the write side and each run's exit code is
+  asserted.
+- `CONTRIBUTING.md`, `ruff.toml` (pyflakes rules), a lint job in CI, and a
+  `report-html-tests` job that runs the render.js tests against the installed
+  wheel.
+
+### Changed
+
+- **One exit-code convention across every tool**: 0 ran and yes, 1 refused, 2
+  dry run, 3 ran and found something. A usage error exits 1 rather than
+  argparse's 2. `blast fire --send` exits 3 when anything is flagged.
+- **`compare` shares the convention**: a usage error is 1 and, with
+  `--fail-on-regression`, a regression is 3.
+- **Ctrl-C** in a long-running command writes what it has (when `--out` was
+  given) with `aborted: true` and exits 3.
+- **Blast verdict**: a 5xx or transport failure in any category is a flag. A
+  4xx on malformed input is recorded and not flagged. Transport failures are
+  reported by reason (refused, DNS, timeout, TLS, other).
+- **`headers` stay in sync** with `subject`, `to` and `from` after every
+  mutation and catalog case.
+- **The default Blast corpus contains attachments** (`--attachments seeded`).
+- **Barrage** ends at `duration + grace` whatever the target does: in-flight
+  requests past the deadline are `abandoned`, queued ones `not_dispatched`,
+  and requests that waited past `--max-queue-seconds` are `dropped_backlog`.
+  Achieved throughput is measured over the real dispatch span. The knee is the
+  first of two buckets in a row that shed (error rate of 5% or more), send over
+  20% fewer requests than planned, or run at over twice the hold-stage median
+  latency. In a closed loop, fewer requests than planned count only when the
+  workers could have sent more.
+- **Routing is not checked unless `--expect-route` is given**; it reports
+  SKIPPED with "no expected route".
+- **Subjects match by containment by default**, so a stored
+  "[Ticket #123] Re: ..." passes. An empty expected body is SKIPPED. HTML
+  bodies are compared after tag stripping. "Last, First <addr>" senders parse.
+- **ledger** reports UNVERIFIED STRAYS when the adapter cannot enumerate,
+  instead of refusing after sending. "Extra" means records carrying this run's
+  tag family that this run did not send; other runs' records are listed
+  separately and untagged records are not strays. The `extra` list is capped
+  at 200 with a count.
+- **`--max-wait` is one budget for the whole readback**: a request or retry
+  that would run past it ends the poll.
+- **A spec adapter** (`--readback module:attr`) may be a class (instantiated),
+  an object with `fetch(probe)`, a one-argument fetch function, or a factory
+  taking nothing or `config`. An import error is a refusal before anything is
+  sent. The class or factory gets the table with the adapter's own keys as
+  written (an `env:NAME` value in them arrives unresolved) plus the built-in
+  keys; header names are included and header values are not.
+- **redeliver** slow-retry sends every original, waits once, then sends every
+  retry; the references scenario checks the full `References` ancestry. Every
+  Message-ID, In-Reply-To and References id it writes is in angle brackets.
+- **loop** enumerates the outbound sink, so an auto-reply it cannot attribute
+  is a finding ("outbound, unattributed") rather than "no reply". The loop
+  bait is an auto-reply from the customer's address to the pipeline's real
+  outbound Message-ID, falling back to a made-up parent when no acknowledgement
+  arrives within `--ack-wait`. `--reply-address` takes precedence over the
+  config file.
+- **steady** headline is per-family majority dissent, with the pairwise flip
+  rate secondary (it gates the exit code only with `--max-flip-rate`). Cross-language pairs are
+  reported separately and every report has a sample-size line. It refuses when
+  every family would read the same constant label.
+- The CLI is a package, one module per command. The command surface is
+  unchanged.
+- `project.license` is the SPDX string `"MIT"`.
+- The sdist ships the package, the docs and the examples, not `tests/`.
+- The suite's meta tests (README wording, CI YAML, docstrings, file absence)
+  are cut to the ones that guard behaviour.
+  `docs/decisions/0003-what-the-suite-protects.md` records what each removed
+  check had encoded.
+
+### Fixed
+
+- **`--allow-public-readback` wins over `allow_public_hosts = false`** in the
+  `[readback]` table of the config file.
+- `steady` runs from an installed wheel: its intent fixture is package data.
+- A negative `--count` is a refusal, not a traceback.
+- The mailbox readback adapter tolerates a partially written trailing line and
+  parses the file once per poll instead of once per tag per poll.
+- A readback list under the wrong `items_key` says so by name.
+- A relative mailbox path resolves against the config file's directory, not
+  the working directory.
+- Undefined names (`Any` in `core/config.py`, `FormPart` in `core/formats.py`),
+  the `record_id_for` export that did not exist, and unused imports.
+
+### Removed
+
+- **The web UI** (`web/`), which exposed only Blast. Its rendering lives
+  on as `report --html`.
+- **The marketing page** (`site/`).
+- `--expect-route` on `loop` and `steady`, which never honoured it.
+- `testinghq/pipeline/corpus.py`, a duplicate of code in `verify.py`.
+
+### Security
+
+- Public targets require explicit consent: `--allow-public-target`, plus
+  `public = true` for Barrage. Synthetic reserved-domain addresses, the
+  dry-run default and the configured-target allow-list are unchanged.
+- Readback URLs on a public host are refused unless `--allow-public-readback`
+  is passed.
+- `[readback.headers]` values must be `env:NAME` references, checked before
+  anything is sent, and an artifact records header names, never values.
+- Any artifact written at a public target records `allow_public_target: true`
+  under `config`.
+- The `report --html` page carries a Content-Security-Policy that allows only
+  its own inline script and style, and escapes every value from the artifact.
+- The CI, e2e and security workflows run with a read-only token and have job
+  timeouts, and cancel superseded pull request runs.
+
+[Unreleased]: https://github.com/saifullah4khan/TestingHQ/compare/v1.0.0...HEAD
+[1.0.0]: https://github.com/saifullah4khan/TestingHQ/releases/tag/v1.0.0

testinghq-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,36 @@
+# Contributing
+
+## Set up
+
+    git clone https://github.com/saifullah4khan/TestingHQ.git
+    cd TestingHQ
+    python -m venv .venv && . .venv/bin/activate
+    pip install -e ".[dev]"
+
+Python 3.10 is the floor. Runtime code uses the standard library only (plus
+`tomli` below 3.11); a new runtime dependency needs a strong case.
+
+## Check
+
+    pytest -q -m "not allow_network"   # hermetic suite, what CI's tests job runs
+    pytest -q -m allow_network         # real loopback sockets, what e2e runs
+    ruff check .                       # pyflakes rules, see ruff.toml
+    node --test tests/report_html/*.test.cjs
+
+The suite blocks outbound sockets by default. A test that needs loopback is
+marked `@pytest.mark.allow_network` and lives in `tests/e2e/` or
+`tests/integration/`; collection fails anywhere else.
+
+Test behaviour through public functions or the CLI. Do not add tests that
+assert README wording, CI YAML, docstrings or that a file is absent; see
+`docs/decisions/0003-what-the-suite-protects.md`.
+
+## Commits and pull requests
+
+- Lowercase `type: subject` (feat, fix, refactor, test, docs, chore, ci),
+  subject under 72 characters, and a body that says why.
+- No em-dashes anywhere, in code, comments, docs or commit messages.
+- Every behaviour change ships with a test that fails before and passes after.
+- Update `CHANGELOG.md` under `[Unreleased]`. A change to a flag, exit code,
+  config key or the `report --json` shape is breaking.
+- Both required checks, `tests` and `end-to-end`, must be green.

testinghq-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Saifullah Khan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

testinghq-1.0.0/MANIFEST.in

@@ -0,0 +1,18 @@
+# What goes in the sdist. The wheel carries only the importable package; the
+# sdist adds what a reader of the source needs: the changelog, the docs and
+# the example configs.
+#
+# The test suite is not shipped. It needs the repository (CI workflows, git,
+# the e2e harness), so from an extracted sdist it could not pass, and a suite
+# that fails out of the box is worse than none. Clone the repository to run it.
+
+include CHANGELOG.md
+include CONTRIBUTING.md
+include LICENSE
+include README.md
+
+recursive-include docs *.md
+recursive-include examples *.toml *.py *.md
+
+prune tests
+prune .github

testinghq-1.0.0/PKG-INFO

@@ -0,0 +1,266 @@
+Metadata-Version: 2.4
+Name: testinghq
+Version: 1.0.0
+Summary: Fuzz, load-test and verify your inbound email webhook. Catches lost, duplicated, mis-threaded and looped mail that a 200 OK hides.
+Author-email: Saifullah Khan <saifullah4khan@gmail.com>
+License-Expression: MIT
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: tomli>=1.1.0; python_version < "3.11"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: setuptools>=77; extra == "dev"
+Dynamic: license-file
+
+# TestingHQ
+
+Fuzz, load-test and verify your inbound email webhook. Catches lost, duplicated,
+mis-threaded and looped mail that a 200 OK hides.
+
+TestingHQ posts provider-shaped inbound email (SendGrid, Mailgun, Postmark or raw
+MIME) at a webhook you run, then asks your system what it actually made of each
+message.
+
+## What it catches
+
+- An email your endpoint answered 200 for and never turned into a ticket.
+- Two tickets for one email: a provider retry, a redelivery, a worker that ran twice.
+- A ticket nobody sent, carrying a test tag.
+- A mangled sender, a changed subject, a truncated body, a dropped attachment.
+- A reply filed as a new conversation, including one that arrives before its original.
+- An auto-reply to an out-of-office, a bounce or a mailing list: how a mail loop starts.
+- A triage label that moves when a signature, a forward or a translation is added.
+- A webhook that accepts unsigned, wrongly signed, stale or replayed requests.
+- A parser that answers 500 to messy, multilingual or malformed mail, or 200 with an error.
+- The request rate at which the endpoint starts slowing down or shedding load.
+
+## Which tool
+
+| Question | Tool | Answer | Exit 3 means |
+| --- | --- | --- | --- |
+| Does my parser survive messy mail? | `blast` | a seeded corpus fired at the webhook, every bad answer flagged | a payload was flagged |
+| How much load before it slows or sheds? | `barrage` | throughput, latency percentiles and the knee | the run was interrupted |
+| Did it store what was sent? | `verify` | six checks per message, read back from your system | a check failed, or the readback never settled |
+| Did we lose or duplicate anything? | `ledger` | missing, duplicated, misparsed and extra, counted | anything but BALANCED |
+| Are retries and out-of-order replies safe? | `redeliver` | four provider misbehaviours, one ticket and one thread each | a scenario failed |
+| Will it answer an auto-reply? | `loop` | twelve kinds of machine mail and a loop bait | a ticket or a reply for machine mail |
+| Is the triage label stable? | `steady` | label dissent across meaning-preserving rewrites | dissent above the gate |
+| Does it check the webhook signature? | `sigcheck` | six signed and mis-signed variants | it accepted one it should refuse |
+| Did my change make it worse? | `compare` | regressions, fixes and changes between two runs | a regression, with `--fail-on-regression` |
+
+## Install
+
+```
+pip install testinghq
+```
+
+Python 3.10 or newer. Standard library only at run time, plus `tomli` on 3.10.
+
+## Quick start
+
+```
+cat > target.toml <<'EOF'
+[targets.local]
+url = "http://127.0.0.1:8000/inbound"   # your webhook
+[targets.mailgun]                        # sigcheck --scheme mailgun needs format = "mailgun"
+url = "http://127.0.0.1:8000/inbound"
+format = "mailgun"
+
+[readback]
+kind = "http"
+url = "http://127.0.0.1:8000/tickets"   # answers GET ?tag=<tag> with the records it made
+EOF
+testinghq config validate target.toml
+testinghq ledger fire --target local --count 50
+testinghq ledger fire --target local --count 50 --send --out ledger.json
+testinghq report ledger.json --html report.html
+```
+
+Every command is a dry run until you pass `--send`. Records are read with the
+field names in [docs/CONFIG.md](docs/CONFIG.md) unless `[readback.fields]` maps them; an unknown config key is refused (exit 1) with the closest known one.
+A target's `format` picks the wire format: `sendgrid` (default), `sendgrid-raw`, `mailgun`, `postmark` or `mime`; see [docs/FORMATS.md](docs/FORMATS.md).
+
+## What a finding looks like
+
+`ledger` against a pipeline whose worker writes every fifth ticket twice. Every
+POST in the run was answered 200. From `python examples/pipeline_demo.py`:
+
+```
+ledger fire: explicit --send flag set
+ledger: UNACCOUNTED  (12 message(s) sent, seed=0, tag prefix 'hq-2610142218-d0f4')
+
+  sent:            12
+  produced:        14
+  exactly once:    10
+  missing:         0
+  duplicated:      2
+  misparsed:       0
+  readback:        2.59s over 4 poll(s), settled
+  extra:           0
+
+  missing: none
+
+  duplicated (2):
+    hq-2610142218-d0f4-0-0004: 2 tickets ['T00005', 'T00006']
+    hq-2610142218-d0f4-0-0009: 2 tickets ['T00011', 'T00012']
+
+  misparsed: none
+```
+
+## Exit codes
+
+| Code | Meaning | A script should |
+| --- | --- | --- |
+| 0 | ran, the answer was yes | carry on |
+| 1 | refused: guardrail, bad config or bad usage; nothing was sent | fix the command |
+| 2 | dry run: no `--send`, nothing was sent | carry on |
+| 3 | ran, the answer was no | read the report |
+
+Every tool uses the same four codes. `report` exits 0 whenever it could read the artifact.
+
+## The tools
+
+**Blast.** Fires a seeded corpus (clean, messy, multilingual, malformed, degenerate) with one to three clean attachments on about a third of payloads (`--attachments none|seeded|adversarial`).
+`--mix clean=20,messy=40,...` sets the category percentages; `--catalog` fires the named edge cases instead, all of them or a comma-separated list.
+A 5xx or no response is flagged in every category, and a missing response says why (refused, DNS, timeout, TLS, other). `--fail-on-body REGEX` flags a 2xx whose body reports failure.
+The same seed and options rebuild a byte-identical corpus, and `blast replay` re-fires it.
+
+```
+testinghq blast generate --count 100 --seed 1 --out corpus
+testinghq blast fire --target local --send --catalog --out blast.json
+testinghq blast replay blast.json --send
+```
+
+**Barrage.** Clean payloads at a controlled rate after a warmup ramp. `--mode open` (default) keeps arrivals on schedule whatever the target does; `--mode closed` runs `--concurrency` workers that each send the next request when the last completes.
+It reports achieved against target throughput, p50/p90/p99 latency, errors over time with 429 counted as shed load, and the knee where the endpoint gives way.
+The run ends `--grace` seconds (default 10) after `--duration`, and a request that waited longer than `--max-queue-seconds` (default 5) for a worker is dropped and counted. Each worker keeps one connection alive unless `--no-keepalive`.
+Ctrl-C writes a partial artifact with `aborted: true` and exits 3. Rate, duration and concurrency are capped at 50/s, 300s and 64 unless `--allow-high-rate`.
+
+```
+testinghq barrage fire --target local --rate 20 --duration 60 --send --out load.json
+testinghq barrage fire --target local --mode closed --concurrency 8 --send
+```
+
+**Verify.** Fires clean tagged messages, reads back what the system stored, and checks that a record exists and that its sender, subject, body, attachments and route match.
+Routing is checked only when you pass `--expect-route`; without it the check is skipped. `--subject-match contains` (the default) accepts helpdesk prefixes like `[Ticket #123]`, and `--attachment-match loose` tolerates sanitised filenames.
+A check that could not run says why (no record found, adapter cannot see field, nothing to check) and never counts as a pass.
+`verify check` re-reads an earlier run without sending anything.
+
+```
+testinghq verify fire --target local --send --expect-route support --out verify.json
+testinghq verify check verify.json
+```
+
+**Ledger.** Sends N uniquely tagged messages and reconciles them: missing, duplicated, misparsed, and extra (a record with this run's tag family that this run never sent).
+Each run draws its own tag prefix, `hq-<yymmddHHMM>-<4 hex>`, so runs never read each other's records; `--tag-prefix` fixes one.
+BALANCED exits 0. A readback that cannot list every record reports UNVERIFIED STRAYS, and one still changing at `--max-wait` reports UNSETTLED; both exit 3.
+Run it while Barrage holds the system under load to see what load loses.
+
+```
+testinghq ledger fire --target local --send --count 50
+```
+
+**Redeliver.** Four scenarios: the same Message-ID delivered twice; every message re-sent after one `--retry-after` wait (default 120s); two replies delivered before their original; a three-message thread whose last reply must carry the full References chain.
+Duplicate tickets and threading are checked separately. `--scenario` picks one, and can be repeated.
+
+```
+testinghq redeliver fire --target local --send --scenario slow-retry --retry-after 30
+```
+
+**Loop.** Sends machine mail a pipeline must not act on (Auto-Submitted, Precedence, out-of-office, a delivery-status bounce, noreply senders, list mail) and a loop bait: an ordinary message, then an autoresponder answering the pipeline's own acknowledgement of it.
+A ticket for machine mail is a finding unless `--ticket-policy allowed`; a reply to machine mail, or to the bait, is always a finding.
+Replies are read from `[loop.outbound]` in the config. Without it the reply check is NOT CHECKED, never passed, and outbound mail it cannot tie to a message is itself a finding.
+
+```
+testinghq loop fire --target local --send --ticket-policy allowed
+```
+
+**Steady.** Sends reviewed intents in English, Urdu and Roman Urdu, each with meaning-preserving edits: a signature, a forward wrapper, quoted history, reflowed whitespace, a reworded greeting.
+It reads back the label (`--label-field category|route|priority`) and reports the share of variants that dissent from their family's majority, by transform and by language. Above `--max-dissent-rate` (default 0.1) it exits 3.
+`--repeats N` measures answers to identical input separately. It measures stability, not correctness.
+
+```
+testinghq steady fire --target local --send --label-field category --repeats 3
+```
+
+**Sigcheck.** Sends one request per variant (valid, wrong-secret, unsigned, stale-timestamp, tampered-body, replayed-token) and reports which the endpoint accepted.
+Schemes: `mailgun` (HMAC of timestamp and token, as form fields); `sendgrid` (ECDSA over timestamp and raw body, on the curve SendGrid documents: give the private key matching the endpoint's public key, and the dry run prints that public key); `postmark-basic` (Postmark does not sign; its protection is Basic credentials).
+The secret is read only from the environment variable `env:NAME` names, and the artifact records the reference, never the value. For Mailgun, accepting a stale timestamp but refusing a replayed token follows Mailgun's own guidance and is informational.
+Exit 3: the endpoint accepted what it should refuse, refused a valid request, or answered 5xx.
+
+```
+testinghq sigcheck fire --target mailgun --scheme mailgun --secret env:MAILGUN_SIGNING_KEY --send
+```
+
+**Compare.** Two Blast runs are lined up payload by payload, with regressions, fixes and changes reported apart. Two Barrage runs are compared on load: error and unsent rate, hold-stage p99, and achieved rate when mode and rate match.
+A dry run on either side, or two runs with nothing in common, is refused.
+
+```
+testinghq compare --baseline before.json --candidate after.json --fail-on-regression
+```
+
+**Reports.** `testinghq report run.json` summarises any tool's artifact, and `testinghq report run.json --json` prints one shape for every tool. `testinghq report run.json --html report.html` writes a self-contained HTML report. `testinghq config validate` shows what the loaders resolved and never prints a header value.
+
+## Readback
+
+The pipeline tools ask your system what it made of each message, found by a tag
+in an `X-TestingHQ-Tag` header and `[testinghq:<tag>]` in the body. Three kinds:
+
+```toml
+[readback]                         # a JSON API: GET url?tag=<tag>
+kind = "http"
+url = "http://127.0.0.1:8000/tickets"
+query = "type:ticket tags:{tag}"   # optional template, for search APIs
+
+[readback.headers]
+Authorization = "env:HQ_READBACK_TOKEN"   # env: references only; a literal is refused
+```
+
+```toml
+[readback]                         # a JSON Lines file your pipeline writes
+kind = "mailbox"
+path = "./mail-sink.jsonl"
+```
+
+Anything else is an import path, `--readback mypkg.adapters:build`: a class, an
+object with `fetch`, or a function taking one probe and returning records.
+
+The readback settles once what it sees (which payloads are found, how many records each, the field
+values the checks read) has held still for `--quiet-window` seconds (default 5); a payload still not
+found then is lost. A store still changing when `--max-wait` (default 60, raised to fit a large run) runs out, or not polled twice by then, is UNSETTLED
+(exit 3), not a loss, and anything already proven, such as a duplicate, is still listed. A run whose
+every lookup was a 404 is READBACK FAILED (exit 3): the url is almost certainly wrong, so nothing is
+reported as lost. For a slow or async pipeline, raise `--quiet-window` above its worst processing
+latency. Lookups retry 429, 5xx and timeouts with backoff, honour `Retry-After`, and are paced to
+`--readback-rate` (default 4 per second). A public readback host needs `--allow-public-readback`.
+Recipes for Zendesk, Freshdesk, Mailpit, Chatwoot, a generic REST API and a mail sink are in
+[examples/readback/](examples/readback/).
+
+## Responsible use
+
+TestingHQ is for endpoints you own or are authorised to test. It never sends
+email: it POSTs webhook payloads, and every address in them is on a reserved
+example domain.
+
+- Dry run is the default; every run that sends needs `--send`. Only a target declared in `[targets.<name>]` can be fired at.
+- A public host needs `--allow-public-target`; Barrage at a public host also needs `public = true` on the target.
+- Every send is rate limited, and Barrage has hard ceilings.
+
+See [docs/SECURITY.md](docs/SECURITY.md) for the full model and how to report a vulnerability.
+
+## Documentation
+
+- [docs/README.md](docs/README.md): index of everything below
+- [docs/CONFIG.md](docs/CONFIG.md): every config key, generated from the code
+- [docs/FORMATS.md](docs/FORMATS.md): the five wire formats, field by field
+- [docs/SECURITY.md](docs/SECURITY.md): guardrails, public targets, threat model
+- [docs/DOGFOOD.md](docs/DOGFOOD.md): running against a real staging deployment
+- [docs/ROADMAP.md](docs/ROADMAP.md): goals, timeline and what comes after 1.0
+- [examples/README.md](examples/README.md): the demo scripts and sample configs
+- [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md)
+
+## License
+
+MIT. See [LICENSE](LICENSE).