undercurrent 0.0.0a0__tar.gz → 0.2.0__tar.gz

undercurrent-0.2.0/.gitignore

@@ -0,0 +1,58 @@
+# Root ignore rules for the whole repository. Keep .dockerignore in sync:
+# anything that must never be committed should also stay out of image
+# build contexts.
+
+# --- Python bytecode and caches ---------------------------------------------
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# --- Build output ------------------------------------------------------------
+build/
+dist/
+*.egg-info/
+# Generated by hatch-vcs at build/install time
+src/undercurrent/_version.py
+# mkdocs build output
+site/
+
+# --- Environments ------------------------------------------------------------
+.venv/
+venv/
+.env
+.env.*
+!.env.example
+
+# --- Coverage ----------------------------------------------------------------
+.coverage*
+htmlcov/
+coverage.xml
+
+# --- Model weights and checkpoints -------------------------------------------
+# Trained checkpoints are private, per-user artefacts and can be tens of MB.
+# Never commit them; publish weights through the Hugging Face Hub instead.
+*.pt
+*.pth
+*.ckpt
+*.safetensors
+*.bin
+*.gguf
+
+# --- Notebooks ---------------------------------------------------------------
+.ipynb_checkpoints/
+
+# --- Local tooling -----------------------------------------------------------
+.claude/worktrees/
+.claude/settings.local.json
+.idea/
+.vscode/
+.DS_Store
+
+# --- Run artefacts -----------------------------------------------------------
+# Observation / metrics sink output
+*.ndjson
+benchmarks/results/
+# examples/train_probe output (probe weights, metrics), wherever it's run from
+**/outputs/train_probe/

undercurrent-0.2.0/CHANGELOG.md

@@ -0,0 +1,254 @@
+# Changelog
+
+All notable changes to Undercurrent are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+While Undercurrent is on 0.x:
+
+- A **minor** release (0.1 → 0.2) may contain breaking changes. Each one is
+  listed under **Changed** or **Removed** with what to do about it.
+- A **patch** release (0.1.0 → 0.1.1) contains only backwards-compatible fixes.
+- A deprecated API keeps working and emits a `DeprecationWarning` for at least
+  one minor release before it is removed.
+
+Versions come from git tags (`vX.Y.Z`). See [RELEASING.md](RELEASING.md) for
+how a release is cut.
+
+## [Unreleased]
+
+## [0.2.0] - 2026-11-01
+
+Performance and correctness work on the vLLM adapter, and experimental
+graph-mode `residual_stream` capture that keeps CUDA graphs and
+`torch.compile`. There are no breaking changes: code written for 0.1.0 runs
+unchanged.
+
+### Added
+
+- vLLM adapter (experimental): graph-mode `residual_stream` capture.
+  `load_model(graph_capture_layers=[...])` (also accepted by
+  `ProbedModel(..., backend="vllm")`) captures the residual stream at the given
+  decoder layers without `enforce_eager=True`, so the engine keeps CUDA graphs
+  and `torch.compile`. It reuses vLLM's EAGLE-3 auxiliary hidden-state outputs,
+  enabled by the `vllm.general_plugins` entry point before compilation. Other
+  tensor types, other layers, models without that interface, and speculative
+  decoding that uses aux outputs are rejected with an error. The benchmark
+  harness gains `--graph-capture`.
+
+### Changed
+
+- vLLM adapter: activation capture on CUDA no longer blocks the host once per
+  captured row. Each forward hook gathers its matched rows on the GPU and
+  starts one non-blocking copy into pinned host memory; records are built after
+  one wait per step (at the end of the step in-process, or when the driver
+  polls on the cross-process path). Probes see the same records, in the same
+  order, with inline aborts taking effect at the same step as before. See
+  "Copying captures to the host" in the vLLM adapter internals docs.
+- vLLM adapter: on the cross-process polling path (vLLM 0.28's default), the
+  worker sends each poll's activations as one packed batch (float32 `bytes`
+  plus per-field lists) instead of one dict per record with the tensor as a
+  nested Python list, and the driver routes each drain in one executor call
+  instead of one per record. In a CPU microbenchmark at a 3,584-wide hidden
+  size, transport fell from about 350 µs to under 15 µs per record. Probes see
+  the same records in the same order; aborts are sent after the drain is
+  routed.
+- vLLM adapter: forward hooks skip (layer, tensor type) pairs no request
+  captures with one set lookup, and position matching runs once per step
+  instead of once per hooked layer; the captured tensor is only extracted when
+  something matched. Hook-path CPU time per step fell by about a third in a
+  microbenchmark (16 requests x 8 probes, 28 layers).
+
+### Fixed
+
+- vLLM adapter: `ActivationRecord.timestamp` is now the capture time, as
+  documented, on every path. On the cross-process polling path the record was
+  rebuilt in the driver with a fresh timestamp, so a probe (or a latency
+  measurement) saw the time of routing, not of capture, and couldn't tell how
+  stale an activation was. The worker now stamps each capture when its hook
+  runs and the packed batch carries it to the driver.
+- vLLM adapter: with `OverflowPolicy.BLOCK` on the cross-process polling path,
+  async probes ran one at a time instead of in parallel. Each drained backlog
+  was routed one request at a time, so `route()` waited on one request's full
+  queue while every other request's worker sat idle. Drained records are now
+  interleaved round-robin across requests, keeping each request's forward-pass
+  order. On an A100 (Qwen2.5-7B, 16 requests, 76 ms probe) the old order made
+  `block` about 13x slower than the probe's own pace. The docs now also say
+  that on this path `block` delays results rather than slowing the engine.
+- vLLM adapter: activations could be silently dropped under concurrency on the
+  polling path (engine core in a subprocess, the vLLM 0.28 default). One
+  request's drain of the shared worker buffer could still be routing another
+  request's records when that request received its final output and ended, so
+  those records were discarded as late. A request that finishes in the step that
+  captured its only extraction point (e.g. `prompt[-1]` with one new token) lost
+  its capture entirely; longer generations could lose their last activations.
+  Drains are now serialised, so every record captured up to a request's final
+  step is routed before the request ends.
+
+## [0.1.0] - 2026-10-02
+
+The first public release of **Undercurrent**, Wrynx's activation-probing
+platform for LLM inference. Earlier versions were internal development builds
+and were never published.
+
+Planned work that is not part of this release (an OpenAI-compatible
+`undercurrent serve` server, Hugging Face Hub integration, a published
+container image, and more) is tracked in the
+[roadmap](https://github.com/wrynx/undercurrent/blob/main/ROADMAP.md).
+
+### Added
+
+- **One installable package.** `pip install undercurrent` installs the
+  `undercurrent` package for Python 3.10+, with the Hugging Face
+  `transformers` backend in the base install. vLLM is never installed by
+  default. Tested on Python 3.10–3.13 with torch 2.1+ and transformers
+  4.40–5.x; see [Compatibility](https://wrynx.github.io/undercurrent/compatibility/).
+- **`ProbedModel`**, the high-level API (`undercurrent.ProbedModel`):
+  `ProbedModel.from_pretrained(model, spec=..., probes=...)`, then
+  `generate()`, which returns a `GenerationOutput` with the text and each
+  extraction point's `ProbeResult`. Backends `"hf"` (default) and `"vllm"`.
+  Pass `on_result=callback` for a simple result hook alongside the sink API.
+- **Extraction-point specs** (`undercurrent.spec`). Declare what to capture
+  in YAML (`load_yaml_file`, `parse_yaml`, `parse_dict`) or construct
+  `ExtractionPoint`s in Python. Specs are validated by pydantic models and
+  round-trip back to YAML (`to_yaml`).
+  - Tensor types `residual_stream`, `attn_out`, `mlp_out` and `final_norm`,
+    selected per layer.
+  - A position-selector grammar: absolute indices (`5`), prompt and generated
+    indices (`prompt[-1]`, `generated[0]`), every generated token
+    (`generated[*]`), generated slices (`generated[5:]`, `generated[2:8]`),
+    and offsets (`prompt[-1]+1`). Malformed selectors raise
+    `PositionSyntaxError` with the accepted forms.
+  - `until` conditions for continuous selectors (`generation_end`,
+    `fixed_count`, `stop_token`).
+  - Example specs in `examples/specs/`.
+- **Probe interface** (`undercurrent.core`). Probes implement the lifecycle
+  `spawn` → `on_start` → `on_activation`* → `on_end` and return
+  `ProbeSignal`s and a final `ProbeResult`. There are two probe kinds,
+  `single_shot` and `trajectory`. A fresh instance is spawned for each
+  request and extraction point, and probe classes that declare mutable
+  class-level attributes are rejected when the class is defined, so instances
+  can't share state across requests. Reference probes are in
+  `undercurrent.core.examples`: `MLPClassifierProbe` (single-shot) and
+  `TrajectoryScoreProbe` (trajectory).
+- **Function probes and a probe registry.** Decorate a plain function
+  `(record) -> float | bool | ProbeSignal | None` with `@probe("name", ...)`
+  for stateless single-shot probes, and register probe classes by name with
+  `@register_probe("name")`. Third-party packages can expose probes through
+  the `undercurrent.probes` entry-point group.
+- **Router** (`undercurrent.router`). Dispatches activation records to
+  per-request probe instances. Each extraction point picks an execution mode:
+  - `inline`: the probe runs on the generation path and its signal is
+    returned to the adapter, so it can abort generation.
+  - `async`: the probe runs out of band and doesn't block the response. Each
+    binding has a bounded queue (`DEFAULT_QUEUE_DEPTH`), served by a shared
+    worker pool, with a configurable `OverflowPolicy` (`drop_oldest`,
+    `drop_newest` or `block`).
+- **Interventions.** An inline probe can return `ProbeAction.ABORT` to stop
+  generation. With the `block_until_signal` intervention policy, the router
+  waits up to `timeout_ms` for the probe, then applies `on_timeout`
+  (`continue` or `abort`). A circuit breaker downgrades an extraction point
+  to observe-only after `circuit_breaker_threshold` consecutive timeouts or
+  probe errors, so a failing probe can't keep stalling generation.
+- **Metrics.** Per-binding queue depth, drops, activation latency and probe
+  errors go to a pluggable `MetricsSink`. `InMemoryMetricsRegistry` provides
+  snapshots.
+- **Observation sinks** (`undercurrent.sinks`) for async probe output:
+  - `FileLogSink` writes NDJSON.
+  - `WebhookLogSink` POSTs JSON from a background thread, so a slow endpoint
+    never adds latency to the caller. Failed deliveries are retried with
+    exponential backoff, records that exhaust their retries are
+    dead-lettered to a file, and the send queue is bounded.
+  - A `LogSink` base class for writing your own sink.
+- **Sink redaction.** `redact_keys()`, `drop_keys()` and `chain()` control
+  what a sink writes. `WebhookLogSink` redacts prompt text by default.
+- **Router context managers.** `Router` and `router.request(...)` can be used
+  in `with` blocks so requests end and workers shut down cleanly.
+- **Hugging Face adapter** (`undercurrent.adapters.hf`, in the base install).
+  `HFEngineAdapter` captures activations from `transformers` models with
+  forward hooks during `generate()`, and aborts generation through a
+  `StoppingCriteria` (`ProbingStoppingCriteria`).
+- **vLLM adapter** (`undercurrent.adapters.vllm`). Install Undercurrent into
+  your existing vLLM environment or image (the recommended production path),
+  or use the `undercurrent[vllm]` extra for a vLLM from the tested range.
+  - Capture is aware of continuous batching: a worker extension installs the
+    hooks, and `SeqIdMapper` translates batch rows to
+    `(request_id, token_pos)`. Abort signals are wired back to the engine.
+  - A `vllm.general_plugins` entry point registers the worker extension.
+  - Constructing the adapter checks the installed vLLM version against the
+    supported range and raises `VLLMAdapterLimitationError` with the
+    installed version and the supported range. Set
+    `UNDERCURRENT_ALLOW_UNSUPPORTED_VLLM=1` to downgrade the error to a
+    warning. See
+    [docs/compatibility.md](https://github.com/wrynx/undercurrent/blob/main/docs/compatibility.md).
+  - Tensor and pipeline parallelism: results from every worker are merged,
+    and events that tensor-parallel ranks duplicate are deduplicated.
+    Multi-worker topologies are refused unless you pass
+    `allow_unsupported_executor=True`, because cross-rank ordering under
+    pipeline parallelism and Ray/distributed executors haven't been
+    validated. See the
+    [tensor and pipeline parallelism notes](https://wrynx.github.io/undercurrent/internals/vllm-parallelism/).
+- **Command-line tool** `undercurrent`: `inspect-model` lists a model's
+  hookable layers and supported tensor types, `validate` checks spec files,
+  and `schema` prints the spec JSON Schema (also in
+  `schema/probe-spec.schema.json` and `undercurrent.spec.json_schema()`).
+- **Curated top-level API.** The common names import from `undercurrent`
+  directly; see [API stability](https://wrynx.github.io/undercurrent/api-stability/)
+  for what is public. `load_spec()`
+  loads a spec from a path, YAML string or dict. The package ships `py.typed`.
+- **Error hierarchy.** Errors Undercurrent raises on purpose derive from
+  `ProbingError` (and the matching built-in type), with messages that say
+  how to fix the problem.
+- **Clear errors for missing backends.** Using an adapter whose backend isn't
+  installed raises `MissingDependencyError`, which says how to install it.
+- **Examples** (in the repository, not installed with the package, and covered
+  by the test suite):
+  - [`examples/content_safety/`](https://github.com/wrynx/undercurrent/tree/main/examples/content_safety):
+    a **demo** content-safety probe in both `single_shot` and `trajectory`
+    variants, with YAML specs, a dummy-checkpoint maker, a vLLM pipeline
+    script and a Dockerfile. It is a demonstration of the probe API, not a
+    trained safety classifier.
+  - [`examples/openai_server/`](https://github.com/wrynx/undercurrent/tree/main/examples/openai_server):
+    a reference OpenAI-compatible wire format for probe verdicts (completion
+    bodies and SSE chunks) and a reference HTTP server built on it.
+- **Probe-training example** (`examples/train_probe/`) and Colab notebooks
+  (`notebooks/quickstart.ipynb`, `notebooks/train_probe.ipynb`).
+
+### Fixed
+
+- **vLLM `residual_stream` on fused-residual layers.** On Llama-family models
+  (and the other vLLM architectures whose decoder layer returns
+  `(hidden_states, residual)`), a `residual_stream` extraction point captured
+  the layer's MLP output. It now captures `hidden_states + residual`, the
+  residual stream after the layer, matching the HF backend. Decoder-layer
+  outputs the adapter can't classify raise `VLLMAdapterLimitationError`
+  instead of being captured silently. Verified on GPU (NVIDIA L4, vLLM 0.28.0)
+  by `tests/adapters/vllm/test_residual_stream_gpu.py`, which compares the
+  vLLM and HF captures token by token.
+
+### Known issues
+
+See [Known issues (v0.1)](https://github.com/wrynx/undercurrent/blob/main/docs/compatibility.md#known-issues-v01)
+for details and workarounds.
+
+- **No GPU CI.** The GPU tests run by hand with `scripts/gpu_check.sh` before
+  each release. 0.1.0 passed on an NVIDIA L4 with vLLM 0.28.0, torch 2.13.0
+  and CUDA 13.0; other GPUs and multi-GPU topologies haven't been run.
+- **Only vLLM 0.28 is supported** (`vllm>=0.28,<0.29`), and vLLM 0.30 is
+  already out. `UNDERCURRENT_ALLOW_UNSUPPORTED_VLLM=1` lets you try a newer
+  vLLM; the range widens only after a GPU validation run.
+
+### Security
+
+- Vulnerabilities can be reported privately to security@wrynx.com or through
+  GitHub private vulnerability reporting. See
+  [SECURITY.md](https://github.com/wrynx/undercurrent/blob/main/SECURITY.md).
+- The content-safety example loads classifier checkpoints only with
+  `torch.load(..., weights_only=True)`, accepts only plain tensor
+  `state_dict`s, and refuses to load on torch older than 2.6 (the first
+  release with a fix for CVE-2025-32434).
+
+[Unreleased]: https://github.com/wrynx/undercurrent/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/wrynx/undercurrent/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/wrynx/undercurrent/releases/tag/v0.1.0

undercurrent-0.2.0/NOTICE

@@ -0,0 +1,4 @@
+Undercurrent, Wrynx's activation-probing platform
+Copyright 2026 Wrynx
+
+This product includes software developed at Wrynx.

undercurrent-0.2.0/PKG-INFO

@@ -0,0 +1,229 @@
+Metadata-Version: 2.5
+Name: undercurrent
+Version: 0.2.0
+Summary: Undercurrent: production-grade LLM activation probing and intervention. Works with Hugging Face Transformers and vLLM.
+Project-URL: Homepage, https://github.com/wrynx/undercurrent
+Project-URL: Source, https://github.com/wrynx/undercurrent
+Project-URL: Issues, https://github.com/wrynx/undercurrent/issues
+Project-URL: Changelog, https://github.com/wrynx/undercurrent/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://wrynx.github.io/undercurrent/
+Author: Wrynx
+Maintainer: Wrynx
+License-Expression: Apache-2.0
+License-File: LICENSE
+License-File: NOTICE
+Keywords: activation-probing,ai-safety,guardrails,interpretability,llm,llm-inference,mechanistic-interpretability,probes,transformers,undercurrent,vllm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.23
+Requires-Dist: packaging>=22
+Requires-Dist: pydantic<3,>=2
+Requires-Dist: pyyaml<7,>=6
+Requires-Dist: torch>=2.1
+Requires-Dist: transformers<6,>=4.40
+Provides-Extra: dev
+Requires-Dist: ipykernel>=6; extra == 'dev'
+Requires-Dist: jsonschema<5,>=4.18; extra == 'dev'
+Requires-Dist: mypy>=1.11; extra == 'dev'
+Requires-Dist: nbclient>=0.10; extra == 'dev'
+Requires-Dist: nbformat>=5.10; extra == 'dev'
+Requires-Dist: pre-commit>=4; extra == 'dev'
+Requires-Dist: pytest-cov>=5; extra == 'dev'
+Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff<0.18,>=0.16; extra == 'dev'
+Requires-Dist: types-pyyaml; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material; extra == 'docs'
+Requires-Dist: mkdocs<2,>=1.6; extra == 'docs'
+Requires-Dist: mkdocstrings[python]; extra == 'docs'
+Requires-Dist: mktestdocs; extra == 'docs'
+Requires-Dist: pymdown-extensions; extra == 'docs'
+Provides-Extra: vllm
+Requires-Dist: vllm<0.29,>=0.28; extra == 'vllm'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/wrynx/undercurrent/main/docs/assets/brand/undercurrent-logo-dark.png">
+    <img alt="Undercurrent" src="https://raw.githubusercontent.com/wrynx/undercurrent/main/docs/assets/brand/undercurrent-logo.png" width="440">
+  </picture>
+</p>
+
+<p align="center"><em>See the undercurrent before it surfaces.</em></p>
+
+[![PyPI](https://img.shields.io/pypi/v/undercurrent)](https://pypi.org/project/undercurrent/)
+[![Python versions](https://img.shields.io/pypi/pyversions/undercurrent)](https://pypi.org/project/undercurrent/)
+[![CI](https://github.com/wrynx/undercurrent/actions/workflows/ci.yml/badge.svg)](https://github.com/wrynx/undercurrent/actions/workflows/ci.yml)
+[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/wrynx/undercurrent/blob/main/LICENSE)
+[![Docs](https://img.shields.io/badge/docs-wrynx.github.io%2Fundercurrent-blue)](https://wrynx.github.io/undercurrent/)
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/wrynx/undercurrent/blob/main/notebooks/quickstart.ipynb)
+
+<sub>An open-source project by [Wrynx](https://github.com/wrynx).</sub>
+
+Undercurrent is production-grade activation probing for live LLM inference.
+It runs your probes on a model's internal activations *while it generates*,
+so a probe can stop a generation mid-flight, or observe asynchronously
+without adding latency to the response.
+
+## Why Undercurrent
+
+- **Declarative YAML specs.** A few lines say which tensor to capture, at
+  which layers and token positions, and which probe receives it. No model
+  surgery, no forked serving code.
+- **Inline abort mid-generation.** An inline probe runs on the generation path
+  and can stop a request before the next token is produced.
+- **Production-ready async execution.** Observe-only probes run off the hot
+  path on bounded per-binding queues with overflow policies, and report to
+  metrics and to file or webhook sinks (with retries, dead-lettering and
+  redaction).
+- **Runs where your model runs.** Inside your existing vLLM deployment, with
+  continuous batching (the vLLM adapter is experimental, see
+  [Deploy with vLLM](https://wrynx.github.io/undercurrent/guides/vllm-deployment/)),
+  and with Hugging Face Transformers on a laptop CPU.
+
+> **Known issues (v0.2):** the vLLM adapter is experimental: its GPU tests
+> pass on an NVIDIA L4 with vLLM 0.28.0, run by hand (there's no GPU CI yet),
+> and only vLLM 0.28 is supported. Fixed in 0.1.0: on vLLM, `residual_stream`
+> on Llama-family (fused-residual) models is captured as
+> `hidden_states + residual`, the same residual stream as the HF backend
+> (GPU-verified). See
+> [Known issues](https://wrynx.github.io/undercurrent/compatibility/#known-issues-v02).
+
+## How it works
+
+In short: an engine adapter (Hugging Face or vLLM) captures the activations your
+YAML spec asks for, and the router hands them to your probes, which either
+abort the generation (inline) or report to sinks and metrics (async).
+
+```mermaid
+flowchart LR
+    spec["YAML spec"] --> adapter
+    subgraph engine["Inference engine (HF Transformers or vLLM)"]
+        model["LLM generating tokens"] -- forward hooks --> adapter["Engine adapter"]
+    end
+    adapter -- ActivationRecords --> router["Router"]
+    router -- inline --> inline["Inline probes"]
+    inline -- ABORT --> model
+    router -- async, bounded queues --> async["Async probes"]
+    async --> sinks["Sinks: file / webhook / metrics"]
+```
+
+## Install
+
+```bash
+pip install undercurrent
+```
+
+Python 3.10+. The Hugging Face Transformers backend is included; there are no
+extras to choose. For vLLM, install Undercurrent into your existing vLLM
+environment or image, or use `pip install "undercurrent[vllm]"` to get a vLLM
+from the tested range (currently `vllm>=0.28,<0.29`). The adapter checks the
+installed vLLM version at runtime; set `UNDERCURRENT_ALLOW_UNSUPPORTED_VLLM=1`
+to try an untested vLLM anyway.
+
+Supported versions, tested combinations and known issues are on the
+[Compatibility](https://wrynx.github.io/undercurrent/compatibility/) page.
+
+## Quickstart
+
+Attach a probe to GPT-2 and read its result. This runs on CPU:
+
+```python
+from undercurrent.core import ActivationRecord, probe
+from undercurrent.model import ProbedModel
+
+SPEC = """
+version: "1"
+extraction_points:
+  - name: last_prompt_token
+    layers: 6
+    tensor_type: residual_stream
+    position: "prompt[-1]"
+    probe_type: norm_gate
+    probe_kind: single_shot
+    execution_mode: inline
+"""
+
+
+@probe("norm_gate", threshold=500.0)
+def norm_gate(record: ActivationRecord) -> float:
+    """L2 norm of the residual stream at this token."""
+    return float(record.tensor.norm())
+
+
+with ProbedModel.from_pretrained("openai-community/gpt2", spec=SPEC, probes={"norm_gate": norm_gate}) as model:
+    out = model.generate("The quick brown fox", max_new_tokens=20, temperature=0)
+
+print(repr(out.text))
+print(out.probe_results["last_prompt_token"].verdict)
+```
+
+A score at or above `threshold` would abort the generation. The
+[Quickstart](https://wrynx.github.io/undercurrent/getting-started/quickstart/)
+continues with a trajectory probe that stops a generation part-way through,
+and an async probe that logs to a file without touching the response.
+
+## How it compares
+
+| | Undercurrent | TransformerLens / pyvene / baukit | nnsight (+ NDIF) | Llama Guard / NeMo Guardrails |
+|---|---|---|---|---|
+| Purpose | Production activation probing | Interpretability research | Interpretability research, local or remote | Text-level safety classification and rails |
+| Runs inside vLLM serving | partial (single worker validated) | ✗ | ✓ | n/a (separate model / wraps the LLM call) |
+| Abort a request mid-generation from a probe | ✓ | partial (edit activations; no built-in abort) | partial (edit activations; no built-in abort) | Llama Guard: n/a (application decides); NeMo: partial (streaming rails end the stream, in text chunks) |
+| Declarative spec | ✓ YAML | pyvene: partial (dict configs); others ✗ | ✗ | NeMo: ✓ (YAML + Colang) |
+| License | Apache-2.0 | MIT / Apache-2.0 / MIT | MIT | Llama 4 Community License / Apache-2.0 |
+
+Last reviewed: 2026-10-01. Footnotes, sources and when to pick something else:
+[full comparison](https://wrynx.github.io/undercurrent/comparison/).
+
+## Documentation
+
+- [Quickstart](https://wrynx.github.io/undercurrent/getting-started/quickstart/): your first probe on GPT-2, in five minutes.
+- [Concepts](https://wrynx.github.io/undercurrent/concepts/architecture/): extraction points, position selectors, probes, inline vs async.
+- [Guides](https://wrynx.github.io/undercurrent/guides/custom-probe/): write, test and package your own probe.
+- [Production](https://wrynx.github.io/undercurrent/guides/vllm-deployment/): vLLM deployment, embedding the router, backpressure, sinks and metrics.
+- [Examples](https://wrynx.github.io/undercurrent/examples/): complete, runnable projects.
+- [API reference](https://wrynx.github.io/undercurrent/reference/): every public class and function, and the CLI.
+- [Intended use and limitations](https://wrynx.github.io/undercurrent/about/intended-use/) and [privacy and data handling](https://wrynx.github.io/undercurrent/about/privacy/): what Undercurrent is and isn't for, and what data it captures and writes.
+- [Colab notebook](https://colab.research.google.com/github/wrynx/undercurrent/blob/main/notebooks/quickstart.ipynb): the quickstart in your browser.
+
+[`examples/content_safety/`](https://github.com/wrynx/undercurrent/tree/main/examples/content_safety)
+is a demo of the wiring (probes, specs and a vLLM pipeline), not a safety
+product. Its probes use random-initialised or dummy weights.
+
+## Project status
+
+Undercurrent is on 0.x: the API may change between minor releases. The
+[API stability policy](https://github.com/wrynx/undercurrent/blob/main/docs/api-stability.md)
+says what is public, what is experimental, and how deprecations work. Planned
+work is in the [roadmap](https://github.com/wrynx/undercurrent/blob/main/ROADMAP.md).
+
+## Contributing and community
+
+- [Contributing guide](https://github.com/wrynx/undercurrent/blob/main/CONTRIBUTING.md)
+- [Security policy](https://github.com/wrynx/undercurrent/blob/main/SECURITY.md): report vulnerabilities privately to security@wrynx.com.
+- [Code of Conduct](https://github.com/wrynx/undercurrent/blob/main/CODE_OF_CONDUCT.md)
+
+## Citation
+
+If you use Undercurrent in research, please cite it using
+[`CITATION.cff`](https://github.com/wrynx/undercurrent/blob/main/CITATION.cff)
+(GitHub's "Cite this repository" button reads it).
+
+## License
+
+Apache-2.0. See [LICENSE](https://github.com/wrynx/undercurrent/blob/main/LICENSE)
+and [NOTICE](https://github.com/wrynx/undercurrent/blob/main/NOTICE). Copyright Wrynx.