ithrl 0.1.0__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ithrl-0.1.0 → ithrl-0.2.0}/.gitignore +1 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/CHANGELOG.md +50 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/CONTRIBUTING.md +123 -34
- ithrl-0.2.0/PKG-INFO +153 -0
- ithrl-0.2.0/README.md +120 -0
- ithrl-0.2.0/ROADMAP.md +126 -0
- ithrl-0.2.0/docs/architecture.md +221 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/docs/cli.md +33 -0
- ithrl-0.2.0/docs/decisions/0001-native-evidence-and-controlled-runners.md +82 -0
- ithrl-0.2.0/docs/decisions/0002-fixed-samples-and-durable-dispatch.md +122 -0
- ithrl-0.2.0/docs/decisions/0003-strict-fresh-baselines.md +71 -0
- ithrl-0.2.0/docs/decisions/0004-single-finding-fuzz-confirmation.md +80 -0
- ithrl-0.2.0/docs/decisions/index.md +21 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/docs/index.md +14 -5
- ithrl-0.2.0/docs/measurements.md +303 -0
- ithrl-0.2.0/docs/native-runs.md +427 -0
- ithrl-0.2.0/docs/research/context-fuzzing.md +345 -0
- ithrl-0.2.0/docs/research/feasibility.md +195 -0
- ithrl-0.2.0/docs/research/index.md +31 -0
- ithrl-0.2.0/docs/research/reduction-design.md +368 -0
- ithrl-0.2.0/docs/results.md +166 -0
- ithrl-0.2.0/docs/troubleshooting.md +155 -0
- ithrl-0.2.0/examples/fuzz/campaign.toml +37 -0
- ithrl-0.2.0/examples/fuzz/codex.jsonl +7 -0
- ithrl-0.2.0/examples/fuzz/copilot.jsonl +9 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/pyproject.toml +36 -3
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/_jsonl.py +30 -27
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/codex/context.py +43 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/copilot/context.py +82 -8
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/registry.py +25 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/behavior.py +27 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/calibration.py +166 -30
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/evidence.py +43 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/__init__.py +63 -15
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/extract.py +75 -7
- ithrl-0.2.0/src/ithrl/cli/fuzz.py +464 -0
- ithrl-0.2.0/src/ithrl/cli/fuzz_output.py +317 -0
- ithrl-0.2.0/src/ithrl/cli/fuzz_run.py +434 -0
- ithrl-0.2.0/src/ithrl/cli/fuzz_session.py +233 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/native.py +46 -3
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/output.py +44 -2
- ithrl-0.2.0/src/ithrl/cli/progress.py +62 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/sampled.py +83 -4
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/sampled_output.py +18 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/sampled_reduction.py +52 -84
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/sampled_session.py +1 -1
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/shrink.py +4 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/context.py +3 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/execution/budget.py +19 -11
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/execution/contracts.py +2 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/execution/preflight.py +14 -4
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/experiment.py +28 -1
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/extraction.py +40 -7
- ithrl-0.2.0/src/ithrl/fuzz/__init__.py +10 -0
- ithrl-0.2.0/src/ithrl/fuzz/campaign.py +534 -0
- ithrl-0.2.0/src/ithrl/fuzz/configuration.py +77 -0
- ithrl-0.2.0/src/ithrl/fuzz/contracts.py +108 -0
- ithrl-0.2.0/src/ithrl/fuzz/deterministic.py +88 -0
- ithrl-0.2.0/src/ithrl/fuzz/generation.py +55 -0
- ithrl-0.2.0/src/ithrl/fuzz/progress.py +117 -0
- ithrl-0.2.0/src/ithrl/fuzz/sampling.py +231 -0
- ithrl-0.2.0/src/ithrl/fuzz/simplify.py +96 -0
- ithrl-0.2.0/src/ithrl/fuzz/statistics.py +91 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/measurement.py +2 -0
- ithrl-0.2.0/src/ithrl/mutation.py +159 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/observations.py +54 -11
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/reduce/contracts.py +8 -6
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/reduce/engine.py +5 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/results/models.py +7 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/results/storage.py +2 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/results/summary.py +2 -9
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/results/validation.py +13 -3
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/run_state.py +146 -32
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/capability.py +13 -3
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/configuration.py +6 -6
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/isolation.py +9 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/materialize.py +37 -12
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/policy.py +2 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/protocol.py +66 -34
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/protocol_contracts.py +8 -1
- ithrl-0.2.0/src/ithrl/runner/codex/relay.py +403 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/runner.py +71 -18
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/runtime_configuration.py +4 -2
- ithrl-0.2.0/src/ithrl/runner/codex/transcript.py +288 -0
- ithrl-0.2.0/src/ithrl/runner/codex/worker.py +334 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/contracts.py +37 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/capability.py +48 -156
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/configuration.py +3 -1
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/isolation.py +64 -25
- ithrl-0.2.0/src/ithrl/runner/copilot/materialize.py +269 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/policy.py +11 -39
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/protocol/__init__.py +1 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/protocol/contracts.py +21 -2
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/protocol/decoding.py +3 -1
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/protocol/parser.py +243 -85
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/protocol/state.py +14 -10
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/runner.py +47 -28
- ithrl-0.2.0/src/ithrl/runner/copilot/sdk_runtime.py +138 -0
- ithrl-0.2.0/src/ithrl/runner/copilot/worker.py +678 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/evidence.py +52 -10
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/fake.py +3 -1
- ithrl-0.2.0/src/ithrl/runner/runtime/certificates.py +58 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/contracts.py +2 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/process.py +31 -10
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/protection.py +9 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/validation.py +1 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/sampling.py +41 -16
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/workflow.py +50 -14
- ithrl-0.1.0/PKG-INFO +0 -118
- ithrl-0.1.0/README.md +0 -89
- ithrl-0.1.0/docs/measurements.md +0 -86
- ithrl-0.1.0/docs/native-runs.md +0 -150
- ithrl-0.1.0/docs/results.md +0 -105
- ithrl-0.1.0/docs/troubleshooting.md +0 -73
- ithrl-0.1.0/src/ithrl/runner/codex/transcript.py +0 -130
- ithrl-0.1.0/src/ithrl/runner/codex/worker.py +0 -175
- ithrl-0.1.0/src/ithrl/runner/copilot/materialize.py +0 -484
- ithrl-0.1.0/src/ithrl/runner/copilot/worker.py +0 -351
- {ithrl-0.1.0 → ithrl-0.2.0}/LICENSE +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/docs/third-party-notices.txt +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/__main__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/codex/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/codex/decisions.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/codex/discovery.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/codex/parser.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/contracts.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/copilot/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/copilot/decisions.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/copilot/discovery.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/copilot/parser.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/discovery.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/adapters/errors.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/contracts.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/filesystem.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/source.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/source_context.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/source_storage.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/case/storage.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/cli/paths.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/decisions.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/examples/synthetic-trace.json +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/execution/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/path_safety.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/reduce/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/reduce/search.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/results/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/codex/schema.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/copilot/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/registry.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/__init__.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/output.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/runner/runtime/state.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/statistics.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/synthetic.py +0 -0
- {ithrl-0.1.0 → ithrl-0.2.0}/src/ithrl/timestamps.py +0 -0
|
@@ -1,8 +1,56 @@
|
|
|
1
1
|
# CHANGELOG
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## v0.2.0 (2026-11-02)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
### Bug Fixes
|
|
6
|
+
|
|
7
|
+
- **codex**: Bound and account for physical provider requests
|
|
8
|
+
([#33](https://github.com/eberkut/ithrl/pull/33),
|
|
9
|
+
[`88529d6`](https://github.com/eberkut/ithrl/commit/88529d6f8b930dbcc546a0011f085c67d0bef1ac))
|
|
10
|
+
|
|
11
|
+
- **copilot**: Recover catalogue connections and qualify fuzzing
|
|
12
|
+
([#37](https://github.com/eberkut/ithrl/pull/37),
|
|
13
|
+
[`ff6cf63`](https://github.com/eberkut/ithrl/commit/ff6cf6324325f7f6dad8f8c9c80ea58bfe4e249a))
|
|
14
|
+
|
|
15
|
+
- **copilot**: Restore native session prefixes through the CLI
|
|
16
|
+
([#34](https://github.com/eberkut/ithrl/pull/34),
|
|
17
|
+
[`36845dd`](https://github.com/eberkut/ithrl/commit/36845ddb8239c0a488552297aaa70e6211979e59))
|
|
18
|
+
|
|
19
|
+
- **fuzz**: Preserve accounting and validate campaign recovery
|
|
20
|
+
([#25](https://github.com/eberkut/ithrl/pull/25),
|
|
21
|
+
[`f1f4793`](https://github.com/eberkut/ithrl/commit/f1f479318fe914f3033667b65381f7101bd3aaea))
|
|
22
|
+
|
|
23
|
+
- **sampling**: Enforce calibration budgets and report durable progress
|
|
24
|
+
([#32](https://github.com/eberkut/ithrl/pull/32),
|
|
25
|
+
[`bd40577`](https://github.com/eberkut/ithrl/commit/bd40577c734d70b36e59a9e6d3babafa1502bfd3))
|
|
26
|
+
|
|
27
|
+
### Features
|
|
28
|
+
|
|
29
|
+
- **extract**: Support literal predicates for native decisions
|
|
30
|
+
([#31](https://github.com/eberkut/ithrl/pull/31),
|
|
31
|
+
[`3b18c2b`](https://github.com/eberkut/ithrl/commit/3b18c2bdd814c4e4c3bf79825a8cab77eb433661))
|
|
32
|
+
|
|
33
|
+
- **fuzz**: Add mutation and sampled comparison services
|
|
34
|
+
([`56ee8b8`](https://github.com/eberkut/ithrl/commit/56ee8b8c5e1370b91db6c9c952eb8ceec19c95ae))
|
|
35
|
+
|
|
36
|
+
- **fuzz**: Expose inspectable campaigns and saved results
|
|
37
|
+
([#24](https://github.com/eberkut/ithrl/pull/24),
|
|
38
|
+
[`e60e472`](https://github.com/eberkut/ithrl/commit/e60e47230b99e50ff42de7cd8fb89b3769e6d6e2))
|
|
39
|
+
|
|
40
|
+
- **fuzz**: Persist campaigns and recover interrupted execution
|
|
41
|
+
([#22](https://github.com/eberkut/ithrl/pull/22),
|
|
42
|
+
[`806667f`](https://github.com/eberkut/ithrl/commit/806667f47e8bae1ccbf9b32fadc511ca559675d0))
|
|
43
|
+
|
|
44
|
+
- **fuzz**: Qualify native Codex text appends ([#28](https://github.com/eberkut/ithrl/pull/28),
|
|
45
|
+
[`a5b880b`](https://github.com/eberkut/ithrl/commit/a5b880b60ef9834bb57b2c4950e18cd17ff0b275))
|
|
46
|
+
|
|
47
|
+
### Performance Improvements
|
|
48
|
+
|
|
49
|
+
- Streamline local workflows and harden result publication
|
|
50
|
+
([#36](https://github.com/eberkut/ithrl/pull/36),
|
|
51
|
+
[`43b9d81`](https://github.com/eberkut/ithrl/commit/43b9d81d0dcff85708da5c914986a68bbe8141db))
|
|
52
|
+
|
|
53
|
+
## v0.1.0 (2026-10-23)
|
|
6
54
|
|
|
7
55
|
### Bug Fixes
|
|
8
56
|
|
|
@@ -1,11 +1,28 @@
|
|
|
1
1
|
# Contributing
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
change. Pull requests are limited to collaborators
|
|
3
|
+
Open an [issue](https://github.com/eberkut/ithrl/issues) for a bug report, question or proposed
|
|
4
|
+
change. Pull requests are limited to collaborators. If you are not a collaborator and want to
|
|
5
|
+
implement a change, agree on scope and ask the maintainer about access in the issue before preparing
|
|
6
|
+
a PR.
|
|
5
7
|
|
|
6
8
|
For bug reports, include the ithrl version, platform, sanitized error code and steps to reproduce.
|
|
7
9
|
Keep full sessions, cases, proprietary source and credentials out of public reports. Report
|
|
8
|
-
suspected vulnerabilities through
|
|
10
|
+
suspected vulnerabilities through
|
|
11
|
+
[private vulnerability reporting](https://github.com/eberkut/ithrl/security/advisories/new).
|
|
12
|
+
|
|
13
|
+
## First contribution
|
|
14
|
+
|
|
15
|
+
1. Find an existing issue or open one, and agree on scope with the maintainer.
|
|
16
|
+
2. Follow [development setup](#development-setup) to prepare the locked environment.
|
|
17
|
+
3. Make a focused change using [Preparing a change](#preparing-a-change), and run the relevant
|
|
18
|
+
[checks](#checks).
|
|
19
|
+
4. Open a PR linking the issue and explaining the change and its validation. Follow the
|
|
20
|
+
[review requirements](#pull-request-review).
|
|
21
|
+
|
|
22
|
+
For task-specific instructions, see [documentation](#documentation),
|
|
23
|
+
[workflows and dependencies](#workflow-and-dependency-checks),
|
|
24
|
+
[integration tests](#credential-free-integration-tests) or
|
|
25
|
+
[native smoke tests](#opt-in-native-smoke-tests).
|
|
9
26
|
|
|
10
27
|
## Development setup
|
|
11
28
|
|
|
@@ -13,7 +30,7 @@ Use [uv](https://docs.astral.sh/uv/) and Python 3.12, 3.13 or 3.14. Python 3.14
|
|
|
13
30
|
development version. From the repository root:
|
|
14
31
|
|
|
15
32
|
```bash
|
|
16
|
-
uv sync --locked --all-groups
|
|
33
|
+
uv sync --locked --all-groups --all-extras
|
|
17
34
|
uv run ithrl --help
|
|
18
35
|
```
|
|
19
36
|
|
|
@@ -28,6 +45,8 @@ Run the same entry points used by CI:
|
|
|
28
45
|
uv lock --check
|
|
29
46
|
uv run ruff check .
|
|
30
47
|
uv run ruff format --check .
|
|
48
|
+
uv run rumdl fmt --check .
|
|
49
|
+
uv run rumdl check .
|
|
31
50
|
uv run pyrefly check
|
|
32
51
|
uv run pytest --ignore=tests/packaging
|
|
33
52
|
uv run radon cc src/ithrl --min B --show-complexity --total-average
|
|
@@ -36,7 +55,8 @@ uv build --sdist --wheel --no-sources
|
|
|
36
55
|
ITHRL_DIST_DIR=dist uv run pytest -o addopts= tests/packaging
|
|
37
56
|
```
|
|
38
57
|
|
|
39
|
-
Non-packaging tests and the synthetic example need no provider credentials or network
|
|
58
|
+
Non-packaging tests and the synthetic example need no provider credentials or external network
|
|
59
|
+
access. Native-runner tests need Linux, Bubblewrap and OpenSSL; their HTTPS fixture runs locally.
|
|
40
60
|
Packaging checks may download build and runtime dependencies. Radon reports are review signals.
|
|
41
61
|
|
|
42
62
|
CI runs tests, dependency audits and installed-package checks on Python 3.12–3.14. It builds one
|
|
@@ -47,6 +67,21 @@ an isolated tool, pass its path to `uv tool install` or `pipx install`.
|
|
|
47
67
|
For local matrix runs, set `UV_PYTHON` on every command. Use separate environments, test caches and
|
|
48
68
|
coverage files for parallel runs; otherwise run versions sequentially.
|
|
49
69
|
|
|
70
|
+
Install the local hooks with `uv run pre-commit install`; run them on the whole repository with
|
|
71
|
+
`uv run pre-commit run --all-files`. Ruff and rumdl format files in place. CI checks formatting
|
|
72
|
+
without modifying files.
|
|
73
|
+
|
|
74
|
+
Recognized Markdown-only PRs skip Python linting, typing, unit-test matrices and dependency audits.
|
|
75
|
+
They retain Markdown formatting/linting, documentation tests, the strict site build, workflow and
|
|
76
|
+
dependency review, and all package build/install checks. Mixed changes, unknown paths, configuration
|
|
77
|
+
changes and PRs with 3,000 or more changed files run full validation. Rename detection considers
|
|
78
|
+
both paths; deletions participate. Failed path detection fails the required Workflows check.
|
|
79
|
+
Main-branch, scheduled, manual and release runs always perform full validation.
|
|
80
|
+
|
|
81
|
+
The required `Python tests` check summarizes the test matrix. It accepts a skipped matrix only
|
|
82
|
+
after successful Markdown-only classification. Failed classification, missing decisions, test
|
|
83
|
+
failures and unexpected skips fail this check.
|
|
84
|
+
|
|
50
85
|
## Workflow and dependency checks
|
|
51
86
|
|
|
52
87
|
For GitHub Actions or Dependabot changes, use the jactionlint release pinned in
|
|
@@ -58,12 +93,13 @@ jactionlint --no-online --strict-exit
|
|
|
58
93
|
|
|
59
94
|
CI uses its default correctness/security profile, with online lookups and automatic fixes disabled.
|
|
60
95
|
|
|
61
|
-
Audit the resolved runtime dependencies and every dependency group, including
|
|
62
|
-
and audit tooling, from `uv.lock`. Run this in Bash with the locked environment
|
|
96
|
+
Audit the resolved runtime dependencies, optional extras and every dependency group, including
|
|
97
|
+
development, docs and audit tooling, from `uv.lock`. Run this in Bash with the locked environment
|
|
98
|
+
synchronized:
|
|
63
99
|
|
|
64
100
|
```bash
|
|
65
101
|
set -o pipefail
|
|
66
|
-
uv export --locked --all-groups --no-emit-project --format requirements.txt |
|
|
102
|
+
uv export --locked --all-groups --all-extras --no-emit-project --format requirements.txt |
|
|
67
103
|
uv run --locked --group audit pip-audit --strict --disable-pip --require-hashes -r /dev/stdin
|
|
68
104
|
```
|
|
69
105
|
|
|
@@ -72,25 +108,48 @@ package names and versions without installing those packages, resolving alternat
|
|
|
72
108
|
fixes. CI evaluates markers on each supported Python version. This covers locked versions rather
|
|
73
109
|
than every version permitted by package dependency ranges.
|
|
74
110
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
111
|
+
For dependency changes, run the lockfile audit above on each supported Python version. GitHub's
|
|
112
|
+
dependency graph may omit packages from `uv.lock`, so Dependabot and Dependency Review cannot
|
|
113
|
+
replace that audit.
|
|
114
|
+
|
|
115
|
+
The PR dependency-review job rejects newly introduced known vulnerabilities at any severity across
|
|
116
|
+
runtime, development and unknown scopes. It uses a read-only token and does not check out or execute
|
|
117
|
+
PR code. If it reports incomplete dependency snapshots, wait for GitHub's graph update and rerun
|
|
118
|
+
the job. License checks are disabled.
|
|
83
119
|
|
|
84
120
|
CI runs weekly on Monday at 06:17 UTC to detect new advisories. Dependabot checks Python, Actions
|
|
85
121
|
and pre-commit dependencies weekly, groups minor/patch updates and leaves majors individually
|
|
86
122
|
reviewable. Ordinary updates have a seven-day cooldown; security updates have none. Updates are
|
|
87
|
-
never automatically merged.
|
|
123
|
+
never automatically merged.
|
|
88
124
|
|
|
89
125
|
## Documentation
|
|
90
126
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
127
|
+
Use `uv run rumdl fmt .` to format Markdown, then `uv run rumdl check .` to lint it. The same
|
|
128
|
+
version and `pyproject.toml` configuration apply to pre-commit and CI. Prose wraps at 100
|
|
129
|
+
characters; commands and tables retain their layout. MkDocs syntax is recognized within `docs/`. The
|
|
130
|
+
generated CLI reference has narrow exceptions for Typer's prompts, option alignment and wrapping;
|
|
131
|
+
its byte-for-byte generation check remains enforced. The generated changelog is checked under the
|
|
132
|
+
shared rules.
|
|
133
|
+
|
|
134
|
+
Keep installation and first-use instructions in README, and detailed user guides and reference in
|
|
135
|
+
`docs/`. Support, security and conduct policies live at the repository root. Keep private sessions,
|
|
136
|
+
cases and credentials out of documentation and generated output.
|
|
137
|
+
|
|
138
|
+
[ROADMAP.md](ROADMAP.md) describes future direction. GitHub issues own actionable scope,
|
|
139
|
+
dependencies, acceptance criteria and status; release milestones group scope and the public
|
|
140
|
+
[Project](https://github.com/users/eberkut/projects/1/views/4) provides views of those same issues.
|
|
141
|
+
Link implementation pull requests to their owning issues and record relevant validation before
|
|
142
|
+
closing them. Keep future plans separate from current support claims.
|
|
143
|
+
|
|
144
|
+
Use [architecture](docs/architecture.md) to locate current contracts,
|
|
145
|
+
[decision records](docs/decisions/index.md) for significant choices and alternatives, and
|
|
146
|
+
[research](docs/research/index.md) for methods, measurements and reproducibility. Amend the owning
|
|
147
|
+
document instead of maintaining a second plan. A new decision record should explain context,
|
|
148
|
+
decision, alternatives and consequences; supersede an old decision explicitly when needed.
|
|
149
|
+
|
|
150
|
+
Before evaluating a new study, declare its tested domain, criteria, seeds, budget and intended
|
|
151
|
+
claims. Report costs across complete workflows, including failed and incomplete work, and document
|
|
152
|
+
the alternatives and limitations needed to interpret the results.
|
|
94
153
|
|
|
95
154
|
The optional `docs` dependency group pins MkDocs and the shadcn theme. Search runs in the browser;
|
|
96
155
|
fonts and scripts are served with the site. GitHub star-count requests and raw Markdown copying
|
|
@@ -100,13 +159,16 @@ When changing the theme or enabled extensions, check that asset list and the
|
|
|
100
159
|
To preview the site locally:
|
|
101
160
|
|
|
102
161
|
```bash
|
|
103
|
-
uv sync --locked --all-groups
|
|
162
|
+
uv sync --locked --all-groups --all-extras
|
|
104
163
|
uv run --group docs mkdocs serve
|
|
105
164
|
```
|
|
106
165
|
|
|
107
|
-
`mkdocs.yml` admits
|
|
108
|
-
|
|
109
|
-
|
|
166
|
+
`mkdocs.yml` admits public pages and Markdown files in the dedicated `docs/decisions/` and
|
|
167
|
+
`docs/research/` directories. These directories contain publication-ready material only. Add new
|
|
168
|
+
pages to navigation; a page within an admitted directory needs no per-file package/test entry.
|
|
169
|
+
When adding a public directory or asset type, update the site/sdist publication boundaries and
|
|
170
|
+
their tests together. Never set `docs_dir` to the repository root. Keep raw evidence and generated
|
|
171
|
+
research outputs outside the documentation sources.
|
|
110
172
|
|
|
111
173
|
After changing CLI options or help, regenerate the committed reference with Typer:
|
|
112
174
|
|
|
@@ -132,7 +194,7 @@ runs in a separate `github-pages` environment job without checking out or execut
|
|
|
132
194
|
Pull requests receive read-only documentation checks through CI and cannot deploy.
|
|
133
195
|
|
|
134
196
|
Repository administrators must select **Settings → Pages → Source: GitHub Actions** and restrict
|
|
135
|
-
the `github-pages` environment to `main
|
|
197
|
+
the `github-pages` environment to `main`. No custom domain, separate
|
|
136
198
|
site repository, `gh-pages` branch or personal access token is needed. After a successful
|
|
137
199
|
deployment, check the published pages, navigation and static assets at the URL above.
|
|
138
200
|
|
|
@@ -153,8 +215,15 @@ accounting and budget refusal:
|
|
|
153
215
|
uv run pytest -o addopts= tests/integration/test_native_calibration.py -q
|
|
154
216
|
```
|
|
155
217
|
|
|
218
|
+
Fuzz CLI tests cover harness-free inspection, consent, bounded collection, unsuccessful outcomes,
|
|
219
|
+
interruption and exact resume without provider requests:
|
|
220
|
+
|
|
221
|
+
```bash
|
|
222
|
+
uv run pytest -o addopts= tests/integration/test_fuzz_cli.py -q
|
|
223
|
+
```
|
|
224
|
+
|
|
156
225
|
These tests establish local contracts. Native qualification is recorded in the
|
|
157
|
-
[
|
|
226
|
+
[measurements](docs/measurements.md).
|
|
158
227
|
|
|
159
228
|
## Opt-in native smoke tests
|
|
160
229
|
|
|
@@ -188,13 +257,13 @@ uv run pytest -o addopts= tests/live -m native_smoke -s \
|
|
|
188
257
|
```
|
|
189
258
|
|
|
190
259
|
Every invocation creates fresh private state, harmless session content and an empty original
|
|
191
|
-
workspace. Calibration requires two matching attempts and permits at most two, each bounded at
|
|
192
|
-
|
|
193
|
-
Copilot
|
|
260
|
+
workspace. Calibration requires two matching attempts and permits at most two, each bounded at 120
|
|
261
|
+
seconds plus cancellation grace. Codex allows up to 32 provider calls across the two attempts;
|
|
262
|
+
either Copilot profile allows up to two. In the shell profile, `pwd` is visible but denied.
|
|
194
263
|
|
|
195
|
-
Returned actions never execute. Hosted tools, MCP and tool-side network effects remain forbidden.
|
|
196
|
-
|
|
197
|
-
|
|
264
|
+
Returned actions never execute. Hosted tools, MCP and tool-side network effects remain forbidden. An
|
|
265
|
+
input-token cap cannot be enforced. Private evidence records known counts and unknown acceptance
|
|
266
|
+
explicitly. Unsupported or unsuccessful profiles fail the test without skipping or retrying.
|
|
198
267
|
|
|
199
268
|
## Preparing a change
|
|
200
269
|
|
|
@@ -206,8 +275,28 @@ Changes to native protocols, reconstruction, permissions or provider accounting
|
|
|
206
275
|
the affected safety boundary. Mock tests establish local contracts; claims of native support also
|
|
207
276
|
need capability, isolation, calibration and live-use evidence for the target profile.
|
|
208
277
|
|
|
278
|
+
### Pull request review
|
|
279
|
+
|
|
280
|
+
Pull requests are squash-merged, with the PR title becoming the commit subject. Use a Conventional
|
|
281
|
+
Commit title, such as `fix(codex): handle overload responses` or
|
|
282
|
+
`docs: explain confirmation results`. The title check reruns on title edits and new pushes; it reads
|
|
283
|
+
metadata without checking out PR code. Make sure the required CI and CodeQL checks pass and resolve
|
|
284
|
+
review conversations before merging.
|
|
285
|
+
|
|
286
|
+
CodeRabbit reviews PRs when they are ready for review and posts feedback in a walkthrough and
|
|
287
|
+
inline comments. Address relevant findings or explain why a suggestion does not apply. The
|
|
288
|
+
maintainer makes the final review and merge decisions.
|
|
289
|
+
|
|
290
|
+
CodeRabbit's optional planning and code-generation features are configured in
|
|
291
|
+
[`.coderabbit.yaml`](.coderabbit.yaml). A paid CodeRabbit account is not required to contribute.
|
|
292
|
+
|
|
209
293
|
## Preparing a release
|
|
210
294
|
|
|
295
|
+
The release procedures below are for maintainers with repository and package-publishing access.
|
|
296
|
+
|
|
297
|
+
For every feature, aim for parity across supported harnesses. Any exception must be clearly
|
|
298
|
+
justified and documented before release.
|
|
299
|
+
|
|
211
300
|
Python Semantic Release, pinned in the `release` dependency group, calculates versions from
|
|
212
301
|
Conventional Commits. `pyproject.toml` is the version source; installed CLI and Codex client
|
|
213
302
|
metadata read that distribution version. The tool generates `CHANGELOG.md` from the full release
|
|
@@ -256,8 +345,8 @@ ownership and the following repository/index settings:
|
|
|
256
345
|
- Release-tag rules permit the maintainer's signed-tag creation and prevent unapproved tag
|
|
257
346
|
replacement or deletion. The workflow never creates a tag.
|
|
258
347
|
|
|
259
|
-
The
|
|
260
|
-
SHA and exact final `X.Y.Z` version. Replace the placeholders with the reviewed values:
|
|
348
|
+
The release workflow accepts dispatches only from the repository maintainer. Supply the full source
|
|
349
|
+
commit SHA and exact final `X.Y.Z` version. Replace the placeholders with the reviewed values:
|
|
261
350
|
|
|
262
351
|
```bash
|
|
263
352
|
gh workflow run release.yml --repo eberkut/ithrl --ref main \
|
ithrl-0.2.0/PKG-INFO
ADDED
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: ithrl
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Find smaller agent contexts that preserve a selected model behavior
|
|
5
|
+
Project-URL: Source, https://github.com/eberkut/ithrl
|
|
6
|
+
Project-URL: Issues, https://github.com/eberkut/ithrl/issues
|
|
7
|
+
Project-URL: Documentation, https://eberkut.github.io/ithrl/
|
|
8
|
+
Project-URL: Changelog, https://github.com/eberkut/ithrl/releases
|
|
9
|
+
Author: Vincent Morel
|
|
10
|
+
License-Expression: Apache-2.0
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Environment :: Console
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
16
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
20
|
+
Classifier: Topic :: Software Development :: Debuggers
|
|
21
|
+
Requires-Python: >=3.12
|
|
22
|
+
Requires-Dist: filelock>=3.32.4
|
|
23
|
+
Requires-Dist: github-copilot-sdk==1.0.28
|
|
24
|
+
Requires-Dist: jsonschema>=4.25
|
|
25
|
+
Requires-Dist: packaging>=24.2
|
|
26
|
+
Requires-Dist: platformdirs>=4.3
|
|
27
|
+
Requires-Dist: pydantic>=2.12
|
|
28
|
+
Requires-Dist: rich>=14
|
|
29
|
+
Requires-Dist: typer>=0.16
|
|
30
|
+
Provides-Extra: fuzz
|
|
31
|
+
Requires-Dist: scipy>=1.18; extra == 'fuzz'
|
|
32
|
+
Description-Content-Type: text/markdown
|
|
33
|
+
|
|
34
|
+
# ithrl
|
|
35
|
+
|
|
36
|
+
[](https://pypi.org/project/ithrl/)
|
|
37
|
+
[](https://github.com/eberkut/ithrl/actions/workflows/ci.yml)
|
|
38
|
+
|
|
39
|
+
`ithrl` turns saved Codex and GitHub Copilot sessions into debugging experiments. Search for a
|
|
40
|
+
smaller context that preserves a chosen response or tool request, or a small text change that alters
|
|
41
|
+
it.
|
|
42
|
+
|
|
43
|
+
[Documentation](https://eberkut.github.io/ithrl/) · [Try it](#try-it) ·
|
|
44
|
+
[Supported inputs](#supported-inputs)
|
|
45
|
+
|
|
46
|
+
## Install
|
|
47
|
+
|
|
48
|
+
With [uv](https://docs.astral.sh/uv/):
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
uv tool install ithrl
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
You can also install with `pipx install ithrl` or, in an existing virtual environment,
|
|
55
|
+
`python -m pip install ithrl`. For context fuzzing, use `'ithrl[fuzz]'` in place of `ithrl` in
|
|
56
|
+
the installation command.
|
|
57
|
+
|
|
58
|
+
Tested on Python 3.12–3.14. Native runners require Linux. APIs and saved formats may change before
|
|
59
|
+
1.0; keep your source sessions and record the version used for each case.
|
|
60
|
+
|
|
61
|
+
## Try it
|
|
62
|
+
|
|
63
|
+
The bundled deterministic example needs no account, agent installation or model access:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
mkdir -m 700 ithrl-demo
|
|
67
|
+
ithrl extract --example --output ithrl-demo/case
|
|
68
|
+
ithrl shrink ithrl-demo/case --observation-store ithrl-demo/observations.sqlite
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
The result removes half the context items and confirms that the target is preserved. Output excerpt:
|
|
72
|
+
|
|
73
|
+
```text
|
|
74
|
+
Reduced and confirmed
|
|
75
|
+
context items 10 -> 5
|
|
76
|
+
original test target observed
|
|
77
|
+
search 1-minimal; all valid single removals tested
|
|
78
|
+
confirmation target preserved
|
|
79
|
+
observations 15
|
|
80
|
+
No network requests or provider calls. No actions executed.
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Inspect the saved result in `ithrl-demo/case/results/`; the
|
|
84
|
+
[results guide](https://github.com/eberkut/ithrl/blob/main/docs/results.md) explains its files.
|
|
85
|
+
Use a new directory to repeat the example; extraction refuses to overwrite an existing case.
|
|
86
|
+
|
|
87
|
+
## Commands
|
|
88
|
+
|
|
89
|
+
- `ithrl extract` browses saved sessions and creates a case from a selected decision.
|
|
90
|
+
- `ithrl shrink CASE` searches for context removals that preserve the selected behavior.
|
|
91
|
+
- `ithrl fuzz CASE --config PRIVATE_TOML` searches for and simplifies text appends that decrease how
|
|
92
|
+
often the selected behavior occurs, with independent confirmation.
|
|
93
|
+
|
|
94
|
+
For example, extract a saved session file:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
ithrl extract path/to/session.jsonl --output .ithrl/case
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Extraction makes no provider calls. Native experiments require Linux, usable Bubblewrap, an
|
|
101
|
+
installed harness and model access. Review the provider, safety controls and call/time budget in the
|
|
102
|
+
[native-run guide](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md) before execution.
|
|
103
|
+
Returned actions are observed and denied. Cases contain copied session content; keep them private.
|
|
104
|
+
|
|
105
|
+
## Supported inputs
|
|
106
|
+
|
|
107
|
+
| Input | Extracted decisions | Reduction | Fuzzing |
|
|
108
|
+
| --- | --- | --- | --- |
|
|
109
|
+
| Bundled synthetic example | Synthetic tool request | Deterministic local runner | Local fake runner |
|
|
110
|
+
| Codex CLI rollouts | Final responses | Experimental native runner | Experimental prior-user text appends |
|
|
111
|
+
| Copilot CLI session events | Final responses and shell requests | Experimental final-response and denied `bash` runner | Experimental prior-user text appends for final-response targets |
|
|
112
|
+
|
|
113
|
+
Native Codex and Copilot final-response fuzzing supports appending text to eligible prior user
|
|
114
|
+
input while preserving the fixed triggering input. The
|
|
115
|
+
[fuzzing walkthrough](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md#context-fuzzing)
|
|
116
|
+
starts with a local example and covers native execution.
|
|
117
|
+
|
|
118
|
+
[Saved-session measurements](https://github.com/eberkut/ithrl/blob/main/docs/measurements.md#saved-session-experiments)
|
|
119
|
+
include a confirmed reduction for one Codex session and its chosen predicate: 50.17% lower mean
|
|
120
|
+
input tokens per confirmation observation, using a 20-percentage-point preservation margin.
|
|
121
|
+
The measurements report resource usage and unsuccessful outcomes alongside completed runs. Check
|
|
122
|
+
[native compatibility](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md#compatibility-and-measurements)
|
|
123
|
+
for the installed harness's requirements.
|
|
124
|
+
|
|
125
|
+
## Documentation
|
|
126
|
+
|
|
127
|
+
Read the [documentation site](https://eberkut.github.io/ithrl/) or browse the guides below.
|
|
128
|
+
|
|
129
|
+
- [Native runs](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md): session selection,
|
|
130
|
+
experiment configuration, preflight, model execution and measured resource usage.
|
|
131
|
+
- [Results and privacy](https://github.com/eberkut/ithrl/blob/main/docs/results.md): saved files,
|
|
132
|
+
result interpretation, budgets, resume and sensitive data.
|
|
133
|
+
- [Contributing](https://github.com/eberkut/ithrl/blob/main/CONTRIBUTING.md): issue-first
|
|
134
|
+
collaboration, checkout setup, checks and smoke tests.
|
|
135
|
+
- [Roadmap](https://github.com/eberkut/ithrl/blob/main/ROADMAP.md): current priorities and planned
|
|
136
|
+
capabilities.
|
|
137
|
+
- [Architecture](https://github.com/eberkut/ithrl/blob/main/docs/architecture.md): implementation
|
|
138
|
+
boundaries and decisions.
|
|
139
|
+
- [Research](https://github.com/eberkut/ithrl/blob/main/docs/research/index.md): feasibility
|
|
140
|
+
studies, statistical methods and measurements.
|
|
141
|
+
|
|
142
|
+
For help, bugs or proposals, see [Support](https://github.com/eberkut/ithrl/blob/main/SUPPORT.md).
|
|
143
|
+
Report vulnerabilities through
|
|
144
|
+
[private reporting](https://github.com/eberkut/ithrl/security/advisories/new).
|
|
145
|
+
|
|
146
|
+
## Name
|
|
147
|
+
|
|
148
|
+
Pronounce `ithrl` as **Ithuriel**, the angel whose spear reveals Satan in disguise in Milton's
|
|
149
|
+
[*Paradise Lost*](https://rpo.library.utoronto.ca/content/paradise-lost-book-iv-1674).
|
|
150
|
+
|
|
151
|
+
## License
|
|
152
|
+
|
|
153
|
+
Licensed under the [Apache License 2.0](https://github.com/eberkut/ithrl/blob/main/LICENSE).
|
ithrl-0.2.0/README.md
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# ithrl
|
|
2
|
+
|
|
3
|
+
[](https://pypi.org/project/ithrl/)
|
|
4
|
+
[](https://github.com/eberkut/ithrl/actions/workflows/ci.yml)
|
|
5
|
+
|
|
6
|
+
`ithrl` turns saved Codex and GitHub Copilot sessions into debugging experiments. Search for a
|
|
7
|
+
smaller context that preserves a chosen response or tool request, or a small text change that alters
|
|
8
|
+
it.
|
|
9
|
+
|
|
10
|
+
[Documentation](https://eberkut.github.io/ithrl/) · [Try it](#try-it) ·
|
|
11
|
+
[Supported inputs](#supported-inputs)
|
|
12
|
+
|
|
13
|
+
## Install
|
|
14
|
+
|
|
15
|
+
With [uv](https://docs.astral.sh/uv/):
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
uv tool install ithrl
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
You can also install with `pipx install ithrl` or, in an existing virtual environment,
|
|
22
|
+
`python -m pip install ithrl`. For context fuzzing, use `'ithrl[fuzz]'` in place of `ithrl` in
|
|
23
|
+
the installation command.
|
|
24
|
+
|
|
25
|
+
Tested on Python 3.12–3.14. Native runners require Linux. APIs and saved formats may change before
|
|
26
|
+
1.0; keep your source sessions and record the version used for each case.
|
|
27
|
+
|
|
28
|
+
## Try it
|
|
29
|
+
|
|
30
|
+
The bundled deterministic example needs no account, agent installation or model access:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
mkdir -m 700 ithrl-demo
|
|
34
|
+
ithrl extract --example --output ithrl-demo/case
|
|
35
|
+
ithrl shrink ithrl-demo/case --observation-store ithrl-demo/observations.sqlite
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
The result removes half the context items and confirms that the target is preserved. Output excerpt:
|
|
39
|
+
|
|
40
|
+
```text
|
|
41
|
+
Reduced and confirmed
|
|
42
|
+
context items 10 -> 5
|
|
43
|
+
original test target observed
|
|
44
|
+
search 1-minimal; all valid single removals tested
|
|
45
|
+
confirmation target preserved
|
|
46
|
+
observations 15
|
|
47
|
+
No network requests or provider calls. No actions executed.
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Inspect the saved result in `ithrl-demo/case/results/`; the
|
|
51
|
+
[results guide](https://github.com/eberkut/ithrl/blob/main/docs/results.md) explains its files.
|
|
52
|
+
Use a new directory to repeat the example; extraction refuses to overwrite an existing case.
|
|
53
|
+
|
|
54
|
+
## Commands
|
|
55
|
+
|
|
56
|
+
- `ithrl extract` browses saved sessions and creates a case from a selected decision.
|
|
57
|
+
- `ithrl shrink CASE` searches for context removals that preserve the selected behavior.
|
|
58
|
+
- `ithrl fuzz CASE --config PRIVATE_TOML` searches for and simplifies text appends that decrease how
|
|
59
|
+
often the selected behavior occurs, with independent confirmation.
|
|
60
|
+
|
|
61
|
+
For example, extract a saved session file:
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
ithrl extract path/to/session.jsonl --output .ithrl/case
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Extraction makes no provider calls. Native experiments require Linux, usable Bubblewrap, an
|
|
68
|
+
installed harness and model access. Review the provider, safety controls and call/time budget in the
|
|
69
|
+
[native-run guide](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md) before execution.
|
|
70
|
+
Returned actions are observed and denied. Cases contain copied session content; keep them private.
|
|
71
|
+
|
|
72
|
+
## Supported inputs
|
|
73
|
+
|
|
74
|
+
| Input | Extracted decisions | Reduction | Fuzzing |
|
|
75
|
+
| --- | --- | --- | --- |
|
|
76
|
+
| Bundled synthetic example | Synthetic tool request | Deterministic local runner | Local fake runner |
|
|
77
|
+
| Codex CLI rollouts | Final responses | Experimental native runner | Experimental prior-user text appends |
|
|
78
|
+
| Copilot CLI session events | Final responses and shell requests | Experimental final-response and denied `bash` runner | Experimental prior-user text appends for final-response targets |
|
|
79
|
+
|
|
80
|
+
Native Codex and Copilot final-response fuzzing supports appending text to eligible prior user
|
|
81
|
+
input while preserving the fixed triggering input. The
|
|
82
|
+
[fuzzing walkthrough](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md#context-fuzzing)
|
|
83
|
+
starts with a local example and covers native execution.
|
|
84
|
+
|
|
85
|
+
[Saved-session measurements](https://github.com/eberkut/ithrl/blob/main/docs/measurements.md#saved-session-experiments)
|
|
86
|
+
include a confirmed reduction for one Codex session and its chosen predicate: 50.17% lower mean
|
|
87
|
+
input tokens per confirmation observation, using a 20-percentage-point preservation margin.
|
|
88
|
+
The measurements report resource usage and unsuccessful outcomes alongside completed runs. Check
|
|
89
|
+
[native compatibility](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md#compatibility-and-measurements)
|
|
90
|
+
for the installed harness's requirements.
|
|
91
|
+
|
|
92
|
+
## Documentation
|
|
93
|
+
|
|
94
|
+
Read the [documentation site](https://eberkut.github.io/ithrl/) or browse the guides below.
|
|
95
|
+
|
|
96
|
+
- [Native runs](https://github.com/eberkut/ithrl/blob/main/docs/native-runs.md): session selection,
|
|
97
|
+
experiment configuration, preflight, model execution and measured resource usage.
|
|
98
|
+
- [Results and privacy](https://github.com/eberkut/ithrl/blob/main/docs/results.md): saved files,
|
|
99
|
+
result interpretation, budgets, resume and sensitive data.
|
|
100
|
+
- [Contributing](https://github.com/eberkut/ithrl/blob/main/CONTRIBUTING.md): issue-first
|
|
101
|
+
collaboration, checkout setup, checks and smoke tests.
|
|
102
|
+
- [Roadmap](https://github.com/eberkut/ithrl/blob/main/ROADMAP.md): current priorities and planned
|
|
103
|
+
capabilities.
|
|
104
|
+
- [Architecture](https://github.com/eberkut/ithrl/blob/main/docs/architecture.md): implementation
|
|
105
|
+
boundaries and decisions.
|
|
106
|
+
- [Research](https://github.com/eberkut/ithrl/blob/main/docs/research/index.md): feasibility
|
|
107
|
+
studies, statistical methods and measurements.
|
|
108
|
+
|
|
109
|
+
For help, bugs or proposals, see [Support](https://github.com/eberkut/ithrl/blob/main/SUPPORT.md).
|
|
110
|
+
Report vulnerabilities through
|
|
111
|
+
[private reporting](https://github.com/eberkut/ithrl/security/advisories/new).
|
|
112
|
+
|
|
113
|
+
## Name
|
|
114
|
+
|
|
115
|
+
Pronounce `ithrl` as **Ithuriel**, the angel whose spear reveals Satan in disguise in Milton's
|
|
116
|
+
[*Paradise Lost*](https://rpo.library.utoronto.ca/content/paradise-lost-book-iv-1674).
|
|
117
|
+
|
|
118
|
+
## License
|
|
119
|
+
|
|
120
|
+
Licensed under the [Apache License 2.0](https://github.com/eberkut/ithrl/blob/main/LICENSE).
|