termverify 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,125 @@
1
+ Metadata-Version: 2.4
2
+ Name: termverify
3
+ Version: 0.1.0
4
+ Summary: Protocol-driven verification for autonomous terminal applications.
5
+ Keywords: agents,testing,terminal,tui,verification
6
+ Author: Matthias Hölzl
7
+ Author-email: Matthias Hölzl <tc@xantira.com>
8
+ License-Expression: Apache-2.0
9
+ Classifier: Development Status :: 2 - Pre-Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Operating System :: OS Independent
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3.12
14
+ Classifier: Programming Language :: Python :: 3.13
15
+ Classifier: Programming Language :: Python :: 3.14
16
+ Classifier: Topic :: Software Development :: Testing
17
+ Requires-Dist: pywinpty>=3.0.5,<4.0 ; sys_platform == 'win32'
18
+ Requires-Dist: rfc8785>=0.1.4
19
+ Requires-Python: >=3.12
20
+ Project-URL: Homepage, https://github.com/hoelzl/termverify
21
+ Project-URL: Repository, https://github.com/hoelzl/termverify
22
+ Project-URL: Issues, https://github.com/hoelzl/termverify/issues
23
+ Description-Content-Type: text/markdown
24
+
25
+ # TermVerify
26
+
27
+ Protocol-driven verification for autonomous terminal applications.
28
+
29
+ TermVerify is a Python library and reference toolkit for testing terminal and TUI applications through reproducible interaction. It combines direct semantic adapters with real PTY-driven runs, then compares structured observations rather than relying only on brittle raw-terminal snapshots.
30
+
31
+ ## Why
32
+
33
+ An autonomous coding agent is most reliable when it can make a change, exercise the actual program, observe meaningful results, and leave behind replayable evidence. Terminal applications need this especially badly: unit tests alone do not prove that key bindings, focus, rendering, prompts, and real interaction work.
34
+
35
+ TermVerify provides a common foundation for:
36
+
37
+ - deterministic interaction transcripts and replay;
38
+ - semantic state and UI observations;
39
+ - property/state-machine testing;
40
+ - reviewed golden snapshots;
41
+ - differential tests against a reference implementation or execution mode;
42
+ - failure minimization and CI-ready artifacts.
43
+
44
+ ## Project status
45
+
46
+ The repository is in its foundation phase. The reviewed transcript design has
47
+ an initial codec, semantic validator, mandatory safe transcript-persistence
48
+ boundary, immutable producer-side adapter contract, and deterministic in-process
49
+ direct runtime. Requested timezone names and semantic key chords use closed,
50
+ protocol-owned v1 registries; semantic keys also have an immutable direct-dispatch
51
+ representation. The canonical transcript schema ships inside the package with a
52
+ public access API, and isolated installation checks verify the wheel and sdist
53
+ resource contract; the schema's `$id` resolves at
54
+ [termverify.dev](https://termverify.dev/schemas/termverify.transcript/v1.schema.json)
55
+ as a byte-identical mirror of the committed resource, and runtime validation
56
+ remains authoritative. A strict no-regression coverage floor gates the full
57
+ suite. Release governance is defined — changelog and pre-1.0 compatibility
58
+ policy, private security disclosure, reviewed release checklist, and a
59
+ tag-triggered attested draft-artifact workflow — although no release is
60
+ authorized and nothing is published to a package index. A Windows ConPTY
61
+ adapter with Windows-matrix evidence covers native pseudoconsole ownership
62
+ and close, end-of-stream draining, process-tree teardown,
63
+ cancellation/recovery, resize epochs, and replayable evidence normalization,
64
+ and the first fully verified terminal run has landed using opt-in
65
+ cooperation-tier constraint ports: the six non-terminal constraints are
66
+ delivered to the subject's environment with truthful `delivered` receipts,
67
+ honored by subject cooperation rather than OS enforcement. OS-level
68
+ containment is an explicit non-goal by recorded owner decision; TermVerify
69
+ verifies applications whose authors control the subject and is not an
70
+ execution sandbox for adversarial code. Configuration values or receipt
71
+ construction alone do not prove constraint enforcement.
72
+
73
+ ## Design principles
74
+
75
+ 1. **Semantic evidence first.** Verify state, events, and explicit UI semantics before comparing raw ANSI output.
76
+ 2. **Production interaction still matters.** PTY/terminal tests validate the application a person or agent actually drives.
77
+ 3. **Determinism is a contract.** Seeds, clock, locale, terminal size, filesystem sandbox, and network policy are explicit.
78
+ 4. **Human review owns baselines.** Agents may propose snapshot updates; they never silently bless them.
79
+ 5. **Harness-neutral by default.** The project works with Hermes, Claude Code, Codex, OpenCode, and ordinary CI without a required proprietary integration.
80
+
81
+ ## Planned architecture
82
+
83
+ ```text
84
+ application under test
85
+ ├── direct semantic adapter ── fast properties, replay, differential tests
86
+ └── PTY adapter ───────────── real terminal interaction and rendering evidence
87
+
88
+ TermVerify
89
+ ├── run configuration and interaction protocol
90
+ ├── observation normalization and comparison
91
+ ├── transcript replay and shrinking
92
+ ├── property/state-machine support
93
+ └── reports and CI artifacts
94
+ ```
95
+
96
+ See [the knowledge bundle](docs/knowledge/index.md) for the durable architecture and verification model.
97
+ Browser bridging is deferred until the terminal vertical slice proves a shared
98
+ abstraction is necessary.
99
+
100
+ ## Development
101
+
102
+ Requirements: [uv](https://docs.astral.sh/uv/) and Python 3.12 or newer. The
103
+ minimum installer version is 3.12; the continuously supported and tested
104
+ versions are currently 3.12 through 3.14. Support for later Python releases is
105
+ not implied until they join the CI matrix.
106
+
107
+ ```bash
108
+ uv --no-config sync --all-groups --locked
109
+ uv --no-config run pytest --cov --cov-report=term-missing
110
+ uv --no-config run ruff check .
111
+ uv --no-config run ruff format --check .
112
+ uv --no-config run mypy src tests scripts
113
+ uv --no-config run pre-commit run --all-files
114
+ uv --no-config run pre-commit run --hook-stage pre-push --all-files
115
+ uv --no-config build
116
+ uv --no-config run pre-commit install --hook-type pre-commit --hook-type pre-push
117
+ ```
118
+
119
+ See [developer workflow](docs/developer-guide/agent-workflow.md) and [contributing guide](CONTRIBUTING.md).
120
+ External subjects implementing the producer contract start with the
121
+ [adapter-author surface](docs/developer-guide/adapter-authors.md).
122
+
123
+ ## License
124
+
125
+ Apache License 2.0. See [LICENSE](LICENSE).
@@ -0,0 +1,101 @@
1
+ # TermVerify
2
+
3
+ Protocol-driven verification for autonomous terminal applications.
4
+
5
+ TermVerify is a Python library and reference toolkit for testing terminal and TUI applications through reproducible interaction. It combines direct semantic adapters with real PTY-driven runs, then compares structured observations rather than relying only on brittle raw-terminal snapshots.
6
+
7
+ ## Why
8
+
9
+ An autonomous coding agent is most reliable when it can make a change, exercise the actual program, observe meaningful results, and leave behind replayable evidence. Terminal applications need this especially badly: unit tests alone do not prove that key bindings, focus, rendering, prompts, and real interaction work.
10
+
11
+ TermVerify provides a common foundation for:
12
+
13
+ - deterministic interaction transcripts and replay;
14
+ - semantic state and UI observations;
15
+ - property/state-machine testing;
16
+ - reviewed golden snapshots;
17
+ - differential tests against a reference implementation or execution mode;
18
+ - failure minimization and CI-ready artifacts.
19
+
20
+ ## Project status
21
+
22
+ The repository is in its foundation phase. The reviewed transcript design has
23
+ an initial codec, semantic validator, mandatory safe transcript-persistence
24
+ boundary, immutable producer-side adapter contract, and deterministic in-process
25
+ direct runtime. Requested timezone names and semantic key chords use closed,
26
+ protocol-owned v1 registries; semantic keys also have an immutable direct-dispatch
27
+ representation. The canonical transcript schema ships inside the package with a
28
+ public access API, and isolated installation checks verify the wheel and sdist
29
+ resource contract; the schema's `$id` resolves at
30
+ [termverify.dev](https://termverify.dev/schemas/termverify.transcript/v1.schema.json)
31
+ as a byte-identical mirror of the committed resource, and runtime validation
32
+ remains authoritative. A strict no-regression coverage floor gates the full
33
+ suite. Release governance is defined — changelog and pre-1.0 compatibility
34
+ policy, private security disclosure, reviewed release checklist, and a
35
+ tag-triggered attested draft-artifact workflow — although no release is
36
+ authorized and nothing is published to a package index. A Windows ConPTY
37
+ adapter with Windows-matrix evidence covers native pseudoconsole ownership
38
+ and close, end-of-stream draining, process-tree teardown,
39
+ cancellation/recovery, resize epochs, and replayable evidence normalization,
40
+ and the first fully verified terminal run has landed using opt-in
41
+ cooperation-tier constraint ports: the six non-terminal constraints are
42
+ delivered to the subject's environment with truthful `delivered` receipts,
43
+ honored by subject cooperation rather than OS enforcement. OS-level
44
+ containment is an explicit non-goal by recorded owner decision; TermVerify
45
+ verifies applications whose authors control the subject and is not an
46
+ execution sandbox for adversarial code. Configuration values or receipt
47
+ construction alone do not prove constraint enforcement.
48
+
49
+ ## Design principles
50
+
51
+ 1. **Semantic evidence first.** Verify state, events, and explicit UI semantics before comparing raw ANSI output.
52
+ 2. **Production interaction still matters.** PTY/terminal tests validate the application a person or agent actually drives.
53
+ 3. **Determinism is a contract.** Seeds, clock, locale, terminal size, filesystem sandbox, and network policy are explicit.
54
+ 4. **Human review owns baselines.** Agents may propose snapshot updates; they never silently bless them.
55
+ 5. **Harness-neutral by default.** The project works with Hermes, Claude Code, Codex, OpenCode, and ordinary CI without a required proprietary integration.
56
+
57
+ ## Planned architecture
58
+
59
+ ```text
60
+ application under test
61
+ ├── direct semantic adapter ── fast properties, replay, differential tests
62
+ └── PTY adapter ───────────── real terminal interaction and rendering evidence
63
+
64
+ TermVerify
65
+ ├── run configuration and interaction protocol
66
+ ├── observation normalization and comparison
67
+ ├── transcript replay and shrinking
68
+ ├── property/state-machine support
69
+ └── reports and CI artifacts
70
+ ```
71
+
72
+ See [the knowledge bundle](docs/knowledge/index.md) for the durable architecture and verification model.
73
+ Browser bridging is deferred until the terminal vertical slice proves a shared
74
+ abstraction is necessary.
75
+
76
+ ## Development
77
+
78
+ Requirements: [uv](https://docs.astral.sh/uv/) and Python 3.12 or newer. The
79
+ minimum installer version is 3.12; the continuously supported and tested
80
+ versions are currently 3.12 through 3.14. Support for later Python releases is
81
+ not implied until they join the CI matrix.
82
+
83
+ ```bash
84
+ uv --no-config sync --all-groups --locked
85
+ uv --no-config run pytest --cov --cov-report=term-missing
86
+ uv --no-config run ruff check .
87
+ uv --no-config run ruff format --check .
88
+ uv --no-config run mypy src tests scripts
89
+ uv --no-config run pre-commit run --all-files
90
+ uv --no-config run pre-commit run --hook-stage pre-push --all-files
91
+ uv --no-config build
92
+ uv --no-config run pre-commit install --hook-type pre-commit --hook-type pre-push
93
+ ```
94
+
95
+ See [developer workflow](docs/developer-guide/agent-workflow.md) and [contributing guide](CONTRIBUTING.md).
96
+ External subjects implementing the producer contract start with the
97
+ [adapter-author surface](docs/developer-guide/adapter-authors.md).
98
+
99
+ ## License
100
+
101
+ Apache License 2.0. See [LICENSE](LICENSE).
@@ -0,0 +1,128 @@
1
+ [project]
2
+ name = "termverify"
3
+ version = "0.1.0"
4
+ description = "Protocol-driven verification for autonomous terminal applications."
5
+ readme = "README.md"
6
+ requires-python = ">=3.12"
7
+ license = "Apache-2.0"
8
+ authors = [
9
+ { name = "Matthias Hölzl", email = "tc@xantira.com" },
10
+ ]
11
+ keywords = ["agents", "testing", "terminal", "tui", "verification"]
12
+ classifiers = [
13
+ "Development Status :: 2 - Pre-Alpha",
14
+ "Intended Audience :: Developers",
15
+ "Operating System :: OS Independent",
16
+ "Programming Language :: Python :: 3",
17
+ "Programming Language :: Python :: 3.12",
18
+ "Programming Language :: Python :: 3.13",
19
+ "Programming Language :: Python :: 3.14",
20
+ "Topic :: Software Development :: Testing",
21
+ ]
22
+ dependencies = [
23
+ "pywinpty>=3.0.5,<4.0; sys_platform == 'win32'",
24
+ "rfc8785>=0.1.4",
25
+ ]
26
+
27
+ [project.urls]
28
+ Homepage = "https://github.com/hoelzl/termverify"
29
+ Repository = "https://github.com/hoelzl/termverify"
30
+ Issues = "https://github.com/hoelzl/termverify/issues"
31
+
32
+ [dependency-groups]
33
+ dev = [
34
+ "bump-my-version>=1.2.4",
35
+ "hypothesis>=6.156.2",
36
+ "jsonschema>=4.23",
37
+ "mypy>=1.0",
38
+ "pre-commit>=4.0",
39
+ "pytest>=8.0",
40
+ "pytest-cov>=5.0",
41
+ "pyyaml>=6.0",
42
+ "ruff>=0.11",
43
+ "types-jsonschema>=4.23",
44
+ "types-pyyaml>=6.0",
45
+ ]
46
+ docs = [
47
+ "mkdocs>=1.6.1",
48
+ "mkdocs-material>=9.7.7",
49
+ ]
50
+
51
+ [build-system]
52
+ requires = ["uv_build>=0.11.28,<0.12.0"]
53
+ build-backend = "uv_build"
54
+
55
+ [tool.pytest.ini_options]
56
+ addopts = "-ra"
57
+ testpaths = ["tests"]
58
+
59
+ [tool.coverage.run]
60
+ branch = true
61
+ source = ["termverify"]
62
+ # The native ConPTY binding executes only on the Windows CI legs; including it
63
+ # would make the cross-platform ratchet floor depend on the host OS. It stays
64
+ # deliberately thin, its behavior is proven by the Windows-only integration
65
+ # tests, and all adapter logic above it must remain fully ratcheted
66
+ # (docs/developer-guide/development.md).
67
+ omit = ["*/termverify/_conpty.py"]
68
+
69
+ [tool.coverage.report]
70
+ show_missing = true
71
+ skip_covered = true
72
+ # No-regression ratchet floor: integer floor of the reviewed observed total
73
+ # (94.43% on 2026-07-17). Raise only for sustained durable coverage; lowering
74
+ # requires explicit owner review (docs/developer-guide/development.md).
75
+ # precision = 2 makes the comparison a strict floor instead of granting a
76
+ # hidden half-point of integer rounding grace.
77
+ fail_under = 94
78
+ precision = 2
79
+
80
+ [tool.ruff]
81
+ line-length = 88
82
+ target-version = "py312"
83
+
84
+ [tool.ruff.lint]
85
+ select = ["E", "F", "I", "UP", "B", "SIM"]
86
+
87
+ [tool.mypy]
88
+ files = ["src", "tests", "scripts"]
89
+ python_version = "3.12"
90
+ strict = true
91
+ warn_unreachable = true
92
+
93
+ [[tool.mypy.overrides]]
94
+ # pywinpty ships no type stubs and is absent on non-Windows checkers; the
95
+ # binding wrapper confines its untyped surface to termverify._conpty.
96
+ module = "winpty.*"
97
+ ignore_missing_imports = true
98
+
99
+ [tool.bumpversion]
100
+ current_version = "0.1.0"
101
+ commit = true
102
+ # No local tag: the Release workflow (.github/workflows/release.yml) creates the
103
+ # authoritative vX.Y.Z tag on the server *after* the CI-green gate, so a red
104
+ # commit is never tagged. bump-my-version only produces the `Bump version …`
105
+ # commit that the workflow keys on. (tag_name documents the format the server
106
+ # uses; it is inert while tag = false.)
107
+ tag = false
108
+ tag_name = "v{new_version}"
109
+ commit_message = "Bump version {current_version} → {new_version}"
110
+
111
+ [[tool.bumpversion.files]]
112
+ # pyproject.toml is the single version source of truth: [project] version is
113
+ # read directly by the Release workflow's resolve job, and `__version__` is
114
+ # derived from the installed distribution metadata at runtime.
115
+ filename = "pyproject.toml"
116
+ search = 'version = "{current_version}"'
117
+ replace = 'version = "{new_version}"'
118
+
119
+ [[tool.bumpversion.files]]
120
+ # uv.lock records the project's own version in its [[package]] block. If it is
121
+ # not updated here, the next `uv sync --locked` fails or silently re-locks.
122
+ # Scope the replacement to the project's own block via the unique two-line
123
+ # name+version pair.
124
+ filename = "uv.lock"
125
+ search = """name = "termverify"
126
+ version = "{current_version}\""""
127
+ replace = """name = "termverify"
128
+ version = "{new_version}\""""
@@ -0,0 +1,137 @@
1
+ """Protocol-driven verification for autonomous terminal applications.
2
+
3
+ The names re-exported here are the curated public surface. Adapter authors
4
+ implement the contract defined in :mod:`termverify.adapter` and, for
5
+ in-process subjects, :mod:`termverify.direct`; every contract name is
6
+ importable from ``termverify`` directly and is identical to its module-path
7
+ definition, so both import styles stay interchangeable. The surface is
8
+ pre-1.0: compatibility intent and changes are recorded in ``CHANGELOG.md``.
9
+ """
10
+
11
+ from importlib.metadata import version
12
+
13
+ from termverify.adapter import (
14
+ ENFORCEMENT_TIERS,
15
+ Adapter,
16
+ AdapterFailure,
17
+ ClockAdvance,
18
+ ClockConfiguration,
19
+ ClockReceipt,
20
+ ConstraintName,
21
+ ConstraintPorts,
22
+ ConstraintUnsupported,
23
+ Cursor,
24
+ DeliveryRecord,
25
+ Diagnostic,
26
+ DispatchInput,
27
+ EnforcedConstraints,
28
+ EnforcementReceipt,
29
+ EnforcementTier,
30
+ EpochCompleted,
31
+ EpochResult,
32
+ Event,
33
+ ExitStatus,
34
+ FilesystemConfiguration,
35
+ FilesystemReceipt,
36
+ Frame,
37
+ FrozenJsonValue,
38
+ JsonInput,
39
+ KeyInput,
40
+ LocaleReceipt,
41
+ ManualTime,
42
+ NetworkConfiguration,
43
+ NetworkEndpoint,
44
+ NetworkReceipt,
45
+ Observation,
46
+ ProcessObservation,
47
+ Region,
48
+ Resize,
49
+ RunConfiguration,
50
+ RunFailed,
51
+ RunFinished,
52
+ SeedReceipt,
53
+ Started,
54
+ StartFailed,
55
+ StartResult,
56
+ StartTerminated,
57
+ StartUnsupported,
58
+ Stop,
59
+ TerminalConfiguration,
60
+ TerminalReceipt,
61
+ TerminalResult,
62
+ TextInput,
63
+ TimezoneReceipt,
64
+ UiObservation,
65
+ freeze_json,
66
+ )
67
+ from termverify.direct import DirectAdapter, DirectApplication
68
+ from termverify.evidence import persist_transcript_evidence
69
+ from termverify.schema import (
70
+ TRANSCRIPT_SCHEMA_V1_ID,
71
+ transcript_schema_v1_bytes,
72
+ transcript_schema_v1_json,
73
+ )
74
+
75
+ __all__ = [
76
+ "Adapter",
77
+ "AdapterFailure",
78
+ "ClockAdvance",
79
+ "ClockConfiguration",
80
+ "ClockReceipt",
81
+ "ConstraintName",
82
+ "ConstraintPorts",
83
+ "ConstraintUnsupported",
84
+ "Cursor",
85
+ "DeliveryRecord",
86
+ "Diagnostic",
87
+ "DirectAdapter",
88
+ "DirectApplication",
89
+ "DispatchInput",
90
+ "ENFORCEMENT_TIERS",
91
+ "EnforcedConstraints",
92
+ "EnforcementReceipt",
93
+ "EnforcementTier",
94
+ "EpochCompleted",
95
+ "EpochResult",
96
+ "Event",
97
+ "ExitStatus",
98
+ "FilesystemConfiguration",
99
+ "FilesystemReceipt",
100
+ "Frame",
101
+ "FrozenJsonValue",
102
+ "JsonInput",
103
+ "KeyInput",
104
+ "LocaleReceipt",
105
+ "ManualTime",
106
+ "NetworkConfiguration",
107
+ "NetworkEndpoint",
108
+ "NetworkReceipt",
109
+ "Observation",
110
+ "ProcessObservation",
111
+ "Region",
112
+ "Resize",
113
+ "RunConfiguration",
114
+ "RunFailed",
115
+ "RunFinished",
116
+ "SeedReceipt",
117
+ "StartFailed",
118
+ "StartResult",
119
+ "StartTerminated",
120
+ "StartUnsupported",
121
+ "Started",
122
+ "Stop",
123
+ "TRANSCRIPT_SCHEMA_V1_ID",
124
+ "TerminalConfiguration",
125
+ "TerminalReceipt",
126
+ "TerminalResult",
127
+ "TextInput",
128
+ "TimezoneReceipt",
129
+ "UiObservation",
130
+ "__version__",
131
+ "freeze_json",
132
+ "persist_transcript_evidence",
133
+ "transcript_schema_v1_bytes",
134
+ "transcript_schema_v1_json",
135
+ ]
136
+
137
+ __version__ = version("termverify")