controlled-text-transfer 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- controlled_text_transfer-0.1.0/.cttignore.example +12 -0
- controlled_text_transfer-0.1.0/CHANGELOG.md +85 -0
- controlled_text_transfer-0.1.0/LICENSE +7 -0
- controlled_text_transfer-0.1.0/MANIFEST.in +4 -0
- controlled_text_transfer-0.1.0/PKG-INFO +223 -0
- controlled_text_transfer-0.1.0/README-quickstart.md +111 -0
- controlled_text_transfer-0.1.0/README.md +187 -0
- controlled_text_transfer-0.1.0/SECURITY.md +102 -0
- controlled_text_transfer-0.1.0/ctt.spec +28 -0
- controlled_text_transfer-0.1.0/ctt.yaml.example +18 -0
- controlled_text_transfer-0.1.0/docs/README.md +16 -0
- controlled_text_transfer-0.1.0/docs/api.md +53 -0
- controlled_text_transfer-0.1.0/docs/cli.md +89 -0
- controlled_text_transfer-0.1.0/docs/decisions/0001-reversible-text-only-transfer.md +58 -0
- controlled_text_transfer-0.1.0/docs/decisions/0002-allowlist-manifest-and-integrity-model.md +63 -0
- controlled_text_transfer-0.1.0/docs/decisions/0003-packaging-and-signing-boundaries.md +57 -0
- controlled_text_transfer-0.1.0/docs/decisions/0004-python-packaging-and-quality-gates.md +55 -0
- controlled_text_transfer-0.1.0/docs/decisions/0005-preflight-atomicity-and-trusted-ingestion.md +66 -0
- controlled_text_transfer-0.1.0/docs/decisions/0006-single-artifact-publication-and-signature-verification.md +51 -0
- controlled_text_transfer-0.1.0/docs/decisions/0007-remove-unused-worker-policy.md +50 -0
- controlled_text_transfer-0.1.0/docs/decisions/0008-controlled-text-transfer-name.md +51 -0
- controlled_text_transfer-0.1.0/docs/decisions/0009-python-312-minimum.md +45 -0
- controlled_text_transfer-0.1.0/docs/decisions/0010-offline-report-dashboard.md +84 -0
- controlled_text_transfer-0.1.0/docs/decisions/0011-fail-closed-ingestion-and-atomic-restoration.md +65 -0
- controlled_text_transfer-0.1.0/docs/decisions/0012-harden-github-automation.md +58 -0
- controlled_text_transfer-0.1.0/docs/decisions/README.md +23 -0
- controlled_text_transfer-0.1.0/docs/operations.md +118 -0
- controlled_text_transfer-0.1.0/docs/policy.md +142 -0
- controlled_text_transfer-0.1.0/pyproject.toml +65 -0
- controlled_text_transfer-0.1.0/scripts/README.md +58 -0
- controlled_text_transfer-0.1.0/scripts/__init__.py +1 -0
- controlled_text_transfer-0.1.0/scripts/check_release.py +90 -0
- controlled_text_transfer-0.1.0/scripts/clean.py +12 -0
- controlled_text_transfer-0.1.0/scripts/report.py +796 -0
- controlled_text_transfer-0.1.0/scripts/run.sh +87 -0
- controlled_text_transfer-0.1.0/setup.cfg +4 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/__init__.py +3 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/__main__.py +8 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/cleanup.py +179 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/cli.py +221 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/core.py +1146 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer/signing.py +112 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/PKG-INFO +223 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/SOURCES.txt +62 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/dependency_links.txt +1 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/entry_points.txt +2 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/requires.txt +15 -0
- controlled_text_transfer-0.1.0/src/controlled_text_transfer.egg-info/top_level.txt +1 -0
- controlled_text_transfer-0.1.0/tests/README.md +22 -0
- controlled_text_transfer-0.1.0/tests/test_archive_security.py +269 -0
- controlled_text_transfer-0.1.0/tests/test_clean.py +283 -0
- controlled_text_transfer-0.1.0/tests/test_cli_and_packaging.py +251 -0
- controlled_text_transfer-0.1.0/tests/test_cli_options.py +274 -0
- controlled_text_transfer-0.1.0/tests/test_core.py +163 -0
- controlled_text_transfer-0.1.0/tests/test_documentation_links.py +126 -0
- controlled_text_transfer-0.1.0/tests/test_package_name.py +44 -0
- controlled_text_transfer-0.1.0/tests/test_policy_and_transformation.py +245 -0
- controlled_text_transfer-0.1.0/tests/test_policy_documentation.py +43 -0
- controlled_text_transfer-0.1.0/tests/test_preflight.py +381 -0
- controlled_text_transfer-0.1.0/tests/test_release.py +210 -0
- controlled_text_transfer-0.1.0/tests/test_report.py +369 -0
- controlled_text_transfer-0.1.0/tests/test_run_script.py +175 -0
- controlled_text_transfer-0.1.0/tests/test_security_invariants.py +505 -0
- controlled_text_transfer-0.1.0/tests/test_signing.py +234 -0
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## [Unreleased]
|
|
4
|
+
|
|
5
|
+
## [0.1.0] - 2026-07-25
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- `prepare`, `verify`, `restore`, and read-only `diff` commands.
|
|
10
|
+
- Strict YAML policies and `.cttignore` filtering; unknown policy fields,
|
|
11
|
+
including `workers`, fail validation.
|
|
12
|
+
- UTF-8 validation, optional transfer BOMs, and original BOM restoration.
|
|
13
|
+
- Allowlist-based text selection and binary/oversize reporting.
|
|
14
|
+
- Deterministic manifest generation.
|
|
15
|
+
- SHA-256 and SHA-512 hashing with optional BLAKE3.
|
|
16
|
+
- Directory, ZIP, TAR, and TAR.GZ package output.
|
|
17
|
+
- Structured JSON audit logging and dry-run modes.
|
|
18
|
+
- Detached signing hooks for externally managed GPG/X.509 workflows.
|
|
19
|
+
- Named compatibility preflight with deterministic JSON decisions, strict
|
|
20
|
+
policy parsing, content checks, aggregate limits, and strict preparation.
|
|
21
|
+
- Guarded repository cleanup with dry-run reporting and optional environment
|
|
22
|
+
removal, including source-checkout operation through an external Python
|
|
23
|
+
interpreter.
|
|
24
|
+
- Stable source capture, staged self-verification, and atomic package
|
|
25
|
+
publication; archive formats publish only the requested archive.
|
|
26
|
+
- Safe direct verification and restoration of ZIP, TAR, and TAR.GZ packages.
|
|
27
|
+
- Enforceable detached manifest signatures using trusted injected signers,
|
|
28
|
+
with trusted verification required by default for signed packages.
|
|
29
|
+
- Pytest coverage, Ruff, Black, MyPy, Bandit, GitHub Actions, PyInstaller,
|
|
30
|
+
release-metadata validation, and distribution checks. The suite enforces
|
|
31
|
+
100% application statement coverage, MyPy checks unreachable code, and
|
|
32
|
+
Bandit scans application and development scripts.
|
|
33
|
+
- A `scripts/run.sh` development dispatcher with setup, test, check, report,
|
|
34
|
+
build, and guarded cleanup commands. The report command generates indexed
|
|
35
|
+
pydoc, test, coverage, static-analysis, security, dependency, and runtime
|
|
36
|
+
reports.
|
|
37
|
+
- Command-scoped CLI options, descriptive built-in help, and executable
|
|
38
|
+
contracts for every advertised option.
|
|
39
|
+
- The Controlled Text Transfer identity: distribution
|
|
40
|
+
`controlled-text-transfer`, package `controlled_text_transfer`, command
|
|
41
|
+
`ctt`, policy `ctt.yaml`, ignore file `.cttignore`, and manifest prefix
|
|
42
|
+
`ctt-manifest`.
|
|
43
|
+
- Python 3.12 as the minimum supported runtime, including native
|
|
44
|
+
junction-safe filesystem inspection.
|
|
45
|
+
- CI using locked dependencies and the canonical quality gate on pushes to
|
|
46
|
+
`dev` and `main` and on pull requests targeting `main`.
|
|
47
|
+
- Weekly Dependabot maintenance for commit-SHA-pinned GitHub Actions.
|
|
48
|
+
- Publication metadata linking to the `dgomez407/ctt` repository.
|
|
49
|
+
- Fail-closed rejection of unsupported package formats before output is
|
|
50
|
+
written.
|
|
51
|
+
- Controlled library and CLI errors for malformed manifests.
|
|
52
|
+
- Publication documentation, ADRs, security guidance, and configuration
|
|
53
|
+
examples in the source distribution.
|
|
54
|
+
- An offline, responsive report dashboard with test and coverage metrics,
|
|
55
|
+
accessible status presentation, formatted raw-output views, and navigation
|
|
56
|
+
from generated API documentation.
|
|
57
|
+
- Responsive, navigable pydoc pages with readable signatures and docstrings,
|
|
58
|
+
mobile layouts, preserved API anchors, and print styling.
|
|
59
|
+
- Fail-closed pydoc link post-processing that preserves generated application
|
|
60
|
+
targets and renders unavailable local targets as code.
|
|
61
|
+
- Complete pydoc coverage for `__main__` and unambiguous
|
|
62
|
+
dashboard/package/module breadcrumbs with full module titles.
|
|
63
|
+
- Security guidance, operational runbook, API reference, and ADRs.
|
|
64
|
+
|
|
65
|
+
### Changed
|
|
66
|
+
|
|
67
|
+
### Fixed
|
|
68
|
+
|
|
69
|
+
### Security
|
|
70
|
+
|
|
71
|
+
- Reject path traversal, symlink payload files, unexpected payload files, and
|
|
72
|
+
existing restore destinations.
|
|
73
|
+
- Reject secret-bearing signing command arguments.
|
|
74
|
+
- Reject oversized source files before reading their contents.
|
|
75
|
+
- Reject linked package roots, metadata sidecars, payload roots, and payload
|
|
76
|
+
descendants.
|
|
77
|
+
- Restrict manifest hashes to SHA-256, SHA-512, or optional BLAKE3 and restored
|
|
78
|
+
modes to ordinary permission bits.
|
|
79
|
+
- Restore through verified sibling staging so failures leave no partial
|
|
80
|
+
destination.
|
|
81
|
+
- Reject secret-bearing external signer flags in assignment form.
|
|
82
|
+
- GitHub Actions use explicit least-privilege permissions and immutable
|
|
83
|
+
dependency references.
|
|
84
|
+
- Releases use locked build tools and reject tagged commits that are not
|
|
85
|
+
contained in `origin/main`.
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Controlled Text Transfer contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files, to deal in the Software without restriction, subject to the following conditions: the above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
|
6
|
+
|
|
7
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.
|
|
@@ -0,0 +1,223 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: controlled-text-transfer
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Auditable text-only preparation and restoration for controlled file transfers
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Project-URL: Homepage, https://github.com/dgomez407/ctt
|
|
7
|
+
Project-URL: Repository, https://github.com/dgomez407/ctt
|
|
8
|
+
Project-URL: Issues, https://github.com/dgomez407/ctt/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/dgomez407/ctt/blob/main/CHANGELOG.md
|
|
10
|
+
Keywords: controlled-transfer,file-integrity,security,text-files
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Operating System :: OS Independent
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
17
|
+
Classifier: Topic :: Security
|
|
18
|
+
Classifier: Topic :: System :: Archiving
|
|
19
|
+
Requires-Python: >=3.12
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: PyYAML>=6.0
|
|
23
|
+
Provides-Extra: blake3
|
|
24
|
+
Requires-Dist: blake3>=0.4; extra == "blake3"
|
|
25
|
+
Provides-Extra: dev
|
|
26
|
+
Requires-Dist: pytest>=8; extra == "dev"
|
|
27
|
+
Requires-Dist: pytest-cov>=5; extra == "dev"
|
|
28
|
+
Requires-Dist: ruff>=0.5; extra == "dev"
|
|
29
|
+
Requires-Dist: black>=24; extra == "dev"
|
|
30
|
+
Requires-Dist: mypy>=1.10; extra == "dev"
|
|
31
|
+
Requires-Dist: bandit>=1.7; extra == "dev"
|
|
32
|
+
Requires-Dist: pyinstaller>=6; extra == "dev"
|
|
33
|
+
Requires-Dist: build>=1; extra == "dev"
|
|
34
|
+
Requires-Dist: twine>=6; extra == "dev"
|
|
35
|
+
Dynamic: license-file
|
|
36
|
+
|
|
37
|
+
# Controlled Text Transfer 0.1.0
|
|
38
|
+
|
|
39
|
+
Controlled Text Transfer (`ctt`) prepares an allowlisted set of text files for
|
|
40
|
+
a controlled transfer that accepts only `.txt` names. It appends `.txt`,
|
|
41
|
+
preserves content and paths in a signed-ready JSON manifest, adds an optional
|
|
42
|
+
UTF-8 BOM only to transfer copies, and restores byte-identical originals.
|
|
43
|
+
Python 3.12 or newer is required.
|
|
44
|
+
|
|
45
|
+
For the shortest path to running the application, see
|
|
46
|
+
[README-quickstart.md](README-quickstart.md).
|
|
47
|
+
|
|
48
|
+
## Documentation
|
|
49
|
+
|
|
50
|
+
- [Documentation index](docs/README.md)
|
|
51
|
+
- [Operations runbook](docs/operations.md)
|
|
52
|
+
- [Policy reference and examples](docs/policy.md)
|
|
53
|
+
- [Complete CLI option reference](docs/cli.md)
|
|
54
|
+
- [CLI and library API](docs/api.md)
|
|
55
|
+
- [Security guidance](SECURITY.md)
|
|
56
|
+
- [Architecture Decision Records](docs/decisions/README.md)
|
|
57
|
+
- [Changelog](CHANGELOG.md)
|
|
58
|
+
- [Contributor and agent guidelines](AGENTS.md)
|
|
59
|
+
- [Repository-specific agent guidance](AGENTS-project.md)
|
|
60
|
+
- [Gemini agent entry point](GEMINI.md)
|
|
61
|
+
|
|
62
|
+
## Repository Map
|
|
63
|
+
|
|
64
|
+
- [GitHub workflow details](.github/workflows/README.md)
|
|
65
|
+
- [Development scripts](scripts/README.md)
|
|
66
|
+
- [Source code](src/README.md)
|
|
67
|
+
- [Test suite](tests/README.md)
|
|
68
|
+
|
|
69
|
+
## GitHub automation
|
|
70
|
+
|
|
71
|
+
GitHub-specific automation enforces the same build, test, security, and
|
|
72
|
+
release requirements documented for local development. It uses
|
|
73
|
+
least-privilege permissions, immutable action references, and locked Python
|
|
74
|
+
dependencies.
|
|
75
|
+
|
|
76
|
+
- [Workflow definitions](.github/workflows/README.md)
|
|
77
|
+
- [`dependabot.yml`](.github/dependabot.yml) maintains pinned GitHub Actions
|
|
78
|
+
dependencies.
|
|
79
|
+
|
|
80
|
+
## Quick start
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
python -m pip install .
|
|
84
|
+
ctt prepare ./source ./transfer --policy ctt.yaml
|
|
85
|
+
ctt preflight ./source --policy ctt.yaml --json
|
|
86
|
+
ctt verify ./transfer
|
|
87
|
+
ctt restore ./transfer ./restored
|
|
88
|
+
ctt diff ./transfer ./source --json
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## Commands
|
|
92
|
+
|
|
93
|
+
| Command | Purpose |
|
|
94
|
+
|---|---|
|
|
95
|
+
| `uv run ctt preflight SOURCE --json` | Produce a read-only compatibility report |
|
|
96
|
+
| `uv run ctt prepare SOURCE TRANSFER --strict` | Build only when every candidate passes |
|
|
97
|
+
| `uv run ctt verify TRANSFER` | Verify a directory or supported archive |
|
|
98
|
+
| `uv run ctt restore TRANSFER DESTINATION` | Restore verified original files |
|
|
99
|
+
| `uv run --extra dev pytest` | Install missing development tools and run all tests |
|
|
100
|
+
| `bash scripts/run.sh check` | Run the complete local quality gate |
|
|
101
|
+
| `bash scripts/run.sh report` | Generate browsable API, coverage, quality, and security reports |
|
|
102
|
+
|
|
103
|
+
Use `--dry-run` with `prepare` or `restore` to inspect the operation without
|
|
104
|
+
writing. Use `--log-json` for machine-readable audit events. Directory format
|
|
105
|
+
publishes the requested directory; `zip`, `tar`, and `tar.gz` publish only the
|
|
106
|
+
corresponding archive.
|
|
107
|
+
|
|
108
|
+
Use `prepare --strict --json-report preflight.json` to publish nothing when a
|
|
109
|
+
candidate is rejected while retaining a deterministic compatibility report.
|
|
110
|
+
Verification and restoration accept canonical directories and generated
|
|
111
|
+
archives directly. Directory verification rejects linked package or payload
|
|
112
|
+
paths, and restoration publishes only after its staged output validates.
|
|
113
|
+
|
|
114
|
+
`diff` is read-only and reports added, removed, modified, and unchanged
|
|
115
|
+
allowlisted files when compared with the source directory.
|
|
116
|
+
|
|
117
|
+
## Architecture
|
|
118
|
+
|
|
119
|
+
The package keeps transformation, integrity verification, packaging, signing
|
|
120
|
+
hooks, and CLI concerns small and separable. The core uses a policy-driven
|
|
121
|
+
allowlist and a versioned manifest; archive creation is optional; signing is
|
|
122
|
+
delegated to externally managed key infrastructure. See ADR-001 through
|
|
123
|
+
ADR-012 for the rationale and rejected alternatives.
|
|
124
|
+
|
|
125
|
+
## Policy and detection
|
|
126
|
+
|
|
127
|
+
The default policy uses an explicit extension/name allowlist, UTF-8 decoding, a 10 MiB per-file limit, and SHA-256. Files that are binary, undecodable, ignored by `.cttignore`, oversized, or not allowlisted are reported in the manifest and never copied. `.cttignore` patterns use simple gitignore-like glob matching.
|
|
128
|
+
|
|
129
|
+
The concrete `generic-text-v1` compatibility profile also checks aggregate
|
|
130
|
+
size, file count, path depth and length, filename characters, line endings,
|
|
131
|
+
control characters, Unicode policy, line length, prohibited textual patterns,
|
|
132
|
+
and the `.txt` transfer extension. See the
|
|
133
|
+
[policy reference](docs/policy.md) for every field, default, and example.
|
|
134
|
+
|
|
135
|
+
SHA-512 is available through `hash_algorithm: sha512`. BLAKE3 is optional
|
|
136
|
+
(`uv sync --extra blake3`) and fails closed with a clear message when
|
|
137
|
+
unavailable.
|
|
138
|
+
|
|
139
|
+
## Security notes
|
|
140
|
+
|
|
141
|
+
The source directory is never modified. Destination directories must not
|
|
142
|
+
already exist. Oversized sources are rejected before content reads. Every
|
|
143
|
+
manifest path is checked against a link-free package root to prevent traversal
|
|
144
|
+
and external reads. Verification hashes transfer bytes before restoration;
|
|
145
|
+
restoration validates a staged destination after removing the transport BOM
|
|
146
|
+
and publishes it atomically. Do not treat a checksum as authenticity: use a
|
|
147
|
+
separately managed digital signature and verify it in the approved environment.
|
|
148
|
+
|
|
149
|
+
Signing is exposed through a detached-signature hook in
|
|
150
|
+
`controlled_text_transfer.signing`;
|
|
151
|
+
integrate approved GPG or X.509 tooling outside this package. Never pass
|
|
152
|
+
private keys or passphrases through CLI arguments, environment variables, or
|
|
153
|
+
manifest content. See [SECURITY.md](SECURITY.md) for the operating guidance.
|
|
154
|
+
Archives should be scanned by the CDS and malware controls; renaming or
|
|
155
|
+
encoding is not a bypass.
|
|
156
|
+
|
|
157
|
+
Packages that declare a signature require a trusted verifier by default.
|
|
158
|
+
`--allow-unverified-signature` performs explicitly unauthenticated,
|
|
159
|
+
integrity-only verification.
|
|
160
|
+
|
|
161
|
+
## Development
|
|
162
|
+
|
|
163
|
+
The Bash dispatcher provides memorable shortcuts:
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
bash scripts/run.sh setup
|
|
167
|
+
bash scripts/run.sh test
|
|
168
|
+
bash scripts/run.sh check
|
|
169
|
+
bash scripts/run.sh build
|
|
170
|
+
bash scripts/run.sh clean --dry-run
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Run `bash scripts/run.sh help` for the command summary. The underlying commands
|
|
174
|
+
remain available directly:
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
uv sync --extra dev
|
|
178
|
+
uv run pytest
|
|
179
|
+
uv run ruff check .
|
|
180
|
+
uv run black --check .
|
|
181
|
+
uv run mypy src
|
|
182
|
+
uv run bandit -r src scripts
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
After deleting `.venv`, either run `uv sync --extra dev` before the quality
|
|
186
|
+
commands or use `uv run --extra dev pytest` to install the development extra
|
|
187
|
+
and run the complete test suite in one step. Plain `uv run pytest` cannot find
|
|
188
|
+
pytest in a newly created runtime-only environment. Use
|
|
189
|
+
`uv run --extra dev pytest -q` as the equivalent concise-output alternative.
|
|
190
|
+
|
|
191
|
+
The package is pipx-friendly because it exposes the `ctt` console script.
|
|
192
|
+
Build a standalone executable with `pyinstaller ctt.spec`; keep
|
|
193
|
+
generated binaries out of source control.
|
|
194
|
+
|
|
195
|
+
### Clean generated artifacts
|
|
196
|
+
|
|
197
|
+
Preview the repository-scoped cleanup, then remove reproducible build and test
|
|
198
|
+
artifacts:
|
|
199
|
+
|
|
200
|
+
```bash
|
|
201
|
+
bash scripts/run.sh clean --dry-run
|
|
202
|
+
bash scripts/run.sh clean
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
The default removes `build/`, `dist/`, tool caches, `.coverage`,
|
|
206
|
+
`__pycache__/`, and `*.egg-info/`; it preserves source, configuration,
|
|
207
|
+
`uv.lock`, and `.venv`. To remove `.venv` too, run
|
|
208
|
+
`bash scripts/run.sh clean --environment`; the dispatcher selects the first
|
|
209
|
+
`python` on `PATH` outside the project environment. The action refuses to
|
|
210
|
+
delete its active interpreter. It never
|
|
211
|
+
cleans uv's shared global cache because that can affect other projects.
|
|
212
|
+
|
|
213
|
+
## Publishing
|
|
214
|
+
|
|
215
|
+
Releases use PyPI Trusted Publishing through
|
|
216
|
+
`.github/workflows/release.yml`. Configure a pending GitHub publisher for
|
|
217
|
+
project `controlled-text-transfer`, owner `dgomez407`, repository `ctt`,
|
|
218
|
+
workflow `release.yml`, and environment `pypi`. After the quality gates pass,
|
|
219
|
+
push a version tag such as `v0.1.0` from a commit contained in `main`. The
|
|
220
|
+
workflow verifies that the tagged commit belongs to `origin/main`, the tag,
|
|
221
|
+
package version, README, and changelog agree; reruns the locked quality gates;
|
|
222
|
+
validates the wheel and source distribution; and publishes them without a
|
|
223
|
+
stored API token.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Controlled Text Transfer Quick Start
|
|
2
|
+
|
|
3
|
+
Run these commands from the repository root. `uv` creates and manages the
|
|
4
|
+
Python 3.12-or-newer environment automatically; no separate installation is
|
|
5
|
+
required.
|
|
6
|
+
|
|
7
|
+
## Show the CLI
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
uv run ctt --help
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Prepare and transfer files
|
|
14
|
+
|
|
15
|
+
Replace `./source` with the directory containing the files to transfer:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
uv run ctt preflight ./source --json
|
|
19
|
+
uv run ctt prepare ./source ./transfer --strict
|
|
20
|
+
uv run ctt verify ./transfer
|
|
21
|
+
uv run ctt restore ./transfer ./restored
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
- `preflight` reports which files will be accepted, transformed, or rejected.
|
|
25
|
+
- `prepare --strict` creates nothing if any candidate is rejected.
|
|
26
|
+
- `verify` checks the manifest, paths, sizes, and hashes.
|
|
27
|
+
- `restore` recreates the original files in a new destination directory.
|
|
28
|
+
|
|
29
|
+
## Use a policy file
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
uv run ctt prepare ./source ./transfer --policy ./ctt.yaml --strict
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
The default `directory` package format creates `./transfer`. ZIP, TAR, and
|
|
36
|
+
TAR.GZ policies instead create `./transfer.zip`, `./transfer.tar`, or
|
|
37
|
+
`./transfer.tar.gz`.
|
|
38
|
+
|
|
39
|
+
For all options:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
uv run ctt prepare --help
|
|
43
|
+
uv run ctt verify --help
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The [complete CLI option reference](docs/cli.md) explains how and why to use
|
|
47
|
+
every option, including signing and machine-readable output.
|
|
48
|
+
|
|
49
|
+
To preview or perform a cleanup of generated repository artifacts:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
uv run python scripts/clean.py --dry-run
|
|
53
|
+
uv run python scripts/clean.py
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Run the tests
|
|
57
|
+
|
|
58
|
+
From Bash, use the development dispatcher:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
bash scripts/run.sh test
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Run every quality gate before review:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
bash scripts/run.sh check
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
After a fresh clone or after removing `.venv`, include the development extra:
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
uv run --extra dev pytest
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
For the same complete suite with concise output:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
uv run --extra dev pytest -q
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Alternatively, install all development tools first and then run commands
|
|
83
|
+
without repeating the extra:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
uv sync --extra dev
|
|
87
|
+
uv run pytest
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The suite enforces 100% aggregate statement coverage. Its terminal report omits
|
|
91
|
+
fully covered modules so remaining gaps are easier to review.
|
|
92
|
+
|
|
93
|
+
Other dispatcher commands are:
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
bash scripts/run.sh setup
|
|
97
|
+
bash scripts/run.sh report
|
|
98
|
+
bash scripts/run.sh build
|
|
99
|
+
bash scripts/run.sh clean --dry-run
|
|
100
|
+
bash scripts/run.sh clean --environment
|
|
101
|
+
bash scripts/run.sh help
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
The report command writes a browsable `reports/index.html`. Use
|
|
105
|
+
`bash scripts/run.sh report --pydoc-only` for fast API documentation, or see
|
|
106
|
+
the [scripts guide](scripts/README.md) for every generated report.
|
|
107
|
+
|
|
108
|
+
See [README.md](./README.md) for architecture and
|
|
109
|
+
[docs/operations.md](./docs/operations.md) for operational guidance. See the
|
|
110
|
+
[policy reference](./docs/policy.md) for every supported field and practical
|
|
111
|
+
examples.
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
# Controlled Text Transfer 0.1.0
|
|
2
|
+
|
|
3
|
+
Controlled Text Transfer (`ctt`) prepares an allowlisted set of text files for
|
|
4
|
+
a controlled transfer that accepts only `.txt` names. It appends `.txt`,
|
|
5
|
+
preserves content and paths in a signed-ready JSON manifest, adds an optional
|
|
6
|
+
UTF-8 BOM only to transfer copies, and restores byte-identical originals.
|
|
7
|
+
Python 3.12 or newer is required.
|
|
8
|
+
|
|
9
|
+
For the shortest path to running the application, see
|
|
10
|
+
[README-quickstart.md](README-quickstart.md).
|
|
11
|
+
|
|
12
|
+
## Documentation
|
|
13
|
+
|
|
14
|
+
- [Documentation index](docs/README.md)
|
|
15
|
+
- [Operations runbook](docs/operations.md)
|
|
16
|
+
- [Policy reference and examples](docs/policy.md)
|
|
17
|
+
- [Complete CLI option reference](docs/cli.md)
|
|
18
|
+
- [CLI and library API](docs/api.md)
|
|
19
|
+
- [Security guidance](SECURITY.md)
|
|
20
|
+
- [Architecture Decision Records](docs/decisions/README.md)
|
|
21
|
+
- [Changelog](CHANGELOG.md)
|
|
22
|
+
- [Contributor and agent guidelines](AGENTS.md)
|
|
23
|
+
- [Repository-specific agent guidance](AGENTS-project.md)
|
|
24
|
+
- [Gemini agent entry point](GEMINI.md)
|
|
25
|
+
|
|
26
|
+
## Repository Map
|
|
27
|
+
|
|
28
|
+
- [GitHub workflow details](.github/workflows/README.md)
|
|
29
|
+
- [Development scripts](scripts/README.md)
|
|
30
|
+
- [Source code](src/README.md)
|
|
31
|
+
- [Test suite](tests/README.md)
|
|
32
|
+
|
|
33
|
+
## GitHub automation
|
|
34
|
+
|
|
35
|
+
GitHub-specific automation enforces the same build, test, security, and
|
|
36
|
+
release requirements documented for local development. It uses
|
|
37
|
+
least-privilege permissions, immutable action references, and locked Python
|
|
38
|
+
dependencies.
|
|
39
|
+
|
|
40
|
+
- [Workflow definitions](.github/workflows/README.md)
|
|
41
|
+
- [`dependabot.yml`](.github/dependabot.yml) maintains pinned GitHub Actions
|
|
42
|
+
dependencies.
|
|
43
|
+
|
|
44
|
+
## Quick start
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
python -m pip install .
|
|
48
|
+
ctt prepare ./source ./transfer --policy ctt.yaml
|
|
49
|
+
ctt preflight ./source --policy ctt.yaml --json
|
|
50
|
+
ctt verify ./transfer
|
|
51
|
+
ctt restore ./transfer ./restored
|
|
52
|
+
ctt diff ./transfer ./source --json
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Commands
|
|
56
|
+
|
|
57
|
+
| Command | Purpose |
|
|
58
|
+
|---|---|
|
|
59
|
+
| `uv run ctt preflight SOURCE --json` | Produce a read-only compatibility report |
|
|
60
|
+
| `uv run ctt prepare SOURCE TRANSFER --strict` | Build only when every candidate passes |
|
|
61
|
+
| `uv run ctt verify TRANSFER` | Verify a directory or supported archive |
|
|
62
|
+
| `uv run ctt restore TRANSFER DESTINATION` | Restore verified original files |
|
|
63
|
+
| `uv run --extra dev pytest` | Install missing development tools and run all tests |
|
|
64
|
+
| `bash scripts/run.sh check` | Run the complete local quality gate |
|
|
65
|
+
| `bash scripts/run.sh report` | Generate browsable API, coverage, quality, and security reports |
|
|
66
|
+
|
|
67
|
+
Use `--dry-run` with `prepare` or `restore` to inspect the operation without
|
|
68
|
+
writing. Use `--log-json` for machine-readable audit events. Directory format
|
|
69
|
+
publishes the requested directory; `zip`, `tar`, and `tar.gz` publish only the
|
|
70
|
+
corresponding archive.
|
|
71
|
+
|
|
72
|
+
Use `prepare --strict --json-report preflight.json` to publish nothing when a
|
|
73
|
+
candidate is rejected while retaining a deterministic compatibility report.
|
|
74
|
+
Verification and restoration accept canonical directories and generated
|
|
75
|
+
archives directly. Directory verification rejects linked package or payload
|
|
76
|
+
paths, and restoration publishes only after its staged output validates.
|
|
77
|
+
|
|
78
|
+
`diff` is read-only and reports added, removed, modified, and unchanged
|
|
79
|
+
allowlisted files when compared with the source directory.
|
|
80
|
+
|
|
81
|
+
## Architecture
|
|
82
|
+
|
|
83
|
+
The package keeps transformation, integrity verification, packaging, signing
|
|
84
|
+
hooks, and CLI concerns small and separable. The core uses a policy-driven
|
|
85
|
+
allowlist and a versioned manifest; archive creation is optional; signing is
|
|
86
|
+
delegated to externally managed key infrastructure. See ADR-001 through
|
|
87
|
+
ADR-012 for the rationale and rejected alternatives.
|
|
88
|
+
|
|
89
|
+
## Policy and detection
|
|
90
|
+
|
|
91
|
+
The default policy uses an explicit extension/name allowlist, UTF-8 decoding, a 10 MiB per-file limit, and SHA-256. Files that are binary, undecodable, ignored by `.cttignore`, oversized, or not allowlisted are reported in the manifest and never copied. `.cttignore` patterns use simple gitignore-like glob matching.
|
|
92
|
+
|
|
93
|
+
The concrete `generic-text-v1` compatibility profile also checks aggregate
|
|
94
|
+
size, file count, path depth and length, filename characters, line endings,
|
|
95
|
+
control characters, Unicode policy, line length, prohibited textual patterns,
|
|
96
|
+
and the `.txt` transfer extension. See the
|
|
97
|
+
[policy reference](docs/policy.md) for every field, default, and example.
|
|
98
|
+
|
|
99
|
+
SHA-512 is available through `hash_algorithm: sha512`. BLAKE3 is optional
|
|
100
|
+
(`uv sync --extra blake3`) and fails closed with a clear message when
|
|
101
|
+
unavailable.
|
|
102
|
+
|
|
103
|
+
## Security notes
|
|
104
|
+
|
|
105
|
+
The source directory is never modified. Destination directories must not
|
|
106
|
+
already exist. Oversized sources are rejected before content reads. Every
|
|
107
|
+
manifest path is checked against a link-free package root to prevent traversal
|
|
108
|
+
and external reads. Verification hashes transfer bytes before restoration;
|
|
109
|
+
restoration validates a staged destination after removing the transport BOM
|
|
110
|
+
and publishes it atomically. Do not treat a checksum as authenticity: use a
|
|
111
|
+
separately managed digital signature and verify it in the approved environment.
|
|
112
|
+
|
|
113
|
+
Signing is exposed through a detached-signature hook in
|
|
114
|
+
`controlled_text_transfer.signing`;
|
|
115
|
+
integrate approved GPG or X.509 tooling outside this package. Never pass
|
|
116
|
+
private keys or passphrases through CLI arguments, environment variables, or
|
|
117
|
+
manifest content. See [SECURITY.md](SECURITY.md) for the operating guidance.
|
|
118
|
+
Archives should be scanned by the CDS and malware controls; renaming or
|
|
119
|
+
encoding is not a bypass.
|
|
120
|
+
|
|
121
|
+
Packages that declare a signature require a trusted verifier by default.
|
|
122
|
+
`--allow-unverified-signature` performs explicitly unauthenticated,
|
|
123
|
+
integrity-only verification.
|
|
124
|
+
|
|
125
|
+
## Development
|
|
126
|
+
|
|
127
|
+
The Bash dispatcher provides memorable shortcuts:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
bash scripts/run.sh setup
|
|
131
|
+
bash scripts/run.sh test
|
|
132
|
+
bash scripts/run.sh check
|
|
133
|
+
bash scripts/run.sh build
|
|
134
|
+
bash scripts/run.sh clean --dry-run
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Run `bash scripts/run.sh help` for the command summary. The underlying commands
|
|
138
|
+
remain available directly:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
uv sync --extra dev
|
|
142
|
+
uv run pytest
|
|
143
|
+
uv run ruff check .
|
|
144
|
+
uv run black --check .
|
|
145
|
+
uv run mypy src
|
|
146
|
+
uv run bandit -r src scripts
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
After deleting `.venv`, either run `uv sync --extra dev` before the quality
|
|
150
|
+
commands or use `uv run --extra dev pytest` to install the development extra
|
|
151
|
+
and run the complete test suite in one step. Plain `uv run pytest` cannot find
|
|
152
|
+
pytest in a newly created runtime-only environment. Use
|
|
153
|
+
`uv run --extra dev pytest -q` as the equivalent concise-output alternative.
|
|
154
|
+
|
|
155
|
+
The package is pipx-friendly because it exposes the `ctt` console script.
|
|
156
|
+
Build a standalone executable with `pyinstaller ctt.spec`; keep
|
|
157
|
+
generated binaries out of source control.
|
|
158
|
+
|
|
159
|
+
### Clean generated artifacts
|
|
160
|
+
|
|
161
|
+
Preview the repository-scoped cleanup, then remove reproducible build and test
|
|
162
|
+
artifacts:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
bash scripts/run.sh clean --dry-run
|
|
166
|
+
bash scripts/run.sh clean
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The default removes `build/`, `dist/`, tool caches, `.coverage`,
|
|
170
|
+
`__pycache__/`, and `*.egg-info/`; it preserves source, configuration,
|
|
171
|
+
`uv.lock`, and `.venv`. To remove `.venv` too, run
|
|
172
|
+
`bash scripts/run.sh clean --environment`; the dispatcher selects the first
|
|
173
|
+
`python` on `PATH` outside the project environment. The action refuses to
|
|
174
|
+
delete its active interpreter. It never
|
|
175
|
+
cleans uv's shared global cache because that can affect other projects.
|
|
176
|
+
|
|
177
|
+
## Publishing
|
|
178
|
+
|
|
179
|
+
Releases use PyPI Trusted Publishing through
|
|
180
|
+
`.github/workflows/release.yml`. Configure a pending GitHub publisher for
|
|
181
|
+
project `controlled-text-transfer`, owner `dgomez407`, repository `ctt`,
|
|
182
|
+
workflow `release.yml`, and environment `pypi`. After the quality gates pass,
|
|
183
|
+
push a version tag such as `v0.1.0` from a commit contained in `main`. The
|
|
184
|
+
workflow verifies that the tagged commit belongs to `origin/main`, the tag,
|
|
185
|
+
package version, README, and changelog agree; reruns the locked quality gates;
|
|
186
|
+
validates the wheel and source distribution; and publishes them without a
|
|
187
|
+
stored API token.
|