sieve-redact 0.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Kai IWASAKI
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,216 @@
1
+ Metadata-Version: 2.4
2
+ Name: sieve-redact
3
+ Version: 0.5.0
4
+ Summary: Deterministic text and image masking — byte/byte-pixel integrity outside matched regions
5
+ Author: Kai IWASAKI
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/neguseatama/sieve-redact
8
+ Project-URL: Repository, https://github.com/neguseatama/sieve-redact.git
9
+ Project-URL: Bug Tracker, https://github.com/neguseatama/sieve-redact/issues
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Intended Audience :: Education
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.9
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Topic :: Security
20
+ Classifier: Topic :: Text Processing :: Filters
21
+ Requires-Python: >=3.9
22
+ Description-Content-Type: text/markdown
23
+ License-File: LICENSE
24
+ Provides-Extra: image
25
+ Requires-Dist: Pillow>=10.0; extra == "image"
26
+ Dynamic: license-file
27
+
28
+ # Sieve Redact v0.5
29
+
30
+ A command-line tool that removes sensitive information from documents —
31
+ **deterministically**, while preserving **byte-for-byte integrity** of
32
+ everything else.
33
+
34
+ [![CI](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml/badge.svg)](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml)
35
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
36
+ [![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/)
37
+
38
+ **English** | [日本語](README.ja.md)
39
+
40
+ ## 💡 Concept
41
+
42
+ Personal information in resumes, reports and logs is converted into a second
43
+ file that is safe to publish and share.
44
+
45
+ - **Deterministic** — Same input plus same rules always produces the same
46
+ output. Character selection in `noise` is derived from SHA-256; it depends
47
+ on nothing else — not on the environment, the clock, or any RNG.
48
+ - **Byte-for-byte integrity** — Bytes outside a match are left untouched,
49
+ down to the last byte. CRLF line endings survive. Integrity is verified
50
+ mechanically before output; `--strict` refuses to write on failure
51
+ (exit code 3).
52
+ - **No guessing** — The user specifies what is sensitive. The engine never
53
+ guesses.
54
+ - **Receipts** — The receipt records the facts of processing (positions,
55
+ lengths, modes, hashes) but never the sensitive body itself. Even
56
+ "no pattern remains" is reported as a measurement
57
+ (`pattern_appearance_in_output`), not a promise.
58
+
59
+ > The name refers to a sieve: drop only what is specified, and leave
60
+ > everything else unharmed, byte for byte.
61
+
62
+ Companion tool to [Sieve Lens](https://github.com/neguseatama/sieve-lens)
63
+ (observation of invisible content) — Lens observes, Redact masks.
64
+
65
+ ---
66
+
67
+ ## 🔥 Features
68
+
69
+ ### Six redaction modes
70
+
71
+ | MODE | Operation | Content |
72
+ |------|-----------|---------|
73
+ | `delete` | remove the region | empty string |
74
+ | `mosaic` | fill per character | `█` repeated for the original character count |
75
+ | `label` | numbered labels | prefix (default `[REDACTED]`) + sequence number |
76
+ | `noise` | same-class replacement | picked deterministically within the same character class (hiragana / katakana / kanji / upper / lower / digit) |
77
+ | `decor` | wrap | `ARG + original + ARG` |
78
+ | `replace` | bulk replace | ARG |
79
+
80
+ ### Built-in matchers (`--builtin`)
81
+
82
+ | NAME | Target |
83
+ |------|--------|
84
+ | `postal-jp` | Japanese postal codes `NNN-NNNN` |
85
+ | `phone-jp` | Japanese phone numbers (`090-…` / `03-…` / `050-…` / `0120-…` / 4-digit area codes `01XX-07XX`) |
86
+ | `email` | Email addresses (ASCII) |
87
+ | `chars` | All occurrences of given codepoints (`U+200B,U+00AD` form) |
88
+
89
+ ### Image masking (`--region`)
90
+
91
+ PNG images (8-bit RGB/RGBA, non-interlaced, non-animated) are masked by
92
+ explicit regions: `--region x,y,w,h:MODE[:ARG]` (repeatable; overlapping
93
+ regions are discarded in declaration order; zero-area or out-of-bounds
94
+ regions are usage errors; `--rule` cannot be combined with `--region`).
95
+
96
+ | MODE | Effect |
97
+ |------|--------|
98
+ | `delete` | opaque white fill |
99
+ | `mosaic` | block average over all channels (blocks align to the region origin; ARG = block size, default 8) |
100
+ | `noise` | per-pixel deterministic replacement — SHA-256(`img:x:y:R,G,B`) drives the new RGB; alpha is kept, so semi-transparent pixels keep their visibility (content changes, visibility does not) |
101
+ | `label` | black band with `[REDACTED]n` (bundled bitmap font) |
102
+ | `decor` | 2px border in the ARG color (RRGGBB hex) |
103
+ | `replace` | ARG text drawn on a white fill (bundled bitmap font) |
104
+
105
+ Image receipts set `byte_integrity_verified` to `null` (not applicable —
106
+ PNG re-encoding changes file bytes structurally) and report pixel
107
+ integrity outside every region as `pixel_integrity_verified`; `--strict`
108
+ exits 3 when it fails. File metadata (EXIF etc.) is always dropped on
109
+ save — a privacy-friendly default by design. Output bytes are
110
+ deterministic across platforms and pinned in CI (with Pillow 12.3.0;
111
+ future Pillow updates may change encoded bytes, never pixels).
112
+
113
+ ### Word-unit matching (`+` prefix)
114
+
115
+ `太郎:label` also matches the `太郎` inside `田中太郎`; `太郎:+label`
116
+ matches only at word boundaries (adjacent characters of the same class, or
117
+ both alphanumeric, cause rejection).
118
+
119
+ ### Config files and Lens integration
120
+
121
+ - `--config rules.cfg`: load rules from a file (`rule <spec>` / `builtin <spec>`
122
+ line format).
123
+ - `--from-lens catalog.json`: generate masking rules from an observation
124
+ catalog (JSON) produced by [Sieve Lens](https://github.com/neguseatama/sieve-lens).
125
+ Invisible characters become deletions; look-alike letters
126
+ (Cyrillic/Greek → Latin) become replacements based on the `twin` field.
127
+
128
+ ---
129
+
130
+ ## 💻 Quick start
131
+
132
+ git clone https://github.com/neguseatama/sieve-redact.git
133
+ cd sieve-redact
134
+
135
+ # Mosaic a name, label phone numbers
136
+ python sieve_redact.py in.txt -o out.txt \
137
+ --rule "田中太郎:mosaic" \
138
+ --builtin phone-jp:label \
139
+ --receipt receipt.json
140
+
141
+ Example receipt (stdout) — `in.txt` contains `x秘密y`, run with
142
+ `--rule "秘密:delete"`:
143
+
144
+ Sieve Redact v0.5 — レシート
145
+ 入力: 8 バイト -> 出力: 2 バイト
146
+ redact 件数: 1
147
+ #1 rule=1 matcher=literal mode=delete pos=1..7 len=6B -> 0B sha256=062a2931da68...
148
+ 他バイト完全性: 検証済み
149
+ 出力内パターン残留: none
150
+
151
+ The sensitive body never appears in the receipt — hashes only. In
152
+ image mode (`--region`) `byte_integrity_verified` is `null` (not
153
+ applicable) and pixel integrity is measured by
154
+ `pixel_integrity_verified` instead.
155
+
156
+ ### Integration with Sieve Lens
157
+
158
+ # (1) Lens produces an observation catalog
159
+ python -m sieve_lens_ext.redact_bridge input.txt catalog.json
160
+ # (2) Redact masks everything
161
+ python sieve_redact.py input.txt -o clean.txt --from-lens catalog.json
162
+
163
+ ---
164
+
165
+ ## 🔬 Determinism guarantees
166
+
167
+ Character selection in `noise` is fully determined by:
168
+
169
+ char = POOL[ SHA-256("<pattern>:<byte_offset>:<original char>").digest()[0] % len(POOL) ]
170
+
171
+ No seed. No environment dependence. Same input, same output, always.
172
+ The bundled tests verify exact agreement with this formula by recomputation.
173
+
174
+ Overlapping matches are resolved deterministically as well: rules are tried
175
+ in declaration order, each rule scans leftmost-first, later matches that
176
+ overlap a confirmed region are discarded, and confirmed regions are applied
177
+ in ascending order of position.
178
+
179
+ ---
180
+
181
+ ## 🔬 Tests
182
+
183
+ python -m unittest discover -s tests -v
184
+
185
+ 113 tests (five modes, built-in matchers, word-unit matching, config files,
186
+ Lens integration, CLI boundaries). No additional dependencies.
187
+
188
+ ---
189
+
190
+ ## ⚠️ Known limitations
191
+
192
+ 1. **No regex** — patterns are literal, exact-match strings only
193
+ 2. **No case-insensitive matching** — matching is case-sensitive
194
+ 3. **No automatic sensitive-data detection** — the user specifies what is
195
+ sensitive (by design)
196
+ 4. **Text and PNG images** — PDF and Office formats are not supported;
197
+ image masking is PNG-only (8-bit RGB/RGBA) and JPEG is explicitly
198
+ out of scope; PDF masking is out of scope (observation is Sieve
199
+ Lens's domain)
200
+ 5. **Full-width digits and non-ASCII hyphens are out of scope** for
201
+ postal/phone
202
+ 6. **4-digit area codes starting with 08/09 (except `0800`) and short
203
+ numbers such as 117/104 are out of scope** for `phone-jp` — match
204
+ them explicitly with `--rule`
205
+
206
+ ---
207
+
208
+ ## 📄 License
209
+
210
+ MIT. See [LICENSE](LICENSE).
211
+
212
+ ---
213
+
214
+ ## 👤 Author
215
+
216
+ * **Kai IWASAKI**
@@ -0,0 +1,189 @@
1
+ # Sieve Redact v0.5
2
+
3
+ A command-line tool that removes sensitive information from documents —
4
+ **deterministically**, while preserving **byte-for-byte integrity** of
5
+ everything else.
6
+
7
+ [![CI](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml/badge.svg)](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml)
8
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
9
+ [![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/)
10
+
11
+ **English** | [日本語](README.ja.md)
12
+
13
+ ## 💡 Concept
14
+
15
+ Personal information in resumes, reports and logs is converted into a second
16
+ file that is safe to publish and share.
17
+
18
+ - **Deterministic** — Same input plus same rules always produces the same
19
+ output. Character selection in `noise` is derived from SHA-256; it depends
20
+ on nothing else — not on the environment, the clock, or any RNG.
21
+ - **Byte-for-byte integrity** — Bytes outside a match are left untouched,
22
+ down to the last byte. CRLF line endings survive. Integrity is verified
23
+ mechanically before output; `--strict` refuses to write on failure
24
+ (exit code 3).
25
+ - **No guessing** — The user specifies what is sensitive. The engine never
26
+ guesses.
27
+ - **Receipts** — The receipt records the facts of processing (positions,
28
+ lengths, modes, hashes) but never the sensitive body itself. Even
29
+ "no pattern remains" is reported as a measurement
30
+ (`pattern_appearance_in_output`), not a promise.
31
+
32
+ > The name refers to a sieve: drop only what is specified, and leave
33
+ > everything else unharmed, byte for byte.
34
+
35
+ Companion tool to [Sieve Lens](https://github.com/neguseatama/sieve-lens)
36
+ (observation of invisible content) — Lens observes, Redact masks.
37
+
38
+ ---
39
+
40
+ ## 🔥 Features
41
+
42
+ ### Six redaction modes
43
+
44
+ | MODE | Operation | Content |
45
+ |------|-----------|---------|
46
+ | `delete` | remove the region | empty string |
47
+ | `mosaic` | fill per character | `█` repeated for the original character count |
48
+ | `label` | numbered labels | prefix (default `[REDACTED]`) + sequence number |
49
+ | `noise` | same-class replacement | picked deterministically within the same character class (hiragana / katakana / kanji / upper / lower / digit) |
50
+ | `decor` | wrap | `ARG + original + ARG` |
51
+ | `replace` | bulk replace | ARG |
52
+
53
+ ### Built-in matchers (`--builtin`)
54
+
55
+ | NAME | Target |
56
+ |------|--------|
57
+ | `postal-jp` | Japanese postal codes `NNN-NNNN` |
58
+ | `phone-jp` | Japanese phone numbers (`090-…` / `03-…` / `050-…` / `0120-…` / 4-digit area codes `01XX-07XX`) |
59
+ | `email` | Email addresses (ASCII) |
60
+ | `chars` | All occurrences of given codepoints (`U+200B,U+00AD` form) |
61
+
62
+ ### Image masking (`--region`)
63
+
64
+ PNG images (8-bit RGB/RGBA, non-interlaced, non-animated) are masked by
65
+ explicit regions: `--region x,y,w,h:MODE[:ARG]` (repeatable; overlapping
66
+ regions are discarded in declaration order; zero-area or out-of-bounds
67
+ regions are usage errors; `--rule` cannot be combined with `--region`).
68
+
69
+ | MODE | Effect |
70
+ |------|--------|
71
+ | `delete` | opaque white fill |
72
+ | `mosaic` | block average over all channels (blocks align to the region origin; ARG = block size, default 8) |
73
+ | `noise` | per-pixel deterministic replacement — SHA-256(`img:x:y:R,G,B`) drives the new RGB; alpha is kept, so semi-transparent pixels keep their visibility (content changes, visibility does not) |
74
+ | `label` | black band with `[REDACTED]n` (bundled bitmap font) |
75
+ | `decor` | 2px border in the ARG color (RRGGBB hex) |
76
+ | `replace` | ARG text drawn on a white fill (bundled bitmap font) |
77
+
78
+ Image receipts set `byte_integrity_verified` to `null` (not applicable —
79
+ PNG re-encoding changes file bytes structurally) and report pixel
80
+ integrity outside every region as `pixel_integrity_verified`; `--strict`
81
+ exits 3 when it fails. File metadata (EXIF etc.) is always dropped on
82
+ save — a privacy-friendly default by design. Output bytes are
83
+ deterministic across platforms and pinned in CI (with Pillow 12.3.0;
84
+ future Pillow updates may change encoded bytes, never pixels).
85
+
86
+ ### Word-unit matching (`+` prefix)
87
+
88
+ `太郎:label` also matches the `太郎` inside `田中太郎`; `太郎:+label`
89
+ matches only at word boundaries (adjacent characters of the same class, or
90
+ both alphanumeric, cause rejection).
91
+
92
+ ### Config files and Lens integration
93
+
94
+ - `--config rules.cfg`: load rules from a file (`rule <spec>` / `builtin <spec>`
95
+ line format).
96
+ - `--from-lens catalog.json`: generate masking rules from an observation
97
+ catalog (JSON) produced by [Sieve Lens](https://github.com/neguseatama/sieve-lens).
98
+ Invisible characters become deletions; look-alike letters
99
+ (Cyrillic/Greek → Latin) become replacements based on the `twin` field.
100
+
101
+ ---
102
+
103
+ ## 💻 Quick start
104
+
105
+ git clone https://github.com/neguseatama/sieve-redact.git
106
+ cd sieve-redact
107
+
108
+ # Mosaic a name, label phone numbers
109
+ python sieve_redact.py in.txt -o out.txt \
110
+ --rule "田中太郎:mosaic" \
111
+ --builtin phone-jp:label \
112
+ --receipt receipt.json
113
+
114
+ Example receipt (stdout) — `in.txt` contains `x秘密y`, run with
115
+ `--rule "秘密:delete"`:
116
+
117
+ Sieve Redact v0.5 — レシート
118
+ 入力: 8 バイト -> 出力: 2 バイト
119
+ redact 件数: 1
120
+ #1 rule=1 matcher=literal mode=delete pos=1..7 len=6B -> 0B sha256=062a2931da68...
121
+ 他バイト完全性: 検証済み
122
+ 出力内パターン残留: none
123
+
124
+ The sensitive body never appears in the receipt — hashes only. In
125
+ image mode (`--region`) `byte_integrity_verified` is `null` (not
126
+ applicable) and pixel integrity is measured by
127
+ `pixel_integrity_verified` instead.
128
+
129
+ ### Integration with Sieve Lens
130
+
131
+ # (1) Lens produces an observation catalog
132
+ python -m sieve_lens_ext.redact_bridge input.txt catalog.json
133
+ # (2) Redact masks everything
134
+ python sieve_redact.py input.txt -o clean.txt --from-lens catalog.json
135
+
136
+ ---
137
+
138
+ ## 🔬 Determinism guarantees
139
+
140
+ Character selection in `noise` is fully determined by:
141
+
142
+ char = POOL[ SHA-256("<pattern>:<byte_offset>:<original char>").digest()[0] % len(POOL) ]
143
+
144
+ No seed. No environment dependence. Same input, same output, always.
145
+ The bundled tests verify exact agreement with this formula by recomputation.
146
+
147
+ Overlapping matches are resolved deterministically as well: rules are tried
148
+ in declaration order, each rule scans leftmost-first, later matches that
149
+ overlap a confirmed region are discarded, and confirmed regions are applied
150
+ in ascending order of position.
151
+
152
+ ---
153
+
154
+ ## 🔬 Tests
155
+
156
+ python -m unittest discover -s tests -v
157
+
158
+ 113 tests (five modes, built-in matchers, word-unit matching, config files,
159
+ Lens integration, CLI boundaries). No additional dependencies.
160
+
161
+ ---
162
+
163
+ ## ⚠️ Known limitations
164
+
165
+ 1. **No regex** — patterns are literal, exact-match strings only
166
+ 2. **No case-insensitive matching** — matching is case-sensitive
167
+ 3. **No automatic sensitive-data detection** — the user specifies what is
168
+ sensitive (by design)
169
+ 4. **Text and PNG images** — PDF and Office formats are not supported;
170
+ image masking is PNG-only (8-bit RGB/RGBA) and JPEG is explicitly
171
+ out of scope; PDF masking is out of scope (observation is Sieve
172
+ Lens's domain)
173
+ 5. **Full-width digits and non-ASCII hyphens are out of scope** for
174
+ postal/phone
175
+ 6. **4-digit area codes starting with 08/09 (except `0800`) and short
176
+ numbers such as 117/104 are out of scope** for `phone-jp` — match
177
+ them explicitly with `--rule`
178
+
179
+ ---
180
+
181
+ ## 📄 License
182
+
183
+ MIT. See [LICENSE](LICENSE).
184
+
185
+ ---
186
+
187
+ ## 👤 Author
188
+
189
+ * **Kai IWASAKI**
@@ -0,0 +1,43 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "sieve-redact"
7
+ dynamic = ["version"]
8
+ description = "Deterministic text and image masking — byte/byte-pixel integrity outside matched regions"
9
+ readme = "README.md"
10
+ authors = [{ name = "Kai IWASAKI" }]
11
+ license = { text = "MIT" }
12
+ classifiers = [
13
+ "Development Status :: 3 - Alpha",
14
+ "Intended Audience :: Developers",
15
+ "Intended Audience :: Education",
16
+ "License :: OSI Approved :: MIT License",
17
+ "Programming Language :: Python :: 3",
18
+ "Programming Language :: Python :: 3.9",
19
+ "Programming Language :: Python :: 3.10",
20
+ "Programming Language :: Python :: 3.11",
21
+ "Programming Language :: Python :: 3.12",
22
+ "Topic :: Security",
23
+ "Topic :: Text Processing :: Filters"
24
+ ]
25
+ requires-python = ">=3.9"
26
+ dependencies = []
27
+
28
+ [project.urls]
29
+ Homepage = "https://github.com/neguseatama/sieve-redact"
30
+ Repository = "https://github.com/neguseatama/sieve-redact.git"
31
+ "Bug Tracker" = "https://github.com/neguseatama/sieve-redact/issues"
32
+
33
+ [project.optional-dependencies]
34
+ image = ["Pillow>=10.0"]
35
+
36
+ [project.scripts]
37
+ sieve-redact = "sieve_redact:main"
38
+
39
+ [tool.setuptools]
40
+ py-modules = ["sieve_redact"]
41
+
42
+ [tool.setuptools.dynamic]
43
+ version = { attr = "sieve_redact.VERSION" }
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,216 @@
1
+ Metadata-Version: 2.4
2
+ Name: sieve-redact
3
+ Version: 0.5.0
4
+ Summary: Deterministic text and image masking — byte/byte-pixel integrity outside matched regions
5
+ Author: Kai IWASAKI
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/neguseatama/sieve-redact
8
+ Project-URL: Repository, https://github.com/neguseatama/sieve-redact.git
9
+ Project-URL: Bug Tracker, https://github.com/neguseatama/sieve-redact/issues
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: Intended Audience :: Education
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.9
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Topic :: Security
20
+ Classifier: Topic :: Text Processing :: Filters
21
+ Requires-Python: >=3.9
22
+ Description-Content-Type: text/markdown
23
+ License-File: LICENSE
24
+ Provides-Extra: image
25
+ Requires-Dist: Pillow>=10.0; extra == "image"
26
+ Dynamic: license-file
27
+
28
+ # Sieve Redact v0.5
29
+
30
+ A command-line tool that removes sensitive information from documents —
31
+ **deterministically**, while preserving **byte-for-byte integrity** of
32
+ everything else.
33
+
34
+ [![CI](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml/badge.svg)](https://github.com/neguseatama/sieve-redact/actions/workflows/test.yml)
35
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
36
+ [![Python Version](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/)
37
+
38
+ **English** | [日本語](README.ja.md)
39
+
40
+ ## 💡 Concept
41
+
42
+ Personal information in resumes, reports and logs is converted into a second
43
+ file that is safe to publish and share.
44
+
45
+ - **Deterministic** — Same input plus same rules always produces the same
46
+ output. Character selection in `noise` is derived from SHA-256; it depends
47
+ on nothing else — not on the environment, the clock, or any RNG.
48
+ - **Byte-for-byte integrity** — Bytes outside a match are left untouched,
49
+ down to the last byte. CRLF line endings survive. Integrity is verified
50
+ mechanically before output; `--strict` refuses to write on failure
51
+ (exit code 3).
52
+ - **No guessing** — The user specifies what is sensitive. The engine never
53
+ guesses.
54
+ - **Receipts** — The receipt records the facts of processing (positions,
55
+ lengths, modes, hashes) but never the sensitive body itself. Even
56
+ "no pattern remains" is reported as a measurement
57
+ (`pattern_appearance_in_output`), not a promise.
58
+
59
+ > The name refers to a sieve: drop only what is specified, and leave
60
+ > everything else unharmed, byte for byte.
61
+
62
+ Companion tool to [Sieve Lens](https://github.com/neguseatama/sieve-lens)
63
+ (observation of invisible content) — Lens observes, Redact masks.
64
+
65
+ ---
66
+
67
+ ## 🔥 Features
68
+
69
+ ### Six redaction modes
70
+
71
+ | MODE | Operation | Content |
72
+ |------|-----------|---------|
73
+ | `delete` | remove the region | empty string |
74
+ | `mosaic` | fill per character | `█` repeated for the original character count |
75
+ | `label` | numbered labels | prefix (default `[REDACTED]`) + sequence number |
76
+ | `noise` | same-class replacement | picked deterministically within the same character class (hiragana / katakana / kanji / upper / lower / digit) |
77
+ | `decor` | wrap | `ARG + original + ARG` |
78
+ | `replace` | bulk replace | ARG |
79
+
80
+ ### Built-in matchers (`--builtin`)
81
+
82
+ | NAME | Target |
83
+ |------|--------|
84
+ | `postal-jp` | Japanese postal codes `NNN-NNNN` |
85
+ | `phone-jp` | Japanese phone numbers (`090-…` / `03-…` / `050-…` / `0120-…` / 4-digit area codes `01XX-07XX`) |
86
+ | `email` | Email addresses (ASCII) |
87
+ | `chars` | All occurrences of given codepoints (`U+200B,U+00AD` form) |
88
+
89
+ ### Image masking (`--region`)
90
+
91
+ PNG images (8-bit RGB/RGBA, non-interlaced, non-animated) are masked by
92
+ explicit regions: `--region x,y,w,h:MODE[:ARG]` (repeatable; overlapping
93
+ regions are discarded in declaration order; zero-area or out-of-bounds
94
+ regions are usage errors; `--rule` cannot be combined with `--region`).
95
+
96
+ | MODE | Effect |
97
+ |------|--------|
98
+ | `delete` | opaque white fill |
99
+ | `mosaic` | block average over all channels (blocks align to the region origin; ARG = block size, default 8) |
100
+ | `noise` | per-pixel deterministic replacement — SHA-256(`img:x:y:R,G,B`) drives the new RGB; alpha is kept, so semi-transparent pixels keep their visibility (content changes, visibility does not) |
101
+ | `label` | black band with `[REDACTED]n` (bundled bitmap font) |
102
+ | `decor` | 2px border in the ARG color (RRGGBB hex) |
103
+ | `replace` | ARG text drawn on a white fill (bundled bitmap font) |
104
+
105
+ Image receipts set `byte_integrity_verified` to `null` (not applicable —
106
+ PNG re-encoding changes file bytes structurally) and report pixel
107
+ integrity outside every region as `pixel_integrity_verified`; `--strict`
108
+ exits 3 when it fails. File metadata (EXIF etc.) is always dropped on
109
+ save — a privacy-friendly default by design. Output bytes are
110
+ deterministic across platforms and pinned in CI (with Pillow 12.3.0;
111
+ future Pillow updates may change encoded bytes, never pixels).
112
+
113
+ ### Word-unit matching (`+` prefix)
114
+
115
+ `太郎:label` also matches the `太郎` inside `田中太郎`; `太郎:+label`
116
+ matches only at word boundaries (adjacent characters of the same class, or
117
+ both alphanumeric, cause rejection).
118
+
119
+ ### Config files and Lens integration
120
+
121
+ - `--config rules.cfg`: load rules from a file (`rule <spec>` / `builtin <spec>`
122
+ line format).
123
+ - `--from-lens catalog.json`: generate masking rules from an observation
124
+ catalog (JSON) produced by [Sieve Lens](https://github.com/neguseatama/sieve-lens).
125
+ Invisible characters become deletions; look-alike letters
126
+ (Cyrillic/Greek → Latin) become replacements based on the `twin` field.
127
+
128
+ ---
129
+
130
+ ## 💻 Quick start
131
+
132
+ git clone https://github.com/neguseatama/sieve-redact.git
133
+ cd sieve-redact
134
+
135
+ # Mosaic a name, label phone numbers
136
+ python sieve_redact.py in.txt -o out.txt \
137
+ --rule "田中太郎:mosaic" \
138
+ --builtin phone-jp:label \
139
+ --receipt receipt.json
140
+
141
+ Example receipt (stdout) — `in.txt` contains `x秘密y`, run with
142
+ `--rule "秘密:delete"`:
143
+
144
+ Sieve Redact v0.5 — レシート
145
+ 入力: 8 バイト -> 出力: 2 バイト
146
+ redact 件数: 1
147
+ #1 rule=1 matcher=literal mode=delete pos=1..7 len=6B -> 0B sha256=062a2931da68...
148
+ 他バイト完全性: 検証済み
149
+ 出力内パターン残留: none
150
+
151
+ The sensitive body never appears in the receipt — hashes only. In
152
+ image mode (`--region`) `byte_integrity_verified` is `null` (not
153
+ applicable) and pixel integrity is measured by
154
+ `pixel_integrity_verified` instead.
155
+
156
+ ### Integration with Sieve Lens
157
+
158
+ # (1) Lens produces an observation catalog
159
+ python -m sieve_lens_ext.redact_bridge input.txt catalog.json
160
+ # (2) Redact masks everything
161
+ python sieve_redact.py input.txt -o clean.txt --from-lens catalog.json
162
+
163
+ ---
164
+
165
+ ## 🔬 Determinism guarantees
166
+
167
+ Character selection in `noise` is fully determined by:
168
+
169
+ char = POOL[ SHA-256("<pattern>:<byte_offset>:<original char>").digest()[0] % len(POOL) ]
170
+
171
+ No seed. No environment dependence. Same input, same output, always.
172
+ The bundled tests verify exact agreement with this formula by recomputation.
173
+
174
+ Overlapping matches are resolved deterministically as well: rules are tried
175
+ in declaration order, each rule scans leftmost-first, later matches that
176
+ overlap a confirmed region are discarded, and confirmed regions are applied
177
+ in ascending order of position.
178
+
179
+ ---
180
+
181
+ ## 🔬 Tests
182
+
183
+ python -m unittest discover -s tests -v
184
+
185
+ 113 tests (five modes, built-in matchers, word-unit matching, config files,
186
+ Lens integration, CLI boundaries). No additional dependencies.
187
+
188
+ ---
189
+
190
+ ## ⚠️ Known limitations
191
+
192
+ 1. **No regex** — patterns are literal, exact-match strings only
193
+ 2. **No case-insensitive matching** — matching is case-sensitive
194
+ 3. **No automatic sensitive-data detection** — the user specifies what is
195
+ sensitive (by design)
196
+ 4. **Text and PNG images** — PDF and Office formats are not supported;
197
+ image masking is PNG-only (8-bit RGB/RGBA) and JPEG is explicitly
198
+ out of scope; PDF masking is out of scope (observation is Sieve
199
+ Lens's domain)
200
+ 5. **Full-width digits and non-ASCII hyphens are out of scope** for
201
+ postal/phone
202
+ 6. **4-digit area codes starting with 08/09 (except `0800`) and short
203
+ numbers such as 117/104 are out of scope** for `phone-jp` — match
204
+ them explicitly with `--rule`
205
+
206
+ ---
207
+
208
+ ## 📄 License
209
+
210
+ MIT. See [LICENSE](LICENSE).
211
+
212
+ ---
213
+
214
+ ## 👤 Author
215
+
216
+ * **Kai IWASAKI**