ciphergate-cli 0.12.0__tar.gz → 0.14.0__tar.gz

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/CHANGELOG.md

@@ -6,6 +6,35 @@ versioning: [PEP 440](https://peps.python.org/pep-0440/) with SemVer meaning (pr
 
 ## [Unreleased]
 
+## [0.14.0] - 2026-11-02
+
+Adds `--extension-format` for mixed directories and tests the optional NER detector against a
+real Presidio install. No existing behaviour changes.
+
+### Added
+
+- `--extension-format EXT=FORMAT` on `inspect`, `plan`, `sanitize` and `rescan` (repeatable) reads
+  files with that extension as FORMAT, for directories and single files. It adds unknown
+  extensions or replaces built-in ones, and cannot be combined with `--format`. The Python API
+  takes `RuntimeSettings(extension_formats=...)`.
+- Tests that run the optional NER detector against a real Presidio install with the
+  `en_core_web_sm` model, including a `sanitize` run with network access disabled. They skip
+  when Presidio is not installed. A new non-required CI job, `ner`, installs the extra and a
+  hash-pinned model and runs them.
+
+## [0.13.0] - 2026-11-02
+
+Fewer random IDs are reported as payment cards. Cards with an unrecognised prefix and no card
+keyword nearby are no longer reported, so check that your data is still covered.
+
+### Changed
+
+- `luhn_card` reports fewer random IDs as cards. A Luhn-valid number now needs a known
+  issuer prefix or a card keyword nearby, and an issuer-prefixed number right after an ID
+  word such as `invoice` or `batch id` needs a card keyword. JCB, UnionPay, Diners Club,
+  Mir and Discover 644-649 prefixes are now recognised. Cards with other prefixes and no
+  keyword are no longer reported.
+
 ## [0.12.0] - 2026-11-02
 
 Blocked runs now say what to do next, and directory runs no longer hide files they never
@@ -350,7 +379,9 @@ Pre-release (alpha). Licensed under Apache-2.0. Not a final release: the securit
 See `docs/limitations.md`. Deferred: optional NER/Presidio detectors, stdin/stdout streaming,
 directory processing, structured log parsers, key files, reversible token vault, lossy decoding.
 
-[Unreleased]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.12.0...HEAD
+[Unreleased]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.14.0...HEAD
+[0.14.0]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.13.0...v0.14.0
+[0.13.0]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.12.0...v0.13.0
 [0.12.0]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.10.0...v0.11.0
 [0.10.0]: https://github.com/aashishkumar-tech/ciphergate/compare/v0.9.0...v0.10.0

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ciphergate-cli
-Version: 0.12.0
+Version: 0.14.0
 Summary: Local-first, environment- and destination-aware inspection and sanitization of logs and test datasets.
 Project-URL: Homepage, https://github.com/aashishkumar-tech/ciphergate
 Project-URL: Source, https://github.com/aashishkumar-tech/ciphergate
@@ -50,7 +50,7 @@ vendor ticket. CipherGate replaces that with a local, explainable, scriptable ga
 inspect  ->  plan  ->  sanitize --apply  ->  rescan
 ```
 
-> **Status: 0.12.0 (alpha).** No detector finds every sensitive value. A passing run means the
+> **Status: 0.14.0 (alpha).** No detector finds every sensitive value. A passing run means the
 > configured processing completed under the selected policy; it is **not** proof that the output
 > contains no sensitive information, and it is not a compliance (GDPR, HIPAA, PCI DSS, ...) or
 > anonymization guarantee. Policy templates must be reviewed before use.
@@ -295,6 +295,10 @@ ciphergate sanitize logs/ --policy ... --output build/logs.sanitized --apply
   `rescan` fails (exit 7) when it skipped any, and `sanitize` leaves them out of the output.
   `--format` on a directory applies to every file, and the run warns when that contradicts an
   extension (for example a CSV read as text loses its column names).
+- `--extension-format EXT=FORMAT` (repeatable) reads files with that extension as FORMAT, for
+  example `--extension-format .dat=csv --extension-format .log=jsonlog`. It adds unknown extensions
+  and replaces built-in ones, works for single files too, and cannot be combined with `--format`.
+  From Python, pass `RuntimeSettings(extension_formats={".dat": InputFormat.CSV})`.
   At most 1000 files by default (`--max-files N`); more fails the run (exit 4) before anything
   is processed. An empty selection fails with `batch_empty` (exit 4).
 - Each file is processed exactly like a single-file run, in sorted path order; a failing file

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/README.md

@@ -11,7 +11,7 @@ vendor ticket. CipherGate replaces that with a local, explainable, scriptable ga
 inspect  ->  plan  ->  sanitize --apply  ->  rescan
 ```
 
-> **Status: 0.12.0 (alpha).** No detector finds every sensitive value. A passing run means the
+> **Status: 0.14.0 (alpha).** No detector finds every sensitive value. A passing run means the
 > configured processing completed under the selected policy; it is **not** proof that the output
 > contains no sensitive information, and it is not a compliance (GDPR, HIPAA, PCI DSS, ...) or
 > anonymization guarantee. Policy templates must be reviewed before use.
@@ -256,6 +256,10 @@ ciphergate sanitize logs/ --policy ... --output build/logs.sanitized --apply
   `rescan` fails (exit 7) when it skipped any, and `sanitize` leaves them out of the output.
   `--format` on a directory applies to every file, and the run warns when that contradicts an
   extension (for example a CSV read as text loses its column names).
+- `--extension-format EXT=FORMAT` (repeatable) reads files with that extension as FORMAT, for
+  example `--extension-format .dat=csv --extension-format .log=jsonlog`. It adds unknown extensions
+  and replaces built-in ones, works for single files too, and cannot be combined with `--format`.
+  From Python, pass `RuntimeSettings(extension_formats={".dat": InputFormat.CSV})`.
   At most 1000 files by default (`--max-files N`); more fails the run (exit 4) before anything
   is processed. An empty selection fails with `batch_empty` (exit 4).
 - Each file is processed exactly like a single-file run, in sorted path order; a failing file

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/docs/detector-coverage.md

@@ -12,7 +12,7 @@ are engineering estimates mapped to bands (`low` < 0.5 <= `medium` < 0.8 <= `hig
 | `email_regex` | `pii.email` | `alice.test@example.com`, `alice.test%40example.com` | `a..b@example.com`, `user@localhost` | 0.9 | ASCII only; no IDN; obfuscated forms (`alice at example dot com`) missed; `%40` is the only percent-encoding recognised outside `accesslog` |
 | `phone_pattern` | `pii.phone` | `+1 202 555 0143` (high), `(202) 555-0143` (medium) | dates, IPs, card numbers, repeated digits, < 7 digits | 0.3 bare digits, 0.55–0.8 prefixed/formatted, +0.2 keyword | no numbering-plan validation; IDs can look like phones (low band) |
 | `ip_address` | `pii.ip_address` (global), `network.telemetry` (private/loopback/link-local) | `8.8.8.8`, `10.0.0.1`, `2606:4700::1111`, `fe80::1%eth0` | `256.1.1.1`, `std::vector`, `12:30:45` | 0.85 / 0.8 | version strings like `1.2.3.4` match; documentation ranges flagged with evidence |
-| `luhn_card` | `pii.payment_card` | `4111 1111 1111 1111` | Luhn-invalid, mixed separators, repeated digits | 0.4 Luhn-only, 0.8 issuer prefix, +0.15 keyword; labelled CVV/CVC values 0.85 | random IDs pass Luhn ~10% (low band); cards split across fields missed |
+| `luhn_card` | `pii.payment_card` | `4111 1111 1111 1111` | Luhn-invalid, mixed separators, repeated digits | 0.8 issuer prefix (Visa, Mastercard, Amex, Discover, JCB, UnionPay, Diners, Mir), +0.15 keyword; 0.55 keyword without prefix; labelled CVV/CVC values 0.85 | a Luhn-valid number with no issuer prefix and no card keyword is not reported, nor is one right after an ID word (`invoice`, `batch id`, `order`) without a card keyword, so such cards are missed (for example Maestro without a keyword); cards split across fields missed |
 | `mac_address` | `pii.device_identifier` | `00:1A:2B:3C:4D:5E`, `00-1a-2b-3c-4d-5e`, `001a.2b3c.4d5e` | all-zero, broadcast, mixed separators, `12:30:45` | 0.85, 0.6 when all digits | bare 12-hex values are missed; digit-only IDs in this shape are flagged |
 | `iban` | `pii.bank_account` | a mod-97-valid IBAN, compact or space-grouped | wrong check digits, lowercase | 0.9 | country length and national format are not checked; about 1 in 97 random strings of this shape pass; lowercase forms are missed |
 | `ssn_pattern` | `pii.government_id` | `219-09-9999` | never-issued areas (`000`, `666`, `9xx`), group `00`, serial `0000`, undashed | 0.8, 0.95 with a keyword | US only; undashed numbers are missed; other dashed IDs in this shape are flagged |
@@ -28,8 +28,12 @@ Needs `pip install 'ciphergate-cli[ner]'` and a spaCy model installed by the use
 `python -m spacy download en_core_web_lg`). CipherGate checks that the model package is importable
 before importing Presidio and fails with `ner_model_missing` instead of letting Presidio download
 it. Strings with no letters or under two characters are not sent to the model. Tests
-(`tests/unit/test_presidio_ner.py`) use a fake engine. Presidio itself is **not** exercised in
-CI, so accuracy on real text has not been measured by this project.
+in `tests/unit/test_presidio_ner.py` use a fake engine. `tests/integration/test_presidio_real.py`
+runs the real Presidio with `en_core_web_sm`: it finds a common synthetic name and place, ignores
+text with no entities, removes them in a `sanitize` run with network access disabled, and fails
+closed when the model is missing. CI runs it in the `ner` job (not a required check). These are
+smoke tests on a few sentences: accuracy on real text has not been measured by this project,
+and the small model misses names that the larger `en_core_web_lg` may find.
 
 ## Text/log specifics
 

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/__init__.py

@@ -24,7 +24,7 @@ Supported public API (everything else is internal and may change without notice)
     rescan_directory(root, policy, context=None, *, recursive=False, strict=None) -> BatchReport
 """
 
-__version__ = "0.12.0"
+__version__ = "0.14.0"
 
 from ciphergate.config import RuntimeSettings
 from ciphergate.constants import (

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/cli.py

@@ -117,6 +117,15 @@ PolicyOpt = Annotated[Path, typer.Option("--policy", "-p", help="Policy YAML fil
 FormatOpt = Annotated[
     InputFormat | None, typer.Option("--format", help="Input format (default: from extension).")
 ]
+ExtFormatOpt = Annotated[
+    list[str] | None,
+    typer.Option(
+        "--extension-format",
+        metavar="EXT=FORMAT",
+        help="Read files with this extension as FORMAT, e.g. .dat=csv or .log=jsonlog. "
+        "Repeatable. Cannot be combined with --format.",
+    ),
+]
 ReportOpt = Annotated[
     Path | None, typer.Option("--report", help="Write the privacy-safe report to this file.")
 ]
@@ -192,18 +201,52 @@ def _runtime(
     encoding: str | None,
     path_display: PathDisplayChoice | None,
     max_files: int | None = None,
+    extension_formats: list[str] | None = None,
+    fmt: InputFormat | None = None,
 ) -> RuntimeSettings:
+    mapping = _parse_extension_formats(extension_formats or [])
+    if mapping and fmt is not None:
+        raise UsageError(
+            "use either --format (one format for every file) or --extension-format, not both",
+            code="format_conflict",
+        )
     try:
         return RuntimeSettings().with_overrides(
             max_input_bytes=max_bytes,
             encoding=encoding,
             path_display=str(path_display) if path_display else None,
             max_batch_files=max_files,
+            extension_formats=mapping or None,
         )
     except ValueError as exc:
         raise UsageError(str(exc), code="invalid_runtime_setting") from None
 
 
+def _parse_extension_formats(values: list[str]) -> dict[str, InputFormat]:
+    mapping: dict[str, InputFormat] = {}
+    for value in values:
+        ext, sep, name = value.partition("=")
+        ext = ext.strip().lower()
+        if not ext.startswith("."):
+            ext = "." + ext
+        try:
+            fmt = InputFormat(name.strip().lower())
+        except ValueError:
+            fmt = None
+        if not sep or fmt is None:
+            raise UsageError(
+                "--extension-format needs EXT=FORMAT with FORMAT one of " + "|".join(InputFormat),
+                code="invalid_extension_format",
+            )
+        if ext in mapping and mapping[ext] != fmt:
+            raise UsageError(
+                "--extension-format maps the same extension to two formats",
+                code="invalid_extension_format",
+            )
+        mapping[ext] = fmt
+    return mapping
+
+
 def _err(message: str) -> None:
     sys.stderr.write(escape_control(message)[:2000] + "\n")
 
@@ -501,6 +544,7 @@ def inspect_command(
     ],
     policy: PolicyOpt,
     fmt: FormatOpt = None,
+    extension_format: ExtFormatOpt = None,
     report: ReportOpt = None,
     report_format: ReportFormatOpt = None,
     environment: EnvOpt = None,
@@ -531,7 +575,7 @@ def inspect_command(
             report,
             report_format,
             _context(environment, target_environment, destination, use_case, profile),
-            _runtime(max_bytes, encoding, path_display, max_files),
+            _runtime(max_bytes, encoding, path_display, max_files, extension_format, fmt),
             quiet,
             verbose,
             force,
@@ -547,6 +591,7 @@ def plan_command(
     ],
     policy: PolicyOpt,
     fmt: FormatOpt = None,
+    extension_format: ExtFormatOpt = None,
     report: ReportOpt = None,
     report_format: ReportFormatOpt = None,
     environment: EnvOpt = None,
@@ -577,7 +622,7 @@ def plan_command(
             report,
             report_format,
             _context(environment, target_environment, destination, use_case, profile),
-            _runtime(max_bytes, encoding, path_display, max_files),
+            _runtime(max_bytes, encoding, path_display, max_files, extension_format, fmt),
             quiet,
             verbose,
             force,
@@ -604,6 +649,7 @@ def sanitize_command(
         bool, typer.Option("--apply", help="Actually write output (else plan).")
     ] = False,
     fmt: FormatOpt = None,
+    extension_format: ExtFormatOpt = None,
     report: ReportOpt = None,
     report_format: ReportFormatOpt = None,
     environment: EnvOpt = None,
@@ -636,7 +682,9 @@ def sanitize_command(
                 "--output - uses stdout for data; write the JSON report with --report PATH",
                 code="stdout_conflict",
             )
-        runtime = _runtime(max_bytes, encoding, path_display)
+        runtime = _runtime(
+            max_bytes, encoding, path_display, extension_formats=extension_format, fmt=fmt
+        )
         _check_report_path(report, [p for p in (input_path, output) if not is_stdio(p)], force)
         loaded = load_policy(policy)
         options = SanitizeOptions(apply=apply, force=force, format=selected_fmt, runtime=runtime)
@@ -689,7 +737,7 @@ def sanitize_command(
             raise UsageError(
                 "a directory INPUT needs --output DIR, not stdout", code="stdout_unsupported"
             )
-        runtime = _runtime(max_bytes, encoding, path_display, max_files)
+        runtime = _runtime(max_bytes, encoding, path_display, max_files, extension_format, fmt)
         _check_batch_report_path(report, [input_path, output], force)
         batch = sanitize_directory(
             input_path,
@@ -731,6 +779,7 @@ def rescan_command(
         ),
     ] = None,
     fmt: FormatOpt = None,
+    extension_format: ExtFormatOpt = None,
     report: ReportOpt = None,
     report_format: ReportFormatOpt = None,
     environment: EnvOpt = None,
@@ -764,7 +813,7 @@ def rescan_command(
             report,
             report_format,
             _context(environment, target_environment, destination, use_case, profile),
-            _runtime(max_bytes, encoding, path_display, max_files),
+            _runtime(max_bytes, encoding, path_display, max_files, extension_format, fmt),
             quiet,
             verbose,
             force,

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/config.py

@@ -2,10 +2,14 @@
 
 from __future__ import annotations
 
+import re
 from dataclasses import dataclass, field, replace
 from typing import Literal
 
+from ciphergate.constants import InputFormat
+
 PathDisplay = Literal["basename", "redacted", "full"]
+_EXTENSION_RE = re.compile(r"^\.[a-z0-9_-]{1,16}$")
 
 MIB = 1024 * 1024
 
@@ -27,6 +31,9 @@ class RuntimeSettings:
     encoding: str = "utf-8"
     csv_delimiter: str = ","
     path_display: PathDisplay = "basename"
+    # Extra or replacement extension -> format mappings, e.g. {".dat": csv}. Keys are
+    # lower-case single suffixes. An explicit --format still wins for every file.
+    extension_formats: dict[str, InputFormat] = field(default_factory=dict)
     # Stored for completeness of the run report; never changes decisions.
     extra: dict[str, str] = field(default_factory=dict)
 
@@ -44,6 +51,12 @@ class RuntimeSettings:
                 raise ValueError(f"{name} must be a positive integer")
         if len(self.csv_delimiter) != 1 or self.csv_delimiter in '\r\n"':
             raise ValueError("csv_delimiter must be a single character other than quote or newline")
+        for ext in self.extension_formats:
+            if not _EXTENSION_RE.match(ext):
+                raise ValueError(
+                    "extension_formats keys must look like '.dat' (a dot, then 1-16 "
+                    "lower-case letters, digits, '_' or '-')"
+                )
 
     def with_overrides(self, **changes: object) -> RuntimeSettings:
         clean = {k: v for k, v in changes.items() if v is not None}

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/detectors/payment_card.py

@@ -1,7 +1,9 @@
 """Payment-card-like detector: 13-19 digits, Luhn-valid, scored by issuer prefix and context.
 
-A Luhn-valid digit run without a known issuer prefix or context keyword is low confidence:
-long numeric identifiers pass Luhn roughly 10% of the time by chance.
+Long numeric identifiers pass Luhn roughly 10% of the time by chance, so a Luhn-valid run is
+reported only with a known issuer prefix or a card keyword nearby. An issuer-prefixed run right
+after an identifier word such as "invoice" or "batch id" is also skipped unless a card keyword
+is present. Cards with other prefixes (for example Maestro) and no keyword are missed.
 """
 
 from __future__ import annotations
@@ -16,6 +18,12 @@ _CVV_RE = re.compile(
     r"(?i)(?<![a-z0-9])(?:cvv2?|cvc2?)(?:\\?[\"'])?\s{0,3}[:=]\s{0,3}(?:\\?[\"'])?(\d{3,4})(?!\d)"
 )
 _KEYWORDS = ("card", "visa", "mastercard", "amex", "pan", "cc", "credit", "debit", "payment")
+# Words that label a number as some other identifier. Whole words only: a substring match
+# would let "paid" (which contains "id") hide a real card.
+_ID_WORD_RE = re.compile(
+    r"(?i)(?<![a-z])(?:id|invoice|order|batch|total|trace|request|ref|seq|sequence|ticket|"
+    r"build|job|txn|transaction)(?![a-z])"
+)
 
 
 def luhn_valid(digits: str) -> bool:
@@ -38,8 +46,15 @@ def known_issuer_prefix(digits: str) -> bool:
         return len(digits) == 16
     if two in (34, 37):
         return len(digits) == 15
-    if digits.startswith("6011") or two == 65:
+    three = int(digits[:3])
+    if digits.startswith("6011") or two == 65 or 644 <= three <= 649:
         return len(digits) in (16, 19)
+    if 3528 <= four <= 3589 or two == 62:  # JCB, UnionPay
+        return 16 <= len(digits) <= 19
+    if 300 <= three <= 305 or two in (36, 38, 39):  # Diners Club
+        return 14 <= len(digits) <= 19
+    if 2200 <= four <= 2204:  # Mir
+        return 16 <= len(digits) <= 19
     return False
 
 
@@ -71,12 +86,18 @@ class PaymentCardDetector:
             seps = {c for c in raw if c in " -"}
             if len(seps) > 1:
                 continue
+            issuer = known_issuer_prefix(digits)
+            keyword = preceding_context(text, m.start(), _KEYWORDS, window=32)
+            if not keyword and (
+                not issuer or _ID_WORD_RE.search(text[max(0, m.start() - 16) : m.start()])
+            ):
+                continue
             evidence = ["pattern_match", "checksum_valid"]
             confidence = 0.4
-            if known_issuer_prefix(digits):
+            if issuer:
                 confidence = 0.8
                 evidence.append("issuer_prefix")
-            if preceding_context(text, m.start(), _KEYWORDS, window=32):
+            if keyword:
                 confidence += 0.15
                 evidence.append("context_keyword")
             out.append(

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/pipeline/batch.py

@@ -24,7 +24,7 @@ import shutil
 import stat
 import sys
 import tempfile
-from collections.abc import Callable
+from collections.abc import Callable, Mapping
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any
@@ -171,7 +171,14 @@ def _is_link(entry: os.DirEntry[str]) -> bool:
     return False
 
 
-def discover(root: Path, *, recursive: bool, fmt: InputFormat | None, max_files: int) -> Discovery:
+def discover(
+    root: Path,
+    *,
+    recursive: bool,
+    fmt: InputFormat | None,
+    max_files: int,
+    extensions: Mapping[str, InputFormat] | None = None,
+) -> Discovery:
     """List the files a batch run will process. Raises on limits; never follows links."""
     try:
         st = os.lstat(root)
@@ -210,7 +217,7 @@ def discover(root: Path, *, recursive: bool, fmt: InputFormat | None, max_files:
                         walk(rel, depth + 1)
                 elif not entry.is_file(follow_symlinks=False):
                     skipped.append((rel, "not_regular_file"))
-                elif fmt is None and rel.suffix.lower() not in EXTENSION_FORMATS:
+                elif fmt is None and not _known_extension(rel, extensions):
                     skipped.append((rel, "unsupported_extension"))
                 else:
                     files.append(rel)
@@ -226,6 +233,11 @@ def discover(root: Path, *, recursive: bool, fmt: InputFormat | None, max_files:
     return Discovery(root, sorted(files, key=lambda p: p.as_posix()), skipped)
 
 
+def _known_extension(rel: Path, extensions: Mapping[str, InputFormat] | None) -> bool:
+    suffix = rel.suffix.lower()
+    return suffix in EXTENSION_FORMATS or suffix in (extensions or {})
+
+
 def display_relative(rel: Path, index: int, root: Path, mode: str) -> str:
     if mode == "redacted":
         return f"<redacted:{index:04d}>{rel.suffix}"
@@ -291,7 +303,8 @@ def coverage_warnings(
             f"{unknown} file(s) were skipped and not scanned because their extension is not "
             "recognised"
             + ("; they are not in the output" if sanitize else "")
-            + "; rename them, or pass --format to read every file as one format"
+            + "; map them with --extension-format EXT=FORMAT, or pass --format to read every "
+            "file as one format"
         )
     if fmt is not None:
         mismatched = sum(
@@ -332,7 +345,13 @@ def analyze_directory(
 ) -> BatchReport:
     """Batch ``inspect`` / ``plan`` / ``rescan``. Read-only, like the single-file commands."""
     root = Path(root)
-    discovery = discover(root, recursive=recursive, fmt=fmt, max_files=runtime.max_batch_files)
+    discovery = discover(
+        root,
+        recursive=recursive,
+        fmt=fmt,
+        max_files=runtime.max_batch_files,
+        extensions=runtime.extension_formats,
+    )
     _require_files(discovery)
 
     def run(path: Path, rel: Path) -> RunReport:
@@ -471,7 +490,11 @@ def sanitize_directory(
     runtime = options.runtime or RuntimeSettings()
     validate_output_directory(root, output)
     discovery = discover(
-        root, recursive=recursive, fmt=options.format, max_files=runtime.max_batch_files
+        root,
+        recursive=recursive,
+        fmt=options.format,
+        max_files=runtime.max_batch_files,
+        extensions=runtime.extension_formats,
     )
     _require_files(discovery)
     mode = runtime.path_display

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/pipeline/inspector.py

@@ -85,7 +85,7 @@ def prepare_run(
     ctx = resolve_context(policy, context)
     path = Path(input_path)
     validate_input_path(path, runtime.max_input_bytes)
-    detected = detect_format(path, fmt)
+    detected = detect_format(path, fmt, runtime.extension_formats)
     options = ReaderOptions(
         runtime=runtime,
         policy_settings=policy.document.settings,

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/src/ciphergate/readers/__init__.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+from collections.abc import Mapping
 from pathlib import Path
 from typing import IO
 
@@ -49,11 +50,19 @@ _JSON_PREFIXES = (
 )
 
 
-def detect_format(path: Path, explicit: InputFormat | None = None) -> InputFormat:
-    """Explicit ``--format`` wins; else use the extension, refusing contradicting content."""
+def detect_format(
+    path: Path,
+    explicit: InputFormat | None = None,
+    extensions: Mapping[str, InputFormat] | None = None,
+) -> InputFormat:
+    """Explicit ``--format`` wins; then ``extensions``, then the built-in extension table.
+
+    Content that contradicts a JSON format is refused either way.
+    """
     if explicit is not None:
         return explicit
-    fmt = EXTENSION_FORMATS.get(path.suffix.lower())
+    suffix = path.suffix.lower()
+    fmt = (extensions or {}).get(suffix) or EXTENSION_FORMATS.get(suffix)
     if fmt is None:
         raise UsageError(
             "cannot infer the input format from this file extension; pass --format "

{ciphergate_cli-0.12.0 → ciphergate_cli-0.14.0}/tests/integration/test_batch.py

@@ -405,3 +405,94 @@ def test_directory_summary_carries_next_steps(tmp_path: Path) -> None:
     (root / "app.log").write_text(SECRET_LINE, encoding="utf-8")
     r = invoke("inspect", root, "--policy", POLICIES / "developer_logs.yml")
     assert r.exit_code == 1 and "next: pass --environment" in r.stdout
+
+
+# --------------------------------------------------------------------------- --extension-format
+
+
+def _mixed(tmp_path: Path) -> Path:
+    root = tmp_path / "in"
+    root.mkdir()
+    (root / "users.dat").write_text("email\nalice@example.com\n", encoding="utf-8")
+    (root / "app.log").write_text(
+        '{"msg": "started", "user": "bob@example.com"}\n', encoding="utf-8"
+    )
+    return root
+
+
+def test_extension_format_adds_an_extension_to_discovery(tmp_path: Path) -> None:
+    root = _mixed(tmp_path)
+    assert [
+        p.as_posix() for p in discover(root, recursive=False, fmt=None, max_files=10).files
+    ] == ["app.log"]
+    mapped = discover(
+        root, recursive=False, fmt=None, max_files=10, extensions={".dat": InputFormat.CSV}
+    )
+    assert [p.as_posix() for p in mapped.files] == ["app.log", "users.dat"] and not mapped.skipped
+
+
+def test_extension_format_sets_each_files_format(tmp_path: Path, env) -> None:
+    root = _mixed(tmp_path)
+    report = tmp_path / "report.json"
+    r = invoke(
+        "inspect", root, *PRD_EXTERNAL, "--report", report,
+        "--extension-format", ".dat=csv", "--extension-format", "LOG=jsonlog",
+        env=env,
+    )  # fmt: skip
+    assert r.exit_code in (0, 1), r.stdout
+    formats = {f["path"]: f["report"]["input"]["format"] for f in _report(report)["files"]}
+    assert formats == {"app.log": "jsonlog", "users.dat": "csv"}
+    assert not _report(report)["skipped"]
+
+
+def test_extension_format_works_for_a_single_file(tmp_path: Path, env) -> None:
+    src = _mixed(tmp_path) / "users.dat"
+    plain = invoke("inspect", src, *PRD_EXTERNAL, env=env)
+    assert plain.exit_code == 2 and "format_required" in plain.stdout + plain.stderr
+    report = tmp_path / "report.json"
+    mapped = invoke(
+        "inspect", src, *PRD_EXTERNAL, "--extension-format", ".dat=csv", "--report", report, env=env
+    )
+    assert mapped.exit_code in (0, 1) and _report(report)["input"]["format"] == "csv"
+
+
+@pytest.mark.parametrize(
+    ("args", "code"),
+    [
+        (["--format", "text", "--extension-format", ".dat=csv"], "format_conflict"),
+        (["--extension-format", ".dat=xml"], "invalid_extension_format"),
+        (["--extension-format", ".dat"], "invalid_extension_format"),
+        (
+            ["--extension-format", ".dat=csv", "--extension-format", ".dat=text"],
+            "invalid_extension_format",
+        ),
+        (["--extension-format", "../x=csv"], "invalid_runtime_setting"),
+    ],
+)
+def test_bad_extension_format_is_a_usage_error(
+    args: list[str], code: str, tmp_path: Path, env
+) -> None:
+    r = invoke("inspect", _mixed(tmp_path), *PRD_EXTERNAL, *args, env=env)
+    assert r.exit_code == 2 and code in r.stdout + r.stderr
+
+
+@pytest.mark.parametrize("command", ["plan", "rescan"])
+def test_extension_format_on_other_commands(command: str, tmp_path: Path, env) -> None:
+    root = _mixed(tmp_path)
+    report = tmp_path / "report.json"
+    invoke(
+        command, root, *PRD_EXTERNAL, "--extension-format", ".dat=csv", "--report", report, env=env
+    )
+    assert [f["path"] for f in _report(report)["files"]] == ["app.log", "users.dat"]
+
+
+def test_sanitize_directory_with_extension_format(tmp_path: Path, env) -> None:
+    root = _mixed(tmp_path)
+    out = tmp_path / "out"
+    r = invoke(
+        "sanitize", root, "--policy", POLICIES / "dev_local.yml", "-o", out, "--apply",
+        "--extension-format", ".dat=csv", env=env,
+    )  # fmt: skip
+    assert r.exit_code == 0, r.stdout
+    assert sorted(p.name for p in out.iterdir()) == ["app.log", "users.dat"]
+    assert "alice@example.com" not in (out / "users.dat").read_text(encoding="utf-8")