datafog 4.8.1__tar.gz → 4.9.0__tar.gz

{datafog-4.8.1 → datafog-4.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: datafog
-Version: 4.8.1
+Version: 4.9.0
 Summary: Lightning-fast PII detection and anonymization library, 100x+ faster than NER-based detection (see benchmarks/)
 Author: Sid Mohan
 Author-email: sid@datafog.ai
@@ -28,6 +28,8 @@ License-File: LICENSE
 Requires-Dist: pydantic<3.0,>=2.0
 Requires-Dist: pydantic-settings>=2.0.0
 Requires-Dist: typing-extensions>=4.0
+Provides-Extra: rust
+Requires-Dist: datafog-core<0.5,>=0.4.0; extra == "rust"
 Provides-Extra: nlp
 Requires-Dist: click<9.0,>=8.0; extra == "nlp"
 Requires-Dist: spacy<4.0,>=3.7.0; extra == "nlp"
@@ -36,6 +38,8 @@ Requires-Dist: gliner>=0.2.5; extra == "nlp-advanced"
 Requires-Dist: torch<2.7,>=2.1.0; extra == "nlp-advanced"
 Requires-Dist: transformers>=4.20.0; extra == "nlp-advanced"
 Requires-Dist: huggingface-hub>=0.16.0; extra == "nlp-advanced"
+Requires-Dist: sentencepiece>=0.2.0; extra == "nlp-advanced"
+Requires-Dist: protobuf>=4.0.0; extra == "nlp-advanced"
 Provides-Extra: ocr
 Requires-Dist: numpy>=1.24.0; extra == "ocr"
 Requires-Dist: pytesseract>=0.3.0; extra == "ocr"
@@ -77,6 +81,8 @@ Requires-Dist: gliner>=0.2.5; extra == "all"
 Requires-Dist: torch<2.7,>=2.1.0; extra == "all"
 Requires-Dist: transformers>=4.20.0; extra == "all"
 Requires-Dist: huggingface-hub>=0.16.0; extra == "all"
+Requires-Dist: sentencepiece>=0.2.0; extra == "all"
+Requires-Dist: protobuf>=4.0.0; extra == "all"
 Requires-Dist: numpy>=1.24.0; extra == "all"
 Requires-Dist: pytesseract>=0.3.0; extra == "all"
 Requires-Dist: Pillow>=12.2.0; extra == "all"
@@ -178,6 +184,9 @@ pip install datafog[distributed]
 pip install datafog[all]
 ```
 
+The `nlp-advanced` extra includes SentencePiece and protobuf for GLiNER's
+multilingual tokenizer; installing the OCR extra is not required for GLiNER.
+
 Python 3.13 support is certified for the core SDK, CLI, `nlp`,
 `nlp-advanced`, and `ocr` install profiles. Donut OCR still requires a model
 that is available locally before runtime use. `distributed` and `all` remain
@@ -293,9 +302,10 @@ scan/redact helpers, or guardrail helpers.
     model.
   - A Java runtime is required by PySpark.
 
-OCR and Spark are not deprecated. Their broader API and packaging overhaul is
-deferred; the 4.x goal is to keep them explicit, documented, and isolated from
-the lightweight core path.
+DataFog 4.9.0 deprecates OCR and Spark with visible use-time
+warnings; their APIs and extras will be removed in 5.0. They remain functional
+in 4.9. Users who need these features can stay on the final 4.x release. See
+the [4.9 migration guide](docs/migration-4.9.md) for the transition plan.
 
 ## Backward-Compatible APIs
 
@@ -361,6 +371,27 @@ Telemetry does not include input text or detected PII values.
 
 ## Development
 
+The [4.9 migration guide](docs/migration-4.9.md) explains opt-in Rust detection,
+the native `datafog.v5` preview, and the revised 5.0 retirement schedule for
+`detect`/`process`, OCR, and Spark. The Python detector remains the default.
+
+The experimental Rust adapter in 4.9.0 requires Core `>=0.4.0,<0.5` and capability
+contract 1. Upgrade with `python -m pip install --upgrade "datafog[rust]==4.9.0"`
+to evaluate it. Base-only users can install `datafog==4.9.0` without Core;
+installing the Rust extra does not change the default backend. See the
+[4.9.0 release notes](RELEASE_NOTES_4.9.0.md).
+Entity labels, locales, and activation settings come from the installed
+Core, allowing compatible releases to add detectors without a Python update.
+German detection is opt-in through locale or entity selection; UUID is opt-in
+through `entity_types=["UUID"]`. Core's structured-only `PERSON` is unavailable
+for explicit Rust text selection. The legacy overlap policy can suppress NPI in
+favor of PHONE; the migration guide explains native alternatives. Lock the Core
+version if detection output must remain reproducible.
+
+The [4.8.1 compatibility contract](docs/migration-4.8.1-contract.md) records
+published Python behavior for the Rust migration, with frozen fixtures and
+instructions for independently reproducing them from the release wheel.
+
 ```bash
 git clone https://github.com/datafog/datafog-python
 cd datafog-python

{datafog-4.8.1 → datafog-4.9.0}/README.md

@@ -70,6 +70,9 @@ pip install datafog[distributed]
 pip install datafog[all]
 ```
 
+The `nlp-advanced` extra includes SentencePiece and protobuf for GLiNER's
+multilingual tokenizer; installing the OCR extra is not required for GLiNER.
+
 Python 3.13 support is certified for the core SDK, CLI, `nlp`,
 `nlp-advanced`, and `ocr` install profiles. Donut OCR still requires a model
 that is available locally before runtime use. `distributed` and `all` remain
@@ -185,9 +188,10 @@ scan/redact helpers, or guardrail helpers.
     model.
   - A Java runtime is required by PySpark.
 
-OCR and Spark are not deprecated. Their broader API and packaging overhaul is
-deferred; the 4.x goal is to keep them explicit, documented, and isolated from
-the lightweight core path.
+DataFog 4.9.0 deprecates OCR and Spark with visible use-time
+warnings; their APIs and extras will be removed in 5.0. They remain functional
+in 4.9. Users who need these features can stay on the final 4.x release. See
+the [4.9 migration guide](docs/migration-4.9.md) for the transition plan.
 
 ## Backward-Compatible APIs
 
@@ -253,6 +257,27 @@ Telemetry does not include input text or detected PII values.
 
 ## Development
 
+The [4.9 migration guide](docs/migration-4.9.md) explains opt-in Rust detection,
+the native `datafog.v5` preview, and the revised 5.0 retirement schedule for
+`detect`/`process`, OCR, and Spark. The Python detector remains the default.
+
+The experimental Rust adapter in 4.9.0 requires Core `>=0.4.0,<0.5` and capability
+contract 1. Upgrade with `python -m pip install --upgrade "datafog[rust]==4.9.0"`
+to evaluate it. Base-only users can install `datafog==4.9.0` without Core;
+installing the Rust extra does not change the default backend. See the
+[4.9.0 release notes](RELEASE_NOTES_4.9.0.md).
+Entity labels, locales, and activation settings come from the installed
+Core, allowing compatible releases to add detectors without a Python update.
+German detection is opt-in through locale or entity selection; UUID is opt-in
+through `entity_types=["UUID"]`. Core's structured-only `PERSON` is unavailable
+for explicit Rust text selection. The legacy overlap policy can suppress NPI in
+favor of PHONE; the migration guide explains native alternatives. Lock the Core
+version if detection output must remain reproducible.
+
+The [4.8.1 compatibility contract](docs/migration-4.8.1-contract.md) records
+published Python behavior for the Rust migration, with frozen fixtures and
+instructions for independently reproducing them from the release wheel.
+
 ```bash
 git clone https://github.com/datafog/datafog-python
 cd datafog-python

datafog-4.9.0/datafog/__about__.py

@@ -0,0 +1 @@
+__version__ = "4.9.0"

{datafog-4.8.1 → datafog-4.9.0}/datafog/__init__.py

@@ -15,11 +15,9 @@ from .__about__ import __version__
 from .agent import create_guardrail, filter_output, sanitize, scan_prompt
 
 # Core API functions - always available (lightweight)
+from .compat import v4 as _compat_v4
 from .core import anonymize_text, detect_pii, get_supported_entities, scan_text
 from .engine import Entity, RedactResult, ScanResult
-from .engine import redact as _redact_entities
-from .engine import scan as _scan
-from .engine import scan_and_redact as _scan_and_redact
 
 # Essential models - always available
 from .models.common import EntityTypes
@@ -130,20 +128,10 @@ _LAZY_EXPORTS = {
 }
 
 
-_REDACT_PRESETS = {
-    "default": "token",
-    "llm": "token",
-    "mask": "mask",
-    "hash": "hash",
-    "replace": "pseudonymize",
-    "pseudonymize": "pseudonymize",
-}
-
-
 def _warn_v5_replacement(old_api: str, replacement: str) -> None:
     warnings.warn(
-        f"datafog.{old_api}() is deprecated for v5. Use {replacement} instead. "
-        "This compatibility shim will remain through the v5.x line.",
+        f"datafog.{old_api}() is deprecated and will be removed in 5.0. Use {replacement} instead. "
+        "The earlier promise to retain this shim through 5.x has been revised.",
         FutureWarning,
         stacklevel=3,
     )
@@ -156,25 +144,18 @@ def scan(
     locales: list[str] | None = None,
     allowlist: list[str] | None = None,
     allowlist_patterns: list[str] | None = None,
+    *,
+    backend: str = "python",
 ) -> ScanResult:
-    """
-    v5-preview scan entrypoint.
-
-    Defaults to the lightweight regex engine so the core install works without
-    optional dependency fallback warnings.
-
-    ``allowlist`` exempts exact entity texts (your own support address, doc
-    placeholders); ``allowlist_patterns`` exempts entities whose full text
-    matches a regex (e.g. ``^\\d{10}$`` so unix timestamps stop matching as
-    phone numbers).
-    """
-    return _scan(
-        text=text,
-        engine=engine,
-        entity_types=entity_types,
-        locales=locales,
-        allowlist=allowlist,
-        allowlist_patterns=allowlist_patterns,
+    """Scan using the 4.x compatibility API; Rust detection is opt-in."""
+    return _compat_v4.scan(
+        text,
+        engine,
+        entity_types,
+        locales,
+        allowlist,
+        allowlist_patterns,
+        backend=backend,
     )
 
 
@@ -188,39 +169,21 @@ def redact(
     locales: list[str] | None = None,
     allowlist: list[str] | None = None,
     allowlist_patterns: list[str] | None = None,
+    *,
+    backend: str = "python",
 ) -> RedactResult:
-    """
-    v5-preview redaction entrypoint.
-
-    If entities are provided, redact those spans. Otherwise, scan text first
-    using the selected engine and redact the detected entities. ``allowlist``
-    and ``allowlist_patterns`` exempt findings from redaction (exact text and
-    full-text regex match respectively); they apply to the scan path and are
-    rejected when explicit ``entities`` are supplied.
-    """
-    if preset is not None:
-        try:
-            strategy = _REDACT_PRESETS[preset]
-        except KeyError as exc:
-            allowed = ", ".join(sorted(_REDACT_PRESETS))
-            raise ValueError(f"preset must be one of: {allowed}") from exc
-
-    if entities is not None:
-        if allowlist or allowlist_patterns:
-            raise ValueError(
-                "allowlist/allowlist_patterns cannot be combined with explicit "
-                "entities; filter the entities before calling redact"
-            )
-        return _redact_entities(text=text, entities=entities, strategy=strategy)
-
-    return _scan_and_redact(
-        text=text,
-        engine=engine,
-        entity_types=entity_types,
-        strategy=strategy,
-        locales=locales,
-        allowlist=allowlist,
-        allowlist_patterns=allowlist_patterns,
+    """Redact using the 4.x compatibility API and legacy strategies."""
+    return _compat_v4.redact(
+        text,
+        entities,
+        engine,
+        entity_types,
+        strategy,
+        preset,
+        locales,
+        allowlist,
+        allowlist_patterns,
+        backend=backend,
     )
 
 

datafog-4.9.0/datafog/_core_capabilities.py

@@ -0,0 +1,141 @@
+"""Translate installed Core capabilities into native text scan configuration."""
+
+from __future__ import annotations
+
+from copy import deepcopy
+from functools import lru_cache
+from typing import Any
+
+_ASCII_LOWER = str.maketrans("ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz")
+
+
+def _locale_key(value: str) -> str:
+    return value.strip(" \t\n\r\v\f").translate(_ASCII_LOWER)
+
+
+def _invalid(detail: str) -> RuntimeError:
+    return RuntimeError(f"Incompatible datafog-core capabilities: {detail}")
+
+
+def _labels(value: Any, field: str) -> set[str]:
+    if not isinstance(value, list) or any(
+        not isinstance(label, str) or not label for label in value
+    ):
+        raise _invalid(f"{field} must be a list of entity labels")
+    return set(value)
+
+
+@lru_cache(maxsize=4)
+def _read_capabilities(capabilities: Any) -> tuple[set[str], dict, dict]:
+    """Snapshot one installed build; replacing its reader invalidates the cache.
+
+    The private snapshot never escapes into native scan configuration. Failed
+    capability reads or validation are not cached by lru_cache.
+    """
+    payload = deepcopy(capabilities())
+    if not isinstance(payload, dict):
+        raise _invalid("capabilities() must return a dictionary")
+    version = payload.get("contract_version")
+    if not isinstance(version, int) or isinstance(version, bool) or version != 1:
+        raise _invalid(f"unsupported contract_version {version!r}; expected 1")
+    supported = _labels(payload.get("supported_entities"), "supported_entities")
+    defaults = _labels(payload.get("default_entities"), "default_entities")
+    if not defaults <= supported:
+        raise _invalid("default_entities contains unsupported labels")
+    advertised_locales = payload.get("locales")
+    metadata = payload.get("entities")
+    if not isinstance(advertised_locales, dict) or not isinstance(metadata, dict):
+        raise _invalid("locales and entities must be dictionaries")
+
+    return supported, _locale_lookup(advertised_locales, supported), metadata
+
+
+def _locale_lookup(advertised_locales: dict, supported: set[str]) -> dict[str, str]:
+    locale_lookup: dict[str, str] = {}
+    for locale, details in advertised_locales.items():
+        if not isinstance(locale, str) or not _locale_key(locale):
+            raise _invalid("locale identifiers must be nonempty strings")
+        if (
+            not isinstance(details, dict)
+            or not _labels(
+                details.get("enabled_entities"), f"locales[{locale!r}].enabled_entities"
+            )
+            <= supported
+        ):
+            raise _invalid(f"invalid locale metadata for {locale!r}")
+        locale_lookup[_locale_key(locale)] = locale
+
+    return locale_lookup
+
+
+def _resolve_locale(value: Any, locale_lookup: dict[str, str]) -> str:
+    if not isinstance(value, str) or _locale_key(value) not in locale_lookup:
+        raise ValueError(f"Unsupported locale for the Rust backend: {value!r}")
+    return value
+
+
+def _requested_locales(locales: Any, lookup: dict[str, str]) -> set[str]:
+    if isinstance(locales, str):
+        locales = [locales]
+    if locales is not None and (
+        not isinstance(locales, (list, tuple))
+        or any(not isinstance(locale, str) for locale in locales)
+    ):
+        raise ValueError("locales must be a list of locale identifiers")
+    return {_resolve_locale(locale, lookup) for locale in locales or []}
+
+
+def _activation_config(label: str, metadata: dict) -> dict:
+    details = metadata.get(label)
+    if not isinstance(details, dict):
+        raise _invalid(f"missing entity metadata for {label!r}")
+    scopes = details.get("scopes")
+    if not isinstance(scopes, list) or any(
+        not isinstance(scope, str) for scope in scopes
+    ):
+        raise _invalid(f"invalid scopes for {label!r}")
+    if "text" not in scopes:
+        raise ValueError(
+            f"Entity {label!r} is not available for Rust text scanning "
+            "(structured-only entity)"
+        )
+    activation = details.get("activation")
+    if not isinstance(activation, dict):
+        raise _invalid(f"missing activation metadata for {label!r}")
+    kind = activation.get("kind")
+    if kind == "default":
+        return {}
+    if kind not in {"locale", "config"}:
+        raise _invalid(f"unsupported activation kind {kind!r} for {label!r}")
+    config = activation.get("scan_config")
+    if not isinstance(config, dict) or not config:
+        raise _invalid(f"missing scan_config for {label!r}")
+    if kind == "locale" and "locale" not in config:
+        raise _invalid(f"missing activation locale for {label!r}")
+    return deepcopy(config)
+
+
+def scan_configs(core: Any, requested: set[str], locales: Any) -> list[dict]:
+    """Build a union of singular-locale scans without duplicating inventories."""
+    capabilities = getattr(core, "capabilities", None)
+    if not callable(capabilities):
+        raise _invalid(
+            "contract version 1 is required; install a compatible Core 0.4.x"
+        )
+    supported, lookup, metadata = _read_capabilities(capabilities)
+    selected_locales = _requested_locales(locales, lookup)
+    common: dict[str, Any] = {}
+    for label in sorted(requested & supported):
+        config = _activation_config(label, metadata)
+        for key, value in config.items():
+            if key == "locale":
+                selected_locales.add(_resolve_locale(value, lookup))
+            elif key in common and common[key] != value:
+                raise _invalid(f"conflicting activation values for {key!r}")
+            else:
+                common[key] = value
+    if not selected_locales:
+        return [common]
+    return [
+        deepcopy(dict(common, locale=locale)) for locale in sorted(selected_locales)
+    ]

datafog-4.9.0/datafog/_legacy_retirement.py

@@ -0,0 +1,17 @@
+"""Dependency-free retirement notices for optional legacy functionality."""
+
+import warnings
+
+
+class LegacySurfaceWarning(FutureWarning):
+    """An optional DataFog surface scheduled for removal in 5.0."""
+
+
+def warn_legacy_surface(surface: str) -> None:
+    """Warn at the caller's public API boundary, before optional imports."""
+    warnings.warn(
+        f"DataFog {surface} support is deprecated in 4.9 and will be removed in 5.0. "
+        "Remain on the final 4.x release if you need continued support.",
+        LegacySurfaceWarning,
+        stacklevel=3,
+    )

{datafog-4.8.1 → datafog-4.9.0}/datafog/client.py

@@ -10,6 +10,8 @@ from typing import List, Optional
 
 import typer
 
+from datafog._legacy_retirement import LegacySurfaceWarning, warn_legacy_surface
+
 from .config import OperationType, get_config
 from .engine import scan_and_redact
 from .main import DataFog
@@ -63,6 +65,7 @@ def scan_image(
 
     Prints results or exits with error on failure.
     """
+    warn_legacy_surface("OCR")
     if not image_urls:
         typer.echo("No image URLs or file paths provided. Please provide at least one.")
         raise typer.Exit(code=1)
@@ -87,6 +90,8 @@ def scan_image(
         except Exception:
             pass
     except Exception as e:
+        if isinstance(e, LegacySurfaceWarning):
+            raise
         logging.exception("Error in run_ocr_pipeline")
         try:
             from .telemetry import track_error

datafog-4.9.0/datafog/compat/__init__.py

@@ -0,0 +1 @@
+"""Compatibility APIs for migrations between DataFog major versions."""

datafog-4.9.0/datafog/compat/v4.py

@@ -0,0 +1,105 @@
+"""Legacy scan/redact facade, preserving the 4.x Python result schema.
+
+The result classes retain their original engine identity. This namespace does
+not include the deprecated detect/process shims.
+"""
+
+from ..engine import Entity, RedactResult, ScanResult
+from ..engine import redact as _redact_entities
+from ..engine import scan as _scan
+from ..engine import scan_and_redact as _scan_and_redact
+
+__all__ = ["Entity", "ScanResult", "RedactResult", "scan", "redact"]
+
+_REDACT_PRESETS = {
+    "default": "token",
+    "llm": "token",
+    "mask": "mask",
+    "hash": "hash",
+    "replace": "pseudonymize",
+    "pseudonymize": "pseudonymize",
+}
+
+
+def scan(
+    text: str,
+    engine: str = "regex",
+    entity_types: list[str] | None = None,
+    locales: list[str] | None = None,
+    allowlist: list[str] | None = None,
+    allowlist_patterns: list[str] | None = None,
+    *,
+    backend: str = "python",
+) -> ScanResult:
+    """
+    Scan with the legacy result schema; Rust detection is opt-in.
+
+    Defaults to the lightweight regex engine so the core install works without
+    optional dependency fallback warnings.
+
+    ``allowlist`` exempts exact entity texts (your own support address, doc
+    placeholders); ``allowlist_patterns`` exempts entities whose full text
+    matches a regex (e.g. ``^\\d{10}$`` so unix timestamps stop matching as
+    phone numbers).
+    """
+    return _scan(
+        text=text,
+        engine=engine,
+        entity_types=entity_types,
+        locales=locales,
+        allowlist=allowlist,
+        allowlist_patterns=allowlist_patterns,
+        backend=backend,
+    )
+
+
+def redact(
+    text: str,
+    entities: list[Entity] | None = None,
+    engine: str = "regex",
+    entity_types: list[str] | None = None,
+    strategy: str = "token",
+    preset: str | None = None,
+    locales: list[str] | None = None,
+    allowlist: list[str] | None = None,
+    allowlist_patterns: list[str] | None = None,
+    *,
+    backend: str = "python",
+) -> RedactResult:
+    """
+    Redact with legacy strategies and result schema.
+
+    If entities are provided, redact those spans. Otherwise, scan text first
+    using the selected engine and redact the detected entities. ``allowlist``
+    and ``allowlist_patterns`` exempt findings from redaction (exact text and
+    full-text regex match respectively); they apply to the scan path and are
+    rejected when explicit ``entities`` are supplied.
+    """
+    if backend not in ("python", "rust"):
+        raise ValueError("backend must be one of: python, rust")
+
+    if preset is not None:
+        try:
+            strategy = _REDACT_PRESETS[preset]
+        except KeyError as exc:
+            allowed = ", ".join(sorted(_REDACT_PRESETS))
+            raise ValueError(f"preset must be one of: {allowed}") from exc
+
+    if entities is not None:
+        if allowlist or allowlist_patterns:
+            raise ValueError(
+                "allowlist/allowlist_patterns cannot be combined with explicit "
+                "entities; filter the entities before calling redact"
+            )
+        return _redact_entities(text=text, entities=entities, strategy=strategy)
+
+    return _scan_and_redact(
+        text=text,
+        engine=engine,
+        entity_types=entity_types,
+        strategy=strategy,
+        locales=locales,
+        allowlist=allowlist,
+        allowlist_patterns=allowlist_patterns,
+        backend=backend,
+    )

{datafog-4.8.1 → datafog-4.9.0}/datafog/engine.py

@@ -209,6 +209,53 @@ def _regex_entities(
     return _suppress_overlapping_entities(entities)
 
 
+def _rust_entities(
+    text: str,
+    entity_types: Optional[list[str]] = None,
+    locales: Optional[list[str]] = None,
+) -> list[Entity]:
+    """Adapt native findings to the legacy regex result contract.
+
+    Core supplies candidate findings. Python applies the legacy overlap
+    suppression before entity selection and allowlists, preserving that order
+    without duplicating the native detectors.
+    """
+    try:
+        import datafog_core
+    except ImportError as exc:
+        raise ImportError(
+            "The Rust backend requires datafog-core. "
+            'Install with: pip install "datafog[rust]"'
+        ) from exc
+
+    from ._core_capabilities import scan_configs
+
+    requested = {_canonical_type(value) for value in entity_types or []}
+    configs = scan_configs(datafog_core, requested, locales)
+    entities: list[Entity] = []
+    seen: set[tuple[str, int, int]] = set()
+    for config in configs:
+        for finding in datafog_core.scan(text, config=config):
+            canonical_type = _canonical_type(finding.entity_type)
+            start = finding.codepoint_range.start
+            end = finding.codepoint_range.end
+            key = (canonical_type, start, end)
+            if key in seen or not finding.matched_text.strip():
+                continue
+            seen.add(key)
+            entities.append(
+                Entity(
+                    type=canonical_type,
+                    text=finding.matched_text,
+                    start=start,
+                    end=end,
+                    confidence=1.0,
+                    engine="regex",
+                )
+            )
+    return _suppress_overlapping_entities(entities)
+
+
 def _spacy_entities(text: str) -> list[Entity]:
     annotator = _get_spacy_annotator()
     if isinstance(annotator, _UnavailableAnnotator):
@@ -357,6 +404,13 @@ def _needs_ner(entity_types: Optional[list[str]]) -> bool:
     return bool(requested & NER_ENTITY_TYPES)
 
 
+def _validate_backend(backend: str, engine: str) -> None:
+    if backend not in {"python", "rust"}:
+        raise ValueError("backend must be one of: python, rust")
+    if backend == "rust" and engine != "regex":
+        raise ValueError("backend='rust' supports only engine='regex'")
+
+
 def scan(
     text: str,
     engine: str = "smart",
@@ -364,9 +418,15 @@ def scan(
     locales: Optional[list[str]] = None,
     allowlist: Optional[list[str]] = None,
     allowlist_patterns: Optional[list[str]] = None,
+    *,
+    backend: str = "python",
 ) -> ScanResult:
     """Scan text for PII entities.
 
+    ``backend="rust"`` opts into experimental native detection for
+    ``engine="regex"``. Results retain legacy regex provenance and confidence;
+    these are compatibility values, not native confidence estimates.
+
     ``allowlist`` exempts exact entity texts (e.g. your own support email);
     ``allowlist_patterns`` exempts entities whose full text matches a regex
     (e.g. ``^\\d{10}$`` to stop unix timestamps matching as phone numbers).
@@ -377,11 +437,14 @@ def scan(
     if engine not in {"regex", "spacy", "gliner", "smart"}:
         raise ValueError("engine must be one of: regex, spacy, gliner, smart")
 
+    _validate_backend(backend, engine)
+
     # Validate patterns up front so config errors fail fast even when the
     # text contains no entities.
     _compile_allowlist_patterns(allowlist_patterns)
 
-    regex_entities = _regex_entities(
+    detector = _rust_entities if backend == "rust" else _regex_entities
+    regex_entities = detector(
         text,
         entity_types=entity_types,
         locales=locales,
@@ -547,8 +610,10 @@ def scan_and_redact(
     locales: Optional[list[str]] = None,
     allowlist: Optional[list[str]] = None,
     allowlist_patterns: Optional[list[str]] = None,
+    *,
+    backend: str = "python",
 ) -> RedactResult:
-    """Convenience wrapper: scan then redact."""
+    """Scan with the selected backend, then apply legacy Python redaction."""
     scan_result = scan(
         text=text,
         engine=engine,
@@ -556,5 +621,6 @@ def scan_and_redact(
         locales=locales,
         allowlist=allowlist,
         allowlist_patterns=allowlist_patterns,
+        backend=backend,
     )
     return redact(text=text, entities=scan_result.entities, strategy=strategy)