sealed-fields 0.2.0__tar.gz → 0.4.0__tar.gz

{sealed_fields-0.2.0 → sealed_fields-0.4.0}/.gitignore

@@ -6,3 +6,4 @@ dist/
 __pycache__/
 *.pyc
 .pytest_cache/
+.venv/

sealed_fields-0.4.0/PKG-INFO

@@ -0,0 +1,141 @@
+Metadata-Version: 2.5
+Name: sealed-fields
+Version: 0.4.0
+Summary: Field-level encryption for PII at rest: AES-256-GCM with versioned key rotation, key namespaces, HMAC blind indexes, and row helpers with systemic-failure detection. Wire-compatible with the sealed-fields npm package.
+Project-URL: Homepage, https://github.com/mpointer/sealed-fields
+Project-URL: Issues, https://github.com/mpointer/sealed-fields/issues
+Author-email: Michael Pointer <mpointer@gmail.com>
+License-Expression: MIT
+Keywords: aes-256-gcm,blind-index,encryption,encryption-at-rest,field-encryption,key-rotation,pii,searchable-encryption,sqlalchemy
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Security :: Cryptography
+Requires-Python: >=3.10
+Requires-Dist: cryptography>=41
+Provides-Extra: sqlalchemy
+Requires-Dist: sqlalchemy>=1.4; extra == 'sqlalchemy'
+Description-Content-Type: text/markdown
+
+# sealed-fields (Python)
+
+Field-level encryption for PII at rest. This is the Python twin of the
+[sealed-fields npm package](https://www.npmjs.com/package/sealed-fields):
+identical wire format, identical API shape, shared cross-language test
+vectors. Data sealed by either implementation unseals in the other.
+
+```
+pip install sealed-fields
+```
+
+```python
+from sealed_fields import create_sealed_fields, generate_key
+
+sf = create_sealed_fields(
+    namespaces={"PII": {"key": os.environ["SEALED_KEY_PII"]}},
+    blind_index_key=os.environ.get("SEALED_BLIND_INDEX_KEY"),
+)
+
+token = sf.seal("free-text note about a real person")  # "v1:..."
+sf.unseal(token)                                        # plaintext
+sf.blind_index("alice@example.com")                     # HMAC lookup token
+```
+
+See the [repository README](https://github.com/mpointer/sealed-fields) for
+the full design: key rotation without a flag day, key namespaces, blind
+index entropy rules, row helpers, legacy-format decoders, and the
+fail-closed default.
+
+## SQLAlchemy column types
+
+```
+pip install "sealed-fields[sqlalchemy]"
+```
+
+Declare the sealing on the column and the ORM does the rest: sealed on
+write, unsealed on read, plaintext rows from before the backfill read
+through unchanged.
+
+```python
+from sqlalchemy import Column, Integer, String
+from sealed_fields import sealed_fields_from_env
+from sealed_fields.sqlalchemy import SealedJSON, SealedString, SealedText
+
+sf = sealed_fields_from_env(["PII"])
+
+class User(Base):
+    __tablename__ = "users"
+    id = Column(Integer, primary_key=True)
+    email = Column(SealedText(sf))
+    email_idx = Column(String(64), index=True)   # sf.blind_index(email), set by the app
+    display_name = Column(SealedString(sf))
+    preferences = Column(SealedJSON(sf))         # list or dict, stored as sealed JSON text
+```
+
+Each type takes a provider, either a `SealedFields` instance or a
+zero-argument callable returning one, and an optional `namespace`. A
+callable is resolved on every read and write, so keys loaded after the
+models are declared, or swapped between tests, are picked up without
+redeclaring any column. Keep the callable cheap: return a cached instance.
+
+```python
+from functools import lru_cache
+from sealed_fields import SealedFields, sealed_fields_from_env
+
+@lru_cache(maxsize=1)
+def current_sf() -> SealedFields:
+    return sealed_fields_from_env(["PII"])   # tests call current_sf.cache_clear()
+
+class Note(Base):
+    body = Column(SealedText(current_sf, namespace="PII"))
+```
+
+What happens at the boundary, each one a production lesson:
+
+- `None` is SQL `NULL` both ways. An empty string is sealed like any other
+  value, so `""` and `NULL` stay distinct.
+- A non-string value raises `TypeError` on write. Sealing needs text; a
+  silent `str()` would hide a bug.
+- With no key configured, the `SealedFields` mode decides. `"strict"` (the
+  default) raises on write, so a dropped environment variable cannot store
+  PII in the clear. `"dormant"` writes plaintext and logs one warning per
+  process. Use dormant only for the deploy window before keys exist, and
+  never in production:
+
+  ```python
+  mode = "strict" if os.environ.get("APP_ENV") == "production" else "dormant"
+  ```
+
+- A tampered, truncated or wrong-key token raises on read. These types do
+  not null the value the way the row helpers do: a single column has no
+  batch to measure a systemic failure against, and a quiet `None` in an
+  ORM attribute is how a wrong key goes unnoticed for months.
+- A token naming a key version that is not in the keyring raises
+  `UnknownKeyVersionError`. During a rotation, build the `SealedFields`
+  with `on_unknown_version="passthrough"` and the token comes back as-is.
+- `SealedJSON` serializes with compact separators and `ensure_ascii=False`
+  before sealing, and parses after unsealing. The backing column is `Text`,
+  not `json` or `jsonb`. If a column is still `json`/`jsonb` because that
+  migration has not run yet, the driver hands back an already-parsed value
+  and the type returns it untouched. Like `JSON`, SQLAlchemy persists a
+  reassigned value but not an in-place mutation.
+- Ciphertext is about 1.4x the UTF-8 byte length plus roughly 50 bytes.
+  Size `SealedString(sf, length=...)` for that, or use `SealedText`.
+
+A project-local alias so columns can say `Column(EncryptedText)` with no
+arguments: subclass, set `provider`, and restate `cache_ok` (SQLAlchemy
+requires every `TypeDecorator` subclass to declare it, or it stops caching
+statements that use the type and warns).
+
+```python
+class EncryptedText(SealedText):
+    cache_ok = True
+    provider = staticmethod(current_sf)
+```
+
+The provider and namespace are part of the SQLAlchemy cache key. Bind and
+result processors are cached with the compiled statement, so two columns
+sealed by different keyrings never share one; the test suite proves it with
+two identical statements and two providers.
+
+Tested against SQLAlchemy 1.4 and 2.x on every push.

sealed_fields-0.4.0/README.md

@@ -0,0 +1,122 @@
+# sealed-fields (Python)
+
+Field-level encryption for PII at rest. This is the Python twin of the
+[sealed-fields npm package](https://www.npmjs.com/package/sealed-fields):
+identical wire format, identical API shape, shared cross-language test
+vectors. Data sealed by either implementation unseals in the other.
+
+```
+pip install sealed-fields
+```
+
+```python
+from sealed_fields import create_sealed_fields, generate_key
+
+sf = create_sealed_fields(
+    namespaces={"PII": {"key": os.environ["SEALED_KEY_PII"]}},
+    blind_index_key=os.environ.get("SEALED_BLIND_INDEX_KEY"),
+)
+
+token = sf.seal("free-text note about a real person")  # "v1:..."
+sf.unseal(token)                                        # plaintext
+sf.blind_index("alice@example.com")                     # HMAC lookup token
+```
+
+See the [repository README](https://github.com/mpointer/sealed-fields) for
+the full design: key rotation without a flag day, key namespaces, blind
+index entropy rules, row helpers, legacy-format decoders, and the
+fail-closed default.
+
+## SQLAlchemy column types
+
+```
+pip install "sealed-fields[sqlalchemy]"
+```
+
+Declare the sealing on the column and the ORM does the rest: sealed on
+write, unsealed on read, plaintext rows from before the backfill read
+through unchanged.
+
+```python
+from sqlalchemy import Column, Integer, String
+from sealed_fields import sealed_fields_from_env
+from sealed_fields.sqlalchemy import SealedJSON, SealedString, SealedText
+
+sf = sealed_fields_from_env(["PII"])
+
+class User(Base):
+    __tablename__ = "users"
+    id = Column(Integer, primary_key=True)
+    email = Column(SealedText(sf))
+    email_idx = Column(String(64), index=True)   # sf.blind_index(email), set by the app
+    display_name = Column(SealedString(sf))
+    preferences = Column(SealedJSON(sf))         # list or dict, stored as sealed JSON text
+```
+
+Each type takes a provider, either a `SealedFields` instance or a
+zero-argument callable returning one, and an optional `namespace`. A
+callable is resolved on every read and write, so keys loaded after the
+models are declared, or swapped between tests, are picked up without
+redeclaring any column. Keep the callable cheap: return a cached instance.
+
+```python
+from functools import lru_cache
+from sealed_fields import SealedFields, sealed_fields_from_env
+
+@lru_cache(maxsize=1)
+def current_sf() -> SealedFields:
+    return sealed_fields_from_env(["PII"])   # tests call current_sf.cache_clear()
+
+class Note(Base):
+    body = Column(SealedText(current_sf, namespace="PII"))
+```
+
+What happens at the boundary, each one a production lesson:
+
+- `None` is SQL `NULL` both ways. An empty string is sealed like any other
+  value, so `""` and `NULL` stay distinct.
+- A non-string value raises `TypeError` on write. Sealing needs text; a
+  silent `str()` would hide a bug.
+- With no key configured, the `SealedFields` mode decides. `"strict"` (the
+  default) raises on write, so a dropped environment variable cannot store
+  PII in the clear. `"dormant"` writes plaintext and logs one warning per
+  process. Use dormant only for the deploy window before keys exist, and
+  never in production:
+
+  ```python
+  mode = "strict" if os.environ.get("APP_ENV") == "production" else "dormant"
+  ```
+
+- A tampered, truncated or wrong-key token raises on read. These types do
+  not null the value the way the row helpers do: a single column has no
+  batch to measure a systemic failure against, and a quiet `None` in an
+  ORM attribute is how a wrong key goes unnoticed for months.
+- A token naming a key version that is not in the keyring raises
+  `UnknownKeyVersionError`. During a rotation, build the `SealedFields`
+  with `on_unknown_version="passthrough"` and the token comes back as-is.
+- `SealedJSON` serializes with compact separators and `ensure_ascii=False`
+  before sealing, and parses after unsealing. The backing column is `Text`,
+  not `json` or `jsonb`. If a column is still `json`/`jsonb` because that
+  migration has not run yet, the driver hands back an already-parsed value
+  and the type returns it untouched. Like `JSON`, SQLAlchemy persists a
+  reassigned value but not an in-place mutation.
+- Ciphertext is about 1.4x the UTF-8 byte length plus roughly 50 bytes.
+  Size `SealedString(sf, length=...)` for that, or use `SealedText`.
+
+A project-local alias so columns can say `Column(EncryptedText)` with no
+arguments: subclass, set `provider`, and restate `cache_ok` (SQLAlchemy
+requires every `TypeDecorator` subclass to declare it, or it stops caching
+statements that use the type and warns).
+
+```python
+class EncryptedText(SealedText):
+    cache_ok = True
+    provider = staticmethod(current_sf)
+```
+
+The provider and namespace are part of the SQLAlchemy cache key. Bind and
+result processors are cached with the compiled statement, so two columns
+sealed by different keyrings never share one; the test suite proves it with
+two identical statements and two providers.
+
+Tested against SQLAlchemy 1.4 and 2.x on every push.

{sealed_fields-0.2.0 → sealed_fields-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sealed-fields"
-version = "0.2.0"
+version = "0.4.0"
 description = "Field-level encryption for PII at rest: AES-256-GCM with versioned key rotation, key namespaces, HMAC blind indexes, and row helpers with systemic-failure detection. Wire-compatible with the sealed-fields npm package."
 readme = "README.md"
 license = "MIT"
@@ -22,6 +22,9 @@ classifiers = [
   "Topic :: Security :: Cryptography",
 ]
 
+[project.optional-dependencies]
+sqlalchemy = ["SQLAlchemy>=1.4"]
+
 [project.urls]
 Homepage = "https://github.com/mpointer/sealed-fields"
 Issues = "https://github.com/mpointer/sealed-fields/issues"

{sealed_fields-0.2.0 → sealed_fields-0.4.0}/src/sealed_fields/__init__.py

@@ -313,6 +313,8 @@ class SealedFields:
                     return value
                 raise UnknownKeyVersionError(version, ns)
             raw = _decode_blob(m.group(2))
+            if len(raw) < NONCE_BYTES + TAG_BYTES:
+                raise SealedFieldsError("unseal: token is too short to be a sealed value.")
             nonce, ct = raw[:NONCE_BYTES], raw[NONCE_BYTES:]
             return AESGCM(key).decrypt(nonce, ct, version.encode("ascii")).decode("utf-8")
         if ring is not None and self._legacy:
@@ -331,7 +333,9 @@ class SealedFields:
             return None
         return self.unseal(value, namespace)
 
-    def blind_index(self, value: str, *, normalize: bool = True) -> str:
+    def blind_index(
+        self, value: str, *, normalize: bool | Callable[[str], str] = True
+    ) -> str:
         """Deterministic HMAC-SHA256 lookup token for an equality query on an
         otherwise-sealed column. ONLY for high-entropy values (email, username,
         phone). Never index a low-entropy field: a deterministic MAC over a
@@ -339,12 +343,21 @@ class SealedFields:
 
         Normalization (default on) is trim + lowercase, matching the npm
         package's String.prototype.toLowerCase for cross-language equality.
+        Pass a callable to normalize differently, for example
+        ``lambda s: s.strip().casefold()`` when existing index tokens were
+        built with casefold. Whatever you pick, every writer and every reader
+        of a column must use the same normalization, or lookups silently miss.
         """
         if self._index_key is None:
             raise KeyNotConfiguredError(
                 "blind_index_key is not configured; cannot build a blind index."
             )
-        normalized = value.strip().lower() if normalize else value
+        if callable(normalize):
+            normalized = normalize(value)
+        elif normalize:
+            normalized = value.strip().lower()
+        else:
+            normalized = value
         return hmac_mod.new(self._index_key, normalized.encode("utf-8"), hashlib.sha256).hexdigest()
 
     # -- row helpers ---------------------------------------------------------
@@ -368,8 +381,8 @@ class SealedFields:
             v = out.get(field)
             if not isinstance(v, str) or v == "":
                 continue
-            if is_sealed(v):
-                continue
+            # Seal unconditionally: a value that merely looks like a token is
+            # untrusted input, not ciphertext. Call seal_row once per write.
             out[field] = self.seal(v, namespace)
         return out
 
@@ -382,7 +395,9 @@ class SealedFields:
         attempts = 0
         for field in (*fields, *json_fields):
             v = out.get(field)
-            if not isinstance(v, str) or v == "" or not is_sealed(v):
+            if not isinstance(v, str) or v == "":
+                continue
+            if not (is_sealed(v) or any(_match_legacy(v, f) for f in self._legacy)):
                 continue
             attempts += 1
             try:

sealed_fields-0.4.0/src/sealed_fields/sqlalchemy.py

@@ -0,0 +1,191 @@
+"""SQLAlchemy column types that seal on write and unseal on read.
+
+    pip install "sealed-fields[sqlalchemy]"
+
+Three ``TypeDecorator`` subclasses: :class:`SealedText` (backed by ``Text``),
+:class:`SealedString` (backed by ``String``) and :class:`SealedJSON` (a JSON
+value serialized to text, then sealed, backed by ``Text``). Each takes a
+*provider*: either a :class:`~sealed_fields.SealedFields` instance or a
+zero-argument callable returning one, plus an optional namespace::
+
+    from sealed_fields import sealed_fields_from_env
+    from sealed_fields.sqlalchemy import SealedText, SealedJSON
+
+    sf = sealed_fields_from_env(["PII"])
+
+    class User(Base):
+        __tablename__ = "users"
+        id = Column(Integer, primary_key=True)
+        email = Column(SealedText(sf))
+        preferences = Column(SealedJSON(sf))
+
+A callable provider is resolved on every write and read, so keys that are
+loaded lazily (after models are declared, or re-read between tests) just
+work. The callable should be cheap: return a cached instance.
+
+Behavior at the boundaries, learned in production and kept deliberately:
+
+* ``None`` is stored as SQL NULL and read back as ``None``. An empty string
+  is sealed like any other value, so ``""`` and ``NULL`` stay distinct.
+* A column may hold plaintext rows mid-migration: ``unseal`` passes any
+  non-token string through unchanged, so swapping a column to these types
+  is a safe deploy before the backfill runs.
+* With no key configured, the SealedFields ``mode`` decides what a write
+  does. ``"strict"`` (the default) raises, so a dropped environment variable
+  cannot silently store PII in the clear. ``"dormant"`` writes plaintext and
+  logs one loud warning per process; choose it only for the deploy window
+  before keys are provisioned, and never in production.
+* A tampered, truncated or wrong-key token raises on read. These types do
+  not null the value the way the row helpers do, because a per-column type
+  has no batch to measure a systemic failure against, and a silent ``None``
+  in an ORM attribute is how ciphertext problems go unnoticed for months.
+* Ciphertext is longer than plaintext (about 1.4x the UTF-8 byte length plus
+  roughly 50 bytes of nonce, tag and version). Size the backing column for
+  that, not for the plaintext.
+"""
+from __future__ import annotations
+
+import json
+from typing import Any, Callable, Optional, Union
+
+from sqlalchemy.types import String, Text, TypeDecorator
+
+from . import SealedFields, SealedFieldsError
+
+__all__ = ["SealedText", "SealedString", "SealedJSON", "Provider"]
+
+Provider = Union[SealedFields, Callable[[], SealedFields]]
+
+
+class _SealedType(TypeDecorator):
+    """Shared plumbing: provider resolution and the seal/unseal calls."""
+
+    cache_ok = True
+
+    #: A subclass may set this so the type can be used without arguments
+    #: (``Column(MyText)``), the way a project-local alias usually is. Every
+    #: TypeDecorator subclass must restate ``cache_ok``, or SQLAlchemy stops
+    #: caching statements that use it and warns::
+    #:
+    #:     class EncryptedText(SealedText):
+    #:         cache_ok = True
+    #:         provider = staticmethod(get_sealed_fields)
+    provider: Optional[Provider] = None
+
+    def __init__(
+        self,
+        provider: Optional[Provider] = None,
+        namespace: Optional[str] = None,
+        **impl_kwargs: Any,
+    ) -> None:
+        # `namespace` is positional-or-keyword on purpose: SQLAlchemy builds
+        # the cache key from positional-or-keyword __init__ names only, and a
+        # keyword-only one would be left out of it.
+        super().__init__(**impl_kwargs)
+        resolved = provider if provider is not None else type(self).provider
+        if resolved is None:
+            raise TypeError(
+                f"{type(self).__name__} needs a provider: a SealedFields instance or a "
+                "zero-argument callable returning one."
+            )
+        if not isinstance(resolved, SealedFields) and not callable(resolved):
+            raise TypeError(
+                f"{type(self).__name__} provider must be a SealedFields instance or a "
+                f"callable, not {type(resolved).__name__}."
+            )
+        # Both attributes are part of the SQLAlchemy cache key (cache_ok=True
+        # keys on the __init__ argument names found in __dict__). Bind and
+        # result processors are cached with the compiled statement, so two
+        # columns sealed by different providers must never share one.
+        self.provider = resolved
+        self.namespace = namespace
+
+    def _sf(self) -> SealedFields:
+        p = self.provider
+        sf = p if isinstance(p, SealedFields) else p()  # type: ignore[operator]
+        if not isinstance(sf, SealedFields):
+            raise SealedFieldsError(
+                f"{type(self).__name__} provider returned {type(sf).__name__}, "
+                "expected a SealedFields instance."
+            )
+        return sf
+
+    def _seal(self, value: str) -> str:
+        return self._sf().seal(value, self.namespace)
+
+    def _unseal(self, value: str) -> str:
+        return self._sf().unseal(value, self.namespace)
+
+
+class SealedText(_SealedType):
+    """Text column sealed at rest.
+
+    Writes seal the string with the namespace's active key. Reads unseal a
+    token and pass plaintext (a row written before the backfill) through
+    unchanged.
+    """
+
+    impl = Text
+    cache_ok = True
+
+    def process_bind_param(self, value: Any, dialect: Any) -> Optional[str]:
+        if value is None:
+            return None
+        if not isinstance(value, str):
+            raise TypeError(
+                f"{type(self).__name__} expects str or None, got {type(value).__name__}."
+            )
+        return self._seal(value)
+
+    def process_result_value(self, value: Any, dialect: Any) -> Optional[str]:
+        if value is None:
+            return None
+        if not isinstance(value, str):
+            return value
+        return self._unseal(value)
+
+
+class SealedString(SealedText):
+    """:class:`SealedText` backed by ``String`` instead of ``Text``, for call
+    sites that previously declared ``String``. Pass ``length=`` sized for
+    the ciphertext, not the plaintext, or leave it unset.
+    """
+
+    impl = String
+    cache_ok = True
+
+
+class SealedJSON(_SealedType):
+    """JSON value (list or dict) sealed at rest.
+
+    The value is serialized to compact JSON text, then sealed, so the backing
+    column is ``Text``, not ``json`` or ``jsonb``. On read, a value the driver
+    already parsed (a column still typed ``json``/``jsonb`` because the
+    migration to text has not run yet) is returned as-is; only strings are
+    unsealed and decoded.
+
+    Like ``JSON``, SQLAlchemy persists a reassigned value (``obj.col = new``)
+    but does not track an in-place mutation (``obj.col.append(x)``).
+    """
+
+    impl = Text
+    cache_ok = True
+
+    def process_bind_param(self, value: Any, dialect: Any) -> Optional[str]:
+        if value is None:
+            return None
+        payload = json.dumps(value, separators=(",", ":"), ensure_ascii=False)
+        return self._seal(payload)
+
+    def process_result_value(self, value: Any, dialect: Any) -> Any:
+        if value is None:
+            return None
+        if not isinstance(value, str):
+            return value
+        decoded = self._unseal(value)
+        if decoded is None or decoded == "":
+            return None
+        try:
+            return json.loads(decoded)
+        except ValueError:
+            return decoded  # legacy text that was never JSON; never lose it

{sealed_fields-0.2.0 → sealed_fields-0.4.0}/tests/test_sealed_fields.py

@@ -154,11 +154,20 @@ class TestRows:
         assert out["extra"] == "untouched"
         assert row["name"] == "Ada"
 
-    def test_no_double_seal(self):
+    def test_token_shaped_input_is_sealed_not_trusted(self):
+        # Regression: seal_row used to skip values that looked sealed, storing
+        # attacker-chosen "tokens" unencrypted and tripping SystemicUnsealError.
         sf = make_sf()
-        once = sf.seal_row({"name": "Ada"}, self.FIELDS)
-        twice = sf.seal_row(once, self.FIELDS)
-        assert twice["name"] == once["name"]
+        evil = "v1:" + base64.urlsafe_b64encode(os.urandom(30)).decode().rstrip("=")
+        assert is_sealed(evil)
+        out = sf.seal_row({"name": evil}, self.FIELDS)
+        assert out["name"] != evil
+        assert sf.unseal_row(out, self.FIELDS)["name"] == evil
+        rows = [
+            sf.seal_row({"name": "v1:" + base64.urlsafe_b64encode(os.urandom(30)).decode()}, self.FIELDS)
+            for _ in range(5)
+        ]
+        sf.unseal_rows(rows, self.FIELDS)  # must not raise
 
     def test_mixed_plaintext_rows_mid_migration(self):
         sf = make_sf()
@@ -245,6 +254,19 @@ class TestLegacy:
         token = self._iv_ct_tag("x", self._key_bytes(KEY_A))
         assert sf.unseal(token) == token
 
+    def test_row_helpers_decode_legacy_tokens(self):
+        sf = make_sf(legacy=["iv-ct-tag", "v1-iv-tag-ct"], logger=lambda m: None)
+        k = self._key_bytes(KEY_A)
+        row = {"a": self._iv_ct_tag("one", k), "b": self._v1_iv_tag_ct("two", k)}
+        assert sf.unseal_row(row, ["a", "b"]) == {"a": "one", "b": "two"}
+
+    def test_row_helper_legacy_failures_count_as_systemic(self):
+        sf = make_sf(legacy=["iv-ct-tag"], logger=lambda m: None)
+        rows = [{"a": self._iv_ct_tag(f"p{i}", self._key_bytes(KEY_B))} for i in range(5)]
+        assert sf.unseal_row(rows[0], ["a"])["a"] is None
+        with pytest.raises(SystemicUnsealError):
+            sf.unseal_rows(rows, ["a"])
+
     def test_ordinary_colon_text_not_mistaken(self):
         sf = make_sf(legacy=["iv-ct-tag", "v1-iv-tag-ct"])
         for s in ["a:b:c", "12:30:45", "v1:looks:like:legacy"]:
@@ -263,6 +285,16 @@ class TestBlindIndexAndUtils:
         expected = hmac_mod.new(raw, b"alice@example.com", hashlib.sha256).hexdigest()
         assert sf.blind_index("alice@example.com") == expected
 
+    def test_blind_index_custom_normalizer(self):
+        sf = make_sf(blind_index_key=KEY_A)
+        casefold = lambda s: s.strip().casefold()
+        # "ß".casefold() is "ss" while lower() leaves it alone, so the two
+        # policies produce different tokens for the same input.
+        assert sf.blind_index(" Straße ", normalize=casefold) == sf.blind_index(
+            "strasse", normalize=False
+        )
+        assert sf.blind_index("Straße", normalize=casefold) != sf.blind_index("Straße")
+
     def test_blind_index_requires_key(self):
         with pytest.raises(KeyNotConfiguredError):
             make_sf().blind_index("x")
@@ -313,3 +345,10 @@ class TestCrossLanguage:
         for name, token in VECTORS["tokens"].items():
             assert sf.unseal(token) == VECTORS["plaintext"], name
         assert sf.blind_index(VECTORS["blindIndex"]["value"]) == VECTORS["blindIndex"]["hex"]
+
+
+def test_short_canonical_token_rejected():
+    sf = make_sf()
+    for t in ["v1:AAAAAA", "v1:AAAAAAAAAAA", "v1:" + "A" * 36]:
+        with pytest.raises(SealedFieldsError, match="too short"):
+            sf.unseal(t)

sealed_fields-0.4.0/tests/test_sqlalchemy.py

@@ -0,0 +1,308 @@
+"""SQLAlchemy column types: sealed at rest, plaintext in the ORM."""
+import base64
+import json
+
+import pytest
+import sqlalchemy as sa
+from sqlalchemy import Column, Integer, MetaData, Table, Text, bindparam, insert, select, text
+from sqlalchemy.orm import Session, declarative_base
+
+from sealed_fields import (
+    KeyNotConfiguredError,
+    SealedFields,
+    SealedFieldsError,
+    UnknownKeyVersionError,
+    create_sealed_fields,
+    generate_key,
+    is_sealed,
+)
+from sealed_fields.sqlalchemy import SealedJSON, SealedString, SealedText
+
+KEY_A = generate_key()
+KEY_B = generate_key()
+
+SF_A = create_sealed_fields(namespaces={"PII": {"key": KEY_A}})
+SF_B = create_sealed_fields(namespaces={"PII": {"key": KEY_B}})
+
+
+class Current:
+    """Stand-in for an app's lazily loaded keyring: tests swap `.sf`."""
+
+    sf = SF_A
+
+
+def current_sf():
+    return Current.sf
+
+
+Base = declarative_base()
+
+
+class Note(Base):
+    __tablename__ = "notes"
+    id = Column(Integer, primary_key=True)
+    body = Column(SealedText(SF_A))
+    author = Column(SealedString(SF_A))
+    snapshot = Column(SealedJSON(SF_A))
+    lazy = Column(SealedText(current_sf))
+
+
+@pytest.fixture
+def engine():
+    eng = sa.create_engine("sqlite://")
+    Base.metadata.create_all(eng)
+    yield eng
+    eng.dispose()
+
+
+@pytest.fixture
+def session(engine):
+    with Session(engine) as s:
+        yield s
+
+
+@pytest.fixture(autouse=True)
+def reset_current():
+    Current.sf = SF_A
+    yield
+    Current.sf = SF_A
+
+
+def raw(session, column, note_id):
+    return session.execute(
+        text(f"SELECT {column} FROM notes WHERE id = :id"), {"id": note_id}
+    ).scalar()
+
+
+class TestSealedText:
+    def test_stored_sealed_read_plain(self, session):
+        secret = "slept nine hours, anxiety way down"
+        n = Note(body=secret, author="Ada")
+        session.add(n)
+        session.commit()
+        stored = raw(session, "body", n.id)
+        assert is_sealed(stored)
+        assert secret not in stored
+        assert is_sealed(raw(session, "author", n.id))
+        session.expire(n)
+        assert n.body == secret
+        assert n.author == "Ada"
+
+    def test_none_stays_null_and_empty_string_is_sealed(self, session):
+        n = Note(body=None, author="")
+        session.add(n)
+        session.commit()
+        assert raw(session, "body", n.id) is None
+        stored_author = raw(session, "author", n.id)
+        assert is_sealed(stored_author)  # "" and NULL stay distinct
+        session.expire(n)
+        assert n.body is None
+        assert n.author == ""
+
+    def test_legacy_plaintext_row_reads_through(self, session):
+        session.execute(
+            text("INSERT INTO notes (body) VALUES (:b)"), {"b": "written before sealing"}
+        )
+        session.commit()
+        n = session.execute(select(Note)).scalar_one()
+        assert n.body == "written before sealing"
+
+    def test_non_string_write_raises(self, session):
+        session.add(Note(body=123))
+        with pytest.raises(sa.exc.StatementError) as excinfo:
+            session.commit()
+        assert isinstance(excinfo.value.orig, TypeError)
+
+    def test_tampered_token_raises_on_read(self, session):
+        n = Note(body="secret")
+        session.add(n)
+        session.commit()
+        stored = raw(session, "body", n.id)
+        version, blob = stored.split(":", 1)
+        bad = bytearray(base64.urlsafe_b64decode(blob + "=" * (-len(blob) % 4)))
+        bad[-1] ^= 0x01
+        forged = f"{version}:{base64.urlsafe_b64encode(bytes(bad)).decode().rstrip('=')}"
+        session.execute(text("UPDATE notes SET body = :b WHERE id = :id"), {"b": forged, "id": n.id})
+        session.commit()
+        session.expire(n)
+        with pytest.raises(Exception):
+            n.body
+
+    def test_lazy_provider_resolves_on_every_call(self, session):
+        n = Note(lazy="under key A")
+        session.add(n)
+        session.commit()
+        stored = raw(session, "lazy", n.id)
+        assert SF_A.unseal(stored) == "under key A"
+
+        Current.sf = SF_B  # rotate: the column must follow, no re-declaration
+        m = Note(lazy="under key B")
+        session.add(m)
+        session.commit()
+        assert SF_B.unseal(raw(session, "lazy", m.id)) == "under key B"
+        session.expire(n)
+        with pytest.raises(Exception):  # key A rows are unreadable under B alone
+            n.lazy
+
+    def test_unknown_version_raises_unless_passthrough(self, session):
+        future = create_sealed_fields(namespaces={"PII": {"key": KEY_A, "version": "v7"}})
+        n = Note(lazy="x")
+        session.add(n)
+        session.commit()
+        token = future.seal("from the future")
+        session.execute(text("UPDATE notes SET lazy = :t WHERE id = :id"), {"t": token, "id": n.id})
+        session.commit()
+        session.expire(n)
+        with pytest.raises(UnknownKeyVersionError):
+            n.lazy
+        Current.sf = create_sealed_fields(
+            namespaces={"PII": {"key": KEY_A}}, on_unknown_version="passthrough"
+        )
+        session.expire(n)
+        assert n.lazy == token  # migration window: the token comes back as-is
+
+
+class TestModes:
+    def test_strict_without_key_refuses_to_write(self, session):
+        Current.sf = create_sealed_fields(namespaces={"PII": {}})
+        session.add(Note(lazy="pii"))
+        with pytest.raises(sa.exc.StatementError) as excinfo:
+            session.commit()
+        assert isinstance(excinfo.value.orig, KeyNotConfiguredError)
+
+    def test_dormant_without_key_writes_plaintext_and_warns_once(self, session):
+        warnings = []
+        Current.sf = create_sealed_fields(
+            namespaces={"PII": {}}, mode="dormant", logger=warnings.append
+        )
+        n = Note(lazy="not yet keyed")
+        m = Note(lazy="still not keyed")
+        session.add_all([n, m])
+        session.commit()
+        assert raw(session, "lazy", n.id) == "not yet keyed"
+        assert len(warnings) == 1 and "PLAINTEXT" in warnings[0]
+        session.expire(n)
+        assert n.lazy == "not yet keyed"
+
+    def test_strict_without_key_cannot_read_a_token(self, session):
+        n = Note(lazy="sealed under A")
+        session.add(n)
+        session.commit()
+        Current.sf = create_sealed_fields(namespaces={"PII": {}})
+        session.expire(n)
+        with pytest.raises(KeyNotConfiguredError):
+            n.lazy
+
+
+class TestSealedJSON:
+    def test_round_trips_list_and_dict(self, session):
+        conditions = ["anxiety", "chronic pain", {"nested": [1, 2.5, None, "ü"]}]
+        n = Note(snapshot=conditions)
+        session.add(n)
+        session.commit()
+        stored = raw(session, "snapshot", n.id)
+        assert is_sealed(stored)
+        assert "anxiety" not in stored
+        assert json.loads(SF_A.unseal(stored)) == conditions
+        session.expire(n)
+        assert n.snapshot == conditions
+
+    def test_empty_list_and_none(self, session):
+        n = Note(snapshot=[])
+        m = Note(snapshot=None)
+        session.add_all([n, m])
+        session.commit()
+        assert raw(session, "snapshot", m.id) is None
+        session.expire(n)
+        session.expire(m)
+        assert n.snapshot == []
+        assert m.snapshot is None
+
+    def test_legacy_json_text_row_decodes(self, session):
+        session.execute(
+            text("INSERT INTO notes (snapshot) VALUES (:s)"), {"s": '["couch-lock","paranoia"]'}
+        )
+        session.commit()
+        n = session.execute(select(Note)).scalar_one()
+        assert n.snapshot == ["couch-lock", "paranoia"]
+
+    def test_already_parsed_value_passes_through(self):
+        # A column still typed json/jsonb hands the ORM a parsed value.
+        t = SealedJSON(SF_A)
+        assert t.process_result_value(["a", "b"], None) == ["a", "b"]
+        assert t.process_result_value({"k": 1}, None) == {"k": 1}
+        assert t.process_result_value(None, None) is None
+        assert t.process_result_value("", None) is None
+        assert t.process_result_value("not json", None) == "not json"
+
+    def test_string_value_survives_as_string(self):
+        t = SealedJSON(SF_A)
+        stored = t.process_bind_param("free text", None)
+        assert t.process_result_value(stored, None) == "free text"
+
+
+class TestProviderAndCaching:
+    def test_requires_a_provider(self):
+        with pytest.raises(TypeError):
+            SealedText()
+        with pytest.raises(TypeError):
+            SealedText(provider=42)
+
+    def test_class_attribute_provider_allows_bare_use(self, engine):
+        class EncryptedText(SealedText):
+            cache_ok = True
+            provider = staticmethod(current_sf)
+
+        md = MetaData()
+        t = Table("bare", md, Column("id", Integer, primary_key=True), Column("v", EncryptedText))
+        md.create_all(engine)
+        with engine.begin() as conn:
+            conn.execute(t.insert().values(v="hello"))
+            stored = conn.execute(text("SELECT v FROM bare")).scalar()
+            assert is_sealed(stored)
+            assert conn.execute(select(t.c.v)).scalar() == "hello"
+
+    def test_provider_must_return_sealed_fields(self):
+        t = SealedText(lambda: "nope")
+        with pytest.raises(SealedFieldsError):
+            t.process_bind_param("x", None)
+
+    def test_namespace_is_honored(self):
+        sf = create_sealed_fields(namespaces={"PII": {"key": KEY_A}, "TOTP": {"key": KEY_B}})
+        t = SealedText(sf, namespace="TOTP")
+        token = t.process_bind_param("secret", None)
+        assert sf.unseal(token, "TOTP") == "secret"
+        with pytest.raises(Exception):
+            sf.unseal(token, "PII")
+        assert t.process_result_value(token, None) == "secret"
+
+    def test_cache_key_includes_provider_and_namespace(self):
+        assert SealedText(SF_A)._static_cache_key == SealedText(SF_A)._static_cache_key
+        assert SealedText(SF_A)._static_cache_key != SealedText(SF_B)._static_cache_key
+        sf = create_sealed_fields(namespaces={"PII": {"key": KEY_A}, "TOTP": {"key": KEY_B}})
+        assert (
+            SealedText(sf, namespace="PII")._static_cache_key
+            != SealedText(sf, namespace="TOTP")._static_cache_key
+        )
+
+    def test_compiled_cache_does_not_cross_providers(self, engine):
+        # Bind processors are cached with the compiled statement. Two
+        # statements with identical shape but different providers must each
+        # seal under their own key.
+        md = MetaData()
+        t = Table("plain", md, Column("id", Integer, primary_key=True), Column("v", Text))
+        md.create_all(engine)
+        with engine.begin() as conn:
+            conn.execute(
+                insert(t).values(id=1, v=bindparam("v", type_=SealedText(SF_A))), {"v": "a"}
+            )
+            conn.execute(
+                insert(t).values(id=2, v=bindparam("v", type_=SealedText(SF_B))), {"v": "b"}
+            )
+            rows = dict(conn.execute(text("SELECT id, v FROM plain")).all())
+        assert SF_A.unseal(rows[1]) == "a"
+        assert SF_B.unseal(rows[2]) == "b"
+
+    def test_string_impl_kwargs_pass_through(self):
+        t = SealedString(SF_A, length=512)
+        assert t.impl.length == 512

sealed_fields-0.2.0/PKG-INFO

@@ -1,45 +0,0 @@
-Metadata-Version: 2.4
-Name: sealed-fields
-Version: 0.2.0
-Summary: Field-level encryption for PII at rest: AES-256-GCM with versioned key rotation, key namespaces, HMAC blind indexes, and row helpers with systemic-failure detection. Wire-compatible with the sealed-fields npm package.
-Project-URL: Homepage, https://github.com/mpointer/sealed-fields
-Project-URL: Issues, https://github.com/mpointer/sealed-fields/issues
-Author-email: Michael Pointer <mpointer@gmail.com>
-License-Expression: MIT
-Keywords: aes-256-gcm,blind-index,encryption,encryption-at-rest,field-encryption,key-rotation,pii,searchable-encryption,sqlalchemy
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3
-Classifier: Topic :: Security :: Cryptography
-Requires-Python: >=3.10
-Requires-Dist: cryptography>=41
-Description-Content-Type: text/markdown
-
-# sealed-fields (Python)
-
-Field-level encryption for PII at rest. This is the Python twin of the
-[sealed-fields npm package](https://www.npmjs.com/package/sealed-fields):
-identical wire format, identical API shape, shared cross-language test
-vectors. Data sealed by either implementation unseals in the other.
-
-```
-pip install sealed-fields
-```
-
-```python
-from sealed_fields import create_sealed_fields, generate_key
-
-sf = create_sealed_fields(
-    namespaces={"PII": {"key": os.environ["SEALED_KEY_PII"]}},
-    blind_index_key=os.environ.get("SEALED_BLIND_INDEX_KEY"),
-)
-
-token = sf.seal("free-text note about a real person")  # "v1:..."
-sf.unseal(token)                                        # plaintext
-sf.blind_index("alice@example.com")                     # HMAC lookup token
-```
-
-See the [repository README](https://github.com/mpointer/sealed-fields) for
-the full design: key rotation without a flag day, key namespaces, blind
-index entropy rules, row helpers, legacy-format decoders, and the
-fail-closed default.

sealed_fields-0.2.0/README.md

@@ -1,28 +0,0 @@
-# sealed-fields (Python)
-
-Field-level encryption for PII at rest. This is the Python twin of the
-[sealed-fields npm package](https://www.npmjs.com/package/sealed-fields):
-identical wire format, identical API shape, shared cross-language test
-vectors. Data sealed by either implementation unseals in the other.
-
-```
-pip install sealed-fields
-```
-
-```python
-from sealed_fields import create_sealed_fields, generate_key
-
-sf = create_sealed_fields(
-    namespaces={"PII": {"key": os.environ["SEALED_KEY_PII"]}},
-    blind_index_key=os.environ.get("SEALED_BLIND_INDEX_KEY"),
-)
-
-token = sf.seal("free-text note about a real person")  # "v1:..."
-sf.unseal(token)                                        # plaintext
-sf.blind_index("alice@example.com")                     # HMAC lookup token
-```
-
-See the [repository README](https://github.com/mpointer/sealed-fields) for
-the full design: key rotation without a flag day, key namespaces, blind
-index entropy rules, row helpers, legacy-format decoders, and the
-fail-closed default.