dataframely 2.0.0__tar.gz → 2.2.0__tar.gz

dataframely-2.2.0/.github/copilot-instructions.md

@@ -0,0 +1,214 @@
+# Dataframely - Coding Agent Instructions
+
+## Project Overview
+
+Dataframely is a declarative, polars-native data frame validation library. It validates schemas and data content in
+polars DataFrames using native polars expressions and a custom Rust-based polars plugin for high performance. It
+supports validating individual data frames via `Schema` classes and interconnected data frames via `Collection` classes.
+
+## Tech Stack
+
+### Core Technologies
+
+- **Python**: Primary language for the public API
+- **Rust**: Backend for polars plugin and custom regex operations
+- **Polars**: Only supported data frame library
+- **pyo3 & maturin**: Rust-Python bindings and build system
+- **pixi**: Primary environment and task manager (NOT pip/conda directly)
+
+### Build System
+
+- **maturin**: Builds the Rust extension module `dataframely._native`
+- **Cargo**: Rust dependency management
+- Rust toolchain specified in `rust-toolchain.toml` with clippy and rustfmt components
+
+## Environment Setup
+
+**CRITICAL**: Always use `pixi` commands - never run `pip`, `conda`, `python`, or `cargo` directly unless specifically
+required for Rust-only operations.
+
+### Initial Setup
+
+Unless already performed via external setup steps:
+
+```bash
+# Install Rust toolchain
+rustup show
+
+# Install pixi environment and dependencies
+pixi install
+
+# Build and install the package locally (REQUIRED after Rust changes)
+pixi run postinstall
+```
+
+### After Rust Code Changes
+
+**Always run** `pixi run postinstall` after modifying any Rust code in `src/` to rebuild the native extension.
+
+## Development Workflow
+
+### Running Tests
+
+```bash
+# Run all tests (excludes S3 tests by default)
+pixi run test
+
+# Run tests with S3 backend (requires moto server)
+pixi run test -m s3
+
+# Run specific test file or directory
+pixi run test tests/schema/
+
+# Run with coverage
+pixi run test-coverage
+
+# Run benchmarks
+pixi run test-bench
+```
+
+### Code Quality
+
+**NEVER** run linters/formatters directly. Use pre-commit:
+
+```bash
+# Run all pre-commit hooks
+pixi run pre-commit run
+```
+
+Pre-commit handles:
+
+- **Python**: ruff (lint & format), mypy (type checking), docformatter
+- **Rust**: cargo fmt, cargo clippy
+- **Other**: prettier (md/yml), taplo (toml), license headers, trailing whitespace
+
+### Building Documentation
+
+```bash
+# Build documentation
+pixi run -e docs postinstall
+pixi run docs
+
+# Open in browser (macOS)
+open docs/_build/html/index.html
+```
+
+## Project Structure
+
+```
+dataframely/              # Python package
+  schema.py              # Core Schema class for DataFrame validation
+  collection/            # Collection class for validating multiple interconnected DataFrames
+  columns/               # Column type definitions (String, Integer, Float, etc.)
+  testing/               # Testing utilities (factories, masks, storage mocks)
+  _storage/              # Storage backends (Parquet, Delta Lake)
+  _rule.py               # Rule decorator for validation rules
+  _plugin.py             # Polars plugin registration
+  _native.pyi            # Type stubs for Rust extension
+
+src/                     # Rust source code
+  lib.rs                 # PyO3 module definition
+  polars_plugin/         # Custom polars plugin for validation
+  regex/                 # Custom regex operations
+
+tests/                   # Unit tests (mirrors dataframely/ structure)
+  benches/               # Benchmark tests
+  conftest.py            # Shared pytest fixtures (including s3_server)
+
+docs/                    # Sphinx documentation
+  guides/                # User guides and examples
+  api/                   # Auto-generated API reference
+```
+
+## Pixi Environments
+
+Multiple environments for different purposes:
+
+- **default**: Base Python + core dependencies
+- **dev**: Includes jupyter for notebooks
+- **test**: Testing dependencies (pytest, moto, boto3, etc.)
+- **docs**: Documentation building (sphinx, myst-parser, etc.)
+- **lint**: Linting and formatting tools
+- **optionals**: Optional dependencies (pydantic, deltalake, pyarrow, sqlalchemy)
+- **py310-py314**: Python version-specific environments
+
+Use `-e <env>` to run commands in specific environments:
+
+```bash
+pixi run -e test test
+pixi run -e docs docs
+```
+
+## API Design Principles
+
+### Critical Guidelines
+
+1. **NO BREAKING CHANGES**: Public API must remain backward compatible
+2. **100% Test Coverage**: All new code requires tests
+3. **Documentation Required**: All public features need docstrings + API docs
+4. **Cautious API Extension**: Avoid adding to public API unless necessary
+
+### Public API
+
+Public exports are in `dataframely/__init__.py`. Main components:
+
+- **Schema classes**: `Schema` for DataFrame validation
+- **Collection classes**: `Collection`, `CollectionMember` for multi-DataFrame validation
+- **Column types**: `String`, `Integer`, `Float`, `Bool`, `Date`, `Datetime`, etc.
+- **Decorators**: `@rule()`, `@filter()`
+- **Type hints**: `DataFrame[Schema]`, `LazyFrame[Schema]`, `Validation`
+
+## Common Pitfalls & Solutions
+
+### S3 Testing
+
+The `s3_server` fixture in `tests/conftest.py` uses `subprocess.Popen` to start moto_server on port 9999. This is a **workaround** for a polars issue with ThreadedMotoServer. When the polars issue is fixed, it should be replaced with ThreadedMotoServer (code is commented in the file).
+
+**Note**: CI skips S3 tests by default. Run with `pixi run test -m s3` when modifying storage backends.
+
+## Testing Strategy
+
+- Tests are organized by module, mirroring the `dataframely/` structure
+- Use `dy.Schema.sample()` for generating test data
+- Test both eager (`DataFrame`) and lazy (`LazyFrame`) execution
+- S3 tests use moto server fixture from `conftest.py`
+- Benchmark tests in `tests/benches/` use pytest-benchmark
+
+## Validation Pattern
+
+Typical usage pattern:
+
+```python
+class MySchema(dy.Schema):
+    col = dy.String(nullable=False)
+
+    @dy.rule()
+    def my_rule(cls) -> pl.Expr:
+        return pl.col("col").str.len_chars() > 0
+
+# Validate and cast
+validated_df: dy.DataFrame[MySchema] = MySchema.validate(df, cast=True)
+```
+
+## Key Configuration Files
+
+- `pixi.toml`: Environment and task definitions
+- `pyproject.toml`: Python package metadata, tool configurations (ruff, mypy, pytest)
+- `Cargo.toml`: Rust dependencies and build settings
+- `.pre-commit-config.yaml`: All code quality checks
+- `rust-toolchain.toml`: Rust nightly version specification
+
+## When Making Changes
+
+1. **Python code**: Run `pixi run pre-commit run` before committing
+2. **Rust code**: Run `pixi run postinstall` to rebuild, then run tests
+3. **Tests**: Ensure `pixi run test` passes. If changes might affect storage backends, use `pixi run test -m s3`.
+4. **Documentation**: Update docstrings
+5. **API changes**: Ensure backward compatibility or document migration path
+
+## Performance Considerations
+
+- Validation uses native polars expressions for performance
+- Custom Rust plugin for advanced validation logic
+- Lazy evaluation supported via `LazyFrame` for large datasets
+- Avoid materializing data unnecessarily in validation rules

dataframely-2.2.0/.github/instructions/tests.instructions.md

@@ -0,0 +1,9 @@
+---
+applyTo: tests/**/*.py
+---
+
+# Testing Guidelines
+
+- Instead of writing new tests, existing tests should ideally be parametrized using `@pytest.mark.parametrize`
+  unless the parametrization is impractical, e.g. by adding more parameters
+- Tests should not use docstrings unless they are _very_ complex

{dataframely-2.0.0 → dataframely-2.2.0}/.github/workflows/build.yml

@@ -17,7 +17,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Set up pixi
-        uses: prefix-dev/setup-pixi@194d461b21b6c5717c722ffc597fa91ed2ff29fa # v0.9.1
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
         with:
           environments: build
       - name: Set version
@@ -25,7 +25,7 @@ jobs:
       - name: Build project
         run: pixi run -e build build-sdist
       - name: Upload package
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
         with:
           name: sdist
           path: dist/*
@@ -42,7 +42,7 @@ jobs:
           - target-platform: linux-aarch64
             os: ubuntu-24.04-arm
           - target-platform: osx-64
-            os: macos-13
+            os: macos-15-intel
           - target-platform: osx-arm64
             os: macos-latest
           - target-platform: win-64
@@ -52,7 +52,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Set up pixi
-        uses: prefix-dev/setup-pixi@194d461b21b6c5717c722ffc597fa91ed2ff29fa # v0.9.1
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
         with:
           environments: build
       - name: Set version
@@ -70,7 +70,7 @@ jobs:
       - name: Check package
         run: pixi run -e build check-wheel
       - name: Upload package
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
         with:
           name: wheel-${{ matrix.target-platform }}
           path: dist/*
@@ -84,7 +84,7 @@ jobs:
       id-token: write
     environment: pypi
     steps:
-      - uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
+      - uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
         with:
           path: dist
           merge-multiple: true

{dataframely-2.0.0 → dataframely-2.2.0}/.github/workflows/ci.yml

@@ -24,7 +24,7 @@ jobs:
           # needed for 'pre-commit-mirrors-insert-license'
           fetch-depth: 0
       - name: Set up pixi
-        uses: prefix-dev/setup-pixi@194d461b21b6c5717c722ffc597fa91ed2ff29fa # v0.9.1
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
         with:
           environments: default lint
       - name: Install Rust
@@ -58,7 +58,7 @@ jobs:
       - name: Checkout branch
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
       - name: Set up pixi
-        uses: prefix-dev/setup-pixi@194d461b21b6c5717c722ffc597fa91ed2ff29fa # v0.9.1
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
         with:
           environments: ${{ matrix.environment }}
           # FIXME: Remove when `s3_server` fixture does not start a process anymore

dataframely-2.2.0/.github/workflows/copilot-setup-steps.yml

@@ -0,0 +1,26 @@
+name: Copilot Setup Steps
+on:
+  pull_request:
+    paths:
+      - .github/workflows/copilot-setup-steps.yml
+  workflow_dispatch:
+
+jobs:
+  copilot-setup-steps:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
+    steps:
+      - name: Checkout branch
+        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+      - name: Set up pixi
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
+        with:
+          environments: default
+      - name: Install Rust
+        run: rustup show
+      - name: Cache Rust dependencies
+        uses: Swatinem/rust-cache@f13886b937689c021905a6b90929199931d60db1 # v2.8.1
+      - name: Install repository
+        run: pixi run postinstall

{dataframely-2.0.0 → dataframely-2.2.0}/.github/workflows/nightly.yml

@@ -25,7 +25,7 @@ jobs:
       - name: Checkout branch
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
       - name: Set up pixi
-        uses: prefix-dev/setup-pixi@194d461b21b6c5717c722ffc597fa91ed2ff29fa # v0.9.1
+        uses: prefix-dev/setup-pixi@28eb668aafebd9dede9d97c4ba1cd9989a4d0004 # v0.9.2
         with:
           environments: nightly
       - name: Install polars nightly

{dataframely-2.0.0 → dataframely-2.2.0}/.github/workflows/scorecard.yml

@@ -65,7 +65,7 @@ jobs:
       # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
       # format to the repository Actions tab.
       - name: "Upload artifact"
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
         with:
           name: SARIF file
           path: results.sarif
@@ -74,6 +74,6 @@ jobs:
       # Upload the results to GitHub's code scanning dashboard (optional).
       # Commenting out will disable upload of results to your repo's Code Scanning dashboard
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.29.5
+        uses: github/codeql-action/upload-sarif@0499de31b99561a6d14a36a5f662c2a54f91beee # v3.29.5
         with:
           sarif_file: results.sarif

{dataframely-2.0.0 → dataframely-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dataframely
-Version: 2.0.0
+Version: 2.2.0
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
@@ -109,5 +109,5 @@ df = pl.DataFrame({
 validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)
 ```
 
-See more advanced usage examples in the [documentation](https://dataframely.readthedocs.io/en/latest/).
+See more advanced usage examples in the [documentation](https://dataframely.readthedocs.io/stable/).
 

{dataframely-2.0.0 → dataframely-2.2.0}/README.md

@@ -78,4 +78,4 @@ df = pl.DataFrame({
 validated_df: dy.DataFrame[HouseSchema] = HouseSchema.validate(df, cast=True)
 ```
 
-See more advanced usage examples in the [documentation](https://dataframely.readthedocs.io/en/latest/).
+See more advanced usage examples in the [documentation](https://dataframely.readthedocs.io/stable/).

{dataframely-2.0.0 → dataframely-2.2.0}/dataframely/__init__.py

@@ -51,6 +51,7 @@ from .columns import (
     UInt64,
 )
 from .config import Config
+from .exc import DeserializationError
 from .filter_result import FailureInfo
 from .functional import (
     concat_collection_members,
@@ -106,4 +107,5 @@ __all__ = [
     "Array",
     "Object",
     "Validation",
+    "DeserializationError",
 ]

{dataframely-2.0.0 → dataframely-2.2.0}/dataframely/_base_schema.py

@@ -8,7 +8,7 @@ import textwrap
 from abc import ABCMeta
 from copy import copy
 from dataclasses import dataclass, field
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
 import polars as pl
 
@@ -164,12 +164,16 @@ class SchemaMeta(ABCMeta):
 
         return cls
 
-    def __getattribute__(cls, name: str) -> Any:
-        val = super().__getattribute__(name)
-        # Dynamically set the name of the column if it is a `Column` instance.
-        if isinstance(val, Column):
-            val._name = val.alias or name
-        return val
+    if not TYPE_CHECKING:
+        # Only define __getattribute__ at runtime to allow type checkers to properly
+        # validate attribute access. When TYPE_CHECKING is True, type checkers will use
+        # the default metaclass behavior which correctly identifies non-existent attributes.
+        def __getattribute__(cls, name: str) -> Any:
+            val = super().__getattribute__(name)
+            # Dynamically set the name of the column if it is a `Column` instance.
+            if isinstance(val, Column):
+                val._name = val.alias or name
+            return val
 
     @staticmethod
     def _get_metadata_recursively(kls: type[object]) -> Metadata:
@@ -199,9 +203,9 @@ class SchemaMeta(ABCMeta):
     def __repr__(cls) -> str:
         parts = [f'[Schema "{cls.__name__}"]']
         parts.append(textwrap.indent("Columns:", prefix=" " * 2))
-        for name, col in cls.columns().items():
+        for name, col in cls.columns().items():  # type: ignore[attr-defined]
             parts.append(textwrap.indent(f'- "{name}": {col!r}', prefix=" " * 4))
-        if validation_rules := cls._schema_validation_rules():
+        if validation_rules := cls._schema_validation_rules():  # type: ignore[attr-defined]
             parts.append(textwrap.indent("Rules:", prefix=" " * 2))
             for name, rule in validation_rules.items():
                 parts.append(textwrap.indent(f'- "{name}": {rule!r}', prefix=" " * 4))

{dataframely-2.0.0 → dataframely-2.2.0}/dataframely/_pydantic.py

@@ -20,12 +20,12 @@ _S = TypeVar("_S", bound=BaseSchema)
 def _dict_to_df(schema_type: type[BaseSchema], data: dict) -> pl.DataFrame:
     return pl.from_dict(
         data,
-        schema=schema_type.to_polars_schema(),
+        schema=schema_type.to_polars_schema(),  # type: ignore[attr-defined]
     )
 
 
 def _validate_df_schema(schema_type: type[_S], df: pl.DataFrame) -> DataFrame[_S]:
-    if not schema_type.is_valid(df):
+    if not schema_type.is_valid(df):  # type: ignore[attr-defined]
         raise ValueError("DataFrame violates schema")
     return df  # type: ignore
 

{dataframely-2.0.0 → dataframely-2.2.0}/dataframely/collection/collection.py

@@ -12,7 +12,7 @@ from collections.abc import Iterable, Mapping, Sequence
 from dataclasses import asdict
 from json import JSONDecodeError
 from pathlib import Path
-from typing import IO, Annotated, Any, Literal, cast
+from typing import IO, Annotated, Any, Literal, cast, overload
 
 import polars as pl
 import polars.exceptions as plexc
@@ -33,7 +33,11 @@ from dataframely._storage.constants import COLLECTION_METADATA_KEY
 from dataframely._storage.delta import DeltaStorageBackend
 from dataframely._storage.parquet import ParquetStorageBackend
 from dataframely._typing import LazyFrame, Validation
-from dataframely.exc import ValidationError, ValidationRequiredError
+from dataframely.exc import (
+    DeserializationError,
+    ValidationError,
+    ValidationRequiredError,
+)
 from dataframely.filter_result import FailureInfo
 from dataframely.random import Generator
 from dataframely.schema import _schema_from_dict
@@ -531,6 +535,25 @@ class Collection(BaseCollection, ABC):
         Raises:
             ValueError: If an insufficient set of input data frames is provided, i.e. if
                 any required member of this collection is missing in the input.
+
+        Example:
+
+            .. code-block:: python
+
+                # Define collection
+                class HospitalInvoiceData(dy.Collection):
+                    invoice: dy.LazyFrame[InvoiceSchema]
+                    ...
+
+                # Filter the data and cast columns to expected types
+                good, failure = HospitalInvoiceData.filter(df, cast=True)
+
+                # Inspect the reasons for the failed rows for member `invoice`
+                print(failure.invoice.counts())
+
+                # Inspect the failed rows
+                failed_df = failure.invoice.invalid()
+                print(failed_df)
         """
         cls._validate_input_keys(data)
 
@@ -872,13 +895,13 @@ class Collection(BaseCollection, ABC):
                 - `"allow"`: The method tries to read the schema data from the parquet
                   files. If the stored collection schema matches this collection
                   schema, the collection is read without validation. If the stored
-                  schema mismatches this schema no metadata can be found in
+                  schema mismatches this schema, no valid metadata can be found in
                   the parquets, or the files have conflicting metadata,
                   this method automatically runs :meth:`validate` with `cast=True`.
                 - `"warn"`: The method behaves similarly to `"allow"`. However,
                   it prints a warning if validation is necessary.
                 - `"forbid"`: The method never runs validation automatically and only
-                  returns if the metadata stores a collection schema that matches
+                  returns if the metadata stores a valid collection schema that matches
                   this collection.
                 - `"skip"`: The method never runs validation and simply reads the
                   data, entrusting the user that the schema is valid. *Use this option
@@ -1165,7 +1188,12 @@ class Collection(BaseCollection, ABC):
                 members=cls.member_schemas().keys(), **kwargs
             )
 
-        collection_types = _deserialize_types(serialized_collection_types)
+        # Use strict=False when validation is "allow", "warn" or "skip" to tolerate
+        # missing or broken collection metadata.
+        strict = validation == "forbid"
+        collection_types = _deserialize_types(
+            serialized_collection_types, strict=strict
+        )
         collection_type = _reconcile_collection_types(collection_types)
 
         if cls._requires_validation_for_reading_parquets(collection_type, validation):
@@ -1226,14 +1254,27 @@ def read_parquet_metadata_collection(
     """
     metadata = pl.read_parquet_metadata(source)
     if (schema_metadata := metadata.get(COLLECTION_METADATA_KEY)) is not None:
-        try:
-            return deserialize_collection(schema_metadata)
-        except (JSONDecodeError, plexc.ComputeError):
-            return None
+        return deserialize_collection(schema_metadata, strict=False)
     return None
 
 
-def deserialize_collection(data: str) -> type[Collection]:
+@overload
+def deserialize_collection(
+    data: str, strict: Literal[True] = True
+) -> type[Collection]: ...
+
+
+@overload
+def deserialize_collection(
+    data: str, strict: Literal[False]
+) -> type[Collection] | None: ...
+
+
+@overload
+def deserialize_collection(data: str, strict: bool) -> type[Collection] | None: ...
+
+
+def deserialize_collection(data: str, strict: bool = True) -> type[Collection] | None:
     """Deserialize a collection from a JSON string.
 
     This method allows to dynamically load a collection from its serialization, without
@@ -1241,12 +1282,14 @@ def deserialize_collection(data: str) -> type[Collection]:
 
     Args:
         data: The JSON string created via :meth:`Collection.serialize`.
+        strict: Whether to raise an exception if the collection cannot be deserialized.
 
     Returns:
         The collection loaded from the JSON data.
 
     Raises:
-        ValueError: If the schema format version is not supported.
+        DeserializationError: If the collection can not be deserialized
+            and `strict=True`.
 
     Attention:
         The returned collection **cannot** be used to create instances of the
@@ -1261,34 +1304,41 @@ def deserialize_collection(data: str) -> type[Collection]:
     See also:
         :meth:`Collection.serialize` for additional information on serialization.
     """
-    decoded = json.loads(data, cls=SchemaJSONDecoder)
-    if (format := decoded["versions"]["format"]) != SERIALIZATION_FORMAT_VERSION:
-        raise ValueError(f"Unsupported schema format version: {format}")
-
-    annotations: dict[str, Any] = {}
-    for name, info in decoded["members"].items():
-        lf_type = LazyFrame[_schema_from_dict(info["schema"])]  # type: ignore
-        if info["is_optional"]:
-            lf_type = lf_type | None  # type: ignore
-        annotations[name] = Annotated[
-            lf_type,
-            CollectionMember(
-                ignored_in_filters=info["ignored_in_filters"],
-                inline_for_sampling=info["inline_for_sampling"],
-            ),
-        ]
-
-    return type(
-        f"{decoded['name']}_dynamic",
-        (Collection,),
-        {
-            "__annotations__": annotations,
-            **{
-                name: Filter(logic=lambda _, logic=logic: logic)  # type: ignore
-                for name, logic in decoded["filters"].items()
+    try:
+        decoded = json.loads(data, cls=SchemaJSONDecoder)
+        if (format := decoded["versions"]["format"]) != SERIALIZATION_FORMAT_VERSION:
+            raise ValueError(f"Unsupported schema format version: {format}")
+
+        annotations: dict[str, Any] = {}
+        for name, info in decoded["members"].items():
+            lf_type = LazyFrame[_schema_from_dict(info["schema"])]  # type: ignore
+            if info["is_optional"]:
+                lf_type = lf_type | None  # type: ignore
+            annotations[name] = Annotated[
+                lf_type,
+                CollectionMember(
+                    ignored_in_filters=info["ignored_in_filters"],
+                    inline_for_sampling=info["inline_for_sampling"],
+                ),
+            ]
+
+        return type(
+            f"{decoded['name']}_dynamic",
+            (Collection,),
+            {
+                "__annotations__": annotations,
+                **{
+                    name: Filter(logic=lambda _, logic=logic: logic)  # type: ignore
+                    for name, logic in decoded["filters"].items()
+                },
             },
-        },
-    )
+        )
+    except (ValueError, TypeError, JSONDecodeError, plexc.ComputeError) as e:
+        if strict:
+            raise DeserializationError(
+                "The Collection metadata could not be deserialized"
+            ) from e
+        return None
 
 
 # --------------------------------------- UTILS -------------------------------------- #
@@ -1314,14 +1364,15 @@ def _extract_keys_if_exist(
 
 def _deserialize_types(
     serialized_collection_types: Iterable[str | None],
+    strict: bool = True,
 ) -> list[type[Collection]]:
     collection_types = []
-    collection_type: type[Collection] | None = None
     for t in serialized_collection_types:
         if t is None:
             continue
-        collection_type = deserialize_collection(t)
-        collection_types.append(collection_type)
+        collection_type = deserialize_collection(t, strict=strict)
+        if collection_type is not None:
+            collection_types.append(collection_type)
 
     return collection_types