datamasque-python 1.2.1__tar.gz → 1.2.3__tar.gz

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/HISTORY.rst

@@ -2,6 +2,27 @@
 History
 =======
 
+1.2.3 (2026-08-12)
+------------------
+
+* Added file-discovery value counts:
+
+  * ``value_count`` on ``FileDiscoveryLocatorResult``, the total over the files in the group.
+  * ``value_counts`` and ``value_count_status`` on ``FileDiscoveryFile``, typed by ``ValueCountStatus``.
+
+Requires server version 3.26.15
+
+1.2.2 (2026-07-31)
+------------------
+
+* Added ``validation_error_details`` to ``DiscoveryConfig``.
+* Added ``usage_count`` to ``DiscoveryConfigLibrary``.
+* ``create_discovery_config``, ``update_discovery_config``, ``create_discovery_config_library``, and
+  ``update_discovery_config_library`` now raise ``ValueError`` when the passed entity has no ``yaml``
+  content, instead of sending a request the server rejects.
+
+Requires server version 3.26.14
+
 1.2.1 (2026-07-30)
 ------------------
 

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: datamasque-python
-Version: 1.2.1
+Version: 1.2.3
 Summary: Official Python client for the DataMasque data-masking API.
 Project-URL: Homepage, https://datamasque.com/
 Project-URL: Documentation, https://datamasque-python.readthedocs.io/

datamasque_python-1.2.3/RELEASING.rst

@@ -0,0 +1,223 @@
+=========
+Releasing
+=========
+
+For DataMasque staff / maintainers of this project.
+Covers production releases to PyPI, dev releases to TestPyPI, and how downstream projects pin ``datamasque-python``.
+
+The assumed starting point is that you have a ``datamasque-python`` feature/fix PR in progress,
+and want to know how to get it pushed up to PyPI once merged,
+or how to build a test release / pin to a particular commit so you can test before merging the PR.
+
+Version numbers live in two places
+==================================
+
+Every release must update both, or CI will reject the tag.
+As part of your PR, update:
+
+1. ``pyproject.toml`` — ``[project] version``.
+   This is the version the built sdist/wheel carries,
+   and the one ``release.yml`` checks the git tag against.
+2. ``uv.lock`` — the ``version`` field of the ``datamasque-python`` package entry.
+   CI runs ``uv sync --frozen``, so a stale lock fails every job.
+
+Edit ``pyproject.toml`` by hand, then run ``uv lock`` to update the lock entry.
+Don't hand-edit ``uv.lock``.
+
+Don't update ``datamasque.client.__version__`` or ``docs/conf.py``.
+These read the installed package metadata.
+
+Updating the changelog
+======================
+
+Do this as part of your PR.
+
+``HISTORY.rst`` needs a new entry for each release to pypi.
+Add a new entry at the top, under the ``History`` heading, with the new version and today's date.
+Use short summaries of additions and changes, written with past tense and worded for a customer consumer of this library.
+For example::
+
+       1.2.2 (2026-08-05)
+       ------------------
+
+       * Added ``get_all_widgets`` API.
+       * Fixed issue where ``list_frobs`` would incorrectly raise a ``ValueError`` when using the filtering options.
+
+       Requires server version 3.26.14.
+
+Call out breaking changes explicitly,
+and note the minimum DataMasque server version if the release requires one.
+
+Releasing to PyPI
+=================
+
+1. Get your PR approved and merged.
+2. Tag the merge commit on ``main`` and push the tag::
+
+       git checkout main
+       git pull --ff-only
+       git tag v1.2.2  # or whatever is your version number
+       git push origin v1.2.2
+
+   The tag must be ``v`` + the exact ``pyproject.toml`` version.
+   ``release.yml`` triggers on ``v*.*.*`` and fails the build if the two disagree.
+   Tag only after the PR has merged, so the tag points at a commit on ``main``.
+3. The admins of the repo will get an automated email asking them to approve the deployment.
+   They can do this by visiting the Workflows page or by clicking the link in the email.
+4. Once approved, the build will appear on PyPI very quickly (around 1 minute).
+   You can verify that the release appears at https://pypi.org/project/datamasque-python/,
+   and Read the Docs has built the new tag
+   at https://datamasque-python.readthedocs.io/.
+5. Create the GitHub Release (see below).
+
+Creating the GitHub Release
+===========================
+
+Do this after the PyPI publish has succeeded,
+so the release never points at a version nobody can install.
+
+1. On the repository home page,
+   find **Releases** in the right-hand sidebar under the *About* box,
+   and click it,
+   then **Draft a new release**.
+2. **Choose a tag** — pick the existing tag you pushed as part of releasing to PyPI.
+   Don't let GitHub create a new one; the tag should already exist from the release flow.
+3. Set the target to ``main``.
+4. Title it the same as the tag, e.g. ``v1.2.2``.
+5. Paste the ``HISTORY.rst`` entry for this version into the body, converted to Markdown.
+   **Generate release notes** is a reasonable starting point for the commit list,
+   but the hand-written changelog is what users read.
+6. Leave *Set as a pre-release* unticked for a normal release.
+7. Click **Publish release**.
+
+Creating releases can also be done from the ``gh`` CLI::
+
+    gh release create v1.2.2 --title v1.2.2 --notes-file notes.md
+
+Releasing a dev build to TestPyPI
+=================================
+
+Use this to let a downstream project try an unreleased change.
+It builds from any ref and never touches PyPI.
+
+1. Push the branch you want to publish.
+   Nothing needs to be merged first.
+2. Go to *Actions* → *Release (TestPyPI)* → **Run workflow**.
+3. Fill in the inputs:
+
+   - **ref** — the branch to build from.
+   - **dev_version** — the version to publish, for example ``1.3.0.dev1``.
+
+   Use a ``.devN`` suffix.
+   The input is validated, so a plain release version like ``1.3.0`` is rejected.
+
+   **Note:** Version numbers cannot be reused. Ensure you increment the suffix each time.
+
+4. No approval is needed — unlike a PyPI release, the build publishes as soon as it passes.
+5. Verify at https://test.pypi.org/project/datamasque-python/.
+
+**Note:** Do **not** create a git tag or a GitHub Release for a TestPyPI build.
+
+Pointing a project at a release
+===============================
+
+Three ways to depend on ``datamasque-python``, depending on what you need.
+
+A released version on PyPI
+--------------------------
+
+The usual case.
+In the consuming project's ``pyproject.toml``::
+
+    dependencies = [
+        "datamasque-python==1.2.2",
+    ]
+
+Always give a version specifier.
+An unpinned ``"datamasque-python"`` means "whatever is newest at the moment this resolves",
+which is prone to breaking when someone releases a ``datamasque-python`` change before the corresponding server release.
+
+Pin exactly (``==1.2.2``) when you want every upgrade to be a deliberate, reviewed change.
+A bounded range (``>=1.2.2,<2``) is acceptable where you need a floor for a specific feature,
+but note that this project does sometimes ship breaking changes
+(e.g. 1.2.0 removed ``config_type`` from the discovery config library APIs),
+so read the ``HISTORY.rst`` entries between your current version and the new one before raising the floor.
+
+A dev build on TestPyPI
+-----------------------
+
+TestPyPI is a separate index, so it must be declared as well as pinned.
+Pin the exact dev version — TestPyPI content is disposable.
+
+**Note:** Always replace a TestPyPI pin with a real PyPI version before merging the consuming project —
+TestPyPI is not a durable index,
+and the downstream project should always use a "proper" (normal PyPI) version of ``datamasque-python`` once merged.
+
+With ``uv``::
+
+    [project]
+    dependencies = [
+        "datamasque-python==1.3.0.dev1",
+    ]
+
+    [[tool.uv.index]]
+    name = "testpypi"
+    url = "https://test.pypi.org/simple/"
+    explicit = true
+
+    [tool.uv.sources]
+    datamasque-python = { index = "testpypi" }
+
+**Note:** Be sure to include the ``explicit = true``:
+without it, ``uv`` will resolve *every* dependency against TestPyPI, which mirrors PyPI only partially.
+
+With Poetry::
+
+    [[tool.poetry.source]]
+    name = "testpypi"
+    url = "https://test.pypi.org/simple/"
+    priority = "explicit"
+
+    [tool.poetry.dependencies]
+    datamasque-python = { version = "1.3.0.dev1", source = "testpypi" }
+
+Ad hoc, without editing anything::
+
+    pip install \
+        --index-url https://test.pypi.org/simple/ \
+        --extra-index-url https://pypi.org/simple/ \
+        datamasque-python==1.3.0.dev1
+
+
+A branch or commit on GitHub
+----------------------------
+
+For testing an unmerged change without publishing anything at all.
+Prefer a commit SHA over a branch name — a branch ref can change under you, and a lockfile pinned to one is not reproducible.
+
+**Note:** Always replace a git pin with a PyPI version before merging the consuming project,
+for the same reasons as above, plus the fact git dependencies break any downstream install that lacks repository access.
+
+``uv`` sources form, which keeps the version constraint readable::
+
+    [project]
+    dependencies = [
+        "datamasque-python",
+    ]
+
+    [tool.uv.sources]
+    datamasque-python = { git = "https://github.com/datamasque/datamasque-python.git", rev = "ab12cd34" }
+
+With Poetry::
+
+    [tool.poetry.dependencies]
+    datamasque-python = { git = "https://github.com/datamasque/datamasque-python.git", rev = "ab12cd34" }
+
+Use ``git+ssh://git@github.com/...`` instead of ``https`` if the consumer builds somewhere without an HTTPS credential for the repository.
+
+PEP 508 direct reference,
+which ``pip`` and ``uv`` both understand::
+
+    dependencies = [
+        "datamasque-python @ git+https://github.com/datamasque/datamasque-python.git@ab12cd34",
+    ]

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/__init__.py

@@ -81,6 +81,7 @@ from datamasque.client.models.discovery import (
     SchemaDiscoveryRequest,
     SchemaDiscoveryResult,
     TableConstraints,
+    ValueCountStatus,
 )
 from datamasque.client.models.discovery_config import DiscoveryConfig, DiscoveryConfigId, DiscoveryConfigType
 from datamasque.client.models.discovery_config_library import DiscoveryConfigLibrary, DiscoveryConfigLibraryId
@@ -319,4 +320,5 @@ __all__ = [
     "ValidationErrorDetails",
     "ValidationErrorType",
     "ValidationStatus",
+    "ValueCountStatus",
 ]

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/discovery_config_libraries.py

@@ -76,15 +76,19 @@ class DiscoveryConfigLibraryClient(BaseClient):
         Creates a new discovery config library on the server.
 
         Sets the library's server-assigned fields
-        (`id`, `is_valid`, `validation_error`, `created`, `modified`) and returns the library.
+        (`id`, `is_valid`, `validation_error`, `usage_count`, `created`, `modified`) and returns the library.
         """
 
+        if not library.yaml:
+            raise ValueError("Cannot create a discovery config library without YAML content (yaml is empty)")
+
         data = library.model_dump(exclude_none=True, by_alias=True, mode="json")
         response = self.make_request("POST", "/api/discovery/config-libraries/", data=data)
         created = DiscoveryConfigLibrary.model_validate(response.json())
         library.id = created.id
         library.is_valid = created.is_valid
         library.validation_error = created.validation_error
+        library.usage_count = created.usage_count
         library.created = created.created
         library.modified = created.modified
         logger.info('Creation of discovery config library "%s" successful', library.name)
@@ -101,9 +105,9 @@ class DiscoveryConfigLibraryClient(BaseClient):
         if library.id is None:
             raise ValueError("Cannot update a discovery config library that has not been created yet (id is None)")
 
-        if library.yaml is None:
+        if not library.yaml:
             raise ValueError(
-                "Cannot update a discovery config library without YAML content (yaml is None); "
+                "Cannot update a discovery config library without YAML content (yaml is empty or unset); "
                 "list results omit YAML, so fetch the full library with `get_discovery_config_library` first"
             )
 
@@ -112,6 +116,7 @@ class DiscoveryConfigLibraryClient(BaseClient):
         updated = DiscoveryConfigLibrary.model_validate(response.json())
         library.is_valid = updated.is_valid
         library.validation_error = updated.validation_error
+        library.usage_count = updated.usage_count
         library.modified = updated.modified
         logger.debug('Update of discovery config library "%s" successful', library.name)
         return library

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/discovery_configs.py

@@ -75,15 +75,20 @@ class DiscoveryConfigClient(BaseClient):
         Creates a new discovery config on the server.
 
         Sets the config's server-assigned fields
-        (`id`, `is_valid`, `validation_error`, `created`, `modified`) and returns the config.
+        (`id`, `is_valid`, `validation_error`, `validation_error_details`, `created`, `modified`)
+        and returns the config.
         """
 
+        if not config.yaml:
+            raise ValueError("Cannot create a discovery config without YAML content (yaml is empty)")
+
         data = config.model_dump(exclude_none=True, by_alias=True, mode="json")
         response = self.make_request("POST", "/api/discovery/configs/", data=data)
         created = DiscoveryConfig.model_validate(response.json())
         config.id = created.id
         config.is_valid = created.is_valid
         config.validation_error = created.validation_error
+        config.validation_error_details = created.validation_error_details
         config.created = created.created
         config.modified = created.modified
         logger.info('Creation of discovery config "%s" successful', config.name)
@@ -94,17 +99,24 @@ class DiscoveryConfigClient(BaseClient):
         Performs a full update of the discovery config.
 
         The config must have its `id` set
-        (i.e., it must have been previously created or retrieved from the server).
+        and its `yaml` content present.
         """
 
         if config.id is None:
             raise ValueError("Cannot update a discovery config that has not been created yet (id is None)")
 
+        if not config.yaml:
+            raise ValueError(
+                "Cannot update a discovery config without YAML content (yaml is empty or unset); "
+                "list results omit YAML, so fetch the full config with `get_discovery_config` first"
+            )
+
         data = config.model_dump(exclude_none=True, by_alias=True, mode="json")
         response = self.make_request("PUT", f"/api/discovery/configs/{config.id}/", data=data)
         updated = DiscoveryConfig.model_validate(response.json())
         config.is_valid = updated.is_valid
         config.validation_error = updated.validation_error
+        config.validation_error_details = updated.validation_error_details
         config.modified = updated.modified
         logger.debug('Update of discovery config "%s" successful', config.name)
         return config

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/models/data_selection.py

@@ -1,5 +1,6 @@
 """Models related to data selection in endpoints such as /api/async-generate-ruleset."""
 
+import json
 from typing import Optional, Union
 
 from pydantic import BaseModel, ConfigDict
@@ -19,6 +20,33 @@ A locator identifying a masked value within a file.
 """
 
 
+def serialize_locator(locator: Locator) -> str:
+    """
+    Returns the string form of `locator` that the API uses as a key, such as in `FileDiscoveryFile.value_counts`.
+
+    A string locator is its own key.
+    A :data:`JsonPath` becomes compact JSON, e.g. `'["employees","*","email"]'`.
+    """
+
+    if isinstance(locator, str):
+        return locator
+    return json.dumps(locator, separators=(",", ":"), ensure_ascii=False)
+
+
+def deserialize_locator(key: str) -> Locator:
+    """
+    Returns the `Locator` that `key`, a serialized locator from the API, identifies.
+
+    A key that holds no JSON array names a column, and comes back as a string.
+    """
+
+    try:
+        parsed = json.loads(key)
+    except json.JSONDecodeError:
+        return key
+    return parsed if isinstance(parsed, list) else key
+
+
 class UserSelection(BaseModel):
     """Information about selected files and locators for file masking ruleset generation."""
 

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/models/discovery.py

@@ -6,7 +6,13 @@ from typing import Any, Optional, Union
 from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
 
 from datamasque.client.models.connection import ConnectionConfig, ConnectionId, unwrap_connection_id
-from datamasque.client.models.data_selection import HashColumnsTableConfig, Locator, UserSelection
+from datamasque.client.models.data_selection import (
+    HashColumnsTableConfig,
+    Locator,
+    UserSelection,
+    deserialize_locator,
+    serialize_locator,
+)
 from datamasque.client.models.discovery_config import DiscoveryConfig, DiscoveryConfigId, unwrap_discovery_config_id
 from datamasque.client.models.pagination import Page
 from datamasque.client.models.rg_config import RGConfig, RGConfigId, unwrap_rg_config_id
@@ -434,6 +440,14 @@ class FileDiscoveryMatch(BaseModel):
     hit_ratio: Optional[int] = None  # None for metadata matches, percentage 0-100 for IDD matches.
 
 
+class ValueCountStatus(Enum):
+    """Whether a file's values were counted, and when they were not, why."""
+
+    counted = "counted"
+    in_data_discovery_disabled = "in_data_discovery_disabled"
+    file_type_has_no_values = "file_type_has_no_values"
+
+
 class FileDiscoveryLocatorResult(BaseModel):
     """A locator (column/path) within a discovered file."""
 
@@ -443,6 +457,7 @@ class FileDiscoveryLocatorResult(BaseModel):
     matches: list[FileDiscoveryMatch]
     data_types: list[str]
     safe_data_preview: Optional[SafeDataPreview] = None
+    value_count: Optional[int] = None
 
 
 class FileDiscoveryFile(BaseModel):
@@ -454,6 +469,23 @@ class FileDiscoveryFile(BaseModel):
     file_type: str
     delimiter: Optional[str] = None
     encoding: Optional[str] = None
+    value_counts: dict[str, int] = Field(default_factory=dict)
+    value_count_status: Optional[ValueCountStatus] = None
+
+    @field_validator("value_count_status", mode="before")
+    @classmethod
+    def _blank_status_is_none(cls, value: Any) -> Any:
+        return value or None
+
+    def get_value_count_of_locator(self, locator: Locator) -> Optional[int]:
+        """Returns how many values this file holds at `locator`, or `None` if this file holds no count for it."""
+
+        return self.value_counts.get(serialize_locator(locator))
+
+    def parse_value_counts(self) -> list[tuple[Locator, int]]:
+        """Returns this file's value counts, with each locator in the form that the discovery results use."""
+
+        return [(deserialize_locator(key), count) for key, count in self.value_counts.items()]
 
 
 class FileDiscoveryResult(BaseModel):

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/models/discovery_config.py

@@ -2,9 +2,9 @@ import enum
 from datetime import datetime
 from typing import Any, NewType, Optional
 
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import AliasChoices, AliasPath, BaseModel, ConfigDict, Field
 
-from datamasque.client.models.status import ValidationStatus
+from datamasque.client.models.status import ValidationErrorDetails, ValidationStatus
 
 DiscoveryConfigId = NewType("DiscoveryConfigId", str)
 
@@ -49,5 +49,14 @@ class DiscoveryConfig(BaseModel):
     """Validation status; may be `in_progress` briefly after creating a large config."""
     validation_error: Optional[str] = Field(default=None, exclude=True)
     """Human-readable validation error, or `None` when valid."""
+    # Deliberately not `validation_errors`:
+    # this is a different shape to `Ruleset.validation_errors`,
+    # and would sit one character from `validation_error` above.
+    validation_error_details: list[ValidationErrorDetails] = Field(
+        default_factory=list,
+        exclude=True,
+        validation_alias=AliasChoices(AliasPath("errors", "config_yaml"), "validation_error_details"),
+    )
+    """Structured, positional validation errors."""
     created: Optional[datetime] = Field(default=None, exclude=True)
     modified: Optional[datetime] = Field(default=None, exclude=True)

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/datamasque/client/models/discovery_config_library.py

@@ -27,5 +27,7 @@ class DiscoveryConfigLibrary(BaseModel):
     """Validation status; libraries are validated synchronously on create/update."""
     validation_error: Optional[str] = Field(default=None, exclude=True)
     """Human-readable validation error, or `None` when valid."""
+    usage_count: Optional[int] = Field(default=None, exclude=True)
+    """Number of active discovery configs that import this library."""
     created: Optional[datetime] = Field(default=None, exclude=True)
     modified: Optional[datetime] = Field(default=None, exclude=True)

{datamasque_python-1.2.1 → datamasque_python-1.2.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "datamasque-python"
-version = "1.2.1"
+version = "1.2.3"
 description = "Official Python client for the DataMasque data-masking API."
 authors = [
     { name = "DataMasque Ltd" },
@@ -47,7 +47,6 @@ Issues = "https://github.com/datamasque/datamasque-python/issues"
 
 [dependency-groups]
 dev = [
-    "bump2version>=1.0.1",
     "ruff>=0.9.0",
     "pytest>=7.4.4",
     "Sphinx>=7.2.6",