mammoth-cli 2.2.31__tar.gz → 2.2.32__tar.gz

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mammoth-cli
-Version: 2.2.31
+Version: 2.2.32
 Summary: Command-line interface for the Mammoth Analytics platform
 Project-URL: homepage, https://mammoth.io
 Project-URL: repository, https://github.com/EdgeMetric/mammothsdk

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/__init__.py

@@ -6,7 +6,7 @@ not implement a second HTTP client and does not call private SDK members.
 
 from __future__ import annotations
 
-__version__ = "2.2.31"
+__version__ = "2.2.32"
 
 # The versioned machine output envelope contract.
 SCHEMA_VERSION = 1

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/bundled_skill/mammoth-cli/references/about-mammoth.md

@@ -146,7 +146,12 @@ No command:
   every source column to an existing target column. A partial mapping, or a
   destination the target lacks, is refused with `append_mapping_incomplete`
   before anything is written (the backend would append only the mapped columns
-  and add a new column for the unknown name). An append is a standing link: it is stored
+  and add a new column for the unknown name). A target column the append
+  leaves unfilled would be NULL in every appended row: it is refused with
+  `append_leaves_columns_blank` unless the input names exactly those columns,
+  e.g. `"blank_columns": ["id","score"]` (a deliberate partial append, such as
+  mapping only `name`); never delete source columns or clone the source view to
+  get around it. An append is a standing link: it is stored
   as a pipeline step of the source view, so every later re-run of that view
   appends again into the target dataset, not just the first run. A column
   with the same name but a different type in the source view and the target

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/bundled_skill/mammoth-cli/references/recipes/exports.md

@@ -55,7 +55,11 @@ there is no separate Auto-Sync endpoint or field to find.
 `target_ds_id` writes into an existing dataset instead of creating one
 (`save_as_mode` `REPLACE_IN_DS` or `APPEND_TO_DS`). Read the delivery back in
 the target project (`view list N --project P`, `view data get`) before
-reporting it. Do not build this with `view export create`: the raw
+reporting it. An append that leaves target columns unfilled (a partial
+`column_mapping`, or a source with fewer columns) is refused with
+`append_leaves_columns_blank` unless the input lists exactly those columns,
+`"blank_columns": ["id","score"]`; confirm that with the user, and never
+clone the source view or delete its columns to get around the refusal. Do not build this with `view export create`: the raw
 `internal_dataset` spec needs `USER_ID` and the `export_project` /
 `project_id` / `source_project_id` trio, and without them the backend answers
 `4GENR007 Validation error` with no detail; the typed command fills them.

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/commands/batch.py

@@ -25,6 +25,10 @@ from mammoth_cli.manifest.loader import command_by_id
 from mammoth_cli.runtime.confirm import POLICY_PROMPT_OR_YES, enforce_confirmation
 from mammoth_cli.runtime.invocation import Invocation
 from mammoth_cli.runtime.session import open_service, require_project
+from mammoth_cli.services.append_blank_columns import (
+    BLANK_COLUMNS_FIELD,
+    acknowledged_blank_columns,
+)
 
 HandlerResult = tuple[Any, dict[str, Any]]
 
@@ -221,15 +225,32 @@ def _reject_type_change(item: dict[str, Any], dest_col: dict[str, Any], dataset_
     )
 
 
+def _filled_destination_names(
+    items: list[dict[str, Any]], dest: list[dict[str, Any]], source: list[dict[str, Any]]
+) -> set[str]:
+    """Destination column names some mapping item fills (``add_column`` items included)."""
+    dest_names = {col["c_name"] for col in dest}
+    filled: set[str] = set()
+    for item in items:
+        _, dest_col = _resolve_item(item, source, dest)
+        if dest_col is not None:
+            filled.add(dest_col["c_name"])
+        elif item.get("action") == "add_column" and item.get("destination_c_name") in dest_names:
+            filled.add(item["destination_c_name"])
+    return filled
+
+
 def _checked_batch_mapping(
-    service: Any, dataset_id: int, source_id: int, mapping: Any
+    service: Any, dataset_id: int, source_id: int, mapping: Any, blank_columns: Any = None
 ) -> tuple[list[dict[str, Any]], list[str]]:
     """Validate a batch mapping against both schemas; return typed items and warnings.
 
     Refuses unmapped source columns and unknown destinations (the backend appends
     only the mapped columns) and stamps every item with the destination's actual
     current type, so no destination column is silently re-typed. A mapped pair
-    whose source and destination types differ is returned as a warning.
+    whose source and destination types differ is returned as a warning. A
+    destination column no item fills is refused unless ``blank_columns`` names
+    exactly those columns; the confirmed blanks are returned as a warning.
     """
     source = _dataset_columns(service, source_id, "source")
     dest = _dataset_columns(service, dataset_id, "target")
@@ -258,7 +279,8 @@ def _checked_batch_mapping(
     _reject_incomplete_batch_mapping(
         (dataset_id, source_id), {col["c_name"] for col in source}, mapped_sources, unknown
     )
-    return typed, warnings
+    blank = sorted({col["c_name"] for col in dest} - _filled_destination_names(typed, dest, source))
+    return typed, acknowledged_blank_columns(dataset_id, blank, blank_columns) + warnings
 
 
 def batch_create(invocation: Invocation) -> HandlerResult:
@@ -297,7 +319,7 @@ def batch_create(invocation: Invocation) -> HandlerResult:
         )
     with open_service(invocation) as (service, auth):
         kwargs["mapping"], warnings = _checked_batch_mapping(
-            service, dataset_id, source_id, mapping
+            service, dataset_id, source_id, mapping, document.get(BLANK_COLUMNS_FIELD)
         )
         data = service.call(_symbol(invocation), **kwargs)
     if warnings and isinstance(data, dict):

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/commands/view.py

@@ -55,6 +55,10 @@ from mammoth_cli.runtime.invocation import Invocation
 from mammoth_cli.runtime.session import open_service, require_project
 from mammoth_cli.services import conditional_format as cf_rules
 from mammoth_cli.services import read_queries, text_dates
+from mammoth_cli.services.append_blank_columns import (
+    BLANK_COLUMNS_FIELD,
+    acknowledged_blank_columns,
+)
 from mammoth_cli.services.conditions import CONDITION_KWARG, compile_condition
 from mammoth_cli.services.dashboard_review import UPLOAD_NOTE, upload_hints
 from mammoth_cli.services.data_quality import column_warnings, duplicate_rows_fact
@@ -3614,6 +3618,8 @@ def view_export_specialized(invocation: Invocation) -> HandlerResult:
     allowed = explicit_fields | common_fields | {_DATASET_ID_FIELD}
     if invocation.command_id in _DB_REPLACE_ROUTES:
         allowed = allowed | {_REPLACE_TABLE_FIELD}
+    if invocation.command_id == "view.export.dataset":
+        allowed = allowed | {BLANK_COLUMNS_FIELD}
     unknown = sorted(set(document) - allowed)
     if unknown:
         raise CliError(
@@ -3647,6 +3653,7 @@ def view_export_specialized(invocation: Invocation) -> HandlerResult:
     kwargs = dict(document)
     kwargs.pop(_DATASET_ID_FIELD, None)
     kwargs.pop(_REPLACE_TABLE_FIELD, None)
+    blank_columns = kwargs.pop(BLANK_COLUMNS_FIELD, None)
     is_dataset_route = invocation.command_id == "view.export.dataset"
     target_ds_id = kwargs.get("target_ds_id") if is_dataset_route else None
     save_as_mode = kwargs.get("save_as_mode") if is_dataset_route else None
@@ -3663,6 +3670,7 @@ def view_export_specialized(invocation: Invocation) -> HandlerResult:
                 int(target_ds_id) if target_ds_id is not None else None,
             )
         target_view_before = None
+        blank_warnings: list[str] = []
         target_only_columns: list[str] = []
         mapped_type_warnings: list[str] = []
         hidden_left_out = (
@@ -3684,6 +3692,9 @@ def view_export_specialized(invocation: Invocation) -> HandlerResult:
                 int(target_ds_id),
                 kwargs.get("column_mapping"),
             )
+            blank_warnings = acknowledged_blank_columns(
+                int(target_ds_id), target_only_columns, blank_columns
+            )
         try:
             data = service.call_view(dataview_id, method, dataset_id=dataset_id, **kwargs)
         except CliError as error:
@@ -3745,14 +3756,7 @@ def view_export_specialized(invocation: Invocation) -> HandlerResult:
                     "hidden column(s) of the source view are not in the new dataset: "
                     + ", ".join(hidden_left_out)
                 )
-            if target_only_columns:
-                # Allowed -- an append never has to cover every target column
-                # -- but worth surfacing rather than leaving silent.
-                warnings.insert(
-                    0,
-                    "target dataset has column(s) the source view does not (kept "
-                    "as-is): " + ", ".join(target_only_columns),
-                )
+            warnings[:0] = blank_warnings
             if warnings:
                 row_check["warnings"] = warnings
             data["row_check"] = row_check

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/errors/envelope.py

@@ -77,6 +77,9 @@ ERROR_SUMMARIES: dict[str, str] = {
         "More than one item matched, so the assistant needs to be more specific."
     ),
     "api_error": "Mammoth couldn't complete this step.",
+    "append_leaves_columns_blank": (
+        "This append would leave columns of the destination empty, and needs them named."
+    ),
     "append_mapping_incomplete": "The column mapping would leave columns out or add new ones.",
     "append_schema_mismatch": "The new data doesn't line up with the existing columns.",
     "append_type_mismatch": "A column has a different type than in the existing data.",

mammoth_cli-2.2.32/mammoth_cli/services/append_blank_columns.py

@@ -0,0 +1,57 @@
+"""The named opt-in for an append that leaves destination columns blank.
+
+An append only writes the columns its mapping fills; every other destination
+column is NULL in each appended row. The caller must name exactly those
+columns in ``blank_columns`` -- the same explicit acknowledgement pattern as
+``replace_table`` on a database export -- so a partial append is deliberate.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+from mammoth_cli.errors.envelope import EXIT_USAGE, CliError
+
+BLANK_COLUMNS_FIELD = "blank_columns"
+
+
+def acknowledged_blank_columns(
+    destination_id: int, blank: list[str], acknowledged: Any
+) -> list[str]:
+    """Return the warnings for ``blank`` (empty when none), refusing an unnamed leave-blank.
+
+    ``acknowledged`` is the input's ``blank_columns`` value; it must list
+    exactly the ``blank`` names as a set. A blanket ``true`` or a mismatched
+    list is refused with the expected list shown.
+    """
+    if not blank:
+        return []
+    expected = sorted(blank)
+    given = (
+        {str(name) for name in acknowledged}
+        if isinstance(acknowledged, list) and not isinstance(acknowledged, bool)
+        else None
+    )
+    if given != set(expected):
+        listed = ", ".join(expected)
+        raise CliError(
+            code="append_leaves_columns_blank",
+            message=(
+                f"Appending into dataset {destination_id} leaves column(s) {listed} "
+                "blank (NULL) in every appended row."
+            ),
+            exit_status=EXIT_USAGE,
+            hint=(
+                "If that is intended, add "
+                f'"{BLANK_COLUMNS_FIELD}": {_json_list(expected)} to the input to confirm; '
+                "otherwise map a source column to each of them."
+            ),
+            details={"blank_columns": expected, "acknowledged": acknowledged},
+        )
+    return [f"appended rows leave column(s) {', '.join(expected)} blank (NULL), as confirmed"]
+
+
+def _json_list(names: list[str]) -> str:
+    """The names as a JSON array literal, for the hint."""
+    return json.dumps(names)

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/mammoth_cli/services/command_contract.py

@@ -743,6 +743,14 @@ _S7_ADDITIONAL_INPUT_FIELDS: dict[str, tuple[FieldSpec, ...]] = {
         )
         for route in ("postgres", "mysql", "mssql", "redshift", "bigquery", "publish-db")
     },
+    # An append that leaves destination columns unfilled needs them named in
+    # ``blank_columns`` (CLI-side acknowledgement, never forwarded to the SDK).
+    **{
+        route: (
+            FieldSpec("blank_columns", required=False, annotation=list[str] | None, default=None),
+        )
+        for route in ("view.export.dataset", "batch.create")
+    },
     # user.update previously pinned name/email here as a guess, from when
     # UserProfileAPI.update took **fields and so had nothing introspectable.
     # The SDK method is now typed (first_name, last_name -- the backend's

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mammoth-cli"
-version = "2.2.31"
+version = "2.2.32"
 description = "Command-line interface for the Mammoth Analytics platform"
 authors = [{name = "Mammoth Analytics", email = "support@mammoth.io"}]
 readme = "README.md"

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/spec/manifests/commands/batch.yaml

@@ -91,7 +91,7 @@ commands:
   - CT-BATCH-CREATE-ERROR
   draft_test: null
   undo_test: null
-  known_restrictions: Every source column must be mapped to an existing destination column; the CLI reads both schemas and refuses (append_mapping_incomplete) an unmapped source column or a destination the dataset lacks, because the backend appends only mapped columns. The CLI sends each item's expected_destination_c_type as the destination's current type (never re-types it; an explicit different value is refused with append_type_mismatch) and warns in mapping_check.warnings when source and destination types differ. SOURCE_ID is a standalone dataset in the same project (4BATC010 INVALID_SOURCE_DATASET for a file id or an append child); to append a file use file upload with append_to_ds_id. delete_source_ds=true changes the mutation class to destructive and requires confirmation.
+  known_restrictions: Every source column must be mapped to an existing destination column; the CLI reads both schemas and refuses (append_mapping_incomplete) an unmapped source column or a destination the dataset lacks, because the backend appends only mapped columns. A destination column no mapping item fills (add_column items count) would be NULL in every appended row, so it is refused (append_leaves_columns_blank) unless the input lists exactly those names in blank_columns; the confirmed blanks are named in mapping_check.warnings. The CLI sends each item's expected_destination_c_type as the destination's current type (never re-types it; an explicit different value is refused with append_type_mismatch) and warns in mapping_check.warnings when source and destination types differ. SOURCE_ID is a standalone dataset in the same project (4BATC010 INVALID_SOURCE_DATASET for a file id or an append child); to append a file use file upload with append_to_ds_id. delete_source_ds=true changes the mutation class to destructive and requires confirmation.
   reviewed_by: primary
   readback:
     command: batch.list

{mammoth_cli-2.2.31 → mammoth_cli-2.2.32}/spec/manifests/commands/view.yaml

@@ -2261,7 +2261,7 @@ commands:
   - CT-VIEW-EXPORT-DATASET-ERROR
   draft_test: null
   undo_test: null
-  known_restrictions: 'Mutation class dynamic: benign_mutation for a new target, destructive when replacing an existing target. The export is appended to the source pipeline (end_of_pipeline) and re-runs with it; target_project_id sends the dataset into another project (cross-project branch-out, proven on release). Result: {dataset_id, project_id, source_view_id, next}. Not a plain copy: this leaves a live export on the source and the new dataset refreshes from the source view, so keep that working view and name it as the new dataset''s source. For an independent copy of a dataset use dataset create with ds_creation_type clone. Writing into an existing dataset (target_ds_id, save_as_mode APPEND_TO_DS or REPLACE_IN_DS; both run the same backend append code, so both get these guards): omit column_mapping and every column is mapped by name; a non-empty column_mapping is used as-is, so it must map every source column to an existing target column, and a partial mapping or an unknown destination is refused (append_mapping_incomplete) before anything is written. A mapped pair of different types is reported in row_check.warnings. A new dataset (no target_ds_id) leaves out the view''s hidden columns; they are named in row_check.warnings.'
+  known_restrictions: 'Mutation class dynamic: benign_mutation for a new target, destructive when replacing an existing target. The export is appended to the source pipeline (end_of_pipeline) and re-runs with it; target_project_id sends the dataset into another project (cross-project branch-out, proven on release). Result: {dataset_id, project_id, source_view_id, next}. Not a plain copy: this leaves a live export on the source and the new dataset refreshes from the source view, so keep that working view and name it as the new dataset''s source. For an independent copy of a dataset use dataset create with ds_creation_type clone. Writing into an existing dataset (target_ds_id, save_as_mode APPEND_TO_DS or REPLACE_IN_DS; both run the same backend append code, so both get these guards): omit column_mapping and every column is mapped by name; a non-empty column_mapping is used as-is, so it must map every source column to an existing target column, and a partial mapping or an unknown destination is refused (append_mapping_incomplete) before anything is written. A target column that nothing fills (a column the source view lacks and column_mapping does not target) would be NULL in every appended row: refused (append_leaves_columns_blank) unless the input lists exactly those column names in blank_columns (a blanket true or a different list is refused); the confirmed blanks are named in row_check.warnings. Do not reshape the source view to dodge this. A mapped pair of different types is reported in row_check.warnings. A new dataset (no target_ds_id) leaves out the view''s hidden columns; they are named in row_check.warnings.'
   reviewed_by: primary
 - command_id: view.export.delete
   command_path: view export delete