tangle-cli 0.1.14__tar.gz → 0.1.16__tar.gz

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: tangle-cli
-Version: 0.1.14
+Version: 0.1.16
 Summary: CLI for Tangle, the open-source ML pipeline orchestration platform
 Author: Alexey Volkov, Tangle authors, Volv Grebennikov
 Author-email: Alexey Volkov <alexey.volkov@ark-kun.com>, Volv Grebennikov <volv.grebennikov@shopify.com>
@@ -379,7 +379,36 @@ uv run tangle sdk published-components publish components/my-component.yaml --dr
 uv run tangle sdk published-components deprecate sha256:old --superseded-by sha256:new
 ```
 
-`publish` accepts `--image`, `--name`, `--description`, `--annotations` (JSON), `--dry-run`, `--published-by`, generic git metadata fields, generic API auth fields, `--log-type`, and `--config`. By default it scopes version checks and automatic old-version deprecation to the current authenticated user via `users_me()`; use `--published-by` to supply an explicit owner/publisher filter. Publishing fails closed if no owner can be determined.
+`publish` accepts `--image`, `--name`, `--description`, `--annotations` (JSON), `--dry-run`, `--allow-downgrade`, `--published-by`, generic git metadata fields, generic API auth fields, `--log-type`, and `--config`. By default it scopes version checks and automatic old-version deprecation to the current authenticated user via `users_me()`; use `--published-by` to supply an explicit owner/publisher filter. Publishing fails closed if no owner can be determined.
+
+#### Monotonic publishing and result digests
+
+Publishing is monotonic against the highest **non-deprecated, owner-scoped** published version of the component (ordering comes from `compare_versions`, which zero-pads shorter versions so `1.0.1 > 1.0`):
+
+| Local vs latest published | Outcome | Notes |
+| --- | --- | --- |
+| nothing published (no non-deprecated owner-scoped version) | `proceed` | first publish |
+| local strictly newer | `proceed` | publishes, then deprecates owner-scoped versions proven older |
+| local equal | `skip` | no create/deprecate calls |
+| local strictly older | `skip` | no-op; never publishes an older version and never deprecates a newer one |
+| published version unreadable, or ambiguous tie at the latest version | `error` | fails closed; no create/deprecate calls |
+
+Every result carries the digest of the version it compared against:
+
+- `digest` — digest of a **newly created** publication (SUCCESS only, unchanged meaning).
+- `latest_digest` — exact digest of the selected latest published version (set on PROCEED/SKIP, and carried through the SUCCESS/ERROR results that follow a version check). JSON output includes it as `latest_digest`.
+- `ProcessingResult.resolved_digest` — the digest a caller should pin: `digest or latest_digest`, but deliberately `None` for any outcome other than SUCCESS/SKIP, so a failed publish never hands back a stale-but-plausible digest.
+
+The check **fails closed** (an `error`, with nothing published and nothing deprecated) whenever the published state cannot be read completely:
+
+- any non-deprecated owner-scoped candidate whose digest is missing, or whose spec/version cannot be fetched or parsed — an unreadable row could be newer than the local version, and must never be deprecated sight-unseen;
+- two or more non-deprecated candidates tied at the selected latest version, where no exact digest can be chosen. The reason names the tied digests instead of guessing from API ordering.
+
+Deprecated components are never selected as "latest", and all digest lists in results/logs are sorted, so diagnostics do not depend on API response order.
+
+The published state is re-read immediately before create, and the same policy is re-applied to that fresh observation: a version that appeared concurrently since the first check can still turn the publish into a skip or an error. After a successful create, only digests **proven strictly older** in that final observation are deprecated — a row first seen after the publish decision is never deprecated on the strength of the earlier one. A race after the final read is not preventable client-side and needs a server-side conditional/CAS operation.
+
+**Contract change:** republishing an older version used to proceed (publishing the older spec and deprecating the newer one). It is now a skip. `--allow-downgrade` publishes the older spec but still never deprecates a strictly newer row; deprecate those explicitly with `published-components deprecate` if that is really intended. Deliberate downgrades must opt in with `--allow-downgrade` on the CLI, or `ComponentPublisher(allow_downgrade=True)` / `allow_downgrade=True` on the `publish_component_to_tangle` / `perform_version_check` wrappers. Republishing the same version is still a skip, as before.
 
 There is no separate OSS `publish-all` command. To publish multiple components, pass a YAML/JSON config list, or `_defaults` + `configs`, to the same `published-components publish` command; the command aggregates results and exits nonzero if any component errors. A top-level `_select` node can choose between such documents per environment (see [Environment-selected configs](#environment-selected-configs-_select)).
 

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/README.md

@@ -353,7 +353,36 @@ uv run tangle sdk published-components publish components/my-component.yaml --dr
 uv run tangle sdk published-components deprecate sha256:old --superseded-by sha256:new
 ```
 
-`publish` accepts `--image`, `--name`, `--description`, `--annotations` (JSON), `--dry-run`, `--published-by`, generic git metadata fields, generic API auth fields, `--log-type`, and `--config`. By default it scopes version checks and automatic old-version deprecation to the current authenticated user via `users_me()`; use `--published-by` to supply an explicit owner/publisher filter. Publishing fails closed if no owner can be determined.
+`publish` accepts `--image`, `--name`, `--description`, `--annotations` (JSON), `--dry-run`, `--allow-downgrade`, `--published-by`, generic git metadata fields, generic API auth fields, `--log-type`, and `--config`. By default it scopes version checks and automatic old-version deprecation to the current authenticated user via `users_me()`; use `--published-by` to supply an explicit owner/publisher filter. Publishing fails closed if no owner can be determined.
+
+#### Monotonic publishing and result digests
+
+Publishing is monotonic against the highest **non-deprecated, owner-scoped** published version of the component (ordering comes from `compare_versions`, which zero-pads shorter versions so `1.0.1 > 1.0`):
+
+| Local vs latest published | Outcome | Notes |
+| --- | --- | --- |
+| nothing published (no non-deprecated owner-scoped version) | `proceed` | first publish |
+| local strictly newer | `proceed` | publishes, then deprecates owner-scoped versions proven older |
+| local equal | `skip` | no create/deprecate calls |
+| local strictly older | `skip` | no-op; never publishes an older version and never deprecates a newer one |
+| published version unreadable, or ambiguous tie at the latest version | `error` | fails closed; no create/deprecate calls |
+
+Every result carries the digest of the version it compared against:
+
+- `digest` — digest of a **newly created** publication (SUCCESS only, unchanged meaning).
+- `latest_digest` — exact digest of the selected latest published version (set on PROCEED/SKIP, and carried through the SUCCESS/ERROR results that follow a version check). JSON output includes it as `latest_digest`.
+- `ProcessingResult.resolved_digest` — the digest a caller should pin: `digest or latest_digest`, but deliberately `None` for any outcome other than SUCCESS/SKIP, so a failed publish never hands back a stale-but-plausible digest.
+
+The check **fails closed** (an `error`, with nothing published and nothing deprecated) whenever the published state cannot be read completely:
+
+- any non-deprecated owner-scoped candidate whose digest is missing, or whose spec/version cannot be fetched or parsed — an unreadable row could be newer than the local version, and must never be deprecated sight-unseen;
+- two or more non-deprecated candidates tied at the selected latest version, where no exact digest can be chosen. The reason names the tied digests instead of guessing from API ordering.
+
+Deprecated components are never selected as "latest", and all digest lists in results/logs are sorted, so diagnostics do not depend on API response order.
+
+The published state is re-read immediately before create, and the same policy is re-applied to that fresh observation: a version that appeared concurrently since the first check can still turn the publish into a skip or an error. After a successful create, only digests **proven strictly older** in that final observation are deprecated — a row first seen after the publish decision is never deprecated on the strength of the earlier one. A race after the final read is not preventable client-side and needs a server-side conditional/CAS operation.
+
+**Contract change:** republishing an older version used to proceed (publishing the older spec and deprecating the newer one). It is now a skip. `--allow-downgrade` publishes the older spec but still never deprecates a strictly newer row; deprecate those explicitly with `published-components deprecate` if that is really intended. Deliberate downgrades must opt in with `--allow-downgrade` on the CLI, or `ComponentPublisher(allow_downgrade=True)` / `allow_downgrade=True` on the `publish_component_to_tangle` / `perform_version_check` wrappers. Republishing the same version is still a skip, as before.
 
 There is no separate OSS `publish-all` command. To publish multiple components, pass a YAML/JSON config list, or `_defaults` + `configs`, to the same `published-components publish` command; the command aggregates results and exits nonzero if any component errors. A top-level `_select` node can choose between such documents per environment (see [Environment-selected configs](#environment-selected-configs-_select)).
 

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/packages/tangle-cli/src/tangle_cli/__init__.py

@@ -14,6 +14,6 @@ from tangle_cli.dynamic_discovery_client import TangleDynamicDiscoveryClient
 try:
     __version__ = metadata_version("tangle-cli")
 except PackageNotFoundError:
-    __version__ = "0.1.14"
+    __version__ = "0.1.16"
 
 __all__ = ["TangleDynamicDiscoveryClient", "__version__"]

tangle_cli-0.1.16/packages/tangle-cli/src/tangle_cli/authenticated_identity.py

@@ -0,0 +1,77 @@
+"""Resolution of the authenticated account, and the symbolic ``me`` publisher.
+
+Compilation is offline, so a compiler cannot write the author's account id into
+a component entry. It writes the symbolic publisher :data:`ME` instead, and the
+account is resolved at hydration time by whoever is actually authenticated.
+
+Two callers need the account: the publisher, which scopes its version check and
+deprecation to the owner, and hydration of entries whose publisher is ``me``.
+They must agree, so the parsing lives here once.
+
+They differ only in what an *unknown* account means, so that choice is left to
+the caller: the publisher degrades, while resolution must fail closed -- see
+:func:`require_authenticated_user_id`.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from typing import Any
+
+#: Symbolic publisher meaning "whoever is authenticated at hydration time".
+#: Matched exactly and case-sensitively, so a literal account id that happens to
+#: differ in case is never mistaken for the sentinel.
+ME = "me"
+
+__all__ = [
+    "ME",
+    "IdentityUnavailableError",
+    "authenticated_user_id",
+    "is_symbolic_me",
+    "require_authenticated_user_id",
+]
+
+
+class IdentityUnavailableError(RuntimeError):
+    """The authenticated account could not be determined."""
+
+
+def is_symbolic_me(publisher: Any) -> bool:
+    """Whether ``publisher`` is the symbolic self-reference rather than an id."""
+    return publisher == ME
+
+
+def authenticated_user_id(client: Any) -> str | None:
+    """Return the current user id, or ``None`` if it cannot be read.
+
+    An empty or missing id is as unusable as no answer at all.
+    """
+    try:
+        user_info = client.users_me()
+    except Exception:
+        return None
+    if user_info is None:
+        return None
+    if isinstance(user_info, Mapping):
+        value = user_info.get("id")
+    else:
+        value = getattr(user_info, "id", None)
+    return str(value) if value else None
+
+
+def require_authenticated_user_id(client: Any) -> str:
+    """Return the current user id, raising when it cannot be determined.
+
+    For owner-scoped *resolution* an unknown account must never widen the
+    search: continuing unscoped is what would let a component published by
+    someone else under the same name be resolved as the author's own code.
+    """
+    user_id = authenticated_user_id(client)
+    if not user_id:
+        raise IdentityUnavailableError(
+            "Cannot determine the authenticated account, so a component "
+            f"published by '{ME}' cannot be resolved. Refusing to fall back to "
+            "an unscoped lookup, which could resolve a component owned by "
+            "someone else. Re-authenticate and retry."
+        )
+    return user_id

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/packages/tangle-cli/src/tangle_cli/client.py

@@ -952,12 +952,22 @@ class TangleApiClient(GeneratedTangleApiOperations):
                     search_digests.add(str(data["digest"]))
 
         publisher_filter = published_by_substring or published_by
+        # The API parameter is a SUBSTRING match. When the caller asked for an
+        # exact owner it is therefore only a prefilter, and accepting its
+        # results verbatim would let a superset owner id (``alice`` matching
+        # ``alice2``) satisfy an exact request. An owner-scoped lookup is an
+        # identity control, so exactness is enforced here on the returned rows
+        # and a row with no owner is never accepted. Explicit
+        # ``published_by_substring`` callers keep substring semantics.
+        exact_owner = published_by if published_by and not published_by_substring else None
         found: dict[str, ComponentInfo] = {}
 
         def add(info: ComponentInfo) -> None:
             key = info.digest or info.name
             if not key:
                 return
+            if exact_owner is not None and info.published_by != exact_owner:
+                return
             found[key] = info
             if verbose:
                 self.logger.info(f"   Found existing component: {info.name} ({key[:16]}...)")

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/packages/tangle-cli/src/tangle_cli/component_from_func.py

@@ -986,8 +986,13 @@ def _is_name_main_test(node: ast.expr) -> bool:
 # _strip_authoring_constructs). ``registered`` marks an op published separately
 # via its own gen_config.yaml; when that same op is baked (through its
 # local_from_python entry) the decorator + its authoring import must be stripped
-# too, exactly like @task.
-_AUTHORING_DECORATOR_NAMES = frozenset({"task", "pipeline", "subpipeline", "registered"})
+# too, exactly like @task. ``Publish`` likewise only records a publication
+# declaration for the compiler; leaving it in the baked program would raise
+# ``NameError`` at container startup, since its import is stripped with the
+# rest of the authoring surface.
+_AUTHORING_DECORATOR_NAMES = frozenset(
+    {"task", "pipeline", "subpipeline", "registered", "Publish"}
+)
 
 # The python-pipeline authoring modules. ONLY imports of these modules (and
 # their submodules) are authoring-only and stripped from the baked source. We
@@ -2015,6 +2020,15 @@ def build_component_dict(
 # ============================================================================
 
 
+def _within(path: Path, root: Path) -> bool:
+    """Whether *path* lies inside *root*."""
+    try:
+        path.resolve().relative_to(root.resolve())
+    except ValueError:
+        return False
+    return True
+
+
 def generate_component_yaml(
     file_path: Path,
     output_path: Path,
@@ -2030,12 +2044,21 @@ def generate_component_yaml(
     emit_generation_annotations: bool = True,
     path_annotation_mode: Literal["oss", "td_legacy"] = "oss",
     unwrapped_inputs: dict[str, Any] | None = None,
+    logical_output_path: Path | None = None,
 ) -> bool:
     """Generate a component YAML file from a Python function.
 
     Args:
         file_path: Path to the Python source file
         output_path: Where to write the generated YAML
+        logical_output_path: Where the component is to be UNDERSTOOD to live,
+            for provenance only. Defaults to ``output_path``. Pass this when
+            the file is written somewhere incidental -- a private staging
+            directory, a scratch area -- so the recorded provenance describes
+            the component rather than the accident of where bytes landed.
+            Provenance is otherwise derived from the physical path, which
+            would embed that location and make the emitted YAML, and therefore
+            its digest, differ between runs of identical source.
         container_image: Docker image reference
         function_name: Function to extract (auto-detected if None)
         dependencies_from: Path to pyproject.toml with pip dependencies
@@ -2111,7 +2134,6 @@ def generate_component_yaml(
             deps = read_dependencies(dependencies_from)
 
         # 4. Build annotations
-        directory = file_path.parent.resolve()
         module_code = file_path.read_text()
 
         annotations: dict[str, str] = {
@@ -2143,9 +2165,18 @@ def generate_component_yaml(
         # basename-only paths outside a git checkout to preserve historical
         # snapshots.
         resolved_source = file_path.resolve()
-        resolved_output = output_path.resolve()
+        # Provenance describes where the component LIVES, which is not always
+        # where this call happens to write it. Kept UNRESOLVED: ``td_legacy``
+        # annotates the lexical basename, so resolving here would rewrite the
+        # recorded name whenever the output is a symlink.
+        annotation_output = logical_output_path if logical_output_path is not None else output_path
+        resolved_output = annotation_output.resolve()
         common_dir = Path(os.path.commonpath([resolved_source, resolved_output]))
-        git_root = get_git_root(directory)
+        # Discover from the SOURCE's real directory: a symlinked source file
+        # whose link lives outside the checkout would otherwise find no repo
+        # and drop every git annotation.
+        source_dir = resolved_source.parent
+        git_root = get_git_root(source_dir)
         use_common_paths = path_annotation_mode == "oss" or git_root is not None
 
         def _path_annotation(path: Path) -> str:
@@ -2158,7 +2189,7 @@ def generate_component_yaml(
 
         if not strip_source_path:
             annotations["python_original_code_path"] = _path_annotation(file_path)
-        annotations["component_yaml_path"] = _path_annotation(output_path)
+        annotations["component_yaml_path"] = _path_annotation(annotation_output)
         if emit_generation_annotations:
             if dependencies_from:
                 annotations["tangle_cli_generation_dependencies_from"] = _path_annotation(dependencies_from)
@@ -2167,7 +2198,12 @@ def generate_component_yaml(
 
         # Git info — use the same common ancestor as git_relative_dir when common paths are active.
         if git_root:
-            git_info = get_git_info(common_dir)
+            # Read the repository from a directory KNOWN to be inside it. The
+            # common ancestor of source and output need not be: any output
+            # outside the checkout (a configured output_folder, a staging dir)
+            # drags it out, and reading git there returns nothing, silently
+            # publishing a component with no repository of origin.
+            git_info = get_git_info(common_dir if _within(common_dir, git_root) else source_dir)
             git_info.pop("_git_root", None)
             # Override git_relative_dir to be the common ancestor
             try:
@@ -2176,7 +2212,7 @@ def generate_component_yaml(
                 pass
             annotations.update(git_info)
         else:
-            git_info = get_git_info(directory)
+            git_info = get_git_info(source_dir)
             git_info.pop("_git_root", None)
             annotations.update(git_info)
 

{tangle_cli-0.1.14 → tangle_cli-0.1.16}/packages/tangle-cli/src/tangle_cli/component_generator.py

@@ -128,6 +128,7 @@ class ComponentGenerator:
         resolve_root: Path | None = None,
         emit_generation_annotations: bool = True,
         unwrapped_inputs: dict[str, Any] | None = None,
+        logical_output_path: Path | None = None,
     ) -> bool:
         """Generate component YAML from a Python function source file.
 
@@ -147,6 +148,8 @@ class ComponentGenerator:
             unwrapped_inputs: Optional persisted unwrap schema. Hydrate forwards
                 this ``local_from_python.unwrapped_inputs`` payload so component
                 generation expands dict parameters exactly as compile did.
+            logical_output_path: Where the component is to be understood to
+                live, for provenance only. Defaults to ``output_path``.
 
         Returns:
             True when generation succeeds, otherwise False.
@@ -154,6 +157,12 @@ class ComponentGenerator:
 
         from tangle_cli.component_from_func import generate_component_yaml
 
+        # Forwarded only when set. These hops are overridable, and an older
+        # override written against the previous signature must keep working
+        # for every caller that does not use the seam.
+        seam: dict[str, Any] = (
+            {} if logical_output_path is None else {"logical_output_path": logical_output_path}
+        )
         return generate_component_yaml(
             file_path=file_path,
             output_path=output_path,
@@ -168,6 +177,7 @@ class ComponentGenerator:
             resolve_root=resolve_root,
             emit_generation_annotations=emit_generation_annotations,
             unwrapped_inputs=unwrapped_inputs,
+            **seam,
         )
 
     def regenerate_yaml(
@@ -184,6 +194,7 @@ class ComponentGenerator:
         resolve_root: Path | None = None,
         emit_generation_annotations: bool = True,
         unwrapped_inputs: dict[str, Any] | None = None,
+        logical_output_path: Path | None = None,
     ) -> bool:
         """Regenerate a YAML component from a Python function source file.
 
@@ -202,6 +213,10 @@ class ComponentGenerator:
             emit_generation_annotations: Whether to emit regeneration metadata.
             unwrapped_inputs: Optional ``local_from_python.unwrapped_inputs``
                 schema to preserve compile-time unwrap expansion at hydrate time.
+            logical_output_path: Where the component is to be understood to
+                live, for provenance only. Defaults to ``output_path``. Note
+                that the image is still read back from the PHYSICAL output, so
+                a caller writing to a fresh location supplies ``image``.
 
         Returns:
             True when regeneration succeeds, otherwise False.
@@ -218,6 +233,9 @@ class ComponentGenerator:
             self._log(f"  Found dependencies: {deps_file}")
 
         final_output.parent.mkdir(parents=True, exist_ok=True)
+        seam: dict[str, Any] = (
+            {} if logical_output_path is None else {"logical_output_path": logical_output_path}
+        )
         return self.run_generation(
             python_file=python_file,
             final_output=final_output,
@@ -231,6 +249,7 @@ class ComponentGenerator:
             resolve_root=resolve_root,
             emit_generation_annotations=emit_generation_annotations,
             unwrapped_inputs=unwrapped_inputs,
+            **seam,
         )
 
     def run_generation(
@@ -248,6 +267,7 @@ class ComponentGenerator:
         resolve_root: Path | None = None,
         emit_generation_annotations: bool = True,
         unwrapped_inputs: dict[str, Any] | None = None,
+        logical_output_path: Path | None = None,
     ) -> bool:
         """Execute component generation and clean up partial output on failure.
 
@@ -265,6 +285,8 @@ class ComponentGenerator:
             emit_generation_annotations: Whether to emit regeneration metadata.
             unwrapped_inputs: Optional persisted unwrap schema forwarded to the
                 low-level generator for compile/hydrate interface parity.
+            logical_output_path: Where the component is to be understood to
+                live, for provenance only. Defaults to ``final_output``.
 
         Returns:
             True when generation succeeds, otherwise False. On failure, any
@@ -274,6 +296,9 @@ class ComponentGenerator:
         try:
             function_detail = f" function {func_name!r}" if func_name else ""
             self._log(f"  Generating component from {python_file.name}{function_detail}...")
+            seam: dict[str, Any] = (
+                {} if logical_output_path is None else {"logical_output_path": logical_output_path}
+            )
             success = self.generate_component_yaml(
                 file_path=python_file,
                 output_path=final_output,
@@ -287,6 +312,7 @@ class ComponentGenerator:
                 resolve_root=resolve_root,
                 emit_generation_annotations=emit_generation_annotations,
                 unwrapped_inputs=unwrapped_inputs,
+                **seam,
             )
             if not success:
                 self._log("  ❌ Failed to generate component", err=True)
@@ -340,6 +366,7 @@ def regenerate_yaml(
     resolve_root: Path | None = None,
     logger: Any | None = None,
     unwrapped_inputs: dict[str, Any] | None = None,
+    logical_output_path: Path | None = None,
 ) -> bool:
     """Regenerate component YAML through the default generator.
 
@@ -358,11 +385,16 @@ def regenerate_yaml(
         logger: Optional logger object used by the generator.
         unwrapped_inputs: Optional persisted unwrap schema from
             ``local_from_python.unwrapped_inputs``.
+        logical_output_path: Where the component is to be understood to live,
+            for provenance only. Defaults to ``output_path``.
 
     Returns:
         True when regeneration succeeds, otherwise False.
     """
 
+    seam: dict[str, Any] = (
+        {} if logical_output_path is None else {"logical_output_path": logical_output_path}
+    )
     return ComponentGenerator(logger=logger, verbose=verbose).regenerate_yaml(
         python_file=python_file,
         output_path=output_path,
@@ -375,6 +407,7 @@ def regenerate_yaml(
         mode=mode,
         resolve_root=resolve_root,
         unwrapped_inputs=unwrapped_inputs,
+        **seam,
     )