mainsequence 6.0.30__tar.gz → 6.0.32__tar.gz

{mainsequence-6.0.30/mainsequence.egg-info → mainsequence-6.0.32}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence
-Version: 6.0.30
+Version: 6.0.32
 Summary: Main Sequence SDK 
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: LicenseRef-MainSequence-Proprietary

{mainsequence-6.0.30 → mainsequence-6.0.32}/agent_scaffold/skills/platform_operations/orchestration_and_releases/SKILL.md

@@ -27,7 +27,8 @@ This skill is for:
 - author backend-managed declarations under `.mainsequence/workflows/`
 - validate workflow files through the ProjectBranch workflow endpoints
 - create or select project images
-- freeze jobs to a project image
+- bind every job to one exact project image
+- configure standalone Job automatic deployment as future exact-image promotion
 - inspect job runs and logs
 - reason about project resources and resource releases
 - decide whether a `ResourceRelease` should opt into `automatic_deployment`
@@ -91,7 +92,9 @@ Before changing orchestration or release behavior, collect or infer:
   - Streamlit dashboard release creation
 - whether a `ResourceRelease` should be manually pinned or opted into repository-sync automatic deployment
 - whether Artifact inputs or outputs are part of the run
-- whether the job should be reproducible against a pinned image
+- who owns the exact initial image: the caller for manual pinning or the
+  backend for automatic deployment
+- whether future qualifying immutable repository events may promote a standalone Job
 - for Streamlit dashboard deployment, the `README.md` resource next to `app.py`
 
 If the execution target or image strategy is unclear, stop before scheduling anything.
@@ -102,7 +105,8 @@ For every non-trivial orchestration task, decide:
 
 1. Is this a one-off/manual workflow or a repository-managed recurring workflow?
 2. Should the jobs live in a `.mainsequence/workflows/*.yaml` declaration?
-3. Which pinned project image should the job use?
+3. Is the Job manually pinned to a caller-selected exact ready image, or does
+   automatic deployment make the backend derive it from the synchronized commit?
 4. Does the workflow depend on Artifacts?
 5. Is the task actually a release/resource problem instead of only a job problem?
 6. For Streamlit dashboard deployment, do the selected app resource, README resource, and project image all refer to the intended pushed commit?
@@ -131,15 +135,37 @@ an existing backend resource.
 
 Do not hide important recurring schedules in ad hoc shell history or one-off manual commands.
 
-### 2. Jobs should run against pinned images
+### 2. Every Job requires one exact image
 
-Jobs should be pinned to a project image.
+Every persisted direct Job has one exact project image, but the creation mode
+determines who supplies it.
 
 Remember:
 
 - images are built from pushed commits
 - if a commit is not on the remote, it cannot be used for an image
-- unpinned jobs are not an acceptable default in a managed Main Sequence project
+- there is no dynamic, blank-image, branch-tip, or `latest` Job mode
+- a manual Job requires the caller to select `related_image_uid`
+- an automatic Job forbids caller image selection; the backend derives the
+  exact initial image from the ProjectBranch's persisted synchronized commit
+
+When standalone Job automatic deployment is enabled during creation, the
+backend prepares the exact initial image and later qualifying immutable
+repository events may atomically promote another exact image. On an existing
+Job, enabling or disabling it retains the current exact image. The backend owns
+repository-event truth, policy evaluation, image preparation, and promotion;
+project code and the SDK must not inspect Git or choose a runtime branch.
+
+For direct creation use:
+
+- `--automatic-deployment` to permit future exact-image promotion
+- `--no-automatic-deployment` to keep future repository events from changing
+  the Job image
+- `--automatic-redeployment-tag-regex` to require a full matching immutable
+  tag; omit the regex for every otherwise-eligible exact event
+
+Use `--related-image-uid` only for manual pinning. Omit it with
+`--automatic-deployment`; passing both is invalid.
 
 ### 3. Jobs must be verifiable after creation
 
@@ -263,6 +289,9 @@ When reviewing an orchestration task, look for:
 - direct job creation where a batch file should exist
 - missing or wrong `related_image_uid`
 - jobs tied to moving repository state instead of a pinned image
+- manual Job creation that omits an exact ready image
+- automatic Job creation that supplies a caller-selected image
+- client code that interprets automatic deployment as a branch-tip or `latest` selector
 - no run/log verification after creation
 - unsafe use of `--strict`
 - workflows depending on laptop-specific file paths instead of Artifacts
@@ -282,7 +311,8 @@ Do not claim success until you have checked:
   - interval
   - crontab
   - one-off
-- the pinned image choice is intentional
+- manual image selection or backend-owned automatic image preparation is intentional
+- standalone Job automatic deployment and tag policy are intentionally enabled or disabled
 - the job exists after direct creation or repository workflow application
 - runs and logs were inspected when execution success matters
 - resources and releases were verified when deployment success matters
@@ -308,6 +338,8 @@ If the workflow uses Artifacts, also check:
 - the execution target is unclear
 - the image strategy is unclear but reproducibility matters
 - the backend rejects the workflow version, resource kind, or requested field
+- a manually pinned Job has no exact ready initial image
+- an automatically deployed Job has no persisted synchronized ProjectBranch commit
 - the workflow depends on local file paths that should be platform Artifacts
 - automatic deployment is requested but the deployment source branch/current synced project version is unclear
 - automatic deployment is requested but the resource path or required README is not stable

{mainsequence-6.0.30 → mainsequence-6.0.32}/mainsequence/cli/api.py

@@ -60,6 +60,7 @@ MCP_MAX_RESOURCE_PAGES = 100
 S = requests.Session()
 S.headers.update({"Content-Type": "application/json"})
 
+
 class ApiError(RuntimeError):
     """Base error for API failures."""
 
@@ -942,9 +943,7 @@ def render_project_branch_default_redeployment_tag(
         {"version": normalized_version},
     )
     if not r.ok:
-        raise ApiError(
-            f"Default redeployment tag rendering failed ({r.status_code})."
-        )
+        raise ApiError(f"Default redeployment tag rendering failed ({r.status_code}).")
     payload = r.json()
     if not isinstance(payload, dict):
         raise ApiError("Default redeployment tag rendering returned an unexpected payload.")
@@ -1123,9 +1122,7 @@ def update_organization_team(
             class_name="Team",
             operation=lambda ClientTeam: ClientTeam.get_by_uid(
                 str(team_uid), timeout=timeout
-            ).patch(
-                **updates
-            ),
+            ).patch(**updates),
         )
         return _sdk_object_to_dict(team)
     except Exception as e:
@@ -2866,7 +2863,6 @@ def list_meta_tables(
         raise ApiError(f"MetaTables fetch failed: {e}") from e
 
 
-
 def _serialize_sdk_search_response(payload: Any) -> dict[str, Any] | list[dict[str, Any]]:
     if isinstance(payload, list):
         return [_sdk_object_to_dict(item) for item in payload]
@@ -4372,6 +4368,8 @@ def create_project_job(
     spot: bool | None = None,
     max_runtime_seconds: int | None = None,
     related_image_uid: str | None = None,
+    automatic_deployment: bool = False,
+    automatic_redeployment_tag_regex: str | None = None,
     timeout: int | None = None,
 ) -> dict[str, Any]:
     """
@@ -4440,6 +4438,10 @@ def create_project_job(
             spot=spot,
             max_runtime_seconds=max_runtime_seconds,
             related_image_uid=related_image_uid,
+            automatic_deployment=automatic_deployment,
+            automatic_redeployment_policy={
+                "tag_regex": automatic_redeployment_tag_regex,
+            },
             timeout=timeout,
         )
         if isinstance(created, dict):
@@ -5031,9 +5033,7 @@ def create_project(
 
     if default_metatables_data_source_uid in (None, ""):
         raise ValueError("default_metatables_data_source_uid is required")
-    payload["default_metatables_data_source_uid"] = str(
-        default_metatables_data_source_uid
-    )
+    payload["default_metatables_data_source_uid"] = str(default_metatables_data_source_uid)
     if default_base_image_uid not in (None, ""):
         payload["default_base_image_uid"] = str(default_base_image_uid)
     if github_org_uid not in (None, ""):

{mainsequence-6.0.30 → mainsequence-6.0.32}/mainsequence/cli/cli.py

@@ -746,11 +746,14 @@ def _render_projects_table(items: list[dict], base_dir: str, org_slug: str) -> s
     for p in items:
         public_id = _project_identity_value(p)
         name = p.get("project_name") or "(unnamed)"
-        branches = ", ".join(
-            str(branch.get("repository_branch") or "-")
-            for branch in list(p.get("branches") or [])
-            if isinstance(branch, dict)
-        ) or "-"
+        branches = (
+            ", ".join(
+                str(branch.get("repository_branch") or "-")
+                for branch in list(p.get("branches") or [])
+                if isinstance(branch, dict)
+            )
+            or "-"
+        )
 
         local_path = _find_local_project_dir(base_dir, org_slug, public_id, name)
         local = "Local" if local_path else "-"
@@ -2454,9 +2457,7 @@ def login(
                     "tool": handoff["mcp_tool"],
                     "arguments": handoff["mcp_arguments"],
                 }
-                info(
-                    "Authorize this CLI from the connected Main Sequence MCP session:"
-                )
+                info("Authorize this CLI from the connected Main Sequence MCP session:")
                 typer.echo(json.dumps(tool_call, separators=(",", ":")))
 
             flow = login_via_mcp_handoff(
@@ -2466,9 +2467,7 @@ def login(
             access = (flow.get("access") or "").strip()
             refresh = (flow.get("refresh") or "").strip()
             if not access or not refresh:
-                raise ApiError(
-                    "MCP handoff login did not return access and refresh tokens."
-                )
+                raise ApiError("MCP handoff login did not return access and refresh tokens.")
 
             persisted = cfg.save_tokens("", access, refresh)
             user_payload = flow.get("user")
@@ -2480,9 +2479,7 @@ def login(
                 if isinstance(profile, dict):
                     username = (profile.get("username") or "").strip()
             if username:
-                persisted = bool(
-                    cfg.save_tokens(username, access, refresh) and persisted
-                )
+                persisted = bool(cfg.save_tokens(username, access, refresh) and persisted)
 
             res = {
                 "username": username,
@@ -2801,9 +2798,7 @@ def _organization_teams_list_impl(
         )
 
     if rows:
-        print_table(
-            "Organization Teams", ["UID", "Name", "Description", "Members", "Active"], rows
-        )
+        print_table("Organization Teams", ["UID", "Name", "Description", "Members", "Active"], rows)
     else:
         info("No organization teams.")
     info(f"Total organization teams: {len(teams_payload)}")
@@ -4013,9 +4008,7 @@ def _agent_session_list_impl(
         command_label="Agent Sessions",
     )
     if agent_uid and any(key in filters for key in ("agent_uid", "agent_uid__in")):
-        error(
-            "Do not pass `--filter agent_uid=...` with `--agent-uid`. " "Use only one agent scope."
-        )
+        error("Do not pass `--filter agent_uid=...` with `--agent-uid`. Use only one agent scope.")
         raise typer.Exit(1)
 
     _require_login()
@@ -4511,7 +4504,6 @@ def _parse_cli_csv_list(values: list[str] | None) -> list[str]:
     return items
 
 
-
 def _data_node_storage_list_impl(
     timeout: int | None,
     filter_entries: list[str] | None,
@@ -4772,7 +4764,6 @@ def _print_data_node_storage_search_section(
     return len(storages)
 
 
-
 @agent.command("list")
 def agent_list_cmd(
     filter_entries: list[str] | None = typer.Option(None, "--filter", help=LIST_FILTER_OPTION_HELP),
@@ -7515,11 +7506,7 @@ def project_create_cmd(
             ds_rows: list[list[str]] = []
             for item in ds_items:
                 uid = _require_item_uid(item, prompt_label="data source uid")
-                ds_name = (
-                    item.get("display_name")
-                    or item.get("class_type")
-                    or f"data-source-{uid}"
-                )
+                ds_name = item.get("display_name") or item.get("class_type") or f"data-source-{uid}"
                 ds_details = (
                     f"class={item.get('class_type') or '-'}, status={item.get('status') or '-'}"
                 )
@@ -8843,7 +8830,9 @@ def project_images_create_cmd(
     path: str | None = typer.Option(
         None, "--path", help="Project repository path (default: current project)"
     ),
-    base_image_uid: str | None = typer.Option(None, "--base-image-uid", help="Project base image UID"),
+    base_image_uid: str | None = typer.Option(
+        None, "--base-image-uid", help="Project base image UID"
+    ),
     timeout: int = typer.Option(
         300, "--timeout", help="Maximum wait time in seconds for the image to become ready."
     ),
@@ -8905,7 +8894,9 @@ def project_create_image_cmd(
     path: str | None = typer.Option(
         None, "--path", help="Project repository path (default: current project)"
     ),
-    base_image_uid: str | None = typer.Option(None, "--base-image-uid", help="Project base image UID"),
+    base_image_uid: str | None = typer.Option(
+        None, "--base-image-uid", help="Project base image UID"
+    ),
     timeout: int = typer.Option(
         300, "--timeout", help="Maximum wait time in seconds for the image to become ready."
     ),
@@ -8990,6 +8981,33 @@ def _project_jobs_list_impl(
             ],
             rows,
         )
+
+        deployment_rows = [
+            [
+                str(job.get("uid") or "-"),
+                str(job.get("project_repo_hash") or "-"),
+                str(job.get("related_image_uid") or job.get("related_image") or "-"),
+                str(job.get("image_status") or "-"),
+                str(bool(job.get("automatic_deployment"))).lower(),
+                str(
+                    (job.get("automatic_redeployment_policy") or {}).get("tag_regex")
+                    or "every qualifying exact event"
+                ),
+            ]
+            for job in jobs
+        ]
+        print_table(
+            "Project Job Image And Promotion State",
+            [
+                "Job UID",
+                "Exact Commit",
+                "Image UID",
+                "Image Status",
+                "Automatic Deployment",
+                "Promotion Tag Regex",
+            ],
+            deployment_rows,
+        )
     else:
         info("No project jobs.")
     info(f"Total jobs: {len(jobs)}")
@@ -9048,6 +9066,21 @@ def _project_job_runs_list_impl(
             ],
             rows,
         )
+
+        snapshot_rows = [
+            [
+                str(run.get("uid") or "-"),
+                str(run.get("runtime_image_uid") or "-"),
+                str(run.get("runtime_image_digest") or "-"),
+                str(run.get("commit_hash") or "-"),
+            ]
+            for run in runs
+        ]
+        print_table(
+            "Immutable Job Run Image Snapshots",
+            ["Job Run UID", "Runtime Image UID", "Runtime Image Digest", "Commit Hash"],
+            snapshot_rows,
+        )
     else:
         info("No job runs.")
     info(f"Total job runs: {len(runs)}")
@@ -9358,6 +9391,8 @@ def _project_jobs_create_impl(
     gpu_type: str | None,
     spot: bool | None,
     max_runtime_seconds: int | None,
+    automatic_deployment: bool,
+    automatic_redeployment_tag_regex: str | None,
     timeout: int | None,
 ) -> None:
     _require_login()
@@ -9384,34 +9419,46 @@ def _project_jobs_create_impl(
         error("Job name is required.")
         raise typer.Exit(1)
 
-    try:
-        project_images = list_project_images(
-            related_project_branch_uid=project_branch_uid,
-            timeout=timeout,
+    if automatic_deployment:
+        if related_image_uid is not None:
+            error(
+                "Do not provide --related-image-uid with --automatic-deployment. "
+                "The backend derives and prepares the initial exact image."
+            )
+            raise typer.Exit(1)
+        info(
+            "Automatic deployment: the backend will derive the initial exact image "
+            "from the ProjectBranch synchronized commit."
         )
-    except ApiError as e:
-        error(f"Project images fetch failed: {e}")
-        raise typer.Exit(1) from e
+    else:
+        try:
+            project_images = list_project_images(
+                related_project_branch_uid=project_branch_uid,
+                timeout=timeout,
+            )
+        except ApiError as e:
+            error(f"Project images fetch failed: {e}")
+            raise typer.Exit(1) from e
 
-    if related_image_uid is None and project_images:
-        image_rows = [
-            [
-                str(img.get("uid") or "-"),
-                str(img.get("project_repo_hash") or "-"),
-                _format_base_image_label(img.get("base_image")),
+        if related_image_uid is None and project_images:
+            image_rows = [
+                [
+                    str(img.get("uid") or "-"),
+                    str(img.get("project_repo_hash") or "-"),
+                    _format_base_image_label(img.get("base_image")),
+                ]
+                for img in project_images
             ]
-            for img in project_images
-        ]
-        related_image_uid = _prompt_select_uid(
-            title="Available Project Images",
-            prompt_label="Related image UID",
-            items=project_images,
-            rows=image_rows,
-        )
+            related_image_uid = _prompt_select_uid(
+                title="Available Project Images",
+                prompt_label="Related image UID",
+                items=project_images,
+                rows=image_rows,
+            )
 
-    if related_image_uid is None:
-        error("related_image_uid is required for jobs.")
-        raise typer.Exit(1)
+        if related_image_uid is None:
+            error("related_image_uid is required when automatic deployment is disabled.")
+            raise typer.Exit(1)
 
     if execution_path is None and app_name is None:
         execution_path = (
@@ -9487,6 +9534,8 @@ def _project_jobs_create_impl(
             spot=spot,
             max_runtime_seconds=max_runtime_seconds,
             related_image_uid=related_image_uid,
+            automatic_deployment=automatic_deployment,
+            automatic_redeployment_tag_regex=automatic_redeployment_tag_regex,
             timeout=timeout,
         )
     except ApiError as e:
@@ -9523,6 +9572,19 @@ def _project_jobs_create_impl(
                 "Max Runtime Seconds",
                 str(created.get("max_runtime_seconds") or max_runtime_seconds),
             ),
+            ("Image Status", str(created.get("image_status") or "-")),
+            ("Exact Commit", str(created.get("project_repo_hash") or "-")),
+            (
+                "Automatic Deployment",
+                str(created.get("automatic_deployment", automatic_deployment)).lower(),
+            ),
+            (
+                "Promotion Tag Regex",
+                str(
+                    (created.get("automatic_redeployment_policy") or {}).get("tag_regex")
+                    or "every qualifying exact event"
+                ),
+            ),
         ],
     )
 
@@ -9548,7 +9610,10 @@ def project_jobs_create_cmd(
         "related_image_uid",
         None,
         "--related-image-uid",
-        extra_help="Use the public project image UID.",
+        extra_help=(
+            "Required for manually pinned Jobs. Omit it with "
+            "--automatic-deployment so the backend owns initial image preparation."
+        ),
     ),
     schedule_type: str | None = pydantic_option(
         INTERVAL_SCHEDULE_MODEL_REF,
@@ -9620,12 +9685,30 @@ def project_jobs_create_cmd(
         "--max-runtime-seconds",
         extra_help="Defaults to 86400 when omitted.",
     ),
+    automatic_deployment: bool = typer.Option(
+        False,
+        "--automatic-deployment/--no-automatic-deployment",
+        help=(
+            "Let the backend derive and prepare the initial exact image from the "
+            "synchronized ProjectBranch commit, then promote future qualifying exact events."
+        ),
+    ),
+    automatic_redeployment_tag_regex: str | None = typer.Option(
+        None,
+        "--automatic-redeployment-tag-regex",
+        help=(
+            "Full-match regex for qualifying immutable Git tags. Omit it to "
+            "allow every qualifying exact event while automatic deployment is enabled."
+        ),
+    ),
     timeout: int | None = typer.Option(None, "--timeout", help="Request timeout in seconds"),
 ):
     """
     Create a job for a project.
 
-    Uses SDK client `Job.create()` as the single source of truth.
+    Uses SDK client `Job.create()` as the single source of truth. Manual Jobs
+    require `--related-image-uid`; automatically deployed Jobs forbid it and
+    delegate initial exact-image preparation to the backend.
     When compute settings are omitted, the CLI applies safe defaults:
     `cpu_request=0.25`, `memory_request=0.5`, `spot=false`, `max_runtime_seconds=86400`.
     If schedule arguments are omitted, the CLI asks whether to build an interval or crontab schedule.
@@ -9657,6 +9740,8 @@ def project_jobs_create_cmd(
         gpu_type=gpu_type,
         spot=spot,
         max_runtime_seconds=max_runtime_seconds,
+        automatic_deployment=automatic_deployment,
+        automatic_redeployment_tag_regex=automatic_redeployment_tag_regex,
         timeout=timeout,
     )
 

{mainsequence-6.0.30 → mainsequence-6.0.32}/mainsequence/client/metatables/core.py

@@ -533,40 +533,16 @@ class MetaTableOperationScope(BasePydanticModel):
         min_length=1,
         validation_alias=AliasChoices("data_source_uid", "dataSourceUid"),
         description=(
-            "Public UID of the DataSource that owns the compiled "
-            "SQL execution connection. If omitted, the SDK resolves the "
-            "configured project/session default data source. Scoped MetaTables "
-            "are the permission contract, not the source of execution routing."
+            "Optional public UID of the DataSource that owns the compiled "
+            "SQL execution connection. If omitted, the backend derives the "
+            "connection from the declared MetaTable scope and rejects mixed "
+            "data sources."
         ),
     )
     tables: list[MetaTableOperationScopeTable] = Field(..., min_length=1)
 
     model_config = ConfigDict(populate_by_name=True)
 
-    @model_validator(mode="after")
-    def resolve_default_data_source_uid(self):
-        if self.data_source_uid not in (None, ""):
-            self.data_source_uid = str(self.data_source_uid)
-            return self
-
-        try:
-            data_source = get_session_data_source()
-        except Exception as exc:
-            raise ValueError(
-                "MetaTable compiled SQL scope requires data_source_uid or a "
-                "configured project/session default data source. "
-                f"Default data source resolution failed: {exc}"
-            ) from exc
-
-        uid = getattr(data_source, "uid", None) or getattr(data_source, "data_source_uid", None)
-        if uid in (None, ""):
-            raise ValueError(
-                "Configured project/session default data source does not expose a uid."
-            )
-        self.data_source_uid = str(uid)
-        return self
-
-
 class MetaTableProjectContextRequest(BaseModel):
     model_config = ConfigDict(extra="forbid", frozen=True)