workly 0.3.0__tar.gz → 0.4.3__tar.gz

{workly-0.3.0 → workly-0.4.3}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: workly
-Version: 0.3.0
+Version: 0.4.3
 Summary: Python SDK for Workly, the open-source HTTP task queue
 Keywords: task queue,background jobs,webhooks,http
 License-Expression: Apache-2.0
 License-File: LICENSE
-Classifier: Development Status :: 3 - Alpha
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: Framework :: AsyncIO
 Classifier: Intended Audience :: Developers
 Classifier: Programming Language :: Python :: 3
@@ -27,7 +27,7 @@ pip install workly          # the SDK
 pip install "workly[cli]"   # the SDK plus the workly binary, for `workly dev`
 ```
 
-Requires Python 3.11+. The only dependency is [httpx](https://www.python-httpx.org). The `cli` extra adds [`workly-cli`](https://pypi.org/project/workly-cli/), the server binary (about 7 MB), which you usually want on dev machines and not in production images.
+Requires Python 3.11+. The only dependency is [httpx](https://www.python-httpx.org). The `cli` extra adds [`workly-cli`](https://pypi.org/project/workly-cli/), the `workly` binary (about 7 MB): the dev server (`workly dev`) and the CLI, including `workly validate`, `plan` and `deploy` for [configuration as code](https://workly.run/docs/deploy/). Install it on dev machines and in CI; production images that only enqueue tasks don't need it.
 
 ## Quickstart
 
@@ -186,24 +186,27 @@ Runs are delivered with `Workly-Run-Id`, `Workly-Schedule` and `Workly-Scheduled
 
 A group tracks a set of tasks and enqueues a callback task once it's closed and they've all finished: a `POST` of the group's status and counts. See [Task groups](https://workly.run/docs/groups/). Like every callback, it goes to Workly's `_callbacks` queue.
 
+`groups.enqueue` does the whole job in one call: it creates the group, enqueues the tasks into it in batches of 100 and closes it, and it's safe to retry with the same `idempotency_key`.
+
 ```python
 from workly import Callback
 
-group = client.groups.create(
+group = client.groups.enqueue(
+    "stock-prices",
+    (
+        {"url": "https://api.example.com/jobs/update-stock-price", "payload": {"symbol": c.symbol}}
+        for c in companies
+    ),
     name="daily-stock-prices",
-    idempotency_key="stock-prices-2026-10-19",  # creating it again returns this group
+    idempotency_key="stock-prices-2026-10-19",  # a retry finishes this group rather than start another
     # or a dict: {"url": "https://api.example.com/jobs/refresh-views"}
     on_complete=Callback(url="https://api.example.com/jobs/refresh-views"),
 )
-for company in companies:
-    client.tasks.enqueue(
-        "stock-prices",
-        url="https://api.example.com/jobs/update-stock-price",
-        payload={"symbol": company.symbol},
-        group_id=group.id,
-    )
-client.groups.close(group.id)  # no more tasks; it completes once they've finished
+```
 
+Or step by step: `client.groups.create(...)`, then `client.tasks.enqueue(..., group_id=group.id)` or `enqueue_many` for each task, then `client.groups.close(group.id)`, after which it completes once its tasks have finished.
+
+```python
 group = client.groups.get(group.id)
 print(group.status, group.total, group.succeeded, group.failed, group.remaining)
 for task in client.tasks.list(group_id=group.id, state="failed"):

{workly-0.3.0 → workly-0.4.3}/README.md

@@ -7,7 +7,7 @@ pip install workly          # the SDK
 pip install "workly[cli]"   # the SDK plus the workly binary, for `workly dev`
 ```
 
-Requires Python 3.11+. The only dependency is [httpx](https://www.python-httpx.org). The `cli` extra adds [`workly-cli`](https://pypi.org/project/workly-cli/), the server binary (about 7 MB), which you usually want on dev machines and not in production images.
+Requires Python 3.11+. The only dependency is [httpx](https://www.python-httpx.org). The `cli` extra adds [`workly-cli`](https://pypi.org/project/workly-cli/), the `workly` binary (about 7 MB): the dev server (`workly dev`) and the CLI, including `workly validate`, `plan` and `deploy` for [configuration as code](https://workly.run/docs/deploy/). Install it on dev machines and in CI; production images that only enqueue tasks don't need it.
 
 ## Quickstart
 
@@ -166,24 +166,27 @@ Runs are delivered with `Workly-Run-Id`, `Workly-Schedule` and `Workly-Scheduled
 
 A group tracks a set of tasks and enqueues a callback task once it's closed and they've all finished: a `POST` of the group's status and counts. See [Task groups](https://workly.run/docs/groups/). Like every callback, it goes to Workly's `_callbacks` queue.
 
+`groups.enqueue` does the whole job in one call: it creates the group, enqueues the tasks into it in batches of 100 and closes it, and it's safe to retry with the same `idempotency_key`.
+
 ```python
 from workly import Callback
 
-group = client.groups.create(
+group = client.groups.enqueue(
+    "stock-prices",
+    (
+        {"url": "https://api.example.com/jobs/update-stock-price", "payload": {"symbol": c.symbol}}
+        for c in companies
+    ),
     name="daily-stock-prices",
-    idempotency_key="stock-prices-2026-10-19",  # creating it again returns this group
+    idempotency_key="stock-prices-2026-10-19",  # a retry finishes this group rather than start another
     # or a dict: {"url": "https://api.example.com/jobs/refresh-views"}
     on_complete=Callback(url="https://api.example.com/jobs/refresh-views"),
 )
-for company in companies:
-    client.tasks.enqueue(
-        "stock-prices",
-        url="https://api.example.com/jobs/update-stock-price",
-        payload={"symbol": company.symbol},
-        group_id=group.id,
-    )
-client.groups.close(group.id)  # no more tasks; it completes once they've finished
+```
 
+Or step by step: `client.groups.create(...)`, then `client.tasks.enqueue(..., group_id=group.id)` or `enqueue_many` for each task, then `client.groups.close(group.id)`, after which it completes once its tasks have finished.
+
+```python
 group = client.groups.get(group.id)
 print(group.status, group.total, group.succeeded, group.failed, group.remaining)
 for task in client.tasks.list(group_id=group.id, state="failed"):

{workly-0.3.0 → workly-0.4.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "workly"
-version = "0.3.0"
+version = "0.4.3"
 description = "Python SDK for Workly, the open-source HTTP task queue"
 readme = "README.md"
 license = "Apache-2.0"
@@ -14,7 +14,7 @@ keywords = [
     "http",
 ]
 classifiers = [
-    "Development Status :: 3 - Alpha",
+    "Development Status :: 5 - Production/Stable",
     "Framework :: AsyncIO",
     "Intended Audience :: Developers",
     "Programming Language :: Python :: 3",

{workly-0.3.0 → workly-0.4.3}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "workly"
-version = "0.3.0"
+version = "0.4.3"
 description = "Python SDK for Workly, the open-source HTTP task queue"
 readme = "README.md"
 license = "Apache-2.0"
@@ -9,7 +9,7 @@ requires-python = ">=3.11"
 dependencies = ["httpx>=0.27,<1"]
 keywords = ["task queue", "background jobs", "webhooks", "http"]
 classifiers = [
-    "Development Status :: 3 - Alpha",
+    "Development Status :: 5 - Production/Stable",
     "Framework :: AsyncIO",
     "Intended Audience :: Developers",
     "Programming Language :: Python :: 3",

{workly-0.3.0 → workly-0.4.3}/src/workly/__init__.py

@@ -7,9 +7,21 @@ from ._async_client import (
     AsyncRuns,
     AsyncSchedules,
     AsyncTasks,
+    AsyncWorkflowRuns,
+    AsyncWorkflows,
     AsyncWorkly,
 )
-from ._client import Alerts, Groups, Queues, Runs, Schedules, Tasks, Workly
+from ._client import (
+    Alerts,
+    Groups,
+    Queues,
+    Runs,
+    Schedules,
+    Tasks,
+    WorkflowRuns,
+    Workflows,
+    Workly,
+)
 from ._errors import (
     APIConnectionError,
     APIError,
@@ -34,6 +46,7 @@ from ._models import (
     AttemptOutcome,
     Callback,
     CronPreview,
+    CronSpan,
     Delivery,
     EnqueueResult,
     FailureGroup,
@@ -55,12 +68,23 @@ from ._models import (
     Schedule,
     SkipReason,
     SortOrder,
+    StepStatus,
     Task,
     TaskPage,
     TaskSort,
     TaskState,
+    TaskWorkflowStep,
+    WaitFor,
+    Workflow,
+    WorkflowRun,
+    WorkflowRunPage,
+    WorkflowRunStatus,
+    WorkflowRunSummary,
+    WorkflowStep,
+    WorkflowStepConfig,
+    WorkflowStepTask,
 )
-from ._requests import CallbackDict, GroupCallbackSpec, TaskSpec, __version__
+from ._requests import CallbackDict, GroupCallbackSpec, TaskSpec, WorkflowStepSpec, __version__
 from ._signing import sign, verify_signature
 
 __all__ = [
@@ -79,6 +103,8 @@ __all__ = [
     "AsyncRuns",
     "AsyncSchedules",
     "AsyncTasks",
+    "AsyncWorkflowRuns",
+    "AsyncWorkflows",
     "AsyncWorkly",
     "Attempt",
     "AttemptOutcome",
@@ -88,6 +114,7 @@ __all__ = [
     "CallbackDict",
     "ConflictError",
     "CronPreview",
+    "CronSpan",
     "Delivery",
     "EnqueueResult",
     "FailureGroup",
@@ -119,12 +146,26 @@ __all__ = [
     "SignatureVerificationError",
     "SkipReason",
     "SortOrder",
+    "StepStatus",
     "Task",
     "TaskPage",
     "TaskSort",
     "TaskSpec",
     "TaskState",
+    "TaskWorkflowStep",
     "Tasks",
+    "WaitFor",
+    "Workflow",
+    "WorkflowRun",
+    "WorkflowRunPage",
+    "WorkflowRunStatus",
+    "WorkflowRunSummary",
+    "WorkflowRuns",
+    "WorkflowStep",
+    "WorkflowStepConfig",
+    "WorkflowStepSpec",
+    "WorkflowStepTask",
+    "Workflows",
     "Workly",
     "WorklyError",
     "__version__",

{workly-0.3.0 → workly-0.4.3}/src/workly/_async_client.py

@@ -39,6 +39,11 @@ from ._models import (
     TaskPage,
     TaskSort,
     TaskState,
+    Workflow,
+    WorkflowRun,
+    WorkflowRunPage,
+    WorkflowRunStatus,
+    WorkflowRunSummary,
 )
 
 
@@ -69,6 +74,8 @@ class AsyncWorkly:
         self.groups = AsyncGroups(self)
         self.schedules = AsyncSchedules(self)
         self.runs = AsyncRuns(self)
+        self.workflows = AsyncWorkflows(self)
+        self.workflow_runs = AsyncWorkflowRuns(self)
         self.alerts = AsyncAlerts(self)
 
     @property
@@ -386,6 +393,37 @@ class AsyncGroups:
         req = r.create_group(name, idempotency_key, on_complete)
         return Group._from_json(await self._client._send(req))
 
+    async def enqueue(
+        self,
+        queue: str,
+        tasks: Iterable[r.TaskSpec],
+        *,
+        name: str | None = None,
+        idempotency_key: str | None = None,
+        on_complete: Callback | r.CallbackDict | None = None,
+        traceparent: str | None = None,
+    ) -> Group:
+        """Create a group, enqueue `tasks` into it and close it: the whole job in one call.
+
+        Tasks go in batches of 100, each joining the group. Returns the closed
+        group, which completes (and enqueues `on_complete`) once its tasks
+        have finished.
+
+        Safe to retry with the same `idempotency_key` and tasks: the group is
+        found again instead of created, a task without its own
+        `idempotency_key` gets one from the group and its position so none is
+        enqueued twice, and a group that is already closed is returned as it
+        is. Without `idempotency_key`, a retry starts a second group.
+        """
+        group = await self.create(
+            name=name, idempotency_key=idempotency_key, on_complete=on_complete
+        )
+        if group.status != "open":
+            return group
+        for batch in r.group_batches(group.id, tasks):
+            await self._client.tasks.enqueue_many(queue, batch, traceparent=traceparent)
+        return await self.close(group.id)
+
     async def get(self, group_id: str) -> Group:
         return Group._from_json(await self._client._send(r.get_group(group_id)))
 
@@ -446,7 +484,8 @@ class AsyncSchedules:
         name: str,
         *,
         cron: str,
-        url: str,
+        url: str | None = None,
+        workflow: str | None = None,
         timezone: str | None = None,
         payload: Any = None,
         body: str | None = None,
@@ -465,6 +504,9 @@ class AsyncSchedules:
     ) -> Schedule:
         """Create or update a schedule that calls `url` on `cron` (read in `timezone`, default UTC).
 
+        With `workflow` instead of `url`, each run starts a run of that
+        workflow, with `payload` as its input.
+
         Declarative: the schedule ends up with exactly these settings, and
         omitted ones are reset to their defaults. It doesn't change whether
         the schedule is paused. `next_runs` on the result shows when it runs
@@ -474,6 +516,7 @@ class AsyncSchedules:
         config = dict(
             cron=cron,
             url=url,
+            workflow=workflow,
             timezone=timezone,
             payload=payload,
             body=body,
@@ -504,12 +547,22 @@ class AsyncSchedules:
         """Fire again from the next slot. Slots that passed while paused aren't run."""
         return Schedule._from_json(await self._client._send(r.resume_schedule(name)))
 
-    async def preview_cron(self, cron: str, *, timezone: str | None = None) -> CronPreview:
+    async def preview_cron(
+        self,
+        cron: str,
+        *,
+        timezone: str | None = None,
+        from_: datetime | None = None,
+        until: datetime | None = None,
+    ) -> CronPreview:
         """The next 5 times a schedule with this cron would run, without creating it.
 
-        `timezone` is an IANA name; the default is UTC.
+        `timezone` is an IANA name; the default is UTC. With `from_` and `until` (at
+        most 35 days apart), `spans` lists every run in that window.
         """
-        return CronPreview._from_json(await self._client._send(r.preview_cron(cron, timezone)))
+        return CronPreview._from_json(
+            await self._client._send(r.preview_cron(cron, timezone, from_, until))
+        )
 
     async def trigger(self, name: str) -> Run:
         """Run the schedule now, even if paused. Doesn't change when it next fires."""
@@ -601,3 +654,106 @@ class AsyncAlerts:
     async def test(self, channel_id: str) -> None:
         """Send a test alert to the channel now. Raises APIError (alert_failed) if it can't."""
         await self._client._send(r.test_alert_channel(channel_id))
+
+
+class AsyncWorkflows:
+    """Workflows: DAGs of HTTP steps, each delivered as a task once its dependencies succeed.
+
+    A step's endpoint can hand its completion to a task group by answering
+    2xx with `Workly-Wait-For: group/<id>`; the step then finishes when the
+    group does. Define workflows in workly.yaml (`workly deploy`) or with `put`.
+    """
+
+    def __init__(self, client: AsyncWorkly) -> None:
+        self._client = client
+
+    async def list(self) -> list[Workflow]:
+        """All workflows, by name, with their latest run."""
+        data = await self._client._send(r.list_workflows())
+        return [Workflow._from_json(w) for w in data["workflows"]]
+
+    async def get(self, name: str) -> Workflow:
+        return Workflow._from_json(await self._client._send(r.get_workflow(name)))
+
+    async def put(
+        self,
+        name: str,
+        *,
+        steps: list[r.WorkflowStepSpec],
+        max_duration_seconds: float | None = None,
+    ) -> Workflow:
+        """Create or update a workflow with exactly these steps.
+
+        Runs already started keep the definition they started with. A run
+        still going after `max_duration_seconds` (default a day) fails.
+        """
+        req = r.put_workflow(name, steps, max_duration_seconds)
+        return Workflow._from_json(await self._client._send(req))
+
+    async def delete(self, name: str) -> None:
+        """Delete the definition. Its runs are kept, and those going finish."""
+        await self._client._send(r.delete_workflow(name))
+
+    async def trigger(
+        self, name: str, *, input: Any = None, idempotency_key: str | None = None
+    ) -> WorkflowRun:
+        """Start a run. `input` (any JSON) is sent to every step without a body of its own.
+
+        With an `idempotency_key` already used for a run of this workflow,
+        that run is returned and nothing starts.
+        """
+        req = r.start_workflow_run(name, input, idempotency_key)
+        return WorkflowRun._from_json(await self._client._send(req))
+
+
+class AsyncWorkflowRuns:
+    """Runs of workflows, with each step's status, task and what it waits for."""
+
+    def __init__(self, client: AsyncWorkly) -> None:
+        self._client = client
+
+    async def list(
+        self,
+        *,
+        workflow: str | None = None,
+        status: WorkflowRunStatus | None = None,
+        page_size: int | None = None,
+    ) -> AsyncIterator[WorkflowRunSummary]:
+        """Runs of every workflow, or of one, newest first, fetching pages as needed."""
+        cursor: str | None = None
+        while True:
+            page = await self.list_page(
+                workflow=workflow, status=status, cursor=cursor, limit=page_size
+            )
+            for run in page.workflow_runs:
+                yield run
+            if page.next_cursor is None:
+                return
+            cursor = page.next_cursor
+
+    async def list_page(
+        self,
+        *,
+        workflow: str | None = None,
+        status: WorkflowRunStatus | None = None,
+        cursor: str | None = None,
+        limit: int | None = None,
+    ) -> WorkflowRunPage:
+        """One page of `list`. Pass the previous page's `next_cursor` as `cursor`."""
+        req = r.list_workflow_runs(workflow=workflow, status=status, cursor=cursor, limit=limit)
+        return WorkflowRunPage._from_json(await self._client._send(req))
+
+    async def get(self, run_id: str) -> WorkflowRun:
+        return WorkflowRun._from_json(await self._client._send(r.get_workflow_run(run_id)))
+
+    async def cancel(self, run_id: str) -> WorkflowRun:
+        """Stop a run: steps not yet started are skipped. Requests in flight finish."""
+        return WorkflowRun._from_json(await self._client._send(r.cancel_workflow_run(run_id)))
+
+    async def retry(self, run_id: str) -> WorkflowRun:
+        """Reopen a failed or cancelled run under the definition it started with.
+
+        Steps that succeeded stay; the rest run again, each on its old task with a
+        fresh set of attempts. A step that was waiting for a group follows the group.
+        """
+        return WorkflowRun._from_json(await self._client._send(r.retry_workflow_run(run_id)))

{workly-0.3.0 → workly-0.4.3}/src/workly/_client.py

@@ -39,6 +39,11 @@ from ._models import (
     TaskPage,
     TaskSort,
     TaskState,
+    Workflow,
+    WorkflowRun,
+    WorkflowRunPage,
+    WorkflowRunStatus,
+    WorkflowRunSummary,
 )
 
 
@@ -69,6 +74,8 @@ class Workly:
         self.groups = Groups(self)
         self.schedules = Schedules(self)
         self.runs = Runs(self)
+        self.workflows = Workflows(self)
+        self.workflow_runs = WorkflowRuns(self)
         self.alerts = Alerts(self)
 
     @property
@@ -385,6 +392,35 @@ class Groups:
         req = r.create_group(name, idempotency_key, on_complete)
         return Group._from_json(self._client._send(req))
 
+    def enqueue(
+        self,
+        queue: str,
+        tasks: Iterable[r.TaskSpec],
+        *,
+        name: str | None = None,
+        idempotency_key: str | None = None,
+        on_complete: Callback | r.CallbackDict | None = None,
+        traceparent: str | None = None,
+    ) -> Group:
+        """Create a group, enqueue `tasks` into it and close it: the whole job in one call.
+
+        Tasks go in batches of 100, each joining the group. Returns the closed
+        group, which completes (and enqueues `on_complete`) once its tasks
+        have finished.
+
+        Safe to retry with the same `idempotency_key` and tasks: the group is
+        found again instead of created, a task without its own
+        `idempotency_key` gets one from the group and its position so none is
+        enqueued twice, and a group that is already closed is returned as it
+        is. Without `idempotency_key`, a retry starts a second group.
+        """
+        group = self.create(name=name, idempotency_key=idempotency_key, on_complete=on_complete)
+        if group.status != "open":
+            return group
+        for batch in r.group_batches(group.id, tasks):
+            self._client.tasks.enqueue_many(queue, batch, traceparent=traceparent)
+        return self.close(group.id)
+
     def get(self, group_id: str) -> Group:
         return Group._from_json(self._client._send(r.get_group(group_id)))
 
@@ -444,7 +480,8 @@ class Schedules:
         name: str,
         *,
         cron: str,
-        url: str,
+        url: str | None = None,
+        workflow: str | None = None,
         timezone: str | None = None,
         payload: Any = None,
         body: str | None = None,
@@ -463,6 +500,9 @@ class Schedules:
     ) -> Schedule:
         """Create or update a schedule that calls `url` on `cron` (read in `timezone`, default UTC).
 
+        With `workflow` instead of `url`, each run starts a run of that
+        workflow, with `payload` as its input.
+
         Declarative: the schedule ends up with exactly these settings, and
         omitted ones are reset to their defaults. It doesn't change whether
         the schedule is paused. `next_runs` on the result shows when it runs
@@ -472,6 +512,7 @@ class Schedules:
         config = dict(
             cron=cron,
             url=url,
+            workflow=workflow,
             timezone=timezone,
             payload=payload,
             body=body,
@@ -502,12 +543,22 @@ class Schedules:
         """Fire again from the next slot. Slots that passed while paused aren't run."""
         return Schedule._from_json(self._client._send(r.resume_schedule(name)))
 
-    def preview_cron(self, cron: str, *, timezone: str | None = None) -> CronPreview:
+    def preview_cron(
+        self,
+        cron: str,
+        *,
+        timezone: str | None = None,
+        from_: datetime | None = None,
+        until: datetime | None = None,
+    ) -> CronPreview:
         """The next 5 times a schedule with this cron would run, without creating it.
 
-        `timezone` is an IANA name; the default is UTC.
+        `timezone` is an IANA name; the default is UTC. With `from_` and `until` (at
+        most 35 days apart), `spans` lists every run in that window.
         """
-        return CronPreview._from_json(self._client._send(r.preview_cron(cron, timezone)))
+        return CronPreview._from_json(
+            self._client._send(r.preview_cron(cron, timezone, from_, until))
+        )
 
     def trigger(self, name: str) -> Run:
         """Run the schedule now, even if paused. Doesn't change when it next fires."""
@@ -594,3 +645,103 @@ class Alerts:
     def test(self, channel_id: str) -> None:
         """Send a test alert to the channel now. Raises APIError (alert_failed) if it can't."""
         self._client._send(r.test_alert_channel(channel_id))
+
+
+class Workflows:
+    """Workflows: DAGs of HTTP steps, each delivered as a task once its dependencies succeed.
+
+    A step's endpoint can hand its completion to a task group by answering
+    2xx with `Workly-Wait-For: group/<id>`; the step then finishes when the
+    group does. Define workflows in workly.yaml (`workly deploy`) or with `put`.
+    """
+
+    def __init__(self, client: Workly) -> None:
+        self._client = client
+
+    def list(self) -> list[Workflow]:
+        """All workflows, by name, with their latest run."""
+        data = self._client._send(r.list_workflows())
+        return [Workflow._from_json(w) for w in data["workflows"]]
+
+    def get(self, name: str) -> Workflow:
+        return Workflow._from_json(self._client._send(r.get_workflow(name)))
+
+    def put(
+        self,
+        name: str,
+        *,
+        steps: list[r.WorkflowStepSpec],
+        max_duration_seconds: float | None = None,
+    ) -> Workflow:
+        """Create or update a workflow with exactly these steps.
+
+        Runs already started keep the definition they started with. A run
+        still going after `max_duration_seconds` (default a day) fails.
+        """
+        req = r.put_workflow(name, steps, max_duration_seconds)
+        return Workflow._from_json(self._client._send(req))
+
+    def delete(self, name: str) -> None:
+        """Delete the definition. Its runs are kept, and those going finish."""
+        self._client._send(r.delete_workflow(name))
+
+    def trigger(
+        self, name: str, *, input: Any = None, idempotency_key: str | None = None
+    ) -> WorkflowRun:
+        """Start a run. `input` (any JSON) is sent to every step without a body of its own.
+
+        With an `idempotency_key` already used for a run of this workflow,
+        that run is returned and nothing starts.
+        """
+        req = r.start_workflow_run(name, input, idempotency_key)
+        return WorkflowRun._from_json(self._client._send(req))
+
+
+class WorkflowRuns:
+    """Runs of workflows, with each step's status, task and what it waits for."""
+
+    def __init__(self, client: Workly) -> None:
+        self._client = client
+
+    def list(
+        self,
+        *,
+        workflow: str | None = None,
+        status: WorkflowRunStatus | None = None,
+        page_size: int | None = None,
+    ) -> Iterator[WorkflowRunSummary]:
+        """Runs of every workflow, or of one, newest first, fetching pages as needed."""
+        cursor: str | None = None
+        while True:
+            page = self.list_page(workflow=workflow, status=status, cursor=cursor, limit=page_size)
+            yield from page.workflow_runs
+            if page.next_cursor is None:
+                return
+            cursor = page.next_cursor
+
+    def list_page(
+        self,
+        *,
+        workflow: str | None = None,
+        status: WorkflowRunStatus | None = None,
+        cursor: str | None = None,
+        limit: int | None = None,
+    ) -> WorkflowRunPage:
+        """One page of `list`. Pass the previous page's `next_cursor` as `cursor`."""
+        req = r.list_workflow_runs(workflow=workflow, status=status, cursor=cursor, limit=limit)
+        return WorkflowRunPage._from_json(self._client._send(req))
+
+    def get(self, run_id: str) -> WorkflowRun:
+        return WorkflowRun._from_json(self._client._send(r.get_workflow_run(run_id)))
+
+    def cancel(self, run_id: str) -> WorkflowRun:
+        """Stop a run: steps not yet started are skipped. Requests in flight finish."""
+        return WorkflowRun._from_json(self._client._send(r.cancel_workflow_run(run_id)))
+
+    def retry(self, run_id: str) -> WorkflowRun:
+        """Reopen a failed or cancelled run under the definition it started with.
+
+        Steps that succeeded stay; the rest run again, each on its old task with a
+        fresh set of attempts. A step that was waiting for a group follows the group.
+        """
+        return WorkflowRun._from_json(self._client._send(r.retry_workflow_run(run_id)))

{workly-0.3.0 → workly-0.4.3}/src/workly/_models.py

@@ -23,12 +23,21 @@ OverlapPolicy = Literal["skip", "allow"]
 MissedPolicy = Literal["skip", "catch_up"]
 GroupStatus = Literal["open", "running", "succeeded", "failed"]
 """`open` accepts tasks; `running` is closed with tasks unfinished; the other two are final."""
+WorkflowRunStatus = Literal["running", "succeeded", "failed", "cancelled"]
+StepStatus = Literal["blocked", "pending", "running", "waiting", "succeeded", "failed", "skipped"]
+"""`blocked` waits for its dependencies; `pending` has its task queued or retrying; `running` is
+being delivered; `waiting` succeeded and waits for its `wait_for` group; `skipped` never ran,
+or stopped waiting, because the run stopped."""
 
 
 def _time(value: str) -> datetime:
     return datetime.fromisoformat(value)
 
 
+def _list(value: Any) -> list[dict[str, Any]]:
+    return value or []
+
+
 def _opt_time(value: str | None) -> datetime | None:
     return None if value is None else _time(value)
 
@@ -70,6 +79,8 @@ class QueueActivity:
     """Median duration of the latest attempt of the succeeded and failed tasks counted."""
     p95_duration_ms: int | None = None
     """95th percentile of the same durations."""
+    duration_samples: int | None = None
+    """How many durations the percentiles are taken over; None from older servers."""
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> QueueActivity:
@@ -81,9 +92,20 @@ class QueueActivity:
             retention_seconds=d.get("retention_seconds"),
             p50_duration_ms=d.get("p50_duration_ms"),
             p95_duration_ms=d.get("p95_duration_ms"),
+            duration_samples=d.get("duration_samples"),
         )
 
 
+@dataclass(frozen=True, slots=True)
+class CronSpan:
+    """`count` runs, `every_seconds` apart, the first at `start`."""
+
+    start: datetime
+    every_seconds: int
+    """0 when `count` is 1."""
+    count: int
+
+
 @dataclass(frozen=True, slots=True)
 class CronPreview:
     """When a cron timetable would run, from now."""
@@ -91,11 +113,23 @@ class CronPreview:
     next_runs: list[datetime]
     interval_seconds: int
     """The shortest time between two of its runs."""
+    spans: list[CronSpan] | None = None
+    """With a window: every run in it, earliest first, as runs at a fixed interval."""
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> CronPreview:
+        spans = d.get("spans")
         return cls(
-            next_runs=[_time(t) for t in d["next_runs"]], interval_seconds=d["interval_seconds"]
+            next_runs=[_time(t) for t in d["next_runs"]],
+            interval_seconds=d["interval_seconds"],
+            spans=None
+            if spans is None
+            else [
+                CronSpan(
+                    start=_time(s["start"]), every_seconds=s["every_seconds"], count=s["count"]
+                )
+                for s in spans
+            ],
         )
 
 
@@ -104,16 +138,20 @@ ActivityRange = Literal["1h", "24h", "7d"]
 
 @dataclass(frozen=True, slots=True)
 class ActivityBucket:
-    """Tasks created from `start` until the next bucket, by queue."""
+    """Tasks created from `start` until the next bucket, by queue, and tasks that finished in it."""
 
     start: datetime
     counts: dict[str, int]
-    """Queues with none are left out."""
+    """Tasks created, by queue. Queues with none are left out."""
+    succeeded: int = 0
+    """Tasks of every queue that finished in the bucket and succeeded."""
+    failed: int = 0
+    """Tasks of every queue that finished in the bucket and failed."""
 
 
 @dataclass(frozen=True, slots=True)
 class Activity:
-    """The workspace's tasks created over time: every bucket of the range, oldest first."""
+    """Tasks created and finished over time: every bucket of the range, oldest first."""
 
     range: ActivityRange
     bucket_seconds: int
@@ -127,7 +165,12 @@ class Activity:
             range=d["range"],
             bucket_seconds=d["bucket_seconds"],
             buckets=[
-                ActivityBucket(start=_time(b["start"]), counts=dict(b["counts"]))
+                ActivityBucket(
+                    start=_time(b["start"]),
+                    counts=dict(b["counts"]),
+                    succeeded=b.get("succeeded", 0),
+                    failed=b.get("failed", 0),
+                )
                 for b in d["buckets"]
             ],
             retention_seconds=d.get("retention_seconds"),
@@ -240,10 +283,13 @@ class Task:
     """Called each time the task finishes."""
     callback_task_id: str | None = None
     """The callback task enqueued when the task last finished, if it has `on_complete`."""
+    workflow_step: TaskWorkflowStep | None = None
+    """The workflow step the task delivers, if it is one."""
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> Task:
         cb = d.get("on_complete")
+        step = d.get("workflow_step")
         return cls(
             id=d["id"],
             queue=d["queue"],
@@ -271,9 +317,26 @@ class Task:
             last_duration_ms=d.get("last_duration_ms"),
             on_complete=None if cb is None else Callback._from_json(cb),
             callback_task_id=d.get("callback_task_id"),
+            workflow_step=None if step is None else TaskWorkflowStep._from_json(step),
         )
 
 
+@dataclass(frozen=True, slots=True)
+class TaskWorkflowStep:
+    """The workflow step a task delivers. Its task can't be retried alone: retry its run."""
+
+    id: str
+    """The step, `wfs_…`."""
+    name: str
+    workflow: str
+    run_id: str
+    """The workflow run, `wfr_…`."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> TaskWorkflowStep:
+        return cls(id=d["id"], name=d["name"], workflow=d["workflow"], run_id=d["run_id"])
+
+
 @dataclass(frozen=True, slots=True)
 class Delivery:
     """The HTTP request that delivers a task or run, freshly signed."""
@@ -405,6 +468,8 @@ class Schedule:
     """While paused, the five slots it would run at if resumed now; None while it runs."""
     on_complete: Callback | None = None
     """Called each time a run finishes, except skipped runs."""
+    workflow: str | None = None
+    """The workflow each run starts, for a schedule that starts one instead of calling `url`."""
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> Schedule:
@@ -441,6 +506,7 @@ class Schedule:
                 else [_time(t) for t in resumed]
             ),
             on_complete=None if cb is None else Callback._from_json(cb),
+            workflow=d.get("workflow"),
         )
 
 
@@ -477,6 +543,8 @@ class Run:
     """How long the latest finished attempt took."""
     callback_task_id: str | None = None
     """The callback task enqueued when the run last finished, if its schedule has `on_complete`."""
+    workflow_run_id: str | None = None
+    """The workflow run it started, for a schedule that starts a workflow."""
 
     @classmethod
     def _from_json(cls, d: dict[str, Any]) -> Run:
@@ -505,6 +573,7 @@ class Run:
             started_at=_opt_time(d.get("started_at")),
             last_duration_ms=d.get("last_duration_ms"),
             callback_task_id=d.get("callback_task_id"),
+            workflow_run_id=d.get("workflow_run_id"),
         )
 
 
@@ -634,3 +703,249 @@ class GroupPage:
             groups=[Group._from_json(g) for g in d["groups"]],
             next_cursor=d.get("next_cursor"),
         )
+
+
+WORKFLOW_QUEUE = "_workflows"
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowStepConfig:
+    """A step of a workflow's definition: the request its task sends."""
+
+    name: str
+    url: str
+    depends_on: list[str]
+    method: HTTPMethod
+    queue: str
+    headers: dict[str, str]
+    payload: Any
+    """The JSON body, decoded. None for a raw `body`, or none: then the run's context is sent."""
+    body: str | None
+    timeout_seconds: float | None
+    """Overrides the queue's timeout."""
+    max_attempts: int | None
+    """Overrides the queue's max attempts."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowStepConfig:
+        return cls(
+            name=d["name"],
+            url=d["url"],
+            depends_on=d.get("depends_on") or [],
+            method=d.get("method") or "POST",
+            queue=d.get("queue") or WORKFLOW_QUEUE,
+            headers=d.get("headers") or {},
+            payload=d.get("payload"),
+            body=d.get("body"),
+            timeout_seconds=d.get("timeout_seconds"),
+            max_attempts=d.get("max_attempts"),
+        )
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowRunSummary:
+    """A workflow run without its steps, as lists show it."""
+
+    id: str
+    workflow: str
+    version: int
+    """The definition's version the run started with."""
+    status: WorkflowRunStatus
+    error: str | None
+    """Why the run failed or was cancelled."""
+    idempotency_key: str | None
+    schedule_run_id: str | None
+    """The schedule run that started it, if any."""
+    created_at: datetime
+    deadline_at: datetime | None
+    """When a run still going fails (`max_duration_seconds`)."""
+    finished_at: datetime | None
+    duration_ms: int | None
+    """From start to finish, or to now while running."""
+    attempt: int = 1
+    """1, plus one for each time the run was retried."""
+    retried_at: datetime | None = None
+    """When the run was last retried."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowRunSummary:
+        return cls(**_run_summary_fields(d))
+
+
+def _run_summary_fields(d: dict[str, Any]) -> dict[str, Any]:
+    return dict(
+        id=d["id"],
+        workflow=d["workflow"],
+        version=d.get("version", 0),
+        status=d["status"],
+        error=d.get("error"),
+        idempotency_key=d.get("idempotency_key"),
+        schedule_run_id=d.get("schedule_run_id"),
+        created_at=_time(d["created_at"]),
+        deadline_at=_opt_time(d.get("deadline_at")),
+        finished_at=_opt_time(d.get("finished_at")),
+        duration_ms=d.get("duration_ms"),
+        attempt=d.get("attempt", 1),
+        retried_at=_opt_time(d.get("retried_at")),
+    )
+
+
+@dataclass(frozen=True, slots=True)
+class Workflow:
+    """A DAG of HTTP steps, each delivered as a task once its dependencies have succeeded."""
+
+    name: str
+    steps: list[WorkflowStepConfig]
+    max_duration_seconds: float
+    version: int
+    """Counts changes to the definition, from 1."""
+    last_run: WorkflowRunSummary | None
+    created_at: datetime
+    updated_at: datetime
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> Workflow:
+        last = d.get("last_run")
+        return cls(
+            name=d["name"],
+            steps=[WorkflowStepConfig._from_json(s) for s in d["steps"]],
+            max_duration_seconds=d["max_duration_seconds"],
+            version=d["version"],
+            last_run=None if last is None else WorkflowRunSummary._from_json(last),
+            created_at=_time(d["created_at"]),
+            updated_at=_time(d["updated_at"]),
+        )
+
+
+@dataclass(frozen=True, slots=True)
+class WaitFor:
+    """The resource a step handed its completion to with `Workly-Wait-For`."""
+
+    type: Literal["group"]
+    id: str
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WaitFor:
+        return cls(type=d["type"], id=d["id"])
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowStepTask:
+    """A step's task, which records its delivery attempts (`tasks.attempts`)."""
+
+    id: str
+    queue: str
+    state: TaskState
+    attempt_count: int
+    last_status: int | None
+    last_outcome: AttemptOutcome | None
+    last_error: str | None
+    run_at: datetime | None
+    """When a pending task is next delivered."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowStepTask:
+        return cls(
+            id=d["id"],
+            queue=d["queue"],
+            state=d["state"],
+            attempt_count=d["attempt_count"],
+            last_status=d.get("last_status"),
+            last_outcome=d.get("last_outcome"),
+            last_error=d.get("last_error"),
+            run_at=_opt_time(d.get("run_at")),
+        )
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowStep:
+    """A step of a run: where it is, and why."""
+
+    id: str
+    """Stable across retries of its request: use it as an idempotency key."""
+    name: str
+    status: StepStatus
+    depends_on: list[str]
+    blocked_by: list[str]
+    """Dependencies that haven't succeeded yet."""
+    task_id: str | None
+    task: WorkflowStepTask | None
+    wait_for: WaitFor | None
+    error: str | None
+    created_at: datetime | None
+    ready_at: datetime | None
+    """When its task was enqueued."""
+    http_done_at: datetime | None
+    """When its request succeeded."""
+    finished_at: datetime | None
+    blocked_ms: int | None
+    waiting_ms: int | None
+    duration_ms: int | None
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowStep:
+        task, wait = d.get("task"), d.get("wait_for")
+        return cls(
+            id=d["id"],
+            name=d["name"],
+            status=d["status"],
+            depends_on=d.get("depends_on") or [],
+            blocked_by=d.get("blocked_by") or [],
+            task_id=d.get("task_id"),
+            task=None if task is None else WorkflowStepTask._from_json(task),
+            wait_for=None if wait is None else WaitFor._from_json(wait),
+            error=d.get("error"),
+            created_at=_opt_time(d.get("created_at")),
+            ready_at=_opt_time(d.get("ready_at")),
+            http_done_at=_opt_time(d.get("http_done_at")),
+            finished_at=_opt_time(d.get("finished_at")),
+            blocked_ms=d.get("blocked_ms"),
+            waiting_ms=d.get("waiting_ms"),
+            duration_ms=d.get("duration_ms"),
+        )
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowRun:
+    """One execution of a workflow, under the definition it started with."""
+
+    id: str
+    workflow: str
+    version: int
+    status: WorkflowRunStatus
+    error: str | None
+    idempotency_key: str | None
+    schedule_run_id: str | None
+    created_at: datetime
+    deadline_at: datetime | None
+    finished_at: datetime | None
+    duration_ms: int | None
+    input: Any
+    steps: list[WorkflowStep]
+    """In definition order."""
+    attempt: int = 1
+    """1, plus one for each time the run was retried."""
+    retried_at: datetime | None = None
+    """When the run was last retried."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowRun:
+        return cls(
+            **_run_summary_fields(d),
+            input=d.get("input"),
+            steps=[WorkflowStep._from_json(s) for s in _list(d.get("steps"))],
+        )
+
+
+@dataclass(frozen=True, slots=True)
+class WorkflowRunPage:
+    workflow_runs: list[WorkflowRunSummary]
+    next_cursor: str | None
+    """Pass as `cursor` to get the next page. None on the last page."""
+
+    @classmethod
+    def _from_json(cls, d: dict[str, Any]) -> WorkflowRunPage:
+        return cls(
+            workflow_runs=[WorkflowRunSummary._from_json(r) for r in d["workflow_runs"]],
+            next_cursor=d.get("next_cursor"),
+        )

{workly-0.3.0 → workly-0.4.3}/src/workly/_requests.py

@@ -7,7 +7,7 @@ send it and decode the response, so the two stay in step.
 from __future__ import annotations
 
 import os
-from collections.abc import Mapping
+from collections.abc import Iterable, Iterator, Mapping
 from dataclasses import dataclass, field
 from datetime import datetime
 from importlib.metadata import PackageNotFoundError, version
@@ -23,6 +23,7 @@ from ._models import (
     SortOrder,
     TaskSort,
     TaskState,
+    WorkflowRunStatus,
 )
 
 DEFAULT_URL = "http://localhost:7337"
@@ -88,6 +89,27 @@ GroupCallbackSpec = CallbackDict
 """Deprecated: use `Callback` or `CallbackDict`."""
 
 
+class WorkflowStepSpec(TypedDict, total=False):
+    """One step of a workflow for `workflows.put`.
+
+    An HTTP request, sent once the steps in `depends_on` have succeeded.
+    Without `payload` or `body`, the step is sent the run's context as JSON.
+    """
+
+    name: Required[str]
+    url: Required[str]
+    depends_on: list[str]
+    method: HTTPMethod
+    headers: Mapping[str, str]
+    payload: Any
+    body: str
+    content_type: str
+    queue: str
+    """The queue its task goes to; `_workflows` by default."""
+    timeout_seconds: float
+    max_attempts: int
+
+
 def callback_body(cb: Callback | CallbackDict) -> dict[str, Any]:
     if isinstance(cb, Callback):
         url, headers = cb.url, cb.headers
@@ -347,8 +369,16 @@ def resume_schedule(name: str) -> Request:
     return Request("POST", f"/v1/schedules/{_q(name)}/resume", idempotent=True)
 
 
-def preview_cron(cron: str, timezone: str | None) -> Request:
-    params = {"cron": cron} if timezone is None else {"cron": cron, "timezone": timezone}
+def preview_cron(
+    cron: str, timezone: str | None, from_: datetime | None, until: datetime | None
+) -> Request:
+    params = {"cron": cron}
+    if timezone is not None:
+        params["timezone"] = timezone
+    if from_ is not None:
+        params["from"] = _timestamp(from_, "from_")
+    if until is not None:
+        params["until"] = _timestamp(until, "until")
     return Request("GET", "/v1/cron/preview", params=params, idempotent=True)
 
 
@@ -409,3 +439,116 @@ def delete_alert_channel(channel_id: str) -> Request:
 def test_alert_channel(channel_id: str) -> Request:
     # Not idempotent: a resend sends a second test message.
     return Request("POST", f"/v1/alert-channels/{_q(channel_id)}/test")
+
+
+# Workflows
+
+
+def list_workflows() -> Request:
+    return Request("GET", "/v1/workflows", idempotent=True)
+
+
+def get_workflow(name: str) -> Request:
+    return Request("GET", f"/v1/workflows/{_q(name)}", idempotent=True)
+
+
+def put_workflow(
+    name: str, steps: list[WorkflowStepSpec], max_duration_seconds: float | None
+) -> Request:
+    body: dict[str, Any] = {"steps": [_step_body(s) for s in steps]}
+    if max_duration_seconds is not None:
+        body["max_duration_seconds"] = max_duration_seconds
+    return Request("PUT", f"/v1/workflows/{_q(name)}", json=body, idempotent=True)
+
+
+def _step_body(step: WorkflowStepSpec) -> dict[str, Any]:
+    body: dict[str, Any] = {"name": step["name"], "url": step["url"]}
+    if "depends_on" in step:
+        body["depends_on"] = list(step["depends_on"])
+    if "headers" in step:
+        body["headers"] = dict(step["headers"])
+    for key in (
+        "method",
+        "payload",
+        "body",
+        "content_type",
+        "queue",
+        "timeout_seconds",
+        "max_attempts",
+    ):
+        if step.get(key) is not None:
+            body[key] = step.get(key)
+    if "payload" in body and "body" in body:
+        raise ValueError(f"step {step['name']!r}: pass payload or body, not both")
+    return body
+
+
+def delete_workflow(name: str) -> Request:
+    return Request("DELETE", f"/v1/workflows/{_q(name)}", idempotent=True)
+
+
+def start_workflow_run(name: str, input: Any, idempotency_key: str | None) -> Request:
+    body: dict[str, Any] = {}
+    if input is not None:
+        body["input"] = input
+    if idempotency_key is not None:
+        body["idempotency_key"] = idempotency_key
+    # Only safe to resend with a key: without one it could start a second run.
+    return Request(
+        "POST", f"/v1/workflows/{_q(name)}/runs", json=body, idempotent=idempotency_key is not None
+    )
+
+
+def list_workflow_runs(
+    *,
+    workflow: str | None,
+    status: WorkflowRunStatus | None,
+    cursor: str | None,
+    limit: int | None,
+) -> Request:
+    params: dict[str, str] = {}
+    if workflow is not None:
+        params["workflow"] = workflow
+    if status is not None:
+        params["status"] = status
+    if cursor is not None:
+        params["cursor"] = cursor
+    if limit is not None:
+        params["limit"] = str(limit)
+    return Request("GET", "/v1/workflow-runs", params=params, idempotent=True)
+
+
+def get_workflow_run(run_id: str) -> Request:
+    return Request("GET", f"/v1/workflow-runs/{_q(run_id)}", idempotent=True)
+
+
+def cancel_workflow_run(run_id: str) -> Request:
+    return Request("POST", f"/v1/workflow-runs/{_q(run_id)}/cancel", idempotent=True)
+
+
+def retry_workflow_run(run_id: str) -> Request:
+    # Not idempotent: once it is running again, retrying is invalid_state.
+    return Request("POST", f"/v1/workflow-runs/{_q(run_id)}/retry")
+
+
+# The most tasks one batch enqueue takes.
+MAX_BATCH = 100
+
+
+def group_batches(group_id: str, specs: Iterable[TaskSpec]) -> Iterator[list[TaskSpec]]:
+    """Split a group's tasks into batches, each task joining the group.
+
+    A task without an idempotency key gets one from the group and its
+    position, so enqueueing the same tasks again (a retry) adds none twice.
+    """
+    batch: list[TaskSpec] = []
+    for i, spec in enumerate(specs):
+        task = TaskSpec(**spec)
+        task["group_id"] = group_id
+        task.setdefault("idempotency_key", f"{group_id}:{i}")
+        batch.append(task)
+        if len(batch) == MAX_BATCH:
+            yield batch
+            batch = []
+    if batch:
+        yield batch