aflib 0.0.6__tar.gz → 0.0.8__tar.gz

{aflib-0.0.6 → aflib-0.0.8}/.gitignore

@@ -5,3 +5,4 @@ build/
 dist/
 .venv/
 .pytest_cache/
+.env

{aflib-0.0.6 → aflib-0.0.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: aflib
-Version: 0.0.6
+Version: 0.0.8
 Summary: Helper library for business logic inside Airflows plpython function bodies
 License: MIT
 License-File: LICENSE
@@ -27,16 +27,57 @@ so low.
 Airflows resolves Python packages from public PyPI by name and version only; there is no
 local-import, git or private-index route. Declare it in
 `models/pythonPackages/pythonPackages.airflows` by pinning **name and version in
-separate fields** — never as a single `aflib==0.0.6` string (that form produced
+separate fields** — never as a single `aflib==0.0.8` string (that form produced
 malformed `name==ver==ver` entries in a real instance's package list):
 
 ```
 name:    aflib
-version: 0.0.6
+version: 0.0.8
 ```
 
 Then import it from a function body.
 
+## What 0.0.8 fixes
+
+`Batch.run` accepts a predicate whose column name contains the letters `_t`.
+`id_tuition IS NOT NULL`, `start_time < now()` and `is_test = false` are
+predicates. 0.0.7 refused them, because the guard searched for `_t` as a
+substring. `_t.col` and `"_t".col` are still refused: that alias exists only on
+the outer `UPDATE`.
+
+## What 0.0.7 gives you
+
+Everything in 0.0.6, plus three capabilities that ran on a real instance on
+2026-10-01 before they were extracted:
+
+- **`Batch(plpy).run(...)`** — one committed chunk. The pending predicate is
+  placed in the outer `WHERE` and in the subquery, and the result carries
+  `done`, `remaining` and `seconds`.
+- **`workflow.start` / `status` / `stuck`** — insert a `PENDING` workflow
+  instance and its variables, read the newest instance for a business key, and
+  list instances that are still `PENDING` or `IN_PROGRESS` after an interval.
+- **`Outbox(plpy, outbox=..., delivery=..., subscription=...).emit` / `pump`** —
+  one event plus a pending delivery per active subscriber, then a chunk of
+  deliveries. Each subscriber runs inside `plpy.subtransaction()`. Failures
+  retry with `attempts`, `next_attempt_at` and `last_error`, and dead-letter
+  after 3 attempts.
+
+```python
+from aflib import workflow
+from aflib.events import Outbox
+
+workflow.start(plpy, "WFP_cadena", "case-58", {
+    "SomeSchema.some_function.p_id": ("INTEGER", 42),
+})
+
+box = Outbox(plpy, outbox="app.outbox", delivery="app.delivery", subscription="app.sub")
+box.emit("order.placed", {"id": 42}, 42)
+box.pump(deliver, limit=50)
+```
+
+`deliver(subscriber, outbox_id, attempt)` does the work and raises to fail the
+attempt. Table names are identifiers you wrote, not request data.
+
 ## What 0.0.6 gives you
 
 0.0.1–0.0.3 (read/result, errors/logging, `transaction.atomic`) plus what
@@ -58,10 +99,10 @@ the 2026-09 platform probes showed, plus two verified fixes:
   resolve.
 - **`identity.current_user(plpy)`** — `SELECT current_user`. Not a User id.
 
-Pin **name and version in separate fields** (`aflib` / `0.0.6`), not
-`aflib==0.0.6` as one string.
+Pin **name and version in separate fields** (`aflib` / `0.0.8`), not
+`aflib==0.0.8` as one string.
 
-## Core API (0.0.1–0.0.3, still in 0.0.6)
+## Core API (0.0.1–0.0.3, still in 0.0.8)
 
 ```python
 from aflib import Db, errors, fail, logging, ok, params

{aflib-0.0.6 → aflib-0.0.8}/README.md

@@ -18,16 +18,57 @@ so low.
 Airflows resolves Python packages from public PyPI by name and version only; there is no
 local-import, git or private-index route. Declare it in
 `models/pythonPackages/pythonPackages.airflows` by pinning **name and version in
-separate fields** — never as a single `aflib==0.0.6` string (that form produced
+separate fields** — never as a single `aflib==0.0.8` string (that form produced
 malformed `name==ver==ver` entries in a real instance's package list):
 
 ```
 name:    aflib
-version: 0.0.6
+version: 0.0.8
 ```
 
 Then import it from a function body.
 
+## What 0.0.8 fixes
+
+`Batch.run` accepts a predicate whose column name contains the letters `_t`.
+`id_tuition IS NOT NULL`, `start_time < now()` and `is_test = false` are
+predicates. 0.0.7 refused them, because the guard searched for `_t` as a
+substring. `_t.col` and `"_t".col` are still refused: that alias exists only on
+the outer `UPDATE`.
+
+## What 0.0.7 gives you
+
+Everything in 0.0.6, plus three capabilities that ran on a real instance on
+2026-10-01 before they were extracted:
+
+- **`Batch(plpy).run(...)`** — one committed chunk. The pending predicate is
+  placed in the outer `WHERE` and in the subquery, and the result carries
+  `done`, `remaining` and `seconds`.
+- **`workflow.start` / `status` / `stuck`** — insert a `PENDING` workflow
+  instance and its variables, read the newest instance for a business key, and
+  list instances that are still `PENDING` or `IN_PROGRESS` after an interval.
+- **`Outbox(plpy, outbox=..., delivery=..., subscription=...).emit` / `pump`** —
+  one event plus a pending delivery per active subscriber, then a chunk of
+  deliveries. Each subscriber runs inside `plpy.subtransaction()`. Failures
+  retry with `attempts`, `next_attempt_at` and `last_error`, and dead-letter
+  after 3 attempts.
+
+```python
+from aflib import workflow
+from aflib.events import Outbox
+
+workflow.start(plpy, "WFP_cadena", "case-58", {
+    "SomeSchema.some_function.p_id": ("INTEGER", 42),
+})
+
+box = Outbox(plpy, outbox="app.outbox", delivery="app.delivery", subscription="app.sub")
+box.emit("order.placed", {"id": 42}, 42)
+box.pump(deliver, limit=50)
+```
+
+`deliver(subscriber, outbox_id, attempt)` does the work and raises to fail the
+attempt. Table names are identifiers you wrote, not request data.
+
 ## What 0.0.6 gives you
 
 0.0.1–0.0.3 (read/result, errors/logging, `transaction.atomic`) plus what
@@ -49,10 +90,10 @@ the 2026-09 platform probes showed, plus two verified fixes:
   resolve.
 - **`identity.current_user(plpy)`** — `SELECT current_user`. Not a User id.
 
-Pin **name and version in separate fields** (`aflib` / `0.0.6`), not
-`aflib==0.0.6` as one string.
+Pin **name and version in separate fields** (`aflib` / `0.0.8`), not
+`aflib==0.0.8` as one string.
 
-## Core API (0.0.1–0.0.3, still in 0.0.6)
+## Core API (0.0.1–0.0.3, still in 0.0.8)
 
 ```python
 from aflib import Db, errors, fail, logging, ok, params

{aflib-0.0.6 → aflib-0.0.8}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "aflib"
-version = "0.0.6"
+version = "0.0.8"
 description = "Helper library for business logic inside Airflows plpython function bodies"
 readme = "README.md"
 # The platform runs Python 3.11.2 (observed, not assumed — see docs/platform-constraints.md).

{aflib-0.0.6 → aflib-0.0.8}/src/aflib/__init__.py

@@ -33,9 +33,11 @@ Python exception.
 
 from aflib import (
     authz,
+    batch,
     codec,
     db,
     errors,
+    events,
     functions,
     identity,
     logging,
@@ -43,23 +45,30 @@ from aflib import (
     result,
     transaction,
     trigger,
+    workflow,
 )
+from aflib.batch import Batch
 from aflib.db import Db
+from aflib.events import Outbox
 from aflib.params import BadParameter
 from aflib.result import fail, ok
 
 # Kept in step with `pyproject.toml` by hand. A deployed body returns this so a run can
 # prove which aflib it actually imported — a stale package otherwise looks exactly like a
 # working one.
-__version__ = "0.0.6"
+__version__ = "0.0.8"
 
 __all__ = [
     "BadParameter",
+    "Batch",
     "Db",
+    "Outbox",
     "authz",
+    "batch",
     "codec",
     "db",
     "errors",
+    "events",
     "fail",
     "functions",
     "identity",
@@ -69,5 +78,6 @@ __all__ = [
     "result",
     "transaction",
     "trigger",
+    "workflow",
     "__version__",
 ]

aflib-0.0.8/src/aflib/batch.py

@@ -0,0 +1,156 @@
+"""Work through a backlog one committed chunk at a time.
+
+    from aflib.batch import Batch
+
+    batch = Batch(plpy)
+
+    progress = batch.run(
+        table="\\"Economic\\".\\"Receipt\\"",
+        set_clause='id_status = $1',
+        pending='id_status = $2 AND (period_from IS NULL OR period_from <= CURRENT_DATE + 5)',
+        args=[pending_id, planned_id],
+        types=["integer", "integer"],
+        limit=1000,
+    )
+
+    return ok({"done": progress.done, "remaining": progress.remaining})
+
+One call is one chunk. The caller decides what drives the next one — a `pg_cron` job, the
+next step of a workflow, a human pressing a button again — and each call is its own
+transaction, so each chunk is **kept** whether or not the next one happens.
+
+**Why a function and not a loop.** A plpgsql or plpython function cannot commit: it runs
+inside a transaction it does not own, and `plpy.commit()` fails with *invalid transaction
+termination*. There are no `PROCEDURE`s on the platform either (401 functions, zero
+procedures, measured 2026-10-01), so the usual `LOOP … COMMIT` shape has nowhere to live.
+A chunk per invocation is not a stylistic choice; it is the only way to commit progress.
+
+**The one rule this exists to enforce: the pending predicate goes in two places.** It
+selects the ids *and* it guards the update. Written the obvious way, with the predicate
+only in the subquery, this is wrong:
+
+    UPDATE t SET … WHERE id IN (SELECT id FROM t WHERE pending LIMIT 1000)   -- BROKEN
+
+Under `READ COMMITTED` the subquery runs against the statement's snapshot, but when the
+update reaches a row another transaction has since changed, Postgres re-reads the latest
+committed version and applies the update to **that**. A row that was pending when selected
+and is now cancelled gets updated anyway. On 2026-10-01 that aborted a production run on
+its second chunk: a receipt was cancelled by a user mid-batch and the trigger refused the
+`Cancelled → Pending` transition. Repeating the predicate in the outer `WHERE` makes
+Postgres re-evaluate it against the new version and skip the row.
+
+`run()` takes the predicate once and places it twice, so getting this wrong requires
+bypassing the helper.
+
+**`table`, `set_clause` and `pending` are SQL, and are interpolated.** They are code
+written by the caller, not values — values go in `args` and are bound. Never build them
+from anything a request supplied.
+
+**Chunk size is measured, not chosen.** Cost per row depends on the triggers the write
+fires, and the figure is not guessable: the same 1.000-row chunk took 19 s on
+`Economic.Receipt`, whose triggers recalculate and audit, and 0.07 ms per row on a bare
+table. Run one chunk, read `progress.seconds`, and size the rest so a chunk finishes well
+inside the environment's `statement_timeout` — 180 s on PRO, no limit on PRE, so **PRE
+cannot tell you whether a chunk fits**.
+"""
+
+import re
+import time
+
+from aflib.db import Db
+
+
+class Progress(object):
+    """What one chunk did, and whether there is more.
+
+    `remaining` is counted **after** the update, in the same transaction, so it already
+    reflects this chunk. A caller that loops until `complete` therefore terminates without
+    needing to subtract anything itself.
+    """
+
+    __slots__ = ("done", "remaining", "seconds")
+
+    def __init__(self, done, remaining, seconds):
+        self.done = done
+        self.remaining = remaining
+        self.seconds = seconds
+
+    @property
+    def complete(self):
+        """True when nothing is left to do."""
+        return self.remaining == 0
+
+    def as_dict(self):
+        return {"done": self.done, "remaining": self.remaining,
+                "seconds": round(self.seconds, 3), "complete": self.complete}
+
+    def __repr__(self):
+        return "Progress(done=%r, remaining=%r, complete=%r)" % (
+            self.done, self.remaining, self.complete)
+
+
+def _alias_as_qualifier(alias, pending):
+    """True when `pending` uses `alias` as a qualifier, not as letters inside a name.
+
+    Searching for the substring `_t` refuses `id_tuition IS NOT NULL`,
+    `start_time < now()` and `is_test = false`. Observed on PRE 2026-10-02,
+    acceptance probe `aflib-007-accept-r1`, case `batch`. The alias is a problem
+    only as `_t.col` or `"_t".col`, with optional space before the dot.
+    """
+    pattern = r'(?<![\w"])"?{0}"?\s*\.'.format(re.escape(alias))
+    return re.search(pattern, pending) is not None
+
+
+class Batch(object):
+    """One committed chunk per call. `plpy` is injected, as everywhere in aflib."""
+
+    def __init__(self, plpy):
+        self._db = Db(plpy)
+
+    ALIAS = "_t"
+
+    def run(self, table, set_clause, pending, args=None, types=None,
+            limit=1000, key="id", order=None):
+        """Update at most `limit` rows matching `pending`, and report what is left.
+
+        `pending` must be **unqualified** — write `id_status = $1`, not
+        `"Receipt".id_status = $1`. That is not a style rule: the same text is placed in
+        the outer `WHERE`, where Postgres resolves it against the update target, and
+        inside the subquery, where it resolves against the subquery's own `FROM`. Both
+        resolutions are what we want, and either qualification breaks one of them. A
+        predicate that qualifies a column with the alias (`_t.col` or `"_t".col`) is
+        rejected rather than silently misplaced. A column whose name merely contains
+        those letters — `id_tuition`, `start_time`, `is_test` — is not a qualification.
+
+        `order` defaults to `key`, which makes the work deterministic and the chunks
+        disjoint. Ordering by something non-unique would let two concurrent callers pick
+        overlapping sets; the row-level guard still keeps them correct, but they would
+        waste each other's work.
+        """
+        if _alias_as_qualifier(self.ALIAS, pending):
+            raise ValueError(
+                "`pending` must be unqualified: it is placed both in the outer WHERE and "
+                "inside the subquery, and %r resolves in only one of them." % self.ALIAS)
+
+        args = list(args or [])
+        types = list(types or [])
+        order = order or key
+
+        started = time.time()
+        updated = self._db.query(
+            "UPDATE {table} AS {alias} SET {set_clause} "
+            "WHERE {pending} "
+            "  AND {alias}.{key} IN (SELECT {key} FROM {table} WHERE {pending} "
+            "                        ORDER BY {order} LIMIT {limit}) "
+            "RETURNING {alias}.{key}".format(
+                table=table, alias=self.ALIAS, set_clause=set_clause,
+                pending=pending, key=key, order=order, limit=int(limit)),
+            args, types).rows()
+        elapsed = time.time() - started
+
+        remaining = self._db.query(
+            "SELECT count(*)::int AS n FROM {table} WHERE {pending}".format(
+                table=table, pending=pending),
+            args, types).scalar("n")
+
+        return Progress(len(updated), remaining, elapsed)

aflib-0.0.8/src/aflib/events.py

@@ -0,0 +1,176 @@
+"""A minimal outbox: emit one event, deliver it with retries.
+
+    from aflib.events import Outbox
+
+    box = Outbox(
+        plpy,
+        outbox="public.aflib_outbox",
+        delivery="public.aflib_delivery",
+        subscription="public.aflib_sub",
+    )
+    box.emit("matricula.finalizada", {"id_tuition": 321633}, 321633)
+
+    def deliver(subscriber, outbox_id, attempt):
+        functions.call(plpy, subscriber, [outbox_id, attempt], ["bigint", "integer"])
+
+    box.pump(deliver, limit=50)
+
+The columns are the ones the probe created and ran on PRE, 2026-10-01. This module
+does not create the tables.
+
+- `outbox`: `id` bigint, `evento` text, `payload` jsonb, `aggregate_id` bigint
+- `delivery`: `id` bigint, `id_outbox` bigint, `suscriptor` text,
+  `estado` text (`pending`, `done`, `dead`), `attempts` int,
+  `next_attempt_at` timestamptz, `last_error` text
+- `subscription`: `evento` text, `suscriptor` text, `activo` boolean
+
+`emit` inserts the event and one `pending` delivery per active subscriber, in the
+caller's transaction. `pump` is one chunk: the caller invokes it again (`pg_cron`,
+the next request) and each invocation is its own transaction.
+
+A subscriber that raises must not take the others with it. The effect runs inside
+`plpy.subtransaction()`. The attempt counter is written **after** that block, so a
+failure rolls the effect back and still records the attempt. Writing the counter
+inside the savepoint would roll the count back too, and the same delivery would
+retry forever. A SQL `SAVEPOINT` is not the way to get that block:
+`plpy.execute("savepoint s")` fails with `SPI_ERROR_TRANSACTION`.
+
+`deliver` is called as `deliver(subscriber, outbox_id, attempt)`. The first attempt
+is 1. A raise is a failed attempt. After `max_attempts` (3 on the run) the row
+becomes `dead`. Before that, `next_attempt_at` moves forward by `2**attempt`
+seconds, which is the backoff the probe wrote. `last_error` keeps the first 200
+characters of the exception text.
+
+Table names are caller-written SQL identifiers, quoted after a check. Event names,
+payloads, ids and error text are bound.
+"""
+
+import re
+
+from aflib.codec import dumps
+from aflib.db import Db
+
+
+_IDENT = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
+_ERROR_LIMIT = 200
+
+
+def _quote_table(name):
+    """`"Schema"."table"` or `"table"`. Anything else is refused.
+
+    The name is interpolated, because an identifier cannot be a bound parameter.
+    Checking it first is what keeps a payload or an error string from being able
+    to ride along in that position.
+    """
+    parts = name.split(".")
+    if not 1 <= len(parts) <= 2 or not all(_IDENT.match(part) for part in parts):
+        raise ValueError(
+            "table name must be an identifier or schema.identifier, got {!r}".format(name)
+        )
+    return ".".join('"{}"'.format(part) for part in parts)
+
+
+class Outbox(object):
+    """`emit` and `pump` against one trio of tables. `plpy` is injected."""
+
+    def __init__(self, plpy, outbox, delivery, subscription):
+        self._plpy = plpy
+        self._db = Db(plpy)
+        self._outbox = _quote_table(outbox)
+        self._delivery = _quote_table(delivery)
+        self._subscription = _quote_table(subscription)
+
+    def emit(self, event, payload, aggregate_id):
+        """One outbox row, plus a pending delivery for each active subscriber.
+
+        Returns `{"outbox": id, "deliveries": n}`. `payload` is encoded with
+        `codec.dumps` and bound as `jsonb`. `aggregate_id` is a bigint, the column
+        the probe declared.
+        """
+        if isinstance(aggregate_id, bool) or not isinstance(aggregate_id, int):
+            raise ValueError("aggregate_id must be an int, got {!r}".format(aggregate_id))
+
+        created = self._db.query(
+            "INSERT INTO {} (evento, payload, aggregate_id) "
+            "VALUES ($1, $2, $3) RETURNING id".format(self._outbox),
+            [event, dumps(payload), aggregate_id],
+            ["text", "jsonb", "bigint"],
+        ).one()
+        deliveries = self._db.query(
+            "INSERT INTO {} (id_outbox, suscriptor) "
+            "SELECT $1, suscriptor FROM {} "
+            "WHERE evento = $2 AND activo "
+            "RETURNING id".format(self._delivery, self._subscription),
+            [created["id"], event],
+            ["bigint", "text"],
+        ).rows()
+        return {"outbox": created["id"], "deliveries": len(deliveries)}
+
+    def pump(self, deliver, limit, max_attempts=3):
+        """Deliver up to `limit` due rows. One subscriber's failure stops at that row.
+
+        Returns `{"seen", "delivered", "retrying", "dead_letter"}`. `limit` is at
+        least 1. `max_attempts` defaults to 3, the count the probe dead-lettered on.
+        """
+        limit = int(limit)
+        max_attempts = int(max_attempts)
+        if limit < 1:
+            raise ValueError("limit must be at least 1, got {}".format(limit))
+        if max_attempts < 1:
+            raise ValueError("max_attempts must be at least 1, got {}".format(max_attempts))
+
+        due = self._db.query(
+            "SELECT id, id_outbox, suscriptor, attempts FROM {} "
+            "WHERE estado = 'pending' AND next_attempt_at <= now() "
+            "ORDER BY id LIMIT $1".format(self._delivery),
+            [limit],
+            ["integer"],
+        ).rows()
+
+        delivered = retrying = dead = 0
+        for row in due:
+            attempt = row["attempts"] + 1
+            try:
+                with self._plpy.subtransaction():
+                    deliver(row["suscriptor"], row["id_outbox"], attempt)
+                self._mark_done(row["id"], attempt)
+                delivered += 1
+            except Exception as exc:
+                message = str(exc)[:_ERROR_LIMIT]
+                if attempt >= max_attempts:
+                    self._mark_dead(row["id"], attempt, message)
+                    dead += 1
+                else:
+                    self._mark_retry(row["id"], attempt, message)
+                    retrying += 1
+        return {
+            "seen": len(due),
+            "delivered": delivered,
+            "retrying": retrying,
+            "dead_letter": dead,
+        }
+
+    def _mark_done(self, delivery_id, attempt):
+        self._db.query(
+            "UPDATE {} SET estado = 'done', attempts = $1, last_error = NULL "
+            "WHERE id = $2".format(self._delivery),
+            [attempt, delivery_id],
+            ["integer", "bigint"],
+        )
+
+    def _mark_dead(self, delivery_id, attempt, message):
+        self._db.query(
+            "UPDATE {} SET estado = 'dead', attempts = $1, last_error = $2 "
+            "WHERE id = $3".format(self._delivery),
+            [attempt, message, delivery_id],
+            ["integer", "text", "bigint"],
+        )
+
+    def _mark_retry(self, delivery_id, attempt, message):
+        self._db.query(
+            "UPDATE {} SET attempts = $1, last_error = $2, "
+            "next_attempt_at = now() + (interval '1 second' * $3) "
+            "WHERE id = $4".format(self._delivery),
+            [attempt, message, 2**attempt, delivery_id],
+            ["integer", "text", "integer", "bigint"],
+        )