@consiliency/agent-board-schema 1.45.0 → 1.48.0

package/fixtures/already-applied.json

@@ -572,6 +572,18 @@
     {
       "id": "20260925000100_agent_board_ownerless_renew_ttl_cap",
       "sha256": "da312f43cabf07c6dfd10dbe2a119fb46f49f24155e21adce4f38e291a38103c"
+    },
+    {
+      "id": "20260926000100_agent_board_ownerless_quota_census",
+      "sha256": "bbfabcb2d45cbcebd1e6e121032a1045e8406ea8763a784762a92d210e1e1043"
+    },
+    {
+      "id": "20260928000100_agent_board_ownerless_quota_renew_convergence",
+      "sha256": "9982efff25378703a744187b15081b5cd53ab38fa6ac7d2950092a471780169e"
+    },
+    {
+      "id": "20260927000100_agent_board_fleet_domain_no_downgrade",
+      "sha256": "ec5c4d89aff2d6d77d44cb3d00c1096b39010c6064c23d72eb3c5c354f8a5e74"
     }
   ]
 }

package/migrations/20260926000100_agent_board_ownerless_quota_census.sql

@@ -0,0 +1,268 @@
+-- #388 — the ownerless quota bounds NEW ACQUISITIONS but never CONVERGES an agent already over it.
+--
+-- THE PROPERTY, stated as the issue states it. `acquire_claim` enforces a per-agent ceiling of
+-- `_claimgate_ownerless_quota()` (50) concurrent active file/tool/custom claims. `renew_claim`
+-- does not, and #387 deliberately did not add it — that PR closed the TTL half only. So the quota
+-- is effective against GROWTH and not against PERSISTENCE: an agent already over it renews every
+-- one of its claims indefinitely and stays over, forever.
+--
+-- THREE WAYS AN AGENT IS ALREADY OVER, all real and none hypothetical:
+--   1. claims acquired while `ownerless_guards` was OFF — the flag defaults disabled and was
+--      flipped in production only recently (#369), so everything accumulated before the flip is
+--      grandfathered;
+--   2. claims acquired under a board that was `claim_policy = 'advisory'` and later set back to
+--      `inherit` — the acquire-time guard is skipped for an advisory board (20260922000100), but
+--      the resulting rows are counted by every LATER check;
+--   3. a quota value lowered after the fact.
+--
+-- ---------------------------------------------------------------------------
+-- WHAT THIS MIGRATION IS, AND THE THREE THINGS IT DELIBERATELY IS NOT
+-- ---------------------------------------------------------------------------
+--
+-- IT IS A READ-ONLY CENSUS. #388 step 3: "which agents are currently over quota on the live
+-- board". Nothing here writes, blocks, clamps or refuses.
+--
+-- IT IS NOT ENFORCEMENT, and that is a decision rather than an omission. Acquire refusing costs
+-- the caller nothing — it asked for a new claim and did not get it. Renew refusing would KILL A
+-- LIVE LEASE the agent is currently relying on, and if an agent is 10 over, the reasonable
+-- behaviour is not "fail 10 renews at random", it is a decision about WHICH claims lose (oldest
+-- first? least recently touched? notify the agent?). That is a product question nobody has
+-- answered. This is the instrument that informs it. Enforcement, if it is chosen, additionally
+-- needs the agent-scoped `pg_advisory_xact_lock` acquire already takes plus a two-session
+-- isolationtester case (#137's harness), because two concurrent renews must not both observe room.
+--
+-- IT IS NOT A SCRIPT, and the alternative was considered rather than skipped. A `scripts/*.mjs`
+-- check with the house 0/1/2 exit contract answers a question about whatever database it is
+-- pointed at; the population that matters is PRODUCTION, and the operator path to production is
+-- `service_role` SQL through the Management API, where a view and a function are directly
+-- readable and a Node script is not runnable at all. A CI invocation would examine an empty
+-- freshly-built database and report "0 of 0" — true, and worthless. And an exit code is a GATE:
+-- 1 means "fail the build", which presumes a policy about what an over-quota agent should cause
+-- to happen. #388 explicitly defers that policy, so there is nothing yet for a non-zero exit to
+-- mean. When the convergence policy exists, a script that gates on this view is the natural next
+-- artifact; it is not this one.
+--
+-- ---------------------------------------------------------------------------
+-- THE NUMERATOR IS ACQUIRE'S OWN COUNT PREDICATE, VERBATIM
+-- ---------------------------------------------------------------------------
+-- A census whose arithmetic differs from the enforcer's is a census about a different question,
+-- and it would be silently wrong rather than visibly wrong. The count below is copied from the
+-- quota block of the CURRENT `acquire_claim` (20260922000100 — note that migration re-creates it
+-- from a `pg_get_functiondef` dump in UPPERCASE, so a lowercase grep for the definer finds
+-- 20260914000100 and reads the wrong body):
+--
+--     where c.agent_id = v_actor_agent_id
+--       and c.resource_kind in ('file', 'tool', 'custom')
+--       and c.status = 'active'
+--       and c.released_at is null
+--       and c.lease_expires_at > now()
+--
+-- NO BOARD-POLICY FILTER, and this is the subtle one. The advisory board opt-out gates WHETHER
+-- the guard runs for the acquisition being attempted; it does not exclude the resulting rows from
+-- the count. So an agent's 60 claims count as 60 even if 20 of them were taken on advisory
+-- boards. Filtering them out here would produce a smaller, friendlier, WRONG number that does not
+-- predict what acquire does. Stated operationally, a row in this view with `acquire_blocked` true
+-- means: this agent's next ownerless acquire, on an `inherit` board, with the guards on, raises
+-- 54000.
+--
+-- `claims_on_advisory_boards` is reported as a SEPARATE column precisely because it is the thing
+-- the convergence decision needs and the thing the headline count must not absorb — refusing a
+-- claim that another board's own lock already granted is us breaking a promise, so an operator
+-- weighing "which claims lose" needs to see that share without it distorting the ceiling test.
+--
+-- OVERAGE AND ACQUIRE_BLOCKED ARE DIFFERENT NUMBERS AND BOTH ARE REPORTED. Acquire refuses at
+-- `count >= quota`, so an agent holding exactly 50 is already frozen while being over by nothing.
+--   * `overage`         = greatest(count - quota, 0) — how many claims exceed the ceiling. This is
+--                         the convergence question: what would have to be given up.
+--   * `acquire_blocked` = count >= quota — acquire's refusal predicate, verbatim. This is the
+--                         "why can my agent not take a lock" question.
+-- Collapsing them into one boolean would answer one of those two questions wrongly.
+--
+-- ---------------------------------------------------------------------------
+-- THE DENOMINATOR IS NOT OPTIONAL
+-- ---------------------------------------------------------------------------
+-- The live answer today is expected to be ZERO agents over quota. A reader that emits "0
+-- offenders" and nothing else is byte-identical whether it examined 400 agents or examined
+-- nothing at all — which is exactly the defect #367 shipped a migration to close, where
+-- `v_ops_presence_heartbeats` returned zero rows about an entirely unmonitored fleet and read as
+-- perfect health. So `ownerless_quota_census()` always emits its population alongside its finding:
+--
+--     0 over quota, of 12 agents holding ownerless claims, across 47 active agents
+--
+-- and it additionally emits `guards_enabled` and `quota`, so the output states the REGIME it
+-- measured under. "0 over quota" with `guards_enabled = false` is a materially different fact
+-- from the same number with the flag on, and a reader that reports the count without the flag
+-- invites exactly the rule-5 misreading — a true statement about a moment, still asserting itself
+-- after the moment moved.
+--
+-- ---------------------------------------------------------------------------
+-- The view. `security_invoker` matching every other view in agent_board; the grant below is what
+-- actually confines it, and the two are not redundant (see the grant comment).
+-- ---------------------------------------------------------------------------
+create or replace view agent_board.v_ownerless_claim_quota
+with (security_invoker = true)
+as
+with holders as (
+  select
+    c.agent_id                                                       as agent_id,
+    count(*)::integer                                                as active_ownerless_claims,
+    -- The advisory share, resolved per claim through its OWN thread. A claim with no thread, or a
+    -- thread with no board, resolves to 'inherit' by the helper's own contract — so absence never
+    -- reads as an opt-out here either.
+    count(*) filter (
+      where agent_board._claimgate_board_claim_policy(c.thread_id) = 'advisory'
+    )::integer                                                       as claims_on_advisory_boards,
+    count(*) filter (where c.resource_kind = 'file')::integer        as file_claims,
+    count(*) filter (where c.resource_kind = 'tool')::integer        as tool_claims,
+    count(*) filter (where c.resource_kind = 'custom')::integer      as custom_claims,
+    min(c.created_at)                                                as oldest_claim_at,
+    max(c.created_at)                                                as newest_claim_at,
+    max(c.lease_expires_at)                                          as latest_lease_expires_at
+  from agent_board.claims c
+  -- ACQUIRE'S PREDICATE, VERBATIM. Every conjunct is load-bearing and each has a near-miss row in
+  -- the pgTAP fixture: a released claim, an expired-status claim, a lapsed-lease claim, and an
+  -- owned-kind claim all sit next to the counted ones so that dropping any conjunct here fails a
+  -- named assertion rather than passing quietly.
+  where c.resource_kind in ('file', 'tool', 'custom')
+    and c.status = 'active'
+    and c.released_at is null
+    and c.lease_expires_at > now()
+  group by c.agent_id
+)
+select
+  h.agent_id,
+  -- Identity sufficient to ACT on the row without a second query: who to contact, and whether the
+  -- agent is even still in service. `status` is REPORTED, never filtered on — a suspended agent
+  -- holding 60 live leases is precisely the row an operator most needs to see, and a
+  -- `status = 'active'` filter here would hide it.
+  a.handle                                                           as agent_handle,
+  a.display_name                                                     as agent_display_name,
+  a.status                                                           as agent_status,
+  h.active_ownerless_claims,
+  agent_board._claimgate_ownerless_quota()                           as quota,
+  greatest(h.active_ownerless_claims - agent_board._claimgate_ownerless_quota(), 0)::integer
+                                                                     as overage,
+  (h.active_ownerless_claims >= agent_board._claimgate_ownerless_quota())
+                                                                     as acquire_blocked,
+  h.claims_on_advisory_boards,
+  h.file_claims,
+  h.tool_claims,
+  h.custom_claims,
+  h.oldest_claim_at,
+  h.newest_claim_at,
+  h.latest_lease_expires_at
+from holders h
+join agent_board.agents a on a.id = h.agent_id;
+
+comment on view agent_board.v_ownerless_claim_quota is
+  '#388 read-only census: one row per agent HOLDING at least one active ownerless (file/tool/'
+  'custom) claim, with its count, the quota, and the overage. The count is acquire_claim''s own '
+  'quota predicate verbatim (kind in file/tool/custom, status active, released_at null, lease '
+  'unexpired) with NO board-policy filter, because the advisory opt-out gates whether the guard '
+  'RUNS for an acquisition, not whether the resulting rows COUNT — so acquire_blocked (count >= '
+  'quota) predicts acquire''s 54000 exactly. overage (count - quota, floored at 0) is the '
+  'different question of how many claims exceed the ceiling; an agent at exactly the quota is '
+  'blocked while over by nothing. claims_on_advisory_boards is reported separately because it '
+  'informs the convergence decision without distorting the ceiling test. agent_status is reported '
+  'and never filtered: a suspended agent holding live leases is the row that matters most. '
+  'REPORT-ONLY — nothing here refuses, clamps or converges anything; see #388 for why refusing a '
+  'renew is a product decision and not a clamp. Read agent_board.ownerless_quota_census() FIRST '
+  'for the denominator. security_invoker, service_role only.';
+
+-- ---------------------------------------------------------------------------
+-- The summary. Its only job is to make an empty result impossible to misread.
+-- ---------------------------------------------------------------------------
+create or replace function agent_board.ownerless_quota_census()
+returns table(
+  guards_enabled                  boolean,
+  quota                           integer,
+  agents_active                   integer,
+  agents_holding_ownerless_claims integer,
+  agents_over_quota               integer,
+  agents_acquire_blocked          integer,
+  total_active_ownerless_claims   integer,
+  total_overage                   integer
+)
+language sql
+stable
+security invoker
+set search_path = agent_board, public
+as $$
+  select
+    -- The REGIME first: a count without the flag that governs it is a rule-5 fact with no
+    -- expiry. "0 over quota" means something different when the guards are off.
+    agent_board._claimgate_ownerless_guards_enabled()                       as guards_enabled,
+    agent_board._claimgate_ownerless_quota()                                as quota,
+    -- The widest denominator: the fleet. Non-zero even when NOBODY holds an ownerless claim, so
+    -- "0 of 0" can still be distinguished from "0 of 0, of 0" — a database this reader cannot
+    -- see into at all.
+    (select count(*)::integer from agent_board.agents where status = 'active')
+                                                                            as agents_active,
+    -- The population actually examined. THIS is the number that makes a zero finding meaningful.
+    -- It is counted from the view directly and NOT derived from the offender set — deriving it
+    -- would make it 0 exactly when the finding is 0, which is the vacuous shape this exists to
+    -- prevent.
+    (select count(*)::integer from agent_board.v_ownerless_claim_quota)
+                                                                            as agents_holding_ownerless_claims,
+    (select count(*) filter (where overage > 0)::integer
+       from agent_board.v_ownerless_claim_quota)                            as agents_over_quota,
+    (select count(*) filter (where acquire_blocked)::integer
+       from agent_board.v_ownerless_claim_quota)                            as agents_acquire_blocked,
+    (select coalesce(sum(active_ownerless_claims), 0)::integer
+       from agent_board.v_ownerless_claim_quota)                            as total_active_ownerless_claims,
+    (select coalesce(sum(overage), 0)::integer
+       from agent_board.v_ownerless_claim_quota)                            as total_overage;
+$$;
+
+comment on function agent_board.ownerless_quota_census() is
+  '#388 denominator. Returns agents_holding_ownerless_claims and agents_active ALONGSIDE '
+  'agents_over_quota, so "0 over quota" can never be rendered as "all clear" — 0 over, of 0 '
+  'holders, across 47 active agents, is a reader that examined nothing, and it reads as one '
+  '(the #367 defect shape). Also emits guards_enabled and quota so the output states the regime '
+  'it measured under. agents_holding_ownerless_claims is counted from the view, NOT derived from '
+  'the offender set: a denominator that collapses to 0 exactly when the finding is 0 proves '
+  'nothing. agents_over_quota counts overage > 0; agents_acquire_blocked counts count >= quota, '
+  'which is acquire_claim''s refusal predicate and is the strictly larger set. '
+  'security invoker + stable: a projection of v_ownerless_claim_quota, inheriting the caller''s '
+  'RLS.';
+
+-- ---------------------------------------------------------------------------
+-- SERVICE_ROLE ONLY, and this is the most important statement in the file.
+--
+-- `security_invoker` does not make an `authenticated` grant safe here; it makes it dangerous, and
+-- for the mirror-image reason #367 found. The RLS SELECT policy on `agent_board.claims` is:
+--
+--     agent_id = current_agent_id()
+--     or exists (select 1 from tasks t where t.id = task_id
+--                 and (t.created_by_agent_id = current_agent_id()
+--                      or t.assigned_agent_id = current_agent_id()))
+--
+-- so an authenticated caller can see essentially ITS OWN claims. Read by that caller this view
+-- returns one row — itself — and the census reports `agents_holding_ownerless_claims = 1`,
+-- `agents_over_quota = 0`. Not an error, not an empty result: a confident, plausible, fleet-shaped
+-- answer that is wrong, and wrong in the reassuring direction. Two agents would get two different
+-- censuses with no way to tell which was right, and the one that mattered — the agent sitting at
+-- 60 claims — would be invisible to everyone except itself.
+--
+-- #367 hit this as OVER-reporting (a LEFT JOIN turned "hidden from you" into an alarm). Here the
+-- join is an inner aggregate, so it is UNDER-reporting, which is worse for a census whose whole
+-- purpose is to find the outliers: over-reporting is noisy and gets investigated, under-reporting
+-- is silence with a denominator attached to make it look diligent.
+--
+-- An authenticated agent cannot answer "who in the fleet is over quota" at all, because it cannot
+-- see anyone else's claims. That is the RLS model working, not a gap here, and the honest response
+-- is to withhold the surface rather than serve a per-caller answer that reads fleet-wide. Quota
+-- convergence is an operations question and this is an operations surface.
+--
+-- `security_invoker` is kept anyway as defence in depth: if a future migration widens the grant,
+-- rows are still RLS-filtered rather than leaked wholesale. OQ-8/OQ-9 pin both the view option and
+-- the grant so widening either fails a test that says why.
+-- ---------------------------------------------------------------------------
+grant select on agent_board.v_ownerless_claim_quota to service_role;
+
+-- NOT REDUNDANT — this is the AL-9b trap from #367, which arrives on a plain
+-- `create or replace function` and not only on a DROP. A newly created function is granted
+-- EXECUTE to PUBLIC by default, so `authenticated` inherits it through PUBLIC and granting only
+-- service_role above would change nothing on its own.
+revoke all on function agent_board.ownerless_quota_census() from public;
+grant execute on function agent_board.ownerless_quota_census() to service_role;