mercury-agent 0.21.0 → 0.22.0

package/container/Dockerfile

@@ -111,6 +111,7 @@ COPY src/cli/mrctl.ts /app/src/cli/mrctl.ts
 COPY src/cli/mrctl-http.ts /app/src/cli/mrctl-http.ts
 COPY src/extensions/reserved.ts /app/src/extensions/reserved.ts
 COPY src/extensions/permission-guard.ts /app/src/extensions/permission-guard.ts
+COPY src/extensions/bash-timeout.ts /app/src/extensions/bash-timeout.ts
 COPY src/text/sanitize-text.ts /app/src/text/sanitize-text.ts
 COPY src/types.ts /app/src/types.ts
 COPY resources/ /app/resources/

package/container/Dockerfile.base

@@ -80,6 +80,7 @@ COPY src/cli/mrctl.ts /app/src/cli/mrctl.ts
 COPY src/cli/mrctl-http.ts /app/src/cli/mrctl-http.ts
 COPY src/extensions/reserved.ts /app/src/extensions/reserved.ts
 COPY src/extensions/permission-guard.ts /app/src/extensions/permission-guard.ts
+COPY src/extensions/bash-timeout.ts /app/src/extensions/bash-timeout.ts
 COPY src/text/sanitize-text.ts /app/src/text/sanitize-text.ts
 COPY src/types.ts /app/src/types.ts
 COPY resources/ /app/resources/

package/docs/configuration.md

@@ -302,6 +302,48 @@ Container env passthrough: explicit `all` override — these vars reach every sp
 
 For secrets that only host-side hooks and jobs need, prefer `mercury.env({ from: "…", hostOnly: true })`, which keeps them out of containers in either mode. For credentials the agent should never hold at all, use a host-side capability handler (`mercury.capability()`), which runs the privileged call on the host and returns only the result.
 
+## Container shared memory (`agent.container_shm_size`)
+
+How big `/dev/shm` is inside every agent container, in Docker's `--shm-size` spelling:
+
+```yaml
+agent:
+  container_shm_size: 512m   # 512m (default) | 1g | any Docker size
+```
+
+Env: `MERCURY_CONTAINER_SHM_SIZE`. Applies to both runtimes — `/dev/shm` is Docker's, not gVisor's.
+
+Docker's own default is **64 MB**, and that is the wrong size for what these containers actually do. Chromium is the workload: the `web-browser` extension drives it, the `poster` extension renders through it, and an agent asked to screenshot a page runs it directly. A large frame (a 1856x2304 poster, say) exhausts 64 MB, and Chromium's response is not an error — it **hangs**. The turn then sits idle until the host kills the container at `container_timeout_ms`, and the user gets no reply, only the time-limit marker. Three customer-facing runs were lost that way on 2026-09-10 before the cause was found.
+
+The default is deliberately generous rather than minimal. A tmpfs is allocated as it is used, so a container that renders nothing pays nothing for the headroom, and under `container_runtime: runsc` the total is bounded by the container's `--memory` limit anyway. Lower it only on a host that is genuinely short of RAM, and expect large renders to hang again if you go back to 64 MB.
+
+Two related things worth knowing:
+
+- Passing `--disable-dev-shm-usage` to Chromium sidesteps `/dev/shm` entirely and works at any size. The `poster` extension already does this, which is why it kept working while direct Chromium calls hung.
+- `mercury preflight`'s round-trip probe spawns with the same value, so the shape it vouches for is the shape a real turn gets.
+
+## Bash call timeout (`agent.container_bash_timeout_seconds`)
+
+The longest a single `bash` tool call may run inside an agent container, in seconds:
+
+```yaml
+agent:
+  container_bash_timeout_seconds: 300   # 300 (default)
+```
+
+Env: `MERCURY_CONTAINER_BASH_TIMEOUT_SECONDS`. It is both a default and a ceiling: a call that names no `timeout` gets this one, and a call that asks for more is clamped down to it. The agent can only ask for *less*.
+
+pi's `bash` tool takes an optional `timeout` and arms no timer at all when the model omits it — its own parameter description says "optional, no default timeout". A command that hangs rather than failing therefore blocks the turn with nothing watching it, until the host kills the whole container at `container_timeout_ms` and the user gets no reply, only the time-limit marker. That is the other half of the 2026-09-10 incident described under `container_shm_size`: the hanging Chromium was the cause, and the missing bound is what turned each hang into a lost 20-minute run instead of a recoverable error.
+
+When the bound fires, pi kills the command's whole process tree and returns `Command timed out after N seconds` as a normal tool error, so the turn can retry differently, fall back, or explain — while it still has budget left to reply.
+
+Two things to keep in mind when changing it:
+
+- **Keep it comfortably below `container_timeout_ms`.** A bound at or above the container limit buys nothing, because the SIGKILL arrives first. The defaults (300 s against 5 minutes) leave that ratio to whoever tunes both.
+- **Long jobs need a different shape, not a bigger number.** Raising this to cover a slow build makes every hang that much more expensive. Prefer backgrounding the work and polling a file for its result.
+
+The value in force is stated to the model in its system prompt, so it can plan around the ceiling instead of meeting it as a dead tool call.
+
 ## Agent run traces (`agent.trace_runs`)
 
 Captures pi's raw output from inside each agent container, with arrival timestamps, and copies it to `<dataDir>/traces/` before the container is reaped:

package/docs/container-lifecycle.md

@@ -100,6 +100,12 @@ When a container exceeds the timeout:
 4. A marker assistant message ("[System: this run was killed at the container time limit before replying. …]") is recorded in chat history, so the next run's agent knows the previous run was cut off and can check the workspace for partial work
 5. Queue unblocks, next message can proceed
 
+A container that hits the timeout was not necessarily busy. The commonest way to burn the whole budget is a child process that hangs rather than fails, and the commonest such child is Chromium on a too-small `/dev/shm`. Every spawn therefore sizes it explicitly — see `container_shm_size` in [configuration.md](configuration.md#container-shared-memory-agentcontainer_shm_size).
+
+| Config | Env Var | Default | Applies to |
+|--------|---------|---------|------------|
+| `containerShmSize` | `MERCURY_CONTAINER_SHM_SIZE` | `512m` | both runtimes |
+
 The host always injects a resolved **model chain** into the container (after `MERCURY_*` passthrough) so retries and fallbacks use the same policy Mercury loaded at startup:
 
 | In-container env | Source (host) | Purpose |

package/docs/deployment.md

@@ -135,6 +135,27 @@ first:
 Mercury's own CLI does not load the WhatsApp stack at startup, so running
 `mercury upgrade` never locks the tree it is replacing.
 
+### Disk: prune the build cache after every upgrade
+
+Each upgrade pulls a new base image (~2 GB of layers) and Mercury rebuilds the
+derived `mercury-agent-ext` image on the next start. Removing the old base with
+`docker rmi` / `docker image prune` does **not** free its layers: the derived
+build leaves BuildKit cache records that keep the old base layers alive, so
+overlay2 grows by roughly 2 GB per release and `docker system df` under-reports
+it. A 38 GB box hit 93% after five releases (2026-09-08).
+
+Once `/health` reports the new version and the journal shows `Built derived
+agent image`, run:
+
+```bash
+docker builder prune -af
+```
+
+Safe with the bot running: it touches no image, container or data. The only
+cost is that the next derived-image build starts cold, which it does after a
+base change anyway. Never `docker image prune -a` on an idle box — with no
+container running it removes the derived image and the base too.
+
 ## Auto-Restart Behavior
 
 Both systemd and launchd are configured to automatically restart Mercury if it crashes:

package/docs/extensions.md

@@ -279,6 +279,39 @@ mercury.widget({
 
 Widgets render HTML fragments in the dashboard overview. Errors show a placeholder — never crash the dashboard.
 
+### `mercury.capability(name, handler)`
+
+Register a host-side handler the agent reaches from inside the container with
+`mrctl capability <name> <action> '<json>'` → `POST /api/capability/:name/:action`.
+Credentials the handler uses stay on the host and never enter the container.
+
+```typescript
+// in extensions/rooms/index.ts
+mercury.permission({ defaultRoles: ["admin"] });
+mercury.capability("rooms", async (req, ctx) => {
+  if (req.action === "book") return { data: await bookRoom(req.callerId, req.body) };
+  return { status: 400, data: { error: "unknown action" } };
+});
+```
+
+- **`name` must equal the extension's own name** — registering under any other
+  name throws at load. The broker authorizes a call with `checkPerm(<name>)`,
+  so the equality is what makes that the *extension's* permission: the same
+  predicate the env-var injection rule and the sensitive-connection guard use.
+  A capability named after some other extension's permission would be reachable
+  by callers those two legs believe cannot reach it.
+- **The extension must also call `mercury.permission()`.** Without a *registered*
+  permission of that name, `checkPerm` refuses every role — including `admin`,
+  whose grant is the enumerated set of built-in plus registered permissions, not
+  a wildcard — so the capability is unreachable, silently.
+- An extension named after a **built-in** permission (`prompt`, `stop`, `clear`,
+  …) cannot register a capability at all: it could not own that permission
+  (`mercury.permission()` refuses a built-in name) while the broker would still
+  authorize the handler with it, and `prompt` is one every member holds.
+- One capability per extension; branch on `req.action` for sub-verbs.
+- The name is resolved host-wide, so two extensions cannot own it — a
+  collision is rejected at load time.
+
 ### `mercury.store`
 
 Scoped key-value store for persistent state.

package/docs/goals/feed-watch-social-sources/decisions.md

@@ -56,7 +56,9 @@
 - **Used by:** feed-watch-telegram-parser, feed-watch-telegram-source-config
 
 ## D-003: `MAX_SOURCES` stays at 20 for Milestone 1
-- **Category:** scope
+- **Category:** scope — **superseded 2026-09-10 by D-006 and D-007**: the
+  cap is 150 and the rationing it implied is gone. Kept as written; the
+  revisit clause it names is exactly what fired.
 - **Decided:** the football manifest holds 14 entries; M1.1 may add at most 3
   and M1.5 at most 3. The cap constant, its validator and its description do
   not move in this milestone.
@@ -91,6 +93,224 @@
 - **Used by:** feed-watch-telegram-source-config,
   feed-watch-telegram-football-deploy
 
+## D-006: `MAX_SOURCES` moves and the poll fetches concurrently, as one story
+- **Category:** scope (revisits D-003; D-005 is reserved for the no-bypass
+  rule named under "Open" below, so this is D-006)
+- **Decided:** 2026-09-10, by the owner: the football space is to watch
+  many sources, and `football-match-day-mode` D-010 dropped its fixtures
+  provider in favour of watched channels, so D-003's revisit clause fires.
+  One story, `feed-watch-source-cap-and-concurrent-poll` (M1.6 on the
+  roadmap; idea filed at
+  `docs/ideas/feed-watch-source-cap-and-concurrent-poll.md`; architecture
+  via `/f-feature-planning`), raises the constant and its description, makes
+  `pollSpace` fetch targets through a bounded pool instead of one after
+  another, and measures the tick against the poll interval at the new cap.
+  The number is decided in that story from the measurement, not here.
+- **Alternatives considered:** rationing the six free slots between the
+  two goals (the 2026-09-07 shape); a per-space override of the cap (a
+  config surface for a bound that should simply be higher).
+- **Reasoning:** the cap guards the tick, not the bill — sequential fetches
+  at `FETCH_TIMEOUT_MS = 5000` inside a 3-minute interval on the host's
+  single thread, and `gnews` fan-out per term. `verify_max_items`,
+  `digest_max_items`, `BUFFER_CAP` and `max_per_hour` bound everything that
+  costs tokens, so more sources cannot make a run dearer. A concurrent fetch
+  removes the reason the bound was low.
+- **Revisit if:** the measured tick at the new cap still overruns the
+  interval — then the interval, not the cap, is the next knob.
+- **Used by:** feed-watch-source-cap-and-concurrent-poll,
+  feed-watch-rung0-manifest-sources (its "stay within 20" clause relaxes
+  once M1.6 lands), football-match-day-mode M1.7
+
+## D-007: 150 sources, a pool of 8, and one Reddit fetch per tick
+- **Category:** implementation (settles the numbers D-006 left to the story)
+- **Decided:** 2026-09-10, in `feed-watch-source-cap-and-concurrent-poll`.
+  `MAX_SOURCES = 150`; `FETCH_POOL_SIZE = 8`, a module constant and not a
+  config key; `BUFFER_CAP = 1500`; hosts in `ONE_PER_TICK_HOSTS` (the three
+  Reddit spellings) get **one** fetch per tick, round-robin from
+  `host_cursor:<spaceId>`, the rest counted as `hostDeferred`.
+- **Alternatives considered:** a pool of 6 (the idea file's first guess — the
+  ratio to the interval is what matters, and 8 makes the worst case a third
+  of it rather than half); making the pool a config key (a property of the
+  host process, not of a space, and no admin has the measurement to set it);
+  spacing same-host fetches 60 s apart *inside* the tick instead of rationing
+  across ticks (three Reddit feeds would put the tick's floor at 120 s, past
+  the slow-tick threshold, so every tick would report itself slow); giving
+  Reddit its own `SourceKind` (a rate limit is a property of the host, not of
+  the parser).
+- **Reasoning:** the cap always guarded the tick, and the pool is what moves
+  the tick's worst case from `due x 5 s` to `ceil(due / 8) x 5 s`. 150 clears
+  the round's 85-entry manifest (§7.I of
+  `docs/notes/2026-09-10-feed-watch-round.md`) with room for the bench tier.
+  1,500 buffered items is a *digest* bound: the daily article is built from
+  the buffer, so an overflow silently drops the oldest matched items of a
+  match day. Reddit's 60 s per-IP ceiling was measured on 2026-09-10 from the
+  box's own network, for both user-agents.
+- **Revisit if:** the `feed-watch: slow poll tick` line appears on a healthy
+  box (then the interval or the pool, in that order, not the cap); or a
+  second space starts watching (see D-008).
+- **Used by:** feed-watch-source-cap-and-concurrent-poll,
+  football-match-day-sources, feed-watch-rung0-manifest-sources
+
+## D-008: `inFlight` stays process-wide, and the tick's cost is now readable
+- **Category:** scope
+- **Decided:** 2026-09-10. The `inFlight` guard in `index.ts` keeps its
+  present meaning — **one tick at a time across every space** — and does not
+  become per space in this story. The reader that would justify changing it
+  ships instead: `feed-watch: slow poll tick` at `info` past half the
+  interval, and an unconditional `last_tick` store row carrying `lastTickMs`
+  and `lastTickAt`.
+- **Alternatives considered:** a per-space `inFlight` map (one slow space
+  would stop costing the others their poll — but the fetch phase is now
+  concurrent *within* a space, so the sum of ticks is far smaller than it
+  was, and a per-space guard would let N spaces put N pools on the host's
+  single thread with nothing bounding the total).
+- **Reasoning:** the football space is the only watcher on the box, so the
+  guard has never actually fired for the reason a per-space version would
+  fix. Splitting it now would trade a bound nobody has hit for an unbounded
+  total, and would do it without a measurement. The measurement is what this
+  story adds.
+- **Revisit if:** a second space enables `feed-watch.enabled`, or the
+  `lastTickMs` row on the box climbs past half the interval.
+- **Used by:** feed-watch-source-cap-and-concurrent-poll
+
+## D-009: liveness is `newestItemAt` plus a once-per-streak pair, and nothing else
+- **Category:** implementation
+- **Decided:** 2026-09-10, in `feed-watch-source-liveness` (M1.7).
+  `SourceStatus` gains `newestItemAt` — the running **maximum** of every
+  item's `publishedAt` a source has served — and `silentAt`. Past
+  `STALE_AFTER_MS = 7 days` with the fetch still succeeding, one
+  `feed-watch: source silent` at `warn`; a fresh item writes one
+  `feed-watch: source active again`, also at `warn`. Both fields are carried
+  through the failure branch untouched.
+- **Alternatives considered:** recording only this tick's newest date (a feed
+  that rotates its window would read as freshly stale every time it dropped
+  its newest item); a boolean `silent` flag (the *time* the silence started is
+  the fact an operator wants, and a boolean cannot be read as a duration);
+  `info` for the recovery line (the pair it mirrors is both-at-warn, and two
+  levels for two ends of one streak makes `grep -c` return a number that reads
+  as unresolved incidents).
+- **Reasoning:** `lastOkAt` is about the fetch, so a page serving a 2023 body
+  with HTTP 200 writes the same row as a live one — `t.me/s/ILRentsTLV` does
+  exactly this, and X's syndication widget did it for months. At 14 sources a
+  person notices; at the ~85 the round ships, the only symptom is an article
+  missing something nobody knew to expect.
+- **Revisit if:** a low-volume source flaps silent/active often enough to be
+  noise — then a per-source threshold, which is config surface nobody needs
+  yet.
+- **Used by:** feed-watch-source-liveness
+
+## D-010: a stale source is **not** treated as failed, for any kind
+- **Category:** scope (answers the roadmap's "Source liveness" row, which
+  asked for stale-as-failed "for kinds known to serve stale caches")
+- **Decided:** 2026-09-10. No backoff, no `nextAttemptAt`, no removal, no
+  exception for Telegram or for any other kind. Visibility only.
+- **Alternatives considered:** treating a stale Telegram channel as failed
+  (the row's own wording); a per-kind flag on `SourceKind` saying whether its
+  transport can serve a stale 200.
+- **Reasoning:** the roadmap row's premise does not survive contact with the
+  data. A *quiet* channel and a *dead* one are byte-identical from outside —
+  the same 200, the same well-formed body, the same old newest post — so
+  "known to serve stale caches" describes the transport, not the source, and
+  every Telegram channel would qualify including the ones that simply post
+  twice a month. Backing one off costs the story it eventually publishes, and
+  at the 96-minute backoff ceiling it would miss the return as well. The
+  failure the row is really about is *nobody noticing*, and a log line plus a
+  store field closes that without touching the poll.
+- **Consequence:** the manual per-channel precondition in
+  `feed-watch-telegram-football-deploy` (M1.5 — "newest `<time datetime>`
+  inside the last 7 days", checked by hand before adding a channel) is
+  **redundant** the moment this deploys: the same week, checked every tick, by
+  the running code. The edit to that document is **not** made here —
+  `football-match-day-sources` owns it this round — and is owed to whoever
+  lands it.
+- **Revisit if:** a watched kind appears whose transport can serve a stale
+  body *and* whose real publication rate is known to be daily, so that
+  silence is unambiguous.
+- **Used by:** feed-watch-source-liveness, feed-watch-telegram-football-deploy,
+  football-match-day-sources
+
+> **Numbering note (2026-09-10).** These three were written as D-007 to D-009
+> by S2 of the 2026-09-10 feed-watch round while S1 claimed D-007 to D-010 in
+> a parallel worktree; the coordinator renumbered them to D-011 to D-013 on
+> landing, together with the citations in `feed-watch-roundup-lane`'s spec.
+
+## D-011: The digest is lane-addressed, and the default lane is the unnamed one
+
+- **Category:** architecture
+- **Decided:** `[feed-watch:digest]` keeps its exact behaviour and its exact
+  cursor key `last_digest_at:<spaceId>`. A second reader carries
+  `[feed-watch:digest:<lane>]` (`lane` matching `[a-z0-9-]{1,32}`) and reads
+  and advances `last_digest_at:<spaceId>:<lane>`, with its own pending mark
+  `digest_until:<spaceId>:<lane>`. The block of a named lane carries
+  `lane="…"`; the default block is byte-for-byte unchanged, so the article's
+  prompt does not move. A marker whose lane will not parse yields **no**
+  digest, no mark and no cursor movement — never a fallback to the default
+  lane — and is warned about once per hook.
+- **Alternatives considered:** a second task on the plain marker (the reason
+  this story exists: `handleAfterContainer` advances the one cursor for any
+  run carrying the marker, so the 09:00 article would see only what arrived
+  since the last roundup — silently, reading as a thin news day); a per-task
+  cursor keyed on the task id (the id is a live row that does not exist until
+  the task is created, so the prompt could not name its own cursor, and a
+  re-created task would silently start from the beginning of the buffer); a
+  config key listing lanes (a second surface to keep in step with the prompts
+  that actually carry the markers).
+- **Reasoning:** the marker is already the contract between a task prompt and
+  the extension, and it is the only thing both hooks can see. Extending it is
+  a suffix, not a mechanism. Making the unnamed lane the default is what keeps
+  the migration empty: nothing on the box has to be renamed.
+- **Revisit if:** a lane ever needs to *read without consuming* (a preview
+  lane). Today every lane advances its own cursor, and that is stated as the
+  point rather than an accident.
+- **Used by:** feed-watch-roundup-lane
+
+## D-012: Each lane owns its window file and its archive
+
+- **Category:** architecture
+- **Decided:** the default lane keeps `knowledge/feed-digest-window.md` and
+  `knowledge/feed-digest.md`; a named lane writes
+  `knowledge/feed-digest-window.<lane>.md` and
+  `knowledge/feed-digest.<lane>.md`. The trailer names the lane's own path.
+- **Alternatives considered:** one shared pair of files with the lane written
+  inside each section. Rejected on two counts: the window file is *named to
+  the model in prose*, so it has to hold the window that model was handed
+  rather than whichever lane wrote last; and the archive is trimmed to
+  `DIGEST_FILE_WINDOWS = 30` sections, so six writers a day would cut the
+  article's readable history from thirty days to five without anything saying
+  so.
+- **Reasoning:** the cost is one string; the failure it removes is silent.
+- **Revisit if:** a space runs enough lanes for the workspace listing to
+  matter — then a `knowledge/feed-digest/<lane>.md` subdirectory, decided as
+  its own change.
+- **Used by:** feed-watch-roundup-lane
+
+## D-013: The roundup releases what has not been reported, on five slots a day
+
+- **Category:** scope
+- **Decided:** the football space's roundup runs
+  `0 10,13,16,19,22 * * *` `Asia/Jerusalem` — five slots, all outside
+  `feed-watch.quiet_hours` (`00:30-07:30`) and none at 09:00, which is the
+  article's. Its window **overlaps** what the immediate verify runs already
+  posted, and that overlap is resolved by the editorial rule that already
+  exists (`AGENTS.md § מניעת כפילויות`: an item already in the notebook's
+  `## Current State` is not reported again unless a concrete detail changed),
+  not by a second cursor. The roundup is not subject to
+  `feed-watch.max_per_hour`, which governs verify runs only.
+- **Alternatives considered:** a cursor that skips items a verify run
+  consumed (feed-watch does not record which items a run posted — only which
+  it was handed — so this would be a new bookkeeping surface for a rule the
+  standard already states); hourly slots (five was chosen against a `NO_UPDATE`
+  run's own cost, below); making the roundup obey `max_per_hour` (it is a
+  `tasks` row, not a batch, and a scheduled release nobody can predict is
+  worse than one that says nothing).
+- **Reasoning:** cost is `runs/day × prefix × $2.50/M` and a run that answers
+  `NO_UPDATE` still pays its whole prefix — five slots is roughly
+  $0.75–1.75/day against the space's $16–20/day, and the slot count is the
+  lever if the 50-source manifest pushes the window up.
+- **Revisit if:** the real group finds five too many or too few after the
+  link — the cron is a live row, so changing it needs no deploy.
+- **Used by:** feed-watch-roundup-lane
+
 ## Open — for the owner, raised by the 2026-09-06 blocked-sources research
 
 Recorded in [blocked-sources-research.md](blocked-sources-research.md); none

package/docs/goals/feed-watch-social-sources/roadmap.md

@@ -1,6 +1,6 @@
 # Roadmap: feed-watch source adapters — social and messaging platforms as a ladder
 
-**Last updated**: 2026-09-07 (M1.2 and M1.3 shipped into local `dev`, unpushed and undeployed; approved by the owner; Milestone 1 architecture filled in the five story specs and D-001–D-004 recorded in [decisions.md](decisions.md). History: drafted by `/f-feature-roadmap` on 2026-09-06, one critic round and revision; the blocked-sources research folded in and a second critic round with its revision — see the two Plan Review sections at the end; later rungs named, not decomposed — see [goal.md](goal.md) for the research that shaped them)
+**Last updated**: 2026-09-10 (**M1.6 added — `feed-watch-source-cap-and-concurrent-poll`, D-006**: the owner wants the football space to watch many sources and `football-match-day-mode` D-010 dropped its fixtures provider for watched channels, so D-003's cap rationing ends; the story raises `MAX_SOURCES` and makes the poll fetch concurrently, idea filed, architecture via `/f-feature-planning`. The rung-0 ranking guard from the other goal is lifted: the official highlights channels may be plain `rss` picks now. M1.4 was pushed as PR #97 and deployed 2026-09-10 on `da35ea9`. Earlier, 2026-09-09: M1.4 shipped into local `dev`, unpushed — `expandSources` now emits `kind: "telegram"`, so the M1.3 parser is reachable by configuration and stops being dead code the moment a channel is written; M1.2 and M1.3 shipped into local `dev`; pushed to `origin/dev` 2026-09-08 and deployed 2026-09-09 in 0.21.0, and inert on the box until M1.4 deploys and a channel is configured (M1.5); approved by the owner; Milestone 1 architecture filled in the five story specs and D-001–D-004 recorded in [decisions.md](decisions.md). History: drafted by `/f-feature-roadmap` on 2026-09-06, one critic round and revision; the blocked-sources research folded in and a second critic round with its revision — see the two Plan Review sections at the end; later rungs named, not decomposed — see [goal.md](goal.md) for the research that shaped them)
 **Last updated**: 2026-09-06 (drafted by `/f-feature-roadmap`: one milestone decomposed, one critic round, one revision; then the blocked-sources research folded in and a second critic round with its revision — see the two Plan Review sections at the end; later rungs named, not decomposed — see [goal.md](goal.md) for the research that shaped them)
 
 > Order chosen so the milestone is useful on its own: M1.1 needs no code and
@@ -21,11 +21,16 @@
 
 | ID | Story | Slug | Depends on | Status |
 |----|-------|------|------------|--------|
-| M1.1 | Rung 0: probe and add Reddit / YouTube / Google-News `site:` sources to the football manifest — config plus a probe of the UA and Reddit's rate; deploys by `apply` alone, no restart | [feed-watch-rung0-manifest-sources](../../backlog/feed-watch-rung0-manifest-sources.md) | — (owner names the candidates; `r/soccer/new/.rss` is the stated minimum) | backlog |
-| M1.2 | Open the parser seam: `SourceKind` in `feeds.ts`, required `FetchTarget.kind`, `parse` / `redirect` options on `fetchFeed`, an exhaustive switch in `index.ts` that picks the whole fetch | [feed-watch-parser-seam](../../archive/feed-watch-social-sources/2026-09-07-feed-watch-parser-seam.md) | — | **done** 2026-09-07 — merged into local `dev`, **not pushed, not deployed**. `SourceKind` + `RedirectMode` + `BodyParser` in `feeds.ts`, `parse`/`redirect` on `fetchFeed`, required `FetchTarget.kind`, one exhaustive switch in `index.ts:179` whose `never` default is negative-tested (a phantom kind fails `typecheck:examples` at `index.ts:197`). **One correction found while building it:** the `MAX_ITEMS_PER_FEED` cap had to move out of the parser and into `fetchFeed` — delegating the parser had silently delegated the bound, and the `=== MAX` report meant an overflowing kind would have uncapped the feed *without* firing `onTruncate`; it is `>=` plus an explicit slice now, a provable no-op for `xml`. **Two deviations from the spec:** `RequestRedirect` is a DOM global this project has no lib for, so the union is spelled out locally as `RedirectMode`; and the `expandSources` `kind` assertion cannot fail until M1.3 widens the union — it is a value check waiting for a second kind, not a runtime guard today. Gate 4208 pass / 0 fail (+6) |
-| M1.3 | Telegram preview-page parser with a trimmed real fixture — newest-first, `js-message_text` only, `redirect: "manual"` so a 302 backs off | [feed-watch-telegram-parser](../../archive/feed-watch-social-sources/2026-09-07-feed-watch-telegram-parser.md) | M1.2 | **done** 2026-09-07 — merged into local `dev`, **not pushed, not deployed, and still inert** (nothing constructs a `telegram` target until M1.4). `parseTelegramPreview` + `TELEGRAM_TITLE_MAX` in `feeds.ts`, `SourceKind` widened, `case "telegram"` in `index.ts` with `parse` + `redirect: "manual"`, and `onTruncate` hoisted now that there are two callers. **Four corrections the live probe forced**, all from 34 channel captures: (1) `t.me/s/espn` serves `data-post="founder/…"` and `t.me/s/TrollFootball` serves `thefootballtroll/…` — Telegram auctions usernames, so reading the channel from `data-post` is load-bearing, not stylistic; (2) `telegram.me` and `telegram.dog` are live aliases and had to join `t.me` in the outbound-link skip list; (3) `data-post` and `<time>` are now anchored to their own elements — unanchored, the real Sky Sports fixture dates two posts wrong off a `message_video_duration` tag; (4) Biome 2.x lints HTML, so the fixtures needed a narrow `!tests/fixtures/**/*.html` exclusion. **Two fixtures, not one:** no channel carried all four block shapes *and* passed the repo content rule — Sky Sports gives text/grouped-media/caption-less-photo, tginfo gives the reply quote and the links. **The one gap M1.3 shipped with is closed.** The `telegram` case's `parse:`/`redirect:` were compile-checked but not runtime-pinned, because the switch was private to a closure in `buildDeps`; the 2026-09-07 review round lifted it into the exported `fetchForTarget` and pinned both with a recording `fetchImpl`, confirmed by deletion. The **review round also removed the outbound-link append**: `description`'s only reader is `haystack` in `match.ts` (`toStoredItem` drops it), so the append reached no digest or verify run and only widened term matching to URL text -> `ideas/feed-watch-telegram-outbound-link.md`. What M1.4 still owes is the leg above the switch — that `expandSources` actually emits `kind: "telegram"`. Diff 771/15 against D-002's 500-line budget — reported, owner's call. Gate 4277 pass / 0 fail |
-| M1.4 | `TelegramSource` in the config union, `expandSources`, the `sources` description, and an end-to-end `pollSpace` test over the fixture | [feed-watch-telegram-source-config](../../backlog/feed-watch-telegram-source-config.md) | M1.3 | backlog |
-| M1.5 | Football manifest: add the owner's channels, deploy once (restart → verify installed copy → apply → check) and verify on the running bot, incl. the 302 drill | [feed-watch-telegram-football-deploy](../../backlog/feed-watch-telegram-football-deploy.md) | M1.4 **+ precondition:** exact usernames, each probed 200 with message blocks **and a newest `<time datetime>` inside the last 7 days** (Sport 5 has no public preview; `ILRentsTLV` serves blocks from 2023) | backlog — blocked on the owner's list |
+| M1.1 | Rung 0: probe and add Reddit / YouTube / Google-News `site:` sources to the football manifest — config plus a probe of the UA and Reddit's rate; deploys by `apply` alone, no restart | [feed-watch-rung0-manifest-sources](../../archive/feed-watch-social-sources/2026-09-10-feed-watch-rung0-manifest-sources.md) | — (owner names the candidates; `r/soccer/new/.rss` is the stated minimum). **2026-09-10:** the official highlights channels (LaLiga, Premier League, UEFA YouTube feeds) are no longer excluded from the picks — `football-match-day-mode` D-010 dropped the fixtures entry that owned them, and its M1.7 wants them as `rss`; whichever story lands first adds them. The "stay within 20, leave 3 free" clause relaxes once M1.6 lands | **absorbed 2026-09-10** into `football-match-day-sources` (`football-match-day-mode` M1.7), never started on its own — the whole researched list shipped in one manifest edit (`docs/notes/2026-09-10-feed-watch-round.md` §7). The spec is kept for its deploy order; its Reddit rate verdict (one request per 60 s per IP, either user-agent) is recorded in the football manifest comment and is what M1.6's per-host rule answers |
+| M1.2 | Open the parser seam: `SourceKind` in `feeds.ts`, required `FetchTarget.kind`, `parse` / `redirect` options on `fetchFeed`, an exhaustive switch in `index.ts` that picks the whole fetch | [feed-watch-parser-seam](../../archive/feed-watch-social-sources/2026-09-07-feed-watch-parser-seam.md) | — | **done** 2026-09-07 — pushed to `origin/dev` 2026-09-08 and **deployed 2026-09-09** on image 0.21.0. `SourceKind` + `RedirectMode` + `BodyParser` in `feeds.ts`, `parse`/`redirect` on `fetchFeed`, required `FetchTarget.kind`, one exhaustive switch in `index.ts:179` whose `never` default is negative-tested (a phantom kind fails `typecheck:examples` at `index.ts:197`). **One correction found while building it:** the `MAX_ITEMS_PER_FEED` cap had to move out of the parser and into `fetchFeed` — delegating the parser had silently delegated the bound, and the `=== MAX` report meant an overflowing kind would have uncapped the feed *without* firing `onTruncate`; it is `>=` plus an explicit slice now, a provable no-op for `xml`. **Two deviations from the spec:** `RequestRedirect` is a DOM global this project has no lib for, so the union is spelled out locally as `RedirectMode`; and the `expandSources` `kind` assertion cannot fail until M1.3 widens the union — it is a value check waiting for a second kind, not a runtime guard today. Gate 4208 pass / 0 fail (+6) |
+| M1.3 | Telegram preview-page parser with a trimmed real fixture — newest-first, `js-message_text` only, `redirect: "manual"` so a 302 backs off | [feed-watch-telegram-parser](../../archive/feed-watch-social-sources/2026-09-07-feed-watch-telegram-parser.md) | M1.2 | **done** 2026-09-07 — pushed to `origin/dev` 2026-09-08 and **deployed 2026-09-09** on image 0.21.0, and inert on the box until M1.4 deploys (nothing constructed a `telegram` target). `parseTelegramPreview` + `TELEGRAM_TITLE_MAX` in `feeds.ts`, `SourceKind` widened, `case "telegram"` in `index.ts` with `parse` + `redirect: "manual"`, and `onTruncate` hoisted now that there are two callers. **Four corrections the live probe forced**, all from 34 channel captures: (1) `t.me/s/espn` serves `data-post="founder/…"` and `t.me/s/TrollFootball` serves `thefootballtroll/…` — Telegram auctions usernames, so reading the channel from `data-post` is load-bearing, not stylistic; (2) `telegram.me` and `telegram.dog` are live aliases and had to join `t.me` in the outbound-link skip list; (3) `data-post` and `<time>` are now anchored to their own elements — unanchored, the real Sky Sports fixture dates two posts wrong off a `message_video_duration` tag; (4) Biome 2.x lints HTML, so the fixtures needed a narrow `!tests/fixtures/**/*.html` exclusion. **Two fixtures, not one:** no channel carried all four block shapes *and* passed the repo content rule — Sky Sports gives text/grouped-media/caption-less-photo, tginfo gives the reply quote and the links. **The one gap M1.3 shipped with is closed.** The `telegram` case's `parse:`/`redirect:` were compile-checked but not runtime-pinned, because the switch was private to a closure in `buildDeps`; the 2026-09-07 review round lifted it into the exported `fetchForTarget` and pinned both with a recording `fetchImpl`, confirmed by deletion. The **review round also removed the outbound-link append**: `description`'s only reader is `haystack` in `match.ts` (`toStoredItem` drops it), so the append reached no digest or verify run and only widened term matching to URL text -> `ideas/feed-watch-telegram-outbound-link.md`. What M1.4 still owes is the leg above the switch — that `expandSources` actually emits `kind: "telegram"`. Diff 771/15 against D-002's 500-line budget — reported, owner's call. Gate 4277 pass / 0 fail |
+| M1.4 | `TelegramSource` in the config union, `expandSources`, the `sources` description, and an end-to-end `pollSpace` test over the fixture | [feed-watch-telegram-source-config](../../archive/feed-watch-social-sources/2026-09-09-feed-watch-telegram-source-config.md) | M1.3 | **done** 2026-09-09 — merged into local `dev`; pushed as PR #97 and **deployed 2026-09-10** on `da35ea9` (restart only — feed-watch is the installed snapshot, not the image). `TelegramSource` + `TELEGRAM_CHANNEL` + the validator branch + the third shape in the `sources` description in `config.ts`; one `telegram` branch in `expandSources` emitting `key: telegram:<channel>`, `url: https://t.me/s/<channel>`, `kind: "telegram"` and `fallbackSource: outlet ?? t.me/<channel>`; the `sources` row of `SKILL.md` names the shape and what `outlet` means. **The leg M1.3 left open is closed**: an end-to-end `pollSpace` run drives the real `fetchForTarget` over a fake transport serving `telegram-preview.html`, and a matched post reaches the buffer and the verify prompt with the post's own time, its `t.me` permalink and the configured outlet label — changing that branch to `kind: "xml"` reddens three tests, confirmed by deletion, which is the failure the type system cannot see. Five guards negative-tested in total (the kind, the outlet fallback, `TELEGRAM_CHANNEL`, the unknown-type rejection, the description). **Still inert on the box**: no channel is in the football manifest until M1.5, so a live target needs a temporary write — see the M1.4 block in `docs/pending-verification.md`. Diff 430 insertions against D-002's 500-line budget, no new fixture. Gate 4438 pass / 9 skip / 0 fail |
+| M1.5 | Football manifest: add the owner's channels, deploy once (restart → verify installed copy → apply → check) and verify on the running bot, incl. the 302 drill | [feed-watch-telegram-football-deploy](../../archive/feed-watch-social-sources/2026-09-10-feed-watch-telegram-football-deploy.md) | M1.4 **+ precondition:** exact usernames, each probed 200 with message blocks **and a newest `<time datetime>` inside the last 7 days** (Sport 5 has no public preview; `ILRentsTLV` serves blocks from 2023) | **absorbed 2026-09-10** into `football-match-day-sources` (`football-match-day-mode` M1.7), never started on its own — the whole researched list shipped in one manifest edit (`docs/notes/2026-09-10-feed-watch-round.md` §7). The spec is kept for its deploy order and its 302 drill, both of which M1.7 reuses; the probed channel list is §7.A of that note and all 20 live channels shipped, not the two or three this spec scoped |
+| M1.6 | Raise `MAX_SOURCES` and fetch sources concurrently: a bounded pool in `pollSpace`'s fetch loop (per-target `status` bookkeeping and the once-per-streak warn preserved), the constant and the `sources` description moved to a number a measured tick supports, a test that one slow target cannot delay the rest past the pool bound, and a reader for the tick's elapsed time | [feed-watch-source-cap-and-concurrent-poll](../../archive/feed-watch-social-sources/2026-09-10-feed-watch-source-cap-and-concurrent-poll.md) | — (D-006; independent of M1.5 — a restart-only deploy of the snapshot, coordinated with M1.5's single restart if they land together) | **done** 2026-09-10 — merged into local `dev` (unpushed, undeployed): built with `MAX_SOURCES = 150`, `FETCH_POOL_SIZE = 8`, `BUFFER_CAP = 1500`, `ONE_PER_TICK_HOSTS` round-robin for Reddit's one-request-per-60 s ceiling, and the reader (`feed-watch: slow poll tick` at info past half the interval, plus an unconditional `last_tick` store row). Numbers in D-007, `inFlight` in D-008 |
+
+| M1.8 | A second digest lane with its own cursor — `[feed-watch:digest:<lane>]` reading and advancing `last_digest_at:<spaceId>:<lane>`, per-lane window and archive files, the plain marker and its cursor byte-for-byte unchanged — plus the football profile's Hebrew `tasks/roundup.md` and a README paragraph with the cron and the id placeholder | [feed-watch-roundup-lane](../../archive/feed-watch-social-sources/2026-09-10-feed-watch-roundup-lane.md) | — (D-011…D-013; independent of M1.6 and M1.5 — a restart-only deploy of the snapshot). The live `tasks` row is created by the coordinator **after** the restart, then the `config.yaml` entry follows | **in-progress** — added 2026-09-10 (S2 of `docs/notes/2026-09-10-feed-watch-round.md`); id may need renumbering if M1.6's second story took M1.7 **and** M1.8 |
+
+> **M1.8 numbering, 2026-09-10.** S1 took M1.7 (`feed-watch-source-liveness`); this row stood as M1.8 when the coordinator landed both.
 
 **Checkpoint:** On the live box, from `~/whatsapp-bot`, after M1.5:
 
@@ -61,6 +66,10 @@
 M1.1 (independent; deploys by apply alone)
 M1.2 → M1.3 → M1.4 → M1.5   (M1.5 also blocked on the owner's probed channel list;
                               it carries the milestone's single service restart)
+M1.6 (independent; restart-only deploy — share M1.5's restart if they land together)
+                            → football-match-day-mode M1.7 (their D-011)
+M1.8 (independent; restart-only deploy — shares the same restart; the live
+      `tasks` row and its `config.yaml` entry come after it)
 ```
 
 ## Future milestones — named, not decomposed
@@ -72,7 +81,7 @@ decomposed at the M1 checkpoint, not before.
 
 | Candidate | What | Why it is on the list |
 |-----------|------|-----------------------|
-| Source liveness | A source whose *newest item* is older than N days is visible and, for kinds known to serve stale caches, treated as failed (the store row records `lastOkAt` for the fetch, not the last item) — `t.me/s/ILRentsTLV` serves a perfect preview and has been silent since 2023-04-12; X's syndication widget returns HTTP 200 with tweets from 2025-10 for Ornstein | A dead source reads as healthy today, and the 2026-09-06 research found the same shape on X. **Required before any source kind whose page can serve a stale body with HTTP 200 — Telegram included**; until it ships, M1.5's per-channel precondition (newest post inside 7 days) is the manual stand-in. Small; could join M1 as a sixth story at the owner's call |
+| Source liveness — **promoted to M1.7, built and merged into local `dev` 2026-09-10 (unpushed, undeployed)** ([spec](../../archive/feed-watch-social-sources/2026-09-10-feed-watch-source-liveness.md)); `newestItemAt` + the once-per-streak `source silent` / `source active again` pair at 7 days (D-009). The "treated as failed" half is **rejected**, not deferred: a quiet channel and a dead one are the same bytes, so the clause would back off every low-volume Telegram channel — D-010, which also makes M1.5's manual precondition redundant | A source whose *newest item* is older than N days is visible and, for kinds known to serve stale caches, treated as failed (the store row records `lastOkAt` for the fetch, not the last item) — `t.me/s/ILRentsTLV` serves a perfect preview and has been silent since 2023-04-12; X's syndication widget returns HTTP 200 with tweets from 2025-10 for Ornstein | A dead source reads as healthy today, and the 2026-09-06 research found the same shape on X. **Required before any source kind whose page can serve a stale body with HTTP 200 — Telegram included**; until it ships, M1.5's per-channel precondition (newest post inside 7 days) is the manual stand-in. Small; could join M1 as a sixth story at the owner's call |
 | Rung 0 — apartment sources | `realta.co.il/feed.xml` (Yad2 + Madlan + Homeless + Komo + Facebook listings as one RSS, verified 2026-09-06) and, for football, Bluesky per-account RSS — both plain `rss` entries. Two measurements first: the minutes Realta's 50-item window covers on a weekday, and which Bluesky handles are live | Zero code; Realta alone answers most of what Yad2 and Madlan were "out of scope" for. Bluesky handles can join M1.1's candidate list |
 | Rung 1b — page watcher | `{"type":"page","url":…,"selector":…}`: poll a server-rendered page, diff its link list against the seen-set; `seenAt` is the clock when the page has no per-item date | homeless.co.il (per-row date) and komo.co.il (no date) are readable — but both are inside Realta's feed, so this rung is now a fallback for the day Realta goes dark |
 | Rung 1c — notification-email bridge | A `sources` kind that reads a mailbox folder (IMAP via the `yahoo-mail` code path, or Gmail via `gws`) and yields one item per Facebook "All Posts" notification and per **Yad2 fast alert** (Yad2 states e-mail within 15 minutes for a saved search) | The zero-risk read of Facebook groups the account is a member of (owner's 2026-09-05 research) — new posts only, digest bundling is a known gap, so it is the *floor*, not the Facebook path — and the sanctioned, complete Yad2 read. Needs the non-HTTP dispatch M1.2 leaves open. Read one raw Yad2 alert mail before decomposing |