ctrlrelay 0.9.0__tar.gz → 0.10.0__tar.gz

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/.github/workflows/publish.yml

@@ -12,9 +12,17 @@ name: Publish to PyPI
 #   Workflow:    publish.yml
 #   Environment: pypi
 #
-# The `pypi` environment adds a manual approval gate — releases pause
-# at the "Publish" step until a repo maintainer approves, so a
-# compromised tag alone can't silently push to PyPI.
+# Publishing is intentionally automatic: tag, publish the GitHub
+# release, and this ships to PyPI with no human step. The `pypi`
+# environment scopes the OIDC trust to this workflow — it is NOT an
+# approval gate and carries no protection rules. Anything that can
+# publish a release here can publish to PyPI.
+#
+# (This comment previously claimed the environment paused for a
+# maintainer. It never did, and someone reading it planned a release
+# around a checkpoint that was not there. If a gate is ever wanted,
+# it is a required reviewer on the `pypi` environment in repo
+# settings — not something this file can assert.)
 
 on:
   release:

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/CHANGELOG.md

@@ -7,6 +7,145 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.10.0] - 2026-09-08
+
+### Added
+
+- **The dispatcher now logs.** `core/dispatcher.py` is the module that
+  spawns and supervises every agent session, and it was the only core
+  module with no structured logging at all — so the one place where a
+  session is born, times out or dies was the one place with no trail.
+  It now emits `dispatcher.session.start` / `.finished` / `.timeout` /
+  `.subprocess_failed` / `.spawn_failed` / `.cancelled`, plus
+  `dispatcher.checkpoint.missing` when a session ends without
+  signalling and `dispatcher.checkpoint.read_failed` when the file
+  won't parse. Every failure event carries `error_type`, so a bare
+  `asyncio.TimeoutError` — whose `str()` is empty — is still
+  identifiable in the log. Prompts and agent output are never logged in
+  plaintext: only a `hash_text()` prefix and a length. See
+  [Tracing one agent session](https://ainvirion.github.io/ctrlrelay/operations/#tracing-one-agent-session).
+- **`dispatcher.binary.fallback` / `.unresolved` warn when `claude`
+  isn't on `PATH`.** Under systemd and launchd the unit's `PATH` is
+  minimal, and falling through to a hard-coded path is the usual
+  precursor to every session failing to spawn. It used to happen
+  silently.
+
+### Changed
+
+- `spawn_session` takes optional `repo` and `issue_number` arguments,
+  which the dev, task and secops pipelines now pass. They are
+  observability-only — they let dispatcher events be correlated with
+  pipeline events without re-parsing the composite session id.
+
+### Fixed
+
+- **A slow post is no longer logged as a failed one.** The bridge ACKs
+  only after Telegram has accepted a question, so a post slower than the
+  transport's wait meant the transport timed out first and logged
+  `dev.question.post_failed` while the bridge logged
+  `dev.question.posted` for the same request — a false claim in the
+  opposite direction to the one being fixed. A timeout now emits
+  `dev.question.post_unknown`, because the outcome genuinely is unknown;
+  a write that actually failed keeps the definite `post_failed`.
+  Delivery is only ever confirmed by the far side's acknowledgement;
+  everything else divides into "the bytes provably never left" (a
+  definite failure) and "they may have" (unknown). A `drain()` failure
+  is unknown — `write()` had already handed the data over — and on the
+  bridge, a Telegram timeout or bare transport error is unknown too,
+  while `BadRequest`/`Forbidden`/`InvalidToken` are Telegram answering
+  "no". Note `BadRequest` subclasses `NetworkError` in
+  python-telegram-bot, so that check cannot be a plain isinstance; it
+  lives in `is_ambiguous_delivery` next to the library that defines the
+  taxonomy. `TransportTimeoutError` subclasses
+  `TransportUnknownDeliveryError` subclasses `TransportError`, so
+  existing handlers are unaffected. The bridge's ERROR response carries
+  which of the two it was (`telegram_delivery_unknown` vs
+  `telegram_api_error`) rather than letting the transport re-derive it —
+  the transport cannot see the Telegram exception, so a generic error
+  forced it to assume the worst and contradict the bridge's own record
+  for the same `request_id`.
+
+  **Known limit:** python-telegram-bot raises `TimedOut` for an HTTPX
+  connection-pool timeout too, where the request was never sent. That
+  case is currently reported as unknown rather than as the definite
+  non-delivery it is. Separating it means matching on library internals,
+  which is more fragile than the mislabelling is harmful.
+
+- **`dev.question.posted` no longer claims a delivery that never
+  happened.** The transport logged it before writing to the socket and the
+  bridge logged it before calling Telegram, so a failed write or a Telegram
+  outage still left a record saying the operator had been asked. The
+  transport now emits it on the bridge's ACK — the only signal that says
+  the question actually reached Telegram — and the bridge emits it after
+  the API accepts the message, tagged with `telegram_msg_id`. Both sides
+  emit `dev.question.post_failed` (with `reason` and a truncated `error`)
+  on the failure path instead. A question that is posted and then goes
+  unanswered still logs only `posted`: the delivery succeeded, the operator
+  just did not reply.
+- **Questions and operator answers are no longer written to logs in
+  plaintext.** Every event already carried `question_hash` / `answer_hash`,
+  and then defeated it by logging the raw text in the field next door.
+  These records go to stdout and are captured into `~/.ctrlrelay/logs` and
+  the systemd journal, so free-text operator replies — and questions that
+  now enumerate every open PR in a repo — persisted there indefinitely with
+  no retention policy tuned for that content. The hash and length remain,
+  which is enough to correlate the same question across the poller and
+  bridge logs. A test walks the source and fails on any `log_event()` call
+  that passes a raw payload field.
+- **A broken `gh` install no longer reports itself as "network
+  unavailable".** `subprocess.run` raising `OSError` means the child could
+  not be exec'd — a non-executable binary, a bad path, a wrong
+  architecture. Python does no networking here, `gh` does, so an `OSError`
+  raised on our side can never mean offline. Reporting it that way was the
+  exact misdiagnosis this classification exists to prevent: it sends the
+  operator to check their wifi while their `gh` install is broken. Only a
+  hung `gh` (timeout) still reads as connectivity — there the child
+  launched and then stopped responding.
+- **`broken pipe` now reads as a network failure.** A peer that vanished
+  mid-write carries none of the other network markers, so it fell through
+  to a generic API error — a miss of the classifier's own purpose. Bare
+  `EOF` is deliberately still not matched: `gh` passes API response
+  bodies through verbatim, so matching it would misclassify real API
+  errors as outages, the same mistake in the other direction.
+- **A resource limit is no longer blamed on the `gh` install.** `EMFILE`
+  and `ENOMEM` also reach the classifier as `OSError` from a failed spawn,
+  but answering those with "install GitHub CLI" is the same misdirection
+  wearing a different hat. Only exec-family errnos say the install is
+  broken; the rest surface the errno.
+- **Go transport failures are no longer reported as API errors.** `gh`
+  renders a failed round-trip as `Get "https://…": <cause>`, which by
+  construction means no HTTP response arrived — so `EOF`, `http2: client
+  connection lost`, `request canceled` and `remote error: tls: handshake
+  failure` were all landing as "GitHub API call failed". Matching the
+  shape rather than each wording catches the family; a real API error
+  carries `(HTTP nnn)` instead and is unaffected. `read tcp …: connection
+  timed out` (a link dropped mid-request) is matched too.
+- **TLS trust failures are no longer called transient.** An untrusted CA
+  or a skewed clock is a persistent local problem, and "retry later" can
+  never fix it. These now report as `tls_trust` with remediation pointing
+  at the CA bundle and the system clock. A certificate valid for the
+  wrong host stays a network failure — that is the captive-portal case.
+- **`gh` stderr is escaped before Rich renders it.** Stderr containing
+  bracketed text (e.g. `[/docs]`) raised `MarkupError`, replacing the
+  diagnostic with a traceback and skipping the exit — losing the raw
+  detail the line exists to show.
+- **The startup `gh` probe is time-bounded.** Without a timeout a hung
+  `gh` wedged `poller start` indefinitely, and the one exception branch
+  that really does mean connectivity could never fire.
+
+- **Startup no longer blames your GitHub auth when you are offline.**
+  `poller start` probes `gh api user` before it does anything else, and
+  every way that probe can fail — unplugged network, expired token, a
+  500 from the API — exits 1, so the CLI printed
+  `Command '[...]' returned non-zero exit status 1.` for all of them.
+  A new `core/network.py` classifies the failure from gh's own stderr
+  into network / auth / rate-limit / API-error (plus "gh not
+  installed"), and the CLI prints the matching message —
+  `Network unavailable — could not reach api.github.com` in yellow when
+  it is transient, a pointer to `gh auth status` when the credentials are the
+  problem, the HTTP status when the API answered. gh's raw stderr is
+  still printed underneath.
+
 ## [0.9.0] - 2026-09-07
 
 ### Added
@@ -1398,7 +1537,8 @@ pipeline).
   per-phase implementation plans (Phase 0 through Phase 4).
 - `docs/Claude_Code_Project_Guide.md` — project development guide.
 
-[Unreleased]: https://github.com/AInvirion/ctrlrelay/compare/v0.9.0...HEAD
+[Unreleased]: https://github.com/AInvirion/ctrlrelay/compare/v0.10.0...HEAD
+[0.10.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.9.0...v0.10.0
 [0.9.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.8.1...v0.9.0
 [0.8.1]: https://github.com/AInvirion/ctrlrelay/compare/v0.8.0...v0.8.1
 [0.8.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.7.0...v0.8.0

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ctrlrelay
-Version: 0.9.0
+Version: 0.10.0
 Summary: Local-first orchestrator for headless coding agents across multiple GitHub repos
 Project-URL: Homepage, https://github.com/AInvirion/ctrlrelay
 Project-URL: Documentation, https://ainvirion.github.io/ctrlrelay/

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/docs/operations.md

@@ -305,6 +305,57 @@ tail -f ~/.ctrlrelay/logs/poller.error.log
 The poller prints one line per detected issue and one line per pipeline outcome.
 The bridge prints connection events and Telegram API errors.
 
+### Tracing one agent session
+
+Log lines are newline-delimited JSON, and the dispatcher — the module that
+spawns and supervises every agent subprocess — tags each of its events with
+the session id, so one grep reconstructs a whole run:
+
+```bash
+grep dev-owner-repo-42- ~/.ctrlrelay/logs/poller.log | jq -c '{ts, level, event}'
+```
+
+The events it emits:
+
+| Event | Level | When |
+|---|---|---|
+| `dispatcher.session.start` | INFO | Subprocess about to launch. Carries `binary`, `timeout`, `resume`, `prompt_hash`, `prompt_len`. |
+| `dispatcher.session.finished` | INFO | Subprocess exited. Carries `exit_code`, `checkpoint_status`, `elapsed_ms`. |
+| `dispatcher.session.subprocess_failed` | ERROR | Non-zero exit, with `stderr_tail`. |
+| `dispatcher.session.timeout` | ERROR | The `timeout` seconds elapsed and the child was killed. |
+| `dispatcher.session.spawn_failed` | ERROR | The binary could not be exec'd at all. |
+| `dispatcher.session.cancelled` | INFO | Daemon shutdown killed the child mid-run. |
+| `dispatcher.checkpoint.missing` | WARNING | The session ended without writing a checkpoint. |
+| `dispatcher.checkpoint.read_failed` | ERROR | A checkpoint file existed but would not parse. |
+| `dispatcher.binary.fallback` | WARNING | `claude` was not on `PATH`; a well-known path was used instead. Expect this when the unit's `PATH` is minimal. |
+| `dispatcher.binary.unresolved` | WARNING | Not on `PATH` and not at any known path — sessions are about to start failing to spawn. |
+
+Prompts and agent output never appear in plaintext: only a short SHA-256
+prefix and a length are logged.
+
+### Blocked-question events
+
+When a session blocks on a question, the transport and the bridge each emit
+their own record:
+
+| Event | When |
+|---|---|
+| `dev.question.posted` | The question reached the operator. The transport emits it on the bridge's ACK; the bridge emits it once the Telegram API has accepted the message, and adds `telegram_msg_id`. |
+| `dev.question.post_failed` | Delivery failed. Carries `reason` (`send_failed`, `bridge_error`, or the exception class on the bridge side) and a truncated `error`. |
+| `dev.answer.received` | The operator's reply came back. |
+
+Neither the question nor the answer is written to the log. Each event carries
+`question_hash` / `question_length` (or `answer_hash` / `answer_length`)
+instead — the hash is stable, so it still joins the poller's record to the
+bridge's record for the same question:
+
+```bash
+grep 44509098fa6eeaf3 ~/.ctrlrelay/logs/*.log | jq -c '{ts, logger, event}'
+```
+
+This matters because these files and the systemd journal have no retention
+policy tuned for human-entered content, and operator replies are free text.
+
 ## Inspecting state
 
 ### `ctrlrelay status`

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ctrlrelay"
-version = "0.9.0"
+version = "0.10.0"
 description = "Local-first orchestrator for headless coding agents across multiple GitHub repos"
 readme = "README.md"
 requires-python = ">=3.12"

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/src/ctrlrelay/bridge/server.py

@@ -19,7 +19,10 @@ from ctrlrelay.bridge.protocol import (
     parse_message,
     serialize_message,
 )
-from ctrlrelay.bridge.telegram_handler import TelegramHandler
+from ctrlrelay.bridge.telegram_handler import (
+    TelegramHandler,
+    is_ambiguous_delivery,
+)
 from ctrlrelay.core.obs import get_logger, hash_text, log_event
 
 if TYPE_CHECKING:
@@ -298,6 +301,23 @@ class BridgeServer:
                 )
 
         if msg.op == BridgeOp.ASK:
+            question = msg.question or ""
+            # Hash + length only: these records land in the journal and in
+            # poller.log, so the question text itself would persist there in
+            # plaintext indefinitely. The hash still joins this event to the
+            # client's own dev.question.posted. Built outside the try so the
+            # failure path can always describe what failed.
+            post_fields = {
+                "session_id": msg.session_id,
+                "repo": msg.repo,
+                "issue_number": msg.issue_number,
+                "transport": "telegram",
+                "destination": f"telegram:chat={self.chat_id}",
+                "request_id": msg.request_id,
+                "question_length": len(question),
+                "question_hash": hash_text(question),
+                "options": msg.options,
+            }
             try:
                 assert self._telegram is not None
                 # Start the expiry clock before the Telegram round trip, not
@@ -308,21 +328,6 @@ class BridgeServer:
                 # reply-to in that window writes the answer into a dead
                 # request.
                 received_at = time.monotonic()
-                question = msg.question or ""
-                log_event(
-                    _logger,
-                    "dev.question.posted",
-                    session_id=msg.session_id,
-                    repo=msg.repo,
-                    issue_number=msg.issue_number,
-                    transport="telegram",
-                    destination=f"telegram:chat={self.chat_id}",
-                    request_id=msg.request_id,
-                    question=question,
-                    question_length=len(question),
-                    question_hash=hash_text(question),
-                    options=msg.options,
-                )
                 telegram_msg_id = await self._telegram.ask(
                     format_question(
                         question,
@@ -350,15 +355,50 @@ class BridgeServer:
                     "bridge: ASK posted request_id=%s telegram_msg_id=%s",
                     msg.request_id, telegram_msg_id,
                 )
+                # Emitted here, not before the send: until Telegram has
+                # accepted the message there is nothing posted to claim.
+                log_event(
+                    _logger,
+                    "dev.question.posted",
+                    **post_fields,
+                    telegram_msg_id=telegram_msg_id,
+                )
                 return BridgeMessage(
                     op=BridgeOp.ACK, request_id=msg.request_id, status="pending",
                 )
             except Exception as e:
                 _log.warning("bridge: ASK failed, request_id=%s err=%s", msg.request_id, e)
+                # Same rule as the transport: only claim a definite
+                # failure when Telegram actually answered. A timeout or a
+                # bare transport error can be raised after Telegram
+                # accepted the message, in which case the question IS on
+                # the operator's phone and "post_failed" is a lie.
+                #
+                # The taxonomy is not intuitive, so it lives next to the
+                # library that defines it — see is_ambiguous_delivery.
+                unknown = is_ambiguous_delivery(e)
+                log_event(
+                    _logger,
+                    "dev.question.post_unknown"
+                    if unknown
+                    else "dev.question.post_failed",
+                    **post_fields,
+                    reason=type(e).__name__,
+                    error=str(e)[:200],
+                )
                 return BridgeMessage(
                     op=BridgeOp.ERROR,
                     request_id=msg.request_id,
-                    error="telegram_api_error",
+                    # Carry the classification instead of letting the
+                    # other side re-derive it: the transport cannot see
+                    # the Telegram exception, so a generic ERROR forced
+                    # it to assume the worst and contradict the
+                    # post_unknown just logged here for the same request.
+                    error=(
+                        "telegram_delivery_unknown"
+                        if unknown
+                        else "telegram_api_error"
+                    ),
                     message=str(e),
                 )
 
@@ -473,7 +513,6 @@ class BridgeServer:
             request_id=match.request_id,
             telegram_msg_id=match.telegram_msg_id,
             reply_to_message_id=reply_to_message_id,
-            answer=text,
             answer_length=len(text),
             answer_hash=hash_text(text),
         )

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/src/ctrlrelay/bridge/telegram_handler.py

@@ -7,12 +7,31 @@ import logging
 from typing import Awaitable, Callable
 
 from telegram import Bot, ReplyKeyboardMarkup, ReplyKeyboardRemove
+from telegram.error import NetworkError, TimedOut
 
 _log = logging.getLogger(__name__)
 
 IncomingMessageHandler = Callable[[str, int | None], Awaitable[None]]
 
 
+
+def is_ambiguous_delivery(exc: BaseException) -> bool:
+    """True when a send may have reached Telegram despite raising.
+
+    Delivery is only confirmed by a response. A timeout or a bare
+    transport error can be raised *after* Telegram accepted the message,
+    so calling those a failed post would put a lie in the log while the
+    question sits on the operator's phone.
+
+    The taxonomy is not intuitive and is worth stating: `BadRequest`
+    subclasses `NetworkError` here, so `isinstance(exc, NetworkError)`
+    would sweep definite rejections into the ambiguous bucket. Only
+    `TimedOut` and a bare `NetworkError` are genuinely unknown —
+    `BadRequest`, `Forbidden`, `InvalidToken` and the rest are Telegram
+    answering "no".
+    """
+    return isinstance(exc, TimedOut) or type(exc) is NetworkError
+
 class TelegramHandler:
     """Handles Telegram Bot API communication — outbound (send/ask) and
     inbound (long-poll get_updates) for answers from the operator."""

{ctrlrelay-0.9.0 → ctrlrelay-0.10.0}/src/ctrlrelay/cli.py

@@ -5,6 +5,7 @@ from pathlib import Path
 
 import typer
 from rich.console import Console
+from rich.markup import escape as rich_escape
 from rich.table import Table
 
 from ctrlrelay import __version__
@@ -13,6 +14,10 @@ from ctrlrelay.core.github import GitHubCLI, GitHubError
 from ctrlrelay.core.pr_verifier import PRVerifier
 from ctrlrelay.core.question_expiry import expire_stale_questions
 
+# Startup probe cap. `GitHubCLI` uses 60s for ordinary calls; this one
+# gates daemon startup, so a hung gh must not wedge it indefinitely.
+_GH_PROBE_TIMEOUT_SECONDS = 60
+
 app = typer.Typer(
     name="ctrlrelay",
     help="Local-first orchestrator for Claude Code across multiple GitHub repos.",
@@ -1035,10 +1040,33 @@ def poller_start(
                 capture_output=True,
                 text=True,
                 check=True,
+                # Without this a hung gh hangs `poller start` forever, and
+                # the TimeoutExpired branch of the classifier — the one
+                # case that really does mean connectivity — can never fire.
+                timeout=_GH_PROBE_TIMEOUT_SECONDS,
             )
             username = result.stdout.strip()
-        except subprocess.CalledProcessError as e:
-            console.print(f"[red]Failed to get GitHub username:[/red] {e}")
+        except (subprocess.SubprocessError, OSError) as e:
+            # `gh` exits 1 for offline, expired-token and API-error alike,
+            # so classify from its stderr — otherwise operators debug auth
+            # when the laptop is simply offline (#32).
+            from ctrlrelay.core.network import gh_failure_from_exception
+
+            failure = gh_failure_from_exception(e)
+            color = "yellow" if failure.is_transient else "red"
+            console.print(
+                f"[{color}]Could not determine GitHub username:[/{color}] "
+                f"{failure.message}"
+            )
+            if failure.stderr:
+                # gh's stderr is arbitrary text and Rich parses [...] as
+                # markup: a stderr containing something like "[/docs]"
+                # raises MarkupError, which replaces the diagnostic with a
+                # traceback and skips the exit — losing exactly the detail
+                # this line exists to show.
+                console.print(
+                    f"[dim]gh: {rich_escape(failure.stderr)}[/dim]"
+                )
             raise typer.Exit(1)
 
         if not username: