catence 0.2.1-beta.2 → 0.2.1-beta.21

package/docs/deployment/local-mcp.md

@@ -81,14 +81,19 @@ catence
 ```
 
 An agent first calls `list_athletes`, then includes `athleteId` with every
-data tool. This is intentional: a shared agent may access the stores you
-configured, but no tool implicitly aggregates or crosses between athletes.
+data tool. Tools act only on the named athlete store and never merge athletes
+implicitly; the MCP server does not authenticate callers — it trusts the
+`athleteId` each call names.
 
 ## Multi-athlete catalog
 
-Catence is designed to serve **several isolated athlete stores** from one
-shared server. Each store has its own DuckDB, raw data, and tokens, so
-athletes never leak into each other.
+Catence serves several athlete stores from one server. Each store has its own
+DuckDB, raw data, and tokens; a call acts on the single store it names and
+never merges athletes implicitly. That is a tool contract, not an access
+boundary: the stdio server does not authenticate callers and trusts the
+`athleteId` they pass — over Streamable HTTP the bearer gate, per-account
+keys, or OAuth authenticate and scope callers; see
+[MCP clients and HTTP](#streamable-http-server).
 
 ### MCP behavior
 
@@ -104,7 +109,9 @@ athletes never leak into each other.
   Strava hydration tools: it spawns a detached `catence-data sync` child for
   the selected athlete and returns immediately with the run handle and log
   file. It refuses to start while another sync run is active (`sync_in_progress`);
-  follow the run with `catence_sync_progress`.
+  follow the run with `catence_sync_progress`. Pass `backfill: true` with a
+  `from` date to spawn `catence-data backfill` instead — history is re-imported
+  backwards from that date without advancing the incremental sync cursors.
 
 ### Adding another athlete
 
@@ -299,7 +306,60 @@ catence serve --host 127.0.0.1 --port 8787
 By default it is loopback-only. `CATENCE_HTTP_HOST` and `CATENCE_HTTP_PORT`
 override the host and port when the flags are omitted.
 
+#### Bearer-token authentication
+
+The server has no authentication of its own; when it must be reachable beyond
+loopback (for example through a Cloudflare Tunnel to an MCP client such as
+Mistral Work), switch on the bearer-token gate. The gate is armed at a token
+file — `<catalog-home>/mcp-token` by default, or the path given by
+`--auth-token-file` (environment equivalent: `CATENCE_MCP_TOKEN_FILE`) — and
+the **file's presence is the switch**: create it to enable, remove it to
+disable, no restart needed. The gate is also armed when the per-account key
+registry holds at least one active key or when OAuth is enabled (see the next
+sections); only with no token file, no keys, and no OAuth configured is the
+server open.
+
+Catence generates the token itself; never hand-craft one:
+
+```sh
+catence-data mcp status    # enabled, token file, endpoint URLs
+catence-data mcp enable    # generate the token file (mode 0600) if missing
+catence-data mcp token      # print the token for the client credential flow
+catence-data mcp disable   # remove the file; remote access is refused again
+```
+
+- Every endpoint except `/health` and `/api/v1/health` (kept open for readiness
+  probes) answers `401` without an exact `Authorization: Bearer <token>`
+  header while the gate is on. A request with no token gets no
+  `WWW-Authenticate` at all, so MCP clients with auth auto-detection (Mistral
+  Work) classify the server as bearer-auth rather than OAuth; with OAuth
+  enabled a token-less `401` instead gains a
+  `WWW-Authenticate: Bearer resource_metadata="<public-url>/.well-known/oauth-protected-resource/mcp"`
+  challenge — see [OAuth 2.1 (ChatGPT/Codex)](#oauth-21-chatgptcodex). A request
+  that presents a wrong or expired token gets
+  `WWW-Authenticate: Bearer error="invalid_token", error_description="The access token is invalid or expired."`
+  with no `resource_metadata`: the client already holds a credential, so it is
+  told the token is bad instead of being steered into the OAuth flow.
+- CORS preflights (`OPTIONS`) pass through without a token.
+- An existing but unreadable or empty token file fails closed: the gate
+  rejects everything except the health probes until the file is fixed or
+  removed.
+- `GET /api/v1/mcp-exposure` reports the same status the Console Settings page
+  shows (including the token); `POST /api/v1/mcp-exposure` with
+  `{"enabled": true|false}` toggles the gate. The Console exposes both to
+  admins only.
+- Rotation: `catence-data mcp disable && catence-data mcp enable` (or rewrite
+  the file), then re-authenticate the MCP client. Keep the token out of
+  `.env`, `config.json`, shell history, and repositories.
+- The Console attaches the same header automatically to its runtime calls and
+  reads the token file lazily, so enabling the gate from the Console or CLI
+  needs no Console restart either.
+
 - `GET /api/v1/athletes` returns IDs and labels only.
+- `POST /api/v1/athletes` creates a catalog athlete from
+  `{"id": "sam", "label": "Sam", "setDefault": false}` and answers with the
+  updated roster; duplicate ids are refused with `409 athlete_exists`, and
+  single-store runtimes without a catalog with `400 catalog_required`.
 - `GET /api/v1/dashboard` requires `athleteId`, for example
   `http://127.0.0.1:8787/api/v1/dashboard?athleteId=alex&days=28` (`endDate`
   is an optional `YYYY-MM-DD`; `days` ranges 1-90 and defaults to 28).
@@ -309,7 +369,11 @@ override the host and port when the flags are omitted.
   returns `202` with the run handle immediately; an already-active run is
   refused with `409 sync_in_progress`. Pass `"refreshModels": true` to also
   refresh OpenCode Go model profiles first (failure is a non-fatal warning in
-  the response).
+  the response). Pass `"backfill": true` with a required `"from"` date
+  (`YYYY-MM-DD`) to run `catence-data backfill` instead: history is re-imported
+  backwards from that date without advancing the incremental cursors. The
+  Console's Status tab exposes the same control (date picker + "Backfill
+  history").
 - `GET /api/v1/sync/status?athleteId=alex` reports live progress heartbeats,
   recent runs, and best-effort last-completion timestamps per provider. It
   keeps working while a sync holds the write lock (`lastSync` becomes `null`
@@ -323,7 +387,122 @@ override the host and port when the flags are omitted.
   packaged Console instead proxies the dashboard through its authenticated
   same-origin route.
 
-Connect HTTP-based clients to `http://127.0.0.1:8787/mcp`. There is **no
-authentication on the MCP server itself** — it trusts the caller's `athleteId`
-parameter — so keep it on loopback or restrict it at the network layer. For
-remote access over SSH: `ssh -L 8787:127.0.0.1:8787 server`.
+#### Per-account API keys
+
+The shared bearer token is a single admin credential. For per-account access,
+create API keys bound to Console accounts (`console/accounts.json`). A key acts
+as its account: an admin account's key sees every athlete, a member account's
+key sees only that account's granted athletes.
+
+The key registry lives at `<catalog-home>/mcp-keys.json` — override with
+`--mcp-keys-file` or `CATENCE_MCP_KEYS_FILE`. In the Docker deployment the
+default is `/data/mcp-keys.json` inside the data volume: runtime-managed, no
+host mount needed. The file is written atomically with mode 0600 and stores
+only SHA-256 token hashes. Any active key in the registry arms the gate,
+exactly like the token file's presence does.
+
+```sh
+catence-data mcp status                                    # gate, token, key counts, OAuth state
+catence-data mcp key add --account coach --label "vibe"   # prints the raw token exactly once
+catence-data mcp key list                                  # ids, labels, accounts — never hashes
+catence-data mcp key roll <id>                             # new raw token, printed exactly once
+catence-data mcp key revoke <id>                           # stops working immediately
+```
+
+All four accept `--file <path>` for a non-default registry. Every Console
+account manages its own keys in Settings → API keys (create with a label — the
+raw token is shown once — plus roll and revoke); admins additionally see the
+whole registry in Settings → MCP Exposure → API keys with an account picker.
+The HTTP API `GET`/`POST`/`DELETE /api/v1/mcp-keys` and `POST
+/api/v1/mcp-keys/roll` enforce the same boundary for direct callers and for
+key- or OAuth-authenticated sessions: an admin principal may manage any
+account's keys, any other principal only its own (`403 admin_required`
+otherwise), and a non-admin's list shows only its own account's keys.
+
+Scope semantics:
+
+- A key is bound to a Console account; its athlete scope is the account's
+  grant list, resolved live from `console/accounts.json` on every request.
+- Scope is enforced in the runtime for both MCP tools and the HTTP APIs: an
+  out-of-scope `athleteId` is refused with `403 athlete_forbidden` over HTTP
+  (`athlete_forbidden` tool error over MCP), and `list_athletes`,
+  `catence://athletes`, and `GET /api/v1/athletes` are filtered to the scope.
+- Admin-only APIs (`POST /api/v1/athletes`, `/api/v1/mcp-exposure`,
+  `/api/v1/models/discover`) require an admin principal: the legacy token is
+  always admin, an account key is admin only when its account is. The key
+  registry APIs are account-scoped instead of admin-only — admins manage every
+  account's keys, any other principal only its own (see above).
+- Fail closed: a revoked key, a deleted account, or an unreadable registry or
+  accounts file rejects the request (`401`) — never an anonymous fallback.
+- The raw token is shown exactly once (on `add` and `roll`); only its hash is
+  stored. Keep it in a password manager, out of `.env`, `config.json`, shell
+  history, and repositories.
+- `mcp disable` only closes the shared-token path: while keys or OAuth are
+  configured, exposure stays armed through them.
+
+#### OAuth 2.1 (ChatGPT/Codex)
+
+`catence serve --oauth --public-url <url>` runs an OAuth 2.1 authorization
+server inside the runtime so ChatGPT and Codex can connect with their native
+OAuth flow. The environment equivalents are `CATENCE_OAUTH_ENABLED=1|true` and
+`CATENCE_PUBLIC_URL=<url>`.
+
+From the Console, open Settings → MCP exposure → **OAuth 2.1 sign-in**: set the
+public URL and flip the switch. The Console writes
+`<catalog-home>/oauth/settings.json`, which the runtime reads on every request —
+the change applies immediately, without a restart. Once a valid settings file
+exists it alone decides whether OAuth is enabled and overrides the
+`CATENCE_OAUTH_ENABLED`/`CATENCE_PUBLIC_URL` (or `--oauth`/`--public-url`)
+startup values; a malformed file fails closed to disabled until the Console
+saves a valid one. `catence-data mcp status` reports the effective state and
+its `source` (`console`, `environment`, or `disabled`).
+
+- The issuer is the public URL: it must be an absolute `https` URL with no
+  query string or fragment (typically the tunnel URL), and it is required when
+  OAuth is enabled — the server refuses to start without it (startup values)
+  and the Console write rejects it (settings values).
+- Clients discover the server through RFC 8414/RFC 9728 metadata served at
+  `/.well-known/oauth-authorization-server` and
+  `/.well-known/oauth-protected-resource[/mcp]`. Dynamic client registration
+  (`POST /oauth/register`, RFC 7591) and client ID metadata documents (CIMD)
+  are enabled; PKCE with S256 is enforced.
+- The browser consent page lives at `<public-url>/oauth/consent`: it logs in
+  with a Console account (username + password, verified against
+  `console/accounts.json`) and asks to approve or deny the client. The access
+  token carries the account username, and the athlete scope is that account's
+  live grants — scoped exactly like an account key.
+- While OAuth is enabled, the `401` for a request with no bearer token gains
+  an RFC 9728 challenge —
+  `WWW-Authenticate: Bearer resource_metadata="<public-url>/.well-known/oauth-protected-resource/mcp"`
+  — so OAuth clients can discover the authorization server. A presented but
+  invalid token gets `invalid_token` without the challenge instead. Without
+  `--oauth` the `401` stays plain on purpose (Mistral connector
+  auto-detection).
+- Clients that register (DCR) or publish a client ID metadata document with
+  `token_endpoint_auth_method: "private_key_jwt"` — OpenAI's ChatGPT/Codex
+  connector does — are accepted as public clients: the declaration is
+  downgraded to `"none"`, no client secret is issued, and PKCE with S256
+  still protects the flow. Catence cannot verify client JWT assertions, so
+  the method is never honored as such.
+- Authorization-server state (registered clients, authorization codes, tokens)
+  lives in `<catalog-home>/oauth/` (directory mode 0700, files 0600).
+
+Note: the `resource_metadata` challenge on token-less `401`s can flip a client
+with auth auto-detection (Mistral Work) from bearer to OAuth — smoke-test the
+Mistral connector after enabling OAuth. A client that already sends a key
+never sees the challenge; it only gets `invalid_token` when the key is wrong.
+
+#### Client matrix
+
+| Client | Setup | Credential |
+| --- | --- | --- |
+| Mistral Vibe (Work custom connector) | Connectors → Add Connector → Custom MCP Connector, URL `https://<host>/mcp` | Bearer credential (the shared token or a per-account API key) |
+| Claude (custom connector) | Authentication: "No sign in"; Request headers: `Authorization: Bearer <key>` | Per-account API key |
+| Claude Code (CLI) | `claude mcp add --transport http --header "Authorization: Bearer <key>" <url>` | Per-account API key |
+| ChatGPT / Codex | Connector OAuth connect flow | OAuth 2.1 (Console: MCP exposure → OAuth 2.1 sign-in, or `--oauth`) |
+
+Connect HTTP-based clients to `http://127.0.0.1:8787/mcp`. Without the bearer
+gate, per-account keys, or OAuth there is **no authentication on the MCP
+server itself** — it trusts the caller's `athleteId` parameter — so keep it on
+loopback or restrict it at the network layer. For remote access over SSH:
+`ssh -L 8787:127.0.0.1:8787 server`.

package/docs/development.md

@@ -321,9 +321,11 @@ Recommended dev defaults:
 - The deployed scaffold's `sync.sh` / `hash-password.sh` / `doctor.sh` helpers are generated by `scripts/deploy-console.sh`; the dev container does not generate them — use the `npm run` / `uv run` equivalents above, or run the same `catence-data --home /data` commands via `docker compose exec`.
 - If the Console settings panel shows no Model dropdown, the data home's `config.json` has no `console.profiles` — re-copy `config.example.json` or re-run the discovery script, then restart the Console.
 
-### Status vs Dashboard
+### Settings, Status, and Dashboard
 
-The **Dashboard** (`/dashboard`) is now a pure analytics view — it renders charts and recent activities from `GET /api/v1/dashboard` and, when a detached sync holds the DuckDB write lock, shows a non-blocking amber banner (“Data temporarily unavailable — a sync is running. View live progress on the Status page.”) instead of a full-page `dashboard_unavailable` error. The **Status** page (`/status`) is the lock-safe control plane: it reads `GET /api/v1/health` (runtime `catenceRuntimeHealth()`), `GET /api/v1/sync/status` (sidecar heartbeats + `lastSync`), and `GET /api/v1/athletes` (roster), so it never needs a successful `dashboardSnapshot` and never 400s under lock. The interactive **Sync** button (with instant grey-out via `startingRef`) lives only on Status; Dashboard keeps a read-only “Syncing…” pill and a “Sync moved to Status →” banner.
+Top-level navigation is **Chat**, **Dashboard**, and **Settings**; the chat-history sidebar only renders on Chat routes, so the other pages use the full width. **Settings** (`/settings`) is a single page whose tabs — Profile, Status, and Models for every account, plus Athletes and Accounts for admins — are deep-linkable as `/settings?tab=<tab>`; asking for a tab the account cannot see falls back to the first visible one.
+
+The **Dashboard** (`/dashboard`) is a pure analytics view — it renders charts and recent activities from `GET /api/v1/dashboard` and, when a detached sync holds the DuckDB write lock, shows a non-blocking amber banner (“Data temporarily unavailable — a sync is running. View live progress in Settings → Status.”) instead of a full-page `dashboard_unavailable` error. **Settings → Status** (`/settings?tab=status`) is the lock-safe control plane: it reads `GET /api/v1/health` (runtime `catenceRuntimeHealth()`), `GET /api/v1/sync/status` (sidecar heartbeats + `lastSync`), and `GET /api/v1/athletes` (roster), so it never needs a successful `dashboardSnapshot` and never 400s under lock. The interactive **Sync** button (with instant grey-out via `startingRef`) lives only there; Dashboard keeps a read-only “Syncing…” pill and a “Sync moved to Settings → Status” banner.
 
 ## Troubleshooting
 
@@ -334,4 +336,4 @@ The **Dashboard** (`/dashboard`) is now a pure analytics view — it renders cha
   1. Runtime status API: `curl -s "http://127.0.0.1:8787/api/v1/sync/status?athleteId=<id>" | jq '.progress.running'` (through the Console instead: `/api/v1/sync/status` behind your login).
   2. Detached-child log files: `ls -t "$CATENCE_HOME/logs" | head`. A successful `POST /api/v1/sync` creates `logs/sync-<athlete>-<stamp>.log` before spawning, so an absent or empty `logs/` directory proves no detached sync ever started.
   3. The MCP progress tools (`catence_sync_progress`) read the same heartbeats.
-- **Models page clutter** — unused shipped profiles can be hidden per device from the Models page (`Hide` on the profile card); hidden profiles disappear from the chat Model dropdown but stay configured, and are restored with `Unhide`. Hiding lives in the Console database, never in `config.json`.
+- **Models page clutter** — unused shipped profiles can be hidden per device from **Settings → Models** (`Hide` on the profile card); hidden profiles disappear from the chat Model dropdown but stay configured, and are restored with `Unhide`. Hiding lives in the Console database, never in `config.json`.

package/docs/llm-providers.md

@@ -203,10 +203,10 @@ of the live catalog are pruned from the managed profiles, so the Console never
 offers a route the gateway no longer serves; if a pruned model was the
 profile's default, discovery selects a current one instead.
 
-You do not have to run the script manually: the Console dashboard's **Sync
-data** button (and `POST /api/v1/sync` with `"refreshModels": true`) refreshes
-these profiles from the live catalog before starting a data sync, and the
-Models page's **Discover OpenCode Go models** button (or
+You do not have to run the script manually: the **Sync data** button in
+**Settings → Status** (and `POST /api/v1/sync` with `"refreshModels": true`)
+refreshes these profiles from the live catalog before starting a data sync, and
+the **Discover OpenCode Go models** button in **Settings → Models** (or
 `POST /api/v1/models/discover`) refreshes them without syncing anything.
 Discovery failures — offline machine, catalog unreachable — are reported as a
 warning and never block or fail the sync itself.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "catence",
-  "version": "0.2.1-beta.2",
+  "version": "0.2.1-beta.21",
   "description": "Local MCP access to normalized Garmin, Intervals.icu, and targeted Strava fitness data.",
   "license": "AGPL-3.0-or-later",
   "repository": {
@@ -55,6 +55,7 @@
     "test:watch": "vitest",
     "catence-data": "tsx src/interfaces/cli/main.ts",
     "mcp": "tsx src/interfaces/mcp/main.ts",
+    "dev": "node scripts/dev.mjs",
     "console": "uv run --project console catence-console serve",
     "console:doctor": "uv run --project console catence-console doctor",
     "discover:opencode-go": "node scripts/discover-opencode-go.mjs",
@@ -70,20 +71,26 @@
   "dependencies": {
     "@duckdb/node-api": "^1.4.4-r.1",
     "@modelcontextprotocol/sdk": "^1.30.0",
+    "bcryptjs": "3.0.3",
     "commander": "^14.0.0",
     "dotenv": "^17.2.2",
+    "express": "5.3.0",
+    "express-rate-limit": "8.7.1",
     "intervals-icu": "^2.2.0",
+    "mcp-oauth-server": "1.0.1",
     "zod": "^4.1.5"
   },
   "devDependencies": {
+    "@types/bcryptjs": "2.4.6",
+    "@types/express": "5.0.6",
     "@types/node": "^24.3.0",
     "tsx": "^4.20.5",
     "typescript": "^5.9.2",
-    "vitest": "^3.2.4"
+    "vitest": "^5.0.0"
   },
   "overrides": {
-    "fast-uri": "3.1.7",
-    "hono": "4.12.34",
+    "fast-uri": "3.1.8",
+    "hono": "4.13.7",
     "nanoid": "3.3.18",
     "qs": "6.16.0"
   }

package/pyproject.toml

@@ -15,7 +15,7 @@ dependencies = [
 
 [tool.uv.sources]
 # fork commit
-garminconnect = { git = "https://github.com/rifusaki/python-garminconnect.git", rev = "cd4e3d9ef81a93681950138e08f57af226fd5735" }
+garminconnect = { git = "https://github.com/rifusaki/python-garminconnect.git", rev = "38b3d5003ae348de384f679976d0527316775f94" }
 
 [tool.uv]
 package = false

package/python/catence/providers/garmin/cli.py

@@ -8,6 +8,7 @@ import signal
 import sys
 from datetime import date
 from pathlib import Path
+from typing import Any
 
 from .progress import ProgressReporter
 from .staging import StagingWriter
@@ -40,7 +41,31 @@ def parse_args() -> argparse.Namespace:
         type=date.fromisoformat,
         help="Fetch calendar events from this date instead of the default short lookback.",
     )
-    return parser.parse_args()
+    parser.add_argument(
+        "--concurrency",
+        type=int,
+        default=5,
+        help="Maximum in-flight Garmin requests (1 disables request parallelization).",
+    )
+    args = parser.parse_args()
+    if args.concurrency < 1:
+        parser.error("--concurrency must be at least 1")
+    return args
+
+
+def _print_run_summary(run_id: str, worker: GarminStagingWorker, reporter: ProgressReporter) -> None:
+    """Emit one stderr record with per-stage timings and capture counts.
+
+    The management layer surfaces stderr in the sync log, so quiet-day runs
+    carry their own performance profile without an extra instrumentation pass.
+    """
+    record: dict[str, Any] = {
+        "kind": "stage_timings",
+        "runId": run_id,
+        "timings": {name: round(seconds, 2) for name, seconds in reporter.stage_summary().items()},
+    }
+    record.update(worker.capture_summary())
+    print(json.dumps(record, separators=(",", ":")), file=sys.stderr, flush=True)
 
 
 def main() -> None:
@@ -50,12 +75,32 @@ def main() -> None:
     email = os.environ.get("GARMIN_EMAIL")
     if not email:
         raise SystemExit("GARMIN_EMAIL is required.")
-    password = os.environ.get("GARMIN_PASSWORD") or getpass.getpass("Garmin password: ")
-    writer = StagingWriter(args.data_dir, args.output)
-    writer.manifest(args.run_id, args.from_date.isoformat())
     token_store = args.data_dir / "auth" / "garmin_tokens.json"
     token_store.parent.mkdir(parents=True, exist_ok=True)
-    client = Garmin(email, password, prompt_mfa=lambda: input("Garmin MFA code: "))
+    if not sys.stdin.isatty():
+        # Unattended auto-sync must never hang on a hidden prompt: fail fast
+        # so the run records a visible error instead.
+        password = os.environ.get("GARMIN_PASSWORD")
+        if not password and not token_store.exists():
+            raise SystemExit(
+                "Garmin login is required (no stored token, no GARMIN_PASSWORD, "
+                "and stdin is not interactive). Run an interactive sync once to "
+                "store the session token, or set GARMIN_PASSWORD."
+            )
+    else:
+        password = os.environ.get("GARMIN_PASSWORD") or getpass.getpass("Garmin password: ")
+
+    def prompt_mfa() -> str:
+        if not sys.stdin.isatty():
+            raise SystemExit(
+                "Garmin MFA is required but stdin is not interactive; the "
+                "session token expired. Run an interactive sync once to renew it."
+            )
+        return input("Garmin MFA code: ")
+
+    writer = StagingWriter(args.data_dir, args.output)
+    writer.manifest(args.run_id, args.from_date.isoformat())
+    client = Garmin(email, password, prompt_mfa=prompt_mfa)
     client.login(str(token_store))
     reporter = ProgressReporter(args.run_id)
     reporter.set_stage("login")
@@ -65,7 +110,7 @@ def main() -> None:
         loaded = json.loads(args.known_activities.read_text(encoding="utf-8"))
         if isinstance(loaded, dict):
             known_hashes = {str(key): str(value) for key, value in loaded.items()}
-    worker = GarminStagingWorker(client, writer, args.data_dir, known_hashes, progress=reporter)
+    worker = GarminStagingWorker(client, writer, args.data_dir, known_hashes, progress=reporter, concurrency=args.concurrency)
     interrupt_signal: int | None = None
 
     def _request_interrupt(signum: int, _frame: object) -> None:
@@ -89,7 +134,10 @@ def main() -> None:
     except WorkerInterrupted:
         reporter.finish("interrupted")
         raise SystemExit(130 if interrupt_signal == signal.SIGINT else 143)
-    reporter.finish("completed")
+    else:
+        reporter.finish("completed")
+    finally:
+        _print_run_summary(args.run_id, worker, reporter)
 
 
 if __name__ == "__main__":

package/python/catence/providers/garmin/progress.py

@@ -2,19 +2,26 @@
 
 Emits single-line JSON progress records (kind: 'progress') on stdout so the
 Node management layer can persist heartbeats without parsing the staging
-JSONL consumed by the importer.
+JSONL consumed by the importer. Captures run on several threads once sync
+concurrency is enabled, so every state mutation happens under a lock while
+the actual emit stays outside it.
+
+The reporter also accumulates per-stage wall-clock timings; the CLI prints
+them to stderr as a single `stage_timings` record so each sync run carries
+its own phase profile.
 """
 
 from __future__ import annotations
 
 import json
+import threading
 import time
 from datetime import datetime, timezone
 from typing import Any, Callable
 
 
 class ProgressReporter:
-    """Throttled progress publisher writing NDJSON records to stdout."""
+    """Throttled, thread-safe progress publisher writing NDJSON records to stdout."""
 
     def __init__(
         self,
@@ -27,59 +34,88 @@ class ProgressReporter:
         self.provider = provider
         self.emit = emit
         self.interval_seconds = interval_seconds
+        self._lock = threading.Lock()
         self._started = time.monotonic()
         self._last_emitted = 0.0
+        self._stage_started = self._started
         self.stage = "starting"
         self.current_step: str | None = None
         self.completed_units = 0
         self.total_units: int | None = None
+        self.stage_timings: dict[str, float] = {}
 
     def set_stage(self, stage: str) -> None:
         """Switch phases, resetting phase-local counters and forcing a publish."""
-        if stage != self.stage:
+        with self._lock:
+            if stage == self.stage:
+                return
+            self._close_stage_locked()
             self.stage = stage
             self.current_step = None
             self.completed_units = 0
             self.total_units = None
-            self.publish(force=True)
+        self.publish(force=True)
 
     def advance(self, completed: int | None = None, total: int | None = None, step: str | None = None) -> None:
-        if completed is not None:
-            self.completed_units = completed
-        if total is not None:
-            self.total_units = total
-        if step is not None:
-            self.current_step = step
+        with self._lock:
+            if completed is not None:
+                self.completed_units = completed
+            if total is not None:
+                self.total_units = total
+            if step is not None:
+                self.current_step = step
+        self.publish()
+
+    def add_completed(self, amount: int = 1) -> None:
+        """Increment the completed-unit counter without replacing it."""
+        with self._lock:
+            self.completed_units += amount
         self.publish()
 
     def publish(self, force: bool = False) -> None:
         now = time.monotonic()
-        if not force and now - self._last_emitted < self.interval_seconds:
-            return
-        self._last_emitted = now
-        elapsed = now - self._started
-        percent = 0.0
-        eta: float | None = None
-        if self.total_units and self.total_units > 0:
-            percent = min(100.0, max(0.0, self.completed_units / self.total_units * 100.0))
-            if self.completed_units > 0 and self.completed_units < self.total_units:
-                rate = elapsed / self.completed_units
-                eta = rate * (self.total_units - self.completed_units)
-        record = {
-            "kind": "progress",
-            "runId": self.run_id,
-            "provider": self.provider,
-            "stage": self.stage,
-            "currentStep": self.current_step,
-            "completedUnits": self.completed_units,
-            "totalUnits": self.total_units,
-            "percentComplete": round(percent, 2),
-            "elapsedSeconds": round(elapsed, 1),
-            "estimatedRemainingSeconds": round(eta, 1) if eta is not None else None,
-            "heartbeatAt": datetime.now(timezone.utc).isoformat(),
-        }
+        with self._lock:
+            if not force and now - self._last_emitted < self.interval_seconds:
+                return
+            self._last_emitted = now
+            elapsed = now - self._started
+            percent = 0.0
+            eta: float | None = None
+            if self.total_units and self.total_units > 0:
+                percent = min(100.0, max(0.0, self.completed_units / self.total_units * 100.0))
+                if 0 < self.completed_units < self.total_units:
+                    rate = elapsed / self.completed_units
+                    eta = rate * (self.total_units - self.completed_units)
+            record: dict[str, Any] = {
+                "kind": "progress",
+                "runId": self.run_id,
+                "provider": self.provider,
+                "stage": self.stage,
+                "currentStep": self.current_step,
+                "completedUnits": self.completed_units,
+                "totalUnits": self.total_units,
+                "percentComplete": round(percent, 2),
+                "elapsedSeconds": round(elapsed, 1),
+                "estimatedRemainingSeconds": round(eta, 1) if eta is not None else None,
+                "heartbeatAt": datetime.now(timezone.utc).isoformat(),
+            }
+        # Emitting outside the lock keeps a slow pipe from stalling captures.
         self.emit(json.dumps(record, separators=(",", ":")))
 
+    def stage_summary(self) -> dict[str, float]:
+        """Per-stage elapsed seconds, including the still-open stage."""
+        with self._lock:
+            summary = dict(self.stage_timings)
+            summary[self.stage] = summary.get(self.stage, 0.0) + (time.monotonic() - self._stage_started)
+        return summary
+
     def finish(self, stage: str = "completed") -> None:
-        self.stage = stage
+        with self._lock:
+            self._close_stage_locked()
+            self.stage = stage
+            self._stage_started = time.monotonic()
         self.publish(force=True)
+
+    def _close_stage_locked(self) -> None:
+        self.stage_timings[self.stage] = self.stage_timings.get(self.stage, 0.0) + (time.monotonic() - self._stage_started)
+        self._stage_started = time.monotonic()

package/python/catence/providers/garmin/staging.py

@@ -3,10 +3,12 @@ from __future__ import annotations
 import hashlib
 import json
 import os
+import threading
 from dataclasses import dataclass
 from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any
+from uuid import uuid4
 
 from . import STAGING_SCHEMA_VERSION
 from .contracts import validate_record
@@ -28,12 +30,15 @@ class StagingWriter:
     def __post_init__(self) -> None:
         self.output.parent.mkdir(parents=True, exist_ok=True)
         self.output.touch(exist_ok=True)
+        self._lock = threading.Lock()
 
     def emit(self, record: dict[str, Any]) -> None:
         record["schemaVersion"] = STAGING_SCHEMA_VERSION
-        with self.output.open("a", encoding="utf-8") as file:
-            file.write(json.dumps(validate_record(record), separators=(",", ":"), default=str))
-            file.write("\n")
+        line = json.dumps(validate_record(record), separators=(",", ":"), default=str) + "\n"
+        # Sync runs fan capture out across threads; serialize whole lines so the
+        # JSONL never interleaves, even though each write still opens the file.
+        with self._lock, self.output.open("a", encoding="utf-8") as file:
+            file.write(line)
 
     def manifest(self, run_id: str, from_date: str) -> None:
         self.emit({
@@ -54,7 +59,9 @@ class StagingWriter:
         destination = self.data_dir / relative
         destination.parent.mkdir(parents=True, exist_ok=True)
         if not destination.exists():
-            temporary = destination.with_suffix(f".{extension}.tmp")
+            # Unique same-directory temp name: concurrent writers can race on
+            # the same content hash, and a shared temp path would clobber.
+            temporary = destination.with_name(f".{destination.name}.{os.getpid()}.{uuid4().hex}.tmp")
             temporary.write_bytes(contents)
             os.replace(temporary, destination)
         self.emit({