@aitne-sh/aitne 0.1.7 → 0.1.9

package/agent-assets/docs/troubleshooting/fallback-keeps-firing.md

@@ -3,52 +3,126 @@ schema_version: 1
 slug: troubleshooting/fallback-keeps-firing
 title: Fallback Keeps Firing
 id: fallback-keeps-firing
+aliases:
+  - fallback firing
+  - backend fallback
+  - main backend down
+  - fallover loop
 category: troubleshooting
 summary: |
-  The router transitioned main → fallback for several runs in a row.
-  Almost always main backend is unhealthy or quota-exhausted.
+  The router keeps switching from your main backend to its fallback,
+  run after run. Almost always the main backend is unhealthy
+  (expired auth) or quota-exhausted. This doc shows how to confirm the
+  cause and get runs back onto the main backend.
 section: fallback-keeps-firing
+status: stable
 tags:
   - troubleshooting
   - backends
-status: stable
+  - routing
+  - health
+  - cost
+keywords:
+  - fallback
+  - BackendQuotaError
+  - BackendDecisiveFailure
+  - router fallover
+  - auth expired
+  - quota exhausted
 ask_examples:
-  - Why does the fallback keep running?
+  - Why does the fallback backend keep running?
   - How do I stop the fallover loop?
-locale: en-US
-created: 2026-04-25
-updated: 2026-04-25
+  - Why isn't my main backend being used?
+ui_anchors:
+  - /settings/models
+  - /analytics
+  - /activity
+process_keys:
+  - message.dm
+  - routine.morning_routine
+api_endpoints:
+  - GET /api/health
+  - POST /api/backends/:backendId/check-auth
+config_keys:
+  - autonomousDailyCostCapUsd
 related:
   - troubleshooting/auth-failed
   - troubleshooting/quota-exhausted
   - features/operations/backend-routing
+  - concepts/auth-health
+  - concepts/backends-and-tiers
+locale: en-US
+created: 2026-04-25
+updated: 2026-05-28
 ---
 
 # Fallback Keeps Firing
 
-## What You See
+The backend router resolves every run to a `{ main, fallback }` pair. When
+the main backend can't take the work, the router transparently switches to
+the fallback so the run still completes. If that happens once, it's normal.
+If it happens run after run, the main backend has a persistent problem.
+
+## What you see
+
+- Multiple recent Activity rows whose backend is *not* your main backend.
+- Notifications that mention a "fallback".
+
+## Most likely causes
+
+1. **Main backend's auth has expired.** The most common cause — the CLI
+   credentials (or API key) for the main backend are no longer valid, so
+   every run raises a `BackendDecisiveFailure("auth")` and the router falls
+   over.
+2. **Main backend's usage window is exhausted.** Claude and Codex meter
+   usage per rolling window; Gemini also hits its per-day free-tier cap.
+   When the limit is reached the backend raises a `BackendQuotaError` and
+   the router falls over until the window resets.
+3. **A repeating `BackendDecisiveFailure` from a config issue** — e.g. the
+   main backend's CLI isn't installed, a pinned model id no longer
+   resolves, or execution-mode settings reject the run.
+
+`BackendQuotaError` and `BackendDecisiveFailure` are the two signals the
+router watches; one of them is firing on every attempt.
+
+## Diagnose
 
-- Multiple recent Activity rows with backend != main.
-- Notifications mentioning "fallback".
+1. **Check auth on `/settings/models`.** Each backend has a card showing
+   its auth status. If the main backend's card flags expired or failed
+   auth, that's your cause — re-verify or re-authenticate it there.
+2. **Check quota/spend on `/analytics` (Cost tab).** Look at the main
+   backend's recent spend. A flat ceiling or a depleted window points to
+   cause 2. If you set `autonomousDailyCostCapUsd`, autonomous runs also
+   stop once the daily cap is hit (reactive DMs are never blocked by it).
+3. **Read the failure detail in `/activity`.** Open the offending row to
+   see the exact error. From the CLI you can do the same:
 
-## Most Likely Causes
+   ```bash
+   # recent runs on the main backend that didn't succeed
+   aitne audit --backend claude --result failed --since 24h
+   ```
 
-1. Main backend's auth has expired.
-2. Main backend's heavy-tier window is depleted.
-3. A `BackendDecisiveFailure` is repeating — usually a config issue.
+   (`--backend` accepts `claude`, `codex`, or `gemini`.)
 
-## Diagnostic Steps
+## Fix
 
-1. Open `/connections` — auth-health card.
-2. Open `/analytics` — main-backend cost / quota.
-3. Look at the failure detail in Activity for the offending row.
+- **Expired auth →** re-authenticate the main backend from its card on
+  `/settings/models`, then run a test. See [Auth Failed](auth-failed.md).
+- **Exhausted quota →** wait for the usage window to reset, raise the cap,
+  or accept the fallback until it clears. See
+  [Quota Exhausted](quota-exhausted.md).
+- **Config issue →** fix the reported problem (install the CLI, repin a
+  valid model, relax execution mode) on `/settings/models`.
 
-## Confirming the Fix
+## Confirm the fix
 
-- The next run uses the main backend.
+- The next run uses the main backend again — visible in `/activity` or via
+  `aitne audit --backend <id> --since 1h`.
 
 ## Related
 
 - [Auth Failed](auth-failed.md)
 - [Quota Exhausted](quota-exhausted.md)
 - [Backend Routing](../features/operations/backend-routing.md)
+- [Auth Health](../concepts/auth-health.md)
+- [Backends and Tiers](../concepts/backends-and-tiers.md)

package/agent-assets/docs/troubleshooting/messaging-not-pairing.md

@@ -3,22 +3,40 @@ schema_version: 1
 slug: troubleshooting/messaging-not-pairing
 title: Messaging Not Pairing
 id: messaging-not-pairing
+aliases:
+  - pairing failed
+  - magic phrase not working
+  - messaging not paired
+  - pairing card stuck waiting
 category: troubleshooting
 summary: |
-  The magic phrase isn't pairing your messaging account. Usually the
-  bot can't see DMs, the phrase expired, or the message went to a
-  group instead of a direct channel.
-section: messaging-not-pairing
+  Your messaging account won't pair: the dashboard's pairing card stays in
+  "waiting". Usually the secret expired (5-minute window), the phrase was
+  wrapped in a sentence instead of sent by itself, the bot can't see your
+  DM, or the message landed in a group instead of a direct channel.
+section: messaging
 tags:
   - troubleshooting
   - messaging
+  - pairing
 status: stable
+locale: en-US
+created: 2026-04-25
+updated: 2026-05-28
+keywords:
+  - magic phrase
+  - owner channel
+  - pairing
+  - bot token
+  - QR pairing
+  - deep link
+ui_anchors:
+  - /connections/messaging
 ask_examples:
   - Why isn't my magic phrase pairing?
   - Where do I send the magic phrase?
-locale: en-US
-created: 2026-04-25
-updated: 2026-04-25
+  - My Telegram QR isn't pairing, what's wrong?
+  - The pairing card is stuck on waiting.
 related:
   - features/messaging/pairing-and-magic-phrase
   - features/messaging/overview
@@ -26,28 +44,65 @@ related:
 
 # Messaging Not Pairing
 
-## What You See
+## What you see
+
+After you send the magic phrase (Slack / Discord) or scan/tap the QR or
+deep link (Telegram / WhatsApp), the pairing card on
+**Connections → Messaging** (`/connections/messaging`) stays in the
+"waiting" state and never flips to "paired".
+
+## Know which secret your platform uses
+
+Not every platform uses the typed phrase, so the fix depends on the
+platform:
+
+| Platform | Secret | How you send it |
+|---|---|---|
+| **Slack** | Magic phrase | DM the 4-word phrase (e.g. `apple-banana-cherry-date`) to the bot, by itself. |
+| **Discord** | Magic phrase | Same as Slack — DM the displayed phrase to the bot. |
+| **Telegram** | QR / deep link | Tap **START** so the bot receives `/start <token>`. |
+| **WhatsApp** | Device QR | Scan the dashboard QR from your phone (Linked Devices). |
 
-- After DMing the magic phrase, the dashboard's pairing card stays in
-  "waiting" state.
+All secrets are single-use and expire after **5 minutes**.
 
-## Most Likely Causes
+## Most likely causes
 
-1. Phrase expired — generate a new one.
-2. Bot does not have DM permission (Discord, Slack).
-3. DM was sent in a group / channel by mistake.
-4. Bot token wrong — paste it in again.
+1. **The secret expired.** The phrase or token is only valid for 5
+   minutes. If you took longer, it has lapsed — regenerate a fresh one.
+2. **You wrapped the phrase in a sentence (Slack / Discord).** The
+   matcher requires the four words to be the **only** content of the
+   message. Sending "my phrase is apple-banana-cherry-date" is rejected
+   by design, and the agent replies asking you to send the phrase on its
+   own. Matching otherwise ignores case, punctuation, and emoji.
+3. **You sent it in a group or channel.** Pairing — and all agent
+   messaging — works only in a one-to-one DM. Group chats are out of
+   scope.
+4. **The bot can't see your DM.** On Discord and Slack the bot needs
+   permission to receive direct messages from you; without it your
+   message never reaches the daemon.
+5. **A token is wrong.** A mistyped or stale bot token means the daemon
+   isn't connected at all. Re-paste it in the platform's card.
 
-## Diagnostic Steps
+## Diagnostic steps
 
-1. Click "Regenerate phrase" on `/connections/messaging`.
-2. Confirm bot privileges in the messaging app.
-3. Send the phrase from a direct chat with the bot.
+1. On `/connections/messaging`, click **Regenerate phrase** (Slack /
+   Discord) or re-open the QR / deep link (Telegram / WhatsApp) so you
+   start a fresh 5-minute window.
+2. For Slack / Discord, send **only** the four words — no surrounding
+   text — from a direct chat with the bot, not a channel.
+3. For Telegram, tap **START** so the bot gets `/start <token>`. For
+   WhatsApp, set the owner phone number first, then click **Pair device**
+   and scan.
+4. If the card still doesn't react at all, confirm the bot's tokens and
+   DM permissions in the messaging app, then re-paste any token on the
+   card.
 
-## Confirming the Fix
+## Confirming the fix
 
-- The pairing card shows your owner identity.
+The pairing card flips to "paired" and shows your owner identity. A DM to
+that channel now reaches the agent.
 
 ## Related
 
-- [Pairing & Magic Phrase](../features/messaging/pairing-and-magic-phrase.md)
+- [Pairing and Magic Phrase](../features/messaging/pairing-and-magic-phrase.md)
+- [Messaging Overview](../features/messaging/overview.md)

package/agent-assets/docs/troubleshooting/morning-routine-didnt-run.md

@@ -9,61 +9,124 @@ aliases:
   - no morning briefing
 category: troubleshooting
 summary: |
-  The most common operator pain. Causes are ordered by frequency:
-  daemon was stopped, fallback failed, quota exhausted, schedule
-  configuration mismatch, day-boundary subtlety.
+  The morning routine produced no fresh state/today.md and no briefing.
+  Causes are ordered by frequency: daemon was stopped at the trigger
+  time, both backends failed, quota exhausted, the routine threw and is
+  mid-retry, or it is a day-boundary subtlety. Most cases self-heal via
+  boot-time catch-up or the retry/stall watchdog.
 section: morning-routine-didnt-run
 tags:
   - troubleshooting
-  - routine
+  - routines
+  - autonomous
+  - scheduler
+  - dispatch
 status: stable
 ask_examples:
   - Why didn't my morning routine fire?
   - Why is today.md empty?
+  - How do I regenerate today.md by hand?
 locale: en-US
 created: 2026-04-25
-updated: 2026-04-25
+updated: 2026-05-28
+keywords:
+  - morning routine didn't run
+  - morning routine skipped
+  - morning routine gate
+  - empty today.md
+  - regenerate today
 related:
   - features/routines/morning-routine
+  - features/routines/hourly-check
+  - concepts/agent-day
   - troubleshooting/auth-failed
   - troubleshooting/quota-exhausted
+process_keys:
+  - routine.morning_routine
+  - routine.morning_routine_today
+  - routine.morning_routine_journal
+api_endpoints:
+  - POST /api/agent/regenerate
+config_keys:
+  - dayBoundaryHour
+context_files:
+  - state/today.md
+ui_anchors:
+  - /settings/routines
+  - /activity
 ---
 
 # Morning Routine Didn't Run
 
 ## What You See
 
-- An empty or stale `today.md`.
+- An empty or stale `state/today.md`.
 - No morning notification.
 - Activity has no `routine.morning_routine` row for today.
 
+## Before You Worry: It Often Self-Heals
+
+The morning routine has two recovery paths that usually fix this
+without any action from you:
+
+- **Boot-time catch-up.** If the daemon was stopped during the trigger
+  window, it fires the unrun morning routine the next time it starts.
+  So a missed routine often resolves itself on the next `aitne start`
+  or `aitne restart`.
+- **Retry on failure.** If the routine runs but fails to produce a
+  fresh `state/today.md`, the daemon retries up to 3 times with an
+  exponential back-off (5, then 10, then 15 minutes). After 3 failed
+  attempts it sends you a DM asking you to regenerate manually. If the
+  wake row sits unfinished for ~2 hours (the stall watchdog threshold),
+  you also get an owner DM so the silence never goes unnoticed.
+
+Give it a few minutes, or restart the daemon, before digging deeper.
+
 ## Most Likely Causes (in probability order)
 
-1. **Daemon was stopped at the trigger time.** Check `pa status`.
-2. **Fallback failed too.** Look for a `fallback-failed` notification.
-3. **Heavy-tier quota exhausted on both backends.** See
-   [Quota Exhausted](quota-exhausted.md).
-4. **Hourly check skip-gate fired during the trigger window.** The
-   morning routine and the hourly check share an atomic flag — if the
-   hourly check was already running at the rollover instant, the
-   morning routine won't double-fire.
-5. **Day-boundary subtlety.** Before `dayBoundaryHour`, the routine
-   still "belongs to" yesterday — see
+1. **Daemon was stopped at the trigger time.** Check `aitne status`.
+   Boot-time catch-up should cover this once the daemon is back up.
+2. **Both backends failed.** The routine tried the main backend, fell
+   back, and the fallback failed too. Check `/activity` for an error
+   outcome and `aitne logs` for the failure.
+3. **Quota exhausted on both backends.** The morning routine runs on
+   the medium tier (Sonnet by default for Claude); a backend that is
+   out of quota fails over, and if the fallback is also out you get no
+   run. See [Quota Exhausted](quota-exhausted.md).
+4. **Mid-retry.** The routine threw and is in the 5/10/15-minute
+   retry window — the row may simply not have landed yet.
+5. **Day-boundary subtlety.** Before `dayBoundaryHour` (default 04:00),
+   the routine still "belongs to" yesterday — see
    [Agent Day](../concepts/agent-day.md).
 
+Note: the morning routine takes priority over the hourly check, not the
+other way around. The hourly check skips itself while the morning
+routine is active, so a running hourly check never blocks the morning
+routine.
+
 ## Diagnostic Steps
 
-1. `pa status` — daemon up?
-2. `/activity` — any row for `routine.morning_routine`?
+1. `aitne status` — is the daemon up?
+2. `/activity` — is there a row for `routine.morning_routine`? An error
+   outcome points to a backend failure (cause 2 or 3).
 3. `/settings/routines` — the routine list shows the next scheduled
    fire and recent runs.
-4. `pa logs` — search for `morning_routine`.
+4. `aitne logs` — search for `morning_routine` to see the trigger,
+   any fallback, and retry scheduling.
+
+## Forcing a Regenerate
+
+If you want today's `state/today.md` rebuilt right now, click
+**Regenerate** on the dashboard (it POSTs to `/api/agent/regenerate`
+with `target: today`). This bypasses the schedule and runs the
+synthesis immediately.
 
 ## Confirming the Fix
 
-- The next morning's routine fires within 60 seconds of
-  `dayBoundaryHour` (default 04:00).
-- Activity shows the row with a non-error outcome.
+- The next morning's routine fires shortly after `dayBoundaryHour`
+  (default 04:00).
+- `/activity` shows a `routine.morning_routine` row with a non-error
+  outcome.
 
 ## When None of the Above Help
 
@@ -73,3 +136,5 @@ related:
 ## Related
 
 - [Morning Routine](../features/routines/morning-routine.md)
+- [Hourly Check](../features/routines/hourly-check.md)
+- [Agent Day](../concepts/agent-day.md)

package/agent-assets/docs/troubleshooting/observation-not-detected.md

@@ -3,55 +3,119 @@ schema_version: 1
 slug: troubleshooting/observation-not-detected
 title: Observation Not Detected
 id: observation-not-detected
+aliases:
+  - observation missing
+  - no observations
+  - polling broken
+  - hourly check empty
 category: troubleshooting
 summary: |
   An expected change (commit, note, calendar move) didn't surface in
   the hourly check. Most often a polling delay, a vault/repo not
-  watched, or the change was authored by the agent itself.
-section: observation-not-detected
+  watched, or a change the agent authored itself (filtered out).
+section: troubleshooting
 tags:
   - troubleshooting
   - observations
+  - polling
+  - hourly-check
 status: stable
 ask_examples:
   - Why didn't the agent notice my new commit?
   - Why didn't a calendar change show up?
+  - Why is the hourly check empty?
 locale: en-US
 created: 2026-04-25
-updated: 2026-04-25
+updated: 2026-05-28
+keywords:
+  - observation
+  - polling
+  - observer
+  - AgentWriteTracker
+  - hourly check
 related:
   - concepts/observations
+  - features/routines/hourly-check
   - features/integrations/git
   - features/integrations/obsidian
-  - features/routines/hourly-check
+config_keys:
+  - hourlyCheckMinObservations
+  - hourlyCheckEnabled
+  - hourlyCheckIntervalMinutes
+process_keys:
+  - routine.hourly_check
+ui_anchors:
+  - /connections/repositories
+  - /connections/calendar
+  - /settings/schedule
+context_files:
+  - state/today.md
 ---
 
 # Observation Not Detected
 
-## What You See
+You made a change by hand — a commit, an Obsidian note, a calendar move — and
+expected the agent to notice it on the next hourly check. It didn't.
+
+This is almost always one of four things: the poller hasn't run yet, the
+source isn't being watched, the agent itself made the change (so it was
+filtered out), or there weren't enough observations to clear the hourly-check
+gate. Work through them in that order.
+
+## How detection actually works
+
+Observers (Obsidian, Git, Notion, Calendar) do **not** fire an event per
+change. They poll on a cadence and call `recordObservation(...)`, which
+appends a row to the `observations` table. The hourly check later reads those
+rows in a single batch. So a "missing" observation is usually a row that was
+never written, or one that was written but filtered.
 
-- A change you made by hand, not echoed by the hourly check.
-- The hourly check ran but did not mention it.
+`AgentWriteTracker` tags every change as `actor='agent'` or `actor='user'`.
+Changes the agent itself wrote are tagged `actor='agent'` and skipped by the
+hourly check — this is the deliberate anti-loop filter that stops the agent
+from observing its own output.
 
-## Most Likely Causes
+## Most likely causes
 
-1. Poll has not yet fired since your change.
-2. The repo / vault is not on the watched list.
-3. The change was tagged `actor='agent'` (anti-loop filter).
-4. Below the `hourlyCheckMinObservations` threshold.
+1. **The poll hasn't fired since your change.** Each integration runs on its
+   own interval; a fresh change can sit for several minutes before the next
+   poll picks it up.
+2. **The repo / vault / calendar isn't on the watched list.** If the source
+   was never connected, nothing polls it.
+3. **The change was tagged `actor='agent'`.** If the agent (not you) authored
+   the commit or note, the anti-loop filter drops it.
+4. **It was below the gate threshold.** The hourly check only runs its full
+   pass when at least `hourlyCheckMinObservations` (default `1`) pending
+   observations exist. A single low-signal change can be held back.
 
-## Diagnostic Steps
+## Diagnostic steps
 
-1. Confirm the integration's "last polled" timestamp on
-   `/connections/...`.
-2. Confirm the observed path is in the watched set.
-3. Lower `hourlyCheckMinObservations` to test.
+1. **Check the "last polled" timestamp.** On the relevant connection page —
+   `/connections/repositories` for Git/GitHub, `/connections/calendar` for
+   calendars, or the matching `/connections/...` page — confirm the source
+   polled *after* you made the change. If the timestamp predates your change,
+   you're just early; wait for the next poll.
+2. **Confirm the source is watched.** Verify the repo/vault/calendar is
+   actually connected and enabled on its connection page. A disabled or
+   never-added source produces no observations.
+3. **Rule out the agent-authored filter.** If the commit or note was written
+   by the agent, that's expected — it's filtered by design. Look for a change
+   *you* made by hand to test detection.
+4. **Lower the gate threshold to test.** Temporarily set
+   `hourlyCheckMinObservations` to `1` (its default) on `/settings/schedule`
+   so even a single observation triggers the check, then make a manual change
+   and wait for the next hourly run.
 
-## Confirming the Fix
+## Confirming the fix
 
-- Future changes record observations and surface in the next
-  hourly check.
+After the next poll runs, a manual change you made should record an
+observation and surface in the next hourly check. You can cross-check recent
+agent activity from the CLI with `aitne audit`, which reads the `agent_actions`
+log read-only.
 
 ## Related
 
-- [Observations](../concepts/observations.md)
+- [Observations](../concepts/observations.md) — what gets recorded and why
+- [Hourly Check](../features/routines/hourly-check.md) — the gate and its thresholds
+- [Git integration](../features/integrations/git.md)
+- [Obsidian integration](../features/integrations/obsidian.md)

package/agent-assets/docs/troubleshooting/quota-exhausted.md

@@ -17,17 +17,33 @@ tags:
   - troubleshooting
   - cost
   - backends
+  - quotas
+  - routing
 status: stable
 ask_examples:
   - Why did Aitne fall over to the fallback backend?
   - Why is my Anthropic API call returning a quota error?
+  - Why did an autonomous run get skipped for the cost cap?
 locale: en-US
 created: 2026-04-25
-updated: 2026-05-04
+updated: 2026-05-28
+keywords:
+  - quota
+  - BackendQuotaError
+  - rate limit
+  - fallback fired
+  - 5h window
+  - cost cap
+ui_anchors:
+  - /analytics
+  - /settings/models
+config_keys:
+  - autonomousDailyCostCapUsd
+  - autonomousMonthlyCostCapUsd
 related:
   - concepts/costs-and-quotas
   - concepts/auth-health
-  - guides/use-cloud-providers
+  - guides/switch-default-backend
 ---
 
 # Quota Exhausted
@@ -36,15 +52,25 @@ related:
 
 - A `BackendQuotaError` row in Activity.
 - Fallback ran the next routine instead of main.
-- Analytics shows the budget close to or past the cap.
+- Or: an autonomous run was skipped with reason
+  `autonomous_cost_cap_exceeded` — this is a *separate* safety net
+  (`autonomousDailyCostCapUsd` / `autonomousMonthlyCostCapUsd`,
+  default off) that skips only autonomous work, never reactive DMs.
+  It is not a provider quota error; raise the cap on `/settings/models`
+  or wait for the next agent day.
 
 ## Most Likely Causes
 
 1. **Provider rate limit** on your API key (Anthropic / OpenAI /
    Google) — the supported path. Check the provider console for the
    per-minute / per-day caps tied to your billing.
-2. **Gemini per-day cap** for `gemini-2.5-flash` reached on the free
-   tier (`GEMINI_API_KEY` without billing).
+2. **Gemini per-agent-day request ceiling** reached. Aitne caps
+   Gemini at 450 model requests per agent day (a conservative fraction
+   of Google's free-tier daily limit on `GEMINI_API_KEY` /
+   `GOOGLE_API_KEY` without billing). This ceiling is Gemini-only and
+   model-agnostic — it is not tied to any one model id. The error reads
+   "Gemini daily-request ceiling reached" and resets at the next
+   agent-day boundary (04:00 local).
 3. **Cloud-provider quota** — Bedrock / Vertex / Foundry / Azure
    OpenAI / Gemini-Vertex enforce their own per-region / per-model
    quotas. The error surfaces the same way as a direct-API quota.
@@ -73,7 +99,11 @@ related:
 
 ## Confirming the Fix
 
-- The next run after the window resets succeeds on the main backend.
+- After a **provider window / Gemini ceiling** resets, the next run
+  succeeds on the main backend (no `BackendQuotaError` row, no
+  fallback).
+- After raising or clearing the **autonomous cost cap**, autonomous
+  runs stop logging `autonomous_cost_cap_exceeded` skips.
 
 ## Related