milaidy 1.0.0

package/docs/install/uninstall.md

@@ -0,0 +1,128 @@
+---
+summary: "Uninstall Milaidy completely (CLI, service, state, workspace)"
+read_when:
+  - You want to remove Milaidy from a machine
+  - The gateway service is still running after uninstall
+title: "Uninstall"
+---
+
+# Uninstall
+
+Two paths:
+
+- **Easy path** if `milaidy` is still installed.
+- **Manual service removal** if the CLI is gone but the service is still running.
+
+## Easy path (CLI still installed)
+
+Recommended: use the built-in uninstaller:
+
+```bash
+milaidy uninstall
+```
+
+Non-interactive (automation / npx):
+
+```bash
+milaidy uninstall --all --yes --non-interactive
+npx -y milaidy uninstall --all --yes --non-interactive
+```
+
+Manual steps (same result):
+
+1. Stop the gateway service:
+
+```bash
+milaidy gateway stop
+```
+
+2. Uninstall the gateway service (launchd/systemd/schtasks):
+
+```bash
+milaidy gateway uninstall
+```
+
+3. Delete state + config:
+
+```bash
+rm -rf "${MILAIDY_STATE_DIR:-$HOME/.milaidy}"
+```
+
+If you set `MILAIDY_CONFIG_PATH` to a custom location outside the state dir, delete that file too.
+
+4. Delete your workspace (optional, removes agent files):
+
+```bash
+rm -rf ~/.milaidy/workspace
+```
+
+5. Remove the CLI install (pick the one you used):
+
+```bash
+npm rm -g milaidy
+pnpm remove -g milaidy
+bun remove -g milaidy
+```
+
+6. If you installed the macOS app:
+
+```bash
+rm -rf /Applications/Milaidy.app
+```
+
+Notes:
+
+- If you used profiles (`--profile` / `MILAIDY_PROFILE`), repeat step 3 for each state dir (defaults are `~/.milaidy-<profile>`).
+- In remote mode, the state dir lives on the **gateway host**, so run steps 1-4 there too.
+
+## Manual service removal (CLI not installed)
+
+Use this if the gateway service keeps running but `milaidy` is missing.
+
+### macOS (launchd)
+
+Default label is `bot.molt.gateway` (or `bot.molt.<profile>`; legacy `com.milaidy.*` may still exist):
+
+```bash
+launchctl bootout gui/$UID/bot.molt.gateway
+rm -f ~/Library/LaunchAgents/bot.molt.gateway.plist
+```
+
+If you used a profile, replace the label and plist name with `bot.molt.<profile>`. Remove any legacy `com.milaidy.*` plists if present.
+
+### Linux (systemd user unit)
+
+Default unit name is `milaidy-gateway.service` (or `milaidy-gateway-<profile>.service`):
+
+```bash
+systemctl --user disable --now milaidy-gateway.service
+rm -f ~/.config/systemd/user/milaidy-gateway.service
+systemctl --user daemon-reload
+```
+
+### Windows (Scheduled Task)
+
+Default task name is `Milaidy Gateway` (or `Milaidy Gateway (<profile>)`).
+The task script lives under your state dir.
+
+```powershell
+schtasks /Delete /F /TN "Milaidy Gateway"
+Remove-Item -Force "$env:USERPROFILE\.milaidy\gateway.cmd"
+```
+
+If you used a profile, delete the matching task name and `~\.milaidy-<profile>\gateway.cmd`.
+
+## Normal install vs source checkout
+
+### Normal install (install.sh / npm / pnpm / bun)
+
+If you used `https://milaidy.ai/install.sh` or `install.ps1`, the CLI was installed with `npm install -g milaidy@latest`.
+Remove it with `npm rm -g milaidy` (or `pnpm remove -g` / `bun remove -g` if you installed that way).
+
+### Source checkout (git clone)
+
+If you run from a repo checkout (`git clone` + `milaidy ...` / `bun run milaidy ...`):
+
+1. Uninstall the gateway service **before** deleting the repo (use the easy path above or manual service removal).
+2. Delete the repo directory.
+3. Remove state + workspace as shown above.

package/docs/install/updating.md

@@ -0,0 +1,228 @@
+---
+summary: "Updating Milaidy safely (global install or source), plus rollback strategy"
+read_when:
+  - Updating Milaidy
+  - Something breaks after an update
+title: "Updating"
+---
+
+# Updating
+
+Milaidy is moving fast (pre “1.0”). Treat updates like shipping infra: update → run checks → restart (or use `milaidy update`, which restarts) → verify.
+
+## Recommended: re-run the website installer (upgrade in place)
+
+The **preferred** update path is to re-run the installer from the website. It
+detects existing installs, upgrades in place, and runs `milaidy doctor` when
+needed.
+
+```bash
+curl -fsSL https://milaidy.ai/install.sh | bash
+```
+
+Notes:
+
+- Add `--no-onboard` if you don’t want the onboarding wizard to run again.
+- For **source installs**, use:
+  ```bash
+  curl -fsSL https://milaidy.ai/install.sh | bash -s -- --install-method git --no-onboard
+  ```
+  The installer will `git pull --rebase` **only** if the repo is clean.
+- For **global installs**, the script uses `npm install -g milaidy@latest` under the hood.
+- Legacy note: `milaidy` remains available as a compatibility shim.
+
+## Before you update
+
+- Know how you installed: **global** (npm/pnpm) vs **from source** (git clone).
+- Know how your Gateway is running: **foreground terminal** vs **supervised service** (launchd/systemd).
+- Snapshot your tailoring:
+  - Config: `~/.milaidy/milaidy.json`
+  - Credentials: `~/.milaidy/credentials/`
+  - Workspace: `~/.milaidy/workspace`
+
+## Update (global install)
+
+Global install (pick one):
+
+```bash
+npm i -g milaidy@latest
+```
+
+```bash
+pnpm add -g milaidy@latest
+```
+
+We do **not** recommend Bun for the Gateway runtime (WhatsApp/Telegram bugs).
+
+To switch update channels (git + npm installs):
+
+```bash
+milaidy update --channel beta
+milaidy update --channel dev
+milaidy update --channel stable
+```
+
+Use `--tag <dist-tag|version>` for a one-off install tag/version.
+
+See [Development channels](/install/development-channels) for channel semantics and release notes.
+
+Note: on npm installs, the gateway logs an update hint on startup (checks the current channel tag). Disable via `update.checkOnStart: false`.
+
+Then:
+
+```bash
+milaidy doctor
+milaidy gateway restart
+milaidy health
+```
+
+Notes:
+
+- If your Gateway runs as a service, `milaidy gateway restart` is preferred over killing PIDs.
+- If you’re pinned to a specific version, see “Rollback / pinning” below.
+
+## Update (`milaidy update`)
+
+For **source installs** (git checkout), prefer:
+
+```bash
+milaidy update
+```
+
+It runs a safe-ish update flow:
+
+- Requires a clean worktree.
+- Switches to the selected channel (tag or branch).
+- Fetches + rebases against the configured upstream (dev channel).
+- Installs deps, builds, builds the Control UI, and runs `milaidy doctor`.
+- Restarts the gateway by default (use `--no-restart` to skip).
+
+If you installed via **npm/pnpm** (no git metadata), `milaidy update` will try to update via your package manager. If it can’t detect the install, use “Update (global install)” instead.
+
+## Update (Control UI / RPC)
+
+The Control UI has **Update & Restart** (RPC: `update.run`). It:
+
+1. Runs the same source-update flow as `milaidy update` (git checkout only).
+2. Writes a restart sentinel with a structured report (stdout/stderr tail).
+3. Restarts the gateway and pings the last active session with the report.
+
+If the rebase fails, the gateway aborts and restarts without applying the update.
+
+## Update (from source)
+
+From the repo checkout:
+
+Preferred:
+
+```bash
+milaidy update
+```
+
+Manual (equivalent-ish):
+
+```bash
+git pull
+pnpm install
+pnpm build
+pnpm ui:build # auto-installs UI deps on first run
+milaidy doctor
+milaidy health
+```
+
+Notes:
+
+- `pnpm build` matters when you run the packaged `milaidy` binary ([`milaidy.mjs`](https://github.com/milaidy/milaidy/blob/main/milaidy.mjs)) or use Node to run `dist/`.
+- If you run from a repo checkout without a global install, use `pnpm milaidy ...` for CLI commands.
+- If you run directly from TypeScript (`pnpm milaidy ...`), a rebuild is usually unnecessary, but **config migrations still apply** → run doctor.
+- Switching between global and git installs is easy: install the other flavor, then run `milaidy doctor` so the gateway service entrypoint is rewritten to the current install.
+
+## Always Run: `milaidy doctor`
+
+Doctor is the “safe update” command. It’s intentionally boring: repair + migrate + warn.
+
+Note: if you’re on a **source install** (git checkout), `milaidy doctor` will offer to run `milaidy update` first.
+
+Typical things it does:
+
+- Migrate deprecated config keys / legacy config file locations.
+- Audit DM policies and warn on risky “open” settings.
+- Check Gateway health and can offer to restart.
+- Detect and migrate older gateway services (launchd/systemd; legacy schtasks) to current Milaidy services.
+- On Linux, ensure systemd user lingering (so the Gateway survives logout).
+
+Details: [Doctor](/gateway/doctor)
+
+## Start / stop / restart the Gateway
+
+CLI (works regardless of OS):
+
+```bash
+milaidy gateway status
+milaidy gateway stop
+milaidy gateway restart
+milaidy gateway --port 18789
+milaidy logs --follow
+```
+
+If you’re supervised:
+
+- macOS launchd (app-bundled LaunchAgent): `launchctl kickstart -k gui/$UID/bot.molt.gateway` (use `bot.molt.<profile>`; legacy `com.milaidy.*` still works)
+- Linux systemd user service: `systemctl --user restart milaidy-gateway[-<profile>].service`
+- Windows (WSL2): `systemctl --user restart milaidy-gateway[-<profile>].service`
+  - `launchctl`/`systemctl` only work if the service is installed; otherwise run `milaidy gateway install`.
+
+Runbook + exact service labels: [Gateway runbook](/gateway)
+
+## Rollback / pinning (when something breaks)
+
+### Pin (global install)
+
+Install a known-good version (replace `<version>` with the last working one):
+
+```bash
+npm i -g milaidy@<version>
+```
+
+```bash
+pnpm add -g milaidy@<version>
+```
+
+Tip: to see the current published version, run `npm view milaidy version`.
+
+Then restart + re-run doctor:
+
+```bash
+milaidy doctor
+milaidy gateway restart
+```
+
+### Pin (source) by date
+
+Pick a commit from a date (example: “state of main as of 2026-01-01”):
+
+```bash
+git fetch origin
+git checkout "$(git rev-list -n 1 --before=\"2026-01-01\" origin/main)"
+```
+
+Then reinstall deps + restart:
+
+```bash
+pnpm install
+pnpm build
+milaidy gateway restart
+```
+
+If you want to go back to latest later:
+
+```bash
+git checkout main
+git pull
+```
+
+## If you’re stuck
+
+- Run `milaidy doctor` again and read the output carefully (it often tells you the fix).
+- Check: [Troubleshooting](/gateway/troubleshooting)
+- Ask in Discord: https://discord.gg/clawd

package/docs/logging.md

@@ -0,0 +1,350 @@
+---
+summary: "Logging overview: file logs, console output, CLI tailing, and the Control UI"
+read_when:
+  - You need a beginner-friendly overview of logging
+  - You want to configure log levels or formats
+  - You are troubleshooting and need to find logs quickly
+title: "Logging"
+---
+
+# Logging
+
+Milaidy logs in two places:
+
+- **File logs** (JSON lines) written by the Gateway.
+- **Console output** shown in terminals and the Control UI.
+
+This page explains where logs live, how to read them, and how to configure log
+levels and formats.
+
+## Where logs live
+
+By default, the Gateway writes a rolling log file under:
+
+`/tmp/milaidy/milaidy-YYYY-MM-DD.log`
+
+The date uses the gateway host's local timezone.
+
+You can override this in `~/.milaidy/milaidy.json`:
+
+```json
+{
+  "logging": {
+    "file": "/path/to/milaidy.log"
+  }
+}
+```
+
+## How to read logs
+
+### CLI: live tail (recommended)
+
+Use the CLI to tail the gateway log file via RPC:
+
+```bash
+milaidy logs --follow
+```
+
+Output modes:
+
+- **TTY sessions**: pretty, colorized, structured log lines.
+- **Non-TTY sessions**: plain text.
+- `--json`: line-delimited JSON (one log event per line).
+- `--plain`: force plain text in TTY sessions.
+- `--no-color`: disable ANSI colors.
+
+In JSON mode, the CLI emits `type`-tagged objects:
+
+- `meta`: stream metadata (file, cursor, size)
+- `log`: parsed log entry
+- `notice`: truncation / rotation hints
+- `raw`: unparsed log line
+
+If the Gateway is unreachable, the CLI prints a short hint to run:
+
+```bash
+milaidy doctor
+```
+
+### Control UI (web)
+
+The Control UI’s **Logs** tab tails the same file using `logs.tail`.
+See [/web/control-ui](/web/control-ui) for how to open it.
+
+### Channel-only logs
+
+To filter channel activity (WhatsApp/Telegram/etc), use:
+
+```bash
+milaidy channels logs --channel whatsapp
+```
+
+## Log formats
+
+### File logs (JSONL)
+
+Each line in the log file is a JSON object. The CLI and Control UI parse these
+entries to render structured output (time, level, subsystem, message).
+
+### Console output
+
+Console logs are **TTY-aware** and formatted for readability:
+
+- Subsystem prefixes (e.g. `gateway/channels/whatsapp`)
+- Level coloring (info/warn/error)
+- Optional compact or JSON mode
+
+Console formatting is controlled by `logging.consoleStyle`.
+
+## Configuring logging
+
+All logging configuration lives under `logging` in `~/.milaidy/milaidy.json`.
+
+```json
+{
+  "logging": {
+    "level": "info",
+    "file": "/tmp/milaidy/milaidy-YYYY-MM-DD.log",
+    "consoleLevel": "info",
+    "consoleStyle": "pretty",
+    "redactSensitive": "tools",
+    "redactPatterns": ["sk-.*"]
+  }
+}
+```
+
+### Log levels
+
+- `logging.level`: **file logs** (JSONL) level.
+- `logging.consoleLevel`: **console** verbosity level.
+
+`--verbose` only affects console output; it does not change file log levels.
+
+### Console styles
+
+`logging.consoleStyle`:
+
+- `pretty`: human-friendly, colored, with timestamps.
+- `compact`: tighter output (best for long sessions).
+- `json`: JSON per line (for log processors).
+
+### Redaction
+
+Tool summaries can redact sensitive tokens before they hit the console:
+
+- `logging.redactSensitive`: `off` | `tools` (default: `tools`)
+- `logging.redactPatterns`: list of regex strings to override the default set
+
+Redaction affects **console output only** and does not alter file logs.
+
+## Diagnostics + OpenTelemetry
+
+Diagnostics are structured, machine-readable events for model runs **and**
+message-flow telemetry (webhooks, queueing, session state). They do **not**
+replace logs; they exist to feed metrics, traces, and other exporters.
+
+Diagnostics events are emitted in-process, but exporters only attach when
+diagnostics + the exporter plugin are enabled.
+
+### OpenTelemetry vs OTLP
+
+- **OpenTelemetry (OTel)**: the data model + SDKs for traces, metrics, and logs.
+- **OTLP**: the wire protocol used to export OTel data to a collector/backend.
+- Milaidy exports via **OTLP/HTTP (protobuf)** today.
+
+### Signals exported
+
+- **Metrics**: counters + histograms (token usage, message flow, queueing).
+- **Traces**: spans for model usage + webhook/message processing.
+- **Logs**: exported over OTLP when `diagnostics.otel.logs` is enabled. Log
+  volume can be high; keep `logging.level` and exporter filters in mind.
+
+### Diagnostic event catalog
+
+Model usage:
+
+- `model.usage`: tokens, cost, duration, context, provider/model/channel, session ids.
+
+Message flow:
+
+- `webhook.received`: webhook ingress per channel.
+- `webhook.processed`: webhook handled + duration.
+- `webhook.error`: webhook handler errors.
+- `message.queued`: message enqueued for processing.
+- `message.processed`: outcome + duration + optional error.
+
+Queue + session:
+
+- `queue.lane.enqueue`: command queue lane enqueue + depth.
+- `queue.lane.dequeue`: command queue lane dequeue + wait time.
+- `session.state`: session state transition + reason.
+- `session.stuck`: session stuck warning + age.
+- `run.attempt`: run retry/attempt metadata.
+- `diagnostic.heartbeat`: aggregate counters (webhooks/queue/session).
+
+### Enable diagnostics (no exporter)
+
+Use this if you want diagnostics events available to plugins or custom sinks:
+
+```json
+{
+  "diagnostics": {
+    "enabled": true
+  }
+}
+```
+
+### Diagnostics flags (targeted logs)
+
+Use flags to turn on extra, targeted debug logs without raising `logging.level`.
+Flags are case-insensitive and support wildcards (e.g. `telegram.*` or `*`).
+
+```json
+{
+  "diagnostics": {
+    "flags": ["telegram.http"]
+  }
+}
+```
+
+Env override (one-off):
+
+```
+MILAIDY_DIAGNOSTICS=telegram.http,telegram.payload
+```
+
+Notes:
+
+- Flag logs go to the standard log file (same as `logging.file`).
+- Output is still redacted according to `logging.redactSensitive`.
+- Full guide: [/diagnostics/flags](/diagnostics/flags).
+
+### Export to OpenTelemetry
+
+Diagnostics can be exported via the `diagnostics-otel` plugin (OTLP/HTTP). This
+works with any OpenTelemetry collector/backend that accepts OTLP/HTTP.
+
+```json
+{
+  "plugins": {
+    "allow": ["diagnostics-otel"],
+    "entries": {
+      "diagnostics-otel": {
+        "enabled": true
+      }
+    }
+  },
+  "diagnostics": {
+    "enabled": true,
+    "otel": {
+      "enabled": true,
+      "endpoint": "http://otel-collector:4318",
+      "protocol": "http/protobuf",
+      "serviceName": "milaidy-gateway",
+      "traces": true,
+      "metrics": true,
+      "logs": true,
+      "sampleRate": 0.2,
+      "flushIntervalMs": 60000
+    }
+  }
+}
+```
+
+Notes:
+
+- You can also enable the plugin with `milaidy plugins enable diagnostics-otel`.
+- `protocol` currently supports `http/protobuf` only. `grpc` is ignored.
+- Metrics include token usage, cost, context size, run duration, and message-flow
+  counters/histograms (webhooks, queueing, session state, queue depth/wait).
+- Traces/metrics can be toggled with `traces` / `metrics` (default: on). Traces
+  include model usage spans plus webhook/message processing spans when enabled.
+- Set `headers` when your collector requires auth.
+- Environment variables supported: `OTEL_EXPORTER_OTLP_ENDPOINT`,
+  `OTEL_SERVICE_NAME`, `OTEL_EXPORTER_OTLP_PROTOCOL`.
+
+### Exported metrics (names + types)
+
+Model usage:
+
+- `milaidy.tokens` (counter, attrs: `milaidy.token`, `milaidy.channel`,
+  `milaidy.provider`, `milaidy.model`)
+- `milaidy.cost.usd` (counter, attrs: `milaidy.channel`, `milaidy.provider`,
+  `milaidy.model`)
+- `milaidy.run.duration_ms` (histogram, attrs: `milaidy.channel`,
+  `milaidy.provider`, `milaidy.model`)
+- `milaidy.context.tokens` (histogram, attrs: `milaidy.context`,
+  `milaidy.channel`, `milaidy.provider`, `milaidy.model`)
+
+Message flow:
+
+- `milaidy.webhook.received` (counter, attrs: `milaidy.channel`,
+  `milaidy.webhook`)
+- `milaidy.webhook.error` (counter, attrs: `milaidy.channel`,
+  `milaidy.webhook`)
+- `milaidy.webhook.duration_ms` (histogram, attrs: `milaidy.channel`,
+  `milaidy.webhook`)
+- `milaidy.message.queued` (counter, attrs: `milaidy.channel`,
+  `milaidy.source`)
+- `milaidy.message.processed` (counter, attrs: `milaidy.channel`,
+  `milaidy.outcome`)
+- `milaidy.message.duration_ms` (histogram, attrs: `milaidy.channel`,
+  `milaidy.outcome`)
+
+Queues + sessions:
+
+- `milaidy.queue.lane.enqueue` (counter, attrs: `milaidy.lane`)
+- `milaidy.queue.lane.dequeue` (counter, attrs: `milaidy.lane`)
+- `milaidy.queue.depth` (histogram, attrs: `milaidy.lane` or
+  `milaidy.channel=heartbeat`)
+- `milaidy.queue.wait_ms` (histogram, attrs: `milaidy.lane`)
+- `milaidy.session.state` (counter, attrs: `milaidy.state`, `milaidy.reason`)
+- `milaidy.session.stuck` (counter, attrs: `milaidy.state`)
+- `milaidy.session.stuck_age_ms` (histogram, attrs: `milaidy.state`)
+- `milaidy.run.attempt` (counter, attrs: `milaidy.attempt`)
+
+### Exported spans (names + key attributes)
+
+- `milaidy.model.usage`
+  - `milaidy.channel`, `milaidy.provider`, `milaidy.model`
+  - `milaidy.sessionKey`, `milaidy.sessionId`
+  - `milaidy.tokens.*` (input/output/cache_read/cache_write/total)
+- `milaidy.webhook.processed`
+  - `milaidy.channel`, `milaidy.webhook`, `milaidy.chatId`
+- `milaidy.webhook.error`
+  - `milaidy.channel`, `milaidy.webhook`, `milaidy.chatId`,
+    `milaidy.error`
+- `milaidy.message.processed`
+  - `milaidy.channel`, `milaidy.outcome`, `milaidy.chatId`,
+    `milaidy.messageId`, `milaidy.sessionKey`, `milaidy.sessionId`,
+    `milaidy.reason`
+- `milaidy.session.stuck`
+  - `milaidy.state`, `milaidy.ageMs`, `milaidy.queueDepth`,
+    `milaidy.sessionKey`, `milaidy.sessionId`
+
+### Sampling + flushing
+
+- Trace sampling: `diagnostics.otel.sampleRate` (0.0–1.0, root spans only).
+- Metric export interval: `diagnostics.otel.flushIntervalMs` (min 1000ms).
+
+### Protocol notes
+
+- OTLP/HTTP endpoints can be set via `diagnostics.otel.endpoint` or
+  `OTEL_EXPORTER_OTLP_ENDPOINT`.
+- If the endpoint already contains `/v1/traces` or `/v1/metrics`, it is used as-is.
+- If the endpoint already contains `/v1/logs`, it is used as-is for logs.
+- `diagnostics.otel.logs` enables OTLP log export for the main logger output.
+
+### Log export behavior
+
+- OTLP logs use the same structured records written to `logging.file`.
+- Respect `logging.level` (file log level). Console redaction does **not** apply
+  to OTLP logs.
+- High-volume installs should prefer OTLP collector sampling/filtering.
+
+## Troubleshooting tips
+
+- **Gateway not reachable?** Run `milaidy doctor` first.
+- **Logs empty?** Check that the Gateway is running and writing to the file path
+  in `logging.file`.
+- **Need more detail?** Set `logging.level` to `debug` or `trace` and retry.