@biaoo/tiangong-wiki 0.2.2 → 0.3.0

package/references/centralized-service-deployment.md

@@ -0,0 +1,482 @@
+# Centralized Service Deployment
+
+Single-host deployment guide for running Tiangong Wiki as a centralized service with:
+
+- a local daemon (`tiangong-wiki daemon run`)
+- a local MCP adapter (`mcp-server/dist/index.js`)
+- an Nginx reverse proxy handling TLS, static Bearer tokens, and trusted actor headers
+
+This guide is the V1 baseline for issue `#040`. It assumes Linux + `systemd` and intentionally does not cover HA, load balancing, or cloud-vendor-specific templates.
+
+---
+
+## Topology
+
+```text
+Remote MCP client
+  -> HTTPS /mcp (Nginx)
+  -> local MCP service (127.0.0.1:9400)
+  -> local daemon (127.0.0.1:8787)
+  -> pages/ + index.db + templates/ + audit.ndjson + local Git repo
+```
+
+Rules for V1:
+
+- Do not expose the daemon directly to the public internet.
+- Only the reverse proxy should terminate TLS and validate Bearer tokens.
+- The proxy must overwrite actor headers before traffic reaches MCP.
+- MCP stays a thin adapter and must not write files directly.
+
+---
+
+## Directory Layout
+
+Recommended server layout:
+
+```text
+/srv/tiangong-wiki/
+├── current/                       # checked-out tiangong-wiki repo + built artifacts
+├── workspace/
+│   ├── pages/
+│   ├── templates/
+│   ├── wiki.config.json
+│   ├── index.db
+│   └── .queue-artifacts/
+├── vault/
+└── .wiki-runtime/
+    └── audit.ndjson
+
+/etc/tiangong-wiki/
+└── centralized.env
+
+/etc/systemd/system/
+├── tiangong-wiki-daemon.service
+└── tiangong-wiki-mcp.service
+```
+
+Recommended ownership:
+
+- application repo: `root:root`
+- runtime workspace and vault: `tiangong-wiki:tiangong-wiki`
+- `systemd` services run as `tiangong-wiki`
+
+---
+
+## Build and Bootstrap
+
+```bash
+git clone <repo-url> /srv/tiangong-wiki/current
+cd /srv/tiangong-wiki/current
+npm ci
+npm run build
+```
+
+Initialize the workspace once:
+
+```bash
+mkdir -p /srv/tiangong-wiki/workspace/pages
+mkdir -p /srv/tiangong-wiki/workspace/templates
+mkdir -p /srv/tiangong-wiki/vault
+
+cat >/etc/tiangong-wiki/centralized.env <<'EOF'
+WIKI_PATH=/srv/tiangong-wiki/workspace/pages
+WIKI_DB_PATH=/srv/tiangong-wiki/workspace/index.db
+WIKI_CONFIG_PATH=/srv/tiangong-wiki/workspace/wiki.config.json
+WIKI_TEMPLATES_PATH=/srv/tiangong-wiki/workspace/templates
+VAULT_PATH=/srv/tiangong-wiki/vault
+
+WIKI_SYNC_INTERVAL=300
+WIKI_DAEMON_PORT=8787
+
+WIKI_MCP_HOST=127.0.0.1
+WIKI_MCP_PORT=9400
+WIKI_MCP_PATH=/mcp
+WIKI_DAEMON_BASE_URL=http://127.0.0.1:8787
+
+WIKI_GIT_AUTO_PUSH=true
+WIKI_GIT_PUSH_REMOTE=origin
+WIKI_GIT_PUSH_DELAY_MS=3000
+EOF
+
+cd /srv/tiangong-wiki/current
+env $(grep -v '^#' /etc/tiangong-wiki/centralized.env | xargs) node dist/index.js init
+```
+
+An example env file is also provided at [references/examples/centralized-service/centralized.env.example](./examples/centralized-service/centralized.env.example).
+
+Important:
+
+- Bearer tokens are not part of `centralized.env`
+- Bearer token validation belongs to Nginx, not the daemon or MCP process
+- Keep real tokens in a private Nginx include file such as `/etc/nginx/snippets/wiki-auth-tokens.conf`
+
+---
+
+## Required Environment Variables
+
+### Core runtime
+
+- `WIKI_PATH`
+- `WIKI_DB_PATH`
+- `WIKI_CONFIG_PATH`
+- `WIKI_TEMPLATES_PATH`
+- `VAULT_PATH`
+- `WIKI_SYNC_INTERVAL`
+
+### Daemon
+
+- `WIKI_DAEMON_PORT`
+
+Notes:
+
+- the daemon bind host is currently fixed to `127.0.0.1`
+- the daemon should stay loopback-only in production
+
+### MCP adapter
+
+- `WIKI_MCP_HOST`
+- `WIKI_MCP_PORT`
+- `WIKI_MCP_PATH`
+- `WIKI_DAEMON_BASE_URL`
+
+Recommended values:
+
+- `WIKI_MCP_HOST=127.0.0.1`
+- `WIKI_MCP_PORT=9400`
+- `WIKI_MCP_PATH=/mcp`
+- `WIKI_DAEMON_BASE_URL=http://127.0.0.1:8787`
+
+### Optional Git push batching
+
+- `WIKI_GIT_AUTO_PUSH=true|false`
+- `WIKI_GIT_PUSH_REMOTE`
+- `WIKI_GIT_PUSH_DELAY_MS`
+
+V1 decision:
+
+- local Git commit is part of the write transaction
+- async push batching stays inside the daemon process
+- external cron is not required for the primary path
+
+---
+
+## Git Repository Setup
+
+The centralized service assumes the wiki workspace itself is a real Git repository.
+
+Current behavior:
+
+- every successful write attempts a local Git commit
+- optional remote push is triggered asynchronously when `WIKI_GIT_AUTO_PUSH=true`
+- push failure does not roll back a successful local write
+
+Initialize the workspace repo:
+
+```bash
+cd /srv/tiangong-wiki/workspace
+
+git init -b main
+git config user.name "tiangong-wiki"
+git config user.email "tiangong-wiki@local"
+```
+
+Notes:
+
+- daemon-generated commits override author/committer identity from the write actor
+- the local repo still should have basic Git config for operator workflows
+
+Configure the GitHub remote:
+
+```bash
+cd /srv/tiangong-wiki/workspace
+git remote add origin git@github.com:YOUR_ORG/YOUR_REPO.git
+git remote -v
+```
+
+If the remote already exists:
+
+```bash
+git remote set-url origin git@github.com:YOUR_ORG/YOUR_REPO.git
+```
+
+Recommended authentication model:
+
+- run the daemon as a dedicated system user such as `tiangong-wiki`
+- give that user an SSH key dedicated to this deployment
+- register the public key in GitHub as a deploy key, bot account key, or GitHub App-managed key
+
+Example SSH bootstrap:
+
+```bash
+sudo -u tiangong-wiki mkdir -p /home/tiangong-wiki/.ssh
+sudo -u tiangong-wiki ssh-keygen -t ed25519 -f /home/tiangong-wiki/.ssh/id_ed25519 -N ""
+sudo -u tiangong-wiki ssh-keyscan github.com >> /home/tiangong-wiki/.ssh/known_hosts
+chmod 600 /home/tiangong-wiki/.ssh/known_hosts
+```
+
+Do not store GitHub personal access tokens in `centralized.env`. Prefer SSH deploy keys.
+
+Before enabling daemon-side auto push, verify Git access as the actual service user:
+
+```bash
+sudo -u tiangong-wiki git -C /srv/tiangong-wiki/workspace status
+sudo -u tiangong-wiki git -C /srv/tiangong-wiki/workspace remote -v
+sudo -u tiangong-wiki git -C /srv/tiangong-wiki/workspace push origin HEAD
+```
+
+If this manual push fails, daemon-side auto push will fail too.
+
+Operational constraints:
+
+- do not leave unrelated unstaged changes in the workspace repo
+- ensure the target branch policy matches service behavior if you enable auto push
+- if GitHub branch protection forbids direct push to the target branch, the current V1 service should push to an allowed branch or keep auto push disabled
+
+---
+
+## `systemd` Units
+
+Example unit files are provided under [references/examples/centralized-service/](./examples/centralized-service/).
+
+Install them:
+
+```bash
+cp /srv/tiangong-wiki/current/references/examples/centralized-service/tiangong-wiki-daemon.service /etc/systemd/system/
+cp /srv/tiangong-wiki/current/references/examples/centralized-service/tiangong-wiki-mcp.service /etc/systemd/system/
+
+systemctl daemon-reload
+systemctl enable --now tiangong-wiki-daemon
+systemctl enable --now tiangong-wiki-mcp
+```
+
+Startup order:
+
+1. Build the repo (`npm ci && npm run build`)
+2. Initialize the workspace (`node dist/index.js init`)
+3. Start `tiangong-wiki-daemon`
+4. Verify daemon health
+5. Start `tiangong-wiki-mcp`
+6. Verify MCP health
+7. Reload Nginx
+
+---
+
+## Nginx Reverse Proxy
+
+V1 authentication model:
+
+- Nginx validates a static Bearer token
+- Nginx maps token -> actor identity
+- Nginx injects:
+  - `X-Wiki-Actor-Id`
+  - `X-Wiki-Actor-Type`
+  - `X-Request-Id`
+- MCP forwards these headers to the daemon on write requests
+
+Important:
+
+- never trust client-supplied actor headers directly
+- always overwrite actor headers in the proxy
+- clear `Authorization` before passing traffic upstream
+
+Example config: [references/examples/centralized-service/nginx-centralized-wiki.conf](./examples/centralized-service/nginx-centralized-wiki.conf)
+
+Recommended token placement:
+
+1. Create a private include file, for example `/etc/nginx/snippets/wiki-auth-tokens.conf`
+2. Move the `map $http_authorization ...` blocks into that file
+3. `include` that file from your main Nginx config inside the `http {}` scope
+4. Keep only placeholder tokens in repo-tracked example configs
+
+This keeps static Bearer tokens out of the service env file and out of the checked-in site config.
+
+Recommended exposure model:
+
+- public: `/mcp` and `/mcp/health`
+- private/admin only: daemon health and daemon HTTP routes
+
+Because MCP uses streamable HTTP/SSE, keep:
+
+- `proxy_http_version 1.1`
+- `proxy_buffering off`
+- generous read/send timeouts
+
+---
+
+## Health Checks
+
+### Local daemon
+
+```bash
+curl http://127.0.0.1:8787/health
+curl http://127.0.0.1:8787/status
+curl http://127.0.0.1:8787/write-queue/summary
+```
+
+### Local MCP
+
+```bash
+curl http://127.0.0.1:9400/health
+```
+
+### Through Nginx
+
+```bash
+curl -H "Authorization: Bearer <token>" https://wiki.example.com/mcp/health
+```
+
+### `systemd`
+
+```bash
+systemctl status tiangong-wiki-daemon
+systemctl status tiangong-wiki-mcp
+
+journalctl -u tiangong-wiki-daemon -f
+journalctl -u tiangong-wiki-mcp -f
+```
+
+---
+
+## Runtime Contracts You Must Expect
+
+These behaviors are intentional and should be documented to operators and MCP clients.
+
+### Revision conflicts
+
+- daemon returns HTTP `409`
+- MCP returns a structured tool error with `code=revision_conflict`
+- callers must re-read the page and retry intentionally
+
+### Queue full
+
+- daemon returns HTTP `503`
+- cause: write queue depth exceeded the configured limit
+- operators should inspect `/write-queue/summary`
+
+### Degraded Git commit failure
+
+- the write may already be applied locally
+- daemon returns a degraded error with `code=git_commit_failed`
+- inspect `audit.ndjson`, local Git state, and journal output before retrying
+
+### Sync failure
+
+- daemon does not commit to Git when sync fails
+- audit log records `sync_failed`
+
+### Audit and journal locations
+
+- audit log: `/srv/tiangong-wiki/.wiki-runtime/audit.ndjson` in the recommended layout
+- Git journal: local repo commit history in `/srv/tiangong-wiki/workspace` or your chosen wiki root repo
+
+---
+
+## Backup and Push Strategy
+
+V1 baseline:
+
+- every successful write attempts a local Git commit
+- optional async push is handled by the daemon when `WIKI_GIT_AUTO_PUSH=true`
+- async push failure does not roll back the successful local write
+
+Recommended operator practice:
+
+- keep the workspace as a real Git repository with a configured `origin`
+- snapshot both the workspace and `/etc/tiangong-wiki/centralized.env`
+- monitor daemon logs for `git push failed`
+
+Manual recovery after push failure:
+
+```bash
+cd /srv/tiangong-wiki/workspace
+git status
+git log --oneline -n 5
+git push origin HEAD
+```
+
+---
+
+## Recovery Playbook
+
+### Daemon is down
+
+```bash
+systemctl restart tiangong-wiki-daemon
+curl http://127.0.0.1:8787/health
+```
+
+If it still fails:
+
+- inspect `journalctl -u tiangong-wiki-daemon -n 200`
+- verify `index.db`, `wiki.config.json`, and Git repo status
+
+### MCP is down but daemon is healthy
+
+```bash
+systemctl restart tiangong-wiki-mcp
+curl http://127.0.0.1:9400/health
+```
+
+Check:
+
+- `WIKI_DAEMON_BASE_URL`
+- `WIKI_MCP_PORT`
+- Nginx upstream target
+
+### Queue remains full
+
+Check:
+
+```bash
+curl http://127.0.0.1:8787/write-queue/summary | jq
+```
+
+Look for:
+
+- a long-running active job
+- repeated failing jobs
+- vault queue pressure caused by large `sync --process` workloads
+
+### Revision conflict reported by clients
+
+This is expected behavior, not an outage.
+
+Action:
+
+1. re-read the page
+2. merge user intent with the latest revision
+3. retry with the new `ifRevision`
+
+### Degraded Git commit failure
+
+Check:
+
+```bash
+journalctl -u tiangong-wiki-daemon -n 200
+tail -n 50 /srv/tiangong-wiki/.wiki-runtime/audit.ndjson
+cd /srv/tiangong-wiki/workspace && git status
+```
+
+Typical causes:
+
+- workspace is not a Git repo
+- Git identity or hooks are broken
+- permissions do not allow commit creation
+
+### Actor metadata missing
+
+If MCP returns `missing_actor`:
+
+- verify Nginx is overwriting the three actor headers
+- verify the client is calling the proxied `/mcp` endpoint, not the loopback service directly
+
+---
+
+## What V1 Does Not Cover
+
+- multi-host deployments
+- active/active daemon replicas
+- cloud-specific Terraform or Helm modules
+- alternate reverse proxy examples beyond Nginx
+
+If those are needed later, treat them as follow-up work after the single-host contract is stable.

package/references/examples/centralized-service/centralized.env.example

@@ -0,0 +1,25 @@
+# Core workspace
+WIKI_PATH=/srv/tiangong-wiki/workspace/pages
+WIKI_DB_PATH=/srv/tiangong-wiki/workspace/index.db
+WIKI_CONFIG_PATH=/srv/tiangong-wiki/workspace/wiki.config.json
+WIKI_TEMPLATES_PATH=/srv/tiangong-wiki/workspace/templates
+VAULT_PATH=/srv/tiangong-wiki/vault
+
+# Daemon
+WIKI_SYNC_INTERVAL=300
+WIKI_DAEMON_PORT=8787
+
+# MCP adapter
+WIKI_MCP_HOST=127.0.0.1
+WIKI_MCP_PORT=9400
+WIKI_MCP_PATH=/mcp
+WIKI_DAEMON_BASE_URL=http://127.0.0.1:8787
+
+# Bearer tokens are NOT configured here.
+# Put them in your reverse proxy config instead, for example:
+# /etc/nginx/snippets/wiki-auth-tokens.conf
+
+# Optional async Git push batching
+WIKI_GIT_AUTO_PUSH=true
+WIKI_GIT_PUSH_REMOTE=origin
+WIKI_GIT_PUSH_DELAY_MS=3000

package/references/examples/centralized-service/nginx-centralized-wiki.conf

@@ -0,0 +1,68 @@
+# Put the `map` blocks in the `http {}` scope.
+# These token values are placeholders only.
+# In production, move them to a private include such as:
+# /etc/nginx/snippets/wiki-auth-tokens.conf
+
+map $http_authorization $wiki_access_allowed {
+    default 0;
+    "Bearer codex-prod-token" 1;
+    "Bearer claude-prod-token" 1;
+}
+
+map $http_authorization $wiki_actor_id {
+    default "";
+    "Bearer codex-prod-token" "agent:codex";
+    "Bearer claude-prod-token" "agent:claude";
+}
+
+map $http_authorization $wiki_actor_type {
+    default "";
+    "Bearer codex-prod-token" "agent";
+    "Bearer claude-prod-token" "agent";
+}
+
+server {
+    listen 443 ssl http2;
+    server_name wiki.example.com;
+
+    ssl_certificate /etc/letsencrypt/live/wiki.example.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/wiki.example.com/privkey.pem;
+
+    proxy_http_version 1.1;
+    proxy_buffering off;
+    proxy_read_timeout 360s;
+    proxy_send_timeout 360s;
+
+    location = /mcp/health {
+        if ($wiki_access_allowed = 0) { return 401; }
+
+        proxy_set_header Authorization "";
+        proxy_pass http://127.0.0.1:9400/health;
+    }
+
+    location /mcp {
+        if ($wiki_access_allowed = 0) { return 401; }
+
+        proxy_set_header Authorization "";
+        proxy_set_header X-Wiki-Actor-Id $wiki_actor_id;
+        proxy_set_header X-Wiki-Actor-Type $wiki_actor_type;
+        proxy_set_header X-Request-Id $request_id;
+
+        proxy_pass http://127.0.0.1:9400;
+    }
+
+    # Optional admin-only daemon routes.
+    # Keep them private or behind an internal allowlist / VPN.
+    location /daemon/ {
+        allow 127.0.0.1;
+        deny all;
+
+        proxy_set_header Authorization "";
+        proxy_set_header X-Wiki-Actor-Id "system:proxy-admin";
+        proxy_set_header X-Wiki-Actor-Type "system";
+        proxy_set_header X-Request-Id $request_id;
+
+        rewrite ^/daemon/(.*)$ /$1 break;
+        proxy_pass http://127.0.0.1:8787;
+    }
+}

package/references/examples/centralized-service/tiangong-wiki-daemon.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Tiangong Wiki daemon
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=tiangong-wiki
+Group=tiangong-wiki
+WorkingDirectory=/srv/tiangong-wiki/current
+EnvironmentFile=/etc/tiangong-wiki/centralized.env
+ExecStart=/usr/bin/env node /srv/tiangong-wiki/current/dist/index.js daemon run
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target

package/references/examples/centralized-service/tiangong-wiki-mcp.service

@@ -0,0 +1,18 @@
+[Unit]
+Description=Tiangong Wiki MCP adapter
+After=network-online.target tiangong-wiki-daemon.service
+Wants=network-online.target
+Requires=tiangong-wiki-daemon.service
+
+[Service]
+Type=simple
+User=tiangong-wiki
+Group=tiangong-wiki
+WorkingDirectory=/srv/tiangong-wiki/current
+EnvironmentFile=/etc/tiangong-wiki/centralized.env
+ExecStart=/usr/bin/env node /srv/tiangong-wiki/current/mcp-server/dist/index.js
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target

package/references/troubleshooting.md

@@ -34,6 +34,21 @@ All configuration is managed through `.wiki.env` (created by `tiangong-wiki setu
 | `WIKI_TEMPLATES_PATH` | Yes | Path to the templates directory |
 | `WIKI_SYNC_INTERVAL` | No | Auto-sync interval in seconds (default: `86400`) |
 
+### Daemon and MCP
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `WIKI_DAEMON_PORT` | No | Loopback daemon port |
+| `WIKI_MCP_HOST` | No | MCP bind host (recommended: `127.0.0.1`) |
+| `WIKI_MCP_PORT` | No | MCP bind port |
+| `WIKI_MCP_PATH` | No | MCP HTTP path (default: `/mcp`) |
+| `WIKI_DAEMON_BASE_URL` | For MCP service | Daemon base URL used by the MCP adapter |
+| `WIKI_GIT_AUTO_PUSH` | No | Enable daemon-side async Git push batching |
+| `WIKI_GIT_PUSH_REMOTE` | No | Git remote name for async push (default: `origin`) |
+| `WIKI_GIT_PUSH_DELAY_MS` | No | Delay before async push (default: `3000`) |
+
+For the full single-host deployment baseline, see [centralized-service-deployment.md](./centralized-service-deployment.md).
+
 ### Vault
 
 | Variable | Required | Description |
@@ -73,8 +88,11 @@ The agent uses [Codex SDK](https://www.npmjs.com/package/@openai/codex-sdk) to p
 | `WIKI_AGENT_API_KEY` | If enabled | API key for the LLM provider |
 | `WIKI_AGENT_MODEL` | No | Model name (e.g. `gpt-5.4`, `Qwen/Qwen3.5-397B-A17B-GPTQ-Int4`) |
 | `WIKI_AGENT_BATCH_SIZE` | No | Max concurrent vault items per batch (default: `5`) |
+| `WIKI_AGENT_SANDBOX_MODE` | No | Codex sandbox mode: `danger-full-access` (default) or `workspace-write` |
 | `WIKI_PARSER_SKILLS` | No | Comma-separated parser skill list (e.g. `pdf,docx,pptx,xlsx`) |
 
+`tiangong-wiki setup` now prompts for `WIKI_AGENT_SANDBOX_MODE` when automatic vault processing is enabled. The default is `danger-full-access`, and the setup wizard highlights that this mode grants full runtime access.
+
 ---
 
 ## Common Issues
@@ -102,6 +120,10 @@ Always run `tiangong-wiki lint --path <page-id> --format json` after mutations.
 
 Parser skills must be installed under `<workspace-root>/.agents/skills/`. Run `tiangong-wiki skill` to inspect installed skills. Use `tiangong-wiki skill update --all` to update.
 
+### Codex workflow sandbox fails to initialize
+
+If the agent workflow fails with `bwrap`, `unshare`, `uid_map`, or similar sandbox startup errors, switch `WIKI_AGENT_SANDBOX_MODE` to `danger-full-access`. Use `workspace-write` only when you explicitly want that sandbox mode and know the host supports it.
+
 ---
 
 ## LLM Provider Setup