@researai/deepscientist 1.5.0 → 1.5.1

package/docs/en/03_QQ_CONNECTOR_GUIDE.md

@@ -0,0 +1,325 @@
+# 03 QQ Connector Guide: Use QQ With DeepScientist
+
+This guide explains how to talk to DeepScientist through QQ and how to configure the QQ connector from the DeepScientist `Settings` page.
+
+This guide assumes the current built-in DeepScientist QQ runtime:
+
+- official QQ Gateway direct connection
+- no public callback URL required
+- no `relay_url` required
+- no extra QQ plugin installation
+
+If you previously followed an OpenClaw or NanoClaw article, note the key difference: DeepScientist already includes the QQ connector runtime. You configure it from `Settings > Connectors > QQ` instead of installing a separate plugin from the CLI.
+
+## 1. What you get after setup
+
+After finishing this guide, you should be able to:
+
+- chat with DeepScientist from QQ private messages
+- let QQ auto-bind to the latest active quest
+- use `/new`, `/use latest`, `/status`, and related commands from QQ
+- see the detected `openid` in the `Settings` page
+- run safe readiness checks and send probes from the `Settings` page
+
+### Deployment checklist before you start
+
+Before configuring anything, make sure all of these are already true:
+
+- DeepScientist is installed and the daemon / web UI is already running
+- you can open `Settings > Connectors`
+- you already have the QQ bot `AppID` and `AppSecret`
+- you have a real QQ account ready to send the first private message to the bot
+- if you changed QQ settings before, `Restart gateway on config change` should stay enabled
+
+If any of the items above is still missing, fix that first. It saves a lot of unnecessary debugging later.
+
+## 2. Register a QQ bot
+
+This section is based on the Tencent Cloud developer article “OpenClaw 接入 QQ 机器人”:
+
+- Source article: https://cloud.tencent.com/developer/article/2635190
+- Official QQ Bot platform: https://bot.q.qq.com/
+
+Use the official QQ Bot platform as the primary source of truth. The screenshot below is included only to make the registration flow easier to recognize.
+
+### 2.1 Open the bot registration entry
+
+Prefer the official platform:
+
+```text
+https://bot.q.qq.com/
+```
+
+The Tencent Cloud article also shows this quick-entry page:
+
+```text
+https://q.qq.com/qqbot/openclaw/login.html
+```
+
+![QQ bot registration entry](../images/qq/tencent-cloud-qq-register.png)
+
+### 2.2 Sign in and create the bot
+
+1. Sign in with QQ by scanning the QR code.
+2. Create a new QQ bot.
+3. Finish the basic bot setup in the Tencent console.
+
+### 2.3 Save `AppID` and `AppSecret` immediately
+
+After creation, record these two values right away:
+
+| Field | Meaning | Required by DeepScientist |
+| --- | --- | --- |
+| `AppID` | Unique bot identifier | Yes |
+| `AppSecret` | Secret used to call the QQ Bot API | Yes |
+
+Important pitfalls:
+
+- `AppSecret` is usually shown only once when created or reset.
+- If you lose it, you will normally need to reset it in the console.
+- DeepScientist only needs these two credentials to start the built-in QQ gateway direct path.
+
+## 3. What you do **not** need in DeepScientist
+
+This matters a lot if you are coming from an OpenClaw-style guide.
+
+In DeepScientist, you do **not** need to:
+
+- install `@sliverp/qqbot`
+- run `openclaw channels add ...`
+- configure a public callback URL
+- configure `relay_url`
+- manually fill `openid` before the first inbound QQ message
+
+DeepScientist currently uses this path:
+
+- fixed `transport: gateway_direct`
+- direct `app_id + app_secret`
+- automatic `openid` detection after the first private QQ message
+
+## 4. Configure QQ from the Settings page
+
+Open:
+
+- [Settings > Connectors > QQ](/settings/connectors#connector-qq)
+
+The link jumps directly to the `QQ` connector card. The page is now organized into these anchored setup steps:
+
+- `#connector-qq-step-credentials`: enter `App ID` and `App Secret`
+- `#connector-qq-step-bind`: save first, then send the first QQ private message
+- `#connector-qq-step-success`: confirm the auto-detected OpenID
+- `#connector-qq-step-advanced`: advanced settings and milestone delivery switches
+
+### 4.1 Recommended field order
+
+| Field | How to fill it | Recommendation |
+| --- | --- | --- |
+| `Enabled` | Turn it on | Required |
+| `Transport` | Keep `gateway_direct` | Fixed, do not change |
+| `App ID` | Paste the `AppID` from the QQ bot console | Required |
+| `App secret` | Paste the `AppSecret` from the QQ bot console | Required |
+| `Detected OpenID` | Leave empty at first | Auto-filled after the first private message |
+
+The remaining settings such as group mention policy, command prefix, auto-binding, and milestone delivery live under `#connector-qq-step-advanced`. Milestone delivery now defaults to fully enabled; only change those switches if you want less outbound push.
+
+### 4.2 Important things to notice before saving
+
+- QQ does not need `public_callback_url`
+- QQ does not need `relay_url`
+- `Detected OpenID` is not something you must fill before the first test
+- It is normal for `Detected OpenID` to stay empty until the bot receives the first private QQ message
+
+## 5. Recommended end-to-end test flow
+
+This is the most reliable path with the fewest surprises.
+
+### Step 1: save the credentials
+
+Fill:
+
+- `App ID`
+- `App secret`
+
+Then save the connector config.
+
+### Step 2: click “Check”
+
+From the `Settings` page, click:
+
+- `Check`
+
+Expected result:
+
+- no more missing-credential errors for `app_id` or `app_secret`
+
+### Step 3: click “Test all” or QQ-specific “Send probe”
+
+Before the first inbound private message, you may still see a warning like:
+
+```text
+QQ readiness is healthy, but no OpenID has been learned yet. Save credentials, then send one private QQ message so DeepScientist can auto-detect and save the `openid`.
+```
+
+This usually does **not** mean your credentials are wrong. It means:
+
+- DeepScientist can already exchange `access_token` and probe `/gateway`
+- but it still does not know which QQ `openid` should receive an active outbound test message
+
+In other words:
+
+- a successful readiness check is not the same as already having a delivery target
+
+### Step 4: send the first private QQ message to the bot
+
+Start with a private chat, not a group.
+
+![QQ private chat with the bot](../images/qq/tencent-cloud-qq-chat.png)
+
+You can send:
+
+```text
+/help
+```
+
+or:
+
+```text
+hello
+```
+
+If the connector is working, DeepScientist will auto-detect the `openid` for that private conversation and save it into `main_chat_id`.
+
+### Step 5: go back to the Settings page and confirm the state
+
+Refresh or reopen `Settings > Connectors > QQ`, or jump back to [#connector-qq-step-success](/settings/connectors#connector-qq-step-success), and check:
+
+- whether `Detected OpenID` is now filled automatically
+- whether the `Snapshot` panel shows something close to:
+  - `Transport = gateway_direct`
+  - `Connection` / `Auth` near `ready`
+  - `Discovered targets > 0`
+  - `Bound target` showing your `openid`
+
+### Step 6: click “Send probe” again
+
+Now click:
+
+- `Send probe`
+
+If it succeeds, DeepScientist can now:
+
+- actively send messages to that QQ user
+- receive new messages from that QQ user
+
+### 5.1 What success looks like
+
+When the connector is fully working, you should usually see all of these:
+
+- the bot replies to `/help`, `/status`, and similar commands
+- `Detected OpenID` is no longer empty in `Settings > Connectors > QQ`
+- the `Snapshot` panel shows a discovered target and the bound target is no longer empty
+- clicking `Send probe` again no longer reports an empty delivery target
+- if a latest quest already exists, plain text continues that quest automatically; if no quest exists yet, the bot returns help instead
+
+### 5.2 Error quick decoder
+
+| Message | What it usually means | What to do |
+| --- | --- | --- |
+| `app_id is required` / `app_secret is required` | The credentials are incomplete | Fill the missing field in Settings and save again |
+| `401` / `invalid credential` / token exchange failed | The `AppID` or `AppSecret` is wrong, or the secret was reset | Recheck the QQ console values and save them again |
+| `QQ readiness is healthy, but no OpenID has been learned yet...` | The credentials probably work, but DeepScientist still does not know which QQ user should receive an active outbound message | Send one private QQ message to the bot first so the runtime can discover the `openid` |
+| `QQ callback flow usually needs public_callback_url...` | This comes from an older callback/relay model, not the current DeepScientist direct path | Keep `transport = gateway_direct` and do not add a public callback URL |
+| `QQ relay mode needs relay_url...` | The transport was switched to relay mode by mistake | Change it back to `gateway_direct` |
+| `Detected OpenID` stays empty | The bot has not received the first private QQ message yet, or the gateway did not restart cleanly after config changes | Save the config first, then send a private QQ message, and restart the gateway if needed |
+
+## 6. How to talk to DeepScientist from QQ
+
+Common commands:
+
+| Command | Meaning |
+| --- | --- |
+| `/help` | Show help |
+| `/projects` or `/list` | List quests |
+| `/use <quest_id>` | Bind a specific quest |
+| `/use latest` | Bind the newest quest |
+| `/new <goal>` | Create a new quest and bind the current QQ conversation |
+| `/status` | Show the current quest status |
+
+Recommended usage:
+
+- if a latest quest already exists, plain text usually continues that quest
+- if there is no quest yet, start with `/new <goal>`
+- if you want to switch to another quest, send `/use <quest_id>` explicitly
+
+## 7. Most common mistakes
+
+### 7.1 Assuming QQ needs a public callback
+
+The current DeepScientist QQ path does **not** require a public callback.
+
+If you see these ideas from older guides, ignore them for DeepScientist:
+
+- `public_callback_url`
+- `relay_url`
+
+### 7.2 Assuming a failed active send test means the credentials are wrong
+
+If the warning says the target is empty, the usual reason is:
+
+- you have not sent the first private QQ message yet
+- the runtime has not auto-detected the `openid` yet
+
+That is a different problem from invalid `app_id` or `app_secret`.
+
+### 7.3 Thinking you must manually discover `openid`
+
+In DeepScientist, the easiest path is not manual lookup. Instead:
+
+1. save `App ID` and `App secret`
+2. send one private QQ message to the bot
+3. let the runtime auto-fill `Detected OpenID`
+
+### 7.4 Bot does not respond in a group
+
+Check:
+
+- whether you actually `@` mentioned the bot
+- whether `Require @ mention in groups` is enabled
+
+If private chat is not working yet, do not start group debugging first.
+
+### 7.5 A different operator now uses QQ, but `main_chat_id` is still the old one
+
+If a previous `main_chat_id` is already saved, a new private user may not automatically overwrite it.
+
+In that case:
+
+- decide who the primary QQ operator should be
+- clear or update the QQ target explicitly if needed
+- then rerun the private-message test flow
+
+## 8. Smallest reliable deployment sequence
+
+If you only want the shortest path that works:
+
+1. create the QQ bot
+2. save `AppID` and `AppSecret`
+3. open [Settings > Connectors > QQ](/settings/connectors#connector-qq)
+4. enable QQ and fill `App ID` / `App secret`
+5. save and click `Check`
+6. send `/help` to the bot from your QQ private chat
+7. verify that `Detected OpenID` is now filled
+8. click `Send probe` again
+9. start using `/new <goal>` or `/use latest`
+
+## 9. References
+
+- Tencent Cloud developer article: “OpenClaw 接入 QQ 机器人”
+  - https://cloud.tencent.com/developer/article/2635190
+- Official QQ Bot platform
+  - https://bot.q.qq.com/
+
+Notes:
+
+- the bot-registration flow and screenshots in this guide are based on the Tencent Cloud article above
+- the actual DeepScientist connector behavior follows the current built-in DeepScientist QQ runtime, not the OpenClaw plugin workflow

package/docs/en/04_LINGZHU_CONNECTOR_GUIDE.md

@@ -0,0 +1,216 @@
+# 04 Lingzhu Connector Guide: Configure Lingzhu for DeepScientist
+
+This guide explains how to configure the Lingzhu companion endpoint from `Settings > Connectors > Lingzhu`.
+
+Scope:
+
+- configuration only
+- OpenClaw-compatible Lingzhu bridge values
+- local probe and SSE smoke test
+- what to paste into the Lingzhu platform
+
+This guide does not cover cloud deployment architecture. You only need one working OpenClaw gateway and one public IP or public domain that the glasses can reach.
+
+Reference sources:
+
+- Rokid forum setup page: https://forum.rokid.com/post/detail/2831
+- packaged bridge root: `assets/connectors/lingzhu/openclaw-bridge`
+- packaged OpenClaw config template: `assets/connectors/lingzhu/openclaw.lingzhu.config.template.json`
+
+## 1. What DeepScientist now provides
+
+The Lingzhu settings card now generates and validates:
+
+- local health URL
+- local SSE URL
+- public SSE URL
+- `auth_ak`
+- OpenClaw config snippet
+- local probe `curl`
+
+The packaged bridge is configured to:
+
+- emit one immediate visible receipt acknowledgement before the model starts
+- keep comment heartbeats alive
+- emit lightweight visible progress heartbeats during long silent upstream phases
+
+DeepScientist treats Lingzhu as a companion endpoint, not as a full bidirectional chat connector like QQ.
+
+## 2. Before you start
+
+Make sure these are already true:
+
+- DeepScientist is installed and running
+- your OpenClaw gateway is already running locally
+- the OpenClaw HTTP gateway port is known, usually `18789`
+- you have a public IP or public domain
+
+Important:
+
+- `127.0.0.1` only works for local health checks
+- Lingzhu devices need a public address
+
+## 3. Open the settings page
+
+Open:
+
+- `Settings > Connectors > Lingzhu`
+
+The card is organized into:
+
+- gateway endpoint
+- auth and identity
+- generated values
+- probe and verify
+- advanced debug
+
+![Lingzhu settings overview](../images/lingzhu/lingzhu-settings-overview.svg)
+
+## 4. Fill the endpoint fields
+
+Recommended values:
+
+| Field | Recommended value | Notes |
+| --- | --- | --- |
+| `Enabled` | `true` | turn on the companion config |
+| `Transport` | `openclaw_sse` | fixed, do not change |
+| `Local host` | `127.0.0.1` | used only for local probe |
+| `Gateway port` | `18789` | match your OpenClaw gateway |
+| `Public base URL` | `http://<public-ip>:18789` | must be reachable from the glasses |
+
+If you do not know what to fill for the local endpoint, click:
+
+- `Use local defaults`
+
+## 5. Generate the AK and identity values
+
+Fill or keep:
+
+| Field | Recommended value |
+| --- | --- |
+| `Auth AK` | click `Generate AK` |
+| `Agent ID` | `main` |
+| `Lingzhu system prompt` | optional |
+
+Use the same `auth_ak` and `agent_id` in both places:
+
+- OpenClaw Lingzhu plugin config
+- Lingzhu platform config
+
+## 6. Use the generated values
+
+DeepScientist shows the exact values you need:
+
+- local health URL
+- local SSE URL
+- public SSE URL
+- OpenClaw config snippet
+- probe curl
+
+The public value is the one you should paste into the Lingzhu platform.
+
+![Lingzhu platform values](../images/lingzhu/lingzhu-platform-values.svg)
+
+## 7. Update OpenClaw
+
+Use either:
+
+- the generated snippet in the settings page
+- or the packaged template at `assets/connectors/lingzhu/openclaw.lingzhu.config.template.json`
+
+Install the packaged bridge with:
+
+```bash
+openclaw plugins install ./assets/connectors/lingzhu/openclaw-bridge
+```
+
+At minimum, OpenClaw must expose:
+
+```json
+{
+  "gateway": {
+    "port": 18789,
+    "http": {
+      "endpoints": {
+        "chatCompletions": {
+          "enabled": true
+        }
+      }
+    }
+  }
+}
+```
+
+DeepScientist also generates the full Lingzhu plugin block for you:
+
+![OpenClaw config snippet](../images/lingzhu/lingzhu-openclaw-config.svg)
+
+## 8. Run the local probe
+
+After saving the values, click:
+
+- `Run Lingzhu probe`
+
+DeepScientist performs:
+
+1. `GET /metis/agent/api/health`
+2. `POST /metis/agent/api/sse`
+
+Expected result:
+
+- `Connection = reachable`
+- `Auth = ready`
+- probe result returns no errors
+
+## 9. What to paste into the Lingzhu platform
+
+Use:
+
+- `Public SSE URL`
+- `Auth AK`
+
+Do not paste:
+
+- `127.0.0.1`
+- any local-only host name
+
+## 10. Common failure causes
+
+### Health is offline
+
+Usually means:
+
+- OpenClaw is not running
+- wrong `gateway_port`
+- wrong `local_host`
+
+### SSE probe fails
+
+Usually means:
+
+- `auth_ak` mismatch
+- Lingzhu endpoint path is not exposed
+- OpenClaw `chatCompletions.enabled` is still off
+
+### Device still cannot connect
+
+Usually means:
+
+- `Public base URL` is not public
+- firewall or reverse proxy still blocks the port
+
+## 11. Practical note
+
+This settings surface is intentionally registry-first:
+
+- Lingzhu stays in `connectors.yaml`
+- generated values come from the same structured config
+- validation, testing, and frontend rendering consume the same connector record
+
+For long-running tasks, the most stable practical behavior is:
+
+- immediate bridge-level receipt acknowledgement
+- visible progress heartbeats from the bridge during silent upstream phases
+- preserved per-user session continuity across follow-up turns
+
+This improves Lingzhu compatibility, but it still does not remove the platform-side single-request timeout limit.

package/docs/en/05_TUI_GUIDE.md

@@ -0,0 +1,141 @@
+# 05 TUI Guide: Use the Terminal Interface
+
+This document is the single repo-level reference for the current DeepScientist TUI workflow.
+
+For the current runtime control flow, prompt/skill execution model, MCP surface, and Canvas reconstruction logic, also see `docs/en/06_RUNTIME_AND_CANVAS.md`.
+
+## Install And Start
+
+Use the current workspace install:
+
+```bash
+pip install -e .
+npm install
+```
+
+Common launcher commands:
+
+```bash
+ds
+ds --tui
+ds --both
+ds --status
+ds --stop
+```
+
+Notes:
+
+- `ds` starts the managed daemon, prints the local web URL, tries to open the browser, and exits.
+- `ds --tui` starts daemon plus the Ink-based terminal workspace.
+- `ds --both` starts the daemon, opens the web workspace, and keeps the terminal workspace attached.
+- `ds --stop` stops the managed daemon itself, not just one quest.
+- `python -m deepscientist.cli ...` remains available for low-level commands, but the launcher is the normal path.
+- from a source checkout, `node bin/ds.js` behaves the same as `ds`
+
+## Core TUI Flow
+
+Inside the TUI:
+
+- `/home`: leave the current quest and return to request mode.
+- `/projects`: open the quest browser.
+- `/use <quest_id>`: bind the current TUI session to one quest.
+- `/new`: create a quest from inside the TUI.
+- `/new <goal>` creates the quest and auto-starts it through the daemon.
+- `/delete <quest_id> --yes`: delete a quest (destructive; requires confirmation).
+- plain text: send a normal user message to the bound quest.
+- `/status`: inspect the current quest state.
+- `/graph`: inspect the current quest graph.
+- `/help`: show the in-TUI command list and control keys.
+- `/config`: open the local config browser.
+- `/pause`: pause the current quest; if no quest is bound, choose one in the panel.
+- `/resume`: resume the current quest; if no quest is bound, choose one in the panel.
+- `/stop`: stop the current quest; if no quest is bound, choose one in the panel.
+- `/stop <quest_id>`: stop a specific quest explicitly.
+- typing `/` in the input box shows a live command list; typing `/re`, `/co`, and similar prefixes filters the visible command rows
+- in home mode, `↑/↓` and `Tab` only change the selected quest preview; they do not hard-switch an already bound quest
+- in home mode, plain text does not create or send anything implicitly; use `/new <goal>` to create and `/projects` or `/use <quest_id>` to bind before chatting
+
+If you are already inside a quest, `/pause`, `/resume`, and `/stop` target that quest automatically.
+
+The footer and welcome banner also expose the main shortcuts directly:
+
+- `Enter`: send or confirm
+- `↑/↓`: browse selectors
+- `Esc`: close the active overlay
+- `Ctrl+O`: open the web workspace
+- `Ctrl+C`: quit the TUI
+
+## Message Delivery Model
+
+The TUI, web UI, and connectors now share the same mailbox semantics.
+
+1. The first plain user message for an idle quest starts a turn directly.
+2. That first launch message is claimed by the run and does not get re-delivered later through the mailbox.
+3. Later user messages that arrive while the agent is already running are written into `.ds/user_message_queue.json`.
+4. Those queued messages are delivered only when the agent calls `artifact.interact(...)`.
+5. When delivery happens, the agent receives the queued bundle as the latest user requirements and the queue entries move from `pending` to completed audit records.
+6. If there is no new user message and the quest is still unfinished, the runtime can auto-start another turn so the agent keeps advancing from durable state instead of idling.
+7. If there is no new user message, the runtime returns recent interaction records plus the instruction that the user did not send anything new.
+
+Durable files used by this flow:
+
+- `.ds/runtime_state.json`
+- `.ds/user_message_queue.json`
+- `.ds/interaction_journal.jsonl`
+- `.ds/events.jsonl`
+
+## Stop, Pause, Resume
+
+Quest-level control:
+
+- `/pause` interrupts the active runner and marks the quest as `paused`.
+- `/resume` moves a `paused` or `stopped` quest back to `active`.
+- `/stop` interrupts the active runner, clears `active_run_id`, and marks the quest as `stopped`.
+- `/pause`, `/resume`, and `/stop` also append a visible assistant-side control notice into the quest chat history and push the same notice to bound connectors using the normal connector routing policy.
+
+`/stop` semantics are intentionally stronger than `/pause`:
+
+- queued but not yet delivered user mailbox messages are cancelled
+- those cancelled messages are preserved in the queue audit trail with terminal status fields such as `cancelled_by_stop`
+- the currently executing launch message is recorded as `accepted_by_run`
+- the next user message after stop starts a fresh turn in the same quest context without silently replaying stale queued text
+- stop does not rewrite Git state: the current branch, worktree, and already written files remain in place, so follow-up work resumes from the same repository context
+
+Daemon-level stop:
+
+- `ds --stop` shuts down the managed daemon
+- it validates both `home` and `daemon_id` before touching the daemon
+- it first asks the daemon to stop cleanly
+- if needed it escalates to `SIGTERM`, then `SIGKILL`
+- if shutdown succeeds, the launcher clears the daemon state file and prints `DeepScientist daemon stopped.`
+- a clean daemon shutdown stops any actively running quests first, emits the same stop notice, and leaves Git/worktree state untouched for the next daemon start
+
+## Artifact Interaction Expectations
+
+The agent should use `artifact.interact(...)` as the long-lived conversation thread:
+
+- `progress` and `milestone` updates are threaded, non-blocking status messages
+- `decision_request` is the blocking form
+- a blocking decision request should provide 1 to 3 concrete options, explain pros and cons, wait up to 1 day when feasible, then self-resolve and notify the user if the timeout expires
+- true quest completion is separate from a normal decision request: the agent should first ask for explicit completion approval with `reply_schema.decision_type = quest_completion_approval`, and only then call `artifact.complete_quest(...)`
+
+## Troubleshooting
+
+If the TUI looks idle after you send text:
+
+- confirm the quest is actually bound
+- check the quest status with `/status`
+- inspect `.ds/runtime_state.json` for `status`, `active_run_id`, and `pending_user_message_count`
+- inspect `.ds/events.jsonl` for `runner.turn_error`, `quest.control`, or `artifact.recorded`
+
+If a follow-up user message does not seem to reach the agent:
+
+- check whether the quest is currently running
+- inspect `.ds/user_message_queue.json`
+- confirm the agent is calling `artifact.interact(...)` during long work
+
+If stop appears ineffective:
+
+- use `/stop` inside the quest first
+- verify the quest becomes `stopped`
+- for a full daemon shutdown use `ds --stop`