@researai/deepscientist 1.5.15 → 1.5.17

package/docs/en/15_CODEX_PROVIDER_SETUP.md

@@ -2,6 +2,8 @@
 
 DeepScientist does not implement separate provider adapters for MiniMax, GLM, Volcengine Ark, or Alibaba Bailian.
 
+For Qwen on Alibaba Bailian, DeepScientist only supports the **Coding Plan** path. The generic Bailian or DashScope Qwen platform API is not supported here.
+
 Instead, it reuses the same Codex CLI setup that already works in your terminal.
 
 The recommended order is always:
@@ -18,7 +20,7 @@ The recommended order is always:
 Use this when your Codex CLI works through the standard OpenAI login flow.
 
 ```bash
-codex --login
+codex login
 ds doctor
 ds
 ```
@@ -61,7 +63,8 @@ codex:
 Important:
 
 - keep `model: inherit` for provider-backed Codex profiles unless you are certain the provider accepts the explicit model id you plan to send
-- DeepScientist will reuse the same `~/.codex/config.toml` and environment that your terminal Codex already uses
+- DeepScientist now launches Codex from an isolated runtime home under `.ds/codex-home`, but that runtime copy inherits your configured `~/.codex` auth, config, skills, agents, and prompts first
+- if the active provider uses `wire_api = "chat"`, DeepScientist now auto-checks that the selected Codex binary is exactly `0.57.0` during startup probe
 
 ## Provider matrix
 
@@ -71,14 +74,14 @@ Important:
 | MiniMax | [MiniMax Codex CLI](https://platform.minimaxi.com/docs/coding-plan/codex-cli) | No | your Codex profile, for example `ds --codex-profile m27` |
 | GLM | [GLM Coding Plan: Other Tools](https://docs.bigmodel.cn/cn/coding-plan/tool/others) | No | a Codex profile that targets the GLM coding endpoint |
 | Volcengine Ark | [Ark Coding Plan Overview](https://www.volcengine.com/docs/82379/1925114?lang=zh) | No | a Codex profile that targets the Ark coding endpoint |
-| Alibaba Bailian | [Bailian Coding Plan: Other Tools](https://help.aliyun.com/zh/model-studio/other-tools-coding-plan) | No | a Codex profile that targets the Bailian coding endpoint |
+| Alibaba Bailian | [Bailian Coding Plan: Other Tools](https://help.aliyun.com/zh/model-studio/other-tools-coding-plan) | No | a Codex profile that targets the Bailian Coding Plan endpoint; do not use the generic Bailian or DashScope Qwen API |
 
 ## OpenAI
 
 ### What to prepare
 
 - a normal Codex CLI install
-- a successful `codex --login` or `codex` interactive first-run setup
+- a successful `codex login` or `codex` interactive first-run setup
 
 ### DeepScientist commands
 
@@ -108,13 +111,13 @@ Official doc:
 
 ### Verified compatibility note
 
-Checked against MiniMax's current Codex CLI doc and local compatibility validation on 2026-03-25:
+Checked against MiniMax's current Codex CLI doc and local compatibility validation on 2026-04-04:
 
 - MiniMax's Codex CLI page currently recommends `@openai/codex@0.57.0`
 - the Coding Plan endpoint to use is `https://api.minimaxi.com/v1`
 - MiniMax's official page uses `m21` as the profile name, but that profile name is only a local alias; this repo uses `m27` consistently in examples
 - the `codex-MiniMax-*` model names shown on MiniMax's page did not pass reliably through Codex CLI in local testing with the provided key
-- the locally verified working path was `MiniMax-M2.7` + `m27` + `model: inherit` + Codex CLI `0.57.0`
+- the locally verified DeepScientist working paths were `MiniMax-M2.7` + `m27` + Codex CLI `0.57.0` and `MiniMax-M2.5` + `m25` + Codex CLI `0.57.0`
 - the current `@openai/codex` latest release still does not line up cleanly with MiniMax's current guide
 
 If you want the most reproducible DeepScientist + MiniMax path today, use Codex CLI `0.57.0`.
@@ -124,7 +127,7 @@ If you want the most reproducible DeepScientist + MiniMax path today, use Codex
 - Codex CLI `0.57.0`
 - a MiniMax `Coding Plan Key`
 - `MINIMAX_API_KEY` available in the shell that starts Codex and DeepScientist
-- the current shell cleared of `OPENAI_API_KEY` and `OPENAI_BASE_URL`
+- for plain terminal `codex --profile <name>` checks, the current shell cleared of `OPENAI_API_KEY` and `OPENAI_BASE_URL`
 - a working Codex profile in `~/.codex/config.toml`
 
 ### Install Codex CLI `0.57.0`
@@ -144,6 +147,8 @@ codex-cli 0.57.0
 
 If you want to keep another Codex version elsewhere, create a small wrapper script and point `runners.codex.binary` at that absolute path.
 
+When DeepScientist detects a MiniMax profile at startup and the active Codex CLI is not `0.57.0`, it now prompts to reinstall `@openai/codex@0.57.0` automatically in interactive terminal launches.
+
 ### Codex-side setup
 
 Use `https://api.minimaxi.com/v1`, not `https://api.minimax.io/v1`.
@@ -156,15 +161,19 @@ unset OPENAI_BASE_URL
 export MINIMAX_API_KEY="..."
 ```
 
+For plain terminal validation, keep doing that exactly as shown above.
+For the DeepScientist path, when the selected provider sets `requires_openai_auth = false`, DeepScientist now strips `OPENAI_API_KEY` and `OPENAI_BASE_URL` automatically during the startup probe and real runner execution.
+
 MiniMax's official page uses `m21` as the example profile name. Since the profile name is only a local alias, this repo rewrites that example to `m27`.
 
 The important difference is the model name:
 
 - MiniMax's page currently shows `codex-MiniMax-M2.5`
-- in local testing, direct MiniMax API calls worked with `MiniMax-M2.7`
-- with the same key, `codex-MiniMax-M2.5` and `codex-MiniMax-M2.7` both failed through Codex CLI
+- in local testing, direct MiniMax API calls worked with `MiniMax-M2.7` and `MiniMax-M2.5`
+- the reproducible DeepScientist paths were `MiniMax-M2.7` on profile `m27` and `MiniMax-M2.5` on profile `m25`
+- for the `m25` path, use `MiniMax-M2.5`, not `codex-MiniMax-M2.5`
 
-So the config below is the currently recommended DeepScientist working configuration:
+So the config below is the currently recommended DeepScientist configuration:
 
 ```toml
 [model_providers.minimax]
@@ -182,9 +191,20 @@ model = "MiniMax-M2.7"
 model_provider = "minimax"
 ```
 
+If you want the same DeepScientist path on `m25`, keep the provider block unchanged and use:
+
+```toml
+[profiles.m25]
+model = "MiniMax-M2.5"
+model_provider = "minimax"
+```
+
 What DeepScientist supports now:
 
-- if you use this profile-only MiniMax config with Codex CLI `0.57.0`, DeepScientist automatically promotes the selected profile's `model_provider` and `model` to the top level inside its probe/runtime copy of `.codex/config.toml`
+- if you use this profile-only MiniMax config with Codex CLI `0.57.0`, DeepScientist automatically promotes the selected profile's `model_provider` and `model` to the top level inside its probe/runtime copy of `config.toml`
+- DeepScientist forces provider-backed MiniMax runs to use `model: inherit`, so it does not accidentally override the profile with a hard-coded OpenAI model
+- when `requires_openai_auth = false`, DeepScientist strips conflicting `OPENAI_API_KEY` and `OPENAI_BASE_URL` values from the probe/runtime environment
+- for chat-wire provider sessions such as MiniMax on Codex CLI `0.57.0`, DeepScientist now injects a compatibility guard that tells Codex to serialize MCP tool calls one at a time instead of bundling multiple tool calls into the same response
 - this means DeepScientist can start even when plain terminal `codex --profile m27` still fails on that exact profile-only shape
 
 If you want plain terminal `codex --profile <name>` to work too, use the explicit top-level compatibility form instead:
@@ -247,6 +267,7 @@ DeepScientist now does two MiniMax-specific compatibility steps for the `0.57.0`
 
 - it downgrades `xhigh` to `high` automatically when the Codex CLI does not support `xhigh`
 - it auto-adapts MiniMax's profile-only `model_provider` / `model` shape inside the temporary DeepScientist Codex home when needed
+- it removes conflicting `OPENAI_*` auth variables automatically when the provider explicitly says `requires_openai_auth = false`
 
 ## GLM
 
@@ -333,6 +354,11 @@ codex:
 
 Bailian documents Coding Plan as an OpenAI-compatible coding endpoint. It requires the Coding Plan-specific key and endpoint, not the generic platform endpoint.
 
+For Qwen specifically:
+
+- supported: Qwen through the Bailian **Coding Plan** endpoint
+- not supported: the generic Bailian or DashScope Qwen platform API
+
 Official docs:
 
 - <https://help.aliyun.com/zh/model-studio/other-tools-coding-plan>

package/docs/en/19_EXTERNAL_CONTROLLER_GUIDE.md

@@ -0,0 +1,226 @@
+# 19 External Controller Guide
+
+DeepScientist already exposes enough durable state to support an outer orchestration layer without patching core runtime code.
+
+This guide explains the minimal public pattern for external controllers that:
+
+- inspect recent quest state
+- decide whether the current run should continue
+- enqueue a routed follow-up message through the quest mailbox
+- optionally stop the current run through `quest_control`
+- record a durable report explaining why the guard fired
+
+This is intentionally lighter than a plugin framework.
+
+## When to use an external controller
+
+Use an external controller when the rule is:
+
+- project-specific
+- expensive to hard-code into global prompts or skills
+- better treated as outer governance than as core runtime behavior
+
+Examples:
+
+- publishability admission rules before paper-facing writing
+- repeated figure-polish loops that monopolize the frontier
+- lab-specific stop / branch policies
+
+## Public contracts you can rely on
+
+The safest extension surface is the existing durable runtime contract:
+
+- quest mailbox
+  - queued user-facing messages are stored under `.ds/user_message_queue.json`
+- recent quest state
+  - runtime state, artifact state, and connector-visible outputs are already durable files
+- daemon quest control
+  - `POST /api/quests/<quest_id>/control`
+- connector-visible durable reports
+  - write your own report under the quest tree so the next turn can cite it
+
+Prefer these contracts over prompt patching, private monkey-patching, or editing installed package files.
+
+## Minimal controller loop
+
+An external controller usually follows this sequence:
+
+1. Read the latest durable quest state.
+2. Decide whether a guard condition is active.
+3. Write a durable report describing:
+   - what was observed
+   - why it matters
+   - the recommended next route
+4. If intervention is needed:
+   - optionally stop the current run through `quest_control`
+   - enqueue one clear routed mailbox message for the next turn
+
+The mailbox message should explain the conclusion, not dump raw logs.
+
+## Example control flow
+
+```text
+read quest state
+-> detect low-yield loop or route violation
+-> write durable report
+-> stop current run if needed
+-> enqueue one mailbox message with the required next route
+```
+
+## Example `quest_control` request
+
+```bash
+curl -X POST http://127.0.0.1:20999/api/quests/<quest_id>/control \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "action": "stop",
+    "source": "external-controller"
+  }'
+```
+
+## Example mailbox intervention shape
+
+The exact queue file is runtime-owned, so your controller should preserve the existing schema and only append a normal user-style message payload.
+
+The message content should be short and actionable, for example:
+
+```text
+Hard control message from external orchestration layer: stop the current figure loop.
+Return to the main line and do one bounded route next:
+1. literature scout
+2. reference expansion
+3. manuscript body revision
+```
+
+## Example controllers you can adapt
+
+### Example 1: publishability admission guard
+
+This controller is useful when a quest is drifting into paper-facing writing before the evidence line is ready.
+
+Typical inputs:
+
+- the latest verification note
+- the current draft / summary state
+- recent baseline or utility results
+
+Typical stop condition:
+
+- the claimed paper direction still has weak support
+- one or more mandatory evidence items are still missing
+
+Typical intervention:
+
+1. Write `reports/publishability_guard.md` explaining the missing support.
+2. Stop the current write-heavy run through `quest_control`.
+3. Enqueue one mailbox message that routes the next turn back to `idea`, `analysis`, or a bounded evidence-repair step.
+
+Example mailbox text:
+
+```text
+External controller: do not continue manuscript-facing writing yet.
+Reason: the current evidence line does not pass the publishability admission gate.
+Next route: return to one bounded evidence-building step before write resumes.
+```
+
+### Example 2: figure-loop guard
+
+This controller is useful when the frontier is monopolized by repeated reopen / polish cycles on the same figure.
+
+Typical inputs:
+
+- recent figure artifact history
+- repeated reopen events
+- the latest review or summary note
+
+Typical stop condition:
+
+- the same figure has been reopened multiple times without improving the main claim
+- the next useful action is no longer figure polish, but evidence repair or manuscript revision
+
+Typical intervention:
+
+1. Write `reports/figure_loop_guard.md` summarizing the loop.
+2. Stop the current figure branch if needed.
+3. Enqueue one mailbox message that names exactly one next route.
+
+Example mailbox text:
+
+```text
+External controller: stop the current figure-polish loop.
+Reason: repeated reopen cycles are no longer improving the main evidence line.
+Next route: return to one bounded manuscript or analysis task.
+```
+
+## Example connector customization surface
+
+The control logic should stay connector-agnostic. In practice, adapting the same controller to a different connector usually means changing only the message profile, not the stop logic or durable report contract.
+
+For example:
+
+```yaml
+connector_profiles:
+  weixin:
+    summary_style: concise
+    max_route_options: 2
+    include_report_path: true
+  telegram:
+    summary_style: concise
+    max_route_options: 3
+    include_report_path: true
+  studio:
+    summary_style: detailed
+    max_route_options: 4
+    include_report_excerpt: true
+```
+
+The controller decision stays the same:
+
+- read durable state
+- decide whether to stop
+- write a durable report
+- enqueue one routed mailbox message
+
+What changes per connector is only how much detail you surface to the human operator.
+
+## Durable report shape
+
+Keep reports simple and auditable.
+
+A good report usually includes:
+
+- `generated_at`
+- `quest_id`
+- `status`
+- `recommended_action`
+- `blockers`
+- `evidence_summary`
+
+Markdown plus a machine-readable JSON companion is a practical pattern.
+
+## What not to rely on
+
+Avoid building controllers that depend on:
+
+- private prompt text offsets
+- internal temporary logs that are not documented durable state
+- patching installed package files inside `site-packages`
+- undocumented frontend-only state
+
+If a controller needs one of those, the contract is not stable enough yet.
+
+## Design recommendations
+
+- keep each controller focused on one question
+- prefer additive reports over hidden side effects
+- stop only when the next routed action is clear
+- keep domain- or lab-specific policy outside core defaults
+- treat external controllers as optional governance, not as required runtime plumbing
+
+## Good first controllers
+
+If you want to start small, begin with one of these:
+
+- a publishability admission guard for paper-mode quests
+- a figure-loop guard that stops repeated reopen cycles
+- a route-drift guard that blocks accidental `write` transitions before evidence is ready

package/docs/en/19_LOCAL_BROWSER_AUTH.md

@@ -0,0 +1,70 @@
+# Local Browser Auth
+
+DeepScientist can optionally protect the local web workspace with a generated 16-character password.
+
+This protection is local-first:
+
+- it applies to the browser UI
+- it also applies to `/api/*` requests from the browser
+- it is disabled by default unless you launch with `ds --auth true` or set `ui.auth_enabled: true`
+
+## What Happens On Startup
+
+When you run `ds --auth true`, DeepScientist starts the daemon and prints two important things in the terminal:
+
+- the local browser URL, such as `http://127.0.0.1:20999`
+- the generated local access password for that launch
+
+The browser URL is no longer expected to carry `?token=...`.
+
+If the browser is not already authenticated, DeepScientist shows a password modal on top of the landing page before it loads quests, docs, settings, or workspace data.
+
+## How To View The Password
+
+Use either of these:
+
+```bash
+ds --status
+```
+
+Read the `auth_token` field from the JSON output, or look at the terminal where `ds` was started.
+
+## How Login Persistence Works
+
+After the first successful login:
+
+- the browser stores the local auth state
+- later visits from the same browser usually open directly
+- restarting or reusing the managed daemon can rotate the password again
+
+If the password rotates, the browser will be asked to log in again.
+
+## How To Disable It
+
+Enable the browser password gate for one launch:
+
+```bash
+ds --auth true
+```
+
+In that mode:
+
+- the password modal is enabled
+- browser requests to `/api/*` require the local auth token
+
+To disable it again explicitly for one launch:
+
+```bash
+ds --auth false
+```
+
+In that mode:
+
+- the password modal is disabled
+- browser requests to `/api/*` do not require the local auth token
+
+## Practical Notes
+
+- This is not intended as internet-grade authentication. It is a local browser gate for machines you control.
+- If you share the machine or expose the port remotely, enable auth explicitly.
+- If you script against the local daemon from a browser client, authenticate first or disable auth explicitly for that launch.

package/docs/en/20_WORKSPACE_MODES_GUIDE.md

@@ -0,0 +1,250 @@
+# 20 Workspace Modes Guide: Copilot vs Autonomous
+
+This page explains the two normal project-start modes in DeepScientist:
+
+- `Copilot`
+- `Autonomous`
+
+Use this when:
+
+- you are about to create a new project
+- you are unsure which launch mode to choose
+- you want to understand why one project waits for you while another starts moving immediately
+
+If you only want the shortest install-and-launch path, read [00 Quick Start](./00_QUICK_START.md) first.
+
+If you want the exact startup payload and form field contract, read [02 Start Research Guide](./02_START_RESEARCH_GUIDE.md) after this page.
+
+## 1. One-sentence summary
+
+- `Copilot`: quiet start, user-directed, stops after the current requested unit unless you ask it to continue
+- `Autonomous`: standard DeepScientist, keeps pushing the quest forward on its own
+
+## 2. Where You Choose This
+
+On the home / projects surface, start a new project and then choose the start style:
+
+- `Copilot Mode`
+- `Autonomous Mode`
+
+Current UI wording may appear in two layers:
+
+- outer launcher step: `Start Research` or `Start Experiment`
+- mode picker title: `Choose the start style`
+
+After that choice, the two flows diverge.
+
+## 3. Copilot Mode
+
+### 3.1 What it is
+
+Copilot mode is the user-directed workspace.
+
+It is for cases where you want DeepScientist to help actively, but you still want to steer each unit of work:
+
+- inspect a repo
+- read a paper
+- debug code
+- design an experiment
+- check a running process
+- rewrite a section
+- summarize a result
+
+The system should not assume that one request means “run the whole autonomous research loop”.
+
+### 3.2 What happens after creation
+
+After you create a Copilot project:
+
+- the project is created
+- the quest stays idle
+- the agent waits for your first real instruction
+
+In practical terms:
+
+- there is no immediate autonomous launch
+- no baseline / experiment / writing loop starts just because the project exists
+- the first substantial action begins when you send the first message
+
+### 3.3 Continuation behavior
+
+Copilot mode is intentionally conservative about continuation.
+
+After the current requested unit is complete, DeepScientist should normally:
+
+- summarize what changed
+- preserve context durably
+- wait for your next message or `/resume`
+
+This is the right mode when you want:
+
+- tight human control
+- short request-scoped help
+- less background churn
+- clearer handoff points
+
+### 3.4 Good fit
+
+Choose `Copilot` when:
+
+- you want to inspect before launching expensive work
+- the task is still ambiguous
+- you expect to iterate interactively
+- you want DeepScientist to behave more like a strong research IDE partner than a long-running autonomous operator
+
+### 3.5 Bad fit
+
+Avoid `Copilot` when:
+
+- you already know the quest should keep running for hours
+- you want detached experiments, monitoring, and follow-up routing without repeated user nudges
+- the goal is full quest ownership, not request-by-request collaboration
+
+## 4. Autonomous Mode
+
+### 4.1 What it is
+
+Autonomous mode is the standard DeepScientist path.
+
+It is meant for quests where the system should keep making ordinary route choices on its own and continue until the next real checkpoint is reached.
+
+This is the right mode for:
+
+- baseline establishment
+- long experiments
+- analysis campaigns
+- durable writing / finalize loops
+- connector-facing project progress over time
+
+### 4.2 What happens after creation
+
+After you create an Autonomous project:
+
+- the quest is created
+- the first turn is launched immediately
+- the system begins turning the startup contract into real work
+
+This may mean:
+
+- reading the baseline and references
+- checking environment and constraints
+- preparing scripts
+- selecting the next route
+- launching detached `bash_exec` work
+
+### 4.3 Continuation behavior
+
+Autonomous mode has two practical continuation regimes.
+
+#### A. No real long-running external task yet
+
+If no real long-running external task exists yet, DeepScientist should not park.
+
+It should keep using the next turns to:
+
+- prepare the real task
+- launch the real task
+- or make a durable route decision about what the real next task must be
+
+This is the “active preparation / launch” phase.
+
+#### B. A real long-running external task is already active
+
+Once a real detached task is already running, continuation changes shape.
+
+At that point, DeepScientist should not busy-loop through rapid model turns just to imitate continuous execution.
+
+Instead:
+
+- the real work should remain alive in detached `bash_exec` sessions or the runtime process they launched
+- agent turns become lower-frequency monitoring / synthesis passes
+- the current default monitoring cadence is roughly every `240` seconds
+
+This is the “background progress monitoring” phase.
+
+### 4.4 Good fit
+
+Choose `Autonomous` when:
+
+- the quest should keep moving on its own
+- you expect real long-running experiment or analysis work
+- you want DeepScientist to keep routing after milestones
+- you want the standard research-operating-system behavior
+
+### 4.5 Bad fit
+
+Avoid `Autonomous` when:
+
+- you only want a quiet project shell first
+- you want to inspect the repo manually before any real movement
+- you want every next unit to be explicitly user-directed
+
+## 5. The Most Important Practical Difference
+
+The easiest way to remember the split is:
+
+- `Copilot` asks: “What single useful unit should I help with right now?”
+- `Autonomous` asks: “What is the next real quest step, and how do I keep the quest moving?”
+
+That difference matters more than the labels themselves.
+
+## 6. How Resume Works
+
+Both modes preserve context durably, but they resume differently.
+
+On later turns, DeepScientist now carries a compact resume spine that can include:
+
+- the latest durable user message
+- the latest assistant checkpoint
+- the latest run summary
+- recent memory cues
+- current `bash_exec` state
+
+But the continuation policy still differs:
+
+- `Copilot`: resume only when you speak again or explicitly resume
+- `Autonomous`: keep going unless a real blocker or explicit waiting state applies
+
+## 7. Choosing Quickly
+
+Use this fast rule:
+
+1. If you want the project to wait quietly until you tell it what to do, choose `Copilot`.
+2. If you want DeepScientist to begin turning the quest contract into real work immediately, choose `Autonomous`.
+3. If you are unsure, start with `Copilot`; you can still move into longer-running work once the route is clearer.
+
+## 8. Common Misunderstandings
+
+### “Autonomous means it should always spin rapidly”
+
+No.
+
+Autonomous means the quest should keep moving.
+
+When there is no real long-running task yet, that can mean rapid preparation / launch turns.
+When a real long-running task is already active, it should usually mean lower-frequency monitoring, not constant model churn.
+
+### “Copilot means it cannot run experiments”
+
+Also no.
+
+Copilot can still help launch, inspect, analyze, and write.
+The difference is that it should not assume long autonomous continuation unless you ask for it.
+
+### “The two modes only change wording”
+
+No.
+
+They affect:
+
+- initial launch behavior
+- continuation policy
+- when the quest parks
+- how much autonomous routing is appropriate
+
+## 9. Related Docs
+
+- [00 Quick Start](./00_QUICK_START.md)
+- [02 Start Research Guide](./02_START_RESEARCH_GUIDE.md)
+- [12 Guided Workflow Tour](./12_GUIDED_WORKFLOW_TOUR.md)
+- [14 Prompt, Skills, and MCP Guide](./14_PROMPT_SKILLS_AND_MCP_GUIDE.md)