@konbakuyomu/smart-search-linux-x64 0.0.0 → 0.1.26

package/runtime/smart-search/_internal/smart_search/assets/skills/smart-search-cli/references/provider-routing.md

@@ -0,0 +1,178 @@
+# Provider Routing
+
+## Intent Routing Diagnostics
+
+`search` and `research` are the two live search modes. Both share the configured channel catalog, execution, deadlines, and evidence records.
+
+- `search`: one routing decision, using Jev or local rules, selects one suitable channel, then returns results and sources. Necessary dependent requests (such as Context7 library resolution followed by docs) and same-capability failure fallback remain; model answers and web search are mutual fallbacks in ordinary search. It never runs evidence assessment, follow-up search, filtering, or synthesis. `--extra-sources N` limits results from that channel (up to 20), not the number of channels. `evidence_assessment.status=not_evaluated`; success is retrieval success, not factual verification. `--validation strict` requires source evidence and can return partial content with an evidence error.
+- `research`: the shared loop selects complementary actions in parallel, reads discovered pages, assesses evidence gaps, and continues while useful and within budget. Jev or local rules choose from program-built original/subquestion/gap query candidates; Jev does not generate arbitrary queries or tool calls. Only fetched/read content enters `citations`. Optional filtering and `SMART_SEARCH_JEV_SYNTHESIZE=false|auto|true` apply only here.
+- `deep` remains an offline planner. Its `steps[].command` entries are suggestions for manual/AI orchestration; `research` records its actual actions separately in `routing_decision.rounds` and `stage_results`.
+
+The default is `SMART_SEARCH_INTENT_ROUTER=jev`. In Jev mode, ordinary search makes at most one logical Jev judgment; without a TypeSafe key or a valid judgment, `--fallback auto` uses local capability rules and reports degradation, while `--fallback off` fails clearly. Explicit `local` needs no TypeSafe key and never calls Jev. Legacy `hybrid` maps to `jev`; `rules` and `off` map to `local`. Old embedding/classifier keys remain stored but unused. `route-calibrate` is a retired, non-network compatibility command.
+
+`route` previews local capabilities without calling a model or retrieval provider. Add `--remote` only for an explicit, potentially billable Jev diagnostic. Direct provider commands keep their explicit behavior.
+
+Only configured, enabled, allowed channels outside cooldown can run. `--providers`, provider enable switches, `SMART_SEARCH_RESEARCH_DISABLED_PROVIDERS`, and explicit-only boundaries are enforced before selection and dispatch. A failed ordinary channel can try another provider of the same capability, with model answers and web search treated as mutual fallbacks; it cannot turn into a research loop. User preferences do not bypass these boundaries.
+
+The shared total timeout defaults to 300 seconds (`SMART_SEARCH_TIMEOUT_SECONDS`, overridden by `search --timeout`). Research quick/standard/deep ceilings are 2/3/10 rounds, 1/2/10 channels per round, 3/5/20 results per channel, and 45/120/configured seconds, further limited by the configured Jev budgets. Sufficient evidence, exhaustion, no progress, or deadline stops the loop. Completed results survive optional judgment/filter/synthesis failure.
+
+For research, `SMART_SEARCH_JEV_SYNTHESIZE` accepts `true`, `false`, and `auto`. Final synthesis uses only the dedicated `SMART_SEARCH_JEV_SYNTHESIS_API_URL`, `_API_KEY`, and `_MODEL` configuration; `_API_MODE` selects `chat-completions` (default) or `responses`. It never uses the main-search model or `--model`. Auto asks Jev after optional filtering whether the retained evidence benefits from synthesis. No allowed dedicated model or a failed judgment returns evidence directly. `synthesis` reports the mode, decision probability, execution status and skip reason. With an explicit `--providers` filter, include `jev-synthesis` to permit the synthesis call.
+
+## Provider Boundaries
+
+- `deep` uses offline rules/local signals only and must not call remote embeddings or classifier components.
+- Official xAI uses `/responses` through `XAI_*`. OpenAI-compatible relays default to `/chat/completions`; `OPENAI_COMPATIBLE_API_MODE=responses` explicitly selects `/responses` for a relay that supports the documented subset.
+- `search --stream` sets `stream=true` for xAI Responses and OpenAI-compatible search, including Grok search selected by Jev. `--no-stream` disables streaming for that request. `OPENAI_COMPATIBLE_STREAM=true` remains the default switch for OpenAI-compatible search and provider-side fetch.
+- Legacy `SMART_SEARCH_API_URL`, `SMART_SEARCH_API_KEY`, `SMART_SEARCH_API_MODE`, `SMART_SEARCH_MODEL`, and `SMART_SEARCH_XAI_TOOLS` are unsupported config keys.
+- xAI Responses mode may use only `XAI_TOOLS=web_search,x_search` and a subset of those tools.
+- `XAI_REASONING_EFFORT` accepts `low`, `medium`, or `high` (case-insensitive, stored lowercase). Optional; when set, sends reasoning.effort in xAI Responses requests; omitted by default. This also applies to Grok search selected by Jev. Unsupported reasoning values reported by the upstream API remain errors; the field is not silently removed and retried.
+- Neither OpenAI-compatible API mode sends xAI `web_search` / `x_search` tools or legacy `search_parameters`; xAI Chat Completions Live Search is deprecated.
+- AnySearch is reported only as optional experimental `vertical_search`; it is not part of the `web_search` fallback and is not required by the `standard` minimum profile.
+- Sciverse is reported only as optional experimental explicit-only `vertical_search`; it is not `docs_search`, not part of default `search` / `research` fallback, and not required by the `standard` minimum profile.
+- Jina Reader is `web_fetch` only, not a general search provider. `JINA_API_KEY` is required before Jina satisfies the standard minimum profile; anonymous `r.jina.ai` is explicit/experimental fetch behavior.
+- Same-capability fallback is allowed; model answers and web search are mutual fallbacks in ordinary search. Context7 is not used for unrelated broad web queries, and page extraction providers are not used as docs search providers.
+- `TAVILY_ENABLED=false` removes Tavily from registered `web_search` and `web_fetch` routes even when its key is present. Direct Tavily search/extract calls, `map`, `doctor`, and smoke must make no Tavily network request; `map` reports a local configuration error. Firecrawl remains independently configured and all fallback stays within its capability.
+- In Jev mode, Jev chooses among allowed capabilities and providers for the question; there is no mandatory main-model call or fixed global provider order. Local routing prioritizes suitable capabilities and configured preferences.
+- `main_search` contains xAI Responses and OpenAI-compatible model channels; `web_search` contains Zhipu REST/MCP, Tavily, Firecrawl and TinyFish; `docs_search` contains Context7 and Exa. Failure fallback stays within the selected capability family; model answers and web search are mutual fallbacks in ordinary search.
+- Ordinary Context7 uses the first library in Context7's ranked candidates, without a second Jev call. Research may use Jev to select the matching library, using the first candidate if Jev fails or no score reaches its threshold. Empty results may fall through to same-capability Exa; explicit Context7 commands remain unchanged.
+- `--extra-sources N` controls the selected channel's result count. It does not add channels, split a budget between providers, or make Tavily/Firecrawl a required extra stage. A known-URL search follows the configured Jev/local routing; local routing prefers page reading.
+- `fetch` tries Tavily first, then Jina with `JINA_API_KEY`, then Zhipu Coding Plan MCP Reader, then Firecrawl, then TinyFish.
+
+TinyFish:
+
+- `TINYFISH_API_KEY` is optional and registers TinyFish for both `web_search` and `web_fetch`. It never satisfies `main_search` or `docs_search`. Jev may select it directly; the direct `fetch` fallback order remains as listed above.
+- `TINYFISH_SEARCH_API_URL` defaults to `https://api.search.tinyfish.ai` and is called as `GET ?query=...`; `TINYFISH_FETCH_API_URL` defaults to `https://api.fetch.tinyfish.ai` and is called as `POST` with `{"urls": [url], "format": "markdown"}`.
+- Both endpoints send `X-API-Key`, not `Authorization: Bearer`. Never log or echo the key.
+- Search results normalize `snippet` to `description`, `site_name` to `source`, and `date` to `published_date`. An empty result set is a successful empty response, not a failure.
+- Fetch reads `results[0].text`; `errors[]` entries are surfaced as `provider_error` with the upstream message. A payload with neither results nor errors is `provider_error`, and a non-object payload is `parse_error`.
+- Challenge pages such as `Checking if the site connection is secure` are reported as `quality_error` so same-capability fallback can continue.
+- `401`/403 map to `auth_error`, `408` to `timeout`, `429` to `rate_limited`, `5xx` to `network_error`, and `400`/422 to `parameter_error`.
+- `map` currently uses Tavily only.
+- `exa-search` and `exa-similar` use Exa only.
+- `context7-library` and `context7-docs` use Context7 only.
+- `anysearch-domains`, `anysearch-search`, `anysearch-extract`, and `anysearch-batch` use AnySearch only. Treat results as acceptance evidence until the target vertical domain is reviewed.
+- `sciverse-catalog`, `sciverse-search`, `sciverse-semantic`, `sciverse-read`, and `sciverse-relations` use Sciverse only. They are explicit academic commands and must not be inserted into default provider fallback.
+- `zhipu-search` uses Zhipu only.
+- `zhipu-mcp-search`, `zhipu-mcp-reader`, and `zhipu-mcp-*` zread commands use Zhipu Coding Plan Remote MCP only.
+- Runtime config priority is environment variables first, then local config file, then defaults.
+- `setup` and `config` read/write the local Smart Search config file and do not call providers.
+- `model current` reports explicit provider model settings. `model set` is retained only as a parameter-error migration guard; use `config set XAI_MODEL ...` or `config set OPENAI_COMPATIBLE_MODEL ...` to change models.
+
+Provider attempt errors:
+
+- Provider exceptions must be visible as `provider_attempts[].status="error"`, never silently converted to empty output. Use the stable taxonomy: HTTP `400`/`422` is `parameter_error`, `401`/`403` is `auth_error`, timeout is `timeout`, `429` is `rate_limited`, `5xx` or request failure is `network_error`, invalid response decoding is `parse_error`, and explicit upstream tool failures are `provider_error`.
+- A successfully decoded response with no normalized candidates or content is `status="empty"`, so same-capability fallback can continue without misreporting a provider failure.
+
+Zhipu Web Search API:
+
+- `zhipu-search` corresponds to the official Zhipu Web Search API route, using `ZHIPU_API_URL` plus `ZHIPU_SEARCH_ENGINE`; it is not Zhipu Chat Completions `tools=[web_search]`, not Search Agent, and not the MCP Server.
+- `ZHIPU_SEARCH_ENGINE` defaults to `search_std`. Official Web Search API service values include `search_std`, `search_pro`, `search_pro_sogou`, and `search_pro_quark`; keep custom values possible because official services may change.
+- `TAVILY_API_URL` only affects Tavily REST calls and does not proxy Zhipu.
+
+Zhipu Coding Plan Remote MCP:
+
+- `ZHIPU_MCP_API_KEY` configures the Coding Plan MCP auth token and must be sent as `Authorization: Bearer ...`; it must never be logged unmasked.
+- `ZHIPU_MCP_SEARCH_API_URL` defaults to `https://open.bigmodel.cn/api/mcp/web_search_prime/mcp` and calls `web_search_prime` for `web_search`.
+- `ZHIPU_MCP_READER_API_URL` defaults to `https://open.bigmodel.cn/api/mcp/web_reader/mcp` and calls `webReader` for `web_fetch`.
+- `ZHIPU_MCP_ZREAD_API_URL` defaults to `https://open.bigmodel.cn/api/mcp/zread/mcp` and calls `search_doc`, `get_repo_structure`, and `read_file` through explicit repo/docs commands.
+- Zhipu Coding Plan MCP must be implemented as a separate Remote MCP-over-HTTP provider layer. Do not route it through the existing `/paas/v4/web_search` Zhipu REST provider.
+- A normal Zhipu Web Search API key is not sufficient evidence of Coding Plan entitlement. If `ZHIPU_MCP_API_KEY` is missing or returns auth/provider errors, MCP providers are skipped or fall through within the same capability; zread remains explicit and does not affect the standard minimum profile.
+- Provider failures must appear in `provider_attempts` and fallback must remain same-capability, with model answers and web search treated as mutual fallbacks in ordinary search.
+- `doctor` should report configured/not-configured, auth, rate-limit, provider, timeout, and network status without exposing the MCP token.
+
+Jina Reader:
+
+- `JINA_READER_API_URL` defaults to `https://r.jina.ai`.
+- `JINA_API_KEY` is required before Jina satisfies `SMART_SEARCH_MINIMUM_PROFILE=standard`.
+- Anonymous Jina Reader calls may be used only as explicit/experimental degraded fetch behavior; they must not make standard setup pass.
+- `JINA_RESPOND_WITH=readerlm-v2` requires `JINA_API_KEY` and should report a configuration error without a network request when the key is missing.
+- Jina Reader is `web_fetch` only, not `web_search`.
+- Jina 401/403, 422, 429, timeout, network errors, and low-quality challenge pages such as `Title: Just a moment...` must be reported as failed provider attempts and allow same-capability fallback.
+
+AnySearch:
+
+- AnySearch uses JSON-RPC 2.0 `tools/call` at `ANYSEARCH_API_URL`, default `https://api.anysearch.com/mcp`.
+- `ANYSEARCH_API_KEY` is optional. If configured, requests include `Authorization: Bearer ...`; if missing, anonymous requests are allowed.
+- `ANYSEARCH_TIMEOUT_SECONDS` defaults to `30`.
+- Live MCP tools are `search`, `batch_search`, `extract`, and `get_sub_domains`. `anysearch-domains` maps to `get_sub_domains` (not the removed `list_domains` tool).
+- `anysearch-domains DOMAIN` calls the live `get_sub_domains` tool; without `DOMAIN`, it reads the `tools/list` schema for the available domains.
+- HTTP 200 responses with `result.isError=true` must return `ok=false`, `error_type=provider_error`, and no successful source results.
+- Markdown URL/title/snippet candidates should be parsed into `results`, while raw text remains in `content` and `raw_content`.
+- Structured results without URLs must be preserved as raw/structured evidence, not dropped.
+- Dotted vertical domain shorthand such as `code.doc` is allowed for simple subdomains and must be normalized to `domain=code` plus `sub_domain=doc`; parameterized subdomains parse `--sub-domain-params` JSON first, then repeatable `--param KEY=VALUE` entries override matching keys. Invalid JSON, non-object JSON, a missing `=`, or an empty key fails before the network request.
+- `anysearch-extract --max-length` sends only `url` to the live tool. A positive value truncates successful top-level and result text fields locally; zero or negative values preserve the normalized payload.
+- `anysearch-batch` accepts at most 5 CLI query strings and returns `error_type=parameter_error` without sending a request when the limit is exceeded.
+
+Sciverse:
+
+- `SCIVERSE_API_URL` defaults to `https://api.sciverse.space`; `SCIVERSE_TIMEOUT_SECONDS` defaults to `30`.
+- `SCIVERSE_API_TOKEN` is required. Missing token returns `error_type=config_error` without a network request; configured requests send `Authorization: Bearer ...` and must never expose the token.
+- Commands map directly to Sciverse OpenAPI: catalog -> `GET /meta-catalog`, search -> `POST /meta-search`, semantic -> `POST /agentic-search`, read -> `GET /content`, relations -> `POST /meta-paper-relations`.
+- Current `meta-catalog` and `meta-search` schemas have no `collection` selector. Legacy `--collection papers` is accepted without being sent upstream; `authors` and `sources` return `parameter_error` before a request. `meta-search` sends only `query`, `filters`, `sort`, `freshness_boost`, `page`, and `page_size`. `--title-contains` / `--abstract-contains` fold into `query`; author, journal, subject, and year conveniences become current filter items.
+- Advanced filters and sorts are structured local contracts: filters require `field` and `value` with a current `operator` such as `FILTER_OP_GTE`; legacy `op` is accepted only when it maps unambiguously. Sort items require `field` and normalize `order` to `SORT_ORDER_ASC` or `SORT_ORDER_DESC`. Full-text `query` and any `sort` conflict, so `--sort-by-year` defaults to `none` and sorting is filter-only.
+- `sciverse-semantic --retrieval hybrid|milvus|es` is the current semantic contract. Deprecated `--mode fast|balanced|quality` maps to `retrieval=hybrid` with a warning; it conflicts with explicit `milvus` or `es`. Semantic `--source-types` accepts only `web,pdf`.
+- `sciverse-read` uses `doc_id`; `sciverse-relations` uses `unique_id`.
+- Relation direction must stay explicit: `CITATIONS` means papers citing the target paper, `REFERENCES` means papers cited by the target paper, and `RELATED_WORKS` means related work suggestions.
+- Sciverse's documented HTTP `504` deadline response is `timeout`; other `5xx` responses remain `network_error`.
+- Local limits reject before network: search `page_size <= 200` and `page * page_size <= 10000`, semantic `top_k <= 100`, read `limit <= 16384`, relations `page_size <= 200`.
+- `--filters-advanced` and `--sort-advanced` must be JSON arrays of valid objects and fail with `parameter_error` before service/provider calls when malformed.
+
+OpenAI-compatible streaming:
+
+- `OPENAI_COMPATIBLE_API_MODE` defaults to `chat-completions` and accepts only `chat-completions` or `responses`. The selected protocol applies consistently to `search()`, provider-side `fetch()`, `describe_url()`, `rank_sources()`, `doctor`, and `diagnose openai-compatible`.
+- Responses mode handles the official `model` + `instructions`/`input` subset, heterogeneous `output` text parts, URL citations, typed stream deltas, and terminal failures. It does not promise `/responses` support from every relay marketed as OpenAI-compatible; accept each named relay with both diagnose stream probes.
+- `OPENAI_COMPATIBLE_STREAM` defaults to `false` and accepts `true`, `1`, or `yes` as true.
+- For OpenAI-compatible search, `search --stream` means "prefer stream first"; stream empty/timeout/retryable protocol failures fall back to the same provider/model with `stream=false`.
+- `search --no-stream` forces `stream=false` for the current invocation.
+- `SMART_SEARCH_TIMEOUT_SECONDS` defines one 300-second default monotonic `search` budget. Jev selection, channel dependencies and failure fallback share its remaining time; no budget is reserved for a mandatory main model. `search --timeout` overrides the saved value for one invocation. Research uses the same deadline with its configured budget ceilings.
+- `OPENAI_COMPATIBLE_FALLBACK_MODELS` is an optional comma-separated ordered list. It is fail-over after a hard primary-model failure, not a time slice. The current candidate keeps the remaining shared main-search budget; extra fallback models must not shrink that budget. `--fallback off` or `--model MODEL` disables this model fallback for the invocation.
+- `timeout_phase`, `phase_attempts`, elapsed/remaining deadline values, and `partial_success` report execution progress. Research judgment/filter/synthesis failures preserve completed evidence; ordinary search has no extra/supplemental phase. Strict validation still reports an evidence error when its source requirements are not met.
+- OpenAI-compatible attempts may include `model`, `transport`, `fallback_from_transport`, `fallback_from_model`, and `breaker_state`. `transport_fallback_used` records stream-to-non-stream recovery separately from provider/model `fallback_used`.
+- Streaming applies to xAI Responses `search()` and OpenAI-compatible `search()` and provider-side `fetch()` calls. `describe_url()` and `rank_sources()` stay non-streaming. xAI Responses uses the shared Responses SSE parser and does not retry with `stream=false`.
+
+Exa domain filters:
+
+- `--include-domains` and `--exclude-domains` accept comma-separated or whitespace-separated domains.
+- Both `--include-domains docs.python.org,developer.mozilla.org` and `--include-domains docs.python.org developer.mozilla.org` normalize to the same Exa domain list.
+- This normalization is intentional for Windows PowerShell, where an unquoted comma expression can be forwarded through `.ps1` wrappers as a space-separated value.
+
+## Provider Failure Cooldown
+
+- Same-capability fallback assumes a failing provider is worth retrying. A provider whose key is revoked, quota is exhausted, or endpoint is down is not, so its failures are persisted in `provider_health.json` and the provider is skipped for `SMART_SEARCH_PROVIDER_COOLDOWN_SECONDS` instead of being called again on every invocation.
+- Hard failures (`auth_error`, `config_error`) open the cooldown immediately for four cooldown windows and are not probed. Soft failures need `SMART_SEARCH_PROVIDER_FAILURE_THRESHOLD` consecutive failures within one window, and automatic search respects cooldown until expiry or an explicit reset. Direct fetch retains its bounded recovery probe.
+- A skipped provider appears in `provider_attempts` with `status=skipped`, the remembered `error_type`, and a `provider_health` block. It is not a silent drop, and it does not change same-capability fallback order.
+- Zhipu reports several failures inside an HTTP `200` body. Those payloads are classified (`auth_error`, `rate_limited`, `provider_error`) instead of being reported as an empty result set, so a dead Zhipu channel is diagnosable and coolable.
+- Recovery paths: `smart-search providers reset PROVIDER`, a credential change through `smart-search config set`, a successful `smart-search doctor` probe, or waiting out the cooldown.
+
+## Provider Output Details
+
+- Exa HTTP `400` or `422` failures are returned as `ok=false` with `error_type=parameter_error`; use this to distinguish bad CLI/domain/date/category arguments from upstream network failures.
+- AnySearch experimental output should preserve structured results without URLs as raw/structured evidence.
+- Sciverse experimental output should preserve raw response data under `raw` while exposing normalized `fields`, `results`, `hits`, `text`, or `items` depending on the command.
+- Diagnostic output should report Firecrawl status as whether `FIRECRAWL_API_KEY` is configured; it is not currently a live Firecrawl request.
+
+## Routing Heuristics
+
+- Use `smart-search route "query" --format markdown` when you need to explain why a query maps to `docs_search`, `web_search`, `web_fetch`, or `vertical_search` without executing providers.
+- Use `exa-search --include-domains` when official documentation domains are known.
+- Use `context7-library` / `context7-docs` for docs/API/SDK/library/framework intent when Context7 is configured.
+- Use `zhipu-search` for Chinese, domestic, current, or domain-filtered source discovery when Zhipu is configured.
+- Use `exa-search --start-published-date` for recency-constrained source discovery.
+- Use `exa-similar` when a known good page is available and adjacent sources are needed.
+- Use `search --format content` when a human wants the selected channel output.
+- Use `fetch --format markdown` or `fetch --format content` for user-supplied URLs or when exact page text matters.
+- Use `map` before fetching many pages from a documentation site.
+- Keep `search --extra-sources` small (`1` to `3`) unless broad coverage is requested.
+- For current news or high-risk claims, prefer source discovery plus `fetch`; do not treat broad `search.content` plus source metadata as claim-level verification.
+
+## Maintenance Guardrails
+
+- Provider architecture changes must be verified as distributable CLI behavior, not as behavior that only works because one developer machine has a specific wrapper, shell profile, or local config file.
+- Register providers by capability first, then route by intent. Fallback is allowed only within the same capability family; model answers and web search are mutual fallbacks in ordinary search.
+- Keep xAI Responses and OpenAI-compatible as peer `main_search` providers. A failed xAI Responses request may fall back to OpenAI-compatible only when `OPENAI_COMPATIBLE_API_URL` and `OPENAI_COMPATIBLE_API_KEY` are separately configured.
+- Do not use Context7 for broad news or generic web facts; do not use Tavily or Firecrawl as documentation semantic-search replacements.
+- After provider-routing changes, run source-checkout regression plus `smart-search smoke --mock --format json`. If live keys were used, run a targeted secret scan for exact key substrings before committing.
+
+`primary_sources` preserves citations from a selected model channel; these are discovery candidates until read. `sources` contains returned source metadata; `extra_sources` is retained as an empty compatibility list.
+
+Explicit cancellation returns `error_type=cancelled` with completed evidence and `partial_success`; do not treat the interrupted request as complete. App cancellation retains the same result and research artifacts.

package/runtime/smart-search/_internal/smart_search/assets/skills/smart-search-cli/references/regression-release.md

@@ -0,0 +1,46 @@
+# Regression And Release
+
+## Table of Contents
+
+- Regression
+- Smoke matrix
+- Release lanes
+- Release closeout lessons
+
+## Regression
+
+Run `smart-search regression` before considering CLI or skill changes complete.
+
+- In a source checkout, it runs offline pytest coverage for CLI, service, smoke, provider, and skill contract behavior.
+- In npm / mise packaged installs, repository test files are not bundled; since v0.1.8 it falls back to built-in mock smoke regression so users can still verify installed CLI health.
+- For release validation, use a source checkout for full pytest-backed regression and use packaged-install regression only as an install-health check.
+- Provider architecture changes must be verified as distributable CLI behavior, not as behavior that only works because one developer machine has a specific wrapper, shell profile, or local config file.
+- After provider-routing changes, run source-checkout regression plus `smart-search smoke --mock --format json`. If live keys were used, run a targeted secret scan for exact key substrings before committing.
+- Read-only CI must cover Ubuntu Node 18/Python 3.10, Ubuntu Node 24/Python 3.12, and Windows Node 22/Python 3.12. It must never publish; it checks source regression, mock smoke, skill parity, pack dry-run, a fresh temporary-prefix tarball install, and committed diff whitespace with `git diff --check HEAD^1 HEAD` (or `git diff-tree --check --root -r --no-commit-id HEAD` for a root commit). A bare `git diff --check` after checkout only sees an empty worktree.
+
+## Smoke Matrix
+
+- Deep Research smoke coverage is mock-full plus live-limited.
+- Mock-full coverage should cover trigger phrases, normal search requests that should not trigger Deep Research, required `research_plan` fields, allowed tool whitelist, `fetch_before_claim`, evidence paths, capability boundaries, `intent_signals`, `capability_plan`, `gap_check`, simple current prompts such as `深度搜索一下最近的比特币行情`, docs/API prompts, claim-verification prompts, user-provided URL fetch-first flows, missing-provider failure guidance, research provider advantage routing, same-capability research fallback, and the rule that fixed topic recipe ids are not required schema.
+- Live-limited coverage should run `doctor`, one broad `search`, one `exa-search`, and one `fetch` when real keys are available and live checks are expected; add one small `research` smoke when configured keys make it stable.
+- If a smoke issue is found, fix the affected docs/code/tests and rerun the affected smoke until it passes or is proven to be an external provider blocker.
+
+## Release Lanes
+
+- Stable releases are pushed as `vX.Y.Z` Git tags and publish npm `X.Y.Z` with dist-tag `latest`.
+- Test releases are pushed from `main` and publish `<package.json version>-beta.N` with dist-tag `next`. The beta counter resets per base version, so `0.1.9-beta.1` and `0.1.10-beta.1` are separate sequences.
+- Stable bump commits should use `chore(release): bump version to X.Y.Z`, but the workflow detects a stable bump by comparing the current stable package version with the first parent. This also skips beta publication for merge and squash release commits; the subject check remains a compatibility fallback.
+- Stable GitHub release notes should be stored as `.github/releases/vX.Y.Z.md` before tagging. The publish workflow appends npm package, dist-tag, and workflow-run metadata to that body automatically.
+- Historical test builds can be backfilled through GitHub Actions `workflow_dispatch` by supplying an explicit `target_ref`, exact `version`, and a non-`latest` npm tag such as `backfill`.
+- npm versions are immutable. Old `*-dev.*` packages cannot be renamed in place; publish replacement `*-beta.N` packages and optionally deprecate the old names when npm owner credentials are available.
+- npm publish runs use a repository-scoped concurrency group and must serialize rather than cancel an in-progress release.
+
+## Release Closeout Lessons
+
+- Always read back npm before and after publishing with `npm view @konbakuyomu/smart-search versions --json` and `npm view @konbakuyomu/smart-search dist-tags --json`. A test release must leave `latest` on the stable version and move only `next` or the explicitly supplied non-`latest` tag.
+- Backfill jobs can publish npm successfully even if GitHub release creation fails because the workflow token cannot access the release API. In that case, leave npm intact and create the missing GitHub prerelease with authenticated local `gh release create ... --prerelease --latest=false`.
+- If concurrent backfill jobs hit npm `E409`, re-dispatch only the affected versions serially after checking whether the version already appeared in the registry.
+- Finish with a diff-style gap check: expected beta version list minus npm versions equals empty, and expected `vX.Y.Z-beta.N` list minus GitHub prereleases equals empty.
+- Local verification after a test release must use an exact install target, such as `mise use -g "npm:@konbakuyomu/smart-search@0.1.10-beta.3" -y --pin`, followed by `mise reshim`, `where.exe smart-search`, `smart-search --version`, packaged `smart-search regression`, and `smart-search smoke --mock --format json`.
+- Also pipe a non-ASCII JSON command such as `smart-search deep "深度搜索一下最近的比特币行情" --format json | ConvertFrom-Json` to verify the Windows npm/mise wrapper is emitting UTF-8 JSON, not locale-encoded bytes.
+- Before any remote release action, run the local temporary-prefix tarball smoke so a package from `npm pack` proves its wrapper, packaged regression fallback, and mock smoke without touching a global install.

package/runtime/smart-search/_internal/smart_search/assets/skills/smart-search-cli/references/setup-config.md

@@ -0,0 +1,133 @@
+# Setup And Config
+
+## Table of Contents
+
+- Config storage
+- Doctor and diagnostics
+- Setup workflow
+- Browser config UI
+- Skill installation sync
+- Provider endpoint setup
+- Intent router setup
+
+## Config Storage
+
+- Prefer the CLI's local config file managed by `smart-search setup` / `smart-search config`.
+- Environment variables remain supported for CI and advanced users, and override the local config file.
+- `SMART_SEARCH_TIMEOUT_SECONDS` persists the total monotonic `search` budget and defaults to `300`; explicit `search --timeout SECONDS` overrides environment, config, and default for one invocation. The budget is also the main-search read ceiling, so slow reasoning models are no longer cut off by a separate fixed provider read timeout.
+- `SMART_SEARCH_PROVIDER_COOLDOWN_SECONDS` defaults to `900` and `SMART_SEARCH_PROVIDER_FAILURE_THRESHOLD` defaults to `2`. They control how long a repeatedly failing optional provider is skipped; `SMART_SEARCH_PROVIDER_COOLDOWN_SECONDS=0` disables the cooldown entirely.
+- Do not ask users to set Windows global API-key environment variables by default.
+- If keys are changed with `smart-search config set`, rerun the CLI; no Codex restart is needed.
+- If PATH is changed, a new terminal or Codex restart may be needed.
+- On Windows, the default local config file is `%LOCALAPPDATA%\smart-search\config.json`. Linux/macOS default to `~/.config/smart-search/config.json`.
+- In sandboxed runtimes where the default config directory is not writable or must be pinned, set `SMART_SEARCH_CONFIG_DIR` to an absolute writable path. The CLI uses it for both config and relative logs and skips default-directory selection.
+- Earlier Windows source defaults used `~\.config\smart-search\config.json`, while some installs were already pinned to `%LOCALAPPDATA%\smart-search` through `SMART_SEARCH_CONFIG_DIR`. If the new default file is missing but the old file exists, `doctor` reports `legacy_windows_home` as the active source so upgrades do not silently lose configuration.
+- When a Windows user reports different config paths, diagnose in this order: `config_dir_source`, `config_dir_override_value`, `config_dir_override_matches_default`, then `legacy_windows_config_exists`. Do not delete either config file or the user-level override until the upgraded CLI has been verified with `config path`, `doctor`, and smoke/regression checks.
+
+## Doctor And Diagnostics
+
+- Use `smart-search doctor --format json` for agent/script parsing and `smart-search doctor --format markdown` when a human wants a detailed diagnostic report.
+- If `smart-search doctor --format json` returns `ok: false`, follow the `error` field's guidance (`smart-search setup` or `smart-search config set KEY VALUE`); do not silently fall back to native web search.
+- `doctor --format markdown` must render a detailed diagnostic report with overall status, active/default/legacy config paths, log path resolution, file-logging status, masked config values with sources, minimum profile, capability status, providers currently on failure cooldown, main-search provider checks, provider connectivity checks, Jev/local router status, model metadata, and full long error/message detail.
+- Use `smart-search diagnose openai-compatible --format markdown` when `doctor` succeeds but OpenAI-compatible `search` appears to hang, returns a timeout, or differs between `--stream` and `--no-stream`. It is the beginner-facing one-command report for upstream/relay compatibility.
+- `diagnose openai-compatible --format markdown` must render a short copy-pasteable troubleshooting report with masked config, the selected API mode and endpoint, a quick selected-endpoint check, real search-shape `stream=false` and `stream=true` checks, fallback-model inventory against `/models`, the remaining-budget timeout policy, a plain-language summary, and a next command.
+
+## Setup Workflow
+
+- Interactive `smart-search setup` is a language-selecting grouped wizard with arrow-key / Space / Enter provider selection. It groups optional providers by capability; configure one retrieval channel, then add Jev or page reading as needed.
+- Default `smart-search setup` shows a Smart Search ASCII banner, asks for `zh` or `en`, offers user-level `smart-search-cli` skill installation, then shows a grouped provider wizard.
+- The grouped wizard should use an arrow-key / Space / Enter selector when packaged TUI dependencies are available, with a text fallback for non-TTY and tests.
+- Use `smart-search setup --lang en` for an English wizard.
+- Use `smart-search setup --advanced` only when low-level config keys must be shown one by one; Jev/local routing is available in the default wizard. `--advanced` does not show the skill prompt unless `--install-skills` is explicit.
+- `--non-interactive` keeps script behavior and only saves values passed as flags.
+- Provider groups are optional choices: configure at least one usable retrieval channel, then add Jev and research page reading as needed.
+- Unchecking a configured provider must not delete existing config values; use `smart-search config unset KEY` for deletion.
+- Interactive output should summarize `minimum_profile_ok`, missing required capabilities, and next-step commands.
+- Beginner filling examples for official-service and relay/pooled-endpoint minimum profiles must appear in the grouped wizard on stderr, not stdout. They must cover `main_search`, `docs_search`, and `web_fetch`.
+
+## Browser Config UI
+
+- `smart-search ui` opens a temporary local config page for users who prefer forms to the terminal wizard. Its test page supports offline route preview and ordinary single-pass search; start research from the App or CLI.
+- It binds `127.0.0.1` on a random port, prints a URL carrying a one-time token, and exits once the browser tab stops sending heartbeats. It is not a daemon and needs no extra install: stdlib `http.server` plus one bundled HTML file.
+- Flags: `--no-browser` (print the URL only), `--port N`, `--idle-timeout SEC` (default `900`, `0` disables), `--lang zh|en`, `--check` (verify the bundled page resolves, then exit), plus the usual `--format`.
+- On a headless host, SSH session, or container the browser is deliberately not opened; the URL is printed and a `ssh -L` forwarding recipe is shown. Use `--port` to pin the port for forwarding.
+- The page never receives unmasked secrets. Keys supplied through environment variables render as locked, because environment variables override `config.json` on every read; the page names the `unset` to run.
+- Per-provider Test buttons call `smart-search providers test PROVIDER` equivalents. Each one is a real, possibly billable API request, so nothing is probed until the user clicks.
+- Do not recommend the UI for scripted or non-interactive work; `smart-search setup --non-interactive` and `smart-search config set` remain the automation path.
+
+## Skill Installation Sync
+
+- Skill installation installs the bundled `smart-search-cli` skill into selected AI-tool skill directories and must not run `trellis init`, create hooks, create agents, create commands, or modify other skills.
+- Targets are user-level/global directories under the current user's home directory, for example Codex `~/.agents/skills/`, Claude Code `~/.claude/skills/`, Cursor `~/.cursor/skills/`, OpenCode `~/.config/opencode/skills/`, GitHub Copilot `~/.copilot/skills/`, and Hermes Agent `~/.hermes/skills/`.
+- Skill targets are `codex`, `claude`, `cursor`, `opencode`, `copilot`, `gemini`, `kiro`, `qoder`, `codebuddy`, `droid`, `pi`, `kilo`, `antigravity`, `windsurf`, and `hermes`.
+- OpenCode's canonical global skill directory is `~/.config/opencode/skills/smart-search-cli`. `skills status` reports a discovered legacy `~/.opencode/skills/smart-search-cli` directory as read-only `legacy_locations` metadata; it never migrates or deletes that tree.
+- `--skip-skills` disables skill installation.
+- `--install-skills codex,claude,cursor,hermes` selects targets explicitly.
+- `--skills-root PATH` is an advanced synthetic home-root override used in portable installs or tests. For OpenCode, `--skills-root T` writes to `T/.config/opencode/skills/smart-search-cli`. Normal users should omit it.
+- `smart-search agent-guide [relative-path]` reads current instructions directly from the installed CLI. A Skill's local invocation prefix also applies to this command.
+- `smart-search skills remove --targets codex --format json` moves selected Skill files into a backup folder and disconnects automatic maintenance. It does not uninstall the CLI or remove configuration.
+- CLI-owned automatic maintenance affects only explicitly connected targets bound to this installation. It preserves personal edits, missing files, extra files and each target's configuration directory. Version/capability probes and `skills status` never perform maintenance.
+- `smart-search skills status --targets codex,claude,cursor,hermes --format json` compares bundled skill files with installed user-level skill directories. Status values are `missing`, `up_to_date`, `stale`, `extra_files`, and `error`. It reports target paths, bundled file count, installed file count, hashes, hash match flags, missing files, stale files, and extra files. OpenCode also reports discovered legacy locations. Status must not write or delete files.
+- `smart-search skills update --targets codex,claude,cursor,hermes --format json` overwrites the managed bundled `smart-search-cli` files for selected targets. For OpenCode, setup and update write managed files only to the canonical config path; legacy trees and user extra files remain untouched. `smart-search skills update --all --format json` selects every target id.
+- This daily sync path must not change provider keys, run setup prompts, create Trellis files, create hooks, create agents, create commands, or delete leftover files. Extra installed files are only reported by `skills status`.
+- `smart-search setup --non-interactive --install-skills codex` remains the first-time setup compatibility path. Prefer `skills status` and `skills update` for routine global skill synchronization after CLI upgrades.
+
+## Provider Endpoint Setup
+
+- Setup and config output should include `ok` and `config_file`. Saved API keys must be masked in command output.
+- Use `smart-search setup --non-interactive --zhipu-api-url "https://open.bigmodel.cn/api" --zhipu-search-engine "search_std"` to save Zhipu Web Search API endpoint and search service without prompts.
+- Interactive setup asks for Zhipu API key, API URL, and search service when the user selects Zhipu in the provider wizard.
+- `config set ZHIPU_SEARCH_ENGINE VALUE` must remain free-form so newly added official services do not require a CLI release.
+- `ZHIPU_API_URL` defaults to `https://open.bigmodel.cn/api`.
+- `ZHIPU_SEARCH_ENGINE` defaults to `search_std`.
+- Official Web Search API service values include `search_std`, `search_pro`, `search_pro_sogou`, and `search_pro_quark`.
+- Use `smart-search setup --non-interactive --jina-key "key"` to let Jina satisfy `web_fetch`; `JINA_RESPOND_WITH=readerlm-v2` also requires `JINA_API_KEY`.
+- Use `smart-search setup --non-interactive --zhipu-mcp-key "key"` only when the user explicitly wants Coding Plan Remote MCP quota.
+- Use `smart-search setup --non-interactive --openai-compatible-api-mode responses` only when a named relay requires `/responses`. The default is `chat-completions` and uses `/chat/completions`; invalid modes fail before setup saves any field.
+- Use `smart-search setup --non-interactive --openai-compatible-stream true` only when an OpenAI-compatible relay benefits from SSE streaming for long requests. Default remains false.
+- Use `smart-search setup --non-interactive --openai-compatible-fallback-models "model-a,model-b"` to save ordered OpenAI-compatible backup models for primary model hard failure. These models do not receive a reserved time slice; the primary model keeps the remaining shared main-search budget. `--fallback off` and `search --model MODEL` disable this model fallback for one invocation.
+- Use `smart-search setup --non-interactive --search-timeout 300` or `smart-search config set SMART_SEARCH_TIMEOUT_SECONDS 300` to persist the normal search budget. Invalid non-positive values fail before provider work.
+- Use `smart-search setup --non-interactive --anysearch-api-url "https://api.anysearch.com/mcp" --anysearch-key "key"` only for experimental AnySearch acceptance; do not add it to the normal minimum-profile setup.
+- Use `smart-search setup --non-interactive --sciverse-token "key" --sciverse-api-url "https://api.sciverse.space"` only for explicit experimental Sciverse academic commands; do not add it to the normal minimum-profile setup.
+- `TAVILY_API_URL` defaults to `https://api.tavily.com` and only affects Tavily REST calls. It does not proxy Zhipu.
+- Use `TAVILY_API_URL=https://<host>/api/tavily` for Tavily Hikari / pooled endpoints. Root host and `/mcp` inputs are normalized by setup; `/mcp` itself is not the REST base Smart Search should call.
+- `TAVILY_TIMEOUT_SECONDS` controls the Tavily `doctor` connectivity timeout and defaults to `30`. Raise it for slower pooled/community Tavily endpoints before judging the provider unhealthy.
+- `ANYSEARCH_API_URL` defaults to `https://api.anysearch.com/mcp`; `ANYSEARCH_TIMEOUT_SECONDS` defaults to `30`.
+- `SCIVERSE_API_URL` defaults to `https://api.sciverse.space`; `SCIVERSE_TIMEOUT_SECONDS` defaults to `30`.
+- `FIRECRAWL_API_URL` defaults to `https://api.firecrawl.dev/v2`. Use it only for a Firecrawl-compatible REST base.
+- Use `smart-search setup --non-interactive --tinyfish-key "key"` to add TinyFish to `web_search` and `web_fetch`. It never satisfies `main_search` or `docs_search`; Jev can select it among the allowed channels.
+- `TINYFISH_SEARCH_API_URL` defaults to `https://api.search.tinyfish.ai`, and `TINYFISH_FETCH_API_URL` defaults to `https://api.fetch.tinyfish.ai`. Both authenticate with the `X-API-Key` header, not a bearer token.
+- `TINYFISH_TIMEOUT_SECONDS` defaults to `60` and covers both TinyFish search and fetch reads.
+
+## Provider Failure Cooldown
+
+- Optional providers (`web_search`, `docs_search`, `web_fetch`, `vertical_search`) are additive. When one keeps failing, Smart Search records it in `provider_health.json` next to `config.json` and skips it instead of re-calling it on every invocation. Search-model channels also respect the shared health gate.
+- A hard failure (`auth_error`, `config_error`) opens the cooldown on the first occurrence and is never probed, because a bad key does not heal itself. Soft failures (timeout, `5xx`, rate limit) need `SMART_SEARCH_PROVIDER_FAILURE_THRESHOLD` consecutive failures and automatic search waits until cooldown expiry or an explicit reset.
+- An empty result set is not a failure and never opens a cooldown.
+- The record is keyed to a fingerprint of the provider's credentials, so re-keying a provider with `smart-search config set` clears its cooldown automatically. A successful `smart-search doctor` probe also clears it.
+- `smart-search providers test PROVIDER [PROVIDER...]` checks whether saved credentials still work for one provider at a time, instead of `doctor`'s all-at-once fan-out. Each run is a real API request. `--timeout` caps each probe. A passing test clears that provider's cooldown, the same way a `doctor` probe does. Firecrawl reports `probe: presence` because only key presence can be checked.
+- Use `smart-search providers status --format json` to see which providers are cooling and why, and `smart-search providers reset PROVIDER` to retry one immediately. `smart-search providers reset` with no argument clears every cooldown.
+- Use `smart-search setup --non-interactive --provider-cooldown 600 --provider-failure-threshold 3`, or `smart-search config set SMART_SEARCH_PROVIDER_COOLDOWN_SECONDS VALUE`, to persist the cooldown policy. A negative or non-numeric cooldown and a non-positive threshold fail before setup saves any field.
+- A skipped provider is still reported: the attempt has `status=skipped` with the remembered `error_type`, and `provider_notices` carries one deduplicated entry per degraded provider.
+
+## Intent Router Setup
+
+- `--search-timeout` (also `--search-timeout-seconds`) is the non-interactive setup flag for the total search budget; it does not change individual provider endpoint timeouts.
+
+## Language and complete handbook
+
+From v0.1.21, Smart Search supports `smart-search --lang en --help`, `smart-search search QUERY --lang zh`, and saved `SMART_SEARCH_LANGUAGE=auto|zh|en` through `config set`. Priority is per-call flag, dedicated environment variable, saved configuration, then system locale. Command/JSON identifiers and source content stay unchanged. App language is independent.
+
+The [bilingual handbook](https://github.com/konbakuyomu/smartsearch/tree/main/docs/guide) contains the [complete command reference](https://github.com/konbakuyomu/smartsearch/blob/main/docs/guide/en/cli-reference.md) and [all saved configuration keys](https://github.com/konbakuyomu/smartsearch/blob/main/docs/guide/en/configuration-reference.md).
+
+## Current routing configuration
+
+`SMART_SEARCH_INTENT_ROUTER=jev|local` defaults to `jev`. `hybrid` maps to `jev`; `rules`/`off` map to `local`. Retired `INTENT_EMBEDDING_*`, `INTENT_CLASSIFIER_*`, and `INTENT_ROUTER_TIMEOUT_SECONDS` keys stay stored, without remote calls or preset changes.
+
+```powershell
+smart-search config set TYPESAFE_API_KEY "your-key"
+smart-search config set SMART_SEARCH_INTENT_ROUTER jev
+smart-search route "React docs" --format json
+```
+
+`TYPESAFE_API_KEY` is optional with `--fallback auto`; local routing needs no router credentials. See `references/provider-routing.md` for the two-mode execution contract.