@fffangchenghao/dsh-web-search 0.11.0 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,90 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.13.0
4
+
5
+ ### Added
6
+ - **Optional JS rendering.** A page whose HTML is an empty shell can be rendered
7
+ by a real browser — but only if `playwright` or `playwright-core` is already
8
+ installed. Nothing is bundled, nothing is imported at load time, and a missing
9
+ module is a silent fallback, not an error. `renderJs` picks `auto` (render only
10
+ suspected shells), `always` or `off`; `renderTimeoutMs` bounds the attempt. A
11
+ render that fails, times out or is aborted returns the earlier rung's result.
12
+ - **`read_page`, for pages longer than one answer.** The host fetch request
13
+ carries only a url, so `web_fetch` can truncate but never continue. This tool
14
+ takes `offset` and returns `nextOffset`, and consecutive chunks concatenate to
15
+ exactly the full converted page — verified live against a long documentation
16
+ page. Pages are fetched once and sliced, so continuing does not download again.
17
+ - **A retrieval quality benchmark.** Twenty queries (ten Chinese, ten English)
18
+ across documentation, releases, packages and news, each with expected domains.
19
+ `scripts/bench.mjs` records real engine output once, then replays it offline so
20
+ a ranking change is a number, not an opinion. Metrics: precision@8, recall@8,
21
+ MRR@8, per-engine contribution and consensus. Baseline 0.3625 / 0.7417 / 0.7600.
22
+
23
+ ### Fixed
24
+ - An anti-bot page served with HTTP 200 was parsed as "zero results", which
25
+ removed an engine silently while keeping its slot reserved and never engaging
26
+ its cooldown. Such pages now report `SEARCH_BLOCKED`, and 360 — walled from
27
+ this network through both the direct and tunnel paths — moved to the fallback
28
+ tier so it no longer slows every search.
29
+
30
+ ### Measured, not assumed
31
+ - `rrfConstant` 40 scores a higher MRR (0.7767) than the default 60 (0.7600) and
32
+ a lower precision@8 (0.3500 vs 0.3625); `rrfConstant` 120 flips that trade in
33
+ the other direction. Twenty queries and one-to-four-hit gaps are not enough to
34
+ justify changing a default, so 60 stays and the numbers are in `bench/`.
35
+ - Dropping authority weighting costs 0.08 precision@8 and breaks Chinese
36
+ results, which is why it remains on by default.
37
+
38
+ ## 0.13.0
39
+
40
+ ### Added
41
+ - **Optional JS rendering.** A page whose HTML is an empty shell can be rendered
42
+ by a real browser — but only if `playwright` or `playwright-core` is already
43
+ installed. Nothing is bundled, nothing is imported at load time, and a missing
44
+ module is a silent fallback, not an error. `renderJs` picks `auto` (render only
45
+ suspected shells), `always` or `off`; `renderTimeoutMs` bounds the attempt. A
46
+ render that fails, times out or is aborted returns the earlier rung's result.
47
+ - **A retrieval quality benchmark.** Twenty queries (ten Chinese, ten English)
48
+ across documentation, releases, packages and news, each with expected domains.
49
+ `scripts/bench.mjs` records real engine output once, then replays it offline so
50
+ a ranking change is a number, not an opinion. Metrics: precision@8, recall@8,
51
+ MRR@8, per-engine contribution and consensus. Baseline 0.3625 / 0.7417 / 0.7600.
52
+
53
+ ### Fixed
54
+ - An anti-bot page served with HTTP 200 was parsed as "zero results", which
55
+ removed an engine silently while keeping its slot reserved and never engaging
56
+ its cooldown. Such pages now report `SEARCH_BLOCKED`, and 360 — walled from
57
+ this network through both the direct and tunnel paths — moved to the fallback
58
+ tier so it no longer slows every search.
59
+
60
+ ## 0.12.0
61
+
62
+ ### Added
63
+ - **Fetched pages come back as article Markdown**, not raw markup. Extraction
64
+ picks the article root, prunes page furniture, renders headings, lists, code
65
+ fences, tables, entities and absolute links, and cuts at `fetchMaxLength` with
66
+ a notice naming the original length. `fetchFormat` can return plain text.
67
+ - **`search_many`** answers one to five queries in a single call with a
68
+ concurrency cap of three and per-query error isolation.
69
+ - **The cache survives a restart.** A versioned snapshot keeps summaries only,
70
+ is written atomically, caps itself at 4 MiB with LRU eviction, and treats a
71
+ corrupt or unreadable file as an empty cache. `cachePersist` and `cacheFile`
72
+ control it.
73
+
74
+ ### Fixed
75
+ - `cacheTtlMs`, `cacheEntries`, `cachePersist` and `cacheFile` are read live, so
76
+ a Settings edit takes effect without a remount, like every other volatile
77
+ field.
78
+
79
+ - `fetchFormat` / `fetchMaxLength` are resolved per request, so the Settings page
80
+ no longer needs a reload for them either; only the cache file path and the
81
+ persist switch are read at mount.
82
+
83
+ ### Known limits
84
+ - The host fetch request carries only a url, so a page cannot be resumed at an
85
+ offset: raise `fetchMaxLength` or fetch a narrower page.
86
+ - Two processes writing the same cache file is last-writer-wins.
87
+
3
88
  ## 0.11.0
4
89
 
5
90
  ### Added