@xbzb404/dsh-context-compression-selector-runtime 0.0.0-stage → 0.2.1-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +13 -3
- package/README.zh.md +13 -0
- package/THIRD_PARTY_NOTICES.md +36 -0
- package/assets/deepseek-v4/LICENSE.DeepSeek-V4-Pro.txt +21 -0
- package/assets/deepseek-v4/manifest.json +22 -0
- package/assets/deepseek-v4/tokenizer.json +267359 -0
- package/assets/deepseek-v4/tokenizer_config.json +34 -0
- package/assets/deepseek-v4.1-flash/LICENSE.DeepSeek-V4.1-Flash.txt +21 -0
- package/assets/deepseek-v4.1-flash/manifest.json +24 -0
- package/assets/deepseek-v4.1-flash/tokenizer.json +267359 -0
- package/assets/deepseek-v4.1-flash/tokenizer_config.json +34 -0
- package/lib/index.d.ts +581 -0
- package/lib/index.js +3871 -0
- package/lib/invariant.d.ts +10 -0
- package/lib/invariant.js +70 -0
- package/lib/tail-trim.js +142 -0
- package/package.json +67 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 ZHB
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,13 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
|
|
1
|
+
# dsh-context-compression-selector-runtime
|
|
2
|
+
|
|
3
|
+
Internal runtime package for the unofficial community Bundle [`dsh-context-compression-selector`](https://github.com/xbzb404/dsh-context-compression-selector). Most users should install the Bundle, not this package directly.
|
|
4
|
+
|
|
5
|
+
It provides deterministic Fresh, Aggregate, History, Native tool-result, and Custom TailTrim compression; a plugin-owned `context_compression_retrieve` recovery tool; structured audit records; and a pinned offline DeepSeek V4 tokenizer. It uses only public DeepSeek Harness APIs and supports both `0.1.1-rc.2` and `0.1.2-alpha.5`; it does not patch Harness core.
|
|
6
|
+
|
|
7
|
+
Lossy rewrites require exact same-revision counts before and after replacement. Verified model ids are `deepseek-flash`, `deepseek-v4-flash`, `deepseek-v4-pro`, and `deepseek-v4-flash-vision-exp`. `deepseek-flash`, `deepseek-v4-flash`, and `deepseek-v4-flash-vision-exp` are served by DeepSeek-V4.1-Flash; `deepseek-v4-pro` keeps its own DeepSeek-V4-Pro tokenizer. Vision text counting uses the `deepseek-ai/DeepSeek-V4.1-Flash` tokenizer. Vision images use the official image-processor arithmetic validated against official golden fixtures and are reported as `tokenizer-estimate`: the block length is exactly `nLlmH * (nLlmW + 1) + 2` and is independent of the serialized position, so the padding minimum and maximum are equal, with 1024 tokens retained as the per-image upper bound; malformed or unevaluable dimensions use a fixed 256-token fallback. The estimate is not exact because the adapter's final image projection is not publicly observable. Image-bearing tool-result candidates remain exact-ineligible and intact; unknown models, unavailable tokenizer assets, unsafe tool groups, and incomplete text measurements still fail open.
|
|
8
|
+
|
|
9
|
+
Audit log records use the prefix `context-compression audit ` and distinguish policy snapshots, skipped/disabled components, committed rewrites, fail-open errors, and observed core `compaction/summary` events. No audit record contains prompt or tool-result content.
|
|
10
|
+
|
|
11
|
+
The first `policy-frozen` record carries the complete settings/deployment snapshot. It is emitted through the Harness logger, so its retention follows the configured log sink; standard prune/replacement Session events remain the durable proof of committed rewrites. This package does not hard-code a private path below `~/.dsh`.
|
|
12
|
+
|
|
13
|
+
See the [repository README](https://github.com/xbzb404/dsh-context-compression-selector#readme) for installation, profiles, compatibility, Adaptive/cache limitations, and release evidence. Tokenizer provenance and checksums are in `assets/deepseek-v4/manifest.json` and `assets/deepseek-v4.1-flash/manifest.json`, summarized in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
|
package/README.zh.md
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# dsh-context-compression-selector-runtime
|
|
2
|
+
|
|
3
|
+
这是非官方社区 Bundle [`dsh-context-compression-selector`](https://github.com/xbzb404/dsh-context-compression-selector) 的内部运行时包。普通用户应安装入口 Bundle,不需要直接安装本包。
|
|
4
|
+
|
|
5
|
+
本包提供确定性的 Fresh、Aggregate、History、Native 工具结果与 Custom TailTrim 压缩,插件自有的 `context_compression_retrieve` 恢复工具,结构化审计记录,以及固定版本的离线 DeepSeek V4 tokenizer。它只使用 DeepSeek Harness 的公开 API,兼容 `0.1.1-rc.2` 与 `0.1.2-alpha.5`,不修改 Harness 核心。
|
|
6
|
+
|
|
7
|
+
有损改写要求 replacement 前后取得同 revision 的 exact count。明确验证过的模型 id 为 `deepseek-flash`、`deepseek-v4-flash`、`deepseek-v4-pro` 和 `deepseek-v4-flash-vision-exp`。`deepseek-flash`、`deepseek-v4-flash` 与 `deepseek-v4-flash-vision-exp` 由 DeepSeek-V4.1-Flash 服务;`deepseek-v4-pro` 保留自有 DeepSeek-V4-Pro tokenizer。视觉模型的文本计数使用 `deepseek-ai/DeepSeek-V4.1-Flash` tokenizer。图片 token 使用由官方 golden fixtures 逐项验证的官方图像处理算术,并以 `tokenizer-estimate` 上报:块长度恒为 `nLlmH * (nLlmW + 1) + 2` 且与序列化位置无关,因此 padding 最小值与最大值相等,每张图片保留 1024 token 上限;尺寸畸形或无法计算时固定计为 256 token。由于 adapter 最终图片投影不可公开观测,该数值不标记为 exact。含图片的工具结果候选仍不具备 exact 资格并保持原样;未知模型、tokenizer 资产不可用、不安全工具组或文本计量不完整时继续 fail-open。
|
|
8
|
+
|
|
9
|
+
审计日志以 `context-compression audit ` 开头,区分策略快照、组件关闭/跳过、已提交 rewrite、fail-open 错误和已观察到的核心 `compaction/summary`。审计记录不包含 prompt 或工具结果正文。
|
|
10
|
+
|
|
11
|
+
首次 `policy-frozen` 会携带完整 settings/deployment 快照。它通过 Harness logger 输出,因此保留期限由部署的日志 sink 决定;标准 prune/replacement Session event 仍是已提交 rewrite 的持久证据。本包不会在 `~/.dsh` 下硬编码私有路径。
|
|
12
|
+
|
|
13
|
+
安装、Profile、兼容性、Adaptive/cache 限制与发布证据见[仓库 README](https://github.com/xbzb404/dsh-context-compression-selector#readme)。Tokenizer 来源与校验和见 `assets/deepseek-v4/manifest.json`、`assets/deepseek-v4.1-flash/manifest.json` 与 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Third-party notices
|
|
2
|
+
|
|
3
|
+
Portions of this package are adapted from the MIT-licensed DeepSeek Harness, verified against `dsh-v0.1.1-rc.2` (`b150a551b8d465e31e418e1b2eaf5e79bbb7d28e`).
|
|
4
|
+
|
|
5
|
+
## DeepSeek Harness MIT notice
|
|
6
|
+
|
|
7
|
+
Copyright (c) 2026 DeepSeek
|
|
8
|
+
|
|
9
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
10
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
11
|
+
in the Software without restriction, including without limitation the rights
|
|
12
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
13
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
14
|
+
furnished to do so, subject to the following conditions:
|
|
15
|
+
|
|
16
|
+
The above copyright notice and this permission notice shall be included in all
|
|
17
|
+
copies or substantial portions of the Software.
|
|
18
|
+
|
|
19
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
20
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
21
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
22
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
23
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
24
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
25
|
+
SOFTWARE.
|
|
26
|
+
|
|
27
|
+
This package also distributes `tokenizer.json` and `tokenizer_config.json` from these pinned official repositories, both under the MIT license:
|
|
28
|
+
|
|
29
|
+
- `deepseek-ai/DeepSeek-V4-Pro` revision `0e1a0e5e52aea73055f50fef6f2423db370265b6` (serves `deepseek-v4-pro`), in `assets/deepseek-v4/`.
|
|
30
|
+
- `deepseek-ai/DeepSeek-V4.1-Flash` revision `dba1be0a40aa45a94ad051997016db3960a90277` (serves `deepseek-flash`, `deepseek-v4-flash`, and `deepseek-v4-flash-vision-exp`), in `assets/deepseek-v4.1-flash/`. This tokenizer is a distinct artifact, not an alias of the V4-Pro tokenizer.
|
|
31
|
+
|
|
32
|
+
The exact provenance and SHA-256 values are in each directory's `manifest.json`; the upstream license text ships beside the assets. The vision image-token arithmetic is a port of the repository's `inference/image_processor.py` at the same pinned revision.
|
|
33
|
+
|
|
34
|
+
The runtime depends on `@huggingface/tokenizers` 0.1.3 under Apache-2.0. Its NPM package carries the dependency license text.
|
|
35
|
+
|
|
36
|
+
This is an unofficial community package and is not endorsed by DeepSeek.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2023 DeepSeek
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
{
|
|
2
|
+
"repository": "deepseek-ai/DeepSeek-V4-Pro",
|
|
3
|
+
"revision": "0e1a0e5e52aea73055f50fef6f2423db370265b6",
|
|
4
|
+
"license": "MIT",
|
|
5
|
+
"modelIds": [
|
|
6
|
+
"deepseek-v4-pro"
|
|
7
|
+
],
|
|
8
|
+
"files": {
|
|
9
|
+
"tokenizer.json": {
|
|
10
|
+
"bytes": 6367146,
|
|
11
|
+
"sha256": "8f9f37ca37fdc4f5fd36d5cf4d3b0e8392edb4e894fd10cc0d70b4957c8633cf"
|
|
12
|
+
},
|
|
13
|
+
"tokenizer_config.json": {
|
|
14
|
+
"bytes": 801,
|
|
15
|
+
"sha256": "6ac8c8dc065ed118161d02dd532749ae3f52c243deac27872134fae2f50d8547"
|
|
16
|
+
},
|
|
17
|
+
"LICENSE.DeepSeek-V4-Pro.txt": {
|
|
18
|
+
"bytes": 1064,
|
|
19
|
+
"sha256": "1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7"
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
}
|