anolisa-tokenless 0.7.9 → 0.7.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -7
- package/adapters/tokenless/claude-code/.claude-plugin/plugin.json +1 -1
- package/adapters/tokenless/codex/.codex-plugin/plugin.json +1 -1
- package/adapters/tokenless/common/cosh-extension.json +1 -1
- package/adapters/tokenless/common/hooks/hook_utils.py +1 -1
- package/adapters/tokenless/common/hooks/tool_ready_hook.sh +3 -3
- package/adapters/tokenless/dsh/package.json +1 -1
- package/adapters/tokenless/hermes/plugin.yaml +1 -1
- package/adapters/tokenless/manifest.json +1 -1
- package/adapters/tokenless/openclaw/openclaw.plugin.json +1 -1
- package/adapters/tokenless/openclaw/package.json +1 -1
- package/adapters/tokenless/qoder/.qoder-plugin/plugin.json +1 -1
- package/adapters/tokenless/qwencode/qwen-extension.json +1 -1
- package/package.json +6 -7
- package/scripts/postinstall.js +3 -3
- package/bin/toon +0 -6
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
Token-Less combines complementary strategies to minimize LLM token consumption:
|
|
8
8
|
|
|
9
9
|
- **Schema & Response Compression** — Compresses OpenAI Function Calling tool definitions and API responses via the `tokenless-schema` library, cutting structural overhead before tokens ever reach the context window.
|
|
10
|
-
- **TOON Context Compression** — Encodes JSON responses to TOON (Token-Oriented Object Notation) format via the `toon`
|
|
10
|
+
- **TOON Context Compression** — Encodes JSON responses to TOON (Token-Oriented Object Notation) format via the `toon-format` library linked into `tokenless`, reducing token usage by 15-40% for structured data.
|
|
11
11
|
- **Command Rewriting** — Integrates [RTK](https://github.com/rtk-ai/rtk) to filter and rewrite CLI command output, eliminating noise that would otherwise waste 60–90% of tokens.
|
|
12
12
|
- **Tool Ready (legacy, hard-disabled)** — Its pre-call dependency checks are retained in source but unconditionally bypassed while the readiness model is redesigned.
|
|
13
13
|
|
|
@@ -119,7 +119,7 @@ Token-Less/
|
|
|
119
119
|
Install the published component with the ANOLISA CLI:
|
|
120
120
|
|
|
121
121
|
The install script places `anolisa` in `~/.local/bin`, and a user-mode
|
|
122
|
-
Tokenless installation places `tokenless
|
|
122
|
+
Tokenless installation places `tokenless` and `rtk` in that same
|
|
123
123
|
directory. Export it once if the current shell has not picked it up yet.
|
|
124
124
|
|
|
125
125
|
```bash
|
|
@@ -180,8 +180,8 @@ cd Token-Less
|
|
|
180
180
|
make setup
|
|
181
181
|
```
|
|
182
182
|
|
|
183
|
-
The source setup installs `tokenless` to `~/.local/bin`, places the `rtk`
|
|
184
|
-
|
|
183
|
+
The source setup installs `tokenless` to `~/.local/bin`, places the `rtk`
|
|
184
|
+
helper alongside it, and deploys all adapters for development.
|
|
185
185
|
|
|
186
186
|
### Build the Python runtime
|
|
187
187
|
|
|
@@ -208,6 +208,28 @@ package is built and tested in this repository but is not yet published to
|
|
|
208
208
|
PyPI. See the [runtime design](docs/design/runtime-library.md)
|
|
209
209
|
and the [user manual](../../docs/user-guide/en/token-saving/tokenless/user-manual.md#build-the-python-runtime-from-source).
|
|
210
210
|
|
|
211
|
+
The same wheel provides typed, read-only statistics queries without requiring
|
|
212
|
+
the CLI. Point `TokenlessStats` at the state directory used by the runtime, or
|
|
213
|
+
use the lazy `sdk.stats` client:
|
|
214
|
+
|
|
215
|
+
```python
|
|
216
|
+
from anolisa_tokenless import TokenlessStats
|
|
217
|
+
|
|
218
|
+
stats = TokenlessStats("/absolute/path/to/tokenless-data")
|
|
219
|
+
summary = stats.summary()
|
|
220
|
+
print(summary.total.tokens_saved, summary.total.tokens_saved_percent)
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Token counts are estimates and only operations with positive savings are
|
|
224
|
+
recorded. `show()` and detailed `diff()` results may contain sensitive tool
|
|
225
|
+
input and output stored in `stats.db`. Read-only describes the API surface:
|
|
226
|
+
opening the client follows CLI initialization and may create or migrate
|
|
227
|
+
`stats.db`, so the data directory must be writable. `summary(limit=None)` and
|
|
228
|
+
`compare(..., limit=None)` inspect at most the newest 10,000 records. For a
|
|
229
|
+
session or tool-use diff, at most the newest 10,000 matching records are read.
|
|
230
|
+
For a meaningful comparison, pass a dry-run session first and an active
|
|
231
|
+
Tokenless session second.
|
|
232
|
+
|
|
211
233
|
## CLI Usage
|
|
212
234
|
|
|
213
235
|
### compress-schema
|
|
@@ -673,9 +695,8 @@ transformed. Tool Ready remains hard-disabled.
|
|
|
673
695
|
|
|
674
696
|
| Target | Description |
|
|
675
697
|
|---|---|
|
|
676
|
-
| `make build` | Build `tokenless` + `rtk`
|
|
698
|
+
| `make build` | Build `tokenless` + `rtk` (release mode) |
|
|
677
699
|
| `make build-tokenless` | Build `tokenless` + `rtk` (via justfile) |
|
|
678
|
-
| `make build-toon` | Install TOON binary via `cargo install toon-format` |
|
|
679
700
|
| `make python-wheel` | Build the native `anolisa-tokenless` wheel |
|
|
680
701
|
| `make agentscope-wheel` | Build the pure-Python AgentScope integration wheel |
|
|
681
702
|
| `make test-python-runtime` | Install and test the wheel in an isolated environment |
|
|
@@ -713,7 +734,7 @@ make install BIN_DIR=/usr/local/bin
|
|
|
713
734
|
|
|
714
735
|
## Raw Packaging
|
|
715
736
|
|
|
716
|
-
Raw packaging accepts already-built `tokenless
|
|
737
|
+
Raw packaging accepts already-built `tokenless` and `rtk`
|
|
717
738
|
executables in one directory and applies the stable component payload layout:
|
|
718
739
|
|
|
719
740
|
```bash
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tokenless",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.10",
|
|
4
4
|
"description": "Token-Less context compression for Claude Code — RTK command rewriting, response/TOON compression, and Tool Ready environment pre-check",
|
|
5
5
|
"author": { "name": "ANOLISA" },
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tokenless",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.10",
|
|
4
4
|
"description": "Intelligent tool response compression, environment error detection, and token optimization. Strips noise (debug fields, nulls, empty values), truncates long strings/arrays, applies TOON encoding for JSON responses, and classifies environment errors with actionable fix hints.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"compression",
|
|
@@ -47,7 +47,7 @@ _RTK_LOCAL_SHARE = _user_path(
|
|
|
47
47
|
)
|
|
48
48
|
_RTK_LOCAL_LIB = _user_path(".local", "lib", "anolisa", "tokenless", "rtk")
|
|
49
49
|
|
|
50
|
-
_TOKENLESS_HELPER_BINARIES = frozenset({"rtk"
|
|
50
|
+
_TOKENLESS_HELPER_BINARIES = frozenset({"rtk"})
|
|
51
51
|
|
|
52
52
|
|
|
53
53
|
def _known_binary_paths(name: str, home: str | None = None) -> tuple[str, ...]:
|
|
@@ -295,7 +295,7 @@ resolve_binary() {
|
|
|
295
295
|
candidates+=("$USER_HOME/.local/bin/$name")
|
|
296
296
|
fi
|
|
297
297
|
case "$name" in
|
|
298
|
-
rtk
|
|
298
|
+
rtk)
|
|
299
299
|
if [ -n "$USER_HOME" ]; then
|
|
300
300
|
candidates+=(
|
|
301
301
|
"$USER_HOME/.local/lib/anolisa/libexec/tokenless/$name"
|
|
@@ -306,13 +306,13 @@ resolve_binary() {
|
|
|
306
306
|
esac
|
|
307
307
|
candidates+=("/usr/local/bin/$name")
|
|
308
308
|
case "$name" in
|
|
309
|
-
rtk
|
|
309
|
+
rtk)
|
|
310
310
|
candidates+=("/usr/local/libexec/anolisa/tokenless/$name")
|
|
311
311
|
;;
|
|
312
312
|
esac
|
|
313
313
|
candidates+=("/usr/bin/$name")
|
|
314
314
|
case "$name" in
|
|
315
|
-
rtk
|
|
315
|
+
rtk)
|
|
316
316
|
candidates+=(
|
|
317
317
|
"/usr/libexec/anolisa/tokenless/$name"
|
|
318
318
|
"/usr/lib/anolisa/tokenless/$name"
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"id": "tokenless",
|
|
3
3
|
"name": "Tokenless",
|
|
4
|
-
"version": "0.7.
|
|
4
|
+
"version": "0.7.10",
|
|
5
5
|
"description": "Unified RTK command rewriting + response/TOON compression + registered but hard-disabled Tool Ready. Wraps tokenless and rtk system binaries via child_process — this is expected and not malicious.",
|
|
6
6
|
"activation": {
|
|
7
7
|
"onCapabilities": ["hook"]
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@tokenless/openclaw-plugin",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.10",
|
|
4
4
|
"description": "Unified OpenClaw plugin — RTK command rewriting + tokenless schema/response compression for 60-90% LLM token savings",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tokenless",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.10",
|
|
4
4
|
"description": "Token-Less context compression for Qwen Code — RTK command rewriting, response/TOON/schema compression, and registered but hard-disabled Tool Ready",
|
|
5
5
|
"author": { "name": "ANOLISA" },
|
|
6
6
|
"license": "Apache-2.0",
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "anolisa-tokenless",
|
|
3
3
|
"type": "module",
|
|
4
|
-
"version": "0.7.
|
|
4
|
+
"version": "0.7.10",
|
|
5
5
|
"description": "Token-Less — LLM token optimization toolkit (schema/response compression, command rewriting, tool readiness)",
|
|
6
6
|
"license": "Apache-2.0",
|
|
7
7
|
"repository": {
|
|
@@ -20,8 +20,7 @@
|
|
|
20
20
|
],
|
|
21
21
|
"bin": {
|
|
22
22
|
"tokenless": "bin/tokenless",
|
|
23
|
-
"rtk": "bin/rtk"
|
|
24
|
-
"toon": "bin/toon"
|
|
23
|
+
"rtk": "bin/rtk"
|
|
25
24
|
},
|
|
26
25
|
"files": [
|
|
27
26
|
"bin/",
|
|
@@ -45,10 +44,10 @@
|
|
|
45
44
|
"arm64"
|
|
46
45
|
],
|
|
47
46
|
"optionalDependencies": {
|
|
48
|
-
"@anolisa/tokenless-linux-x64": "0.7.
|
|
49
|
-
"@anolisa/tokenless-linux-arm64": "0.7.
|
|
50
|
-
"@anolisa/tokenless-darwin-x64": "0.7.
|
|
51
|
-
"@anolisa/tokenless-darwin-arm64": "0.7.
|
|
47
|
+
"@anolisa/tokenless-linux-x64": "0.7.10",
|
|
48
|
+
"@anolisa/tokenless-linux-arm64": "0.7.10",
|
|
49
|
+
"@anolisa/tokenless-darwin-x64": "0.7.10",
|
|
50
|
+
"@anolisa/tokenless-darwin-arm64": "0.7.10"
|
|
52
51
|
},
|
|
53
52
|
"publishConfig": {
|
|
54
53
|
"registry": "https://registry.npmjs.org/",
|
package/scripts/postinstall.js
CHANGED
|
@@ -17,8 +17,8 @@
|
|
|
17
17
|
* Platform packages follow the naming convention:
|
|
18
18
|
* @anolisa/tokenless-{os}-{arch}
|
|
19
19
|
*
|
|
20
|
-
* Each platform package ships
|
|
21
|
-
* bin/tokenless, bin/rtk
|
|
20
|
+
* Each platform package ships two native binaries:
|
|
21
|
+
* bin/tokenless, bin/rtk
|
|
22
22
|
*
|
|
23
23
|
* Exit codes: on a supported platform, a missing platform package or a
|
|
24
24
|
* missing binary is a hard failure (non-zero exit) so `npm install` fails
|
|
@@ -47,7 +47,7 @@ const require = createRequire(import.meta.url);
|
|
|
47
47
|
const packageRoot = join(__dirname, '..');
|
|
48
48
|
const binDir = join(packageRoot, 'bin');
|
|
49
49
|
|
|
50
|
-
const BINARIES = ['tokenless', 'rtk'
|
|
50
|
+
const BINARIES = ['tokenless', 'rtk'];
|
|
51
51
|
|
|
52
52
|
// Map Node.js platform/arch to package names
|
|
53
53
|
const PLATFORM_MAP = {
|