anolisa-tokenless 0.7.9 → 0.7.11

package/README.md

@@ -7,7 +7,7 @@
 Token-Less combines complementary strategies to minimize LLM token consumption:
 
 - **Schema & Response Compression** — Compresses OpenAI Function Calling tool definitions and API responses via the `tokenless-schema` library, cutting structural overhead before tokens ever reach the context window.
-- **TOON Context Compression** — Encodes JSON responses to TOON (Token-Oriented Object Notation) format via the `toon` binary, reducing token usage by 15-40% for structured data.
+- **TOON Context Compression** — Encodes JSON responses to TOON (Token-Oriented Object Notation) format via the `toon-format` library linked into `tokenless`, reducing token usage by 15-40% for structured data.
 - **Command Rewriting** — Integrates [RTK](https://github.com/rtk-ai/rtk) to filter and rewrite CLI command output, eliminating noise that would otherwise waste 60–90% of tokens.
 - **Tool Ready (legacy, hard-disabled)** — Its pre-call dependency checks are retained in source but unconditionally bypassed while the readiness model is redesigned.
 
@@ -62,7 +62,13 @@ tokenless only removes redundancy from **tool call responses** before they enter
 ### Where it pays little or doesn't apply
 
 - **Chat-heavy / few tool calls**: tool-response share is tiny, overall savings approach 0.
-- **Already-short responses**: when `after >= before`, the CLI emits the original and records no stats (expected).
+- **No fixed minimum payload**: `compress-schema` and `compress-response` build a
+  candidate for every accepted valid JSON input. In active mode, they emit it only
+  when its estimated token count is strictly lower than the original. A small input
+  with removable content can still compress, while a larger already-compact input can
+  pass through unchanged; the CLI writes the reason to stderr and records no stats.
+  In dry-run mode, the CLI always emits the original and may record a smaller candidate
+  as a predicted saving.
 - **Model inference tokens / billed tokens**: outside what tokenless touches.
 
 ### Estimating the effect
@@ -119,7 +125,7 @@ Token-Less/
 Install the published component with the ANOLISA CLI:
 
 The install script places `anolisa` in `~/.local/bin`, and a user-mode
-Tokenless installation places `tokenless`, `rtk`, and `toon` in that same
+Tokenless installation places `tokenless` and `rtk` in that same
 directory. Export it once if the current shell has not picked it up yet.
 
 ```bash
@@ -180,8 +186,8 @@ cd Token-Less
 make setup
 ```
 
-The source setup installs `tokenless` to `~/.local/bin`, places the `rtk` and
-`toon` helpers alongside it, and deploys all adapters for development.
+The source setup installs `tokenless` to `~/.local/bin`, places the `rtk`
+helper alongside it, and deploys all adapters for development.
 
 ### Build the Python runtime
 
@@ -208,8 +214,38 @@ package is built and tested in this repository but is not yet published to
 PyPI. See the [runtime design](docs/design/runtime-library.md)
 and the [user manual](../../docs/user-guide/en/token-saving/tokenless/user-manual.md#build-the-python-runtime-from-source).
 
+The same wheel provides typed, read-only statistics queries without requiring
+the CLI. Point `TokenlessStats` at the state directory used by the runtime, or
+use the lazy `sdk.stats` client:
+
+```python
+from anolisa_tokenless import TokenlessStats
+
+stats = TokenlessStats("/absolute/path/to/tokenless-data")
+summary = stats.summary()
+print(summary.total.tokens_saved, summary.total.tokens_saved_percent)
+```
+
+Token counts are estimates and only operations with positive savings are
+recorded. `show()` and detailed `diff()` results may contain sensitive tool
+input and output stored in `stats.db`. Read-only describes the API surface:
+opening the client follows CLI initialization and may create or migrate
+`stats.db`, so the data directory must be writable. `summary(limit=None)` and
+`compare(..., limit=None)` inspect at most the newest 10,000 records. For a
+session or tool-use diff, at most the newest 10,000 matching records are read.
+For a meaningful comparison, pass a dry-run session first and an active
+Tokenless session second.
+
 ## CLI Usage
 
+The standalone `compress-schema` and `compress-response` commands use this
+content-dependent savings check rather than a fixed byte or character minimum.
+The description, string, array, and depth limits in the
+[CLI reference](../../docs/user-guide/en/token-saving/tokenless/cli-reference.md)
+trigger individual transformations; they are not minimum total payload sizes.
+Agent adapters may apply separate pre-check thresholds; see the
+[framework integration guide](../../docs/user-guide/en/token-saving/tokenless/framework-integration.md#adapter-processing-rules).
+
 ### compress-schema
 
 Compress a single tool schema:
@@ -673,9 +709,8 @@ transformed. Tool Ready remains hard-disabled.
 
 | Target | Description |
 |---|---|
-| `make build` | Build `tokenless` + `rtk` + `toon` (release mode) |
+| `make build` | Build `tokenless` + `rtk` (release mode) |
 | `make build-tokenless` | Build `tokenless` + `rtk` (via justfile) |
-| `make build-toon` | Install TOON binary via `cargo install toon-format` |
 | `make python-wheel` | Build the native `anolisa-tokenless` wheel |
 | `make agentscope-wheel` | Build the pure-Python AgentScope integration wheel |
 | `make test-python-runtime` | Install and test the wheel in an isolated environment |
@@ -713,7 +748,7 @@ make install BIN_DIR=/usr/local/bin
 
 ## Raw Packaging
 
-Raw packaging accepts already-built `tokenless`, `rtk`, and `toon`
+Raw packaging accepts already-built `tokenless` and `rtk`
 executables in one directory and applies the stable component payload layout:
 
 ```bash

package/adapters/tokenless/claude-code/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Token-Less context compression for Claude Code — RTK command rewriting, response/TOON compression, and Tool Ready environment pre-check",
   "author": { "name": "ANOLISA" },
   "license": "Apache-2.0",

package/adapters/tokenless/codex/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Intelligent tool response compression, environment error detection, and token optimization. Strips noise (debug fields, nulls, empty values), truncates long strings/arrays, applies TOON encoding for JSON responses, and classifies environment errors with actionable fix hints.",
   "keywords": [
     "compression",

package/adapters/tokenless/common/cosh-extension.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "contextFileName": "COPILOT.md",
   "hooks": {
     "PreToolUse": [

package/adapters/tokenless/common/hooks/hook_utils.py

@@ -47,7 +47,7 @@ _RTK_LOCAL_SHARE = _user_path(
 )
 _RTK_LOCAL_LIB = _user_path(".local", "lib", "anolisa", "tokenless", "rtk")
 
-_TOKENLESS_HELPER_BINARIES = frozenset({"rtk", "toon"})
+_TOKENLESS_HELPER_BINARIES = frozenset({"rtk"})
 
 
 def _known_binary_paths(name: str, home: str | None = None) -> tuple[str, ...]:

package/adapters/tokenless/common/hooks/tool_ready_hook.sh

@@ -295,7 +295,7 @@ resolve_binary() {
     candidates+=("$USER_HOME/.local/bin/$name")
   fi
   case "$name" in
-    rtk|toon)
+    rtk)
       if [ -n "$USER_HOME" ]; then
         candidates+=(
           "$USER_HOME/.local/lib/anolisa/libexec/tokenless/$name"
@@ -306,13 +306,13 @@ resolve_binary() {
   esac
   candidates+=("/usr/local/bin/$name")
   case "$name" in
-    rtk|toon)
+    rtk)
       candidates+=("/usr/local/libexec/anolisa/tokenless/$name")
       ;;
   esac
   candidates+=("/usr/bin/$name")
   case "$name" in
-    rtk|toon)
+    rtk)
       candidates+=(
         "/usr/libexec/anolisa/tokenless/$name"
         "/usr/lib/anolisa/tokenless/$name"

package/adapters/tokenless/dsh/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@anolisa/dsh-tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Native DeepSeek Harness response compression for Tokenless",
   "type": "module",
   "main": "./dist/index.js",

package/adapters/tokenless/hermes/plugin.yaml

@@ -1,5 +1,5 @@
 name: tokenless
-version: "0.7.9"
+version: "0.7.11"
 description: "Token-Less context compression for Hermes Agent — response compression, TOON encoding, command rewriting, and registered but hard-disabled Tool Ready"
 author: ANOLISA
 requires_env: []

package/adapters/tokenless/manifest.json

@@ -1,6 +1,6 @@
 {
   "component": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "targets": {
     "cosh": {
       "compatibleVersions": "*",

package/adapters/tokenless/openclaw/openclaw.plugin.json

@@ -1,7 +1,7 @@
 {
   "id": "tokenless",
   "name": "Tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Unified RTK command rewriting + response/TOON compression + registered but hard-disabled Tool Ready. Wraps tokenless and rtk system binaries via child_process — this is expected and not malicious.",
   "activation": {
     "onCapabilities": ["hook"]

package/adapters/tokenless/openclaw/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tokenless/openclaw-plugin",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Unified OpenClaw plugin — RTK command rewriting + tokenless schema/response compression for 60-90% LLM token savings",
   "type": "module",
   "main": "dist/index.js",

package/adapters/tokenless/qoder/.qoder-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Token-Less LLM Token Optimization Toolkit for Qoder CLI",
   "author": {
     "name": "Shile Zhang",

package/adapters/tokenless/qwencode/qwen-extension.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenless",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Token-Less context compression for Qwen Code — RTK command rewriting, response/TOON/schema compression, and registered but hard-disabled Tool Ready",
   "author": { "name": "ANOLISA" },
   "license": "Apache-2.0",

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "anolisa-tokenless",
   "type": "module",
-  "version": "0.7.9",
+  "version": "0.7.11",
   "description": "Token-Less — LLM token optimization toolkit (schema/response compression, command rewriting, tool readiness)",
   "license": "Apache-2.0",
   "repository": {
@@ -20,8 +20,7 @@
   ],
   "bin": {
     "tokenless": "bin/tokenless",
-    "rtk": "bin/rtk",
-    "toon": "bin/toon"
+    "rtk": "bin/rtk"
   },
   "files": [
     "bin/",
@@ -45,10 +44,10 @@
     "arm64"
   ],
   "optionalDependencies": {
-    "@anolisa/tokenless-linux-x64": "0.7.9",
-    "@anolisa/tokenless-linux-arm64": "0.7.9",
-    "@anolisa/tokenless-darwin-x64": "0.7.9",
-    "@anolisa/tokenless-darwin-arm64": "0.7.9"
+    "@anolisa/tokenless-linux-x64": "0.7.11",
+    "@anolisa/tokenless-linux-arm64": "0.7.11",
+    "@anolisa/tokenless-darwin-x64": "0.7.11",
+    "@anolisa/tokenless-darwin-arm64": "0.7.11"
   },
   "publishConfig": {
     "registry": "https://registry.npmjs.org/",

package/scripts/postinstall.js

@@ -17,8 +17,8 @@
  * Platform packages follow the naming convention:
  *   @anolisa/tokenless-{os}-{arch}
  *
- * Each platform package ships three native binaries:
- *   bin/tokenless, bin/rtk, bin/toon
+ * Each platform package ships two native binaries:
+ *   bin/tokenless, bin/rtk
  *
  * Exit codes: on a supported platform, a missing platform package or a
  * missing binary is a hard failure (non-zero exit) so `npm install` fails
@@ -47,7 +47,7 @@ const require = createRequire(import.meta.url);
 const packageRoot = join(__dirname, '..');
 const binDir = join(packageRoot, 'bin');
 
-const BINARIES = ['tokenless', 'rtk', 'toon'];
+const BINARIES = ['tokenless', 'rtk'];
 
 // Map Node.js platform/arch to package names
 const PLATFORM_MAP = {

package/bin/toon

@@ -1,6 +0,0 @@
-#!/usr/bin/env node
-console.error(
-  'anolisa-tokenless: postinstall has not run yet. ' +
-  'Run "npm rebuild anolisa-tokenless" to fix.',
-);
-process.exit(1);