llm-token-saver 0.1.0__tar.gz

llm_token_saver-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 李泉
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

llm_token_saver-0.1.0/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.4
+Name: llm-token-saver
+Version: 0.1.0
+Summary: A seamless LLM API middleware to maximize context cache hits and reduce token costs.
+License: MIT
+Project-URL: Homepage, https://github.com/quan0618/token-saver
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai>=1.0.0
+Dynamic: license-file
+
+```markdown
+# 🛡️ Token-Saver
+[![PyPI version](https://img.shields.io/badge/pypi-v0.1.0-blue.svg)](https://pypi.org/project/token-saver/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+**Token-Saver** 是一个极致轻量、无侵入的 LLM API 中间件。只需一行代码，就能自动优化你的 Prompt 结构，大幅提升上下文缓存（Context Cache）命中率，并自动修剪冗长的对话历史，帮你直接砍掉 50%+ 的 API 账单！
+## ✨ 核心特性
+- **⚡ 零侵入接入**：完全兼容 OpenAI 官方 SDK 格式，无需重写业务逻辑。
+- **🎯 前缀稳定器 (Prefix Stabilizer)**：自动提取 System Prompt 中的动态变量（如时间戳、随机 ID）到末尾，确保请求前缀字节级一致，强制触发缓存命中。
+- **✂️ 上下文瘦身 (Context Pruner)**：自动检测并截断超长对话历史，防止输入 Token 无限膨胀。
+- **🤐 输出防废话 (Output Constraint)**：自动注入隐藏提示词，让大模型直接给结果，减少输出 Token 浪费。
+
+## 📦 安装
+
+```bash
+pip install token-saver
+```
+
+## 🚀 极简用法
+
+只需要用 `TokenSaver` 包装你现有的官方客户端，其他代码完全不用改：
+
+```python
+from openai import OpenAI
+from token_saver import TokenSaver
+
+# 1. 初始化原生客户端
+raw_client = OpenAI(api_key="your-api-key", base_url="https://api.deepseek.com/v1")
+
+# 2. 用 TokenSaver 包裹它！
+client = TokenSaver(client=raw_client)
+
+# 3. 像平常一样调用 (Token-Saver 会在后台自动优化)
+response = client.chat.completions.create(
+    model="deepseek-chat",
+    messages=[
+        {"role": "system", "content": "你是一个助手。当前时间：2024-05-20 12:00:00"},
+        {"role": "user", "content": "给我讲个笑话"}
+    ]
+)
+```
+
+## 🛠️ 高级配置
+
+你可以自定义需要保留的上下文轮数：
+
+```python
+# 只保留最近的 5 条历史消息
+client = TokenSaver(client=raw_client, max_history=5)
+```
+
+## 🤝 贡献与反馈
+
+欢迎提交 Issue 或 Pull Request！如果你觉得这个项目帮你省了钱，请给个 ⭐️ Star 支持一下！
+```
+
+---
+
+### 2. 添加 `pyproject.toml`（安装配置文件）
+
+在你的本地仓库根目录，新建一个文件叫 `pyproject.toml`，把下面的代码贴进去。这是目前 Python 打包的标准格式：
+
+```toml
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "token-saver"
+version = "0.1.0"
+description = "A seamless LLM API middleware to maximize context cache hits and reduce token costs."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+dependencies = [
+    "openai>=1.0.0",
+]
+
+```

llm_token_saver-0.1.0/README.md

@@ -0,0 +1,80 @@
+```markdown
+# 🛡️ Token-Saver
+[![PyPI version](https://img.shields.io/badge/pypi-v0.1.0-blue.svg)](https://pypi.org/project/token-saver/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+**Token-Saver** 是一个极致轻量、无侵入的 LLM API 中间件。只需一行代码，就能自动优化你的 Prompt 结构，大幅提升上下文缓存（Context Cache）命中率，并自动修剪冗长的对话历史，帮你直接砍掉 50%+ 的 API 账单！
+## ✨ 核心特性
+- **⚡ 零侵入接入**：完全兼容 OpenAI 官方 SDK 格式，无需重写业务逻辑。
+- **🎯 前缀稳定器 (Prefix Stabilizer)**：自动提取 System Prompt 中的动态变量（如时间戳、随机 ID）到末尾，确保请求前缀字节级一致，强制触发缓存命中。
+- **✂️ 上下文瘦身 (Context Pruner)**：自动检测并截断超长对话历史，防止输入 Token 无限膨胀。
+- **🤐 输出防废话 (Output Constraint)**：自动注入隐藏提示词，让大模型直接给结果，减少输出 Token 浪费。
+
+## 📦 安装
+
+```bash
+pip install token-saver
+```
+
+## 🚀 极简用法
+
+只需要用 `TokenSaver` 包装你现有的官方客户端，其他代码完全不用改：
+
+```python
+from openai import OpenAI
+from token_saver import TokenSaver
+
+# 1. 初始化原生客户端
+raw_client = OpenAI(api_key="your-api-key", base_url="https://api.deepseek.com/v1")
+
+# 2. 用 TokenSaver 包裹它！
+client = TokenSaver(client=raw_client)
+
+# 3. 像平常一样调用 (Token-Saver 会在后台自动优化)
+response = client.chat.completions.create(
+    model="deepseek-chat",
+    messages=[
+        {"role": "system", "content": "你是一个助手。当前时间：2024-05-20 12:00:00"},
+        {"role": "user", "content": "给我讲个笑话"}
+    ]
+)
+```
+
+## 🛠️ 高级配置
+
+你可以自定义需要保留的上下文轮数：
+
+```python
+# 只保留最近的 5 条历史消息
+client = TokenSaver(client=raw_client, max_history=5)
+```
+
+## 🤝 贡献与反馈
+
+欢迎提交 Issue 或 Pull Request！如果你觉得这个项目帮你省了钱，请给个 ⭐️ Star 支持一下！
+```
+
+---
+
+### 2. 添加 `pyproject.toml`（安装配置文件）
+
+在你的本地仓库根目录，新建一个文件叫 `pyproject.toml`，把下面的代码贴进去。这是目前 Python 打包的标准格式：
+
+```toml
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "token-saver"
+version = "0.1.0"
+description = "A seamless LLM API middleware to maximize context cache hits and reduce token costs."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+dependencies = [
+    "openai>=1.0.0",
+]
+
+```

llm_token_saver-0.1.0/llm_token_saver.egg-info/PKG-INFO

@@ -0,0 +1,92 @@
+Metadata-Version: 2.4
+Name: llm-token-saver
+Version: 0.1.0
+Summary: A seamless LLM API middleware to maximize context cache hits and reduce token costs.
+License: MIT
+Project-URL: Homepage, https://github.com/quan0618/token-saver
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai>=1.0.0
+Dynamic: license-file
+
+```markdown
+# 🛡️ Token-Saver
+[![PyPI version](https://img.shields.io/badge/pypi-v0.1.0-blue.svg)](https://pypi.org/project/token-saver/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+**Token-Saver** 是一个极致轻量、无侵入的 LLM API 中间件。只需一行代码，就能自动优化你的 Prompt 结构，大幅提升上下文缓存（Context Cache）命中率，并自动修剪冗长的对话历史，帮你直接砍掉 50%+ 的 API 账单！
+## ✨ 核心特性
+- **⚡ 零侵入接入**：完全兼容 OpenAI 官方 SDK 格式，无需重写业务逻辑。
+- **🎯 前缀稳定器 (Prefix Stabilizer)**：自动提取 System Prompt 中的动态变量（如时间戳、随机 ID）到末尾，确保请求前缀字节级一致，强制触发缓存命中。
+- **✂️ 上下文瘦身 (Context Pruner)**：自动检测并截断超长对话历史，防止输入 Token 无限膨胀。
+- **🤐 输出防废话 (Output Constraint)**：自动注入隐藏提示词，让大模型直接给结果，减少输出 Token 浪费。
+
+## 📦 安装
+
+```bash
+pip install token-saver
+```
+
+## 🚀 极简用法
+
+只需要用 `TokenSaver` 包装你现有的官方客户端，其他代码完全不用改：
+
+```python
+from openai import OpenAI
+from token_saver import TokenSaver
+
+# 1. 初始化原生客户端
+raw_client = OpenAI(api_key="your-api-key", base_url="https://api.deepseek.com/v1")
+
+# 2. 用 TokenSaver 包裹它！
+client = TokenSaver(client=raw_client)
+
+# 3. 像平常一样调用 (Token-Saver 会在后台自动优化)
+response = client.chat.completions.create(
+    model="deepseek-chat",
+    messages=[
+        {"role": "system", "content": "你是一个助手。当前时间：2024-05-20 12:00:00"},
+        {"role": "user", "content": "给我讲个笑话"}
+    ]
+)
+```
+
+## 🛠️ 高级配置
+
+你可以自定义需要保留的上下文轮数：
+
+```python
+# 只保留最近的 5 条历史消息
+client = TokenSaver(client=raw_client, max_history=5)
+```
+
+## 🤝 贡献与反馈
+
+欢迎提交 Issue 或 Pull Request！如果你觉得这个项目帮你省了钱，请给个 ⭐️ Star 支持一下！
+```
+
+---
+
+### 2. 添加 `pyproject.toml`（安装配置文件）
+
+在你的本地仓库根目录，新建一个文件叫 `pyproject.toml`，把下面的代码贴进去。这是目前 Python 打包的标准格式：
+
+```toml
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "token-saver"
+version = "0.1.0"
+description = "A seamless LLM API middleware to maximize context cache hits and reduce token costs."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+dependencies = [
+    "openai>=1.0.0",
+]
+
+```

llm_token_saver-0.1.0/llm_token_saver.egg-info/SOURCES.txt

@@ -0,0 +1,15 @@
+LICENSE
+README.md
+pyproject.toml
+llm_token_saver.egg-info/PKG-INFO
+llm_token_saver.egg-info/SOURCES.txt
+llm_token_saver.egg-info/dependency_links.txt
+llm_token_saver.egg-info/requires.txt
+llm_token_saver.egg-info/top_level.txt
+token_saver/__init__.py
+token_saver/client.py
+token_saver/optimizers/__init__.py
+token_saver/optimizers/base.py
+token_saver/optimizers/context.py
+token_saver/optimizers/output.py
+token_saver/optimizers/prefix.py

llm_token_saver-0.1.0/llm_token_saver.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

llm_token_saver-0.1.0/llm_token_saver.egg-info/requires.txt

@@ -0,0 +1 @@
+openai>=1.0.0

llm_token_saver-0.1.0/llm_token_saver.egg-info/top_level.txt

@@ -0,0 +1 @@
+token_saver

llm_token_saver-0.1.0/pyproject.toml

@@ -0,0 +1,17 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "llm-token-saver"
+version = "0.1.0"
+description = "A seamless LLM API middleware to maximize context cache hits and reduce token costs."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+dependencies = [
+    "openai>=1.0.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/quan0618/token-saver"

llm_token_saver-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

llm_token_saver-0.1.0/token_saver/__init__.py

@@ -0,0 +1,4 @@
+from .client import TokenSaver
+
+__version__ = "0.1.0"
+__all__ = ["TokenSaver"]

llm_token_saver-0.1.0/token_saver/client.py

@@ -0,0 +1,51 @@
+from typing import Any
+from .optimizers import PrefixStabilizer, ContextPruner, OutputConstraint
+
+class TokenSaver:
+    """包装官方的 OpenAI 客户端，实现无缝 Token 优化。"""
+    
+    def __init__(self, client: Any, max_history: int = 10):
+        self._client = client
+        
+        # 装配优化器链
+        self.optimizers = [
+            PrefixStabilizer(),
+            ContextPruner(max_messages=max_history),
+            OutputConstraint()
+        ]
+        
+        # 注意这里：直接用 _ChatProxy 类名，不要加 self.
+        self.chat = _ChatProxy(self._client.chat, self.optimizers)
+        
+    def __getattr__(self, name):
+        # 将其他未修改的属性（如 models）原样返回给官方客户端
+        return getattr(self._client, name)
+
+class _ChatProxy:
+    def __init__(self, original_chat, optimizers):
+        self._original_chat = original_chat
+        self._optimizers = optimizers
+        
+        # 注意这里：直接用 _CompletionsProxy 类名
+        self.completions = _CompletionsProxy(original_chat.completions, optimizers)
+
+    def __getattr__(self, name):
+        return getattr(self._original_chat, name)
+
+class _CompletionsProxy:
+    def __init__(self, original_completions, optimizers):
+        self._original_completions = original_completions
+        self._optimizers = optimizers
+
+    def create(self, *args, **kwargs):
+        # 1. 拦截参数中的 messages
+        if "messages" in kwargs:
+            messages = kwargs["messages"]
+            # 2. 依次通过优化器链
+            for opt in self._optimizers:
+                messages = opt.optimize(messages)
+            # 3. 替换为优化后的 messages
+            kwargs["messages"] = messages
+            
+        # 4. 放行请求给官方客户端
+        return self._original_completions.create(*args, **kwargs)

llm_token_saver-0.1.0/token_saver/optimizers/__init__.py

@@ -0,0 +1,5 @@
+from .prefix import PrefixStabilizer
+from .context import ContextPruner
+from .output import OutputConstraint
+
+__all__ = ["PrefixStabilizer", "ContextPruner", "OutputConstraint"]

llm_token_saver-0.1.0/token_saver/optimizers/base.py

@@ -0,0 +1,10 @@
+from abc import ABC, abstractmethod
+from typing import List, Dict
+
+class BaseOptimizer(ABC):
+    """所有优化器的基础抽象类"""
+    
+    @abstractmethod
+    def optimize(self, messages: List[Dict[str, str]]) -> List[Dict[str, str]]:
+        """接收并返回优化后的 messages"""
+        pass

llm_token_saver-0.1.0/token_saver/optimizers/context.py

@@ -0,0 +1,20 @@
+from typing import List, Dict
+from .base import BaseOptimizer
+
+class ContextPruner(BaseOptimizer):
+    def __init__(self, max_messages: int = 10):
+        self.max_messages = max_messages
+
+    def optimize(self, messages: List[Dict[str, str]]) -> List[Dict[str, str]]:
+        if len(messages) <= self.max_messages:
+            return messages
+        
+        # 必须保留 system 开头的消息，裁剪后面的
+        system_msgs = [m for m in messages if m.get("role") == "system"]
+        other_msgs = [m for m in messages if m.get("role") != "system"]
+        
+        # 只保留最近的 N 条对话
+        keep_count = self.max_messages - len(system_msgs)
+        pruned_others = other_msgs[-keep_count:] if keep_count > 0 else []
+        
+        return system_msgs + pruned_others

llm_token_saver-0.1.0/token_saver/optimizers/output.py

@@ -0,0 +1,11 @@
+from typing import List, Dict
+from .base import BaseOptimizer
+
+class OutputConstraint(BaseOptimizer):
+    def optimize(self, messages: List[Dict[str, str]]) -> List[Dict[str, str]]:
+        # 找到最后一条 user 消息，追加约束
+        for msg in reversed(messages):
+            if msg.get("role") == "user" and isinstance(msg.get("content"), str):
+                msg["content"] += "\n\n(注意：请直接给出答案，不要输出客套话或废话。)"
+                break
+        return messages

llm_token_saver-0.1.0/token_saver/optimizers/prefix.py

@@ -0,0 +1,26 @@
+import re
+from typing import List, Dict
+from .base import BaseOptimizer
+
+class PrefixStabilizer(BaseOptimizer):
+    def __init__(self):
+        # 匹配常见的日期时间格式，如 "当前时间：2024-05-20 12:00:00"
+        self.time_pattern = r'(当前时间[：:]\s*[\d\-\/:年月日时分秒\s]+)'
+
+    def optimize(self, messages: List[Dict[str, str]]) -> List[Dict[str, str]]:
+        if not messages:
+            return messages
+
+        for msg in messages:
+            if msg.get("role") == "system" and isinstance(msg.get("content"), str):
+                content = msg["content"]
+                match = re.search(self.time_pattern, content)
+                
+                if match:
+                    dynamic_part = match.group(1)
+                    # 剥离动态部分
+                    static_content = content.replace(dynamic_part, "").strip()
+                    # 拼接回末尾
+                    msg["content"] = f"{static_content}\n\n[系统动态信息]: {dynamic_part}"
+                    
+        return messages