Autarkia-Textkit 0.1.0__py3-none-any.whl

autarkia_textkit-0.1.0.dist-info/METADATA

@@ -0,0 +1,266 @@
+Metadata-Version: 2.5
+Name: Autarkia-Textkit
+Version: 0.1.0
+Summary: Chinese chat text segmentation and set similarity measures: bubble splitting, cleaning, numbered lists, emoji gluing, Markdown span repair, IoMax/containment/Jaccard/ngram.
+Project-URL: Homepage, https://qishihao.icu/projects/autarkia
+Project-URL: Repository, https://github.com/hakituo/Autarkia-Textkit
+Project-URL: Roadmap, https://github.com/hakituo/Autarkia-tool/blob/main/ROADMAP.md
+Project-URL: Autarkia Runtime, https://github.com/hakituo/Autarkia
+Project-URL: Author, https://qishihao.icu
+Author-email: Shihao Qi <qishihao512@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: autarkia,chat,chinese,containment,jaccard,nlp,similarity,text-segmentation
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Provides-Extra: segmenter
+Requires-Dist: jieba>=0.42; extra == 'segmenter'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/hakituo/Autarkia-Textkit/main/autarkia-dark.svg">
+    <img src="https://raw.githubusercontent.com/hakituo/Autarkia-Textkit/main/autarkia.svg" alt="Autarkia" width="300">
+  </picture>
+</p>
+
+<p align="center">
+  <b>Autarkia-Textkit</b><br>
+  <sub>中文聊天消息切分 + 集合相似度 · Chinese chat bubble segmentation &amp; set similarity</sub>
+</p>
+
+<p align="center">
+  <a href="https://pypi.org/project/Autarkia-Textkit/"><img alt="PyPI" src="https://img.shields.io/pypi/v/autarkia-textkit.svg"></a>
+  <a href="https://pypi.org/project/Autarkia-Textkit/"><img alt="Python" src="https://img.shields.io/pypi/pyversions/autarkia-textkit.svg"></a>
+  <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-blue.svg"></a>
+</p>
+
+<p align="center">
+  <a href="#中文">简体中文</a> · <a href="#english">English</a>
+</p>
+
+---
+
+## 中文
+
+把模型**一段回复**切成**几条像人打出来的消息**，外加几个分母说得清的集合相似度函数。
+
+来自长期在线的 [Autarkia](https://qishihao.icu/projects/autarkia) agent 运行时——那里的切分规则是对着真实 QQ / Android 聊天行为一条条调出来的。整个系列见[组件库路线图](https://github.com/hakituo/Autarkia-tool/blob/main/ROADMAP.md)。
+
+### 为什么：切气泡不是 `text.split("。")`
+
+下面每条规则都对应一次线上翻车：
+
+- **标点要攒够字数才切**，否则「啊。好。」会变成两条机器人气泡。
+- **省略号是情绪，不是句号**：`就是……戳废了` 保持一条，`……哈？！` 不会被撕开。
+- **括号、引号、数学区间是原子**：`[1, 3]` 不会变成 `[1` + `3]`。
+- **编号列表各自成条**，`1. … 2. … 3. …` 不会从条目中间断开。
+- **标点后的 emoji 跟着句子走**：`真的吗？😊` 不拆。
+- **Markdown 结构整体保留**：代码块、表格、标题、`$$` 行都不切。
+- **模型抄进来的时间戳会清掉**：`[今天 20:00] 早啊` → `早啊`。
+- **句尾的句号/逗号去掉，省略号保留**。
+- **承接词合并回去**：以「然后…」开头的碎片不会单独成条。
+
+相似度一起发，是因为运行时里「相似度」被不同分母实现了四遍。这里把分母写进函数名：
+
+| 函数 | 分母 | 用途 |
+| --- | --- | --- |
+| `intersection_over_max` | `max(|A|, |B|)` | 宽容的重叠度 |
+| `containment` | `min(|A|, |B|)` | 「短文本是否被长文本覆盖」 |
+| `jaccard` | `|A ∪ B|` | 经典 Jaccard |
+
+### 安装
+
+```bash
+pip install autarkia-textkit
+```
+
+可选的承接词词性判断：
+
+```bash
+pip install "autarkia-textkit[segmenter]"
+```
+
+没装 `jieba` 时自动退回一套紧凑的连词前缀兜底逻辑。
+
+### 快速开始
+
+```python
+from autarkia_textkit import split_chat_message, split_for_display
+
+split_chat_message("我觉得你今天真的特别辛苦，早点休息吧。明天还要早起呢。", min_split_len=10)
+# ["我觉得你今天真的特别辛苦，早点休息吧", "明天还要早起呢"]
+
+split_for_display("[今天 20:00] 好呀好呀。")
+# ["好呀好呀"]
+
+split_for_display("看这个：\n```python\nprint(1)\n```")
+# ["看这个：", "```python\nprint(1)\n```"]
+```
+
+确定性切分（把随机源交出去）：
+
+```python
+import random
+split_chat_message(text, comma_split_prob=0.2, rng=random.Random(7))
+```
+
+相似度：
+
+```python
+from autarkia_textkit import char_ngrams, containment, intersection_over_max, jaccard
+
+a = char_ngrams("外卖点了吗")
+b = char_ngrams("外卖点好没")
+containment(a, b)           # 短文本覆盖度
+intersection_over_max(a, b) # max 分母的重叠
+jaccard(a, b)               # 并集分母
+```
+
+### 模块分工
+
+| 组件 | 模块 | 做什么 |
+| --- | --- | --- |
+| `split_chat_message` | `autarkia_textkit.segmenter` | 全部边界规则下的气泡切分 |
+| `clean_chat_text` | `autarkia_textkit.clean` | 时间戳与句尾标点清理 |
+| `split_for_display` | `autarkia_textkit.clean` | 清理 → 归并 Markdown 块 → 切分 |
+| `normalize_numbered_list` | `autarkia_textkit.numbered` | 行内 `1. 2. 3.` 摊成多行 |
+| `merge_continuation_chunks` | `autarkia_textkit.merge` | 合并不自然的断点 |
+| `repair_broken_inline_spans` | `autarkia_textkit.force` | 修复被切开的 `**粗体**` |
+| `intersection_over_max` 等 | `autarkia_textkit.similarity` | 集合度量 |
+
+### 设计说明
+
+- **不绑渠道**：`preprocess` 与 `pass_through_markers` 都是参数，QQ 专有的清洗/透传标记留在适配层。
+- **只有一个随机缝**：`rng=` 取代模块级 `random`，行为可测。
+- **必需依赖为零**：`jieba` 只是可选 extra，且仅用于词性判断的承接词检测。
+
+### 系列与链接
+
+- 项目页：<https://qishihao.icu/projects/autarkia>
+- 运行时仓库：<https://github.com/hakituo/Autarkia>
+- 本包：<https://github.com/hakituo/Autarkia-Textkit>
+- 组件库路线图：<https://github.com/hakituo/Autarkia-tool/blob/main/ROADMAP.md>
+- 作者：Shihao Qi — <https://qishihao.icu>
+
+### 许可
+
+MIT
+
+---
+
+## English
+
+Turn **one model reply** into **several messages that read like a human typed them** — plus set-similarity functions whose names state their actual denominator.
+
+Extracted from the [Autarkia](https://qishihao.icu/projects/autarkia) agent runtime, where these rules were tuned against real QQ/Android chat behavior. See the [component roadmap](https://github.com/hakituo/Autarkia-tool/blob/main/ROADMAP.md).
+
+### Why
+
+Splitting a reply into chat bubbles is not `text.split("。")`. The rules here exist because each naive rule failed in production:
+
+- **Sentence punctuation only splits after enough text.** Otherwise "啊。好。" becomes two robotic bubbles.
+- **Ellipses are emotion, not punctuation.** `就是……戳废了` stays one bubble; `……哈？！` is never torn apart.
+- **Brackets, quotes and math ranges are atomic.** `[1, 3]` does not become `[1` + `3]`.
+- **Numbered lists become separate bubbles.** `1. ... 2. ... 3. ...` never breaks mid-item.
+- **Emoji after punctuation sticks to the sentence.** `真的吗？😊` stays together.
+- **Markdown structure survives.** Fenced code, tables, headings and `$$` lines are kept whole.
+- **Model-copied timestamps are stripped.** `[今天 20:00] 早啊` -> `早啊`.
+- **Trailing periods/commas are removed, ellipses preserved.**
+- **Continuation words are merged back.** "然后..." does not start a lonely bubble.
+
+Set similarity is included because "similarity" was implemented four times with different denominators in the runtime. Here the names are explicit:
+
+| Function | Denominator | Use |
+| --- | --- | --- |
+| `intersection_over_max` | `max(|A|, |B|)` | tolerant overlap |
+| `containment` | `min(|A|, |B|)` | "is the short text covered by the long one" |
+| `jaccard` | `|A ∪ B|` | classic Jaccard |
+
+### Install
+
+```bash
+pip install autarkia-textkit
+```
+
+Optional continuation-word POS detection:
+
+```bash
+pip install "autarkia-textkit[segmenter]"
+```
+
+Without `jieba`, a compact conjunction-prefix fallback is used.
+
+### Quick start
+
+```python
+from autarkia_textkit import split_chat_message, split_for_display
+
+split_chat_message("我觉得你今天真的特别辛苦，早点休息吧。明天还要早起呢。", min_split_len=10)
+# ["我觉得你今天真的特别辛苦，早点休息吧", "明天还要早起呢"]
+
+split_for_display("[今天 20:00] 好呀好呀。")
+# ["好呀好呀"]
+
+split_for_display("看这个：\n```python\nprint(1)\n```")
+# ["看这个：", "```python\nprint(1)\n```"]
+```
+
+Deterministic splitting:
+
+```python
+import random
+split_chat_message(text, comma_split_prob=0.2, rng=random.Random(7))
+```
+
+Similarity:
+
+```python
+from autarkia_textkit import char_ngrams, containment, intersection_over_max, jaccard
+
+a = char_ngrams("外卖点了吗")
+b = char_ngrams("外卖点好没")
+containment(a, b)           # short text coverage
+intersection_over_max(a, b) # max-denominator overlap
+jaccard(a, b)               # union denominator
+```
+
+### Concepts
+
+| Piece | Module | What it does |
+| --- | --- | --- |
+| `split_chat_message` | `autarkia_textkit.segmenter` | bubble splitting with all boundary rules |
+| `clean_chat_text` | `autarkia_textkit.clean` | timestamp + trailing punctuation cleanup |
+| `split_for_display` | `autarkia_textkit.clean` | clean -> group Markdown blocks -> split |
+| `normalize_numbered_list` | `autarkia_textkit.numbered` | inline `1. 2. 3.` -> lines |
+| `merge_continuation_chunks` | `autarkia_textkit.merge` | rejoin unnatural breaks |
+| `repair_broken_inline_spans` | `autarkia_textkit.force` | rejoin split `**bold**` |
+| `intersection_over_max` etc. | `autarkia_textkit.similarity` | set measures |
+
+### Design notes
+
+- **No channel coupling.** `preprocess` and `pass_through_markers` are parameters, so QQ-specific stripping or passthrough codes stay in the adapter.
+- **One RNG seam.** `rng=` replaces the module-global `random`, making behavior testable.
+- **Zero required dependencies.** `jieba` is an optional extra used only for POS-based continuation detection.
+
+### The bigger picture
+
+- Project page: <https://qishihao.icu/projects/autarkia>
+- Runtime repository: <https://github.com/hakituo/Autarkia>
+- This package: <https://github.com/hakituo/Autarkia-Textkit>
+- Roadmap: <https://github.com/hakituo/Autarkia-tool/blob/main/ROADMAP.md>
+- Author: Shihao Qi — <https://qishihao.icu>
+
+### License
+
+MIT

autarkia_textkit-0.1.0.dist-info/RECORD

@@ -0,0 +1,14 @@
+autarkia_textkit/__init__.py,sha256=JH6qcYhxxavxuhrTv5mtsaHmRcqZKnKFErPmtF_Ul-E,2412
+autarkia_textkit/clean.py,sha256=U-3OyLjeNGDACCoaOJMX_5TKWGZF61JTiQJde_kjBwg,4544
+autarkia_textkit/emoji.py,sha256=9Egf752IEPNk4FfzfiUShTTjErlCFv36drl5q4O8eJY,1979
+autarkia_textkit/force.py,sha256=MXljmcSp_fu3lZ8CqCkMqQ7U_jhnEn5nFOe-cfQq3nE,4203
+autarkia_textkit/merge.py,sha256=1LFQseHgk920en8Z2s0pLga7g5VA4Gm7ZJIKih2T2e0,6793
+autarkia_textkit/numbered.py,sha256=ZVwkE0tRRKPh7Vx46pFN0YMhdqE2S8qju-tuEOFthwQ,3091
+autarkia_textkit/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+autarkia_textkit/rules.py,sha256=1yXVSxF1mto6QYLTupLdYaXv8Je-wHedRHVMEn2sJVw,2572
+autarkia_textkit/segmenter.py,sha256=S5Gnk59lrVOz9uIijVukUydRJBEhIxGvwdN0HszEtQw,17352
+autarkia_textkit/similarity.py,sha256=1CFIz3TrsD8rn-SICb2wzh8aqKppRP09QCAevoHgj9I,2388
+autarkia_textkit-0.1.0.dist-info/METADATA,sha256=anD5ceqWNtDrBspx_BrN1g8bhmjaRcu81urM6mpL_HQ,10962
+autarkia_textkit-0.1.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+autarkia_textkit-0.1.0.dist-info/licenses/LICENSE,sha256=KjCC0TPNgxUUjuiaiDm4bNjiDFPtghA1apRPKt_8cU0,1066
+autarkia_textkit-0.1.0.dist-info/RECORD,,

autarkia_textkit-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.4
+Root-Is-Purelib: true
+Tag: py3-none-any

autarkia_textkit-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shihao Qi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.