scrollbreaker 0.1.0__tar.gz

scrollbreaker-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 scrollcap contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

scrollbreaker-0.1.0/PKG-INFO

@@ -0,0 +1,231 @@
+Metadata-Version: 2.4
+Name: scrollbreaker
+Version: 0.1.0
+Summary: Scroll-aware screen content collector for LLM agents — 把滚动的屏幕切成能读的页
+Author: scrollbreaker contributors
+License: MIT
+Keywords: ocr,screen,scroll,llm,agent,capture,information-collector,content-collector
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: image
+Requires-Dist: pillow>=9; extra == "image"
+Requires-Dist: mss>=9; extra == "image"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# ScrollBreaker
+
+**A scroll-aware screen content collector for LLM agents** — 一个会分辨页面的信息采集器。
+
+> **Your LLM is eating garbage. The screenshots are the problem.**
+>
+> Every scroll-capture tool on the planet splices frames together by line overlap.
+> That works — until it silently fuses an article and a chat log into one record
+> that looks perfectly normal. ScrollBreaker stops that with **four gates**.
+
+Most **information collectors** answer the question *"how do I grab the screen?"*
+ScrollBreaker answers the harder one: *"are these two grabs the same page at all?"*
+
+
+Screenshot → OCR → merge is the standard pipeline for feeding visual content to an
+agent. It has one failure mode that poisons your dataset and never warns you:
+**unrelated pages that happen to share lines.**
+
+```
+Article page : 正常 / 好的 / 收到 / 在看 / +关注 / body paragraph...
+Chat page    : 正常 / 好的 / 收到 / Alice: stop machine 4 / Bob: on it
+Shared lines : 3
+```
+
+Any tool that merges on "overlap ≥ 2" fuses these two. The result is one record
+containing an article with a factory conversation stapled into its middle. It parses
+fine. It embeds fine. It is **wrong**, and nothing downstream will tell you.
+
+ScrollBreaker refuses to merge unless four gates all pass.
+
+## The four gates
+
+| # | Gate | Rule | Default |
+|---|------|------|---------|
+| 1 | Time window | Frames further apart than N seconds are a different reading session | `180s` |
+| 2 | Same type | `article` / `chat` / `list` / `text` must match | — |
+| 3 | Same anchor | Same page header (relaxed for articles — titles scroll away) | — |
+| 4 | Real overlap | ≥ N **identical** lines, because 1 line is noise | `2` |
+
+Fail any one → **new record, no merge. Ever.**
+
+## Install
+
+No PyPI account needed — the wheel is attached to the release:
+
+```bash
+pip install https://github.com/xiangjianhuan007/scrollbreaker/releases/download/v0.1.0/scrollbreaker-0.1.0-py3-none-any.whl
+```
+
+Or from source, if you want to poke at it:
+
+```bash
+git clone https://github.com/xiangjianhuan007/scrollbreaker
+cd scrollbreaker
+pip install -e ".[dev]"
+pytest -q
+```
+
+Verify the install the same way CI does:
+
+```bash
+scrollbreaker --version
+scrollbreaker replay examples/frames.jsonl
+```
+
+Zero runtime dependencies. The `image` extra (`pillow`, `mss`) is only needed if you
+want to feed it real screenshots instead of OCR'd text.
+
+
+There is a fifth state, `mixed`: a frame containing both article and chat/list
+signatures. That frame *is* a page transition in progress, so it can neither start
+an accumulation nor be absorbed into one.
+
+## Quick start
+
+```bash
+pip install -e .
+# for live capture: pip install mss pillow
+```
+
+```python
+from scrollbreaker import Accumulator, Frame, Mode
+
+acc = Accumulator()                      # every threshold lives in Config
+
+for snap in screenshots:                 # one frame every N seconds
+    m = acc.push(Frame(lines=snap.lines, ts=snap.ts))
+
+    if m.mode is Mode.ROLL:
+        pass                             # same page scrolling; m.text is the full text
+    else:
+        handle_page(m.text)              # page changed; previous record is complete
+```
+
+Every decision is auditable — `m.gate_log` tells you which gate failed:
+
+```python
+{"time_window": True, "same_type": False, "same_anchor": False,
+ "anchor_relaxed": True, "overlap_enough": False}
+```
+
+### CLI
+
+```bash
+# offline replay — tune thresholds without capturing anything
+scrollbreaker replay examples/frames.jsonl
+
+# live capture
+scrollbreaker watch --interval 10 \
+  --ocr-cmd tesseract {img} stdout -l eng --psm 6 \
+  --output records.jsonl
+```
+
+Real output from `examples/frames.jsonl`:
+
+```
+frame 1  new   chat     anchor=Alice|Bob
+frame 2  roll  chat     anchor=Alice|Bob   overlap=2       <- same conversation, merged
+frame 3  new   article  same_type=false                    <- switched pages, hard break
+```
+
+That third line is the whole point: **wrong type, no merge — no matter the overlap.**
+
+## Android capture app (optional)
+
+`android/` contains the phone half of the pair: interval screen capture → dHash
+dedup → refuse password/payment screens → POST the frame to an endpoint you control.
+
+It is source, not a shipped app — you build it yourself, and **nothing leaves the
+device until you enter an endpoint**. Requires Android 11+ (API 30) because
+`AccessibilityService.takeScreenshot()` is the only unattended capture API.
+
+```
+android/          769 lines Kotlin, zero dependencies
+  CaptureService.kt   the loop + the four gates + the privacy gate
+  ImageSig.kt         dHash + blank detection
+  Uplink.kt           one POST, no SDK
+  CaptureConfig.kt    everything configurable, nothing hard-coded
+  MainActivity.kt     settings screen (built in code, no layouts)
+```
+
+See [android/README.md](android/README.md) for the permission trade-off, the payload
+shape and a 10-line receiver.
+
+
+## Output shape
+
+```json
+{"ts": 1000.0, "mode": "roll", "page": "chat", "anchor": "Alice|Bob",
+ "scene": "chat|Alice|Bob", "overlap": 2, "text": "..."}
+```
+
+`scene` is the field that matters downstream. Group by `scene` when you store,
+embed, or summarize, and unrelated captures can no longer contaminate each other.
+
+## Design trade-offs
+
+**OCR is a plug-in.** The whole contract is `read(bytes) -> str`. Use tesseract,
+PaddleOCR, a cloud API, a local VLM — ScrollBreaker never looks at pixels, only lines.
+
+**Screen only, no database spelunking.** No file parsing, no process injection,
+no root. You lose some fidelity versus reading the source directly; you gain
+application-agnostic capture that survives app updates.
+
+**Thresholds are empirical, not laws.** `180s / 80 lines / 2 overlap` came from real
+long-form reading and group-chat scrolling. Monitoring a dashboard or a log tail?
+Change `Config`.
+
+**Chat anchors degrade gracefully.** A chat header scrolls out of view, at which point
+the first line is a message, not a header. ScrollBreaker falls back to the set of
+speakers seen in the frame — stable within a conversation, rarely shared across two.
+
+## Test suite
+
+```bash
+pip install -e ".[dev]" && pytest -q
+```
+
+9 tests covering each gate. The one that matters is `test_article_to_chat_never_merged`:
+two frames sharing **6 lines** with different page types must still produce two records.
+It exists because that exact fusion happened and corrupted real data. Run it before
+touching the merge logic.
+
+## Compliance
+
+- Reads **only what is visible on screen**. No application data files, no databases,
+  no process memory.
+- **No network, no telemetry.** Everything stays local; output is a JSONL on your disk.
+- Capturing other people's screens, or redistributing captured content, may violate
+  local law and the target platform's terms. **Use it on your own device, your own
+  account, your own visible content.**
+- This project provides no authentication bypass, no database decryption, and no
+  process injection — and will not accept PRs that add them.
+
+## 中文说明
+
+**ScrollBreaker（破滚者）** —— 把滚动的屏幕切成能读的页。
+
+核心不是「截图 + OCR」，是**判断两屏到底是不是同一页**。绝大多数采集工具按「有
+两行重合」就拼，于是把一篇文章和一段聊天糊成一条记录——数据被污染了还看不出来。
+
+四道闸门（时间窗 / 类型一致 / 页头一致 / 实质重合）全部通过才允许合并，任一条不过
+就果断断开。`mixed`（一屏里同时出现文章和聊天特征）是切页中间态，既不累积也不被吸收。
+
+阈值都在 `Config` 里，改成你的场景就行。
+
+## License
+
+MIT — do whatever you want, just don't blame us when your OCR is worse than ours.

scrollbreaker-0.1.0/README.md

@@ -0,0 +1,209 @@
+# ScrollBreaker
+
+**A scroll-aware screen content collector for LLM agents** — 一个会分辨页面的信息采集器。
+
+> **Your LLM is eating garbage. The screenshots are the problem.**
+>
+> Every scroll-capture tool on the planet splices frames together by line overlap.
+> That works — until it silently fuses an article and a chat log into one record
+> that looks perfectly normal. ScrollBreaker stops that with **four gates**.
+
+Most **information collectors** answer the question *"how do I grab the screen?"*
+ScrollBreaker answers the harder one: *"are these two grabs the same page at all?"*
+
+
+Screenshot → OCR → merge is the standard pipeline for feeding visual content to an
+agent. It has one failure mode that poisons your dataset and never warns you:
+**unrelated pages that happen to share lines.**
+
+```
+Article page : 正常 / 好的 / 收到 / 在看 / +关注 / body paragraph...
+Chat page    : 正常 / 好的 / 收到 / Alice: stop machine 4 / Bob: on it
+Shared lines : 3
+```
+
+Any tool that merges on "overlap ≥ 2" fuses these two. The result is one record
+containing an article with a factory conversation stapled into its middle. It parses
+fine. It embeds fine. It is **wrong**, and nothing downstream will tell you.
+
+ScrollBreaker refuses to merge unless four gates all pass.
+
+## The four gates
+
+| # | Gate | Rule | Default |
+|---|------|------|---------|
+| 1 | Time window | Frames further apart than N seconds are a different reading session | `180s` |
+| 2 | Same type | `article` / `chat` / `list` / `text` must match | — |
+| 3 | Same anchor | Same page header (relaxed for articles — titles scroll away) | — |
+| 4 | Real overlap | ≥ N **identical** lines, because 1 line is noise | `2` |
+
+Fail any one → **new record, no merge. Ever.**
+
+## Install
+
+No PyPI account needed — the wheel is attached to the release:
+
+```bash
+pip install https://github.com/xiangjianhuan007/scrollbreaker/releases/download/v0.1.0/scrollbreaker-0.1.0-py3-none-any.whl
+```
+
+Or from source, if you want to poke at it:
+
+```bash
+git clone https://github.com/xiangjianhuan007/scrollbreaker
+cd scrollbreaker
+pip install -e ".[dev]"
+pytest -q
+```
+
+Verify the install the same way CI does:
+
+```bash
+scrollbreaker --version
+scrollbreaker replay examples/frames.jsonl
+```
+
+Zero runtime dependencies. The `image` extra (`pillow`, `mss`) is only needed if you
+want to feed it real screenshots instead of OCR'd text.
+
+
+There is a fifth state, `mixed`: a frame containing both article and chat/list
+signatures. That frame *is* a page transition in progress, so it can neither start
+an accumulation nor be absorbed into one.
+
+## Quick start
+
+```bash
+pip install -e .
+# for live capture: pip install mss pillow
+```
+
+```python
+from scrollbreaker import Accumulator, Frame, Mode
+
+acc = Accumulator()                      # every threshold lives in Config
+
+for snap in screenshots:                 # one frame every N seconds
+    m = acc.push(Frame(lines=snap.lines, ts=snap.ts))
+
+    if m.mode is Mode.ROLL:
+        pass                             # same page scrolling; m.text is the full text
+    else:
+        handle_page(m.text)              # page changed; previous record is complete
+```
+
+Every decision is auditable — `m.gate_log` tells you which gate failed:
+
+```python
+{"time_window": True, "same_type": False, "same_anchor": False,
+ "anchor_relaxed": True, "overlap_enough": False}
+```
+
+### CLI
+
+```bash
+# offline replay — tune thresholds without capturing anything
+scrollbreaker replay examples/frames.jsonl
+
+# live capture
+scrollbreaker watch --interval 10 \
+  --ocr-cmd tesseract {img} stdout -l eng --psm 6 \
+  --output records.jsonl
+```
+
+Real output from `examples/frames.jsonl`:
+
+```
+frame 1  new   chat     anchor=Alice|Bob
+frame 2  roll  chat     anchor=Alice|Bob   overlap=2       <- same conversation, merged
+frame 3  new   article  same_type=false                    <- switched pages, hard break
+```
+
+That third line is the whole point: **wrong type, no merge — no matter the overlap.**
+
+## Android capture app (optional)
+
+`android/` contains the phone half of the pair: interval screen capture → dHash
+dedup → refuse password/payment screens → POST the frame to an endpoint you control.
+
+It is source, not a shipped app — you build it yourself, and **nothing leaves the
+device until you enter an endpoint**. Requires Android 11+ (API 30) because
+`AccessibilityService.takeScreenshot()` is the only unattended capture API.
+
+```
+android/          769 lines Kotlin, zero dependencies
+  CaptureService.kt   the loop + the four gates + the privacy gate
+  ImageSig.kt         dHash + blank detection
+  Uplink.kt           one POST, no SDK
+  CaptureConfig.kt    everything configurable, nothing hard-coded
+  MainActivity.kt     settings screen (built in code, no layouts)
+```
+
+See [android/README.md](android/README.md) for the permission trade-off, the payload
+shape and a 10-line receiver.
+
+
+## Output shape
+
+```json
+{"ts": 1000.0, "mode": "roll", "page": "chat", "anchor": "Alice|Bob",
+ "scene": "chat|Alice|Bob", "overlap": 2, "text": "..."}
+```
+
+`scene` is the field that matters downstream. Group by `scene` when you store,
+embed, or summarize, and unrelated captures can no longer contaminate each other.
+
+## Design trade-offs
+
+**OCR is a plug-in.** The whole contract is `read(bytes) -> str`. Use tesseract,
+PaddleOCR, a cloud API, a local VLM — ScrollBreaker never looks at pixels, only lines.
+
+**Screen only, no database spelunking.** No file parsing, no process injection,
+no root. You lose some fidelity versus reading the source directly; you gain
+application-agnostic capture that survives app updates.
+
+**Thresholds are empirical, not laws.** `180s / 80 lines / 2 overlap` came from real
+long-form reading and group-chat scrolling. Monitoring a dashboard or a log tail?
+Change `Config`.
+
+**Chat anchors degrade gracefully.** A chat header scrolls out of view, at which point
+the first line is a message, not a header. ScrollBreaker falls back to the set of
+speakers seen in the frame — stable within a conversation, rarely shared across two.
+
+## Test suite
+
+```bash
+pip install -e ".[dev]" && pytest -q
+```
+
+9 tests covering each gate. The one that matters is `test_article_to_chat_never_merged`:
+two frames sharing **6 lines** with different page types must still produce two records.
+It exists because that exact fusion happened and corrupted real data. Run it before
+touching the merge logic.
+
+## Compliance
+
+- Reads **only what is visible on screen**. No application data files, no databases,
+  no process memory.
+- **No network, no telemetry.** Everything stays local; output is a JSONL on your disk.
+- Capturing other people's screens, or redistributing captured content, may violate
+  local law and the target platform's terms. **Use it on your own device, your own
+  account, your own visible content.**
+- This project provides no authentication bypass, no database decryption, and no
+  process injection — and will not accept PRs that add them.
+
+## 中文说明
+
+**ScrollBreaker（破滚者）** —— 把滚动的屏幕切成能读的页。
+
+核心不是「截图 + OCR」，是**判断两屏到底是不是同一页**。绝大多数采集工具按「有
+两行重合」就拼，于是把一篇文章和一段聊天糊成一条记录——数据被污染了还看不出来。
+
+四道闸门（时间窗 / 类型一致 / 页头一致 / 实质重合）全部通过才允许合并，任一条不过
+就果断断开。`mixed`（一屏里同时出现文章和聊天特征）是切页中间态，既不累积也不被吸收。
+
+阈值都在 `Config` 里，改成你的场景就行。
+
+## License
+
+MIT — do whatever you want, just don't blame us when your OCR is worse than ours.

scrollbreaker-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "scrollbreaker"
+version = "0.1.0"
+description = "Scroll-aware screen content collector for LLM agents — 把滚动的屏幕切成能读的页"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "scrollbreaker contributors" }]
+keywords = ["ocr", "screen", "scroll", "llm", "agent", "capture", "information-collector", "content-collector"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Text Processing",
+]
+dependencies = []
+
+[project.optional-dependencies]
+image = ["pillow>=9", "mss>=9"]
+dev = ["pytest>=7"]
+
+[project.scripts]
+scrollbreaker = "scrollbreaker.cli:main"
+
+[tool.setuptools.packages.find]
+include = ["scrollbreaker*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

scrollbreaker-0.1.0/scrollbreaker/__init__.py

@@ -0,0 +1,32 @@
+"""scrollbreaker —— 把滚动的屏幕，切成能读的页。
+
+核心问题：手机/电脑上读长内容时，一屏只能看到一小段。连续截屏会有三种噪声：
+  1. 屏幕没动，重复截到同一屏（浪费 token）
+  2. 内容滚动了，前后两屏有一段重叠（直接拼会重复）
+  3. **换页面了**，但上一屏和这一屏恰好有相同的一行（直接拼会把两件事糊成一件）
+
+前两个好解决，第三个才是真正会毁掉数据的问题。scrollbreaker 用四道闸门判定
+「这两屏到底是不是同一页」，只有全过才拼，否则另起一条。
+"""
+
+from .config import Config, DEFAULT_CONFIG
+from .pagetype import page_type, head_anchor, body_lines
+from .accumulate import Accumulator, Frame, Merge, Mode
+from .dedup import dhash, is_duplicate, is_blank
+
+__all__ = [
+    "Config",
+    "DEFAULT_CONFIG",
+    "page_type",
+    "head_anchor",
+    "body_lines",
+    "Accumulator",
+    "Frame",
+    "Merge",
+    "Mode",
+    "dhash",
+    "is_duplicate",
+    "is_blank",
+]
+
+__version__ = "0.1.0"