wavi-lib 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- wavi_lib-0.2.0/.githooks/pre-commit +17 -0
- wavi_lib-0.2.0/.githooks/pre-push +21 -0
- wavi_lib-0.2.0/.github/workflows/test.yml +24 -0
- wavi_lib-0.2.0/.gitignore +42 -0
- wavi_lib-0.2.0/LICENSE +21 -0
- wavi_lib-0.2.0/Makefile +47 -0
- wavi_lib-0.2.0/PKG-INFO +166 -0
- wavi_lib-0.2.0/README.md +127 -0
- wavi_lib-0.2.0/docs/adr/ADR-001-vision-over-dom.md +29 -0
- wavi_lib-0.2.0/docs/adr/ADR-002-viewport-1280x1920.md +57 -0
- wavi_lib-0.2.0/docs/adr/ADR-003-chrome-daemon.md +26 -0
- wavi_lib-0.2.0/docs/adr/ADR-004-dom-anchor-dedup.md +22 -0
- wavi_lib-0.2.0/docs/adr/ADR-005-arm64-chrome.md +23 -0
- wavi_lib-0.2.0/docs/adr/ADR-006-optimistic-headless-connect.md +24 -0
- wavi_lib-0.2.0/docs/adr/ADR-007-transcribe-after-close.md +23 -0
- wavi_lib-0.2.0/docs/adr/ADR-008-port-registry.md +27 -0
- wavi_lib-0.2.0/docs/audit-checklist.md +81 -0
- wavi_lib-0.2.0/docs/boarding.html +1967 -0
- wavi_lib-0.2.0/docs/capability-matrix.md +69 -0
- wavi_lib-0.2.0/docs/library-integration.md +224 -0
- wavi_lib-0.2.0/docs/plan-mejoras.md +313 -0
- wavi_lib-0.2.0/docs/pulpo-migration-prompt.md +252 -0
- wavi_lib-0.2.0/docs/pulpo-onboarding-prompt.md +128 -0
- wavi_lib-0.2.0/docs/reports/test_results.js +1005 -0
- wavi_lib-0.2.0/pyproject.toml +85 -0
- wavi_lib-0.2.0/scripts/debug_audio.py +120 -0
- wavi_lib-0.2.0/scripts/make_corpus_case.py +86 -0
- wavi_lib-0.2.0/swift/ocr_vision.swift +52 -0
- wavi_lib-0.2.0/tests/__init__.py +0 -0
- wavi_lib-0.2.0/tests/conftest.py +90 -0
- wavi_lib-0.2.0/tests/corpus/README.md +73 -0
- wavi_lib-0.2.0/tests/corpus_utils.py +128 -0
- wavi_lib-0.2.0/tests/test_corpus.py +79 -0
- wavi_lib-0.2.0/tests/test_lazy_session.py +82 -0
- wavi_lib-0.2.0/tests/test_runner.py +1182 -0
- wavi_lib-0.2.0/tests/test_session.py +454 -0
- wavi_lib-0.2.0/tests/test_transcription.py +279 -0
- wavi_lib-0.2.0/tests/test_vision.py +429 -0
- wavi_lib-0.2.0/uv.lock +1137 -0
- wavi_lib-0.2.0/wavi/__init__.py +14 -0
- wavi_lib-0.2.0/wavi/cli.py +1230 -0
- wavi_lib-0.2.0/wavi/element_detector.py +214 -0
- wavi_lib-0.2.0/wavi/queue.py +102 -0
- wavi_lib-0.2.0/wavi/runner.py +1238 -0
- wavi_lib-0.2.0/wavi/server.py +232 -0
- wavi_lib-0.2.0/wavi/session.py +992 -0
- wavi_lib-0.2.0/wavi/transcription.py +47 -0
- wavi_lib-0.2.0/wavi/vision.py +600 -0
- wavi_lib-0.2.0/wavi-client/package.json +27 -0
- wavi_lib-0.2.0/wavi-client/src/index.ts +210 -0
- wavi_lib-0.2.0/wavi-client/tsconfig.json +14 -0
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
# Lint before every commit. Bypass with: git commit --no-verify
|
|
3
|
+
set -e
|
|
4
|
+
|
|
5
|
+
cd "$(git rev-parse --show-toplevel)"
|
|
6
|
+
|
|
7
|
+
if [ -x .venv/bin/ruff ]; then
|
|
8
|
+
RUFF=.venv/bin/ruff
|
|
9
|
+
elif command -v ruff >/dev/null 2>&1; then
|
|
10
|
+
RUFF=ruff
|
|
11
|
+
else
|
|
12
|
+
echo "[pre-commit] ruff not found — skipping lint (run: make install-dev)" >&2
|
|
13
|
+
exit 0
|
|
14
|
+
fi
|
|
15
|
+
|
|
16
|
+
echo "[pre-commit] ruff check..."
|
|
17
|
+
$RUFF check wavi/ tests/ scripts/
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
# Lint + full offline test suite before every push (~1s).
|
|
3
|
+
# WAVI_NO_REPORT=1 keeps conftest from rewriting boarding.html/test_results.js.
|
|
4
|
+
# Bypass with: git push --no-verify
|
|
5
|
+
set -e
|
|
6
|
+
|
|
7
|
+
cd "$(git rev-parse --show-toplevel)"
|
|
8
|
+
|
|
9
|
+
if [ -x .venv/bin/python ]; then
|
|
10
|
+
PY=.venv/bin/python
|
|
11
|
+
else
|
|
12
|
+
PY=python3
|
|
13
|
+
fi
|
|
14
|
+
|
|
15
|
+
if [ -x .venv/bin/ruff ]; then
|
|
16
|
+
echo "[pre-push] ruff check..."
|
|
17
|
+
.venv/bin/ruff check wavi/ tests/ scripts/
|
|
18
|
+
fi
|
|
19
|
+
|
|
20
|
+
echo "[pre-push] pytest..."
|
|
21
|
+
WAVI_NO_REPORT=1 $PY -m pytest -q
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
name: test
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
|
|
8
|
+
jobs:
|
|
9
|
+
test:
|
|
10
|
+
# Unit tests are fully offline (mocked browser, synthetic images) and
|
|
11
|
+
# platform-independent. The vision corpus (real Apple Vision OCR) is
|
|
12
|
+
# gated by WAVI_CORPUS=1 and runs only locally — see tests/corpus/README.md.
|
|
13
|
+
runs-on: ubuntu-latest
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
- uses: actions/setup-python@v5
|
|
17
|
+
with:
|
|
18
|
+
python-version: "3.11"
|
|
19
|
+
- name: Install
|
|
20
|
+
run: pip install -e ".[dev]"
|
|
21
|
+
- name: Lint
|
|
22
|
+
run: ruff check wavi/ tests/ scripts/
|
|
23
|
+
- name: Test
|
|
24
|
+
run: pytest -v
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
__pycache__/
|
|
2
|
+
*.pyc
|
|
3
|
+
*.pyo
|
|
4
|
+
.venv/
|
|
5
|
+
venv/
|
|
6
|
+
dist/
|
|
7
|
+
build/
|
|
8
|
+
*.egg-info/
|
|
9
|
+
.DS_Store
|
|
10
|
+
|
|
11
|
+
# Chrome profiles and generated data (contain WA session — never commit)
|
|
12
|
+
data/
|
|
13
|
+
|
|
14
|
+
# Generated assets
|
|
15
|
+
assets/*.png
|
|
16
|
+
assets/*.json
|
|
17
|
+
assets/*.ogg
|
|
18
|
+
assets/current.png
|
|
19
|
+
output/
|
|
20
|
+
|
|
21
|
+
# Secrets
|
|
22
|
+
.env
|
|
23
|
+
|
|
24
|
+
# Local agent config
|
|
25
|
+
.agent.json
|
|
26
|
+
|
|
27
|
+
# Playwright CLI logs
|
|
28
|
+
.playwright-cli/
|
|
29
|
+
docs/.playwright-cli/
|
|
30
|
+
|
|
31
|
+
# Session files (local state)
|
|
32
|
+
session/
|
|
33
|
+
|
|
34
|
+
# Compiled OCR binary (make ocr)
|
|
35
|
+
bin/
|
|
36
|
+
|
|
37
|
+
# Vision eval corpus — real WA screenshots, never commit (privacy).
|
|
38
|
+
# The harness (tests/test_corpus.py) and docs stay tracked.
|
|
39
|
+
tests/corpus/cases/
|
|
40
|
+
|
|
41
|
+
# Claude Code local config + worktrees
|
|
42
|
+
.claude/
|
wavi_lib-0.2.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 José Tabuyo
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
wavi_lib-0.2.0/Makefile
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
.PHONY: install install-dev uninstall test report boarding lint ocr corpus-baseline hooks
|
|
2
|
+
|
|
3
|
+
# Enable versioned git hooks: lint on commit, lint+tests on push.
|
|
4
|
+
# Bypass per-invocation with --no-verify.
|
|
5
|
+
hooks:
|
|
6
|
+
chmod +x .githooks/pre-commit .githooks/pre-push
|
|
7
|
+
git config core.hooksPath .githooks
|
|
8
|
+
@echo "OK → pre-commit (ruff) + pre-push (ruff + pytest)"
|
|
9
|
+
|
|
10
|
+
# Global install — isolated venv via pipx, wavi available everywhere
|
|
11
|
+
install:
|
|
12
|
+
@command -v pipx >/dev/null 2>&1 || { echo "pipx not found. Run: brew install pipx && pipx ensurepath"; exit 1; }
|
|
13
|
+
pipx install --editable .
|
|
14
|
+
|
|
15
|
+
# Compile the Apple Vision OCR helper to a native arm64 binary.
|
|
16
|
+
# ~6x less startup overhead than interpreting the script, and Vision runs
|
|
17
|
+
# natively instead of under Rosetta. vision.py picks it up automatically and
|
|
18
|
+
# falls back to `swift swift/ocr_vision.swift` if bin/ is missing or stale.
|
|
19
|
+
ocr:
|
|
20
|
+
mkdir -p bin
|
|
21
|
+
arch -arm64 swiftc -O swift/ocr_vision.swift -o bin/ocr_vision
|
|
22
|
+
@echo "OK → bin/ocr_vision"
|
|
23
|
+
|
|
24
|
+
# Dev install — editable inside the project venv (changes take effect immediately)
|
|
25
|
+
install-dev:
|
|
26
|
+
pip install -e ".[dev]"
|
|
27
|
+
|
|
28
|
+
uninstall:
|
|
29
|
+
pipx uninstall wavi
|
|
30
|
+
|
|
31
|
+
# Run tests (conftest.py writes docs/reports/test_results.js automatically)
|
|
32
|
+
test:
|
|
33
|
+
pytest
|
|
34
|
+
|
|
35
|
+
# Vision eval on the golden corpus (real OCR, macOS only, ~5-10s per case)
|
|
36
|
+
corpus: ocr
|
|
37
|
+
WAVI_CORPUS=1 pytest tests/test_corpus.py -v
|
|
38
|
+
|
|
39
|
+
# Run tests and open boarding page to inspect results
|
|
40
|
+
report:
|
|
41
|
+
pytest && wavi boarding
|
|
42
|
+
|
|
43
|
+
boarding:
|
|
44
|
+
wavi boarding
|
|
45
|
+
|
|
46
|
+
lint:
|
|
47
|
+
ruff check wavi/ tests/ scripts/
|
wavi_lib-0.2.0/PKG-INFO
ADDED
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: wavi-lib
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: WhatsApp automation via vision: screenshot → OCR → click
|
|
5
|
+
Project-URL: Homepage, https://github.com/josetabuyo/wavi
|
|
6
|
+
Project-URL: Source, https://github.com/josetabuyo/wavi
|
|
7
|
+
Project-URL: Issues, https://github.com/josetabuyo/wavi/issues
|
|
8
|
+
Author-email: José Tabuyo <josetabuyo@gmail.com>
|
|
9
|
+
License: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Keywords: automation,cli,ocr,vision,whatsapp
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
20
|
+
Classifier: Topic :: Communications :: Chat
|
|
21
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
22
|
+
Requires-Python: >=3.11
|
|
23
|
+
Requires-Dist: click>=8.1
|
|
24
|
+
Requires-Dist: groq>=0.11
|
|
25
|
+
Requires-Dist: numpy>=1.24
|
|
26
|
+
Requires-Dist: pillow>=10.0
|
|
27
|
+
Requires-Dist: playwright>=1.40
|
|
28
|
+
Requires-Dist: python-dotenv>=1.2.2
|
|
29
|
+
Requires-Dist: scipy>=1.11
|
|
30
|
+
Provides-Extra: dev
|
|
31
|
+
Requires-Dist: httpx>=0.27; extra == 'dev'
|
|
32
|
+
Requires-Dist: pytest-asyncio>=1.4.0; extra == 'dev'
|
|
33
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
34
|
+
Requires-Dist: ruff>=0.8; extra == 'dev'
|
|
35
|
+
Provides-Extra: server
|
|
36
|
+
Requires-Dist: fastapi>=0.111; extra == 'server'
|
|
37
|
+
Requires-Dist: uvicorn[standard]>=0.29; extra == 'server'
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
# wavi — WhatsApp Web Automation via Vision
|
|
41
|
+
|
|
42
|
+
CLI tool for WhatsApp Web automation. Extracts message history using a vision pipeline (screenshot → OCR → bubbles), and handles navigation and sidebar state via DOM scraping.
|
|
43
|
+
|
|
44
|
+
## Commands
|
|
45
|
+
|
|
46
|
+
| Command | What it does | Approach |
|
|
47
|
+
|---|---|---|
|
|
48
|
+
| `wavi connect [session]` | Start Chrome daemon, authenticate via QR | — |
|
|
49
|
+
| `wavi status [session]` | Check if daemon is alive and authenticated | DOM |
|
|
50
|
+
| `wavi get <contact>` | Extract full message history from a chat (`--grow` to page through in chunks) | **Vision** |
|
|
51
|
+
| `wavi send <contact> <message>` | Send a message | DOM + keyboard |
|
|
52
|
+
| `wavi check-updates [session]` | Detect new inbound messages in sidebar | DOM |
|
|
53
|
+
| `wavi list-contacts [session]` | List all contacts in the "New chat" panel | DOM |
|
|
54
|
+
| `wavi queue [session]` | Show operation queue status | — |
|
|
55
|
+
| `wavi stop [session]` | Gracefully shut down the Chrome daemon | — |
|
|
56
|
+
|
|
57
|
+
## Architecture
|
|
58
|
+
|
|
59
|
+
### Vision pipeline (`wavi get`)
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
Screenshot → Crop chat panel → Color-mask detection → Bbox extraction
|
|
63
|
+
↓
|
|
64
|
+
OCR (tiled) → Timestamp extraction → Message classification → Bubble list
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Used for message content because WhatsApp Web obfuscates the message DOM in ways that make direct scraping unreliable.
|
|
68
|
+
|
|
69
|
+
Key files: `element_detector.py`, `vision.py`, `runner.py`
|
|
70
|
+
|
|
71
|
+
### DOM scraping
|
|
72
|
+
|
|
73
|
+
Navigation and sidebar state use JavaScript evaluated directly on the page. Each JS constant in `session.py` has a comment documenting its key selector and the vision-based fallback to implement if the selector breaks after a WA update. When a DOM-scraped feature stops working, check `session.py` → "DOM scraping inventory" block at the top.
|
|
74
|
+
|
|
75
|
+
### Chrome daemon
|
|
76
|
+
|
|
77
|
+
Chrome runs as a long-lived background process (started by `wavi connect`). Playwright connects and disconnects for each operation without ever killing Chrome. Killing Chrome mid-session corrupts WA's IndexedDB and invalidates the session. Shutdown is done only via `wavi stop`, which navigates to `about:blank` first so WA can flush state.
|
|
78
|
+
|
|
79
|
+
## Setup
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
# Install uv if needed
|
|
83
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
84
|
+
|
|
85
|
+
git clone <repo> && cd wavi
|
|
86
|
+
uv sync
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
## Quick start
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
# 1. Start daemon and scan QR
|
|
93
|
+
wavi connect
|
|
94
|
+
|
|
95
|
+
# 2. Extract message history
|
|
96
|
+
wavi get "Contact Name"
|
|
97
|
+
|
|
98
|
+
# 2b. Long chat — page through in blocks of 10 iterations
|
|
99
|
+
wavi get "Contact Name" --grow --max-iter 10 # block 1
|
|
100
|
+
wavi get "Contact Name" --grow --max-iter 10 # block 2 (continues where block 1 stopped)
|
|
101
|
+
# repeat until "history is now complete" or no more messages
|
|
102
|
+
|
|
103
|
+
# 3. Poll for new messages
|
|
104
|
+
wavi check-updates # first run: saves baseline
|
|
105
|
+
wavi check-updates # subsequent: no_updates or updates + contact list
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
## wavi get flags
|
|
109
|
+
|
|
110
|
+
| Flag | Behavior |
|
|
111
|
+
|---|---|
|
|
112
|
+
| `--max-iter N` | Stop after N scroll iterations (default 300). In `--grow` mode, N counts only **new-content** iterations per run. |
|
|
113
|
+
| `--from YYYY-MM-DD` | Stop scrolling when the oldest visible day pill is before this date. Drop bubbles older than the date. |
|
|
114
|
+
| `--newest` | Load existing `history_bubbles.json` and stop the moment a known message is found. Prepends new messages. Goes toward the **present**. |
|
|
115
|
+
| `--grow` | Load existing history, fast-forward past known content, then capture N more iterations toward the **past**. Saves a `grow_checkpoint.json` so each run continues where the last one stopped. Incompatible with `--newest`. |
|
|
116
|
+
| `--assets DIR` | Override the output directory (default `output/<session>/<contact>/`). |
|
|
117
|
+
| `--json-out` | Print the bubble list as JSON to stdout instead of the summary table. |
|
|
118
|
+
|
|
119
|
+
### `--grow` workflow for long chats
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
wavi get "Contact" --grow --max-iter 10 # run 1: captures first 10 new-content iterations
|
|
123
|
+
wavi get "Contact" --grow --max-iter 10 # run 2: fast-forwards to boundary, captures next 10
|
|
124
|
+
# repeat — prints "history is now complete" when scrollTop reaches 0
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
State is stored in `output/<session>/<contact>/grow_checkpoint.json`. Delete it to restart from scratch (also delete `history_bubbles.json`).
|
|
128
|
+
|
|
129
|
+
## check-updates behavior
|
|
130
|
+
|
|
131
|
+
Compares the sidebar snapshot (last message + timestamp per chat) against the previous saved state. Reports a contact as updated only when:
|
|
132
|
+
- its `last_message` changed, **and**
|
|
133
|
+
- `direction == "inbound"` (outbound messages and re-reads are ignored)
|
|
134
|
+
|
|
135
|
+
Direction is inferred from tick icons (`msg-check`, `msg-dbl-check`, etc.) — present → outbound; absent → inbound.
|
|
136
|
+
|
|
137
|
+
**Limitation**: only the last visible message per chat is tracked. If multiple messages arrive between two checks, only the most recent is reported per contact. Use `wavi get <contact>` to retrieve the full history after detection.
|
|
138
|
+
|
|
139
|
+
## Development
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
make ocr # compile the OCR helper to bin/ocr_vision (arm64, ~4x faster pipeline)
|
|
143
|
+
make hooks # git hooks: ruff on commit, ruff+pytest on push (bypass: --no-verify)
|
|
144
|
+
uv run pytest tests/ -v # unit tests (offline, mocked browser)
|
|
145
|
+
make corpus # vision eval on golden screenshots (real OCR, see tests/corpus/README.md)
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
`WAVI_TIMING=1` prints a per-stage timing breakdown of each `analyze()` run.
|
|
149
|
+
Roadmap and audit: `docs/plan-mejoras.md`, `docs/audit-checklist.md`.
|
|
150
|
+
|
|
151
|
+
Key files:
|
|
152
|
+
- `session.py` — Chrome CDP connection + all DOM scraping JS (see inventory block)
|
|
153
|
+
- `runner.py` — Orchestration: vision pipeline, `check_updates`, `list_contacts`
|
|
154
|
+
- `element_detector.py` — Color-mask morphology for bubble detection
|
|
155
|
+
- `vision.py` — OCR, classification, timestamp extraction
|
|
156
|
+
|
|
157
|
+
## Debugging
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
wavi bubbles /path/to/screenshot.png --debug
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Produces `screenshot_debug.png` with annotated boxes:
|
|
164
|
+
- Green: sent messages
|
|
165
|
+
- Blue: received messages
|
|
166
|
+
- Red crosses: audio play button targets
|
wavi_lib-0.2.0/README.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
# wavi — WhatsApp Web Automation via Vision
|
|
2
|
+
|
|
3
|
+
CLI tool for WhatsApp Web automation. Extracts message history using a vision pipeline (screenshot → OCR → bubbles), and handles navigation and sidebar state via DOM scraping.
|
|
4
|
+
|
|
5
|
+
## Commands
|
|
6
|
+
|
|
7
|
+
| Command | What it does | Approach |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| `wavi connect [session]` | Start Chrome daemon, authenticate via QR | — |
|
|
10
|
+
| `wavi status [session]` | Check if daemon is alive and authenticated | DOM |
|
|
11
|
+
| `wavi get <contact>` | Extract full message history from a chat (`--grow` to page through in chunks) | **Vision** |
|
|
12
|
+
| `wavi send <contact> <message>` | Send a message | DOM + keyboard |
|
|
13
|
+
| `wavi check-updates [session]` | Detect new inbound messages in sidebar | DOM |
|
|
14
|
+
| `wavi list-contacts [session]` | List all contacts in the "New chat" panel | DOM |
|
|
15
|
+
| `wavi queue [session]` | Show operation queue status | — |
|
|
16
|
+
| `wavi stop [session]` | Gracefully shut down the Chrome daemon | — |
|
|
17
|
+
|
|
18
|
+
## Architecture
|
|
19
|
+
|
|
20
|
+
### Vision pipeline (`wavi get`)
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
Screenshot → Crop chat panel → Color-mask detection → Bbox extraction
|
|
24
|
+
↓
|
|
25
|
+
OCR (tiled) → Timestamp extraction → Message classification → Bubble list
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Used for message content because WhatsApp Web obfuscates the message DOM in ways that make direct scraping unreliable.
|
|
29
|
+
|
|
30
|
+
Key files: `element_detector.py`, `vision.py`, `runner.py`
|
|
31
|
+
|
|
32
|
+
### DOM scraping
|
|
33
|
+
|
|
34
|
+
Navigation and sidebar state use JavaScript evaluated directly on the page. Each JS constant in `session.py` has a comment documenting its key selector and the vision-based fallback to implement if the selector breaks after a WA update. When a DOM-scraped feature stops working, check `session.py` → "DOM scraping inventory" block at the top.
|
|
35
|
+
|
|
36
|
+
### Chrome daemon
|
|
37
|
+
|
|
38
|
+
Chrome runs as a long-lived background process (started by `wavi connect`). Playwright connects and disconnects for each operation without ever killing Chrome. Killing Chrome mid-session corrupts WA's IndexedDB and invalidates the session. Shutdown is done only via `wavi stop`, which navigates to `about:blank` first so WA can flush state.
|
|
39
|
+
|
|
40
|
+
## Setup
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
# Install uv if needed
|
|
44
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
45
|
+
|
|
46
|
+
git clone <repo> && cd wavi
|
|
47
|
+
uv sync
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## Quick start
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
# 1. Start daemon and scan QR
|
|
54
|
+
wavi connect
|
|
55
|
+
|
|
56
|
+
# 2. Extract message history
|
|
57
|
+
wavi get "Contact Name"
|
|
58
|
+
|
|
59
|
+
# 2b. Long chat — page through in blocks of 10 iterations
|
|
60
|
+
wavi get "Contact Name" --grow --max-iter 10 # block 1
|
|
61
|
+
wavi get "Contact Name" --grow --max-iter 10 # block 2 (continues where block 1 stopped)
|
|
62
|
+
# repeat until "history is now complete" or no more messages
|
|
63
|
+
|
|
64
|
+
# 3. Poll for new messages
|
|
65
|
+
wavi check-updates # first run: saves baseline
|
|
66
|
+
wavi check-updates # subsequent: no_updates or updates + contact list
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## wavi get flags
|
|
70
|
+
|
|
71
|
+
| Flag | Behavior |
|
|
72
|
+
|---|---|
|
|
73
|
+
| `--max-iter N` | Stop after N scroll iterations (default 300). In `--grow` mode, N counts only **new-content** iterations per run. |
|
|
74
|
+
| `--from YYYY-MM-DD` | Stop scrolling when the oldest visible day pill is before this date. Drop bubbles older than the date. |
|
|
75
|
+
| `--newest` | Load existing `history_bubbles.json` and stop the moment a known message is found. Prepends new messages. Goes toward the **present**. |
|
|
76
|
+
| `--grow` | Load existing history, fast-forward past known content, then capture N more iterations toward the **past**. Saves a `grow_checkpoint.json` so each run continues where the last one stopped. Incompatible with `--newest`. |
|
|
77
|
+
| `--assets DIR` | Override the output directory (default `output/<session>/<contact>/`). |
|
|
78
|
+
| `--json-out` | Print the bubble list as JSON to stdout instead of the summary table. |
|
|
79
|
+
|
|
80
|
+
### `--grow` workflow for long chats
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
wavi get "Contact" --grow --max-iter 10 # run 1: captures first 10 new-content iterations
|
|
84
|
+
wavi get "Contact" --grow --max-iter 10 # run 2: fast-forwards to boundary, captures next 10
|
|
85
|
+
# repeat — prints "history is now complete" when scrollTop reaches 0
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
State is stored in `output/<session>/<contact>/grow_checkpoint.json`. Delete it to restart from scratch (also delete `history_bubbles.json`).
|
|
89
|
+
|
|
90
|
+
## check-updates behavior
|
|
91
|
+
|
|
92
|
+
Compares the sidebar snapshot (last message + timestamp per chat) against the previous saved state. Reports a contact as updated only when:
|
|
93
|
+
- its `last_message` changed, **and**
|
|
94
|
+
- `direction == "inbound"` (outbound messages and re-reads are ignored)
|
|
95
|
+
|
|
96
|
+
Direction is inferred from tick icons (`msg-check`, `msg-dbl-check`, etc.) — present → outbound; absent → inbound.
|
|
97
|
+
|
|
98
|
+
**Limitation**: only the last visible message per chat is tracked. If multiple messages arrive between two checks, only the most recent is reported per contact. Use `wavi get <contact>` to retrieve the full history after detection.
|
|
99
|
+
|
|
100
|
+
## Development
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
make ocr # compile the OCR helper to bin/ocr_vision (arm64, ~4x faster pipeline)
|
|
104
|
+
make hooks # git hooks: ruff on commit, ruff+pytest on push (bypass: --no-verify)
|
|
105
|
+
uv run pytest tests/ -v # unit tests (offline, mocked browser)
|
|
106
|
+
make corpus # vision eval on golden screenshots (real OCR, see tests/corpus/README.md)
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
`WAVI_TIMING=1` prints a per-stage timing breakdown of each `analyze()` run.
|
|
110
|
+
Roadmap and audit: `docs/plan-mejoras.md`, `docs/audit-checklist.md`.
|
|
111
|
+
|
|
112
|
+
Key files:
|
|
113
|
+
- `session.py` — Chrome CDP connection + all DOM scraping JS (see inventory block)
|
|
114
|
+
- `runner.py` — Orchestration: vision pipeline, `check_updates`, `list_contacts`
|
|
115
|
+
- `element_detector.py` — Color-mask morphology for bubble detection
|
|
116
|
+
- `vision.py` — OCR, classification, timestamp extraction
|
|
117
|
+
|
|
118
|
+
## Debugging
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
wavi bubbles /path/to/screenshot.png --debug
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Produces `screenshot_debug.png` with annotated boxes:
|
|
125
|
+
- Green: sent messages
|
|
126
|
+
- Blue: received messages
|
|
127
|
+
- Red crosses: audio play button targets
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# ADR-001: Visión sobre DOM para extracción de contenido
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05 (formalizado como archivo 2026-06-11)
|
|
5
|
+
|
|
6
|
+
## Contexto
|
|
7
|
+
|
|
8
|
+
El DOM de WhatsApp Web está minificado y ofuscado, y cambia con cada deploy. Los
|
|
9
|
+
selectores CSS y APIs JS que funcionan hoy se rompen silenciosamente mañana. El
|
|
10
|
+
rendering visual, en cambio, está anclado a la percepción humana: las burbujas se
|
|
11
|
+
ven como burbujas, los timestamps como timestamps — eso casi no cambia entre
|
|
12
|
+
versiones de WA.
|
|
13
|
+
|
|
14
|
+
## Decisión
|
|
15
|
+
|
|
16
|
+
Extraer el contenido de mensajes vía screenshots + OCR (pipeline de visión), no
|
|
17
|
+
vía selectores DOM. La interacción usa coordenadas físicas y eventos de teclado,
|
|
18
|
+
no `page.locator()`.
|
|
19
|
+
|
|
20
|
+
El DOM se usa solo como **acelerador** donde no hay alternativa razonable hoy:
|
|
21
|
+
coordenadas de elementos dinámicos (compose box), estado de scroll, `data-id`
|
|
22
|
+
para dedup, y captura de blobs de audio vía JS. Cada señal DOM tiene su fallback
|
|
23
|
+
visual documentado en el inventario de `session.py`.
|
|
24
|
+
|
|
25
|
+
## Consecuencias
|
|
26
|
+
|
|
27
|
+
- Más lento que parsear DOM, pero resiliente a cualquier cambio no-visual de WA.
|
|
28
|
+
- Requiere calibración de geometría/colores (ver ADR-002 y plan-mejoras Fase 2).
|
|
29
|
+
- La captura de audio (bytes .ogg) es la única capacidad sin camino visual posible.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# ADR-002: Viewport 1280×1920 con DPR=1 forzado
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05-30
|
|
5
|
+
**Relacionado:** ADR-001 (navigate_to_contact sin locators)
|
|
6
|
+
|
|
7
|
+
## Contexto
|
|
8
|
+
|
|
9
|
+
El pipeline de visión captura screenshots de WhatsApp Web y los analiza para detectar
|
|
10
|
+
burbujas. Cuantos más mensajes quepan en un screenshot, menos iteraciones de scroll
|
|
11
|
+
necesita `full-sync-enhanced` para capturar el historial completo. La resolución del
|
|
12
|
+
screenshot determina directamente la cantidad de mensajes capturados por pantalla.
|
|
13
|
+
|
|
14
|
+
En macOS con pantalla Retina (DPR=2), Chrome headless sin configuración explícita
|
|
15
|
+
produce screenshots de ~876px de alto (limitado por la altura física de la pantalla
|
|
16
|
+
dividida por DPR), en lugar de los 1920px que necesitamos.
|
|
17
|
+
|
|
18
|
+
## Decisión
|
|
19
|
+
|
|
20
|
+
1. **WINDOW_W = 1280, WINDOW_H = 1920** — constantes en `session.py`, fuente de verdad
|
|
21
|
+
para todo el sistema.
|
|
22
|
+
|
|
23
|
+
2. **`--force-device-scale-factor=1`** en los args de Chrome (tanto `wavi connect`
|
|
24
|
+
como el fallback en `WASession.connect()`). Fuerza DPR=1 para que
|
|
25
|
+
`--window-size=1280,1920` mapee directamente a 1280×1920 píxeles CSS. Sin este
|
|
26
|
+
flag, en Mac Retina el viewport efectivo es ~640×960 CSS (DPR=2 divide el window-size).
|
|
27
|
+
|
|
28
|
+
3. **`set_viewport_size(WINDOW_W, WINDOW_H)` solo antes de la primera carga de WA**
|
|
29
|
+
(cuando la página está en `about:blank`). NO se llama en reconexiones porque
|
|
30
|
+
provoca pantalla blanca en WA Web ya cargado (confirmado, commit a092e41).
|
|
31
|
+
Con `--force-device-scale-factor=1`, el viewport de Chrome es el correcto
|
|
32
|
+
sin necesitar emulación de Playwright en reconexiones.
|
|
33
|
+
|
|
34
|
+
4. **WINDOW_W = 1280 es la base calibrada de la fórmula del sidebar** en `vision.py`:
|
|
35
|
+
`sidebar_x = screenshot_w * (SIDEBAR_PX / WINDOW_W)`. Esta fórmula escala
|
|
36
|
+
correctamente con DPR (screenshot_w = WINDOW_W * DPR). No cambiar WINDOW_W
|
|
37
|
+
sin recalibrar SIDEBAR_PX.
|
|
38
|
+
|
|
39
|
+
## Consecuencias
|
|
40
|
+
|
|
41
|
+
- Screenshots de 1280×1920 px consistentes entre ejecuciones y reconexiones.
|
|
42
|
+
- ~10–12 burbujas visibles por screenshot (vs ~4 con viewport pequeño).
|
|
43
|
+
- El sidebar cropping formula en vision.py sigue siendo válida para DPR=1 y DPR=2.
|
|
44
|
+
- `--force-device-scale-factor=1` debe estar en TODOS los puntos de lanzamiento
|
|
45
|
+
de Chrome (CLI `connect` y fallback de `WASession.connect()`). Un punto que lo
|
|
46
|
+
omita produce imágenes "enanas" sin error visible — regresión silenciosa.
|
|
47
|
+
|
|
48
|
+
## Tests de regresión
|
|
49
|
+
|
|
50
|
+
`tests/test_session.py::TestViewportRegression` cubre:
|
|
51
|
+
- WINDOW_W == 1280, WINDOW_H == 1920
|
|
52
|
+
- `--force-device-scale-factor=1` presente en args de launch (CLI y fallback)
|
|
53
|
+
- `--window-size=1280,1920` presente en args de launch
|
|
54
|
+
- Screenshot dimensions verificados contra WINDOW_W × WINDOW_H
|
|
55
|
+
|
|
56
|
+
Si alguno de estos tests falla, la imagen de debug tendrá menos mensajes de lo
|
|
57
|
+
esperado y la captura de historial necesitará más iteraciones de scroll.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# ADR-003: Chrome como daemon de larga vida
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05 (formalizado como archivo 2026-06-11)
|
|
5
|
+
|
|
6
|
+
## Contexto
|
|
7
|
+
|
|
8
|
+
WhatsApp guarda su estado de autenticación en IndexedDB dentro del user-data-dir
|
|
9
|
+
de Chrome. Matar Chrome en medio de una sesión corrompe IndexedDB y fuerza
|
|
10
|
+
re-autenticación por QR.
|
|
11
|
+
|
|
12
|
+
## Decisión
|
|
13
|
+
|
|
14
|
+
`wavi connect` inicia Chrome una sola vez como proceso background. Todos los
|
|
15
|
+
demás comandos se conectan/desconectan vía CDP sin matar Chrome nunca. El
|
|
16
|
+
shutdown se hace solo con `wavi stop`, que navega a `about:blank` primero para
|
|
17
|
+
que WA haga flush de IndexedDB, y recién entonces envía SIGTERM (SIGKILL solo
|
|
18
|
+
tras 10s de espera).
|
|
19
|
+
|
|
20
|
+
## Consecuencias
|
|
21
|
+
|
|
22
|
+
- Conectar/desconectar Playwright es barato; reiniciar Chrome es caro y con
|
|
23
|
+
riesgo de pérdida de sesión.
|
|
24
|
+
- Nunca usar `kill -9` directamente sobre un Chrome con sesión WA.
|
|
25
|
+
- Los comandos lazy (`_lazy_session`) auto-inician y auto-detienen Chrome solo
|
|
26
|
+
si no había daemon previo.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# ADR-004: Dedup por ancla DOM con fallback OCR
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05 (formalizado como archivo 2026-06-11)
|
|
5
|
+
|
|
6
|
+
## Contexto
|
|
7
|
+
|
|
8
|
+
Las regiones de scroll se solapan ~15%: un mensaje visible al fondo del frame N
|
|
9
|
+
reaparece arriba del frame N+1. El texto OCR del mismo mensaje puede variar entre
|
|
10
|
+
capturas (ruido, puntuación), así que una clave de contenido sola no alcanza.
|
|
11
|
+
|
|
12
|
+
## Decisión
|
|
13
|
+
|
|
14
|
+
Clave primaria de dedup = `dom_id` (atributo `data-id` de WA, inmutable por
|
|
15
|
+
mensaje). Fallback cuando no se pudo asignar dom_id (elemento fuera de pantalla):
|
|
16
|
+
clave OCR = `(sender, msg_type, text[:80], timestamp)`.
|
|
17
|
+
|
|
18
|
+
## Consecuencias
|
|
19
|
+
|
|
20
|
+
- Cero duplicados aun con OCR imperfecto.
|
|
21
|
+
- Dependencia residual del DOM para corrección — el plan (Fase 3) agrega una
|
|
22
|
+
tercera clave por hash perceptual (dhash) del crop para eliminar esa dependencia.
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# ADR-005: Chrome ARM64 nativo en macOS
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05 (formalizado como archivo 2026-06-11)
|
|
5
|
+
|
|
6
|
+
## Contexto
|
|
7
|
+
|
|
8
|
+
En Apple Silicon, Chrome lanzado sin arch explícito puede correr bajo Rosetta
|
|
9
|
+
(emulación x86). El service worker y el IndexedDB de WA Web se comportan de forma
|
|
10
|
+
inconsistente bajo Rosetta: la sesión aparece autenticada pero la lista de chats
|
|
11
|
+
nunca carga (congelada).
|
|
12
|
+
|
|
13
|
+
## Decisión
|
|
14
|
+
|
|
15
|
+
Lanzar Chrome siempre con `arch -arm64` (en `wavi connect` y en el fallback de
|
|
16
|
+
`WASession.connect()`).
|
|
17
|
+
|
|
18
|
+
## Consecuencias
|
|
19
|
+
|
|
20
|
+
- Sesiones estables; sin cuelgues de chat-list.
|
|
21
|
+
- El mismo principio aplica a otros binarios: el OCR Swift también se compila
|
|
22
|
+
arm64 nativo (`make ocr`) — bajo Rosetta, Apple Vision rinde peor y devuelve
|
|
23
|
+
resultados ligeramente distintos.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# ADR-006: Connect headless optimista
|
|
2
|
+
|
|
3
|
+
**Estado:** Aceptado
|
|
4
|
+
**Fecha:** 2026-05 (formalizado como archivo 2026-06-11)
|
|
5
|
+
|
|
6
|
+
## Contexto
|
|
7
|
+
|
|
8
|
+
Una vez autenticada, la sesión vive en el user-data-dir de Chrome y headless la
|
|
9
|
+
restaura silenciosamente. Abrir una ventana visible sin necesidad es disruptivo
|
|
10
|
+
y más lento.
|
|
11
|
+
|
|
12
|
+
## Decisión
|
|
13
|
+
|
|
14
|
+
`wavi connect` siempre intenta headless primero. Solo si hace falta QR, lo
|
|
15
|
+
captura igualmente en headless y escribe `data/qr.html` (QR en base64 + countdown
|
|
16
|
+
de 60s) para escanear desde el browser del usuario — nunca se abre una ventana
|
|
17
|
+
de Chrome visible. La expiración del QR se detecta por cambio del atributo
|
|
18
|
+
`data-ref` (fallback: timer de 65s).
|
|
19
|
+
|
|
20
|
+
## Consecuencias
|
|
21
|
+
|
|
22
|
+
- Cero ventanas visibles en el flujo completo.
|
|
23
|
+
- Tras el escaneo, la carpeta de sesión se renombra al número de teléfono
|
|
24
|
+
detectado y el alias `.default` apunta a ella.
|