msgsearch 0.2.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {msgsearch-0.2.0 → msgsearch-0.2.1}/CHANGELOG.md +18 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/PKG-INFO +89 -22
- {msgsearch-0.2.0 → msgsearch-0.2.1}/README.md +88 -21
- {msgsearch-0.2.0 → msgsearch-0.2.1}/pyproject.toml +1 -1
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/__init__.py +1 -1
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/agents/corpus-explorer.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/agents/retrieval-critic.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/commands/bench.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/commands/go.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/commands/recon.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/hooks/verify-retrieval.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.claude/settings.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.github/workflows/ci.yml +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.github/workflows/release.yml +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/.gitignore +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/AGENTS.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/ARCHITECTURE.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/CLAUDE.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/CONTRIBUTING.md +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/LICENSE +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/__init__.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/bench.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/gold.jsonl +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/label.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/results/baseline.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/results/retriever.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/results/retriever_bm25.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/results/retriever_dense.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/results/retriever_norerank.json +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/retriever.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/retriever_bm25.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/retriever_dense.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/eval/retriever_norerank.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/attributed_body.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/chunk.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/cli.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/config.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/contacts.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/doctor.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/embedder.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/explore.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/extract.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/index.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/py.typed +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/search.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/sync.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/src/msgsearch/tagging.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_chunk.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_cli.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_contacts.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_index.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_search.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_sync.py +0 -0
- {msgsearch-0.2.0 → msgsearch-0.2.1}/tests/test_tagging.py +0 -0
|
@@ -3,6 +3,24 @@
|
|
|
3
3
|
Notable changes to msgsearch. Retrieval changes carry the measurement that
|
|
4
4
|
justified them; see `ARCHITECTURE.md` for the full evaluation.
|
|
5
5
|
|
|
6
|
+
## [0.2.1] — 2026-09-06
|
|
7
|
+
|
|
8
|
+
Documentation only. PyPI freezes a project's description at publish time, so
|
|
9
|
+
correcting the README required a release.
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
- The install instructions recommended `pipx install --python <arm64 python>`,
|
|
13
|
+
which does not work on Apple silicon with an Intel Homebrew. Python from
|
|
14
|
+
python.org is a universal binary that runs as whichever architecture its
|
|
15
|
+
parent process is, so an Intel-built pipx launches it as x86_64 whichever
|
|
16
|
+
interpreter you name, and PyTorch has no x86_64 macOS wheels. The working
|
|
17
|
+
recipe creates the venv under `arch -arm64`.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
- A quickstart covering install through first search in one block, and an
|
|
21
|
+
explicit note that Full Disk Access and Contacts are separate permissions
|
|
22
|
+
granted to the application rather than the shell.
|
|
23
|
+
|
|
6
24
|
## [0.2.0] — 2026-09-06
|
|
7
25
|
|
|
8
26
|
### Added
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: msgsearch
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: Search your iMessage history by meaning, entirely on your own machine.
|
|
5
5
|
Project-URL: Homepage, https://github.com/dhruv1707/msgsearch
|
|
6
6
|
Project-URL: Documentation, https://github.com/dhruv1707/msgsearch/blob/main/README.md
|
|
@@ -63,46 +63,113 @@ up to make committing it difficult. Treat that directory the way you would treat
|
|
|
63
63
|
password manager's database. Both models run locally, so nothing is uploaded, but
|
|
64
64
|
what lands on your disk is unencrypted.
|
|
65
65
|
|
|
66
|
-
##
|
|
66
|
+
## Quickstart
|
|
67
67
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
on Apple silicon. Check with:
|
|
68
|
+
```bash
|
|
69
|
+
# 1. install (needs an arm64 Python — see Install if this errors)
|
|
70
|
+
pipx install msgsearch
|
|
72
71
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
72
|
+
# 2. get the embedding model
|
|
73
|
+
# accept the licence at huggingface.co/google/embeddinggemma-300m, then:
|
|
74
|
+
hf auth login
|
|
76
75
|
|
|
77
|
-
|
|
76
|
+
# 3. snapshot your messages [needs Full Disk Access]
|
|
77
|
+
msgsearch sync
|
|
78
78
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
```
|
|
79
|
+
# 4. check the setup, and fix whatever it names
|
|
80
|
+
msgsearch doctor
|
|
82
81
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
82
|
+
# 5. build the index [~30 min per 100k messages, once]
|
|
83
|
+
msgsearch index
|
|
84
|
+
|
|
85
|
+
# 6. search
|
|
86
|
+
msgsearch search "that restaurant we talked about"
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Afterwards, one command keeps it current — only genuinely new text is embedded,
|
|
90
|
+
so it takes seconds:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
msgsearch sync --index
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
**Two permissions, and they are different.** `sync` needs **Full Disk Access** to
|
|
97
|
+
read `~/Library/Messages`. Resolving names needs **Contacts**. macOS grants both
|
|
98
|
+
to the *application*, not the shell, so if you run from an editor's integrated
|
|
99
|
+
terminal the grant has to go to the editor — Terminal.app is the simple option.
|
|
100
|
+
Both live in System Settings → Privacy & Security, and you must restart the app
|
|
101
|
+
afterwards.
|
|
102
|
+
|
|
103
|
+
## Requirements
|
|
104
|
+
|
|
105
|
+
- **macOS on Apple silicon.** PyTorch no longer publishes x86_64 macOS wheels.
|
|
106
|
+
- **Python 3.10 or newer, running as arm64.** See the note below — this is the
|
|
107
|
+
one thing that reliably goes wrong.
|
|
86
108
|
- About 3 GB of disk for dependencies and model weights.
|
|
87
109
|
|
|
88
110
|
## Install
|
|
89
111
|
|
|
90
112
|
```bash
|
|
91
|
-
|
|
92
|
-
|
|
113
|
+
pipx install msgsearch
|
|
114
|
+
msgsearch doctor
|
|
93
115
|
```
|
|
94
116
|
|
|
95
|
-
|
|
96
|
-
|
|
117
|
+
This pulls PyTorch, around 1 GB, so the first install is slow. Everything after
|
|
118
|
+
that is local and fast.
|
|
119
|
+
|
|
120
|
+
<details>
|
|
121
|
+
<summary><b>If that fails with "No matching distribution found for torch"</b></summary>
|
|
97
122
|
|
|
98
|
-
|
|
123
|
+
You have an Intel-built Python, which is common on Apple silicon after migrating
|
|
124
|
+
from an Intel Mac. Check:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
python3 -c "import platform; print(platform.machine())" # must say arm64
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
The confusing part is that `pipx install --python /path/to/arm64/python` often
|
|
131
|
+
**does not fix it**. Python from python.org is a *universal* binary that runs as
|
|
132
|
+
whichever architecture its parent process is, and if pipx itself was installed by
|
|
133
|
+
an Intel Homebrew (`/usr/local/...`), it launches that Python as x86_64. pip then
|
|
134
|
+
looks for x86_64 wheels that PyTorch does not publish.
|
|
135
|
+
|
|
136
|
+
Install into a venv created explicitly under arm64 instead:
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
arch -arm64 /Library/Frameworks/Python.framework/Versions/3.12/bin/python3 \
|
|
140
|
+
-m venv ~/.msgsearch-venv
|
|
141
|
+
~/.msgsearch-venv/bin/pip install msgsearch
|
|
142
|
+
ln -sf ~/.msgsearch-venv/bin/msgsearch /usr/local/bin/msgsearch
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Adjust the interpreter path to any arm64 Python 3.10+. To undo:
|
|
146
|
+
`rm /usr/local/bin/msgsearch && rm -rf ~/.msgsearch-venv`.
|
|
147
|
+
|
|
148
|
+
Architecture cannot be expressed in package metadata, which is why this surfaces
|
|
149
|
+
as a dependency-resolution wall rather than a useful error. `msgsearch doctor`
|
|
150
|
+
reports it in one line.
|
|
151
|
+
</details>
|
|
152
|
+
|
|
153
|
+
To work on msgsearch rather than use it:
|
|
99
154
|
|
|
100
155
|
```bash
|
|
101
156
|
git clone https://github.com/dhruv1707/msgsearch && cd msgsearch
|
|
102
|
-
python3 -m venv .venv
|
|
157
|
+
arch -arm64 python3 -m venv .venv # must be an arm64 interpreter
|
|
103
158
|
./.venv/bin/python -m pip install -e ".[dev]"
|
|
104
159
|
```
|
|
105
160
|
|
|
161
|
+
## Setup
|
|
162
|
+
|
|
163
|
+
Get access to the embedding model. `google/embeddinggemma-300m` is gated, so
|
|
164
|
+
accept the licence at <https://huggingface.co/google/embeddinggemma-300m>, then:
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
hf auth login
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Any sentence-transformers model works instead, for example
|
|
171
|
+
`MSGSEARCH_EMBED_MODEL=BAAI/bge-small-en-v1.5`.
|
|
172
|
+
|
|
106
173
|
Take a snapshot of the Messages database. **Never point this tool at
|
|
107
174
|
`~/Library/Messages`**: that file is live, Messages.app holds locks on it, and it
|
|
108
175
|
is irreplaceable.
|
|
@@ -17,46 +17,113 @@ up to make committing it difficult. Treat that directory the way you would treat
|
|
|
17
17
|
password manager's database. Both models run locally, so nothing is uploaded, but
|
|
18
18
|
what lands on your disk is unencrypted.
|
|
19
19
|
|
|
20
|
-
##
|
|
20
|
+
## Quickstart
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
on Apple silicon. Check with:
|
|
22
|
+
```bash
|
|
23
|
+
# 1. install (needs an arm64 Python — see Install if this errors)
|
|
24
|
+
pipx install msgsearch
|
|
26
25
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
26
|
+
# 2. get the embedding model
|
|
27
|
+
# accept the licence at huggingface.co/google/embeddinggemma-300m, then:
|
|
28
|
+
hf auth login
|
|
30
29
|
|
|
31
|
-
|
|
30
|
+
# 3. snapshot your messages [needs Full Disk Access]
|
|
31
|
+
msgsearch sync
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
```
|
|
33
|
+
# 4. check the setup, and fix whatever it names
|
|
34
|
+
msgsearch doctor
|
|
36
35
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
36
|
+
# 5. build the index [~30 min per 100k messages, once]
|
|
37
|
+
msgsearch index
|
|
38
|
+
|
|
39
|
+
# 6. search
|
|
40
|
+
msgsearch search "that restaurant we talked about"
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Afterwards, one command keeps it current — only genuinely new text is embedded,
|
|
44
|
+
so it takes seconds:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
msgsearch sync --index
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
**Two permissions, and they are different.** `sync` needs **Full Disk Access** to
|
|
51
|
+
read `~/Library/Messages`. Resolving names needs **Contacts**. macOS grants both
|
|
52
|
+
to the *application*, not the shell, so if you run from an editor's integrated
|
|
53
|
+
terminal the grant has to go to the editor — Terminal.app is the simple option.
|
|
54
|
+
Both live in System Settings → Privacy & Security, and you must restart the app
|
|
55
|
+
afterwards.
|
|
56
|
+
|
|
57
|
+
## Requirements
|
|
58
|
+
|
|
59
|
+
- **macOS on Apple silicon.** PyTorch no longer publishes x86_64 macOS wheels.
|
|
60
|
+
- **Python 3.10 or newer, running as arm64.** See the note below — this is the
|
|
61
|
+
one thing that reliably goes wrong.
|
|
40
62
|
- About 3 GB of disk for dependencies and model weights.
|
|
41
63
|
|
|
42
64
|
## Install
|
|
43
65
|
|
|
44
66
|
```bash
|
|
45
|
-
|
|
46
|
-
|
|
67
|
+
pipx install msgsearch
|
|
68
|
+
msgsearch doctor
|
|
47
69
|
```
|
|
48
70
|
|
|
49
|
-
|
|
50
|
-
|
|
71
|
+
This pulls PyTorch, around 1 GB, so the first install is slow. Everything after
|
|
72
|
+
that is local and fast.
|
|
73
|
+
|
|
74
|
+
<details>
|
|
75
|
+
<summary><b>If that fails with "No matching distribution found for torch"</b></summary>
|
|
51
76
|
|
|
52
|
-
|
|
77
|
+
You have an Intel-built Python, which is common on Apple silicon after migrating
|
|
78
|
+
from an Intel Mac. Check:
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
python3 -c "import platform; print(platform.machine())" # must say arm64
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The confusing part is that `pipx install --python /path/to/arm64/python` often
|
|
85
|
+
**does not fix it**. Python from python.org is a *universal* binary that runs as
|
|
86
|
+
whichever architecture its parent process is, and if pipx itself was installed by
|
|
87
|
+
an Intel Homebrew (`/usr/local/...`), it launches that Python as x86_64. pip then
|
|
88
|
+
looks for x86_64 wheels that PyTorch does not publish.
|
|
89
|
+
|
|
90
|
+
Install into a venv created explicitly under arm64 instead:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
arch -arm64 /Library/Frameworks/Python.framework/Versions/3.12/bin/python3 \
|
|
94
|
+
-m venv ~/.msgsearch-venv
|
|
95
|
+
~/.msgsearch-venv/bin/pip install msgsearch
|
|
96
|
+
ln -sf ~/.msgsearch-venv/bin/msgsearch /usr/local/bin/msgsearch
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Adjust the interpreter path to any arm64 Python 3.10+. To undo:
|
|
100
|
+
`rm /usr/local/bin/msgsearch && rm -rf ~/.msgsearch-venv`.
|
|
101
|
+
|
|
102
|
+
Architecture cannot be expressed in package metadata, which is why this surfaces
|
|
103
|
+
as a dependency-resolution wall rather than a useful error. `msgsearch doctor`
|
|
104
|
+
reports it in one line.
|
|
105
|
+
</details>
|
|
106
|
+
|
|
107
|
+
To work on msgsearch rather than use it:
|
|
53
108
|
|
|
54
109
|
```bash
|
|
55
110
|
git clone https://github.com/dhruv1707/msgsearch && cd msgsearch
|
|
56
|
-
python3 -m venv .venv
|
|
111
|
+
arch -arm64 python3 -m venv .venv # must be an arm64 interpreter
|
|
57
112
|
./.venv/bin/python -m pip install -e ".[dev]"
|
|
58
113
|
```
|
|
59
114
|
|
|
115
|
+
## Setup
|
|
116
|
+
|
|
117
|
+
Get access to the embedding model. `google/embeddinggemma-300m` is gated, so
|
|
118
|
+
accept the licence at <https://huggingface.co/google/embeddinggemma-300m>, then:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
hf auth login
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Any sentence-transformers model works instead, for example
|
|
125
|
+
`MSGSEARCH_EMBED_MODEL=BAAI/bge-small-en-v1.5`.
|
|
126
|
+
|
|
60
127
|
Take a snapshot of the Messages database. **Never point this tool at
|
|
61
128
|
`~/Library/Messages`**: that file is live, Messages.app holds locks on it, and it
|
|
62
129
|
is irreplaceable.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|