lensgrep 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of lensgrep might be problematic. Click here for more details.

@@ -0,0 +1,32 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+ branches: [main]
8
+
9
+ jobs:
10
+ lint-test:
11
+ runs-on: ${{ matrix.os }}
12
+ strategy:
13
+ fail-fast: false
14
+ matrix:
15
+ os: [ubuntu-latest, windows-latest, macos-latest]
16
+ python-version: ["3.10", "3.11", "3.12"]
17
+ steps:
18
+ - uses: actions/checkout@v4
19
+ - uses: actions/setup-python@v5
20
+ with:
21
+ python-version: ${{ matrix.python-version }}
22
+ cache: pip
23
+ - name: Install
24
+ run: |
25
+ python -m pip install --upgrade pip
26
+ pip install -e ".[dev]"
27
+ - name: Lint (ruff)
28
+ run: ruff check .
29
+ - name: Format check (ruff)
30
+ run: ruff format --check .
31
+ - name: Tests
32
+ run: pytest -q
@@ -0,0 +1,31 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ *$py.class
4
+ *.so
5
+ .Python
6
+ build/
7
+ dist/
8
+ *.egg-info/
9
+ .eggs/
10
+ .venv/
11
+ venv/
12
+ .env
13
+ .coverage
14
+ htmlcov/
15
+ .pytest_cache/
16
+ .ruff_cache/
17
+ .mypy_cache/
18
+ .idea/
19
+ .vscode/
20
+ *.swp
21
+ *.swo
22
+ .DS_Store
23
+ Thumbs.db
24
+ # Local data
25
+ *.db
26
+ *.sqlite
27
+ *.sqlite3
28
+ *.faiss
29
+ .lensgrep/
30
+ # Models (download on first run)
31
+ *.onnx
lensgrep-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Krzysiek Ołdak
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,307 @@
1
+ Metadata-Version: 2.4
2
+ Name: lensgrep
3
+ Version: 0.1.0
4
+ Summary: Grep for your photo library — find any person across thousands of photos, locally and privately. Built-in LoRA dataset builder.
5
+ Project-URL: Homepage, https://github.com/Kris-O/lensgrep
6
+ Project-URL: Repository, https://github.com/Kris-O/lensgrep
7
+ Project-URL: Issues, https://github.com/Kris-O/lensgrep/issues
8
+ Project-URL: Changelog, https://github.com/Kris-O/lensgrep/blob/main/CHANGELOG.md
9
+ Author-email: Krzysiek Ołdak <krzysiek.oldak@gmail.com>
10
+ License-Expression: MIT
11
+ License-File: LICENSE
12
+ Keywords: arcface,cli,face-recognition,flux,insightface,lora,photo-library,privacy,stable-diffusion
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Intended Audience :: End Users/Desktop
17
+ Classifier: License :: OSI Approved :: MIT License
18
+ Classifier: Operating System :: OS Independent
19
+ Classifier: Programming Language :: Python :: 3
20
+ Classifier: Programming Language :: Python :: 3.10
21
+ Classifier: Programming Language :: Python :: 3.11
22
+ Classifier: Programming Language :: Python :: 3.12
23
+ Classifier: Topic :: Multimedia :: Graphics
24
+ Classifier: Topic :: Scientific/Engineering :: Image Recognition
25
+ Classifier: Topic :: Utilities
26
+ Requires-Python: >=3.10
27
+ Requires-Dist: faiss-cpu>=1.8.0
28
+ Requires-Dist: imagehash>=4.3.1
29
+ Requires-Dist: insightface>=0.7.3
30
+ Requires-Dist: numpy>=1.26.0
31
+ Requires-Dist: onnxruntime>=1.17.0
32
+ Requires-Dist: opencv-python>=4.9.0
33
+ Requires-Dist: pillow>=10.0.0
34
+ Requires-Dist: platformdirs>=4.2.0
35
+ Requires-Dist: rich>=13.7.0
36
+ Requires-Dist: scikit-learn>=1.4.0
37
+ Requires-Dist: tqdm>=4.66.0
38
+ Requires-Dist: typer>=0.12.0
39
+ Provides-Extra: caption
40
+ Requires-Dist: torch>=2.2.0; extra == 'caption'
41
+ Requires-Dist: transformers>=4.40.0; extra == 'caption'
42
+ Provides-Extra: dev
43
+ Requires-Dist: mypy>=1.10.0; extra == 'dev'
44
+ Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
45
+ Requires-Dist: pytest>=8.0.0; extra == 'dev'
46
+ Requires-Dist: ruff>=0.5.0; extra == 'dev'
47
+ Provides-Extra: gpu
48
+ Requires-Dist: onnxruntime-gpu>=1.17.0; extra == 'gpu'
49
+ Description-Content-Type: text/markdown
50
+
51
+ # lensgrep
52
+
53
+ > **`grep` for your photo library.** Find people. Remove duplicates. Build LoRA datasets. All locally, all private, no cloud.
54
+
55
+ [![PyPI](https://img.shields.io/pypi/v/lensgrep.svg)](https://pypi.org/project/lensgrep/)
56
+ [![Python](https://img.shields.io/pypi/pyversions/lensgrep.svg)](https://pypi.org/project/lensgrep/)
57
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
58
+ [![CI](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml/badge.svg)](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml)
59
+
60
+ ```
61
+ $ lensgrep scan ~/Photos
62
+ scanned 12,847 files faces+= 38,219 clustered 412 groups
63
+
64
+ $ lensgrep name 3 "Mom"
65
+ ✓ 'Mom' ← cluster 3, assigned 1,204 face(s)
66
+
67
+ $ lensgrep find Mom -n 5 --paths
68
+ /home/me/Photos/2023/birthday/IMG_0341.jpg
69
+ /home/me/Photos/2024/cabin/DSC_8821.jpg
70
+ /home/me/Photos/2024/cabin/DSC_8824.jpg
71
+ /home/me/Photos/2025/holidays/IMG_1129.jpg
72
+ /home/me/Photos/2025/holidays/IMG_1131.jpg
73
+
74
+ $ lensgrep lora Mom ./mom_dataset --size 50
75
+ ✓ LoRA dataset: 50 images at 1024×1024 → ./mom_dataset
76
+ Token: mom
77
+ Next: open AI Toolkit / kohya_ss → point dataset path here → train.
78
+
79
+ $ lensgrep dupes --near --threshold 5
80
+ Found 412 group(s), 1,847 extra file(s), could free 14,203.5 MB.
81
+
82
+ Read-only listing. Re-run with --mode delete or --mode symlink to act.
83
+ ```
84
+
85
+ ## What it is
86
+
87
+ **lensgrep** is a single-purpose tool that does one thing well: **find photos of a specific person in a folder of photos**, no matter how big the folder is. SQLite under the hood, [InsightFace](https://github.com/deepinsight/insightface) ArcFace embeddings, FAISS for vector search. Nothing leaves your machine.
88
+
89
+ It's intentionally small. If you want a full self-hosted Google Photos clone, run [PhotoPrism](https://github.com/photoprism/photoprism) or [Immich](https://github.com/immich-app/immich). If you want `grep` for faces, run this.
90
+
91
+ ## Killer features
92
+
93
+ ### Duplicate detection — reclaim disk space
94
+
95
+ Most photo libraries have 10–30% wasted space: phone-sync dupes, edited copies, burst shots, JPEG-vs-RAW pairs, "the same file in three folders because cloud sync went sideways."
96
+
97
+ ```bash
98
+ lensgrep dupes # exact (bit-identical) only — read-only listing
99
+ lensgrep dupes --near --threshold 5 # near-duplicates: resized / re-saved / minor edits
100
+ lensgrep dupes --near --mode delete # actually delete the victims
101
+ lensgrep dupes --near --mode symlink # replace victims with symlinks to the keeper
102
+ lensgrep dupes --keep newest # alternative: oldest | newest | shortest-path | largest (default)
103
+ ```
104
+
105
+ **Two detection modes:**
106
+ - **Exact** — SHA-256 hash of the file, zero false positives, catches bit-identical copies regardless of filename.
107
+ - **Near** — perceptual hash (pHash) with configurable Hamming-distance threshold. Catches resized copies, JPEG re-saves, minor color edits, thumbnails — the stuff that exact match misses.
108
+
109
+ **Safety by default:** without `--mode delete` or `--mode symlink`, the command is read-only. You'll always see what it *would* do before it does anything.
110
+
111
+ ### LoRA dataset builder — flagship
112
+
113
+ Training a face LoRA (Flux / SDXL) usually means manually picking 40–60 diverse photos of one person from your library. lensgrep does this in one command:
114
+
115
+ ```bash
116
+ lensgrep lora "Krzysiek" ./dataset --size 50 --resolution 1024
117
+ ```
118
+
119
+ This:
120
+ 1. Pulls all photos containing Krzysiek from the indexed library
121
+ 2. Scores quality (sharpness, face area, detection confidence)
122
+ 3. Picks 50 **diverse** photos via greedy farthest-point sampling on ArcFace embeddings — no near-duplicates, balanced mix of closeup / portrait / half-body / full-body
123
+ 4. Crops to 1024×1024 with the face in the upper third (rule-of-thirds)
124
+ 5. Writes `.txt` caption files next to each image, ready for [AI Toolkit](https://github.com/ostris/ai-toolkit) or [kohya_ss](https://github.com/bmaltais/kohya_ss)
125
+
126
+ You go from "8 000 raw photos" to "trainable Flux LoRA dataset" in **one command**.
127
+
128
+ ## Install
129
+
130
+ ```bash
131
+ pip install lensgrep
132
+ ```
133
+
134
+ Optional: GPU support (10× faster scan)
135
+
136
+ ```bash
137
+ pip install "lensgrep[gpu]"
138
+ ```
139
+
140
+ Requires Python 3.10+. First run downloads ~280 MB of InsightFace models to `~/.insightface/`.
141
+
142
+ ## Quickstart
143
+
144
+ ```bash
145
+ # 1. scan your photo library (one time, ~1.5h CPU / ~10 min GPU for 10k photos)
146
+ lensgrep scan ~/Photos
147
+
148
+ # 2. browse detected face clusters
149
+ lensgrep people --min-size 10
150
+
151
+ # 3. name the ones you care about
152
+ lensgrep name 3 "Mom"
153
+ lensgrep name 7 "Dad"
154
+ lensgrep name 12 "Krzysiek"
155
+
156
+ # 4. find photos of someone
157
+ lensgrep find Mom -n 100
158
+ lensgrep find Mom --paths | xargs -I{} cp {} ./mom_photos/
159
+
160
+ # 5. export all photos of a person
161
+ lensgrep export Mom ./mom_photos --mode symlink
162
+
163
+ # 6. build a LoRA training dataset
164
+ lensgrep lora Krzysiek ./krzysiek_dataset --size 50
165
+
166
+ # 7. quick stats
167
+ lensgrep stats
168
+ ```
169
+
170
+ ## Commands
171
+
172
+ | Command | What it does |
173
+ |---------|--------------|
174
+ | `scan <PATH>` | Recursively scan folders for faces. Incremental by default (skips unchanged files). |
175
+ | `people` | List detected face clusters, with face counts. |
176
+ | `name <CLUSTER> <NAME>` | Tag a cluster with a person's name. |
177
+ | `find <NAME>` | List photos of a tagged person, ranked by face quality. |
178
+ | `export <NAME> <DIR>` | Copy/symlink/hardlink all matching photos to a folder. |
179
+ | `lora <NAME> <DIR>` | Build a Flux/SDXL LoRA training dataset. |
180
+ | `dupes` | Find (and optionally remove) duplicate photos — exact or perceptual. |
181
+ | `recluster` | Re-run DBSCAN with custom parameters. |
182
+ | `stats` | Database statistics. |
183
+
184
+ Run `lensgrep <command> --help` for full options on any command.
185
+
186
+ ## How it works
187
+
188
+ ```
189
+ ┌────────────────┐
190
+ ~/Photos ───▶ │ scan │ ──▶ images table (SQLite)
191
+ │ - detect │
192
+ │ - embed (512d)│ ──▶ faces table (SQLite, embedding as BLOB)
193
+ │ - score │
194
+ └────────────────┘
195
+
196
+
197
+ ┌────────────────┐
198
+ │ cluster │ ──▶ persons table (named clusters)
199
+ │ (DBSCAN) │
200
+ └────────────────┘
201
+
202
+
203
+ ┌────────────────┐
204
+ │ find / export │
205
+ │ / lora │
206
+ └────────────────┘
207
+ ```
208
+
209
+ - **Detection + embedding:** InsightFace `buffalo_l` (SCRFD detector + ArcFace r50 recognition). State-of-the-art, ONNX, runs on CPU or CUDA.
210
+ - **Storage:** Plain SQLite — no Docker, no daemons, no cloud accounts. The whole index lives in one file at the platform-standard data dir.
211
+ - **Vector search:** FAISS `IndexFlatIP` on L2-normalized embeddings — exact cosine similarity, sub-millisecond on a million faces.
212
+ - **Clustering:** DBSCAN over cosine distance, `eps=0.35`, `min_samples=5`. Tune via `lensgrep recluster --eps 0.4 --min-samples 8`.
213
+ - **LoRA selection:** Greedy farthest-point sampling on embeddings — maximizes diversity. Quality scoring weights sharpness (Laplacian variance), face area, detection confidence.
214
+
215
+ ## Why this exists
216
+
217
+ In 2026, the photo-management landscape has two extremes:
218
+
219
+ - **Bloated cloud apps** that ingest your library and "do AI on it": Google Photos, Apple Photos, Amazon Photos. Hand over your private life.
220
+ - **Self-hosted galleries with face features bolted on**: PhotoPrism, Immich, digiKam. Powerful but heavy — Docker, databases, web UIs, hours of setup, weeks of curating.
221
+
222
+ Neither helps if you just want to answer one question:
223
+
224
+ > "Where are all the photos of [person] in this big folder?"
225
+
226
+ …or:
227
+
228
+ > "Why does this folder weigh 80 GB? How many of these photos are actual duplicates?"
229
+
230
+ …or:
231
+
232
+ > "I'm training a Flux LoRA of my partner. Give me 50 diverse, high-quality photos."
233
+
234
+ That's the niche. **One job. Done well.**
235
+
236
+ ## Comparison
237
+
238
+ | | lensgrep | PhotoPrism | Immich | digiKam | face_recognition |
239
+ |---|---|---|---|---|---|
240
+ | CLI-first | ✅ | ❌ | ❌ | ❌ | ✅ (library) |
241
+ | No Docker / DB setup | ✅ | ❌ | ❌ | ✅ | ✅ |
242
+ | Duplicate detection (exact + perceptual) | ✅ | partial | partial | partial | ❌ |
243
+ | LoRA dataset export | ✅ | ❌ | ❌ | ❌ | ❌ |
244
+ | FAISS vector search | ✅ | ❌ | ✅ | ❌ | ❌ |
245
+ | Modern face model (ArcFace) | ✅ | ✅ | ✅ | ❌ (older) | ❌ (dlib) |
246
+ | Local-only / no cloud | ✅ | ✅ | ✅ | ✅ | ✅ |
247
+ | Install in 10 seconds | ✅ | ❌ | ❌ | ❌ | ✅ |
248
+
249
+ ## Privacy
250
+
251
+ Everything stays on your machine. No telemetry, no analytics, no network calls except the one-time model download from GitHub releases on first run. The SQLite database is plain text — you can `sqlite3 lensgrep.db ".dump"` and read it.
252
+
253
+ If you want to verify: `pip install lensgrep` ships ~600 lines of pure Python source. Audit it in an afternoon.
254
+
255
+ ## Performance
256
+
257
+ Rough numbers on a 2020 laptop (Quadro RTX 5000 mobile, 16 GB VRAM):
258
+
259
+ | Library size | CPU scan | GPU scan (CUDA) |
260
+ |---|---|---|
261
+ | 1 000 photos | ~3 min | ~30 s |
262
+ | 10 000 photos | ~30 min | ~5 min |
263
+ | 100 000 photos | ~5 h | ~50 min |
264
+
265
+ Scan is the only slow part. After indexing, `find` / `export` / `lora` are sub-second on hundreds of thousands of faces.
266
+
267
+ CPU works fine if you're willing to leave it running overnight once. GPU setup needs CUDA Toolkit 12 + cuDNN 9 (`pip install "lensgrep[gpu]"` is necessary but not sufficient — see [docs/gpu.md](docs/gpu.md)).
268
+
269
+ ## Roadmap
270
+
271
+ - [x] CLI MVP (scan, people, name, find, export, lora, dupes, stats)
272
+ - [ ] Per-person nearest-neighbor add: `lensgrep find-similar <photo.jpg>`
273
+ - [ ] EXIF date timeline filter: `lensgrep find Mom --year 2024`
274
+ - [ ] EXIF GPS map output
275
+ - [ ] HEIC support
276
+ - [ ] `lensgrep serve` — read-only web UI (FastAPI + minimal frontend) for browsing
277
+ - [ ] BLIP / Florence-2 auto-captions for richer LoRA training data
278
+
279
+ If the project gains traction, the web UI is next. Open an issue if you want to push it forward.
280
+
281
+ ## Contributing
282
+
283
+ PRs welcome. Project conventions:
284
+
285
+ - Ruff for lint + format (`ruff check . && ruff format .`)
286
+ - Pytest for tests (`pytest`)
287
+ - Conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`)
288
+
289
+ ```bash
290
+ git clone https://github.com/Kris-O/lensgrep
291
+ cd lensgrep
292
+ pip install -e ".[dev]"
293
+ pytest
294
+ ```
295
+
296
+ ## License
297
+
298
+ MIT — see [LICENSE](LICENSE).
299
+
300
+ ## Acknowledgments
301
+
302
+ Standing on the shoulders of:
303
+ - [InsightFace](https://github.com/deepinsight/insightface) — face detection + recognition
304
+ - [FAISS](https://github.com/facebookresearch/faiss) — vector search
305
+ - [Typer](https://github.com/fastapi/typer) + [Rich](https://github.com/Textualize/rich) — joy of CLI dev
306
+
307
+ Inspired by the question: *why isn't this just one command yet?*
@@ -0,0 +1,257 @@
1
+ # lensgrep
2
+
3
+ > **`grep` for your photo library.** Find people. Remove duplicates. Build LoRA datasets. All locally, all private, no cloud.
4
+
5
+ [![PyPI](https://img.shields.io/pypi/v/lensgrep.svg)](https://pypi.org/project/lensgrep/)
6
+ [![Python](https://img.shields.io/pypi/pyversions/lensgrep.svg)](https://pypi.org/project/lensgrep/)
7
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
8
+ [![CI](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml/badge.svg)](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml)
9
+
10
+ ```
11
+ $ lensgrep scan ~/Photos
12
+ scanned 12,847 files faces+= 38,219 clustered 412 groups
13
+
14
+ $ lensgrep name 3 "Mom"
15
+ ✓ 'Mom' ← cluster 3, assigned 1,204 face(s)
16
+
17
+ $ lensgrep find Mom -n 5 --paths
18
+ /home/me/Photos/2023/birthday/IMG_0341.jpg
19
+ /home/me/Photos/2024/cabin/DSC_8821.jpg
20
+ /home/me/Photos/2024/cabin/DSC_8824.jpg
21
+ /home/me/Photos/2025/holidays/IMG_1129.jpg
22
+ /home/me/Photos/2025/holidays/IMG_1131.jpg
23
+
24
+ $ lensgrep lora Mom ./mom_dataset --size 50
25
+ ✓ LoRA dataset: 50 images at 1024×1024 → ./mom_dataset
26
+ Token: mom
27
+ Next: open AI Toolkit / kohya_ss → point dataset path here → train.
28
+
29
+ $ lensgrep dupes --near --threshold 5
30
+ Found 412 group(s), 1,847 extra file(s), could free 14,203.5 MB.
31
+
32
+ Read-only listing. Re-run with --mode delete or --mode symlink to act.
33
+ ```
34
+
35
+ ## What it is
36
+
37
+ **lensgrep** is a single-purpose tool that does one thing well: **find photos of a specific person in a folder of photos**, no matter how big the folder is. SQLite under the hood, [InsightFace](https://github.com/deepinsight/insightface) ArcFace embeddings, FAISS for vector search. Nothing leaves your machine.
38
+
39
+ It's intentionally small. If you want a full self-hosted Google Photos clone, run [PhotoPrism](https://github.com/photoprism/photoprism) or [Immich](https://github.com/immich-app/immich). If you want `grep` for faces, run this.
40
+
41
+ ## Killer features
42
+
43
+ ### Duplicate detection — reclaim disk space
44
+
45
+ Most photo libraries have 10–30% wasted space: phone-sync dupes, edited copies, burst shots, JPEG-vs-RAW pairs, "the same file in three folders because cloud sync went sideways."
46
+
47
+ ```bash
48
+ lensgrep dupes # exact (bit-identical) only — read-only listing
49
+ lensgrep dupes --near --threshold 5 # near-duplicates: resized / re-saved / minor edits
50
+ lensgrep dupes --near --mode delete # actually delete the victims
51
+ lensgrep dupes --near --mode symlink # replace victims with symlinks to the keeper
52
+ lensgrep dupes --keep newest # alternative: oldest | newest | shortest-path | largest (default)
53
+ ```
54
+
55
+ **Two detection modes:**
56
+ - **Exact** — SHA-256 hash of the file, zero false positives, catches bit-identical copies regardless of filename.
57
+ - **Near** — perceptual hash (pHash) with configurable Hamming-distance threshold. Catches resized copies, JPEG re-saves, minor color edits, thumbnails — the stuff that exact match misses.
58
+
59
+ **Safety by default:** without `--mode delete` or `--mode symlink`, the command is read-only. You'll always see what it *would* do before it does anything.
60
+
61
+ ### LoRA dataset builder — flagship
62
+
63
+ Training a face LoRA (Flux / SDXL) usually means manually picking 40–60 diverse photos of one person from your library. lensgrep does this in one command:
64
+
65
+ ```bash
66
+ lensgrep lora "Krzysiek" ./dataset --size 50 --resolution 1024
67
+ ```
68
+
69
+ This:
70
+ 1. Pulls all photos containing Krzysiek from the indexed library
71
+ 2. Scores quality (sharpness, face area, detection confidence)
72
+ 3. Picks 50 **diverse** photos via greedy farthest-point sampling on ArcFace embeddings — no near-duplicates, balanced mix of closeup / portrait / half-body / full-body
73
+ 4. Crops to 1024×1024 with the face in the upper third (rule-of-thirds)
74
+ 5. Writes `.txt` caption files next to each image, ready for [AI Toolkit](https://github.com/ostris/ai-toolkit) or [kohya_ss](https://github.com/bmaltais/kohya_ss)
75
+
76
+ You go from "8 000 raw photos" to "trainable Flux LoRA dataset" in **one command**.
77
+
78
+ ## Install
79
+
80
+ ```bash
81
+ pip install lensgrep
82
+ ```
83
+
84
+ Optional: GPU support (10× faster scan)
85
+
86
+ ```bash
87
+ pip install "lensgrep[gpu]"
88
+ ```
89
+
90
+ Requires Python 3.10+. First run downloads ~280 MB of InsightFace models to `~/.insightface/`.
91
+
92
+ ## Quickstart
93
+
94
+ ```bash
95
+ # 1. scan your photo library (one time, ~1.5h CPU / ~10 min GPU for 10k photos)
96
+ lensgrep scan ~/Photos
97
+
98
+ # 2. browse detected face clusters
99
+ lensgrep people --min-size 10
100
+
101
+ # 3. name the ones you care about
102
+ lensgrep name 3 "Mom"
103
+ lensgrep name 7 "Dad"
104
+ lensgrep name 12 "Krzysiek"
105
+
106
+ # 4. find photos of someone
107
+ lensgrep find Mom -n 100
108
+ lensgrep find Mom --paths | xargs -I{} cp {} ./mom_photos/
109
+
110
+ # 5. export all photos of a person
111
+ lensgrep export Mom ./mom_photos --mode symlink
112
+
113
+ # 6. build a LoRA training dataset
114
+ lensgrep lora Krzysiek ./krzysiek_dataset --size 50
115
+
116
+ # 7. quick stats
117
+ lensgrep stats
118
+ ```
119
+
120
+ ## Commands
121
+
122
+ | Command | What it does |
123
+ |---------|--------------|
124
+ | `scan <PATH>` | Recursively scan folders for faces. Incremental by default (skips unchanged files). |
125
+ | `people` | List detected face clusters, with face counts. |
126
+ | `name <CLUSTER> <NAME>` | Tag a cluster with a person's name. |
127
+ | `find <NAME>` | List photos of a tagged person, ranked by face quality. |
128
+ | `export <NAME> <DIR>` | Copy/symlink/hardlink all matching photos to a folder. |
129
+ | `lora <NAME> <DIR>` | Build a Flux/SDXL LoRA training dataset. |
130
+ | `dupes` | Find (and optionally remove) duplicate photos — exact or perceptual. |
131
+ | `recluster` | Re-run DBSCAN with custom parameters. |
132
+ | `stats` | Database statistics. |
133
+
134
+ Run `lensgrep <command> --help` for full options on any command.
135
+
136
+ ## How it works
137
+
138
+ ```
139
+ ┌────────────────┐
140
+ ~/Photos ───▶ │ scan │ ──▶ images table (SQLite)
141
+ │ - detect │
142
+ │ - embed (512d)│ ──▶ faces table (SQLite, embedding as BLOB)
143
+ │ - score │
144
+ └────────────────┘
145
+
146
+
147
+ ┌────────────────┐
148
+ │ cluster │ ──▶ persons table (named clusters)
149
+ │ (DBSCAN) │
150
+ └────────────────┘
151
+
152
+
153
+ ┌────────────────┐
154
+ │ find / export │
155
+ │ / lora │
156
+ └────────────────┘
157
+ ```
158
+
159
+ - **Detection + embedding:** InsightFace `buffalo_l` (SCRFD detector + ArcFace r50 recognition). State-of-the-art, ONNX, runs on CPU or CUDA.
160
+ - **Storage:** Plain SQLite — no Docker, no daemons, no cloud accounts. The whole index lives in one file at the platform-standard data dir.
161
+ - **Vector search:** FAISS `IndexFlatIP` on L2-normalized embeddings — exact cosine similarity, sub-millisecond on a million faces.
162
+ - **Clustering:** DBSCAN over cosine distance, `eps=0.35`, `min_samples=5`. Tune via `lensgrep recluster --eps 0.4 --min-samples 8`.
163
+ - **LoRA selection:** Greedy farthest-point sampling on embeddings — maximizes diversity. Quality scoring weights sharpness (Laplacian variance), face area, detection confidence.
164
+
165
+ ## Why this exists
166
+
167
+ In 2026, the photo-management landscape has two extremes:
168
+
169
+ - **Bloated cloud apps** that ingest your library and "do AI on it": Google Photos, Apple Photos, Amazon Photos. Hand over your private life.
170
+ - **Self-hosted galleries with face features bolted on**: PhotoPrism, Immich, digiKam. Powerful but heavy — Docker, databases, web UIs, hours of setup, weeks of curating.
171
+
172
+ Neither helps if you just want to answer one question:
173
+
174
+ > "Where are all the photos of [person] in this big folder?"
175
+
176
+ …or:
177
+
178
+ > "Why does this folder weigh 80 GB? How many of these photos are actual duplicates?"
179
+
180
+ …or:
181
+
182
+ > "I'm training a Flux LoRA of my partner. Give me 50 diverse, high-quality photos."
183
+
184
+ That's the niche. **One job. Done well.**
185
+
186
+ ## Comparison
187
+
188
+ | | lensgrep | PhotoPrism | Immich | digiKam | face_recognition |
189
+ |---|---|---|---|---|---|
190
+ | CLI-first | ✅ | ❌ | ❌ | ❌ | ✅ (library) |
191
+ | No Docker / DB setup | ✅ | ❌ | ❌ | ✅ | ✅ |
192
+ | Duplicate detection (exact + perceptual) | ✅ | partial | partial | partial | ❌ |
193
+ | LoRA dataset export | ✅ | ❌ | ❌ | ❌ | ❌ |
194
+ | FAISS vector search | ✅ | ❌ | ✅ | ❌ | ❌ |
195
+ | Modern face model (ArcFace) | ✅ | ✅ | ✅ | ❌ (older) | ❌ (dlib) |
196
+ | Local-only / no cloud | ✅ | ✅ | ✅ | ✅ | ✅ |
197
+ | Install in 10 seconds | ✅ | ❌ | ❌ | ❌ | ✅ |
198
+
199
+ ## Privacy
200
+
201
+ Everything stays on your machine. No telemetry, no analytics, no network calls except the one-time model download from GitHub releases on first run. The SQLite database is plain text — you can `sqlite3 lensgrep.db ".dump"` and read it.
202
+
203
+ If you want to verify: `pip install lensgrep` ships ~600 lines of pure Python source. Audit it in an afternoon.
204
+
205
+ ## Performance
206
+
207
+ Rough numbers on a 2020 laptop (Quadro RTX 5000 mobile, 16 GB VRAM):
208
+
209
+ | Library size | CPU scan | GPU scan (CUDA) |
210
+ |---|---|---|
211
+ | 1 000 photos | ~3 min | ~30 s |
212
+ | 10 000 photos | ~30 min | ~5 min |
213
+ | 100 000 photos | ~5 h | ~50 min |
214
+
215
+ Scan is the only slow part. After indexing, `find` / `export` / `lora` are sub-second on hundreds of thousands of faces.
216
+
217
+ CPU works fine if you're willing to leave it running overnight once. GPU setup needs CUDA Toolkit 12 + cuDNN 9 (`pip install "lensgrep[gpu]"` is necessary but not sufficient — see [docs/gpu.md](docs/gpu.md)).
218
+
219
+ ## Roadmap
220
+
221
+ - [x] CLI MVP (scan, people, name, find, export, lora, dupes, stats)
222
+ - [ ] Per-person nearest-neighbor add: `lensgrep find-similar <photo.jpg>`
223
+ - [ ] EXIF date timeline filter: `lensgrep find Mom --year 2024`
224
+ - [ ] EXIF GPS map output
225
+ - [ ] HEIC support
226
+ - [ ] `lensgrep serve` — read-only web UI (FastAPI + minimal frontend) for browsing
227
+ - [ ] BLIP / Florence-2 auto-captions for richer LoRA training data
228
+
229
+ If the project gains traction, the web UI is next. Open an issue if you want to push it forward.
230
+
231
+ ## Contributing
232
+
233
+ PRs welcome. Project conventions:
234
+
235
+ - Ruff for lint + format (`ruff check . && ruff format .`)
236
+ - Pytest for tests (`pytest`)
237
+ - Conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`)
238
+
239
+ ```bash
240
+ git clone https://github.com/Kris-O/lensgrep
241
+ cd lensgrep
242
+ pip install -e ".[dev]"
243
+ pytest
244
+ ```
245
+
246
+ ## License
247
+
248
+ MIT — see [LICENSE](LICENSE).
249
+
250
+ ## Acknowledgments
251
+
252
+ Standing on the shoulders of:
253
+ - [InsightFace](https://github.com/deepinsight/insightface) — face detection + recognition
254
+ - [FAISS](https://github.com/facebookresearch/faiss) — vector search
255
+ - [Typer](https://github.com/fastapi/typer) + [Rich](https://github.com/Textualize/rich) — joy of CLI dev
256
+
257
+ Inspired by the question: *why isn't this just one command yet?*