findakin 2.0.0a1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. findakin-2.0.0a1/.gitignore +231 -0
  2. findakin-2.0.0a1/CHANGELOG.md +32 -0
  3. findakin-2.0.0a1/LICENSE +21 -0
  4. findakin-2.0.0a1/PKG-INFO +165 -0
  5. findakin-2.0.0a1/README.md +110 -0
  6. findakin-2.0.0a1/docs/README.md +35 -0
  7. findakin-2.0.0a1/docs/archive/v1-webapp/reference/README.md +60 -0
  8. findakin-2.0.0a1/docs/roadmap/README.md +77 -0
  9. findakin-2.0.0a1/pyproject.toml +80 -0
  10. findakin-2.0.0a1/src/findakin/__init__.py +19 -0
  11. findakin-2.0.0a1/src/findakin/align.py +292 -0
  12. findakin-2.0.0a1/src/findakin/analysis/__init__.py +37 -0
  13. findakin-2.0.0a1/src/findakin/analysis/anomaly.py +397 -0
  14. findakin-2.0.0a1/src/findakin/analysis/clustering.py +631 -0
  15. findakin-2.0.0a1/src/findakin/analysis/projection.py +533 -0
  16. findakin-2.0.0a1/src/findakin/benchmark.py +90 -0
  17. findakin-2.0.0a1/src/findakin/cli.py +139 -0
  18. findakin-2.0.0a1/src/findakin/connectors/__init__.py +41 -0
  19. findakin-2.0.0a1/src/findakin/connectors/base.py +61 -0
  20. findakin-2.0.0a1/src/findakin/connectors/ssap.py +793 -0
  21. findakin-2.0.0a1/src/findakin/connectors/staros.py +88 -0
  22. findakin-2.0.0a1/src/findakin/connectors/vo_service.py +227 -0
  23. findakin-2.0.0a1/src/findakin/embedding.py +325 -0
  24. findakin-2.0.0a1/src/findakin/io/__init__.py +11 -0
  25. findakin-2.0.0a1/src/findakin/io/fits_parser.py +776 -0
  26. findakin-2.0.0a1/src/findakin/io/metadata.py +166 -0
  27. findakin-2.0.0a1/src/findakin/store.py +522 -0
  28. findakin-2.0.0a1/src/findakin/timeseries.py +42 -0
  29. findakin-2.0.0a1/tests/test_align.py +345 -0
  30. findakin-2.0.0a1/tests/test_core_analysis.py +63 -0
  31. findakin-2.0.0a1/tests/test_embedding_parity.py +152 -0
  32. findakin-2.0.0a1/tests/test_store.py +218 -0
  33. findakin-2.0.0a1/website/README.md +36 -0
@@ -0,0 +1,231 @@
1
+ # Byte-compiled / optimized / DLL files
2
+ __pycache__/
3
+ *.py[codz]
4
+ *$py.class
5
+
6
+ # C extensions
7
+ *.so
8
+
9
+ # Distribution / packaging
10
+ .Python
11
+ build/
12
+ develop-eggs/
13
+ dist/
14
+ downloads/
15
+ eggs/
16
+ .eggs/
17
+ #lib/
18
+ lib64/
19
+ parts/
20
+ sdist/
21
+ var/
22
+ wheels/
23
+ share/python-wheels/
24
+ *.egg-info/
25
+ .installed.cfg
26
+ *.egg
27
+ MANIFEST
28
+
29
+ # PyInstaller
30
+ # Usually these files are written by a python script from a template
31
+ # before PyInstaller builds the exe, so as to inject date/other infos into it.
32
+ *.manifest
33
+ *.spec
34
+
35
+ # Installer logs
36
+ pip-log.txt
37
+ pip-delete-this-directory.txt
38
+
39
+ # Unit test / coverage reports
40
+ htmlcov/
41
+ .tox/
42
+ .nox/
43
+ .coverage
44
+ .coverage.*
45
+ .cache
46
+ nosetests.xml
47
+ coverage.xml
48
+ *.cover
49
+ *.py.cover
50
+ .hypothesis/
51
+ .pytest_cache/
52
+ cover/
53
+
54
+ # Translations
55
+ *.mo
56
+ *.pot
57
+
58
+ # Django stuff:
59
+ *.log
60
+ local_settings.py
61
+ db.sqlite3
62
+ db.sqlite3-journal
63
+
64
+ # Flask stuff:
65
+ instance/
66
+ .webassets-cache
67
+
68
+ # Scrapy stuff:
69
+ .scrapy
70
+
71
+ # Sphinx documentation
72
+ docs/_build/
73
+
74
+ # PyBuilder
75
+ .pybuilder/
76
+ target/
77
+
78
+ # Jupyter Notebook
79
+ .ipynb_checkpoints
80
+
81
+ # IPython
82
+ profile_default/
83
+ ipython_config.py
84
+
85
+ # pyenv
86
+ # For a library or package, you might want to ignore these files since the code is
87
+ # intended to run in multiple environments; otherwise, check them in:
88
+ # .python-version
89
+
90
+ # pipenv
91
+ # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92
+ # However, in case of collaboration, if having platform-specific dependencies or dependencies
93
+ # having no cross-platform support, pipenv may install dependencies that don't work, or not
94
+ # install all needed dependencies.
95
+ #Pipfile.lock
96
+
97
+ # UV
98
+ # Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
99
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
100
+ # commonly ignored for libraries.
101
+ #uv.lock
102
+
103
+ # poetry
104
+ # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
106
+ # commonly ignored for libraries.
107
+ # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108
+ #poetry.lock
109
+ #poetry.toml
110
+
111
+ # pdm
112
+ # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
113
+ # pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
114
+ # https://pdm-project.org/en/latest/usage/project/#working-with-version-control
115
+ #pdm.lock
116
+ #pdm.toml
117
+ .pdm-python
118
+ .pdm-build/
119
+
120
+ # pixi
121
+ # Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
122
+ #pixi.lock
123
+ # Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
124
+ # in the .venv directory. It is recommended not to include this directory in version control.
125
+ .pixi
126
+
127
+ # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
128
+ __pypackages__/
129
+
130
+ # Celery stuff
131
+ celerybeat-schedule
132
+ celerybeat.pid
133
+
134
+ # SageMath parsed files
135
+ *.sage.py
136
+
137
+ # Environments
138
+ .env
139
+ .envrc
140
+ .venv
141
+ env/
142
+ venv/
143
+ ENV/
144
+ env.bak/
145
+ venv.bak/
146
+
147
+ # Spyder project settings
148
+ .spyderproject
149
+ .spyproject
150
+
151
+ # Rope project settings
152
+ .ropeproject
153
+
154
+ # mkdocs documentation
155
+ /site
156
+
157
+ # mypy
158
+ .mypy_cache/
159
+ .dmypy.json
160
+ dmypy.json
161
+
162
+ # Pyre type checker
163
+ .pyre/
164
+
165
+ # pytype static type analyzer
166
+ .pytype/
167
+
168
+ # Cython debug symbols
169
+ cython_debug/
170
+
171
+ # PyCharm
172
+ # JetBrains specific template is maintained in a separate JetBrains.gitignore that can
173
+ # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
174
+ # and can be added to the global gitignore or merged into this file. For a more nuclear
175
+ # option (not recommended) you can uncomment the following to ignore the entire idea folder.
176
+ #.idea/
177
+
178
+ # Abstra
179
+ # Abstra is an AI-powered process automation framework.
180
+ # Ignore directories containing user credentials, local state, and settings.
181
+ # Learn more at https://abstra.io/docs
182
+ .abstra/
183
+
184
+ # Visual Studio Code
185
+ # Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore
186
+ # that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
187
+ # and can be added to the global gitignore or merged into this file. However, if you prefer,
188
+ # you could uncomment the following to ignore the entire vscode folder
189
+ # .vscode/
190
+
191
+ # Ruff stuff:
192
+ .ruff_cache/
193
+
194
+ # PyPI configuration file
195
+ .pypirc
196
+
197
+ # Cursor
198
+ # Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
199
+ # exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
200
+ # refer to https://docs.cursor.com/context/ignore-files
201
+ .cursorignore
202
+ .cursorindexingignore
203
+
204
+ # Marimo
205
+ marimo/_static/
206
+ marimo/_lsp/
207
+ __marimo__/
208
+
209
+ # Production secrets
210
+ .env.production
211
+ .env.local
212
+ .env.*.local
213
+ *.pem
214
+ *.key
215
+
216
+ # DO App Platform
217
+ .do/
218
+
219
+ stuff/*
220
+ # OS noise / local data archives
221
+ .DS_Store
222
+ *.zip
223
+
224
+ # Library tooling / local stores
225
+ .venv/
226
+ *.egg-info/
227
+ /dist/
228
+ /build/
229
+ website/dist/
230
+ website/node_modules/
231
+ *.lance/
@@ -0,0 +1,32 @@
1
+ # Changelog
2
+
3
+ All notable changes to Findakin. The format follows [Keep a Changelog](https://keepachangelog.com/).
4
+
5
+ ## [Unreleased]
6
+
7
+ ## [2.0.0a1] — 2026-10-22
8
+
9
+ Findakin becomes a **local Python library** on LanceDB. The hosted web application (v1) is
10
+ archived under the git tag `v1-webapp` and the Digital Ocean deployment is retired.
11
+
12
+ ### Added
13
+ - `findakin.Store`: a LanceDB directory holding `spectra` (metadata, content-hash ids),
14
+ `flux_native` (wavelength / flux arrays), one `emb__<encoder>` table per encoder and the
15
+ `encoders` registry.
16
+ - `Store.ingest_folder()` with content deduplication, header metadata extraction tolerant to
17
+ BeSS / ISIS / specINTI / Demetra dialects, and `series_from` = object | night | folder.
18
+ - `Store.embed()` over every spectro-kernel strategy (`naive`, `dct`, `multiscale_dct`,
19
+ `log_lambda`, `band_power`, `continuum_subtracted`, `lick`, `wavelets`, `pretrained`,
20
+ `remote`); several encoders coexist and are idempotent.
21
+ - `Store.vectors()` (numpy matrix for population analyses), `Store.neighbours()` (k-NN by id,
22
+ by file or by vector, with SQL pre-filters), `Store.embed_file()`.
23
+ - `findakin` CLI: `info`, `ingest`, `embed`, `encoders`, `search`.
24
+ - `findakin.analysis` (clustering, anomalies, projection), `findakin.benchmark`,
25
+ `findakin.align` (common λ grids) and `findakin.connectors` (SSAP, STAROS) carried over
26
+ from v1 as pure library modules.
27
+ - Landing page in `website/` (findakin.lelain.net), deployable with Netlify.
28
+ - Release workflow publishing to PyPI through a Trusted Publisher.
29
+
30
+ ### Removed
31
+ - FastAPI backend, MongoDB models, Clerk authentication, Cloudflare R2 storage, Qdrant /
32
+ ChromaDB services, job queue, React application, Docker / Digital Ocean deployment files.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Matthieu Le Lain
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,165 @@
1
+ Metadata-Version: 2.5
2
+ Name: findakin
3
+ Version: 2.0.0a1
4
+ Summary: Population-level questions on collections of astronomical spectra: similarity search, clustering, anomalies, trajectories and encoder benchmarks on a local LanceDB store, with embeddings from spectro-kernel.
5
+ Project-URL: Homepage, https://findakin.lelain.net
6
+ Project-URL: Repository, https://github.com/matthieulel/findakin
7
+ Project-URL: Documentation, https://github.com/matthieulel/findakin/tree/main/docs
8
+ Project-URL: Changelog, https://github.com/matthieulel/findakin/blob/main/CHANGELOG.md
9
+ Author: matthieulel
10
+ License: MIT
11
+ License-File: LICENSE
12
+ Keywords: anomaly-detection,astronomy,clustering,embeddings,lancedb,mcp,similarity-search,spectra,spectroscopy
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: License :: OSI Approved :: MIT License
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Topic :: Scientific/Engineering :: Astronomy
21
+ Classifier: Topic :: Scientific/Engineering :: Physics
22
+ Requires-Python: >=3.11
23
+ Requires-Dist: astropy>=6.0
24
+ Requires-Dist: httpx>=0.27
25
+ Requires-Dist: lancedb>=0.15
26
+ Requires-Dist: numpy>=1.26
27
+ Requires-Dist: pandas>=2.0
28
+ Requires-Dist: pyarrow>=15
29
+ Requires-Dist: pydantic>=2.6
30
+ Requires-Dist: scikit-learn>=1.4
31
+ Requires-Dist: scipy>=1.12
32
+ Requires-Dist: spectro-kernel>=0.8
33
+ Requires-Dist: typer>=0.12
34
+ Provides-Extra: all
35
+ Requires-Dist: hdbscan>=0.8.38; extra == 'all'
36
+ Requires-Dist: plotly>=5.20; extra == 'all'
37
+ Requires-Dist: pyvo>=1.5; extra == 'all'
38
+ Requires-Dist: umap-learn>=0.5.6; extra == 'all'
39
+ Provides-Extra: analysis
40
+ Requires-Dist: hdbscan>=0.8.38; extra == 'analysis'
41
+ Requires-Dist: umap-learn>=0.5.6; extra == 'analysis'
42
+ Provides-Extra: dev
43
+ Requires-Dist: pytest-cov>=5; extra == 'dev'
44
+ Requires-Dist: pytest>=8.3; extra == 'dev'
45
+ Requires-Dist: ruff>=0.8; extra == 'dev'
46
+ Provides-Extra: encoders
47
+ Requires-Dist: torch>=2.2; extra == 'encoders'
48
+ Provides-Extra: mcp
49
+ Requires-Dist: fastmcp>=2.0; extra == 'mcp'
50
+ Provides-Extra: viz
51
+ Requires-Dist: plotly>=5.20; extra == 'viz'
52
+ Provides-Extra: vo
53
+ Requires-Dist: pyvo>=1.5; extra == 'vo'
54
+ Description-Content-Type: text/markdown
55
+
56
+ # Findakin
57
+
58
+ **Find a kin for every spectrum.**
59
+
60
+ Findakin answers *population* questions about collections of astronomical spectra:
61
+ which look alike, what structure they form, which stand out, which changed in time, and
62
+ which representation (encoder) is the most physical. It is a local Python library built on
63
+ [spectro-kernel](https://github.com/matthieulel/spectro-kernel) (one spectrum → one vector)
64
+ and [LanceDB](https://lancedb.com) (an embedded, versioned, columnar store with a vector
65
+ index). No server, no account: your spectra stay on your disk.
66
+
67
+ > **Status: v2 alpha.** The hosted web application (v1, FastAPI + MongoDB + Qdrant + React)
68
+ > is archived under the git tag `v1-webapp`. The rewrite into a library is documented in
69
+ > [docs/roadmap/findakin-v2-local-first.md](docs/roadmap/findakin-v2-local-first.md).
70
+ > Landing page: [findakin.lelain.net](https://findakin.lelain.net).
71
+
72
+ ```
73
+ spectro-kernel = algorithms on ONE spectrum (embeddings, I/O, lines, RV, reduction…)
74
+ Findakin = questions on MANY spectra (index, neighbours, clusters, anomalies,
75
+ trajectories, encoder benchmarks)
76
+ ```
77
+
78
+ ## Install
79
+
80
+ ```bash
81
+ pip install findakin # core: store, ingestion, kernel recipes, k-NN
82
+ pip install "findakin[analysis]" # + UMAP projection, HDBSCAN clustering
83
+ pip install "findakin[vo]" # + SSAP / Virtual Observatory connectors
84
+ pip install "findakin[encoders]" # + local PyTorch encoders (spectro-kernel embed_pretrained)
85
+ ```
86
+
87
+ Python ≥ 3.11.
88
+
89
+ ## Quick start
90
+
91
+ ```python
92
+ from findakin import Store
93
+
94
+ st = Store.open("~/findakin/store") # a directory; created if missing
95
+
96
+ # 1. ingest — FITS (BeSS / ISIS / specINTI / Demetra dialects), CSV, JSON, TXT.
97
+ # Spectra are deduplicated by content; metadata (object, time, instrument,
98
+ # observer, site, resolution, coverage) is extracted from the headers.
99
+ report = st.ingest_folder("~/spectra/alphadra", source="staros", tags=["campaign:alphadra"])
100
+ print(report) # 1012 files → 987 added, 25 duplicates, 0 errors
101
+
102
+ # 2. embed — any spectro-kernel strategy; several encoders coexist for the same spectra
103
+ enc = st.embed(strategy="dct", dim=256, norm_method="z_score") # → 'dct:v1:256:nogrid:z_score'
104
+ enc_ll = st.embed(strategy="log_lambda", dim=256) # Doppler-robust recipe
105
+ # tip: with min_max the continuum term dominates the vector and every pair scores ≈ 0.99;
106
+ # z_score (zero mean) or continuum_subtracted let the lines drive the similarity.
107
+
108
+ # 3. ask
109
+ st.neighbours("~/spectra/new_alfdra.fits", k=10, encoder=enc) # k-NN by file…
110
+ st.neighbours(some_id, k=10, encoder=enc, where="observer = 'mlelain'") # …or by id, filtered
111
+ ids, X = st.vectors(enc, where="series_id = 'alf dra'") # numpy matrix for analysis
112
+
113
+ from findakin.analysis import cluster_embeddings, detect_anomalies, project_embeddings
114
+ labels = cluster_embeddings(X, method="hdbscan")
115
+ outliers = detect_anomalies(X, method="lof", contamination=0.02)
116
+ xy = project_embeddings(X, method="umap")
117
+ ```
118
+
119
+ Same verbs on the command line:
120
+
121
+ ```bash
122
+ findakin ingest ~/spectra/alphadra --source staros --tag campaign:alphadra
123
+ findakin embed --strategy dct --dim 256
124
+ findakin search ~/spectra/new_alfdra.fits -k 10
125
+ findakin info
126
+ ```
127
+
128
+ ## What is in the store
129
+
130
+ A LanceDB directory, versioned, rebuildable from your files:
131
+
132
+ | Table | Content |
133
+ |---|---|
134
+ | `spectra` | one row per spectrum: `id` (content hash), `source`, `object`, `series_id`, `t_mjd`, `instrument`, `observer`, `site`, `resolution`, `wl_min/wl_max`, `file_uri`, `tags`, raw header (JSON) |
135
+ | `flux_native` | the native wavelength / flux arrays (re-embed with a new encoder without re-reading files) |
136
+ | `emb__<encoder>` | one table per encoder: `(id, vector, provenance)` — `encoder_id = strategy:model_version:dim:grid:norm` |
137
+ | `encoders` | the encoder registry (config, dimension) |
138
+
139
+ `series_id` groups spectra that form a trajectory in time: an object followed over epochs
140
+ (default), a night of sky spectra (`series_from="night"`, e.g. aurorae), or a folder.
141
+
142
+ ## Roadmap (short)
143
+
144
+ Phase 0 — store, ingestion, kernel encoders, k-NN, CLI ✅ · Phase 1 — common λ grids
145
+ (inter-instrument alignment), `flux__<grid>` tables, vector indexes · Phase 2 — named
146
+ populations, clusters / anomalies / projections stored as results, **trajectory anomalies**,
147
+ Plotly notebook helpers · Phase 3 — learned encoders registry, encoder benchmark with an
148
+ *instrument-leakage* score · Phase 4 — `findakin-mcp` over the store · Phase 5 — thin HTTP
149
+ façade for STAROS / SpecRelay. Details: [docs/roadmap/](docs/roadmap/).
150
+
151
+ ## Development
152
+
153
+ ```bash
154
+ uv venv .venv --python 3.12 && source .venv/bin/activate
155
+ uv pip install -e ".[dev]"
156
+ pytest -q
157
+ ruff check src tests
158
+ ```
159
+
160
+ The landing page lives in [website/](website/) (Vite + React, deployed separately). Project
161
+ notes and decisions: [docs/](docs/) and [stuff/](stuff/).
162
+
163
+ ## License
164
+
165
+ MIT — © 2026 Matthieu Le Lain.
@@ -0,0 +1,110 @@
1
+ # Findakin
2
+
3
+ **Find a kin for every spectrum.**
4
+
5
+ Findakin answers *population* questions about collections of astronomical spectra:
6
+ which look alike, what structure they form, which stand out, which changed in time, and
7
+ which representation (encoder) is the most physical. It is a local Python library built on
8
+ [spectro-kernel](https://github.com/matthieulel/spectro-kernel) (one spectrum → one vector)
9
+ and [LanceDB](https://lancedb.com) (an embedded, versioned, columnar store with a vector
10
+ index). No server, no account: your spectra stay on your disk.
11
+
12
+ > **Status: v2 alpha.** The hosted web application (v1, FastAPI + MongoDB + Qdrant + React)
13
+ > is archived under the git tag `v1-webapp`. The rewrite into a library is documented in
14
+ > [docs/roadmap/findakin-v2-local-first.md](docs/roadmap/findakin-v2-local-first.md).
15
+ > Landing page: [findakin.lelain.net](https://findakin.lelain.net).
16
+
17
+ ```
18
+ spectro-kernel = algorithms on ONE spectrum (embeddings, I/O, lines, RV, reduction…)
19
+ Findakin = questions on MANY spectra (index, neighbours, clusters, anomalies,
20
+ trajectories, encoder benchmarks)
21
+ ```
22
+
23
+ ## Install
24
+
25
+ ```bash
26
+ pip install findakin # core: store, ingestion, kernel recipes, k-NN
27
+ pip install "findakin[analysis]" # + UMAP projection, HDBSCAN clustering
28
+ pip install "findakin[vo]" # + SSAP / Virtual Observatory connectors
29
+ pip install "findakin[encoders]" # + local PyTorch encoders (spectro-kernel embed_pretrained)
30
+ ```
31
+
32
+ Python ≥ 3.11.
33
+
34
+ ## Quick start
35
+
36
+ ```python
37
+ from findakin import Store
38
+
39
+ st = Store.open("~/findakin/store") # a directory; created if missing
40
+
41
+ # 1. ingest — FITS (BeSS / ISIS / specINTI / Demetra dialects), CSV, JSON, TXT.
42
+ # Spectra are deduplicated by content; metadata (object, time, instrument,
43
+ # observer, site, resolution, coverage) is extracted from the headers.
44
+ report = st.ingest_folder("~/spectra/alphadra", source="staros", tags=["campaign:alphadra"])
45
+ print(report) # 1012 files → 987 added, 25 duplicates, 0 errors
46
+
47
+ # 2. embed — any spectro-kernel strategy; several encoders coexist for the same spectra
48
+ enc = st.embed(strategy="dct", dim=256, norm_method="z_score") # → 'dct:v1:256:nogrid:z_score'
49
+ enc_ll = st.embed(strategy="log_lambda", dim=256) # Doppler-robust recipe
50
+ # tip: with min_max the continuum term dominates the vector and every pair scores ≈ 0.99;
51
+ # z_score (zero mean) or continuum_subtracted let the lines drive the similarity.
52
+
53
+ # 3. ask
54
+ st.neighbours("~/spectra/new_alfdra.fits", k=10, encoder=enc) # k-NN by file…
55
+ st.neighbours(some_id, k=10, encoder=enc, where="observer = 'mlelain'") # …or by id, filtered
56
+ ids, X = st.vectors(enc, where="series_id = 'alf dra'") # numpy matrix for analysis
57
+
58
+ from findakin.analysis import cluster_embeddings, detect_anomalies, project_embeddings
59
+ labels = cluster_embeddings(X, method="hdbscan")
60
+ outliers = detect_anomalies(X, method="lof", contamination=0.02)
61
+ xy = project_embeddings(X, method="umap")
62
+ ```
63
+
64
+ Same verbs on the command line:
65
+
66
+ ```bash
67
+ findakin ingest ~/spectra/alphadra --source staros --tag campaign:alphadra
68
+ findakin embed --strategy dct --dim 256
69
+ findakin search ~/spectra/new_alfdra.fits -k 10
70
+ findakin info
71
+ ```
72
+
73
+ ## What is in the store
74
+
75
+ A LanceDB directory, versioned, rebuildable from your files:
76
+
77
+ | Table | Content |
78
+ |---|---|
79
+ | `spectra` | one row per spectrum: `id` (content hash), `source`, `object`, `series_id`, `t_mjd`, `instrument`, `observer`, `site`, `resolution`, `wl_min/wl_max`, `file_uri`, `tags`, raw header (JSON) |
80
+ | `flux_native` | the native wavelength / flux arrays (re-embed with a new encoder without re-reading files) |
81
+ | `emb__<encoder>` | one table per encoder: `(id, vector, provenance)` — `encoder_id = strategy:model_version:dim:grid:norm` |
82
+ | `encoders` | the encoder registry (config, dimension) |
83
+
84
+ `series_id` groups spectra that form a trajectory in time: an object followed over epochs
85
+ (default), a night of sky spectra (`series_from="night"`, e.g. aurorae), or a folder.
86
+
87
+ ## Roadmap (short)
88
+
89
+ Phase 0 — store, ingestion, kernel encoders, k-NN, CLI ✅ · Phase 1 — common λ grids
90
+ (inter-instrument alignment), `flux__<grid>` tables, vector indexes · Phase 2 — named
91
+ populations, clusters / anomalies / projections stored as results, **trajectory anomalies**,
92
+ Plotly notebook helpers · Phase 3 — learned encoders registry, encoder benchmark with an
93
+ *instrument-leakage* score · Phase 4 — `findakin-mcp` over the store · Phase 5 — thin HTTP
94
+ façade for STAROS / SpecRelay. Details: [docs/roadmap/](docs/roadmap/).
95
+
96
+ ## Development
97
+
98
+ ```bash
99
+ uv venv .venv --python 3.12 && source .venv/bin/activate
100
+ uv pip install -e ".[dev]"
101
+ pytest -q
102
+ ruff check src tests
103
+ ```
104
+
105
+ The landing page lives in [website/](website/) (Vite + React, deployed separately). Project
106
+ notes and decisions: [docs/](docs/) and [stuff/](stuff/).
107
+
108
+ ## License
109
+
110
+ MIT — © 2026 Matthieu Le Lain.
@@ -0,0 +1,35 @@
1
+ # 📖 Documentation Findakin
2
+
3
+ > **Bibliothèque Python locale** de recherche de similarité et d'analyse de population sur des
4
+ > spectres astronomiques, adossée à spectro-kernel (un spectre → un vecteur) et LanceDB (store
5
+ > local versionné). Le code de l'application web v1 est archivé sous le tag git `v1-webapp`.
6
+
7
+ ---
8
+
9
+ ## 🗂️ Structure
10
+
11
+ ```
12
+ docs/
13
+ ├── roadmap/ # 🗺️ Décisions et évolutions
14
+ │ ├── findakin-v2-local-first.md # ★ La décision d'architecture v2 (lib + LanceDB) et le plan
15
+ │ ├── kernel-vs-findakin.md # ★ Frontière spectro-kernel ↔ Findakin
16
+ │ ├── vision.md, backlog.md, … # Réflexions antérieures (époque SaaS, à lire comme historique)
17
+ │
18
+ └── archive/
19
+ └── v1-webapp/ # 📦 Documentation de l'application web v1 (API REST, infra cloud)
20
+ ├── reference/
21
+ └── app_description.md
22
+ ```
23
+
24
+ ---
25
+
26
+ ## 🚀 Démarrage rapide
27
+
28
+ → Le [README](../README.md) à la racine : installation, `Store`, CLI.
29
+ → [roadmap/findakin-v2-local-first.md](roadmap/findakin-v2-local-first.md) : pourquoi cette forme, tables du store, plan par phases.
30
+
31
+ ---
32
+
33
+ ## 📅 Dernière mise à jour
34
+
35
+ 28 janvier 2026
@@ -0,0 +1,60 @@
1
+ # 📚 Documentation Findakin
2
+
3
+ > **Documentation technique de référence**
4
+
5
+ ---
6
+
7
+ ## ⚠️ Important : Findakin n'est pas une solution d'archivage
8
+
9
+ **Findakin est un service d'indexation vectorielle pour la recherche par similarité**, pas un système d'archivage ou de sauvegarde de spectres.
10
+
11
+ - Les spectres peuvent être supprimés, réindexés ou modifiés à tout moment
12
+ - Le service conserve une copie des spectres pour l'affichage, mais ce n'est pas sa fonction principale
13
+ - Les spectres sont stockés de manière optimisée pour la recherche, pas pour la préservation à long terme
14
+ - **Conservez toujours vos fichiers originaux dans un système de sauvegarde approprié** (serveur de fichiers, système de versioning, archive institutionnelle, etc.)
15
+
16
+ Findakin est conçu pour :
17
+ - ✅ Rechercher rapidement des spectres similaires dans de grandes collections
18
+ - ✅ Expérimenter avec différentes stratégies de vectorisation
19
+ - ✅ Analyser les relations entre spectres via leurs embeddings
20
+ - ❌ **Pas** pour remplacer vos systèmes d'archivage classiques
21
+
22
+ ---
23
+
24
+ ## 📖 Contenu
25
+
26
+ Cette documentation décrit l'état actuel d'Findakin et évolue avec chaque nouvelle fonctionnalité implémentée.
27
+
28
+ | Document | Description |
29
+ |----------|-------------|
30
+ | [Architecture](architecture.md) | Stack technique, composants, décisions d'architecture (ADRs) |
31
+ | [Concepts](concepts.md) | Embeddings, vecteurs, similarité, stratégies |
32
+ | [API](api.md) | Documentation complète de l'API REST v1 |
33
+ | [Infrastructure](infrastructure.md) | Services cloud, Docker, déploiement |
34
+
35
+ ---
36
+
37
+ ## 🏗️ Architecture en bref
38
+
39
+ ```
40
+ Frontend (React) → API (FastAPI) → Worker de jobs (file MongoDB)
41
+ ↓
42
+ ┌─────────────┼─────────────┐
43
+ ↓ ↓ ↓
44
+ MongoDB Atlas Qdrant Cloud Cloudflare R2
45
+ (métadonnées) (vecteurs) (fichiers)
46
+ ```
47
+
48
+ ---
49
+
50
+ ## 🔗 Liens rapides
51
+
52
+ - **API Docs** : `/docs` (Swagger UI)
53
+ - **Roadmap** : [../roadmap/](../roadmap/)
54
+ - **Archive** : [../archive/](../archive/)
55
+
56
+ ---
57
+
58
+ ## 📅 Dernière mise à jour
59
+
60
+ 28 janvier 2026