JCcoder0901semantic-search 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,149 @@
1
+ Metadata-Version: 2.4
2
+ Name: JCcoder0901semantic_search
3
+ Version: 0.1.0
4
+ Summary: Search your own files by meaning, entirely offline, through a simple local web UI.
5
+ Author: Jcthecoder200
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/Jcthecoder200/semantic_search
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Intended Audience :: End Users/Desktop
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Classifier: Operating System :: OS Independent
12
+ Classifier: Programming Language :: Python :: 3
13
+ Requires-Python: >=3.9
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: fastapi
17
+ Requires-Dist: uvicorn
18
+ Requires-Dist: fastembed
19
+ Requires-Dist: numpy
20
+ Dynamic: license-file
21
+
22
+ # Local File Search
23
+
24
+ Search your own files by *meaning*, not just keyword matching — entirely on
25
+ your own machine, nothing uploaded anywhere.
26
+
27
+ ## Install
28
+
29
+ **Recommended: [pipx](https://pipx.pypa.io)** — installs the command and
30
+ automatically makes it available in your terminal, avoiding a common
31
+ Windows/Mac/Linux gotcha where plain `pip install` puts the command
32
+ somewhere your terminal doesn't look (see Troubleshooting below if you hit
33
+ this).
34
+
35
+ ```
36
+ pip install pipx
37
+ pipx ensurepath
38
+ pipx install JCcoder0901semantic_search
39
+ ```
40
+ *(rename `JCcoder0901semantic_search` once you've picked and published under your
41
+ chosen name)*
42
+
43
+ **Alternative: plain pip**
44
+ ```
45
+ pip install JCcoder0901semantic_search
46
+ ```
47
+
48
+ ## Run
49
+
50
+ ```
51
+ localsearch
52
+ ```
53
+
54
+ If that command isn't found, see **Troubleshooting** below — this is a very
55
+ common first-run snag with Python command-line tools in general, not
56
+ specific to this one, and there's a guaranteed-to-work fallback.
57
+
58
+ Then open your browser to:
59
+ ```
60
+ http://127.0.0.1:9120
61
+ ```
62
+
63
+ You'll see a simple page:
64
+
65
+ 1. **Index a folder** — type the full path to a folder you want searchable
66
+ (e.g. `C:\Users\you\Documents\notes`), pick which extensions to include if
67
+ you want more than the `.txt`/`.md` defaults, click **Index**.
68
+
69
+ 2. **Search** — pick a mode (file name + content, file name only, or content
70
+ only), type a plain-English question, click **Search**. Results are
71
+ ranked by how close their *meaning* is to your question, not exact word
72
+ matches — a file can show up even without containing your exact search
73
+ terms.
74
+
75
+ That's the whole workflow — no curl, no JSON, no terminal commands after the
76
+ initial `localsearch`.
77
+
78
+ First run downloads a small embedding model (~50MB via `fastembed`), needs
79
+ internet once, then works fully offline.
80
+
81
+ Your search index is saved to `~/.local_file_search/index.pkl`, so it
82
+ survives restarts — you don't need to re-index folders you already indexed.
83
+
84
+ ## Troubleshooting: "localsearch is not recognized"
85
+
86
+ This happens when `pip` (not `pipx`) installs the command into a folder your
87
+ terminal's PATH doesn't include — it's a well-known Python packaging quirk,
88
+ most common on Windows, and not something wrong with your installation.
89
+
90
+ **Guaranteed fix, works every time regardless of PATH:**
91
+ ```
92
+ python -m search_api.main
93
+ ```
94
+
95
+ **Permanent fix:** switch to `pipx` (see Install above) — it's built
96
+ specifically to solve this, or manually add Python's Scripts folder to your
97
+ PATH:
98
+ ```
99
+ python -c "import sysconfig; print(sysconfig.get_path('scripts'))"
100
+ ```
101
+ Add the folder that prints to your system PATH (search "environment
102
+ variables" in the Start menu on Windows), then open a *new* terminal window.
103
+
104
+ ## Running via Docker instead (optional)
105
+
106
+ If you'd rather not install anything into your system Python, a `Dockerfile`
107
+ is included.
108
+
109
+ ```powershell
110
+ docker build -t local-file-search .
111
+ docker run -p 9120:9120 -v "${PWD}/data:/data" -v "${env:USERPROFILE}:/host" local-file-search
112
+ ```
113
+ `${env:USERPROFILE}` mounts your whole Windows user folder as `/host` inside
114
+ the container, so a folder like `Documents\notes` in your profile becomes
115
+ `/host/Documents/notes` when typed into the app. `${PWD}/data:/data`
116
+ persists the index outside the container so it survives restarts.
117
+
118
+ ## Notes on scope
119
+
120
+ - Only files you explicitly `Index` get searched — nothing is scanned
121
+ automatically just because it exists on disk.
122
+ - Point Index at specific folders rather than an entire drive — indexing
123
+ everything would be slow and pull in a lot of irrelevant files.
124
+ - Supported file types by default: `.txt` and `.md`. Add more via the
125
+ `extensions` field in the Index request, or by editing
126
+ `search_api/main.py`.
127
+
128
+ ## How it works, briefly
129
+
130
+ - **Index**: for each matching file, embeds the filename and the content as
131
+ two *separate* vectors (lists of numbers capturing meaning) using a small
132
+ local model (via `fastembed`, no PyTorch/GPU required), and saves them to
133
+ `index.pkl`.
134
+ - **Search**: embeds your question the same way, compares it against the
135
+ stored vectors by cosine similarity, and returns the closest matches. The
136
+ mode selector controls whether "closest" is judged by filename, content,
137
+ or whichever of the two is the stronger match.
138
+
139
+ ## Publishing this yourself (for the maintainer)
140
+
141
+ ```
142
+ pip install build twine
143
+ python -m build
144
+ twine upload dist/*
145
+ ```
146
+ Before your first upload: pick a unique name at pypi.org, update it in
147
+ `pyproject.toml` (`name` and the `localsearch` install command above), fill
148
+ in your name in `pyproject.toml` and `LICENSE`, and set up a PyPI API token
149
+ for `twine` to use.
@@ -0,0 +1,13 @@
1
+ LICENSE
2
+ MANIFEST.in
3
+ README.md
4
+ pyproject.toml
5
+ JCcoder0901semantic_search.egg-info/PKG-INFO
6
+ JCcoder0901semantic_search.egg-info/SOURCES.txt
7
+ JCcoder0901semantic_search.egg-info/dependency_links.txt
8
+ JCcoder0901semantic_search.egg-info/entry_points.txt
9
+ JCcoder0901semantic_search.egg-info/requires.txt
10
+ JCcoder0901semantic_search.egg-info/top_level.txt
11
+ search_api/__init__.py
12
+ search_api/main.py
13
+ search_api/static/index.html
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ localsearch = search_api.main:start
@@ -0,0 +1,4 @@
1
+ fastapi
2
+ uvicorn
3
+ fastembed
4
+ numpy
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Your Name
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ recursive-include search_api/static *
@@ -0,0 +1,149 @@
1
+ Metadata-Version: 2.4
2
+ Name: JCcoder0901semantic_search
3
+ Version: 0.1.0
4
+ Summary: Search your own files by meaning, entirely offline, through a simple local web UI.
5
+ Author: Jcthecoder200
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/Jcthecoder200/semantic_search
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Intended Audience :: End Users/Desktop
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Classifier: Operating System :: OS Independent
12
+ Classifier: Programming Language :: Python :: 3
13
+ Requires-Python: >=3.9
14
+ Description-Content-Type: text/markdown
15
+ License-File: LICENSE
16
+ Requires-Dist: fastapi
17
+ Requires-Dist: uvicorn
18
+ Requires-Dist: fastembed
19
+ Requires-Dist: numpy
20
+ Dynamic: license-file
21
+
22
+ # Local File Search
23
+
24
+ Search your own files by *meaning*, not just keyword matching — entirely on
25
+ your own machine, nothing uploaded anywhere.
26
+
27
+ ## Install
28
+
29
+ **Recommended: [pipx](https://pipx.pypa.io)** — installs the command and
30
+ automatically makes it available in your terminal, avoiding a common
31
+ Windows/Mac/Linux gotcha where plain `pip install` puts the command
32
+ somewhere your terminal doesn't look (see Troubleshooting below if you hit
33
+ this).
34
+
35
+ ```
36
+ pip install pipx
37
+ pipx ensurepath
38
+ pipx install JCcoder0901semantic_search
39
+ ```
40
+ *(rename `JCcoder0901semantic_search` once you've picked and published under your
41
+ chosen name)*
42
+
43
+ **Alternative: plain pip**
44
+ ```
45
+ pip install JCcoder0901semantic_search
46
+ ```
47
+
48
+ ## Run
49
+
50
+ ```
51
+ localsearch
52
+ ```
53
+
54
+ If that command isn't found, see **Troubleshooting** below — this is a very
55
+ common first-run snag with Python command-line tools in general, not
56
+ specific to this one, and there's a guaranteed-to-work fallback.
57
+
58
+ Then open your browser to:
59
+ ```
60
+ http://127.0.0.1:9120
61
+ ```
62
+
63
+ You'll see a simple page:
64
+
65
+ 1. **Index a folder** — type the full path to a folder you want searchable
66
+ (e.g. `C:\Users\you\Documents\notes`), pick which extensions to include if
67
+ you want more than the `.txt`/`.md` defaults, click **Index**.
68
+
69
+ 2. **Search** — pick a mode (file name + content, file name only, or content
70
+ only), type a plain-English question, click **Search**. Results are
71
+ ranked by how close their *meaning* is to your question, not exact word
72
+ matches — a file can show up even without containing your exact search
73
+ terms.
74
+
75
+ That's the whole workflow — no curl, no JSON, no terminal commands after the
76
+ initial `localsearch`.
77
+
78
+ First run downloads a small embedding model (~50MB via `fastembed`), needs
79
+ internet once, then works fully offline.
80
+
81
+ Your search index is saved to `~/.local_file_search/index.pkl`, so it
82
+ survives restarts — you don't need to re-index folders you already indexed.
83
+
84
+ ## Troubleshooting: "localsearch is not recognized"
85
+
86
+ This happens when `pip` (not `pipx`) installs the command into a folder your
87
+ terminal's PATH doesn't include — it's a well-known Python packaging quirk,
88
+ most common on Windows, and not something wrong with your installation.
89
+
90
+ **Guaranteed fix, works every time regardless of PATH:**
91
+ ```
92
+ python -m search_api.main
93
+ ```
94
+
95
+ **Permanent fix:** switch to `pipx` (see Install above) — it's built
96
+ specifically to solve this, or manually add Python's Scripts folder to your
97
+ PATH:
98
+ ```
99
+ python -c "import sysconfig; print(sysconfig.get_path('scripts'))"
100
+ ```
101
+ Add the folder that prints to your system PATH (search "environment
102
+ variables" in the Start menu on Windows), then open a *new* terminal window.
103
+
104
+ ## Running via Docker instead (optional)
105
+
106
+ If you'd rather not install anything into your system Python, a `Dockerfile`
107
+ is included.
108
+
109
+ ```powershell
110
+ docker build -t local-file-search .
111
+ docker run -p 9120:9120 -v "${PWD}/data:/data" -v "${env:USERPROFILE}:/host" local-file-search
112
+ ```
113
+ `${env:USERPROFILE}` mounts your whole Windows user folder as `/host` inside
114
+ the container, so a folder like `Documents\notes` in your profile becomes
115
+ `/host/Documents/notes` when typed into the app. `${PWD}/data:/data`
116
+ persists the index outside the container so it survives restarts.
117
+
118
+ ## Notes on scope
119
+
120
+ - Only files you explicitly `Index` get searched — nothing is scanned
121
+ automatically just because it exists on disk.
122
+ - Point Index at specific folders rather than an entire drive — indexing
123
+ everything would be slow and pull in a lot of irrelevant files.
124
+ - Supported file types by default: `.txt` and `.md`. Add more via the
125
+ `extensions` field in the Index request, or by editing
126
+ `search_api/main.py`.
127
+
128
+ ## How it works, briefly
129
+
130
+ - **Index**: for each matching file, embeds the filename and the content as
131
+ two *separate* vectors (lists of numbers capturing meaning) using a small
132
+ local model (via `fastembed`, no PyTorch/GPU required), and saves them to
133
+ `index.pkl`.
134
+ - **Search**: embeds your question the same way, compares it against the
135
+ stored vectors by cosine similarity, and returns the closest matches. The
136
+ mode selector controls whether "closest" is judged by filename, content,
137
+ or whichever of the two is the stronger match.
138
+
139
+ ## Publishing this yourself (for the maintainer)
140
+
141
+ ```
142
+ pip install build twine
143
+ python -m build
144
+ twine upload dist/*
145
+ ```
146
+ Before your first upload: pick a unique name at pypi.org, update it in
147
+ `pyproject.toml` (`name` and the `localsearch` install command above), fill
148
+ in your name in `pyproject.toml` and `LICENSE`, and set up a PyPI API token
149
+ for `twine` to use.
@@ -0,0 +1,128 @@
1
+ # Local File Search
2
+
3
+ Search your own files by *meaning*, not just keyword matching — entirely on
4
+ your own machine, nothing uploaded anywhere.
5
+
6
+ ## Install
7
+
8
+ **Recommended: [pipx](https://pipx.pypa.io)** — installs the command and
9
+ automatically makes it available in your terminal, avoiding a common
10
+ Windows/Mac/Linux gotcha where plain `pip install` puts the command
11
+ somewhere your terminal doesn't look (see Troubleshooting below if you hit
12
+ this).
13
+
14
+ ```
15
+ pip install pipx
16
+ pipx ensurepath
17
+ pipx install JCcoder0901semantic_search
18
+ ```
19
+ *(rename `JCcoder0901semantic_search` once you've picked and published under your
20
+ chosen name)*
21
+
22
+ **Alternative: plain pip**
23
+ ```
24
+ pip install JCcoder0901semantic_search
25
+ ```
26
+
27
+ ## Run
28
+
29
+ ```
30
+ localsearch
31
+ ```
32
+
33
+ If that command isn't found, see **Troubleshooting** below — this is a very
34
+ common first-run snag with Python command-line tools in general, not
35
+ specific to this one, and there's a guaranteed-to-work fallback.
36
+
37
+ Then open your browser to:
38
+ ```
39
+ http://127.0.0.1:9120
40
+ ```
41
+
42
+ You'll see a simple page:
43
+
44
+ 1. **Index a folder** — type the full path to a folder you want searchable
45
+ (e.g. `C:\Users\you\Documents\notes`), pick which extensions to include if
46
+ you want more than the `.txt`/`.md` defaults, click **Index**.
47
+
48
+ 2. **Search** — pick a mode (file name + content, file name only, or content
49
+ only), type a plain-English question, click **Search**. Results are
50
+ ranked by how close their *meaning* is to your question, not exact word
51
+ matches — a file can show up even without containing your exact search
52
+ terms.
53
+
54
+ That's the whole workflow — no curl, no JSON, no terminal commands after the
55
+ initial `localsearch`.
56
+
57
+ First run downloads a small embedding model (~50MB via `fastembed`), needs
58
+ internet once, then works fully offline.
59
+
60
+ Your search index is saved to `~/.local_file_search/index.pkl`, so it
61
+ survives restarts — you don't need to re-index folders you already indexed.
62
+
63
+ ## Troubleshooting: "localsearch is not recognized"
64
+
65
+ This happens when `pip` (not `pipx`) installs the command into a folder your
66
+ terminal's PATH doesn't include — it's a well-known Python packaging quirk,
67
+ most common on Windows, and not something wrong with your installation.
68
+
69
+ **Guaranteed fix, works every time regardless of PATH:**
70
+ ```
71
+ python -m search_api.main
72
+ ```
73
+
74
+ **Permanent fix:** switch to `pipx` (see Install above) — it's built
75
+ specifically to solve this, or manually add Python's Scripts folder to your
76
+ PATH:
77
+ ```
78
+ python -c "import sysconfig; print(sysconfig.get_path('scripts'))"
79
+ ```
80
+ Add the folder that prints to your system PATH (search "environment
81
+ variables" in the Start menu on Windows), then open a *new* terminal window.
82
+
83
+ ## Running via Docker instead (optional)
84
+
85
+ If you'd rather not install anything into your system Python, a `Dockerfile`
86
+ is included.
87
+
88
+ ```powershell
89
+ docker build -t local-file-search .
90
+ docker run -p 9120:9120 -v "${PWD}/data:/data" -v "${env:USERPROFILE}:/host" local-file-search
91
+ ```
92
+ `${env:USERPROFILE}` mounts your whole Windows user folder as `/host` inside
93
+ the container, so a folder like `Documents\notes` in your profile becomes
94
+ `/host/Documents/notes` when typed into the app. `${PWD}/data:/data`
95
+ persists the index outside the container so it survives restarts.
96
+
97
+ ## Notes on scope
98
+
99
+ - Only files you explicitly `Index` get searched — nothing is scanned
100
+ automatically just because it exists on disk.
101
+ - Point Index at specific folders rather than an entire drive — indexing
102
+ everything would be slow and pull in a lot of irrelevant files.
103
+ - Supported file types by default: `.txt` and `.md`. Add more via the
104
+ `extensions` field in the Index request, or by editing
105
+ `search_api/main.py`.
106
+
107
+ ## How it works, briefly
108
+
109
+ - **Index**: for each matching file, embeds the filename and the content as
110
+ two *separate* vectors (lists of numbers capturing meaning) using a small
111
+ local model (via `fastembed`, no PyTorch/GPU required), and saves them to
112
+ `index.pkl`.
113
+ - **Search**: embeds your question the same way, compares it against the
114
+ stored vectors by cosine similarity, and returns the closest matches. The
115
+ mode selector controls whether "closest" is judged by filename, content,
116
+ or whichever of the two is the stronger match.
117
+
118
+ ## Publishing this yourself (for the maintainer)
119
+
120
+ ```
121
+ pip install build twine
122
+ python -m build
123
+ twine upload dist/*
124
+ ```
125
+ Before your first upload: pick a unique name at pypi.org, update it in
126
+ `pyproject.toml` (`name` and the `localsearch` install command above), fill
127
+ in your name in `pyproject.toml` and `LICENSE`, and set up a PyPI API token
128
+ for `twine` to use.
@@ -0,0 +1,45 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ # CHANGE THIS: pick a name not already taken on PyPI (check at pypi.org first).
7
+ name = "JCcoder0901semantic_search"
8
+ version = "0.1.0"
9
+ description = "Search your own files by meaning, entirely offline, through a simple local web UI."
10
+ readme = "README.md"
11
+ requires-python = ">=3.9"
12
+ license = { text = "MIT" }
13
+ # CHANGE THIS: put your own name/contact here (or leave email out if you'd rather not share it).
14
+ authors = [
15
+ { name = "Jcthecoder200" }
16
+ ]
17
+ classifiers = [
18
+ "Development Status :: 3 - Alpha",
19
+ "Intended Audience :: End Users/Desktop",
20
+ "License :: OSI Approved :: MIT License",
21
+ "Operating System :: OS Independent",
22
+ "Programming Language :: Python :: 3",
23
+ ]
24
+ dependencies = [
25
+ "fastapi",
26
+ "uvicorn",
27
+ "fastembed",
28
+ "numpy",
29
+ ]
30
+
31
+ [project.urls]
32
+ # CHANGE THIS once you have a GitHub repo for it — PyPI shows this on your project page.
33
+ Homepage = "https://github.com/Jcthecoder200/semantic_search"
34
+
35
+ [project.scripts]
36
+ # This is the command people type after `pip install JCcoder0901semantic_search`.
37
+ localsearch = "search_api.main:start"
38
+
39
+ [tool.setuptools.packages.find]
40
+ include = ["search_api*"]
41
+
42
+ [tool.setuptools.package-data]
43
+ # Ships the browser UI (index.html) inside the installed package —
44
+ # without this, pip install would silently omit it and / would 404.
45
+ search_api = ["static/*"]
@@ -0,0 +1,239 @@
1
+ import os
2
+ import pickle
3
+ from typing import List
4
+
5
+ import numpy as np
6
+ import uvicorn
7
+ from fastapi import FastAPI, Query
8
+ from fastapi.responses import FileResponse
9
+ from fastapi.staticfiles import StaticFiles
10
+ from fastembed import TextEmbedding
11
+ from pydantic import BaseModel
12
+
13
+ app = FastAPI(title="Local File Search")
14
+
15
+ STATIC_DIR = os.path.join(os.path.dirname(__file__), "static")
16
+ app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
17
+
18
+
19
+ @app.get("/")
20
+ def home():
21
+ """Serve the simple browser UI instead of making people use curl/Swagger."""
22
+ return FileResponse(os.path.join(STATIC_DIR, "index.html"))
23
+
24
+ # fastembed uses ONNX Runtime instead of PyTorch — much smaller install,
25
+ # no compiled CUDA headers, and critically: no Windows long-path issues
26
+ # during install, which torch is prone to.
27
+ MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"
28
+
29
+ # Default: a hidden folder in the user's home directory, so the index
30
+ # persists regardless of which folder someone runs the command from.
31
+ # Overridable via env var — Docker setups point this at a mounted volume.
32
+ DEFAULT_INDEX_DIR = os.path.join(os.path.expanduser("~"), ".local_file_search")
33
+ INDEX_PATH = os.environ.get("INDEX_PATH", os.path.join(DEFAULT_INDEX_DIR, "index.pkl"))
34
+
35
+ model = TextEmbedding(model_name=MODEL_NAME)
36
+
37
+
38
+ def embed(text: str) -> np.ndarray:
39
+ """fastembed's .embed() returns a generator — pull the single result out."""
40
+ return next(model.embed([text]))
41
+
42
+
43
+ class IndexRequest(BaseModel):
44
+ folder_path: str
45
+ extensions: List[str] = [".txt", ".md"]
46
+
47
+
48
+ def load_index() -> dict:
49
+ """Index format: { file_path: {"snippet": str, "embedding": np.ndarray} }"""
50
+ if os.path.exists(INDEX_PATH):
51
+ with open(INDEX_PATH, "rb") as f:
52
+ return pickle.load(f)
53
+ return {}
54
+
55
+
56
+ def save_index(index: dict) -> None:
57
+ directory = os.path.dirname(INDEX_PATH)
58
+ if directory:
59
+ os.makedirs(directory, exist_ok=True)
60
+ with open(INDEX_PATH, "wb") as f:
61
+ pickle.dump(index, f)
62
+
63
+
64
+ @app.post("/index")
65
+ def index_folder(req: IndexRequest):
66
+ """Walk a folder, embed every matching text file, save the index to disk."""
67
+
68
+ # Fail loudly instead of silently returning 0 for a bad path.
69
+ if not os.path.exists(req.folder_path):
70
+ return {
71
+ "indexed_files": 0,
72
+ "error": f"Path does not exist: '{req.folder_path}'. If you're running "
73
+ f"this via Docker, remember your real folder needs to be under "
74
+ f"the mounted path (e.g. /host/...). If running directly, use "
75
+ f"the normal path on your computer.",
76
+ }
77
+ if not os.path.isdir(req.folder_path):
78
+ return {
79
+ "indexed_files": 0,
80
+ "error": f"'{req.folder_path}' exists but is not a folder.",
81
+ }
82
+
83
+ index = load_index()
84
+ indexed = 0
85
+ scanned = 0
86
+ empty_count = 0
87
+ unreadable_count = 0
88
+ skipped_extensions = set()
89
+
90
+ for root, _, files in os.walk(req.folder_path):
91
+ for fname in files:
92
+ scanned += 1
93
+ if not any(fname.endswith(ext) for ext in req.extensions):
94
+ ext = os.path.splitext(fname)[1] or "(no extension)"
95
+ skipped_extensions.add(ext)
96
+ continue
97
+
98
+ path = os.path.join(root, fname)
99
+ try:
100
+ with open(path, "r", encoding="utf-8", errors="ignore") as f:
101
+ text = f.read()
102
+ except OSError:
103
+ unreadable_count += 1
104
+ continue
105
+
106
+ if not text.strip():
107
+ empty_count += 1
108
+ continue
109
+
110
+ # Store filename and content as SEPARATE embeddings, so search can
111
+ # target either one independently instead of one blended vector.
112
+ name_without_ext = os.path.splitext(fname)[0].replace("_", " ").replace("-", " ")
113
+ name_embedding = embed(name_without_ext)
114
+ content_embedding = embed(text[:2000])
115
+
116
+ index[path] = {
117
+ "snippet": text[:200],
118
+ "name_embedding": name_embedding,
119
+ "content_embedding": content_embedding,
120
+ }
121
+ indexed += 1
122
+
123
+ if indexed == 0 and scanned > 0:
124
+ reasons = []
125
+ if skipped_extensions:
126
+ reasons.append(
127
+ f"{len(skipped_extensions)} file(s) had a non-matching extension "
128
+ f"(saw: {sorted(skipped_extensions)}, looking for: {req.extensions})"
129
+ )
130
+ if empty_count:
131
+ reasons.append(f"{empty_count} matching file(s) were empty (no text inside)")
132
+ if unreadable_count:
133
+ reasons.append(f"{unreadable_count} matching file(s) couldn't be read")
134
+ reason_text = "; ".join(reasons) if reasons else "no readable text was found"
135
+ return {
136
+ "indexed_files": 0,
137
+ "total_in_index": len(index),
138
+ "warning": f"Found {scanned} file(s) in that folder, but indexed none. "
139
+ f"Reason: {reason_text}.",
140
+ }
141
+ if scanned == 0:
142
+ return {
143
+ "indexed_files": 0,
144
+ "total_in_index": len(index),
145
+ "warning": f"The folder '{req.folder_path}' exists but contains no files "
146
+ f"(checked subfolders too). Double check it's the right path.",
147
+ }
148
+
149
+ save_index(index)
150
+ return {"indexed_files": indexed, "total_in_index": len(index)}
151
+
152
+
153
+ def cosine(a: np.ndarray, b: np.ndarray) -> float:
154
+ return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-8))
155
+
156
+
157
+ @app.get("/search")
158
+ def search(
159
+ q: str = Query(..., description="Natural language question"),
160
+ top_k: int = 5,
161
+ mode: str = Query("both", description="'filename', 'content', or 'both'"),
162
+ ):
163
+ """Embed the query, rank every indexed file by cosine similarity."""
164
+ index = load_index()
165
+ if not index:
166
+ return {"results": [], "message": "Index is empty — call POST /index first."}
167
+
168
+ if mode not in ("filename", "content", "both"):
169
+ return {"results": [], "error": f"Invalid mode '{mode}'. Use 'filename', 'content', or 'both'."}
170
+
171
+ # Handle indexes built before the filename/content split existed.
172
+ stale_entries = [p for p, d in index.items() if "name_embedding" not in d]
173
+ if stale_entries:
174
+ return {
175
+ "results": [],
176
+ "error": f"{len(stale_entries)} indexed file(s) were indexed with an older "
177
+ f"version of this app and need to be re-indexed before search will "
178
+ f"work. Click Index again on the same folder(s).",
179
+ }
180
+
181
+ query_embedding = embed(q)
182
+
183
+ scored = []
184
+ for path, data in index.items():
185
+ name_score = cosine(query_embedding, data["name_embedding"])
186
+ content_score = cosine(query_embedding, data["content_embedding"])
187
+
188
+ if mode == "filename":
189
+ combined = name_score
190
+ elif mode == "content":
191
+ combined = content_score
192
+ else: # both — a file counts as relevant if EITHER its name or its content is a strong match
193
+ combined = max(name_score, content_score)
194
+
195
+ scored.append((combined, name_score, content_score, path, data["snippet"]))
196
+
197
+ scored.sort(key=lambda x: x[0], reverse=True)
198
+ top_results = scored[:top_k]
199
+
200
+ response = {
201
+ "results": [
202
+ {
203
+ "file": path,
204
+ "score": round(combined, 4),
205
+ "name_score": round(name_score, 4),
206
+ "content_score": round(content_score, 4),
207
+ "snippet": snippet,
208
+ }
209
+ for combined, name_score, content_score, path, snippet in top_results
210
+ ],
211
+ "mode": mode,
212
+ }
213
+
214
+ # A real semantic match is typically 0.3+. Below that, results are likely
215
+ # noise rather than genuinely relevant — flag it instead of implying
216
+ # confidence the ranking doesn't actually have.
217
+ if top_results and top_results[0][0] < 0.3:
218
+ response["note"] = (
219
+ "Low confidence: none of your indexed files closely match this "
220
+ "query in meaning. These are just the least-unrelated of what's "
221
+ "indexed, not strong matches."
222
+ )
223
+
224
+ return response
225
+
226
+
227
+ @app.get("/status")
228
+ def status():
229
+ index = load_index()
230
+ return {"indexed_files": len(index), "model": MODEL_NAME}
231
+
232
+
233
+ def start():
234
+ print("Starting local semantic search API on http://0.0.0.0:9120 ...")
235
+ uvicorn.run(app, host="0.0.0.0", port=9120)
236
+
237
+
238
+ if __name__ == "__main__":
239
+ start()
@@ -0,0 +1,257 @@
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <title>File Search</title>
6
+ <style>
7
+ :root {
8
+ --bg: #f6f5f2;
9
+ --panel: #ffffff;
10
+ --ink: #1c1c1c;
11
+ --muted: #6b6b6b;
12
+ --accent: #2d5c4d;
13
+ --accent-hover: #234a3e;
14
+ --border: #e2e0da;
15
+ }
16
+ * { box-sizing: border-box; }
17
+ body {
18
+ margin: 0;
19
+ font-family: -apple-system, "Segoe UI", Helvetica, Arial, sans-serif;
20
+ background: var(--bg);
21
+ color: var(--ink);
22
+ display: flex;
23
+ justify-content: center;
24
+ padding: 48px 20px;
25
+ }
26
+ .app {
27
+ width: 100%;
28
+ max-width: 640px;
29
+ }
30
+ h1 {
31
+ font-size: 20px;
32
+ font-weight: 600;
33
+ margin: 0 0 4px 0;
34
+ }
35
+ .subtitle {
36
+ color: var(--muted);
37
+ font-size: 13px;
38
+ margin-bottom: 28px;
39
+ }
40
+ .panel {
41
+ background: var(--panel);
42
+ border: 1px solid var(--border);
43
+ border-radius: 10px;
44
+ padding: 20px;
45
+ margin-bottom: 16px;
46
+ }
47
+ .panel h2 {
48
+ font-size: 13px;
49
+ text-transform: uppercase;
50
+ letter-spacing: 0.04em;
51
+ color: var(--muted);
52
+ margin: 0 0 12px 0;
53
+ font-weight: 600;
54
+ }
55
+ .row {
56
+ display: flex;
57
+ gap: 8px;
58
+ }
59
+ input[type="text"] {
60
+ flex: 1;
61
+ padding: 10px 12px;
62
+ border: 1px solid var(--border);
63
+ border-radius: 7px;
64
+ font-size: 14px;
65
+ background: #fbfaf8;
66
+ }
67
+ input[type="text"]:focus {
68
+ outline: none;
69
+ border-color: var(--accent);
70
+ }
71
+ button {
72
+ padding: 10px 16px;
73
+ border: none;
74
+ border-radius: 7px;
75
+ background: var(--accent);
76
+ color: white;
77
+ font-size: 14px;
78
+ font-weight: 500;
79
+ cursor: pointer;
80
+ }
81
+ button:hover { background: var(--accent-hover); }
82
+ button:disabled { background: #b7b7b7; cursor: default; }
83
+ .status {
84
+ font-size: 13px;
85
+ color: var(--muted);
86
+ margin-top: 10px;
87
+ min-height: 16px;
88
+ }
89
+ .results { margin-top: 8px; }
90
+ .result {
91
+ padding: 12px;
92
+ border: 1px solid var(--border);
93
+ border-radius: 8px;
94
+ margin-bottom: 8px;
95
+ background: #fdfdfc;
96
+ }
97
+ .result .path {
98
+ font-size: 13px;
99
+ font-weight: 600;
100
+ word-break: break-all;
101
+ margin-bottom: 4px;
102
+ }
103
+ .result .snippet {
104
+ font-size: 13px;
105
+ color: var(--muted);
106
+ line-height: 1.4;
107
+ }
108
+ .result .score {
109
+ font-size: 11px;
110
+ color: var(--accent);
111
+ font-weight: 600;
112
+ margin-top: 6px;
113
+ }
114
+ .hint {
115
+ font-size: 12px;
116
+ color: var(--muted);
117
+ margin-top: 8px;
118
+ }
119
+ .mode-row {
120
+ display: flex;
121
+ gap: 16px;
122
+ margin-top: 10px;
123
+ font-size: 13px;
124
+ color: var(--muted);
125
+ }
126
+ .mode-row label {
127
+ display: flex;
128
+ align-items: center;
129
+ gap: 5px;
130
+ cursor: pointer;
131
+ }
132
+ .mode-row input[type="radio"] {
133
+ accent-color: var(--accent);
134
+ cursor: pointer;
135
+ }
136
+ </style>
137
+ </head>
138
+ <body>
139
+ <div class="app">
140
+ <h1>File Search</h1>
141
+ <div class="subtitle">Search your files by meaning, not just keywords. Runs entirely on your machine.</div>
142
+
143
+ <div class="panel">
144
+ <h2>1. Index a folder</h2>
145
+ <div class="row">
146
+ <input id="folderInput" type="text" placeholder="C:\Users\you\Documents\notes" />
147
+ <button id="indexBtn">Index</button>
148
+ </div>
149
+ <div class="hint">Type the full path to a folder on your computer. (Running via Docker instead? Prefix with <code>/host/</code>.) Only <code>.txt</code> and <code>.md</code> files are indexed by default.</div>
150
+ <div class="status" id="indexStatus"></div>
151
+ </div>
152
+
153
+ <div class="panel">
154
+ <h2>2. Search</h2>
155
+ <div class="row">
156
+ <input id="searchInput" type="text" placeholder="e.g. notes about budgeting" />
157
+ <button id="searchBtn">Search</button>
158
+ </div>
159
+ <div class="mode-row">
160
+ <label><input type="radio" name="searchMode" value="both" checked /> File name + content</label>
161
+ <label><input type="radio" name="searchMode" value="filename" /> File name only</label>
162
+ <label><input type="radio" name="searchMode" value="content" /> Content only</label>
163
+ </div>
164
+ <div class="status" id="searchStatus"></div>
165
+ <div class="results" id="results"></div>
166
+ </div>
167
+ </div>
168
+
169
+ <script>
170
+ const indexBtn = document.getElementById('indexBtn');
171
+ const searchBtn = document.getElementById('searchBtn');
172
+ const folderInput = document.getElementById('folderInput');
173
+ const searchInput = document.getElementById('searchInput');
174
+ const indexStatus = document.getElementById('indexStatus');
175
+ const searchStatus = document.getElementById('searchStatus');
176
+ const results = document.getElementById('results');
177
+
178
+ async function indexFolder() {
179
+ const folder_path = folderInput.value.trim();
180
+ if (!folder_path) { indexStatus.textContent = 'Enter a folder path first.'; return; }
181
+ indexBtn.disabled = true;
182
+ indexStatus.textContent = 'Indexing... this can take a moment for large folders.';
183
+ try {
184
+ const res = await fetch('/index', {
185
+ method: 'POST',
186
+ headers: { 'Content-Type': 'application/json' },
187
+ body: JSON.stringify({ folder_path })
188
+ });
189
+ const data = await res.json();
190
+ if (!res.ok) {
191
+ indexStatus.textContent = 'Error: ' + (data.detail ? JSON.stringify(data.detail) : res.status);
192
+ } else if (data.error) {
193
+ indexStatus.textContent = '⚠️ ' + data.error;
194
+ } else if (data.warning) {
195
+ indexStatus.textContent = '⚠️ ' + data.warning;
196
+ } else {
197
+ indexStatus.textContent = `Indexed ${data.indexed_files} files (${data.total_in_index} total in index).`;
198
+ }
199
+ } catch (err) {
200
+ indexStatus.textContent = 'Request failed: ' + err.message;
201
+ } finally {
202
+ indexBtn.disabled = false;
203
+ }
204
+ }
205
+
206
+ async function search() {
207
+ const q = searchInput.value.trim();
208
+ if (!q) { searchStatus.textContent = 'Type a search first.'; return; }
209
+ const mode = document.querySelector('input[name="searchMode"]:checked').value;
210
+ searchBtn.disabled = true;
211
+ searchStatus.textContent = 'Searching...';
212
+ results.innerHTML = '';
213
+ try {
214
+ const res = await fetch('/search?q=' + encodeURIComponent(q) + '&mode=' + encodeURIComponent(mode));
215
+ const data = await res.json();
216
+ if (!res.ok) {
217
+ searchStatus.textContent = 'Error: ' + (data.detail ? JSON.stringify(data.detail) : res.status);
218
+ return;
219
+ }
220
+ if (data.error) {
221
+ searchStatus.textContent = '⚠️ ' + data.error;
222
+ return;
223
+ }
224
+ if (!data.results || data.results.length === 0) {
225
+ searchStatus.textContent = data.message || 'No results.';
226
+ return;
227
+ }
228
+ searchStatus.textContent = `${data.results.length} result(s).` + (data.note ? ' ⚠️ ' + data.note : '');
229
+ for (const r of data.results) {
230
+ const div = document.createElement('div');
231
+ div.className = 'result';
232
+ div.innerHTML = `
233
+ <div class="path">${r.file}</div>
234
+ <div class="snippet">${r.snippet.replace(/</g, '&lt;')}</div>
235
+ <div class="score">match score: ${r.score} &nbsp;(name: ${r.name_score}, content: ${r.content_score})</div>
236
+ `;
237
+ results.appendChild(div);
238
+ }
239
+ } catch (err) {
240
+ searchStatus.textContent = 'Request failed: ' + err.message;
241
+ } finally {
242
+ searchBtn.disabled = false;
243
+ }
244
+ }
245
+
246
+ folderInput.addEventListener('focus', () => {
247
+ const len = folderInput.value.length;
248
+ folderInput.setSelectionRange(len, len);
249
+ });
250
+
251
+ indexBtn.addEventListener('click', indexFolder);
252
+ searchBtn.addEventListener('click', search);
253
+ folderInput.addEventListener('keydown', e => { if (e.key === 'Enter') indexFolder(); });
254
+ searchInput.addEventListener('keydown', e => { if (e.key === 'Enter') search(); });
255
+ </script>
256
+ </body>
257
+ </html>
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+