chunksilo 2.3.4__tar.gz → 2.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. {chunksilo-2.3.4/src/chunksilo.egg-info → chunksilo-2.4.0}/PKG-INFO +11 -10
  2. {chunksilo-2.3.4 → chunksilo-2.4.0}/README.md +8 -7
  3. {chunksilo-2.3.4 → chunksilo-2.4.0}/pyproject.toml +6 -3
  4. {chunksilo-2.3.4 → chunksilo-2.4.0}/requirements.txt +2 -0
  5. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/__init__.py +1 -1
  6. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/cfgload.py +19 -7
  7. {chunksilo-2.3.4 → chunksilo-2.4.0/src/chunksilo.egg-info}/PKG-INFO +11 -10
  8. {chunksilo-2.3.4 → chunksilo-2.4.0}/LICENSE +0 -0
  9. {chunksilo-2.3.4 → chunksilo-2.4.0}/NOTICE +0 -0
  10. {chunksilo-2.3.4 → chunksilo-2.4.0}/setup.cfg +0 -0
  11. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/__main__.py +0 -0
  12. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/cli.py +0 -0
  13. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/confluence_html_formatter.py +0 -0
  14. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/docx_utils.py +0 -0
  15. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/index.py +0 -0
  16. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/models.py +0 -0
  17. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/search.py +0 -0
  18. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/server.py +0 -0
  19. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo/ui.py +0 -0
  20. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo.egg-info/SOURCES.txt +0 -0
  21. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo.egg-info/dependency_links.txt +0 -0
  22. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo.egg-info/entry_points.txt +0 -0
  23. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo.egg-info/requires.txt +2 -2
  24. {chunksilo-2.3.4 → chunksilo-2.4.0}/src/chunksilo.egg-info/top_level.txt +0 -0
  25. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_chunk_location.py +0 -0
  26. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_cli.py +0 -0
  27. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_confluence_html_formatter.py +0 -0
  28. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_docx_utils.py +0 -0
  29. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_error_handling.py +0 -0
  30. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_heading_path_integration.py +0 -0
  31. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_incremental_ingest.py +0 -0
  32. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_indexing_benchmark.py +0 -0
  33. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_indexing_ui.py +0 -0
  34. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_jira_integration.py +0 -0
  35. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_quoted_phrases.py +0 -0
  36. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_rag_metrics.py +0 -0
  37. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_retrieval_only.py +0 -0
  38. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_scan_timeouts.py +0 -0
  39. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_server.py +0 -0
  40. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_system.py +0 -0
  41. {chunksilo-2.3.4 → chunksilo-2.4.0}/test/test_utils.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: chunksilo
3
- Version: 2.3.4
3
+ Version: 2.4.0
4
4
  Summary: Local RAG-based semantic document search with MCP server interface
5
5
  Author: Fredrik Reveny
6
6
  License-Expression: Apache-2.0
@@ -30,10 +30,10 @@ Requires-Dist: huggingface-hub<2,>=0.22.0
30
30
  Requires-Dist: flashrank<1,>=0.1.0
31
31
  Requires-Dist: fastembed<1,>=0.5.0
32
32
  Requires-Dist: pyyaml<7,>=6.0
33
+ Requires-Dist: llama-index-readers-confluence<1,>=0.6.0
34
+ Requires-Dist: jira<4,>=3.5.0
33
35
  Provides-Extra: confluence
34
- Requires-Dist: llama-index-readers-confluence<1,>=0.6.0; extra == "confluence"
35
36
  Provides-Extra: jira
36
- Requires-Dist: jira<4,>=3.5.0; extra == "jira"
37
37
  Provides-Extra: test
38
38
  Requires-Dist: pytest<9,>=7.4.0; extra == "test"
39
39
  Requires-Dist: pytest-cov>=4.0; extra == "test"
@@ -93,11 +93,12 @@ Requires Python 3.11 or later. Models are downloaded automatically on first run
93
93
 
94
94
  ```bash
95
95
  pip install chunksilo
96
-
97
- # Or with Confluence and Jira support:
98
- pip install chunksilo[confluence,jira]
99
96
  ```
100
97
 
98
+ Confluence and Jira support is included by default — just provide a config file
99
+ to enable them. (`pip install chunksilo[confluence,jira]` still works as an alias
100
+ for backward compatibility.)
101
+
101
102
  Then:
102
103
  1. **Create** a config file at `~/.config/chunksilo/config.yaml` (see [Configuration](#configuration))
103
104
  2. **Build** the index: `chunksilo --build-index`
@@ -226,7 +227,7 @@ All settings are optional and have sensible defaults.
226
227
 
227
228
  #### Confluence Settings (optional)
228
229
 
229
- > **Note:** Confluence integration requires the optional dependency. Install with: `pip install chunksilo[confluence]`
230
+ > **Note:** Confluence support is installed by default; just set the values below to enable it.
230
231
 
231
232
  | Setting | Description |
232
233
  | :--- | :--- |
@@ -244,7 +245,7 @@ All settings are optional and have sensible defaults.
244
245
 
245
246
  #### Jira Settings (optional)
246
247
 
247
- > **Note:** Jira integration requires the optional dependency. Install with: `pip install chunksilo[jira]`
248
+ > **Note:** Jira support is installed by default; just set the values below to enable it.
248
249
 
249
250
  | Setting | Description |
250
251
  | :--- | :--- |
@@ -419,8 +420,8 @@ Add to `mcp_settings.json` (typically in `~/.config/Code/User/globalStorage/roov
419
420
  - **Index missing**: Run `chunksilo --build-index` (PyPI install) or `./venv/bin/chunksilo --build-index` (offline bundle).
420
421
  - **Retrieval errors**: Check paths in your MCP client configuration.
421
422
  - **Offline mode**: PyPI installs default to `offline: false` (models auto-download). The offline bundle includes pre-downloaded models and sets `offline: true`. Set `retrieval.offline: true` in `config.yaml` to prevent network calls after initial model download.
422
- - **Confluence Integration**: Install with `pip install chunksilo[confluence]`, then set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
423
- - **Jira Integration**: Install with `pip install chunksilo[jira]`, then set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
423
+ - **Confluence Integration**: Included by default set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
424
+ - **Jira Integration**: Included by default set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
424
425
  - **Custom CA Bundle**: Set `ssl.ca_bundle_path` in `config.yaml` for custom certificates.
425
426
  - **Network mounts**: Unavailable directories are skipped with a warning; indexing continues with available directories.
426
427
  - **Legacy .doc files**: Requires LibreOffice to be installed for automatic conversion to .docx. If LibreOffice is not found, .doc files are skipped with a warning. Full heading extraction is supported.
@@ -51,11 +51,12 @@ Requires Python 3.11 or later. Models are downloaded automatically on first run
51
51
 
52
52
  ```bash
53
53
  pip install chunksilo
54
-
55
- # Or with Confluence and Jira support:
56
- pip install chunksilo[confluence,jira]
57
54
  ```
58
55
 
56
+ Confluence and Jira support is included by default — just provide a config file
57
+ to enable them. (`pip install chunksilo[confluence,jira]` still works as an alias
58
+ for backward compatibility.)
59
+
59
60
  Then:
60
61
  1. **Create** a config file at `~/.config/chunksilo/config.yaml` (see [Configuration](#configuration))
61
62
  2. **Build** the index: `chunksilo --build-index`
@@ -184,7 +185,7 @@ All settings are optional and have sensible defaults.
184
185
 
185
186
  #### Confluence Settings (optional)
186
187
 
187
- > **Note:** Confluence integration requires the optional dependency. Install with: `pip install chunksilo[confluence]`
188
+ > **Note:** Confluence support is installed by default; just set the values below to enable it.
188
189
 
189
190
  | Setting | Description |
190
191
  | :--- | :--- |
@@ -202,7 +203,7 @@ All settings are optional and have sensible defaults.
202
203
 
203
204
  #### Jira Settings (optional)
204
205
 
205
- > **Note:** Jira integration requires the optional dependency. Install with: `pip install chunksilo[jira]`
206
+ > **Note:** Jira support is installed by default; just set the values below to enable it.
206
207
 
207
208
  | Setting | Description |
208
209
  | :--- | :--- |
@@ -377,8 +378,8 @@ Add to `mcp_settings.json` (typically in `~/.config/Code/User/globalStorage/roov
377
378
  - **Index missing**: Run `chunksilo --build-index` (PyPI install) or `./venv/bin/chunksilo --build-index` (offline bundle).
378
379
  - **Retrieval errors**: Check paths in your MCP client configuration.
379
380
  - **Offline mode**: PyPI installs default to `offline: false` (models auto-download). The offline bundle includes pre-downloaded models and sets `offline: true`. Set `retrieval.offline: true` in `config.yaml` to prevent network calls after initial model download.
380
- - **Confluence Integration**: Install with `pip install chunksilo[confluence]`, then set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
381
- - **Jira Integration**: Install with `pip install chunksilo[jira]`, then set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
381
+ - **Confluence Integration**: Included by default set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
382
+ - **Jira Integration**: Included by default set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
382
383
  - **Custom CA Bundle**: Set `ssl.ca_bundle_path` in `config.yaml` for custom certificates.
383
384
  - **Network mounts**: Unavailable directories are skipped with a warning; indexing continues with available directories.
384
385
  - **Legacy .doc files**: Requires LibreOffice to be installed for automatic conversion to .docx. If LibreOffice is not found, .doc files are skipped with a warning. Full heading extraction is supported.
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "chunksilo"
7
- version = "2.3.4"
7
+ version = "2.4.0"
8
8
  description = "Local RAG-based semantic document search with MCP server interface"
9
9
  license = "Apache-2.0"
10
10
  requires-python = ">=3.11"
@@ -25,8 +25,11 @@ classifiers = [
25
25
  dynamic = ["dependencies"]
26
26
 
27
27
  [project.optional-dependencies]
28
- confluence = ["llama-index-readers-confluence>=0.6.0,<1"]
29
- jira = ["jira>=3.5.0,<4"]
28
+ # Confluence and Jira support is now installed by default (see requirements.txt).
29
+ # These extras are kept as empty aliases so existing `pip install chunksilo[confluence,jira]`
30
+ # commands keep working without a warning.
31
+ confluence = []
32
+ jira = []
30
33
  test = ["pytest>=7.4.0,<9", "pytest-cov>=4.0", "requests>=2.31.0,<3"]
31
34
 
32
35
  [project.scripts]
@@ -10,3 +10,5 @@ huggingface-hub>=0.22.0,<2
10
10
  flashrank>=0.1.0,<1
11
11
  fastembed>=0.5.0,<1
12
12
  pyyaml>=6.0,<7
13
+ llama-index-readers-confluence>=0.6.0,<1
14
+ jira>=3.5.0,<4
@@ -1,4 +1,4 @@
1
1
  # SPDX-License-Identifier: Apache-2.0
2
2
  """ChunkSilo - Local RAG-based semantic document search."""
3
3
 
4
- __version__ = "2.3.4"
4
+ __version__ = "2.4.0"
@@ -142,6 +142,14 @@ def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any
142
142
  return result
143
143
 
144
144
 
145
+ def _has_embedding_cache(cache_dir: Path) -> bool:
146
+ """True if ``cache_dir`` holds a HuggingFace-style embedding model cache."""
147
+ try:
148
+ return any(cache_dir.glob("models--*"))
149
+ except OSError:
150
+ return False
151
+
152
+
145
153
  def _resolve_bundled_models(config: dict[str, Any]) -> dict[str, Any]:
146
154
  """Transparently fall back to models bundled in the package.
147
155
 
@@ -150,16 +158,20 @@ def _resolve_bundled_models(config: dict[str, Any]) -> dict[str, Any]:
150
158
  offline mode. This lets a pip-installed wheel run in an air-gapped environment
151
159
  with zero extra configuration. A user-configured cache that already holds the
152
160
  models always wins, so explicit setups are never overridden.
153
- """
154
- embed_name = config["retrieval"]["embed_model_name"]
155
- hf_dirname = f"models--{embed_name.replace('/', '--')}"
156
161
 
157
- if not (_BUNDLED_MODELS_DIR / hf_dirname).exists():
158
- return config # No bundled models for this embedding model.
162
+ Detection looks for any HuggingFace-style ``models--*`` cache directory rather
163
+ than matching the configured embedding name: fastembed stores the model under
164
+ its upstream source repo (e.g. ``models--qdrant--bge-small-en-v1.5-onnx-q``),
165
+ not under the configured alias ``BAAI/bge-small-en-v1.5``.
166
+ """
167
+ if not _has_embedding_cache(_BUNDLED_MODELS_DIR):
168
+ return config # No models bundled in this install.
159
169
 
160
170
  configured = Path(config["storage"]["model_cache_dir"]).expanduser()
161
- if (configured / hf_dirname).exists():
162
- return config # Configured cache already has the models; respect it.
171
+ if configured.resolve() != _BUNDLED_MODELS_DIR.resolve() and _has_embedding_cache(
172
+ configured
173
+ ):
174
+ return config # Configured cache already has models; respect it.
163
175
 
164
176
  result = copy.deepcopy(config)
165
177
  result["storage"]["model_cache_dir"] = str(_BUNDLED_MODELS_DIR)
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: chunksilo
3
- Version: 2.3.4
3
+ Version: 2.4.0
4
4
  Summary: Local RAG-based semantic document search with MCP server interface
5
5
  Author: Fredrik Reveny
6
6
  License-Expression: Apache-2.0
@@ -30,10 +30,10 @@ Requires-Dist: huggingface-hub<2,>=0.22.0
30
30
  Requires-Dist: flashrank<1,>=0.1.0
31
31
  Requires-Dist: fastembed<1,>=0.5.0
32
32
  Requires-Dist: pyyaml<7,>=6.0
33
+ Requires-Dist: llama-index-readers-confluence<1,>=0.6.0
34
+ Requires-Dist: jira<4,>=3.5.0
33
35
  Provides-Extra: confluence
34
- Requires-Dist: llama-index-readers-confluence<1,>=0.6.0; extra == "confluence"
35
36
  Provides-Extra: jira
36
- Requires-Dist: jira<4,>=3.5.0; extra == "jira"
37
37
  Provides-Extra: test
38
38
  Requires-Dist: pytest<9,>=7.4.0; extra == "test"
39
39
  Requires-Dist: pytest-cov>=4.0; extra == "test"
@@ -93,11 +93,12 @@ Requires Python 3.11 or later. Models are downloaded automatically on first run
93
93
 
94
94
  ```bash
95
95
  pip install chunksilo
96
-
97
- # Or with Confluence and Jira support:
98
- pip install chunksilo[confluence,jira]
99
96
  ```
100
97
 
98
+ Confluence and Jira support is included by default — just provide a config file
99
+ to enable them. (`pip install chunksilo[confluence,jira]` still works as an alias
100
+ for backward compatibility.)
101
+
101
102
  Then:
102
103
  1. **Create** a config file at `~/.config/chunksilo/config.yaml` (see [Configuration](#configuration))
103
104
  2. **Build** the index: `chunksilo --build-index`
@@ -226,7 +227,7 @@ All settings are optional and have sensible defaults.
226
227
 
227
228
  #### Confluence Settings (optional)
228
229
 
229
- > **Note:** Confluence integration requires the optional dependency. Install with: `pip install chunksilo[confluence]`
230
+ > **Note:** Confluence support is installed by default; just set the values below to enable it.
230
231
 
231
232
  | Setting | Description |
232
233
  | :--- | :--- |
@@ -244,7 +245,7 @@ All settings are optional and have sensible defaults.
244
245
 
245
246
  #### Jira Settings (optional)
246
247
 
247
- > **Note:** Jira integration requires the optional dependency. Install with: `pip install chunksilo[jira]`
248
+ > **Note:** Jira support is installed by default; just set the values below to enable it.
248
249
 
249
250
  | Setting | Description |
250
251
  | :--- | :--- |
@@ -419,8 +420,8 @@ Add to `mcp_settings.json` (typically in `~/.config/Code/User/globalStorage/roov
419
420
  - **Index missing**: Run `chunksilo --build-index` (PyPI install) or `./venv/bin/chunksilo --build-index` (offline bundle).
420
421
  - **Retrieval errors**: Check paths in your MCP client configuration.
421
422
  - **Offline mode**: PyPI installs default to `offline: false` (models auto-download). The offline bundle includes pre-downloaded models and sets `offline: true`. Set `retrieval.offline: true` in `config.yaml` to prevent network calls after initial model download.
422
- - **Confluence Integration**: Install with `pip install chunksilo[confluence]`, then set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
423
- - **Jira Integration**: Install with `pip install chunksilo[jira]`, then set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
423
+ - **Confluence Integration**: Included by default set `confluence.url`, `confluence.username`, and `confluence.api_token` in `config.yaml`.
424
+ - **Jira Integration**: Included by default set `jira.url`, `jira.username`, and `jira.api_token` in `config.yaml`. Optionally configure `jira.projects` to restrict search to specific project keys.
424
425
  - **Custom CA Bundle**: Set `ssl.ca_bundle_path` in `config.yaml` for custom certificates.
425
426
  - **Network mounts**: Unavailable directories are skipped with a warning; indexing continues with available directories.
426
427
  - **Legacy .doc files**: Requires LibreOffice to be installed for automatic conversion to .docx. If LibreOffice is not found, .doc files are skipped with a warning. Full heading extraction is supported.
File without changes
File without changes
File without changes
File without changes
@@ -10,12 +10,12 @@ huggingface-hub<2,>=0.22.0
10
10
  flashrank<1,>=0.1.0
11
11
  fastembed<1,>=0.5.0
12
12
  pyyaml<7,>=6.0
13
+ llama-index-readers-confluence<1,>=0.6.0
14
+ jira<4,>=3.5.0
13
15
 
14
16
  [confluence]
15
- llama-index-readers-confluence<1,>=0.6.0
16
17
 
17
18
  [jira]
18
- jira<4,>=3.5.0
19
19
 
20
20
  [test]
21
21
  pytest<9,>=7.4.0
File without changes
File without changes
File without changes
File without changes