biblicus 1.0.0__py3-none-any.whl → 1.1.0__py3-none-any.whl

{biblicus-1.0.0.dist-info → biblicus-1.1.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: biblicus
-Version: 1.0.0
+Version: 1.1.0
 Summary: Command line interface and Python library for corpus ingestion, retrieval, and evaluation.
 License: MIT
 Requires-Python: >=3.9
@@ -80,7 +80,7 @@ See [retrieval augmented generation overview] for a short introduction to the id
 ## Analysis highlights
 
 - `biblicus analyze markov` learns a directed, weighted state transition graph over segmented text.
-- YAML recipes support cascading composition plus dotted `--config key=value` overrides.
+- YAML configurations support cascading composition plus dotted `--config key=value` overrides.
 - Text extract splits long texts with an LLM by inserting XML tags in-place for structured spans.
 - See `docs/MARKOV_ANALYSIS.md` for Markov analysis details and runnable demos.
 - See `docs/TEXT_EXTRACT.md` for the text extract utility and examples.
@@ -167,7 +167,7 @@ sequenceDiagram
 
 - You can ingest raw material once, then try many retrieval approaches over time.
 - You can keep raw files readable and portable, without locking your data inside a database.
-- You can evaluate retrieval runs against shared datasets and compare backends using the same corpus.
+- You can evaluate retrieval snapshots against shared datasets and compare backends using the same corpus.
 
 ## Typical flow
 
@@ -176,7 +176,7 @@ sequenceDiagram
 - Crawl a website section into corpus items when you want a repeatable “import from the web” workflow.
 - Run extraction when you want derived text artifacts from non-text sources.
 - Reindex to refresh the catalog after edits.
-- Build a retrieval run with a backend.
+- Build a retrieval snapshot with a backend.
 - Query the run to collect evidence and evaluate it with datasets.
 
 ## Install
@@ -292,7 +292,7 @@ for note_title, note_text in notes:
     corpus.ingest_note(note_text, title=note_title, tags=["memory"])
 
 backend = get_backend("scan")
-run = backend.build_run(corpus, recipe_name="Story demo", config={})
+run = backend.build_run(corpus, configuration_name="Story demo", config={})
 budget = QueryBudget(max_total_items=5, maximum_total_characters=2000, max_items_per_source=None)
 result = backend.query(
     corpus,
@@ -336,8 +336,8 @@ Example output:
     "maximum_total_characters": 2000,
     "max_items_per_source": null
   },
-  "run_id": "RUN_ID",
-  "recipe_id": "RECIPE_ID",
+  "snapshot_id": "RUN_ID",
+  "configuration_id": "RECIPE_ID",
   "backend_id": "scan",
   "generated_at": "2026-01-29T00:00:00.000000Z",
   "evidence": [
@@ -352,8 +352,8 @@ Example output:
       "span_start": null,
       "span_end": null,
       "stage": "scan",
-      "recipe_id": "RECIPE_ID",
-      "run_id": "RUN_ID",
+      "configuration_id": "RECIPE_ID",
+      "snapshot_id": "RUN_ID",
       "hash": null
     }
   ],
@@ -422,7 +422,7 @@ flowchart TB
 
       subgraph RowExtraction[Pluggable: extraction pipeline]
         direction TB
-        Catalog --> Extract[Extract pipeline] --> ExtractedText[Extracted text artifacts] --> ExtractionRun[Extraction run manifest]
+        Catalog --> Extract[Extract pipeline] --> ExtractedText[Extracted text artifacts] --> ExtractionRun[Extraction snapshot manifest]
       end
 
       subgraph RowRetrieval[Pluggable: retrieval backend]
@@ -484,7 +484,7 @@ From Python, the same flow is available through the Corpus class and backend int
 - Ingest notes with `Corpus.ingest_note`.
 - Ingest files or web addresses with `Corpus.ingest_source`.
 - List items with `Corpus.list_items`.
-- Build a retrieval run with `get_backend` and `backend.build_run`.
+- Build a retrieval snapshot with `get_backend` and `backend.build_run`.
 - Query a run with `backend.query`.
 - Evaluate with `evaluate_run`.
 
@@ -530,13 +530,13 @@ corpus/
     runs/
       extraction/
         pipeline/
-          <run id>/
+          <snapshot id>/
             manifest.json
             text/
               <item id>.txt
       retrieval/
         <backend id>/
-          <run id>/
+          <snapshot id>/
             manifest.json
 ```
 
@@ -552,7 +552,7 @@ For detailed documentation including configuration options, performance characte
 
 ## Retrieval documentation
 
-For the retrieval pipeline overview and run artifacts, see `docs/RETRIEVAL.md`. For retrieval quality upgrades
+For the retrieval pipeline overview and snapshot artifacts, see `docs/RETRIEVAL.md`. For retrieval quality upgrades
 (tuned lexical baseline, reranking, hybrid retrieval), see `docs/RETRIEVAL_QUALITY.md`. For evaluation workflows
 and dataset formats, see `docs/RETRIEVAL_EVALUATION.md`. For a runnable walkthrough, use the retrieval evaluation lab
 script (`scripts/retrieval_evaluation_lab.py`).
@@ -615,26 +615,26 @@ See `docs/TEXT_SLICE.md` for the utility API and examples.
 
 Biblicus can run analysis pipelines on extracted text without changing the raw corpus. Profiling and topic modeling
 are the first analysis backends. Profiling summarizes corpus composition and extraction coverage. Topic modeling reads
-an extraction run, optionally applies an LLM-driven extraction pass, applies lexical processing, runs BERTopic, and
+an extraction snapshot, optionally applies an LLM-driven extraction pass, applies lexical processing, runs BERTopic, and
 optionally applies an LLM fine-tuning pass to label topics. The output is structured JavaScript Object Notation.
 
 See `docs/ANALYSIS.md` for the analysis pipeline overview, `docs/PROFILING.md` for profiling, and
 `docs/TOPIC_MODELING.md` for topic modeling details.
 
-Run a topic analysis using a recipe file:
+Run a topic analysis using a configuration file:
 
 ```
-biblicus analyze topics --corpus corpora/example --recipe recipes/topic-modeling.yml --extraction-run pipeline:<run_id>
+biblicus analyze topics --corpus corpora/example --configuration configurations/topic-modeling.yml --extraction-run pipeline:<snapshot_id>
 ```
 
-If `--extraction-run` is omitted, Biblicus uses the most recent extraction run and emits a warning about
+If `--extraction-run` is omitted, Biblicus uses the most recent extraction snapshot and emits a warning about
 reproducibility. The analysis output is stored under:
 
 ```
-.biblicus/runs/analysis/topic-modeling/<run_id>/output.json
+.biblicus/runs/analysis/topic-modeling/<snapshot_id>/output.json
 ```
 
-Minimal recipe example:
+Minimal configuration example:
 
 ```yaml
 schema_version: 1
@@ -659,7 +659,7 @@ llm_fine_tuning:
 ```
 
 LLM extraction and fine-tuning require `biblicus[openai]` and a configured OpenAI API key.
-Recipe files are validated strictly against the topic modeling schema, so type mismatches or unknown fields are errors.
+Configuration files are validated strictly against the topic modeling schema, so type mismatches or unknown fields are errors.
 AG News integration runs require `biblicus[datasets]` in addition to `biblicus[topic-modeling]`.
 
 For a repeatable, real-world integration run that downloads AG News and executes topic modeling, use:
@@ -712,6 +712,15 @@ Build the documentation:
 python -m sphinx -b html docs docs/_build/html
 ```
 
+Preview the documentation locally:
+
+```
+cd docs/_build/html
+python -m http.server
+```
+
+Open `http://localhost:8000` in your browser.
+
 ## License
 
 License terms are in `LICENSE`.

biblicus-1.1.0.dist-info/RECORD

@@ -0,0 +1,91 @@
+biblicus/__init__.py,sha256=O9FlaC1aaafCfDoI3sIsbtUsjNKJpBI6sP-RTp_kCaI,1013
+biblicus/__main__.py,sha256=ipfkUoTlocVnrQDM69C7TeBqQxmHVeiWMRaT3G9rtnk,117
+biblicus/chunking.py,sha256=GdJr0skAAI0Su99mr7dXqCgR7eJ0sJu8n2XesVGyddY,13206
+biblicus/cli.py,sha256=GN7L0-s0k9tAj_lthvBrJlfo_DG9y53vYc6k_IhSea0,45797
+biblicus/configuration.py,sha256=JzQU-2pzO4hY7pBw8J79Ci0Glc9cvh4KrRvzSMK2d5w,4329
+biblicus/constants.py,sha256=VVjfZvdmoiCNsiQv0JVI-cA6JKXWUsvGL_IjnTxlEI8,386
+biblicus/context.py,sha256=I7L86ag2AbNr_QgiP5YSt1uwwULGx1cH73eR2nE9T3g,10842
+biblicus/corpus.py,sha256=D9O1Z8lQ7yFNXQDkaKR9fSTRDMSwtrYTGavh_GM7Eww,60374
+biblicus/crawl.py,sha256=n8rXBMnziBK9vtKQQCXYOpBzqsPCswj2PzVJUb370KY,6250
+biblicus/embedding_providers.py,sha256=phWEsq1vryyTFRRs6uZ0sx9FhrqWIkDsS3I52I64zqM,3839
+biblicus/errors.py,sha256=7fAGJbe_pCD8ygnfbTn6bNRV6pam0Vx3xjIpLrxrucg,1382
+biblicus/evaluation.py,sha256=XnQKPbUcUBnELllh7cNEzvTK8EKU1Ub0q3u_sIhXB5E,8372
+biblicus/evidence_processing.py,sha256=sJe6T1nLxvU0xs9yMH8JZZS19zHXMR-Fpr5lWi5ndUM,6120
+biblicus/extraction.py,sha256=YiJqLWY3mglYokSJqA8-oIxpFBPW4Hz0TEeeNp0PtWA,20581
+biblicus/extraction_evaluation.py,sha256=kFbyKcHzZK_z0OgCmQ3Olj55zgGoxin0Ir3dUA50TLI,10641
+biblicus/frontmatter.py,sha256=uFC4iIrgpnTDiP1gvAnT_CbFYdNuUVtETX7tZ3a9g-Y,2517
+biblicus/hook_logging.py,sha256=IMvde-JhVWrx9tNz3eDJ1CY_rr5Sj7DZ2YNomYCZbz0,5366
+biblicus/hook_manager.py,sha256=ZCAkE5wLvn4lnQz8jho_o0HGEC9KdQd9qitkAEUQRcw,6997
+biblicus/hooks.py,sha256=-ZcKZ4scK9ctas_PcseOmJJOLCkwxpnIxrACcz1qUus,7907
+biblicus/ignore.py,sha256=fyjt34E6tWNNrm1FseOhgH2MgryyVBQVzxhKL5s4aio,1800
+biblicus/inference.py,sha256=_k00AIPoXD2lruiTB-JUagtY4f_WKcdzA3axwiq1tck,3512
+biblicus/knowledge_base.py,sha256=pDZQlihjMB7AF61LccVG21rWEAjisgRtkEcn-dymZTM,6915
+biblicus/models.py,sha256=UlaqdvdqPZHd2__4Gcd4pryA_DBVgSiv86uI2AYD8Ag,16990
+biblicus/retrieval.py,sha256=9RA3KGw43dBOD1EFZwt9sqcVf334UtXb1qNHUqYW6As,4646
+biblicus/sources.py,sha256=FNwW1FWts0jxWIL3AHon7D6c5ZatyG9AGFqzn1Id5mE,8504
+biblicus/time.py,sha256=3BSKOSo7R10K-0Dzrbdtl3fh5_yShTYqfdlKvvdkx7M,485
+biblicus/uris.py,sha256=xXD77lqsT9NxbyzI1spX9Y5a3-U6sLYMnpeSAV7g-nM,2013
+biblicus/user_config.py,sha256=UXUYBNUN4FR37ggZGJG1wv3K8XzsMR8pXW1T18lrivw,6495
+biblicus/_vendor/dotyaml/__init__.py,sha256=OVv6IsuCvsjaUznLzuit4UbSLVg4TiTVm9cOPY1Y2Cs,409
+biblicus/_vendor/dotyaml/interpolation.py,sha256=FVUkdQr_KbXjoFPvGTv6I5v0X5iZkJe5yhZtYKRbYzI,1991
+biblicus/_vendor/dotyaml/loader.py,sha256=vFfnhbvHYYyOKzl5iq2FH97GSHH2GvEHmGiPnE0g0kA,6954
+biblicus/_vendor/dotyaml/transformer.py,sha256=RWNrm_KAsanG409HEIWquTH9i_jz-ZFK9fM86emXeF4,3724
+biblicus/ai/__init__.py,sha256=HY8PKhqRLIDYJYlL9A2JjqKxQaujITNLYgIytNUhnrU,1161
+biblicus/ai/embeddings.py,sha256=n2xlonZOHcmDrP1XMhGcja5Hzr8r87PF-IecH-Yhu98,3703
+biblicus/ai/llm.py,sha256=g724_UAxmicB_W-Z7Uu9SRsI9-aVNZUlYIjvnlE17VE,4712
+biblicus/ai/models.py,sha256=6newnT0NJf3uf9FvWXVC-9Gkk5xRB-PjXDZpeBHA04Y,7857
+biblicus/analysis/__init__.py,sha256=d1q11tEx3JkrOPMaiGMNCHhN9tCOTr_QpQP-tI1J2Wk,1389
+biblicus/analysis/base.py,sha256=HErFLn3gv1qf9ckAUxbolHF2k9sJDNZjPjdboCMhyBE,1349
+biblicus/analysis/markov.py,sha256=pLtKvt4gtsqa1CASizh8bBJ4CQW2e0wGaQ-BgdP7Pfg,63766
+biblicus/analysis/models.py,sha256=dYnm5gwUzTk5HvrHZjQx4vug_TZLnXU9qN6CLIRyLng,56495
+biblicus/analysis/profiling.py,sha256=IynvrgcopqFj6lMUPHS1prwd0FxN8FzIa5p3JInDFCc,11185
+biblicus/analysis/schema.py,sha256=MCiAQJmijVk8iM8rOUYbzyaDwsMR-Oo86iZU5NCbDMM,435
+biblicus/analysis/topic_modeling.py,sha256=dsNHuqxcEoCKO_8aDAM9yEOa0kWCjPWS2NvcQayIyXQ,22623
+biblicus/context_engine/__init__.py,sha256=cIJWTUwOewW1x13a2n0YKfr4-XU0IwlVdAH_0pckfKk,1337
+biblicus/context_engine/assembler.py,sha256=E7VPdqUJ9peZUoonM0Ooa1wsaklFOuLCt2IH9nFxAfM,44260
+biblicus/context_engine/compaction.py,sha256=2bLaCpT48d1TL7vt9rrcRCgfdHeWWp9LX85Cgij12o0,2921
+biblicus/context_engine/models.py,sha256=jesVd83ZQcatO-7yNlzwKkactSQ-e1znYuWof4rxVFg,12762
+biblicus/context_engine/retrieval.py,sha256=A0w6C5uPrDY_aeGeirRkSGr6I-gU0U0cY6ElvrLhe0Q,4425
+biblicus/extractors/__init__.py,sha256=ci3oldbdQZ8meAfHccM48CqQtZsPSRg3HkPrBSZF15M,2673
+biblicus/extractors/base.py,sha256=ka-nz_1zHPr4TS9sU4JfOoY-PJh7lbHPBOEBrbQFGSc,2171
+biblicus/extractors/deepgram_stt.py,sha256=xx_zrROGRHotF5aht23Qey9dpCnU3KHZ0unwa933Pto,6358
+biblicus/extractors/docling_granite_text.py,sha256=iDHWZVgqZd86Q3Zu-fcdCq7ia00xTpbFbTU7JbDNZ38,6953
+biblicus/extractors/docling_smol_text.py,sha256=qI7m93Odrjmob0RW-Yvnt5Ck4AgFcgdVjwatLQA5krI,6885
+biblicus/extractors/markitdown_text.py,sha256=kYixZbVxaIyeWtpezocnrSxC3z_9KqWuBzeK8sI4s1o,4567
+biblicus/extractors/metadata_text.py,sha256=7FbEPp0K1mXc7FH1_c0KhPhPexF9U6eLd3TVY1vTp1s,3537
+biblicus/extractors/openai_stt.py,sha256=d2CaVhxapfkXaeI_QZcoXwdVm5Bj5YwGLdmTZDcqgTc,7197
+biblicus/extractors/paddleocr_vl_text.py,sha256=2xoHA1Jviw8zzeBvHBI74Lkx4SX_vSarCe3wxvYf6c4,11794
+biblicus/extractors/pass_through_text.py,sha256=DNxkCwpH2bbXjPGPEQwsx8kfqXi6rIxXNY_n3TU2-WI,2777
+biblicus/extractors/pdf_text.py,sha256=YtUphgLVxyWJXew6ZsJ8wBRh67Y5ri4ZTRlMmq3g1Bk,3255
+biblicus/extractors/pipeline.py,sha256=qdlBBSUVNdg2V4izHacv_8a2DikCGlVMAdpkZkzNvyY,3288
+biblicus/extractors/rapidocr_text.py,sha256=5adSCiOmyHiCgX3jBMcl1OiQlGzYLxmgJQzo9GHSecs,4791
+biblicus/extractors/select_longest_text.py,sha256=wRveXAfYLdj7CpGuo4RoD7zE6SIfylRCbv40z2azO0k,3702
+biblicus/extractors/select_override.py,sha256=gSpffFmn1ux9pGtFvHD5Uu_LO8TmmJC4L_mvjehiSec,4014
+biblicus/extractors/select_smart_override.py,sha256=-sLMnNoeXbCB3dO9zflQq324eHuLbd6hpveSwduXP-U,6763
+biblicus/extractors/select_text.py,sha256=w0ATmDy3tWWbOObzW87jGZuHbgXllUhotX5XyySLs-o,3395
+biblicus/extractors/unstructured_text.py,sha256=WXr_fu4KQ0NODkbb05e4HrAX-trOWRKiDOmznh9_pLI,3579
+biblicus/retrievers/__init__.py,sha256=LOxhUYgph1sPAeY6PmSmXH4Os4bIGGOtw88iOdI9S2k,1704
+biblicus/retrievers/base.py,sha256=DSf5Ve5IFeunIyV9zt7T1vEUvSkJWO4iBj96co5F0Qo,1891
+biblicus/retrievers/embedding_index_common.py,sha256=63_dUds-yIALhq2L9_0oDNpoh-_h7v2j1kexbdVw1-o,11756
+biblicus/retrievers/embedding_index_file.py,sha256=mvtXqRX-_eQpi9bRxQ2yqFxY26YhP8Vn2WGcoWVtMtc,10668
+biblicus/retrievers/embedding_index_inmemory.py,sha256=8csrdjCGkkR7DgvmLZ72oD1gm4duWWUnxJsjw6nLicU,10525
+biblicus/retrievers/hybrid.py,sha256=kaH-kIi4wxYyUWnKNFT7UNBbHFkRtcGlwjjiJpx-TJY,11789
+biblicus/retrievers/scan.py,sha256=ccDGVnqBS9a2ymKeBEXdfJz8XLahsBeYWYyYXQcg2KQ,13147
+biblicus/retrievers/sqlite_full_text_search.py,sha256=7rzYfzpRhPbsKuXjXi8x2-rmq8-z1em3amUF9UPAomI,25392
+biblicus/retrievers/tf_vector.py,sha256=rkcRG1GU5S_3t8GRbQTBThITj-eHT5fs1dyVzXPLg8w,15776
+biblicus/text/__init__.py,sha256=MiaGAY7xWlUCeBzDzNz6pJnSMiU_Ge5EmlSiEzhqTRo,947
+biblicus/text/annotate.py,sha256=asmpj3_s_t8hl6stEg99apmqxAhDTkoPzHhZNggYE3Y,8355
+biblicus/text/extract.py,sha256=pdnUiZWtfCUj7kZK5zhd-tjqokgmhYYheWhyN3iShRU,7669
+biblicus/text/link.py,sha256=2IdOi3WgyBKPFau0bpS1eToV1q2v_6wq5RK5_P_qUDg,20448
+biblicus/text/markup.py,sha256=8jj9aX03HiZTOWdPs_VC4JLpQ7TlPHgGuXj_QUQIHVw,6265
+biblicus/text/models.py,sha256=REp6RowUWFdV-6y437JENP7XtGKt57BOvVtF91KmUqI,10853
+biblicus/text/prompts.py,sha256=9dx1cWpJb6oBY4AhDHxlkRUYs7DfbySH0gb-uBTNvtk,7567
+biblicus/text/redact.py,sha256=tkDRmA0VvOZwMryEmBPLEHf3Z6VHJkkaWjBaNIMyGZ0,8415
+biblicus/text/slice.py,sha256=dlHxGO8c5P8BszXGwlNQoQ-cyWjJf6PfS1LUBJXXGEE,5762
+biblicus/text/tool_loop.py,sha256=dFeIEcCUA-yR8GMqsJ_n4007fHVmn9zK2hhlm6NlWyg,14161
+biblicus-1.1.0.dist-info/licenses/LICENSE,sha256=lw44GXFG_Q0fS8m5VoEvv_xtdBXK26pBcbSPUCXee_Q,1078
+biblicus-1.1.0.dist-info/METADATA,sha256=8hRnC6tlf8crtWxf6FPbGANZH9lxL6kiAtOtcxqJ3Ig,31202
+biblicus-1.1.0.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
+biblicus-1.1.0.dist-info/entry_points.txt,sha256=BZmO4H8Uz00fyi1RAFryOCGfZgX7eHWkY2NE-G54U5A,47
+biblicus-1.1.0.dist-info/top_level.txt,sha256=sUD_XVZwDxZ29-FBv1MknTGh4mgDXznGuP28KJY_WKc,9
+biblicus-1.1.0.dist-info/RECORD,,

biblicus/backends/__init__.py

@@ -1,50 +0,0 @@
-"""
-Backend registry for Biblicus retrieval engines.
-"""
-
-from __future__ import annotations
-
-from typing import Dict, Type
-
-from .base import RetrievalBackend
-from .embedding_index_file import EmbeddingIndexFileBackend
-from .embedding_index_inmemory import EmbeddingIndexInMemoryBackend
-from .hybrid import HybridBackend
-from .scan import ScanBackend
-from .sqlite_full_text_search import SqliteFullTextSearchBackend
-from .tf_vector import TfVectorBackend
-
-
-def available_backends() -> Dict[str, Type[RetrievalBackend]]:
-    """
-    Return the registered retrieval backends.
-
-    :return: Mapping of backend identifiers to backend classes.
-    :rtype: dict[str, Type[RetrievalBackend]]
-    """
-    return {
-        EmbeddingIndexFileBackend.backend_id: EmbeddingIndexFileBackend,
-        EmbeddingIndexInMemoryBackend.backend_id: EmbeddingIndexInMemoryBackend,
-        HybridBackend.backend_id: HybridBackend,
-        ScanBackend.backend_id: ScanBackend,
-        SqliteFullTextSearchBackend.backend_id: SqliteFullTextSearchBackend,
-        TfVectorBackend.backend_id: TfVectorBackend,
-    }
-
-
-def get_backend(backend_id: str) -> RetrievalBackend:
-    """
-    Instantiate a retrieval backend by identifier.
-
-    :param backend_id: Backend identifier.
-    :type backend_id: str
-    :return: Backend instance.
-    :rtype: RetrievalBackend
-    :raises KeyError: If the backend identifier is unknown.
-    """
-    registry = available_backends()
-    backend_class = registry.get(backend_id)
-    if backend_class is None:
-        known = ", ".join(sorted(registry))
-        raise KeyError(f"Unknown backend '{backend_id}'. Known backends: {known}")
-    return backend_class()

biblicus/backends/base.py

@@ -1,65 +0,0 @@
-"""
-Backend interface for Biblicus retrieval engines.
-"""
-
-from __future__ import annotations
-
-from abc import ABC, abstractmethod
-from typing import Dict
-
-from ..corpus import Corpus
-from ..models import QueryBudget, RetrievalResult, RetrievalRun
-
-
-class RetrievalBackend(ABC):
-    """
-    Abstract interface for retrieval backends.
-
-    :ivar backend_id: Identifier string for the backend.
-    :vartype backend_id: str
-    """
-
-    backend_id: str
-
-    @abstractmethod
-    def build_run(
-        self, corpus: Corpus, *, recipe_name: str, config: Dict[str, object]
-    ) -> RetrievalRun:
-        """
-        Build or register a retrieval run for the backend.
-
-        :param corpus: Corpus to build against.
-        :type corpus: Corpus
-        :param recipe_name: Human name for the recipe.
-        :type recipe_name: str
-        :param config: Backend-specific configuration values.
-        :type config: dict[str, object]
-        :return: Run manifest describing the build.
-        :rtype: RetrievalRun
-        """
-        raise NotImplementedError
-
-    @abstractmethod
-    def query(
-        self,
-        corpus: Corpus,
-        *,
-        run: RetrievalRun,
-        query_text: str,
-        budget: QueryBudget,
-    ) -> RetrievalResult:
-        """
-        Run a retrieval query against a backend.
-
-        :param corpus: Corpus associated with the run.
-        :type corpus: Corpus
-        :param run: Run manifest to use for querying.
-        :type run: RetrievalRun
-        :param query_text: Query text to execute.
-        :type query_text: str
-        :param budget: Evidence selection budget.
-        :type budget: QueryBudget
-        :return: Retrieval results containing evidence.
-        :rtype: RetrievalResult
-        """
-        raise NotImplementedError

biblicus/backends/hybrid.py

@@ -1,292 +0,0 @@
-"""
-Hybrid retrieval backend combining lexical and vector results.
-"""
-
-from __future__ import annotations
-
-from typing import Dict, List, Optional
-
-from pydantic import BaseModel, ConfigDict, Field, model_validator
-
-from ..corpus import Corpus
-from ..models import Evidence, QueryBudget, RetrievalResult, RetrievalRun
-from ..retrieval import apply_budget, create_recipe_manifest, create_run_manifest
-from ..time import utc_now_iso
-
-
-class HybridRecipeConfig(BaseModel):
-    """
-    Configuration for hybrid retrieval fusion.
-
-    :ivar lexical_backend: Backend identifier for lexical retrieval.
-    :vartype lexical_backend: str
-    :ivar embedding_backend: Backend identifier for embedding retrieval.
-    :vartype embedding_backend: str
-    :ivar lexical_weight: Weight for lexical scores.
-    :vartype lexical_weight: float
-    :ivar embedding_weight: Weight for embedding scores.
-    :vartype embedding_weight: float
-    :ivar lexical_config: Optional lexical backend configuration.
-    :vartype lexical_config: dict[str, object]
-    :ivar embedding_config: Optional embedding backend configuration.
-    :vartype embedding_config: dict[str, object]
-    """
-
-    model_config = ConfigDict(extra="forbid")
-
-    lexical_backend: str = Field(default="sqlite-full-text-search", min_length=1)
-    embedding_backend: str = Field(default="tf-vector", min_length=1)
-    lexical_weight: float = Field(default=0.5, ge=0, le=1)
-    embedding_weight: float = Field(default=0.5, ge=0, le=1)
-    lexical_config: Dict[str, object] = Field(default_factory=dict)
-    embedding_config: Dict[str, object] = Field(default_factory=dict)
-
-    @model_validator(mode="after")
-    def _validate_weights(self) -> "HybridRecipeConfig":
-        if abs((self.lexical_weight + self.embedding_weight) - 1.0) > 1e-6:
-            raise ValueError("weights must sum to 1")
-        return self
-
-
-class HybridBackend:
-    """
-    Hybrid backend that fuses lexical and embedding retrieval.
-
-    :ivar backend_id: Backend identifier.
-    :vartype backend_id: str
-    """
-
-    backend_id = "hybrid"
-
-    def build_run(
-        self, corpus: Corpus, *, recipe_name: str, config: Dict[str, object]
-    ) -> RetrievalRun:
-        """
-        Build or register a hybrid retrieval run.
-
-        :param corpus: Corpus to build against.
-        :type corpus: Corpus
-        :param recipe_name: Human-readable recipe name.
-        :type recipe_name: str
-        :param config: Backend-specific configuration values.
-        :type config: dict[str, object]
-        :return: Run manifest describing the build.
-        :rtype: RetrievalRun
-        """
-        recipe_config = HybridRecipeConfig.model_validate(config)
-        _ensure_backend_supported(recipe_config)
-        lexical_backend = _resolve_backend(recipe_config.lexical_backend)
-        embedding_backend = _resolve_backend(recipe_config.embedding_backend)
-        lexical_run = lexical_backend.build_run(
-            corpus, recipe_name=f"{recipe_name}-lexical", config=recipe_config.lexical_config
-        )
-        embedding_run = embedding_backend.build_run(
-            corpus, recipe_name=f"{recipe_name}-embedding", config=recipe_config.embedding_config
-        )
-        recipe = create_recipe_manifest(
-            backend_id=self.backend_id,
-            name=recipe_name,
-            config=recipe_config.model_dump(),
-        )
-        stats = {
-            "lexical_run_id": lexical_run.run_id,
-            "embedding_run_id": embedding_run.run_id,
-        }
-        run = create_run_manifest(corpus, recipe=recipe, stats=stats, artifact_paths=[])
-        corpus.write_run(run)
-        return run
-
-    def query(
-        self,
-        corpus: Corpus,
-        *,
-        run: RetrievalRun,
-        query_text: str,
-        budget: QueryBudget,
-    ) -> RetrievalResult:
-        """
-        Query using both lexical and embedding backends and fuse scores.
-
-        :param corpus: Corpus associated with the run.
-        :type corpus: Corpus
-        :param run: Run manifest to use for querying.
-        :type run: RetrievalRun
-        :param query_text: Query text to execute.
-        :type query_text: str
-        :param budget: Evidence selection budget.
-        :type budget: QueryBudget
-        :return: Retrieval results containing evidence.
-        :rtype: RetrievalResult
-        """
-        recipe_config = HybridRecipeConfig.model_validate(run.recipe.config)
-        _ensure_backend_supported(recipe_config)
-        lexical_backend = _resolve_backend(recipe_config.lexical_backend)
-        embedding_backend = _resolve_backend(recipe_config.embedding_backend)
-        lexical_run_id = run.stats.get("lexical_run_id")
-        embedding_run_id = run.stats.get("embedding_run_id")
-        if not lexical_run_id or not embedding_run_id:
-            raise ValueError("Hybrid run missing lexical or embedding run identifiers")
-        lexical_run = corpus.load_run(str(lexical_run_id))
-        embedding_run = corpus.load_run(str(embedding_run_id))
-        component_budget = _expand_component_budget(budget)
-        lexical_result = lexical_backend.query(
-            corpus, run=lexical_run, query_text=query_text, budget=component_budget
-        )
-        embedding_result = embedding_backend.query(
-            corpus, run=embedding_run, query_text=query_text, budget=component_budget
-        )
-        candidates = _fuse_evidence(
-            lexical_result.evidence,
-            embedding_result.evidence,
-            lexical_weight=recipe_config.lexical_weight,
-            embedding_weight=recipe_config.embedding_weight,
-        )
-        sorted_candidates = sorted(
-            candidates,
-            key=lambda evidence_item: (-evidence_item.score, evidence_item.item_id),
-        )
-        ranked = [
-            evidence_item.model_copy(
-                update={
-                    "rank": index,
-                    "recipe_id": run.recipe.recipe_id,
-                    "run_id": run.run_id,
-                }
-            )
-            for index, evidence_item in enumerate(sorted_candidates, start=1)
-        ]
-        evidence = apply_budget(ranked, budget)
-        stats = {
-            "candidates": len(sorted_candidates),
-            "returned": len(evidence),
-            "fusion_weights": {
-                "lexical": recipe_config.lexical_weight,
-                "embedding": recipe_config.embedding_weight,
-            },
-        }
-        return RetrievalResult(
-            query_text=query_text,
-            budget=budget,
-            run_id=run.run_id,
-            recipe_id=run.recipe.recipe_id,
-            backend_id=self.backend_id,
-            generated_at=utc_now_iso(),
-            evidence=evidence,
-            stats=stats,
-        )
-
-
-def _ensure_backend_supported(recipe_config: HybridRecipeConfig) -> None:
-    """
-    Validate that hybrid backends do not reference the hybrid backend itself.
-
-    :param recipe_config: Parsed hybrid recipe configuration.
-    :type recipe_config: HybridRecipeConfig
-    :return: None.
-    :rtype: None
-    :raises ValueError: If hybrid is used as a component backend.
-    """
-    if recipe_config.lexical_backend == HybridBackend.backend_id:
-        raise ValueError("Hybrid backend cannot use itself as the lexical backend")
-    if recipe_config.embedding_backend == HybridBackend.backend_id:
-        raise ValueError("Hybrid backend cannot use itself as the embedding backend")
-
-
-def _resolve_backend(backend_id: str):
-    """
-    Resolve a backend by identifier.
-
-    :param backend_id: Backend identifier.
-    :type backend_id: str
-    :return: Backend instance.
-    :rtype: object
-    """
-    from . import get_backend
-
-    return get_backend(backend_id)
-
-
-def _expand_component_budget(budget: QueryBudget, *, multiplier: int = 5) -> QueryBudget:
-    """
-    Expand a final budget to collect more candidates for fusion.
-
-    :param budget: Final evidence budget.
-    :type budget: QueryBudget
-    :param multiplier: Candidate expansion multiplier.
-    :type multiplier: int
-    :return: Expanded budget for component backends.
-    :rtype: QueryBudget
-    """
-    maximum_total_characters = budget.maximum_total_characters
-    expanded_characters = (
-        maximum_total_characters * multiplier if maximum_total_characters is not None else None
-    )
-    expanded_max_items_per_source = (
-        budget.max_items_per_source * multiplier
-        if budget.max_items_per_source is not None
-        else None
-    )
-    requested_items = budget.max_total_items + budget.offset
-    return QueryBudget(
-        max_total_items=requested_items * multiplier,
-        offset=0,
-        maximum_total_characters=expanded_characters,
-        max_items_per_source=expanded_max_items_per_source,
-    )
-
-
-def _fuse_evidence(
-    lexical: List[Evidence],
-    embedding: List[Evidence],
-    *,
-    lexical_weight: float,
-    embedding_weight: float,
-) -> List[Evidence]:
-    """
-    Fuse lexical and embedding evidence lists into hybrid candidates.
-
-    :param lexical: Lexical evidence list.
-    :type lexical: list[Evidence]
-    :param embedding: Embedding evidence list.
-    :type embedding: list[Evidence]
-    :param lexical_weight: Lexical score weight.
-    :type lexical_weight: float
-    :param embedding_weight: Embedding score weight.
-    :type embedding_weight: float
-    :return: Hybrid evidence list.
-    :rtype: list[Evidence]
-    """
-    merged: Dict[str, Dict[str, Optional[Evidence]]] = {}
-    for evidence_item in lexical:
-        merged.setdefault(evidence_item.item_id, {})["lexical"] = evidence_item
-    for evidence_item in embedding:
-        merged.setdefault(evidence_item.item_id, {})["embedding"] = evidence_item
-
-    candidates: List[Evidence] = []
-    for item_id, sources in merged.items():
-        lexical_evidence = sources.get("lexical")
-        embedding_evidence = sources.get("embedding")
-        lexical_score = lexical_evidence.score if lexical_evidence else 0.0
-        embedding_score = embedding_evidence.score if embedding_evidence else 0.0
-        combined_score = (lexical_score * lexical_weight) + (embedding_score * embedding_weight)
-        base_evidence = lexical_evidence or embedding_evidence
-        candidates.append(
-            Evidence(
-                item_id=item_id,
-                source_uri=base_evidence.source_uri,
-                media_type=base_evidence.media_type,
-                score=combined_score,
-                rank=1,
-                text=base_evidence.text,
-                content_ref=base_evidence.content_ref,
-                span_start=base_evidence.span_start,
-                span_end=base_evidence.span_end,
-                stage="hybrid",
-                stage_scores={"lexical": lexical_score, "embedding": embedding_score},
-                recipe_id="",
-                run_id="",
-                metadata=base_evidence.metadata,
-                hash=base_evidence.hash,
-            )
-        )
-    return candidates