jekyll-client-search 0.3.5 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4bd0e3f73bbbf16d364c059483ad2ad40382232dda34ca1e8a0f83b0286fa605
-  data.tar.gz: 643dfe41f83ab07d100fa9f841e7b80f07ce1a8db5d5e3d2e5e43d4a29880a2e
+  metadata.gz: 749a1708fe34bce1624db0eb4d91ea58a569f2663f8d78b8200365188faf7ae4
+  data.tar.gz: 833b1c87a57b2f78a05f42103efef6f9aa2ec09ee0726cad734e0f28f268fc50
 SHA512:
-  metadata.gz: 2e0e6fbfde2eaf9695f31bc4f79980f5a0d5e88b299253fde5523faa137be6606c60fecf25de2d2ed4b0e3e05d08d2afc5a9033346dd64b25c3dbde8156a44b4
-  data.tar.gz: b2cdf16bbca7714c81b6ce4a8054f94fa654a30d3e30f5c926fde2f0f3c312cc28ed66d249f3c9dcd38f03cbd9ccb40bf437ce6180c13f867928afa16c6a1f30
+  metadata.gz: 169936c38c680b2f4ec657243909aaad0a902b6bbbc52e9c586d3d986651e1529fdfb2d2a9dcde69f997075b8c901c5178b7148d86f064242ee018a228b2e457
+  data.tar.gz: 3e75798e1b41dac7982ca34e83e61e8b28b03b347f6ca60b6e527570a72d4a69a9f5ddf6293197410c6935a699b54a7001b601145c57c7e09302e4f1edf4d426

data/CHANGELOG.md

@@ -1,6 +1,63 @@
 # Changelog
 
-## Unreleased
+## 0.4.0 — 2026-10-24
+
+- Dev-tooling bump: `jsdom` 30.1.3 (patch).
+
+### Added
+
+- `benchmarks/embedding_model_benchmark.rb`,
+  `benchmarks/related_model_benchmark.rb`, and
+  `benchmarks/related_quality_judge.rb` — measure Ollama embedding
+  latency per model on a real site, compare `related` output across
+  models, and score judged relevance with a blind LLM judge
+  (10 local embedding models benchmarked on a 152-document site).
+- `embeddinggemma-2` document/query prefixes (same prompt format as
+  `embeddinggemma`), `voyage-4-nano` prefixes (open-weight Voyage
+  model via community GGUF), and `qwen3-embedding` prefixes (official
+  instruct-format query prefix; empty document prefix).
+- README model-selection guidance: measured per-document latency table,
+  LLM-judged quality ratings, quality-per-cost value index, amortized
+  yearly cost, `qwen3-embedding` build-time-only notes, and the
+  `localhost` → `127.0.0.1` loopback gotcha.
+- Split index/related embedding models: `semantic.index_model` selects
+  the Ollama model whose vectors go into `search-index.json` (must match
+  the browser query embedder); `related.index_model` selects the model
+  used for related-article analysis. Both default to `embedding.model`;
+  equal models share one embedding pass, divergent models embed each
+  document twice into separate cached vector fields.
+  `semantic.document_prefix` / `semantic.query_prefix` override the
+  index model's prefixes. Related-model vectors are cached separately
+  in `.jekyll-client-search-related-cache.json` and never written to
+  the index.
+- `embeddinggemma-2` (`:270m`, `:570m`, untagged) maps to
+  `onnx-community/embeddinggemma-2-ONNX` for the transformers.js query
+  embedder. `test/onnx-parity.test.js` is an opt-in check
+  (`ONNX_PARITY=1`, requires `@huggingface/transformers` and a running
+  Ollama) that compares ONNX and Ollama vectors and rankings;
+  `embeddinggemma:300m` remains the verified mapping.
+
+### Changed
+
+- Default `embedding.model` is now `embeddinggemma-2:570m` (was
+  `embeddinggemma:300m`) — the judged-quality leader among license-free
+  local models (111 vs 96) at ~2.5× the per-document latency; its text
+  vectors are byte-identical to `embeddinggemma-2:270m`. The
+  transformers.js query embedder maps it to
+  `onnx-community/embeddinggemma-2-ONNX`; sites that prefer the verified
+  browser mapping can set `semantic.index_model: embeddinggemma:300m`
+  while related analysis stays on the default model.
+- CI: Semgrep moved from an inline `pip3 install semgrep==1.176.0` pin to hashed `.github/requirements-ci.txt` (compiled from `requirements-ci.in`, Python 3.12), installed with `--require-hashes`. Bumped Semgrep to 1.180.0; the version is now Dependabot-visible via a new `pip` ecosystem entry for `/.github`.
+
+### Fixed
+
+- A `localhost` `embedding.base_url` (the default) is resolved once to
+  the loopback literal that accepts a TCP connection — `127.0.0.1` or
+  `[::1]` — because `ollama-ruby`/Excon does not fall back across
+  address families and Ollama commonly listens on IPv4 loopback only.
+  Embedding calls that died with `Excon::Error::Socket "Broken pipe"`
+  when `localhost` resolved to `::1` first now connect on both
+  IPv4-only and IPv6-only servers.
 
 ## 0.3.5 — 2026-10-06
 

data/README.developer.md

@@ -237,9 +237,10 @@ To run the full integration tests that call a real Ollama server:
 
 1. Install [Ollama](https://ollama.ai/)
 
-2. Pull the required model:
+2. Pull the required models:
 
    ```bash
+   ollama pull embeddinggemma-2:570m
    ollama pull embeddinggemma:300m
    ```
 
@@ -250,6 +251,16 @@ To run the full integration tests that call a real Ollama server:
    OLLAMA_INTEGRATION=1 bundle exec rspec spec/llm_injection_spec.rb
    ```
 
+To verify cross-runtime parity between the transformers.js ONNX twin and
+the Ollama build model (opt-in — needs the model downloaded once):
+
+```bash
+npm i -D @huggingface/transformers@3.8.1
+ONNX_PARITY=1 node --test test/onnx-parity.test.js
+```
+
+Set `OLLAMA_MODEL`, `ONNX_MODEL`, or `ONNX_DTYPE` to check other mappings.
+
 These tests:
 
 - Build the full 80-post index with real Ollama embeddings
@@ -450,7 +461,8 @@ This check also runs in the release workflow — the publish will fail if
    ```
 
    The Ollama integration tests require a running [Ollama](https://ollama.ai/)
-   server with the `embeddinggemma:300m` model pulled (`ollama pull
+   server with the `embeddinggemma-2:570m` and `embeddinggemma:300m` models
+   pulled (`ollama pull embeddinggemma-2:570m` and `ollama pull
    embeddinggemma:300m`). Do not publish a release until the Ollama
    integration tests pass locally — they are skipped by default and will
    not block CI, so they must be run manually before each release.

data/README.md

@@ -15,7 +15,6 @@
 [![Semgrep](https://img.shields.io/badge/semgrep-scanning-brightgreen.svg)](https://semgrep.dev)
 [![CodeFactor](https://www.codefactor.io/repository/github/gundestrup/jekyll-client-search/badge)](https://www.codefactor.io/repository/github/gundestrup/jekyll-client-search)
 [![SonarQube](https://sonarcloud.io/api/project_badges/measure?project=gundestrup_jekyll-client-search&metric=alert_status)](https://sonarcloud.io/project/overview?id=gundestrup_jekyll-client-search)
-[![codecov](https://codecov.io/gh/gundestrup/jekyll-client-search/branch/main/graph/badge.svg)](https://codecov.io/gh/gundestrup/jekyll-client-search)
 
 A Jekyll plugin that generates a JSON document index for client-side search.
 Supports lexical search ([MiniSearch](https://lucaong.github.io/minisearch/),
@@ -450,15 +449,18 @@ client_search:
     output: search-relations.json
     minimum_similarity: 0.55
     max_items: null
+    index_model: null             # default: embedding.model
   embedding:
     enabled: false
-    model: embeddinggemma:300m
+    model: embeddinggemma-2:570m
     base_url: http://localhost:11434
     connect_timeout: 5
     read_timeout: 120
     fail_on_error: true
     query_embedder:
       type: transformers
+  semantic:
+    index_model: null             # default: embedding.model
 ```
 
 All options:
@@ -487,9 +489,13 @@ All options:
 | `related.semantic` | `true` | Include vector relations when document embeddings are available. |
 | `related.minimum_similarity` | `0.55` | Cosine cutoff for semantic relations; no fixed relation count. |
 | `related.max_items` | `null` | Optional safety cap after cutoff; `null` keeps every matching relation. |
+| `related.index_model` | `embedding.model` | Ollama model for related-article similarity. May differ from the index model since relations never reach the browser. |
+| `semantic.index_model` | `embedding.model` | Ollama model whose vectors go into the index; must match the browser query model. |
+| `semantic.document_prefix` | Model-specific | Document prefix for `semantic.index_model` (defaults to `embedding.document_prefix` when the models match). |
+| `semantic.query_prefix` | Model-specific | Query prefix for `semantic.index_model` (defaults to `embedding.query_prefix` when the models match). |
 | `embedding.enabled` | `false` | Generate embedding vectors at build time via Ollama. |
-| `embedding.model` | `embeddinggemma:300m` | Ollama model name for embedding generation. |
-| `embedding.base_url` | `http://localhost:11434` | Ollama server URL. |
+| `embedding.model` | `embeddinggemma-2:570m` | Ollama model name for embedding generation. |
+| `embedding.base_url` | `http://localhost:11434` | Ollama server URL. `localhost` is resolved once to the loopback literal (`127.0.0.1` or `[::1]`) that accepts a TCP connection, so IPv4-only and IPv6-only servers both work. |
 | `embedding.connect_timeout` | `5` | Ollama connection timeout in seconds. |
 | `embedding.read_timeout` | `120` | Ollama response timeout in seconds. |
 | `embedding.fail_on_error` | `true` | Fail the build when an embedding cannot be generated. |
@@ -610,7 +616,7 @@ Jekyll build time using a local [Ollama](https://ollama.ai/) server and the
 1. Install [Ollama](https://ollama.ai/) and pull an embedding model:
 
    ```bash
-   ollama pull embeddinggemma:300m
+   ollama pull embeddinggemma-2:570m
    ```
 
 2. Add `ollama-ruby` to your site's Gemfile (optional dependency):
@@ -626,10 +632,18 @@ Jekyll build time using a local [Ollama](https://ollama.ai/) server and the
      engine: semantic
      embedding:
        enabled: true
-       model: embeddinggemma:300m
        base_url: http://localhost:11434
    ```
 
+   With `query_embedder.type: transformers` (the default), the build
+   model needs a packaged browser ONNX mapping — `embeddinggemma` and
+   `embeddinggemma-2` models have one (see
+   [Browser query models](#browser-query-models)); any other
+   `embedding.model` requires an explicit `query_embedder.model` or
+   `query_embedder.type: ollama_api`. For build-time-only embeddings — a
+   lexical engine with `related` analysis — any Ollama model works
+   as-is.
+
 ### How it works
 
 - At build time, the `OllamaEmbeddingAdapter` sends each document's text
@@ -642,34 +656,175 @@ Jekyll build time using a local [Ollama](https://ollama.ai/) server and the
 - A content-hash cache (`.jekyll-client-search-cache.json` in the site
   source) tracks which documents have already been embedded. Unchanged
   documents reuse the cached embedding — only new or modified documents
-  are sent to the model.
+  are sent to the model. When the index and related models differ, the
+  related vectors are cached separately in
+  `.jekyll-client-search-related-cache.json`.
 - Cache entries include the provider, model, endpoint, and cache schema. A
   model or embedding configuration change automatically invalidates old
   vectors and re-embeds the documents.
-- Cache writes are atomic. The cache file is git-ignored and safe to delete
-  (it will be rebuilt on the next `jekyll build`).
+- Cache writes are atomic. The cache files are git-ignored and safe to
+  delete (they will be rebuilt on the next `jekyll build`).
 - By default, an embedding failure fails the build rather than silently
   producing an unusable semantic index. Set `embedding.fail_on_error: false`
   only when warning-only behavior is intentional.
 
 ### Choosing a model
 
-Any Ollama-compatible embedding model works. Common choices:
-
-| Model | Dimensions | Good for |
-| --- | --- | --- |
-| `embeddinggemma:300m` | 768 | Default, multilingual, good quality |
-| `nomic-embed-text` | 768 | Multilingual, good quality |
-| `bge-m3` | 1024 | Multilingual, high quality |
-| `all-minilm` | 384 | English, fast, lightweight |
+Any Ollama-compatible embedding model works. Hosted embedding APIs (OpenAI
+`text-embedding-3-small`, Gemini Embedding, the Voyage API) are not
+supported — the adapter only speaks Ollama's `/api/embed` protocol, and
+local embeddings keep the corpus on your machine. Open-weight models that
+ship GGUF builds (e.g. `voyage-4-nano`) can still be tested through a
+local Ollama server.
+
+Measured on a 152-document site (Apple Silicon, sequential `/api/embed`
+calls, GPU-resident models, `related.minimum_similarity: 0.55`):
+
+| Model | Dims | ~ms/doc | Full corpus | Quality* | Value† | ~Cost/yr‡ | Notes |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| `embeddinggemma-2:570m` | 768 | ~246 | ~37 s | 111 | 0.50 | ~63 s | **Default** — best judged relatedness short of non-commercial jina; text vectors byte-identical to 270m; ONNX twin packaged (verify parity) |
+| `embeddinggemma-2:270m` | 768 | ~260 | ~40 s | 111 | 0.47 | ~67 s | Same text vectors as 570m; shares its ONNX twin (verify parity) |
+| `embeddinggemma:300m` | 768 | ~105 | ~16 s | 96 | 1.00 | ~27 s | Fast; verified transformers.js browser mapping |
+| `nomic-embed-text` | 768 | ~100 | ~15 s | 105 | **1.16** | ~25 s | Fast; best measured value |
+| `voyage-4-nano` | 1024 | ~115 | ~17 s | 107 | 1.03 | ~29 s | Open-weight (Apache 2.0); community GGUF at its 1024-dim base — official 2048 dim (post-hoc linear head) retested, judged equivalent (103) |
+| `bge-m3` | 1024 | ~305 | ~46 s | 107 | 0.38 | ~78 s | Multilingual; strong judged quality |
+| `jina-embeddings-v3` | 1024 | ~241 | ~37 s | **112** | 0.51 | ~62 s | CC-BY-NC-4.0 (non-commercial); community GGUF base weights — best judged picks + 88% corroboration; text-matching LoRA tested via llama.cpp: worse (98) |
+| `bge-large` | 1024 | ~115 | ~17 s | 106 | 1.01 | ~29 s | English-only yet still strong judged quality |
+| `mxbai-embed-large` | 1024 | ~340 | ~52 s | 107 | 0.35 | ~87 s | Strong judged quality |
+| `snowflake-arctic-embed2` | 1024 | ~505 | ~77 s | 81 | 0.17 | ~130 s | Conservative scores — few edges pass the 0.55 cutoff, weakest judged picks |
+| `all-minilm` | 384 | — | — | — | — | — | English only; not measured |
+| `qwen3-embedding:0.6b` | 1024 | ~350 | ~53 s | 90 | 0.28 | ~89 s | Strong multilingual results upstream |
+| `qwen3-embedding:4b` | 2560 | ~4000 | ~10 min | 87 | 0.02 | ~17 min | Poor value: ~38× the cost for below-average judged picks |
+| `qwen3-embedding:8b` | 4096 | ~10800 | ~27 min | 100 | 0.01 | ~46 min | Reference quality but ~103× the cost; build-time only |
+
+\* judged relevance: `gpt-oss:120b-cloud` scored every deduplicated
+related-article suggestion 0–2, blind to which model proposed it
+(`benchmarks/related_quality_judge.rb`); mean precision normalized to
+`qwen3-embedding:8b` = 100. Raw mean scores ranged 0.70–0.96 of 2.00.
+† quality ÷ ms/doc, normalized to `embeddinggemma:300m` = 1.00.
+‡ amortized embedding time per year assuming one cold re-embed plus weekly
+builds changing two posts — at this scale every model is affordable, so
+quality dominates the choice.
+
+The index cache amortizes cold-corpus cost — only new or changed documents
+re-embed — but every changed post pays the per-document latency at each
+build.
+
+Judged quality inverted the earlier agreement-based ranking: models with
+"hot" similarity distributions (`embeddinggemma-2`, `nomic-embed-text`,
+`voyage-4-nano`, `bge-m3`, `mxbai-embed-large`) were not producing noise —
+the judge rated their novel picks as the *most* relevant. Conservative
+models (snowflake, the small Qwen3s) scored worst on judged precision
+despite high agreement with `qwen3-embedding:8b`. Practical takeaway for
+this corpus size: `nomic-embed-text`, `voyage-4-nano`, and
+`embeddinggemma:300m` offer the best measured value; the default
+`embeddinggemma-2:570m` (and its identical-text-vector twin `:270m`)
+trades ~2.5× the per-document latency for the best judged relatedness of
+any license-free model; the large Qwen3 models do not pay for themselves
+below several thousand documents.
+
+"Build-time only" models are too large to run in the browser for
+`engine: semantic` — use them only with a lexical engine, where embeddings
+serve `related` analysis and never reach the visitor's browser, or with
+`query_embedder.type: ollama_api` where every visitor can reach the API.
 
 The model used at build time must match the query model's weights, vector
 space, dimensions, and preprocessing. Cross-runtime compatibility is verified
-for the default `embeddinggemma:300m` mapping. An `all-minilm` mapping is also
-provided, but sites should independently verify ranking when changing models.
-For any other Ollama model, set `embedding.query_embedder.model` to a
-compatible Transformers.js model explicitly or use `ollama_api` so both paths
-call the same Ollama model.
+for the `embeddinggemma:300m` → `onnx-community/embeddinggemma-300m-ONNX`
+mapping. The `embeddinggemma-2` family maps to
+`onnx-community/embeddinggemma-2-ONNX` — packaged but not yet parity-verified
+against the Ollama vectors, so confirm ranking on your own corpus. An
+`all-minilm` mapping is also provided. For any other Ollama model, set
+`embedding.query_embedder.model` to a compatible transformers.js model
+explicitly or use `ollama_api` so both paths call the same Ollama model.
+
+### Splitting the index and related models
+
+`search-relations.json` is computed entirely at build time, so related
+analysis can use a different vector space than the semantic index — the
+index vectors are the only ones the browser query embedder must match.
+Two optional keys control this; both default to `embedding.model`:
+
+```yaml
+client_search:
+  engine: semantic
+  embedding:
+    enabled: true
+    model: embeddinggemma-2:570m      # default build model
+  semantic:
+    index_model: embeddinggemma:300m  # vectors written into the index
+  related:
+    enabled: true
+    index_model: embeddinggemma-2:570m # vectors used for related analysis
+```
+
+When `semantic.index_model` and `related.index_model` resolve to the same
+model, documents are embedded once and the vector serves both outputs —
+there is no extra cost. When they differ, each document is embedded twice
+(once per model) and the related vectors are kept in a separate field,
+analyzed by `RelatedAnalyzer`, then stripped before `search-index.json`
+is written. Each model gets its own content-hash cache, so per-model
+vectors still amortize across builds. `semantic.document_prefix` /
+`semantic.query_prefix` can override the index model's prefixes; explicit
+`embedding.document_prefix` / `embedding.query_prefix` apply to
+`embedding.model`.
+
+`semantic.index_model` matters whenever vectors go into the index —
+including a lexical engine with `embedding.include_in_index: true`. It
+must always match the browser query model; `semantic.index_model:
+embeddinggemma:300m` is the safe choice for the verified transformers.js
+mapping.
+
+### Browser query models
+
+With `query_embedder.type: transformers`, the query model runs in the
+visitor's browser and must be a transformers.js-compatible ONNX build of
+the *index* model's weights. These Ollama models have a packaged mapping:
+
+| `embedding.model` / `semantic.index_model` | transformers.js model |
+| --- | --- |
+| `embeddinggemma:300m` | `onnx-community/embeddinggemma-300m-ONNX` (verified) |
+| `embeddinggemma-2`, `embeddinggemma-2:270m`, `embeddinggemma-2:570m` | `onnx-community/embeddinggemma-2-ONNX` (verify parity) |
+| `all-minilm` | `Xenova/all-MiniLM-L6-v2` |
+
+For anything else, set `embedding.query_embedder.model` to a compatible
+transformers.js model or use `query_embedder.type: ollama_api`. The full
+set of supported architectures and tasks is in the
+[transformers.js docs](https://huggingface.co/docs/transformers.js/index);
+compatible ONNX weights are tagged
+[`library:transformers.js` on the Hub](https://huggingface.co/models?library=transformers.js).
+
+Document/query prefixes are applied automatically for `embeddinggemma`,
+`embeddinggemma-2`, `nomic-embed-text`, and `voyage-4-nano`; other models
+get no prefix.
+Qwen3 embedding models take no document prefix; for semantic search the
+instruct-format query prefix must be set via `embedding.query_prefix`.
+Cosine similarity distributions differ substantially per model — some run
+"narrow" (snowflake: almost nothing passes 0.55) and some run "hot"
+(nomic, embeddinggemma-2: nearly everything does) — so retune
+`related.minimum_similarity` and the semantic threshold when switching
+rather than comparing at the default.
+
+To benchmark your own site and corpus:
+
+```bash
+cd jekyll-client-search
+bundle install
+OLLAMA_BASE_URL=http://127.0.0.1:11434 \
+  EMBEDDING_MODELS="embeddinggemma:300m,nomic-embed-text,bge-m3" \
+  bundle exec ruby benchmarks/related_model_benchmark.rb /path/to/site
+```
+
+The script times `OllamaEmbeddingAdapter.embed` per document and runs the
+real `RelatedAnalyzer`, then diffs each model's related lists against the
+baseline (`BASELINE_MODEL` env, default `embeddinggemma-2:570m`). Raw vectors
+are cached under `benchmarks/.related-embeddings/` so re-runs are cheap.
+
+A `localhost` `base_url` is resolved automatically to the loopback literal
+that accepts a TCP connection (`127.0.0.1` or `[::1]`), because the HTTP
+client does not fall back across address families and Ollama often listens
+on IPv4 loopback only. If `ollama embed` calls still fail with a socket
+error, set `base_url: http://127.0.0.1:11434` explicitly.
 
 ## Related articles reference
 
@@ -977,7 +1132,7 @@ client_search:
   engine: semantic
   embedding:
     enabled: true
-    model: embeddinggemma:300m
+    model: embeddinggemma-2:570m
     query_embedder:
       type: ollama_api
       api_url: https://embeddings.example.com/api/embed

data/assets/query-embedders/ollama-api.js

@@ -2,7 +2,7 @@
     "use strict";
 
     var config = window.ClientSearchEmbedderConfig || {};
-    var model = config.model || config.buildModel || "embeddinggemma:300m";
+    var model = config.model || config.buildModel || "embeddinggemma-2:570m";
     var apiUrl = config.apiUrl || "http://localhost:11434/api/embed";
     var activeController = null;
 

data/lib/jekyll/client_search/configuration.rb

@@ -27,7 +27,7 @@ module Jekyll
         "icon_field" => "icon_url",
         "embedding" => {
           "enabled" => false,
-          "model" => "embeddinggemma:300m",
+          "model" => "embeddinggemma-2:570m",
           "base_url" => "http://localhost:11434",
           "connect_timeout" => 5,
           "read_timeout" => 120,
@@ -41,9 +41,7 @@ module Jekyll
       def initialize(site)
         configured = normalize_configured(site.config["client_search"])
         @values = DEFAULTS.merge(configured)
-        @live_search = LiveSearchConfiguration.new(configured["live_search"] || {}, engine: engine)
-        @related = RelatedConfiguration.new(configured["related"] || {})
-        @dropdown = DropdownConfiguration.new(configured["dropdown"] || {})
+        build_sub_configurations(configured)
         merge_embedding_config(configured["embedding"])
         validate_engine!
         return unless embedding_enabled?
@@ -54,6 +52,13 @@ module Jekyll
 
       private
 
+      def build_sub_configurations(configured)
+        @live_search = LiveSearchConfiguration.new(configured["live_search"] || {}, engine: engine)
+        @related = RelatedConfiguration.new(configured["related"] || {})
+        @semantic = SemanticConfiguration.new(configured["semantic"] || {})
+        @dropdown = DropdownConfiguration.new(configured["dropdown"] || {})
+      end
+
       def normalize_configured(value)
         return { "enabled" => false } if value == false
         return value if value.is_a?(Hash)
@@ -70,13 +75,9 @@ module Jekyll
                 "client_search embedding configuration must be a mapping"
         end
         @values["embedding"] = DEFAULTS.fetch("embedding").merge(embedding)
-        merge_query_embedder_config(embedding["query_embedder"])
-      end
-
-      def merge_query_embedder_config(configured_query_embedder)
         @query_embedder = QueryEmbedderConfiguration.new(
-          configured_query_embedder || {},
-          build_model: embedding_model,
+          embedding["query_embedder"] || {},
+          build_model: embedding_index_model,
           build_base_url: embedding_base_url,
           query_prefix: embedding_query_prefix
         )

data/lib/jekyll/client_search/configuration_accessors.rb

@@ -20,6 +20,19 @@ module Jekyll
         @related
       end
 
+      # The Ollama model whose vectors are written into the search index and
+      # must match the browser query embedder's vector space. Defaults to
+      # embedding.model; override with semantic.index_model.
+      def embedding_index_model
+        @semantic.index_model || embedding_model
+      end
+
+      # The Ollama model used for related-article similarity analysis.
+      # Defaults to embedding.model; override with related.index_model.
+      def embedding_related_model
+        @related.index_model || embedding_model
+      end
+
       def query_embedder_type
         @query_embedder.type
       end

data/lib/jekyll/client_search/embedding_configuration.rb

@@ -9,9 +9,25 @@ module Jekyll
           "document" => "title: none | text: ",
           "query" => "task: search result | query: "
         },
+        "embeddinggemma-2" => {
+          "document" => "title: none | text: ",
+          "query" => "task: search result | query: "
+        },
         "nomic-embed-text" => {
           "document" => "search_document: ",
           "query" => "search_query: "
+        },
+        "voyage-4-nano" => {
+          "document" => "Represent the document for retrieval: ",
+          "query" => "Represent the query for retrieving supporting documents: "
+        },
+        "qwen3-embedding" => {
+          "document" => "",
+          "query" => "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: "
+        },
+        "bge-large" => {
+          "document" => "",
+          "query" => "Represent this sentence for searching relevant passages: "
         }
       }.freeze
 
@@ -44,43 +60,63 @@ module Jekyll
         configured.nil? ? engine == "semantic" : configured == true
       end
 
+      # Document prefix for the index model (semantic.index_model).
       def embedding_document_prefix
-        configured_prefix("document_prefix", "document")
+        configured_prefix("document", model: embedding_index_model, override: @semantic.document_prefix)
       end
 
+      # Query prefix for the index model — must match the browser query
+      # embedder's vector space.
       def embedding_query_prefix
-        configured_prefix("query_prefix", "query")
+        configured_prefix("query", model: embedding_index_model, override: @semantic.query_prefix)
+      end
+
+      # Document prefix for the related-analysis model (related.index_model).
+      def embedding_related_document_prefix
+        configured_prefix("document", model: embedding_related_model)
       end
 
       def embedding_identity
+        embedding_identity_for(model: embedding_index_model,
+                               document_prefix: embedding_document_prefix,
+                               query_prefix: embedding_query_prefix)
+      end
+
+      def embedding_related_identity
+        embedding_identity_for(model: embedding_related_model,
+                               document_prefix: embedding_related_document_prefix,
+                               query_prefix: configured_prefix("query", model: embedding_related_model))
+      end
+
+      private
+
+      def embedding_identity_for(model:, document_prefix:, query_prefix:)
         {
           "provider" => "ollama",
-          "model" => embedding_model,
+          "model" => model,
           "base_url" => embedding_base_url,
-          "document_prefix" => embedding_document_prefix,
-          "query_prefix" => embedding_query_prefix,
+          "document_prefix" => document_prefix,
+          "query_prefix" => query_prefix,
           "schema" => 2
         }
       end
 
-      private
+      def configured_prefix(role, model:, override: nil)
+        return override.to_s if override
 
-      def configured_prefix(config_key, role)
         embedding = @values.fetch("embedding")
-        return embedding.fetch(config_key).to_s if embedding.key?(config_key)
+        config_key = "#{role}_prefix"
+        return embedding[config_key].to_s if model == embedding_model && embedding.key?(config_key)
 
-        model_name = embedding_model.split(":").first
-        PREFIXES.fetch(model_name, {}).fetch(role, "")
+        PREFIXES.fetch(model.split(":").first, {}).fetch(role, "")
       end
 
       def positive_timeout(key)
         value = @values.fetch("embedding").fetch(key).to_f
-        unless value.positive? && value.finite?
-          raise Jekyll::Errors::FatalException,
-                "embedding #{key} must be a finite value greater than zero"
-        end
+        return value if value.positive? && value.finite?
 
-        value
+        raise Jekyll::Errors::FatalException,
+              "embedding #{key} must be a finite value greater than zero"
       end
 
       def validate_embedding!

data/lib/jekyll/client_search/generator.rb

@@ -25,6 +25,7 @@ module Jekyll
 
       def add_generated_pages(site, documents, configuration)
         add_related_page(site, documents, configuration) if configuration.related_enabled?
+        documents.each { |document| document.delete("related_embedding") }
         remove_embeddings(documents) unless configuration.embedding_include_in_index?
         site.pages << SearchIndexPage.new(site, configuration.output, documents)
         site.pages << RuntimeConfigPage.new(site, configuration) if configuration.copy_runtime?
@@ -39,18 +40,60 @@ module Jekyll
       end
 
       def enhance_with_embeddings(site, documents, configuration)
-        cache = IndexCache.new(site.source, embedding_identity: configuration.embedding_identity)
-        adapter = build_embedding_adapter(configuration)
-
-        documents.each { |entry| embed_document(entry, cache, adapter, configuration) }
-        cache.prune(documents.map { |doc| doc["id"] })
-        cache.save
+        embedding_jobs(configuration).each do |job|
+          cache = IndexCache.new(site.source, embedding_identity: job[:identity],
+                                              file_name: job[:cache_file])
+          adapter = build_embedding_adapter(configuration, model: job[:model])
+
+          documents.each { |entry| embed_document(entry, cache, adapter, configuration, job) }
+          cache.prune(documents.map { |doc| doc["id"] })
+          cache.save
+        end
         documents
       end
 
-      def build_embedding_adapter(configuration)
+      # Each job embeds every document once per model into a target field.
+      # The index model writes "embedding" (kept when include_in_index); a
+      # divergent related model writes "related_embedding" (stripped before
+      # the index is emitted). Identical models share a single pass.
+      def embedding_jobs(configuration)
+        jobs = []
+        jobs << index_embedding_job(configuration) if configuration.embedding_include_in_index?
+        jobs.concat(related_embedding_jobs(configuration)) if configuration.related_enabled?
+        jobs
+      end
+
+      def index_embedding_job(configuration)
+        {
+          model: configuration.embedding_index_model,
+          field: "embedding",
+          document_prefix: configuration.embedding_document_prefix,
+          identity: configuration.embedding_identity,
+          cache_file: IndexCache::CACHE_FILE
+        }
+      end
+
+      def related_embedding_jobs(configuration)
+        return [related_embedding_job("embedding", IndexCache::CACHE_FILE, configuration)] unless
+          configuration.embedding_include_in_index?
+        return [] if configuration.embedding_related_model == configuration.embedding_index_model
+
+        [related_embedding_job("related_embedding", IndexCache::RELATED_CACHE_FILE, configuration)]
+      end
+
+      def related_embedding_job(field, cache_file, configuration)
+        {
+          model: configuration.embedding_related_model,
+          field: field,
+          document_prefix: configuration.embedding_related_document_prefix,
+          identity: configuration.embedding_related_identity,
+          cache_file: cache_file
+        }
+      end
+
+      def build_embedding_adapter(configuration, model: configuration.embedding_model)
         OllamaEmbeddingAdapter.new(
-          model: configuration.embedding_model,
+          model: model,
           base_url: configuration.embedding_base_url,
           connect_timeout: configuration.embedding_connect_timeout,
           read_timeout: configuration.embedding_read_timeout
@@ -66,17 +109,13 @@ module Jekyll
         documents.each { |document| document.delete("embedding") }
       end
 
-      def embed_document(document, cache, adapter, configuration)
-        hash = IndexCache.content_hash(document)
-        cached = cache.lookup(document["id"], hash)
-        if cached && cached["embedding"]
-          document["embedding"] = cached["embedding"]
-          return
-        end
+      def embed_document(document, cache, adapter, configuration, job)
+        hash = IndexCache.content_hash(document.except("embedding", "related_embedding"))
+        return if embedding_restored?(document, cache, hash, job[:field])
 
-        embedding = adapter.embed(embedding_text(document, configuration))
+        embedding = adapter.embed(embedding_text(document, job[:document_prefix]))
         if embedding
-          document["embedding"] = embedding
+          document[job[:field]] = embedding
           cache.store(document["id"], hash, embedding)
         elsif configuration.embedding_fail_on_error?
           raise Jekyll::Errors::FatalException,
@@ -84,9 +123,17 @@ module Jekyll
         end
       end
 
-      def embedding_text(document, configuration)
+      def embedding_restored?(document, cache, hash, field)
+        cached = cache.lookup(document["id"], hash)
+        return false unless cached && cached["embedding"]
+
+        document[field] = cached["embedding"]
+        true
+      end
+
+      def embedding_text(document, document_prefix)
         text = [document["title"], document["excerpt"], document["content"]].compact.join(" ")
-        "#{configuration.embedding_document_prefix}#{text}"
+        "#{document_prefix}#{text}"
       end
 
       def collection_documents(site, configuration, builder)

data/lib/jekyll/client_search/index_cache.rb

@@ -12,11 +12,12 @@ module Jekyll
     # should be git-ignored.
     class IndexCache
       CACHE_FILE = ".jekyll-client-search-cache.json"
+      RELATED_CACHE_FILE = ".jekyll-client-search-related-cache.json"
 
       attr_reader :path
 
-      def initialize(site_source, embedding_identity: nil)
-        @path = File.join(site_source, CACHE_FILE)
+      def initialize(site_source, embedding_identity: nil, file_name: CACHE_FILE)
+        @path = File.join(site_source, file_name)
         @embedding_identity = embedding_identity
         @entries = load
       end

data/lib/jekyll/client_search/ollama_embedding_adapter.rb

@@ -11,6 +11,11 @@ module Jekyll
     #     enabled: true
     #     model: all-minilm          # or nomic-embed-text, bge-m3, etc.
     #     base_url: http://localhost:11434
+    #
+    # A +localhost+ base_url is resolved once to the loopback literal
+    # (127.0.0.1 or ::1) that accepts a TCP connection, because Excon
+    # does not fall back across address families when the DNS name maps
+    # to a family the server is not listening on.
     class OllamaEmbeddingAdapter
       attr_reader :model, :base_url, :connect_timeout, :read_timeout
 
@@ -48,12 +53,53 @@ module Jekyll
         @client ||= begin
           require "ollama"
           Ollama::Client.new(
-            base_url: @base_url,
+            base_url: resolved_base_url,
             connect_timeout: @connect_timeout,
             read_timeout: @read_timeout
           )
         end
       end
+
+      # "localhost" may resolve to ::1 while the server only listens on
+      # 127.0.0.1, or the reverse. Excon does not fall back across
+      # address families, so probe both loopback literals once and use
+      # the one that accepts a TCP connection. Any other host name is
+      # used exactly as configured.
+      def resolved_base_url
+        @resolved_base_url ||= resolve_localhost(@base_url)
+      end
+
+      def resolve_localhost(base_url)
+        require "uri"
+        uri = URI.parse(base_url)
+        return base_url unless uri.host.to_s.chomp(".").casecmp("localhost").zero?
+
+        resolved = loopback_base_url(uri)
+        return base_url unless resolved
+
+        Jekyll.logger.info "ClientSearch:", "resolved localhost to #{resolved}"
+        resolved
+      rescue URI::InvalidURIError
+        base_url
+      end
+
+      def loopback_base_url(uri)
+        %w[127.0.0.1 ::1].each do |address|
+          next unless tcp_reachable?(address, uri.port)
+
+          resolved = uri.dup
+          resolved.host = address.include?(":") ? "[#{address}]" : address
+          return resolved.to_s
+        end
+        nil
+      end
+
+      def tcp_reachable?(address, port)
+        require "socket"
+        Socket.tcp(address, port, connect_timeout: @connect_timeout) { true }
+      rescue SystemCallError, SocketError
+        false
+      end
     end
   end
 end

data/lib/jekyll/client_search/query_embedder_configuration.rb

@@ -8,6 +8,9 @@ module Jekyll
 
       MODEL_MAP = {
         "embeddinggemma:300m" => "onnx-community/embeddinggemma-300m-ONNX",
+        "embeddinggemma-2" => "onnx-community/embeddinggemma-2-ONNX",
+        "embeddinggemma-2:270m" => "onnx-community/embeddinggemma-2-ONNX",
+        "embeddinggemma-2:570m" => "onnx-community/embeddinggemma-2-ONNX",
         "all-minilm" => "Xenova/all-MiniLM-L6-v2"
       }.freeze
 

data/lib/jekyll/client_search/related_analyzer.rb

@@ -29,7 +29,10 @@ module Jekyll
 
       def build_relation(source, target)
         metadata = metadata_relation(source, target)
-        semantic_similarity = similarity(source["embedding"], target["embedding"])
+        semantic_similarity = similarity(
+          source["related_embedding"] || source["embedding"],
+          target["related_embedding"] || target["embedding"]
+        )
         return unless metadata || semantic_match?(semantic_similarity)
 
         relation = base_relation(target, metadata, semantic_similarity)

data/lib/jekyll/client_search/related_configuration.rb

@@ -62,12 +62,27 @@ module Jekyll
         @values.fetch("max_items")
       end
 
+      # Ollama model used for related-article similarity analysis. nil means
+      # the site uses the general embedding.model.
+      def index_model
+        value = @values["index_model"]
+        value&.to_s
+      end
+
       private
 
       def validate!
         validate_booleans!
         validate_similarity!
         validate_max_items!
+        validate_index_model!
+      end
+
+      def validate_index_model!
+        return if index_model.nil? || !index_model.empty?
+
+        raise Jekyll::Errors::FatalException,
+              "client_search related index_model must not be empty"
       end
 
       def validate_booleans!

data/lib/jekyll/client_search/semantic_configuration.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module Jekyll
+  module ClientSearch
+    # Validates semantic-engine index settings: which Ollama model's vectors
+    # are written into the search index and must match the browser query
+    # embedder's vector space.
+    class SemanticConfiguration
+      def initialize(config)
+        unless config.is_a?(Hash)
+          raise Jekyll::Errors::FatalException,
+                "client_search semantic configuration must be a mapping"
+        end
+
+        @values = config
+        validate!
+      end
+
+      # Ollama model embedded into the search index. nil means the site uses
+      # the general embedding.model.
+      def index_model
+        value = @values["index_model"]
+        value&.to_s
+      end
+
+      def document_prefix
+        value = @values["document_prefix"]
+        value&.to_s
+      end
+
+      def query_prefix
+        value = @values["query_prefix"]
+        value&.to_s
+      end
+
+      private
+
+      def validate!
+        return if index_model.nil? || !index_model.empty?
+
+        raise Jekyll::Errors::FatalException,
+              "client_search semantic index_model must not be empty"
+      end
+    end
+  end
+end

data/lib/jekyll/client_search/version.rb

@@ -2,6 +2,6 @@
 
 module Jekyll
   module ClientSearch
-    VERSION = "0.3.5"
+    VERSION = "0.4.0"
   end
 end

data/lib/jekyll/client_search.rb

@@ -9,6 +9,7 @@ require_relative "client_search/version"
 require_relative "client_search/embedding_configuration"
 require_relative "client_search/live_search_configuration"
 require_relative "client_search/related_configuration"
+require_relative "client_search/semantic_configuration"
 require_relative "client_search/dropdown_configuration"
 require_relative "client_search/query_embedder_configuration"
 require_relative "client_search/configuration_accessors"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: jekyll-client-search
 version: !ruby/object:Gem::Version
-  version: 0.3.5
+  version: 0.4.0
 platform: ruby
 authors:
 - Svend Gundestrup
@@ -78,6 +78,7 @@ files:
 - lib/jekyll/client_search/script_tags.rb
 - lib/jekyll/client_search/search_index_page.rb
 - lib/jekyll/client_search/search_tag.rb
+- lib/jekyll/client_search/semantic_configuration.rb
 - lib/jekyll/client_search/tasks.rb
 - lib/jekyll/client_search/version.rb
 homepage: https://github.com/gundestrup/jekyll-client-search