jekyll-client-search 0.3.4 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 74262c7813a4c946b9a4c9c7d7a19aa1d866dedfdf82781131e5edbe9aabbdef
-  data.tar.gz: 6ebbd6018cfc419d586d50d60d7d346c5a6c0a7b442275caf55fc22d3420a797
+  metadata.gz: 749a1708fe34bce1624db0eb4d91ea58a569f2663f8d78b8200365188faf7ae4
+  data.tar.gz: 833b1c87a57b2f78a05f42103efef6f9aa2ec09ee0726cad734e0f28f268fc50
 SHA512:
-  metadata.gz: 402221144b4bbf95e1daf759430c6a59fad8025d650f4f84624146ed44951de2e34755a1e335f478aeabe2fbec693f2b95c74c64aad1ff19e54b3a1e30213ee8
-  data.tar.gz: 84d405d332fdd45895885b43dc408a46574b97f26f2b1742513ad97f9910e0ed96f6bb4ddff0c4cf0148751b2a0348b97b5adaa41aab57ba5c724c07f0827d5e
+  metadata.gz: 169936c38c680b2f4ec657243909aaad0a902b6bbbc52e9c586d3d986651e1529fdfb2d2a9dcde69f997075b8c901c5178b7148d86f064242ee018a228b2e457
+  data.tar.gz: 3e75798e1b41dac7982ca34e83e61e8b28b03b347f6ca60b6e527570a72d4a69a9f5ddf6293197410c6935a699b54a7001b601145c57c7e09302e4f1edf4d426

data/CHANGELOG.md

@@ -1,13 +1,109 @@
 # Changelog
 
-## Unreleased
+## 0.4.0 — 2026-10-24
+
+- Dev-tooling bump: `jsdom` 30.1.3 (patch).
+
+### Added
+
+- `benchmarks/embedding_model_benchmark.rb`,
+  `benchmarks/related_model_benchmark.rb`, and
+  `benchmarks/related_quality_judge.rb` — measure Ollama embedding
+  latency per model on a real site, compare `related` output across
+  models, and score judged relevance with a blind LLM judge
+  (10 local embedding models benchmarked on a 152-document site).
+- `embeddinggemma-2` document/query prefixes (same prompt format as
+  `embeddinggemma`), `voyage-4-nano` prefixes (open-weight Voyage
+  model via community GGUF), and `qwen3-embedding` prefixes (official
+  instruct-format query prefix; empty document prefix).
+- README model-selection guidance: measured per-document latency table,
+  LLM-judged quality ratings, quality-per-cost value index, amortized
+  yearly cost, `qwen3-embedding` build-time-only notes, and the
+  `localhost` → `127.0.0.1` loopback gotcha.
+- Split index/related embedding models: `semantic.index_model` selects
+  the Ollama model whose vectors go into `search-index.json` (must match
+  the browser query embedder); `related.index_model` selects the model
+  used for related-article analysis. Both default to `embedding.model`;
+  equal models share one embedding pass, divergent models embed each
+  document twice into separate cached vector fields.
+  `semantic.document_prefix` / `semantic.query_prefix` override the
+  index model's prefixes. Related-model vectors are cached separately
+  in `.jekyll-client-search-related-cache.json` and never written to
+  the index.
+- `embeddinggemma-2` (`:270m`, `:570m`, untagged) maps to
+  `onnx-community/embeddinggemma-2-ONNX` for the transformers.js query
+  embedder. `test/onnx-parity.test.js` is an opt-in check
+  (`ONNX_PARITY=1`, requires `@huggingface/transformers` and a running
+  Ollama) that compares ONNX and Ollama vectors and rankings;
+  `embeddinggemma:300m` remains the verified mapping.
+
+### Changed
+
+- Default `embedding.model` is now `embeddinggemma-2:570m` (was
+  `embeddinggemma:300m`) — the judged-quality leader among license-free
+  local models (111 vs 96) at ~2.5× the per-document latency; its text
+  vectors are byte-identical to `embeddinggemma-2:270m`. The
+  transformers.js query embedder maps it to
+  `onnx-community/embeddinggemma-2-ONNX`; sites that prefer the verified
+  browser mapping can set `semantic.index_model: embeddinggemma:300m`
+  while related analysis stays on the default model.
+- CI: Semgrep moved from an inline `pip3 install semgrep==1.176.0` pin to hashed `.github/requirements-ci.txt` (compiled from `requirements-ci.in`, Python 3.12), installed with `--require-hashes`. Bumped Semgrep to 1.180.0; the version is now Dependabot-visible via a new `pip` ecosystem entry for `/.github`.
+
+### Fixed
+
+- A `localhost` `embedding.base_url` (the default) is resolved once to
+  the loopback literal that accepts a TCP connection — `127.0.0.1` or
+  `[::1]` — because `ollama-ruby`/Excon does not fall back across
+  address families and Ollama commonly listens on IPv4 loopback only.
+  Embedding calls that died with `Excon::Error::Socket "Broken pipe"`
+  when `localhost` resolved to `::1` first now connect on both
+  IPv4-only and IPv6-only servers.
+
+## 0.3.5 — 2026-10-06
+
+### Fixed
+
+- `LICENSE` now contains the verbatim AGPL-3.0 text (was a 22-line
+  stub that license scanners could not detect); README keeps the
+  copyright attribution.
+
+### Added
+
+- Codecov coverage reporting: CI uploads Cobertura XML (via `simplecov-cobertura`) from the Ruby 3.4 / Node 24 matrix job, with `codecov.yml` status checks (85% project/patch targets) and a README badge
+- `rake version:check_consistency` — two-axis drift check: doc floor
+  literals (`Ruby >= X.Y`, `TargetRubyVersion`) against the gemspec,
+  full `X.Y.Z` pins against `.ruby-version`. `rake version:pre_release`
+  runs it with `check_changelog`.
+
+### Changed
+
+- `release.yml` no longer hardcodes the Ruby pin — `setup-ruby` reads
+  `.ruby-version` (single source for dev/release; the 3.2 floor stays a
+  deliberate gemspec literal proven by the CI matrix).
+- `rbenv` instructions in README/README.developer.md now derive from
+  `.ruby-version` instead of restating the pin.
+- Git hooks moved to tracked `bin/hooks/` enabled via `core.hooksPath`
+  — `bin/install-hooks.sh` no longer generates copies into `.git/hooks`.
+- `spec/spec_helper.rb` autoloads `spec/support/**/*.rb`.
+- Removed stale literal counts from docs (test examples, Semgrep rule
+  counts) — numbers that change on their own aren't restated.
+
+### Security
+
+- Dev-tooling audit gate: `scripts/npm-audit.mjs` + `.audit-allow.json`
+  replace the bare `npm audit` CI step — patchable transitive deps are
+  forced to fixed versions via `overrides` (smol-toml, katex), and only
+  explicitly allowlisted unpatched advisories are tolerated (braces,
+  dev-only, no patched release).
 
 ## 0.3.4 — 2026-09-11
 
 ### Added
+
 - SonarQube Cloud badge in README.md (project key: `gundestrup_jekyll-client-search`)
 
 ### Fixed
+
 - Verify `event.origin` in transformers-worker.js `message` listener to prevent cross-origin message injection (SonarQube: `javascript:S2819`, CRITICAL)
 - Add `--ignore-scripts` to `npm ci` in CI to prevent lifecycle script execution during dependency installation (SonarQube: `githubactions:S6505`)
 - Pin Semgrep to `1.176.0` with `--only-binary :all:` in CI to prevent setup script execution and lock resolved versions (SonarQube: `githubactions:S8541`, `githubactions:S8544`)
@@ -18,11 +114,13 @@
 ## 0.3.3 — 2026-09-11
 
 ### Refactored
+
 - Extract duplicated `normalize` function from `client-search-base.js` and `client-search-dropdown.js` into new shared module `assets/client-search-shared.js` (exposes `window.ClientSearchShared.normalize` and `window.ClientSearchShared.coreFields`) — eliminates CodeFactor duplicate-code finding and ensures consistent normalization across both runtimes
 
 ## 0.3.2 — 2026-09-10
 
 ### Added
+
 - Semgrep security scan in CI (`semgrep ci` job in `.github/workflows/ci.yml`) — runs on every push and pull request using the `SEMGREP_APP_TOKEN` secret and the Semgrep Pro engine with the organization's full policy (2,900+ rules)
 - `.semgrep.yml` with custom Ruby ReDoS detection rules and usage documentation
 - Semgrep step in pre-commit hook (`bin/install-hooks.sh`)
@@ -32,20 +130,24 @@
 - Semgrep usage guide in README.developer.md (local commands, nosemgrep annotations, SHA pinning)
 
 ### Fixed
+
 - Pin all GitHub Actions to full 40-character commit SHAs with version comments to prevent supply-chain attacks via mutable tag repointing (Semgrep: `github-actions-mutable-action-tag`)
 - Replace `Object.assign({}, entry, ...)` in benchmark with explicit field construction to avoid mass-assignment taint (Semgrep: `insecure-object-assign`)
 - Replace manual arXiv query string with `URI.encode_www_form` for correct URL encoding (Semgrep: `avoid-tainted-http-request`)
 
 ### Changed
+
 - Add `nosemgrep` annotations with justifications for false positives (static ERB template, array-form `Open3.capture2`, fixed-host arXiv request) and acceptable risks (local loopback Ollama HTTP in tests and benchmarks)
 
 ## 0.3.1 — 2026-09-09
 
 ### Fixed
+
 - Fix CodeQL `rb/bad-tag-filter` (high) in `DocumentBuilder#clean` — closing tag regex `</script\s*>` only matched whitespace before `>`, missing browser-accepted tags like `</script\t\n bar>`; changed to `</script[^>]*>` and `</style[^>]*>` to match any non-`>` content
 - Fix CodeQL `rb/polynomial-redos` (high) in `RelatedTag` — `\s*` patterns between optional groups in SYNTAX regex caused O(n²) backtracking on input with many spaces; replaced single regex with token-split parser that validates each token individually
 
 ### Changed
+
 - Update `simplecov` 1.1.1 → 1.2.0
 - Update `google-protobuf` 4.36.0 → 4.36.1 (transitive)
 - Update `sass-embedded` 1.103.1 → 1.104.0 (transitive)
@@ -55,6 +157,7 @@
 ## 0.3.0 — 2026-09-02
 
 ### Added
+
 - `{% search_dropdown %}` Liquid tag — compact live-search dropdown for navbars and headers with keyboard navigation (Arrow/Enter/Escape), lazy index loading, icon support, and framework-agnostic semantic HTML (no Bulma/Bootstrap/Tailwind classes)
 - `DropdownConfiguration` class with `enabled`, `max_items`, `min_chars`, `debounce_ms`, `redirect_url` options
 - `assets/client-search-dropdown.js` runtime (434 lines) — two-stage search (AND first, fuzzy OR fallback), shared index cache with the base runtime, multi-instance support via `[data-client-search-dropdown]` attributes, icon rendering from `icon_field`
@@ -69,6 +172,7 @@
 - System test assertions for embeddings/retrieval content matching and non-ASCII query robustness
 
 ### Changed
+
 - `related.max_items` default changed from `nil` (unlimited) to `5`
 - `Configuration#initialize` refactored — extracted `normalize_configured` helper for cleaner config validation
 - Runtime config page now merges `dropdown` config alongside `liveSearch`
@@ -80,6 +184,7 @@
 - README.md updated with dropdown section, related `max:N` docs, and CSS styling examples for dropdown
 
 ### Fixed
+
 - `SearchTag#render` no longer raises on nil site (now returns empty string)
 - `RelatedTag#render` no longer raises on nil site or `client_search: false` config
 - Meta test race condition — `forEach` callback couldn't `await settle()`, causing flaky cross-engine comparisons; fixed by switching to `for` loops
@@ -87,6 +192,7 @@
 ## 0.2.0 — 2026-08-27
 
 ### Added
+
 - Generic `passthrough_fields` config option: external plugins declare which document `data` fields to forward into the search index (e.g. `file_type`, `icon_url`, `icon_set` from jekyll-documents)
 - Field renaming in `passthrough_fields`: entries can be strings (same name) or hashes (`{source => target}`) for integration with other search conventions
 - `icon_field` config option (default: `"icon_url"`): tells the browser runtime which field to render as an `<img>` icon before the result title; set to `null` to disable
@@ -94,10 +200,12 @@
 - Result `<article>` elements carry `data-source`, `data-categories`, `data-tags`, plus `data-*` for all passthrough fields (snake_case → data-kebab-case) for CSS-based result customization
 
 ### Fixed
+
 - Allow whitespace in script/style end tags in `DocumentBuilder#clean` (`</script\s*>` instead of `</script>`) — CodeQL: bad HTML filtering regexp
 - Fix polynomial ReDoS in `SearchTag` and `RelatedTag` syntax regexes — strip markup before matching, remove leading/trailing `\s*` patterns that caused O(n²) backtracking (CodeQL: polynomial regex on uncontrolled data)
 
 ### Changed
+
 - Simplified release workflow to tag-push trigger (`push: tags: v*`) — no manual `gh release create` needed
 - Switched to RubyGems trusted publishing (`rubygems/release-gem@v1` with OIDC)
 - Centralized version in `version.rb` as single source of truth — removed `version` field from `package.json`