@shoalrun/cli 0.1.0-alpha.15 → 0.1.0-alpha.16

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## 0.1.0-alpha.16 — 2026-11-04
+
+- Set `outputs.html.search.provider: semantic` to connect the search box to
+  Shoalrun's semantic endpoint without writing a custom browser module. The
+  included keyword index remains available when the endpoint is absent or
+  unavailable. Configure an embedding provider, build the passage index, and
+  run or deploy the search endpoint; selecting the browser provider alone does
+  not provision a backend.
+
+- Qualify stored passage IDs by publication so repeated page and section IDs
+  across releases cannot alias embedding rows or citation digests. Rebuild
+  passage indexes with `shoalrun index`; when using semantic retrieval, rerun
+  indexing with `--embed` and the same embedding configuration to reuse cached
+  inputs. Regenerate answer cards after rebuilding their cited passage IDs.
+  Ambiguous legacy indexes now fail with rebuild instructions.
+- Include publication membership, routes, reader marks, and prepared search
+  evidence in passage-index generation hashes without changing embedding inputs.
+- Read publication-specific passage counts and `sourceDigest` from the index
+  command's JSON summary to check which retained releases were indexed.
+- Stream hosted search scores into page selection, retaining at most three
+  candidates per page and a result-sized final ranking. Large galleries no
+  longer require an unbounded sorted passage array. Exact identifiers reserve
+  a slot among displayable pages and keep literal table/code landing evidence.
+  File-backed retrieval still scans the selected publication; external vector
+  storage and publication-sharded browser downloads are not included yet.
+
 ## 0.1.0-alpha.15 — 2026-11-03
 
 - Opt into `profiles.requireClassification` to reject unclassified public pages

package/README.md

@@ -102,6 +102,44 @@ for main, developer, and regional sites built from one source directory.
 For deployment-owned SKU selections, direct or grandfathered feature grants, and
 branding supplied outside the shared repository, see [deployment profiles](DEPLOYMENT-PROFILES.md).
 
+## Enable semantic search
+
+The default search runs in the browser with no server or API key. To use
+Shoalrun's semantic endpoint, select the included provider:
+
+```yaml
+outputs:
+  html:
+    search:
+      provider: semantic
+retrieval:
+  embeddings:
+    provider: voyage
+    model: voyage-4-large
+    queryModel: voyage-4-lite
+    dimension: 1024
+```
+
+Set `VOYAGE_API_KEY` in the build and server environment, then run:
+
+```sh
+shoalrun build docs.config.yaml site-dist
+shoalrun index site-dist --embed --config docs.config.yaml
+shoalrun serve site-dist --search --config docs.config.yaml
+```
+
+No custom browser adapter or chat model is needed. The browser sends the selected
+publication and reader view to the same-origin endpoint and uses keyword search
+if it is missing or unavailable. Each semantic query calls the embedding
+provider; keys stay server-side. Reindex changed content and restart the preview
+after rebuilding indexes. Existing matching embedding inputs are reused.
+
+`provider: semantic` configures the browser connection; production hosting still
+needs a search endpoint. The current Vercel integration supplies search through
+its chat-function deployment, which does not yet support deployment profiles.
+A static deployment remains usable through keyword fallback. External search
+services can continue to use `provider: custom`.
+
 ## Export content into an application
 
 Export pages from the same source into a portable directory for an application's

package/assets/search-semantic.js

@@ -0,0 +1,34 @@
+// Built-in semantic connection. Credentials and query embeddings stay server-side.
+import * as lexical from './search-lexical.js';
+const endpoint = new URL('../__shoalrun/search', import.meta.url).pathname;
+let endpointMissing = false;
+export const prepare = lexical.prepare;
+
+export async function search(query, context) {
+  const fallback = () => lexical.search(query, context);
+  context.signal?.throwIfAborted();
+  if (endpointMissing || query.trim().length < 3) return fallback();
+  // Debounce requests before spending query-embedding credits. Every listener
+  // is removed on completion, including when a keystroke cancels this request.
+  await new Promise((resolve, reject) => {
+    const abort = () => { clearTimeout(timer); context.signal?.removeEventListener('abort', abort); reject(context.signal.reason); };
+    const timer = setTimeout(() => { context.signal?.removeEventListener('abort', abort); resolve(); }, 200);
+    context.signal?.addEventListener('abort', abort, { once: true });
+  });
+  context.signal?.throwIfAborted();
+  try {
+    const signal = AbortSignal.any([AbortSignal.timeout(4000), ...(context.signal ? [context.signal] : [])]);
+    const response = await fetch(endpoint, {
+      method: 'POST', headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ query, variant: context.variant, limit: context.limit, readerSelection: context.readerSelection }), signal
+    });
+    if (response.status === 404) endpointMissing = true;
+    if (!response.ok) return fallback();
+    const body = await response.json();
+    context.signal?.throwIfAborted();
+    return Array.isArray(body.results) ? body.results : fallback();
+  } catch {
+    context.signal?.throwIfAborted();
+    return fallback();
+  }
+}

package/dist/artifact-bundle.d.ts

@@ -32,7 +32,7 @@ export type ArtifactAssemblyInput = {
     generatedPaletteCss?: string;
     generatedBadgeCss?: string;
     search: {
-        provider: 'lexical' | 'custom' | 'none';
+        provider: 'lexical' | 'semantic' | 'custom' | 'none';
         index: boolean;
         module?: string;
     };

package/dist/artifact-bundle.js

@@ -122,7 +122,7 @@ export async function assembleArtifacts(input) {
         const compact = { schemaVersion: 1, documents: index.documents.map(({ id, demo, url, markdownUrl, title, summary, type, variant, headings, sections, navigationTab }) => ({ id, demo, url, markdownUrl, title, summary, type, variant, headings, ...(navigationTab ? { navigationTab } : {}), ...(sections ? { sections } : {}), ...sizes.get(markdownUrl) })) };
         addMetadataArtifact('/search-index.min.json', JSON.stringify(compact) + '\n');
     }
-    if (search.provider === 'custom' && search.index)
+    if ((search.provider === 'custom' || search.provider === 'semantic') && search.index)
         addPublicArtifact('/assets/search-lexical.js', await readFile(builtInSearchProvider, 'utf8'));
     if (search.module)
         addPublicArtifact('/assets/search-provider.js', await readFile(search.module, 'utf8'));

package/dist/cli.js

@@ -238,7 +238,7 @@ try {
                 embedQuery = async (query, signal) => (await provider.embed([query], 'query', signal))[0];
             }
             extras = { search: createSearchHandler({ retriever, embedQuery, hosts, readerFilters: validateReaderFilters(settings.readerFilters), pageSummaries }) };
-            banner = `Search is on: ${embeddings ? 'hybrid retrieval' : 'lexical retrieval; run shoalrun index --embed for semantic results'}.\n`;
+            banner = `Search is on: ${embeddings ? 'semantic retrieval' : 'lexical retrieval; run shoalrun index --embed for semantic results'}.\n`;
         }
         if (ask) {
             // The preview spends your API credits on every question, so it answers only on this machine.
@@ -425,7 +425,11 @@ try {
             const provider = createProvider(embeddingProviderConfig(retrieval, { provider: values['--provider'], model: values['--model'], dimension: values['--dimension'] === undefined ? undefined : Number(values['--dimension']), baseUrl: values['--base-url'], keyEnv: values['--key-env'] }));
             embeddings = await embedChunkIndex(subcommand, result.index, provider, embeddingInput);
         }
-        process.stdout.write(JSON.stringify({ status: 'ok', documents: result.documents, chunks: result.index.chunks.length, output: result.output, ...(embeddings ? { embeddings } : {}) }, null, 2) + '\n');
+        const counts = new Map();
+        for (const chunk of result.index.chunks)
+            counts.set(chunk.variant, (counts.get(chunk.variant) ?? 0) + 1);
+        const publications = [...counts].sort(([a], [b]) => a.localeCompare(b)).map(([variant, passages]) => ({ variant, passages }));
+        process.stdout.write(JSON.stringify({ status: 'ok', documents: result.documents, chunks: result.index.chunks.length, sourceDigest: result.index.sourceDigest, publications, output: result.output, ...(embeddings ? { embeddings } : {}) }, null, 2) + '\n');
     }
     else if (command === 'ask') {
         const usage = 'Usage: shoalrun ask <site-directory> "<question>" [--config FILE] [--model MODEL] [--passages N] [--reader-view FACET=CHOICE[,CHOICE]]... [--channel C --version V --language L] [--no-rerank] [--no-direct] [--no-escalate] [--no-cards] [--json]';

package/dist/compiler.js

@@ -66,6 +66,8 @@ function validateSearch(value, declaringFile) {
         return { provider, index: false };
     if (provider === 'lexical')
         return { provider, module: builtInSearchProvider, index: true };
+    if (provider === 'semantic')
+        return { provider, module: path.resolve(path.dirname(builtInSearchProvider), 'search-semantic.js'), index: true };
     if (provider !== 'custom')
         throw new Error(`Unknown search provider ${provider}`);
     if (typeof config.module !== 'string' || !/\.m?js$/.test(config.module))

package/dist/index-build.js

@@ -164,7 +164,9 @@ export function chunkMarkdown(markdown, document, anchors = [], options = {}) {
     return chunks;
 }
 function makeChunk(document, headingPath, anchor, text, used, part = 0) {
-    const base = `${document.id}#${slug(headingPath.join(' ')) || 'intro'}${part ? `~${part + 1}` : ''}`;
+    // Logical page IDs repeat across releases. Stored passage IDs must not: vector
+    // rows and answer citations are keyed globally within this site's index.
+    const base = `${encodeURIComponent(document.variant)}:${encodeURIComponent(document.id)}#${slug(headingPath.join(' ')) || 'intro'}${part ? `~${part + 1}` : ''}`;
     const seen = used.get(base) ?? 0;
     used.set(base, seen + 1);
     const chunk = { id: seen ? `${base}-${seen + 1}` : base, docId: document.id, title: document.title, type: document.type, variant: document.variant, headingPath, url: anchor ? `${document.url}#${anchor}` : document.url, markdownUrl: document.markdownUrl, text };
@@ -219,8 +221,10 @@ export async function buildChunkIndex(directory, options = {}) {
         const { channel, version, language } = document.variant;
         chunks.push(...chunkMarkdown(markdown, { ...document, variant: `${channel}|${version}|${language}` }, anchors.get(document.url), options));
     }
-    const source = manifest.documents.map(document => `${document.id}:${document.digest}`).sort().join('\n');
-    const index = { schemaVersion: 1, sourceDigest: sha(source + (synonyms.length ? `\n${JSON.stringify(synonyms)}` : '')), chunks, ...(synonyms.length ? { synonyms } : {}) };
+    // Generation identity includes publication membership, routes, reader marks,
+    // and prepared search evidence. Embedding digests remain content-only.
+    const source = chunks.map(({ id, variant, digest, url, markdownUrl, readerFilters, search }) => ({ id, variant, digest, url, markdownUrl, readerFilters: Object.fromEntries(Object.entries(readerFilters ?? {}).sort(([a], [b]) => a.localeCompare(b)).map(([facet, values]) => [facet, [...values].sort()])), search })).sort((a, b) => a.id.localeCompare(b.id));
+    const index = { schemaVersion: 1, sourceDigest: sha(JSON.stringify({ identityVersion: 2, source, synonyms })), chunks, ...(synonyms.length ? { synonyms } : {}) };
     const output = path.join(await aiDirectory(root), 'chunks.json');
     await writeFile(output, JSON.stringify(index) + '\n');
     return { index, output, documents: manifest.documents.length };

package/dist/publication-render.d.ts

@@ -52,7 +52,7 @@ export declare function renderPublicationPages({ rendered, docs, relationships,
     header: HeaderConfig;
     footer?: FooterConfig;
     search: {
-        provider: 'lexical' | 'custom' | 'none';
+        provider: 'lexical' | 'semantic' | 'custom' | 'none';
     };
     readerFilters?: ReaderFiltersConfig;
     variants: Variant[];

package/dist/retrieval.d.ts

@@ -21,6 +21,7 @@ export declare class LexicalIndex {
     private readonly average;
     constructor(chunks: Chunk[], synonyms?: SynonymGroups);
     search(query: string, limit?: number, keep?: Keep): Hit[];
+    scan(query: string, keep?: Keep): IterableIterator<Hit>;
 }
 type VectorRow = {
     chunk: Chunk;
@@ -30,7 +31,9 @@ export declare class VectorIndex {
     private readonly rows;
     constructor(rows: VectorRow[]);
     search(query: Float32Array, limit?: number, keep?: Keep): Hit[];
+    scan(query: Float32Array, keep?: Keep): IterableIterator<Hit>;
 }
+export declare const compareHits: (left: Hit, right: Hit) => number;
 export declare function fuse(lists: Hit[][], limit?: number, k?: number): Hit[];
 export type Coverage = {
     total: number;
@@ -53,6 +56,7 @@ export declare class Retriever {
     private variant;
     pageIntroduction(docId: string, options?: LexicalOptions): Chunk | undefined;
     lexical(query: string, options?: LexicalOptions): Hit[];
+    searchPassages(query: string, queryVector: Float32Array | undefined, options?: LexicalOptions): Iterable<Hit>;
     vector(queryVector: Float32Array, options?: LexicalOptions): Hit[];
     hasVectors(options?: LexicalOptions): boolean;
     hybrid(query: string, queryVector: Float32Array | undefined, options?: LexicalOptions): Hit[];

package/dist/retrieval.js

@@ -67,10 +67,12 @@ export class LexicalIndex {
         this.average = this.entries.reduce((sum, entry) => sum + entry.length, 0) / (this.entries.length || 1);
     }
     search(query, limit = 8, keep) {
+        return [...this.scan(query, keep)].sort(compareHits).slice(0, limit);
+    }
+    *scan(query, keep) {
         const terms = [...new Set(tokenize(query))];
         const units = queryUnits(query, this.synonyms).filter(unit => unit.alternatives.length);
         const total = this.entries.length;
-        const hits = [];
         for (const entry of this.entries) {
             if (keep && !keep(entry.chunk))
                 continue;
@@ -93,9 +95,8 @@ export class LexicalIndex {
                 }
             }
             if (score > 0)
-                hits.push({ chunk: entry.chunk, score });
+                yield { chunk: entry.chunk, score };
         }
-        return hits.sort((left, right) => right.score - left.score || left.chunk.id.localeCompare(right.chunk.id)).slice(0, limit);
     }
 }
 function unit(vector) {
@@ -113,8 +114,10 @@ export class VectorIndex {
         this.rows = rows;
     }
     search(query, limit = 8, keep) {
+        return [...this.scan(query, keep)].sort(compareHits).slice(0, limit);
+    }
+    *scan(query, keep) {
         const q = unit(query);
-        const hits = [];
         for (const row of this.rows) {
             if (keep && !keep(row.chunk))
                 continue;
@@ -123,11 +126,11 @@ export class VectorIndex {
             let score = 0;
             for (let at = 0; at < q.length; at++)
                 score += q[at] * row.vector[at];
-            hits.push({ chunk: row.chunk, score });
+            yield { chunk: row.chunk, score };
         }
-        return hits.sort((left, right) => right.score - left.score || left.chunk.id.localeCompare(right.chunk.id)).slice(0, limit);
     }
 }
+export const compareHits = (left, right) => right.score - left.score || left.chunk.id.localeCompare(right.chunk.id);
 // Reciprocal rank fusion. Lexical and vector scores are not comparable, so only ranks are combined.
 export function fuse(lists, limit = 8, k = 60) {
     const scores = new Map();
@@ -149,10 +152,16 @@ export class Retriever {
     constructor(index, embeddings) {
         if (index?.schemaVersion !== 1 || !Array.isArray(index.chunks))
             throw new Error('Unsupported or invalid chunks.json');
+        if (new Set(index.chunks.map(chunk => chunk.id)).size !== index.chunks.length)
+            throw new Error('Ambiguous passage IDs in chunks.json; run shoalrun index <site-directory> to rebuild publication-qualified IDs. Existing passage embeddings can be reused.');
         const synonyms = validateSearchSynonyms(index.synonyms);
         const groups = new Map();
-        for (const chunk of index.chunks)
-            groups.set(chunk.variant, [...(groups.get(chunk.variant) ?? []), chunk]);
+        for (const chunk of index.chunks) {
+            let passages = groups.get(chunk.variant);
+            if (!passages)
+                groups.set(chunk.variant, passages = []);
+            passages.push(chunk);
+        }
         this.counts = { total: index.chunks.length, embedded: 0, stale: 0, ...(embeddings ? { model: embeddings.meta.model, dimension: embeddings.meta.dimension } : {}) };
         const rowOf = new Map((embeddings?.meta.ids ?? []).map((id, row) => [id, row]));
         for (const chunk of index.chunks)
@@ -209,6 +218,17 @@ export class Retriever {
     lexical(query, options = {}) {
         return this.byVariant.get(this.variant(options)).search(query, options.limit, keeping(options));
     }
+    // File-backed search still scans its selected publication, but streams scores
+    // into page selection instead of sorting/materializing every passage first.
+    searchPassages(query, queryVector, options = {}) {
+        const variant = this.variant(options);
+        if (!queryVector)
+            return this.byVariant.get(variant).scan(query, keeping(options));
+        const index = this.vectors.get(variant);
+        if (!index)
+            throw new Error('No current embeddings for this publication; run shoalrun index --embed');
+        return index.scan(queryVector, keeping(options));
+    }
     vector(queryVector, options = {}) {
         const index = this.vectors.get(this.variant(options));
         if (!index)

package/dist/search-handler.js

@@ -1,6 +1,7 @@
+import { compareHits } from './retrieval.js';
 import { validateReaderSelection } from './reader-filters.js';
 import { searchSnippetFromParagraphs } from './search-snippet.js';
-import { identifierRescue, searchIdentifiers } from './search-identifier.js';
+import { identifierMatch } from './search-identifier.js';
 import { createSearchResultSelector } from './search-results.js';
 export function createSearchHandler(options) {
     const selectResults = createSearchResultSelector(options.retriever);
@@ -91,17 +92,16 @@ export function createSearchHandler(options) {
             }
             if (!vector && fallbackReason)
                 options.onFallback?.(fallbackReason);
-            // Rank pages after inspecting their passages. A large gallery can otherwise
-            // consume the entire candidate window before page deduplication happens.
-            const retrievalOptions = { ...searchOptions, limit: Number.MAX_SAFE_INTEGER };
-            let hits = vector
-                ? options.retriever.vector(vector, retrievalOptions)
-                : options.retriever.lexical(query.trim(), retrievalOptions);
-            if (vector && limit >= 3 && searchIdentifiers(query).length) {
-                const lexical = options.retriever.lexical(query.trim(), retrievalOptions);
-                hits = identifierRescue(query.trim(), hits, lexical);
+            let candidate;
+            const matchesIdentifier = identifierMatch(query);
+            if (vector && limit >= 3 && matchesIdentifier) {
+                for (const hit of options.retriever.searchPassages(query.trim(), undefined, searchOptions)) {
+                    if (matchesIdentifier(hit) && (!candidate || compareHits(hit, candidate) < 0))
+                        candidate = hit;
+                }
             }
-            const results = selectResults(hits, query.trim(), searchOptions).map(({ chunk, literalSummary }) => ({ url: chunk.url, title: chunk.title, section: chunk.headingPath.at(-1), summary: literalSummary ?? searchSnippetFromParagraphs(chunk.search.paragraphs, query.trim(), options.pageSummaries?.get(`${chunk.variant}:${chunk.docId}`)) }));
+            const hits = selectResults(options.retriever.searchPassages(query.trim(), vector, searchOptions), query.trim(), searchOptions, candidate);
+            const results = hits.map(({ chunk, literalSummary }) => ({ url: chunk.url, title: chunk.title, section: chunk.headingPath.at(-1), summary: literalSummary ?? searchSnippetFromParagraphs(chunk.search.paragraphs, query.trim(), options.pageSummaries?.get(`${chunk.variant}:${chunk.docId}`)) }));
             return json(200, { results, mode: vector ? 'vector' : 'lexical', ...(!vector ? { fallbackReason } : {}) });
         }
         catch {

package/dist/search-identifier.d.ts

@@ -1,3 +1,4 @@
 import type { Hit } from './retrieval.js';
 export declare function searchIdentifiers(query: string): string[];
-export declare function identifierRescue(query: string, semantic: Hit[], lexical: Hit[]): Hit[];
+export declare function identifierRescue<T extends Hit>(query: string, semantic: T[], lexical: T[]): T[];
+export declare function identifierMatch(query: string): ((hit: Hit) => boolean) | undefined;

package/dist/search-identifier.js

@@ -7,11 +7,10 @@ export function searchIdentifiers(query) {
     return [...new Set(identifiers.filter(id => !/\.(?:txt|json|ya?ml|mdx?|[cm]?js|tsx?|html?|css|svg|png|jpe?g|com|org|net)$/i.test(id)))];
 }
 export function identifierRescue(query, semantic, lexical) {
-    const identifiers = searchIdentifiers(query);
-    if (!identifiers.length)
+    const matches = identifierMatch(query);
+    if (!matches)
         return semantic;
-    const matches = identifiers.map(id => new RegExp(`(?<![\\w$.-])${id.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?![\\w$-]|\\.[\\w$])`, 'i'));
-    const candidate = lexical.find(hit => (hit.chunk.search?.paragraphs ?? searchParagraphs(hit.chunk.text)).some(p => matches.every(re => re.test(p))));
+    const candidate = lexical.find(matches);
     if (!candidate)
         return semantic;
     const pages = [...new Set(semantic.map(h => h.chunk.docId))];
@@ -23,3 +22,10 @@ export function identifierRescue(query, semantic, lexical) {
     const position = at < 0 ? remaining.length : at;
     return [...remaining.slice(0, position), candidate, ...remaining.slice(position)];
 }
+export function identifierMatch(query) {
+    const identifiers = searchIdentifiers(query);
+    if (!identifiers.length)
+        return undefined;
+    const matches = identifiers.map(id => new RegExp(`(?<![\\w$.-])${id.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?![\\w$-]|\\.[\\w$])`, 'i'));
+    return hit => (hit.chunk.search?.paragraphs ?? searchParagraphs(hit.chunk.text)).some(p => matches.every(re => re.test(p)));
+}

package/dist/search-results.d.ts

@@ -2,5 +2,5 @@ import type { Hit, LexicalOptions, Retriever } from './retrieval.js';
 type SearchHit = Hit & {
     literalSummary?: string;
 };
-export declare function createSearchResultSelector(retriever: Retriever): (hits: Hit[], query: string, options: LexicalOptions) => SearchHit[];
+export declare function createSearchResultSelector(retriever: Retriever): (hits: Iterable<Hit>, query: string, options: LexicalOptions, rescue?: Hit) => SearchHit[];
 export {};

package/dist/search-results.js

@@ -1,45 +1,59 @@
-import { tokenize } from './retrieval.js';
+import { compareHits, tokenize } from './retrieval.js';
 import { literalSearch } from './search-literal.js';
+import { identifierRescue } from './search-identifier.js';
 // Select landing passages for the search box, without changing passage retrieval
 // used by Ask. A gallery's repeated examples must not crowd other pages out.
 export function createSearchResultSelector(retriever) {
     retriever.requirePreparedSearch();
     const normalized = (text) => tokenize(text).join(' ');
-    return (hits, query, options) => {
+    return (hits, query, options, rescue) => {
         const pages = new Map();
+        const normalizedQuery = normalized(query);
         const literal = literalSearch(query);
-        const literalPages = new Map();
-        if (literal) {
-            for (const hit of hits) {
-                if (literalPages.has(hit.chunk.docId))
-                    continue;
-                const literalSummary = literal(hit.chunk.search.literalLines);
-                if (literalSummary)
-                    literalPages.set(hit.chunk.docId, { ...hit, literalSummary });
-            }
-        }
+        let exact;
+        // Retain at most three candidates per page, independent of its passage
+        // count. Input scores need not be sorted or collected into an array.
         for (const hit of hits) {
-            if (pages.has(hit.chunk.docId))
+            const page = pages.get(hit.chunk.docId) ?? { exactTitle: normalized(hit.chunk.title) === normalizedQuery };
+            if (!page.first || compareHits(hit, page.first) < 0)
+                page.first = hit;
+            if (hit.chunk.search.paragraphs.length && (!page.prose || compareHits(hit, page.prose) < 0))
+                page.prose = hit;
+            const literalSummary = literal?.(hit.chunk.search.literalLines);
+            if (literalSummary && (!page.literal || compareHits(hit, page.literal) < 0))
+                page.literal = { ...hit, literalSummary };
+            pages.set(hit.chunk.docId, page);
+            if (page.exactTitle && (!exact || compareHits(hit, exact) < 0))
+                exact = hit;
+        }
+        const limit = options.limit ?? 8;
+        const candidates = [];
+        for (const page of pages.values()) {
+            const candidate = page.literal ? { rank: page.first, hit: { ...page.literal, score: page.first.score } } : page.prose ? { rank: page.prose, hit: page.prose } : undefined;
+            if (!candidate)
                 continue;
-            // Keep the page's retrieved position, but land on the passage that actually
-            // contains the literal. A nearby token table must not hide the correct row.
-            const exactPassage = literalPages.get(hit.chunk.docId);
-            if (exactPassage)
-                pages.set(hit.chunk.docId, { ...exactPassage, score: hit.score });
-            else if (hit.chunk.search.paragraphs.length)
-                pages.set(hit.chunk.docId, hit);
+            const at = candidates.findIndex(other => compareHits(candidate.rank, other.rank) < 0);
+            candidates.splice(at < 0 ? candidates.length : at, 0, candidate);
+            if (candidates.length > limit)
+                candidates.pop();
+        }
+        let selected = candidates.map(item => item.hit);
+        if (rescue) {
+            selected = identifierRescue(query, selected, [rescue]);
+            const landing = pages.get(rescue.chunk.docId)?.literal;
+            if (landing)
+                selected = selected.map(hit => hit.chunk.docId === rescue.chunk.docId ? { ...landing, score: hit.score } : hit);
         }
         // An exact page-title query is a request for that topic's entry point.
         // Ordinary semantic queries retain their ranking; incidental words do not qualify.
-        const exact = hits.find(hit => normalized(hit.chunk.title) === normalized(query));
         if (exact) {
             const introduction = retriever.pageIntroduction(exact.chunk.docId, options);
-            const landing = introduction ? { chunk: introduction, score: exact.score } : pages.get(exact.chunk.docId);
+            const page = pages.get(exact.chunk.docId);
+            const landing = introduction ? { chunk: introduction, score: exact.score } : page?.literal ?? page?.prose;
             if (landing) {
-                pages.delete(exact.chunk.docId);
-                return [landing, ...pages.values()].slice(0, options.limit ?? 8);
+                return [landing, ...selected.filter(hit => hit.chunk.docId !== exact.chunk.docId)].slice(0, options.limit ?? 8);
             }
         }
-        return [...pages.values()].slice(0, options.limit ?? 8);
+        return selected.slice(0, limit);
     };
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@shoalrun/cli",
-  "version": "0.1.0-alpha.15",
+  "version": "0.1.0-alpha.16",
   "publishConfig": {
     "registry": "https://registry.npmjs.org",
     "access": "public"

package/skills/shoalrun-setup/SKILL.md

@@ -33,4 +33,6 @@ For a new OpenAPI source, read `guide.api-references` and complete its API-sourc
 
 Check the generated HTML, Markdown twins, and `documents.json`. Confirm navigation and API links resolve, and inspect assets inside the components that contain them. Run `shoalrun audit docs.config.yaml` when project checks or API audits are configured; documentation behavior tests can use Doc Detective through `audit.tests` or another external CLI.
 
+For optional semantic search, use `outputs.html.search.provider: semantic` (available since `0.1.0-alpha.16`) with `retrieval.embeddings`; do not generate a custom browser adapter for Shoalrun's own endpoint. Build the site, run `shoalrun index <output> --embed --config docs.config.yaml`, then `shoalrun serve <output> --search --config docs.config.yaml` with the provider key only in the environment. Confirm the endpoint returns `mode: vector`; a visible result or keyword fallback does not prove semantic retrieval. Selecting the browser provider does not provision production hosting, and profiled Vercel chat-function builds remain unsupported.
+
 Report the files created, commands run, and anything still needing an owner decision. Do not deploy, publish a release, or claim factual accuracy from a successful build alone.