revit-api-docs 0.2.0__tar.gz → 0.3.0__tar.gz

{revit_api_docs-0.2.0 → revit_api_docs-0.3.0}/CHANGELOG.md

@@ -6,6 +6,116 @@ All notable changes to this project are documented here. The format follows
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-10-26
+
+Vector hits now come back as the entries they describe and count also
+without a reranker; reranking picks its provider from the keys that are set.
+The data set `v1.1-data` adds constructor and interface member pages, and its
+vector store lines up with the SQLite rows.
+
+**Upgrading from 0.2.0** (`uvx --refresh revit-api-docs`): on the first start the server replaces an installed
+`v1.0-data` (its `manifest.json` names the old release) with `v1.1-data`, a
+download of 267 MB (all four files); a data directory without a
+manifest (a hand-copied tree) is used as it is. With an embedding key and a
+tree that still holds `v1.0-data`, the first vector query logs the warning
+`vector_store_misaligned`: its record ids do not line up with the SQLite rows,
+and hits are matched by entry name. Without any key nothing changes beyond the
+new entries. Reranker choice: with only `COHERE_API_KEY` set Cohere still
+reranks; with `TYPESAFE_API_KEY` set Jev now reranks without
+`REVIT_API_DOCS_RERANK_PROVIDER=jev` (set the provider to `cohere` to keep
+Cohere).
+
+### Added
+
+- Data set `v1.1-data` (`RELEASE_TAG`; the download URL and SHA-256 list in
+  `data.py` follow it): the 27,596 entries of `v1.0-data` are kept unchanged
+  under the same ids, and 1,266 pages of the same reference are appended
+  (ids 27,597-28,862): 1,040 constructor pages (`XYZ(Double, Double, Double)
+  Constructor`, `Transaction(Document, String) Constructor`, overload lists)
+  and 226 members whose C# signature has no `public`, mostly interface
+  members (`IExternalEventHandler.Execute`, `IConnector.Origin`,
+  `IFailuresPreprocessor.PreprocessFailures`) and explicit interface
+  implementations. 28,862 entries in all. Source: `RevitAPI.chm` of the
+  Revit 2026.3 SDK, whose pages are the same set as the Revit 2026 reference
+  the first release was built from. The API vector store is rebuilt from the
+  new SQLite with the same model (`text-embedding-3-large`, 3072 dimensions):
+  one record per row and every record id is its row id, so the first vector
+  query no longer logs `vector_store_misaligned` (`download` reports
+  `aligned`). The SDK files (`revit_sdk.db`, `chromadb_code.tar.gz`) are the
+  `v1.0-data` ones.
+- The vector documents are now `name : summary` (`v1.0-data` embedded mostly
+  the entry name alone: 21,952 of its 28,863 records); on the 223 test
+  queries this raised fused nDCG@5 without a reranker from 0.610 to 0.647
+  (task requests 0.438 to 0.459) and left it with Jev at 0.842 against 0.846
+  (task requests 0.778 against 0.794).
+- `pipeline/build_data_v1_1.py` (sdist only): builds `v1.1-data` from the
+  `v1.0-data` SQLite and the CHM pages: appends the pages the base lacks
+  (pages whose name is in the base under another `full_id`, or whose content
+  is a base row's under a name the v1.0 rewrite changed, are listed and not
+  appended), runs the quality agent on the new rows only, re-embeds every
+  row, packages the four release files with `SHA256SUMS.txt` and checks the
+  vector store against the rows without an API call.
+
+### Fixed
+
+- Vector search returned unrelated entries. The vector store of the
+  `v1.0-data` release numbers its 28,863 API records independently of the
+  27,596 rows of `revit_api.db` (2 ids name the same entry in both), and hits
+  were looked up by record id. A hit is now matched to the row whose entry
+  name equals the record's `name` metadata (the row with the record's own id
+  when it carries that name; among rows sharing a name the one with the
+  record's `full_id` or `info`, else the lowest id). 27,591 records (95.6%)
+  reach 27,512 rows (99.7%); records without a row are dropped. The SDK sample
+  store's ids match its rows, and matching by project name keeps them. The
+  name map is built on the first vector query and kept in memory.
+- Consistency probe: on the first vector query 50 records of each vector
+  collection are compared with SQLite. When fewer than 90% of the sampled ids
+  name the same entry as the row with that id, the warning
+  `vector_store_misaligned` is logged and hits are matched by name (`v1.0-data`:
+  0/50 ids, 49/50 names). When fewer than 50% of the sampled names are found
+  either, vector search is off for the process and keyword search answers
+  alone. `revit-api-docs download` prints the check for the installed data set.
+- A second retriever in the same process could not open the vector store
+  ("An instance of Chroma already exists ... with different settings"): each
+  ChromaDB client now gets its own `Settings` object.
+
+### Changed
+
+- Hybrid ranking by reciprocal rank fusion: the vector hits and the BM25 hits,
+  `retrieval.api.top_k` (30) of each, are fused with score = sum of
+  1 / (60 + rank), and the best 30 go to the reranker; without a reranker they
+  are cut to `top_k`. On equal scores a keyword hit comes first. Before, the
+  two lists were merged by distances on different scales, so without a
+  reranker vector-only hits never reached the top 5, and up to 60 candidates
+  went to the reranker. Without an embedding key the BM25 order and results
+  are unchanged. On 223 English test queries without reranking (measured on
+  `v1.1-data`): nDCG@5 0.557 with BM25 alone, 0.647 fused, and 0.355 -> 0.459
+  on the 113 multi-step task requests, whose answer is among the top 30 for
+  82% instead of 72% of them.
+- The default rerank provider is `auto`: Jev when `TYPESAFE_API_KEY` is set,
+  otherwise Cohere when `COHERE_API_KEY` is set, otherwise none.
+  `REVIT_API_DOCS_RERANK_PROVIDER` (now `auto`, `jev`, `cohere` or `none`)
+  still wins, and the `Rerank:` line names the provider that ran. README:
+  Jev is the recommended reranker, Cohere optional and the fallback for Jev.
+- Cohere requests time out after 20 s instead of 10 s (the same as Jev); 10 s
+  cut off about 1% of calls.
+- `search_revit_api` and `get_code_examples` are unchanged: names,
+  parameters and the first output line. The tool description now counts
+  28,862 entries.
+- CHM parser (`pipeline/api_parser/parse_chm.py`): `parse_single_html` and
+  `parse_all_api_html` take `keep_constructors` (default `False`, the old
+  behaviour); the C# signature no longer has to contain `public` (interface
+  members such as `void Execute(UIApplication app)` were skipped as a whole
+  page); a page with neither a signature nor a members table is still
+  skipped.
+- `pipeline.embedder.embed.embed_api_data(..., resume=False,
+  document_template="NS")`: with `resume=True` the collection is kept and
+  only missing row ids are embedded; `document_template` picks the embedded
+  text per API row (`NS` `name : summary`, the default and the `v1.1-data`
+  store; `FS` `full_id : summary`, the earlier behaviour; `N` the entry name)
+  and is recorded in `meta.json`. `pipeline/build_data_v1_1.py
+  --document-template` passes it on (default `NS`).
+
 ## [0.2.0] - 2026-10-26
 
 Better candidates without any key, and an opt-in relevance check: BM25 keyword
@@ -147,6 +257,7 @@ no git history).
 - CI (tests + build on Linux/Windows, Python 3.11/3.13) and PyPI Trusted
   Publishing on version tags.
 
-[Unreleased]: https://github.com/revitbridge/revit-api-docs/compare/v0.2.0...HEAD
+[Unreleased]: https://github.com/revitbridge/revit-api-docs/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/revitbridge/revit-api-docs/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/revitbridge/revit-api-docs/compare/v0.1.0...v0.2.0
 [0.1.0]: https://github.com/revitbridge/revit-api-docs/releases/tag/v0.1.0

{revit_api_docs-0.2.0 → revit_api_docs-0.3.0}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.5
 Name: revit-api-docs
-Version: 0.2.0
+Version: 0.3.0
 Summary: Revit API reference search for MCP hosts: local semantic + keyword lookup over the Revit 2026 API docs and SDK samples
 Project-URL: Homepage, https://github.com/revitbridge/revit-api-docs
 Project-URL: Issues, https://github.com/revitbridge/revit-api-docs/issues
 Project-URL: Changelog, https://github.com/revitbridge/revit-api-docs/blob/main/CHANGELOG.md
-Project-URL: Data, https://github.com/revitbridge/revit-api-docs/releases/tag/v1.0-data
+Project-URL: Data, https://github.com/revitbridge/revit-api-docs/releases/tag/v1.1-data
 Author: revitbridge
 License-Expression: MIT
 License-File: LICENSE
@@ -35,8 +35,8 @@ Description-Content-Type: text/markdown
 # revit-api-docs
 
 Revit API reference search as an MCP server: two read-only tools over a local
-copy of the Revit 2026 API documentation (27,596 classes, methods, properties
-and enums) and the official SDK samples (153 projects).
+copy of the Revit 2026 API documentation (28,862 classes, constructors,
+methods, properties and enums) and the official SDK samples (153 projects).
 
 **Optional.** It exists for models that do not know the Revit API well. If
 your model already writes correct Revit code, you do not need it;
@@ -97,8 +97,8 @@ keep it on a loopback address, or put it behind a reverse proxy that
 authenticates; the server logs a warning when `--host` is not a loopback
 address.
 
-On the first start the server downloads the data set (335 MB, four files) from
-this repository's [`v1.0-data` release](https://github.com/revitbridge/revit-api-docs/releases/tag/v1.0-data)
+On the first start the server downloads the data set (267 MB, four files) from
+this repository's [`v1.1-data` release](https://github.com/revitbridge/revit-api-docs/releases/tag/v1.1-data)
 into `%LOCALAPPDATA%\revit-api-docs` (Windows) or `~/.local/share/revit-api-docs`
 (Linux; `~/Library/Application Support/revit-api-docs` on macOS), verifies each
 file's SHA-256 and unpacks the vector stores. Tool calls made while that runs
@@ -133,11 +133,13 @@ reference: BM25 over a full-text index of entry names, IDs, summaries and
 signatures, where identifiers also match by their camel-case parts, so a
 misspelt or unknown name still gets candidates. The index (11.8 MB) is built
 next to the data set on first use (under a second). With an embedding key it
-adds semantic search over the vector store, and with a Cohere key it reranks
-the merged candidates; with a TypeSafe key it can rerank with Jev instead, which
-returns a calibrated relevance per entry. With a threshold set and no entry
-reaching it, the output starts with an abstention line and then lists the
-candidates with their relevance, for reference. See Configure.
+adds semantic search over the vector store and fuses the two rankings by rank,
+which helps most with task descriptions that do not name the API. With a
+TypeSafe key it reranks the candidates with Jev, which returns a calibrated
+relevance per entry; with only a Cohere key it reranks with Cohere. With a Jev
+threshold set and no entry reaching it, the output starts with an abstention
+line and then lists the candidates with their relevance, for reference. See
+Configure.
 
 Try it from a terminal (installs the data set if needed):
 
@@ -158,39 +160,72 @@ All settings are environment variables; there is no config file to edit.
 | `REVIT_API_DOCS_EMBEDDING_API_KEY` | *(unset)* | Key for the embedding endpoint; turns on semantic (vector) search. `OPENROUTER_API_KEY` is used when this is unset |
 | `REVIT_API_DOCS_EMBEDDING_BASE_URL` | `https://openrouter.ai/api/v1` | Any OpenAI-compatible `/embeddings` endpoint |
 | `REVIT_API_DOCS_EMBEDDING_MODEL` | `openai/text-embedding-3-large` | Must produce the same 3072-dimensional space the shipped vectors were built with (`text-embedding-3-large`); another model makes vector search return noise |
-| `COHERE_API_KEY` | *(unset)* | Enables reranking with Cohere `rerank-v3.5` (native REST API) |
-| `REVIT_API_DOCS_RERANK_PROVIDER` | `cohere` | `cohere`, `jev` or `none`. With `jev`, a failed or unconfigured Jev call falls back to Cohere when `COHERE_API_KEY` is set, otherwise to no reranking; the first output line says which one ran |
-| `TYPESAFE_API_KEY` | *(unset)* | Key for TypeSafe System One (Jev), used when the provider is `jev`. At most 30 candidates per request, 2 requests in flight, 2 retries, 20 s timeout |
+| `TYPESAFE_API_KEY` | *(unset)* | Key for TypeSafe System One (Jev), the recommended reranker: with the default provider `auto` setting this key is enough. At most 30 candidates per request, 2 requests in flight, 2 retries, 20 s timeout |
+| `COHERE_API_KEY` | *(unset)* | Optional. Cohere `rerank-v3.5` (native REST API, 20 s timeout): reranks when it is the only rerank key set, and is the fallback for a failed Jev call when both are set |
+| `REVIT_API_DOCS_RERANK_PROVIDER` | `auto` | `auto`, `jev`, `cohere` or `none`. `auto` reranks with Jev when `TYPESAFE_API_KEY` is set, otherwise with Cohere when `COHERE_API_KEY` is set, otherwise not at all; an explicit value wins. With Jev, a failed or unconfigured call falls back to Cohere when `COHERE_API_KEY` is set, otherwise to no reranking; the first output line says which one ran |
 | `REVIT_API_DOCS_JEV_MODEL` | `jev-latest` | Jev model or alias; pin a version such as `jev-1.14.0` for reproducible results |
 | `REVIT_API_DOCS_MIN_RELEVANCE` | *(unset)* | Probability 0-1 (`retrieval.api.min_relevance`). Only honoured while Jev ranks the results, and it only decides whether to abstain: no entry is dropped, every result keeps its `Relevance` line. When no entry reaches it, `search_revit_api` starts with "No entry in the Revit 2026 data set scored high enough for this query", notes that the member may be absent from this version or listed under a different name or wording, and lists the low-scoring candidates for reference. No effect with Cohere or without reranking. See "Reranking with Jev" below |
-| `REVIT_API_DOCS_RELEASE_URL` | this repository's `v1.0-data` release | Mirror for the data files; the SHA-256 checks still apply |
+| `REVIT_API_DOCS_RELEASE_URL` | this repository's `v1.1-data` release | Mirror for the data files; the SHA-256 checks still apply |
 | `HTTPS_PROXY` / `HTTP_PROXY` | *(unset)* | Proxy for the embedding and rerank calls |
 
 Pass them through the `env` block of your MCP host's server entry.
 
-**Reranking with Jev.** If you have a `TYPESAFE_API_KEY`,
-`REVIT_API_DOCS_RERANK_PROVIDER=jev` is the recommended setting (the default
-provider stays `cohere`). On 60 English test queries (50 with a known answer)
-over the keyword (BM25) candidates, with no embedding key, Jev raised nDCG@5
-from 0.675 to 0.871 and never turned a correct top-5 answer into a miss.
-Input tokens cost about $0.23 per 1,000 queries.
+Recommended setup: a `TYPESAFE_API_KEY` (Jev reranking), plus an embedding key
+when requests describe multi-step tasks rather than name APIs. Without any key
+the server runs on BM25 keyword search alone.
+
+**Reranking with Jev.** Jev is the recommended reranker, and the default
+provider (`auto`) uses it whenever `TYPESAFE_API_KEY` is set. Cohere is
+optional: `auto` uses it when only `COHERE_API_KEY` is set, and with both keys
+it is the fallback for a failed Jev call. Measured on `v1.0-data`: on 60
+English test queries (50 with a known answer) over the keyword (BM25)
+candidates, with no embedding key, Jev raised nDCG@5 from 0.675 to 0.871 and
+never turned a correct top-5 answer into a miss. On 223 further queries over
+the same candidates Jev reached an nDCG@5 of 0.788 against Cohere's 0.674. Input tokens cost about $0.23 per 1,000
+queries; Cohere bills $2 per 1,000 searches.
 `REVIT_API_DOCS_MIN_RELEVANCE` stays unset by default. Set to 0.15 it filters
 out clearly unrelated queries (questions that are not about Revit, types this
 data set does not have), abstaining on about 2.5% of the queries that do have
-an answer. It cannot spot a misspelt or misplaced member name: such a query
-usually gets the closest real API as the first result, to be confirmed with a
-compile check. **Do not use 0.5**: it abstained on 14% of the queries that
-have an answer. What TypeSafe keeps: see the `jev-data-handling` note
+an answer (measured on `v1.0-data`). It cannot spot a misspelt or misplaced
+member name: such a query usually gets the closest real API as the first
+result, to be confirmed with a compile check. **Do not use 0.5**: it
+abstained on 14% of the queries that have an answer. What TypeSafe keeps: see the `jev-data-handling` note
 (`uvx revit-api-docs environment typesafe`).
 
+**Vector search.** An embedding key (`REVIT_API_DOCS_EMBEDDING_API_KEY`, or
+`OPENROUTER_API_KEY`) turns it on; each query then costs one embedding request
+(about 0.9 s). The vector hits and the BM25 hits, 30 of each, are fused by
+reciprocal rank (score = sum of 1 / (60 + rank)) and the best 30 go to the
+reranker; without a reranker they are cut to `top_k`. On 223 English test
+queries without reranking (measured on `v1.1-data`), fusion raises nDCG@5
+from 0.557 (BM25 alone) to 0.647, and on the 113 of them that describe a
+multi-step task from 0.355 to 0.459, with an answer among the top 30 for 82%
+of those instead of 72%. Each
+vector hit is matched to its entry by name. On the first vector query the
+server samples 50 records of each vector collection: when the record ids do
+not line up with the rows it logs the warning `vector_store_misaligned` and
+keeps matching by name (the older `v1.0-data` vector store numbers its records
+differently from the rows of `revit_api.db`; `v1.1-data` is aligned and logs
+nothing); when the names are not found either, it turns vector search off
+until the server restarts and answers from keyword search. `revit-api-docs download`
+prints the same check for the installed data set.
+
 Privacy: with an embedding, Cohere or TypeSafe key the query text and the
 candidate API entries (public documentation) are sent to that provider. Keep
 queries to API names and generic task descriptions, in English.
 
-Data set: the SQLite databases and ChromaDB indexes were built from the Revit
-2026 API CHM and the Revit 2026 SDK samples with the pipeline in `pipeline/`
-(`uv sync --extra pipeline`; needs the CHM, the SDK and OpenRouter/Cohere keys
-in the environment). The server does not need any of that.
+Data set (`v1.1-data`): the API reference comes from `RevitAPI.chm` of the
+Revit 2026.3 SDK (the same pages as the Revit 2026 reference) and includes
+constructor pages and interface members such as
+`IExternalEventHandler.Execute`: 28,862 entries, of which the 27,596 of
+`v1.0-data` are unchanged under the same ids. The SDK samples (153 projects)
+come from the Revit 2026 SDK. The SQLite databases and ChromaDB indexes were
+built with the pipeline in `pipeline/` (`uv sync --extra pipeline`; needs the
+CHM, the SDK and OpenRouter/Cohere keys in the environment). `v1.1-data` was
+built from `v1.0-data` with `uv run python -m pipeline.build_data_v1_1
+--html-dir <CHM html dir> --base-db <v1.0-data dir>/sqlite/revit_api.db`, which
+appends the pages `v1.0-data` lacked and rebuilds the API vector store so that
+every record id is its row id. The server does not need any of that.
 
 Development:
 

{revit_api_docs-0.2.0 → revit_api_docs-0.3.0}/README.md

@@ -1,8 +1,8 @@
 # revit-api-docs
 
 Revit API reference search as an MCP server: two read-only tools over a local
-copy of the Revit 2026 API documentation (27,596 classes, methods, properties
-and enums) and the official SDK samples (153 projects).
+copy of the Revit 2026 API documentation (28,862 classes, constructors,
+methods, properties and enums) and the official SDK samples (153 projects).
 
 **Optional.** It exists for models that do not know the Revit API well. If
 your model already writes correct Revit code, you do not need it;
@@ -63,8 +63,8 @@ keep it on a loopback address, or put it behind a reverse proxy that
 authenticates; the server logs a warning when `--host` is not a loopback
 address.
 
-On the first start the server downloads the data set (335 MB, four files) from
-this repository's [`v1.0-data` release](https://github.com/revitbridge/revit-api-docs/releases/tag/v1.0-data)
+On the first start the server downloads the data set (267 MB, four files) from
+this repository's [`v1.1-data` release](https://github.com/revitbridge/revit-api-docs/releases/tag/v1.1-data)
 into `%LOCALAPPDATA%\revit-api-docs` (Windows) or `~/.local/share/revit-api-docs`
 (Linux; `~/Library/Application Support/revit-api-docs` on macOS), verifies each
 file's SHA-256 and unpacks the vector stores. Tool calls made while that runs
@@ -99,11 +99,13 @@ reference: BM25 over a full-text index of entry names, IDs, summaries and
 signatures, where identifiers also match by their camel-case parts, so a
 misspelt or unknown name still gets candidates. The index (11.8 MB) is built
 next to the data set on first use (under a second). With an embedding key it
-adds semantic search over the vector store, and with a Cohere key it reranks
-the merged candidates; with a TypeSafe key it can rerank with Jev instead, which
-returns a calibrated relevance per entry. With a threshold set and no entry
-reaching it, the output starts with an abstention line and then lists the
-candidates with their relevance, for reference. See Configure.
+adds semantic search over the vector store and fuses the two rankings by rank,
+which helps most with task descriptions that do not name the API. With a
+TypeSafe key it reranks the candidates with Jev, which returns a calibrated
+relevance per entry; with only a Cohere key it reranks with Cohere. With a Jev
+threshold set and no entry reaching it, the output starts with an abstention
+line and then lists the candidates with their relevance, for reference. See
+Configure.
 
 Try it from a terminal (installs the data set if needed):
 
@@ -124,39 +126,72 @@ All settings are environment variables; there is no config file to edit.
 | `REVIT_API_DOCS_EMBEDDING_API_KEY` | *(unset)* | Key for the embedding endpoint; turns on semantic (vector) search. `OPENROUTER_API_KEY` is used when this is unset |
 | `REVIT_API_DOCS_EMBEDDING_BASE_URL` | `https://openrouter.ai/api/v1` | Any OpenAI-compatible `/embeddings` endpoint |
 | `REVIT_API_DOCS_EMBEDDING_MODEL` | `openai/text-embedding-3-large` | Must produce the same 3072-dimensional space the shipped vectors were built with (`text-embedding-3-large`); another model makes vector search return noise |
-| `COHERE_API_KEY` | *(unset)* | Enables reranking with Cohere `rerank-v3.5` (native REST API) |
-| `REVIT_API_DOCS_RERANK_PROVIDER` | `cohere` | `cohere`, `jev` or `none`. With `jev`, a failed or unconfigured Jev call falls back to Cohere when `COHERE_API_KEY` is set, otherwise to no reranking; the first output line says which one ran |
-| `TYPESAFE_API_KEY` | *(unset)* | Key for TypeSafe System One (Jev), used when the provider is `jev`. At most 30 candidates per request, 2 requests in flight, 2 retries, 20 s timeout |
+| `TYPESAFE_API_KEY` | *(unset)* | Key for TypeSafe System One (Jev), the recommended reranker: with the default provider `auto` setting this key is enough. At most 30 candidates per request, 2 requests in flight, 2 retries, 20 s timeout |
+| `COHERE_API_KEY` | *(unset)* | Optional. Cohere `rerank-v3.5` (native REST API, 20 s timeout): reranks when it is the only rerank key set, and is the fallback for a failed Jev call when both are set |
+| `REVIT_API_DOCS_RERANK_PROVIDER` | `auto` | `auto`, `jev`, `cohere` or `none`. `auto` reranks with Jev when `TYPESAFE_API_KEY` is set, otherwise with Cohere when `COHERE_API_KEY` is set, otherwise not at all; an explicit value wins. With Jev, a failed or unconfigured call falls back to Cohere when `COHERE_API_KEY` is set, otherwise to no reranking; the first output line says which one ran |
 | `REVIT_API_DOCS_JEV_MODEL` | `jev-latest` | Jev model or alias; pin a version such as `jev-1.14.0` for reproducible results |
 | `REVIT_API_DOCS_MIN_RELEVANCE` | *(unset)* | Probability 0-1 (`retrieval.api.min_relevance`). Only honoured while Jev ranks the results, and it only decides whether to abstain: no entry is dropped, every result keeps its `Relevance` line. When no entry reaches it, `search_revit_api` starts with "No entry in the Revit 2026 data set scored high enough for this query", notes that the member may be absent from this version or listed under a different name or wording, and lists the low-scoring candidates for reference. No effect with Cohere or without reranking. See "Reranking with Jev" below |
-| `REVIT_API_DOCS_RELEASE_URL` | this repository's `v1.0-data` release | Mirror for the data files; the SHA-256 checks still apply |
+| `REVIT_API_DOCS_RELEASE_URL` | this repository's `v1.1-data` release | Mirror for the data files; the SHA-256 checks still apply |
 | `HTTPS_PROXY` / `HTTP_PROXY` | *(unset)* | Proxy for the embedding and rerank calls |
 
 Pass them through the `env` block of your MCP host's server entry.
 
-**Reranking with Jev.** If you have a `TYPESAFE_API_KEY`,
-`REVIT_API_DOCS_RERANK_PROVIDER=jev` is the recommended setting (the default
-provider stays `cohere`). On 60 English test queries (50 with a known answer)
-over the keyword (BM25) candidates, with no embedding key, Jev raised nDCG@5
-from 0.675 to 0.871 and never turned a correct top-5 answer into a miss.
-Input tokens cost about $0.23 per 1,000 queries.
+Recommended setup: a `TYPESAFE_API_KEY` (Jev reranking), plus an embedding key
+when requests describe multi-step tasks rather than name APIs. Without any key
+the server runs on BM25 keyword search alone.
+
+**Reranking with Jev.** Jev is the recommended reranker, and the default
+provider (`auto`) uses it whenever `TYPESAFE_API_KEY` is set. Cohere is
+optional: `auto` uses it when only `COHERE_API_KEY` is set, and with both keys
+it is the fallback for a failed Jev call. Measured on `v1.0-data`: on 60
+English test queries (50 with a known answer) over the keyword (BM25)
+candidates, with no embedding key, Jev raised nDCG@5 from 0.675 to 0.871 and
+never turned a correct top-5 answer into a miss. On 223 further queries over
+the same candidates Jev reached an nDCG@5 of 0.788 against Cohere's 0.674. Input tokens cost about $0.23 per 1,000
+queries; Cohere bills $2 per 1,000 searches.
 `REVIT_API_DOCS_MIN_RELEVANCE` stays unset by default. Set to 0.15 it filters
 out clearly unrelated queries (questions that are not about Revit, types this
 data set does not have), abstaining on about 2.5% of the queries that do have
-an answer. It cannot spot a misspelt or misplaced member name: such a query
-usually gets the closest real API as the first result, to be confirmed with a
-compile check. **Do not use 0.5**: it abstained on 14% of the queries that
-have an answer. What TypeSafe keeps: see the `jev-data-handling` note
+an answer (measured on `v1.0-data`). It cannot spot a misspelt or misplaced
+member name: such a query usually gets the closest real API as the first
+result, to be confirmed with a compile check. **Do not use 0.5**: it
+abstained on 14% of the queries that have an answer. What TypeSafe keeps: see the `jev-data-handling` note
 (`uvx revit-api-docs environment typesafe`).
 
+**Vector search.** An embedding key (`REVIT_API_DOCS_EMBEDDING_API_KEY`, or
+`OPENROUTER_API_KEY`) turns it on; each query then costs one embedding request
+(about 0.9 s). The vector hits and the BM25 hits, 30 of each, are fused by
+reciprocal rank (score = sum of 1 / (60 + rank)) and the best 30 go to the
+reranker; without a reranker they are cut to `top_k`. On 223 English test
+queries without reranking (measured on `v1.1-data`), fusion raises nDCG@5
+from 0.557 (BM25 alone) to 0.647, and on the 113 of them that describe a
+multi-step task from 0.355 to 0.459, with an answer among the top 30 for 82%
+of those instead of 72%. Each
+vector hit is matched to its entry by name. On the first vector query the
+server samples 50 records of each vector collection: when the record ids do
+not line up with the rows it logs the warning `vector_store_misaligned` and
+keeps matching by name (the older `v1.0-data` vector store numbers its records
+differently from the rows of `revit_api.db`; `v1.1-data` is aligned and logs
+nothing); when the names are not found either, it turns vector search off
+until the server restarts and answers from keyword search. `revit-api-docs download`
+prints the same check for the installed data set.
+
 Privacy: with an embedding, Cohere or TypeSafe key the query text and the
 candidate API entries (public documentation) are sent to that provider. Keep
 queries to API names and generic task descriptions, in English.
 
-Data set: the SQLite databases and ChromaDB indexes were built from the Revit
-2026 API CHM and the Revit 2026 SDK samples with the pipeline in `pipeline/`
-(`uv sync --extra pipeline`; needs the CHM, the SDK and OpenRouter/Cohere keys
-in the environment). The server does not need any of that.
+Data set (`v1.1-data`): the API reference comes from `RevitAPI.chm` of the
+Revit 2026.3 SDK (the same pages as the Revit 2026 reference) and includes
+constructor pages and interface members such as
+`IExternalEventHandler.Execute`: 28,862 entries, of which the 27,596 of
+`v1.0-data` are unchanged under the same ids. The SDK samples (153 projects)
+come from the Revit 2026 SDK. The SQLite databases and ChromaDB indexes were
+built with the pipeline in `pipeline/` (`uv sync --extra pipeline`; needs the
+CHM, the SDK and OpenRouter/Cohere keys in the environment). `v1.1-data` was
+built from `v1.0-data` with `uv run python -m pipeline.build_data_v1_1
+--html-dir <CHM html dir> --base-db <v1.0-data dir>/sqlite/revit_api.db`, which
+appends the pages `v1.0-data` lacked and rebuilds the API vector store so that
+every record id is its row id. The server does not need any of that.
 
 Development:
 

{revit_api_docs-0.2.0 → revit_api_docs-0.3.0}/pipeline/api_parser/parse_chm.py

@@ -19,13 +19,19 @@ from pathlib import Path
 from bs4 import BeautifulSoup
 
 
-def parse_single_html(html_path: str) -> dict | None:
+def parse_single_html(html_path: str, *, keep_constructors: bool = False) -> dict | None:
     """
     解析单个 API HTML 文件，提取结构化数据
 
+    Args:
+        html_path:         HTML 文件路径
+        keep_constructors: True 时保留构造函数页（data release v1.1 起收录）；
+                           默认 False，与旧行为相同（丢弃构造函数页）
+
     Returns:
         dict with keys: name, full_id, namespace, content_type, info, summary,
-                        remark, parameters, exceptions, members, keywords
+                        remark, parameters, exceptions, members, keywords,
+                        _source_file（源 HTML 路径，供 quality_agent 读取原文；不入 SQLite）
         如果不是有效的 API 文档页面返回 None
     """
     try:
@@ -52,8 +58,9 @@ def parse_single_html(html_path: str) -> dict | None:
         return None
 
     # --- 过滤构造函数页面（ctor） ---
-    # 构造函数仅描述如何实例化对象，不含方法逻辑，对 RAG 检索意义有限
-    if _is_constructor_page(title, full_id):
+    # 旧版默认丢弃构造函数页；keep_constructors=True 时保留（v1.1-data 起收录，
+    # `new XYZ(x, y, z)`、`new Transaction(doc, name)` 之类的查询需要它们）
+    if not keep_constructors and _is_constructor_page(title, full_id):
         return None
 
     # 命名空间
@@ -184,8 +191,10 @@ def _extract_api_fullname(soup: BeautifulSoup, title: str) -> str | None:
 
 def _extract_csharp_signature(soup: BeautifulSoup) -> str:
     """
-    从 HTML 中提取 C# 方法签名，仅保留包含 public 的签名。
-    对应旧版 get_csharp_full_name 逻辑。
+    从 HTML 中提取 C# 签名（IDAB_code_Div1 的 <pre>，非空即可）。
+    对应旧版 get_csharp_full_name 逻辑；旧版只保留含 public 的签名，接口成员
+    （如 `void Execute(UIApplication app)`）与显式接口实现因此被整页跳过，
+    现在不再要求 public。
     """
     div = soup.find("div", id="IDAB_code_Div1")
     if not div:
@@ -196,7 +205,7 @@ def _extract_csharp_signature(soup: BeautifulSoup) -> str:
         return ""
 
     text = pre.get_text(separator=" ", strip=True)
-    if "public" not in text:
+    if not text:
         return ""
 
     # 归一化空白
@@ -345,16 +354,18 @@ def _extract_members_table(topic_content) -> str:
     return "\n".join(members)
 
 
-def parse_all_api_html(html_dir: str, *, limit: int | None = None) -> list[dict]:
+def parse_all_api_html(html_dir: str, *, limit: int | None = None,
+                       keep_constructors: bool = False) -> list[dict]:
     """
     解析目录下所有（或前 limit 个）API HTML 文件
 
     Args:
-        html_dir: CHM 解压后的 HTML 文件目录
-        limit:    仅解析前 N 个文件（调试用），None 表示全量
+        html_dir:          CHM 解压后的 HTML 文件目录
+        limit:             仅解析前 N 个文件（调试用），None 表示全量
+        keep_constructors: True 时保留构造函数页（见 parse_single_html）
 
     Returns:
-        list of parsed API data dicts（已过滤构造函数、命名空间页等噪音）
+        list of parsed API data dicts（已过滤命名空间页等噪音；默认也过滤构造函数）
     """
     results = []
     html_dir = Path(html_dir)
@@ -372,7 +383,7 @@ def parse_all_api_html(html_dir: str, *, limit: int | None = None) -> list[dict]
         if i % 1000 == 0 and i > 0:
             print(f"  解析进度: {i}/{total}  有效={len(results)}  ctor跳过={skipped_ctor}  噪音跳过={skipped_noise}")
         try:
-            data = parse_single_html(str(html_file))
+            data = parse_single_html(str(html_file), keep_constructors=keep_constructors)
             if data:
                 results.append(data)
             else: