doc3gpp 0.3.2__tar.gz → 0.3.4__tar.gz

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/AGENTS.md

@@ -275,6 +275,10 @@ Workflows in one line (full prose in `docs/architecture.md`):
   the result is still emitted with a warning). The zip cache is
   keyed on the **original (sanitized) filename** (D10 fix) so
   multiple revisions of the same tdoc_id never collide. When
+  the URL targets the canonical 3GPP FTP site, HTTPS is required (the
+  bare `www.3gpp.org/ftp/...` form is normalized), `.docx`/`.zip` files
+  must have a TDoc-shaped basename including `RP-...` plenary ids, and
+  any other canonical path is a folder with an optional trailing slash.
   `Settings.sync.auto_sync` is enabled, `tdoc parse --from-url`
   on a 3GPP FTP URL extracts tdoc_id candidates from the URL
   (basename for file URLs; BFS up to `--max-depth` / `--recursive`
@@ -356,8 +360,8 @@ Workflows in one line (full prose in `docs/architecture.md`):
    with `--resume`, `--stale-only`, and `--batch`, or `IndexStatus` with no
    rebuild flag. Hybrid weighting comes from
    `Settings.semantic_search.fts5_weight`.
- - `doc3gpp spec doc parse --spec <id> [--release R] [--version V] [--force]` → `SpecDocService.parse_many` → resolve the numeric-newest `SpecVersion` with a non-empty `.zip` URL via `resolve_spec_doc_version` (after optional release filtering); explicit `--version` is strict and fails clearly when its ZIP link is missing. Fetch the ZIP with internal `fetch_spec_doc_zip` when absent (or use the zip cache at `{spec_doc.cache_dir}/zips/<spec_id>/<version>.zip`), and record the download. `--force` re-downloads before re-parsing, including pairs that already carry `parsed_at`.
- - `doc3gpp spec doc parse --spec <id> [--release R] [--version V] [--force]` → per spec: immutable-skip when `(spec_id, version)` already carries `parsed_at` (unless `--force`) → `parse` = fetch-if-missing + `list_spec_docx` + `convert_document_to_blocks` + `order_spec_files` + `extract_spec_toc` + `chunk_blocks` → `upsert_toc` + `replace_chunks` + per-file markdown cache (`{spec_doc.cache_dir}/markdown/<spec_id>/<version>/`) + `record_parsed` + best-effort auto-index (`upsert_for_version`) / auto-embed (`index_for_version`). Oversized zips → skip bucket; unknown spec/version or empty (no-docx) zips → failure bucket; one spec never aborts the batch (`SpecDocBatchResult(successes/skipped/failures)`). Prints `ok/skipped/failed` buckets only (`--format/--output/--compact` accepted but ignored). Background `parse_spec_docs` jobs log and store per-spec errors in `result.failure_details`; all-failed batches have job status `failed`, while partial successes remain `succeeded`. Chunk/search metadata uses newline-delimited combined `sections` and `tables`; TOC entries retain `section_no` / `title`.
+  - `doc3gpp spec doc parse --spec <id> [--release R] [--version V] [--force]` accepts exactly one spec and calls `SpecDocService.parse_many` with a one-item list → resolve the numeric-newest `SpecVersion` with a non-empty `.zip` URL via `resolve_spec_doc_version` (after optional release filtering); when both `--release` and `--version` are given, their release numbers must align (for example, `Rel-18` with `18.5.0`); explicit `--version` is strict and fails clearly when its ZIP link is missing. Fetch the ZIP with internal `fetch_spec_doc_zip` when absent (or use the zip cache at `{spec_doc.cache_dir}/zips/<spec_id>/<version>.zip`), and record the download. `--force` re-downloads before re-parsing, including pairs that already carry `parsed_at`.
+  - `doc3gpp spec doc parse --spec <id> [--release R] [--version V] [--force]` → immutable-skip when `(spec_id, version)` already carries `parsed_at` (unless `--force`) → `parse` = fetch-if-missing + `list_spec_docx` + `convert_document_to_blocks` + `order_spec_files` + `extract_spec_toc` + `chunk_blocks` → `upsert_toc` + `replace_chunks` + per-file markdown cache (`{spec_doc.cache_dir}/markdown/<spec_id>/<version>/`) + `record_parsed` + best-effort auto-index (`upsert_for_version`) / auto-embed (`index_for_version`). Oversized zips → skip bucket; unknown spec/version or empty (no-docx) zips → failure bucket. Prints `ok/skipped/failed` buckets only (`--format/--output/--compact` accepted but ignored). Background `parse_spec_docs` jobs use scalar `spec_id` / optional `release` / `version` / `force`, log and store the result in `result.failure_details`. Chunk/search metadata uses newline-delimited combined `sections` and `tables`; TOC entries retain `section_no` / `title`.
  - `doc3gpp spec doc toc show --spec <id> --version <v> [--fields] [--format/--output/--compact]` → `SpecDocService.get_toc` (stored `(spec_id, version)` rows; miss → `BadParameter`). Default fields `section_no, title, level, source_file` (`[output.fields] spec_doc_toc`).
   - `doc3gpp spec doc search [--text TEXT] [--semantic TEXT] [filters]` →
     `SpecDocSearchFacade.search` with the same four mode-selection rules and

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: doc3gpp
-Version: 0.3.2
+Version: 0.3.4
 Summary: Provide 3GPP TDoc information with full text search capability — a Python CLI/library, a local web UI, and an MCP server for AI clients.
 Project-URL: Homepage, https://github.com/jerrywang121/doc3gpp
 Project-URL: Repository, https://github.com/jerrywang121/doc3gpp
@@ -296,6 +296,8 @@ doc3gpp tdoc parse --from-url https://www.3gpp.org/ftp/.../R5s260009.zip # 3GPP
 doc3gpp tdoc parse --from-url https://example.com/some.zip --format json -o /tmp/out.json  # non-3GPP URL → in-memory only
 doc3gpp tdoc parse --from-path ./tdocs --output ./parsed --recursive --format json         # local batch
 doc3gpp tdoc parse --from-url https://www.3gpp.org/ftp/.../Docs/ --recursive --output ./parsed  # online batch
+# 3GPP URLs also accept bare www.3gpp.org/ftp/...; HTTPS is required,
+# and .docx/.zip basenames must contain a TDoc id (including RP-... plenary ids).
 
 # show — --tdoc and --ftp-url are mutually exclusive
 doc3gpp tdoc show --tdoc R5s260009 --format json -o r5s260009.json
@@ -366,7 +368,7 @@ values on every version row.
 
 ```bash
 # use newest version with a published .zip, fetch-if-missing, convert every .docx, chunk, and auto-index
-doc3gpp spec doc parse --spec 38.331 --spec 38.523-1
+doc3gpp spec doc parse --spec 38.331 --release Rel-18 --version 18.5.0
 doc3gpp spec doc parse --spec 38.331 --force  # re-download and re-parse
 
 # inspect the stored TOC and search chunk text/combined metadata
@@ -377,10 +379,10 @@ doc3gpp spec doc index --rebuild
 doc3gpp spec doc schema --format json
 ```
 
-Background `parse_spec_docs` jobs retain per-spec failure reasons in the job
-result (`failure_details`) and log. A batch where every spec fails has status
-`failed`; a partial success retains status `succeeded` with the failed specs
-listed in the result.
+Background `parse_spec_docs` jobs retain the selected spec's failure reason in
+the job result (`failure_details`) and log. Each job targets one spec; when
+both release and version are supplied, their release numbers must align (for
+example, `Rel-18` with `18.5.0`).
 
 Spec-document rows live in a separate sibling database, normally
 `<main-stem>_specdata.db`, and are cached below the dedicated

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/README.md

@@ -213,6 +213,8 @@ doc3gpp tdoc parse --from-url https://www.3gpp.org/ftp/.../R5s260009.zip # 3GPP
 doc3gpp tdoc parse --from-url https://example.com/some.zip --format json -o /tmp/out.json  # non-3GPP URL → in-memory only
 doc3gpp tdoc parse --from-path ./tdocs --output ./parsed --recursive --format json         # local batch
 doc3gpp tdoc parse --from-url https://www.3gpp.org/ftp/.../Docs/ --recursive --output ./parsed  # online batch
+# 3GPP URLs also accept bare www.3gpp.org/ftp/...; HTTPS is required,
+# and .docx/.zip basenames must contain a TDoc id (including RP-... plenary ids).
 
 # show — --tdoc and --ftp-url are mutually exclusive
 doc3gpp tdoc show --tdoc R5s260009 --format json -o r5s260009.json
@@ -283,7 +285,7 @@ values on every version row.
 
 ```bash
 # use newest version with a published .zip, fetch-if-missing, convert every .docx, chunk, and auto-index
-doc3gpp spec doc parse --spec 38.331 --spec 38.523-1
+doc3gpp spec doc parse --spec 38.331 --release Rel-18 --version 18.5.0
 doc3gpp spec doc parse --spec 38.331 --force  # re-download and re-parse
 
 # inspect the stored TOC and search chunk text/combined metadata
@@ -294,10 +296,10 @@ doc3gpp spec doc index --rebuild
 doc3gpp spec doc schema --format json
 ```
 
-Background `parse_spec_docs` jobs retain per-spec failure reasons in the job
-result (`failure_details`) and log. A batch where every spec fails has status
-`failed`; a partial success retains status `succeeded` with the failed specs
-listed in the result.
+Background `parse_spec_docs` jobs retain the selected spec's failure reason in
+the job result (`failure_details`) and log. Each job targets one spec; when
+both release and version are supplied, their release numbers must align (for
+example, `Rel-18` with `18.5.0`).
 
 Spec-document rows live in a separate sibling database, normally
 `<main-stem>_specdata.db`, and are cached below the dedicated

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/docs/architecture.md

@@ -622,12 +622,15 @@ and syncs each through the `--tsg` path below.
 
 ### Spec-document corpus (parse → TOC → search)
 
-1. `doc3gpp spec doc parse --spec <id>... [--release R] [--version V]
-   [--force]` calls `SpecDocService.parse_many`. The version resolves via
+1. `doc3gpp spec doc parse --spec <id> [--release R] [--version V]
+   [--force]` accepts exactly one spec and calls `SpecDocService.parse_many`
+   with a one-item list. The version resolves via
    `resolve_spec_doc_version` — for automatic selection, filters to rows with
    a non-empty `.zip` URL, then applies numeric sort on `SpecVersion.version`
    (segment-wise ints, non-numeric → 0); optional `release` filters before
-   selection. An explicit `version` is strict and raises
+    selection. When both `release` and `version` are supplied, their release
+    numbers must agree (for example, `Rel-18` with `18.5.0`). An explicit
+    `version` is strict and raises
    `SpecDocUnknownVersionError` if its ZIP link is missing. A miss raises
    `SpecDocUnknownSpecError` (no
    `spec_versions` rows — run `spec sync --spec-id` first) or
@@ -1093,8 +1096,8 @@ the `spec doc` / `toc` / `search` triplet, 40 commands) plus the `server` group
     - `sync` — `--tsg`
     - `list` — filters by `--tsg`, `--name`, `--acronym`, `--release`
  - `spec doc` (nested under `spec` as `spec doc ...`):
-     - `parse` — repeatable `--spec` (at least one required), `--release`,
-       `--version`, `--force/-f`; resolves the numeric-newest or pinned
+     - `parse` — exactly one `--spec` (required), `--release`, `--version`,
+        `--force/-f`; resolves the numeric-newest or pinned
        version, fetches the ZIP when missing, and records the download;
        `--force` re-downloads and re-parses, including already-parsed pairs.
        Also accepts `--format/--output/--compact` (currently ignored); prints

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/docs/cli.md

@@ -1269,18 +1269,23 @@ Options:
   detected: a file is parsed as a single source, a directory is
   processed as a batch. The cache and database are never touched for
   local files.
-- `--from-url URL`: download the URL (HTTP or HTTPS) and parse it.
-  When the URL is on the canonical 3GPP FTP root
-  (`https://www.3gpp.org/ftp/...`) the result is cached on disk and
-  written to `tdoc_extracts` + `tdoc_cr_cover_page` (and to
+- `--from-url URL`: download the URL and parse it. For the canonical
+  3GPP FTP site, only HTTPS URLs under
+  `https://www.3gpp.org/ftp/...` are accepted; the bare form
+  `www.3gpp.org/ftp/...` is also accepted and normalized to HTTPS,
+  while an explicit `http://www.3gpp.org/ftp/...` URL is rejected.
+  A canonical `.docx` or `.zip` file must have a TDoc-shaped basename
+  (including plenary ids such as `RP-2601234`); a spec archive such as
+  `36508-j31.zip` is rejected. Every other canonical 3GPP path is
+  treated as a folder, and its trailing slash is optional. Canonical
+  single files are parsed directly; folders are processed as batches.
+  Canonical 3GPP results are cached on disk and written to
+  `tdoc_extracts` + `tdoc_cr_cover_page` (and to
   `tdoc_cr_ttcn_details` when the parser produced a TTCN sidecar —
-  subject to the FK matrix below); any other URL is parsed
-  in-memory only. Schemes other than `http(s)` (e.g. `ftp://`,
-  `file://`) are rejected — download out of band and use
-  `--from-path` instead. A URL ending in `/` is treated as a 3GPP
-  FTP folder and processed as a batch; URLs ending in `.docx`/`.zip`
-  are treated as single files;
-  ambiguous URLs are probed once and routed accordingly.
+  subject to the FK matrix below). Other HTTP(S) URLs retain the
+  existing in-memory parsing behavior. Schemes other than `http(s)`
+  (e.g. `ftp://`, `file://`) are rejected — download out of band and
+  use `--from-path` instead.
 - `--format {table,markdown,json,raw}`: output format. Default
   `table` (tab-separated header + single data row, matching
   `tdoc list --format table`). `markdown` writes a single-row GFM
@@ -1379,7 +1384,7 @@ FK-aware behaviour matrix (3GPP URL):
 |---|---|---|---|---|---|---|
 | `tdoc_id ∈ tdocs` (extracted from filename) | yes | yes | yes (skipped when `--format raw`) | yes — only when the parser emitted a TTCN sidecar (i.e. `is_ttcn_tdoc(tdoc_id)` and the document contained a TTCN overview / corrections section) | always | no |
 | `tdoc_id ∉ tdocs` (extracted but no FK target) | no | no | no | no | always | yes — actionable `meeting sync --tsg R5` recipe |
-| No `tdoc_id` pattern in filename | no | no | no | no | always | yes — pattern-miss notice |
+| Canonical `.docx`/`.zip` basename has no `tdoc_id` pattern | no | no | no | no | no — rejected at the URL boundary | yes — validation error mentioning TDoc |
 
 Local files and non-3GPP URLs always emit output and never touch
 the cache or the database.
@@ -1390,13 +1395,14 @@ the URL **before** dispatching to the parse helpers, then fires
 `trigger_auto_sync(...)` with the candidate set:
 
 - **3GPP FTP file URL** (`.docx` / `.zip` suffix) → one candidate parsed
-  out of the basename via `extract_tdoc_id_from_filename()`. Empty when
-  the basename has no recognised `tdoc_id` pattern.
-- **3GPP FTP folder URL** (trailing `/`) → BFS via
+  out of the TDoc-shaped basename via
+  `extract_tdoc_id_from_filename()`.
+- **3GPP FTP folder URL** (trailing `/` optional) → BFS via
   `TDocCrService.collect_3gpp_file_urls()` (up to `--max-depth` /
   `--recursive`), then extract `tdoc_id`s per file. Empty when BFS
   fails (warning only — the parse still proceeds).
-- **3GPP URL of unknown shape** → best-effort basename extraction.
+- **Other canonical 3GPP paths** → treated as folders, including paths
+  without a trailing `/`.
 - **Non-3GPP URL** → empty. The CLI gates the call on
   `is_3gpp_ftp_url()` so non-3GPP URLs never trigger the candidate
   helper.
@@ -2415,12 +2421,12 @@ strict and fails clearly if that version has no ZIP link. Run
 
 Purpose:
 
-- Fetch-if-missing, convert, chunk, and index one or more specs; prints one
-  `ok` / `skipped <reason>` / `failed <reason>` (stderr) line per spec.
+- Fetch-if-missing, convert, chunk, and index one spec; prints one `ok`,
+  `skipped <reason>`, or `failed <reason>` (stderr) line.
 
 Options:
 
-- --spec: spec id to parse; repeat per spec (at least one required).
+- --spec: spec id to parse; exactly one is required.
 - --release: release marker, e.g. `Rel-18`. Optional pin.
 - --version: exact version, e.g. `18.5.0`. Optional pin.
 - --force, -f: re-download and re-parse the resolved version, including
@@ -2439,11 +2445,12 @@ Behavior:
   `spec_doc_sources.downloaded_at` before conversion.
 - `--force` re-downloads the resolved ZIP and then re-parses it, including when
   the `(spec_id, version)` pair already has `parsed_at`.
-- Per spec: immutable-skip when the `(spec_id, version)` source row already
-  carries `parsed_at` (unless `--force`); oversized zips land in the `skipped`
-  bucket; unknown spec/version, empty (no-`.docx`) zips, and every other error
-  land in the `failures` bucket. One spec never aborts the batch
-  (`SpecDocBatchResult(successes/skipped/failures)`).
+- The `(spec_id, version)` source row is skipped when it already carries
+  `parsed_at` (unless `--force`); oversized zips land in the `skipped` bucket;
+  unknown spec/version, empty (no-`.docx`) zips, and every other error land in
+  the `failures` bucket. When both `--release` and `--version` are supplied,
+  their release numbers must agree (for example, `Rel-18` requires a version
+  beginning with `18.`).
 - `parse` = fetch-if-missing + `list_spec_docx` (`.doc` entries warn-skip) +
   `convert_document_to_blocks` + `order_spec_files` (front-matter TOC file first,
   then section-tuple sort, TOC tiebreak, unnumbered lexicographic tail) +
@@ -2468,8 +2475,8 @@ Examples:
 # Parse the newest version of 38.331 with a published ZIP download link.
 doc3gpp spec doc parse --spec 38.331
 
-# Parse two specs at a pinned release.
-doc3gpp spec doc parse --spec 38.331 --spec 38.523-1 --release Rel-18
+# Parse one spec at a pinned release and version.
+doc3gpp spec doc parse --spec 38.331 --release Rel-18 --version 18.5.0
 
 # Re-parse even when already parsed.
 doc3gpp spec doc parse --spec 38.331 --force

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/docs/code-map.md

@@ -115,7 +115,7 @@ table below is for navigation only.
 | `list_spec_docx` / `order_spec_files` / `extract_spec_toc` | functions | `parsers/spec_doc.py` | List `.docx` zip members, order split spec files by first section/TOC rules while preserving alphanumeric identifiers such as `7.2A.3A`, and build the ordered TOC/file map. |
 | `chunk_blocks` | function | `parsers/spec_doc_chunker.py` | Pure paragraph/sentence/table-row chunker using semantic token budgets plus the spec-doc character ceiling; carries newline-delimited combined `sections` / `tables` and source-file metadata. |
 | `parse_cr_details` | function | `parsers/cr_parser.py` | Markdown → `TDocCRDetails` (cover-page, optional TTCN overview, optional corrections). |
-| `is_3gpp_ftp_url` / `direct_parse_bytes` / `derive_zip_cache_key` / `extract_tdoc_id_from_filename` | functions | `parsers/direct_extractor.py` | Helpers for the `tdoc parse --from-path/--from-url` direct path. `is_3gpp_ftp_url` is the 3GPP-FTP detection rule; `direct_parse_bytes` glues docx conversion + cover-page parsing. |
+| `is_3gpp_ftp_url` / `normalize_3gpp_ftp_url` / `validate_tdoc_ftp_url` / `direct_parse_bytes` / `derive_zip_cache_key` / `extract_tdoc_id_from_filename` | functions | `parsers/direct_extractor.py` | Helpers for the `tdoc parse --from-path/--from-url` direct path. The URL normalizer/validator is the shared HTTPS canonical-site, file-vs-folder, and TDoc-basename policy; `direct_parse_bytes` glues docx conversion + cover-page parsing. |
 | `TDocParser` Protocol / `TDocParserRegistry` / `build_default_registry` | Protocol / class / function | `parsers/tdoc_parsers.py` | `TDocParser.parse()` returns a `TDocCRParseResult` (cover + optional TTCN sidecar). `build_default_registry` registers `TTCNCRParser` before the generic `CRParser` so TTCN CRs route to the overview + corrections parser and everything else falls through. |
 | `CRParserBase` / `CRParser` / `TTCNCRParser` | classes | `parsers/cr/cr_parsers.py` | Concrete parser implementations. `CRParserBase.parse()` returns `TDocCRParseResult`; `TTCNCRParser` populates the sidecar from the TTCN overview + corrections section payloads; `CRParser` always emits `ttcn=None`. |
 | `ttcn_functions` module | module | `parsers/cr/ttcn_functions.py` | Regex-driven helpers for the `tdoc_cr_ttcn_details.changed_functions` aggregate: `extract_module_basename` (handles Unix paths, Windows paths, bare basenames), `extract_function_name` (matches `f_`/`fl_`/`fx_`/`a_`/`tsc_`/`cs_`/`cr_`/`crs_`/`cas_`/`car_`/`cds_`/`cdr_`/`cms_`/`cmr_`/`cads_` prefixes with a `*_type` fallback), and the thin aggregator `extract_changed_functions` that returns a sorted, deduplicated `list[str]` of `"<module_basename>.<function_name>"` entries derived from `required_changes`. Partial-extraction markers (`"<module>."` trailing-dot, `".<function>"` leading-dot) cover the cases where one regex returns `None`; entries where both regexes return `None` are dropped. Consumed by `CRParserBase.parse()` when building the `TDocCRTTCNDetails` sidecar. |

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/docs/web-server.md

@@ -206,17 +206,16 @@ the target is missing or not `X-Doc3gpp-Managed` by doc3gpp.
 | POST | `/jobs/tdocs/index` | Enqueue unified TDoc FTS5/vector index maintenance. |
 | POST | `/jobs/spec-docs/index` | Enqueue unified spec-doc FTS5/vector index maintenance. |
 | POST | `/jobs/cache/purge` | Enqueue `cache_purge` (requires `yes: true`). |
-| POST | `/jobs/parse/tdoc-url` | Enqueue `parse_tdoc_url` (a single 3GPP FTP URL or folder; `url` must be `https://www.3gpp.org/ftp/...`, `recursive` XOR `max_depth`). |
-| POST | `/jobs/parse/spec-docs` | Enqueue `parse_spec_docs` for one or more spec ids (`spec_ids`, optional `release`, `version`, and `force`); progress is streamed through the normal job SSE endpoint. This job is intentionally not a panel on the eleven-form `/sync` hub. |
+| POST | `/jobs/parse/tdoc-url` | Enqueue `parse_tdoc_url` (a single TDoc file or folder on the canonical 3GPP FTP site; accepts HTTPS `https://www.3gpp.org/ftp/...` or bare `www.3gpp.org/ftp/...`, normalizes the latter, rejects explicit HTTP and non-TDoc spec archives, and permits an optional folder trailing slash; `recursive` XOR `max_depth`). |
+| POST | `/jobs/parse/spec-docs` | Enqueue `parse_spec_docs` for one spec id (`spec_id`, optional `release`, `version`, and `force`); progress is streamed through the normal job SSE endpoint. This job is intentionally not a panel on the eleven-form `/sync` hub. |
 | POST | `/jobs/{id}/cancel` | Cancel a queued/running job. Accepts `?format=html` to return the refreshed job row as an `outerHTML` swap target for the list page's per-row Cancel button. JSON otherwise. |
 | POST | `/jobs/sync_tdocs` | Flat alias for `sync_tdocs` (form or JSON). |
 | GET | `/sync` | Sync hub page: eleven enqueue forms (meetings, tdocs, all-tdocs, specs-by-tsg, specs-by-id, parse-tdocs, parse-tdoc-url, TDoc index maintenance, spec-document index maintenance, cache-purge, testcases) + a "Recent sync jobs" table. |
 | GET | `/sync?format=fragment` | Recent-jobs table fragment (wrapped in `<div id="recent-jobs">`) for HTMX `outerHTML` swap. |
 
-`parse_spec_docs` records per-spec errors in `result.failure_details` and the
-job log. If every requested spec fails, the job is `failed` with a populated
-`error` and result; partial successes remain `succeeded` with failure details
-in the result.
+`parse_spec_docs` records the spec error in `result.failure_details` and the
+job log. A failed parse job has a populated `error` and result; a successful or
+skipped parse reports its corresponding one-spec result.
 
 Append `?format=json` to any list/detail route to get the CLI-equivalent
 JSON. Append `?format=html` (or omit) for the browsable HTML view.
@@ -279,8 +278,8 @@ specdata source row has not been downloaded or parsed; a downloaded source is
 waiting for parsing; and a parsed source shows its parse timestamp, DOCX and
 chunk counts. The only document action is `Parse document` (or `Force
 re-parse`, available only for a parsed source. It enqueues the existing
-`POST /jobs/parse/spec-docs` job with `spec_ids`, the selected exact `version`,
-and `force`, then
+`POST /jobs/parse/spec-docs` job with scalar `spec_id`, the selected exact
+`version`, and `force`, then
 polls the normal job status. The parser downloads a missing ZIP internally;
 there is intentionally no standalone spec-document fetch route or fetch
 control.
@@ -530,8 +529,8 @@ The spec-document tools use the separate specdata database. `get_spec_toc`
 reads one parsed `(spec_id, version)` TOC, `search_spec_docs` selects among
 FTS5, vector, hybrid, and filter-only chunk search, and `get_spec_doc_index`
 reports index status. `parse_spec_docs` enqueues the same
-`PARSE_SPEC_DOCS` job as `POST /jobs/parse/spec-docs`; it accepts
-`spec_ids`, optional `release`/`version`, and `force`.
+`PARSE_SPEC_DOCS` job as `POST /jobs/parse/spec-docs`; it accepts one
+`spec_id`, optional `release`/`version`, and `force`.
 
 The MCP `serverInfo` block returned on every `initialize` handshake carries `name` ("doc3gpp"), `version` (from `importlib.metadata.version("doc3gpp")`, falling back to `doc3gpp.__version__`), `title`, `description`, and `website_url`. Clients do not need a tool call to read the version.
 

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/pyproject.toml

@@ -7,7 +7,7 @@ allow-direct-references = true
 
 [project]
 name = "doc3gpp"
-version = "0.3.2"
+version = "0.3.4"
 description = "Provide 3GPP TDoc information with full text search capability — a Python CLI/library, a local web UI, and an MCP server for AI clients."
 readme = "README.md"
 requires-python = ">=3.10"

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/__init__.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "0.3.0"
+__version__ = "0.3.4"

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/cli.py

@@ -30,13 +30,15 @@ from doc3gpp.cli_server import server_app
 from doc3gpp.cli_url_helpers import (
     _looks_like_3gpp_file_url,
     _looks_like_3gpp_folder_url,
+    is_3gpp_ftp_candidate_url,
     is_3gpp_ftp_url,
+    validate_tdoc_ftp_url,
 )
 from doc3gpp.config import get_settings
 from doc3gpp.models.meeting import Meeting
 from doc3gpp.models.schema_info import SCHEMA_FIELDS, schema_payload
 from doc3gpp.models.spec import Spec, SpecVersion
-from doc3gpp.models.spec_doc import SpecDocSearchFilters
+from doc3gpp.models.spec_doc import SpecDocSearchFilters, SpecDocSelectionError
 from doc3gpp.models.sync import BulkSyncOutcome, SyncOutcome
 from doc3gpp.models.tdoc import TDoc, TDocWithMeeting
 from doc3gpp.models.tdoc_cr import (
@@ -55,6 +57,7 @@ from doc3gpp.parsers.docx_converter import PythonDocxNotInstalledError
 from doc3gpp.parsers.normalizers import normalize_ftp_path
 from doc3gpp.scraping.cache import CacheStatus, TDocCache
 from doc3gpp.scraping.cache_keys import derive_cache_file
+from doc3gpp.scraping.spec_doc_source import validate_release_version_alignment
 from doc3gpp.scraping.tdoc_zip_source import canonicalise_tdoc_id
 from doc3gpp.services.factory import (
     build_meeting_service,
@@ -2237,6 +2240,11 @@ def tdoc_parse(
                     f"--from-path is neither a file nor a directory: {from_path}"
                 )
         else:
+            if is_3gpp_ftp_candidate_url(from_url):
+                try:
+                    from_url = validate_tdoc_ftp_url(from_url)
+                except ValueError as exc:
+                    raise typer.BadParameter(str(exc)) from exc
             effective_depth = _resolve_url_batch_depth(
                 recursive=recursive,
                 max_depth=max_depth,
@@ -5132,7 +5140,7 @@ def spec_doc_parse(
     spec: list[str] | None = typer.Option(  # noqa: B008 - Typer option declaration
         None,
         "--spec",
-        help="Spec id to parse; repeat per spec (at least one required).",
+        help="Spec id to parse (exactly one required).",
     ),
     release: str | None = typer.Option(
         None, "--release", help="Release marker, e.g. Rel-18.",
@@ -5160,10 +5168,16 @@ def spec_doc_parse(
         help="Strip output formatting: JSON drops indent and operator-space. No-op for ``table``.",
     ),
 ) -> None:
-    """Batch form: --spec may repeat; success/skip/failure buckets printed."""
+    """Parse one spec; print its success/skip/failure result."""
     create_schema("all")
     if not spec:
-        raise typer.BadParameter("pass at least one --spec <id>")
+        raise typer.BadParameter("pass exactly one --spec <id>")
+    if len(spec) != 1:
+        raise typer.BadParameter("--spec may be provided only once")
+    try:
+        validate_release_version_alignment(release, version)
+    except SpecDocSelectionError as exc:
+        raise typer.BadParameter(str(exc)) from exc
     svc = build_spec_doc_service()
     result = svc.parse_many(spec, release=release, version=version, force=force)
     for sid in result.successes:

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/cli_auto_sync.py

@@ -19,8 +19,8 @@ from typing import TYPE_CHECKING
 from doc3gpp.cli_filters import parse_tdoc_id
 from doc3gpp.cli_url_helpers import (
     _looks_like_3gpp_file_url,
-    _looks_like_3gpp_folder_url,
-    is_3gpp_ftp_url,
+    is_3gpp_ftp_candidate_url,
+    validate_tdoc_ftp_url,
 )
 from doc3gpp.models.meeting import Meeting
 from doc3gpp.parsers.direct_extractor import (
@@ -38,7 +38,7 @@ if TYPE_CHECKING:
 
 logger = logging.getLogger(__name__)
 
-_TSG_PREFIX_RE = re.compile(r"^([RSC][1-9])", re.IGNORECASE)
+_TSG_PREFIX_RE = re.compile(r"^([RSC][1-9P])", re.IGNORECASE)
 
 
 def _build_meeting_url(tsg: str, ext: str = "htm") -> str:
@@ -176,7 +176,7 @@ def sync_meeting_internal(
 def collect_tdoc_candidates_for_url(
     url: str,
     *,
-    tdoc_service: "TDocCrService | None" = None,
+    tdoc_service: TDocCrService | None = None,
     max_depth: int = 0,
 ) -> set[str]:
     """Return distinct tdoc_ids implied by ``url``.
@@ -188,12 +188,12 @@ def collect_tdoc_candidates_for_url(
     - 3GPP FTP file URL (ends with ``.docx`` / ``.zip``) → the tdoc_id
       parsed out of the basename, or empty if the basename has no
       recognised TDoc-id pattern.
-    - 3GPP FTP folder URL (ends with ``/``) → BFS over the folder up to
-      ``max_depth`` via the supplied service and extract tdoc_ids from
-      each file's basename. Returns empty when ``tdoc_service`` is
+    - 3GPP FTP folder URL (trailing ``/`` optional) → BFS over the folder
+      up to ``max_depth`` via the supplied service and extract tdoc_ids
+      from each file's basename. Returns empty when ``tdoc_service`` is
       ``None`` (the helper is a pure no-op in that case).
-    - 3GPP URL of unknown shape → try basename extraction; empty if it
-      doesn't parse.
+    - Any other canonical 3GPP FTP path → treat it as a folder and scan
+      breadth-first using the supplied service.
 
     Network or BFS failures are logged at ``WARNING`` and the helper
     returns whatever candidates were already collected — it never raises
@@ -215,7 +215,12 @@ def collect_tdoc_candidates_for_url(
         failure mode.
     """
     candidates: set[str] = set()
-    if not is_3gpp_ftp_url(url):
+    if not is_3gpp_ftp_candidate_url(url):
+        return candidates
+    try:
+        url = validate_tdoc_ftp_url(url)
+    except ValueError as exc:
+        logger.warning("Auto-sync candidate collection rejected %s: %s", url, exc)
         return candidates
 
     if _looks_like_3gpp_file_url(url):
@@ -224,18 +229,12 @@ def collect_tdoc_candidates_for_url(
             candidates.add(extracted)
         return candidates
 
-    if _looks_like_3gpp_folder_url(url):
-        return _collect_from_3gpp_folder(url, tdoc_service, max_depth)
-
-    extracted = extract_tdoc_id_from_filename(url)
-    if extracted is not None:
-        candidates.add(extracted)
-    return candidates
+    return _collect_from_3gpp_folder(url, tdoc_service, max_depth)
 
 
 def _collect_from_3gpp_folder(
     url: str,
-    tdoc_service: "TDocCrService | None",
+    tdoc_service: TDocCrService | None,
     max_depth: int,
 ) -> set[str]:
     """BFS-collect tdoc_ids from a 3GPP FTP folder URL; empty on any failure."""

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/cli_filters.py

@@ -67,13 +67,13 @@ DATE_FILTER_RE = re.compile(
 )
 
 
-# CR-shape TDoc identifier: ``[RSC][1-9][-sw]\d{6,7}`` — TSG group
-# initial (R/S/C), non-zero TSG digit, shape marker (- / s / w for
+# CR-shape TDoc identifier: ``[RSC][1-9P][-sw]\d{6,7}`` — TSG group
+# initial (R/S/C), non-zero TSG digit or plenary marker P, shape marker (- / s / w for
 # canonical / TTCN / workshop), 6- or 7-digit sequence number.
 # 3GPP RAN4 has used 7-digit numbers since 2016 (e.g. ``R4-2607922``);
 # every other working group is still on 6 digits. Do not narrow without
 # re-checking the RAN4 DynaReport.
-TDOC_ID_RE = re.compile(r"[RSC][1-9][-sw]\d{6,7}", re.IGNORECASE)
+TDOC_ID_RE = re.compile(r"[RSC][1-9P][-sw]\d{6,7}", re.IGNORECASE)
 
 
 def is_null_token(value: str) -> bool:
@@ -142,8 +142,9 @@ def parse_tdoc_id(value: str) -> tuple[str, int]:
         raise ValueError(
             f"Invalid TDoc id {value!r}. Expected a 9- or 10-character "
             f"CR-shape id like 'R5-260013', 'R5s260009', 'R5w260013', or "
-            f"'R4-2607922' — TSG group initial (R/S/C), non-zero TSG digit, "
-            f"shape marker (-/s/w), then 6 or 7 decimal digits."
+            f"'R4-2607922' or 'RP-2601234' — TSG group initial (R/S/C), "
+            f"non-zero TSG digit or plenary marker P, shape marker "
+            f"(-/s/w), then 6 or 7 decimal digits."
         )
     return stripped[:3], int(stripped[3:])
 
@@ -360,4 +361,4 @@ class SearchQueryBuilder:
                 out.append("\\" + ch)
             else:
                 out.append(ch)
-        return "".join(out)
+        return "".join(out)

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/cli_url_helpers.py

@@ -2,8 +2,8 @@
 
 The CLI's ``--from-url`` direct-parse path needs a tiny set of
 predicate functions to decide whether a URL points at a 3GPP FTP
-asset, whether it ends in a known file extension, and whether it
-trails with a slash (a folder marker). The same predicates are
+asset, whether it ends in a known file extension, and whether it is a
+folder path (with or without a trailing slash). The same predicates are
 needed from :mod:`doc3gpp.cli_auto_sync` so the auto-sync helper can
 derive TDoc-id candidates from a URL without dragging the full
 parser module into its import surface.
@@ -16,26 +16,36 @@ import one consistent set of names:
   parser implementation stays in one place.
 * :func:`_looks_like_3gpp_file_url` — true when the URL ends with a
   known document extension.
-* :func:`_looks_like_3gpp_folder_url` — true when the URL ends with
-  a slash (folder marker).
+* :func:`_looks_like_3gpp_folder_url` — true for any other canonical
+  3GPP FTP path.
 """
 
 from __future__ import annotations
 
-from doc3gpp.parsers.direct_extractor import is_3gpp_ftp_url
+from doc3gpp.parsers.direct_extractor import (
+    is_3gpp_file_url,
+    is_3gpp_ftp_candidate_url,
+    is_3gpp_ftp_url,
+    normalize_3gpp_ftp_url,
+    validate_tdoc_ftp_url,
+)
 
 __all__ = [
     "_looks_like_3gpp_file_url",
     "_looks_like_3gpp_folder_url",
+    "is_3gpp_file_url",
+    "is_3gpp_ftp_candidate_url",
     "is_3gpp_ftp_url",
+    "normalize_3gpp_ftp_url",
+    "validate_tdoc_ftp_url",
 ]
 
 
 def _looks_like_3gpp_folder_url(url: str) -> bool:
-    """Return ``True`` when ``url`` is unambiguously a folder path."""
-    return url.endswith("/")
+    """Return ``True`` for a canonical non-file FTP path."""
+    return is_3gpp_ftp_url(url) and not is_3gpp_file_url(url)
 
 
 def _looks_like_3gpp_file_url(url: str) -> bool:
-    """Return ``True`` when ``url`` ends with a known file extension."""
-    return url.lower().endswith((".docx", ".zip"))
+    """Return ``True`` when a canonical URL ends with a known extension."""
+    return is_3gpp_file_url(url)

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/data/skills/doc3gpp/SKILL.md

@@ -20,7 +20,7 @@ Read tools return a single text item containing JSON (except `get_tdoc_content`,
 | Meeting calendar | `list_meetings` with `tsg`, `name`, or `year`; `get_meeting` with numeric `meeting_id` | `sync_meetings({"tsg":"R5"})` fetches and parses that TSG's calendar | Re-list, then use `get_meeting` to obtain its meeting ID and FTP URL. There is no separate meeting-parse tool. |
 | Meeting TDocs | `list_tdocs` with `meeting_id`, `tdoc_id`, `title`, `spec`, etc.; `get_tdoc` with `tdoc_id` or `ftp_url` | `sync_tdocs({"meeting_id":260013})` after the meeting exists; alternatively `sync_tdocs_by_meeting({"meeting":"RAN5#106"})`. For document extraction, `parse_tdocs({"filter":{"tdoc_id":"R5-260013"}})`; for a direct 3GPP FTP URL use `parse_tdoc_url({"url":"https://www.3gpp.org/ftp/..."})`. | `get_tdoc` for cover/extract metadata; `get_tdoc_content({"tdoc_id":"R5-260013"})` for cached markdown if available; `search_tdoc` for indexed content and filters. |
 | Specifications | `list_specs` with `tsg`, `spec_id`, `title`, or `parsed`; `get_spec` with dotted `spec_id` | `sync_specs({"tsg":"R5"})` or `sync_specs({"spec_id":"38.331"})` (exactly one selector). `force` bypasses the sync interval; `per_version_details` fetches extra PDF/CR-list details. | `get_spec` returns versions, release markers, ZIP URLs, and per-version `parsed` state. Paginate versions with `limit`/`offset` if needed. |
-| Specification documents | Start with `get_spec` to identify a version with a ZIP URL; `get_spec_toc` needs an already parsed exact `(spec_id, version)` | `parse_spec_docs({"spec_ids":["38.331"],"version":"18.5.0"})` downloads if needed and parses; omit `version` to resolve a suitable stored version (optionally constrain with `release`). `force` re-downloads and re-parses. | `get_spec_toc({"spec_id":"38.331","version":"18.5.0"})`; `search_spec_docs` with `spec_id`, `version`, `text` or `semantic`, `sections`, `tables`. Note '%' wildcards to match section / table titles to improve search. If search section numbers or table numbers, use '%' at the end to match sub-sections or sub-tables. e.g. `"section":"5.3.2%"` or `"table":"6.1.1.2.3-%"`. |
+| Specification documents | Start with `get_spec` to identify a version with a ZIP URL; `get_spec_toc` needs an already parsed exact `(spec_id, version)` | `parse_spec_docs({"spec_id":"38.331","version":"18.5.0"})` downloads if needed and parses one spec; omit `version` to resolve a suitable stored version (optionally constrain with `release`). When both `release` and `version` are supplied, their release numbers must align, e.g. `Rel-18` with `18.5.0`. `force` re-downloads and re-parses. | `get_spec_toc({"spec_id":"38.331","version":"18.5.0"})`; `search_spec_docs` with `spec_id`, `version`, `text` or `semantic`, `sections`, `tables`. Note '%' wildcards to match section / table titles to improve search. If search section numbers or table numbers, use '%' at the end to match sub-sections or sub-tables. e.g. `"section":"5.3.2%"` or `"table":"6.1.1.2.3-%"`. |
 | RAN5 testcases | `list_testcases` with `testcase` (ID pattern), `title`, `spec`, `group`, `status` (TTCN), or `gcf_status` (GCF/PTCRB) | `sync_testcases({})` fetches and parses the latest RAN5 status snapshot in one job; `{"force":true}` re-downloads and re-parses a snapshot already recorded. No separate testcase parse tool. | `get_testcase({"testcase_id":"<id>"})` returns all stored groups; pass `group` for one group. Each result has nested `statuses` entries containing `path`, `gcf_ptcrb`, and `ttcn_status`. |
 
 Filters on list/search tools generally accept SQL-LIKE `%` wildcards (for example `title:"%handover%"`); plain values match exactly. Testcase `group` is an exact match: `5G`, `LTE`, `IMS`, `UTRA`, `POS`, or `MCX`. `list_meetings`, `list_tdocs`, `list_specs`, and `list_testcases` default to 50 rows: use `limit` and `offset` to page. `search_tdoc` and `search_spec_docs` select FTS5 with `text`, vector search with `semantic`, hybrid with both, and filter-only with neither. Semantic search requires its configured embedding service. If text search reports an unavailable or corrupt index, inspect `get_tdoc_index` or `get_spec_doc_index`; use `index_tdocs({"rebuild":true})` or `index_spec_docs({"rebuild":true})` when rebuilding is needed, then poll that job. Parse normally attempts automatic indexing, but indexing can fail independently.
@@ -44,7 +44,7 @@ search_tdoc         {"text":"handover","meeting_id":<meeting_id>}
 sync_specs          {"spec_id":"38.331"}
 get_job             {"job_id":"<spec-sync job_id>"}    # poll until terminal
 get_spec            {"spec_id":"38.331"}
-parse_spec_docs     {"spec_ids":["38.331"],"version":"<ZIP-backed version from get_spec>"}
+parse_spec_docs     {"spec_id":"38.331","version":"<ZIP-backed version from get_spec>"}
 get_job             {"job_id":"<spec-doc-parse job_id>"} # poll until terminal
 get_spec_toc        {"spec_id":"38.331","version":"<parsed version>"}
 search_spec_docs    {"text":"handover","spec_id":"38.331","version":"<parsed version>", "sections":"5.2.2.3%"}

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/models/spec_doc.py

@@ -1,5 +1,6 @@
 """DTOs for the spec-document corpus (separate specdata sqlite file)."""
 from __future__ import annotations
+
 from dataclasses import dataclass, field
 from datetime import datetime
 
@@ -18,6 +19,10 @@ class SpecDocUnknownVersionError(SpecDocError):
         self.available = available or []
 
 
+class SpecDocSelectionError(SpecDocError):
+    """Raised when a release/version selection is internally inconsistent."""
+
+
 class SpecDocNoDocxError(SpecDocError):
     pass
 

{doc3gpp-0.3.2 → doc3gpp-0.3.4}/src/doc3gpp/parsers/cr/header.py

@@ -21,9 +21,10 @@ class CRHeaderMissingError(ValueError):
 
 
 # Union: ``R5-227476`` (dash) AND ``R5s260009`` / ``R5w260176`` (single
-# letter) AND ``C6-250028`` (dash) AND RAN4's 7-digit ``R4-2607922``.
+# letter) AND ``C6-250028`` (dash), plenary ``RP-2601234`` IDs, and
+# RAN4's 7-digit ``R4-2607922``.
 # Matches in the document header near the top of the markdown.
-_TDOC_HEADER_PATTERN = re.compile(r"([RSC][1-6](?:[-sw])\d{6,7})", re.IGNORECASE)
+_TDOC_HEADER_PATTERN = re.compile(r"([RSC][1-6P](?:[-sw])\d{6,7})", re.IGNORECASE)
 # Email-meeting TTCN pattern. Overview / corrections are only parsed
 # for TDocs matching this shape.
 _TTCN_TDOC_PATTERN = re.compile(r"R5s\d{6}", re.IGNORECASE)