@pikaa-ai/pikaa 0.2.4 → 0.3.0

package/skills/imaging-data-commons/references/licensing_and_citation.md

@@ -0,0 +1,230 @@
+# Licensing and Citation Guide for IDC
+
+## When to Use This Guide
+
+Load this guide when:
+- A user asks whether IDC data can be used commercially, redistributed, or included in a product
+- You are assembling a cohort that mixes collections and need to know which terms govern it
+- A user is publishing results and needs formatted citations (APA, BibTeX, CSL JSON, RDF Turtle)
+- You need the parameters or output formats for citation generation
+
+The obligation summary — check the license, generate citations — lives in `SKILL.md`. This
+guide holds the detail behind it.
+
+**These are the two IDC tasks least tied to any one access path.** Licenses and citations are
+available identically from `idc-index`, the REST API, and the hosted MCP server. Use whichever
+route the session is already on rather than installing Python to answer a licensing question,
+or dropping out of an MCP session to run a script.
+
+| Task | `idc-index` (Python) | REST API | MCP server |
+|------|----------------------|----------|------------|
+| License breakdown for a selection | `sql_query` on `license_short_name` | `POST /v3/licenses` | `get_licenses` |
+| Citations for a selection | `citations_from_selection()` | `POST /v3/citations` | `get_citations` |
+
+Route-specific detail lives in `references/rest_api_guide.md` (endpoint reference, filter
+syntax, and the body-shape pitfall that makes a mis-shaped filter return all of IDC) and
+`references/mcp_guide.md` (tool inventory). The license semantics below apply to all three.
+
+## Licenses in IDC
+
+Every DICOM file in IDC is tagged with its license in the file metadata, and every row in the
+`index` table carries a `license_short_name` column. There is no single IDC-wide license.
+
+| License | Share of data | Commercial use | Attribution required |
+|---------|---------------|----------------|----------------------|
+| CC BY 4.0 | 74.7% | Yes | Yes |
+| CC BY 3.0 | 22.1% | Yes | Yes |
+| CC BY-NC 4.0 | 2.1% | **No** | Yes |
+| CC BY-NC 3.0 | 0.8% | **No** | Yes |
+| NLM Terms and Conditions | 0.3% | Read the terms | Yes |
+
+About 97% of IDC data by size permits commercial reuse; just under 3% is non-commercial. Treat
+any `license_short_name` that is not a recognizable Creative Commons string as custom, and
+report the exact value to the user rather than assuming it permits reuse.
+
+**Licenses attach to individual series, not to whole collections.** 39 of IDC's 176 collections
+carry more than one license — analysis results and original images within one collection can
+differ, as can series from different sources. Never conclude that a collection is
+commercially usable from one series, or from the collection's headline license: group by
+`license_short_name` over the exact selection you intend to use.
+
+**When a cohort mixes licenses, the most restrictive term governs the combined dataset.** If a
+selection contains any CC BY-NC series, either drop those series or tell the user the whole
+derived dataset is non-commercial.
+
+Commercially restricted data is also physically separated in cloud storage: the
+`idc-open-data-cr` (AWS) / `idc-open-cr` (GCS) buckets hold the CC BY-NC collections. See
+`references/cloud_storage_guide.md` for bucket details.
+
+## Checking licenses
+
+### Via `idc-index`
+
+```python
+from idc_index import IDCClient
+client = IDCClient()
+
+# Licenses across all collections
+licenses = client.sql_query("""
+    SELECT DISTINCT
+      collection_id,
+      license_short_name,
+      COUNT(DISTINCT SeriesInstanceUID) as series_count
+    FROM index
+    GROUP BY collection_id, license_short_name
+    ORDER BY collection_id
+""")
+print(licenses)
+```
+
+```python
+# Licenses present in one specific cohort — run this before handing a dataset to a user
+cohort_licenses = client.sql_query("""
+    SELECT license_short_name, COUNT(DISTINCT SeriesInstanceUID) as series_count
+    FROM index
+    WHERE Modality = 'MR' AND BodyPartExamined = 'BREAST'
+    GROUP BY license_short_name
+""")
+print(cohort_licenses)
+```
+
+```python
+# Commercial-safe subset: exclude non-commercial collections outright
+commercial_ok = client.sql_query("""
+    SELECT collection_id, SeriesInstanceUID
+    FROM index
+    WHERE Modality = 'CT'
+      AND license_short_name NOT LIKE '%NC%'
+    LIMIT 20
+""")
+```
+
+### Via the REST API
+
+`POST /v3/licenses` takes the filter object **directly** (not wrapped in a `filters` key) and
+returns the per-license breakdown with series counts and sizes:
+
+```bash
+B=https://api.imaging.datacommons.cancer.gov/v3
+curl -s $B/licenses \
+  -H 'content-type: application/json' \
+  -d '{"terms": {"Modality": ["MR"], "BodyPartExamined": ["BREAST"]}}'
+```
+
+Response shape: `licenses[{license_short_name, series, size_TB}]`. A collection's licenses are
+also included in `GET /v3/collections/{id}`.
+
+### Via the MCP server
+
+Call `get_licenses` with the same selection you built with `build_cohort`. The result carries
+the same per-license breakdown; the CC BY vs CC BY-NC distinction above applies unchanged.
+
+## Citations and attribution
+
+The `source_DOI` column links to the publications describing how each dataset was generated.
+All three routes turn a selection into formatted citations that satisfy the attribution
+requirement common to every IDC license.
+
+Generate citations from the *same* selection you downloaded, not from the collection as a
+whole — a five-series subset of a collection that spans several source publications should
+cite only the publications it actually draws on.
+
+### Via `idc-index`
+
+```python
+# Citations for a collection (APA is the default format)
+citations = client.citations_from_selection(collection_id="rider_pilot")
+for citation in citations:
+    print(citation)
+```
+
+```python
+# Citations for a specific set of series — matches what you actually downloaded
+results = client.sql_query("""
+    SELECT SeriesInstanceUID FROM index
+    WHERE collection_id = 'tcga_luad' LIMIT 5
+""")
+citations = client.citations_from_selection(
+    seriesInstanceUID=list(results['SeriesInstanceUID'].values)
+)
+```
+
+```python
+# BibTeX, for LaTeX manuscripts
+bibtex_citations = client.citations_from_selection(
+    collection_id="tcga_luad",
+    citation_format=IDCClient.CITATION_FORMAT_BIBTEX
+)
+```
+
+`citations_from_selection()` takes the same selection filters as the download methods —
+`collection_id`, `patientId`, `studyInstanceUID`, `seriesInstanceUID` — plus `citation_format`.
+
+### Via the REST API
+
+`POST /v3/citations` **wraps** the filter in a `filters` key (unlike `/v3/licenses` — this
+asymmetry is the single most common REST mistake; see `references/rest_api_guide.md`):
+
+```bash
+curl -s $B/citations \
+  -H 'content-type: application/json' \
+  -d '{"filters": {"terms": {"collection_id": ["rider_pilot"]}}, "citation_format": "bibtex"}'
+```
+
+The response separates the per-dataset `citations[]` from `idc_acknowledgment` (the IDC paper)
+and `recommendation`. Include both parts — see *What to include when publishing* below.
+
+### Via the MCP server
+
+Call `get_citations` for a selection. It returns the per-dataset citations plus the IDC paper
+to acknowledge IDC itself, matching the REST response.
+
+### Citation formats
+
+| `idc-index` constant | REST / MCP `citation_format` | Output |
+|----------------------|------------------------------|--------|
+| `IDCClient.CITATION_FORMAT_APA` | `apa` (default) | APA string |
+| `IDCClient.CITATION_FORMAT_BIBTEX` | `bibtex` | BibTeX entry, for LaTeX |
+| `IDCClient.CITATION_FORMAT_JSON` | `csl-json` | CSL JSON |
+| `IDCClient.CITATION_FORMAT_TURTLE` | `turtle` | RDF Turtle |
+
+## What to include when publishing
+
+1. **The dataset citations** for every collection or series set used.
+2. **The IDC data version** — `client.get_idc_version()`, `GET /v3/version`, or the MCP
+   `get_idc_version` tool. IDC releases are versioned and series are added and revised between
+   them, so the version is what makes the selection reproducible.
+3. **The IDC platform citation**, to acknowledge IDC itself. The REST and MCP routes return
+   this as `idc_acknowledgment`; when using `idc-index`, add it yourself:
+
+   > Fedorov, A., et al. "National Cancer Institute Imaging Data Commons: Toward Transparency,
+   > Reproducibility, and Scalability in Imaging Artificial Intelligence." *RadioGraphics* 43.12
+   > (2023). https://doi.org/10.1148/rg.230180
+
+4. **The series manifest** — save the `SeriesInstanceUID` list alongside the analysis so the
+   exact cohort can be rebuilt.
+
+## Troubleshooting
+
+### Issue: Fewer citations returned than collections selected
+
+- **Cause:** Citations are derived from `source_DOI`, and several collections can share one
+  DOI, so a multi-collection selection may legitimately produce a shorter list.
+- **Solution:** Query `SELECT DISTINCT collection_id, source_DOI FROM index WHERE ...` to see
+  the mapping directly.
+
+### Issue: `POST /v3/citations` returns citations for all of IDC
+
+- **Cause:** The filter was passed directly instead of wrapped in `filters`. `/v3/licenses`
+  takes the filter directly; `/v3/citations` wraps it. A mis-shaped body is not an error — it
+  is treated as an empty filter.
+- **Solution:** Check the response counts against a `POST /v3/cohort/counts` for the same
+  selection. See `references/rest_api_guide.md`.
+
+## Resources
+
+- **IDC Portal** — https://portal.imaging.datacommons.cancer.gov/
+- **IDC data licensing documentation** — https://learn.canceridc.dev/data/licensing
+- **`references/rest_api_guide.md`** — `/v3/licenses` and `/v3/citations` endpoint reference
+- **`references/mcp_guide.md`** — `get_licenses` and `get_citations` tool inventory
+- **`references/cloud_storage_guide.md`** — bucket separation for commercially restricted data

package/skills/imaging-data-commons/references/mcp_guide.md

@@ -0,0 +1,181 @@
+# IDC MCP Server Guide
+
+IDC operates a hosted [Model Context Protocol](https://modelcontextprotocol.io/) server that
+exposes IDC discovery and metadata as agent tools. This guide covers how to recognize it, how
+to divide work between it and `idc-index`, and what to hand off across that boundary.
+
+The server is optional. Everything in `SKILL.md` works without it.
+
+## Endpoint
+
+| Property | Value |
+|----------|-------|
+| URL | `https://api.imaging.datacommons.cancer.gov/mcp` |
+| Transport | Streamable HTTP (`streamable-http`, sometimes spelled `http`) |
+| Authentication | None |
+| Server identity | `IDC (Imaging Data Commons)` |
+
+## Identifying the server
+
+Tool names and resource URIs are defined by the server, so they are the same on every host.
+Use them, not host-specific naming conventions, to decide whether the server is present.
+
+**Strongest signal — resource URIs.** The server publishes two resources under an `idc://`
+scheme:
+
+| URI | Content |
+|-----|---------|
+| `idc://guide` | Data model and recommended workflow (Markdown) |
+| `idc://tables` | Tables available to `run_sql`, with descriptions and column counts (JSON) |
+
+If the host can enumerate MCP resources, a resource with URI `idc://guide` identifies the
+server unambiguously.
+
+**Fallback — tool-name fingerprint.** Require three or more of `build_cohort`,
+`get_cohort_urls`, `list_analysis_results`, and `get_idc_version`. Do not treat `run_sql`,
+`get_stats`, `list_tables`, or `get_citations` as evidence on their own; those names are
+generic enough that another server could expose them.
+
+**This is disambiguation, not authentication.** No runtime check can prove the server on the
+other end is operated by NCI — a hostile server could serve `idc://guide` and name its tools
+anything. The trust anchor is the URL the user configured plus TLS, which is established when
+the server is added, not when the skill runs. That is sufficient for routing: the check only
+has to distinguish IDC from the user's other installed servers.
+
+**Fail soft.** If identification is ambiguous, or a tool call fails, fall back rather than
+reporting an error — to the REST API (`rest_api_guide.md`) for read-only metadata, which is the
+same service with no configuration, or to `idc-index` when it is already installed or the task
+needs downloads or local analysis. Tool names may change as the server matures.
+
+## Tool inventory
+
+Verified against server version `3.0.0b3`. Treat this as a snapshot, not a contract — call
+the server's own listing rather than assuming this list is current.
+
+| Group | Tools |
+|-------|-------|
+| Version and scale | `get_idc_version`, `get_stats` |
+| Collections | `list_collections`, `get_collection`, `list_analysis_results` |
+| Attribute grounding | `list_attributes`, `get_attribute_values` |
+| Cohorts | `build_cohort`, `get_cohort_urls` |
+| SQL | `list_tables`, `get_table_schema`, `run_sql` |
+| Clinical data | `list_clinical_tables`, `get_clinical_table_schema`, `get_clinical_table` |
+| Attribution | `get_citations`, `get_licenses` |
+| Visualization | `get_viewer_url` |
+
+The server ships its own usage instructions, which most hosts inject automatically. Follow
+those instructions for tool sequencing (ground with `list_attributes` /
+`get_attribute_values` before filtering; check `list_tables` before writing SQL). Do not
+re-derive that workflow from `SKILL.md` — the two would drift apart on the server's next
+release.
+
+**Cohort results report their own filters.** `build_cohort` and `get_cohort_urls` require at
+least one filter predicate and fail cleanly without one, rather than returning the whole archive;
+results echo the filters actually applied along with warnings for any predicate that was dropped
+or any value whose casing did not match. Read those warnings before reporting a count — a zero
+with no warning means the filter matched nothing, which is a real answer. Same contract as the
+REST endpoints they wrap; see `rest_api_guide.md`.
+
+## Division of labor
+
+The server and `idc-index` overlap on metadata queries and diverge everywhere else.
+
+| Task | Use |
+|------|-----|
+| IDC data version, collection and series counts | Server (`get_idc_version`, `get_stats`) |
+| Valid filter values before building a query | Server (`get_attribute_values`) |
+| Cohort selection by attribute filters | Server (`build_cohort`) |
+| One-off metadata SQL, answer consumed as prose | Server (`run_sql`) |
+| Metadata SQL whose result feeds local Python | `idc-index` (`client.sql_query`) |
+| Downloading DICOM files | `idc-index` (`client.download_from_selection`) |
+| pandas / notebook analysis, plotting | `idc-index` |
+| Reading pixel data (pydicom, SimpleITK) | `idc-index` + local files |
+| DICOMweb, BigQuery, direct S3/GCS, Parquet | `idc-index` and the relevant reference guide |
+| Digital pathology tiling and annotation workflows | `idc-index` + `digital_pathology_guide.md` |
+| Reproducible scripts a user will re-run | `idc-index` (a script outlives the session) |
+
+Two rules resolve the overlap:
+
+- **Prefer the server for discovery.** It is hosted against a current IDC release, so it does
+  not depend on the `idc-index` version pinned in `SKILL.md`.
+- **Prefer `idc-index` when the result must become a Python object.** Round-tripping a
+  DataFrame through tool output wastes context and loses types.
+
+## Handing off from the server to `idc-index`
+
+The boundary artifact is a list of `SeriesInstanceUID` values.
+
+```python
+# UIDs obtained from the MCP server's build_cohort / run_sql output
+series_uids = [
+    "1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463",
+    # ...
+]
+
+from idc_index import IDCClient
+client = IDCClient()
+
+# Confirm size before downloading — the server reports size_TB, but re-check locally
+sizes = client.sql_query(f"""
+    SELECT COUNT(*) AS series, SUM(series_size_MB)/1000 AS size_GB
+    FROM index
+    WHERE SeriesInstanceUID IN ({','.join(f"'{u}'" for u in series_uids)})
+""")
+print(sizes)
+
+client.download_from_selection(
+    downloadDir="./data",
+    seriesInstanceUID=series_uids,       # a list, not a DataFrame
+    dirTemplate="%collection_id/%PatientID/%Modality",
+)
+```
+
+Run `python scripts/check_version.py` before the first `idc-index` call in a session, even if
+discovery happened server-side — the two components version independently.
+
+`get_cohort_urls` also returns ready-made `idc` CLI commands. Those are the better handoff
+when the user wants a shell command they can re-run outside the session; see
+`references/cli_guide.md`.
+
+Going the other direction, `idc-index` results are already local, so there is rarely a reason
+to send them back to the server.
+
+## Version authority
+
+When the server is present, it is the authority on the IDC data version: call
+`get_idc_version` rather than quoting the `idc-data-version` value in the `SKILL.md`
+frontmatter, which records the release the skill was last verified against.
+
+If the server and a locally installed `idc-index` report different versions, say so and name
+both. The mismatch is real — the hosted server tracks IDC releases independently of the user's
+installed package — and it changes which answers about "what's new" are correct.
+
+## Host-specific notes
+
+Everything above is portable. The items below are not, and apply only to specific agent
+environments.
+
+### Claude Code
+
+- **Tool naming.** MCP tools are exposed as `mcp__<server>__<tool>`, where `<server>` is the
+  configured server name with every character outside `A-Za-z0-9_-` replaced by `_`. A CLI
+  install named `idc` yields `mcp__idc__build_cohort`; a claude.ai connector named
+  `IDC MCP prod` yields `mcp__claude_ai_IDC_MCP_prod__build_cohort`.
+- **Enumerating resources.** `ListMcpResourcesTool` returns each resource with a `server`
+  field, which is how to find the `idc://guide` resource and the owning server name in one
+  call.
+- **Adding the server.**
+  `claude mcp add --transport http idc https://api.imaging.datacommons.cancer.gov/mcp`
+- **Permission rules.** Allow rules need a literal, glob-free server segment: `mcp__idc__*`
+  works, `mcp__*` does not. Connector installs need their own
+  `mcp__claude_ai_<name>__*` rule, so the rule differs by install path.
+- **Detecting via the CLI does not work.** `claude mcp list` reads only file-based
+  configuration (`~/.claude.json`, `.mcp.json`). It reports "No MCP servers configured" for a
+  claude.ai connector that is connected and working in the same session, so it cannot be used
+  as a presence check.
+
+### Other hosts
+
+Any agent that supports MCP over streamable HTTP can use the server. Consult that agent's
+documentation for how servers are registered and how tool names are namespaced; the endpoint
+URL and the absence of authentication are all the configuration it needs.

package/skills/imaging-data-commons/references/parquet_access_guide.md

@@ -0,0 +1,200 @@
+# Direct Parquet Access Guide for IDC
+
+**Tested with:** idc-index-data 24.2.2 (IDC data version v24), DuckDB 1.5
+
+All idc-index metadata tables are published as Parquet files to a public GCS bucket with unrestricted CORS access. This enables metadata queries with DuckDB or pandas without installing idc-index.
+
+**Limitation:** download helpers (`download_from_selection()`), viewer URLs (`get_viewer_URL()`), and citation generation require the idc-index client and are not available from raw Parquet files.
+
+**This is not the first no-install option to reach for.** It still needs DuckDB installed, and the per-collection clinical tables are not published here — only the `clinical_index` dictionary. For ad-hoc metadata with nothing installed, the REST API (`rest_api_guide.md`) needs no install at all and reaches `clinical.<table>` through `POST /sql`.
+
+## When to Use This Guide
+
+Load this guide when you need to:
+- Pin queries to a specific IDC data version (see *Pinning to a Specific Version* below) rather than whatever the hosted API currently serves
+- Return more rows than the REST `/sql` ceiling of 10 000
+- Run heavy or repeated local DuckDB analysis without driving the hosted API
+- Query IDC metadata where DuckDB is available but idc-index is not
+
+For downloads, viewer URLs, and citations, use idc-index as documented in the main SKILL.md.
+
+## URL Pattern
+
+```
+https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts/{filename}.parquet
+```
+
+`current/` always resolves to the latest data release. To pin to a specific version, replace `current` with the data version number (e.g., `23.10.1`).
+
+## Available Files
+
+| File | Approximate Size | Description |
+|------|-----------------|-------------|
+| `idc_index.parquet` | ~70 MB | Primary index (all DICOM series metadata) |
+| `volume_geometry_index.parquet` | ~5 MB | 3D geometry validation for CT/MR/PT series |
+| `rtstruct_index.parquet` | ~2 MB | RT Structure Set ROI metadata |
+| `seg_index.parquet` | ~6 MB | DICOM Segmentation cross-references |
+| `sm_index.parquet` | ~2 MB | Slide microscopy series metadata |
+| `contrast_index.parquet` | ~1 MB | Contrast agent metadata |
+| `ann_index.parquet` | ~0.2 MB | Microscopy annotation series metadata |
+| `ann_group_index.parquet` | ~0.5 MB | Annotation group metadata |
+| `collections_index.parquet` | — | Collection-level metadata |
+| `analysis_results_index.parquet` | — | Derived dataset metadata |
+| `clinical_index.parquet` | ~0.2 MB | Clinical data column dictionary |
+| `ct_index.parquet` | — | CT acquisition/reconstruction parameters |
+| `mr_index.parquet` | — | MR sequence/acquisition parameters |
+| `pt_index.parquet` | — | PET acquisition/radiopharmaceutical parameters |
+| `prior_versions_index.parquet` | — | Series from previous IDC releases |
+
+**Note:** the main index file is named `idc_index.parquet`, not `index.parquet`. Reference it with an alias in SQL queries (e.g., `FROM read_parquet(...) AS index`).
+
+## Prerequisites
+
+Install the Python `duckdb` package, using whatever installer manages the environment you are
+running in.
+
+DuckDB reads Parquet directly from HTTPS URLs using HTTP range requests — no GCS client library or authentication required.
+
+## Basic Queries
+
+```python
+import duckdb
+
+BASE = "https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts"
+
+# Discover modalities and series counts
+duckdb.sql(f"""
+    SELECT Modality, COUNT(*) as series_count, ROUND(SUM(series_size_MB)/1000, 1) as size_GB
+    FROM read_parquet('{BASE}/idc_index.parquet')
+    GROUP BY Modality
+    ORDER BY series_count DESC
+""").df()
+
+# Collections with CT data, ordered by size
+duckdb.sql(f"""
+    SELECT collection_id,
+           COUNT(DISTINCT PatientID) as patients,
+           COUNT(*) as series,
+           ROUND(SUM(series_size_MB)/1000, 1) as size_GB
+    FROM read_parquet('{BASE}/idc_index.parquet')
+    WHERE Modality = 'CT'
+    GROUP BY collection_id
+    ORDER BY size_GB DESC
+    LIMIT 10
+""").df()
+```
+
+## Volume Geometry Validation
+
+`volume_geometry_index` covers single-frame CT, MR, and PT series. Each row has boolean checks for orientation, spacing, dimensions, and slice positions, plus a composite `regularly_spaced_3d_volume` flag.
+
+```python
+import duckdb
+
+BASE = "https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts"
+
+# CT series that form a valid 3D volume (can be loaded without resampling)
+duckdb.sql(f"""
+    SELECT i.collection_id, i.SeriesInstanceUID, i.BodyPartExamined,
+           v.obliquity_degrees, v.regularly_spaced_3d_volume
+    FROM read_parquet('{BASE}/idc_index.parquet') i
+    JOIN read_parquet('{BASE}/volume_geometry_index.parquet') v
+        ON i.SeriesInstanceUID = v.SeriesInstanceUID
+    WHERE i.Modality = 'CT'
+      AND v.regularly_spaced_3d_volume = TRUE
+    LIMIT 10
+""").df()
+
+# Fraction of 3D-valid series per collection and modality
+duckdb.sql(f"""
+    SELECT i.collection_id, i.Modality,
+           COUNT(*) as total,
+           SUM(CASE WHEN v.regularly_spaced_3d_volume THEN 1 ELSE 0 END) as valid_3d,
+           ROUND(100.0 * SUM(CASE WHEN v.regularly_spaced_3d_volume THEN 1 ELSE 0 END) / COUNT(*), 1) as pct_valid
+    FROM read_parquet('{BASE}/idc_index.parquet') i
+    JOIN read_parquet('{BASE}/volume_geometry_index.parquet') v
+        ON i.SeriesInstanceUID = v.SeriesInstanceUID
+    WHERE i.Modality IN ('CT', 'MR', 'PT')
+    GROUP BY i.collection_id, i.Modality
+    ORDER BY total DESC
+    LIMIT 10
+""").df()
+```
+
+Key columns in `volume_geometry_index`:
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `SeriesInstanceUID` | STRING | Join key |
+| `single_orientation` | BOOLEAN | All instances share the same ImageOrientationPatient |
+| `orthogonal_orientation` | BOOLEAN | Orientation direction cosines are orthogonal |
+| `unique_slice_positions` | BOOLEAN | No duplicate or overlapping slices |
+| `consistent_pixel_spacing` | BOOLEAN | All instances share the same PixelSpacing |
+| `consistent_image_dimensions` | BOOLEAN | All instances share the same Rows and Columns |
+| `uniform_slice_spacing` | BOOLEAN | Spacing between consecutive slices is constant |
+| `obliquity_degrees` | FLOAT | Angle between slice normal and nearest cardinal axis (0 = pure axial/sagittal/coronal) |
+| `regularly_spaced_3d_volume` | BOOLEAN | Composite: TRUE if all checks pass |
+
+## RT Structure Sets
+
+`rtstruct_index` has one row per RTSTRUCT series with aggregated ROI metadata.
+
+```python
+import duckdb
+
+BASE = "https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts"
+
+# RTSTRUCT series with ROI details
+duckdb.sql(f"""
+    SELECT i.collection_id, i.SeriesInstanceUID,
+           r.total_rois, r.ROINames, r.RTROIInterpretedTypes,
+           r.referenced_SeriesInstanceUID
+    FROM read_parquet('{BASE}/idc_index.parquet') i
+    JOIN read_parquet('{BASE}/rtstruct_index.parquet') r
+        ON i.SeriesInstanceUID = r.SeriesInstanceUID
+    WHERE i.Modality = 'RTSTRUCT'
+    LIMIT 5
+""").df()
+
+# Collections with the most RTSTRUCT series
+duckdb.sql(f"""
+    SELECT i.collection_id,
+           COUNT(*) as rtstruct_series,
+           ROUND(AVG(r.total_rois), 1) as avg_rois_per_struct
+    FROM read_parquet('{BASE}/idc_index.parquet') i
+    JOIN read_parquet('{BASE}/rtstruct_index.parquet') r
+        ON i.SeriesInstanceUID = r.SeriesInstanceUID
+    GROUP BY i.collection_id
+    ORDER BY rtstruct_series DESC
+    LIMIT 10
+""").df()
+```
+
+Key columns in `rtstruct_index`:
+
+| Column | Type | Description |
+|--------|------|-------------|
+| `SeriesInstanceUID` | STRING | Join key (the RTSTRUCT series) |
+| `total_rois` | INTEGER | Number of ROIs in the structure set |
+| `ROINames` | STRING (array) | Distinct ROI names (e.g., `["GTV", "Heart", "PTV"]`) |
+| `ROIGenerationAlgorithms` | STRING (array) | Distinct generation algorithms (e.g., `["AUTOMATIC", "MANUAL"]`) |
+| `RTROIInterpretedTypes` | STRING (array) | Distinct ROI types (e.g., `["GTV", "ORGAN", "PTV"]`) |
+| `referenced_SeriesInstanceUID` | STRING | SeriesInstanceUID of the referenced source image series |
+
+## Pinning to a Specific Version
+
+```python
+import duckdb
+
+# Use a specific data release instead of 'current'
+VERSION = "23.10.1"
+BASE = f"https://storage.googleapis.com/idc-index-data-artifacts/{VERSION}/release_artifacts"
+
+duckdb.sql(f"SELECT COUNT(*) FROM read_parquet('{BASE}/idc_index.parquet')").df()
+```
+
+## Resources
+
+- idc-index-data releases: https://github.com/ImagingDataCommons/idc-index-data/releases
+- idc-index documentation: https://idc-index.readthedocs.io/
+- IDC Portal: https://portal.imaging.datacommons.cancer.gov/