wotann 0.5.0 → 0.5.39

package/skills/scientific/database-lookup/references/geo.md

@@ -0,0 +1,130 @@
+# NCBI GEO (Gene Expression Omnibus) via E-utilities
+
+## Base URLs
+
+| Purpose | URL |
+|---|---|
+| E-utilities | `https://eutils.ncbi.nlm.nih.gov/entrez/eutils/` |
+| GEO direct query | `https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi` |
+
+## Important: The database name is `gds`
+
+The Entrez database for GEO is `gds` (not `geo`). It contains all GEO record types: GDS datasets, GSE series, GPL platforms, GSM samples. Filter by type using `[ETYP]` in the search term.
+
+## Key Endpoints
+
+### eSearch — Search GEO
+
+```
+GET /esearch.fcgi?db=gds&term={query}&retmode=json&retmax={n}
+```
+
+Parameters:
+- `db=gds` (required)
+- `term` — search query with field tags
+- `retmax` — max results (default 20)
+- `retstart` — offset for pagination
+- `retmode=json` — get JSON response
+- `usehistory=y` — store results server-side for large queries
+- `api_key` — NCBI API key (optional, raises rate limit)
+
+#### Entry type filters (`[ETYP]`)
+- `gds[ETYP]` — curated GEO DataSets
+- `gse[ETYP]` — GEO Series (most common, use this by default)
+- `gpl[ETYP]` — platforms
+- `gsm[ETYP]` — samples
+
+#### Other field tags
+- `[Organism]` — e.g. `"Homo sapiens"[Organism]`
+- `[PDAT]` — publication date
+- `[Title]` — title search
+- Boolean: `AND`, `OR`, `NOT` (uppercase)
+
+Example — cancer GSE series in human:
+```
+/esearch.fcgi?db=gds&term=cancer+AND+gse[ETYP]+AND+"Homo+sapiens"[Organism]&retmax=10&retmode=json
+```
+
+Response:
+```json
+{
+  "esearchresult": {
+    "count": "15432",
+    "retmax": "10",
+    "idlist": ["200012345", "200067890"],
+    "querytranslation": "cancer AND gse[ETYP]"
+  }
+}
+```
+
+The IDs returned are numeric UIDs (not accession numbers). For GSE records: UID = 200000000 + GSE_number.
+
+### eSummary — Get metadata for UIDs
+
+```
+GET /esummary.fcgi?db=gds&id={uid_list}&retmode=json
+```
+
+Key response fields per record:
+- `Accession` — e.g. "GSE12345"
+- `title`, `summary`
+- `taxon` — organism
+- `entrytype` — "GDS", "GSE", "GPL", "GSM"
+- `gdstype` — e.g. "Expression profiling by array"
+- `n_samples` — sample count
+- `pubmedids` — linked PubMed IDs
+- `PDAT` — publication date
+- `Samples` — array of sample objects
+- `FTPLink` — data download path
+
+Example:
+```
+/esummary.fcgi?db=gds&id=200012345&retmode=json
+```
+
+### GEO Direct Query — Full records by accession
+
+```
+GET https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc={accession}&form={format}&view={detail}
+```
+
+Parameters:
+- `acc` — GEO accession (GSE12345, GDS1234, GPL570, GSM12345)
+- `targ` — `self`, `gsm` (samples), `gpl` (platform), `gse` (series)
+- `form` — `text` (SOFT format), `xml` (MINiML), `html`
+- `view` — `quick`, `brief`, `full`, `data`
+
+Example — series metadata in SOFT:
+```
+acc.cgi?acc=GSE53757&targ=self&form=text&view=brief
+```
+
+Note: acc.cgi does not return JSON. Use eSearch + eSummary for JSON results. Use acc.cgi when you need full SOFT/MINiML records.
+
+### eLink — Cross-reference with other NCBI databases
+
+```
+GET /elink.fcgi?dbfrom=gds&db=pubmed&id={uid}&retmode=json
+```
+
+## Practical Workflow
+
+For most queries, use this two-step approach:
+
+1. **eSearch** to find UIDs matching the query
+2. **eSummary** to get metadata for those UIDs
+
+This gives you JSON throughout.
+
+## Important Notes
+
+- GDS records are mostly frozen — NCBI stopped curating new GDS. Use `gse[ETYP]` for comprehensive results.
+- eFetch has limited support for the `gds` database. Use eSummary for metadata or acc.cgi for full records.
+- URL-encode spaces as `+` and quotes as `%22`.
+
+## Rate Limits
+
+- **Without API key**: 3 requests/second
+- **With API key**: 10 requests/second (free registration at ncbi.nlm.nih.gov/account/settings)
+- Include `&email=user@example.com` as a courtesy
+- For large result sets, use the History server (`usehistory=y` then pass `WebEnv` and `query_key` to eSummary)

package/skills/scientific/database-lookup/references/gnomad.md

@@ -0,0 +1,93 @@
+# gnomAD (Genome Aggregation Database) API Reference
+
+## Overview
+gnomAD aggregates exome and genome sequencing data to provide allele frequencies
+and variant annotations across diverse populations.
+
+## API Type: GraphQL
+- **Endpoint**: `https://gnomad.broadinstitute.org/api`
+- **Method**: POST with JSON body containing GraphQL query
+- **Auth**: None required (public, unauthenticated)
+- **Response format**: JSON (`data` wrapper with GraphQL structure)
+
+## Key Queries
+
+### Variant lookup by variant ID
+Variant IDs use format: `{chrom}-{pos}-{ref}-{alt}` (GRCh37 or GRCh38).
+
+```
+POST https://gnomad.broadinstitute.org/api
+Content-Type: application/json
+
+{
+  "query": "{ variant(variantId: \"1-55516888-G-A\", dataset: gnomad_r4) { variant_id rsids chrom pos ref alt exome { ac an af } genome { ac an af } } }"
+}
+```
+
+### Gene lookup
+```json
+{
+  "query": "{ gene(gene_symbol: \"BRCA1\", reference_genome: GRCh38) { gene_id symbol chrom start stop strand } }"
+}
+```
+
+### Variants in a gene
+```json
+{
+  "query": "{ gene(gene_symbol: \"PCSK9\", reference_genome: GRCh38) { variants(dataset: gnomad_r4) { variant_id consequence rsids exome { ac an af } genome { ac an af } } } }"
+}
+```
+
+### Variants in a region
+```json
+{
+  "query": "{ region(chrom: \"1\", start: 55505222, stop: 55530526, reference_genome: GRCh38) { variants(dataset: gnomad_r4) { variant_id rsids consequence exome { ac af } genome { ac af } } } }"
+}
+```
+
+### Transcript lookup
+```json
+{
+  "query": "{ transcript(transcript_id: \"ENST00000357654\", reference_genome: GRCh38) { transcript_id gene_id chrom start stop strand } }"
+}
+```
+
+## Dataset values
+- `gnomad_r4` -- gnomAD v4 (GRCh38, latest major release)
+- `gnomad_r3` -- gnomAD v3.1.2 (GRCh38, genomes only)
+- `gnomad_r2_1` -- gnomAD v2.1.1 (GRCh37, exomes + genomes)
+
+## Population frequency fields
+Within `exome` or `genome` objects, population-specific frequencies are available via
+`populations { id ac an af }` where `id` values include: `afr`, `amr`, `asj`, `eas`,
+`fin`, `mid`, `nfe`, `oth`, `sas`.
+
+## Response example (variant)
+```json
+{
+  "data": {
+    "variant": {
+      "variant_id": "1-55516888-G-A",
+      "rsids": ["rs11591147"],
+      "chrom": "1",
+      "pos": 55516888,
+      "ref": "G",
+      "alt": "A",
+      "exome": { "ac": 1234, "an": 250000, "af": 0.004936 },
+      "genome": { "ac": 456, "an": 150000, "af": 0.00304 }
+    }
+  }
+}
+```
+
+## Rate Limits
+- No published rate limits, but aggressive querying will be throttled
+- Use reasonable request pacing (~1 req/sec recommended)
+- For bulk downloads, use gnomAD's Hail tables on Google Cloud or download VCFs
+
+## Notes
+- The GraphQL schema is not versioned separately; it tracks the gnomAD web interface
+- Use the browser's network inspector on gnomad.broadinstitute.org to discover
+  additional query fields and structures
+- Structural variants (SV) have a separate query structure (`structural_variant`)
+- Constraint metrics (pLI, LOEUF) are available on gene queries via `gnomad_constraint`

package/skills/scientific/database-lookup/references/gtex.md

@@ -0,0 +1,136 @@
+# GTEx (Genotype-Tissue Expression) API Reference
+
+## Overview
+GTEx catalogs gene expression levels across human tissues from postmortem donors,
+enabling study of tissue-specific gene regulation and eQTLs.
+
+## Base URL
+`https://gtexportal.org/api/v2`
+
+## Auth
+None required (public, unauthenticated).
+
+## Response Format
+JSON. Most endpoints return paginated results with structure:
+```json
+{
+  "data": [ ... ],
+  "paging_info": {
+    "numberOfPages": 10,
+    "page": 0,
+    "maxItemsPerPage": 250
+  }
+}
+```
+
+## Pagination Parameters (common to most endpoints)
+- `page` -- 0-indexed page number (default: 0)
+- `itemsPerPage` -- results per page (default: 250, max: 250)
+
+## Key Endpoints
+
+### Gene expression (median by tissue)
+```
+GET /expression/medianGeneExpression?gencodeId=ENSG00000139618.17&datasetId=gtex_v8
+```
+Parameters:
+- `gencodeId` -- Versioned Ensembl gene ID (required)
+- `datasetId` -- `gtex_v8` (required)
+- `tissueSiteDetailId` -- filter to specific tissue (optional)
+
+Returns median TPM per tissue for the gene.
+
+### Gene expression (all, for a tissue)
+```
+GET /expression/medianGeneExpression?tissueSiteDetailId=Liver&datasetId=gtex_v8
+```
+
+### Single-tissue eQTLs
+```
+GET /association/singleTissueEqtl?gencodeId=ENSG00000139618.17&tissueSiteDetailId=Whole_Blood&datasetId=gtex_v8
+```
+Parameters:
+- `gencodeId` -- Versioned Ensembl gene ID (required)
+- `tissueSiteDetailId` -- tissue ID (required)
+- `datasetId` -- `gtex_v8` (required)
+
+### Multi-tissue eQTLs
+```
+GET /association/multiTissueEqtl?gencodeId=ENSG00000139618.17&datasetId=gtex_v8
+```
+
+### Gene search
+```
+GET /reference/gene?geneId=BRCA2&gencodeVersion=v26&genomeBuild=GRCh38/hg38
+```
+Parameters:
+- `geneId` -- gene symbol or Ensembl ID
+- `gencodeVersion` -- `v26` for GTEx v8
+- `genomeBuild` -- `GRCh38/hg38`
+
+### List tissues
+```
+GET /dataset/tissueSiteDetail?datasetId=gtex_v8
+```
+Returns all tissue site detail IDs, names, colors, sample counts.
+
+### Exon expression
+```
+GET /expression/medianExonExpression?gencodeId=ENSG00000139618.17&datasetId=gtex_v8
+```
+
+### Transcript expression
+```
+GET /expression/medianTranscriptExpression?gencodeId=ENSG00000139618.17&datasetId=gtex_v8
+```
+
+### Top expressed genes in a tissue
+```
+GET /expression/topExpressedGene?tissueSiteDetailId=Brain_Cortex&datasetId=gtex_v8&filterMtGene=true
+```
+
+### Variant by location (dyadic)
+```
+GET /association/dyneqtl?variantId=chr1_1000000_A_G_b38&gencodeId=ENSG00000139618.17&tissueSiteDetailId=Whole_Blood&datasetId=gtex_v8
+```
+
+## Tissue ID examples
+Use the underscore-separated names exactly:
+- `Whole_Blood`, `Liver`, `Brain_Cortex`, `Heart_Left_Ventricle`
+- `Muscle_Skeletal`, `Adipose_Subcutaneous`, `Lung`, `Skin_Sun_Exposed_Lower_leg`
+
+## Example response (median gene expression)
+```json
+{
+  "data": [
+    {
+      "datasetId": "gtex_v8",
+      "gencodeId": "ENSG00000139618.17",
+      "geneSymbol": "BRCA2",
+      "median": 4.523,
+      "tissueSiteDetailId": "Whole_Blood",
+      "unit": "TPM"
+    },
+    {
+      "datasetId": "gtex_v8",
+      "gencodeId": "ENSG00000139618.17",
+      "geneSymbol": "BRCA2",
+      "median": 12.87,
+      "tissueSiteDetailId": "Testis",
+      "unit": "TPM"
+    }
+  ],
+  "paging_info": { "numberOfPages": 1, "page": 0, "maxItemsPerPage": 250 }
+}
+```
+
+## Rate Limits
+- No published rate limits
+- Reasonable request pacing recommended (~1-2 req/sec)
+- For bulk analysis, download full datasets from the GTEx Portal downloads page
+
+## Notes
+- GTEx v8 is the primary dataset; always specify `datasetId=gtex_v8`
+- Gene IDs must be versioned GENCODE IDs (e.g., ENSG00000139618.17)
+- Use the gene search endpoint to resolve symbols to versioned GENCODE IDs
+- `gencodeVersion=v26` corresponds to GTEx v8

package/skills/scientific/database-lookup/references/gwas-catalog.md

@@ -0,0 +1,46 @@
+# GWAS Catalog (EBI)
+
+## Base URL
+```
+https://www.ebi.ac.uk/gwas/rest/api
+```
+
+## Auth
+No API key required.
+
+## Note: Responses use HAL+JSON format with `_links` and `_embedded` keys.
+
+## Key Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `/studies/{accession}` | Single study (e.g. GCST001633) |
+| `/studies/search/findByPubmedId?pubmedId={id}` | Studies by PubMed ID |
+| `/singleNucleotidePolymorphisms/{rsId}` | SNP details |
+| `/singleNucleotidePolymorphisms/{rsId}/associations` | Associations for a SNP |
+| `/singleNucleotidePolymorphisms/search/findByRsId?rsId={rsId}` | Search by rsID |
+| `/associations` | List associations |
+| `/associations/{id}` | Single association |
+| `/efoTraits` | List EFO traits |
+| `/efoTraits/search/findByEfoTrait?trait={name}` | Search traits |
+
+## Pagination
+`?page=0&size=20` (zero-indexed, max ~500)
+
+## Example Calls
+```
+# Get a study
+https://www.ebi.ac.uk/gwas/rest/api/studies/GCST001633
+
+# Associations for a SNP
+https://www.ebi.ac.uk/gwas/rest/api/singleNucleotidePolymorphisms/rs7329174/associations
+
+# Search traits
+https://www.ebi.ac.uk/gwas/rest/api/efoTraits/search/findByEfoTrait?trait=diabetes&page=0&size=5
+```
+
+## Response Format
+HAL+JSON. Results in `_embedded.studies[]` or `_embedded.associations[]`. Key fields: `pvalue`, `riskFrequency`, `orPerCopyNum`, `betaNum`.
+
+## Rate Limits
+No published limit. Bulk data via FTP at ftp.ebi.ac.uk/pub/databases/gwas/

package/skills/scientific/database-lookup/references/hca.md

@@ -0,0 +1,35 @@
+# Human Cell Atlas (HCA)
+
+## Base URL
+```
+https://service.azul.data.humancellatlas.org/
+```
+
+## Auth
+No auth required.
+
+## Key Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `/index/projects?size={n}&catalog=dcp2` | List/search projects |
+| `/index/samples?size={n}&catalog=dcp2` | List/search samples |
+| `/index/files?size={n}&catalog=dcp2` | List/search files |
+| `/index/summary?catalog=dcp2` | Summary statistics |
+
+## Example Calls
+```
+# List projects
+https://service.azul.data.humancellatlas.org/index/projects?size=5&catalog=dcp2
+
+# Summary stats
+https://service.azul.data.humancellatlas.org/index/summary?catalog=dcp2
+```
+
+Supports JSON filter parameters for organ, species, library construction, etc.
+
+## Response Format
+JSON. `hits` array with project/sample/file metadata + pagination.
+
+## Rate Limits
+No published limits. Be reasonable.

package/skills/scientific/database-lookup/references/hpo.md

@@ -0,0 +1,48 @@
+# HPO (Human Phenotype Ontology)
+
+## Base URL
+```
+https://ontology.jax.org/api/hp
+```
+
+## Auth
+No API key required.
+
+## Important: URL-encode colons in HP IDs — `HP:0001250` becomes `HP%3A0001250`
+
+## Key Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `/hpo/search?q={query}&max={n}` | Search HPO terms by name |
+| `/hpo/term/{id}` | Term details |
+| `/hpo/term/{id}/genes` | Genes associated with a phenotype |
+| `/hpo/term/{id}/diseases` | Diseases associated with a phenotype |
+| `/hpo/term/{id}/children` | Child terms in hierarchy |
+| `/hpo/term/{id}/parents` | Parent terms |
+| `/hpo/gene/{gene_id}` | Phenotypes for a gene (Entrez ID) |
+| `/hpo/disease/{disease_id}` | Phenotypes for a disease (OMIM/ORPHA) |
+
+## Example Calls
+```
+# Search for "seizure"
+https://ontology.jax.org/api/hp/hpo/search?q=seizure&max=5
+
+# Term details for Seizure
+https://ontology.jax.org/api/hp/hpo/term/HP%3A0001250
+
+# Genes associated with Seizure
+https://ontology.jax.org/api/hp/hpo/term/HP%3A0001250/genes
+
+# Diseases for Seizure
+https://ontology.jax.org/api/hp/hpo/term/HP%3A0001250/diseases
+
+# Phenotypes for SCN1A (Entrez 6323)
+https://ontology.jax.org/api/hp/hpo/gene/6323
+```
+
+## Response Format
+JSON. Terms: `id`, `name`, `definition`, `synonyms`. Gene associations: `genes[]` with `geneId`, `geneSymbol`. Diseases: `diseases[]` with `diseaseId`, `diseaseName`.
+
+## Rate Limits
+No published limits. Bulk annotation files at https://hpo.jax.org/data/annotations

package/skills/scientific/database-lookup/references/human-protein-atlas.md

@@ -0,0 +1,57 @@
+# Human Protein Atlas (HPA)
+
+## Base URL
+```
+https://www.proteinatlas.org
+```
+
+## Auth
+No API key required.
+
+## Key Endpoints
+
+| Purpose | URL Pattern |
+|---|---|
+| Gene data by Ensembl ID | `/{ENSEMBL_ID}.json` |
+| Gene data by symbol | `/{GENE_NAME}.json` |
+| Search (JSON) | `/search/{QUERY}?format=json` |
+| Search (XML) | `/search/{QUERY}?format=xml` |
+
+## Example Calls
+
+```
+# Gene data by Ensembl ID
+https://www.proteinatlas.org/ENSG00000141510.json
+
+# Gene data by symbol
+https://www.proteinatlas.org/TP53.json
+
+# Search
+https://www.proteinatlas.org/search/TP53?format=json
+```
+
+## Response Format (JSON, gene endpoint)
+```json
+{
+  "Gene": "TP53",
+  "Gene synonym": ["p53", "LFS1"],
+  "Ensembl": "ENSG00000141510",
+  "Gene description": "tumor protein p53",
+  "Uniprot": ["P04637"],
+  "Chromosome": "17",
+  "Protein class": ["Transcription factors"],
+  "RNA tissue specificity": "Low tissue specificity",
+  "Subcellular location": ["Nucleoplasm"],
+  "Pathology prognostics": [...]
+}
+```
+
+## Bulk Downloads
+For large-scale work, use TSV files from https://www.proteinatlas.org/about/download:
+- `normal_tissue.tsv` — IHC tissue expression
+- `rna_tissue_consensus.tsv` — RNA consensus
+- `subcellular_location.tsv`
+- `pathology.tsv` — cancer prognostics
+
+## Rate Limits
+No published limits. Be reasonable. Prefer bulk downloads for large queries.

package/skills/scientific/database-lookup/references/interpro.md

@@ -0,0 +1,120 @@
+# InterPro API Reference
+
+## Base URL
+```
+https://www.ebi.ac.uk/interpro/api
+```
+
+## Authentication
+None required. Fully public API.
+
+## Rate Limits
+No published hard limits. EBI general guidance: be reasonable, use bulk downloads for large datasets.
+
+## Response Format
+JSON by default. Some endpoints support `?format=json` explicitly.
+
+## Key Endpoints
+
+### 1. Entry Lookup (by accession)
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro/{accession}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro/IPR000504
+```
+Returns JSON with entry name, type (family/domain/site/etc.), description, GO terms, literature references.
+
+### 2. Entry Lookup by Member Database
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/pfam/{pfam_accession}
+GET https://www.ebi.ac.uk/interpro/api/entry/smart/{smart_accession}
+GET https://www.ebi.ac.uk/interpro/api/entry/prosite/{prosite_accession}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/pfam/PF00076
+```
+
+### 3. Search / List Entries
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro?search={query}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro?search=kinase
+```
+Returns paginated list of matching InterPro entries.
+
+### 4. Protein Annotations — Get InterPro Entries for a Protein
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro/protein/uniprot/{uniprot_accession}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro/protein/uniprot/P12345
+```
+Returns all InterPro entries annotating that protein.
+
+### 5. Proteins with a Given Entry
+```
+GET https://www.ebi.ac.uk/interpro/api/protein/uniprot/entry/interpro/{accession}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/protein/uniprot/entry/interpro/IPR000504
+```
+Returns paginated list of UniProt proteins annotated with that entry.
+
+### 6. Structure Mappings
+```
+GET https://www.ebi.ac.uk/interpro/api/structure/pdb/entry/interpro/{accession}
+```
+Example:
+```
+GET https://www.ebi.ac.uk/interpro/api/structure/pdb/entry/interpro/IPR000504
+```
+
+### 7. Entry by Type Filter
+```
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro?type=domain
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro?type=family
+GET https://www.ebi.ac.uk/interpro/api/entry/interpro?type=homologous_superfamily
+```
+
+### 8. Taxonomy Cross-Reference
+```
+GET https://www.ebi.ac.uk/interpro/api/taxonomy/uniprot/entry/interpro/{accession}
+```
+
+## Pagination
+Responses include `next` and `previous` URLs:
+```json
+{
+  "count": 1234,
+  "next": "https://www.ebi.ac.uk/interpro/api/entry/interpro?cursor=...&page_size=20",
+  "previous": null,
+  "results": [...]
+}
+```
+Use `?page_size=N` to control page size (default 20).
+
+## Entry Response Key Fields
+```json
+{
+  "metadata": {
+    "accession": "IPR000504",
+    "name": "RNA recognition motif domain",
+    "type": "domain",
+    "source_database": "interpro",
+    "member_databases": {"pfam": {"PF00076": "RRM_1"}},
+    "go_terms": [{"identifier": "GO:0003723", "name": "RNA binding"}],
+    "description": ["<p>The RNA recognition motif...</p>"]
+  }
+}
+```
+
+## Notes
+- The API follows a composable URL pattern: combine entity types (entry, protein, structure, taxonomy) to create cross-reference queries.
+- Member databases: pfam, smart, prosite, prints, panther, cdd, hamap, tigrfam, pirsf, sfld, ncbifam.