@synsci/cli-darwin-x64-baseline 1.1.76 → 1.1.78

package/bin/skills/uniprot-database/references/query_syntax.md

@@ -0,0 +1,256 @@
+# UniProt Query Syntax Reference
+
+Comprehensive guide to UniProt search query syntax for constructing complex searches.
+
+## Basic Syntax
+
+### Simple Queries
+```
+insulin
+kinase
+```
+
+### Field-Specific Searches
+```
+gene:BRCA1
+accession:P12345
+organism_name:human
+protein_name:kinase
+```
+
+## Boolean Operators
+
+### AND (both terms must be present)
+```
+insulin AND diabetes
+kinase AND human
+gene:BRCA1 AND reviewed:true
+```
+
+### OR (either term can be present)
+```
+diabetes OR insulin
+(cancer OR tumor) AND human
+```
+
+### NOT (exclude terms)
+```
+kinase NOT human
+protein_name:kinase NOT organism_name:mouse
+```
+
+### Grouping with Parentheses
+```
+(diabetes OR insulin) AND reviewed:true
+(gene:BRCA1 OR gene:BRCA2) AND organism_id:9606
+```
+
+## Common Search Fields
+
+### Identification
+- `accession:P12345` - UniProt accession number
+- `id:INSR_HUMAN` - Entry name
+- `gene:BRCA1` - Gene name
+- `gene_exact:BRCA1` - Exact gene name match
+
+### Organism/Taxonomy
+- `organism_name:human` - Organism name
+- `organism_name:"Homo sapiens"` - Exact organism name (use quotes for multi-word)
+- `organism_id:9606` - NCBI taxonomy ID
+- `taxonomy_id:9606` - Same as organism_id
+- `taxonomy_name:"Homo sapiens"` - Taxonomy name
+
+### Protein Information
+- `protein_name:insulin` - Protein name
+- `protein_name:"insulin receptor"` - Exact protein name
+- `reviewed:true` - Only Swiss-Prot (reviewed) entries
+- `reviewed:false` - Only TrEMBL (unreviewed) entries
+
+### Sequence Properties
+- `length:[100 TO 500]` - Sequence length range
+- `mass:[50000 TO 100000]` - Molecular mass in Daltons
+- `sequence:MVLSPADKTNVK` - Exact sequence match
+- `fragment:false` - Exclude fragment sequences
+
+### Gene Ontology (GO)
+- `go:0005515` - GO term ID (0005515 = protein binding)
+- `go_f:* ` - Any molecular function
+- `go_p:*` - Any biological process
+- `go_c:*` - Any cellular component
+
+### Annotations
+- `annotation:(type:signal)` - Has signal peptide annotation
+- `annotation:(type:transmem)` - Has transmembrane region
+- `cc_function:*` - Has function comment
+- `cc_interaction:*` - Has interaction comment
+- `ft_domain:*` - Has domain feature
+
+### Database Cross-References
+- `xref:pdb` - Has PDB structure
+- `xref:ensembl` - Has Ensembl reference
+- `database:pdb` - Same as xref
+- `database:(type:pdb)` - Alternative syntax
+
+### Protein Families and Domains
+- `family:"protein kinase"` - Protein family
+- `keyword:"Protein kinase"` - Keyword annotation
+- `cc_similarity:*` - Has similarity comment
+
+## Range Queries
+
+### Numeric Ranges
+```
+length:[100 TO 500]          # Between 100 and 500
+mass:[* TO 50000]            # Less than or equal to 50000
+created:[2023-01-01 TO *]   # Created after Jan 1, 2023
+```
+
+### Date Ranges
+```
+created:[2023-01-01 TO 2023-12-31]
+modified:[2024-01-01 TO *]
+```
+
+## Wildcards
+
+### Single Character (?)
+```
+gene:BRCA?      # Matches BRCA1, BRCA2, etc.
+```
+
+### Multiple Characters (*)
+```
+gene:BRCA*      # Matches BRCA1, BRCA2, BRCA1P1, etc.
+protein_name:kinase*
+organism_name:Homo*
+```
+
+## Advanced Searches
+
+### Existence Queries
+```
+cc_function:*              # Has any function annotation
+ft_domain:*                # Has any domain feature
+xref:pdb                   # Has PDB structure
+```
+
+### Combined Complex Queries
+```
+# Human reviewed kinases with PDB structure
+(protein_name:kinase OR family:kinase) AND organism_id:9606 AND reviewed:true AND xref:pdb
+
+# Cancer-related proteins excluding mice
+(disease:cancer OR keyword:cancer) NOT organism_name:mouse
+
+# Membrane proteins with signal peptides
+annotation:(type:transmem) AND annotation:(type:signal) AND reviewed:true
+
+# Recently updated human proteins
+organism_id:9606 AND modified:[2024-01-01 TO *] AND reviewed:true
+```
+
+## Field-Specific Examples
+
+### Protein Names
+```
+protein_name:"insulin receptor"    # Exact phrase
+protein_name:insulin*              # Starts with insulin
+recommended_name:insulin           # Recommended name only
+alternative_name:insulin           # Alternative names only
+```
+
+### Genes
+```
+gene:BRCA1                        # Gene symbol
+gene_exact:BRCA1                  # Exact gene match
+olnName:BRCA1                     # Ordered locus name
+orfName:BRCA1                     # ORF name
+```
+
+### Organisms
+```
+organism_name:human               # Common name
+organism_name:"Homo sapiens"      # Scientific name
+organism_id:9606                  # Taxonomy ID
+lineage:primates                  # Taxonomic lineage
+```
+
+### Features
+```
+ft_signal:*                       # Signal peptide
+ft_transmem:*                     # Transmembrane region
+ft_domain:"Protein kinase"        # Specific domain
+ft_binding:*                      # Binding site
+ft_site:*                         # Any site
+```
+
+### Comments (cc_)
+```
+cc_function:*                     # Function description
+cc_catalytic_activity:*           # Catalytic activity
+cc_pathway:*                      # Pathway involvement
+cc_interaction:*                  # Protein interactions
+cc_subcellular_location:*         # Subcellular location
+cc_tissue_specificity:*           # Tissue specificity
+cc_disease:cancer                 # Disease association
+```
+
+## Tips and Best Practices
+
+1. **Use quotes for exact phrases**: `organism_name:"Homo sapiens"` not `organism_name:Homo sapiens`
+
+2. **Filter by review status**: Add `AND reviewed:true` for high-quality Swiss-Prot entries
+
+3. **Combine wildcards carefully**: `*kinase*` may be too broad; `kinase*` is more specific
+
+4. **Use parentheses for complex logic**: `(A OR B) AND (C OR D)` is clearer than `A OR B AND C OR D`
+
+5. **Numeric ranges are inclusive**: `length:[100 TO 500]` includes both 100 and 500
+
+6. **Field prefixes**: Learn common prefixes:
+   - `cc_` = Comments
+   - `ft_` = Features
+   - `go_` = Gene Ontology
+   - `xref_` = Cross-references
+
+7. **Check field names**: Use the API's `/configure/uniprotkb/result-fields` endpoint to see all available fields
+
+## Query Validation
+
+Test queries using:
+- **Web interface**: https://www.uniprot.org/uniprotkb
+- **API**: https://rest.uniprot.org/uniprotkb/search?query=YOUR_QUERY
+- **API documentation**: https://www.uniprot.org/help/query-fields
+
+## Common Patterns
+
+### Find well-characterized proteins
+```
+reviewed:true AND xref:pdb AND cc_function:*
+```
+
+### Find disease-associated proteins
+```
+cc_disease:* AND organism_id:9606 AND reviewed:true
+```
+
+### Find proteins with experimental evidence
+```
+existence:"Evidence at protein level" AND reviewed:true
+```
+
+### Find secreted proteins
+```
+cc_subcellular_location:secreted AND reviewed:true
+```
+
+### Find drug targets
+```
+keyword:"Pharmaceutical" OR keyword:"Drug target"
+```
+
+## Resources
+
+- Full query field reference: https://www.uniprot.org/help/query-fields
+- API query documentation: https://www.uniprot.org/help/api_queries
+- Text search documentation: https://www.uniprot.org/help/text-search

package/bin/skills/uniprot-database/scripts/uniprot_client.py

@@ -0,0 +1,341 @@
+#!/usr/bin/env python3
+"""
+UniProt REST API Client
+
+A Python client for interacting with the UniProt REST API.
+Provides helper functions for common operations including search,
+retrieval, ID mapping, and streaming.
+
+Usage examples:
+    # Search for proteins
+    results = search_proteins("insulin AND organism_name:human", format="json")
+
+    # Get a single protein
+    protein = get_protein("P12345", format="fasta")
+
+    # Map IDs
+    mapped = map_ids(["P12345", "P04637"], from_db="UniProtKB_AC-ID", to_db="PDB")
+
+    # Stream large results
+    for batch in stream_results("taxonomy_id:9606 AND reviewed:true", format="fasta"):
+        process(batch)
+"""
+
+import requests
+import sys
+import time
+import json
+from typing import List, Dict, Optional, Generator
+from urllib.parse import urlencode
+
+BASE_URL = "https://rest.uniprot.org"
+POLLING_INTERVAL = 3  # seconds
+
+
+def search_proteins(query: str, format: str = "json",
+                   fields: Optional[List[str]] = None,
+                   size: int = 25) -> Dict:
+    """
+    Search UniProt database with a query.
+
+    Args:
+        query: Search query (e.g., "insulin AND organism_name:human")
+        format: Response format (json, tsv, xlsx, xml, fasta, txt, rdf)
+        fields: List of fields to return (e.g., ["accession", "gene_names", "organism_name"])
+        size: Number of results per page (default 25, max 500)
+
+    Returns:
+        Response data in requested format
+    """
+    endpoint = f"{BASE_URL}/uniprotkb/search"
+
+    params = {
+        "query": query,
+        "format": format,
+        "size": size
+    }
+
+    if fields:
+        params["fields"] = ",".join(fields)
+
+    response = requests.get(endpoint, params=params)
+    response.raise_for_status()
+
+    if format == "json":
+        return response.json()
+    else:
+        return response.text
+
+
+def get_protein(accession: str, format: str = "json") -> str:
+    """
+    Retrieve a single protein entry by accession number.
+
+    Args:
+        accession: UniProt accession number (e.g., "P12345")
+        format: Response format (json, txt, xml, fasta, gff, rdf)
+
+    Returns:
+        Protein data in requested format
+    """
+    endpoint = f"{BASE_URL}/uniprotkb/{accession}.{format}"
+
+    response = requests.get(endpoint)
+    response.raise_for_status()
+
+    if format == "json":
+        return response.json()
+    else:
+        return response.text
+
+
+def batch_retrieve(accessions: List[str], format: str = "json",
+                  fields: Optional[List[str]] = None) -> str:
+    """
+    Retrieve multiple protein entries efficiently.
+
+    Args:
+        accessions: List of UniProt accession numbers
+        format: Response format
+        fields: List of fields to return
+
+    Returns:
+        Combined results in requested format
+    """
+    query = " OR ".join([f"accession:{acc}" for acc in accessions])
+    return search_proteins(query, format=format, fields=fields, size=len(accessions))
+
+
+def stream_results(query: str, format: str = "fasta",
+                  fields: Optional[List[str]] = None,
+                  chunk_size: int = 8192) -> Generator[str, None, None]:
+    """
+    Stream large result sets without pagination.
+
+    Args:
+        query: Search query
+        format: Response format
+        fields: List of fields to return
+        chunk_size: Size of chunks to yield
+
+    Yields:
+        Chunks of response data
+    """
+    endpoint = f"{BASE_URL}/uniprotkb/stream"
+
+    params = {
+        "query": query,
+        "format": format
+    }
+
+    if fields:
+        params["fields"] = ",".join(fields)
+
+    response = requests.get(endpoint, params=params, stream=True)
+    response.raise_for_status()
+
+    for chunk in response.iter_content(chunk_size=chunk_size, decode_unicode=True):
+        if chunk:
+            yield chunk
+
+
+def map_ids(ids: List[str], from_db: str, to_db: str,
+           format: str = "json") -> Dict:
+    """
+    Map protein identifiers between different database systems.
+
+    Args:
+        ids: List of identifiers to map (max 100,000)
+        from_db: Source database (e.g., "UniProtKB_AC-ID", "Gene_Name")
+        to_db: Target database (e.g., "PDB", "Ensembl", "RefSeq_Protein")
+        format: Response format
+
+    Returns:
+        Mapping results
+
+    Note:
+        - Maximum 100,000 IDs per job
+        - Results stored for 7 days
+        - See id_mapping_databases.md for all supported databases
+    """
+    if len(ids) > 100000:
+        raise ValueError("Maximum 100,000 IDs allowed per mapping job")
+
+    # Step 1: Submit job
+    submit_endpoint = f"{BASE_URL}/idmapping/run"
+
+    data = {
+        "from": from_db,
+        "to": to_db,
+        "ids": ",".join(ids)
+    }
+
+    response = requests.post(submit_endpoint, data=data)
+    response.raise_for_status()
+    job_id = response.json()["jobId"]
+
+    # Step 2: Poll for completion
+    status_endpoint = f"{BASE_URL}/idmapping/status/{job_id}"
+
+    while True:
+        response = requests.get(status_endpoint)
+        response.raise_for_status()
+        status = response.json()
+
+        if "results" in status or "failedIds" in status:
+            break
+
+        time.sleep(POLLING_INTERVAL)
+
+    # Step 3: Retrieve results
+    results_endpoint = f"{BASE_URL}/idmapping/results/{job_id}"
+
+    params = {"format": format}
+    response = requests.get(results_endpoint, params=params)
+    response.raise_for_status()
+
+    if format == "json":
+        return response.json()
+    else:
+        return response.text
+
+
+def get_available_fields() -> List[Dict]:
+    """
+    Get list of all available fields for queries.
+
+    Returns:
+        List of field definitions with names and descriptions
+    """
+    endpoint = f"{BASE_URL}/configure/uniprotkb/result-fields"
+
+    response = requests.get(endpoint)
+    response.raise_for_status()
+
+    return response.json()
+
+
+def get_id_mapping_databases() -> Dict:
+    """
+    Get list of all supported databases for ID mapping.
+
+    Returns:
+        Dictionary of database groups and their supported databases
+    """
+    endpoint = f"{BASE_URL}/configure/idmapping/fields"
+
+    response = requests.get(endpoint)
+    response.raise_for_status()
+
+    return response.json()
+
+
+def main():
+    """Command-line interface for UniProt database queries."""
+    import argparse
+
+    parser = argparse.ArgumentParser(
+        description='Query UniProt database using REST API',
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  # Search for proteins
+  %(prog)s --search "insulin AND organism_name:human" --format json
+
+  # Get a specific protein
+  %(prog)s --get P01308 --format fasta
+
+  # Map IDs from UniProt to PDB
+  %(prog)s --map P01308,P04637 --from UniProtKB_AC-ID --to PDB
+
+  # Stream large results
+  %(prog)s --stream "taxonomy_id:9606 AND reviewed:true" --format fasta
+
+  # List available fields
+  %(prog)s --list-fields
+
+  # List mapping databases
+  %(prog)s --list-databases
+        """
+    )
+
+    # Main operation arguments (mutually exclusive)
+    group = parser.add_mutually_exclusive_group(required=True)
+    group.add_argument('--search', '-s', help='Search query string')
+    group.add_argument('--get', '-g', help='Get protein by accession number')
+    group.add_argument('--map', '-m', help='Map IDs (comma-separated)')
+    group.add_argument('--stream', help='Stream large result sets')
+    group.add_argument('--list-fields', action='store_true',
+                      help='List all available query fields')
+    group.add_argument('--list-databases', action='store_true',
+                      help='List all ID mapping databases')
+
+    # Format options
+    parser.add_argument('--format', '-f', default='json',
+                       help='Output format (json, tsv, xlsx, xml, fasta, txt, rdf)')
+
+    # Search-specific options
+    parser.add_argument('--fields', help='Comma-separated list of fields to return')
+    parser.add_argument('--size', type=int, default=25,
+                       help='Number of results (default: 25, max: 500)')
+
+    # Mapping-specific options
+    parser.add_argument('--from', dest='from_db',
+                       help='Source database for ID mapping')
+    parser.add_argument('--to', dest='to_db',
+                       help='Target database for ID mapping')
+
+    args = parser.parse_args()
+
+    try:
+        if args.list_fields:
+            fields = get_available_fields()
+            print(json.dumps(fields, indent=2))
+
+        elif args.list_databases:
+            databases = get_id_mapping_databases()
+            print(json.dumps(databases, indent=2))
+
+        elif args.search:
+            fields_list = args.fields.split(',') if args.fields else None
+            results = search_proteins(
+                args.search,
+                format=args.format,
+                fields=fields_list,
+                size=args.size
+            )
+            if args.format == 'json':
+                print(json.dumps(results, indent=2))
+            else:
+                print(results)
+
+        elif args.get:
+            protein = get_protein(args.get, format=args.format)
+            if args.format == 'json':
+                print(json.dumps(protein, indent=2))
+            else:
+                print(protein)
+
+        elif args.map:
+            if not args.from_db or not args.to_db:
+                parser.error("--map requires --from and --to arguments")
+
+            ids = [id.strip() for id in args.map.split(',')]
+            mapping = map_ids(ids, args.from_db, args.to_db, format=args.format)
+            if args.format == 'json':
+                print(json.dumps(mapping, indent=2))
+            else:
+                print(mapping)
+
+        elif args.stream:
+            fields_list = args.fields.split(',') if args.fields else None
+            for chunk in stream_results(args.stream, format=args.format, fields=fields_list):
+                print(chunk, end='')
+
+    except Exception as e:
+        print(f"Error: {e}", file=sys.stderr)
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()