clearai-dsh 0.3.0 → 0.4.0

package/presets/clearai/template/skills/paper-lookup/references/arxiv.md

@@ -1,161 +0,0 @@
-# arXiv API
-
-arXiv is a preprint server for physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering, and economics.
-
-**Important:** The arXiv API returns **Atom XML**, not JSON. There is no JSON option.
-
-## Base URL
-
-```
-https://export.arxiv.org/api/query
-```
-
-## Authentication
-
-None required. Fully public.
-
-## Query Parameters
-
-```
-GET https://export.arxiv.org/api/query?search_query={query}&start={n}&max_results={n}
-```
-
-| Parameter | Required | Default | Description |
-|-----------|----------|---------|-------------|
-| `search_query` | Yes* | -- | Search using field prefixes + boolean operators |
-| `id_list` | Yes* | -- | Comma-separated arXiv IDs (e.g., `2103.15348,2005.14165`) |
-| `start` | No | 0 | Pagination offset (0-based) |
-| `max_results` | No | 10 | Results per request (max 2000; absolute max 30000) |
-| `sortBy` | No | `relevance` | `relevance`, `lastUpdatedDate`, `submittedDate` |
-| `sortOrder` | No | `descending` | `ascending` or `descending` |
-
-*At least one of `search_query` or `id_list` must be provided. They can be combined (intersection).
-
-## Search Field Prefixes
-
-| Prefix | Searches |
-|--------|----------|
-| `ti:` | Title |
-| `au:` | Author |
-| `abs:` | Abstract |
-| `co:` | Comment |
-| `jr:` | Journal reference |
-| `cat:` | Subject category |
-| `rn:` | Report number |
-| `all:` | All fields |
-
-## Boolean Operators
-
-- `AND` -- both conditions
-- `OR` -- either condition
-- `ANDNOT` -- exclude
-- Parentheses for grouping (URL-encode as `%28` / `%29`)
-- Quoted phrases (URL-encode as `%22`)
-
-## Example Queries
-
-**Search all fields:**
-```
-https://export.arxiv.org/api/query?search_query=all:transformer+attention&max_results=5
-```
-
-**Author + category:**
-```
-https://export.arxiv.org/api/query?search_query=au:hinton+AND+cat:cs.LG&max_results=10
-```
-
-**Title search:**
-```
-https://export.arxiv.org/api/query?search_query=ti:%22attention+is+all+you+need%22
-```
-
-**By ID:**
-```
-https://export.arxiv.org/api/query?id_list=2103.15348
-```
-
-**Multiple IDs:**
-```
-https://export.arxiv.org/api/query?id_list=2103.15348,2005.14165,1706.03762
-```
-
-**Date range:**
-```
-https://export.arxiv.org/api/query?search_query=cat:cs.AI+AND+submittedDate:[202401010000+TO+202412312359]
-```
-
-## Response Format (Atom XML)
-
-```xml
-<feed xmlns="http://www.w3.org/2005/Atom">
-  <opensearch:totalResults>1234</opensearch:totalResults>
-  <opensearch:startIndex>0</opensearch:startIndex>
-  <opensearch:itemsPerPage>10</opensearch:itemsPerPage>
-
-  <entry>
-    <id>http://arxiv.org/abs/1706.03762v7</id>
-    <title>Attention Is All You Need</title>
-    <summary>The dominant sequence transduction models are based on...</summary>
-    <published>2017-06-12T17:57:34Z</published>
-    <updated>2023-08-02T00:00:12Z</updated>
-    <author><name>Ashish Vaswani</name></author>
-    <author><name>Noam Shazeer</name></author>
-    <!-- more authors -->
-    <category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
-    <arxiv:primary_category term="cs.CL"/>
-    <link rel="alternate" href="http://arxiv.org/abs/1706.03762v7"/>
-    <link rel="related" type="application/pdf" href="http://arxiv.org/pdf/1706.03762v7"/>
-    <arxiv:doi>10.48550/arXiv.1706.03762</arxiv:doi>
-    <arxiv:comment>15 pages, 5 figures</arxiv:comment>
-    <arxiv:journal_ref>Advances in Neural Information Processing Systems 30 (NIPS 2017)</arxiv:journal_ref>
-  </entry>
-</feed>
-```
-
-### Key XML elements per entry
-
-| Element | Description |
-|---------|-------------|
-| `<id>` | arXiv URL: `http://arxiv.org/abs/{id}` |
-| `<title>` | Paper title |
-| `<summary>` | Abstract |
-| `<published>` | Original submission date (ISO 8601) |
-| `<updated>` | Date of latest version |
-| `<author><name>` | One per author |
-| `<category term="...">` | Subject categories |
-| `<arxiv:primary_category>` | Primary classification |
-| `<link rel="alternate">` | Abstract page URL |
-| `<link rel="related" title="pdf">` | PDF URL |
-| `<arxiv:doi>` | DOI (when available) |
-| `<arxiv:comment>` | Author comments |
-| `<arxiv:journal_ref>` | Journal reference |
-
-## Parsing Tips
-
-Since arXiv returns XML, you'll need to parse it. With `curl`, you can pipe the output and extract what you need. The XML namespace is `http://www.w3.org/2005/Atom` with arXiv extensions in `http://arxiv.org/schemas/atom`.
-
-For practical extraction, the key data is in `<entry>` elements. Each entry's `<id>` contains the arXiv ID in the URL path.
-
-## Common Categories
-
-| Category | Field |
-|----------|-------|
-| `cs.AI` | Artificial Intelligence |
-| `cs.CL` | Computation and Language (NLP) |
-| `cs.CV` | Computer Vision |
-| `cs.LG` | Machine Learning |
-| `stat.ML` | Machine Learning (Statistics) |
-| `q-bio` | Quantitative Biology |
-| `physics` | Physics (all subcategories) |
-| `math` | Mathematics (all subcategories) |
-| `econ` | Economics |
-| `eess` | Electrical Engineering and Systems Science |
-
-Full list: https://arxiv.org/category_taxonomy
-
-## Rate Limits
-
-- **1 request every 3 seconds** (hard limit)
-- Single connection at a time
-- Search results are cached daily -- same query won't show new results within 24 hours
-- For bulk data, use the OAI-PMH interface instead

package/presets/clearai/template/skills/paper-lookup/references/biorxiv.md

@@ -1,118 +0,0 @@
-# bioRxiv API
-
-bioRxiv is a preprint server for biology. The API provides metadata for preprints, including title, authors, abstract, DOI, and publication status.
-
-**Important:** The bioRxiv API has **no keyword search**. It supports date-range browsing and DOI lookup only. For keyword search of bioRxiv preprints, use Semantic Scholar, OpenAlex, or CORE instead.
-
-## Base URL
-
-```
-https://api.biorxiv.org
-```
-
-## Authentication
-
-None required. Fully public API.
-
-## Key Endpoints
-
-### 1. Content Detail -- Browse by date range
-
-```
-GET /details/biorxiv/{interval}/{cursor}/{format}
-```
-
-| Parameter | Values | Description |
-|-----------|--------|-------------|
-| `interval` | `YYYY-MM-DD/YYYY-MM-DD` | Date range (inclusive). Keep ranges narrow (1-3 days) to avoid timeouts. |
-| | `N` (integer) | N most recent preprints |
-| | `Nd` (integer + "d") | Last N days |
-| `cursor` | Integer (default `0`) | Pagination offset (100 results per page) |
-| `format` | `json` (default), `xml` | Response format |
-
-Optional query parameter: `?category=neuroscience` (filter by category, use underscores for spaces)
-
-**Examples:**
-```
-https://api.biorxiv.org/details/biorxiv/2024-01-01/2024-01-31/0
-https://api.biorxiv.org/details/biorxiv/5
-https://api.biorxiv.org/details/biorxiv/10d
-https://api.biorxiv.org/details/biorxiv/2024-01-01/2024-01-31?category=neuroscience
-```
-
-### 2. Content Detail -- DOI lookup
-
-```
-GET /details/biorxiv/{doi}/na/{format}
-```
-
-**Example:**
-```
-https://api.biorxiv.org/details/biorxiv/10.1101/2024.01.16.575895/na/json
-```
-
-### 3. Published Article Links
-
-```
-GET /pubs/biorxiv/{interval}/{cursor}
-GET /pubs/biorxiv/{doi}/na
-```
-
-Links preprints to their published journal versions. Accepts both preprint DOI and published DOI.
-
-### 4. Publisher Filter
-
-```
-GET /publisher/{prefix}/{interval}/{cursor}
-```
-
-Find bioRxiv papers published by a specific publisher (by DOI prefix).
-
-**Example:**
-```
-https://api.biorxiv.org/publisher/10.15252/2024-01-01/2024-06-01/0
-```
-
-## Response Format
-
-```json
-{
-  "messages": [{
-    "status": "ok",
-    "count": 100,
-    "total": "1029",
-    "cursor": 0
-  }],
-  "collection": [{
-    "title": "Paper title...",
-    "authors": "Surname, A.; Surname, B.",
-    "author_corresponding": "Full Name",
-    "author_corresponding_institution": "Institution",
-    "doi": "10.1101/2024.01.16.575895",
-    "date": "2024-01-20",
-    "version": "1",
-    "type": "new results",
-    "license": "cc_no",
-    "category": "cancer biology",
-    "jatsxml": "https://www.biorxiv.org/content/early/.../source.xml",
-    "abstract": "Full abstract text...",
-    "published": "10.1158/2159-8290.CD-24-0187",
-    "server": "bioRxiv"
-  }]
-}
-```
-
-- `published` is `"NA"` if not yet published in a journal, or the published DOI if it has been.
-- `type` values: `new results`, `confirmatory results`, `contradictory results`
-
-## Pagination
-
-All multi-result endpoints return **100 results per page**. Use `cursor` to paginate. The `messages` object tells you the `total` count.
-
-## Rate Limits
-
-No documented rate limits. No authentication required. Be reasonable with request frequency.
-
-## Categories
-
-`animal-behavior-and-cognition`, `biochemistry`, `bioengineering`, `bioinformatics`, `biophysics`, `cancer-biology`, `cell-biology`, `clinical-trials`, `developmental-biology`, `ecology`, `epidemiology`, `evolutionary-biology`, `genetics`, `genomics`, `immunology`, `microbiology`, `molecular-biology`, `neuroscience`, `paleontology`, `pathology`, `pharmacology-and-toxicology`, `physiology`, `plant-biology`, `scientific-communication-and-education`, `synthetic-biology`, `systems-biology`, `zoology`

package/presets/clearai/template/skills/paper-lookup/references/core.md

@@ -1,150 +0,0 @@
-# CORE API
-
-CORE aggregates open access research from 15,000+ repositories worldwide. It provides **full text** for 37M+ articles and metadata for 368M+ papers.
-
-## Base URL
-
-```
-https://api.core.ac.uk/v3
-```
-
-**Important:** GET search paths require a **trailing slash** (e.g., `/v3/search/works/` not `/v3/search/works`).
-
-## Authentication
-
-- **Header:** `Authorization: Bearer YOUR_API_KEY`
-- **Query param:** `?api_key=YOUR_API_KEY`
-- Register at: https://core.ac.uk/services/api
-
-**Without auth:** Basic metadata queries work, but full text is NOT available (returns "Not available for public API users").
-
-## Rate Limits (token-based)
-
-| User Type | Daily Tokens | Per-Minute Max |
-|-----------|-------------|----------------|
-| Unauthenticated | 100/day | 10/min |
-| Registered Personal | 1,000/day | 25/min |
-| Registered Academic | 5,000/day | 10/min |
-
-Simple queries cost 1 token. Downloads and scroll pagination cost 3-5 tokens.
-
-## Key Endpoints
-
-### 1. Search works
-
-```
-GET /v3/search/works/?q={query}&limit={n}&offset={n}
-```
-
-| Parameter | Default | Description |
-|-----------|---------|-------------|
-| `q` | required | Search query (supports field lookups, boolean operators) |
-| `limit` | 10 | Results per page (max 100) |
-| `offset` | 0 | Pagination offset |
-| `scroll` | false | Enable scroll pagination for >10,000 results |
-| `sort` | relevance | `relevance` or `recency` |
-
-**POST alternative** (for complex queries):
-```
-POST /v3/search/works
-Content-Type: application/json
-
-{"q": "machine learning", "limit": 10, "offset": 0}
-```
-
-**Example:**
-```
-https://api.core.ac.uk/v3/search/works/?q=CRISPR+gene+therapy&limit=10
-```
-
-### 2. Query language
-
-| Operator | Example | Description |
-|----------|---------|-------------|
-| AND | `title:"AI" AND authors:"Smith"` | Both conditions |
-| OR | `title:"AI" OR fullText:"Deep Learning"` | Either condition |
-| Grouping | `(title:"AI" OR title:"ML") AND yearPublished>"2020"` | Precedence |
-| Field lookup | `title:"Machine Learning"` | Search specific field |
-| Range | `yearPublished>2018` | Numeric comparison |
-| Exists | `_exists_:fullText` | Field must exist |
-| Phrase | `title:"Attention is all you need"` | Exact phrase |
-
-**Searchable fields:** `abstract`, `arxivId`, `authors`, `contributors`, `createdDate`, `dataProviders`, `depositedDate`, `documentType`, `doi`, `fullText`, `id`, `language`, `license`, `oai`, `title`, `yearPublished`
-
-### 3. Get work by ID
-
-```
-GET /v3/works/{id}
-```
-
-`id` is a CORE Work ID (integer). Example: `/v3/works/267312`
-
-### 4. Get output by ID
-
-```
-GET /v3/outputs/{id}
-```
-
-### 5. Download full text
-
-```
-GET /v3/outputs/{id}/download
-```
-
-Returns binary PDF. Requires authentication.
-
-```
-GET /v3/works/tei/{id}
-```
-
-Returns TEI XML format.
-
-### 6. Search outputs
-
-```
-GET /v3/search/outputs/?q={query}&limit={n}&offset={n}
-```
-
-Search by DOI: `q=doi:10.1038/nature12373`
-
-## Response Format
-
-### Search response
-```json
-{
-  "totalHits": 2281337,
-  "limit": 10,
-  "offset": 0,
-  "scrollId": null,
-  "results": [...]
-}
-```
-
-### Work object (key fields)
-```json
-{
-  "id": 8848131,
-  "title": "Attention Is All You Need",
-  "authors": [{"name": "Ashish Vaswani"}, ...],
-  "abstract": "The dominant sequence...",
-  "doi": "10.48550/arXiv.1706.03762",
-  "arxivId": "1706.03762",
-  "yearPublished": 2017,
-  "downloadUrl": "https://core.ac.uk/download/...",
-  "fullText": "Full text content (when authenticated)...",
-  "language": {"code": "en", "name": "English"},
-  "documentType": "research",
-  "citationCount": 145678,
-  "dataProviders": [{"name": "arXiv"}],
-  "links": [{"type": "download", "url": "..."}]
-}
-```
-
-## Pagination
-
-- **Standard:** `offset` + `limit` (max 10,000 results)
-- **Scroll:** Set `scroll=true`. Response includes `scrollId`. Use in subsequent requests to page beyond 10,000 (costs more tokens).
-
-## Error Handling
-
-Under heavy load, the API may return partial shard failure messages. These are transient -- retry after a brief wait.

package/presets/clearai/template/skills/paper-lookup/references/crossref.md

@@ -1,181 +0,0 @@
-# Crossref API
-
-Crossref is the DOI registration agency for scholarly content. It provides metadata for 150M+ works including journal articles, books, conference papers, datasets, and preprints.
-
-## Base URL
-
-```
-https://api.crossref.org
-```
-
-## Authentication
-
-None required. Add `mailto=you@example.com` to get into the **polite pool** (2x faster rate limits).
-
-## Rate Limits
-
-| Pool | Rate | Concurrency |
-|------|------|-------------|
-| Public (no mailto) | 5 req/sec | 1 concurrent |
-| Polite (with mailto) | 10 req/sec | 3 concurrent |
-
-HTTP 429 = temporarily blocked.
-
-## Key Endpoints
-
-### 1. Search works
-
-```
-GET /works?query={text}&rows={n}&mailto=you@example.com
-```
-
-| Parameter | Default | Description |
-|-----------|---------|-------------|
-| `query` | -- | Free-text search across all fields |
-| `query.author` | -- | Search author names |
-| `query.bibliographic` | -- | Search titles, authors, ISSNs, years |
-| `query.affiliation` | -- | Search affiliations |
-| `query.container-title` | -- | Search journal names |
-| `filter` | -- | Comma-separated `name:value` pairs |
-| `sort` | `score` | `score`, `published`, `issued`, `deposited`, `updated`, `is-referenced-by-count`, `references-count` |
-| `order` | `desc` | `asc` or `desc` |
-| `rows` | 20 | Results per page (max 1000) |
-| `offset` | 0 | Skip N results (max 10,000) |
-| `cursor` | -- | Use `*` for cursor-based deep pagination |
-| `select` | -- | Comma-separated field names to return |
-| `facet` | -- | Facet counts, e.g. `type-name:10` |
-| `sample` | -- | Return N random items (max 100) |
-
-**Example:**
-```
-https://api.crossref.org/works?query=CRISPR+gene+therapy&filter=from-pub-date:2024-01-01,type:journal-article,has-abstract:true&rows=5&sort=published&order=desc&mailto=you@example.com
-```
-
-### 2. Get work by DOI
-
-```
-GET /works/{doi}?mailto=you@example.com
-```
-
-URL-encode the DOI: `10.1038/nature12373` becomes `10.1038%2Fnature12373`
-
-**Example:**
-```
-https://api.crossref.org/works/10.1038%2Fnature12373?mailto=you@example.com
-```
-
-### 3. Journals
-
-```
-GET /journals?query={name}&rows={n}
-GET /journals/{issn}
-GET /journals/{issn}/works?query={text}&rows={n}
-```
-
-### 4. Funders
-
-```
-GET /funders?query={name}
-GET /funders/{id}
-GET /funders/{id}/works?rows={n}
-```
-
-Funder IDs are from the Funder Registry (e.g., `100000001` for NSF).
-
-### 5. Members (publishers)
-
-```
-GET /members?query={name}
-GET /members/{id}/works?rows={n}
-```
-
-## Key Filters
-
-### Date filters (accept `YYYY`, `YYYY-MM`, `YYYY-MM-DD`)
-| Filter | Description |
-|--------|-------------|
-| `from-pub-date` / `until-pub-date` | Publication date |
-| `from-print-pub-date` / `until-print-pub-date` | Print publication date |
-| `from-online-pub-date` / `until-online-pub-date` | Online publication date |
-| `from-posted-date` / `until-posted-date` | Posted date (preprints) |
-
-### Boolean filters
-| Filter | Description |
-|--------|-------------|
-| `has-abstract` | Has an abstract |
-| `has-orcid` | Has ORCID IDs |
-| `has-funder` | Has funder info |
-| `has-full-text` | Has full-text links |
-| `has-references` | Has reference list |
-| `has-license` | Has license info |
-
-### Value filters
-| Filter | Description |
-|--------|-------------|
-| `type` | `journal-article`, `posted-content`, `book-chapter`, `proceedings-article`, etc. |
-| `issn` | Journal ISSN |
-| `doi` | Specific DOI |
-| `orcid` | Contributor ORCID |
-| `funder` | Funder Registry ID |
-| `member` | Crossref member ID |
-| `prefix` | DOI prefix |
-| `license.url` | License URL |
-| `update-type` | `correction`, `retraction` |
-
-**Syntax:** `filter=name1:value1,name2:value2`
-
-## Pagination
-
-### Offset-based (max 10,000)
-```
-/works?query=cancer&rows=100&offset=200
-```
-
-### Cursor-based (unlimited)
-1. First request: `?cursor=*&rows=100`
-2. Response includes `next-cursor`
-3. Next request: `?cursor={next-cursor-value}&rows=100`
-4. Cursors expire after 5 minutes
-
-## Response Format
-
-### List response
-```json
-{
-  "status": "ok",
-  "message-type": "work-list",
-  "message": {
-    "total-results": 2779116,
-    "items-per-page": 20,
-    "next-cursor": "...",
-    "items": [...]
-  }
-}
-```
-
-### Work object (key fields)
-```json
-{
-  "DOI": "10.1038/nature12373",
-  "title": ["Nanometre-scale thermometry in a living cell"],
-  "author": [{"given": "G.", "family": "Kucsko", "sequence": "first"}],
-  "publisher": "Springer Science and Business Media LLC",
-  "type": "journal-article",
-  "published": {"date-parts": [[2013, 7, 31]]},
-  "container-title": ["Nature"],
-  "ISSN": ["0028-0836", "1476-4687"],
-  "volume": "500",
-  "issue": "7460",
-  "page": "54-58",
-  "is-referenced-by-count": 1745,
-  "references-count": 30,
-  "abstract": "<p>Abstract text with HTML tags...</p>",
-  "license": [{"URL": "...", "content-version": "vor"}],
-  "link": [{"URL": "...", "content-type": "application/pdf"}],
-  "reference": [{"key": "...", "doi-asserted-by": "crossref", "DOI": "..."}],
-  "subject": ["Multidisciplinary"],
-  "language": "en"
-}
-```
-
-Note: `title` and `container-title` are arrays. `published.date-parts` is `[[year, month, day]]`. Abstract may contain HTML tags.