@wentorai/research-plugins 1.1.0 → 1.2.0

package/skills/literature/fulltext/dataverse-api/SKILL.md

@@ -0,0 +1,215 @@
+---
+name: dataverse-api
+description: "Deposit and discover research datasets via Harvard Dataverse API"
+metadata:
+  openclaw:
+    emoji: "🗄️"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["Dataverse", "research data", "data repository", "Harvard", "dataset deposit", "data sharing"]
+    source: "https://dataverse.org/"
+---
+
+# Harvard Dataverse API
+
+## Overview
+
+Dataverse is an open-source research data repository platform developed by Harvard IQSS, hosting 150K+ datasets across 80+ installations worldwide. The Harvard Dataverse alone has 130K+ datasets covering social science, natural science, and humanities. The API supports search, metadata retrieval, file download, and dataset deposit. Free, no authentication for read access.
+
+## API Endpoints
+
+### Base URL
+
+```
+https://dataverse.harvard.edu/api
+```
+
+### Search
+
+```bash
+# Search datasets
+curl "https://dataverse.harvard.edu/api/search?q=climate+change&type=dataset&per_page=20"
+
+# Search files within datasets
+curl "https://dataverse.harvard.edu/api/search?q=temperature+data&type=file&per_page=20"
+
+# Filter by subject
+curl "https://dataverse.harvard.edu/api/search?q=survey+data&type=dataset&\
+fq=subject_ss:\"Social Sciences\""
+
+# Filter by publication date
+curl "https://dataverse.harvard.edu/api/search?q=genomics&type=dataset&\
+fq=dateSort:[2024-01-01T00:00:00Z TO *]"
+
+# Sort by relevance or date
+curl "https://dataverse.harvard.edu/api/search?q=machine+learning&type=dataset&\
+sort=date&order=desc"
+```
+
+### Get Dataset Metadata
+
+```bash
+# By persistent ID (DOI)
+curl "https://dataverse.harvard.edu/api/datasets/:persistentId/?persistentId=doi:10.7910/DVN/EXAMPLE"
+
+# By dataset ID
+curl "https://dataverse.harvard.edu/api/datasets/12345"
+
+# Get dataset versions
+curl "https://dataverse.harvard.edu/api/datasets/:persistentId/versions?persistentId=doi:10.7910/DVN/EXAMPLE"
+```
+
+### Download Files
+
+```bash
+# Download a specific file by ID
+curl -O "https://dataverse.harvard.edu/api/access/datafile/67890"
+
+# Download with original format
+curl -O "https://dataverse.harvard.edu/api/access/datafile/67890?format=original"
+
+# Download all files in a dataset (as zip)
+curl -O "https://dataverse.harvard.edu/api/access/dataset/:persistentId/?persistentId=doi:10.7910/DVN/EXAMPLE"
+```
+
+### Query Parameters (Search)
+
+| Parameter | Description | Example |
+|-----------|-------------|---------|
+| `q` | Search query | `q=voter+turnout` |
+| `type` | Item type | `dataset`, `file`, `dataverse` |
+| `per_page` | Results per page (max 1000) | `per_page=50` |
+| `start` | Pagination offset | `start=50` |
+| `sort` | Sort field | `name`, `date` |
+| `order` | Sort order | `asc`, `desc` |
+| `fq` | Filter query (Solr) | `fq=subject_ss:"Medicine"` |
+
+## Response Structure
+
+```json
+{
+  "status": "OK",
+  "data": {
+    "q": "climate change",
+    "total_count": 2450,
+    "items": [
+      {
+        "name": "Global Temperature Dataset 2024",
+        "type": "dataset",
+        "url": "https://doi.org/10.7910/DVN/EXAMPLE",
+        "global_id": "doi:10.7910/DVN/EXAMPLE",
+        "description": "Monthly global temperature anomalies...",
+        "published_at": "2024-03-15",
+        "publisher": "Harvard Dataverse",
+        "subjects": ["Earth and Environmental Sciences"],
+        "fileCount": 12,
+        "citation": "Smith, J. (2024). Global Temperature Dataset..."
+      }
+    ]
+  }
+}
+```
+
+## Python Usage
+
+```python
+import requests
+
+BASE_URL = "https://dataverse.harvard.edu/api"
+
+
+def search_datasets(query: str, per_page: int = 20,
+                    subject: str = None) -> list:
+    """Search Harvard Dataverse for datasets."""
+    params = {
+        "q": query,
+        "type": "dataset",
+        "per_page": per_page,
+        "sort": "date",
+        "order": "desc",
+    }
+    if subject:
+        params["fq"] = f'subject_ss:"{subject}"'
+
+    resp = requests.get(f"{BASE_URL}/search", params=params)
+    resp.raise_for_status()
+    data = resp.json()
+
+    results = []
+    for item in data.get("data", {}).get("items", []):
+        results.append({
+            "name": item.get("name"),
+            "doi": item.get("global_id"),
+            "description": item.get("description", "")[:300],
+            "published": item.get("published_at"),
+            "subjects": item.get("subjects", []),
+            "files": item.get("fileCount", 0),
+            "url": item.get("url"),
+        })
+    return results
+
+
+def get_dataset_files(doi: str) -> list:
+    """List files in a dataset."""
+    resp = requests.get(
+        f"{BASE_URL}/datasets/:persistentId/",
+        params={"persistentId": doi},
+    )
+    resp.raise_for_status()
+    data = resp.json().get("data", {})
+
+    files = []
+    version = data.get("latestVersion", {})
+    for f in version.get("files", []):
+        df = f.get("dataFile", {})
+        files.append({
+            "id": df.get("id"),
+            "filename": df.get("filename"),
+            "size": df.get("filesize"),
+            "content_type": df.get("contentType"),
+            "md5": df.get("md5"),
+        })
+    return files
+
+
+def download_file(file_id: int, output_path: str):
+    """Download a file from Dataverse."""
+    resp = requests.get(
+        f"{BASE_URL}/access/datafile/{file_id}",
+        stream=True,
+    )
+    resp.raise_for_status()
+    with open(output_path, "wb") as f:
+        for chunk in resp.iter_content(chunk_size=8192):
+            f.write(chunk)
+
+
+# Example: find social science datasets
+datasets = search_datasets("income inequality",
+                           subject="Social Sciences")
+for ds in datasets:
+    print(f"[{ds['published']}] {ds['name']} ({ds['files']} files)")
+    print(f"  DOI: {ds['doi']}")
+
+# Example: list files in a dataset
+# files = get_dataset_files("doi:10.7910/DVN/EXAMPLE")
+# for f in files:
+#     print(f"  {f['filename']} ({f['size']} bytes)")
+```
+
+## Other Dataverse Installations
+
+| Installation | URL | Focus |
+|-------------|-----|-------|
+| Harvard Dataverse | dataverse.harvard.edu | Multi-discipline |
+| UNC Dataverse | dataverse.unc.edu | Social science |
+| AUSSDA | data.aussda.at | Austrian social science |
+| Borealis (Canada) | borealisdata.ca | Canadian research |
+| DataverseNL | dataverse.nl | Dutch research |
+
+## References
+
+- [Harvard Dataverse](https://dataverse.harvard.edu/)
+- [Dataverse API Guide](https://guides.dataverse.org/en/latest/api/)
+- [Dataverse Project](https://dataverse.org/)
+- King, G. (2007). "An Introduction to the Dataverse Network." *Sociological Methods & Research* 36(2).

package/skills/literature/fulltext/hal-archive-api/SKILL.md

@@ -0,0 +1,218 @@
+---
+name: hal-archive-api
+description: "Access French and European research via the HAL open archive API"
+metadata:
+  openclaw:
+    emoji: "🇫🇷"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["HAL", "French research", "open archive", "CNRS", "European research", "institutional repository"]
+    source: "https://api.archives-ouvertes.fr/"
+---
+
+# HAL Open Archive API
+
+## Overview
+
+HAL (Hyper Articles en Ligne) is France's national open archive for scholarly deposits. Managed by CNRS, it hosts 4M+ full-text documents from French research institutions and international collaborators. The API provides Solr-based search with full metadata, PDF links, and OAI-PMH harvesting. Free, no authentication required.
+
+## API Endpoints
+
+### Search API
+
+```bash
+# Keyword search
+curl "https://api.archives-ouvertes.fr/search/?q=machine+learning&rows=20&wt=json"
+
+# Search specific fields
+curl "https://api.archives-ouvertes.fr/search/?q=title_s:\"deep learning\"&wt=json"
+
+# Filter by document type
+curl "https://api.archives-ouvertes.fr/search/?q=neural+networks&\
+fq=docType_s:ART&rows=20&wt=json"
+
+# Filter by year and language
+curl "https://api.archives-ouvertes.fr/search/?q=climate+change&\
+fq=producedDateY_i:[2023 TO 2026]&fq=language_s:en&wt=json"
+
+# Filter by institution
+curl "https://api.archives-ouvertes.fr/search/?q=robotics&\
+fq=structId_i:441569&wt=json"
+
+# Return specific fields
+curl "https://api.archives-ouvertes.fr/search/?q=CRISPR&\
+fl=halId_s,title_s,authFullName_s,producedDateY_i,uri_s,files_s&wt=json"
+```
+
+### Search Fields
+
+| Field | Description | Example |
+|-------|-------------|---------|
+| `title_s` | Title | `title_s:"attention mechanism"` |
+| `authFullName_s` | Author name | `authFullName_s:"Yann LeCun"` |
+| `abstract_s` | Abstract | `abstract_s:transformer` |
+| `keyword_s` | Keywords | `keyword_s:"natural language"` |
+| `producedDateY_i` | Year | `producedDateY_i:2024` |
+| `docType_s` | Document type | `docType_s:ART` |
+| `language_s` | Language | `language_s:en` |
+| `domain_s` | Domain/subject | `domain_s:info.info-ai` |
+| `journalTitle_s` | Journal name | `journalTitle_s:"Nature"` |
+| `structId_i` | Institution ID | Lab/university ID |
+
+### Document Types
+
+| Code | Type |
+|------|------|
+| `ART` | Journal article |
+| `COMM` | Conference paper |
+| `THESE` | PhD thesis |
+| `HDR` | Habilitation thesis |
+| `REPORT` | Report |
+| `COUV` | Book chapter |
+| `OUV` | Book |
+| `POSTER` | Poster |
+| `UNDEFINED` | Preprint/other |
+
+### Query Parameters
+
+| Parameter | Description |
+|-----------|-------------|
+| `q` | Solr query |
+| `fq` | Filter query |
+| `fl` | Fields to return |
+| `rows` | Results per page (max 10000) |
+| `start` | Pagination offset |
+| `sort` | Sort order (e.g., `producedDateY_i desc`) |
+| `wt` | Format: `json`, `xml`, `csv` |
+
+## Response Structure
+
+```json
+{
+  "response": {
+    "numFound": 12500,
+    "start": 0,
+    "docs": [
+      {
+        "halId_s": "hal-01234567",
+        "title_s": ["Deep Learning for Climate Modeling"],
+        "authFullName_s": ["Marie Dupont", "Jean Martin"],
+        "producedDateY_i": 2024,
+        "docType_s": "ART",
+        "journalTitle_s": "Environmental Modelling",
+        "uri_s": "https://hal.science/hal-01234567",
+        "files_s": ["https://hal.science/hal-01234567/document"],
+        "domain_s": ["sde.es", "info.info-ai"],
+        "abstract_s": ["We propose a novel deep learning approach..."],
+        "language_s": ["en"]
+      }
+    ]
+  }
+}
+```
+
+## Python Usage
+
+```python
+import requests
+
+BASE_URL = "https://api.archives-ouvertes.fr/search/"
+
+
+def search_hal(query: str, rows: int = 20,
+               doc_type: str = None, from_year: int = None,
+               language: str = None) -> list:
+    """Search HAL open archive."""
+    params = {
+        "q": query,
+        "wt": "json",
+        "rows": rows,
+        "fl": "halId_s,title_s,authFullName_s,producedDateY_i,"
+              "uri_s,files_s,docType_s,journalTitle_s,abstract_s",
+        "sort": "producedDateY_i desc",
+    }
+
+    fq = []
+    if doc_type:
+        fq.append(f"docType_s:{doc_type}")
+    if from_year:
+        fq.append(f"producedDateY_i:[{from_year} TO 2030]")
+    if language:
+        fq.append(f"language_s:{language}")
+    if fq:
+        params["fq"] = fq
+
+    resp = requests.get(BASE_URL, params=params)
+    resp.raise_for_status()
+    data = resp.json()
+
+    results = []
+    for doc in data.get("response", {}).get("docs", []):
+        title = doc.get("title_s", [""])[0] if isinstance(
+            doc.get("title_s"), list) else doc.get("title_s", "")
+        results.append({
+            "hal_id": doc.get("halId_s"),
+            "title": title,
+            "authors": doc.get("authFullName_s", []),
+            "year": doc.get("producedDateY_i"),
+            "type": doc.get("docType_s"),
+            "journal": doc.get("journalTitle_s"),
+            "url": doc.get("uri_s"),
+            "pdf": doc.get("files_s", [None])[0],
+        })
+    return results
+
+
+def search_theses(topic: str, from_year: int = 2020) -> list:
+    """Find French PhD theses on a topic."""
+    return search_hal(topic, rows=50, doc_type="THESE",
+                      from_year=from_year)
+
+
+def get_institution_publications(struct_id: int,
+                                 from_year: int = 2023) -> list:
+    """Get publications from a specific institution."""
+    params = {
+        "q": "*:*",
+        "fq": [f"structId_i:{struct_id}",
+               f"producedDateY_i:[{from_year} TO 2030]"],
+        "wt": "json",
+        "rows": 100,
+        "fl": "halId_s,title_s,authFullName_s,producedDateY_i,docType_s",
+        "sort": "producedDateY_i desc",
+    }
+    resp = requests.get(BASE_URL, params=params)
+    resp.raise_for_status()
+    return resp.json().get("response", {}).get("docs", [])
+
+
+# Example: find recent French AI research
+papers = search_hal("intelligence artificielle", from_year=2024)
+for p in papers:
+    pdf = " [PDF]" if p["pdf"] else ""
+    print(f"[{p['year']}] {p['title']}{pdf}")
+
+# Example: find PhD theses on NLP
+theses = search_theses("natural language processing")
+for t in theses:
+    print(f"{t['title']} — {', '.join(t['authors'][:2])}")
+```
+
+## HAL Domains
+
+| Code | Domain |
+|------|--------|
+| `info` | Computer Science |
+| `math` | Mathematics |
+| `phys` | Physics |
+| `sde` | Environmental Sciences |
+| `sdv` | Life Sciences |
+| `shs` | Social Sciences & Humanities |
+| `chim` | Chemistry |
+| `spi` | Engineering Sciences |
+
+## References
+
+- [HAL Open Archive](https://hal.science/)
+- [HAL API Documentation](https://api.archives-ouvertes.fr/docs)
+- [HAL Search Guide](https://doc.archives-ouvertes.fr/en/)

package/skills/literature/fulltext/osf-api/SKILL.md

@@ -0,0 +1,212 @@
+---
+name: osf-api
+description: "Manage open science projects and preprints via the OSF REST API"
+metadata:
+  openclaw:
+    emoji: "🔓"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["OSF", "Open Science Framework", "preprints", "open data", "reproducibility", "COS"]
+    source: "https://osf.io/"
+---
+
+# OSF (Open Science Framework) API
+
+## Overview
+
+The Open Science Framework by the Center for Open Science provides infrastructure for the entire research lifecycle — project management, file storage, preprint hosting, and registrations. The API enables search, project creation, file management, and preprint discovery across OSF Preprints, PsyArXiv, SocArXiv, and 25+ community preprint servers. Free, no auth for read access.
+
+## API Endpoints
+
+### Base URL
+
+```
+https://api.osf.io/v2
+```
+
+### Search
+
+```bash
+# Search across all OSF content
+curl "https://api.osf.io/v2/search/?q=replication+crisis&page[size]=20"
+
+# Search preprints
+curl "https://api.osf.io/v2/preprints/?filter[q]=machine+learning&page[size]=20"
+
+# Filter by preprint provider
+curl "https://api.osf.io/v2/preprints/?filter[provider]=psyarxiv&filter[q]=cognitive+bias"
+
+# Search registrations (pre-registered studies)
+curl "https://api.osf.io/v2/registrations/?filter[q]=randomized+controlled+trial"
+```
+
+### Projects
+
+```bash
+# Get public projects
+curl "https://api.osf.io/v2/nodes/?filter[public]=true&filter[q]=neuroimaging"
+
+# Get project details
+curl "https://api.osf.io/v2/nodes/{node_id}/"
+
+# Get project files
+curl "https://api.osf.io/v2/nodes/{node_id}/files/"
+
+# Get project contributors
+curl "https://api.osf.io/v2/nodes/{node_id}/contributors/"
+```
+
+### Preprint Providers
+
+| Provider | Filter | Disciplines |
+|----------|--------|-------------|
+| OSF Preprints | `osf` | Multidisciplinary |
+| PsyArXiv | `psyarxiv` | Psychology |
+| SocArXiv | `socarxiv` | Social sciences |
+| EarthArXiv | `eartharxiv` | Earth sciences |
+| BioHackrXiv | `biohackrxiv` | Bioinformatics |
+| engrXiv | `engrxiv` | Engineering |
+| MedArXiv | `medarxiv` | Medical sciences |
+| NutriXiv | `nutrixiv` | Nutrition |
+
+### Query Parameters
+
+| Parameter | Description | Example |
+|-----------|-------------|---------|
+| `filter[q]` | Text search | `filter[q]=open+data` |
+| `filter[provider]` | Preprint server | `filter[provider]=psyarxiv` |
+| `filter[subjects]` | Subject filter | Subject taxonomy ID |
+| `filter[date_created]` | Date filter | `filter[date_created][gte]=2024-01-01` |
+| `page[size]` | Results per page (max 100) | `page[size]=50` |
+| `page` | Page number | `page=2` |
+
+## Response Structure (Preprint)
+
+```json
+{
+  "data": [
+    {
+      "id": "abc12",
+      "type": "preprints",
+      "attributes": {
+        "title": "Replication of the Ego Depletion Effect",
+        "description": "We attempted to replicate...",
+        "date_created": "2024-06-15T10:00:00Z",
+        "date_published": "2024-06-16T08:00:00Z",
+        "doi": "10.31234/osf.io/abc12",
+        "is_published": true,
+        "subjects": [["Social and Behavioral Sciences", "Psychology"]],
+        "tags": ["replication", "ego depletion"]
+      },
+      "relationships": {
+        "contributors": {"links": {"related": {"href": "..."}}},
+        "primary_file": {"links": {"related": {"href": "..."}}}
+      }
+    }
+  ]
+}
+```
+
+## Python Usage
+
+```python
+import requests
+
+BASE_URL = "https://api.osf.io/v2"
+
+
+def search_preprints(query: str, provider: str = None,
+                     page_size: int = 20) -> list:
+    """Search OSF preprints across providers."""
+    params = {
+        "filter[q]": query,
+        "page[size]": page_size,
+    }
+    if provider:
+        params["filter[provider]"] = provider
+
+    resp = requests.get(f"{BASE_URL}/preprints/", params=params)
+    resp.raise_for_status()
+    data = resp.json()
+
+    results = []
+    for item in data.get("data", []):
+        attrs = item.get("attributes", {})
+        results.append({
+            "id": item.get("id"),
+            "title": attrs.get("title"),
+            "description": (attrs.get("description") or "")[:300],
+            "doi": attrs.get("doi"),
+            "date": attrs.get("date_published", "")[:10],
+            "tags": attrs.get("tags", []),
+            "url": f"https://osf.io/{item.get('id')}/",
+        })
+    return results
+
+
+def search_registrations(query: str,
+                         page_size: int = 20) -> list:
+    """Search pre-registered studies on OSF."""
+    params = {
+        "filter[q]": query,
+        "page[size]": page_size,
+    }
+    resp = requests.get(f"{BASE_URL}/registrations/", params=params)
+    resp.raise_for_status()
+    data = resp.json()
+
+    results = []
+    for item in data.get("data", []):
+        attrs = item.get("attributes", {})
+        results.append({
+            "id": item.get("id"),
+            "title": attrs.get("title"),
+            "description": (attrs.get("description") or "")[:300],
+            "date_registered": attrs.get("date_registered", "")[:10],
+            "registration_schema": attrs.get("registration_supplement"),
+        })
+    return results
+
+
+def get_project_files(node_id: str) -> list:
+    """List files in an OSF project."""
+    resp = requests.get(f"{BASE_URL}/nodes/{node_id}/files/")
+    resp.raise_for_status()
+    data = resp.json()
+
+    providers = []
+    for item in data.get("data", []):
+        attrs = item.get("attributes", {})
+        providers.append({
+            "provider": attrs.get("provider"),
+            "name": attrs.get("name"),
+        })
+    return providers
+
+
+# Example: search psychology preprints
+preprints = search_preprints("cognitive load", provider="psyarxiv")
+for p in preprints[:5]:
+    print(f"[{p['date']}] {p['title']}")
+    print(f"  DOI: {p['doi']}")
+
+# Example: find pre-registered clinical trials
+regs = search_registrations("randomized placebo")
+for r in regs[:5]:
+    print(f"[{r['date_registered']}] {r['title']}")
+```
+
+## Use Cases
+
+1. **Preprint discovery**: Search across 25+ preprint servers
+2. **Pre-registration search**: Find registered study protocols
+3. **Open data access**: Download shared research datasets
+4. **Reproducibility**: Access materials, data, and code for published studies
+5. **Project management**: Programmatic project and file management
+
+## References
+
+- [OSF](https://osf.io/)
+- [OSF API Documentation](https://developer.osf.io/)
+- [OSF Preprints](https://osf.io/preprints/)
+- [Center for Open Science](https://cos.io/)