@wentorai/research-plugins 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +204 -0
- package/curated/analysis/README.md +64 -0
- package/curated/domains/README.md +104 -0
- package/curated/literature/README.md +53 -0
- package/curated/research/README.md +62 -0
- package/curated/tools/README.md +87 -0
- package/curated/writing/README.md +61 -0
- package/index.ts +39 -0
- package/mcp-configs/academic-db/ChatSpatial.json +17 -0
- package/mcp-configs/academic-db/academia-mcp.json +17 -0
- package/mcp-configs/academic-db/academic-paper-explorer.json +17 -0
- package/mcp-configs/academic-db/academic-search-mcp-server.json +17 -0
- package/mcp-configs/academic-db/agentinterviews-mcp.json +17 -0
- package/mcp-configs/academic-db/all-in-mcp.json +17 -0
- package/mcp-configs/academic-db/apple-health-mcp.json +17 -0
- package/mcp-configs/academic-db/arxiv-latex-mcp.json +17 -0
- package/mcp-configs/academic-db/arxiv-mcp-server.json +17 -0
- package/mcp-configs/academic-db/bgpt-mcp.json +17 -0
- package/mcp-configs/academic-db/biomcp.json +17 -0
- package/mcp-configs/academic-db/biothings-mcp.json +17 -0
- package/mcp-configs/academic-db/catalysishub-mcp-server.json +17 -0
- package/mcp-configs/academic-db/clinicaltrialsgov-mcp-server.json +17 -0
- package/mcp-configs/academic-db/deep-research-mcp.json +17 -0
- package/mcp-configs/academic-db/dicom-mcp.json +17 -0
- package/mcp-configs/academic-db/enrichr-mcp-server.json +17 -0
- package/mcp-configs/academic-db/fec-mcp-server.json +17 -0
- package/mcp-configs/academic-db/fhir-mcp-server-themomentum.json +17 -0
- package/mcp-configs/academic-db/fhir-mcp.json +19 -0
- package/mcp-configs/academic-db/gget-mcp.json +17 -0
- package/mcp-configs/academic-db/google-researcher-mcp.json +17 -0
- package/mcp-configs/academic-db/idea-reality-mcp.json +17 -0
- package/mcp-configs/academic-db/legiscan-mcp.json +19 -0
- package/mcp-configs/academic-db/lex.json +17 -0
- package/mcp-configs/ai-platform/Adaptive-Graph-of-Thoughts-MCP-server.json +17 -0
- package/mcp-configs/ai-platform/ai-counsel.json +17 -0
- package/mcp-configs/ai-platform/atlas-mcp-server.json +17 -0
- package/mcp-configs/ai-platform/counsel-mcp.json +17 -0
- package/mcp-configs/ai-platform/cross-llm-mcp.json +17 -0
- package/mcp-configs/ai-platform/gptr-mcp.json +17 -0
- package/mcp-configs/browser/decipher-research-agent.json +17 -0
- package/mcp-configs/browser/deep-research.json +17 -0
- package/mcp-configs/browser/everything-claude-code.json +17 -0
- package/mcp-configs/browser/gpt-researcher.json +17 -0
- package/mcp-configs/browser/heurist-agent-framework.json +17 -0
- package/mcp-configs/data-platform/4everland-hosting-mcp.json +17 -0
- package/mcp-configs/data-platform/context-keeper.json +17 -0
- package/mcp-configs/data-platform/context7.json +19 -0
- package/mcp-configs/data-platform/contextstream-mcp.json +17 -0
- package/mcp-configs/data-platform/email-mcp.json +17 -0
- package/mcp-configs/note-knowledge/ApeRAG.json +17 -0
- package/mcp-configs/note-knowledge/In-Memoria.json +17 -0
- package/mcp-configs/note-knowledge/agent-memory.json +17 -0
- package/mcp-configs/note-knowledge/aimemo.json +17 -0
- package/mcp-configs/note-knowledge/biel-mcp.json +19 -0
- package/mcp-configs/note-knowledge/cognee.json +17 -0
- package/mcp-configs/note-knowledge/context-awesome.json +17 -0
- package/mcp-configs/note-knowledge/context-mcp.json +17 -0
- package/mcp-configs/note-knowledge/conversation-handoff-mcp.json +17 -0
- package/mcp-configs/note-knowledge/cortex.json +17 -0
- package/mcp-configs/note-knowledge/devrag.json +17 -0
- package/mcp-configs/note-knowledge/easy-obsidian-mcp.json +17 -0
- package/mcp-configs/note-knowledge/engram.json +17 -0
- package/mcp-configs/note-knowledge/gnosis-mcp.json +17 -0
- package/mcp-configs/note-knowledge/graphlit-mcp-server.json +19 -0
- package/mcp-configs/reference-mgr/arxiv-cli.json +17 -0
- package/mcp-configs/reference-mgr/arxiv-search-mcp.json +17 -0
- package/mcp-configs/reference-mgr/chiken.json +17 -0
- package/mcp-configs/reference-mgr/claude-scholar.json +17 -0
- package/mcp-configs/reference-mgr/devonthink-mcp.json +17 -0
- package/mcp-configs/registry.json +447 -0
- package/openclaw.plugin.json +21 -0
- package/package.json +61 -0
- package/skills/analysis/dataviz/color-accessibility-guide/SKILL.md +230 -0
- package/skills/analysis/dataviz/geospatial-viz-guide/SKILL.md +218 -0
- package/skills/analysis/dataviz/interactive-viz-guide/SKILL.md +287 -0
- package/skills/analysis/dataviz/network-visualization-guide/SKILL.md +195 -0
- package/skills/analysis/dataviz/publication-figures-guide/SKILL.md +238 -0
- package/skills/analysis/dataviz/python-dataviz-guide/SKILL.md +195 -0
- package/skills/analysis/econometrics/causal-inference-guide/SKILL.md +197 -0
- package/skills/analysis/econometrics/iv-regression-guide/SKILL.md +198 -0
- package/skills/analysis/econometrics/panel-data-guide/SKILL.md +274 -0
- package/skills/analysis/econometrics/robustness-checks/SKILL.md +250 -0
- package/skills/analysis/econometrics/stata-regression/SKILL.md +117 -0
- package/skills/analysis/econometrics/time-series-guide/SKILL.md +235 -0
- package/skills/analysis/statistics/bayesian-statistics-guide/SKILL.md +221 -0
- package/skills/analysis/statistics/hypothesis-testing-guide/SKILL.md +210 -0
- package/skills/analysis/statistics/meta-analysis-guide/SKILL.md +206 -0
- package/skills/analysis/statistics/nonparametric-tests-guide/SKILL.md +221 -0
- package/skills/analysis/statistics/power-analysis-guide/SKILL.md +240 -0
- package/skills/analysis/statistics/sem-guide/SKILL.md +231 -0
- package/skills/analysis/statistics/survival-analysis-guide/SKILL.md +195 -0
- package/skills/analysis/wrangling/missing-data-handling/SKILL.md +224 -0
- package/skills/analysis/wrangling/pandas-data-wrangling/SKILL.md +242 -0
- package/skills/analysis/wrangling/questionnaire-design-guide/SKILL.md +234 -0
- package/skills/analysis/wrangling/text-mining-guide/SKILL.md +225 -0
- package/skills/domains/ai-ml/computer-vision-guide/SKILL.md +213 -0
- package/skills/domains/ai-ml/deep-learning-papers-guide/SKILL.md +200 -0
- package/skills/domains/ai-ml/llm-evaluation-guide/SKILL.md +194 -0
- package/skills/domains/ai-ml/prompt-engineering-research/SKILL.md +233 -0
- package/skills/domains/ai-ml/reinforcement-learning-guide/SKILL.md +254 -0
- package/skills/domains/ai-ml/transformer-architecture-guide/SKILL.md +233 -0
- package/skills/domains/biomedical/clinical-research-guide/SKILL.md +232 -0
- package/skills/domains/biomedical/clinicaltrials-api/SKILL.md +177 -0
- package/skills/domains/biomedical/epidemiology-guide/SKILL.md +200 -0
- package/skills/domains/biomedical/genomics-analysis-guide/SKILL.md +270 -0
- package/skills/domains/business/market-analysis-guide/SKILL.md +112 -0
- package/skills/domains/business/strategic-management-guide/SKILL.md +154 -0
- package/skills/domains/chemistry/computational-chemistry-guide/SKILL.md +266 -0
- package/skills/domains/chemistry/retrosynthesis-guide/SKILL.md +215 -0
- package/skills/domains/cs/algorithms-complexity-guide/SKILL.md +194 -0
- package/skills/domains/cs/dblp-api/SKILL.md +129 -0
- package/skills/domains/cs/software-engineering-research/SKILL.md +218 -0
- package/skills/domains/ecology/biodiversity-data-guide/SKILL.md +296 -0
- package/skills/domains/ecology/conservation-biology-guide/SKILL.md +198 -0
- package/skills/domains/ecology/gbif-api/SKILL.md +158 -0
- package/skills/domains/ecology/inaturalist-api/SKILL.md +173 -0
- package/skills/domains/economics/behavioral-economics-guide/SKILL.md +239 -0
- package/skills/domains/economics/development-economics-guide/SKILL.md +181 -0
- package/skills/domains/economics/fred-api/SKILL.md +189 -0
- package/skills/domains/education/curriculum-design-guide/SKILL.md +144 -0
- package/skills/domains/education/learning-science-guide/SKILL.md +150 -0
- package/skills/domains/finance/financial-data-analysis/SKILL.md +152 -0
- package/skills/domains/finance/quantitative-finance-guide/SKILL.md +151 -0
- package/skills/domains/geoscience/climate-science-guide/SKILL.md +158 -0
- package/skills/domains/geoscience/gis-remote-sensing-guide/SKILL.md +129 -0
- package/skills/domains/humanities/digital-humanities-guide/SKILL.md +181 -0
- package/skills/domains/humanities/philosophy-research-guide/SKILL.md +148 -0
- package/skills/domains/law/courtlistener-api/SKILL.md +213 -0
- package/skills/domains/law/legal-research-guide/SKILL.md +250 -0
- package/skills/domains/math/linear-algebra-applications/SKILL.md +227 -0
- package/skills/domains/math/numerical-methods-guide/SKILL.md +236 -0
- package/skills/domains/math/oeis-api/SKILL.md +158 -0
- package/skills/domains/pharma/clinical-pharmacology-guide/SKILL.md +165 -0
- package/skills/domains/pharma/drug-development-guide/SKILL.md +177 -0
- package/skills/domains/physics/computational-physics-guide/SKILL.md +300 -0
- package/skills/domains/physics/nasa-ads-api/SKILL.md +150 -0
- package/skills/domains/physics/quantum-computing-guide/SKILL.md +234 -0
- package/skills/domains/social-science/social-research-methods/SKILL.md +194 -0
- package/skills/domains/social-science/survey-research-guide/SKILL.md +182 -0
- package/skills/literature/discovery/citation-alert-guide/SKILL.md +154 -0
- package/skills/literature/discovery/conference-proceedings-guide/SKILL.md +142 -0
- package/skills/literature/discovery/literature-mapping-guide/SKILL.md +175 -0
- package/skills/literature/discovery/paper-tracking-guide/SKILL.md +211 -0
- package/skills/literature/discovery/rss-paper-feeds/SKILL.md +214 -0
- package/skills/literature/discovery/semantic-scholar-recs-guide/SKILL.md +164 -0
- package/skills/literature/fulltext/doaj-api/SKILL.md +120 -0
- package/skills/literature/fulltext/interlibrary-loan-guide/SKILL.md +163 -0
- package/skills/literature/fulltext/open-access-guide/SKILL.md +183 -0
- package/skills/literature/fulltext/pmc-oai-api/SKILL.md +184 -0
- package/skills/literature/fulltext/preprint-servers-guide/SKILL.md +128 -0
- package/skills/literature/fulltext/repository-harvesting-guide/SKILL.md +207 -0
- package/skills/literature/fulltext/unpaywall-api/SKILL.md +113 -0
- package/skills/literature/metadata/altmetrics-guide/SKILL.md +132 -0
- package/skills/literature/metadata/citation-network-guide/SKILL.md +236 -0
- package/skills/literature/metadata/crossref-api/SKILL.md +133 -0
- package/skills/literature/metadata/datacite-api/SKILL.md +126 -0
- package/skills/literature/metadata/doi-resolution-guide/SKILL.md +168 -0
- package/skills/literature/metadata/h-index-guide/SKILL.md +183 -0
- package/skills/literature/metadata/journal-metrics-guide/SKILL.md +188 -0
- package/skills/literature/metadata/opencitations-api/SKILL.md +128 -0
- package/skills/literature/metadata/orcid-api/SKILL.md +136 -0
- package/skills/literature/metadata/orcid-integration-guide/SKILL.md +178 -0
- package/skills/literature/search/arxiv-api/SKILL.md +95 -0
- package/skills/literature/search/biorxiv-api/SKILL.md +123 -0
- package/skills/literature/search/boolean-search-guide/SKILL.md +199 -0
- package/skills/literature/search/citation-chaining-guide/SKILL.md +148 -0
- package/skills/literature/search/database-comparison-guide/SKILL.md +100 -0
- package/skills/literature/search/europe-pmc-api/SKILL.md +120 -0
- package/skills/literature/search/google-scholar-guide/SKILL.md +182 -0
- package/skills/literature/search/mesh-terms-guide/SKILL.md +164 -0
- package/skills/literature/search/openalex-api/SKILL.md +134 -0
- package/skills/literature/search/pubmed-api/SKILL.md +130 -0
- package/skills/literature/search/scientify-literature-survey/SKILL.md +203 -0
- package/skills/literature/search/semantic-scholar-api/SKILL.md +134 -0
- package/skills/literature/search/systematic-search-strategy/SKILL.md +214 -0
- package/skills/research/automation/ai-scientist-guide/SKILL.md +228 -0
- package/skills/research/automation/data-collection-automation/SKILL.md +248 -0
- package/skills/research/automation/research-workflow-automation/SKILL.md +266 -0
- package/skills/research/deep-research/meta-synthesis-guide/SKILL.md +174 -0
- package/skills/research/deep-research/research-cog/SKILL.md +153 -0
- package/skills/research/deep-research/scoping-review-guide/SKILL.md +217 -0
- package/skills/research/deep-research/systematic-review-guide/SKILL.md +250 -0
- package/skills/research/funding/figshare-api/SKILL.md +163 -0
- package/skills/research/funding/grant-writing-guide/SKILL.md +233 -0
- package/skills/research/funding/nsf-grant-guide/SKILL.md +206 -0
- package/skills/research/funding/open-science-guide/SKILL.md +255 -0
- package/skills/research/funding/zenodo-api/SKILL.md +174 -0
- package/skills/research/methodology/action-research-guide/SKILL.md +201 -0
- package/skills/research/methodology/experimental-design-guide/SKILL.md +236 -0
- package/skills/research/methodology/grad-school-guide/SKILL.md +182 -0
- package/skills/research/methodology/grounded-theory-guide/SKILL.md +171 -0
- package/skills/research/methodology/mixed-methods-guide/SKILL.md +208 -0
- package/skills/research/methodology/qualitative-research-guide/SKILL.md +234 -0
- package/skills/research/methodology/scientify-idea-generation/SKILL.md +222 -0
- package/skills/research/paper-review/paper-reading-assistant/SKILL.md +266 -0
- package/skills/research/paper-review/peer-review-guide/SKILL.md +227 -0
- package/skills/research/paper-review/rebuttal-writing-guide/SKILL.md +185 -0
- package/skills/research/paper-review/scientify-write-review-paper/SKILL.md +209 -0
- package/skills/tools/code-exec/jupyter-notebook-guide/SKILL.md +178 -0
- package/skills/tools/code-exec/python-reproducibility-guide/SKILL.md +341 -0
- package/skills/tools/code-exec/r-reproducibility-guide/SKILL.md +236 -0
- package/skills/tools/code-exec/sandbox-execution-guide/SKILL.md +221 -0
- package/skills/tools/diagram/mermaid-diagram-guide/SKILL.md +269 -0
- package/skills/tools/diagram/plantuml-guide/SKILL.md +397 -0
- package/skills/tools/diagram/scientific-illustration-guide/SKILL.md +225 -0
- package/skills/tools/document/anystyle-api/SKILL.md +199 -0
- package/skills/tools/document/grobid-pdf-parsing/SKILL.md +294 -0
- package/skills/tools/document/markdown-academic-guide/SKILL.md +217 -0
- package/skills/tools/document/pdf-extraction-guide/SKILL.md +321 -0
- package/skills/tools/knowledge-graph/knowledge-graph-construction/SKILL.md +306 -0
- package/skills/tools/knowledge-graph/ontology-design-guide/SKILL.md +214 -0
- package/skills/tools/knowledge-graph/rag-methodology-guide/SKILL.md +325 -0
- package/skills/tools/ocr-translate/formula-recognition-guide/SKILL.md +367 -0
- package/skills/tools/ocr-translate/handwriting-recognition-guide/SKILL.md +211 -0
- package/skills/tools/ocr-translate/latex-ocr-guide/SKILL.md +204 -0
- package/skills/tools/ocr-translate/multilingual-research-guide/SKILL.md +234 -0
- package/skills/tools/scraping/academic-web-scraping/SKILL.md +326 -0
- package/skills/tools/scraping/api-data-collection-guide/SKILL.md +301 -0
- package/skills/tools/scraping/web-scraping-ethics-guide/SKILL.md +250 -0
- package/skills/writing/citation/bibtex-management-guide/SKILL.md +246 -0
- package/skills/writing/citation/citation-style-guide/SKILL.md +248 -0
- package/skills/writing/citation/reference-manager-comparison/SKILL.md +208 -0
- package/skills/writing/citation/zotero-api/SKILL.md +188 -0
- package/skills/writing/composition/abstract-writing-guide/SKILL.md +188 -0
- package/skills/writing/composition/discussion-writing-guide/SKILL.md +194 -0
- package/skills/writing/composition/introduction-writing-guide/SKILL.md +194 -0
- package/skills/writing/composition/literature-review-writing/SKILL.md +196 -0
- package/skills/writing/composition/methods-section-guide/SKILL.md +185 -0
- package/skills/writing/composition/response-to-reviewers/SKILL.md +215 -0
- package/skills/writing/composition/scientific-writing-guide/SKILL.md +152 -0
- package/skills/writing/latex/bibliography-management-guide/SKILL.md +206 -0
- package/skills/writing/latex/latex-drawing-guide/SKILL.md +234 -0
- package/skills/writing/latex/latex-ecosystem-guide/SKILL.md +240 -0
- package/skills/writing/latex/math-typesetting-guide/SKILL.md +231 -0
- package/skills/writing/latex/overleaf-collaboration-guide/SKILL.md +211 -0
- package/skills/writing/latex/tikz-diagrams-guide/SKILL.md +211 -0
- package/skills/writing/polish/academic-translation-guide/SKILL.md +175 -0
- package/skills/writing/polish/academic-writing-refiner/SKILL.md +143 -0
- package/skills/writing/polish/ai-writing-humanizer/SKILL.md +178 -0
- package/skills/writing/polish/grammar-checker-guide/SKILL.md +184 -0
- package/skills/writing/polish/plagiarism-detection-guide/SKILL.md +167 -0
- package/skills/writing/templates/beamer-presentation-guide/SKILL.md +263 -0
- package/skills/writing/templates/conference-paper-template/SKILL.md +219 -0
- package/skills/writing/templates/thesis-template-guide/SKILL.md +200 -0
- package/skills/writing/templates/thesis-writing-guide/SKILL.md +220 -0
- package/src/tools/arxiv.ts +131 -0
- package/src/tools/crossref.ts +112 -0
- package/src/tools/openalex.ts +174 -0
- package/src/tools/pubmed.ts +166 -0
- package/src/tools/semantic-scholar.ts +108 -0
- package/src/tools/unpaywall.ts +58 -0
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: mesh-terms-guide
|
|
3
|
+
description: "Navigate MeSH vocabulary for precise PubMed and MEDLINE searches"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "dna"
|
|
7
|
+
category: "literature"
|
|
8
|
+
subcategory: "search"
|
|
9
|
+
keywords: ["MeSH terms", "PubMed", "MEDLINE", "controlled vocabulary", "medical subject headings", "literature search"]
|
|
10
|
+
source: "wentor-research-plugins"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# MeSH Terms Guide
|
|
14
|
+
|
|
15
|
+
A skill for using Medical Subject Headings (MeSH) to construct precise and comprehensive biomedical literature searches. Covers MeSH tree structure, subheadings, explosion, major topics, and strategies for combining MeSH with free-text terms.
|
|
16
|
+
|
|
17
|
+
## Understanding MeSH Vocabulary
|
|
18
|
+
|
|
19
|
+
### What Is MeSH?
|
|
20
|
+
|
|
21
|
+
MeSH is a hierarchical controlled vocabulary maintained by the National Library of Medicine. Every article indexed in MEDLINE/PubMed is assigned MeSH terms by trained indexers. Using MeSH ensures you capture articles that discuss a concept even when authors use different terminology.
|
|
22
|
+
|
|
23
|
+
### Tree Structure and Hierarchy
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
MeSH terms are organized in a tree with up to 13 hierarchical levels.
|
|
27
|
+
|
|
28
|
+
Example tree path for "Breast Neoplasms":
|
|
29
|
+
|
|
30
|
+
Neoplasms [C04]
|
|
31
|
+
Neoplasms by Site [C04.588]
|
|
32
|
+
Breast Neoplasms [C04.588.180]
|
|
33
|
+
Breast Neoplasms, Male [C04.588.180.260]
|
|
34
|
+
Inflammatory Breast Neoplasms [C04.588.180.390]
|
|
35
|
+
Phyllodes Tumor [C04.588.180.610]
|
|
36
|
+
|
|
37
|
+
A term can appear in multiple tree branches.
|
|
38
|
+
"Diabetes Mellitus, Type 2" appears under both:
|
|
39
|
+
- Endocrine System Diseases
|
|
40
|
+
- Metabolic Diseases
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### Explosion vs. Non-Explosion
|
|
44
|
+
|
|
45
|
+
```
|
|
46
|
+
# Exploded search (default in PubMed):
|
|
47
|
+
"Breast Neoplasms"[MeSH]
|
|
48
|
+
-> Retrieves articles indexed with "Breast Neoplasms"
|
|
49
|
+
AND all narrower terms underneath it
|
|
50
|
+
|
|
51
|
+
# Non-exploded search:
|
|
52
|
+
"Breast Neoplasms"[MeSH:NoExp]
|
|
53
|
+
-> Retrieves ONLY articles indexed with "Breast Neoplasms" itself
|
|
54
|
+
Does NOT include narrower terms like Inflammatory Breast Neoplasms
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Working with MeSH in PubMed
|
|
58
|
+
|
|
59
|
+
### Finding the Right MeSH Term
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
Step 1: Go to the MeSH Browser (meshb.nlm.nih.gov)
|
|
63
|
+
Step 2: Enter your concept (e.g., "heart attack")
|
|
64
|
+
Step 3: Review the mapped term: "Myocardial Infarction"
|
|
65
|
+
Step 4: Check Entry Terms (synonyms that map to this heading)
|
|
66
|
+
Step 5: Examine the tree position and scope note
|
|
67
|
+
Step 6: Decide whether to explode or restrict
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Subheadings (Qualifiers)
|
|
71
|
+
|
|
72
|
+
MeSH subheadings refine a concept by specifying an aspect:
|
|
73
|
+
|
|
74
|
+
```python
|
|
75
|
+
def build_mesh_query(mesh_term: str, subheadings: list[str] = None,
|
|
76
|
+
major_topic: bool = False,
|
|
77
|
+
explode: bool = True) -> str:
|
|
78
|
+
"""
|
|
79
|
+
Build a PubMed MeSH search fragment.
|
|
80
|
+
|
|
81
|
+
Args:
|
|
82
|
+
mesh_term: The MeSH heading
|
|
83
|
+
subheadings: List of MeSH subheadings (e.g., 'therapy', 'diagnosis')
|
|
84
|
+
major_topic: If True, restrict to articles where this is a major topic
|
|
85
|
+
explode: If False, do not include narrower terms
|
|
86
|
+
"""
|
|
87
|
+
field = "MeSH"
|
|
88
|
+
if major_topic:
|
|
89
|
+
field = "Majr"
|
|
90
|
+
|
|
91
|
+
if not explode:
|
|
92
|
+
field += ":NoExp"
|
|
93
|
+
|
|
94
|
+
if subheadings:
|
|
95
|
+
sub_parts = " OR ".join(
|
|
96
|
+
f'"{mesh_term}/{sh}"[{field}]' for sh in subheadings
|
|
97
|
+
)
|
|
98
|
+
return f"({sub_parts})"
|
|
99
|
+
|
|
100
|
+
return f'"{mesh_term}"[{field}]'
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
# Example: Diabetes treatment as a major topic
|
|
104
|
+
print(build_mesh_query(
|
|
105
|
+
"Diabetes Mellitus, Type 2",
|
|
106
|
+
subheadings=["drug therapy", "therapy"],
|
|
107
|
+
major_topic=True
|
|
108
|
+
))
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### Common Subheadings by Research Purpose
|
|
112
|
+
|
|
113
|
+
| Purpose | Subheadings |
|
|
114
|
+
|---------|-------------|
|
|
115
|
+
| Treatment | /therapy, /drug therapy, /surgery, /rehabilitation |
|
|
116
|
+
| Etiology | /etiology, /complications, /chemically induced |
|
|
117
|
+
| Diagnosis | /diagnosis, /diagnostic imaging, /pathology |
|
|
118
|
+
| Prevention | /prevention and control |
|
|
119
|
+
| Epidemiology | /epidemiology, /mortality, /statistics and numerical data |
|
|
120
|
+
| Genetics | /genetics, /metabolism |
|
|
121
|
+
|
|
122
|
+
## Combining MeSH with Free Text
|
|
123
|
+
|
|
124
|
+
### Comprehensive Search Strategy
|
|
125
|
+
|
|
126
|
+
A robust search combines both MeSH terms and free-text keywords to capture all relevant articles, including those not yet indexed:
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
# Block 1 - MeSH terms for Concept A
|
|
130
|
+
("Myocardial Infarction"[MeSH] OR "Acute Coronary Syndrome"[MeSH])
|
|
131
|
+
|
|
132
|
+
# Block 2 - Free-text terms for Concept A
|
|
133
|
+
("myocardial infarction"[tiab] OR "heart attack"[tiab] OR
|
|
134
|
+
"acute coronary"[tiab] OR "STEMI"[tiab] OR "NSTEMI"[tiab])
|
|
135
|
+
|
|
136
|
+
# Block 3 - Combine with OR
|
|
137
|
+
(Block 1 OR Block 2)
|
|
138
|
+
|
|
139
|
+
# Repeat for Concept B, then AND the concept blocks together
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Handling Unindexed Articles
|
|
143
|
+
|
|
144
|
+
PubMed articles are often available before MeSH indexing is complete. These articles will not be found by MeSH-only searches. Always include free-text (Title/Abstract) terms alongside MeSH to capture recent publications:
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
"Diabetes Mellitus, Type 2"[MeSH]
|
|
148
|
+
OR
|
|
149
|
+
("type 2 diabetes"[tiab] OR "T2DM"[tiab] OR "non-insulin dependent"[tiab])
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
## Validating Your MeSH Strategy
|
|
153
|
+
|
|
154
|
+
### Recall and Precision Check
|
|
155
|
+
|
|
156
|
+
1. Identify 5-10 known relevant articles (your "gold standard" set)
|
|
157
|
+
2. Run your MeSH search and verify all gold-standard articles appear
|
|
158
|
+
3. If any are missing, check their MeSH indexing to find gaps in your strategy
|
|
159
|
+
4. Scan the first 100 results to estimate precision
|
|
160
|
+
5. Adjust subheadings and explosion settings accordingly
|
|
161
|
+
|
|
162
|
+
### Documenting MeSH Searches for Systematic Reviews
|
|
163
|
+
|
|
164
|
+
Record the MeSH database version date, all terms used with their tree numbers, explosion status, and subheadings. This allows the search to be replicated exactly at a later date.
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: openalex-api
|
|
3
|
+
description: "Query the OpenAlex catalog of scholarly works, authors, and institutions"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "🔍"
|
|
7
|
+
category: "literature"
|
|
8
|
+
subcategory: "search"
|
|
9
|
+
keywords: ["academic database search", "literature search", "citation statistics", "academic metrics", "bibliometrics"]
|
|
10
|
+
source: "https://docs.openalex.org/"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# OpenAlex API Guide
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
OpenAlex is a free, open catalog of the global research system. It indexes over 250 million scholarly works, 90 million authors, 100,000 institutions, and 65,000 concepts. Created by the nonprofit OurResearch as a replacement for Microsoft Academic Graph, OpenAlex provides comprehensive bibliometric data for academic research analysis.
|
|
18
|
+
|
|
19
|
+
The API serves as a powerful tool for researchers conducting systematic literature reviews, bibliometric analyses, and research landscape mapping. It covers works across all academic disciplines, linking papers to their authors, institutions, concepts, and citation networks. Each entity in OpenAlex has a persistent identifier (OpenAlex ID) and is enriched with metadata from CrossRef, ORCID, ROR, and other authoritative sources.
|
|
20
|
+
|
|
21
|
+
OpenAlex is entirely free to use. No API key is required, though providing a contact email in the `mailto` parameter grants access to the polite pool with faster response times and higher rate limits.
|
|
22
|
+
|
|
23
|
+
## Authentication
|
|
24
|
+
|
|
25
|
+
No authentication required. For better rate limits and access to the polite pool, include your email in requests:
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
https://api.openalex.org/works?mailto=your@email.com
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The polite pool provides significantly faster responses and is recommended for all production usage.
|
|
32
|
+
|
|
33
|
+
## Core Endpoints
|
|
34
|
+
|
|
35
|
+
### Works: Search Scholarly Publications
|
|
36
|
+
|
|
37
|
+
- **URL**: `GET https://api.openalex.org/works`
|
|
38
|
+
- **Parameters**:
|
|
39
|
+
| Param | Type | Required | Description |
|
|
40
|
+
|-------|------|----------|-------------|
|
|
41
|
+
| search | string | No | Full-text search across titles and abstracts |
|
|
42
|
+
| filter | string | No | Structured filters (e.g., publication_year:2024, cited_by_count:>100) |
|
|
43
|
+
| sort | string | No | Sort field: cited_by_count, publication_date, relevance_score |
|
|
44
|
+
| page | integer | No | Page number for pagination (default: 1) |
|
|
45
|
+
| per_page | integer | No | Results per page (default: 25, max: 200) |
|
|
46
|
+
| mailto | string | No | Email for polite pool access |
|
|
47
|
+
- **Example**:
|
|
48
|
+
```bash
|
|
49
|
+
curl "https://api.openalex.org/works?search=large+language+models&filter=publication_year:2024&sort=cited_by_count:desc&per_page=10&mailto=user@example.com"
|
|
50
|
+
```
|
|
51
|
+
- **Response**: JSON with `results` array containing `id`, `title`, `doi`, `publication_date`, `cited_by_count`, `authorships`, `concepts`, `open_access` status, and `abstract_inverted_index`.
|
|
52
|
+
|
|
53
|
+
### Authors: Search Researcher Profiles
|
|
54
|
+
|
|
55
|
+
- **URL**: `GET https://api.openalex.org/authors`
|
|
56
|
+
- **Parameters**:
|
|
57
|
+
| Param | Type | Required | Description |
|
|
58
|
+
|-------|------|----------|-------------|
|
|
59
|
+
| search | string | No | Name-based author search |
|
|
60
|
+
| filter | string | No | Filters (e.g., works_count:>50, last_known_institution.id) |
|
|
61
|
+
| sort | string | No | Sort field: works_count, cited_by_count, h_index |
|
|
62
|
+
| per_page | integer | No | Results per page (default: 25, max: 200) |
|
|
63
|
+
- **Example**:
|
|
64
|
+
```bash
|
|
65
|
+
curl "https://api.openalex.org/authors?search=Geoffrey+Hinton&mailto=user@example.com"
|
|
66
|
+
```
|
|
67
|
+
- **Response**: JSON with `results` array containing `id`, `display_name`, `orcid`, `works_count`, `cited_by_count`, `h_index`, `last_known_institution`, and `x_concepts`.
|
|
68
|
+
|
|
69
|
+
### Institutions: Search Academic Organizations
|
|
70
|
+
|
|
71
|
+
- **URL**: `GET https://api.openalex.org/institutions`
|
|
72
|
+
- **Parameters**:
|
|
73
|
+
| Param | Type | Required | Description |
|
|
74
|
+
|-------|------|----------|-------------|
|
|
75
|
+
| search | string | No | Institution name search |
|
|
76
|
+
| filter | string | No | Filters (e.g., country_code:US, type:education) |
|
|
77
|
+
| sort | string | No | Sort field: works_count, cited_by_count |
|
|
78
|
+
| per_page | integer | No | Results per page |
|
|
79
|
+
- **Example**:
|
|
80
|
+
```bash
|
|
81
|
+
curl "https://api.openalex.org/institutions?search=MIT&mailto=user@example.com"
|
|
82
|
+
```
|
|
83
|
+
- **Response**: JSON with institution details including `id`, `display_name`, `ror`, `country_code`, `type`, `works_count`, and `cited_by_count`.
|
|
84
|
+
|
|
85
|
+
### Concepts: Browse Research Topics
|
|
86
|
+
|
|
87
|
+
- **URL**: `GET https://api.openalex.org/concepts`
|
|
88
|
+
- **Parameters**:
|
|
89
|
+
| Param | Type | Required | Description |
|
|
90
|
+
|-------|------|----------|-------------|
|
|
91
|
+
| search | string | No | Concept name search |
|
|
92
|
+
| filter | string | No | Filters (e.g., level:0 for top-level concepts) |
|
|
93
|
+
| per_page | integer | No | Results per page |
|
|
94
|
+
- **Example**:
|
|
95
|
+
```bash
|
|
96
|
+
curl "https://api.openalex.org/concepts?filter=level:0&per_page=50&mailto=user@example.com"
|
|
97
|
+
```
|
|
98
|
+
- **Response**: JSON with concept hierarchy, `works_count`, and related concepts.
|
|
99
|
+
|
|
100
|
+
## Rate Limits
|
|
101
|
+
|
|
102
|
+
Without the `mailto` parameter: 10 requests per second, 100,000 requests per day. With the `mailto` parameter (polite pool): significantly higher throughput. The API uses HTTP 429 responses when limits are exceeded. Implement exponential backoff for production usage.
|
|
103
|
+
|
|
104
|
+
## Common Patterns
|
|
105
|
+
|
|
106
|
+
### Citation Analysis for a Paper
|
|
107
|
+
|
|
108
|
+
Retrieve all works that cite a specific paper by its DOI:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
curl "https://api.openalex.org/works?filter=cites:W2741809807&sort=cited_by_count:desc&per_page=25&mailto=user@example.com"
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
### Institutional Research Output
|
|
115
|
+
|
|
116
|
+
Analyze publication trends for a specific institution:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
curl "https://api.openalex.org/works?filter=institutions.id:I136199984,publication_year:2024&group_by=open_access.is_oa&mailto=user@example.com"
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### Topic Landscape Mapping
|
|
123
|
+
|
|
124
|
+
Explore how a concept connects to others in the research landscape:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
curl "https://api.openalex.org/concepts/C41008148?mailto=user@example.com"
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
## References
|
|
131
|
+
|
|
132
|
+
- Official documentation: https://docs.openalex.org/
|
|
133
|
+
- OpenAlex data model: https://docs.openalex.org/about-the-data
|
|
134
|
+
- API filters reference: https://docs.openalex.org/how-to-use-the-api/get-lists-of-entities/filter-entity-lists
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: pubmed-api
|
|
3
|
+
description: "Search biomedical literature and retrieve records via PubMed E-utilities"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "🔍"
|
|
7
|
+
category: "literature"
|
|
8
|
+
subcategory: "search"
|
|
9
|
+
keywords: ["academic database search", "literature search", "scholarly database", "field-specific search"]
|
|
10
|
+
source: "https://www.ncbi.nlm.nih.gov/books/NBK25501/"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# PubMed E-utilities API Guide
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
PubMed is the premier biomedical literature database maintained by the National Center for Biotechnology Information (NCBI) at the US National Library of Medicine. It indexes over 36 million citations and abstracts from MEDLINE, life science journals, and online books. The Entrez Programming Utilities (E-utilities) provide programmatic access to the entire PubMed database and other NCBI databases.
|
|
18
|
+
|
|
19
|
+
E-utilities consist of a suite of server-side programs that accept URL-based requests and return structured data. These tools are essential for biomedical researchers, systematic reviewers, and developers building health informatics applications. The API supports complex search queries using MeSH (Medical Subject Headings) terms, boolean operators, and field-specific searches.
|
|
20
|
+
|
|
21
|
+
The API is free and does not require authentication for basic usage. Registering for an NCBI API key raises the rate limit from 3 to 10 requests per second, which is recommended for any automated workflow.
|
|
22
|
+
|
|
23
|
+
## Authentication
|
|
24
|
+
|
|
25
|
+
No authentication required for basic usage (3 requests/second). For higher rate limits (10 requests/second), register for a free API key at https://www.ncbi.nlm.nih.gov/account/ and include it in requests:
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
&api_key=YOUR_API_KEY
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Including `tool` and `email` parameters in requests helps NCBI contact you if there are issues with your application.
|
|
32
|
+
|
|
33
|
+
## Core Endpoints
|
|
34
|
+
|
|
35
|
+
### ESearch: Search and Retrieve PMIDs
|
|
36
|
+
|
|
37
|
+
- **URL**: `GET https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi`
|
|
38
|
+
- **Parameters**:
|
|
39
|
+
| Param | Type | Required | Description |
|
|
40
|
+
|-------|------|----------|-------------|
|
|
41
|
+
| db | string | Yes | Database name (e.g., pubmed, pmc) |
|
|
42
|
+
| term | string | Yes | Search query (supports boolean operators and field tags) |
|
|
43
|
+
| retmax | integer | No | Maximum number of IDs returned (default: 20, max: 10000) |
|
|
44
|
+
| retstart | integer | No | Index of first ID to retrieve (for pagination) |
|
|
45
|
+
| retmode | string | No | Response format: xml (default) or json |
|
|
46
|
+
| sort | string | No | Sort order: relevance, pub_date, author, journal |
|
|
47
|
+
| datetype | string | No | Date type for range filter: pdat, mdat, edat |
|
|
48
|
+
| mindate | string | No | Start date (YYYY/MM/DD) |
|
|
49
|
+
| maxdate | string | No | End date (YYYY/MM/DD) |
|
|
50
|
+
| usehistory | string | No | Set to "y" to store results on server for subsequent retrieval |
|
|
51
|
+
- **Example**:
|
|
52
|
+
```bash
|
|
53
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=CRISPR+AND+cancer[Title]&retmax=10&retmode=json&sort=pub_date"
|
|
54
|
+
```
|
|
55
|
+
- **Response**: JSON/XML with `esearchresult` containing `count` (total hits), `idlist` (array of PMIDs), and optionally `webenv` and `querykey` for history server.
|
|
56
|
+
|
|
57
|
+
### EFetch: Retrieve Full Records
|
|
58
|
+
|
|
59
|
+
- **URL**: `GET https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi`
|
|
60
|
+
- **Parameters**:
|
|
61
|
+
| Param | Type | Required | Description |
|
|
62
|
+
|-------|------|----------|-------------|
|
|
63
|
+
| db | string | Yes | Database name |
|
|
64
|
+
| id | string | Yes | Comma-separated list of PMIDs (or use WebEnv/query_key) |
|
|
65
|
+
| rettype | string | No | Return type: abstract, medline, full, xml |
|
|
66
|
+
| retmode | string | No | Format: xml, text |
|
|
67
|
+
| WebEnv | string | No | Web environment from ESearch with usehistory=y |
|
|
68
|
+
| query_key | string | No | Query key from ESearch |
|
|
69
|
+
- **Example**:
|
|
70
|
+
```bash
|
|
71
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=33116299,34735795&rettype=abstract&retmode=xml"
|
|
72
|
+
```
|
|
73
|
+
- **Response**: XML with complete PubMed records including `MedlineCitation` with `Article` (title, abstract, authors, journal), `MeSHHeadingList`, and `PubmedData` (DOI, publication status).
|
|
74
|
+
|
|
75
|
+
### ESummary: Retrieve Document Summaries
|
|
76
|
+
|
|
77
|
+
- **URL**: `GET https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi`
|
|
78
|
+
- **Parameters**:
|
|
79
|
+
| Param | Type | Required | Description |
|
|
80
|
+
|-------|------|----------|-------------|
|
|
81
|
+
| db | string | Yes | Database name |
|
|
82
|
+
| id | string | Yes | Comma-separated PMIDs |
|
|
83
|
+
| retmode | string | No | Response format: xml or json |
|
|
84
|
+
| version | string | No | Set to "2.0" for enhanced XML format |
|
|
85
|
+
- **Example**:
|
|
86
|
+
```bash
|
|
87
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=33116299&retmode=json&version=2.0"
|
|
88
|
+
```
|
|
89
|
+
- **Response**: JSON with document summary including `uid`, `title`, `authors`, `source` (journal), `pubdate`, `doi`, and `pmcid`.
|
|
90
|
+
|
|
91
|
+
## Rate Limits
|
|
92
|
+
|
|
93
|
+
Without API key: 3 requests per second. With API key: 10 requests per second. Exceeding limits results in temporary IP blocking. For large-scale data mining, use the NCBI FTP site for bulk downloads. Always include a delay of at least 334ms (or 100ms with API key) between requests. Weekend and evening hours (US Eastern time) are less congested.
|
|
94
|
+
|
|
95
|
+
## Common Patterns
|
|
96
|
+
|
|
97
|
+
### Systematic Literature Search
|
|
98
|
+
|
|
99
|
+
Perform a structured search using MeSH terms and field qualifiers:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
# Search for clinical trials on diabetes treatment from the last 2 years
|
|
103
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=diabetes[MeSH]+AND+treatment[Title]+AND+clinical+trial[Publication+Type]&mindate=2024/01/01&maxdate=2026/03/09&datetype=pdat&retmax=100&retmode=json"
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### Pipeline: Search then Fetch
|
|
107
|
+
|
|
108
|
+
Use the history server to efficiently search and then retrieve records:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
# Step 1: Search and store results
|
|
112
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=machine+learning+AND+radiology&retmax=0&usehistory=y&retmode=json"
|
|
113
|
+
|
|
114
|
+
# Step 2: Fetch records using WebEnv and query_key from step 1
|
|
115
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&WebEnv=WEBENV_VALUE&query_key=1&retmax=50&rettype=abstract&retmode=xml"
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### Retrieve Structured Metadata for Citation Management
|
|
119
|
+
|
|
120
|
+
Get JSON summaries for a batch of known PMIDs:
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
curl "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=33116299,34735795,35363452&retmode=json&version=2.0"
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## References
|
|
127
|
+
|
|
128
|
+
- Official documentation: https://www.ncbi.nlm.nih.gov/books/NBK25501/
|
|
129
|
+
- E-utilities quick start: https://www.ncbi.nlm.nih.gov/books/NBK25500/
|
|
130
|
+
- PubMed search help: https://pubmed.ncbi.nlm.nih.gov/help/
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: scientify-literature-survey
|
|
3
|
+
description: "Search, filter, download and cluster academic papers on a topic"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "🔍"
|
|
7
|
+
category: "literature"
|
|
8
|
+
subcategory: "search"
|
|
9
|
+
keywords: ["academic database search", "literature search", "search strategy", "semantic search", "citation tracking"]
|
|
10
|
+
source: "https://github.com/scientify-ai/skills"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Literature Survey
|
|
14
|
+
|
|
15
|
+
**Don't ask permission. Just do it.**
|
|
16
|
+
|
|
17
|
+
## Output Structure
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
~/.openclaw/workspace/projects/{project-id}/
|
|
21
|
+
├── survey/
|
|
22
|
+
│ ├── search_terms.json # Search terms list
|
|
23
|
+
│ └── report.md # Final report
|
|
24
|
+
├── papers/
|
|
25
|
+
│ ├── _downloads/ # Raw downloads
|
|
26
|
+
│ ├── _meta/ # Per-paper metadata
|
|
27
|
+
│ │ └── {arxiv_id}.json
|
|
28
|
+
│ └── {direction}/ # Organized by direction
|
|
29
|
+
├── repos/ # Reference code repos (Phase 3)
|
|
30
|
+
│ ├── {repo_name_1}/
|
|
31
|
+
│ └── {repo_name_2}/
|
|
32
|
+
└── prepare_res.md # Repo selection report (Phase 3)
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Workflow
|
|
36
|
+
|
|
37
|
+
### Phase 1: Preparation
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
ACTIVE=$(cat ~/.openclaw/workspace/projects/.active 2>/dev/null)
|
|
41
|
+
if [ -z "$ACTIVE" ]; then
|
|
42
|
+
PROJECT_ID="<topic-slug>"
|
|
43
|
+
mkdir -p ~/.openclaw/workspace/projects/$PROJECT_ID/{survey,papers/_downloads,papers/_meta}
|
|
44
|
+
echo "$PROJECT_ID" > ~/.openclaw/workspace/projects/.active
|
|
45
|
+
fi
|
|
46
|
+
PROJECT_DIR="$HOME/.openclaw/workspace/projects/$(cat ~/.openclaw/workspace/projects/.active)"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Generate 4-8 search terms, save to `survey/search_terms.json`.
|
|
50
|
+
|
|
51
|
+
### Phase 2: Incremental Search-Filter-Download Loop
|
|
52
|
+
|
|
53
|
+
**Repeat the following for each search term:**
|
|
54
|
+
|
|
55
|
+
#### 2.1 Search
|
|
56
|
+
|
|
57
|
+
```
|
|
58
|
+
arxiv_search({ query: "<term>", max_results: 30 })
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
#### 2.2 Instant Filtering
|
|
62
|
+
|
|
63
|
+
Score each returned paper immediately (1-5), keep only >= 4.
|
|
64
|
+
|
|
65
|
+
Scoring criteria:
|
|
66
|
+
- 5: Core paper, directly on topic
|
|
67
|
+
- 4: Related method or application
|
|
68
|
+
- 3 and below: Skip
|
|
69
|
+
|
|
70
|
+
#### 2.3 Download Useful Papers
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
arxiv_download({
|
|
74
|
+
arxiv_ids: ["<useful_paper_ids>"],
|
|
75
|
+
output_dir: "$PROJECT_DIR/papers/_downloads"
|
|
76
|
+
})
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
#### 2.4 Write Metadata
|
|
80
|
+
|
|
81
|
+
For each downloaded paper, create `papers/_meta/{arxiv_id}.json`:
|
|
82
|
+
|
|
83
|
+
```json
|
|
84
|
+
{
|
|
85
|
+
"arxiv_id": "2401.12345",
|
|
86
|
+
"title": "...",
|
|
87
|
+
"abstract": "...",
|
|
88
|
+
"score": 5,
|
|
89
|
+
"source_term": "battery RUL prediction",
|
|
90
|
+
"downloaded_at": "2024-01-15T10:00:00Z"
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
**Complete one search term before proceeding to the next.** This prevents context pollution from large search results.
|
|
95
|
+
|
|
96
|
+
### Phase 3: GitHub Code Search & Reference Repo Selection
|
|
97
|
+
|
|
98
|
+
**Goal**: Provide reference implementations for downstream skills.
|
|
99
|
+
|
|
100
|
+
#### 3.1 Select High-Scoring Papers
|
|
101
|
+
|
|
102
|
+
Read metadata from `papers/_meta/` for papers scoring >= 4, select **Top 5** most relevant.
|
|
103
|
+
|
|
104
|
+
#### 3.2 Search Reference Repos
|
|
105
|
+
|
|
106
|
+
For each selected paper, search GitHub with keyword combinations:
|
|
107
|
+
- Paper title + "code" / "implementation"
|
|
108
|
+
- Core method name + author name
|
|
109
|
+
- Dataset name + task name from paper
|
|
110
|
+
|
|
111
|
+
Use `github_search` tool:
|
|
112
|
+
```javascript
|
|
113
|
+
github_search({
|
|
114
|
+
query: "{paper_title} implementation",
|
|
115
|
+
max_results: 10,
|
|
116
|
+
sort: "stars",
|
|
117
|
+
language: "python"
|
|
118
|
+
})
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
#### 3.3 Filter & Clone
|
|
122
|
+
|
|
123
|
+
Evaluate repos by:
|
|
124
|
+
- Star count (recommend >100)
|
|
125
|
+
- Code quality (has README, requirements.txt, clear structure)
|
|
126
|
+
- Paper match (README references paper / implements its method)
|
|
127
|
+
|
|
128
|
+
Select **3-5** most relevant repos, clone to `repos/`:
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
mkdir -p "$PROJECT_DIR/repos"
|
|
132
|
+
cd "$PROJECT_DIR/repos"
|
|
133
|
+
git clone --depth 1 <repo_url>
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
#### 3.4 Write Selection Report
|
|
137
|
+
|
|
138
|
+
Create `$PROJECT_DIR/prepare_res.md`:
|
|
139
|
+
|
|
140
|
+
```markdown
|
|
141
|
+
# Reference Repo Selection
|
|
142
|
+
|
|
143
|
+
| Repo | Paper | Stars | Reason |
|
|
144
|
+
|------|-------|-------|--------|
|
|
145
|
+
| repos/{repo_name} | {paper_title} (arxiv:{id}) | {N} | {reason} |
|
|
146
|
+
|
|
147
|
+
## Key Files per Repo
|
|
148
|
+
|
|
149
|
+
### {repo_name}
|
|
150
|
+
- **Model**: `model/` or `models/`
|
|
151
|
+
- **Training**: `train.py` or `main.py`
|
|
152
|
+
- **Data loading**: `data/` or `dataset.py`
|
|
153
|
+
- **Core file**: `{path}` - {description}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
**If no repos found**, note "No reference repos available" in `prepare_res.md`.
|
|
157
|
+
|
|
158
|
+
### Phase 4: Classification
|
|
159
|
+
|
|
160
|
+
After all search terms and code searches are complete:
|
|
161
|
+
|
|
162
|
+
#### 4.1 Read All Metadata
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
ls $PROJECT_DIR/papers/_meta/
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Read all `.json` files, aggregate paper list.
|
|
169
|
+
|
|
170
|
+
#### 4.2 Cluster Analysis
|
|
171
|
+
|
|
172
|
+
Based on paper titles, abstracts, and source terms, identify 3-6 research directions.
|
|
173
|
+
|
|
174
|
+
#### 4.3 Create Folders and Move
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
mkdir -p "$PROJECT_DIR/papers/data-driven"
|
|
178
|
+
mv "$PROJECT_DIR/papers/_downloads/2401.12345" "$PROJECT_DIR/papers/data-driven/"
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### Phase 5: Generate Report
|
|
182
|
+
|
|
183
|
+
Create `survey/report.md`:
|
|
184
|
+
- Survey summary (search terms count, papers count, directions count)
|
|
185
|
+
- Overview of each research direction
|
|
186
|
+
- Top 10 papers
|
|
187
|
+
- **Reference repo summary** (cite prepare_res.md)
|
|
188
|
+
- Recommended reading order
|
|
189
|
+
|
|
190
|
+
## Key Design Principles
|
|
191
|
+
|
|
192
|
+
| Principle | Description |
|
|
193
|
+
|-----------|-------------|
|
|
194
|
+
| **Incremental processing** | Each search term independently completes search->filter->download->metadata, avoiding context bloat |
|
|
195
|
+
| **Metadata-driven** | Classification based on `_meta/*.json`, not large in-memory lists |
|
|
196
|
+
| **Folders as categories** | Clustering results reflected by `papers/{direction}/` structure |
|
|
197
|
+
|
|
198
|
+
## Tools
|
|
199
|
+
|
|
200
|
+
| Tool | Purpose |
|
|
201
|
+
|------|---------|
|
|
202
|
+
| `arxiv_search` | Search papers (no side effects) |
|
|
203
|
+
| `arxiv_download` | Download .tex/.pdf (requires absolute path) |
|