@wentorai/research-plugins 1.0.0 → 1.2.0

package/skills/tools/ocr-translate/zotero-pdf-translate-guide/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: zotero-pdf-translate-guide
+description: "Guide to Zotero PDF Translate for multilingual PDF and annotation translation"
+metadata:
+  openclaw:
+    emoji: "🌐"
+    category: "tools"
+    subcategory: "ocr-translate"
+    keywords: ["zotero", "pdf-translate", "multilingual", "annotation", "academic-translation"]
+    source: "https://github.com/windingwind/zotero-pdf-translate"
+---
+
+# Zotero PDF Translate Guide
+
+## Overview
+
+Zotero PDF Translate is one of the most popular Zotero plugins with over 10,000 stars on GitHub, providing seamless translation capabilities directly within the Zotero PDF reader. It enables researchers to translate selected text, annotations, metadata, and even entire pages without leaving their reference management workflow.
+
+The plugin supports a wide range of translation engines including Google Translate, DeepL, Microsoft Translator, OpenAI, and numerous other services. This flexibility allows researchers to choose the engine that best suits their language pair, domain, and accuracy requirements. For academic work involving technical terminology, the ability to switch between engines or use specialized services is invaluable.
+
+Zotero PDF Translate goes beyond simple text translation. It handles EPub documents, web pages saved to Zotero, item metadata fields, and annotation notes. This makes it an essential tool for any researcher who regularly reads papers in languages other than their primary working language, or who collaborates across linguistic boundaries.
+
+## Installation and Setup
+
+Install Zotero PDF Translate through the Zotero Add-ons Manager:
+
+1. Download the latest `.xpi` file from the GitHub releases page at https://github.com/windingwind/zotero-pdf-translate/releases
+2. In Zotero, navigate to Tools > Add-ons > gear icon > Install Add-on From File
+3. Select the downloaded `.xpi` file and restart Zotero when prompted
+
+After installation, configure your preferred translation engines:
+
+- Open Zotero Preferences > PDF Translate
+- Select your primary translation engine from the dropdown
+- For engines requiring API access, set your credentials via environment variables:
+  - DeepL: configure `$DEEPL_API_KEY` in your environment
+  - OpenAI: configure `$OPENAI_API_KEY` in your environment
+  - Microsoft: configure `$AZURE_TRANSLATOR_KEY` in your environment
+- Set your source and target languages
+- Configure auto-translate behavior (on selection, on annotation, etc.)
+
+For best results with academic content, consider using DeepL or OpenAI-based translation, as these engines generally handle technical vocabulary and complex sentence structures better than basic machine translation services.
+
+## Core Features
+
+**In-Reader Translation**: Select any text in the Zotero PDF reader and the translation appears instantly in a sidebar panel. The original and translated text are displayed side by side, making it easy to verify accuracy and understand context.
+
+**Annotation Translation**: When you highlight text and create annotations, Zotero PDF Translate can automatically translate the annotation content. This is particularly useful when building a collection of notes from foreign-language papers, as your annotation library remains in your working language.
+
+**Metadata Translation**: Translate item titles, abstracts, and other metadata fields directly within your Zotero library. This helps when organizing a large collection that includes papers from multiple languages, making search and browsing more efficient.
+
+**Multiple Engine Support**: The plugin supports over 15 translation engines:
+- Free engines: Google Translate, Bing, Yandex, LibreTranslate
+- Premium engines: DeepL, OpenAI GPT, Azure Translator
+- Chinese-specialized: Baidu, Tencent, Youdao, Caiyun
+- Academic: CNKI translation services
+
+**Batch Translation**: Translate multiple items or annotations at once rather than processing them one by one. This is essential when processing a large reading list of foreign-language papers.
+
+**Custom Dictionaries**: Define custom term mappings to ensure domain-specific terminology is translated consistently. For example, you can map technical terms in your field to their correct translations rather than relying on generic machine translation.
+
+## Academic Workflow Integration
+
+A typical multilingual research workflow with Zotero PDF Translate involves several stages:
+
+**Literature Discovery**: When you encounter papers in unfamiliar languages through citation tracking or database searches, add them to Zotero normally. Use metadata translation to translate titles and abstracts, helping you triage which papers deserve full reading.
+
+**Deep Reading**: Open a foreign-language PDF in the Zotero reader. As you read, select passages for instant translation. For particularly important sections, create annotations that automatically include both the original text and translation.
+
+**Note Compilation**: After reading, your Zotero annotations contain bilingual notes. Export these to your note-taking system with both original quotes and translations preserved, ensuring you can always trace back to the source text.
+
+**Collaborative Research**: When working with international collaborators, use batch translation to prepare shared reading lists with translated metadata, making it easier for team members to navigate references outside their language expertise.
+
+**Configuration Tips for Researchers**:
+- Set keyboard shortcuts for quick translation toggling
+- Enable auto-translate on text selection for fluid reading
+- Configure fallback engines in case the primary service is unavailable
+- Use custom dictionaries for your specific research domain terminology
+
+## Troubleshooting Common Issues
+
+**Translation Not Appearing**: Ensure the plugin is enabled in Tools > Add-ons. Check that your selected translation engine is properly configured and that any required API keys are set in environment variables.
+
+**Slow Translation Speed**: Some engines have rate limits. If translation is slow, try switching to a different engine or enabling caching in the plugin settings to avoid re-translating previously seen text.
+
+**Incorrect Technical Terms**: Academic translation often struggles with domain-specific vocabulary. Use the custom dictionary feature to define correct translations for key terms in your field. This builds up over time into a valuable domain-specific translation resource.
+
+**Character Encoding Issues**: Some older PDFs may have character encoding problems that affect text extraction and translation. Try re-saving the PDF or using Zotero's built-in PDF processing to improve text extraction quality.
+
+## References
+
+- GitHub Repository: https://github.com/windingwind/zotero-pdf-translate
+- Zotero Plugin Directory: https://www.zotero.org/support/plugins
+- Zotero Forums Translation Discussion: https://forums.zotero.org
+- DeepL API Documentation: https://www.deepl.com/docs-api

package/skills/tools/ocr-translate/zotero-pdf2zh-guide/SKILL.md

@@ -0,0 +1,143 @@
+---
+name: zotero-pdf2zh-guide
+description: "PDF Chinese translation plugin for Zotero reference manager"
+version: 1.0.0
+author: wentor-community
+source: https://github.com/guaguastandup/zotero-pdf2zh
+metadata:
+  openclaw:
+    category: "tools"
+    subcategory: "ocr-translate"
+    keywords:
+      - pdf-translation
+      - zotero-plugin
+      - chinese-translation
+      - ocr
+      - academic-papers
+      - bilingual-reading
+---
+
+# Zotero PDF2ZH Guide
+
+A skill for using the Zotero PDF2ZH plugin to translate academic PDF documents between Chinese and English while preserving the original layout, figures, and mathematical notation. Based on zotero-pdf2zh (3K stars), this skill enables researchers to bridge the language barrier in academic literature.
+
+## Overview
+
+Academic research is inherently global, yet language barriers remain a significant obstacle. Chinese-speaking researchers need to read English-language papers, and English-speaking researchers increasingly need access to Chinese-language publications, particularly in fields where China produces significant research output (materials science, AI, engineering, traditional medicine). PDF2ZH addresses this by providing high-quality translation directly within the Zotero reference management workflow, producing bilingual documents that preserve the original formatting.
+
+The plugin supports bidirectional translation between Chinese and English, leveraging modern neural machine translation while maintaining the structural integrity of academic documents including equations, tables, figures, and references.
+
+## Installation and Setup
+
+**Prerequisites**
+- Zotero 6 or Zotero 7 installed and configured
+- An active internet connection for translation API access
+- Sufficient storage for translated PDF copies (approximately 2x per document)
+
+**Installation Steps**
+- Download the latest .xpi file from the project's releases page
+- In Zotero, navigate to Tools then Add-ons
+- Click the gear icon and select Install Add-on From File
+- Select the downloaded .xpi file and confirm installation
+- Restart Zotero to activate the plugin
+- Configure translation settings in the plugin preferences panel
+
+**Configuration Options**
+- Select the default translation direction (EN to ZH or ZH to EN)
+- Choose the translation engine (multiple backends supported)
+- Configure output format (side-by-side bilingual or translated-only)
+- Set quality preferences (speed versus accuracy trade-off)
+- Define custom terminology dictionaries for domain-specific terms
+
+## Translation Workflow
+
+**Single Document Translation**
+- Right-click a PDF item in the Zotero library
+- Select the PDF2ZH translation option from the context menu
+- Choose the target language if different from the default
+- The plugin processes the document and creates a translated copy
+- The translated PDF is automatically attached to the same Zotero item
+- Original and translated versions are both accessible from the item entry
+
+**Batch Translation**
+- Select multiple items in the Zotero library
+- Apply translation to the entire selection
+- The plugin queues documents and processes them sequentially
+- Progress is displayed in the Zotero status bar
+- Failed translations are flagged for manual review
+
+**Translation Quality Features**
+- Mathematical equations are detected and preserved without translation
+- Figures and their captions are handled separately for better accuracy
+- Tables maintain their structure with cell-level translation
+- Reference lists are preserved in their original form
+- Headers, footers, and page numbers are excluded from translation
+
+## Handling Academic Content
+
+The plugin includes special handling for academic document elements:
+
+**Mathematical Notation**
+- LaTeX-style equations are detected and excluded from translation
+- Inline math symbols and variables are preserved in context
+- Equation numbers and cross-references maintain their original format
+- Greek letters and mathematical operators are not transliterated
+
+**Technical Terminology**
+- Domain-specific terms can be added to custom dictionaries
+- The plugin supports glossary files for consistent term translation
+- Ambiguous terms are handled based on the document's detected field
+- Acronyms are expanded on first occurrence in the translation
+- Standard academic phrases have pre-verified translations
+
+**Figures and Tables**
+- Figure labels and captions are translated while images are preserved
+- Table headers and cell contents are translated maintaining structure
+- Chart axis labels and legends within embedded images are not modified
+- Cross-references to figures and tables maintain their numbering
+
+## Bilingual Reading Mode
+
+The side-by-side bilingual output is particularly useful for research:
+
+**Layout Options**
+- Parallel paragraphs: original and translation appear side by side
+- Interleaved paragraphs: translation appears below each original paragraph
+- Margin annotations: brief translations appear in the margin alongside originals
+- Separate documents: full original and full translation as independent PDFs
+
+**Reading Strategies**
+- Use bilingual mode to learn technical vocabulary in the target language
+- Reference the original when the translation of a technical passage is unclear
+- Compare sentence structures to understand discipline-specific writing conventions
+- Build personal glossaries from translated terms in your research area
+
+## Domain-Specific Considerations
+
+Translation quality varies by academic domain:
+
+**High Accuracy Domains** - Computer science, mathematics, physics (standardized terminology)
+**Medium Accuracy Domains** - Biology, chemistry, engineering (mixed standardized and specialized terms)
+**Challenging Domains** - Social sciences, humanities, law (culturally embedded terminology)
+**Specialized Domains** - Traditional Chinese medicine, regional studies (unique conceptual frameworks)
+
+For challenging domains, always verify key translated terms against established bilingual dictionaries or domain experts.
+
+## Integration with Research-Claw
+
+This skill enhances the Research-Claw multilingual research workflow:
+
+- Translate papers discovered through literature search skills
+- Feed translated content to paper analysis skills for knowledge extraction
+- Support bilingual literature reviews spanning Chinese and English sources
+- Connect with citation management skills for multilingual bibliographies
+- Enable cross-language comparison of research findings on the same topic
+
+## Best Practices
+
+- Always verify critical translations against the original, especially for methodology details
+- Build and maintain domain-specific glossaries to improve translation consistency
+- Use bilingual mode for important papers to catch translation nuances
+- Batch-translate collection items during off-peak hours to manage API load
+- Keep both original and translated versions in Zotero for reference
+- Report translation errors to improve the tool's accuracy over time

package/skills/tools/scraping/dataset-finder-guide/SKILL.md

@@ -0,0 +1,253 @@
+---
+name: dataset-finder-guide
+description: "Search and download research datasets from Kaggle, HuggingFace, and repos"
+metadata:
+  openclaw:
+    emoji: "database"
+    category: "tools"
+    subcategory: "scraping"
+    keywords: ["dataset", "Kaggle", "data download", "HuggingFace", "data repository", "open data"]
+    source: "https://github.com/wentor-ai/research-plugins"
+---
+
+# Dataset Finder Guide
+
+Search, evaluate, and download research datasets from major repositories including Kaggle, Hugging Face, Google Dataset Search, Zenodo, UCI Machine Learning Repository, and domain-specific archives. This skill helps researchers locate the right data for their experiments efficiently.
+
+## Overview
+
+Finding suitable datasets is often one of the most time-consuming phases of empirical research. Datasets are scattered across dozens of platforms, each with different APIs, licensing terms, download mechanisms, and metadata standards. A single research project might require datasets from Kaggle for benchmarking, Hugging Face for NLP tasks, Zenodo for supplementary materials from published papers, and government open data portals for demographic or economic variables.
+
+This skill provides a unified approach to dataset discovery: formulating search queries, evaluating dataset quality and suitability, understanding licensing implications, and efficiently downloading and organizing data. It covers both general-purpose repositories and domain-specific archives that researchers in various fields need.
+
+The emphasis is on reproducibility -- every dataset used in research should be citable, versioned, and documented. This skill includes patterns for recording dataset provenance, creating data cards, and managing dataset versions across experiments.
+
+## Dataset Repositories
+
+### General-Purpose Repositories
+
+| Repository | Strengths | API | Citation Support |
+|------------|-----------|-----|-----------------|
+| Kaggle | ML benchmarks, competitions, community kernels | REST + CLI | DOI via dataset cards |
+| Hugging Face Datasets | NLP, CV, audio; streaming support | Python library | Built-in citation |
+| Zenodo | Any research data, DOI minting, EU-funded | REST API | Automatic DOI |
+| Google Dataset Search | Meta-search across repositories | Web only | Links to source |
+| UCI ML Repository | Classic ML benchmarks | Direct download | BibTeX provided |
+| Figshare | Figures, datasets, media, preprints | REST API | DOI per item |
+| Dryad | Ecology, biology, environmental science | REST API | DOI per dataset |
+| ICPSR | Social science survey data | Restricted API | Persistent IDs |
+| Harvard Dataverse | Multi-discipline, institutional | REST API | DOI per dataset |
+
+### Domain-Specific Archives
+
+| Domain | Repository | Notable Datasets |
+|--------|-----------|-----------------|
+| Genomics | NCBI GEO, ENA | Gene expression, sequencing data |
+| Astronomy | NASA archives, SDSS | Sky surveys, spectral data |
+| Economics | FRED, World Bank, IMF | Time series, macro indicators |
+| Climate | NOAA, CMIP6 | Temperature, precipitation records |
+| Linguistics | LDC, CLARIN | Corpora, treebanks |
+| Medical | PhysioNet, MIMIC | Clinical records, ECG/EEG |
+| Chemistry | PubChem, ChEMBL | Molecular structures, bioassays |
+
+## Searching for Datasets
+
+### Kaggle CLI
+
+```bash
+# Install and configure
+pip install kaggle
+# Place kaggle.json in ~/.kaggle/
+
+# Search datasets
+kaggle datasets list -s "sentiment analysis" --sort-by votes
+kaggle datasets list -s "medical imaging" --file-type csv --min-size 100MB
+
+# Get dataset details
+kaggle datasets metadata -d stanford/imdb-review-dataset
+
+# Download dataset
+kaggle datasets download -d stanford/imdb-review-dataset -p ./data/
+unzip ./data/imdb-review-dataset.zip -d ./data/imdb/
+
+# Download competition data
+kaggle competitions download -c titanic -p ./data/
+```
+
+### Hugging Face Datasets
+
+```python
+from datasets import load_dataset, list_datasets
+
+# Search for datasets by task
+from huggingface_hub import HfApi
+api = HfApi()
+datasets = api.list_datasets(
+    search="scientific papers",
+    sort="downloads",
+    direction=-1,
+    limit=20
+)
+for ds in datasets:
+    print(f"{ds.id}: {ds.downloads} downloads")
+
+# Load a dataset (with streaming for large datasets)
+dataset = load_dataset("scientific_papers", "arxiv", streaming=True)
+
+# Inspect structure
+print(dataset["train"].features)
+print(f"Number of examples: {dataset['train'].num_rows}")
+
+# Load specific split and subset
+validation = load_dataset(
+    "scientific_papers", "arxiv",
+    split="validation[:1000]"
+)
+```
+
+### Google Dataset Search (Programmatic)
+
+```python
+import requests
+from bs4 import BeautifulSoup
+
+def search_google_datasets(query, num_results=10):
+    """Search Google Dataset Search and extract results."""
+    url = f"https://datasetsearch.research.google.com/search"
+    params = {"query": query, "docid": ""}
+    # Note: Google Dataset Search does not have an official API
+    # Use the web interface or alternative approaches
+    print(f"Search at: {url}?query={query.replace(' ', '+')}")
+    return url
+```
+
+### Zenodo API
+
+```python
+import requests
+
+def search_zenodo(query, resource_type="dataset", size=10):
+    """Search Zenodo for research datasets."""
+    url = "https://zenodo.org/api/records"
+    params = {
+        "q": query,
+        "type": resource_type,
+        "size": size,
+        "sort": "mostrecent",
+        "access_right": "open"
+    }
+    response = requests.get(url, params=params)
+    results = response.json()
+
+    for hit in results.get("hits", {}).get("hits", []):
+        meta = hit["metadata"]
+        print(f"Title: {meta['title']}")
+        print(f"DOI: {meta.get('doi', 'N/A')}")
+        print(f"License: {meta.get('license', {}).get('id', 'N/A')}")
+        print(f"Size: {sum(f['size'] for f in hit.get('files', []))/1e6:.1f} MB")
+        print("---")
+
+    return results
+```
+
+## Dataset Evaluation Checklist
+
+Before using a dataset in research, verify the following:
+
+### Quality Assessment
+
+- **Completeness**: What percentage of values are missing? Are missing values random or systematic?
+- **Accuracy**: Are values within expected ranges? Are there obvious errors?
+- **Consistency**: Are formats uniform (dates, categories, units)?
+- **Timeliness**: When was the data collected? Is it current enough for your research question?
+- **Sample size**: Is the dataset large enough for your intended analysis?
+
+### Licensing and Ethics
+
+| License | Commercial Use | Modification | Attribution Required |
+|---------|---------------|-------------|---------------------|
+| CC0 | Yes | Yes | No |
+| CC-BY 4.0 | Yes | Yes | Yes |
+| CC-BY-SA 4.0 | Yes | Yes (share-alike) | Yes |
+| CC-BY-NC 4.0 | No | Yes | Yes |
+| ODC-ODbL | Yes | Yes (share-alike) | Yes |
+| Custom/Restricted | Varies | Varies | Varies |
+
+### Reproducibility Documentation
+
+```markdown
+## Data Card
+
+**Dataset**: [Name]
+**Source**: [URL]
+**Version**: [Version/Date]
+**DOI**: [DOI if available]
+**License**: [License name]
+**Downloaded**: [YYYY-MM-DD]
+**Size**: [X rows, Y columns, Z MB]
+**Description**: [Brief description]
+**Preprocessing**: [Steps applied before use]
+**Citation**: [BibTeX entry]
+```
+
+## Download and Organization
+
+### Project Data Structure
+
+```
+project/
+  data/
+    raw/              # Original downloaded data (never modify)
+      dataset_v1.csv
+      README.md       # Data card with provenance
+    processed/        # Cleaned and transformed data
+      train.csv
+      test.csv
+    external/         # Third-party reference data
+  scripts/
+    download_data.py  # Reproducible download script
+    preprocess.py     # Data cleaning pipeline
+```
+
+### Reproducible Download Script
+
+```python
+"""download_data.py - Reproducible dataset download."""
+import hashlib
+from pathlib import Path
+import requests
+
+DATASETS = {
+    "main_dataset": {
+        "url": "https://zenodo.org/record/12345/files/data.csv",
+        "sha256": "abc123...",
+        "filename": "raw/main_dataset.csv"
+    }
+}
+
+DATA_DIR = Path("data")
+
+for name, info in DATASETS.items():
+    path = DATA_DIR / info["filename"]
+    if path.exists():
+        print(f"Already downloaded: {name}")
+        continue
+
+    path.parent.mkdir(parents=True, exist_ok=True)
+    print(f"Downloading {name}...")
+    response = requests.get(info["url"])
+    path.write_bytes(response.content)
+
+    # Verify integrity
+    sha256 = hashlib.sha256(response.content).hexdigest()
+    assert sha256 == info["sha256"], f"Checksum mismatch for {name}"
+    print(f"Verified: {name}")
+```
+
+## References
+
+- Kaggle API documentation: https://github.com/Kaggle/kaggle-api
+- Hugging Face Datasets: https://huggingface.co/docs/datasets
+- Zenodo API: https://developers.zenodo.org
+- Google Dataset Search: https://datasetsearch.research.google.com
+- Gebru et al., "Datasheets for Datasets" (2021): https://arxiv.org/abs/1803.09010