@wentorai/research-plugins 1.0.0 → 1.2.0

package/skills/research/paper-review/automated-review-guide/SKILL.md

@@ -0,0 +1,281 @@
+---
+name: automated-review-guide
+description: "AI-assisted peer review tools, workflows, and quality standards"
+metadata:
+  openclaw:
+    emoji: "robot"
+    category: "research"
+    subcategory: "paper-review"
+    keywords: ["automated review", "AI peer review", "LLM review", "review quality", "manuscript screening", "editorial workflow"]
+    source: "wentor-research-plugins"
+---
+
+# Automated Review Guide
+
+A skill for leveraging AI-assisted tools in the peer review process, covering both author-side self-review and editor-side manuscript screening. Addresses tool selection, prompt engineering for review tasks, limitations and biases of LLM-generated reviews, quality assurance workflows, and ethical guidelines for AI use in peer review.
+
+## Overview of AI in Peer Review
+
+### Current Landscape
+
+AI-assisted peer review tools operate at multiple stages of the publication pipeline. Understanding where automation adds genuine value and where it introduces risk is essential for responsible adoption.
+
+```
+Where AI assists in peer review:
+
+Author-side (pre-submission):
+  - Grammar and style checking (Grammarly, Writefull)
+  - Statistical result verification (statcheck, GRIM/SPRITE)
+  - Reference completeness checking
+  - Plagiarism detection (iThenticate, Turnitin)
+  - Readability scoring
+  - Structural completeness (IMRAD compliance)
+
+Editor-side (triage and assignment):
+  - Desk rejection screening (scope, quality threshold)
+  - Reviewer matching (expertise alignment)
+  - Conflict of interest detection
+  - Duplicate submission detection
+  - Plagiarism and image manipulation screening
+
+Reviewer-side (review assistance):
+  - Paper summarization for rapid assessment
+  - Statistical claim verification
+  - Reference checking (do cited papers support claims)
+  - Comparison with related work
+  - Structured review template generation
+
+Post-review:
+  - Decision consistency analysis
+  - Review quality assessment
+  - Revision compliance checking
+```
+
+## Self-Review with AI Before Submission
+
+### Structured Self-Review Prompts
+
+```
+Pre-submission AI review checklist:
+
+1. Abstract completeness check:
+   Prompt: "Analyze this abstract. Does it contain:
+   (a) background/motivation, (b) research gap,
+   (c) methodology summary, (d) key results with
+   numbers, (e) conclusion/implication? Identify
+   any missing elements."
+
+2. Claim-evidence alignment:
+   Prompt: "For each claim in the Discussion section,
+   identify the specific result (table, figure, or
+   statistical test) that supports it. Flag any claims
+   without corresponding evidence in the Results."
+
+3. Methods reproducibility:
+   Prompt: "Read the Methods section and list every
+   piece of information that another researcher would
+   need to replicate this study. Identify any gaps:
+   missing sample sizes, unspecified parameters,
+   ambiguous procedures, unnamed software versions."
+
+4. Statistical reporting:
+   Prompt: "Check all statistical results in this paper
+   for completeness. Each test should report: test name,
+   test statistic, degrees of freedom, p-value, and
+   effect size. List any incomplete reports."
+
+5. Reference audit:
+   Prompt: "For each citation in the Introduction, verify
+   that the cited claim matches the in-text description.
+   Flag any cases where the citation might not support
+   the specific claim being made."
+```
+
+### Automated Statistical Checking
+
+```python
+import re
+
+def check_statistical_reporting(text):
+    """
+    Check for common statistical reporting issues.
+
+    Verifies:
+    - p-values are reported with test statistics
+    - Degrees of freedom are included
+    - Effect sizes are reported
+    - Exact p-values (not just p < .05)
+    """
+    issues = []
+
+    # Find p-value reports
+    p_pattern = r'p\s*[<=<>]\s*\.?\d+'
+    p_matches = re.finditer(p_pattern, text, re.IGNORECASE)
+
+    for match in p_matches:
+        # Check context (100 chars before) for test statistic
+        start = max(0, match.start() - 100)
+        context = text[start:match.end()]
+
+        has_test_stat = any(
+            stat in context for stat in
+            ["t(", "F(", "chi", "r(", "r =", "z =",
+             "U =", "W =", "H(", "d =", "eta"]
+        )
+
+        if not has_test_stat:
+            issues.append({
+                "location": match.start(),
+                "text": context[-50:],
+                "issue": "p-value without test statistic"
+            })
+
+    # Check for "p < .05" without exact values
+    vague_p = re.findall(r'p\s*<\s*\.05(?!\d)', text)
+    if vague_p:
+        issues.append({
+            "issue": f"Found {len(vague_p)} instances of 'p < .05' "
+                     "without exact p-values. APA recommends exact values."
+        })
+
+    return issues
+```
+
+## Limitations and Biases of AI Reviews
+
+### Known Failure Modes
+
+```
+AI review limitations to be aware of:
+
+1. Hallucinated references:
+   - LLMs may claim a paper cites X when it does not
+   - Always verify any reference claims made by AI
+   - LLMs cannot actually read PDFs behind paywalls
+
+2. False confidence in statistical judgments:
+   - LLMs may incorrectly flag valid statistical approaches
+   - They may miss subtle errors that require domain expertise
+   - Statistical verification tools (statcheck) are more reliable
+
+3. Novelty assessment failures:
+   - LLMs have knowledge cutoff dates and cannot assess true novelty
+   - They may flag well-known methods as novel or novel methods as
+     well-known, depending on training data coverage
+   - Human expertise is essential for novelty evaluation
+
+4. Disciplinary bias:
+   - LLMs trained primarily on English text from well-resourced fields
+   - May apply STEM conventions to humanities papers inappropriately
+   - May not recognize valid methodologies in underrepresented fields
+
+5. Sycophancy:
+   - Tendency to agree with the framing of the prompt
+   - "Review this excellent paper" vs "Review this paper" yields
+     systematically different feedback
+   - Use neutral prompts and ask for both strengths and weaknesses
+
+6. Reproducibility of reviews:
+   - Same paper reviewed twice may get different feedback
+   - Temperature settings affect consistency
+   - Document model, version, and prompt for reproducibility
+```
+
+## Quality Assurance Workflow
+
+### Human-AI Hybrid Review
+
+```
+Recommended workflow for responsible AI-assisted review:
+
+Step 1 - Human first read (30 minutes):
+  Read the paper yourself without AI assistance.
+  Form your own initial impressions about strengths,
+  weaknesses, and significance.
+
+Step 2 - AI-assisted deep dive (20 minutes):
+  Use AI to check specific aspects:
+  - Statistical reporting completeness
+  - Methods section gaps
+  - Reference verification
+  - Structural issues
+
+Step 3 - Human synthesis (30 minutes):
+  Integrate your own assessment with AI-flagged issues.
+  Verify every AI-identified issue before including it.
+  Discard AI suggestions that are incorrect or irrelevant.
+  Write the review in your own voice.
+
+Step 4 - Disclosure:
+  If journal policy requires it, disclose AI tool usage.
+  Many journals now have explicit policies on AI in review.
+
+Key principle: AI should help you be MORE thorough,
+not replace your expert judgment. The review is YOUR
+professional responsibility.
+```
+
+## Ethical Guidelines
+
+### Responsible Use Framework
+
+```
+Ethics of AI in peer review:
+
+Transparency:
+  - Disclose AI tool usage per journal policy
+  - Do not present AI-generated text as your own analysis
+  - Note which aspects of the review were AI-assisted
+
+Confidentiality:
+  - NEVER upload full manuscripts to public AI services
+  - Use on-premises or privacy-preserving tools
+  - Manuscripts under review are confidential documents
+  - Check with the journal before using any AI tool
+
+Accountability:
+  - The reviewer, not the AI, is responsible for the review
+  - Verify all AI-generated claims and suggestions
+  - Do not blindly copy AI output into review reports
+  - You must understand and agree with every point in your review
+
+Fairness:
+  - Apply AI-assisted scrutiny equally to all papers
+  - Be aware of AI biases against non-native English text
+  - Do not use AI to generate reject recommendations automatically
+  - Maintain the same standards you would without AI assistance
+
+Journal policies (check before using AI):
+  - Nature: allows AI tools, requires disclosure
+  - Science: allows AI tools for editing, not for review content
+  - Many journals have not yet issued explicit guidance
+  - When in doubt, disclose and ask the editor
+```
+
+## Tools and Resources
+
+```
+AI review assistance tools:
+
+General-purpose:
+  - Writefull: academic language and style checking
+  - Paperpal: manuscript readiness assessment
+  - SciSpace: paper reading and comprehension assistance
+
+Statistical checking:
+  - statcheck: automatic verification of statistical results
+  - GRIM test: granularity-related inconsistency of means
+  - SPRITE: sample parameter reconstruction via iterative techniques
+
+Plagiarism and integrity:
+  - iThenticate: similarity detection (industry standard)
+  - Turnitin: similarity detection (education-focused)
+  - Imagetwin: image duplication detection
+
+Reference management:
+  - scite.ai: smart citation analysis (supporting/contrasting)
+  - Semantic Scholar: related work discovery
+  - Connected Papers: citation graph visualization
+```
+
+AI-assisted review tools are most valuable when they augment rather than replace human expertise. They excel at systematic, repetitive checks (statistical reporting, reference formatting, structural completeness) but cannot substitute for the domain knowledge, contextual understanding, and scholarly judgment that define quality peer review.

package/skills/research/paper-review/latte-review-guide/SKILL.md

@@ -0,0 +1,175 @@
+---
+name: latte-review-guide
+description: "Automate systematic literature reviews with LatteReview AI agents"
+metadata:
+  openclaw:
+    emoji: "☕"
+    category: "research"
+    subcategory: "paper-review"
+    keywords: ["LatteReview", "systematic review", "literature screening", "AI review", "title screening", "PRISMA"]
+    source: "https://github.com/PouriaRouzrokh/LatteReview"
+---
+
+# LatteReview Guide
+
+## Overview
+
+LatteReview is a low-code Python package that uses AI agents to automate systematic literature reviews. It handles title/abstract screening, full-text assessment, data extraction, and PRISMA-compliant reporting — tasks that typically consume hundreds of researcher-hours. Supports multiple LLM backends (Anthropic, OpenAI, local models).
+
+## Installation
+
+```bash
+pip install lattereview
+```
+
+## Core Workflow
+
+### Step 1: Initialize Review
+
+```python
+from lattereview import ReviewProject
+
+# Create a new review project
+project = ReviewProject(
+    name="ML in Medical Imaging Review",
+    research_question="What deep learning architectures are used for "
+                      "medical image segmentation?",
+    inclusion_criteria=[
+        "Uses deep learning for medical image segmentation",
+        "Published in peer-reviewed venue",
+        "Reports quantitative evaluation metrics",
+    ],
+    exclusion_criteria=[
+        "Review/survey articles",
+        "Non-English publications",
+        "Conference abstracts only",
+    ],
+)
+```
+
+### Step 2: Import Papers
+
+```python
+# Import from various sources
+project.import_papers("scopus_export.csv", source="scopus")
+project.import_papers("pubmed_export.csv", source="pubmed")
+
+# Or from a DataFrame
+import pandas as pd
+df = pd.read_csv("papers.csv")
+project.import_from_dataframe(df,
+    title_col="title",
+    abstract_col="abstract",
+    year_col="year",
+)
+
+print(f"Imported {project.total_papers} papers")
+```
+
+### Step 3: AI Screening
+
+```python
+from lattereview.agents import ScreeningAgent
+
+# Configure screening agent
+screener = ScreeningAgent(
+    llm_provider="anthropic",
+    model="claude-sonnet-4-20250514",
+    criteria=project.inclusion_criteria,
+    exclusion=project.exclusion_criteria,
+)
+
+# Title/abstract screening
+results = screener.screen(
+    project.papers,
+    mode="title_abstract",
+    confidence_threshold=0.7,
+)
+
+# Results include: decision, confidence, reasoning
+for paper in results[:3]:
+    print(f"{paper.title}")
+    print(f"  Decision: {paper.decision} "
+          f"(confidence: {paper.confidence:.2f})")
+    print(f"  Reason: {paper.reasoning}")
+```
+
+### Step 4: Data Extraction
+
+```python
+from lattereview.agents import ExtractionAgent
+
+extractor = ExtractionAgent(
+    llm_provider="anthropic",
+    fields={
+        "architecture": "Deep learning architecture used",
+        "dataset": "Medical imaging dataset",
+        "modality": "Imaging modality (CT, MRI, X-ray, etc.)",
+        "dice_score": "Best Dice similarity coefficient reported",
+        "sample_size": "Number of images/patients",
+    },
+)
+
+extracted = extractor.extract(project.included_papers)
+
+# Export structured data
+extracted.to_csv("extracted_data.csv")
+```
+
+### Step 5: Generate Report
+
+```python
+# PRISMA flow diagram
+project.generate_prisma_diagram("prisma.png")
+
+# Summary statistics
+summary = project.summarize()
+print(f"Screened: {summary['screened']}")
+print(f"Included: {summary['included']}")
+print(f"Excluded: {summary['excluded']}")
+```
+
+## Configuration
+
+```python
+# Use different LLM providers
+screener = ScreeningAgent(
+    llm_provider="openai",
+    model="gpt-4o",
+)
+
+# Local models via Ollama
+screener = ScreeningAgent(
+    llm_provider="ollama",
+    model="llama3",
+    base_url="http://localhost:11434",
+)
+```
+
+## Dual-Reviewer Mode
+
+```python
+# Simulate dual-reviewer screening for reliability
+results = screener.dual_screen(
+    project.papers,
+    models=["claude-sonnet-4-20250514", "gpt-4o"],
+    agreement_threshold=0.8,
+)
+
+# Papers with disagreement flagged for human review
+conflicts = [p for p in results if p.agreement < 0.8]
+print(f"{len(conflicts)} papers need human adjudication")
+```
+
+## Use Cases
+
+1. **Systematic reviews**: PRISMA-compliant literature reviews
+2. **Scoping reviews**: Rapid evidence mapping
+3. **Meta-analysis preparation**: Structured data extraction
+4. **Grant applications**: Quick literature landscape assessment
+
+## References
+
+- [LatteReview GitHub](https://github.com/PouriaRouzrokh/LatteReview)
+- [LatteReview Documentation](https://lattereview.readthedocs.io/)
+- Rouzrokh, P. et al. (2024). "LatteReview: AI-Assisted Systematic Literature Reviews."

package/skills/research/paper-review/paper-compare-guide/SKILL.md

@@ -0,0 +1,238 @@
+---
+name: paper-compare-guide
+description: "Compare research papers side-by-side on methodology and findings"
+metadata:
+  openclaw:
+    emoji: "⚖️"
+    category: "research"
+    subcategory: "paper-review"
+    keywords: ["paper comparison", "side-by-side analysis", "methodology comparison", "research synthesis", "critical analysis"]
+    source: "https://github.com/AcademicSkills/paper-compare-guide"
+---
+
+# Paper Compare Guide
+
+A skill for conducting structured side-by-side comparisons of research papers. Extracts and aligns key dimensions across papers including research questions, methodologies, datasets, results, and conclusions, producing comparison matrices and synthesis narratives that highlight agreements, contradictions, and complementary contributions.
+
+## Overview
+
+Comparing multiple papers on the same topic is a fundamental task in academic research, required for literature reviews, related work sections, research gap identification, and method selection. However, ad hoc comparisons often miss important dimensions or apply inconsistent criteria across papers. This skill provides a systematic framework that ensures comprehensive, fair comparisons by defining comparison dimensions upfront and extracting standardized data from each paper.
+
+The approach works for comparing 2-10 papers and produces two types of output: a structured comparison matrix (tabular, suitable for inclusion in papers) and a narrative synthesis (prose, suitable for literature review sections). It handles comparisons across papers that address the same question with different methods, papers that use the same method on different data, and papers that reach conflicting conclusions about the same phenomenon.
+
+## Comparison Framework
+
+### Defining Comparison Dimensions
+
+Before reading the papers, define the dimensions you will compare:
+
+```python
+COMPARISON_DIMENSIONS = {
+    'basic': {
+        'title': str,
+        'authors': list,
+        'year': int,
+        'venue': str,
+        'citation_count': int,
+    },
+    'research_design': {
+        'research_question': str,
+        'hypothesis': str,
+        'study_type': str,  # RCT, observational, simulation, etc.
+        'theoretical_framework': str,
+    },
+    'methodology': {
+        'data_source': str,
+        'sample_size': str,
+        'sampling_method': str,
+        'variables': {
+            'independent': list,
+            'dependent': list,
+            'control': list,
+        },
+        'analysis_method': str,
+        'tools_used': list,
+    },
+    'results': {
+        'main_finding': str,
+        'effect_size': str,
+        'statistical_significance': str,
+        'secondary_findings': list,
+    },
+    'quality': {
+        'strengths': list,
+        'limitations': list,
+        'reproducibility': str,  # high, medium, low
+        'generalizability': str,
+    }
+}
+```
+
+### Extraction Protocol
+
+For each paper, systematically extract data for every dimension:
+
+```
+Paper A: Smith et al. (2024)
+  research_question: "Does intervention X improve outcome Y in population Z?"
+  study_type: "Randomized controlled trial"
+  sample_size: "n=200 (100 treatment, 100 control)"
+  analysis_method: "Mixed-effects logistic regression"
+  main_finding: "Intervention X improved Y by 23% (OR=1.45, 95% CI [1.12, 1.88])"
+  limitations: ["Single-site study", "Short follow-up (6 months)"]
+
+Paper B: Jones et al. (2025)
+  research_question: "What is the effect of intervention X on outcome Y?"
+  study_type: "Quasi-experimental (pre-post with control group)"
+  sample_size: "n=350 (200 treatment, 150 control)"
+  analysis_method: "Difference-in-differences regression"
+  main_finding: "Intervention X improved Y by 18% (beta=0.18, p<0.01)"
+  limitations: ["Non-random assignment", "Potential selection bias"]
+```
+
+## Comparison Matrix Generation
+
+### Tabular Comparison
+
+| Dimension | Smith et al. (2024) | Jones et al. (2025) | Chen et al. (2025) |
+|-----------|-------------------|-------------------|--------------------|
+| **Study type** | RCT | Quasi-experimental | Observational cohort |
+| **Sample size** | n=200 | n=350 | n=1,200 |
+| **Population** | University students | Working adults | General population |
+| **Intervention** | X (standardized) | X (adapted version) | X (self-selected) |
+| **Primary outcome** | Y (binary) | Y (continuous) | Y (composite score) |
+| **Main effect** | OR=1.45 | beta=0.18 | HR=1.32 |
+| **Significant?** | Yes (p=.003) | Yes (p<.01) | Yes (p=.02) |
+| **Follow-up** | 6 months | 12 months | 24 months |
+| **Key limitation** | Single-site | Non-random | Self-selection |
+| **Code available** | Yes (GitHub) | No | Data only (Zenodo) |
+
+### Automated Matrix Builder
+
+```python
+import pandas as pd
+
+def build_comparison_matrix(papers: list, dimensions: list) -> pd.DataFrame:
+    """
+    Build a comparison matrix from extracted paper data.
+
+    Args:
+        papers: List of dicts, each containing extracted dimensions
+        dimensions: List of dimension keys to include in the matrix
+    """
+    matrix = {}
+    for paper in papers:
+        label = f"{paper['authors'][0].split()[-1]} et al. ({paper['year']})"
+        matrix[label] = {}
+        for dim in dimensions:
+            value = paper.get(dim, 'Not reported')
+            if isinstance(value, list):
+                value = '; '.join(str(v) for v in value)
+            matrix[label][dim] = value
+
+    df = pd.DataFrame(matrix).T
+    return df
+```
+
+## Synthesis Narratives
+
+### Agreement Synthesis
+
+When papers agree on a finding:
+
+```
+Template:
+"Multiple studies converge on the finding that [finding]. [Author A] (year)
+demonstrated this using [method A] with [sample A], finding [specific result].
+This was corroborated by [Author B] (year) using [different method/sample],
+who reported [similar result]. The consistency across [different methods/
+populations/contexts] strengthens confidence in this finding."
+```
+
+### Contradiction Synthesis
+
+When papers disagree:
+
+```
+Template:
+"The evidence on [topic] is mixed. [Author A] (year) found [finding A]
+using [method], while [Author B] (year) reported [contradictory finding B].
+Several factors may explain this discrepancy: (1) [methodological difference],
+(2) [population difference], (3) [measurement difference]. Further research
+using [suggested approach] is needed to resolve this inconsistency."
+```
+
+### Complementary Synthesis
+
+When papers address different aspects of the same topic:
+
+```
+Template:
+"The studies contribute complementary evidence on [topic]. [Author A] (year)
+addressed [aspect 1] by [method], establishing that [finding]. Building on
+this, [Author B] (year) examined [aspect 2] and found [finding]. Together,
+these studies suggest that [integrated conclusion], though [gap] remains
+unaddressed."
+```
+
+## Advanced Comparison Techniques
+
+### Methodological Quality Comparison
+
+Apply a standardized quality assessment tool to all papers:
+
+| Quality Criterion | Paper A | Paper B | Paper C |
+|------------------|---------|---------|---------|
+| Clear research question | Yes | Yes | Partially |
+| Appropriate design | Yes | Mostly | Yes |
+| Adequate sample size | No (underpowered) | Yes | Yes |
+| Valid measurement | Yes | Yes | Questionable |
+| Controls for confounders | Yes | Partially | No |
+| Appropriate analysis | Yes | Yes | Yes |
+| Effect size reported | Yes | No | Yes |
+| Limitations discussed | Yes | Yes | Minimal |
+| **Overall quality** | **High** | **Medium** | **Medium** |
+
+### Visualizing Paper Relationships
+
+```
+Conceptual Map:
+
+  Smith (2024) ──agrees──→ Jones (2025)
+       │                        │
+       │                        │
+   extends                  contradicts
+       │                        │
+       ↓                        ↓
+  Lee (2023)              Chen (2025)
+       │
+   replicates
+       │
+       ↓
+  Park (2022) [foundational study]
+```
+
+## Output Formats
+
+The comparison can be output in several formats depending on the use case:
+
+1. **Comparison table**: For inclusion in a paper's related work section or supplementary materials.
+2. **Narrative synthesis**: For the body of a literature review chapter.
+3. **Gap analysis**: For identifying your own research contribution.
+4. **Presentation slide**: A single-slide summary for lab meetings or conference talks.
+5. **Decision matrix**: For choosing which method to adopt in your own research.
+
+## Best Practices
+
+- Define comparison dimensions before reading the papers to avoid post hoc cherry-picking.
+- Compare no more than 8-10 papers in a single matrix; beyond that, group papers by theme.
+- Always note when a dimension is "not reported" rather than leaving it blank.
+- Be fair: apply the same critical lens to all papers, including those you agree with.
+- When results conflict, investigate whether the conflict is real (different findings) or apparent (different operationalizations of the same concept).
+- Include both quantitative comparisons (effect sizes, sample sizes) and qualitative assessments (strengths, limitations).
+
+## References
+
+- Snyder, H. (2019). Literature Review as a Research Methodology. *Journal of Business Research*, 104, 333-339.
+- Templier, M. & Pare, G. (2015). A Framework for Guiding and Evaluating Literature Reviews. *Communications of the AIS*, 37, 112-137.
+- Webster, J. & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review. *MIS Quarterly*, 26(2), xiii-xxiii.