flonat-research 0.1.0

package/skills/literature/references/openalex-common-queries.md

@@ -0,0 +1,381 @@
+# Common OpenAlex Query Examples
+
+This document provides practical examples for common research queries using OpenAlex.
+
+## Finding Papers by Author
+
+**User query**: "Find papers by Albert Einstein"
+
+**Approach**: Two-step pattern
+1. Search for author to get ID
+2. Filter works by author ID
+
+**Python example**:
+```python
+from scripts.openalex_client import OpenAlexClient
+from scripts.query_helpers import find_author_works
+
+client = OpenAlexClient(email="your-email@example.edu")
+works = find_author_works("Albert Einstein", client, limit=100)
+
+for work in works:
+    print(f"{work['title']} ({work['publication_year']})")
+```
+
+## Finding Papers from an Institution
+
+**User query**: "What papers has MIT published in the last year?"
+
+**Approach**: Two-step pattern with date filter
+1. Search for institution to get ID
+2. Filter works by institution ID and year
+
+**Python example**:
+```python
+from scripts.query_helpers import find_institution_works
+
+works = find_institution_works("MIT", client, limit=200)
+
+# Filter for recent papers
+import datetime
+current_year = datetime.datetime.now().year
+recent_works = [w for w in works if w['publication_year'] == current_year]
+```
+
+## Highly Cited Papers on a Topic
+
+**User query**: "Find the most cited papers on CRISPR from the last 5 years"
+
+**Approach**: Search + filter + sort
+
+**Python example**:
+```python
+works = client.search_works(
+    search="CRISPR",
+    filter_params={
+        "publication_year": ">2019"
+    },
+    sort="cited_by_count:desc",
+    per_page=100
+)
+
+for work in works['results']:
+    title = work['title']
+    citations = work['cited_by_count']
+    year = work['publication_year']
+    print(f"{title} ({year}): {citations} citations")
+```
+
+## Open Access Papers on a Topic
+
+**User query**: "Find open access papers about climate change"
+
+**Approach**: Search + OA filter
+
+**Python example**:
+```python
+from scripts.query_helpers import get_open_access_papers
+
+papers = get_open_access_papers(
+    search_term="climate change",
+    client=client,
+    oa_status="any",  # or "gold", "green", "hybrid", "bronze"
+    limit=200
+)
+
+for paper in papers:
+    print(f"{paper['title']}")
+    print(f"  OA Status: {paper['open_access']['oa_status']}")
+    print(f"  URL: {paper['open_access']['oa_url']}")
+```
+
+## Publication Trends Analysis
+
+**User query**: "Show me publication trends for machine learning over the years"
+
+**Approach**: Use group_by to aggregate by year
+
+**Python example**:
+```python
+from scripts.query_helpers import get_publication_trends
+
+trends = get_publication_trends(
+    search_term="machine learning",
+    client=client
+)
+
+# Sort by year
+trends_sorted = sorted(trends, key=lambda x: x['key'])
+
+for trend in trends_sorted[-10:]:  # Last 10 years
+    year = trend['key']
+    count = trend['count']
+    print(f"{year}: {count} publications")
+```
+
+## Analyzing Research Output
+
+**User query**: "Analyze the research output of Stanford University from 2020-2024"
+
+**Approach**: Multiple aggregations for comprehensive analysis
+
+**Python example**:
+```python
+from scripts.query_helpers import analyze_research_output
+
+analysis = analyze_research_output(
+    entity_type='institution',
+    entity_name='Stanford University',
+    client=client,
+    years='2020-2024'
+)
+
+print(f"Institution: {analysis['entity_name']}")
+print(f"Total works: {analysis['total_works']}")
+print(f"Open access: {analysis['open_access_percentage']}%")
+print("\nTop topics:")
+for topic in analysis['top_topics'][:5]:
+    print(f"  - {topic['key_display_name']}: {topic['count']} works")
+```
+
+## Finding Papers by DOI (Batch)
+
+**User query**: "Get information for these 10 DOIs: ..."
+
+**Approach**: Batch lookup with pipe separator
+
+**Python example**:
+```python
+dois = [
+    "https://doi.org/10.1371/journal.pone.0266781",
+    "https://doi.org/10.1371/journal.pone.0267149",
+    "https://doi.org/10.1038/s41586-021-03819-2",
+    # ... up to 50 DOIs
+]
+
+works = client.batch_lookup(
+    entity_type='works',
+    ids=dois,
+    id_field='doi'
+)
+
+for work in works:
+    print(f"{work['title']} - {work['publication_year']}")
+```
+
+## Random Sample of Papers
+
+**User query**: "Give me 50 random papers from 2023"
+
+**Approach**: Use sample parameter with seed for reproducibility
+
+**Python example**:
+```python
+works = client.sample_works(
+    sample_size=50,
+    seed=42,  # For reproducibility
+    filter_params={
+        "publication_year": "2023",
+        "is_oa": "true"
+    }
+)
+
+print(f"Got {len(works)} random papers from 2023")
+```
+
+## Papers from Multiple Institutions
+
+**User query**: "Find papers with authors from both MIT and Stanford"
+
+**Approach**: Use + operator for AND within same attribute
+
+**Python example**:
+```python
+# First, get institution IDs
+mit_response = client._make_request(
+    '/institutions',
+    params={'search': 'MIT', 'per-page': 1}
+)
+mit_id = mit_response['results'][0]['id'].split('/')[-1]
+
+stanford_response = client._make_request(
+    '/institutions',
+    params={'search': 'Stanford', 'per-page': 1}
+)
+stanford_id = stanford_response['results'][0]['id'].split('/')[-1]
+
+# Find works with authors from both institutions
+works = client.search_works(
+    filter_params={
+        "authorships.institutions.id": f"{mit_id}+{stanford_id}"
+    },
+    per_page=100
+)
+
+print(f"Found {works['meta']['count']} collaborative papers")
+```
+
+## Papers in a Specific Journal
+
+**User query**: "Get all papers from Nature published in 2023"
+
+**Approach**: Two-step - find journal ID, then filter works
+
+**Python example**:
+```python
+# Step 1: Find journal source ID
+source_response = client._make_request(
+    '/sources',
+    params={'search': 'Nature', 'per-page': 1}
+)
+source = source_response['results'][0]
+source_id = source['id'].split('/')[-1]
+
+print(f"Found journal: {source['display_name']} (ID: {source_id})")
+
+# Step 2: Get works from that source
+works = client.search_works(
+    filter_params={
+        "primary_location.source.id": source_id,
+        "publication_year": "2023"
+    },
+    per_page=200
+)
+
+print(f"Found {works['meta']['count']} papers from Nature in 2023")
+```
+
+## Topic Analysis by Institution
+
+**User query**: "What topics does MIT research most?"
+
+**Approach**: Filter by institution, group by topics
+
+**Python example**:
+```python
+# Get MIT ID
+inst_response = client._make_request(
+    '/institutions',
+    params={'search': 'MIT', 'per-page': 1}
+)
+mit_id = inst_response['results'][0]['id'].split('/')[-1]
+
+# Group by topics
+topics = client.group_by(
+    entity_type='works',
+    group_field='topics.id',
+    filter_params={
+        "authorships.institutions.id": mit_id,
+        "publication_year": ">2020"
+    }
+)
+
+print("Top research topics at MIT (2020+):")
+for i, topic in enumerate(topics[:10], 1):
+    print(f"{i}. {topic['key_display_name']}: {topic['count']} works")
+```
+
+## Citation Analysis
+
+**User query**: "Find papers that cite this specific DOI"
+
+**Approach**: Get work by DOI, then use cited_by_api_url
+
+**Python example**:
+```python
+# Get the work
+doi = "https://doi.org/10.1038/s41586-021-03819-2"
+work = client.get_entity('works', doi)
+
+# Get papers that cite it
+cited_by_url = work['cited_by_api_url']
+
+# Extract just the query part and use it
+import requests
+response = requests.get(cited_by_url, params={'mailto': client.email})
+citing_works = response.json()
+
+print(f"{work['title']}")
+print(f"Total citations: {work['cited_by_count']}")
+print(f"\nRecent citing papers:")
+for citing_work in citing_works['results'][:5]:
+    print(f"  - {citing_work['title']} ({citing_work['publication_year']})")
+```
+
+## Large-Scale Data Extraction
+
+**User query**: "Get all papers on quantum computing from the last 3 years"
+
+**Approach**: Paginate through all results
+
+**Python example**:
+```python
+all_papers = client.paginate_all(
+    endpoint='/works',
+    params={
+        'search': 'quantum computing',
+        'filter': 'publication_year:2022-2024'
+    },
+    max_results=10000  # Limit to prevent excessive API calls
+)
+
+print(f"Retrieved {len(all_papers)} papers")
+
+# Save to CSV
+import csv
+with open('quantum_papers.csv', 'w', newline='') as f:
+    writer = csv.writer(f)
+    writer.writerow(['Title', 'Year', 'Citations', 'DOI', 'OA Status'])
+
+    for paper in all_papers:
+        writer.writerow([
+            paper['title'],
+            paper['publication_year'],
+            paper['cited_by_count'],
+            paper.get('doi', 'N/A'),
+            paper['open_access']['oa_status']
+        ])
+```
+
+## Complex Multi-Filter Query
+
+**User query**: "Find recent, highly-cited, open access papers on AI from top institutions"
+
+**Approach**: Combine multiple filters
+
+**Python example**:
+```python
+# Get IDs for top institutions
+top_institutions = ['MIT', 'Stanford', 'Oxford']
+inst_ids = []
+
+for inst_name in top_institutions:
+    response = client._make_request(
+        '/institutions',
+        params={'search': inst_name, 'per-page': 1}
+    )
+    if response['results']:
+        inst_id = response['results'][0]['id'].split('/')[-1]
+        inst_ids.append(inst_id)
+
+# Combine with pipe for OR
+inst_filter = '|'.join(inst_ids)
+
+# Complex query
+works = client.search_works(
+    search="artificial intelligence",
+    filter_params={
+        "publication_year": ">2022",
+        "cited_by_count": ">50",
+        "is_oa": "true",
+        "authorships.institutions.id": inst_filter
+    },
+    sort="cited_by_count:desc",
+    per_page=200
+)
+
+print(f"Found {works['meta']['count']} papers matching criteria")
+for work in works['results'][:10]:
+    print(f"{work['title']}")
+    print(f"  Citations: {work['cited_by_count']}, Year: {work['publication_year']}")
+```

package/skills/literature/references/openalex-workflows.md

@@ -0,0 +1,248 @@
+# OpenAlex Structured Query Workflows
+
+> Extracted from the former `/openalex-search` skill. These workflows use the shared Python client in `.scripts/openalex/openalex_client.py` and helpers in `.scripts/openalex/query_helpers.py`.
+
+## Setup
+
+```python
+import sys
+sys.path.insert(0, ".scripts/openalex")
+from openalex_client import OpenAlexClient
+from query_helpers import find_highly_cited_recent_papers, get_open_access_papers, \
+    find_author_works, find_institution_works, analyze_research_output, get_publication_trends
+
+client = OpenAlexClient(email="user@example.edu")
+```
+
+Install dependency: `uv pip install requests`
+
+---
+
+## When to Use OpenAlex Queries
+
+| Scenario | OpenAlex workflow | Full `/literature` instead? |
+|----------|-------------------|----------------------------|
+| Find highly-cited papers on a topic | Workflow 1 | No — quick structured query |
+| Build a full literature review with narrative | — | Yes — use full 7-phase pipeline |
+| Citation count / trend analysis | Workflow 4 | No — API-only |
+| Verify a specific citation exists | Workflow 7 | Or Phase 4 of `/literature` |
+| Author or institution research output | Workflows 2–3 | No — API-only |
+| Find open-access versions of papers | Workflow 5 | No — API-only |
+| Comprehensive lit search + .bib assembly | — | Yes — calls OpenAlex as Phase 2 agent |
+| Bibliometric analysis for a paper | Workflow 6 | No — API-only |
+
+**Rule of thumb:** Use these workflows directly when you need *structured data* (counts, rankings, trends, metadata). Use the full `/literature` pipeline when you need a *complete workflow* (search, verify, download, synthesize).
+
+---
+
+## Core Workflows
+
+### 1. Find Highly-Cited Papers on a Topic
+
+```python
+papers = find_highly_cited_recent_papers(
+    topic="multi-criteria decision making",
+    years=">2018",
+    client=client,
+    limit=50
+)
+
+for p in papers[:10]:
+    print(f"[{p['cited_by_count']}] {p['title']} ({p['publication_year']})")
+```
+
+### 2. Author Publication Record
+
+```python
+works = find_author_works("Salvatore Greco", client=client, limit=100)
+```
+
+Uses the two-step pattern: search author -> get ID -> filter works by ID.
+
+### 3. Institution Research Output
+
+```python
+analysis = analyze_research_output(
+    entity_type='institution',
+    entity_name='University of Example',
+    client=client,
+    years='>2020'
+)
+
+print(f"Total works: {analysis['total_works']}")
+print(f"Open access: {analysis['open_access_percentage']}%")
+```
+
+### 4. Publication Trends
+
+```python
+trends = get_publication_trends(
+    search_term="human-AI collaboration",
+    client=client
+)
+
+for t in sorted(trends, key=lambda x: x['key'])[-10:]:
+    print(f"{t['key']}: {t['count']} publications")
+```
+
+### 5. Open-Access Paper Discovery
+
+```python
+papers = get_open_access_papers(
+    search_term="staggered difference-in-differences",
+    client=client,
+    oa_status="any",
+    limit=50
+)
+```
+
+### 6. Citation Network (Who Cites This Paper?)
+
+```python
+import requests
+
+work = client.get_entity('works', 'https://doi.org/10.1016/j.ejor.2023.01.001')
+citing = requests.get(
+    work['cited_by_api_url'],
+    params={'mailto': client.email, 'per-page': 200}
+).json()['results']
+```
+
+### 7. Batch DOI Lookup
+
+For verifying multiple papers at once — useful for feeding into `/bib-validate`:
+
+```python
+dois = [
+    "https://doi.org/10.1016/j.ejor.2023.01.001",
+    "https://doi.org/10.1287/mnsc.2022.4321",
+]
+
+works = client.batch_lookup('works', dois, 'doi')
+```
+
+---
+
+## Key API Patterns
+
+### Filters
+
+```python
+# Single year
+filter_params={"publication_year": "2023"}
+
+# Range
+filter_params={"publication_year": "2020-2025"}
+
+# Multiple conditions (AND)
+filter_params={"publication_year": ">2020", "is_oa": "true", "cited_by_count": ">50"}
+
+# Multiple values (OR)
+filter_params={"authorships.institutions.id": "I123|I456"}
+
+# Collaboration (AND within attribute)
+filter_params={"authorships.institutions.id": "I123+I456"}
+
+# Negation
+filter_params={"type": "!paratext"}
+```
+
+### Two-Step Pattern (Always for Entity Lookups)
+
+Never filter by entity names directly — always get the ID first:
+
+```python
+# Step 1: Search -> get ID
+author_resp = client._make_request('/authors', params={'search': 'Name', 'per-page': 1})
+author_id = author_resp['results'][0]['id'].split('/')[-1]
+
+# Step 2: Filter by ID
+works = client.search_works(filter_params={"authorships.author.id": author_id})
+```
+
+### External IDs
+
+```python
+# DOI
+client.get_entity('works', 'https://doi.org/10.1016/j.ejor.2023.01.001')
+
+# ORCID
+client.get_entity('authors', 'https://orcid.org/0000-0003-1613-5981')
+
+# ROR (institution)
+client.get_entity('institutions', 'https://ror.org/02y3ad647')
+
+# ISSN (journal)
+client.get_entity('sources', 'issn:0377-2217')  # [Journal]
+```
+
+---
+
+## Output Formats
+
+When presenting results, format as a markdown table:
+
+```markdown
+| # | Title | Authors | Year | Journal | Citations | DOI |
+|---|-------|---------|------|---------|-----------|-----|
+| 1 | ... | ... | 2024 | [Journal] | 42 | 10.1016/... |
+```
+
+For BibTeX export (to feed into project `.bib`):
+
+```python
+for work in papers:
+    authors = " and ".join(
+        a['author']['display_name']
+        for a in work.get('authorships', [])
+    )
+    last_name = work['authorships'][0]['author']['display_name'].split()[-1]
+    key = last_name + str(work['publication_year'])
+    print(f"@article{{{key},")
+    print(f"  author = {{{authors}}},")
+    print(f"  title = {{{work['title']}}},")
+    print(f"  year = {{{work['publication_year']}}},")
+    if work.get('doi'):
+        print(f"  doi = {{{work['doi'].replace('https://doi.org/', '')}}},")
+    print("}")
+```
+
+---
+
+## Rate Limits
+
+| Pool | Rate | How to access |
+|------|------|---------------|
+| Default | 1 req/sec, 100k/day | No email |
+| Polite | 10 req/sec, 100k/day | Pass email to client |
+
+Always use the polite pool.
+
+---
+
+## MCP Server (Preferred Over Python Client)
+
+**Always prefer MCP tools when available.** The bibliography MCP server (`packages/mcp-bibliography/`) exposes all OpenAlex functionality plus cross-source search — no Python boilerplate needed.
+
+| Task | MCP tool (preferred) | Python client (fallback) |
+|------|---------------------|-------------------------|
+| Search papers by topic | `scholarly_search` (all sources) | `find_highly_cited_recent_papers()` |
+| Look up a DOI | `openalex_lookup_doi` | `client.get_entity()` |
+| Batch DOI verification | `scholarly_verify_dois` | Manual loop |
+| Find similar papers | `scholarly_similar_works` | N/A |
+| Author publications | `openalex_author_works` | `find_author_works()` |
+| Author profile | `openalex_author_profile` | `analyze_research_output()` |
+| Institution output | `openalex_institution_output` | `analyze_research_output()` |
+| Publication trends | `openalex_trends` | `get_publication_trends()` |
+| Forward citations | `openalex_citing_works` | Manual `cited_by_api_url` fetch |
+| Check active sources | `scholarly_source_status` | N/A |
+
+Use the Python client below only for workflows not yet exposed via MCP (custom filter combinations, batch entity lookups by non-DOI IDs). See [`docs/mcp-servers.md`](../../../docs/mcp-servers.md) for full MCP documentation.
+
+---
+
+## Complementary Skills
+
+- **`/literature`** (parent skill) — OpenAlex serves as Phase 2 Agent 3 alongside Google Scholar and Semantic Scholar. Its API returns structured metadata that web scraping often misses.
+- **`/bib-validate`** — Use batch DOI lookup (Workflow 7) to verify metadata for flagged entries.
+- **`/split-pdf`** — Use OA discovery (Workflow 5) to find downloadable PDFs before split-reading.

package/skills/literature/references/reference-manager-sync.md

@@ -0,0 +1,36 @@
+# Reference Manager Sync — Phase 6c Details
+
+> Referenced from: `literature/SKILL.md` Phase 6c
+
+After assembling and validating the `.bib`, sync new references to the user's reference libraries. Follow the filing sequence defined in [`../shared/reference-resolution.md`](../../shared/reference-resolution.md).
+
+## Zotero (Active Write Target)
+
+For each new entry not marked **ALREADY IN ZOTERO** from Phase 1:
+
+1. **Add to Zotero** — call `mcp__refpile__add_item` with full metadata (title, authors, year, journal, DOI, itemType).
+2. **File into topic collection** — resolve the topic collection key (from Phase 1 step 6) and call `mcp__refpile__add_to_collection(collection_key=<topic_key>)`.
+3. **Tag for review** — also call `mcp__refpile__add_to_collection` with the `_Needs Review` collection key. Items go into both the topic collection and `_Needs Review`.
+4. **Report results** — show a summary table of what was added and which collections they were filed into.
+
+## Paperpile (Read-Only Cross-Reference)
+
+For each new entry not already in Paperpile:
+
+1. **Report additions needed** — list entries that need manual import into Paperpile (Paperpile MCP is read-only; the user adds via the Paperpile app or browser extension).
+2. **Export BibTeX** — if Paperpile has entries with better metadata than what was assembled, call `mcp__paperpile__export_bib` and use those entries instead.
+
+## Migration Candidates
+
+For entries found in Paperpile but not in Zotero (status `MIGRATE_TO_ZOTERO` from Phase 1):
+
+1. **Auto-add to Zotero** using Paperpile metadata.
+2. **File into topic collection + `_Needs Review`** per the filing sequence.
+3. **Report** the migration count.
+
+## Post-Run Maintenance
+
+1. **Update `zotero-collections.md`** — increment item counts for affected collections.
+2. **Report Paperpile gaps** — items the user should manually add to Paperpile.
+
+**Graceful degradation:** If either MCP is unavailable, skip that source with a warning. The `.bib` file on disk is still the primary output.