@pikaa-ai/pikaa 0.3.22 → 0.3.24

package/skills/liteparse/SKILL.md

@@ -1,295 +0,0 @@
----
-name: liteparse
-description: Local document and PDF parsing that returns spatial text with bounding boxes. Use for extracting text from PDFs, DOCX, Office files, and images; running OCR on scans; producing layout-preserved JSON for RAG; batch-ingesting folders of papers; or rendering pages to PNG for multimodal agents. Distinguishing capabilities are per-token bounding boxes, page raster output, and fully local processing with no cloud API.
-license: Apache-2.0
-allowed-tools: Read Write Edit Bash
-compatibility: Python 3.10+. Optional LibreOffice (Office formats) and ImageMagick (images). Bundled Tesseract for OCR. All processing is local — no cloud API required.
-metadata:
-  version: "1.1"
-  skill-author: K-Dense Inc.
----
-
-# LiteParse — Local Document Parsing
-
-## Overview
-
-LiteParse is a fast, open-source document parser (Rust core, Python/Node bindings) focused on **local, layout-aware text extraction** with bounding boxes. It does not produce Markdown and does not call cloud LLMs. Outputs are **plain text** (layout-preserved) or **structured JSON** with per-page `text_items` (position, font metadata, optional confidence).
-
-**Version note:** Examples target **liteparse 2.0.0** (PyPI, May 2026). The upstream V1 branch is legacy; this skill documents **V2 / main** only.
-
-For parser selection vs MarkItDown, the `pdf` skill, or LlamaParse, see `references/choosing_a_parser.md`.
-
-## When to Use This Skill
-
-Use LiteParse when you need:
-
-- **Fast local parsing** of PDFs or converted Office/image files without cloud dependencies
-- **Spatial text** with bounding boxes for layout-aware RAG, citation grounding, or figure/table region logic
-- **OCR** on scanned PDFs or images (bundled Tesseract, or a user-run HTTP OCR server)
-- **Page screenshots** (PNG) for multimodal agents that must see charts, figures, or handwriting
-- **Batch ingestion** of literature folders, supplementary PDFs, or protocol libraries
-- **Page subsets** or **password-protected** PDFs
-
-## When Not to Use
-
-| Task | Use instead |
-|------|-------------|
-| Markdown for LLM ingestion (EPUB, audio, YouTube, HTML) | `markitdown` skill |
-| Merge/split PDFs, forms, watermarks, rotation | `pdf` skill |
-| Dense tables, handwriting, production cloud pipelines | [LlamaParse](https://docs.cloud.llamaindex.ai/llamaparse/overview) (cloud; sign up separately) |
-
-## Installation
-
-```bash
-uv pip install "liteparse==2.0.0"
-```
-
-This installs the Python bindings and the **`lit`** CLI. Verify:
-
-```bash
-lit --help
-python -c "import liteparse; print(liteparse.__version__)"
-```
-
-**Optional system tools** (for non-PDF inputs):
-
-- **LibreOffice** — Word, Excel, PowerPoint, OpenDocument, CSV/TSV
-- **ImageMagick** — PNG, JPEG, TIFF, WebP, SVG, etc.
-
-Install commands are in `references/ocr_and_formats.md`.
-
-**Node.js / TypeScript** (optional): `npm i @llamaindex/liteparse` — see `references/api_reference.md`.
-
----
-
-## Quick Start
-
-### Python
-
-```python
-from liteparse import LiteParse
-
-parser = LiteParse(quiet=True)
-result = parser.parse("paper.pdf")
-print(result.text)
-
-for page in result.pages:
-    print(f"Page {page.page_num}: {len(page.text_items)} items")
-```
-
-### CLI
-
-```bash
-# Layout-preserved text (default)
-lit parse paper.pdf
-
-# Structured JSON with bounding boxes
-lit parse paper.pdf --format json -o paper.json
-
-# Disable OCR on text-native PDFs (faster)
-lit parse paper.pdf --no-ocr
-```
-
----
-
-## Core Workflows
-
-### 1. Parse to layout-preserved text
-
-Best for quick full-document text or feeding chunkers that do not need coordinates.
-
-```python
-parser = LiteParse(ocr_enabled=True, quiet=True)
-result = parser.parse("document.pdf")
-full_text = result.text
-```
-
-```bash
-lit parse document.pdf -o output.txt
-```
-
-### 2. Parse to structured JSON (bounding boxes)
-
-Use when building layout-aware RAG, highlighting source regions, or joining text with screenshots.
-
-```python
-import json
-from liteparse import LiteParse
-
-parser = LiteParse(output_format="json", quiet=True)
-result = parser.parse("document.pdf")
-
-# Programmatic access
-for page in result.pages:
-    for item in page.text_items:
-        bbox = (item.x, item.y, item.width, item.height)
-        # item.text, item.confidence, item.font_name, item.font_size
-```
-
-```bash
-lit parse document.pdf --format json -o document.json
-```
-
-JSON field layout: `references/output_formats.md`.
-
-### 3. Parse specific pages
-
-```python
-parser = LiteParse(target_pages="1-5,10,15-20", quiet=True)
-result = parser.parse("long_paper.pdf")
-```
-
-```bash
-lit parse long_paper.pdf --target-pages "1-5,10"
-```
-
-### 4. Parse from bytes or stdin
-
-Useful for uploads, S3 downloads, or piping remote PDFs.
-
-```python
-with open("document.pdf", "rb") as f:
-    result = parser.parse(f.read())
-```
-
-```bash
-curl -sL https://example.com/report.pdf | lit parse -
-```
-
-### 5. Page screenshots for multimodal agents
-
-Screenshots capture visual content that text extraction alone misses (figures, complex tables, handwriting).
-
-```python
-from pathlib import Path
-
-parser = LiteParse(dpi=150, quiet=True)
-shots = parser.screenshot("document.pdf", page_numbers=[1, 2, 3])
-out = Path("screenshots")
-out.mkdir(exist_ok=True)
-for s in shots:
-    (out / f"page_{s.page_num}.png").write_bytes(s.image_bytes)
-```
-
-```bash
-lit screenshot document.pdf --target-pages "1,3,5" -o ./screenshots
-lit screenshot document.pdf --dpi 300 -o ./screenshots
-```
-
-Combine **JSON parse + screenshots** when an agent needs both coordinates and pixels for the same pages.
-
-### 6. Batch-parse a directory
-
-For large corpora, prefer the CLI (parallel OCR workers) or the bundled script.
-
-```bash
-lit batch-parse ./papers ./parsed --format json --recursive
-lit batch-parse ./papers ./parsed --extension .pdf --no-ocr
-```
-
-```bash
-python scripts/batch_parse_dir.py ./papers ./parsed --format json --recursive
-```
-
-See `scripts/batch_parse_dir.py` for a Python batch wrapper without network calls.
-
-### 7. OCR configuration
-
-OCR is **on by default**. Tesseract is bundled; no extra install for basic English OCR.
-
-```python
-parser = LiteParse(
-    ocr_enabled=True,
-    ocr_language="eng",       # Tesseract codes: fra, deu, etc.
-    num_workers=4,            # parallel OCR (default: CPU cores - 1)
-    dpi=150,                  # higher DPI → better OCR, slower
-)
-```
-
-```bash
-lit parse scan.pdf --ocr-language fra
-lit parse scan.pdf --no-ocr
-lit parse scan.pdf --ocr-server-url http://localhost:8080/ocr
-```
-
-**Offline / air-gapped:** set `TESSDATA_PREFIX` to a directory of `.traineddata` files, or pass `--tessdata-path`. Details: `references/ocr_and_formats.md`.
-
-### 8. Encrypted PDFs
-
-```python
-parser = LiteParse(password="secret", quiet=True)
-result = parser.parse("protected.pdf")
-```
-
-```bash
-lit parse protected.pdf --password secret
-```
-
-### 9. Search text items by phrase
-
-Merge adjacent items and return combined bounding boxes for a phrase (e.g. section titles).
-
-```python
-from liteparse import search_items
-
-page = result.get_page(1)
-matches = search_items(page.text_items, "Materials and Methods", case_sensitive=False)
-```
-
----
-
-## Multi-Format Inputs
-
-| Category | Extensions (examples) | Requirement |
-|----------|----------------------|-------------|
-| PDF | `.pdf` | Native |
-| Office | `.docx`, `.xlsx`, `.pptx`, `.doc`, `.odt`, … | LibreOffice |
-| Images | `.png`, `.jpg`, `.tiff`, `.webp`, `.svg`, … | ImageMagick |
-
-Files are converted to PDF internally, then parsed. If conversion tools are missing, parsing fails with an actionable error — install the dependency and retry.
-
----
-
-## Performance Tips
-
-- **`--no-ocr`** on born-digital PDFs — largest speedup
-- **`target_pages`** — parse only methods/supplement sections
-- **`num_workers`** — scale OCR across CPU cores
-- **`max_pages`** — cap very large files (default 1000)
-- **`lit batch-parse`** — directory-scale jobs with `--recursive` and `--extension`
-- Lower **`dpi`** (e.g. 100) when OCR quality is already sufficient
-
----
-
-## Reference Files
-
-| File | Read when |
-|------|-----------|
-| `references/choosing_a_parser.md` | Unsure whether to use LiteParse, MarkItDown, pdf, or LlamaParse |
-| `references/api_reference.md` | Python/TypeScript API, types, `search_items` |
-| `references/cli_reference.md` | Full `lit` command flags |
-| `references/output_formats.md` | JSON schema, bboxes, confidence scores |
-| `references/ocr_and_formats.md` | Tesseract, HTTP OCR, LibreOffice, ImageMagick |
-
----
-
-## Troubleshooting
-
-| Issue | Fix |
-|-------|-----|
-| Office file fails | Install LibreOffice; ensure `soffice` is on PATH (Windows: add LibreOffice `program` dir) |
-| Image fails | Install ImageMagick; verify `convert` or `magick` works |
-| OCR poor quality | Increase `--dpi`; try `--ocr-language`; or HTTP OCR server |
-| OCR slow | `--no-ocr` if not needed; reduce pages; increase `num_workers` |
-| Air-gapped OCR | `export TESSDATA_PREFIX=/path/to/tessdata` or `--tessdata-path` |
-| `ParseError` on bytes | Ensure input is valid PDF bytes (Office bytes need a file path + conversion) |
-
----
-
-## Resources
-
-- **GitHub**: https://github.com/run-llama/liteparse
-- **Docs**: https://developers.llamaindex.ai/liteparse/
-- **PyPI**: https://pypi.org/project/liteparse/2.0.0/
-- **npm**: https://www.npmjs.com/package/@llamaindex/liteparse
-- **OCR API spec**: https://github.com/run-llama/liteparse/blob/main/OCR_API_SPEC.md

package/skills/literature-review/SKILL.md

@@ -1,263 +0,0 @@
----
-name: literature-review
-description: Conduct comprehensive, systematic literature reviews using multiple academic databases (PubMed, arXiv, bioRxiv, Semantic Scholar, etc.). This skill should be used when conducting systematic literature reviews, meta-analyses, research synthesis, or comprehensive literature searches across biomedical, scientific, and technical domains. Creates professionally formatted markdown documents and PDFs with verified citations in multiple citation styles (APA, Nature, Vancouver, etc.).
-allowed-tools: Read Write Edit Bash
-license: MIT license
-metadata:
-  version: "1.7"
-  skill-author: K-Dense Inc.
-  openclaw:
-    primaryEnv: OPENROUTER_API_KEY
-    envVars:
-    - name: OPENROUTER_API_KEY
-      required: false
-      description: OpenRouter API key for the skill's LLM-powered steps.
----
-
-# Literature Review
-
-## Overview
-
-Conduct systematic, comprehensive literature reviews following rigorous academic methodology. Search multiple literature databases, synthesize findings thematically, verify all citations for accuracy, and generate professional output documents in markdown and PDF formats.
-
-This skill uses the **parallel-web skill** (`parallel-cli search`) as the primary web search tool for broad academic literature discovery, supplemented by specialized database access skills (gget, bioservices, datacommons-client). It provides specialized tools for citation verification, result aggregation, and document generation.
-
-## When to Use This Skill
-
-Use this skill when:
-- Conducting a systematic literature review for research or publication
-- Synthesizing current knowledge on a specific topic across multiple sources
-- Performing meta-analysis or scoping reviews
-- Writing the literature review section of a research paper or thesis
-- Investigating the state of the art in a research domain
-- Identifying research gaps and future directions
-- Requiring verified citations and professional formatting
-
-## Visual Enhancement with Scientific Schematics
-
-**⚠️ MANDATORY: Every literature review MUST include at least 1-2 AI-generated figures using the scientific-schematics skill.**
-
-This is not optional. Literature reviews without visual elements are incomplete. Before finalizing any document:
-1. Generate at minimum ONE schematic or diagram (e.g., PRISMA flow diagram for systematic reviews)
-2. Prefer 2-3 figures for comprehensive reviews (search strategy flowchart, thematic synthesis diagram, conceptual framework)
-
-**How to generate figures:**
-- Use the **scientific-schematics** skill to generate AI-powered publication-quality diagrams
-- Simply describe your desired diagram in natural language
-- Nano Banana Pro will automatically generate, review, and refine the schematic
-
-**How to generate schematics:**
-```bash
-python scripts/generate_schematic.py "your diagram description" -o figures/output.png
-```
-
-The AI will automatically:
-- Create publication-quality images with proper formatting
-- Review and refine through multiple iterations
-- Ensure accessibility (colorblind-friendly, high contrast)
-- Save outputs in the figures/ directory
-
-**When to add schematics:**
-- PRISMA flow diagrams for systematic reviews
-- Literature search strategy flowcharts
-- Thematic synthesis diagrams
-- Research gap visualization maps
-- Citation network diagrams
-- Conceptual framework illustrations
-- Any complex concept that benefits from visualization
-
-For detailed guidance on creating schematics, refer to the scientific-schematics skill documentation.
-
----
-
-## Core Workflow
-
-A literature review runs in seven phases, documented in full with commands and templates
-in [references/core_workflow.md](references/core_workflow.md):
-
-1. **Planning and scoping** — the question, inclusion and exclusion criteria, and scope.
-2. **Systematic literature search** — multi-database searching with recorded queries.
-3. **Screening and selection** — title/abstract then full-text screening with counts kept
-   for the PRISMA flow.
-4. **Data extraction and quality assessment** — structured extraction and risk-of-bias
-   or quality appraisal.
-5. **Synthesis and analysis** — thematic or quantitative synthesis across studies.
-6. **Citation verification** — every citation checked against the actual source.
-7. **Document generation** — assembling the review with a complete bibliography.
-
-Record every search string and date as you go: a review that cannot reproduce its own
-search is not systematic. Per-database search guidance and citation styles are in
-[references/search_and_citation.md](references/search_and_citation.md), and a full worked
-review is in [references/example_workflow.md](references/example_workflow.md).
-
-## Best Practices
-
-### Search Strategy
-1. **Start with parallel-web**: Use `parallel-cli search` with academic domains for initial broad coverage before querying specialized databases
-2. **Use multiple databases** (minimum 3): Ensures comprehensive coverage — parallel-web counts as one source
-3. **Include preprint servers**: Captures latest unpublished findings
-4. **Document everything**: Search strings, dates, result counts for reproducibility — save all parallel-cli output to `sources/`
-5. **Test and refine**: Run pilot searches, review results, adjust search terms
-6. **Sort by citations**: When available, sort search results by citation count to surface influential work first
-7. **Use parallel-cli extract**: Fetch full content from promising URLs found during search to verify relevance before full-text screening
-
-### Screening and Selection
-1. **Use multiple databases** (minimum 3): Ensures comprehensive coverage
-2. **Include preprint servers**: Captures latest unpublished findings
-3. **Document everything**: Search strings, dates, result counts for reproducibility
-4. **Test and refine**: Run pilot searches, review results, adjust search terms
-
-### Screening and Selection
-1. **Use clear criteria**: Document inclusion/exclusion criteria before screening
-2. **Screen systematically**: Title → Abstract → Full text
-3. **Document exclusions**: Record reasons for excluding studies
-4. **Consider dual screening**: For systematic reviews, have two reviewers screen independently
-
-### Synthesis
-1. **Organize thematically**: Group by themes, NOT by individual studies
-2. **Synthesize across studies**: Compare, contrast, identify patterns
-3. **Be critical**: Evaluate quality and consistency of evidence
-4. **Identify gaps**: Note what's missing or understudied
-
-### Quality and Reproducibility
-1. **Assess study quality**: Use appropriate quality assessment tools
-2. **Verify all citations**: Run verify_citations.py script
-3. **Document methodology**: Provide enough detail for others to reproduce
-4. **Follow guidelines**: Use PRISMA for systematic reviews
-
-### Writing
-1. **Be objective**: Present evidence fairly, acknowledge limitations
-2. **Be systematic**: Follow structured template
-3. **Be specific**: Include numbers, statistics, effect sizes where available
-4. **Be clear**: Use clear headings, logical flow, thematic organization
-
-## Common Pitfalls to Avoid
-
-1. **Single database search**: Misses relevant papers; always search multiple databases
-2. **No search documentation**: Makes review irreproducible; document all searches
-3. **Study-by-study summary**: Lacks synthesis; organize thematically instead
-4. **Unverified citations**: Leads to errors; always run verify_citations.py
-5. **Too broad search**: Yields thousands of irrelevant results; refine with specific terms
-6. **Too narrow search**: Misses relevant papers; include synonyms and related terms
-7. **Ignoring preprints**: Misses latest findings; include bioRxiv, medRxiv, arXiv
-8. **No quality assessment**: Treats all evidence equally; assess and report quality
-9. **Publication bias**: Only positive results published; note potential bias
-10. **Outdated search**: Field evolves rapidly; clearly state search date
-
-## Integration with Other Skills
-
-This skill works seamlessly with other scientific skills:
-
-### Web Search & Extraction (parallel-web skill — PRIMARY)
-- **parallel-cli search**: Broad academic and general web search with domain filtering — use for initial scoping, finding papers, citation chaining, and supplementary searches
-- **parallel-cli extract**: Fetch full content from paper URLs, journal websites, and preprint servers — use for reading abstracts, extracting reference lists, and verifying paper details
-- **parallel-cli search --include-domains**: Academic-focused search across scholarly domains (arxiv.org, pubmed, nature.com, etc.)
-
-### Database Access Skills
-- **gget**: PubMed, bioRxiv, COSMIC, AlphaFold, Ensembl, UniProt
-- **bioservices**: ChEMBL, KEGG, Reactome, UniProt, PubChem
-- **datacommons-client**: Demographics, economics, health statistics
-
-### Analysis Skills
-- **pydeseq2**: RNA-seq differential expression (for methods sections)
-- **scanpy**: Single-cell analysis (for methods sections)
-- **anndata**: Single-cell data (for methods sections)
-- **biopython**: Sequence analysis (for background sections)
-
-### Visualization Skills
-- **matplotlib**: Generate figures and plots for review
-- **seaborn**: Statistical visualizations
-
-### Writing Skills
-- **brand-guidelines**: Apply institutional branding to PDF
-- **internal-comms**: Adapt review for different audiences
-- **venue-templates**: Access venue-specific writing style guides when preparing reviews for publication
-
-### Venue-Specific Writing Styles
-
-When preparing a literature review for a specific journal, consult the **venue-templates** skill for writing style guidance:
-- `venue_writing_styles.md`: Master style comparison across venues
-- `nature_science_style.md`: Nature/Science flowing abstract style, story-driven structure
-- `cell_press_style.md`: Cell Press graphical abstracts, Highlights format
-- `medical_journal_styles.md`: NEJM/Lancet/JAMA structured abstracts, PRISMA compliance
-
-These guides help adapt your review's tone, abstract format, and structure to match the target venue's expectations.
-
-## Resources
-
-### Bundled Resources
-
-**Scripts:**
-- `scripts/verify_citations.py`: Verify DOIs and generate formatted citations
-- `scripts/generate_pdf.py`: Convert markdown to professional PDF
-- `scripts/search_databases.py`: Process, deduplicate, and format search results
-
-**References:**
-- `references/citation_styles.md`: Detailed citation formatting guide (APA, Nature, Vancouver, Chicago, IEEE)
-- `references/database_strategies.md`: Comprehensive database search strategies
-
-**Assets:**
-- `assets/review_template.md`: Complete literature review template with all sections
-
-### External Resources
-
-**Guidelines:**
-- PRISMA (Systematic Reviews): http://www.prisma-statement.org/
-- Cochrane Handbook: https://training.cochrane.org/handbook
-- AMSTAR 2 (Review Quality): https://amstar.ca/
-
-**Tools:**
-- MeSH Browser: https://meshb.nlm.nih.gov/search
-- PubMed Advanced Search: https://pubmed.ncbi.nlm.nih.gov/advanced/
-- Boolean Search Guide: https://www.ncbi.nlm.nih.gov/books/NBK3827/
-
-**Citation Styles:**
-- APA Style: https://apastyle.apa.org/
-- Nature Portfolio: https://www.nature.com/nature-portfolio/editorial-policies/reporting-standards
-- NLM/Vancouver: https://www.nlm.nih.gov/bsd/uniform_requirements.html
-
-## Dependencies
-
-### Required CLI Tools
-```bash
-# parallel-cli (PRIMARY — for web search and URL extraction)
-curl -fsSL https://parallel.ai/install.sh | bash
-# Or: uv tool install "parallel-web-tools[cli]"
-# Authenticate: parallel-cli auth
-```
-
-### Required Python Packages
-```bash
-uv pip install requests  # For citation verification
-```
-
-### Required System Tools
-```bash
-# For PDF generation
-brew install pandoc  # macOS
-apt-get install pandoc  # Linux
-
-# For LaTeX (PDF generation)
-brew install --cask mactex  # macOS
-apt-get install texlive-xetex  # Linux
-```
-
-Check dependencies:
-```bash
-python scripts/generate_pdf.py --check-deps
-```
-
-## Summary
-
-This literature-review skill provides:
-
-1. **Systematic methodology** following academic best practices
-2. **Parallel-web powered search** using `parallel-cli search` for fast, broad academic literature discovery with scholarly domain filtering
-3. **Multi-database integration** via existing scientific skills (gget, bioservices, datacommons-client)
-4. **Citation verification** ensuring accuracy and credibility
-5. **Professional output** in markdown and PDF formats
-6. **Comprehensive guidance** covering the entire review process
-7. **Quality assurance** with verification and validation tools
-8. **Reproducibility** through detailed documentation requirements
-
-Conduct thorough, rigorous literature reviews that meet academic standards and provide comprehensive synthesis of current knowledge in any domain.