@wentorai/research-plugins 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -18
- package/curated/analysis/README.md +12 -1
- package/curated/domains/README.md +48 -1
- package/curated/literature/README.md +46 -1
- package/curated/research/README.md +16 -1
- package/curated/tools/README.md +20 -1
- package/curated/writing/README.md +13 -1
- package/mcp-configs/academic-db/alphafold-mcp.json +20 -0
- package/mcp-configs/academic-db/brightspace-mcp.json +21 -0
- package/mcp-configs/academic-db/climatiq-mcp.json +20 -0
- package/mcp-configs/academic-db/gibs-mcp.json +20 -0
- package/mcp-configs/academic-db/gis-mcp-server.json +22 -0
- package/mcp-configs/academic-db/google-earth-engine-mcp.json +21 -0
- package/mcp-configs/academic-db/m4-clinical-mcp.json +21 -0
- package/mcp-configs/academic-db/medical-mcp.json +21 -0
- package/mcp-configs/academic-db/nexonco-mcp.json +20 -0
- package/mcp-configs/academic-db/omop-mcp.json +20 -0
- package/mcp-configs/academic-db/onekgpd-mcp.json +20 -0
- package/mcp-configs/academic-db/openedu-mcp.json +20 -0
- package/mcp-configs/academic-db/opengenes-mcp.json +20 -0
- package/mcp-configs/academic-db/openstax-mcp.json +21 -0
- package/mcp-configs/academic-db/openstreetmap-mcp.json +21 -0
- package/mcp-configs/academic-db/opentargets-mcp.json +21 -0
- package/mcp-configs/academic-db/pdb-mcp.json +21 -0
- package/mcp-configs/academic-db/smithsonian-mcp.json +20 -0
- package/mcp-configs/ai-platform/magi-researchers.json +21 -0
- package/mcp-configs/ai-platform/mcp-academic-researcher.json +22 -0
- package/mcp-configs/ai-platform/open-paper-machine.json +21 -0
- package/mcp-configs/ai-platform/paper-intelligence.json +21 -0
- package/mcp-configs/ai-platform/paper-reader.json +21 -0
- package/mcp-configs/ai-platform/paperdebugger.json +21 -0
- package/mcp-configs/browser/exa-mcp.json +20 -0
- package/mcp-configs/browser/mcp-searxng.json +21 -0
- package/mcp-configs/browser/mcp-webresearch.json +20 -0
- package/mcp-configs/communication/discourse-mcp.json +21 -0
- package/mcp-configs/data-platform/automl-stat-mcp.json +21 -0
- package/mcp-configs/data-platform/jefferson-stats-mcp.json +22 -0
- package/mcp-configs/data-platform/mcp-excel-server.json +21 -0
- package/mcp-configs/data-platform/mcp-stata.json +21 -0
- package/mcp-configs/data-platform/mcpstack-jupyter.json +21 -0
- package/mcp-configs/data-platform/ml-mcp.json +21 -0
- package/mcp-configs/data-platform/nasdaq-data-link-mcp.json +20 -0
- package/mcp-configs/data-platform/numpy-mcp.json +21 -0
- package/mcp-configs/dev-platform/geogebra-mcp.json +21 -0
- package/mcp-configs/dev-platform/latex-mcp-server.json +21 -0
- package/mcp-configs/dev-platform/manim-mcp.json +20 -0
- package/mcp-configs/dev-platform/mcp-echarts.json +20 -0
- package/mcp-configs/dev-platform/panel-viz-mcp.json +20 -0
- package/mcp-configs/dev-platform/paperbanana.json +20 -0
- package/mcp-configs/dev-platform/texflow-mcp.json +20 -0
- package/mcp-configs/dev-platform/texmcp.json +20 -0
- package/mcp-configs/dev-platform/typst-mcp.json +21 -0
- package/mcp-configs/dev-platform/vizro-mcp.json +20 -0
- package/mcp-configs/note-knowledge/local-faiss-mcp.json +21 -0
- package/mcp-configs/note-knowledge/mcp-memory-service.json +21 -0
- package/mcp-configs/note-knowledge/mcp-obsidian.json +23 -0
- package/mcp-configs/note-knowledge/mcp-ragdocs.json +20 -0
- package/mcp-configs/note-knowledge/mcp-summarizer.json +21 -0
- package/mcp-configs/note-knowledge/mediawiki-mcp.json +21 -0
- package/mcp-configs/note-knowledge/openzim-mcp.json +20 -0
- package/mcp-configs/note-knowledge/zettelkasten-mcp.json +21 -0
- package/mcp-configs/reference-mgr/academic-paper-mcp-http.json +20 -0
- package/mcp-configs/reference-mgr/academix.json +20 -0
- package/mcp-configs/reference-mgr/arxiv-research-mcp.json +21 -0
- package/mcp-configs/reference-mgr/google-scholar-abstract-mcp.json +19 -0
- package/mcp-configs/reference-mgr/google-scholar-mcp.json +20 -0
- package/mcp-configs/reference-mgr/mcp-paperswithcode.json +21 -0
- package/mcp-configs/reference-mgr/mcp-scholarly.json +20 -0
- package/mcp-configs/reference-mgr/mcp-simple-arxiv.json +20 -0
- package/mcp-configs/reference-mgr/mcp-simple-pubmed.json +20 -0
- package/mcp-configs/reference-mgr/mcp-zotero.json +21 -0
- package/mcp-configs/reference-mgr/mendeley-mcp.json +20 -0
- package/mcp-configs/reference-mgr/ncbi-mcp-server.json +22 -0
- package/mcp-configs/reference-mgr/onecite.json +21 -0
- package/mcp-configs/reference-mgr/paper-search-mcp.json +21 -0
- package/mcp-configs/reference-mgr/pubmed-search-mcp.json +21 -0
- package/mcp-configs/reference-mgr/scholar-mcp.json +21 -0
- package/mcp-configs/reference-mgr/scholar-multi-mcp.json +21 -0
- package/mcp-configs/reference-mgr/seerai.json +21 -0
- package/mcp-configs/reference-mgr/semantic-scholar-fastmcp.json +21 -0
- package/mcp-configs/reference-mgr/sourcelibrary.json +20 -0
- package/openclaw.plugin.json +2 -2
- package/package.json +2 -2
- package/skills/analysis/dataviz/citation-map-guide/SKILL.md +184 -0
- package/skills/analysis/dataviz/data-visualization-principles/SKILL.md +171 -0
- package/skills/analysis/econometrics/econml-causal-guide/SKILL.md +2 -2
- package/skills/analysis/econometrics/empirical-paper-analysis/SKILL.md +192 -0
- package/skills/analysis/econometrics/mostly-harmless-guide/SKILL.md +2 -2
- package/skills/analysis/econometrics/panel-data-regression-workflow/SKILL.md +267 -0
- package/skills/analysis/econometrics/python-causality-guide/SKILL.md +2 -2
- package/skills/analysis/econometrics/stata-reference-guide/SKILL.md +293 -0
- package/skills/analysis/statistics/general-statistics-guide/SKILL.md +226 -0
- package/skills/analysis/statistics/infiagent-benchmark-guide/SKILL.md +106 -0
- package/skills/analysis/wrangling/claude-data-analysis-guide/SKILL.md +100 -0
- package/skills/analysis/wrangling/open-data-scientist-guide/SKILL.md +197 -0
- package/skills/analysis/wrangling/streamline-analyst-guide/SKILL.md +119 -0
- package/skills/domains/ai-ml/ai-agent-papers-guide/SKILL.md +146 -0
- package/skills/domains/ai-ml/anomaly-detection-papers-guide/SKILL.md +167 -0
- package/skills/domains/ai-ml/autonomous-agents-papers-guide/SKILL.md +178 -0
- package/skills/domains/ai-ml/domain-adaptation-papers-guide/SKILL.md +173 -0
- package/skills/domains/ai-ml/generative-ai-guide/SKILL.md +2 -2
- package/skills/domains/ai-ml/graph-learning-papers-guide/SKILL.md +125 -0
- package/skills/domains/ai-ml/kolmogorov-arnold-networks-guide/SKILL.md +185 -0
- package/skills/domains/ai-ml/npcpy-research-guide/SKILL.md +137 -0
- package/skills/domains/ai-ml/responsible-ai-guide/SKILL.md +126 -0
- package/skills/domains/ai-ml/vmas-simulator-guide/SKILL.md +129 -0
- package/skills/domains/biomedical/clawbio-guide/SKILL.md +167 -0
- package/skills/domains/biomedical/clinical-dialogue-agents-guide/SKILL.md +145 -0
- package/skills/domains/biomedical/ena-sequence-api/SKILL.md +175 -0
- package/skills/domains/biomedical/genomas-guide/SKILL.md +126 -0
- package/skills/domains/biomedical/genotex-benchmark-guide/SKILL.md +125 -0
- package/skills/domains/biomedical/med-researcher-guide/SKILL.md +161 -0
- package/skills/domains/biomedical/med-researcher-r1-guide/SKILL.md +146 -0
- package/skills/domains/biomedical/ncbi-blast-api/SKILL.md +195 -0
- package/skills/domains/biomedical/ncbi-datasets-api/SKILL.md +220 -0
- package/skills/domains/biomedical/quickgo-api/SKILL.md +181 -0
- package/skills/domains/business/xpert-bi-guide/SKILL.md +84 -0
- package/skills/domains/chemistry/cactus-cheminformatics-guide/SKILL.md +89 -0
- package/skills/domains/chemistry/chemeagle-guide/SKILL.md +147 -0
- package/skills/domains/chemistry/chemgraph-agent-guide/SKILL.md +120 -0
- package/skills/domains/cs/ai-security-papers-guide/SKILL.md +103 -0
- package/skills/domains/cs/code-llm-papers-guide/SKILL.md +131 -0
- package/skills/domains/cs/gaussian-splatting-papers-guide/SKILL.md +158 -0
- package/skills/domains/cs/llm-aiops-guide/SKILL.md +70 -0
- package/skills/domains/cs/software-heritage-api/SKILL.md +200 -0
- package/skills/domains/economics/nber-working-papers-api/SKILL.md +177 -0
- package/skills/domains/economics/repec-economics-api/SKILL.md +188 -0
- package/skills/domains/education/academic-study-methods/SKILL.md +228 -0
- package/skills/domains/education/edumcp-guide/SKILL.md +74 -0
- package/skills/domains/education/open-syllabus-api/SKILL.md +171 -0
- package/skills/domains/finance/akshare-finance-data/SKILL.md +207 -0
- package/skills/domains/finance/finsight-research-guide/SKILL.md +113 -0
- package/skills/domains/finance/options-analytics-agent-guide/SKILL.md +117 -0
- package/skills/domains/geoscience/pangaea-data-api/SKILL.md +197 -0
- package/skills/domains/humanities/digital-humanities-methods/SKILL.md +232 -0
- package/skills/domains/law/caselaw-access-api/SKILL.md +149 -0
- package/skills/domains/law/legal-agent-skills-guide/SKILL.md +132 -0
- package/skills/domains/law/legal-research-methods/SKILL.md +190 -0
- package/skills/domains/law/opencontracts-guide/SKILL.md +168 -0
- package/skills/domains/math/lean-theorem-proving-guide/SKILL.md +140 -0
- package/skills/domains/pharma/madd-drug-discovery-guide/SKILL.md +153 -0
- package/skills/domains/social-science/ipums-microdata-api/SKILL.md +211 -0
- package/skills/domains/social-science/sociology-research-methods/SKILL.md +181 -0
- package/skills/literature/discovery/arxiv-paper-monitoring/SKILL.md +233 -0
- package/skills/literature/discovery/papers-we-love-guide/SKILL.md +169 -0
- package/skills/literature/discovery/zotero-arxiv-daily-guide/SKILL.md +2 -2
- package/skills/literature/fulltext/bioc-pmc-api/SKILL.md +146 -0
- package/skills/literature/fulltext/dataverse-api/SKILL.md +215 -0
- package/skills/literature/fulltext/hal-archive-api/SKILL.md +218 -0
- package/skills/literature/fulltext/osf-api/SKILL.md +212 -0
- package/skills/literature/fulltext/pmc-ftp-bulk-download/SKILL.md +182 -0
- package/skills/literature/fulltext/zotero-ai-butler-guide/SKILL.md +166 -0
- package/skills/literature/fulltext/zotero-scihub-guide/SKILL.md +168 -0
- package/skills/literature/metadata/bibliometrix-guide/SKILL.md +164 -0
- package/skills/literature/metadata/crossref-event-data-api/SKILL.md +183 -0
- package/skills/literature/metadata/doi-content-negotiation/SKILL.md +202 -0
- package/skills/literature/metadata/orkg-api/SKILL.md +153 -0
- package/skills/literature/metadata/plumx-metrics-api/SKILL.md +188 -0
- package/skills/literature/metadata/ror-organization-api/SKILL.md +208 -0
- package/skills/literature/metadata/sophosia-reference-guide/SKILL.md +110 -0
- package/skills/literature/metadata/viaf-authority-api/SKILL.md +209 -0
- package/skills/literature/metadata/zoplicate-dedup-guide/SKILL.md +147 -0
- package/skills/literature/metadata/zotero-actions-tags-guide/SKILL.md +212 -0
- package/skills/literature/metadata/zotmoov-guide/SKILL.md +120 -0
- package/skills/literature/metadata/zutilo-guide/SKILL.md +140 -0
- package/skills/literature/search/arxiv-cli-tools/SKILL.md +172 -0
- package/skills/literature/search/arxiv-osiris/SKILL.md +199 -0
- package/skills/literature/search/base-academic-search/SKILL.md +196 -0
- package/skills/literature/search/chatpaper-guide/SKILL.md +2 -2
- package/skills/literature/search/citeseerx-api/SKILL.md +183 -0
- package/skills/literature/search/deepgit-search-guide/SKILL.md +2 -2
- package/skills/literature/search/eric-education-api/SKILL.md +199 -0
- package/skills/literature/search/findpapers-guide/SKILL.md +177 -0
- package/skills/literature/search/ieee-xplore-api/SKILL.md +177 -0
- package/skills/literature/search/lens-scholarly-api/SKILL.md +211 -0
- package/skills/literature/search/multi-database-literature-search/SKILL.md +198 -0
- package/skills/literature/search/open-library-api/SKILL.md +196 -0
- package/skills/literature/search/open-semantic-search-guide/SKILL.md +190 -0
- package/skills/literature/search/openaire-api/SKILL.md +141 -0
- package/skills/literature/search/paper-search-mcp-guide/SKILL.md +107 -0
- package/skills/literature/search/papers-chat-guide/SKILL.md +194 -0
- package/skills/literature/search/pasa-paper-search-guide/SKILL.md +2 -2
- package/skills/literature/search/plos-open-access-api/SKILL.md +203 -0
- package/skills/literature/search/scielo-api/SKILL.md +182 -0
- package/skills/literature/search/share-research-api/SKILL.md +129 -0
- package/skills/literature/search/worldcat-search-api/SKILL.md +224 -0
- package/skills/research/automation/aim-experiment-guide/SKILL.md +2 -2
- package/skills/research/automation/claude-academic-workflow-guide/SKILL.md +202 -0
- package/skills/research/automation/coexist-ai-guide/SKILL.md +149 -0
- package/skills/research/automation/datagen-research-guide/SKILL.md +2 -2
- package/skills/research/automation/foam-agent-guide/SKILL.md +203 -0
- package/skills/research/automation/kedro-pipeline-guide/SKILL.md +2 -2
- package/skills/research/automation/mle-agent-guide/SKILL.md +2 -2
- package/skills/research/automation/paper-to-agent-guide/SKILL.md +2 -2
- package/skills/research/deep-research/auto-deep-research-guide/SKILL.md +2 -2
- package/skills/research/deep-research/cognitive-kernel-guide/SKILL.md +200 -0
- package/skills/research/deep-research/corvus-research-guide/SKILL.md +132 -0
- package/skills/research/deep-research/in-depth-research-guide/SKILL.md +205 -0
- package/skills/research/deep-research/kosmos-scientist-guide/SKILL.md +185 -0
- package/skills/research/deep-research/llm-scientific-discovery-guide/SKILL.md +178 -0
- package/skills/research/deep-research/open-researcher-guide/SKILL.md +138 -0
- package/skills/research/methodology/claude-scientific-guide/SKILL.md +2 -2
- package/skills/research/methodology/parsifal-slr-guide/SKILL.md +154 -0
- package/skills/research/methodology/research-pipeline-units-guide/SKILL.md +169 -0
- package/skills/research/methodology/slr-automation-guide/SKILL.md +235 -0
- package/skills/research/paper-review/latte-review-guide/SKILL.md +175 -0
- package/skills/research/paper-review/paper-critique-framework/SKILL.md +181 -0
- package/skills/tools/code-exec/contextplus-mcp-guide/SKILL.md +110 -0
- package/skills/tools/diagram/clawphd-guide/SKILL.md +149 -0
- package/skills/tools/diagram/kroki-diagram-api/SKILL.md +198 -0
- package/skills/tools/diagram/scientific-graphical-abstract/SKILL.md +201 -0
- package/skills/tools/document/docsgpt-guide/SKILL.md +2 -2
- package/skills/tools/document/md2pdf-xelatex/SKILL.md +212 -0
- package/skills/tools/document/openpaper-guide/SKILL.md +232 -0
- package/skills/tools/document/weknora-guide/SKILL.md +216 -0
- package/skills/tools/document/zotero-addon-market-guide/SKILL.md +108 -0
- package/skills/tools/document/zotero-night-theme-guide/SKILL.md +142 -0
- package/skills/tools/document/zotero-style-guide/SKILL.md +217 -0
- package/skills/tools/knowledge-graph/graphiti-guide/SKILL.md +2 -2
- package/skills/tools/knowledge-graph/mimir-memory-guide/SKILL.md +135 -0
- package/skills/tools/knowledge-graph/notero-zotero-notion-guide/SKILL.md +187 -0
- package/skills/tools/knowledge-graph/open-webui-tools-guide/SKILL.md +156 -0
- package/skills/tools/knowledge-graph/openspg-guide/SKILL.md +210 -0
- package/skills/tools/knowledge-graph/paperpile-notion-guide/SKILL.md +84 -0
- package/skills/tools/knowledge-graph/zotero-markdb-connect-guide/SKILL.md +162 -0
- package/skills/tools/ocr-translate/latex-translation-guide/SKILL.md +176 -0
- package/skills/tools/ocr-translate/math-equation-renderer/SKILL.md +198 -0
- package/skills/tools/ocr-translate/pdf-math-translate-guide/SKILL.md +2 -2
- package/skills/tools/ocr-translate/zotero-pdf-translate-guide/SKILL.md +2 -2
- package/skills/tools/ocr-translate/zotero-pdf2zh-guide/SKILL.md +2 -2
- package/skills/writing/citation/academic-citation-manager-guide/SKILL.md +182 -0
- package/skills/writing/citation/citation-assistant-skill/SKILL.md +192 -0
- package/skills/writing/citation/jabref-reference-guide/SKILL.md +2 -2
- package/skills/writing/citation/jasminum-zotero-guide/SKILL.md +2 -2
- package/skills/writing/citation/mendeley-api/SKILL.md +231 -0
- package/skills/writing/citation/obsidian-citation-guide/SKILL.md +2 -2
- package/skills/writing/citation/obsidian-zotero-guide/SKILL.md +2 -2
- package/skills/writing/citation/onecite-reference-guide/SKILL.md +168 -0
- package/skills/writing/citation/papersgpt-zotero-guide/SKILL.md +2 -2
- package/skills/writing/citation/papis-cli-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-better-bibtex-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-better-notes-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-gpt-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-mcp-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-mdnotes-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotero-reference-guide/SKILL.md +2 -2
- package/skills/writing/citation/zotfile-attachment-guide/SKILL.md +2 -2
- package/skills/writing/composition/opendraft-thesis-guide/SKILL.md +200 -0
- package/skills/writing/composition/paper-debugger-guide/SKILL.md +2 -2
- package/skills/writing/composition/paperforge-guide/SKILL.md +205 -0
- package/skills/writing/composition/research-paper-writer/SKILL.md +226 -0
- package/skills/writing/composition/scientific-writing-resources/SKILL.md +2 -2
- package/skills/writing/latex/academic-writing-latex/SKILL.md +285 -0
- package/skills/writing/latex/latex-drawing-collection/SKILL.md +2 -2
- package/skills/writing/latex/latex-templates-collection/SKILL.md +2 -2
- package/skills/writing/polish/chinese-text-humanizer/SKILL.md +140 -0
- package/skills/writing/templates/arxiv-preprint-template/SKILL.md +184 -0
- package/skills/writing/templates/elegant-paper-template/SKILL.md +141 -0
- package/skills/writing/templates/novathesis-guide/SKILL.md +2 -2
- package/skills/writing/templates/sjtuthesis-guide/SKILL.md +2 -2
- package/skills/writing/templates/thuthesis-guide/SKILL.md +2 -2
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: finsight-research-guide
|
|
3
|
+
description: "Deep financial research with the FinSight multi-agent system"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "💰"
|
|
7
|
+
category: "domains"
|
|
8
|
+
subcategory: "finance"
|
|
9
|
+
keywords: ["FinSight", "financial analysis", "deep research", "market analysis", "financial reports", "multi-agent"]
|
|
10
|
+
source: "https://github.com/RUC-NLPIR/FinSight"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# FinSight Research Guide
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
FinSight is a deep research agent designed specifically for financial analysis. Developed by RUC-NLPIR, it combines multi-source data retrieval, financial reasoning, and report generation to produce publication-ready financial research. It handles market analysis, company fundamentals, sector comparisons, and macroeconomic assessment through specialized agents.
|
|
18
|
+
|
|
19
|
+
## Installation
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
git clone https://github.com/RUC-NLPIR/FinSight.git
|
|
23
|
+
cd FinSight && pip install -e .
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Core Capabilities
|
|
27
|
+
|
|
28
|
+
### Research Query to Report
|
|
29
|
+
|
|
30
|
+
```python
|
|
31
|
+
from finsight import FinSightAgent
|
|
32
|
+
|
|
33
|
+
agent = FinSightAgent(llm_provider="anthropic")
|
|
34
|
+
|
|
35
|
+
# Generate comprehensive financial analysis
|
|
36
|
+
report = agent.research(
|
|
37
|
+
"Analyze the competitive landscape of the global EV battery "
|
|
38
|
+
"market. Compare CATL, LG Energy, and Panasonic on market "
|
|
39
|
+
"share, technology, margins, and growth outlook."
|
|
40
|
+
)
|
|
41
|
+
|
|
42
|
+
print(report.summary)
|
|
43
|
+
report.save("ev_battery_analysis.pdf")
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### Agent Architecture
|
|
47
|
+
|
|
48
|
+
| Agent | Role |
|
|
49
|
+
|-------|------|
|
|
50
|
+
| **Retrieval Agent** | Fetches data from SEC filings, financial APIs, news |
|
|
51
|
+
| **Data Agent** | Processes financial statements, ratios, time series |
|
|
52
|
+
| **Analysis Agent** | Performs fundamental, technical, and comparative analysis |
|
|
53
|
+
| **Reasoning Agent** | Synthesizes findings, identifies trends and risks |
|
|
54
|
+
| **Report Agent** | Generates structured research reports with citations |
|
|
55
|
+
|
|
56
|
+
### Financial Data Sources
|
|
57
|
+
|
|
58
|
+
```python
|
|
59
|
+
# FinSight integrates with multiple data sources
|
|
60
|
+
config = {
|
|
61
|
+
"sec_edgar": True, # SEC filings (free)
|
|
62
|
+
"fred": True, # Federal Reserve economic data
|
|
63
|
+
"yahoo_finance": True, # Market data (free)
|
|
64
|
+
"news_api": True, # Financial news
|
|
65
|
+
"world_bank": True, # Macro indicators
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Analysis Types
|
|
70
|
+
|
|
71
|
+
```python
|
|
72
|
+
# Company fundamental analysis
|
|
73
|
+
report = agent.research(
|
|
74
|
+
"Provide a fundamental analysis of NVIDIA including "
|
|
75
|
+
"revenue trends, margin analysis, valuation multiples, "
|
|
76
|
+
"and competitive moat assessment."
|
|
77
|
+
)
|
|
78
|
+
|
|
79
|
+
# Sector analysis
|
|
80
|
+
report = agent.research(
|
|
81
|
+
"Compare the top 5 cloud computing companies by revenue "
|
|
82
|
+
"growth, operating margins, and R&D investment intensity."
|
|
83
|
+
)
|
|
84
|
+
|
|
85
|
+
# Macro analysis
|
|
86
|
+
report = agent.research(
|
|
87
|
+
"Analyze the impact of rising interest rates on US "
|
|
88
|
+
"commercial real estate valuations since 2022."
|
|
89
|
+
)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Report Structure
|
|
93
|
+
|
|
94
|
+
Generated reports typically include:
|
|
95
|
+
|
|
96
|
+
1. **Executive Summary** — Key findings in 3-5 bullets
|
|
97
|
+
2. **Market Overview** — Industry size, growth, trends
|
|
98
|
+
3. **Company Analysis** — Financials, competitive position
|
|
99
|
+
4. **Risk Assessment** — Key risks and mitigation
|
|
100
|
+
5. **Outlook** — Forward-looking analysis with scenarios
|
|
101
|
+
6. **Sources** — Cited data sources and references
|
|
102
|
+
|
|
103
|
+
## Use Cases
|
|
104
|
+
|
|
105
|
+
1. **Investment research**: Company and sector deep dives
|
|
106
|
+
2. **Due diligence**: Comprehensive target company analysis
|
|
107
|
+
3. **Academic research**: Financial economics research support
|
|
108
|
+
4. **Market intelligence**: Competitive landscape mapping
|
|
109
|
+
|
|
110
|
+
## References
|
|
111
|
+
|
|
112
|
+
- [FinSight GitHub](https://github.com/RUC-NLPIR/FinSight)
|
|
113
|
+
- [RUC-NLPIR Lab](http://playbigdata.ruc.edu.cn/)
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: options-analytics-agent-guide
|
|
3
|
+
description: "AI agent for options pricing, Greeks, and strategy analysis"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "📉"
|
|
7
|
+
category: "domains"
|
|
8
|
+
subcategory: "finance"
|
|
9
|
+
keywords: ["options analytics", "derivatives", "Greeks", "Black-Scholes", "strategy analysis", "financial agent"]
|
|
10
|
+
source: "https://github.com/options-analytics/options-agent"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Options Analytics Agent Guide
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
An AI agent for options pricing, risk analysis, and strategy evaluation. It combines Black-Scholes and binomial models, Greeks calculations, implied volatility surfaces, and portfolio risk analytics into a conversational interface. Researchers and quantitative analysts can query options data, price exotic derivatives, and evaluate trading strategies through natural language.
|
|
18
|
+
|
|
19
|
+
## Core Capabilities
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
from options_agent import OptionsAgent
|
|
23
|
+
|
|
24
|
+
agent = OptionsAgent(llm_provider="anthropic")
|
|
25
|
+
|
|
26
|
+
# Price an option
|
|
27
|
+
result = agent.price(
|
|
28
|
+
option_type="call",
|
|
29
|
+
strike=100,
|
|
30
|
+
spot=105,
|
|
31
|
+
expiry_days=30,
|
|
32
|
+
risk_free_rate=0.05,
|
|
33
|
+
volatility=0.20,
|
|
34
|
+
model="black_scholes",
|
|
35
|
+
)
|
|
36
|
+
|
|
37
|
+
print(f"Price: ${result.price:.2f}")
|
|
38
|
+
print(f"Delta: {result.delta:.4f}")
|
|
39
|
+
print(f"Gamma: {result.gamma:.4f}")
|
|
40
|
+
print(f"Theta: {result.theta:.4f}")
|
|
41
|
+
print(f"Vega: {result.vega:.4f}")
|
|
42
|
+
print(f"Rho: {result.rho:.4f}")
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Greeks Analysis
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
# Full Greeks surface
|
|
49
|
+
surface = agent.greeks_surface(
|
|
50
|
+
strike=100,
|
|
51
|
+
spot_range=(80, 120),
|
|
52
|
+
expiry_range=(7, 90), # days
|
|
53
|
+
volatility=0.25,
|
|
54
|
+
)
|
|
55
|
+
|
|
56
|
+
surface.plot_delta_surface("delta_surface.png")
|
|
57
|
+
surface.plot_gamma_surface("gamma_surface.png")
|
|
58
|
+
surface.plot_theta_decay("theta_decay.png")
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Strategy Evaluation
|
|
62
|
+
|
|
63
|
+
```python
|
|
64
|
+
# Evaluate an options strategy
|
|
65
|
+
strategy = agent.evaluate_strategy(
|
|
66
|
+
legs=[
|
|
67
|
+
{"type": "call", "strike": 100, "action": "buy", "qty": 1},
|
|
68
|
+
{"type": "call", "strike": 110, "action": "sell", "qty": 1},
|
|
69
|
+
],
|
|
70
|
+
spot=105,
|
|
71
|
+
expiry_days=30,
|
|
72
|
+
volatility=0.20,
|
|
73
|
+
)
|
|
74
|
+
|
|
75
|
+
print(f"Strategy: {strategy.name}") # Bull Call Spread
|
|
76
|
+
print(f"Max profit: ${strategy.max_profit:.2f}")
|
|
77
|
+
print(f"Max loss: ${strategy.max_loss:.2f}")
|
|
78
|
+
print(f"Breakeven: ${strategy.breakeven:.2f}")
|
|
79
|
+
|
|
80
|
+
strategy.plot_payoff("payoff.png")
|
|
81
|
+
strategy.plot_pnl_scenarios("scenarios.png")
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Implied Volatility
|
|
85
|
+
|
|
86
|
+
```python
|
|
87
|
+
# Calculate implied volatility
|
|
88
|
+
iv = agent.implied_volatility(
|
|
89
|
+
market_price=5.50,
|
|
90
|
+
option_type="call",
|
|
91
|
+
strike=100,
|
|
92
|
+
spot=105,
|
|
93
|
+
expiry_days=30,
|
|
94
|
+
risk_free_rate=0.05,
|
|
95
|
+
)
|
|
96
|
+
print(f"Implied volatility: {iv:.2%}")
|
|
97
|
+
|
|
98
|
+
# Volatility smile/surface
|
|
99
|
+
vol_surface = agent.volatility_surface(
|
|
100
|
+
ticker="SPY",
|
|
101
|
+
date="2025-03-10",
|
|
102
|
+
)
|
|
103
|
+
vol_surface.plot("vol_surface.png")
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
## Use Cases
|
|
107
|
+
|
|
108
|
+
1. **Options pricing**: Black-Scholes and numerical methods
|
|
109
|
+
2. **Risk management**: Greeks and portfolio risk metrics
|
|
110
|
+
3. **Strategy analysis**: P&L profiles and breakeven analysis
|
|
111
|
+
4. **Volatility analysis**: IV surfaces and skew analysis
|
|
112
|
+
5. **Education**: Interactive derivatives teaching tool
|
|
113
|
+
|
|
114
|
+
## References
|
|
115
|
+
|
|
116
|
+
- [Options Analytics Agent](https://github.com/options-analytics/options-agent)
|
|
117
|
+
- [QuantLib](https://www.quantlib.org/) — Quantitative finance library
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: pangaea-data-api
|
|
3
|
+
description: "Access earth and environmental science datasets via PANGAEA API"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "🌍"
|
|
7
|
+
category: "domains"
|
|
8
|
+
subcategory: "geoscience"
|
|
9
|
+
keywords: ["PANGAEA", "earth science data", "oceanography", "paleoclimate", "environmental data", "geoscience repository"]
|
|
10
|
+
source: "https://www.pangaea.de/"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# PANGAEA Data Repository API
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
PANGAEA is the world's leading data repository for earth and environmental sciences, hosting 400K+ datasets with 20B+ data points. It archives research data from oceanography, paleoclimatology, geology, ecology, and atmospheric science. Each dataset has a DOI and is linked to the originating publication. The API provides search, metadata retrieval, and data download. Free, no authentication required.
|
|
18
|
+
|
|
19
|
+
## API Endpoints
|
|
20
|
+
|
|
21
|
+
### Search API
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
# Search datasets by keyword
|
|
25
|
+
curl "https://www.pangaea.de/advanced/search.php?q=ocean+temperature&count=20&type=json"
|
|
26
|
+
|
|
27
|
+
# Search with geographic bounding box
|
|
28
|
+
curl "https://www.pangaea.de/advanced/search.php?\
|
|
29
|
+
q=sediment+core&minlat=-60&maxlat=-30&minlon=-180&maxlon=180&type=json"
|
|
30
|
+
|
|
31
|
+
# Filter by parameter (measurement type)
|
|
32
|
+
curl "https://www.pangaea.de/advanced/search.php?\
|
|
33
|
+
q=carbon+dioxide¶m=Atmospheric+CO2&type=json"
|
|
34
|
+
|
|
35
|
+
# Filter by date range
|
|
36
|
+
curl "https://www.pangaea.de/advanced/search.php?\
|
|
37
|
+
q=Arctic+ice&mindate=2020-01-01&maxdate=2026-12-31&type=json"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### ElasticSearch API
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
# Full-text search via Elasticsearch
|
|
44
|
+
curl -X POST "https://ws.pangaea.de/es/pangaea/panmd/_search" \
|
|
45
|
+
-H "Content-Type: application/json" \
|
|
46
|
+
-d '{
|
|
47
|
+
"query": {
|
|
48
|
+
"bool": {
|
|
49
|
+
"must": [
|
|
50
|
+
{"match": {"citation.title": "ocean temperature"}}
|
|
51
|
+
],
|
|
52
|
+
"filter": [
|
|
53
|
+
{"range": {"citation.year": {"gte": 2020}}}
|
|
54
|
+
]
|
|
55
|
+
}
|
|
56
|
+
},
|
|
57
|
+
"size": 20
|
|
58
|
+
}'
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
### Dataset Access
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
# Get dataset metadata
|
|
65
|
+
curl "https://doi.pangaea.de/10.1594/PANGAEA.123456?format=metainfo_json"
|
|
66
|
+
|
|
67
|
+
# Download dataset as tab-delimited text
|
|
68
|
+
curl "https://doi.pangaea.de/10.1594/PANGAEA.123456?format=textfile"
|
|
69
|
+
|
|
70
|
+
# Download as CSV
|
|
71
|
+
curl "https://doi.pangaea.de/10.1594/PANGAEA.123456?format=csv"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### OAI-PMH Harvesting
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
# List records
|
|
78
|
+
curl "https://ws.pangaea.de/oai/provider?verb=ListRecords&metadataPrefix=oai_dc"
|
|
79
|
+
|
|
80
|
+
# Get specific record
|
|
81
|
+
curl "https://ws.pangaea.de/oai/provider?verb=GetRecord&identifier=oai:pangaea.de:doi:10.1594/PANGAEA.123456&metadataPrefix=oai_dc"
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### Query Parameters (Search API)
|
|
85
|
+
|
|
86
|
+
| Parameter | Description | Example |
|
|
87
|
+
|-----------|-------------|---------|
|
|
88
|
+
| `q` | Search query | `q=coral+reef+bleaching` |
|
|
89
|
+
| `count` | Results per page | `count=50` |
|
|
90
|
+
| `offset` | Pagination offset | `offset=20` |
|
|
91
|
+
| `minlat/maxlat` | Latitude bounds | `-90` to `90` |
|
|
92
|
+
| `minlon/maxlon` | Longitude bounds | `-180` to `180` |
|
|
93
|
+
| `mindate/maxdate` | Temporal filter | `2020-01-01` |
|
|
94
|
+
| `param` | Parameter/measurement | `Temperature` |
|
|
95
|
+
| `topic` | Topic filter | `Atmosphere`, `Biosphere` |
|
|
96
|
+
| `type` | Response format | `json`, `xml` |
|
|
97
|
+
|
|
98
|
+
## Python Usage
|
|
99
|
+
|
|
100
|
+
```python
|
|
101
|
+
import requests
|
|
102
|
+
import pandas as pd
|
|
103
|
+
from io import StringIO
|
|
104
|
+
|
|
105
|
+
SEARCH_URL = "https://www.pangaea.de/advanced/search.php"
|
|
106
|
+
ES_URL = "https://ws.pangaea.de/es/pangaea/panmd/_search"
|
|
107
|
+
|
|
108
|
+
|
|
109
|
+
def search_pangaea(query: str, count: int = 20,
|
|
110
|
+
bbox: dict = None) -> list:
|
|
111
|
+
"""Search PANGAEA for earth science datasets."""
|
|
112
|
+
params = {"q": query, "count": count, "type": "json"}
|
|
113
|
+
if bbox:
|
|
114
|
+
params.update({
|
|
115
|
+
"minlat": bbox.get("south", -90),
|
|
116
|
+
"maxlat": bbox.get("north", 90),
|
|
117
|
+
"minlon": bbox.get("west", -180),
|
|
118
|
+
"maxlon": bbox.get("east", 180),
|
|
119
|
+
})
|
|
120
|
+
|
|
121
|
+
resp = requests.get(SEARCH_URL, params=params, timeout=30)
|
|
122
|
+
resp.raise_for_status()
|
|
123
|
+
data = resp.json()
|
|
124
|
+
|
|
125
|
+
results = []
|
|
126
|
+
for item in data.get("results", []):
|
|
127
|
+
results.append({
|
|
128
|
+
"doi": item.get("URI", ""),
|
|
129
|
+
"title": item.get("citation", ""),
|
|
130
|
+
"year": item.get("year"),
|
|
131
|
+
"size": item.get("size"),
|
|
132
|
+
"parameters": item.get("params", []),
|
|
133
|
+
"score": item.get("score"),
|
|
134
|
+
})
|
|
135
|
+
return results
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
def download_dataset(doi: str) -> pd.DataFrame:
|
|
139
|
+
"""Download a PANGAEA dataset as a pandas DataFrame."""
|
|
140
|
+
url = f"https://doi.pangaea.de/{doi}?format=textfile"
|
|
141
|
+
resp = requests.get(url, timeout=60)
|
|
142
|
+
resp.raise_for_status()
|
|
143
|
+
|
|
144
|
+
lines = resp.text.split("\n")
|
|
145
|
+
header_end = next(
|
|
146
|
+
(i for i, line in enumerate(lines) if line.startswith("*/")),
|
|
147
|
+
-1,
|
|
148
|
+
)
|
|
149
|
+
data_text = "\n".join(lines[header_end + 1:])
|
|
150
|
+
return pd.read_csv(StringIO(data_text), sep="\t")
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
def search_by_location(query: str, lat: float, lon: float,
|
|
154
|
+
radius_deg: float = 5.0) -> list:
|
|
155
|
+
"""Search datasets near a geographic location."""
|
|
156
|
+
bbox = {
|
|
157
|
+
"south": lat - radius_deg,
|
|
158
|
+
"north": lat + radius_deg,
|
|
159
|
+
"west": lon - radius_deg,
|
|
160
|
+
"east": lon + radius_deg,
|
|
161
|
+
}
|
|
162
|
+
return search_pangaea(query, bbox=bbox)
|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
# Example: find ocean temperature datasets
|
|
166
|
+
datasets = search_pangaea("sea surface temperature", count=5)
|
|
167
|
+
for ds in datasets:
|
|
168
|
+
print(f"[{ds['year']}] {ds['title'][:80]}...")
|
|
169
|
+
print(f" DOI: {ds['doi']} | Size: {ds['size']}")
|
|
170
|
+
|
|
171
|
+
# Example: download a specific dataset
|
|
172
|
+
# df = download_dataset("10.1594/PANGAEA.123456")
|
|
173
|
+
# print(df.head())
|
|
174
|
+
|
|
175
|
+
# Example: find Arctic research data
|
|
176
|
+
arctic = search_by_location("permafrost", lat=70, lon=25)
|
|
177
|
+
for ds in arctic[:3]:
|
|
178
|
+
print(f"{ds['title'][:80]}...")
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Data Topics
|
|
182
|
+
|
|
183
|
+
| Topic | Coverage |
|
|
184
|
+
|-------|----------|
|
|
185
|
+
| Oceans | Temperature, salinity, currents, chemistry |
|
|
186
|
+
| Paleoclimate | Ice cores, sediment cores, tree rings |
|
|
187
|
+
| Atmosphere | CO2, aerosols, weather observations |
|
|
188
|
+
| Lithosphere | Geology, tectonics, geochemistry |
|
|
189
|
+
| Biosphere | Biodiversity, ecology, marine biology |
|
|
190
|
+
| Cryosphere | Sea ice, glaciers, permafrost |
|
|
191
|
+
|
|
192
|
+
## References
|
|
193
|
+
|
|
194
|
+
- [PANGAEA](https://www.pangaea.de/)
|
|
195
|
+
- [PANGAEA API](https://wiki.pangaea.de/wiki/PANGAEA_API)
|
|
196
|
+
- [PANGAEA Elasticsearch](https://wiki.pangaea.de/wiki/Elasticsearch_API)
|
|
197
|
+
- Diepenbroek, M. et al. (2002). "PANGAEA — an information system for environmental sciences." *C&G* 28(10).
|
|
@@ -0,0 +1,232 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: digital-humanities-methods
|
|
3
|
+
description: "Computational methods for humanities research with text and network analysis"
|
|
4
|
+
metadata:
|
|
5
|
+
openclaw:
|
|
6
|
+
emoji: "📜"
|
|
7
|
+
category: "domains"
|
|
8
|
+
subcategory: "humanities"
|
|
9
|
+
keywords: ["digital humanities", "text analysis", "corpus linguistics", "network analysis", "cultural analytics", "computational methods"]
|
|
10
|
+
source: "https://clawhub.ai/digital-humanities"
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Digital Humanities Methods
|
|
14
|
+
|
|
15
|
+
## Overview
|
|
16
|
+
|
|
17
|
+
Digital Humanities (DH) applies computational methods to humanistic inquiry — analyzing literary texts, historical records, cultural artifacts, and social networks at scale. This guide covers the core computational methods used in DH research: text analysis, topic modeling, network analysis, spatial analysis, and corpus linguistics. These methods complement rather than replace traditional close reading and archival research.
|
|
18
|
+
|
|
19
|
+
## Text Analysis
|
|
20
|
+
|
|
21
|
+
### Preprocessing Pipeline
|
|
22
|
+
|
|
23
|
+
```python
|
|
24
|
+
import re
|
|
25
|
+
from collections import Counter
|
|
26
|
+
|
|
27
|
+
def preprocess_text(text: str, language: str = "en") -> list:
|
|
28
|
+
"""Standard preprocessing for humanities text analysis."""
|
|
29
|
+
# Lowercase
|
|
30
|
+
text = text.lower()
|
|
31
|
+
|
|
32
|
+
# Remove metadata markers (page numbers, headers)
|
|
33
|
+
text = re.sub(r'\[page \d+\]', '', text)
|
|
34
|
+
text = re.sub(r'\n{3,}', '\n\n', text)
|
|
35
|
+
|
|
36
|
+
# Tokenize (simple whitespace + punctuation split)
|
|
37
|
+
tokens = re.findall(r'\b[a-z]+\b', text)
|
|
38
|
+
|
|
39
|
+
# Remove stopwords (customize per corpus!)
|
|
40
|
+
# Standard lists often remove words meaningful in literary analysis
|
|
41
|
+
# e.g., "not", "but", "never" carry sentiment — keep them if relevant
|
|
42
|
+
stopwords = {"the", "a", "an", "is", "are", "was", "were", "in",
|
|
43
|
+
"on", "at", "to", "for", "of", "with", "and", "or"}
|
|
44
|
+
tokens = [t for t in tokens if t not in stopwords and len(t) > 2]
|
|
45
|
+
|
|
46
|
+
return tokens
|
|
47
|
+
|
|
48
|
+
# Word frequency analysis
|
|
49
|
+
def word_frequencies(tokens: list, top_n: int = 50) -> list:
|
|
50
|
+
return Counter(tokens).most_common(top_n)
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### Stylometry (Authorship Analysis)
|
|
54
|
+
|
|
55
|
+
```python
|
|
56
|
+
"""Stylometric features for authorship attribution."""
|
|
57
|
+
|
|
58
|
+
def extract_style_features(text: str) -> dict:
|
|
59
|
+
"""Extract stylistic features from a text."""
|
|
60
|
+
sentences = text.split('.')
|
|
61
|
+
words = text.split()
|
|
62
|
+
chars = list(text)
|
|
63
|
+
|
|
64
|
+
return {
|
|
65
|
+
"avg_sentence_length": len(words) / max(len(sentences), 1),
|
|
66
|
+
"avg_word_length": sum(len(w) for w in words) / max(len(words), 1),
|
|
67
|
+
"vocabulary_richness": len(set(words)) / max(len(words), 1),
|
|
68
|
+
"hapax_ratio": sum(1 for w, c in Counter(words).items() if c == 1) / max(len(set(words)), 1),
|
|
69
|
+
"comma_rate": text.count(',') / max(len(words), 1),
|
|
70
|
+
"semicolon_rate": text.count(';') / max(len(words), 1),
|
|
71
|
+
"question_rate": text.count('?') / max(len(sentences), 1),
|
|
72
|
+
"exclamation_rate": text.count('!') / max(len(sentences), 1),
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Sentiment Analysis for Historical Texts
|
|
77
|
+
|
|
78
|
+
```python
|
|
79
|
+
# Note: Modern NLP sentiment tools are trained on contemporary text.
|
|
80
|
+
# For historical texts, consider:
|
|
81
|
+
# 1. Historical sentiment lexicons (e.g., NRC Emotion Lexicon adapted)
|
|
82
|
+
# 2. Period-specific word lists
|
|
83
|
+
# 3. Manual validation on a sample before scaling
|
|
84
|
+
|
|
85
|
+
from transformers import pipeline
|
|
86
|
+
|
|
87
|
+
# Modern text sentiment (use with caution on historical texts)
|
|
88
|
+
sentiment = pipeline("sentiment-analysis")
|
|
89
|
+
result = sentiment("It was the best of times, it was the worst of times.")
|
|
90
|
+
|
|
91
|
+
# Better: keyword-based approach with custom lexicons
|
|
92
|
+
POSITIVE = {"virtue", "honor", "glory", "triumph", "beauty", "noble"}
|
|
93
|
+
NEGATIVE = {"vice", "shame", "defeat", "ruin", "wretched", "base"}
|
|
94
|
+
|
|
95
|
+
def lexicon_sentiment(tokens: list, pos: set, neg: set) -> float:
|
|
96
|
+
"""Simple lexicon-based sentiment score."""
|
|
97
|
+
pos_count = sum(1 for t in tokens if t in pos)
|
|
98
|
+
neg_count = sum(1 for t in tokens if t in neg)
|
|
99
|
+
total = pos_count + neg_count
|
|
100
|
+
if total == 0:
|
|
101
|
+
return 0.0
|
|
102
|
+
return (pos_count - neg_count) / total
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Topic Modeling
|
|
106
|
+
|
|
107
|
+
```python
|
|
108
|
+
from sklearn.feature_extraction.text import CountVectorizer
|
|
109
|
+
from sklearn.decomposition import LatentDirichletAllocation
|
|
110
|
+
|
|
111
|
+
def run_topic_model(documents: list, n_topics: int = 10,
|
|
112
|
+
n_top_words: int = 15):
|
|
113
|
+
"""LDA topic modeling on a corpus of documents."""
|
|
114
|
+
# Vectorize
|
|
115
|
+
vectorizer = CountVectorizer(max_df=0.95, min_df=2,
|
|
116
|
+
max_features=5000,
|
|
117
|
+
stop_words="english")
|
|
118
|
+
dtm = vectorizer.fit_transform(documents)
|
|
119
|
+
feature_names = vectorizer.get_feature_names_out()
|
|
120
|
+
|
|
121
|
+
# Fit LDA
|
|
122
|
+
lda = LatentDirichletAllocation(n_components=n_topics,
|
|
123
|
+
random_state=42,
|
|
124
|
+
max_iter=50)
|
|
125
|
+
lda.fit(dtm)
|
|
126
|
+
|
|
127
|
+
# Display topics
|
|
128
|
+
topics = {}
|
|
129
|
+
for topic_idx, topic in enumerate(lda.components_):
|
|
130
|
+
top_words = [feature_names[i] for i in topic.argsort()[-n_top_words:]]
|
|
131
|
+
topics[f"Topic {topic_idx}"] = top_words
|
|
132
|
+
print(f"Topic {topic_idx}: {', '.join(top_words)}")
|
|
133
|
+
|
|
134
|
+
return lda, topics, dtm
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
## Network Analysis
|
|
138
|
+
|
|
139
|
+
### Social Network from Letters/Correspondence
|
|
140
|
+
|
|
141
|
+
```python
|
|
142
|
+
import networkx as nx
|
|
143
|
+
|
|
144
|
+
def build_correspondence_network(letters: list) -> nx.Graph:
|
|
145
|
+
"""Build a social network from letter metadata.
|
|
146
|
+
|
|
147
|
+
Args:
|
|
148
|
+
letters: list of dicts with 'sender', 'recipient', 'date', 'location'
|
|
149
|
+
"""
|
|
150
|
+
G = nx.Graph()
|
|
151
|
+
|
|
152
|
+
for letter in letters:
|
|
153
|
+
sender = letter["sender"]
|
|
154
|
+
recipient = letter["recipient"]
|
|
155
|
+
|
|
156
|
+
if G.has_edge(sender, recipient):
|
|
157
|
+
G[sender][recipient]["weight"] += 1
|
|
158
|
+
else:
|
|
159
|
+
G.add_edge(sender, recipient, weight=1)
|
|
160
|
+
|
|
161
|
+
# Compute centrality measures
|
|
162
|
+
centrality = nx.degree_centrality(G)
|
|
163
|
+
betweenness = nx.betweenness_centrality(G)
|
|
164
|
+
|
|
165
|
+
print(f"Network: {G.number_of_nodes()} individuals, "
|
|
166
|
+
f"{G.number_of_edges()} connections")
|
|
167
|
+
print(f"Most connected: {max(centrality, key=centrality.get)}")
|
|
168
|
+
print(f"Most bridging: {max(betweenness, key=betweenness.get)}")
|
|
169
|
+
|
|
170
|
+
return G
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
## Spatial Analysis (GIS for Humanities)
|
|
174
|
+
|
|
175
|
+
Common applications:
|
|
176
|
+
- Mapping historical events or migration patterns
|
|
177
|
+
- Georeferencing historical maps
|
|
178
|
+
- Spatial analysis of literary settings
|
|
179
|
+
|
|
180
|
+
```python
|
|
181
|
+
import folium
|
|
182
|
+
|
|
183
|
+
def create_historical_map(events: list, title: str = "Historical Events"):
|
|
184
|
+
"""Create an interactive map of historical events.
|
|
185
|
+
|
|
186
|
+
Args:
|
|
187
|
+
events: list of dicts with 'name', 'lat', 'lon', 'date', 'description'
|
|
188
|
+
"""
|
|
189
|
+
center_lat = sum(e["lat"] for e in events) / len(events)
|
|
190
|
+
center_lon = sum(e["lon"] for e in events) / len(events)
|
|
191
|
+
|
|
192
|
+
m = folium.Map(location=[center_lat, center_lon], zoom_start=5)
|
|
193
|
+
|
|
194
|
+
for event in events:
|
|
195
|
+
popup = f"<b>{event['name']}</b><br>{event['date']}<br>{event['description']}"
|
|
196
|
+
folium.Marker(
|
|
197
|
+
location=[event["lat"], event["lon"]],
|
|
198
|
+
popup=popup,
|
|
199
|
+
tooltip=event["name"]
|
|
200
|
+
).add_to(m)
|
|
201
|
+
|
|
202
|
+
m.save(f"{title.replace(' ', '_').lower()}.html")
|
|
203
|
+
return m
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
## Key Data Sources for DH
|
|
207
|
+
|
|
208
|
+
| Source | Content | Access |
|
|
209
|
+
|--------|---------|--------|
|
|
210
|
+
| **Project Gutenberg** | 70,000+ free ebooks | gutenberg.org |
|
|
211
|
+
| **HathiTrust** | 17M+ digitized volumes | hathitrust.org |
|
|
212
|
+
| **Internet Archive** | Books, media, web archives | archive.org |
|
|
213
|
+
| **EEBO / ECCO** | Early English books (1475-1800) | Institutional |
|
|
214
|
+
| **Perseus Digital Library** | Greek and Latin classics | perseus.tufts.edu |
|
|
215
|
+
| **Europeana** | European cultural heritage | europeana.eu |
|
|
216
|
+
| **DPLA** | US digital public library | dp.la |
|
|
217
|
+
| **Old Bailey Online** | London criminal trials (1674-1913) | oldbaileyonline.org |
|
|
218
|
+
|
|
219
|
+
## Methodological Considerations
|
|
220
|
+
|
|
221
|
+
1. **Close reading still matters**: Computational methods reveal patterns; interpretation requires humanistic expertise
|
|
222
|
+
2. **Corpus bias**: Digitized collections over-represent certain periods, languages, and genres
|
|
223
|
+
3. **OCR quality**: Historical texts often have high OCR error rates — validate before analysis
|
|
224
|
+
4. **Anachronism**: Modern NLP tools may misinterpret historical language use
|
|
225
|
+
5. **Interdisciplinary collaboration**: Best DH work pairs domain expertise with technical skills
|
|
226
|
+
|
|
227
|
+
## References
|
|
228
|
+
|
|
229
|
+
- Moretti, F. (2013). *Distant Reading*. Verso.
|
|
230
|
+
- Jockers, M. L. (2013). *Macroanalysis: Digital Methods and Literary History*. University of Illinois Press.
|
|
231
|
+
- [Programming Historian](https://programminghistorian.org/) — Free tutorials for DH methods
|
|
232
|
+
- [DH Tools Directory](https://dirtdirectory.org/)
|