haiku.rag 0.19.3__py3-none-any.whl → 0.23.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- README.md +63 -97
- haiku_rag-0.23.0.dist-info/METADATA +162 -0
- haiku_rag-0.23.0.dist-info/RECORD +6 -0
- haiku_rag-0.19.3.dist-info/METADATA +0 -196
- haiku_rag-0.19.3.dist-info/RECORD +0 -6
- {haiku_rag-0.19.3.dist-info → haiku_rag-0.23.0.dist-info}/WHEEL +0 -0
- {haiku_rag-0.19.3.dist-info → haiku_rag-0.23.0.dist-info}/entry_points.txt +0 -0
- {haiku_rag-0.19.3.dist-info → haiku_rag-0.23.0.dist-info}/licenses/LICENSE +0 -0
README.md
CHANGED
|
@@ -1,22 +1,22 @@
|
|
|
1
1
|
# Haiku RAG
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
`haiku.rag` is a Retrieval-Augmented Generation (RAG) library built to work with LanceDB as a local vector database. It uses LanceDB for storing embeddings and performs semantic (vector) search as well as full-text search combined through native hybrid search with Reciprocal Rank Fusion. Both open-source (Ollama) as well as commercial (OpenAI, VoyageAI) embedding providers are supported.
|
|
3
|
+
Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/).
|
|
6
4
|
|
|
7
5
|
## Features
|
|
8
6
|
|
|
9
|
-
- **
|
|
10
|
-
- **
|
|
11
|
-
- **
|
|
12
|
-
- **
|
|
13
|
-
- **
|
|
14
|
-
- **
|
|
15
|
-
- **
|
|
16
|
-
- **
|
|
17
|
-
- **
|
|
18
|
-
- **MCP server
|
|
19
|
-
- **
|
|
7
|
+
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
|
8
|
+
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
|
9
|
+
- **Question answering** — QA agents with citations (page numbers, section headings)
|
|
10
|
+
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
|
11
|
+
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
|
12
|
+
- **Visual grounding** — View chunks highlighted on original page images
|
|
13
|
+
- **Time travel** — Query the database at any historical point with `--before`
|
|
14
|
+
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
|
15
|
+
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
|
16
|
+
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
|
17
|
+
- **File monitoring** — Watch directories and auto-index on changes
|
|
18
|
+
- **Inspector** — TUI for browsing documents, chunks, and search results
|
|
19
|
+
- **CLI & Python API** — Full functionality from command line or code
|
|
20
20
|
|
|
21
21
|
## Installation
|
|
22
22
|
|
|
@@ -41,113 +41,76 @@ Install only the extras you need. See the [Installation](https://ggozad.github.i
|
|
|
41
41
|
## Quick Start
|
|
42
42
|
|
|
43
43
|
```bash
|
|
44
|
-
#
|
|
45
|
-
haiku-rag add
|
|
46
|
-
haiku-rag add "Your content here" --meta author=alice --meta topic=notes
|
|
47
|
-
haiku-rag add-src document.pdf --meta source=manual
|
|
44
|
+
# Index a PDF
|
|
45
|
+
haiku-rag add-src paper.pdf
|
|
48
46
|
|
|
49
47
|
# Search
|
|
50
|
-
haiku-rag search "
|
|
51
|
-
|
|
52
|
-
# Search with filters
|
|
53
|
-
haiku-rag search "query" --filter "uri LIKE '%.pdf' AND title LIKE '%paper%'"
|
|
54
|
-
|
|
55
|
-
# Ask questions
|
|
56
|
-
haiku-rag ask "Who is the author of haiku.rag?"
|
|
48
|
+
haiku-rag search "attention mechanism"
|
|
57
49
|
|
|
58
50
|
# Ask questions with citations
|
|
59
|
-
haiku-rag ask "
|
|
51
|
+
haiku-rag ask "What datasets were used for evaluation?" --cite
|
|
60
52
|
|
|
61
|
-
# Deep QA
|
|
62
|
-
haiku-rag ask "
|
|
53
|
+
# Deep QA — decomposes complex questions into sub-queries
|
|
54
|
+
haiku-rag ask "How does the proposed method compare to the baseline on MMLU?" --deep
|
|
63
55
|
|
|
64
|
-
#
|
|
65
|
-
haiku-rag
|
|
56
|
+
# Research mode — iterative planning and search
|
|
57
|
+
haiku-rag research "What are the limitations of the approach?" --verbose
|
|
66
58
|
|
|
67
|
-
#
|
|
68
|
-
haiku-rag research
|
|
69
|
-
"What are the main drivers and trends of global temperature anomalies since 1990?" \
|
|
70
|
-
--max-iterations 2 \
|
|
71
|
-
--confidence-threshold 0.8 \
|
|
72
|
-
--max-concurrency 3 \
|
|
73
|
-
--verbose
|
|
59
|
+
# Interactive research — human-in-the-loop with decision points
|
|
60
|
+
haiku-rag research "Compare the approaches discussed" --interactive
|
|
74
61
|
|
|
75
|
-
#
|
|
76
|
-
haiku-rag rebuild
|
|
77
|
-
|
|
78
|
-
# Start server with file monitoring
|
|
62
|
+
# Watch a directory for changes
|
|
79
63
|
haiku-rag serve --monitor
|
|
80
64
|
```
|
|
81
65
|
|
|
82
|
-
|
|
66
|
+
See [Configuration](https://ggozad.github.io/haiku.rag/configuration/) for customization options.
|
|
83
67
|
|
|
84
|
-
## Python
|
|
68
|
+
## Python API
|
|
85
69
|
|
|
86
70
|
```python
|
|
87
71
|
from haiku.rag.client import HaikuRAG
|
|
88
|
-
from haiku.rag.config import Config
|
|
89
|
-
from haiku.rag.graph.agui import stream_graph
|
|
90
|
-
from haiku.rag.graph.research import (
|
|
91
|
-
ResearchContext,
|
|
92
|
-
ResearchDeps,
|
|
93
|
-
ResearchState,
|
|
94
|
-
build_research_graph,
|
|
95
|
-
)
|
|
96
|
-
|
|
97
|
-
async with HaikuRAG("database.lancedb") as client:
|
|
98
|
-
# Add document
|
|
99
|
-
doc = await client.create_document("Your content")
|
|
100
|
-
|
|
101
|
-
# Search (reranking enabled by default)
|
|
102
|
-
results = await client.search("query")
|
|
103
|
-
for chunk, score in results:
|
|
104
|
-
print(f"{score:.3f}: {chunk.content}")
|
|
105
|
-
|
|
106
|
-
# Ask questions
|
|
107
|
-
answer = await client.ask("Who is the author of haiku.rag?")
|
|
108
|
-
print(answer)
|
|
109
72
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
73
|
+
async with HaikuRAG("research.lancedb", create=True) as rag:
|
|
74
|
+
# Index documents
|
|
75
|
+
await rag.create_document_from_source("paper.pdf")
|
|
76
|
+
await rag.create_document_from_source("https://arxiv.org/pdf/1706.03762")
|
|
113
77
|
|
|
114
|
-
#
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
)
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
# Blocking run (final result only)
|
|
126
|
-
report = await graph.run(state=state, deps=deps)
|
|
127
|
-
print(report.title)
|
|
128
|
-
|
|
129
|
-
# Streaming progress (AG-UI events)
|
|
130
|
-
async for event in stream_graph(graph, state, deps):
|
|
131
|
-
if event["type"] == "STEP_STARTED":
|
|
132
|
-
print(f"Starting step: {event['stepName']}")
|
|
133
|
-
elif event["type"] == "ACTIVITY_SNAPSHOT":
|
|
134
|
-
print(f" {event['content']}")
|
|
135
|
-
elif event["type"] == "RUN_FINISHED":
|
|
136
|
-
print("\nResearch complete!\n")
|
|
137
|
-
result = event["result"]
|
|
138
|
-
print(result["title"])
|
|
139
|
-
print(result["executive_summary"])
|
|
78
|
+
# Search — returns chunks with provenance
|
|
79
|
+
results = await rag.search("self-attention")
|
|
80
|
+
for result in results:
|
|
81
|
+
print(f"{result.score:.2f} | p.{result.page_numbers} | {result.content[:100]}")
|
|
82
|
+
|
|
83
|
+
# QA with citations
|
|
84
|
+
answer, citations = await rag.ask("What is the complexity of self-attention?")
|
|
85
|
+
print(answer)
|
|
86
|
+
for cite in citations:
|
|
87
|
+
print(f" [{cite.chunk_id}] p.{cite.page_numbers}: {cite.content[:80]}")
|
|
140
88
|
```
|
|
141
89
|
|
|
90
|
+
For research agents and streaming with [AG-UI](https://docs.ag-ui.com/), see the [Agents docs](https://ggozad.github.io/haiku.rag/agents/).
|
|
91
|
+
|
|
142
92
|
## MCP Server
|
|
143
93
|
|
|
144
94
|
Use with AI assistants like Claude Desktop:
|
|
145
95
|
|
|
146
96
|
```bash
|
|
147
|
-
haiku-rag serve --stdio
|
|
97
|
+
haiku-rag serve --mcp --stdio
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Add to your Claude Desktop configuration:
|
|
101
|
+
|
|
102
|
+
```json
|
|
103
|
+
{
|
|
104
|
+
"mcpServers": {
|
|
105
|
+
"haiku-rag": {
|
|
106
|
+
"command": "haiku-rag",
|
|
107
|
+
"args": ["serve", "--mcp", "--stdio"]
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
}
|
|
148
111
|
```
|
|
149
112
|
|
|
150
|
-
Provides tools for document management and
|
|
113
|
+
Provides tools for document management, search, QA, and research directly in your AI assistant.
|
|
151
114
|
|
|
152
115
|
## Examples
|
|
153
116
|
|
|
@@ -166,7 +129,10 @@ Full documentation at: https://ggozad.github.io/haiku.rag/
|
|
|
166
129
|
- [CLI](https://ggozad.github.io/haiku.rag/cli/) - Command reference
|
|
167
130
|
- [Python API](https://ggozad.github.io/haiku.rag/python/) - Complete API docs
|
|
168
131
|
- [Agents](https://ggozad.github.io/haiku.rag/agents/) - QA agent and multi-agent research
|
|
169
|
-
- [
|
|
170
|
-
- [
|
|
132
|
+
- [Server](https://ggozad.github.io/haiku.rag/server/) - File monitoring, MCP, and AG-UI
|
|
133
|
+
- [MCP](https://ggozad.github.io/haiku.rag/mcp/) - Model Context Protocol integration
|
|
134
|
+
- [Inspector](https://ggozad.github.io/haiku.rag/inspector/) - Database browser TUI
|
|
135
|
+
- [Benchmarks](https://ggozad.github.io/haiku.rag/benchmarks/) - Performance benchmarks
|
|
136
|
+
- [Changelog](https://ggozad.github.io/haiku.rag/changelog/) - Version history
|
|
171
137
|
|
|
172
138
|
mcp-name: io.github.ggozad/haiku-rag
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: haiku.rag
|
|
3
|
+
Version: 0.23.0
|
|
4
|
+
Summary: Opinionated agentic RAG powered by LanceDB, Pydantic AI, and Docling
|
|
5
|
+
Author-email: Yiorgis Gozadinos <ggozadinos@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
License-File: LICENSE
|
|
8
|
+
Keywords: RAG,docling,lancedb,mcp,ml,pydantic-ai,vector-database
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Environment :: Console
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: Operating System :: MacOS
|
|
13
|
+
Classifier: Operating System :: Microsoft :: Windows :: Windows 10
|
|
14
|
+
Classifier: Operating System :: Microsoft :: Windows :: Windows 11
|
|
15
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
18
|
+
Classifier: Typing :: Typed
|
|
19
|
+
Requires-Python: >=3.12
|
|
20
|
+
Requires-Dist: haiku-rag-slim[cohere,docling,inspector,mxbai,voyageai,zeroentropy]==0.23.0
|
|
21
|
+
Provides-Extra: inspector
|
|
22
|
+
Requires-Dist: textual>=1.0.0; extra == 'inspector'
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
|
|
25
|
+
# Haiku RAG
|
|
26
|
+
|
|
27
|
+
Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/).
|
|
28
|
+
|
|
29
|
+
## Features
|
|
30
|
+
|
|
31
|
+
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
|
32
|
+
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
|
33
|
+
- **Question answering** — QA agents with citations (page numbers, section headings)
|
|
34
|
+
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
|
35
|
+
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
|
36
|
+
- **Visual grounding** — View chunks highlighted on original page images
|
|
37
|
+
- **Time travel** — Query the database at any historical point with `--before`
|
|
38
|
+
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
|
39
|
+
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
|
40
|
+
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
|
41
|
+
- **File monitoring** — Watch directories and auto-index on changes
|
|
42
|
+
- **Inspector** — TUI for browsing documents, chunks, and search results
|
|
43
|
+
- **CLI & Python API** — Full functionality from command line or code
|
|
44
|
+
|
|
45
|
+
## Installation
|
|
46
|
+
|
|
47
|
+
**Python 3.12 or newer required**
|
|
48
|
+
|
|
49
|
+
### Full Package (Recommended)
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
uv pip install haiku.rag
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Includes all features: document processing, all embedding providers, and rerankers.
|
|
56
|
+
|
|
57
|
+
### Slim Package (Minimal Dependencies)
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
uv pip install haiku.rag-slim
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Install only the extras you need. See the [Installation](https://ggozad.github.io/haiku.rag/installation/) documentation for available options
|
|
64
|
+
|
|
65
|
+
## Quick Start
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
# Index a PDF
|
|
69
|
+
haiku-rag add-src paper.pdf
|
|
70
|
+
|
|
71
|
+
# Search
|
|
72
|
+
haiku-rag search "attention mechanism"
|
|
73
|
+
|
|
74
|
+
# Ask questions with citations
|
|
75
|
+
haiku-rag ask "What datasets were used for evaluation?" --cite
|
|
76
|
+
|
|
77
|
+
# Deep QA — decomposes complex questions into sub-queries
|
|
78
|
+
haiku-rag ask "How does the proposed method compare to the baseline on MMLU?" --deep
|
|
79
|
+
|
|
80
|
+
# Research mode — iterative planning and search
|
|
81
|
+
haiku-rag research "What are the limitations of the approach?" --verbose
|
|
82
|
+
|
|
83
|
+
# Interactive research — human-in-the-loop with decision points
|
|
84
|
+
haiku-rag research "Compare the approaches discussed" --interactive
|
|
85
|
+
|
|
86
|
+
# Watch a directory for changes
|
|
87
|
+
haiku-rag serve --monitor
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
See [Configuration](https://ggozad.github.io/haiku.rag/configuration/) for customization options.
|
|
91
|
+
|
|
92
|
+
## Python API
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
from haiku.rag.client import HaikuRAG
|
|
96
|
+
|
|
97
|
+
async with HaikuRAG("research.lancedb", create=True) as rag:
|
|
98
|
+
# Index documents
|
|
99
|
+
await rag.create_document_from_source("paper.pdf")
|
|
100
|
+
await rag.create_document_from_source("https://arxiv.org/pdf/1706.03762")
|
|
101
|
+
|
|
102
|
+
# Search — returns chunks with provenance
|
|
103
|
+
results = await rag.search("self-attention")
|
|
104
|
+
for result in results:
|
|
105
|
+
print(f"{result.score:.2f} | p.{result.page_numbers} | {result.content[:100]}")
|
|
106
|
+
|
|
107
|
+
# QA with citations
|
|
108
|
+
answer, citations = await rag.ask("What is the complexity of self-attention?")
|
|
109
|
+
print(answer)
|
|
110
|
+
for cite in citations:
|
|
111
|
+
print(f" [{cite.chunk_id}] p.{cite.page_numbers}: {cite.content[:80]}")
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
For research agents and streaming with [AG-UI](https://docs.ag-ui.com/), see the [Agents docs](https://ggozad.github.io/haiku.rag/agents/).
|
|
115
|
+
|
|
116
|
+
## MCP Server
|
|
117
|
+
|
|
118
|
+
Use with AI assistants like Claude Desktop:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
haiku-rag serve --mcp --stdio
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Add to your Claude Desktop configuration:
|
|
125
|
+
|
|
126
|
+
```json
|
|
127
|
+
{
|
|
128
|
+
"mcpServers": {
|
|
129
|
+
"haiku-rag": {
|
|
130
|
+
"command": "haiku-rag",
|
|
131
|
+
"args": ["serve", "--mcp", "--stdio"]
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Provides tools for document management, search, QA, and research directly in your AI assistant.
|
|
138
|
+
|
|
139
|
+
## Examples
|
|
140
|
+
|
|
141
|
+
See the [examples directory](examples/) for working examples:
|
|
142
|
+
|
|
143
|
+
- **[Interactive Research Assistant](examples/ag-ui-research/)** - Full-stack research assistant with Pydantic AI and AG-UI featuring human-in-the-loop approval and real-time state synchronization
|
|
144
|
+
- **[Docker Setup](examples/docker/)** - Complete Docker deployment with file monitoring and MCP server
|
|
145
|
+
- **[A2A Server](examples/a2a-server/)** - Self-contained A2A protocol server package with conversational agent interface
|
|
146
|
+
|
|
147
|
+
## Documentation
|
|
148
|
+
|
|
149
|
+
Full documentation at: https://ggozad.github.io/haiku.rag/
|
|
150
|
+
|
|
151
|
+
- [Installation](https://ggozad.github.io/haiku.rag/installation/) - Provider setup
|
|
152
|
+
- [Configuration](https://ggozad.github.io/haiku.rag/configuration/) - YAML configuration
|
|
153
|
+
- [CLI](https://ggozad.github.io/haiku.rag/cli/) - Command reference
|
|
154
|
+
- [Python API](https://ggozad.github.io/haiku.rag/python/) - Complete API docs
|
|
155
|
+
- [Agents](https://ggozad.github.io/haiku.rag/agents/) - QA agent and multi-agent research
|
|
156
|
+
- [Server](https://ggozad.github.io/haiku.rag/server/) - File monitoring, MCP, and AG-UI
|
|
157
|
+
- [MCP](https://ggozad.github.io/haiku.rag/mcp/) - Model Context Protocol integration
|
|
158
|
+
- [Inspector](https://ggozad.github.io/haiku.rag/inspector/) - Database browser TUI
|
|
159
|
+
- [Benchmarks](https://ggozad.github.io/haiku.rag/benchmarks/) - Performance benchmarks
|
|
160
|
+
- [Changelog](https://ggozad.github.io/haiku.rag/changelog/) - Version history
|
|
161
|
+
|
|
162
|
+
mcp-name: io.github.ggozad/haiku-rag
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
README.md,sha256=aFW4_ghsf-SUJzlYFq9RC2NO5nu7BFjqG6n5-cRYcyQ,5287
|
|
2
|
+
haiku_rag-0.23.0.dist-info/METADATA,sha256=txl5cEtHupw_PJPl3k-a4hj-s-7aYkLFXlKc4voHBmI,6279
|
|
3
|
+
haiku_rag-0.23.0.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
|
|
4
|
+
haiku_rag-0.23.0.dist-info/entry_points.txt,sha256=G1U3nAkNd5YDYd4v0tuYFbriz0i-JheCsFuT9kIoGCI,48
|
|
5
|
+
haiku_rag-0.23.0.dist-info/licenses/LICENSE,sha256=eXZrWjSk9PwYFNK9yUczl3oPl95Z4V9UXH7bPN46iPo,1065
|
|
6
|
+
haiku_rag-0.23.0.dist-info/RECORD,,
|
|
@@ -1,196 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: haiku.rag
|
|
3
|
-
Version: 0.19.3
|
|
4
|
-
Summary: Agentic Retrieval Augmented Generation (RAG) with LanceDB
|
|
5
|
-
Author-email: Yiorgis Gozadinos <ggozadinos@gmail.com>
|
|
6
|
-
License: MIT
|
|
7
|
-
License-File: LICENSE
|
|
8
|
-
Keywords: RAG,lancedb,mcp,ml,vector-database
|
|
9
|
-
Classifier: Development Status :: 4 - Beta
|
|
10
|
-
Classifier: Environment :: Console
|
|
11
|
-
Classifier: Intended Audience :: Developers
|
|
12
|
-
Classifier: Operating System :: MacOS
|
|
13
|
-
Classifier: Operating System :: Microsoft :: Windows :: Windows 10
|
|
14
|
-
Classifier: Operating System :: Microsoft :: Windows :: Windows 11
|
|
15
|
-
Classifier: Operating System :: POSIX :: Linux
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
18
|
-
Classifier: Typing :: Typed
|
|
19
|
-
Requires-Python: >=3.12
|
|
20
|
-
Requires-Dist: haiku-rag-slim[cohere,docling,inspector,mxbai,voyageai,zeroentropy]==0.19.3
|
|
21
|
-
Provides-Extra: inspector
|
|
22
|
-
Requires-Dist: textual>=1.0.0; extra == 'inspector'
|
|
23
|
-
Description-Content-Type: text/markdown
|
|
24
|
-
|
|
25
|
-
# Haiku RAG
|
|
26
|
-
|
|
27
|
-
Retrieval-Augmented Generation (RAG) library built on LanceDB.
|
|
28
|
-
|
|
29
|
-
`haiku.rag` is a Retrieval-Augmented Generation (RAG) library built to work with LanceDB as a local vector database. It uses LanceDB for storing embeddings and performs semantic (vector) search as well as full-text search combined through native hybrid search with Reciprocal Rank Fusion. Both open-source (Ollama) as well as commercial (OpenAI, VoyageAI) embedding providers are supported.
|
|
30
|
-
|
|
31
|
-
## Features
|
|
32
|
-
|
|
33
|
-
- **Local LanceDB**: No external servers required, supports also LanceDB cloud storage, S3, Google Cloud & Azure
|
|
34
|
-
- **Multiple embedding providers**: Ollama, LM Studio, VoyageAI, OpenAI, vLLM
|
|
35
|
-
- **Multiple QA providers**: Any provider/model supported by Pydantic AI (Ollama, LM Studio, OpenAI, Anthropic, etc.)
|
|
36
|
-
- **Native hybrid search**: Vector + full-text search with native LanceDB RRF reranking
|
|
37
|
-
- **Reranking**: Default search result reranking with MixedBread AI, Cohere, Zero Entropy, or vLLM
|
|
38
|
-
- **Question answering**: Built-in QA agents on your documents
|
|
39
|
-
- **Research graph (multi‑agent)**: Plan → Search → Evaluate → Synthesize with agentic AI
|
|
40
|
-
- **File monitoring**: Auto-index files when run as server
|
|
41
|
-
- **CLI & Python API**: Use from command line or Python
|
|
42
|
-
- **MCP server**: Expose as tools for AI assistants
|
|
43
|
-
- **Flexible document processing**: Local (docling) or remote (docling-serve) processing
|
|
44
|
-
|
|
45
|
-
## Installation
|
|
46
|
-
|
|
47
|
-
**Python 3.12 or newer required**
|
|
48
|
-
|
|
49
|
-
### Full Package (Recommended)
|
|
50
|
-
|
|
51
|
-
```bash
|
|
52
|
-
uv pip install haiku.rag
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
Includes all features: document processing, all embedding providers, and rerankers.
|
|
56
|
-
|
|
57
|
-
### Slim Package (Minimal Dependencies)
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
uv pip install haiku.rag-slim
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
Install only the extras you need. See the [Installation](https://ggozad.github.io/haiku.rag/installation/) documentation for available options
|
|
64
|
-
|
|
65
|
-
## Quick Start
|
|
66
|
-
|
|
67
|
-
```bash
|
|
68
|
-
# Add documents
|
|
69
|
-
haiku-rag add "Your content here"
|
|
70
|
-
haiku-rag add "Your content here" --meta author=alice --meta topic=notes
|
|
71
|
-
haiku-rag add-src document.pdf --meta source=manual
|
|
72
|
-
|
|
73
|
-
# Search
|
|
74
|
-
haiku-rag search "query"
|
|
75
|
-
|
|
76
|
-
# Search with filters
|
|
77
|
-
haiku-rag search "query" --filter "uri LIKE '%.pdf' AND title LIKE '%paper%'"
|
|
78
|
-
|
|
79
|
-
# Ask questions
|
|
80
|
-
haiku-rag ask "Who is the author of haiku.rag?"
|
|
81
|
-
|
|
82
|
-
# Ask questions with citations
|
|
83
|
-
haiku-rag ask "Who is the author of haiku.rag?" --cite
|
|
84
|
-
|
|
85
|
-
# Deep QA (multi-agent question decomposition)
|
|
86
|
-
haiku-rag ask "Who is the author of haiku.rag?" --deep --cite
|
|
87
|
-
|
|
88
|
-
# Deep QA with verbose output
|
|
89
|
-
haiku-rag ask "Who is the author of haiku.rag?" --deep --verbose
|
|
90
|
-
|
|
91
|
-
# Multi‑agent research (iterative plan/search/evaluate)
|
|
92
|
-
haiku-rag research \
|
|
93
|
-
"What are the main drivers and trends of global temperature anomalies since 1990?" \
|
|
94
|
-
--max-iterations 2 \
|
|
95
|
-
--confidence-threshold 0.8 \
|
|
96
|
-
--max-concurrency 3 \
|
|
97
|
-
--verbose
|
|
98
|
-
|
|
99
|
-
# Rebuild database (re-chunk and re-embed all documents)
|
|
100
|
-
haiku-rag rebuild
|
|
101
|
-
|
|
102
|
-
# Start server with file monitoring
|
|
103
|
-
haiku-rag serve --monitor
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
To customize settings, create a `haiku.rag.yaml` config file (see [Configuration](https://ggozad.github.io/haiku.rag/configuration/)).
|
|
107
|
-
|
|
108
|
-
## Python Usage
|
|
109
|
-
|
|
110
|
-
```python
|
|
111
|
-
from haiku.rag.client import HaikuRAG
|
|
112
|
-
from haiku.rag.config import Config
|
|
113
|
-
from haiku.rag.graph.agui import stream_graph
|
|
114
|
-
from haiku.rag.graph.research import (
|
|
115
|
-
ResearchContext,
|
|
116
|
-
ResearchDeps,
|
|
117
|
-
ResearchState,
|
|
118
|
-
build_research_graph,
|
|
119
|
-
)
|
|
120
|
-
|
|
121
|
-
async with HaikuRAG("database.lancedb") as client:
|
|
122
|
-
# Add document
|
|
123
|
-
doc = await client.create_document("Your content")
|
|
124
|
-
|
|
125
|
-
# Search (reranking enabled by default)
|
|
126
|
-
results = await client.search("query")
|
|
127
|
-
for chunk, score in results:
|
|
128
|
-
print(f"{score:.3f}: {chunk.content}")
|
|
129
|
-
|
|
130
|
-
# Ask questions
|
|
131
|
-
answer = await client.ask("Who is the author of haiku.rag?")
|
|
132
|
-
print(answer)
|
|
133
|
-
|
|
134
|
-
# Ask questions with citations
|
|
135
|
-
answer = await client.ask("Who is the author of haiku.rag?", cite=True)
|
|
136
|
-
print(answer)
|
|
137
|
-
|
|
138
|
-
# Multi‑agent research pipeline (Plan → Search → Evaluate → Synthesize)
|
|
139
|
-
# Graph settings (provider, model, max_iterations, etc.) come from config
|
|
140
|
-
graph = build_research_graph(config=Config)
|
|
141
|
-
question = (
|
|
142
|
-
"What are the main drivers and trends of global temperature "
|
|
143
|
-
"anomalies since 1990?"
|
|
144
|
-
)
|
|
145
|
-
context = ResearchContext(original_question=question)
|
|
146
|
-
state = ResearchState.from_config(context=context, config=Config)
|
|
147
|
-
deps = ResearchDeps(client=client)
|
|
148
|
-
|
|
149
|
-
# Blocking run (final result only)
|
|
150
|
-
report = await graph.run(state=state, deps=deps)
|
|
151
|
-
print(report.title)
|
|
152
|
-
|
|
153
|
-
# Streaming progress (AG-UI events)
|
|
154
|
-
async for event in stream_graph(graph, state, deps):
|
|
155
|
-
if event["type"] == "STEP_STARTED":
|
|
156
|
-
print(f"Starting step: {event['stepName']}")
|
|
157
|
-
elif event["type"] == "ACTIVITY_SNAPSHOT":
|
|
158
|
-
print(f" {event['content']}")
|
|
159
|
-
elif event["type"] == "RUN_FINISHED":
|
|
160
|
-
print("\nResearch complete!\n")
|
|
161
|
-
result = event["result"]
|
|
162
|
-
print(result["title"])
|
|
163
|
-
print(result["executive_summary"])
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
## MCP Server
|
|
167
|
-
|
|
168
|
-
Use with AI assistants like Claude Desktop:
|
|
169
|
-
|
|
170
|
-
```bash
|
|
171
|
-
haiku-rag serve --stdio
|
|
172
|
-
```
|
|
173
|
-
|
|
174
|
-
Provides tools for document management and search directly in your AI assistant.
|
|
175
|
-
|
|
176
|
-
## Examples
|
|
177
|
-
|
|
178
|
-
See the [examples directory](examples/) for working examples:
|
|
179
|
-
|
|
180
|
-
- **[Interactive Research Assistant](examples/ag-ui-research/)** - Full-stack research assistant with Pydantic AI and AG-UI featuring human-in-the-loop approval and real-time state synchronization
|
|
181
|
-
- **[Docker Setup](examples/docker/)** - Complete Docker deployment with file monitoring and MCP server
|
|
182
|
-
- **[A2A Server](examples/a2a-server/)** - Self-contained A2A protocol server package with conversational agent interface
|
|
183
|
-
|
|
184
|
-
## Documentation
|
|
185
|
-
|
|
186
|
-
Full documentation at: https://ggozad.github.io/haiku.rag/
|
|
187
|
-
|
|
188
|
-
- [Installation](https://ggozad.github.io/haiku.rag/installation/) - Provider setup
|
|
189
|
-
- [Configuration](https://ggozad.github.io/haiku.rag/configuration/) - YAML configuration
|
|
190
|
-
- [CLI](https://ggozad.github.io/haiku.rag/cli/) - Command reference
|
|
191
|
-
- [Python API](https://ggozad.github.io/haiku.rag/python/) - Complete API docs
|
|
192
|
-
- [Agents](https://ggozad.github.io/haiku.rag/agents/) - QA agent and multi-agent research
|
|
193
|
-
- [MCP Server](https://ggozad.github.io/haiku.rag/mcp/) - Model Context Protocol integration
|
|
194
|
-
- [Benchmarks](https://ggozad.github.io/haiku.rag/benchmarks/) - Performance Benchmarks
|
|
195
|
-
|
|
196
|
-
mcp-name: io.github.ggozad/haiku-rag
|
|
@@ -1,6 +0,0 @@
|
|
|
1
|
-
README.md,sha256=3Y4bUYJ-gnWPH_zeCdHK_NZcej9JvA2JdnKoSY0eu6o,6377
|
|
2
|
-
haiku_rag-0.19.3.dist-info/METADATA,sha256=BUpAqkIkKsZTQ1mGopYaSKlvlUC6cRBQtpvLfz-5h5M,7338
|
|
3
|
-
haiku_rag-0.19.3.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
|
|
4
|
-
haiku_rag-0.19.3.dist-info/entry_points.txt,sha256=G1U3nAkNd5YDYd4v0tuYFbriz0i-JheCsFuT9kIoGCI,48
|
|
5
|
-
haiku_rag-0.19.3.dist-info/licenses/LICENSE,sha256=eXZrWjSk9PwYFNK9yUczl3oPl95Z4V9UXH7bPN46iPo,1065
|
|
6
|
-
haiku_rag-0.19.3.dist-info/RECORD,,
|
|
File without changes
|
|
File without changes
|
|
File without changes
|