longloom 1.0.0__tar.gz

longloom-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 LongLoom Maintainers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

longloom-1.0.0/PKG-INFO

@@ -0,0 +1,358 @@
+Metadata-Version: 2.4
+Name: longloom
+Version: 1.0.0
+Summary: Weave documents of any length through LLM map/merge/repair — any LLM, any storage, measured no-loss
+Author: LongLoom Maintainers
+License: MIT
+Project-URL: Homepage, https://github.com/sumitnemade/longloom
+Project-URL: Documentation, https://github.com/sumitnemade/longloom#readme
+Project-URL: Changelog, https://github.com/sumitnemade/longloom/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/sumitnemade/longloom/issues
+Keywords: llm,long-context,map-reduce,langgraph,chunking,rag,summarization,nvidia,openai,postgresql,mongodb
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: General
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: langgraph>=0.2.0
+Requires-Dist: langchain-core>=0.3.0
+Provides-Extra: tiktoken
+Requires-Dist: tiktoken>=0.7.0; extra == "tiktoken"
+Provides-Extra: nvidia
+Requires-Dist: requests>=2.31; extra == "nvidia"
+Provides-Extra: openai
+Requires-Dist: langchain-openai>=0.2.0; extra == "openai"
+Provides-Extra: sql
+Requires-Dist: sqlalchemy>=2.0; extra == "sql"
+Provides-Extra: postgres
+Requires-Dist: sqlalchemy>=2.0; extra == "postgres"
+Requires-Dist: psycopg2-binary>=2.9; extra == "postgres"
+Provides-Extra: mysql
+Requires-Dist: sqlalchemy>=2.0; extra == "mysql"
+Requires-Dist: pymysql>=1.1; extra == "mysql"
+Provides-Extra: mongodb
+Requires-Dist: pymongo>=4.6; extra == "mongodb"
+Provides-Extra: supabase
+Requires-Dist: supabase>=2.4; extra == "supabase"
+Provides-Extra: databases
+Requires-Dist: sqlalchemy>=2.0; extra == "databases"
+Requires-Dist: psycopg2-binary>=2.9; extra == "databases"
+Requires-Dist: pymysql>=1.1; extra == "databases"
+Requires-Dist: pymongo>=4.6; extra == "databases"
+Requires-Dist: supabase>=2.4; extra == "databases"
+Provides-Extra: all
+Requires-Dist: tiktoken>=0.7.0; extra == "all"
+Requires-Dist: requests>=2.31; extra == "all"
+Requires-Dist: langchain-openai>=0.2.0; extra == "all"
+Requires-Dist: sqlalchemy>=2.0; extra == "all"
+Requires-Dist: psycopg2-binary>=2.9; extra == "all"
+Requires-Dist: pymysql>=1.1; extra == "all"
+Requires-Dist: pymongo>=4.6; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: tiktoken>=0.7.0; extra == "dev"
+Requires-Dist: pymongo>=4.6; extra == "dev"
+Requires-Dist: sqlalchemy>=2.0; extra == "dev"
+Requires-Dist: psycopg2-binary>=2.9; extra == "dev"
+Requires-Dist: pymysql>=1.1; extra == "dev"
+Dynamic: license-file
+
+<div align="center">
+
+# 🧶 LongLoom
+
+**Weave documents of any length through LLM map/merge/repair — without losing a thread.**
+
+[![PyPI](https://img.shields.io/pypi/v/longloom)](https://pypi.org/project/longloom/)
+[![Python](https://img.shields.io/pypi/pyversions/longloom)](https://pypi.org/project/longloom/)
+[![CI](https://github.com/sumitnemade/longloom/actions/workflows/ci.yml/badge.svg)](https://github.com/sumitnemade/longloom/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Typed](https://img.shields.io/badge/typing-typed-blue)](longloom/py.typed)
+[![Tests](https://img.shields.io/badge/tests-50%20passing-brightgreen)](#test-it-yourself)
+
+*Any LLM · Any storage · Measured no-loss · 2 core dependencies*
+
+</div>
+
+---
+
+Your model accepts 8K, 128K, or 200K tokens — your documents don't care.
+Annual reports, BRDs, legal contracts, log archives, codebases: they all blow
+past the window, and naive truncation silently drops the facts you need.
+
+**LongLoom** splits the problem the way humans do — read in chunks, extract,
+combine, condense — as a LangGraph pipeline with **measured** information
+preservation, over **any LLM** you already use and **any storage** you already run.
+
+```text
+              ┌───────────────┐
+ big document ─► CHUNK + STORE ─► every byte persisted (SQLite / Postgres / Mongo / …)
+              └───────┬───────┘
+                      ▼   MAP      one LLM call per chunk — verbatim or summarizing
+              ┌───────────────┐
+              │  map results  │
+              └───────┬───────┘
+                      ▼   MERGE   window-sized hierarchical combine calls
+              ┌───────────────┐           (scales past any single context window)
+              │  merged draft │
+              └───────┬───────┘
+                      ▼   REPAIR  bounded condensation loop, only if still too big
+              ┌───────────────┐
+              │ final result  │  +  verification: every fact accounted for
+              └───────────────┘
+```
+
+## Install
+
+```bash
+pip install "longloom[all]"      # batteries included
+```
+
+or just the core (two dependencies: `langgraph` + `langchain-core`):
+
+```bash
+pip install longloom
+```
+
+Python ≥ 3.10 · Optional extras: `[tiktoken]` `[nvidia]` `[openai]` `[postgres]` `[mysql]` `[mongodb]` `[supabase]` `[databases]`
+
+## 60-second quickstart
+
+```python
+import os
+from longloom import MapReduceProcessor, SQLiteContextStore, init_llm
+
+processor = MapReduceProcessor(
+    llm=init_llm("nvidia", "nvidia/nemotron-3-ultra-550b-a55b",
+                 api_key=os.environ["NVIDIA_API_KEY"],   # your app owns the key
+                 thinking=False),
+    store=SQLiteContextStore("ctx.db"),
+    max_chunk_tokens=3000,
+    max_context_tokens=8192,          # your model's real window
+)
+
+result = processor.process(
+    annual_report,                    # 50K tokens? 500K? doesn't matter
+    task="Summarize the key findings",
+)
+print(result["final_result"])
+print(f"{result['num_chunks']} chunks · {result['total_input_tokens']:,} tokens · "
+      f"{result['repair_attempts']} repairs")
+```
+
+Every run is **persisted and queryable** afterwards:
+
+```python
+processor.search_session(result["session_id"], "revenue")   # which chunks mention it
+processor.retrieve_chunks(result["session_id"])              # original text, byte-faithful
+processor.get_session_info(result["session_id"])             # chunk/token stats
+```
+
+## When facts must not be lost
+
+Summarizing prompts measurably drop facts (we measured 2–10% on messy
+documents — different facts every run). When you need *everything*:
+
+```python
+from longloom import PromptTemplates
+
+processor = MapReduceProcessor(
+    llm=llm,
+    store=store,
+    prompts=PromptTemplates.for_extraction(),   # verbatim line-copy mode
+    ...
+)
+```
+
+**Measured result: 100% fact preservation (41/41 planted facts in a log/JSON/
+email/table/document mix — stable across repeated real-LLM runs on NVIDIA and
+OpenAI).** The trade-off is output length: extraction copies, it doesn't
+compress. Summaries → `DEFAULT_PROMPTS`; completeness → `for_extraction()`.
+
+## Any LLM
+
+```python
+init_llm("nvidia",    "nvidia/nemotron-3-ultra-550b-a55b",
+         api_key=os.environ["NVIDIA_API_KEY"], thinking=False)
+init_llm("openai",    "gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
+init_llm("anthropic", "claude-sonnet-4-20250514", api_key=os.environ["ANTHROPIC_API_KEY"])
+init_llm("groq" | "mistral" | "deepseek" | "together", ..., api_key=...)
+init_llm("ollama", "llama3.1")                               # no key needed
+```
+
+> 🔐 **Keys are constructor arguments, never environment reads.** LongLoom
+> itself never touches `os.environ` for credentials — your application reads
+> the secret (vault, env, wherever) and passes it explicitly. Explicit beats
+> implicit when credentials are involved.
+
+…or bring your own: any LangChain `BaseChatModel`, or a bare function —
+the minimum contract is `str → str`:
+
+```python
+processor = MapReduceProcessor(llm=lambda prompt: my_gateway(prompt), store=store)
+```
+
+Provider quirks are handled for you: NVIDIA reasoning models get real model
+IDs, a `thinking` toggle, and a `content=None` starvation guard; OpenAI
+`extra_body` and temperature restrictions are abstracted away.
+
+## Any storage
+
+```python
+from longloom import create_store
+
+store = create_store("sqlite:///ctx.db")                     # default, single file
+store = create_store("postgresql://user:pw@host/longloom")
+store = create_store("mysql://user:pw@host/longloom")
+store = create_store("mongodb://host:27017/longloom")
+store = create_store("https://myteam.supabase.co")           # + SUPABASE_KEY
+store = create_store("json://data")                          # human-readable JSONL
+store = create_store("memory://")                            # ephemeral
+```
+
+| Backend | Extra | Notes |
+|---|---|---|
+| In-memory | — | zero-dependency default |
+| SQLite | — (stdlib) | inline content or file-backed |
+| JSONL files | — | one readable file per session |
+| **PostgreSQL** | `[postgres]` | SQLAlchemy or psycopg2/psycopg |
+| **MySQL / MariaDB** | `[mysql]` | SQLAlchemy or PyMySQL |
+| **MongoDB** | `[mongodb]` | server-side regex search, `$group` stats |
+| **Supabase** | `[supabase]` | REST via supabase-py — no DB connection needed |
+| **Your own** | — | subclass `BaseContextStore` — implement four methods |
+
+All backends pass the identical contract suite (round-trip, filters, budget,
+upsert idempotency, search, stats, delete).
+
+## Embedding in existing databases
+
+No dedicated database needed — LongLoom lives inside **your** tables with
+**your** column names, next to your application's own data:
+
+```python
+store = create_store(
+    "postgresql://app@host/mydb",        # the database you already run
+    table="app_document_chunks",         # your table
+    columns={                            # your column names
+        "id": "chunk_pk", "content": "text_body",
+        "metadata": "meta_json", "session_id": "sess",
+    },
+    namespace="support-agent",           # several apps share one table safely
+    init_schema=False,                   # your migrations own the DDL
+)
+```
+
+| Option | Meaning | Supported by |
+|---|---|---|
+| `table=` | your table/collection name | SQL, MongoDB, Supabase |
+| `columns=` | logical → physical field mapping | SQL, MongoDB, Supabase |
+| `namespace=` + `namespace_field=` | discriminator so multiple apps share one table | SQL, MongoDB, Supabase |
+| `init_schema=False` | your migrations own the DDL | SQL backends |
+| `SqlContextStore.ddl()` / `table_sql(...)` | generate DDL matching your names, for your migration tooling | SQL, Supabase |
+
+Same table, two apps, zero cross-talk — namespaces isolate sessions even with
+identical session IDs. See `examples/17_embed_existing_database.py`.
+
+## Built for production
+
+| | |
+|---|---|
+| **Hierarchical merge** | Verbatim runs beyond one window still complete — merge calls are grouped and combined recursively (verified: a 45K-token extraction survives an 8K window). |
+| **Retry + pacing** | Exponential backoff for 429/5xx/timeouts, optional min-interval between calls for long matrix runs. |
+| **Bounded repair** | Oversized results condense with a capped attempt count — no infinite loops, no silent truncation. |
+| **Token-count guardrails** | tiktoken exact or zero-dep estimator; chunk sizes clamped so map results are always combinable. |
+| **Typed & versioned** | `py.typed`, semantic versioning, CHANGELOG, per-provider extras. |
+
+### Honest limits
+
+- Map/merge **aggregates**; it doesn't reason globally — cross-chunk conflicts
+  (two versions of the same figure) are preserved side by side, not reconciled.
+- Final results don't cite source chunks (the store does — provenance output is
+  on the roadmap).
+- The map phase is sequential; parallelize externally for very large corpora.
+- tiktoken approximates non-OpenAI tokenizers — keep 2× headroom on chunk sizes.
+
+## Examples — every claim is a runnable script
+
+```bash
+git clone https://github.com/sumitnemade/longloom && cd longloom
+pip install -e ".[dev]"
+export NVIDIA_API_KEY=...        # or OPENAI_API_KEY=...
+python examples/run_all.py       # 17 examples; skips gracefully what's missing
+```
+
+| # | Example | One-liner |
+|---|---|---|
+| 01 | quickstart | minimal pipeline, zero setup |
+| 02 | all storage backends | every store via one URL |
+| 03–06 | NVIDIA · OpenAI · Anthropic · local Ollama | same workflow, four vendors |
+| 07 | custom callable | a bare function as the LLM (routing pattern) |
+| 08–11 | PostgreSQL · MySQL · MongoDB · Supabase | one example per database |
+| 12 | custom backend | a working Redis store in ~40 lines |
+| 13 | advanced | custom prompts, token counters, brace-safety |
+| 14 | **NVIDIA × every database** | real Nemotron over 8 backends |
+| 15 | **OpenAI × every database** | real GPT over 8 backends |
+| 16 | **no-loss extraction** | 41/41 facts on a hostile document mix |
+| 17 | **embed in existing database** | your table, your columns, namespaces |
+| 18 | **enterprise gateway** | corporate LLM gateway + existing DB + tenant namespace |
+
+## Test it yourself
+
+```bash
+pip install -e ".[dev]"
+pytest tests/ -v                       # 50 tests; network-dependent ones self-skip
+NVIDIA_API_KEY=... pytest tests/ -v    # + real NVIDIA, Postgres, MySQL, MongoDB
+OPENAI_API_KEY=... pytest tests/ -v    # + real OpenAI end-to-end
+```
+
+## FAQ
+
+**Do I need LangChain?** Only `langchain-core` (a lightweight dependency).
+Your LLM can even be a plain function.
+
+**What if my output is bigger than the window?** The repair loop condenses it,
+and the merge stage itself groups results hierarchically — oversized inputs are
+the library's job, not yours.
+
+**Is my data sent anywhere?** Only to the LLM provider you configure. Storage
+stays in whatever database *you* chose.
+
+**Why not just use a 200K-token model?** Cost, latency, and attention decay.
+Chunked extraction is cheaper per token and verifiably complete — and LongLoom
+works with *any* window, so you're never locked to a model generation.
+
+## Releasing (maintainers)
+
+```bash
+scripts/release.sh 1.2.0 --push     # tests → bump → changelog → tag → CI publishes to PyPI
+```
+
+One-time setup:
+1. Create the project on pypi.org, then **Trusted Publishing**: owner =
+   your GitHub username/org, repo = `longloom`, workflow = `release.yml`,
+   environment = `pypi` — no API tokens live in the repository.
+2. Push the tag (`--push`) — the release workflow builds, checks metadata,
+   verifies tag↔version, publishes, and attaches artifacts to a GitHub release.
+3. Optional local path: `scripts/release.sh 1.2.0 --publish` with
+   `PYPI_TOKEN` set (TestPyPI: `--testpypi` with `TESTPYPI_TOKEN`).
+
+## Contributing
+
+Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for setup and
+conventions (every fix ships with a test; honest verification is the house
+style). By participating you agree to the
+[Code of Conduct](CODE_OF_CONDUCT.md).
+
+## License
+
+[MIT](LICENSE) — © The LongLoom Maintainers

longloom-1.0.0/README.md

@@ -0,0 +1,289 @@
+<div align="center">
+
+# 🧶 LongLoom
+
+**Weave documents of any length through LLM map/merge/repair — without losing a thread.**
+
+[![PyPI](https://img.shields.io/pypi/v/longloom)](https://pypi.org/project/longloom/)
+[![Python](https://img.shields.io/pypi/pyversions/longloom)](https://pypi.org/project/longloom/)
+[![CI](https://github.com/sumitnemade/longloom/actions/workflows/ci.yml/badge.svg)](https://github.com/sumitnemade/longloom/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![Typed](https://img.shields.io/badge/typing-typed-blue)](longloom/py.typed)
+[![Tests](https://img.shields.io/badge/tests-50%20passing-brightgreen)](#test-it-yourself)
+
+*Any LLM · Any storage · Measured no-loss · 2 core dependencies*
+
+</div>
+
+---
+
+Your model accepts 8K, 128K, or 200K tokens — your documents don't care.
+Annual reports, BRDs, legal contracts, log archives, codebases: they all blow
+past the window, and naive truncation silently drops the facts you need.
+
+**LongLoom** splits the problem the way humans do — read in chunks, extract,
+combine, condense — as a LangGraph pipeline with **measured** information
+preservation, over **any LLM** you already use and **any storage** you already run.
+
+```text
+              ┌───────────────┐
+ big document ─► CHUNK + STORE ─► every byte persisted (SQLite / Postgres / Mongo / …)
+              └───────┬───────┘
+                      ▼   MAP      one LLM call per chunk — verbatim or summarizing
+              ┌───────────────┐
+              │  map results  │
+              └───────┬───────┘
+                      ▼   MERGE   window-sized hierarchical combine calls
+              ┌───────────────┐           (scales past any single context window)
+              │  merged draft │
+              └───────┬───────┘
+                      ▼   REPAIR  bounded condensation loop, only if still too big
+              ┌───────────────┐
+              │ final result  │  +  verification: every fact accounted for
+              └───────────────┘
+```
+
+## Install
+
+```bash
+pip install "longloom[all]"      # batteries included
+```
+
+or just the core (two dependencies: `langgraph` + `langchain-core`):
+
+```bash
+pip install longloom
+```
+
+Python ≥ 3.10 · Optional extras: `[tiktoken]` `[nvidia]` `[openai]` `[postgres]` `[mysql]` `[mongodb]` `[supabase]` `[databases]`
+
+## 60-second quickstart
+
+```python
+import os
+from longloom import MapReduceProcessor, SQLiteContextStore, init_llm
+
+processor = MapReduceProcessor(
+    llm=init_llm("nvidia", "nvidia/nemotron-3-ultra-550b-a55b",
+                 api_key=os.environ["NVIDIA_API_KEY"],   # your app owns the key
+                 thinking=False),
+    store=SQLiteContextStore("ctx.db"),
+    max_chunk_tokens=3000,
+    max_context_tokens=8192,          # your model's real window
+)
+
+result = processor.process(
+    annual_report,                    # 50K tokens? 500K? doesn't matter
+    task="Summarize the key findings",
+)
+print(result["final_result"])
+print(f"{result['num_chunks']} chunks · {result['total_input_tokens']:,} tokens · "
+      f"{result['repair_attempts']} repairs")
+```
+
+Every run is **persisted and queryable** afterwards:
+
+```python
+processor.search_session(result["session_id"], "revenue")   # which chunks mention it
+processor.retrieve_chunks(result["session_id"])              # original text, byte-faithful
+processor.get_session_info(result["session_id"])             # chunk/token stats
+```
+
+## When facts must not be lost
+
+Summarizing prompts measurably drop facts (we measured 2–10% on messy
+documents — different facts every run). When you need *everything*:
+
+```python
+from longloom import PromptTemplates
+
+processor = MapReduceProcessor(
+    llm=llm,
+    store=store,
+    prompts=PromptTemplates.for_extraction(),   # verbatim line-copy mode
+    ...
+)
+```
+
+**Measured result: 100% fact preservation (41/41 planted facts in a log/JSON/
+email/table/document mix — stable across repeated real-LLM runs on NVIDIA and
+OpenAI).** The trade-off is output length: extraction copies, it doesn't
+compress. Summaries → `DEFAULT_PROMPTS`; completeness → `for_extraction()`.
+
+## Any LLM
+
+```python
+init_llm("nvidia",    "nvidia/nemotron-3-ultra-550b-a55b",
+         api_key=os.environ["NVIDIA_API_KEY"], thinking=False)
+init_llm("openai",    "gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
+init_llm("anthropic", "claude-sonnet-4-20250514", api_key=os.environ["ANTHROPIC_API_KEY"])
+init_llm("groq" | "mistral" | "deepseek" | "together", ..., api_key=...)
+init_llm("ollama", "llama3.1")                               # no key needed
+```
+
+> 🔐 **Keys are constructor arguments, never environment reads.** LongLoom
+> itself never touches `os.environ` for credentials — your application reads
+> the secret (vault, env, wherever) and passes it explicitly. Explicit beats
+> implicit when credentials are involved.
+
+…or bring your own: any LangChain `BaseChatModel`, or a bare function —
+the minimum contract is `str → str`:
+
+```python
+processor = MapReduceProcessor(llm=lambda prompt: my_gateway(prompt), store=store)
+```
+
+Provider quirks are handled for you: NVIDIA reasoning models get real model
+IDs, a `thinking` toggle, and a `content=None` starvation guard; OpenAI
+`extra_body` and temperature restrictions are abstracted away.
+
+## Any storage
+
+```python
+from longloom import create_store
+
+store = create_store("sqlite:///ctx.db")                     # default, single file
+store = create_store("postgresql://user:pw@host/longloom")
+store = create_store("mysql://user:pw@host/longloom")
+store = create_store("mongodb://host:27017/longloom")
+store = create_store("https://myteam.supabase.co")           # + SUPABASE_KEY
+store = create_store("json://data")                          # human-readable JSONL
+store = create_store("memory://")                            # ephemeral
+```
+
+| Backend | Extra | Notes |
+|---|---|---|
+| In-memory | — | zero-dependency default |
+| SQLite | — (stdlib) | inline content or file-backed |
+| JSONL files | — | one readable file per session |
+| **PostgreSQL** | `[postgres]` | SQLAlchemy or psycopg2/psycopg |
+| **MySQL / MariaDB** | `[mysql]` | SQLAlchemy or PyMySQL |
+| **MongoDB** | `[mongodb]` | server-side regex search, `$group` stats |
+| **Supabase** | `[supabase]` | REST via supabase-py — no DB connection needed |
+| **Your own** | — | subclass `BaseContextStore` — implement four methods |
+
+All backends pass the identical contract suite (round-trip, filters, budget,
+upsert idempotency, search, stats, delete).
+
+## Embedding in existing databases
+
+No dedicated database needed — LongLoom lives inside **your** tables with
+**your** column names, next to your application's own data:
+
+```python
+store = create_store(
+    "postgresql://app@host/mydb",        # the database you already run
+    table="app_document_chunks",         # your table
+    columns={                            # your column names
+        "id": "chunk_pk", "content": "text_body",
+        "metadata": "meta_json", "session_id": "sess",
+    },
+    namespace="support-agent",           # several apps share one table safely
+    init_schema=False,                   # your migrations own the DDL
+)
+```
+
+| Option | Meaning | Supported by |
+|---|---|---|
+| `table=` | your table/collection name | SQL, MongoDB, Supabase |
+| `columns=` | logical → physical field mapping | SQL, MongoDB, Supabase |
+| `namespace=` + `namespace_field=` | discriminator so multiple apps share one table | SQL, MongoDB, Supabase |
+| `init_schema=False` | your migrations own the DDL | SQL backends |
+| `SqlContextStore.ddl()` / `table_sql(...)` | generate DDL matching your names, for your migration tooling | SQL, Supabase |
+
+Same table, two apps, zero cross-talk — namespaces isolate sessions even with
+identical session IDs. See `examples/17_embed_existing_database.py`.
+
+## Built for production
+
+| | |
+|---|---|
+| **Hierarchical merge** | Verbatim runs beyond one window still complete — merge calls are grouped and combined recursively (verified: a 45K-token extraction survives an 8K window). |
+| **Retry + pacing** | Exponential backoff for 429/5xx/timeouts, optional min-interval between calls for long matrix runs. |
+| **Bounded repair** | Oversized results condense with a capped attempt count — no infinite loops, no silent truncation. |
+| **Token-count guardrails** | tiktoken exact or zero-dep estimator; chunk sizes clamped so map results are always combinable. |
+| **Typed & versioned** | `py.typed`, semantic versioning, CHANGELOG, per-provider extras. |
+
+### Honest limits
+
+- Map/merge **aggregates**; it doesn't reason globally — cross-chunk conflicts
+  (two versions of the same figure) are preserved side by side, not reconciled.
+- Final results don't cite source chunks (the store does — provenance output is
+  on the roadmap).
+- The map phase is sequential; parallelize externally for very large corpora.
+- tiktoken approximates non-OpenAI tokenizers — keep 2× headroom on chunk sizes.
+
+## Examples — every claim is a runnable script
+
+```bash
+git clone https://github.com/sumitnemade/longloom && cd longloom
+pip install -e ".[dev]"
+export NVIDIA_API_KEY=...        # or OPENAI_API_KEY=...
+python examples/run_all.py       # 17 examples; skips gracefully what's missing
+```
+
+| # | Example | One-liner |
+|---|---|---|
+| 01 | quickstart | minimal pipeline, zero setup |
+| 02 | all storage backends | every store via one URL |
+| 03–06 | NVIDIA · OpenAI · Anthropic · local Ollama | same workflow, four vendors |
+| 07 | custom callable | a bare function as the LLM (routing pattern) |
+| 08–11 | PostgreSQL · MySQL · MongoDB · Supabase | one example per database |
+| 12 | custom backend | a working Redis store in ~40 lines |
+| 13 | advanced | custom prompts, token counters, brace-safety |
+| 14 | **NVIDIA × every database** | real Nemotron over 8 backends |
+| 15 | **OpenAI × every database** | real GPT over 8 backends |
+| 16 | **no-loss extraction** | 41/41 facts on a hostile document mix |
+| 17 | **embed in existing database** | your table, your columns, namespaces |
+| 18 | **enterprise gateway** | corporate LLM gateway + existing DB + tenant namespace |
+
+## Test it yourself
+
+```bash
+pip install -e ".[dev]"
+pytest tests/ -v                       # 50 tests; network-dependent ones self-skip
+NVIDIA_API_KEY=... pytest tests/ -v    # + real NVIDIA, Postgres, MySQL, MongoDB
+OPENAI_API_KEY=... pytest tests/ -v    # + real OpenAI end-to-end
+```
+
+## FAQ
+
+**Do I need LangChain?** Only `langchain-core` (a lightweight dependency).
+Your LLM can even be a plain function.
+
+**What if my output is bigger than the window?** The repair loop condenses it,
+and the merge stage itself groups results hierarchically — oversized inputs are
+the library's job, not yours.
+
+**Is my data sent anywhere?** Only to the LLM provider you configure. Storage
+stays in whatever database *you* chose.
+
+**Why not just use a 200K-token model?** Cost, latency, and attention decay.
+Chunked extraction is cheaper per token and verifiably complete — and LongLoom
+works with *any* window, so you're never locked to a model generation.
+
+## Releasing (maintainers)
+
+```bash
+scripts/release.sh 1.2.0 --push     # tests → bump → changelog → tag → CI publishes to PyPI
+```
+
+One-time setup:
+1. Create the project on pypi.org, then **Trusted Publishing**: owner =
+   your GitHub username/org, repo = `longloom`, workflow = `release.yml`,
+   environment = `pypi` — no API tokens live in the repository.
+2. Push the tag (`--push`) — the release workflow builds, checks metadata,
+   verifies tag↔version, publishes, and attaches artifacts to a GitHub release.
+3. Optional local path: `scripts/release.sh 1.2.0 --publish` with
+   `PYPI_TOKEN` set (TestPyPI: `--testpypi` with `TESTPYPI_TOKEN`).
+
+## Contributing
+
+Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for setup and
+conventions (every fix ships with a test; honest verification is the house
+style). By participating you agree to the
+[Code of Conduct](CODE_OF_CONDUCT.md).
+
+## License
+
+[MIT](LICENSE) — © The LongLoom Maintainers