cognee 1.6.2__tar.gz → 1.6.4.dev0__tar.gz

cognee-1.6.4.dev0/.agents/skills/cognee-improve-sessions/SKILL.md

@@ -0,0 +1,229 @@
+---
+name: cognee-improve-sessions
+description: Use when working with cognee's session memory or improve() — storing conversation turns, agent traces and feedback with session_id, bridging sessions into the permanent graph, reading an ImproveResult, understanding why an improve stage was skipped, already_completed or lock_held, or tuning the IMPROVE_* settings.
+---
+
+# Session memory and improve()
+
+cognee has two kinds of memory:
+
+- **Session memory**: a fast cache of conversation turns, agent traces, and
+  feedback, keyed by `session_id`. Writing is instant, with no LLM
+  extraction.
+- **The permanent graph**: what `remember()` builds without a session.
+
+`improve()` connects them. It bridges session content into the graph and
+enriches the graph itself. `remember()` calls it automatically, so most
+users never call it directly.
+
+```python
+import cognee
+
+# Session write: returns immediately; improve() bridges it in the background
+await cognee.remember("User prefers dark mode.", session_id="chat_1")
+
+# Session-aware query: session cache first, then the graph
+results = await cognee.recall("What does the user prefer?", session_id="chat_1")
+
+# Bridge sessions into a dataset's graph explicitly
+result = await cognee.improve(dataset="main_dataset", session_ids=["chat_1"])
+print(result.status, result.stage_summary())
+
+await cognee.wait_for_background_tasks()  # before a script exits
+```
+
+## Use it
+
+### Writing session memory
+
+| What | How |
+|---|---|
+| A fact or note | `remember(text, session_id=...)` (stored as a Q&A entry with the text as the answer) |
+| A Q&A turn | `recall(query, session_id=...)` with a completion search type saves the turn itself; or `remember(cognee.QAEntry(question=..., answer=...), session_id=...)` |
+| An agent step | `remember(cognee.TraceEntry(origin_function=..., status="success", ...), session_id=...)`, or the `@cognee.agent_memory(save_session_traces=True)` decorator |
+| Feedback on an answer | `remember(cognee.FeedbackEntry(qa_id=..., feedback_score=...), session_id=...)` or `cognee.session.add_feedback(session_id, qa_id, feedback_text=..., feedback_score=...)` |
+| Read a session | `cognee.session.get_session(session_id, last_n=...)` |
+
+Requirements: `CACHING=true` (default). The cache backend is
+`CACHE_BACKEND`, one of `sqlite` (default), `postgres`, `redis`, `fs`,
+`tapes`. Sessions expire after `SESSION_TTL_SECONDS` (default 7 days).
+
+### What improve() does: nine stages, in order
+
+Every run goes through the same ordered stages
+(`cognee/modules/improve/registry.py`). Each stage checks a gate before it
+spends any LLM or embedding cost, and reports one `StageResult`.
+
+| # | Stage | What it does | Runs when |
+|---|---|---|---|
+| 1 | `feedback_weights` | Adjusts the weight of graph elements that rated answers used | `session_ids` given; adapter supports feedback weights |
+| 2 | `persist_session_qa` | Turns session Q&A into graph content (node set `user_sessions_from_cache`) | `session_ids` given. **The only fatal stage** |
+| 3 | `persist_agent_traces` | Turns agent-trace feedback into graph content | `session_ids` given |
+| 4 | `extract_agent_context` | Drafts agent-profile lessons from traces | `session_ids`, `CACHING` + `AUTO_FEEDBACK`, an LLM |
+| 5 | `distill_sessions` | Distills session learnings into the graph | `session_ids`, an LLM |
+| 6 | `update_user_preferences` | Folds rated turns into per-user preferences | `session_ids`, `PERSONALIZATION_ENABLED=true` (default false) |
+| 7 | `build_truth_subspace` | Builds the truth subspace from distilled learnings | `session_ids`, `build_truth_subspace=True`, a Ladybug graph |
+| 8 | `triplet_enrichment` | Triplet embeddings over the graph (memify) | `TRIPLET_EMBEDDING=true` (**default false**), or custom tasks/data passed |
+| 9 | `global_context_index` | Bucket and root summaries for global questions | `build_global_context_index=True`, an LLM |
+
+Stages 1–7 need `session_ids`; 8 and 9 work on the graph alone. The order
+matters: 4 feeds 5, 5 feeds 7, and 7 runs before 8.
+
+### Reading the result
+
+`improve()` returns an `ImproveResult`, and so do `POST /api/v1/improve`, the
+CLI, and `RememberResult.improve`.
+
+- `result.status`: `completed`, `errored` (any stage errored), `skipped`
+  (every stage skipped), or `running` (background, not finished).
+- `result.stages`: one `StageResult` per stage, with `stage`, `status`
+  (`completed` / `already_completed` / `skipped` / `errored`), `reason`,
+  `error`, `counts`, `duration_ms`.
+- `result.stage("distill_sessions")`, `result.stage_summary()`,
+  `result.lock_held`, `result.rerun_requested`, `result.rerun_passes`.
+- `await result.wait()` finishes a background run (no-op otherwise).
+
+**Skip and no-op reasons:**
+
+| Reason | Meaning / fix |
+|---|---|
+| `no_session_ids` | Session stage, no `session_ids` passed |
+| `disabled_by_config` | Listed in `IMPROVE_STAGES_DISABLED` |
+| `triplet_embedding_disabled` | Set `TRIPLET_EMBEDDING=true` to enable stage 8 |
+| `opt_in_disabled` | Pass `build_truth_subspace=True` / `build_global_context_index=True` |
+| `personalization_disabled` | Set `PERSONALIZATION_ENABLED=true` |
+| `auto_feedback_disabled` | Stage 4 needs `CACHING=true` and `AUTO_FEEDBACK=true` |
+| `no_llm_configured` | Stages 4, 5, 9 draft text with an LLM; none configured |
+| `backend_unsupported` | The graph adapter lacks the feature (feedback weights, truth subspace) |
+| `session_manager_unavailable` | The session cache is not reachable |
+| `lock_held` | Another improve for the same dataset or session is running (below) |
+| `aborted_by_fatal_stage` | Stage 2 failed, so the rest did not run |
+| `budget_exhausted` | An earlier stage failed because the LLM budget is exhausted (a 402); the rest would fail the same way. Top up, then run improve again |
+| `no_new_entries`, `no_new_trace_steps`, `no_writes_since_last_improve` | With status `already_completed`: nothing new since the last run |
+
+### How remember() triggers improve
+
+- **Without a session:** `add`, then `cognify`, then a foreground `improve()`.
+  Its outcome is on `result.improve` / `result.improve_error`. A failed
+  improve never marks the remember as errored.
+- **With `session_id`:** the text is cached, then a background
+  `improve(dataset, session_ids=[session_id])` starts if the debounce allows
+  it. `result.improve` fills in only after `await result` or
+  `wait_for_background_tasks()`.
+- `self_improvement=False` turns it off per call; `IMPROVE_AUTO_ENABLED=false`
+  turns it off everywhere and overrides `self_improvement=True`.
+- An application embedding cognee can decline one automatic improve before it
+  starts: `cognee.modules.improve.register_auto_improve_admission(check)`
+  registers one async check that `remember()` awaits on both paths. Returning
+  a reason string skips the improve and sets `result.improve_skipped`
+  (`result.improve` stays `None`); the data is stored either way. A check that
+  raises allows the improve. Explicit `improve()` calls are never gated.
+
+### Settings (`IMPROVE_*`, `cognee/modules/improve/config.py`)
+
+| Env var | Default | Meaning |
+|---|---|---|
+| `IMPROVE_AUTO_ENABLED` | `true` | Automatic improve after `remember()` |
+| `IMPROVE_DEBOUNCE_ENTRIES` | `1` | Session auto-improve fires after this many new entries |
+| `IMPROVE_DEBOUNCE_SECONDS` | `0` | ...or this many seconds since the last one. Seconds alone (entries left at 1) means time-only |
+| `IMPROVE_STAGES_DISABLED` | empty | CSV of stage names to skip |
+| `IMPROVE_FEEDBACK_ALPHA` | `0.1` | Feedback learning rate, in (0, 1] |
+
+There is no debounce timer: held-back entries wait for the next
+`remember()` with that session, or an explicit `improve()`.
+
+## Pitfalls
+
+- **A plain `improve(dataset)` often does nothing.** Without `session_ids`,
+  only stages 8 and 9 can run, and both are off by default. Result: every
+  stage `skipped`. That is expected, not an error.
+- **Typed entries do not auto-improve.** `remember(QAEntry/TraceEntry/
+  FeedbackEntry, session_id=...)` stores the entry but never starts an
+  improve. Call `improve(session_ids=[...])` yourself.
+- **`lock_held` does not wait.** Improves for the same dataset or session
+  run one at a time: a second call returns at once with every stage
+  `skipped: lock_held`. If it shares a session with the running one, it sets
+  `rerun_requested=True` and the holder runs up to 2 extra passes (3 passes
+  in total; see `rerun_passes`). The bound is `IMPROVE_MAX_RERUN_PASSES`, a
+  constant in `cognee/api/v1/improve/improve.py`, not an env var. The lock is per process only; multiple API workers do not
+  share it.
+- **Sessions bridge once.** Q&A and trace persistence are tracked per user
+  and session, not per dataset, so bridging a session into dataset A and
+  then into dataset B persists no new Q&A/traces into B. Distillation is
+  tracked per (session, dataset) and still runs into B.
+- **`IMPROVE_STAGES_DISABLED` is validated.** An unknown stage name, or
+  `persist_session_qa` (fatal, cannot be disabled), raises `ValueError`, and
+  the API server refuses to start.
+- **Session writes with the cache off.** `remember(text, session_id=...)`
+  with `CACHING=false` only logs a warning and stores nothing; typed entries
+  raise `RuntimeError`. An explicit `improve(session_ids=...)` then fails in
+  the fatal stage.
+- **Fatal stage failure.** If stage 2 errors, `improve()` raises (HTTP 409)
+  and the error carries `.improve_result`. In background mode it is recorded
+  on `result.error` instead. Other stages failing only mark themselves
+  `errored`; HTTP still returns 200, so check `status`.
+- **Remote mode.** After `cognee.serve(url)`, a background improve is
+  fire-and-forget: `status` stays `running` and there is no polling.
+- `cognee/modules/session_bridge/` is gone (only stale `__pycache__` may be
+  left). The bridging now lives in the improve stages; some test names still
+  say "session_bridge".
+
+## How it works
+
+`improve()` resolves the dataset once (creating it if the name is new),
+claims the improve lock for `dataset:<id>` plus every
+`session:<user>:<session_id>`, probes the graph adapter's capabilities, and
+runs the stages with one frozen `ImproveRunInputs`. Each stage is a gate
+plus a call into existing pipelines plus a result mapping. Stages never own
+retries or ordering.
+
+Watermarks keep repeat runs cheap:
+
+- Session stages track how many entries were already persisted, per user
+  and session.
+- Stage 8 compares the last completed enrichment against later write
+  pipelines for the dataset. A `node_name` or custom-task run bypasses that
+  check.
+
+The improve operation row is written when the run finishes: `failed` if any
+stage errored (a retry is never gated off), `noop` if nothing ran.
+
+- Orchestrator: `cognee/api/v1/improve/improve.py` (HTTP:
+  `routers/get_improve_router.py`; CLI: `cognee/cli/commands/improve_command.py`)
+- Stages, registry, results: `cognee/modules/improve/` (`stages.py`,
+  `registry.py`, `stage.py`, `result.py`, `inputs.py`, `capabilities.py`,
+  `config.py`, `graph_changes.py`, `constants.py`)
+- Lock: `cognee/infrastructure/locks/session_lock.py`
+- Session store and watermarks: `cognee/infrastructure/session/`
+  (`session_manager.py`, `session_persist_watermark.py`,
+  `feedback_detection.py`); cache backends in
+  `cognee/infrastructure/databases/cache/`
+- Auto-improve from remember: `cognee/api/v1/remember/remember.py`,
+  `cognee/api/v1/remember/auto_improve_debounce.py`
+- Entry types: `cognee/memory/entries.py`
+- Background tasks: `cognee/infrastructure/background_tasks.py`
+
+Examples: `examples/guides/improve_quickstart.py`, `sessions.py`,
+`session_distillation.py`, `global_context_index.py`,
+`agent_memory_quickstart.py`, and
+`examples/advanced_guides/remember_recall_improve_example.py`.
+
+## Extending it
+
+Adding a stage:
+
+1. Subclass `BaseStage` in `cognee/modules/improve/stages.py`. Set `name`,
+   `needs_sessions`, and `fatal` (leave it `False`; exactly one fatal stage
+   is enforced at import). Implement `gate()` (return a skip reason
+   constant, or `None`, before any LLM/embedding cost) and `run()` (call
+   existing pipeline code, return a `StageResult`).
+2. Insert it into `DEFAULT_STAGES` in `registry.py` at the right position.
+   The order is pinned by `cognee/tests/unit/modules/improve/test_registry_order.py`;
+   update it deliberately.
+3. If it can re-run cheaply, give it a watermark so a repeat run reports
+   `already_completed`.
+4. If it writes graph data under a new pipeline name, add that name to
+   `WRITE_PIPELINE_NAMES` in `graph_changes.py`, or stage 8 will not notice
+   the change.
+5. Tests: `cognee/tests/unit/modules/improve/` (gates, results, config) and
+   `cognee/tests/unit/api/v1/improve/` (orchestration, rerun, router).

cognee-1.6.4.dev0/.agents/skills/cognee-ingestion/SKILL.md

@@ -0,0 +1,209 @@
+---
+name: cognee-ingestion
+description: Use when putting data into cognee memory with remember() — choosing inputs (text, files, folders, URLs, repos, databases), datasets and node_sets, loaders, ontologies, the graph extractor (LLM or GLiNER), chunking, dry-run cost estimates, or when remember() raises on a keyword argument.
+---
+
+# Ingest data with remember()
+
+`remember()` is cognee's ingestion API. One call stores the data, builds the
+knowledge graph, and enriches it. Use it for all ingestion; every option in
+this skill is a `remember()` argument unless it says otherwise.
+
+```python
+import cognee
+
+result = await cognee.remember("Einstein was born in Ulm.")  # text
+result = await cognee.remember(
+    ["./notes.md", "./report.pdf"],  # files
+    dataset_name="research",
+)
+print(result.status, result.dataset_id)  # "completed", UUID
+```
+
+All cognee functions are async. Without `dataset_name` data goes to
+`main_dataset`. Needs `LLM_API_KEY` unless you use the GLiNER extractor
+(below).
+
+## Use it
+
+### Inputs
+
+`data` accepts a string, a list of strings, file paths (absolute, `file://`,
+`s3://`), http(s) URLs, binary streams, or a list mixing them.
+
+- **URLs** are fetched and scraped (needs `ALLOW_HTTP_REQUESTS=true`, the
+  default).
+- **Folders** are ingested file by file. A folder that looks like a code
+  project, or a GitHub/GitLab URL, becomes one code repository (needs `git`
+  on PATH).
+- **Code files** (`.py`, `.ts`, `.go`, …) go down the code-graph route: a
+  deterministic graph, no LLM calls, searchable only with
+  `SearchType.CODE`. To index a whole repository explicitly, pass
+  `content_type="code"`.
+- **Databases and dlt sources**: a SQL connection string, a dlt
+  `DltResource` / `DltSource`, or a CSV. dlt is a core dependency, so no
+  extra is needed (`cognee[dlt]` is an empty compatibility extra). Options: `primary_key` (default `"id"`),
+  `write_disposition` (`"replace"` default, or `"append"`), `query`,
+  `max_rows_per_table`.
+- **Skill playbooks** (`SKILL.md` files): `content_type="skills"`; ingests
+  into the target dataset (default `main_dataset`), so pass `dataset_name`
+  to keep skills in their own dataset.
+
+### Where the data goes
+
+| Argument | What it does |
+|---|---|
+| `dataset_name` / `dataset_id` | Target dataset. `dataset_id` wins. A dataset is the unit of permissions and isolation. |
+| `node_set=["AI", "FinTech"]` | Tags the data so recall can filter to it later with `recall(..., node_name=["AI"])`. |
+| `session_id="chat_1"` | Writes to the fast session cache instead of the graph; `improve()` bridges it into the graph in the background. See the `cognee-improve-sessions` skill. Requires `CACHING=true`. |
+
+### How the graph is built
+
+| Argument | What it does |
+|---|---|
+| `extractor` | `"llm"` or `"gliner_demo"` (alias `"gliner"`). Default is `GRAPH_EXTRACTOR=auto`: the LLM when an API key is configured, otherwise GLiNER. |
+| `graph_model=MyModel` | Extract into your own DataPoint model instead of the generic `KnowledgeGraph`. See the `cognee-custom-graph-models` skill. |
+| `custom_prompt` | Replaces the entity-extraction prompt (ignored by GLiNER). |
+| `config={"ontology_config": {...}}` | Ground entities in an OWL ontology (below). |
+| `chunk_size`, `chunker` | Max tokens per chunk (default: derived from the embedding and LLM limits) and the chunker class (default `TextChunker`). |
+| `preferred_loaders` | Choose a loader per file type (below). |
+| `self_improvement` | Default `True`: runs `improve()` after the graph is built. Its outcome is on `result.improve` / `result.improve_error`; a failed improve never fails the remember. |
+| `run_in_background=True` | Returns immediately with `status="running"`; `await result` to wait. |
+
+### Ontologies
+
+```python
+from cognee.modules.ontology.rdf_xml.RDFLibOntologyResolver import RDFLibOntologyResolver
+
+config = {
+    "ontology_config": {
+        "ontology_resolver": RDFLibOntologyResolver(ontology_file="./my.owl"),
+        # "ontology_mode": "strict",   # drop entities with no ontology match
+    }
+}
+await cognee.remember(texts, config=config)
+```
+
+Or set `ONTOLOGY_FILE_PATH` (plus `ONTOLOGY_MODE`, `MATCHING_STRATEGY`) in
+`.env`. `annotate` (default) only enriches; `strict` drops entities that
+match no ontology class or individual. It prunes only the graph, chunk text
+is still stored. Strict mode with an empty or missing ontology file is a hard
+error. Over HTTP, upload the ontology to `/api/v1/ontologies` and pass its
+`ontology_key` to `POST /api/v1/remember`. Example:
+`examples/guides/ontology_quickstart.py`.
+
+### Loaders
+
+Each file is claimed by the first loader that accepts it. Default order:
+code, text, pypdf, image, audio, video, dlt_csv, csv, unstructured,
+advanced_pdf, docling. Names: `text_loader`, `code_loader`, `csv_loader`,
+`dlt_csv_loader`, `pypdf_loader`, `image_loader`, `audio_loader`,
+`video_loader`, `unstructured_loader`, `advanced_pdf_loader`,
+`docling_loader`, `beautiful_soup_loader`.
+
+```python
+# Treat a code file as a plain document (chunking + LLM extraction):
+await cognee.remember("./script.py", preferred_loaders={"text_loader": {}})
+```
+
+Office formats (DOCX, PPTX, …) need the `docs` (unstructured) or `docling`
+extra. A preferred loader that is not installed is skipped with only an info
+log, so check the extra is installed when a file comes out wrong.
+
+### Check the cost first
+
+`dry_run=True` returns a token and cost estimate without ingesting anything
+or calling the LLM. It excludes the calls `improve()` makes. Not supported
+with GLiNER, sessions, or a remote instance.
+
+`dry_run="presort"` on a folder returns a `PresortReport` (junk, duplicates,
+version candidates, possible personal data, proposed dataset groups). Apply
+it with `await cognee.remember(report)`, or pass `auto_apply=True`.
+
+### Without an LLM: GLiNER
+
+`extractor="gliner"` builds the graph and summaries with a local GLiNER2
+model, with no LLM call (embeddings still run). Install
+`pip install "cognee[gliner]"`; the model (about 750 MB) downloads on first use.
+It cannot be combined with a custom `graph_model`, `dry_run`,
+`session_id`, or a remote instance.
+
+> **For production:** the open-source GLiNER extractor is a demo. cognee's
+> enterprise GLiNER extraction is more accurate and covers more labels. The
+> same goes for the Postgres graph adapter (`postgres_demo`). Contact
+> social@cognee.ai.
+
+## Pitfalls
+
+- **Unknown keyword arguments raise.** `remember()` forwards kwargs through
+  a fixed allow-list and raises `TypeError: Unexpected keyword arguments`
+  for anything else. These real options are not on it yet:
+
+  | Option | Workaround through remember() |
+  |---|---|
+  | `ontology_file_path` | `config={"ontology_config": ...}` or `ONTOLOGY_FILE_PATH` (above) |
+  | `functional_relationships`, `chunk_attachment` | None yet. Only `cognee.cognify()` accepts them. |
+  | `extraction_rules` | Pass it through the loader: `preferred_loaders={"beautiful_soup_loader": {"extraction_rules": {...}}}` (works in `remember()` and `add()`). Needs the `scraping` extra: without it the loader is not registered and the rules are silently ignored |
+  | `tavily_config`, `soup_crawler_config` | Not honoured by `add()` or `remember()`; only the `cognee/tasks/web_scraper` tasks use them |
+  | `column_value_columns` (dlt) | None yet. Only `cognee.add()` accepts it. |
+
+  If a user needs one with no workaround, say so plainly: the option exists
+  on the lower-level `add()` / `cognify()` but not on `remember()` yet.
+- **Changed files raise `DocumentUpdateRequiredError`.** Re-remembering the
+  same path (or the same filename for an upload) with different content is
+  an update, not a new document. Use
+  `cognee.update(data_id=..., data=..., dataset_id=...)`, which re-extracts
+  only the changed chunks and keeps the document's id. Identical content is
+  a no-op.
+- **`content_type` is strict.** Only `None`, `"skills"`, or `"code"`.
+  `"code"` rejects `session_id` and needs repository paths or git URLs;
+  `"skills"` ingests into the target dataset like any other call (default
+  `main_dataset`); pass `dataset_name` to keep skills in their own dataset.
+- **Session mode needs `CACHING=true`**, and `extractor` cannot be combined
+  with `session_id`.
+- **Remote mode.** After `cognee.serve(url)`, calls go to the server:
+  `extractor` and `session_ids` raise, and other options the client does not
+  forward (including `graph_model`, `node_set`, and ontology `config`) are
+  dropped without an error.
+- **Every remember runs `improve()`** unless `self_improvement=False` or
+  `IMPROVE_AUTO_ENABLED=false`. In scripts, call
+  `await cognee.wait_for_background_tasks()` before exiting.
+
+## How it works
+
+`remember(data)` runs `add()` (store raw data and create `Data` rows), then
+`cognify()` (classify documents, chunk, extract the graph and summaries,
+store in graph and vector DBs), then `improve()`. `remember(data,
+session_id=...)` writes to the session cache instead.
+
+- Entry point and kwarg routing: `cognee/api/v1/remember/remember.py`
+  (`RememberKwargs`, `_ADD_ONLY` / `_COGNIFY_ONLY` / `_SHARED`)
+- Storage: `cognee/api/v1/add/add.py`, `cognee/tasks/ingestion/ingest_data.py`
+- Graph build: `cognee/api/v1/cognify/cognify.py`,
+  `cognee/tasks/graph/extract_graph_from_data.py`,
+  `cognee/tasks/storage/add_data_points.py`
+- Extractor choice: `cognee/modules/cognify/config.py:resolve_extractor`;
+  GLiNER package: `cognee/tasks/graph/gliner_demo/`
+- Ontologies: `cognee/modules/ontology/`
+- Loaders: `cognee/infrastructure/loaders/` (`supported_loaders.py`,
+  `LoaderEngine.py`)
+- dlt: `cognee/tasks/ingestion/resolve_dlt_sources.py`
+
+Examples in `examples/guides/`: `simple_cognee_example.py`,
+`nodeset_grouping_example.py`, `ontology_quickstart.py`,
+`gliner_demo_llm_free_cognify.py`, `no_llm_remember_recall.py`,
+`temporal_recall.py`, `presort_downloads.py`,
+`web_url_content_ingestion_example.py`, `code_graph_example.py`.
+
+## Extending it
+
+- **New remember() option:** add it to `RememberKwargs` and to the matching
+  routing set in `remember.py`. An option on `add()`/`cognify()` that is not
+  in a routing set raises `TypeError` from `remember()`.
+- **New loader:** implement `LoaderInterface`
+  (`cognee/infrastructure/loaders/LoaderInterface.py`), register it in
+  `supported_loaders.py` (extras-gated loaders go under `external/`), and
+  add it to the priority list in `LoaderEngine.py` if it should run by
+  default.
+- **New cognify task:** see the `cognee-custom-pipelines` skill and
+  `cognee/tasks/README.md`.

cognee-1.6.4.dev0/.agents/skills/cognee-performance/SKILL.md

@@ -0,0 +1,186 @@
+---
+name: cognee-performance
+description: Use when cognee is slow, expensive, or hitting rate limits — speeding up or throttling ingestion (remember/cognify batching, chunk size, LLM and embedding rate limits, per-stage models), cutting read latency, estimating cost before ingesting, running work in the background, or planning how to scale a deployment.
+---
+
+# Tune cognee's performance
+
+**The LLM is the bottleneck.** Building the graph costs two LLM calls per
+chunk (graph extraction and summarization), and almost every other tuning
+question comes down to how many of those calls run, how fast, and on which
+model. Database and embedding work is small next to that.
+
+## Use it
+
+### 1. Estimate before you ingest
+
+```python
+estimate = await cognee.remember("./docs", dry_run=True)
+print(estimate)  # per-stage token counts and approximate cost; no LLM calls
+```
+
+The estimate covers graph extraction and summarization only, not
+`improve()`, embeddings, or contradiction detection. Not available with
+GLiNER or a remote instance.
+
+### 2. Ingestion knobs
+
+All are `remember()` arguments (and `cognify()` ones):
+
+| Knob | Default | What it controls |
+|---|---|---|
+| `data_per_batch` | 20 | How many documents are processed **at the same time** (a concurrency limit, not a batch). Lower it to reduce load; raise it for many small files. |
+| `chunks_per_batch` | 2000 (env `CHUNKS_PER_BATCH`) | How many chunks each extraction/storage call receives. Lower it to spread load and fail smaller. The DLT pipeline defaults to 100. |
+| `chunk_size` | min(embedding max tokens, LLM max tokens / 2); about 8191 tokens with default OpenAI models | Max tokens per chunk. Fewer, bigger chunks mean fewer LLM calls; smaller chunks mean finer-grained graphs. |
+| `run_in_background` | `False` | Return immediately (`status="running"`); `await result` later. |
+| `self_improvement` | `True` | `improve()` after the graph is built. `False` (or `IMPROVE_AUTO_ENABLED=false`) skips that extra work. |
+
+Worst-case concurrent LLM calls are about
+`data_per_batch × min(chunks per document, chunks_per_batch) × 2`, and there
+is no separate concurrency cap. The rate limiter below is the brake.
+
+Several datasets in one `cognify(datasets=[...])` call run one after
+another, not in parallel.
+
+### 3. LLM throttling and models
+
+| Env var | Default | Notes |
+|---|---|---|
+| `LLM_RATE_LIMIT_ENABLED` | `false` | Turn on a requests-per-interval limit |
+| `LLM_RATE_LIMIT_REQUESTS` / `LLM_RATE_LIMIT_INTERVAL` | 60 / 60 s | 10 requests for local providers (Ollama, llama.cpp, LM Studio) |
+| `AUTO_RATE_LIMIT` | `true` | On a 429/503/529 or timeout, switches the limiter on for 15 minutes (extended while errors continue) |
+| `LLM_EXTRACTION_MODEL` / `_PROVIDER` / `_ENDPOINT` / `_API_KEY` / `_API_VERSION` | the main model | Use a cheaper or faster model just for graph extraction |
+| `LLM_SUMMARIZATION_*`, `LLM_QUERY_*` | the main model | Same, for summaries and for answering queries |
+| `LLM_MAX_COMPLETION_TOKENS` | 16384 | Also caps the default chunk size (half of it) |
+| `LLM_ARGS` | — | Extra litellm arguments merged into every call, e.g. a request `timeout` |
+
+Structured-output calls retry with exponential backoff for at least two
+attempts and four minutes before failing; auth, not-found, and quota errors
+are not retried.
+
+The rate limiter is built from the main model's settings. A stage routed to
+a local model does not get the local default of 10 requests; set
+`LLM_RATE_LIMIT_REQUESTS` yourself.
+
+### 4. Embeddings
+
+| Env var | Default |
+|---|---|
+| `EMBEDDING_BATCH_SIZE` | 36 texts per request |
+| `EMBEDDING_MAX_CONCURRENT_DATA_POINTS` | 150 (so 150 / 36 = 4 concurrent requests) |
+| `EMBEDDING_RATE_LIMIT_ENABLED` / `_REQUESTS` / `_INTERVAL` | off / 60 / 60 s |
+
+Embedding calls retry transient errors, rate-limit 429s included, with
+exponential jitter for up to 128 s (budget-exhaustion errors are terminal,
+not retried), but `AUTO_RATE_LIMIT` does not apply to them; enable
+`EMBEDDING_RATE_LIMIT_ENABLED` yourself for sustained load. Without any
+credentials cognee embeds locally with fastembed (`BAAI/bge-small-en-v1.5`),
+which runs on the CPU and blocks while it works.
+
+### 5. Skip the LLM for extraction
+
+`remember(data, extractor="gliner")` builds the graph and summaries with a
+local GLiNER model: no LLM calls, embeddings still run. The runtime installs
+on first use (`GLINER_AUTO_INSTALL=false` to disable; then `pip install
+"cognee[gliner]"`). See the `cognee-ingestion` skill for its limits.
+
+### 6. Read latency
+
+- **`AUTO_FEEDBACK=false`** is the biggest win for chat-style use: by default
+  every answered turn (the default session is used when no `session_id` is
+  passed) makes one extra LLM call to analyze feedback. Keep `CACHING=true` so session memory still works.
+- **Pick a type without an LLM** when you need passages, not an answer:
+  `CHUNKS`, `CHUNKS_LEXICAL`, `SUMMARIES`, `CODE`, `SKILLS`. Completion types
+  make one LLM call; `GRAPH_COMPLETION_COT`, `_DECOMPOSITION`,
+  `_CONTEXT_EXTENSION`, `GRAPH_SUMMARY_COMPLETION` and `TEMPORAL` make
+  several; `NATURAL_LANGUAGE` makes one and retries (up to 3 attempts) only
+  on an empty or failed query; `FEELING_LUCKY` adds one to pick the type.
+- **Narrow the search**: pass `datasets=[...]` (one search runs per dataset)
+  and a smaller `top_k` (per dataset, default 15; HYBRID caps each lane at
+  10, so only values below 10 shrink its context).
+- `SESSION_SEARCH_MODE=concurrent` (default) overlaps the feedback analysis
+  with the answer; `sequential` runs them back to back.
+
+### 7. Several datasets at once
+
+With access control on (the default), each dataset has its own databases.
+`DATASET_QUEUE_MAX_CONCURRENT` (default 6, from `DATABASE_MAX_LRU_CACHE_SIZE`)
+caps how many datasets are processed at once in one process, and
+`SUBPROCESS_IDLE_TTL_SECONDS` (600) keeps idle database workers warm.
+`DATASET_QUEUE_ENABLED` (default true) enforces that cap, releases
+subprocess engines when a dataset's last scope exits (they stay warm for
+`SUBPROCESS_IDLE_TTL_SECONDS` and close at once only when it is 0) and pins
+in-use engines against eviction.
+Setting it false removes the cap rather than disabling parallelism, and
+risks file-lock leaks and engine eviction under parallel load, so keep it
+on.
+
+### 8. Scaling a deployment
+
+Open-source cognee scales **vertically, in one process**:
+
+- The API server runs one worker (`gunicorn -w 1`), and the improve and
+  session locks, caches and semaphores are in-process only. More workers or
+  replicas against the same data are not coordinated (the Helm chart README
+  says to validate before scaling past one replica).
+- The default embedded databases (SQLite, LanceDB, Ladybug) live on local
+  disk. For production, use Postgres + PGVector and a graph-native backend
+  such as Neo4j.
+- `distributed/deploy/` holds one-click deploy templates (Modal, Fly,
+  Railway, Render, Daytona), not a distributed runner.
+
+> **For production scale:** horizontal scaling (distributed ingestion across
+> workers), cognee's enterprise GLiNER extraction, and the production
+> Postgres graph adapter are part of cognee's proprietary offering. Contact
+> social@cognee.ai.
+
+## Pitfalls
+
+- **`CHUNK_SIZE`, `CHUNK_OVERLAP`, `CHUNK_STRATEGY`** (in `.env.template`) and
+  `cognee.config.set_chunk_size()` etc. **have no effect on ingestion.** Pass
+  `chunk_size=` to `remember()` instead.
+- **`LLM_RATE_LIMIT_TOKENS` and `EMBEDDING_RATE_LIMIT_TOKENS` do nothing**;
+  only request-count limits are enforced.
+- **`FALLBACK_MODEL`** is used only when the main model rejects content on
+  policy grounds, not when it is slow, down, or rate-limited.
+- **`data_per_batch=None` crashes.** Omit the argument to get the default.
+- **A background `cognify()`** is not tracked by
+  `cognee.wait_for_background_tasks()` (a background `remember()` or
+  `improve()` is). Await its pipeline status yourself.
+- **Big default chunks with a small local embedder.** The default chunk size
+  follows the configured embedding limit (8191), even for fastembed's
+  512-token model. Pass a smaller `chunk_size` for local embedding models.
+- Benchmark with `CACHING=true`: with it off you measure cognee without its
+  memory layer.
+
+## How it works
+
+`remember()` → `add()` → `cognify()`: every document of the dataset is
+scheduled at once, `data_per_batch` of them run concurrently, and each runs
+the task chain (classify, chunk, extract graph + summarize, store). Tasks
+receive their input in batches of `chunks_per_batch`.
+`extract_graph_and_summarize` runs extraction and summarization together,
+each gathering one LLM call per chunk in the batch. Every LLM call passes
+through one process-wide rate limiter.
+
+- Cognify task list and defaults: `cognee/api/v1/cognify/cognify.py`
+- Concurrency (`data_per_batch` semaphore): `cognee/modules/pipelines/operations/run_tasks.py`
+- Task batching: `cognee/modules/pipelines/tasks/task.py`, `run_tasks_base.py`
+- LLM calls per chunk: `cognee/tasks/graph/extract_graph_and_summarize.py`
+- Chunk-size default: `cognee/infrastructure/llm/utils.py:get_max_chunk_tokens`
+- LLM config, rate limits, stage routing: `cognee/infrastructure/llm/config.py`,
+  `cognee/shared/rate_limiting.py`, `cognee/infrastructure/llm/overload_policy.py`,
+  `cognee/infrastructure/llm/retry_config.py`
+- Embedding config: `cognee/infrastructure/databases/vector/embeddings/config.py`
+- Dataset queue: `cognee/infrastructure/databases/dataset_queue/queue.py`
+- Cost estimate: `cognee/modules/cognify/estimator.py`
+
+## Extending it
+
+- Benchmarks: `cognee/tests/performance/` (`batch_add_cognify_test.py`,
+  `locust_performance_analysis.py`, and `statistics_percentile/` for p50–p99
+  runs with mocked or replayed LLM calls). Measure before and after a change.
+- Retrieval quality and cost: `cognee/eval_framework/` (`python -m
+  cognee.eval_framework`; Modal runs cost money).
+- A new pipeline task that calls an LLM should keep `needs_llm=True` and go
+  through `LLMGateway`, so it shares the rate limiter and retry policy.