indx 0.0.1__tar.gz

indx-0.0.1/.gitignore

@@ -0,0 +1,37 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+
+# Tooling caches
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.nox/
+.coverage
+htmlcov/
+
+# indx
+*.indx
+/scratch/
+tests/corpus/_heavy/*
+!tests/corpus/_heavy/.gitkeep
+
+# indx app — the Next.js bundle is build-time only (scripts/build_webapp.sh regenerates it).
+# Only the tracked .gitkeep is committed; `next build` clobbers index.html with a real bundle
+# that references the (ignored) _next/ assets, so we never track it. The server serves an
+# inline "not built yet" fallback (server.py) when the bundle is absent.
+src/indx/app/static/*
+!src/indx/app/static/.gitkeep
+
+# webapp (Next.js frontend) build artifacts & deps
+webapp/node_modules/
+webapp/.next/
+webapp/out/
+webapp/.turbo/
+*.tsbuildinfo

indx-0.0.1/CHANGELOG.md

@@ -0,0 +1,67 @@
+# Changelog
+
+All notable changes to `indx` are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+See [Versioning & compatibility](https://docs.indx.jp) for what
+major/minor/patch mean for the CLI, the SDK, and the `.indx` artifact format.
+
+## [Unreleased]
+
+Nothing yet.
+
+## [0.0.1] — 2026-06-07
+
+First public release on PyPI: the light pure-Python core, the full
+Walk → Parse → Chunk → Relate → Enrich → Embed+Pack pipeline, the registry of swappable
+typed slots, and the complete set of optional cloud/local backends. The `.indx` archive
+format is at `schema_version` `"1"`. See
+[Versioning & compatibility](https://docs.indx.jp) for what
+major/minor/patch mean for the CLI, the SDK, and the artifact format.
+
+### Added
+
+- Core pipeline and the `.indx` archive format (`schema_version` `"1"`): the light
+  pure-Python Walk → Parse → Chunk → Relate → Enrich → Embed+Pack flow, the registry, and
+  the swappable typed slots (parser / LLM / VLM / embedder / store / writer).
+- `--offline` CLI profile that selects the pure-Python, air-gapped stack (plaintext
+  parser, `hash` embedder, `jsonl` store, `llm none`) without touching the documented
+  cloud defaults. This is the supported way to run with zero network calls — the slot
+  defaults in `src/indx/config/schema.py` are deliberately left as-is.
+- `indx demo` command that produces an instant, zero-config knowledge space so the
+  "magical moment" is reachable in one command.
+- Per-stage build timings, emitted as structured data via a build `--json` flag, for
+  local DX measurement without telemetry.
+- JSONL-directory `inspect` and `query` support so a space materialized as JSONL can be
+  explored without re-sealing it into a `.indx` archive.
+- `CONTRIBUTING` guide and runnable `examples/` covering real use cases.
+- **AI-agent connectors** (`indx.agent`): `connect(archive)` turns a portable `.indx`
+  knowledge space into live tools for an agent — one method each for LangChain
+  (`.langchain()` / `.langchain_retriever()`), the OpenAI Agents SDK (`.openai()`),
+  Pydantic AI (`.pydantic_ai()`), and the Claude Agent SDK (`.claude()`), plus raw tool
+  specs (`.openai_schema()` / `.anthropic_schema()`) and a `.call()` dispatcher for the bare
+  Chat Completions / Messages API. New extras `indx[mcp]`, `indx[pydantic-ai]`,
+  `indx[openai-agents]`, `indx[claude-agent]`, and the `indx[agent]` umbrella; each adapter
+  is lazily imported and gated by its extra, so core stays light.
+- **`indx mcp <archive>`** — serve a knowledge space over the Model Context Protocol so any
+  MCP client (Claude Desktop, Cursor, the TypeScript Mastra framework) can search it with no
+  Python glue. Built on FastMCP (prefers the standalone `fastmcp` package, falls back to the
+  FastMCP bundled in the `mcp` SDK). Needs `indx[mcp]` (or `indx[agent]`).
+- **LiteLLM backends** (`indx[litellm]`): an `litellm` LLM adapter and embedder that reach
+  any provider through one interface — on-prem (Ollama, vLLM), and AWS Bedrock, Azure
+  OpenAI, GCP Vertex, or any hosted model — selected by LiteLLM's `provider/model` strings
+  (e.g. `--llm litellm:bedrock/anthropic.claude-3` or `--embedder litellm:azure/<deployment>`).
+
+### Changed
+
+- README quickstart reworked so the first copy-pasted command succeeds offline, instead
+  of failing on a missing cloud extra.
+
+### Docs
+
+- Overview/reference pages corrected against the real code shapes (e.g. `Document.doc_type`,
+  `KnowledgeSpace.documents()` as a method with the list stored under `documents_`,
+  `Manifest` carrying `schema_version` + `indx_version`).
+- Added this CHANGELOG and a [Versioning & compatibility](https://docs.indx.jp)
+  reference chapter documenting SemVer intent and the `.indx` schema-compat policy.

indx-0.0.1/CONTRIBUTING.md

@@ -0,0 +1,207 @@
+# Contributing to indx
+
+Thanks for helping build indx — the tool that turns a directory into a portable,
+AI-ready knowledge space. This guide gets you from a fresh clone to a green PR, and walks
+through the one workflow most contributors come for: **adding a backend without forking**.
+
+indx is a light pure-Python core that orchestrates a fixed
+**Walk → Parse → Chunk → Relate → Enrich → Embed+Pack** pipeline. Every external capability
+(parser, LLM, VLM, embedder, vector store, output writer) is a **swappable typed slot**
+resolved by name through a registry. Four constraints decide every change:
+**local-first / air-gapped**, **no vendor lock-in**, **light core**, **determinism**. Keep
+them in mind — most review feedback traces back to one of them.
+
+## Dev setup
+
+```bash
+python -m venv .venv
+. .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+That installs indx plus the dev toolchain (ruff, mypy, pytest, nox). The core stays tiny;
+heavy/optional backends (docling, torch, openai, qdrant, ...) are **extras** you install
+only when you need them. You never need any extra to run the offline core or the test suite.
+
+Want to see it work right now? The [`examples/`](examples/) directory has a runnable,
+fully-offline quickstart:
+
+```bash
+python examples/quickstart.py
+```
+
+## The gates
+
+Three checks gate every change. Run them locally before you push:
+
+```bash
+ruff check src tests     # lint
+mypy src/indx            # strict type-check on the public API
+pytest -q                # fast offline suite: unit + corpus
+```
+
+Also run the formatter check the way CI does:
+
+```bash
+ruff format --check src tests
+```
+
+All three must be green. The suite is **offline by design** — it never reaches the network
+and uses the deterministic core stack (`plaintext` / `hash` / `jsonl`), so it runs the same
+on any machine.
+
+### nox sessions
+
+[`noxfile.py`](noxfile.py) wraps the gates and the heavier suites into reproducible
+sessions. Run `nox -l` to list them; the real sessions are:
+
+| session           | what it does                                                          |
+| ----------------- | -------------------------------------------------------------------- |
+| `tests`           | fast offline suite (unit + corpus); the default `pytest` run. Matrixed across 3.11/3.12/3.13. |
+| `lint`            | `ruff check` + `ruff format --check` on `src` and `tests`.           |
+| `typecheck`       | `mypy src/indx` (strict).                                            |
+| `integration`     | real backends via docker compose (qdrant / postgres / ollama).       |
+| `docker`          | packaged-wheel distribution tests in clean containers.               |
+| `airgap`          | default stack with **no egress**, in a network-isolated container.   |
+| `live`            | real models end to end; nightly/manual, may require network.         |
+| `record-fixtures` | regenerate recorded ParsedDoc / embedding fixtures (deliberate).     |
+
+Day to day you only need `tests`, `lint`, and `typecheck`. The rest run in CI or on demand.
+
+```bash
+nox -s tests          # fast offline suite
+nox -s lint           # ruff lint + format check
+nox -s typecheck      # mypy strict
+nox -l                # list every session
+```
+
+## Coding standards
+
+A few rules carry most of the review weight:
+
+- **Never `import` a heavy/optional vendor SDK at module top level.** Lazy-import it inside
+  `__init__` and gate it with `require_extra(...)` first (see the plugin how-to below).
+- **Raise typed `IndxError` subclasses, never bare `ValueError`.** Library code must not
+  `print()` — only the CLI layer (`src/indx/cli/`) renders, via Rich.
+- **Determinism is non-negotiable.** Same inputs + same component selection ⇒ byte-identical
+  `index.json`. Sort before assigning ids; thread the `seed`; pin golden files.
+- **Models are Pydantic v2** and the public API is `mypy --strict` clean.
+
+For the full picture:
+
+- [`docs/reference/`](docs/reference/) — the "textbook": start with
+  [`00-overview.md`](docs/reference/00-overview.md) (dependency philosophy) and
+  [`13-tooling.md`](docs/reference/13-tooling.md) (ruff / mypy / nox), then the per-slot
+  chapters.
+- The docsite contributing pages —
+  [`coding-standards`](docsite/src/content/docs/contributing/coding-standards.md) and
+  [`adding-a-backend`](docsite/src/content/docs/contributing/adding-a-backend.md).
+
+## Add a backend without forking (plugin how-to)
+
+This is the core extension story: a third-party package adds a new slot implementation
+*without editing indx*. It is exactly what [`examples/custom_plugin/`](examples/custom_plugin/)
+demonstrates — read it alongside this section.
+
+### 1. Implement the slot's `typing.Protocol` — structurally
+
+Each slot has a `typing.Protocol` you satisfy **by shape**, not by subclassing. For example,
+the parser contract (`indx.parsers.base.Parser`) is just:
+
+```python
+from pathlib import Path
+from indx.core.parsed import Block, ParsedDoc
+
+
+class UppercaseParser:
+    name = "uppercase"               # the slot name users select
+    version = "1"
+
+    def parse(self, path: Path) -> ParsedDoc:
+        text = path.read_text(encoding="utf-8")
+        return ParsedDoc(
+            source_path=str(path),
+            parser=self.name,
+            parser_version=self.version,
+            blocks=[Block(kind="text", text=text.upper().strip(), order=0)],
+        )
+```
+
+The other slots follow the same idea: an embedder exposes `name`, `dim`, and
+`embed(texts) -> list[list[float]]`; a store exposes `upsert` / `search` / `delete`; etc.
+See [`docs/reference/04-plugins-registry.md`](docs/reference/04-plugins-registry.md) and the
+per-slot chapters (`05-parsers.md`, `07-embeddings.md`, `08-vector-stores.md`,
+`09-output-writers.md`).
+
+### 2. Lazy-import the vendor SDK inside `__init__`, gated first
+
+If your backend wraps a heavy/optional dependency, do **not** import it at module top level —
+that would break a bare core install. Import it inside `__init__`, and call
+`indx.utils.lazy.require_extra(slot, name, extra, *module_names)` **first** so a missing
+dependency surfaces as one actionable `MissingExtraError` only when the slot is selected:
+
+```python
+class MyVendorParser:
+    name = "myvendor"
+
+    def __init__(self) -> None:
+        from indx.utils.lazy import require_extra
+
+        require_extra("parser", "myvendor", "myvendor", "myvendor_sdk")
+        import myvendor_sdk          # imported here, never at module top level
+
+        self._client = myvendor_sdk.Client()
+```
+
+### 3. Register via an entry point in the correct group
+
+Advertise your class under the slot's entry-point group in your package's `pyproject.toml`.
+The group names are defined in `src/indx/registry/builtins.py` (`ENTRY_POINT_GROUPS` /
+`STAGE_ENTRY_POINT_GROUP`):
+
+| slot     | entry-point group                      |
+| -------- | -------------------------------------- |
+| parser   | `indx.parsers`                         |
+| llm      | `indx.llms`                            |
+| vlm      | `indx.vlms`                            |
+| embedder | `indx.embedders`                       |
+| store    | `indx.stores`                          |
+| writer   | `indx.outputs` (legacy alias: `indx.writers`) |
+| stage    | `indx.stages`                          |
+
+```toml
+# in your plugin package's pyproject.toml
+[project.entry-points."indx.parsers"]
+uppercase = "indx_uppercase_parser:UppercaseParser"
+```
+
+Once your package is `pip install`ed, users select it by name — `--parser uppercase` on the
+CLI, or `DirectoryPipeline(parser="uppercase", ...)` from the SDK.
+
+### 4. How the registry resolves it
+
+When a slot name is requested, `src/indx/registry/registry.py` resolves it in this order:
+
+1. **First-party builtins** (`BUILTINS[slot]`) — loaded lazily by `module:attr`.
+2. **Third-party entry points** for the slot's group(s) (`ENTRY_POINT_GROUPS[slot]`).
+
+**Built-ins always win name collisions**, so a plugin can never silently shadow a built-in
+name. If nothing matches, you get a `RegistryError` listing the known names; a selected
+backend whose extra is missing raises `MissingExtraError` (from the gate in step 2).
+
+## PR expectations
+
+Before you open a PR:
+
+- **Gates green.** `ruff check src tests`, `ruff format --check src tests`, `mypy src/indx`,
+  and `pytest -q` all pass (or `nox -s lint typecheck tests`).
+- **Tests added.** New behavior ships with tests. Backends get unit tests; deterministic
+  output paths get golden-file coverage.
+- **Determinism respected.** No new nondeterminism in serialized output — sort before
+  assigning ids, thread the `seed`, and update golden fixtures deliberately
+  (`nox -s record-fixtures`), never by accident.
+- **Constraints honored.** No heavy import at module top level; no bare `ValueError`; no
+  `print()` outside the CLI layer; the offline core stays reachable.
+
+Thanks again — clear, well-tested, deterministic contributions keep indx portable for
+everyone.

indx-0.0.1/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 indx contributors
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

indx-0.0.1/NOTICE

@@ -0,0 +1,16 @@
+indx
+Copyright 2026 indx contributors
+
+This product includes software developed by the indx contributors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.