reindex 0.3.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- reindex-0.3.1/PKG-INFO +108 -0
- reindex-0.3.1/README.md +84 -0
- reindex-0.3.1/pyproject.toml +46 -0
- reindex-0.3.1/setup.cfg +4 -0
- reindex-0.3.1/src/reindex/__init__.py +12 -0
- reindex-0.3.1/src/reindex.egg-info/PKG-INFO +108 -0
- reindex-0.3.1/src/reindex.egg-info/SOURCES.txt +52 -0
- reindex-0.3.1/src/reindex.egg-info/dependency_links.txt +1 -0
- reindex-0.3.1/src/reindex.egg-info/entry_points.txt +3 -0
- reindex-0.3.1/src/reindex.egg-info/requires.txt +4 -0
- reindex-0.3.1/src/reindex.egg-info/top_level.txt +2 -0
- reindex-0.3.1/src/reindex_cli/__init__.py +6 -0
- reindex-0.3.1/src/reindex_cli/__main__.py +3 -0
- reindex-0.3.1/src/reindex_cli/api.py +69 -0
- reindex-0.3.1/src/reindex_cli/archives.py +97 -0
- reindex-0.3.1/src/reindex_cli/bundled_skills/reindex-create/SKILL.md +12 -0
- reindex-0.3.1/src/reindex_cli/bundled_skills/reindex-data/SKILL.md +22 -0
- reindex-0.3.1/src/reindex_cli/bundled_skills/reindex-scan/SKILL.md +13 -0
- reindex-0.3.1/src/reindex_cli/cli.py +176 -0
- reindex-0.3.1/src/reindex_cli/collection/__init__.py +9 -0
- reindex-0.3.1/src/reindex_cli/collection/resolver.py +51 -0
- reindex-0.3.1/src/reindex_cli/collection/state.py +88 -0
- reindex-0.3.1/src/reindex_cli/config.py +49 -0
- reindex-0.3.1/src/reindex_cli/errors.py +10 -0
- reindex-0.3.1/src/reindex_cli/get_ops.py +183 -0
- reindex-0.3.1/src/reindex_cli/manifest/__init__.py +4 -0
- reindex-0.3.1/src/reindex_cli/manifest/models.py +31 -0
- reindex-0.3.1/src/reindex_cli/manifest/parser.py +211 -0
- reindex-0.3.1/src/reindex_cli/manifest/yaml_support.py +52 -0
- reindex-0.3.1/src/reindex_cli/package/__init__.py +15 -0
- reindex-0.3.1/src/reindex_cli/package/cards.py +41 -0
- reindex-0.3.1/src/reindex_cli/package/renderer.py +190 -0
- reindex-0.3.1/src/reindex_cli/package/validation.py +211 -0
- reindex-0.3.1/src/reindex_cli/parsers/__init__.py +1 -0
- reindex-0.3.1/src/reindex_cli/parsers/common.py +44 -0
- reindex-0.3.1/src/reindex_cli/parsers/csv_parser.py +76 -0
- reindex-0.3.1/src/reindex_cli/parsers/docling_pdf.py +222 -0
- reindex-0.3.1/src/reindex_cli/parsers/generic.py +25 -0
- reindex-0.3.1/src/reindex_cli/parsers/markdown.py +38 -0
- reindex-0.3.1/src/reindex_cli/parsers/profiles.py +77 -0
- reindex-0.3.1/src/reindex_cli/parsers/registry.py +49 -0
- reindex-0.3.1/src/reindex_cli/pipeline/__init__.py +3 -0
- reindex-0.3.1/src/reindex_cli/pipeline/assembly.py +172 -0
- reindex-0.3.1/src/reindex_cli/pipeline/discovery.py +73 -0
- reindex-0.3.1/src/reindex_cli/pipeline/inspection.py +124 -0
- reindex-0.3.1/src/reindex_cli/pipeline/models.py +102 -0
- reindex-0.3.1/src/reindex_cli/pipeline/parsing.py +63 -0
- reindex-0.3.1/src/reindex_cli/pipeline/planning.py +54 -0
- reindex-0.3.1/src/reindex_cli/pipeline/publish.py +26 -0
- reindex-0.3.1/src/reindex_cli/pipeline/runner.py +188 -0
- reindex-0.3.1/src/reindex_cli/remote_ops.py +109 -0
- reindex-0.3.1/src/reindex_cli/remote_state.py +39 -0
- reindex-0.3.1/src/reindex_cli/skills.py +92 -0
- reindex-0.3.1/src/reindex_cli/util.py +50 -0
reindex-0.3.1/PKG-INFO
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: reindex
|
|
3
|
+
Version: 0.3.1
|
|
4
|
+
Summary: Python package and CLI for building and using ReIndex knowledge Collections.
|
|
5
|
+
Project-URL: Homepage, https://github.com/yechenyan/reIndex
|
|
6
|
+
Project-URL: Documentation, https://github.com/yechenyan/reIndex/tree/main/wiki
|
|
7
|
+
Project-URL: Repository, https://github.com/yechenyan/reIndex
|
|
8
|
+
Project-URL: Issues, https://github.com/yechenyan/reIndex/issues
|
|
9
|
+
Keywords: agents,cli,knowledge,search
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Environment :: Console
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Topic :: Text Processing :: Indexing
|
|
17
|
+
Classifier: Topic :: Utilities
|
|
18
|
+
Requires-Python: >=3.12
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
Requires-Dist: docling<3,>=2
|
|
21
|
+
Requires-Dist: httpx<1,>=0.27
|
|
22
|
+
Requires-Dist: pypdfium2<6,>=5
|
|
23
|
+
Requires-Dist: pyyaml<7,>=6
|
|
24
|
+
|
|
25
|
+
# ReIndex
|
|
26
|
+
|
|
27
|
+
The `reindex` distribution provides the ReIndex Python package and the `rei`
|
|
28
|
+
CLI. `reindex` is also a CLI alias.
|
|
29
|
+
|
|
30
|
+
## Install and initialize
|
|
31
|
+
|
|
32
|
+
Give an AI Agent this sentence:
|
|
33
|
+
|
|
34
|
+
> Run `uv tool install --upgrade reindex`, then run `rei init <data-directory> --agent <current-agent>` to create or update the ReIndex skills.
|
|
35
|
+
|
|
36
|
+
The installation is complete when `rei --help` works directly, without
|
|
37
|
+
`uv run`. `pipx install reindex` and `python -m pip install reindex`
|
|
38
|
+
are also supported. Python 3.12 or newer is required; the Docling dependency
|
|
39
|
+
makes the first installation and first PDF scan larger than a typical CLI.
|
|
40
|
+
|
|
41
|
+
`<current-agent>` is `codex`, `claude`, `cursor`, or `copilot`. `init` is idempotent, creates or reuses local identity, and installs or safely updates the three bundled skills. It does not scan, push, or download tutorial data.
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
rei init <collection-dir> [--name <name>] [--agent codex]
|
|
45
|
+
rei rename <collection-dir> <new-name>
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Local commands
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
rei inspect <path>
|
|
52
|
+
rei scan <path> [--collection-root <collection-dir>]
|
|
53
|
+
rei check <path>
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`inspect` is read-only. `scan` stages, validates, and atomically publishes a package. `check` verifies package resources, protected Node metadata, and current inputs.
|
|
57
|
+
|
|
58
|
+
For the normal author path, `push` runs the same package check again, so an
|
|
59
|
+
explicit `check` is mainly useful after manually editing cards or in CI:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
rei init <collection-dir> --name <name> --agent codex
|
|
63
|
+
rei inspect <collection-dir>
|
|
64
|
+
rei scan <collection-dir>
|
|
65
|
+
rei push <collection-dir> --api-url <api-url>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Remote commands
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
rei set-api <base-url>
|
|
72
|
+
rei push [path]
|
|
73
|
+
rei pull <name> [--output <directory>]
|
|
74
|
+
rei search "<query>" [--remote <name>] [--mode lexical|semantic|hybrid]
|
|
75
|
+
rei get <node-path> [--target card|source|content|asset]
|
|
76
|
+
rei get raw://<path>
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
`push` synchronously sends the complete validated package and exactly referenced raw sources. `pull` downloads only the complete `.node.md` tree. `get` checks a complete local package, local authoring source, and the SHA-256 cache before downloading one exact resource.
|
|
80
|
+
|
|
81
|
+
Collection names are the user-facing remote identifier. UUIDs remain internal stable identity. Resource logical paths stay Collection-relative and are scoped by the internal Collection UUID.
|
|
82
|
+
|
|
83
|
+
## Skills
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
rei skills install --agent codex
|
|
87
|
+
rei skills update --agent codex
|
|
88
|
+
rei skills update --agent codex --force
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Normal updates overwrite only unmodified ReIndex-managed copies. A modified skill reports a conflict unless `--force` is explicit.
|
|
92
|
+
|
|
93
|
+
## Python API
|
|
94
|
+
|
|
95
|
+
The stable Python namespace is `reindex`. It currently exposes the HTTP client
|
|
96
|
+
and installed package version; higher-level Collection APIs can grow here
|
|
97
|
+
without changing the CLI implementation package.
|
|
98
|
+
|
|
99
|
+
```python
|
|
100
|
+
from reindex import ApiClient, __version__
|
|
101
|
+
|
|
102
|
+
client = ApiClient("http://127.0.0.1:8000")
|
|
103
|
+
print(__version__, client.health())
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The CLI does not include or provision a hosted API. Remote commands use the
|
|
107
|
+
configured ReIndex API; without one, the default is the local development
|
|
108
|
+
endpoint at `http://127.0.0.1:8000`.
|
reindex-0.3.1/README.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# ReIndex
|
|
2
|
+
|
|
3
|
+
The `reindex` distribution provides the ReIndex Python package and the `rei`
|
|
4
|
+
CLI. `reindex` is also a CLI alias.
|
|
5
|
+
|
|
6
|
+
## Install and initialize
|
|
7
|
+
|
|
8
|
+
Give an AI Agent this sentence:
|
|
9
|
+
|
|
10
|
+
> Run `uv tool install --upgrade reindex`, then run `rei init <data-directory> --agent <current-agent>` to create or update the ReIndex skills.
|
|
11
|
+
|
|
12
|
+
The installation is complete when `rei --help` works directly, without
|
|
13
|
+
`uv run`. `pipx install reindex` and `python -m pip install reindex`
|
|
14
|
+
are also supported. Python 3.12 or newer is required; the Docling dependency
|
|
15
|
+
makes the first installation and first PDF scan larger than a typical CLI.
|
|
16
|
+
|
|
17
|
+
`<current-agent>` is `codex`, `claude`, `cursor`, or `copilot`. `init` is idempotent, creates or reuses local identity, and installs or safely updates the three bundled skills. It does not scan, push, or download tutorial data.
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
rei init <collection-dir> [--name <name>] [--agent codex]
|
|
21
|
+
rei rename <collection-dir> <new-name>
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Local commands
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
rei inspect <path>
|
|
28
|
+
rei scan <path> [--collection-root <collection-dir>]
|
|
29
|
+
rei check <path>
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
`inspect` is read-only. `scan` stages, validates, and atomically publishes a package. `check` verifies package resources, protected Node metadata, and current inputs.
|
|
33
|
+
|
|
34
|
+
For the normal author path, `push` runs the same package check again, so an
|
|
35
|
+
explicit `check` is mainly useful after manually editing cards or in CI:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
rei init <collection-dir> --name <name> --agent codex
|
|
39
|
+
rei inspect <collection-dir>
|
|
40
|
+
rei scan <collection-dir>
|
|
41
|
+
rei push <collection-dir> --api-url <api-url>
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## Remote commands
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
rei set-api <base-url>
|
|
48
|
+
rei push [path]
|
|
49
|
+
rei pull <name> [--output <directory>]
|
|
50
|
+
rei search "<query>" [--remote <name>] [--mode lexical|semantic|hybrid]
|
|
51
|
+
rei get <node-path> [--target card|source|content|asset]
|
|
52
|
+
rei get raw://<path>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`push` synchronously sends the complete validated package and exactly referenced raw sources. `pull` downloads only the complete `.node.md` tree. `get` checks a complete local package, local authoring source, and the SHA-256 cache before downloading one exact resource.
|
|
56
|
+
|
|
57
|
+
Collection names are the user-facing remote identifier. UUIDs remain internal stable identity. Resource logical paths stay Collection-relative and are scoped by the internal Collection UUID.
|
|
58
|
+
|
|
59
|
+
## Skills
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
rei skills install --agent codex
|
|
63
|
+
rei skills update --agent codex
|
|
64
|
+
rei skills update --agent codex --force
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Normal updates overwrite only unmodified ReIndex-managed copies. A modified skill reports a conflict unless `--force` is explicit.
|
|
68
|
+
|
|
69
|
+
## Python API
|
|
70
|
+
|
|
71
|
+
The stable Python namespace is `reindex`. It currently exposes the HTTP client
|
|
72
|
+
and installed package version; higher-level Collection APIs can grow here
|
|
73
|
+
without changing the CLI implementation package.
|
|
74
|
+
|
|
75
|
+
```python
|
|
76
|
+
from reindex import ApiClient, __version__
|
|
77
|
+
|
|
78
|
+
client = ApiClient("http://127.0.0.1:8000")
|
|
79
|
+
print(__version__, client.health())
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
The CLI does not include or provision a hosted API. Remote commands use the
|
|
83
|
+
configured ReIndex API; without one, the default is the local development
|
|
84
|
+
endpoint at `http://127.0.0.1:8000`.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "reindex"
|
|
7
|
+
version = "0.3.1"
|
|
8
|
+
description = "Python package and CLI for building and using ReIndex knowledge Collections."
|
|
9
|
+
requires-python = ">=3.12"
|
|
10
|
+
readme = "README.md"
|
|
11
|
+
keywords = ["agents", "cli", "knowledge", "search"]
|
|
12
|
+
classifiers = [
|
|
13
|
+
"Development Status :: 3 - Alpha",
|
|
14
|
+
"Environment :: Console",
|
|
15
|
+
"Intended Audience :: Developers",
|
|
16
|
+
"Operating System :: OS Independent",
|
|
17
|
+
"Programming Language :: Python :: 3",
|
|
18
|
+
"Programming Language :: Python :: 3.12",
|
|
19
|
+
"Topic :: Text Processing :: Indexing",
|
|
20
|
+
"Topic :: Utilities",
|
|
21
|
+
]
|
|
22
|
+
dependencies = [
|
|
23
|
+
"docling>=2,<3",
|
|
24
|
+
"httpx>=0.27,<1",
|
|
25
|
+
"pypdfium2>=5,<6",
|
|
26
|
+
"pyyaml>=6,<7",
|
|
27
|
+
]
|
|
28
|
+
|
|
29
|
+
[project.urls]
|
|
30
|
+
Homepage = "https://github.com/yechenyan/reIndex"
|
|
31
|
+
Documentation = "https://github.com/yechenyan/reIndex/tree/main/wiki"
|
|
32
|
+
Repository = "https://github.com/yechenyan/reIndex"
|
|
33
|
+
Issues = "https://github.com/yechenyan/reIndex/issues"
|
|
34
|
+
|
|
35
|
+
[project.scripts]
|
|
36
|
+
rei = "reindex_cli.cli:main"
|
|
37
|
+
reindex = "reindex_cli.cli:main"
|
|
38
|
+
|
|
39
|
+
[tool.setuptools.package-dir]
|
|
40
|
+
"" = "src"
|
|
41
|
+
|
|
42
|
+
[tool.setuptools.packages.find]
|
|
43
|
+
where = ["src"]
|
|
44
|
+
|
|
45
|
+
[tool.setuptools.package-data]
|
|
46
|
+
reindex_cli = ["bundled_skills/*/SKILL.md"]
|
reindex-0.3.1/setup.cfg
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
"""Public Python API for ReIndex."""
|
|
2
|
+
|
|
3
|
+
from importlib.metadata import PackageNotFoundError, version
|
|
4
|
+
|
|
5
|
+
from reindex_cli.api import ApiClient
|
|
6
|
+
|
|
7
|
+
try:
|
|
8
|
+
__version__ = version("reindex")
|
|
9
|
+
except PackageNotFoundError: # Source-only execution without an installed distribution.
|
|
10
|
+
__version__ = "0.0.0+local"
|
|
11
|
+
|
|
12
|
+
__all__ = ["ApiClient", "__version__"]
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: reindex
|
|
3
|
+
Version: 0.3.1
|
|
4
|
+
Summary: Python package and CLI for building and using ReIndex knowledge Collections.
|
|
5
|
+
Project-URL: Homepage, https://github.com/yechenyan/reIndex
|
|
6
|
+
Project-URL: Documentation, https://github.com/yechenyan/reIndex/tree/main/wiki
|
|
7
|
+
Project-URL: Repository, https://github.com/yechenyan/reIndex
|
|
8
|
+
Project-URL: Issues, https://github.com/yechenyan/reIndex/issues
|
|
9
|
+
Keywords: agents,cli,knowledge,search
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Environment :: Console
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Operating System :: OS Independent
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Topic :: Text Processing :: Indexing
|
|
17
|
+
Classifier: Topic :: Utilities
|
|
18
|
+
Requires-Python: >=3.12
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
Requires-Dist: docling<3,>=2
|
|
21
|
+
Requires-Dist: httpx<1,>=0.27
|
|
22
|
+
Requires-Dist: pypdfium2<6,>=5
|
|
23
|
+
Requires-Dist: pyyaml<7,>=6
|
|
24
|
+
|
|
25
|
+
# ReIndex
|
|
26
|
+
|
|
27
|
+
The `reindex` distribution provides the ReIndex Python package and the `rei`
|
|
28
|
+
CLI. `reindex` is also a CLI alias.
|
|
29
|
+
|
|
30
|
+
## Install and initialize
|
|
31
|
+
|
|
32
|
+
Give an AI Agent this sentence:
|
|
33
|
+
|
|
34
|
+
> Run `uv tool install --upgrade reindex`, then run `rei init <data-directory> --agent <current-agent>` to create or update the ReIndex skills.
|
|
35
|
+
|
|
36
|
+
The installation is complete when `rei --help` works directly, without
|
|
37
|
+
`uv run`. `pipx install reindex` and `python -m pip install reindex`
|
|
38
|
+
are also supported. Python 3.12 or newer is required; the Docling dependency
|
|
39
|
+
makes the first installation and first PDF scan larger than a typical CLI.
|
|
40
|
+
|
|
41
|
+
`<current-agent>` is `codex`, `claude`, `cursor`, or `copilot`. `init` is idempotent, creates or reuses local identity, and installs or safely updates the three bundled skills. It does not scan, push, or download tutorial data.
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
rei init <collection-dir> [--name <name>] [--agent codex]
|
|
45
|
+
rei rename <collection-dir> <new-name>
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Local commands
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
rei inspect <path>
|
|
52
|
+
rei scan <path> [--collection-root <collection-dir>]
|
|
53
|
+
rei check <path>
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
`inspect` is read-only. `scan` stages, validates, and atomically publishes a package. `check` verifies package resources, protected Node metadata, and current inputs.
|
|
57
|
+
|
|
58
|
+
For the normal author path, `push` runs the same package check again, so an
|
|
59
|
+
explicit `check` is mainly useful after manually editing cards or in CI:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
rei init <collection-dir> --name <name> --agent codex
|
|
63
|
+
rei inspect <collection-dir>
|
|
64
|
+
rei scan <collection-dir>
|
|
65
|
+
rei push <collection-dir> --api-url <api-url>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Remote commands
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
rei set-api <base-url>
|
|
72
|
+
rei push [path]
|
|
73
|
+
rei pull <name> [--output <directory>]
|
|
74
|
+
rei search "<query>" [--remote <name>] [--mode lexical|semantic|hybrid]
|
|
75
|
+
rei get <node-path> [--target card|source|content|asset]
|
|
76
|
+
rei get raw://<path>
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
`push` synchronously sends the complete validated package and exactly referenced raw sources. `pull` downloads only the complete `.node.md` tree. `get` checks a complete local package, local authoring source, and the SHA-256 cache before downloading one exact resource.
|
|
80
|
+
|
|
81
|
+
Collection names are the user-facing remote identifier. UUIDs remain internal stable identity. Resource logical paths stay Collection-relative and are scoped by the internal Collection UUID.
|
|
82
|
+
|
|
83
|
+
## Skills
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
rei skills install --agent codex
|
|
87
|
+
rei skills update --agent codex
|
|
88
|
+
rei skills update --agent codex --force
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Normal updates overwrite only unmodified ReIndex-managed copies. A modified skill reports a conflict unless `--force` is explicit.
|
|
92
|
+
|
|
93
|
+
## Python API
|
|
94
|
+
|
|
95
|
+
The stable Python namespace is `reindex`. It currently exposes the HTTP client
|
|
96
|
+
and installed package version; higher-level Collection APIs can grow here
|
|
97
|
+
without changing the CLI implementation package.
|
|
98
|
+
|
|
99
|
+
```python
|
|
100
|
+
from reindex import ApiClient, __version__
|
|
101
|
+
|
|
102
|
+
client = ApiClient("http://127.0.0.1:8000")
|
|
103
|
+
print(__version__, client.health())
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The CLI does not include or provision a hosted API. Remote commands use the
|
|
107
|
+
configured ReIndex API; without one, the default is the local development
|
|
108
|
+
endpoint at `http://127.0.0.1:8000`.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
README.md
|
|
2
|
+
pyproject.toml
|
|
3
|
+
src/reindex/__init__.py
|
|
4
|
+
src/reindex.egg-info/PKG-INFO
|
|
5
|
+
src/reindex.egg-info/SOURCES.txt
|
|
6
|
+
src/reindex.egg-info/dependency_links.txt
|
|
7
|
+
src/reindex.egg-info/entry_points.txt
|
|
8
|
+
src/reindex.egg-info/requires.txt
|
|
9
|
+
src/reindex.egg-info/top_level.txt
|
|
10
|
+
src/reindex_cli/__init__.py
|
|
11
|
+
src/reindex_cli/__main__.py
|
|
12
|
+
src/reindex_cli/api.py
|
|
13
|
+
src/reindex_cli/archives.py
|
|
14
|
+
src/reindex_cli/cli.py
|
|
15
|
+
src/reindex_cli/config.py
|
|
16
|
+
src/reindex_cli/errors.py
|
|
17
|
+
src/reindex_cli/get_ops.py
|
|
18
|
+
src/reindex_cli/remote_ops.py
|
|
19
|
+
src/reindex_cli/remote_state.py
|
|
20
|
+
src/reindex_cli/skills.py
|
|
21
|
+
src/reindex_cli/util.py
|
|
22
|
+
src/reindex_cli/bundled_skills/reindex-create/SKILL.md
|
|
23
|
+
src/reindex_cli/bundled_skills/reindex-data/SKILL.md
|
|
24
|
+
src/reindex_cli/bundled_skills/reindex-scan/SKILL.md
|
|
25
|
+
src/reindex_cli/collection/__init__.py
|
|
26
|
+
src/reindex_cli/collection/resolver.py
|
|
27
|
+
src/reindex_cli/collection/state.py
|
|
28
|
+
src/reindex_cli/manifest/__init__.py
|
|
29
|
+
src/reindex_cli/manifest/models.py
|
|
30
|
+
src/reindex_cli/manifest/parser.py
|
|
31
|
+
src/reindex_cli/manifest/yaml_support.py
|
|
32
|
+
src/reindex_cli/package/__init__.py
|
|
33
|
+
src/reindex_cli/package/cards.py
|
|
34
|
+
src/reindex_cli/package/renderer.py
|
|
35
|
+
src/reindex_cli/package/validation.py
|
|
36
|
+
src/reindex_cli/parsers/__init__.py
|
|
37
|
+
src/reindex_cli/parsers/common.py
|
|
38
|
+
src/reindex_cli/parsers/csv_parser.py
|
|
39
|
+
src/reindex_cli/parsers/docling_pdf.py
|
|
40
|
+
src/reindex_cli/parsers/generic.py
|
|
41
|
+
src/reindex_cli/parsers/markdown.py
|
|
42
|
+
src/reindex_cli/parsers/profiles.py
|
|
43
|
+
src/reindex_cli/parsers/registry.py
|
|
44
|
+
src/reindex_cli/pipeline/__init__.py
|
|
45
|
+
src/reindex_cli/pipeline/assembly.py
|
|
46
|
+
src/reindex_cli/pipeline/discovery.py
|
|
47
|
+
src/reindex_cli/pipeline/inspection.py
|
|
48
|
+
src/reindex_cli/pipeline/models.py
|
|
49
|
+
src/reindex_cli/pipeline/parsing.py
|
|
50
|
+
src/reindex_cli/pipeline/planning.py
|
|
51
|
+
src/reindex_cli/pipeline/publish.py
|
|
52
|
+
src/reindex_cli/pipeline/runner.py
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
3
|
+
import json
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
|
|
6
|
+
import httpx
|
|
7
|
+
|
|
8
|
+
from reindex_cli.errors import ReIndexError
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
class ApiClient:
|
|
12
|
+
def __init__(self, base_url: str, timeout: float = 1800.0) -> None:
|
|
13
|
+
self.base_url = base_url.rstrip("/")
|
|
14
|
+
self.timeout = timeout
|
|
15
|
+
|
|
16
|
+
def health(self) -> dict:
|
|
17
|
+
with httpx.Client(base_url=self.base_url, timeout=30.0) as client:
|
|
18
|
+
return self._json(client.get("/health"))
|
|
19
|
+
|
|
20
|
+
def push(self, name: str, package: Path, sources: Path) -> dict:
|
|
21
|
+
with (
|
|
22
|
+
package.open("rb") as package_stream,
|
|
23
|
+
sources.open("rb") as source_stream,
|
|
24
|
+
httpx.Client(base_url=self.base_url, timeout=self.timeout) as client,
|
|
25
|
+
):
|
|
26
|
+
response = client.post(
|
|
27
|
+
"/v1/push",
|
|
28
|
+
data={"name": name},
|
|
29
|
+
files={
|
|
30
|
+
"package": ("package.zip", package_stream, "application/zip"),
|
|
31
|
+
"sources": ("sources.zip", source_stream, "application/zip"),
|
|
32
|
+
},
|
|
33
|
+
)
|
|
34
|
+
return self._json(response)
|
|
35
|
+
|
|
36
|
+
def json(self, path: str, payload: dict) -> dict:
|
|
37
|
+
with httpx.Client(base_url=self.base_url, timeout=self.timeout) as client:
|
|
38
|
+
return self._json(client.post(path, json=payload))
|
|
39
|
+
|
|
40
|
+
def bytes(self, path: str, payload: dict) -> tuple[bytes, dict[str, str]]:
|
|
41
|
+
with httpx.Client(base_url=self.base_url, timeout=self.timeout) as client:
|
|
42
|
+
response = client.post(path, json=payload)
|
|
43
|
+
self._raise(response)
|
|
44
|
+
return response.content, dict(response.headers)
|
|
45
|
+
|
|
46
|
+
def _json(self, response: httpx.Response) -> dict:
|
|
47
|
+
self._raise(response)
|
|
48
|
+
try:
|
|
49
|
+
value = response.json()
|
|
50
|
+
except json.JSONDecodeError as error:
|
|
51
|
+
raise ReIndexError("Server returned invalid JSON") from error
|
|
52
|
+
if not isinstance(value, dict):
|
|
53
|
+
raise ReIndexError("Server returned an invalid response")
|
|
54
|
+
return value
|
|
55
|
+
|
|
56
|
+
@staticmethod
|
|
57
|
+
def _raise(response: httpx.Response) -> None:
|
|
58
|
+
if response.is_success:
|
|
59
|
+
return
|
|
60
|
+
try:
|
|
61
|
+
body = response.json()
|
|
62
|
+
detail = body.get("error", body)
|
|
63
|
+
message = detail.get("message") if isinstance(detail, dict) else None
|
|
64
|
+
except (json.JSONDecodeError, AttributeError):
|
|
65
|
+
message = None
|
|
66
|
+
raise ReIndexError(
|
|
67
|
+
f"Server request failed ({response.status_code}): "
|
|
68
|
+
f"{message or response.text or response.reason_phrase}"
|
|
69
|
+
)
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
3
|
+
import tempfile
|
|
4
|
+
import zipfile
|
|
5
|
+
from pathlib import Path, PurePosixPath
|
|
6
|
+
|
|
7
|
+
from reindex_cli.errors import ReIndexError
|
|
8
|
+
from reindex_cli.package.cards import parse_card
|
|
9
|
+
from reindex_cli.util import sha256_file
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def build_push_archives(context) -> tuple[Path, Path, tempfile.TemporaryDirectory]:
|
|
13
|
+
temporary = tempfile.TemporaryDirectory(prefix="rei-push-")
|
|
14
|
+
root = Path(temporary.name)
|
|
15
|
+
package_zip = root / "package.zip"
|
|
16
|
+
source_zip = root / "sources.zip"
|
|
17
|
+
with zipfile.ZipFile(package_zip, "w", zipfile.ZIP_DEFLATED) as bundle:
|
|
18
|
+
for file in sorted(context.output_dir.rglob("*")):
|
|
19
|
+
if file.is_file():
|
|
20
|
+
relative = Path(context.output_dir.name) / file.relative_to(
|
|
21
|
+
context.output_dir
|
|
22
|
+
)
|
|
23
|
+
bundle.write(file, relative.as_posix())
|
|
24
|
+
references = raw_references(context.output_dir)
|
|
25
|
+
with zipfile.ZipFile(source_zip, "w", zipfile.ZIP_DEFLATED) as bundle:
|
|
26
|
+
for relative, expected in sorted(references.items()):
|
|
27
|
+
source = context.root / relative
|
|
28
|
+
if not source.is_file() or sha256_file(source) != expected:
|
|
29
|
+
temporary.cleanup()
|
|
30
|
+
raise ReIndexError(f"Missing or changed source: {relative}")
|
|
31
|
+
bundle.write(source, relative)
|
|
32
|
+
return package_zip, source_zip, temporary
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def raw_references(package: Path) -> dict[str, str]:
|
|
36
|
+
result: dict[str, str] = {}
|
|
37
|
+
for card_path in package.rglob("*.node.md"):
|
|
38
|
+
metadata, _body = parse_card(card_path)
|
|
39
|
+
values = [
|
|
40
|
+
metadata.get("source"),
|
|
41
|
+
metadata.get("content"),
|
|
42
|
+
*metadata.get("assets", []),
|
|
43
|
+
]
|
|
44
|
+
for value in values:
|
|
45
|
+
if not isinstance(value, dict):
|
|
46
|
+
continue
|
|
47
|
+
uri = str(value.get("uri", ""))
|
|
48
|
+
if not uri.startswith("raw://"):
|
|
49
|
+
continue
|
|
50
|
+
relative = _safe(uri.removeprefix("raw://"))
|
|
51
|
+
digest = str(value.get("sha256", ""))
|
|
52
|
+
if relative in result and result[relative] != digest:
|
|
53
|
+
raise ReIndexError(f"Conflicting source hashes: {relative}")
|
|
54
|
+
result[relative] = digest
|
|
55
|
+
return result
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
def extract_node_archive(content: bytes, target: Path) -> int:
|
|
59
|
+
target.mkdir(parents=True, exist_ok=True)
|
|
60
|
+
archive_path = target / ".download.zip"
|
|
61
|
+
archive_path.write_bytes(content)
|
|
62
|
+
count = 0
|
|
63
|
+
try:
|
|
64
|
+
with zipfile.ZipFile(archive_path) as bundle:
|
|
65
|
+
names: set[str] = set()
|
|
66
|
+
for item in bundle.infolist():
|
|
67
|
+
path = PurePosixPath(item.filename)
|
|
68
|
+
if (
|
|
69
|
+
item.filename in names
|
|
70
|
+
or path.is_absolute()
|
|
71
|
+
or ".." in path.parts
|
|
72
|
+
or not item.filename.endswith(".node.md")
|
|
73
|
+
or (item.external_attr >> 16) & 0o170000 == 0o120000
|
|
74
|
+
):
|
|
75
|
+
raise ReIndexError(
|
|
76
|
+
f"Unsafe or non-Node pull entry: {item.filename}"
|
|
77
|
+
)
|
|
78
|
+
names.add(item.filename)
|
|
79
|
+
if item.is_dir():
|
|
80
|
+
continue
|
|
81
|
+
destination = target.joinpath(*path.parts)
|
|
82
|
+
destination.parent.mkdir(parents=True, exist_ok=True)
|
|
83
|
+
destination.write_bytes(bundle.read(item))
|
|
84
|
+
parse_card(destination)
|
|
85
|
+
count += 1
|
|
86
|
+
finally:
|
|
87
|
+
archive_path.unlink(missing_ok=True)
|
|
88
|
+
if not (target / "index.node.md").is_file():
|
|
89
|
+
raise ReIndexError("Pulled Node tree has no root index.node.md")
|
|
90
|
+
return count
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
def _safe(value: str) -> str:
|
|
94
|
+
path = PurePosixPath(value)
|
|
95
|
+
if path.is_absolute() or ".." in path.parts or not path.parts:
|
|
96
|
+
raise ReIndexError(f"Unsafe raw path: {value}")
|
|
97
|
+
return path.as_posix()
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: reindex-create
|
|
3
|
+
description: Initialize a ReIndex project when the user asks to set up ReIndex or run rei init/create.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# ReIndex create
|
|
7
|
+
|
|
8
|
+
1. Resolve the exact directory named by the user.
|
|
9
|
+
2. Prefer `rei init <directory> --agent <current-agent>` for normal setup. It is idempotent and manages skills.
|
|
10
|
+
3. Use `rei create <directory>` only when identity-only initialization was explicitly requested.
|
|
11
|
+
4. Report the Collection name and whether it was created or reused. UUID is internal and normally omitted.
|
|
12
|
+
5. Do not scan, push, delete source files, or replace `.rei/collection.json` during initialization.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: reindex-data
|
|
3
|
+
description: Push, pull, search, and get exact ReIndex data when the user asks to publish or use remote ReIndex knowledge.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# ReIndex data
|
|
7
|
+
|
|
8
|
+
## Publish
|
|
9
|
+
|
|
10
|
+
1. Run `rei check <path>` before `rei push <path>`; scan first if stale.
|
|
11
|
+
2. `rei push` synchronously sends the validated package and exactly referenced sources.
|
|
12
|
+
3. Report the user-facing Collection name and ready status, not internal UUID details.
|
|
13
|
+
|
|
14
|
+
## Find and fetch
|
|
15
|
+
|
|
16
|
+
1. Use `rei search "<question>"` before downloading large files.
|
|
17
|
+
2. Select the result whose Evidence supports the task.
|
|
18
|
+
3. Use the result's Node path with `rei get <node-path> --target content`; use `source` only when original bytes are required.
|
|
19
|
+
4. Use `rei get raw://<path>` for an explicitly named raw source.
|
|
20
|
+
5. Answer from the fetched file and cite the Collection name and Node or raw path.
|
|
21
|
+
|
|
22
|
+
`rei pull <name>` downloads the complete Node tree only. It intentionally excludes source, content and assets; fetch exact resources later with `rei get`.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: reindex-scan
|
|
3
|
+
description: Scan raw files into ReIndex when the user asks to scan, compile, ingest, or convert local data.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# ReIndex scan
|
|
7
|
+
|
|
8
|
+
1. Run `rei inspect <path>` and review effective inputs, relationships and changes.
|
|
9
|
+
2. Apply only evidence-backed manifest corrections; never delete raw files just to pass validation.
|
|
10
|
+
3. Run `rei scan <path>` and review changes, warnings and generated Node cards.
|
|
11
|
+
4. Edit Markdown card bodies only. The CLI owns YAML frontmatter.
|
|
12
|
+
5. Run `rei check <path>` after manual card edits.
|
|
13
|
+
6. Report the Collection name, Node count, warnings and package location. Passing checks is not human approval.
|