wexample-storage 1.2.4__tar.gz → 1.2.6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- wexample_storage-1.2.6/PKG-INFO +119 -0
- wexample_storage-1.2.6/README.md +96 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/pyproject.toml +3 -3
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/__init__.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/__init__.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/abstract_file_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/abstract_graph_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/abstract_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/abstract_vector_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/filesystem_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/neo4j_graph_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/postgres_vector_storage.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/storage_registry.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/vector_match.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/__init__.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/capabilities_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/credential_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/database_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/label_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/path_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/storage_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/storages_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/table_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/type_config_option.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/const/__init__.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/const/capability.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/exception/__init__.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/exception/invalid_storage_config_exception.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/exception/invalid_storage_key_exception.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/exception/no_storage_for_capability_exception.py +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/py.typed +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/tests/.gitkeep +0 -0
- {wexample_storage-1.2.4 → wexample_storage-1.2.6}/tests/test_storage.py +0 -0
- wexample_storage-1.2.4/PKG-INFO +0 -257
- wexample_storage-1.2.4/README.md +0 -234
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: wexample-storage
|
|
3
|
+
Version: 1.2.6
|
|
4
|
+
Author-Email: weeger <contact@wexample.com>
|
|
5
|
+
License: MIT
|
|
6
|
+
Classifier: Programming Language :: Python :: 3
|
|
7
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
8
|
+
Classifier: Operating System :: OS Independent
|
|
9
|
+
Requires-Python: >=3.10
|
|
10
|
+
Requires-Dist: wexample-config
|
|
11
|
+
Requires-Dist: wexample-helpers>=20.6.0
|
|
12
|
+
Requires-Dist: wexample-remote>=1.1.6
|
|
13
|
+
Provides-Extra: neo4j
|
|
14
|
+
Requires-Dist: neo4j>=5; extra == "neo4j"
|
|
15
|
+
Provides-Extra: postgres
|
|
16
|
+
Requires-Dist: psycopg2-binary; extra == "postgres"
|
|
17
|
+
Requires-Dist: sqlalchemy>=2; extra == "postgres"
|
|
18
|
+
Provides-Extra: dev
|
|
19
|
+
Requires-Dist: pytest; extra == "dev"
|
|
20
|
+
Requires-Dist: pytest-cov; extra == "dev"
|
|
21
|
+
Requires-Dist: wexample-wallet; extra == "dev"
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
|
|
24
|
+
# storage
|
|
25
|
+
|
|
26
|
+
Version: 1.2.6
|
|
27
|
+
|
|
28
|
+
## Table of Contents
|
|
29
|
+
|
|
30
|
+
- [Introduction](#introduction)
|
|
31
|
+
- [Install](#install)
|
|
32
|
+
- [Tests](#tests)
|
|
33
|
+
- [Roadmap](#roadmap)
|
|
34
|
+
- [Status Compatibility](#status-compatibility)
|
|
35
|
+
- [Useful Links](#useful-links)
|
|
36
|
+
|
|
37
|
+
`wexample-storage` gives an application the places it keeps contents in, behind one interface per kind of content: files written and read back by key, vectors searched by similarity, graphs queried in their own language. Each storage has a name and the capabilities it accepts (`files`, `vectors`, `graph`), and a registry picks the storage for a capability — so the code needing a place to write never chooses the backend itself.
|
|
38
|
+
|
|
39
|
+
Every storage is a `wexample-remote` remote, so it can be checked like any other service. Backends: a local directory, PostgreSQL with pgvector (the `postgres` extra), Neo4j (the `neo4j` extra).
|
|
40
|
+
|
|
41
|
+
## Installation
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
pip install wexample-storage
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Requires Python >=3.10.
|
|
48
|
+
|
|
49
|
+
## Tests
|
|
50
|
+
|
|
51
|
+
This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.
|
|
52
|
+
|
|
53
|
+
### Installation
|
|
54
|
+
|
|
55
|
+
First, install the required testing dependencies:
|
|
56
|
+
```bash
|
|
57
|
+
.venv/bin/python -m pip install pytest pytest-cov
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### Basic Usage
|
|
61
|
+
|
|
62
|
+
Run all tests with coverage:
|
|
63
|
+
```bash
|
|
64
|
+
.venv/bin/python -m pytest --cov --cov-report=html
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Common Commands
|
|
68
|
+
```bash
|
|
69
|
+
# Run tests with coverage for a specific module
|
|
70
|
+
.venv/bin/python -m pytest --cov=your_module
|
|
71
|
+
|
|
72
|
+
# Show which lines are not covered
|
|
73
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing
|
|
74
|
+
|
|
75
|
+
# Generate an HTML coverage report
|
|
76
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=html
|
|
77
|
+
|
|
78
|
+
# Combine terminal and HTML reports
|
|
79
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html
|
|
80
|
+
|
|
81
|
+
# Run specific test file with coverage
|
|
82
|
+
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
### Viewing HTML Reports
|
|
86
|
+
|
|
87
|
+
After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.
|
|
88
|
+
|
|
89
|
+
### Coverage Threshold
|
|
90
|
+
|
|
91
|
+
To enforce a minimum coverage percentage:
|
|
92
|
+
```bash
|
|
93
|
+
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
This will cause the test suite to fail if coverage drops below 80%.
|
|
97
|
+
|
|
98
|
+
## Known Limitations & Roadmap
|
|
99
|
+
|
|
100
|
+
Current limitations and planned features are tracked in the GitHub issues.
|
|
101
|
+
|
|
102
|
+
See the [project roadmap](https://github.com/wexample/python-storage/issues) for upcoming features and improvements.
|
|
103
|
+
|
|
104
|
+
## Status & Compatibility
|
|
105
|
+
|
|
106
|
+
**Maturity**: Production-ready
|
|
107
|
+
|
|
108
|
+
**Python Support**: >=3.10
|
|
109
|
+
|
|
110
|
+
**OS Support**: Linux, macOS, Windows
|
|
111
|
+
|
|
112
|
+
**Status**: Actively maintained
|
|
113
|
+
|
|
114
|
+
## Useful Links
|
|
115
|
+
|
|
116
|
+
- **Homepage**: https://github.com/wexample/python-storage
|
|
117
|
+
- **Issue Tracker**: https://github.com/wexample/python-storage/issues
|
|
118
|
+
- **Discussions**: https://github.com/wexample/python-storage/discussions
|
|
119
|
+
- **PyPI**: [pypi.org/project/wexample-storage](https://pypi.org/project/wexample-storage/)
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# storage
|
|
2
|
+
|
|
3
|
+
Version: 1.2.6
|
|
4
|
+
|
|
5
|
+
## Table of Contents
|
|
6
|
+
|
|
7
|
+
- [Introduction](#introduction)
|
|
8
|
+
- [Install](#install)
|
|
9
|
+
- [Tests](#tests)
|
|
10
|
+
- [Roadmap](#roadmap)
|
|
11
|
+
- [Status Compatibility](#status-compatibility)
|
|
12
|
+
- [Useful Links](#useful-links)
|
|
13
|
+
|
|
14
|
+
`wexample-storage` gives an application the places it keeps contents in, behind one interface per kind of content: files written and read back by key, vectors searched by similarity, graphs queried in their own language. Each storage has a name and the capabilities it accepts (`files`, `vectors`, `graph`), and a registry picks the storage for a capability — so the code needing a place to write never chooses the backend itself.
|
|
15
|
+
|
|
16
|
+
Every storage is a `wexample-remote` remote, so it can be checked like any other service. Backends: a local directory, PostgreSQL with pgvector (the `postgres` extra), Neo4j (the `neo4j` extra).
|
|
17
|
+
|
|
18
|
+
## Installation
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
pip install wexample-storage
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Requires Python >=3.10.
|
|
25
|
+
|
|
26
|
+
## Tests
|
|
27
|
+
|
|
28
|
+
This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.
|
|
29
|
+
|
|
30
|
+
### Installation
|
|
31
|
+
|
|
32
|
+
First, install the required testing dependencies:
|
|
33
|
+
```bash
|
|
34
|
+
.venv/bin/python -m pip install pytest pytest-cov
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
### Basic Usage
|
|
38
|
+
|
|
39
|
+
Run all tests with coverage:
|
|
40
|
+
```bash
|
|
41
|
+
.venv/bin/python -m pytest --cov --cov-report=html
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Common Commands
|
|
45
|
+
```bash
|
|
46
|
+
# Run tests with coverage for a specific module
|
|
47
|
+
.venv/bin/python -m pytest --cov=your_module
|
|
48
|
+
|
|
49
|
+
# Show which lines are not covered
|
|
50
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing
|
|
51
|
+
|
|
52
|
+
# Generate an HTML coverage report
|
|
53
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=html
|
|
54
|
+
|
|
55
|
+
# Combine terminal and HTML reports
|
|
56
|
+
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html
|
|
57
|
+
|
|
58
|
+
# Run specific test file with coverage
|
|
59
|
+
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Viewing HTML Reports
|
|
63
|
+
|
|
64
|
+
After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.
|
|
65
|
+
|
|
66
|
+
### Coverage Threshold
|
|
67
|
+
|
|
68
|
+
To enforce a minimum coverage percentage:
|
|
69
|
+
```bash
|
|
70
|
+
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
This will cause the test suite to fail if coverage drops below 80%.
|
|
74
|
+
|
|
75
|
+
## Known Limitations & Roadmap
|
|
76
|
+
|
|
77
|
+
Current limitations and planned features are tracked in the GitHub issues.
|
|
78
|
+
|
|
79
|
+
See the [project roadmap](https://github.com/wexample/python-storage/issues) for upcoming features and improvements.
|
|
80
|
+
|
|
81
|
+
## Status & Compatibility
|
|
82
|
+
|
|
83
|
+
**Maturity**: Production-ready
|
|
84
|
+
|
|
85
|
+
**Python Support**: >=3.10
|
|
86
|
+
|
|
87
|
+
**OS Support**: Linux, macOS, Windows
|
|
88
|
+
|
|
89
|
+
**Status**: Actively maintained
|
|
90
|
+
|
|
91
|
+
## Useful Links
|
|
92
|
+
|
|
93
|
+
- **Homepage**: https://github.com/wexample/python-storage
|
|
94
|
+
- **Issue Tracker**: https://github.com/wexample/python-storage/issues
|
|
95
|
+
- **Discussions**: https://github.com/wexample/python-storage/discussions
|
|
96
|
+
- **PyPI**: [pypi.org/project/wexample-storage](https://pypi.org/project/wexample-storage/)
|
|
@@ -6,7 +6,7 @@ build-backend = "pdm.backend"
|
|
|
6
6
|
|
|
7
7
|
[project]
|
|
8
8
|
name = "wexample-storage"
|
|
9
|
-
version = "1.2.
|
|
9
|
+
version = "1.2.6"
|
|
10
10
|
authors = [
|
|
11
11
|
{ name = "weeger", email = "contact@wexample.com" },
|
|
12
12
|
]
|
|
@@ -18,8 +18,8 @@ classifiers = [
|
|
|
18
18
|
]
|
|
19
19
|
dependencies = [
|
|
20
20
|
"wexample-config",
|
|
21
|
-
"wexample-helpers>=20.
|
|
22
|
-
"wexample-remote>=1.1.
|
|
21
|
+
"wexample-helpers>=20.6.0",
|
|
22
|
+
"wexample-remote>=1.1.6",
|
|
23
23
|
]
|
|
24
24
|
|
|
25
25
|
[project.readme]
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/abstract_storage.py
RENAMED
|
File without changes
|
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/filesystem_storage.py
RENAMED
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/neo4j_graph_storage.py
RENAMED
|
File without changes
|
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/storage_registry.py
RENAMED
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/common/vector_match.py
RENAMED
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/config_option/__init__.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{wexample_storage-1.2.4 → wexample_storage-1.2.6}/src/wexample_storage/exception/__init__.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
wexample_storage-1.2.4/PKG-INFO
DELETED
|
@@ -1,257 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.1
|
|
2
|
-
Name: wexample-storage
|
|
3
|
-
Version: 1.2.4
|
|
4
|
-
Author-Email: weeger <contact@wexample.com>
|
|
5
|
-
License: MIT
|
|
6
|
-
Classifier: Programming Language :: Python :: 3
|
|
7
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
8
|
-
Classifier: Operating System :: OS Independent
|
|
9
|
-
Requires-Python: >=3.10
|
|
10
|
-
Requires-Dist: wexample-config
|
|
11
|
-
Requires-Dist: wexample-helpers>=20.5.0
|
|
12
|
-
Requires-Dist: wexample-remote>=1.1.0
|
|
13
|
-
Provides-Extra: neo4j
|
|
14
|
-
Requires-Dist: neo4j>=5; extra == "neo4j"
|
|
15
|
-
Provides-Extra: postgres
|
|
16
|
-
Requires-Dist: psycopg2-binary; extra == "postgres"
|
|
17
|
-
Requires-Dist: sqlalchemy>=2; extra == "postgres"
|
|
18
|
-
Provides-Extra: dev
|
|
19
|
-
Requires-Dist: pytest; extra == "dev"
|
|
20
|
-
Requires-Dist: pytest-cov; extra == "dev"
|
|
21
|
-
Requires-Dist: wexample-wallet; extra == "dev"
|
|
22
|
-
Description-Content-Type: text/markdown
|
|
23
|
-
|
|
24
|
-
# storage
|
|
25
|
-
|
|
26
|
-
Version: 1.2.4
|
|
27
|
-
|
|
28
|
-
## Files
|
|
29
|
-
|
|
30
|
-
```python
|
|
31
|
-
from wexample_storage import FilesystemStorage
|
|
32
|
-
|
|
33
|
-
storage = FilesystemStorage(name="uploads", path="/var/app/uploads")
|
|
34
|
-
storage.write("report.json", '{"score": 42}')
|
|
35
|
-
storage.read_text("report.json")
|
|
36
|
-
storage.exists("report.json")
|
|
37
|
-
storage.delete("report.json")
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
The directory is created on first write. A key may not leave it: `../x` raises `InvalidStorageKeyException`.
|
|
41
|
-
|
|
42
|
-
## Vectors
|
|
43
|
-
|
|
44
|
-
```python
|
|
45
|
-
from wexample_storage import PostgresVectorStorage
|
|
46
|
-
|
|
47
|
-
vectors = PostgresVectorStorage(
|
|
48
|
-
database="store", host="localhost", name="vectors", password="...", user="postgres"
|
|
49
|
-
)
|
|
50
|
-
vectors.save_vector("doc-1", "chunk-a", [0.1, 0.8, ...], "The text the vector was made from")
|
|
51
|
-
vectors.search_vectors("doc-1", query_vector, limit=5, max_distance=0.4) # closest first
|
|
52
|
-
vectors.get_vector_keys("doc-1")
|
|
53
|
-
vectors.delete_vectors("doc-1")
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
Vectors are grouped in namespaces, one per document for instance. The table (`wexample_storage_vector` unless `table` says otherwise) and the pgvector extension are created on first use; the column has no fixed dimension, so each embedding model keeps its own. Needs the `postgres` extra.
|
|
57
|
-
|
|
58
|
-
## Graphs
|
|
59
|
-
|
|
60
|
-
```python
|
|
61
|
-
from wexample_storage import Neo4jGraphStorage
|
|
62
|
-
|
|
63
|
-
graph = Neo4jGraphStorage(name="graph", url="bolt://localhost:7687", user="neo4j", password="...")
|
|
64
|
-
graph.run_query("MATCH (p:Player) RETURN p.name AS name") # [{"name": ...}, ...]
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
Needs the `neo4j` extra.
|
|
68
|
-
|
|
69
|
-
## Choosing a storage
|
|
70
|
-
|
|
71
|
-
```python
|
|
72
|
-
from wexample_storage import StorageRegistry
|
|
73
|
-
|
|
74
|
-
registry = StorageRegistry([uploads, vectors, graph])
|
|
75
|
-
registry.get_for_capability("vectors") # the first storage accepting it
|
|
76
|
-
registry.check_all() # statuses, as any remote registry
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
A storage accepts the capabilities of its kind unless given others: `FilesystemStorage(capabilities=["archives"], ...)` takes archives only. Without a storage for a capability, `NoStorageForCapabilityException` names those registered.
|
|
80
|
-
|
|
81
|
-
## From configuration
|
|
82
|
-
|
|
83
|
-
```python
|
|
84
|
-
from wexample_storage.config_option.storages_config_option import StoragesConfigOption
|
|
85
|
-
|
|
86
|
-
storages = StoragesConfigOption(
|
|
87
|
-
value=[
|
|
88
|
-
{"name": "files", "type": "filesystem", "path": "/var/app/files"},
|
|
89
|
-
{"name": "vectors", "type": "postgres", "credential": "store", "table": "vectors"},
|
|
90
|
-
{"name": "graph", "type": "neo4j", "credential": "graph"},
|
|
91
|
-
]
|
|
92
|
-
).create_storages(wallet)
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
The configuration names the `wexample-wallet` credential holding a backend's secrets; `PostgresVectorStorage.from_credential()` and `Neo4jGraphStorage.from_credential()` do the same from code. `StoragesConfigOption` is a `wexample-config` option, so an application's own schema can include it. A storage missing what its type needs raises `InvalidStorageConfigException`.
|
|
96
|
-
|
|
97
|
-
## Table of Contents
|
|
98
|
-
|
|
99
|
-
- [Files](#files)
|
|
100
|
-
- [Vectors](#vectors)
|
|
101
|
-
- [Graphs](#graphs)
|
|
102
|
-
- [Choosing a storage](#choosing-a-storage)
|
|
103
|
-
- [From configuration](#from-configuration)
|
|
104
|
-
- [Installation](#installation)
|
|
105
|
-
- [Tests](#tests)
|
|
106
|
-
- [Architecture](#architecture)
|
|
107
|
-
- [Integration in the Suite](#integration-in-the-suite)
|
|
108
|
-
- [Dependencies](#dependencies)
|
|
109
|
-
- [Versioning & Compatibility Policy](#versioning--compatibility-policy)
|
|
110
|
-
- [License](#license)
|
|
111
|
-
- [About us](#about-us)
|
|
112
|
-
- [Known Limitations & Roadmap](#known-limitations--roadmap)
|
|
113
|
-
- [Status & Compatibility](#status--compatibility)
|
|
114
|
-
- [Useful Links](#useful-links)
|
|
115
|
-
- [Migration Notes](#migration-notes)
|
|
116
|
-
|
|
117
|
-
## Installation
|
|
118
|
-
|
|
119
|
-
```bash
|
|
120
|
-
pip install wexample-storage
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
Requires Python >=3.10.
|
|
124
|
-
|
|
125
|
-
## Tests
|
|
126
|
-
|
|
127
|
-
This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.
|
|
128
|
-
|
|
129
|
-
### Installation
|
|
130
|
-
|
|
131
|
-
First, install the required testing dependencies:
|
|
132
|
-
```bash
|
|
133
|
-
.venv/bin/python -m pip install pytest pytest-cov
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
### Basic Usage
|
|
137
|
-
|
|
138
|
-
Run all tests with coverage:
|
|
139
|
-
```bash
|
|
140
|
-
.venv/bin/python -m pytest --cov --cov-report=html
|
|
141
|
-
```
|
|
142
|
-
|
|
143
|
-
### Common Commands
|
|
144
|
-
```bash
|
|
145
|
-
# Run tests with coverage for a specific module
|
|
146
|
-
.venv/bin/python -m pytest --cov=your_module
|
|
147
|
-
|
|
148
|
-
# Show which lines are not covered
|
|
149
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing
|
|
150
|
-
|
|
151
|
-
# Generate an HTML coverage report
|
|
152
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=html
|
|
153
|
-
|
|
154
|
-
# Combine terminal and HTML reports
|
|
155
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html
|
|
156
|
-
|
|
157
|
-
# Run specific test file with coverage
|
|
158
|
-
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
### Viewing HTML Reports
|
|
162
|
-
|
|
163
|
-
After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.
|
|
164
|
-
|
|
165
|
-
### Coverage Threshold
|
|
166
|
-
|
|
167
|
-
To enforce a minimum coverage percentage:
|
|
168
|
-
```bash
|
|
169
|
-
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
|
|
170
|
-
```
|
|
171
|
-
|
|
172
|
-
This will cause the test suite to fail if coverage drops below 80%.
|
|
173
|
-
|
|
174
|
-
## Architecture
|
|
175
|
-
|
|
176
|
-
src/wexample_storage/common/abstract_storage.py is a `wexample-remote` remote with a name, a label and capabilities. A storage's capabilities are those given, or else those of its kind (`get_default_capabilities()`), so that a vector storage never accepts files by accident.
|
|
177
|
-
|
|
178
|
-
Each kind is an abstract class of its own, since a key-value store, a vector index and a graph share nothing but being storages:
|
|
179
|
-
|
|
180
|
-
- src/wexample_storage/common/abstract_file_storage.py: `write`, `read`, `exists`, `delete`; implemented by src/wexample_storage/common/filesystem_storage.py.
|
|
181
|
-
- src/wexample_storage/common/abstract_vector_storage.py: vectors by namespace and key, searched by cosine distance into src/wexample_storage/common/vector_match.py; implemented by src/wexample_storage/common/postgres_vector_storage.py, in raw SQL through SQLAlchemy so that no pgvector binding is needed.
|
|
182
|
-
- src/wexample_storage/common/abstract_graph_storage.py: `run_query()`; implemented by src/wexample_storage/common/neo4j_graph_storage.py, whose records come back as plain dicts.
|
|
183
|
-
|
|
184
|
-
src/wexample_storage/config_option/storages_config_option.py and src/wexample_storage/config_option/storage_config_option.py are the `wexample-config` schema of a list of storages: a name, a type, a path or the name of a wallet credential. They build the storages, taking secrets from the wallet given, so that a configuration file never holds one.
|
|
185
|
-
|
|
186
|
-
src/wexample_storage/common/storage_registry.py extends the remote registry with the choice by capability. Capability names are in src/wexample_storage/const/capability.py.
|
|
187
|
-
|
|
188
|
-
Backend libraries are imported inside the methods using them and declared as extras, so that a filesystem-only application installs none. The PostgreSQL and Neo4j tests run only when `WEXAMPLE_STORAGE_TEST_POSTGRES` (`host:port:database:user:password`) or `WEXAMPLE_STORAGE_TEST_NEO4J` (`url|user|password`) point at a server.
|
|
189
|
-
|
|
190
|
-
## Integration in the Suite
|
|
191
|
-
|
|
192
|
-
This package is part of the Wexample Suite — a collection of high-quality, modular tools designed to work seamlessly together across multiple languages and environments.
|
|
193
|
-
|
|
194
|
-
### Related Packages
|
|
195
|
-
|
|
196
|
-
The suite includes packages for configuration management, file handling, prompts, and more. Each package can be used independently or as part of the integrated suite.
|
|
197
|
-
|
|
198
|
-
## Dependencies
|
|
199
|
-
|
|
200
|
-
- wexample-config:
|
|
201
|
-
- wexample-helpers: >=20.5.0
|
|
202
|
-
- wexample-remote: >=1.1.0
|
|
203
|
-
|
|
204
|
-
## Versioning & Compatibility Policy
|
|
205
|
-
|
|
206
|
-
Wexample packages follow **Semantic Versioning** (SemVer):
|
|
207
|
-
|
|
208
|
-
- **MAJOR**: Breaking changes
|
|
209
|
-
- **MINOR**: New features, backward compatible
|
|
210
|
-
- **PATCH**: Bug fixes, backward compatible
|
|
211
|
-
|
|
212
|
-
We maintain backward compatibility within major versions and provide clear migration guides for breaking changes.
|
|
213
|
-
|
|
214
|
-
## License
|
|
215
|
-
|
|
216
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
217
|
-
|
|
218
|
-
Free to use in both personal and commercial projects.
|
|
219
|
-
|
|
220
|
-
## About us
|
|
221
|
-
|
|
222
|
-
[Wexample](https://wexample.com) stands as a cornerstone of the digital ecosystem — a collective of seasoned engineers, researchers, and creators driven by a relentless pursuit of technological excellence. More than a media platform, it has grown into a vibrant community where innovation meets craftsmanship, and where every line of code reflects a commitment to clarity, durability, and shared intelligence.
|
|
223
|
-
|
|
224
|
-
This packages suite embodies this spirit. Trusted by professionals and enthusiasts alike, it delivers a consistent, high-quality foundation for modern development — open, elegant, and battle-tested. Its reputation is built on years of collaboration, refinement, and rigorous attention to detail, making it a natural choice for those who demand both robustness and beauty in their tools.
|
|
225
|
-
|
|
226
|
-
Wexample cultivates a culture of mastery. Each package, each contribution carries the mark of a community that values precision, ethics, and innovation — a community proud to shape the future of digital craftsmanship.
|
|
227
|
-
|
|
228
|
-
Built and maintained by [Romain Weeger](https://weeger.io).
|
|
229
|
-
|
|
230
|
-
## Known Limitations & Roadmap
|
|
231
|
-
|
|
232
|
-
Current limitations and planned features are tracked in the GitHub issues.
|
|
233
|
-
|
|
234
|
-
See the [project roadmap](https://github.com/wexample/python-storage/issues) for upcoming features and improvements.
|
|
235
|
-
|
|
236
|
-
## Status & Compatibility
|
|
237
|
-
|
|
238
|
-
**Maturity**: Production-ready
|
|
239
|
-
|
|
240
|
-
**Python Support**: >=3.10
|
|
241
|
-
|
|
242
|
-
**OS Support**: Linux, macOS, Windows
|
|
243
|
-
|
|
244
|
-
**Status**: Actively maintained
|
|
245
|
-
|
|
246
|
-
## Useful Links
|
|
247
|
-
|
|
248
|
-
- **Homepage**: https://github.com/wexample/python-storage
|
|
249
|
-
- **Issue Tracker**: https://github.com/wexample/python-storage/issues
|
|
250
|
-
- **Discussions**: https://github.com/wexample/python-storage/discussions
|
|
251
|
-
- **PyPI**: [pypi.org/project/wexample-storage](https://pypi.org/project/wexample-storage/)
|
|
252
|
-
|
|
253
|
-
## Migration Notes
|
|
254
|
-
|
|
255
|
-
When upgrading between major versions, refer to the migration guides in the documentation.
|
|
256
|
-
|
|
257
|
-
Breaking changes are clearly documented with upgrade paths and examples.
|
wexample_storage-1.2.4/README.md
DELETED
|
@@ -1,234 +0,0 @@
|
|
|
1
|
-
# storage
|
|
2
|
-
|
|
3
|
-
Version: 1.2.4
|
|
4
|
-
|
|
5
|
-
## Files
|
|
6
|
-
|
|
7
|
-
```python
|
|
8
|
-
from wexample_storage import FilesystemStorage
|
|
9
|
-
|
|
10
|
-
storage = FilesystemStorage(name="uploads", path="/var/app/uploads")
|
|
11
|
-
storage.write("report.json", '{"score": 42}')
|
|
12
|
-
storage.read_text("report.json")
|
|
13
|
-
storage.exists("report.json")
|
|
14
|
-
storage.delete("report.json")
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
The directory is created on first write. A key may not leave it: `../x` raises `InvalidStorageKeyException`.
|
|
18
|
-
|
|
19
|
-
## Vectors
|
|
20
|
-
|
|
21
|
-
```python
|
|
22
|
-
from wexample_storage import PostgresVectorStorage
|
|
23
|
-
|
|
24
|
-
vectors = PostgresVectorStorage(
|
|
25
|
-
database="store", host="localhost", name="vectors", password="...", user="postgres"
|
|
26
|
-
)
|
|
27
|
-
vectors.save_vector("doc-1", "chunk-a", [0.1, 0.8, ...], "The text the vector was made from")
|
|
28
|
-
vectors.search_vectors("doc-1", query_vector, limit=5, max_distance=0.4) # closest first
|
|
29
|
-
vectors.get_vector_keys("doc-1")
|
|
30
|
-
vectors.delete_vectors("doc-1")
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
Vectors are grouped in namespaces, one per document for instance. The table (`wexample_storage_vector` unless `table` says otherwise) and the pgvector extension are created on first use; the column has no fixed dimension, so each embedding model keeps its own. Needs the `postgres` extra.
|
|
34
|
-
|
|
35
|
-
## Graphs
|
|
36
|
-
|
|
37
|
-
```python
|
|
38
|
-
from wexample_storage import Neo4jGraphStorage
|
|
39
|
-
|
|
40
|
-
graph = Neo4jGraphStorage(name="graph", url="bolt://localhost:7687", user="neo4j", password="...")
|
|
41
|
-
graph.run_query("MATCH (p:Player) RETURN p.name AS name") # [{"name": ...}, ...]
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
Needs the `neo4j` extra.
|
|
45
|
-
|
|
46
|
-
## Choosing a storage
|
|
47
|
-
|
|
48
|
-
```python
|
|
49
|
-
from wexample_storage import StorageRegistry
|
|
50
|
-
|
|
51
|
-
registry = StorageRegistry([uploads, vectors, graph])
|
|
52
|
-
registry.get_for_capability("vectors") # the first storage accepting it
|
|
53
|
-
registry.check_all() # statuses, as any remote registry
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
A storage accepts the capabilities of its kind unless given others: `FilesystemStorage(capabilities=["archives"], ...)` takes archives only. Without a storage for a capability, `NoStorageForCapabilityException` names those registered.
|
|
57
|
-
|
|
58
|
-
## From configuration
|
|
59
|
-
|
|
60
|
-
```python
|
|
61
|
-
from wexample_storage.config_option.storages_config_option import StoragesConfigOption
|
|
62
|
-
|
|
63
|
-
storages = StoragesConfigOption(
|
|
64
|
-
value=[
|
|
65
|
-
{"name": "files", "type": "filesystem", "path": "/var/app/files"},
|
|
66
|
-
{"name": "vectors", "type": "postgres", "credential": "store", "table": "vectors"},
|
|
67
|
-
{"name": "graph", "type": "neo4j", "credential": "graph"},
|
|
68
|
-
]
|
|
69
|
-
).create_storages(wallet)
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
The configuration names the `wexample-wallet` credential holding a backend's secrets; `PostgresVectorStorage.from_credential()` and `Neo4jGraphStorage.from_credential()` do the same from code. `StoragesConfigOption` is a `wexample-config` option, so an application's own schema can include it. A storage missing what its type needs raises `InvalidStorageConfigException`.
|
|
73
|
-
|
|
74
|
-
## Table of Contents
|
|
75
|
-
|
|
76
|
-
- [Files](#files)
|
|
77
|
-
- [Vectors](#vectors)
|
|
78
|
-
- [Graphs](#graphs)
|
|
79
|
-
- [Choosing a storage](#choosing-a-storage)
|
|
80
|
-
- [From configuration](#from-configuration)
|
|
81
|
-
- [Installation](#installation)
|
|
82
|
-
- [Tests](#tests)
|
|
83
|
-
- [Architecture](#architecture)
|
|
84
|
-
- [Integration in the Suite](#integration-in-the-suite)
|
|
85
|
-
- [Dependencies](#dependencies)
|
|
86
|
-
- [Versioning & Compatibility Policy](#versioning--compatibility-policy)
|
|
87
|
-
- [License](#license)
|
|
88
|
-
- [About us](#about-us)
|
|
89
|
-
- [Known Limitations & Roadmap](#known-limitations--roadmap)
|
|
90
|
-
- [Status & Compatibility](#status--compatibility)
|
|
91
|
-
- [Useful Links](#useful-links)
|
|
92
|
-
- [Migration Notes](#migration-notes)
|
|
93
|
-
|
|
94
|
-
## Installation
|
|
95
|
-
|
|
96
|
-
```bash
|
|
97
|
-
pip install wexample-storage
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
Requires Python >=3.10.
|
|
101
|
-
|
|
102
|
-
## Tests
|
|
103
|
-
|
|
104
|
-
This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.
|
|
105
|
-
|
|
106
|
-
### Installation
|
|
107
|
-
|
|
108
|
-
First, install the required testing dependencies:
|
|
109
|
-
```bash
|
|
110
|
-
.venv/bin/python -m pip install pytest pytest-cov
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
### Basic Usage
|
|
114
|
-
|
|
115
|
-
Run all tests with coverage:
|
|
116
|
-
```bash
|
|
117
|
-
.venv/bin/python -m pytest --cov --cov-report=html
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
### Common Commands
|
|
121
|
-
```bash
|
|
122
|
-
# Run tests with coverage for a specific module
|
|
123
|
-
.venv/bin/python -m pytest --cov=your_module
|
|
124
|
-
|
|
125
|
-
# Show which lines are not covered
|
|
126
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing
|
|
127
|
-
|
|
128
|
-
# Generate an HTML coverage report
|
|
129
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=html
|
|
130
|
-
|
|
131
|
-
# Combine terminal and HTML reports
|
|
132
|
-
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html
|
|
133
|
-
|
|
134
|
-
# Run specific test file with coverage
|
|
135
|
-
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
### Viewing HTML Reports
|
|
139
|
-
|
|
140
|
-
After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.
|
|
141
|
-
|
|
142
|
-
### Coverage Threshold
|
|
143
|
-
|
|
144
|
-
To enforce a minimum coverage percentage:
|
|
145
|
-
```bash
|
|
146
|
-
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
This will cause the test suite to fail if coverage drops below 80%.
|
|
150
|
-
|
|
151
|
-
## Architecture
|
|
152
|
-
|
|
153
|
-
src/wexample_storage/common/abstract_storage.py is a `wexample-remote` remote with a name, a label and capabilities. A storage's capabilities are those given, or else those of its kind (`get_default_capabilities()`), so that a vector storage never accepts files by accident.
|
|
154
|
-
|
|
155
|
-
Each kind is an abstract class of its own, since a key-value store, a vector index and a graph share nothing but being storages:
|
|
156
|
-
|
|
157
|
-
- src/wexample_storage/common/abstract_file_storage.py: `write`, `read`, `exists`, `delete`; implemented by src/wexample_storage/common/filesystem_storage.py.
|
|
158
|
-
- src/wexample_storage/common/abstract_vector_storage.py: vectors by namespace and key, searched by cosine distance into src/wexample_storage/common/vector_match.py; implemented by src/wexample_storage/common/postgres_vector_storage.py, in raw SQL through SQLAlchemy so that no pgvector binding is needed.
|
|
159
|
-
- src/wexample_storage/common/abstract_graph_storage.py: `run_query()`; implemented by src/wexample_storage/common/neo4j_graph_storage.py, whose records come back as plain dicts.
|
|
160
|
-
|
|
161
|
-
src/wexample_storage/config_option/storages_config_option.py and src/wexample_storage/config_option/storage_config_option.py are the `wexample-config` schema of a list of storages: a name, a type, a path or the name of a wallet credential. They build the storages, taking secrets from the wallet given, so that a configuration file never holds one.
|
|
162
|
-
|
|
163
|
-
src/wexample_storage/common/storage_registry.py extends the remote registry with the choice by capability. Capability names are in src/wexample_storage/const/capability.py.
|
|
164
|
-
|
|
165
|
-
Backend libraries are imported inside the methods using them and declared as extras, so that a filesystem-only application installs none. The PostgreSQL and Neo4j tests run only when `WEXAMPLE_STORAGE_TEST_POSTGRES` (`host:port:database:user:password`) or `WEXAMPLE_STORAGE_TEST_NEO4J` (`url|user|password`) point at a server.
|
|
166
|
-
|
|
167
|
-
## Integration in the Suite
|
|
168
|
-
|
|
169
|
-
This package is part of the Wexample Suite — a collection of high-quality, modular tools designed to work seamlessly together across multiple languages and environments.
|
|
170
|
-
|
|
171
|
-
### Related Packages
|
|
172
|
-
|
|
173
|
-
The suite includes packages for configuration management, file handling, prompts, and more. Each package can be used independently or as part of the integrated suite.
|
|
174
|
-
|
|
175
|
-
## Dependencies
|
|
176
|
-
|
|
177
|
-
- wexample-config:
|
|
178
|
-
- wexample-helpers: >=20.5.0
|
|
179
|
-
- wexample-remote: >=1.1.0
|
|
180
|
-
|
|
181
|
-
## Versioning & Compatibility Policy
|
|
182
|
-
|
|
183
|
-
Wexample packages follow **Semantic Versioning** (SemVer):
|
|
184
|
-
|
|
185
|
-
- **MAJOR**: Breaking changes
|
|
186
|
-
- **MINOR**: New features, backward compatible
|
|
187
|
-
- **PATCH**: Bug fixes, backward compatible
|
|
188
|
-
|
|
189
|
-
We maintain backward compatibility within major versions and provide clear migration guides for breaking changes.
|
|
190
|
-
|
|
191
|
-
## License
|
|
192
|
-
|
|
193
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
194
|
-
|
|
195
|
-
Free to use in both personal and commercial projects.
|
|
196
|
-
|
|
197
|
-
## About us
|
|
198
|
-
|
|
199
|
-
[Wexample](https://wexample.com) stands as a cornerstone of the digital ecosystem — a collective of seasoned engineers, researchers, and creators driven by a relentless pursuit of technological excellence. More than a media platform, it has grown into a vibrant community where innovation meets craftsmanship, and where every line of code reflects a commitment to clarity, durability, and shared intelligence.
|
|
200
|
-
|
|
201
|
-
This packages suite embodies this spirit. Trusted by professionals and enthusiasts alike, it delivers a consistent, high-quality foundation for modern development — open, elegant, and battle-tested. Its reputation is built on years of collaboration, refinement, and rigorous attention to detail, making it a natural choice for those who demand both robustness and beauty in their tools.
|
|
202
|
-
|
|
203
|
-
Wexample cultivates a culture of mastery. Each package, each contribution carries the mark of a community that values precision, ethics, and innovation — a community proud to shape the future of digital craftsmanship.
|
|
204
|
-
|
|
205
|
-
Built and maintained by [Romain Weeger](https://weeger.io).
|
|
206
|
-
|
|
207
|
-
## Known Limitations & Roadmap
|
|
208
|
-
|
|
209
|
-
Current limitations and planned features are tracked in the GitHub issues.
|
|
210
|
-
|
|
211
|
-
See the [project roadmap](https://github.com/wexample/python-storage/issues) for upcoming features and improvements.
|
|
212
|
-
|
|
213
|
-
## Status & Compatibility
|
|
214
|
-
|
|
215
|
-
**Maturity**: Production-ready
|
|
216
|
-
|
|
217
|
-
**Python Support**: >=3.10
|
|
218
|
-
|
|
219
|
-
**OS Support**: Linux, macOS, Windows
|
|
220
|
-
|
|
221
|
-
**Status**: Actively maintained
|
|
222
|
-
|
|
223
|
-
## Useful Links
|
|
224
|
-
|
|
225
|
-
- **Homepage**: https://github.com/wexample/python-storage
|
|
226
|
-
- **Issue Tracker**: https://github.com/wexample/python-storage/issues
|
|
227
|
-
- **Discussions**: https://github.com/wexample/python-storage/discussions
|
|
228
|
-
- **PyPI**: [pypi.org/project/wexample-storage](https://pypi.org/project/wexample-storage/)
|
|
229
|
-
|
|
230
|
-
## Migration Notes
|
|
231
|
-
|
|
232
|
-
When upgrading between major versions, refer to the migration guides in the documentation.
|
|
233
|
-
|
|
234
|
-
Breaking changes are clearly documented with upgrade paths and examples.
|