conta 0.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- conta-0.0.1/.gitignore +105 -0
- conta-0.0.1/PKG-INFO +60 -0
- conta-0.0.1/README.md +51 -0
- conta-0.0.1/conta/__init__.py +626 -0
- conta-0.0.1/pyproject.toml +22 -0
- conta-0.0.1/tests/test_conta.py +754 -0
conta-0.0.1/.gitignore
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
/cmake-build-debug
|
|
2
|
+
/cmake-build-release
|
|
3
|
+
/cmake-build-debug-cuda
|
|
4
|
+
/cmake-build-release-clang
|
|
5
|
+
/cmake-build-release-clang-16
|
|
6
|
+
/cmake-build-release-clang-16-package
|
|
7
|
+
/cmake-build-release-gcc11
|
|
8
|
+
/build_test
|
|
9
|
+
/build-coverage
|
|
10
|
+
/cmake-build-*
|
|
11
|
+
/build_export
|
|
12
|
+
/conan_build
|
|
13
|
+
/build_car
|
|
14
|
+
/install
|
|
15
|
+
/test_results
|
|
16
|
+
/conan_provider.cmake
|
|
17
|
+
/**/*tfevents*
|
|
18
|
+
/lightning_logs
|
|
19
|
+
/deepsweep_runs
|
|
20
|
+
.dependencies
|
|
21
|
+
*.dot
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
/.idea
|
|
25
|
+
|
|
26
|
+
.vscode/settings.json
|
|
27
|
+
|
|
28
|
+
/**/*.h5
|
|
29
|
+
/**/*.hdf5
|
|
30
|
+
/plots
|
|
31
|
+
/build
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
/**/*.swp
|
|
35
|
+
/Testing
|
|
36
|
+
/tests/docker/context
|
|
37
|
+
/**/*.png
|
|
38
|
+
/logs
|
|
39
|
+
/experiments
|
|
40
|
+
|
|
41
|
+
/data
|
|
42
|
+
/**/__pycache__
|
|
43
|
+
/**/.gdb_history
|
|
44
|
+
/logs_old
|
|
45
|
+
/MUJOCO_LOG.TXT
|
|
46
|
+
/**/*.tfevents
|
|
47
|
+
|
|
48
|
+
/build-wasm
|
|
49
|
+
/.vs
|
|
50
|
+
/out
|
|
51
|
+
/build_conan
|
|
52
|
+
/CMakeUserPresets.json
|
|
53
|
+
|
|
54
|
+
/checkpoints
|
|
55
|
+
/checkpoints_old
|
|
56
|
+
/**/.DS_Store
|
|
57
|
+
/**/*.json
|
|
58
|
+
!/static/ide/package.json
|
|
59
|
+
!/static/ide/examples.json
|
|
60
|
+
!/tools/ide/dependencies.json
|
|
61
|
+
/tmp
|
|
62
|
+
/lldb_type_dump.txt
|
|
63
|
+
/wandb
|
|
64
|
+
/your_project_name
|
|
65
|
+
/**/*.out
|
|
66
|
+
/**/*.pdf
|
|
67
|
+
/**/*.tar.gz
|
|
68
|
+
/experiments*
|
|
69
|
+
/**/*.nsys-rep
|
|
70
|
+
/**/*.sqlite
|
|
71
|
+
/**/*.sqlite-wal
|
|
72
|
+
/**/*.sqlite-shm
|
|
73
|
+
/**/*.qdstrm
|
|
74
|
+
/perf.data
|
|
75
|
+
/.venv*
|
|
76
|
+
/**/.ipynb_checkpoints
|
|
77
|
+
/**/*.ipynb
|
|
78
|
+
!/interface/python/hyperdrone/hyperdrone/examples/*.ipynb
|
|
79
|
+
/**/*.pth
|
|
80
|
+
/foundation-model-experiments
|
|
81
|
+
/1k-experiments
|
|
82
|
+
/*.tar
|
|
83
|
+
/build*
|
|
84
|
+
/resnet_ast.txt
|
|
85
|
+
/runs
|
|
86
|
+
/videos
|
|
87
|
+
*.bin
|
|
88
|
+
/**/*.glb
|
|
89
|
+
/**/*.mp4
|
|
90
|
+
.cache
|
|
91
|
+
/regression-study
|
|
92
|
+
.claude
|
|
93
|
+
/**/*.tflite
|
|
94
|
+
/**/*.csv
|
|
95
|
+
raytracing_pose_trace*
|
|
96
|
+
camera_poses.txt
|
|
97
|
+
/static/hyperdrone/blob
|
|
98
|
+
!/static/hyperdrone/pose.json
|
|
99
|
+
!/static/hyperdrone/registry.json
|
|
100
|
+
|
|
101
|
+
# the hyperdrone python golden corpus is checked in (the cross-language stability anchor)
|
|
102
|
+
!interface/python/hyperdrone/tests/env/golden/*
|
|
103
|
+
|
|
104
|
+
# browser IDE artifacts (tools/ide/toolchain superbuild, tools/ide/bundle.sh)
|
|
105
|
+
static/ide/build
|
conta-0.0.1/PKG-INFO
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: conta
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: Content Addressed Blob Storage
|
|
5
|
+
Author-email: Jonas Eschmann <jonas.eschmann@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Requires-Python: >=3.9
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
|
|
10
|
+
## Conta: Content Addressed Blob Storage
|
|
11
|
+
|
|
12
|
+
Use `load()` when you want the blob's bytes in memory. It keeps downloaded gzip copies compressed in the cache by default, saving disk space. Use `resolve()` when a library needs a filename, or for seeking, mmap, and blobs too large to load into memory.
|
|
13
|
+
|
|
14
|
+
```python
|
|
15
|
+
import conta
|
|
16
|
+
|
|
17
|
+
data = conta.load(reference) # bytes
|
|
18
|
+
path = conta.resolve(reference) # pathlib.Path, plain file
|
|
19
|
+
data = conta.load(reference, compressed_cache=False) # bytes, keep a plain cache
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`reference` is a SHA-1 or a manifest reference accepted by the existing path API. `resolve_all(references)` and the `conta` CLI still return plain filesystem paths.
|
|
23
|
+
|
|
24
|
+
Both APIs share the same cache directory: `<sha1>` contains the original bytes and `<sha1>.gz` contains gzip bytes. Existing plain files take precedence. When only gzip is cached, `load()` decodes directly into memory; `resolve()` and `load(..., compressed_cache=False)` materialize a verified plain file for subsequent calls. Existing alternate copies are preserved. A cold gzip download retains only the requested cache representation; raw-only sources remain raw, with no local recompression or automatic cache migration.
|
|
25
|
+
|
|
26
|
+
To avoid repeated decompression while keeping the memory API, set `CONTA_COMPRESSED_CACHE=0` (default: `1`), or set `Config.compressed_cache=False`. Python's `compressed_cache` keyword overrides the supplied config. Without an explicit config, both APIs use `config_from_environment()`; an explicit config takes precedence over environment settings. The setting changes `load()`'s cache policy, never its returned bytes, and does not change `resolve()`.
|
|
27
|
+
|
|
28
|
+
```python
|
|
29
|
+
config = conta.config_from_environment()
|
|
30
|
+
config.compressed_cache = False
|
|
31
|
+
data = conta.load(reference, config)
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`CONTA_CACHE` selects the cache directory. `CONTA_ROOT` selects an authoritative offline store: plain root files are used directly, and gzip root files can be loaded into memory or expanded into `CONTA_CACHE` for path access. Conta never writes into the root; if neither representation exists there, the lookup fails without network access. `CONTA_URL` remains a direct raw-blob URL override that bypasses index discovery.
|
|
35
|
+
|
|
36
|
+
The v1 index advertises per-location alternatives as `{"id": "primary", "compressed": ["gz"]}`. Clients try supported compressed formats before raw at each location and ignore unknown extensions. Downloads and gzip decoding verify the original content's SHA-1 and, when available, SHA-256 before returning data or publishing cache files atomically. Python uses its standard-library gzip decoder.
|
|
37
|
+
|
|
38
|
+
### C++
|
|
39
|
+
|
|
40
|
+
```cpp
|
|
41
|
+
#include <conta/conta.h>
|
|
42
|
+
#include <stdexcept>
|
|
43
|
+
|
|
44
|
+
std::vector<std::uint8_t> data;
|
|
45
|
+
std::string path, error;
|
|
46
|
+
if(!conta::load(sha1, data, error)) {
|
|
47
|
+
throw std::runtime_error(error);
|
|
48
|
+
}
|
|
49
|
+
if(!conta::resolve(sha1, path, error)) {
|
|
50
|
+
throw std::runtime_error(error);
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
auto config = conta::config_from_environment();
|
|
54
|
+
config.compressed_cache = false;
|
|
55
|
+
if(!conta::load(config, sha1, data, error)) {
|
|
56
|
+
throw std::runtime_error(error);
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The C++ API uses SHA-1 strings and returns success as `bool`, with diagnostics in `error`. RLtools enables gzip through its existing optional zlib dependency. Standalone header consumers can define `CONTA_ENABLE_ZLIB` and link zlib; without it, online lookups use raw copies, and a gzip-only offline store reports that decoding requires zlib.
|
conta-0.0.1/README.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
## Conta: Content Addressed Blob Storage
|
|
2
|
+
|
|
3
|
+
Use `load()` when you want the blob's bytes in memory. It keeps downloaded gzip copies compressed in the cache by default, saving disk space. Use `resolve()` when a library needs a filename, or for seeking, mmap, and blobs too large to load into memory.
|
|
4
|
+
|
|
5
|
+
```python
|
|
6
|
+
import conta
|
|
7
|
+
|
|
8
|
+
data = conta.load(reference) # bytes
|
|
9
|
+
path = conta.resolve(reference) # pathlib.Path, plain file
|
|
10
|
+
data = conta.load(reference, compressed_cache=False) # bytes, keep a plain cache
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`reference` is a SHA-1 or a manifest reference accepted by the existing path API. `resolve_all(references)` and the `conta` CLI still return plain filesystem paths.
|
|
14
|
+
|
|
15
|
+
Both APIs share the same cache directory: `<sha1>` contains the original bytes and `<sha1>.gz` contains gzip bytes. Existing plain files take precedence. When only gzip is cached, `load()` decodes directly into memory; `resolve()` and `load(..., compressed_cache=False)` materialize a verified plain file for subsequent calls. Existing alternate copies are preserved. A cold gzip download retains only the requested cache representation; raw-only sources remain raw, with no local recompression or automatic cache migration.
|
|
16
|
+
|
|
17
|
+
To avoid repeated decompression while keeping the memory API, set `CONTA_COMPRESSED_CACHE=0` (default: `1`), or set `Config.compressed_cache=False`. Python's `compressed_cache` keyword overrides the supplied config. Without an explicit config, both APIs use `config_from_environment()`; an explicit config takes precedence over environment settings. The setting changes `load()`'s cache policy, never its returned bytes, and does not change `resolve()`.
|
|
18
|
+
|
|
19
|
+
```python
|
|
20
|
+
config = conta.config_from_environment()
|
|
21
|
+
config.compressed_cache = False
|
|
22
|
+
data = conta.load(reference, config)
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
`CONTA_CACHE` selects the cache directory. `CONTA_ROOT` selects an authoritative offline store: plain root files are used directly, and gzip root files can be loaded into memory or expanded into `CONTA_CACHE` for path access. Conta never writes into the root; if neither representation exists there, the lookup fails without network access. `CONTA_URL` remains a direct raw-blob URL override that bypasses index discovery.
|
|
26
|
+
|
|
27
|
+
The v1 index advertises per-location alternatives as `{"id": "primary", "compressed": ["gz"]}`. Clients try supported compressed formats before raw at each location and ignore unknown extensions. Downloads and gzip decoding verify the original content's SHA-1 and, when available, SHA-256 before returning data or publishing cache files atomically. Python uses its standard-library gzip decoder.
|
|
28
|
+
|
|
29
|
+
### C++
|
|
30
|
+
|
|
31
|
+
```cpp
|
|
32
|
+
#include <conta/conta.h>
|
|
33
|
+
#include <stdexcept>
|
|
34
|
+
|
|
35
|
+
std::vector<std::uint8_t> data;
|
|
36
|
+
std::string path, error;
|
|
37
|
+
if(!conta::load(sha1, data, error)) {
|
|
38
|
+
throw std::runtime_error(error);
|
|
39
|
+
}
|
|
40
|
+
if(!conta::resolve(sha1, path, error)) {
|
|
41
|
+
throw std::runtime_error(error);
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
auto config = conta::config_from_environment();
|
|
45
|
+
config.compressed_cache = false;
|
|
46
|
+
if(!conta::load(config, sha1, data, error)) {
|
|
47
|
+
throw std::runtime_error(error);
|
|
48
|
+
}
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
The C++ API uses SHA-1 strings and returns success as `bool`, with diagnostics in `error`. RLtools enables gzip through its existing optional zlib dependency. Standalone header consumers can define `CONTA_ENABLE_ZLIB` and link zlib; without it, online lookups use raw copies, and a gzip-only offline store reports that decoding requires zlib.
|