object-storage-client 0.4.2__tar.gz → 0.4.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/Cargo.lock +1 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/Cargo.toml +1 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/PKG-INFO +139 -110
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/README.md +138 -109
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/BEHAVIOR.md +34 -17
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/CONFIGURATION.md +69 -25
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/DEVELOPMENT.md +35 -24
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/ENCRYPTION.md +56 -34
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/GUI.md +56 -32
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/TESTING.md +22 -11
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/docs/UPGRADING.md +60 -9
- object_storage_client-0.4.4/docs/images/osc-gui.png +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/python/object_storage_client/__init__.pyi +3 -2
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/azure.rs +271 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/mod.rs +150 -7
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/s3.rs +394 -22
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/client.rs +203 -33
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/connection/aws.rs +59 -3
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/connection/mod.rs +90 -2
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/tests.rs +3 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/keys.rs +5 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/model.rs +68 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/ops/tests.rs +74 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/ops.rs +14 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/address_bar.rs +1 -7
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/panel.rs +58 -13
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/tests.rs +155 -4
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/worker/tests.rs +8 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/python/mod.rs +1 -1
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/gui_ops.rs +53 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_plain_client.py +37 -0
- object_storage_client-0.4.2/docs/images/osc-gui.png +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.cargo/config.toml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.docker/garage.toml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.gitattributes +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.github/dependabot.yml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.github/workflows/ci.yaml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.github/workflows/perf.yaml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.github/workflows/tests.yaml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.gitignore +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/.python-version +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/LICENSE +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/benches/alloc-counter/Cargo.toml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/benches/alloc-counter/src/lib.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/benches/common/mod.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/benches/throughput.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/compose.yml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/deny.toml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/proptest-regressions/encryption/context.txt +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/pyproject.toml +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/python/object_storage_client/__init__.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/python/object_storage_client/_stream.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/python/object_storage_client/_types.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/python/object_storage_client/py.typed +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/scripts/esdk_oracle.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/scripts/hostile-env.sh +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/scripts/perf_smoke.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/aws_profile.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/body.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/connector.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/gcs.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/http.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/local.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/backend/stub.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/bin/osc-gui.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/args.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/enc/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/enc.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/harden.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/keyring.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/output.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/transfer/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/transfer.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/tree/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli/tree.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/cli.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/client/local_delete.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/codec.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/connection/azure.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/connection/gcp.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/connection/sso.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/context.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/envelope/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/envelope.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/fixtures.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/frame.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/fuzzing.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/header.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/keys/keyring/tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/keys/keyring.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/keys/mod.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/keys/raw_aes.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/mod.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/suite.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/encryption/tamper_tests.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/forward.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/fsmode.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/close.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/conn.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/job.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/nav.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/tests/close.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/tests/conn.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/tests/jobs.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action/tests/nav.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/action.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/app.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/args.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/display.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/location.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/startup.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/dialogs.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/startup_error.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/status.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view/toolbar.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/view.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui/worker.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/gui.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/hex.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/lib.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/main.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/python/client.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/python/encryption.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/python/stream.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/registry.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/sign.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/text.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/src/upload.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/aws_profile.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/cli.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/client.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/encryption.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/golden.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/sign.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional/suite.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/functional.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/azure_storage_ops.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/common/azurite.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/common/gcs.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/common/mod.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/common/provider.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/encryption.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/file_roundtrip.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/gcs_storage_ops.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/reencrypt.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/s3_storage_ops.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration/storage_ops.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/integration.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/perf_guards.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/conftest.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_abort_at_exit.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_encrypted_client.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_encryption_runtime.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_free_threading.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_gil_contention.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_list_stream.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_pipe_native.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_profile.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_roundtrip.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_shared_client.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_stream_leak.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_stream_sticky.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_stub_conformance.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_subinterpreter.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/python/test_worker_threads.py +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/support/mini_http.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/support/mock.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/support/mod.rs +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/README.md +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/ctx_long.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/ctx_min.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/ctx_utf8.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/manifest.json +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p0_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p0_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p1_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p1_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p200000_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p200000_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4095_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4095_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4096_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4096_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4097_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p4097_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65535_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65535_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65536_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65536_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65537_f4096.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/esdk/p65537_f65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/README.md +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/context.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/keyring +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/manifest.json +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/old_key.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/size_0.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/size_1.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/size_200000.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/golden/v1/size_65536.bin +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/keyrings/README.md +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/keyrings/one.keyring +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/keyrings/other.keyring +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/tests/vectors/keyrings/two.keyring +0 -0
- {object_storage_client-0.4.2 → object_storage_client-0.4.4}/uv.lock +0 -0
|
@@ -9,7 +9,7 @@ license = "GPL-3.0-only"
|
|
|
9
9
|
name = "object-storage-client"
|
|
10
10
|
readme = "README.md"
|
|
11
11
|
repository = "https://github.com/bixority/object-storage-client"
|
|
12
|
-
version = "0.4.
|
|
12
|
+
version = "0.4.4"
|
|
13
13
|
|
|
14
14
|
[lib]
|
|
15
15
|
# maturin adds the cdylib crate type itself; do not list it here.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: object-storage-client
|
|
3
|
-
Version: 0.4.
|
|
3
|
+
Version: 0.4.4
|
|
4
4
|
Classifier: Programming Language :: Rust
|
|
5
5
|
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
6
6
|
Classifier: Programming Language :: Python :: 3
|
|
@@ -21,12 +21,13 @@ Project-URL: Repository, https://github.com/bixority/object-storage-client
|
|
|
21
21
|
# Object Storage Client
|
|
22
22
|
|
|
23
23
|
One URL-based API for AWS S3, Google Cloud Storage, Azure Blob Storage,
|
|
24
|
-
HTTP(S) and the local filesystem, as a Rust library, Python bindings
|
|
25
|
-
`osc` command-line tool. The scheme of each
|
|
26
|
-
client can copy and move objects between
|
|
27
|
-
bounded memory (see
|
|
28
|
-
for the limits) and
|
|
29
|
-
|
|
24
|
+
HTTP(S) and the local filesystem, as a Rust library, Python bindings, the
|
|
25
|
+
`osc` command-line tool and the `osc-gui` desktop browser. The scheme of each
|
|
26
|
+
URL picks the backend, so a single client can copy and move objects between
|
|
27
|
+
providers, stream large objects in bounded memory (see
|
|
28
|
+
[Behaviour](docs/BEHAVIOR.md#large-objects-and-streaming) for the limits) and
|
|
29
|
+
generate pre-signed URLs. Optional client-side encryption keeps object contents
|
|
30
|
+
hidden from the storage provider.
|
|
30
31
|
|
|
31
32
|
- **One API for every backend**, including cross-provider copy and move.
|
|
32
33
|
- **Streaming** uploads and downloads, with multipart uploads for large objects.
|
|
@@ -55,6 +56,123 @@ itself: list it, create it or check that it exists. Credentials come from
|
|
|
55
56
|
environment variables or, for `s3://`, from a named AWS profile; see
|
|
56
57
|
[Configuration](docs/CONFIGURATION.md).
|
|
57
58
|
|
|
59
|
+
## Desktop browser (`osc-gui`)
|
|
60
|
+
|
|
61
|
+
`osc-gui` is a two-panel desktop browser for the same stores as `osc`. Each
|
|
62
|
+
panel shows one folder on its own connection, and you copy, move, rename and
|
|
63
|
+
delete objects and whole folders within a panel or between the two, also across
|
|
64
|
+
accounts and providers. It finds the connections already set up on the machine
|
|
65
|
+
and takes the same encryption options as `osc`.
|
|
66
|
+
|
|
67
|
+

|
|
68
|
+
|
|
69
|
+
### Install
|
|
70
|
+
|
|
71
|
+
The browser is behind the optional `gui` feature, so `osc` alone is unaffected.
|
|
72
|
+
Install both binaries from the registry:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
76
|
+
cargo install object-storage-client --registry bixority --features gui
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Or from a checkout:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
cargo install --path . --features gui
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
On Linux, `osc-gui` needs a Wayland session; X11 is not supported.
|
|
86
|
+
|
|
87
|
+
### Run
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
osc-gui # both panels on Local, the current directory
|
|
91
|
+
osc-gui --keyring keyring # copies encrypt and decrypt
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Each panel has a connection selector for Local and what is found on the
|
|
95
|
+
machine: AWS profiles, provider environment variables, gcloud configurations and
|
|
96
|
+
Azure CLI storage accounts; a cloud connection lists its buckets. Type any
|
|
97
|
+
matching URL or path into a panel's address bar and press Enter. Archived
|
|
98
|
+
objects (S3 Glacier Flexible Retrieval and Deep Archive, Azure Archive tier)
|
|
99
|
+
show ❄ and objects being restored show ⌛; copy, move and rename skip them and
|
|
100
|
+
say why.
|
|
101
|
+
If a connection cannot authenticate, an AWS SSO profile can log in from the
|
|
102
|
+
panel's error banner; for gcloud and Azure the banner shows the command to run.
|
|
103
|
+
The keys and shortcuts are listed in [Desktop browser](docs/GUI.md), which also
|
|
104
|
+
covers limits and closing behaviour.
|
|
105
|
+
|
|
106
|
+
## CLI (`osc`)
|
|
107
|
+
|
|
108
|
+
### Install
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
112
|
+
cargo install object-storage-client --registry bixority
|
|
113
|
+
# or: cargo install --git https://codeberg.org/bixority/object-storage-client
|
|
114
|
+
# or, from a checkout: cargo install --path .
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### Run
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
osc put my_file.txt s3://my-bucket/remote_file.txt # upload a file
|
|
121
|
+
osc put --recursive my_dir s3://my-bucket/remote_dir/ # upload a directory
|
|
122
|
+
osc get gs://my-bucket/data.json ./local_data.json # download
|
|
123
|
+
osc get-stream gs://my-bucket/large_file.bin > large_file.bin # stream to stdout; progress on stderr
|
|
124
|
+
osc --profile dev ls s3://my-bucket/ # credentials from the AWS profile "dev"
|
|
125
|
+
osc cp s3://source-bucket/image.png az://dest-container/image.png # copy between providers
|
|
126
|
+
osc mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt
|
|
127
|
+
osc ls s3://my-bucket/logs/ # names; --url prints full object URLs
|
|
128
|
+
osc rm s3://my-bucket/temp_file.tmp
|
|
129
|
+
osc exists s3://my-bucket/report.pdf # prints true or false
|
|
130
|
+
osc mb s3://my-new-bucket # create a bucket (a directory for local paths)
|
|
131
|
+
osc bucket-exists s3://my-bucket # prints true or false
|
|
132
|
+
|
|
133
|
+
# Pre-signed URLs (S3, GCS, Azure): a download link valid for the default
|
|
134
|
+
# 1 hour, and an upload (PUT) link valid for 15 minutes. With temporary
|
|
135
|
+
# credentials (SSO, an assumed role) an s3:// link stops working when that
|
|
136
|
+
# session ends, whatever the expiry
|
|
137
|
+
osc sign s3://my-bucket/report.pdf
|
|
138
|
+
osc sign --method PUT --expires-in 900 s3://my-bucket/upload.bin
|
|
139
|
+
# S3 only: the upload is rejected unless Content-Length and Content-Type match
|
|
140
|
+
osc sign --method PUT --content-length 1048576 \
|
|
141
|
+
--content-type application/pdf s3://my-bucket/upload.pdf
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`get-stream` writes each chunk to standard output as it arrives, so its output
|
|
145
|
+
is complete only if `osc` exits 0. A failed read, or a truncated or tampered
|
|
146
|
+
encrypted object, stops the output part way: every byte written was
|
|
147
|
+
authenticated, but the object may be cut short. In a pipeline use
|
|
148
|
+
`set -o pipefail`, or give a FILE for an all-or-nothing download.
|
|
149
|
+
|
|
150
|
+
### Encryption
|
|
151
|
+
|
|
152
|
+
With a keyring, `put`, `get`, `get-stream`, `cp` and `mv` encrypt and decrypt
|
|
153
|
+
transparently:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
umask 077 # keyring files must not be readable by others
|
|
157
|
+
osc keyring new > keyring # a keyring with one fresh key
|
|
158
|
+
osc --keyring keyring put report.csv s3://my-bucket/report.csv
|
|
159
|
+
osc --keyring keyring get s3://my-bucket/report.csv report.csv
|
|
160
|
+
osc enc inspect s3://my-bucket/report.csv # header details; no keys needed
|
|
161
|
+
|
|
162
|
+
# Rotation: add a key, wait until every reader has it, activate it, then move
|
|
163
|
+
# objects off the old one (a single reader may use `osc keyring rotate`)
|
|
164
|
+
osc keyring add < keyring > keyring.new && mv keyring.new keyring
|
|
165
|
+
osc keyring activate <new key ID> < keyring > keyring.new && mv keyring.new keyring
|
|
166
|
+
osc --keyring keyring enc reencrypt s3://my-bucket/ --from <old key ID>
|
|
167
|
+
osc enc scan s3://my-bucket/ # objects per key ID; retire the old key only when it has none and the scan exits 0
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
`--keyring` can also come from `OSC_KEYRING_FILE`, or from a command that
|
|
171
|
+
prints the keyring (`OSC_KEYRING_COMMAND`), so the keyring never touches the
|
|
172
|
+
disk. With `--require-encryption` (or `OSC_REQUIRE_ENCRYPTION=1`), `put`, `get`,
|
|
173
|
+
`get-stream`, `cp`, `mv` and `sign` fail when no keyring is configured, instead
|
|
174
|
+
of sending or reading plaintext.
|
|
175
|
+
|
|
58
176
|
## Rust
|
|
59
177
|
|
|
60
178
|
The crate is published to the
|
|
@@ -96,6 +214,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
|
96
214
|
let retrieved = client.get("s3://my-bucket/hello.txt").await?;
|
|
97
215
|
println!("Retrieved: {}", String::from_utf8_lossy(&retrieved));
|
|
98
216
|
|
|
217
|
+
// Stream a large object in bounded memory: the download feeds the upload
|
|
218
|
+
let stream = client.get_stream("s3://my-bucket/hello.txt").await?;
|
|
219
|
+
client.put_stream("file:///tmp/hello_copy.txt", stream).await?;
|
|
220
|
+
|
|
99
221
|
// List the whole bucket from a bucket-root URL (flat, recursive: every key
|
|
100
222
|
// is returned). Pass a prefix such as "s3://my-bucket/logs/" to narrow it.
|
|
101
223
|
let keys = client.list("s3://my-bucket").await?;
|
|
@@ -176,7 +298,7 @@ let health = keyring.status(); // active key, key ages, last reload error
|
|
|
176
298
|
## Python
|
|
177
299
|
|
|
178
300
|
The package is published on [PyPI](https://pypi.org/project/object-storage-client/)
|
|
179
|
-
and needs Python 3.13
|
|
301
|
+
and needs Python 3.13.9 to 3.15:
|
|
180
302
|
|
|
181
303
|
```bash
|
|
182
304
|
pip install object-storage-client
|
|
@@ -231,6 +353,13 @@ async def main():
|
|
|
231
353
|
async for chunk in stream:
|
|
232
354
|
print(f"Chunk size: {len(chunk)}")
|
|
233
355
|
|
|
356
|
+
# Stream an upload from any async iterable of bytes (size_hint= is optional);
|
|
357
|
+
# a ByteStream from get_object_stream is piped natively
|
|
358
|
+
await client.put_object_stream(
|
|
359
|
+
"s3://my-bucket/hello-copy.txt",
|
|
360
|
+
await client.get_object_stream("s3://my-bucket/hello.txt"),
|
|
361
|
+
)
|
|
362
|
+
|
|
234
363
|
# --- Move data across providers with one client ---
|
|
235
364
|
|
|
236
365
|
# Copy S3 -> Google Cloud Storage (source is left in place)
|
|
@@ -274,107 +403,6 @@ data = await secure.get_object("s3://my-bucket/payroll.csv") # authentic
|
|
|
274
403
|
health = keyring.status() # active key, key ages, last reload error
|
|
275
404
|
```
|
|
276
405
|
|
|
277
|
-
## CLI (`osc`)
|
|
278
|
-
|
|
279
|
-
```bash
|
|
280
|
-
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
281
|
-
cargo install object-storage-client --registry bixority
|
|
282
|
-
# or: cargo install --git https://codeberg.org/bixority/object-storage-client
|
|
283
|
-
# or, from a checkout: cargo install --path .
|
|
284
|
-
```
|
|
285
|
-
|
|
286
|
-
```bash
|
|
287
|
-
osc put my_file.txt s3://my-bucket/remote_file.txt # upload a file
|
|
288
|
-
osc put --recursive my_dir s3://my-bucket/remote_dir/ # upload a directory
|
|
289
|
-
osc get gs://my-bucket/data.json ./local_data.json # download
|
|
290
|
-
osc get-stream gs://my-bucket/large_file.bin > large_file.bin # stream to stdout; progress on stderr
|
|
291
|
-
osc --profile dev ls s3://my-bucket/ # credentials from the AWS profile "dev"
|
|
292
|
-
osc cp s3://source-bucket/image.png az://dest-container/image.png # copy between providers
|
|
293
|
-
osc mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt
|
|
294
|
-
osc ls s3://my-bucket/logs/ # names; --url prints full object URLs
|
|
295
|
-
osc rm s3://my-bucket/temp_file.tmp
|
|
296
|
-
osc exists s3://my-bucket/report.pdf # prints true or false
|
|
297
|
-
osc mb s3://my-new-bucket # create a bucket (a directory for local paths)
|
|
298
|
-
osc bucket-exists s3://my-bucket # prints true or false
|
|
299
|
-
|
|
300
|
-
# Pre-signed URLs (S3, GCS, Azure): a download link valid for the default
|
|
301
|
-
# 1 hour, and an upload (PUT) link valid for 15 minutes. With temporary
|
|
302
|
-
# credentials (SSO, an assumed role) an s3:// link stops working when that
|
|
303
|
-
# session ends, whatever the expiry
|
|
304
|
-
osc sign s3://my-bucket/report.pdf
|
|
305
|
-
osc sign --method PUT --expires-in 900 s3://my-bucket/upload.bin
|
|
306
|
-
# S3 only: the upload is rejected unless Content-Length and Content-Type match
|
|
307
|
-
osc sign --method PUT --content-length 1048576 \
|
|
308
|
-
--content-type application/pdf s3://my-bucket/upload.pdf
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
`get-stream` writes each chunk to standard output as it arrives, so its output
|
|
312
|
-
is complete only if `osc` exits 0. A failed read, or a truncated or tampered
|
|
313
|
-
encrypted object, stops the output part way: every byte written was
|
|
314
|
-
authenticated, but the object may be cut short. In a pipeline use
|
|
315
|
-
`set -o pipefail`, or give a FILE for an all-or-nothing download.
|
|
316
|
-
|
|
317
|
-
With a keyring, `put`, `get`, `get-stream`, `cp` and `mv` encrypt and decrypt
|
|
318
|
-
transparently:
|
|
319
|
-
|
|
320
|
-
```bash
|
|
321
|
-
umask 077 # keyring files must not be readable by others
|
|
322
|
-
osc keyring new > keyring # a keyring with one fresh key
|
|
323
|
-
osc --keyring keyring put report.csv s3://my-bucket/report.csv
|
|
324
|
-
osc --keyring keyring get s3://my-bucket/report.csv report.csv
|
|
325
|
-
osc enc inspect s3://my-bucket/report.csv # header details; no keys needed
|
|
326
|
-
|
|
327
|
-
# Rotation: add a key, wait until every reader has it, activate it, then move
|
|
328
|
-
# objects off the old one (a single reader may use `osc keyring rotate`)
|
|
329
|
-
osc keyring add < keyring > keyring.new && mv keyring.new keyring
|
|
330
|
-
osc keyring activate <new key ID> < keyring > keyring.new && mv keyring.new keyring
|
|
331
|
-
osc --keyring keyring enc reencrypt s3://my-bucket/ --from <old key ID>
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
`--keyring` can also come from `OSC_KEYRING_FILE`, or from a command that
|
|
335
|
-
prints the keyring (`OSC_KEYRING_COMMAND`), so the keyring never touches the
|
|
336
|
-
disk.
|
|
337
|
-
|
|
338
|
-
## Desktop browser (`osc-gui`)
|
|
339
|
-
|
|
340
|
-
`osc-gui` is a two-panel desktop browser for the same stores as `osc`. Each
|
|
341
|
-
panel shows one folder, and you copy, move, rename and delete objects and whole
|
|
342
|
-
folders within a panel or between the two. It uses the same credentials and
|
|
343
|
-
encryption options as `osc`.
|
|
344
|
-
|
|
345
|
-

|
|
346
|
-
|
|
347
|
-
### Install
|
|
348
|
-
|
|
349
|
-
The browser is behind the optional `gui` feature, so `osc` alone is unaffected.
|
|
350
|
-
Install both binaries from the registry:
|
|
351
|
-
|
|
352
|
-
```bash
|
|
353
|
-
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
354
|
-
cargo install object-storage-client --registry bixority --features gui
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
Or from a checkout:
|
|
358
|
-
|
|
359
|
-
```bash
|
|
360
|
-
cargo install --path . --features gui
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
On Linux, `osc-gui` needs a Wayland session; X11 is not supported.
|
|
364
|
-
|
|
365
|
-
### Run
|
|
366
|
-
|
|
367
|
-
```bash
|
|
368
|
-
osc-gui # both panels on Local, the current directory
|
|
369
|
-
osc-gui --keyring keyring # copies encrypt and decrypt
|
|
370
|
-
```
|
|
371
|
-
|
|
372
|
-
Each panel has a connection selector for Local and the AWS profiles, gcloud
|
|
373
|
-
configurations and Azure storage accounts found on the machine; a cloud connection
|
|
374
|
-
lists its buckets. Type any matching URL or path into a panel's address bar and
|
|
375
|
-
press Enter. The keys and shortcuts are listed in
|
|
376
|
-
[Desktop browser](docs/GUI.md), which also covers limits and closing behaviour.
|
|
377
|
-
|
|
378
406
|
## Documentation
|
|
379
407
|
|
|
380
408
|
- [Configuration](docs/CONFIGURATION.md): credentials and environment variables
|
|
@@ -382,7 +410,8 @@ press Enter. The keys and shortcuts are listed in
|
|
|
382
410
|
- [Encryption](docs/ENCRYPTION.md): what it protects, keyrings, Kubernetes, key
|
|
383
411
|
rotation, responding to a key leak, the format and interop.
|
|
384
412
|
- [Behaviour and limits](docs/BEHAVIOR.md): URL rules, copy and move, deletes,
|
|
385
|
-
file permissions, large objects, pre-signing, CLI and
|
|
413
|
+
archived objects, file permissions, large objects, pre-signing, CLI and
|
|
414
|
+
Python details.
|
|
386
415
|
- [Desktop browser](docs/GUI.md): `osc-gui`, its keys, limits and closing.
|
|
387
416
|
- [Upgrading](docs/UPGRADING.md): breaking changes between releases.
|
|
388
417
|
- [Development](docs/DEVELOPMENT.md) and [Testing](docs/TESTING.md).
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
# Object Storage Client
|
|
2
2
|
|
|
3
3
|
One URL-based API for AWS S3, Google Cloud Storage, Azure Blob Storage,
|
|
4
|
-
HTTP(S) and the local filesystem, as a Rust library, Python bindings
|
|
5
|
-
`osc` command-line tool. The scheme of each
|
|
6
|
-
client can copy and move objects between
|
|
7
|
-
bounded memory (see
|
|
8
|
-
for the limits) and
|
|
9
|
-
|
|
4
|
+
HTTP(S) and the local filesystem, as a Rust library, Python bindings, the
|
|
5
|
+
`osc` command-line tool and the `osc-gui` desktop browser. The scheme of each
|
|
6
|
+
URL picks the backend, so a single client can copy and move objects between
|
|
7
|
+
providers, stream large objects in bounded memory (see
|
|
8
|
+
[Behaviour](docs/BEHAVIOR.md#large-objects-and-streaming) for the limits) and
|
|
9
|
+
generate pre-signed URLs. Optional client-side encryption keeps object contents
|
|
10
|
+
hidden from the storage provider.
|
|
10
11
|
|
|
11
12
|
- **One API for every backend**, including cross-provider copy and move.
|
|
12
13
|
- **Streaming** uploads and downloads, with multipart uploads for large objects.
|
|
@@ -35,6 +36,123 @@ itself: list it, create it or check that it exists. Credentials come from
|
|
|
35
36
|
environment variables or, for `s3://`, from a named AWS profile; see
|
|
36
37
|
[Configuration](docs/CONFIGURATION.md).
|
|
37
38
|
|
|
39
|
+
## Desktop browser (`osc-gui`)
|
|
40
|
+
|
|
41
|
+
`osc-gui` is a two-panel desktop browser for the same stores as `osc`. Each
|
|
42
|
+
panel shows one folder on its own connection, and you copy, move, rename and
|
|
43
|
+
delete objects and whole folders within a panel or between the two, also across
|
|
44
|
+
accounts and providers. It finds the connections already set up on the machine
|
|
45
|
+
and takes the same encryption options as `osc`.
|
|
46
|
+
|
|
47
|
+

|
|
48
|
+
|
|
49
|
+
### Install
|
|
50
|
+
|
|
51
|
+
The browser is behind the optional `gui` feature, so `osc` alone is unaffected.
|
|
52
|
+
Install both binaries from the registry:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
56
|
+
cargo install object-storage-client --registry bixority --features gui
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Or from a checkout:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
cargo install --path . --features gui
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
On Linux, `osc-gui` needs a Wayland session; X11 is not supported.
|
|
66
|
+
|
|
67
|
+
### Run
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
osc-gui # both panels on Local, the current directory
|
|
71
|
+
osc-gui --keyring keyring # copies encrypt and decrypt
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Each panel has a connection selector for Local and what is found on the
|
|
75
|
+
machine: AWS profiles, provider environment variables, gcloud configurations and
|
|
76
|
+
Azure CLI storage accounts; a cloud connection lists its buckets. Type any
|
|
77
|
+
matching URL or path into a panel's address bar and press Enter. Archived
|
|
78
|
+
objects (S3 Glacier Flexible Retrieval and Deep Archive, Azure Archive tier)
|
|
79
|
+
show ❄ and objects being restored show ⌛; copy, move and rename skip them and
|
|
80
|
+
say why.
|
|
81
|
+
If a connection cannot authenticate, an AWS SSO profile can log in from the
|
|
82
|
+
panel's error banner; for gcloud and Azure the banner shows the command to run.
|
|
83
|
+
The keys and shortcuts are listed in [Desktop browser](docs/GUI.md), which also
|
|
84
|
+
covers limits and closing behaviour.
|
|
85
|
+
|
|
86
|
+
## CLI (`osc`)
|
|
87
|
+
|
|
88
|
+
### Install
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
92
|
+
cargo install object-storage-client --registry bixority
|
|
93
|
+
# or: cargo install --git https://codeberg.org/bixority/object-storage-client
|
|
94
|
+
# or, from a checkout: cargo install --path .
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Run
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
osc put my_file.txt s3://my-bucket/remote_file.txt # upload a file
|
|
101
|
+
osc put --recursive my_dir s3://my-bucket/remote_dir/ # upload a directory
|
|
102
|
+
osc get gs://my-bucket/data.json ./local_data.json # download
|
|
103
|
+
osc get-stream gs://my-bucket/large_file.bin > large_file.bin # stream to stdout; progress on stderr
|
|
104
|
+
osc --profile dev ls s3://my-bucket/ # credentials from the AWS profile "dev"
|
|
105
|
+
osc cp s3://source-bucket/image.png az://dest-container/image.png # copy between providers
|
|
106
|
+
osc mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt
|
|
107
|
+
osc ls s3://my-bucket/logs/ # names; --url prints full object URLs
|
|
108
|
+
osc rm s3://my-bucket/temp_file.tmp
|
|
109
|
+
osc exists s3://my-bucket/report.pdf # prints true or false
|
|
110
|
+
osc mb s3://my-new-bucket # create a bucket (a directory for local paths)
|
|
111
|
+
osc bucket-exists s3://my-bucket # prints true or false
|
|
112
|
+
|
|
113
|
+
# Pre-signed URLs (S3, GCS, Azure): a download link valid for the default
|
|
114
|
+
# 1 hour, and an upload (PUT) link valid for 15 minutes. With temporary
|
|
115
|
+
# credentials (SSO, an assumed role) an s3:// link stops working when that
|
|
116
|
+
# session ends, whatever the expiry
|
|
117
|
+
osc sign s3://my-bucket/report.pdf
|
|
118
|
+
osc sign --method PUT --expires-in 900 s3://my-bucket/upload.bin
|
|
119
|
+
# S3 only: the upload is rejected unless Content-Length and Content-Type match
|
|
120
|
+
osc sign --method PUT --content-length 1048576 \
|
|
121
|
+
--content-type application/pdf s3://my-bucket/upload.pdf
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`get-stream` writes each chunk to standard output as it arrives, so its output
|
|
125
|
+
is complete only if `osc` exits 0. A failed read, or a truncated or tampered
|
|
126
|
+
encrypted object, stops the output part way: every byte written was
|
|
127
|
+
authenticated, but the object may be cut short. In a pipeline use
|
|
128
|
+
`set -o pipefail`, or give a FILE for an all-or-nothing download.
|
|
129
|
+
|
|
130
|
+
### Encryption
|
|
131
|
+
|
|
132
|
+
With a keyring, `put`, `get`, `get-stream`, `cp` and `mv` encrypt and decrypt
|
|
133
|
+
transparently:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
umask 077 # keyring files must not be readable by others
|
|
137
|
+
osc keyring new > keyring # a keyring with one fresh key
|
|
138
|
+
osc --keyring keyring put report.csv s3://my-bucket/report.csv
|
|
139
|
+
osc --keyring keyring get s3://my-bucket/report.csv report.csv
|
|
140
|
+
osc enc inspect s3://my-bucket/report.csv # header details; no keys needed
|
|
141
|
+
|
|
142
|
+
# Rotation: add a key, wait until every reader has it, activate it, then move
|
|
143
|
+
# objects off the old one (a single reader may use `osc keyring rotate`)
|
|
144
|
+
osc keyring add < keyring > keyring.new && mv keyring.new keyring
|
|
145
|
+
osc keyring activate <new key ID> < keyring > keyring.new && mv keyring.new keyring
|
|
146
|
+
osc --keyring keyring enc reencrypt s3://my-bucket/ --from <old key ID>
|
|
147
|
+
osc enc scan s3://my-bucket/ # objects per key ID; retire the old key only when it has none and the scan exits 0
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
`--keyring` can also come from `OSC_KEYRING_FILE`, or from a command that
|
|
151
|
+
prints the keyring (`OSC_KEYRING_COMMAND`), so the keyring never touches the
|
|
152
|
+
disk. With `--require-encryption` (or `OSC_REQUIRE_ENCRYPTION=1`), `put`, `get`,
|
|
153
|
+
`get-stream`, `cp`, `mv` and `sign` fail when no keyring is configured, instead
|
|
154
|
+
of sending or reading plaintext.
|
|
155
|
+
|
|
38
156
|
## Rust
|
|
39
157
|
|
|
40
158
|
The crate is published to the
|
|
@@ -76,6 +194,10 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
|
76
194
|
let retrieved = client.get("s3://my-bucket/hello.txt").await?;
|
|
77
195
|
println!("Retrieved: {}", String::from_utf8_lossy(&retrieved));
|
|
78
196
|
|
|
197
|
+
// Stream a large object in bounded memory: the download feeds the upload
|
|
198
|
+
let stream = client.get_stream("s3://my-bucket/hello.txt").await?;
|
|
199
|
+
client.put_stream("file:///tmp/hello_copy.txt", stream).await?;
|
|
200
|
+
|
|
79
201
|
// List the whole bucket from a bucket-root URL (flat, recursive: every key
|
|
80
202
|
// is returned). Pass a prefix such as "s3://my-bucket/logs/" to narrow it.
|
|
81
203
|
let keys = client.list("s3://my-bucket").await?;
|
|
@@ -156,7 +278,7 @@ let health = keyring.status(); // active key, key ages, last reload error
|
|
|
156
278
|
## Python
|
|
157
279
|
|
|
158
280
|
The package is published on [PyPI](https://pypi.org/project/object-storage-client/)
|
|
159
|
-
and needs Python 3.13
|
|
281
|
+
and needs Python 3.13.9 to 3.15:
|
|
160
282
|
|
|
161
283
|
```bash
|
|
162
284
|
pip install object-storage-client
|
|
@@ -211,6 +333,13 @@ async def main():
|
|
|
211
333
|
async for chunk in stream:
|
|
212
334
|
print(f"Chunk size: {len(chunk)}")
|
|
213
335
|
|
|
336
|
+
# Stream an upload from any async iterable of bytes (size_hint= is optional);
|
|
337
|
+
# a ByteStream from get_object_stream is piped natively
|
|
338
|
+
await client.put_object_stream(
|
|
339
|
+
"s3://my-bucket/hello-copy.txt",
|
|
340
|
+
await client.get_object_stream("s3://my-bucket/hello.txt"),
|
|
341
|
+
)
|
|
342
|
+
|
|
214
343
|
# --- Move data across providers with one client ---
|
|
215
344
|
|
|
216
345
|
# Copy S3 -> Google Cloud Storage (source is left in place)
|
|
@@ -254,107 +383,6 @@ data = await secure.get_object("s3://my-bucket/payroll.csv") # authentic
|
|
|
254
383
|
health = keyring.status() # active key, key ages, last reload error
|
|
255
384
|
```
|
|
256
385
|
|
|
257
|
-
## CLI (`osc`)
|
|
258
|
-
|
|
259
|
-
```bash
|
|
260
|
-
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
261
|
-
cargo install object-storage-client --registry bixority
|
|
262
|
-
# or: cargo install --git https://codeberg.org/bixority/object-storage-client
|
|
263
|
-
# or, from a checkout: cargo install --path .
|
|
264
|
-
```
|
|
265
|
-
|
|
266
|
-
```bash
|
|
267
|
-
osc put my_file.txt s3://my-bucket/remote_file.txt # upload a file
|
|
268
|
-
osc put --recursive my_dir s3://my-bucket/remote_dir/ # upload a directory
|
|
269
|
-
osc get gs://my-bucket/data.json ./local_data.json # download
|
|
270
|
-
osc get-stream gs://my-bucket/large_file.bin > large_file.bin # stream to stdout; progress on stderr
|
|
271
|
-
osc --profile dev ls s3://my-bucket/ # credentials from the AWS profile "dev"
|
|
272
|
-
osc cp s3://source-bucket/image.png az://dest-container/image.png # copy between providers
|
|
273
|
-
osc mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt
|
|
274
|
-
osc ls s3://my-bucket/logs/ # names; --url prints full object URLs
|
|
275
|
-
osc rm s3://my-bucket/temp_file.tmp
|
|
276
|
-
osc exists s3://my-bucket/report.pdf # prints true or false
|
|
277
|
-
osc mb s3://my-new-bucket # create a bucket (a directory for local paths)
|
|
278
|
-
osc bucket-exists s3://my-bucket # prints true or false
|
|
279
|
-
|
|
280
|
-
# Pre-signed URLs (S3, GCS, Azure): a download link valid for the default
|
|
281
|
-
# 1 hour, and an upload (PUT) link valid for 15 minutes. With temporary
|
|
282
|
-
# credentials (SSO, an assumed role) an s3:// link stops working when that
|
|
283
|
-
# session ends, whatever the expiry
|
|
284
|
-
osc sign s3://my-bucket/report.pdf
|
|
285
|
-
osc sign --method PUT --expires-in 900 s3://my-bucket/upload.bin
|
|
286
|
-
# S3 only: the upload is rejected unless Content-Length and Content-Type match
|
|
287
|
-
osc sign --method PUT --content-length 1048576 \
|
|
288
|
-
--content-type application/pdf s3://my-bucket/upload.pdf
|
|
289
|
-
```
|
|
290
|
-
|
|
291
|
-
`get-stream` writes each chunk to standard output as it arrives, so its output
|
|
292
|
-
is complete only if `osc` exits 0. A failed read, or a truncated or tampered
|
|
293
|
-
encrypted object, stops the output part way: every byte written was
|
|
294
|
-
authenticated, but the object may be cut short. In a pipeline use
|
|
295
|
-
`set -o pipefail`, or give a FILE for an all-or-nothing download.
|
|
296
|
-
|
|
297
|
-
With a keyring, `put`, `get`, `get-stream`, `cp` and `mv` encrypt and decrypt
|
|
298
|
-
transparently:
|
|
299
|
-
|
|
300
|
-
```bash
|
|
301
|
-
umask 077 # keyring files must not be readable by others
|
|
302
|
-
osc keyring new > keyring # a keyring with one fresh key
|
|
303
|
-
osc --keyring keyring put report.csv s3://my-bucket/report.csv
|
|
304
|
-
osc --keyring keyring get s3://my-bucket/report.csv report.csv
|
|
305
|
-
osc enc inspect s3://my-bucket/report.csv # header details; no keys needed
|
|
306
|
-
|
|
307
|
-
# Rotation: add a key, wait until every reader has it, activate it, then move
|
|
308
|
-
# objects off the old one (a single reader may use `osc keyring rotate`)
|
|
309
|
-
osc keyring add < keyring > keyring.new && mv keyring.new keyring
|
|
310
|
-
osc keyring activate <new key ID> < keyring > keyring.new && mv keyring.new keyring
|
|
311
|
-
osc --keyring keyring enc reencrypt s3://my-bucket/ --from <old key ID>
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
`--keyring` can also come from `OSC_KEYRING_FILE`, or from a command that
|
|
315
|
-
prints the keyring (`OSC_KEYRING_COMMAND`), so the keyring never touches the
|
|
316
|
-
disk.
|
|
317
|
-
|
|
318
|
-
## Desktop browser (`osc-gui`)
|
|
319
|
-
|
|
320
|
-
`osc-gui` is a two-panel desktop browser for the same stores as `osc`. Each
|
|
321
|
-
panel shows one folder, and you copy, move, rename and delete objects and whole
|
|
322
|
-
folders within a panel or between the two. It uses the same credentials and
|
|
323
|
-
encryption options as `osc`.
|
|
324
|
-
|
|
325
|
-

|
|
326
|
-
|
|
327
|
-
### Install
|
|
328
|
-
|
|
329
|
-
The browser is behind the optional `gui` feature, so `osc` alone is unaffected.
|
|
330
|
-
Install both binaries from the registry:
|
|
331
|
-
|
|
332
|
-
```bash
|
|
333
|
-
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
|
|
334
|
-
cargo install object-storage-client --registry bixority --features gui
|
|
335
|
-
```
|
|
336
|
-
|
|
337
|
-
Or from a checkout:
|
|
338
|
-
|
|
339
|
-
```bash
|
|
340
|
-
cargo install --path . --features gui
|
|
341
|
-
```
|
|
342
|
-
|
|
343
|
-
On Linux, `osc-gui` needs a Wayland session; X11 is not supported.
|
|
344
|
-
|
|
345
|
-
### Run
|
|
346
|
-
|
|
347
|
-
```bash
|
|
348
|
-
osc-gui # both panels on Local, the current directory
|
|
349
|
-
osc-gui --keyring keyring # copies encrypt and decrypt
|
|
350
|
-
```
|
|
351
|
-
|
|
352
|
-
Each panel has a connection selector for Local and the AWS profiles, gcloud
|
|
353
|
-
configurations and Azure storage accounts found on the machine; a cloud connection
|
|
354
|
-
lists its buckets. Type any matching URL or path into a panel's address bar and
|
|
355
|
-
press Enter. The keys and shortcuts are listed in
|
|
356
|
-
[Desktop browser](docs/GUI.md), which also covers limits and closing behaviour.
|
|
357
|
-
|
|
358
386
|
## Documentation
|
|
359
387
|
|
|
360
388
|
- [Configuration](docs/CONFIGURATION.md): credentials and environment variables
|
|
@@ -362,7 +390,8 @@ press Enter. The keys and shortcuts are listed in
|
|
|
362
390
|
- [Encryption](docs/ENCRYPTION.md): what it protects, keyrings, Kubernetes, key
|
|
363
391
|
rotation, responding to a key leak, the format and interop.
|
|
364
392
|
- [Behaviour and limits](docs/BEHAVIOR.md): URL rules, copy and move, deletes,
|
|
365
|
-
file permissions, large objects, pre-signing, CLI and
|
|
393
|
+
archived objects, file permissions, large objects, pre-signing, CLI and
|
|
394
|
+
Python details.
|
|
366
395
|
- [Desktop browser](docs/GUI.md): `osc-gui`, its keys, limits and closing.
|
|
367
396
|
- [Upgrading](docs/UPGRADING.md): breaking changes between releases.
|
|
368
397
|
- [Development](docs/DEVELOPMENT.md) and [Testing](docs/TESTING.md).
|