occystrap 0.3.0__py3-none-any.whl → 0.4.1__py3-none-any.whl

occystrap-0.4.1.dist-info/METADATA

@@ -0,0 +1,444 @@
+Metadata-Version: 2.4
+Name: occystrap
+Version: 0.4.1
+Summary: occystrap: docker and OCI container tools
+Author-email: Michael Still <mikal@stillhq.com>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/shakenfist/occystrap
+Project-URL: Bug Tracker, https://github.com/shakenfist/occystrap/issues
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: AUTHORS
+Requires-Dist: click>=7.1.1
+Requires-Dist: requests
+Requires-Dist: requests-unixsocket
+Requires-Dist: prettytable
+Requires-Dist: oslo.concurrency
+Requires-Dist: shakenfist-utilities
+Provides-Extra: test
+Requires-Dist: coverage; extra == "test"
+Requires-Dist: testtools; extra == "test"
+Requires-Dist: mock; extra == "test"
+Requires-Dist: stestr; extra == "test"
+Requires-Dist: flake8; extra == "test"
+Dynamic: license-file
+
+# Occy Strap
+
+Occy Strap is a simple set of Docker and OCI container tools, which can be used
+either for container forensics or for implementing an OCI orchestrator,
+depending on your needs. This is a very early implementation, so be braced for
+impact.
+
+## Quick Start with URI-Style Commands
+
+The recommended way to use Occy Strap is with the new URI-style `process` and
+`search` commands:
+
+```
+# Download from registry to tarball
+occystrap process registry://docker.io/library/busybox:latest tar://busybox.tar
+
+# Download from registry to directory
+occystrap process registry://docker.io/library/centos:7 dir://centos7
+
+# Export from local Docker to tarball with timestamp normalization
+occystrap process docker://myimage:v1 tar://output.tar -f normalize-timestamps
+
+# Search for files in an image
+occystrap search registry://docker.io/library/busybox:latest "bin/*sh"
+```
+
+## The `process` Command
+
+The `process` command takes a source URI, a destination URI, and optional
+filters:
+
+```
+occystrap process SOURCE DESTINATION [-f FILTER]...
+```
+
+### Input URI Schemes
+
+- `registry://HOST/IMAGE:TAG` - Docker/OCI registry
+- `docker://IMAGE:TAG` - Local Docker daemon
+- `tar:///path/to/file.tar` - Docker-save format tarball
+
+### Output URI Schemes
+
+- `tar:///path/to/output.tar` - Create tarball
+- `dir:///path/to/directory` - Extract to directory
+- `oci:///path/to/bundle` - Create OCI runtime bundle
+- `mounts:///path/to/directory` - Create overlay mounts
+- `docker://IMAGE:TAG` - Load into local Docker daemon
+- `registry://HOST/IMAGE:TAG` - Push to Docker/OCI registry
+
+### URI Options
+
+Options can be passed as query parameters:
+
+```
+# Extract with unique names and expansion
+occystrap process registry://docker.io/library/busybox:latest \
+    "dir://merged?unique_names=true&expand=true"
+
+# Use custom Docker socket
+occystrap process "docker://myimage:v1?socket=/run/podman/podman.sock" \
+    tar://output.tar
+```
+
+### Filters
+
+Filters transform or inspect image elements as they pass through the pipeline:
+
+```
+# Normalize timestamps for reproducible builds
+occystrap process registry://docker.io/library/busybox:latest \
+    tar://busybox.tar -f normalize-timestamps
+
+# Normalize with custom timestamp
+occystrap process registry://docker.io/library/busybox:latest \
+    tar://busybox.tar -f "normalize-timestamps:ts=1609459200"
+
+# Search while creating output (prints matches AND creates tarball)
+occystrap process registry://docker.io/library/busybox:latest \
+    tar://busybox.tar -f "search:pattern=*.conf"
+
+# Chain multiple filters
+occystrap process registry://docker.io/library/busybox:latest \
+    tar://busybox.tar -f normalize-timestamps -f "search:pattern=bin/*"
+
+# Record layer metadata to a JSONL file (inspect filter)
+occystrap process docker://myimage:v1 registry://myregistry/myimage:v1 \
+    -f "inspect:file=layers-before.jsonl" \
+    -f normalize-timestamps \
+    -f "inspect:file=layers-after.jsonl"
+
+# Exclude files matching glob patterns from layers
+occystrap process registry://docker.io/library/python:3.11 \
+    tar://python.tar -f "exclude:pattern=**/.git/**"
+
+# Exclude multiple patterns (comma-separated)
+occystrap process registry://docker.io/library/python:3.11 \
+    tar://python.tar -f "exclude:pattern=**/.git/**,**/__pycache__/**,**/*.pyc"
+
+# Load image directly into local Docker daemon
+occystrap process registry://docker.io/library/busybox:latest \
+    docker://busybox:latest
+
+# Load into Podman
+occystrap process registry://docker.io/library/busybox:latest \
+    "docker://busybox:latest?socket=/run/podman/podman.sock"
+
+# Push image to a registry
+occystrap process docker://myimage:v1 \
+    registry://myregistry.example.com/myuser/myimage:v1
+
+# Push to registry with authentication
+occystrap --username myuser --password mytoken \
+    process tar://image.tar registry://ghcr.io/myorg/myimage:latest
+```
+
+## The `search` Command
+
+Search for files in container image layers:
+
+```
+occystrap search SOURCE PATTERN [--regex] [--script-friendly]
+```
+
+Examples:
+
+```
+# Search registry image
+occystrap search registry://docker.io/library/busybox:latest "bin/*sh"
+
+# Search local Docker image
+occystrap search docker://myimage:v1 "*.conf"
+
+# Search tarball with regex
+occystrap search --regex tar://image.tar ".*\.py$"
+
+# Machine-parseable output
+occystrap search --script-friendly registry://docker.io/library/busybox:latest "*sh"
+```
+
+## Legacy Commands (Deprecated)
+
+The following commands are deprecated but still work for backwards
+compatibility. They will be removed in a future version.
+
+### Downloading an image from a repository and storing as a tarball
+
+Let's say we want to download an image from a repository and store it as a
+local tarball. This is a common thing to want to do in airgapped environments
+for example. You could do this with docker with a `docker pull; docker save`.
+The Occy Strap equivalent is:
+
+```
+occystrap fetch-to-tarfile registry-1.docker.io library/busybox latest busybox.tar
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/library/busybox:latest tar://busybox.tar
+```
+
+In this example we're pulling from the Docker Hub (registry-1.docker.io), and
+are downloading busybox's latest version into a tarball named `busybox.tar`.
+This tarball can be loaded with `docker load -i busybox.tar` on an airgapped
+Docker environment.
+
+### Repeatable builds with normalized timestamps
+
+To make builds more repeatable, you can normalize file access and modification
+times in the image layers. This is useful when you want to ensure that the
+same image content produces the same tarball hash, regardless of when the
+files were originally created:
+
+```
+occystrap fetch-to-tarfile --normalize-timestamps registry-1.docker.io library/busybox latest busybox.tar
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/library/busybox:latest tar://busybox.tar -f normalize-timestamps
+```
+
+This will set all timestamps in the layer tarballs to 0 (Unix epoch: January
+1, 1970). You can also specify a custom timestamp:
+
+```
+occystrap fetch-to-tarfile --normalize-timestamps --timestamp 1609459200 registry-1.docker.io library/busybox latest busybox.tar
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/library/busybox:latest tar://busybox.tar -f "normalize-timestamps:ts=1609459200"
+```
+
+When timestamps are normalized, the layer SHAs are recalculated and the
+manifest is updated to reflect the new hashes. This ensures the tarball
+structure remains consistent and valid.
+
+### Downloading an image from a repository and storing as an extracted tarball
+
+The format of the tarball in the previous example is two JSON configuration
+files and a series of image layers as tarballs inside the main tarball. You
+can write these elements to a directory instead of to a tarball if you'd like
+to inspect them:
+
+```
+occystrap fetch-to-extracted registry-1.docker.io library/centos 7 centos7
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/library/centos:7 dir://centos7
+```
+
+### Downloading an image to a merged directory
+
+In scenarios where image layers are likely to be reused between images, you
+can save disk space by downloading images to a directory which contains more
+than one image:
+
+```
+occystrap fetch-to-extracted --use-unique-names registry-1.docker.io \
+    homeassistant/home-assistant latest merged_images
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/homeassistant/home-assistant:latest \
+    "dir://merged_images?unique_names=true"
+```
+
+### Storing an image tarfile in a merged directory
+
+Sometimes you have image tarfiles instead of images in a registry:
+
+```
+occystrap tarfile-to-extracted --use-unique-names file.tar merged_images
+```
+
+**New equivalent:**
+```
+occystrap process tar://file.tar "dir://merged_images?unique_names=true"
+```
+
+### Exploring the contents of layers and overwritten files
+
+If you'd like the layers to be expanded from their tarballs to the filesystem:
+
+```
+occystrap fetch-to-extracted --expand quay.io \
+    ukhomeofficedigital/centos-base latest ukhomeoffice-centos
+```
+
+**New equivalent:**
+```
+occystrap process registry://quay.io/ukhomeofficedigital/centos-base:latest \
+    "dir://ukhomeoffice-centos?expand=true"
+```
+
+### Generating an OCI runtime bundle
+
+```
+occystrap fetch-to-oci registry-1.docker.io library/hello-world latest bar
+```
+
+**New equivalent:**
+```
+occystrap process registry://registry-1.docker.io/library/hello-world:latest oci://bar
+```
+
+### Searching image layers for files
+
+```
+occystrap search-layers registry-1.docker.io library/busybox latest "bin/*sh"
+```
+
+**New equivalent:**
+```
+occystrap search registry://registry-1.docker.io/library/busybox:latest "bin/*sh"
+```
+
+### Working with local Docker or Podman daemon
+
+```
+occystrap docker-to-tarfile library/busybox latest busybox.tar
+```
+
+**New equivalent:**
+```
+occystrap process docker://library/busybox:latest tar://busybox.tar
+```
+
+For Podman:
+```
+occystrap process "docker://myimage:latest?socket=/run/podman/podman.sock" tar://output.tar
+```
+
+Note: Podman doesn't run a daemon by default. You need to start the socket
+service first:
+
+```
+# For rootless Podman
+systemctl --user start podman.socket
+
+# For rootful Podman
+sudo systemctl start podman.socket
+```
+
+## Authenticating with private registries
+
+To fetch images from private registries (such as GitLab Container Registry,
+AWS ECR, or private Docker Hub repositories), use the `--username` and
+`--password` global options:
+
+```
+occystrap --username myuser --password mytoken \
+    process registry://registry.gitlab.com/mygroup/myimage:latest tar://output.tar
+```
+
+You can also use environment variables to avoid putting credentials on the
+command line:
+
+```
+export OCCYSTRAP_USERNAME=myuser
+export OCCYSTRAP_PASSWORD=mytoken
+occystrap process registry://registry.gitlab.com/mygroup/myimage:latest tar://output.tar
+```
+
+For GitLab Container Registry, the username is typically your GitLab username
+and the password is a personal access token with `read_registry` scope.
+
+## Supporting non-default architectures
+
+Docker image repositories can store multiple versions of a single image, with
+each image corresponding to a different (operating system, cpu architecture,
+cpu variant) tuple. Occy Strap supports letting you specify which to use with
+global command line flags. Occy Strap defaults to linux amd64 if you don't
+specify something different:
+
+```
+occystrap --os linux --architecture arm64 --variant v8 \
+    process registry://registry-1.docker.io/library/busybox:latest dir://busybox
+```
+
+Or via URI query parameters:
+
+```
+occystrap process "registry://registry-1.docker.io/library/busybox:latest?os=linux&arch=arm64&variant=v8" \
+    dir://busybox
+```
+
+## Development
+
+### Install for Development
+
+```
+pip install -e ".[test]"
+```
+
+### Pre-commit Hooks
+
+This project uses pre-commit hooks to validate code before commits. Install them
+with:
+
+```
+pip install pre-commit
+pre-commit install
+```
+
+The hooks run:
+- `actionlint` - GitHub Actions workflow validation
+- `shellcheck` - Shell script linting
+- `tox -eflake8` - Python code style checks
+- `tox -epy3` - Unit tests
+
+To run the hooks manually:
+
+```
+pre-commit run --all-files
+```
+
+### Running Tests
+
+Unit tests are in `occystrap/tests/` and can be run with:
+
+```
+tox -epy3
+```
+
+Functional tests are in `deploy/occystrap_ci/tests/` and are run in CI.
+
+### Releasing
+
+Releases are automated via GitHub Actions. Push a version tag to trigger the
+pipeline:
+
+```
+git tag -s v0.5.0 -m "Release v0.5.0"
+git push origin v0.5.0
+```
+
+The workflow builds the package, signs the tag with Sigstore, publishes to
+PyPI, and creates a GitHub Release. See [RELEASE-SETUP.md](RELEASE-SETUP.md)
+for one-time configuration steps.
+
+## Documentation
+
+For more detailed documentation, see the [docs/](docs/) directory:
+
+- [Installation](docs/installation.md) - Getting started guide
+- [Command Reference](docs/command-reference.md) - Complete CLI reference
+- [Pipeline Architecture](docs/pipeline.md) - How the pipeline works
+- [Use Cases](docs/use-cases.md) - Common scenarios and examples

occystrap-0.4.1.dist-info/RECORD

@@ -0,0 +1,38 @@
+occystrap/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+occystrap/_version.py,sha256=k7cu0JKra64gmMNU_UfA5sw2eNc_GRvf3QmesiYAy8g,704
+occystrap/common.py,sha256=Zm4hHpn8RgSXp0W86HhZzpyXq19QIsLJgp9SxK_1QQg,1300
+occystrap/constants.py,sha256=kmOt-12settGbDTW1efpT3UENRQouG9f0ZjgOqWdrIA,4399
+occystrap/main.py,sha256=bo87uuAJYCZq0YL-EnjpNUsbiaySyvpYPgi-SQF2uaE,15521
+occystrap/pipeline.py,sha256=oiWXB6Wrjl32jC0MI0HksATjxcUQ7o5ntKm864BZxnA,10849
+occystrap/tarformat.py,sha256=Fw5GPntymn7mNzDNiCQr55iNtDQ2GYqes9yBYT1XtJE,4113
+occystrap/uri.py,sha256=uKwy2hb0i1Pjf2QpwSBTobgcH2ve1WNsYSK0n2GpkK8,6858
+occystrap/util.py,sha256=eYZOkamk6hWyFuN4W8NoimR23A_FG1cMl_AfT7vJbEw,3987
+occystrap/filters/__init__.py,sha256=ZT9J7QX4tuEeEJEnty-pGcQQPxv_7lj6ymELleMR_wU,383
+occystrap/filters/base.py,sha256=D1Li3mgNFzrn6BBL7pz35IUGUVWcqwWBfHYXqfQQ1-8,2375
+occystrap/filters/exclude.py,sha256=RrISggbuSMIWOEi1SPKpnorTa6cJVKlIBwAPHxI99D0,4749
+occystrap/filters/inspect.py,sha256=saxsO5vCcZ1-7exkboTa4bkzmglFXY9Fi6U9Q66iV8g,6101
+occystrap/filters/normalize_timestamps.py,sha256=-oM05xtwuf8qjiAZ2jUJDqRLdom10H5KE9_QlPJOQUk,4663
+occystrap/filters/search.py,sha256=ftxTTsHS44L0BvxgFb09hFKC1t0znIj8zEDCK6-5lbo,6782
+occystrap/inputs/__init__.py,sha256=mbLK9EG2LauQqSJ-hqCyW0P2VCZNeG68mqT14FI8jmA,66
+occystrap/inputs/base.py,sha256=xLXp19fSO1Ks8eXETvOQkI3gUu42rI1OdhM_yuIJSts,1271
+occystrap/inputs/docker.py,sha256=QRGJ_LGLbuSaNN-HsgRembHrIAoP3JbyQwH7hF470vI,6420
+occystrap/inputs/registry.py,sha256=hQaqQ09sruuhkZl0KcZdCNIES2dZgJVWt8i20kIyDHk,10572
+occystrap/inputs/tarfile.py,sha256=7OHT4TYckI6zyAMwpUr-8B3Mg8T4M9qTVTHCJ_pFlDI,2898
+occystrap/outputs/__init__.py,sha256=E_-OfS5WtQi-yWMQTSviAlC8enO6IH88ZcOYCY_IjCw,38
+occystrap/outputs/base.py,sha256=TiucjT_XB_f8-QjWOFhhwSFyrS66mK6T8W8GYT1uKYI,1471
+occystrap/outputs/directory.py,sha256=MGZVIDqc3sNk5p3rd1wMgEO3u9_yvfLgE_y5_8Bu81w,11496
+occystrap/outputs/docker.py,sha256=AxCpYotlMnaBci5NObzKLEsHbXNYP3TuzPMffBxHQw0,4585
+occystrap/outputs/mounts.py,sha256=-rMNbetDW_rZO5gkLFtTYtuPTDhuzyhYg9CZXeuq_TQ,6433
+occystrap/outputs/ocibundle.py,sha256=tK09wZ7bCjeVnqoGsz6MF38Ma36hI9OofHEzG5HBtI4,2155
+occystrap/outputs/registry.py,sha256=qlhSaa4y7G_zC31MqItEtkWz3Ks83a_g4JlBj8RHleI,8633
+occystrap/outputs/tarfile.py,sha256=AHY6UvBo2WPG7nj5FHE7GRzouuxpFnQkwZEHz_KCLRY,2310
+occystrap/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+occystrap/tests/test_inspect.py,sha256=HA1ogCLyy5SCFWpesyVkhp6mL95k2XhGuCcAPQ5kMSc,11209
+occystrap/tests/test_tarformat.py,sha256=6q3tG7UtLpX274rjngvgQ8HCUk7lN17TMiXXjxdN2OA,7156
+occystrap-0.4.1.dist-info/licenses/AUTHORS,sha256=toKLUaf9c-NkNow00B_akwMGcGtm-S_ihcC_eql9qWc,34
+occystrap-0.4.1.dist-info/licenses/LICENSE,sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ,11357
+occystrap-0.4.1.dist-info/METADATA,sha256=HDKdBAasvpNlirXiQUt4qXOuQRBvaLSNl4DG-yovdRw,13194
+occystrap-0.4.1.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
+occystrap-0.4.1.dist-info/entry_points.txt,sha256=Uba7wHWknje_I2ZN1sGOpmP4twuQ1XEsOa5o28bvc2Y,49
+occystrap-0.4.1.dist-info/top_level.txt,sha256=06nN7FHq2z_Jpp2PZNm3rGOGUA1cIGlUr6MEZrqgOlc,10
+occystrap-0.4.1.dist-info/RECORD,,

{occystrap-0.3.0.dist-info → occystrap-0.4.1.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: bdist_wheel (0.34.2)
+Generator: setuptools (80.10.2)
 Root-Is-Purelib: true
 Tag: py3-none-any
 

{occystrap-0.3.0.dist-info → occystrap-0.4.1.dist-info}/entry_points.txt

@@ -1,3 +1,2 @@
 [console_scripts]
 occystrap = occystrap.main:cli
-

occystrap/docker_extract.py

@@ -1,36 +0,0 @@
-#!/usr/bin/python3
-
-# Call me like this:
-#  docker-image-extract tarfile.tar extracted
-
-import tarfile
-import json
-import os
-import sys
-
-image_path = sys.argv[1]
-extracted_path = sys.argv[2]
-
-with tarfile.open(image_path) as image:
-    manifest = json.loads(image.extractfile('manifest.json').read())
-    print('Manifest: %s' % manifest)
-
-    config = json.loads(image.extractfile(manifest[0]['Config']).read())
-    print('Config: %s' % config)
-
-    for layer in manifest[0]['Layers']:
-        print('Found layer: %s' % layer)
-        layer_tar = tarfile.open(fileobj=image.extractfile(layer))
-
-        for tarinfo in layer_tar:
-            print('  ... %s' % tarinfo.name)
-            if tarinfo.isdev():
-                print('  --> skip device files')
-                continue
-
-            dest = os.path.join(extracted_path, tarinfo.name)
-            if not tarinfo.isdir() and os.path.exists(dest):
-                print('  --> remove old version of file')
-                os.unlink(dest)
-
-            layer_tar.extract(tarinfo, path=extracted_path)