octopize.deploy_tool 2.7.1__tar.gz → 2.7.3__tar.gz

octopize_deploy_tool-2.7.3/PKG-INFO

@@ -0,0 +1,620 @@
+Metadata-Version: 2.4
+Name: octopize.deploy_tool
+Version: 2.7.3
+Summary: Deployment configuration tool for Octopize Avatar platform
+Project-URL: Homepage, https://octopize.io
+Project-URL: Documentation, https://docs.octopize.io/docs/deploying/self-hosted
+Project-URL: Repository, https://github.com/octopize/avatar
+Project-URL: Issues, https://github.com/octopize/avatar/issues
+Author-email: Octopize <contact@octopize.io>
+License: MIT
+Keywords: avatar,configuration,deployment,octopize
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: System :: Installation/Setup
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.13
+Requires-Dist: httpx>=0.28.0
+Requires-Dist: jinja2>=3.1.0
+Requires-Dist: pandas>=3.0.0
+Requires-Dist: python-on-whales>=0.76.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich-argparse>=1.8.0
+Requires-Dist: rich>=13.0.0
+Description-Content-Type: text/markdown
+
+# Octopize Avatar Deployment Tool
+
+This package provides the `octopize-deploy-tool` CLI for operating a self-hosted Octopize Avatar
+deployment. It bundles deployment templates for each release so it works without fetching files
+from GitHub.
+
+Available commands:
+
+- **`install`** — first-time setup: collects configuration interactively and generates all
+  deployment files under a versioned app root directory.
+- **`update`** — reconfigure an existing deployment: re-runs the configuration wizard seeded from
+  the current state and regenerates the deployment files.
+- **`migrate`** — migrates an existing flat deployment (from the legacy layout) to the new
+  versioned directory structure.
+- **`start`** — creates required Docker volumes and starts the deployment stack with
+  `docker compose up`.
+- **`stop`** — suspends all running services with `docker compose stop`, preserving
+  containers and volumes for a fast restart.
+- **`generate-env`** — generates per-component `.env` files for local development.
+- **`authentik-migrate`** — migrates existing user data from CSV exports into
+  [authentik](https://goauthentik.io/), the identity provider bundled with Avatar.
+
+## Installation
+
+### Option 1 — uvx (no install required, recommended for one-shot use)
+
+[uv](https://docs.astral.sh/uv/) can run the tool directly from PyPI without a permanent install.
+If you don't have `uv` yet:
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+Then run any command directly:
+
+```bash
+uvx octopize-deploy-tool install --app-root ./app/avatar
+```
+
+### Option 2 — virtual environment
+
+Modern Linux distributions and macOS prevent installing packages into the system Python
+(PEP 668 — "externally managed environment"). A virtual environment avoids that:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+pip install octopize-deploy-tool
+octopize-deploy-tool install --app-root ./app/avatar
+```
+
+### Option 3 — global pip
+
+If your system Python allows global installs:
+
+```bash
+pip install octopize-deploy-tool
+```
+
+Verify the CLI is available:
+
+```bash
+octopize-deploy-tool --help
+```
+
+### Option 4 — Docker (if PyPI is not accessible)
+
+If PyPI is blocked in your environment (air-gapped networks, corporate proxies, or registries
+that allow [quay.io](https://quay.io/octopize) but not PyPI), a pre-built Docker image is
+available:
+
+```bash
+docker pull quay.io/octopize/deploy-tool:latest
+```
+
+See the [Docker](#docker) section below for full usage examples for each command.
+
+## Quick Start
+
+Set up a new deployment in two steps:
+
+1. Run the configuration wizard to generate all deployment files:
+
+   ```bash
+   octopize-deploy-tool install --app-root ./app/avatar
+   ```
+
+2. Start the deployment:
+
+   ```bash
+   octopize-deploy-tool start --app-root ./app/avatar
+   ```
+
+## Commands: When to Use What
+
+| Command   | Use when                                                                         |
+| --------- | -------------------------------------------------------------------------------- |
+| `install` | Setting up a new Avatar deployment for the first time                            |
+| `update`  | Upgrading an existing deployment to a newer version of the Avatar application    |
+| `migrate` | Moving a deployment set up with the legacy layout to the new versioned structure |
+| `start`   | Bringing a configured deployment live, or restarting after a host reboot         |
+| `stop`    | Suspending the running stack temporarily (preserves data; fast to restart)       |
+
+### `install`
+
+Use `install` when you are setting up Avatar for the first time. It runs the full configuration
+wizard, collects all required values interactively (or from a seed config file), generates all
+deployment files under a versioned directory tree, and leaves the deployment ready to start.
+
+```bash
+octopize-deploy-tool install --app-root ./app/avatar
+```
+
+To pre-populate answers from a YAML config file:
+
+```bash
+octopize-deploy-tool install \
+  --app-root ./app/avatar \
+  --from-config config.yaml \
+  --non-interactive
+```
+
+Common `install` options:
+
+```text
+--app-root PATH        Root directory for the deployment (default: current directory)
+--from-config PATH     YAML seed config to pre-populate answers
+--non-interactive      Run without prompts; fails fast if any required value is missing
+--verbose              Show detailed progress output
+```
+
+### `update`
+
+Use `update` when you are upgrading to a newer version of the Avatar application. It carries your
+existing configuration forward unchanged, re-renders the deployment files using the new templates
+bundled in the updated deploy tool, diffs the result against the currently running files, and
+applies the changes.
+
+```bash
+octopize-deploy-tool update --app-root ./app/avatar
+```
+
+After `update`, run `start` to apply the new files to the running stack:
+
+```bash
+octopize-deploy-tool start --app-root ./app/avatar
+```
+
+> **Need to change a configuration value** (URL, TLS certificate, email settings, …)?  
+> `update` is not the right tool for that — it carries existing values forward without
+> re-asking questions. To go through the configuration wizard again, use:
+>
+> ```bash
+> octopize-deploy-tool configure --fresh --app-root ./app/avatar
+> ```
+
+Common `update` options:
+
+```text
+--app-root PATH        Root directory for the deployment (default: current directory)
+--non-interactive      Run without prompts
+--verbose              Show detailed progress output
+```
+
+### `migrate`
+
+Use `migrate` when you have a deployment that was generated by the legacy flat layout (a directory
+containing `.env` and `.secrets/`) and want to bring it under the new versioned directory
+structure. `migrate` reads the existing `.env` and `.secrets/` files to pre-seed all configuration
+values so you do not have to re-enter them.
+
+```bash
+octopize-deploy-tool migrate \
+  --app-root ./app/avatar-new \
+  --from-legacy ./app/avatar-old
+```
+
+After `migrate`, run `start` to bring the migrated deployment live:
+
+```bash
+octopize-deploy-tool start --app-root ./app/avatar-new
+```
+
+Common `migrate` options:
+
+```text
+--app-root PATH        Root directory for the new versioned deployment (default: current directory)
+--from-legacy DIR      Path to the existing flat deployment directory (required)
+--non-interactive      Run without prompts
+--verbose              Show detailed progress output
+```
+
+### `start`
+
+Use `start` to create the required Docker volumes and bring the deployment stack live. Run it after
+`install`, `migrate`, or `update`. `start` is idempotent — if a volume already exists it is left
+unchanged, making it safe to re-run after a host reboot.
+
+```bash
+octopize-deploy-tool start --app-root ./app/avatar
+```
+
+Common `start` options:
+
+```text
+--app-root PATH        Root directory for the deployment (default: current directory)
+--verbose              Show detailed progress output
+```
+
+### `stop`
+
+Use `stop` to suspend the running stack without removing containers or volumes. This is the
+complement of `start` — services can be brought back up quickly with `start` without losing
+any data.
+
+```bash
+octopize-deploy-tool stop --app-root ./app/avatar
+```
+
+Common `stop` options:
+
+```text
+--app-root PATH        Root directory for the deployment (default: current directory)
+--verbose              Show detailed progress output
+```
+
+## Docker
+
+If you don't have Python installed, you can use the published Docker image instead:
+
+```
+quay.io/octopize/deploy-tool:latest
+```
+
+> **Note on file ownership:** For commands that write files to a bind-mounted directory, run the
+> container with `--user "$(id -u):$(id -g)"` so generated files are owned by your host user.
+
+### `install`
+
+**Interactive:**
+
+```bash
+docker run -it --rm \
+  --user "$(id -u):$(id -g)" \
+  -v "$(pwd)/avatar:/app-root" \
+  quay.io/octopize/deploy-tool:latest install --app-root /app-root
+```
+
+**Non-interactive, seed config inside the app-root:**
+
+```bash
+docker run --rm \
+  --user "$(id -u):$(id -g)" \
+  -v "$(pwd)/avatar:/app-root" \
+  quay.io/octopize/deploy-tool:latest install \
+    --app-root /app-root \
+    --from-config /app-root/config.yaml \
+    --non-interactive
+```
+
+**Non-interactive, seed config stored outside the app-root:**
+
+Mount the directory containing the config file as a separate read-only volume:
+
+```bash
+docker run --rm \
+  --user "$(id -u):$(id -g)" \
+  -v "$(pwd)/avatar:/app-root" \
+  -v "$(pwd)/config:/input:ro" \
+  quay.io/octopize/deploy-tool:latest install \
+    --app-root /app-root \
+    --from-config /input/config.yaml \
+    --non-interactive
+```
+
+> **Important — enter host paths during configuration:** When the wizard asks for filesystem
+> paths — such as the path to your TLS certificate or private key — enter the path where that
+> file exists on your **host machine**, not the path inside this container.
+>
+> For example, if your TLS certificate lives at `/etc/ssl/avatar/server.crt` on the host,
+> enter `/etc/ssl/avatar/server.crt` at the prompt. Docker Compose will mount those paths from
+> the host when it starts the Avatar services.
+
+### `update`
+
+Mount the existing app-root. If your `--from-config` override file lives outside the app-root,
+add a second read-only mount (same pattern as `install`):
+
+```bash
+docker run -it --rm \
+  --user "$(id -u):$(id -g)" \
+  -v "$(pwd)/avatar:/app-root" \
+  quay.io/octopize/deploy-tool:latest update --app-root /app-root
+```
+
+### `migrate`
+
+Mount both the new app-root **and** the legacy deployment directory as a separate read-only volume.
+The `--from-legacy` path must point to its container-side location:
+
+```bash
+docker run -it --rm \
+  --user "$(id -u):$(id -g)" \
+  -v "$(pwd)/avatar-new:/app-root" \
+  -v "$(pwd)/avatar-old:/legacy:ro" \
+  quay.io/octopize/deploy-tool:latest migrate \
+    --app-root /app-root \
+    --from-legacy /legacy
+```
+
+### `start`
+
+`start` calls Docker directly (to create volumes and run `docker compose up`), so the host Docker
+socket must be mounted:
+
+```bash
+docker run --rm \
+  -v "$(pwd)/avatar:/app-root" \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  quay.io/octopize/deploy-tool:latest start --app-root /app-root
+```
+
+> **Note:** `--user` is omitted here because the process needs access to the Docker socket. If you
+> see a permission error, add `--group-add $(stat -c '%g' /var/run/docker.sock)` to grant the
+> container access to the socket's group on the host.
+
+### `stop`
+
+Like `start`, `stop` calls Docker directly and requires the host Docker socket:
+
+```bash
+docker run --rm \
+  -v "$(pwd)/avatar:/app-root" \
+  -v /var/run/docker.sock:/var/run/docker.sock \
+  quay.io/octopize/deploy-tool:latest stop --app-root /app-root
+```
+
+## Non-Interactive Usage
+
+Provide a YAML configuration file to skip interactive prompts. Any value not covered by the
+config file will use the built-in default.
+
+Example seed config:
+
+```yaml
+PUBLIC_URL: avatar.example.com
+ENV_NAME: prod
+ORGANIZATION_NAME: MyCompany
+```
+
+Then run:
+
+```bash
+octopize-deploy-tool install \
+  --app-root ./app/avatar \
+  --from-config config.yaml \
+  --non-interactive
+```
+
+## Generated Files
+
+The `install` command writes deployment files under a versioned directory tree:
+
+```text
+./app/avatar/
+├── .deployment_states/
+│   └── v1/
+│       ├── deployment_state.yaml
+│       └── generated/             # reference copy of rendered files (read-only)
+├── customer_inputs/               # operator-owned configuration overrides
+└── generated_files/               # what Docker Compose actually uses
+    ├── .env
+    ├── docker-compose.yml
+    ├── nginx/nginx.conf
+    ├── authentik/
+    │   ├── octopize-avatar-blueprint.yaml
+    │   ├── custom-templates/
+    │   └── branding/
+    └── .secrets/
+```
+
+Each subsequent `update` or `migrate` run adds a new versioned slot (e.g. `v2/`), keeping a full
+history of prior states.
+
+The `generate-env` command writes component `.env` files directly to their resolved destinations:
+
+```text
+./avatar-local/
+├── api/.env
+└── web/.env
+```
+
+## Typical Deployment Workflow
+
+1. Generate configuration:
+
+   ```bash
+   octopize-deploy-tool install --app-root ./app/avatar
+   ```
+
+2. Review the generated files:
+
+   ```bash
+   ls ./app/avatar/generated_files/
+   cat ./app/avatar/generated_files/.env
+   ls -la ./app/avatar/generated_files/.secrets/
+   ```
+
+3. Add any required TLS certificates for production.
+
+4. Start the services:
+
+   ```bash
+   octopize-deploy-tool start --app-root ./app/avatar
+   ```
+
+5. Verify the deployment:
+
+   ```bash
+   docker compose -f ./app/avatar/generated_files/docker-compose.yml ps
+   docker compose -f ./app/avatar/generated_files/docker-compose.yml logs -f
+   ```
+
+## `generate-env`
+
+Use `generate-env` to create per-component `.env` files for local development without generating
+the full deployment bundle.
+
+Example:
+
+```bash
+octopize-deploy-tool generate-env \
+  --component api \
+  --api-output-path ./avatar-local/api/.env \
+  --component web \
+  --web-output-path ./avatar-local/web/.env
+```
+
+Common `generate-env` options:
+
+```text
+--config FILE              YAML configuration file to load
+--non-interactive          Run without prompts, using config/defaults
+--verbose                  Show detailed progress output
+--component NAME           Generate only the selected component (repeatable; defaults to all)
+--api-output-path PATH     Override the API env output path for this run
+--web-output-path PATH     Override the web env output path for this run
+--python-client-output-path PATH
+                           Override the python_client env output path for this run
+--output-path COMPONENT=PATH
+                           Repeatable generic output-path override
+--target NAME              Load named URLs from the environments config section
+--api-url URL              Override the API URL
+--storage-url URL          Override the storage public URL
+--sso-url URL              Override the SSO provider URL
+```
+
+## Troubleshooting
+
+### Services fail to start after `start`
+
+Check that Docker is running and the generated files are present:
+
+```bash
+ls ./app/avatar/generated_files/
+docker compose -f ./app/avatar/generated_files/docker-compose.yml ps
+docker compose -f ./app/avatar/generated_files/docker-compose.yml logs -f
+```
+
+### Existing containers cause bind-mount or startup issues
+
+Stop and remove old containers and volumes, then restart:
+
+```bash
+docker compose -f ./app/avatar/generated_files/docker-compose.yml down --volumes --remove-orphans
+octopize-deploy-tool start --app-root ./app/avatar
+```
+
+### Verbose output for debugging
+
+Add `--verbose` to any command to see detailed progress and template rendering output:
+
+```bash
+octopize-deploy-tool install --app-root ./app/avatar --verbose
+```
+
+## Migrating Users to authentik
+
+When upgrading from an older Avatar deployment that managed users directly in the Avatar API
+database, use `octopize-deploy-tool authentik-migrate` to import those users into authentik.
+
+### Overview
+
+The tool reads three CSV exports from the Avatar database and creates the equivalent users, groups,
+and group assignments in authentik via its REST API:
+
+| Source CSV          | authentik entity             |
+| ------------------- | ---------------------------- |
+| `organizations.csv` | Groups (one per org)         |
+| `licenses.csv`      | Group `attributes.license`   |
+| `users.csv`         | Users with `attributes.role` |
+
+Users that already have an `authentik_id` value in the CSV are automatically skipped, making the
+tool safe to run incrementally.
+
+### Extracting CSV Data
+
+Exec into the API container and export the required tables:
+
+```bash
+docker compose exec -it api bash
+cd /app/avatar
+python bin/dbtool.py shell
+```
+
+Then in the pgcli shell:
+
+```sql
+\copy (SELECT * FROM licenses) TO '/tmp/licenses.csv' WITH CSV HEADER
+\copy (SELECT * FROM users) TO '/tmp/users.csv' WITH CSV HEADER
+\copy (SELECT * FROM organizations) TO '/tmp/organizations.csv' WITH CSV HEADER
+```
+
+### Usage
+
+**Dry run** — preview all operations without making any changes:
+
+```bash
+octopize-deploy-tool authentik-migrate \
+  --users /tmp/users.csv \
+  --orgs /tmp/organizations.csv \
+  --licenses /tmp/licenses.csv \
+  --dry-run
+```
+
+**Full migration** — execute against a live authentik instance:
+
+```bash
+octopize-deploy-tool authentik-migrate \
+  --users /tmp/users.csv \
+  --orgs /tmp/organizations.csv \
+  --licenses /tmp/licenses.csv \
+  --authentik-url https://avatar.example.com/sso \
+  --authentik-token YOUR_API_TOKEN
+```
+
+The authentik URL is the `/sso` path of your Avatar instance. The API token is generated
+automatically by the deploy tool and stored in the `.env` file.
+
+Every run writes a JSONL log (`migration_log.jsonl` by default). If some operations fail, retry
+just the failed ones:
+
+```bash
+octopize-deploy-tool authentik-migrate \
+  --from-log migration_log.jsonl \
+  --failed-only \
+  --authentik-url https://avatar.example.com/sso \
+  --authentik-token YOUR_API_TOKEN
+```
+
+### Getting an authentik API Token
+
+1. Log in to your authentik instance
+2. Navigate to **Admin** → **Tokens & App passwords**
+3. Create a token with the following permissions:
+   - `core:groups:create`
+   - `core:users:create`
+   - `core:groups:update` (for user assignments)
+
+### Migration CLI Reference
+
+```text
+octopize-deploy-tool authentik-migrate [options]
+
+  --users PATH           Path to users CSV file
+  --orgs PATH            Path to organizations CSV file
+  --licenses PATH        Path to licenses CSV file
+  --authentik-url URL    Base URL of the authentik instance
+                         (e.g. https://avatar.example.com/sso)
+  --authentik-token TOK  API token for authentik
+  --dry-run              Preview operations without executing
+  --log PATH             JSONL log output path (default: migration_log.jsonl)
+  --from-log PATH        Replay operations from a previous log instead of CSV
+  --failed-only          With --from-log, only replay previously-failed ops
+```
+
+## Further Information
+
+- Source repository: <https://github.com/octopize/avatar>
+- Deployment documentation: <https://docs.octopize.io/docs/deploying/self-hosted>
+- Network topology reference: [docs/network-topology.md](docs/network-topology.md)