djevops 0.0.1__tar.gz

djevops-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michael Herrmann
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

djevops-0.0.1/PKG-INFO

@@ -0,0 +1,205 @@
+Metadata-Version: 2.4
+Name: djevops
+Version: 0.0.1
+Summary: Host Django without Docker
+Author: Michael Herrmann
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/mherrmann/djevops
+Project-URL: Repository, https://github.com/mherrmann/djevops
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: PyYAML==6.0.3
+Provides-Extra: test
+Requires-Dist: boto3; extra == "test"
+Requires-Dist: celery==5.5.3; extra == "test"
+Requires-Dist: django==5.1.2; extra == "test"
+Requires-Dist: dnsimple==2.15.0; extra == "test"
+Requires-Dist: hcloud==2.13.0; extra == "test"
+Requires-Dist: requests==2.32.5; extra == "test"
+Dynamic: license-file
+
+# djevops: Host Django on bare metal
+
+djevops is a command-line tool for deploying Django web apps to Linux VPSs.
+Unlike other tools, djevops runs Django "on bare metal". That is, without
+Docker. This makes development faster and easier.
+
+To get started with djevops, all you need is SSH root access to a Linux VPS
+running Ubuntu or Debian. Install djevops on your local machine with
+`pip install djevops`. Then, execute `djevops init` in your Django app's Git
+repository. You get a config file that looks similar to the following:
+
+```
+server: 1.2.3.4
+
+git:
+  repo: githubuser/reponame
+  branch: main
+
+services:
+  web:
+    type: django
+    env:
+      clear:
+        ALLOWED_HOSTS: your.website.com
+      secret:
+        - DJANGO_SECRET_KEY
+
+db:
+  type: sqlite
+
+mail:
+  host: smtp.gmail.com
+  user: SMTP_USER
+  password: SMTP_PASSWORD
+```
+
+Secrets such as `DJANGO_SECRET_KEY` or `SMTP_PASSWORD` can be specified as
+constants in file `djevops/secrets.py`.
+
+Most config values are optional. Fill in the ones you want and run
+`djevops deploy`. djevops then clones your Git repo on the `server` and starts
+all services. As you work on your Django app and push new commits to Git, simply
+run `djevops deploy` again to apply them to your server.
+
+## Features
+
+<details>
+<summary>Automatic SSL certificates</summary>
+
+djevops generates and automatically renews SSL certificates for any domains you
+specify in Django setting `ALLOWED_HOSTS`. The domains need to be tied to your
+server's IP address.
+</details>
+
+<details>
+<summary>Error emails</summary>
+
+If you filled in the `mail` section in the config file, then you can make Django
+email you when errors occur. To do so, set `ADMINS` in Django's `settings.py` as
+follows:
+
+```
+ADMINS = [('Your Name', 'your@email.com)]
+```
+
+Error emails require Django setting `DEBUG` to be `False`.
+</details>
+
+<details>
+<summary>Automatic database backups</summary>
+
+You can set up automatic database backups by adding a `backup` element to the
+`db` section in the djevops config file. For example:
+
+```
+db:
+  type: sqlite
+  backup:
+    type: s3
+    bucket: mybackup
+    access-key-id: S3_BACKUP_ACCESS_KEY
+    secret-access-key: S3_BACKUP_SECRET_KEY
+    path: db
+    region: us-east-1
+```
+
+Backups are created continuously while your server is running. If you ever
+re-install your server, then the latest backup is automatically restored.
+
+djevops uses [Litestream](https://litestream.io/) for SQLite backups. Litestream
+can store backups in S3, Azure Blob Storage and many others. The keys you add to
+the `backup` element above get copied into a `replica` element in Litestream's
+config. For more information about the available options, please
+see [Litestream's documentation](https://litestream.io/reference/config/).
+</details>
+
+<details>
+<summary>Background tasks via Celery and Redis</summary>
+
+If your Django app uses the `celery` Python package, then you can add a Celery
+worker by adding the following item to the djevops config:
+
+```
+services:
+  web:
+    # as before
+  celery:
+    type: celery
+    env:
+      inherit: web
+```
+
+To install Redis on the server (which many Django apps use as Celery's backend),
+add an empty `redis` block:
+
+```
+redis:
+```
+
+This setup lets you run Python functions asynchronously and on a schedule such
+as "every five hours". The service of type `celery` also runs the necessary
+`beat` scheduler.
+</details>
+
+<details>
+<summary>Easy access to log files</summary>
+
+djevops writes the log file for each service to `/var/log/<service>.log`. To
+read it, simply SSH into the server and do `less`, `tail -f`, etc. To prevent
+log files from filling up your server's disk space, djevops also rotates and
+compresses log files.
+</details>
+
+<details>
+<summary>Secret handling</summary>
+
+Very often, you have secrets that you need on the server but should not commit
+to Git. djevops lets you specify such values in the file `djevops/secrets.py`,
+and refer to them from your config file. The way this works is that `secrets.py`
+gets executed on your local machine, and the produced values then get uploaded
+as constants to the server. This gives you a lot of flexibility. You can
+hardcode values in `secrets.py` and not commit that file to Git. Or you can for
+example make `secrets.py` read from environment variables that are available
+when you do `djevops deploy`:
+
+```
+import os
+MY_SECRET = os.environ['MY_SECRET']
+```
+
+You can also invoke password managers in `secrets.py`, etc.
+</details>
+
+<details>
+<summary>Secure defaults</summary>
+
+djevops uses secure defaults whenever possible. For example, each `service` runs
+as a separate user. This means that environment variables cannot leak from one
+service to another. djevops also makes sure that no unintended ports are open,
+such as for example port 25 when using Postfix for sending emails.
+</details>
+
+<details>
+<summary>Automatic OS updates</summary>
+
+djevops sets up automatic OS updates to keep your server up-to-date and secure.
+This does not apply major version upgrades, which could introduce potentially
+breaking changes.
+</details>
+
+## Development
+
+Install the `test` dependencies from
+[`djevops/pyproject.toml`](djevops/pyproject.toml). The easiest way I know for
+doing this is with [`uv`](https://docs.astral.sh/uv/):
+
+```
+uv venv
+source .venv/bin/activate
+uv sync --no-install-project --extra test
+```
+
+Then, you can do `python -m unittest` to run tests. This requires some API keys
+and environment variables.

djevops-0.0.1/README.md

@@ -0,0 +1,184 @@
+# djevops: Host Django on bare metal
+
+djevops is a command-line tool for deploying Django web apps to Linux VPSs.
+Unlike other tools, djevops runs Django "on bare metal". That is, without
+Docker. This makes development faster and easier.
+
+To get started with djevops, all you need is SSH root access to a Linux VPS
+running Ubuntu or Debian. Install djevops on your local machine with
+`pip install djevops`. Then, execute `djevops init` in your Django app's Git
+repository. You get a config file that looks similar to the following:
+
+```
+server: 1.2.3.4
+
+git:
+  repo: githubuser/reponame
+  branch: main
+
+services:
+  web:
+    type: django
+    env:
+      clear:
+        ALLOWED_HOSTS: your.website.com
+      secret:
+        - DJANGO_SECRET_KEY
+
+db:
+  type: sqlite
+
+mail:
+  host: smtp.gmail.com
+  user: SMTP_USER
+  password: SMTP_PASSWORD
+```
+
+Secrets such as `DJANGO_SECRET_KEY` or `SMTP_PASSWORD` can be specified as
+constants in file `djevops/secrets.py`.
+
+Most config values are optional. Fill in the ones you want and run
+`djevops deploy`. djevops then clones your Git repo on the `server` and starts
+all services. As you work on your Django app and push new commits to Git, simply
+run `djevops deploy` again to apply them to your server.
+
+## Features
+
+<details>
+<summary>Automatic SSL certificates</summary>
+
+djevops generates and automatically renews SSL certificates for any domains you
+specify in Django setting `ALLOWED_HOSTS`. The domains need to be tied to your
+server's IP address.
+</details>
+
+<details>
+<summary>Error emails</summary>
+
+If you filled in the `mail` section in the config file, then you can make Django
+email you when errors occur. To do so, set `ADMINS` in Django's `settings.py` as
+follows:
+
+```
+ADMINS = [('Your Name', 'your@email.com)]
+```
+
+Error emails require Django setting `DEBUG` to be `False`.
+</details>
+
+<details>
+<summary>Automatic database backups</summary>
+
+You can set up automatic database backups by adding a `backup` element to the
+`db` section in the djevops config file. For example:
+
+```
+db:
+  type: sqlite
+  backup:
+    type: s3
+    bucket: mybackup
+    access-key-id: S3_BACKUP_ACCESS_KEY
+    secret-access-key: S3_BACKUP_SECRET_KEY
+    path: db
+    region: us-east-1
+```
+
+Backups are created continuously while your server is running. If you ever
+re-install your server, then the latest backup is automatically restored.
+
+djevops uses [Litestream](https://litestream.io/) for SQLite backups. Litestream
+can store backups in S3, Azure Blob Storage and many others. The keys you add to
+the `backup` element above get copied into a `replica` element in Litestream's
+config. For more information about the available options, please
+see [Litestream's documentation](https://litestream.io/reference/config/).
+</details>
+
+<details>
+<summary>Background tasks via Celery and Redis</summary>
+
+If your Django app uses the `celery` Python package, then you can add a Celery
+worker by adding the following item to the djevops config:
+
+```
+services:
+  web:
+    # as before
+  celery:
+    type: celery
+    env:
+      inherit: web
+```
+
+To install Redis on the server (which many Django apps use as Celery's backend),
+add an empty `redis` block:
+
+```
+redis:
+```
+
+This setup lets you run Python functions asynchronously and on a schedule such
+as "every five hours". The service of type `celery` also runs the necessary
+`beat` scheduler.
+</details>
+
+<details>
+<summary>Easy access to log files</summary>
+
+djevops writes the log file for each service to `/var/log/<service>.log`. To
+read it, simply SSH into the server and do `less`, `tail -f`, etc. To prevent
+log files from filling up your server's disk space, djevops also rotates and
+compresses log files.
+</details>
+
+<details>
+<summary>Secret handling</summary>
+
+Very often, you have secrets that you need on the server but should not commit
+to Git. djevops lets you specify such values in the file `djevops/secrets.py`,
+and refer to them from your config file. The way this works is that `secrets.py`
+gets executed on your local machine, and the produced values then get uploaded
+as constants to the server. This gives you a lot of flexibility. You can
+hardcode values in `secrets.py` and not commit that file to Git. Or you can for
+example make `secrets.py` read from environment variables that are available
+when you do `djevops deploy`:
+
+```
+import os
+MY_SECRET = os.environ['MY_SECRET']
+```
+
+You can also invoke password managers in `secrets.py`, etc.
+</details>
+
+<details>
+<summary>Secure defaults</summary>
+
+djevops uses secure defaults whenever possible. For example, each `service` runs
+as a separate user. This means that environment variables cannot leak from one
+service to another. djevops also makes sure that no unintended ports are open,
+such as for example port 25 when using Postfix for sending emails.
+</details>
+
+<details>
+<summary>Automatic OS updates</summary>
+
+djevops sets up automatic OS updates to keep your server up-to-date and secure.
+This does not apply major version upgrades, which could introduce potentially
+breaking changes.
+</details>
+
+## Development
+
+Install the `test` dependencies from
+[`djevops/pyproject.toml`](djevops/pyproject.toml). The easiest way I know for
+doing this is with [`uv`](https://docs.astral.sh/uv/):
+
+```
+uv venv
+source .venv/bin/activate
+uv sync --no-install-project --extra test
+```
+
+Then, you can do `python -m unittest` to run tests. This requires some API keys
+and environment variables.

djevops-0.0.1/djevops/__init__.py

@@ -0,0 +1 @@
+GIT_HINT = "Don't forget to commit *and push* your changes to Git."

djevops-0.0.1/djevops/__main__.py

@@ -0,0 +1,247 @@
+from djevops import GIT_HINT
+from djevops.config import get_services_users_envs, get_django_service
+from djevops.util import git, get_apt_install_cmd, run_in_django_shell, \
+    run_silently
+from functools import partial
+from os import remove, makedirs
+from os.path import dirname, exists
+from runpy import run_path
+from shlex import quote
+from subprocess import run
+from tempfile import NamedTemporaryFile
+from urllib.parse import urlparse
+
+import json
+import os
+import re
+import sys
+import yaml
+
+SAMPLE_SERVER_IP = '0.0.0.0'
+
+SAMPLE_SECRETS_PY = """
+# This file lets you store secrets that you can then refer to from your djevops
+# config. For example:
+#
+# DJANGO_SECRET_KEY = '1234...'
+#
+# The motivation for keeping secrets in a separate file is that you usually do
+# not want to commit them to Git. One approach you can use is to hard-code your
+# secrets here and only store djevops' deploy.yml file in Git.
+#
+# Another approach is to read secrets from environment variables. For example:
+#
+# import os
+# MY_SECRET = os.environ['MY_SECRET']
+#
+# This works if the environment variables are available when you execute
+# `djevops deploy`. The produced values are uploaded as constants to your
+# server. If all your secrets are read from environment variables in this way,
+# then you can consider committing this file to Git.
+#
+# (Feel free to remove these comments once you are done reading them.)
+""".lstrip()
+
+SECRETS_NAME_RE = r'^[A-Z][A-Z0-9_]+$'
+
+class CommandError(Exception):
+    pass
+
+def init(quiet=False):
+    if not exists('.git'):
+        raise CommandError('This directory is not a Git repository.')
+    remotes = git('remote').splitlines()
+    if not remotes:
+        raise CommandError(
+            "This Git repository has no remotes. If you add one, don't forget "
+            "to run `git push` after."
+        )
+    if not exists('manage.py'):
+        raise CommandError(
+            "There is no manage.py file in the current directory. If you add "
+            "one, don't forget to commit *and push* your changes to Git."
+        )
+    if not exists('requirements.txt'):
+        raise CommandError(
+            'Please create a requirements.txt file. For example, by running:\n'
+            '    pip freeze > requirements.txt\n' + GIT_HINT
+        )
+    with open('requirements.txt') as f:
+        requirements = f.read()
+    for dep in ('django', 'gunicorn'):
+        if not re.search(
+            rf'^{dep}\s*\b', requirements, re.MULTILINE | re.IGNORECASE
+        ):
+            raise CommandError(
+                f'Please add `{dep}` to your requirements.txt file. ' + GIT_HINT
+            )
+    remote = remotes[0]
+    remote_url = git('remote', 'get-url', remote)
+    if remote_url.startswith('https://'):
+        parsed_url = urlparse(remote_url)
+        git_server = parsed_url.hostname
+        git_repo = parsed_url.path.lstrip('/')
+    else:
+        m = re.match(r'git@([^:]+):(.*)$', remote_url)
+        if not m:
+            raise CommandError(f'Invalid Git remote URL: {remote_url}')
+        git_server = m.group(1)
+        git_repo = m.group(2)
+    if git_server == 'github.com' and git_repo.endswith('.git'):
+        git_repo = git_repo[:-4]
+    git_config = {}
+    if git_server != 'github.com':
+        git_config['server'] = git_server
+    git_config['repo'] = git_repo
+    git_config['branch'] = git('branch', '--show-current')
+    deploy_yml = {
+        'server': SAMPLE_SERVER_IP,
+        'git': git_config,
+        'services': {
+            'web': {
+                'type': 'django'
+            }
+        }
+    }
+    for path in ('djevops/deploy.yml', 'djevops/secrets.py'):
+        if exists(path):
+            raise CommandError(f'{path} already exists.')
+    makedirs('djevops', exist_ok=True)
+    with open('djevops/deploy.yml', 'w') as f:
+        f.write(yaml.dump(deploy_yml, sort_keys=False))
+    with open('djevops/secrets.py', 'w') as f:
+        f.write(SAMPLE_SECRETS_PY)
+    if not quiet:
+        print('Created djevops/deploy.yml')
+        print('Created djevops/secrets.py')
+        print(f'To deploy your Django app to a server, run: djevops deploy')
+
+def deploy(quiet=False, dry_run=False):
+    deploy_yml = 'djevops/deploy.yml'
+    with open(deploy_yml) as f:
+        deploy_config = yaml.safe_load(f)
+    secrets = get_secrets('djevops/secrets.py')
+
+    check_config(deploy_config, secrets)
+
+    if dry_run:
+        return
+
+    server = deploy_config['server']
+    install_djevops_on_server('root', server, quiet)
+    rsync('-a', deploy_yml, f'root@{server}:/root/deploy.yml')
+
+    secrets_json = NamedTemporaryFile(mode='w', delete=False, suffix='.json')
+    json.dump(secrets, secrets_json, indent=2, sort_keys=True)
+    secrets_json.close()
+    try:
+        rsync('-a', secrets_json.name, f'root@{server}:/root/secrets.json')
+    finally:
+        remove(secrets_json.name)
+
+    run_with_djevops_venv(
+        'root', server, 'python -u -m djevops.remote.deploy', quiet
+    )
+
+def check_config(deploy_config, secrets):
+    server_ip = deploy_config.get('server')
+    if not server_ip or server_ip == SAMPLE_SERVER_IP:
+        raise CommandError(
+            "Please set your server's IP address in djevops/deploy.yml. For "
+            "example:\n"
+            "    server: 1.2.3.4"
+        )
+
+    try:
+        django_service_name, django_service = get_django_service(deploy_config)
+    except LookupError:
+        raise CommandError(
+            'Please add at least one service of type `django` to '
+            'djevops/deploy.yml. For example:\n'
+            '    services:\n'
+            '      web:\n'
+            '        type: django'
+        )
+
+    user_envs = get_services_users_envs(deploy_config, secrets)
+    django_env = user_envs[django_service_name][1]
+
+    has_db = bool(deploy_config.get('db'))
+
+    # Ensure `djevops.check_django_settings` is loadable:
+    django_shell_env = django_env | {'PYTHONPATH': ':'.join(sys.path)}
+    error_msg = run_in_django_shell([
+        'from djevops.check_django_settings import main',
+        f'main({server_ip!r}, {has_db})',
+    ], env=django_shell_env)
+    if error_msg:
+        raise CommandError(error_msg)
+
+def install_djevops_on_server(user, host, quiet):
+    ssh_ = lambda cmd: ssh(user, host, cmd, quiet)
+    ssh_(get_apt_install_cmd('rsync'))
+    ssh_(
+        'command -v uv >/dev/null 2>&1 || '
+        'curl -LsSf https://astral.sh/uv/install.sh | sh >/dev/null 2>&1'
+    )
+    rsync(
+        '-raL',
+        dirname(__file__) + '/',
+        f'{user}@{host}:/opt/djevops/',
+        "--include=**.gitignore",
+        "--filter=:- .gitignore",
+        "--delete-after"
+    )
+    ssh_(
+        'ln -sf /opt/djevops/pyproject.toml /opt/pyproject.toml && '
+        'ln -sf /opt/djevops/uv.lock /opt/uv.lock && '
+        'cd /opt && '
+        'UV_PROJECT_ENVIRONMENT=/opt/djevops/.venv ~/.local/bin/uv sync -q && '
+        'rm /opt/pyproject.toml /opt/uv.lock'
+    )
+
+def get_secrets(path):
+    if not exists(path):
+        return {}
+    return {
+        k: v for k, v in run_path(path).items() if re.match(SECRETS_NAME_RE, k)
+    }
+
+def run_with_djevops_venv(user, host, cmd, quiet):
+    ssh(user, host, f'/opt/djevops/.venv/bin/{cmd}', quiet)
+
+def rsync(*args):
+    ssh_cmd = get_ssh_command()
+    extra_rsync_args = [] if ssh_cmd == 'ssh' else ['-e', ssh_cmd]
+    run_silently(['rsync', *extra_rsync_args, *args])
+
+def ssh(user, host, cmd, quiet):
+    ssh_cmd = get_ssh_command()
+    run_ = run_silently if quiet else partial(run, check=True)
+    run_(f'{ssh_cmd} {user}@{host} {quote(cmd)}', shell=True)
+
+def get_ssh_command():
+    try:
+        return os.environ['DJEVOPS_SSH_COMMAND']
+    except KeyError:
+        return 'ssh'
+
+def main():
+    if len(sys.argv) < 2 or len(sys.argv) > 3:
+        print('Usage: djevops init|deploy [--quiet]')
+        sys.exit(0)
+    command = sys.argv[1]
+    quiet = len(sys.argv) == 3 and sys.argv[2] == '--quiet'
+    try:
+        if command == 'init':
+            init(quiet)
+        elif command == 'deploy':
+            deploy(quiet)
+        else:
+            raise CommandError(f'Unknown command: {command}')
+    except CommandError as e:
+        sys.stderr.write(e.args[0] + '\n')
+        sys.exit(1)
+
+if __name__ == '__main__':
+    main()

djevops-0.0.1/djevops/bin/celery.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+set -eo pipefail
+
+SERVICE=$1
+
+DJANGO_PROJECT=$(find . -name wsgi.py | head -1 | xargs dirname | sed 's|^\./||')
+
+exec celery -A $DJANGO_PROJECT worker -B \
+  -s /var/run/djevops/$SERVICE-schedule \
+  --pidfile /var/run/djevops/$SERVICE.pid \
+  -l info