sg-event-daemon 0.1.0__tar.gz

sg_event_daemon-0.1.0/.gitignore

@@ -0,0 +1,46 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+dist/
+.eggs/
+*.egg-info/
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage*
+.pytest_cache/
+.cache/
+
+# Type checker / linter caches
+.pyright/
+.mypy_cache/
+.ruff_cache/
+pyrightconfig.json
+
+# Editors
+.vscode/
+.idea/
+
+# OS
+.DS_Store
+Thumbs.db

sg_event_daemon-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 the sg-event-daemon authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sg_event_daemon-0.1.0/PKG-INFO

@@ -0,0 +1,224 @@
+Metadata-Version: 2.5
+Name: sg-event-daemon
+Version: 0.1.0
+Summary: Event daemon for Autodesk Flow Production Tracking
+Project-URL: Documentation, https://github.com/dactylo/sg-event-daemon/tree/main/docs
+Project-URL: Homepage, https://github.com/dactylo/sg-event-daemon
+Project-URL: Issues, https://github.com/dactylo/sg-event-daemon/issues
+Project-URL: Repository, https://github.com/dactylo/sg-event-daemon
+License-Expression: MIT
+License-File: LICENSE
+Keywords: daemon,events,flow-production-tracking,shotgrid,shotgun
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Provides-Extra: sg-python-api
+Requires-Dist: sg-python-api<0.2,>=0.1; extra == 'sg-python-api'
+Provides-Extra: sg-python-api-async
+Requires-Dist: sg-python-api[async]<0.2,>=0.1; extra == 'sg-python-api-async'
+Provides-Extra: shotgun-api3
+Requires-Dist: shotgun-api3<4,>=3.10; extra == 'shotgun-api3'
+Description-Content-Type: text/markdown
+
+# sg-event-daemon
+
+sg-event-daemon runs your Python plugins in response to Autodesk Flow Production Tracking events.
+It watches `EventLogEntry` records, delivers each event to the plugins that want it, and records
+delivery progress in SQLite. After a crash or restart it resumes where it stopped, including an
+event whose callbacks had only partly finished.
+
+It is for pipeline teams whose event-driven integrations need controlled recovery and health
+checks. One foreground supervisor can serve several sites and plugin groups. Groups can share one
+event poller while keeping separate ordering, failure handling, and recovery, so a failing plugin
+stops only its own group.
+
+This documentation uses SG for Flow Production Tracking, formerly named Shotgun and ShotGrid.
+
+## Why use it?
+
+The first-party [shotgunEvents](https://github.com/shotgunsoftware/shotgunevents) daemon remains a
+good fit for existing plugins and simple deployments. It loads its plugins directly, reloads
+changed plugin files, daemonizes itself, and can send logs to files and email. sg-event-daemon
+makes different tradeoffs:
+
+| Need                  | sg-event-daemon behavior                                                                                                              |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
+| Recover after a crash | Records its position, the event in progress, and each completed callback in SQLite. Event contents are refetched from SG, not stored. |
+| Isolate workloads     | Polls each event source once for any number of plugin groups. Each group delivers serially and fails independently.                   |
+| Control ordering      | Offers low-latency `eager` delivery, or `strict` delivery in increasing event ID order at the cost of a holdback delay.               |
+| Operate failures      | Provides status and health probes, callback deadlines, worker restart limits, and audited recovery commands.                          |
+| Write modern plugins  | Runs synchronous callbacks, and native asynchronous callbacks with the `sg-python-api-async` extra.                                   |
+
+Weigh these limits before you adopt it:
+
+- shotgunEvents plugins do not run unmodified. Port each one to explicit registration, as the
+  [porting guide](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/porting.md) shows.
+- A new plugin group starts at the latest visible event. The daemon does not import the event
+  position saved by shotgunEvents.
+- Plugin files are not reloaded while the daemon runs. Restart the daemon to load changed plugin
+  code.
+- The daemon runs in the foreground. A service manager such as systemd or Kubernetes owns process
+  restarts, log capture, and alerts.
+- Delivery is at least once. A callback can run again after a crash, retry, or replay, so its side
+  effects must tolerate repetition.
+
+## Installation
+
+You need:
+
+- Linux or macOS.
+- Python 3.11 or newer whose SQLite library is version 3.51.3 or newer. The daemon checks the
+  SQLite version before it opens state.
+- Exactly one SG API provider. `shotgun-api3` supports synchronous groups; `sg-python-api` supports
+  synchronous and asynchronous groups.
+- An SG site and script credentials that can read events and perform your plugins' work.
+- A private, persistent state directory on local storage, owned by the user that runs the daemon.
+
+From a checkout, install the command with [uv](https://docs.astral.sh/uv/) and the standard
+Autodesk `shotgun-api3` provider:
+
+```sh
+uv tool install '.[shotgun-api3]'
+```
+
+To use the `sg-python-api` provider instead, install the `sg-python-api` extra for synchronous
+groups or the `sg-python-api-async` extra for asynchronous groups. Install only one provider: both
+supply the `shotgun_api3` module, and the daemon refuses to run when both are present.
+
+Python's version alone does not determine its SQLite version. Check the SQLite library in the
+installed tool's environment:
+
+```sh
+"$(uv tool dir)/sg-event-daemon/bin/python" -c 'import sqlite3; print(sqlite3.sqlite_version)'
+```
+
+## Quick start
+
+This example sets up one plugin that logs each new event, then starts the daemon in the foreground.
+
+### Write a plugin
+
+Save this plugin as `/etc/sg-event-daemon/plugins/observe.py`:
+
+```python
+def on_event(context):
+    context.logger.info('EventLogEntry %s', context.event['id'])
+
+
+def register(registrar):
+    registrar.add_callback(id='log-event', callback=on_event)
+```
+
+### Configure the daemon
+
+Save this configuration as `/etc/sg-event-daemon/daemon.toml`, replacing the site URL and script
+name with your own:
+
+```toml
+[daemon]
+state_dir = "/var/lib/sg-event-daemon"
+sg_api_provider = "shotgun-api3"
+
+[sites.studio]
+server = "https://studio.example.invalid"
+default_source = "primary"
+
+[sites.studio.credentials.event-reader]
+script_name = "event-daemon-reader"
+api_key = { env = "SG_EVENT_READER_KEY" }
+
+[sites.studio.sources.primary]
+credentials = "event-reader"
+
+[sites.studio.groups.observers]
+source = "primary"
+callback_credentials = "event-reader"
+execution = "sync"
+
+[[sites.studio.groups.observers.plugins]]
+id = "event-log"
+path = "plugins/observe.py"
+```
+
+The script key comes from the `SG_EVENT_READER_KEY` environment variable, so the configuration file
+holds no secrets. Relative plugin paths and `state_dir` resolve from the directory of the
+configuration file after symlinks are resolved. If `state_dir` is missing, the daemon creates it,
+which requires write access to its parent. The directory must be owned by the daemon's user with
+mode `0700`.
+
+### Validate and start
+
+With `SG_EVENT_READER_KEY` set in the environment, validate the configuration and plugin, then
+start the supervisor:
+
+```sh
+sg-event-daemon validate-config --config /etc/sg-event-daemon/daemon.toml
+sg-event-daemon supervise --config /etc/sg-event-daemon/daemon.toml
+```
+
+Validation imports the plugin without contacting SG. It warns that importing plugin code can have
+side effects, then prints `{"provider":"shotgun-api3","sites":["studio"],"validated":true}`.
+Invalid configuration, including a missing plugin file, makes either command print `error:` and
+exit with status 2. Invalid plugin registration makes `validate-config` exit with status 2. During
+`supervise`, a plugin import or registration failure stops its worker; the supervisor restarts that
+worker until its breaker opens.
+
+The supervisor runs until you stop it with Ctrl-C or SIGTERM, which shuts down its workers and
+exits with status 0. The new group starts at the latest visible event rather than processing
+existing history. When someone changes something in SG, the supervisor's output shows an
+`EventLogEntry` line from the plugin.
+
+### Check status
+
+In another shell, ask the running supervisor for its status:
+
+```sh
+sg-event-daemon status --config /etc/sg-event-daemon/daemon.toml
+```
+
+The command prints one line of JSON. `primary_state` summarizes overall health, for example `idle`
+while the group waits for events, and `liveness` and `readiness` report the health checks. If no
+supervisor is running for that state directory, the command prints one line that starts with
+`error:` and exits with status 3.
+
+This example only logs events. A plugin that writes to SG should use separate callback credentials
+with the permissions it needs; see
+[separate reader and writer credentials](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#separate-reader-and-writer-credentials).
+
+## Documentation
+
+| If you want to...                              | Read                                                                     |
+| ---------------------------------------------- | ------------------------------------------------------------------------ |
+| Port a shotgunEvents plugin or write a new one | [Porting plugins](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/porting.md)                                       |
+| Test a plugin without a daemon or SG site      | [Offline harness](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/porting.md#test-plugins-offline)                  |
+| Give plugins separate credentials for writing  | [Credentials](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#separate-reader-and-writer-credentials) |
+| Choose between eager and strict ordering       | [Ordering](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#choose-eager-or-strict-ordering)           |
+| Monitor health or configure probes             | [Status and probes](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#check-status-and-health)          |
+| Recover a failed or stopped group              | [Recovery](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#recover-a-group-or-source)                 |
+| Back up state or plan storage                  | [State and backups](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#sqlite-state-and-backups)         |
+| Run on Kubernetes                              | [Kubernetes](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#kubernetes)                              |
+| Size queues, buffers, and memory               | [Capacity](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/operations.md#capacity)                                  |
+| See changes by version                         | [Changelog](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/CHANGELOG.md)                                                |
+
+## Development
+
+See the [development guide](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/docs/development.md) for the required tools, CI, and dependency locks.
+With those tools installed, run:
+
+```sh
+make setup
+make check
+```
+
+The checks never contact a live SG site; the development guide describes the opt-in live tests.
+
+## License
+
+sg-event-daemon is available under the [MIT License](https://github.com/dactylo/sg-event-daemon/blob/v0.1.0/LICENSE). It is an independent project and is
+not affiliated with or endorsed by Autodesk. Flow Production Tracking is a product of Autodesk,
+Inc.

sg_event_daemon-0.1.0/README.md

@@ -0,0 +1,197 @@
+# sg-event-daemon
+
+sg-event-daemon runs your Python plugins in response to Autodesk Flow Production Tracking events.
+It watches `EventLogEntry` records, delivers each event to the plugins that want it, and records
+delivery progress in SQLite. After a crash or restart it resumes where it stopped, including an
+event whose callbacks had only partly finished.
+
+It is for pipeline teams whose event-driven integrations need controlled recovery and health
+checks. One foreground supervisor can serve several sites and plugin groups. Groups can share one
+event poller while keeping separate ordering, failure handling, and recovery, so a failing plugin
+stops only its own group.
+
+This documentation uses SG for Flow Production Tracking, formerly named Shotgun and ShotGrid.
+
+## Why use it?
+
+The first-party [shotgunEvents](https://github.com/shotgunsoftware/shotgunevents) daemon remains a
+good fit for existing plugins and simple deployments. It loads its plugins directly, reloads
+changed plugin files, daemonizes itself, and can send logs to files and email. sg-event-daemon
+makes different tradeoffs:
+
+| Need                  | sg-event-daemon behavior                                                                                                              |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
+| Recover after a crash | Records its position, the event in progress, and each completed callback in SQLite. Event contents are refetched from SG, not stored. |
+| Isolate workloads     | Polls each event source once for any number of plugin groups. Each group delivers serially and fails independently.                   |
+| Control ordering      | Offers low-latency `eager` delivery, or `strict` delivery in increasing event ID order at the cost of a holdback delay.               |
+| Operate failures      | Provides status and health probes, callback deadlines, worker restart limits, and audited recovery commands.                          |
+| Write modern plugins  | Runs synchronous callbacks, and native asynchronous callbacks with the `sg-python-api-async` extra.                                   |
+
+Weigh these limits before you adopt it:
+
+- shotgunEvents plugins do not run unmodified. Port each one to explicit registration, as the
+  [porting guide](docs/porting.md) shows.
+- A new plugin group starts at the latest visible event. The daemon does not import the event
+  position saved by shotgunEvents.
+- Plugin files are not reloaded while the daemon runs. Restart the daemon to load changed plugin
+  code.
+- The daemon runs in the foreground. A service manager such as systemd or Kubernetes owns process
+  restarts, log capture, and alerts.
+- Delivery is at least once. A callback can run again after a crash, retry, or replay, so its side
+  effects must tolerate repetition.
+
+## Installation
+
+You need:
+
+- Linux or macOS.
+- Python 3.11 or newer whose SQLite library is version 3.51.3 or newer. The daemon checks the
+  SQLite version before it opens state.
+- Exactly one SG API provider. `shotgun-api3` supports synchronous groups; `sg-python-api` supports
+  synchronous and asynchronous groups.
+- An SG site and script credentials that can read events and perform your plugins' work.
+- A private, persistent state directory on local storage, owned by the user that runs the daemon.
+
+From a checkout, install the command with [uv](https://docs.astral.sh/uv/) and the standard
+Autodesk `shotgun-api3` provider:
+
+```sh
+uv tool install '.[shotgun-api3]'
+```
+
+To use the `sg-python-api` provider instead, install the `sg-python-api` extra for synchronous
+groups or the `sg-python-api-async` extra for asynchronous groups. Install only one provider: both
+supply the `shotgun_api3` module, and the daemon refuses to run when both are present.
+
+Python's version alone does not determine its SQLite version. Check the SQLite library in the
+installed tool's environment:
+
+```sh
+"$(uv tool dir)/sg-event-daemon/bin/python" -c 'import sqlite3; print(sqlite3.sqlite_version)'
+```
+
+## Quick start
+
+This example sets up one plugin that logs each new event, then starts the daemon in the foreground.
+
+### Write a plugin
+
+Save this plugin as `/etc/sg-event-daemon/plugins/observe.py`:
+
+```python
+def on_event(context):
+    context.logger.info('EventLogEntry %s', context.event['id'])
+
+
+def register(registrar):
+    registrar.add_callback(id='log-event', callback=on_event)
+```
+
+### Configure the daemon
+
+Save this configuration as `/etc/sg-event-daemon/daemon.toml`, replacing the site URL and script
+name with your own:
+
+```toml
+[daemon]
+state_dir = "/var/lib/sg-event-daemon"
+sg_api_provider = "shotgun-api3"
+
+[sites.studio]
+server = "https://studio.example.invalid"
+default_source = "primary"
+
+[sites.studio.credentials.event-reader]
+script_name = "event-daemon-reader"
+api_key = { env = "SG_EVENT_READER_KEY" }
+
+[sites.studio.sources.primary]
+credentials = "event-reader"
+
+[sites.studio.groups.observers]
+source = "primary"
+callback_credentials = "event-reader"
+execution = "sync"
+
+[[sites.studio.groups.observers.plugins]]
+id = "event-log"
+path = "plugins/observe.py"
+```
+
+The script key comes from the `SG_EVENT_READER_KEY` environment variable, so the configuration file
+holds no secrets. Relative plugin paths and `state_dir` resolve from the directory of the
+configuration file after symlinks are resolved. If `state_dir` is missing, the daemon creates it,
+which requires write access to its parent. The directory must be owned by the daemon's user with
+mode `0700`.
+
+### Validate and start
+
+With `SG_EVENT_READER_KEY` set in the environment, validate the configuration and plugin, then
+start the supervisor:
+
+```sh
+sg-event-daemon validate-config --config /etc/sg-event-daemon/daemon.toml
+sg-event-daemon supervise --config /etc/sg-event-daemon/daemon.toml
+```
+
+Validation imports the plugin without contacting SG. It warns that importing plugin code can have
+side effects, then prints `{"provider":"shotgun-api3","sites":["studio"],"validated":true}`.
+Invalid configuration, including a missing plugin file, makes either command print `error:` and
+exit with status 2. Invalid plugin registration makes `validate-config` exit with status 2. During
+`supervise`, a plugin import or registration failure stops its worker; the supervisor restarts that
+worker until its breaker opens.
+
+The supervisor runs until you stop it with Ctrl-C or SIGTERM, which shuts down its workers and
+exits with status 0. The new group starts at the latest visible event rather than processing
+existing history. When someone changes something in SG, the supervisor's output shows an
+`EventLogEntry` line from the plugin.
+
+### Check status
+
+In another shell, ask the running supervisor for its status:
+
+```sh
+sg-event-daemon status --config /etc/sg-event-daemon/daemon.toml
+```
+
+The command prints one line of JSON. `primary_state` summarizes overall health, for example `idle`
+while the group waits for events, and `liveness` and `readiness` report the health checks. If no
+supervisor is running for that state directory, the command prints one line that starts with
+`error:` and exits with status 3.
+
+This example only logs events. A plugin that writes to SG should use separate callback credentials
+with the permissions it needs; see
+[separate reader and writer credentials](docs/operations.md#separate-reader-and-writer-credentials).
+
+## Documentation
+
+| If you want to...                              | Read                                                                     |
+| ---------------------------------------------- | ------------------------------------------------------------------------ |
+| Port a shotgunEvents plugin or write a new one | [Porting plugins](docs/porting.md)                                       |
+| Test a plugin without a daemon or SG site      | [Offline harness](docs/porting.md#test-plugins-offline)                  |
+| Give plugins separate credentials for writing  | [Credentials](docs/operations.md#separate-reader-and-writer-credentials) |
+| Choose between eager and strict ordering       | [Ordering](docs/operations.md#choose-eager-or-strict-ordering)           |
+| Monitor health or configure probes             | [Status and probes](docs/operations.md#check-status-and-health)          |
+| Recover a failed or stopped group              | [Recovery](docs/operations.md#recover-a-group-or-source)                 |
+| Back up state or plan storage                  | [State and backups](docs/operations.md#sqlite-state-and-backups)         |
+| Run on Kubernetes                              | [Kubernetes](docs/operations.md#kubernetes)                              |
+| Size queues, buffers, and memory               | [Capacity](docs/operations.md#capacity)                                  |
+| See changes by version                         | [Changelog](CHANGELOG.md)                                                |
+
+## Development
+
+See the [development guide](docs/development.md) for the required tools, CI, and dependency locks.
+With those tools installed, run:
+
+```sh
+make setup
+make check
+```
+
+The checks never contact a live SG site; the development guide describes the opt-in live tests.
+
+## License
+
+sg-event-daemon is available under the [MIT License](LICENSE). It is an independent project and is
+not affiliated with or endorsed by Autodesk. Flow Production Tracking is a product of Autodesk,
+Inc.