imbi-plugin-github 2.9.1__tar.gz → 2.9.3__tar.gz

{imbi_plugin_github-2.9.1 → imbi_plugin_github-2.9.3}/.pre-commit-config.yaml

@@ -21,6 +21,9 @@ repos:
     rev: v0.7.25
     hooks:
       - id: tombi-format
+        # uv owns uv.lock's formatting; tombi reflows it (2- vs 4-space),
+        # producing whole-file churn on every dependency change.
+        exclude: ^uv\.lock$
 
   - repo: local
     hooks:

{imbi_plugin_github-2.9.1 → imbi_plugin_github-2.9.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: imbi-plugin-github
-Version: 2.9.1
+Version: 2.9.3
 Summary: GitHub identity plugin for Imbi (github.com / GHEC / GHES)
 Author-email: "Gavin M. Roy" <gavinr@aweber.com>
 License: BSD-3-Clause
@@ -13,8 +13,9 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.14
 Requires-Python: >=3.14
 Requires-Dist: httpx>=0.27
-Requires-Dist: imbi-common[databases]==2.9.1
+Requires-Dist: imbi-common[databases]==2.9.2
 Requires-Dist: pydantic>=2
+Requires-Dist: pyjwt[crypto]>=2.8
 Description-Content-Type: text/markdown
 
 # imbi-plugin-github
@@ -30,6 +31,7 @@ projects to the right backend.
 | Identity   | `github`, `github-enterprise-cloud`, `github-enterprise-server`      |
 | Deployment | `github-deployment`, `github-deployment-ec`, `github-deployment-es`  |
 | Lifecycle  | `github-lifecycle`, `github-lifecycle-ec`, `github-lifecycle-es`     |
+| Webhook    | `github-commit-sync`                                                  |
 
 ### Identity
 
@@ -63,6 +65,37 @@ org.
 Archiving requires admin scope on the repo; transferring additionally
 requires admin permission on the target organization.
 
+### Webhook (commit / tag sync)
+
+A single `github-commit-sync` webhook-action plugin exposes two actions
+the gateway dispatches on `push` deliveries:
+
+| Action         | Handler                          | Records into ClickHouse |
+| -------------- | -------------------------------- | ----------------------- |
+| `sync_commits` | `github-commit-sync#sync_commits`| `commits`               |
+| `sync_tags`    | `github-commit-sync#sync_tags`   | `tags`                  |
+
+`sync_commits` fetches the full set of commits in a push via the compare
+API (paginated, so it isn't capped by the 20-commit inline payload limit);
+`sync_tags` records the pushed tag and, with `reconcile_all`, the repo's
+full tag list. Branch/tag gating is the rule's CEL `filter_expression`
+(e.g. `ref == "refs/heads/main"`, `ref.startsWith("refs/tags/")`). The API
+flavor (github.com / GHEC / GHES) is resolved at runtime — explicit
+`api_base_url`, else a connected GitHub plugin on the same service, else
+the service endpoint, else the payload's `repository.url`.
+
+Unlike identity/deployment/lifecycle (which act as the OAuth user),
+commit-sync runs without an actor and authenticates with a **service**
+credential in one of two modes, resolved per call:
+
+- **PAT** — a static `access_token`.
+- **GitHub App** — `app_id` + `private_key`; the plugin signs an App JWT
+  and mints a short-lived **installation token** (cached process-wide
+  until shortly before it expires), so no static, expiring token is
+  stored. `installation_id` is optional — when unset it is discovered
+  from the pushed repository (`GET /repos/{owner}/{repo}/installation`).
+  The App needs **Contents: Read-only**.
+
 ## Manifest options (identity)
 
 | Option           | Required  | Description                                                                |
@@ -77,6 +110,18 @@ requires admin permission on the target organization.
 | `client_id`      | yes      |
 | `client_secret`  | yes      |
 
+## Credentials (commit-sync)
+
+Provide **either** the PAT field **or** the GitHub App fields (all
+individually optional; validated per call):
+
+| Field             | Mode | Description                                            |
+| ----------------- | ---- | ------------------------------------------------------ |
+| `access_token`    | PAT  | Static personal/service token.                         |
+| `app_id`          | App  | GitHub App identifier.                                 |
+| `private_key`     | App  | App private key — raw PEM or base64-encoded PEM.       |
+| `installation_id` | App  | Optional; discovered from the repo when unset.         |
+
 ## License
 
 BSD-3-Clause.

{imbi_plugin_github-2.9.1 → imbi_plugin_github-2.9.3}/README.md

@@ -11,6 +11,7 @@ projects to the right backend.
 | Identity   | `github`, `github-enterprise-cloud`, `github-enterprise-server`      |
 | Deployment | `github-deployment`, `github-deployment-ec`, `github-deployment-es`  |
 | Lifecycle  | `github-lifecycle`, `github-lifecycle-ec`, `github-lifecycle-es`     |
+| Webhook    | `github-commit-sync`                                                  |
 
 ### Identity
 
@@ -44,6 +45,37 @@ org.
 Archiving requires admin scope on the repo; transferring additionally
 requires admin permission on the target organization.
 
+### Webhook (commit / tag sync)
+
+A single `github-commit-sync` webhook-action plugin exposes two actions
+the gateway dispatches on `push` deliveries:
+
+| Action         | Handler                          | Records into ClickHouse |
+| -------------- | -------------------------------- | ----------------------- |
+| `sync_commits` | `github-commit-sync#sync_commits`| `commits`               |
+| `sync_tags`    | `github-commit-sync#sync_tags`   | `tags`                  |
+
+`sync_commits` fetches the full set of commits in a push via the compare
+API (paginated, so it isn't capped by the 20-commit inline payload limit);
+`sync_tags` records the pushed tag and, with `reconcile_all`, the repo's
+full tag list. Branch/tag gating is the rule's CEL `filter_expression`
+(e.g. `ref == "refs/heads/main"`, `ref.startsWith("refs/tags/")`). The API
+flavor (github.com / GHEC / GHES) is resolved at runtime — explicit
+`api_base_url`, else a connected GitHub plugin on the same service, else
+the service endpoint, else the payload's `repository.url`.
+
+Unlike identity/deployment/lifecycle (which act as the OAuth user),
+commit-sync runs without an actor and authenticates with a **service**
+credential in one of two modes, resolved per call:
+
+- **PAT** — a static `access_token`.
+- **GitHub App** — `app_id` + `private_key`; the plugin signs an App JWT
+  and mints a short-lived **installation token** (cached process-wide
+  until shortly before it expires), so no static, expiring token is
+  stored. `installation_id` is optional — when unset it is discovered
+  from the pushed repository (`GET /repos/{owner}/{repo}/installation`).
+  The App needs **Contents: Read-only**.
+
 ## Manifest options (identity)
 
 | Option           | Required  | Description                                                                |
@@ -58,6 +90,18 @@ requires admin permission on the target organization.
 | `client_id`      | yes      |
 | `client_secret`  | yes      |
 
+## Credentials (commit-sync)
+
+Provide **either** the PAT field **or** the GitHub App fields (all
+individually optional; validated per call):
+
+| Field             | Mode | Description                                            |
+| ----------------- | ---- | ------------------------------------------------------ |
+| `access_token`    | PAT  | Static personal/service token.                         |
+| `app_id`          | App  | GitHub App identifier.                                 |
+| `private_key`     | App  | App private key — raw PEM or base64-encoded PEM.       |
+| `installation_id` | App  | Optional; discovered from the repo when unset.         |
+
 ## License
 
 BSD-3-Clause.

{imbi_plugin_github-2.9.1 → imbi_plugin_github-2.9.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "imbi-plugin-github"
-version = "2.9.1"
+version = "2.9.3"
 description = "GitHub identity plugin for Imbi (github.com / GHEC / GHES)"
 readme = "README.md"
 requires-python = ">=3.14"
@@ -16,8 +16,9 @@ classifiers = [
 ]
 dependencies = [
   "httpx>=0.27",
-  "imbi-common[databases]==2.9.1",
+  "imbi-common[databases]==2.9.2",
   "pydantic>=2",
+  "pyjwt[crypto]>=2.8",
 ]
 
 [project.entry-points."imbi.plugins"]
@@ -36,7 +37,7 @@ github-commit-sync = "imbi_plugin_github.commits:GitHubCommitSyncPlugin"
 dev = [
   "basedpyright",
   "coverage[toml]",
-  "imbi-common[server,databases]==2.9.1",
+  "imbi-common[server,databases]==2.9.2",
   "pre-commit",
   "pytest",
   "pytest-asyncio",

imbi_plugin_github-2.9.3/src/imbi_plugin_github/_app_auth.py

@@ -0,0 +1,195 @@
+"""GitHub App installation-token minting for webhook actions.
+
+The commit-sync webhook plugin has no acting user, so when it is
+configured with GitHub App credentials (``app_id`` + ``private_key``) it
+mints a short-lived *installation* access token per call instead of
+carrying a static PAT.  Tokens are cached process-wide until shortly
+before they expire, so a busy org makes one token-exchange round-trip
+per hour per ``(app, installation, host)`` rather than one per webhook
+delivery.
+
+All three GitHub flavors work unchanged: the caller resolves the API
+base via :func:`imbi_plugin_github._hosts.host_to_api_base` and passes
+it in, so the JWT exchange hits ``api.github.com``,
+``api.<tenant>.ghe.com``, or ``<ghes>/api/v3`` as appropriate.
+"""
+
+from __future__ import annotations
+
+import base64
+import binascii
+import datetime
+import logging
+import time
+import typing
+
+import httpx
+import jwt
+
+from imbi_plugin_github.deployment import (
+    _auth_headers,  # pyright: ignore[reportPrivateUsage]
+    _raise_on_401,  # pyright: ignore[reportPrivateUsage]
+)
+
+LOGGER = logging.getLogger(__name__)
+
+_HTTP_TIMEOUT_SECONDS = 10.0
+# GitHub rejects an App JWT whose ``exp`` is more than 10 minutes out;
+# sign for 9 to leave room for clock skew between us and GitHub.
+_JWT_TTL_SECONDS = 540
+# Re-mint an installation token this many seconds before it actually
+# expires so an in-flight request never races the expiry boundary.
+_TOKEN_REFRESH_MARGIN_SECONDS = 300.0
+# Installation tokens last an hour; assume ~55 minutes when GitHub omits
+# (or we can't parse) the ``expires_at`` field.
+_DEFAULT_TOKEN_TTL_SECONDS = 3300.0
+
+# Process-wide caches.  Token cache values are ``(token, deadline)`` where
+# ``deadline`` is a ``time.monotonic()`` instant; the installation cache
+# avoids re-discovering the installation id on every delivery.
+_TOKEN_CACHE: dict[tuple[str, str, str], tuple[str, float]] = {}
+_INSTALL_CACHE: dict[tuple[str, str, str, str], str] = {}
+
+
+def reset_cache() -> None:
+    """Clear the process-wide token / installation caches (tests)."""
+    _TOKEN_CACHE.clear()
+    _INSTALL_CACHE.clear()
+
+
+def _load_private_key(raw: str) -> str:
+    """Return a PEM private key from raw PEM or a base64-encoded PEM.
+
+    Operators may paste the key GitHub generated directly, or a
+    single-line base64 encoding of it (handy where the config UI lacks a
+    multi-line field).  Raises ``ValueError`` for anything else.
+    """
+    value = raw.strip()
+    if '-----BEGIN' in value:
+        return value
+    try:
+        decoded = base64.b64decode(value, validate=True).decode('utf-8')
+    except (binascii.Error, ValueError, UnicodeDecodeError) as exc:
+        raise ValueError(
+            'github-commit-sync private_key is neither a PEM nor a '
+            'base64-encoded PEM'
+        ) from exc
+    if '-----BEGIN' not in decoded:
+        raise ValueError(
+            'github-commit-sync private_key decoded but is not a PEM'
+        )
+    return decoded
+
+
+def _app_jwt(app_id: str, private_key: str) -> str:
+    now = int(time.time())
+    return jwt.encode(
+        {'iat': now - 60, 'exp': now + _JWT_TTL_SECONDS, 'iss': app_id},
+        _load_private_key(private_key),
+        algorithm='RS256',
+    )
+
+
+def _token_deadline(expires_at: object) -> float:
+    """Map GitHub's ISO ``expires_at`` to a monotonic cache deadline."""
+    now = time.monotonic()
+    if not isinstance(expires_at, str):
+        return now + _DEFAULT_TOKEN_TTL_SECONDS
+    try:
+        exp = datetime.datetime.fromisoformat(expires_at)
+    except ValueError:
+        return now + _DEFAULT_TOKEN_TTL_SECONDS
+    remaining = (exp - datetime.datetime.now(datetime.UTC)).total_seconds()
+    return now + max(0.0, remaining - _TOKEN_REFRESH_MARGIN_SECONDS)
+
+
+def _cached_token(key: tuple[str, str, str]) -> str | None:
+    entry = _TOKEN_CACHE.get(key)
+    if entry is not None and entry[1] > time.monotonic():
+        return entry[0]
+    return None
+
+
+async def _discover_installation_id(
+    client: httpx.AsyncClient, owner: str, repo: str
+) -> str:
+    resp = await client.get(f'/repos/{owner}/{repo}/installation')
+    resp.raise_for_status()
+    data = typing.cast('dict[str, typing.Any]', resp.json())
+    install_id = data.get('id')
+    if install_id is None:
+        raise ValueError(
+            f'no GitHub App installation found for {owner}/{repo}'
+        )
+    return str(install_id)
+
+
+async def _mint(
+    client: httpx.AsyncClient, installation_id: str
+) -> tuple[str, object]:
+    resp = await client.post(
+        f'/app/installations/{installation_id}/access_tokens'
+    )
+    resp.raise_for_status()
+    data = typing.cast('dict[str, typing.Any]', resp.json())
+    return str(data['token']), data.get('expires_at')
+
+
+async def installation_token(
+    *,
+    base: str,
+    app_id: str,
+    private_key: str,
+    installation_id: str | None,
+    owner: str,
+    repo: str,
+) -> str:
+    """Return a valid installation token, minting/caching as needed.
+
+    ``installation_id`` may be ``None``, in which case the installation
+    is discovered from the target repo (and cached).  The resulting
+    token is cached until shortly before it expires.
+    """
+    install = installation_id
+    if install is not None and (
+        cached := _cached_token((app_id, install, base))
+    ):
+        return cached
+    if install is None:
+        install = _INSTALL_CACHE.get((app_id, base, owner, repo))
+        if install is not None and (
+            cached := _cached_token((app_id, install, base))
+        ):
+            return cached
+
+    install_was_cached = install is not None and installation_id is None
+    app_token = _app_jwt(app_id, private_key)
+    async with httpx.AsyncClient(
+        base_url=base,
+        headers=_auth_headers(app_token),
+        timeout=_HTTP_TIMEOUT_SECONDS,
+        event_hooks={'response': [_raise_on_401]},
+    ) as client:
+        if install is None:
+            install = await _discover_installation_id(client, owner, repo)
+            _INSTALL_CACHE[(app_id, base, owner, repo)] = install
+            if cached := _cached_token((app_id, install, base)):
+                return cached
+        try:
+            token, expires_at = await _mint(client, install)
+        except httpx.HTTPStatusError as exc:
+            # A 404 (or 401, surfaced as PluginAuthenticationFailed by the
+            # response hook) against a *cached* installation id means the
+            # app was uninstalled/reinstalled or transferred. Evict the
+            # stale id and rediscover once before giving up.
+            if not install_was_cached or exc.response.status_code != 404:
+                raise
+            _INSTALL_CACHE.pop((app_id, base, owner, repo), None)
+            install = await _discover_installation_id(client, owner, repo)
+            _INSTALL_CACHE[(app_id, base, owner, repo)] = install
+            token, expires_at = await _mint(client, install)
+    _TOKEN_CACHE[(app_id, install, base)] = (
+        token,
+        _token_deadline(expires_at),
+    )
+    return token