bkflow-django-webhook 1.0.0__tar.gz

bkflow_django_webhook-1.0.0/PKG-INFO

@@ -0,0 +1,133 @@
+Metadata-Version: 2.1
+Name: bkflow-django-webhook
+Version: 1.0.0
+Summary: A Django app to make it easy for integrating webhook into service.
+Author-email: normal-wls <weishi.swee@qq.com>
+Description-Content-Type: text/markdown
+Requires-Dist: Django >2, <4
+Requires-Dist: pyyaml >5, <7
+Requires-Dist: pydantic <3
+Requires-Dist: pytest >=7.0.1,<8 ; extra == "test"
+Requires-Dist: pytest-django>=4.5.2,<5.0 ; extra == "test"
+Requires-Dist: djangorestframework >3, <4 ; extra == "test"
+Project-URL: Home, https://github.com/TencentBlueKing/bkflow-django-webhook
+Provides-Extra: test
+
+# bkflow-django-webhook
+
+## 简介
+bkflow-django-webhook 是一款支持系统快速集成 webhook 功能的 Django app。
+
+webhook 是一种在特定事件发生时，通过 HTTP 请求将数据发送到指定的 URL 的实时通信的机制。
+
+webhook 的使用场景非常广泛。例如，当用户在某个网站上进行了特定操作（如提交表单、创建账户或进行支付）时，网站可以使用 webhook 将相关数据发送给其他应用程序或服务。这样，接收方应用程序就能够实时获取到这些数据，并根据需要进行处理。
+
+集成 bkflow-django-webhook，可以让 Django 应用快速获得 webhook 所需要的能力，帮助应用实现基于事件的服务自动化调用。
+
+## 相关概念
+![bkflow_django_webhook_concepts](docs/pics/bkflow_django_webhook_concepts.png)
+
+上图描述了从事件触发到最终请求的过程，涉及以下几个概念：
+1. 请求配置 (Webhook) : 记录发送请求所需要的 endpoint、token 等信息，根据该配置发送 HTTP POST 请求完成对外部服务的调用。
+2. 事件 (Event) : 系统触发请求的类型，需要提前定义。
+3. 领域 (Scope) : `事件`的触发往往是因为资源的变更，而资源会有`领域`的划分，不同`领域`的资源往往需要发送不同的请求。（上图的 template 是资源，template 属于不同的业务，业务就是 Scope，不同业务下的资源事件可能会触发不同的 webhook 请求。比如 biz_1 业务下的 template 变更时，会通过 webhook 1 请求 service 1；而 biz_2 业务下的 template 变更时，会通过 webhook 2 请求 service 2。）
+4. 订阅 (Subscription) : 记录不同`事件`、`领域`和`请求配置`订阅关系的配置，用于在`事件`触发时筛选出对应的`请求配置`进行请求调用。
+
+在 bkflow-django-webhook 的设计中，资源 (图中 template) 的变更触发了事件 (Event)，根据事件、资源所属领域 (Scope, 图中 biz) 和订阅 (Subscription) 的记录，筛选出命中的请求配置 (Webhook)，最终对对应的服务 (service) 发送请求。
+
+
+## 快速上手
+
+下面以上图为例，说明如何快速上手。
+
+#### 安装
+
+```shell
+pip install bkflow-django-webhook
+```
+
+#### 添加到 Django 项目中
+
+将 `webhook` 添加到 Django 项目中
+
+```python
+INSTALLED_APPS = [
+    ...
+    'webhook',
+]
+```
+
+建表
+
+```shell
+python manage.py migrate webhook
+```
+
+#### 事件定义
+
+创建定义资源文件，如 `webhook_events.yaml`
+
+```yaml
+version: 1
+events:
+  - code: template_update
+    name: 模板更新
+  - code: template_create
+    name: 模板创建
+```
+
+目前支持对多个事件进行定义，其中 `code` 定义事件唯一键，`name` 定义事件名称。
+
+#### 事件同步
+
+通过 `sync_webhook_events` 命令进行事件同步（可以在每次应用启动时调用进行同步）
+
+```shell
+python manage.py sync_webhook_events . webhook_events.yaml
+```
+
+`sync_webhook_events` 命令包含两个位置参数：
+- base_path: 资源文件所在的目录地址
+- filename: 资源文件名
+
+#### 资源注册
+
+通过 bkflow-django-webhook 的 api 注册 请求配置 和 订阅关系。
+
+```python
+from webhook.api import apply_scope_subscriptions, apply_scope_webhooks
+
+webhook_configs = [
+    {"code": "webhook1", "name": "webhook1", "endpoint": "https://xxx.com"},  # endpoint 是接收请求的服务地址
+    {"code": "webhook2", "name": "webhook2", "endpoint": "https://xxx.com"}
+]
+
+subscription_configs = {"webhook1": ["*"], "webhook2": ["template_update"]}  # "*" 表示订阅所有事件
+
+
+apply_scope_webhooks(scope_type="biz", scope_code="biz1", webhooks=webhook_configs)
+apply_scope_subscriptions(scope_type="biz", scope_code="biz1", subscription_configs=subscription_configs)
+
+```
+
+#### 发起请求
+
+```python
+from webhook.signals import event_broadcast_signal
+
+# 在对应的业务逻辑中触发调用
+event_broadcast_signal.send(
+    sender="template_update",
+    scopes=[("biz", "biz1")],
+    extra_info={"template_id": "template1"},
+)
+```
+
+根据触发的事件(sender)、领域(scopes，可支持多个)过滤出对应的请求配置，并同步发送请求(extra_info将作为请求参数)。
+
+请求完成后，可在项目的 django admin 页面 Webhook/History 中查看请求历史。
+
+
+
+## 资料
+[Release](release.md)

bkflow_django_webhook-1.0.0/README.md

@@ -0,0 +1,118 @@
+# bkflow-django-webhook
+
+## 简介
+bkflow-django-webhook 是一款支持系统快速集成 webhook 功能的 Django app。
+
+webhook 是一种在特定事件发生时，通过 HTTP 请求将数据发送到指定的 URL 的实时通信的机制。
+
+webhook 的使用场景非常广泛。例如，当用户在某个网站上进行了特定操作（如提交表单、创建账户或进行支付）时，网站可以使用 webhook 将相关数据发送给其他应用程序或服务。这样，接收方应用程序就能够实时获取到这些数据，并根据需要进行处理。
+
+集成 bkflow-django-webhook，可以让 Django 应用快速获得 webhook 所需要的能力，帮助应用实现基于事件的服务自动化调用。
+
+## 相关概念
+![bkflow_django_webhook_concepts](docs/pics/bkflow_django_webhook_concepts.png)
+
+上图描述了从事件触发到最终请求的过程，涉及以下几个概念：
+1. 请求配置 (Webhook) : 记录发送请求所需要的 endpoint、token 等信息，根据该配置发送 HTTP POST 请求完成对外部服务的调用。
+2. 事件 (Event) : 系统触发请求的类型，需要提前定义。
+3. 领域 (Scope) : `事件`的触发往往是因为资源的变更，而资源会有`领域`的划分，不同`领域`的资源往往需要发送不同的请求。（上图的 template 是资源，template 属于不同的业务，业务就是 Scope，不同业务下的资源事件可能会触发不同的 webhook 请求。比如 biz_1 业务下的 template 变更时，会通过 webhook 1 请求 service 1；而 biz_2 业务下的 template 变更时，会通过 webhook 2 请求 service 2。）
+4. 订阅 (Subscription) : 记录不同`事件`、`领域`和`请求配置`订阅关系的配置，用于在`事件`触发时筛选出对应的`请求配置`进行请求调用。
+
+在 bkflow-django-webhook 的设计中，资源 (图中 template) 的变更触发了事件 (Event)，根据事件、资源所属领域 (Scope, 图中 biz) 和订阅 (Subscription) 的记录，筛选出命中的请求配置 (Webhook)，最终对对应的服务 (service) 发送请求。
+
+
+## 快速上手
+
+下面以上图为例，说明如何快速上手。
+
+#### 安装
+
+```shell
+pip install bkflow-django-webhook
+```
+
+#### 添加到 Django 项目中
+
+将 `webhook` 添加到 Django 项目中
+
+```python
+INSTALLED_APPS = [
+    ...
+    'webhook',
+]
+```
+
+建表
+
+```shell
+python manage.py migrate webhook
+```
+
+#### 事件定义
+
+创建定义资源文件，如 `webhook_events.yaml`
+
+```yaml
+version: 1
+events:
+  - code: template_update
+    name: 模板更新
+  - code: template_create
+    name: 模板创建
+```
+
+目前支持对多个事件进行定义，其中 `code` 定义事件唯一键，`name` 定义事件名称。
+
+#### 事件同步
+
+通过 `sync_webhook_events` 命令进行事件同步（可以在每次应用启动时调用进行同步）
+
+```shell
+python manage.py sync_webhook_events . webhook_events.yaml
+```
+
+`sync_webhook_events` 命令包含两个位置参数：
+- base_path: 资源文件所在的目录地址
+- filename: 资源文件名
+
+#### 资源注册
+
+通过 bkflow-django-webhook 的 api 注册 请求配置 和 订阅关系。
+
+```python
+from webhook.api import apply_scope_subscriptions, apply_scope_webhooks
+
+webhook_configs = [
+    {"code": "webhook1", "name": "webhook1", "endpoint": "https://xxx.com"},  # endpoint 是接收请求的服务地址
+    {"code": "webhook2", "name": "webhook2", "endpoint": "https://xxx.com"}
+]
+
+subscription_configs = {"webhook1": ["*"], "webhook2": ["template_update"]}  # "*" 表示订阅所有事件
+
+
+apply_scope_webhooks(scope_type="biz", scope_code="biz1", webhooks=webhook_configs)
+apply_scope_subscriptions(scope_type="biz", scope_code="biz1", subscription_configs=subscription_configs)
+
+```
+
+#### 发起请求
+
+```python
+from webhook.signals import event_broadcast_signal
+
+# 在对应的业务逻辑中触发调用
+event_broadcast_signal.send(
+    sender="template_update",
+    scopes=[("biz", "biz1")],
+    extra_info={"template_id": "template1"},
+)
+```
+
+根据触发的事件(sender)、领域(scopes，可支持多个)过滤出对应的请求配置，并同步发送请求(extra_info将作为请求参数)。
+
+请求完成后，可在项目的 django admin 页面 Webhook/History 中查看请求历史。
+
+
+
+## 资料
+[Release](release.md)

bkflow_django_webhook-1.0.0/pyproject.toml

@@ -0,0 +1,36 @@
+[build-system]
+requires = ["flit_core >=3.2,<4"]
+build-backend = "flit_core.buildapi"
+
+[project]
+name = "bkflow-django-webhook"
+description = "A Django app to make it easy for integrating webhook into service."
+authors = [{name = "normal-wls", email = "weishi.swee@qq.com"}]
+readme = "README.md"
+dynamic = ["version"]
+dependencies = [
+    "Django >2, <4",
+    "pyyaml >5, <7",
+    "pydantic <3",
+]
+
+[project.urls]
+Home = "https://github.com/TencentBlueKing/bkflow-django-webhook"
+
+[project.optional-dependencies]
+test = [
+    "pytest >=7.0.1,<8",
+    "pytest-django>=4.5.2,<5.0",
+    "djangorestframework >3, <4",
+]
+
+[tool.flit.module]
+name = "webhook"
+
+
+[tool.black]
+line-length = 120
+fast = true
+
+[tool.isort]
+line_length = 120

bkflow_django_webhook-1.0.0/setup.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python
+# setup.py generated by flit for tools that don't yet use PEP 517
+
+from distutils.core import setup
+
+packages = \
+['webhook',
+ 'webhook.contrib',
+ 'webhook.contrib.drf',
+ 'webhook.management',
+ 'webhook.management.commands',
+ 'webhook.migrations',
+ 'webhook.requester',
+ 'webhook.signals']
+
+package_data = \
+{'': ['*']}
+
+install_requires = \
+['Django >2, <4', 'pyyaml >5, <7', 'pydantic <3']
+
+extras_require = \
+{'test': ['pytest >=7.0.1,<8',
+          'pytest-django>=4.5.2,<5.0',
+          'djangorestframework >3, <4']}
+
+setup(name='bkflow-django-webhook',
+      version='1.0.0',
+      description='A Django app to make it easy for integrating webhook into service.',
+      author=None,
+      author_email='normal-wls <weishi.swee@qq.com>',
+      url=None,
+      packages=packages,
+      package_data=package_data,
+      install_requires=install_requires,
+      extras_require=extras_require,
+     )

bkflow_django_webhook-1.0.0/webhook/__init__.py

@@ -0,0 +1,3 @@
+# -*- coding: utf-8 -*-
+
+__version__ = "1.0.0"

bkflow_django_webhook-1.0.0/webhook/admin.py

@@ -0,0 +1,37 @@
+# -*- coding: utf-8 -*-
+from django.contrib import admin
+
+from webhook import models
+
+
+@admin.register(models.Event)
+class EventAdmin(admin.ModelAdmin):
+    list_display = [field.name for field in models.Event._meta.fields]
+    search_fields = ["code", "name"]
+
+
+@admin.register(models.Webhook)
+class WebhookAdmin(admin.ModelAdmin):
+    list_display = [field.name for field in models.Webhook._meta.fields]
+    search_fields = ["code", "name", "scope_code"]
+
+
+@admin.register(models.History)
+class HistoryAdmin(admin.ModelAdmin):
+    list_display = [field.name for field in models.History._meta.fields]
+    search_fields = ["delivery_id", "webhook_code", "event_code", "scope_code"]
+    list_filter = ["success", "status_code"]
+
+
+@admin.register(models.Subscription)
+class SubscriptionAdmin(admin.ModelAdmin):
+    list_display = [field.name for field in models.Subscription._meta.fields]
+    search_fields = ["webhook_code", "event_code", "scope_code"]
+    list_filter = ["event_code"]
+
+
+@admin.register(models.Scope)
+class ScopeAdmin(admin.ModelAdmin):
+    list_display = [field.name for field in models.Scope._meta.fields]
+    search_fields = ["code"]
+    list_filter = ["type"]

bkflow_django_webhook-1.0.0/webhook/api.py

@@ -0,0 +1,67 @@
+# -*- coding: utf-8 -*-
+import logging
+from typing import List, Tuple, Union, Dict
+
+from webhook.base_models import Event, Scope, Webhook
+from webhook.handlers import EventHandler
+from webhook.models import Event as EventModel
+from webhook.models import Subscription
+from webhook.models import Webhook as WebhookModel
+
+logger = logging.getLogger(__name__)
+
+
+def get_scope_webhooks(scope_type: str, scope_code: str) -> List[Webhook]:
+    """
+    get webhooks of scope
+    """
+    scope = Scope(scope_type=scope_type, scope_code=scope_code)
+    return [
+        Webhook.from_orm(webhook)
+        for webhook in WebhookModel.objects.filter(scope_type=scope.type, scope_code=scope.code)
+    ]
+
+
+def apply_scope_webhooks(scope_type: str, scope_code: str, webhooks: List[Dict]) -> None:
+    """
+    update or create webhooks by given list and delete others
+    """
+    scope = Scope(type=scope_type, code=scope_code)
+    webhooks = [Webhook(**webhook, scope_type=scope_type, scope_code=scope_code) for webhook in webhooks]
+    WebhookModel.objects.apply_scope_webhooks(scope, webhooks)
+
+
+def apply_scope_subscriptions(scope_type: str, scope_code: str, subscription_configs: Dict) -> None:
+    scope = Scope(type=scope_type, code=scope_code)
+    Subscription.objects.apply_scope_subscriptions(scope, subscription_configs)
+
+
+def event_broadcast(event: Union[Event, str], scopes: List[Union[Scope, Tuple[str, str]]], *args, **kwargs):
+    """
+    broadcast event to make subscription webhooks send requests
+    """
+    logger.info(f"[event broadcasting...] event: {event}, scopes: {scopes}, args: {args}, kwargs: {kwargs}")
+    if isinstance(event, str):
+        event_instance = EventModel.objects.filter(code=event).first()
+        if not event_instance:
+            logger.error(f"event {event} not found")
+            return
+        event = Event.from_orm(event_instance)
+
+        extra_info = kwargs.get("extra_info", {})
+        if event.info:
+            event.info.update(extra_info)
+        else:
+            event.info = extra_info
+    if not isinstance(event, Event):
+        logger.error(f"event {event} is not a Event instance")
+        return
+
+    if not all([isinstance(scope, (Scope, tuple)) for scope in scopes]):
+        logger.error(f"scopes {scopes} is not a Scope or tuple instance")
+        return
+
+    scopes = [Scope(type=scope[0], code=scope[1]) if isinstance(scope, tuple) else scope for scope in scopes]
+
+    event_handler = EventHandler(event)
+    event_handler.handle(scopes=scopes)

bkflow_django_webhook-1.0.0/webhook/apps.py

@@ -0,0 +1,16 @@
+from django.apps import AppConfig
+
+
+class WebhookConfig(AppConfig):
+    default_auto_field = "django.db.models.BigAutoField"
+    name = "webhook"
+
+    def ready(self):
+        from .signals import (  # noqa
+            event_broadcast_signal,
+            post_event_broadcast_signal,
+            post_send_request_signal,
+            pre_event_broadcast_signal,
+            pre_send_request_signal,
+        )
+        from .signals.handlers import handle_event_broadcast  # noqa

bkflow_django_webhook-1.0.0/webhook/base_models.py

@@ -0,0 +1,38 @@
+# -*- coding: utf-8 -*-
+from typing import Optional
+
+from pydantic import BaseModel
+
+
+class Event(BaseModel):
+    code: str
+    name: str
+    description: Optional[str] = None
+    info: Optional[dict] = None
+
+    class Config:
+        orm_mode = True
+        from_attributes = True
+
+
+class Webhook(BaseModel):
+    code: str
+    name: str
+    endpoint: str
+    scope_type: str
+    scope_code: str
+    token: Optional[str] = None
+    extra_info: Optional[dict] = None
+
+    class Config:
+        orm_mode = True
+        from_attributes = True
+
+
+class Scope(BaseModel):
+    type: str
+    code: str
+
+    class Config:
+        orm_mode = True
+        from_attributes = True

bkflow_django_webhook-1.0.0/webhook/config.py

@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-
+from django.conf import settings
+
+DEFAULT_SETTINGS = {
+    "MODE": "SYNC",  # SYNC or ASYNC
+    "ALL_EVENTS_KEY": "*",
+    "EVENT": {
+        "DATA_SOURCE": "DB",  # DB or SETTINGS
+    },
+    "REQUEST": {
+        "TIMEOUT": 30,
+    },
+}
+
+
+class WebhookSettings:
+    SETTING_PREFIX = "WEBHOOK"
+    NESTING_SEPARATOR = "_"
+
+    def __init__(self, default_settings=None):
+        self.project_settings = self.get_flatten_settings(getattr(settings, self.SETTING_PREFIX, {}))
+        self.default_settings = self.get_flatten_settings(default_settings or DEFAULT_SETTINGS)
+
+    def __getattr__(self, key):
+        if key not in self.project_settings and key not in self.default_settings:
+            raise AttributeError
+
+        value = self.project_settings.get(key) or self.default_settings.get(key)
+        if value is not None:
+            setattr(self, key, value)
+        return value
+
+    def get_flatten_settings(self, inputted_settings: dict, cur_prefix: str = ""):
+        def get_cur_key(cur_key):
+            return f"{cur_prefix}{self.NESTING_SEPARATOR}{cur_key}" if cur_prefix else cur_key
+
+        flatten_settings = {}
+        for key, value in inputted_settings.items():
+            if isinstance(value, dict):
+                flatten_sub_settings = self.get_flatten_settings(value, key)
+                flatten_settings.update(
+                    {
+                        get_cur_key(flatten_key): flatten_value
+                        for flatten_key, flatten_value in flatten_sub_settings.items()
+                    }
+                )
+            else:
+                flatten_settings[get_cur_key(key)] = value
+        return flatten_settings
+
+
+webhook_settings = WebhookSettings(DEFAULT_SETTINGS)

bkflow_django_webhook-1.0.0/webhook/contrib/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

bkflow_django_webhook-1.0.0/webhook/contrib/drf/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

bkflow_django_webhook-1.0.0/webhook/contrib/drf/serializers.py

@@ -0,0 +1,32 @@
+# -*- coding: utf-8 -*-
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from webhook.config import webhook_settings
+from webhook.models import Event
+
+
+class WebhookSerializer(serializers.Serializer):
+    code = serializers.CharField(help_text=_("webhook编码"), max_length=255, required=True)
+    name = serializers.CharField(help_text=_("webhook名称"), max_length=255, required=True)
+    endpoint = serializers.URLField(help_text=_("webhook endpoint"), max_length=255, required=True)
+    token = serializers.CharField(help_text=_("webhook token"), max_length=255, required=False)
+    extra_info = serializers.JSONField(help_text=_("额外扩展信息"), required=False)
+
+
+class WebhookConfigsSerializer(serializers.Serializer):
+    webhooks = serializers.ListField(help_text=_("webhook列表"), child=WebhookSerializer(), required=True)
+
+
+class WebhookWithEventsSerializer(WebhookSerializer):
+    events = serializers.ListField(help_text=_("webhook事件列表"), required=True)
+
+    def validate_events(self, events: list):
+        not_support_events = set(events) - set(Event.objects.all_events() + [webhook_settings.ALL_EVENTS_KEY])
+        if not_support_events:
+            raise serializers.ValidationError(_(f"校验失败，events中包含不支持的事件类型, 不支持事件类型: {not_support_events}"))
+        return events
+
+
+class WebhookConfigsWithEventsSerializer(serializers.Serializer):
+    webhooks = serializers.ListField(help_text=_("webhook列表"), child=WebhookWithEventsSerializer(), required=True)

bkflow_django_webhook-1.0.0/webhook/handlers.py

@@ -0,0 +1,103 @@
+# -*- coding: utf-8 -*-
+import abc
+import uuid
+from typing import Iterable, List
+
+from django.db.models import Q
+
+from webhook.base_models import Event, Scope, Webhook
+from webhook.config import webhook_settings
+from webhook.models import History, Subscription
+from webhook.models import Webhook as WebHookModel
+from webhook.requester import RequestConfig, Requester, RequestResult
+from webhook.signals import (
+    post_event_broadcast_signal,
+    post_send_request_signal,
+    pre_event_broadcast_signal,
+    pre_send_request_signal,
+)
+
+
+class HandlerInterface(abc.ABC):
+    @abc.abstractmethod
+    def handle(self, *args, **kwargs):
+        pass
+
+
+class EventHandler(HandlerInterface):
+    def __init__(self, event: Event):
+        self.event = event
+
+    def handle(self, scopes: List[Scope], *args, **kwargs) -> None:
+        self._broadcast(scopes)
+
+    def _broadcast(self, scopes: List[Scope], *args, **kwargs):
+        pre_event_broadcast_signal.send(sender=self.__class__, event=self.event, scopes=scopes)
+
+        subscription_query = Q()
+        for scope in scopes:
+            subscription_query |= Q(
+                scope_type=scope.type,
+                scope_code=scope.code,
+                event_code__in=[self.event.code, webhook_settings.ALL_EVENTS_KEY],
+            )
+        subscriptions = (
+            list(
+                Subscription.objects.filter(subscription_query).values_list("webhook_code", "scope_type", "scope_code")
+            )
+            if scopes
+            else []
+        )
+        webhook_query = Q()
+        for webhook_code, scope_type, scope_code in subscriptions:
+            webhook_query |= Q(code=webhook_code, scope_type=scope_type, scope_code=scope_code)
+        webhooks = (
+            [Webhook.from_orm(webhook) for webhook in WebHookModel.objects.filter(webhook_query)]
+            if subscriptions
+            else []
+        )
+
+        # TODO: 优化支持异步实现
+        self._sync_webhooker_handle(webhooks=webhooks)
+
+        post_event_broadcast_signal.send(sender=self.__class__, event=self.event, scopes=scopes)
+
+    def _sync_webhooker_handle(self, webhooks: Iterable[Webhook], *args, **kwargs):
+        for webhook in webhooks:
+            webhooker = Webhooker(webhook)
+            webhooker.handle(event=self.event, *args, **kwargs)
+
+
+class Webhooker(HandlerInterface):
+    def __init__(self, webhook: Webhook):
+        self.webhook = webhook
+
+    def handle(self, event: Event, *args, **kwargs) -> RequestResult:
+        return self._request(event=event, *args, **kwargs)
+
+    def _request(self, event: Event, *args, **kwargs):
+        pre_send_request_signal.send(sender=self.__class__, webhook=self.webhook, event=event)
+        delivery_id = kwargs.pop("delivery_id", uuid.uuid4().hex)
+        headers = kwargs.pop("headers", {})
+        request_config = RequestConfig(
+            url=self.webhook.endpoint, data={"event": event.dict(), "delivery_id": delivery_id}, headers=headers
+        )
+        request_result: RequestResult = Requester(config=request_config).request()
+        history_extra_info = {
+            "request": request_config.dict(),
+            "response": request_result.json_response(),
+            **kwargs.get("history_extra_info", {}),
+        }
+        # TODO: 可配置
+        History.objects.create(
+            webhook_code=self.webhook.code,
+            event_code=event.code,
+            success=request_result.ok,
+            status_code=request_result.response_status_code,
+            delivery_id=delivery_id,
+            scope_type=self.webhook.scope_type,
+            scope_code=self.webhook.scope_code,
+            extra_info=history_extra_info,
+        )
+        post_send_request_signal.send(sender=self.__class__, webhook=self.webhook, event=event)
+        return request_result

bkflow_django_webhook-1.0.0/webhook/management/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

bkflow_django_webhook-1.0.0/webhook/management/commands/__init__.py

@@ -0,0 +1 @@
+# -*- coding: utf-8 -*-

bkflow_django_webhook-1.0.0/webhook/management/commands/sync_webhook_events.py

@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+from os import path
+
+import yaml
+from django.core.management.base import BaseCommand
+
+from webhook.models import Event
+
+
+class Command(BaseCommand):
+    """
+    python manage.py sync_webhook_events base_path filename
+    """
+
+    def add_arguments(self, parser):
+        parser.add_argument("base_path", type=str, help="base_path")
+        parser.add_argument("filename", type=str, help="filename")
+
+    def handle(self, *args, **kwargs):
+        base_path = kwargs["base_path"]
+        filename = kwargs["filename"]
+        self.stdout.write(f"sync events and scopes with filename: {filename}")
+
+        with open(path.join(base_path, filename), "r", encoding="utf-8") as f:
+            webhook_config = yaml.load(f, Loader=yaml.FullLoader)
+
+        # sync events
+        events = webhook_config.get("events", [])
+        for event in events:
+            if not event.get("code"):
+                self.stdout.error(f"event code is required, event: {event}")
+                continue
+            Event.objects.get_or_create(
+                code=event["code"], name=event["name"], description=event.get("description", "")
+            )

bkflow_django_webhook-1.0.0/webhook/migrations/0001_initial.py

@@ -0,0 +1,80 @@
+# Generated by Django 3.2.15 on 2023-07-15 17:58
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = []
+
+    operations = [
+        migrations.CreateModel(
+            name="Event",
+            fields=[
+                ("code", models.CharField(max_length=255, primary_key=True, serialize=False)),
+                ("name", models.CharField(max_length=255)),
+                ("description", models.TextField(blank=True, null=True)),
+                ("info", models.JSONField(blank=True, null=True)),
+            ],
+        ),
+        migrations.CreateModel(
+            name="Webhook",
+            fields=[
+                ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+                ("code", models.CharField(max_length=255)),
+                ("name", models.CharField(max_length=255)),
+                ("endpoint", models.URLField()),
+                ("scope_type", models.CharField(max_length=64)),
+                ("scope_code", models.CharField(max_length=64)),
+                ("token", models.CharField(blank=True, max_length=255, null=True)),
+                ("extra_info", models.JSONField(blank=True, null=True)),
+            ],
+            options={
+                "unique_together": {("scope_type", "scope_code", "code")},
+            },
+        ),
+        migrations.CreateModel(
+            name="Subscription",
+            fields=[
+                ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+                ("webhook_code", models.CharField(db_index=True, max_length=255)),
+                ("event_code", models.CharField(db_index=True, max_length=255)),
+                ("scope_type", models.CharField(max_length=64)),
+                ("scope_code", models.CharField(max_length=64)),
+            ],
+            options={
+                "index_together": {("scope_type", "scope_code", "event_code", "webhook_code")},
+            },
+        ),
+        migrations.CreateModel(
+            name="Scope",
+            fields=[
+                ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+                ("type", models.CharField(max_length=64)),
+                ("code", models.CharField(max_length=64)),
+            ],
+            options={
+                "unique_together": {("type", "code")},
+            },
+        ),
+        migrations.CreateModel(
+            name="History",
+            fields=[
+                ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
+                ("webhook_code", models.CharField(db_index=True, max_length=255)),
+                ("event_code", models.CharField(db_index=True, max_length=255)),
+                ("created_at", models.DateTimeField(auto_now_add=True, db_index=True)),
+                ("success", models.BooleanField()),
+                ("status_code", models.IntegerField(blank=True, default=None, null=True)),
+                ("delivery_id", models.CharField(db_index=True, max_length=64)),
+                ("scope_type", models.CharField(max_length=64)),
+                ("scope_code", models.CharField(max_length=64)),
+                ("extra_info", models.JSONField(blank=True, null=True)),
+            ],
+            options={
+                "index_together": {("scope_type", "scope_code", "webhook_code", "event_code")},
+            },
+        ),
+    ]

bkflow_django_webhook-1.0.0/webhook/migrations/0002_auto_20240202_1148.py

@@ -0,0 +1,25 @@
+# Generated by Django 3.2.15 on 2024-02-02 03:48
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('webhook', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='history',
+            options={'ordering': ['-id']},
+        ),
+        migrations.AlterModelOptions(
+            name='subscription',
+            options={'ordering': ['-id']},
+        ),
+        migrations.AlterModelOptions(
+            name='webhook',
+            options={'ordering': ['-id']},
+        ),
+    ]

bkflow_django_webhook-1.0.0/webhook/models.py

@@ -0,0 +1,136 @@
+# -*- coding: utf-8 -*-
+from typing import Dict, List
+
+from django.db import models, transaction
+from django.db.models import Q
+
+from webhook.base_models import Scope as ScopeBaseModel
+from webhook.base_models import Webhook as WebhookBaseModel
+
+
+class WebhookManager(models.Manager):
+    def apply_scope_webhooks(self, scope: ScopeBaseModel, webhooks: List[WebhookBaseModel]):
+        """
+        apply the given webhooks to the specified scope
+        need to guarantee the scope in webhooks is the same as the scope parameter
+        """
+        codes = set([webhook.code for webhook in webhooks])
+        with transaction.atomic():
+            if not Scope.objects.filter(type=scope.type, code=scope.code).exists():
+                Scope.objects.create(type=scope.type, code=scope.code)
+            # delete ones
+            self.filter(scope_type=scope.type, scope_code=scope.code).exclude(code__in=codes).delete()
+
+            # create or update ones
+            existing_webhook_info = self.filter(scope_type=scope.type, scope_code=scope.code).values("code", "id")
+            existing_webhook_code2id = {info["code"]: info["id"] for info in existing_webhook_info}
+            create_ones = [
+                Webhook(**webhook.dict()) for webhook in webhooks if webhook.code not in existing_webhook_code2id
+            ]
+            self.bulk_create(create_ones)
+            # update ones
+            update_ones = [
+                Webhook(id=existing_webhook_code2id[webhook.code], **webhook.dict())
+                for webhook in webhooks
+                if webhook.code in existing_webhook_code2id
+            ]
+            self.bulk_update(update_ones, fields=["name", "endpoint", "token", "extra_info"])
+
+
+class Webhook(models.Model):
+    code = models.CharField(max_length=255)
+    name = models.CharField(max_length=255)
+    endpoint = models.URLField()
+    scope_type = models.CharField(max_length=64)
+    scope_code = models.CharField(max_length=64)
+    token = models.CharField(max_length=255, null=True, blank=True)
+    extra_info = models.JSONField(null=True, blank=True)
+
+    objects = WebhookManager()
+
+    class Meta:
+        unique_together = ("scope_type", "scope_code", "code")
+        ordering = ["-id"]
+
+
+class History(models.Model):
+    webhook_code = models.CharField(max_length=255, db_index=True)
+    event_code = models.CharField(max_length=255, db_index=True)
+    created_at = models.DateTimeField(auto_now_add=True, db_index=True)
+    success = models.BooleanField()
+    status_code = models.IntegerField(null=True, blank=True, default=None)
+    delivery_id = models.CharField(max_length=64, db_index=True)
+    scope_type = models.CharField(max_length=64)
+    scope_code = models.CharField(max_length=64)
+    extra_info = models.JSONField(null=True, blank=True)  # request、response、time
+
+    class Meta:
+        index_together = ("scope_type", "scope_code", "webhook_code", "event_code")
+        ordering = ["-id"]
+
+
+class SubscriptionManager(models.Manager):
+    def apply_scope_subscriptions(self, scope: ScopeBaseModel, subscription_configs: Dict[str, List[str]]):
+        """
+        subscription_configs: {webhook_code: [event_code]}
+        """
+        with transaction.atomic():
+            flatten_subscription_configs = [
+                (webhook_code, event_code)
+                for webhook_code, event_codes in subscription_configs.items()
+                for event_code in event_codes
+            ]
+            query_filter = Q()
+            for webhook_code, event_code in flatten_subscription_configs:
+                query_filter |= Q(webhook_code=webhook_code, event_code=event_code)
+
+            # delete
+            self.filter(scope_type=scope.type, scope_code=scope.code).exclude(query_filter).delete()
+
+            # create
+            existing_ones = self.filter(scope_type=scope.type, scope_code=scope.code).values_list(
+                "webhook_code", "event_code"
+            )
+            create_configs = set(flatten_subscription_configs) - set(existing_ones)
+            create_ones = [
+                Subscription(
+                    scope_type=scope.type, scope_code=scope.code, webhook_code=webhook_code, event_code=event_code
+                )
+                for webhook_code, event_code in create_configs
+            ]
+            self.bulk_create(create_ones)
+
+
+class Subscription(models.Model):
+    webhook_code = models.CharField(max_length=255, db_index=True)
+    event_code = models.CharField(max_length=255, db_index=True)
+    scope_type = models.CharField(max_length=64)
+    scope_code = models.CharField(max_length=64)
+
+    objects = SubscriptionManager()
+
+    class Meta:
+        index_together = ("scope_type", "scope_code", "event_code", "webhook_code")
+        ordering = ["-id"]
+
+
+class EventManager(models.Manager):
+    def all_events(self) -> list:
+        return list(self.all().values_list("code", flat=True))
+
+
+class Event(models.Model):
+    code = models.CharField(max_length=255, primary_key=True)
+    name = models.CharField(max_length=255)
+    description = models.TextField(null=True, blank=True)
+    info = models.JSONField(null=True, blank=True)
+
+    objects = EventManager()
+
+
+class Scope(models.Model):
+    type = models.CharField(max_length=64)
+    code = models.CharField(max_length=64)
+
+    class Meta:
+        unique_together = ("type", "code")

bkflow_django_webhook-1.0.0/webhook/requester/__init__.py

@@ -0,0 +1,3 @@
+# -*- coding: utf-8 -*-
+
+from .base import RequestConfig, Requester, RequestResult  # noqa

bkflow_django_webhook-1.0.0/webhook/requester/base.py

@@ -0,0 +1,80 @@
+# -*- coding: utf-8 -*-
+import logging
+
+import requests
+from pydantic import BaseModel
+
+logger = logging.getLogger(__name__)
+
+JSON_CONTENT_TYPE = "application/json"
+FORM_CONTENT_TYPE = "application/x-www-form-urlencoded"
+
+
+class RequestConfig(BaseModel):
+    url: str
+    method: str = "post"
+    content_type: str = JSON_CONTENT_TYPE
+    headers: dict = None
+    data: dict = {}  # convert to json when content_type is JSON_CONTENT_TYPE
+    verify: bool = False
+    timeout: int = None
+
+    @staticmethod
+    def _gen_default_headers(content_type):
+        return {"Content-Type": content_type}
+
+    def __init__(self, **kwargs):
+        kwargs["headers"] = {
+            **self._gen_default_headers(content_type=kwargs.get("content_type", JSON_CONTENT_TYPE)),
+            **kwargs.get("headers", {}),
+        }
+        super().__init__(**kwargs)
+
+    def dict(self, **kwargs):
+        result = super().dict(**kwargs)
+        content_type = result.pop("content_type")
+        if content_type == JSON_CONTENT_TYPE:
+            result["json"] = result.pop("data")
+        return result
+
+
+class RequestResult(BaseModel):
+    result: bool
+    response: requests.Response = None
+    exe_data: str = None
+
+    class Config:
+        arbitrary_types_allowed = True
+
+    @property
+    def ok(self):
+        return self.result and self.response.ok
+
+    @property
+    def response_status_code(self):
+        if self.response is None:
+            return None
+        return self.response.status_code
+
+    def json_response(self):
+        if self.response is None:
+            return None
+        try:
+            return self.response.json()
+        except Exception as e:
+            logger.exception(f"[RequestResult.json_response error] {e}")
+            return self.response.text
+
+
+class Requester:
+    def __init__(self, config: RequestConfig):
+        self.request_config = config
+
+    def request(self, *args, **kwargs) -> RequestResult:
+        try:
+            response = requests.request(**self.request_config.dict())
+        except Exception as e:
+            logger.exception(f"[Requester.request error] {e}")
+            return RequestResult(result=False, exe_data=str(e))
+        else:
+            return RequestResult(result=True, response=response)

bkflow_django_webhook-1.0.0/webhook/signals/__init__.py

@@ -0,0 +1,10 @@
+# -*- coding: utf-8 -*-
+
+from django.dispatch import Signal
+
+event_broadcast_signal = Signal(providing_args=["scopes", "extra_info"])
+
+pre_event_broadcast_signal = Signal(providing_args=["event", "scope"])
+post_event_broadcast_signal = Signal(providing_args=["event", "scope"])
+pre_send_request_signal = Signal(providing_args=["webhook", "event"])
+post_send_request_signal = Signal(providing_args=["webhook", "event"])

bkflow_django_webhook-1.0.0/webhook/signals/handlers.py

@@ -0,0 +1,15 @@
+# -*- coding: utf-8 -*-
+from typing import List, Tuple, Union
+
+from django.dispatch import receiver
+
+from webhook.api import event_broadcast
+from webhook.base_models import Event, Scope
+from webhook.signals import event_broadcast_signal
+
+
+@receiver(event_broadcast_signal)
+def handle_event_broadcast(
+    sender: Union[Event, str], scopes: List[Union[Scope, Tuple[str, str]]], extra_info: dict = None, **kwargs
+):
+    event_broadcast(event=sender, scopes=scopes, extra_info=extra_info)