ipfabric_netbox 3.1.2__tar.gz

ipfabric_netbox-3.1.2/PKG-INFO

@@ -0,0 +1,88 @@
+Metadata-Version: 2.1
+Name: ipfabric_netbox
+Version: 3.1.2
+Summary: NetBox plugin to sync IP Fabric data into NetBox
+Home-page: https://gitlab.com/ip-fabric/integrations/ipfabric-netbox-sync
+License: MIT
+Keywords: netbox,ipfabric,plugin,sync
+Author: Solution Architecture
+Author-email: solution.architecture@ipfabric.io
+Requires-Python: >=3.10,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: ipfabric (>=6.6.4)
+Requires-Dist: netutils
+Project-URL: Bug Tracker, https://gitlab.com/ip-fabric/integrations/ipfabric-netbox-sync/-/issues
+Project-URL: Repository, https://gitlab.com/ip-fabric/integrations/ipfabric-netbox-sync
+Description-Content-Type: text/markdown
+
+# IP Fabric Netbox Plugin
+
+## IP Fabric
+
+IP Fabric is a vendor-neutral network assurance platform that automates the
+holistic discovery, verification, visualization, and documentation of
+large-scale enterprise networks, reducing the associated costs and required
+resources whilst improving security and efficiency.
+
+It supports your engineering and operations teams, underpinning migration and
+transformation projects. IP Fabric will revolutionize how you approach network
+visibility and assurance, security assurance, automation, multi-cloud
+networking, and trouble resolution.
+
+**Integrations or scripts should not be installed directly on the IP Fabric VM unless directly communicated from the
+IP Fabric Support or Solution Architect teams.  Any action on the Command-Line Interface (CLI) using the root, osadmin,
+or autoboss account may cause irreversible, detrimental changes to the product and can render the system unusable.**
+
+## Overview
+
+This plugin allows the integration and data synchronization between IP Fabric and NetBox.
+
+The plugin uses IP Fabric collect network data utilizing the [IP Fabric Python SDK](https://gitlab.com/ip-fabric/integrations/python-ipfabric). This plugin relies on helpful features in NetBox like [Staged Changes](https://netboxlabs.com/docs/netbox/en/stable/plugins/development/staged-changes/) and [Background Tasks](https://netboxlabs.com/docs/netbox/en/stable/plugins/development/background-tasks/) to make the job of bringing in data to NetBox easier.
+
+- Multiple IP Fabric Sources
+- Transform Maps
+- Scheduled Synchronization
+- Diff Visualization
+
+## NetBox Compatibility
+
+| Netbox Version | Plugin Version |
+| -------------- | -------------- |
+| 3.4            | <=1.0.11       |
+| 3.5            | <=1.0.11       |
+| 3.6            | <=1.0.11       |
+| 3.7            | >=2.0.0        |
+| 4.0            | >=3.0.0        |
+| 4.0.1          | >=3.0.1        |
+| 4.1.0          | >=3.1.0        |
+| 4.1.5          | >=3.1.1        |
+
+## Screenshots
+
+![Source](docs/images/user_guide/source_sync.png)
+
+![Snapshots](docs/images/user_guide/snapshot_detail.png)
+
+![Transform Maps](docs/images/user_guide/tm_edit_hostname.png)
+
+![Ingesion](docs/images/user_guide/ingestion_detail.png)
+
+![Diff](docs/images/user_guide/branch_changes_update_diff.png)
+
+## Documentation
+
+Full documentation for this plugin can be found at [IP Fabric Docs](https://docs.ipfabric.io/main/integrations/netbox/).
+
+- User Guide
+- Administrator Guide
+
+## Contributing
+
+If you would like to contribute to this plugin, please see the [CONTRIBUTING.md](CONTRIBUTING.md) file.
+

ipfabric_netbox-3.1.2/README.md

@@ -0,0 +1,64 @@
+# IP Fabric Netbox Plugin
+
+## IP Fabric
+
+IP Fabric is a vendor-neutral network assurance platform that automates the
+holistic discovery, verification, visualization, and documentation of
+large-scale enterprise networks, reducing the associated costs and required
+resources whilst improving security and efficiency.
+
+It supports your engineering and operations teams, underpinning migration and
+transformation projects. IP Fabric will revolutionize how you approach network
+visibility and assurance, security assurance, automation, multi-cloud
+networking, and trouble resolution.
+
+**Integrations or scripts should not be installed directly on the IP Fabric VM unless directly communicated from the
+IP Fabric Support or Solution Architect teams.  Any action on the Command-Line Interface (CLI) using the root, osadmin,
+or autoboss account may cause irreversible, detrimental changes to the product and can render the system unusable.**
+
+## Overview
+
+This plugin allows the integration and data synchronization between IP Fabric and NetBox.
+
+The plugin uses IP Fabric collect network data utilizing the [IP Fabric Python SDK](https://gitlab.com/ip-fabric/integrations/python-ipfabric). This plugin relies on helpful features in NetBox like [Staged Changes](https://netboxlabs.com/docs/netbox/en/stable/plugins/development/staged-changes/) and [Background Tasks](https://netboxlabs.com/docs/netbox/en/stable/plugins/development/background-tasks/) to make the job of bringing in data to NetBox easier.
+
+- Multiple IP Fabric Sources
+- Transform Maps
+- Scheduled Synchronization
+- Diff Visualization
+
+## NetBox Compatibility
+
+| Netbox Version | Plugin Version |
+| -------------- | -------------- |
+| 3.4            | <=1.0.11       |
+| 3.5            | <=1.0.11       |
+| 3.6            | <=1.0.11       |
+| 3.7            | >=2.0.0        |
+| 4.0            | >=3.0.0        |
+| 4.0.1          | >=3.0.1        |
+| 4.1.0          | >=3.1.0        |
+| 4.1.5          | >=3.1.1        |
+
+## Screenshots
+
+![Source](docs/images/user_guide/source_sync.png)
+
+![Snapshots](docs/images/user_guide/snapshot_detail.png)
+
+![Transform Maps](docs/images/user_guide/tm_edit_hostname.png)
+
+![Ingesion](docs/images/user_guide/ingestion_detail.png)
+
+![Diff](docs/images/user_guide/branch_changes_update_diff.png)
+
+## Documentation
+
+Full documentation for this plugin can be found at [IP Fabric Docs](https://docs.ipfabric.io/main/integrations/netbox/).
+
+- User Guide
+- Administrator Guide
+
+## Contributing
+
+If you would like to contribute to this plugin, please see the [CONTRIBUTING.md](CONTRIBUTING.md) file.

ipfabric_netbox-3.1.2/ipfabric_netbox/__init__.py

@@ -0,0 +1,42 @@
+import requests
+from django.db import connection
+from netbox.plugins import PluginConfig
+
+
+class NetboxIPFabricConfig(PluginConfig):
+    name = "ipfabric_netbox"
+    verbose_name = "NetBox IP Fabric SoT Plugin"
+    description = "Sync IP Fabric into NetBox"
+    version = "3.1.1"
+    base_url = "ipfabric"
+    min_version = "4.1"
+
+    def ready(self):
+        super().ready()
+        try:
+            from ipfabric_netbox.signals import ipfabric_netbox_init
+
+            all_tables = connection.introspection.table_names()
+            if all(
+                item in all_tables
+                for item in [
+                    "extras_customfield",
+                    "ipfabric_netbox_ipfabricbranch",
+                    "ipfabric_netbox_ipfabricsource",
+                ]
+            ):
+                ipfabric_netbox_init()
+            if "ipfabric_netbox_ipfabrictransformmap" in all_tables:
+                from ipfabric_netbox.models import IPFabricTransformMap
+                from ipfabric_netbox.utilities.transform_map import BuildTransformMaps
+
+                if IPFabricTransformMap.objects.count() == 0:
+                    data = requests.get(
+                        "https://gitlab.com/ip-fabric/integrations/ipfabric-netbox/-/raw/main/scripts/transform_map.json"
+                    ).json()
+                    BuildTransformMaps(data=data)
+        except Exception as e:
+            print(f"Failed to initialize IP Fabric plugin: {e}.")
+
+
+config = NetboxIPFabricConfig

ipfabric_netbox-3.1.2/ipfabric_netbox/api/__init__.py

@@ -0,0 +1,2 @@
+from .nested_serializers import *  # noqa: F401, F403
+from .serializers import *  # noqa: F401, F403

ipfabric_netbox-3.1.2/ipfabric_netbox/api/nested_serializers.py

@@ -0,0 +1,99 @@
+from drf_spectacular.types import OpenApiTypes
+from drf_spectacular.utils import extend_schema_field
+from netbox.api.fields import ContentTypeField
+from netbox.api.serializers import NetBoxModelSerializer
+from netbox.api.serializers import WritableNestedSerializer
+from rest_framework import serializers
+from users.api.serializers_.nested import NestedUserSerializer
+
+from ipfabric_netbox.models import IPFabricBranch
+from ipfabric_netbox.models import IPFabricSnapshot
+from ipfabric_netbox.models import IPFabricSource
+from ipfabric_netbox.models import IPFabricSync
+from ipfabric_netbox.models import IPFabricTransformMap
+
+# from .serializers import IPFabricSyncSerializer
+
+__all__ = (
+    "NestedIPFabricSourceSerializer",
+    "NestedIPFabricSnapshotSerializer",
+    "NestedIPFabricTransformMapSerializer",
+    "NestedIPFabricBranchSerializer",
+    "NestedIPFabricSyncSerializer",
+)
+
+
+class NestedIPFabricSourceSerializer(WritableNestedSerializer):
+    url = serializers.URLField()
+
+    class Meta:
+        model = IPFabricSource
+        fields = ["id", "url", "display", "name", "type"]
+
+
+class NestedIPFabricSnapshotSerializer(NetBoxModelSerializer):
+    source = NestedIPFabricSourceSerializer(read_only=True)
+    display = serializers.SerializerMethodField(read_only=True)
+
+    class Meta:
+        model = IPFabricSnapshot
+        fields = [
+            "id",
+            "name",
+            "source",
+            "snapshot_id",
+            "status",
+            "date",
+            "display",
+            "sites",
+        ]
+
+    @extend_schema_field(OpenApiTypes.STR)
+    def get_display(self, obj):
+        return f"{obj.name} ({obj.snapshot_id})"
+
+
+class NestedIPFabricSyncSerializer(NetBoxModelSerializer):
+    snapshot_data = NestedIPFabricSnapshotSerializer(read_only=True)
+
+    class Meta:
+        model = IPFabricSync
+        fields = [
+            "id",
+            "name",
+            "display",
+            "snapshot_data",
+            "type",
+            "status",
+            "parameters",
+            "last_synced",
+        ]
+
+
+class NestedIPFabricTransformMapSerializer(NetBoxModelSerializer):
+    target_model = ContentTypeField(read_only=True)
+
+    class Meta:
+        model = IPFabricTransformMap
+        fields = [
+            "id",
+            "source_model",
+            "target_model",
+            "status",
+        ]
+
+
+class NestedIPFabricBranchSerializer(NetBoxModelSerializer):
+    user = NestedUserSerializer(read_only=True)
+    sync = NestedIPFabricSyncSerializer(read_only=True)
+
+    class Meta:
+        model = IPFabricBranch
+        fields = [
+            "id",
+            "name",
+            "display",
+            "sync",
+            "description",
+            "user",
+        ]

ipfabric_netbox-3.1.2/ipfabric_netbox/api/serializers.py

@@ -0,0 +1,160 @@
+from core.choices import DataSourceStatusChoices
+from netbox.api.fields import ChoiceField
+from netbox.api.fields import ContentTypeField
+from netbox.api.serializers import NetBoxModelSerializer
+from rest_framework import serializers
+from users.api.serializers_.nested import NestedUserSerializer
+
+from .nested_serializers import NestedIPFabricBranchSerializer  # noqa: F401
+from .nested_serializers import NestedIPFabricSnapshotSerializer
+from .nested_serializers import NestedIPFabricSourceSerializer
+from .nested_serializers import NestedIPFabricTransformMapSerializer
+from ipfabric_netbox.models import IPFabricBranch
+from ipfabric_netbox.models import IPFabricRelationshipField
+from ipfabric_netbox.models import IPFabricSnapshot
+from ipfabric_netbox.models import IPFabricSource
+from ipfabric_netbox.models import IPFabricSync
+from ipfabric_netbox.models import IPFabricTransformField
+from ipfabric_netbox.models import IPFabricTransformMap
+
+__all__ = (
+    "IPFabricSyncSerializer",
+    "IPFabricSnapshotSerializer",
+    "IPFabricRelationshipFieldSerializer",
+    "IPFabricTransformFieldSerializer",
+    "IPFabricTransformMapSerializer",
+    "IPFabricBranchSerializer",
+    "IPFabricSourceSerializer",
+)
+
+
+class IPFabricSyncSerializer(NetBoxModelSerializer):
+    snapshot_data = NestedIPFabricSnapshotSerializer(read_only=True)
+
+    class Meta:
+        model = IPFabricSync
+        fields = [
+            "id",
+            "name",
+            "display",
+            "snapshot_data",
+            "type",
+            "status",
+            "parameters",
+            "last_synced",
+            "created",
+            "last_updated",
+        ]
+
+
+class IPFabricSnapshotSerializer(NetBoxModelSerializer):
+    source = NestedIPFabricSourceSerializer()
+    data = serializers.JSONField()
+    date = serializers.DateTimeField()
+    last_updated = serializers.DateTimeField()
+
+    class Meta:
+        model = IPFabricSnapshot
+        fields = [
+            "id",
+            "name",
+            "source",
+            "snapshot_id",
+            "status",
+            "date",
+            "display",
+            "sites",
+            "data",
+            "created",
+            "last_updated",
+        ]
+
+        extra_kwargs = {
+            "raw_data": {"write_only": True},
+        }
+
+
+class IPFabricRelationshipFieldSerializer(NetBoxModelSerializer):
+    transform_map = NestedIPFabricTransformMapSerializer(read_only=True)
+    source_model = ContentTypeField(read_only=True)
+
+    class Meta:
+        model = IPFabricRelationshipField
+        fields = [
+            "id",
+            "transform_map",
+            "source_model",
+            "target_field",
+            "coalesce",
+            "template",
+        ]
+
+
+class IPFabricTransformMapSerializer(NetBoxModelSerializer):
+    target_model = ContentTypeField(read_only=True)
+
+    class Meta:
+        model = IPFabricTransformMap
+        fields = [
+            "id",
+            "source_model",
+            "target_model",
+            "status",
+            "created",
+            "last_updated",
+        ]
+
+
+class IPFabricTransformFieldSerializer(NetBoxModelSerializer):
+    transform_map = NestedIPFabricTransformMapSerializer(read_only=True)
+
+    class Meta:
+        model = IPFabricTransformField
+        fields = [
+            "id",
+            "transform_map",
+            "source_field",
+            "target_field",
+            "coalesce",
+            "template",
+        ]
+
+
+class IPFabricBranchSerializer(NetBoxModelSerializer):
+    user = NestedUserSerializer(read_only=True)
+    sync = IPFabricSyncSerializer(read_only=True)
+
+    class Meta:
+        model = IPFabricBranch
+        fields = [
+            "id",
+            "name",
+            "display",
+            "sync",
+            "description",
+            "user",
+            "created",
+            "last_updated",
+        ]
+
+
+class IPFabricSourceSerializer(NetBoxModelSerializer):
+    status = ChoiceField(choices=DataSourceStatusChoices)
+    url = serializers.URLField()
+
+    class Meta:
+        model = IPFabricSource
+        fields = [
+            "id",
+            "url",
+            "display",
+            "name",
+            "type",
+            "status",
+            "last_synced",
+            "description",
+            "comments",
+            "parameters",
+            "created",
+            "last_updated",
+        ]

ipfabric_netbox-3.1.2/ipfabric_netbox/api/urls.py

@@ -0,0 +1,21 @@
+# api/urls.py
+from netbox.api.routers import NetBoxRouter
+
+from ipfabric_netbox.api.views import IPFabricBranchViewSet
+from ipfabric_netbox.api.views import IPFabricRelationshipFieldiewSet
+from ipfabric_netbox.api.views import IPFabricSnapshotViewSet
+from ipfabric_netbox.api.views import IPFabricSourceViewSet
+from ipfabric_netbox.api.views import IPFabricSyncViewSet
+from ipfabric_netbox.api.views import IPFabricTransformFieldiewSet
+from ipfabric_netbox.api.views import IPFabricTransformMapViewSet
+
+
+router = NetBoxRouter()
+router.register("source", IPFabricSourceViewSet)
+router.register("snapshot", IPFabricSnapshotViewSet)
+router.register("transform-map", IPFabricTransformMapViewSet)
+router.register("sync", IPFabricSyncViewSet)
+router.register("branch", IPFabricBranchViewSet)
+router.register("transform-field", IPFabricTransformFieldiewSet)
+router.register("relationship-field", IPFabricRelationshipFieldiewSet)
+urlpatterns = router.urls

ipfabric_netbox-3.1.2/ipfabric_netbox/api/views.py

@@ -0,0 +1,111 @@
+from django.db import transaction
+from netbox.api.viewsets import NetBoxModelViewSet
+from netbox.api.viewsets import NetBoxReadOnlyModelViewSet
+from rest_framework.decorators import action
+from rest_framework.response import Response
+from utilities.query import count_related
+
+from .serializers import IPFabricBranchSerializer
+from .serializers import IPFabricRelationshipFieldSerializer
+from .serializers import IPFabricSnapshotSerializer
+from .serializers import IPFabricSourceSerializer
+from .serializers import IPFabricSyncSerializer
+from .serializers import IPFabricTransformFieldSerializer
+from .serializers import IPFabricTransformMapSerializer
+from ipfabric_netbox.filtersets import IPFabricSnapshotFilterSet
+from ipfabric_netbox.filtersets import IPFabricSourceFilterSet
+from ipfabric_netbox.filtersets import IPFabricTransformFieldFilterSet
+from ipfabric_netbox.models import IPFabricBranch
+from ipfabric_netbox.models import IPFabricData
+from ipfabric_netbox.models import IPFabricRelationshipField
+from ipfabric_netbox.models import IPFabricSnapshot
+from ipfabric_netbox.models import IPFabricSource
+from ipfabric_netbox.models import IPFabricSync
+from ipfabric_netbox.models import IPFabricTransformField
+from ipfabric_netbox.models import IPFabricTransformMap
+
+
+class IPFabricTransformMapViewSet(NetBoxReadOnlyModelViewSet):
+    queryset = IPFabricTransformMap.objects.all()
+    serializer_class = IPFabricTransformMapSerializer
+
+
+class IPFabricTransformFieldiewSet(NetBoxReadOnlyModelViewSet):
+    queryset = IPFabricTransformField.objects.all()
+    serializer_class = IPFabricTransformFieldSerializer
+    filterset_class = IPFabricTransformFieldFilterSet
+
+
+class IPFabricRelationshipFieldiewSet(NetBoxReadOnlyModelViewSet):
+    queryset = IPFabricRelationshipField.objects.all()
+    serializer_class = IPFabricRelationshipFieldSerializer
+    filterset_class = IPFabricTransformFieldFilterSet
+
+
+class IPFabricSyncViewSet(NetBoxReadOnlyModelViewSet):
+    queryset = IPFabricSync.objects.all()
+    serializer_class = IPFabricSyncSerializer
+
+
+class IPFabricBranchViewSet(NetBoxReadOnlyModelViewSet):
+    queryset = IPFabricBranch.objects.all()
+    serializer_class = IPFabricBranchSerializer
+
+
+class IPFabricSnapshotViewSet(NetBoxModelViewSet):
+    queryset = IPFabricSnapshot.objects.all()
+    serializer_class = IPFabricSnapshotSerializer
+    filterset_class = IPFabricSnapshotFilterSet
+
+    @action(detail=True, methods=["patch", "delete"], url_path="raw")
+    def raw(self, request, pk):
+        snapshot = self.get_object()
+        if request.method == "DELETE":
+            raw_data = IPFabricData.objects.filter(snapshot_data=snapshot)
+            raw_data._raw_delete(raw_data.db)
+            return Response({"status": "success"})
+        elif request.method == "PATCH":
+            transaction.set_autocommit(False)
+            IPFabricData.objects.bulk_create(
+                [
+                    IPFabricData(
+                        snapshot_data=snapshot, data=item["data"], type=item["type"]
+                    )
+                    for item in request.data["data"]
+                ],
+                batch_size=5000,
+            )
+            transaction.commit()
+            return Response({"status": "success"})
+
+    @action(detail=True, methods=["get"], url_path="sites")
+    def sites(self, request, pk):
+        q = request.GET.get("q", None)
+        snapshot = IPFabricSnapshot.objects.get(pk=pk)
+        new_sites = {"results": []}
+        if snapshot.data:
+            sites = snapshot.data.get("sites", None)
+            num = 0
+            if sites:
+                for site in sites:
+                    if q:
+                        if q.lower() in site.lower():
+                            new_sites["results"].append(
+                                {"display": site, "name": site, "id": site}
+                            )
+                    else:
+                        new_sites["results"].append(
+                            {"display": site, "name": site, "id": site}
+                        )
+                    num += 1
+                return Response(new_sites)
+        else:
+            return Response([])
+
+
+class IPFabricSourceViewSet(NetBoxModelViewSet):
+    queryset = IPFabricSource.objects.annotate(
+        snapshot_count=count_related(IPFabricSnapshot, "source")
+    )
+    serializer_class = IPFabricSourceSerializer
+    filterset_class = IPFabricSourceFilterSet