forward-netbox 2.9.2__tar.gz → 2.9.4__tar.gz

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: forward-netbox
-Version: 2.9.2
+Version: 2.9.4
 Summary: NetBox plugin to sync Forward data into NetBox via built-in NQE queries
 License: MIT
 License-File: LICENSE
@@ -29,7 +29,7 @@ Provides-Extra: routing
 Provides-Extra: validity
 Requires-Dist: httpx (>0.26,<0.29)
 Requires-Dist: netbox-cisco-aci (==0.4.0) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "aci" or extra == "integrations")
-Requires-Dist: netbox-dlm (>=0.4.1,<0.10.0) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "dlm" or extra == "integrations")
+Requires-Dist: netbox-dlm (>=0.4.1,<0.11.0) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "dlm" or extra == "integrations")
 Requires-Dist: netbox-peering-manager (==0.3.0) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "peering" or extra == "integrations")
 Requires-Dist: netbox-routing (==0.4.3) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "routing" or extra == "integrations")
 Requires-Dist: netbox-validity (==3.5.2) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "validity" or extra == "integrations")
@@ -108,15 +108,17 @@ See the
 
 ## Release Compatibility
 
-The `2.9.2` release candidate requires NetBox `4.6.x` (>= `4.6.5`, tested on `4.6.8`) and `netbox-branching` `1.1.x` (tested on `1.1.2`). Expand for the published release history and candidate notes.
+The `2.9.4` release candidate requires NetBox `4.6.x` (>= `4.6.5`, tested on `4.6.8`) and `netbox-branching` `1.1.x` (tested on `1.1.2`). Expand for the published release history and candidate notes.
 
 <details>
 <summary>Release compatibility history</summary>
 
 | Plugin Release | NetBox Version | Status |
 | --- | --- | --- |
-| `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Release candidate; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
-| `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Current release; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |
+| `v2.9.4` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Release candidate; Fix: every operator diagnostic and remediation is now reachable in the NetBox GUI - the uncovered-device cleanup 2.9.3 shipped as a command flag is a gated button, a wedged sync has a recovery control, ingestion issues say which of them block the baseline, the two device sets the panel counted but would not list are listable in full, a device that refuses a manual delete explains which records refuse it and which of those this plugin can release, and the support bundle carries all of it so a problem we have to look at arrives with its evidence. |
+| `v2.9.3` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Current release; Feature: **uncovered devices can be cleaned up**, not only counted - `forward_device_scope_reconciliation_audit --prune-uncovered` deletes devices this sync created that Forward no longer reports at all. The orphan prune never reached them: it acts on devices claimed by the run that produced the current result, so a deployment whose orphan count reads zero had no way to clear a growing uncovered count. Only devices the census marks absent are eligible, and only once their absence has outlasted the quarantine - a device Forward still reports without an include tag is a scoping decision and is never deleted, and neither is one this sync did not create. Dry run unless `--apply` is passed. Fix: **OSPF interface rows converge instead of drifting forever**, and the interface lookup behind MAC-address apply uses an index instead of scanning every interface in the estate. Feature: **netbox-dlm `0.10.0` is supported**. No migration; upgrade and re-run the sync. |
+| `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.3`; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
+| `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.2`; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |
 | `v2.9.0` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.1`; Feature: back up device configurations from Forward into a git data source for Validity golden-config checks, plus the runtime-validation refactor and health check it required. |
 | `v2.8.9` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.0`; Fix: a sync deletes a device only on the same quarantined evidence the operator prune requires, and never stages a delete the database will refuse. |
 | `v2.8.8` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.8.9`; Fix: the dependency preview runs again on deployments with optional plugins installed. |

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/README.md

@@ -64,15 +64,17 @@ See the
 
 ## Release Compatibility
 
-The `2.9.2` release candidate requires NetBox `4.6.x` (>= `4.6.5`, tested on `4.6.8`) and `netbox-branching` `1.1.x` (tested on `1.1.2`). Expand for the published release history and candidate notes.
+The `2.9.4` release candidate requires NetBox `4.6.x` (>= `4.6.5`, tested on `4.6.8`) and `netbox-branching` `1.1.x` (tested on `1.1.2`). Expand for the published release history and candidate notes.
 
 <details>
 <summary>Release compatibility history</summary>
 
 | Plugin Release | NetBox Version | Status |
 | --- | --- | --- |
-| `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Release candidate; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
-| `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Current release; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |
+| `v2.9.4` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Release candidate; Fix: every operator diagnostic and remediation is now reachable in the NetBox GUI - the uncovered-device cleanup 2.9.3 shipped as a command flag is a gated button, a wedged sync has a recovery control, ingestion issues say which of them block the baseline, the two device sets the panel counted but would not list are listable in full, a device that refuses a manual delete explains which records refuse it and which of those this plugin can release, and the support bundle carries all of it so a problem we have to look at arrives with its evidence. |
+| `v2.9.3` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Current release; Feature: **uncovered devices can be cleaned up**, not only counted - `forward_device_scope_reconciliation_audit --prune-uncovered` deletes devices this sync created that Forward no longer reports at all. The orphan prune never reached them: it acts on devices claimed by the run that produced the current result, so a deployment whose orphan count reads zero had no way to clear a growing uncovered count. Only devices the census marks absent are eligible, and only once their absence has outlasted the quarantine - a device Forward still reports without an include tag is a scoping decision and is never deleted, and neither is one this sync did not create. Dry run unless `--apply` is passed. Fix: **OSPF interface rows converge instead of drifting forever**, and the interface lookup behind MAC-address apply uses an index instead of scanning every interface in the estate. Feature: **netbox-dlm `0.10.0` is supported**. No migration; upgrade and re-run the sync. |
+| `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.3`; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
+| `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.2`; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |
 | `v2.9.0` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.1`; Feature: back up device configurations from Forward into a git data source for Validity golden-config checks, plus the runtime-validation refactor and health check it required. |
 | `v2.8.9` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.0`; Fix: a sync deletes a device only on the same quarantined evidence the operator prune requires, and never stages a delete the database will refuse. |
 | `v2.8.8` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.8.9`; Fix: the dependency preview runs again on deployments with optional plugins installed. |

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/__init__.py

@@ -5,7 +5,7 @@ class NetboxForwardConfig(PluginConfig):
     name = "forward_netbox"
     verbose_name = "Forward"
     description = "Sync Forward data into NetBox using built-in NQE queries."
-    version = "2.9.2"
+    version = "2.9.4"
     base_url = "forward"
     min_version = "4.6.5"
     max_version = "4.6.99"

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/api/views.py

@@ -928,6 +928,53 @@ class ForwardSyncViewSet(NetBoxModelViewSet):
     def tag_delete_eligible_ipam(self, request, pk):
         return self._enqueue_button_job_response(request, "tag_delete_eligible_ipam")
 
+    @extend_schema(
+        methods=["post"],
+        request=EmptySerializer(),
+        responses={
+            201: JobSerializer(),
+            202: OpenApiResponse(
+                description='{"status": "already_running", "job_id": N}'
+            ),
+        },
+    )
+    @action(detail=True, methods=["post"], url_path="recover-stuck-sync")
+    def recover_stuck_sync(self, request, pk):
+        return self._enqueue_button_job_response(request, "recover_stuck_sync")
+
+    @extend_schema(
+        methods=["post"],
+        request=EmptySerializer(),
+        responses={
+            201: JobSerializer(),
+            202: OpenApiResponse(
+                description=(
+                    '{"status": "already_running"|"blocked_by_sync_run", '
+                    '"job_id": N}'
+                )
+            ),
+        },
+    )
+    @action(detail=True, methods=["post"], url_path="prune-uncovered")
+    def prune_uncovered(self, request, pk):
+        return self._enqueue_button_job_response(request, "prune_uncovered")
+
+    @extend_schema(
+        methods=["post"],
+        request=EmptySerializer(),
+        responses={
+            201: JobSerializer(),
+            202: OpenApiResponse(
+                description='{"status": "already_running", "job_id": N}'
+            ),
+        },
+    )
+    @action(detail=True, methods=["post"], url_path="prune-stale-hardware-notices")
+    def prune_stale_hardware_notices(self, request, pk):
+        return self._enqueue_button_job_response(
+            request, "prune_stale_hardware_notices"
+        )
+
 
 class ForwardIngestionViewSet(NetBoxReadOnlyModelViewSet):
     queryset = ForwardIngestion.objects.all()

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/filtersets.py

@@ -162,11 +162,32 @@ class ForwardIngestionIssueFilterSet(BaseFilterSet):
         choices=ForwardIngestionPhaseChoices,
         null_value=None,
     )
+    # "Which of these actually stops the baseline" was answerable only by
+    # running a command against the ingestion. Same predicate as readiness.
+    blocking = django_filters.BooleanFilter(
+        method="filter_blocking",
+        label="Blocks baseline readiness",
+    )
 
     class Meta:
         model = ForwardIngestionIssue
         fields = ("phase", "model", "timestamp", "exception", "message")
 
+    def filter_blocking(self, queryset, name, value):
+        from django.db.models import Q
+
+        from .utilities.ingestion_issues import blocking_issue_q
+
+        if value is None:
+            return queryset
+        # Matches the column: a row whose ingestion promoted its baseline
+        # blocked nothing, so it is not offered as blocking however its class
+        # is scored. Filtering to rows the column calls Blocking and getting
+        # back rows it calls Promoted over is the same disagreement in the
+        # other direction.
+        predicate = blocking_issue_q() & Q(ingestion__baseline_ready=False)
+        return queryset.filter(predicate if value else ~predicate)
+
     def search(self, queryset, name, value):
         if not value.strip():
             return queryset

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/forms.py

@@ -6,6 +6,7 @@ from netbox.forms import NetBoxModelForm
 from utilities.datetime import local_now
 from utilities.forms import add_blank_choice
 from utilities.forms import ConfirmationForm
+from utilities.forms import FilterForm
 from utilities.forms import get_field_value
 from utilities.forms.fields import CommentField
 from utilities.forms.rendering import FieldSet
@@ -18,12 +19,14 @@ from utilities.forms.widgets import NumberWithOptions
 from .choices import forward_configured_models
 from .choices import FORWARD_OPTIONAL_MODELS
 from .choices import ForwardDiffFallbackModeChoices
+from .choices import ForwardIngestionPhaseChoices
 from .choices import ForwardSourceDeploymentChoices
 from .choices import ForwardSourceStatusChoices
 from .choices import ForwardSyncStatusChoices
 from .exceptions import ForwardConnectivityError
 from .exceptions import ForwardSyncError
 from .models import ForwardDriftPolicy
+from .models import ForwardIngestionIssue
 from .models import ForwardNQEMap
 from .models import ForwardSource
 from .models import ForwardSync
@@ -1210,9 +1213,11 @@ class ForwardSyncForm(NetBoxModelForm):
         label="Use fast first-baseline load",
         help_text=(
             "Exact-version, first-full-snapshot opt-in for an empty NetBox. "
-            "It omits branch review, per-row audit evidence, and branch rollback; "
-            "run `python manage.py forward_fast_baseline_preflight --sync <id>` "
-            "and verify a database backup before enabling it."
+            "It omits branch review, per-row audit evidence, and branch "
+            "rollback, so a bad first load cannot be unwound - verify a "
+            "database backup before enabling it. Only use it on a NetBox with "
+            "no inventory of its own, and leave it off for every run after the "
+            "first."
         ),
     )
     require_fast_baseline_eligibility = forms.BooleanField(
@@ -2065,3 +2070,40 @@ class ForwardDriftPolicyBulkEditForm(NetBoxModelBulkEditForm):
     enabled = forms.NullBooleanField(required=False, label="Enabled")
     model = ForwardDriftPolicy
     fields = ("enabled",)
+
+
+class ForwardIngestionIssueFilterForm(FilterForm):
+    """Filter tab for the ingestion-issue list.
+
+    The plugin's other list views carry no filter form, so this one is here for
+    a specific reason: `blocking` is the only field on the page that separates
+    the handful of rows holding a baseline back from the several hundred that
+    are noise, and a filter reachable only by hand-editing the query string is
+    not reachable. `forward_blocker_audit` was the previous answer.
+    """
+
+    model = ForwardIngestionIssue
+    fieldsets = (
+        FieldSet("q", "blocking"),
+        FieldSet("phase", "exception", name="Failure"),
+    )
+    q = forms.CharField(required=False, label="Search")
+    blocking = forms.NullBooleanField(
+        required=False,
+        label="Blocks baseline readiness",
+        widget=forms.Select(
+            choices=(
+                ("", "---------"),
+                ("true", "Blocking only"),
+                ("false", "Non-blocking only"),
+            )
+        ),
+    )
+    phase = forms.MultipleChoiceField(
+        choices=ForwardIngestionPhaseChoices,
+        required=False,
+    )
+    # No field for the NetBox model: `model` is FilterForm's own attribute for
+    # the model being filtered, and a form field of that name shadows it. The
+    # search box already matches `model` (see the filterset's `search`).
+    exception = forms.CharField(required=False, label="Exception")

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/jobs.py

@@ -516,8 +516,8 @@ def _enqueue_post_sync_overlays(
             exception_type(exc),
         )
         raise ForwardOwnershipDispatchError(
-            "Ownership reconciliation is durable but could not be enqueued; "
-            "run forward_stuck_job_recover --apply to redispatch it."
+            "Ownership reconciliation is durable but could not be enqueued. "
+            "Use Recover stuck sync on this sync's page to redispatch it."
         ) from exc
 
 
@@ -864,6 +864,135 @@ def _validate_forwardsync_work(job):
         raise
 
 
+def _recover_stuck_sync_work(job):
+    """Recover one sync wedged by a worker that died mid-run.
+
+    An RQ worker killed during a merge leaves the sync `MERGING` with no live
+    execution behind it, and nothing times out. The classification and the
+    recovery already existed and were reachable only from a command, so the
+    only person who could unstick a sync was one with shell access - and the
+    failure that causes it is announced in a job log, where a shell is exactly
+    what the reader does not have.
+    """
+    from .utilities.stuck_recovery import classify_stuck_sync
+    from .utilities.stuck_recovery import recover_stuck_sync
+
+    sync = ForwardSync.objects.get(pk=job.object_id)
+    verdict = classify_stuck_sync(sync)
+    if verdict is None:
+        # Not an error: the operator clicked a button offered by a report that
+        # has since gone stale, or another recovery got there first.
+        job.data = {
+            "recovered": False,
+            "reason": "nothing-stuck",
+            "detail": (
+                "This sync has no wedged job to recover. It either recovered on "
+                "its own or another recovery already ran."
+            ),
+        }
+        job.save(update_fields=["data"])
+        return
+
+    result = recover_stuck_sync(sync, verdict, user=job.user)
+    job.data = {
+        "recovered": True,
+        "action": verdict.get("action"),
+        "reason": verdict.get("reason"),
+        "result": result,
+    }
+    job.save(update_fields=["data"])
+
+
+def _prune_uncovered_devices_work(job, *, include_quarantined=False):
+    """Run reviewed uncovered-device pruning for a JobRunner-managed sync job.
+
+    Same shape as the orphan prune, and the same reason for the flag: it
+    arrives only from the operator button, and only when that operator ticked
+    the box beside a named list of held devices. Nothing schedules this, so
+    there is no unattended path that could override the quarantine.
+    """
+    from .utilities.scope_reconciliation import compute_scope_reconciliation
+    from .utilities.scope_reconciliation import EmptyForwardScopeError
+    from .utilities.scope_reconciliation import prune_uncovered_devices
+    from .utilities.scope_reconciliation import ScopeCensusUnavailableError
+    from .utilities.scope_reconciliation import ScopeShrinkGuardError
+
+    sync = ForwardSync.objects.get(pk=job.object_id)
+    try:
+        report = compute_scope_reconciliation(sync)
+        result = prune_uncovered_devices(
+            sync,
+            report=report,
+            include_quarantined=include_quarantined,
+        )
+        job.data = {
+            "pruned_device_count": result.get("pruned_device_count", 0),
+            "pruned_object_count": result.get("pruned_object_count", 0),
+            "out_of_scope_sample": result.get("out_of_scope_sample", []),
+            "ownership_blocked_device_count": result.get(
+                "ownership_blocked_device_count", 0
+            ),
+            "protected_device_count": result.get("protected_device_count", 0),
+            # WHICH model refused, not just how many devices were refused. The
+            # orphan prune has carried this since 2.5.5 and this one shipped
+            # without it, so a device held by another plugin's rows - ten
+            # netbox_routing BGP peers on its addresses, in the case that found
+            # this - reported a bare count and named nothing to act on.
+            "protected_by_model": result.get("protected_by_model", {}),
+            "quarantine_held_device_count": result.get(
+                "quarantine_held_device_count", 0
+            ),
+            "quarantine_overridden_device_count": result.get(
+                "quarantine_overridden_device_count", 0
+            ),
+            "quarantine_required_runs": result.get("quarantine_required_runs"),
+            "quarantine_required_hours": result.get("quarantine_required_hours"),
+        }
+        job.save(update_fields=["data"])
+    except ScopeCensusUnavailableError as exc:
+        # Not a failure to hide behind a class name: without the census nothing
+        # is known about WHY these devices are uncovered, and absent is the only
+        # eligible reason. The message says exactly that.
+        job.data = {"error": str(exc), "error_type": exception_type(exc)}
+        job.save(update_fields=["data"])
+        logger.error(
+            "Forward uncovered pruning refused without a census (%s).",
+            exception_type(exc),
+        )
+        raise
+    except ScopeShrinkGuardError as exc:
+        # Counts and a percentage only, never a device name, so the operator
+        # gets the reason rather than a class name.
+        job.data = {
+            "error": str(exc),
+            "error_type": exception_type(exc),
+        }
+        job.save(update_fields=["data"])
+        logger.error(
+            "Forward uncovered pruning refused a shrinking scope (%s).",
+            exception_type(exc),
+        )
+        raise
+    except EmptyForwardScopeError as exc:
+        job.data = {
+            "error": safe_operation_failure("Forward uncovered pruning", exc),
+            "error_type": exception_type(exc),
+        }
+        job.save(update_fields=["data"])
+        logger.error(
+            "Forward uncovered pruning rejected an empty scope (%s).",
+            exception_type(exc),
+        )
+        raise
+    except Exception as exc:
+        job.data = {
+            "error": safe_operation_failure("Forward uncovered pruning", exc),
+            "error_type": exception_type(exc),
+        }
+        job.save(update_fields=["data"])
+        raise
+
+
 def _prune_forward_orphans_work(job, *, include_quarantined=False):
     """Run reviewed orphan pruning for a JobRunner-managed sync job.
 
@@ -1989,6 +2118,32 @@ class VirtualParentReconciliationJob(ForwardJobRunner):
         _link_forward_vsys_parents_work(self.job, *args, **kwargs)
 
 
+class RecoverStuckSyncJob(ForwardJobRunner):
+    """Operator-triggered recovery of a sync wedged by a dead worker."""
+
+    class Meta:
+        # Byte-identical to BUTTON_JOB_SPECS["recover_stuck_sync"][1].
+        name = "recover stuck sync"
+
+    def run(self, *args, **kwargs):
+        _recover_stuck_sync_work(self.job)
+
+
+class PruneUncoveredDevicesJob(ForwardJobRunner):
+    """Reviewed uncovered-device prune runner used by the HTML action."""
+
+    class Meta:
+        # Byte-identical to BUTTON_JOB_SPECS["prune_uncovered"][1]: the overlap
+        # guard's exact-name arm depends on it.
+        name = "prune uncovered devices"
+
+    def run(self, *args, **kwargs):
+        extra = {}
+        if "include_quarantined" in kwargs:
+            extra["include_quarantined"] = bool(kwargs["include_quarantined"])
+        _prune_uncovered_devices_work(self.job, **extra)
+
+
 class PruneOrphansJob(ForwardJobRunner):
     """Reviewed orphan-prune runner used by the HTML and REST actions."""
 

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/management/commands/forward_device_scope_reconciliation_audit.py

@@ -6,6 +6,7 @@ from django.core.management.base import CommandError
 from forward_netbox.models import ForwardSync
 from forward_netbox.utilities.scope_reconciliation import compute_scope_reconciliation
 from forward_netbox.utilities.scope_reconciliation import prune_orphan_devices
+from forward_netbox.utilities.scope_reconciliation import prune_uncovered_devices
 
 
 class Command(BaseCommand):
@@ -33,10 +34,25 @@ class Command(BaseCommand):
                 "also passed. Tagged-but-backfilled devices are never pruned."
             ),
         )
+        parser.add_argument(
+            "--prune-uncovered",
+            action="store_true",
+            help=(
+                "Delete devices this sync created that Forward no longer "
+                "reports at all. Only devices the census marks absent are "
+                "eligible: one Forward still returns without an include tag is "
+                "a scoping question and is never touched, and a device this "
+                "sync did not create is never touched. Reports what would be "
+                "deleted unless --apply is also passed."
+            ),
+        )
         parser.add_argument(
             "--apply",
             action="store_true",
-            help="With --prune-orphans, actually delete instead of dry-run.",
+            help=(
+                "With --prune-orphans or --prune-uncovered, actually delete "
+                "instead of dry-run."
+            ),
         )
         parser.add_argument(
             "--include-quarantined",
@@ -172,6 +188,46 @@ class Command(BaseCommand):
                     "Dry run: re-run with --apply to delete these devices."
                 )
 
+        if options["prune_uncovered"]:
+            owned_absence = (report.get("unmanaged") or {}).get("owned_absence") or {}
+            candidates = int(owned_absence.get("absent_from_snapshot") or 0)
+            payload["uncovered_prune_requested"] = True
+            payload["uncovered_prune_applied"] = False
+            payload["uncovered_prune_candidate_count"] = candidates
+            if not owned_absence.get("available", False):
+                # Without the census nothing is known about WHY these devices
+                # are uncovered, and "absent" is the only eligible reason.
+                payload["uncovered_prune_aborted"] = "census-unavailable"
+                payload["uncovered_prune_abort_reason"] = (
+                    "The Forward census that says whether each uncovered device "
+                    "is absent or merely untagged did not run, so eligibility "
+                    "cannot be established. Retry once Forward is reachable."
+                )
+                self.stdout.write(json.dumps(payload, indent=2, default=str))
+                raise SystemExit(2)
+            if options["apply"] and candidates:
+                result = prune_uncovered_devices(
+                    sync,
+                    report=report,
+                    allow_scope_shrink=options["allow_scope_shrink"],
+                    include_quarantined=options["include_quarantined"],
+                )
+                payload["uncovered_prune_applied"] = True
+                payload["uncovered_pruned_device_count"] = result["pruned_device_count"]
+                payload["uncovered_pruned_object_count"] = result["pruned_object_count"]
+                payload["uncovered_quarantine_held_device_count"] = result[
+                    "quarantine_held_device_count"
+                ]
+                payload["uncovered_quarantine_overridden_device_count"] = result[
+                    "quarantine_overridden_device_count"
+                ]
+            elif candidates:
+                payload["uncovered_prune_dry_run_note"] = (
+                    "Dry run: re-run with --apply to delete these devices. Only "
+                    f"the {candidates} absent from Forward are eligible; "
+                    "uncovered devices Forward still reports are not."
+                )
+
         self.stdout.write(json.dumps(payload, indent=2, default=str))
 
         if options["fail_on_drift"] and out_of_scope:

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/management/commands/forward_profile_merge.py

@@ -1,5 +1,6 @@
 """Profile real branch merges with anonymized production-shaped fixtures."""
 
+import contextlib
 import hashlib
 import json
 import logging
@@ -25,6 +26,40 @@ from forward_netbox.models import ForwardSync
 from forward_netbox.utilities.merge import merge_branch
 from forward_netbox.utilities.merge_observability import begin_merge_attempt
 from forward_netbox.utilities.merge_profiling import MergeProfileRecorder
+from forward_netbox.utilities.merge_profiling import profile_scope
+
+
+@contextlib.contextmanager
+def _changediff_save_measured():
+    """Attribute `ChangeDiff.save()` to its own scope for the merge.
+
+    The per-row cost sits inside upstream's `ObjectChange.apply`, which the
+    recorder already times as one opaque `objectchange_apply` bucket - so
+    "is branching 1.1.3's ChangeDiff saving measurable on a real merge"
+    could not be answered from a profile. Wrapping the method here rather
+    than in the merge path keeps production untouched: this is a profiling
+    command and the instrumentation belongs with the measurement.
+    """
+    from netbox_branching.models import ChangeDiff
+
+    original = ChangeDiff.save
+    calls = {"installed": True, "count": 0}
+
+    def measured(self, *args, **kwargs):
+        calls["count"] += 1
+        with profile_scope(
+            "changediff_save",
+            owner="upstream_netbox_branching",
+            rows=1,
+        ):
+            return original(self, *args, **kwargs)
+
+    ChangeDiff.save = measured
+    try:
+        yield calls
+    finally:
+        ChangeDiff.save = original
+
 
 LOG = logging.getLogger("forward_netbox.merge_profile")
 STAGE_CHUNK_SIZE = 500
@@ -40,6 +75,11 @@ FIXTURE_WEIGHTS = (
     ("ipam.ipaddress", 0.053),
     ("ipam.prefix", 0.036),
     ("dcim.site", 0.001),
+    # A tree model, so `_is_bulk_safe` (bulk_merge.py) always refuses it and
+    # the row goes through the per-object upstream fallback - the only path
+    # that reaches ChangeDiff.save(). Without it this fixture is CREATE-only
+    # and bulk-safe, so it measured everything EXCEPT the cost in question.
+    ("dcim.region", 0.002),
 )
 
 
@@ -47,6 +87,7 @@ def _allocated_counts(volume):
     counts = {model: int(volume * weight) for model, weight in FIXTURE_WEIGHTS}
     counts["dcim.device"] = max(1, counts["dcim.device"])
     counts["dcim.site"] = max(1, counts["dcim.site"])
+    counts["dcim.region"] = max(1, counts["dcim.region"])
     assigned = sum(counts.values())
     counts["dcim.interface"] += volume - assigned
     return counts
@@ -156,6 +197,7 @@ class Command(BaseCommand):
             InventoryItem,
             MACAddress,
             Manufacturer,
+            Region,
             Site,
         )
         from ipam.models import IPAddress, Prefix
@@ -267,6 +309,16 @@ class Command(BaseCommand):
             ),
         )
 
+        self._stage_rows(
+            branch,
+            request,
+            counts["dcim.region"],
+            lambda index: Region.objects.create(
+                name=f"Profile Branch Region {short} {index}",
+                slug=f"profile-branch-region-{short}-{index}",
+            ),
+        )
+
         Branch.objects.filter(pk=branch.pk).update(status=BranchStatusChoices.READY)
         branch.refresh_from_db()
         staged_changes = branch.get_unmerged_changes().count()
@@ -296,7 +348,7 @@ class Command(BaseCommand):
                 "staged_changes": staged_changes,
             }
         )
-        with recorder.activate():
+        with recorder.activate(), _changediff_save_measured() as changediff_calls:
             merge_branch(
                 ingestion,
                 user=user,
@@ -304,6 +356,10 @@ class Command(BaseCommand):
             )
         after_postgres = _postgres_snapshot()
         result = recorder.result()
+        # Recorded explicitly: an absent `changediff_save` bucket otherwise
+        # cannot be told apart from instrumentation that never installed.
+        result["changediff_save_calls"] = changediff_calls["count"]
+        result["changediff_save_instrumented"] = changediff_calls["installed"]
         result["postgres"] = _snapshot_delta(before_postgres, after_postgres)
         result["changes_per_second"] = (
             volume / result["wall_seconds"] if result["wall_seconds"] else 0.0

{forward_netbox-2.9.2 → forward_netbox-2.9.4}/forward_netbox/navigation.py

@@ -38,6 +38,15 @@ ingestion = PluginMenuItem(
     permissions=["forward_netbox.view_forwardingestion"],
 )
 
+# The route existed and nothing reached it: issues were visible only through
+# the ingestion that produced them, so "which failures are blocking anything,
+# anywhere" had no page at all and was answered by a command.
+ingestion_issue = PluginMenuItem(
+    link="plugins:forward_netbox:forwardingestionissue_list",
+    link_text=_("Ingestion Issues"),
+    permissions=["forward_netbox.view_forwardingestionissue"],
+)
+
 validation_run = PluginMenuItem(
     link="plugins:forward_netbox:forwardvalidationrun_list",
     link_text=_("Validation Runs"),
@@ -82,7 +91,7 @@ menu = PluginMenu(
     label="Forward",
     icon_class="mdi mdi-cloud-sync",
     groups=(
-        ("Data Sync", (source, sync, ingestion, validation_run)),
+        ("Data Sync", (source, sync, ingestion, ingestion_issue, validation_run)),
         ("Analysis", (device_analysis,)),
         ("Configuration", (nqe_map, drift_policy)),
     ),