forward-netbox 3.0.2__tar.gz → 3.0.4__tar.gz

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: forward-netbox
-Version: 3.0.2
+Version: 3.0.4
 Summary: NetBox plugin to sync Forward data into NetBox via built-in NQE queries
 License: MIT
 License-File: LICENSE
@@ -36,6 +36,7 @@ Requires-Dist: netbox-routing (>=0.5.0,<0.6.0) ; (python_version >= "3.12" and p
 Requires-Dist: netbox-validity (==3.6.0) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "validity" or extra == "integrations")
 Requires-Dist: netboxlabs-netbox-branching (>=1.2.0,<1.3.0) ; python_version >= "3.12" and python_version < "3.15"
 Requires-Dist: pyzipper (>=0.4.0,<0.5.0)
+Requires-Dist: scrapli (>=2022.7.30,<2026.10) ; (python_version >= "3.12" and python_version < "3.15") and (extra == "validity" or extra == "integrations")
 Project-URL: Changelog, https://github.com/forwardnetworks/forward-netbox/blob/main/CHANGELOG.md
 Project-URL: Documentation, https://github.com/forwardnetworks/forward-netbox/tree/main/docs
 Project-URL: Homepage, https://github.com/forwardnetworks/forward-netbox
@@ -109,15 +110,17 @@ See the
 
 ## Release Compatibility
 
-The `3.0.2` release candidate requires NetBox `4.7.x` (tested on `4.7.1`) and `netbox-branching` `1.2.x` (tested on `1.2.0`). Expand for the published release history and candidate notes.
+The `3.0.4` release candidate requires NetBox `4.7.x` (tested on `4.7.1`) and `netbox-branching` `1.2.x` (tested on `1.2.0`). Expand for the published release history and candidate notes.
 
 <details>
 <summary>Release compatibility history</summary>
 
 | Plugin Release | NetBox Version | Status |
 | --- | --- | --- |
-| `v3.0.2` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Release candidate; Port of 2.9.13's PeeringSession crash + endpoint /32 fixes, plus the 3.0.1 upgrade-runtime override fix |
-| `v3.0.1` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Current release; Release candidate; Fix: every fix shipped on the 2.9.x line as 2.9.7 through 2.9.11 now ships on NetBox 4.7 - the 3.x line had none of them. Fix: renaming a device's site in Forward moves the existing device instead of creating a duplicate, devices are matched case-insensitively the way NetBox identifies them, and the new Scope Reconciliation action Merge site-relabel duplicates repairs pairs already created (keeps the older device at the site Forward reports; held pairs are listed with a reason and never pruned). Fix: scope membership is by name, so no in-scope device is called out of scope. Feature: duplicates and other silent problems are no longer silent - a Health check, a sync-page banner, a Scope Reconciliation card and a job-log line. Fix: config backup reaches the repository through the proxy NetBox uses, no longer writes credentials to the worker log, fails when the remote refuses the branch, and names the cause of a failure in the job, Health and the support bundle. Fix: the fast baseline, COPY/SQL and set-based merge run on any subset of the validated plugins. Fix: routing-policy scope counts every device that holds a definition, pending removals are what the next sync removes, and Scope Reconciliation offers Remove out-of-scope routing policy (an allowlisted, gated operator action). Fix: the SDK repository lookup read attributes the forward-sdk RepositoryQuery does not have, so committed-query lookups by path failed against a real Forward; forward-sdk is now 0.1.20. Fix: protection against deleting a sync or ingestion while a job runs, and support bundles and log exports no longer carry operator values such as include-tag names. Feature: the support bundle carries a diagnostics section, and a live query-drift check flags a published query whose parameter list no longer matches this release. Feature: Auto merge stops for review when a run is far larger than the sync has been - only with Auto merge on, only after the sync has finished a run before, and only for a model that stages at least 10,000 changes at 3x the table's size (or 10x the typical earlier run); a held run stays staged, with the ingestion's Merge and Delete actions as Merge anyway and Discard. Feature: Health and the support bundle show Forward's row count next to NetBox's per model, the last three ingestions with job and merge status, statistics and merged changes by model and action, and a BGP peer duplicate count, and flag an abnormal run, a baseline that never completed and a NetBox table far smaller than Forward's. Feature: the config backup folder is configurable (nested folders allowed), Health shows the Validity device_config_path for it, and the last backup reports rows, written, unchanged, unmapped, managed devices with no collected configuration and files in the folder. Fix: the Include unmanaged configuration checkbox never persisted from the source form. Migrations 0053 through 0057 and 0060 adopt the 2.9.x schema history. Org-backed deployments: run Publish Bundled Queries after upgrading, because the interface and IPv4 address queries changed their parameter lists (a stale published copy fails every execution against it until republished) and the three routing-policy queries gained an output column. |
+| `v3.0.4` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Release candidate; OSPF instance name clashes keep a stable per-process name instead of being skipped; Infoblox and Palo Alto endpoints are named by vendor; netbox-dlm 0.10.1 and 0.10.2 accepted; the UI gate runs on agent-browser; support bundle gains endpoint mask, site placement and fallback-reason figures; tested on NetBox 4.7.2 and netbox-branching 1.2.1 |
+| `v3.0.3` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Current release; Static routes in the global table import; site-relabel repair releases BGP address families; primary IP falls back to the management address when a tag cannot resolve; interfaces holding a VLAN from a previous site are cleared instead of refused on every sync; scrapli capped so NetBox starts with netbox-validity; OSPF instances named per VRF for netbox-routing 0.5.0; support bundle gains dependency versions, scope-report age, site mismatch, a primary-IP split and mask breakdown, and per-rule issue counts; tested on NetBox 4.7.2 and netbox-branching 1.2.1 |
+| `v3.0.2` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.3`; Port of 2.9.13's PeeringSession crash + endpoint /32 fixes, plus the 3.0.1 upgrade-runtime override fix |
+| `v3.0.1` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.2`; Release candidate; Fix: every fix shipped on the 2.9.x line as 2.9.7 through 2.9.11 now ships on NetBox 4.7 - the 3.x line had none of them. Fix: renaming a device's site in Forward moves the existing device instead of creating a duplicate, devices are matched case-insensitively the way NetBox identifies them, and the new Scope Reconciliation action Merge site-relabel duplicates repairs pairs already created (keeps the older device at the site Forward reports; held pairs are listed with a reason and never pruned). Fix: scope membership is by name, so no in-scope device is called out of scope. Feature: duplicates and other silent problems are no longer silent - a Health check, a sync-page banner, a Scope Reconciliation card and a job-log line. Fix: config backup reaches the repository through the proxy NetBox uses, no longer writes credentials to the worker log, fails when the remote refuses the branch, and names the cause of a failure in the job, Health and the support bundle. Fix: the fast baseline, COPY/SQL and set-based merge run on any subset of the validated plugins. Fix: routing-policy scope counts every device that holds a definition, pending removals are what the next sync removes, and Scope Reconciliation offers Remove out-of-scope routing policy (an allowlisted, gated operator action). Fix: the SDK repository lookup read attributes the forward-sdk RepositoryQuery does not have, so committed-query lookups by path failed against a real Forward; forward-sdk is now 0.1.20. Fix: protection against deleting a sync or ingestion while a job runs, and support bundles and log exports no longer carry operator values such as include-tag names. Feature: the support bundle carries a diagnostics section, and a live query-drift check flags a published query whose parameter list no longer matches this release. Feature: Auto merge stops for review when a run is far larger than the sync has been - only with Auto merge on, only after the sync has finished a run before, and only for a model that stages at least 10,000 changes at 3x the table's size (or 10x the typical earlier run); a held run stays staged, with the ingestion's Merge and Delete actions as Merge anyway and Discard. Feature: Health and the support bundle show Forward's row count next to NetBox's per model, the last three ingestions with job and merge status, statistics and merged changes by model and action, and a BGP peer duplicate count, and flag an abnormal run, a baseline that never completed and a NetBox table far smaller than Forward's. Feature: the config backup folder is configurable (nested folders allowed), Health shows the Validity device_config_path for it, and the last backup reports rows, written, unchanged, unmapped, managed devices with no collected configuration and files in the folder. Fix: the Include unmanaged configuration checkbox never persisted from the source form. Migrations 0053 through 0057 and 0060 adopt the 2.9.x schema history. Org-backed deployments: run Publish Bundled Queries after upgrading, because the interface and IPv4 address queries changed their parameter lists (a stale published copy fails every execution against it until republished) and the three routing-policy queries gained an output column. |
 | `v3.0.0` | NetBox `4.7.x` supported, tested on `4.7.0`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.1`; NetBox 4.7 with netbox-branching 1.2, and change control gated on what Forward saw before and after the change. |
 | `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v3.0.0`; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
 | `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.2`; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/README.md

@@ -64,15 +64,17 @@ See the
 
 ## Release Compatibility
 
-The `3.0.2` release candidate requires NetBox `4.7.x` (tested on `4.7.1`) and `netbox-branching` `1.2.x` (tested on `1.2.0`). Expand for the published release history and candidate notes.
+The `3.0.4` release candidate requires NetBox `4.7.x` (tested on `4.7.1`) and `netbox-branching` `1.2.x` (tested on `1.2.0`). Expand for the published release history and candidate notes.
 
 <details>
 <summary>Release compatibility history</summary>
 
 | Plugin Release | NetBox Version | Status |
 | --- | --- | --- |
-| `v3.0.2` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Release candidate; Port of 2.9.13's PeeringSession crash + endpoint /32 fixes, plus the 3.0.1 upgrade-runtime override fix |
-| `v3.0.1` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Current release; Release candidate; Fix: every fix shipped on the 2.9.x line as 2.9.7 through 2.9.11 now ships on NetBox 4.7 - the 3.x line had none of them. Fix: renaming a device's site in Forward moves the existing device instead of creating a duplicate, devices are matched case-insensitively the way NetBox identifies them, and the new Scope Reconciliation action Merge site-relabel duplicates repairs pairs already created (keeps the older device at the site Forward reports; held pairs are listed with a reason and never pruned). Fix: scope membership is by name, so no in-scope device is called out of scope. Feature: duplicates and other silent problems are no longer silent - a Health check, a sync-page banner, a Scope Reconciliation card and a job-log line. Fix: config backup reaches the repository through the proxy NetBox uses, no longer writes credentials to the worker log, fails when the remote refuses the branch, and names the cause of a failure in the job, Health and the support bundle. Fix: the fast baseline, COPY/SQL and set-based merge run on any subset of the validated plugins. Fix: routing-policy scope counts every device that holds a definition, pending removals are what the next sync removes, and Scope Reconciliation offers Remove out-of-scope routing policy (an allowlisted, gated operator action). Fix: the SDK repository lookup read attributes the forward-sdk RepositoryQuery does not have, so committed-query lookups by path failed against a real Forward; forward-sdk is now 0.1.20. Fix: protection against deleting a sync or ingestion while a job runs, and support bundles and log exports no longer carry operator values such as include-tag names. Feature: the support bundle carries a diagnostics section, and a live query-drift check flags a published query whose parameter list no longer matches this release. Feature: Auto merge stops for review when a run is far larger than the sync has been - only with Auto merge on, only after the sync has finished a run before, and only for a model that stages at least 10,000 changes at 3x the table's size (or 10x the typical earlier run); a held run stays staged, with the ingestion's Merge and Delete actions as Merge anyway and Discard. Feature: Health and the support bundle show Forward's row count next to NetBox's per model, the last three ingestions with job and merge status, statistics and merged changes by model and action, and a BGP peer duplicate count, and flag an abnormal run, a baseline that never completed and a NetBox table far smaller than Forward's. Feature: the config backup folder is configurable (nested folders allowed), Health shows the Validity device_config_path for it, and the last backup reports rows, written, unchanged, unmapped, managed devices with no collected configuration and files in the folder. Fix: the Include unmanaged configuration checkbox never persisted from the source form. Migrations 0053 through 0057 and 0060 adopt the 2.9.x schema history. Org-backed deployments: run Publish Bundled Queries after upgrading, because the interface and IPv4 address queries changed their parameter lists (a stale published copy fails every execution against it until republished) and the three routing-policy queries gained an output column. |
+| `v3.0.4` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Release candidate; OSPF instance name clashes keep a stable per-process name instead of being skipped; Infoblox and Palo Alto endpoints are named by vendor; netbox-dlm 0.10.1 and 0.10.2 accepted; the UI gate runs on agent-browser; support bundle gains endpoint mask, site placement and fallback-reason figures; tested on NetBox 4.7.2 and netbox-branching 1.2.1 |
+| `v3.0.3` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Current release; Static routes in the global table import; site-relabel repair releases BGP address families; primary IP falls back to the management address when a tag cannot resolve; interfaces holding a VLAN from a previous site are cleared instead of refused on every sync; scrapli capped so NetBox starts with netbox-validity; OSPF instances named per VRF for netbox-routing 0.5.0; support bundle gains dependency versions, scope-report age, site mismatch, a primary-IP split and mask breakdown, and per-rule issue counts; tested on NetBox 4.7.2 and netbox-branching 1.2.1 |
+| `v3.0.2` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.3`; Port of 2.9.13's PeeringSession crash + endpoint /32 fixes, plus the 3.0.1 upgrade-runtime override fix |
+| `v3.0.1` | NetBox `4.7.x` supported, tested on `4.7.1`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.2`; Release candidate; Fix: every fix shipped on the 2.9.x line as 2.9.7 through 2.9.11 now ships on NetBox 4.7 - the 3.x line had none of them. Fix: renaming a device's site in Forward moves the existing device instead of creating a duplicate, devices are matched case-insensitively the way NetBox identifies them, and the new Scope Reconciliation action Merge site-relabel duplicates repairs pairs already created (keeps the older device at the site Forward reports; held pairs are listed with a reason and never pruned). Fix: scope membership is by name, so no in-scope device is called out of scope. Feature: duplicates and other silent problems are no longer silent - a Health check, a sync-page banner, a Scope Reconciliation card and a job-log line. Fix: config backup reaches the repository through the proxy NetBox uses, no longer writes credentials to the worker log, fails when the remote refuses the branch, and names the cause of a failure in the job, Health and the support bundle. Fix: the fast baseline, COPY/SQL and set-based merge run on any subset of the validated plugins. Fix: routing-policy scope counts every device that holds a definition, pending removals are what the next sync removes, and Scope Reconciliation offers Remove out-of-scope routing policy (an allowlisted, gated operator action). Fix: the SDK repository lookup read attributes the forward-sdk RepositoryQuery does not have, so committed-query lookups by path failed against a real Forward; forward-sdk is now 0.1.20. Fix: protection against deleting a sync or ingestion while a job runs, and support bundles and log exports no longer carry operator values such as include-tag names. Feature: the support bundle carries a diagnostics section, and a live query-drift check flags a published query whose parameter list no longer matches this release. Feature: Auto merge stops for review when a run is far larger than the sync has been - only with Auto merge on, only after the sync has finished a run before, and only for a model that stages at least 10,000 changes at 3x the table's size (or 10x the typical earlier run); a held run stays staged, with the ingestion's Merge and Delete actions as Merge anyway and Discard. Feature: Health and the support bundle show Forward's row count next to NetBox's per model, the last three ingestions with job and merge status, statistics and merged changes by model and action, and a BGP peer duplicate count, and flag an abnormal run, a baseline that never completed and a NetBox table far smaller than Forward's. Feature: the config backup folder is configurable (nested folders allowed), Health shows the Validity device_config_path for it, and the last backup reports rows, written, unchanged, unmapped, managed devices with no collected configuration and files in the folder. Fix: the Include unmanaged configuration checkbox never persisted from the source form. Migrations 0053 through 0057 and 0060 adopt the 2.9.x schema history. Org-backed deployments: run Publish Bundled Queries after upgrading, because the interface and IPv4 address queries changed their parameter lists (a stale published copy fails every execution against it until republished) and the three routing-policy queries gained an output column. |
 | `v3.0.0` | NetBox `4.7.x` supported, tested on `4.7.0`; needs netbox-branching `1.2.x`, tested on `1.2.0` | Superseded by `v3.0.1`; NetBox 4.7 with netbox-branching 1.2, and change control gated on what Forward saw before and after the change. |
 | `v2.9.2` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v3.0.0`; Drift is measured exactly for every fetched model, the uncovered-device count says why it grows and warns when it does, and five delete and diagnostic defects are closed. |
 | `v2.9.1` | `4.6.x` (>= `4.6.5`) supported, tested on `4.6.8`; needs netbox-branching `1.1.x`, tested on `1.1.2` | Superseded by `v2.9.2`; Accepts netbox-branching 1.1.3 and records the validated runtime in the lockfile. |

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/__init__.py

@@ -5,7 +5,7 @@ class NetboxForwardConfig(PluginConfig):
     name = "forward_netbox"
     verbose_name = "Forward"
     description = "Sync Forward data into NetBox using built-in NQE queries."
-    version = "3.0.2"
+    version = "3.0.4"
     base_url = "forward"
     min_version = "4.7.0"
     max_version = "4.7.99"

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/management/commands/forward_seed_ui_harness.py

@@ -27,7 +27,7 @@ from forward_netbox.utilities.ownership import required_ownership_domains
 
 
 class Command(BaseCommand):
-    help = "Seed synthetic Forward NetBox records for the Playwright UI harness."
+    help = "Seed synthetic Forward NetBox records for the agent-browser UI harness."
 
     def add_arguments(self, parser):
         parser.add_argument(
@@ -59,7 +59,7 @@ class Command(BaseCommand):
         if os.getenv("FORWARD_UI_HARNESS_ISOLATED", "").lower() != "true":
             raise CommandError(
                 "Synthetic UI fixtures may only be seeded in the isolated "
-                "`invoke playwright-test` runtime."
+                "`invoke ui-test` runtime."
             )
         user = self._ensure_superuser(
             username=options["username"],

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/queries/forward_devices.nqe

@@ -111,8 +111,17 @@ foreach row in (
     // overlay like Avocent/Opengear (these are ordinary SNMP-GENERIC-class
     // endpoints, ungated by sync_generic_endpoints), just a manufacturer
     // signal for when SNMP returned no sysDescr/sysObjectID to read one from.
-    let isInfoblox = matches(endpointProfileName, "*infoblox*")
-    let isPaloAlto = matches(endpointProfileName, "*pano*")
+    // These appliances answer sysDescr, but with text that does not name the
+    // vendor: an Infoblox NIOS box reports `Linux <hostname> <kernel>` (so the
+    // first token read as the manufacturer made it "Linux"), and a Panorama
+    // reports `Palo Alto Networks M-Series Appliance` (first token "Palo").
+    // The vendor's enterprise OID and the profile name identify them instead.
+    let isInfoblox = matches(sysObjId, "1.3.6.1.4.1.7779.*")
+      || matches(sdLower, "*infoblox*")
+      || matches(endpointProfileName, "*infoblox*")
+    let isPaloAlto = matches(sysObjId, "1.3.6.1.4.1.25461.*")
+      || matches(sdLower, "*palo alto*")
+      || matches(endpointProfileName, "*pano*")
       || matches(endpointProfileName, "*panorama*")
     // Manufacturer: the overlay/profile match when one applies; else the
     // first token of the cleaned sysDescr; "Unknown" only when none of the
@@ -120,9 +129,9 @@ foreach row in (
     // profile is generic).
     let ep_manuf_raw = if isAvocent then "Avocent"
       else if isOpengear then "Opengear"
-      else if hasDescr then substring(replaceRegexMatches(sysDescrClean, re` .*`, ""), 0, 100)
       else if isInfoblox then "Infoblox"
       else if isPaloAlto then "Palo Alto Networks"
+      else if hasDescr then substring(replaceRegexMatches(sysDescrClean, re` .*`, ""), 0, 100)
       else "Unknown"
     let ep_manuf = if ep_manuf_raw == "" then "Unknown" else ep_manuf_raw
     let ep_manuf_slug_raw = slugify(ep_manuf)

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/queries/forward_devices_with_netbox_aliases.nqe

@@ -152,8 +152,17 @@ foreach row in (
     // overlay like Avocent/Opengear (these are ordinary SNMP-GENERIC-class
     // endpoints, ungated by sync_generic_endpoints), just a manufacturer
     // signal for when SNMP returned no sysDescr/sysObjectID to read one from.
-    let isInfoblox = matches(endpointProfileName, "*infoblox*")
-    let isPaloAlto = matches(endpointProfileName, "*pano*")
+    // These appliances answer sysDescr, but with text that does not name the
+    // vendor: an Infoblox NIOS box reports `Linux <hostname> <kernel>` (so the
+    // first token read as the manufacturer made it "Linux"), and a Panorama
+    // reports `Palo Alto Networks M-Series Appliance` (first token "Palo").
+    // The vendor's enterprise OID and the profile name identify them instead.
+    let isInfoblox = matches(sysObjId, "1.3.6.1.4.1.7779.*")
+      || matches(sdLower, "*infoblox*")
+      || matches(endpointProfileName, "*infoblox*")
+    let isPaloAlto = matches(sysObjId, "1.3.6.1.4.1.25461.*")
+      || matches(sdLower, "*palo alto*")
+      || matches(endpointProfileName, "*pano*")
       || matches(endpointProfileName, "*panorama*")
     // Manufacturer: the overlay/profile match when one applies; else the
     // first token of the cleaned sysDescr; "Unknown" only when none of the
@@ -161,9 +170,9 @@ foreach row in (
     // profile is generic).
     let ep_manuf_raw = if isAvocent then "Avocent"
       else if isOpengear then "Opengear"
-      else if hasDescr then substring(replaceRegexMatches(sysDescrClean, re` .*`, ""), 0, 100)
       else if isInfoblox then "Infoblox"
       else if isPaloAlto then "Palo Alto Networks"
+      else if hasDescr then substring(replaceRegexMatches(sysDescrClean, re` .*`, ""), 0, 100)
       else "Unknown"
     let ep_manuf = if ep_manuf_raw == "" then "Unknown" else ep_manuf_raw
     let ep_manuf_slug_raw = slugify(ep_manuf)

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/templates/forward_netbox/forwardsync_scope_reconciliation.html

@@ -489,7 +489,11 @@
           </p>
           <ul class="mb-0">
             {% for entry in site_relabel_pairs.held_by_reason %}
-              <li><strong>{{ entry.count }}</strong> - {{ entry.remedy|default:entry.reason }}</li>
+              <li><strong>{{ entry.count }}</strong> - {{ entry.remedy|default:entry.reason }}
+                {% if entry.names %}
+                  <div class="text-muted small">{{ entry.names|join:", " }}{% if entry.more_names %}, +{{ entry.more_names }} {% trans "more" %}{% endif %}</div>
+                {% endif %}
+              </li>
             {% endfor %}
           </ul>
         {% endif %}

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/tests/test_apply_engine.py

@@ -174,6 +174,66 @@ class ForwardBulkOrmApplyEngineTest(TestCase):
         self.assertEqual(Site.objects.filter(name="Bad").count(), 0)
         runner._record_issue.assert_called_once()
 
+    def test_isolate_bulk_objects_indexes_a_rescued_shard_once(self):
+        # A shard that fell back to row-by-row saves must cost the search cache
+        # one update, not one per row: every "Search cache update" is a queued
+        # job, and a large shard used to queue thousands.
+        from netbox.search import deferred
+
+        from forward_netbox.utilities.apply_engine_bulk import _isolate_bulk_objects
+
+        flushed = []
+        sites = [Site(name=f"Rescued {i}", slug=f"rescued-{i}") for i in range(25)]
+
+        with patch.object(
+            deferred, "_flush", lambda batch, using: flushed.append(dict(batch))
+        ):
+            with self.captureOnCommitCallbacks(execute=True):
+                _isolate_bulk_objects(
+                    Site, sites, "create", self._runner(), "dcim.site"
+                )
+
+        self.assertEqual(Site.objects.filter(slug__startswith="rescued-").count(), 25)
+        self.assertEqual(len(flushed), 1)
+        self.assertEqual(len(flushed[0]), 25)
+
+    def test_isolate_bulk_objects_does_not_index_a_row_that_failed(self):
+        from netbox.search import deferred
+
+        from forward_netbox.utilities.apply_engine_bulk import _isolate_bulk_objects
+
+        Site.objects.create(name="Taken", slug="taken")
+        flushed = []
+        good = Site(name="Good", slug="good")
+        bad = Site(name="Bad", slug="taken")  # duplicate slug -> IntegrityError
+
+        with patch.object(
+            deferred, "_flush", lambda batch, using: flushed.append(dict(batch))
+        ):
+            with self.captureOnCommitCallbacks(execute=True):
+                _isolate_bulk_objects(
+                    Site, [good, bad], "create", self._runner(), "dcim.site"
+                )
+
+        indexed_pks = {pk for batch in flushed for (_type, pk) in batch}
+        self.assertEqual(indexed_pks, {good.pk})
+
+    def test_isolate_bulk_objects_restores_the_search_handler(self):
+        from netbox.search.backends import search_backend
+
+        from forward_netbox.utilities.apply_engine_bulk import _isolate_bulk_objects
+
+        before = search_backend.caching_handler
+        _isolate_bulk_objects(
+            Site,
+            [Site(name="Restore", slug="restore")],
+            "create",
+            self._runner(),
+            "dcim.site",
+        )
+        self.assertEqual(search_backend.caching_handler, before)
+        self.assertNotIn("caching_handler", vars(search_backend))
+
     def test_bulk_orm_creates_and_updates_manufacturers(self):
         self.sync.parameters["enable_bulk_orm"] = True
         self.sync.save(update_fields=["parameters"])

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/tests/test_bulk_merge.py

@@ -4516,8 +4516,19 @@ class Phase4BulkStageTest(CleanTransactionTestCase):
             request_id=merge_request_id,
         )
         # The member is audited as CREATE plus the internal deferred-LAG UPDATE;
-        # the two audit rows still count as one logical branch change.
-        self.assertEqual(audits.count(), 4)
+        # the two audit rows still count as one logical branch change. Deleting
+        # the cable also clears the parent's cached cable fields (`cable`,
+        # `_path`, `cable_end`), which netbox-branching 1.2.1 records as its own
+        # UPDATE: five rows, still all attributed to the invoking user.
+        self.assertEqual(audits.count(), 5)
+        self.assertEqual(
+            audits.filter(
+                changed_object_type=interface_type,
+                changed_object_id=main_parent.pk,
+                action="update",
+            ).count(),
+            2,
+        )
         self.assertEqual(
             set(audits.values_list("user_id", flat=True)), {invoking_user.pk}
         )
@@ -4530,7 +4541,7 @@ class Phase4BulkStageTest(CleanTransactionTestCase):
         )
         self.assertEqual(
             branch.get_merged_changes().filter(pk__in=audits).count(),
-            4,
+            5,
         )
         self.assertEqual(
             list(

forward_netbox-3.0.4/forward_netbox/tests/test_bundle_triage_diagnostics.py

@@ -0,0 +1,208 @@
+# The support bundle answers the questions a customer report actually asks.
+#
+# A customer's report this cycle could not be matched to its bundle: a three-day-old
+# scope report sat beside a same-day sync, the primary-IP count had no split to
+# reconcile against the device list, 147 issues had to be tallied by hand to find
+# that 130 were one rule, and a dependency release that stopped NetBox starting left
+# no version in the file. These pin the figures that close each gap, and that the
+# export still carries counts rather than names.
+import json
+
+from dcim.models import Device
+from dcim.models import DeviceRole
+from dcim.models import DeviceType
+from dcim.models import Manufacturer
+from dcim.models import Site
+from django.test import TestCase
+from extras.models import Tag
+
+from forward_netbox.models import ForwardDeviceIdentity
+from forward_netbox.models import ForwardIngestion
+from forward_netbox.models import ForwardIngestionIssue
+from forward_netbox.models import ForwardSource
+from forward_netbox.models import ForwardSync
+from forward_netbox.utilities.bundle_diagnostics import bundle_diagnostics
+from forward_netbox.utilities.export_redaction import export_safe_payload
+from forward_netbox.views import _environment_bundle_payload
+from forward_netbox.views import _ingestion_issue_bundle_payload
+from forward_netbox.views import _primary_ip_bundle_payload
+from forward_netbox.views import _scope_reconciliation_bundle_payload
+
+
+class BundleTriageDiagnosticsTest(TestCase):
+    def setUp(self):
+        self.site = Site.objects.create(name="Secret Street Site", slug="secret")
+        mfr = Manufacturer.objects.create(name="Cisco", slug="cisco")
+        self.device_type = DeviceType.objects.create(
+            manufacturer=mfr, model="M1", slug="m1"
+        )
+        self.role = DeviceRole.objects.create(name="firewall", slug="firewall")
+        source = ForwardSource.objects.create(
+            name="src", url="https://fwd.example.invalid"
+        )
+        self.sync = ForwardSync.objects.create(name="sync", source=source)
+
+    def _device(self, name, *, managed=True, tag=None):
+        device = Device.objects.create(
+            name=name,
+            site=self.site,
+            device_type=self.device_type,
+            role=self.role,
+            status="active",
+        )
+        if managed:
+            ForwardDeviceIdentity.objects.create(
+                sync=self.sync, device=device, source_device_key=name
+            )
+        if tag:
+            device.tags.add(Tag.objects.get_or_create(name=tag, slug=tag.lower())[0])
+        return device
+
+    def test_the_primary_ip_count_is_split_so_it_can_be_reconciled(self):
+        self._device("fw-uncovered", tag="forward-uncovered")
+        self._device("fw-mgmt", tag="Mgmt_Vl211")
+        self._device("fw-plain")
+        self._device("unmanaged", managed=False)
+
+        payload = _primary_ip_bundle_payload(self.sync)
+
+        self.assertEqual(payload["sync_devices"], 3)
+        self.assertEqual(payload["without_primary_ip"], 3)
+        split = payload["without_primary_ip_split"]
+        self.assertEqual(split["tagged_uncovered"], 1)
+        self.assertEqual(split["with_a_mgmt_tag"], 1)
+        self.assertEqual(split["with_no_ip_at_all"], 3)
+        self.assertEqual(split["with_an_interface_ip"], 0)
+        # The figure on the device list covers devices this sync does not own.
+        self.assertEqual(payload["netbox_devices_without_primary_ip"], 4)
+
+    def test_primary_addresses_are_counted_by_mask_and_host_masks_by_role(self):
+        from ipam.models import IPAddress
+
+        loopback = self._device("fw-loopback")
+        subnet = self._device("fw-subnet")
+        for device, address in ((loopback, "10.0.0.1/32"), (subnet, "10.0.1.1/24")):
+            ip = IPAddress.objects.create(address=address)
+            Device.objects.filter(pk=device.pk).update(primary_ip4=ip)
+
+        masks = _primary_ip_bundle_payload(self.sync)["primary_ip4_prefix_lengths"]
+
+        self.assertEqual(
+            {row["prefix_length"]: row["devices"] for row in masks["by_prefix_length"]},
+            {32: 1, 24: 1},
+        )
+        self.assertEqual(
+            masks["host_mask_by_role"], [{"role": "firewall", "devices": 1}]
+        )
+
+    def test_interface_addresses_are_counted_by_mask_and_the_default_site_is_listed(
+        self,
+    ):
+        from dcim.models import Interface
+        from ipam.models import IPAddress
+
+        default = Site.objects.create(name="Default", slug="default")
+        endpoint = self._device("endpoint-1")
+        Device.objects.filter(pk=endpoint.pk).update(site=default)
+        switch = self._device("switch-1")
+        for device, address in ((endpoint, "10.0.0.9/32"), (switch, "10.0.1.1/24")):
+            interface = Interface.objects.create(
+                device=device, name="Gi0/1", type="1000base-t"
+            )
+            IPAddress.objects.create(address=address, assigned_object=interface)
+
+        payload = _primary_ip_bundle_payload(self.sync)
+
+        masks = payload["interface_ip4_prefix_lengths"]
+        self.assertEqual(
+            {
+                row["prefix_length"]: row["addresses"]
+                for row in masks["by_prefix_length"]
+            },
+            {32: 1, 24: 1},
+        )
+        self.assertEqual(
+            masks["host_mask_by_role"], [{"role": "firewall", "addresses": 1}]
+        )
+        placement = payload["site_placement"]
+        self.assertEqual(placement["at_default_site"], 1)
+        self.assertEqual(placement["at_default_site_by_role"], {"firewall": 1})
+        self.assertEqual(placement["at_default_site_without_primary_ip"], 1)
+        self.assertNotIn("endpoint-1", json.dumps(payload))
+
+    def test_issues_are_tallied_by_model_exception_and_rule_over_every_row(self):
+        ingestion = ForwardIngestion.objects.create(sync=self.sync)
+        for _ in range(3):
+            ForwardIngestionIssue.objects.create(
+                ingestion=ingestion,
+                model="dcim.interface",
+                exception="ValidationError",
+                message="Merge failed",
+                raw_data={"validation_rules": ["untagged-vlan-outside-device-site"]},
+            )
+        ForwardIngestionIssue.objects.create(
+            ingestion=ingestion,
+            model="dcim.site",
+            exception="ForwardDependencySkipError",
+            message="skipped",
+            raw_data={},
+        )
+
+        summary = _ingestion_issue_bundle_payload(ingestion)["summary"]
+
+        self.assertEqual(summary["counted"], 4)
+        self.assertEqual(
+            summary["by_model_and_exception"][0],
+            {"model": "dcim.interface", "exception": "ValidationError", "count": 3},
+        )
+        self.assertEqual(
+            summary["by_validation_rule"],
+            [
+                {
+                    "model": "dcim.interface",
+                    "rule": "untagged-vlan-outside-device-site",
+                    "count": 3,
+                }
+            ],
+        )
+
+    def test_the_environment_names_the_dependency_versions(self):
+        deps = _environment_bundle_payload()["dependency_versions"]
+
+        for name in ("scrapli", "scrapli-netconf", "forward-sdk", "django"):
+            self.assertIn(name, deps)
+        # A package that is not installed reads None, never an error.
+        self.assertTrue(all(v is None or isinstance(v, str) for v in deps.values()))
+        self.assertTrue(deps["django"])
+
+    def test_a_missing_scope_report_has_no_age_rather_than_a_wrong_one(self):
+        payload = _scope_reconciliation_bundle_payload(self.sync)
+
+        self.assertIsNone(payload["generated_at"])
+        self.assertIsNone(payload["age_hours"])
+        self.assertIsNone(payload["older_than_last_sync"])
+
+    def test_routing_name_collisions_reports_whether_the_plugin_is_there(self):
+        out = bundle_diagnostics(self.sync)
+
+        self.assertEqual(out["errors"], {})
+        collisions = out["routing_name_collisions"]
+        self.assertIn("installed", collisions)
+        if collisions["installed"]:
+            self.assertEqual(collisions["duplicate_name_groups"], 0)
+
+    def test_the_new_figures_export_without_names(self):
+        self._device("customer-firewall-01", tag="forward-uncovered")
+        text = json.dumps(
+            export_safe_payload(
+                {
+                    "primary_ip": _primary_ip_bundle_payload(self.sync),
+                    "diagnostics": bundle_diagnostics(self.sync),
+                    "environment": _environment_bundle_payload(),
+                }
+            ),
+            default=str,
+        )
+
+        self.assertNotIn("customer-firewall-01", text)
+        self.assertNotIn("Secret Street", text)

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/tests/test_endpoints_import.py

@@ -421,6 +421,32 @@ class EndpointBranchIncludeScopeTest(SimpleTestCase):
             params = _default_query_parameters(filename)
             self.assertIs(params["sync_generic_endpoints"], False, filename)
 
+    def test_vendor_overlays_name_the_manufacturer_before_the_sysdescr_token(self):
+        # An Infoblox box reports `Linux <host> <kernel>` and a Panorama reports
+        # `Palo Alto Networks ...`, so reading the first sysDescr token first
+        # made them "Linux" and "Palo". The enterprise OID and the profile name
+        # come before that token.
+        for filename in (
+            "forward_devices.nqe",
+            "forward_devices_with_netbox_aliases.nqe",
+        ):
+            endpoint_branch = _read_query(filename).split("network.endpoints", 1)[1]
+            self.assertIn('matches(sysObjId, "1.3.6.1.4.1.7779.*")', endpoint_branch)
+            self.assertIn('matches(sysObjId, "1.3.6.1.4.1.25461.*")', endpoint_branch)
+            chain = endpoint_branch.split("let ep_manuf_raw", 1)[1].split(
+                "let ep_manuf =", 1
+            )[0]
+            self.assertLess(
+                chain.index('then "Infoblox"'),
+                chain.index("else if hasDescr"),
+                filename,
+            )
+            self.assertLess(
+                chain.index('then "Palo Alto Networks"'),
+                chain.index("else if hasDescr"),
+                filename,
+            )
+
     def test_device_query_declares_toggle_default_off(self):
         for filename in (
             "forward_devices.nqe",

{forward_netbox-3.0.2 → forward_netbox-3.0.4}/forward_netbox/tests/test_merge_rule_rejection.py

@@ -398,6 +398,18 @@ class FieldScopedValidationRuleTests(SimpleTestCase):
             ["untagged-vlan-outside-device-site"], diagnosis["validation_rules"]
         )
 
+    def test_an_ospf_name_collision_is_named(self):
+        # netbox-routing 0.5.0 unique (device, name): reported on a customer sync
+        # as an unrecognised rule until this slug existed.
+        diagnosis = structured_failure_diagnosis(
+            ValidationError(
+                "Name must be unique per device Only a single empty name is "
+                "permitted per device"
+            )
+        )
+        self.assertEqual(["name-not-unique-per-device"], diagnosis["validation_rules"])
+        self.assertNotIn("unrecognized_validation_rules", diagnosis)
+
     def test_untagged_vlan_without_mode_is_named(self):
         diagnosis = structured_failure_diagnosis(self.UNTAGGED_VLAN_MODE)
         self.assertEqual(