sc-napalm 0.2.0__tar.gz

sc_napalm-0.2.0/.gitignore

@@ -0,0 +1,7 @@
+.env
+.venv
+__pycache__
+*.egg-info
+*.swp
+*.python-version
+dist/

sc_napalm-0.2.0/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.4
+Name: sc-napalm
+Version: 0.2.0
+Summary: Add your description here
+Requires-Python: >=3.8
+Requires-Dist: gitpython>=3.1.45
+Requires-Dist: lxml>=6.0.2
+Requires-Dist: napalm-srl>=1.0.5
+Requires-Dist: napalm-sros>=1.0.2
+Requires-Dist: napalm>=5.1.0
+Requires-Dist: pynautobot>=2.1.1
+Requires-Dist: python-decouple>=3.8
+Description-Content-Type: text/markdown
+
+# sc-napalm
+
+This repo is for building custom [NAPALM](https://napalm.readthedocs.io/en/latest/) "getters" to pull operational data from network devices, as well as
+override existing NAPALM getters with custom code if needed. At the moment only one custom getter is implemented: `get_inventory`. This getter
+pulls model and serial numbers of device components: fans, psus, and optics.
+
+What's cool about NAPALM is that if you nest your custom drivers under `custom_napalm` as is done in this project, you can install your custom drivers
+in the same virtual environment as the main NAPALM package, and they will override NAPALM's core drivers. This allows us to leverage NAPALM in a pretty 
+seamless way - by that I mean applications that leverage NAPALM (like Nautobot or Nornir) can be easily altered to use this code instead.
+
+
+## Using sc-napalm
+The package comes with a cli script called `sc-napalm-get` that will run a particular getter against a particular device and output the results
+to your terminal.
+
+Right now to use this code you must do the following things:
+1. [Install uv](https://docs.astral.sh/uv/getting-started/installation/)
+2. Clone the repo onto your local machine, then do a local development install of the repo into a uv virtual environment.
+```
+git clone https://scinet.supercomputing.org:8443/automation/sc25/sc-napalm.git
+cd sc-napalm
+uv run pip install -e .
+```
+3. Copy `sample_env` file into `.env` and fill out `.env` with the credentials you need. Note that you could also specify these credentials
+when you run the script, or in your local environment. The script prefers manual input, then `.env` then environment values.
+
+4. Run the script with `uv run sc-napalm-get`.
+```
+amylieb@m-n76107kqmh sc-napalm % uv run sc-napalm-get -h
+usage: sc-napalm-get [-h] [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL} | -L {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--logfile LOGFILE] [--username USERNAME]
+                     [--password PASSWORD]
+                     device {iosxr,nxos,junos,sros,srl,eos} {get_config,get_facts,get_lldp_neighbors,get_inventory}
+
+Run a specific sc_napalm "getter" against a device.
+
+positional arguments:
+  device                device hostname or IP address
+  {iosxr,nxos,junos,sros,srl,eos}
+                        The platform of this device
+  {get_config,get_facts,get_lldp_neighbors,get_inventory}
+                        The getter command to run against this device
+
+options:
+  -h, --help            show this help message and exit
+  -l, --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
+                        Set log level for sc_napalm only
+  -L, --LOG-LEVEL {DEBUG,INFO,WARNING,ERROR,CRITICAL}
+                        set global log level
+  --logfile LOGFILE     Save logging to a file (specified by name) instead of to stdout
+  --username USERNAME   Specify credentials
+  --password PASSWORD   Specify credentials
+aliebowitz@sysauto:~/src/sc-napalm$ uv run sc-napalm-get 2001:468:1f07:ff19::1d eos get_inventory
+[{'name': 'Ethernet45',
+  'part_number': 'QSFP-100G-LR4',
+  'serial_number': 'XYL252206819',
+  'subtype': 'QSFP-100G-LR4',
+  'type': 'optic'},
+ {'name': 'Ethernet46',
+  'part_number': 'QSFP-100G-LR4',
+  'serial_number': 'XYL252206822',
+  'subtype': 'QSFP-100G-LR4',
+  'type': 'optic'},
+ {'name': 'PSU 1',
+  'part_number': 'PWR-511-AC-RED',
+  'serial_number': 'EEWT2420216960',
+  'subtype': None,
+  'type': 'psu'},
+ ...
+  .....
+
+```
+
+## Developing sc-napalm
+Currently, the getters that are exposed as options in the `get` script are defined in the [base class](https://scinet.supercomputing.org:8443/automation/sc25/sc-napalm/-/blob/main/src/custom_napalm/base.py?ref_type=heads) of the custom drivers.
+Note that because all the custom classes inherit the NAPALM getters, we could easily define all the other NAPALM getters there, but I've only included ones I think are obviously useful to us.
+
+My hope is that instead of just printing out results we can write code that saves data in Nautobot, or some other place.
+This could be done with Nornir or Nautobot jobs.
+
+## To-dos
+* Decide where this code is run and how (periodically? Manually? From Nautobot? if so, how?)
+* Decide what output we want to capture (eg "show version"? "show lldp neighbor"?) and where it should go
+* Fuctions/tasks that transform output and save it appropriately
+* `get_inventory` methods for sros
+* Test/mock classes for custom getters
+

sc_napalm-0.2.0/README.md

@@ -0,0 +1,86 @@
+# sc-napalm
+
+This repo is for building custom [NAPALM](https://napalm.readthedocs.io/en/latest/) "getters" to pull operational data from network devices, as well as
+override existing NAPALM getters with custom code if needed. At the moment only one custom getter is implemented: `get_inventory`. This getter
+pulls model and serial numbers of device components: fans, psus, and optics.
+
+What's cool about NAPALM is that if you nest your custom drivers under `custom_napalm` as is done in this project, you can install your custom drivers
+in the same virtual environment as the main NAPALM package, and they will override NAPALM's core drivers. This allows us to leverage NAPALM in a pretty 
+seamless way - by that I mean applications that leverage NAPALM (like Nautobot or Nornir) can be easily altered to use this code instead.
+
+
+## Using sc-napalm
+The package comes with a cli script called `sc-napalm-get` that will run a particular getter against a particular device and output the results
+to your terminal.
+
+Right now to use this code you must do the following things:
+1. [Install uv](https://docs.astral.sh/uv/getting-started/installation/)
+2. Clone the repo onto your local machine, then do a local development install of the repo into a uv virtual environment.
+```
+git clone https://scinet.supercomputing.org:8443/automation/sc25/sc-napalm.git
+cd sc-napalm
+uv run pip install -e .
+```
+3. Copy `sample_env` file into `.env` and fill out `.env` with the credentials you need. Note that you could also specify these credentials
+when you run the script, or in your local environment. The script prefers manual input, then `.env` then environment values.
+
+4. Run the script with `uv run sc-napalm-get`.
+```
+amylieb@m-n76107kqmh sc-napalm % uv run sc-napalm-get -h
+usage: sc-napalm-get [-h] [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL} | -L {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--logfile LOGFILE] [--username USERNAME]
+                     [--password PASSWORD]
+                     device {iosxr,nxos,junos,sros,srl,eos} {get_config,get_facts,get_lldp_neighbors,get_inventory}
+
+Run a specific sc_napalm "getter" against a device.
+
+positional arguments:
+  device                device hostname or IP address
+  {iosxr,nxos,junos,sros,srl,eos}
+                        The platform of this device
+  {get_config,get_facts,get_lldp_neighbors,get_inventory}
+                        The getter command to run against this device
+
+options:
+  -h, --help            show this help message and exit
+  -l, --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
+                        Set log level for sc_napalm only
+  -L, --LOG-LEVEL {DEBUG,INFO,WARNING,ERROR,CRITICAL}
+                        set global log level
+  --logfile LOGFILE     Save logging to a file (specified by name) instead of to stdout
+  --username USERNAME   Specify credentials
+  --password PASSWORD   Specify credentials
+aliebowitz@sysauto:~/src/sc-napalm$ uv run sc-napalm-get 2001:468:1f07:ff19::1d eos get_inventory
+[{'name': 'Ethernet45',
+  'part_number': 'QSFP-100G-LR4',
+  'serial_number': 'XYL252206819',
+  'subtype': 'QSFP-100G-LR4',
+  'type': 'optic'},
+ {'name': 'Ethernet46',
+  'part_number': 'QSFP-100G-LR4',
+  'serial_number': 'XYL252206822',
+  'subtype': 'QSFP-100G-LR4',
+  'type': 'optic'},
+ {'name': 'PSU 1',
+  'part_number': 'PWR-511-AC-RED',
+  'serial_number': 'EEWT2420216960',
+  'subtype': None,
+  'type': 'psu'},
+ ...
+  .....
+
+```
+
+## Developing sc-napalm
+Currently, the getters that are exposed as options in the `get` script are defined in the [base class](https://scinet.supercomputing.org:8443/automation/sc25/sc-napalm/-/blob/main/src/custom_napalm/base.py?ref_type=heads) of the custom drivers.
+Note that because all the custom classes inherit the NAPALM getters, we could easily define all the other NAPALM getters there, but I've only included ones I think are obviously useful to us.
+
+My hope is that instead of just printing out results we can write code that saves data in Nautobot, or some other place.
+This could be done with Nornir or Nautobot jobs.
+
+## To-dos
+* Decide where this code is run and how (periodically? Manually? From Nautobot? if so, how?)
+* Decide what output we want to capture (eg "show version"? "show lldp neighbor"?) and where it should go
+* Fuctions/tasks that transform output and save it appropriately
+* `get_inventory` methods for sros
+* Test/mock classes for custom getters
+

sc_napalm-0.2.0/pyproject.toml

@@ -0,0 +1,45 @@
+[project]
+name = "sc-napalm"
+version = "0.2.0"
+description = "Add your description here"
+readme = "README.md"
+requires-python = ">=3.8"
+dependencies = [
+    "gitpython>=3.1.45",
+    "lxml>=6.0.2",
+    "napalm>=5.1.0",
+    "napalm-srl>=1.0.5",
+    "napalm-sros>=1.0.2",
+    "pynautobot>=2.1.1",
+    "python-decouple>=3.8",
+]
+
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project.scripts]
+sc-napalm-get = "custom_napalm.get:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/custom_napalm"]
+
+#[tool.hatch.build]
+#include = [
+#    "src/custom_napalm/junos/*.yml",
+#    "src/custom_napalm/nxos/*.tpl",
+#]
+
+[dependency-groups]
+dev = [
+    "pytest>=8.3.5",
+    "ruff>=0.14.3",
+]
+
+[tool.pytest.ini_options]
+filterwarnings = [
+    "ignore::DeprecationWarning",
+]
+
+  

sc_napalm-0.2.0/sample_env

@@ -0,0 +1,2 @@
+SC_USERNAME=automaton
+SC_PASSWORD=SCinetisCool

sc_napalm-0.2.0/src/custom_napalm/__init__.py

@@ -0,0 +1,28 @@
+from .iosxr import SCIOSXR
+from .junos import SCJunOS
+from .nxos import SCNXOS
+from .sros import SCNokiaSROSDriver
+from .srl import SCNokiaSRLDriver
+from .eos import SCEOSDriver
+from .nos import SCNOSDriver
+
+PLATFORM_MAP = {
+    "iosxr": SCIOSXR,
+    "nxos": SCNXOS,
+    "junos": SCJunOS,
+    "sros": SCNokiaSROSDriver,
+    "srl": SCNokiaSRLDriver,
+    "eos": SCEOSDriver,
+    "nos": SCNOSDriver,
+}
+
+
+def get_network_driver(platform: str):
+    """
+    Returns network driver based on platform string.
+    """
+    for valid_platform, driver in PLATFORM_MAP.items():
+        if valid_platform == platform:
+            return driver
+
+    raise NotImplementedError(f"Unsupported platform {platform}")

sc_napalm-0.2.0/src/custom_napalm/base.py

@@ -0,0 +1,108 @@
+from typing import Dict, List
+import re
+from napalm.base import models as napalm_models
+from netmiko import ConnectHandler
+
+from . import models as sc_models
+
+
+class SCBaseDriver:
+    """
+    Base class that all children should inherit
+    """
+
+    # populate in child classes
+    INVENTORY_TO_TYPE = {}
+
+    def _get_inventory_type(self, name: str) -> str:
+        """
+        Maps the name of the inventory item to its type
+        """
+        for pattern, inv_type in self.INVENTORY_TO_TYPE.items():
+            if inv_type and inv_type not in sc_models.VALID_INVENTORY_TYPES.__args__:
+                raise TypeError(f"Invalid Inventory type {inv_type}")
+            if re.search(pattern, name):
+                return inv_type
+
+        raise ValueError(f"Unknown inventory item {name}")
+
+    def get_config(self) -> napalm_models.ConfigDict:
+        """napalm get config"""
+        raise NotImplementedError
+
+    def get_facts(self) -> napalm_models.FactsDict:
+        """napalm get facts"""
+        raise NotImplementedError
+
+    def get_optics(self) -> Dict[str, napalm_models.OpticsDict]:
+        """napalm get optics"""
+        raise NotImplementedError
+
+    def get_lldp_neighbors(self) -> Dict[str, List[napalm_models.LLDPNeighborDict]]:
+        """napalm get lldp neighbors"""
+        raise NotImplementedError
+
+    def get_inventory(self) -> List[sc_models.InventoryDict]:
+        """sc get inventory"""
+        raise NotImplementedError
+
+
+class SCBaseNetconfDriver(SCBaseDriver):
+    """
+    Inclues some helper xml functions
+    """
+
+    def ssh_conn(self) -> ConnectHandler:
+        """
+        Ugly workaround for getting stuff via the cli over ssh.
+        Starts a netmiko ssh connection handler and returns it,
+        allowing you to interact with the CLI of the device.
+        """
+
+        args = {
+            "device_type": "linux",
+            "host":self.hostname,
+            "username":self.username,
+            "password":self.password, 
+            "ssh_config_file":self.optional_args.get("ssh_config", None),  
+        }
+        return ConnectHandler(**args)
+
+    def _find_txt(self, xml_tree, path, default=None, namespaces=None):
+        """
+        Stolen from the napalm iosxr driver
+        Extract the text value from a leaf in an XML tree using XPath.
+
+        Will return a default value if leaf path not matched.
+        :param xml_tree:the XML Tree object. <type'lxml.etree._Element'>.
+        :param path: XPath to be applied in order to extract the desired data.
+        :param default:  Value to be returned in case of a no match.
+        :param namespaces: namespace dictionary.
+        :return: a str value or None if leaf path not matched.
+        """
+        value = None
+        xpath_applied = xml_tree.xpath(path, namespaces=namespaces)
+
+        if xpath_applied:
+            if not len(xpath_applied[0]):
+                if xpath_applied[0].text is not None:
+                    value = xpath_applied[0].text.strip()
+                else:
+                    value = ""
+        else:
+            value = default
+
+        return value
+
+    # Helper xml methods that always pass in our namespaces by default
+    def _text(self, xml_tree, path, default=None):
+        return self._find_txt(xml_tree, path, default, namespaces=self.NS)
+
+    def _xpath(self, xml_tree, path):
+        return getattr(xml_tree, "xpath")(path, namespaces=self.NS)
+
+    def _find(self, xml_tree, element):
+        return getattr(xml_tree, "find")(element, namespaces=self.NS)
+
+    def _iterfind(self, xml_tree, element):
+        return getattr(xml_tree, "iterfind")(element, namespaces=self.NS)

sc_napalm-0.2.0/src/custom_napalm/eos/__init__.py

@@ -0,0 +1 @@
+from .eos import SCEOSDriver

sc_napalm-0.2.0/src/custom_napalm/eos/eos.py

@@ -0,0 +1,94 @@
+from typing import List
+from napalm.eos import EOSDriver
+from ..base import SCBaseDriver
+from ..models import InventoryDict
+
+
+class SCEOSDriver(EOSDriver, SCBaseDriver):
+    ("fabric_module",)
+    ("fan",)
+    ("linecard",)
+    ("optic",)
+    ("psu",)
+    ("re",)
+    ("stack_cable",)
+    ("stack_member",)
+    ("uplink_module",)
+    ("aoc",)
+    ("dac",)
+
+    INVENTORY_TO_TYPE = {
+        r"Fabric": "fabric_module",
+        r"Linecard": "linecard",
+        r"Supervisor": "re",
+    }
+
+    def __init__(self, hostname, username, password, timeout=60, optional_args=None):
+        """
+        Forcing ssh transport since we don't enable the web interface
+        """
+        optional_args = optional_args if optional_args else {}
+        optional_args["transport"] = "ssh"
+
+
+        super().__init__(
+            hostname, username, password, timeout=timeout, optional_args=optional_args
+        )
+
+    def get_inventory(self) -> List[InventoryDict]:
+        inventory = self._run_commands(["show inventory"], encoding="json")
+
+        results = []
+
+        ### optics
+        for slot, optic in inventory[0]["xcvrSlots"].items():
+            if optic.get("modelName"):
+                results.append(
+                    {
+                        "type": "optic",
+                        "subtype": optic["modelName"],
+                        "name": f"Ethernet{slot}",
+                        "part_number": optic["modelName"],
+                        "serial_number": optic["serialNum"],
+                    },
+                )
+
+        ### line cards
+        for slot, card in inventory[0]["cardSlots"].items():
+            if card.get("serialNum"):
+                results.append(
+                    {
+                        "type": self._get_inventory_type(slot),
+                        "subtype": card["modelName"],
+                        "name": f"Ethernet{slot}",
+                        "part_number": card["modelName"],
+                        "serial_number": card["serialNum"],
+                    },
+                )
+
+        ### PSUs
+        for slot, psu in inventory[0]["powerSupplySlots"].items():
+            if psu.get("serialNum"):
+                results.append(
+                    {
+                        "type": "psu",
+                        "subtype": None,
+                        "name": f"PSU {slot}",
+                        "part_number": psu["name"],
+                        "serial_number": psu["serialNum"],
+                    },
+                )
+
+        ### FANs
+        for slot, fan in inventory[0]["fanTraySlots"].items():
+            if fan.get("serialNum"):
+                results.append(
+                    {
+                        "type": "fan",
+                        "subtype": None,
+                        "name": f"FAN {slot}",
+                        "part_number": fan["name"],
+                        "serial_number": fan["serialNum"],
+                    },
+                )
+        return results

sc_napalm-0.2.0/src/custom_napalm/get.py

@@ -0,0 +1,85 @@
+import argparse
+from pprint import pprint
+
+from napalm import get_network_driver
+
+from custom_napalm import PLATFORM_MAP
+from custom_napalm.base import SCBaseDriver
+from custom_napalm.utils import configure_logging, LOG_LEVELS, get_from_args_or_env
+
+# list of getters to run
+GETTERS = [attr for attr in SCBaseDriver.__dict__ if attr.startswith("get")]
+
+cred_args = {"sc_username": True, "sc_password": True}
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="""
+Run a specific sc_napalm "getter" against a device.
+"""
+    )
+    parser.add_argument("device", help="device hostname or IP address")
+    parser.add_argument(
+        "sc_napalm_platform",
+        choices=PLATFORM_MAP,
+        help="The platform of this device",
+    )
+    parser.add_argument(
+        "cmd",
+        choices=GETTERS,
+        help="The getter command to run against this device",
+    )
+    parser.add_argument(
+        "--ssh-cfg", help="Use SSH config file to connect", type=str
+    )
+    log_args = parser.add_mutually_exclusive_group()
+    log_args.add_argument(
+        "-l", "--log-level", help="Set log level for sc_napalm only", choices=LOG_LEVELS
+    )
+    log_args.add_argument(
+        "-L", "--LOG-LEVEL", help="set global log level", choices=LOG_LEVELS
+    )
+    parser.add_argument(
+        "--logfile",
+        type=str,
+        help="Save logging to a file (specified by name) instead of to stdout",
+    )
+
+    for cred_arg in cred_args:
+        parser.add_argument(f"--{cred_arg}", help="Specify credentials")
+    args = parser.parse_args()
+
+    log_level = args.log_level if args.log_level else args.LOG_LEVEL
+    if log_level:
+        configure_logging(
+            log_level,
+            log_globally=bool(args.LOG_LEVEL),
+            log_file=args.logfile,
+            log_to_console=not (bool(args.logfile)),
+        )
+
+    creds = {
+        cred: get_from_args_or_env(cred, args, required=reqd)
+        for cred, reqd in cred_args.items()
+    }
+    Driver = get_network_driver(args.sc_napalm_platform)
+
+    # setting up connection details
+    optional_args = {"look_for_keys": False}
+    if args.ssh_cfg:
+        optional_args = {"ssh_config_file": args.ssh_cfg}
+
+    with Driver(
+        args.device,
+        creds["sc_username"],
+        creds["sc_password"],
+        timeout=60,
+        optional_args=optional_args,
+    ) as conn:
+        result = getattr(conn, args.cmd)()
+        pprint(result)
+
+
+if __name__ == "__main__":
+    main()

sc_napalm-0.2.0/src/custom_napalm/iosxr/__init__.py

@@ -0,0 +1 @@
+from .iosxr import SCIOSXR