oxidized 0.29.1 → 0.37.0

data/docs/Inputs.md

@@ -0,0 +1,237 @@
+# Inputs
+## Index
+* Configuration
+  * [SSH](#ssh)
+  * [SCP](#scp)
+  * [FTP](#ftp)
+  * telnet
+  * http
+  * tftp
+  * exec
+* [Debugging](#debugging)
+
+## SSH
+### Disabling SSH exec channels
+
+Oxidized uses exec channels to make information extraction simpler, but there
+are some situations where this doesn't work well, e.g. configuring devices. This
+feature can be turned off by setting the `ssh_no_exec`
+variable.
+
+```yaml
+vars:
+  ssh_no_exec: true
+```
+
+### Disabling SSH keepalives
+
+Oxidized SSH input makes use of SSH keepalives to prevent timeouts from slower
+devices and to quickly tear down stale sessions in larger deployments. There
+have been reports of SSH keepalives breaking compatibility with certain OS
+types. They can be disabled using the `ssh_no_keepalive` variable on a per-node
+basis (by specifying it in the source) or configured application-wide.
+
+```yaml
+vars:
+  ssh_no_keepalive: true
+```
+
+### SSH Auth Methods
+
+By default, Oxidized registers the following auth methods: `none`, `publickey` and `password`. However you can configure this globally, by groups, models or nodes.
+
+```yaml
+vars:
+  auth_methods: [ "none", "publickey", "password", "keyboard-interactive" ]
+```
+
+### Public Key Authentication with SSH
+
+Instead of password-based login, Oxidized can make use of key-based SSH
+authentication.
+
+You can tell Oxidized to use one or more private keys globally, or specify the
+key to be used on a per-node basis. The latter can be done by mapping the
+`ssh_keys` variable through the active source.
+
+Global:
+
+```yaml
+vars:
+  ssh_keys: "~/.ssh/id_rsa"
+```
+
+Per-Node:
+
+```yaml
+# ...
+map:
+  name: 0
+  model: 1
+vars_map:
+  enable: 2
+  ssh_keys: 3
+# ...
+```
+
+If you are using a non-standard path, especially when copying the private key
+via a secured channel, make sure that the permissions are set correctly:
+
+```bash
+foo@bar:~$ ls -la ~/.ssh/
+total 20
+drwx------ 2 oxidized oxidized 4096 Mar 13 17:03 .
+drwx------ 5 oxidized oxidized 4096 Mar 13 21:40 ..
+-r-------- 1 oxidized oxidized  103 Mar 13 17:03 authorized_keys
+-rw------- 1 oxidized oxidized  399 Mar 13 17:02 id_ed25519
+-rw-r--r-- 1 oxidized oxidized   94 Mar 13 17:02 id_ed25519.pub
+```
+
+Finally, multiple private keys can be specified as an array of file paths, such
+as `["~/.ssh/id_rsa", "~/.ssh/id_another_rsa"]`.
+
+### SSH Proxy Command
+
+Oxidized can `ssh` through a proxy as well. To do so we just need to set
+`ssh_proxy` variable with the proxy host information and optionally set the
+`ssh_proxy_port` with the SSH port if it is not listening on port 22.
+
+This can be provided on a per-node basis by mapping the proper fields from your
+source.
+
+An example for a `csv` input source that maps the 4th field as the `ssh_proxy`
+value and the 5th field as `ssh_proxy_port`.
+
+```yaml
+# ...
+map:
+  name: 0
+  model: 1
+vars_map:
+  enable: 2
+  ssh_proxy: 3
+  ssh_proxy_port: 4
+# ...
+```
+
+### SSH enabling legacy algorithms
+
+When connecting to older firmware over SSH, it is sometimes necessary to enable
+legacy/disabled settings like KexAlgorithms, HostKeyAlgorithms, MAC or the
+Encryption.
+
+These settings can be provided on a per-node basis by mapping the ssh_kex,
+ssh_host_key, ssh_hmac and the ssh_encryption fields from you source.
+
+```yaml
+# ...
+map:
+  name: 0
+  model: 1
+vars_map:
+  enable: 2
+  ssh_kex: 3
+  ssh_host_key: 4
+  ssh_hmac: 5
+  ssh_encryption: 6
+# ...
+```
+
+### Custom SSH port
+Set the variable `ssh_port` to the desired value (default is 22).
+
+### SSH Host key verification
+With the configuration `secure', you can set the ssh key verification:
+- `true`: strict host verification, looking up the known host files
+- `false` (default): disable host verification, accept any ssh key
+
+```
+input:
+  ssh:
+    secure: true
+```
+
+## SCP
+### SSH Host key verification (SCP)
+Same as for [SSH host key verification](#ssh-host-key-verification)
+
+```
+input:
+  scp:
+    secure: true
+```
+
+### Custom SCP port
+Set the variable `ssh_port` to the desired value (default is 22).
+
+## FTP
+### FTP Passive Mode
+
+Oxidized uses ftp passive mode by default. Some devices require passive mode to
+be disabled. To do so, we can set `input.ftp.passive` to false - this will make
+use of FTP active mode.
+
+```yaml
+input:
+  ftp:
+    passive: false
+```
+
+## HTTP
+### Supported HTTP Methods
+
+The HTTP input supports the following HTTP methods:
+- `:get`  - for GET requests
+- `:post` - for POST requests
+
+These methods are used internally by models that require HTTP-based 
+configuration retrieval. Models can use `get_http()` and `post_http()` methods 
+provided by the HTTP input.
+
+Example usage in a model:
+
+```ruby
+cfg :http do
+  post_response = post_http('/some/path', payload, 'Some-Extra-Header' => 'value')
+  get_response  = get_http('/some/path')
+end
+```
+
+HTTP input can be enabled by adding this block to the configuration file:
+
+```yaml
+input:
+  http:
+    scheme: https
+    ssl_verify: true
+    timeout: 30
+```
+
+## Debugging
+In case a model plugin doesn't work correctly (ios, procurve, etc.), you can
+enable live debugging of SSH and Telnet sessions with the `debug` option of
+the `input` section.
+
+Starting with version 0.37.0, `debug` can take different values:
+- `text`: log input and output to a text file (ssh, telnet)
+- `yaml`: produce a yaml simulation file (ssh, scp)
+- `library`: activate debug logging of the underlying library
+- a combination of the options above (`text, yaml`)
+- `true`; activate all debugging options (Only option for versions prior 0.37.0)
+
+The log files will be created in `~/.config/oxidized/logs/` (or `$OXIDIZED_LOGS/logs/`).
+
+The following example will log an active ssh/telnet session to
+`~/.config/oxidized/logs/<IP-Address>-<PROTOCOL>-<timestamp>.txt` and for ssh
+`~/.config/oxidized/logs/<IP-Address>-<PROTOCOL>-<timestamp>.yaml`. A new file
+is created for each session.
+
+```yaml
+input:
+  default: ssh, telnet
+  debug: yaml, text
+  ssh:
+    secure: false
+  http:
+    ssl_verify: true
+```

data/docs/Issues.md

@@ -0,0 +1,123 @@
+# Writing good issues
+If you're experiencing a problem with Oxidized or need a new feature, you can
+[submit an issue on github](https://github.com/ytti/oxidized/issues). We have
+a great community where users help each other through the issue system.
+
+This guide provides tips on writing your issue to make it easier for the
+community and developers to understand and respond effectively.
+
+Why write good issues?
+- A clear and detailed issue improves the chances of getting your problem
+  resolved.
+- By spending time to write a good issue, you save developers time, contributing
+  to Oxidized’s progress without writing a line of code.
+
+## Rules
+The project receives too many incomplete issues, unnecessarily taking time that
+could be invested in new code. Therefore, issues will be worked on with the
+following rules:
+
+- Use the predefined templates for bugs, feature requests and support requests.
+- If you don't provide the necessary information (read this file, fill in the
+  questions in the templates), expect your issue to be closed without a comment.
+- Inactive issues will be marked "stale" automatically after 90 days. Issues
+  are not closed automatically; this is a manual action by a maintainer.
+- A feature request may be closed after some time of inactivity, as obviously
+  no one has found the time to implement it. Consider contributing code in a
+  Pull Request instead.
+- While it is OK to ask for help (using the support request template), don't be
+  disappointed if no one finds the time to answer your question. Stale questions
+  (after 90 days) will be closed without a comment.
+
+## Submit to the correct project
+Choose the appropriate GitHub project based on your issue:
+
+- For issues with the web frontend or REST API, go to
+  [oxidized-web](https://github.com/ytti/oxidized-web/).
+- For issues with oxidized-script, use
+  [oxidized-script](https://github.com/ytti/oxidized-script). (note: as of
+  November 2024, oxidized-script is not actively maintained).
+- For issues with third-party software relying on Oxidized, open an issue in
+  that specific project.
+- For issues with Oxidized itself, go to
+  [oxidized](https://github.com/ytti/oxidized).
+
+## Use the latest version
+If you are using an old version of oxidized, you may encounter issues that have
+been solved. No support will be provided for older versions of oxidized.
+
+If you can, please also test against the latest git version, or at least read
+[CHANGELOG.md](/CHANGELOG.md) to see if your problem has been solved on master.
+
+## Format your issue
+- Use [GitHub Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax) to format your issue.
+- Preview your text before submitting to ensure it renders correctly.
+- Avoid screenshots of text. Instead, use [code formating](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code) for any relevant code snippets.
+
+## Choose your title well
+Keep the title brief yet descriptive. Aim to summarize the main issue or request in a few words.
+
+## Provide detailled informations
+Include as many relevant details as possible. At a minimum, specify:
+
+- Oxidized version and operating system.
+- Which model (oxidized name AND Manufacturer name) is the issue about.
+- Relevant parts of your Oxidized configuration and a brief explanation of your setup.
+- Output of the error, if relevant.
+- For issues related to specific devices, consider creating a YAML Simulation file (instructions below).
+
+Also, provide clear steps to reproduce the issue, if applicable.
+
+## Making feature requests
+Feature requests are welcome, but please understand that unaddressed requests
+may be closed after some time. If you need a feature urgently, consider
+contributing code via a pull request (PR) or hiring a developer.
+
+## Sumbit a YAML Simulation File
+To help developers troubleshoot device-specific issues, you may be asked to submit a
+[YAML simulation file](/docs/DeviceSimulation.md#creating-a-yaml-file-with-device2yamlrb) for your device.
+
+Here's a brief overview how to do it, you can find more details in the link
+above.
+- Fork Oxidized on github
+- Install dependencies (git and Ruby's Net::SSH):
+```
+# Adapt when not using a debian-based distro
+sudo apt install git ruby-net-ssh
+```
+- Clone your forked Oxidized repository:
+```
+git clone git@github.com:<your github user>/oxidized.git
+```
+- run the `extra/device2yaml.rb` script (you’ll be provided with the command to
+run) from the repository root:
+
+```
+extra/device2yaml.rb oxidized@r61 -c "terminal length 0
+terminal width 0
+show version
+show vtp status
+show inventory
+show running-config
+exit" -o spec/model/data/ios:C8200L_16.12.1:simulation.yaml
+```
+
+- The script waits 5 seconds between commands, and outputs the response of the
+  device. You can press "ESC" if you see the prompt and want to pass to next
+  command without waiting for the timeout.
+- The result will be stored in `spec/model/data/`.
+- Replace any sensitive information with placeholder values in the output file.
+- Commit & push the file to github
+```
+git add spec/model/data/ios:C8200L_16.12.1:simulation.yaml
+git commit -m "Device simulation for C8200L"
+git push
+```
+- Create a pull request (PR) in GitHub, referencing the issue number (e.g.,
+  "YAML simulation file for issue #1234").
+
+
+
+
+
+

data/docs/Model-Notes/APC.md

@@ -0,0 +1,72 @@
+# APC Configuration
+The configuration of APC Network Management Cards can be downloaded using FTP
+and SCP. You can retrieve serial numbers and OS version information through
+an SSH connection.
+
+APC OS does not have the ability to display the config.ini within an SSH shell.
+A ticket was opened with APC support to enable "cat config.ini"
+within an SSH shell, but APC declined to implement this feature.
+
+To overcome this limitation, a capability to run against multiple inputs (SSH + SCP)
+has been implemented in Oxidized and in the [model ApcAos](/lib/oxidized/model/apcaos.rb).
+
+The old model apc_aos (SCP/FTP only) is deprecated and will be removed in a
+future release. Migrate to ApcAos.
+
+## How do I activate FTP/SCP input?
+To download the configuration with FTP or SCP, you must activate it
+as an input in the Oxidized configuration. If you don't activate the input,
+Oxidized will fail for the node with an error.
+
+You probably also need to increase the default timeout to something about 60
+seconds, as the APC are really slow, and need about 30 seconds to complete.
+
+The configuration can be done either globally or only for the ApcAos model.
+
+### Global Configuration
+The global configuration would look like this. Note that Oxidized will try every
+input type in the given order until it succeeds, or it will report a failure.
+```yaml
+timeout: 60
+input:
+  default: ssh, ftp, scp
+```
+The order in the configuration is relevant. With this configuration, the ApcAos
+model will run SSH first, then it will try FTP, and if FTP fails SCP.
+
+### Model-Specific Configuration
+
+Configuration for activating only the SCP input for ApcAos only:
+```yaml
+input:
+  default: ssh
+models:
+  apcaos:
+    input: ssh, scp
+    timeout: 60
+```
+
+### Setting Specific Credentials
+You can also set a specific username and password for ApcAos only:
+```yaml
+username: default-user
+password: default-password
+input:
+  default: ssh
+models:
+  ApcAos:
+    username: apc-user
+    password: apc-password
+    input: ssh, scp
+    timeout: 60
+```
+
+## Why do I partially get CR + LF?
+The config.ini file has a DOS-Format (CR + LF), and is saved without
+modifications, so that it can be uploaded to the device.
+
+Outputs from ssh are stored without CR, so the first part of the file is
+without CR and config.ini with CR + LF.
+
+This is expected behavior and should not affect the functionality of the backup
+or restore process.

data/docs/Model-Notes/Cumulus.md

@@ -11,6 +11,9 @@ A variable has been added to enable users running Cumulus Linux > 3.4.0 to targe
 ## NCLU
 It is possible to switch to [NCLU](https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-44/System-Configuration/Network-Command-Line-Utility-NCLU/) as a configuration collecting method, by setting `cumulus_use_nclu` to true
 
+## NVUE
+It is also possible to use [NVUE](https://docs.nvidia.com/networking-ethernet-software/knowledge-base/Setup-and-Getting-Started/NVUE-Cheat-Sheet/) as a configuration collecting method, by setting `cumulus_use_nvue` to true.
+
 ### Example usage
 
 ```yaml
@@ -43,4 +46,6 @@ The default value for `cumulus_routing_daemon` is `quagga` so existing installat
 
 The default value for `cumulus_use_nclu` is `false`, in case NCLU is not installed.
 
+The default value for `cumulus_use_nvue` is `false`, in case NVUE is not installed.
+
 Back to [Model-Notes](README.md)

data/docs/Model-Notes/EatonNetwork.md

@@ -0,0 +1,18 @@
+# EatonNetwork Configuration
+
+This model uses the command `save_configuration -p <passphrase>` to get the backup config. The `-p` option is a required passphrase used to encrypted sensitive parts of the config data, the encrypted data is nondeterministic and changes with each run. The passphrase used is the auth password for the node.
+
+See the [Eaton Network-M3 user's guide](https://www.eaton.com/content/dam/eaton/products/backup-power-ups-surge-it-power-distribution/power-management-software-connectivity/eaton-gigabit-network-card/network-m3/resources/eaton-network-m3-user-guide.pdf) section 7.7.14 (page 260) for more information.
+
+To not have the backup change on each for all `eatonnetwork` node run set a model var in the config for the `eatonnetwork` model to [remove secrets](../Configuration.md#removing-secrets):
+
+```yaml
+models:
+  eatonnetwork:
+    vars:
+      remove_secret: true
+```
+
+See the [Eaton Network-M3 user's guide](https://www.eaton.com/content/dam/eaton/products/backup-power-ups-surge-it-power-distribution/power-management-software-connectivity/eaton-gigabit-network-card/network-m3/resources/eaton-network-m3-user-guide.pdf) section 3.20 (page 111) for details on JSON configuration structure, and restoring without sensitive/secrets.
+
+Back to [Model-Notes](README.md)

data/docs/Model-Notes/ExaLink.md

@@ -0,0 +1,43 @@
+# Cisco Nexus 3550-F (ExaLink Fusion)
+
+The Cisco Nexus 3550-F (formerly Exablaze ExaLink Fusion) is an ultra-low-latency
+Layer 1/2 switch platform based on FPGA technology, primarily used in high-frequency
+trading and HPC environments. It runs a custom Linux-based OS with a proprietary CLI
+and JSON RPC API.
+
+## Device Configuration
+
+Create a read-only user for Oxidized on the device:
+
+```
+admin@N3550-F> configure user oxidized password <password>
+admin@N3550-F> configure user oxidized privilege read-only
+```
+
+## Oxidized Configuration
+
+```yaml
+source:
+  default: csv
+  csv:
+    file: "/home/oxidized/.config/oxidized/router.db"
+    delimiter: !ruby/regexp /:/
+    map:
+      name: 0
+      model: 1
+```
+
+Example `router.db` entry:
+
+```bash
+myswitch.example.com:exalink
+```
+
+## Notes
+
+- Both SSH and Telnet are supported. SSH is recommended.
+- The model collects `show version` (excluding uptime to avoid noisy diffs),
+  `show port`, and `show running-config`.
+- Timestamps (`!Time:`) are stripped from the running config to avoid noisy diffs.
+- The device prompt format is `hostname#` or `hostname>`.
+- This model was developed and tested against software version 1.16.0.

data/docs/Model-Notes/FSOS.md

@@ -0,0 +1,17 @@
+# Fiberstore (fs.com) FSOS notes
+
+This has been tested against the following models and OS versions
+
+|Model               |OS Version and Build          |
+|--------------------|------------------------------|
+|S3400-48T4SP        |Version 2.0.2J Build 81736    |
+|S3400-48T4SP        |Version 2.0.2J Build 95262    |
+|S3400-48T6SP        |Version 2.2.0F Build 109661   |
+|S3400-48T4SP        |Version 2.0.2J Build 120538   |
+|S3410-24TS-P        |S3410_FSOS 11.4(1)B74S5       |
+|S5850-48T4Q         |Version 7.0.4.34              |
+|S5800-48MBQ         |Version 7.0.4.21              |
+|S5810-48TS-P        |S5810_FSOS 11.4(1)B74S8, Release(10200711)              |
+|S5860-20SQ          |S5860_FSOS 12.4(1)B0101P1S4   |
+
+Back to [Model-Notes](README.md)

data/docs/Model-Notes/Fortinet.md

@@ -0,0 +1,75 @@
+# Fortinet models
+There are two models for Fortinet devices:
+- fortigate: for the FortiGate firewalls
+- fortios: for VM-Based appliances (FortiManager, FortiADC, FortiAnalyzer...)
+
+# Notes for both models
+## Configuration changes / hiding passwords
+Fortigate and Fortios re-encrypt their passwords every time the configuration is shown.
+This results in a lot of apparent configuration changes on every pull.
+
+To avoid this, you have two options:
+- remove secrets
+- save significant changes only
+
+### Remove secrets
+If you don't want to have a new version every time the configuration is
+downloaded, you can hide all secrets. Beware that you won't have a full backup, as all passwords will be replaced with <configuration removed>
+
+```yaml
+models:
+  fortigate:
+    vars:
+      remove_secret: true
+```
+
+### Save significant changes only
+You can [store the configuration only on significant changes](/docs/Configuration.md#store-configuration-only-on-significant-changes)
+by setting the [variable](#options-credentials-vars-etc-precedence)
+`output_store_mode` to `on_significant`. On FortiGate and FortiOS, this
+prevents Oxidized from saving a configuration when there were only changes to
+the encrypted passwords. Beware that you won't have the last backup if you only
+changed a password.
+
+```yaml
+vars:
+  output_store_mode: on_significant
+```
+
+# Notes for the FortiGate model
+## Create user oxidized with ED25519 public key
+You can use a user/password for retrieving the configuration or use a SSH public key:
+
+```text
+config system admin
+edit oxidized
+set trusthost1 192.0.2.1 255.255.255.255
+set accprofile "super_admin_readonly"
+set ssh-public-key1 "ssh-ed25519 AAAAThisIsJustAnExampleKey_UseYourOxidizedPUBLICKEY oxidized@librenms"
+end
+```
+
+## config vs. full config
+On FortiGate, you can get a configuration without default values (`show`) or
+including all default values (`show full-configuration`).
+
+The full configuration can be long and may cause timeouts.
+Starting with with oxidized 0.30.1, the default is to get the short configuration.
+
+If you need the full configuration, you can activate it in oxidized config file:
+```yaml
+models:
+  fortigate:
+    vars:
+      fullconfig: true
+```
+## Autoupdate
+You can get the result of `diagnose autoupdate version` by setting the [variable](#options-credentials-vars-etc-precedence) `fortigate_autoupdate` to `true`:
+
+```yaml
+vars:
+  fortigate_autoupdate: true
+```
+
+Note that the variable `fortios_autoupdate` is deprecated and will be removed
+in a future Version of Oxidized. Use `fortigate_autoupdate` instead.

data/docs/Model-Notes/GrandstreamHT8xx.md

@@ -0,0 +1,8 @@
+# Grandstream HT8xx
+
+You need to have enabled access to device by SSH.
+Connection using user/password for retrieve the configuration containing XML file with all params stored in device memory.
+
+Tested on hardware: v1 and software version: 1.0.61.2.
+
+Back to [Model-Notes](README.md)

data/docs/Model-Notes/HPEAruba.md

@@ -0,0 +1,32 @@
+# HPE Aruba Networking devices
+HPE Aruba offers various networking devices with different operating systems.
+
+## HPE Aruba Networking Instant Mode (Aruba Instant)
+[Aruba Instant](https://www.arubanetworks.com/techdocs/ArubaDocPortal/content/cons-instant-home.htm)
+runs on IAPs (Instant Access points).
+
+The Oxidized model is [ArubaInstant](/lib/oxidized/model/arubainstant.rb).
+When run on the virtual WLAN controller, it will also collect the list of the
+WLAN-AP linked to the controller.
+
+The aosw model for AOS 8 used to be used for Aruba Instant, but it does not work
+as well and may stop working in the future.
+
+## HPE Aruba Networking Wireless Operating System 8 (AOS 8)
+[AOS 8](https://www.arubanetworks.com/techdocs/ArubaDocPortal/content/cons-aos-home.htm)
+runs on WLAN controllers (mobility controllers) and controller-managed access
+points.
+
+The Oxidized model is [aosw](/lib/oxidized/model/aosw.rb).
+
+## HPE Aruba Networking CX Switch Operating System (AOS-CX)
+[AOS-CX](https://www.arubanetworks.com/techdocs/AOS-CX/help_portal/Content/home.htm)
+is the operating system for the newer CX-Series.
+
+The Oxidized model is [aoscx](/lib/oxidized/model/aoscx.rb).
+
+## Older Models
+Older Devices like [ProCurve](/lib/oxidized/model/procurve.rb) or 3Com/Comware
+are listed under the Vendor "HP" in the
+[Supported OS Types](/docs/Supported-OS-Types.md) list.
+