oxidized 0.29.1 → 0.37.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (236) hide show
  1. checksums.yaml +4 -4
  2. data/.coderabbit.yaml +21 -0
  3. data/.github/ISSUE_TEMPLATE/bug_report.md +45 -0
  4. data/.github/ISSUE_TEMPLATE/feature_request.md +22 -0
  5. data/.github/ISSUE_TEMPLATE/support-request.md +39 -0
  6. data/.github/workflows/codeql.yml +4 -4
  7. data/.github/workflows/publishdocker.yml +108 -13
  8. data/.github/workflows/ruby.yml +3 -12
  9. data/.github/workflows/stale.yml +6 -1
  10. data/.gitignore +2 -0
  11. data/.rubocop.yml +69 -13
  12. data/.rubocop_todo.yml +13 -143
  13. data/CHANGELOG.md +431 -4
  14. data/CONTRIBUTING.md +189 -0
  15. data/Dockerfile +89 -34
  16. data/README.md +59 -188
  17. data/Rakefile +58 -12
  18. data/bin/oxidized +2 -5
  19. data/docs/Configuration.md +382 -149
  20. data/docs/Creating-Models.md +262 -10
  21. data/docs/DeviceSimulation.md +196 -0
  22. data/docs/Docker.md +246 -0
  23. data/docs/Hooks.md +146 -51
  24. data/docs/Inputs.md +237 -0
  25. data/docs/Issues.md +123 -0
  26. data/docs/Model-Notes/APC.md +72 -0
  27. data/docs/Model-Notes/Cumulus.md +5 -0
  28. data/docs/Model-Notes/EatonNetwork.md +18 -0
  29. data/docs/Model-Notes/ExaLink.md +43 -0
  30. data/docs/Model-Notes/FSOS.md +17 -0
  31. data/docs/Model-Notes/Fortinet.md +75 -0
  32. data/docs/Model-Notes/GrandstreamHT8xx.md +8 -0
  33. data/docs/Model-Notes/HPEAruba.md +32 -0
  34. data/docs/Model-Notes/IvantiConnectSecure.md +59 -0
  35. data/docs/Model-Notes/OS6.md +10 -0
  36. data/docs/Model-Notes/README.md +1 -20
  37. data/docs/Model-Notes/RouterOS.md +28 -0
  38. data/docs/Model-Notes/SikluMHTG.md +7 -0
  39. data/docs/Model-Notes/TrueNAS.md +23 -0
  40. data/docs/Model-Notes/XGS4600-Zyxel.md +5 -0
  41. data/docs/ModelUnitTests.md +219 -0
  42. data/docs/Outputs.md +102 -5
  43. data/docs/Release.md +102 -0
  44. data/docs/Ruby-API.md +135 -2
  45. data/docs/Sources.md +34 -0
  46. data/docs/Supported-OS-Types.md +207 -270
  47. data/docs/Troubleshooting.md +43 -1
  48. data/extra/device2yaml.rb +259 -0
  49. data/extra/gitdiff-msteams.sh +32 -5
  50. data/extra/hooks/modelrules.rb +55 -0
  51. data/extra/hooks/modelrulesadvanced.rb +167 -0
  52. data/extra/hooks/srcipmap.rb +54 -0
  53. data/extra/nagios_check_failing_nodes.rb +1 -1
  54. data/extra/oxidized.runit +1 -1
  55. data/extra/rest_client.rb +5 -4
  56. data/extra/syslog.rb +11 -6
  57. data/lib/oxidized/cli/support.rb +152 -0
  58. data/lib/oxidized/cli.rb +21 -6
  59. data/lib/oxidized/config/vars.rb +33 -14
  60. data/lib/oxidized/config.rb +28 -17
  61. data/lib/oxidized/core.rb +50 -8
  62. data/lib/oxidized/hook/ciscosparkdiff.rb +11 -9
  63. data/lib/oxidized/hook/exec.rb +5 -4
  64. data/lib/oxidized/hook/githubrepo.rb +55 -18
  65. data/lib/oxidized/hook/noophook.rb +2 -2
  66. data/lib/oxidized/hook/slackdiff.rb +50 -14
  67. data/lib/oxidized/hook/xmppdiff.rb +9 -9
  68. data/lib/oxidized/hook.rb +68 -16
  69. data/lib/oxidized/input/cli.rb +8 -3
  70. data/lib/oxidized/input/debugtext.rb +40 -0
  71. data/lib/oxidized/input/debugyaml.rb +82 -0
  72. data/lib/oxidized/input/exec.rb +2 -11
  73. data/lib/oxidized/input/ftp.rb +2 -19
  74. data/lib/oxidized/input/http.rb +67 -17
  75. data/lib/oxidized/input/input.rb +35 -14
  76. data/lib/oxidized/input/scp.rb +27 -0
  77. data/lib/oxidized/input/ssh.rb +59 -83
  78. data/lib/oxidized/input/sshbase.rb +102 -0
  79. data/lib/oxidized/input/telnet.rb +20 -19
  80. data/lib/oxidized/input/tftp.rb +8 -8
  81. data/lib/oxidized/job.rb +18 -7
  82. data/lib/oxidized/jobs.rb +11 -1
  83. data/lib/oxidized/logger.rb +51 -0
  84. data/lib/oxidized/manager.rb +18 -17
  85. data/lib/oxidized/model/acos.rb +2 -1
  86. data/lib/oxidized/model/addpack.rb +26 -0
  87. data/lib/oxidized/model/adtran.rb +5 -1
  88. data/lib/oxidized/model/adva.rb +3 -3
  89. data/lib/oxidized/model/airfiber.rb +1 -1
  90. data/lib/oxidized/model/aos7.rb +11 -0
  91. data/lib/oxidized/model/aoscx.rb +37 -29
  92. data/lib/oxidized/model/aosw.rb +39 -29
  93. data/lib/oxidized/model/apc_aos.rb +7 -2
  94. data/lib/oxidized/model/apcaos.rb +39 -0
  95. data/lib/oxidized/model/aricentiss.rb +9 -2
  96. data/lib/oxidized/model/arubainstant.rb +93 -0
  97. data/lib/oxidized/model/asa.rb +11 -10
  98. data/lib/oxidized/model/asternos.rb +22 -0
  99. data/lib/oxidized/model/asyncos.rb +3 -3
  100. data/lib/oxidized/model/audiocodes.rb +2 -2
  101. data/lib/oxidized/model/awplus.rb +15 -12
  102. data/lib/oxidized/model/axos.rb +1 -1
  103. data/lib/oxidized/model/bdcom.rb +1 -0
  104. data/lib/oxidized/model/c4cmts.rb +1 -2
  105. data/lib/oxidized/model/ciscosma.rb +1 -1
  106. data/lib/oxidized/model/ciscosmb.rb +6 -1
  107. data/lib/oxidized/model/cnos.rb +13 -10
  108. data/lib/oxidized/model/comtrol.rb +1 -1
  109. data/lib/oxidized/model/comware.rb +5 -3
  110. data/lib/oxidized/model/cumulus.rb +14 -2
  111. data/lib/oxidized/model/defacto.rb +26 -0
  112. data/lib/oxidized/model/dellx.rb +1 -1
  113. data/lib/oxidized/model/dlink.rb +5 -2
  114. data/lib/oxidized/model/dlinknextgen.rb +55 -0
  115. data/lib/oxidized/model/dnos.rb +3 -0
  116. data/lib/oxidized/model/dslcommands.rb +93 -0
  117. data/lib/oxidized/model/dslsetup.rb +102 -0
  118. data/lib/oxidized/model/eatonnetwork.rb +70 -0
  119. data/lib/oxidized/model/edgecos.rb +3 -1
  120. data/lib/oxidized/model/edgeos.rb +7 -6
  121. data/lib/oxidized/model/edgeswitch.rb +3 -1
  122. data/lib/oxidized/model/efos.rb +41 -0
  123. data/lib/oxidized/model/eltex.rb +4 -2
  124. data/lib/oxidized/model/enterasys800.rb +2 -2
  125. data/lib/oxidized/model/enterprise_sonic.rb +46 -0
  126. data/lib/oxidized/model/eos.rb +3 -1
  127. data/lib/oxidized/model/exalink.rb +36 -0
  128. data/lib/oxidized/model/f5os.rb +17 -0
  129. data/lib/oxidized/model/fabricos.rb +2 -2
  130. data/lib/oxidized/model/fastiron.rb +5 -3
  131. data/lib/oxidized/model/firebrick.rb +2 -2
  132. data/lib/oxidized/model/firelinuxos.rb +13 -6
  133. data/lib/oxidized/model/firewareos.rb +10 -1
  134. data/lib/oxidized/model/fortigate.rb +160 -0
  135. data/lib/oxidized/model/fortios.rb +38 -48
  136. data/lib/oxidized/model/fsos.rb +46 -0
  137. data/lib/oxidized/model/gaiaos.rb +4 -4
  138. data/lib/oxidized/model/garderos.rb +38 -0
  139. data/lib/oxidized/model/grandstreamht8xx.rb +19 -0
  140. data/lib/oxidized/model/h3c.rb +1 -1
  141. data/lib/oxidized/model/ibos.rb +1 -0
  142. data/lib/oxidized/model/ingate.rb +47 -0
  143. data/lib/oxidized/model/ios.rb +57 -30
  144. data/lib/oxidized/model/iosxr.rb +2 -2
  145. data/lib/oxidized/model/ironware.rb +7 -5
  146. data/lib/oxidized/model/ivanti.rb +54 -0
  147. data/lib/oxidized/model/junos.rb +12 -5
  148. data/lib/oxidized/model/kornfeldos.rb +33 -0
  149. data/lib/oxidized/model/lenovonos.rb +2 -0
  150. data/lib/oxidized/model/linksyssrw.rb +3 -3
  151. data/lib/oxidized/model/linuxgeneric.rb +4 -2
  152. data/lib/oxidized/model/macros.rb +60 -0
  153. data/lib/oxidized/model/mimosab11.rb +34 -0
  154. data/lib/oxidized/model/ml66.rb +33 -0
  155. data/lib/oxidized/model/mlnxos.rb +18 -14
  156. data/lib/oxidized/model/model.rb +98 -103
  157. data/lib/oxidized/model/mtrlrfs.rb +1 -0
  158. data/lib/oxidized/model/ndms.rb +6 -0
  159. data/lib/oxidized/model/netgear.rb +14 -3
  160. data/lib/oxidized/model/netonix.rb +1 -1
  161. data/lib/oxidized/model/netscaler.rb +1 -1
  162. data/lib/oxidized/model/nodegrid.rb +1 -1
  163. data/lib/oxidized/model/nsxdfw.rb +31 -0
  164. data/lib/oxidized/model/nsxfirewall.rb +2 -1
  165. data/lib/oxidized/model/nxos.rb +20 -15
  166. data/lib/oxidized/model/ocnos.rb +42 -0
  167. data/lib/oxidized/model/onefinity.rb +18 -0
  168. data/lib/oxidized/model/openbsd.rb +1 -1
  169. data/lib/oxidized/model/opengear.rb +36 -1
  170. data/lib/oxidized/model/openwrt.rb +6 -6
  171. data/lib/oxidized/model/opnsense.rb +9 -1
  172. data/lib/oxidized/model/os6.rb +48 -0
  173. data/lib/oxidized/model/outputs.rb +5 -0
  174. data/lib/oxidized/model/panos.rb +2 -0
  175. data/lib/oxidized/model/perle.rb +41 -0
  176. data/lib/oxidized/model/pfsense.rb +9 -0
  177. data/lib/oxidized/model/powerconnect.rb +32 -11
  178. data/lib/oxidized/model/procurve.rb +6 -2
  179. data/lib/oxidized/model/qtech.rb +3 -1
  180. data/lib/oxidized/model/quantaos.rb +8 -6
  181. data/lib/oxidized/model/rgos.rb +33 -0
  182. data/lib/oxidized/model/riverbed.rb +104 -0
  183. data/lib/oxidized/model/routeros.rb +18 -11
  184. data/lib/oxidized/model/saos.rb +18 -1
  185. data/lib/oxidized/model/saos10.rb +38 -0
  186. data/lib/oxidized/model/siklu.rb +1 -1
  187. data/lib/oxidized/model/siklumhtg.rb +22 -0
  188. data/lib/oxidized/model/sixwind.rb +28 -0
  189. data/lib/oxidized/model/slxos.rb +3 -3
  190. data/lib/oxidized/model/smartbyte.rb +48 -0
  191. data/lib/oxidized/model/sonicos.rb +27 -20
  192. data/lib/oxidized/model/speedtouch.rb +1 -1
  193. data/lib/oxidized/model/sros.rb +3 -3
  194. data/lib/oxidized/model/srosmd.rb +1 -1
  195. data/lib/oxidized/model/supermicro.rb +1 -1
  196. data/lib/oxidized/model/telco.rb +1 -1
  197. data/lib/oxidized/model/tmos.rb +1 -0
  198. data/lib/oxidized/model/tnsr.rb +57 -0
  199. data/lib/oxidized/model/tplink.rb +5 -5
  200. data/lib/oxidized/model/trango.rb +4 -2
  201. data/lib/oxidized/model/truenas.rb +64 -3
  202. data/lib/oxidized/model/ucs.rb +1 -1
  203. data/lib/oxidized/model/unifiap.rb +144 -0
  204. data/lib/oxidized/model/uplinkolt.rb +46 -0
  205. data/lib/oxidized/model/voltaire.rb +1 -1
  206. data/lib/oxidized/model/voss.rb +3 -0
  207. data/lib/oxidized/model/vrp.rb +5 -1
  208. data/lib/oxidized/model/vyatta.rb +9 -3
  209. data/lib/oxidized/model/vyos.rb +37 -0
  210. data/lib/oxidized/model/xos.rb +10 -1
  211. data/lib/oxidized/model/zhoneolt.rb +4 -2
  212. data/lib/oxidized/model/zynos.rb +67 -3
  213. data/lib/oxidized/model/zynosadsl.rb +14 -0
  214. data/lib/oxidized/model/zynoscli.rb +1 -1
  215. data/lib/oxidized/model/zynosgs.rb +2 -0
  216. data/lib/oxidized/model/zynosmgs.rb +32 -0
  217. data/lib/oxidized/node.rb +99 -64
  218. data/lib/oxidized/nodes.rb +40 -19
  219. data/lib/oxidized/output/file.rb +80 -43
  220. data/lib/oxidized/output/git.rb +295 -151
  221. data/lib/oxidized/output/gitcrypt.rb +193 -185
  222. data/lib/oxidized/output/http.rb +63 -51
  223. data/lib/oxidized/output/output.rb +20 -4
  224. data/lib/oxidized/signals.rb +44 -0
  225. data/lib/oxidized/source/csv.rb +49 -49
  226. data/lib/oxidized/source/http.rb +76 -63
  227. data/lib/oxidized/source/jsonfile.rb +68 -0
  228. data/lib/oxidized/source/source.rb +73 -18
  229. data/lib/oxidized/source/sql.rb +71 -59
  230. data/lib/oxidized/version.rb +6 -4
  231. data/lib/oxidized/worker.rb +57 -29
  232. data/lib/oxidized.rb +4 -24
  233. data/lib/refinements.rb +21 -1
  234. data/oxidized.gemspec +45 -21
  235. metadata +248 -41
  236. data/lib/oxidized/model/timos.rb +0 -10
@@ -0,0 +1,59 @@
1
+ ### Ivanti Connect Secure (ICS)
2
+
3
+ #### Overview
4
+
5
+ This model provides support for Ivanti Connect Secure (ICS) appliances using REST API ([official documentation](https://help.ivanti.com/ps/help/en_US/ICS/22.x/22.7R2/22.xICSAG.pdf)).
6
+ ICS stores its configuration as a binary ZIP archive (with `system.cfg` and `user.cfg` files) which is retrieved using the `/api/v1/system/binary-configuration` endpoint.
7
+
8
+ The model performs an initial authentication against `/api/v1/realm_auth` using Basic Auth (`username`/`password`) and retrieves a temporary `api_key`.
9
+ This key is then used for all further API requests during the Oxidized collection cycle.
10
+
11
+ The model is designed to work with standard ICS deployments without requiring command-line access to the device.
12
+
13
+ #### How Configuration Is Retrieved
14
+
15
+ 1. Oxidized authenticates using:
16
+
17
+ ```bash
18
+ POST /api/v1/realm_auth
19
+ ```
20
+
21
+ with:
22
+ - Basic Auth: `username` + `password`
23
+ - JSON body `{"realm": "<realm>"}`
24
+
25
+
26
+ 2. ICS returns a temporary:
27
+
28
+ ```json
29
+ { "api_key": "<token>" }
30
+ ```
31
+
32
+
33
+ 3. The configuration is fetched from:
34
+
35
+ ```bash
36
+ GET /api/v1/system/binary-configuration
37
+ ```
38
+
39
+ with:
40
+ - `api_key` as `username`
41
+ - `''` as `password`
42
+
43
+ ICS responds with a BASE64-encoded ZIP archive containing the device configuration.
44
+ The model stores this BASE64 value as a single uninterrupted line.
45
+
46
+
47
+ #### Required Node Configuration
48
+
49
+ In source (CSV, HTTP, SQL, etc.), simply define:
50
+
51
+ ```yaml
52
+ model: ivanti
53
+ username: <your username>
54
+ password: <your password>
55
+ vars:
56
+ realm: <your realm> # Optional, default = "Users"
57
+ ```
58
+
59
+ The model will automatically handle authentication and obtain the API key as stated above.
@@ -0,0 +1,10 @@
1
+ # OS6 Configuration
2
+
3
+ The commands Oxidized executes are:
4
+
5
+ 1. terminal length 0
6
+ 2. show version
7
+ 3. show interfaces transceiver properties
8
+ 4. show running-config
9
+
10
+ Back to [Model-Notes](README.md)
@@ -2,25 +2,6 @@
2
2
 
3
3
  This directory contains implementation notes and caveats to assist you in your oxidized deployment.
4
4
 
5
- Use the table below for more information on the Vendor/Model caveats.
6
-
7
- Vendor | Model |Updated
8
- ----------------|-----------------|----------------
9
- 3COM|[Comware](Comware.md)|15 Feb 2018
10
- AireOS|[AireOS](AireOS.md)|29 Nov 2017
11
- Arbor Networks|[ArbOS](ArbOS.md)|27 Feb 2018
12
- Arista|[EOS](EOS.md)|05 Feb 2018
13
- Cumulus|[Cumulus](Cumulus.md)|11 Jun 2018
14
- Huawei|[VRP](VRP-Huawei.md)|17 Nov 2017
15
- Huawei|[SmartAX series](SmartAX-Huawei.md)|21 Jan 2019
16
- Cisco IOS|[IOS](IOS.md)|29 Mar 2019
17
- Juniper|[MX/QFX/EX/SRX/J Series](JunOS.md)|18 Jan 2018
18
- Netgear|[Netgear](Netgear.md)|11 Apr 2018
19
- Nokia|[Nokia ISAM](Nokia.md)|22 Aug 2018
20
- Dell/EMC|[Dell EMC Networking OS10](OS10.md)|07 Dec 2021
21
- Viptela|[Viptela](Viptela.md)|1 Jul 2018
22
- Zyxel|[XGS4600 Series](XGS4600-Zyxel.md)|1 Feb 2018
23
- Linux|[LinuxGeneric](LinuxGeneric.md)|10 Jun 2019
24
- Lenovo|[Lenovo Network OS](LenovoNOS.md)|5 Apr 2022
5
+ The model notes are linked from the [list of supported OS types](/docs/Supported-OS-Types.md).
25
6
 
26
7
  If you discover additional caveats or problems please make sure to consult the [GitHub issues for oxidized](https://github.com/ytti/oxidized/issues) known issues.
@@ -0,0 +1,28 @@
1
+ # Mikrotik RouterOS Configuration
2
+
3
+ RouterOS 7.12 and later support ED25519 keys.
4
+
5
+ Create a key pair, save the public key (``id_ed25519.pub``) and save it on flash. Create a user
6
+ and attach the public key.
7
+
8
+ ```text
9
+ [admin@mikrotik] > /user add name=oxidized group=read disabled=no
10
+ [admin@mikrotik] > /user ssh-keys import public-key-file=id_ed25519.pub user=oxidized
11
+ ```
12
+
13
+ Oxidized can now retrieve your configuration!
14
+
15
+ ## Save significant changes only
16
+
17
+ You can [store the configuration only on significant changes](/docs/Configuration.md#store-configuration-only-on-significant-changes)
18
+ by setting the [variable](/docs/Configuration.md#options-credentials-vars-etc-precedence)
19
+ `output_store_mode` to `on_significant`. On RouterOS, this prevents Oxidized from saving a
20
+ new configuration version when only the system history has changed without any actual
21
+ configuration change.
22
+
23
+ ```yaml
24
+ vars:
25
+ output_store_mode: on_significant
26
+ ```
27
+
28
+ Back to [Model-Notes](README.md)
@@ -0,0 +1,7 @@
1
+ # Siklu Multihaul Terragraph Radio Line
2
+
3
+ The Siklu Multihaul TG radios use a different command set than the other Siklu radios.
4
+
5
+ To use this model, your source must designate the model as siklumhtg instead of siklu. It also requires that the MH-TG radio be running at least version 2.1.2.
6
+
7
+ Back to [Model-Notes](README.md)
@@ -0,0 +1,23 @@
1
+ # TrueNAS
2
+
3
+ This should support both older TrueNAS CORE (FreeBSD-based) and newer
4
+ TrueNAS SCALE (Linux-based) devices.
5
+
6
+ ## Authentication
7
+
8
+ Ensure that the user configured for oxidized to login to your device has the
9
+ permissions to read the configuration database. On older CORE instances, this
10
+ would just work without sudo. On newer devices, the `/data/freenas-v1.db` file
11
+ can only be read by the root user.
12
+
13
+ On SCALE devices with Apps support, it's also necessary to add some privileges
14
+ to read the container configurations for any apps you have installed, which can
15
+ be found under `/mnt/.ix-apps`.
16
+
17
+ You can make sure that the user that oxidized uses to login (`oxidized` in this
18
+ example) can dump the configuration using `sudo` by adding something like this
19
+ to your `/etc/sudoers` file:
20
+
21
+ ```
22
+ oxidized ALL=(ALL) NOPASSWD: /usr/bin/find /mnt/.ix-apps/app_configs *, /usr/bin/sqlite3 -readonly file\:/data/freenas-v1.db *
23
+ ```
@@ -5,6 +5,11 @@
5
5
  FTP access is only possible as admin, other users can login but cannot pull the files.
6
6
  For the XGS4600 series the config file is _config_ and not _config-0_
7
7
 
8
+ To enable FTP backup, uncomment the following line in _oxidized/lib/oxidized/model/zynos.rb_
9
+ ```text
10
+ # cmd 'config-0'
11
+ ```
12
+
8
13
  The following line in _oxidized/lib/oxidized/model/zynos.rb_ will need changing
9
14
 
10
15
  ```text
@@ -0,0 +1,219 @@
1
+ # Model Unit Tests
2
+ Oxidized includes automated unit tests for its models, which require very little
3
+ effort to use. There are three different default unit tests for models:
4
+ - [Device Simulation](ModelUnitTests.md#device-simulation)
5
+ - [Device Prompt](ModelUnitTests.md#device-prompt)
6
+ - [Secrets](ModelUnitTests.md#secrets)
7
+ - [Significant Changes](ModelUnitTests.md#significant-changes)
8
+
9
+ You only need to provide test files under [/spec/model/data](/spec/model/data),
10
+ and the tests will be run automatically with `rake test`. See
11
+ [CONTRIBUTING.md](/CONTRIBUTING.md) for explanations on how to set up a
12
+ development environment. In the following examples, we use
13
+ [Bundler](https://bundler.io/) and prefix all commands with `bundle exec`.
14
+
15
+ In addition, you can write [custom unit tests](ModelUnitTests.md#custom-tests)
16
+ for your model
17
+
18
+ ## Device Simulation
19
+ ### YAML Simulation File
20
+ You need a [YAML simulation file](/docs/DeviceSimulation.md) for your device.
21
+ See the link for instructions on how to produce it.
22
+
23
+ The YAML simulation files are stored under
24
+ [/spec/model/data/](/spec/model/data/), with the naming convention
25
+ `<model>#<description>#simulation.yaml`, where `<model>` is the lowercase name
26
+ of the Oxidized model and `<description>` is the name of the test case.
27
+ `<description>` is generally formatted as `<hardware>_<software>` or
28
+ `<hardware>_<software>_<information>`.
29
+
30
+ Using a correct name for the file is important to ensure it is included in
31
+ automatic model unit tests.
32
+
33
+ ### Expected Output
34
+ You need a second file that contains the expected output, which has the same
35
+ name as the YAML simulation file but ends with `#output.txt` instead of
36
+ `#simulation.yaml`.
37
+
38
+ You can automatically produce an output file based on the current model for all
39
+ YAML simulation files missing an `#output.txt`:
40
+ ```shell
41
+ bundle exec ruby spec/model/atoms_generate.rb
42
+ ```
43
+
44
+ In the following example,
45
+ `spec/model/data/aoscx#R8N85A-C6000-48G-CL4_PL.10.08.1010#output.txt` (the
46
+ second file in the list) was missing:
47
+
48
+ ```shell
49
+ oxidized$ bundle exec ruby spec/model/atoms_generate.rb
50
+ Run options: --seed 57811
51
+
52
+ # Running:
53
+
54
+ Generating output file for aoscx#R0X25A-6410_FL.10.10.1100#simulation... SKIP, output already exists
55
+ Generating output file for aoscx#R8N85A-C6000-48G-CL4_PL.10.08.1010#simulation... OK
56
+ Generating output file for arubainstant#IAP515_8.10.0.6_VWLC#simulation... SKIP, output already exists
57
+ Generating output file for asa#5512_9.12-4-67_single-context#simulation... SKIP, output already exists
58
+ Generating output file for garderos#R7709_003_006_068#simulation... SKIP, output already exists
59
+ Generating output file for ios#C8200L_16.12.1#simulation... FAIL, no simulation file
60
+ Generating output file for ios#C9200L-24P-4G_17.09.04a#simulation... SKIP, output already exists
61
+ Generating output file for ios#C9800-L-F-K9_17.06.05#simulation... SKIP, output already exists
62
+ Generating output file for ios#asr920_16.8.1b#simulation... SKIP, output already exists
63
+ Generating output file for junos#srx300_22.4#simulation... SKIP, output already exists
64
+ Generating output file for opnsense#nano_23.7#simulation... SKIP, output already exists
65
+ Generating output file for pfsense#CE_2.7.2#simulation... SKIP, output already exists
66
+ Generating output file for routeros#CHR_7.10.1#simulation... SKIP, output already exists
67
+ Generating output file for routeros#CHR_7.16#simulation... SKIP, output already exists
68
+ Generating output file for routeros#L009UiGS_7.15.2#simulation... SKIP, output already exists
69
+ .
70
+
71
+ Finished in 0.904792s, 1.1052 runs/s, 0.0000 assertions/s.
72
+
73
+ 1 runs, 0 assertions, 0 failures, 0 errors, 0 skips
74
+ Coverage report generated for RSpec to /home/xxx/oxidized/coverage/coverage.xml. 651 / 1122 LOC (58.02%) covered
75
+ Coverage report generated for RSpec to /home/xxx/oxidized/coverage.
76
+ Line Coverage: 58.02% (651 / 1122)
77
+ ```
78
+
79
+ ### Running the Tests
80
+ You can modify the `#output.txt` file to match your expectations and modify the
81
+ model accordingly. Run `bundle exec rake` to run the tests.
82
+
83
+ Here is an example when the output of the VTP command is missing in the expected
84
+ output:
85
+
86
+ ```
87
+ oxidized$ bundle exec rake test
88
+ /usr/bin/ruby3.1 -I"lib:spec" /home/xxx/oxidized/vendor/bundle/ruby/3.1.0/gems/rake-13.2.1/lib/rake/rake_test_loader.rb "spec/cli_spec.rb" "spec/config_spec.rb" "spec/hook/githubrepo_spec.rb" "spec/input/ssh_spec.rb" "spec/manager_spec.rb" "spec/model/apc_aos_spec.rb" "spec/model/model_atoms_spec.rb" "spec/model/model_helper_spec.rb" "spec/node_spec.rb" "spec/nodes_spec.rb" "spec/output/file_spec.rb" "spec/output/git_spec.rb" "spec/refinements_spec.rb" "spec/source/csv_spec.rb" "spec/source/http_spec.rb" "spec/source/jsonfile_spec.rb" "spec/source/sql_spec.rb"
89
+ Run options: --seed 31447
90
+
91
+ # Running:
92
+
93
+ ...............................................................SS..................F.............................SS..
94
+
95
+ Finished in 7.963602s, 14.6918 runs/s, 48.7217 assertions/s.
96
+
97
+ 1) Failure:
98
+ ATOMS tests#test_0006_ios#C9200L-24P-4G_17.09.04a#output has expected output [spec/model/model_atoms_spec.rb:12]:
99
+ --- expected
100
+ +++ actual
101
+ @@ -9,6 +9,21 @@
102
+ ! CPU: ARM64
103
+ ! Memory: nvram 2048K
104
+ !
105
+ +! VTP: VTP Version capable : 1 to 3
106
+ +! VTP: VTP version running : 1
107
+ +! VTP: VTP Domain Name : Oxidized
108
+ +! VTP: VTP Pruning Mode : Disabled (Operationally Disabled)
109
+ +! VTP: VTP Traps Generation : Disabled
110
+ +! VTP: Device ID : 40f0.7800.0000
111
+ +! VTP: Feature VLAN:
112
+ +! VTP: --------------
113
+ +! VTP: VTP Operating Mode : Transparent
114
+ +! VTP: Maximum VLANs supported locally : 1005
115
+ +! VTP: Number of existing VLANs : 10
116
+ +! VTP: Configuration Revision : 0
117
+ +! VTP: MD5 digest : 0x35 0x00 0x00 0x00 0x00 0x00 0x7F 0xB4
118
+ +! VTP: 0x07 0x00 0x00 0x00 0x00 0x00 0x09 0x6D
119
+ +!
120
+ ! NAME: \"c92xxL Stack\", DESCR: \"c92xxL Stack\"
121
+ ! PID: C9200L-24P-4G , VID: V01 , SN: JAE24FFFFFF
122
+ !
123
+
124
+
125
+ 117 runs, 388 assertions, 1 failures, 0 errors, 4 skips
126
+
127
+ You have skipped tests. Run with --verbose for details.
128
+ Coverage report generated for RSpec to /home/xxx/oxidized/coverage/coverage.xml. 2167 / 3131 LOC (69.21%) covered
129
+ Coverage report generated for RSpec to /home/xxx/oxidized/coverage.
130
+ Line Coverage: 69.21% (2167 / 3131)
131
+ rake aborted!
132
+ Command failed with status (1): [ruby -I"lib:spec" /home/xxx/oxidized/vendor/bundle/ruby/3.1.0/gems/rake-13.2.1/lib/rake/rake_test_loader.rb "spec/cli_spec.rb" "spec/config_spec.rb" "spec/hook/githubrepo_spec.rb" "spec/input/ssh_spec.rb" "spec/manager_spec.rb" "spec/model/apc_aos_spec.rb" "spec/model/model_atoms_spec.rb" "spec/model/model_helper_spec.rb" "spec/node_spec.rb" "spec/nodes_spec.rb" "spec/output/file_spec.rb" "spec/output/git_spec.rb" "spec/refinements_spec.rb" "spec/source/csv_spec.rb" "spec/source/http_spec.rb" "spec/source/jsonfile_spec.rb" "spec/source/sql_spec.rb" ]
133
+ /home/xxx/oxidized/vendor/bundle/ruby/3.1.0/gems/rake-13.2.1/exe/rake:27:in `<top (required)>'
134
+ Tasks: TOP => test
135
+ (See full trace by running task with --trace)
136
+ ```
137
+
138
+ If you want to be sure that your model has been tested, run
139
+ `bundle exec rake test TESTOPTS="--verbose"` and search for your models unter
140
+ `ATOMS tests`
141
+
142
+ ### Running only one test
143
+ If you want to run only one test while debuging your model, you can select it
144
+ with the option `--name=/regexp/`:
145
+ ```
146
+ bundle exec rake test TESTOPTS="--verbose --name=/ios#C9800.*output/"
147
+ ```
148
+
149
+ You can also set `Oxidized.asetus.cfg.debug = true` in
150
+ `spec/model/model_helper.rb` to activate debug logs.
151
+
152
+ ## Device Prompt
153
+ You can specify device prompts to test in a YAML file named
154
+ `spec/model/data/<model>#generic#prompt.yaml`.
155
+
156
+ The YAML file has three sections containing a list of prompts to test:
157
+ - pass: these prompts will pass the prompt regexp.
158
+ - pass_with_expect: these prompts will pass the prompt regexp after having been
159
+ cleaned by the expect commands.
160
+ - fail: these prompts will fail the prompt regexp.
161
+
162
+ Here is an example:
163
+ ```yaml
164
+ pass:
165
+ - "LAB-R1234_Garderos# "
166
+ pass_with_expect:
167
+ - "\e[4m\rLAB-R1234_Garderos#\e[m "
168
+ fail:
169
+ - "\e[4m\rLAB-R1234_Garderos#\e[m "
170
+ - "#LAB#"
171
+ ```
172
+
173
+ ## Secrets
174
+ You can test if the model effectively removes secrets from your YAML simulation
175
+ file with a YAML file named like the YAML simulation, but with the suffix
176
+ `#secret.yaml`.
177
+
178
+ The YAML file has two sections containing a list of strings to test:
179
+ - fail: the test will fail if the output contains these strings.
180
+ - pass: the test will pass only if the output contains these strings.
181
+
182
+ ```yaml
183
+ fail:
184
+ - 'AAAAAAAAAABBBBBBBBBBCCCCCCCCCC'
185
+ pass:
186
+ - 'snmp-server host 10.10.42.12 version 2c <secret removed> inform'
187
+ - 'hash-mgmt-user oxidized password hash <secret removed>'
188
+ - 'hash-mgmt-user rocks password hash <secret removed> usertype read-only'
189
+ ```
190
+
191
+ ## Significant Changes
192
+ You can test if the model correctly detects significant changes from a YAML
193
+ simulation file (`#simulation.yaml`) when run with variable
194
+ `output_store_mode` set to `on_significant`.
195
+
196
+ The output is checked against a file with the same
197
+ prefix as the yaml simulation file, but with the suffix
198
+ `#significant_changes.yaml`.
199
+
200
+ The `#significant_changes.yaml` file contains two sections with a list of
201
+ strings or regular expressions to test:
202
+ - pass: the test passes only if the output contains these strings (significant changes).
203
+ - fail: the test fails if the output contain these strings (non-significant changes).
204
+
205
+ ```yaml
206
+ pass:
207
+ - "! Processor ID: FCL2XXXXXXX"
208
+ fail:
209
+ - "! Last configuration change at 13:57:08 CET Wed Mar 13 2024"
210
+ - "! NVRAM config last updated at 15:26:39 CET Wed Mar 13 2024 by oxidized"
211
+ ```
212
+
213
+ ## Custom tests
214
+ When you write custom tests for your models, please do not use the filenames
215
+ mentioned above, as it will interfere with the standard tests. If you need to
216
+ store a custom simulation file, use `model#description#custom_simulation.yaml`.
217
+
218
+ The [cumulus test](/spec/model/cumulus_spec.rb) is an example of a custom
219
+ test.
data/docs/Outputs.md CHANGED
@@ -10,9 +10,48 @@ output:
10
10
  directory: /var/lib/oxidized/configs
11
11
  ```
12
12
 
13
+ ### Groups
14
+ If you use groups, the nodes will be stored in directories named after the
15
+ groups. The directories are stored one level above the directory for configurations
16
+ without groups.
17
+
18
+ Example:
19
+ ```
20
+ /var/lib/oxidized/
21
+ + configs/ # Configurations of groupless nodes
22
+ + group1/ # Configurations of nodes in group1
23
+ + group2/ # Configurations of nodes in group2
24
+ ```
25
+
26
+ ### Clean obsolete nodes
27
+ The `file` output can automatically remove the configuration of nodes no
28
+ longer present in the [source](Sources.md).
29
+
30
+ > :warning: **Warning:** this might be a dangerous operation: oxidized
31
+ > will remove **any** file not matching the hostname of the nodes configured
32
+ > in the source.
33
+
34
+ When using groups, it will remove any files not matching the hostnames of the
35
+ nodes from the groups directories (which are on the same level as the default
36
+ directory). As a safety measure, oxidized will only clean configuration out of
37
+ active groups. If the group `example` isn't used anymore, oxidized won't clean
38
+ the configurations out of the directory `../example/`.
39
+
40
+ Configuration:
41
+
42
+ ```yaml
43
+ output:
44
+ default: file
45
+ clean_obsolete_nodes: true
46
+ file:
47
+ directory: "~/.config/oxidized/configs/default"
48
+ ```
49
+
50
+
13
51
  ## Output: Git
14
52
 
15
- This uses the rugged/libgit2 interface. So you should remember that normal Git hooks will not be executed.
53
+ This uses the rugged/libgit2 interface. So you should remember that normal Git
54
+ hooks will not be executed.
16
55
 
17
56
  For a single repository containing all devices:
18
57
 
@@ -63,6 +102,50 @@ output:
63
102
 
64
103
  ```
65
104
 
105
+ ### Git performance issues with large device counts
106
+ When you use git to store your configurations, the size of your repository will
107
+ grow over time. This growth may lead to performance issues. If you encounter
108
+ such issues, you should perform a Git garbage collection on your repository.
109
+
110
+ Follow these steps to do so:
111
+
112
+ 1. Stop oxidized (no one should access the git repository while running garbage
113
+ collection)
114
+ 2. Make a backup of your oxidized data, especially the Git repository
115
+ 3. Change directory your oxidized git repository (as configured in oxidized
116
+ configuration file)
117
+ 4. Execute the command `git gc` to run the garbage collection
118
+ 5. Restart oxidized - you're done!
119
+
120
+
121
+ ### Clean obsolete nodes
122
+ The `git` output can automatically remove the configuration of nodes no
123
+ longer present in the [source](Sources.md).
124
+
125
+ > :warning: **Limitations**
126
+ > - this currently only works with `single_repo: true`
127
+ > - it will ignore configurations saved as [output types](#output-types) in
128
+ > a separate repository.
129
+ > - oxidized will refuse to remove old configurations
130
+ > when saving [output types](#output-types) in a subdirectory of the git
131
+ > repository (`type_as_directory: true`), or it would remove the output
132
+ > type directories
133
+
134
+ Oxidized will remove **any** file within the git repository not matching the
135
+ group and hostname of the nodes configured in the source and will then commit
136
+ the change into git.
137
+
138
+ Configuration:
139
+
140
+ ```yaml
141
+ output:
142
+ default: git
143
+ clean_obsolete_nodes: true
144
+ git:
145
+ single_repo: true
146
+ repo: "~/.config/oxidized/devices.git"
147
+ ```
148
+
66
149
  ## Output: Git-Crypt
67
150
 
68
151
  This uses the gem git and system git-crypt interfaces. Have a look at [GIT-Crypt](https://www.agwa.name/projects/git-crypt/) documentation to know how to install it.
@@ -128,9 +211,9 @@ output:
128
211
 
129
212
  Please note that user list is only updated once at creation.
130
213
 
131
- ## Output: Http
214
+ ## Output: HTTP
132
215
 
133
- The HTTP output will POST a config to the specified HTTP URL. Basic username/password authentication is supported.
216
+ The HTTP output will POST a config as JSON to the specified HTTP URL. It supports HTTP Basic Authentication, custom headers, and SSL/TLS verification control.
134
217
 
135
218
  Example HTTP output configuration:
136
219
 
@@ -138,11 +221,25 @@ Example HTTP output configuration:
138
221
  output:
139
222
  default: http
140
223
  http:
141
- user: admin
142
- password: changeit
143
224
  url: "http://192.168.162.50:8080/db/coll"
225
+ user: admin # Optional - for HTTP basic auth
226
+ password: changeit # Optional - for HTTP basic auth
227
+ ssl_verify: false # Optional - verify SSL certs (default: false)
228
+ headers: # Optional - custom HTTP headers
229
+ X-Custom-Header: "value"
230
+ X-API-Key: "secret"
144
231
  ```
145
232
 
233
+ ### Configuration Options
234
+
235
+ | Option | Required | Description |
236
+ |--------------|----------|---------------------------------------------------------|
237
+ | `url` | Yes | Full HTTP/HTTPS URL to POST the config to |
238
+ | `user` | No | Username for HTTP Basic Authentication |
239
+ | `password` | No | Password for HTTP Basic Authentication |
240
+ | `ssl_verify` | No | When `true`, verify SSL certificates (default: `false`) |
241
+ | `headers` | No | Hash of custom HTTP headers to include in the request |
242
+
146
243
  ## Output types
147
244
 
148
245
  If you prefer to have different outputs in different files and/or directories, you can easily do this by modifying the corresponding model. To change the behaviour for IOS, you would edit `lib/oxidized/model/ios.rb` (run `gem contents oxidized` to find out the full file path).
data/docs/Release.md ADDED
@@ -0,0 +1,102 @@
1
+ # How to release a new version of Oxidized?
2
+ This document is targeted at oxidized maintainers. It describes the release process.
3
+
4
+ ## Version numbering
5
+ Oxidized versions are numbered like major.minor.patch
6
+ - currently, the major version is 0.
7
+ - minor is incremented when releasing new features.
8
+ - patch is incremented when releasing fixes only.
9
+
10
+ ## Create a release branch
11
+ Name the release branch `release/0.xx.yy`
12
+
13
+ ## Review changes
14
+ Run `git diff 0.30.0` (where `0.30.0` is to be changed to the last release) and review
15
+ all the changes that have been done. Have a specific look at changes you don't understand.
16
+
17
+ For a graphical compare, use `git difftool -d 0.30.0`.
18
+
19
+ Commit fixes to the release branch
20
+
21
+ ## Update the gem dependencies to the latest versions
22
+ ```
23
+ bundle outdated
24
+ bundle update
25
+ bundle outdated
26
+ ```
27
+
28
+ ## Update rubocup .rubocop_todo.yml
29
+ Run `bundle exec rubocop --auto-gen-config`,
30
+ and make sure `bundle exec rake` passes after it.
31
+
32
+ If you change some code => Restart the release process at the beginning ;-)
33
+
34
+ ## Make sure the file permissions are correct
35
+ Run `bundle exec rake chmod`
36
+
37
+ ## Test !
38
+ Test the git code and the container against as much device types and
39
+ environments as you can.
40
+
41
+ ## Bump the version
42
+ Update CHANGELOG.md:
43
+ - review it
44
+ - add release notes
45
+ - set the new version (replace `[Unreleased]` with `[0.xx.yy – 202Y-MM-DD]`)
46
+
47
+ Change the version in `lib/oxidized/version.rb`
48
+
49
+ Upload the branch to github, make a Pull Request for it.
50
+
51
+ ## Make sure you pass all GitHub CI
52
+ They test different ruby versions an run security checks on the code (codeql).
53
+
54
+ ## Prepare the release in your working repository
55
+ 1. Merge the Pull Request into master with the commit message
56
+ `chore(release): release version 0.3x.y`
57
+ 2. `git pull` on master
58
+ 3. Tag the commit with `git tag -a 0.xx.yy -m "Release 0.xx.yy"` or `rake tag`
59
+ 4. Build the gem with ‘rake build’
60
+ 5. Run `git diff` to check if there have been more changes (there shouldn't)
61
+ 6. Install an test the gem locally
62
+ ```shell
63
+ gem install --user-install pkg/oxidized-0.xx.yy.gem
64
+ ~/.local/share/gem/ruby/3.3.0/bin/oxidized
65
+ ```
66
+
67
+ ## Release in github
68
+ Push the tag to github:
69
+ ```
70
+ git push origin 0.xx.yy
71
+ ```
72
+
73
+ Make a release from the tag in github.
74
+ - Name the release `0.xx.yy`
75
+ - Generate release notes
76
+ - Remove `## What's changed`
77
+ - Take the release notes from CHANGELOG.md
78
+ - List new contributors (generated automatically)
79
+ - Keep the Full Changelog (generated automatically)
80
+ - Preview
81
+ - Publish
82
+
83
+ Close the corresponding milestone in github.
84
+
85
+ ## Release in rubygems
86
+ Push the gem with ‘rake push’
87
+
88
+ ## Release in docker.io
89
+ The OCI-Containter is automatically build and pushed to docker.io by github
90
+
91
+ ## Update CHANGELOG.md for next release
92
+ Add
93
+ ```
94
+ ## [Unreleased]
95
+
96
+ ### Added
97
+
98
+ ### Changed
99
+
100
+ ### Fixed
101
+
102
+ ```