oxidized 0.32.1 → 0.34.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (127) hide show
  1. checksums.yaml +4 -4
  2. data/.github/ISSUE_TEMPLATE/bug_report.md +45 -0
  3. data/.github/ISSUE_TEMPLATE/feature_request.md +22 -0
  4. data/.github/ISSUE_TEMPLATE/support-request.md +39 -0
  5. data/.github/workflows/publishdocker.yml +35 -16
  6. data/.github/workflows/ruby.yml +4 -2
  7. data/.gitignore +2 -0
  8. data/.rubocop.yml +29 -8
  9. data/.rubocop_todo.yml +1 -60
  10. data/CHANGELOG.md +103 -2
  11. data/CONTRIBUTING.md +20 -10
  12. data/Dockerfile +37 -64
  13. data/README.md +47 -141
  14. data/Rakefile +9 -11
  15. data/docs/Configuration.md +236 -27
  16. data/docs/DeviceSimulation.md +19 -7
  17. data/docs/Docker.md +245 -0
  18. data/docs/Issues.md +27 -1
  19. data/docs/Model-Notes/EatonNetwork.md +18 -0
  20. data/docs/Model-Notes/HPEAruba.md +3 -2
  21. data/docs/ModelUnitTests.md +35 -25
  22. data/docs/Outputs.md +83 -2
  23. data/docs/Release.md +34 -24
  24. data/docs/Supported-OS-Types.md +7 -0
  25. data/docs/Troubleshooting.md +4 -13
  26. data/extra/device2yaml.rb +24 -9
  27. data/extra/rest_client.rb +3 -2
  28. data/extra/syslog.rb +8 -3
  29. data/lib/oxidized/cli.rb +7 -3
  30. data/lib/oxidized/config/vars.rb +22 -14
  31. data/lib/oxidized/config.rb +3 -2
  32. data/lib/oxidized/core.rb +30 -8
  33. data/lib/oxidized/hook/ciscosparkdiff.rb +11 -9
  34. data/lib/oxidized/hook/exec.rb +5 -4
  35. data/lib/oxidized/hook/githubrepo.rb +23 -17
  36. data/lib/oxidized/hook/noophook.rb +2 -2
  37. data/lib/oxidized/hook/slackdiff.rb +9 -8
  38. data/lib/oxidized/hook/xmppdiff.rb +9 -9
  39. data/lib/oxidized/hook.rb +10 -8
  40. data/lib/oxidized/input/cli.rb +8 -3
  41. data/lib/oxidized/input/exec.rb +1 -1
  42. data/lib/oxidized/input/ftp.rb +2 -2
  43. data/lib/oxidized/input/http.rb +6 -6
  44. data/lib/oxidized/input/input.rb +1 -0
  45. data/lib/oxidized/input/scp.rb +2 -2
  46. data/lib/oxidized/input/ssh.rb +21 -14
  47. data/lib/oxidized/input/telnet.rb +3 -3
  48. data/lib/oxidized/input/tftp.rb +1 -1
  49. data/lib/oxidized/job.rb +7 -4
  50. data/lib/oxidized/logger.rb +51 -0
  51. data/lib/oxidized/model/acos.rb +1 -0
  52. data/lib/oxidized/model/aos7.rb +9 -0
  53. data/lib/oxidized/model/aoscx.rb +2 -0
  54. data/lib/oxidized/model/aosw.rb +22 -17
  55. data/lib/oxidized/model/aricentiss.rb +2 -2
  56. data/lib/oxidized/model/asa.rb +3 -3
  57. data/lib/oxidized/model/awplus.rb +13 -10
  58. data/lib/oxidized/model/eatonnetwork.rb +65 -0
  59. data/lib/oxidized/model/edgecos.rb +2 -1
  60. data/lib/oxidized/model/edgeos.rb +7 -6
  61. data/lib/oxidized/model/edgeswitch.rb +3 -1
  62. data/lib/oxidized/model/efos.rb +41 -0
  63. data/lib/oxidized/model/eltex.rb +1 -1
  64. data/lib/oxidized/model/fabricos.rb +1 -1
  65. data/lib/oxidized/model/fastiron.rb +3 -1
  66. data/lib/oxidized/model/firelinuxos.rb +12 -3
  67. data/lib/oxidized/model/fortios.rb +5 -4
  68. data/lib/oxidized/model/gaiaos.rb +4 -4
  69. data/lib/oxidized/model/ingate.rb +47 -0
  70. data/lib/oxidized/model/ios.rb +16 -5
  71. data/lib/oxidized/model/ironware.rb +1 -1
  72. data/lib/oxidized/model/junos.rb +4 -0
  73. data/lib/oxidized/model/linksyssrw.rb +3 -3
  74. data/lib/oxidized/model/mlnxos.rb +14 -7
  75. data/lib/oxidized/model/model.rb +4 -3
  76. data/lib/oxidized/model/netgear.rb +8 -0
  77. data/lib/oxidized/model/nsxdfw.rb +2 -1
  78. data/lib/oxidized/model/nsxfirewall.rb +2 -1
  79. data/lib/oxidized/model/nxos.rb +2 -2
  80. data/lib/oxidized/model/openwrt.rb +6 -6
  81. data/lib/oxidized/model/powerconnect.rb +31 -10
  82. data/lib/oxidized/model/procurve.rb +3 -1
  83. data/lib/oxidized/model/qtech.rb +3 -1
  84. data/lib/oxidized/model/quantaos.rb +8 -6
  85. data/lib/oxidized/model/routeros.rb +3 -2
  86. data/lib/oxidized/model/saos10.rb +38 -0
  87. data/lib/oxidized/model/sixwind.rb +28 -0
  88. data/lib/oxidized/model/sonicos.rb +1 -1
  89. data/lib/oxidized/model/srosmd.rb +1 -1
  90. data/lib/oxidized/model/supermicro.rb +1 -1
  91. data/lib/oxidized/model/timos.rb +1 -1
  92. data/lib/oxidized/model/tmos.rb +1 -0
  93. data/lib/oxidized/model/tnsr.rb +53 -0
  94. data/lib/oxidized/model/trango.rb +3 -1
  95. data/lib/oxidized/model/unifiap.rb +144 -0
  96. data/lib/oxidized/model/vrp.rb +3 -1
  97. data/lib/oxidized/model/xos.rb +3 -1
  98. data/lib/oxidized/model/zhoneolt.rb +3 -1
  99. data/lib/oxidized/model/zynos.rb +3 -3
  100. data/lib/oxidized/node.rb +44 -27
  101. data/lib/oxidized/nodes.rb +8 -4
  102. data/lib/oxidized/output/file.rb +28 -0
  103. data/lib/oxidized/output/git.rb +148 -41
  104. data/lib/oxidized/output/gitcrypt.rb +18 -13
  105. data/lib/oxidized/output/http.rb +5 -4
  106. data/lib/oxidized/output/output.rb +14 -0
  107. data/lib/oxidized/source/http.rb +4 -2
  108. data/lib/oxidized/version.rb +6 -4
  109. data/lib/oxidized/worker.rb +13 -13
  110. data/lib/oxidized.rb +3 -24
  111. data/lib/refinements.rb +2 -0
  112. data/oxidized.gemspec +10 -8
  113. metadata +74 -41
  114. data/examples/podman-compose/Makefile +0 -103
  115. data/examples/podman-compose/README.md +0 -94
  116. data/examples/podman-compose/docker-compose.yml +0 -30
  117. data/examples/podman-compose/gitserver/.gitignore +0 -1
  118. data/examples/podman-compose/gitserver/Dockerfile +0 -14
  119. data/examples/podman-compose/model-simulation/Dockerfile-model +0 -13
  120. data/examples/podman-compose/model-simulation/asternos.sh +0 -36
  121. data/examples/podman-compose/oxidized-config/.gitignore +0 -10
  122. data/examples/podman-compose/oxidized-config/config +0 -46
  123. data/examples/podman-compose/oxidized-config/config_csv-file +0 -46
  124. data/examples/podman-compose/oxidized-config/config_csv-gitserver +0 -56
  125. data/examples/podman-compose/oxidized-config/router.db +0 -1
  126. data/examples/podman-compose/oxidized-ssh/.gitignore +0 -1
  127. data/examples/podman-compose/oxidized-ssh/README.md +0 -14
data/docs/Docker.md ADDED
@@ -0,0 +1,245 @@
1
+ # Running oxidized within an OCI container (docker, podman...)
2
+
3
+ ## Docker image
4
+ The official Docker image is automatically built and pushed to hub.docker.com
5
+ as [oxidized/oxidized](https://hub.docker.com/r/oxidized/oxidized/) with a
6
+ [GitHub CI](/.github/workflows/publishdocker.yml).
7
+
8
+ There are three different types of tags:
9
+ - Each commit to the master branch will be published with the tag
10
+ `master-(git sha oid)`
11
+ - Each release will be published with the version as a tag
12
+ - Latest is the latest release, either from a commit or a release tag
13
+
14
+ Currently, Docker Hub automatically builds the master branch for linux/amd64 and
15
+ linux/arm64 platforms as
16
+ [oxidized/oxidized](https://hub.docker.com/r/oxidized/oxidized/). You can make
17
+ use of this container or build your own.
18
+
19
+ ## Choose a container running environment
20
+ There are many options to run containers. Two main options are
21
+ [docker](https://www.docker.com/) and [podman](https://podman.io/). A main
22
+ difference is that docker requires root rights to run, and podman can be run
23
+ by a local user. Both work with oxidized, so the choice is up to you.
24
+
25
+ Oxidized has also been reported to work with
26
+ [Portainer](https://www.portainer.io/).
27
+
28
+ ## File rights in the container userspace and host userspace
29
+ As oxidized runs under the user "oxidized" (UID: 30000) in the container
30
+ userspace, docker and podman will map this UID in the shared volumes, producing
31
+ weird UIDs in the host userspace.
32
+
33
+ ### docker
34
+ When docker runs the container as root, the mapping to the UIDs in the host
35
+ userspace will be the same as in the container, so the files produced by the
36
+ oxidized user in the container will have UID 30000 in the host.
37
+
38
+ If you map a volume between the host and the container and need it to be
39
+ accessed by the oxidized user, you need to fix the UIDs:
40
+ ```
41
+ sudo chown 30000:30000 ~/oxidized-config
42
+ ```
43
+
44
+ ### podman
45
+ When podman is run as a user, the mapping of UIDs between the container and the
46
+ linux host will depend on your UID on the host.
47
+
48
+ If you map a volume between the host and the container and need it to be
49
+ accessed by the oxidized user, you need to fix the UIDs:
50
+
51
+ ```
52
+ podman unshare chown 30000:30000 ~/oxidized-config
53
+ ```
54
+
55
+ If you need to access the files from the linux host, you can do this by
56
+ prefixing `podman unshare` to your shell commands.
57
+
58
+ ## Build you own container image
59
+ To build your own container image, clone the git repository:
60
+
61
+ ```shell
62
+ git clone https://github.com/ytti/oxidized
63
+ ```
64
+
65
+ Then, build the container locally:
66
+
67
+ ```shell
68
+ sudo docker build -q -t oxidized/oxidized:latest oxidized/
69
+ ```
70
+
71
+ - `-q` stands for quiet; remove it if you want to see the build process.
72
+ - `-t oxidized/oxidized:latest` tags the image as `oxidized/oxidized:latest`
73
+
74
+ You can also build with podman:
75
+ ```
76
+ podman build -t oxidized:latest oxidized/
77
+ ```
78
+
79
+ Within the oxidized repository, using `rake build_container` will automatically
80
+ build the container (with podman or docker), name it `localhost/oxidized` and
81
+ give it the tags `latest` and `<branchname>-<sha-tag>`, for example
82
+ `localhost/oxidized:master-65baab9`.
83
+
84
+ ## Set up an environment for the container
85
+ Once you've built the container (or chosen to make use of the automatically
86
+ built container in Docker Hub, which will be downloaded for you by docker on the
87
+ first `run` command had you not built it), you need to set up an environment
88
+ for the container.
89
+
90
+ First, you need a configuration directory in the host system that you can map
91
+ in the container. You can choose any directory you want, we'll take
92
+ `~/oxidized-config` in our example. Don't forget to adjust the permissions as
93
+ explained above.
94
+
95
+ If you already have a configuration for oxidized (`config`), you can skip this
96
+ step. Just save it under `~/oxidized-config` and run the container (see below).
97
+
98
+ If you don't have a configuration, you can make oxidized produce one for you, so
99
+ that you just have to adapt it to your needs.
100
+
101
+ ```shell
102
+ sudo docker run --rm -v ~/oxidized-config:/home/oxidized/.config/oxidized docker.io/oxidized/oxidized:latest su - oxidized -c oxidized
103
+ ```
104
+ ```shell
105
+ podman run --rm -v ~/oxidized-config:/home/oxidized/.config/oxidized docker.io/oxidized/oxidized:latest su - oxidized -c oxidized
106
+ ```
107
+
108
+ - `--rm` tells docker to automatically remove the container when he exits
109
+ - `-v ~/oxidized-config:/home/oxidized/.config/oxidized` maps your local
110
+ `~/oxidized-config` into `/home/oxidized/.config/oxidized`in the container
111
+ environment.
112
+ - `su - oxidized -c oxidized` runs oxidized under the user oxidized, so that it
113
+ can produce a configuration under `/home/oxidized/.config/oxidized`
114
+
115
+
116
+ This will return `edit /home/oxidized/.config/oxidized/config`, which is the
117
+ path in the container context. Now you can edit `~/oxidized-config/config` to
118
+ fit your needs.
119
+
120
+ You can reiterate this process a few times, until oxidized is happy with the
121
+ config, an then you're finished with setting up the environment.
122
+
123
+
124
+ You also need to create the `router.db` file under
125
+ `~/oxidized-config/config/router.db` (see
126
+ [CSV Source](/docs/Sources.md#source-csv) for further info) or configure another
127
+ source to suit your needs. Don't forget to set the file permissions (owner)
128
+ properly!
129
+
130
+
131
+
132
+ ## Run the container
133
+ Now you can run the container without specifying an entry point. It will
134
+ automatically start oxidized and every other process needed.
135
+ ```shell
136
+ sudo docker run --rm -v ~/oxidized-config:/home/oxidized/.config/oxidized -p 8888:8888/tcp docker.io/oxidized/oxidized:latest
137
+ ```
138
+ ```shell
139
+ podman run --rm -v ~/oxidized-config:/home/oxidized/.config/oxidized -p 8888:8888/tcp docker.io/oxidized/oxidized:latest
140
+ ```
141
+
142
+ `-p 8888:8888/tcp` maps the TCP port 8888 in the container with the port
143
+ 8888 on the host, so that you can access the RESTful API and Web Interface
144
+ from the host.
145
+ If the RESTful API and Web Interface should be enabled, edit the
146
+ configuration (in our example `~/oxidized-config/config`) and modify
147
+ `rest: 127.0.0.1:8888` to `rest: 0.0.0.0:8888`. This will bind port 8888 to all
148
+ interfaces, and expose the port so that it can be accessed externally.
149
+ [(Issue #445)](https://github.com/ytti/oxidized/issues/445)
150
+
151
+
152
+ ## Run with with docker-compose / podman-compose
153
+ Alternatively, you can use docker-compose or podman-compose to run the
154
+ container:
155
+
156
+ ```yaml
157
+ # docker-compose.yml
158
+ # docker-compose file example for oxidized that will start along with docker daemon
159
+ ---
160
+ version: "3"
161
+ services:
162
+ oxidized:
163
+ restart: always
164
+ image: docker.io/oxidized/oxidized:latest
165
+ ports:
166
+ - 8888:8888/tcp
167
+ environment:
168
+ # Reload hosts list once per day
169
+ CONFIG_RELOAD_INTERVAL: 86400
170
+ volumes:
171
+ - ~/oxidized-config/config:/home/oxidized/.config/oxidized/
172
+ ```
173
+
174
+ To start the pod, use `docker-compose up` or `podman-compose down`.
175
+
176
+ ## Special configurations of the official container
177
+ ### Reload the configuration
178
+ If you want to have the config automatically reloaded (e.g. when using a http
179
+ source that changes), you need to set the environment variable
180
+ CONFIG_RELOAD_INTERVAL. This can be done in `docker-compose.yml` (see above) or
181
+ on the command line:
182
+
183
+ ```shell
184
+ sudo docker run -v ~/oxidized-config:/home/oxidized/.config/oxidized -p 8888:8888/tcp -e CONFIG_RELOAD_INTERVAL=3600 docker.io/oxidized/oxidized:latest
185
+ ```
186
+ ### Use an internal CA
187
+ If you need to use an internal CA (e.g. to connect to an private github instance):
188
+
189
+ ```shell
190
+ docker run -v /etc/oxidized:/home/oxidized/.config/oxidized -v /path/to/MY-CA.crt:/usr/local/share/ca-certificates/MY-CA.crt -p 8888:8888/tcp -e UPDATE_CA_CERTIFICATES=true -t oxidized/oxidized:latest
191
+ ```
192
+
193
+ ### Pass the ssh passphrase for a remote git
194
+ If you don't want to authenticate with user & password but with a ssh-key, you
195
+ can set the ssh passphrase with the environment variable
196
+ `OXIDIZED_SSH_PASSPHRASE`
197
+
198
+ ## Tipps & tricks
199
+ ### podman & Debian Bookworm
200
+ To install podman in Debian Bookwork, you need following packages:
201
+ ```shell
202
+ sudo apt install podman containers-storage podman-compose
203
+ ```
204
+
205
+ Ensure Podman is using the overlay driver for image storage.
206
+ Without this driver, Podman may save every container layer separately rather
207
+ than only the changes, which can quickly consume disk space.
208
+
209
+ This issue can occur if podman was run before installing the
210
+ `container-storage` package.
211
+
212
+ ```shell
213
+ podman info | grep graphDriverName
214
+ ```
215
+
216
+ You should get this reply
217
+ ```shell
218
+ graphDriverName: overlay
219
+ ```
220
+
221
+ If not, a quick way to solve it is to delete `~/.local/share/containers/`.
222
+ Beware - this will delete **all** your containers!
223
+
224
+ ### Store the ssh keys a remote git repository
225
+ When you use the githubrepo hook to upload your configs to a remote git
226
+ repository, you have to store your ssh-key and the public keys of the remote
227
+ server. Create a directory `~/oxidized-ssh` and map it to `/home/oxidized/.ssh`.
228
+
229
+
230
+ To generate an ssh-key, run:
231
+ ```shell
232
+ ssh-keygen -q -t ed25519 -C "Oxidized Push Key@`hostname`" -N "YOURPASSPHRASE" -m PEM -f ~/oxidized-ssh/oxidized-key
233
+ ```
234
+
235
+ You also need to store the public keys of the remote git server in known_hosts.
236
+ If you don't store the keys, oxidized will refuse to push to the remote Git with
237
+ the error
238
+ `#<Rugged::SshError: invalid or unknown remote ssh hostkey>`, see Issue #2753.
239
+
240
+ ```shell
241
+ ssh-keyscan git-server.example.com > ~/oxidized-ssh/known_hosts
242
+ ```
243
+
244
+ Don't forget to set the permission (owner) of the files for the user oxidized
245
+ inside the container, or this will not work!
data/docs/Issues.md CHANGED
@@ -7,10 +7,28 @@ This guide provides tips on writing your issue to make it easier for the
7
7
  community and developers to understand and respond effectively.
8
8
 
9
9
  Why write good issues?
10
- - A clear and detailed issue improves the chances of getting your problem resolved.
10
+ - A clear and detailed issue improves the chances of getting your problem
11
+ resolved.
11
12
  - By spending time to write a good issue, you save developers time, contributing
12
13
  to Oxidized’s progress without writing a line of code.
13
14
 
15
+ ## Rules
16
+ The project receives too many incomplete issues, unnecessarily taking time that
17
+ could be invested in new code. Therefore, issues will be worked on with the
18
+ following rules:
19
+
20
+ - Use the predefined templates for bugs, feature requests and support requests.
21
+ - If you don't provide the necessary information (read this file, fill in the
22
+ questions in the templates), expect your issue to be closed without a comment.
23
+ - Inactive issues will be marked "stale" automatically after 90 days. Issues
24
+ are not closed automatically; this is a manual action by a maintainer.
25
+ - A feature request may be closed after some time of inactivity, as obviously
26
+ no one has found the time to implement it. Consider contributing code in a
27
+ Pull Request instead.
28
+ - While it is OK to ask for help (using the support request template), don't be
29
+ disappointed if no one finds the time to answer your question. Stale questions
30
+ (after 90 days) will be closed without a comment.
31
+
14
32
  ## Submit to the correct project
15
33
  Choose the appropriate GitHub project based on your issue:
16
34
 
@@ -24,6 +42,13 @@ Choose the appropriate GitHub project based on your issue:
24
42
  - For issues with Oxidized itself, go to
25
43
  [oxidized](https://github.com/ytti/oxidized).
26
44
 
45
+ ## Use the latest version
46
+ If you are using an old version of oxidized, you may encounter issues that have
47
+ been solved. No support will be provided for older versions of oxidized.
48
+
49
+ If you can, please also test against the latest git version, or at least read
50
+ [CHANGELOG.md](/CHANGELOG.md) to see if your problem has been solved on master.
51
+
27
52
  ## Format your issue
28
53
  - Use [GitHub Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax) to format your issue.
29
54
  - Preview your text before submitting to ensure it renders correctly.
@@ -36,6 +61,7 @@ Keep the title brief yet descriptive. Aim to summarize the main issue or request
36
61
  Include as many relevant details as possible. At a minimum, specify:
37
62
 
38
63
  - Oxidized version and operating system.
64
+ - Which model (oxidized name AND Manufacturer name) is the issue about.
39
65
  - Relevant parts of your Oxidized configuration and a brief explanation of your setup.
40
66
  - Output of the error, if relevant.
41
67
  - For issues related to specific devices, consider creating a YAML Simulation file (instructions below).
@@ -0,0 +1,18 @@
1
+ # EatonNetwork Configuration
2
+
3
+ This model uses the command `save_configuration -p <passphrase>` to get the backup config. The `-p` option is a required passphrase used to encrypted sensitive parts of the config data, the encrypted data is nondeterministic and changes with each run. The passphrase used is the auth password for the node.
4
+
5
+ See the [Eaton Network-M3 user's guide](https://www.eaton.com/content/dam/eaton/products/backup-power-ups-surge-it-power-distribution/power-management-software-connectivity/eaton-gigabit-network-card/network-m3/resources/eaton-network-m3-user-guide.pdf) section 7.7.14 (page 260) for more information.
6
+
7
+ To not have the backup change on each for all `eatonnetwork` node run set a model var in the config for the `eatonnetwork` model to [remove secrets](../Configuration.md#removing-secrets):
8
+
9
+ ```yaml
10
+ models:
11
+ eatonnetwork:
12
+ vars:
13
+ remove_secret: true
14
+ ```
15
+
16
+ See the [Eaton Network-M3 user's guide](https://www.eaton.com/content/dam/eaton/products/backup-power-ups-surge-it-power-distribution/power-management-software-connectivity/eaton-gigabit-network-card/network-m3/resources/eaton-network-m3-user-guide.pdf) section 3.20 (page 111) for details on JSON configuration structure, and restoring without sensitive/secrets.
17
+
18
+ Back to [Model-Notes](README.md)
@@ -26,6 +26,7 @@ is the operating system for the newer CX-Series.
26
26
  The Oxidized model is [aoscx](/lib/oxidized/model/aoscx.rb).
27
27
 
28
28
  ## Older Models
29
- Older Devices like ProCurve or 3Com/Comware are listed under the Vendor "HP" in
30
- the [Supported OS Types](docs/Supported-OS-Types.md) list.
29
+ Older Devices like [ProCurve](/lib/oxidized/model/procurve.rb) or 3Com/Comware
30
+ are listed under the Vendor "HP" in the
31
+ [Supported OS Types](/docs/Supported-OS-Types.md) list.
31
32
 
@@ -21,7 +21,7 @@ See the link for instructions on how to produce it.
21
21
 
22
22
  The YAML simulation files are stored under
23
23
  [/spec/model/data/](/spec/model/data/), with the naming convention
24
- `<model>:<description>:simulation.yaml`, where `<model>` is the lowercase name
24
+ `<model>#<description>#simulation.yaml`, where `<model>` is the lowercase name
25
25
  of the Oxidized model and `<description>` is the name of the test case.
26
26
  `<description>` is generally formatted as `<hardware>_<software>` or
27
27
  `<hardware>_<software>_<information>`.
@@ -31,17 +31,17 @@ automatic model unit tests.
31
31
 
32
32
  ### Expected Output
33
33
  You need a second file that contains the expected output, which has the same
34
- name as the YAML simulation file but ends with `:output.txt` instead of
35
- `:simulation.yaml`.
34
+ name as the YAML simulation file but ends with `#output.txt` instead of
35
+ `#simulation.yaml`.
36
36
 
37
37
  You can automatically produce an output file based on the current model for all
38
- YAML simulation files missing an `:output.txt`:
38
+ YAML simulation files missing an `#output.txt`:
39
39
  ```shell
40
40
  bundle exec ruby spec/model/atoms_generate.rb
41
41
  ```
42
42
 
43
43
  In the following example,
44
- `spec/model/data/aoscx:R8N85A-C6000-48G-CL4_PL.10.08.1010:output.txt` (the
44
+ `spec/model/data/aoscx#R8N85A-C6000-48G-CL4_PL.10.08.1010#output.txt` (the
45
45
  second file in the list) was missing:
46
46
 
47
47
  ```shell
@@ -50,21 +50,21 @@ Run options: --seed 57811
50
50
 
51
51
  # Running:
52
52
 
53
- Generating output file for aoscx:R0X25A-6410_FL.10.10.1100:simulation... SKIP, output already exists
54
- Generating output file for aoscx:R8N85A-C6000-48G-CL4_PL.10.08.1010:simulation... OK
55
- Generating output file for arubainstant:IAP515_8.10.0.6_VWLC:simulation... SKIP, output already exists
56
- Generating output file for asa:5512_9.12-4-67_single-context:simulation... SKIP, output already exists
57
- Generating output file for garderos:R7709_003_006_068:simulation... SKIP, output already exists
58
- Generating output file for ios:C8200L_16.12.1:simulation... FAIL, no simulation file
59
- Generating output file for ios:C9200L-24P-4G_17.09.04a:simulation... SKIP, output already exists
60
- Generating output file for ios:C9800-L-F-K9_17.06.05:simulation... SKIP, output already exists
61
- Generating output file for ios:asr920_16.8.1b:simulation... SKIP, output already exists
62
- Generating output file for junos:srx300_22.4:simulation... SKIP, output already exists
63
- Generating output file for opnsense:nano_23.7:simulation... SKIP, output already exists
64
- Generating output file for pfsense:CE_2.7.2:simulation... SKIP, output already exists
65
- Generating output file for routeros:CHR_7.10.1:simulation... SKIP, output already exists
66
- Generating output file for routeros:CHR_7.16:simulation... SKIP, output already exists
67
- Generating output file for routeros:L009UiGS_7.15.2:simulation... SKIP, output already exists
53
+ Generating output file for aoscx#R0X25A-6410_FL.10.10.1100#simulation... SKIP, output already exists
54
+ Generating output file for aoscx#R8N85A-C6000-48G-CL4_PL.10.08.1010#simulation... OK
55
+ Generating output file for arubainstant#IAP515_8.10.0.6_VWLC#simulation... SKIP, output already exists
56
+ Generating output file for asa#5512_9.12-4-67_single-context#simulation... SKIP, output already exists
57
+ Generating output file for garderos#R7709_003_006_068#simulation... SKIP, output already exists
58
+ Generating output file for ios#C8200L_16.12.1#simulation... FAIL, no simulation file
59
+ Generating output file for ios#C9200L-24P-4G_17.09.04a#simulation... SKIP, output already exists
60
+ Generating output file for ios#C9800-L-F-K9_17.06.05#simulation... SKIP, output already exists
61
+ Generating output file for ios#asr920_16.8.1b#simulation... SKIP, output already exists
62
+ Generating output file for junos#srx300_22.4#simulation... SKIP, output already exists
63
+ Generating output file for opnsense#nano_23.7#simulation... SKIP, output already exists
64
+ Generating output file for pfsense#CE_2.7.2#simulation... SKIP, output already exists
65
+ Generating output file for routeros#CHR_7.10.1#simulation... SKIP, output already exists
66
+ Generating output file for routeros#CHR_7.16#simulation... SKIP, output already exists
67
+ Generating output file for routeros#L009UiGS_7.15.2#simulation... SKIP, output already exists
68
68
  .
69
69
 
70
70
  Finished in 0.904792s, 1.1052 runs/s, 0.0000 assertions/s.
@@ -76,7 +76,7 @@ Line Coverage: 58.02% (651 / 1122)
76
76
  ```
77
77
 
78
78
  ### Running the Tests
79
- You can modify the `:output.txt` file to match your expectations and modify the
79
+ You can modify the `#output.txt` file to match your expectations and modify the
80
80
  model accordingly. Run `bundle exec rake` to run the tests.
81
81
 
82
82
  Here is an example when the output of the VTP command is missing in the expected
@@ -94,7 +94,7 @@ Run options: --seed 31447
94
94
  Finished in 7.963602s, 14.6918 runs/s, 48.7217 assertions/s.
95
95
 
96
96
  1) Failure:
97
- ATOMS tests#test_0006_ios:C9200L-24P-4G_17.09.04a:output has expected output [spec/model/model_atoms_spec.rb:12]:
97
+ ATOMS tests#test_0006_ios#C9200L-24P-4G_17.09.04a#output has expected output [spec/model/model_atoms_spec.rb:12]:
98
98
  --- expected
99
99
  +++ actual
100
100
  @@ -9,6 +9,21 @@
@@ -138,9 +138,19 @@ If you want to be sure that your model has been tested, run
138
138
  `bundle exec rake test TESTOPTS="--verbose"` and search for your models unter
139
139
  `ATOMS tests`
140
140
 
141
+ ### Running only one test
142
+ If you want to run only one test while debuging your model, you can select it
143
+ with the option `--name=/regexp/`:
144
+ ```
145
+ bundle exec rake test TESTOPTS="--verbose --name=/ios#C9800.*output/"
146
+ ```
147
+
148
+ You can also set `Oxidized.asetus.cfg.debug = true` in
149
+ `spec/model/model_helper.rb` to activate debug logs.
150
+
141
151
  ## Device Prompt
142
152
  You can specify device prompts to test in a YAML file named
143
- `spec/model/data/<model>:generic:prompt.yaml`.
153
+ `spec/model/data/<model>#generic#prompt.yaml`.
144
154
 
145
155
  The YAML file has three sections containing a list of prompts to test:
146
156
  - pass: these prompts will pass the prompt regexp.
@@ -162,7 +172,7 @@ fail:
162
172
  ## Secrets
163
173
  You can test if the model effectively removes secrets from your YAML simulation
164
174
  file with a YAML file named like the YAML simulation, but with the suffix
165
- `:secret.yaml`.
175
+ `#secret.yaml`.
166
176
 
167
177
  The YAML file has two sections containing a list of strings to test:
168
178
  - fail: the test will fail if the output contains these strings.
@@ -180,7 +190,7 @@ pass:
180
190
  ## Custom tests
181
191
  When you write custom tests for your models, please do not use the filenames
182
192
  mentioned above, as it will interfere with the standard tests. If you need to
183
- store a custom simulation file, use `model:description:custom_simulation.yaml`.
193
+ store a custom simulation file, use `model#description#custom_simulation.yaml`.
184
194
 
185
195
  The [cumulus test](/spec/model/cumulus_spec.rb) is an example of a custom
186
196
  test.
data/docs/Outputs.md CHANGED
@@ -10,9 +10,48 @@ output:
10
10
  directory: /var/lib/oxidized/configs
11
11
  ```
12
12
 
13
+ ### Groups
14
+ If you use groups, the nodes will be stored in directories named after the
15
+ groups. The directories are stored one level above the directory for configurations
16
+ without groups.
17
+
18
+ Example:
19
+ ```
20
+ /var/lib/oxidized/
21
+ + configs/ # Configurations of groupless nodes
22
+ + group1/ # Configurations of nodes in group1
23
+ + group2/ # Configurations of nodes in group2
24
+ ```
25
+
26
+ ### Clean obsolete nodes
27
+ The `file` output can automatically remove the configuration of nodes no
28
+ longer present in the [source](Sources.md).
29
+
30
+ > :warning: **Warning:** this might be a dangerous operation: oxidized
31
+ > will remove **any** file not matching the hostname of the nodes configured
32
+ > in the source.
33
+
34
+ When using groups, it will remove any files not matching the hostnames of the
35
+ nodes from the groups directories (which are on the same level as the default
36
+ directory). As a safety measure, oxidized will only clean configuration out of
37
+ active groups. If the group `example` isn't used anymore, oxidized won't clean
38
+ the configurations out of the directory `../example/`.
39
+
40
+ Configuration:
41
+
42
+ ```yaml
43
+ output:
44
+ default: file
45
+ clean_obsolete_nodes: true
46
+ file:
47
+ directory: "~/.config/oxidized/configs/default"
48
+ ```
49
+
50
+
13
51
  ## Output: Git
14
52
 
15
- This uses the rugged/libgit2 interface. So you should remember that normal Git hooks will not be executed.
53
+ This uses the rugged/libgit2 interface. So you should remember that normal Git
54
+ hooks will not be executed.
16
55
 
17
56
  For a single repository containing all devices:
18
57
 
@@ -63,7 +102,49 @@ output:
63
102
 
64
103
  ```
65
104
 
66
- Over time, your Git repository will expand, potentially leading to performance issues. For instructions on how to address this, see [git performance issues with large device counts](Troubleshooting.md#git-performance-issues-with-large-device-counts).
105
+ ### Git performance issues with large device counts
106
+ When you use git to store your configurations, the size of your repository will
107
+ grow over time. This growth may lead to performance issues. If you encounter
108
+ such issues, you should perform a Git garbage collection on your repository.
109
+
110
+ Follow these steps to do so:
111
+
112
+ 1. Stop oxidized (no one should access the git repository while running garbage
113
+ collection)
114
+ 2. Make a backup of your oxidized data, especially the Git repository
115
+ 3. Change directory your oxidized git repository (as configured in oxidized
116
+ configuration file)
117
+ 4. Execute the command `git gc` to run the garbage collection
118
+ 5. Restart oxidized - you're done!
119
+
120
+
121
+ ### Clean obsolete nodes
122
+ The `git` output can automatically remove the configuration of nodes no
123
+ longer present in the [source](Sources.md).
124
+
125
+ > :warning: **Limitations**
126
+ > - this currently only works with `single_repo: true`
127
+ > - it will ignore configurations saved as [output types](#output-types) in
128
+ > a separate repository.
129
+ > - oxidized will refuse to remove old configurations
130
+ > when saving [output types](#output-types) in a subdirectory of the git
131
+ > repository (`type_as_directory: true`), or it would remove the output
132
+ > type directories
133
+
134
+ Oxidized will remove **any** file within the git repository not matching the
135
+ group and hostname of the nodes configured in the source and will then commit
136
+ the change into git.
137
+
138
+ Configuration:
139
+
140
+ ```yaml
141
+ output:
142
+ default: git
143
+ clean_obsolete_nodes: true
144
+ git:
145
+ single_repo: true
146
+ repo: "~/.config/oxidized/devices.git"
147
+ ```
67
148
 
68
149
  ## Output: Git-Crypt
69
150
 
data/docs/Release.md CHANGED
@@ -1,6 +1,12 @@
1
1
  # How to release a new version of Oxidized?
2
2
  This document is targeted at oxidized maintainers. It describes the release process.
3
3
 
4
+ ## Version numbering
5
+ Oxidized versions are nummered like major.minor.patch
6
+ - currently, the major version is 0.
7
+ - minor is incremented when releasing new features.
8
+ - patch is incremented when releasing fixes only.
9
+
4
10
  ## Review changes
5
11
  Run `git diff 0.30.0..master` (where `0.30.0` is to be changed to the last release) and review
6
12
  all the changes that have been done. Have a specific look at changes you don't understand.
@@ -23,43 +29,47 @@ If you change some code => Restart the release process at the beginning ;-)
23
29
  ## Make sure the file permissions are correct
24
30
  Run `bundle exec rake chmod`
25
31
 
26
- ## Test !
27
- Test the git code and the container against as much device types and environments as you can.
32
+ ## Create a release branch
33
+ Name the release branch `release/0.xx.yy`
28
34
 
29
- Do not integrate late PRs into master if they do not fix issues for the release. The must wait for the next release.
35
+ Update CHANGELOG.md:
36
+ - review it
37
+ - add release notes
38
+ - set the new version (replace `[Unreleased]` with `[0.xx.yy – 202Y-MM-DD]`)
30
39
 
31
- ## Version numbering
32
- Oxidized versions are nummered like major.minor.patch
33
- - currently, the major version is 0.
34
- - minor is incremented when releasing new features.
35
- - patch is incremented when releasing fixes only.
40
+ Change the version in `lib/oxidized/version.rb`
41
+
42
+ Upload the branch to github, make a Pull Request for it.
43
+
44
+ ## Make sure you pass all GitHub CI
45
+ They test different ruby versions, the docker build process and codeql.
46
+
47
+ ## Test !
48
+ Test the git code and the container against as much device types and
49
+ environments as you can.
36
50
 
37
51
  ## Prepare the release in your working repository
38
- 1. Checkout the master branch of oxidized. Make sure you are up to date with origin.
39
- 2. Change the version in lib/oxidized/version.rb
40
- 3. Change CHANGELOG.md to replace [Unreleased] with [0.xx.yy 202Y-MM-DD]
41
- 4. Run `git diff` to check your changes
42
- 5. Commit the changes to the local git repository with a commit message “chore(release): release version 0.xx.yy”
43
- 6. Tag the commit with `git tag -a 0.xx.yy -m "Release 0.xx.yy"`
44
- 7. Build the gem with ‘rake build’
45
- 8. Run `git diff` to check if there have been more changes (there shouldn't)
46
- 9. Install an test the gem locally
47
- ```
48
- gem install --user-install pkg/oxidized-0.30.0.gem
52
+ 1. Merge the Pull Request into master
53
+ 2. `git pull` master
54
+ 3. Tag the commit with `git tag -a 0.xx.yy -m "Release 0.xx.yy"`
55
+ 4. Build the gem with ‘rake build’
56
+ 5. Run `git diff` to check if there have been more changes (there shouldn't)
57
+ 6. Install an test the gem locally
58
+ ```shell
59
+ gem install --user-install pkg/oxidized-0.xx.yy.gem
49
60
  ~/.local/share/gem/ruby/3.1.0/bin/oxidized
50
61
  ```
51
62
 
52
63
  ## Release in github
53
- Push the change and the tag to github:
64
+ Push the tag to github:
54
65
  ```
55
- git push
56
66
  git push origin 0.xx.yy
57
67
  ```
58
68
 
59
- Make a release from the tag in github
60
- - Thank the contributors
61
- - Only describe major changes, and refer to CHANGELOG.md
69
+ Make a release from the tag in github.
70
+ - Take the release notes frm CHANGELOG.md
62
71
  - List new contributors (generated automatically)
72
+ - Keep the Full Changelog (generated automatically)
63
73
 
64
74
  ## Release in rubygems
65
75
  Push the gem with ‘rake push’