oxidized 0.21.0 → 0.22.0

data/docs/Sources.md

@@ -1,10 +1,10 @@
-## Source
+# Sources
 
-### Source: CSV
+## Source: CSV
 
-One line per device, colon seperated. If `ip` isn't present, a DNS lookup will be done against `name`.  For large installations, setting `ip` will dramatically reduce startup time.
+One line per device, colon separated. If `ip` isn't present, a DNS lookup will be done against `name`.  For large installations, setting `ip` will dramatically reduce startup time.
 
-```
+```yaml
 source:
   default: csv
   csv:
@@ -22,19 +22,53 @@ source:
 
 Example csv `/var/lib/oxidized/router.db`:
 
+```text
+rtr01.local:192.168.1.1:ios:oxidized:5uP3R53cR3T:T0p53cR3t
+```
+
+If you would like to use a GPG encrypted file as the source then you can use the following example:
+
+```yaml
+source:
+  default: csv
+  csv:
+    file: ~/.config/oxidized/router.db
+    delimiter: !ruby/regexp /:/
+    gpg: true
+    gpg_password: 'password'
+    map:
+      name: 0
+      model: 1
+```
+
+Please note, if you are running GPG v2 then you will be prompted for your gpg password on start up, if you use GPG >= 2.1 then you can add the following config to stop that behaviour:
+
+Within `~/.gnupg/gpg-agent.conf`
+
+```text
+allow-loopback-pinentry
 ```
-rtr01.local,192.168.1.1,ios,oxidized,5uP3R53cR3T,T0p53cR3t
+
+and within: `~/.gnupg/gpg.conf`
+
+```text
+pinentry-mode loopback
 ```
 
-### Source: SQL
+## Source: SQL
+
  Oxidized uses the `sequel` ruby gem. You can use a variety of databases that aren't explicitly listed. For more information visit https://github.com/jeremyevans/sequel Make sure you have the correct adapter!
-### Source: MYSQL
+
+**NOTE** - Many database engines have reserved keywords that may conflict with Oxidized configuration field names (such as 'name', 'group', etc). Pay attention to any names that are used and observed proper quoting methods to avoid errors or unpredictable results.
+
+
+## Source: MYSQL
 
 `sudo apt-get install libmysqlclient-dev`
 
 The values correspond to your fields in the DB such that ip, model, etc are field names in the DB
 
-```
+```yaml
 source:
   default: sql
   sql:
@@ -52,17 +86,17 @@ source:
       enable: enable
 ```
 
-### Source: SQLite
+## Source: SQLite
 
 One row per device, filtered by hostname.
 
-```
+```yaml
 source:
   default: sql
   sql:
     adapter: sqlite
-    database: "/var/lib/oxidized/devices.db"
-    table: devices
+    database: "/var/lib/oxidized/nodes.db"
+    table: nodes
     map:
       name: fqdn
       model: model
@@ -72,13 +106,41 @@ source:
       enable: enable
 ```
 
-### Source: HTTP
+## Custom SQL Query Support
+
+You may also implement a custom SQL query to retrieve the nodelist using  SQL syntax with the `query:` configuration parameter under the `sql:` stanza.
+
+### Custom SQL Query Examples
+
+You may have a table named `nodes` which contains a boolean to indicate if the nodes should be enabled (fetched via oxidized). This can be used in the custom SQL query to avoid fetching from known impacted nodes.
+
+In your configuration, you would add the `query:` parameter and specify the SQL query. Make sure to put this within the `sql:` configuration section.
+
+```sql
+query: "SELECT * FROM nodes WHERE enabled = True"
+```
+
+Since this is an SQL query, you can also provide a more advanced query to assist in more complicated oxidized deployments. The exact deployment is up to you on how you design your database and oxidized fetchers.
+
+In this example we limit the nodes to two "POPs" of `mypop1` and `mypop2`. We also require the nodes to have the `enabled` boolean set to `True`.
+
+```sql
+query: "SELECT * FROM nodes WHERE pop IN ('mypop1','mypop2') AND enabled = True"
+```
+
+The order of the nodes returned will influence the order that nodes are fetched by oxidized. You can use standard SQL `ORDER BY` clauses to influence the node order.
+
+You should always test your SQL query before using it in the oxidized configuration as there is no syntax or error checking performed before sending it to the database engine.
+
+Consult your database documentation for more information on query language and table optimization.
+
+## Source: HTTP
 
 One object per device.
 
 HTTP Supports basic auth, configure the user and pass you want to use under the http: section.
 
-```
+```yaml
 source:
   default: http
   http:
@@ -100,11 +162,11 @@ source:
 
 You can also pass `secure: false` if you want to disable ssl certificate verification:
 
-```
+```yaml
 source:
   default: http
   http:
     url: https://url/api
     scheme: https
     secure: false
-```
+```

data/docs/Supported-OS-Types.md

@@ -1,149 +1,172 @@
 # Supported OS types
- * Vendor
-   * OS model
- * A10 Networks
-   * [ACOS](/lib/oxidized/model/acos.rb)
- * Accedian Performance Elements (NIDs)
-   * [AEN](/lib/oxidized/model/aen.rb)
- * Alcatel-Lucent
-   * [AOS](/lib/oxidized/model/aos.rb)
-   * [AOS7](/lib/oxidized/model/aos7.rb)
-   * [ISAM](/lib/oxidized/model/isam.rb)
-   * [SR OS (Formerly TiMOS)](/lib/oxidized/model/timos.rb)
-   * Wireless
- * Alvarion
-   * [BreezeACCESS](/lib/oxidized/model/alvarion.rb)
- * APC
-   * [AOS](/lib/oxidized/model/apc_aos.rb)
- * Arista
-   * [EOS](/lib/oxidized/model/eos.rb)
- * Arris
-   * [C4CMTS](/lib/oxidized/model/c4cmts.rb)
- * Aruba
-   * [AOSW](/lib/oxidized/model/aosw.rb)
- * AudioCodes
-   * [AudioCodes](/lib/oxdized/model/audiocodes.rb)
- * Avaya
-   * [VOSS (VSP Operating System Software)](/lib/oxidized/model/voss.rb)
-   * [BOSS (Baystack Operating System Software)](/lib/oxidized/model/boss.rb)
- * Brocade
-   * [FabricOS](/lib/oxidized/model/fabricos.rb)
-   * [Ironware](/lib/oxidized/model/ironware.rb)
-   * [NOS (Network Operating System)](/lib/oxidized/model/nos.rb)
-   * [Vyatta](/lib/oxidized/model/vyatta.rb)
-   * [6910](/lib/oxidized/model/br6910.rb)
- * Casa
-   * [Casa](/lib/oxidized/model/casa.rb)
- * Check Point
-   * [GaiaOS](/lib/oxidized/model/gaiaos.rb)
- * Ciena
-   * [SAOS](/lib/oxidized/model/saos.rb)
- * Cisco
-   * [ACSW](/lib/oxidized/model/acsw.rb)
-   * [AireOS](/lib/oxidized/model/aireos.rb)
-   * [ASA](/lib/oxidized/model/asa.rb)
-   * [AsyncOS](/lib/oxidized/model/asyncos.rb)
-   * [CatOS](/lib/oxidized/model/catos.rb)
-   * [IOS](/lib/oxidized/model/ios.rb)
-   * [IOSXR](/lib/oxidized/model/iosxr.rb)
-   * [NGA](/lib/oxidized/model/cisconga.rb)
-   * [NXOS](/lib/oxidized/model/nxos.rb)
-   * [SMA](/lib/oxidized/model/ciscosma.rb)
-   * [SMB (Nikola series)](/lib/oxidized/model/ciscosmb.rb)
-   * [UCS](/lib/oxidized/model/ucs.rb)
- * Citrix
-   * [NetScaler (Virtual Applicance)](/lib/oxidized/model/netscaler.rb)
- * Coriant (former Tellabs)
-   * [TMOS (8800)](/lib/oxidized/model/corianttmos.rb)
-   * [8600](/lib/oxidized/model/coriant8600.rb)
-   * [Groove](/lib/oxidized/model/coriantgroove.rb)
- * Cumulus
-   * [Linux](/lib/oxidized/model/cumulus.rb)
- * DataCom
-   * [DmSwitch 3000](/lib/oxidized/model/datacom.rb)
- * DCN
-   * [DCN](/lib/oxidized/model/ios.rb) - Map this to ios.
- * DELL
-   * [PowerConnect](/lib/oxidized/model/powerconnect.rb)
-   * [AOSW](/lib/oxidized/model/aosw.rb)
- * D-Link
-   * [D-Link](/lib/oxidized/model/dlink.rb)
- * Ericsson/Redback
-   * [IPOS (former SEOS)](/lib/oxidized/model/ipos.rb)
- * Extreme Networks
-   * [Enterasys](/lib/oxidized/model/enterasys.rb)
-   * [WM](/lib/oxidized/model/mtrlrfs.rb)
-   * [XOS](/lib/oxidized/model/xos.rb)
- * F5
-   * [TMOS](/lib/oxidized/model/tmos.rb)
- * Force10
-   * [DNOS](/lib/oxidized/model/dnos.rb)
-   * [FTOS](/lib/oxidized/model/ftos.rb)
- * FortiGate
-   * [FortiOS](/lib/oxidized/model/fortios.rb)
- * Fujitsu
-   * [PRIMERGY Blade switch 1/10Gbe](/lib/oxidized/model/fujitsupy.rb)
- * Hatteras
-   * [Hatteras](/lib/oxidized/model/hatteras.rb)
- * Hirschmann
-   * [HiOS](/lib/oxidized/model/hirschmann.rb)
- * HP
-   * [Comware (HP A-series, H3C, 3Com)](/lib/oxidized/model/comware.rb)
-   * [Procurve](/lib/oxidized/model/procurve.rb)
-   * [BladeSystem (Onboard Administrator)](/lib/oxidized/model/hpebladesystem.rb)
-   * [MSA](/lib/oxidized/model/hpemsa.rb)
- * Huawei
-   * [VRP](/lib/oxidized/model/vrp.rb)
- * Juniper
-   * [JunOS](/lib/oxidized/model/junos.rb)
-   * [ScreenOS (Netscreen)](/lib/oxidized/model/screenos.rb)
- * Mellanox
-   * [MLNX-OS](/lib/oxidized/model/mlnxos.rb)
-   * [Voltaire](/lib/oxidized/model/voltaire.rb)
- * Mikrotik
-   * [RouterOS](/lib/oxidized/model/routeros.rb)
- * Motorola
-   * [RFS](/lib/oxidized/model/mtrlrfs.rb)
- * MRV
-   * [MasterOS](/lib/oxidized/model/masteros.rb)
-   * [FiberDriver](/lib/oxidized/model/fiberdriver.rb)
- * Netgear
-   * [Netgear](/lib/oxidized/model/netgear.rb)
- * Netonix
-   * [WISP Switch (As Netonix)](/lib/oxidized/model/netonix.rb)
- * Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent)
-   * [SR OS (TiMOS)](/lib/oxidized/model/timos.rb)
- * OneAccess
-   * [OneOS](/lib/oxidized/model/oneos.rb)
- * Opengear
-   * [Opengear](/lib/oxidized/model/opengear.rb)
- * Palo Alto
-   * [PANOS](/lib/oxidized/model/panos.rb)
- * [PLANET SG/SGS Switches](/lib/oxidized/model/planet.rb)
- * [pfSense](/lib/oxidized/model/pfsense.rb)
- * Radware
-   * [AlteonOS](/lib/oxidized/model/alteonos.rb)
- * Quanta
-   * [Quanta / VxWorks 6.6 (1.1.0.8)](/lib/oxidized/model/quantaos.rb)
- * Siklu
-   * [EtherHaul](/lib/oxidized/model/siklu.rb)
- * Supermicro
-   * [Supermicro](/lib/oxidized/model/supermicro.rb)
- * Symantec
-   * [Blue Coat ProxySG / Security Gateway OS (SGOS)](/lib/oxidized/model/sgos.rb)
- * Trango Systems
-   * [Trango](/lib/oxidized/model/trango.rb)
- * TPLink
-   * [TPLink](/lib/oxidized/model/tplink.rb)
- * Ubiquiti
-   * [AirOS](/lib/oxidized/model/airos.rb)
-   * [Edgeos](/lib/oxidized/model/edgeos.rb)
-   * [EdgeSwitch](/lib/oxidized/model/edgeswitch.rb)
- * Watchguard
-   * [Fireware OS](/lib/oxidized/model/firewareos.rb)
- * Westell
-   * [Westell 8178G, Westell 8266G](/lib/oxidized/model/weos.rb)
- * Zhone
-   * [Zhone (OLT and MX)](/lib/oxidized/model/zhoneolt.rb)
- * Zyxel
-   * [ZyNOS](/lib/oxidized/model/zynos.rb)
+
+* Vendor
+  * OS model
+* A10 Networks
+  * [ACOS](/lib/oxidized/model/acos.rb)
+* Accedian Performance Elements (NIDs)
+  * [AEN](/lib/oxidized/model/aen.rb)
+* Alcatel-Lucent
+  * [AOS](/lib/oxidized/model/aos.rb)
+  * [AOS7](/lib/oxidized/model/aos7.rb)
+  * [ISAM](/lib/oxidized/model/isam.rb)
+  * [SR OS (Formerly TiMOS)](/lib/oxidized/model/sros.rb)
+  * Wireless
+* Allied Telesis
+  * [Alliedware Plus](/lib/oxidized/model/awplus.rb)
+* Alvarion
+  * [BreezeACCESS](/lib/oxidized/model/alvarion.rb)
+* APC
+  * [AOS](/lib/oxidized/model/apc_aos.rb)
+* Arbor Networks
+  * [ArbOS](/lib/oxidized/model/arbos.rb)
+* Arista
+  * [EOS](/lib/oxidized/model/eos.rb)
+* Arris
+  * [C4CMTS](/lib/oxidized/model/c4cmts.rb)
+* Aruba
+  * [AOSW](/lib/oxidized/model/aosw.rb)
+* AudioCodes
+  * [AudioCodes](/lib/oxdized/model/audiocodes.rb)
+* Avaya
+  * [VOSS (VSP Operating System Software)](/lib/oxidized/model/voss.rb)
+  * [BOSS (Baystack Operating System Software)](/lib/oxidized/model/boss.rb)
+* Brocade
+  * [FabricOS](/lib/oxidized/model/fabricos.rb)
+  * [Ironware](/lib/oxidized/model/ironware.rb)
+  * [NOS (Network Operating System)](/lib/oxidized/model/nos.rb)
+  * [Vyatta](/lib/oxidized/model/vyatta.rb)
+  * [6910](/lib/oxidized/model/br6910.rb)
+  * [SLX-OS](/lib/oxidized/model/slxos.rb)
+* Casa
+  * [Casa](/lib/oxidized/model/casa.rb)
+* Check Point
+  * [GaiaOS](/lib/oxidized/model/gaiaos.rb)
+* Ciena
+  * [SAOS](/lib/oxidized/model/saos.rb)
+* Cisco
+  * [ACSW](/lib/oxidized/model/acsw.rb)
+  * [AireOS](/lib/oxidized/model/aireos.rb)
+  * [ASA](/lib/oxidized/model/asa.rb)
+  * [AsyncOS](/lib/oxidized/model/asyncos.rb)
+  * [CatOS](/lib/oxidized/model/catos.rb)
+  * [IOS](/lib/oxidized/model/ios.rb)
+  * [IOSXR](/lib/oxidized/model/iosxr.rb)
+  * [NGA](/lib/oxidized/model/cisconga.rb)
+  * [NXOS](/lib/oxidized/model/nxos.rb)
+  * [SMA](/lib/oxidized/model/ciscosma.rb)
+  * [SMB (Nikola series)](/lib/oxidized/model/ciscosmb.rb)
+  * [UCS](/lib/oxidized/model/ucs.rb)
+* Citrix
+  * [NetScaler (Virtual Applicance)](/lib/oxidized/model/netscaler.rb)
+* Coriant (former Tellabs)
+  * [TMOS (8800)](/lib/oxidized/model/corianttmos.rb)
+  * [8600](/lib/oxidized/model/coriant8600.rb)
+  * [Groove](/lib/oxidized/model/coriantgroove.rb)
+* ComNet
+  * [ComNet Microsemi Switch](/lib/oxidized/model/comnetms.rb)
+* Cumulus
+  * [Linux](/lib/oxidized/model/cumulus.rb)
+* DataCom
+  * [DmSwitch 3000](/lib/oxidized/model/datacom.rb)
+* DCN
+  * [DCN](/lib/oxidized/model/ios.rb) - Map this to ios.
+* DELL
+  * [PowerConnect](/lib/oxidized/model/powerconnect.rb)
+  * [AOSW](/lib/oxidized/model/aosw.rb)
+* D-Link
+  * [D-Link](/lib/oxidized/model/dlink.rb)
+* Ericsson/Redback
+  * [IPOS (former SEOS)](/lib/oxidized/model/ipos.rb)
+* Extreme Networks
+  * [Enterasys](/lib/oxidized/model/enterasys.rb)
+  * [WM](/lib/oxidized/model/mtrlrfs.rb)
+  * [XOS](/lib/oxidized/model/xos.rb)
+* F5
+  * [TMOS](/lib/oxidized/model/tmos.rb)
+* Fiberstore
+  * [S3800](/lib/oxidized/model/gcombnps.rb)
+* Force10
+  * [DNOS](/lib/oxidized/model/dnos.rb)
+  * [FTOS](/lib/oxidized/model/ftos.rb)
+* FortiGate
+  * [FortiOS](/lib/oxidized/model/fortios.rb)
+* Fujitsu
+  * [PRIMERGY Blade switch 1/10Gbe](/lib/oxidized/model/fujitsupy.rb)
+* GCOM Technologies
+  * [Broadband Network Platform Software](/lib/oxidized/model/gcombnps.rb)
+* Hatteras
+  * [Hatteras](/lib/oxidized/model/hatteras.rb)
+* Hillstone Networks
+  * [StoneOS](/lib/oxidized/model/stoneos.rb)
+* Hirschmann
+  * [HiOS](/lib/oxidized/model/hirschmann.rb)
+* HP
+  * [Comware (HP A-series, H3C, 3Com)](/lib/oxidized/model/comware.rb)
+  * [Procurve](/lib/oxidized/model/procurve.rb)
+  * [BladeSystem (Onboard Administrator)](/lib/oxidized/model/hpebladesystem.rb)
+  * [MSA](/lib/oxidized/model/hpemsa.rb)
+* Huawei
+  * [VRP](/lib/oxidized/model/vrp.rb)
+* Juniper
+  * [JunOS](/lib/oxidized/model/junos.rb)
+  * [ScreenOS (Netscreen)](/lib/oxidized/model/screenos.rb)
+* Mellanox
+  * [MLNX-OS](/lib/oxidized/model/mlnxos.rb)
+  * [Voltaire](/lib/oxidized/model/voltaire.rb)
+* Mikrotik
+  * [RouterOS](/lib/oxidized/model/routeros.rb)
+* Motorola
+  * [RFS](/lib/oxidized/model/mtrlrfs.rb)
+* MRV
+  * [MasterOS](/lib/oxidized/model/masteros.rb)
+  * [FiberDriver](/lib/oxidized/model/fiberdriver.rb)
+* Netgear
+  * [Netgear](/lib/oxidized/model/netgear.rb)
+* Netonix
+  * [WISP Switch (As Netonix)](/lib/oxidized/model/netonix.rb)
+* Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent)
+  * [SR OS (TiMOS)](/lib/oxidized/model/sros.rb)
+* OneAccess
+  * [OneOS](/lib/oxidized/model/oneos.rb)
+* [OpenBSD](/lib/oxidized/model/openbsd.rb)
+* Opengear
+  * [Opengear](/lib/oxidized/model/opengear.rb)
+* [OpenWRT](/lib/oxidized/model/openwrt.rb)
+* [OPNsense](/lib/oxidized/model/opnsense.rb)
+* Palo Alto
+  * [PANOS](/lib/oxidized/model/panos.rb)
+* [PLANET SG/SGS Switches](/lib/oxidized/model/planet.rb)
+* [pfSense](/lib/oxidized/model/pfsense.rb)
+* Radware
+  * [AlteonOS](/lib/oxidized/model/alteonos.rb)
+* Quanta
+  * [Quanta / VxWorks 6.6 (1.1.0.8)](/lib/oxidized/model/quantaos.rb)
+* Siklu
+  * [EtherHaul](/lib/oxidized/model/siklu.rb)
+* SNR
+  * [SNR-S300G, S2xxx, S3xxx, S4xxx](/lib/oxidized/model/dcnos.rb)
+* Supermicro
+  * [SSE-G2252, G2252P](/lib/oxidized/model/edgecos.rb)
+  * [SSE-G48-TG4, G24-TG4](/lib/oxidized/model/aricentiss.rb)
+  * [SSE-X24S, X24SR, X3348S, X3348SR, X3348T, X3348TR](/lib/oxidized/model/aricentiss.rb)
+  * [SBM-GEM-X2C, GEM-X2C+, GEM-X3S+, XEM-X10SM](/lib/oxidized/model/aricentiss.rb)
+* Symantec
+  * [Blue Coat ProxySG / Security Gateway OS (SGOS)](/lib/oxidized/model/sgos.rb)
+* Trango Systems
+  * [Trango](/lib/oxidized/model/trango.rb)
+* TPLink
+  * [TPLink](/lib/oxidized/model/tplink.rb)
+* Ubiquiti
+  * [AirOS](/lib/oxidized/model/airos.rb)
+  * [Edgeos](/lib/oxidized/model/edgeos.rb)
+  * [EdgeSwitch](/lib/oxidized/model/edgeswitch.rb)
+* Watchguard
+  * [Fireware OS](/lib/oxidized/model/firewareos.rb)
+* Westell
+  * [Westell 8178G, Westell 8266G](/lib/oxidized/model/weos.rb)
+* Zhone
+  * [Zhone (OLT and MX)](/lib/oxidized/model/zhoneolt.rb)
+* Zyxel
+  * [ZyNOS](/lib/oxidized/model/zynos.rb)
+  * [NDMS](/lib/oxidized/model/ndms.rb)