junos-ez-stdlib 0.0.10

data/LICENSE

@@ -0,0 +1,26 @@
+LICENSE (BSD-2)
+===============
+Copyright (c) 2013, Jeremy Schulman, Juniper Networks
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+    Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+
+    Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,181 @@
+# OVERVIEW
+
+Ruby framework to support Junos OS based device management automation.  
+
+This is the "standard library" or "core" set of functionality that should work on most/all Junos OS based devices.  
+
+This framework is build on top of the NETCONF gem which uses XML as the fundamental data-exchange.  So no 
+"automating the CLI" or using SNMP.  The purpose of this framework is to **enable automation development 
+without requiring specific Junos XML knowledge**.
+
+Further documentation can be found in the *docs* subdirectory.
+
+# FRAMEWORK
+
+The framework is comprised of these basic eloements:
+
+  - Facts: 
+
+    A Hash of name/value pairs of information auto-collected.  Fact values can be Hash structures as well
+    so you can have deeply nested fact data.  You can also define your own facts in addition to the "stdlib" facts.
+    The facts are used by the framework to create a platform indepent layer of abstraction.  This means
+    that managing a VLAN, for example, is the same regardless of the underlying hardware platofrm (EX, QFX,
+    MX, SRX, ...)
+    
+  - Resources: 
+
+    Resources allow you to easily configure and perform operational functions on specific items within Junos, 
+    for example VLANs, or switch ports.  A resource has *properties* that you manipuate as Hash.  You can
+    interact with Junos using resource methods like `read!`, `write!`, `delete!`, `activate!`, `deactivate!`, etc. 
+    For a complete listing of resource methods, refer to the *docs* directory
+    
+  - Providers:
+
+    Providers allow you to manage a collection of resource, and most commonly, select a resource.  
+    The purpose of a provider/resource is to automate the life-cycle of common changes, like adding
+    VLANs, or ports to a VLAN.  A provider also allows you to obtain a `list` of resources 
+    (Array of *names*) or a `catalog` (Hash of resource properties).  Providers may include resource 
+    specific functionality, like using complex YAML/Hash data for easy import/export and provisioning 
+    with Junos.  If you need the ability to simply apply config-snippets that you do not need to model
+    as resources (as you might for initial device commissioning), the Utilities library is where you 
+    want to start.
+  
+  - Utilities:
+
+    Utilities are simply collections of functions.  The **configuration** utilities, for example, will
+    allow you to easily push config snippets in "curly-brace", "set", or XML formats.  Very useful
+    for unmanaged provider/resources (like initial configuration of the device).  The
+    **routing-engine** utilities, for example, will allow you to easily upgrade software, check
+    memory usage, and do `ping` operations.
+  
+# EXAMPLE USAGE
+  
+```ruby
+require 'pp'
+require 'net/netconf/jnpr'
+require 'junos-ez/stdlib'
+
+unless ARGV[0]
+   puts "You must specify a target"
+   exit 1
+end
+
+# login information for NETCONF session 
+login = { :target => ARGV[0], :username => 'jeremy',  :password => 'jeremy1',  }
+
+## create a NETCONF object to manage the device and open the connection ...
+
+ndev = Netconf::SSH.new( login )
+print "Connecting to device #{login[:target]} ... "
+ndev.open
+puts "OK!"
+
+## Now bind providers to the device object. The 'Junos::Ez::Provider' must be first.
+## This will retrieve the device 'facts'.  The other providers allow you to define the 
+## provider variables; so this example is using 'l1_ports' and 'ip_ports', but you could name 
+## them what you like, yo!
+
+Junos::Ez::Provider( ndev )
+Junos::Ez::L1ports::Provider( ndev, :l1_ports )
+Junos::Ez::IPports::Provider( ndev, :ip_ports )
+Junoz::Ez::Config::Utils( ndev, :cu )
+
+# -----------------------------------------------------------
+# Facts ...
+# -----------------------------------------------------------
+
+# show the device softare version fact
+pp ndev.fact :version
+
+# show the device serial-number face
+pp ndev.fact :serialnumber
+
+# get a list of all available facts (Array)
+pp ndev.facts.list
+
+# get a hash of all facts and their associated values
+pp ndev.facts.catalog
+
+# -----------------------------------------------------------
+# Layer 1 (physical ports) Resources ...
+# -----------------------------------------------------------
+
+pp ndev.l1_ports.list
+pp ndev.l1_ports.catalog
+
+# select port 'ge-0/0/0' and display the contents
+# of the properties (like port, speed, description)
+
+ge_0 = ndev.l1_ports['ge-0/0/0']
+pp ge_0.to_h
+
+# change port to disable, this will write the change
+# but not commit it.
+
+ge_0[:admin] = :down
+ge_0.write!
+
+# show the diff of the change to the screen
+
+puts ndev.cu.diff?
+
+# now rollback the change, since we don't want to save it.
+
+ndev.cu.rollback!
+
+ndev.close
+```
+  
+# PROVIDERS
+
+Providers manage access to individual resources and their associated properties.  Providers/resources exists
+for managing life-cycle common changes that you generally need as part of a larger workflow process.  For more
+documentation on Providers/Resources, see the *docs* directory.
+
+  - L1ports: Physical port management
+  - L2ports: Ethernet port (VLAN) management
+  - Vlans: VLAN resource management
+  - IPports: IP v4 port management
+  - StaticHosts: Static Hosts [system static-host-mapping ...]  
+  - StaticRoutes: Static Routes [routing-options static ...] 
+
+# UTILITIES
+
+  - Config:
+    
+    These functions allow you to load config snippets, do commit checks, look at config diffs, etc.
+    Generally speaking, you would want to use the Providers/Resources framework to manage specific 
+    items in the config.  This utility library is very useful when doing the initial commissioning
+    process, where you do not (cannot) model every aspect of Junos.  These utilities can also be
+    used in conjunction with Providers/Resources, specifically around locking/unlocking and committing
+    the configuration.
+  
+  - Filesystem:
+  
+    These functions provide you "unix-like" commands that return data in Hash forms rather than
+    as string output you'd normally have to screen-scraps.  These methods include `ls`, `df`, `pwd`,
+    `cwd`, `cleanup`, and `cleanup!`
+
+  - Routing-Engine:
+  
+    These functions provide a general collection to information and functioanlity for handling 
+    routing-engine (RE) processes.  These functions `reboot!`, `shutdown!`, `install_software!`, 
+    `ping`.  Information gathering such as memory-usage, current users, and RE status information
+    is also made available through this collection.
+
+# DEPENDENCIES
+
+  * gem netconf
+  * Junos OS based products
+  
+# INSTALLATION 
+
+  * gem install junos-ez-stdlib
+
+# CONTRIBUTORS
+
+  * Jeremy Schulman, @nwkautomaniac
+
+# LICENSES
+
+   BSD-2, See LICENSE file

data/docs/Config_Utils.md

@@ -0,0 +1,3 @@
+# Configuration Utilities
+
+Docs comming soon ...

data/docs/Facts.md

@@ -0,0 +1,106 @@
+# Fact Keeping
+
+This framework is *fact based*, meaning that the provider libraries will have access to information about each
+target.  Facts enable the framework to abstract the physical differences of the underlying hardware.  
+
+For example,the `Junos::Ez::Vlans::Provider` allows you to manage vlans without having to worry about the differences between the EX product family and the MX product family.  To you, the programmer, you simply obtain a resource and manage the associated properties.
+
+There are collection of standard facts that are always read by the framework.  You can find a list of these and the assocaited code in the *libs/../facts* subdirectory.  These facts are also avaialble to your program as well.  So you can make programmatic decisions based on the facts of the device.
+
+You can also define your own facts, and then go on to building your own provider libraries (but we're getting ahead of ourselfs here ...)
+
+# Usage
+
+Usage rule: you **MUST** call `Junos::Ez::Provider` on your netconf object:
+
+  - **AFTER** the object has connected to the target, since it will read facts
+
+  - **BEFORE** you add any other providers, since these may use the facts
+  
+Here is a basic example:
+
+```ruby
+require 'pp'
+require 'net/netconf/jnpr'
+require 'junos-ez/stdlib'
+
+login = { :target => ARGV[0], :username => 'jeremy',  :password => 'jeremy1',  }
+
+# create a NETCONF object to manage the device
+
+ndev = Netconf::SSH.new( login )
+
+# open the NETCONF connetion, if this fails, the object will throw an exception
+ndev.open
+
+# now that the netconf session has been established, initialize the object for the
+# Junos::Ez framework.  This will add an instance variable called `@facts` and
+# retrieve all known facts from the target
+
+Junos::Ez::Provider( ndev )
+
+# do a quick dump of all facts
+
+pp ndev.facts.catalog
+
+-> {:hardwaremodel=>"SRX210H",
+ :serialnumber=>"AD2909AA0096",
+ :hostname=>"srx210",
+ :domain=>"",
+ :fqdn=>"srx210",
+ :RE=>
+  {:status=>"OK",
+   :model=>"RE-SRX210H",
+   :up_time=>"26 days, 15 hours, 46 minutes, 4 seconds",
+   :last_reboot_reason=>"0x200:normal shutdown"},
+ :personality=>:SRX_BRANCH,
+ :ifd_style=>:CLASSIC,
+ :switch_style=>:VLAN,
+ :version=>"12.1X44-D10.4"}
+```
+
+# Methods
+  
+  - `read!` - reloads the facts from the target
+  - `[]` - retrieve a specific fact from the keeper
+  - `fact` - alternative method to retrieve a specific fact from the keeper
+  - `list`, `list!` - returns an Array of fact names (symbols)
+  - `catalog`, `catalog!` - returns a Hash of fact names and values
+  
+The bang (!) indicates that the method will re-read the value from the target, which the non-bang method uses the values cached in memory.  If the cache does not exist, the framework will read the values. The use of the bang-methods are handy if/when you have facts whose values change at runtime, like the `ndev.fact(:RE)[:up_time]`
+
+# Definig Custom Facts
+
+You can define your own facts using `Junos::Ez::Facts::Keeper.define`.  You can review the stdlib facts by looking the *libs/../facts* subdirectory of this repo.  Here is the code for the `:chassis` fact.  What is interesting about this example, is this code will actually create multiple facts about the chassis.  So there is not a required one-to-one relationship between createing a custom fact and the actual number of facts it creates, yo!
+
+When you define your fact, you must give it a unique "fact name*, and a block.  The block takee two arguments: the first is the netconf object, which will provide you access to the underlying Junos XML netconf, an a Hash that allows you to write your facts into the Keeper:
+
+```ruby
+Junos::Ez::Facts::Keeper.define( :chassis ) do |ndev, facts|
+  
+  inv_info = ndev.rpc.get_chassis_inventory
+  chassis = inv_info.xpath('chassis')
+  
+  facts[:hardwaremodel] = chassis.xpath('description').text
+  facts[:serialnumber] = chassis.xpath('serial-number').text           
+  
+  cfg = ndev.rpc.get_configuration{|xml|
+    xml.system {
+      xml.send(:'host-name')
+      xml.send(:'domain-name')
+    }
+  }
+  
+  facts[:hostname] = cfg.xpath('//host-name').text
+  facts[:domain] = cfg.xpath('//domain-name').text
+  facts[:fqdn] = facts[:hostname]
+  facts[:fqdn] += ".#{facts[:domain]}" unless facts[:domain].empty?
+  
+end
+```
+
+
+
+
+
+

data/docs/Filesys_Utils.md

@@ -0,0 +1,3 @@
+# Filesystem Utilities
+
+Docs coming soon ...

data/docs/IPports.md

@@ -0,0 +1,3 @@
+# IP v4 Port Provider
+
+Docs comming soon...

data/docs/L1ports.md

@@ -0,0 +1,3 @@
+# Layer 1 Physical Port Provider
+
+Docs comming soon ...

data/docs/L2ports.md

@@ -0,0 +1,3 @@
+# Layer 2 Switching Port Provider
+
+Docs comming soon ...

data/docs/Providers_Resources.md

@@ -0,0 +1,304 @@
+# RESOURCES
+
+Let's start with Resources before we get to Providers.  Resources are generally where all "the action" happens. 
+The primary purpose of a resource is to allow you to make configuration changes without having to know
+the underlying Junos XML configuration.  When you have a resource, you can always get a list of the properties 
+available to you.  
+
+Here's an example of looking at a an ethernet switching port using the L2ports provider.
+This is just a snippet of code, and they use the (->) notation to indicate standard output results.
+
+For the following example assume that `l2_ports` is the provider assigned to the `ndev` object.  The `ndev` object is of class `Netconf::SSH` (or has this as a parent class)
+
+```ruby
+
+# select a port by name from the provider
+
+port = ndev.l2_ports['ge-0/0/0']
+
+# check to see if this actually exists in the config
+
+unless port.exists?
+  puts "This port #{port.name} does not exist!"
+  exit 1
+end
+
+# now pretty-print the properties associated with this resource (which would be the same list for
+# any L2port resource
+
+pp port.properties
+
+-> [:_exist, :_active, :description, :untagged_vlan, :tagged_vlans, :vlan_tagging]
+
+# now look at the specific values for this resource by pp the assocaite hash
+
+pp port.to_h
+
+-> {"ge-0/0/0"=>
+  {:_active=>true,
+   :_exist=>true,
+   :vlan_tagging=>true,
+   :tagged_vlans=>["Red", "Green", "Blue"],
+   :untagged_vlan=>nil}}
+```
+
+## Resource Methods
+
+  - `read!` - reads the contents into the resource read-hash (@has)
+  - `write!` - writes the contents from the resource write-hash (@should) to the device
+  - `delete!` - delete the resource from the configuration
+  - `activate!` - activates the configuration for this resource
+  - `deactivate!` - deactivates the configuration for this resource
+  - `rename!` - renames the resource in the configuration
+  - `reorder!` - reorders (Junos `insert`) the resource in the configuration
+  - `exists?` - indicates if this resource exists in the configuration
+  - `active?` - indicates if this resource is active in the configuration
+  - `to_h` - return the read-from and write-to hash values
+
+## Instance Variables
+
+Each resource include a hash structure containing the values read-from the device.  This is
+the `@has` variable.  When modifying properties, the changed values are stored in the
+write-to hash `@should` variable.  These instance variable are made accessible, but you
+should not access them directly.  See next section for changing the property values.
+
+## Reading Properties
+
+You can obtain the entire `@has` property hash has using the `to_h` method.  This example selects the "ge-0/0/0" physical port and dumps the property hash:
+
+```ruby
+port = ndev.l1_ports["ge-0/0/1"]
+
+pp port.to_h
+
+-> {"ge-0/0/1"=>
+  {:_active=>true,
+   :_exist=>true,
+   :admin=>:up,
+   :duplex=>:auto,
+   :speed=>:auto,
+   :unit_count=>26}}
+
+```
+
+You can also obtain just a specific property using the `[]` operator:
+
+```ruby
+pp port[:admin]
+
+-> :up
+```
+
+
+
+## Modifying Properties
+
+Modifying the resource property is simply making use of the `[]=` operator.  For example,
+setting the `:untagged_vlan` to "Black" and writing that back to the device would
+look something like this:
+
+```ruby
+port[:untagged_vlan] = "Black"
+port.write!
+```
+
+You can also obtain the `@should` property hash using the `to_h` method and providing the optional `:write` argument:
+
+```ruby
+port[:admin] = :down
+
+pp port.to_h( :write )
+
+-> {"ge-0/0/1"=>{:admin=>:down}}
+```
+
+_NOTE: The `@should` property hash only contains the changes that will be applied, not every property value._
+
+When you execute the `write!` method, the framework will examine the contents of
+the `@should` hash to determine what changes need to be made.  On success the
+values are then transfered into the `@has` hash.
+
+If you're going to make changes to an array property, you would need to do something like this:
+
+```ruby
+port[:tagged_vlans] += ["Purple"]
+port[:tagged_vlans] -= ["Red", "Green"]
+port.write!
+```
+
+_NOTE: for Array values, do not use array methods like `delete` as they will operate
+on the `@has` hash._
+
+## Special Properties
+
+All resources include two special properties `_exist` and `_active`.  These control whether or
+not a resource exists (or should) and whether or not the resource is active (or deactive).  
+Generally speaking you should not modify these properties directly.  Rather you should use the
+`delete!` method to remove the resource and the `activate!` and `deactivate!` methods respectively.
+That being said, if you have a Hash/YAML dataset that explicity has `:_exist => false` when that
+resource is applied to the device, the resource is deleted.  This is handy to ensure a specific
+resource does not exist, for example.
+
+# PROVIDERS
+
+Providers enable access to, and information about resources.  So how to you bind a provider to a 
+Netconf::SSH (netconf) object? This is done by the provider's `Provider` method.  There are
+two techniques for *bindng* a provider to a netconf object.  
+
+One method is to bind the provider after the call to `Netconf::SSH#open`.  For example, if you want 
+to use the L2port provider, you bind it to the netconf object like so:
+
+```ruby
+
+# creating a netconf object, here login is a hash defining login info
+
+ndev = Netconf::SSH.new( login )
+
+# connect to the target
+
+ndev.open
+
+# bind providers to this object
+
+Junos::Ez::Provider( ndev )
+Junos::Ez::L2ports::Provider( ndev, :l2_ports )
+```
+
+But let's say that you want to create multiple `Netconf::SSH` objects and you don't want to
+programmatically do the binding each time?  You can define a new class inheriting `Netconf::SSH` and
+overload the `open` method, for example:
+
+```ruby
+class MyJunosSwitch < Netconf::SSH
+   def open
+      # must be first to open the connection to the target
+      super                      
+      
+      # bind init provider, this will retrieve facts
+      Junos::Ez::Provider( self )  
+      
+      # bind other providers you want this object to have
+      Junos::Ez::L2ports::Provider( self, :l2_ports )
+      Junos::Ez::Vlans::Provider( self, :vlans )
+   end
+end
+
+# now open a few devices ...
+
+dev1 = MyJunosSwitch.new( login )
+dev2 = MyJunosSwitch.new( login )
+
+dev1.open
+dev2.open
+
+pp dev1.vlans.list
+pp dev2.vlans.list
+
+dev1.close
+dev2.close
+```
+
+There are a few things to note on these example:
+
+  1.  This framework is built around the NETCONF gem as all of the underlying code access the Junos XML
+      API via the NETCONF protocol
+
+  2.  You **MUST** use the `Junos::Ez::Provider` before any other providers as this sets up the `Netconf::SSH`
+      object for future bindings and reads the `facts` from the target. These facts can then be
+      used by other provider libraries to abstract target specific differences
+
+  3.  **You** get to chose the provider instance variable name (in this case `l2_ports`, there are is no 
+      hard-coding going on in this framework, yo! (except for the `facts` variable)
+
+## Listing Providers
+
+When you bind providers to a netconf object, you can always get a list of what exists:
+
+```ruby
+pp ndev.providers
+
+-> [:l1_ports, :ip_ports, :l2_ports]
+```
+
+## Resource List
+
+You can obtain a list of managed resources using the `list` or `list!` method.  This method
+will return an Array of names.  Again these names could be simple strings or complex values.
+The `list!` method causes the framework to re-read from the device.  The `list` method uses the cached value.  
+If there is no-cached value, the framework will read from the device, so you don't need to explicity
+use `list!` unless you need to force the cache update.
+
+```ruby
+pp ndev.l2_ports.list
+
+-> ["fe-0/0/2", "fe-0/0/3", "fe-0/0/6"]
+```
+
+## Resource Catalog
+
+You can also obtain the provider's catalog, which is a Hash of resources keyed by name and
+each value is the Hash of the associated properties.  The `catalog` and `catalog!` methods
+work the same as described in the list section above.
+
+```ruby
+pp ndev.l2_ports.catalog
+
+-> {"fe-0/0/2"=>
+  {:_active=>true,
+   :_exist=>true,
+   :vlan_tagging=>true,
+   :tagged_vlans=>["Red", "Green", "Blue"]},
+ "fe-0/0/3"=>{:_active=>true, :_exist=>true, :vlan_tagging=>false},
+ "fe-0/0/6"=>
+  {:_active=>true,
+   :_exist=>true,
+   :vlan_tagging=>false,
+   :untagged_vlan=>"Blue"}}
+```
+
+## Selecting a Resource from a Provider
+
+You select a resource from a provider using the `[]` operator and identifying the resource by *name*.  The 
+name could be a simple string as shown in the previous example, or could be a complex name
+like an Array.  If you take a look a the SRX library (separate repo), you can see that
+the `Junos::Ez::SRX::Policies::Provider` name is an Array of [ from_zone_name, to_zone_name ].  
+
+Here is an example of selecting an ethernet switching port, "ge-0/0/0":
+
+```ruby
+port = ndev.l2_ports['ge-0/0/0']
+```
+
+When a resource is selected, the framework will automatically `read!` retrieve the configuration.
+
+## Creating a new Resource
+
+There are two ways to create a resource.  One is using the `create` or `create!` methods.
+The `create` method is used to create the new resource but not write it do the device.  The
+`create!` method does both the creation and the write to the device.  The `create` method
+can also be given a block, so you can setup the contents of the new resource.  Here's an
+example that creates some config and then deactivates it.
+
+```ruby
+ndev.l2_ports.create('ge-0/0/20') do |port|
+   port[:description] = "I am port 20"
+   port[:untagged_vlan] = "Blue"
+   port.write!
+   port.deactivate!
+end
+```
+The above example will also return the new resource object.   So you could use the Ruby
+block as a default initializer, and then continue to make changes to the resource.  
+
+The second way is to simply select a resource by name that doesn't exist.  So let's
+say you want to create a new L2port for `ge-0/0/20`.  It would look something like this:
+
+```ruby
+port = ndev.l2_ports['ge-0/0/20']
+
+puts "I don't exist" unless port.exists?
+
+port[:description] = "I am port 20"
+port[:untagged_vlan] = "Storage"
+port.write!
+```

data/docs/RE_utils.md

@@ -0,0 +1,3 @@
+# Routing-Engine Utilities
+
+Docs comming soon ...

data/docs/StaticHosts.md

@@ -0,0 +1,3 @@
+# Static Host Entry Provider
+
+Docs comming soon ...

data/docs/StaticRoutes.md

@@ -0,0 +1,3 @@
+# Static Routes Provider
+
+Docs comming soon ...

data/docs/Vlans.md

@@ -0,0 +1,3 @@
+# VLANs Provider
+
+Docs comming soon ...