puppet 4.2.0 → 4.3.0

data/lib/puppet/module_tool/skeleton/templates/generator/README.md.erb

@@ -2,78 +2,82 @@
 
 #### Table of Contents
 
-1. [Overview](#overview)
-2. [Module Description - What the module does and why it is useful](#module-description)
-3. [Setup - The basics of getting started with <%= metadata.name %>](#setup)
+1. [Description](#description)
+1. [Setup - The basics of getting started with <%= metadata.name %>](#setup)
     * [What <%= metadata.name %> affects](#what-<%= metadata.name %>-affects)
     * [Setup requirements](#setup-requirements)
     * [Beginning with <%= metadata.name %>](#beginning-with-<%= metadata.name %>)
-4. [Usage - Configuration options and additional functionality](#usage)
-5. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
-5. [Limitations - OS compatibility, etc.](#limitations)
-6. [Development - Guide for contributing to the module](#development)
+1. [Usage - Configuration options and additional functionality](#usage)
+1. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
+1. [Limitations - OS compatibility, etc.](#limitations)
+1. [Development - Guide for contributing to the module](#development)
 
-## Overview
+## Description
 
-A one-maybe-two sentence summary of what the module does/what problem it solves.
-This is your 30 second elevator pitch for your module. Consider including
-OS/Puppet version it works with.
+Start with a one- or two-sentence summary of what the module does and/or what
+problem it solves. This is your 30-second elevator pitch for your module.
+Consider including OS/Puppet version it works with.
 
-## Module Description
-
-If applicable, this section should have a brief description of the technology
-the module integrates with and what that integration enables. This section
+You can give more descriptive information in a second paragraph. This paragraph
 should answer the questions: "What does this module *do*?" and "Why would I use
-it?"
-
-If your module has a range of functionality (installation, configuration,
-management, etc.) this is the time to mention it.
+it?" If your module has a range of functionality (installation, configuration,
+management, etc.), this is the time to mention it.
 
 ## Setup
 
-### What <%= metadata.name %> affects
+### What <%= metadata.name %> affects **OPTIONAL**
+
+If it's obvious what your module touches, you can skip this section. For
+example, folks can probably figure out that your mysql_instance module affects
+their MySQL instances.
+
+If there's more that they should know about, though, this is the place to mention:
 
 * A list of files, packages, services, or operations that the module will alter,
-  impact, or execute on the system it's installed on.
-* This is a great place to stick any warnings.
-* Can be in list or paragraph form.
+  impact, or execute.
+* Dependencies that your module automatically installs.
+* Warnings or other important notices.
 
 ### Setup Requirements **OPTIONAL**
 
 If your module requires anything extra before setting up (pluginsync enabled,
 etc.), mention it here.
 
-### Beginning with <%= metadata.name %>
+If your most recent release breaks compatibility or requires particular steps
+for upgrading, you might want to include an additional "Upgrading" section
+here.
 
-The very basic steps needed for a user to get the module up and running.
+### Beginning with <%= metadata.name %>
 
-If your most recent release breaks compatibility or requires particular steps
-for upgrading, you may wish to include an additional section here: Upgrading
-(For an example, see http://forge.puppetlabs.com/puppetlabs/firewall).
+The very basic steps needed for a user to get the module up and running. This
+can include setup steps, if necessary, or it can be an example of the most
+basic use of the module.
 
 ## Usage
 
-Put the classes, types, and resources for customizing, configuring, and doing
-the fancy stuff with your module here.
+This section is where you describe how to customize, configure, and do the
+fancy stuff with your module here. It's especially helpful if you include usage
+examples and code samples for doing things with your module.
 
 ## Reference
 
-Here, list the classes, types, providers, facts, etc contained in your module.
-This section should include all of the under-the-hood workings of your module so
-people know what the module is touching on their system but don't need to mess
-with things. (We are working on automating this section!)
+Here, include a complete list of your module's classes, types, providers,
+facts, along with the parameters for each. Users refer to this section (thus
+the name "Reference") to find specific details; most users don't read it per
+se.
 
 ## Limitations
 
-This is where you list OS compatibility, version compatibility, etc.
+This is where you list OS compatibility, version compatibility, etc. If there
+are Known Issues, you might want to include them under their own heading here.
 
 ## Development
 
 Since your module is awesome, other users will want to play with it. Let them
 know what the ground rules for contributing are.
 
-## Release Notes/Contributors/Etc **Optional**
+## Release Notes/Contributors/Etc. **Optional**
 
 If you aren't using changelog, put your release notes here (though you should
-consider using changelog). You may also add any additional sections you feel are
-necessary or important to include here. Please use the `## ` header.
+consider using changelog). You can also add any additional sections you feel
+are necessary or important to include here. Please use the `## ` header.

data/lib/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb

@@ -1,37 +1,44 @@
-# == Class: <%= metadata.name %>
+# Class: <%= metadata.name %>
+# ===========================
 #
 # Full description of class <%= metadata.name %> here.
 #
-# === Parameters
+# Parameters
+# ----------
 #
 # Document parameters here.
 #
-# [*sample_parameter*]
-#   Explanation of what this parameter affects and what it defaults to.
-#   e.g. "Specify one or more upstream ntp servers as an array."
+# * `sample parameter`
+# Explanation of what this parameter affects and what it defaults to.
+# e.g. "Specify one or more upstream ntp servers as an array."
 #
-# === Variables
+# Variables
+# ----------
 #
 # Here you should define a list of variables that this module would require.
 #
-# [*sample_variable*]
-#   Explanation of how this variable affects the function of this class and if
-#   it has a default. e.g. "The parameter enc_ntp_servers must be set by the
-#   External Node Classifier as a comma separated list of hostnames." (Note,
-#   global variables should be avoided in favor of class parameters as
-#   of Puppet 2.6.)
+# * `sample variable`
+#  Explanation of how this variable affects the function of this class and if
+#  it has a default. e.g. "The parameter enc_ntp_servers must be set by the
+#  External Node Classifier as a comma separated list of hostnames." (Note,
+#  global variables should be avoided in favor of class parameters as
+#  of Puppet 2.6.)
 #
-# === Examples
+# Examples
+# --------
 #
-#  class { '<%= metadata.name %>':
-#    servers => [ 'pool.ntp.org', 'ntp.local.company.com' ],
-#  }
+# @example
+#    class { '<%= metadata.name %>':
+#      servers => [ 'pool.ntp.org', 'ntp.local.company.com' ],
+#    }
 #
-# === Authors
+# Authors
+# -------
 #
 # Author Name <author@domain.com>
 #
-# === Copyright
+# Copyright
+# ---------
 #
 # Copyright <%= Time.now.year %> Your name here, unless otherwise noted.
 #

data/lib/puppet/network/authconfig.rb

@@ -3,7 +3,7 @@ require 'puppet/network/http'
 
 module Puppet
   class ConfigurationError < Puppet::Error; end
-  class Network::AuthConfig
+  class Network::DefaultAuthProvider
     attr_accessor :rights
 
     def self.master_url_prefix
@@ -89,4 +89,24 @@ module Puppet
       insert_default_acl
     end
   end
+
+  class Network::AuthConfig
+    @@authprovider_class = nil
+
+    def self.authprovider_class=(klass)
+      @@authprovider_class = klass
+    end
+
+    def self.authprovider_class
+      @@authprovider_class || Puppet::Network::DefaultAuthProvider
+    end
+
+    def initialize(rights=nil)
+      @authprovider = self.class.authprovider_class.new(rights)
+    end
+
+    def check_authorization(method, path, params)
+      @authprovider.check_authorization(method, path, params)
+    end
+  end
 end

data/lib/puppet/network/authorization.rb

@@ -21,8 +21,15 @@ module Puppet::Network
   end
 
   module Authorization
+    @@authconfigloader_class = nil
+
+    def self.authconfigloader_class=(klass)
+      @@authconfigloader_class = klass
+    end
+
     def authconfig
-      AuthConfigLoader.authconfig
+      authconfigloader = @@authconfigloader_class || AuthConfigLoader
+      authconfigloader.authconfig
     end
 
     # Verify that our client has access.

data/lib/puppet/network/http/api/master/v3/environment.rb

@@ -0,0 +1,59 @@
+require 'json'
+require 'puppet/parser/environment_compiler'
+
+class Puppet::Network::HTTP::API::Master::V3::Environment
+  def call(request, response)
+    env_name = request.routing_path.split('/').last
+    env = Puppet.lookup(:environments).get(env_name)
+
+    if env.nil?
+      raise Puppet::Network::HTTP::Error::HTTPNotFoundError.new("#{env_name} is not a known environment", Puppet::Network::HTTP::Issues::RESOURCE_NOT_FOUND)
+    end
+
+    catalog = Puppet::Parser::EnvironmentCompiler.compile(env).to_resource
+
+    env_graph = {:environment => env.name, :applications => {}}
+    applications = catalog.resources.select do |res|
+      type = res.resource_type
+      type.is_a?(Puppet::Resource::Type) && type.application?
+    end
+    applications.each do |app|
+      app_components = {}
+      # Turn the 'nodes' hash into a map component ref => node name
+      node_mapping = {}
+      app['nodes'].each do |node, comps|
+        comps = [comps] unless comps.is_a?(Array)
+        comps.each do |comp|
+          raise Puppet::ParseError, "Application #{app} maps component #{comp} to multiple nodes" if node_mapping.include?(comp.ref)
+          node_mapping[comp.ref] = node.title
+        end
+      end
+
+      catalog.direct_dependents_of(app).each do |comp|
+        mapped_node = node_mapping[comp.ref]
+        if mapped_node.nil?
+          raise Puppet::ParseError, "Component #{comp} is not mapped to any node"
+        end
+        app_components[comp.ref] = {
+          :produces => comp.export.map(&:ref),
+          :consumes => prerequisites(comp).map(&:ref),
+          :node => mapped_node
+        }
+      end
+      env_graph[:applications][app.ref] = app_components
+    end
+    response.respond_with(200, "application/json", JSON.dump(env_graph))
+  end
+
+  private
+
+  # Finds all the prerequisites of component +comp+. They are all the
+  # capability resources that +comp+ depends on; this includes resources
+  # that +comp+ consumes but also resources it merely requires
+  def prerequisites(comp)
+    params = Puppet::Type.relationship_params.select { |p| p.direction == :in }.map(&:name)
+    params.map { |rel| comp[rel] }.flatten.compact.select do |rel|
+      rel.resource_type && rel.resource_type.is_capability?
+    end
+  end
+end

data/lib/puppet/network/http/api/master/v3.rb

@@ -1,6 +1,7 @@
 class Puppet::Network::HTTP::API::Master::V3
   require 'puppet/network/http/api/master/v3/authorization'
   require 'puppet/network/http/api/master/v3/environments'
+  require 'puppet/network/http/api/master/v3/environment'
   require 'puppet/network/http/api/indirected_routes'
 
   AUTHZ = Authorization.new
@@ -14,9 +15,14 @@ class Puppet::Network::HTTP::API::Master::V3
     Environments.new(Puppet.lookup(:environments))
   end)
 
+  ENVIRONMENT = Puppet::Network::HTTP::Route.
+      path(%r{/environment/[^/]+$}).get(AUTHZ.wrap do
+    Environment.new
+  end)
+
   def self.routes
     Puppet::Network::HTTP::Route.path(%r{v3}).
         any.
-        chain(ENVIRONMENTS, INDIRECTED)
+        chain(ENVIRONMENTS, ENVIRONMENT, INDIRECTED)
   end
 end

data/lib/puppet/node/environment.rb

@@ -182,11 +182,18 @@ class Puppet::Node::Environment
   # @api private
   def conflicting_manifest_settings?
     return false if !Puppet[:disable_per_environment_manifest]
-    environment_conf = Puppet.lookup(:environments).get_conf(name)
-    original_manifest = environment_conf.raw_setting(:manifest)
+    original_manifest = configuration.raw_setting(:manifest)
     !original_manifest.nil? && !original_manifest.empty? && original_manifest != Puppet[:default_manifest]
   end
 
+  # Return the environment configuration
+  # @return [Puppet::Settings::EnvironmentConf] The configuration
+  #
+  # @api private
+  def configuration
+    Puppet.lookup(:environments).get_conf(name)
+  end
+
   # Checks the environment and settings for any conflicts
   # @return [Array<String>] an array of validation errors
   # @api public

data/lib/puppet/node.rb

@@ -9,6 +9,9 @@ class Puppet::Node
   # the node sources.
   extend Puppet::Indirector
 
+  # Asymmetric serialization/deserialization required in this class via to/from datahash
+  include Puppet::Util::PsychSupport
+
   # Use the node source as the indirection terminus.
   indirects :node, :terminus_setting => :node_terminus, :doc => "Where to find node information.
     A node is composed of its name, its facts, and its environment."
@@ -18,13 +21,16 @@ class Puppet::Node
 
   attr_reader :server_facts
 
-  def self.from_data_hash(data)
-    raise ArgumentError, "No name provided in serialized data" unless name = data['name']
+  def initialize_from_hash(data)
+    @name       = data['name']       || (raise ArgumentError, "No name provided in serialized data")
+    @classes    = data['classes']    || []
+    @parameters = data['parameters'] || {}
+    @environment_name = data['environment']
+  end
 
+  def self.from_data_hash(data)
     node = new(name)
-    node.classes = data['classes']
-    node.parameters = data['parameters']
-    node.environment_name = data['environment']
+    node.initialize_from_hash(data)
     node
   end
 

data/lib/puppet/parser/ast/pops_bridge.rb

@@ -88,12 +88,18 @@ class Puppet::Parser::AST::PopsBridge
           instantiate_HostClassDefinition(d, modname)
         when Puppet::Pops::Model::ResourceTypeDefinition
           instantiate_ResourceTypeDefinition(d, modname)
+        when Puppet::Pops::Model::CapabilityMapping
+          instantiate_CapabilityMapping(d, modname)
         when Puppet::Pops::Model::NodeDefinition
           instantiate_NodeDefinition(d, modname)
+        when Puppet::Pops::Model::SiteDefinition
+            instantiate_SiteDefinition(d, modname)
         when Puppet::Pops::Model::FunctionDefinition
           instantiate_FunctionDefinition(d, modname)
           # The 3x logic calling this will not know what to do with the result, it is compacted away at the end
           next
+        when Puppet::Pops::Model::Application
+          instantiate_ApplicationDefinition(d, modname)
         else
           raise Puppet::ParseError, "Internal Error: Unknown type of definition - got '#{d.class}'"
         end
@@ -182,6 +188,27 @@ class Puppet::Parser::AST::PopsBridge
       Puppet::Resource::Type.new(:definition, o.name, @context.merge(args_from_definition(o, modname)))
     end
 
+    def instantiate_CapabilityMapping(o, modname)
+      # Use an intermediate 'capability_mapping' type to pass this info to the compiler where the
+      # actual mapping takes place
+      Puppet::Resource::Type.new(:capability_mapping, "#{o.component} #{o.kind} #{o.capability}", { :arguments => {
+        'component'     => o.component,
+        'kind'          => o.kind,
+        'blueprint'     => {
+          :capability => o.capability,
+          :mappings   => o.mappings.reduce({}) do |memo, mapping|
+            memo[mapping.attribute_name] =
+              Expression.new(:value => mapping.value_expr)
+            memo
+          end
+      }}})
+    end
+
+    def instantiate_ApplicationDefinition(o, modname)
+      args = args_from_definition(o, modname)
+      Puppet::Resource::Type.new(:application, o.name, @context.merge(args))
+    end
+
     def instantiate_NodeDefinition(o, modname)
       args = { :module_name => modname }
 
@@ -192,14 +219,25 @@ class Puppet::Parser::AST::PopsBridge
       unless is_nop?(o.parent)
         args[:parent] = @ast_transformer.hostname(o.parent)
       end
+      args = @ast_transformer.merge_location(args, o)
 
       host_matches = @ast_transformer.hostname(o.host_matches)
-      @ast_transformer.merge_location(args, o)
       host_matches.collect do |name|
         Puppet::Resource::Type.new(:node, name, @context.merge(args))
       end
     end
 
+    def instantiate_SiteDefinition(o, modname)
+      args = { :module_name => modname }
+
+      unless is_nop?(o.body)
+        args[:code] = Expression.new(:value => o.body)
+      end
+
+      args = @ast_transformer.merge_location(args, o)
+      Puppet::Resource::Type.new(:site, 'site', @context.merge(args))
+    end
+
     # Propagates a found Function to the appropriate loader.
     # This is for 4x evaluator/loader
     #

data/lib/puppet/parser/compiler/catalog_validator/env_relationship_validator.rb

@@ -0,0 +1,64 @@
+class Puppet::Parser::Compiler
+  # Validator that asserts that all capability resources that are referenced by 'consume' or 'require' has
+  # been exported by some other resource in the environment
+ class CatalogValidator::EnvironmentRelationshipValidator < CatalogValidator
+
+    def validate
+      assumed_exports = {}
+      exported = {}
+      catalog.resources.each do |resource|
+        next unless resource.is_a?(Puppet::Parser::Resource)
+        resource.eachparam do |param|
+          pclass = Puppet::Type.metaparamclass(param.name)
+          validate_relationship(resource, param, assumed_exports, exported) if !pclass.nil? && pclass < Puppet::Type::RelationshipMetaparam
+        end
+      end
+      assumed_exports.each_pair do |key, (param, cap)|
+        raise CatalogValidationError.new("Capability '#{cap}' referenced by '#{param.name}' is never exported", param.file, param.line) unless exported.include?(key)
+      end
+      nil
+    end
+
+    private
+
+    def validate_relationship(resource, param, assumed_exports, exported)
+      case param.name
+      when :require, :consume
+        add_capability_ref(param, param.value, assumed_exports)
+      when :export
+        add_exported(resource, param, param.value, exported)
+      end
+    end
+
+    def add_capability_ref(param, value, assumed_exports)
+      case value
+      when Array
+        value.each { |v| add_capability_ref(param, v, assumed_exports) }
+      when Puppet::Resource
+        rt = value.resource_type
+        unless rt.nil? || !rt.is_capability?
+          title_key = catalog.title_key_for_ref(value.ref)
+          assumed_exports[title_key] = [param, value]
+        end
+        nil
+      end
+    end
+
+    def add_exported(resource, param, value, hash)
+      case value
+      when Array
+        value.each { |v| add_exported(resource, param, v, hash) }
+      when Puppet::Resource
+        rt = value.resource_type
+        unless rt.nil? || !rt.is_capability?
+          title_key = catalog.title_key_for_ref(value.ref)
+          if hash.include?(title_key)
+            raise CatalogValidationError.new("'#{value}' is exported by both '#{hash[title_key]}' and '#{resource}'", param.file, param.line)
+          else
+            hash[title_key] = resource
+          end
+        end
+      end
+    end
+  end
+end

data/lib/puppet/parser/compiler/catalog_validator/relationship_validator.rb

@@ -0,0 +1,38 @@
+class Puppet::Parser::Compiler
+  # Validator that asserts that only 'require', 'consume', and 'export' is used when declaring relationships
+  # to capability resources.
+  class CatalogValidator::RelationshipValidator < CatalogValidator
+    def validate
+      catalog.resources.each do |resource|
+        next unless resource.is_a?(Puppet::Parser::Resource)
+        resource.eachparam do |param|
+          pclass = Puppet::Type.metaparamclass(param.name)
+          validate_relationship(param) if !pclass.nil? && pclass < Puppet::Type::RelationshipMetaparam
+        end
+      end
+      nil
+    end
+
+    private
+
+    def validate_relationship(param)
+      unless [:require, :consume, :export].find {|pname| pname == param.name }
+        raise CatalogValidationError.new("'#{param.name}' is not a valid relationship to a capability", param.file, param.line) if has_capability?(param.value)
+      end
+    end
+
+    def has_capability?(value)
+      case value
+      when Array
+        value.find { |v| has_capability?(v) }
+      when Puppet::Resource
+        rt = value.resource_type
+        !rt.nil? && rt.is_capability?
+      else
+        false
+      end
+    end
+  end
+end
+
+

data/lib/puppet/parser/compiler/catalog_validator/site_validator.rb

@@ -0,0 +1,20 @@
+class Puppet::Parser::Compiler
+  # Validator that asserts that only application components can appear inside a site.
+  class CatalogValidator::SiteValidator < CatalogValidator
+    def self.validation_stage?(stage)
+      PRE_FINISH.equal?(stage)
+    end
+
+    def validate
+      the_site_resource = catalog.resource('Site', 'site')
+      return unless the_site_resource
+
+      catalog.downstream_from_vertex(the_site_resource).keys.each do |r|
+        unless r.is_application_component? || r.resource_type.application?
+          raise CatalogValidationError.new("Only application components can appear inside a site - #{r} is not allowed", r.file, r.line)
+        end
+      end
+    end
+  end
+end
+

data/lib/puppet/parser/compiler/catalog_validator.rb

@@ -0,0 +1,33 @@
+# Abstract class for a catalog validator that can be registered with the compiler to run at
+# a certain stage.
+class Puppet::Parser::Compiler
+  class CatalogValidator
+    PRE_FINISH = :pre_finish
+    FINAL = :final
+
+    # Returns true if the validator should run at the given stage. The default
+    # implementation will only run at stage `FINAL`
+    #
+    # @param stage [Symbol] One of the stage constants defined in this class
+    # @return [Boolean] true if the validator should run at the given stage
+    #
+    def self.validation_stage?(stage)
+      FINAL.equal?(stage)
+    end
+
+    attr_reader :catalog
+
+    # @param catalog [Puppet::Resource::Catalog] The catalog to validate
+    def initialize(catalog)
+      @catalog = catalog
+    end
+
+    # Validate some aspect of the catalog and raise a `CatalogValidationError` on failure
+    def validate
+    end
+  end
+
+  class CatalogValidationError < Puppet::Error
+    include Puppet::ExternalFileError
+  end
+end