puppet-resource_api 1.2.0 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fd035e8b155f1bf4307be10cd13021a16f91aae8d8ef3bfd652f5a77a67a3556
-  data.tar.gz: 43d002ab24ab5e320357684b297c8e01f6f79f661f7ed062410619ed37146a94
+  metadata.gz: 48ed58d882063171b3d9cd9b0711403cf6d8bc0c213aa98d0583d2b0bf8746a7
+  data.tar.gz: c6acb1edb2ee5b1cefe1cb0449090ee3072e380632952d82948b6eb2455a49b4
 SHA512:
-  metadata.gz: c316e50d658c0c65501defdf61392f7d2ee586e0ebf4b1b78f9a1da98b3a289d65b6265f2ba68a8fdf3d23f2ab612591c280a7ca25846d1db66d6e530228ab9f
-  data.tar.gz: e4f60efe74b30a0770e07acb9e98024f0bcbaf6ea3d5bc5214b59835510c8c0edd757c3da50e88ab7bf65947c63a9fb9f0ea97315ae3051fb063af6ecf18f44f
+  metadata.gz: 4fc8c73ce49206569836f00aa1c52380b1571055ccc3f6842577a6c890c657b19280ad9b6c09e376db8acc3b63daa700cebe6d64c47571d464df797ba1b0e482
+  data.tar.gz: 9e64c5cc2d4474014ed8f45d66b44ad0416874f29b6b2f6c5b0a0da01e93179525d3eee33029f3e28eaa29acb5f5ea2d537935265f608d23610e63d51e5b3ec3

data/.rubocop.yml

@@ -4,7 +4,7 @@ inherit_from: .rubocop_todo.yml
 AllCops:
   TargetRubyVersion: '2.1'
   Include:
-  - "./**/*.rb"
+  - "**/*.rb"
   Exclude:
   - bin/*
   - ".vendor/**/*"

data/.travis.yml

@@ -8,6 +8,9 @@ script:
   - echo travis_fold:end:DEBUG
   - COVERAGE=yes bundle exec rake $CHECK
 cache: bundler
+branches:
+  except:
+  - release-prep
 matrix:
   include:
     - rvm: 2.4.3

data/CHANGELOG.md

@@ -3,6 +3,28 @@
 All significant changes to this repo will be summarized in this file.
 
 
+## [v1.3.0](https://github.com/puppetlabs/puppet-resource_api/tree/v1.3.0) (2018-05-24)
+[Full Changelog](https://github.com/puppetlabs/puppet-resource_api/compare/v1.2.0...v1.3.0)
+
+**Implemented enhancements:**
+
+- Check for more attributes that puppet can't use [\#84](https://github.com/puppetlabs/puppet-resource_api/pull/84) ([DavidS](https://github.com/DavidS))
+- \(PDK-531\) Support for composite namevars [\#82](https://github.com/puppetlabs/puppet-resource_api/pull/82) ([da-ar](https://github.com/da-ar))
+- \(PDK-889\) Write support for multiple namevars [\#79](https://github.com/puppetlabs/puppet-resource_api/pull/79) ([da-ar](https://github.com/da-ar))
+- \(PDK-889\) Read-only support for multiple namevars [\#76](https://github.com/puppetlabs/puppet-resource_api/pull/76) ([da-ar](https://github.com/da-ar))
+
+**Fixed bugs:**
+
+- Ignore `provider` attribute when calculating target state [\#83](https://github.com/puppetlabs/puppet-resource_api/pull/83) ([DavidS](https://github.com/DavidS))
+- Add check to handle absent resources through puppet apply [\#81](https://github.com/puppetlabs/puppet-resource_api/pull/81) ([da-ar](https://github.com/da-ar))
+- \(PDK-988\) restrain mungify from non-`puppet resource` workflows [\#80](https://github.com/puppetlabs/puppet-resource_api/pull/80) ([DavidS](https://github.com/DavidS))
+
+**Merged pull requests:**
+
+- Update fixtures module to PDK v1.5 [\#78](https://github.com/puppetlabs/puppet-resource_api/pull/78) ([DavidS](https://github.com/DavidS))
+- Some glue fixes: announcement, to\_manifest, to\_hierayaml [\#77](https://github.com/puppetlabs/puppet-resource_api/pull/77) ([DavidS](https://github.com/DavidS))
+- Release prep for v1.2.0 [\#75](https://github.com/puppetlabs/puppet-resource_api/pull/75) ([DavidS](https://github.com/DavidS))
+
 ## [v1.2.0](https://github.com/puppetlabs/puppet-resource_api/tree/v1.2.0) (2018-05-08)
 [Full Changelog](https://github.com/puppetlabs/puppet-resource_api/compare/v1.1.0...v1.2.0)
 

data/README.md

@@ -209,8 +209,9 @@ There are still a few notable gaps between the implementation and the specificat
 * Complex data types, like Hash, Tuple or Struct are not yet implemented.
 * Only a single runtime environment (the Puppet commands) is currently implemented.
 
-Restrictions of running under puppet:
-* `supports_noop` is not effective, as puppet doesn't call into the type under noop.
+Restrictions of puppet:
+* `supports_noop` is not effective, as puppet doesn't call into the type under noop at all.
+* Attributes cannot be called `title`, `provider`, or any of the [metaparameters](https://puppet.com/docs/puppet/5.5/metaparameter.html), as those are reserved by puppet itself.
 
 Future possibilities:
 * [Composite Namevars](https://tickets.puppetlabs.com/browse/PDK-531)
@@ -239,9 +240,8 @@ To cut a new release, from a current `master` checkout:
   * double check the PRs to make sure they're all tagged correctly (using the new CHANGELOG for cross-checking)
 * Check README and other materials for up-to-date-ness
 * Commit changes with title "Release prep for v<VERSION>"
-* Upload and PR the release prep to github through the main repo starting with `git push upstream`
-  * make sure to use the name of your git remote pointing to main repo instead of "upstream"
+* Upload and PR the release-prep branch to the puppetlabs GitHub repo
 * Check that CI is green and merge the PR
 * Run `rake release[upstream]` to release from your checkout
-  * make sure to use the name of your git remote pointing to main repo instead of "upstream"
+  * make sure to use the name of your git remote pointing to the puppetlabs GitHub repo
 * Remove the release-prep branch

data/appveyor.yml

@@ -1,5 +1,9 @@
 # version: 1.0.{build}-{branch}
 
+branches:
+  except:
+  - release-prep
+
 environment:
   matrix:
     - RUBY_VERSION: 24-x64

data/lib/puppet/resource_api.rb

@@ -7,9 +7,17 @@ require 'puppet/type'
 
 module Puppet::ResourceApi
   def register_type(definition)
-    raise Puppet::DevError, 'requires a Hash as definition, not %{other_type}' % { other_type: definition.class } unless definition.is_a? Hash
-    raise Puppet::DevError, 'requires a name' unless definition.key? :name
-    raise Puppet::DevError, 'requires attributes' unless definition.key? :attributes
+    raise Puppet::DevError, 'requires a hash as definition, not `%{other_type}`' % { other_type: definition.class } unless definition.is_a? Hash
+    raise Puppet::DevError, 'requires a `:name`' unless definition.key? :name
+    raise Puppet::DevError, 'requires `:attributes`' unless definition.key? :attributes
+    raise Puppet::DevError, '`:attributes` must be a hash, not `%{other_type}`' % { other_type: definition[:attributes].class } unless definition[:attributes].is_a?(Hash)
+    [:title, :provider, :alias, :audit, :before, :consume, :export, :loglevel, :noop, :notify, :require, :schedule, :stage, :subscribe, :tag].each do |name|
+      raise Puppet::DevError, 'must not define an attribute called `%{name}`' % { name: name.inspect } if definition[:attributes].key? name
+    end
+    if definition.key?(:title_patterns) && !definition[:title_patterns].is_a?(Array)
+      raise Puppet::DevError, '`:title_patterns` must be an array, not `%{other_type}`' % { other_type: definition[:title_patterns].class }
+    end
+
     validate_ensure(definition)
 
     definition[:features] ||= []
@@ -37,8 +45,6 @@ module Puppet::ResourceApi
 
     Puppet::Type.newtype(definition[:name].to_sym) do
       @docs = definition[:docs]
-      has_namevar = false
-      namevar_name = nil
 
       # Keeps a copy of the provider around. Weird naming to avoid clashes with puppet's own `provider` member
       define_singleton_method(:my_provider) do
@@ -65,15 +71,26 @@ module Puppet::ResourceApi
       define_method(:initialize) do |attributes|
         # $stderr.puts "A: #{attributes.inspect}"
         if attributes.is_a? Puppet::Resource
+          @title = attributes.title
           attributes = attributes.to_hash
         else
-          @called_from_resource = true
+          @ral_find_absent = true
         end
         # $stderr.puts "B: #{attributes.inspect}"
         if type_definition.feature?('canonicalize')
           attributes = my_provider.canonicalize(context, [attributes])[0]
         end
-        # $stderr.puts "C: #{attributes.inspect}"
+
+        # the `Puppet::Resource::Ral.find` method, when `instances` does not return a match, uses a Hash with a `:name` key to create
+        # an "absent" resource. This is often hit by `puppet resource`. This needs to work, even if the namevar is not called `name`.
+        # This bit here relies on the default `title_patterns` (see below) to match the title back to the first (and often only) namevar
+        if definition[:attributes][:name].nil? && attributes[:title].nil?
+          attributes[:title] = attributes.delete(:name)
+          if attributes[:title].nil? && !type_definition.namevars.empty?
+            attributes[:title] = @title
+          end
+        end
+
         super(attributes)
       end
 
@@ -81,6 +98,10 @@ module Puppet::ResourceApi
         # enforce mandatory attributes
         @missing_attrs = []
         @missing_params = []
+
+        # do not validate on known-absent instances
+        return if @ral_find_absent
+
         definition[:attributes].each do |name, options|
           type = Puppet::Pops::Types::TypeParser.singleton.parse(options[:type])
 
@@ -98,7 +119,7 @@ module Puppet::ResourceApi
           end
         end
 
-        @missing_attrs -= [:ensure] if @called_from_resource
+        @missing_attrs -= [:ensure]
 
         raise_missing_params if @missing_params.any?
       end
@@ -115,7 +136,7 @@ module Puppet::ResourceApi
         # TODO: using newparam everywhere would suppress change reporting
         #       that would allow more fine-grained reporting through context,
         #       but require more invest in hooking up the infrastructure to emulate existing data
-        param_or_property = if [:read_only, :parameter, :namevar].include? options[:behaviour]
+        param_or_property = if [:read_only, :parameter].include? options[:behaviour]
                               :newparam
                             else
                               :newproperty
@@ -133,11 +154,7 @@ module Puppet::ResourceApi
           end
 
           if options[:behaviour] == :namevar
-            # puts 'setting namevar'
-            # raise Puppet::DevError, "namevar must be called 'name', not '#{name}'" if name.to_s != 'name'
             isnamevar
-            has_namevar = true
-            namevar_name = name
           end
 
           # read-only values do not need type checking, but can have default values
@@ -199,16 +216,16 @@ module Puppet::ResourceApi
             end
           end
 
-          if type.instance_of? Puppet::Pops::Types::POptionalType
-            type = type.type
-          end
-
           # puppet symbolizes some values through puppet/parameter/value.rb (see .convert()), but (especially) Enums
           # are strings. specifying a munge block here skips the value_collection fallback in puppet/parameter.rb's
           # default .unsafe_munge() implementation.
           munge { |v| v }
 
           # provide hints to `puppet type generate` for better parsing
+          if type.instance_of? Puppet::Pops::Types::POptionalType
+            type = type.type
+          end
+
           case type
           when Puppet::Pops::Types::PStringType
             # require any string value
@@ -250,10 +267,12 @@ module Puppet::ResourceApi
             property = definition[:attributes][key.first]
             attr_def[key.first] = property
           end
-          if resource_hash[namevar_name].nil?
-            raise Puppet::ResourceError, "`#{name}.get` did not return a value for the `#{namevar_name}` namevar attribute"
+          context.type.namevars.each do |namevar|
+            if resource_hash[namevar].nil?
+              raise Puppet::ResourceError, "`#{name}.get` did not return a value for the `#{namevar}` namevar attribute"
+            end
           end
-          Puppet::ResourceApi::TypeShim.new(resource_hash[namevar_name], resource_hash, name, namevar_name, attr_def)
+          Puppet::ResourceApi::TypeShim.new(resource_hash, name, context.type.namevars, attr_def)
         end
       end
 
@@ -264,7 +283,7 @@ module Puppet::ResourceApi
         current_state = if type_definition.feature?('simple_get_filter')
                           my_provider.get(context, [title]).first
                         else
-                          my_provider.get(context).find { |h| h[namevar_name] == title }
+                          my_provider.get(context).find { |h| namevar_match?(h) }
                         end
 
         strict_check(current_state) if current_state && type_definition.feature?('canonicalize')
@@ -274,7 +293,7 @@ module Puppet::ResourceApi
             result[k] = v
           end
         else
-          result[namevar_name] = title
+          result[:title] = title
           result[:ensure] = :absent if type_definition.ensurable?
         end
 
@@ -288,12 +307,18 @@ module Puppet::ResourceApi
         result
       end
 
+      define_method(:namevar_match?) do |item|
+        context.type.namevars.all? do |namevar|
+          item[namevar] == @parameters[namevar].value if @parameters[namevar].respond_to? :value
+        end
+      end
+
       define_method(:flush) do
         raise_missing_attrs
 
         # puts 'flush'
         # skip puppet's injected metaparams
-        target_state = Hash[@parameters.reject { |k, _v| [:loglevel, :noop].include? k }.map { |k, v| [k, v.rs_value] }]
+        target_state = Hash[@parameters.reject { |k, _v| [:loglevel, :noop, :provider].include? k }.map { |k, v| [k, v.rs_value] }]
         target_state = my_provider.canonicalize(context, [target_state]).first if type_definition.feature?('canonicalize')
 
         retrieve unless @rapi_current_state
@@ -348,7 +373,7 @@ module Puppet::ResourceApi
         #:nocov:
         # codecov fails to register this multiline as covered, even though simplecov does.
         message = <<MESSAGE.strip
-#{definition[:name]}[#{current_state[namevar_name]}]#get has not provided canonicalized values.
+#{definition[:name]}[#{@title}]#get has not provided canonicalized values.
 Returned values:       #{current_state.inspect}
 Canonicalized values:  #{state_clone.inspect}
 MESSAGE
@@ -372,6 +397,31 @@ MESSAGE
         self.class.context
       end
 
+      define_singleton_method(:title_patterns) do
+        @title_patterns ||= if definition.key? :title_patterns
+                              parse_title_patterns(definition[:title_patterns])
+                            else
+                              [[%r{(.*)}m, [[type_definition.namevars.first]]]]
+                            end
+      end
+
+      # Creates a `title_pattern` compatible data structure to pass to the underlying puppet runtime environment.
+      # It uses the named items in the regular expression to connect the dots
+      #
+      # @example `[ %r{^(?<package>.*[^-])-(?<manager>.*)$} ]` becomes
+      #   [
+      #     [
+      #       %r{^(?<package>.*[^-])-(?<manager>.*)$},
+      #       [ [:package], [:manager] ]
+      #     ],
+      #   ]
+      def self.parse_title_patterns(patterns)
+        patterns.map do |item|
+          regex = Regexp.new(item[:pattern])
+          [item[:pattern], regex.names.map { |x| [x.to_sym] }]
+        end
+      end
+
       [:autorequire, :autobefore, :autosubscribe, :autonotify].each do |auto|
         next unless definition[auto]
 
@@ -420,18 +470,32 @@ MESSAGE
     end
   end
 
-  # Validates and munges values coming from puppet into a shape palatable to the provider.
-  # This includes parsing values from strings, e.g. when running in `puppet resource`.
+  def self.caller_is_resource_app?
+    caller.any? { |c| c.match(%r{application/resource.rb:}) }
+  end
+
+  # This method handles translating values from the runtime environment to the expected types for the provider.
+  # When being called from `puppet resource`, it tries to transform the strings from the command line into their
+  # expected ruby representations, e.g. `"2"` (a string), will be transformed to `2` (the number) if (and only if)
+  # the target `type` is `Integer`.
+  # Additionally this function also validates that the passed in (and optionally transformed) value matches the
+  # specified type.
   # @param type[Puppet::Pops::Types::TypedModelObject] the type to check/clean against
   # @param value the value to clean
   # @param error_msg_prefix[String] a prefix for the error messages
   # @return [type] the cleaned value
   # @raise [Puppet::ResourceError] if `value` could not be parsed into `type`
   def self.mungify(type, value, error_msg_prefix)
-    cleaned_value, error = try_mungify(type, value, error_msg_prefix)
-
-    raise Puppet::ResourceError, error if error
-
+    if caller_is_resource_app?
+      # When the provider is exercised from the `puppet resource` CLI, we need to unpack strings into
+      # the correct types, e.g. "1" (a string) to 1 (an integer)
+      cleaned_value, error_msg = try_mungify(type, value, error_msg_prefix)
+      raise Puppet::ResourceError, error_msg if error_msg
+    else
+      # Every other time, we can use the values as is
+      cleaned_value = value
+    end
+    Puppet::ResourceApi.validate(type, cleaned_value, error_msg_prefix)
     cleaned_value
   end
 
@@ -494,13 +558,41 @@ MESSAGE
       # fall through to default handling
     end
 
-    # a match!
-    return [value, nil] if type.instance?(value)
+    error_msg = try_validate(type, value, error_msg_prefix)
+    if error_msg
+      # an error :-(
+      [nil, error_msg]
+    else
+      # a match!
+      [value, nil]
+    end
+  end
+
+  # Validates the `value` against the specified `type`.
+  # @param type[Puppet::Pops::Types::TypedModelObject] the type to check against
+  # @param value the value to clean
+  # @param error_msg_prefix[String] a prefix for the error messages
+  # @raise [Puppet::ResourceError] if `value` is not of type `type`
+  # @private
+  def self.validate(type, value, error_msg_prefix)
+    error_msg = try_validate(type, value, error_msg_prefix)
+
+    raise Puppet::ResourceError, error_msg if error_msg
+  end
+
+  # Tries to validate the `value` against the specified `type`.
+  # @param type[Puppet::Pops::Types::TypedModelObject] the type to check against
+  # @param value the value to clean
+  # @param error_msg_prefix[String] a prefix for the error messages
+  # @return [String, nil] a error message indicating the problem, or `nil` if the value was valid.
+  # @private
+  def self.try_validate(type, value, error_msg_prefix)
+    return nil if type.instance?(value)
 
     # an error :-(
     inferred_type = Puppet::Pops::Types::TypeCalculator.infer_set(value)
     error_msg = Puppet::Pops::Types::TypeMismatchDescriber.new.describe_mismatch(error_msg_prefix, type, inferred_type)
-    [nil, error_msg]
+    error_msg
   end
 
   def self.validate_ensure(definition)

data/lib/puppet/resource_api/glue.rb

@@ -1,43 +1,43 @@
+require 'yaml'
+
 module Puppet; end # rubocop:disable Style/Documentation
 module Puppet::ResourceApi
   # A trivial class to provide the functionality required to push data through the existing type/provider parts of puppet
   class TypeShim
-    attr_reader :values, :typename, :namevar, :attr_def
+    attr_reader :values, :typename, :namevars, :attr_def
 
-    def initialize(title, resource_hash, typename, namevarname, attr_def)
+    def initialize(resource_hash, typename, namevars, attr_def)
       # internalize and protect - needs to go deeper
-      @values = resource_hash.dup
-      # "name" is a privileged key
-      @values[namevarname] = title
-      @values.freeze
+      @values = resource_hash.dup.freeze
 
       @typename = typename
-      @namevar = namevarname
+      @namevars = namevars
       @attr_def = attr_def
+      @resource = ResourceShim.new(@values, @typename, @namevars, @attr_def)
     end
 
     def to_resource
-      ResourceShim.new(@values, @typename, @namevar, @attr_def)
+      @resource
     end
 
     def name
-      values[@namevar]
+      @resource.title
     end
   end
 
   # A trivial class to provide the functionality required to push data through the existing type/provider parts of puppet
   class ResourceShim
-    attr_reader :values, :typename, :namevar, :attr_def
+    attr_reader :values, :typename, :namevars, :attr_def
 
-    def initialize(resource_hash, typename, namevarname, attr_def)
+    def initialize(resource_hash, typename, namevars, attr_def)
       @values = resource_hash.dup.freeze # whatevs
       @typename = typename
-      @namevar = namevarname
+      @namevars = namevars
       @attr_def = attr_def
     end
 
     def title
-      values[@namevar]
+      values[@namevars.first]
     end
 
     def prune_parameters(*_args)
@@ -46,7 +46,7 @@ module Puppet::ResourceApi
     end
 
     def to_manifest
-      (["#{@typename} { #{values[@namevar].inspect}: "] + values.keys.reject { |k| k == @namevar }.map do |k|
+      (["#{@typename} { #{Puppet::Parameter.format_value_for_display(title)}: "] + filtered_keys.map do |k|
         cs = ' '
         ce = ''
         if attr_def[k][:behaviour] && attr_def[k][:behaviour] == :read_only
@@ -59,7 +59,13 @@ module Puppet::ResourceApi
 
     # Convert our resource to yaml for Hiera purposes.
     def to_hierayaml
-      (["  #{values[@namevar]}: "] + values.keys.reject { |k| k == @namevar }.map { |k| "    #{k}: #{Puppet::Parameter.format_value_for_display(values[k])}" }).join("\n") + "\n"
+      attributes = Hash[filtered_keys.map { |k| [k.to_s, values[k]] }]
+      YAML.dump('type' => { title => attributes }).split("\n").drop(2).join("\n") + "\n"
+    end
+
+    # attribute names that are not title or namevars
+    def filtered_keys
+      values.keys.reject { |k| k == :title || attr_def[k][:behaviour] == :namevar && @namevars.size == 1 }
     end
   end
 end

data/lib/puppet/resource_api/type_definition.rb

@@ -15,6 +15,12 @@ class Puppet::ResourceApi::TypeDefinition
     @definition[:attributes].key?(:ensure)
   end
 
+  def namevars
+    @namevars ||= @definition[:attributes].select { |_name, options|
+      options.key?(:behaviour) && options[:behaviour] == :namevar
+    }.keys
+  end
+
   # rubocop complains when this is named has_feature?
   def feature?(feature)
     supported = (definition[:features] && definition[:features].include?(feature))

data/lib/puppet/resource_api/version.rb

@@ -1,5 +1,5 @@
 module Puppet
   module ResourceApi
-    VERSION = '1.2.0'.freeze
+    VERSION = '1.3.0'.freeze
   end
 end

data/misc/ANNOUNCEMENT_TEMPLATE.md

@@ -16,7 +16,7 @@ Hi all,
 
 We're pleased to announce that version X.Y.Z of the Resource API is being released today.
 
-The Resource API provides a simple way to create new native resources in the form of types and providers for Puppet. It is provided as a Ruby gem to be referenced within modules. Support for it has been included as an experimental feature in version 1.4 of the Puppet Development Kit (`pdk new provider`). Use the [resource_api module](https://forge.puppet.com/puppetlabs/resource_api) to deploy it in your infrastructure.
+The Resource API provides a simple way to create new native resources in the form of types and providers for Puppet. It is provided as a Ruby gem to be referenced within modules. Support for it has been included as an experimental feature in version 1.4 of the Puppet Development Kit (`pdk new provider`). Use the [resource_api module](https://forge.puppet.com/puppetlabs/resource_api) or the [puppet 6 nightly packages](https://groups.google.com/d/msg/puppet-users/N3LJGhsrqkU/TUEsq7VfDQAJ) to deploy it in your infrastructure.
 
 The new release of the Resource API provides the following enhancements:
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: puppet-resource_api
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.3.0
 platform: ruby
 authors:
 - David Schmitt
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-05-08 00:00:00.000000000 Z
+date: 2018-05-24 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hocon
@@ -81,7 +81,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.7.6
+rubygems_version: 2.7.3
 signing_key: 
 specification_version: 4
 summary: This library provides a simple way to write new native resources for puppet.