puppet-resource_api 0.1.0

data/NOTICE

@@ -0,0 +1,17 @@
+Puppet Resource API
+
+Copyright (C) 2017 Puppet, Inc.
+
+Puppet Labs can be contacted at: info@puppetlabs.com
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -0,0 +1,50 @@
+# Puppet::ResourceApi [![Build Status](https://travis-ci.org/puppetlabs/puppet-resource_api.svg?branch=master)](https://travis-ci.org/puppetlabs/puppet-resource_api) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qvor6rkh0d1e4suc?svg=true)](https://ci.appveyor.com/project/puppetlabs/puppet-resource-api) [![Coverage Status](https://coveralls.io/repos/github/puppetlabs/puppet-resource_api/badge.svg?branch=master)](https://coveralls.io/github/puppetlabs/puppet-resource_api?branch=master) [![codecov](https://codecov.io/gh/puppetlabs/puppet-resource_api/branch/master/graph/badge.svg)](https://codecov.io/gh/puppetlabs/puppet-resource_api)
+
+
+This is an implementation of the [Resource API](https://github.com/DavidS/puppet-specifications/blob/resourceapi/language/resource-api/README.md) proposal. Find a working example of a new-style provider in the [experimental puppetlabs-apt branch](https://github.com/DavidS/puppetlabs-apt/blob/resource-api-experiments/lib/puppet/provider/apt_key2/apt_key2.rb). There is also the corresponding [type](https://github.com/DavidS/puppetlabs-apt/blob/resource-api-experiments/lib/puppet/type/apt_key2.rb), and [new unit tests](https://github.com/DavidS/puppetlabs-apt/blob/resource-api-experiments/spec/unit/puppet/provider/apt_key2/apt_key2_spec.rb) for 100% coverage.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'puppet-resource_api'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install puppet-resource_api
+
+## Usage
+
+The [Resource API](https://github.com/DavidS/puppet-specifications/blob/resourceapi/language/resource-api/README.md) explains the usage and capabilities of this gem.
+
+Already working:
+* basic type and provider definition, using `name`, `desc`, and `attributes`
+* the `canonicalize` and `remote_resource` features
+* all the logging facilities
+* executing the new provider under any of the following commands:
+  * `puppet apply`
+  * `puppet resource`
+  * `puppet agent`
+  * `puppet device` (if applicable)
+
+
+There are still a few notable gaps between the implementation, and the specification:
+* Only a single runtime environment (the puppet commands) is currently implemented.
+* `auto*` definitions
+* the Commands API is mostly implemented, but deployment is blocked on upstream work (PDK-580). You can use regular Ruby `system()` calls as a workaround, with all their attendant encoding, and safety issues.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/puppet-resource_api.

data/Rakefile

@@ -0,0 +1,24 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require 'rubocop/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+RuboCop::RakeTask.new(:rubocop) do |t|
+  t.options = ['--display-cop-names']
+end
+
+namespace :spec do
+  desc 'Run RSpec code examples with coverage collection'
+  task :coverage do
+    ENV['COVERAGE'] = 'yes'
+    Rake::Task['spec'].execute
+  end
+end
+
+desc 'Check for unapproved licenses in dependencies'
+task(:license_finder) do
+  system('license_finder --decisions-file=.dependency_decisions.yml') || raise(StandardError, 'Unapproved license(s) found on dependencies')
+end
+
+task :default => :spec

data/appveyor.yml

@@ -0,0 +1,39 @@
+# version: 1.0.{build}-{branch}
+
+environment:
+  matrix:
+    - RUBY_VERSION: 24-x64
+    - RUBY_VERSION: 21-x64
+
+install:
+  - set PATH=C:\Ruby%RUBY_VERSION%\bin;C:\mingw-w64\x86_64-6.3.0-posix-seh-rt_v5-rev1\mingw64\bin;%PATH%
+  - SET LOG_SPEC_ORDER=true
+  - SET COVERAGE=yes
+  # Due to a bug in the version of OpenSSL shipped with Ruby 2.4.1 on Windows
+  # (https://bugs.ruby-lang.org/issues/11033). Errors are ignored because the
+  # mingw gem calls out to pacman to install OpenSSL which is already
+  # installed, causing gem to raise a warning that powershell determines to be
+  # a fatal error.
+  - ps: |
+      $ErrorActionPreference = "SilentlyContinue"
+      if($env:RUBY_VERSION -eq "24-x64") {
+        gem install openssl "~> 2.0.4" --no-rdoc --no-ri -- --with-openssl-dir=C:\msys64\mingw64
+      }
+      $host.SetShouldExit(0)
+  - bundle install --jobs 4 --retry 2 --without development
+
+build: off
+
+before_test:
+  - ruby -v
+  - gem -v
+  - bundle -v
+  - type Gemfile.lock
+
+test_script:
+  - bundle exec rake
+
+# Uncomment this block to enable RDP access to the AppVeyor test instance for
+# debugging purposes.
+#on_finish:
+#  - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "puppet/resource_api"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/ext/mkrf_conf.rb

@@ -0,0 +1,20 @@
+# Based on the example from https://en.wikibooks.org/wiki/Ruby_Programming/RubyGems#How_to_install_different_versions_of_gems_depending_on_which_version_of_ruby_the_installee_is_using
+require 'rubygems'
+require 'rubygems/command.rb'
+require 'rubygems/dependency_installer.rb'
+begin
+  Gem::Command.build_args = ARGV
+rescue NoMethodError # rubocop:disable Lint/HandleExceptions
+end
+inst = Gem::DependencyInstaller.new
+begin
+  if RbConfig::CONFIG['host_os'] =~ %r{mswin|msys|mingw32}i
+    inst.install 'childprocess', '~> 0.7'
+  end
+rescue # rubocop:disable Lint/RescueWithoutErrorClass
+  exit(1)
+end
+
+f = File.open(File.join(File.dirname(__FILE__), 'Rakefile'), 'w') # create dummy rakefile to indicate success
+f.write("task :default\n")
+f.close

data/lib/puppet/resource_api.rb

@@ -0,0 +1,236 @@
+require 'pathname'
+# require 'puppet/resource_api/command'
+require 'puppet/resource_api/errors'
+require 'puppet/resource_api/glue'
+require 'puppet/resource_api/puppet_context'
+require 'puppet/resource_api/version'
+require 'puppet/type'
+
+module Puppet::ResourceApi
+  def register_type(definition)
+    raise Puppet::DevError, 'requires a Hash as definition, not %{other_type}' % { other_type: definition.class } unless definition.is_a? Hash
+    raise Puppet::DevError, 'requires a name' unless definition.key? :name
+    raise Puppet::DevError, 'requires attributes' unless definition.key? :attributes
+
+    # prepare the ruby module for the provider
+    # this has to happen before Puppet::Type.newtype starts autoloading providers
+    # it also needs to be guarded against the namespace already being defined by something
+    # else to avoid ruby warnings
+    unless Puppet::Provider.const_defined?(class_name_from_type_name(definition[:name]))
+      Puppet::Provider.const_set(class_name_from_type_name(definition[:name]), Module.new)
+    end
+
+    Puppet::Type.newtype(definition[:name].to_sym) do
+      @docs = definition[:docs]
+      has_namevar = false
+      namevar_name = nil
+
+      # Keeps a copy of the provider around. Weird naming to avoid clashes with puppet's own `provider` member
+      define_singleton_method(:my_provider) do
+        @my_provider ||= Puppet::ResourceApi.load_provider(definition[:name]).new
+      end
+
+      # make the provider available in the instance's namespace
+      def my_provider
+        self.class.my_provider
+      end
+
+      if definition[:features] && definition[:features].include?('remote_resource')
+        apply_to_device
+      end
+
+      define_method(:initialize) do |attributes|
+        # $stderr.puts "A: #{attributes.inspect}"
+        attributes = attributes.to_hash if attributes.is_a? Puppet::Resource
+        # $stderr.puts "B: #{attributes.inspect}"
+        if definition.key?(:features) && definition[:features].include?('canonicalize')
+          attributes = my_provider.canonicalize(context, [attributes])[0]
+        end
+        # $stderr.puts "C: #{attributes.inspect}"
+        super(attributes)
+      end
+
+      definition[:attributes].each do |name, options|
+        # puts "#{name}: #{options.inspect}"
+
+        # TODO: using newparam everywhere would suppress change reporting
+        #       that would allow more fine-grained reporting through context,
+        #       but require more invest in hooking up the infrastructure to emulate existing data
+        param_or_property = if [:read_only, :parameter, :namevar].include? options[:behaviour]
+                              :newparam
+                            else
+                              :newproperty
+                            end
+        send(param_or_property, name.to_sym) do
+          unless options[:type]
+            raise Puppet::DevError, "#{definition[:name]}.#{name} has no type"
+          end
+
+          if options[:desc]
+            desc "#{options[:desc]} (a #{options[:type]})"
+          else
+            warn("#{definition[:name]}.#{name} has no docs")
+          end
+
+          if options[:behaviour] == :namevar
+            # puts 'setting namevar'
+            # raise Puppet::DevError, "namevar must be called 'name', not '#{name}'" if name.to_s != 'name'
+            isnamevar
+            has_namevar = true
+            namevar_name = name
+          end
+
+          # read-only values do not need type checking, but can have default values
+          if options[:behaviour] != :read_only
+            # TODO: this should use Pops infrastructure to avoid hardcoding stuff, and enhance type fidelity
+            # validate do |v|
+            #   type = Puppet::Pops::Types::TypeParser.singleton.parse(options[:type]).normalize
+            #   if type.instance?(v)
+            #     return true
+            #   else
+            #     inferred_type = Puppet::Pops::Types::TypeCalculator.infer_set(value)
+            #     error_msg = Puppet::Pops::Types::TypeMismatchDescriber.new.describe_mismatch("#{DEFINITION[:name]}.#{name}", type, inferred_type)
+            #     raise Puppet::ResourceError, error_msg
+            #   end
+            # end
+
+            if options.key? :default
+              defaultto options[:default]
+            end
+
+            case options[:type]
+            when 'String'
+              # require any string value
+              newvalues %r{} do
+              end
+            # rubocop:disable Lint/BooleanSymbol
+            when 'Boolean'
+              newvalues 'true', 'false', :true, :false, true, false
+
+              munge do |v|
+                case v
+                when 'true', :true
+                  true
+                when 'false', :false
+                  false
+                else
+                  v
+                end
+              end
+            # rubocop:enable Lint/BooleanSymbol
+            when 'Integer'
+              newvalue %r{^-?\d+$} do
+              end
+              munge do |v|
+                Puppet::Pops::Utils.to_n(v)
+              end
+            when 'Float', 'Numeric'
+              newvalue Puppet::Pops::Patterns::NUMERIC do
+              end
+              munge do |v|
+                Puppet::Pops::Utils.to_n(v)
+              end
+            when 'Enum[present, absent]'
+              newvalue :absent do
+              end
+              newvalue :present do
+              end
+            when 'Variant[Pattern[/\A(0x)?[0-9a-fA-F]{8}\Z/], Pattern[/\A(0x)?[0-9a-fA-F]{16}\Z/], Pattern[/\A(0x)?[0-9a-fA-F]{40}\Z/]]'
+              # the namevar needs to be a Parameter, which only has newvalue*s*
+              newvalues(%r{\A(0x)?[0-9a-fA-F]{8}\Z}, %r{\A(0x)?[0-9a-fA-F]{16}\Z}, %r{\A(0x)?[0-9a-fA-F]{40}\Z})
+            when 'Optional[String]'
+              newvalues(%r{}, :undef) do
+              end
+            when 'Variant[Stdlib::Absolutepath, Pattern[/\A(https?|ftp):\/\//]]'
+              # TODO: this is wrong, but matches original implementation
+              [/^\//, /\A(https?|ftp):\/\//].each do |v| # rubocop:disable Style/RegexpLiteral
+                newvalues v do
+                end
+              end
+            when 'Pattern[/\A((hkp|http|https):\/\/)?([a-z\d])([a-z\d-]{0,61}\.)+[a-z\d]+(:\d{2,5})?$/]'
+              newvalues(/\A((hkp|http|https):\/\/)?([a-z\d])([a-z\d-]{0,61}\.)+[a-z\d]+(:\d{2,5})?$/) # rubocop:disable Style/RegexpLiteral
+            else
+              raise Puppet::DevError, "Datatype #{options[:type]} is not yet supported in this prototype"
+            end
+          end
+        end
+      end
+
+      define_singleton_method(:instances) do
+        # puts 'instances'
+        # force autoloading of the provider
+        provider(name)
+        my_provider.get(context).map do |resource_hash|
+          Puppet::ResourceApi::TypeShim.new(resource_hash[namevar_name], resource_hash)
+        end
+      end
+
+      define_method(:retrieve) do
+        # puts "retrieve(#{title.inspect})"
+        result        = Puppet::Resource.new(self.class, title)
+        current_state = my_provider.get(context).find { |h| h[namevar_name] == title }
+
+        # require 'pry'; binding.pry
+
+        if current_state
+          current_state.each do |k, v|
+            result[k] = v
+          end
+        else
+          result[:name] = title
+          result[:ensure] = :absent
+        end
+
+        # puts "retrieved #{current_state.inspect}"
+
+        @rapi_current_state = current_state
+        result
+      end
+
+      define_method(:flush) do
+        # puts 'flush'
+        # require'pry';binding.pry
+        target_state = Hash[@parameters.map { |k, v| [k, v.value] }]
+        # remove puppet's injected metaparams
+        target_state.delete(:loglevel)
+        target_state = my_provider.canonicalize(context, [target_state]).first if definition.key?(:features) && definition[:features].include?('canonicalize')
+
+        retrieve unless @rapi_current_state
+
+        # require 'pry'; binding.pry
+        return if @rapi_current_state == target_state
+
+        # puts "@rapi_current_state: #{@rapi_current_state.inspect}"
+        # puts "target_state: #{target_state.inspect}"
+
+        my_provider.set(context, title => { is: @rapi_current_state, should: target_state })
+        raise 'Execution encountered an error' if context.failed?
+      end
+
+      define_singleton_method(:context) do
+        @context ||= PuppetContext.new(definition[:name])
+      end
+
+      def context
+        self.class.context
+      end
+    end
+  end
+  module_function :register_type
+
+  def load_provider(type_name)
+    class_name = class_name_from_type_name(type_name)
+    type_name_sym = type_name.to_sym
+
+    # loads the "puppet/provider/#{type_name}/#{type_name}" file through puppet
+    Puppet::Type.type(type_name_sym).provider(type_name_sym)
+    Puppet::Provider.const_get(class_name).const_get(class_name)
+  rescue NameError
+    raise Puppet::DevError, "class #{class_name} not found in puppet/provider/#{type_name}/#{type_name}"
+  end
+  module_function :load_provider
+
+  def self.class_name_from_type_name(type_name)
+    type_name.to_s.split('_').map(&:capitalize).join
+  end
+end

data/lib/puppet/resource_api/base_context.rb

@@ -0,0 +1,138 @@
+require 'puppet/util'
+require 'puppet/util/network_device'
+
+class Puppet::ResourceApi::BaseContext
+  def initialize(typename)
+    @typename = typename
+  end
+
+  def device
+    # TODO: evaluate facter_url setting for loading config if there is no `current` NetworkDevice
+    raise 'no device configured' unless Puppet::Util::NetworkDevice.current
+    Puppet::Util::NetworkDevice.current
+  end
+
+  def failed?
+    @failed
+  end
+
+  [:debug, :info, :notice, :warning, :err].each do |level|
+    define_method(level) do |*args|
+      if args.length == 1
+        message = "#{@context || @typename}: #{args.last}"
+      elsif args.length == 2
+        resources = format_titles(args.first)
+        message = "#{resources}: #{args.last}"
+      else
+        message = args.map(&:to_s).join(', ')
+      end
+      send_log(level, message)
+    end
+  end
+
+  [:creating, :updating, :deleting].each do |method|
+    define_method(method) do |titles, message: method.to_s.capitalize, &block|
+      start_time = Time.now
+      setup_context(titles, message)
+      begin
+        debug('Start')
+        block.call
+        notice("Finished in #{format_seconds(Time.now - start_time)} seconds")
+      rescue StandardError => e
+        err("Failed after #{format_seconds(Time.now - start_time)} seconds: #{e}")
+        @failed = true
+      ensure
+        @context = nil
+      end
+    end
+  end
+
+  def failing(titles, message: 'Failing')
+    start_time = Time.now
+    setup_context(titles, message)
+    begin
+      debug('Start')
+      yield
+      warning("Finished failing in #{format_seconds(Time.now - start_time)} seconds")
+    rescue StandardError => e
+      err("Failed after #{format_seconds(Time.now - start_time)} seconds: #{e}")
+      @failed = true
+    ensure
+      @context = nil
+    end
+  end
+
+  def processing(title, is, should, message: 'Processing')
+    raise "#{__method__} only accepts a single resource title" if title.respond_to?(:each)
+    start_time = Time.now
+    setup_context(title, message)
+    begin
+      debug("Starting processing of #{title} from #{is} to #{should}")
+      yield
+      notice("Finished processing #{title} in #{format_seconds(Time.now - start_time)} seconds: #{should}")
+    rescue StandardError => e
+      err("Failed processing #{title} after #{format_seconds(Time.now - start_time)} seconds: #{e}")
+      @failed = true
+    ensure
+      @context = nil
+    end
+  end
+
+  [:created, :updated, :deleted].each do |method|
+    define_method(method) do |titles, message: method.to_s.capitalize|
+      notice("#{message}: #{titles}")
+    end
+  end
+
+  def processed(title, is, should)
+    raise "#{__method__} only accepts a single resource title" if title.respond_to?(:each)
+    notice("Processed #{title} from #{is} to #{should}")
+  end
+
+  def attribute_changed(title, attribute, is, should, message: nil)
+    raise "#{__method__} only accepts a single resource title" if title.respond_to?(:each)
+    printable_is = 'nil'
+    printable_should = 'nil'
+    if is
+      printable_is = is.is_a?(Numeric) ? is : "'#{is}'"
+    end
+    if should
+      printable_should = should.is_a?(Numeric) ? should : "'#{should}'"
+    end
+    notice("#{title}: attribute '#{attribute}' changed from #{printable_is} to #{printable_should}#{message ? ": #{message}" : ''}")
+  end
+
+  def failed(titles, message: 'Updating has failed')
+    setup_context(titles)
+    begin
+      err(message)
+      # raise message
+    ensure
+      @context = nil
+    end
+  end
+
+  def send_log(_level, _message)
+    raise 'Received send_log() on an unprepared BaseContext. Use IOContext, or PuppetContext instead.'
+  end
+
+  private
+
+  def format_titles(titles)
+    if titles.length.zero? && !titles.is_a?(String)
+      @typename
+    else
+      "#{@typename}[#{[titles].flatten.compact.join(', ')}]"
+    end
+  end
+
+  def setup_context(titles, message = nil)
+    @context = format_titles(titles)
+    @context += ": #{message}" if message
+  end
+
+  def format_seconds(seconds)
+    return '%.6f' % seconds if seconds < 1
+    '%.2f' % seconds
+  end
+end