dropsonde 0.0.5 → 0.0.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 643def01638761803e04cf758bbf627ac821bb091fc7dad50c5b161251534728
-  data.tar.gz: f0c8a36548214da7beeecee9dc583dcd8d81eceb640b967078fadf5b2978f987
+  metadata.gz: 05a4a694cd3915ec7c3683c7c8250fd1f857d07ef8fb6cae80c832404b35e3ce
+  data.tar.gz: 95245d11a85ef50539f86182cc458246178ffb24ff409bd04fbef9a0a23342ef
 SHA512:
-  metadata.gz: 7c260e1617666fb83d7b0be1a3d73ad13aaee5c33844d5eb4117b186c6eec23a9aad6dc03ff3df393168b6a3715bf492ce677624fdbe536e4961d956d1e9cdcb
-  data.tar.gz: cffbe7901afe08afa593ee73fa749358257efcd03d612564b3aac507684897d2d0db3e504a599d1eab7251508c3f455e236985501ea9f6722f108bb02cf80e2e
+  metadata.gz: a8eed2636ad7735accf3e3f6bacbcb16bae84514b3503e718eb96426ca0bbe6a39a06a9ea06727714b6e578ac4a3bd9ddd6fe76f9bebb4c1d63c084b3503c44b
+  data.tar.gz: 2d1c1cf87cef36cdcb196cab0c135970020389462b99b0ff8447dcefe3ae1f381da31a93ad29a34fedbfd6134eeba2b8c58a3ee30069021c74e3972a857f5950

data/CHANGELOG.md

@@ -0,0 +1,72 @@
+# Changelog
+
+## [0.0.8](https://github.com/puppetlabs/dropsonde/tree/0.0.8) (2022-08-29)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.7...0.0.8)
+
+**Merged pull requests:**
+
+- \[maint\] correct cache [\#29](https://github.com/puppetlabs/dropsonde/pull/29) ([binford2k](https://github.com/binford2k))
+- \[maint\] rubocop style fixes [\#28](https://github.com/puppetlabs/dropsonde/pull/28) ([binford2k](https://github.com/binford2k))
+- Fix example generation [\#27](https://github.com/puppetlabs/dropsonde/pull/27) ([binford2k](https://github.com/binford2k))
+- unused modules [\#26](https://github.com/puppetlabs/dropsonde/pull/26) ([binford2k](https://github.com/binford2k))
+- Adding changelog generation to Rakefile [\#23](https://github.com/puppetlabs/dropsonde/pull/23) ([binford2k](https://github.com/binford2k))
+
+## [v0.0.7](https://github.com/puppetlabs/dropsonde/tree/v0.0.7) (2022-02-10)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.6...v0.0.7)
+
+**Merged pull requests:**
+
+- Releasing v00.7 [\#22](https://github.com/puppetlabs/dropsonde/pull/22) ([binford2k](https://github.com/binford2k))
+- Releasing v00.7 [\#21](https://github.com/puppetlabs/dropsonde/pull/21) ([binford2k](https://github.com/binford2k))
+- update server language [\#20](https://github.com/puppetlabs/dropsonde/pull/20) ([binford2k](https://github.com/binford2k))
+- Updates the telemetry endpoint to a stable URL [\#19](https://github.com/puppetlabs/dropsonde/pull/19) ([MikaelSmith](https://github.com/MikaelSmith))
+- docs updates now that it's shipped [\#18](https://github.com/puppetlabs/dropsonde/pull/18) ([binford2k](https://github.com/binford2k))
+
+## [v0.0.6](https://github.com/puppetlabs/dropsonde/tree/v0.0.6) (2021-11-17)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.5...v0.0.6)
+
+**Merged pull requests:**
+
+- \(MAINT\) Update gemspec license param to 'Apache-2.0' [\#17](https://github.com/puppetlabs/dropsonde/pull/17) ([sanfrancrisko](https://github.com/sanfrancrisko))
+- \(MAINT\) Release prep for 0.0.6 [\#16](https://github.com/puppetlabs/dropsonde/pull/16) ([sanfrancrisko](https://github.com/sanfrancrisko))
+- \(MAINT\) Fix Rubocop violations [\#15](https://github.com/puppetlabs/dropsonde/pull/15) ([sanfrancrisko](https://github.com/sanfrancrisko))
+- \(SERVER-3079\) Allow overriding Puppet's base settings [\#14](https://github.com/puppetlabs/dropsonde/pull/14) ([Magisus](https://github.com/Magisus))
+- Load Ruby's CA certificates instead of using httpclient defaults [\#13](https://github.com/puppetlabs/dropsonde/pull/13) ([Magisus](https://github.com/Magisus))
+- \(IAC-1676\) - Add nightly workflow [\#12](https://github.com/puppetlabs/dropsonde/pull/12) ([adrianiurca](https://github.com/adrianiurca))
+- \(IAC-1659\) - Add Acceptance Tests [\#11](https://github.com/puppetlabs/dropsonde/pull/11) ([adrianiurca](https://github.com/adrianiurca))
+- \(IAC-1624\) - Expand unit test case suit [\#9](https://github.com/puppetlabs/dropsonde/pull/9) ([adrianiurca](https://github.com/adrianiurca))
+- \(IAC-1620\) - replace travis workflow with github actions [\#8](https://github.com/puppetlabs/dropsonde/pull/8) ([adrianiurca](https://github.com/adrianiurca))
+- \(IAC-1605\) - add static checks - rubocop [\#5](https://github.com/puppetlabs/dropsonde/pull/5) ([adrianiurca](https://github.com/adrianiurca))
+- Allow to specify siteid with a flag [\#4](https://github.com/puppetlabs/dropsonde/pull/4) ([raphink](https://github.com/raphink))
+
+## [v0.0.5](https://github.com/puppetlabs/dropsonde/tree/v0.0.5) (2020-06-11)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.4...v0.0.5)
+
+## [v0.0.4](https://github.com/puppetlabs/dropsonde/tree/v0.0.4) (2020-06-05)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.3...v0.0.4)
+
+## [v0.0.3](https://github.com/puppetlabs/dropsonde/tree/v0.0.3) (2020-05-24)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.2...v0.0.3)
+
+## [v0.0.2](https://github.com/puppetlabs/dropsonde/tree/v0.0.2) (2020-05-08)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/v0.0.1...v0.0.2)
+
+## [v0.0.1](https://github.com/puppetlabs/dropsonde/tree/v0.0.1) (2020-02-27)
+
+[Full Changelog](https://github.com/puppetlabs/dropsonde/compare/502bce39de366c8d6816676e0cf7ca6284b90734...v0.0.1)
+
+**Merged pull requests:**
+
+- Add simple puppetfile metric [\#2](https://github.com/puppetlabs/dropsonde/pull/2) ([binford2k](https://github.com/binford2k))
+- MVP workable tool [\#1](https://github.com/puppetlabs/dropsonde/pull/1) ([binford2k](https://github.com/binford2k))
+
+
+
+\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*

data/README.md

@@ -15,8 +15,7 @@ data in a way that lets you signal approval simply by using a module is the
 primary goal of this project.
 
 > **This means that the best way for you to help yourself find new modules is to
-> install this telemetry tool and, along with your peers, share your module usage
-> data.** ✅
+> opt in and use this telemetry tool to share your module usage data.** ✅
 
 
 ## Design
@@ -34,11 +33,16 @@ You can see exactly what will be phoned home by running the command:
 $ dropsonde preview
 ```
 
-Dropsonde is a simple telemetry probe designed to run as a regular cron job.
-Metrics are defined by plugins that gather data, but also export a schema that
-constrains the data allowed to be reported on. Dropsonde will reject metrics
-that don't meet these constraints. The backend database is also defined by this
-schema so the system cannot gather any data that's not described in the schema.
+> *Note:* as of Puppetserver 7.5.0, Dropsonde is bundled in as a standard `puppetserver`
+> command. You should run it as `puppetserver dropsonde` (or `/opt/puppetlabs/bin/puppetserver
+> dropsonde` if your `$PATH` isn't configured.)
+
+Dropsonde is a simple telemetry probe designed to run both as a command-line
+tool and optionally as a regularly scheduled task. Metrics are defined by
+plugins that gather data, but also export a schema that constrains the data
+allowed to be reported on. Dropsonde will reject metrics that don't meet these
+constraints. The backend database is also defined by this schema so the system
+cannot gather any data that's not described in the schema.
 
 See the full schema of all enabled plugins by running the command:
 
@@ -87,9 +91,11 @@ possible: [privacy@puppet.com](mailto:privacy@puppet.com)
 
 ## Installation
 
-This is distributed as a Ruby gem. Simply `gem install dropsonde`. There's a
-[Puppet module](https://github.com/puppetlabs/puppetlabs-dropsonde) to manage it
-if that's more your thing.
+As of Puppetserver 7.5.0, Dropsonde is bundled in as a standard `puppetserver`
+command. If you're running an older version, you can install it yourself with
+`gem install dropsonde` or let the
+[Puppet module](https://github.com/puppetlabs/puppetlabs-dropsonde) manage it
+for you.
 
 
 ## Configuration
@@ -106,7 +112,29 @@ will not report the `:puppetfiles` metrics.
   - puppetfiles
 ```
 
-The `puppetlabs-dropsonde` Puppet module manages this configuration for you.
+The [`puppetlabs-dropsonde`](https://github.com/puppetlabs/puppetlabs-dropsonde)
+Puppet module manages this configuration for you.
+
+
+## Scheduled Reporting
+
+Dropsonde is intended to run on a regular schedule to submit usage reports back
+to Puppet. The [`puppetlabs-dropsonde`](https://github.com/puppetlabs/puppetlabs-dropsonde)
+Puppet module will manage this for you, or you can follow the instructions below
+to configure a schedule manually.
+
+If you're running Puppetserver 7.5.0 or above, it's already set up so you can
+simply opt in. Add or update the following stanza of
+`/etc/puppetlabs/puppetserver/conf.d/puppetserver.conf`:
+
+``` hocon
+dropsonde: {
+    enabled: true
+}
+```
+
+If you're running an older version of Puppetserver, then create a cron job that
+runs `dropsonde submit` once a week.
 
 
 ## Running
@@ -139,7 +167,7 @@ Developer comands
 
 ## Architecture
 
-Dropsonde is a simple telemetry probe designed to run as a regular cron job. It
+Dropsonde is a simple telemetry probe designed to run as a scheduled task. It
 will gather metrics defined by self-contained plugins that each defines its own
 partial schema and then gathers the data to meet that schema.
 
@@ -155,7 +183,8 @@ of modules is updated.
 
 ## Limitations
 
-This is super early in development and has not yet been battle tested.
+The output format of individual metrics is not yet formally defined. If you write
+tooling to use it, then make your tooling resilient to changes.
 
 
 Contact

data/bin/dropsonde

@@ -5,6 +5,9 @@ require 'dropsonde'
 class Dropsonde
   extend GLI::App
 
+  @cache = nil
+  @puppetdb_session = Dropsonde.new
+
   program_desc 'A simple telemetry tool for Puppet infrastructures'
   config_file  "#{File.dirname(Puppet.settings[:confdir])}/telemetry.yaml"
   version      Dropsonde::VERSION
@@ -30,15 +33,18 @@ class Dropsonde
   desc 'Any number or string used to generate the randomized site ID.'
   flag [:seed]
 
+  desc 'Static site ID'
+  flag [:siteid]
+
   pre do |global, command, options, args|
     Dropsonde.settings = global
-    Dropsonde::Cache.init(global[:cachepath], global[:ttl], global[:update])
+    @cache = Dropsonde::Cache.new(global[:cachepath], global[:ttl], global[:update])
   end
 
   desc 'Manually update the Forge module name cache'
   command :update do |c|
     c.action do |global, options, args|
-      Dropsonde::Cache.update
+      @cache.update
     end
   end
 
@@ -55,21 +61,21 @@ class Dropsonde
     c.flag [:format], :default_value => 'human'
 
     c.action do |global, options, args|
-      Dropsonde::Cache.autoupdate
-      Dropsonde.generate_report(options[:format])
+      @cache.autoupdate
+      Dropsonde.generate_report(options[:format], @puppetdb_session)
     end
   end
 
   desc 'Submit a telemetry report'
   command :submit do |c|
     c.desc 'Telemetry endpoint'
-    c.flag [:endpoint], :default_value => 'https://prod.dujour.k8s.puppet.net'
+    c.flag [:endpoint], :default_value => 'https://updates.puppet.com'
 
     c.desc 'Telemetry port'
     c.flag [:port], :default_value => 443, :type => Integer
 
     c.action do |global, options, args|
-      Dropsonde::Cache.autoupdate
+      @cache.autoupdate
       Dropsonde.submit_report(options[:endpoint], options[:port])
     end
   end
@@ -111,7 +117,7 @@ class Dropsonde
       c.flag [:filename], :default_value => 'example.jsonl'
 
       c.action do |global, options, args|
-        Dropsonde::Cache.autoupdate
+        @cache.autoupdate
         Dropsonde.generate_example(options[:size], options[:filename])
       end
     end

data/lib/dropsonde/cache.rb

@@ -1,34 +1,47 @@
+# frozen_string_literal: true
+
 require 'date'
 require 'json'
 require 'fileutils'
 require 'puppet_forge'
 
+# cache class
 class Dropsonde::Cache
-  @@autoupdate = false
+  @@cache = nil # rubocop:disable Style/ClassVars
+  @autoupdate = false
 
-  def self.init(path, ttl, autoupdate)
+  def initialize(path = '~/.dropsonde', ttl = 7, autoupdate = true) # rubocop:disable Style/OptionalBooleanParameter
+    path = File.expand_path(path)
     FileUtils.mkdir_p(path)
-    @@path = "#{File.expand_path(path)}/forge.json"
-    @@ttl  = ttl
-    @@autoupdate = autoupdate
-
-    if File.file? @@path
-      @@cache = JSON.parse(File.read(@@path))
-    else
-      @@cache = {
-                'timestamp' => '2000-1-1',  # long before any puppet modules were released!
-                'modules'   => [],
-              }
-    end
-
-    PuppetForge.user_agent = "Dropsonde Telemetry Client/0.0.1"
+    @path = "#{path}/forge.json"
+    @ttl  = ttl
+    @autoupdate = autoupdate
+
+    @@cache = if File.file? @path # rubocop:disable Style/ClassVars
+                JSON.parse(File.read(@path))
+              else
+                {
+                  'timestamp' => '2000-1-1', # long before any puppet modules were released!
+                  'modules' => [],
+                }
+              end
+
+    PuppetForge.user_agent = 'Dropsonde Telemetry Client/0.0.1'
   end
 
   def self.modules
     @@cache['modules']
   end
 
-  def self.forgeModule?(mod)
+  def modules
+    Dropsonde::Cache.modules
+  end
+
+  def cache
+    @@cache
+  end
+
+  def self.forge_module?(mod)
     case mod
     when Puppet::Module
       modname = mod.forge_slug
@@ -39,12 +52,12 @@ class Dropsonde::Cache
     end
     return unless modname
 
-    modules.include? modname.tr('/','-')
+    @@cache['modules'].include? modname.tr('/', '-')
   end
 
-  def self.update
-    puts "Updating module cache..."
-    iter   = PuppetForge::Module.all(:sort_by => 'latest_release')
+  def update
+    puts 'Updating module cache...'
+    iter   = PuppetForge::Module.all(sort_by: 'latest_release')
     newest = DateTime.parse(@@cache['timestamp'])
 
     @@cache['timestamp'] = iter.first.updated_at
@@ -53,7 +66,7 @@ class Dropsonde::Cache
       # stop once we reach modules we've already cached
       break if DateTime.parse(iter.first.updated_at) <= newest
 
-      @@cache['modules'].concat iter.map {|mod| mod.slug }
+      @@cache['modules'].concat(iter.map { |mod| mod.slug })
 
       iter = iter.next
       print '.'
@@ -62,22 +75,19 @@ class Dropsonde::Cache
     @@cache['modules'].sort!
     @@cache['modules'].uniq!
 
-    File.write(@@path, JSON.pretty_generate(@@cache))
+    File.write(@path, JSON.pretty_generate(@@cache))
   end
 
-  def self.autoupdate
-    return unless @@autoupdate
+  def autoupdate
+    return unless @autoupdate
 
-    unless File.file? @@path
-      puts "Dropsonde caches a list of all Forge modules to ensure that it only reports"
-      puts "usage data on public modules. Generating this cache may take some time on"
+    unless File.file? @path
+      puts 'Dropsonde caches a list of all Forge modules to ensure that it only reports'
+      puts 'usage data on public modules. Generating this cache may take some time on'
       puts "the first run and you'll see your screen fill up with dots."
       update
     end
 
-    if (Date.today - File.mtime(@@path).to_date).to_i > @@ttl
-      update
-    end
+    return update if (Date.today - File.mtime(@path).to_date).to_i > @ttl
   end
-
 end

data/lib/dropsonde/metrics/dependencies.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+
+# dependencies plugin
 class Dropsonde::Metrics::Dependencies
   def self.initialize_dependencies
     # require any libraries needed here -- no need to load puppet; it's already initialized
@@ -5,10 +8,10 @@ class Dropsonde::Metrics::Dependencies
   end
 
   def self.description
-    <<~EOF
+    <<~DESCRIPTION
       This group of metrics discovers dependencies between modules in all
       environments. It will omit dependencies on private modules.
-    EOF
+    DESCRIPTION
   end
 
   def self.schema
@@ -18,23 +21,23 @@ class Dropsonde::Metrics::Dependencies
       {
         "fields": [
           {
-            "description": "The depended on module name",
-            "mode": "NULLABLE",
-            "name": "name",
-            "type": "STRING"
+            "description": 'The depended on module name',
+            "mode": 'NULLABLE',
+            "name": 'name',
+            "type": 'STRING',
           },
           {
-            "description": "The depended on module version requirement",
-            "mode": "NULLABLE",
-            "name": "version_requirement",
-            "type": "STRING"
-          }
+            "description": 'The depended on module version requirement',
+            "mode": 'NULLABLE',
+            "name": 'version_requirement',
+            "type": 'STRING',
+          },
         ],
-        "description": "List of modules that private modules in all environments depend on.",
-        "mode": "REPEATED",
-        "name": "dependencies",
-        "type": "RECORD"
-      }
+        "description": 'List of modules that private modules in all environments depend on.',
+        "mode": 'REPEATED',
+        "name": 'dependencies',
+        "type": 'RECORD',
+      },
     ]
   end
 
@@ -42,26 +45,25 @@ class Dropsonde::Metrics::Dependencies
     # run just before generating this metric
   end
 
-  def self.run
+  def self.run(_puppetdb_session = nil)
     # return an array of hashes representing the data to be merged into the combined checkin
-    environments = Puppet.lookup(:environments).list.map{|e|e.name}
-    modules = environments.map do |env|
+    environments = Puppet.lookup(:environments).list.map { |e| e.name }
+    modules = environments.map { |env|
       Puppet.lookup(:environments).get(env).modules
-    end.flatten
+    }.flatten
 
     # we want only PUBLIC modules that PRIVATE modules depend on
-    dependencies = modules.map do|mod|
+    dependencies = modules.map { |mod|
       next unless mod.dependencies
-      next if Dropsonde::Cache.forgeModule? mod  # skip unless this is a private module
+      next if Dropsonde::Cache.forge_module? mod # skip unless this is a private module
 
       # and return a list of all public modules it depends on
-      mod.dependencies.select {|mod| Dropsonde::Cache.forgeModule? mod }
-    end.flatten.compact
+      mod.dependencies.select { |dep| Dropsonde::Cache.forge_module? dep }
+    }.flatten.compact
 
     [
-      { :dependencies => dependencies },
+      { dependencies: dependencies },
     ]
-
   end
 
   def self.example
@@ -71,12 +73,14 @@ class Dropsonde::Metrics::Dependencies
 
     versions = ['>= 1.5.2', '>= 4.3.2', '>= 3.0.0 < 4.0.0', '>= 2.2.1 < 5.0.0', '>= 5.0.0 < 7.0.0', '>= 4.11.0']
     [
-      :dependencies => Dropsonde::Cache.modules
-                                  .sample(rand(250))
-                                  .map {|item| {
-                                    :name                => item,
-                                    :version_requirement => versions.sample,
-                                  }},
+      dependencies: Dropsonde::Cache.modules
+                                    .sample(rand(250))
+                                    .map do |item|
+                      {
+                        name: item,
+                        version_requirement: versions.sample,
+                      }
+                    end,
     ]
   end
 

data/lib/dropsonde/metrics/environments.rb

@@ -1,3 +1,6 @@
+# frozen_string_literal: true
+
+# environments plugin
 class Dropsonde::Metrics::Environments
   def self.initialize_environments
     # Require any libraries needed here -- no need to load puppet or puppetdb;
@@ -7,20 +10,20 @@ class Dropsonde::Metrics::Environments
 
   def self.description
     # This is a Ruby squiggle heredoc; just a multi-line string with indentation removed
-    <<~EOF
+    <<~DESCRIPTION
       This group of metrics gathers information about environments.
-    EOF
+    DESCRIPTION
   end
 
   def self.schema
     # return an array of hashes of a partial schema to be merged into the complete schema
     [
       {
-        "description": "The number of environments",
-        "mode": "NULLABLE",
-        "name": "environment_count",
-        "type": "INTEGER"
-      }
+        "description": 'The number of environments',
+        "mode": 'NULLABLE',
+        "name": 'environment_count',
+        "type": 'INTEGER',
+      },
     ]
   end
 
@@ -28,10 +31,10 @@ class Dropsonde::Metrics::Environments
     # run just before generating this metric
   end
 
-  def self.run
+  def self.run(_puppetdb_session = nil)
     # return an array of hashes representing the data to be merged into the combined checkin
     [
-      :environment_count => Puppet.lookup(:environments).list.count,
+      environment_count: Puppet.lookup(:environments).list.count,
     ]
   end
 
@@ -40,7 +43,7 @@ class Dropsonde::Metrics::Environments
     # make it easier to write data aggregation queries without access to the
     # actual private data that users have submitted.
     [
-      :environment_count => rand(1..100),
+      environment_count: rand(1..100),
     ]
   end