librato-rack 1.1.1 → 2.0.0

data/CHANGELOG.md

@@ -1,3 +1,6 @@
+### Version 2.0.0
+* Add support for tagged measurements (#54)
+
 ### Version 1.1.0
 * Fix deprecation warnings in ruby 2.4 (#57, Ben Radler)
 

data/README.md

@@ -49,7 +49,7 @@ If you don't have a Metrics account already, [sign up](https://metrics.librato.c
 
 By default you can use `LIBRATO_USER` and `LIBRATO_TOKEN` to pass your account data to the middleware. While these are the only required variables, there are a few more optional environment variables you may find useful.
 
-* `LIBRATO_SOURCE` - the default source to use for submitted metrics. If this is not set, hostname of the executing machine will be the default source
+* `LIBRATO_TAGS` - the default tags to use for submitted metrics. Format is comma-separated key=value pairs, e.g. `region=us-east,az=b`. If not set, `host` of the executing machine is detected and set as default tag
 * `LIBRATO_SUITES` - manage which metrics librato-rack will report. See more in [metrics suites](#metric-suites).
 * `LIBRATO_PREFIX` - a prefix which will be prepended to all metric names
 * `LIBRATO_LOG_LEVEL` - see logging section for more
@@ -74,19 +74,36 @@ See the [configuration class](https://github.com/librato/librato-rack/blob/maste
 
 If you are using the [Librato Metrics Heroku addon](https://addons.heroku.com/librato), your `LIBRATO_USER` and `LIBRATO_TOKEN` environment variables will already be set in your Heroku environment. If you are running without the addon you will need to provide them yourself.
 
-You must also specify a custom source for your app to track properly. If an explicit source is not set, `librato-rack` will not start. You can set the source in your environment:
+NOTE: if Heroku idles your application no measurements will be sent until it receives another request and is restarted. If you see intermittent gaps in your measurements during periods of low traffic this is the most likely cause.
 
-    heroku config:add LIBRATO_SOURCE=myappname
+## Default Tags
 
-NOTE: if Heroku idles your application no measurements will be sent until it receives another request and is restarted. If you see intermittent gaps in your measurements during periods of low traffic this is the most likely cause.
+Librato Metrics supports tagged measurements that are associated with a metric, one or more tag pairs, and a point in time. For more information on tagged measurements, visit our [API documentation](https://www.librato.com/docs/api/#measurements).
+
+##### Detected Tags
+
+By default, `host` is detected and applied as a default tag for submitted measurements. Optionally, you can override the detected values, e.g. `LIBRATO_TAGS=host=myapp-prod-1`
+
+##### Custom Tags
+
+In addition to the default tags, you can also provide custom tags:
+
+```ruby
+config = Librato::Rack::Configuration.new
+config.user = 'myuser@mysite.com'
+config.token = 'mytoken'
+config.tags = { service: 'myapp', environment: 'production', host: 'myapp-prod-1' }
+
+use Librato::Rack, :config => config
+```
 
 ##### Metric Suites
 
 The metrics recorded by `librato-rack` are organized into named metric suites that can be selectively enabled/disabled:
 
 * `rack`: The `rack.request.total`, `rack.request.time`, `rack.request.slow`, and `rack.request.queue.time` metrics
-* `rack_status`: All of the `rack.request.status` metrics
-* `rack_method`: All of the `rack.request.method` metrics
+* `rack_status`: `rack.request.status` metric with `status` tag name and HTTP status code tag value, e.g. `status=200`
+* `rack_method`: `rack.request.method` metric with `method` tag name and HTTP method tag value, e.g. `method=POST`
 
 All three of the metric suites listed above are enabled by default.
 
@@ -116,14 +133,23 @@ Tracking anything that interests you is easy with Metrics. There are four primar
 
 Use for tracking a running total of something _across_ requests, examples:
 
-    # increment the 'sales_completed' metric by one
-    Librato.increment 'sales.completed'
+```ruby
+# increment the 'sales_completed' metric by one
+Librato.increment 'sales.completed'
+# => {:host=>"myapp-prod-1"}
 
-    # increment by five
-    Librato.increment 'items.purchased', :by => 5
+# increment by five
+Librato.increment 'items.purchased', by: 5
+# => {:host=>"myapp-prod-1"}
 
-    # increment with a custom source
-    Librato.increment 'user.purchases', :source => user.id
+# increment with custom per-measurement tags
+Librato.increment 'user.purchases', tags: { user_id: user.id, currency: 'USD' }
+# => {:user_id=>43, :currency=>"USD"}
+
+# increment with custom per-measurement tags and inherited default tags
+Librato.increment 'user.purchases', tags: { user_id: user.id, currency: 'USD' }, inherit_tags: true
+# => {:host=>"myapp-prod-1", :user_id=>43, :currency=>"USD"}
+```
 
 Other things you might track this way: user signups, requests of a certain type or to a certain route, total jobs queued or processed, emails sent or received
 
@@ -131,10 +157,10 @@ Other things you might track this way: user signups, requests of a certain type
 
 Note that `increment` is primarily used for tracking the rate of occurrence of some event. Given this increment metrics are _continuous by default_: after being called on a metric once they will report on every interval, reporting zeros for any interval when increment was not called on the metric.
 
-Especially with custom sources you may want the opposite behavior - reporting a measurement only during intervals where `increment` was called on the metric:
+Especially with custom per-measurement tags you may want the opposite behavior - reporting a measurement only during intervals where `increment` was called on the metric:
 
     # report a value for 'user.uploaded_file' only during non-zero intervals
-    Librato.increment 'user.uploaded_file', :source => user.id, :sporadic => true
+    Librato.increment 'user.uploaded_file', tags: { user: user.id, bucket: bucket.name }, sporadic: true
 
 #### measure
 
@@ -142,8 +168,8 @@ Use when you want to track an average value _per_-request. Examples:
 
     Librato.measure 'user.social_graph.nodes', 212
 
-    # report from a custom source
-    Librato.measure 'jobs.queued', 3, :source => 'worker.12'
+    # report from custom per-measurement tags
+    Librato.measure 'jobs.queued', 3, tags: { priority: 'high', worker: 'worker.12' }
 
 #### timing
 
@@ -205,14 +231,11 @@ If you are using `librato-rack` with sidekiq, [see these notes about setup](http
 
 ## Cross-Process Aggregation
 
-`librato-rack` submits measurements back to the Librato platform on a _per-process_ basis. By default these measurements are then combined into a single measurement per source (default is your hostname) before persisting the data.
+`librato-rack` submits measurements back to the Librato platform on a _per-process_ basis. By default these measurements are then combined into a single measurement per default tags (detects `host`) before persisting the data.
 
 For example if you have 4 hosts with 8 unicorn instances each (i.e. 32 processes total), on the Metrics site you'll find 4 data streams (1 per host) instead of 32.
 Current pricing applies after aggregation, so in this case you will be charged for 4 streams instead of 32.
 
-If you want to report per-process instead, you can set `source_pids` to `true` in
-your config, which will append the process id to the source name used by each thread.
-
 ## Troubleshooting
 
 Note that it may take 2-3 minutes for the first results to show up in your Metrics account after you have started your servers with `librato-rack` enabled and the first request has been received.

data/lib/librato/collector/aggregator.rb

@@ -5,16 +5,19 @@ module Librato
     # maintains storage of timing and measurement type measurements
     #
     class Aggregator
-      SOURCE_SEPARATOR = '$$'
+      SEPARATOR = "$$"
 
       extend Forwardable
 
       def_delegators :@cache, :empty?, :prefix, :prefix=
 
+      attr_reader :default_tags
+
       def initialize(options={})
         @cache = Librato::Metrics::Aggregator.new(prefix: options[:prefix])
         @percentiles = {}
         @lock = Mutex.new
+        @default_tags = options.fetch(:default_tags, {})
       end
 
       def [](key)
@@ -26,13 +29,13 @@ module Librato
       def fetch(key, options={})
         return nil if @cache.empty?
         return fetch_percentile(key, options) if options[:percentile]
-        gauges = nil
-        source = options[:source]
-        @lock.synchronize { gauges = @cache.queued[:gauges] }
-        gauges.each do |metric|
+        measurements = nil
+        tags = options[:tags] || @default_tags
+        @lock.synchronize { measurements = @cache.queued[:measurements] }
+        measurements.each do |metric|
           if metric[:name] == key.to_s
-            return metric if !source && !metric[:source]
-            return metric if source.to_s == metric[:source]
+            return metric if !tags && !metric[:tags]
+            return metric if tags == metric[:tags]
           end
         end
         nil
@@ -89,18 +92,27 @@ module Librato
         if args.length > 1 and args[-1].respond_to?(:each)
           options = args[-1]
         end
-        source = options[:source]
-        percentiles = Array(options[:percentile])
 
-        @lock.synchronize do
-          if source
-            @cache.add event => {source: source, value: value}
+        percentiles = Array(options[:percentile])
+        source = options[:source]
+        tags_option = options[:tags]
+        tags_option = { source: source } if source && !tags_option
+        tags =
+          if tags_option && options[:inherit_tags]
+            @default_tags.merge(tags_option)
+          elsif tags_option
+            tags_option
           else
-            @cache.add event => value
+            @default_tags
           end
 
+        @lock.synchronize do
+          payload = { value: value }
+          payload.merge!({ tags: tags }) if tags
+          @cache.add event => payload
+
           percentiles.each do |perc|
-            store = fetch_percentile_store(event, source)
+            store = fetch_percentile_store(event, payload)
             store[:reservoir] << value
             track_percentile(store, perc)
           end
@@ -120,30 +132,38 @@ module Librato
       end
 
       def fetch_percentile(key, options)
-        store = fetch_percentile_store(key, options[:source])
+        store = fetch_percentile_store(key, options)
         return nil unless store
         store[:reservoir].percentile(options[:percentile])
       end
 
-      def fetch_percentile_store(event, source)
-        keyname = source ? "#{event}#{SOURCE_SEPARATOR}#{source}" : event
+      def fetch_percentile_store(event, options)
+        keyname = event
+
+        if options[:tags]
+          keyname = Librato::Metrics::Util.build_key_for(keyname, options[:tags])
+        end
+
         @percentiles[keyname] ||= {
+          name: event,
           reservoir: Hetchy::Reservoir.new(size: 1000),
           percs: Set.new
         }
+        @percentiles[keyname].merge!({ tags: options[:tags] }) if options && options[:tags]
+        @percentiles[keyname]
       end
 
       def flush_percentiles(queue, opts)
         @percentiles.each do |key, val|
-          metric, source = key.split(SOURCE_SEPARATOR)
           val[:percs].each do |perc|
             perc_name = perc.to_s[0,5].gsub('.','')
-            payload = if source
-              { value: val[:reservoir].percentile(perc), source: source }
-            else
-              val[:reservoir].percentile(perc)
-            end
-            queue.add "#{metric}.p#{perc_name}" => payload
+            payload =
+              if val[:tags]
+                { value: val[:reservoir].percentile(perc), tags: val[:tags] }
+              else
+                val[:reservoir].percentile(perc)
+              end
+            queue.add "#{val[:name]}.p#{perc_name}" => payload
           end
         end
       end
@@ -158,4 +178,4 @@ module Librato
     end
 
   end
-end
+end

data/lib/librato/collector/counter_cache.rb

@@ -1,3 +1,5 @@
+require "json"
+
 module Librato
   class Collector
     # maintains storage of a set of incrementable, counter-like
@@ -11,10 +13,13 @@ module Librato
 
       def_delegators :@cache, :empty?
 
-      def initialize
+      attr_reader :default_tags
+
+      def initialize(options={})
         @cache = {}
         @lock = Mutex.new
         @sporadics = Set.new
+        @default_tags = options.fetch(:default_tags, {})
       end
 
       # Retrieve the current value for a given metric. This is a short
@@ -34,14 +39,15 @@ module Librato
         @lock.synchronize { @cache.clear }
       end
 
-
       def fetch(key, options={})
-        if options[:source]
-          key = "#{key}#{SEPARATOR}#{options[:source]}"
-        end
-        @lock.synchronize do
-          @cache[key.to_s]
-        end
+        key = key.to_s
+        key =
+          if options[:tags]
+            Librato::Metrics::Util.build_key_for(key, options[:tags])
+          elsif @default_tags
+            Librato::Metrics::Util.build_key_for(key, @default_tags)
+          end
+        @lock.synchronize { @cache[key] }
       end
 
       # transfer all measurements to queue and reset internal status
@@ -50,16 +56,13 @@ module Librato
         @lock.synchronize do
           # work off of a duplicate data set so we block for
           # as little time as possible
-          counts = @cache.dup
+          # requires a deep copy of data set
+          counts = JSON.parse(@cache.dup.to_json, symbolize_names: true)
           reset_cache unless opts[:preserve]
         end
-        counts.each do |metric, value|
-          metric, source = metric.split(SEPARATOR)
-          if source
-            queue.add metric => {value: value, source: source}
-          else
-            queue.add metric => value
-          end
+        counts.each do |metric, payload|
+          metric = metric.to_s.split(SEPARATOR).first
+          queue.add metric => payload
         end
       end
 
@@ -75,22 +78,33 @@ module Librato
       #   increment :foo, :source => user.id
       #
       def increment(counter, options={})
+        metric = counter.to_s
         if options.is_a?(INTEGER_CLASS)
           # suppport legacy style
           options = {by: options}
         end
         by = options[:by] || 1
-        if options[:source]
-          metric = "#{counter}#{SEPARATOR}#{options[:source]}"
-        else
-          metric = counter.to_s
-        end
+        source = options[:source]
+        tags_option = options[:tags]
+        tags_option = { source: source } if source && !tags_option
+        tags =
+          if tags_option && options[:inherit_tags]
+            @default_tags.merge(tags_option)
+          elsif tags_option
+            tags_option
+          else
+            @default_tags
+          end
+        metric = Librato::Metrics::Util.build_key_for(metric, tags) if tags
         if options[:sporadic]
           make_sporadic(metric)
         end
         @lock.synchronize do
-          @cache[metric] ||= 0
-          @cache[metric] += by
+          @cache[metric] = {} unless @cache[metric]
+          @cache[metric][:name] ||= metric
+          @cache[metric][:value] ||= 0
+          @cache[metric][:value] += by
+          @cache[metric][:tags] = tags if tags
         end
       end
 
@@ -105,7 +119,7 @@ module Librato
         @sporadics.each { |metric| @cache.delete(metric) }
         @sporadics.clear
         # reset all continuous source/metric pairs to 0
-        @cache.each_key { |key| @cache[key] = 0 }
+        @cache.each_key { |key| @cache[key][:value] = 0 }
       end
 
     end

data/lib/librato/collector.rb

@@ -10,14 +10,20 @@ module Librato
     def_delegators :counters, :increment
     def_delegators :aggregate, :measure, :timing
 
+    attr_reader :tags
+
+    def initialize(options={})
+      @tags = options[:tags]
+    end
+
     # access to internal aggregator object
     def aggregate
-      @aggregator_cache ||= Aggregator.new(prefix: @prefix)
+      @aggregator_cache ||= Aggregator.new(prefix: @prefix, default_tags: @tags)
     end
 
     # access to internal counters object
     def counters
-      @counter_cache ||= CounterCache.new
+      @counter_cache ||= CounterCache.new(default_tags: @tags)
     end
 
     # remove any accumulated but unsent metrics
@@ -48,4 +54,4 @@ end
 require_relative 'collector/aggregator'
 require_relative 'collector/counter_cache'
 require_relative 'collector/exceptions'
-require_relative 'collector/group'
+require_relative 'collector/group'

data/lib/librato/rack/configuration.rb

@@ -1,5 +1,3 @@
-
-
 module Librato
   class Rack
     # Holds configuration for Librato::Rack middleware to use.
@@ -16,17 +14,17 @@ module Librato
 
       DEFAULT_SUITES = [:rack, :rack_method, :rack_status]
 
-      attr_accessor :user, :token, :autorun, :api_endpoint, :tracker,
-                    :source_pids, :log_level, :log_prefix, :log_target,
-                    :disable_rack_metrics, :flush_interval, :proxy, :suites
-      attr_reader :prefix, :source, :deprecations
+      attr_accessor :api_endpoint, :autorun, :disable_rack_metrics,
+                    :flush_interval, :log_level, :log_prefix,
+                    :log_target, :proxy, :suites,
+                    :tags, :token, :tracker, :user
+      attr_reader :deprecations, :prefix
 
       def initialize
         # set up defaults
         self.tracker = nil
         self.api_endpoint = Librato::Metrics.api_endpoint
         self.flush_interval = 60
-        self.source_pids = false
         self.log_prefix = '[librato-rack] '
         @listeners = []
         @deprecations = []
@@ -50,8 +48,8 @@ module Librato
         end
       end
 
-      def explicit_source?
-        !!@explicit_source
+      def has_tags?
+        @tags && !@tags.empty?
       end
 
       # check environment variables and capture current state
@@ -61,7 +59,7 @@ module Librato
         self.token = ENV['LIBRATO_TOKEN']
         self.autorun = detect_autorun
         self.prefix = ENV['LIBRATO_PREFIX']
-        self.source = ENV['LIBRATO_SOURCE']
+        self.tags = build_tags
         self.log_level = ENV['LIBRATO_LOG_LEVEL'] || :info
         self.proxy = ENV['LIBRATO_PROXY'] || ENV['https_proxy'] || ENV['http_proxy']
         self.event_mode = ENV['LIBRATO_EVENT_MODE']
@@ -78,14 +76,9 @@ module Librato
         @listeners << listener
       end
 
-      def source=(src)
-        @source = src
-        @explicit_source = !!@source
-      end
-
       def dump
         fields = {}
-        %w{user token log_level source prefix flush_interval source_pids suites}.each do |field|
+        %w{flush_interval log_level prefix suites tags token user}.each do |field|
           fields[field.to_sym] = self.send(field)
         end
         fields[:metric_suites] = metric_suites.fields
@@ -130,6 +123,21 @@ module Librato
         end
       end
 
+      def build_tags
+        tags = {}
+        tags.tap do
+          if ENV["LIBRATO_TAGS"]
+            ENV["LIBRATO_TAGS"].split(",")
+              .map { |pairs| pairs.split("=") }
+                .map { |k,v| tags[k.to_sym] = v }
+
+            if tags.all? {|k,v| k.nil? || v.nil? }
+              raise InvalidTagConfiguration, "Invalid tag configuration format. Example: foo=bar,baz=qux"
+            end
+          end
+        end
+      end
+
     end
   end
 end

data/lib/librato/rack/errors.rb

@@ -5,5 +5,7 @@ module Librato
 
     class InvalidSuiteConfiguration < RuntimeError; end
 
+    class InvalidTagConfiguration < RuntimeError; end
+
   end
 end

data/lib/librato/rack/tracker.rb

@@ -5,8 +5,6 @@ module Librato
     class Tracker
       extend Forwardable
 
-      SOURCE_REGEX = /\A[-:A-Za-z0-9_.]{1,255}\z/
-
       def_delegators :collector, :increment, :measure, :timing, :group
       def_delegators :logger, :log
 
@@ -39,7 +37,7 @@ module Librato
 
       # primary collector object used by this tracker
       def collector
-        @collector ||= Librato::Collector.new
+        @collector ||= Librato::Collector.new(tags: tags)
       end
 
       # log a deprecation message
@@ -59,11 +57,6 @@ module Librato
         log :error, "submission failed permanently: #{error}"
       end
 
-      # source including process pid if indicated
-      def qualified_source
-        config.source_pids ? "#{source}.#{$$}" : source
-      end
-
       # current local instrumentation to be sent on next flush
       # this is for debugging, don't call rapidly in production as it
       # may introduce latency
@@ -78,10 +71,12 @@ module Librato
           log :debug, 'halting: credentials not present.'
         elsif config.autorun == false
           log :debug, 'halting: LIBRATO_AUTORUN disabled startup'
-        elsif qualified_source !~ SOURCE_REGEX
-          log :warn, "halting: '#{qualified_source}' is an invalid source name."
-        elsif on_heroku && !config.explicit_source?
-          log :warn, 'halting: source must be provided in configuration.'
+        elsif tags.any? { |k,v| k.to_s !~ ValidatingQueue::TAGS_KEY_REGEX || v.to_s !~ ValidatingQueue::TAGS_VALUE_REGEX }
+          log :warn, "halting: '#{tags}' are invalid tags."
+        elsif tags.keys.length > ValidatingQueue::DEFAULT_TAGS_LIMIT
+          log :warn, "halting: cannot exceed default tags limit of #{ValidatingQueue::DEFAULT_TAGS_LIMIT} tag names per measurement."
+        elsif on_heroku && !config.has_tags?
+          log :warn, 'halting: tags must be provided in configuration.'
         else
           return true
         end
@@ -125,12 +120,12 @@ module Librato
       end
 
       def build_flush_queue(collector, preserve=false)
-        queue = ValidatingQueue.new( client: client, source: qualified_source,
+        queue = ValidatingQueue.new( client: client,
           prefix: config.prefix, skip_measurement_times: true )
         [collector.counters, collector.aggregate].each do |cache|
           cache.flush_to(queue, preserve: preserve)
         end
-        queue.add 'rack.processes' => 1
+        queue.add 'rack.processes' => { value: 1, tags: tags }
         trace_queued(queue.queued) #if should_log?(:trace)
         queue
       end
@@ -166,8 +161,8 @@ module Librato
         RUBY_DESCRIPTION.split[0]
       end
 
-      def source
-        @source ||= (config.source || Socket.gethostname).downcase
+      def tags
+        @tags ||= config.has_tags? ? config.tags : { host: Socket.gethostname.downcase }
       end
 
       # should we spin up a worker? wrap this in a process check

data/lib/librato/rack/validating_queue.rb

@@ -5,21 +5,23 @@ module Librato
     # to work out the kinks
     #
     class ValidatingQueue < Librato::Metrics::Queue
+      DEFAULT_TAGS_LIMIT = 4
       METRIC_NAME_REGEX = /\A[-.:_\w]{1,255}\z/
-      SOURCE_NAME_REGEX = /\A[-:A-Za-z0-9_.]{1,255}\z/
+      TAGS_KEY_REGEX = /\A[-.:_\w]{1,64}\z/
+      TAGS_VALUE_REGEX = /\A[-.:_\w]{1,256}\z/
 
       attr_accessor :logger
 
       # screen all measurements for validity before sending
       def submit
-        @queued[:gauges].delete_if do |entry|
+        @queued[:measurements].delete_if do |entry|
           name = entry[:name].to_s
-          source = entry[:source] && entry[:source].to_s
+          tags = entry[:tags]
           if name !~ METRIC_NAME_REGEX
             log :warn, "invalid metric name '#{name}', not sending."
             true # delete
-          elsif source && source !~ SOURCE_NAME_REGEX
-            log :warn, "invalid source name '#{source}', not sending."
+          elsif tags && tags.any? { |k,v| k.to_s !~ TAGS_KEY_REGEX || v.to_s !~ TAGS_VALUE_REGEX }
+            log :warn, "halting: '#{tags}' are invalid tags."
             true # delete
           else
             false # preserve
@@ -38,4 +40,4 @@ module Librato
 
     end
   end
-end
+end

data/lib/librato/rack/version.rb

@@ -1,5 +1,5 @@
 module Librato
   class Rack
-    VERSION = "1.1.1"
+    VERSION = "2.0.0"
   end
 end

data/lib/librato/rack.rb

@@ -44,21 +44,15 @@ module Librato
     EOS
 
     RECORD_RACK_STATUS_BODY = <<-'EOS'
-      group.group 'status' do |s|
-        s.increment status
-        s.increment "#{status.to_s[0]}xx"
-
-        s.timing "#{status}.time", duration
-        s.timing "#{status.to_s[0]}xx.time", duration
-      end
+      status_tags = { status: status }
+      tracker.increment "rack.request.status", tags: status_tags, inherit_tags: true
+      tracker.timing "rack.request.status.time", duration, tags: status_tags, inherit_tags: true
     EOS
 
     RECORD_RACK_METHOD_BODY = <<-'EOS'
-      group.group 'method' do |m|
-        method = http_method.downcase
-        m.increment method
-        m.timing "#{method}.time", duration
-      end
+      method_tags = { method: http_method.downcase! }
+      tracker.increment "rack.request.method", tags: method_tags, inherit_tags: true
+      tracker.timing "rack.request.method.time", duration, tags: method_tags, inherit_tags: true
     EOS
 
     attr_reader :config, :tracker

data/test/apps/custom.ru

@@ -5,6 +5,10 @@ use Librato::Rack
 
 def application(env)
   case env['PATH_INFO']
+  when '/tags'
+    tags = { region: "us-east-1" }
+    Librato.increment "requests", tags: tags
+    Librato.timing "requests.time", 3, tags: tags
   when '/increment'
     Librato.increment :hits
   when '/measure'
@@ -24,4 +28,4 @@ def application(env)
   [200, {"Content-Type" => 'text/html'}, ["Hello!"]]
 end
 
-run method(:application)
+run method(:application)