tsdb_time_series 4.1.2

checksums.yaml

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    OTY2MTQyZTM3YWMwYzc3YWZlOWQyNDk1ZTBhNWM5MGNjNzYzNmE2Zg==
+  data.tar.gz: !binary |-
+    YmVkMjlhN2U1ODM5ODBmZGIzYzRkNDY5ZGViYjk1MWI5NGU5YWE2OQ==
+SHA512:
+  metadata.gz: !binary |-
+    MzE5ZDk3OWNmMzMxZWYwOGFkNTM1YzA5OTdmNTg3NDI4MTNiNWE4NmYyNjIx
+    ZDZhNTc5ZDZhYjZiNTY4YzE4N2Q1NWI2MDg2NjE1ZmFhZjEwMmVmY2QyNjhm
+    ZmEwZmExNzU4MzgwMmU0N2E1ZjI4NjVmZWRlMDAwOWMyMDRlZTg=
+  data.tar.gz: !binary |-
+    ODNmOTdlODViODczNGQ1YzY1ZWEwMGM1NzQyNDExZjM1ODBhMGQ5OWNmNTQ2
+    NDRjNWVhYmUzZDUwZTAxMTg4NjhlMGQzZjY3MDQyNzAzN2YzZWY4Mzk5NTlj
+    NGQxZjBhMzQ0MjMzOWU5MzMxNzYxOGViMWRjOWNhMDUyMTdkYWM=

data/.gitignore

@@ -0,0 +1,22 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*.swp
+*.swo
+bin/
+vendor/
+.DS_Store

data/.rubocop.yml

@@ -0,0 +1,17 @@
+LineLength:
+  Enabled: true
+  Max: 128
+
+Encoding:
+  Enabled: false
+
+CaseIndentation:
+  Enabled: false
+
+# Annoying warning when dealing with Unix timestamps
+NumericLiterals:
+  Enabled: false
+
+# MethodLength is superseded by Reek's TooManyStatements smell-finder.
+MethodLength:
+  Enabled: false

data/.rvmrc

@@ -0,0 +1 @@
+rvm use ruby-1.9.3-p545

data/CHANGELOG.md

@@ -0,0 +1,38 @@
+# v4.1.2
+- Renamed gem to `tsdb_time_series` for public use. (`time_series` is already taken)
+- Cleaned up README.md a little bit
+
+# v4.1.1
+- Bumped Rubocop to `~> 0.28.0` and updated dependencies
+- Fixed new Rubocop warnings
+
+# v4.1.0
+- Bumped RSpec to `~> 3.0`
+- Updated tests to follow betterspecs.org guidelines
+- Added integration tests using Docker
+
+# v4.0.0
+- Added initial support for synthetic metrics through formula calculations.
+- Dropped OpenTSDB 1.1 support (includes dropping ASCII output support!)
+- Dropped client-side validation support (done by OpenTSDB now)
+
+# v3.0.0
+- Updated response signatures to include status code, result count, and explicit error messages.
+
+# v2.4.0
+- Added multi-query support (persistent, pipelined HTTP requests)
+
+# v2.3.0
+- Deprecrated client-side validation for OpenTSDB 2.0 clients
+
+# v2.2.0
+- Switched to Excon from HTTParty
+
+# v2.1.0
+- Upgraded to Ruby 1.9.3
+
+# v2.0.1
+- Added proper rate support
+
+# v2.0.0
+- Initial OpenTSDB 2.0 support

data/CONTRIBUTING.md

@@ -0,0 +1,22 @@
+## Contributing to TimeSeries
+
+The best way to contribute code is to fork the main repo and send a pull request on GitHub.  
+
+Bug fixes should be done in the master branch. New features or major changes should be done in a feature branch.  Alternatively, you can send a plain-text patch to the [mailing list](https://groups.google.com/forum/#!forum/opentsdb-clients).
+
+Please break down your changes into as many small commits as possible. 
+
+Please respect the coding style of the code you're changing. TimeSeries uses Rubocop and reek to adhere to the Ruby style guide.
+
+## Pull Request Requirements
+
+*Before* sending a pull request, please make sure your changes meet the following criteria:
+
+- You have run `bundle exec rake build` and your code:
+    - Maintains code coverage remains at 100%
+    - Has fully been fully documented (`yard` coverage at 100%)
+    - Has no `reek` warnings
+    - Has no `rubocop` warnings
+    - All RSpec tests pass
+
+

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in time_series.gemspec
+gemspec

data/Guardfile

@@ -0,0 +1,27 @@
+# vim: ft=ruby
+# More info at https://github.com/guard/guard#readme
+#
+# More info also at https://github.com/guard/guard-rspec -- this one in
+# particular details configuration options such as whether to run all tests
+# after a failing test starts passing
+
+guard :rspec, cli: '--tag ~slow' do
+  watch(/^spec\/.+_spec\.rb/)
+  watch(/^lib\/(.+)\.rb$/) do |match|
+    %w(unit integration acceptance).map do |kind|
+      "spec/#{kind}/lib/#{match[1]}_spec.rb"
+    end
+  end
+  watch('spec/spec_helper.rb')  { 'spec' }
+  watch(%r{^spec/(fixtures|resources)(/|.rb)}) { 'spec' }
+end
+
+guard :rubocop, all_on_start: false do
+  watch('Guardfile')
+  watch(/.+\.rb$/)
+  watch(/(?:.+\/)?\.rubocop\.yml$/) { |m| File.dirname(m[0]) }
+end
+
+guard :reek do
+  watch(/^lib\/(.+)\.rb$/)
+end

data/README.md

@@ -0,0 +1,232 @@
+## tsdb_time_series
+
+`tsdb_time_series` is a Ruby Gem for OpenTSDB that provides core tools when working with an OpenTSDB data store. With `tsdb_time_series`, you can search for registered metrics, tag keys and tag values, read from and write to the OpenTSDB, and submit multiple simultaneous queries to an OpenTSDB cluster.
+
+### Installation
+
+Download the Gem from a standard Gem server like rubygems.org and install it:
+
+    gem install tsdb_time_series
+
+Alternatively, build it from source and install it:
+
+    git clone https://github.com/opower/time_series.git
+    cd time_series
+    gem build tsdb_time_series.gemspec
+    gem install tsdb_time_series-4.1.3.gem
+
+### Usage
+Once you have the OpenTSDB cluster set up, we can configure the TimeSeries Gem to talk to the API. The first step would be configuring a TimeSeries client. If no host is specified, the client connects to localhost by default. The client connects to port 4242 by default.
+
+#### Configuring a TimeSeries Client
+
+```ruby
+client = Opower::TimeSeries::TSClient.new('opentsdb.foo.com', 4242)
+client.configure({ version: '2.0', dry_run: false, validation: true })
+```
+
+Here is a table that lists options supported by the TimeSeries client:
+
+| Option | Type | Description| Default value |
+| ------------- | ------------- | ------------- | ------------- |
+| :version | Float | version of the OpentSDB cluster the client will be talking to.  If you wish to use the new 2.0 endpoints, set version to 2.0 or higher. | 2.0 |
+| :dry_run | Boolean | If set to true, this gem will not run any commands, only output the generated URLs or calls to OpenTSDB. | false |
+| :validation | Boolean | With this flag set to true, client performs a check to validate the metric name.  | false | 
+
+#### Search for a registered metric/tagk/tagv
+
+Using a properly configured client, you can search an OpenTSDB cluster to find suggestions for a metric, tag key, or tag value. This employs the `/api/suggest` end point of the OpenTSDB API and works as a simple namespace search. It is useful when you do not know what metric labels are being written to the OpenTSDB.
+
+```ruby
+client.suggest('proc.stat.cpu') # suggest a metric
+client.suggest('proc.stat.cpu', 'tagk') # suggest a tagk
+client.suggest('proc.stat.cpu', 'tagv') # suggest a tagv
+```
+
+#### Writing to OpenTSDB
+
+You can use a TimeSeries client to push telnet/netcat style writes to OpenTSDB. If no hostname and port are specified, this gem defaults to 127.0.0.1:4242. To insert a metric into OpenTSDB using a configured client, create a new `Metric` object first and then use the `client.write` call as shown below :
+
+```ruby
+metric_config = {
+        name: 'proc.stat.cpu',
+        timestamp: Time.now.to_i,
+        value: 10,
+        tags: { host: 'somehost.foo.com', type: 'iowait' }
+}
+
+metric = Opower::TimeSeries::Metric.new(metric_config)
+client.write(metric)
+```
+
+
+#### Reading from OpenTSDB
+
+We can use a TimeSeries client to read metric data from an OpenTSDB cluster. To read data from OpenTSDB, you would first create a query object to run against the specified client. This query object supports all the standard options in the 2.0 API. Here is an example :
+
+```ruby
+query_config = {
+        format: :png,
+        start: '2013/01/01-01:00:00',
+        end: '2013/02/01-01:00:00',
+        m: [{ aggregator: 'sum', metric: 'proc.stat.cpu', tags: { type: 'iowait' } }],
+        nocache: true
+}
+
+query = Opower::TimeSeries::Query.new(query_config)
+client.run_query(query)
+```
+
+The `Query` object accepts the following parameters:
+
+
+| Option | Type | Description| Default value |
+| ------------- | ------------- | ------------- | ------------- |
+| :format | String | Specifies the output format. supported values include : `ascii`, `json`, `png`. | 'json' |
+| :start | `String` / `Integer` / `DateTime` | The query's start date/time expressed as '2013/01/01-01:00:00' (string), '5m-ago' (String, indicating data for last 5 minutes), 1232323232 (time since epoch, Integer) or as a Ruby DateTime object. This is a required field. | none | 
+| :end | `String` / `Integer` / `DateTime` | The query's end date. This field supports the same types as `:start` field. This field is optional | Time.now (current time) |
+| :m | `Array` | Array of JSON objects with the `aggregator`, `metrics`, and `tags` as fields: | none |
+
+Here is a sample metrics object , that goes into the :m object .
+```ruby
+m: [{ aggregator: 'sum', metric: 'proc.stat.cpu', tags: { type: 'iowait', version: 2.1 } }]
+```
+
+Other options available to the REST API can be used here as well. Here is a list of options that have been tested to work with this gem. See the [OpenTSDB documentation](http://opentsdb.net/http-api.html#/q_Parameters) for more information :
+
+```
+
+ # * o       Rendering options.
+ # * wxh     The dimensions of the graph.
+ # * yrange  The range of the left Y axis.
+ # * y2range The range of the right Y axis.
+ # * ylabel  Label for the left Y axis.
+ # * y2label Label for the right Y axis.
+ # * yformat Format string for the left Y axis.
+ # * y2formatFormat string for the right Y axis.
+ # * ylog    Enables log scale for the left Y axis.
+ # * y2log   Enables log scale for the right Y axis.
+ # * key     Options for the key (legend) of the graph.
+ # * nokey   Removes the key (legend) from the graph.
+ # * nocache Forces TSD to ignore cache and fetch results from HBase.
+
+```
+
+
+
+#### Example Queries
+
+```ruby
+query_config = {
+        format: :ascii,
+        start: 14535353,
+        end: 16786786,
+        m: [{ aggregator: 'sum', metric: 'proc.stat.cpu', rate: true, tags: { type: 'iowait' } }]
+}
+
+query = Opower::TimeSeries::Query.new(query_config)
+client.run_query(query)
+```
+
+```ruby
+query_config = {
+        format: :json,
+        start: '3m-ago',
+        m: [{ aggregator: 'max', metric: 'proc.stat.cpu', tags: { type: 'iowait' } }],
+        nocache: true
+}
+
+query = Opower::TimeSeries::Query.new(query_config)
+client.run_query(query)
+```
+
+#### Running Multiple Queries Simultaneously
+
+If you need to query multiple metrics at the same time, TimeSeries provides support for that as well:
+
+```ruby
+queries = []
+3.times do
+    query_config = {
+            format: :ascii,
+            start: 14535353,
+            end: 16786786,
+            m: [{ aggregator: 'sum', metric: 'proc.stat.cpu', rate: true, tags: { type: 'iowait' } }]
+    }
+
+    queries << Opower::TimeSeries::Query.new(query_config)
+end
+
+client.run_queries(queries)
+```
+
+#### Running Synthetic Metric Queries
+
+Sometimes, you might need to create a new time series using metrics data from existing time series. We call these 'Synthetic Metric Queries'. Some examples could be disk utilization (expressed as disk used/total disk available) or CPU itilization (expressed as cpu cycles used/total cpu cycles).
+
+TimeSeries also provides the capability to create synthetic metric queries through the use of a formula and any number of queries against OpenTSDB. Here is an example that creates a formula which adds two time series ( `x + y` ) and feeds the calculation with data from OpenTSDB :
+
+```ruby
+metric_x = [{ metric: 'sys.numa.allocation', tags: { host: 'somehost.foo.com' } }]
+query_config_x = { format: :json, start: '1h-ago', m: metric_x }
+@query_metric_x = Opower::TimeSeries::Query.new(query_config_x)
+
+metric_y = [{ metric: 'sys.numa.foreign_allocs', tags: { host: 'somehost.foo.com' } }]
+query_config_y = { format: :json, start: '1h-ago', m: metric_y }
+@query_metric_y = Opower::TimeSeries::Query.new(query_config_y)
+
+name = 'My Synthetic Metric Alias'
+formula = 'x + y'
+query_hash = { x: @query_metric_x, y: @query_metric_y }
+client.run_synthetic_query(name, formula, query_hash)
+```
+
+The above example illustrates how you pass in a hash object to the client in order to run a sythentic query. This also indicates how the key maps to the parameters in the formula, with their corresponding values consisting of a Query object. When the calculation is performed, it will only operate on matching timestamps. If there are no matching data-points, it will return nothing.
+
+For more information about what can be done with the formula parameters, read the documentation for the [Dentaku Calculator](https://github.com/rubysolo/dentaku). This gem expects any parameter in the formula to have a matching query in the query hash.
+
+##### Built-in Ruby Math support
+
+```ruby
+name = 'My Synthetic Metric Alias'
+formula = 'cos(x) + y'
+query_hash = { x: @query_metric_x, y: @query_metric_y }
+client.run_synthetic_query(name, formula, query_hash)
+```
+
+Formulas in time-series can use all of the basic methods provided by the Math module from Ruby.
+
+You must wrap nested mathematical expressions in formulas or Dentaku will attempt to pass them as separate arguments into the lambda below!
+
+For example:
+Assume x = 1, y = 2
+ - cos(x + y) is translated into cos(1, 'add', 2) - this calls Math.cos(1, 'add', 2) - this obviously throws an error
+ - cos((x + y)) is translated into cos(3) - this correctly calls Math.cos(3)
+
+This is due to the way Dentaku handles the order of precedence; unless you wrap nested arguments, it will pass them separately.
+
+#### Testing time_series
+
+Test cases should be added for any new code added to this project.
+
+Run acceptance/unit tests locally:
+
+```
+rake spec
+```
+
+Running integration tests:
+```
+docker pull opower/opentsdb
+rake integration
+```
+
+Integration tests requires you have a `Docker` installed and have ran `docker pull opower/opentsdb` before-hand.
+
+#### Generating Documentation
+
+To generate the documentation for this gem, run the following:
+
+```
+yard doc
+```

data/Rakefile

@@ -0,0 +1,44 @@
+# rubocop: disable LeadingCommentSpace
+#! /usr/bin/env rake
+# rubocop: enable LeadingCommentSpace
+require 'bundler/gem_tasks'
+require 'yard'
+require 'rspec/core/rake_task'
+require 'reek/rake/task'
+require 'rubocop/rake_task'
+
+task default: :build
+
+# If there are test failures, you'll need to write code to address them.
+# So no point in continuing to run the style tests.
+desc 'Runs unit/acceptance tests'
+RSpec::Core::RakeTask.new(:spec) do |task|
+  task.pattern = 'spec/acceptance/**/*.rb,spec/unit/**/*.rb'
+end
+
+desc 'Runs integration tests'
+RSpec::Core::RakeTask.new(:integration) do |task|
+  task.pattern = 'spec/integration/**/*.rb'
+end
+
+desc 'Runs yard'
+YARD::Rake::YardocTask.new(:yard)
+
+desc 'smells the lib directory, which Reek defaults to anyway'
+Reek::Rake::Task.new(:reek) do |task|
+  task.verbose = true
+end
+
+desc 'smells the spec directory, which is less important than lib'
+Reek::Rake::Task.new(:reek_tests) do |task|
+  task.source_files = 'spec/**/*.rb'
+  task.verbose = true
+end
+
+desc 'runs Rubocop'
+RuboCop::RakeTask.new
+
+desc 'Runs test and code cleanliness suite: Rubocop, Reek, rspec, and yard'
+task run_guards: [:spec, :yard, :reek, :reek_tests, :rubocop]
+
+task build: :run_guards

data/lib/time_series.rb

@@ -0,0 +1,7 @@
+# -*- encoding: utf-8 -*-
+require 'time_series/metric'
+require 'time_series/query'
+require 'time_series/results/result'
+require 'time_series/results/synthetic_result'
+require 'time_series/ts_client'
+require 'time_series/version'

data/lib/time_series/metric.rb

@@ -0,0 +1,56 @@
+# -*- encoding: utf-8 -*-
+
+module Opower
+  module TimeSeries
+    # Represents a metric that can be inserted into OpenTSDB instance through a [TSDBClient] object.
+    class Metric
+      attr_reader :name, :value, :timestamp, :tags
+
+      # Initializer for the Metric class.
+      #
+      # @param [Hash] config configuration hash consisting of the following values:
+      # @option config [String] :name The metric name (required)
+      # @option config [String] :value The metric value (required)
+      # @option config [String, Integer, Timestamp] :timestamp The timestamp in either epoch or a TimeStamp object.
+      # @option config [Array] :tags Array of tags to set for this metric. (tag_key => value)
+      #
+      # @return [Metric] a new Metric object
+      def initialize(config = {})
+        validate(config, [:name, :value])
+
+        @name = config[:name]
+        @value = config[:value]
+        @timestamp = config[:timestamp] ||  Time.now.to_i
+        @tags = config[:tags] || {}
+      end
+
+      # Converts the metric into the format required for use by `put` to insert into OpenTSDB.
+      #
+      # @return [String] put string
+      def to_s
+        result = ''
+        # Format the string for OpenTSDB
+        @tags.each { |key, value| result += "#{key}=#{value} " }
+        [@name, @timestamp, @value, result.rstrip].join(' ')
+      end
+
+      private
+
+      # Validates the metric inputs
+      #
+      # @param [Hash] config The configuration to validate.
+      # @param [Array] required_fields The required fields to be set inside the configuration.
+      def validate(config = {}, required_fields)
+        # Required fields check
+        required_fields.each do |field|
+          next if config.include?(field)
+          fail(ArgumentError, "#{field} is required to write into OpenTSDB.")
+        end
+
+        # Reject if user provided timestamp as not numeric
+        timestamp = config[:timestamp]
+        fail(ArgumentError, 'Timestamp must be numeric') if timestamp && !(timestamp.is_a? Fixnum)
+      end
+    end
+  end
+end