graphite-api 0.4.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76010192ae0104397b2f2f069f03ee6f5895f42473f8bab79068ce7808104a58
-  data.tar.gz: da60c3a2a69ed0d1e35232d4b1e4fb90709a7d16c87d241801824b67094cf4b0
+  metadata.gz: 0f81b82a2dff25c88b118c8734da6711683cd3dbe4befa2d5faea3172d3c752b
+  data.tar.gz: f0c6278c05daba7691b3a854953b2a6f5d8abb0338298fddc76afa2ae8892d24
 SHA512:
-  metadata.gz: 29d966b90c0d676c604e2556a05992667ec6a083e078ae2a2a4cb75a9524ea042919585038ff18dcf1b90b7ed049c414ebefdb5dbc0389787a2976960cf534b9
-  data.tar.gz: 4c1c95a572507ecd6df22581559f615fecdefc8cb8c098707bdaac0ed2bc39f51c6dcbeaa13b3dee441a3b36a0e86cbf30d0e916676214aa0aa40aa38480edc8
+  metadata.gz: ea462f77942592a9b31067f091b417b3e6bc13a945f68e11c5d283744e7d585a3ee4136400175c5cd9778ccb012ccfca3bd3a9ed569eba3c7e83d861736dee89
+  data.tar.gz: 951db12f2528aeff81202121809c09d96394e25c65a2ca14a79297c0f3789c3f8109f931c28575303ae05d875b0edc0b24d9d5d2851d87f1735dec7cf4322e01

data/README.md

@@ -1,14 +1,12 @@
 # Description
-**GraphiteAPI** provides two ways for interacting with **Graphite's Carbon Daemon**, the first is for Ruby applications using the **GraphiteAPI::Client**, the second is through **GraphiteAPI-Middleware** daemon, both methods implements Graphite's [plaintext protocol](http://graphite.readthedocs.org/en/1.0/feeding-carbon.html).
+**GraphiteAPI** provides a way for interacting with **Graphite's Carbon Daemon** using the **GraphiteAPI::Client**, that implements Graphite's [plaintext protocol](http://graphite.readthedocs.org/en/1.0/feeding-carbon.html).
 
 ## Package Content
 * Includes a **simple** client for ruby.
-* Ships with a **GraphiteAPI-Middleware**, which is a lightweight, event-driven, aggregator daemon.
 * Utilities like scheduling and caching.
 
 ## Key Features
-* **Multiple Graphite Servers Support** - GraphiteAPI-Middleware supports sending aggregated data to multiple graphite servers, in a multiplex fashion, useful for large data centers and backup purposes
-* **Reanimation mode** - support cases which the same keys (same timestamps as well) can be received simultaneously and asynchronously from multiple input sources, in these cases GraphiteAPI-Middleware will "reanimate" old records (records that were already sent to Graphite server), and will send the sum of the reanimated record value + the value of the record that was just received to the graphite server; this new summed record should override the key with the new value on Graphite database.
+* **Reanimation mode** - support cases which the same keys (same timestamps as well) can be received simultaneously and asynchronously from multiple input sources, in these cases GraphiteAPI will "reanimate" old records (records that were already sent to Graphite server), and will send the sum of the reanimated record value + the value of the record that was just received to the graphite server; this new summed record should override the key with the new value on Graphite database.
 * **non-blocking I/O** ( EventMachine aware ).
 * **Thread-Safe** client.
 
@@ -47,7 +45,7 @@ options = {
   # Optional: set the max age in seconds for records reanimation ( default is 12 hours )
   cache: 4 * 60 * 60,
 
-  # Optional: The default aggregation method for multiple reports in the same slice (default is :add).
+  # Optional: The default aggregation method for multiple reports in the same slice (default is :sum).
   # Possible options: :sum, :avg, :replace
   default_aggregation_method: :avg
 }
@@ -126,19 +124,7 @@ require 'graphite-api'
 client = GraphiteAPI.new graphite: 'udp://UNKNOWN-HOST:1234'
 client.check!
 
-SocketError: udp://UNKNOWN-HOST:1234: getaddrinfo: nodename nor servname provided, or not known
-	from graphite-api/lib/graphite-api/connector.rb:71:in `connect'
-	from graphite-api/lib/graphite-api/connector.rb:71:in `block in init_udp'
-	from graphite-api/lib/graphite-api/connector.rb:71:in `tap'
-	from graphite-api/lib/graphite-api/connector.rb:71:in `init_udp'
-	from graphite-api/lib/graphite-api/connector.rb:37:in `socket'
-	from graphite-api/lib/graphite-api/connector.rb:30:in `check!'
-	from graphite-api/lib/graphite-api/connector.rb:88:in `block in check!'
-	from graphite-api/lib/graphite-api/connector.rb:86:in `each'
-	from github/graphite-api/lib/graphite-api/connector.rb:86:in `check!'
-	from graphite-api/lib/graphite-api/client.rb:22:in `check!'
-	from (irb):4
-	from ~/.rvm/rubies/ruby-2.3.3/bin/irb:11:in `<main>'
+# SocketError: udp://UNKNOWN-HOST:1234: getaddrinfo: nodename nor servname provided, or not known
 ```
 
 Increment records
@@ -218,69 +204,6 @@ GraphiteAPI::Logger.logger.level = ::Logger::DEBUG
 GraphiteAPI::Logger.init level: :debug, dev: 'logger.out' # or STDOUT | STDERR
 ```
 
-## GraphiteAPI-Middleware Usage
-* After installing GraphiteAPI gem, the `graphite-middleware` command should be available.
-
-```
-[root@graphite-middleware-node]# graphite-middleware --help
-
-GraphiteAPI Middleware Server
-
-Usage: graphite-middleware [options]
-    -g, --graphite HOST:PORT         graphite host, in HOST:PORT format (can be specified multiple times)
-    -p, --port PORT                  listening port (default 2003)
-    -l, --log-file FILE              log file
-    -L, --log-level LEVEL            log level (default warn)
-    -P, --pid-file FILE              pid file (default /var/run/graphite-middleware.pid)
-    -d, --daemonize                  run in background
-    -i, --interval INT               report every X seconds (default 60)
-    -s, --slice SECONDS              send to graphite in X seconds slices (default 60)
-    -r, --reanimation HOURS          reanimate records that are younger than X hours, please see README
-    -m, --aggregation-method method  The aggregation method (sum, avg or replace) for multiple reports in the same time slice (default sum)
-
-More Info @ https://github.com/kontera-technologies/graphite-api
-```
-
-* launch **GraphiteAPI-Middleware** daemon
-
-```
-[root@graphite-middleware-node]# graphite-middleware              \
-  --port 2005                                                     \
-  --interval 60                                                   \
-  --log-level debug                                               \
-  --log-file /tmp/graphite-middleware.out                         \
-  --daemonize                                                     \
-  --graphite graphite-server:2003                                 \
-  --graphite graphite-backup-server:2003
-```
-
-* Send metrics via **UDP/TCP sockets**
-
-```
-[root@graphite-middleware-node] telnet localhost 2005
-Trying 127.0.0.1...
-Connected to localhost.
-Escape character is '^]'.
-example.middleware.value 10.2 1335008343
-example.middleware.value2 99 1334929231
-^C
-[root@graphite-middleware-node]
-```
-
-* Or via **GraphtieAPI client**
-
-```ruby
-require 'graphite-api'
-client = GraphiteAPI.new(:graphite => 'tcp://graphite-middleware-node:2005')
-client.example.middleware.value 10.2
-client.example.middleware.value2 27
-client.bla.bla.value2 27
-```
-
-## Example Setup
-<br/>
-<img src="https://raw.github.com/kontera-technologies/graphite-api/master/examples/middleware_t1.png" align="center">
-
 ## Bugs
 
 If you find a bug, feel free to report it @ our [issues tracker](https://github.com/kontera-technologies/graphite-api/issues) on github.

data/Rakefile

@@ -2,54 +2,36 @@ $:.unshift File.join(File.dirname(__FILE__), 'lib')
 Dir.chdir File.dirname __FILE__
 
 require 'graphite-api'
+require "bundler/gem_tasks"
 require "rake/testtask"
 require 'rubygems/package_task'
 
-def message msg
-  puts "*** #{msg} ***"
+Rake::TestTask.new(:prepare) do |t|
+  t.libs << "tests"
 end
 
-task(:test => :functional)
+Rake::TestTask.new(:unit => :prepare) do |t|
+  t.pattern = "tests/unit/*_test.rb"
+end
 
-Rake::TestTask.new do |t|
-  t.libs << "tests"
+Rake::TestTask.new(:functional => :prepare) do |t|
+  t.pattern = "tests/functional/*_test.rb"
+end
+
+Rake::TestTask.new(:all => :prepare) do |t|
   t.pattern = "tests/**/*_test.rb"
 end
 
-task :functional do
-  some_failed = false
-
-  next unless ENV['SKIP_FUNC'].nil?
-
-  unless RUBY_COPYRIGHT.end_with?("Matsumoto")
-    puts("Functional tests are enabled only on MRI...")
-    next
-  end
-
-  message "Executing GraphiteAPI Functional Tests"
-  message "( You can skip them by passing SKIP_FUNC=true )"
-
-  Dir[File.expand_path("../tests/functional/*",__FILE__)].each do |file|
-    next unless file.end_with?(".rb")
-    now = Time.now.to_i
-    name = File.basename(file)
-    message "Executing #{name}"
-    Process.waitpid(Process.spawn("ruby", File.expand_path(file)))
-    took = "took #{Time.now.to_i - now} seconds"
-    if $?.success?
-      message "[PASS] #{name}, #{took}"
-    else
-      message "[FAIL] #{name}, #{took}"
-      some_failed = true
-    end
-  end
-  message "Done Executing GraphiteAPI Functional Tests"
-  abort "Some functional tests failed..." if some_failed
+# Disable functional tests on JRuby
+if RUBY_PLATFORM == "java"
+  task(:test => :unit)
+else
+  task(:test => :all)
 end
 
 task :default => :test
 
-task :gem => [:test,:clobber_package]
+task :gem => [:test, :clobber_package]
 
 GraphiteAPI::GemSpec = eval File.read 'graphite-api.gemspec'
 

data/lib/graphite-api.rb

@@ -3,7 +3,6 @@ require 'graphite-api/version'
 require 'graphite-api/client'
 require 'graphite-api/cache'
 require 'graphite-api/connector'
-require 'graphite-api/middleware'
 require 'graphite-api/buffer'
 require 'graphite-api/logger'
 
@@ -12,9 +11,9 @@ module GraphiteAPI
   def self.version
     GraphiteAPI::VERSION
   end
-  
+
   def self.new options
     Client.new options
   end
-  
+
 end

data/lib/graphite-api/client.rb

@@ -5,8 +5,7 @@ module GraphiteAPI
   class Client
     extend Forwardable
 
-    def_delegator Zscheduler, :loop, :join
-    def_delegator Zscheduler, :stop
+    def_delegator Zscheduler, :loop, :stop
 
     attr_reader :options, :buffer, :connectors, :mu
     private     :options, :buffer, :connectors, :mu

data/lib/graphite-api/connector.rb

@@ -8,6 +8,7 @@ module GraphiteAPI
     def initialize uri
       @uri = URI.parse uri
       @uri = @uri.host ? @uri : URI.parse("udp://#{uri}")
+      @socket = nil
     end
 
     def puts message

data/lib/graphite-api/version.rb

@@ -1,3 +1,3 @@
 module GraphiteAPI
-  VERSION = "0.4.0"
+  VERSION = "1.0.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: graphite-api
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 1.0.0
 platform: ruby
 authors:
 - Eran Barak Levi
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-02-20 00:00:00.000000000 Z
+date: 2019-03-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: eventmachine
@@ -40,15 +40,13 @@ dependencies:
         version: 0.0.7
 description: Graphite API - A Simple ruby client, aggregator daemon and API tools
 email: eran@kontera.com
-executables:
-- graphite-middleware
+executables: []
 extensions: []
 extra_rdoc_files: []
 files:
 - LICENSE
 - README.md
 - Rakefile
-- bin/graphite-middleware
 - lib/graphite-api.rb
 - lib/graphite-api/buffer.rb
 - lib/graphite-api/cache.rb
@@ -56,7 +54,6 @@ files:
 - lib/graphite-api/connector.rb
 - lib/graphite-api/core_ext/numeric.rb
 - lib/graphite-api/logger.rb
-- lib/graphite-api/middleware.rb
 - lib/graphite-api/version.rb
 homepage: http://www.kontera.com
 licenses:
@@ -70,15 +67,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 1.9.3
+      version: '2.3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: graphite-api
-rubygems_version: 2.7.8
+rubygems_version: 3.0.3
 signing_key: 
 specification_version: 4
 summary: Graphite Ruby Client

data/bin/graphite-middleware

@@ -1,78 +0,0 @@
-#!/usr/bin/env ruby
-$:.unshift File.expand_path('../../lib',__FILE__)
-
-require 'graphite-api'
-require 'optparse'
-
-options = GraphiteAPI::Client.default_options.merge interval: 60
-
-OptionParser.new do |opts|
-  opts.banner = "GraphiteAPI Middleware Server"
-  opts.define_head "Usage: graphite-middleware [options]"
-  opts.define_head ""
-
-  opts.on("-g", "--graphite [udp,tcp]://host:port", "can be specified multiple times") do |graphite|
-    options[:backends].push graphite
-  end
-
-  opts.on("-p", "--port PORT","listening port (default #{options[:port]})") do |port|
-    options[:port] = port
-  end
-
-  opts.on("-l", "--log-file FILE","log file") do |file|
-    options[:log_file] = File::expand_path(file)
-  end
-
-  opts.on("-L", "--log-level LEVEL","log level (default warn)") do |level|
-    options[:log_level] = level
-  end
-
-  opts.on("-P", "--pid-file FILE","pid file (default #{options[:pid]})") do |pid_file|
-    options[:pid] = pid_file
-  end
-
-  opts.on("-d", "--daemonize","run in background") do
-    options[:daemonize] = true
-  end
-
-  opts.on("-i", "--interval INT","report every X seconds (default #{options[:interval]})") do |x_seconds|
-    options[:interval] = x_seconds.to_i if x_seconds.to_i > 0
-  end
-
-  opts.on("-s", "--slice SECONDS","send to graphite in X seconds slices (default #{options[:slice]})") do |slice|
-    options[:slice] = slice.to_i if slice.to_i >= 0
-  end
-
-  opts.on("-r", "--reanimation HOURS","reanimate records that are younger than X hours, please see README") do |exp|
-    (options[:cache] = exp.to_i * 3600) if exp.to_i > 0
-  end
-
-  opts.on("-m", "--aggregation-method method","The aggregation method (sum, avg or replace) for multiple reports in the same time slice (default sum)") do |exp|
-    raise "Invalid aggregation method. Valid values are sum, avg or replace." unless ["sum", "avg", "replace"].include? exp
-    options[:default_aggregation_method] = exp.to_sym
-  end
-
-  opts.on("-v", "--version","Show version and exit") do |exp|
-    puts "Version #{GraphiteAPI.version}"
-    exit
-  end
-
-  opts.separator ""
-  opts.separator "More Info @ https://github.com/kontera-technologies/graphite-api"
-  opts.separator ""
-
-  opts.define_tail ""
-  opts.define_tail ""
-end.parse! ARGV
-
-abort "You must specify at least one graphite host" if options[:backends].empty?
-
-GraphiteAPI::Logger.init Hash[[:dev,:level].zip options.values_at(:log_file, :log_level) ]
-
-begin
-  Process.daemon if options[:daemonize]
-  GraphiteAPI::Middleware.start options
-rescue Interrupt
-  GraphiteAPI::Logger.info "Shutting down..."
-  Zscheduler.stop
-end

data/lib/graphite-api/middleware.rb

@@ -1,73 +0,0 @@
-# -----------------------------------------------------
-# Graphite Middleware Server
-# Should be placed between graphite server and graphite clients
-# - Data Aggregator
-# - Caching
-# - Data Manipulation
-# -----------------------------------------------------
-# Usage:
-#   GraphiteAPI::Middleware.start(options)
-# 
-# Options:
-#   graphite         target graphite hostname
-#   reanimation_exp  cache lifetime in seconds  (default is 43200 seconds)
-#   prefix           add prefix to each key
-#   interval         report to graphite every X seconds (default is 60)
-#   slice            send to graphite in X seconds slices (default is 60)
-#   log_level        info 
-# -----------------------------------------------------
-require 'eventmachine'
-require 'socket'
-
-module GraphiteAPI
-  class Middleware < EventMachine::Connection
-
-    def initialize buffer
-      @buffer = buffer and super
-    end
-    
-    attr_reader :buffer, :client_id
-    
-    def post_init
-      @client_id = peername
-      Logger.debug [:middleware, :connecting, client_id]
-    end
-
-    def receive_data data
-      Logger.debug [:middleware, :message, client_id, data]
-      buffer.stream data, client_id
-    end
-
-    def unbind
-      Logger.debug [:middleware, :disconnecting, client_id]
-    end
-    
-    def peername
-      port, *ip = get_peername[2,6].unpack "nC4"
-      [ip.join("."),port].join ":"
-    end
-    
-    private :peername
-    
-    def self.start options
-      EventMachine.run do
-        GraphiteAPI::Logger.info "Server running on port #{options[:port]}"
-        
-        buffer = GraphiteAPI::Buffer.new options
-        group  = GraphiteAPI::Connector::Group.new options
-        
-        # Starting server
-        [:start_server, :open_datagram_socket].each do |m|
-          EventMachine.send(m,'0.0.0.0',options[:port],self,buffer)
-        end
-
-        # Send metrics to graphite every X seconds
-        Zscheduler.every(options[:interval], :on_shutdown => true) do
-          group.publish buffer.pull :string if buffer.new_records?
-        end
-
-      end  
-    end
-    
-  end 
-end