growthforecast 0.0.1 → 0.0.2

data/README.md

@@ -1,11 +1,16 @@
 # GrowthForecast
 
-Client library to operate GrowthForecast.
+Client library and command to operate GrowthForecast.
 
 * http://kazeburo.github.com/GrowthForecast/ (Japanese)
 * https://github.com/kazeburo/growthforecast
 
-Update graph value, or create/edit/delete basic graphs and complex graphs.
+Features:
+
+* Update graph value, or create/edit/delete basic graphs and complex graphs from your own code
+* Check, or edit/create basic/complex graphs with YAML specs, keywords and `gfclient` command
+
+**USE GrowthForecast v0.33 or later**
 
 ## Installation
 
@@ -23,6 +28,8 @@ Or install it yourself as:
 
 ## Usage
 
+### Client Library
+
 Update graph's value:
 
 ```ruby
@@ -179,11 +186,154 @@ complex_spec.description = '....'
 gf.add(complex_spec) # add() ignores 'id' attribute
 ```
 
+### Basic Authentication
+
+Set `username` and `password`:
+
+```ruby
+gf = GrowthForecast.new(hostname, portnum)
+gf.username = 'whoami'
+gf.password = 'secret'
+
+# ...
+```
+
+### gfclient
+
+```
+usage: gfclient SPEC_PATH TARGET_NAME [KEYWORD1 KEYWORD2 ...]
+  -f, --force     create/edit graphs with spec (default: check only)
+
+  -H, --host=HOST  set growthforecast hostname (overwrite configuration in spec)
+  -P, --port=PORT  set growthforecast port number
+
+  -u, --user=USER  set username for growthforecast basic authentication
+  -p, --pass=PASS  set password
+
+      --prefix=PREFIX  set growthforecast uri prefix
+
+  -l, --list=PATH  set keywords list file path (keywords set per line)
+                     do check/edit many times with this option
+                     (ignore default keywords)
+  -a, --get-all    get and cache all graph data at first (default: get incrementally)
+  -g, --debug      enable debug mode
+
+  -h, --help      show this message
+```
+
+`gfclient` checks all patterns of `TARGET_NAME` in spec yaml file's `specs -> check` section and `specs -> edit` section. Spec file examples are `example/testspec.yaml` and `example/spec.yaml`.
+
+Minimal example is:
+
+```yaml
+config:
+  host: your.gf.host.local
+  port: 5125
+specs:
+  - name: 'example1'
+    keywords:
+      - 'target'
+      - 'groupname'
+    check:
+      - name: 'metrics1'
+        path: 'pageviews/${target}/total'
+        color: '#1111ff'
+      - name: 'metrics2'
+        path: 'pageviews/${target}/bot'
+        color: '#ff1111'
+    edit:
+      - name: 'all metrics'
+        path: '${groupname}/pageviews/all'
+        complex: true
+        description: 'Pageviews graph (team: ${groupname}, service: ${target})'
+        stack: false
+        type: 'LINE2'
+        data:
+          - path: 'pageviews/${target}/total'
+          - path: 'pageviews/${target}/bot'
+            type: 'LINE1'
+```
+
+With configurations above and command line below:
+
+```sh
+ $ gfclient spec.yaml example1 myservice super_team
+```
+
+`gfclient` works with GrowthForecast at http://your.gf.host.local:5125/ like this:
+
+1. check `pageviews/myservice/total` and `pageviews/myservice/bot` exists or not
+2. check these are configured with specified configuration parameters or not
+3. **check** `super_team/pageviews/all` complex graph exists or not, and is configured correctly or not
+4. report result
+
+`gfclient` do check only without options. With **-f or --force** option, graphs specified in `edit` section are created/editted.
+
+1. check `pageviews/myservice/total` and `pageviews/myservice/bot` exists or not, and are configured correctly or not
+2. **abort** when any mismatches found for `check` section specs
+3. check graph specs in `edit` section
+4. **create** graph if specified path is not found
+5. **edit** graph if specified graph's configuration doesn't matches with spec
+
+`name` and `path` attributes must be specified in each check/edit items, and `complex: true` must be specified for complex graph. Others are optional (missing configuration attributes are ignored for check, and specified as default value for create).
+
+(bold item is required)
+
+* basic graph
+  * **name** (label of this spec item)
+  * **path** (string like `service/section/graph`)
+  * description (text)
+  * mode (string: 'gauge', 'subtract' or 'both')
+  * sort (number: 19-0)
+  * color (string like '#0088ff')
+  * type (string: 'AREA', 'LINE1' or 'LINE2') (LINE2 meas bold)
+  * ulimit, llimit (number: effective range upper/lower limit)
+  * stype (string: 'AREA', 'LINE1' or 'LINE2') (mode of subtract graph)
+  * sulimit, llimit (number: effective range of subtract graph)
+* complex graph
+  * **name** (label of this spec item)
+  * **path** (string like `service/section/graph`)
+  * **complex** (true or false: true must be specified for complex graph)
+  * description (text)
+  * sort (number: 19-0)
+  * sumup (true or false: display sum up value or not)
+  * mode/type/stack (global options for items of `data`, and may be overwritten by mode/type/stack of each items of `data`)
+  * **data** (list of basic graph in this complex graph)
+    * **path** (string: path of graph like `service/section/graph`)
+    * mode (string: 'gauge' or 'subtract')
+    * type (string: 'AREA', 'LINE1' or 'LINE2')
+    * stack (true or false)
+
+Spec file configurations (all of these are optional, and may be overwritten by command line options):
+
+```yaml
+config:
+  host: 'hostname.of.growthforecast' # default: localhost
+  port: 80         # default: 5125
+  prefix: '/gf'    # default: '/' (for cases if you mount GrowthForecast on subpath of web server)
+  username: 'name' # username of GrowthForecast's HTTP Basic authentication (default: none)
+  password: 'pass' # password (default: none)
+  debug: false     # show errors in growthforecast http response or not (default: false)
+  getall: false    # get and cache all graph informations before all checks
+                   # (default false, but you should specify true if your gf has many graphs)
+```
+
+You can check/edit many graphs with keywords list file like `gfclient -l listfile spec.yaml targetname`:
+
+```
+# keyword1 keyword2
+xx1        yy1
+xx2        yy2
+
+# blank and comments are ok (but invalid for line-end comment)
+aa1   bb1
+aa2   bb2
+```
+
 ## TODO
 
 * validations of specs
 * diff of 2 graph objects
-* bin/gfclient
 * tests
 
 ## Copyright

data/bin/gfclient

@@ -0,0 +1,76 @@
+#!/usr/bin/env ruby
+
+# gfclient spec.yaml TARGET ARG1 ARG2
+
+$0 = 'gfclient'
+
+$LOAD_PATH.push('lib')
+
+options = {}
+
+require 'optparse'
+opt = OptionParser.new
+opt.on('-f', '--force'){|v| options[:force] = v }
+opt.on('-H HOST', '--host=HOST'){|v| options[:host] = v }
+opt.on('-P PORT', '--port=PORT'){|v| options[:port] = v }
+opt.on('-u USER', '--user=USER'){|v| options[:username] = v }
+opt.on('-p PASS', '--pass=PASS'){|v| options[:password] = v }
+opt.on('--prefix=PREFIX'){|v| options[:prefix] = v }
+
+opt.on('-l PATH', '--list=PATH'){|v| options[:list] = v }
+opt.on('-a', '--get-all'){|v| options[:getall] = v }
+opt.on('-g', '--debug'){|v| options[:debug] = v }
+opt.on('-h', '--help'){|v| options[:help] = v }
+begin
+  opt.parse!(ARGV)
+rescue OptionParser::ParseError
+  options[:error] = true
+end
+
+specfile_path = ARGV[0]
+target_name = ARGV[1]
+keywords = ARGV[2,ARGV.size]
+
+unless specfile_path && target_name && keywords && !options[:error] && !options[:help]
+  puts <<"EOH"
+usage: gfclient SPEC_PATH TARGET_NAME [KEYWORD1 KEYWORD2 ...]
+  -f, --force     create/edit graphs with spec (default: check only)
+
+  -H, --host=HOST  set growthforecast hostname (overwrite configuration in spec)
+  -P, --port=PORT  set growthforecast port number
+
+  -u, --user=USER  set username for growthforecast basic authentication
+  -p, --pass=PASS  set password
+
+      --prefix=PREFIX  set growthforecast uri prefix
+
+  -l, --list=PATH  set keywords list file path (keywords set per line)
+                     do check/edit many times with this option
+                     (ignore default keywords)
+  -a, --get-all    get and cache all graph data at first (default: get incrementally)
+  -g, --debug      enable debug mode
+
+  -h, --help      show this message
+EOH
+  exit(0) if options[:help]
+  exit(1)
+end
+
+begin
+  require 'growthforecast/client'
+rescue LoadError
+  require 'rubygems'
+  require 'growthforecast/client'
+end
+
+retval = if options[:list]
+           unless File.file?(options[:list])
+             warn "keywords list file not found: #{options[:list]}"
+             exit(1)
+           end
+           keywords_list = File.open(options[:list]){|file| file.readlines }.select{|s| s !~ /^(#.*|\s+)$/}.map{|line| line.split(/\s+/)}
+           GrowthForecast::Client.execute_bulk(options[:force], specfile_path, target_name, keywords_list, options)
+         else
+           GrowthForecast::Client.execute(options[:force], specfile_path, target_name, keywords, options)
+         end
+exit retval

data/example/list

@@ -0,0 +1,6 @@
+a1 a2
+b1 b2
+c1 c2
+d1 d2
+e1 e2
+f1 f2

data/example/spec.yaml

@@ -0,0 +1,149 @@
+# gfclient [options] set_name keyword1 [keyword2 ...]
+config:
+  host: gf.local
+  port: 5127
+  debug: true
+  getall: true
+specs:
+  - name: 'httpstatus'
+    keywords:
+      - 'servicename'
+    check:
+      - name: 'httpstatus_2xx_percentage'
+        path: 'accesslog/httpstatus/${servicename}_2xx_percentage'
+        color: '#1111cc'
+      - name: 'httpstatus_2xx_rate'
+        path: 'accesslog/httpstatus/${servicename}_2xx_rate'
+        color: '#1111cc'
+      - name: 'httpstatus_3xx_percentage'
+        path: 'accesslog/httpstatus/${servicename}_3xx_percentage'
+        color: '#11cc11'
+      - name: 'httpstatus_3xx_rate'
+        path: 'accesslog/httpstatus/${servicename}_3xx_rate'
+        color: '#11cc11'
+      - name: 'httpstatus_429_percentage'
+        path: 'accesslog/httpstatus/${servicename}_429_percentage'
+        color: '#cccc77'
+      - name: 'httpstatus_429_rate'
+        path: 'accesslog/httpstatus/${servicename}_429_rate'
+        color: '#cccc77'
+      - name: 'httpstatus_4xx_percentage'
+        path: 'accesslog/httpstatus/${servicename}_4xx_percentage'
+        color: '#cc7711'
+      - name: 'httpstatus_4xx_rate'
+        path: 'accesslog/httpstatus/${servicename}_4xx_rate'
+        color: '#cc7711'
+      - name: 'httpstatus_5xx_percentage'
+        path: 'accesslog/httpstatus/${servicename}_5xx_percentage'
+        color: '#cc1111'
+      - name: 'httpstatus_5xx_rate'
+        path: 'accesslog/httpstatus/${servicename}_5xx_rate'
+        color: '#cc1111'
+      - name: 'httpstatus_unmatched_percentage'
+        path: 'accesslog/httpstatus/${servicename}_unmatched_percentage'
+        color: '#7711cc'
+      - name: 'httpstatus_unmatched_rate'
+        path: 'accesslog/httpstatus/${servicename}_unmatched_rate'
+        color: '#7711cc'
+      - name: 'httpstatus'
+        path: '${servicename}/accesslog/status'
+        complex: true
+        description: 'レスポンスのステータスコード別の割合'
+        sort: 19
+        gmode: 'gauge'
+        stack: true
+        type: 'AREA'
+        data:
+          - path: 'accesslog/httpstatus/${servicename}_2xx_percentage'
+          - path: 'accesslog/httpstatus/${servicename}_3xx_percentage'
+          - path: 'accesslog/httpstatus/${servicename}_429_percentage'
+          - path: 'accesslog/httpstatus/${servicename}_4xx_percentage'
+          - path: 'accesslog/httpstatus/${servicename}_5xx_percentage'
+          - path: 'accesslog/httpstatus/${servicename}_unmatched_percentage'
+  - name: 'responsetime'
+    keywords:
+      - 'servicename'
+    check:
+      - name: 'min'
+        path: 'accesslog/responsetime/${servicename}_min'
+        color: '#cccc77'
+      - name: 'avg'
+        path: 'accesslog/responsetime/${servicename}_avg'
+        color: '#1111cc'
+      - name: 'max'
+        path: 'accesslog/responsetime/${servicename}_max'
+        color: '#77cccc'
+      - name: 'percentile_50'
+        path: 'accesslog/responsetime/${servicename}_percentile_50'
+        color: '#11cc11'
+      - name: 'percentile_90'
+        path: 'accesslog/responsetime/${servicename}_percentile_90'
+        color: '#cc1111'
+      - name: 'percentile_95'
+        path: 'accesslog/responsetime/${servicename}_percentile_95'
+        color: '#336699'
+      - name: 'percentile_98'
+        path: 'accesslog/responsetime/${servicename}_percentile_98'
+        color: '#cc1177'
+      - name: 'percentile_99'
+        path: 'accesslog/responsetime/${servicename}_percentile_99'
+        color: '#cccc11'
+      - name: 'u100ms'
+        path: 'accesslog/responsetime/${servicename}_u100ms_percentage'
+        color: '#11cccc'
+      - name: 'u500ms'
+        path: 'accesslog/responsetime/${servicename}_u500ms_percentage'
+        color: '#1111cc'
+      - name: 'u1s'
+        path: 'accesslog/responsetime/${servicename}_u1s_percentage'
+        color: '#77cc11'
+      - name: 'u3s'
+        path: 'accesslog/responsetime/${servicename}_u3s_percentage'
+        color: '#cc7711'
+      - name: 'long'
+        path: 'accesslog/responsetime/${servicename}_long_percentage'
+        color: '#cc1111'
+      - name: 'unmatched'
+        path: 'accesslog/responsetime/${servicename}_unmatched_percentage'
+        color: '#7711cc'
+      - name: 'responsetime_percentage'
+        path: '${servicename}/accesslog/response_time_percentage'
+        complex: true
+        description: 'レスポンス所要時間(duration)毎の割合'
+        sort: 18
+        type: 'AREA'
+        stack: true
+        data:
+          - path: 'accesslog/responsetime/${servicename}_unmatched_percentage'
+          - path: 'accesslog/responsetime/${servicename}_u100ms_percentage'
+          - path: 'accesslog/responsetime/${servicename}_u500ms_percentage'
+          - path: 'accesslog/responsetime/${servicename}_u1s_percentage'
+          - path: 'accesslog/responsetime/${servicename}_u3s_percentage'
+          - path: 'accesslog/responsetime/${servicename}_long_percentage'
+      - name: 'responsetime'
+        path: '${servicename}/accesslog/response_time'
+        complex: true
+        description: 'レスポンス所要時間(duration:マイクロ秒)統計'
+        sort: 17
+        type: 'LINE2'
+        stack: false
+        data:
+          - path: 'accesslog/responsetime/${servicename}_min'
+          - path: 'accesslog/responsetime/${servicename}_avg'
+          - path: 'accesslog/responsetime/${servicename}_percentile_50'
+          - path: 'accesslog/responsetime/${servicename}_percentile_90'
+          - path: 'accesslog/responsetime/${servicename}_percentile_95'
+          - path: 'accesslog/responsetime/${servicename}_percentile_98'
+          - path: 'accesslog/responsetime/${servicename}_percentile_99'
+      - name: 'responsetime2'
+        path: '${servicename}/accesslog/response_time2'
+        complex: true
+        description: 'レスポンス所要時間(duration:マイクロ秒) avg/99%tile/max'
+        sort: 16
+        type: 'LINE2'
+        stack: false
+        data:
+          - path: 'accesslog/responsetime/${servicename}_avg'
+          - path: 'accesslog/responsetime/${servicename}_percentile_99'
+          - path: 'accesslog/responsetime/${servicename}_max'
+

data/example/testspec.yaml

@@ -0,0 +1,72 @@
+config:
+  host: localhost
+  port: 5125
+specs:
+  - name: 'spectest'
+    keywords:
+      - 'arg1'
+      - 'arg2'
+    check:
+      - name: 'test_first_graph'
+        path: 'example/spectest/${arg1}'
+        description: 'test1 for ${arg1} and ${arg2}'
+        color: '#ff1111'
+      - name: 'test_second_graph'
+        path: 'example/spectest/${arg1}_second'
+        description: 'test2 for ${arg1} and ${arg2}'
+        color: '#1111ff'
+      - name: 'test1_complex'
+        path: 'example/spectest/comp1_${arg1}'
+        complex: true
+        description: 'complex test for ${arg1} and ${arg2}'
+        stack: true
+        mode: 'gauge'
+        type: 'LINE1'
+        data:
+          - path: 'example/spectest/${arg1}'
+          - path: 'example/spectest/${arg1}_second'
+            type: 'LINE2'
+      - name: 'test2_complex'
+        path: 'example/spectest/comp2_${arg1}'
+        complex: true
+        description: 'complex test 2 for ${arg1} and ${arg2}'
+        stack: false
+        mode: 'gauge'
+        type: 'AREA'
+        data:
+          - path: 'example/spectest/${arg1}'
+          - path: 'example/spectest/${arg1}_second'
+  - name: 'spectest2'
+    keywords:
+      - 'arg1'
+      - 'arg2'
+    edit:
+      - name: 'test_first_graph'
+        path: 'example/spectest/${arg1}'
+        description: 'test1 for ${arg1} and ${arg2}'
+        color: '#ff1111'
+      - name: 'test_second_graph'
+        path: 'example/spectest/${arg1}_second'
+        description: 'test2 for ${arg1} and ${arg2}'
+        color: '#1111ff'
+      - name: 'test1_complex'
+        path: 'example/spectest/comp1_${arg1}'
+        complex: true
+        description: 'complex test for ${arg1} and ${arg2}'
+        stack: true
+        mode: 'gauge'
+        type: 'LINE1'
+        data:
+          - path: 'example/spectest/${arg1}'
+          - path: 'example/spectest/${arg1}_second'
+            type: 'LINE2'
+      - name: 'test2_complex'
+        path: 'example/spectest/comp2_${arg1}'
+        complex: true
+        description: 'complex test 2 for ${arg1} and ${arg2}'
+        stack: false
+        mode: 'gauge'
+        type: 'AREA'
+        data:
+          - path: 'example/spectest/${arg1}'
+          - path: 'example/spectest/${arg1}_second'

data/lib/growthforecast.rb

@@ -14,8 +14,9 @@ require 'json'
 
 class GrowthForecast
   attr_accessor :host, :port, :prefix, :timeout, :debug
+  attr_accessor :username, :password
 
-  def initialize(host='localhost', port=5125, prefix=nil, timeout=30, debug=false)
+  def initialize(host='localhost', port=5125, prefix=nil, timeout=30, debug=false, username=nil, password=nil)
     @host = host
     @port = port.to_i
     @prefix = if prefix && (prefix =~ /^\//) then prefix
@@ -25,11 +26,14 @@ class GrowthForecast
     @prefix.chop! if @prefix =~ /\/$/
     @timeout = timeout.to_i
     @debug = debug ? true : false
+
+    @username = username
+    @password = password
   end
 
   def debug(mode=nil)
     if mode.nil?
-      return GrowthForecast.new(@host,@port,@prefix,@timeout,true)
+      return GrowthForecast.new(@host,@port,@prefix,@timeout,true,@username,@password)
     end
     @mode = mode ? true : false
     self
@@ -68,6 +72,7 @@ class GrowthForecast
 
   def complexes
     list = request('GET', "/json/list/complex", {}, '', true)
+    return nil if list.nil?
     list.each do |path|
       path.complex = true
     end
@@ -172,8 +177,8 @@ class GrowthForecast
       obj
     when obj.is_a?(Array)
       obj.map{|e| concrete(e)}
-    when GrowthForecast::Path.path?(obj)
-      GrowthForecast::Path.new(obj)
+    when Path.path?(obj)
+      Path.new(obj)
     when obj['complex']
       Complex.new(obj)
     else
@@ -185,11 +190,25 @@ class GrowthForecast
     concrete(http_request(method, path, header, content, getlist))
   end
 
-  def http_request(method, path, header={}, content='', getlist=false)
+  def http_request(method, path, header={}, content=nil, getlist=false)
     conn = Net::HTTP.new(@host, @port)
     conn.open_timeout = conn.read_timeout = @timeout
     request_path = @prefix + path
-    res = conn.send_request(method, request_path, content, header)
+    req = case method
+          when 'GET'
+            Net::HTTP::Get.new(request_path, header)
+          when 'POST'
+            Net::HTTP::Post.new(request_path, header)
+          else
+            raise ArgumentError, "Invalid HTTP method for GrowthForecast: '#{method}'"
+          end
+    if content
+      req.body = content
+    end
+    if @username || @password
+      req.basic_auth(@username, @password)
+    end
+    res = conn.request(req)
 
     unless res.is_a?(Net::HTTPSuccess)
       return [] if getlist and res.code == '404'

data/lib/growthforecast/cache.rb

@@ -0,0 +1,46 @@
+class GrowthForecast::Cache
+  def initialize(client, tree=nil, reload=true)
+    @client = client
+    @tree = tree
+    @reload = reload
+
+    @graphs = []
+    @complexes = []
+  end
+
+  def get(service, section, graph, force_reload=false)
+    if @tree && (!force_reload)
+      return @tree.fetch(service, {}).fetch(section, {})[graph]
+    end
+
+    item = check_cached_path(service, section, graph)
+    if item.is_a?(GrowthForecast::Path)
+      if item.complex?
+        item = @client.complex(item.id)
+      else
+        item = @client.graph(item.id)
+      end
+    end
+    item
+  end
+
+  def check_cached_path(service, section, graph, atfirst=true)
+    list = @graphs.select{|item| item.service_name == service && item.section_name == section && item.graph_name == graph}
+    unless list.empty?
+      return list.first
+    end
+
+    list = @complexes.select{|item| item.service_name == service && item.section_name == section && item.graph_name == graph}
+    unless list.empty?
+      return list.first
+    end
+
+    if @reload && atfirst
+      @graphs = @client.graphs
+      @complexes = @client.complexes
+      return check_cached_path(service, section, graph, false)
+    end
+
+    nil
+  end
+end

data/lib/growthforecast/client.rb

@@ -0,0 +1,142 @@
+require 'growthforecast'
+require 'growthforecast/cache'
+require 'growthforecast/spec'
+
+require 'yaml'
+
+module GrowthForecast::Client
+  def self.execute_bulk(force, spec_path, target, keywords_list, options)
+    gf, cache, specs = setup(spec_path, options)
+    spec = specs.select{|s| s['name'] == target}
+    if spec.size != 1
+      warn "#{spec.size} specs with name '#{target}'" if spec.size > 1
+      warn "Spec #{target} not found" if spec.size < 1
+      return 2
+    end
+    spec = spec.first
+
+    keywords_list.each do |keywords|
+      retval = execute_once(gf, cache, force, spec, keywords)
+      if force && retval != 0
+        warn "aborting batch mode."
+        return retval
+      end
+    end
+    return 0
+  end
+
+  def self.execute(force, spec_path, target, keywords, options)
+    gf, cache, specs = setup(spec_path, options)
+    spec = specs.select{|s| s['name'] == target}
+    if spec.size != 1
+      warn "#{spec.size} specs with name '#{target}'" if spec.size > 1
+      warn "Spec #{target} not found" if spec.size < 1
+      return 2
+    end
+    spec = spec.first
+
+    execute_once(gf, cache, force, spec, keywords)
+  end
+
+  def self.execute_once(gf, cache, force, spec, keywords)
+    # generate dictionary
+    if spec['keywords'].size != keywords.size
+      warn "Keyword mismatch, in spec: #{spec.keywords.join('/')}"
+      exit(2)
+    end
+    dic = Hash[[spec['keywords'], keywords].transpose]
+
+    check_success = true
+    # check [and fix] all specs
+    check_target = if force
+                     spec['check'] || []
+                   else
+                     (spec['check'] || []) + (spec['edit'] || [])
+                   end
+    check_target.each do |check|
+      c = GrowthForecast::Spec.new(dic, check)
+      result,errors = c.check(cache)
+      unless result
+        errors.each{|e|
+          warn "#{c.name}(#{c.service_name}/#{c.section_name}/#{c.graph_name} [#{c.complex? ? 'complex' : 'graph'}]) #{e}"
+        }
+        check_success = false
+      end
+    end
+
+    unless check_success
+      warn "Some check failure exists, aborting."
+      return 1
+    end
+    return 0 unless force
+
+    (spec['edit'] || []).each do |edit|
+      e = GrowthForecast::Spec.new(dic, edit)
+      result,errors = e.check(cache)
+      next if result
+      
+      target = cache.get(e.service_name, e.section_name, e.graph_name)
+      if target
+        if target.complex? ^ e.complex?
+          warn "#{e.name}(#{e.service_name}/#{e.section_name}/#{e.graph_name}) complex type is not match: skip."
+          next
+        end
+        # edit
+        warn "update #{e.name}(#{e.service_name}/#{e.section_name}/#{e.graph_name})"
+        target = e.merge(cache, target)
+        unless target
+          warn "aborting with error."
+          return 1
+        end
+        gf.debug.edit(target)
+      else
+        # generate
+        warn "create #{e.name}(#{e.service_name}/#{e.section_name}/#{e.graph_name})"
+        target = e.merge(cache, nil)
+        unless target
+          warn "aborting with error."
+          return 1
+        end
+        gf.debug.add(target)
+        unless target.complex? # basic graph creation cannot handle options (except for 'color')
+          target = e.merge(cache, cache.get(e.service_name, e.section_name, e.graph_name, true))
+          gf.debug.edit(target)
+        end
+      end
+    end
+    return 0
+  end
+
+  def self.setup(spec_path, options)
+    spec_data = nil
+    begin
+      spec_data = YAML.load_file(spec_path)
+    rescue => e
+      warn "Spec file load error: #{e.message}"
+      exit(2)
+    end
+    specs = spec_data['specs']
+    spec_config = spec_data['config']
+
+    config = lambda{|name| options[name] || spec_config[name.to_s]}
+
+    host = config.call(:host) || 'localhost'
+    port = config.call(:port) || 5125
+    prefix = config.call(:prefix)
+    gf = GrowthForecast.new(host, port, prefix)
+
+    gf.debug = true if config.call(:debug)
+
+    gf.username = config.call(:username)
+    gf.password = config.call(:password)
+
+    tree = if config.call(:getall)
+             gf.tree()
+           else
+             nil
+           end
+    cache = GrowthForecast::Cache.new(gf, tree)
+
+    return gf, cache, specs
+  end
+end

data/lib/growthforecast/spec.rb

@@ -0,0 +1,239 @@
+class GrowthForecast::Spec
+  attr_accessor :name, :path, :complex
+  attr_accessor :service_name, :section_name, :graph_name
+
+  def complex?
+    @complex
+  end
+  
+  def initialize(dic, spec_yaml, complex=false)
+    @dic = dic
+
+    @spec = spec_yaml.dup
+
+    @name = @spec.delete('name')
+    @path = @spec.delete('path')
+    @complex = @spec.delete('complex') || complex
+    
+    @service_name, @section_name, @graph_name = replace_keywords(@path).split('/')
+
+    unless @service_name and @section_name and @graph_name and
+        not @service_name.empty? and not @section_name.empty? and not @graph_name.empty?
+      raise ArgumentError, "'path' must be as service/section/graph (#{@path})"
+    end
+  end
+
+  def replace_keywords(str)
+    @dic.reduce(str){|r, pair| r.gsub('${' + pair[0] + '}', pair[1])}
+  end
+
+  # return [true/false, [error notifications]]
+  def check(cache)
+    target = cache.get(@service_name, @section_name, @graph_name)
+
+    if target.nil?
+      return false, ["target path #{@service_name}/#{@section_name}/#{@graph_name} not exists"]
+    elsif self.complex? ^ target.complex?
+      return false, ["complex type is not match"]
+    end
+
+    if self.complex?
+      self.check_complex(cache, target)
+    else
+      self.check_graph(cache, target)
+    end
+  end
+
+  def merge(cache, target)
+    if self.complex?
+      self.merge_complex(cache, target)
+    else
+      self.merge_graph(cache, target)
+    end
+  end
+
+  GRAPH_ATTRIBUTES = [
+    :description, :mode, :sort, :color, :gmode,
+    :type, :ulimit, :llimit, :stype, :sulimit, :sllimit,
+    :adjust, :adjustval, :unit,
+  ]
+  def check_graph(cache, target)
+    errors = []
+    GRAPH_ATTRIBUTES.each do |attr|
+      next unless @spec.has_key?(attr.to_s)
+      target_val = target.send(attr)
+      spec_val = if (attr == :description) && @spec[attr.to_s]
+                   replace_keywords(@spec[attr.to_s])
+                 else
+                   @spec[attr.to_s]
+                 end
+      unless target_val == spec_val
+        errors.push("attribute #{attr} value mismatch, spec '#{spec_val}' but '#{target_val}'")
+      end
+    end
+    return errors.empty?, errors
+  end
+
+  def merge_graph(cache, target)
+    if target.nil?
+      target = GrowthForecast::Graph.new({
+          service_name: @service_name, section_name: @section_name, graph_name: @graph_name,
+          description: '',
+        })
+    end
+    GRAPH_ATTRIBUTES.each do |attr|
+      next unless @spec.has_key?(attr.to_s)
+      val = if attr == :description
+              replace_keywords(@spec[attr.to_s])
+            else
+              @spec[attr.to_s]
+            end
+      target.send((attr.to_s + '=').to_sym, val)
+    end
+    target
+  end
+
+  COMPLEX_ATTRIBUTES = [ :description, :sort, :sumup ]
+  def check_complex(cache, target)
+    errors = []
+    COMPLEX_ATTRIBUTES.each do |attr|
+      next unless @spec.has_key?(attr.to_s)
+      target_val = target.send(attr)
+      spec_val = if (attr == :description) && @spec[attr.to_s]
+                   replace_keywords(@spec[attr.to_s])
+                 else
+                   @spec[attr.to_s]
+                 end
+      unless target_val == spec_val
+        errors.push("attribute #{attr} value mismatch, spec '#{spec_val}' but '#{target_val}'")
+      end
+    end
+
+    unless @spec.has_key?('data')
+      return errors.empty?, errors
+    end
+
+    # @spec has 'data'
+    spec_data_tmpl = {}
+    spec_data_tmpl['gmode'] = @spec['gmode'] if @spec.has_key?('gmode')
+    spec_data_tmpl['stack'] = @spec['stack'] if @spec.has_key?('stack')
+    spec_data_tmpl['type'] = @spec['type'] if @spec.has_key?('type')
+
+    target_data = target.send(:data)
+
+    @spec['data'].each_with_index do |item, index|
+      specitem = spec_data_tmpl.merge(item)
+
+      #path
+      unless specitem['path']
+        errors.push("data[#{index}]: path missing")
+        next
+      end
+      replaced_path = replace_keywords(specitem['path'])
+      path_element = replaced_path.split('/')
+      unless path_element.size == 3
+        errors.push("data[#{index}]: path is not like SERVICE/SECTION/GRAPH")
+      end
+      specitem_graph = cache.get(*path_element)
+      unless specitem_graph
+        errors.push("data[#{index}]: specified graph '#{replaced_path}' not found")
+      end
+
+      #data(sub graph) size
+      unless target_data[index]
+        errors.push("data[#{index}]: graph member missing")
+        next
+      end
+
+      #data graph_id
+      if specitem_graph.id.to_i != target_data[index].graph_id.to_i
+        errors.push("data[#{index}]: mismatch, spec '#{replaced_path}'(graph id #{specitem_graph.id}) but id '#{target_data[index].graph_id}'")
+      end
+      #gmode, type
+      if specitem.has_key?('gmode') && specitem['gmode'] != target_data[index].gmode
+        errors.push("data[#{index}]: gmode mismatch, spec '#{specitem['gmode']}' but '#{target_data[index].gmode}'")
+      end
+      if specitem.has_key?('type') && specitem['type'] != target_data[index].type
+        errors.push("data[#{index}]: type mismatch, spec '#{specitem['type']}' but '#{target_data[index].type}'")
+      end
+      #stack: stack of first data item is nonsense
+      if index > 0 && specitem.has_key?('stack') && specitem['stack'] != target_data[index].stack
+        errors.push("data[#{index}]: stack mismatch, spec '#{specitem['stack']}' but '#{target_data[index].stack}'")
+      end
+    end
+
+    return errors.empty?, errors
+  end
+
+  def merge_complex(cache, target)
+    if target.nil?
+      target = GrowthForecast::Complex.new({
+          complex: true,
+          service_name: @service_name, section_name: @section_name, graph_name: @graph_name,
+          description: '',
+        })
+    end
+
+    COMPLEX_ATTRIBUTES.each do |attr|
+      next unless @spec.has_key?(attr.to_s)
+
+      val = if attr == :description
+              replace_keywords(@spec[attr.to_s])
+            else
+              @spec[attr.to_s]
+            end
+      target.send((attr.to_s + '=').to_sym, val)
+    end
+
+    unless @spec.has_key?('data')
+      return target
+    end
+
+    # @spec has 'data'
+    spec_data_tmpl = {}
+    spec_data_tmpl['gmode'] = @spec['gmode'] if @spec.has_key?('gmode')
+    spec_data_tmpl['stack'] = @spec['stack'] if @spec.has_key?('stack')
+    spec_data_tmpl['type'] = @spec['type'] if @spec.has_key?('type')
+
+    target.data = target.data.dup
+
+    @spec['data'].each_with_index do |item, index|
+      specitem = spec_data_tmpl.merge(item)
+
+      #path
+      unless specitem['path']
+        warn "data[#{index}]: path missing"
+        return nil
+      end
+      replaced_path = replace_keywords(specitem['path'])
+      path_element = replaced_path.split('/')
+      unless path_element.size == 3
+        warn "data[#{index}]: path '#{replaced_path}' is not like SERVICE/SECTION/GRAPH"
+        return nil
+      end
+      specitem_graph = cache.get(*path_element)
+      unless specitem_graph
+        warn "data[#{index}]: path '#{replaced_path}' not found"
+        return nil
+      end
+
+      #data(sub graph) size
+      unless target.data[index]
+        target.data[index] = GrowthForecast::Complex::Item.new({
+            graph_id: specitem_graph.id,
+            gmode: (specitem['gmode'] || nil), # nil: default
+            stack: (specitem.has_key?('stack') ? specitem['stack'] : nil),
+            type: (specitem['type'] || nil),
+          })
+        next
+      end
+
+      target.data[index].graph_id = specitem_graph.id if specitem_graph.id != target.data[index].graph_id
+      target.data[index].gmode = specitem['gmode'] if specitem.has_key?('gmode') && specitem['gmode'] != target.data[index].gmode
+      target.data[index].type  = specitem['type']  if specitem.has_key?('type') && specitem['type'] != target.data[index].type
+      target.data[index].stack = specitem['stack'] if index > 0 && specitem.has_key?('stack') && specitem['stack'] != target.data[index].stack
+    end
+
+    target
+  end
+end

data/lib/growthforecast/version.rb

@@ -1,3 +1,3 @@
 class GrowthForecast
-  VERSION = "0.0.1"
+  VERSION = "0.0.2"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: growthforecast
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
   prerelease: 
 platform: ruby
 authors:
@@ -9,13 +9,14 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-01-03 00:00:00.000000000 Z
+date: 2013-02-04 00:00:00.000000000 Z
 dependencies: []
 description: Client library and tool to update values, create/edit/delete graphs of
   GrowthForecast
 email:
 - tagomoris@gmail.com
-executables: []
+executables:
+- gfclient
 extensions: []
 extra_rdoc_files: []
 files:
@@ -24,12 +25,19 @@ files:
 - LICENSE
 - README.md
 - Rakefile
+- bin/gfclient
+- example/list
+- example/spec.yaml
 - example/test1.rb
+- example/testspec.yaml
 - growthforecast.gemspec
 - lib/growthforecast.rb
+- lib/growthforecast/cache.rb
+- lib/growthforecast/client.rb
 - lib/growthforecast/complex.rb
 - lib/growthforecast/graph.rb
 - lib/growthforecast/path.rb
+- lib/growthforecast/spec.rb
 - lib/growthforecast/version.rb
 homepage: https://github.com/tagomoris/rb-growthforecast
 licenses: []