MuranoCLI 2.0.0

data/TODO.taskpaper

@@ -0,0 +1,122 @@
+Readme:
+	- Add more walk-thrus of common actions. (create new solution, …)
+	- Look into using VCR for testing. @pri(low)
+
+Commands:
+	- Init command. @done(2016-11-28)
+	- Empty sub-commands should return help. @done(2016-11-21)
+	There are a bunch of empty sub-commands that prefix another layer. Such as
+	assign, content, product, and others.  Those should be impemented as a ‘help’
+	only command.  That is they should return help like the plain `mr` command, but
+	just for their sub-section of things
+	- Errors and Warnings should get sent to STDERR @done(2016-11-03)
+	- Need a more consistent output format. 'pp' is still used in many places. @done(2016-11-03)
+	Maybe have a tool setting for output format? json, yaml, pp, csv, table ?
+	- Do we need rainbow since highline does ANSI color too? @done(2016-11-02)
+	- Status will show {modules,eventhandlers} have changes when they don’t. @done(2016-09-23)
+	- First time run needs to be smoothed out. @done(2016-08-09)
+	- Default for most commands should be -same @done(2016-08-02)
+	- Add Diff Command @done(2016-07-27)
+
+Account:
+	- token is refetched many times. Do this once per run. @done(2016-09-12)
+	- Netrc library (or the netrc format) doesn't allow '#' in passwords. @done(2016-08-10)
+
+Endpoints:
+	- In fetch(); add content_type to script header if not application/json @done(2016-12-06)
+	- Add support for multiple endpoints in one file @pri(high) @done(2016-11-18)
+	- Add directory support like in modules @done(2016-07-26)
+
+Files:
+	- Switch to mime-types v3 @pri(low)
+	- Add ignore patterns to config @done(2016-12-20)
+	- Figure out how to make the hexed-sha checksum faster. @done(2016-09-23)
+	- Fix upload. @done(2016-08-01)
+	- Files won't update, they always delete then add. @done(2016-07-28)
+
+Users and Roles:
+	Much of this is stuck until we get more docs on the User/Role management
+	- Figure out how to upload (create and update) user info.
+	- Figure out how to add Roles to Users in the local data and upload it.
+	- Fix diff for Users and Roles.
+	- Have hash keys in the yaml be strings not symbols. (don't start with colon) @done(2016-07-27)
+
+CORS:
+	- Get working with sync up/down. @done(2016-10-28)
+	- GET&PUT /cors data @done(2016-09-08)
+
+TSDB:
+	- For query, if no metrics on cmdline, then do listMetrics and use all. @done(2016-11-21)
+	well, the first 1000 or whatever we get from a single call to listMetrics
+	- Query should handle tags prefixed with @ to match write. @done(2016-11-21)
+	- Add support for new TSDB service. @pri(high) @done(2016-11-03)
+
+Timeseries:
+	- Add CSV output option. @done(2016-09-09)
+
+Product:
+	- Add option for progress bar when uploading content files.
+	- Support multiple products. @pri(low)
+	Think about how this would work.  There is the syncing of the resources, and then
+	some of the commands that use product.id.
+	Which product.id maps to which resource file?
+	- Auto convert exoline spec files into murano spec files on upload? @done(2016-10-27)
+	Not doing this. Convert is there if you need it.
+	- write alias command @done(2016-09-26)
+	- Need to add way to set the product ID on a device eventhandler. @done(2016-08-01)
+
+Service Device:
+	- When listing and business.id is missing, gracefully fall back to --idonly @done(2016-09-12)
+
+Config:
+	- Store passwords in system Keychain on system that have a Keychain.
+	mac OS does, various Linux desktops have a couple differnet ones.  Not sure about
+	Windows.
+	MacOS: https://github.com/xli/mac-keychain
+	Windows8+: https://msdn.microsoft.com/en-us/library/windows/apps/xaml/hh465069.aspx
+	Linux: https://en.wikipedia.org/wiki/GNOME_Keyring
+	       https://en.wikipedia.org/wiki/KWallet
+
+	Plus things like 1Password, LastPass, KeePass, and others.
+
+	- Add config sync.bydefault to set which things sync{up,down} by default @done(2017-01-05)
+	It is internally hardcoded to be -s, -a, -m, -e right now.
+	- Add ENV['MR_CONFIGFILE'] path to file to load like --configfile @done(2016-09-22)
+	- Maybe add dotenv support. @done(2016-09-22)
+	- Think about adding dev,staging,prod system; how would that work? @done(2016-09-16)
+
+SyncUpDown:
+	- Document the hash keys for an item. @pri(high)
+	Also consider turning that hash into a Struct
+	- Allow specifying local files to limit actions to. @done(2017-03-03)
+
+SolutionBase:
+	- All network traffic is serialized. Make some parallel.
+	This might break some things.
+	- Errors from the server should be displayed prettier. @done(2016-09-26)
+	- JSON parse should use symbols for keys. @done(2016-09-01)
+	- Add the --curl verbose option. @done(2016-08-12)
+	- Rebuild how local names and paths are computed from remote items. @done(2016-07-27)
+
+Windows:
+	- Need to test with http://rubyinstaller.org on Windows. @pri(high) @done(2016-12-21)
+	- Look into http://ocra.rubyforge.org for building an exec. @pri(high) @done(2016-12-21)
+
+Bundles:
+	- Revisit this idea. Its complexity may not be worth its value.
+	- Test syncdown behavor.
+	- Work on design @done(2016-08-09)
+	Thinking of something like VIM bundles.  A directory of directories. Each with a
+	manafest file? (maybe)  A Bundle is a group of modules, endpoints, static files
+	and the other things.
+
+	There needs to be some layering logic added, where the bundles are stacked and
+	then the top-level files are stack on top of that. This builds the final map of
+	what gets uploaded to the server.
+
+	For syncdown, bundles are considered to be read-only.
+
+	The goal is to have things like Users or Debug that you just include into a
+	project.  And it gives all of the library, routes, statics and whatnot that you
+	need.
+

data/bin/mr

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+require 'highline'
+
+puts HighLine.color("!"*80, :yellow)
+puts HighLine.color("The command 'mr' is deprecated.  Use 'murano' instead.", :yellow)
+puts HighLine.color("!"*80, :yellow)
+
+exec 'murano', *ARGV

data/bin/murano

@@ -0,0 +1,84 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
+require 'commander/import'
+require 'pathname'
+require 'MrMurano'
+require 'pp'
+require 'dotenv'
+Dotenv.load
+
+Signal.trap('INT', 'EXIT') # Don't drop traces on ^C
+
+program :version, MrMurano::VERSION
+program :description, %{Manage a Solution and Product in Exosite's Murano}
+
+global_option('-V', '--verbose', 'Be chatty') {
+  $cfg['tool.verbose'] = true
+}
+global_option('-n', '--dry', %{Don't run actions that make changes}) {
+  $cfg['tool.dry'] = true
+  $cfg['tool.verbose'] = true # dry implies verbose
+}
+global_option('-L', '--curl', 'Print out a curl command for each network call') {
+  $cfg['tool.curldebug'] = true
+}
+global_option '--skip-plugins', %{Don't load plugins. Good for when one goes bad.}
+
+global_option('-C', '--configfile FILE', %{Load additional configuration file}) {|file|
+  # this is called after all of the top level code in this file.
+  $cfg.load_specific(file)
+}
+global_option('-c', '--config KEY=VALUE', %{Set a single config key}) {|param|
+  key, value = param.split('=', 2)
+  # a=b :> ["a","b"]
+  # a= :> ["a",""]
+  # a :> ["a"]
+  raise "Bad config '#{param}'" if key.nil?
+  if value.nil? then
+    $cfg[key] = 'true'
+  else
+    $cfg[key] = value
+  end
+}
+
+default_command :help
+#default_command :syncup
+
+$cfg = MrMurano::Config.new
+$cfg.load
+$project = MrMurano::ProjectFile.new
+$project.load
+
+# Basic command support is:
+# - read/write config file in [Project, User, System] (all are optional)
+# - Introspection for tab completion.
+# - Look for tools in PATH that are +x and "mr-foo..."
+
+
+# Look for plug-ins
+pgds = [
+  Pathname.new(Dir.home) + '.mrmurano' + 'plugins',
+  Pathname.new(Dir.home) + '.murano' + 'plugins'
+]
+# Add plugin dirs from configs
+# This is run before the command line options are parsed, so need to check old way.
+if not ARGV.include? '--skip-plugins' then
+  pgds << Pathname.new(ENV['MR_MURANO_PLUGIN_DIR']) if ENV.has_key? 'MR_MURANO_PLUGIN_DIR'
+  pgds << Pathname.new(ENV['MURANO_PLUGIN_DIR']) if ENV.has_key? 'MURANO_PLUGIN_DIR'
+  pgds.each do |path|
+    next unless path.exist?
+    path.each_child do |plugin|
+      next if plugin.directory?
+      next unless plugin.readable?
+      next if plugin.basename.fnmatch('.*') # don't read anything starting with .
+      begin
+        require plugin.to_s
+      rescue Exception => e
+        $stderr.puts "Failed to load plugin at #{plugin} because #{e}"
+      end
+    end
+  end
+end
+
+#  vim: set ai et sw=2 ts=2 :

data/docs/demo.md

@@ -0,0 +1,109 @@
+
+
+# Every sub-command will --help
+
+# Start anew
+- Clone project: `git clone https://github.com/tadpol/GWE-Multitool.git demo01`
+- `cd demo01`
+
+- Pick a bussiness: `mr business list`
+- Set it: `mr config business.id ZZZZZZZZZ`
+
+- Create a product: `mr product create myawesomeproduct`
+- Save the result: `mr config product.id YYYYYYYYY`
+
+- Set the product definition: `mr config product.spec gwe-multitool.yaml`
+- Set the directory to look for specs. `mr config location.specs spec`
+- Sync the product definition up: `mr syncup -V --specs`
+
+- Create a solution: `mr solution create myawesomesolution`
+- Save the result: `mr config solution.id XXXXXX`
+- Assign the product to the solution: `mr assign set`
+
+# What got configured?
+`mr config --dump`
+
+
+
+
+
+
+# <voice type='orc'>Work Work</voice>
+
+- What is going to change? `mr status`
+- Sync solution code up: `mr syncup -V`
+
+- Change a file
+- What is going to change? `mr status`
+- Details of change: `mr diff`
+
+
+
+
+# Devices
+- Add a real device: `mr product device enable 42:42:42:42:42:42`
+- !!!cheet and activate by hand: `mr product device activate 42:42:42:42:42:42`
+- Which resources are there? `mr product spec pull`
+- What did the device write to that one resource?
+  `mr product device read 42:42:42:42:42:42 update_interval`
+- `mr product device write 42:42:42:42:42:42 update_interval 300`
+
+
+
+# Multiple configs
+Because Developing, Staging, Production.
+
+Set addition config file to load with `MR_CONFIGFILE`
+
+Also supports '.env'
+
+
+
+
+# Device Content Area
+
+See GWE.
+
+
+
+
+
+# Debugging
+
+## Logs
+- `mr logs`
+- `mr logs --follow`
+
+
+
+
+
+
+## Keystore
+### What is in the Keystore?
+`mr keystore list`
+
+### Write and Read a Key
+- `mr keystore set test greebled`
+- `mr keystore get test`
+
+### Write to a Set
+- `mr keystore command sadd myset greebled`
+Or any other supported Redis command.
+
+### Remove just the ones with 'socketmap'
+`mr keystore list | grep socketmap | xargs -L1 mr keystore delete`
+
+
+
+
+## TSDB
+
+- `mr tsdb list metrics`
+- `mr tsdb list tags`
+- `mr tsdb query @sn=1 temp0`
+- `mr tsdb query @sn=1 temp0 --limit=4`
+- `mr tsdb query @sn=1 --limit=10 -c outformat=csv --epoch ms`
+
+
+

data/lib/MrMurano/Account.rb

@@ -0,0 +1,211 @@
+require 'uri'
+require 'net/http'
+require 'json'
+require 'date'
+require 'pathname'
+require 'yaml'
+require 'MrMurano/Config'
+require 'MrMurano/http'
+require 'MrMurano/verbosing'
+
+module MrMurano
+  class Passwords
+    include Verbose
+    def initialize(path=nil)
+      path = $cfg.file_at('passwords', :user) if path.nil?
+      path = Pathname.new(path) unless path.kind_of? Pathname
+      @path = path
+      @data = nil
+    end
+    def load()
+      if @path.exist? then
+        @path.chmod(0600)
+        @path.open('rb') do |io|
+          @data = YAML.load(io)
+        end
+      end
+    end
+    def save()
+      @path.dirname.mkpath unless @path.dirname.exist?
+      @path.open('wb') do |io|
+        io << @data.to_yaml
+      end
+      @path.chmod(0600)
+    end
+    def set(host, user, pass)
+      unless @data.kind_of? Hash then
+        @data = {host=>{user=>pass}}
+        return
+      end
+      hd = @data[host]
+      if hd.nil? or not hd.kind_of?(Hash) then
+        @data[host] = {user=>pass}
+        return
+      end
+      @data[host][user] = pass
+      return
+    end
+    def get(host, user)
+      return ENV['MURANO_PASSWORD'] unless ENV['MURANO_PASSWORD'].nil?
+      unless ENV['MR_PASSWORD'].nil? then
+        warning %{Using depercated ENV "MR_PASSWORD", please rename to "MURANO_PASSWORD"}
+        return ENV['MR_PASSWORD']
+      end
+      return nil unless @data.kind_of? Hash
+      return nil unless @data.has_key? host
+      return nil unless @data[host].kind_of? Hash
+      return nil unless @data[host].has_key? user
+      return @data[host][user]
+    end
+
+    ## Remove the password for a user
+    def remove(host, user)
+      if @data.kind_of? Hash then
+        hd = @data[host]
+        if not hd.nil? and hd.kind_of?(Hash) then
+          if hd.has_key? user then
+            @data[host].delete user
+          end
+        end
+      end
+    end
+
+    ## Get all hosts and usernames. (does not return the passwords)
+    def list
+      ret = {}
+      @data.each_pair{|key,value| ret[key] = value.keys} unless @data.nil?
+      ret
+    end
+  end
+
+  class Account
+    include Http
+    include Verbose
+
+    def endPoint(path)
+      URI('https://' + $cfg['net.host'] + '/api:1/' + path.to_s)
+    end
+
+    def _loginInfo
+      host = $cfg['net.host']
+      user = $cfg['user.name']
+      if user.nil? or user.empty? then
+        error("No Murano user account found; please login")
+        user = ask("User name: ")
+        $cfg.set('user.name', user, :user)
+      end
+      pff = $cfg.file_at('passwords', :user)
+      pf = Passwords.new(pff)
+      pf.load
+      pws = pf.get(host, user)
+      if pws.nil? then
+        error("Couldn't find password for #{user}")
+        pws = ask("Password:  ") { |q| q.echo = "*" }
+        pf.set(host, user, pws)
+        pf.save
+      end
+      {
+        :email => $cfg['user.name'],
+        :password => pws
+      }
+    end
+
+    # Store the token in a class variable so that we only fetch it once per run
+    # session of this tool
+    @@token = nil
+    def token
+      if @@token.nil? then
+        # Cannot have token call token, so cannot use workit.
+        uri = endPoint('token/')
+        request = Net::HTTP::Post.new(uri)
+        request['User-Agent'] = "MrMurano/#{MrMurano::VERSION}"
+        request.content_type = 'application/json'
+        curldebug(request)
+        #request.basic_auth(username(), password())
+        request.body = JSON.generate(_loginInfo)
+
+        response = http.request(request)
+        case response
+        when Net::HTTPSuccess
+          token = JSON.parse(response.body, json_opts)
+          @@token = token[:token]
+        else
+          showHttpError(request, response)
+          error "Check to see if username and password are correct."
+          @@token = nil
+        end
+      end
+      @@token
+    end
+
+    def token_reset(value=nil)
+      @@token = value
+    end
+
+    def new_account(email, name, company="")
+      post('/key/', {
+        :email=>email,
+        :name=>name,
+        :company=>company,
+        :source=>'signup',
+      })
+    end
+
+    def reset_account(email)
+      post('/key/', { :email=>email, :source=>'reset' })
+    end
+
+    def accept_account(token, password)
+      post("/key/#{token}", {:password=>password})
+    end
+
+    def businesses
+      _loginInfo if $cfg['user.name'].nil?
+      get('user/' + $cfg['user.name'] + '/membership/')
+    end
+
+    def new_business(name)
+      post('/business/', {:name=>name})
+    end
+
+    def delete_business(id)
+      delete("/business/#{id}")
+    end
+
+    def products
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      get('business/' + $cfg['business.id'] + '/product/')
+    end
+
+    ## Create a new product in the current business
+    def new_product(name, type='onepModel')
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      post('business/' + $cfg['business.id'] + '/product/', {:label=>name, :type=>type})
+    end
+
+    def delete_product(modelId)
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      delete('business/' + $cfg['business.id'] + '/product/' + modelId)
+    end
+
+    def solutions
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      get('business/' + $cfg['business.id'] + '/solution/')
+    end
+
+    ## Create a new solution
+    def new_solution(name, type='dataApi')
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      raise "Solution name must be a valid domain name component" unless name.match(/^[a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9]{0,1}|[a-zA-Z0-9]{0,62})$/)
+      post('business/' + $cfg['business.id'] + '/solution/', {:label=>name, :type=>type})
+    end
+
+    def delete_solution(apiId)
+      raise "Missing Business ID" if $cfg['business.id'].nil?
+      delete('business/' + $cfg['business.id'] + '/solution/' + apiId)
+    end
+
+  end
+end
+
+#  vim: set ai et sw=2 ts=2 :