flow-lite 0.1.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: fd6dc6a4982583a8267aea617e5d7852321b7a00
-  data.tar.gz: 5e050df47d5f1d52881c2666f1bb732f39cb4981
+  metadata.gz: 48d2755dfa5c6d365387d7a788804dd458d0fedf
+  data.tar.gz: '04586b4969ec5b3aa278ba933904c62ea753f765'
 SHA512:
-  metadata.gz: 7778366b484f614248c91c33d9aa3e01377c6861408277e5f9e49391e0b5640edcc3dee685c7ed415484bc0a3f2f9c3a2aff852bbbba777b9087e7e49e8803c0
-  data.tar.gz: 1b0d6f6148183a97326edcf11c1a0b7b140573b0d758d157c5c06d43947c4e2ca26831c24973f5f56688b06fae421188e44648a94bcde84c81ba27ebd7d10319
+  metadata.gz: 10c26e5b61781002e063f4695d676d8847e666950b25c6fe451a1d4e396b14c5b85f5892ecd2cbcc1ac35840f3823462edcb95e0ff221acf06021e30a0503db1
+  data.tar.gz: b7fdb1a505e1b9eb92dd219e8e86af7813b17874562f1f45ab0b9f18be8f4f3cf1c44a8d2571c56cb8b17912960ff58b82de1f1af379b92c3cb62a1452c456ce

data/Manifest.txt

@@ -4,4 +4,6 @@ README.md
 Rakefile
 bin/flow
 lib/flow-lite.rb
+lib/flow-lite/base.rb
+lib/flow-lite/tool.rb
 lib/flow-lite/version.rb

data/README.md

@@ -3,13 +3,14 @@
 flow-lite gem - (yet) another (lite) workflow engine; let's you define your workflow steps in Flowfiles; incl. the flow command line tool
 
 
-* home  :: [github.com/rubycoco/git](https://github.com/rubycoco/git)
-* bugs  :: [github.com/rubycoco/git/issues](https://github.com/rubycoco/git/issues)
+* home  :: [github.com/rubycoco/flow](https://github.com/rubycoco/flow)
+* bugs  :: [github.com/rubycoco/flow/issues](https://github.com/rubycoco/flow/issues)
 * gem   :: [rubygems.org/gems/flow-lite](https://rubygems.org/gems/flow-lite)
 * rdoc  :: [rubydoc.info/gems/flow-lite](http://rubydoc.info/gems/flow-lite)
 
 
 
+
 ## Usage
 
 
@@ -19,12 +20,12 @@ Define the workflow steps in a Flowfile. Example:
 ``` ruby
 step :first_step do
   puts "first_step"
-  second_step    # note: you can call other steps like methods
+  step :second_step    # note: you can call other steps with step
 end
 
 step :second_step do
   puts "second_step"
-  third_step
+  step :third_step
 end
 
 step :third_step do
@@ -32,14 +33,40 @@ step :third_step do
 end
 ```
 
-And than use the `flow` command line tool to run a step.
+And then use the `flow` command line tool to run a step.
 Example:
 
 ```
 $ flow first_step
 ```
 
-Note: By default the `flow` command line tool reads in the `Flowfile`. Use `-f/--flowfile` option to use a different file.
+Note: By default the `flow` command line tool reads in and looks for `flowfile`, `Flowfile`, `flowfile.rb`, `Flowfile.rb`
+in that order.
+Use the `-f/--flowfile` option to use a different file.
+
+
+
+**Prelude / Prolog**
+
+Use the `-r/--require` option to (auto-)require
+some extra libraries or scripts.
+By default for now the "prelude / prolog" that always
+gets auto-required includes:
+
+``` ruby
+require 'pp'
+require 'time'
+require 'date'
+require 'json'
+require 'yaml'
+require 'fileutils'
+
+require 'uri'
+require 'net/http'
+require 'net/https'
+```
+
+Tip: See the [`flow-lite.rb`](https://github.com/rubycoco/flow/blob/master/flow-lite/lib/flow-lite.rb) source for the definite always up-to-date list.
 
 
 That's it for now.
@@ -66,14 +93,17 @@ flowfile = Flow::Flowfile.load( <<TXT )
   end
 TXT
 
-flow = flowfile.flow   # (auto-)builds a flow class (see Note 1)
-                       # and constructs/returns an instance
-flow.hello             #=> "Hello, world!"
-flow.hola              #=> "¡Hola, mundo!"
+flow = flowfile.flow_class.new   # (auto-)build a flow class (see Note 1)
+                                 # and construct/return a new instance
+flow.step_hello             #=> "Hello, world!"
+flow.step_hola              #=> "¡Hola, mundo!"
+flow.step( :hello )         #=> "Hello, world!"
+flow.step( :hola )          #=> "¡Hola, mundo!"
+flow.class.step_methods     #=> [:hello, :hola]
 
 # or use ruby's (regular) message/metaprogramming machinery
-flow.send( :hello )    #=> "Hello, world!"
-flow.send( :hola  )    #=> "¡Hola, mundo!"
+flow.send( :step_hello )    #=> "Hello, world!"
+flow.send( :step_hola  )    #=> "¡Hola, mundo!"
 # or try
 flow.class.instance_methods.grep( /^step_/ ) #=> [:step_hello, :step_hola]
 # ...
@@ -95,27 +125,48 @@ gets used to (auto-) build (via metaprogramming) a flow class like:
 
 ``` ruby
 class Greeter < Flow::Base
-  def hello
+  def step_hello
+    puts "Hello, world!"
+  end
+
+  def step_hola
+    puts "¡Hola, mundo!"
+  end
+end
+```
+
+
+Note: Behind the stage the metaprogramming "class macro"
+`define_step( symbol, method )`
+or `define_step( symbol ) { block }` defined in `Flow::Base`
+gets used, thus,
+if you want to create steps in a "hand-coded" class
+use:
+
+
+``` ruby
+class Greeter < Flow::Base
+  define_step :hello do
     puts "Hello, world!"
   end
-  alias_method :step_hello, :hello
 
-  def hola
+  define_step :hola do
     puts "¡Hola, mundo!"
   end
-  alias_method :step_hola, :hola
 end
 ```
 
 
 
+
+
 **Tips & Tricks**
 
 Auto-include pre-build / pre-defined steps. Use a (regular) Module e.g.:
 
 ``` ruby
 module GitHubActions
-  def setup
+  def step_setup
      # setup ssh
      ssh_key  = ENV['SSH_KEY']
      ssh_path = File.expand_path( '~/.ssh' )
@@ -133,8 +184,6 @@ module GitHubActions
 
      # ...
   end
-  # optional - for auto-discovery add a step alias
-  alias_method :step_setup, :setup
 end
 ```
 
@@ -144,9 +193,60 @@ and than use (regular) include e.g.:
 class Flow::Base
   include GitHubActions
 end
+
+#-or-
+
+Flow::Base.include( GitHubActions )
+```
+
+Now all your flows can (re)use `setup` or any other step methods you define.
+
+
+
+
+## Real World Examples
+
+**Collect GitHub Statistics**
+
+Use GitHub Actions to collect
+GitHub Statistics via GitHub API calls
+and update the JSON datasets
+in the `/cache.github` repo at the `yorobot` org(anization):
+
+``` ruby
+step :clone do
+  Mono.clone( '@yorobot/cache.github' )
+end
+
+
+step :update do
+  Hubba.config.data_dir = Mono.real_path( '@yorobot/cache.github' )
+
+  username = 'geraldb'
+  h = Hubba.reposet( username )
+  pp h
+
+  Hubba.update_stats( h )
+  Hubba.update_traffic( h )
+  puts "Done."
+end
+
+
+step :push do
+  msg = "auto-update week #{Date.today.cweek}"
+
+  Mono.open( '@yorobot/cache.github' ) do |proj|
+    if proj.changes?
+      proj.add( "." )
+      proj.commit( msg )
+      proj.push
+    end
+  end
+end
 ```
 
-Now all your flows can (re)use `setup` or any other methods you define.
+(Sources: [`Flowfile`](https://github.com/yorobot/backup/blob/master/Flowfile.rb), [`workflows/update.yml`](https://github.com/yorobot/backup/blob/master/.github/workflows/update.yml) @ `yorobot/backup`)
+
 
 
 

data/Rakefile

@@ -8,7 +8,7 @@ Hoe.spec 'flow-lite' do
   self.summary = "flow-lite gem - (yet) another (lite) workflow engine; let's you define your workflow steps in Flowfiles; incl. the flow command line tool"
   self.description = summary
 
-  self.urls    = { home: 'https://github.com/rubycoco/git' }
+  self.urls    = { home: 'https://github.com/rubycoco/flow' }
 
   self.author  = 'Gerald Bauer'
   self.email   = 'ruby-talk@ruby-lang.org'
@@ -17,11 +17,7 @@ Hoe.spec 'flow-lite' do
   self.readme_file  = 'README.md'
   self.history_file = 'CHANGELOG.md'
 
-  self.extra_deps = [
-    ['gitti', '>= 0.6.1' ],
-    ['hubba', '>= 1.0.0' ],
-    ['monos', '>= 0.5.0' ],
-  ]
+  self.extra_deps = []
 
   self.licenses = ['Public Domain']
 

data/lib/flow-lite.rb

@@ -1,21 +1,28 @@
+##
+## "prelude / prolog " add some common used stdlibs
+##   add more - why? why not?
 require 'pp'
-require 'optparse'
+require 'time'
+require 'date'
+require 'json'
+require 'yaml'
+require 'fileutils'
 
+require 'uri'
+require 'net/http'
+require 'net/https'
 
 
-####################
-# "prelude/prolog" add some 3rd party libs/gems
-# -- our own
-require 'gitti'
-require 'hubba'  ## todo/fix: rename to gitti-api/gitti-apis
-require 'mono'
-# -- some more ???
+require 'optparse'
 
 
 
 #####################
 # our own code
 require 'flow-lite/version'   # note: let version always go first
+require 'flow-lite/base'
+require 'flow-lite/tool'
+
 
 
 
@@ -38,27 +45,22 @@ class Step
 end # class Step
 
 
+class Flowfile
 
-class Base    ## base class for flow class (auto)-constructed/build from flowfile
-  def self.define_step( step )
-    name = step.names[0]
-    puts "    adding step  >#{name}<..."
-    define_method( name, &step.block )
-    alias_method( :"step_#{name}", name )  ## (auto-)add step_<name> alias
-
-    alt_names = step.names[1..-1]
-    alt_names.each do |alt_name|
-      puts "      adding alias >#{alt_name}< for >#{name}<..."
-      alias_method( alt_name, name )
+  ## find flowfile path by convention
+  ## check for name by convention in this order:
+  NAMES = ['flowfile',    'Flowfile',
+           'flowfile.rb', 'Flowfile.rb']
+  def self.find_file
+    NAMES.each do |name|
+      return "./#{name}"   if File.exist?( "./#{name}" )
     end
-  end
-end  # class Base
-
+    nil
+  end # method self.find_file
 
 
-class Flowfile
   ## convenience method - use like Flowfile.load_file()
-  def self.load_file( path='./Flowfile' )
+  def self.load_file( path )
     code = File.open( path, 'r:utf-8' ) { |f| f.read }
     load( code )
   end
@@ -72,17 +74,21 @@ class Flowfile
 
 
 
-  def flow  ## build flow class
+  def flow
+    ## todo/check: always return a new instance why? why not?
+    flow_class.new
+  end
+
+  def flow_class
     @flow_class ||= build_flow_class
-    @flow_class.new   ## todo/check: always return a new instance why? why not?
   end
 
   def build_flow_class
-    puts "  building flow class..."
+    puts "[flow]  building flow class..."
     klass = Class.new( Base )
 
     steps.each do |step|
-      klass.define_step( step )
+      klass.define_step( step.names, &step.block )
     end
 
     klass
@@ -102,41 +108,11 @@ class Flowfile
     @steps << Step.new( name, block )
   end
 
-
   def run( name )
-    name = name.to_sym  ## make sure we always use symbols
-    if flow.respond_to?( name )
-       flow.send( name )
-    else
-      puts "!! ERROR: step definition >#{name}< not found; cannot run/execute - known steps include:"
-      pp @steps
-      exit 1
-    end
+    ## todo/check: always return/use a new instance why? why not?
+    flow_class.new.step( name )
   end
 end # class Flowfile
-
-
-
-class Tool
-  def self.main( args=ARGV )
-    options = {}
-    OptionParser.new do |parser|
-      parser.on( '-f FILENAME', '--flowfile FILENAME' ) do |filename|
-        options[:flowfile] = filename
-      end
-    end.parse!( args )
-
-    path  =  options[:flowfile] || './Flowfile'
-    flowfile = Flowfile.load_file( path )
-
-    ## allow multipe steps getting called - why? why not?
-    ##   flow setup clone update   etc??
-    args.each do |arg|
-      flowfile.run( arg )
-    end
-  end
-end # class Tool
-
 end # module Flow
 
 

data/lib/flow-lite/base.rb

@@ -0,0 +1,89 @@
+
+module Flow
+
+class Base    ## base class for flow class (auto)-constructed/build from flowfile
+  def self.define_step( name_or_names, &block )
+    names =  if name_or_names.is_a?( Array )
+               name_or_names
+             else
+               [name_or_names]   ## assume single symbol (name); wrap in array
+             end
+    names = names.map {|name| name.to_sym }   ## make sure we always use symbols
+
+
+    name = names[0]
+    puts "[flow]    adding step  >#{name}<..."
+    define_method( :"step_#{name}", &block )
+
+    alt_names = names[1..-1]
+    alt_names.each do |alt_name|
+      puts "[flow]      adding alias >#{alt_name}< for >#{name}<..."
+      alias_method( :"step_#{alt_name}", :"step_#{name}" )
+    end
+  end  # method self.define_step
+
+
+
+
+  TRUE_VALUES = [
+    'true', 't',
+    'yes', 'y',
+    'on',
+    '1',
+  ]
+
+  ### include / check for ruby debug flag too - why? why not?
+  def debug?
+    value = ENV['DEBUG']
+    if value && TRUE_VALUES.include?( value.downcase )
+      true
+    else
+      false
+    end
+  end
+
+
+
+  ## run step by symbol/name (e.g. step :hello - etc.)
+  ##
+  ##  todo/check:  allow (re)entrant calls to step (step calling step etc.) - why? why not?
+  def step( name )
+    step_name = :"step_#{name}"  ## note: make sure we always use symbols
+    if respond_to?( step_name )
+      #######
+      ## check: track (and report) call stack - why? why not?
+      ## e.g.
+      ##   [flow >(1) first_step)] step >first_step< - starting...
+      ##   [flow >(2) ..first_step > second_step)] step >second_step< - starting...
+      ##   [flow >(3) ....first_step > second_step > third_step)] step >third_step< - starting...
+      @stack ||= []   ## use a call stack of step names
+      @stack.push( name )
+
+      puts "[flow >(#{@stack.size}) #{'..'*(@stack.size-1)}#{@stack.join(' > ')})] step >#{name}< - starting..."
+      start_time = Time.now   ## todo: use Timer? t = Timer.start / stop / diff etc. - why? why not?
+
+      __send__( step_name )
+
+      end_time = Time.now
+      diff_time = end_time - start_time
+      puts "[flow <(#{@stack.size}) #{'..'*(@stack.size-1)}#{@stack.join(' < ')})] step >#{name}< - done in #{diff_time} sec(s)"
+      @stack.pop
+    else
+      puts "!! ERROR: step definition >#{name}< not found; cannot run/execute - known (defined) steps include:"
+      pp self.class.step_methods  #=> e.g. [:hello, ...]
+      exit 1
+    end
+  end  # method step
+
+
+  def self.step_methods
+    names = instance_methods.reduce([]) do |names, name|
+                                          names << $1.to_sym  if name =~ /^step_(.+)/
+                                          names
+                                        end
+    names
+  end
+end  # class Base
+
+
+end # module Flow

data/lib/flow-lite/tool.rb

@@ -0,0 +1,130 @@
+module Flow
+
+class Tool
+  def self.main( args=ARGV )
+    options = {}
+    OptionParser.new do |parser|
+      parser.on( '-f FILE', '--file FILE', '--flowfile FILE',
+                    'Read FILE as a flowfile.'
+               ) do |file|
+        options[:flowfile] = file
+      end
+
+      ## note:
+      ##  you can add many/multiple modules
+      ##  e.g. -r gitti -r mono etc.
+      parser.on( '-r NAME', '--require NAME') do |name|
+        options[:requires] ||= []
+        options[:requires] << name
+      end
+
+      parser.on( '-d', '--debug') do |debug|
+        options[:debug] = debug
+        ## note: for no auto-set env here - why? why not
+        ENV['DEBUG']='1'   ## use t or true or such - why? why not?
+        puts "[flow] setting >DEBUG< env variable to >#{1}<"
+      end
+
+      ## todo/check/reserve:  add  -e/--env(iornmanet)  e.g.  dev/test/prod(uction) etc.
+      ##             plus (convenience) shortcuts  e.g.  --dev, --prod(uction), --test
+    end.parse!( args )
+
+
+    ## check for any (dynamic/auto) requires
+    if options[:requires]
+      names = options[:requires]
+      names.each do |name|
+        ## todo/check: add some error/exception handling here - why? why not?
+        puts "[flow] auto-require >#{name}<..."
+        require( name )
+      end
+    else  ## use/try defaults
+      config_path = "./config.rb"
+      if File.exist?( config_path )
+        puts "[flow] auto-require (default) >#{config_path}<..."
+        require( config_path )
+      end
+    end
+
+
+    path = nil
+    if options[:flowfile]
+      path = options[:flowfile]
+    else
+      path = Flowfile.find_file
+
+      if path.nil?
+        STDERR.puts "!! ERROR - no flowfile found, sorry - looking for: #{Flowfile::NAMES.join(', ')} in (#{Dir.pwd})"
+        exit 1
+      end
+    end
+
+
+    ### split args into vars and steps
+    vars = []
+    args = args.select do |arg|
+      ## note: mark freestanding = as fatal error (empty var)
+      if arg == '='
+        STDERR.puts "!! ERROR - empty var (=) in args; sorry - make sure there are NO spaces before ="
+        exit 1
+      elsif arg =~ /^[A-Za-z][A-Za-z0-9_]*=/
+        vars << arg
+        false    ## filter -- do NOT include
+      else
+        true
+      end
+    end
+
+    if args.size > 0
+      puts "[flow] #{args.size} arg(s):"
+      pp args
+    end
+
+    if vars.size > 0
+      puts "[flow] #{vars.size} var(s):"
+      pp vars
+
+      ## auto-add vars to ENV
+      ##   note: if the value is empty e.g. DEBUG= or such
+      ##         the variable gets deleted / undefined / unset!!
+      vars.each do |var|
+        pos = var.index('=')   ## split on first =
+        name  = var[0..(pos-1)]
+        value = var[(pos+1)..-1]
+        # print "[flow] splitting "
+        # print "%-24s  "  % ">#{var}<"
+        # print "=>  name: "
+        # print "%-18s " % ">#{name}<,"
+        # print "value: >#{value}< (#{value.class.name})"
+        # print "\n"
+
+        if value.empty?  ## note: variable gets deleted / undefined / unset
+           puts "[flow]   UNSET >#{name}< env variable"
+           ENV[ name ] = nil
+        else
+           puts "[flow]   set >#{name}< env variable to >#{value}<"
+           ENV[ name ] = value
+        end
+      end
+    end
+
+
+
+    puts "[flow] loading >#{path}<..."
+    flowfile = Flowfile.load_file( path )
+
+
+
+    ## allow multipe steps getting called - why? why not?
+    ##   flow setup clone update   etc.  - yes, yes
+    ##    follow make model and allow variables with FOO= or bar= too
+    ##    note:  mark freestanding = as fatal error (empty var)
+
+    args.each do |arg|
+      flowfile.run( arg )
+    end
+  end # method self.main
+end # class Tool
+
+
+end # module Flow

data/lib/flow-lite/version.rb

@@ -1,6 +1,6 @@
 module FlowLite
 
-  MAJOR = 0    ## todo: namespace inside version or something - why? why not??
+  MAJOR = 1    ## todo: namespace inside version or something - why? why not??
   MINOR = 1
   PATCH = 0
   VERSION = [MAJOR,MINOR,PATCH].join('.')

metadata

@@ -1,57 +1,15 @@
 --- !ruby/object:Gem::Specification
 name: flow-lite
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Gerald Bauer
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-10-23 00:00:00.000000000 Z
+date: 2020-11-04 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: gitti
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.6.1
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.6.1
-- !ruby/object:Gem::Dependency
-  name: hubba
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.0.0
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.0.0
-- !ruby/object:Gem::Dependency
-  name: monos
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.5.0
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.5.0
 - !ruby/object:Gem::Dependency
   name: rdoc
   requirement: !ruby/object:Gem::Requirement
@@ -103,8 +61,10 @@ files:
 - Rakefile
 - bin/flow
 - lib/flow-lite.rb
+- lib/flow-lite/base.rb
+- lib/flow-lite/tool.rb
 - lib/flow-lite/version.rb
-homepage: https://github.com/rubycoco/git
+homepage: https://github.com/rubycoco/flow
 licenses:
 - Public Domain
 metadata: {}