lipsiadmin 3.3.4 → 3.4.0

data/CHANGELOG

@@ -1,3 +1,17 @@
+2009-04-23
+  * Fix rdoc for grid view
+  * Fix a bug override default ext component methods
+
+2009-04-20
+  * Fix compatibility issues with ruby 1.9
+  * Added a new module for background workers
+  * Little formatting to tab helper for accept custom options
+  * Fix rdoc for open_window helper
+
+2009-04-10
+  * Added startup test for account
+  * Added raketasks for init test and autotest
+
 2009-03-24
   * Fix landescape mode in pdf builder
 

data/README

@@ -45,7 +45,7 @@ now run script/generate and you can see some like:
 
   Installed Generators
     Builtin: controller, integration_test, mailer, migration, model, etc..
-    Lipsiadmin: attachment, backend, backend_page, pdf
+    Lipsiadmin: attachment, backend, backend_page, pdf, loops
 
 So you can do:
 

data/Rakefile

@@ -42,7 +42,7 @@ spec = Gem::Specification.new do |s|
   s.homepage          = "http://groups.google.com/group/lipsiadmin"
   s.rubyforge_project = "lipsiadmin"
   s.platform          = Gem::Platform::RUBY
-  s.summary           = "Lipsiadmin is a new revolutionary admin for your projects.Lipsiadmin is based on Ext Js 2.0. framework (with prototype adapter) and is ready for Rails 2.0. This admin is for newbie developper but also for experts, is not entirely written in javascript because the aim of developper wose build in a agile way web/site apps so we use extjs in a new intelligent way a mixin of “old” html and new ajax functions, for example ext manage the layout of page, grids, tree and errors, but form are in html code."
+  s.summary           = "Lipsiadmin is a new revolutionary admin for your projects.Lipsiadmin is based on Ext Js 2.0. framework (with prototype adapter) and is ready for Rails 2.0. This admin is for newbie developper but also for experts, is not entirely written in javascript because the aim of developper wose build in a agile way web/site apps so we use extjs in a new intelligent way a mixin of 'old' html and new ajax functions, for example ext manage the layout of page, grids, tree and errors, but form are in html code."
   s.files             = FileList["CHANGELOG", "README", "MIT-LICENSE", "Rakefile", "init.rb", "{lipsiadmin_generators,lib,resources,tasks}/**/*"].to_a
   s.has_rdoc          = true
   s.requirements     << 'Haml'

data/lib/controller/ext.rb

@@ -71,7 +71,7 @@ module Lipsiadmin
             options[:query]   = false
           end
           
-          # Reforma DataIndex
+          # Reformat DataIndex
           if options[:dataIndex].is_a?(Array)
             options[:dataIndex] = options[:dataIndex].collect do |f| 
               f.is_a?(Symbol) ? "#{@model.table_name}.#{f}" : f 

data/lib/data_base/attachment/storage.rb

@@ -205,12 +205,9 @@ module Lipsiadmin
 
         def find_credentials(creds)
           case creds
-          when File:
-            YAML.load_file(creds.path)
-          when String:
-            YAML.load_file(creds)
-          when Hash:
-            creds
+          when File   then YAML.load_file(creds.path)
+          when String then YAML.load_file(creds)
+          when Hash   then creds
           else
             raise ArgumentError, "Credentials are not a path, file, or hash."
           end

data/lib/loops.rb

@@ -0,0 +1,277 @@
+require 'yaml'
+
+module Lipsiadmin
+  
+  # ==Simple background loops framework for rails
+  # 
+  # Loops is a small and lightweight framework for Ruby on Rails created to support simple 
+  # background loops in your application which are usually used to do some background data processing 
+  # on your servers (queue workers, batch tasks processors, etc).
+  # 
+  # Authors:: Alexey Kovyrin and Dmytro Shteflyuk
+  # Comments:: This plugin has been created in Scribd.com for internal use.
+  # 
+  # == What tasks could you use it for?
+  # 
+  # Originally loops plugin was created to make our own loops code more organized. We used to have tens 
+  # of different modules with methods that were called with script/runner and then used with nohup and 
+  # other not so convenient backgrounding techniques. When you have such a number of loops/workers to 
+  # run in background it becomes a nightmare to manage them on a regular basis (restarts, code upgrades, 
+  # status/health checking, etc).
+  # 
+  # After a short time of writing our loops in more organized ways we were able to generalize most of the 
+  # loops code so now our loops look like a classes with a single mandatory public method called *run*. 
+  # Everything else (spawning many workers, managing them, logging, backgrounding, pid-files management, 
+  # etc) is handled by the plugin it
+  # 
+  # 
+  # == But there are dozens of libraries like this! Why do we need one more?
+  # 
+  # The major idea behind this small project was to create a deadly simple and yet robust framework to 
+  # be able to run some tasks in background and do not think about spawning many workers, restarting 
+  # them when they die, etc. So, if you need to be able to run either one or many copies of your worker or 
+  # you do not want to think about re-spawning dead workers and do not want to spend megabytes of RAM on 
+  # separate copies of Ruby interpreter (when you run each copy of your loop as a separate process 
+  # controlled by monit/god/etc), then I'd recommend you to try this framework -- you'd like it.
+  # 
+  # 
+  # == How to use?
+  # 
+  # Generate binary and configuration files by running 
+  # 
+  #   script/generate loops 
+  # 
+  # This will create the following list of files:
+  # 
+  #   script/loops        # binary file that will be used to manage your loops
+  #   config/loops.yml    # example configuration file
+  #   app/loops/simple.rb # REALLY simple loop example
+  # 
+  # Here is a simple loop scaffold for you to start from (put this file to app/loops/hello_world_loop.rb):
+  # 
+  #   class HelloWorldLoop < Lipsiadmin::Loops::Base
+  #     def run
+  #       debug("Hello, debug log!")
+  #       sleep(config['sleep_period']) # Do something "useful" and make it configurable
+  #       debug("Hello, debug log (yes, once again)!")
+  #     end
+  #   end
+  # 
+  # When you have your loop ready to use, add the following lines to your (maybe empty yet) config/loops.yml 
+  # file:
+  # 
+  #   hello_world:
+  #     sleep_period: 10
+  # 
+  # This is it! To start your loop, just run one of the following commands:
+  # 
+  #   # Generates: list all configured loops:
+  #   $ script/loops -L
+  #   
+  #   # Generates: run all enabled (actually non-disabled) loops in foreground:
+  #   $ script/loops -a
+  #   
+  #   # Generates: run all enabled loops in background:
+  #   $ script/loops -d -a
+  #   
+  #   # Generates: run specific loop in background:
+  #   $ ./script/loops -d -l hello_world
+  #   
+  #   # Generates: all possible options:
+  #   $ ./script/loops -h
+  # 
+  # 
+  # == How to run more than one worker?
+  # 
+  # If you want to have more than one copy of your worker running, that is as simple as adding one 
+  # option to your loop configuration:
+  # 
+  #   hello_world:
+  #     sleep_period: 10
+  #     workers_number: 1  
+  # 
+  # This _workers_number_ option would say loops manager to spawn more than one copy of your loop 
+  # and run them in parallel. The only thing you'd need to do is to think about concurrent work of 
+  # your loops. For example, if you have some kind of database table with elements you need to 
+  # process, you can create a simple database-based locks system or use any memcache-based locks.
+  # 
+  # 
+  # == There is this <tt>workers_engine</tt> option in config file. What it could be used for?
+  # 
+  # There are two so called "workers engines" in this plugin: <tt>fork</tt> and <tt>thread</tt>. They're used 
+  # to control the way process manager would spawn new loops workers: with <tt>fork</tt> engine we'll 
+  # load all loops classes and then fork ruby interpreter as many times as many workers we need. 
+  # With <tt>thread</tt> engine we'd do Thread.new instead of forks. Thread engine could be useful if you 
+  # are sure your loop won't lock ruby interpreter (it does not do native calls, etc) or if you 
+  # use some interpreter that does not support forks (like jruby).
+  # 
+  # Default engine is <tt>fork</tt>.
+  # 
+  # 
+  # == What Ruby implementations does it work for?
+  # 
+  # We've tested and used the plugin on MRI 1.8.6 and on JRuby 1.1.5. At this point we do not support 
+  # demonization in JRuby and never tested the code on Ruby 1.9. Obviously because of JVM limitations 
+  # you won't be able to use +fork+ workers engine in JRuby, but threaded workers do pretty well.
+  #
+  module Loops
+    
+    class << self
+      
+      # Set/Return the main config
+      def config
+        @@config
+      end
+      
+      # Set/Return the loops config
+      def loops_config
+        @@loops_config
+      end
+
+      # Set/Return the global config
+      def global_config
+        @@global_config
+      end
+      
+      # Load the yml config file, default config/loops.yml
+      def load_config(file)
+        @@config = YAML.load_file(file)
+        @@global_config = @@config['global']
+        @@loops_config = @@config['loops']
+
+        @@logger = create_logger('global', global_config)
+      end
+      
+      # Start loops, default :all
+      def start_loops!(loops_to_start = :all)
+        @@running_loops = []
+        @@pm = Loops::ProcessManager.new(global_config, @@logger)
+
+        # Start all loops
+        loops_config.each do |name, config|
+          next if config['disabled']
+          next unless loops_to_start == :all || loops_to_start.member?(name)
+          klass = load_loop_class(name)
+          next unless klass
+
+          start_loop(name, klass, config) 
+          @@running_loops << name
+        end
+
+        # Do not continue if there is nothing to run
+        if @@running_loops.empty?
+          puts "WARNING: No loops to run! Exiting..."
+          return
+        end
+
+        # Start monitoring loop
+        setup_signals
+        @@pm.monitor_workers
+
+        info "Loops are stopped now!"
+      end
+
+    private
+
+      # Proxy logger calls to the default loops logger
+      [ :debug, :error, :fatal, :info, :warn ].each do |meth_name|
+        class_eval <<-EVAL
+          def #{meth_name}(message)
+            LOOPS_DEFAULT_LOGGER.#{meth_name} "\#{Time.now}: loops[RUNNER/\#{Process.pid}]: \#{message}"
+          end
+        EVAL
+      end
+
+      def load_loop_class(name)
+        begin
+          klass_file = LOOPS_ROOT + "/app/loops/#{name}.rb" 
+          debug "Loading class file: #{klass_file}"
+          require(klass_file)
+        rescue Exception
+          error "Can't load the class file: #{klass_file}. Worker #{name} won't be started!"
+          return false
+        end
+
+        klass_name = "#{name}".classify
+        klass = klass_name.constantize rescue nil
+
+        unless klass
+          error "Can't find class: #{klass_name}. Worker #{name} won't be started!"
+          return false
+        end
+
+        begin
+          klass.check_dependencies
+        rescue Exception => e
+          error "Loop #{name} dependencies check failed: #{e} at #{e.backtrace.first}"
+          return false
+        end
+
+        return klass
+      end
+
+      def start_loop(name, klass, config)
+        puts "Starting loop: #{name}"
+        info "Starting loop: #{name}"
+        info " - config: #{config.inspect}"
+
+        @@pm.start_workers(name, config['workers_number'] || 1) do
+          debug "Instantiating class: #{klass}"
+          looop = klass.new(create_logger(name, config))
+          looop.name = name
+          looop.config = config
+
+          debug "Starting the loop #{name}!"
+          fix_ar_after_fork
+          looop.run
+        end
+      end
+
+      def create_logger(loop_name, config)
+        config['logger'] ||= (global_config['logger'] || 'default')
+
+        return LOOPS_DEFAULT_LOGGER if config['logger'] == 'default'
+        return Logger.new(STDOUT)   if config['logger'] == 'stdout'
+        return Logger.new(STDERR)   if config['logger'] == 'stderr'
+
+        config['logger'] = File.join(LOOPS_ROOT, config['logger']) unless config['logger'] =~ /^\//
+        Logger.new(config['logger'])
+
+      rescue Exception => e
+        message = "Can't create a logger for the #{loop_name} loop! Will log to the default logger!"
+        puts "ERROR: #{message}"
+
+        message << "\nException: #{e} at #{e.backtrace.first}"
+        error(message)
+
+        return LOOPS_DEFAULT_LOGGER
+      end
+
+      def setup_signals
+        trap('TERM') {
+          warn "Received a TERM signal... stopping..."
+          @@pm.stop_workers!
+        }
+
+        trap('INT') { 
+          warn "Received an INT signal... stopping..."
+          @@pm.stop_workers!
+        }
+
+        trap('EXIT') { 
+          warn "Received a EXIT 'signal'... stopping..."
+          @@pm.stop_workers!
+        }
+      end
+
+      def fix_ar_after_fork
+        ActiveRecord::Base.clear_active_connections!
+        ActiveRecord::Base.verify_active_connections!
+      end
+      
+    end
+  end
+end
+
+require 'loops/process_manager'
+require 'loops/base'

data/lib/loops/base.rb

@@ -0,0 +1,52 @@
+module Lipsiadmin
+  module Loops
+    # This is the base class for make a simple loop with its own custom run method
+    #
+    # Examples:
+    #
+    #   class SimpleLoop < Lipsiadmin::Loops::Base
+    #     def run
+    #       debug(Time.now)
+    #     end
+    #   end
+    #
+    class Base#:nodoc:
+      attr_accessor :name, :config, :logger
+      
+      # The initialize method, default we pass the logger
+      def initialize(logger)
+        self.logger = logger
+      end
+
+      # Ovveride this method for check dependencies
+      def self.check_dependencies; end
+
+      # Proxy logger calls to our logger
+      [ :debug, :error, :fatal, :info, :warn ].each do |meth_name|
+        class_eval <<-EVAL
+          def #{meth_name}(message)
+            logger.#{meth_name}("\#{Time.now}: loop[\#{name}/\#{Process.pid}]: \#{message}")
+          end
+        EVAL
+      end
+
+      def with_lock(entity_id, loop_id, timeout, entity_name = '')#:nodoc:
+        entity_name = 'item' if entity_name.to_s.empty?
+
+        debug("Locking #{entity_name} #{entity_id}")
+        lock = LoopLock.lock(:entity_id => entity_id, :loop => loop_id.to_s, :timeout => timeout)
+        unless lock
+          warn("Race condition detected for the #{entity_name}: #{entity_id}. Skipping the item.")
+          return
+        end
+
+        begin
+          yield
+        ensure
+          debug("Unlocking #{entity_name} #{entity_id}")
+          LoopLock.unlock(:entity_id => entity_id, :loop => loop_id.to_s)
+        end
+      end
+    end
+  end
+end

data/lib/loops/daemonize.rb

@@ -0,0 +1,72 @@
+module Lipsiadmin
+  module Loops
+    module Daemonize#:nodoc:
+      def self.read_pid(pid_file)
+        File.open(pid_file) do |f| 
+          f.gets.to_i 
+        end
+      rescue Errno::ENOENT
+        0
+      end
+
+      def self.check_pid(pid_file)
+        pid = read_pid(pid_file)
+        return false if pid.zero?
+        if defined?(::JRuby)
+          system "kill -0 #{pid} &> /dev/null"
+          return $? == 0
+        else 
+          Process.kill(0, pid)
+        end
+        true
+      rescue Errno::ESRCH, Errno::ECHILD, Errno::EPERM
+        false
+      end
+
+      def self.create_pid(pid_file)
+        if File.exist?(pid_file)
+          puts "Pid file #{pid_file} exists! Checking the process..."
+          if check_pid(pid_file)
+            puts "Can't create new pid file because another process is runnig!"
+            return false
+          end
+          puts "Stale pid file! Removing..."
+          File.delete(pid_file)
+        end
+
+        puts "Creating pid file..."
+        File.open(pid_file, 'w') do |f|
+          f.puts(Process.pid)
+        end
+
+        return true
+      end
+
+      def self.daemonize(app_name)
+        if defined?(::JRuby)
+          puts "WARNING: daemonize method is not implemented for JRuby (yet), please consider using nohup."
+          return
+        end
+
+        srand # Split rand streams between spawning and daemonized process
+        fork && exit # Fork and exit from the parent
+
+        # Detach from the controlling terminal
+        unless sess_id = Process.setsid
+          raise Daemons.RuntimeException.new('cannot detach from controlling terminal')
+        end
+
+        # Prevent the possibility of acquiring a controlling terminal
+        trap 'SIGHUP', 'IGNORE'
+        exit if pid = fork
+
+        $0 = app_name if app_name
+
+        Dir.chdir(LOOPS_ROOT) # Make sure we're in the working directory
+        File.umask(0000) # Insure sensible umask
+
+        return sess_id
+      end    
+    end
+  end
+end