resque-telework 0.2.1 → 0.3.0

data/README.md

@@ -5,10 +5,12 @@ Resque Telework
 
 Telework depends on Resque 1.20+ and Redis 2.2+
 
+Telework 0.3 has a new auto feature that is in beta and currently under testing - please report bugs to [gip.github@gmail.com](gip.github@gmail.com)
+
 Description
 -----------
 
-Telework is a [Resque](https://github.com/defunkt/resque) plugin aimed at controlling Resque workers from the web UI. It makes it easy to manage workers on a complex systems that includes several hosts, different queue(s) and an evolving source code that is deployed several times a day. Beyond starting and stopping workers on remote hosts, the plugin makes it easy to switch between code revisions, gives a partial view of each worker's log (stdout and stderr) and maintains a status of each workers.
+Telework is a [Resque](https://github.com/defunkt/resque) plugin aimed at controlling Resque workers from the web UI. It makes it easy to manage workers on a complex systems that includes several hosts, different queue(s) and an evolving source code that is deployed several times a day. Beyond starting and stopping workers on remote hosts, the plugin makes it easy to switch between code revisions, gives a partial view of each worker's log (stdout and stderr) and maintains a status of each workers. Version 0.3 adds an auto mode that is able to start workers depending on how full a given queue (or several queues) are. The workers are stopped once all the jobs are processed, allowing a better memory usage.
 
 Telework comes with three main components
 
@@ -16,7 +18,9 @@ Telework comes with three main components
 * A daemon process to be started on each host (`rake telework:start_daemon` starts a new daemon and returns while `rake telework:daemon` runs the daemon interactively)
 * A registration command (`rake telework:register_revision`) to be called by the deployment script when a new revision is added on the host
 
-Note that currently (Telework 0.0.1), the daemon process is included in the main app, which is not really elegant as the full Rails environment needs to be loaded to run the daemon. A light-weight daemon is currently being developed and should be ready in the coming weeks.
+Note that currently (Telework 0.3), the daemon process is included in the main app, which is not really elegant as the full Rails environment needs to be loaded to run the daemon. A light-weight daemon is currently being developed and should be ready in the coming weeks.
+
+Telework has been successfully used in production at Entelo for more that a year with up to 10 servers.
 
 Overview of the WebUI
 ---------------------
@@ -120,12 +124,20 @@ gilles@myworkhost0 $ rake telework:register_revision
 ```
 Note that it is not necessary to stop and restart the daemon. Restarting the daemon is only required when the Telework gem is updated.
 
+Auto Mode
+---------
+
+The auto mode is still under testing as of version 0.3. Starting/stopping workers in auto mode is done by the daemon using a simple heuristic. Parameters may be modified as the task page. The main parameters controlling the auto mode are
+* `Auto_delay`: this is the minimum amount of time, in second, that the daemon has to wait before to make a new change to the workers (e.g. start or stop workers). A large number prevents the overhead of stopping/starting workers too often
+* `Worker_count`, `Auto_job_per_worker` and `Auto_min_worker`: these parameters control the number of workers needed to process jobs from a given queue. Given a queue with `Q` pending jobs, the number of workers started by the daemon at a given time will be:
+
+  min( `Worker_count`, max( `Auto_min_worker`, ceil( `Q` / `Auto_job_per_worker` ) ) )
+
 Known Issues
 ------------
 
-For version 0.2:
+For version 0.3:
 
-* Buttons are not aligned in the web-UI
 * The daemon crashes if any of the log directories do not exist
 
 Bugs
@@ -136,10 +148,6 @@ Please report bugs on [github](https://github.com/gip/resque-telework/issues) or
 Todo
 ----
 
-The following features are are being developed and should be available shortly:
-
-* Improved window layout
-
 The following features are planned for future releases:
 
 * Light-weight daemon in Haskell

data/lib/resque-telework/global.rb

@@ -3,7 +3,7 @@ module Resque
     module Telework
       Name = NAME = 'resque-telework'
       Nickname = NICKNAME = 'telework'
-      Version = VERSION = '0.2.1'
+      Version = VERSION = '0.3.0'
       RedisInterfaceVersion = REDIS_INTERFACE_VERSION = '2'
     end
   end

data/lib/resque-telework/manager.rb

@@ -11,6 +11,7 @@ module Resque
           @SLEEP= cfg['daemon_pooling_interval']
           @WORKERS= {}
           @STOPPED= []
+          @AUTO= {}
         end
         
         # The manager (e.g. daemon) main loop
@@ -33,6 +34,7 @@ module Resque
               while cmd= cmds_pop( @HOST ) do    # Pop a command in the command queue
                 do_command(cmd)                  # Execute it
               end
+              check_auto                         # Deal with the task in auto mode
               sleep @SLEEP                       # Sleep
             end
                                                  # A stop request has been received
@@ -41,6 +43,7 @@ module Resque
             send_status( 'Error', "A stop request has been received by the #{@HOST} daemon but there are still running worker(s) so it will keep running") unless @WORKERS.empty?
             @RUN_DAEMON= true
           end
+          i_am_dead
         rescue Interrupt         # Control-C
           send_status( 'Info', "Interruption for #{@HOST} daemon, exiting gracefully") if @WORKERS.empty?
           send_status( 'Error', "Interruption for #{@HOST} daemon, exiting, running workers may now unexpectedly terminate") unless @WORKERS.empty?
@@ -79,18 +82,108 @@ module Resque
             start_worker( cmd, find_revision(cmd['revision']) )
           when 'signal_worker'
             manage_worker( cmd )
+          when 'start_auto'
+            start_auto( cmd, find_revision(cmd['revision']) )
+          when 'stop_auto'
+            stop_auto( cmd )
           when 'stop_daemon'
             @RUN_DAEMON= false
           when 'kill_daemon'
             send_status( 'Error', "A kill request has been received, the daemon on #{@HOST} is now brutally terminating by calling exit()")
+            i_am_dead
             exit # Bye
           else
             send_status( 'Error', "Unknown command '#{cmd['command']}'" )
           end
         end
+
+        def stop_auto( auto )
+          id= auto['task_id']
+          auto['worker_id'].each do |wid|
+            if @WORKERS[wid]
+              manage_worker( { 'worker_id' => wid, 'action' => 'QUIT'} )
+            end
+          end
+          @AUTO.delete(auto['task_id'])
+          autos_rem( @HOST, id )
+          send_status( 'Info', "Task #{id} is now in manual mode")
+        end
+
+        def status_auto( id, auto )
+          n= nvoid= nrun= 0
+          auto['worker_status']= []
+          auto['worker_id'].each do |id|
+            s= @WORKERS[id] ? @WORKERS[id]['status'] : 'VOID'
+            nvoid+= 1 if 'VOID'==s
+            nrun+= 1 if 'RUN'==s
+            n+= 1
+            auto['worker_status'] << s
+          end
+          auto['worker_run']= nrun
+          auto['worker_void']= nvoid
+          auto['worker_unknown']= n-nrun-nvoid
+          @AUTO[id]= auto
+          auto
+        end
+
+        def manage_auto( auto, status, action, n0 )
+          n= 0
+          auto['worker_status'].each_with_index do |s, i|
+            if s==status
+              cmd= auto.clone
+              cmd['worker_id']= auto['worker_id'][i]
+              if 'START'==action 
+                start_worker( cmd, cmd['rev_info'], true )
+              else
+                cmd['action']= action
+                manage_worker( cmd )
+              end
+              n+= 1
+            end
+            break if n==n0
+          end
+        end
+
+        def check_auto
+          @AUTO.keys.each do |id|
+            auto= @AUTO[id]
+            autos_add( @HOST, id, auto )
+            next unless auto['last_action']+auto['auto_delay'].to_i <= Time.now
+            auto= status_auto( id, @AUTO[id] )  # Compute the new status..
+            ql= get_queue_length( auto['queue'] )
+            ideal= [(ql.to_f / auto['auto_max_waiting_job_per_worker'].to_f).ceil, auto['auto_worker_min'].to_i].max
+            count= auto['worker_count'].to_i
+            case ideal <=> (count-auto['worker_void'])
+            when 0  # Do nothing
+            when 1 # Increase number of workers if possible
+              inc= [ideal-auto['worker_run'], auto['worker_void']].min
+              manage_auto( auto, 'VOID', 'START', inc ) if inc>0
+            when -1  # Decrease number of workers if possible
+              dec= [auto['worker_run']-ideal, auto['worker_run']].min
+              manage_auto( auto, 'RUN', 'QUIT', dec ) if dec>0
+            end
+          end
+        end
+
+        # Start auto session
+        def start_auto( cmd0, rev_info )
+          auto_def= { 'auto_max_waiting_job_per_worker' => 1,'auto_worker_min' => 0, 'auto_delay' => 15 }
+          cmd= auto_def.merge( cmd0 )
+          id= cmd['task_id']
+          if @AUTO[id]
+            send_status( 'Error', "Task #{id} is already running in auto mode")
+            return
+          end
+          send_status( 'Info', "Task #{id} is now in auto mode")
+          auto= cmd                       # Should be defined in cmd: task_id, worker_count, worker_id, queue, rails_env, exec
+          auto['rev_info']= rev_info
+          # Get status for the workers
+          auto['last_action']= Time.now - auto['auto_delay'].to_i
+          @AUTO[id]= auto       
+        end
         
         # Start a task
-        def start_worker( cmd, rev_info )
+        def start_worker( cmd, rev_info, auto=false )
           # Retrieving args
           path= rev_info['revision_path']
           log_path= rev_info['revision_log_path']
@@ -113,8 +206,9 @@ module Resque
           pid= spawn( env, exec, opt) # Start it!
           info= { 'pid' => pid, 'status' => 'RUN', 'environment' => env, 'options' => opt, 'revision_info' => rev_info }
           # Log snapshot
-          info['log_snapshot_period']= cmd['log_snapshot_period'] if cmd['log_snapshot_period']
-          info['log_snapshort_lines']= cmd['log_snapshot_lines'] if cmd['log_snapshot_lines']
+          info['log_snapshot_period']= cmd['log_snapshot_period'].to_i if cmd['log_snapshot_period']
+          info['log_snapshort_lines']= cmd['log_snapshot_lines'].to_i if cmd['log_snapshot_lines']
+          info['mode']= auto ? 'Auto' : 'Manual'
           @WORKERS[id]= info
           workers_add( @HOST, id, info )
           send_status( 'Info', "Starting worker #{id} (PID #{pid})" )
@@ -175,6 +269,12 @@ module Resque
             end            
           end
         end
+
+        def get_queue_length( qs )
+          ql= qs.split(",")
+          l= ql.include?("*") ? queue_list : ql
+          l.inject(0) { |a,e| a+queue_length(e) }
+        end
         
         def update_log_snapshot( id )
           ls= @WORKERS[id]['log_snapshot_period']

data/lib/resque-telework/redis.rb

@@ -7,6 +7,10 @@ module Resque
         "plugins:#{Resque::Plugins::Telework::Nickname}"
       end
 
+      def queue_prefix
+        "queue:"
+      end
+
       def redis_interface_key # String
         "#{key_prefix}:redisif"
       end
@@ -27,6 +31,10 @@ module Resque
         "#{key_prefix}:host:#{h}:workers"
       end
 
+      def autos_key( h ) # Hash
+        "#{key_prefix}:host:#{h}:autos"
+      end
+
       def tasks_key( h ) # Hash
         "#{key_prefix}:host:#{h}:tasks"
       end
@@ -74,14 +82,20 @@ module Resque
       # Clients (hosts) side
     
       def i_am_alive( info= {}, ttl=10 )
-        h= @HOST
         t= Time.now
         info= info.merge( { 'date' => t, 'version' => Resque::Plugins::Telework::Version } )
-        k= alive_key(h)
-        hosts_add(h)
+        k= alive_key(@HOST)
+        hosts_add(@HOST)
         Resque.redis.set(k, info.to_json )
         Resque.redis.expire(k, ttl)
-        Resque.redis.set(last_seen_key(h), t)
+        Resque.redis.set(last_seen_key(@HOST), t)
+      end
+
+      def i_am_dead
+        t= Time.now
+        hosts_add( @HOST )
+        Resque.redis.del( alive_key( @HOST ) )
+        Resque.redis.set(last_seen_key( @HOST ), t)        
       end
       
       def register_revision( h, rev, lim=9 )
@@ -129,12 +143,23 @@ module Resque
         Resque.redis.hset(k, id, info.to_json )
         Resque.redis.expire(k, ttl)
       end
+
+      def autos_add( h, id, info, ttl=10 )
+        k= autos_key(h)
+        Resque.redis.hset(k, id, info.to_json )
+        Resque.redis.expire(k, ttl)
+      end
       
       def workers_rem( h , id )
         k= workers_key(h)
         Resque.redis.hdel(k, id)
       end
 
+      def autos_rem( h , id )
+        k= autos_key(h)
+        Resque.redis.hdel(k, id)
+      end
+
       def tasks_add( h, id, info)
         k= tasks_key(h)
         Resque.redis.hset(k, id, info.to_json )
@@ -195,7 +220,8 @@ module Resque
       # This function must be idempotent
       def reconcile
         hosts.each do |h|
-          tasks(h).each do |id, info|
+          tasks(h).each do |tid, info|
+            auto= autos_by_id( h, tid )
             statuses= []
             pids= []
             tstatus= info['worker_status']              # Task status
@@ -217,17 +243,22 @@ module Resque
                 "Running"
               when "STOP"
                 "Stopped"
+              when "AUTO"
+                "Auto"
               else
                 "Unknown"
               end
               statuses << ws
               pids << worker['pid'] if worker
             end
-            ts= statuses.uniq * ","
-            if ts!=tstatus #&& (tstatus!="Starting" || wstatus!="STOP")
+            ts= if statuses.uniq.length==1 then statuses[0] else statuses * "," end
+            mode= auto ? 'Auto' : 'Manual'
+            if ts!=tstatus || info['mode']!=mode #&& (tstatus!="Starting" || wstatus!="STOP")
+              info['last_changed']= Time.now
               info['worker_status']= ts
               info['worker_pid']= pids
-              tasks_add( h, id, info )
+              info['mode']= mode
+              tasks_add( h, tid, info )
             end
           end
         end
@@ -236,11 +267,18 @@ module Resque
       def workers( h )
          Resque.redis.hgetall(workers_key(h)).collect { |id, info| [id,  ActiveSupport::JSON.decode(info)] }
       end
-      
-      def workers_by_id( h, id )
-        k= workers_key(h)
+
+      def get_by_id( k, id)
         info= Resque.redis.hget(k, id)
-        info ? ActiveSupport::JSON.decode(info) : nil
+        info ? ActiveSupport::JSON.decode(info) : nil        
+      end
+
+      def autos_by_id( h, id )
+        get_by_id( autos_key(h), id)
+      end
+
+      def workers_by_id( h, id )
+        get_by_id( workers_key(h), id)
       end
 
       def tasks( h )
@@ -248,15 +286,11 @@ module Resque
       end
       
       def tasks_by_id( h, id )
-        k= tasks_key(h)
-        info= Resque.redis.hget(k, id)
-        info ? ActiveSupport::JSON.decode(info) : nil
+        get_by_id( tasks_key(h), id)
       end      
       
       def logs_by_id( h, id )
-        k= logs_key(h)
-        info= Resque.redis.hget(k, id)
-        info ? ActiveSupport::JSON.decode(info) : nil      
+        get_by_id( logs_key(h), id)     
       end
       
       def unique_id
@@ -326,6 +360,14 @@ module Resque
         Resque.redis.keys("#{key_prefix}:*").length
       end
       
+      def queue_length( q )
+        Resque.redis.llen("#{queue_prefix}#{q}")
+      end
+
+      def queue_list
+        Resque.redis.smembers("queues")
+      end
+
       def fmt_date( t, rel=false ) # This is not redis-specific and should be moved to another class!
         begin
           if rel

data/lib/resque-telework/server/views/task.erb

@@ -0,0 +1,20 @@
+<h1>Task <%= @task_id %></h1>
+
+<% task= redis.tasks_by_id( @host, @task_id) %>
+
+<form style="margin:0px;float:center;" id="taskm" name="taskm" method="post" action="/resque/telework_mod_task/<%= @task_id %>" >
+<table>
+	<tr><th>Parameter</th><th>Value</th><th>Description</th>
+	<tr><td>Mode</td><td><%= "#{task['mode']}" %></td><td>Current task mode</td>
+    <tr><td>Environment</td><td><%= "#{task['rails_env']}" %></td><td>Rails environment</td>
+    <tr><td>Queues</td><td><%= "#{task['queue']}" %></td><td>Queues</td>    	
+    <tr><td>Worker count</td><td><input id="worker_count" name="worker_count" type="number" value="<%= "#{task['worker_count']}" %>" /></td><td>In manual mode, this is the the number of workers started when the task in on<br/>In auto mode, this is the maximum number of workers that can be started</td>   	
+	<tr><td>Exec</td><td><input id="exec" name="exec" size=60 type="text" value="<%= "#{task['exec']}" %>" /></td><td>Command to be executed to start a worker</td>
+	<tr><td>Log period</td><td><input id="log_snapshot_period" name="log_snapshot_period" type="number" value="<%= "#{task['log_snapshot_period']}" %>" /></td><td>Period in second between two worker log capture</td>
+	<tr><td>Log lines</td><td><input id="log_snapshot_lines" name="log_snapshot_lines" type="number" value="<%= "#{task['log_snapshot_lines']}" %>" /></td><td>Number of lines captured at the end of the worker log</td>
+	<tr><td>Auto delay</td><td><input id="auto_delay" name="auto_delay" type="number" value="<%= "#{task['auto_delay']}" %>" /></td><td>Reaction time in second for the auto mode</td>
+	<tr><td>Auto job per worker</td><td><input id="auto_max_waiting_job_per_worker" name="auto_max_waiting_job_per_worker" type="number" value="<%= "#{task['auto_max_waiting_job_per_worker']}" %>" /></td><td>Max number of waiting job for earch worker in auto mode</td>
+	<tr><td>Auto min worker</td><td><input id="auto_worker_min" name="auto_worker_min" type="number" value="<%= "#{task['auto_worker_min']}" %>" /></td><td>Min number of worker in auto mode</td>		
+</table>
+<input type="submit" value= "Save" />
+</form>

data/lib/resque-telework/server/views/telework.erb

@@ -35,7 +35,8 @@ function action(val) {
     var url = "";
     switch(params[0]) {
 	  case "start":
-	    url = "/start?task="+params[1]+"&host="+params[2]+"&rev="+document.getElementById("rev_filter_"+params[1]).value;
+	    url = "/start?task="+params[1]+"&host="+params[2]+"&rev="+document.getElementById("rev_filter_"+params[1]).value+
+	                                                      "&count="+document.getElementById("count_filter_"+params[1]).value;
 	    break;
 	  case "pause":
 	    url = "/pause?task="+params[1]+"&host="+params[2];
@@ -51,7 +52,14 @@ function action(val) {
 	    break;
 	  case "delete":
 	    url = "/delete?task="+params[1]+"&host="+params[2];
-	    break;	
+	    break;
+	  case "auto":
+	    url = "/start_auto?task="+params[1]+"&host="+params[2]+"&rev="+document.getElementById("rev_filter_"+params[1]).value+
+	                                                      "&count="+document.getElementById("count_filter_"+params[1]).value;
+	    break;
+	  case "autostop":
+	    url = "/stop_auto?kill=true&task="+params[1]+"&host="+params[2];
+	    break;	    
 	  default:
 	    return true;
 	}
@@ -87,7 +95,7 @@ $(document).ready(function() {
 	    <td><%= "#{redis.fmt_date(note['date'], true)}" %></td>
 	    <td><%= note['note'] %></td>
         <td>
-           <form id="deln" name="deln" method="post" action="/resque/telework_del_note/<%= id %>" ><input type="submit" value= "Delete" /></form>
+           <form style="margin:0px;float:center;" id="deln" name="deln" method="post" action="/resque/telework_del_note/<%= id %>" ><input type="submit" value= "Delete" /></form>
         </td>
 	</tr>
     <% end %>
@@ -141,6 +149,11 @@ $(document).ready(function() {
 <% end %>
 
 <h1>Hosts, Revisions, Tasks and Workers</h1>
+<form id="stopping_all" name="stopping_all" method="post" action="/resque/telework/stop_all">
+	<%= generic_filter("mode_filter", "m", ["Stop", "Kill"]) %>
+	 all workers on host <%= generic_filter("host_filter", "h", redis.hosts <<  "[All hosts]") %>
+	<span class="stopping_all_submit"><input type="submit" value="Go"/></span>
+</form></br>
 <table>
     <tr>
       <th>Host</th>
@@ -156,11 +169,9 @@ $(document).ready(function() {
 	      <% end %>
 	  </td>
       <% if 'Alive'==status %>
-        <td> <table><tr><td><%= "Alive (v#{info['version']})" %></td></tr>
-	                <tr><td>
-	                	<form id="stopd" name="stopd" method="post" action="/resque/telework_stopitd/<%= host %>" ><input type="submit" value= <%="\"Stop\""%> /></form>
-	                </td></tr>
-	         </table>
+        <td> <center><%= "Alive (v#{info['version']})" %><br/><br/>
+             <form style="margin:0px;float:center;" id="stopd" name="stopd" method="post" action="/resque/telework_stopitd/<%= host %>" align="middle"><input type="submit" value= <%="\"Stop\""%> /></form></center>
+
         </td>
       <% else %>
         <td><%= status%></td>
@@ -191,14 +202,25 @@ $(document).ready(function() {
 		     <th>Worker(s)</th>
 		<tr>
 	  <% for id, info in redis.tasks(host) %>
+	  	 <% auto= info['mode']=='Auto' %>
 	     <% status= info['worker_status'] %>
 	     <% status= 'Stopped' if status.blank? %>
 	     <% running= status!='Stopped' %>
 	     <% paused=  status=='Paused' %>
-             <td><%= "<a hreftbd=\"/resque/telework/task/#{host}/#{id}\">#{id}</a>" %>
-	    </td><td><%= "#{status}" %>
-	    </td><td><%= "#{info['queue'].gsub(/,/,'<br/>')}" %>
-	    </td><td><%= "#{info['worker_count']}" %>
+             <td><%= "<a href=\"/resque/telework/task/#{host}/#{id}\">#{id}</a>" %>
+	    </td><td><center><%= auto ? "<b>Auto</b><br/><br/>" : "" %><%= "#{status.gsub(/,/,'<br/>')}" %> </center>
+	    </td><td><% for q in info['queue'].split(",") %>
+	    <% if q=="*" %>*
+	    <% else %> <% ql= redis.queue_length(q) %>
+	    <% if ql==0 %><font color="grey"><span title="Queue is empty"><%= q %></span></font>
+	    <% else %><span title="<%= "#{ql} pending jobs" %>" ><%= q %></span>
+	    <% end %><% end %><br/><% end %>
+	    </td><td><% if running%><%= "#{info['worker_count']}" %>
+                    <% else %><select id= <%= "\"count_filter_#{id}\"" %> name="c"/>
+                               <% for co in (1..8).to_a %>
+                                 <option <%= co==info['worker_count'].to_i ? 'selected="selected"' : '' %>value=<%= "#{co}"%> ><%= co %></option>
+                             <% end %>
+                    </select><% end %>
 		</td><td><% if running %><a href= <%= "/resque/telework/revision/#{info['revision']}" %> ><%= "#{info['revision_small']}" %></a>
 			     <% else %> <select id= <%= "\"rev_filter_#{id}\"" %> name="r"/>
 			                   <% for rev in redis.revisions(host) %>
@@ -209,18 +231,20 @@ $(document).ready(function() {
         </td><td>
 	        <select id="action" name="a" onchange="javascript: action(this.options[this.selectedIndex].value);" >
 	        	<option select="selected" value="select">Select</option>
-	            <option <%= 'disabled="disabled"'     if running %> value=<%= "\"start,#{id},#{host}\"" %> >Start worker</option>
-	            <option <%= 'disabled="disabled"' unless running %> value=<%= "\"stop,#{id},#{host}\"" %> >Stop worker</option>
-	            <option <%= 'disabled="disabled"' unless running %> value=<%= "\"kill,#{id},#{host}\"" %> >Kill worker</option>
-	            <option <%= 'disabled="disabled"'     if (!running || paused) %> value=<%= "\"pause,#{id},#{host}\"" %> >Pause worker</option>
-	            <option <%= 'disabled="disabled"' unless paused %> value=<%= "\"cont,#{id},#{host}\"" %> >Resume worker</option>
-	            <option <%= 'disabled="disabled"'     if running %> value=<%= "\"delete,#{id},#{host}\"" %> >Delete</option>
+	            <option <%= 'disabled="disabled"'     if running || auto %> value=<%= "\"start,#{id},#{host}\"" %> >Start worker</option>
+	            <option <%= 'disabled="disabled"' unless running && !auto %> value=<%= "\"stop,#{id},#{host}\"" %> >Stop worker</option>
+	            <option <%= 'disabled="disabled"' unless running && !auto %> value=<%= "\"kill,#{id},#{host}\"" %> >Kill worker</option>
+	            <option <%= 'disabled="disabled"'     if (!running || paused || auto) %> value=<%= "\"pause,#{id},#{host}\"" %> >Pause worker</option>
+	            <option <%= 'disabled="disabled"' unless paused && !auto %> value=<%= "\"cont,#{id},#{host}\"" %> >Resume worker</option>
+	            <option <%= 'disabled="disabled"'     if auto || running %> value=<%= "\"auto,#{id},#{host}\"" %> >Start auto</option>
+	            <option <%= 'disabled="disabled"'     unless auto %> value=<%= "\"autostop,#{id},#{host}\"" %> >Stop auto</option>	    
+	            <option <%= 'disabled="disabled"'     if running || auto %> value=<%= "\"delete,#{id},#{host}\"" %> >Delete</option>        
 	        </select>
 	     </td><td align="center"><p align="center"><font size="1">
-		    <% info['worker_id'].zip(info['worker_pid']) do |id,pid| %>
+		    <% info['worker_id'].zip(info['worker_pid']) do |id,pid| %><% unless pid.blank? %>
 		      <%= "<a href=\"/resque/telework/worker/#{host}/#{id}\">Worker #{id}</a>" if running %><br/>
 	          <font size=1><% l= "#{host}:#{pid}" %><%= "<a href=\"/resque/workers/#{l}:#{info['queue']}\">#{l}</a>" if running %></font><br/>
-	        <% end %>
+	        <% end %><% end %>
 	        </font></p>
 	   </td></tr>
 	  <%end%>

data/lib/resque-telework/server.rb

@@ -38,7 +38,11 @@ module Resque
               end
               html += "</select>"
             end
-            
+            def task_default
+              { 'auto_max_waiting_job_per_worker' => 1,'auto_worker_min' => 0, 'auto_delay' => 15,
+                'log_snapshot_period' => 30, 'log_snapshot_lines' => 40, 'exec' => "bundle exec rake resque:work --trace"
+              }
+            end
           end
 
           app.get "/#{appn.downcase}" do
@@ -46,8 +50,12 @@ module Resque
           end
           
           app.get "/#{appn.downcase}/Overview" do
-            @status_messages= 100
             @refresh= 10
+            if params[:refresh]
+              @refresh= params[:refresh].to_i
+              @refresh= nil if @refresh==0
+            end
+            @status_messages= 100
             @scheduling= nil
             my_show appn.downcase
           end
@@ -72,6 +80,12 @@ module Resque
             @host= params[:host]
             my_show 'worker' 
           end
+
+          app.get "/#{appn.downcase}/task/:host/:task_id" do
+            @task_id= params[:task_id]
+            @host= params[:host]
+            my_show 'task' 
+          end
           
           app.get "/#{appn.downcase}/config" do
             content_type :json
@@ -108,6 +122,24 @@ module Resque
             my_show 'stopit'
           end
 
+          app.post "/#{appn.downcase}_mod_task/:task" do
+            @task_id= params[:task]
+            @host= nil
+            redis.hosts.each do |h|
+              redis.tasks(h).each do |id, info|
+                @host= h if id==@task_id # TODO: break nested loops
+              end
+            end
+            @task= redis.tasks_by_id( @host, @task_id )
+            all= ['log_snapshot_period', 'log_snapshot_lines', 'exec', 'worker_count',
+                  'auto_delay', 'auto_max_waiting_job_per_worker', 'auto_worker_min' ]
+            all.each do |a|
+              @task[a]= params[a]
+            end
+            redis.tasks_add( @host , @task_id, @task )
+            redirect "/resque/#{appn.downcase}"
+          end
+
           app.post "/#{appn.downcase}_killit/:worker" do
             @worker= params[:worker]
             @host= nil
@@ -145,12 +177,10 @@ module Resque
             @env= params[:e]
             @q= @qmanual.blank? ? @queue : @qmanual
             id= redis.unique_id.to_s
-            redis.tasks_add( @host , id, { 'task_id' => id, 'worker_count' => @count,
-                                           'rails_env' => @env, 'queue' => @q,
-                                           'exec' => "bundle exec rake resque:work --trace",
-                                           'worker_id' => [], 'worker_status' => 'Stopped',
-                                           'log_snapshot_period' => 30,
-                                           'log_snapshot_lines' => 40 } )
+            t= task_default
+            redis.tasks_add( @host , id, t.merge( { 'task_id' => id, 'worker_count' => @count,
+                                                    'rails_env' => @env, 'queue' => @q,
+                                                    'worker_id' => [], 'worker_status' => 'Stopped'} ) )
             redirect "/resque/#{appn.downcase}"          
           end
           
@@ -167,7 +197,7 @@ module Resque
             @host= params[:host]
             @rev= params[:rev].split(',')
             @task= redis.tasks_by_id(@host, @task_id)
-            count= @task['worker_count'] || 1
+            count= params[:count]
             id= []
             for i in 1..count.to_i do
               w= @task
@@ -181,10 +211,46 @@ module Resque
               redis.cmds_push( @host, w )
             end
             @task['worker_id']= id
+            @task['worker_count']= count
             redis.tasks_add( @host, @task_id, @task )
             redirect "/resque/#{appn.downcase}"
           end
 
+          app.post "/#{appn.downcase}/start_auto" do
+            @task_id= params[:task]
+            @host= params[:host]
+            @rev= params[:rev].split(',')
+            @task= redis.tasks_by_id(@host, @task_id)
+            count= params[:count]        
+            wid= []
+            for i in 1..count.to_i do
+              wid << redis.unique_id.to_s
+              #redis.cmds_push( @host, w )
+            end
+            @task['worker_id']= wid
+            @task['worker_count']= count
+            @task['mode']= 'auto'
+            cmd= @task
+            cmd['task_id']= @task_id
+            cmd['revision']= @rev[0]
+            cmd['revision_small']= @rev[1]
+            cmd['command']= 'start_auto'
+            redis.cmds_push( @host, cmd )
+            redis.tasks_add( @host, @task_id, @task )
+            redirect "/resque/#{appn.downcase}"
+          end
+
+          app.post "/#{appn.downcase}/stop_auto" do
+            @task_id= params[:task]
+            @host= params[:host]
+            @task= redis.tasks_by_id(@host, @task_id)
+            cmd= @task
+            cmd['command']= 'stop_auto' 
+            redis.cmds_push( @host, cmd )
+            redirect "/resque/#{appn.downcase}"                
+          end
+
+
           app.post "/#{appn.downcase}/pause" do
             @task_id= params[:task]
             @host= params[:host]

metadata

@@ -4,9 +4,9 @@ version: !ruby/object:Gem::Version
   prerelease: false
   segments: 
   - 0
-  - 2
-  - 1
-  version: 0.2.1
+  - 3
+  - 0
+  version: 0.3.0
 platform: ruby
 authors: 
 - Gilles Pirio
@@ -14,7 +14,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2013-02-04 00:00:00 -08:00
+date: 2013-02-22 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -68,6 +68,7 @@ files:
 - lib/resque-telework/server/views/misc.erb
 - lib/resque-telework/server/views/revision.erb
 - lib/resque-telework/server/views/stopit.erb
+- lib/resque-telework/server/views/task.erb
 - lib/resque-telework/server/views/telework.erb
 - lib/resque-telework/server/views/worker.erb
 - lib/tasks/telework.rake