naf 2.0.4 → 2.1.0

data/README.rdoc

@@ -1,7 +1,6 @@
 = Network Application Framework (Naf)
 
 Distributed Rails-based fault tolerant script scheduling framework, providing:
-
 * Quick prototyping
 * Logging
 * Scheduling
@@ -10,14 +9,39 @@ Distributed Rails-based fault tolerant script scheduling framework, providing:
 * Alarming
 
 With a management console, providing:
-
 * Machine Management
 * Live Queue Management
 * Process Management
 * Script Management
 * Schedule Management
 
-=== Please find up-to-date information on Naf here: {Naf Wiki}[http://www.github.com/fiksu/naf/wiki]
+=== Built on top of Af
+Naf is built on top of Af, which provides a supporting infrastructure through its Application class.
+* Command line option parsing sets instance and class variables. Options can either have default values, can provide their own value parser or select from a number of pre-defined parsers.
+* Logging is supported and done via log4r.
+* Tight integration with Postgres allows administrators to easily find which scripts are executing db queries.
+* Application components, such as ActiveRecord models, can add loggers and command line options.
+
+=== What is N/Af
+Naf is simply a framework that we use to schedule, run, and monitor Af programs across a distributed network. It provides:
+* Machine node redundancy (i.e. if a machine runner goes down, the other machine runners in the naf cluster can still run jobs).
+* Script run dependencies allow applications to have prerequisites in order to prevent one application from running before another.
+* Run restrictions, which prevent competing applications from running at the same time (i.e. if you have a scraper which is only allowed to have a single login active at a time, you can ensure it is only run when no other versions of the script is currently executing).
+* A nice UI, where you can control all these features.
+
+=== Runners
+Runners will queue an application if the application schedule is ready and it will check the queue every 30 seconds to see if there are available jobs to be run. Administrators can be alarmed when scripts fail to run, and when runners are down. In order to ensure that applications run on specific machines, use affinities.
 
+=== Affinities
+Machines have affinity slots (which can be required) and applications have affinity tabs. For an application to be able to run on a machine, the machine’s affinity slots need to match all of the application's affinity tabs. And for a machine to be able to run an application, the application’s affinity tabs need to match all of the machine's required affinity slots.
+
+Machines have an allocate-able number of slots of a specific affinity. Applications will allocate a number of these slots at run time, otherwise they will stay queued. This is a way to achieve load balancing, and prevent machines from running out of resources.
+
+=== Tags
+Scripts have tags to describe the stage/progress of the process. They can be system or custom tags. All tags are visible by the user, so set tags at different stages of your script. This is a way of knowing what is happening in the script without having to take a look at the logs.
+
+
+=== Please find up-to-date information on Naf here: {Naf Wiki}[http://www.github.com/fiksu/naf/wiki]
 === Instructions on upgrading to Naf v1.1.4: {Instructions}[https://github.com/fiksu/naf/wiki/Upgrading-to-Naf-v1.1.4]
-=== Instructions on upgrading to Naf v2.0.0: {Instructions}[https://github.com/fiksu/naf/wiki/Upgrading-to-Naf-v2.0]
+=== Instructions on upgrading to Naf v2.0: {Instructions}[https://github.com/fiksu/naf/wiki/Upgrading-to-Naf-v2.0]
+=== Instructions on upgrading to Naf v2.1: {Instructions}[https://github.com/fiksu/naf/wiki/Upgrading-to-Naf-v2.1]

data/RELEASE_NOTES.rdoc

@@ -1,5 +1,13 @@
 = Release Notes
 
+=== Version 2.1.0
+
+Featurs/Changes:
+* Run interval styles determines the type schedule interval (daily, hourly, run, constant)
+* Application can have many schedules
+* Application run group quantum specifies the number of jobs queued during a schedule
+* UI changes
+
 === Version 2.0.0
 
 Featurs/Changes:

data/app/assets/javascripts/dataTablesTemplates/application_schedules.js

@@ -0,0 +1,72 @@
+// When document is ready
+jQuery(document).ready(function() {
+
+  // Prepare for setup the datatable.
+  var dataTableOptions = {
+    "sAjaxSource": sAjaxSource,
+    "fnInitComplete" : function() {
+      initPageSelect();
+    },
+    "bAutoWidth": false,
+    "bSort": true,
+    "aoColumnDefs": [
+      { "bSortable": false, "aTargets": [9, 10] },  // turn off sorting
+      { "sClass": "center", "aTargets": [6, 7, 8] }
+    ],
+    "aoColumns": [
+        { "sWidth": "2%"},      // Id
+        { "sWidth": "15%"},     // Application
+        { "sWidth": "15%"},     // Run Group Name
+        { "sWidth": "9%"},      // Run Group Restriction
+        { "sWidth": "10%"},     // Run Interval Style
+        { "sWidth": "4%"},      // Run Interval
+        { "sWidth": "5%"},      // Run Group Quantum
+        { "sWidth": "5%"},      // Run Group Limit
+        { "sWidth": "5%"},      // Enqueue Backlogs
+        { "sWidth": "10%"},     // Affinities
+        { "sWidth": "10%"}      // Prerequesites
+    ],
+    "fnServerData": function ( sSource, aoData, fnCallback ) {
+      _.each(jQuery('.datatable_variable').serializeArray(), function(dv) { aoData.push(dv); });
+      jQuery.getJSON( sSource, aoData, function (json) {
+        fnCallback(json);
+        initPaging();
+        addTitles();
+      });
+    },
+    "fnRowCallback": function( nRow, aData, iDisplayIndex, iDisplayIndexFull ) {
+      colorizationDisabled(nRow, aData);
+      return nRow;
+    },
+    "sDom": "Rlfrtip"
+  };
+
+  // Setup the datatable
+  jQuery('#datatable').addDataTable(dataTableOptions);
+
+  jQuery(document).delegate('.enqueue', "click", function(){
+    var answer = confirm("Adding application as a job on the queue?");
+    if (!answer) {
+      return false;
+    }
+    jQuery.post(postSource, { "historical_job[application_id]": jQuery(this).attr('id') }, function(data) {
+      if (data.success) {
+        jQuery("<p id='notice'>Congratulations, a Job " + data.title + " was added!</p>").
+                appendTo('#flash_message').slideDown().delay(5000).slideUp();
+        jQuery('#datatable').dataTable().fnDraw();
+      }
+      else {
+        jQuery("<div class='error'>Sorry, \'" +  data.title + "\' cannot add a Job to the queue right now!</div>").
+                appendTo('#flash_message').slideDown().delay(5000).slideUp();
+        jQuery('#datatable').dataTable().fnDraw();
+      }
+    });
+  });
+});
+
+function colorizationDisabled(nRow, aData) {
+  if (aData[11] == false) {
+    jQuery(nRow).addClass('deleted_or_hidden');
+  }
+}
+

data/app/assets/javascripts/dataTablesTemplates/applications.js

@@ -9,26 +9,17 @@ jQuery(document).ready(function() {
     },
     "bAutoWidth": false,
     "aoColumnDefs": [
-      { "bVisible": false, "aTargets": [ 14, 15 ] },    // turn off visibility
-      { "sClass": "center", "aTargets": [ 6, 7, 8, 9, 12, 13 ] }
+      { "sClass": "center", "aTargets": [ 6 ] }
     ],
     "aoColumns": [
-        { "sWidth": "2%"},      // Id
-        { "sWidth": "12%"},     // Title
-        { "sWidth": "7%"},      // Short Name
-        { "sWidth": "6%"},      // Script Type Name
-        null,                   // Application Run Group Name
-        { "sWidth": "9%"},      // Application Run Group Restriction Name
-        { "sWidth": "7%"},      // Application Run Group Limit
-        { "sWidth": "4%"},      // Enabled
-        { "sWidth": "4%"},      // Enqueue Backlogs
-        { "sWidth": "8%"},      // Run Time
-        { "sWidth": "10%"},     // Last Queued At
-        { "sWidth": "8%"},      // Affinities
-        { "sWidth": "8%"},      // Prerequisites
-        { "sWidth": "4%"},      // Actions
-        null,
-        null
+        { "sWidth": "3%"},      // Id
+        null,                   // Title
+        { "sWidth": "15%"},     // Short Name
+        { "sWidth": "10%"},     // Script Type Name
+        { "sWidth": "15%"},     // Application Schedules
+        { "sWidth": "15%"},     // Last Queued At
+        { "sWidth": "5%"}       // Actions
+
     ],
     "fnServerData": function ( sSource, aoData, fnCallback ) {
       _.each(jQuery('.datatable_variable').serializeArray(), function(dv) { aoData.push(dv); });
@@ -77,13 +68,13 @@ function addLinkToApplication(nRow, aData) {
 }
 
 function colorizationDeletedOrHidden(nRow, aData) {
-  if (aData[11] == 'true' || aData[12] == 'false' || aData[13] == '') {
+  if (aData[6] == 'true') {
     jQuery(nRow).addClass('deleted_or_hidden');
   }
 }
 
 function checkTimeFormat(nRow, aData) {
-  var l_q_a_array = jQuery(aData[10]).text().split(',');
+  var l_q_a_array = jQuery(aData[5]).text().split(',');
   var last_queued_at;
   if(jQuery('#time_format').val() == 'lexically') {
     last_queued_at = l_q_a_array[0];
@@ -91,5 +82,5 @@ function checkTimeFormat(nRow, aData) {
     last_queued_at = l_q_a_array[1];
   }
 
-  jQuery('td:nth-child(11)', nRow).empty().append(jQuery(aData[10]).text(last_queued_at));
+  jQuery('td:nth-child(6)', nRow).empty().append(jQuery(aData[5]).text(last_queued_at));
 }

data/app/controllers/naf/application_schedule_affinity_tabs_controller.rb

@@ -33,16 +33,14 @@ module Naf
     end
 
     def create
-      route_params = { application_id: params[:application_id],
-                       application_schedule_id: params[:application_schedule_id] }
+      route_params = { application_schedule_id: params[:application_schedule_id] }
       @application_schedule = Naf::ApplicationSchedule.find(params[:application_schedule_id])
-      @application = Naf::Application.find(params[:application_id])
       @tab = Naf::ApplicationScheduleAffinityTab.new(params[:application_schedule_affinity_tab])
       if @tab.save
-        redirect_to(naf.application_application_schedule_application_schedule_affinity_tab_path(@application, @application_schedule, @tab),
+        redirect_to(naf.application_schedule_application_schedule_affinity_tab_path(@application_schedule, @tab),
                     notice: "Application Schedule Affinity Tab '#{@tab.affinity_name}' was successfully created.")
       else
-        render route_params.merge(action: "new")
+        render route_params.merge(action: 'new')
       end
     end
 
@@ -55,12 +53,11 @@ module Naf
     def update
       @tab = Naf::ApplicationScheduleAffinityTab.find(params[:id])
       @application_schedule = Naf::ApplicationSchedule.find(params[:application_schedule_id])
-      @application = Naf::Application.find(params[:application_id])
       if @tab.update_attributes(params[:application_schedule_affinity_tab])
-        redirect_to(naf.application_application_schedule_application_schedule_affinity_tab_path(@application, @application_schedule, @tab),
+        redirect_to(naf.application_schedule_application_schedule_affinity_tab_path(@application_schedule, @tab),
                     notice: "Application Schedule Affinity Tab '#{@tab.affinity_name}' was successfully updated.")
       else
-        render action: "edit", application_schedule_id: @application_schedule.id, application_id: @application.id
+        render action: :edit, application_schedule_id: @application_schedule.id
       end
     end
 

data/app/controllers/naf/application_schedules_controller.rb

@@ -0,0 +1,98 @@
+module Naf
+  class ApplicationSchedulesController < Naf::ApplicationController
+
+    def index
+      respond_to do |format|
+        format.html
+        format.json do
+          application_schedules = []
+          application_schedule = []
+          @total_records = Naf::ApplicationSchedule.count(:all)
+          Logical::Naf::ApplicationSchedule.search(params).map(&:to_hash).map do |hash|
+            hash.map do |key, value|
+              value = '' if value.nil?
+              application_schedule << value
+            end
+            application_schedules << application_schedule
+            application_schedule = []
+          end
+
+          @total_display_records = application_schedules.count
+          @application_schedules = application_schedules.paginate(page: @page, per_page: @rows_per_page)
+
+          render layout: 'naf/layouts/jquery_datatables'
+        end
+      end
+    end
+
+    def show
+      schedule = ::Naf::ApplicationSchedule.find(params[:id])
+      @application_schedule = ::Logical::Naf::ApplicationSchedule.new(schedule)
+    end
+
+    def new
+      @application_schedule = Naf::ApplicationSchedule.new(application_id: params[:application_id])
+    end
+
+    def create
+      @application_schedule = Naf::ApplicationSchedule.new(params[:application_schedule])
+      check_application_run_group_name
+      set_application_run_group_name
+      if @application_schedule.save
+        redirect_to(@application_schedule.application, notice: "Application Schedule was successfully created.")
+      else
+        render action: :new
+      end
+    end
+
+    def edit
+      @application_schedule = ::Naf::ApplicationSchedule.find(params[:id])
+      check_application_run_group_name
+      if @application_schedule.application_schedule_prerequisites.blank?
+        @application_schedule.application_schedule_prerequisites.build
+      else
+        @show_prerequisite = true
+      end
+    end
+
+    def update
+      @application_schedule = ::Naf::ApplicationSchedule.find(params[:id])
+      set_application_run_group_name
+      if @application_schedule.update_attributes(params[:application_schedule])
+        redirect_to(@application_schedule, notice: "Application Schedule was successfully updated.")
+      else
+        render action: :edit
+      end
+    end
+
+    def destroy
+      @application_schedule = ::Naf::ApplicationSchedule.find(params[:id])
+      @application_schedule.destroy
+      flash[:notice] = 'Application Schedule was successfully deleted.'
+      redirect_to action: :index
+    end
+
+    def check_application_run_group_name
+      case @application_schedule.application_run_group_name
+        when @application_schedule.application.command
+          @run_group_name_type = 'command'
+        when nil, ''
+          @run_group_name_type = 'not set'
+        else
+          @run_group_name_type = 'custom'
+      end
+    end
+
+    def set_application_run_group_name
+      run_group_name_type = params[:run_group_name_type]
+      case run_group_name_type
+        when 'command'
+          params[:application_schedule][:application_run_group_name] =
+            ::Naf::Application.find_by_id(params[:application_schedule][:application_id]).command
+        when 'not set'
+          params[:application_schedule][:application_run_group_name] = nil
+      end
+    end
+
+  end
+end

data/app/controllers/naf/applications_controller.rb

@@ -1,7 +1,6 @@
 module Naf
   class ApplicationsController < Naf::ApplicationController
 
-    before_filter :set_cols_and_attributes
     before_filter :set_rows_per_page
 
     def index
@@ -12,8 +11,7 @@ module Naf
 
           applications = []
           application = []
-          params[:search][:visible] = params[:search][:visible] ? false : true
-          params[:search][:deleted] = params[:search][:deleted] ? false : "false"
+          params[:search][:deleted] = params[:search][:deleted] ? false : 'false'
           @total_records = Naf::Application.count(:all)
           Logical::Naf::Application.search(params[:search]).map(&:to_hash).map do |hash|
             hash.map do |key, value|
@@ -21,7 +19,7 @@ module Naf
               application << value
             end
             applications << application
-            application =[]
+            application = []
           end
           @total_display_records = applications.count
           @applications = applications.paginate(page: @page, per_page: @rows_per_page)
@@ -38,119 +36,29 @@ module Naf
 
     def new
       @application = Naf::Application.new
-      app_schedule = @application.build_application_schedule
-      app_schedule.application_schedule_prerequisites.build
     end
 
     def create
       @application = Naf::Application.new(params[:application])
-      check_application_run_group_name
-      set_application_run_group_name
       if @application.save
-        app_schedule = @application.application_schedule
-        @application.update_attributes(params[:application])
-        if app_schedule.present?
-          prerequisites =
-          app_schedule.prerequisites.map do |prerequisite|
-            prerequisite.title
-          end.join(', ')
-        end
-        redirect_to(@application,
-                    notice: "Application #{@application.title} was successfully created. #{'Prerequisites: ' + prerequisites if app_schedule.try(:prerequisites).try(:present?) }")
+        redirect_to(@application, notice: "Application #{@application.title} was successfully created.")
       else
-        set_shown_schedule_and_prerequisite
-        @application.build_application_schedule unless params[:application].try(:[], :application_schedule_attributes).try(:[], :_destroy) == "0"
-        render action: "new"
+        render action: :new
       end
     end
 
     def edit
       @application = Naf::Application.find(params[:id])
-      check_application_run_group_name
-      app_schedule = @application.application_schedule
-      if app_schedule.blank?
-        build_app_schedule = @application.build_application_schedule
-        build_app_schedule.application_schedule_prerequisites.build
-      else
-        @show_app_schedule = true
-        if app_schedule.application_schedule_prerequisites.blank?
-          app_schedule.application_schedule_prerequisites.build
-        else
-          @show_prerequisite = true
-        end
-      end
     end
 
     def update
       @application = Naf::Application.find(params[:id])
-      set_application_run_group_name
       if @application.update_attributes(params[:application])
-        app_schedule = @application.application_schedule
-        if app_schedule.present?
-          prerequisites =
-          app_schedule.prerequisites.map do |prerequisite|
-            prerequisite.title
-          end.join(', ')
-        end
-        redirect_to(@application,
-                    notice: "Application #{@application.title} was successfully updated. #{'Prerequisites: ' + prerequisites if app_schedule.try(:prerequisites).try(:present?) }")
+        redirect_to(@application, notice: "Application #{@application.title} was successfully updated.")
       else
-        set_shown_schedule_and_prerequisite
-        render action: "edit"
-      end
-    end
-
-    private
-
-    def set_cols_and_attributes
-      more_attributes = [:script_type_name,
-                         :application_run_group_name,
-                         :application_run_group_restriction_name,
-                         :run_interval,
-                         :run_start_minute,
-                         :priority,
-                         :application_run_group_limit,
-                         :visible,
-                         :enabled,
-                         :enqueue_backlogs ]
-      @attributes = Naf::Application.attribute_names.map(&:to_sym) | more_attributes
-      @cols = Logical::Naf::Application::COLUMNS
-    end
-
-    def set_application_run_group_name
-      @run_group_name_type = params[:run_group_name_type]
-      case @run_group_name_type
-        when "command"
-          params[:application][:application_schedule_attributes][:application_run_group_name] = params[:application][:command]
-        when "not set"
-          params[:application][:application_schedule_attributes][:application_run_group_name] = nil
-      end
-    end
-
-    def check_application_run_group_name
-      case @application.application_schedule.try(:application_run_group_name)
-        when @application.command
-          @run_group_name_type = "command"
-        when nil, ''
-          @run_group_name_type = "not set"
-        else
-          @run_group_name_type = "custom"
-      end
-    end
-
-    def set_shown_schedule_and_prerequisite
-      if params[:application].try(:[], :application_schedule_attributes).try(:[], :_destroy) == "0"
-        @show_app_schedule = true
-        unless params[:application].try(:[], :application_schedule_attributes).
-               try(:[], :application_schedule_prerequisites_attributes).
-               try(:select) do |key, value|
-                  value.try(:[], :_destroy) == "false"
-               end.blank?
-          @show_prerequisite = true
-        end
+        render action: :edit
       end
     end
 
   end
-
 end