rakish 0.9.01.beta

data/doc/RakishOverview.html

@@ -0,0 +1,8 @@
+== Package Overview
+
+Rakish is a build system package extending the capability of the ::Rake depedency based task management system.
+
+Rakish extends the capabilitiues of ::Rake with build module configuration and management features and a host of useful 
+utilites for creating tasks and a facile ability for writing and including custom modules for building different types of output.
+
+Several types of modules are included here in the base ::Rakish package for building Java, Jar, Zip, and Apache Tomact War file outputs.

data/doc/SimpleJavaSamples.txt

@@ -0,0 +1,448 @@
+
+==Java Project file Samples
+
+As I haven't written the real "manual" other than the rdocs on the ruby files so I figure this will help
+This is actual production code for me.
+
+Hopefully easy enough to sus out - I am too close to it now to judge :)
+And yes ruby people will hate me 'cause I am am a CamelCasingPerson and not an underbar_word_separartor_person
+and I tend to use it for all "dot delimited" languanges
+
+The JavaProjectModule does export some standard targets :compile, :javadoc, :dist, :libs with
+
+   java.addLibraryTargets(...)
+
+===Global Configuration file used by below samples basically from my current project
+
+    myDir = File.dirname(__FILE__);
+    require "rakish"
+    require "rakish/CppProjects.rb";
+    require "rakish/JavaProjects.rb";
+    require "rakish/IntellijConfig.rb";
+    require "#{myDir}/OMS/build-scripts/rakish/ServeletProject.rb"; # a custom module for this project
+
+    module Rakish  # not needed just to avoid having to use full paths for module names etc.
+
+
+        Rakish.Configuration :include=>[ IntellijConfig, JavaProjectConfig] do |cfg|
+
+            # set up global target dependencies.
+
+            task :autogen
+            task :compile =>[:autogen]
+            task :libs =>[:compile]
+            task :dist =>[:libs]
+            task :deploy=>[:dist] # for tomcat servlets and generating docs etc.
+            task :undeploy=>[:libs] # tomcat
+            task :cleanRebuild =>[:reallyClean, :dist]
+
+            myDir = File.dirname(File.expand_path(__FILE__));
+
+            isWindows = Rakish.HostIsWindows_;
+
+            log = Rakish.log;
+
+            # cfg.verbose = true; # default is false
+
+            cfg.orlandoRoot = File.expand_path("#{myDir}");
+
+            if(cfg.intellij) # we are running this from an intellij UI
+                cfg.buildDir = cfg.intellij.outputPath;
+            else
+                cfg.buildDir = "#{cfg.orlandoRoot}/build";
+            end
+
+            cfg.resourceDir = "#{cfg.buildDir}/production/.orlando";
+
+            # Java configuration items.
+
+            cfg.java_home = ENV['JAVA_HOME'];
+            cfg.ant_home = ENV['ANT_HOME'] || '/usr/share/ant';
+            # cfg.catalina_home = ENV['CATALINA_HOME'] || '/usr/share/tomcat7';
+            java = cfg.java;
+            java.addJarSearchPath("#{myDir}/third-party-jars");
+            java.addClassPaths( 'log4j-1.2.17.jar' ); # everything will inherit this
+
+            # tomcat deployment options
+
+            tomcatConfig = BuildConfig.new
+            tomcatConfig.enableNewFields do |cfg|
+                cfg.managerURL = "http://localhost:8080/manager/text";
+                cfg.managerUsername = "for me only";
+                cfg.managerPassword = "for me only";
+            end
+
+            # deployment configurations for the three categories of apps
+            cfg.omsConfig = BuildConfig.new(tomcatConfig) do |cfg|
+                cfg.managerURL = "http://localhost:8080/manager/text"
+            end
+
+            cfg.routerConfig = BuildConfig.new(tomcatConfig) do |cfg|
+                cfg.managerURL = "http://localhost:8081/manager/text"
+            end
+
+            (cfg.servicesConfig = BuildConfig.new(tomcatConfig)).enableNewFields do |cfg|;
+                cfg.managerURL = "http://localhost:8082/manager/text";
+            end
+        end
+    end
+
+===Root project file that has all the libraries in it
+
+All this is is the dependencies and the reallyClean target
+
+    myDir = File.dirname(__FILE__);
+    require "#{myDir}/build-options.rb";
+    require "rakish/RakishProject.rb";
+
+    Rakish.Project(
+        :name=>'libs',
+        :dependsUpon=> [
+            'artd-util',
+            'artd-vecmath',
+            'artd-net',
+            'artd-bml-generator',
+            'artd-bml-generator/artd-bmlgen-test',
+            'artd-bml-base',
+            'artd-bml-interpreter',
+            'artd-bml-net',
+            'artd-bml-servlet',
+            'artd-uiscene',
+        ]
+    ) do
+
+        task :reallyClean do |t|
+            log.info("deleting #{buildDir()}");
+            FileUtils.rm_rf("#{buildDir()}");
+        end
+
+        export :reallyClean
+
+    end
+
+===Project file for a simple java Jar file library from a source folder
+
+    myDir = File.dirname(__FILE__);
+    require "#{myDir}/../build-options.rb";
+
+    Rakish.Project(
+        :includes=>[Rakish::JavaProjectModule],
+        :name=>'artd-util',
+        :dependsUpon=> [
+        ]
+    ) do
+
+        export task :resources => [];
+
+        java.addSourceRoots("#{projectDir}/src");
+
+        java.addLibraryTargets();
+
+    end
+
+===Project file with dependencies and generated code using a custom tool
+
+    myDir = File.dirname(__FILE__);
+    require "#{myDir}/../build-options.rb";
+    require "#{myDir}/../artd-bml-generator/BmlGeneratorModule.rb"
+
+    Rakish.Project(
+        :includes=>[Rakish::JavaProjectModule, Rakish::BmlGeneratorModule],
+        :name=>'artd-bml-interpreter',
+        :dependsUpon=> [
+            '../artd-bml-generator',  # note it is dependedn on the generator this is also built.
+            '../artd-util',
+            '../artd-net',
+            '../artd-bml-base'
+        ]
+    ) do
+
+        export task :resources => [];
+
+        task :cleanautogen do |t|
+            FileUtils.rm_rf "#{projectDir}/gensrc";
+        end
+
+        task :cleanAll => [:clean, :cleanautogen ] do
+            FileUtils.rm_rf "#{buildDir()}/production/#{moduleName}";
+        end
+
+        generatedSource = createBmlgenTasks('./bmlsrc/*.xml',
+                                            :outPath=>'./gensrc',
+                                            :defaultPackage=>'com.artd.bml.interfaces');
+
+        export(task :autogen => generatedSource);
+
+        task :compile => [:autogen];
+
+        java.addSourceRoots("#{projectDir}/src");
+        java.addSourceRoots("#{projectDir}/gensrc", :generated=>true);
+
+        java.addProjectOutputClasspaths(
+            'artd-util',
+            'artd-net',
+            'artd-bml-base'
+        );
+
+        java.addLibraryTargets();
+
+        export (task :reallyClean => [:cleanAll])
+
+    end
+
+===The custom build module used for the gcode geenration from interface definitions
+
+    # Note to be included in in Rakish projects as a module
+
+    require 'rexml/streamlistener'
+
+    module Rakish
+
+        module BmlGeneratorModule
+
+            addInitBlock do |pnt,opts|   # this is called when the module is included on a project's class instance
+                if(pnt != nil)           # pnt parent configration or project
+                    @java_home = pnt.get(:java_home);
+                end
+                @java_home ||= File.expand_path(ENV['JAVA_HOME']);
+            end
+
+            class XMLListener
+                include Rakish::Util
+                include REXML::StreamListener
+
+                attr_accessor(:defaultPackage)
+                attr_accessor(:javaOutputDir)
+                attr_reader(:javaOutputFiles)
+
+
+                @@outPath = [ 'BMLGEN', 'Interface' ];
+
+                def initialize()
+                    @tagPath=[];
+                    @skipping=nil;
+                    @javaOutputFiles=FileSet.new();
+                end
+
+                def tag_start(name, attributes)
+                    @tagPath.push(name);
+
+                    if(@tagPath.length === @@outPath.length && @tagPath[1] === @@outPath[1])
+                        javaFileName = attributes['name'];
+                        package = attributes['java.package'] || defaultPackage;
+                        packagePath = package.gsub('.','/');
+                        javaOutputFiles.add("#{javaOutputDir}/#{packagePath}/#{javaFileName}.java");
+                    end
+                end
+                def tag_end(name)
+                    @tagPath.pop;
+                end
+            end
+
+            def createBmlgenTasks(*xmlsource)
+
+                opts = (Hash === xmlsource.last) ? xmlsource.pop : {}
+
+                opts[:outPath]||="./gensrc";
+                defaultPackage = opts[:defaultPackage]||="com.artd.bml.interfaces";
+                xmlIncludePath = opts[:xmlIncludePath]||=[];
+
+                outdir = File.expand_path(opts[:outPath]);
+
+                files = FileSet.new;
+                files.include(xmlsource);
+
+                xmlIncludePath.map! do |path|
+                    path = File.expand_path(path);
+                    path
+                end
+
+                xmlsources = [];
+                classname = "";
+
+                listener = XMLListener.new();
+                listener.defaultPackage = defaultPackage;
+                listener.javaOutputDir = outdir;
+
+                # collect up all java files this invocation of the generator will create
+                # from top level "Interface" tags in all the XML source files.
+
+                files.each do |srcname|
+                    # puts("creating task for #{srcname}");
+                    parser = REXML::Parsers::StreamParser.new(File.new(srcname), listener)
+                    parser.parse
+                    xmlsources << srcname;
+
+                end
+
+                # create one task for the combined compile for all the xml source files
+
+                batchTask = Rake::Task::define_unique_task;
+                batchTask.enhance("#{binDir()}/artd-bml-generator.jar"); # make sure this is built
+
+                ensureDirectoryTask(outdir);
+                batchTask.enhance([outdir]);
+
+                batchTask.enhance(batchTask.sources=xmlsources) do |t|
+                    cfg = t.config;
+                    FileUtils.cd cfg.projectDir do
+                        args = " -java javadir=#{outdir} java.package=#{defaultPackage}";
+                        if(xmlIncludePath.length > 0)
+                             args += " \"src.includepath=#{xmlIncludePath.join(";")}\"";
+                        end
+                        sourcexml = "\"#{t.sources.join("\" \"")}\"";
+
+                        cmd = "\"#{java_home}/bin/java\" -cp \"#{cfg.binDir}/artd-bml-generator.jar\"  com.artd.bml.CodeGenerator.XMLToCode #{args} #{sourcexml}";
+                        system(cmd);
+                    end
+                end
+                batchTask.config = self;
+
+                # create a task for each output java file dependent on the source files.
+                # if any java file is out of date invoke the combined task for all the java files once.
+
+                tasks = [];
+
+                listener.javaOutputFiles.each do |javaFile|
+                    tsk = Rake::FileTask.define_task javaFile => [xmlsources] do |t|
+                        genTask = t.config;
+                        genTask.invoke();
+                    end
+                    tsk.config = batchTask; # note this used to pass in task to be completed
+                    tasks << tsk;
+                end
+
+                task :clean do
+                    addCleanFiles(tasks);
+                end
+
+                if(tasks.length > 0)
+                    task :autogen => tasks;
+                end
+
+                tasks   # pass back the list of tasks for creating the java files.
+            end
+
+        end
+
+    end # Rakish
+
+===A project that builds a tomcat war file
+
+Though the ServletProjectModule isn't in the gem (yet) I put this here to
+show building an archive (.war file) and searching the jar file library path to
+get them.  moduleJars are built bt dependent projects and they are in the "binDir" the others come
+from the library search path.
+
+    myDir = File.dirname(__FILE__);
+    require "#{myDir}/../build-options.rb";
+    require "rakish/TomcatProjects.rb";
+
+    Rakish.Project(
+        :includes=>[Rakish::ServeletProjectModule], # note not in gem at present
+        :name=>'omp-router',
+        :dependsUpon=> [
+            '../../libs',
+            '../omp-servlet',
+            '../oms-Interfaces',
+            '../omp-smpp'
+        ]
+    ) do
+
+        export task :resources => [];
+
+        java.addSourceRoots("#{projectDir}/src");
+
+        java.addClassPaths("servlet-api.jar");
+
+        java.addProjectOutputClasspaths(
+            'artd-util',
+            'artd-net',
+            'artd-bml-base',
+            'artd-bml-interpreter',
+            'artd-bml-servlet',
+            'oms-Interfaces',
+            'omp-servlet',
+            'omp-smpp'
+        );
+
+        javac = java.javacTask
+
+        moduleJars = [
+                'artd-util.jar',
+                'artd-net.jar',
+                'artd-bml-servlet.jar',
+                'artd-bml-interpreter.jar',
+                'artd-bml-net.jar',
+                'artd-bml-generator.jar',
+                'artd-bml-base.jar',
+                'oi-xml.jar',
+                'omp-smpp.jar',
+                'omp-net.jar',
+                'oms-Dao.jar',
+                'oms-Connector.jar',
+                'omp-servlet.jar',
+                'oms-Protocol.jar',
+                'oms-Interfaces.jar'
+            ].map! { |name| File.join(binDir(), name); };
+
+        libJars = java.resolveJarsWithPath([
+              'SMPPLibrary.jar',
+              'commons-logging-1.1.3.jar',
+              'commons-logging-api-1.1.jar',
+              'commons-validator-1.3.1.jar',
+              'displaytag-1.2.jar',
+              'dom4j-1.6.1.jar',
+              'freemarker-2.3.19.jar',
+              'hibernate-c3p0-4.1.8.final.jar',
+              'hibernate-commons-annotations-4.0.5.Final.jar',
+              'hibernate-core-4.3.8.Final.jar',
+              'antlr-2.7.7.jar',
+              'hibernate-jpa-2.1-api-1.0.0.Final.jar',
+              'antlr-runtime-3.0.1.jar',
+              'jandex-1.1.0.Final.jar',
+              'javassist-3.11.0.GA.jar',
+              'javassist-3.18.1-GA.jar',
+              'javax.mail-1.4.5.jar',
+              'jboss-logging-3.1.3.GA.jar',
+              'jboss-logging-annotations-1.2.0.Beta1.jar',
+              'jboss-transaction-api_1.2_spec-1.0.0.Final.jar',
+              'jta-4.0.jar',
+              'asm-3.2.jar',
+              'log4j-1.2.17.jar',
+              'asm-attrs.jar',
+              'ognl-3.0.6.jar',
+              'c3p0-0.9.1.jar',
+              'org-apache-commons-logging.jar',
+              'cglib-2.1.96.jar',
+              'slf4j-api-1.7.10.jar',
+              'commons-fileupload-1.3.1.jar',
+              'slf4j-log4j12-1.7.10.jar',
+              'commons-httpclient-3.1.jar',
+              'struts2-core-2.3.20.jar',
+              'commons-io-2.2.jar',
+              'struts2yuiplugin-0.1-ALPHA-7.jar',
+              'commons-lang-2.4.jar',
+              'xwork-core-2.3.20.jar',
+              'commons-lang3-3.2.jar'
+            ],
+            :errorOnMissing=>true  # if one is not found this will raise an exception.
+        );
+
+        warBuilder = createWarBuilder
+
+        warBuilder.addFiles('/WEB-INF/lib', moduleJars, libJars );
+        warBuilder.addFileTree('/WEB-INF/classes', java.outputClasspath, "#{java.outputClasspath}/**/*" );
+        warBuilder.addFileTree('/', './web', './web/WEB-INF/**/*');
+
+        warFile = warBuilder.warFileTask "#{binDir()}/omp-router-0.1-dev.war" => [ :compile, :libs ];
+
+        export task :dist => [ warFile ];
+
+        createDeploymentTasks( routerConfig,
+                                :sourceWar=>warFile.name,
+                                :webPath=>"a.l" );
+
+
+    end
+

data/doc/ToDoItems.txt

@@ -0,0 +1,18 @@
+
+==Items To Do
+
+===Of course make a decent C++/C/asm module for msc and gcc g++ etc
+
+===Make a usable MultiProcessTask for spawning multiple asynchronous tasks
+The one in Rake actually slows things down when too
+many are spawned due to the thread swithing overhead so it really isn't practical.  What it should do is maintain a semaphone
+managed queue and only spawn as many tasks as there are processor cores on the machine to handle the big processes they
+spawn ( compilers and the like ) the little ruby tasks are better off being run in the primary thread unless they are
+doing something with a lot of waiting involved like uploading a file. In any case limiting the number of threads
+would help this too.
+
+===Make tomcat servlet module releasable
+===A real manual
+===Make intellij integration module releasable
+
+

data/doc/UserGuide

@@ -0,0 +1,15 @@
+
+= Rakish - Build system user guide
+
+
+:include:doc/RakishOverview.html
+
+===See why I did it at the end if curious.
+
+:include:doc/SimpleJavaSamples.txt
+
+:include:doc/WhyIDidIt.txt
+
+:include:doc/ToDoItems.txt
+
+

data/doc/WhyIDidIt.txt

@@ -0,0 +1,96 @@
+
+==Why I did it
+
+I have over many years had to use various build scripting systems IDEs and wiite scripts for them.
+
+Many projects after they got at all big required lots of things to be done,
+ - signing files,
+ - building custom tools
+ - genrating code with those custom tools
+ - auto generating version Ids
+ - generating documentation
+ - inserting special comment headers
+ - creating distribution packages
+ - invoking automated testing.
+ - build on checkin on various build servers
+
+And many other little things.
+
+Makefiles though I managed to do what was needed are arcane,
+don't handle strings with spaces in them, have no sense of arrays or collections,
+and really don't handle large trees of dependent modules well having to reload
+everything in every subprocess and re-scan all the dependencies each time.
+and one has to deal with backslash hell and the differences beween windows and
+unix shells. and there is no sense at all of scoped variables or namespaces. I could go on :)
+
+That said the big advantage was it is open, available on all platforms, and either part of the os or
+brain dead easy to install.  The goal being tell a new team member "download this", "install make"
+go to the root folder and enter "make all" ir whatever and it builds the whole massive project.
+
+That all said makefiles have been superior at divvying up processes such as C++
+compiles efficiently on multi-core machines for the absolute fastest rebuild build times.
+However the big projects were surely an undocumented hell for anyone trying to figure
+out how all the makefiles were put together.
+
+With Java I got really annoyed with Ant as XML is just not a programming system.
+
+And yes I know ther are all sorts of Mavens, IDEs and then like.  Many either platform proprietary
+or so oriented to a particular language like Ant with Java that builing and manging mixeld lanquage
+projects with them was a bear.  And I found that one always ended up havint to write programs in
+other languanges or scripting systems to make it all hang together and one eneded up with a hard
+to document and mantain blob of stuff again.
+
+So maybe 8 years ago I decided I woudl bite the bullet and make what I wanted in a "makefile" system.
+not dependent an a particular IDE or proprietary product or using a limited specialty languange etc.
+
+I discovered Ruby and it's sistem module Rake which are very nice.  Ruby is coherant, well documented,
+standard, cross platform, easy to install, has great text processing capabilities, regexes, collections, namespaces,
+scoped variables, hashes "closures - ie: blocks" and many other great features.  Easier to manage that ECMA
+script etc.  Anyway you Ruby people will know what I mean.
+
+Rake though it covers the basics of "makefiles" is limited in scope but does handle task management and dependency,
+checking well and has namespaces and useful utilities for paths and files. And it is built on Ruby.
+
+So Rakish as tha name inplies is my attempt at making Rake a bit sexier.
+
+Using it all I have "Project"s which are the items used to define medium build atoms larger than tasks.
+A library from source, a jar file, a set of classes, etc and contauined in a single folder.
+And then projects are dependent on other projects.  Each project has it's own internal
+namespece so internal tasks unless "exported" don't conflict with others outside.
+Also I can load the entire tree of projects in one go fast and can check dependencies globally on the whole
+mess for faster incremental build times and a tree of makefiles.  Project loading s lazy so if one builds
+a module with few dependencies only what it is dependent on will be loaded.
+
+The "Configurations" are managed through a "PropertyBag" object I built from ruby.  Property bags can have fields
+added dynamically, and can inherit values from "parents" and what I might call "uncles" ( parents are inherited by
+their children. "Uncles" are only available at the level they are attached to and are not passed down
+to children at the next level.  This enables a configuration override structure so for example each project
+though it might inherit the global intermediate output path appends it's name etc so it has it's own sandbox.
+
+I also made a superclass constructor system so multiple modules included in Projects are intialized
+on an instance of a project in much the same way as Java or C++ superclasses, yet they are still mixins
+offering nice multiple inheritance.
+
+"Projects" are basiclally property bags and hosts for these modules, depedency initialization code and
+containers and builders of tasks. Projects include modules that customize the projects for different types of
+capabilities.  One can add custom capabilites to any project by writing a module for it to the pattern of the others.
+
+The ArchiveBuilder (Zip and Jar modules ) and the Java one in the gem is mature and tested. I re-did it as a
+pattern of how I now think best to structure these modules.  It consists of a configuration Module and a
+Project module.  I am now using it for my current Java projects integrated with intellij.
+
+There is a C++ module I converted from old makefiles but it is
+not really ready for prime time as it has too many messy dependencies on the local enviropnment just like the makefiles
+it is included more just so people know it is there.  And I have a tomcat servlet module and a few others.
+I would have improved it but Java web programs has been the majority of work for the past several years and the
+C++ stuff still works to build my libraries and native code for Java bindings.
+
+I have glue modules to run these builds integrated with intellij, have done so with eclipse, and Microsoft
+Developers Studio.
+
+Anyway maybe this can grow into a community resource so I can use it and clients won't be afraid of it :)
+And maybe convice people I am worth hiring :)
+
+All the best
+Peter K.
+