gigantron 0.1.1 → 0.1.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (62) hide show
  1. data/test/test_generator_helper.rb +29 -29
  2. data/test/test_gigantron_generator.rb +118 -114
  3. data/test/test_helper.rb +4 -4
  4. data/test/test_mapreduce_task_generator.rb +50 -50
  5. data/test/test_migration_generator.rb +49 -49
  6. data/test/test_model_generator.rb +53 -53
  7. data/test/test_task_generator.rb +48 -48
  8. metadata +48 -109
  9. data/History.txt +0 -15
  10. data/License.txt +0 -20
  11. data/Manifest.txt +0 -62
  12. data/PostInstall.txt +0 -4
  13. data/README.txt +0 -77
  14. data/Rakefile +0 -4
  15. data/app_generators/gigantron/USAGE +0 -7
  16. data/app_generators/gigantron/gigantron_generator.rb +0 -81
  17. data/app_generators/gigantron/templates/Rakefile +0 -12
  18. data/app_generators/gigantron/templates/database.yml +0 -8
  19. data/app_generators/gigantron/templates/initialize.rb +0 -34
  20. data/app_generators/gigantron/templates/tasks/import.rake +0 -10
  21. data/app_generators/gigantron/templates/test/tasks/test_import.rb +0 -23
  22. data/app_generators/gigantron/templates/test/test_helper.rb +0 -7
  23. data/bin/gigantron +0 -15
  24. data/config/hoe.rb +0 -82
  25. data/config/requirements.rb +0 -15
  26. data/gigantron_generators/mapreduce_task/USAGE +0 -5
  27. data/gigantron_generators/mapreduce_task/mapreduce_task_generator.rb +0 -54
  28. data/gigantron_generators/mapreduce_task/templates/mapreduce/mr_task.rb +0 -22
  29. data/gigantron_generators/mapreduce_task/templates/tasks/task.rake +0 -5
  30. data/gigantron_generators/mapreduce_task/templates/test/tasks/test_task.rb +0 -22
  31. data/gigantron_generators/migration/USAGE +0 -5
  32. data/gigantron_generators/migration/migration_generator.rb +0 -61
  33. data/gigantron_generators/migration/templates/db/migrate/migration.rb +0 -7
  34. data/gigantron_generators/model/USAGE +0 -11
  35. data/gigantron_generators/model/model_generator.rb +0 -54
  36. data/gigantron_generators/model/templates/models/model.rb +0 -3
  37. data/gigantron_generators/model/templates/test/models/test_model.rb +0 -13
  38. data/gigantron_generators/task/USAGE +0 -10
  39. data/gigantron_generators/task/task_generator.rb +0 -51
  40. data/gigantron_generators/task/templates/tasks/task.rake +0 -4
  41. data/gigantron_generators/task/templates/test/tasks/test_task.rb +0 -22
  42. data/lib/gigantron/migrator.rb +0 -10
  43. data/lib/gigantron/tasks/db.rb +0 -11
  44. data/lib/gigantron/tasks/test.rb +0 -30
  45. data/lib/gigantron/version.rb +0 -9
  46. data/lib/gigantron.rb +0 -0
  47. data/script/console +0 -10
  48. data/script/destroy +0 -14
  49. data/script/generate +0 -14
  50. data/script/txt2html +0 -82
  51. data/setup.rb +0 -1585
  52. data/tasks/deployment.rake +0 -34
  53. data/tasks/environment.rake +0 -7
  54. data/tasks/website.rake +0 -17
  55. data/test/template_database.yml +0 -9
  56. data/test/template_database.yml.example +0 -9
  57. data/test/template_migration.rb +0 -16
  58. data/website/index.html +0 -213
  59. data/website/index.txt +0 -145
  60. data/website/javascripts/rounded_corners_lite.inc.js +0 -285
  61. data/website/stylesheets/screen.css +0 -138
  62. data/website/template.html.erb +0 -48
@@ -1,34 +0,0 @@
1
- desc 'Release the website and new gem version'
2
- task :deploy => [:check_version, :website, :release] do
3
- puts "Remember to create SVN tag:"
4
- puts "svn copy svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/trunk " +
5
- "svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/tags/REL-#{VERS} "
6
- puts "Suggested comment:"
7
- puts "Tagging release #{CHANGES}"
8
- end
9
-
10
- desc 'Runs tasks website_generate and install_gem as a local deployment of the gem'
11
- task :local_deploy => [:website_generate, :install_gem]
12
-
13
- task :check_version do
14
- unless ENV['VERSION']
15
- puts 'Must pass a VERSION=x.y.z release version'
16
- exit
17
- end
18
- unless ENV['VERSION'] == VERS
19
- puts "Please update your version.rb to match the release version, currently #{VERS}"
20
- exit
21
- end
22
- end
23
-
24
- desc 'Install the package as a gem, without generating documentation(ri/rdoc)'
25
- task :install_gem_no_doc => [:clean, :package] do
26
- sh "#{'sudo ' unless Hoe::WINDOZE }gem install pkg/*.gem --no-rdoc --no-ri"
27
- end
28
-
29
- namespace :manifest do
30
- desc 'Recreate Manifest.txt to include ALL files'
31
- task :refresh do
32
- `rake check_manifest | patch -p0 > Manifest.txt`
33
- end
34
- end
@@ -1,7 +0,0 @@
1
- task :ruby_env do
2
- RUBY_APP = if RUBY_PLATFORM =~ /java/
3
- "jruby"
4
- else
5
- "ruby"
6
- end unless defined? RUBY_APP
7
- end
data/tasks/website.rake DELETED
@@ -1,17 +0,0 @@
1
- desc 'Generate website files'
2
- task :website_generate => :ruby_env do
3
- (Dir['website/**/*.txt'] - Dir['website/version*.txt']).each do |txt|
4
- sh %{ #{RUBY_APP} script/txt2html #{txt} > #{txt.gsub(/txt$/,'html')} }
5
- end
6
- end
7
-
8
- desc 'Upload website files to rubyforge'
9
- task :website_upload do
10
- host = "#{rubyforge_username}@rubyforge.org"
11
- remote_dir = "/var/www/gforge-projects/#{PATH}/"
12
- local_dir = 'website'
13
- sh %{rsync -aCv #{local_dir}/ #{host}:#{remote_dir}}
14
- end
15
-
16
- desc 'Generate and upload website files'
17
- task :website => [:website_generate, :website_upload, :publish_docs]
@@ -1,9 +0,0 @@
1
- # JRuby
2
- :test:
3
- :adapter: jdbcsqlite3
4
- :url: jdbc:sqlite:<%= GTRON_ROOT %>/db/test.sqlite3
5
-
6
- # Ruby 1.8
7
- #:test:
8
- # :adapter: sqlite3
9
- # :database: <%= File.expand_path GTRON_ROOT %>/db/test.sqlite3
@@ -1,9 +0,0 @@
1
- # JRuby
2
- #:test:
3
- # :adapter: jdbcsqlite3
4
- # :url: jdbc:sqlite:<%= GTRON_ROOT %>/db/test.sqlite3
5
-
6
- # Ruby 1.8
7
- #:test:
8
- # :adapter: sqlite3
9
- # :database: <%= File.expand_path GTRON_ROOT %>/db/test.sqlite3
@@ -1,16 +0,0 @@
1
- class CreateFoos < ActiveRecord::Migration
2
- def self.up
3
- create_table :foos do |t|
4
- t.string :title
5
- end
6
-
7
- %w(Foo Bar).each do |t|
8
- Foo.new(:title => t).save
9
- end
10
- end
11
-
12
- def self.down
13
- drop_table :foos
14
- end
15
- end
16
-
data/website/index.html DELETED
@@ -1,213 +0,0 @@
1
- <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
2
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3
- <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4
- <head>
5
- <link rel="stylesheet" href="stylesheets/screen.css" type="text/css" media="screen" />
6
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
7
- <title>
8
- Gigantron: Processor of Data
9
- </title>
10
- <script src="javascripts/rounded_corners_lite.inc.js" type="text/javascript"></script>
11
- <style>
12
-
13
- </style>
14
- <script type="text/javascript">
15
- window.onload = function() {
16
- settings = {
17
- tl: { radius: 10 },
18
- tr: { radius: 10 },
19
- bl: { radius: 10 },
20
- br: { radius: 10 },
21
- antiAlias: true,
22
- autoPad: true,
23
- validTags: ["div"]
24
- }
25
- var versionBox = new curvyCorners(settings, document.getElementById("version"));
26
- versionBox.applyCornersToAll();
27
- }
28
- </script>
29
- </head>
30
- <body>
31
- <div id="main">
32
-
33
- <h1>Gigantron: Processor of Data</h1>
34
- <div id="version" class="clickable" onclick='document.location = "http://rubyforge.org/projects/gigantron"; return false'>
35
- <p>Get Version</p>
36
- <a href="http://rubyforge.org/projects/gigantron" class="numbers">0.1.1</a>
37
- </div>
38
- <h1>&#x2192; &#8216;gigantron&#8217;</h1>
39
-
40
-
41
- <h2>What</h2>
42
-
43
-
44
- <p>Gigantron is a simple framework for the creation and organization of
45
- data processing projects. Data-processing transforms are created as Rake tasks
46
- and data is handled through ActiveRecord models. (DataMapper was the original
47
- plan, but it has problems playing nicely with JRuby for now).</p>
48
-
49
-
50
- <p>Ruby is great for exploratory data processing. Data processing projects tend
51
- to grow up and encompass large numbers of random scripts and input files. It
52
- is easy to get lost in coding and lose organization. Gigantron is an attempt
53
- to use code generation and random magic to make maintaining organized DP
54
- projects simple. Code is separated into data (models) and operations on the
55
- data (tasks). Code generators stub out these files and the associated tests
56
- for the user.</p>
57
-
58
-
59
- <p>Gigantron was written for my own needs working with atmospheric data and will
60
- evolve through use to reduce the trivialities that can sometimes dominate the
61
- work of developers.</p>
62
-
63
-
64
- <h2>Installing</h2>
65
-
66
-
67
- <p><pre class='syntax'><span class="ident">sudo</span> <span class="ident">gem</span> <span class="ident">install</span> <span class="ident">gigantron</span></pre></p>
68
-
69
-
70
- <h2>The basics</h2>
71
-
72
-
73
- <pre>
74
- # Generate new project
75
- shell&gt; $ gigantron project
76
- create
77
- create tasks
78
- create db
79
- create models
80
- create lib
81
- create test
82
- create Rakefile
83
- create database.yml
84
- create initialize.rb
85
- create tasks/import.rake
86
- create test/test_helper.rb
87
- create test/models
88
- create test/tasks
89
- create test/tasks/test_import.rb
90
- dependency install_rubigen_scripts
91
- create script
92
- create script/generate
93
- create script/destroy
94
- shell&gt; $ cd project
95
- # Create new model
96
- shell&gt; $ script/generate model modis
97
- exists models/
98
- create models/modis.rb
99
- exists test/
100
- exists test/models/
101
- create test/models/test_modis.rb
102
- shell&gt; $ script/generate task modis_to_kml
103
- exists tasks/
104
- create tasks/modis_to_kml.rake
105
- exists test/
106
- exists test/tasks/
107
- create test/tasks/test_modis_to_kml.rb
108
- </pre>
109
-
110
- <p>One can edit these files to add functionality. Gigantron by default includes
111
- ActiveSupport for convenience.</p>
112
-
113
-
114
- <h2>Hacking</h2>
115
-
116
-
117
- <p>Gigantron is super minimal now, so modifying it is pretty easy. The gigantron
118
- application generator (that which is invoked by the <code>gigantron</code> command) lives
119
- in <code>app_generators/gigantron/</code>. The template files and the template dir
120
- structure are in <code>app_generators/gigantron/templates/</code>. When adding new
121
- templates, directories, or files, add the names first to the tests in
122
- <code>test/test_gigantron_generator.rb</code>, then the stubs to
123
- <code>app_generators/gigantron/templates</code>, and then finally describe them in the
124
- manifest section of <code>app_generators/gigantron/gigantron_generator.rb</code>. It
125
- should look something like</p>
126
-
127
-
128
- <p><pre class='syntax'>
129
- <span class="keyword">def </span><span class="method">manifest</span>
130
- <span class="ident">record</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">m</span><span class="punct">|</span>
131
- <span class="punct">...</span>
132
- <span class="ident">m</span><span class="punct">.</span><span class="ident">file</span> <span class="punct">&quot;</span><span class="string">new_file</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">new_file</span><span class="punct">&quot;</span> <span class="comment">#straight file copy</span>
133
- <span class="ident">m</span><span class="punct">.</span><span class="ident">directory</span> <span class="punct">&quot;</span><span class="string">my_new_dir</span><span class="punct">&quot;</span> <span class="comment">#create directory</span>
134
- <span class="ident">m</span><span class="punct">.</span><span class="ident">template</span> <span class="punct">&quot;</span><span class="string">new_thing</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">new_thing</span><span class="punct">&quot;</span> <span class="comment">#runs file through ERB when copying</span>
135
- <span class="keyword">end</span>
136
- <span class="keyword">end</span>
137
- </pre></p>
138
-
139
-
140
- <p>It might be handy to know that in <code>gigantron_generator.rb</code> the name provided to the generator can be referenced as <code>@name</code>. This can be used like</p>
141
-
142
-
143
- <p><pre class='syntax'>
144
- <span class="ident">record</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">m</span><span class="punct">|</span>
145
- <span class="ident">m</span><span class="punct">.</span><span class="ident">file</span> <span class="punct">&quot;</span><span class="string">renamed_file</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string"><span class="expr">#{@name}</span>_file</span><span class="punct">&quot;</span>
146
- <span class="keyword">end</span>
147
- </pre></p>
148
-
149
-
150
- <p>The same value is available to your templates as just <code>name</code>. You can template
151
- a file like</p>
152
-
153
-
154
- <p><pre class='syntax'>
155
- <span class="keyword">class </span><span class="class">Test</span><span class="punct">&lt;%=</span><span class="string"> name.camelcase %&gt; &lt; Test::Unit::TestCase
156
- ...
157
- end<span class="normal">
158
- </span></span></pre></p>
159
-
160
-
161
- <p>The same process applies to the model and task generator for gigantron
162
- projects. These generators live in <code>gigantron_generators/</code>. Modifying them
163
- is exactly the same as modifying the application generator.</p>
164
-
165
-
166
- <p>All of this is pretty vanilla <a href="http://rubigen.rubyforge.org">RubiGen</a>, so if in doubt, check out the docos on that fine piece of work.</p>
167
-
168
-
169
- <p>The only other place for code in Gigantron is is <code>lib/gigantron/tasks/</code> where a few boilerplate test and db tasks live. I think I ripped the test tasks off of rails.</p>
170
-
171
-
172
- <p>If you have any questions, do contact me. I am interested in anything that will make Gigantron suck less and be useful to people.</p>
173
-
174
-
175
- <h2>How to submit patches</h2>
176
-
177
-
178
- <ul>
179
- <li>github: <a href="http://github.com/schleyfox/gigantron/tree/master">http://github.com/schleyfox/gigantron/tree/master</a></li>
180
- </ul>
181
-
182
-
183
- <pre>git clone git://github.com/schleyfox/gigantron.git</pre>
184
-
185
- <h3>Build and test instructions</h3>
186
-
187
-
188
- <pre>cd gigantron
189
- cp test/template_database.yml.example test/template_database.yml
190
- vim test/template_database.yml
191
- rake test
192
- rake install_gem</pre>
193
-
194
- <h2>License</h2>
195
-
196
-
197
- <p>This code is free to use under the terms of the <span class="caps">MIT</span> license.</p>
198
-
199
-
200
- <h2>Contact</h2>
201
-
202
-
203
- <p>Comments are welcome. Send an email to <a href="mailto:ben@pixelmachine.org">Ben Hughes</a></p>
204
- <p class="coda">
205
- <a href="ben@pixelmachine.org">Ben Hughes</a>, 19th June 2008<br>
206
- Theme extended from <a href="http://rb2js.rubyforge.org/">Paul Battley</a>
207
- </p>
208
- </div>
209
-
210
- <!-- insert site tracking codes here, like Google Urchin -->
211
-
212
- </body>
213
- </html>
data/website/index.txt DELETED
@@ -1,145 +0,0 @@
1
- h1. Gigantron: Processor of Data
2
-
3
- h1. &#x2192; 'gigantron'
4
-
5
-
6
- h2. What
7
-
8
- Gigantron is a simple framework for the creation and organization of
9
- data processing projects. Data-processing transforms are created as Rake tasks
10
- and data is handled through ActiveRecord models. (DataMapper was the original
11
- plan, but it has problems playing nicely with JRuby for now).
12
-
13
- Ruby is great for exploratory data processing. Data processing projects tend
14
- to grow up and encompass large numbers of random scripts and input files. It
15
- is easy to get lost in coding and lose organization. Gigantron is an attempt
16
- to use code generation and random magic to make maintaining organized DP
17
- projects simple. Code is separated into data (models) and operations on the
18
- data (tasks). Code generators stub out these files and the associated tests
19
- for the user.
20
-
21
- Gigantron was written for my own needs working with atmospheric data and will
22
- evolve through use to reduce the trivialities that can sometimes dominate the
23
- work of developers.
24
-
25
- h2. Installing
26
-
27
- <pre syntax="ruby">sudo gem install gigantron</pre>
28
-
29
- h2. The basics
30
-
31
- <pre>
32
- # Generate new project
33
- shell> $ gigantron project
34
- create
35
- create tasks
36
- create db
37
- create models
38
- create lib
39
- create test
40
- create Rakefile
41
- create database.yml
42
- create initialize.rb
43
- create tasks/import.rake
44
- create test/test_helper.rb
45
- create test/models
46
- create test/tasks
47
- create test/tasks/test_import.rb
48
- dependency install_rubigen_scripts
49
- create script
50
- create script/generate
51
- create script/destroy
52
- shell> $ cd project
53
- # Create new model
54
- shell> $ script/generate model modis
55
- exists models/
56
- create models/modis.rb
57
- exists test/
58
- exists test/models/
59
- create test/models/test_modis.rb
60
- shell> $ script/generate task modis_to_kml
61
- exists tasks/
62
- create tasks/modis_to_kml.rake
63
- exists test/
64
- exists test/tasks/
65
- create test/tasks/test_modis_to_kml.rb
66
- </pre>
67
-
68
- One can edit these files to add functionality. Gigantron by default includes
69
- ActiveSupport for convenience.
70
-
71
- h2. Hacking
72
-
73
- Gigantron is super minimal now, so modifying it is pretty easy. The gigantron
74
- application generator (that which is invoked by the @gigantron@ command) lives
75
- in @app_generators/gigantron/@. The template files and the template dir
76
- structure are in @app_generators/gigantron/templates/@. When adding new
77
- templates, directories, or files, add the names first to the tests in
78
- @test/test_gigantron_generator.rb@, then the stubs to
79
- @app_generators/gigantron/templates@, and then finally describe them in the
80
- manifest section of @app_generators/gigantron/gigantron_generator.rb@. It
81
- should look something like
82
-
83
- <pre syntax="ruby">
84
- def manifest
85
- record do |m|
86
- ...
87
- m.file "new_file", "new_file" #straight file copy
88
- m.directory "my_new_dir" #create directory
89
- m.template "new_thing", "new_thing" #runs file through ERB when copying
90
- end
91
- end
92
- </pre>
93
-
94
- It might be handy to know that in @gigantron_generator.rb@ the name provided to the generator can be referenced as @@name@. This can be used like
95
-
96
- <pre syntax="ruby">
97
- record do |m|
98
- m.file "renamed_file", "#{@name}_file"
99
- end
100
- </pre>
101
-
102
- The same value is available to your templates as just @name@. You can template
103
- a file like
104
-
105
- <pre syntax="ruby">
106
- class Test<%%= name.camelcase %> < Test::Unit::TestCase
107
- ...
108
- end
109
- </pre>
110
-
111
- The same process applies to the model and task generator for gigantron
112
- projects. These generators live in @gigantron_generators/@. Modifying them
113
- is exactly the same as modifying the application generator.
114
-
115
- All of this is pretty vanilla "RubiGen":http://rubigen.rubyforge.org, so if in doubt, check out the docos on that fine piece of work.
116
-
117
- The only other place for code in Gigantron is is @lib/gigantron/tasks/@ where a few boilerplate test and db tasks live. I think I ripped the test tasks off of rails.
118
-
119
- If you have any questions, do contact me. I am interested in anything that will make Gigantron suck less and be useful to people.
120
-
121
- h2. How to submit patches
122
-
123
-
124
- * github: "http://github.com/schleyfox/gigantron/tree/master":http://github.com/schleyfox/gigantron/tree/master
125
-
126
- <pre>git clone git://github.com/schleyfox/gigantron.git</pre>
127
-
128
-
129
- h3. Build and test instructions
130
-
131
- <pre>cd gigantron
132
- cp test/template_database.yml.example test/template_database.yml
133
- vim test/template_database.yml
134
- rake test
135
- rake install_gem</pre>
136
-
137
-
138
- h2. License
139
-
140
- This code is free to use under the terms of the MIT license.
141
-
142
- h2. Contact
143
-
144
- Comments are welcome. Send an email to "Ben Hughes":mailto:ben@pixelmachine.org
145
-