gigantron 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (62) hide show
  1. data/test/test_generator_helper.rb +29 -29
  2. data/test/test_gigantron_generator.rb +118 -114
  3. data/test/test_helper.rb +4 -4
  4. data/test/test_mapreduce_task_generator.rb +50 -50
  5. data/test/test_migration_generator.rb +49 -49
  6. data/test/test_model_generator.rb +53 -53
  7. data/test/test_task_generator.rb +48 -48
  8. metadata +48 -109
  9. data/History.txt +0 -15
  10. data/License.txt +0 -20
  11. data/Manifest.txt +0 -62
  12. data/PostInstall.txt +0 -4
  13. data/README.txt +0 -77
  14. data/Rakefile +0 -4
  15. data/app_generators/gigantron/USAGE +0 -7
  16. data/app_generators/gigantron/gigantron_generator.rb +0 -81
  17. data/app_generators/gigantron/templates/Rakefile +0 -12
  18. data/app_generators/gigantron/templates/database.yml +0 -8
  19. data/app_generators/gigantron/templates/initialize.rb +0 -34
  20. data/app_generators/gigantron/templates/tasks/import.rake +0 -10
  21. data/app_generators/gigantron/templates/test/tasks/test_import.rb +0 -23
  22. data/app_generators/gigantron/templates/test/test_helper.rb +0 -7
  23. data/bin/gigantron +0 -15
  24. data/config/hoe.rb +0 -82
  25. data/config/requirements.rb +0 -15
  26. data/gigantron_generators/mapreduce_task/USAGE +0 -5
  27. data/gigantron_generators/mapreduce_task/mapreduce_task_generator.rb +0 -54
  28. data/gigantron_generators/mapreduce_task/templates/mapreduce/mr_task.rb +0 -22
  29. data/gigantron_generators/mapreduce_task/templates/tasks/task.rake +0 -5
  30. data/gigantron_generators/mapreduce_task/templates/test/tasks/test_task.rb +0 -22
  31. data/gigantron_generators/migration/USAGE +0 -5
  32. data/gigantron_generators/migration/migration_generator.rb +0 -61
  33. data/gigantron_generators/migration/templates/db/migrate/migration.rb +0 -7
  34. data/gigantron_generators/model/USAGE +0 -11
  35. data/gigantron_generators/model/model_generator.rb +0 -54
  36. data/gigantron_generators/model/templates/models/model.rb +0 -3
  37. data/gigantron_generators/model/templates/test/models/test_model.rb +0 -13
  38. data/gigantron_generators/task/USAGE +0 -10
  39. data/gigantron_generators/task/task_generator.rb +0 -51
  40. data/gigantron_generators/task/templates/tasks/task.rake +0 -4
  41. data/gigantron_generators/task/templates/test/tasks/test_task.rb +0 -22
  42. data/lib/gigantron/migrator.rb +0 -10
  43. data/lib/gigantron/tasks/db.rb +0 -11
  44. data/lib/gigantron/tasks/test.rb +0 -30
  45. data/lib/gigantron/version.rb +0 -9
  46. data/lib/gigantron.rb +0 -0
  47. data/script/console +0 -10
  48. data/script/destroy +0 -14
  49. data/script/generate +0 -14
  50. data/script/txt2html +0 -82
  51. data/setup.rb +0 -1585
  52. data/tasks/deployment.rake +0 -34
  53. data/tasks/environment.rake +0 -7
  54. data/tasks/website.rake +0 -17
  55. data/test/template_database.yml +0 -9
  56. data/test/template_database.yml.example +0 -9
  57. data/test/template_migration.rb +0 -16
  58. data/website/index.html +0 -213
  59. data/website/index.txt +0 -145
  60. data/website/javascripts/rounded_corners_lite.inc.js +0 -285
  61. data/website/stylesheets/screen.css +0 -138
  62. data/website/template.html.erb +0 -48
@@ -1,34 +0,0 @@
1
- desc 'Release the website and new gem version'
2
- task :deploy => [:check_version, :website, :release] do
3
- puts "Remember to create SVN tag:"
4
- puts "svn copy svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/trunk " +
5
- "svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/tags/REL-#{VERS} "
6
- puts "Suggested comment:"
7
- puts "Tagging release #{CHANGES}"
8
- end
9
-
10
- desc 'Runs tasks website_generate and install_gem as a local deployment of the gem'
11
- task :local_deploy => [:website_generate, :install_gem]
12
-
13
- task :check_version do
14
- unless ENV['VERSION']
15
- puts 'Must pass a VERSION=x.y.z release version'
16
- exit
17
- end
18
- unless ENV['VERSION'] == VERS
19
- puts "Please update your version.rb to match the release version, currently #{VERS}"
20
- exit
21
- end
22
- end
23
-
24
- desc 'Install the package as a gem, without generating documentation(ri/rdoc)'
25
- task :install_gem_no_doc => [:clean, :package] do
26
- sh "#{'sudo ' unless Hoe::WINDOZE }gem install pkg/*.gem --no-rdoc --no-ri"
27
- end
28
-
29
- namespace :manifest do
30
- desc 'Recreate Manifest.txt to include ALL files'
31
- task :refresh do
32
- `rake check_manifest | patch -p0 > Manifest.txt`
33
- end
34
- end
@@ -1,7 +0,0 @@
1
- task :ruby_env do
2
- RUBY_APP = if RUBY_PLATFORM =~ /java/
3
- "jruby"
4
- else
5
- "ruby"
6
- end unless defined? RUBY_APP
7
- end
data/tasks/website.rake DELETED
@@ -1,17 +0,0 @@
1
- desc 'Generate website files'
2
- task :website_generate => :ruby_env do
3
- (Dir['website/**/*.txt'] - Dir['website/version*.txt']).each do |txt|
4
- sh %{ #{RUBY_APP} script/txt2html #{txt} > #{txt.gsub(/txt$/,'html')} }
5
- end
6
- end
7
-
8
- desc 'Upload website files to rubyforge'
9
- task :website_upload do
10
- host = "#{rubyforge_username}@rubyforge.org"
11
- remote_dir = "/var/www/gforge-projects/#{PATH}/"
12
- local_dir = 'website'
13
- sh %{rsync -aCv #{local_dir}/ #{host}:#{remote_dir}}
14
- end
15
-
16
- desc 'Generate and upload website files'
17
- task :website => [:website_generate, :website_upload, :publish_docs]
@@ -1,9 +0,0 @@
1
- # JRuby
2
- :test:
3
- :adapter: jdbcsqlite3
4
- :url: jdbc:sqlite:<%= GTRON_ROOT %>/db/test.sqlite3
5
-
6
- # Ruby 1.8
7
- #:test:
8
- # :adapter: sqlite3
9
- # :database: <%= File.expand_path GTRON_ROOT %>/db/test.sqlite3
@@ -1,9 +0,0 @@
1
- # JRuby
2
- #:test:
3
- # :adapter: jdbcsqlite3
4
- # :url: jdbc:sqlite:<%= GTRON_ROOT %>/db/test.sqlite3
5
-
6
- # Ruby 1.8
7
- #:test:
8
- # :adapter: sqlite3
9
- # :database: <%= File.expand_path GTRON_ROOT %>/db/test.sqlite3
@@ -1,16 +0,0 @@
1
- class CreateFoos < ActiveRecord::Migration
2
- def self.up
3
- create_table :foos do |t|
4
- t.string :title
5
- end
6
-
7
- %w(Foo Bar).each do |t|
8
- Foo.new(:title => t).save
9
- end
10
- end
11
-
12
- def self.down
13
- drop_table :foos
14
- end
15
- end
16
-
data/website/index.html DELETED
@@ -1,213 +0,0 @@
1
- <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
2
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
3
- <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4
- <head>
5
- <link rel="stylesheet" href="stylesheets/screen.css" type="text/css" media="screen" />
6
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
7
- <title>
8
- Gigantron: Processor of Data
9
- </title>
10
- <script src="javascripts/rounded_corners_lite.inc.js" type="text/javascript"></script>
11
- <style>
12
-
13
- </style>
14
- <script type="text/javascript">
15
- window.onload = function() {
16
- settings = {
17
- tl: { radius: 10 },
18
- tr: { radius: 10 },
19
- bl: { radius: 10 },
20
- br: { radius: 10 },
21
- antiAlias: true,
22
- autoPad: true,
23
- validTags: ["div"]
24
- }
25
- var versionBox = new curvyCorners(settings, document.getElementById("version"));
26
- versionBox.applyCornersToAll();
27
- }
28
- </script>
29
- </head>
30
- <body>
31
- <div id="main">
32
-
33
- <h1>Gigantron: Processor of Data</h1>
34
- <div id="version" class="clickable" onclick='document.location = "http://rubyforge.org/projects/gigantron"; return false'>
35
- <p>Get Version</p>
36
- <a href="http://rubyforge.org/projects/gigantron" class="numbers">0.1.1</a>
37
- </div>
38
- <h1>&#x2192; &#8216;gigantron&#8217;</h1>
39
-
40
-
41
- <h2>What</h2>
42
-
43
-
44
- <p>Gigantron is a simple framework for the creation and organization of
45
- data processing projects. Data-processing transforms are created as Rake tasks
46
- and data is handled through ActiveRecord models. (DataMapper was the original
47
- plan, but it has problems playing nicely with JRuby for now).</p>
48
-
49
-
50
- <p>Ruby is great for exploratory data processing. Data processing projects tend
51
- to grow up and encompass large numbers of random scripts and input files. It
52
- is easy to get lost in coding and lose organization. Gigantron is an attempt
53
- to use code generation and random magic to make maintaining organized DP
54
- projects simple. Code is separated into data (models) and operations on the
55
- data (tasks). Code generators stub out these files and the associated tests
56
- for the user.</p>
57
-
58
-
59
- <p>Gigantron was written for my own needs working with atmospheric data and will
60
- evolve through use to reduce the trivialities that can sometimes dominate the
61
- work of developers.</p>
62
-
63
-
64
- <h2>Installing</h2>
65
-
66
-
67
- <p><pre class='syntax'><span class="ident">sudo</span> <span class="ident">gem</span> <span class="ident">install</span> <span class="ident">gigantron</span></pre></p>
68
-
69
-
70
- <h2>The basics</h2>
71
-
72
-
73
- <pre>
74
- # Generate new project
75
- shell&gt; $ gigantron project
76
- create
77
- create tasks
78
- create db
79
- create models
80
- create lib
81
- create test
82
- create Rakefile
83
- create database.yml
84
- create initialize.rb
85
- create tasks/import.rake
86
- create test/test_helper.rb
87
- create test/models
88
- create test/tasks
89
- create test/tasks/test_import.rb
90
- dependency install_rubigen_scripts
91
- create script
92
- create script/generate
93
- create script/destroy
94
- shell&gt; $ cd project
95
- # Create new model
96
- shell&gt; $ script/generate model modis
97
- exists models/
98
- create models/modis.rb
99
- exists test/
100
- exists test/models/
101
- create test/models/test_modis.rb
102
- shell&gt; $ script/generate task modis_to_kml
103
- exists tasks/
104
- create tasks/modis_to_kml.rake
105
- exists test/
106
- exists test/tasks/
107
- create test/tasks/test_modis_to_kml.rb
108
- </pre>
109
-
110
- <p>One can edit these files to add functionality. Gigantron by default includes
111
- ActiveSupport for convenience.</p>
112
-
113
-
114
- <h2>Hacking</h2>
115
-
116
-
117
- <p>Gigantron is super minimal now, so modifying it is pretty easy. The gigantron
118
- application generator (that which is invoked by the <code>gigantron</code> command) lives
119
- in <code>app_generators/gigantron/</code>. The template files and the template dir
120
- structure are in <code>app_generators/gigantron/templates/</code>. When adding new
121
- templates, directories, or files, add the names first to the tests in
122
- <code>test/test_gigantron_generator.rb</code>, then the stubs to
123
- <code>app_generators/gigantron/templates</code>, and then finally describe them in the
124
- manifest section of <code>app_generators/gigantron/gigantron_generator.rb</code>. It
125
- should look something like</p>
126
-
127
-
128
- <p><pre class='syntax'>
129
- <span class="keyword">def </span><span class="method">manifest</span>
130
- <span class="ident">record</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">m</span><span class="punct">|</span>
131
- <span class="punct">...</span>
132
- <span class="ident">m</span><span class="punct">.</span><span class="ident">file</span> <span class="punct">&quot;</span><span class="string">new_file</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">new_file</span><span class="punct">&quot;</span> <span class="comment">#straight file copy</span>
133
- <span class="ident">m</span><span class="punct">.</span><span class="ident">directory</span> <span class="punct">&quot;</span><span class="string">my_new_dir</span><span class="punct">&quot;</span> <span class="comment">#create directory</span>
134
- <span class="ident">m</span><span class="punct">.</span><span class="ident">template</span> <span class="punct">&quot;</span><span class="string">new_thing</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">new_thing</span><span class="punct">&quot;</span> <span class="comment">#runs file through ERB when copying</span>
135
- <span class="keyword">end</span>
136
- <span class="keyword">end</span>
137
- </pre></p>
138
-
139
-
140
- <p>It might be handy to know that in <code>gigantron_generator.rb</code> the name provided to the generator can be referenced as <code>@name</code>. This can be used like</p>
141
-
142
-
143
- <p><pre class='syntax'>
144
- <span class="ident">record</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">m</span><span class="punct">|</span>
145
- <span class="ident">m</span><span class="punct">.</span><span class="ident">file</span> <span class="punct">&quot;</span><span class="string">renamed_file</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string"><span class="expr">#{@name}</span>_file</span><span class="punct">&quot;</span>
146
- <span class="keyword">end</span>
147
- </pre></p>
148
-
149
-
150
- <p>The same value is available to your templates as just <code>name</code>. You can template
151
- a file like</p>
152
-
153
-
154
- <p><pre class='syntax'>
155
- <span class="keyword">class </span><span class="class">Test</span><span class="punct">&lt;%=</span><span class="string"> name.camelcase %&gt; &lt; Test::Unit::TestCase
156
- ...
157
- end<span class="normal">
158
- </span></span></pre></p>
159
-
160
-
161
- <p>The same process applies to the model and task generator for gigantron
162
- projects. These generators live in <code>gigantron_generators/</code>. Modifying them
163
- is exactly the same as modifying the application generator.</p>
164
-
165
-
166
- <p>All of this is pretty vanilla <a href="http://rubigen.rubyforge.org">RubiGen</a>, so if in doubt, check out the docos on that fine piece of work.</p>
167
-
168
-
169
- <p>The only other place for code in Gigantron is is <code>lib/gigantron/tasks/</code> where a few boilerplate test and db tasks live. I think I ripped the test tasks off of rails.</p>
170
-
171
-
172
- <p>If you have any questions, do contact me. I am interested in anything that will make Gigantron suck less and be useful to people.</p>
173
-
174
-
175
- <h2>How to submit patches</h2>
176
-
177
-
178
- <ul>
179
- <li>github: <a href="http://github.com/schleyfox/gigantron/tree/master">http://github.com/schleyfox/gigantron/tree/master</a></li>
180
- </ul>
181
-
182
-
183
- <pre>git clone git://github.com/schleyfox/gigantron.git</pre>
184
-
185
- <h3>Build and test instructions</h3>
186
-
187
-
188
- <pre>cd gigantron
189
- cp test/template_database.yml.example test/template_database.yml
190
- vim test/template_database.yml
191
- rake test
192
- rake install_gem</pre>
193
-
194
- <h2>License</h2>
195
-
196
-
197
- <p>This code is free to use under the terms of the <span class="caps">MIT</span> license.</p>
198
-
199
-
200
- <h2>Contact</h2>
201
-
202
-
203
- <p>Comments are welcome. Send an email to <a href="mailto:ben@pixelmachine.org">Ben Hughes</a></p>
204
- <p class="coda">
205
- <a href="ben@pixelmachine.org">Ben Hughes</a>, 19th June 2008<br>
206
- Theme extended from <a href="http://rb2js.rubyforge.org/">Paul Battley</a>
207
- </p>
208
- </div>
209
-
210
- <!-- insert site tracking codes here, like Google Urchin -->
211
-
212
- </body>
213
- </html>
data/website/index.txt DELETED
@@ -1,145 +0,0 @@
1
- h1. Gigantron: Processor of Data
2
-
3
- h1. &#x2192; 'gigantron'
4
-
5
-
6
- h2. What
7
-
8
- Gigantron is a simple framework for the creation and organization of
9
- data processing projects. Data-processing transforms are created as Rake tasks
10
- and data is handled through ActiveRecord models. (DataMapper was the original
11
- plan, but it has problems playing nicely with JRuby for now).
12
-
13
- Ruby is great for exploratory data processing. Data processing projects tend
14
- to grow up and encompass large numbers of random scripts and input files. It
15
- is easy to get lost in coding and lose organization. Gigantron is an attempt
16
- to use code generation and random magic to make maintaining organized DP
17
- projects simple. Code is separated into data (models) and operations on the
18
- data (tasks). Code generators stub out these files and the associated tests
19
- for the user.
20
-
21
- Gigantron was written for my own needs working with atmospheric data and will
22
- evolve through use to reduce the trivialities that can sometimes dominate the
23
- work of developers.
24
-
25
- h2. Installing
26
-
27
- <pre syntax="ruby">sudo gem install gigantron</pre>
28
-
29
- h2. The basics
30
-
31
- <pre>
32
- # Generate new project
33
- shell> $ gigantron project
34
- create
35
- create tasks
36
- create db
37
- create models
38
- create lib
39
- create test
40
- create Rakefile
41
- create database.yml
42
- create initialize.rb
43
- create tasks/import.rake
44
- create test/test_helper.rb
45
- create test/models
46
- create test/tasks
47
- create test/tasks/test_import.rb
48
- dependency install_rubigen_scripts
49
- create script
50
- create script/generate
51
- create script/destroy
52
- shell> $ cd project
53
- # Create new model
54
- shell> $ script/generate model modis
55
- exists models/
56
- create models/modis.rb
57
- exists test/
58
- exists test/models/
59
- create test/models/test_modis.rb
60
- shell> $ script/generate task modis_to_kml
61
- exists tasks/
62
- create tasks/modis_to_kml.rake
63
- exists test/
64
- exists test/tasks/
65
- create test/tasks/test_modis_to_kml.rb
66
- </pre>
67
-
68
- One can edit these files to add functionality. Gigantron by default includes
69
- ActiveSupport for convenience.
70
-
71
- h2. Hacking
72
-
73
- Gigantron is super minimal now, so modifying it is pretty easy. The gigantron
74
- application generator (that which is invoked by the @gigantron@ command) lives
75
- in @app_generators/gigantron/@. The template files and the template dir
76
- structure are in @app_generators/gigantron/templates/@. When adding new
77
- templates, directories, or files, add the names first to the tests in
78
- @test/test_gigantron_generator.rb@, then the stubs to
79
- @app_generators/gigantron/templates@, and then finally describe them in the
80
- manifest section of @app_generators/gigantron/gigantron_generator.rb@. It
81
- should look something like
82
-
83
- <pre syntax="ruby">
84
- def manifest
85
- record do |m|
86
- ...
87
- m.file "new_file", "new_file" #straight file copy
88
- m.directory "my_new_dir" #create directory
89
- m.template "new_thing", "new_thing" #runs file through ERB when copying
90
- end
91
- end
92
- </pre>
93
-
94
- It might be handy to know that in @gigantron_generator.rb@ the name provided to the generator can be referenced as @@name@. This can be used like
95
-
96
- <pre syntax="ruby">
97
- record do |m|
98
- m.file "renamed_file", "#{@name}_file"
99
- end
100
- </pre>
101
-
102
- The same value is available to your templates as just @name@. You can template
103
- a file like
104
-
105
- <pre syntax="ruby">
106
- class Test<%%= name.camelcase %> < Test::Unit::TestCase
107
- ...
108
- end
109
- </pre>
110
-
111
- The same process applies to the model and task generator for gigantron
112
- projects. These generators live in @gigantron_generators/@. Modifying them
113
- is exactly the same as modifying the application generator.
114
-
115
- All of this is pretty vanilla "RubiGen":http://rubigen.rubyforge.org, so if in doubt, check out the docos on that fine piece of work.
116
-
117
- The only other place for code in Gigantron is is @lib/gigantron/tasks/@ where a few boilerplate test and db tasks live. I think I ripped the test tasks off of rails.
118
-
119
- If you have any questions, do contact me. I am interested in anything that will make Gigantron suck less and be useful to people.
120
-
121
- h2. How to submit patches
122
-
123
-
124
- * github: "http://github.com/schleyfox/gigantron/tree/master":http://github.com/schleyfox/gigantron/tree/master
125
-
126
- <pre>git clone git://github.com/schleyfox/gigantron.git</pre>
127
-
128
-
129
- h3. Build and test instructions
130
-
131
- <pre>cd gigantron
132
- cp test/template_database.yml.example test/template_database.yml
133
- vim test/template_database.yml
134
- rake test
135
- rake install_gem</pre>
136
-
137
-
138
- h2. License
139
-
140
- This code is free to use under the terms of the MIT license.
141
-
142
- h2. Contact
143
-
144
- Comments are welcome. Send an email to "Ben Hughes":mailto:ben@pixelmachine.org
145
-