ardumper 1.0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- data/LICENSE +20 -0
- data/README +80 -0
- data/Rakefile +79 -0
- data/init.rb +3 -0
- data/lib/ar_dumper.rb +4 -0
- data/lib/ar_dumper_active_record.rb +92 -0
- data/lib/ar_dumper_base.rb +465 -0
- data/lib/ar_dumper_controller.rb +51 -0
- metadata +61 -0
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2007 Blythe Dunham
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README
ADDED
@@ -0,0 +1,80 @@
|
|
1
|
+
AR DUMP HER!!!!!
|
2
|
+
|
3
|
+
Project Full Name: ArDumper
|
4
|
+
Submitted Description: ArDumper is designed to dump volume rails activerecord data to a file in chunks.
|
5
|
+
|
6
|
+
Options include:
|
7
|
+
* paginating arguments to control page size
|
8
|
+
* finder options to provide control over the data set dumped
|
9
|
+
* field/method options provide control over content dumped for each activerecord.
|
10
|
+
|
11
|
+
Current supported file types are csv, xml and yaml.
|
12
|
+
|
13
|
+
Other features include:
|
14
|
+
* send_file support for ActiveController to send the file
|
15
|
+
|
16
|
+
|
17
|
+
|
18
|
+
#############################################################################
|
19
|
+
ArDumper
|
20
|
+
############################################################################
|
21
|
+
Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string. Specify the page_size used to paginate
|
22
|
+
records and flush files.
|
23
|
+
|
24
|
+
SPECIFY OUTPUT
|
25
|
+
* <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
26
|
+
* <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
27
|
+
* <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
28
|
+
* <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
29
|
+
* <tt>:methods</tt> - a list of the methods to be called on the object
|
30
|
+
* <tt>:procs</tt> - hash of header name to Proc object
|
31
|
+
|
32
|
+
FINDER METHODS
|
33
|
+
* <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
34
|
+
|
35
|
+
FORMAT HEADER
|
36
|
+
* <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
37
|
+
when an array is specified uses this instead of the fields
|
38
|
+
when true or by default prints the fields
|
39
|
+
when false does not include a header
|
40
|
+
* <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
41
|
+
:titleize is chosen, then the Header value is "Email Address"
|
42
|
+
|
43
|
+
* <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
44
|
+
the objects. Each record will be root_id. For example, contact_2348
|
45
|
+
|
46
|
+
FILENAME AND TARGET
|
47
|
+
<tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
48
|
+
* :string prints to string. Do not use with large data sets
|
49
|
+
* :tmp_file. Use a temporary file that is destroyed when the process exists
|
50
|
+
* :file. Use a standard file
|
51
|
+
<tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
52
|
+
<tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
53
|
+
:file_extension is only available when +:target_type_type => :file+
|
54
|
+
<tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
55
|
+
|
56
|
+
Specify which attributes to include and exclude
|
57
|
+
Girlfriend.dumper :yml, :only => [:name, :rating]
|
58
|
+
|
59
|
+
Boyfriend.dumper :csv, :except => [:personality]
|
60
|
+
|
61
|
+
|
62
|
+
Use :methods to include methods on the record
|
63
|
+
|
64
|
+
BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
65
|
+
Output..
|
66
|
+
..other attributes.., 25, doughnuts
|
67
|
+
|
68
|
+
To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
69
|
+
record - the active record
|
70
|
+
result_set - the current result set
|
71
|
+
counter - the number of the record
|
72
|
+
page_num - the page number
|
73
|
+
target - the file/string target
|
74
|
+
proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
75
|
+
Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
76
|
+
|
77
|
+
<girlfriend>
|
78
|
+
... other attributes and methods ...
|
79
|
+
<my_rating>HIGHLY RATED</my_rating>
|
80
|
+
</girlfriend>
|
data/Rakefile
ADDED
@@ -0,0 +1,79 @@
|
|
1
|
+
require 'rake/testtask'
|
2
|
+
require 'rake/rdoctask'
|
3
|
+
require 'rake/gempackagetask'
|
4
|
+
require 'rake/contrib/sshpublisher'
|
5
|
+
|
6
|
+
PKG_NAME = "ardumper"
|
7
|
+
PKG_VERSION = "1.0.1.1"
|
8
|
+
PKG_FILE_NAME = "#{PKG_NAME}-#{PKG_VERSION}"
|
9
|
+
RUBY_FORGE_PROJECT = "ardumper"
|
10
|
+
|
11
|
+
spec = Gem::Specification.new do |s|
|
12
|
+
s.name = PKG_NAME
|
13
|
+
s.version = PKG_VERSION
|
14
|
+
s.platform = Gem::Platform::RUBY
|
15
|
+
s.summary = "Provides developers with the ability to define any dependencies on external plugins directly from within another plugin."
|
16
|
+
s.files = FileList["{lib,tasks}/**/*"].to_a + %w(init.rb LICENSE Rakefile README)
|
17
|
+
s.require_path = "lib"
|
18
|
+
s.autorequire = PKG_NAME
|
19
|
+
s.has_rdoc = true
|
20
|
+
s.test_files = nil#Dir["test/**/*_test.rb"]
|
21
|
+
s.add_dependency "rails", ">= 1.2.0"
|
22
|
+
|
23
|
+
s.author = "Blythe Dunham"
|
24
|
+
s.email = "blythe@spongecell.com"
|
25
|
+
s.homepage = "http://spongetech.wordpress.com/"
|
26
|
+
end
|
27
|
+
|
28
|
+
desc 'Default: run unit tests.'
|
29
|
+
task :default => :test
|
30
|
+
|
31
|
+
desc 'Test the ar_dumper plugin.'
|
32
|
+
Rake::TestTask.new(:test) do |t|
|
33
|
+
t.libs << '.\lib'
|
34
|
+
t.pattern = 'test/**/*_test.rb'
|
35
|
+
t.verbose = true
|
36
|
+
end
|
37
|
+
|
38
|
+
desc 'Generate documentation for the ar_test plugin.'
|
39
|
+
Rake::RDocTask.new(:rdoc) do |rdoc|
|
40
|
+
rdoc.rdoc_dir = 'rdoc'
|
41
|
+
rdoc.title = 'ArP'
|
42
|
+
rdoc.options << '--line-numbers' << '--inline-source'
|
43
|
+
rdoc.rdoc_files.include('README')
|
44
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
45
|
+
end
|
46
|
+
|
47
|
+
Rake::GemPackageTask.new(spec) do |p|
|
48
|
+
p.gem_spec = spec
|
49
|
+
p.need_tar = true
|
50
|
+
p.need_zip = true
|
51
|
+
end
|
52
|
+
|
53
|
+
desc "Publish the API documentation"
|
54
|
+
task :pdoc => [:rdoc] do
|
55
|
+
#Rake::SshDirPublisher.new("spongecell.com", "/var/home/#{PKG_NAME}", "rdoc").upload
|
56
|
+
Rake::RubyForgePublisher.new(RUBY_FORGE_PROJECT, RUBY_FORGE_USER).upload
|
57
|
+
end
|
58
|
+
|
59
|
+
desc "Publish the API docs and gem"
|
60
|
+
task :publish => [:pdoc, :release]
|
61
|
+
|
62
|
+
|
63
|
+
|
64
|
+
desc "Publish the release files to RubyForge"
|
65
|
+
task :release => [:gem, :package] do
|
66
|
+
require 'rubyforge'
|
67
|
+
|
68
|
+
options = {"cookie_jar" => RubyForge::COOKIE_F}
|
69
|
+
options["password"] = ENV["RUBY_FORGE_PASSWORD"] if ENV["RUBY_FORGE_PASSWORD"]
|
70
|
+
ruby_forge = RubyForge.new
|
71
|
+
ruby_forge.login
|
72
|
+
|
73
|
+
%w( gem tgz zip ).each do |ext|
|
74
|
+
file = "pkg/#{PKG_FILE_NAME}.#{ext}"
|
75
|
+
puts "Releasing #{File.basename(file)}..."
|
76
|
+
|
77
|
+
ruby_forge.add_release(RUBY_FORGE_PROJECT, PKG_NAME, PKG_VERSION, file)
|
78
|
+
end
|
79
|
+
end
|
data/init.rb
ADDED
data/lib/ar_dumper.rb
ADDED
@@ -0,0 +1,92 @@
|
|
1
|
+
#############################################################################
|
2
|
+
# ArDumper
|
3
|
+
#############################################################################
|
4
|
+
# Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string. Specify the page_size used to paginate
|
5
|
+
# records and flush files.
|
6
|
+
#
|
7
|
+
# SPECIFY OUTPUT
|
8
|
+
# * <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
9
|
+
# * <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
10
|
+
# * <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
11
|
+
# * <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
12
|
+
# * <tt>:methods</tt> - a list of the methods to be called on the object
|
13
|
+
# * <tt>:procs</tt> - hash of header name to Proc object
|
14
|
+
#
|
15
|
+
# FINDER METHODS
|
16
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
17
|
+
#
|
18
|
+
# FORMAT HEADER
|
19
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
20
|
+
# when an array is specified uses this instead of the fields
|
21
|
+
# when true or by default prints the fields
|
22
|
+
# when false does not include a header
|
23
|
+
# * <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
24
|
+
# :titleize is chosen, then the Header value is "Email Address"
|
25
|
+
#
|
26
|
+
# * <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
27
|
+
# the objects. Each record will be root_id. For example, contact_2348
|
28
|
+
#
|
29
|
+
# FILENAME AND TARGET
|
30
|
+
# <tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
31
|
+
# * :string prints to string. Do not use with large data sets
|
32
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
33
|
+
# * :file. Use a standard file
|
34
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
35
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
36
|
+
# :file_extension is only available when +:target_type_type => :file+
|
37
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
38
|
+
#
|
39
|
+
# Specify which attributes to include and exclude
|
40
|
+
# Girlfriend.dumper :yml, :only => [:name, :rating]
|
41
|
+
#
|
42
|
+
# Boyfriend.dumper :csv, :except => [:personality]
|
43
|
+
#
|
44
|
+
#
|
45
|
+
# Use :methods to include methods on the record
|
46
|
+
#
|
47
|
+
# BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
48
|
+
# Output..
|
49
|
+
# ..other attributes.., 25, doughnuts
|
50
|
+
#
|
51
|
+
# To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
52
|
+
# record - the active record
|
53
|
+
# result_set - the current result set
|
54
|
+
# counter - the number of the record
|
55
|
+
# page_num - the page number
|
56
|
+
# target - the file/string target
|
57
|
+
# proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
58
|
+
# Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
59
|
+
#
|
60
|
+
# <girlfriend>
|
61
|
+
# # ... other attributes and methods ...
|
62
|
+
# <my_rating>HIGHLY RATED</my_rating>
|
63
|
+
# </girlfriend>
|
64
|
+
|
65
|
+
|
66
|
+
|
67
|
+
class ActiveRecord::Base
|
68
|
+
def self.to_csv(options={})
|
69
|
+
dump_to_string :csv, options
|
70
|
+
end
|
71
|
+
|
72
|
+
def self.dump_to_yaml(options={})
|
73
|
+
dumper :yml, options
|
74
|
+
end
|
75
|
+
|
76
|
+
#dump to specified format
|
77
|
+
def self.dumper(format, options={})
|
78
|
+
ArDumper.new(self, options).dump(format)
|
79
|
+
end
|
80
|
+
|
81
|
+
#dump to string
|
82
|
+
def self.dumper_to_string(format, options={})
|
83
|
+
dumper(format, options.update({:target_type => :string}))
|
84
|
+
end
|
85
|
+
|
86
|
+
#dump_him dump_her and dump_em for fun
|
87
|
+
class << self
|
88
|
+
%w( dump_her dump_him dump_em ).each {|meth| alias_method meth, :dumper}
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
|
@@ -0,0 +1,465 @@
|
|
1
|
+
|
2
|
+
|
3
|
+
require 'csv'
|
4
|
+
|
5
|
+
|
6
|
+
class ArDumperException < Exception
|
7
|
+
end
|
8
|
+
|
9
|
+
|
10
|
+
#############################################################################
|
11
|
+
# ArDumper
|
12
|
+
#############################################################################
|
13
|
+
# Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string
|
14
|
+
#
|
15
|
+
#
|
16
|
+
# * <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
17
|
+
# * <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
18
|
+
|
19
|
+
# * <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
20
|
+
# * <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
21
|
+
# * <tt>:methods</tt> - a list of the methods to be called on the object
|
22
|
+
# * <tt>:procs</tt> - hash of header name to Proc object
|
23
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
24
|
+
#
|
25
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
26
|
+
# when an array is specified uses this instead of the fields
|
27
|
+
# when true or by default prints the fields
|
28
|
+
# when false does not include a header
|
29
|
+
# * <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
30
|
+
# :titleize is chosen, then the Header value is "Email Address"
|
31
|
+
#
|
32
|
+
# * <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
33
|
+
# the objects. Each record will be root_id. For example, contact_2348
|
34
|
+
#
|
35
|
+
# Filename and target options
|
36
|
+
# <tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
37
|
+
# * :string prints to string. Do not use with large data sets
|
38
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
39
|
+
# * :file. Use a standard file
|
40
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
41
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
42
|
+
# :file_extension is only available when +:target_type_type => :file+
|
43
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
44
|
+
#
|
45
|
+
#
|
46
|
+
# Use :methods to include methods on the record
|
47
|
+
#
|
48
|
+
# BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
49
|
+
# ..other attributes.., 25, doughnuts
|
50
|
+
#
|
51
|
+
# To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
52
|
+
# record - the active record
|
53
|
+
# result_set - the current result set
|
54
|
+
# counter - the number of the record
|
55
|
+
# page_num - the page number
|
56
|
+
# target - the file/string target
|
57
|
+
|
58
|
+
# proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
59
|
+
# Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
60
|
+
#
|
61
|
+
# <girlfriend>
|
62
|
+
# # ... other attributes and methods ...
|
63
|
+
# <my_rating>HIGHLY RATED</my_rating>
|
64
|
+
# </girlfriend>
|
65
|
+
#
|
66
|
+
class ArDumper
|
67
|
+
|
68
|
+
cattr_accessor :dumper_page_size #default page size
|
69
|
+
@@dumper_page_size ||= 50
|
70
|
+
|
71
|
+
cattr_accessor :dumper_file_path #default file path
|
72
|
+
@@dumper_file_path ||= ENV['TMPDIR']||ENV['TMP']||ENV['TEMP']||'.'
|
73
|
+
|
74
|
+
cattr_accessor :dumper_tmp_file_basename #default basename of temporary files
|
75
|
+
@@dumper_tmp_file_basename ||= 'ardumper'
|
76
|
+
|
77
|
+
cattr_accessor :csv_writer #default to faster csv if it is available
|
78
|
+
|
79
|
+
|
80
|
+
attr_reader :fields
|
81
|
+
attr_reader :klass
|
82
|
+
attr_reader :options
|
83
|
+
|
84
|
+
def initialize(klass, dump_options={})
|
85
|
+
@klass = klass
|
86
|
+
@options = dump_options
|
87
|
+
build_attribute_list
|
88
|
+
|
89
|
+
unless options[:text_format].nil? || String.new.respond_to?(options[:text_format])
|
90
|
+
raise ArDumperException.new("Invalid value for option :text_format #{options[:text_format]}")
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
# build a list of attributes, methods and procs
|
95
|
+
def build_attribute_list#:nodoc:
|
96
|
+
if options[:only]
|
97
|
+
options[:attributes] = Array(options[:only])
|
98
|
+
else
|
99
|
+
options[:attributes] = @klass.column_names - Array(options[:except]).collect { |e| e.to_s }
|
100
|
+
end
|
101
|
+
|
102
|
+
options[:attributes] = options[:attributes].collect{|attr| "#{attr}"}
|
103
|
+
options[:methods] = options[:methods].is_a?(Hash) ? options[:methods].values : Array(options[:methods])
|
104
|
+
|
105
|
+
#if procs are specified as an array separate the headers(keys) from the procs(values)
|
106
|
+
if options[:procs].is_a?(Hash)
|
107
|
+
options[:proc_headers]= options[:procs].keys
|
108
|
+
options[:procs]= options[:procs].values
|
109
|
+
else
|
110
|
+
options[:procs] = Array(options[:procs])
|
111
|
+
options[:proc_headers]||= Array.new
|
112
|
+
0.upto(options[:procs].size - options[:proc_headers].size - 1) {|idx| options[:proc_headers] << "proc_#{idx}" }
|
113
|
+
end
|
114
|
+
|
115
|
+
end
|
116
|
+
|
117
|
+
|
118
|
+
|
119
|
+
|
120
|
+
def dump(format)
|
121
|
+
case format.to_sym
|
122
|
+
when :csv
|
123
|
+
dump_to_csv
|
124
|
+
|
125
|
+
when :xml
|
126
|
+
dump_to_xml
|
127
|
+
|
128
|
+
when :yaml, :fixture, :yml
|
129
|
+
dump_to_fixture
|
130
|
+
|
131
|
+
else
|
132
|
+
raise ArDumperException.new("Unknown format #{format}. Please specify :csv, :xml, or :yml ")
|
133
|
+
end
|
134
|
+
end
|
135
|
+
|
136
|
+
|
137
|
+
# Wrapper around the outside of dumper
|
138
|
+
# See options in dumper
|
139
|
+
def dumper(file_extension=nil, header = nil, footer = nil, &block)
|
140
|
+
|
141
|
+
options[:counter] = -1
|
142
|
+
begin
|
143
|
+
#get the file parameters
|
144
|
+
target = prepare_target(file_extension)
|
145
|
+
target << header if header
|
146
|
+
ArDumper.paginate_dump_records(@klass, @options) do |records, page_num|
|
147
|
+
|
148
|
+
#save state on options to make it accessible by
|
149
|
+
#class and procs
|
150
|
+
options[:result_set] = records
|
151
|
+
options[:page_num] = page_num
|
152
|
+
|
153
|
+
records.each do |record|
|
154
|
+
options[:record] = record
|
155
|
+
yield record
|
156
|
+
end
|
157
|
+
|
158
|
+
#flush after each set
|
159
|
+
target.flush if target.respond_to?(:flush)
|
160
|
+
end
|
161
|
+
target << footer if footer
|
162
|
+
|
163
|
+
#final step close the options[:target]
|
164
|
+
ensure
|
165
|
+
target.close if target && target.respond_to?(:close)
|
166
|
+
end
|
167
|
+
|
168
|
+
options[:full_file_name]||target
|
169
|
+
end
|
170
|
+
|
171
|
+
#collect the record data into an array
|
172
|
+
def dump_record(record)
|
173
|
+
record_values = @options[:attributes].inject([]){|values, attr| values << record["#{attr}"]; values }
|
174
|
+
record_values = @options[:methods].inject(record_values) {|values, method| values << record.send(method); values }
|
175
|
+
record_values = @options[:procs].inject(record_values){|values, proc| values << proc.call(options); values }
|
176
|
+
record_values
|
177
|
+
end
|
178
|
+
|
179
|
+
|
180
|
+
#############################################################################
|
181
|
+
# XML Dumper
|
182
|
+
#############################################################################
|
183
|
+
#
|
184
|
+
# Dumps the data to an xml file
|
185
|
+
#
|
186
|
+
# Using the ActiveRecord version of dumper so we CANNOT specify fields that are not attributes
|
187
|
+
#
|
188
|
+
# In addition to options listed in +dumper+:
|
189
|
+
# * <tt>:xml</tt> - xml options for ActiveRecord .to_xml
|
190
|
+
def dump_to_xml
|
191
|
+
|
192
|
+
#preserve the original skip instruct
|
193
|
+
skip_instruct = @options[:skip_instruct].is_a?(TrueClass)
|
194
|
+
|
195
|
+
#use the fields if :only is not specified in the xml options
|
196
|
+
xml_options ={:only => @options[:only],
|
197
|
+
:except => @options[:except],
|
198
|
+
:methods => @options[:methods],
|
199
|
+
:procs => @options[:procs]
|
200
|
+
}
|
201
|
+
|
202
|
+
xml_options.update(@options[:xml]) if @options[:xml]
|
203
|
+
|
204
|
+
#do not instruct for each set
|
205
|
+
xml_options[:skip_instruct] = true
|
206
|
+
xml_options[:indent]||=2
|
207
|
+
xml_options[:margin] = xml_options[:margin].to_i + 1
|
208
|
+
|
209
|
+
#set the variable on the options
|
210
|
+
options[:xml] = xml_options
|
211
|
+
|
212
|
+
|
213
|
+
#builder for header and footer
|
214
|
+
builder_options = {
|
215
|
+
:margin => xml_options[:margin] - 1,
|
216
|
+
:indent => xml_options[:indent]
|
217
|
+
}
|
218
|
+
|
219
|
+
options[:root] = (options[:root] || @klass.to_s.underscore.pluralize).to_s
|
220
|
+
|
221
|
+
|
222
|
+
#use the builder to make sure we are indented properly
|
223
|
+
builder = Builder::XmlMarkup.new(builder_options.clone)
|
224
|
+
builder.instruct! unless skip_instruct
|
225
|
+
builder << "<#{options[:root]}>\n"
|
226
|
+
header = builder.target!
|
227
|
+
|
228
|
+
#get the footer. Using the builder will make sure we are indented properly
|
229
|
+
builder = Builder::XmlMarkup.new(builder_options)
|
230
|
+
builder << "</#{options[:root]}>"
|
231
|
+
footer = builder.target!
|
232
|
+
|
233
|
+
dumper(:xml, header, footer) do |record|
|
234
|
+
options[:target] << record.to_xml(xml_options.dup)
|
235
|
+
end
|
236
|
+
end
|
237
|
+
|
238
|
+
#############################################################################
|
239
|
+
# Yaml/Fixture Dumper
|
240
|
+
#############################################################################
|
241
|
+
# dumps the data to a fixture file
|
242
|
+
# In addition to options listed in +dumper+:
|
243
|
+
# <tt>:root</tt> Basename of the record. Defaults to the class name so each record is named customer_1
|
244
|
+
def dump_to_fixture
|
245
|
+
basename = @options[:root]||@klass.table_name.singularize
|
246
|
+
header_list = build_header_list
|
247
|
+
|
248
|
+
# doctor the yaml a bit to print the hash header at the top
|
249
|
+
# instead of each record
|
250
|
+
dumper(:yml, "---\s") do |record|
|
251
|
+
record_data = Hash.new
|
252
|
+
dump_record(record).each_with_index{|field, idx| record_data[header_list[idx].to_s] = field.to_s }
|
253
|
+
options[:target] << {"#{basename}_#{record.id}" => record_data}.to_yaml.gsub(/^---\s\n/, "\n")
|
254
|
+
end
|
255
|
+
end
|
256
|
+
|
257
|
+
|
258
|
+
|
259
|
+
#############################################################################
|
260
|
+
# CSV DUMPER
|
261
|
+
#############################################################################
|
262
|
+
#
|
263
|
+
# Dump csv data
|
264
|
+
#
|
265
|
+
# * <tt>:csv</tt> - any options to pass to csv parser.
|
266
|
+
# :col_sep Example + :csv => {:col_sep => "\t"} +
|
267
|
+
# :row_sep Row seperator
|
268
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50
|
269
|
+
def dump_to_csv
|
270
|
+
header = nil
|
271
|
+
@options[:csv]||={}
|
272
|
+
|
273
|
+
if !@options[:header].is_a?(FalseClass)
|
274
|
+
header_list = build_header_list
|
275
|
+
#print the header unless set to false
|
276
|
+
header = write_csv_row(header_list)
|
277
|
+
end
|
278
|
+
|
279
|
+
dumper(:csv, header) do |record|
|
280
|
+
options[:target] << write_csv_row(dump_record(record))
|
281
|
+
end
|
282
|
+
end
|
283
|
+
|
284
|
+
# Write out the csv row using the selected csv writer
|
285
|
+
def write_csv_row(row_data, header_list=[])#:nodoc:
|
286
|
+
if csv_writer == :faster
|
287
|
+
::FasterCSV::Row.new(header_list, row_data).to_csv(@options[:csv])
|
288
|
+
else
|
289
|
+
::CSV.generate_line(row_data, @options[:csv][:col_sep], @options[:csv][:row_sep]) + (@options[:csv][:row_sep]||"\n")
|
290
|
+
end
|
291
|
+
end
|
292
|
+
|
293
|
+
#Try to use the FasterCSV if it exists
|
294
|
+
#otherwise use csv
|
295
|
+
def csv_writer #:nodoc:
|
296
|
+
unless @@csv_writer
|
297
|
+
@@csv_writer = :faster
|
298
|
+
begin
|
299
|
+
require 'faster_csv'
|
300
|
+
::FasterCSV
|
301
|
+
rescue Exception => exc
|
302
|
+
@@csv_writer = :normal
|
303
|
+
end
|
304
|
+
end
|
305
|
+
@@csv_writer
|
306
|
+
end
|
307
|
+
|
308
|
+
# Returns an array with the header names
|
309
|
+
# This will be in the same order as the data returned by dump_record
|
310
|
+
# attributes + methods + procs
|
311
|
+
#
|
312
|
+
# <tt>:header</tt> The header defaults to the attributes and method names. When set
|
313
|
+
# to false no header is specified
|
314
|
+
# * +hash+ A map from attribute or method name to Header column name
|
315
|
+
# * +array+ A list in the same order that is used to display record data
|
316
|
+
#
|
317
|
+
# <tt>:procs</tt> If a hash, then the keys are the names. If an array, then use proc_1, proc_2, etc
|
318
|
+
# <tt>:text_format</tt> Format names with a text format such as +:titlieze+, +:dasherize+, +:underscore+
|
319
|
+
def build_header_list#:nodoc:
|
320
|
+
|
321
|
+
header_options = options[:header]
|
322
|
+
columns = @options[:attributes] + @options[:methods]
|
323
|
+
header_names =
|
324
|
+
if header_options.is_a?(Hash)
|
325
|
+
header_options.symbolize_keys!
|
326
|
+
|
327
|
+
#Get the header for each attribute and method
|
328
|
+
columns.collect{|field|(header_options[field.to_sym]||field).to_s}
|
329
|
+
|
330
|
+
#ordered by attributes, methods, then procs
|
331
|
+
elsif header_options.is_a?(Array)
|
332
|
+
header_names = header_options
|
333
|
+
header_names.concat(columns[header_options.length..-1]) if header_names.length < columns.length
|
334
|
+
|
335
|
+
#default to column names
|
336
|
+
else
|
337
|
+
columns
|
338
|
+
end
|
339
|
+
|
340
|
+
#add process names
|
341
|
+
header_names.concat(options[:proc_headers])
|
342
|
+
|
343
|
+
#format names with a text format such as titlieze, dasherize, underscore
|
344
|
+
header_names.collect!{|n| n.to_s.send(options[:text_format])} if options[:text_format]
|
345
|
+
|
346
|
+
header_names
|
347
|
+
end
|
348
|
+
|
349
|
+
|
350
|
+
|
351
|
+
|
352
|
+
|
353
|
+
# Create the options[:target](file) based on these options. The options[:target] must respond to <<
|
354
|
+
# Current options[:target]s are :string, :file, :tempfile
|
355
|
+
#
|
356
|
+
# <tt> :target_type</tt> The options[:target] for the data. Defaults to +:file+
|
357
|
+
# * :string prints to string. Do not use with large data sets
|
358
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
359
|
+
# * :file. Use a standard file
|
360
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
361
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
362
|
+
# :file_extension is only available when +:target_type_type => :file+
|
363
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
364
|
+
|
365
|
+
def prepare_target(file_extension = nil)
|
366
|
+
|
367
|
+
options[:target] = case options[:target_type]
|
368
|
+
#to string option dumps to a string instead of a file
|
369
|
+
when :string
|
370
|
+
String.new
|
371
|
+
|
372
|
+
#use a temporary file
|
373
|
+
#open a temporary file with the basename specified by filename
|
374
|
+
#defaults to the value of one of the environment variables TMPDIR, TMP, or TEMP
|
375
|
+
when :tmp_file
|
376
|
+
|
377
|
+
Tempfile.open(options[:filename]||(@@dumper_tmp_file_basename+@klass.name.downcase),
|
378
|
+
options[:file_path]||@@dumper_file_path)
|
379
|
+
|
380
|
+
#default to a real file
|
381
|
+
else
|
382
|
+
extension = options[:file_extension]||file_extension
|
383
|
+
mode = options[:append_to_file].is_a?(TrueClass)? 'a' : 'w'
|
384
|
+
filename = options[:filename]||"#{@@dumper_tmp_file_basename}.#{@klass.name.downcase}.#{Time.now.to_f.to_s}.#{extension}"
|
385
|
+
|
386
|
+
#append an extension unless one already exists
|
387
|
+
filename += ".#{extension}" if extension && !filename =~ /\.\w*$/
|
388
|
+
|
389
|
+
#get the file path if the filename does not contain one
|
390
|
+
if File.basename(filename) == filename
|
391
|
+
path = options[:file_path]||@@dumper_file_path
|
392
|
+
filename = File.join(path, filename) unless path.blank?
|
393
|
+
end
|
394
|
+
|
395
|
+
|
396
|
+
File.open(filename, mode)
|
397
|
+
end
|
398
|
+
|
399
|
+
options[:full_file_name] = options[:target].path if options[:target].respond_to?(:path)
|
400
|
+
options[:target]
|
401
|
+
end
|
402
|
+
|
403
|
+
|
404
|
+
|
405
|
+
#############################################################################
|
406
|
+
# Pagination Helpers... OR YOU COULD INSTALL paginate_find plugin!
|
407
|
+
#############################################################################
|
408
|
+
#
|
409
|
+
#
|
410
|
+
# Quick and dirty paginate to loop thru the records page by page
|
411
|
+
# Options are:
|
412
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
413
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50. Set to false to disable pagination
|
414
|
+
#
|
415
|
+
def self.paginate_dump_records(klass, options={}, &block)
|
416
|
+
finder_options = (options[:find]||{}).clone
|
417
|
+
|
418
|
+
#pagination is not needed when :page_size => false
|
419
|
+
if options[:page_size].is_a?(FalseClass)
|
420
|
+
yield klass.find(:all, finder_options), 0
|
421
|
+
return
|
422
|
+
end
|
423
|
+
|
424
|
+
options[:page_size]||= dumper_page_size
|
425
|
+
|
426
|
+
#limit becomes the maximum amount of records to pull
|
427
|
+
max_records = finder_options[:limit]
|
428
|
+
page_num = 0
|
429
|
+
finder_options[:limit] = compute_page_size(max_records, page_num, options[:page_size])
|
430
|
+
records = []
|
431
|
+
while (finder_options[:limit] > 0 && (page_num == 0 || records.length == options[:page_size]))
|
432
|
+
records = klass.find :all, finder_options.update(:offset => page_num * options[:page_size])
|
433
|
+
|
434
|
+
yield records, page_num
|
435
|
+
page_num = page_num + 1
|
436
|
+
|
437
|
+
#calculate the limit if an original limit (max_records) was set
|
438
|
+
finder_options[:limit] = compute_page_size(max_records, page_num, options[:page_size])
|
439
|
+
end
|
440
|
+
end
|
441
|
+
|
442
|
+
def self.compute_page_size(max_records, page_num, page_size)#:nodoc:
|
443
|
+
max_records ? [(max_records - (page_num * page_size)), page_size].min : page_size
|
444
|
+
end
|
445
|
+
|
446
|
+
# Quick and dirty paginate to loop thru each page
|
447
|
+
# Options are:
|
448
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
449
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50. Set to false to disable pagination
|
450
|
+
def self.paginate_each_record(klass, options={}, &block)
|
451
|
+
counter = -1
|
452
|
+
paginate_dump_records(klass, options) do |records, page_num|
|
453
|
+
records.each do |record|
|
454
|
+
yield record, (counter +=1)
|
455
|
+
end
|
456
|
+
end
|
457
|
+
end
|
458
|
+
end
|
459
|
+
|
460
|
+
|
461
|
+
|
462
|
+
|
463
|
+
|
464
|
+
|
465
|
+
|
@@ -0,0 +1,51 @@
|
|
1
|
+
class ActionController::Base
|
2
|
+
|
3
|
+
#Change to :file to use regular files
|
4
|
+
cattr_accessor :dumper_target
|
5
|
+
@@dumper_target = :tmp_file
|
6
|
+
|
7
|
+
|
8
|
+
# send a dumped file
|
9
|
+
# Options are:
|
10
|
+
# * <tt>klass</tt> - the active record to use. Animal or Contact or Author or Tumama
|
11
|
+
# * <tt>send_options</tt> send_options include send_file options such as +:type+ and +:disposition+
|
12
|
+
# * <tt>dump_options</tt> options to send to the dumper. These options are in ArDumper.dump_to_csv
|
13
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['features.hair = ?', 'blonde'], :include => :features} +
|
14
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
15
|
+
# when an array is specified uses this instead of the fields
|
16
|
+
# when true or by default prints the fields
|
17
|
+
# when false does not include a header
|
18
|
+
# * <tt>:csv</tt> - any options to pass to csv parser. Example + :csv => {:col_sep => "\t"} +
|
19
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50
|
20
|
+
def send_file_dump(format, klass, send_options={}, dump_options={})
|
21
|
+
|
22
|
+
#use the base name of the dump file name if specified
|
23
|
+
send_options[:filename] ||= File.basename(dump_options[:filename]) if dump_options[:filename]
|
24
|
+
|
25
|
+
#never delete the file
|
26
|
+
dump_options[:delete_file] = false
|
27
|
+
|
28
|
+
#use temporary files unless otherwise specified
|
29
|
+
dump_options[:target_type]||= @@dumper_target
|
30
|
+
|
31
|
+
#send_options[:type] = 'application/xml; charset=utf-8;'
|
32
|
+
if send_options[:type].nil? && send_options[:disposition] == 'inline'
|
33
|
+
send_options[:type] =
|
34
|
+
case format.to_sym
|
35
|
+
when :xml then 'application/xml; charset=utf-8;'
|
36
|
+
when :csv then 'application/csv; charset=utf-8;'
|
37
|
+
when :yml then 'text/html; charset=utf-8;'
|
38
|
+
end
|
39
|
+
|
40
|
+
end
|
41
|
+
|
42
|
+
target = klass.dumper(format, dump_options)
|
43
|
+
|
44
|
+
if dump_options[:target_type] == :string
|
45
|
+
send_data(target, send_options)
|
46
|
+
else
|
47
|
+
send_file(target, send_options)
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
end
|
metadata
ADDED
@@ -0,0 +1,61 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
rubygems_version: 0.9.4
|
3
|
+
specification_version: 1
|
4
|
+
name: ardumper
|
5
|
+
version: !ruby/object:Gem::Version
|
6
|
+
version: 1.0.1.1
|
7
|
+
date: 2007-08-09 00:00:00 -07:00
|
8
|
+
summary: Provides developers with the ability to define any dependencies on external plugins directly from within another plugin.
|
9
|
+
require_paths:
|
10
|
+
- lib
|
11
|
+
email: blythe@spongecell.com
|
12
|
+
homepage: http://spongetech.wordpress.com/
|
13
|
+
rubyforge_project:
|
14
|
+
description:
|
15
|
+
autorequire: ardumper
|
16
|
+
default_executable:
|
17
|
+
bindir: bin
|
18
|
+
has_rdoc: true
|
19
|
+
required_ruby_version: !ruby/object:Gem::Version::Requirement
|
20
|
+
requirements:
|
21
|
+
- - ">"
|
22
|
+
- !ruby/object:Gem::Version
|
23
|
+
version: 0.0.0
|
24
|
+
version:
|
25
|
+
platform: ruby
|
26
|
+
signing_key:
|
27
|
+
cert_chain:
|
28
|
+
post_install_message:
|
29
|
+
authors:
|
30
|
+
- Blythe Dunham
|
31
|
+
files:
|
32
|
+
- lib/ar_dumper.rb
|
33
|
+
- lib/ar_dumper_active_record.rb
|
34
|
+
- lib/ar_dumper_base.rb
|
35
|
+
- lib/ar_dumper_controller.rb
|
36
|
+
- init.rb
|
37
|
+
- LICENSE
|
38
|
+
- Rakefile
|
39
|
+
- README
|
40
|
+
test_files: []
|
41
|
+
|
42
|
+
rdoc_options: []
|
43
|
+
|
44
|
+
extra_rdoc_files: []
|
45
|
+
|
46
|
+
executables: []
|
47
|
+
|
48
|
+
extensions: []
|
49
|
+
|
50
|
+
requirements: []
|
51
|
+
|
52
|
+
dependencies:
|
53
|
+
- !ruby/object:Gem::Dependency
|
54
|
+
name: rails
|
55
|
+
version_requirement:
|
56
|
+
version_requirements: !ruby/object:Gem::Version::Requirement
|
57
|
+
requirements:
|
58
|
+
- - ">="
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: 1.2.0
|
61
|
+
version:
|