ardumper 1.0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/LICENSE +20 -0
- data/README +80 -0
- data/Rakefile +79 -0
- data/init.rb +3 -0
- data/lib/ar_dumper.rb +4 -0
- data/lib/ar_dumper_active_record.rb +92 -0
- data/lib/ar_dumper_base.rb +465 -0
- data/lib/ar_dumper_controller.rb +51 -0
- metadata +61 -0
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2007 Blythe Dunham
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README
ADDED
@@ -0,0 +1,80 @@
|
|
1
|
+
AR DUMP HER!!!!!
|
2
|
+
|
3
|
+
Project Full Name: ArDumper
|
4
|
+
Submitted Description: ArDumper is designed to dump volume rails activerecord data to a file in chunks.
|
5
|
+
|
6
|
+
Options include:
|
7
|
+
* paginating arguments to control page size
|
8
|
+
* finder options to provide control over the data set dumped
|
9
|
+
* field/method options provide control over content dumped for each activerecord.
|
10
|
+
|
11
|
+
Current supported file types are csv, xml and yaml.
|
12
|
+
|
13
|
+
Other features include:
|
14
|
+
* send_file support for ActiveController to send the file
|
15
|
+
|
16
|
+
|
17
|
+
|
18
|
+
#############################################################################
|
19
|
+
ArDumper
|
20
|
+
############################################################################
|
21
|
+
Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string. Specify the page_size used to paginate
|
22
|
+
records and flush files.
|
23
|
+
|
24
|
+
SPECIFY OUTPUT
|
25
|
+
* <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
26
|
+
* <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
27
|
+
* <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
28
|
+
* <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
29
|
+
* <tt>:methods</tt> - a list of the methods to be called on the object
|
30
|
+
* <tt>:procs</tt> - hash of header name to Proc object
|
31
|
+
|
32
|
+
FINDER METHODS
|
33
|
+
* <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
34
|
+
|
35
|
+
FORMAT HEADER
|
36
|
+
* <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
37
|
+
when an array is specified uses this instead of the fields
|
38
|
+
when true or by default prints the fields
|
39
|
+
when false does not include a header
|
40
|
+
* <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
41
|
+
:titleize is chosen, then the Header value is "Email Address"
|
42
|
+
|
43
|
+
* <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
44
|
+
the objects. Each record will be root_id. For example, contact_2348
|
45
|
+
|
46
|
+
FILENAME AND TARGET
|
47
|
+
<tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
48
|
+
* :string prints to string. Do not use with large data sets
|
49
|
+
* :tmp_file. Use a temporary file that is destroyed when the process exists
|
50
|
+
* :file. Use a standard file
|
51
|
+
<tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
52
|
+
<tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
53
|
+
:file_extension is only available when +:target_type_type => :file+
|
54
|
+
<tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
55
|
+
|
56
|
+
Specify which attributes to include and exclude
|
57
|
+
Girlfriend.dumper :yml, :only => [:name, :rating]
|
58
|
+
|
59
|
+
Boyfriend.dumper :csv, :except => [:personality]
|
60
|
+
|
61
|
+
|
62
|
+
Use :methods to include methods on the record
|
63
|
+
|
64
|
+
BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
65
|
+
Output..
|
66
|
+
..other attributes.., 25, doughnuts
|
67
|
+
|
68
|
+
To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
69
|
+
record - the active record
|
70
|
+
result_set - the current result set
|
71
|
+
counter - the number of the record
|
72
|
+
page_num - the page number
|
73
|
+
target - the file/string target
|
74
|
+
proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
75
|
+
Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
76
|
+
|
77
|
+
<girlfriend>
|
78
|
+
... other attributes and methods ...
|
79
|
+
<my_rating>HIGHLY RATED</my_rating>
|
80
|
+
</girlfriend>
|
data/Rakefile
ADDED
@@ -0,0 +1,79 @@
|
|
1
|
+
require 'rake/testtask'
|
2
|
+
require 'rake/rdoctask'
|
3
|
+
require 'rake/gempackagetask'
|
4
|
+
require 'rake/contrib/sshpublisher'
|
5
|
+
|
6
|
+
PKG_NAME = "ardumper"
|
7
|
+
PKG_VERSION = "1.0.1.1"
|
8
|
+
PKG_FILE_NAME = "#{PKG_NAME}-#{PKG_VERSION}"
|
9
|
+
RUBY_FORGE_PROJECT = "ardumper"
|
10
|
+
|
11
|
+
spec = Gem::Specification.new do |s|
|
12
|
+
s.name = PKG_NAME
|
13
|
+
s.version = PKG_VERSION
|
14
|
+
s.platform = Gem::Platform::RUBY
|
15
|
+
s.summary = "Provides developers with the ability to define any dependencies on external plugins directly from within another plugin."
|
16
|
+
s.files = FileList["{lib,tasks}/**/*"].to_a + %w(init.rb LICENSE Rakefile README)
|
17
|
+
s.require_path = "lib"
|
18
|
+
s.autorequire = PKG_NAME
|
19
|
+
s.has_rdoc = true
|
20
|
+
s.test_files = nil#Dir["test/**/*_test.rb"]
|
21
|
+
s.add_dependency "rails", ">= 1.2.0"
|
22
|
+
|
23
|
+
s.author = "Blythe Dunham"
|
24
|
+
s.email = "blythe@spongecell.com"
|
25
|
+
s.homepage = "http://spongetech.wordpress.com/"
|
26
|
+
end
|
27
|
+
|
28
|
+
desc 'Default: run unit tests.'
|
29
|
+
task :default => :test
|
30
|
+
|
31
|
+
desc 'Test the ar_dumper plugin.'
|
32
|
+
Rake::TestTask.new(:test) do |t|
|
33
|
+
t.libs << '.\lib'
|
34
|
+
t.pattern = 'test/**/*_test.rb'
|
35
|
+
t.verbose = true
|
36
|
+
end
|
37
|
+
|
38
|
+
desc 'Generate documentation for the ar_test plugin.'
|
39
|
+
Rake::RDocTask.new(:rdoc) do |rdoc|
|
40
|
+
rdoc.rdoc_dir = 'rdoc'
|
41
|
+
rdoc.title = 'ArP'
|
42
|
+
rdoc.options << '--line-numbers' << '--inline-source'
|
43
|
+
rdoc.rdoc_files.include('README')
|
44
|
+
rdoc.rdoc_files.include('lib/**/*.rb')
|
45
|
+
end
|
46
|
+
|
47
|
+
Rake::GemPackageTask.new(spec) do |p|
|
48
|
+
p.gem_spec = spec
|
49
|
+
p.need_tar = true
|
50
|
+
p.need_zip = true
|
51
|
+
end
|
52
|
+
|
53
|
+
desc "Publish the API documentation"
|
54
|
+
task :pdoc => [:rdoc] do
|
55
|
+
#Rake::SshDirPublisher.new("spongecell.com", "/var/home/#{PKG_NAME}", "rdoc").upload
|
56
|
+
Rake::RubyForgePublisher.new(RUBY_FORGE_PROJECT, RUBY_FORGE_USER).upload
|
57
|
+
end
|
58
|
+
|
59
|
+
desc "Publish the API docs and gem"
|
60
|
+
task :publish => [:pdoc, :release]
|
61
|
+
|
62
|
+
|
63
|
+
|
64
|
+
desc "Publish the release files to RubyForge"
|
65
|
+
task :release => [:gem, :package] do
|
66
|
+
require 'rubyforge'
|
67
|
+
|
68
|
+
options = {"cookie_jar" => RubyForge::COOKIE_F}
|
69
|
+
options["password"] = ENV["RUBY_FORGE_PASSWORD"] if ENV["RUBY_FORGE_PASSWORD"]
|
70
|
+
ruby_forge = RubyForge.new
|
71
|
+
ruby_forge.login
|
72
|
+
|
73
|
+
%w( gem tgz zip ).each do |ext|
|
74
|
+
file = "pkg/#{PKG_FILE_NAME}.#{ext}"
|
75
|
+
puts "Releasing #{File.basename(file)}..."
|
76
|
+
|
77
|
+
ruby_forge.add_release(RUBY_FORGE_PROJECT, PKG_NAME, PKG_VERSION, file)
|
78
|
+
end
|
79
|
+
end
|
data/init.rb
ADDED
data/lib/ar_dumper.rb
ADDED
@@ -0,0 +1,92 @@
|
|
1
|
+
#############################################################################
|
2
|
+
# ArDumper
|
3
|
+
#############################################################################
|
4
|
+
# Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string. Specify the page_size used to paginate
|
5
|
+
# records and flush files.
|
6
|
+
#
|
7
|
+
# SPECIFY OUTPUT
|
8
|
+
# * <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
9
|
+
# * <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
10
|
+
# * <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
11
|
+
# * <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
12
|
+
# * <tt>:methods</tt> - a list of the methods to be called on the object
|
13
|
+
# * <tt>:procs</tt> - hash of header name to Proc object
|
14
|
+
#
|
15
|
+
# FINDER METHODS
|
16
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
17
|
+
#
|
18
|
+
# FORMAT HEADER
|
19
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
20
|
+
# when an array is specified uses this instead of the fields
|
21
|
+
# when true or by default prints the fields
|
22
|
+
# when false does not include a header
|
23
|
+
# * <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
24
|
+
# :titleize is chosen, then the Header value is "Email Address"
|
25
|
+
#
|
26
|
+
# * <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
27
|
+
# the objects. Each record will be root_id. For example, contact_2348
|
28
|
+
#
|
29
|
+
# FILENAME AND TARGET
|
30
|
+
# <tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
31
|
+
# * :string prints to string. Do not use with large data sets
|
32
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
33
|
+
# * :file. Use a standard file
|
34
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
35
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
36
|
+
# :file_extension is only available when +:target_type_type => :file+
|
37
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
38
|
+
#
|
39
|
+
# Specify which attributes to include and exclude
|
40
|
+
# Girlfriend.dumper :yml, :only => [:name, :rating]
|
41
|
+
#
|
42
|
+
# Boyfriend.dumper :csv, :except => [:personality]
|
43
|
+
#
|
44
|
+
#
|
45
|
+
# Use :methods to include methods on the record
|
46
|
+
#
|
47
|
+
# BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
48
|
+
# Output..
|
49
|
+
# ..other attributes.., 25, doughnuts
|
50
|
+
#
|
51
|
+
# To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
52
|
+
# record - the active record
|
53
|
+
# result_set - the current result set
|
54
|
+
# counter - the number of the record
|
55
|
+
# page_num - the page number
|
56
|
+
# target - the file/string target
|
57
|
+
# proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
58
|
+
# Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
59
|
+
#
|
60
|
+
# <girlfriend>
|
61
|
+
# # ... other attributes and methods ...
|
62
|
+
# <my_rating>HIGHLY RATED</my_rating>
|
63
|
+
# </girlfriend>
|
64
|
+
|
65
|
+
|
66
|
+
|
67
|
+
class ActiveRecord::Base
|
68
|
+
def self.to_csv(options={})
|
69
|
+
dump_to_string :csv, options
|
70
|
+
end
|
71
|
+
|
72
|
+
def self.dump_to_yaml(options={})
|
73
|
+
dumper :yml, options
|
74
|
+
end
|
75
|
+
|
76
|
+
#dump to specified format
|
77
|
+
def self.dumper(format, options={})
|
78
|
+
ArDumper.new(self, options).dump(format)
|
79
|
+
end
|
80
|
+
|
81
|
+
#dump to string
|
82
|
+
def self.dumper_to_string(format, options={})
|
83
|
+
dumper(format, options.update({:target_type => :string}))
|
84
|
+
end
|
85
|
+
|
86
|
+
#dump_him dump_her and dump_em for fun
|
87
|
+
class << self
|
88
|
+
%w( dump_her dump_him dump_em ).each {|meth| alias_method meth, :dumper}
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
|
@@ -0,0 +1,465 @@
|
|
1
|
+
|
2
|
+
|
3
|
+
require 'csv'
|
4
|
+
|
5
|
+
|
6
|
+
class ArDumperException < Exception
|
7
|
+
end
|
8
|
+
|
9
|
+
|
10
|
+
#############################################################################
|
11
|
+
# ArDumper
|
12
|
+
#############################################################################
|
13
|
+
# Formats ActiveRecord data in chunks and dumps it to a file, temporary file, or string
|
14
|
+
#
|
15
|
+
#
|
16
|
+
# * <tt>:filename</tt> - the name of the file to create. By default will create a file based on the timestamp.
|
17
|
+
# * <tt>:file_extension</tt> - appends file extension unless one exists in file name
|
18
|
+
|
19
|
+
# * <tt>:only</tt> - a list of the attributes to be included. By default, all column_names are used.
|
20
|
+
# * <tt>:except</tt> - a list of the attributes to be excluded. By default, all column_names are used. This option is not available if +:only+ is used
|
21
|
+
# * <tt>:methods</tt> - a list of the methods to be called on the object
|
22
|
+
# * <tt>:procs</tt> - hash of header name to Proc object
|
23
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
24
|
+
#
|
25
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
26
|
+
# when an array is specified uses this instead of the fields
|
27
|
+
# when true or by default prints the fields
|
28
|
+
# when false does not include a header
|
29
|
+
# * <tt>:text_format</tt> - a string method such as :titleize, :dasherize, :underscore to format the on all the headers. If an attribute is :email_address and
|
30
|
+
# :titleize is chosen, then the Header value is "Email Address"
|
31
|
+
#
|
32
|
+
# * <tt>:root</tt> In xml, this is the name of the highest level list object. The plural of the class name is the default. For yml, this is the base name of the
|
33
|
+
# the objects. Each record will be root_id. For example, contact_2348
|
34
|
+
#
|
35
|
+
# Filename and target options
|
36
|
+
# <tt> :target_type</tt> The target_type for the data. Defaults to +:file+.
|
37
|
+
# * :string prints to string. Do not use with large data sets
|
38
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
39
|
+
# * :file. Use a standard file
|
40
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
41
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
42
|
+
# :file_extension is only available when +:target_type_type => :file+
|
43
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
44
|
+
#
|
45
|
+
#
|
46
|
+
# Use :methods to include methods on the record
|
47
|
+
#
|
48
|
+
# BigOle.dumper :csv, :methods => [:age, :favorite_food]
|
49
|
+
# ..other attributes.., 25, doughnuts
|
50
|
+
#
|
51
|
+
# To call any Proc's on the object(s) use :procs. All options are availabe to the process including
|
52
|
+
# record - the active record
|
53
|
+
# result_set - the current result set
|
54
|
+
# counter - the number of the record
|
55
|
+
# page_num - the page number
|
56
|
+
# target - the file/string target
|
57
|
+
|
58
|
+
# proc = Proc.new { |options| options[:record].rating > 8 ? 'HIGHLY RATED' ? 'AVERAGE'}
|
59
|
+
# Girlfriend.dumper :xml, :procs => {'my_rating' => proc}
|
60
|
+
#
|
61
|
+
# <girlfriend>
|
62
|
+
# # ... other attributes and methods ...
|
63
|
+
# <my_rating>HIGHLY RATED</my_rating>
|
64
|
+
# </girlfriend>
|
65
|
+
#
|
66
|
+
class ArDumper
|
67
|
+
|
68
|
+
cattr_accessor :dumper_page_size #default page size
|
69
|
+
@@dumper_page_size ||= 50
|
70
|
+
|
71
|
+
cattr_accessor :dumper_file_path #default file path
|
72
|
+
@@dumper_file_path ||= ENV['TMPDIR']||ENV['TMP']||ENV['TEMP']||'.'
|
73
|
+
|
74
|
+
cattr_accessor :dumper_tmp_file_basename #default basename of temporary files
|
75
|
+
@@dumper_tmp_file_basename ||= 'ardumper'
|
76
|
+
|
77
|
+
cattr_accessor :csv_writer #default to faster csv if it is available
|
78
|
+
|
79
|
+
|
80
|
+
attr_reader :fields
|
81
|
+
attr_reader :klass
|
82
|
+
attr_reader :options
|
83
|
+
|
84
|
+
def initialize(klass, dump_options={})
|
85
|
+
@klass = klass
|
86
|
+
@options = dump_options
|
87
|
+
build_attribute_list
|
88
|
+
|
89
|
+
unless options[:text_format].nil? || String.new.respond_to?(options[:text_format])
|
90
|
+
raise ArDumperException.new("Invalid value for option :text_format #{options[:text_format]}")
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
# build a list of attributes, methods and procs
|
95
|
+
def build_attribute_list#:nodoc:
|
96
|
+
if options[:only]
|
97
|
+
options[:attributes] = Array(options[:only])
|
98
|
+
else
|
99
|
+
options[:attributes] = @klass.column_names - Array(options[:except]).collect { |e| e.to_s }
|
100
|
+
end
|
101
|
+
|
102
|
+
options[:attributes] = options[:attributes].collect{|attr| "#{attr}"}
|
103
|
+
options[:methods] = options[:methods].is_a?(Hash) ? options[:methods].values : Array(options[:methods])
|
104
|
+
|
105
|
+
#if procs are specified as an array separate the headers(keys) from the procs(values)
|
106
|
+
if options[:procs].is_a?(Hash)
|
107
|
+
options[:proc_headers]= options[:procs].keys
|
108
|
+
options[:procs]= options[:procs].values
|
109
|
+
else
|
110
|
+
options[:procs] = Array(options[:procs])
|
111
|
+
options[:proc_headers]||= Array.new
|
112
|
+
0.upto(options[:procs].size - options[:proc_headers].size - 1) {|idx| options[:proc_headers] << "proc_#{idx}" }
|
113
|
+
end
|
114
|
+
|
115
|
+
end
|
116
|
+
|
117
|
+
|
118
|
+
|
119
|
+
|
120
|
+
def dump(format)
|
121
|
+
case format.to_sym
|
122
|
+
when :csv
|
123
|
+
dump_to_csv
|
124
|
+
|
125
|
+
when :xml
|
126
|
+
dump_to_xml
|
127
|
+
|
128
|
+
when :yaml, :fixture, :yml
|
129
|
+
dump_to_fixture
|
130
|
+
|
131
|
+
else
|
132
|
+
raise ArDumperException.new("Unknown format #{format}. Please specify :csv, :xml, or :yml ")
|
133
|
+
end
|
134
|
+
end
|
135
|
+
|
136
|
+
|
137
|
+
# Wrapper around the outside of dumper
|
138
|
+
# See options in dumper
|
139
|
+
def dumper(file_extension=nil, header = nil, footer = nil, &block)
|
140
|
+
|
141
|
+
options[:counter] = -1
|
142
|
+
begin
|
143
|
+
#get the file parameters
|
144
|
+
target = prepare_target(file_extension)
|
145
|
+
target << header if header
|
146
|
+
ArDumper.paginate_dump_records(@klass, @options) do |records, page_num|
|
147
|
+
|
148
|
+
#save state on options to make it accessible by
|
149
|
+
#class and procs
|
150
|
+
options[:result_set] = records
|
151
|
+
options[:page_num] = page_num
|
152
|
+
|
153
|
+
records.each do |record|
|
154
|
+
options[:record] = record
|
155
|
+
yield record
|
156
|
+
end
|
157
|
+
|
158
|
+
#flush after each set
|
159
|
+
target.flush if target.respond_to?(:flush)
|
160
|
+
end
|
161
|
+
target << footer if footer
|
162
|
+
|
163
|
+
#final step close the options[:target]
|
164
|
+
ensure
|
165
|
+
target.close if target && target.respond_to?(:close)
|
166
|
+
end
|
167
|
+
|
168
|
+
options[:full_file_name]||target
|
169
|
+
end
|
170
|
+
|
171
|
+
#collect the record data into an array
|
172
|
+
def dump_record(record)
|
173
|
+
record_values = @options[:attributes].inject([]){|values, attr| values << record["#{attr}"]; values }
|
174
|
+
record_values = @options[:methods].inject(record_values) {|values, method| values << record.send(method); values }
|
175
|
+
record_values = @options[:procs].inject(record_values){|values, proc| values << proc.call(options); values }
|
176
|
+
record_values
|
177
|
+
end
|
178
|
+
|
179
|
+
|
180
|
+
#############################################################################
|
181
|
+
# XML Dumper
|
182
|
+
#############################################################################
|
183
|
+
#
|
184
|
+
# Dumps the data to an xml file
|
185
|
+
#
|
186
|
+
# Using the ActiveRecord version of dumper so we CANNOT specify fields that are not attributes
|
187
|
+
#
|
188
|
+
# In addition to options listed in +dumper+:
|
189
|
+
# * <tt>:xml</tt> - xml options for ActiveRecord .to_xml
|
190
|
+
def dump_to_xml
|
191
|
+
|
192
|
+
#preserve the original skip instruct
|
193
|
+
skip_instruct = @options[:skip_instruct].is_a?(TrueClass)
|
194
|
+
|
195
|
+
#use the fields if :only is not specified in the xml options
|
196
|
+
xml_options ={:only => @options[:only],
|
197
|
+
:except => @options[:except],
|
198
|
+
:methods => @options[:methods],
|
199
|
+
:procs => @options[:procs]
|
200
|
+
}
|
201
|
+
|
202
|
+
xml_options.update(@options[:xml]) if @options[:xml]
|
203
|
+
|
204
|
+
#do not instruct for each set
|
205
|
+
xml_options[:skip_instruct] = true
|
206
|
+
xml_options[:indent]||=2
|
207
|
+
xml_options[:margin] = xml_options[:margin].to_i + 1
|
208
|
+
|
209
|
+
#set the variable on the options
|
210
|
+
options[:xml] = xml_options
|
211
|
+
|
212
|
+
|
213
|
+
#builder for header and footer
|
214
|
+
builder_options = {
|
215
|
+
:margin => xml_options[:margin] - 1,
|
216
|
+
:indent => xml_options[:indent]
|
217
|
+
}
|
218
|
+
|
219
|
+
options[:root] = (options[:root] || @klass.to_s.underscore.pluralize).to_s
|
220
|
+
|
221
|
+
|
222
|
+
#use the builder to make sure we are indented properly
|
223
|
+
builder = Builder::XmlMarkup.new(builder_options.clone)
|
224
|
+
builder.instruct! unless skip_instruct
|
225
|
+
builder << "<#{options[:root]}>\n"
|
226
|
+
header = builder.target!
|
227
|
+
|
228
|
+
#get the footer. Using the builder will make sure we are indented properly
|
229
|
+
builder = Builder::XmlMarkup.new(builder_options)
|
230
|
+
builder << "</#{options[:root]}>"
|
231
|
+
footer = builder.target!
|
232
|
+
|
233
|
+
dumper(:xml, header, footer) do |record|
|
234
|
+
options[:target] << record.to_xml(xml_options.dup)
|
235
|
+
end
|
236
|
+
end
|
237
|
+
|
238
|
+
#############################################################################
|
239
|
+
# Yaml/Fixture Dumper
|
240
|
+
#############################################################################
|
241
|
+
# dumps the data to a fixture file
|
242
|
+
# In addition to options listed in +dumper+:
|
243
|
+
# <tt>:root</tt> Basename of the record. Defaults to the class name so each record is named customer_1
|
244
|
+
def dump_to_fixture
|
245
|
+
basename = @options[:root]||@klass.table_name.singularize
|
246
|
+
header_list = build_header_list
|
247
|
+
|
248
|
+
# doctor the yaml a bit to print the hash header at the top
|
249
|
+
# instead of each record
|
250
|
+
dumper(:yml, "---\s") do |record|
|
251
|
+
record_data = Hash.new
|
252
|
+
dump_record(record).each_with_index{|field, idx| record_data[header_list[idx].to_s] = field.to_s }
|
253
|
+
options[:target] << {"#{basename}_#{record.id}" => record_data}.to_yaml.gsub(/^---\s\n/, "\n")
|
254
|
+
end
|
255
|
+
end
|
256
|
+
|
257
|
+
|
258
|
+
|
259
|
+
#############################################################################
|
260
|
+
# CSV DUMPER
|
261
|
+
#############################################################################
|
262
|
+
#
|
263
|
+
# Dump csv data
|
264
|
+
#
|
265
|
+
# * <tt>:csv</tt> - any options to pass to csv parser.
|
266
|
+
# :col_sep Example + :csv => {:col_sep => "\t"} +
|
267
|
+
# :row_sep Row seperator
|
268
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50
|
269
|
+
def dump_to_csv
|
270
|
+
header = nil
|
271
|
+
@options[:csv]||={}
|
272
|
+
|
273
|
+
if !@options[:header].is_a?(FalseClass)
|
274
|
+
header_list = build_header_list
|
275
|
+
#print the header unless set to false
|
276
|
+
header = write_csv_row(header_list)
|
277
|
+
end
|
278
|
+
|
279
|
+
dumper(:csv, header) do |record|
|
280
|
+
options[:target] << write_csv_row(dump_record(record))
|
281
|
+
end
|
282
|
+
end
|
283
|
+
|
284
|
+
# Write out the csv row using the selected csv writer
|
285
|
+
def write_csv_row(row_data, header_list=[])#:nodoc:
|
286
|
+
if csv_writer == :faster
|
287
|
+
::FasterCSV::Row.new(header_list, row_data).to_csv(@options[:csv])
|
288
|
+
else
|
289
|
+
::CSV.generate_line(row_data, @options[:csv][:col_sep], @options[:csv][:row_sep]) + (@options[:csv][:row_sep]||"\n")
|
290
|
+
end
|
291
|
+
end
|
292
|
+
|
293
|
+
#Try to use the FasterCSV if it exists
|
294
|
+
#otherwise use csv
|
295
|
+
def csv_writer #:nodoc:
|
296
|
+
unless @@csv_writer
|
297
|
+
@@csv_writer = :faster
|
298
|
+
begin
|
299
|
+
require 'faster_csv'
|
300
|
+
::FasterCSV
|
301
|
+
rescue Exception => exc
|
302
|
+
@@csv_writer = :normal
|
303
|
+
end
|
304
|
+
end
|
305
|
+
@@csv_writer
|
306
|
+
end
|
307
|
+
|
308
|
+
# Returns an array with the header names
|
309
|
+
# This will be in the same order as the data returned by dump_record
|
310
|
+
# attributes + methods + procs
|
311
|
+
#
|
312
|
+
# <tt>:header</tt> The header defaults to the attributes and method names. When set
|
313
|
+
# to false no header is specified
|
314
|
+
# * +hash+ A map from attribute or method name to Header column name
|
315
|
+
# * +array+ A list in the same order that is used to display record data
|
316
|
+
#
|
317
|
+
# <tt>:procs</tt> If a hash, then the keys are the names. If an array, then use proc_1, proc_2, etc
|
318
|
+
# <tt>:text_format</tt> Format names with a text format such as +:titlieze+, +:dasherize+, +:underscore+
|
319
|
+
def build_header_list#:nodoc:
|
320
|
+
|
321
|
+
header_options = options[:header]
|
322
|
+
columns = @options[:attributes] + @options[:methods]
|
323
|
+
header_names =
|
324
|
+
if header_options.is_a?(Hash)
|
325
|
+
header_options.symbolize_keys!
|
326
|
+
|
327
|
+
#Get the header for each attribute and method
|
328
|
+
columns.collect{|field|(header_options[field.to_sym]||field).to_s}
|
329
|
+
|
330
|
+
#ordered by attributes, methods, then procs
|
331
|
+
elsif header_options.is_a?(Array)
|
332
|
+
header_names = header_options
|
333
|
+
header_names.concat(columns[header_options.length..-1]) if header_names.length < columns.length
|
334
|
+
|
335
|
+
#default to column names
|
336
|
+
else
|
337
|
+
columns
|
338
|
+
end
|
339
|
+
|
340
|
+
#add process names
|
341
|
+
header_names.concat(options[:proc_headers])
|
342
|
+
|
343
|
+
#format names with a text format such as titlieze, dasherize, underscore
|
344
|
+
header_names.collect!{|n| n.to_s.send(options[:text_format])} if options[:text_format]
|
345
|
+
|
346
|
+
header_names
|
347
|
+
end
|
348
|
+
|
349
|
+
|
350
|
+
|
351
|
+
|
352
|
+
|
353
|
+
# Create the options[:target](file) based on these options. The options[:target] must respond to <<
|
354
|
+
# Current options[:target]s are :string, :file, :tempfile
|
355
|
+
#
|
356
|
+
# <tt> :target_type</tt> The options[:target] for the data. Defaults to +:file+
|
357
|
+
# * :string prints to string. Do not use with large data sets
|
358
|
+
# * :tmp_file. Use a temporary file that is destroyed when the process exists
|
359
|
+
# * :file. Use a standard file
|
360
|
+
# <tt> :filename </tt> basename of the file. Defaults to random time based string for non-temporary files
|
361
|
+
# <tt> :file_extension </tt> Extension (suffix) like .csv, .xml. Added only if the basename has no suffix.
|
362
|
+
# :file_extension is only available when +:target_type_type => :file+
|
363
|
+
# <tt> :file_path </tt> path or directory of the file. Defaults to dumper_file_path or temporary directories
|
364
|
+
|
365
|
+
def prepare_target(file_extension = nil)
|
366
|
+
|
367
|
+
options[:target] = case options[:target_type]
|
368
|
+
#to string option dumps to a string instead of a file
|
369
|
+
when :string
|
370
|
+
String.new
|
371
|
+
|
372
|
+
#use a temporary file
|
373
|
+
#open a temporary file with the basename specified by filename
|
374
|
+
#defaults to the value of one of the environment variables TMPDIR, TMP, or TEMP
|
375
|
+
when :tmp_file
|
376
|
+
|
377
|
+
Tempfile.open(options[:filename]||(@@dumper_tmp_file_basename+@klass.name.downcase),
|
378
|
+
options[:file_path]||@@dumper_file_path)
|
379
|
+
|
380
|
+
#default to a real file
|
381
|
+
else
|
382
|
+
extension = options[:file_extension]||file_extension
|
383
|
+
mode = options[:append_to_file].is_a?(TrueClass)? 'a' : 'w'
|
384
|
+
filename = options[:filename]||"#{@@dumper_tmp_file_basename}.#{@klass.name.downcase}.#{Time.now.to_f.to_s}.#{extension}"
|
385
|
+
|
386
|
+
#append an extension unless one already exists
|
387
|
+
filename += ".#{extension}" if extension && !filename =~ /\.\w*$/
|
388
|
+
|
389
|
+
#get the file path if the filename does not contain one
|
390
|
+
if File.basename(filename) == filename
|
391
|
+
path = options[:file_path]||@@dumper_file_path
|
392
|
+
filename = File.join(path, filename) unless path.blank?
|
393
|
+
end
|
394
|
+
|
395
|
+
|
396
|
+
File.open(filename, mode)
|
397
|
+
end
|
398
|
+
|
399
|
+
options[:full_file_name] = options[:target].path if options[:target].respond_to?(:path)
|
400
|
+
options[:target]
|
401
|
+
end
|
402
|
+
|
403
|
+
|
404
|
+
|
405
|
+
#############################################################################
|
406
|
+
# Pagination Helpers... OR YOU COULD INSTALL paginate_find plugin!
|
407
|
+
#############################################################################
|
408
|
+
#
|
409
|
+
#
|
410
|
+
# Quick and dirty paginate to loop thru the records page by page
|
411
|
+
# Options are:
|
412
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
413
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50. Set to false to disable pagination
|
414
|
+
#
|
415
|
+
def self.paginate_dump_records(klass, options={}, &block)
|
416
|
+
finder_options = (options[:find]||{}).clone
|
417
|
+
|
418
|
+
#pagination is not needed when :page_size => false
|
419
|
+
if options[:page_size].is_a?(FalseClass)
|
420
|
+
yield klass.find(:all, finder_options), 0
|
421
|
+
return
|
422
|
+
end
|
423
|
+
|
424
|
+
options[:page_size]||= dumper_page_size
|
425
|
+
|
426
|
+
#limit becomes the maximum amount of records to pull
|
427
|
+
max_records = finder_options[:limit]
|
428
|
+
page_num = 0
|
429
|
+
finder_options[:limit] = compute_page_size(max_records, page_num, options[:page_size])
|
430
|
+
records = []
|
431
|
+
while (finder_options[:limit] > 0 && (page_num == 0 || records.length == options[:page_size]))
|
432
|
+
records = klass.find :all, finder_options.update(:offset => page_num * options[:page_size])
|
433
|
+
|
434
|
+
yield records, page_num
|
435
|
+
page_num = page_num + 1
|
436
|
+
|
437
|
+
#calculate the limit if an original limit (max_records) was set
|
438
|
+
finder_options[:limit] = compute_page_size(max_records, page_num, options[:page_size])
|
439
|
+
end
|
440
|
+
end
|
441
|
+
|
442
|
+
def self.compute_page_size(max_records, page_num, page_size)#:nodoc:
|
443
|
+
max_records ? [(max_records - (page_num * page_size)), page_size].min : page_size
|
444
|
+
end
|
445
|
+
|
446
|
+
# Quick and dirty paginate to loop thru each page
|
447
|
+
# Options are:
|
448
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['hairy = ?', 'of course'], :include => :rodents} +
|
449
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50. Set to false to disable pagination
|
450
|
+
def self.paginate_each_record(klass, options={}, &block)
|
451
|
+
counter = -1
|
452
|
+
paginate_dump_records(klass, options) do |records, page_num|
|
453
|
+
records.each do |record|
|
454
|
+
yield record, (counter +=1)
|
455
|
+
end
|
456
|
+
end
|
457
|
+
end
|
458
|
+
end
|
459
|
+
|
460
|
+
|
461
|
+
|
462
|
+
|
463
|
+
|
464
|
+
|
465
|
+
|
@@ -0,0 +1,51 @@
|
|
1
|
+
class ActionController::Base
|
2
|
+
|
3
|
+
#Change to :file to use regular files
|
4
|
+
cattr_accessor :dumper_target
|
5
|
+
@@dumper_target = :tmp_file
|
6
|
+
|
7
|
+
|
8
|
+
# send a dumped file
|
9
|
+
# Options are:
|
10
|
+
# * <tt>klass</tt> - the active record to use. Animal or Contact or Author or Tumama
|
11
|
+
# * <tt>send_options</tt> send_options include send_file options such as +:type+ and +:disposition+
|
12
|
+
# * <tt>dump_options</tt> options to send to the dumper. These options are in ArDumper.dump_to_csv
|
13
|
+
# * <tt>:find</tt> - a map of the finder options passed to find. For example, + {:conditions => ['features.hair = ?', 'blonde'], :include => :features} +
|
14
|
+
# * <tt>:header</tt> - when a hash is specified, maps the field name to the header name. For example {:a => 'COL A', :b => 'COL B'} would print 'COL A', 'COL B'
|
15
|
+
# when an array is specified uses this instead of the fields
|
16
|
+
# when true or by default prints the fields
|
17
|
+
# when false does not include a header
|
18
|
+
# * <tt>:csv</tt> - any options to pass to csv parser. Example + :csv => {:col_sep => "\t"} +
|
19
|
+
# * <tt>:page_size</tt> - the page size to use. Defaults to dumper_page_size or 50
|
20
|
+
def send_file_dump(format, klass, send_options={}, dump_options={})
|
21
|
+
|
22
|
+
#use the base name of the dump file name if specified
|
23
|
+
send_options[:filename] ||= File.basename(dump_options[:filename]) if dump_options[:filename]
|
24
|
+
|
25
|
+
#never delete the file
|
26
|
+
dump_options[:delete_file] = false
|
27
|
+
|
28
|
+
#use temporary files unless otherwise specified
|
29
|
+
dump_options[:target_type]||= @@dumper_target
|
30
|
+
|
31
|
+
#send_options[:type] = 'application/xml; charset=utf-8;'
|
32
|
+
if send_options[:type].nil? && send_options[:disposition] == 'inline'
|
33
|
+
send_options[:type] =
|
34
|
+
case format.to_sym
|
35
|
+
when :xml then 'application/xml; charset=utf-8;'
|
36
|
+
when :csv then 'application/csv; charset=utf-8;'
|
37
|
+
when :yml then 'text/html; charset=utf-8;'
|
38
|
+
end
|
39
|
+
|
40
|
+
end
|
41
|
+
|
42
|
+
target = klass.dumper(format, dump_options)
|
43
|
+
|
44
|
+
if dump_options[:target_type] == :string
|
45
|
+
send_data(target, send_options)
|
46
|
+
else
|
47
|
+
send_file(target, send_options)
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
end
|
metadata
ADDED
@@ -0,0 +1,61 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
rubygems_version: 0.9.4
|
3
|
+
specification_version: 1
|
4
|
+
name: ardumper
|
5
|
+
version: !ruby/object:Gem::Version
|
6
|
+
version: 1.0.1.1
|
7
|
+
date: 2007-08-09 00:00:00 -07:00
|
8
|
+
summary: Provides developers with the ability to define any dependencies on external plugins directly from within another plugin.
|
9
|
+
require_paths:
|
10
|
+
- lib
|
11
|
+
email: blythe@spongecell.com
|
12
|
+
homepage: http://spongetech.wordpress.com/
|
13
|
+
rubyforge_project:
|
14
|
+
description:
|
15
|
+
autorequire: ardumper
|
16
|
+
default_executable:
|
17
|
+
bindir: bin
|
18
|
+
has_rdoc: true
|
19
|
+
required_ruby_version: !ruby/object:Gem::Version::Requirement
|
20
|
+
requirements:
|
21
|
+
- - ">"
|
22
|
+
- !ruby/object:Gem::Version
|
23
|
+
version: 0.0.0
|
24
|
+
version:
|
25
|
+
platform: ruby
|
26
|
+
signing_key:
|
27
|
+
cert_chain:
|
28
|
+
post_install_message:
|
29
|
+
authors:
|
30
|
+
- Blythe Dunham
|
31
|
+
files:
|
32
|
+
- lib/ar_dumper.rb
|
33
|
+
- lib/ar_dumper_active_record.rb
|
34
|
+
- lib/ar_dumper_base.rb
|
35
|
+
- lib/ar_dumper_controller.rb
|
36
|
+
- init.rb
|
37
|
+
- LICENSE
|
38
|
+
- Rakefile
|
39
|
+
- README
|
40
|
+
test_files: []
|
41
|
+
|
42
|
+
rdoc_options: []
|
43
|
+
|
44
|
+
extra_rdoc_files: []
|
45
|
+
|
46
|
+
executables: []
|
47
|
+
|
48
|
+
extensions: []
|
49
|
+
|
50
|
+
requirements: []
|
51
|
+
|
52
|
+
dependencies:
|
53
|
+
- !ruby/object:Gem::Dependency
|
54
|
+
name: rails
|
55
|
+
version_requirement:
|
56
|
+
version_requirements: !ruby/object:Gem::Version::Requirement
|
57
|
+
requirements:
|
58
|
+
- - ">="
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: 1.2.0
|
61
|
+
version:
|