dumb_delimited 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +9 -0
- data/.travis.yml +5 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +21 -0
- data/README.md +111 -0
- data/Rakefile +23 -0
- data/dumb_delimited.gemspec +29 -0
- data/lib/dumb_delimited.rb +158 -0
- data/lib/dumb_delimited/version.rb +3 -0
- metadata +123 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 4522a4b2571eae677938c2c9ff3a20ed7d0a2b22
|
4
|
+
data.tar.gz: 7d832b4eaf31490dd95f6e547ddf9e74e677f712
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 3cd992cc109129c7342815b227c6ebe4ac167c0635917d894774b5509fc92e7f5f119fe5bb9fc8aeeedd2f3d9937bc031ea71059172fb44d5f1d8fc942492cad
|
7
|
+
data.tar.gz: eef544b4373011f28996a3a902d25a354baf961aa4655f3819889132660adad59031811b776afe52bbba22fc36440e5731f270b10779323f805c27f4472f81c0
|
data/.gitignore
ADDED
data/.travis.yml
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2017 Jonathan Hefner
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,111 @@
|
|
1
|
+
# dumb_delimited
|
2
|
+
|
3
|
+
A library for unsophisticated delimited flat file IO. *dumb_delimited*
|
4
|
+
mixes models and persistence in that "probably wrong but feels so right"
|
5
|
+
kind of way.
|
6
|
+
|
7
|
+
|
8
|
+
## Usage Example
|
9
|
+
|
10
|
+
Let's say we have a products file "products.csv", and a customers
|
11
|
+
file "customers.psv".
|
12
|
+
|
13
|
+
"products.csv" is a comma-delimited flat file and has four columns: SKU,
|
14
|
+
Product Name, Base Price, and Sale Price. An example row from
|
15
|
+
"products.csv" might be:
|
16
|
+
|
17
|
+
```
|
18
|
+
AB81H0F,Widget Alpha,899.99,499.99
|
19
|
+
```
|
20
|
+
|
21
|
+
"customers.psv" is a pipe-delimited flat file and has three columns:
|
22
|
+
Customer Name, Email, and Address. An example row from "customers.psv"
|
23
|
+
might be:
|
24
|
+
|
25
|
+
```
|
26
|
+
Bob Bobbington|best_bob@bobbers.bob|808 Bounce Lane, Austin, TX 78703
|
27
|
+
```
|
28
|
+
|
29
|
+
To interact with these files, we create model classes via the
|
30
|
+
`DumbDelimited#[]` method. Note that a created class can either be used
|
31
|
+
as a superclass or simply assigned to a constant.
|
32
|
+
|
33
|
+
```ruby
|
34
|
+
class Product < DumbDelimited[:sku, :name, :base_price, :sale_price]
|
35
|
+
def on_sale?
|
36
|
+
sale_price < base_price
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
Customer = DumbDelimited[:name, :email, :address]
|
41
|
+
Customer.delimiter = "|"
|
42
|
+
```
|
43
|
+
|
44
|
+
Because "customers.psv" is pipe-delimited, we also set the delimiter
|
45
|
+
for the Customer class. By default, model classes use comma (`","`) as
|
46
|
+
the delimiter. Whenever a delimiter is set, it applies to all future
|
47
|
+
IO operations for that model class.
|
48
|
+
|
49
|
+
Now we can read each flat file, and recieve an array of model objects.
|
50
|
+
|
51
|
+
```ruby
|
52
|
+
products = Product.parse_file("products.csv")
|
53
|
+
customers = Customer.parse_file("customers.psv")
|
54
|
+
```
|
55
|
+
|
56
|
+
This, however, will load the entire contents of each file into memory.
|
57
|
+
Let's say our customers file is very large, and we would prefer to
|
58
|
+
iterate over it rather than load it all into memory at once. To do so,
|
59
|
+
we can use the `each_in_file` method that the model class provides.
|
60
|
+
Below is a complete example in which we load our product data, create a
|
61
|
+
listing of products on sale, and iterate over our customers, notifying
|
62
|
+
each customer of the sale products:
|
63
|
+
|
64
|
+
```ruby
|
65
|
+
products = Product.parse_file("products.csv")
|
66
|
+
|
67
|
+
listing = products.select(&:on_sale?).map do |product|
|
68
|
+
"* #{product.name} (#{product.sale_price})"
|
69
|
+
end.join("\n")
|
70
|
+
|
71
|
+
Customer.each_in_file("customers.psv") do |customer|
|
72
|
+
message =
|
73
|
+
"Hi #{customer.name}!\n\n" \
|
74
|
+
"The following products are on sale:\n\n#{listing}"
|
75
|
+
|
76
|
+
notify(customer.email, message)
|
77
|
+
end
|
78
|
+
```
|
79
|
+
|
80
|
+
Let's say the sale is now over, and we want to change our sale prices
|
81
|
+
back to our base prices. *dumb_delimited* includes the
|
82
|
+
[*pleasant_path*](https://github.com/jonathanhefner/pleasant_path) gem,
|
83
|
+
which offers a fluent API for writing files. To finish our task, we use
|
84
|
+
the `Array#write_to_file` method provided by *pleasant_path*, which in
|
85
|
+
turn invokes `Product#to_s` (provided by *dumb_delimited*) on each model
|
86
|
+
object.
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
Product.parse_file("products.csv").each do |product|
|
90
|
+
product.sale_price = product.base_price
|
91
|
+
end.write_to_file("products.csv")
|
92
|
+
```
|
93
|
+
|
94
|
+
For a more detailed explanation of the *dumb_delimited* API, browse the
|
95
|
+
[full documentation](http://www.rubydoc.info/gems/dumb_delimited/).
|
96
|
+
|
97
|
+
|
98
|
+
## Installation
|
99
|
+
|
100
|
+
$ gem install dumb_delimited
|
101
|
+
|
102
|
+
|
103
|
+
## Development
|
104
|
+
|
105
|
+
Run `rake test` to run the tests. You can also run `rake irb` for an
|
106
|
+
interactive prompt that pre-loads the project code.
|
107
|
+
|
108
|
+
|
109
|
+
## License
|
110
|
+
|
111
|
+
[MIT License](http://opensource.org/licenses/MIT)
|
data/Rakefile
ADDED
@@ -0,0 +1,23 @@
|
|
1
|
+
require "bundler/gem_tasks"
|
2
|
+
require "rake/testtask"
|
3
|
+
require "yard"
|
4
|
+
|
5
|
+
|
6
|
+
YARD::Rake::YardocTask.new(:doc) do |t|
|
7
|
+
end
|
8
|
+
|
9
|
+
desc "Launch IRB with this gem pre-loaded"
|
10
|
+
task :irb do
|
11
|
+
require "dumb_delimited"
|
12
|
+
require "irb"
|
13
|
+
ARGV.clear
|
14
|
+
IRB.start
|
15
|
+
end
|
16
|
+
|
17
|
+
Rake::TestTask.new(:test) do |t|
|
18
|
+
t.libs << "test"
|
19
|
+
t.libs << "lib"
|
20
|
+
t.test_files = FileList["test/**/*_test.rb"]
|
21
|
+
end
|
22
|
+
|
23
|
+
task :default => :test
|
@@ -0,0 +1,29 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path("../lib", __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require "dumb_delimited/version"
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "dumb_delimited"
|
8
|
+
spec.version = DumbDelimited::VERSION
|
9
|
+
spec.authors = ["Jonathan Hefner"]
|
10
|
+
spec.email = ["jonathan.hefner@gmail.com"]
|
11
|
+
|
12
|
+
spec.summary = %q{Library for unsophisticated delimited flat file IO}
|
13
|
+
spec.homepage = "https://github.com/jonathanhefner/dumb_delimited"
|
14
|
+
spec.license = "MIT"
|
15
|
+
|
16
|
+
spec.files = `git ls-files -z`.split("\x0").reject do |f|
|
17
|
+
f.match(%r{^(test|spec|features)/})
|
18
|
+
end
|
19
|
+
spec.bindir = "exe"
|
20
|
+
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
21
|
+
spec.require_paths = ["lib"]
|
22
|
+
|
23
|
+
spec.add_runtime_dependency "pleasant_path", "~> 1.0"
|
24
|
+
|
25
|
+
spec.add_development_dependency "bundler", "~> 1.15"
|
26
|
+
spec.add_development_dependency "rake", "~> 10.0"
|
27
|
+
spec.add_development_dependency "minitest", "~> 5.0"
|
28
|
+
spec.add_development_dependency "yard", "~> 0.9"
|
29
|
+
end
|
@@ -0,0 +1,158 @@
|
|
1
|
+
require "CSV"
|
2
|
+
require "pleasant_path"
|
3
|
+
require "dumb_delimited/version"
|
4
|
+
|
5
|
+
|
6
|
+
module DumbDelimited
|
7
|
+
|
8
|
+
# Returns a model class for delimited data consisting of the specified
|
9
|
+
# columns. The returned class inherits from Ruby's
|
10
|
+
# {https://ruby-doc.org/core/Struct.html +Struct+}, allowing data
|
11
|
+
# manipulation via accessor methods, via indexing by column name, and
|
12
|
+
# via indexing by column number. See {ClassMethods} and
|
13
|
+
# {InstanceMethods} for the IO methods the returned class provides.
|
14
|
+
#
|
15
|
+
# @example
|
16
|
+
# class Product < DumbDelimited[:sku, :name, :base_price, :sale_price]
|
17
|
+
# def on_sale?
|
18
|
+
# sale_price < base_price
|
19
|
+
# end
|
20
|
+
# end
|
21
|
+
#
|
22
|
+
# Customer = DumbDelimited[:name, :email, :address]
|
23
|
+
#
|
24
|
+
# @param columns [*Symbol]
|
25
|
+
# @return [Class]
|
26
|
+
def self.[](*columns)
|
27
|
+
Struct.new(*columns) do
|
28
|
+
extend DumbDelimited::ClassMethods
|
29
|
+
include DumbDelimited::InstanceMethods
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
end
|
34
|
+
|
35
|
+
|
36
|
+
module DumbDelimited::ClassMethods
|
37
|
+
|
38
|
+
# Returns the advanced options Hash. The Hash is not +dup+ed and can
|
39
|
+
# be modified directly. Any modifications will be applied to all
|
40
|
+
# future IO operations for the model class. For detailed information
|
41
|
+
# about available options, see Ruby's
|
42
|
+
# {http://ruby-doc.org/stdlib/libdoc/csv/rdoc/CSV.html#method-c-new
|
43
|
+
# CSV module}.
|
44
|
+
#
|
45
|
+
# @return [Hash]
|
46
|
+
def options
|
47
|
+
@options ||= {
|
48
|
+
col_sep: ',',
|
49
|
+
skip_blanks: true,
|
50
|
+
converters: :numeric,
|
51
|
+
}
|
52
|
+
end
|
53
|
+
|
54
|
+
# Sets the advanced options Hash. The entire Hash is replaced, and
|
55
|
+
# the new value will be applied to all future IO operations for the
|
56
|
+
# model class. To set options individually, see {options}. For
|
57
|
+
# detailed information about available options, see Ruby's
|
58
|
+
# {http://ruby-doc.org/stdlib/libdoc/csv/rdoc/CSV.html#method-c-new
|
59
|
+
# CSV module}.
|
60
|
+
#
|
61
|
+
# @param o [Hash]
|
62
|
+
def options=(o)
|
63
|
+
@options = o
|
64
|
+
end
|
65
|
+
|
66
|
+
# Returns the column delimiter used in IO operations. Defaults to a
|
67
|
+
# comma (<code>","</code>).
|
68
|
+
#
|
69
|
+
# @return [String]
|
70
|
+
def delimiter
|
71
|
+
self.options[:col_sep]
|
72
|
+
end
|
73
|
+
|
74
|
+
# Sets the column delimiter used in IO operations. The new value will
|
75
|
+
# be used in all future IO operations for the model class. Any
|
76
|
+
# delimiter can be safely chosen, and all IO operations will quote
|
77
|
+
# field values as necessary.
|
78
|
+
#
|
79
|
+
# @example
|
80
|
+
# Point = DumbDelimited[:x, :y, :z]
|
81
|
+
# p = Point.new(1, 2, 3)
|
82
|
+
# p.to_s # == "1,2,3"
|
83
|
+
# Point.delimiter = "|"
|
84
|
+
# p.to_s # == "1|2|3"
|
85
|
+
#
|
86
|
+
# @param d [String]
|
87
|
+
def delimiter=(d)
|
88
|
+
self.options[:col_sep] = d
|
89
|
+
end
|
90
|
+
|
91
|
+
# Parses a single delimited line into a model object.
|
92
|
+
#
|
93
|
+
# @example
|
94
|
+
# Point = DumbDelimited[:x, :y, :z]
|
95
|
+
# p = Point.parse_line("1,2,3")
|
96
|
+
# p.is_a?(Point) # == true
|
97
|
+
# p.to_a # == [1, 2, 3]
|
98
|
+
#
|
99
|
+
# @param line [String]
|
100
|
+
# @return [self]
|
101
|
+
def parse_line(line)
|
102
|
+
self.new(*CSV.parse_line(line, self.options))
|
103
|
+
end
|
104
|
+
|
105
|
+
# Parses an entire delimited file into an array of model objects.
|
106
|
+
# This will load the entire contents of the file into memory, and may
|
107
|
+
# not be suitable for large files. To iterate over file contents
|
108
|
+
# without loading it all into memory at once, use {each_in_file}.
|
109
|
+
#
|
110
|
+
# @example
|
111
|
+
# # CONTENTS OF FILE "points.csv":
|
112
|
+
# # 1,2,3
|
113
|
+
# # 4,5,6
|
114
|
+
# # 7,8,9
|
115
|
+
#
|
116
|
+
# Point = DumbDelimited[:x, :y, :z]
|
117
|
+
# points = Point.parse_file("points.csv")
|
118
|
+
# points.map(&:x) # == [1, 4, 7]
|
119
|
+
#
|
120
|
+
# @param path [String, Pathname]
|
121
|
+
# @return [Array<self>]
|
122
|
+
def parse_file(path)
|
123
|
+
each_in_file(path).to_a
|
124
|
+
end
|
125
|
+
|
126
|
+
# Iterates over a delimited file, parsing one row at a time into model
|
127
|
+
# objects. This avoids loading the entire contents of the file into
|
128
|
+
# memory at once. If a block is given, it will be passed a model
|
129
|
+
# object for each row in the file. Otherwise, if a block is not
|
130
|
+
# given, an Enumerator will be returned. Note that some Enumerator
|
131
|
+
# methods, such as +Enumerator#to_a+, will cause the entire contents
|
132
|
+
# of the file to be loaded into memory regardless.
|
133
|
+
#
|
134
|
+
# @param path [String, Pathname]
|
135
|
+
# @yieldparam [self] current model object
|
136
|
+
# @return [Enumerator<self>, nil]
|
137
|
+
def each_in_file(path)
|
138
|
+
return to_enum(__method__, path) unless block_given?
|
139
|
+
|
140
|
+
CSV.foreach(path, self.options) do |row|
|
141
|
+
yield self.new(*row)
|
142
|
+
end
|
143
|
+
end
|
144
|
+
|
145
|
+
end
|
146
|
+
|
147
|
+
|
148
|
+
module DumbDelimited::InstanceMethods
|
149
|
+
|
150
|
+
# Serializes a model object to a delimited string, using the delimiter
|
151
|
+
# specified by {ClassMethods.delimiter}.
|
152
|
+
#
|
153
|
+
# @return [String]
|
154
|
+
def to_s
|
155
|
+
CSV.generate_line(self, self.class.options).chomp!
|
156
|
+
end
|
157
|
+
|
158
|
+
end
|
metadata
ADDED
@@ -0,0 +1,123 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: dumb_delimited
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Jonathan Hefner
|
8
|
+
autorequire:
|
9
|
+
bindir: exe
|
10
|
+
cert_chain: []
|
11
|
+
date: 2017-07-02 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: pleasant_path
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '1.0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '1.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: bundler
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '1.15'
|
34
|
+
type: :development
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '1.15'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rake
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '10.0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '10.0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: minitest
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '5.0'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '5.0'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: yard
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '0.9'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '0.9'
|
83
|
+
description:
|
84
|
+
email:
|
85
|
+
- jonathan.hefner@gmail.com
|
86
|
+
executables: []
|
87
|
+
extensions: []
|
88
|
+
extra_rdoc_files: []
|
89
|
+
files:
|
90
|
+
- ".gitignore"
|
91
|
+
- ".travis.yml"
|
92
|
+
- Gemfile
|
93
|
+
- LICENSE.txt
|
94
|
+
- README.md
|
95
|
+
- Rakefile
|
96
|
+
- dumb_delimited.gemspec
|
97
|
+
- lib/dumb_delimited.rb
|
98
|
+
- lib/dumb_delimited/version.rb
|
99
|
+
homepage: https://github.com/jonathanhefner/dumb_delimited
|
100
|
+
licenses:
|
101
|
+
- MIT
|
102
|
+
metadata: {}
|
103
|
+
post_install_message:
|
104
|
+
rdoc_options: []
|
105
|
+
require_paths:
|
106
|
+
- lib
|
107
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
108
|
+
requirements:
|
109
|
+
- - ">="
|
110
|
+
- !ruby/object:Gem::Version
|
111
|
+
version: '0'
|
112
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
113
|
+
requirements:
|
114
|
+
- - ">="
|
115
|
+
- !ruby/object:Gem::Version
|
116
|
+
version: '0'
|
117
|
+
requirements: []
|
118
|
+
rubyforge_project:
|
119
|
+
rubygems_version: 2.4.8
|
120
|
+
signing_key:
|
121
|
+
specification_version: 4
|
122
|
+
summary: Library for unsophisticated delimited flat file IO
|
123
|
+
test_files: []
|