gtfs-reader 0.2.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: cd067fc13cd77eac671d1b07114b886d1b524c4c
4
+ data.tar.gz: 04d9c564134ba37201753a9094ae715c4d39662f
5
+ SHA512:
6
+ metadata.gz: 4fdf6b70e9ff9725c6b6d6d4086c5747b60d4aaf5ccd580cfb7f4ac2a2e23478d25acc5031627214064a4259fca55469c70f35ba53783a0adc7025cdf531cddd
7
+ data.tar.gz: 75d82e8b6f75a366232ab96adae9e97b0a9735e6d27ed59b9d928d7834974d6beaf568451694927ddeb796e2d660ded0e492dc5920e842ab78ee2b042196fdd7
data/LICENSE ADDED
@@ -0,0 +1,284 @@
1
+ GNU GENERAL PUBLIC LICENSE
2
+ Version 2, June 1991
3
+
4
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
5
+ 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
6
+ Everyone is permitted to copy and distribute verbatim copies
7
+ of this license document, but changing it is not allowed.
8
+
9
+ Preamble
10
+
11
+ The licenses for most software are designed to take away your
12
+ freedom to share and change it. By contrast, the GNU General Public
13
+ License is intended to guarantee your freedom to share and change free
14
+ software--to make sure the software is free for all its users. This
15
+ General Public License applies to most of the Free Software
16
+ Foundation's software and to any other program whose authors commit to
17
+ using it. (Some other Free Software Foundation software is covered by
18
+ the GNU Library General Public License instead.) You can apply it to
19
+ your programs, too.
20
+
21
+ When we speak of free software, we are referring to freedom, not
22
+ price. Our General Public Licenses are designed to make sure that you
23
+ have the freedom to distribute copies of free software (and charge for
24
+ this service if you wish), that you receive source code or can get it
25
+ if you want it, that you can change the software or use pieces of it
26
+ in new free programs; and that you know you can do these things.
27
+
28
+ To protect your rights, we need to make restrictions that forbid
29
+ anyone to deny you these rights or to ask you to surrender the rights.
30
+ These restrictions translate to certain responsibilities for you if you
31
+ distribute copies of the software, or if you modify it.
32
+
33
+ For example, if you distribute copies of such a program, whether
34
+ gratis or for a fee, you must give the recipients all the rights that
35
+ you have. You must make sure that they, too, receive or can get the
36
+ source code. And you must show them these terms so they know their
37
+ rights.
38
+
39
+ We protect your rights with two steps: (1) copyright the software, and
40
+ (2) offer you this license which gives you legal permission to copy,
41
+ distribute and/or modify the software.
42
+
43
+ Also, for each author's protection and ours, we want to make certain
44
+ that everyone understands that there is no warranty for this free
45
+ software. If the software is modified by someone else and passed on, we
46
+ want its recipients to know that what they have is not the original, so
47
+ that any problems introduced by others will not reflect on the original
48
+ authors' reputations.
49
+
50
+ Finally, any free program is threatened constantly by software
51
+ patents. We wish to avoid the danger that redistributors of a free
52
+ program will individually obtain patent licenses, in effect making the
53
+ program proprietary. To prevent this, we have made it clear that any
54
+ patent must be licensed for everyone's free use or not licensed at all.
55
+
56
+ The precise terms and conditions for copying, distribution and
57
+ modification follow.
58
+
59
+
60
+ GNU GENERAL PUBLIC LICENSE
61
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
62
+
63
+ 0. This License applies to any program or other work which contains
64
+ a notice placed by the copyright holder saying it may be distributed
65
+ under the terms of this General Public License. The "Program", below,
66
+ refers to any such program or work, and a "work based on the Program"
67
+ means either the Program or any derivative work under copyright law:
68
+ that is to say, a work containing the Program or a portion of it,
69
+ either verbatim or with modifications and/or translated into another
70
+ language. (Hereinafter, translation is included without limitation in
71
+ the term "modification".) Each licensee is addressed as "you".
72
+
73
+ Activities other than copying, distribution and modification are not
74
+ covered by this License; they are outside its scope. The act of
75
+ running the Program is not restricted, and the output from the Program
76
+ is covered only if its contents constitute a work based on the
77
+ Program (independent of having been made by running the Program).
78
+ Whether that is true depends on what the Program does.
79
+
80
+ 1. You may copy and distribute verbatim copies of the Program's
81
+ source code as you receive it, in any medium, provided that you
82
+ conspicuously and appropriately publish on each copy an appropriate
83
+ copyright notice and disclaimer of warranty; keep intact all the
84
+ notices that refer to this License and to the absence of any warranty;
85
+ and give any other recipients of the Program a copy of this License
86
+ along with the Program.
87
+
88
+ You may charge a fee for the physical act of transferring a copy, and
89
+ you may at your option offer warranty protection in exchange for a fee.
90
+
91
+ 2. You may modify your copy or copies of the Program or any portion
92
+ of it, thus forming a work based on the Program, and copy and
93
+ distribute such modifications or work under the terms of Section 1
94
+ above, provided that you also meet all of these conditions:
95
+
96
+ a) You must cause the modified files to carry prominent notices
97
+ stating that you changed the files and the date of any change.
98
+
99
+ b) You must cause any work that you distribute or publish, that in
100
+ whole or in part contains or is derived from the Program or any
101
+ part thereof, to be licensed as a whole at no charge to all third
102
+ parties under the terms of this License.
103
+
104
+ c) If the modified program normally reads commands interactively
105
+ when run, you must cause it, when started running for such
106
+ interactive use in the most ordinary way, to print or display an
107
+ announcement including an appropriate copyright notice and a
108
+ notice that there is no warranty (or else, saying that you provide
109
+ a warranty) and that users may redistribute the program under
110
+ these conditions, and telling the user how to view a copy of this
111
+ License. (Exception: if the Program itself is interactive but
112
+ does not normally print such an announcement, your work based on
113
+ the Program is not required to print an announcement.)
114
+
115
+
116
+ These requirements apply to the modified work as a whole. If
117
+ identifiable sections of that work are not derived from the Program,
118
+ and can be reasonably considered independent and separate works in
119
+ themselves, then this License, and its terms, do not apply to those
120
+ sections when you distribute them as separate works. But when you
121
+ distribute the same sections as part of a whole which is a work based
122
+ on the Program, the distribution of the whole must be on the terms of
123
+ this License, whose permissions for other licensees extend to the
124
+ entire whole, and thus to each and every part regardless of who wrote it.
125
+
126
+ Thus, it is not the intent of this section to claim rights or contest
127
+ your rights to work written entirely by you; rather, the intent is to
128
+ exercise the right to control the distribution of derivative or
129
+ collective works based on the Program.
130
+
131
+ In addition, mere aggregation of another work not based on the Program
132
+ with the Program (or with a work based on the Program) on a volume of
133
+ a storage or distribution medium does not bring the other work under
134
+ the scope of this License.
135
+
136
+ 3. You may copy and distribute the Program (or a work based on it,
137
+ under Section 2) in object code or executable form under the terms of
138
+ Sections 1 and 2 above provided that you also do one of the following:
139
+
140
+ a) Accompany it with the complete corresponding machine-readable
141
+ source code, which must be distributed under the terms of Sections
142
+ 1 and 2 above on a medium customarily used for software interchange; or,
143
+
144
+ b) Accompany it with a written offer, valid for at least three
145
+ years, to give any third party, for a charge no more than your
146
+ cost of physically performing source distribution, a complete
147
+ machine-readable copy of the corresponding source code, to be
148
+ distributed under the terms of Sections 1 and 2 above on a medium
149
+ customarily used for software interchange; or,
150
+
151
+ c) Accompany it with the information you received as to the offer
152
+ to distribute corresponding source code. (This alternative is
153
+ allowed only for noncommercial distribution and only if you
154
+ received the program in object code or executable form with such
155
+ an offer, in accord with Subsection b above.)
156
+
157
+ The source code for a work means the preferred form of the work for
158
+ making modifications to it. For an executable work, complete source
159
+ code means all the source code for all modules it contains, plus any
160
+ associated interface definition files, plus the scripts used to
161
+ control compilation and installation of the executable. However, as a
162
+ special exception, the source code distributed need not include
163
+ anything that is normally distributed (in either source or binary
164
+ form) with the major components (compiler, kernel, and so on) of the
165
+ operating system on which the executable runs, unless that component
166
+ itself accompanies the executable.
167
+
168
+ If distribution of executable or object code is made by offering
169
+ access to copy from a designated place, then offering equivalent
170
+ access to copy the source code from the same place counts as
171
+ distribution of the source code, even though third parties are not
172
+ compelled to copy the source along with the object code.
173
+
174
+
175
+ 4. You may not copy, modify, sublicense, or distribute the Program
176
+ except as expressly provided under this License. Any attempt
177
+ otherwise to copy, modify, sublicense or distribute the Program is
178
+ void, and will automatically terminate your rights under this License.
179
+ However, parties who have received copies, or rights, from you under
180
+ this License will not have their licenses terminated so long as such
181
+ parties remain in full compliance.
182
+
183
+ 5. You are not required to accept this License, since you have not
184
+ signed it. However, nothing else grants you permission to modify or
185
+ distribute the Program or its derivative works. These actions are
186
+ prohibited by law if you do not accept this License. Therefore, by
187
+ modifying or distributing the Program (or any work based on the
188
+ Program), you indicate your acceptance of this License to do so, and
189
+ all its terms and conditions for copying, distributing or modifying
190
+ the Program or works based on it.
191
+
192
+ 6. Each time you redistribute the Program (or any work based on the
193
+ Program), the recipient automatically receives a license from the
194
+ original licensor to copy, distribute or modify the Program subject to
195
+ these terms and conditions. You may not impose any further
196
+ restrictions on the recipients' exercise of the rights granted herein.
197
+ You are not responsible for enforcing compliance by third parties to
198
+ this License.
199
+
200
+ 7. If, as a consequence of a court judgment or allegation of patent
201
+ infringement or for any other reason (not limited to patent issues),
202
+ conditions are imposed on you (whether by court order, agreement or
203
+ otherwise) that contradict the conditions of this License, they do not
204
+ excuse you from the conditions of this License. If you cannot
205
+ distribute so as to satisfy simultaneously your obligations under this
206
+ License and any other pertinent obligations, then as a consequence you
207
+ may not distribute the Program at all. For example, if a patent
208
+ license would not permit royalty-free redistribution of the Program by
209
+ all those who receive copies directly or indirectly through you, then
210
+ the only way you could satisfy both it and this License would be to
211
+ refrain entirely from distribution of the Program.
212
+
213
+ If any portion of this section is held invalid or unenforceable under
214
+ any particular circumstance, the balance of the section is intended to
215
+ apply and the section as a whole is intended to apply in other
216
+ circumstances.
217
+
218
+ It is not the purpose of this section to induce you to infringe any
219
+ patents or other property right claims or to contest validity of any
220
+ such claims; this section has the sole purpose of protecting the
221
+ integrity of the free software distribution system, which is
222
+ implemented by public license practices. Many people have made
223
+ generous contributions to the wide range of software distributed
224
+ through that system in reliance on consistent application of that
225
+ system; it is up to the author/donor to decide if he or she is willing
226
+ to distribute software through any other system and a licensee cannot
227
+ impose that choice.
228
+
229
+ This section is intended to make thoroughly clear what is believed to
230
+ be a consequence of the rest of this License.
231
+
232
+
233
+ 8. If the distribution and/or use of the Program is restricted in
234
+ certain countries either by patents or by copyrighted interfaces, the
235
+ original copyright holder who places the Program under this License
236
+ may add an explicit geographical distribution limitation excluding
237
+ those countries, so that distribution is permitted only in or among
238
+ countries not thus excluded. In such case, this License incorporates
239
+ the limitation as if written in the body of this License.
240
+
241
+ 9. The Free Software Foundation may publish revised and/or new versions
242
+ of the General Public License from time to time. Such new versions will
243
+ be similar in spirit to the present version, but may differ in detail to
244
+ address new problems or concerns.
245
+
246
+ Each version is given a distinguishing version number. If the Program
247
+ specifies a version number of this License which applies to it and "any
248
+ later version", you have the option of following the terms and conditions
249
+ either of that version or of any later version published by the Free
250
+ Software Foundation. If the Program does not specify a version number of
251
+ this License, you may choose any version ever published by the Free Software
252
+ Foundation.
253
+
254
+ 10. If you wish to incorporate parts of the Program into other free
255
+ programs whose distribution conditions are different, write to the author
256
+ to ask for permission. For software which is copyrighted by the Free
257
+ Software Foundation, write to the Free Software Foundation; we sometimes
258
+ make exceptions for this. Our decision will be guided by the two goals
259
+ of preserving the free status of all derivatives of our free software and
260
+ of promoting the sharing and reuse of software generally.
261
+
262
+ NO WARRANTY
263
+
264
+ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
265
+ FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
266
+ OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
267
+ PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
268
+ OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
269
+ MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
270
+ TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
271
+ PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
272
+ REPAIR OR CORRECTION.
273
+
274
+ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
275
+ WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
276
+ REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
277
+ INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
278
+ OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
279
+ TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
280
+ YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
281
+ PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
282
+ POSSIBILITY OF SUCH DAMAGES.
283
+
284
+ END OF TERMS AND CONDITIONS
data/README.md ADDED
@@ -0,0 +1,57 @@
1
+ # GTFS Reader
2
+
3
+ ```ruby
4
+ gem 'gtfs-reader' git: 'git@github.com:sangster/gtfs-reader.git'
5
+ ```
6
+
7
+ GTFS Reader is a gem designed to help process the contents of a "[GTFS
8
+ Feed](https://developers.google.com/transit/gtfs)":
9
+
10
+ > The General Transit Feed Specification (GTFS) defines a common format for
11
+ > public transportation schedules and associated geographic information. GTFS
12
+ > "feeds" allow public transit agencies to publish their transit data and
13
+ > developers to write applications that consume that data in an interoperable
14
+ > way.
15
+
16
+ Essentially, a GTFS feed is a ZIP file containing
17
+ [CSV-formatted](https://en.wikipedia.org/wiki/Comma-separated_values) .txt
18
+ files which following the specification.
19
+
20
+ ## Usage
21
+
22
+ ## Simple Example
23
+
24
+ ```ruby
25
+ require 'gtfs_reader'
26
+
27
+ GtfsReader.config do
28
+ return_hashes true
29
+
30
+ sources do
31
+ sample do
32
+ url 'http://localhost/sample-feed.zip'
33
+ before { |etag| puts "Processing source with tag #{etag}..." }
34
+ handlers do
35
+ agency {|row| puts "Read Agency: #{row[:agency_name]}" }
36
+ routes {|row| puts "Read Route: #{row[:route_long_name]}" }
37
+ end
38
+ end
39
+ end
40
+ end
41
+
42
+ GtfsReader.update :sample # or GtfsReader.update_all!
43
+ ```
44
+
45
+ Assuming that `http://localhost/sample-feed.zip` returns the [Example Feed]
46
+ (https://developers.google.com/transit/gtfs/examples/gtfs-feed), this script
47
+ will print the following:
48
+
49
+ ```
50
+ Processing source with tag 4d9d3040c284f0581cd5620d5c131109...
51
+ Read Agency: Demo Transit Authority
52
+ Read Route: Airport - Bullfrog
53
+ Read Route: Bullfrog - Furnace Creek Resort
54
+ Read Route: Stagecoach - Airport Shuttle
55
+ Read Route: City
56
+ Read Route: Airport - Amargosa Valley
57
+ ```
data/Rakefile ADDED
@@ -0,0 +1,53 @@
1
+ # encoding: utf-8
2
+
3
+ require 'rubygems'
4
+ require 'bundler'
5
+ require 'rake'
6
+ require 'jeweler'
7
+
8
+ require_relative 'lib/gtfs_reader'
9
+ require_relative 'lib/gtfs_reader/version'
10
+
11
+ begin
12
+ Bundler.setup :default, :development
13
+ rescue Bundler::BundlerError => e
14
+ $stderr.puts e.message
15
+ $stderr.puts 'Run `bundle install` to install missing gems'
16
+ exit e.status_code
17
+ end
18
+
19
+ Jeweler::Tasks.new do |gem|
20
+ gem.name = 'gtfs-reader'
21
+ gem.version = GtfsReader::Version.to_s
22
+ gem.homepage = 'http://github.com/sangster/gtfs-reader'
23
+ gem.license = 'GPL 3'
24
+ gem.summary = 'Read General Transit Feed Specification zip files'
25
+ gem.description = <<-EOF.strip.gsub /\s+/, ' '
26
+ Reads and parses zip files conforming to Google's GTFS spec. Such files can
27
+ take up quite a bit of memory when inflated, so this gem prefers to read
28
+ them as a stream of rows.
29
+
30
+ GTFS Spec: https://developers.google.com/transit/gtfs
31
+ EOF
32
+ gem.email = 'jon@ertt.ca'
33
+ gem.authors = ['Jon Sangster']
34
+
35
+ gem.files = Dir['{lib}/**/*', 'Rakefile', 'README.md', 'LICENSE']
36
+ end
37
+
38
+ Jeweler::RubygemsDotOrgTasks.new
39
+
40
+ task :default => :pry
41
+
42
+ task :pry do
43
+ exec 'pry --gem'
44
+ end
45
+
46
+ task bump: ['bump:patch']
47
+ namespace :bump do
48
+ %i[major minor patch].each do |part|
49
+ bumper = GtfsReader::Version::Bumper.new part
50
+ desc "Bump the version to #{bumper}"
51
+ task(part) { bumper.bump }
52
+ end
53
+ end
@@ -0,0 +1,7 @@
1
+ module GtfsReader
2
+ end
3
+
4
+ require_relative 'gtfs_reader/exceptions'
5
+ require_relative 'gtfs_reader/core'
6
+ require_relative 'gtfs_reader/config/defaults/gtfs_feed_definition'
7
+ require_relative 'gtfs_reader/log'
@@ -0,0 +1,94 @@
1
+ module GtfsReader
2
+ class BulkFeedHandler
3
+ def initialize(bulk_size, args=[], &block)
4
+ @bulk_size = bulk_size
5
+ @callbacks = {}
6
+ BulkFeedHandlerDsl.new(self).instance_exec *args, &block
7
+ end
8
+
9
+ def handler?(filename)
10
+ @callbacks.key? filename
11
+ end
12
+
13
+ def handle_file(filename, reader)
14
+ unless @callbacks.key? filename
15
+ Log.warn { "No handler registered for #{filename.to_s.red}.txt" }
16
+ return
17
+ end
18
+
19
+ calls = callbacks filename
20
+ calls[:before].call if calls.key? :before
21
+ read_row = calls[:read]
22
+
23
+ values = []
24
+ total = bulk_count = 0
25
+ cols = reader.col_names
26
+ reader.each do |row|
27
+ values << (read_row ? read_row.call(row) : row)
28
+ bulk_count += 1
29
+
30
+ if bulk_count == @bulk_size
31
+ total += bulk_count
32
+ calls[:bulk].call values, bulk_count, total, cols
33
+ bulk_count = 0
34
+ values = []
35
+ end
36
+ end
37
+
38
+ unless bulk_count == 0
39
+ calls[:bulk].call values, bulk_count, (total + bulk_count), cols
40
+ end
41
+ nil
42
+ end
43
+
44
+ def create_callback(kind, filename, block)
45
+ Log.debug do
46
+ kind_str = kind.to_s.center(:before.to_s.length).magenta
47
+ "create callback (#{kind_str}) #{filename.to_s.red}"
48
+ end
49
+ callbacks(filename)[kind] = block
50
+ end
51
+
52
+ def callback?(kind, filename)
53
+ @callbacks.key? filename and @callbacks[filename].key? kind
54
+ end
55
+
56
+ private
57
+
58
+ def callbacks(filename)
59
+ @callbacks[filename] ||= {}
60
+ end
61
+ end
62
+
63
+ class BulkFeedHandlerDsl
64
+ def initialize(feed_handler)
65
+ @feed_handler = feed_handler
66
+ end
67
+
68
+ def method_missing(filename, *args, &block)
69
+ BulkDsl.new(@feed_handler, filename).instance_exec &block
70
+
71
+ unless @feed_handler.callback? :bulk, filename
72
+ raise HandlerMissingError, "No bulk block for #{filename}"
73
+ end
74
+ end
75
+ end
76
+
77
+ class BulkDsl
78
+ def initialize(feed_handler, filename)
79
+ @feed_handler, @filename = feed_handler, filename
80
+ end
81
+
82
+ def before(&block)
83
+ @feed_handler.create_callback :before, @filename, block
84
+ end
85
+
86
+ def read(&block)
87
+ @feed_handler.create_callback :read, @filename, block
88
+ end
89
+
90
+ def bulk(&block)
91
+ @feed_handler.create_callback :bulk, @filename, block
92
+ end
93
+ end
94
+ end