json-streamer 1.1.2 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +36 -14
- data/lib/json/streamer.rb +3 -1
- data/lib/json/streamer/json_streamer.rb +5 -1
- data/lib/json/streamer/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3c35a9dba96aa722b2fb6cb9077110e878b021a6
|
4
|
+
data.tar.gz: 88ccd53d4b71d5c4ae77a4dbc7ce269c727f7a0a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: '09599f0613fae47060a80144e95ff2ef2aecee3bc6e7beb53580185f702d8606fe5b5ffa9665462a1cd3eae7739a77e7cfa6284a5e0fc26a12bc5a3142d625c0'
|
7
|
+
data.tar.gz: 2a8bba6340930503cdd41d4702403a6d4f3d235fcc691a95bf81d14590a92d67be0239308a563df3158d72544251c1a03c0ba9248a03379b0cc849df6244ef80
|
data/README.md
CHANGED
@@ -47,18 +47,25 @@ Or install it yourself as:
|
|
47
47
|
|
48
48
|
```ruby
|
49
49
|
require 'json/streamer'
|
50
|
+
```
|
51
|
+
|
52
|
+
### v1.1.3 API
|
53
|
+
|
54
|
+
Check the unit tests for more examples ([spec/streamer_spec.rb](spec/json/streamer/json_streamer_spec.rb)).
|
55
|
+
|
56
|
+
#### Passing IO upfront
|
50
57
|
|
58
|
+
```ruby
|
51
59
|
file_stream = File.open('data.json', 'r')
|
60
|
+
chunk_size = 500 # defaults to 1000
|
52
61
|
|
53
|
-
|
54
|
-
# Default chunk size in 1000
|
55
|
-
streamer = Json::Streamer::JsonStreamer.new(file_stream, 500)
|
62
|
+
streamer = Json::Streamer.parser(file_io: file_stream, chunk_size: chunk_size)
|
56
63
|
```
|
57
64
|
|
58
65
|
#### Get objects based on nesting level
|
59
66
|
|
60
67
|
```ruby
|
61
|
-
# Level zero
|
68
|
+
# Level zero yields the full JSON, first level yields data within the JSON 1-by-1, etc.
|
62
69
|
streamer.get(nesting_level:1) do |object|
|
63
70
|
p object
|
64
71
|
end
|
@@ -109,7 +116,7 @@ Output:
|
|
109
116
|
{"desired_key" : "value3"}
|
110
117
|
```
|
111
118
|
|
112
|
-
#### Skip values
|
119
|
+
#### Skip values
|
113
120
|
|
114
121
|
```ruby
|
115
122
|
streamer.get(nesting_level:1, yield_values:false) do |object|
|
@@ -130,12 +137,12 @@ Output:
|
|
130
137
|
{}
|
131
138
|
```
|
132
139
|
|
133
|
-
#### EventMachine-style
|
140
|
+
#### Passing IO later (EventMachine-style)
|
134
141
|
|
135
142
|
```ruby
|
136
143
|
# Get a JsonStreamer object that provides access to the parser
|
137
144
|
# but does not start processing immediately
|
138
|
-
streamer = Json::Streamer
|
145
|
+
streamer = Json::Streamer.parser
|
139
146
|
|
140
147
|
streamer.get(nesting_level:1) do |object|
|
141
148
|
p object
|
@@ -146,11 +153,23 @@ Then later in your EventMachine handler:
|
|
146
153
|
|
147
154
|
```ruby
|
148
155
|
def receive_data(data)
|
149
|
-
streamer
|
156
|
+
streamer << data
|
150
157
|
end
|
151
158
|
```
|
152
159
|
|
153
|
-
|
160
|
+
### Legacy API (pre-v1.1.3)
|
161
|
+
|
162
|
+
This functionality is deprecated but kept for compatibility reasons.
|
163
|
+
|
164
|
+
```ruby
|
165
|
+
# Same as Json::Streamer.parser
|
166
|
+
streamer = Json::Streamer::JsonStreamer.new
|
167
|
+
```
|
168
|
+
|
169
|
+
```ruby
|
170
|
+
# Same as streamer << data
|
171
|
+
streamer.parser << data
|
172
|
+
```
|
154
173
|
|
155
174
|
## Feedback
|
156
175
|
|
@@ -160,17 +179,20 @@ I can only tailor this project to fit use-cases I know about - which are usually
|
|
160
179
|
|
161
180
|
Please let me know if you make use of this project so that I can prioritize further efforts.
|
162
181
|
|
163
|
-
##
|
182
|
+
## Conventions
|
164
183
|
|
165
|
-
|
166
|
-
|
167
|
-
|
184
|
+
This gem is developed using the following conventions:
|
185
|
+
- [Bundler's guide for developing a gem](http://bundler.io/v1.14/guides/creating_gem.html)
|
186
|
+
- [Better Specs](http://www.betterspecs.org/)
|
187
|
+
- [Semantic versioning](http://semver.org/)
|
188
|
+
- [RubyGems' guide on gem naming](http://guides.rubygems.org/name-your-gem/)
|
189
|
+
- [RFC memo about key words used to Indicate Requirement Levels](https://tools.ietf.org/html/rfc2119)
|
190
|
+
- [Bundler improvements](https://github.com/thisismydesign/bundler-improvements)
|
168
191
|
|
169
192
|
## Contributing
|
170
193
|
|
171
194
|
Bug reports and pull requests are welcome on GitHub at https://github.com/thisismydesign/json-streamer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
172
195
|
|
173
|
-
|
174
196
|
## License
|
175
197
|
|
176
198
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|
data/lib/json/streamer.rb
CHANGED
@@ -22,8 +22,12 @@ module Json
|
|
22
22
|
@parser.key {|k| key(k)}
|
23
23
|
end
|
24
24
|
|
25
|
+
def <<(data)
|
26
|
+
@parser << data
|
27
|
+
end
|
28
|
+
|
25
29
|
# Callbacks containing `yield` have to be defined in the method called via block otherwise yield won't work
|
26
|
-
def get(nesting_level
|
30
|
+
def get(nesting_level: -1, key: nil, yield_values: true)
|
27
31
|
@yield_level = nesting_level
|
28
32
|
@yield_key = key
|
29
33
|
@yield_values = yield_values
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: json-streamer
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- thisismydesign
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-09-
|
11
|
+
date: 2017-09-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|