json-emitter 0.0.4 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5ffd5b387bbbf0ce7f40a9b725dbaaea83a479ac53ada2ede1f01231c5be28e1
4
- data.tar.gz: c0af5bb60b80c67da129f2ba14fea69a99c97ba69a47e508de165ebb12bf11e7
3
+ metadata.gz: d2228e23ca76886db68f546e171db2e9c318f35c02e4262b19adf162d43e1151
4
+ data.tar.gz: c137299a78e76f0ffdeba789d9477ef94a4b161d0718287ba074dcbc90f4c82d
5
5
  SHA512:
6
- metadata.gz: 6ffb66b5cae058582b8d0904b6ae55272b7065ea6f412a2ddc7ab6db89a030f1e13c7801894e672f182c2da0b72d9e7211b9751f500f05fa91c8356679416403
7
- data.tar.gz: e7a855f40029fbd7343cd4bfebf0e59594d81257fd2d0a232b8bf9cee4d0ac82210c8fa8907f6c5ae0eeaf4ee3acb68ca5b58adaa9a7525c1c094ac8f67e6d35
6
+ metadata.gz: b2a21066822a1e4f84df8facee734dd60951dbd489d5e80444be11546e9851e77f9d8c9edc6f9fd1900fd55407cf96cbc193c4402223aa78459265a119ab75a5
7
+ data.tar.gz: 2b025e42af1069efc578d5138e6535b35b507659e931ce59dd9164d96adc7dd1d1ceec334b22aa710a233198b63d8cad033b67e244d52a78854d553383e41bea
data/README.md CHANGED
@@ -1,12 +1,12 @@
1
- # JsonEmitter
1
+ # JsonEmitter [![Build Status](https://travis-ci.org/jhollinger/json-emitter.svg?branch=master)](https://travis-ci.org/jhollinger/json-emitter)
2
2
 
3
3
  JsonEmitter is a library for efficiently generating very large bits of JSON in Ruby. Need to generate a JSON array of 10,000 database records without eating up all your RAM? No problem! Objects? Nested structures? JsonEmitter has you covered.
4
4
 
5
- Use JsonEmitter in your Rack/Rails/Sinatra/Grape API to stream large JSON responses without worrying about RAM or HTTP timeouts. Use it to write large JSON objects to your filesystem, S3, or ~~a 3D printer~~ anywhere else!
5
+ Use JsonEmitter in your Rack/Rails/Sinatra/Grape API to stream large JSON responses without worrying (much) about RAM or HTTP timeouts. Use it to write large JSON objects to your filesystem, S3, or ~~a 3D printer~~ anywhere else!
6
6
 
7
7
  # HTTP Chunked Responses
8
8
 
9
- These examples will use the Order enumerator to generate chunks of JSON and send them to the client as more chunks are generated. No more than 500 orders will be in memory at a time, regardless of how many orders there are. And only small portions of the JSON will be in memory at once, no matter how much we're generating.
9
+ These examples will use the Order enumerator to generate chunks of JSON and send them to the client as more chunks are generated. No more than 500 orders will be in memory at a time, regardless of how many orders there are. And only small portions of the JSON will be in memory at once, no matter how much we're generating. (NOTE There **are** limits to how long something will stream - your app server, nginx/apache, your browser, etc. may cut it off eventually.)
10
10
 
11
11
  ```ruby
12
12
  enumerator = Order.
@@ -70,7 +70,7 @@ JsonEmitter.array(enumerator).each { |json_chunk|
70
70
  }
71
71
  ```
72
72
 
73
- `JsonEmitter.object` takes a `Hash` and returns a stream that generates chunks of JSON.
73
+ `JsonEmitter.object` takes a `Hash` and returns a stream that generates chunks of JSON. Hash values can be literals, Enumerators, Arrays, other Hashes, or Procs that return any of those. Hashes and arrays may be nested.
74
74
 
75
75
  ```ruby
76
76
  JsonEmitter.object({
@@ -97,3 +97,42 @@ File.open("~/out.json", "w+") { |f|
97
97
  JsonEmitter.array(enumerator).write f
98
98
  }
99
99
  ```
100
+
101
+ # Deailing with errors
102
+
103
+ When streaming an HTTP response, you can't change the response code once you start sending data. So if you hit an error after you start, you need another way to communicate errors to the client.
104
+
105
+ One way is to always steam an object that includes an `errors` field. Any errors will be collected while the `Enumerator` is running. After it's finished, they'll be added to the JSON object.
106
+
107
+ ```ruby
108
+ def get_data
109
+ errors = []
110
+ enum = Enumerator.new { |y|
111
+ finished = false
112
+ until finished
113
+ data, errs, finished = get_data_chunk
114
+ if errs
115
+ errors += errs
116
+ finished = true
117
+ next
118
+ end
119
+ data.each { x| y << x }
120
+ end
121
+ }
122
+ return enum, -> { errors }
123
+ end
124
+
125
+ items_enum, errors_proc = get_data
126
+ JsonEmitter.object({
127
+ items: items_enum,
128
+ errors: errors_proc,
129
+ })
130
+ ```
131
+
132
+ # License
133
+
134
+ MIT License. See LICENSE for details.
135
+
136
+ # Copyright
137
+
138
+ Copywrite (c) 2019 Jordan Hollinger.
@@ -8,7 +8,11 @@ module JsonEmitter
8
8
  # and JsonEmitter.error to add critical behavior back in.
9
9
  #
10
10
  class Context
11
- def initialize
11
+ # @return [Hash] The Rack environment Hash (if present)
12
+ attr_reader :rack_env
13
+
14
+ def initialize(rack_env: nil)
15
+ @rack_env = rack_env
12
16
  @wrappers = JsonEmitter.wrappers.map(&:call).compact
13
17
  @error_handlers = JsonEmitter.error_handlers
14
18
  @pass_through_errors = []
@@ -29,6 +33,13 @@ module JsonEmitter
29
33
  @error_handlers += [handler]
30
34
  end
31
35
 
36
+ # Returns a Rack::Request from rack_env (if rack_env was given).
37
+ def request
38
+ if rack_env
39
+ @request ||= Rack::Request.new(rack_env)
40
+ end
41
+ end
42
+
32
43
  # Execute a block within this context.
33
44
  def execute(&inner)
34
45
  @wrappers.reduce(inner) { |f, outer_wrapper|
@@ -38,7 +49,7 @@ module JsonEmitter
38
49
  rescue *@pass_through_errors => e
39
50
  raise e
40
51
  rescue => e
41
- @error_handlers.each { |h| h.call(e) }
52
+ @error_handlers.each { |h| h.call(e, self) }
42
53
  raise e
43
54
  end
44
55
  end
@@ -1,4 +1,6 @@
1
1
  module JsonEmitter
2
+ COMMA = ",".freeze
3
+
2
4
  #
3
5
  # Builds Enumerators that yield JSON from Ruby Arrays or Hashes.
4
6
  #
@@ -6,8 +8,8 @@ module JsonEmitter
6
8
  # @return [JsonEmitter::Context]
7
9
  attr_reader :context
8
10
 
9
- def initialize
10
- @context = Context.new
11
+ def initialize(rack_env: nil)
12
+ @context = Context.new(rack_env: rack_env)
11
13
  end
12
14
 
13
15
  #
@@ -51,7 +53,7 @@ module JsonEmitter
51
53
 
52
54
  first = true
53
55
  enum.each { |val|
54
- y << ",".freeze unless first
56
+ y << COMMA unless first
55
57
  first = false if first
56
58
 
57
59
  mapped_val = mapper ? mapper.call(val) : val
@@ -70,7 +72,7 @@ module JsonEmitter
70
72
 
71
73
  first = true
72
74
  hash.each { |key, val|
73
- y << ",".freeze unless first
75
+ y << COMMA unless first
74
76
  first = false if first
75
77
 
76
78
  json_key = MultiJson.dump(key.to_s)
@@ -1,4 +1,4 @@
1
1
  module JsonEmitter
2
2
  # Library version
3
- VERSION = "0.0.4".freeze
3
+ VERSION = "1.0.0".freeze
4
4
  end
data/lib/json-emitter.rb CHANGED
@@ -51,11 +51,12 @@ module JsonEmitter
51
51
  # @param enum [Enumerable] Something that can be enumerated over, like an Array or Enumerator. Each element should be something that can be rendered as JSON (e.g. a number, string, boolean, Array, or Hash).
52
52
  # @param buffer_size [Integer] The buffer size in kb. This is a size *hint*, not a hard limit.
53
53
  # @param buffer_unit [Symbol] :bytes | :kb (default) | :mb
54
+ # @param rack [Hash] optional Rack env for error handling
54
55
  # @yield If a block is given, it will be yielded each value in the array. The return value from the block will be converted to JSON instead of the original value.
55
56
  # @return [JsonEmitter::BufferedStream]
56
57
  #
57
- def self.array(enum, buffer_size: 16, buffer_unit: :kb, &mapper)
58
- emitter = Emitter.new.array(enum, &mapper)
58
+ def self.array(enum, buffer_size: 16, buffer_unit: :kb, rack: nil, &mapper)
59
+ emitter = Emitter.new(rack_env: rack).array(enum, &mapper)
59
60
  BufferedStream.new(emitter, buffer_size, unit: buffer_unit)
60
61
  end
61
62
 
@@ -95,10 +96,11 @@ module JsonEmitter
95
96
  # @param hash [Hash] Keys should be Strings or Symbols and values should be any JSON-compatible value like a number, string, boolean, Array, or Hash.
96
97
  # @param buffer_size [Integer] The buffer size in kb. This is a size *hint*, not a hard limit.
97
98
  # @param buffer_unit [Symbol] :bytes | :kb (default) | :mb
99
+ # @param rack [Hash] optional Rack env for error handling
98
100
  # @return [JsonEmitter::BufferedStream]
99
101
  #
100
- def self.object(hash, buffer_size: 16, buffer_unit: :kb)
101
- emitter = Emitter.new.object(hash)
102
+ def self.object(hash, buffer_size: 16, buffer_unit: :kb, rack: nil)
103
+ emitter = Emitter.new(rack_env: rack).object(hash)
102
104
  BufferedStream.new(emitter, buffer_size, unit: buffer_unit)
103
105
  end
104
106
 
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: json-emitter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.4
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Jordan Hollinger
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-01-31 00:00:00.000000000 Z
11
+ date: 2022-07-31 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: multi_json
@@ -56,8 +56,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
56
56
  - !ruby/object:Gem::Version
57
57
  version: '0'
58
58
  requirements: []
59
- rubyforge_project:
60
- rubygems_version: 2.7.6
59
+ rubygems_version: 3.0.3.1
61
60
  signing_key:
62
61
  specification_version: 4
63
62
  summary: Efficiently generate tons of JSON