elasticgraph-indexer_lambda 1.0.0.rc3 → 1.0.0.rc4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +8 -21
  3. metadata +12 -12
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9e1d3f12436c1cd61f87692018f725ebcd45791fbb27e7866f7258ea558829d1
4
- data.tar.gz: a7f30ebf536d235ead8d0dd62139510bf8579d766c3eb24383c3de2cb7a31b9a
3
+ metadata.gz: 5c9198223d77296719652035f66cb255cbee4c5a101a680e21223f1c54eaf556
4
+ data.tar.gz: ef4dd620eeb509c4effda57101d92cc615a7d5780bd4d2f34799f56bb075f853
5
5
  SHA512:
6
- metadata.gz: c353394d040a2996f236a0c1a9c8bf89f5db71203b2d5badcf7c6baeb7467be4ebcebefba2e3c96b99789415f419a94a1d642ec2a2d27cc2e53b3cce0e07ea0c
7
- data.tar.gz: 52853ea123e634c1e2df1527e4d4da826d67bf884a61eeb8dfddfdd1e80ff6ada1f4fd7fb30d4bc4263b7628986f9137f92e86c178871bb35681e53449a77a9f
6
+ metadata.gz: c8d91aa2af7eef9e97f7f4a4df7f5a38c8b9fff496a64944a58037e241d02e67ec56b36836eedde6ade452dab063ce60263f7797ea88a9957c5d123c34b370f0
7
+ data.tar.gz: 4cbaeadc8a1cd2e52fb2e8a614f5ae290513524e46cf620627ec999dbb2a9f076ed7d938be8dc6b4c1aae0513fc2bf8a6ec544fc43e32f83712095a5c2e06263
data/README.md CHANGED
@@ -29,28 +29,15 @@ graph LR;
29
29
 
30
30
  ## SQS Message Payload Format
31
31
 
32
- We use [JSON Lines](http://jsonlines.org/) to encode our indexing events. It is just individual JSON objects
32
+ This gem is designed to run in an AWS lambda that consumes from an SQS queue. Messages in the SQS queue should use
33
+ [JSON Lines](https://jsonlines.org/) format to encode indexing events.
34
+
35
+ JSON lines format contains individual JSON objects
33
36
  delimited by a newline control character(not the `\n` string sequence), such as:
34
37
 
35
- ```jsonl
36
- {"op": "upsert", "__typename": "Payment", "id": "123", "version": "1", "record": {...} }
37
- {"op": "upsert", "__typename": "Payment", "id": "123", "version": "2", record: {...} }
38
- {"op": "delete", "__typename": "Payment", "id": "123", "version": "3"}
39
38
  ```
40
-
41
- However, due to SQS message size limit, we have to batch our events carefully so each batch is below the size limit.
42
- This makes payload encoding a bit more complicated on the publisher side because each message has a size limit.
43
- The following code snippet respects the max message size limit and sends JSON Lines payloads with proper size:
44
-
45
- ```ruby
46
- def partition_into_acceptably_sized_chunks(batch, max_size_per_chunk)
47
- chunk_size = 0
48
- batch
49
- .map { |item| JSON.generate(item) }
50
- .slice_before do |json|
51
- chunk_size += (json.bytesize + 1)
52
- (chunk_size > max_size_per_chunk).tap { |chunk_done| chunk_size = 0 if chunk_done }
53
- end
54
- .map { |chunk| chunk.join("\n") }
55
- end
39
+ {"op": "upsert", "__typename": "Widget", "id": "123", "version": 1, "record": {...} }
40
+ {"op": "upsert", "__typename": "Widget", "id": "123", "version": 2, record: {...} }
56
41
  ```
42
+
43
+ When publishing into SQS, be sure to keep messages under the [256 KiB SQS message limit](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html).
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: elasticgraph-indexer_lambda
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.0.0.rc3
4
+ version: 1.0.0.rc4
5
5
  platform: ruby
6
6
  authors:
7
7
  - Myron Marston
@@ -17,42 +17,42 @@ dependencies:
17
17
  requirements:
18
18
  - - '='
19
19
  - !ruby/object:Gem::Version
20
- version: 1.0.0.rc3
20
+ version: 1.0.0.rc4
21
21
  type: :runtime
22
22
  prerelease: false
23
23
  version_requirements: !ruby/object:Gem::Requirement
24
24
  requirements:
25
25
  - - '='
26
26
  - !ruby/object:Gem::Version
27
- version: 1.0.0.rc3
27
+ version: 1.0.0.rc4
28
28
  - !ruby/object:Gem::Dependency
29
29
  name: elasticgraph-lambda_support
30
30
  requirement: !ruby/object:Gem::Requirement
31
31
  requirements:
32
32
  - - '='
33
33
  - !ruby/object:Gem::Version
34
- version: 1.0.0.rc3
34
+ version: 1.0.0.rc4
35
35
  type: :runtime
36
36
  prerelease: false
37
37
  version_requirements: !ruby/object:Gem::Requirement
38
38
  requirements:
39
39
  - - '='
40
40
  - !ruby/object:Gem::Version
41
- version: 1.0.0.rc3
41
+ version: 1.0.0.rc4
42
42
  - !ruby/object:Gem::Dependency
43
43
  name: aws-sdk-s3
44
44
  requirement: !ruby/object:Gem::Requirement
45
45
  requirements:
46
46
  - - "~>"
47
47
  - !ruby/object:Gem::Version
48
- version: '1.192'
48
+ version: '1.195'
49
49
  type: :runtime
50
50
  prerelease: false
51
51
  version_requirements: !ruby/object:Gem::Requirement
52
52
  requirements:
53
53
  - - "~>"
54
54
  - !ruby/object:Gem::Version
55
- version: '1.192'
55
+ version: '1.195'
56
56
  - !ruby/object:Gem::Dependency
57
57
  name: ox
58
58
  requirement: !ruby/object:Gem::Requirement
@@ -89,10 +89,10 @@ licenses:
89
89
  - MIT
90
90
  metadata:
91
91
  bug_tracker_uri: https://github.com/block/elasticgraph/issues
92
- changelog_uri: https://github.com/block/elasticgraph/releases/tag/v1.0.0.rc3
93
- documentation_uri: https://block.github.io/elasticgraph/api-docs/v1.0.0.rc3/
92
+ changelog_uri: https://github.com/block/elasticgraph/releases/tag/v1.0.0.rc4
93
+ documentation_uri: https://block.github.io/elasticgraph/api-docs/v1.0.0.rc4/
94
94
  homepage_uri: https://block.github.io/elasticgraph/
95
- source_code_uri: https://github.com/block/elasticgraph/tree/v1.0.0.rc3/elasticgraph-indexer_lambda
95
+ source_code_uri: https://github.com/block/elasticgraph/tree/v1.0.0.rc4/elasticgraph-indexer_lambda
96
96
  gem_category: lambda
97
97
  rdoc_options: []
98
98
  require_paths:
@@ -111,7 +111,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
111
111
  - !ruby/object:Gem::Version
112
112
  version: '0'
113
113
  requirements: []
114
- rubygems_version: 3.6.7
114
+ rubygems_version: 3.6.9
115
115
  specification_version: 4
116
- summary: Provides an AWS Lambda interface for an elasticgraph API
116
+ summary: Adapts elasticgraph-indexer to run in an AWS Lambda.
117
117
  test_files: []