propono 2.0.0.rc3 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 97673d01a5c39db6689660e59a8667a342c759ab
4
- data.tar.gz: 3e221dff98ec598ee9bd3144bca3dbae9a286de2
2
+ SHA256:
3
+ metadata.gz: a7790d050fa3ee543affb106f6ef8478f7576c07270a7c285ffb3dc60dc65635
4
+ data.tar.gz: 9fab2edbfda2def0ce81b27ff12a9da4d57ecc3b4e45fded5d1513b2a02b855e
5
5
  SHA512:
6
- metadata.gz: 94bbdc2ef73278a7258ef1d404c225e26e40db5f62cd5f5f3d1b7a35380b6867e256fc39dbda5755291bca3c5333444190a516a86ffcacaac98a21559cc0371c
7
- data.tar.gz: cac7956743dbf2c7eab5237966471ccdc86cc460756d9f0d113284637d360746af27abbb1dc4eea65a962cca18a0a6696a4a74d279e819ad7a152ec8af215af8
6
+ metadata.gz: beb2e62ada80b404c4b590b88e2166034217f812483c097e1d1b86585076c261e7bc6b947a1a6e6cfdf3af362c63cdeeb9760d85c3c4dcfed5016b18b14bdd5b
7
+ data.tar.gz: 694302c72bd5aaada4f5472707f52c491d13d2707c7cc3d69575c9330961cd8049e5dcb4baf1c08f24ed5b9d8ae07173849995c922eae559a6746ee806460efb
@@ -0,0 +1,30 @@
1
+ name: Tests
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+ branches: [main]
8
+
9
+ jobs:
10
+ test:
11
+ name: Ruby ${{ matrix.ruby-version }} - ${{ matrix.os }} - ${{ github.event_name }}
12
+ runs-on: ${{ matrix.os }}
13
+ strategy:
14
+ fail-fast: false
15
+ matrix:
16
+ os:
17
+ - ubuntu-latest
18
+ ruby-version: [2.6, 2.7]
19
+
20
+ steps:
21
+ - uses: actions/checkout@v2
22
+
23
+ - name: Set up Ruby
24
+ uses: ruby/setup-ruby@a699edbce608a2c128dedad88e3b6a0e28687b3c
25
+ with:
26
+ ruby-version: ${{ matrix.ruby-version }}
27
+ bundler-cache: true
28
+
29
+ - name: Test
30
+ run: bundle exec rake test:local
data/.ruby-version ADDED
@@ -0,0 +1 @@
1
+ 2.6.6
data/CHANGELOG.md CHANGED
@@ -1,3 +1,12 @@
1
+ # 3.0.0 / 2021-01-10
2
+ * [FEATURE] Improve AWS configuration.
3
+
4
+ # 2.2.0 / 2019-09-21
5
+ * [FEATURE] Add setting to disable slow queue
6
+
7
+ # 2.1.0 / 2017-03-14
8
+ * [FEATURE] Added visibility_timeout to listen
9
+
1
10
  # 2.0.0 / 2017-03-14
2
11
  * [FEATURE] Remove UDP and TCP support
3
12
  * [FEATURE] Change default publish behaviour from async to sync
data/README.md CHANGED
@@ -21,13 +21,9 @@ Propono::Client.new.publish('some-topic', "The Best Message Ever")
21
21
  # - "I just received The Best Message Ever"
22
22
  ```
23
23
 
24
- ## Changes from v1 to v2
24
+ ## Upgrading
25
25
 
26
- Version 2 of Propono changed a few things:
27
- - We moved from a global interface to a client interface. Rather than calling `publish` and equivalent on `Propono`, you should now initialize a `Propono::Client` and then call everything on that client. This fixes issues with thread safety and global config.
28
- - We have also removed the dependancy on Fog and instead switch to the `sns` and `sqs` mini-gems of `aws-sdk`.
29
- - UDP and TCP support have been removed, and `subscribe_by_post` has been removed.
30
- - We are now using long-polling. This makes Propono **significantly** faster (10-100x).
26
+ Upgrades from v1 to v2, and v2 to v3 are covered in the [upgrade documentation](docs/upgrading.md).
31
27
 
32
28
  ## Installation
33
29
 
@@ -41,18 +37,12 @@ And then execute:
41
37
 
42
38
  ## Usage
43
39
 
44
- The first thing to do is setup some configuration keys for Propono. It's best to do this in an initializer, or at the start of your application.
40
+ The first thing to do is setup some configuration for Propono.
41
+ It's best to do this in an initializer, or at the start of your application.
42
+ If you need to setup AWS authentication, see the [AWS Configuration](#aws-configuration) section.
45
43
 
46
44
  ```ruby
47
45
  client = Propono::Client.new
48
- client.config.access_key = "access-key" # From AWS
49
- client.config.secret_key = "secret-key" # From AWS
50
- client.config.queue_region = "queue-region" # From AWS
51
-
52
- # Or use the IAM profile of the machine
53
- client.config.use_iam_profile = true
54
- client.config.queue_region = "queue-region" # From AWS
55
-
56
46
  ```
57
47
 
58
48
  You can then start publishing messages easily from anywhere in your codebase.
@@ -81,28 +71,64 @@ This is because a queue is established for each application_name/topic combinati
81
71
  * subscribers that share the same `application_name` will act as multiple workers on the same queue. Only one will get to process each message.
82
72
  * subscribers that have a different `application_name` will each get a copy of a message to process independently i.e. acts as a one-to-many broadcast.
83
73
 
84
- ### Configuration
74
+ ### AWS Configuration
75
+
76
+ By default, Propono will create SQS and SNS clients with no options.
77
+ In the absence of options, these clients will make use of the credentials on the current host.
78
+ See the [AWS SDK For Ruby Configuration documentation](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html) for more details.
79
+
80
+ To manually configure options for use with AWS, use `aws_options`, which sets options to be passed to both clients. For example:
81
+
82
+ client = Propono::Client.new do |config|
83
+ config.aws_options = {
84
+ region: 'aws_region',
85
+ access_key_id: 'your_access_key_id',
86
+ secret_access_key: 'your_secret_access_key'
87
+ }
88
+ end
85
89
 
86
- The following configuration settings are available:
90
+ In addition to this, there are also `sqs_options` and `sns_options`, used to configure each client independently.
91
+ See the [SQS Client](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/SQS/Client.html#initialize-instance_method) and [SNS Client](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/SNS/Client.html#initialize-instance_method) documentation for available options.
92
+ These individual options are merged with `aws_options` with the per-client options taking precendence.
93
+
94
+ ### General Configuration
87
95
 
88
96
  ```
89
97
  Propono::Client.new do |config|
90
- # Use AWS access and secret keys
91
- config.access_key = "An AWS access key"
92
- config.secret_key = "A AWS secret key"
93
- # Or use AWS IAM profile of the machine
94
- config.use_iam_profile = true
98
+ # AWS Configuration, see above.
99
+ config.aws_options = {...}
100
+ config.sqs_options = {...}
101
+ config.sns_options = {...}
95
102
 
96
- config.queue_region = "An AWS queue region"
97
103
  config.application_name = "A name unique in your network"
98
104
  config.logger = "A logger such as Log4r or Rails.logger"
99
105
 
100
106
  config.max_retries = "The number of retries if a message raises an exception before being placed on the failed queue"
101
107
  config.num_messages_per_poll = "The number of messages retrieved per poll to SQS"
108
+
109
+ config.slow_queue_enabled = true
110
+ end
111
+ ```
112
+
113
+ ### Options
114
+
115
+ #### Async
116
+
117
+ By default messages are posted inline, blocking the main thread. The `async: true` option can be sent when posting a message, which will spawn a new thread for the message networking calls, and unblocking the main thread.
118
+
119
+ #### Visiblity Timeout
120
+
121
+ For certain tasks (e.g. video processing), being able to hold messages for longer is important. To achieve this, the [visibility timeout of a message](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html) can be changed on the call to listen. e.g.
122
+
123
+ ```
124
+ client.listen('long-running-tasks', visiblity_timeout: 3600) do |message|
125
+ puts "I just received: #{message}"
102
126
  end
103
127
  ```
104
128
 
105
- These can all also be set using the `client.config.access_key = "..."` syntax.
129
+ ### Slow Queue
130
+
131
+ The slow queue can be disabled by setting `slow_queue_enabled` to `false`. This will yield performance improvements if you do not make use of the "slow queue" functionality.
106
132
 
107
133
  ### Is it any good?
108
134
 
data/docs/upgrading.md ADDED
@@ -0,0 +1,48 @@
1
+ # Upgrading
2
+
3
+ ## Changes from v2 to v3
4
+
5
+ Version 3 changed the way configuration options for the two AWS services are
6
+ passed to the client gems. Instead of Propono attempting to guess which
7
+ configuration options you might want, it now accepts hashes for AWS
8
+ configuration which are passed directly to the appropriate clients.
9
+
10
+ If you are upgrading from v2 to v3, and using the configuration as previously
11
+ given in the README, you need to change from:
12
+
13
+ ```ruby
14
+ client = Propono::Client.new
15
+ client.config.queue_region = 'aws_region'
16
+ client.config.access_key = 'your_access_key_id'
17
+ client.config.secret_key = 'your_secret_access_key'
18
+ ```
19
+
20
+ To:
21
+
22
+ ```ruby
23
+ client = Propono::Client.new do |config|
24
+ config.aws_options = {
25
+ region: 'aws_region',
26
+ access_key_id: 'your_access_key_id',
27
+ secret_access_key: 'your_secret_access_key'
28
+ }
29
+ end
30
+ ```
31
+
32
+ For a full rundown, see the [AWS Configuration
33
+ section](../README.md#aws-configuration) of the README.
34
+
35
+
36
+ ## Changes from v1 to v2
37
+
38
+ Version 2 of Propono changed a few things:
39
+ - We moved from a global interface to a client interface. Rather than calling
40
+ `publish` and equivalent on `Propono`, you should now initialize a
41
+ `Propono::Client` and then call everything on that client. This fixes issues
42
+ with thread safety and global config.
43
+ - We have also removed the dependancy on Fog and instead switch to the `sns`
44
+ and `sqs` mini-gems of `aws-sdk`.
45
+ - UDP and TCP support have been removed, and `subscribe_by_post` has been
46
+ removed.
47
+ - We are now using long-polling. This makes Propono **significantly** faster
48
+ (10-100x).
@@ -43,16 +43,18 @@ module Propono
43
43
  def set_sqs_policy(queue, policy)
44
44
  sqs_client.set_queue_attributes(
45
45
  queue_url: queue.url,
46
- attributes: { "Policy": policy }
46
+ attributes: { Policy: policy }
47
47
  )
48
48
  end
49
49
 
50
- def read_from_sqs(queue, num_messages, long_poll: true)
50
+ def read_from_sqs(queue, num_messages, long_poll: true, visibility_timeout: nil)
51
51
  wait_time_seconds = long_poll ? 20 : 0
52
+ visibility_timeout ||= 30
52
53
  sqs_client.receive_message(
53
54
  queue_url: queue.url,
54
55
  wait_time_seconds: wait_time_seconds,
55
- max_number_of_messages: num_messages
56
+ max_number_of_messages: num_messages,
57
+ visibility_timeout: visibility_timeout
56
58
  ).messages
57
59
  end
58
60
 
@@ -66,11 +68,11 @@ module Propono
66
68
  private
67
69
 
68
70
  def sns_client
69
- @sns_client ||= Aws::SNS::Client.new(aws_config.aws_options)
71
+ @sns_client ||= Aws::SNS::Client.new(aws_config.sns_options)
70
72
  end
71
73
 
72
74
  def sqs_client
73
- @sqs_client ||= Aws::SQS::Client.new(aws_config.aws_options)
75
+ @sqs_client ||= Aws::SQS::Client.new(aws_config.sqs_options)
74
76
  end
75
77
  end
76
78
  end
@@ -5,19 +5,13 @@ module Propono
5
5
  @config = config
6
6
  end
7
7
 
8
- def aws_options
9
- if @config.use_iam_profile
10
- {
11
- :use_iam_profile => true,
12
- :region => @config.queue_region
13
- }
14
- else
15
- {
16
- :access_key_id => @config.access_key,
17
- :secret_access_key => @config.secret_key,
18
- :region => @config.queue_region
19
- }
20
- end
8
+ def sqs_options
9
+ @config.aws_options.merge(@config.sqs_options)
21
10
  end
11
+
12
+ def sns_options
13
+ @config.aws_options.merge(@config.sns_options)
14
+ end
15
+
22
16
  end
23
17
  end
@@ -1,23 +1,28 @@
1
1
  module Propono
2
2
  class Client
3
3
 
4
- # Propono configuration settings.
4
+ # Propono configuration.
5
5
  #
6
6
  # Settings should be set in an initializer or using some
7
- # other method that insures they are set before any
8
- # Propono code is used. They can be set as followed:
7
+ # other method that ensures they are set before any
8
+ # Propono code is used.
9
9
  #
10
- # Propono.config.access_key = "my-access-key"
10
+ # They can be set in one of the following ways:
11
11
  #
12
- # The following settings are allowed:
12
+ # 1. As options passed to <tt>new</tt> as a hash.
13
13
  #
14
- # * <tt>:access_key</tt> - The AWS access key
15
- # * <tt>:secret_key</tt> - The AWS secret key
16
- # * <tt>:queue_region</tt> - The AWS region
17
- # * <tt>:application_name</tt> - The name of the application Propono
18
- # is included in.
19
- # * <tt>:queue_suffix</tt> - Optional string to append to topic and queue names.
20
- # * <tt>:logger</tt> - A logger object that responds to puts.
14
+ # Propono::Client.new(application_name: 'my-application')
15
+ #
16
+ # 2. As options passed to <tt>new</tt> using a block.
17
+ #
18
+ # Propono::Client.new do |config"
19
+ # config.application_name: 'my-application'
20
+ # end
21
+ #
22
+ # 3. By calling the <tt>Propono::Client#configure</tt>.
23
+ # client.configure do |config|
24
+ # config.access_key = "my-access-key"
25
+ # end
21
26
 
22
27
  attr_reader :config, :aws_client
23
28
  def initialize(settings = {}, &block)
@@ -33,7 +38,7 @@ module Propono
33
38
  @aws_client = AwsClient.new(AwsConfig.new(config))
34
39
  end
35
40
 
36
- def configure(&block)
41
+ def configure
37
42
  yield config
38
43
  end
39
44
 
@@ -69,8 +74,8 @@ module Propono
69
74
  #
70
75
  # @param [String] topic The topic to subscribe to.
71
76
  # @param &message_processor The block to yield for each message.
72
- def listen(topic_name, &message_processor)
73
- QueueListener.listen(aws_client, config, topic_name, &message_processor)
77
+ def listen(topic_name, options = {}, &message_processor)
78
+ QueueListener.listen(aws_client, config, topic_name, options, &message_processor)
74
79
  end
75
80
 
76
81
  # Listens on a queue and yields for each message
@@ -15,7 +15,7 @@ module Propono
15
15
  @topic_name = topic_name
16
16
  @suffixed_topic_name = "#{topic_name}#{propono_config.queue_suffix}"
17
17
  @suffixed_slow_topic_name = "#{topic_name}#{propono_config.queue_suffix}-slow"
18
- @queue_name = "#{propono_config.application_name.gsub(" ", "_")}-#{@suffixed_topic_name}"
18
+ @queue_name = "#{propono_config.application_name.tr(" ", "_")}-#{@suffixed_topic_name}"
19
19
  end
20
20
 
21
21
  def create
@@ -15,24 +15,26 @@ module Propono
15
15
  end
16
16
  end
17
17
 
18
- add_setting :access_key
19
- add_setting :secret_key
20
- add_setting :queue_region
18
+ add_setting :aws_options
19
+ add_setting :sqs_options
20
+ add_setting :sns_options
21
21
  add_setting :application_name
22
22
  add_setting :logger
23
23
  add_setting :max_retries
24
24
  add_setting :num_messages_per_poll
25
-
26
- add_setting :use_iam_profile, required: false
27
- add_setting :queue_suffix, required: false
25
+ add_setting :slow_queue_enabled, required: false
26
+ add_setting :queue_suffix, required: false
28
27
 
29
28
  def initialize
30
29
  @settings = {
30
+ aws_options: {},
31
+ sqs_options: {},
32
+ sns_options: {},
31
33
  logger: Propono::Logger.new,
32
34
  queue_suffix: "",
33
- use_iam_profile: false,
34
35
  max_retries: 0,
35
- num_messages_per_poll: 10
36
+ num_messages_per_poll: 1,
37
+ slow_queue_enabled: true
36
38
  }
37
39
  end
38
40
 
@@ -9,12 +9,13 @@ module Propono
9
9
  new(*args, &message_processor).drain
10
10
  end
11
11
 
12
- attr_reader :aws_client, :propono_config, :topic_name, :message_processor
13
- def initialize(aws_client, propono_config, topic_name, &message_processor)
12
+ attr_reader :aws_client, :propono_config, :topic_name, :visibility_timeout, :message_processor
13
+ def initialize(aws_client, propono_config, topic_name, options = {}, &message_processor)
14
14
  @aws_client = aws_client
15
15
  @propono_config = propono_config
16
16
  @topic_name = topic_name
17
17
  @message_processor = message_processor
18
+ @visibility_timeout = options[:visibility_timeout] || nil
18
19
  end
19
20
 
20
21
  def listen
@@ -27,18 +28,18 @@ module Propono
27
28
  def drain
28
29
  raise ProponoError.new("topic_name is nil") unless topic_name
29
30
  true while read_messages_from_queue(main_queue, 10, long_poll: false)
30
- true while read_messages_from_queue(slow_queue, 10, long_poll: false)
31
+ true while read_messages_from_queue(slow_queue, 10, long_poll: false) if propono_config.slow_queue_enabled
31
32
  end
32
33
 
33
34
  private
34
35
 
35
36
  def read_messages
36
37
  read_messages_from_queue(main_queue, propono_config.num_messages_per_poll) ||
37
- read_messages_from_queue(slow_queue, 1)
38
+ (propono_config.slow_queue_enabled ? read_messages_from_queue(slow_queue, 1) : nil)
38
39
  end
39
40
 
40
41
  def read_messages_from_queue(queue, num_messages, long_poll: true)
41
- messages = aws_client.read_from_sqs(queue, num_messages, long_poll: long_poll)
42
+ messages = aws_client.read_from_sqs(queue, num_messages, long_poll: long_poll, visibility_timeout: visibility_timeout)
42
43
  if messages.empty?
43
44
  false
44
45
  else
@@ -1,3 +1,3 @@
1
1
  module Propono
2
- VERSION = "2.0.0.rc3"
2
+ VERSION = "3.0.0"
3
3
  end
data/propono.gemspec CHANGED
@@ -6,11 +6,11 @@ require 'propono/version'
6
6
  Gem::Specification.new do |spec|
7
7
  spec.name = "propono"
8
8
  spec.version = Propono::VERSION
9
- spec.authors = ["MalcyL", "iHiD"]
10
- spec.email = ["jez.walker@gmail.com", "c.p.care@gmail.com", "malcolm@landonsonline.me.uk"]
9
+ spec.authors = ["iHiD", "dougal", "ccare", "MalcyL"]
10
+ spec.email = ["jez.walker@gmail.com"]
11
11
  spec.description = %q{Pub / Sub Library using Amazon Web Services}
12
12
  spec.summary = %q{General purpose pub/sub library built on top of AWS SNS and SQS}
13
- spec.homepage = ""
13
+ spec.homepage = "https://github.com/iHiD/propono/"
14
14
  spec.license = "MIT"
15
15
 
16
16
  spec.files = `git ls-files`.split($/)
@@ -21,7 +21,7 @@ Gem::Specification.new do |spec|
21
21
  spec.add_dependency "aws-sdk-sns"
22
22
  spec.add_dependency "aws-sdk-sqs"
23
23
 
24
- spec.add_development_dependency "bundler", "~> 1.3"
24
+ spec.add_development_dependency "bundler", "~> 2.1"
25
25
  spec.add_development_dependency "rake"
26
26
  spec.add_development_dependency "mocha"
27
27
  spec.add_development_dependency "yard"
@@ -7,47 +7,27 @@ module Propono
7
7
  super
8
8
  @config = Propono::Configuration.new
9
9
 
10
- @config.access_key = "test-access-key"
11
- @config.secret_key = "test-secret-key"
12
- @config.queue_region = "test-queue-region"
10
+ @config.aws_options = { a: 'any', b: 'aws-specific' }
11
+ @config.sqs_options = { a: 'sqs', c: 'sqs-specific' }
12
+ @config.sns_options = { a: 'sns', c: 'sns-specific' }
13
13
 
14
14
  @aws_config = Propono::AwsConfig.new(@config)
15
15
  end
16
16
 
17
- def test_access_key
18
- assert_equal "test-access-key", @aws_config.aws_options[:access_key_id]
17
+ def test_overwritten_keys_take_precendence
18
+ assert_equal 'sqs', @aws_config.sqs_options[:a]
19
+ assert_equal 'sns', @aws_config.sns_options[:a]
19
20
  end
20
21
 
21
- def test_secret_key
22
- assert_equal "test-secret-key", @aws_config.aws_options[:secret_access_key]
22
+ def test_common_keys_remain
23
+ assert_equal 'aws-specific', @aws_config.sqs_options[:b]
24
+ assert_equal 'aws-specific', @aws_config.sns_options[:b]
23
25
  end
24
26
 
25
- def test_region
26
- assert_equal "test-queue-region", @aws_config.aws_options[:region]
27
+ def test_specific_keys_remain
28
+ assert_equal 'sqs-specific', @aws_config.sqs_options[:c]
29
+ assert_equal 'sns-specific', @aws_config.sns_options[:c]
27
30
  end
28
31
 
29
- def test_no_iam_profile_selected
30
- assert ! @aws_config.aws_options.has_key?(:use_iam_profile)
31
- end
32
-
33
- def test_use_iam_profile
34
- @config.use_iam_profile = true
35
- assert @aws_config.aws_options[:use_iam_profile]
36
- end
37
-
38
- def test_selecting_use_iam_profile_results_in_no_access_key
39
- @config.use_iam_profile = true
40
- assert ! @aws_config.aws_options.has_key?(:access_key_id)
41
- end
42
-
43
- def test_selecting_use_iam_profile_results_in_no_secret_key
44
- @config.use_iam_profile = true
45
- assert ! @aws_config.aws_options.has_key?(:secret_access_key)
46
- end
47
-
48
- def test_region_when_using_iam_profile
49
- @config.use_iam_profile = true
50
- assert_equal "test-queue-region", @aws_config.aws_options[:region]
51
- end
52
32
  end
53
33
  end
@@ -40,11 +40,28 @@ module Propono
40
40
  QueueListener.expects(:listen).with(
41
41
  client.aws_client,
42
42
  client.config,
43
- topic
43
+ topic,
44
+ {}
44
45
  )
45
46
  client.listen(topic)
46
47
  end
47
48
 
49
+ def test_listen_calls_queue_listener_with_options
50
+ topic = 'foobar'
51
+ options = {foo: 'bar'}
52
+
53
+ client = Propono::Client.new
54
+ QueueListener.expects(:listen).with(
55
+ client.aws_client,
56
+ client.config,
57
+ topic,
58
+ options
59
+ )
60
+ client.listen(topic, options)
61
+ end
62
+
63
+
64
+
48
65
  def test_drain_queue_calls_queue_listener
49
66
  topic = 'foobar'
50
67
 
@@ -58,11 +75,11 @@ module Propono
58
75
  end
59
76
 
60
77
  def test_block_configuration_syntax
61
- test_key = "foobar-123-access"
78
+ test_application_name = "my-application"
62
79
  client = Propono::Client.new do |config|
63
- config.access_key = test_key
80
+ config.application_name = test_application_name
64
81
  end
65
- assert_equal test_key, client.config.access_key
82
+ assert_equal test_application_name, client.config.application_name
66
83
  end
67
84
  end
68
85
  end
@@ -1,4 +1,23 @@
1
- access_key: test-aws-access-key
2
- secret_key: test-aws-secret-key
3
- queue_region: test-aws-region
4
1
  application_name: tests-yourinitials
2
+
3
+ # Whatever keys are in aws_options are passed directly to the AWS clients.
4
+
5
+ # Option 1 - Do nothing.
6
+ # AWS clients will either use the default profile in ~/.aws/credentials, or use an IAM Role if on EC2.
7
+
8
+ # Option 2 - Use environment variables
9
+ # You can set a non-default local profile with the AWS_PROFILE environment variable.
10
+ # You can also set keys directly: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html#aws-ruby-sdk-credentials-environment
11
+
12
+ # Option 3 - Set values directly.
13
+ aws_options:
14
+ # Required
15
+ region: 'test-aws-region'
16
+
17
+ # Set keys:
18
+ # access_key_id: test-aws-access-key
19
+ # secret_access_key: test-aws-secret-key
20
+
21
+ # Or set a profile:
22
+ # profile: profile-name
23
+
@@ -12,37 +12,40 @@ module Propono
12
12
  refute propono_config.nil?
13
13
  end
14
14
 
15
- def test_use_iam_profile_defaults_false
16
- assert ! propono_config.use_iam_profile
15
+ def test_application_name
16
+ application_name = "test-application-name"
17
+ propono_config.application_name = application_name
18
+ assert_equal application_name, propono_config.application_name
17
19
  end
18
20
 
19
- def test_use_iam_profile
20
- propono_config.use_iam_profile = true
21
- assert propono_config.use_iam_profile
21
+ def test_default_aws_options
22
+ assert_equal({}, propono_config.aws_options)
22
23
  end
23
24
 
24
- def test_access_key
25
- access_key = "test-access-key"
26
- propono_config.access_key = access_key
27
- assert_equal access_key, propono_config.access_key
25
+ def test_aws_options
26
+ opts = { foo: 'bar' }
27
+ propono_config.aws_options = opts
28
+ assert_equal opts, propono_config.aws_options
28
29
  end
29
30
 
30
- def test_secret_key
31
- secret_key = "test-secret-key"
32
- propono_config.secret_key = secret_key
33
- assert_equal secret_key, propono_config.secret_key
31
+ def test_default_sqs_options
32
+ assert_equal({}, propono_config.sqs_options)
34
33
  end
35
34
 
36
- def test_queue_region
37
- queue_region = "test-queue-region"
38
- propono_config.queue_region = queue_region
39
- assert_equal queue_region, propono_config.queue_region
35
+ def test_sqs_options
36
+ opts = { foo: 'bar' }
37
+ propono_config.sqs_options = opts
38
+ assert_equal opts, propono_config.sqs_options
40
39
  end
41
40
 
42
- def test_application_name
43
- application_name = "test-application-name"
44
- propono_config.application_name = application_name
45
- assert_equal application_name, propono_config.application_name
41
+ def test_default_sns_options
42
+ assert_equal({}, propono_config.sns_options)
43
+ end
44
+
45
+ def test_sns_options
46
+ opts = { foo: 'bar' }
47
+ propono_config.sns_options = opts
48
+ assert_equal opts, propono_config.sns_options
46
49
  end
47
50
 
48
51
  def test_default_logger
@@ -65,7 +68,7 @@ module Propono
65
68
  end
66
69
 
67
70
  def test_default_num_messages_per_poll
68
- assert_equal 10, propono_config.num_messages_per_poll
71
+ assert_equal 1, propono_config.num_messages_per_poll
69
72
  end
70
73
 
71
74
  def test_num_messages_per_poll
@@ -74,24 +77,6 @@ module Propono
74
77
  assert_equal val, propono_config.num_messages_per_poll
75
78
  end
76
79
 
77
- def test_missing_access_key_throws_exception
78
- assert_raises(ProponoConfigurationError) do
79
- propono_config.access_key
80
- end
81
- end
82
-
83
- def test_missing_secret_key_throws_exception
84
- assert_raises(ProponoConfigurationError) do
85
- propono_config.secret_key
86
- end
87
- end
88
-
89
- def test_missing_queue_region_throws_exception
90
- assert_raises(ProponoConfigurationError) do
91
- propono_config.queue_region
92
- end
93
- end
94
-
95
80
  def test_missing_application_name_throws_exception
96
81
  assert_raises(ProponoConfigurationError) do
97
82
  propono_config.application_name
@@ -6,9 +6,7 @@ module Propono
6
6
  def propono_client
7
7
  config_file = YAML.load_file( File.expand_path('../../config.yml', __FILE__))
8
8
  @propono_client ||= Propono::Client.new do |config|
9
- config.access_key = config_file['access_key']
10
- config.secret_key = config_file['secret_key']
11
- config.queue_region = config_file['queue_region']
9
+ config.aws_options = config_file['aws_options']
12
10
  config.application_name = config_file['application_name']
13
11
  end
14
12
  end
@@ -58,6 +58,7 @@ module Propono
58
58
  end
59
59
  end
60
60
 
61
+ # Keep this test in sync with the one below, just with the config enabled
61
62
  def test_drain_should_continue_if_queue_empty
62
63
  @listener.expects(:read_messages_from_queue).with(@slow_queue, 10, long_poll: false).returns(false)
63
64
  @listener.expects(:read_messages_from_queue).with(@queue, 10, long_poll: false).returns(false)
@@ -65,6 +66,16 @@ module Propono
65
66
  assert true
66
67
  end
67
68
 
69
+ # Keep this test in sync with the one above, just with the config disabled
70
+ def test_drain_ignores_slow_queue_if_disabled
71
+ propono_config.slow_queue_enabled = false
72
+
73
+ @listener.expects(:read_messages_from_queue).with(@slow_queue, 10, long_poll: false).never
74
+ @listener.expects(:read_messages_from_queue).with(@queue, 10, long_poll: false).returns(false)
75
+ @listener.drain
76
+ assert true
77
+ end
78
+
68
79
  def test_drain_raises_with_nil_topic
69
80
  listener = QueueListener.new(aws_client, propono_config, nil) {}
70
81
  assert_raises ProponoError do
@@ -81,7 +92,7 @@ module Propono
81
92
 
82
93
  def test_read_message_from_sqs
83
94
  max_number_of_messages = 5
84
- aws_client.expects(:read_from_sqs).with(@queue, max_number_of_messages, long_poll: true)
95
+ aws_client.expects(:read_from_sqs).with(@queue, max_number_of_messages, long_poll: true, visibility_timeout: nil)
85
96
  @listener.send(:read_messages_from_queue, @queue, max_number_of_messages)
86
97
  end
87
98
 
@@ -217,6 +228,7 @@ module Propono
217
228
  @listener.send(:move_to_corrupt_queue, @sqs_message1)
218
229
  end
219
230
 
231
+ # Keep this test in sync with the one below, just with the config enabled
220
232
  def test_if_no_messages_read_from_normal_queue_read_from_slow_queue
221
233
  main_queue = mock
222
234
  @listener.stubs(main_queue: main_queue)
@@ -228,6 +240,20 @@ module Propono
228
240
  @listener.send(:read_messages)
229
241
  end
230
242
 
243
+ # Keep this test in sync with the one above, just with the config disabled
244
+ def ignore_slow_queue_if_disabled
245
+ propono_config.slow_queue_enabled = false
246
+
247
+ main_queue = mock
248
+ @listener.stubs(main_queue: main_queue)
249
+ slow_queue = mock
250
+ @listener.stubs(slow_queue: slow_queue)
251
+
252
+ @listener.expects(:read_messages_from_queue).with(main_queue, propono_config.num_messages_per_poll).returns(false)
253
+ @listener.expects(:read_messages_from_queue).with(slow_queue, 1).never
254
+ @listener.send(:read_messages)
255
+ end
256
+
231
257
  def test_if_read_messages_from_normal_do_not_read_from_slow_queue
232
258
  main_queue = mock
233
259
  @listener.stubs(main_queue: main_queue)
data/test/test_helper.rb CHANGED
@@ -15,9 +15,6 @@ class Minitest::Test
15
15
 
16
16
  def propono_config
17
17
  @propono_config ||= Propono::Configuration.new.tap do |c|
18
- c.access_key = "test-access-key"
19
- c.secret_key = "test-secret-key"
20
- c.queue_region = "us-east-1"
21
18
  c.application_name = "MyApp"
22
19
  c.queue_suffix = ""
23
20
 
metadata CHANGED
@@ -1,15 +1,17 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: propono
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0.rc3
4
+ version: 3.0.0
5
5
  platform: ruby
6
6
  authors:
7
- - MalcyL
8
7
  - iHiD
9
- autorequire:
8
+ - dougal
9
+ - ccare
10
+ - MalcyL
11
+ autorequire:
10
12
  bindir: bin
11
13
  cert_chain: []
12
- date: 2017-10-11 00:00:00.000000000 Z
14
+ date: 2021-02-10 00:00:00.000000000 Z
13
15
  dependencies:
14
16
  - !ruby/object:Gem::Dependency
15
17
  name: aws-sdk-sns
@@ -45,14 +47,14 @@ dependencies:
45
47
  requirements:
46
48
  - - "~>"
47
49
  - !ruby/object:Gem::Version
48
- version: '1.3'
50
+ version: '2.1'
49
51
  type: :development
50
52
  prerelease: false
51
53
  version_requirements: !ruby/object:Gem::Requirement
52
54
  requirements:
53
55
  - - "~>"
54
56
  - !ruby/object:Gem::Version
55
- version: '1.3'
57
+ version: '2.1'
56
58
  - !ruby/object:Gem::Dependency
57
59
  name: rake
58
60
  requirement: !ruby/object:Gem::Requirement
@@ -112,20 +114,20 @@ dependencies:
112
114
  description: Pub / Sub Library using Amazon Web Services
113
115
  email:
114
116
  - jez.walker@gmail.com
115
- - c.p.care@gmail.com
116
- - malcolm@landonsonline.me.uk
117
117
  executables: []
118
118
  extensions: []
119
119
  extra_rdoc_files: []
120
120
  files:
121
+ - ".github/workflows/tests.yml"
121
122
  - ".gitignore"
122
- - ".travis.yml"
123
+ - ".ruby-version"
123
124
  - CHANGELOG.md
124
125
  - CONTRIBUTING.md
125
126
  - Gemfile
126
127
  - LICENCE.md
127
128
  - README.md
128
129
  - Rakefile
130
+ - docs/upgrading.md
129
131
  - lib/propono.rb
130
132
  - lib/propono/components/aws_client.rb
131
133
  - lib/propono/components/aws_config.rb
@@ -158,11 +160,11 @@ files:
158
160
  - test/services/queue_listener_test.rb
159
161
  - test/test_helper.rb
160
162
  - test/utils/hash_test.rb
161
- homepage: ''
163
+ homepage: https://github.com/iHiD/propono/
162
164
  licenses:
163
165
  - MIT
164
166
  metadata: {}
165
- post_install_message:
167
+ post_install_message:
166
168
  rdoc_options: []
167
169
  require_paths:
168
170
  - lib
@@ -173,13 +175,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
173
175
  version: '0'
174
176
  required_rubygems_version: !ruby/object:Gem::Requirement
175
177
  requirements:
176
- - - ">"
178
+ - - ">="
177
179
  - !ruby/object:Gem::Version
178
- version: 1.3.1
180
+ version: '0'
179
181
  requirements: []
180
- rubyforge_project:
181
- rubygems_version: 2.6.13
182
- signing_key:
182
+ rubygems_version: 3.0.3
183
+ signing_key:
183
184
  specification_version: 4
184
185
  summary: General purpose pub/sub library built on top of AWS SNS and SQS
185
186
  test_files:
@@ -199,4 +200,3 @@ test_files:
199
200
  - test/services/queue_listener_test.rb
200
201
  - test/test_helper.rb
201
202
  - test/utils/hash_test.rb
202
- has_rdoc:
data/.travis.yml DELETED
@@ -1,7 +0,0 @@
1
- language: ruby
2
- cache: bundler
3
- rvm:
4
- - 2.2.8
5
- - 2.4.2
6
- script:
7
- - bundle exec rake test:local