streamforce 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c34310d1629ebc82065f442b4e4ef30eb91fef7f28aa42b01afbc062b6fd3a1e
4
- data.tar.gz: 53ae2b66be382330501458097c322c88a70826400af18808ae5f1b062966b6dd
3
+ metadata.gz: 5d03ff265d884babb70b8db70637c9814a5819834ed83109b2b26537cb5c992b
4
+ data.tar.gz: b7b87b62347f92e9a27139536655ee95339e8a3b22bf2c1980db1eca00fa66e9
5
5
  SHA512:
6
- metadata.gz: 4fb0fd547750fb0c26cc027407227bec6091e5a640431bbd26b31eac34866914aaa8fe9b2f0a99551b598aea6ae0f11de0f2ff54e85986fc2cd96e20c3e7a1af
7
- data.tar.gz: 55b68bf69239eba0af1ceabfd225184c2ef8e838154273086614acbad4c685645f37bbb8ed6a60da04033f8cc965191a053a719a864815746ad06bfb868b30a3
6
+ metadata.gz: c4dc737d70d28c061ee3c89379b97274983c14008184acbe7c99139b59c3ed79b3ef79893ba3d486d3f142172d30461260237d171bdb7a230c6f92471769a605
7
+ data.tar.gz: 6a3bbd281947060861c34e5cceab6814cfad52a80e964c80ef4da4956a3c40287e1315ef05419c69b31e60ac2bd20b394336e85a087c8fd41e22691543f6f7d7
data/.rubocop.yml CHANGED
@@ -1,8 +1,3 @@
1
- AllCops:
2
- TargetRubyVersion: 3.0
3
-
4
- Style/StringLiterals:
5
- EnforcedStyle: double_quotes
6
-
7
- Style/StringLiteralsInInterpolation:
8
- EnforcedStyle: double_quotes
1
+ # Omakase Ruby styling for Rails
2
+ inherit_gem:
3
+ rubocop-rails-omakase: rubocop.yml
data/README.md CHANGED
@@ -1,8 +1,76 @@
1
1
  # Streamforce
2
2
 
3
+ In most cases, processing events received from the Salesforce Streaming API can be
4
+ broken down into three very specific steps:
5
+
6
+ 1. Connecting to the Salesforce API and listening for messaging
7
+ 2. Ingesting received messages into some sort of internal event bus (e.g. RabbitMQ,
8
+ Kafka, Redis, etc)
9
+ 3. Processing the stored messages
10
+
11
+ Streamforce aims to handle #1 and simplify the work done for #2.
12
+
13
+ [Restforce](https://github.com/restforce/restforce) provides a simple API to connect
14
+ to the Streaming API and consume messages:
15
+
16
+ ```ruby
17
+ # Restforce uses faye as the underlying implementation for CometD.
18
+ require 'faye'
19
+
20
+ # Initialize a client with your username/password/oauth token/etc.
21
+ client = Restforce.new(username: 'foo',
22
+ password: 'bar',
23
+ security_token: 'security token',
24
+ client_id: 'client_id',
25
+ client_secret: 'client_secret')
26
+
27
+ EM.run do
28
+ # Subscribe to the PushTopic.
29
+ client.subscription '/topic/AllAccounts' do |message|
30
+ puts message.inspect
31
+ end
32
+ end
33
+ ```
34
+
35
+ However, the above code is usable in a production environment because:
36
+
37
+ * The interactions with the Streaming API need to be logged using the correct severity
38
+ (e.g. handshakes should use `Logger::DEBUG` while subscription errors should use
39
+ `Logger::ERROR` for better visibility)
40
+ * Replay IDs need to be stored using a persistent storage like Redis and not in-memory
41
+
42
+ Streamforce comes with all the batteries included.
43
+
44
+ ## Usage
45
+
46
+ A very simple client, which automatically connects based on the following environment
47
+ variables:
48
+
49
+ * `SALESFORCE_USERNAME`
50
+ * `SALESFORCE_PASSWORD`
51
+ * `SALESFORCE_SECURITY_TOKEN`
52
+ * `SALESFORCE_CLIENT_ID`
53
+ * `SALESFORCE_CLIENT_SECRET`
54
+ * `REDIS_URL`
55
+
56
+ ```ruby
57
+ require "streamforce"
58
+
59
+ client = Streamforce::Client.new
60
+
61
+ subscriptions = %w[
62
+ /topic/account-monitor
63
+ /event/AccountUpdated__e
64
+ ]
65
+
66
+ client.subscribe(subscriptions) do |subscription, message|
67
+ # Your code
68
+ end
69
+ ```
70
+
3
71
  ## Contributing
4
72
 
5
- Bug reports and pull requests are welcome on GitHub at https://github.com/andreimaxim/streamforce.
73
+ Bug reports and pull requests are welcome on GitHub at <https://github.com/andreimaxim/streamforce>.
6
74
 
7
75
  ## License
8
76
 
@@ -1,79 +1,79 @@
1
1
  class Streamforce::Client
2
- class << self
3
- def host
4
- ENV["SALESFORCE_HOST"]
5
- end
6
-
7
- def client_id
8
- ENV["SALESFORCE_CLIENT_ID"]
9
- end
10
-
11
- def client_secret
12
- ENV["SALESFORCE_CLIENT_SECRET"]
13
- end
14
-
15
- def username
16
- ENV["SALESFORCE_USERNAME"]
17
- end
18
-
19
- def password
20
- ENV["SALESFORCE_PASSWORD"]
21
- end
22
-
23
- def security_token
24
- ENV["SALESFORCE_SECURITY_TOKEN"]
25
- end
2
+ attr_reader :host, :username, :password, :client_id, :client_secret, :security_token,
3
+ :api_version
4
+
5
+ def initialize(opts = {})
6
+ @host = opts.fetch(:host, ENV["SALESFORCE_HOST"])
7
+ @username = opts.fetch(:username, ENV["SALESFORCE_USERNAME"])
8
+ @password = opts.fetch(:password, ENV["SALESFORCE_PASSWORD"])
9
+ @client_id = opts.fetch(:client_id, ENV["SALESFORCE_CLIENT_ID"])
10
+ @client_secret = opts.fetch(:client_secret, ENV["SALESFORCE_CLIENT_SECRET"])
11
+ @security_token = opts.fetch(:security_token, ENV["SALESFORCE_SECURITY_TOKEN"])
12
+ @api_version = opts.fetch(:api_version, ENV["SALESFORCE_API_VERSION"])
13
+
14
+ @logger = opts.fetch(:logger, Logger.new($stdout))
15
+ @logger.level = ENV.fetch("STREAMFORCE_LOG_LEVEL", Logger::INFO)
16
+ end
26
17
 
27
- def version
28
- ENV.fetch("SALESFORCE_API_VERSION", "61.0")
29
- end
18
+ def subscribe(channels = [], &blk)
19
+ EM.run { subscribe_to_channels(faye, Array(channels), &blk) }
20
+ end
30
21
 
31
- def authentication_url
32
- URI.parse("https://#{host}/services/oauth2/token")
33
- end
22
+ private
34
23
 
35
- def authentication_params
36
- {
37
- grant_type: "password",
38
- client_id: client_id,
39
- client_secret: client_secret,
40
- username: username,
41
- password: "#{password}#{security_token}"
42
- }
43
- end
24
+ def instance_url
25
+ authentication["instance_url"]
44
26
  end
45
27
 
46
- def set_authentication_credentials!
47
- response = Net::HTTP.post_form(Streamforce::Client.authentication_url, Streamforce::Client.authentication_params)
48
- credentials = JSON.parse(response.body)
28
+ def access_token
29
+ authentication["access_token"]
30
+ end
49
31
 
50
- @access_token = credentials["access_token"]
51
- @instance_url = "#{credentials["instance_url"]}/cometd/#{Streamforce::Client.version}"
32
+ def authentication_url
33
+ URI.parse("https://#{host}/services/oauth2/token")
52
34
  end
53
35
 
54
- def subscribe(channels = [], &blk)
55
- channels = Array(channels)
36
+ def authentication_params
37
+ {
38
+ grant_type: "password",
39
+ username: username,
40
+ password: "#{password}#{security_token}",
41
+ client_id: client_id,
42
+ client_secret: client_secret
43
+ }
44
+ end
56
45
 
57
- set_authentication_credentials!
46
+ def authentication
47
+ @authentication ||= fetch_authentication_credentials
48
+ end
58
49
 
59
- EM.run { subscribe_to_channels(faye, channels, &blk) }
50
+ def fetch_authentication_credentials
51
+ response = Net::HTTP.post_form(authentication_url, authentication_params)
52
+ JSON.parse(response.body)
60
53
  end
61
54
 
62
55
  def faye
63
- @faye ||= Faye::Client.new(@instance_url).tap do |client|
64
- client.set_header "Authorization", "OAuth #{@access_token}"
56
+ @faye ||= Faye::Client.new("#{instance_url}/cometd/#{api_version}").tap do |client|
57
+ client.set_header "Authorization", "OAuth #{access_token}"
65
58
 
66
59
  client.add_extension Streamforce::Extension::Replay.new
67
- client.add_extension Streamforce::Extension::SubscriptionTracking.new
68
- client.add_extension Streamforce::Extension::Logging.new
60
+ client.add_extension Streamforce::Extension::Logging.new(@logger)
69
61
  end
70
62
  end
71
63
 
72
64
  def subscribe_to_channels(client, channels, &blk)
73
65
  return if channels.empty?
74
66
 
75
- client
76
- .subscribe(channels.shift, &blk)
77
- .callback { subscribe_to_channels(client, channels, &blk)}
67
+ # Subscribe to a single channel, otherwise Salesforce will return a 403 Unknown Client error
68
+ subscription = client.subscribe(channels.shift)
69
+
70
+ # Allow clients to receive [ channel, message ] block params
71
+ subscription.with_channel(&blk)
72
+
73
+ # Continue subscribing to the rest of the channels, regadless of the current subscription
74
+ # status
75
+ subscription
76
+ .callback { subscribe_to_channels(client, channels, &blk) }
77
+ .errback { subscribe_to_channels(client, channels, &blk) }
78
78
  end
79
79
  end
@@ -1,18 +1,97 @@
1
1
  class Streamforce::Extension::Logging
2
- def initialize(log_level = Logger::DEBUG)
3
- @logger = Logger.new($stdout)
4
- @logger.level = log_level
2
+ def initialize(logger)
3
+ @logger = logger
5
4
  end
6
5
 
7
- def incoming(message, callback)
8
- @logger.debug "Receving message: #{message.inspect}"
6
+ def incoming(payload, callback)
7
+ message = Streamforce::Message.new(payload)
8
+ log_incoming_message(message)
9
9
 
10
- callback.call(message)
10
+ callback.call(payload)
11
11
  end
12
12
 
13
- def outgoing(message, callback)
14
- @logger.debug "Sending message: #{message.inspect}"
13
+ def outgoing(payload, callback)
14
+ message = Streamforce::Message.new(payload)
15
+ log_outgoing_message(message)
15
16
 
16
- callback.call(message)
17
+ callback.call(payload)
18
+ end
19
+
20
+ def log_incoming_message(message)
21
+ if message.channel_type == "meta"
22
+ public_send("log_incoming_#{message.channel_name}", message)
23
+ else
24
+ @logger.debug "[#{message.channel_name}] #{message.data}"
25
+ end
26
+ end
27
+
28
+ def log_outgoing_message(message)
29
+ if message.channel_type == "meta"
30
+ public_send("log_outgoing_#{message.channel_name}", message)
31
+ else
32
+ @logger.debug "[#{message.channel_name}] #{message.data}"
33
+ end
34
+ end
35
+
36
+ def log_outgoing_handshake(message)
37
+ @logger.debug "[Client] Handshake requested..."
38
+ end
39
+
40
+ def log_incoming_handshake(message)
41
+ if message.success?
42
+ @logger.debug "[Server] Handshake accepted, assigning client_id=#{message.client_id}"
43
+ else
44
+ @logger.error "[Server] Connection was refused: #{message.error_message}"
45
+ end
46
+ end
47
+
48
+ def log_outgoing_connect(message)
49
+ debug message, "Sending connection request"
50
+ end
51
+
52
+ def log_incoming_connect(message)
53
+ if message.success?
54
+ debug message, "Connection successful!"
55
+ else
56
+ error message, "Connection failed: #{message.error_message}"
57
+ end
58
+ end
59
+
60
+ def log_outgoing_subscribe(message)
61
+ replay_id = message.replay_id
62
+
63
+ replay_info = if replay_id == -1
64
+ "for all new messages"
65
+ elsif replay_id == -2
66
+ "and requesting all stored messages"
67
+ else
68
+ "and requesting all messages newer than ##{replay_id}"
69
+ end
70
+
71
+ info message, "Subscribing to #{message.subscription} #{replay_info}"
72
+ end
73
+
74
+ def log_incoming_subscribe(message)
75
+ if message.success?
76
+ info message, "Successfully subscribed to #{message.subscription}"
77
+ else
78
+ error message, "Subscription for #{message.subscription} failed: #{message.error_message}"
79
+ end
80
+ end
81
+
82
+ def debug(message, text)
83
+ @logger.debug "[#{message.client_id}##{message.id}] #{text}"
84
+ end
85
+
86
+ def info(message, text)
87
+ @logger.info "[#{message.client_id}##{message.id}] #{text}"
88
+ end
89
+
90
+ def warn(message, text)
91
+ @logger.warn "[#{message.client_id}##{message.id}] #{text}"
92
+ end
93
+
94
+ def error(message, text)
95
+ @logger.error "[#{message.client_id}##{message.id}] #{text}"
17
96
  end
18
97
  end
@@ -1,14 +1,36 @@
1
+ require "redis"
2
+
1
3
  class Streamforce::Extension::Replay
2
4
  def initialize(log_level = Logger::INFO)
3
5
  @logger = Logger.new($stdout)
4
6
  @logger.level = log_level
7
+ @redis = Redis.new
5
8
  end
6
9
 
7
10
  def incoming(message, callback)
11
+ replay_id = message.dig "data", "event", "replayId"
12
+ channel = message["channel"]
13
+
14
+ store(channel, replay_id)
8
15
  callback.call(message)
9
16
  end
10
17
 
11
18
  def outgoing(message, callback)
19
+ return callback.call(message) unless message["channel"] == "/meta/subscribe"
20
+
21
+ channel = message["subscription"]
22
+ message["ext"] = { "replay" => { channel => replay_id(channel).to_i } }
23
+
12
24
  callback.call(message)
13
25
  end
26
+
27
+ def store(channel, replay_id)
28
+ return if channel.nil? || replay_id.nil?
29
+
30
+ @redis.set channel, replay_id, ex: 86400
31
+ end
32
+
33
+ def replay_id(channel)
34
+ @redis.get(channel) || -1
35
+ end
14
36
  end
@@ -0,0 +1,49 @@
1
+ class Streamforce::Message
2
+ def initialize(payload)
3
+ @payload = payload
4
+ end
5
+
6
+ def success?
7
+ @payload["successful"]
8
+ end
9
+
10
+ def id
11
+ @payload["id"]
12
+ end
13
+
14
+ def replay_id
15
+ @payload.dig "ext", "replay", subscription
16
+ end
17
+
18
+ def channel
19
+ @payload["channel"]
20
+ end
21
+
22
+ def client_id
23
+ @payload["clientId"]
24
+ end
25
+
26
+ def channel_type
27
+ channel.split("/")[1]
28
+ end
29
+
30
+ def channel_name
31
+ channel.split("/")[2]
32
+ end
33
+
34
+ def data
35
+ @payload["data"]
36
+ end
37
+
38
+ def subscription
39
+ @payload["subscription"]
40
+ end
41
+
42
+ def subscription?
43
+ channel == "/meta/subscribe"
44
+ end
45
+
46
+ def error_message
47
+ @payload["error"]
48
+ end
49
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Streamforce
4
- VERSION = "0.0.1"
4
+ VERSION = "0.1.0"
5
5
  end
data/lib/streamforce.rb CHANGED
@@ -6,7 +6,6 @@ require "uri"
6
6
  require "net/http"
7
7
  require "logger"
8
8
  require "json"
9
- require "base64"
10
9
 
11
10
  require "faye"
12
11
  require "relaxed_cookiejar"
metadata CHANGED
@@ -1,17 +1,17 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: streamforce
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Andrei Maxim
8
- autorequire:
8
+ autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2024-06-12 00:00:00.000000000 Z
11
+ date: 2024-06-17 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
- name: base64
14
+ name: zeitwerk
15
15
  requirement: !ruby/object:Gem::Requirement
16
16
  requirements:
17
17
  - - ">="
@@ -25,7 +25,7 @@ dependencies:
25
25
  - !ruby/object:Gem::Version
26
26
  version: '0'
27
27
  - !ruby/object:Gem::Dependency
28
- name: zeitwerk
28
+ name: faye
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
31
  - - ">="
@@ -39,7 +39,7 @@ dependencies:
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
41
  - !ruby/object:Gem::Dependency
42
- name: faye
42
+ name: relaxed_cookiejar
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
45
  - - ">="
@@ -53,7 +53,7 @@ dependencies:
53
53
  - !ruby/object:Gem::Version
54
54
  version: '0'
55
55
  - !ruby/object:Gem::Dependency
56
- name: relaxed_cookiejar
56
+ name: redis
57
57
  requirement: !ruby/object:Gem::Requirement
58
58
  requirements:
59
59
  - - ">="
@@ -66,7 +66,49 @@ dependencies:
66
66
  - - ">="
67
67
  - !ruby/object:Gem::Version
68
68
  version: '0'
69
- description:
69
+ - !ruby/object:Gem::Dependency
70
+ name: rake
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '13.0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '13.0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: minitest
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: rubocop-rails-omakase
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ description:
70
112
  email:
71
113
  - andrei@andreimaxim.ro
72
114
  executables: []
@@ -82,7 +124,7 @@ files:
82
124
  - lib/streamforce/client.rb
83
125
  - lib/streamforce/extension/logging.rb
84
126
  - lib/streamforce/extension/replay.rb
85
- - lib/streamforce/extension/subscription_tracking.rb
127
+ - lib/streamforce/message.rb
86
128
  - lib/streamforce/version.rb
87
129
  homepage: https://github.com/andreimaxim/streamforce
88
130
  licenses:
@@ -91,7 +133,7 @@ metadata:
91
133
  homepage_uri: https://github.com/andreimaxim/streamforce
92
134
  source_code_uri: https://github.com/andreimaxim/streamforce
93
135
  changelog_uri: https://github.com/andreimaxim/streamforce/blob/main/CHANGELOG.md
94
- post_install_message:
136
+ post_install_message:
95
137
  rdoc_options: []
96
138
  require_paths:
97
139
  - lib
@@ -99,7 +141,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
99
141
  requirements:
100
142
  - - ">="
101
143
  - !ruby/object:Gem::Version
102
- version: 3.3.0
144
+ version: 3.0.0
103
145
  required_rubygems_version: !ruby/object:Gem::Requirement
104
146
  requirements:
105
147
  - - ">="
@@ -107,7 +149,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
107
149
  version: '0'
108
150
  requirements: []
109
151
  rubygems_version: 3.5.9
110
- signing_key:
152
+ signing_key:
111
153
  specification_version: 4
112
154
  summary: Small wrapper over the Salesforce Streaming API
113
155
  test_files: []
@@ -1,58 +0,0 @@
1
- class Streamforce::Extension::SubscriptionTracking
2
- def initialize(log_level = Logger::DEBUG)
3
- @logger = Logger.new($stdout)
4
- @logger.level = log_level
5
-
6
- @subscriptions = []
7
- end
8
-
9
- def incoming(message, callback)
10
- if subscription?(message)
11
- log_subscription_status(message)
12
- else
13
- log_subscription_payload(message)
14
- end
15
-
16
- callback.call(message)
17
- end
18
-
19
- def outgoing(message, callback)
20
- @logger.debug "Requested subscription for #{message["subscription"]}" if subscription?(message)
21
-
22
- callback.call(message)
23
- end
24
-
25
- def log_subscription_status(message)
26
- subscription = message["subscription"]
27
-
28
- if message["successful"]
29
- @subscriptions << subscription
30
-
31
- @logger.info "Subscription for #{subscription} was successful"
32
- else
33
- @logger.warn "Subscription for #{subscription} failed: #{message["error"]}"
34
- end
35
- end
36
-
37
- def log_subscription_payload(message)
38
- channel = message["channel"]
39
- payload = message["data"].to_json
40
-
41
- type = case channel.split("/")[1]
42
- when "topic"
43
- "PushTopic"
44
- when "event"
45
- "PlatformEvent"
46
- else
47
- "Unknown"
48
- end
49
-
50
- name = channel.split("/")[2]
51
-
52
- @logger.info "[#{type}][#{name}]: #{payload}" if @subscriptions.include?(channel)
53
- end
54
-
55
- def subscription?(message)
56
- message["channel"] == "/meta/subscribe"
57
- end
58
- end