heathrow 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +58 -0
- data/README.md +205 -0
- data/bin/heathrow +42 -0
- data/bin/heathrowd +283 -0
- data/docs/ARCHITECTURE.md +1172 -0
- data/docs/DATABASE_SCHEMA.md +685 -0
- data/docs/DEVELOPMENT_WORKFLOW.md +867 -0
- data/docs/DISCORD_SETUP.md +142 -0
- data/docs/GMAIL_OAUTH_SETUP.md +120 -0
- data/docs/PLUGIN_SYSTEM.md +1370 -0
- data/docs/PROJECT_PLAN.md +1022 -0
- data/docs/README.md +417 -0
- data/docs/REDDIT_SETUP.md +174 -0
- data/docs/REPLY_FORWARD.md +182 -0
- data/docs/WHATSAPP_TELEGRAM_SETUP.md +306 -0
- data/heathrow.gemspec +34 -0
- data/heathrowd.service +21 -0
- data/img/heathrow.svg +95 -0
- data/img/rss_threaded.png +0 -0
- data/img/sources.png +0 -0
- data/lib/heathrow/address_book.rb +42 -0
- data/lib/heathrow/config.rb +332 -0
- data/lib/heathrow/database.rb +731 -0
- data/lib/heathrow/database_new.rb +392 -0
- data/lib/heathrow/event_bus.rb +175 -0
- data/lib/heathrow/logger.rb +122 -0
- data/lib/heathrow/message.rb +176 -0
- data/lib/heathrow/message_composer.rb +399 -0
- data/lib/heathrow/message_organizer.rb +774 -0
- data/lib/heathrow/migrations/001_initial_schema.rb +248 -0
- data/lib/heathrow/notmuch.rb +45 -0
- data/lib/heathrow/oauth2_smtp.rb +254 -0
- data/lib/heathrow/plugin/base.rb +212 -0
- data/lib/heathrow/plugin_manager.rb +141 -0
- data/lib/heathrow/poller.rb +93 -0
- data/lib/heathrow/smtp_sender.rb +204 -0
- data/lib/heathrow/source.rb +39 -0
- data/lib/heathrow/sources/base.rb +74 -0
- data/lib/heathrow/sources/discord.rb +357 -0
- data/lib/heathrow/sources/gmail.rb +294 -0
- data/lib/heathrow/sources/imap.rb +198 -0
- data/lib/heathrow/sources/instagram.rb +307 -0
- data/lib/heathrow/sources/instagram_fetch.py +101 -0
- data/lib/heathrow/sources/instagram_send.py +55 -0
- data/lib/heathrow/sources/instagram_send_marionette.py +104 -0
- data/lib/heathrow/sources/maildir.rb +606 -0
- data/lib/heathrow/sources/messenger.rb +212 -0
- data/lib/heathrow/sources/messenger_fetch.js +297 -0
- data/lib/heathrow/sources/messenger_fetch_marionette.py +138 -0
- data/lib/heathrow/sources/messenger_send.js +32 -0
- data/lib/heathrow/sources/messenger_send.py +100 -0
- data/lib/heathrow/sources/reddit.rb +461 -0
- data/lib/heathrow/sources/rss.rb +299 -0
- data/lib/heathrow/sources/slack.rb +375 -0
- data/lib/heathrow/sources/source_manager.rb +328 -0
- data/lib/heathrow/sources/telegram.rb +498 -0
- data/lib/heathrow/sources/webpage.rb +207 -0
- data/lib/heathrow/sources/weechat.rb +479 -0
- data/lib/heathrow/sources/whatsapp.rb +474 -0
- data/lib/heathrow/ui/application.rb +8098 -0
- data/lib/heathrow/ui/navigation.rb +8 -0
- data/lib/heathrow/ui/panes.rb +8 -0
- data/lib/heathrow/ui/source_wizard.rb +567 -0
- data/lib/heathrow/ui/threaded_view.rb +780 -0
- data/lib/heathrow/ui/views.rb +8 -0
- data/lib/heathrow/version.rb +3 -0
- data/lib/heathrow/wizards/discord_wizard.rb +193 -0
- data/lib/heathrow/wizards/slack_wizard.rb +140 -0
- data/lib/heathrow.rb +55 -0
- metadata +147 -0
|
@@ -0,0 +1,1370 @@
|
|
|
1
|
+
# Heathrow Plugin System
|
|
2
|
+
|
|
3
|
+
**Philosophy:** Every communication platform is a self-contained plugin with zero coupling to other plugins.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Table of Contents
|
|
8
|
+
|
|
9
|
+
1. [Architecture](#architecture)
|
|
10
|
+
2. [Plugin Lifecycle](#plugin-lifecycle)
|
|
11
|
+
3. [Plugin Interface](#plugin-interface)
|
|
12
|
+
4. [Plugin Discovery](#plugin-discovery)
|
|
13
|
+
5. [Error Handling](#error-handling)
|
|
14
|
+
6. [Testing Plugins](#testing-plugins)
|
|
15
|
+
7. [Example Plugins](#example-plugins)
|
|
16
|
+
8. [Plugin Development Guide](#plugin-development-guide)
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Architecture
|
|
21
|
+
|
|
22
|
+
### Core Principles
|
|
23
|
+
|
|
24
|
+
1. **Zero Coupling** - Plugins cannot interact with each other directly
|
|
25
|
+
2. **Fail Independently** - Plugin crash must not crash core or other plugins
|
|
26
|
+
3. **Hot Reload** - Plugins can be loaded/unloaded without restart
|
|
27
|
+
4. **Version Compatibility** - Old plugins work with new core (within major version)
|
|
28
|
+
5. **Sandboxed** - Plugins have limited access to system resources
|
|
29
|
+
|
|
30
|
+
### Plugin Isolation
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
┌─────────────────────────────────────────────┐
|
|
34
|
+
│ Core Application │
|
|
35
|
+
│ ┌────────────────────────────────────┐ │
|
|
36
|
+
│ │ Plugin Manager │ │
|
|
37
|
+
│ │ ┌──────────────────────────────┐ │ │
|
|
38
|
+
│ │ │ Error Boundary (rescue) │ │ │
|
|
39
|
+
│ │ │ ┌────────────────────────┐ │ │ │
|
|
40
|
+
│ │ │ │ Plugin Instance │ │ │ │
|
|
41
|
+
│ │ │ │ - fetch_messages │ │ │ │
|
|
42
|
+
│ │ │ │ - send_message │ │ │ │
|
|
43
|
+
│ │ │ └────────────────────────┘ │ │ │
|
|
44
|
+
│ │ └──────────────────────────────┘ │ │
|
|
45
|
+
│ └────────────────────────────────────┘ │
|
|
46
|
+
└─────────────────────────────────────────────┘
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
**If plugin crashes:**
|
|
50
|
+
1. Error caught by boundary
|
|
51
|
+
2. Error logged
|
|
52
|
+
3. Plugin marked as failed
|
|
53
|
+
4. Other plugins continue working
|
|
54
|
+
5. UI shows error notification
|
|
55
|
+
6. User can retry or disable plugin
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## Plugin Lifecycle
|
|
60
|
+
|
|
61
|
+
### States
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
┌──────────┐
|
|
65
|
+
│ Unloaded │
|
|
66
|
+
└────┬─────┘
|
|
67
|
+
│ load_plugin(name)
|
|
68
|
+
▼
|
|
69
|
+
┌──────────┐ start() fails ┌────────┐
|
|
70
|
+
│ Loaded │────────────────>│ Failed │
|
|
71
|
+
└────┬─────┘ └────┬───┘
|
|
72
|
+
│ start() succeeds │
|
|
73
|
+
▼ │ retry
|
|
74
|
+
┌──────────┐ │
|
|
75
|
+
│ Running │<─────────────────────┘
|
|
76
|
+
└────┬─────┘
|
|
77
|
+
│ stop() or error
|
|
78
|
+
▼
|
|
79
|
+
┌──────────┐
|
|
80
|
+
│ Stopped │
|
|
81
|
+
└────┬─────┘
|
|
82
|
+
│ unload_plugin(name)
|
|
83
|
+
▼
|
|
84
|
+
┌──────────┐
|
|
85
|
+
│ Unloaded │
|
|
86
|
+
└──────────┘
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
### Lifecycle Methods
|
|
90
|
+
|
|
91
|
+
```ruby
|
|
92
|
+
class MyPlugin < Heathrow::Plugin::Base
|
|
93
|
+
# Called once when plugin is loaded
|
|
94
|
+
def initialize(config, event_bus, db)
|
|
95
|
+
super
|
|
96
|
+
@connection = nil
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# Called when plugin is started (connects to service)
|
|
100
|
+
def start
|
|
101
|
+
@connection = connect_to_service
|
|
102
|
+
log(:info, "Connected successfully")
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
# Called when plugin is stopped (cleanup)
|
|
106
|
+
def stop
|
|
107
|
+
@connection&.close
|
|
108
|
+
log(:info, "Disconnected")
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
# Main work methods
|
|
112
|
+
def fetch_messages
|
|
113
|
+
# ...
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def send_message(message, target)
|
|
117
|
+
# ...
|
|
118
|
+
end
|
|
119
|
+
end
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## Plugin Interface
|
|
125
|
+
|
|
126
|
+
### Base Class: `Heathrow::Plugin::Base`
|
|
127
|
+
|
|
128
|
+
**File:** `lib/heathrow/plugin/base.rb`
|
|
129
|
+
|
|
130
|
+
```ruby
|
|
131
|
+
module Heathrow
|
|
132
|
+
module Plugin
|
|
133
|
+
class Base
|
|
134
|
+
attr_reader :name, :config, :capabilities, :state
|
|
135
|
+
|
|
136
|
+
# Initialize plugin (do not connect to service here)
|
|
137
|
+
# @param config [Hash] Plugin-specific configuration
|
|
138
|
+
# @param event_bus [Heathrow::EventBus] For emitting events
|
|
139
|
+
# @param db [Heathrow::Database] For storing data
|
|
140
|
+
def initialize(config, event_bus, db)
|
|
141
|
+
@config = config
|
|
142
|
+
@event_bus = event_bus
|
|
143
|
+
@db = db
|
|
144
|
+
@state = :stopped
|
|
145
|
+
@name = self.class.name.split('::').last.downcase
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
# Start the plugin (connect to service)
|
|
149
|
+
# @raise [PluginError] If connection fails
|
|
150
|
+
def start
|
|
151
|
+
@state = :running
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Stop the plugin (disconnect, cleanup)
|
|
155
|
+
def stop
|
|
156
|
+
@state = :stopped
|
|
157
|
+
end
|
|
158
|
+
|
|
159
|
+
# Fetch new messages from the service
|
|
160
|
+
# @param since [Integer, nil] Unix timestamp to fetch from
|
|
161
|
+
# @return [Array<Heathrow::Message>] Array of messages
|
|
162
|
+
def fetch_messages(since: nil)
|
|
163
|
+
raise NotImplementedError, "#{self.class} must implement #fetch_messages"
|
|
164
|
+
end
|
|
165
|
+
|
|
166
|
+
# Send a message through this service
|
|
167
|
+
# @param message [Heathrow::Message] Message to send
|
|
168
|
+
# @param target [String, Hash] Target identifier (email, channel, etc.)
|
|
169
|
+
# @return [Boolean] True if sent successfully
|
|
170
|
+
# @raise [PluginError] If send fails
|
|
171
|
+
def send_message(message, target)
|
|
172
|
+
raise NotImplementedError, "#{self.class} must implement #send_message"
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
# Get plugin capabilities
|
|
176
|
+
# @return [Array<Symbol>] Capability symbols
|
|
177
|
+
def capabilities
|
|
178
|
+
[]
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# Get current plugin status
|
|
182
|
+
# @return [Hash] Status information
|
|
183
|
+
def status
|
|
184
|
+
{
|
|
185
|
+
state: @state,
|
|
186
|
+
name: @name,
|
|
187
|
+
connected: @state == :running,
|
|
188
|
+
last_sync: @last_sync,
|
|
189
|
+
last_error: @last_error
|
|
190
|
+
}
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
# Get plugin configuration schema (for UI)
|
|
194
|
+
# @return [Hash] JSON Schema for config
|
|
195
|
+
def self.config_schema
|
|
196
|
+
{}
|
|
197
|
+
end
|
|
198
|
+
|
|
199
|
+
# Get plugin metadata
|
|
200
|
+
# @return [Hash] Plugin information
|
|
201
|
+
def self.metadata
|
|
202
|
+
{
|
|
203
|
+
name: name.split('::').last,
|
|
204
|
+
description: "A Heathrow plugin",
|
|
205
|
+
version: "1.0.0",
|
|
206
|
+
author: "Unknown",
|
|
207
|
+
two_way: false
|
|
208
|
+
}
|
|
209
|
+
end
|
|
210
|
+
|
|
211
|
+
protected
|
|
212
|
+
|
|
213
|
+
# Log a message
|
|
214
|
+
# @param level [Symbol] :debug, :info, :warn, :error
|
|
215
|
+
# @param message [String] Log message
|
|
216
|
+
def log(level, message)
|
|
217
|
+
@event_bus.publish(:log, {
|
|
218
|
+
level: level,
|
|
219
|
+
component: @name,
|
|
220
|
+
message: message,
|
|
221
|
+
timestamp: Time.now.to_i
|
|
222
|
+
})
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
# Emit an event
|
|
226
|
+
# @param type [Symbol] Event type
|
|
227
|
+
# @param data [Hash] Event data
|
|
228
|
+
def emit_event(type, data)
|
|
229
|
+
@event_bus.publish(type, data.merge(plugin: @name))
|
|
230
|
+
end
|
|
231
|
+
|
|
232
|
+
# Store encrypted credential
|
|
233
|
+
# @param key [String] Credential key
|
|
234
|
+
# @param value [String] Credential value
|
|
235
|
+
def store_credential(key, value)
|
|
236
|
+
# Encrypt and store in database
|
|
237
|
+
encrypted = Heathrow::Crypto.encrypt(value, encryption_key)
|
|
238
|
+
@db.exec(
|
|
239
|
+
"UPDATE sources SET config = json_set(config, '$.credentials.#{key}', ?) WHERE plugin_type = ?",
|
|
240
|
+
[encrypted, @name]
|
|
241
|
+
)
|
|
242
|
+
end
|
|
243
|
+
|
|
244
|
+
# Retrieve encrypted credential
|
|
245
|
+
# @param key [String] Credential key
|
|
246
|
+
# @return [String, nil] Decrypted credential
|
|
247
|
+
def retrieve_credential(key)
|
|
248
|
+
result = @db.query(
|
|
249
|
+
"SELECT json_extract(config, '$.credentials.#{key}') FROM sources WHERE plugin_type = ?",
|
|
250
|
+
[@name]
|
|
251
|
+
).first&.first
|
|
252
|
+
|
|
253
|
+
return nil unless result
|
|
254
|
+
Heathrow::Crypto.decrypt(result, encryption_key)
|
|
255
|
+
end
|
|
256
|
+
|
|
257
|
+
# Get encryption key from keychain
|
|
258
|
+
# @return [String] Encryption key
|
|
259
|
+
def encryption_key
|
|
260
|
+
Heathrow::Keychain.get_or_create("heathrow.encryption_key")
|
|
261
|
+
end
|
|
262
|
+
|
|
263
|
+
# Normalize external message to Heathrow::Message format
|
|
264
|
+
# @param external_msg [Object] Platform-specific message object
|
|
265
|
+
# @return [Heathrow::Message] Normalized message
|
|
266
|
+
def normalize_message(external_msg)
|
|
267
|
+
raise NotImplementedError, "#{self.class} must implement #normalize_message"
|
|
268
|
+
end
|
|
269
|
+
end
|
|
270
|
+
end
|
|
271
|
+
end
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
### Required Methods
|
|
275
|
+
|
|
276
|
+
Plugins **must** implement:
|
|
277
|
+
|
|
278
|
+
1. `fetch_messages(since: nil)` - Return array of `Heathrow::Message` objects
|
|
279
|
+
2. `capabilities` - Return array of capability symbols
|
|
280
|
+
|
|
281
|
+
Plugins **should** implement (if two-way):
|
|
282
|
+
|
|
283
|
+
3. `send_message(message, target)` - Send a message
|
|
284
|
+
|
|
285
|
+
### Optional Methods
|
|
286
|
+
|
|
287
|
+
Plugins **may** implement:
|
|
288
|
+
|
|
289
|
+
1. `start` - Custom connection logic
|
|
290
|
+
2. `stop` - Custom cleanup logic
|
|
291
|
+
3. `normalize_message(external)` - Convert platform format to Heathrow format
|
|
292
|
+
4. `search(query)` - Server-side search
|
|
293
|
+
5. `mark_read(message_id)` - Mark message as read on server
|
|
294
|
+
6. `delete_message(message_id)` - Delete message on server
|
|
295
|
+
7. `get_thread(thread_id)` - Fetch entire thread
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## Plugin Discovery
|
|
300
|
+
|
|
301
|
+
### Automatic Discovery
|
|
302
|
+
|
|
303
|
+
Plugins are automatically discovered from:
|
|
304
|
+
|
|
305
|
+
1. `lib/heathrow/plugins/*.rb` (built-in)
|
|
306
|
+
2. `~/.heathrow/plugins/*.rb` (user-installed)
|
|
307
|
+
3. Gem plugins: `heathrow-plugin-*` (community)
|
|
308
|
+
|
|
309
|
+
### Registration
|
|
310
|
+
|
|
311
|
+
```ruby
|
|
312
|
+
# lib/heathrow/plugin/registry.rb
|
|
313
|
+
module Heathrow
|
|
314
|
+
module Plugin
|
|
315
|
+
class Registry
|
|
316
|
+
@plugins = {}
|
|
317
|
+
|
|
318
|
+
# Auto-register on class definition
|
|
319
|
+
def self.register(plugin_class)
|
|
320
|
+
name = plugin_class.name.split('::').last.downcase
|
|
321
|
+
@plugins[name] = plugin_class
|
|
322
|
+
end
|
|
323
|
+
|
|
324
|
+
def self.get(name)
|
|
325
|
+
@plugins[name]
|
|
326
|
+
end
|
|
327
|
+
|
|
328
|
+
def self.all
|
|
329
|
+
@plugins.values
|
|
330
|
+
end
|
|
331
|
+
end
|
|
332
|
+
|
|
333
|
+
class Base
|
|
334
|
+
# Automatically register when subclassed
|
|
335
|
+
def self.inherited(subclass)
|
|
336
|
+
Registry.register(subclass)
|
|
337
|
+
end
|
|
338
|
+
end
|
|
339
|
+
end
|
|
340
|
+
end
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
### Plugin Loading
|
|
344
|
+
|
|
345
|
+
```ruby
|
|
346
|
+
# lib/heathrow/plugin_manager.rb
|
|
347
|
+
module Heathrow
|
|
348
|
+
class PluginManager
|
|
349
|
+
def initialize(event_bus, config, db)
|
|
350
|
+
@event_bus = event_bus
|
|
351
|
+
@config = config
|
|
352
|
+
@db = db
|
|
353
|
+
@plugins = {}
|
|
354
|
+
end
|
|
355
|
+
|
|
356
|
+
def load_all
|
|
357
|
+
# Load built-in plugins
|
|
358
|
+
Dir["lib/heathrow/plugins/*.rb"].each { |f| require f }
|
|
359
|
+
|
|
360
|
+
# Load user plugins
|
|
361
|
+
user_plugin_dir = File.expand_path("~/.heathrow/plugins")
|
|
362
|
+
Dir["#{user_plugin_dir}/*.rb"].each { |f| require f } if Dir.exist?(user_plugin_dir)
|
|
363
|
+
|
|
364
|
+
# Load gem plugins
|
|
365
|
+
Gem.find_files("heathrow/plugin/*.rb").each { |f| require f }
|
|
366
|
+
|
|
367
|
+
# Instantiate configured plugins
|
|
368
|
+
sources = @db.query("SELECT * FROM sources WHERE enabled = 1")
|
|
369
|
+
sources.each do |source|
|
|
370
|
+
load_plugin(source['plugin_type'], source['id'])
|
|
371
|
+
end
|
|
372
|
+
end
|
|
373
|
+
|
|
374
|
+
def load_plugin(plugin_type, source_id)
|
|
375
|
+
plugin_class = Plugin::Registry.get(plugin_type)
|
|
376
|
+
raise "Unknown plugin: #{plugin_type}" unless plugin_class
|
|
377
|
+
|
|
378
|
+
# Get source configuration
|
|
379
|
+
source = @db.query("SELECT * FROM sources WHERE id = ?", [source_id]).first
|
|
380
|
+
config = JSON.parse(source['config'])
|
|
381
|
+
|
|
382
|
+
# Instantiate plugin with error boundary
|
|
383
|
+
plugin = plugin_class.new(config, @event_bus, @db)
|
|
384
|
+
plugin.start
|
|
385
|
+
|
|
386
|
+
@plugins[source_id] = plugin
|
|
387
|
+
@event_bus.publish(:plugin_loaded, {plugin: plugin_type, source_id: source_id})
|
|
388
|
+
|
|
389
|
+
rescue => e
|
|
390
|
+
@event_bus.publish(:plugin_error, {
|
|
391
|
+
plugin: plugin_type,
|
|
392
|
+
source_id: source_id,
|
|
393
|
+
error: e.message,
|
|
394
|
+
backtrace: e.backtrace
|
|
395
|
+
})
|
|
396
|
+
nil
|
|
397
|
+
end
|
|
398
|
+
|
|
399
|
+
def get_plugin(source_id)
|
|
400
|
+
@plugins[source_id]
|
|
401
|
+
end
|
|
402
|
+
|
|
403
|
+
def unload_plugin(source_id)
|
|
404
|
+
plugin = @plugins.delete(source_id)
|
|
405
|
+
return unless plugin
|
|
406
|
+
|
|
407
|
+
plugin.stop rescue nil # Ignore errors during shutdown
|
|
408
|
+
@event_bus.publish(:plugin_unloaded, {source_id: source_id})
|
|
409
|
+
end
|
|
410
|
+
|
|
411
|
+
def reload_plugin(source_id)
|
|
412
|
+
unload_plugin(source_id)
|
|
413
|
+
|
|
414
|
+
# Get plugin type for this source
|
|
415
|
+
source = @db.query("SELECT plugin_type FROM sources WHERE id = ?", [source_id]).first
|
|
416
|
+
load_plugin(source['plugin_type'], source_id)
|
|
417
|
+
end
|
|
418
|
+
end
|
|
419
|
+
end
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
---
|
|
423
|
+
|
|
424
|
+
## Error Handling
|
|
425
|
+
|
|
426
|
+
### Error Boundary Pattern
|
|
427
|
+
|
|
428
|
+
```ruby
|
|
429
|
+
# Every plugin method is wrapped in error boundary
|
|
430
|
+
def safe_fetch_messages(plugin, since: nil)
|
|
431
|
+
plugin.fetch_messages(since: since)
|
|
432
|
+
rescue => e
|
|
433
|
+
log_plugin_error(plugin, e)
|
|
434
|
+
[] # Return empty array, don't crash
|
|
435
|
+
end
|
|
436
|
+
|
|
437
|
+
def log_plugin_error(plugin, error)
|
|
438
|
+
@db.exec(
|
|
439
|
+
"UPDATE sources SET last_error = ?, updated_at = ? WHERE id = ?",
|
|
440
|
+
[error.message, Time.now.to_i, plugin.source_id]
|
|
441
|
+
)
|
|
442
|
+
|
|
443
|
+
@event_bus.publish(:plugin_error, {
|
|
444
|
+
plugin: plugin.name,
|
|
445
|
+
error: error.message,
|
|
446
|
+
backtrace: error.backtrace.first(5)
|
|
447
|
+
})
|
|
448
|
+
end
|
|
449
|
+
```
|
|
450
|
+
|
|
451
|
+
### Plugin-Specific Errors
|
|
452
|
+
|
|
453
|
+
```ruby
|
|
454
|
+
# lib/heathrow/plugin/errors.rb
|
|
455
|
+
module Heathrow
|
|
456
|
+
module Plugin
|
|
457
|
+
class Error < StandardError; end
|
|
458
|
+
class ConnectionError < Error; end
|
|
459
|
+
class AuthenticationError < Error; end
|
|
460
|
+
class RateLimitError < Error; end
|
|
461
|
+
class NotFoundError < Error; end
|
|
462
|
+
class ValidationError < Error; end
|
|
463
|
+
end
|
|
464
|
+
end
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
### Retry Strategy
|
|
468
|
+
|
|
469
|
+
```ruby
|
|
470
|
+
def fetch_with_retry(plugin, max_retries: 3)
|
|
471
|
+
retries = 0
|
|
472
|
+
|
|
473
|
+
begin
|
|
474
|
+
plugin.fetch_messages
|
|
475
|
+
rescue Plugin::RateLimitError => e
|
|
476
|
+
# Wait and retry
|
|
477
|
+
sleep_time = 2 ** retries
|
|
478
|
+
log(:warn, "Rate limited, waiting #{sleep_time}s")
|
|
479
|
+
sleep(sleep_time)
|
|
480
|
+
retries += 1
|
|
481
|
+
retry if retries < max_retries
|
|
482
|
+
|
|
483
|
+
raise
|
|
484
|
+
rescue Plugin::ConnectionError => e
|
|
485
|
+
# Temporary network issue, retry
|
|
486
|
+
retries += 1
|
|
487
|
+
retry if retries < max_retries
|
|
488
|
+
|
|
489
|
+
raise
|
|
490
|
+
rescue Plugin::AuthenticationError => e
|
|
491
|
+
# Don't retry, needs user intervention
|
|
492
|
+
raise
|
|
493
|
+
end
|
|
494
|
+
end
|
|
495
|
+
```
|
|
496
|
+
|
|
497
|
+
---
|
|
498
|
+
|
|
499
|
+
## Testing Plugins
|
|
500
|
+
|
|
501
|
+
### Unit Testing
|
|
502
|
+
|
|
503
|
+
```ruby
|
|
504
|
+
# test/plugins/test_gmail.rb
|
|
505
|
+
require 'minitest/autorun'
|
|
506
|
+
require_relative '../test_helper'
|
|
507
|
+
|
|
508
|
+
class TestGmailPlugin < Minitest::Test
|
|
509
|
+
def setup
|
|
510
|
+
@config = {
|
|
511
|
+
'email' => 'test@example.com',
|
|
512
|
+
'credentials' => 'mock_token'
|
|
513
|
+
}
|
|
514
|
+
@event_bus = MockEventBus.new
|
|
515
|
+
@db = MockDatabase.new
|
|
516
|
+
|
|
517
|
+
@plugin = Heathrow::Plugin::Gmail.new(@config, @event_bus, @db)
|
|
518
|
+
end
|
|
519
|
+
|
|
520
|
+
def test_capabilities
|
|
521
|
+
assert_includes @plugin.capabilities, :read
|
|
522
|
+
assert_includes @plugin.capabilities, :write
|
|
523
|
+
assert_includes @plugin.capabilities, :attachments
|
|
524
|
+
end
|
|
525
|
+
|
|
526
|
+
def test_fetch_messages
|
|
527
|
+
# Mock Gmail API response
|
|
528
|
+
stub_gmail_api do
|
|
529
|
+
messages = @plugin.fetch_messages
|
|
530
|
+
assert_instance_of Array, messages
|
|
531
|
+
assert messages.all? { |m| m.is_a?(Heathrow::Message) }
|
|
532
|
+
end
|
|
533
|
+
end
|
|
534
|
+
|
|
535
|
+
def test_send_message
|
|
536
|
+
message = Heathrow::Message.new(
|
|
537
|
+
recipients: ['recipient@example.com'],
|
|
538
|
+
subject: 'Test',
|
|
539
|
+
content: 'Test message'
|
|
540
|
+
)
|
|
541
|
+
|
|
542
|
+
stub_gmail_api do
|
|
543
|
+
assert @plugin.send_message(message, nil)
|
|
544
|
+
end
|
|
545
|
+
end
|
|
546
|
+
|
|
547
|
+
def test_error_handling
|
|
548
|
+
# Simulate network error
|
|
549
|
+
stub_gmail_api_error(Net::ReadTimeout) do
|
|
550
|
+
assert_raises(Heathrow::Plugin::ConnectionError) do
|
|
551
|
+
@plugin.fetch_messages
|
|
552
|
+
end
|
|
553
|
+
end
|
|
554
|
+
end
|
|
555
|
+
end
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
### Integration Testing
|
|
559
|
+
|
|
560
|
+
```ruby
|
|
561
|
+
# test/integration/test_gmail_live.rb
|
|
562
|
+
# These tests require real credentials and are skipped in CI
|
|
563
|
+
|
|
564
|
+
class TestGmailLive < Minitest::Test
|
|
565
|
+
def setup
|
|
566
|
+
skip unless ENV['HEATHROW_GMAIL_TEST_TOKEN']
|
|
567
|
+
|
|
568
|
+
@config = {
|
|
569
|
+
'email' => ENV['HEATHROW_GMAIL_TEST_EMAIL'],
|
|
570
|
+
'credentials' => ENV['HEATHROW_GMAIL_TEST_TOKEN']
|
|
571
|
+
}
|
|
572
|
+
# ... setup
|
|
573
|
+
end
|
|
574
|
+
|
|
575
|
+
def test_real_connection
|
|
576
|
+
@plugin.start
|
|
577
|
+
assert_equal :running, @plugin.status[:state]
|
|
578
|
+
end
|
|
579
|
+
|
|
580
|
+
def test_real_fetch
|
|
581
|
+
messages = @plugin.fetch_messages
|
|
582
|
+
# Just verify it doesn't crash, actual count varies
|
|
583
|
+
assert_instance_of Array, messages
|
|
584
|
+
end
|
|
585
|
+
end
|
|
586
|
+
```
|
|
587
|
+
|
|
588
|
+
### Mock Helpers
|
|
589
|
+
|
|
590
|
+
```ruby
|
|
591
|
+
# test/mocks.rb
|
|
592
|
+
class MockEventBus
|
|
593
|
+
def initialize
|
|
594
|
+
@events = []
|
|
595
|
+
end
|
|
596
|
+
|
|
597
|
+
def publish(type, data)
|
|
598
|
+
@events << {type: type, data: data}
|
|
599
|
+
end
|
|
600
|
+
|
|
601
|
+
def events_of_type(type)
|
|
602
|
+
@events.select { |e| e[:type] == type }
|
|
603
|
+
end
|
|
604
|
+
end
|
|
605
|
+
|
|
606
|
+
class MockDatabase
|
|
607
|
+
def initialize
|
|
608
|
+
@data = {}
|
|
609
|
+
end
|
|
610
|
+
|
|
611
|
+
def query(sql, params = [])
|
|
612
|
+
# Simple mock implementation
|
|
613
|
+
[]
|
|
614
|
+
end
|
|
615
|
+
|
|
616
|
+
def exec(sql, params = [])
|
|
617
|
+
# No-op
|
|
618
|
+
end
|
|
619
|
+
end
|
|
620
|
+
```
|
|
621
|
+
|
|
622
|
+
---
|
|
623
|
+
|
|
624
|
+
## Example Plugins
|
|
625
|
+
|
|
626
|
+
### 1. Gmail Plugin (Two-way)
|
|
627
|
+
|
|
628
|
+
**File:** `lib/heathrow/plugins/gmail.rb`
|
|
629
|
+
|
|
630
|
+
```ruby
|
|
631
|
+
require 'google/apis/gmail_v1'
|
|
632
|
+
require 'googleauth'
|
|
633
|
+
|
|
634
|
+
module Heathrow
|
|
635
|
+
module Plugin
|
|
636
|
+
class Gmail < Base
|
|
637
|
+
def self.metadata
|
|
638
|
+
{
|
|
639
|
+
name: "Gmail",
|
|
640
|
+
description: "Gmail email integration via Google API",
|
|
641
|
+
version: "1.0.0",
|
|
642
|
+
author: "Heathrow Team",
|
|
643
|
+
two_way: true
|
|
644
|
+
}
|
|
645
|
+
end
|
|
646
|
+
|
|
647
|
+
def self.config_schema
|
|
648
|
+
{
|
|
649
|
+
type: "object",
|
|
650
|
+
required: ["email"],
|
|
651
|
+
properties: {
|
|
652
|
+
email: {
|
|
653
|
+
type: "string",
|
|
654
|
+
format: "email",
|
|
655
|
+
description: "Gmail address"
|
|
656
|
+
},
|
|
657
|
+
sync_labels: {
|
|
658
|
+
type: "array",
|
|
659
|
+
items: {type: "string"},
|
|
660
|
+
default: ["INBOX"],
|
|
661
|
+
description: "Labels to sync"
|
|
662
|
+
},
|
|
663
|
+
sync_days: {
|
|
664
|
+
type: "integer",
|
|
665
|
+
default: 30,
|
|
666
|
+
description: "Days of history to sync"
|
|
667
|
+
}
|
|
668
|
+
}
|
|
669
|
+
}
|
|
670
|
+
end
|
|
671
|
+
|
|
672
|
+
def initialize(config, event_bus, db)
|
|
673
|
+
super
|
|
674
|
+
@service = nil
|
|
675
|
+
end
|
|
676
|
+
|
|
677
|
+
def start
|
|
678
|
+
super
|
|
679
|
+
|
|
680
|
+
# Initialize Gmail API service
|
|
681
|
+
@service = Google::Apis::GmailV1::GmailService.new
|
|
682
|
+
@service.authorization = get_authorization
|
|
683
|
+
|
|
684
|
+
# Test connection
|
|
685
|
+
@service.get_user_profile('me')
|
|
686
|
+
|
|
687
|
+
log(:info, "Connected to Gmail: #{@config['email']}")
|
|
688
|
+
rescue Google::Apis::AuthorizationError => e
|
|
689
|
+
raise Plugin::AuthenticationError, "Gmail auth failed: #{e.message}"
|
|
690
|
+
rescue => e
|
|
691
|
+
raise Plugin::ConnectionError, "Gmail connection failed: #{e.message}"
|
|
692
|
+
end
|
|
693
|
+
|
|
694
|
+
def stop
|
|
695
|
+
super
|
|
696
|
+
@service = nil
|
|
697
|
+
end
|
|
698
|
+
|
|
699
|
+
def capabilities
|
|
700
|
+
[:read, :write, :attachments, :threads, :search]
|
|
701
|
+
end
|
|
702
|
+
|
|
703
|
+
def fetch_messages(since: nil)
|
|
704
|
+
query = build_query(since)
|
|
705
|
+
|
|
706
|
+
message_ids = fetch_message_ids(query)
|
|
707
|
+
messages = message_ids.map { |id| fetch_message_detail(id) }
|
|
708
|
+
|
|
709
|
+
messages.compact.map { |m| normalize_message(m) }
|
|
710
|
+
|
|
711
|
+
rescue Google::Apis::RateLimitError => e
|
|
712
|
+
raise Plugin::RateLimitError, "Gmail rate limit: #{e.message}"
|
|
713
|
+
rescue => e
|
|
714
|
+
log(:error, "Fetch failed: #{e.message}")
|
|
715
|
+
raise Plugin::Error, e.message
|
|
716
|
+
end
|
|
717
|
+
|
|
718
|
+
def send_message(message, target = nil)
|
|
719
|
+
raw_message = build_raw_message(message)
|
|
720
|
+
|
|
721
|
+
@service.send_user_message(
|
|
722
|
+
'me',
|
|
723
|
+
Google::Apis::GmailV1::Message.new(raw: raw_message)
|
|
724
|
+
)
|
|
725
|
+
|
|
726
|
+
emit_event(:message_sent, {
|
|
727
|
+
recipients: message.recipients,
|
|
728
|
+
subject: message.subject
|
|
729
|
+
})
|
|
730
|
+
|
|
731
|
+
true
|
|
732
|
+
rescue => e
|
|
733
|
+
log(:error, "Send failed: #{e.message}")
|
|
734
|
+
raise Plugin::Error, e.message
|
|
735
|
+
end
|
|
736
|
+
|
|
737
|
+
def search(query)
|
|
738
|
+
# Implement Gmail search syntax
|
|
739
|
+
fetch_messages(query: query)
|
|
740
|
+
end
|
|
741
|
+
|
|
742
|
+
private
|
|
743
|
+
|
|
744
|
+
def get_authorization
|
|
745
|
+
token = retrieve_credential('oauth_token')
|
|
746
|
+
refresh_token = retrieve_credential('refresh_token')
|
|
747
|
+
|
|
748
|
+
# Create OAuth2 credentials
|
|
749
|
+
credentials = Google::Auth::UserRefreshCredentials.new(
|
|
750
|
+
client_id: ENV['GMAIL_CLIENT_ID'],
|
|
751
|
+
client_secret: ENV['GMAIL_CLIENT_SECRET'],
|
|
752
|
+
refresh_token: refresh_token,
|
|
753
|
+
access_token: token
|
|
754
|
+
)
|
|
755
|
+
|
|
756
|
+
# Refresh if needed
|
|
757
|
+
credentials.refresh! if credentials.expired?
|
|
758
|
+
|
|
759
|
+
# Store new token
|
|
760
|
+
store_credential('oauth_token', credentials.access_token) if credentials.access_token != token
|
|
761
|
+
|
|
762
|
+
credentials
|
|
763
|
+
end
|
|
764
|
+
|
|
765
|
+
def build_query(since)
|
|
766
|
+
parts = []
|
|
767
|
+
|
|
768
|
+
# Sync specific labels
|
|
769
|
+
labels = @config['sync_labels'] || ['INBOX']
|
|
770
|
+
parts << "label:(#{labels.join(' OR ')})"
|
|
771
|
+
|
|
772
|
+
# Since timestamp
|
|
773
|
+
if since
|
|
774
|
+
date = Time.at(since).strftime('%Y/%m/%d')
|
|
775
|
+
parts << "after:#{date}"
|
|
776
|
+
else
|
|
777
|
+
# Default: last N days
|
|
778
|
+
days = @config['sync_days'] || 30
|
|
779
|
+
parts << "newer_than:#{days}d"
|
|
780
|
+
end
|
|
781
|
+
|
|
782
|
+
parts.join(' ')
|
|
783
|
+
end
|
|
784
|
+
|
|
785
|
+
def fetch_message_ids(query)
|
|
786
|
+
ids = []
|
|
787
|
+
page_token = nil
|
|
788
|
+
|
|
789
|
+
loop do
|
|
790
|
+
result = @service.list_user_messages('me', q: query, page_token: page_token)
|
|
791
|
+
ids.concat(result.messages.map(&:id)) if result.messages
|
|
792
|
+
|
|
793
|
+
page_token = result.next_page_token
|
|
794
|
+
break unless page_token
|
|
795
|
+
end
|
|
796
|
+
|
|
797
|
+
ids
|
|
798
|
+
end
|
|
799
|
+
|
|
800
|
+
def fetch_message_detail(message_id)
|
|
801
|
+
@service.get_user_message('me', message_id, format: 'full')
|
|
802
|
+
end
|
|
803
|
+
|
|
804
|
+
def normalize_message(gmail_msg)
|
|
805
|
+
headers = headers_to_hash(gmail_msg.payload.headers)
|
|
806
|
+
|
|
807
|
+
Heathrow::Message.new(
|
|
808
|
+
external_id: gmail_msg.id,
|
|
809
|
+
thread_id: gmail_msg.thread_id,
|
|
810
|
+
sender: headers['from'],
|
|
811
|
+
recipients: parse_recipients(headers['to']),
|
|
812
|
+
subject: headers['subject'],
|
|
813
|
+
content: extract_body(gmail_msg.payload),
|
|
814
|
+
html_content: extract_html(gmail_msg.payload),
|
|
815
|
+
timestamp: gmail_msg.internal_date / 1000,
|
|
816
|
+
received_at: Time.now.to_i,
|
|
817
|
+
labels: gmail_msg.label_ids,
|
|
818
|
+
metadata: {
|
|
819
|
+
gmail_message_id: headers['message-id'],
|
|
820
|
+
gmail_thread_id: gmail_msg.thread_id,
|
|
821
|
+
gmail_labels: gmail_msg.label_ids
|
|
822
|
+
}.to_json
|
|
823
|
+
)
|
|
824
|
+
end
|
|
825
|
+
|
|
826
|
+
def headers_to_hash(headers)
|
|
827
|
+
headers.each_with_object({}) do |h, hash|
|
|
828
|
+
hash[h.name.downcase] = h.value
|
|
829
|
+
end
|
|
830
|
+
end
|
|
831
|
+
|
|
832
|
+
def parse_recipients(to_header)
|
|
833
|
+
return [] unless to_header
|
|
834
|
+
to_header.split(',').map(&:strip)
|
|
835
|
+
end
|
|
836
|
+
|
|
837
|
+
def extract_body(payload)
|
|
838
|
+
if payload.parts
|
|
839
|
+
text_part = payload.parts.find { |p| p.mime_type == 'text/plain' }
|
|
840
|
+
return decode_body(text_part.body.data) if text_part
|
|
841
|
+
end
|
|
842
|
+
|
|
843
|
+
decode_body(payload.body.data) if payload.body&.data
|
|
844
|
+
end
|
|
845
|
+
|
|
846
|
+
def extract_html(payload)
|
|
847
|
+
if payload.parts
|
|
848
|
+
html_part = payload.parts.find { |p| p.mime_type == 'text/html' }
|
|
849
|
+
return decode_body(html_part.body.data) if html_part
|
|
850
|
+
end
|
|
851
|
+
|
|
852
|
+
nil
|
|
853
|
+
end
|
|
854
|
+
|
|
855
|
+
def decode_body(data)
|
|
856
|
+
return nil unless data
|
|
857
|
+
Base64.urlsafe_decode64(data)
|
|
858
|
+
end
|
|
859
|
+
|
|
860
|
+
def build_raw_message(message)
|
|
861
|
+
# Build RFC 2822 email
|
|
862
|
+
mail = <<~EMAIL
|
|
863
|
+
From: #{@config['email']}
|
|
864
|
+
To: #{message.recipients.join(', ')}
|
|
865
|
+
Subject: #{message.subject}
|
|
866
|
+
Content-Type: text/plain; charset=UTF-8
|
|
867
|
+
|
|
868
|
+
#{message.content}
|
|
869
|
+
EMAIL
|
|
870
|
+
|
|
871
|
+
Base64.urlsafe_encode64(mail, padding: false)
|
|
872
|
+
end
|
|
873
|
+
end
|
|
874
|
+
end
|
|
875
|
+
end
|
|
876
|
+
```
|
|
877
|
+
|
|
878
|
+
### 2. RSS Plugin (One-way)
|
|
879
|
+
|
|
880
|
+
**File:** `lib/heathrow/plugins/rss.rb`
|
|
881
|
+
|
|
882
|
+
```ruby
|
|
883
|
+
require 'rss'
|
|
884
|
+
require 'open-uri'
|
|
885
|
+
|
|
886
|
+
module Heathrow
|
|
887
|
+
module Plugin
|
|
888
|
+
class Rss < Base
|
|
889
|
+
def self.metadata
|
|
890
|
+
{
|
|
891
|
+
name: "RSS",
|
|
892
|
+
description: "RSS/Atom feed reader",
|
|
893
|
+
version: "1.0.0",
|
|
894
|
+
author: "Heathrow Team",
|
|
895
|
+
two_way: false
|
|
896
|
+
}
|
|
897
|
+
end
|
|
898
|
+
|
|
899
|
+
def self.config_schema
|
|
900
|
+
{
|
|
901
|
+
type: "object",
|
|
902
|
+
required: ["feed_url"],
|
|
903
|
+
properties: {
|
|
904
|
+
feed_url: {
|
|
905
|
+
type: "string",
|
|
906
|
+
format: "uri",
|
|
907
|
+
description: "RSS feed URL"
|
|
908
|
+
},
|
|
909
|
+
update_interval: {
|
|
910
|
+
type: "integer",
|
|
911
|
+
default: 3600,
|
|
912
|
+
description: "Update interval in seconds"
|
|
913
|
+
}
|
|
914
|
+
}
|
|
915
|
+
}
|
|
916
|
+
end
|
|
917
|
+
|
|
918
|
+
def capabilities
|
|
919
|
+
[:read]
|
|
920
|
+
end
|
|
921
|
+
|
|
922
|
+
def fetch_messages(since: nil)
|
|
923
|
+
feed = fetch_feed
|
|
924
|
+
items = feed.items
|
|
925
|
+
|
|
926
|
+
# Filter by timestamp if provided
|
|
927
|
+
items = items.select { |i| i.pubDate.to_i > since } if since
|
|
928
|
+
|
|
929
|
+
items.map { |item| normalize_message(item) }
|
|
930
|
+
|
|
931
|
+
rescue => e
|
|
932
|
+
log(:error, "Feed fetch failed: #{e.message}")
|
|
933
|
+
[]
|
|
934
|
+
end
|
|
935
|
+
|
|
936
|
+
private
|
|
937
|
+
|
|
938
|
+
def fetch_feed
|
|
939
|
+
URI.open(@config['feed_url']) do |rss|
|
|
940
|
+
RSS::Parser.parse(rss)
|
|
941
|
+
end
|
|
942
|
+
end
|
|
943
|
+
|
|
944
|
+
def normalize_message(item)
|
|
945
|
+
Heathrow::Message.new(
|
|
946
|
+
external_id: item.guid&.content || item.link,
|
|
947
|
+
sender: @config['feed_url'],
|
|
948
|
+
recipients: [],
|
|
949
|
+
subject: item.title,
|
|
950
|
+
content: strip_html(item.description || item.content_encoded),
|
|
951
|
+
html_content: item.description || item.content_encoded,
|
|
952
|
+
timestamp: item.pubDate.to_i,
|
|
953
|
+
received_at: Time.now.to_i,
|
|
954
|
+
metadata: {
|
|
955
|
+
rss_link: item.link,
|
|
956
|
+
rss_categories: item.categories.map(&:content)
|
|
957
|
+
}.to_json
|
|
958
|
+
)
|
|
959
|
+
end
|
|
960
|
+
|
|
961
|
+
def strip_html(html)
|
|
962
|
+
return '' unless html
|
|
963
|
+
html.gsub(/<[^>]+>/, '').strip
|
|
964
|
+
end
|
|
965
|
+
end
|
|
966
|
+
end
|
|
967
|
+
end
|
|
968
|
+
```
|
|
969
|
+
|
|
970
|
+
### 3. Slack Plugin (Two-way, Real-time)
|
|
971
|
+
|
|
972
|
+
**File:** `lib/heathrow/plugins/slack.rb`
|
|
973
|
+
|
|
974
|
+
```ruby
|
|
975
|
+
require 'slack-ruby-client'
|
|
976
|
+
|
|
977
|
+
module Heathrow
|
|
978
|
+
module Plugin
|
|
979
|
+
class Slack < Base
|
|
980
|
+
def self.metadata
|
|
981
|
+
{
|
|
982
|
+
name: "Slack",
|
|
983
|
+
description: "Slack workspace integration",
|
|
984
|
+
version: "1.0.0",
|
|
985
|
+
author: "Heathrow Team",
|
|
986
|
+
two_way: true
|
|
987
|
+
}
|
|
988
|
+
end
|
|
989
|
+
|
|
990
|
+
def capabilities
|
|
991
|
+
[:read, :write, :real_time, :threads, :reactions]
|
|
992
|
+
end
|
|
993
|
+
|
|
994
|
+
def start
|
|
995
|
+
super
|
|
996
|
+
|
|
997
|
+
# Initialize Slack client
|
|
998
|
+
::Slack.configure do |config|
|
|
999
|
+
config.token = retrieve_credential('token')
|
|
1000
|
+
end
|
|
1001
|
+
|
|
1002
|
+
@client = ::Slack::Web::Client.new
|
|
1003
|
+
@rtm_client = ::Slack::RealTime::Client.new
|
|
1004
|
+
|
|
1005
|
+
# Test connection
|
|
1006
|
+
auth = @client.auth_test
|
|
1007
|
+
log(:info, "Connected to Slack: #{auth.team}")
|
|
1008
|
+
|
|
1009
|
+
# Start real-time client
|
|
1010
|
+
start_rtm
|
|
1011
|
+
|
|
1012
|
+
rescue ::Slack::Web::Api::Errors::SlackError => e
|
|
1013
|
+
raise Plugin::AuthenticationError, "Slack auth failed: #{e.message}"
|
|
1014
|
+
end
|
|
1015
|
+
|
|
1016
|
+
def stop
|
|
1017
|
+
super
|
|
1018
|
+
@rtm_client&.stop!
|
|
1019
|
+
end
|
|
1020
|
+
|
|
1021
|
+
def fetch_messages(since: nil)
|
|
1022
|
+
messages = []
|
|
1023
|
+
|
|
1024
|
+
# Fetch from configured channels
|
|
1025
|
+
channels = @config['sync_channels'] || []
|
|
1026
|
+
channels.each do |channel_name|
|
|
1027
|
+
channel = find_channel(channel_name)
|
|
1028
|
+
next unless channel
|
|
1029
|
+
|
|
1030
|
+
history = @client.conversations_history(
|
|
1031
|
+
channel: channel['id'],
|
|
1032
|
+
oldest: since || (Time.now.to_i - 86400) # Default: last 24h
|
|
1033
|
+
)
|
|
1034
|
+
|
|
1035
|
+
messages.concat(history.messages.map { |m| normalize_message(m, channel) })
|
|
1036
|
+
end
|
|
1037
|
+
|
|
1038
|
+
messages
|
|
1039
|
+
end
|
|
1040
|
+
|
|
1041
|
+
def send_message(message, target)
|
|
1042
|
+
channel = find_channel(target)
|
|
1043
|
+
raise Plugin::NotFoundError, "Channel not found: #{target}" unless channel
|
|
1044
|
+
|
|
1045
|
+
@client.chat_postMessage(
|
|
1046
|
+
channel: channel['id'],
|
|
1047
|
+
text: message.content,
|
|
1048
|
+
thread_ts: message.metadata&.dig('slack_thread_ts')
|
|
1049
|
+
)
|
|
1050
|
+
|
|
1051
|
+
true
|
|
1052
|
+
end
|
|
1053
|
+
|
|
1054
|
+
private
|
|
1055
|
+
|
|
1056
|
+
def start_rtm
|
|
1057
|
+
@rtm_client.on :message do |data|
|
|
1058
|
+
next if data.subtype # Skip edited, deleted, etc.
|
|
1059
|
+
|
|
1060
|
+
channel = find_channel_by_id(data.channel)
|
|
1061
|
+
message = normalize_message(data, channel)
|
|
1062
|
+
|
|
1063
|
+
emit_event(:message_received, {message: message})
|
|
1064
|
+
end
|
|
1065
|
+
|
|
1066
|
+
Thread.new { @rtm_client.start! }
|
|
1067
|
+
end
|
|
1068
|
+
|
|
1069
|
+
def find_channel(name)
|
|
1070
|
+
@channels_cache ||= @client.conversations_list.channels
|
|
1071
|
+
@channels_cache.find { |c| c.name == name }
|
|
1072
|
+
end
|
|
1073
|
+
|
|
1074
|
+
def find_channel_by_id(id)
|
|
1075
|
+
@channels_cache ||= @client.conversations_list.channels
|
|
1076
|
+
@channels_cache.find { |c| c.id == id }
|
|
1077
|
+
end
|
|
1078
|
+
|
|
1079
|
+
def normalize_message(slack_msg, channel)
|
|
1080
|
+
Heathrow::Message.new(
|
|
1081
|
+
external_id: slack_msg.ts,
|
|
1082
|
+
thread_id: slack_msg.thread_ts || slack_msg.ts,
|
|
1083
|
+
sender: get_user_name(slack_msg.user),
|
|
1084
|
+
recipients: [channel['name']],
|
|
1085
|
+
subject: "#{channel['name']} - Slack",
|
|
1086
|
+
content: slack_msg.text,
|
|
1087
|
+
timestamp: slack_msg.ts.to_f.to_i,
|
|
1088
|
+
received_at: Time.now.to_i,
|
|
1089
|
+
metadata: {
|
|
1090
|
+
slack_channel: channel['name'],
|
|
1091
|
+
slack_channel_id: channel['id'],
|
|
1092
|
+
slack_ts: slack_msg.ts,
|
|
1093
|
+
slack_thread_ts: slack_msg.thread_ts
|
|
1094
|
+
}.to_json
|
|
1095
|
+
)
|
|
1096
|
+
end
|
|
1097
|
+
|
|
1098
|
+
def get_user_name(user_id)
|
|
1099
|
+
@users_cache ||= {}
|
|
1100
|
+
@users_cache[user_id] ||= begin
|
|
1101
|
+
user = @client.users_info(user: user_id).user
|
|
1102
|
+
user.profile.display_name.empty? ? user.name : user.profile.display_name
|
|
1103
|
+
end
|
|
1104
|
+
end
|
|
1105
|
+
end
|
|
1106
|
+
end
|
|
1107
|
+
end
|
|
1108
|
+
```
|
|
1109
|
+
|
|
1110
|
+
---
|
|
1111
|
+
|
|
1112
|
+
## Plugin Development Guide
|
|
1113
|
+
|
|
1114
|
+
### Step-by-Step: Creating a New Plugin
|
|
1115
|
+
|
|
1116
|
+
#### 1. Create Plugin File
|
|
1117
|
+
|
|
1118
|
+
```bash
|
|
1119
|
+
touch lib/heathrow/plugins/myservice.rb
|
|
1120
|
+
```
|
|
1121
|
+
|
|
1122
|
+
#### 2. Define Plugin Class
|
|
1123
|
+
|
|
1124
|
+
```ruby
|
|
1125
|
+
module Heathrow
|
|
1126
|
+
module Plugin
|
|
1127
|
+
class Myservice < Base
|
|
1128
|
+
def self.metadata
|
|
1129
|
+
{
|
|
1130
|
+
name: "MyService",
|
|
1131
|
+
description: "Integration with MyService platform",
|
|
1132
|
+
version: "1.0.0",
|
|
1133
|
+
author: "Your Name",
|
|
1134
|
+
two_way: true # or false for read-only
|
|
1135
|
+
}
|
|
1136
|
+
end
|
|
1137
|
+
|
|
1138
|
+
def capabilities
|
|
1139
|
+
[:read, :write] # Adjust based on what you implement
|
|
1140
|
+
end
|
|
1141
|
+
|
|
1142
|
+
# Implement required methods...
|
|
1143
|
+
end
|
|
1144
|
+
end
|
|
1145
|
+
end
|
|
1146
|
+
```
|
|
1147
|
+
|
|
1148
|
+
#### 3. Implement Required Methods
|
|
1149
|
+
|
|
1150
|
+
```ruby
|
|
1151
|
+
def fetch_messages(since: nil)
|
|
1152
|
+
# 1. Connect to service API
|
|
1153
|
+
# 2. Fetch messages since timestamp
|
|
1154
|
+
# 3. Convert to Heathrow::Message objects
|
|
1155
|
+
# 4. Return array
|
|
1156
|
+
[]
|
|
1157
|
+
end
|
|
1158
|
+
|
|
1159
|
+
def send_message(message, target)
|
|
1160
|
+
# 1. Connect to service API
|
|
1161
|
+
# 2. Send message to target
|
|
1162
|
+
# 3. Return true on success
|
|
1163
|
+
# 4. Raise Plugin::Error on failure
|
|
1164
|
+
true
|
|
1165
|
+
end
|
|
1166
|
+
```
|
|
1167
|
+
|
|
1168
|
+
#### 4. Add Configuration Schema
|
|
1169
|
+
|
|
1170
|
+
```ruby
|
|
1171
|
+
def self.config_schema
|
|
1172
|
+
{
|
|
1173
|
+
type: "object",
|
|
1174
|
+
required: ["api_key"],
|
|
1175
|
+
properties: {
|
|
1176
|
+
api_key: {
|
|
1177
|
+
type: "string",
|
|
1178
|
+
description: "MyService API key"
|
|
1179
|
+
},
|
|
1180
|
+
sync_interval: {
|
|
1181
|
+
type: "integer",
|
|
1182
|
+
default: 300,
|
|
1183
|
+
description: "Sync interval in seconds"
|
|
1184
|
+
}
|
|
1185
|
+
}
|
|
1186
|
+
}
|
|
1187
|
+
end
|
|
1188
|
+
```
|
|
1189
|
+
|
|
1190
|
+
#### 5. Test Your Plugin
|
|
1191
|
+
|
|
1192
|
+
```ruby
|
|
1193
|
+
# test/plugins/test_myservice.rb
|
|
1194
|
+
class TestMyservice < Minitest::Test
|
|
1195
|
+
def test_fetch_messages
|
|
1196
|
+
# Mock API, test fetch logic
|
|
1197
|
+
end
|
|
1198
|
+
|
|
1199
|
+
def test_send_message
|
|
1200
|
+
# Mock API, test send logic
|
|
1201
|
+
end
|
|
1202
|
+
end
|
|
1203
|
+
```
|
|
1204
|
+
|
|
1205
|
+
#### 6. Document Your Plugin
|
|
1206
|
+
|
|
1207
|
+
```ruby
|
|
1208
|
+
# Add to docs/PLUGINS.md
|
|
1209
|
+
## MyService Plugin
|
|
1210
|
+
|
|
1211
|
+
**Type:** Two-way
|
|
1212
|
+
**Capabilities:** read, write
|
|
1213
|
+
|
|
1214
|
+
### Configuration
|
|
1215
|
+
|
|
1216
|
+
- `api_key` (required): Your MyService API key
|
|
1217
|
+
- `sync_interval` (optional): Sync interval in seconds (default: 300)
|
|
1218
|
+
|
|
1219
|
+
### Setup
|
|
1220
|
+
|
|
1221
|
+
1. Get API key from https://myservice.com/api
|
|
1222
|
+
2. Add source in Heathrow configuration
|
|
1223
|
+
3. Test connection
|
|
1224
|
+
```
|
|
1225
|
+
|
|
1226
|
+
---
|
|
1227
|
+
|
|
1228
|
+
## Best Practices
|
|
1229
|
+
|
|
1230
|
+
### 1. Error Handling
|
|
1231
|
+
|
|
1232
|
+
Always use specific error types:
|
|
1233
|
+
|
|
1234
|
+
```ruby
|
|
1235
|
+
def fetch_messages(since: nil)
|
|
1236
|
+
response = api_call
|
|
1237
|
+
|
|
1238
|
+
case response.code
|
|
1239
|
+
when 401
|
|
1240
|
+
raise Plugin::AuthenticationError, "Invalid credentials"
|
|
1241
|
+
when 429
|
|
1242
|
+
raise Plugin::RateLimitError, "Rate limit exceeded"
|
|
1243
|
+
when 404
|
|
1244
|
+
raise Plugin::NotFoundError, "Resource not found"
|
|
1245
|
+
when 500..599
|
|
1246
|
+
raise Plugin::ConnectionError, "Server error: #{response.code}"
|
|
1247
|
+
end
|
|
1248
|
+
|
|
1249
|
+
# Process response...
|
|
1250
|
+
rescue Timeout::Error
|
|
1251
|
+
raise Plugin::ConnectionError, "Request timeout"
|
|
1252
|
+
rescue JSON::ParserError => e
|
|
1253
|
+
raise Plugin::Error, "Invalid response format: #{e.message}"
|
|
1254
|
+
end
|
|
1255
|
+
```
|
|
1256
|
+
|
|
1257
|
+
### 2. Credentials
|
|
1258
|
+
|
|
1259
|
+
Never log or expose credentials:
|
|
1260
|
+
|
|
1261
|
+
```ruby
|
|
1262
|
+
def start
|
|
1263
|
+
api_key = retrieve_credential('api_key')
|
|
1264
|
+
|
|
1265
|
+
# Good
|
|
1266
|
+
log(:info, "Connecting to service...")
|
|
1267
|
+
|
|
1268
|
+
# Bad - leaks credential
|
|
1269
|
+
# log(:info, "Using API key: #{api_key}")
|
|
1270
|
+
|
|
1271
|
+
connect(api_key)
|
|
1272
|
+
end
|
|
1273
|
+
```
|
|
1274
|
+
|
|
1275
|
+
### 3. Rate Limiting
|
|
1276
|
+
|
|
1277
|
+
Respect API rate limits:
|
|
1278
|
+
|
|
1279
|
+
```ruby
|
|
1280
|
+
def fetch_messages(since: nil)
|
|
1281
|
+
# Implement backoff
|
|
1282
|
+
sleep(rate_limit_delay) if rate_limited?
|
|
1283
|
+
|
|
1284
|
+
# Batch requests
|
|
1285
|
+
message_ids.each_slice(100) do |batch|
|
|
1286
|
+
fetch_batch(batch)
|
|
1287
|
+
end
|
|
1288
|
+
end
|
|
1289
|
+
```
|
|
1290
|
+
|
|
1291
|
+
### 4. Caching
|
|
1292
|
+
|
|
1293
|
+
Cache expensive operations:
|
|
1294
|
+
|
|
1295
|
+
```ruby
|
|
1296
|
+
def get_user_name(user_id)
|
|
1297
|
+
@user_cache ||= {}
|
|
1298
|
+
@user_cache[user_id] ||= fetch_user_from_api(user_id).name
|
|
1299
|
+
end
|
|
1300
|
+
```
|
|
1301
|
+
|
|
1302
|
+
### 5. Logging
|
|
1303
|
+
|
|
1304
|
+
Log important events:
|
|
1305
|
+
|
|
1306
|
+
```ruby
|
|
1307
|
+
def fetch_messages(since: nil)
|
|
1308
|
+
log(:debug, "Fetching messages since #{since || 'beginning'}")
|
|
1309
|
+
|
|
1310
|
+
messages = do_fetch
|
|
1311
|
+
|
|
1312
|
+
log(:info, "Fetched #{messages.count} messages")
|
|
1313
|
+
|
|
1314
|
+
messages
|
|
1315
|
+
rescue => e
|
|
1316
|
+
log(:error, "Fetch failed: #{e.message}")
|
|
1317
|
+
raise
|
|
1318
|
+
end
|
|
1319
|
+
```
|
|
1320
|
+
|
|
1321
|
+
---
|
|
1322
|
+
|
|
1323
|
+
## Plugin Testing Checklist
|
|
1324
|
+
|
|
1325
|
+
Before submitting a plugin, test:
|
|
1326
|
+
|
|
1327
|
+
- [ ] Connection/authentication
|
|
1328
|
+
- [ ] Fetch messages (empty, small, large batches)
|
|
1329
|
+
- [ ] Send message
|
|
1330
|
+
- [ ] Handle network errors gracefully
|
|
1331
|
+
- [ ] Handle rate limits
|
|
1332
|
+
- [ ] Handle authentication errors
|
|
1333
|
+
- [ ] Plugin can be stopped and restarted
|
|
1334
|
+
- [ ] No memory leaks on long runs
|
|
1335
|
+
- [ ] Credentials are encrypted
|
|
1336
|
+
- [ ] No credentials in logs
|
|
1337
|
+
- [ ] Works with proxy (if applicable)
|
|
1338
|
+
- [ ] Handles malformed API responses
|
|
1339
|
+
- [ ] Thread-safe (if using real-time)
|
|
1340
|
+
|
|
1341
|
+
---
|
|
1342
|
+
|
|
1343
|
+
## Community Plugins
|
|
1344
|
+
|
|
1345
|
+
To publish a community plugin:
|
|
1346
|
+
|
|
1347
|
+
1. Create gem: `heathrow-plugin-myservice`
|
|
1348
|
+
2. Include plugin class in `lib/heathrow/plugin/myservice.rb`
|
|
1349
|
+
3. Add README with setup instructions
|
|
1350
|
+
4. Publish to RubyGems
|
|
1351
|
+
5. Submit PR to add to official plugin directory
|
|
1352
|
+
|
|
1353
|
+
**Plugin Gem Template:**
|
|
1354
|
+
|
|
1355
|
+
```
|
|
1356
|
+
heathrow-plugin-myservice/
|
|
1357
|
+
├── lib/
|
|
1358
|
+
│ └── heathrow/
|
|
1359
|
+
│ └── plugin/
|
|
1360
|
+
│ └── myservice.rb
|
|
1361
|
+
├── test/
|
|
1362
|
+
│ └── test_myservice.rb
|
|
1363
|
+
├── README.md
|
|
1364
|
+
├── LICENSE
|
|
1365
|
+
└── heathrow-plugin-myservice.gemspec
|
|
1366
|
+
```
|
|
1367
|
+
|
|
1368
|
+
---
|
|
1369
|
+
|
|
1370
|
+
This plugin system ensures every integration is isolated, testable, and can fail independently without bringing down Heathrow.
|