midi-smtp-server 2.3.3 → 3.0.3

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -19,8 +19,7 @@ MidiSmtpServer is the highly customizable ruby SMTP-Server and SMTP-Service libr
19
19
 
20
20
  As a library it is mainly designed to be integrated into your projects as serving a SMTP-Server service. The lib will do nothing with your mail and you have to create your own event functions to handle and operate on incoming mails. We are using this in conjunction with [Mikel Lindsaar](https://github.com/mikel) great Mail component (https://github.com/mikel/mail). Time to run your own SMTP-Server service.
21
21
 
22
- With version 2 the library gots a lot of improvements (2.3.x Multiple ports and addresses, 2.2.x Encryption [StartTLS], 2.1.0 Authentication [AUTH], 2.1.1 significant speed improvement, etc.).
23
- Please checkout section [changes and updates](https://github.com/4commerce-technologies-AG/midi-smtp-server#changes-and-updates) to get the news.
22
+ Checkout all the features and improvements (3.0.1 Logging enhancement, 2.3.x Multiple ports and addresses, 2.2.x Encryption [StartTLS], 2.1.0 Authentication [AUTH], 2.1.1 significant speed improvement, etc.) and get more details from section [changes and updates](https://github.com/4commerce-technologies-AG/midi-smtp-server#changes-and-updates).
24
23
 
25
24
  MidiSmtpServer is an extremely flexible library and almost any aspect of SMTP communications can be handled by deriving its events and using its configuration options.
26
25
 
@@ -88,995 +87,81 @@ Use the component in your project sources by:
88
87
  <br>
89
88
 
90
89
 
91
- ## Customizing the server
90
+ ## Library documentation
92
91
 
93
- MidiSmtpServer can be easily customized via subclassing. Simply subclass the `MidiSmtpServer` class as given in the example above and re-define event handlers:
94
-
95
- ```ruby
96
- # event on CONNECTION
97
- # you may change the ctx[:server][:local_response] and
98
- # you may change the ctx[:server][:helo_response] in here so
99
- # that these will be used as local welcome and greeting strings
100
- # the values are not allowed to return CR nor LF chars and will be stripped
101
- def on_connect_event(ctx)
102
- end
103
-
104
- # event before DISONNECT
105
- def on_disconnect_event(ctx)
106
- end
107
-
108
- # event on HELO/EHLO
109
- # you may change the ctx[:server][:helo_response] in here so
110
- # that this will be used as greeting string
111
- # the value is not allowed to return CR nor LF chars and will be stripped
112
- def on_helo_event(ctx, helo_data)
113
- end
114
-
115
- # get address send in MAIL FROM
116
- # if any value returned, that will be used for ongoing processing
117
- # otherwise the original value will be used
118
- def on_mail_from_event(ctx, mail_from_data)
119
- end
120
-
121
- # get each address send in RCPT TO
122
- # if any value returned, that will be used for ongoing processing
123
- # otherwise the original value will be used
124
- def on_rcpt_to_event(ctx, rcpt_to_data)
125
- end
126
-
127
- # event when beginning with message DATA
128
- def on_message_data_start_event(ctx)
129
- end
130
-
131
- # event while receiving message DATA
132
- def on_message_data_receiving_event(ctx)
133
- end
134
-
135
- # event when headers are received while receiving message DATA
136
- def on_message_data_headers_event(ctx)
137
- end
138
-
139
- # get each message after DATA <message>
140
- def on_message_data_event(ctx)
141
- end
142
-
143
- # event when process_line identifies an unknown command line
144
- # allows to abort sessions for a series of unknown activities to
145
- # prevent denial of service attacks etc.
146
- def on_process_line_unknown_event(ctx, line)
147
- end
148
- ```
149
-
150
- <br>
151
-
152
-
153
- ## IPv4 and IPv6 ready
154
-
155
- The underlaying ruby component [TCPServer](https://ruby-doc.org/stdlib-2.5.0/libdoc/socket/rdoc/TCPServer.html) allows support for IPv4 and IPv6 communication. If using the `DEFAULT_SMTPD_HOST` as your hosts option than explicitely IPv4 `127.0.0.1` will be enabled. If using the string `localhost` it depends on your _hosts_ file. If that contains a line like `::1 localhost` you might enable your server instance on IPv6 localhost only. Be aware of that when accessing your service.
156
-
157
- <br>
158
-
159
-
160
- ## Multiple ports and addresses
161
-
162
- Since version 2.3.0 you may define multiple ports and hosts or ip addresses at once when initializing the class. The ports and hosts arguments may be comma seperated strings with multiple ports and addresses like:
163
-
164
- ``` ruby
165
- # use port 2525 on all addresses
166
- server = MySmtpd.new('2525', '127.0.0.1, ::1, 192.168.0.1')
167
- # use ports 2525 and 3535 on all addresses
168
- server = MySmtpd.new('2525:3535', '127.0.0.1, ::1, 192.168.0.1')
169
- # use port 2525 on first address 127.0.0.1 and port 3535 on second address (and above)
170
- server = MySmtpd.new('2525, 3535', '127.0.0.1, ::1, 192.168.0.1')
171
- # use port 2525 on first address, port 3535 on second address, port 2525 on third
172
- server = MySmtpd.new('2525, 3535, 2525', '127.0.0.1, ::1, 192.168.0.1')
173
- # use port 2525 on first address, ports 2525 and 3535 on second address, port 2525 on third
174
- server = MySmtpd.new('2525, 2525:3535, 2525', '127.0.0.1, ::1, 192.168.0.1')
175
- ```
176
-
177
- You may write any combination of ports and addresses that should be served. That allows complex servers with optionally different services identified by different ports and addresses.
178
-
179
- There are also a `ports` and `hosts` reader for this values. Please be aware that we will drop the old attributes of `port` and `host` within the next minor release.
180
-
181
- <br>
182
-
183
-
184
- ## Hosts, hosts wildcard and interface detection
185
-
186
- Since version 2.3.2 the `hosts` parameter use the new `"*"` instead of the old empty (blank) `""` wildcard. This was updated to make sure that the new `"*"` wildcard should really identifiy and service on all (local) system interfaces. The new function will identify all valid IPv4 and IPv6 addresses on all (local) system interfaces. In addition the initialization will resolve all IPv4 and IPv6 addresses for all given hostnames. During startup a debug log message will print out the information to be aware of the listening ports and addresses. If an address is defined more than once like when using `"localhost, 127.0.0.1, ::1"`, the component will raise an exception that port and address is already in use.
187
-
188
- For production usage it is highly suggested to use defined IPv4 and IPv6 addresses for your services.
189
-
190
- <br>
191
-
192
-
193
- ## Utilization of connections and processings
194
-
195
- The options `max_processings` and `opts { max_connections }` allows to define the utilization of the running service. The value of `max_processings` will allow to queue processings while active processings have reached the maximum value. The additional (optional) value of `max_connections` will block any additional concurrent TCP connection and respond with SMTP error code 421 on more connections.
196
-
197
- E.g.:
198
-
199
- ``` ruby
200
- server = MySmtpd.new('2525', '127.0.0.1', 4, {max_connections: 100})
201
- ```
202
-
203
- In this example the service will allow 100 concurrent TCP connections but just process 4 of them simultaneously until all connections have been handled. If there are more than 100 concurrent TCP connections, those will be closed by error `421 Service too busy or not available`. That error code will _normally_ ensure, that the sender would try again after a while.
204
-
205
- This allows to calculate the utilization of your service by limiting the connections and processings.
206
-
207
- #### Calculate utilization
208
-
209
- It depends on the system resources (RAM, CPU) how many threads and connections your service may handle simultaniously but it should reflect also how many messages it has to proceed per time interval.
210
-
211
- For processing 1.000.000 mails per 24 hours, it may divided by seconds per day (24 * 60 * 60 = 86.400). This results in 11.5 mails per second. If the average processing time per mail is 15 seconds (long runner), then the service might have an overlap of 15 times 11.5 connections simultaniously. If that is expected, then `max_processings` of 172 should be fine.
212
-
213
- If you need 1.000.000 mail per hour than propably 416 simultaniously processed threads should be fine.
214
-
215
- The number of `max_connections` should always be equal or higher than `max_processings`. In the above examples it should be fine to use 512 or 1024 if your system does fit with its resources. If an unlimited number of concurrent TCP connections should be allowed, then set the value for `max_connections` to `nil` (which is also the default when not specified).
216
-
217
- <br>
218
-
219
-
220
- ## Modifying welcome and greeting responses
221
-
222
- While connecting from a client, the server will show up with a first local welcome message and after HELO or EHLO with a greeting message as well as the capabilities (EHLO). The response messages are build and stored in `ctx` values. You may change the content during `on_connect_event` and `on_helo_event`.
223
-
224
- ``` ruby
225
- # update local welcome and helo response
226
- def on_connect_event(ctx)
227
- ctx[:server][:local_response] = 'My welcome message!'
228
- ctx[:server][:helo_response] = 'My greeting message!'
229
- end
230
- ```
231
-
232
- If you want to show your local_ip or hostname etc. you may also include the context vars for that. Be aware to expose only necessary internal information and addresses etc.
233
-
234
- ``` ruby
235
- # update local welcome and helo response
236
- def on_connect_event(ctx)
237
- ctx[:server][:local_response] = "#{ctx[:server][:local_host]} [#{ctx[:server][:local_ip]}] says welcome!"
238
- ctx[:server][:helo_response] = "#{ctx[:server][:local_host]} [#{ctx[:server][:local_ip]}] is serving you!"
239
- end
240
- ```
241
-
242
- <br>
243
-
244
-
245
- ## Modifying MAIL FROM and RCPT TO addresses
246
-
247
- Since release `1.1.4` the `on_mail_from_event` and `on_rcpt_to_event` allows to return values that should be added to the lists. This is useful if you want to e.g. normalize all incoming addresses. Format defined by RFC for `<path>` as a `MAIL FROM` or `RCPT TO` addresses is:
248
-
249
- ```
250
- "<" | <path> | ">"
251
- ```
252
-
253
- Most mail servers allows also `<path>` only given addresses without leading and ending `< >`.
254
-
255
- To make it easier for processing addresses, you are able to normalize them like:
256
-
257
- ```ruby
258
- # simple rewrite and return value
259
- def on_mail_from_event(ctx, mail_from_data)
260
- # strip and normalize addresses like: <path> to path
261
- mail_from_data.gsub!(/^\s*<\s*(.*)\s*>\s*$/, '\1')
262
- # we believe in downcased addresses
263
- mail_from_data.downcase!
264
- # return address
265
- mail_from_data
266
- end
267
-
268
- # rewrite, process more checks and return value
269
- def on_rcpt_to_event(ctx, rcpt_to_data)
270
- # strip and normalize addresses like: <path> to path
271
- rcpt_to_data.gsub!(/^\s*<\s*(.*)\s*>\s*$/, '\1')
272
- # we believe in downcased addresses
273
- rcpt_to_data.downcase!
274
- # Output for debug
275
- puts "Normalized to: [#{rcpt_to_data}]..."
276
- # return address
277
- rcpt_to_data
278
- end
279
- ```
280
-
281
- <br>
282
-
283
-
284
- ## Adding and testing headers
285
-
286
- Since release `2.3.1` the `on_message_data_start_event` and `on_message_data_headers_event` enable the injection of additional headers like `Received` on DATA streaming. To add a `Received` header before any incoming header, use:
287
-
288
- ```ruby
289
- # event when beginning with message DATA
290
- def on_message_data_start_event(ctx)
291
- ctx[:message][:data] <<
292
- "Received: " <<
293
- "from #{ctx[:server][:remote_host]} (#{ctx[:server][:remote_ip]}) " <<
294
- "by #{ctx[:server][:local_host]} (#{ctx[:server][:local_ip]}) " <<
295
- "with MySmtpd Server; " <<
296
- Time.now.strftime("%a, %d %b %Y %H:%M:%S %z") <<
297
- ctx[:message][:crlf]
298
- end
299
- ```
300
-
301
- The `Received` header may be given with more or less additional information like encryption, recipient, sender etc. This should be done while being aware of system safety. Don't reveal too much internal information and choose wisely the published atrributes.
302
-
303
- Samples for `Received` headers are:
304
-
305
- ```
306
- Received: from localhost ([127.0.0.1])
307
- by mail.domain.test with esmtp (Exim 4.86)
308
- (envelope-from <user@sample.com>)
309
- id 3gIFk7-0006RC-FG
310
- for my.user@mydomain.net; Thu, 01 Nov 2018 12:00:00 +0000
311
- ```
312
-
313
- ```
314
- Received: from localhost ([127.0.0.1:10025])
315
- by mail.domain.test with ESMTPSA id 3gIFk7-0006RC-FG
316
- for <my.user@mydomain.net>
317
- (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
318
- Thu, 01 Nov 2018 12:00:00 +0000
319
- ```
320
-
321
- To append special headers or do some checks on transmitted headers, the `on_message_data_headers_event` is called when end of header transmission was automatically discovered.
322
-
323
- ```ruby
324
- # event when headers are received while receiving message DATA
325
- def on_message_data_headers_event(ctx)
326
- ctx[:message][:data] << 'X-MyHeader: 1.0' << ctx[:message][:crlf]
327
- end
328
- ```
329
-
330
- <br>
331
-
332
-
333
- ## Responding with errors on special conditions
334
-
335
- If you return from event class without an exception, the server will respond to client with the appropriate success code, otherwise the client will be noticed about an error.
336
-
337
- So you can build SPAM protection, when raising exception while getting `RCPT TO` events.
338
-
339
- ```ruby
340
- # get each address send in RCPT TO:
341
- def on_rcpt_to_event(ctx, rcpt_to_data)
342
- raise MidiSmtpServer::Smtpd550Exception if rcpt_to_data == "not.name@domain.con"
343
- end
344
- ```
345
-
346
- You are able to use exceptions on any level of events, so for an example you could raise an exception on `on_message_data_event` if you checked attachments for a pdf-document and fail or so on. If you use the defined `MidiSmtpServer::Smtpd???Exception` classes the remote client get's correct SMTP Server results. For logging purpose the default Exception.message is written to log.
347
-
348
- When using `MidiSmtpServer::Smtpd421Exception` you are able to abort the active connection to the client by replying `421 Service not available, closing transmission channel`. Be aware, that this Exception will actively close the current connection to the client. For logging purposes you may append a message to yourself, this will not be transmitted to the client.
349
-
350
- ```ruby
351
- # drop connection immediately on SPAM
352
- def on_rcpt_to_event(ctx, rcpt_to_data)
353
- raise MidiSmtpServer::Smtpd421Exception.new("421 Abort: Identified spammer!") if rcpt_to_data == "not.name@domain.con"
354
- end
355
- ```
356
-
357
- Please check RFC821 and additional for correct response dialog sequences:
358
-
359
- ```
360
- COMMAND-REPLY SEQUENCES
361
-
362
- Each command is listed with its possible replies. The prefixes
363
- used before the possible replies are "P" for preliminary (not
364
- used in SMTP), "I" for intermediate, "S" for success, "F" for
365
- failure, and "E" for error. The 421 reply (service not
366
- available, closing transmission channel) may be given to any
367
- command if the SMTP-receiver knows it must shut down. This
368
- listing forms the basis for the State Diagrams in Section 4.4.
369
-
370
- CONNECTION ESTABLISHMENT
371
- S: 220
372
- F: 421
373
- HELO
374
- S: 250
375
- E: 500, 501, 504, 421
376
- MAIL
377
- S: 250
378
- F: 552, 451, 452
379
- E: 500, 501, 421
380
- RCPT
381
- S: 250, 251
382
- F: 550, 551, 552, 553, 450, 451, 452
383
- E: 500, 501, 503, 421
384
- DATA
385
- I: 354 -> data -> S: 250
386
- F: 552, 554, 451, 452
387
- F: 451, 554
388
- E: 500, 501, 503, 421
389
- RSET
390
- S: 250
391
- E: 500, 501, 504, 421
392
- NOOP
393
- S: 250
394
- E: 500, 421
395
- QUIT
396
- S: 221
397
- E: 500
398
- AUTH
399
- S: 235
400
- F: 530, 534, 535, 454
401
- E: 500, 421
402
- ```
403
-
404
- <br>
405
-
406
-
407
- ## Access to server values and context
408
-
409
- You can access some important client and server values by using the `ctx` array when in event methods:
410
-
411
- ```ruby
412
- # welcome, helo/ehlo (client) and response strings
413
- ctx[:server][:local_response]
414
- ctx[:server][:helo]
415
- ctx[:server][:helo_response]
416
-
417
- # local (server's) infos
418
- ctx[:server][:local_ip]
419
- ctx[:server][:local_host]
420
- ctx[:server][:local_port]
421
-
422
- # remote (client) infos
423
- ctx[:server][:remote_ip]
424
- ctx[:server][:remote_host]
425
- ctx[:server][:remote_port]
426
-
427
- # connection timestamp (utc)
428
- ctx[:server][:connected]
429
-
430
- # counter (int) of exceptions / unknown commands
431
- ctx[:server][:exceptions]
432
-
433
- # authentification infos
434
- ctx[:server][:authorization_id]
435
- ctx[:server][:authentication_id]
436
-
437
- # successful authentication timestamp (utc)
438
- ctx[:server][:authenticated]
439
-
440
- # timestamp (utc) when encryption was established
441
- ctx[:server][:encrypted]
442
-
443
- # envelope mail from
444
- ctx[:envelope][:from]
445
-
446
- # envelope rcpt_to array
447
- ctx[:envelope][:to][0]
448
-
449
- # envelope enconding settings
450
- ctx[:message][:encoding_body]
451
- ctx[:message][:encoding_utf8]
452
-
453
- # timestamp (utc) when message data was initialized
454
- ctx[:message][:received]
455
-
456
- # timestamp (utc) when message data was completely received
457
- ctx[:message][:delivered]
458
-
459
- # flag to identify if headers already completed while receiving message data stream
460
- ctx[:message][:headers]
461
-
462
- # access message data size when message data was completely received
463
- ctx[:message][:bytesize]
464
-
465
- # string sequence for message data line-breaks
466
- ctx[:message][:crlf]
467
-
468
- # access message data while receiving message stream
469
- ctx[:message][:data]
470
-
471
- ```
92
+ Read the [MidiSmtpServer Documentation](https://midi-smtp-server.readthedocs.io/) for a complete library documentation.
472
93
 
473
94
  <br>
474
95
 
475
96
 
476
- ## Incoming data validation
477
-
478
- With release 2.2.3 there is an extended control about incoming data before processing. New options allow to set a timeout and maximum size of io_buffer for receiving client data up to a complete data line.
479
-
480
- ```ruby
481
- # timeout in seconds before a data line has to be completely sent by client or connection abort
482
- opts = { io_cmd_timeout: DEFAULT_IO_CMD_TIMEOUT }
483
-
484
- # maximum size in bytes to read in buffer for a complete data line from client or connection abort
485
- opts = { io_buffer_max_size: DEFAULT_IO_BUFFER_MAX_SIZE }
486
- ```
487
-
488
- There are new events `on_process_line_unknown_event` and `on_message_data_receiving_event` to handle the incoming transmission of unknown commands and message data.
489
-
490
- As an example to abort on to many unknown commands to prevent a denial of service attack etc.:
491
-
492
- ```ruby
493
- # event if process_line has identified an unknown command line
494
- def on_process_line_unknown_event(ctx, line)
495
- # check
496
- raise MidiSmtpServer::Smtpd421Exception.new("421 Abort: too many unknown commands where sent!") if ctx[:server][:exceptions] >= 5
497
- # otherwise call the super method
498
- super
499
- end
500
- ```
501
-
502
- As an example while receiving message data: abort when message data is going to exceed a maximum size:
503
-
504
- ```ruby
505
- # event while receiving message DATA
506
- def on_message_data_receiving_event(ctx)
507
- raise MidiSmtpServer::Smtpd552Exception if ctx[:message][:data].bytesize > MAX_MSG_SIZE
508
- end
509
- ```
510
-
511
- Or to implement something like a Teergrube for spammers etc.:
512
-
513
- ```ruby
514
- # event while receiving message DATA
515
- def on_message_data_receiving_event(ctx)
516
- # don't allow the spammer to continue fast
517
- # let him wait always 15 seconds before sending next data line
518
- sleep 15 if ctx[:server][:helo] =~ /domain/
519
- end
520
- ```
521
-
522
- Or to check already the message headers before receiving the complete message data. And lots more.
523
-
524
- <br>
525
-
526
-
527
- ## 8BITMIME and SMTPUTF8 support
528
-
529
- Since version 2.3.0 there is builtin optional internationalization support via SMTP 8BITMIME and SMTPUTF8 extension described in [RFC6152](https://tools.ietf.org/html/rfc6152) and [RFC6531](https://tools.ietf.org/html/rfc6531).
530
-
531
- The extensions are disabled by default and could be enabled by:
532
-
533
- ```ruby
534
- # enable internationalization SMTP extensions
535
- opts = { internationalization_extensions: true }
536
- ```
537
-
538
- When enabled and sender is using the 8BITMIME and SMTPUTF8 capabilities, the given enconding information about body and message encoding are set by `MAIL FROM` command. The encodings are read by MidiSmtpServer and published at context vars `ctx[:envelope][:encoding_body]` and `ctx[:envelope][:encoding_utf8]`.
539
-
540
- Possible values for `ctx[:envelope][:encoding_body]` are:
541
-
542
- 1. `""` (default, not set by client)
543
- 2. `"7bit"` (strictly 7bit)
544
- 3. `"8bitmime"` (strictly 8bit)
545
-
546
- Possible values for `ctx[:envelope][:encoding_utf8]` are:
547
-
548
- 1. `""` (default, not set by client)
549
- 2. `"utf8"` (utf-8 is enabled for headers and body)
550
-
551
- Even when `"8bitmime"` was set, you have to decide the correct encoding like `utf-8` or `iso-8859-1` etc. If also `"utf8"` was set, then encoding should be `utf-8`.
552
-
553
- <br>
554
-
555
-
556
- ## Authentication support
557
-
558
- There is built-in authentication support for `AUTH LOGIN` and `AUTH PLAIN` since release `2.1.0`. If you want to enable authentication you have to set the appropriate value to `auth_mode` opts.
559
-
560
- Allowed values are:
561
-
562
- ```ruby
563
- # no authentication is allowed (mostly for internal services)
564
- opts = { auth_mode: :AUTH_FORBIDDEN }
565
-
566
- # authentication is optional (you may grant higher possibilities if authenticated)
567
- opts = { auth_mode: :AUTH_OPTIONAL }
568
-
569
- # session must be authenticated before service may be used for mail transport
570
- opts = { auth_mode: :AUTH_REQUIRED }
571
- ```
572
-
573
- You may initialize your server class like:
574
-
575
- ```ruby
576
- server = MySmtpd.new(2525, '127.0.0.1', 4, auth_mode: :AUTH_REQUIRED)
577
- ```
578
-
579
- If you have enabled authentication you should provide your own user and access methods to grant access to your server. The default event method will deny all access per default.
580
-
581
- Your own server class should implement the `on_auth_event`:
582
-
583
- ```ruby
584
- # check the authentification
585
- # if any value returned, that will be used for ongoing processing
586
- # otherwise the original value will be used for authorization_id
587
- def on_auth_event(ctx, authorization_id, authentication_id, authentication)
588
- if authentication_id == "test" && authentication == "demo"
589
- return authentication_id
590
- else
591
- raise Smtpd535Exception
592
- end
593
- end
594
- ```
595
-
596
- Most of the time the `authorization_id` field will be empty. It allows optional (like described in [RFC 4954](http://www.ietf.org/rfc/rfc4954.txt)) to define an _authorization role_ which will be used, when the _authentication id_ has successfully entered. So the `authorization_id` is a request to become a role after authentication. In case that the `authorization_id` is empty it is supposed to be the same as the `authentication_id`.
597
-
598
- We suggest you to return the `authentication_id` on a successful auth event if you do not have special interests on other usage.
599
-
600
- <br>
601
-
602
-
603
- ## Authentication status in mixed mode
604
-
605
- If you have enabled optional authentication like described before, you may access helpers and values from context `ctx` while processing events to check the status of currents session authentication.
606
-
607
- ```ruby
608
- def on_rcpt_to_event(ctx, rcpt_to_data)
609
- # check if this session was authenticated already
610
- if authenticated?(ctx)
611
- # yes
612
- puts "Proceed with authorized id: #{ctx[:server][:authorization_id]}"
613
- puts "and authentication id: #{ctx[:server][:authentication_id]}"
614
- else
615
- # no
616
- puts "Proceed with anonymoous credentials"
617
- end
618
- end
619
- ```
620
-
621
- <br>
622
-
623
-
624
- ## Encryption
625
-
626
- Since release `2.2.1` the SMTP-Server supports STARTTLS by using `openssl` gem.
627
- If you want to enable encryption you have to set the appropriate value to `tls_mode` opts.
628
-
629
- Allowed values are:
630
-
631
- ```ruby
632
- # no encryption is allowed (mostly for internal services)
633
- opts = { tls_mode: :TLS_FORBIDDEN }
634
-
635
- # encryption is optional
636
- opts = { tls_mode: :TLS_OPTIONAL }
637
-
638
- # client must initialize encryption before service may be used for mail exchange
639
- opts = { tls_mode: :TLS_REQUIRED }
640
- ```
641
-
642
- You may enable TLS on your server class like:
643
-
644
- ```ruby
645
- server = MySmtpd.new(2525, '127.0.0.1', 4, tls_mode: :TLS_OPTIONAL)
646
- ```
647
-
648
- Do not forget to also install or require the `openssl` gem if you want to enable encryption.
649
-
650
- When using `tls_mode: :TLS_REQUIRED` your server will enforce the client to always use STARTTLS before accepting transmission of data like described in [RFC 3207](https://tools.ietf.org/html/rfc3207).
651
-
652
- For security reasons check the "Table of the ciphers (and their priorities)" on [OWASP Foundation](https://www.owasp.org/index.php/TLS_Cipher_String_Cheat_Sheet). Per default the `Advanced+ (A+)` cipher-string will be used as well as `TLSv1.2 only`.
653
-
654
- You may change ciphers and methods on your server class like:
655
-
656
- ```ruby
657
- server = MySmtpd.new(2525, '127.0.0.1', 4, { tls_mode: :TLS_OPTIONAL, tls_ciphers: TLS_CIPHERS_ADVANCED_PLUS, tls_methods: TLS_METHODS_ADVANCED })
658
- ```
659
-
660
- Predefined ciphers and methods strings are available as CONSTs:
661
-
662
- ```ruby
663
- # Advanced+ (A+) _Default_
664
- opts = { tls_ciphers: TLS_CIPHERS_ADVANCED_PLUS, tls_methods: TLS_METHODS_ADVANCED }
665
-
666
- # Advanced (A)
667
- opts = { tls_ciphers: TLS_CIPHERS_ADVANCED, tls_methods: TLS_METHODS_ADVANCED }
668
-
669
- # Broad Compatibility (B)
670
- opts = { tls_ciphers: TLS_CIPHERS_BROAD, tls_methods: TLS_METHODS_ADVANCED }
671
-
672
- # Widest Compatibility (C)
673
- opts = { tls_ciphers: TLS_CIPHERS_WIDEST, tls_methods: TLS_METHODS_LEGACY }
674
-
675
- # Legacy (C-)
676
- opts = { tls_ciphers: TLS_CIPHERS_LEGACY, tls_methods: TLS_METHODS_LEGACY }
677
- ```
678
-
679
- <br>
680
-
681
-
682
- ## Certificates
683
-
684
- As long as `tls_mode` is set to `:TLS_OPTIONAL` or `:TLS_REQUIRED` and no certificate or key path is given on class initialization, the internal TlsTransport class will create a certificate by itself. This should be only used for testing or debugging purposes and not in production environments. The memory only certificate is valid for 90 days from instantiating the class.
685
-
686
- To prevent client errors like `hostname does not match` the certificate is enriched by `subjectAltNames` and will include all hostnames and addresses which were identified on initialization. The automatic certificate subject and subjectAltName may also be manually set by `tls_cert_cn` and `tls_cert_san` parameter.
687
-
688
- In general and for production you better should generate a certificate by your own authority or use a professional trust-center like [LetsEncrypt](https://letsencrypt.org/) and more.
689
-
690
- #### Quick guide to create a certificate
691
-
692
- If interested in detail, read the whole story at [www.thenativeweb.io](https://www.thenativeweb.io/blog/2017-12-29-11-51-the-openssl-beginners-guide-to-creating-ssl-certificates/). Please check also the information about SSL-SAN like [support.dnsimple.com](https://support.dnsimple.com/articles/what-is-ssl-san/).
693
-
694
- ```bash
695
- # create a private key
696
- openssl genrsa -out key.pem 4096
697
- # create a certificate signing request (CSR)
698
- openssl req -new -key key.pem -out csr.pem
699
- # create a SSL certificate
700
- openssl x509 -in csr.pem -out cert.pem -req -signkey key.pem -days 90
701
- ```
702
-
703
- You may use your certificate and key on your server class like:
704
-
705
- ```ruby
706
- server = MySmtpd.new(2525, '127.0.0.1', 4, { tls_mode: :TLS_OPTIONAL, tls_cert_path: 'cert.pem', tls_key_path: 'key.pem' })
707
- ```
708
-
709
- <br>
710
-
97
+ ## Reliable code
711
98
 
712
- ## Test encrypted communication
99
+ Since version 2.3 implementation and integration tests by minitest framework are added to this repository. While the implementation tests are mostly checking the components, the integration tests try to verify the correct exchange of messages for different scenarios. In addtion all sources are checked by rubocop to ensure they fit to the style guides.
713
100
 
714
- While working with encrypted communication it is sometimes hard to test and check during development or debugging. Therefore you should look at the GNU tool `gnutls-cli`. Use this tool to connect to your running SMTP-server and proceed with encrypted communication.
101
+ You may run all rubocop tests through the `rake` helper:
715
102
 
716
- ```bash
717
- # use --insecure when using self created certificates
718
- gnutls-cli --insecure -s -p 2525 127.0.0.1
103
+ ``` bash
104
+ bundle exec rake rubocop
719
105
  ```
720
106
 
721
- After launching `gnutls-cli` start the SMTP dialog by sending `EHLO` and `STARTSSL` commands. Next press Ctrl-D on your keyboard to run the handshake for SSL communication between `gnutls-cli` and your server. When ready you may follow up with the delivery dialog for SMTP.
722
-
723
- <br>
724
-
725
-
726
- ## Attacks on email communication
727
-
728
- You should take care of your project and the communication which it will handle. At least there are a number of attack possibilities even against email communication. It is important to know some of the attacks to write safe codes. Here are just a few starting links about that:
729
-
730
- 1. [SMTP Injection via recipient (and sender) email addresses](https://www.mbsd.jp/Whitepaper/smtpi.pdf)
731
- 1. [Measuring E-Mail Header Injections on the World Wide Web](https://www.cs.ucsb.edu/~vigna/publications/2018_SAC_MailHeaderInjection.pdf)
732
- 1. [DDoS Protections for SMTP Servers](https://pdfs.semanticscholar.org/e942/d110f9686a438fccbac1d97db48c24ab84a7.pdf)
733
- 1. [Use timeouts to prevent SMTP DoS attacks](https://security.stackexchange.com/a/180267)
734
- 1. [Check HELO/EHLO arguments](https://serverfault.com/a/667555)
735
-
736
- Be aware that with enabled option of [PIPELINING](https://tools.ietf.org/html/rfc2920) you can't figure out sender or recipient address injection by the SMTP server. From point of security PIPELINING should be disabled as it is per default since version 2.3.0 on this component.
107
+ You may also run all tests through the `rake` helper:
737
108
 
738
- ```ruby
739
- # PIPELINING ist not allowed (false) per _Default_
740
- opts = { pipelining_extension: DEFAULT_PIPELINING_EXTENSION }
109
+ ``` bash
110
+ bundle exec rake test:all
741
111
  ```
742
112
 
743
- <br>
744
-
745
-
746
- ## RFC(2)822 - CR LF modes
747
-
748
- There is a difference between the conformity of RFC 2822 and best practise.
749
-
750
- In [RFC 2822](https://www.ietf.org/rfc/rfc2822.txt) it says that strictly each line has to end up by CR (code 13) followed by LF (code 10). And in addition that the chars CR (code 13) and LF (code 10) should not be used particulary. If looking on Qmails implementation, they will revoke any traffic which is not conform to the above per default.
751
-
752
- In real world, it is established, that also a line ending with single LF (code 10) is good practise. So if trying other mailservers like Exim or Exchange or Gmail, you may enter your message either ended by CRLF or single LF.
753
-
754
- Also the DATA ending sequence of CRLF.CRLF (CR LF DOT CR LF) may be send as LF.LF (LF DOT LF).
755
-
756
- Since version 2.3.0 the component allows to decide by option `crlf_mode` how to handle the line termination codes. Be aware that `CRLF_ENSURE` is enabled by default.
757
-
758
- ```ruby
759
- # Allow CRLF and LF but always make sure that CRLF is added to message data. _Default_
760
- opts = { crlf_mode: CRLF_ENSURE }
761
-
762
- # Allow CRLF and LF and do not change the incoming data.
763
- opts = { crlf_mode: CRLF_LEAVE }
113
+ or with more verbose output:
764
114
 
765
- # Only allow CRLF otherwise raise an exception
766
- opts = { crlf_mode: CRLF_STRICT }
115
+ ``` bash
116
+ bundle exec rake test:all v=1
767
117
  ```
768
118
 
769
- To understand the modes in details:
770
-
771
- #### CRLF_ENSURE
772
-
773
- 1. Read input buffer and search for LF (code 10)
774
- 2. Use bytes from buffer start to LF as TEXTLINE
775
- 3. Heal by deleting any occurence of char CR (code 13) and char LF (code 10) from TEXTLINE
776
- 4. Append cleaned TEXTLINE and RFC conform pair of CRLF to message data buffer
777
-
778
- * As result you will have a clean RFC 2822 conform message input data
779
- * In best case the data is 100% equal to the original input because that already was CRLF conform
780
- * Other input data maybe have changed for the linebreaks but the message is conform yet
781
-
782
- #### CRLF_LEAVE
783
-
784
- 1. Read input buffer and search for LF (code 10)
785
- 2. Use bytes from buffer start to LF as TEXTLINE
786
- 3. Append TEXTLINE as is to message data buffer
787
-
788
- * As result you may have a non clean RFC 2822 conform message input data
789
- * Other libraries like `Mail` may have parsing errors
790
-
791
- #### CRLF_STRICT
792
-
793
- 1. Read input buffer and search for CRLF (code 13 code 10)
794
- 2. Use bytes from buffer start to CRLF as TEXTLINE
795
- 3. Raise exception if TEXTLINE contains any single CR or LF
796
- 3. Append TEXTLINE as is to message data buffer
797
-
798
- * As result you will have a clean RFC 2822 conform message input data
799
- * The data is 100% equal to the original input because that already was CRLF conform
800
- * You maybe drop mails while in real world not all senders are working RFC conform
801
-
802
- <br>
803
-
804
-
805
- ## Reliable code
806
-
807
- Since version 2.3 implementation and integration tests by minitest framework are added to this repository. While the implementation tests are mostly checking the components, the integration tests try to verify the correct exchange of messages for different scenarios.
808
-
809
- You may run all tests through the `test_runner.rb` helper:
119
+ To just run just a part of the tests, you may select the `specs`, `unit` or `integration` tests:
810
120
 
811
121
  ``` bash
812
- ruby -I lib test/test_runner.rb
122
+ bundle exec rake test:specs
813
123
  ```
814
124
 
815
- or with more verbose output:
125
+ To just run some selected (by regular expression) tests, you may use the `T=filter` option. The example will run only the tests and specs containing the word _connections_ in their method_name or describe_text:
816
126
 
817
127
  ``` bash
818
- ruby -I lib test/test_runner.rb -v
128
+ bundle exec rake test:all v=1 T=connections
819
129
  ```
820
130
 
821
- To just run some selected (by regular expression) tests, you may use the `-n filter` option. The example will run only the tests and specs containing the word _connections_ in their method_name or describe_text:
131
+ _Be aware that the parameters and filter are case sensitive._
822
132
 
823
- ``` bash
824
- ruby -I lib test/test_runner.rb -v -n /connections/
825
- ```
133
+ #### Style guide links
826
134
 
827
- Be aware that the filter is case sensitive.
135
+ 1. [Ruby style guide](https://rubystyle.guide)
136
+ 2. [Minitest style guide](https://minitest.rubystyle.guide)
137
+ 3. [Rubocop/Cop documentation](https://docs.rubocop.org)
138
+ 4. [Rubocop/Minitest](https://docs.rubocop.org/rubocop-minitest/)
828
139
 
829
140
  <br>
830
141
 
831
142
 
832
143
  ## Changes and updates
833
144
 
834
- We suggest everybody using MidiSmtpServer 1.x or 2.x to switch at least to latest 2.3.y. The update is painless and without any source code changes if already using some 2.x release :sunglasses:
835
-
836
- For upgrades from version 1.x or from _Mini_SmtpServer you may follow the guides (see appendix) how to change your existing code to be compatible with the latest 2.x releases.
837
-
838
- #### 2.3.3 (2022-02-12)
839
-
840
- 1. Critical fix for thread safety ([check issue 39](https://github.com/4commerce-technologies-AG/midi-smtp-server/issues/39))
841
-
842
- #### 2.3.2 (2020-01-21)
843
-
844
- 1. New [hosts wildcard and interface detection](https://github.com/4commerce-technologies-AG/midi-smtp-server#hosts-hosts-wildcard-and-interface-detection)
845
- 2. Extended [Certificates](https://github.com/4commerce-technologies-AG/midi-smtp-server#certificates) with subjectAltName
846
- 3. Bound to ruby 2.3+
847
- 4. Full support for `# frozen_string_literal: true` optimization
848
- 5. Updated rubocop linter
849
- 6. Rich enhancements to tests
145
+ We suggest everybody using MidiSmtpServer to switch at least to latest 2.3.y. or best to 3.x. The update is painless and mostly without any source code changes :sunglasses:
850
146
 
147
+ For upgrades from previous versions or outdated _MiniSmtpServer_ gem you may follow the guides (see appendix) how to change your existing code to be compatible with the latest releases.
851
148
 
852
- #### 2.3.1 (2018-11-01)
149
+ #### Latest release: 3.0.3 (2022-02-12)
853
150
 
854
- 1. New [events for header inspection and addons](https://github.com/4commerce-technologies-AG/midi-smtp-server#adding-and-testing-headers)
855
- 2. New [MidiSmtpServer micro homepage](https://4commerce-technologies-ag.github.io/midi-smtp-server/)
856
- 3. New [ReadTheDocs manual](https://midi-smtp-server.readthedocs.io/)
857
- 4. New [Recipe for Slack MTA](https://midi-smtp-server.readthedocs.io/cookbook_recipe_slack_mta/)
151
+ 1. Critical fix for thread safety ([check issue 29](https://github.com/4commerce-technologies-AG/midi-smtp-server/issues/39))
152
+ 2. Fix tests using net/smtp '>= 0.3.1'
858
153
 
859
154
 
860
- #### 2.3.0 (2018-10-17)
155
+ #### Changelog history
861
156
 
862
- 1. Support [IPv4 and IPv6 (documentation)](https://github.com/4commerce-technologies-AG/midi-smtp-server#ipv4-and-ipv6-ready)
863
- 2. Support binding of [multiple ports and hosts / ip addresses](https://github.com/4commerce-technologies-AG/midi-smtp-server#multiple-ports-and-addresses)
864
- 3. Handle [utilization of connections and processings](https://github.com/4commerce-technologies-AG/midi-smtp-server#utilization-of-connections-and-processings)
865
- 4. Support of RFC(2)822 [CR LF modes](https://github.com/4commerce-technologies-AG/midi-smtp-server#rfc2822---cr-lf-modes)
866
- 5. Support (optionally) SMTP [PIPELINING](https://tools.ietf.org/html/rfc2920) extension
867
- 6. Support (optionally) SMTP [8BITMIME](https://github.com/4commerce-technologies-AG/midi-smtp-server#8bitmime-and-smtputf8-support) extension
868
- 7. Support (optionally) SMTP [SMTPUTF8](https://github.com/4commerce-technologies-AG/midi-smtp-server#8bitmime-and-smtputf8-support) extension
869
- 8. SMTP PIPELINING, 8BITMIME and SMTPUTF8 extensions are _disabled_ by default
870
- 9. Support modification of local welcome and greeting messages
871
- 10. Documentation and Links about security and [email attacks](https://github.com/4commerce-technologies-AG/midi-smtp-server#attacks-on-email-communication)
872
- 11. Added [implementation and integration testing](https://github.com/4commerce-technologies-AG/midi-smtp-server#reliable-code)
873
-
874
-
875
- #### 2.2.3
876
-
877
- 1. Control and validation on incoming data [see Incoming data validation](https://github.com/4commerce-technologies-AG/midi-smtp-server#incoming-data-validation)
878
-
879
-
880
- #### 2.2.1
881
-
882
- 1. Builtin optional support of STARTTLS encryption
883
- 2. Added examples for a simple midi-smtp-server with TLS support
884
-
885
-
886
- #### 2.2.x
887
-
888
- 1. Rubocop configuration and passed source code verification
889
- 2. Modified examples for a simple midi-smtp-server with and without auth
890
- 3. Enhanced `serve_service` (previously `start`)
891
- 4. Optionally gracefully shutdown when service `stop` (default gracefully)
892
-
893
-
894
- #### 2.1.1
895
-
896
- 1. Huge speed improvement on receiving large message data (1.000+ faster)
897
-
898
-
899
- #### 2.1.0
900
-
901
- 1. Authentication PLAIN, LOGIN
902
- 2. Safe `join` will catch and rescue `Interrupt`
903
-
904
-
905
- #### 2.x
906
-
907
- 1. Modulized
908
- 2. Removed dependency to GServer
909
- 3. Additional events to interact with
910
- 4. Use logger to log several messages from severity :debug up to :fatal
157
+ A complete list of updates and features can be read in the [CHANGELOG](https://github.com/4commerce-technologies-AG/midi-smtp-server/blob/master/CHANGELOG.md).
911
158
 
912
159
  <br>
913
160
 
914
161
 
915
- ## Upgrade to 2.x
916
-
917
- If you are already using MidiSmtpServer it might be only some straight forward work to get your code ready for MidiSmtpServer version 2.x. Also if you are a _Mini_SmtpServer user, it should request only some few work on your codes.
918
-
919
-
920
- #### Upgrade from 1.x
921
-
922
- <details>
923
- <summary>Open / Close details</summary>
924
-
925
- #### Class
926
-
927
- ##### 1.x
928
-
929
- ```ruby
930
- MidiSmtpServer.new
931
- ```
932
-
933
- ##### 2.x
934
-
935
- ```ruby
936
- MidiSmtpServer::Smtpd.new
937
- ```
938
-
939
- #### Class initialize
940
-
941
- ##### 1.x
942
-
943
- ```ruby
944
- def initialize(port = 2525, host = "127.0.0.1", max_connections = 4, do_smtp_server_reverse_lookup = true, *args)
945
- ```
946
-
947
- ##### 2.x
948
-
949
- ```ruby
950
- def initialize(ports = DEFAULT_SMTPD_PORT, hosts = DEFAULT_SMTPD_HOST, max_connections = 4, opts = {})
951
- # opts may include
952
- opts = { do_dns_reverse_lookup: true }
953
- opts = { logger: myLoggerObject }
954
- ```
955
-
956
- #### On_event arguments order
957
-
958
- ##### 1.x
959
-
960
- ```ruby
961
- def on_helo_event(helo_data, ctx)
962
- def on_mail_from_event(mail_from_data, ctx)
963
- def on_rcpt_to_event(rcpt_to_data, ctx)
964
- ```
965
-
966
- ##### 2.x
967
-
968
- ```ruby
969
- def on_helo_event(ctx, helo_data)
970
- def on_mail_from_event(ctx, mail_from_data)
971
- def on_rcpt_to_event(ctx, rcpt_to_data)
972
- ```
973
-
974
- #### Exceptions
975
-
976
- ##### 1.x
977
-
978
- ```ruby
979
- MidiSmtpServerException
980
- MidiSmtpServer???Exception
981
- ```
982
-
983
- ##### 2.x
984
-
985
- ```ruby
986
- MidiSmtpServer::SmtpdException
987
- MidiSmtpServer::Smtpd???Exception
988
- ```
989
-
990
- #### Removed elements
991
-
992
- ##### 1.x
993
-
994
- ```ruby
995
- # class vars from gserver
996
- audit
997
- debug
998
- ```
999
-
1000
- ##### 2.x
1001
-
1002
- ```ruby
1003
- # not available anymore, is now controlled by Logger
1004
- ```
1005
- </details>
1006
-
1007
-
1008
- #### Upgrade from MiniSmtpServer
1009
-
1010
- <details>
1011
- <summary>Open / Close details</summary>
1012
-
1013
- #### Class
1014
-
1015
- ##### MiniSmtpServer
1016
-
1017
- ```ruby
1018
- MiniSmtpServer.new
1019
- ```
1020
-
1021
- ##### MidiSmtpServer
1022
-
1023
- ```ruby
1024
- MidiSmtpServer::Smtpd.new
1025
- ```
1026
-
1027
- #### Class initialize
162
+ ## Upgrading from previous releases :small_red_triangle:
1028
163
 
1029
- ##### MiniSmtpServer
1030
-
1031
- ```ruby
1032
- def initialize(port = 2525, host = "127.0.0.1", max_connections = 4, *args)
1033
- ```
1034
-
1035
- ##### MidiSmtpServer
1036
-
1037
- ```ruby
1038
- def initialize(ports = DEFAULT_SMTPD_PORT, hosts = DEFAULT_SMTPD_HOST, max_connections = 4, opts = {})
1039
- # opts may include
1040
- opts = { do_dns_reverse_lookup: true }
1041
- opts = { logger: myLoggerObject }
1042
- ```
1043
-
1044
- #### On_event methods
1045
-
1046
- ##### MiniSmtpServer
1047
-
1048
- ```ruby
1049
- def new_message_event(message_hash)
1050
- # message_hash[:from]
1051
- # message_hash[:to]
1052
- # message_hash[:data]
1053
- ```
1054
-
1055
- ##### MidiSmtpServer
1056
-
1057
- ```ruby
1058
- def on_message_data_event(ctx)
1059
- ctx[:envelope][:from]
1060
- ctx[:envelope][:to]
1061
- ctx[:message][:data]
1062
- ```
1063
-
1064
- #### Removed elements
1065
-
1066
- ##### MiniSmtpServer
1067
-
1068
- ```ruby
1069
- # class vars from gserver
1070
- audit
1071
- debug
1072
- ```
1073
-
1074
- ##### MidiSmtpServer
1075
-
1076
- ```ruby
1077
- # not available anymore, is now controlled by Logger
1078
- ```
1079
- </details>
164
+ Checkout the [Appendix Upgrade](https://midi-smtp-server.readthedocs.io/appendix_upgrade/) to get your code ready for the latest releases and read about any incompatibilities.
1080
165
 
1081
166
  <br>
1082
167
 
@@ -1095,7 +180,7 @@ You may find, use and download the gem package on [RubyGems.org](http://rubygems
1095
180
 
1096
181
  **[Class documentation](http://www.rubydoc.info/gems/midi-smtp-server/MidiSmtpServer/Smtpd)** - you will find a detailed description at [RubyDoc](http://www.rubydoc.info/gems/midi-smtp-server/MidiSmtpServer/Smtpd)
1097
182
 
1098
- **[Library manual](https://midi-smtp-server.readthedocs.io/)** - you will find a manual (in progress) at [ReadTheDocs](https://midi-smtp-server.readthedocs.io/)
183
+ **[Library manual](https://midi-smtp-server.readthedocs.io/)** - you will find a manual at [ReadTheDocs](https://midi-smtp-server.readthedocs.io/)
1099
184
 
1100
185
  <br>
1101
186
 
@@ -1106,4 +191,4 @@ Author: [Tom Freudenberg](http://about.me/tom.freudenberg)
1106
191
 
1107
192
  [MidiSmtpServer Class](https://github.com/4commerce-technologies-AG/midi-smtp-server/) is inspired from [MiniSmtpServer Class](https://github.com/aarongough/mini-smtp-server) and code written by [Aaron Gough](https://github.com/aarongough) and [Peter Cooper](http://peterc.org/)
1108
193
 
1109
- Copyright (c) 2014-2018 [Tom Freudenberg](http://www.4commerce.de/), [4commerce technologies AG](http://www.4commerce.de/), released under the MIT license
194
+ Copyright (c) 2014-2022 [Tom Freudenberg](http://www.4commerce.de/), [4commerce technologies AG](http://www.4commerce.de/), released under the MIT license