macaw_framework 1.0.4 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +11 -0
- data/CONTRIBUTING.md +30 -0
- data/README.md +9 -1
- data/lib/macaw_framework/aspects/logging_aspect.rb +2 -0
- data/lib/macaw_framework/core/server.rb +71 -17
- data/lib/macaw_framework/data_filters/log_data_filter.rb +1 -0
- data/lib/macaw_framework/version.rb +1 -1
- data/lib/macaw_framework.rb +24 -11
- data/sig/macaw_framework/macaw.rbs +2 -2
- data/sig/server.rbs +1 -1
- metadata +3 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 23fe3e98800399fc740bf5a76603fab28e53502d8f5041d894ecac5bef96f00c
|
4
|
+
data.tar.gz: 5b8f46d645b2e274d005368e9f144bca0b63eac552942ea3176172a1479ac4a4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 1757594571b01f8110c8b839c906a63488aa25120adc1b96f7a87066cbaea0473d0628398d9d134e4950d3b6dcf19ddd07bc806620608f2214404aee0950587e
|
7
|
+
data.tar.gz: 8326f4957ba9d8baa95603d5c4c4300339e6e4a4d6797fc34945e88d9e07dc9e102ac0ba5a0dbce90ea581f8ab7f5fc4e8d8f22c8cf6e0d2013863f2f634a0b2
|
data/CHANGELOG.md
CHANGED
@@ -63,3 +63,14 @@
|
|
63
63
|
## [1.0.4] - 2023-05-11
|
64
64
|
|
65
65
|
- Fixing issue with response body returning always a blank line at the beginning
|
66
|
+
|
67
|
+
## [1.0.5] - 2023-05-12
|
68
|
+
|
69
|
+
- Fixing critical bug where threads were being killed and not respawning after abrupt client connection shutdown
|
70
|
+
|
71
|
+
## [1.1.0] - 2023-xx-xx
|
72
|
+
|
73
|
+
- Adding support for other SSL/TSL keys other than RSA
|
74
|
+
- New mechanism to handle server shutdown properly
|
75
|
+
- Improving log readability
|
76
|
+
- Automatic logging is now optional
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,30 @@
|
|
1
|
+
# Contributing to Macaw Framework
|
2
|
+
|
3
|
+
First off, thank you for considering contributing to Macaw Framework.
|
4
|
+
|
5
|
+
## Getting Started
|
6
|
+
|
7
|
+
- Submit an issue for your problem or suggestion, assuming one does not already exist.
|
8
|
+
- Clearly describe the issue including steps to reproduce when it is a bug.
|
9
|
+
- Make sure you fill in the earliest version that you know has the issue.
|
10
|
+
|
11
|
+
## Making Changes
|
12
|
+
|
13
|
+
- Fork the repository on GitHub.
|
14
|
+
- Create a topic branch from where you want to base your work. This is usually the main branch.
|
15
|
+
- To quickly create a topic branch based on main; `git checkout -b fix/main/my_contribution main`. Please avoid working directly on the `main` branch.
|
16
|
+
- Make commits of logical units.
|
17
|
+
- Check for unnecessary whitespace with `git diff --check` before committing.
|
18
|
+
- Make sure your commit messages are in the proper format.
|
19
|
+
- Make sure you have added the necessary tests for your changes.
|
20
|
+
- Run _all_ the tests to assure nothing else was accidentally broken.
|
21
|
+
|
22
|
+
## Submitting Changes
|
23
|
+
|
24
|
+
- Run RuboCop to ensure your code adheres to our code style conventions. You can do this by running `rubocop` in your terminal.
|
25
|
+
- Push your changes to a topic branch in your fork of the repository.
|
26
|
+
- Submit a pull request to the repository in my GitHub account.
|
27
|
+
- Our automatic CI/CD pipeline will run all tests and lint for 3 different ruby versions before merges.
|
28
|
+
- I'm constantly reviewing Pull Requests and will provide feedback as soon as possible.
|
29
|
+
|
30
|
+
Thanks for contributing!
|
data/README.md
CHANGED
@@ -108,6 +108,7 @@ end
|
|
108
108
|
"ssl": {
|
109
109
|
"min": "SSL3",
|
110
110
|
"max": "TLS1.3",
|
111
|
+
"key_type": "EC",
|
111
112
|
"cert_file_name": "path/to/cert/file/file.crt",
|
112
113
|
"key_file_name": "path/to/cert/key/file.key"
|
113
114
|
}
|
@@ -123,6 +124,12 @@ curl http://localhost:8080/metrics
|
|
123
124
|
|
124
125
|
### Tips
|
125
126
|
|
127
|
+
The automatic logging and log aspect are now optional. To disable them, simply start Macaw with `custom_log` set to nil.
|
128
|
+
|
129
|
+
```ruby
|
130
|
+
MacawFramework::Macaw.new(custom_log: nil)
|
131
|
+
```
|
132
|
+
|
126
133
|
Cache invalidation time should be specified in seconds. In order to enable caching, The application.json file
|
127
134
|
should exist in the app main directory and it need the `cache_invalidation` config set. It is possible to
|
128
135
|
provide a list of strings in the property `ignore_headers`. All the client headers with the same name of any
|
@@ -143,7 +150,8 @@ exists inside `application.json`.
|
|
143
150
|
If the SSL configuration is provided in the `application.json` file with valid certificate and key files, the TCP server
|
144
151
|
will be wrapped with HTTPS security using the provided certificate.
|
145
152
|
|
146
|
-
The supported values for `min` and `max` in the SSL configuration are: `SSL2`, `SSL3`, `TLS1.1`, `TLS1.2` and `TLS1.3
|
153
|
+
The supported values for `min` and `max` in the SSL configuration are: `SSL2`, `SSL3`, `TLS1.1`, `TLS1.2` and `TLS1.3`,
|
154
|
+
and the supported values for `key_type` are `RSA` and `EC`.
|
147
155
|
|
148
156
|
If prometheus is enabled, a get endpoint will be defined at path `/metrics` to collect prometheus metrics. This path
|
149
157
|
is configurable via the `application.json` file.
|
@@ -9,6 +9,8 @@ require_relative "../data_filters/log_data_filter"
|
|
9
9
|
# in the framework.
|
10
10
|
module LoggingAspect
|
11
11
|
def call_endpoint(logger, *args)
|
12
|
+
return super(*args) if logger.nil?
|
13
|
+
|
12
14
|
endpoint_name = args[1].split(".")[1..].join("/")
|
13
15
|
logger.info(LogDataFilter.sanitize_for_logging(
|
14
16
|
"Request received for #{endpoint_name} with arguments: #{args[2..]}"
|
@@ -55,21 +55,22 @@ class Server
|
|
55
55
|
def run
|
56
56
|
@server = TCPServer.new(@bind, @port)
|
57
57
|
@server = OpenSSL::SSL::SSLServer.new(@server, @context) if @context
|
58
|
+
@workers_mutex = Mutex.new
|
58
59
|
@num_threads.times do
|
59
|
-
|
60
|
-
|
61
|
-
client = @work_queue.pop
|
62
|
-
break if client == :shutdown
|
60
|
+
spawn_worker
|
61
|
+
end
|
63
62
|
|
64
|
-
|
65
|
-
|
63
|
+
Thread.new do
|
64
|
+
loop do
|
65
|
+
sleep 10
|
66
|
+
maintain_worker_pool
|
66
67
|
end
|
67
68
|
end
|
68
69
|
|
69
70
|
loop do
|
70
71
|
@work_queue << @server.accept
|
71
72
|
rescue OpenSSL::SSL::SSLError => e
|
72
|
-
@macaw_log
|
73
|
+
@macaw_log&.error("SSL error: #{e.message}")
|
73
74
|
rescue IOError, Errno::EBADF
|
74
75
|
break
|
75
76
|
end
|
@@ -78,9 +79,7 @@ class Server
|
|
78
79
|
##
|
79
80
|
# Method Responsible for closing the TCP server.
|
80
81
|
def close
|
81
|
-
|
82
|
-
@num_threads.times { @work_queue << :shutdown }
|
83
|
-
@workers.each(&:join)
|
82
|
+
shutdown
|
84
83
|
end
|
85
84
|
|
86
85
|
private
|
@@ -93,22 +92,28 @@ class Server
|
|
93
92
|
declare_client_session(client)
|
94
93
|
client_data = get_client_data(body, headers, parameters)
|
95
94
|
|
96
|
-
@macaw_log
|
95
|
+
@macaw_log&.info("Running #{path.gsub("\n", "").gsub("\r", "")}")
|
97
96
|
message, status, response_headers = call_endpoint(@prometheus_middleware, @macaw_log, @cache,
|
98
97
|
method_name, client_data, client.peeraddr[3])
|
99
98
|
status ||= 200
|
100
99
|
message ||= nil
|
101
100
|
response_headers ||= nil
|
102
101
|
client.puts ResponseDataFilter.mount_response(status, response_headers, message)
|
102
|
+
rescue IOError, Errno::EPIPE => e
|
103
|
+
@macaw_log&.error("Error writing to client: #{e.message}")
|
103
104
|
rescue TooManyRequestsError
|
104
105
|
client.print "HTTP/1.1 429 Too Many Requests\r\n\r\n"
|
105
106
|
rescue EndpointNotMappedError
|
106
107
|
client.print "HTTP/1.1 404 Not Found\r\n\r\n"
|
107
108
|
rescue StandardError => e
|
108
109
|
client.print "HTTP/1.1 500 Internal Server Error\r\n\r\n"
|
109
|
-
@macaw_log
|
110
|
+
@macaw_log&.info("Error: #{e}")
|
110
111
|
ensure
|
111
|
-
|
112
|
+
begin
|
113
|
+
client.close
|
114
|
+
rescue IOError => e
|
115
|
+
@macaw_log&.error("Error closing client: #{e.message}")
|
116
|
+
end
|
112
117
|
end
|
113
118
|
|
114
119
|
def declare_client_session(client)
|
@@ -140,13 +145,20 @@ class Server
|
|
140
145
|
@context.min_version = SupportedSSLVersions::VERSIONS[version_config[:min]] unless version_config[:min].nil?
|
141
146
|
@context.max_version = SupportedSSLVersions::VERSIONS[version_config[:max]] unless version_config[:max].nil?
|
142
147
|
@context.cert = OpenSSL::X509::Certificate.new(File.read(ssl_config["cert_file_name"]))
|
143
|
-
|
148
|
+
|
149
|
+
if ssl_config["key_type"] == "RSA" || ssl_config["key_type"].nil?
|
150
|
+
@context.key = OpenSSL::PKey::RSA.new(File.read(ssl_config["key_file_name"]))
|
151
|
+
elsif ssl_config["key_type"] == "EC"
|
152
|
+
@context.key = OpenSSL::PKey::EC.new(File.read(ssl_config["key_file_name"]))
|
153
|
+
else
|
154
|
+
raise ArgumentError, "Unsupported SSL/TLS key type: #{ssl_config["key_type"]}"
|
155
|
+
end
|
144
156
|
end
|
145
157
|
@context ||= nil
|
146
158
|
rescue IOError => e
|
147
|
-
@macaw_log
|
148
|
-
#{@macaw.config["macaw"]["ssl"]["key_file_name"]}. Please assure the files
|
149
|
-
@macaw_log
|
159
|
+
@macaw_log&.error("It was not possible to read files #{@macaw.config["macaw"]["ssl"]["cert_file_name"]} and
|
160
|
+
#{@macaw.config["macaw"]["ssl"]["key_file_name"]}. Please assure the files exist and their names are correct.")
|
161
|
+
@macaw_log&.error(e.backtrace)
|
150
162
|
raise e
|
151
163
|
end
|
152
164
|
|
@@ -161,6 +173,7 @@ class Server
|
|
161
173
|
end
|
162
174
|
|
163
175
|
def set_features
|
176
|
+
@is_shutting_down = false
|
164
177
|
set_rate_limiting
|
165
178
|
set_session
|
166
179
|
set_ssl
|
@@ -181,4 +194,45 @@ class Server
|
|
181
194
|
def get_client_data(body, headers, parameters)
|
182
195
|
{ body: body, headers: headers, params: parameters }
|
183
196
|
end
|
197
|
+
|
198
|
+
def spawn_worker
|
199
|
+
@workers_mutex.synchronize do
|
200
|
+
@workers << Thread.new do
|
201
|
+
loop do
|
202
|
+
client = @work_queue.pop
|
203
|
+
break if client == :shutdown
|
204
|
+
|
205
|
+
handle_client(client)
|
206
|
+
end
|
207
|
+
end
|
208
|
+
end
|
209
|
+
end
|
210
|
+
|
211
|
+
def maintain_worker_pool
|
212
|
+
@workers_mutex.synchronize do
|
213
|
+
@workers.each_with_index do |worker, index|
|
214
|
+
unless worker.alive?
|
215
|
+
if @is_shutting_down
|
216
|
+
@macaw_log&.info("Worker thread #{index} finished, not respawning due to server shutdown.")
|
217
|
+
else
|
218
|
+
@macaw_log&.error("Worker thread #{index} died, respawning...")
|
219
|
+
@workers[index] = spawn_worker
|
220
|
+
end
|
221
|
+
end
|
222
|
+
end
|
223
|
+
end
|
224
|
+
end
|
225
|
+
|
226
|
+
def shutdown
|
227
|
+
@is_shutting_down = true
|
228
|
+
loop do
|
229
|
+
break if @work_queue.empty?
|
230
|
+
|
231
|
+
sleep 0.1
|
232
|
+
end
|
233
|
+
|
234
|
+
@num_threads.times { @work_queue << :shutdown }
|
235
|
+
@workers.each(&:join)
|
236
|
+
@server.close
|
237
|
+
end
|
184
238
|
end
|
data/lib/macaw_framework.rb
CHANGED
@@ -22,10 +22,10 @@ module MacawFramework
|
|
22
22
|
|
23
23
|
##
|
24
24
|
# @param {Logger} custom_log
|
25
|
-
def initialize(custom_log:
|
25
|
+
def initialize(custom_log: Logger.new($stdout), server: Server)
|
26
26
|
begin
|
27
27
|
@routes = []
|
28
|
-
@macaw_log ||= custom_log
|
28
|
+
@macaw_log ||= custom_log
|
29
29
|
@config = JSON.parse(File.read("application.json"))
|
30
30
|
@port = @config["macaw"]["port"] || 8080
|
31
31
|
@bind = @config["macaw"]["bind"] || "localhost"
|
@@ -37,7 +37,7 @@ module MacawFramework
|
|
37
37
|
@prometheus_middleware = PrometheusMiddleware.new if @config["macaw"]["prometheus"]
|
38
38
|
@prometheus_middleware.configure_prometheus(@prometheus, @config, self) if @config["macaw"]["prometheus"]
|
39
39
|
rescue StandardError => e
|
40
|
-
@macaw_log
|
40
|
+
@macaw_log&.warn(e.message)
|
41
41
|
end
|
42
42
|
@port ||= 8080
|
43
43
|
@bind ||= "localhost"
|
@@ -103,15 +103,28 @@ module MacawFramework
|
|
103
103
|
##
|
104
104
|
# Starts the web server
|
105
105
|
def start!
|
106
|
-
@macaw_log.
|
107
|
-
|
108
|
-
|
109
|
-
|
106
|
+
if @macaw_log.nil?
|
107
|
+
puts("---------------------------------")
|
108
|
+
puts("Starting server at port #{@port}")
|
109
|
+
puts("Number of threads: #{@threads}")
|
110
|
+
puts("---------------------------------")
|
111
|
+
else
|
112
|
+
@macaw_log.info("---------------------------------")
|
113
|
+
@macaw_log.info("Starting server at port #{@port}")
|
114
|
+
@macaw_log.info("Number of threads: #{@threads}")
|
115
|
+
@macaw_log.info("---------------------------------")
|
116
|
+
end
|
110
117
|
server_loop(@server)
|
111
118
|
rescue Interrupt
|
112
|
-
@macaw_log.
|
113
|
-
|
114
|
-
|
119
|
+
if @macaw_log.nil?
|
120
|
+
puts("Stopping server")
|
121
|
+
@server.close
|
122
|
+
puts("Macaw stop flying for some seeds...")
|
123
|
+
else
|
124
|
+
@macaw_log.info("Stopping server")
|
125
|
+
@server.close
|
126
|
+
@macaw_log.info("Macaw stop flying for some seeds...")
|
127
|
+
end
|
115
128
|
end
|
116
129
|
|
117
130
|
private
|
@@ -123,7 +136,7 @@ module MacawFramework
|
|
123
136
|
def map_new_endpoint(prefix, cache, path, &block)
|
124
137
|
@endpoints_to_cache << "#{prefix}.#{RequestDataFiltering.sanitize_method_name(path)}" if cache
|
125
138
|
path_clean = RequestDataFiltering.extract_path(path)
|
126
|
-
@macaw_log
|
139
|
+
@macaw_log&.info("Defining #{prefix.upcase} endpoint at /#{path}")
|
127
140
|
define_singleton_method("#{prefix}.#{path_clean}", block || lambda {
|
128
141
|
|context = { headers: {}, body: "", params: {} }|
|
129
142
|
})
|
@@ -4,7 +4,7 @@ module MacawFramework
|
|
4
4
|
@cache: untyped
|
5
5
|
@config: Hash[String, untyped]
|
6
6
|
@endpoints_to_cache: Array[String]
|
7
|
-
@macaw_log: Logger
|
7
|
+
@macaw_log: Logger?
|
8
8
|
|
9
9
|
@prometheus: untyped
|
10
10
|
@prometheus_middleware: untyped
|
@@ -14,7 +14,7 @@ module MacawFramework
|
|
14
14
|
|
15
15
|
attr_reader bind: String
|
16
16
|
attr_reader config: Hash[String, untyped]
|
17
|
-
attr_reader macaw_log: Logger
|
17
|
+
attr_reader macaw_log: Logger?
|
18
18
|
attr_reader port: Integer
|
19
19
|
attr_reader routes: Array[String]
|
20
20
|
|
data/sig/server.rbs
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: macaw_framework
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Aria Diniz
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-05-
|
11
|
+
date: 2023-05-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: prometheus-client
|
@@ -36,6 +36,7 @@ files:
|
|
36
36
|
- ".rubocop.yml"
|
37
37
|
- CHANGELOG.md
|
38
38
|
- CODE_OF_CONDUCT.md
|
39
|
+
- CONTRIBUTING.md
|
39
40
|
- Gemfile
|
40
41
|
- LICENSE.txt
|
41
42
|
- README.md
|