marty 9.5.1 → 10.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +2 -0
- data/README.md +40 -31
- data/app/components/marty/background_job/schedule_jobs_dashboard.rb +2 -2
- data/app/components/marty/background_job/schedule_jobs_grid.rb +22 -5
- data/app/components/marty/background_job/schedule_jobs_logs.rb +12 -0
- data/app/components/marty/base_rule_view.rb +0 -4
- data/app/components/marty/delorean_rule_view.rb +11 -5
- data/app/components/marty/promise_view.rb +15 -0
- data/app/jobs/marty/cron_job.rb +30 -28
- data/app/jobs/marty/delorean_background_job.rb +8 -0
- data/app/models/marty/background_job/schedule.rb +24 -1
- data/app/models/marty/delorean_rule.rb +1 -1
- data/app/models/marty/vw_promise.rb +6 -4
- data/app/services/marty/background_job/fetch_missing_in_schedule_cron_jobs.rb +8 -6
- data/app/services/marty/background_job/update_schedule.rb +15 -8
- data/app/services/marty/jobs/schedule.rb +8 -4
- data/app/services/marty/promises/delorean/create.rb +2 -1
- data/app/services/marty/promises/ruby/create.rb +2 -1
- data/config/locales/en.yml +1 -0
- data/db/migrate/510_schedule_job_to_remove_old_promises.rb +4 -2
- data/db/migrate/523_add_timeout_to_promises.rb +5 -0
- data/db/migrate/524_add_timeout_to_promise_view.rb +45 -0
- data/db/migrate/525_add_arguments_to_jobs_schedules.rb +10 -0
- data/db/migrate/526_add_schedule_id_to_delayed_jobs.rb +14 -0
- data/lib/marty/delayed_job/queue_adapter.rb +38 -0
- data/lib/marty/delayed_job/scheduled_job_plugin.rb +9 -6
- data/lib/marty/diagnostic/environment_variables.rb +1 -1
- data/lib/marty/monkey.rb +11 -0
- data/lib/marty/promise_job.rb +2 -1
- data/lib/marty/promise_ruby_job.rb +2 -1
- data/lib/marty/version.rb +1 -1
- data/spec/dummy/.foreman +2 -0
- data/spec/dummy/Procfile +2 -0
- data/spec/dummy/app/jobs/test_failing_job.rb +5 -1
- data/spec/dummy/app/models/gemini/my_rule.rb +2 -0
- data/spec/dummy/app/models/gemini/xyz_rule.rb +2 -0
- data/spec/dummy/delorean/jobs.dl +6 -0
- data/spec/features/delayed_jobs_grid_spec.rb +3 -2
- data/spec/features/schedule_jobs_dashboard_spec.rb +12 -12
- data/spec/jobs/cron_job_spec.rb +16 -12
- data/spec/jobs/delorean_background_job_spec.rb +50 -0
- data/spec/models/promise_spec.rb +1 -0
- data/spec/services/background_job/fetch_missing_in_schedule_cron_jobs_spec.rb +5 -3
- data/spec/services/jobs/schedule_spec.rb +5 -4
- metadata +12 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f9a92073fa74d9777548dbafce731dbb8100918a4b3e696cdf00b06b1b20bbd8
|
4
|
+
data.tar.gz: b2b048bd619a886a1ebafbfaba5e73f017c44fdaaa1f55190ade7d7abb32cea7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 82e24b64bee5533356e3c4a0f6d22bb850d9fe24fd300e9d921f98beb7193b119ccc9a31ba11ee766bb0ade94d2fa8a961318b828805ea611dbfedc375c61503
|
7
|
+
data.tar.gz: 76d6757d910bc340d53a4930f846f698a9fa855c8827ec8fff1bd9a82c21346d55f703c41fc5d0f4f7f8871593282125ad628ced7610eee743518a15858dfc49
|
data/.rubocop.yml
CHANGED
data/README.md
CHANGED
@@ -18,44 +18,45 @@ and delorean scripts.
|
|
18
18
|
To create the correct migrations for a Marty-based application (see below for
|
19
19
|
getting the internal dummy application to work):
|
20
20
|
|
21
|
-
```
|
22
|
-
|
21
|
+
```bash
|
22
|
+
rake marty:install:migrations
|
23
23
|
```
|
24
24
|
|
25
25
|
The Marty database needs to be seeded with specific configuration
|
26
26
|
information. Before running your Marty application for the first time you will
|
27
27
|
need to run:
|
28
28
|
|
29
|
-
```
|
30
|
-
|
29
|
+
```bash
|
30
|
+
rake marty:seed
|
31
31
|
```
|
32
32
|
|
33
33
|
If you are using Delorean scripts in your application you can load them
|
34
34
|
with a rake task. By default these scripts will be picked up from
|
35
35
|
`app/delorean_scripts`. To load scripts:
|
36
36
|
|
37
|
-
```
|
38
|
-
|
37
|
+
```bash
|
38
|
+
rake marty:load_scripts
|
39
39
|
```
|
40
40
|
|
41
41
|
You can override the default directory by setting the `LOAD_DIR` environment
|
42
42
|
variable:
|
43
43
|
|
44
|
-
```
|
45
|
-
|
44
|
+
```bash
|
45
|
+
LOAD_DIR=<delorean script directory> marty:load_scripts
|
46
46
|
```
|
47
47
|
|
48
48
|
To delete scripts:
|
49
49
|
|
50
|
+
``` bash
|
51
|
+
rake marty:delete_scripts
|
50
52
|
```
|
51
|
-
|
52
|
-
```
|
53
|
+
|
53
54
|
# Scheduled Job
|
54
55
|
|
55
|
-
To use scheduled
|
56
|
+
To use scheduled background jobs, add to `config/application.rb`:
|
56
57
|
|
57
|
-
```
|
58
|
-
|
58
|
+
```bash
|
59
|
+
config.active_job.queue_adapter = :delayed_job
|
59
60
|
```
|
60
61
|
|
61
62
|
# Dummy Application & Testing
|
@@ -66,45 +67,53 @@ dummy app at spec/dummy/public.
|
|
66
67
|
Docker doesn't support symlinks, so in order to run it in Docker you will have to copy extjs files.
|
67
68
|
|
68
69
|
```bash
|
69
|
-
|
70
|
+
cp -r PATH/TO/YOUR/EXTJS/FILES spec/dummy/public/extjs
|
70
71
|
```
|
71
72
|
|
72
|
-
You can run it with
|
73
|
+
You can run it with Docker:
|
73
74
|
|
74
75
|
```bash
|
75
|
-
|
76
|
-
|
77
|
-
$ make app-start
|
76
|
+
make app-initialise-docker
|
77
|
+
make app-start
|
78
78
|
```
|
79
79
|
|
80
80
|
To run tests:
|
81
81
|
|
82
82
|
```bash
|
83
|
-
|
84
|
-
|
85
|
-
$ HEADLESS=true rspec
|
83
|
+
make app-bash
|
84
|
+
HEADLESS=true rspec
|
86
85
|
```
|
87
86
|
|
88
|
-
To run without
|
87
|
+
To run without Docker:
|
89
88
|
|
90
|
-
Marty currently only runs with
|
89
|
+
Marty currently only runs with PostgreSQL. To be able to run the tests
|
91
90
|
you will first need to create a `database.yml` file in `spec/dummy/config`.
|
92
91
|
You can use the example file by doing:
|
93
92
|
|
94
93
|
```bash
|
95
|
-
|
94
|
+
cp spec/dummy/config/database.yml.example spec/dummy/config/database.yml
|
96
95
|
```
|
97
96
|
|
98
97
|
To initialize the dummy application for a demo run:
|
99
98
|
|
100
99
|
```bash
|
101
|
-
|
102
|
-
|
103
|
-
|
104
|
-
|
100
|
+
bundle install
|
101
|
+
bundle exec rake db:create db:migrate db:seed app:marty:load_scripts
|
102
|
+
cd spec/dummy
|
103
|
+
bundle exec rails s
|
104
|
+
```
|
105
|
+
|
106
|
+
Or, to run the application with the job workers running:
|
107
|
+
|
108
|
+
```bash
|
109
|
+
bundle install
|
110
|
+
bundle exec rake db:create db:migrate db:seed app:marty:load_scripts
|
111
|
+
cd spec/dummy
|
112
|
+
gem install foreman
|
113
|
+
foreman start
|
105
114
|
```
|
106
115
|
|
107
|
-
The
|
116
|
+
The Marty dummy app should now be accessible in your browser:
|
108
117
|
`localhost:3000`
|
109
118
|
|
110
119
|
You can log in using `marty` as both user and password.
|
@@ -112,13 +121,13 @@ You can log in using `marty` as both user and password.
|
|
112
121
|
To create the test database in prepartion to run your tests:
|
113
122
|
|
114
123
|
```bash
|
115
|
-
|
124
|
+
RAILS_ENV=test bundle exec rake db:create
|
116
125
|
```
|
117
126
|
|
118
127
|
Then to run the tests:
|
119
128
|
|
120
129
|
```bash
|
121
|
-
|
130
|
+
bundle exec rspec
|
122
131
|
```
|
123
132
|
|
124
133
|
# History & Status
|
@@ -20,8 +20,8 @@ module Marty
|
|
20
20
|
handler_str = dj.handler[/job_class.*\n/]
|
21
21
|
job_class = handler_str.gsub('job_class:', '').strip
|
22
22
|
|
23
|
-
"#{job_class} with cron #{dj.cron}
|
24
|
-
'in delayed_jobs table, but is missing in the Dashboard.'
|
23
|
+
"#{job_class} with cron #{dj.cron} and schedule_id #{dj.schedule_id}" \
|
24
|
+
'is present in delayed_jobs table, but is missing in the Dashboard.'
|
25
25
|
end
|
26
26
|
|
27
27
|
messages.join('<br>')
|
@@ -29,6 +29,7 @@ module Marty
|
|
29
29
|
|
30
30
|
c.attributes = [
|
31
31
|
:job_class,
|
32
|
+
:arguments,
|
32
33
|
:cron,
|
33
34
|
:state
|
34
35
|
]
|
@@ -46,6 +47,19 @@ module Marty
|
|
46
47
|
c.width = 400
|
47
48
|
end
|
48
49
|
|
50
|
+
attribute :arguments do |c|
|
51
|
+
c.width = 400
|
52
|
+
|
53
|
+
c.getter = lambda do |record|
|
54
|
+
record.arguments.to_json
|
55
|
+
end
|
56
|
+
|
57
|
+
c.setter = lambda do |record, value|
|
58
|
+
# FIXME: hacky way to parse JSON with single quotes
|
59
|
+
record.arguments = JSON.parse(value.tr("'", '"'))
|
60
|
+
end
|
61
|
+
end
|
62
|
+
|
49
63
|
attribute :cron do |c|
|
50
64
|
c.width = 400
|
51
65
|
end
|
@@ -76,6 +90,8 @@ module Marty
|
|
76
90
|
end
|
77
91
|
|
78
92
|
endpoint :edit_window__edit_form__submit do |params|
|
93
|
+
id = JSON.parse(params['data'])['id']
|
94
|
+
|
79
95
|
result = super(params)
|
80
96
|
next result if result.empty?
|
81
97
|
|
@@ -83,7 +99,7 @@ module Marty
|
|
83
99
|
|
84
100
|
Marty::BackgroundJob::UpdateSchedule.call(
|
85
101
|
id: obj_hash['id'],
|
86
|
-
job_class: obj_hash['job_class']
|
102
|
+
job_class: obj_hash['job_class'],
|
87
103
|
)
|
88
104
|
|
89
105
|
result
|
@@ -97,7 +113,7 @@ module Marty
|
|
97
113
|
|
98
114
|
Marty::BackgroundJob::UpdateSchedule.call(
|
99
115
|
id: obj_hash['id'],
|
100
|
-
job_class: obj_hash['job_class']
|
116
|
+
job_class: obj_hash['job_class'],
|
101
117
|
)
|
102
118
|
|
103
119
|
result
|
@@ -109,7 +125,8 @@ module Marty
|
|
109
125
|
|
110
126
|
endpoint :destroy do |params|
|
111
127
|
res = params.each_with_object({}) do |id, hash|
|
112
|
-
|
128
|
+
record = model.find_by(id: id)
|
129
|
+
job_class = record&.job_class
|
113
130
|
result = super([id])
|
114
131
|
|
115
132
|
# Do nothing If it wasn't destroyed
|
@@ -117,7 +134,7 @@ module Marty
|
|
117
134
|
|
118
135
|
Marty::BackgroundJob::UpdateSchedule.call(
|
119
136
|
id: id,
|
120
|
-
job_class: job_class
|
137
|
+
job_class: job_class,
|
121
138
|
)
|
122
139
|
|
123
140
|
hash.merge(result)
|
@@ -130,7 +147,7 @@ module Marty
|
|
130
147
|
begin
|
131
148
|
s = Marty::BackgroundJob::Schedule.find(client_config['selected'])
|
132
149
|
klass = s.job_class
|
133
|
-
klass.constantize.new.perform
|
150
|
+
klass.constantize.new.perform(*s.arguments)
|
134
151
|
rescue StandardError => e
|
135
152
|
next client.netzke_notify(e.message)
|
136
153
|
end
|
@@ -26,6 +26,7 @@ module Marty
|
|
26
26
|
|
27
27
|
c.attributes = [
|
28
28
|
:job_class,
|
29
|
+
:arguments,
|
29
30
|
:status,
|
30
31
|
:error,
|
31
32
|
:created_at
|
@@ -47,6 +48,17 @@ module Marty
|
|
47
48
|
c.read_only = true
|
48
49
|
end
|
49
50
|
|
51
|
+
attribute :arguments do |c|
|
52
|
+
c.getter = lambda do |record|
|
53
|
+
record.arguments.to_json
|
54
|
+
end
|
55
|
+
|
56
|
+
c.setter = lambda do |record, value|
|
57
|
+
# FIXME: hacky way to parse JSON with single quotes
|
58
|
+
record.arguments = JSON.parse(value.tr("'", '"'))
|
59
|
+
end
|
60
|
+
end
|
61
|
+
|
50
62
|
attribute :status do |c|
|
51
63
|
c.read_only = true
|
52
64
|
end
|
@@ -410,10 +410,6 @@ class Marty::BaseRuleView < Marty::McflyGridPanel
|
|
410
410
|
c.format = 'Y-m-d H:i'
|
411
411
|
end
|
412
412
|
|
413
|
-
attribute :rule_type do |c|
|
414
|
-
c.width = 200
|
415
|
-
end
|
416
|
-
|
417
413
|
def self.init_fields
|
418
414
|
klass.guard_info.each do |namestr, h|
|
419
415
|
field_maker(namestr, h, :simple_guards)
|
@@ -6,7 +6,17 @@ class Marty::DeloreanRuleView < Marty::BaseRuleView
|
|
6
6
|
end
|
7
7
|
|
8
8
|
def self.base_fields
|
9
|
-
|
9
|
+
fields = if klass&.column_names&.include?('rule_type')
|
10
|
+
[:rule_type, :start_dt, :end_dt]
|
11
|
+
else
|
12
|
+
[:start_dt, :end_dt]
|
13
|
+
end
|
14
|
+
|
15
|
+
super + fields
|
16
|
+
end
|
17
|
+
|
18
|
+
attribute :rule_type do |c|
|
19
|
+
c.width = 200
|
10
20
|
end
|
11
21
|
|
12
22
|
attribute :start_dt do |c|
|
@@ -18,8 +28,4 @@ class Marty::DeloreanRuleView < Marty::BaseRuleView
|
|
18
28
|
c.width = 150
|
19
29
|
c.format = 'Y-m-d H:i'
|
20
30
|
end
|
21
|
-
|
22
|
-
attribute :rule_type do |c|
|
23
|
-
c.width = 200
|
24
|
-
end
|
25
31
|
end
|
@@ -29,9 +29,11 @@ class Marty::PromiseView < Marty::Tree
|
|
29
29
|
:priority,
|
30
30
|
:start_dt,
|
31
31
|
:end_dt,
|
32
|
+
:total_time,
|
32
33
|
:status,
|
33
34
|
:cformat,
|
34
35
|
:error,
|
36
|
+
:timeout
|
35
37
|
]
|
36
38
|
c.root_visible = false
|
37
39
|
c.paging = :none
|
@@ -122,6 +124,19 @@ class Marty::PromiseView < Marty::Tree
|
|
122
124
|
config.text = I18n.t('jobs.end_dt')
|
123
125
|
end
|
124
126
|
|
127
|
+
attribute :total_time do |config|
|
128
|
+
config.text = I18n.t('jobs.total_time')
|
129
|
+
config.width = 90
|
130
|
+
|
131
|
+
config.getter = ->(record) do
|
132
|
+
next unless record.start_dt
|
133
|
+
next unless record.end_dt
|
134
|
+
|
135
|
+
time_diff = record.end_dt - record.start_dt
|
136
|
+
Time.zone.at(time_diff.to_i.abs).utc.strftime '%H:%M:%S'
|
137
|
+
end
|
138
|
+
end
|
139
|
+
|
125
140
|
attribute :status do |config|
|
126
141
|
config.hidden = true
|
127
142
|
end
|
data/app/jobs/marty/cron_job.rb
CHANGED
@@ -1,15 +1,23 @@
|
|
1
1
|
class Marty::CronJob < ActiveJob::Base
|
2
|
-
|
2
|
+
attr_accessor :schedule_id
|
3
|
+
|
4
|
+
def enqueue(options = {})
|
5
|
+
self.cron = options[:cron] if options[:cron]
|
6
|
+
self.schedule_id = options[:schedule_id] if options[:schedule_id]
|
7
|
+
super
|
8
|
+
end
|
9
|
+
|
10
|
+
around_perform do |job, block|
|
3
11
|
begin
|
4
12
|
block.call
|
5
|
-
log_success
|
13
|
+
log_success(job.arguments)
|
6
14
|
rescue StandardError => e
|
7
|
-
log_failure(e)
|
15
|
+
log_failure(e, job.arguments)
|
8
16
|
raise e
|
9
17
|
end
|
10
18
|
end
|
11
19
|
|
12
|
-
def log_failure(exception)
|
20
|
+
def log_failure(exception, arguments)
|
13
21
|
error = {
|
14
22
|
message: exception.message,
|
15
23
|
backtrace: exception.backtrace
|
@@ -17,55 +25,49 @@ class Marty::CronJob < ActiveJob::Base
|
|
17
25
|
|
18
26
|
::Marty::BackgroundJob::Log.create!(
|
19
27
|
job_class: self.class.name,
|
28
|
+
arguments: arguments,
|
20
29
|
status: :failure,
|
21
30
|
error: error
|
22
31
|
)
|
23
32
|
end
|
24
33
|
|
25
|
-
def log_success
|
34
|
+
def log_success(arguments)
|
26
35
|
::Marty::BackgroundJob::Log.create!(
|
27
36
|
job_class: self.class.name,
|
37
|
+
arguments: arguments,
|
28
38
|
status: :success
|
29
39
|
)
|
30
40
|
end
|
31
41
|
|
32
42
|
class << self
|
33
|
-
def schedule
|
34
|
-
|
43
|
+
def schedule(schedule_obj:)
|
44
|
+
dj = schedule_obj.delayed_job
|
45
|
+
|
46
|
+
return reschedule_obj(schedule_obj: schedule_obj) if dj.present?
|
35
47
|
|
36
|
-
cron =
|
48
|
+
cron = schedule_obj.cron
|
37
49
|
|
38
50
|
return if cron.blank?
|
39
51
|
|
40
|
-
set(cron: cron).perform_later
|
52
|
+
set(cron: cron, schedule_id: schedule_obj.id).perform_later(*schedule_obj.arguments)
|
41
53
|
end
|
42
54
|
|
43
|
-
def reschedule
|
44
|
-
dj = delayed_job
|
45
|
-
return dj.update(cron:
|
55
|
+
def reschedule(schedule_obj:)
|
56
|
+
dj = schedule_obj.delayed_job
|
57
|
+
return dj.update(cron: schedule_obj.cron) if dj.locked_by?
|
46
58
|
|
47
|
-
remove
|
48
|
-
schedule
|
59
|
+
remove(dj)
|
60
|
+
schedule(schedule_obj: schedule_obj)
|
49
61
|
end
|
50
62
|
|
51
|
-
def remove
|
52
|
-
|
63
|
+
def remove(dj)
|
64
|
+
dj.destroy if dj.present?
|
53
65
|
end
|
54
66
|
|
55
67
|
alias remove_schedule remove
|
56
68
|
|
57
|
-
def scheduled?
|
58
|
-
|
59
|
-
end
|
60
|
-
|
61
|
-
def delayed_job
|
62
|
-
Delayed::Job.
|
63
|
-
where('handler LIKE ?', "%job_class: #{name}\n%").
|
64
|
-
first
|
65
|
-
end
|
66
|
-
|
67
|
-
def cron_expression
|
68
|
-
::Marty::BackgroundJob::Schedule.on.find_by(job_class: name)&.cron
|
69
|
+
def scheduled?(schedule_id:)
|
70
|
+
Delayed::Job.find_by(schedule_id: schedule_id).present?
|
69
71
|
end
|
70
72
|
end
|
71
73
|
end
|