acidic_job 1.0.0.rc3 → 1.0.0.rc5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +6 -6
  3. data/app/models/acidic_job/entry.rb +4 -2
  4. data/app/models/acidic_job/execution.rb +25 -12
  5. data/app/models/acidic_job/value.rb +2 -0
  6. data/lib/acidic_job/builder.rb +2 -2
  7. data/lib/acidic_job/context.rb +36 -20
  8. data/lib/acidic_job/errors.rb +2 -3
  9. data/lib/acidic_job/log_subscriber.rb +4 -6
  10. data/lib/acidic_job/plugin_context.rb +26 -3
  11. data/lib/acidic_job/plugins/transactional_step.rb +4 -4
  12. data/lib/acidic_job/serializer.rb +28 -0
  13. data/lib/acidic_job/serializers/job_serializer.rb +1 -1
  14. data/lib/acidic_job/serializers/range_serializer.rb +1 -3
  15. data/lib/acidic_job/testing.rb +10 -12
  16. data/lib/acidic_job/version.rb +1 -1
  17. data/lib/acidic_job/workflow.rb +44 -32
  18. data/lib/acidic_job.rb +3 -0
  19. data/lib/generators/acidic_job/install_generator.rb +5 -5
  20. data/lib/generators/acidic_job/templates/create_acidic_job_tables_migration.rb.erb +6 -6
  21. metadata +7 -55
  22. data/.codacy.yml +0 -4
  23. data/.github/FUNDING.yml +0 -13
  24. data/.github/workflows/main.yml +0 -38
  25. data/.gitignore +0 -18
  26. data/.rubocop.yml +0 -83
  27. data/.ruby-version +0 -1
  28. data/Gemfile +0 -5
  29. data/Gemfile.lock +0 -201
  30. data/Rakefile +0 -16
  31. data/TODO +0 -77
  32. data/UPGRADE_GUIDE.md +0 -81
  33. data/acidic_job.gemspec +0 -52
  34. data/bin/console +0 -21
  35. data/bin/setup +0 -8
  36. data/bin/test_all +0 -26
  37. data/blog_post.md +0 -28
  38. data/combustion/log/test.log +0 -0
  39. data/gemfiles/rails_7.0.gemfile +0 -11
  40. data/gemfiles/rails_7.1.gemfile +0 -11
  41. data/gemfiles/rails_7.2.gemfile +0 -11
  42. data/gemfiles/rails_8.0.gemfile +0 -11
data/UPGRADE_GUIDE.md DELETED
@@ -1,81 +0,0 @@
1
- # AcidicJob Upgrade Guide
2
-
3
- ## 1. Update version requirements in `Gemfile`
4
-
5
- ```diff
6
- - gem "acidic_job"
7
- + gem "acidic_job", "~> 1.0.0.pre1"
8
- ```
9
-
10
- result:
11
- ```
12
- Installing acidic_job 1.0.0.pre4 (was 0.7.7)
13
- Bundle updated!
14
- ```
15
-
16
- ## 2. Generate migration for new `AcidicJob::Run` model
17
-
18
- ```bash
19
- rails generate acidic_job
20
- ```
21
-
22
- result:
23
- ```
24
- create db/migrate/#{yyyymmddhhmmss}_create_acidic_job_runs.rb
25
- ```
26
-
27
- ## 3. Delete any unneeded `AcidicJob::Key` records
28
-
29
- Typically, records that are already finished do not need to be retained. Sometimes, however, applications key finished records around for some amount of time for debugging or metrics aggregation. Whatever your application's logic is for whether or not an `AcidicJob::Key` record is still needed, for all unneeded records, delete them.
30
-
31
- For example, this would delete all finished `Key` records over 1 month old:
32
-
33
- ```ruby
34
- AcidicJob::Key.where(recovery_point: AcidicJob::Key::RECOVERY_POINT_FINISHED, last_run_at: ..1.month.ago).delete_all
35
- ```
36
-
37
- ## 4. Migrate `AcidicJob::Key` to `AcidicJob::Run`
38
-
39
- `AcidicJob` ships with an upgrade module that provides a script to migrate older `Key` records to the new `Run` model.
40
-
41
- ```ruby
42
- AcidicJob::UpgradeService.execute
43
- ```
44
-
45
- This script will prepare an `insert_all` command for `Run` records by mapping the older `Key` data to the new `Run` schema. It also creates the new `Run` records with the same `id` as their `Key` counterparts, and then deletes all `Key` records successfully mapped over. Any `Key` records that were failed to be mapped over will be reported, along with the exception, in the `errored_keys` portion of the resulting hash.
46
-
47
- result:
48
- ```
49
- {
50
- run_records: <Integer>,
51
- key_records: <Integer>,
52
- errored_keys: <Array>
53
- }
54
- ```
55
-
56
- ## 5. Triage remaining `AcidicJob::Key` records
57
-
58
- If there were any `AcidicJob::Key` records that failed to be mapped to the new `Run` model, you will need to manually triage whatever the exception was. In all likelihood, the exception would be relating to the translation of the `Key#job_args` field to the `Run#serialized_job` field, as all other fields have a fairly straight-forward mapping. If you can't resolve the issue, please open an Issue in GitHub.
59
-
60
- ## 6. Ensure all `AcidicJob::Staged` records are processed
61
-
62
- `AcidicJob` still ships with an upgrade module that provides the older `Key` and `Staged` records, so this functionality will still be present to handle any existing records in your database when you deploy the updated version.
63
-
64
- ## 7. Remove the old tables
65
-
66
- Once you have successfully migrated everything over and the new system has been running smoothly for some time, you should drop the old `acidic_job_keys` and `staged_acidic_jobs` tables. We provide a migration generator just for this purpose:
67
-
68
- ```bash
69
- rails generate acidic_job:drop_tables
70
- ```
71
-
72
- result:
73
- ```
74
- create db/migrate/#{yyyymmddhhmmss}_drop_old_acidic_job_tables.rb
75
- ```
76
-
77
- You can then run the migration to have those tables removed:
78
-
79
- ```bash
80
- rails db:migrate
81
- ```
data/acidic_job.gemspec DELETED
@@ -1,52 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require_relative "lib/acidic_job/version"
4
-
5
- Gem::Specification.new do |spec|
6
- spec.name = "acidic_job"
7
- spec.version = AcidicJob::VERSION
8
- spec.authors = ["fractaledmind"]
9
- spec.email = ["stephen.margheim@gmail.com"]
10
-
11
- spec.summary = "Idempotent operations for Rails apps, built on top of ActiveJob."
12
- spec.description = "Idempotent operations for Rails apps, built on top of ActiveJob."
13
- spec.homepage = "https://github.com/fractaledmind/acidic_job"
14
- spec.license = "MIT"
15
- spec.required_ruby_version = Gem::Requirement.new(">= 2.7.0")
16
-
17
- spec.metadata["homepage_uri"] = spec.homepage
18
- spec.metadata["source_code_uri"] = "https://github.com/fractaledmind/acidic_job"
19
- spec.metadata["changelog_uri"] = "https://github.com/fractaledmind/acidic_job/CHANGELOG.md"
20
-
21
- # Specify which files should be added to the gem when it is released.
22
- # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
23
- spec.files = Dir.chdir(File.expand_path(__dir__)) do
24
- `git ls-files -z`.split("\x0").reject { |f| f.match(%r{\A(?:test|spec|features)/}) }
25
- end
26
- spec.bindir = "exe"
27
- spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
28
- spec.require_paths = ["lib"]
29
-
30
- spec.add_dependency "json", ">= 2.7.0" # see: https://github.com/ruby/json/pull/519
31
- ">= 7.1".tap do |rails_version|
32
- spec.add_dependency "activejob", rails_version
33
- spec.add_dependency "activerecord", rails_version
34
- spec.add_dependency "activesupport", rails_version
35
- spec.add_dependency "railties", rails_version
36
- spec.add_development_dependency "actionmailer", rails_version
37
- end
38
-
39
- spec.add_development_dependency "chaotic_job"
40
- spec.add_development_dependency "combustion"
41
- spec.add_development_dependency "minitest"
42
- spec.add_development_dependency "rake"
43
- spec.add_development_dependency "rubocop"
44
- spec.add_development_dependency "rubocop-minitest"
45
- spec.add_development_dependency "rubocop-rake"
46
- spec.add_development_dependency "simplecov"
47
- spec.add_development_dependency "sqlite3"
48
-
49
- # For more information and examples about making a new gem, checkout our
50
- # guide at: https://bundler.io/guides/creating_gem.html
51
- spec.metadata["rubygems_mfa_required"] = "true"
52
- end
data/bin/console DELETED
@@ -1,21 +0,0 @@
1
- #!/usr/bin/env ruby
2
- # frozen_string_literal: true
3
-
4
- require "bundler/setup"
5
- require "rails"
6
- require "acidic_job"
7
-
8
- # You can add fixtures and/or initialization code here to make experimenting
9
- # with your gem easier. You can also use a different console, if you like.
10
-
11
- require "combustion"
12
- require "sqlite3"
13
- Combustion.path = "test/combustion"
14
- Combustion.initialize! :active_record
15
-
16
- # (If you use this, don't forget to add pry to your Gemfile!)
17
- # require "pry"
18
- # Pry.start
19
-
20
- require "irb"
21
- IRB.start(__FILE__)
data/bin/setup DELETED
@@ -1,8 +0,0 @@
1
- #!/usr/bin/env bash
2
- set -euo pipefail
3
- IFS=$'\n\t'
4
- set -vx
5
-
6
- bundle install
7
-
8
- # Do any other automated setup that you need to do here
data/bin/test_all DELETED
@@ -1,26 +0,0 @@
1
- run_tests() {
2
- if [ "$#" -ne 2 ]; then
3
- echo "Usage: run_tests RUBY_VERSION RAILS_VERSION"
4
- echo "Example: run_tests 3.0.7 7.1"
5
- return 1
6
- fi
7
-
8
- local ruby="$1"
9
- local gemfile="gemfiles/rails_$2.gemfile"
10
-
11
- echo "**************************************************"
12
- echo "Running tests with Ruby $1 and Rails $2..."
13
- ASDF_RUBY_VERSION=$ruby BUNDLE_GEMFILE=$gemfile bundle check ||
14
- ASDF_RUBY_VERSION=$ruby BUNDLE_GEMFILE=$gemfile bundle install &&
15
- ASDF_RUBY_VERSION=$ruby BUNDLE_GEMFILE=$gemfile bundle exec rake
16
- }
17
-
18
- run_tests "3.0.7" "7.1"
19
- run_tests "3.1.6" "7.1"
20
- run_tests "3.1.6" "7.2"
21
- run_tests "3.2.5" "7.1"
22
- run_tests "3.2.5" "7.2"
23
- run_tests "3.2.5" "8.0"
24
- run_tests "3.3.5" "7.1"
25
- run_tests "3.3.5" "7.2"
26
- run_tests "3.3.5" "8.0"
data/blog_post.md DELETED
@@ -1,28 +0,0 @@
1
- # ACIDic Operations in Rails
2
-
3
- At the conceptual heart of basically any software are "operations"—the discrete actions the software performs. At the horizon of basically any software is the goal to make that sofware _robust_. Typically, one makes a software system robust by making each of its operations robust. Moreover, typically, robustness in software is considered as the software being "ACIDic"—atomic, consistent, isolated, durable.
4
-
5
- In a loosely collected series of articles, Brandur Leach lays out the core techniques and principles required to make an HTTP API properly ACIDic:
6
-
7
- 1. https://brandur.org/acid
8
- 2. https://brandur.org/http-transactions
9
- 3. https://brandur.org/job-drain
10
- 4. https://brandur.org/idempotency-keys
11
-
12
- With these techniques and principles in mind, our challenge is bring them into the world of a standard Rails application. This will require us to conceptually map the concepts of an HTTP request, an API server action, and an HTTP response into the world of a running Rails process.
13
-
14
- We can begin to make this mapping by observing that an API server action is a specific instantiation of the general concept of an "operation". Like all operations, it has a "trigger" (the HTTP request) and a "response" (the HTTP response). So, what we need is a foundation upon which to build our Rails "operations".
15
-
16
- In order to help us find that tool, let us consider the necessary characteristics we need. We need something that we can easily trigger from other Ruby code throughout our Rails application (controller actions, model methods, model callbacks, etc.). It should also be able to be run both synchronously (blocking execution and then returning its response to the caller) and asychronously (non-blocking and the caller doesn't know its response). It should then also be able to retry a specific operation (in much the way that an API consumer can "retry an operation" by hitting the same endpoint with the same request).
17
-
18
- As we lay out these characteristics, I imagine your mind is going where mine went—`ActiveJob` gives us a solid foundation upon which we can build "ACIDic" operations.
19
-
20
- So, our challenge to build tooling which will allow us to make "operational" jobs _robust_.
21
-
22
- What we need primarily is to be able to make our jobs *idempotent*, and one of the simplest yet still most powerful tools for making an operation idempotent is the idempotency key. As laid out in the article linked above, an idempotency key is a record that we store in our database to uniquely identify a particular execution of an operation and a related "recovery point" for where we are in the process of that operation.
23
-
24
-
25
-
26
-
27
-
28
-
File without changes
@@ -1,11 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- source "https://rubygems.org"
4
-
5
- gem "activejob", "~> 7.0.0"
6
- gem "activerecord", "~> 7.0.0"
7
- gem "activesupport", "~> 7.0.0"
8
- gem "railties", "~> 7.0.0"
9
- gem "sqlite3", "~> 1.7.3"
10
-
11
- gemspec path: "../"
@@ -1,11 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- source "https://rubygems.org"
4
-
5
- gem "activejob", "~> 7.1.0"
6
- gem "activerecord", "~> 7.1.0"
7
- gem "activesupport", "~> 7.1.0"
8
- gem "railties", "~> 7.1.0"
9
- gem "sqlite3", ">= 2.0.0"
10
-
11
- gemspec path: "../"
@@ -1,11 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- source "https://rubygems.org"
4
-
5
- gem "activejob", "~> 7.2.0"
6
- gem "activerecord", "~> 7.2.0"
7
- gem "activesupport", "~> 7.2.0"
8
- gem "railties", "~> 7.2.0"
9
- gem "sqlite3", ">= 2.0.0"
10
-
11
- gemspec path: "../"
@@ -1,11 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- source "https://rubygems.org"
4
-
5
- gem "activejob", "~> 8.0.0.rc1"
6
- gem "activerecord", "~> 8.0.0.rc1"
7
- gem "activesupport", "~> 8.0.0.rc1"
8
- gem "railties", "~> 8.0.0.rc1"
9
- gem "sqlite3", ">= 2.1.0"
10
-
11
- gemspec path: "../"