kiqchestra 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: ee0fcc0d1b95edbe89f701de4360704c050c1cd66bc19825afb9f7c94da758a7
+  data.tar.gz: 54828682b4d25ed828f8cd3a531dc86d8475e43fe42f863d28ec975e6e57e876
+SHA512:
+  metadata.gz: ae4249497721d90095c82d977a0b7e5d968d2ce10ee90e1a07b4bfa74b33e9053d2f9d3ef8eccc456088038646f09ff289200404c4e1d9cc71df8dbfe9ee1018
+  data.tar.gz: 70b2a244001ec267bea317316a6f5a31ea2fab82c109561bcf3260226a75667990f4c902fa80d4a30fa20bacd0548bb170f986e5e972d888b7bd89234a43080e

data/.rspec

@@ -0,0 +1,3 @@
+--format documentation
+--color
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,17 @@
+AllCops:
+  TargetRubyVersion: 3.0
+
+Layout/LineLength:
+  Exclude:
+    - Gemfile
+
+Metrics/BlockLength:
+  Exclude:
+    - kiqchestra.gemspec
+    - spec/**/*
+
+Style/StringLiterals:
+  EnforcedStyle: double_quotes
+
+Style/StringLiteralsInInterpolation:
+  EnforcedStyle: double_quotes

data/CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2025-01-17
+
+### Added
+
+- Initial release of Kiqchestra, a Sidekiq-based job orchestration framework.
+- Support for defining workflows with job dependencies.
+- Redis-based default metadata store for job and workflow progress tracking.
+- Ability to implement custom stores for workflow metadata and progress.
+- Progress tracking for workflows.
+
+### Documentation
+
+- Comprehensive README with usage examples, installation instructions, and contribution guidelines.

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,132 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official email address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[INSERT CONTACT METHOD].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 aries
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,97 @@
+# Kiqchestra - Sidekiq Job Orchestration Framework
+
+Kiqchestra is a lightweight Sidekiq-based job orchestration framework that allows you to manage complex job workflows with dependencies between tasks. Unlike traditional background job frameworks, Kiqchestra simplifies job orchestration by allowing jobs to be executed based on the completion of other jobs, ensuring smooth and controlled execution of task sequences.
+
+## Features
+
+- **Job Dependencies**: Easily manage dependency relationship between jobs and ensure they run in the correct order.
+- **Progress Tracking**: Track the progress of each job in a workflow. Customizable for your need.
+
+## Installation
+
+Add the following line to your `Gemfile`:
+
+```ruby
+gem 'kiqchestra'
+```
+
+Run `bundle install` to install the gem.
+Alternatively, you can install the gem manually:
+
+```bash
+gem install kiqchestra
+```
+
+## Configuration
+
+By default, Kiqchestra uses a Redis-backed store (`DefaultWorkflowStore`) that caches workflow metadata and progress for 7 days, but you can provide your own custom store if needed. To use `DefaultWorkflowStore`, make sure to have `ENV[REDIS_URL]` - if not defined, `redis://localhost:6379/0` will be used (see [Kiqchestra::RedisClient](https://github.com/ariesjchang/kiqchestra/blob/main/lib/kiqchestra/redis_client.rb)).
+
+### Customizing WorkflowStore
+
+You can create your own store by subclassing Kiqchestra::WorkflowStore and implementing the required methods (`read_metadata`, `write_metadata`, `read_progress`, `write_progress`).
+
+For example:
+
+```ruby
+class MyCustomWorkflowStore < Kiqchestra::WorkflowStore
+  def read_metadata(workflow_id)
+    # Your implementation
+  end
+
+  def write_metadata(workflow_id, metadata)
+    # Your implementation
+  end
+
+  def read_progress(workflow_id)
+    # Your implementation
+  end
+
+  def write_progress(workflow_id, progress)
+    # Your implementation
+  end
+end
+
+Kiqchestra.configure do |config|
+  config.store = MyCustomWorkflowStore.new
+end
+```
+
+## Usage
+
+### Defining a Workflow
+
+A workflow is defined by a unique workflow_id and a set of metadata that describes the jobs and their dependencies. The metadata for each job should include:
+- `deps`: An array of jobs that must complete before the current job can start.
+- `args`: Arguments to be passed to the job.
+
+Here's an example of a workflow metadata definition:
+
+```ruby
+metadata = {
+  a_job: { deps: [], args: [1, 2, 3] },
+  b_job: { deps: [:a_job], args: [4, 5] },
+  c_job: { deps: [:a_job], args: nil },
+  d_job: { deps: [:b_job, :c_job], args: [6] }
+}
+```
+
+Make sure each job name corresponds to an actual existing class that is a subclass of `Kiqchestra::BaseJob` (ex. `a_job` would be `AJob`).
+You can create and run a workflow by:
+
+```ruby
+workflow = Kiqchestra::Workflow.new('workflow_123', metadata)
+workflow.execute
+```
+
+## Contributing
+Contributions to Kiqchestra are welcome! To contribute:
+
+1. Fork the repository.
+2. Create a new branch.
+3. Make your changes.
+4. Run tests.
+5. Open a pull request.
+
+## License
+
+Kiqchestra is released under the MIT License.

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/lib/kiqchestra/base_job.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+require "kiqchestra/workflow"
+
+module Kiqchestra
+  # BaseJob provides a standard interface for all workflow jobs.
+  # It includes the necessary callback mechanism to notify the Workflow
+  # once a job has completed and trigger any follow-up jobs based on the dependencies.
+  #
+  # Subclasses are expected to implement the `perform` method to define
+  # the specific logic for the job. This method will be automatically
+  # invoked as part of the job execution flow, and callbacks like `on_complete`
+  # will be triggered when the job finishes.
+  class BaseJob
+    include Sidekiq::Worker
+
+    # Perform the job and trigger workflow callbacks on completion.
+    #
+    # @param workflow_id [String] The unique ID for the workflow
+    # @param args [Array] Arguments to be passed to the specific job's perform method
+    def perform(workflow_id, *args)
+      @workflow_id = workflow_id
+      @args = args
+
+      log_info "Starting job #{job_name} in workflow #{@workflow_id} with args: #{@args.inspect}"
+
+      begin
+        # Delegate the actual job work to the subclass's perform method
+        perform_job(*@args)
+
+        workflow.handle_completed_job job_name
+      rescue StandardError => e
+        log_error "#{job_name} failed: #{e.message}"
+
+        # Re-raise to let Sidekiq handle retries
+        raise e
+      end
+    end
+
+    # Subclasses should define the actual job logic here.
+    # This is called by the `perform` method of BaseJob.
+    #
+    # @param args [Array] Arguments passed to the job
+    def perform_job(*args)
+      raise NotImplementedError, "Subclasses must implement `perform_job`"
+    end
+
+    private
+
+    # Fetch the workflow instance lazily, using pre-fetched workflow data.
+    def workflow
+      @workflow ||= Workflow.new @workflow_id, metadata
+    end
+
+    # Extract job name from the class
+    # Equivalent to Rails's .demodulize.underscore
+    def job_name
+      self.class.name.split("::").last.gsub(/([A-Z]+)([A-Z][a-z])/, '\1_\2')
+          .gsub(/([a-z\d])([A-Z])/, '\1_\2').tr("-", "_").downcase
+    end
+
+    # Fetch the cached workflow data from redis
+    def metadata
+      return @metadata if @metadata&.present?
+
+      metadata = Kiqchestra.config.store.read_metadata @workflow_id
+      @metadata = metadata.transform_keys(&:to_sym).transform_values do |data|
+        { deps: data["deps"].map(&:to_sym), args: data["args"] }
+      end
+    end
+
+    # Logs an info-level message
+    def log_info(message)
+      Sidekiq.logger.info "kiqchestra: #{message}"
+    end
+
+    # Logs an error-level message
+    def log_error(message)
+      Sidekiq.logger.error "kiqchestra: #{message}"
+    end
+  end
+end

data/lib/kiqchestra/config.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require "kiqchestra/default_workflow_store"
+
+module Kiqchestra
+  # The Config class provides a configuration object for Kiqchestra, allowing users
+  # to customize various aspects of the library's behavior.
+  #
+  # By default, it initializes with a DefaultWorkflowStore instance for managing
+  # workflow storage (dependencies, progress, etc.). Users can override this with
+  # a custom store by setting the `workflow_store` attribute.
+  #
+  # Attributes:
+  # - `workflow_store`: The object responsible for storing workflow-related data.
+  #   Defaults to an instance of `DefaultWorkflowStore`.
+  #
+  # Example Usage:
+  #   Kiqchestra.configure do |config|
+  #     config.store = MyCustomWorkflowStore.new
+  #   end
+  class Config
+    attr_accessor :store
+
+    def initialize
+      @store = DefaultWorkflowStore.new
+    end
+  end
+end

data/lib/kiqchestra/default_workflow_store.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+
+require "json"
+require "kiqchestra/redis_client"
+require "kiqchestra/workflow_store"
+
+module Kiqchestra
+  # The DefaultWorkflowStore class is the default implementation of WorkflowStore
+  # for storing workflow dependencies, progress, and arguments in a Redis-backed store.
+  #
+  # This implementation uses Redis for persistence, providing an efficient and
+  # scalable storage system for workflows. Users can customize this by implementing
+  # their own subclass of WorkflowStore if needed.
+  #
+  # Example Usage:
+  #   store = Kiqchestra::DefaultWorkflowStore.new
+  #   store.write_metadata "workflow_123", { a_job: { deps: [], args: [1, 2, 3] } }
+  #   metadata = store.read_metadata "workflow_123"
+  #   store.write_progress "workflow_123", { a_job: "complete" }
+  #   progress = store.read_progress "workflow_123"
+  class DefaultWorkflowStore < WorkflowStore
+    # Reads the metadata for a workflow from Redis.
+    #
+    # @param workflow_id [String] The workflow ID to retrieve workflow data for.
+    # @return [Hash] A hash representing the workflow metadata, where each key is
+    #                a task ID and the value is a hash with keys `:deps` and `:args`.
+    def read_metadata(workflow_id)
+      raw_data = Kiqchestra::RedisClient.client.get metadata_key(workflow_id)
+      JSON.parse(raw_data || "{}")
+    end
+
+    # Writes the metadata for a workflow to Redis.
+    #
+    # @param workflow_id [String] The workflow ID to store dependencies for.
+    # @param metadata [Hash] A hash representing the metadata to store.
+    # @example { a_job: { deps: [], args: [1, 2, 3] }, b_job: { deps: [:a_job], args: nil } }
+    def write_metadata(workflow_id, metadata)
+      Kiqchestra::RedisClient.client.set metadata_key(workflow_id),
+                                         metadata.to_json,
+                                         ex: 604_800 # Default TTL: 7 days
+    end
+
+    # Reads the progress of a workflow from Redis.
+    #
+    # @param workflow_id [String] The workflow ID to retrieve progress for.
+    # @return [Hash] A hash representing the progress of the workflow, where each
+    #                key is a task ID and the value indicates the status.
+    def read_progress(workflow_id)
+      raw_data = Kiqchestra::RedisClient.client.get progress_key(workflow_id)
+      JSON.parse(raw_data || "{}")
+    end
+
+    # Writes the progress of a workflow to Redis.
+    #
+    # @param workflow_id [String] The workflow ID to store progress for.
+    # @param progress [Hash] A hash representing the progress to store.
+    # @example { a_worker: "complete", b_worker: "in_progress" }
+    def write_progress(workflow_id, progress)
+      Kiqchestra::RedisClient.client.set progress_key(workflow_id),
+                                         progress.to_json,
+                                         ex: 604_800 # Default TTL: 7 days
+    end
+
+    private
+
+    # Generates the Redis key for storing metadata of a specific workflow.
+    #
+    # @param workflow_id [String] The workflow ID.
+    # @return [String] The Redis key for metadata.
+    def metadata_key(workflow_id)
+      "workflow:#{workflow_id}:metadata"
+    end
+
+    # Generates the Redis key for storing progress of a specific workflow.
+    #
+    # @param workflow_id [String] The workflow ID.
+    # @return [String] The Redis key for progress.
+    def progress_key(workflow_id)
+      "workflow:#{workflow_id}:progress"
+    end
+  end
+end

data/lib/kiqchestra/redis_client.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require "redis"
+
+module Kiqchestra
+  # Redis client used for caching workflow dependencies and progress.
+  #
+  # Dependencies:
+  # - `redis` gem: Ensure this is included in your Gemfile.
+  class RedisClient
+    def self.client
+      @client ||= Redis.new(url: ENV["REDIS_URL"] || "redis://localhost:6379/0")
+    end
+  end
+end

data/lib/kiqchestra/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Kiqchestra
+  VERSION = "1.0.0"
+end

data/lib/kiqchestra/workflow.rb

@@ -0,0 +1,185 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+require "redis"
+
+module Kiqchestra
+  # The Workflow class provides functionality for orchestrating
+  # Sidekiq-based job workflows. It manages task dependencies
+  # and tracks their completion status.
+  class Workflow
+    # Initializes the workflow with workflow id and data.
+    #
+    # @param workflow_id [String] Unique ID for the workflow
+    # @param metadata [Hash] Hash defining jobs' dependencies and arguments
+    # @example {
+    #   a_job: { deps: [], args: a_job_args },
+    #   b_job: { deps: [:a_job], args: b_job_args },
+    #   c_job: { deps: [:a_job], args: c_job_args },
+    #   d_job: { deps: %i[b_job c_job], args: d_job_args }
+    # }
+    def initialize(workflow_id, metadata)
+      @workflow_id = workflow_id
+      @metadata = metadata
+      @logger = Logger.new($stdout)
+
+      validate_metadata
+      cache_metadata metadata
+    end
+
+    # Starts the workflow execution.
+    def execute
+      read_progress
+      @metadata.each do |job, job_data|
+        process_job job, job_data
+      end
+
+      conclude_workflow if workflow_complete?
+    end
+
+    # Handles the completion of a job and triggers the next jobs if dependencies are met.
+    #
+    # @param job [String] The completed job name (ex. "example_job")
+    def handle_completed_job(job)
+      update_progress job, "complete"
+      log_info "#{job} completed for workflow #{@workflow_id}"
+
+      execute
+    end
+
+    private
+
+    # Validates the workflow data structure for a workflow.
+    # Ensures that all job metadata, dependencies, and arguments conform to the expected format.
+    def validate_metadata
+      # Ensure the root structure is a hash
+      raise ArgumentError, "Metadata must be a hash" unless @metadata.is_a?(Hash)
+
+      # Validate each job's metadata
+      @metadata.each do |job, data|
+        validate_job_metadata job, data
+      end
+    end
+
+    # Validates the metadata for a specific job.
+    # Ensures that the metadata is a hash and its components (deps and args) are correctly structured.
+    #
+    # @param job [Symbol, String] The job identifier
+    # @param data [Hash] Metadata for the job (includes deps and args)
+    def validate_job_metadata(job, data)
+      # Check if the metadata is a hash
+      raise ArgumentError, "Metadata for #{job} must be a hash" unless data.is_a?(Hash)
+
+      # Validate dependencies and arguments separately
+      validate_deps job, data[:deps]
+      validate_args job, data[:args]
+    end
+
+    # Validates the dependencies (`deps`) for a specific job.
+    # Ensures that dependencies are an array of symbols or strings.
+    #
+    # @param job [Symbol, String] The job identifier
+    # @param deps [Array<Symbol, String>] Dependencies for the job
+    def validate_deps(job, deps)
+      return if deps.is_a?(Array) && deps.all? { |dep| dep.is_a?(String) || dep.is_a?(Symbol) }
+
+      raise ArgumentError, "Dependencies for #{job} must be an array of strings or symbols"
+    end
+
+    # Validates the arguments (`args`) for a specific job.
+    # Ensures that arguments are either an array or nil.
+    #
+    # @param job [Symbol, String] The job identifier
+    # @param args [Array, nil] Arguments for the job
+    def validate_args(job, args)
+      return if args.nil? || args.is_a?(Array)
+
+      raise ArgumentError, "Arguments for #{job} must be an array or nil"
+    end
+
+    # Caches the workflow data using the configured workflow store.
+    def cache_metadata(metadata)
+      Kiqchestra.config.store.write_metadata @workflow_id, metadata
+    end
+
+    # Processes a single job during workflow execution.
+    #
+    # @param job [String] The job identifier
+    # @param job_metadata [Hash] Metadata for the job (dependencies and arguments)
+    def process_job(job, job_metadata)
+      return if job_already_processed? job
+
+      return unless ready_to_execute? job_metadata[:deps]
+
+      args = job_metadata[:args] || []
+      enqueue_job job, args
+    end
+
+    # Checks if a job is already processed (completed or in_progress).
+    #
+    # @param job [String] The job identifier
+    # @return [Boolean] True if the job is completed or in progress
+    def job_already_processed?(job)
+      progress = read_progress
+      %w[complete in_progress].include? progress[job.to_s]
+    end
+
+    # Checks if a job is ready for execution (no dependencies or all dependencies completed).
+    #
+    # @param deps [Array<Symbol, String>] Dependencies for the job
+    # @return [Boolean] True if the job is ready to execute
+    def ready_to_execute?(deps)
+      progress = read_progress
+      deps.empty? || deps.all? { |dep| progress[dep.to_s] == "complete" }
+    end
+
+    # Enqueues a Sidekiq job for execution and saves the job's status as "in_progress".
+    #
+    # @param job [String] Job name in snake_case.
+    # @param args [Array] Arguments to pass to the job's perform method (default: empty array).
+    def enqueue_job(job, args = [])
+      return if job_already_processed? job
+
+      update_progress job, "in_progress"
+
+      job_class_name = job.to_s.split("_").map(&:capitalize).join
+      job_class = Object.const_get(job_class_name)
+      job_class.perform_async @workflow_id, *args
+    rescue NameError
+      raise "Class for job '#{job}' not defined"
+    end
+
+    # Logs a message if a logger is configured.
+    def log_info(message)
+      @logger&.info "kiqchestra: #{message}"
+    end
+
+    # Updates the workflow progress for a job.
+    #
+    # @param job [String] job name in snake_case
+    # @param status [String] job status ("in_progress", "completed")
+    def update_progress(job, status)
+      progress = read_progress
+      progress[job] = status
+      Kiqchestra.config.store.write_progress @workflow_id, progress
+    end
+
+    # Reads the current workflow progress from Redis.
+    #
+    # @return [Hash] The current progress of the workflow
+    def read_progress
+      Kiqchestra.config.store.read_progress @workflow_id
+    end
+
+    # Checks if the workflow is complete (all jobs are completed) and logs the workflow's end.
+    def workflow_complete?
+      progress = read_progress
+      @metadata.keys.all? { |job| progress[job.to_s] == "complete" }
+    end
+
+    # Executes the customizable on-complete procedure for the workflow.
+    def conclude_workflow
+      log_info "Workflow #{@workflow_id} has completed successfully."
+    end
+  end
+end

data/lib/kiqchestra/workflow_store.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+module Kiqchestra
+  # The WorkflowStore class serves as an abstract base class for implementing
+  # custom storage mechanisms for workflow progress, dependencies, and arguments.
+  # It defines abstract methods that must be implemented by subclasses.
+  #
+  # By default, Kiqchestra uses DefaultWorkflowStore. Users are free to create
+  # their own custom implementation of WorkflowStore to change storage system
+  # (file-based, database, etc.) or specific details of implementations.
+  class WorkflowStore
+    # Ensures subclasses properly implement required methods.
+    def initialize(*_args)
+      # No initialization logic here
+    end
+
+    # Reads the workflow data for a workflow.
+    #
+    # @return [Hash] A hash representing the metadata for the workflow, where each key
+    #                is a task ID and the value is a hash with keys `:deps` and `:args`.
+    # @raise [NotImplementedError] This method must be implemented by a subclass.
+    def read_metadata
+      raise NotImplementedError, "Subclasses must implement the read_workflow_data method"
+    end
+
+    # Writes the workflow data for a workflow to the store.
+    #
+    # @param workflow_data [Hash] A hash representing the workflow data to store, where each key
+    #                             is a task ID and the value is a hash with keys `:deps` and `:args`.
+    # @example { a_job: { deps: [], args: [1, 2, 3] }, b_job: { deps: [:a_job], args: nil } }
+    # @raise [NotImplementedError] This method must be implemented by a subclass.
+    def write_metadata(_metadata)
+      raise NotImplementedError, "Subclasses must implement the write_workflow_data method"
+    end
+
+    # Reads the progress of a workflow.
+    #
+    # @return [Hash] A hash representing the progress of the workflow, where each key
+    #                is a task ID and the value indicates the completion status.
+    # @raise [NotImplementedError] This method must be implemented by a subclass.
+    def read_progress
+      raise NotImplementedError, "Subclasses must implement the read_progress method"
+    end
+
+    # Writes the progress of a workflow to the store.
+    #
+    # @param progress [Hash] A hash representing the progress to store, where each key
+    #                        is a task ID and the value indicates the completion status.
+    # @example { a_worker: 'complete', b_worker: 'in_progress' }
+    # @raise [NotImplementedError] This method must be implemented by a subclass.
+    def write_progress(_progress)
+      raise NotImplementedError, "Subclasses must implement the write_progress method"
+    end
+  end
+end

data/lib/kiqchestra.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+require "kiqchestra/base_job"
+require "kiqchestra/config"
+require "kiqchestra/version"
+require "kiqchestra/workflow"
+require "kiqchestra/workflow_store"
+
+# Kiqchestra is a Sidekiq-based job orchestration framework designed for
+# workflows where tasks depend on the completion of other tasks.
+# It simplifies the process of managing complex job dependencies, enabling developers
+# to focus on business logic rather than the intricacies of dependency management.
+module Kiqchestra
+  class << self
+    # Yields the configuration object to allow customization of settings.
+    def configure
+      yield config
+    end
+
+    # Returns the configuration object, initializing it if necessary.
+    def config
+      @config ||= Config.new
+    end
+  end
+end

data/sig/kiqchestra.rbs

@@ -0,0 +1,4 @@
+module Kiqchestra
+  VERSION: String
+  # See the writing guide of rbs: https://github.com/ruby/rbs#guides
+end

metadata

@@ -0,0 +1,78 @@
+--- !ruby/object:Gem::Specification
+name: kiqchestra
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- aries
+autorequire:
+bindir: exe
+cert_chain: []
+date: 2025-01-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.5.5
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.5.5
+description: Kiqchestra enhances the power of Sidekiq by introducing a job orchestration
+  framework designed to handle complex workflows with ease and efficiency.
+email:
+- edwardjhchang@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".rspec"
+- ".rubocop.yml"
+- CHANGELOG.md
+- CODE_OF_CONDUCT.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/kiqchestra.rb
+- lib/kiqchestra/base_job.rb
+- lib/kiqchestra/config.rb
+- lib/kiqchestra/default_workflow_store.rb
+- lib/kiqchestra/redis_client.rb
+- lib/kiqchestra/version.rb
+- lib/kiqchestra/workflow.rb
+- lib/kiqchestra/workflow_store.rb
+- sig/kiqchestra.rbs
+homepage: https://github.com/ariesjchang/kiqchestra.
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+  homepage_uri: https://github.com/ariesjchang/kiqchestra
+  source_code_uri: https://github.com/ariesjchang/kiqchestra
+  changelog_uri: https://github.com/ariesjchang/kiqchestra/blob/main/CHANGELOG.md
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.0.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.16
+signing_key:
+specification_version: 4
+summary: Job Workflow Orchestration Layer for Sidekiq
+test_files: []