karafka 1.2.0

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at maciej@coditsu.io. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/CONTRIBUTING.md

@@ -0,0 +1,41 @@
+# Contribute
+
+## Introduction
+
+First, thank you for considering contributing to karafka! It's people like you that make the open source community such a great community! 😊
+
+We welcome any type of contribution, not only code. You can help with 
+- **QA**: file bug reports, the more details you can give the better (e.g. screenshots with the console open)
+- **Marketing**: writing blog posts, howto's, printing stickers, ...
+- **Community**: presenting the project at meetups, organizing a dedicated meetup for the local community, ...
+- **Code**: take a look at the [open issues](issues). Even if you can't write code, commenting on them, showing that you care about a given issue matters. It helps us triage them.
+
+## Your First Contribution
+
+Working on your first Pull Request? You can learn how from this *free* series, [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github).
+
+## Submitting code
+
+Any code change should be submitted as a pull request. The description should explain what the code does and give steps to execute it. The pull request should also contain tests.
+
+## Code review process
+
+Each pull request must pass all the rspec specs and meet our quality requirements.
+
+To check if everything is as it should be, we use [Coditsu](https://coditsu.io) that combines multiple linters and code analyzers for both code and documentation. Once you're done with your changes, submit a pull request.
+
+Coditsu will automatically check your work against our quality standards. You can find your commit check results on the [builds page](https://app.coditsu.io/karafka/commit_builds) of Karafka organization.
+
+[![coditsu](https://coditsu.io/assets/quality_bar.svg)](https://app.coditsu.io/karafka/commit_builds)
+
+## Questions
+
+If you have any questions, create an [issue](issue) (protip: do a quick search first to see if someone else didn't ask the same question before!).
+You can also reach us at hello@karafka.opencollective.com.
+
+## Credits
+
+### Contributors
+
+Thank you to all the people who have already contributed to karafka!
+<a href="graphs/contributors"><img src="https://opencollective.com/karafka/contributors.svg?width=890" /></a>

data/Gemfile

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+gemspec
+
+group :development, :test do
+  gem 'rspec'
+  gem 'simplecov'
+  gem 'timecop'
+end

data/Gemfile.lock

@@ -0,0 +1,123 @@
+PATH
+  remote: .
+  specs:
+    karafka (1.2.0)
+      activesupport (>= 4.0)
+      dry-configurable (~> 0.7)
+      dry-inflector (~> 0.1.1)
+      dry-monitor (~> 0.1)
+      dry-validation (~> 0.11)
+      envlogic (~> 1.0)
+      multi_json (>= 1.12)
+      rake (>= 11.3)
+      require_all (>= 1.4)
+      ruby-kafka (>= 0.5.3)
+      thor (~> 0.19)
+      waterdrop (~> 1.2)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (5.1.5)
+      concurrent-ruby (~> 1.0, >= 1.0.2)
+      i18n (~> 0.7)
+      minitest (~> 5.1)
+      tzinfo (~> 1.1)
+    concurrent-ruby (1.0.5)
+    delivery_boy (0.2.4)
+      king_konf (~> 0.1.8)
+      ruby-kafka (~> 0.5.1)
+    diff-lcs (1.3)
+    docile (1.3.0)
+    dry-configurable (0.7.0)
+      concurrent-ruby (~> 1.0)
+    dry-container (0.6.0)
+      concurrent-ruby (~> 1.0)
+      dry-configurable (~> 0.1, >= 0.1.3)
+    dry-core (0.4.5)
+      concurrent-ruby (~> 1.0)
+    dry-equalizer (0.2.0)
+    dry-events (0.1.0)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 0.4)
+      dry-equalizer (~> 0.2)
+    dry-inflector (0.1.1)
+    dry-logic (0.4.2)
+      dry-container (~> 0.2, >= 0.2.6)
+      dry-core (~> 0.2)
+      dry-equalizer (~> 0.2)
+    dry-monitor (0.1.2)
+      dry-configurable (~> 0.5)
+      dry-equalizer (~> 0.2)
+      dry-events (~> 0.1)
+      rouge (~> 2.0, >= 2.2.1)
+    dry-types (0.12.2)
+      concurrent-ruby (~> 1.0)
+      dry-configurable (~> 0.1)
+      dry-container (~> 0.3)
+      dry-core (~> 0.2, >= 0.2.1)
+      dry-equalizer (~> 0.2)
+      dry-logic (~> 0.4, >= 0.4.2)
+      inflecto (~> 0.0.0, >= 0.0.2)
+    dry-validation (0.11.1)
+      concurrent-ruby (~> 1.0)
+      dry-configurable (~> 0.1, >= 0.1.3)
+      dry-core (~> 0.2, >= 0.2.1)
+      dry-equalizer (~> 0.2)
+      dry-logic (~> 0.4, >= 0.4.0)
+      dry-types (~> 0.12.0)
+    envlogic (1.1.0)
+      dry-inflector (~> 0.1)
+    i18n (0.9.5)
+      concurrent-ruby (~> 1.0)
+    inflecto (0.0.2)
+    json (2.1.0)
+    king_konf (0.1.10)
+    minitest (5.11.3)
+    multi_json (1.13.1)
+    null-logger (0.1.5)
+    rake (12.3.1)
+    require_all (2.0.0)
+    rouge (2.2.1)
+    rspec (3.7.0)
+      rspec-core (~> 3.7.0)
+      rspec-expectations (~> 3.7.0)
+      rspec-mocks (~> 3.7.0)
+    rspec-core (3.7.1)
+      rspec-support (~> 3.7.0)
+    rspec-expectations (3.7.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.7.0)
+    rspec-mocks (3.7.0)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.7.0)
+    rspec-support (3.7.1)
+    ruby-kafka (0.5.4)
+    simplecov (0.16.1)
+      docile (~> 1.1)
+      json (>= 1.8, < 3)
+      simplecov-html (~> 0.10.0)
+    simplecov-html (0.10.2)
+    thor (0.20.0)
+    thread_safe (0.3.6)
+    timecop (0.9.1)
+    tzinfo (1.2.5)
+      thread_safe (~> 0.1)
+    waterdrop (1.2.0)
+      delivery_boy (~> 0.2)
+      dry-configurable (~> 0.7)
+      dry-monitor (~> 0.1)
+      dry-validation (~> 0.11)
+      null-logger
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  karafka!
+  rspec
+  simplecov
+  timecop
+
+BUNDLED WITH
+   1.16.1

data/MIT-LICENCE

@@ -0,0 +1,18 @@
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,89 @@
+![karafka logo](https://raw.githubusercontent.com/karafka/misc/master/logo/karafka_logotype_transparent2.png)
+
+[![Build Status](https://travis-ci.org/karafka/karafka.svg?branch=master)](https://travis-ci.org/karafka/karafka)
+
+Framework used to simplify Apache Kafka based Ruby applications development.
+
+Karafka allows you to capture everything that happens in your systems in large scale, providing you with a seamless and stable core for consuming and processing this data, without having to focus on things that are not your business domain.
+
+Karafka not only handles incoming messages but also provides tools for building complex data-flow applications that receive and send messages.
+
+**Warning**: Wiki and all the docs refer to the 1.2.0.beta4. Sorry for the inconvenience. We will release the stable 1.2.0 version soon.
+
+## How does it work
+
+Karafka provides a higher-level abstraction that allows you to focus on your business logic development, instead of focusing on implementing lower level abstraction layers. It provides developers with a set of tools that are dedicated for building multi-topic applications similarly to how Rails applications are being built.
+
+### Some things you might wonder about:
+
+- You can integrate Karafka with **any** Ruby based application.
+- Karafka does **not** require Sidekiq or any other third party software (apart from Kafka itself).
+- Karafka works with Ruby on Rails but it is a **standalone** framework that can work without it.
+- Karafka has a **minimal** set of dependencies, so adding it won't be a huge burden for your already existing applications.
+- Karafka processes can be executed for a **given subset** of consumer groups and/or topics, so you can fine tune it depending on your business logic.
+
+Karafka based applications can be easily deployed to any type of infrastructure, including those based on:
+
+* Heroku
+* Capistrano
+* Docker
+
+## Support
+
+Karafka has a [Wiki pages](https://github.com/karafka/karafka/wiki) for almost everything and a pretty decent [FAQ](https://github.com/karafka/karafka/wiki/FAQ). It covers the whole installation, setup and deployment along with other useful details on how to run Karafka.
+
+If you have any questions about using Karafka, feel free to join our [Gitter](https://gitter.im/karafka/karafka) chat channel.
+
+## Getting started
+
+If you want to get started with Kafka and Karafka as fast as possible, then the best idea is to just clone our example repository:
+
+```bash
+git clone https://github.com/karafka/karafka-example-app ./example_app
+```
+
+then, just bundle install all the dependencies:
+
+```bash
+cd ./example_app
+bundle install
+```
+
+and follow the instructions from the [example app Wiki](https://github.com/karafka/karafka-example-app/blob/master/README.md).
+
+**Note**: you need to ensure, that you have Kafka up and running and you need to configure Kafka seed_brokers in the ```karafka.rb``` file.
+
+If you need more details and know how on how to start Karafka with a clean installation, read the [Getting started page](https://github.com/karafka/karafka/wiki/Getting-started) section of our Wiki.
+
+## Notice
+
+Karafka framework and Karafka team are __not__ related to Kafka streaming service called CloudKarafka in any matter. We don't recommend nor discourage usage of their platform.
+
+## References
+
+* [Karafka framework](https://github.com/karafka/karafka)
+* [Karafka Travis CI](https://travis-ci.org/karafka/karafka)
+* [Karafka Coditsu](https://app.coditsu.io/karafka/repositories/karafka)
+
+## Note on contributions
+
+First, thank you for considering contributing to Karafka! It's people like you that make the open source community such a great community!
+
+Each pull request must pass all the rspec specs and meet our quality requirements.
+
+To check if everything is as it should be, we use [Coditsu](https://coditsu.io) that combines multiple linters and code analyzers for both code and documentation. Once you're done with your changes, submit a pull request.
+
+Coditsu will automatically check your work against our quality standards. You can find your commit check results on the [builds page](https://app.coditsu.io/karafka/commit_builds) of Karafka organization.
+
+[![coditsu](https://coditsu.io/assets/quality_bar.svg)](https://app.coditsu.io/karafka/commit_builds)
+
+## Contributors
+
+This project exists thanks to all the people who contribute.
+<a href="https://github.com/karafka/karafka/graphs/contributors"><img src="https://opencollective.com/karafka/contributors.svg?width=890" /></a>
+
+## Sponsors
+
+We are looking for sustainable sponsorship. If your company is relying on Karafka framework or simply want to see Karafka evolve faster to meet your requirements, please consider backing the project.
+
+Please contact [Maciej Mensfeld](mailto:maciej@coditsu.io) directly for more details.

data/bin/karafka

@@ -0,0 +1,19 @@
+#!/usr/bin/env ruby
+
+require 'karafka'
+
+# If there is a boot file, we need to require it as we expect it to contain
+# Karafka app setup, routes, etc
+if File.exist?(Karafka.boot_file)
+  require Karafka.boot_file.to_s
+else
+  # However when it is unavailable, we still want to be able to run help command
+  # and install command as they don't require configured app itself to run
+  raise(
+    Karafka::Errors::MissingBootFile,
+    Karafka.boot_file
+  ) unless %w[-h install].include?(ARGV[0])
+end
+
+Karafka::Cli.prepare
+Karafka::Cli.start

data/config/errors.yml

@@ -0,0 +1,6 @@
+en:
+  errors:
+    broker_schema?: >
+      has an invalid format.
+      Expected schema, host and port number.
+      Example: kafka://127.0.0.1:9092 or kafka+ssl://127.0.0.1:9092

data/karafka.gemspec

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+lib = File.expand_path('lib', __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require 'karafka/version'
+
+Gem::Specification.new do |spec|
+  spec.name        = 'karafka'
+  spec.version     = ::Karafka::VERSION
+  spec.platform    = Gem::Platform::RUBY
+  spec.authors     = ['Maciej Mensfeld', 'Pavlo Vavruk', 'Adam Gwozdowski']
+  spec.email       = %w[maciej@coditsu.io pavlo.vavruk@gmail.com adam99g@gmail.com]
+  spec.homepage    = 'https://github.com/karafka/karafka'
+  spec.summary     = 'Ruby based framework for working with Apache Kafka'
+  spec.description = 'Framework used to simplify Apache Kafka based Ruby applications development'
+  spec.license     = 'MIT'
+
+  spec.add_dependency 'activesupport', '>= 4.0'
+  spec.add_dependency 'dry-configurable', '~> 0.7'
+  spec.add_dependency 'dry-inflector', '~> 0.1.1'
+  spec.add_dependency 'dry-monitor', '~> 0.1'
+  spec.add_dependency 'dry-validation', '~> 0.11'
+  spec.add_dependency 'envlogic', '~> 1.0'
+  spec.add_dependency 'multi_json', '>= 1.12'
+  spec.add_dependency 'rake', '>= 11.3'
+  spec.add_dependency 'require_all', '>= 1.4'
+  spec.add_dependency 'ruby-kafka', '>= 0.5.3'
+  spec.add_dependency 'thor', '~> 0.19'
+  spec.add_dependency 'waterdrop', '~> 1.2'
+
+  spec.required_ruby_version = '>= 2.3.0'
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(spec)/}) }
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.require_paths = %w[lib]
+end

data/lib/karafka.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+%w[
+  English
+  waterdrop
+  kafka
+  envlogic
+  thor
+  fileutils
+  multi_json
+  require_all
+  dry-configurable
+  dry-validation
+  dry/inflector
+  dry/monitor/notifications
+  active_support/callbacks
+  karafka/loader
+].each(&method(:require))
+
+# Karafka library
+module Karafka
+  extend Envlogic
+
+  class << self
+    # @return [Logger] logger that we want to use. Will use ::Karafka::Logger by default
+    def logger
+      @logger ||= App.config.logger
+    end
+
+    # @return [::Karafka::Monitor] monitor that we want to use
+    def monitor
+      @monitor ||= App.config.monitor
+    end
+
+    # @return [String] root path of this gem
+    def gem_root
+      Pathname.new(File.expand_path('..', __dir__))
+    end
+
+    # @return [String] Karafka app root path (user application path)
+    def root
+      Pathname.new(ENV['KARAFKA_ROOT_DIR'] || File.dirname(ENV['BUNDLE_GEMFILE']))
+    end
+
+    # @return [String] path to Karafka gem root core
+    def core_root
+      Pathname.new(File.expand_path('karafka', __dir__))
+    end
+
+    # @return [String] path to a default file that contains booting procedure etc
+    # @note By default it is a file called 'karafka.rb' but it can be specified as you wish if you
+    #   have Karafka that is merged into a Sinatra/Rails app and karafka.rb is taken.
+    #   It will be used for console/consumers/etc
+    # @example Standard only-Karafka case
+    #   Karafka.boot_file #=> '/home/app_path/karafka.rb'
+    # @example Non standard case
+    #   KARAFKA_BOOT_FILE='/home/app_path/app.rb'
+    #   Karafka.boot_file #=> '/home/app_path/app.rb'
+    def boot_file
+      Pathname.new(ENV['KARAFKA_BOOT_FILE'] || File.join(Karafka.root, 'karafka.rb'))
+    end
+  end
+end
+
+%w[
+  callbacks
+  callbacks/*
+  setup/dsl
+  setup/config
+  status
+  schemas/config
+  schemas/consumer_group_topic
+  schemas/consumer_group
+].each { |path| require_all File.join(Karafka.core_root, path + '.rb') }
+
+Karafka::Loader.load!(Karafka.core_root)
+Kafka::Consumer.prepend(Karafka::Patches::RubyKafka)
+Dry::Configurable::Config.prepend(Karafka::Patches::DryConfigurable)