karafka 1.4.0.rc1

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at maciej@mensfeld.pl. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/CONTRIBUTING.md

@@ -0,0 +1,41 @@
+# Contribute
+
+## Introduction
+
+First, thank you for considering contributing to karafka! It's people like you that make the open source community such a great community! 😊
+
+We welcome any type of contribution, not only code. You can help with:
+- **QA**: file bug reports, the more details you can give the better (e.g. screenshots with the console open)
+- **Marketing**: writing blog posts, howto's, printing stickers, ...
+- **Community**: presenting the project at meetups, organizing a dedicated meetup for the local community, ...
+- **Code**: take a look at the [open issues](issues). Even if you can't write code, commenting on them, showing that you care about a given issue matters. It helps us triage them.
+
+## Your First Contribution
+
+Working on your first Pull Request? You can learn how from this *free* series, [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github).
+
+## Submitting code
+
+Any code change should be submitted as a pull request. The description should explain what the code does and give steps to execute it. The pull request should also contain tests.
+
+## Code review process
+
+Each pull request must pass all the rspec specs and meet our quality requirements.
+
+To check if everything is as it should be, we use [Coditsu](https://coditsu.io) that combines multiple linters and code analyzers for both code and documentation. Once you're done with your changes, submit a pull request.
+
+Coditsu will automatically check your work against our quality standards. You can find your commit check results on the [builds page](https://app.coditsu.io/karafka/commit_builds) of Karafka organization.
+
+[![coditsu](https://coditsu.io/assets/quality_bar.svg)](https://app.coditsu.io/karafka/commit_builds)
+
+## Questions
+
+If you have any questions, create an [issue](issue) (protip: do a quick search first to see if someone else didn't ask the same question before!).
+You can also reach us at hello@karafka.opencollective.com.
+
+## Credits
+
+### Contributors
+
+Thank you to all the people who have already contributed to karafka!
+<a href="graphs/contributors"><img src="https://opencollective.com/karafka/contributors.svg?width=890" /></a>

data/Gemfile

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
+
+plugin 'diffend'
+
+gemspec
+
+group :test do
+  gem 'byebug'
+  gem 'factory_bot'
+  gem 'rspec'
+  gem 'simplecov'
+end

data/Gemfile.lock

@@ -0,0 +1,139 @@
+PATH
+  remote: .
+  specs:
+    karafka (1.4.0.rc1)
+      dry-configurable (~> 0.8)
+      dry-inflector (~> 0.1)
+      dry-monitor (~> 0.3)
+      dry-validation (~> 1.2)
+      envlogic (~> 1.1)
+      irb (~> 1.0)
+      rake (>= 11.3)
+      ruby-kafka (>= 1.0.0)
+      thor (>= 0.20)
+      waterdrop (~> 1.4.0)
+      zeitwerk (~> 2.1)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (6.0.3.2)
+      concurrent-ruby (~> 1.0, >= 1.0.2)
+      i18n (>= 0.7, < 2)
+      minitest (~> 5.1)
+      tzinfo (~> 1.1)
+      zeitwerk (~> 2.2, >= 2.2.2)
+    byebug (11.1.3)
+    concurrent-ruby (1.1.7)
+    delivery_boy (1.0.1)
+      king_konf (~> 0.3)
+      ruby-kafka (~> 1.0)
+    diff-lcs (1.4.4)
+    digest-crc (0.6.1)
+      rake (~> 13.0)
+    docile (1.3.2)
+    dry-configurable (0.11.6)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 0.4, >= 0.4.7)
+      dry-equalizer (~> 0.2)
+    dry-container (0.7.2)
+      concurrent-ruby (~> 1.0)
+      dry-configurable (~> 0.1, >= 0.1.3)
+    dry-core (0.4.9)
+      concurrent-ruby (~> 1.0)
+    dry-equalizer (0.3.0)
+    dry-events (0.2.0)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 0.4)
+      dry-equalizer (~> 0.2)
+    dry-inflector (0.2.0)
+    dry-initializer (3.0.3)
+    dry-logic (1.0.7)
+      concurrent-ruby (~> 1.0)
+      dry-core (~> 0.2)
+      dry-equalizer (~> 0.2)
+    dry-monitor (0.3.2)
+      dry-configurable (~> 0.5)
+      dry-core (~> 0.4)
+      dry-equalizer (~> 0.2)
+      dry-events (~> 0.2)
+    dry-schema (1.5.3)
+      concurrent-ruby (~> 1.0)
+      dry-configurable (~> 0.8, >= 0.8.3)
+      dry-core (~> 0.4)
+      dry-equalizer (~> 0.2)
+      dry-initializer (~> 3.0)
+      dry-logic (~> 1.0)
+      dry-types (~> 1.4)
+    dry-types (1.4.0)
+      concurrent-ruby (~> 1.0)
+      dry-container (~> 0.3)
+      dry-core (~> 0.4, >= 0.4.4)
+      dry-equalizer (~> 0.3)
+      dry-inflector (~> 0.1, >= 0.1.2)
+      dry-logic (~> 1.0, >= 1.0.2)
+    dry-validation (1.5.4)
+      concurrent-ruby (~> 1.0)
+      dry-container (~> 0.7, >= 0.7.1)
+      dry-core (~> 0.4)
+      dry-equalizer (~> 0.2)
+      dry-initializer (~> 3.0)
+      dry-schema (~> 1.5)
+    envlogic (1.1.2)
+      dry-inflector (~> 0.1)
+    factory_bot (6.1.0)
+      activesupport (>= 5.0.0)
+    i18n (1.8.5)
+      concurrent-ruby (~> 1.0)
+    io-console (0.5.6)
+    irb (1.2.4)
+      reline (>= 0.0.1)
+    king_konf (0.3.7)
+    minitest (5.14.1)
+    rake (13.0.1)
+    reline (0.1.4)
+      io-console (~> 0.5)
+    rspec (3.9.0)
+      rspec-core (~> 3.9.0)
+      rspec-expectations (~> 3.9.0)
+      rspec-mocks (~> 3.9.0)
+    rspec-core (3.9.2)
+      rspec-support (~> 3.9.3)
+    rspec-expectations (3.9.2)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.9.0)
+    rspec-mocks (3.9.1)
+      diff-lcs (>= 1.2.0, < 2.0)
+      rspec-support (~> 3.9.0)
+    rspec-support (3.9.3)
+    ruby-kafka (1.2.0)
+      digest-crc
+    simplecov (0.19.0)
+      docile (~> 1.1)
+      simplecov-html (~> 0.11)
+    simplecov-html (0.12.2)
+    thor (1.0.1)
+    thread_safe (0.3.6)
+    tzinfo (1.2.7)
+      thread_safe (~> 0.1)
+    waterdrop (1.4.0)
+      delivery_boy (>= 0.2, < 2.x)
+      dry-configurable (~> 0.8)
+      dry-monitor (~> 0.3)
+      dry-validation (~> 1.2)
+      ruby-kafka (>= 0.7.8)
+      zeitwerk (~> 2.1)
+    zeitwerk (2.4.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  byebug
+  factory_bot
+  karafka!
+  rspec
+  simplecov
+
+BUNDLED WITH
+   2.1.4

data/MIT-LICENCE

@@ -0,0 +1,18 @@
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,99 @@
+![karafka logo](https://raw.githubusercontent.com/karafka/misc/master/logo/karafka_logotype_transparent2.png)
+
+[![Build Status](https://travis-ci.org/karafka/karafka.svg?branch=master)](https://travis-ci.org/karafka/karafka)
+
+**Note**: Documentation presented here refers to **not yet** released Karafka `1.4.x`.
+
+If you are looking for the documentation for Karafka `1.3.x`, it can be found [here](https://github.com/karafka/wiki/tree/1.3).
+
+## About Karafka
+
+Framework used to simplify Apache Kafka based Ruby applications development.
+
+Karafka allows you to capture everything that happens in your systems in large scale, providing you with a seamless and stable core for consuming and processing this data, without having to focus on things that are not your business domain.
+
+Karafka not only handles incoming messages but also provides tools for building complex data-flow applications that receive and send messages.
+
+## How does it work
+
+Karafka provides a higher-level abstraction that allows you to focus on your business logic development, instead of focusing on implementing lower level abstraction layers. It provides developers with a set of tools that are dedicated for building multi-topic applications similar to how Rails applications are being built.
+
+### Some things you might wonder about:
+
+- You can integrate Karafka with **any** Ruby-based application.
+- Karafka does **not** require Sidekiq or any other third party software (apart from Kafka itself).
+- Karafka works with Ruby on Rails but it is a **standalone** framework that can work without it.
+- Karafka has a **minimal** set of dependencies, so adding it won't be a huge burden for your already existing applications.
+- Karafka processes can be executed for a **given subset** of consumer groups and/or topics, so you can fine tune it depending on your business logic.
+
+Karafka based applications can be easily deployed to any type of infrastructure, including those based on:
+
+* Heroku
+* Capistrano
+* Docker
+* Terraform
+
+## Support
+
+Karafka has a [Wiki pages](https://github.com/karafka/karafka/wiki) for almost everything and a pretty decent [FAQ](https://github.com/karafka/karafka/wiki/FAQ). It covers the whole installation, setup, and deployment along with other useful details on how to run Karafka.
+
+If you have any questions about using Karafka, feel free to join our [Gitter](https://gitter.im/karafka/karafka) chat channel.
+
+## Getting started
+
+If you're completely new to the subject, you can start with our "Kafka on Rails" articles series, that will get you up and running with the terminology and basic ideas behind using Kafka:
+
+- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 1 – Kafka basics and its advantages](https://mensfeld.pl/2017/11/kafka-on-rails-using-kafka-with-ruby-on-rails-part-1-kafka-basics-and-its-advantages/)
+- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 2 – Getting started with Ruby and Kafka](https://mensfeld.pl/2018/01/kafka-on-rails-using-kafka-with-ruby-on-rails-part-2-getting-started-with-ruby-and-kafka/)
+
+If you want to get started with Kafka and Karafka as fast as possible, then the best idea is to just clone our example repository:
+
+```bash
+git clone https://github.com/karafka/example-app ./example_app
+```
+
+then, just bundle install all the dependencies:
+
+```bash
+cd ./example_app
+bundle install
+```
+
+and follow the instructions from the [example app Wiki](https://github.com/karafka/example-app/blob/master/README.md).
+
+**Note**: you need to ensure, that you have Kafka up and running and you need to configure Kafka seed_brokers in the ```karafka.rb``` file.
+
+If you need more details and know how on how to start Karafka with a clean installation, read the [Getting started page](https://github.com/karafka/karafka/wiki/Getting-started) section of our Wiki.
+
+## Notice
+
+Karafka framework and Karafka team are __not__ related to Kafka streaming service called CloudKarafka in any matter. We don't recommend nor discourage usage of their platform.
+
+## References
+
+* [Karafka framework](https://github.com/karafka/karafka)
+* [Karafka Travis CI](https://travis-ci.org/karafka/karafka)
+* [Karafka Coditsu](https://app.coditsu.io/karafka/repositories/karafka)
+
+## Note on contributions
+
+First, thank you for considering contributing to Karafka! It's people like you that make the open source community such a great community!
+
+Each pull request must pass all the RSpec specs and meet our quality requirements.
+
+To check if everything is as it should be, we use [Coditsu](https://coditsu.io) that combines multiple linters and code analyzers for both code and documentation. Once you're done with your changes, submit a pull request.
+
+Coditsu will automatically check your work against our quality standards. You can find your commit check results on the [builds page](https://app.coditsu.io/karafka/commit_builds) of Karafka organization.
+
+[![coditsu](https://coditsu.io/assets/quality_bar.svg)](https://app.coditsu.io/karafka/commit_builds)
+
+## Contributors
+
+This project exists thanks to all the people who contribute.
+<a href="https://github.com/karafka/karafka/graphs/contributors"><img src="https://opencollective.com/karafka/contributors.svg?width=890" /></a>
+
+## Sponsors
+
+We are looking for sustainable sponsorship. If your company is relying on Karafka framework or simply want to see Karafka evolve faster to meet your requirements, please consider backing the project.
+
+Please contact [Maciej Mensfeld](mailto:maciej@mensfeld.pl) directly for more details.

data/bin/karafka

@@ -0,0 +1,19 @@
+#!/usr/bin/env ruby
+
+require 'karafka'
+
+# If there is a boot file, we need to require it as we expect it to contain
+# Karafka app setup, routes, etc
+if File.exist?(Karafka.boot_file)
+  require Karafka.boot_file.to_s
+else
+  # However when it is unavailable, we still want to be able to run help command
+  # and install command as they don't require configured app itself to run
+  raise(
+    Karafka::Errors::MissingBootFileError,
+    Karafka.boot_file
+  ) unless %w[-h install].include?(ARGV[0])
+end
+
+Karafka::Cli.prepare
+Karafka::Cli.start

data/certs/mensfeld.pem

@@ -0,0 +1,25 @@
+-----BEGIN CERTIFICATE-----
+MIIEODCCAqCgAwIBAgIBATANBgkqhkiG9w0BAQsFADAjMSEwHwYDVQQDDBhtYWNp
+ZWovREM9bWVuc2ZlbGQvREM9cGwwHhcNMjAwODExMDkxNTM3WhcNMjEwODExMDkx
+NTM3WjAjMSEwHwYDVQQDDBhtYWNpZWovREM9bWVuc2ZlbGQvREM9cGwwggGiMA0G
+CSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQDCpXsCgmINb6lHBXXBdyrgsBPSxC4/
+2H+weJ6L9CruTiv2+2/ZkQGtnLcDgrD14rdLIHK7t0o3EKYlDT5GhD/XUVhI15JE
+N7IqnPUgexe1fbZArwQ51afxz2AmPQN2BkB2oeQHXxnSWUGMhvcEZpfbxCCJH26w
+hS0Ccsma8yxA6hSlGVhFVDuCr7c2L1di6cK2CtIDpfDaWqnVNJEwBYHIxrCoWK5g
+sIGekVt/admS9gRhIMaIBg+Mshth5/DEyWO2QjteTodItlxfTctrfmiAl8X8T5JP
+VXeLp5SSOJ5JXE80nShMJp3RFnGw5fqjX/ffjtISYh78/By4xF3a25HdWH9+qO2Z
+tx0wSGc9/4gqNM0APQnjN/4YXrGZ4IeSjtE+OrrX07l0TiyikzSLFOkZCAp8oBJi
+Fhlosz8xQDJf7mhNxOaZziqASzp/hJTU/tuDKl5+ql2icnMv5iV/i6SlmvU29QNg
+LCV71pUv0pWzN+OZbHZKWepGhEQ3cG9MwvkCAwEAAaN3MHUwCQYDVR0TBAIwADAL
+BgNVHQ8EBAMCBLAwHQYDVR0OBBYEFImGed2AXS070ohfRidiCEhXEUN+MB0GA1Ud
+EQQWMBSBEm1hY2llakBtZW5zZmVsZC5wbDAdBgNVHRIEFjAUgRJtYWNpZWpAbWVu
+c2ZlbGQucGwwDQYJKoZIhvcNAQELBQADggGBAKiHpwoENVrMi94V1zD4o8/6G3AU
+gWz4udkPYHTZLUy3dLznc/sNjdkJFWT3E6NKYq7c60EpJ0m0vAEg5+F5pmNOsvD3
+2pXLj9kisEeYhR516HwXAvtngboUcb75skqvBCU++4Pu7BRAPjO1/ihLSBexbwSS
+fF+J5OWNuyHHCQp+kGPLtXJe2yUYyvSWDj3I2//Vk0VhNOIlaCS1+5/P3ZJThOtm
+zJUBI7h3HgovwRpcnmk2mXTmU4Zx/bCzX8EA6VY0khEvnmiq7S6eBF0H9qH8KyQ6
+EkVLpvmUDFcf/uNaBQdazEMB5jYtwoA8gQlANETNGPi51KlkukhKgaIEDMkBDJOx
+65N7DzmkcyY0/GwjIVIxmRhcrCt1YeCUElmfFx0iida1/YRm6sB2AXqScc1+ECRi
+2DND//YJUikn1zwbz1kT70XmHd97B4Eytpln7K+M1u2g1pHVEPW4owD/ammXNpUy
+nt70FcDD4yxJQ+0YNiHd0N8IcVBM1TMIVctMNQ==
+-----END CERTIFICATE-----

data/config/errors.yml

@@ -0,0 +1,39 @@
+en:
+  dry_validation:
+    errors:
+      invalid_broker_schema: >
+        has an invalid format
+        Expected schema, host and port number
+        Example: kafka://127.0.0.1:9092 or kafka+ssl://127.0.0.1:9092
+      invalid_certificate: >
+        is not a valid certificate
+      invalid_certificate_from_path: >
+        is not a valid certificate
+      invalid_private_key: >
+        is not a valid private key
+      max_timeout_size_for_exponential: >
+        pause_timeout cannot be more than pause_max_timeout
+      max_wait_time_limit:
+        max_wait_time cannot be more than socket_timeout
+      topics_names_not_unique: >
+        all topic names within a single consumer group must be unique
+      ssl_client_cert_with_ssl_client_cert_key: >
+        Both ssl_client_cert and ssl_client_cert_key need to be provided
+      ssl_client_cert_key_with_ssl_client_cert: >
+        Both ssl_client_cert_key and ssl_client_cert need to be provided
+      ssl_client_cert_chain_with_ssl_client_cert: >
+        Both ssl_client_cert_chain and ssl_client_cert need to be provided
+      ssl_client_cert_chain_with_ssl_client_cert_key: >
+        Both ssl_client_cert_chain and ssl_client_cert_key need to be provided
+      ssl_client_cert_key_password_with_ssl_client_cert_key: >
+        Both ssl_client_cert_key_password and ssl_client_cert_key need to be provided
+      does_not_respond_to_token: >
+        needs to respond to a #token method
+      required_usage_count: >
+        Given topic must be used at least once
+      pid_already_exists: >
+        Pidfile already exists
+      consumer_groups_inclusion: >
+        Unknown consumer group
+      does_not_exist:
+        Given file does not exist or cannot be read