karafka 1.4.14 → 1.4.15
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +28 -0
- data/README.md +14 -91
- data/lib/karafka/version.rb +1 -1
- data/lib/karafka.rb +30 -0
- data.tar.gz.sig +0 -0
- metadata +3 -3
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 611ba9b66a65ca5ba6b0b39201b416d522bfd4fcef20d7241fa64337f9b34431
|
|
4
|
+
data.tar.gz: 0bb87fe1569b54d03b42d2921b03f722fc6996f46d94c4ff1037896a42d9c7b4
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 69c3fb16f1213d25bc40f65d2ddfec0aa7f16cf4be00d0f79330e8ec2d399d18f2e4171d337acd68990550613bc32c64a56a874fd6f4fd40e6673950fa1295a5
|
|
7
|
+
data.tar.gz: 24d3398e30f84d10a6b975758cbb32eeb33ed3ae2d0bc5c2d05615156e32ca6de954aa38e5cc5c46cb03e7a0e8382346c3c9070e35b2dde3427818493f9b1b9a
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,33 @@
|
|
|
1
1
|
# Karafka framework changelog
|
|
2
2
|
|
|
3
|
+
## 1.4.15 (2023-06-19)
|
|
4
|
+
|
|
5
|
+
```
|
|
6
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
7
|
+
~~~ CRITICAL NOTICE ON KARAFKA 1.4 reliability ~~~
|
|
8
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
9
|
+
|
|
10
|
+
Karafka 1.4 is no longer supported and contains critical errors, included but not limited to:
|
|
11
|
+
|
|
12
|
+
- Double processing of messages
|
|
13
|
+
- Skipping messages
|
|
14
|
+
- Hanging during processing
|
|
15
|
+
- Unexpectedly stopping message processing
|
|
16
|
+
- Failure to deliver messages to Kafka
|
|
17
|
+
- Resetting the consumer group and starting from the beginning
|
|
18
|
+
|
|
19
|
+
To resolve these issues, it is highly recommended to upgrade to Karafka 2.1 or higher.
|
|
20
|
+
|
|
21
|
+
If you want to ignore this message and continue, set the I_ACCEPT_CRITICAL_ERRORS_IN_KARAFKA_1_4 env variable to true.
|
|
22
|
+
|
|
23
|
+
Apologies for any inconvenience caused by this release.
|
|
24
|
+
There is no other way to make sure, that you are notified about those issues and their severity.
|
|
25
|
+
|
|
26
|
+
If you need help with the upgrade, we do have a Slack channel you can join: https://slack.karafka.io/
|
|
27
|
+
|
|
28
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
29
|
+
```
|
|
30
|
+
|
|
3
31
|
## 1.4.14 (2022-10-14)
|
|
4
32
|
- Fix `concurrent-ruby` missing as a dependency (Azdaroth)
|
|
5
33
|
- Warn about upcoming end of 1.4 support.
|
data/README.md
CHANGED
|
@@ -1,98 +1,21 @@
|
|
|
1
|
-
|
|
1
|
+
## CRITICAL NOTICE ON KARAFKA 1.4 reliability
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
[](http://badge.fury.io/rb/karafka)
|
|
5
|
-
[](https://slack.karafka.io)
|
|
3
|
+
You are looking at the `1.4` version of Karafka that is no longer supported.
|
|
6
4
|
|
|
7
|
-
|
|
5
|
+
Karafka `1.4` is no longer supported and contains critical errors, included but not limited to:
|
|
8
6
|
|
|
9
|
-
|
|
7
|
+
- Double processing of messages
|
|
8
|
+
- Skipping messages
|
|
9
|
+
- Hanging during processing
|
|
10
|
+
- Unexpectedly stopping message processing
|
|
11
|
+
- Failure to deliver messages to Kafka
|
|
12
|
+
- Resetting the consumer group and starting from the beginning
|
|
10
13
|
|
|
11
|
-
|
|
14
|
+
To resolve these issues, it is highly recommended to upgrade to Karafka 2.1 or higher.
|
|
12
15
|
|
|
13
|
-
|
|
14
|
-
# Define what topics you want to consume with which consumers
|
|
15
|
-
Karafka::App.consumer_groups.draw do
|
|
16
|
-
topic 'system_events' do
|
|
17
|
-
consumer EventsConsumer
|
|
18
|
-
end
|
|
19
|
-
end
|
|
16
|
+
If you want to ignore this message and continue, set the I_ACCEPT_CRITICAL_ERRORS_IN_KARAFKA_1_4 env variable to true.
|
|
20
17
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
# Example that utilizes ActiveRecord#insert_all and Karafka batch processing
|
|
24
|
-
def consume
|
|
25
|
-
# Store all of the incoming Kafka events locally in an efficient way
|
|
26
|
-
Event.insert_all params_batch.payloads
|
|
27
|
-
end
|
|
28
|
-
end
|
|
29
|
-
```
|
|
18
|
+
Apologies for any inconvenience caused by this release.
|
|
19
|
+
There is no other way to make sure, that you are notified about those issues and their severity.
|
|
30
20
|
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
Karafka not only handles incoming messages but also provides tools for building complex data-flow applications that receive and send messages.
|
|
34
|
-
|
|
35
|
-
## How does it work
|
|
36
|
-
|
|
37
|
-
Karafka provides a higher-level abstraction that allows you to focus on your business logic development, instead of focusing on implementing lower level abstraction layers. It provides developers with a set of tools that are dedicated for building multi-topic applications similar to how Rails applications are being built.
|
|
38
|
-
|
|
39
|
-
### Some things you might wonder about:
|
|
40
|
-
|
|
41
|
-
- You can integrate Karafka with **any** Ruby-based application.
|
|
42
|
-
- Karafka does **not** require Sidekiq or any other third party software (apart from Kafka itself).
|
|
43
|
-
- Karafka works with Ruby on Rails but it is a **standalone** framework that can work without it.
|
|
44
|
-
- Karafka has a **minimal** set of dependencies, so adding it won't be a huge burden for your already existing applications.
|
|
45
|
-
- Karafka processes can be executed for a **given subset** of consumer groups and/or topics, so you can fine tune it depending on your business logic.
|
|
46
|
-
|
|
47
|
-
Karafka based applications can be easily deployed to any type of infrastructure, including those based on:
|
|
48
|
-
|
|
49
|
-
* Heroku
|
|
50
|
-
* Capistrano
|
|
51
|
-
* Docker
|
|
52
|
-
* Terraform
|
|
53
|
-
|
|
54
|
-
## Support
|
|
55
|
-
|
|
56
|
-
Karafka has [Wiki pages](https://github.com/karafka/karafka/wiki) for almost everything and a pretty decent [FAQ](https://github.com/karafka/karafka/wiki/FAQ). It covers the whole installation, setup, and deployment along with other useful details on how to run Karafka.
|
|
57
|
-
|
|
58
|
-
If you have any questions about using Karafka, feel free to join our [Slack](https://slack.karafka.io) chat channel.
|
|
59
|
-
|
|
60
|
-
## Getting started
|
|
61
|
-
|
|
62
|
-
If you're completely new to the subject, you can start with our "Kafka on Rails" articles series, that will get you up and running with the terminology and basic ideas behind using Kafka:
|
|
63
|
-
|
|
64
|
-
- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 1 – Kafka basics and its advantages](https://mensfeld.pl/2017/11/kafka-on-rails-using-kafka-with-ruby-on-rails-part-1-kafka-basics-and-its-advantages/)
|
|
65
|
-
- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 2 – Getting started with Ruby and Kafka](https://mensfeld.pl/2018/01/kafka-on-rails-using-kafka-with-ruby-on-rails-part-2-getting-started-with-ruby-and-kafka/)
|
|
66
|
-
|
|
67
|
-
If you want to get started with Kafka and Karafka as fast as possible, then the best idea is to just clone our example repository:
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
git clone https://github.com/karafka/example-app ./example_app
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
then, just bundle install all the dependencies:
|
|
74
|
-
|
|
75
|
-
```bash
|
|
76
|
-
cd ./example_app
|
|
77
|
-
bundle install
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
and follow the instructions from the [example app Wiki](https://github.com/karafka/example-app/blob/master/README.md).
|
|
81
|
-
|
|
82
|
-
**Note**: you need to ensure, that you have Kafka up and running and you need to configure Kafka seed_brokers in the ```karafka.rb``` file.
|
|
83
|
-
|
|
84
|
-
If you need more details and know how on how to start Karafka with a clean installation, read the [Getting started page](https://github.com/karafka/karafka/wiki/Getting-started) section of our Wiki.
|
|
85
|
-
|
|
86
|
-
## References
|
|
87
|
-
|
|
88
|
-
* [Karafka framework](https://github.com/karafka/karafka)
|
|
89
|
-
* [Karafka GitHub Actions](https://github.com/karafka/karafka/actions)
|
|
90
|
-
* [Karafka Coditsu](https://app.coditsu.io/karafka/repositories/karafka)
|
|
91
|
-
|
|
92
|
-
## Note on contributions
|
|
93
|
-
|
|
94
|
-
First, thank you for considering contributing to the Karafka ecosystem! It's people like you that make the open source community such a great community!
|
|
95
|
-
|
|
96
|
-
Each pull request must pass all the RSpec specs, integration tests and meet our quality requirements.
|
|
97
|
-
|
|
98
|
-
Fork it, update and wait for the Github Actions results.
|
|
21
|
+
If you need help with the upgrade, we do have a Slack channel you can join: https://slack.karafka.io/
|
data/lib/karafka/version.rb
CHANGED
data/lib/karafka.rb
CHANGED
|
@@ -20,6 +20,36 @@
|
|
|
20
20
|
zeitwerk
|
|
21
21
|
].each(&method(:require))
|
|
22
22
|
|
|
23
|
+
unless ENV.key?('I_ACCEPT_CRITICAL_ERRORS_IN_KARAFKA_1_4')
|
|
24
|
+
3.times { puts }
|
|
25
|
+
puts '~' * 50
|
|
26
|
+
puts "~~~ \033[31mCRITICAL\033[0m NOTICE ON KARAFKA 1.4 reliability ~~~"
|
|
27
|
+
puts '~' * 50
|
|
28
|
+
puts
|
|
29
|
+
puts 'Karafka 1.4 is no longer supported and contains critical errors, included but not limited to:'
|
|
30
|
+
puts
|
|
31
|
+
puts ' - Double processing of messages'
|
|
32
|
+
puts ' - Skipping messages'
|
|
33
|
+
puts ' - Hanging during processing'
|
|
34
|
+
puts ' - Unexpectedly stopping message processing'
|
|
35
|
+
puts ' - Failure to deliver messages to Kafka'
|
|
36
|
+
puts ' - Resetting the consumer group and starting from the beginning'
|
|
37
|
+
puts
|
|
38
|
+
puts 'To resolve these issues, it is highly recommended to upgrade to Karafka 2.1 or higher.'
|
|
39
|
+
puts
|
|
40
|
+
puts 'If you want to ignore this message and continue, set the I_ACCEPT_CRITICAL_ERRORS_IN_KARAFKA_1_4 env variable to true.'
|
|
41
|
+
puts
|
|
42
|
+
puts 'Apologies for any inconvenience caused by this release.'
|
|
43
|
+
puts 'There is no other way to make sure, that you are notified about those issues and their severity.'
|
|
44
|
+
puts
|
|
45
|
+
puts 'If you need help with the upgrade, we do have a Slack channel you can join: https://slack.karafka.io/'
|
|
46
|
+
puts
|
|
47
|
+
puts '~' * 50
|
|
48
|
+
puts
|
|
49
|
+
|
|
50
|
+
raise RuntimeError
|
|
51
|
+
end
|
|
52
|
+
|
|
23
53
|
# Karafka library
|
|
24
54
|
module Karafka
|
|
25
55
|
extend Envlogic
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: karafka
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.4.
|
|
4
|
+
version: 1.4.15
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Maciej Mensfeld
|
|
@@ -37,7 +37,7 @@ cert_chain:
|
|
|
37
37
|
Qf04B9ceLUaC4fPVEz10FyobjaFoY4i32xRto3XnrzeAgfEe4swLq8bQsR3w/EF3
|
|
38
38
|
MGU0FeSV2Yj7Xc2x/7BzLK8xQn5l7Yy75iPF+KP3vVmDHnNl
|
|
39
39
|
-----END CERTIFICATE-----
|
|
40
|
-
date:
|
|
40
|
+
date: 2023-06-19 00:00:00.000000000 Z
|
|
41
41
|
dependencies:
|
|
42
42
|
- !ruby/object:Gem::Dependency
|
|
43
43
|
name: concurrent-ruby
|
|
@@ -311,7 +311,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
311
311
|
- !ruby/object:Gem::Version
|
|
312
312
|
version: '0'
|
|
313
313
|
requirements: []
|
|
314
|
-
rubygems_version: 3.3.
|
|
314
|
+
rubygems_version: 3.3.4
|
|
315
315
|
signing_key:
|
|
316
316
|
specification_version: 4
|
|
317
317
|
summary: Ruby based framework for working with Apache Kafka
|
metadata.gz.sig
CHANGED
|
Binary file
|