karafka 2.0.0.rc6 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +86 -2
- data/Gemfile.lock +1 -1
- data/README.md +2 -2
- data/lib/karafka/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +3 -3
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 06f96b3ba14b7910d3cb23a90bdc135927f1483d5af4d079d59b3d6940d391b9
|
4
|
+
data.tar.gz: 2d936d4ddac360e229004c05f6ee51d004018e3c4cf4203067a99e6df93df92e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: caeb9bcf1f0301f31442025176f3da631aa8ff9c21164e112b9ce3112372309d4b7fe59863cf7cf511e6a6bc4448f61448f3bc2024b101d4ae2668a358e5bbe9
|
7
|
+
data.tar.gz: cfc422fae74512c2142ef36f9d4b5c3024bf318d79dc83456e27553aebc177cec7a3772f6378160c8ecdf5a3b2626d862ba91d823fae6e30cbb593d4198474a9
|
checksums.yaml.gz.sig
CHANGED
Binary file
|
data/CHANGELOG.md
CHANGED
@@ -1,7 +1,91 @@
|
|
1
1
|
# Karafka framework changelog
|
2
2
|
|
3
|
+
## 2.0.0 (2022-08-5)
|
4
|
+
|
5
|
+
This changelog describes changes between `1.4` and `2.0`. Please refer to appropriate release notes for changes between particular `rc` releases.
|
6
|
+
|
7
|
+
Karafka 2.0 is a **major** rewrite that brings many new things to the table but also removes specific concepts that happened not to be as good as I initially thought when I created them.
|
8
|
+
|
9
|
+
Please consider getting a Pro version if you want to **support** my work on the Karafka ecosystem!
|
10
|
+
|
11
|
+
For anyone worried that I will start converting regular features into Pro: This will **not** happen. Anything free and fully OSS in Karafka 1.4 will **forever** remain free. Most additions and improvements to the ecosystem are to its free parts. Any feature that is introduced as a free and open one will not become paid.
|
12
|
+
|
13
|
+
### Additions
|
14
|
+
|
15
|
+
This section describes **new** things and concepts introduced with Karafka 2.0.
|
16
|
+
|
17
|
+
Karafka 2.0:
|
18
|
+
|
19
|
+
- Introduces multi-threaded support for [concurrent work](https://github.com/karafka/karafka/wiki/Concurrency-and-multithreading) consumption for separate partitions as well as for single partition work via [Virtual Partitions](https://github.com/karafka/karafka/wiki/Pro-Virtual-Partitions).
|
20
|
+
- Introduces [Active Job adapter](https://github.com/karafka/karafka/wiki/Active-Job) for using Karafka as a jobs backend with Ruby on Rails Active Job.
|
21
|
+
- Introduces fully automatic integration end-to-end [test suite](https://github.com/karafka/karafka/tree/master/spec/integrations) that checks any case I could imagine.
|
22
|
+
- Introduces [Virtual Partitions](https://github.com/karafka/karafka/wiki/Pro-Virtual-Partitions) for ability to parallelize work of a single partition.
|
23
|
+
- Introduces [Long-Running Jobs](https://github.com/karafka/karafka/wiki/Pro-Long-Running-Jobs) to allow for work that would otherwise exceed the `max.poll.interval.ms`.
|
24
|
+
- Introduces the [Enhanced Scheduler](https://github.com/karafka/karafka/wiki/Pro-Enhanced-Scheduler) that uses a non-preemptive LJF (Longest Job First) algorithm instead of a a FIFO (First-In, First-Out) one.
|
25
|
+
- Introduces [Enhanced Active Job adapter](https://github.com/karafka/karafka/wiki/Pro-Enhanced-Active-Job) that is optimized and allows for strong ordering of jobs and more.
|
26
|
+
- Introduces seamless [Ruby on Rails integration](https://github.com/karafka/karafka/wiki/Integrating-with-Ruby-on-Rails-and-other-frameworks) via `Rails::Railte` without need for any extra configuration.
|
27
|
+
- Provides `#revoked` [method](https://github.com/karafka/karafka/wiki/Consuming-messages#shutdown-and-partition-revocation-handlers) for taking actions upon topic revocation.
|
28
|
+
- Emits underlying async errors emitted from `librdkafka` via the standardized `error.occurred` [monitor channel](https://github.com/karafka/karafka/wiki/Error-handling-and-back-off-policy#error-tracking).
|
29
|
+
- Replaces `ruby-kafka` with `librdkafka` as an underlying driver.
|
30
|
+
- Introduces official [EOL policies](https://github.com/karafka/karafka/wiki/Versions-Lifecycle-and-EOL).
|
31
|
+
- Introduces [benchmarks](https://github.com/karafka/karafka/tree/master/spec/benchmarks) that can be used to profile Karafka.
|
32
|
+
- Introduces a requirement that the end user code **needs** to be [thread-safe](https://github.com/karafka/karafka/wiki/FAQ#does-karafka-require-gems-to-be-thread-safe).
|
33
|
+
- Introduces a [Pro subscription](https://github.com/karafka/karafka/wiki/Build-vs.-Buy) with a [commercial license](https://github.com/karafka/karafka/blob/master/LICENSE-COMM) to fund further ecosystem development.
|
34
|
+
|
35
|
+
### Deletions
|
36
|
+
|
37
|
+
This section describes things that are **no longer** part of the Karafka ecosystem.
|
38
|
+
|
39
|
+
Karafka 2.0:
|
40
|
+
|
41
|
+
- Removes topics mappers concept completely.
|
42
|
+
- Removes pidfiles support.
|
43
|
+
- Removes daemonization support.
|
44
|
+
- Removes support for using `sidekiq-backend` due to introduction of [multi-threading](https://github.com/karafka/karafka/wiki/Concurrency-and-multithreading).
|
45
|
+
- Removes the `Responders` concept in favour of WaterDrop producer usage.
|
46
|
+
- Removes completely all the callbacks in favour of finalizer method `#shutdown`.
|
47
|
+
- Removes single message consumption mode in favour of [documentation](https://github.com/karafka/karafka/wiki/Consuming-messages#one-at-a-time) on how to do it easily by yourself.
|
48
|
+
|
49
|
+
### Changes
|
50
|
+
|
51
|
+
This section describes things that were **changed** in Karafka but are still present.
|
52
|
+
|
53
|
+
Karafka 2.0:
|
54
|
+
|
55
|
+
- Uses only instrumentation that comes from Karafka. This applies also to notifications coming natively from `librdkafka`. They are now piped through Karafka prior to being dispatched.
|
56
|
+
- Integrates WaterDrop `2.x` tightly with autoconfiguration inheritance and an option to redefine it.
|
57
|
+
- Integrates with the `karafka-testing` gem for RSpec that also has been updated.
|
58
|
+
- Updates `cli info` to reflect the `2.0` details.
|
59
|
+
- Stops validating `kafka` configuration beyond minimum as the rest is handled by `librdkafka`.
|
60
|
+
- No longer uses `dry-validation`.
|
61
|
+
- No longer uses `dry-monitor`.
|
62
|
+
- No longer uses `dry-configurable`.
|
63
|
+
- Lowers general external dependencies three **heavily**.
|
64
|
+
- Renames `Karafka::Params::BatchMetadata` to `Karafka::Messages::BatchMetadata`.
|
65
|
+
- Renames `Karafka::Params::Params` to `Karafka::Messages::Message`.
|
66
|
+
- Renames `#params_batch` in consumers to `#messages`.
|
67
|
+
- Renames `Karafka::Params::Metadata` to `Karafka::Messages::Metadata`.
|
68
|
+
- Renames `Karafka::Fetcher` to `Karafka::Runner` and align notifications key names.
|
69
|
+
- Renames `StdoutListener` to `LoggerListener`.
|
70
|
+
- Reorganizes [monitoring and logging](https://github.com/karafka/karafka/wiki/Monitoring-and-logging) to match new concepts.
|
71
|
+
- Notifies on fatal worker processing errors.
|
72
|
+
- Contains updated install templates for Rails and no-non Rails.
|
73
|
+
- Changes how the routing style (`0.5`) behaves. It now builds a single consumer group instead of one per topic.
|
74
|
+
- Introduces changes that will allow me to build full web-UI in the upcoming `2.1`.
|
75
|
+
- Contains updated example apps.
|
76
|
+
- Standardizes error hooks for all error reporting (`error.occurred`).
|
77
|
+
- Changes license to `LGPL-3.0`.
|
78
|
+
- Introduces a `karafka-core` dependency that contains common code used across the ecosystem.
|
79
|
+
- Contains updated [wiki](https://github.com/karafka/karafka/wiki) on everything I could think of.
|
80
|
+
|
81
|
+
### What's ahead
|
82
|
+
|
83
|
+
Karafka 2.0 is just the beginning.
|
84
|
+
|
85
|
+
There are several things in the plan already for 2.1 and beyond, including a web dashboard, at-rest encryption, transactions support, and more.
|
86
|
+
|
3
87
|
## 2.0.0.rc6 (2022-08-05)
|
4
|
-
- Update licenser to use a gem based approach based on `karafka
|
88
|
+
- Update licenser to use a gem based approach based on `karafka-license`.
|
5
89
|
- Do not mark intermediate jobs as consumed when Karafka runs Enhanced Active Job with Virtual Partitions.
|
6
90
|
- Improve development experience by adding fast cluster state changes refresh (#944)
|
7
91
|
- Improve the license loading.
|
@@ -59,7 +143,7 @@
|
|
59
143
|
- Add more integration specs related to polling limits.
|
60
144
|
- Remove auto-detection of re-assigned partitions upon rebalance as for too fast rebalances it could not be accurate enough. It would also mess up in case of rebalances that would happen right after a `#seek` was issued for a partition.
|
61
145
|
- Optimize the removal of pre-buffered lost partitions data.
|
62
|
-
- Always
|
146
|
+
- Always run `#revoked` when rebalance with revocation happens.
|
63
147
|
- Evict executors upon rebalance, to prevent race-conditions.
|
64
148
|
- Align topics names for integration specs.
|
65
149
|
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -4,7 +4,7 @@
|
|
4
4
|
[![Gem Version](https://badge.fury.io/rb/karafka.svg)](http://badge.fury.io/rb/karafka)
|
5
5
|
[![Join the chat at https://slack.karafka.io](https://raw.githubusercontent.com/karafka/misc/master/slack.svg)](https://slack.karafka.io)
|
6
6
|
|
7
|
-
**Note**: All of the documentation here refers to Karafka `2.0.0
|
7
|
+
**Note**: All of the documentation here refers to Karafka `2.0.0` or higher. If you are looking for the documentation for Karafka `1.4`, please click [here](https://github.com/karafka/wiki/tree/1.4).
|
8
8
|
|
9
9
|
## About Karafka
|
10
10
|
|
@@ -55,7 +55,7 @@ We also maintain many [integration specs](https://github.com/karafka/karafka/tre
|
|
55
55
|
1. Add and install Karafka:
|
56
56
|
|
57
57
|
```bash
|
58
|
-
bundle add karafka
|
58
|
+
bundle add karafka
|
59
59
|
|
60
60
|
bundle exec karafka install
|
61
61
|
```
|
data/lib/karafka/version.rb
CHANGED
data.tar.gz.sig
CHANGED
Binary file
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: karafka
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0.0
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Maciej Mensfeld
|
@@ -287,9 +287,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
287
287
|
version: 2.7.0
|
288
288
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
289
289
|
requirements:
|
290
|
-
- - "
|
290
|
+
- - ">="
|
291
291
|
- !ruby/object:Gem::Version
|
292
|
-
version:
|
292
|
+
version: '0'
|
293
293
|
requirements: []
|
294
294
|
rubygems_version: 3.3.7
|
295
295
|
signing_key:
|
metadata.gz.sig
CHANGED
Binary file
|