deimos-temp-fork 0.0.1

data/.ruby-gemset

@@ -0,0 +1 @@
+deimos

data/.ruby-version

@@ -0,0 +1 @@
+2.5.3

data/CHANGELOG.md

@@ -0,0 +1,349 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## UNRELEASED
+### Roadmap :car:
+
+- TestHelper does not automatically reset Deimos config before each test. [#120](https://github.com/flipp-oss/deimos/pull/120).
+  **Please note that this is a breaking change**
+
+## 1.10.2 - 2021-07-20
+
+- ### Fixes :wrench:
+
+- Fixed issue where producers would stay in an error state after e.g. authorization failures for one topic that wouldn't apply to other topics.
+
+## 1.10.1 - 2021-06-21
+
+- ### Fixes :wrench:
+
+- Fixed crash when trying to decode a nil payload (e.g. during instrumentation of `send_produce_error`.)
+
+## 1.10.0 - 2021-03-22
+
+- ### Roadmap :car:
+
+- Extracted the configuration piece into a separate gem, [fig_tree](https://www.github.com/flipp-oss/fig_tree).
+- Added a `save_record` method to ActiveRecordConsumer in case calling code wants to work with the record before saving.
+
+- ### Fixes :wrench:
+
+- Fixed a regression where the default values for consumer / Phobos listener configs were not correct (they were all nil). This is technically a breaking change, but it puts the configs back the way they were at version 1.4 and matches the documentation.
+
+## 1.9.2 - 2021-01-29
+
+- ### Fixes :wrench:
+
+- Fix for `uninitialized constant ActiveSupport::Autoload` in certain circumstances
+- Removed unnecessary monkey patch which was crashing on newer versions of ruby-kafka
+
+## 1.9.0 - 2021-01-28
+
+- ### Roadmap :car:
+
+- Bumped the version of ruby-kafka to latest
+
+- ### Fixes :wrench:
+
+- Prevents DB Poller from reconnecting to DB if there is an open transaction
+- Replaces `before` by `prepend_before` for more consistent test setups.
+- Adds validation in the `kafka_producers` method (fixes [#90](https://github.com/flipp-oss/deimos/issues/90))
+
+## 1.8.7 - 2021-01-14
+
+- ### Roadmap :car:
+- Update Phobos version to allow version 1.9 or 2.x.
+
+## 1.8.6 - 2021-01-14
+
+- ### Fixes :wrench:
+- Fix for configuration bug with Ruby 3.0 (** instead of passing hash)
+
+## 1.8.5 - 2021-01-13
+
+- ### Fixes :wrench:
+- Fixes for Rails 6.1 (remove usage of `update_attributes!`)
+
+## 1.8.4 - 2020-12-02
+
+### Features :star:
+- Add overridable "process_message?" method to ActiveRecordConsumer to allow for skipping of saving/updating records
+
+### Fixes :wrench:
+
+- Do not apply type coercion to `timestamp-millis` and `timestamp-micros` logical types (fixes [#97](https://github.com/flipp-oss/deimos/issues/97))
+
+## 1.8.3 - 2020-11-18
+
+### Fixes :wrench:
+- Do not resend already sent messages when splitting up batches
+  (fixes [#24](https://github.com/flipp-oss/deimos/issues/24))
+- KafkaSource crashing on bulk-imports if import hooks are disabled
+  (fixes [#73](https://github.com/flipp-oss/deimos/issues/73))
+- #96 Use string-safe encoding for partition keys
+- Retry on offset seek failures in inline consumer
+  (fixes [#5](Inline consumer should use retries when seeking))
+
+## 1.8.2 - 2020-09-25
+
+### Features :star:
+- Add "disabled" config field to consumers to allow disabling
+  individual consumers without having to comment out their
+  entries and possibly affecting unit tests.
+
+### Fixes :wrench:
+- Prepend topic_prefix while encoding messages
+  (fixes [#37](https://github.com/flipp-oss/deimos/issues/37))
+- Raise error if producing without a topic
+  (fixes [#50](https://github.com/flipp-oss/deimos/issues/50))
+- Don't try to load producers/consumers when running rake tasks involving webpacker or assets
+
+## 1.8.2-beta2 - 2020-09-15
+
+### Features :star:
+
+- Add details on using schema backend directly in README.
+- Default to the provided schema if topic is not provided when
+  encoding to `AvroSchemaRegistry`.
+- Add mapping syntax for the `schema` call in `SchemaControllerMixin`.
+
+## 1.8.2-beta1 - 2020-09-09
+
+### Features :star:
+
+- Added the ability to specify the topic for `publish`
+and `publish_list` in a producer
+
+## 1.8.1-beta9 - 2020-08-27
+
+### Fixes :wrench:
+- Moved the TestHelpers hook to `before(:suite)` to allow for
+  overriding e.g. in integration tests.
+
+## 1.8.1-beta7 - 2020-08-25
+
+### Fixes :wrench:
+- Fix for crash when sending pending metrics with DB producer.
+- Fix for compacting messages if all the keys are already unique
+  (fixes [#75](https://github.com/flipp-oss/deimos/issues/75))
+
+## 1.8.1-beta6 - 2020-08-13
+
+### Fixes :wrench:
+
+- Fix for consuming nil payloads with Ruby 2.3.
+
+## 1.8.1-beta5 - 2020-08-13
+
+### Fixes :wrench:
+- Fix regression bug which introduces backwards incompatibility
+  with ActiveRecordProducer's `record_attributes` method.
+
+## 1.8.1-beta4 - 2020-08-12
+
+### Fixes :wrench:
+- Fix regression bug where arrays were not being encoded
+
+## 1.8.1-beta3 - 2020-08-05
+
+### Fixes :wrench:
+- Simplify decoding messages and handle producer not found
+- Consolidate types in sub-records recursively
+  (fixes [#72](https://github.com/flipp-oss/deimos/issues/72))
+
+## 1.8.1-beta2 - 2020-07-28
+
+### Features :star:
+- Add `SchemaControllerMixin` to encode and decode schema-encoded
+  payloads in Rails controllers.
+
+## 1.8.1-beta1 - 2020-07-22
+
+### Fixes :wrench:
+- Retry deleting messages without resending the batch if the
+  delete fails (fixes [#34](https://github.com/flipp-oss/deimos/issues/34))
+- Delete messages in batches rather than all at once to
+  cut down on the chance of a deadlock.
+
+### Features :star:
+- Add `last_processed_at` to `kafka_topic_info` to ensure
+  that wait metrics are accurate in cases where records
+  get created with an old `created_at` time (e.g. for
+  long-running transactions).
+- Add generator for ActiveRecord models and migrations (fixes [#6](https://github.com/flipp-oss/deimos/issues/6))
+
+## 1.8.0-beta2 - 2020-07-08
+
+### Fixes :wrench:
+- Fix crash with batch consumption due to not having ActiveSupport::Concern
+
+### Features :star:
+- Add `first_offset` to the metadata sent via the batch
+
+## 1.8.0-beta1 - 2020-07-06
+### Features :star:
+- Added `ActiveRecordConsumer` batch mode
+
+### Fixes :wrench:
+- Fixes `send_produce_error` to decode `failed_messages` with built-in decoder. 
+- Lag calculation can be incorrect if no messages are being consumed.
+- Fixed bug where printing messages on a MessageSizeTooLarge
+  error didn't work.
+
+### Roadmap
+- Moved SignalHandler and Executor to the `sigurd` gem.
+
+## 1.7.0-beta1 - 2020-05-12
+### Features :star:
+- Added the DB Poller feature / process.
+
+## 1.6.4 - 2020-05-11
+- Fixed the payload logging fix for errored messages as well.
+
+## 1.6.3 - 2020-05-04
+### Fixes :wrench:
+- Fixed the payload logging fix.
+
+## 1.6.2 - 2020-05-04
+### Fixes :wrench:
+- When saving records via `ActiveRecordConsumer`, update `updated_at` to today's time
+  even if nothing else was saved.
+- When logging payloads and metadata, decode them first.
+- Fixes bug in `KafkaSource` that crashes when importing a mix of existing and new records with the `:on_duplicate_key_update` option.
+
+## [1.6.1] - 2020-04-20
+### Fixes :wrench:
+- Re-consuming a message after crashing would try to re-decode message keys.
+
+# [1.6.0] - 2020-03-05
+### Roadmap :car:
+- Removed `was_message_sent?` method from `TestHelpers`.
+
+# [1.6.0-beta1] - 2020-02-05
+### Roadmap :car:
+- Updated dependency for Phobos to 1.9.0-beta3. This ensures compatibility with
+  Phobos 2.0.
+### Fixes :wrench:
+- Fixed RSpec warning when using `test_consume_invalid_message`.
+
+# [1.5.0-beta2] - 2020-01-17
+### Roadmap :car:
+- Added schema backends, which should simplify Avro encoding and make it
+  more flexible for unit tests and local development.
+### Features :star:
+- Add `:test` producer backend which replaces the existing TestHelpers
+  functionality of writing messages to an in-memory hash.
+
+# [1.4.0-beta7] - 2019-12-16
+### Fixes :wrench:
+- Clone loggers when assigning to multiple levels.
+
+# [1.4.0-beta6] - 2019-12-16
+### Features :star:
+- Added default for max_bytes_per_partition.
+
+# [1.4.0-beta4] - 2019-11-26
+### Features :star:
+- Added `define_settings` to define settings without invoking callbacks.
+
+# [1.4.0-beta2] - 2019-11-22
+### Fixes :wrench:
+- Settings with default_proc were being called immediately
+  instead of being lazy-evaluated.
+
+# [1.4.0-beta1] - 2019-11-22
+### Roadmap :car:
+- Complete revamp of configuration method.
+
+# [1.3.0-beta5] - 2020-01-14
+### Features :star:
+- Added `db_producer.insert` and `db_producer.process` metrics.
+
+# [1.3.0-beta4] - 2019-12-02
+### Fixes :wrench:
+- Fixed bug where by running `rake deimos:start` without
+  specifying a producer backend would crash.
+
+# [1.3.0-beta3] - 2019-11-26
+### Fixes :wrench:
+- Fixed bug in TestHelpers where key_decoder was not stubbed out.
+
+# [1.3.0-beta2] - 2019-11-22
+### Fixes :wrench:
+- Fixed bug where consumers would require a key config in all cases
+  even though it's optional if they don't use keys.
+
+# [1.3.0-beta1] - 2019-11-21
+### Features :star:
+- Added `fetch_record` and `assign_key` methods to ActiveRecordConsumer.
+
+# [1.2.0-beta1] - 2019-09-12
+### Features :star:
+- Added `fatal_error` to both global config and consumer classes.
+- Changed `pending_db_messages_max_wait` metric to send per topic.
+- Added config to compact messages in the DB producer.
+- Added config to log messages in the DB producer.
+- Added config to provide a separate logger to the DB producer.
+
+# [1.1.0-beta2] - 2019-09-11
+### Fixes :wrench:
+- Fixed bug where ActiveRecordConsumer was not using `unscoped` to update
+  via primary key and causing duplicate record errors.
+
+# [1.1.0-beta1] - 2019-09-10
+### Features :star:
+- Added BatchConsumer.
+
+## [1.0.0] - 2019-09-03
+### Roadmap :car:
+- Official release of Deimos 1.0!
+
+## [1.0.0-beta26] - 2019-08-29
+- Recover from Kafka::MessageSizeTooLarge in the DB producer.
+- Shut down sync producers correctly when persistent_connections is true.
+- Notify when messages fail to produce in the DB producer.
+- Delete messages on failure and rely on notification.
+
+## [1.0.0-beta25] - 2019-08-28
+- Fix bug where crashing would cause producers to stay disabled
+
+## [1.0.0-beta24] - 2019-08-26
+- Reconnect DB backend if database goes away.
+- Sleep only 5 seconds between attempts instead of using exponential backoff.
+- Fix for null payload being Avro-encoded.
+
+## [1.0.0-beta23] - 2019-08-22
+- Fix bug where nil payloads were not being saved to the DB.
+- Fix DB producer rake task looking at THREADS env var instead of THREAD_COUNT.
+- Debug messages in the DB producer if debug logs are turned on.
+- Changed logger in specs to info.
+
+## [1.0.0-beta22] - 2019-08-09
+- Add `pending_db_messages_max_wait` metric for the DB producer.
+- Fix mock metrics to allow optional option hashes.
+
+## [1.0.0-beta21] - 2019-08-08
+- Handle Phobos `persistent_connections` setting in handling buffer overflows
+
+## [1.0.0-beta20] - 2019-08-07
+- Catch buffer overflows when producing via the DB producer and split the
+  batch up.
+
+## [1.0.0-beta19] - 2019-08-06
+- Fix for DB producer crashing on error in Rails 3.
+
+## [1.0.0-beta18] - 2019-08-02
+- Fixed crash when sending metrics in a couple of places.
+
+## [1.0.0-beta17] - 2019-07-31
+- Added `rails deimos:db_producer` rake task.
+- Fixed the DB producer so it runs inline instead of on a separate thread.
+  Calling code should run it on a thread manually if that is the desired
+  behavior.
+
+## [1.0.0-beta15] - 2019-07-08
+- Initial release.

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,77 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies within all project spaces, and it also applies when
+an individual is representing the project or its community in public spaces.
+Examples of representing a project or community include using an official
+project e-mail address, posting via an official social media account, or acting
+as an appointed representative at an online or offline event. Representation of
+a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq
+

data/Dockerfile

@@ -0,0 +1,23 @@
+FROM ruby:2.5.5-stretch
+
+RUN apt-get update && \
+  apt-get -y install curl git openssh-client openssl nodejs awscli
+RUN apt-get install -yq libpq-dev net-tools mysql-client wait-for-it
+ENV DOCKERIZE_VERSION v0.6.1
+RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
+    && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
+    && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
+
+WORKDIR /opt/deimos
+COPY deimos.gemspec /opt/deimos/deimos.gemspec
+COPY lib/deimos/version.rb /opt/deimos/lib/deimos/version.rb
+COPY Gemfile /opt/deimos/Gemfile
+COPY Gemfile.lock /opt/deimos/Gemfile.lock
+
+RUN bundle install
+
+ADD . .
+
+ENTRYPOINT ["bundle", "exec"]
+
+CMD ["bundle", "exec", "rspec"]