avro_pinions 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 25f8d3d09843b69a825498c6a3671a041c4ca3ff
4
+ data.tar.gz: 8f23561aa6d9f8b6047c373bb1d9d30a8579bb50
5
+ SHA512:
6
+ metadata.gz: f9fc2f77ea7b91f5ba95fc13bf7794c6c5b35affbf307e76cd5b242e0ba248f592725aa70eacdd70935fed8ca89faeae4a4c925b953aa9a8200c47cea745a0bf
7
+ data.tar.gz: 0f291763c9ab210b55918a5ca17579d18ab6797c545d013c0f2f7d24c835308302e7534df3577e8c498fda4e1eea6f32201f45436812bb27e22833ff831bb4dd
@@ -0,0 +1,12 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /Gemfile.lock
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /spec/reports/
9
+ /tmp/
10
+
11
+ # rspec failure tracking
12
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --format documentation
2
+ --color
@@ -0,0 +1,5 @@
1
+ sudo: false
2
+ language: ruby
3
+ rvm:
4
+ - 2.4.2
5
+ before_install: gem install bundler -v 1.14.6
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at jess@optoro.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in avro_pinions.gemspec
4
+ gemspec
@@ -0,0 +1,42 @@
1
+ # A sample Guardfile
2
+ # More info at https://github.com/guard/guard#readme
3
+
4
+ ## Uncomment and set this to only include directories you want to watch
5
+ # directories %w(app lib config test spec features) \
6
+ # .select{|d| Dir.exists?(d) ? d : UI.warning("Directory #{d} does not exist")}
7
+
8
+ ## Note: if you are using the `directories` clause above and you are not
9
+ ## watching the project directory ('.'), then you will want to move
10
+ ## the Guardfile to a watched dir and symlink it back, e.g.
11
+ #
12
+ # $ mkdir config
13
+ # $ mv Guardfile config/
14
+ # $ ln -s config/Guardfile .
15
+ #
16
+ # and, you'll have to watch "config/Guardfile" instead of "Guardfile"
17
+
18
+ # Note: The cmd option is now required due to the increasing number of ways
19
+ # rspec may be run, below are examples of the most common uses.
20
+ # * bundler: 'bundle exec rspec'
21
+ # * bundler binstubs: 'bin/rspec'
22
+ # * spring: 'bin/rspec' (This will use spring if running and you have
23
+ # installed the spring binstubs per the docs)
24
+ # * zeus: 'zeus rspec' (requires the server to be started separately)
25
+ # * 'just' rspec: 'rspec'
26
+
27
+ guard :rspec, cmd: "bundle exec rspec" do
28
+ require "guard/rspec/dsl"
29
+ dsl = Guard::RSpec::Dsl.new(self)
30
+
31
+ # Feel free to open issues for suggestions and improvements
32
+
33
+ # RSpec files
34
+ rspec = dsl.rspec
35
+ watch(rspec.spec_helper) { rspec.spec_dir }
36
+ watch(rspec.spec_support) { rspec.spec_dir }
37
+ watch(rspec.spec_files)
38
+
39
+ # Ruby files
40
+ ruby = dsl.ruby
41
+ dsl.watch_spec_files_for(ruby.lib_files)
42
+ end
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2017 Jess Szmajda
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
@@ -0,0 +1,116 @@
1
+ # AvroPinions
2
+
3
+ Opinionated stuff around AVRO in Ruby. Goals:
4
+
5
+ * Support some generic schema registry. You plug it in, conform to my API spec.
6
+ * Support [Single-Object Encoding](http://avro.apache.org/docs/1.8.2/spec.html#single_object_encoding) out of the box
7
+ * Writing for now. Maybe reading later.
8
+
9
+
10
+ ## Installation
11
+
12
+ Add this line to your application's Gemfile:
13
+
14
+ ```ruby
15
+ gem 'avro_pinions'
16
+ ```
17
+
18
+ And then execute:
19
+
20
+ $ bundle
21
+
22
+ Or install it yourself as:
23
+
24
+ $ gem install avro_pinions
25
+
26
+ ## Usage
27
+
28
+ Here's a full example:
29
+
30
+ ```ruby
31
+ # define a publisher. It has to have one method: `#publish`
32
+ class MyPublisher
33
+ # publish needs receive a topic and a message
34
+ # message is binary encoded AVRO data
35
+ def publish(topic, message)
36
+ # depending on how your transport mechanism works you may need to convert
37
+ # binary data to ascii characters or something. I recommend Base64 if you
38
+ # have to do that
39
+ extra_encoded = make_binary_data_wire_safe(message)
40
+ SomeTransferMechanismIOwn.send(topic, extra_encoded)
41
+ end
42
+ end
43
+
44
+ # configure AvroPinions
45
+ AvroPinions.configure({
46
+ wire_format: :single_object,
47
+ schema_registry: {
48
+ type: :file,
49
+ schema_path: 'doc/schemas'
50
+ },
51
+ publisher: MyPublisher.new
52
+ })
53
+
54
+ # define a message. They inherit from AvroPinions::Message
55
+ class SomeMessage < AvroPinions::Message
56
+ # define these constants or override `topic`, `schema`, and `namespace`
57
+ # methods
58
+ TOPIC = "some_topic"
59
+ SCHEMA = "SchemaName"
60
+ NAMESPACE = "com.company.namespace"
61
+
62
+ def initialize(data)
63
+ @data = data
64
+ end
65
+
66
+ # this method must return a json object that will be encoded with the AVRO
67
+ # schema
68
+ def record
69
+ { key: @data.value }
70
+ end
71
+ end
72
+
73
+ message = SomeMessage.new({ data... })
74
+ if message.valid?
75
+ AvroPinions.publish(message)
76
+ end
77
+ ```
78
+
79
+ Or you can provide things more manually if you want:
80
+ ```ruby
81
+ AvroPinions.configure({ wire_format: :single_object })
82
+
83
+ class Message < AvroPinions::Message
84
+ def initialize(avro_schema, data)
85
+ @avro_schema = avro_schema
86
+ @data = data
87
+ end
88
+
89
+ def record
90
+ @data
91
+ end
92
+
93
+ def avro_schema
94
+ @avro_schema
95
+ end
96
+ end
97
+
98
+ avro_schema = Avro::Schema.real_parse(json_schema_object)
99
+ Message.new(avro_schema, some_data).encode
100
+ ```
101
+
102
+ ## Development
103
+
104
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
105
+
106
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
107
+
108
+ ## Contributing
109
+
110
+ Bug reports and pull requests are welcome on GitHub at https://github.com/jszmajda/avro_pinions. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
111
+
112
+
113
+ ## License
114
+
115
+ The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
116
+
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
@@ -0,0 +1,33 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'avro_pinions/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "avro_pinions"
8
+ spec.version = AvroPinions::VERSION
9
+ spec.authors = ["Jess Szmajda"]
10
+ spec.email = ["jess@optoro.com"]
11
+
12
+ spec.summary = %q{Opinionated way to use AVRO in Ruby}
13
+ #spec.description = %q{TODO: Write a longer description or delete this line.}
14
+ spec.homepage = "https://github.com/jszmajda/avro_pinions"
15
+ spec.license = "MIT"
16
+
17
+ spec.files = `git ls-files -z`.split("\x0").reject do |f|
18
+ f.match(%r{^(test|spec|features)/})
19
+ end
20
+ spec.bindir = "exe"
21
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
22
+ spec.require_paths = ["lib"]
23
+
24
+ spec.add_dependency "avro", "~> 1.8"
25
+ spec.add_dependency "avro-patches", "~> 0.3"
26
+
27
+ spec.add_development_dependency "bundler", "~> 1.14"
28
+ spec.add_development_dependency "rake", "~> 10.0"
29
+ spec.add_development_dependency "rspec", "~> 3.0"
30
+ spec.add_development_dependency "pry"
31
+ spec.add_development_dependency "guard"
32
+ spec.add_development_dependency "guard-rspec"
33
+ end
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "avro_pinions"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start(__FILE__)
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,51 @@
1
+ require "ext/avro_patches"
2
+ require "avro_pinions/version"
3
+ require 'avro_pinions/configuration'
4
+ require 'avro_pinions/message'
5
+ require 'avro_pinions/schema_registry'
6
+ require 'avro_pinions/codec'
7
+
8
+ module AvroPinions
9
+ # Your code goes here...
10
+ module_function
11
+
12
+ def configuration
13
+ AvroPinions::Configuration.instance
14
+ end
15
+
16
+ def configure(messages)
17
+ if messages.respond_to?(:each_pair)
18
+ messages.each_pair do |key, value|
19
+ if value && configuration.respond_to?("#{key}=")
20
+ configuration.send("#{key}=", value)
21
+ elsif configuration.respond_to?(key)
22
+ configuration.send(key, *value)
23
+ end
24
+ end
25
+ else
26
+ configuration.send(messages)
27
+ end
28
+ end
29
+
30
+ def publisher
31
+ configuration.publisher
32
+ end
33
+
34
+ def schema_registry
35
+ configuration.schema_registry
36
+ end
37
+
38
+ def publish(message)
39
+ topic = message.topic
40
+ encoded = message.encode
41
+
42
+ publisher.publish(topic, encoded)
43
+ end
44
+
45
+ class NotFullyImplementedError < StandardError
46
+ end
47
+ class UnableToSerializeError < StandardError
48
+ end
49
+ class ConfigurationError < StandardError
50
+ end
51
+ end
@@ -0,0 +1,61 @@
1
+ module AvroPinions
2
+ class Codec
3
+
4
+ SUPPORTED_WIRE_FORMATS = [:single_object]
5
+
6
+ def self.pedantic_data(data)
7
+ if data.respond_to?(:each_pair)
8
+ data.each_pair.inject({}) do |result, (k,v)|
9
+ strrep = v.to_s
10
+ result[k.to_s] = if strrep.to_sym == v
11
+ strrep
12
+ else
13
+ v
14
+ end
15
+ result
16
+ end
17
+ else
18
+ data
19
+ end
20
+ end
21
+
22
+ attr_accessor :schema
23
+ def initialize(schema)
24
+ @schema = schema
25
+ end
26
+
27
+ def encode(data)
28
+ self.send("encode_as_#{wire_format}", data)
29
+ end
30
+
31
+ def wire_format
32
+ AvroPinions.configuration.wire_format
33
+ end
34
+
35
+ def encode_as_single_object(data)
36
+ soc = SingleObjectCodec.new(schema)
37
+ soc.encode(self.class.pedantic_data(data))
38
+ end
39
+
40
+ class SingleObjectCodec
41
+ attr_reader :schema
42
+ def initialize(schema)
43
+ @schema = schema
44
+ end
45
+
46
+ def encode(data)
47
+ stream = StringIO.new
48
+ writer = Avro::IO::DatumWriter.new(@schema)
49
+ encoder = Avro::IO::BinaryEncoder.new(stream)
50
+
51
+ smd = Avro::IO::SingleObjectMessage.new(@schema)
52
+ smd.header.each do |segment|
53
+ stream.write( segment.chr )
54
+ end
55
+ writer.write(data, encoder)
56
+
57
+ stream.string
58
+ end
59
+ end
60
+ end
61
+ end
@@ -0,0 +1,39 @@
1
+ require 'singleton'
2
+
3
+ module AvroPinions
4
+ class Configuration
5
+ include Singleton
6
+
7
+ attr_accessor :publisher, :schema_registry, :wire_format
8
+
9
+ def initialize
10
+ end
11
+
12
+ def publisher=(pub)
13
+ unless pub.respond_to?(:publish) && pub.method(:publish).arity == 2
14
+ raise InvalidConfiguration.new("Please define a publisher that responds to publish and receives a topic and message")
15
+ end
16
+ @publisher = pub
17
+ end
18
+
19
+ # :type gets pulled off and constructs the given registy
20
+ # all of the rest of the options get passed to an instance of hte type.kklk
21
+ def schema_registry=(options = {})
22
+ klass = options.delete(:type).to_s.split(/_/).map(&:capitalize).join
23
+ const = Object.const_get("AvroPinions::#{klass}Registry")
24
+ sr = const.new(options)
25
+ @schema_registry = sr
26
+ end
27
+
28
+ def wire_format=(format)
29
+ unless AvroPinions::Codec::SUPPORTED_WIRE_FORMATS.include?(format)
30
+ raise InvalidConfiguration.new("Please supply a valid wire format. Valid options are: #{AvroPinions::Codec::SUPPORTED_WIRE_FORMATS}")
31
+ end
32
+
33
+ @wire_format = format
34
+ end
35
+
36
+ class InvalidConfiguration < StandardError
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,57 @@
1
+ module AvroPinions
2
+ class Message
3
+ def topic
4
+ # self.class::SYMBOL looks up a class constant in a child class
5
+ if defined?(self.class::TOPIC)
6
+ self.class::TOPIC
7
+ else
8
+ raise AvroPinions::NotFullyImplementedError, 'No Topic defined'
9
+ end
10
+ end
11
+
12
+ # Schema is inferred to be the .avsc file in the schemas folder with the
13
+ # name as given
14
+ def schema
15
+ if defined?(self.class::SCHEMA)
16
+ self.class::SCHEMA
17
+ else
18
+ raise AvroPinions::NotFullyImplementedError, 'No Schema defined'
19
+ end
20
+ end
21
+
22
+ def namespace
23
+ if defined?(self.class::NAMESPACE)
24
+ self.class::NAMESPACE
25
+ else
26
+ raise AvroPinions::NotFullyImplementedError, 'No Namespace defined'
27
+ end
28
+ end
29
+
30
+ def record
31
+ raise AvroPinions::NotFullyImplementedError, 'record method not implemented'
32
+ end
33
+
34
+ def publish
35
+ AvroPinions.publisher.publish(topic, record)
36
+ end
37
+
38
+ def encode
39
+ codec.encode(record)
40
+ end
41
+
42
+ def valid?
43
+ Avro::Schema.validate(avro_schema, AvroPinions::Codec.pedantic_data(record))
44
+ end
45
+
46
+ def codec
47
+ @codec ||= AvroPinions::Codec.new(avro_schema)
48
+ end
49
+
50
+ def avro_schema
51
+ @avro_schema ||= AvroPinions.schema_registry.schema(schema, namespace)
52
+ end
53
+
54
+ class InvalidRecord < StandardError
55
+ end
56
+ end
57
+ end
@@ -0,0 +1,34 @@
1
+ require 'json'
2
+ require 'avro'
3
+ require 'avro-patches'
4
+
5
+ module AvroPinions
6
+ class FileRegistry < SchemaRegistry
7
+ def initialize(options = {})
8
+ super()
9
+ @path = options[:schema_path] or raise AvroPinions::ConfigurationError.new("please define the path to load schemas from")
10
+ end
11
+
12
+ def load_schema(schema_name, namespace)
13
+ data = File.read(file_path(schema_name, namespace))
14
+ json = JSON.parse(data)
15
+ Avro::Schema.real_parse(json)
16
+ end
17
+
18
+ def file_path(schema_name, namespace)
19
+ schema_file = "#{schema_name}.avsc"
20
+ boring_filename = File.join(@path, schema_file)
21
+ if File.exists?(boring_filename)
22
+ boring_filename
23
+ else
24
+ fancy_filename = File.join(@path, *namespace.split(/\./), schema_file)
25
+ if File.exists?(fancy_filename)
26
+ fancy_filename
27
+ else
28
+ raise "File not found for namespace #{namespace} and schema #{schema_name}. Tried #{fancy_filename} and #{boring_filename}"
29
+ end
30
+ end
31
+ end
32
+
33
+ end
34
+ end
@@ -0,0 +1,38 @@
1
+ module AvroPinions
2
+ class SchemaRegistry
3
+ def initialize
4
+ @cache = {}
5
+ end
6
+
7
+ def schema(schema_name, namespace = "")
8
+ with_cache(schema_name, namespace) do
9
+ load_schema(schema_name, namespace)
10
+ end
11
+ end
12
+
13
+ def canonical_name(schema_name, namespace = "")
14
+ if namespace.length > 0
15
+ "#{namespace}.#{schema_name}"
16
+ else
17
+ schema_name
18
+ end
19
+ end
20
+
21
+ def load_schema(schema_name, namespace)
22
+ raise AvroPinions::NotFullyImplementedError.new("Please define how to load schemas")
23
+ end
24
+
25
+ def with_cache(schema_name, namespace, &block)
26
+ key = canonical_name(schema_name, namespace)
27
+
28
+ unless @cache.has_key?(key)
29
+ @cache[key] = block.call
30
+ end
31
+
32
+ @cache[key]
33
+ end
34
+ end
35
+ end
36
+
37
+ # has to go below because these classes inherit. Yeah..
38
+ require 'avro_pinions/schema_registries/file_registry'
@@ -0,0 +1,3 @@
1
+ module AvroPinions
2
+ VERSION = "1.0.0"
3
+ end
@@ -0,0 +1,61 @@
1
+ # I have a PR open on the avro core to include these capabilities, but for now
2
+ # whatever I guess..
3
+ require 'avro'
4
+ require 'avro-patches'
5
+ module Avro
6
+ class Schema
7
+
8
+ CRC_EMPTY = 0xc15d213aa4d7a795
9
+
10
+ # The java library caches this value after initialized, so this pattern
11
+ # mimics that.
12
+ @@fp_table = nil
13
+ def initFPTable
14
+ @@fp_table = Array.new(256)
15
+ 256.times do |i|
16
+ fp = i
17
+ 8.times do |j|
18
+ fp = (fp >> 1) ^ ( CRC_EMPTY & -( fp & 1 ) )
19
+ end
20
+ @@fp_table[i] = fp
21
+ end
22
+ end
23
+
24
+ def crc_64_avro_fingerprint
25
+ parsing_form = Avro::SchemaNormalization.to_parsing_form(self)
26
+ data_bytes = parsing_form.unpack("C*")
27
+
28
+ initFPTable unless @@fp_table
29
+
30
+ fp = CRC_EMPTY
31
+ data_bytes.each do |b|
32
+ fp = (fp >> 8) ^ @@fp_table[ (fp ^ b) & 0xff ]
33
+ end
34
+ fp
35
+ end
36
+
37
+ end
38
+
39
+ module IO
40
+ class SingleObjectMessage
41
+ def initialize(schema)
42
+ @schema = schema
43
+ end
44
+
45
+ SINGLE_OBJECT_MAGIC_NUMBER = [0xC3, 0x01]
46
+ def header
47
+ [SINGLE_OBJECT_MAGIC_NUMBER, schema_fingerprint].flatten
48
+ end
49
+ def schema_fingerprint
50
+ working = @schema.crc_64_avro_fingerprint
51
+ bytes = Array.new(8)
52
+ 8.times do |i|
53
+ bytes[7 - i] = (working & 0xff)
54
+ working = working >> 8
55
+ end
56
+ bytes
57
+ end
58
+ end
59
+ end
60
+ end
61
+
metadata ADDED
@@ -0,0 +1,176 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: avro_pinions
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Jess Szmajda
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2017-12-20 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: avro
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.8'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.8'
27
+ - !ruby/object:Gem::Dependency
28
+ name: avro-patches
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '0.3'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '0.3'
41
+ - !ruby/object:Gem::Dependency
42
+ name: bundler
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '1.14'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '1.14'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rake
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '10.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '10.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: rspec
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '3.0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '3.0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: pry
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: guard
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ - !ruby/object:Gem::Dependency
112
+ name: guard-rspec
113
+ requirement: !ruby/object:Gem::Requirement
114
+ requirements:
115
+ - - ">="
116
+ - !ruby/object:Gem::Version
117
+ version: '0'
118
+ type: :development
119
+ prerelease: false
120
+ version_requirements: !ruby/object:Gem::Requirement
121
+ requirements:
122
+ - - ">="
123
+ - !ruby/object:Gem::Version
124
+ version: '0'
125
+ description:
126
+ email:
127
+ - jess@optoro.com
128
+ executables: []
129
+ extensions: []
130
+ extra_rdoc_files: []
131
+ files:
132
+ - ".gitignore"
133
+ - ".rspec"
134
+ - ".travis.yml"
135
+ - CODE_OF_CONDUCT.md
136
+ - Gemfile
137
+ - Guardfile
138
+ - LICENSE.txt
139
+ - README.md
140
+ - Rakefile
141
+ - avro_pinions.gemspec
142
+ - bin/console
143
+ - bin/setup
144
+ - lib/avro_pinions.rb
145
+ - lib/avro_pinions/codec.rb
146
+ - lib/avro_pinions/configuration.rb
147
+ - lib/avro_pinions/message.rb
148
+ - lib/avro_pinions/schema_registries/file_registry.rb
149
+ - lib/avro_pinions/schema_registry.rb
150
+ - lib/avro_pinions/version.rb
151
+ - lib/ext/avro_patches.rb
152
+ homepage: https://github.com/jszmajda/avro_pinions
153
+ licenses:
154
+ - MIT
155
+ metadata: {}
156
+ post_install_message:
157
+ rdoc_options: []
158
+ require_paths:
159
+ - lib
160
+ required_ruby_version: !ruby/object:Gem::Requirement
161
+ requirements:
162
+ - - ">="
163
+ - !ruby/object:Gem::Version
164
+ version: '0'
165
+ required_rubygems_version: !ruby/object:Gem::Requirement
166
+ requirements:
167
+ - - ">="
168
+ - !ruby/object:Gem::Version
169
+ version: '0'
170
+ requirements: []
171
+ rubyforge_project:
172
+ rubygems_version: 2.6.10
173
+ signing_key:
174
+ specification_version: 4
175
+ summary: Opinionated way to use AVRO in Ruby
176
+ test_files: []