avro_pinions 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 25f8d3d09843b69a825498c6a3671a041c4ca3ff
4
+ data.tar.gz: 8f23561aa6d9f8b6047c373bb1d9d30a8579bb50
5
+ SHA512:
6
+ metadata.gz: f9fc2f77ea7b91f5ba95fc13bf7794c6c5b35affbf307e76cd5b242e0ba248f592725aa70eacdd70935fed8ca89faeae4a4c925b953aa9a8200c47cea745a0bf
7
+ data.tar.gz: 0f291763c9ab210b55918a5ca17579d18ab6797c545d013c0f2f7d24c835308302e7534df3577e8c498fda4e1eea6f32201f45436812bb27e22833ff831bb4dd
@@ -0,0 +1,12 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /Gemfile.lock
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /spec/reports/
9
+ /tmp/
10
+
11
+ # rspec failure tracking
12
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --format documentation
2
+ --color
@@ -0,0 +1,5 @@
1
+ sudo: false
2
+ language: ruby
3
+ rvm:
4
+ - 2.4.2
5
+ before_install: gem install bundler -v 1.14.6
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at jess@optoro.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in avro_pinions.gemspec
4
+ gemspec
@@ -0,0 +1,42 @@
1
+ # A sample Guardfile
2
+ # More info at https://github.com/guard/guard#readme
3
+
4
+ ## Uncomment and set this to only include directories you want to watch
5
+ # directories %w(app lib config test spec features) \
6
+ # .select{|d| Dir.exists?(d) ? d : UI.warning("Directory #{d} does not exist")}
7
+
8
+ ## Note: if you are using the `directories` clause above and you are not
9
+ ## watching the project directory ('.'), then you will want to move
10
+ ## the Guardfile to a watched dir and symlink it back, e.g.
11
+ #
12
+ # $ mkdir config
13
+ # $ mv Guardfile config/
14
+ # $ ln -s config/Guardfile .
15
+ #
16
+ # and, you'll have to watch "config/Guardfile" instead of "Guardfile"
17
+
18
+ # Note: The cmd option is now required due to the increasing number of ways
19
+ # rspec may be run, below are examples of the most common uses.
20
+ # * bundler: 'bundle exec rspec'
21
+ # * bundler binstubs: 'bin/rspec'
22
+ # * spring: 'bin/rspec' (This will use spring if running and you have
23
+ # installed the spring binstubs per the docs)
24
+ # * zeus: 'zeus rspec' (requires the server to be started separately)
25
+ # * 'just' rspec: 'rspec'
26
+
27
+ guard :rspec, cmd: "bundle exec rspec" do
28
+ require "guard/rspec/dsl"
29
+ dsl = Guard::RSpec::Dsl.new(self)
30
+
31
+ # Feel free to open issues for suggestions and improvements
32
+
33
+ # RSpec files
34
+ rspec = dsl.rspec
35
+ watch(rspec.spec_helper) { rspec.spec_dir }
36
+ watch(rspec.spec_support) { rspec.spec_dir }
37
+ watch(rspec.spec_files)
38
+
39
+ # Ruby files
40
+ ruby = dsl.ruby
41
+ dsl.watch_spec_files_for(ruby.lib_files)
42
+ end
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2017 Jess Szmajda
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
@@ -0,0 +1,116 @@
1
+ # AvroPinions
2
+
3
+ Opinionated stuff around AVRO in Ruby. Goals:
4
+
5
+ * Support some generic schema registry. You plug it in, conform to my API spec.
6
+ * Support [Single-Object Encoding](http://avro.apache.org/docs/1.8.2/spec.html#single_object_encoding) out of the box
7
+ * Writing for now. Maybe reading later.
8
+
9
+
10
+ ## Installation
11
+
12
+ Add this line to your application's Gemfile:
13
+
14
+ ```ruby
15
+ gem 'avro_pinions'
16
+ ```
17
+
18
+ And then execute:
19
+
20
+ $ bundle
21
+
22
+ Or install it yourself as:
23
+
24
+ $ gem install avro_pinions
25
+
26
+ ## Usage
27
+
28
+ Here's a full example:
29
+
30
+ ```ruby
31
+ # define a publisher. It has to have one method: `#publish`
32
+ class MyPublisher
33
+ # publish needs receive a topic and a message
34
+ # message is binary encoded AVRO data
35
+ def publish(topic, message)
36
+ # depending on how your transport mechanism works you may need to convert
37
+ # binary data to ascii characters or something. I recommend Base64 if you
38
+ # have to do that
39
+ extra_encoded = make_binary_data_wire_safe(message)
40
+ SomeTransferMechanismIOwn.send(topic, extra_encoded)
41
+ end
42
+ end
43
+
44
+ # configure AvroPinions
45
+ AvroPinions.configure({
46
+ wire_format: :single_object,
47
+ schema_registry: {
48
+ type: :file,
49
+ schema_path: 'doc/schemas'
50
+ },
51
+ publisher: MyPublisher.new
52
+ })
53
+
54
+ # define a message. They inherit from AvroPinions::Message
55
+ class SomeMessage < AvroPinions::Message
56
+ # define these constants or override `topic`, `schema`, and `namespace`
57
+ # methods
58
+ TOPIC = "some_topic"
59
+ SCHEMA = "SchemaName"
60
+ NAMESPACE = "com.company.namespace"
61
+
62
+ def initialize(data)
63
+ @data = data
64
+ end
65
+
66
+ # this method must return a json object that will be encoded with the AVRO
67
+ # schema
68
+ def record
69
+ { key: @data.value }
70
+ end
71
+ end
72
+
73
+ message = SomeMessage.new({ data... })
74
+ if message.valid?
75
+ AvroPinions.publish(message)
76
+ end
77
+ ```
78
+
79
+ Or you can provide things more manually if you want:
80
+ ```ruby
81
+ AvroPinions.configure({ wire_format: :single_object })
82
+
83
+ class Message < AvroPinions::Message
84
+ def initialize(avro_schema, data)
85
+ @avro_schema = avro_schema
86
+ @data = data
87
+ end
88
+
89
+ def record
90
+ @data
91
+ end
92
+
93
+ def avro_schema
94
+ @avro_schema
95
+ end
96
+ end
97
+
98
+ avro_schema = Avro::Schema.real_parse(json_schema_object)
99
+ Message.new(avro_schema, some_data).encode
100
+ ```
101
+
102
+ ## Development
103
+
104
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
105
+
106
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
107
+
108
+ ## Contributing
109
+
110
+ Bug reports and pull requests are welcome on GitHub at https://github.com/jszmajda/avro_pinions. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
111
+
112
+
113
+ ## License
114
+
115
+ The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
116
+
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
@@ -0,0 +1,33 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'avro_pinions/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "avro_pinions"
8
+ spec.version = AvroPinions::VERSION
9
+ spec.authors = ["Jess Szmajda"]
10
+ spec.email = ["jess@optoro.com"]
11
+
12
+ spec.summary = %q{Opinionated way to use AVRO in Ruby}
13
+ #spec.description = %q{TODO: Write a longer description or delete this line.}
14
+ spec.homepage = "https://github.com/jszmajda/avro_pinions"
15
+ spec.license = "MIT"
16
+
17
+ spec.files = `git ls-files -z`.split("\x0").reject do |f|
18
+ f.match(%r{^(test|spec|features)/})
19
+ end
20
+ spec.bindir = "exe"
21
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
22
+ spec.require_paths = ["lib"]
23
+
24
+ spec.add_dependency "avro", "~> 1.8"
25
+ spec.add_dependency "avro-patches", "~> 0.3"
26
+
27
+ spec.add_development_dependency "bundler", "~> 1.14"
28
+ spec.add_development_dependency "rake", "~> 10.0"
29
+ spec.add_development_dependency "rspec", "~> 3.0"
30
+ spec.add_development_dependency "pry"
31
+ spec.add_development_dependency "guard"
32
+ spec.add_development_dependency "guard-rspec"
33
+ end
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "avro_pinions"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start(__FILE__)
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,51 @@
1
+ require "ext/avro_patches"
2
+ require "avro_pinions/version"
3
+ require 'avro_pinions/configuration'
4
+ require 'avro_pinions/message'
5
+ require 'avro_pinions/schema_registry'
6
+ require 'avro_pinions/codec'
7
+
8
+ module AvroPinions
9
+ # Your code goes here...
10
+ module_function
11
+
12
+ def configuration
13
+ AvroPinions::Configuration.instance
14
+ end
15
+
16
+ def configure(messages)
17
+ if messages.respond_to?(:each_pair)
18
+ messages.each_pair do |key, value|
19
+ if value && configuration.respond_to?("#{key}=")
20
+ configuration.send("#{key}=", value)
21
+ elsif configuration.respond_to?(key)
22
+ configuration.send(key, *value)
23
+ end
24
+ end
25
+ else
26
+ configuration.send(messages)
27
+ end
28
+ end
29
+
30
+ def publisher
31
+ configuration.publisher
32
+ end
33
+
34
+ def schema_registry
35
+ configuration.schema_registry
36
+ end
37
+
38
+ def publish(message)
39
+ topic = message.topic
40
+ encoded = message.encode
41
+
42
+ publisher.publish(topic, encoded)
43
+ end
44
+
45
+ class NotFullyImplementedError < StandardError
46
+ end
47
+ class UnableToSerializeError < StandardError
48
+ end
49
+ class ConfigurationError < StandardError
50
+ end
51
+ end
@@ -0,0 +1,61 @@
1
+ module AvroPinions
2
+ class Codec
3
+
4
+ SUPPORTED_WIRE_FORMATS = [:single_object]
5
+
6
+ def self.pedantic_data(data)
7
+ if data.respond_to?(:each_pair)
8
+ data.each_pair.inject({}) do |result, (k,v)|
9
+ strrep = v.to_s
10
+ result[k.to_s] = if strrep.to_sym == v
11
+ strrep
12
+ else
13
+ v
14
+ end
15
+ result
16
+ end
17
+ else
18
+ data
19
+ end
20
+ end
21
+
22
+ attr_accessor :schema
23
+ def initialize(schema)
24
+ @schema = schema
25
+ end
26
+
27
+ def encode(data)
28
+ self.send("encode_as_#{wire_format}", data)
29
+ end
30
+
31
+ def wire_format
32
+ AvroPinions.configuration.wire_format
33
+ end
34
+
35
+ def encode_as_single_object(data)
36
+ soc = SingleObjectCodec.new(schema)
37
+ soc.encode(self.class.pedantic_data(data))
38
+ end
39
+
40
+ class SingleObjectCodec
41
+ attr_reader :schema
42
+ def initialize(schema)
43
+ @schema = schema
44
+ end
45
+
46
+ def encode(data)
47
+ stream = StringIO.new
48
+ writer = Avro::IO::DatumWriter.new(@schema)
49
+ encoder = Avro::IO::BinaryEncoder.new(stream)
50
+
51
+ smd = Avro::IO::SingleObjectMessage.new(@schema)
52
+ smd.header.each do |segment|
53
+ stream.write( segment.chr )
54
+ end
55
+ writer.write(data, encoder)
56
+
57
+ stream.string
58
+ end
59
+ end
60
+ end
61
+ end
@@ -0,0 +1,39 @@
1
+ require 'singleton'
2
+
3
+ module AvroPinions
4
+ class Configuration
5
+ include Singleton
6
+
7
+ attr_accessor :publisher, :schema_registry, :wire_format
8
+
9
+ def initialize
10
+ end
11
+
12
+ def publisher=(pub)
13
+ unless pub.respond_to?(:publish) && pub.method(:publish).arity == 2
14
+ raise InvalidConfiguration.new("Please define a publisher that responds to publish and receives a topic and message")
15
+ end
16
+ @publisher = pub
17
+ end
18
+
19
+ # :type gets pulled off and constructs the given registy
20
+ # all of the rest of the options get passed to an instance of hte type.kklk
21
+ def schema_registry=(options = {})
22
+ klass = options.delete(:type).to_s.split(/_/).map(&:capitalize).join
23
+ const = Object.const_get("AvroPinions::#{klass}Registry")
24
+ sr = const.new(options)
25
+ @schema_registry = sr
26
+ end
27
+
28
+ def wire_format=(format)
29
+ unless AvroPinions::Codec::SUPPORTED_WIRE_FORMATS.include?(format)
30
+ raise InvalidConfiguration.new("Please supply a valid wire format. Valid options are: #{AvroPinions::Codec::SUPPORTED_WIRE_FORMATS}")
31
+ end
32
+
33
+ @wire_format = format
34
+ end
35
+
36
+ class InvalidConfiguration < StandardError
37
+ end
38
+ end
39
+ end
@@ -0,0 +1,57 @@
1
+ module AvroPinions
2
+ class Message
3
+ def topic
4
+ # self.class::SYMBOL looks up a class constant in a child class
5
+ if defined?(self.class::TOPIC)
6
+ self.class::TOPIC
7
+ else
8
+ raise AvroPinions::NotFullyImplementedError, 'No Topic defined'
9
+ end
10
+ end
11
+
12
+ # Schema is inferred to be the .avsc file in the schemas folder with the
13
+ # name as given
14
+ def schema
15
+ if defined?(self.class::SCHEMA)
16
+ self.class::SCHEMA
17
+ else
18
+ raise AvroPinions::NotFullyImplementedError, 'No Schema defined'
19
+ end
20
+ end
21
+
22
+ def namespace
23
+ if defined?(self.class::NAMESPACE)
24
+ self.class::NAMESPACE
25
+ else
26
+ raise AvroPinions::NotFullyImplementedError, 'No Namespace defined'
27
+ end
28
+ end
29
+
30
+ def record
31
+ raise AvroPinions::NotFullyImplementedError, 'record method not implemented'
32
+ end
33
+
34
+ def publish
35
+ AvroPinions.publisher.publish(topic, record)
36
+ end
37
+
38
+ def encode
39
+ codec.encode(record)
40
+ end
41
+
42
+ def valid?
43
+ Avro::Schema.validate(avro_schema, AvroPinions::Codec.pedantic_data(record))
44
+ end
45
+
46
+ def codec
47
+ @codec ||= AvroPinions::Codec.new(avro_schema)
48
+ end
49
+
50
+ def avro_schema
51
+ @avro_schema ||= AvroPinions.schema_registry.schema(schema, namespace)
52
+ end
53
+
54
+ class InvalidRecord < StandardError
55
+ end
56
+ end
57
+ end
@@ -0,0 +1,34 @@
1
+ require 'json'
2
+ require 'avro'
3
+ require 'avro-patches'
4
+
5
+ module AvroPinions
6
+ class FileRegistry < SchemaRegistry
7
+ def initialize(options = {})
8
+ super()
9
+ @path = options[:schema_path] or raise AvroPinions::ConfigurationError.new("please define the path to load schemas from")
10
+ end
11
+
12
+ def load_schema(schema_name, namespace)
13
+ data = File.read(file_path(schema_name, namespace))
14
+ json = JSON.parse(data)
15
+ Avro::Schema.real_parse(json)
16
+ end
17
+
18
+ def file_path(schema_name, namespace)
19
+ schema_file = "#{schema_name}.avsc"
20
+ boring_filename = File.join(@path, schema_file)
21
+ if File.exists?(boring_filename)
22
+ boring_filename
23
+ else
24
+ fancy_filename = File.join(@path, *namespace.split(/\./), schema_file)
25
+ if File.exists?(fancy_filename)
26
+ fancy_filename
27
+ else
28
+ raise "File not found for namespace #{namespace} and schema #{schema_name}. Tried #{fancy_filename} and #{boring_filename}"
29
+ end
30
+ end
31
+ end
32
+
33
+ end
34
+ end
@@ -0,0 +1,38 @@
1
+ module AvroPinions
2
+ class SchemaRegistry
3
+ def initialize
4
+ @cache = {}
5
+ end
6
+
7
+ def schema(schema_name, namespace = "")
8
+ with_cache(schema_name, namespace) do
9
+ load_schema(schema_name, namespace)
10
+ end
11
+ end
12
+
13
+ def canonical_name(schema_name, namespace = "")
14
+ if namespace.length > 0
15
+ "#{namespace}.#{schema_name}"
16
+ else
17
+ schema_name
18
+ end
19
+ end
20
+
21
+ def load_schema(schema_name, namespace)
22
+ raise AvroPinions::NotFullyImplementedError.new("Please define how to load schemas")
23
+ end
24
+
25
+ def with_cache(schema_name, namespace, &block)
26
+ key = canonical_name(schema_name, namespace)
27
+
28
+ unless @cache.has_key?(key)
29
+ @cache[key] = block.call
30
+ end
31
+
32
+ @cache[key]
33
+ end
34
+ end
35
+ end
36
+
37
+ # has to go below because these classes inherit. Yeah..
38
+ require 'avro_pinions/schema_registries/file_registry'
@@ -0,0 +1,3 @@
1
+ module AvroPinions
2
+ VERSION = "1.0.0"
3
+ end
@@ -0,0 +1,61 @@
1
+ # I have a PR open on the avro core to include these capabilities, but for now
2
+ # whatever I guess..
3
+ require 'avro'
4
+ require 'avro-patches'
5
+ module Avro
6
+ class Schema
7
+
8
+ CRC_EMPTY = 0xc15d213aa4d7a795
9
+
10
+ # The java library caches this value after initialized, so this pattern
11
+ # mimics that.
12
+ @@fp_table = nil
13
+ def initFPTable
14
+ @@fp_table = Array.new(256)
15
+ 256.times do |i|
16
+ fp = i
17
+ 8.times do |j|
18
+ fp = (fp >> 1) ^ ( CRC_EMPTY & -( fp & 1 ) )
19
+ end
20
+ @@fp_table[i] = fp
21
+ end
22
+ end
23
+
24
+ def crc_64_avro_fingerprint
25
+ parsing_form = Avro::SchemaNormalization.to_parsing_form(self)
26
+ data_bytes = parsing_form.unpack("C*")
27
+
28
+ initFPTable unless @@fp_table
29
+
30
+ fp = CRC_EMPTY
31
+ data_bytes.each do |b|
32
+ fp = (fp >> 8) ^ @@fp_table[ (fp ^ b) & 0xff ]
33
+ end
34
+ fp
35
+ end
36
+
37
+ end
38
+
39
+ module IO
40
+ class SingleObjectMessage
41
+ def initialize(schema)
42
+ @schema = schema
43
+ end
44
+
45
+ SINGLE_OBJECT_MAGIC_NUMBER = [0xC3, 0x01]
46
+ def header
47
+ [SINGLE_OBJECT_MAGIC_NUMBER, schema_fingerprint].flatten
48
+ end
49
+ def schema_fingerprint
50
+ working = @schema.crc_64_avro_fingerprint
51
+ bytes = Array.new(8)
52
+ 8.times do |i|
53
+ bytes[7 - i] = (working & 0xff)
54
+ working = working >> 8
55
+ end
56
+ bytes
57
+ end
58
+ end
59
+ end
60
+ end
61
+
metadata ADDED
@@ -0,0 +1,176 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: avro_pinions
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - Jess Szmajda
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2017-12-20 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: avro
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.8'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.8'
27
+ - !ruby/object:Gem::Dependency
28
+ name: avro-patches
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '0.3'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '0.3'
41
+ - !ruby/object:Gem::Dependency
42
+ name: bundler
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '1.14'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '1.14'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rake
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '10.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '10.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: rspec
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '3.0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '3.0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: pry
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: guard
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ - !ruby/object:Gem::Dependency
112
+ name: guard-rspec
113
+ requirement: !ruby/object:Gem::Requirement
114
+ requirements:
115
+ - - ">="
116
+ - !ruby/object:Gem::Version
117
+ version: '0'
118
+ type: :development
119
+ prerelease: false
120
+ version_requirements: !ruby/object:Gem::Requirement
121
+ requirements:
122
+ - - ">="
123
+ - !ruby/object:Gem::Version
124
+ version: '0'
125
+ description:
126
+ email:
127
+ - jess@optoro.com
128
+ executables: []
129
+ extensions: []
130
+ extra_rdoc_files: []
131
+ files:
132
+ - ".gitignore"
133
+ - ".rspec"
134
+ - ".travis.yml"
135
+ - CODE_OF_CONDUCT.md
136
+ - Gemfile
137
+ - Guardfile
138
+ - LICENSE.txt
139
+ - README.md
140
+ - Rakefile
141
+ - avro_pinions.gemspec
142
+ - bin/console
143
+ - bin/setup
144
+ - lib/avro_pinions.rb
145
+ - lib/avro_pinions/codec.rb
146
+ - lib/avro_pinions/configuration.rb
147
+ - lib/avro_pinions/message.rb
148
+ - lib/avro_pinions/schema_registries/file_registry.rb
149
+ - lib/avro_pinions/schema_registry.rb
150
+ - lib/avro_pinions/version.rb
151
+ - lib/ext/avro_patches.rb
152
+ homepage: https://github.com/jszmajda/avro_pinions
153
+ licenses:
154
+ - MIT
155
+ metadata: {}
156
+ post_install_message:
157
+ rdoc_options: []
158
+ require_paths:
159
+ - lib
160
+ required_ruby_version: !ruby/object:Gem::Requirement
161
+ requirements:
162
+ - - ">="
163
+ - !ruby/object:Gem::Version
164
+ version: '0'
165
+ required_rubygems_version: !ruby/object:Gem::Requirement
166
+ requirements:
167
+ - - ">="
168
+ - !ruby/object:Gem::Version
169
+ version: '0'
170
+ requirements: []
171
+ rubyforge_project:
172
+ rubygems_version: 2.6.10
173
+ signing_key:
174
+ specification_version: 4
175
+ summary: Opinionated way to use AVRO in Ruby
176
+ test_files: []