avromatic 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: eda3c903923a23dfb499d2a2f0d60a063b1b305d
-  data.tar.gz: 5b7cb13c8615dc6e8549892c80ec1dab6ccc8f3f
+  metadata.gz: ca2b490c9b6978f339e2f6ea50c5a3476796fcae
+  data.tar.gz: 083c69aace3a37ceb0cce8f4c88fc5e5ade94611
 SHA512:
-  metadata.gz: c1d8cb9fcafb46abb997d1d4e72d4abc9a2fbb7171022f90c5207a466e3f254ea76e1d5e96b9a48013b66ab2b15a96e285795895a07415f9a67a26d24664a3dd
-  data.tar.gz: 5c04568b634ac3c7be07662100d71281c62d06cc923bd74ddeca8bf47907ead80d787384593b7a0d5cbc23dd96ca144e2a32ce97f68f64459470693d0267ebe0
+  metadata.gz: aa9927585c4e68af3905fd6b54f219bc1b4831c3f2d1b85e600fe8a44ea84367bafd248dc1e007dbdbdd45e39a6aa6b98deef727a587de13a39c48b4fe1384de
+  data.tar.gz: 46086b401b88aedc6ddef32ecd0fb946d5f8767536686cf6807d95631e9095d070b591268eccc162f87f000ca52b17327c52616f741e31dedfa40f30880a00a7

data/CHANGELOG.md

@@ -1,5 +1,9 @@
 # avromatic changelog
 
+## v0.4.0
+- Allow the specification of a custom type, including conversion to/from Avro,
+  for named types.
+
 ## v0.3.0
 - Remove dependency on the `private_attr` gem.
 

data/README.md

@@ -4,8 +4,8 @@
 
 [travis]: http://travis-ci.org/salsify/avromatic
 
-`Avromatic` generates Ruby models from Avro schemas and provides utilities to
-encode and decode them.
+`Avromatic` generates Ruby models from [Avro](http://avro.apache.org/) schemas
+and provides utilities to encode and decode them.
 
 ## Installation
 
@@ -31,21 +31,31 @@ Or install it yourself as:
 
 * schema_registry: An `AvroTurf::SchemaRegistry` object used to store Avro schemas 
   so that they can be referenced by id. Either `schema_registry` or 
-  `registry_url` must be configured.
+  `registry_url` must be configured. See [Confluent Schema Registry](http://docs.confluent.io/2.0.1/schema-registry/docs/intro.html).
 * registry_url: URL for the schema registry. The schema registry is used to store
   Avro schemas so that they can be referenced by id.  Either `schema_registry` or 
   `registry_url` must be configured.
 * schema_store: The schema store is used to load Avro schemas from the filesystem.
   It should be an object that responds to `find(name, namespace = nil)` and
-  returns an `Avro::Schema` object.
+  returns an `Avro::Schema` object. An `AvroTurf::SchemaStore` can be used.
 * messaging: An `AvroTurf::Messaging` object to be shared by all generated models.
   The `build_messaging!` method may be used to create a `Messaging` instance based
   on the other configuration values.
-* logger: The logger is for the schema registry client.
+* logger: The logger to use for the schema registry client.
+* [Custom Types](#custom-types)
+
+Example:
+
+```ruby
+Avromatic.configure do |config|
+  config.schema_store = AvroTurf::SchemaStore.new(path: 'avro/schemas')
+  config.registry_url = Rails.configuration.x.avro_schema_registry_url
+  config.build_messaging!
+```
 
 ### Models
 
-Models may be defined based on an Avro schema for a record.
+Models are defined based on an Avro schema for a record.
 
 The Avro schema can be specified by name and loaded using the schema store:
 
@@ -61,13 +71,12 @@ Or an `Avro::Schema` object can be specified directly:
 class MyModel
   include Avromatic::Model.build(schema: schema_object)
 end
-
 ```
 
 Models are generated as [Virtus](https://github.com/solnic/virtus) value
 objects. `Virtus` attributes are added for each field in the Avro schema
 including any default values defined in the schema. `ActiveModel` validations
-are used to define validations on certain types of fields.
+are used to define validations on certain types of fields ([see below](#validations)).
 
 A model may be defined with both a key and a value schema:
 
@@ -88,6 +97,45 @@ constant:
 MyModel = Avromatic::Model.model(schema_name :my_model)
 ```
 
+#### Custom Types
+
+Custom types can be configured for fields of named types (record, enum, fixed).
+These customizations are registered on the `Avromatic` module. Once a custom type
+is registered, it is used for all models with a schema that references that type.
+It is recommended to register types within a block passed to `Avromatic.configure`:
+
+```ruby
+Avromatic.configure do |config|
+  config.register_type('com.example.my_string', MyString)
+end
+```
+
+The full name of the type and an optional class may be specified. When a class is
+provided then values for attributes of that type are defined using the specified 
+class.
+
+If the provided class responds to the class methods `from_avro` and `to_avro`
+then those methods are used to convert values when assigning to the model and 
+before encoding using Avro respectively.
+
+`from_avro` and `to_avro` methods may be also be specified as Procs when 
+registering the type:
+
+```ruby
+Avromatic.configure do |config|
+  config.register_type('com.example.updown_string') do |type|
+    type.from_avro = ->(value) { value.upcase }
+    type.to_avro = ->(value) { value.downcase }
+  end
+end
+```
+
+Nil handling is not required as the conversion methods are not be called if the
+inbound or outbound value is nil.
+
+If a custom type is registered for a record-type field, then any `to_avro` 
+method/Proc should return a Hash with string keys for encoding using Avro.
+
 #### Encode/Decode
 
 Models can be encoded using Avro leveraging a schema registry to encode a schema
@@ -97,7 +145,7 @@ id at the beginning of the value.
 model.avro_message_value
 ```
 
-If a model has a Avro schema for a key, then the key can also be encoded
+If a model has an Avro schema for a key, then the key can also be encoded
 prefixed with a schema id.
 
 ```ruby

data/lib/avromatic/model/attributes.rb

@@ -9,6 +9,15 @@ module Avromatic
     module Attributes
       extend ActiveSupport::Concern
 
+      def self.first_union_schema(field_type)
+        # TODO: This is a hack until I find a better solution for unions with
+        # Virtus. This only handles a union for an optional field with :null
+        # and one other type.
+        schemas = field_type.schemas.reject { |schema| schema.type_sym == :null }
+        raise "Only the union of null with one other type is supported #{field_type}" if schemas.size > 1
+        schemas.first
+      end
+
       module ClassMethods
         def add_avro_fields
           if key_avro_schema
@@ -48,6 +57,7 @@ module Avromatic
                       avro_field_options(field))
 
             add_validation(field)
+            add_serializer(field)
           end
         end
 
@@ -81,6 +91,9 @@ module Avromatic
         end
 
         def avro_field_class(field_type)
+          custom_type = Avromatic.type_registry.fetch(field_type)
+          return custom_type.value_class if custom_type.value_class
+
           case field_type.type_sym
           when :string, :bytes, :fixed
             String
@@ -112,23 +125,28 @@ module Avromatic
         end
 
         def union_field_class(field_type)
-          # TODO: This is a hack until I find a better solution for unions with
-          # Virtus. This only handles a union for a optional field with :null
-          # and one other type.
-          schemas = field_type.schemas.reject { |schema| schema.type_sym == :null }
-          raise "Only the union of null with one other type is supported #{field_type}" if schemas.size > 1
-          avro_field_class(schemas.first)
+          avro_field_class(Avromatic::Model::Attributes.first_union_schema(field_type))
         end
 
         def avro_field_options(field)
+          options = {}
+
+          custom_type = Avromatic.type_registry.fetch(field)
+          coercer = custom_type.deserializer
+          options[:coercer] = coercer if coercer
+
           if field.default
-            {
-              default: default_for(field.default),
-              lazy: true
-            }
-          else
-            { }
+            options.merge!(default: default_for(field.default), lazy: true)
           end
+
+          options
+        end
+
+        def add_serializer(field)
+          custom_type = Avromatic.type_registry.fetch(field)
+          serializer = custom_type.serializer
+
+          avro_serializer[field.name.to_sym] = serializer if serializer
         end
 
         def default_for(value)

data/lib/avromatic/model/configuration.rb

@@ -1,7 +1,7 @@
 module Avromatic
   module Model
 
-    # This class holds configuration for a model build from Avro schema(s).
+    # This class holds configuration for a model built from Avro schema(s).
     class Configuration
 
       attr_reader :avro_schema, :key_avro_schema
@@ -17,7 +17,6 @@ module Avromatic
       # @option options [String, Symbol] :value_schema_name
       # @option options [Avro::Schema] :key_schema
       # @option options [String, Symbol] :key_schema_name
-      # @option options [schema store] :schema_store
       def initialize(**options)
         @avro_schema = find_avro_schema(**options)
         raise ArgumentError.new('value_schema(_name) or schema(_name) must be specified') unless avro_schema

data/lib/avromatic/model/custom_type.rb

@@ -0,0 +1,55 @@
+require 'avromatic/model/null_custom_type'
+
+module Avromatic
+  module Model
+
+    # Instances of this class contains the configuration for custom handling of
+    # a named type (record, enum, fixed).
+    class CustomType
+
+      attr_accessor :to_avro, :from_avro, :value_class
+
+      def initialize(value_class)
+        @value_class = value_class
+      end
+
+      # A deserializer method is used when assigning to the model. It is used both when
+      # deserializing a model instance from Avro and when directly instantiating
+      # an instance. The deserializer method must accept a single argument and return
+      # the value to store in the model for the attribute.
+      def deserializer
+        proc = from_avro_proc
+        wrap_proc(proc) if proc
+      end
+
+      # A serializer method is used when preparing attributes to be serialized using
+      # Avro. The serializer method must accept a single argument of the model value
+      # for the attribute and return a value in a form that Avro can serialize
+      # for the attribute.
+      def serializer
+        proc = to_avro_proc
+        wrap_proc(proc) if proc
+      end
+
+      private
+
+      def to_avro_proc
+        to_avro || value_class_method(:to_avro)
+      end
+
+      def from_avro_proc
+        from_avro || value_class_method(:from_avro)
+      end
+
+      def value_class_method(method_name)
+        value_class && value_class.respond_to?(method_name) &&
+          value_class.method(method_name).to_proc
+      end
+
+      # Wrap the supplied Proc to handle nil.
+      def wrap_proc(proc)
+        ->(value) { proc.call(value) if value }
+      end
+    end
+  end
+end

data/lib/avromatic/model/null_custom_type.rb

@@ -0,0 +1,21 @@
+module Avromatic
+  module Model
+
+    # This module is used to implement the null object pattern for a CustomType.
+    module NullCustomType
+      class << self
+        def value_class
+          nil
+        end
+
+        def deserializer
+          nil
+        end
+
+        def serializer
+          nil
+        end
+      end
+    end
+  end
+end

data/lib/avromatic/model/passthrough_serializer.rb

@@ -0,0 +1,10 @@
+module Avromatic
+  module Model
+    # This trivial serializer simply returns the value provided.
+    module PassthroughSerializer
+      def self.call(value)
+        value
+      end
+    end
+  end
+end

data/lib/avromatic/model/serialization.rb

@@ -1,4 +1,5 @@
 require 'avro_turf/messaging'
+require 'avromatic/model/passthrough_serializer'
 
 module Avromatic
   module Model
@@ -32,6 +33,8 @@ module Avromatic
 
         private
 
+        delegate :avro_serializer, to: :class
+
         def key_attributes_for_avro
           avro_hash(key_avro_field_names)
         end
@@ -41,7 +44,7 @@ module Avromatic
             result[key.to_s] = if value.is_a?(Avromatic::Model::Attributes)
                                  value.value_attributes_for_avro
                                else
-                                 value
+                                 avro_serializer[key].call(value)
                                end
           end
         end
@@ -70,6 +73,13 @@ module Avromatic
         delegate :messaging, to: :Avromatic
 
         include Decode
+
+        # Store a hash of Procs by field name (as a symbol) to convert
+        # the value before Avro serialization.
+        # Returns the default PassthroughSerializer if a key is not present.
+        def avro_serializer
+          @avro_serializer ||= Hash.new(PassthroughSerializer)
+        end
       end
     end
   end

data/lib/avromatic/model/type_registry.rb

@@ -0,0 +1,42 @@
+require 'avromatic/model/custom_type'
+
+module Avromatic
+  module Model
+    class TypeRegistry
+
+      delegate :clear, to: :custom_types
+
+      def initialize
+        @custom_types = {}
+      end
+
+      # @param fullname [String] The fullname of the Avro type.
+      # @param value_class [Class] Optional class to use for the attribute.
+      #   If unspecified then the default class for the Avro field is used.
+      # @block If a block is specified then the CustomType is yielded for
+      #   additional configuration.
+      def register_type(fullname, value_class = nil)
+        custom_types[fullname.to_s] = Avromatic::Model::CustomType.new(value_class).tap do |type|
+          yield(type) if block_given?
+        end
+      end
+
+      # @object [Avro::Schema] Custom type may be fetched based on a Avro field
+      #   or schema. If there is no custom type, then NullCustomType is returned.
+      def fetch(object)
+        field_type = object.is_a?(Avro::Schema::Field) ? object.type : object
+
+        if field_type.type_sym == :union
+          field_type = Avromatic::Model::Attributes.first_union_schema(field_type)
+        end
+
+        fullname = field_type.fullname if field_type.is_a?(Avro::Schema::NamedSchema)
+        custom_types.fetch(fullname, NullCustomType)
+      end
+
+      private
+
+      attr_reader :custom_types
+    end
+  end
+end

data/lib/avromatic/model.rb

@@ -1,5 +1,6 @@
 require 'avromatic/model/builder'
 require 'avromatic/model/decoder'
+require 'avromatic/model/type_registry'
 
 module Avromatic
   module Model

data/lib/avromatic/version.rb

@@ -1,3 +1,3 @@
 module Avromatic
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/lib/avromatic.rb

@@ -5,10 +5,14 @@ require 'avro_turf/messaging'
 
 module Avromatic
   class << self
-    attr_accessor :schema_registry, :registry_url, :schema_store, :logger, :messaging
+    attr_accessor :schema_registry, :registry_url, :schema_store, :logger,
+                  :messaging, :type_registry
+
+    delegate :register_type, to: :type_registry
   end
 
   self.logger = Logger.new($stdout)
+  self.type_registry = Avromatic::Model::TypeRegistry.new
 
   def self.configure
     yield self

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: avromatic
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Salsify Engineering
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-05-16 00:00:00.000000000 Z
+date: 2016-05-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: avro
@@ -202,8 +202,12 @@ files:
 - lib/avromatic/model/builder.rb
 - lib/avromatic/model/configurable.rb
 - lib/avromatic/model/configuration.rb
+- lib/avromatic/model/custom_type.rb
 - lib/avromatic/model/decoder.rb
+- lib/avromatic/model/null_custom_type.rb
+- lib/avromatic/model/passthrough_serializer.rb
 - lib/avromatic/model/serialization.rb
+- lib/avromatic/model/type_registry.rb
 - lib/avromatic/model/value_object.rb
 - lib/avromatic/railtie.rb
 - lib/avromatic/version.rb