jsi 0.0.4 → 0.4.0

data/README.md

@@ -1,11 +1,13 @@
-# JSI: JSON-Schema Instantiation
+# JSI: JSON Schema Instantiation
 
 [![Build Status](https://travis-ci.org/notEthan/jsi.svg?branch=master)](https://travis-ci.org/notEthan/jsi)
 [![Coverage Status](https://coveralls.io/repos/github/notEthan/jsi/badge.svg)](https://coveralls.io/github/notEthan/jsi)
 
-JSI represents JSON-schemas as ruby classes, and schema instances as instances of those classes.
+JSI offers an Object-Oriented representation for JSON data using JSON Schemas. Given your JSON Schemas, JSI constructs Ruby modules and classes which are used to instantiate your JSON data. These modules let you use JSON with all the niceties of OOP such as property accessors and application-defined instance methods.
 
-A JSI class aims to be a fairly unobtrusive wrapper around its instance. It adds accessors for known property names, validation methods, and a few other nice things. Mostly though, you use a JSI as you would use its underlying data, calling the same methods (e.g. `#[]`, `#map`, `#repeated_permutation`) and passing it to anything that duck-types expecting #to_ary or #to_hash.
+To learn more about JSON Schema see [https://json-schema.org/]().
+
+A JSI class aims to be a fairly unobtrusive wrapper around its instance - "instance" here meaning the JSON data, usually a Hash or Array, which instantiate the JSON Schema. JSI schema modules and classes add accessors for property names described by its schema, schema validation, and other nice things. Mostly though, you use a JSI as you would use its underlying data, calling the same methods (e.g. `#[]`, `#map`, `#repeated_permutation`) and passing it to anything that duck-types expecting `#to_ary` or `#to_hash`.
 
 ## Example
 
@@ -19,28 +21,48 @@ properties:
   phone:
     type: "array"
     items:
+      description: "A phone number"
       type: "object"
       properties:
         location: {type: "string"}
         number: {type: "string"}
 ```
 
-And here's the class for that schema from JSI:
+Using that schema, we instantiate a JSI::Schema to represent it:
+
+```ruby
+# this would usually use a YAML.load/JSON.parse/whatever; it's inlined for copypastability.
+contact_schema = JSI::Schema.new({"description" => "A Contact", "type" => "object", "properties" => {"name" => {"type" => "string"}, "phone" => {"type" => "array", "items" => {"type" => "object", "properties" => {"location" => {"type" => "string"}, "number" => {"type" => "string"}}}}}})
+```
+
+We name the module that JSI will use when instantiating a contact. Named modules are better to work with, and JSI will indicate the names of schema modules in its `#inspect` output.
 
 ```ruby
-Contact = JSI.class_for_schema(YAML.load_file('contact.schema.yml'))
-# you can copy/paste this line instead, to follow along in irb:
-Contact = JSI.class_for_schema({"description" => "A Contact", "type" => "object", "properties" => {"name" => {"type" => "string"}, "phone" => {"type" => "array", "items" => {"type" => "object", "properties" => {"location" => {"type" => "string"}, "number" => {"type" => "string"}}}}}})
+Contact = contact_schema.jsi_schema_module
+```
+
+To instantiate the schema, we need some JSON data (expressed here as YAML)
+
+```yaml
+name: bill
+phone:
+- location: home
+  number: "555"
+nickname: big b
 ```
 
-This definition gives you not just the Contact class, but classes for the whole nested structure. So, if we construct an instance like:
+So, if we construct an instance like:
 
 ```ruby
-bill = Contact.new('name' => 'bill', 'phone' => [{'location' => 'home', 'number' => '555'}], 'nickname' => 'big b')
-# => #{<Contact fragment="#">
-# #{<Contact fragment="#">
-#   "phone" => #[<JSI::SchemaClasses["1f97#/properties/phone"] fragment="#/phone">
-#     #{<JSI::SchemaClasses["1f97#/properties/phone/items"] fragment="#/phone/0"> "location" => "home", "number" => "555"}
+# this would usually use a YAML.load/JSON.parse/whatever; it's inlined for copypastability.
+bill = Contact.new_jsi({"name" => "bill", "phone" => [{"location" => "home", "number" => "555"}], "nickname" => "big b"})
+# => #{<JSI (Contact)>
+#   "name" => "bill",
+#   "phone" => #[<JSI>
+#     #{<JSI>
+#       "location" => "home",
+#       "number" => "555"
+#     }
 #   ],
 #   "nickname" => "big b"
 # }
@@ -48,8 +70,6 @@ bill = Contact.new('name' => 'bill', 'phone' => [{'location' => 'home', 'number'
 
 Note that the keys are strings. JSI, being designed with JSON in mind, is geared toward string keys. Symbol keys will not match to schema properties, and so act the same as any other key not recognized from the schema.
 
-The nested classes can be seen as `JSI::SchemaClasses[schema_id]` where schema_id is a generated value.
-
 We get accessors for the Contact:
 
 ```ruby
@@ -74,23 +94,25 @@ bill.validate
 ... and validations on the nested schema instances (#phone here), showing in this example validation failure:
 
 ```ruby
-bad = Contact.new('phone' => [{'number' => [5, 5, 5]}])
-# => #{<Contact fragment="#">
-#   "phone" => #[<JSI::SchemaClasses["1f97#/properties/phone"] fragment="#/phone">
-#     #{<JSI::SchemaClasses["1f97#/properties/phone/items"] fragment="#/phone/0">
-#       "number" => #[<JSI::SchemaClasses["1f97#/properties/phone/items/properties/number"] fragment="#/phone/0/number"> 5, 5, 5]
+bad = Contact.new_jsi({'phone' => [{'number' => [5, 5, 5]}]})
+# => #{<JSI (Contact)>
+#   "phone" => #[<JSI>
+#     #{<JSI>
+#       "number" => #[<JSI> 5, 5, 5]
 #     }
 #   ]
 # }
 bad.phone.fully_validate
-# => ["The property '#/0/number' of type array did not match the following type: string in schema 1f97"]
+# => ["The property '#/0/number' of type array did not match the following type: string in schema 594126e3"]
 ```
 
 These validations are done by the [`json-schema` gem](https://github.com/ruby-json-schema/json-schema) - JSI does not do validations on its own.
 
-Since the underlying instance is a ruby hash (json object), we can use it like a hash with #[] or, say, #transform_values:
+Since the underlying instance is a ruby hash (json object), we can use it like a hash with `#[]` or, say, `#transform_values`:
 
 ```ruby
+# note that #size here is actually referring to multiple different methods; for name and nickname
+# it is String#size but for phone it is Array#size.
 bill.transform_values(&:size)
 # => {"name" => 4, "phone" => 1, "nickname" => 5}
 bill['nickname']
@@ -101,21 +123,22 @@ There's plenty more JSI has to offer, but this should give you a pretty good ide
 
 ## Terminology and Concepts
 
-- JSI::Base is the base class from which other classes representing JSON-Schemas inherit.
-- a JSI class refers to a class representing a schema, a subclass of JSI::Base.
+- `JSI::Base` is the base class for each JSI class representing a JSON Schema.
+- a "JSI class" is a subclass of `JSI::Base` representing a JSON schema.
+- a "JSI schema module" is a module representing a schema, included on a JSI class.
 - "instance" is a term that is significantly overloaded in this space, so documentation will attempt to be clear what kind of instance is meant:
-  - a schema instance refers broadly to a data structure that is described by a json-schema.
-  - a JSI instance (or just "a JSI") is a ruby object instantiating a JSI class. it has a method #instance which contains the underlying data.
-- a schema refers to a json-schema. a JSI::Schema represents such a json-schema. a JSI class allows instantiation of such a schema.
+  - a schema instance refers broadly to a data structure that is described by a JSON schema.
+  - a JSI instance (or just "a JSI") is a ruby object instantiating a JSI class. it has a method `#jsi_instance` which contains the underlying data.
+- a schema refers to a JSON schema. `JSI::Schema` is a module which extends schemas. A schema is usually a `JSI::Base` instance, and that schema JSI's schema is a metaschema (see the sections on Metaschemas below).
 
-## JSI classes
+## JSI and Object Oriented Programming
 
-A JSI class (that is, subclass of JSI::Base) is a starting point but obviously you want your own methods, so you reopen the class as you would any other. referring back to the Example section above, we reopen the Contact class:
+Instantiating your schema is a starting point. But, since the major point of object-oriented programming is applying methods to your objects, of course you want to be able to define your own methods. To do this we reopen the JSI module we defined. Referring back to the Example section above, we reopen the `Contact` module:
 
 ```ruby
-class Contact
-  def full_address
-    address.values.join(", ")
+module Contact
+  def phone_numbers
+    phone.map(&:number)
   end
   def name
     super + ' esq.'
@@ -131,15 +154,17 @@ bill.name = 'rob esq.'
 # => "rob esq."
 bill['name']
 # => "rob"
+bill.phone_numbers
+# => ["555"]
 ```
 
-Note the use of `super` - you can call to accessors defined by JSI and make your accessors act as wrappers (these accessor methods are defined on an included module instead of the JSI class for this reason). You can also use [] and []=, of course, with the same effect.
+Note the use of `super` - you can call to accessors defined by JSI and make your accessors act as wrappers. You can alternatively use `[]` and `[]=` with the same effect.
 
-If you want to add methods to a subschema, get the class_for_schema for that schema and open up that class. You can leave the class anonymous, as in this example:
+You can also add methods to a subschema using the same method `#jsi_schema_module` which we used to define the `Contact` module above.
 
 ```ruby
-phone_schema = Contact.schema['properties']['phone']['items']
-JSI.class_for_schema(phone_schema).class_eval do
+phone_schema = Contact.schema.properties['phone'].items
+phone_schema.jsi_schema_module.module_eval do
   def number_with_dashes
     number.split(//).join('-')
   end
@@ -148,11 +173,11 @@ bill.phone.first.number_with_dashes
 # => "5-5-5"
 ```
 
-If you want to name the class, this works:
+If you want to name the module, this works:
 
 ```ruby
-Phone = JSI.class_for_schema(Contact.schema['properties']['phone']['items'])
-class Phone
+ContactPhone = Contact.schema.properties['phone'].items.jsi_schema_module
+module ContactPhone
   def number_with_dashes
     number.split(//).join('-')
   end
@@ -161,23 +186,33 @@ end
 
 Either syntax is slightly cumbersome and a better syntax is in the works.
 
+## Metaschemas
+
+A metaschema is a schema which describes schemas. Likewise, a schema is an instance of a metaschema.
+
+In JSI, a schema is generally a JSI::Base instance whose schema is a metaschema.
+
+A self-descriptive metaschema - most commonly one of the JSON schema draft metaschemas - is an object whose schema is itself. This is instantiated in JSI as a JSI::MetaschemaNode (not a JSI::Base).
+
 ## ActiveRecord serialization
 
 A really excellent place to use JSI is when dealing with serialized columns in ActiveRecord.
 
-Let's say you're sticking to json types in the database - you have to do so if you're using json columns, or json serialization, and if you have dealt with arbitrary yaml- or marshal-serialized objects in ruby, you have probably found that approach has its shortcomings when the implementation of your classes changes.
+Let's say you're sticking to JSON types in the database - you have to do so if you're using JSON columns, or JSON serialization, and if you have dealt with arbitrary yaml- or marshal-serialized objects in ruby, you have probably found that approach has its shortcomings when the implementation of your classes changes.
 
-But if your database contains json, then your deserialized objects in ruby are likewise Hash / Array / basic types. You have to use subscripts instead of accessors, and you don't have any way to add methods to your data types.
+But if your database contains JSON, then your deserialized objects in ruby are likewise Hash / Array / basic types. You have to use subscripts instead of accessors, and you don't have any way to add methods to your data types.
 
-JSI gives you the best of both with SchemaInstanceJSONCoder. The objects in your database are simple json types, and your ruby classes are extensible and have the accessors you get from a JSI class hierarchy. Here's an example:
+JSI gives you the best of both with JSICoder. This coder dumps objects which are simple JSON types, and loads instances of a specified JSI schema. Here's an example:
 
 ```ruby
 class User < ActiveRecord::Base
-  serialize :contacts, JSI::SchemaInstanceJSONCoder.new(Contact, array: true)
+  serialize :contact_info, JSI::JSICoder.new(Contact)
 end
 ```
 
-Now `user.contacts` will return an array of Contact instances, from the json type in the database, with Contact's accessors, validations, and user-defined instance methods.
+Now `user.contacts` will return an array of Contact instances, from the JSON type in the database, with Contact's accessors, validations, and user-defined instance methods.
+
+See the gem [`arms`](https://github.com/notEthan/arms) if you wish to serialize the dumped JSON-compatible objects further as text.
 
 ## Keying Hashes (JSON Objects)
 
@@ -189,4 +224,8 @@ Issues and pull requests are welcome on GitHub at https://github.com/notEthan/js
 
 ## License
 
-JSI is open source software available under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+[<img align="right" src="https://github.com/notEthan/jsi/raw/master/resources/icons/AGPL-3.0.png">](https://www.gnu.org/licenses/agpl-3.0.html)
+
+JSI is licensed under the terms of the [GNU Affero General Public License version 3](https://www.gnu.org/licenses/agpl-3.0.html).
+
+Unlike the MIT or BSD licenses more commonly used with Ruby gems, this license requires that if you modify JSI and propagate your changes, e.g. by including it in a web application, your modified version must be publicly available. The common path of forking on Github should satisfy this requirement.

data/jsi.gemspec

@@ -6,26 +6,23 @@ Gem::Specification.new do |spec|
   spec.name    = "jsi"
   spec.version = JSI::VERSION
   spec.authors = ["Ethan"]
-  spec.email   = ["ethan@unth"]
+  spec.email   = ["ethan.jsi@unth.net"]
 
-  spec.summary     = "JSI: JSON-Schema instantiation"
-  spec.description = "JSI represents json-schemas as ruby classes and json-schema instances as instances of those classes"
+  spec.summary     = "JSI: JSON Schema Instantiation"
+  spec.description = "JSI offers an Object-Oriented representation for JSON data using JSON Schemas"
   spec.homepage    = "https://github.com/notEthan/jsi"
-  spec.license     = "MIT"
-  ignore_files = %w(.gitignore .travis.yml Gemfile test)
-  ignore_files_re = %r{\A(#{ignore_files.map { |f| Regexp.escape(f) }.join('|')})(/|\z)}
-  Dir.chdir(File.expand_path('..', __FILE__)) do
-    spec.files      = `git ls-files -z`.split("\x0").reject { |f| f.match(ignore_files_re) }
-    spec.test_files = `git ls-files -z test`.split("\x0")
-  end
-  spec.bindir        = "exe"
-  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.license     = "AGPL-3.0"
+  ignore_files     = %w(.gitignore .travis.yml Gemfile test)
+  ignore_files_re  = %r{\A(#{ignore_files.map { |f| Regexp.escape(f) }.join('|')})(/|\z)}
+  spec.files       = `git ls-files -z`.split("\x0").reject { |f| f.match(ignore_files_re) }
+  spec.test_files   = `git ls-files -z test`.split("\x0")
   spec.require_paths = ["lib"]
 
   # we are monkey patching json-schema with a fix that has not been merged in a timely fashion.
   spec.add_dependency "json-schema", "~> 2.8"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "minitest", "~> 5.0"
+  spec.add_development_dependency "rake"
+  spec.add_development_dependency "minitest"
   spec.add_development_dependency "minitest-around"
   spec.add_development_dependency "minitest-reporters"
+  spec.add_development_dependency "scorpio"
 end

data/lib/jsi.rb

@@ -1,31 +1,50 @@
+# frozen_string_literal: true
+
 require "jsi/version"
 require "pp"
+require "set"
+require "json"
+require "pathname"
+require "addressable/uri"
+
 require "jsi/json-schema-fragments"
+
 require "jsi/util"
+require "jsi/typelike_modules"
 
 module JSI
   # generally put in code paths that are not expected to be valid control flow paths.
   # rather a NotImplementedCorrectlyError. but that's too long.
+  #
+  # if you've found this class because JSI has raised this error, please open an issue with the backtrace
+  # and any context you can provide at https://github.com/notEthan/jsi/issues
   class Bug < NotImplementedError
   end
 
+  ROOT_PATH = Pathname.new(__FILE__).dirname.parent.expand_path
+  RESOURCES_PATH = ROOT_PATH.join('resources')
+
   autoload :JSON, 'jsi/json'
+  autoload :PathedNode, 'jsi/pathed_node'
   autoload :Typelike, 'jsi/typelike_modules'
   autoload :Hashlike, 'jsi/typelike_modules'
   autoload :Arraylike, 'jsi/typelike_modules'
   autoload :Schema, 'jsi/schema'
   autoload :Base, 'jsi/base'
-  autoload :BaseArray, 'jsi/base'
-  autoload :BaseHash, 'jsi/base'
-  autoload :SchemaClasses, 'jsi/base'
-  autoload :ObjectJSONCoder, 'jsi/schema_instance_json_coder'
-  autoload :StructJSONCoder, 'jsi/struct_json_coder'
-  autoload :SchemaInstanceJSONCoder, 'jsi/schema_instance_json_coder'
-
-  # @return [Class subclassing JSI::Base] a JSI class which represents the
-  #   given schema. instances of the class represent JSON Schema instances
-  #   for the given schema.
-  def self.class_for_schema(*a, &b)
-    SchemaClasses.class_for_schema(*a, &b)
+  autoload :Metaschema, 'jsi/metaschema'
+  autoload :MetaschemaNode, 'jsi/metaschema_node'
+  autoload :SchemaClasses, 'jsi/schema_classes'
+  autoload :JSICoder, 'jsi/jsi_coder'
+
+  autoload :JSONSchemaOrgDraft04, 'schemas/json-schema.org/draft-04/schema'
+  autoload :JSONSchemaOrgDraft06, 'schemas/json-schema.org/draft-06/schema'
+
+  autoload :SimpleWrap, 'jsi/simple_wrap'
+
+  # @param schemas [Enumerable<JSI::Schema, #to_hash, Boolean>] schemas to represent with the class
+  # @return [Class subclassing JSI::Base] a JSI class which represents the given schemas.
+  #   an instance of the class represents a JSON Schema instance described by all of the given schemas.
+  def self.class_for_schemas(*schemas)
+    SchemaClasses.class_for_schemas(*schemas)
   end
 end

data/lib/jsi/base.rb

@@ -1,5 +1,4 @@
-require 'json'
-require 'jsi/typelike_modules'
+# frozen_string_literal: true
 
 module JSI
   # the base class for representing and instantiating a JSON Schema.
@@ -11,123 +10,302 @@ module JSI
   # are dynamically created for schemas using {JSI.class_for_schema}, and these
   # are what are used to instantiate and represent JSON schema instances.
   class Base
-    include Memoize
+    include Util::Memoize
     include Enumerable
+    include PathedNode
+    class CannotSubscriptError < StandardError
+    end
 
     class << self
-      # @return [String] absolute schema_id of the schema this class represents.
-      #   see {Schema#schema_id}.
-      def schema_id
-        schema.schema_id
+      # JSI::Base.new_jsi behaves the same as .new, and is defined for compatibility so you may call #new_jsi
+      # on any of a JSI::Schema, a JSI::SchemaModule, or a JSI schema class.
+      # @return [JSI::Base] a JSI whose instance is the given instance
+      def new_jsi(instance, *a, &b)
+        new(instance, *a, &b)
       end
 
-      # @return [String] a string representing the class, with schema_id
-      def inspect
-        if !respond_to?(:schema)
-          super
-        elsif !name || name =~ /\AJSI::SchemaClasses::/
-          %Q(#{SchemaClasses.inspect}[#{schema_id.inspect}])
-        else
-          %Q(#{name} (#{schema_id}))
-        end
+      # is the constant JSI::SchemaClasses::{self.schema_classes_const_name} defined?
+      # (if so, we will prefer to use something more human-readable than that ugly mess.)
+      def in_schema_classes
+        # #name sets @in_schema_classes
+        name
+        @in_schema_classes
       end
 
-      # @return [String] a string representing the class - a class name if one
-      #   was explicitly defined, otherwise a reference to JSI::SchemaClasses
-      def to_s
-        if !respond_to?(:schema)
+      # @return [String] a string representing the class, indicating the schemas represented by their module
+      #   name or a URI
+      def inspect
+        if !respond_to?(:jsi_class_schemas)
           super
-        elsif !name || name =~ /\AJSI::SchemaClasses::/
-          %Q(#{SchemaClasses.inspect}[#{schema_id.inspect}])
         else
-          name
+          schema_names = jsi_class_schemas.map do |schema|
+            mod = schema.jsi_schema_module
+            if mod.name && schema.schema_id
+              "#{mod.name} (#{schema.schema_id})"
+            elsif mod.name
+              mod.name
+            elsif schema.schema_id
+              schema.schema_id
+            else
+              schema.node_ptr.uri
+            end
+          end
+
+          if name && !in_schema_classes
+            if jsi_class_schemas.empty?
+              "#{name} (0 schemas)"
+            else
+              "#{name} (#{schema_names.join(', ')})"
+            end
+          else
+            if schema_names.empty?
+              "(JSI Schema Class for 0 schemas)"
+            else
+              "(JSI Schema Class: #{schema_names.join(', ')})"
+            end
+          end
         end
       end
 
-      # @return [String] a name for a constant for this class, generated from the
-      #   schema_id. only used if the class is not assigned to another constant.
+      alias_method :to_s, :inspect
+
+      # @return [String, nil] a name for a constant for this class, generated from the constant name
+      #   or schema id of each schema this class represents. nil if any represented schema has no constant
+      #   name or schema id.
       def schema_classes_const_name
-        name = schema.schema_id.gsub(/[^\w]/, '_')
-        name = 'X' + name unless name[/\A[a-zA-Z_]/]
-        name = name[0].upcase + name[1..-1]
-        name
+        if respond_to?(:jsi_class_schemas)
+          schema_names = jsi_class_schemas.map do |schema|
+            if schema.jsi_schema_module.name
+              schema.jsi_schema_module.name
+            elsif schema.schema_id
+              schema.schema_id
+            else
+              nil
+            end
+          end
+          if !schema_names.any?(&:nil?) && !schema_names.empty?
+            schema_names.sort.map { |n| 'X' + n.gsub(/[^\w]/, '_') }.join('')
+          end
+        end
       end
 
       # @return [String] a constant name of this class
       def name
-        unless super || SchemaClasses.const_defined?(schema_classes_const_name)
-          SchemaClasses.const_set(schema_classes_const_name, self)
+        unless instance_variable_defined?(:@in_schema_classes)
+          const_name = schema_classes_const_name
+          if super || !const_name || SchemaClasses.const_defined?(const_name)
+            @in_schema_classes = false
+          else
+            SchemaClasses.const_set(const_name, self)
+            @in_schema_classes = true
+          end
         end
         super
       end
     end
 
-    # initializes this JSI from the given instance. the instance will be
-    # wrapped as a {JSI::JSON::Node JSI::JSON::Node} (unless what you pass is
-    # a Node already).
+    # NOINSTANCE is a magic value passed to #initialize when instantiating a JSI
+    # from a document and JSON Pointer.
+    NOINSTANCE = Object.new.tap { |o| [:inspect, :to_s].each(&(-> (s, m) { o.define_singleton_method(m) { s } }.curry.([JSI::Base.name, 'NOINSTANCE'].join('::')))) }
+
+    # initializes this JSI from the given instance - instance is most commonly
+    # a parsed JSON document consisting of Hash, Array, or sometimes a basic
+    # type, but this is in no way enforced and a JSI may wrap any object.
     #
     # @param instance [Object] the JSON Schema instance being represented
-    # @param ancestor [JSI::Base] for internal use, specifies an ancestor
-    #   from which this JSI originated to calculate #parents
-    def initialize(instance, ancestor: nil)
-      unless respond_to?(:schema)
-        raise(TypeError, "cannot instantiate #{self.class.inspect} which has no method #schema. please use JSI.class_for_schema")
+    # @param jsi_document [Object] for internal use. the instance may be specified as a
+    #   node in the `jsi_document` param, pointed to by `jsi_ptr`. the param `instance`
+    #   MUST be `NOINSTANCE` to use the jsi_document + jsi_ptr form. `jsi_document` MUST
+    #   NOT be passed if `instance` is anything other than `NOINSTANCE`.
+    # @param jsi_ptr [JSI::JSON::Pointer] for internal use. a JSON pointer specifying
+    #   the path of this instance in the `jsi_document` param. `jsi_ptr` must be passed
+    #   iff `jsi_document` is passed, i.e. when `instance` is `NOINSTANCE`
+    # @param jsi_root_node [JSI::Base] for internal use, specifies the JSI at the root of the document
+    def initialize(instance, jsi_document: nil, jsi_ptr: nil, jsi_root_node: nil)
+      unless respond_to?(:jsi_schemas)
+        raise(TypeError, "cannot instantiate #{self.class.inspect} which has no method #jsi_schemas. it is recommended to instantiate JSIs from a schema using JSI::Schema#new_jsi.")
+      end
+
+      if instance.is_a?(JSI::Schema)
+        raise(TypeError, "assigning a schema to a #{self.class.inspect} instance is incorrect. received: #{instance.pretty_inspect.chomp}")
+      elsif instance.is_a?(JSI::Base)
+        raise(TypeError, "assigning another JSI::Base instance to a #{self.class.inspect} instance is incorrect. received: #{instance.pretty_inspect.chomp}")
+      end
+
+      if instance == NOINSTANCE
+        @jsi_document = jsi_document
+        unless jsi_ptr.is_a?(JSI::JSON::Pointer)
+          raise(TypeError, "jsi_ptr must be a JSI::JSON::Pointer; got: #{jsi_ptr.inspect}")
+        end
+        @jsi_ptr = jsi_ptr
+        if @jsi_ptr.root?
+          raise(Bug, "jsi_root_node cannot be specified for root JSI") if jsi_root_node
+          @jsi_root_node = self
+        else
+          if !jsi_root_node.is_a?(JSI::Base)
+            raise(TypeError, "jsi_root_node must be a JSI::Base; got: #{jsi_root_node.inspect}")
+          end
+          if !jsi_root_node.jsi_ptr.root?
+            raise(Bug, "jsi_root_node ptr #{jsi_root_node.jsi_ptr.inspect} is not root")
+          end
+          @jsi_root_node = jsi_root_node
+        end
+      else
+        raise(Bug, 'incorrect usage') if jsi_document || jsi_ptr || jsi_root_node
+        @jsi_document = instance
+        @jsi_ptr = JSI::JSON::Pointer[]
+        @jsi_root_node = self
       end
 
-      @ancestor = ancestor || self
-      self.instance = instance
+      if self.jsi_instance.respond_to?(:to_hash)
+        extend PathedHashNode
+      elsif self.jsi_instance.respond_to?(:to_ary)
+        extend PathedArrayNode
+      end
 
-      if @instance.is_a?(JSI::JSON::HashNode)
-        extend BaseHash
-      elsif @instance.is_a?(JSI::JSON::ArrayNode)
-        extend BaseArray
+      jsi_schemas.each do |schema|
+        if schema.describes_schema?
+          extend JSI::Schema
+        end
       end
     end
 
-    # the instance of the json-schema. this is a JSI::JSON::Node.
-    attr_reader :instance
+    # document containing the instance of this JSI
+    attr_reader :jsi_document
+
+    # JSI::JSON::Pointer pointing to this JSI's instance within the jsi_document
+    attr_reader :jsi_ptr
+
+    # the JSI at the root of this JSI's document
+    attr_reader :jsi_root_node
 
-    # a JSI which is an ancestor of this
-    attr_reader :ancestor
+    alias_method :node_document, :jsi_document
+    alias_method :node_ptr, :jsi_ptr
+    alias_method :document_root_node, :jsi_root_node
 
-    # each is overridden by BaseHash or BaseArray when appropriate. the base
+    # the instance of the json-schema - the underlying JSON data used to instantiate this JSI
+    alias_method :jsi_instance, :node_content
+    alias_method :instance, :node_content
+
+    # each is overridden by PathedHashNode or PathedArrayNode when appropriate. the base
     # #each is not actually implemented, along with all the methods of Enumerable.
     def each
-      raise NoMethodError, "Enumerable methods and #each not implemented for instance that is not like a hash or array: #{instance.pretty_inspect.chomp}"
+      raise NoMethodError, "Enumerable methods and #each not implemented for instance that is not like a hash or array: #{jsi_instance.pretty_inspect.chomp}"
     end
 
-    # an array of JSI instances above this one in the document. empty if this
-    # JSI is at the root or was instantiated from a source that does not have
-    # a document (e.g. a plain hash or array).
+    # an array of JSI instances above this one in the document.
     #
     # @return [Array<JSI::Base>]
-    def parents
-      parent = @ancestor
-      (@ancestor.instance.path.size...self.instance.path.size).map do |i|
+    def parent_jsis
+      parent = jsi_root_node
+
+      jsi_ptr.reference_tokens.map do |token|
         parent.tap do
-          parent = parent[self.instance.path[i]]
+          parent = parent[token]
         end
       end.reverse
     end
 
-    # the immediate parent of this JSI. nil if no parent(s) are known.
+    # the immediate parent of this JSI. nil if there is no parent.
     #
     # @return [JSI::Base, nil]
-    def parent
-      parents.first
+    def parent_jsi
+      parent_jsis.first
+    end
+
+    alias_method :parent_node, :parent_jsi
+
+    # @deprecated
+    alias_method :parents, :parent_jsis
+    # @deprecated
+    alias_method :parent, :parent_jsi
+
+    # @param token [String, Integer, Object] the token to subscript
+    # @return [JSI::Base, Object] the instance's subscript value at the given token.
+    #   if this JSI's schemas define subschemas which apply for the given token, and the value is complex,
+    #   returns the subscript value as a JSI instantiation of those subschemas. otherwise, the plain instance
+    #   value is returned.
+    def [](token)
+      if respond_to?(:to_hash)
+        token_in_range = node_content_hash_pubsend(:key?, token)
+        value = node_content_hash_pubsend(:[], token)
+      elsif respond_to?(:to_ary)
+        token_in_range = node_content_ary_pubsend(:each_index).include?(token)
+        value = node_content_ary_pubsend(:[], token)
+      else
+        raise(CannotSubscriptError, "cannot subcript (using token: #{token.inspect}) from instance: #{jsi_instance.pretty_inspect.chomp}")
+      end
+
+      result = jsi_memoize(:[], token, value, token_in_range) do |token, value, token_in_range|
+        if respond_to?(:to_ary)
+          token_schemas = jsi_schemas.map { |schema| schema.subschemas_for_index(token) }.inject(Set.new, &:|)
+        else
+          token_schemas = jsi_schemas.map { |schema| schema.subschemas_for_property_name(token) }.inject(Set.new, &:|)
+        end
+        token_schemas = token_schemas.map { |schema| schema.match_to_instance(value) }.inject(Set.new, &:|)
+
+        if token_in_range
+          complex_value = token_schemas.any? && (value.respond_to?(:to_hash) || value.respond_to?(:to_ary))
+          schema_value = token_schemas.any? { |token_schema| token_schema.describes_schema? }
+
+          if complex_value || schema_value
+            JSI::SchemaClasses.class_for_schemas(token_schemas).new(Base::NOINSTANCE, jsi_document: @jsi_document, jsi_ptr: @jsi_ptr[token], jsi_root_node: @jsi_root_node)
+          else
+            value
+          end
+        else
+          defaults = Set.new
+          token_schemas.each do |token_schema|
+            if token_schema.respond_to?(:to_hash) && token_schema.key?('default')
+              defaults << token_schema['default']
+            end
+          end
+
+          if defaults.size == 1
+            # use the default value
+            # we are using #dup so that we get a modified copy of self, in which we set dup[token]=default.
+            dup.tap { |o| o[token] = defaults.first }[token]
+          else
+            # I kind of want to just return nil here. the preferred mechanism for
+            # a JSI's default value should be its schema. but returning nil ignores
+            # any value returned by Hash#default/#default_proc. there's no compelling
+            # reason not to support both, so I'll return that.
+            value
+          end
+        end
+      end
+      result
+    end
+
+    # assigns the subscript of the instance identified by the given token to the given value.
+    # if the value is a JSI, its instance is assigned instead of the JSI value itself.
+    #
+    # @param token [String, Integer, Object] token identifying the subscript to assign
+    # @param value [JSI::Base, Object] the value to be assigned
+    def []=(token, value)
+      unless respond_to?(:to_hash) || respond_to?(:to_ary)
+        raise(NoMethodError, "cannot assign subcript (using token: #{token.inspect}) to instance: #{jsi_instance.pretty_inspect.chomp}")
+      end
+      jsi_clear_memo(:[])
+      if value.is_a?(Base)
+        self[token] = value.jsi_instance
+      else
+        jsi_instance[token] = value
+      end
     end
 
     # if this JSI is a $ref then the $ref is followed. otherwise this JSI
     # is returned.
     #
+    # @yield [JSI::Base] if a block is given (optional), this will yield a deref'd JSI. if this
+    #   JSI is not a $ref object, the block is not called. if we are a $ref which cannot be followed
+    #   (e.g. a $ref to an external document, which is not yet supported), the block is not called.
     # @return [JSI::Base, self]
-    def deref
-      derefed = instance.deref
-      if derefed.object_id == instance.object_id
-        self
-      else
-        self.class.new(derefed, ancestor: @ancestor)
+    def deref(&block)
+      node_ptr_deref do |deref_ptr|
+        deref_ptr.evaluate(jsi_root_node).tap(&(block || Util::NOOP))
       end
+      return self
     end
 
     # yields the content of the underlying instance. the block must result in
@@ -138,22 +316,25 @@ module JSI
     #   in a (nondestructively) modified copy of this.
     # @return [JSI::Base subclass the same as self] the modified copy of self
     def modified_copy(&block)
-      modified_instance = instance.modified_copy(&block)
-      self.class.new(modified_instance, ancestor: @ancestor)
-    end
-
-    def fragment
-      instance.fragment
+      if node_ptr.root?
+        modified_document = @jsi_ptr.modified_document_copy(@jsi_document, &block)
+        self.class.new(Base::NOINSTANCE, jsi_document: modified_document, jsi_ptr: @jsi_ptr)
+      else
+        modified_jsi_root_node = @jsi_root_node.modified_copy do |root|
+          @jsi_ptr.modified_document_copy(root, &block)
+        end
+        self.class.new(Base::NOINSTANCE, jsi_document: modified_jsi_root_node.node_document, jsi_ptr: @jsi_ptr, jsi_root_node: modified_jsi_root_node)
+      end
     end
 
-    # @return [Array<String>] array of schema validation error messages for this instance
-    def fully_validate
-      schema.fully_validate(instance)
+    # @return [Array] array of schema validation errors for this instance
+    def fully_validate(errors_as_objects: false)
+      jsi_schemas.map { |schema| schema.fully_validate_instance(jsi_instance, errors_as_objects: errors_as_objects) }.inject([], &:+)
     end
 
     # @return [true, false] whether the instance validates against its schema
     def validate
-      schema.validate(instance)
+      jsi_schemas.all? { |schema| schema.validate_instance(jsi_instance) }
     end
 
     # @return [true] if this method does not raise, it returns true to
@@ -161,299 +342,84 @@ module JSI
     # @raise [::JSON::Schema::ValidationError] raises if the instance has
     #   validation errors
     def validate!
-      schema.validate!(instance)
+      jsi_schemas.each { |schema| schema.validate_instance!(jsi_instance) }
+      true
+    end
+
+    def dup
+      modified_copy(&:dup)
     end
 
     # @return [String] a string representing this JSI, indicating its class
     #   and inspecting its instance
     def inspect
-      "\#<#{self.class.to_s} #{instance.inspect}>"
+      "\#<#{object_group_text.join(' ')} #{jsi_instance.inspect}>"
     end
 
     # pretty-prints a representation this JSI to the given printer
     # @return [void]
     def pretty_print(q)
-      q.instance_exec(self) do |obj|
-        text "\#<#{obj.class.to_s}"
-        group_sub {
-          nest(2) {
-            breakable ' '
-            pp obj.instance
-          }
+      q.text '#<'
+      q.text object_group_text.join(' ')
+      q.group_sub {
+        q.nest(2) {
+          q.breakable ' '
+          q.pp jsi_instance
         }
-        breakable ''
-        text '>'
-      end
+      }
+      q.breakable ''
+      q.text '>'
     end
 
-    # @return [String] the instance's object_group_text
+    # @return [Array<String>]
     def object_group_text
-      instance.object_group_text
-    end
-
-    # @return [Object] a jsonifiable representation of the instance
-    def as_json(*opt)
-      Typelike.as_json(instance, *opt)
-    end
-
-    # @return [Object] an opaque fingerprint of this JSI for FingerprintHash
-    def fingerprint
-      {class: self.class, instance: instance}
-    end
-    include FingerprintHash
-
-    private
-
-    # assigns @instance to the given thing, raising if the thing is not appropriate for a JSI instance
-    # @param thing [Object] a JSON schema instance for this class's schema
-    def instance=(thing)
-      if instance_variable_defined?(:@instance)
-        raise(JSI::Bug, "overwriting instance is not supported")
-      end
-      if thing.is_a?(Base)
-        warn "assigning instance to a Base instance is incorrect. received: #{thing.pretty_inspect.chomp}"
-        @instance = thing.instance
-      elsif thing.is_a?(JSI::JSON::Node)
-        @instance = thing
-      else
-        @instance = JSI::JSON::Node.new_doc(thing)
-      end
-    end
-
-    # assigns a subscript, taking care of memoization and unwrapping a JSI if given.
-    # @param subscript [Object] the bit between the [ and ]
-    # @param value [JSI::Base, Object] the value to be assigned
-    def subscript_assign(subscript, value)
-      clear_memo(:[], subscript)
-      if value.is_a?(Base)
-        instance[subscript] = value.instance
-      else
-        instance[subscript] = value
-      end
-    end
-
-    # this is an instance method in order to allow subclasses of JSI classes to
-    # override it to point to other subclasses corresponding to other schemas.
-    def class_for_schema(schema)
-      JSI.class_for_schema(schema)
-    end
-  end
-
-  # this module is just a namespace for schema classes.
-  module SchemaClasses
-    extend Memoize
-
-    # JSI::SchemaClasses[schema_id] returns a class for the schema with the
-    # given id, the same class as returned from JSI.class_for_schema.
-    #
-    # @param schema_id [String] absolute schema id as returned by {JSI::Schema#schema_id}
-    # @return [Class subclassing JSI::Base] the class for that schema
-    def self.[](schema_id)
-      @classes_by_id[schema_id]
-    end
-    @classes_by_id = {}
-  end
-
-  # see {JSI.class_for_schema}
-  def SchemaClasses.class_for_schema(schema_object)
-    memoize(:class_for_schema, JSI::Schema.from_object(schema_object)) do |schema_|
-      begin
-        begin
-          Class.new(Base).instance_exec(schema_) do |schema|
-            begin
-              include(JSI::SchemaClasses.module_for_schema(schema))
-
-              SchemaClasses.instance_exec(self) { |klass| @classes_by_id[klass.schema_id] = klass }
-
-              self
-            end
-          end
-        end
-      end
-    end
-  end
-
-  # a module for the given schema, with accessor methods for any object
-  # property names the schema identifies. also has class and instance
-  # methods called #schema to access the {JSI::Schema} this module
-  # represents.
-  #
-  # accessor methods are defined on these modules so that methods can be
-  # defined on {JSI.class_for_schema} classes without method redefinition
-  # warnings. additionally, these overriding instance methods can call
-  # `super` to invoke the normal accessor behavior.
-  #
-  # no property names that are the same as existing method names on the JSI
-  # class will be defined. users should use #[] and #[]= to access properties
-  # whose names conflict with existing methods.
-  def SchemaClasses.module_for_schema(schema_object)
-    memoize(:module_for_schema, JSI::Schema.from_object(schema_object)) do |schema_|
-      Module.new.tap do |m|
-        m.instance_exec(schema_) do |schema|
-          define_method(:schema) { schema }
-          define_singleton_method(:schema) { schema }
-          define_singleton_method(:included) do |includer|
-            includer.send(:define_singleton_method, :schema) { schema }
-          end
-
-          define_singleton_method(:schema_id) do
-            schema.schema_id
-          end
-          define_singleton_method(:inspect) do
-            %Q(#<Module for Schema: #{schema_id}>)
+      class_name = self.class.name unless self.class.in_schema_classes
+      class_txt = begin
+        if class_name
+          # ignore ID
+          schema_module_names = jsi_schemas.map { |schema| schema.jsi_schema_module.name }.compact
+          if schema_module_names.empty?
+            class_name
+          else
+            "#{class_name} (#{schema_module_names.join(', ')})"
           end
-
-          instance_method_modules = [m, Base, BaseArray, BaseHash]
-          instance_methods = instance_method_modules.map do |mod|
-            mod.instance_methods + mod.private_instance_methods
-          end.inject(Set.new, &:|)
-          accessors_to_define = schema.described_object_property_names.map(&:to_s) - instance_methods.map(&:to_s)
-          accessors_to_define.each do |property_name|
-            define_method(property_name) do
-              if respond_to?(:[])
-                self[property_name]
-              else
-                raise(NoMethodError, "instance does not respond to []; cannot call reader `#{property_name}' for: #{pretty_inspect.chomp}")
-              end
-            end
-            define_method("#{property_name}=") do |value|
-              if respond_to?(:[]=)
-                self[property_name] = value
-              else
-                raise(NoMethodError, "instance does not respond to []=; cannot call writer `#{property_name}=' for: #{pretty_inspect.chomp}")
-              end
-            end
+        else
+          schema_names = jsi_schemas.map { |schema| schema.jsi_schema_module.name || schema.schema_id }.compact
+          if schema_names.empty?
+            "JSI"
+          else
+            "JSI (#{schema_names.join(', ')})"
           end
         end
       end
-    end
-  end
-
-  # module extending a {JSI::Base} object when its schema instance is Hash-like (responds to #to_hash)
-  module BaseHash
-    # yields each key and value of this JSI.
-    # each yielded key is the same as a key of the instance, and each yielded
-    # value is the result of self[key] (see #[]).
-    # returns an Enumerator if no block is given.
-    # @yield [Object, Object] each key and value of this JSI hash
-    # @return [self, Enumerator]
-    def each
-      return to_enum(__method__) { instance.size } unless block_given?
-      instance.each_key { |k| yield(k, self[k]) }
-      self
-    end
-
-    # @return [Hash] a hash in which each key is a key of the instance hash and
-    #   each value is the result of self[key] (see #[]).
-    def to_hash
-      inject({}) { |h, (k, v)| h[k] = v; h }
-    end
-
-    include Hashlike
 
-    # methods that don't look at the value; can skip the overhead of #[] (invoked by #to_hash)
-    SAFE_KEY_ONLY_METHODS.each do |method_name|
-      define_method(method_name) { |*a, &b| instance.public_send(method_name, *a, &b) }
-    end
-
-    # @return [JSI::Base, Object] the instance's subscript value at the given
-    #   key property_name_. if there is a subschema defined for that property
-    #   on this JSI's schema, returns the instance's subscript as a JSI
-    #   instiation of that subschema.
-    def [](property_name_)
-      memoize(:[], property_name_) do |property_name|
-        begin
-          property_schema = schema.subschema_for_property(property_name)
-          property_schema = property_schema && property_schema.match_to_instance(instance[property_name])
-
-          if !instance.key?(property_name) && property_schema && property_schema.schema_object.key?('default')
-            # use the default value
-            default = property_schema.schema_object['default']
-            if default.respond_to?(:to_hash) || default.respond_to?(:to_ary)
-              class_for_schema(property_schema).new(default, ancestor: @ancestor)
-            else
-              default
-            end
-          elsif property_schema && (instance[property_name].respond_to?(:to_hash) || instance[property_name].respond_to?(:to_ary))
-            class_for_schema(property_schema).new(instance[property_name], ancestor: @ancestor)
-          else
-            instance[property_name]
-          end
+      if (is_a?(PathedArrayNode) || is_a?(PathedHashNode)) && ![Array, Hash].include?(node_content.class)
+        if node_content.respond_to?(:object_group_text)
+          node_content_txt = node_content.object_group_text
+        else
+          node_content_txt = [node_content.class.to_s]
         end
+      else
+        node_content_txt = []
       end
-    end
-
-    # assigns the given property name of the instance to the given value.
-    # if the value is a JSI, its instance is assigned.
-    # @param property_name [Object] this should generally be a String, but JSI
-    #   does not enforce any constraint on it.
-    # @param value [Object] the value to be assigned to the given subscript
-    #   property_name
-    def []=(property_name, value)
-      subscript_assign(property_name, value)
-    end
-  end
-
-  # module extending a {JSI::Base} object when its schema instance is Array-like (responds to #to_ary)
-  module BaseArray
-    # yields each element. each yielded element is the result of self[index]
-    # for each index of the instance (see #[]).
-    # returns an Enumerator if no block is given.
-    # @yield [Object] each element of this JSI array
-    # @return [self, Enumerator]
-    def each
-      return to_enum(__method__) { instance.size } unless block_given?
-      instance.each_index { |i| yield(self[i]) }
-      self
-    end
-
-    # @return [Array] an array, the same size as the instance, in which the
-    #   element at each index is the result of self[index] (see #[])
-    def to_ary
-      to_a
-    end
 
-    include Arraylike
-
-    # methods that don't look at the value; can skip the overhead of #[] (invoked by #to_a).
-    # we override these methods from Arraylike
-    SAFE_INDEX_ONLY_METHODS.each do |method_name|
-      define_method(method_name) { |*a, &b| instance.public_send(method_name, *a, &b) }
+      [
+        class_txt,
+        is_a?(Metaschema) ? "Metaschema" : is_a?(Schema) ? "Schema" : nil,
+        *node_content_txt,
+      ].compact
     end
 
-    # @return [Object] returns the instance's subscript value at the given index
-    #   i_. if there is a subschema defined for that index on this JSI's schema,
-    #   returns the instance's subscript as a JSI instiation of that subschema.
-    # @param i_ the array index to subscript
-    def [](i_)
-      memoize(:[], i_) do |i|
-        begin
-          index_schema = schema.subschema_for_index(i)
-          index_schema = index_schema && index_schema.match_to_instance(instance[i])
-
-          if !instance.each_index.to_a.include?(i) && index_schema && index_schema.schema_object.key?('default')
-            # use the default value
-            default = index_schema.schema_object['default']
-            if default.respond_to?(:to_hash) || default.respond_to?(:to_ary)
-              class_for_schema(index_schema).new(default, ancestor: @ancestor)
-            else
-              default
-            end
-          elsif index_schema && (instance[i].respond_to?(:to_hash) || instance[i].respond_to?(:to_ary))
-            class_for_schema(index_schema).new(instance[i], ancestor: @ancestor)
-          else
-            instance[i]
-          end
-        end
-      end
+    # @return [Object] a jsonifiable representation of the instance
+    def as_json(*opt)
+      Typelike.as_json(jsi_instance, *opt)
     end
 
-    # assigns the given index of the instance to the given value.
-    # if the value is a JSI, its instance is assigned.
-    # @param i [Object] the array index to assign
-    # @param value [Object] the value to be assigned to the given subscript i
-    def []=(i, value)
-      subscript_assign(i, value)
+    # @return [Object] an opaque fingerprint of this JSI for FingerprintHash. JSIs are equal
+    #   if their instances are equal, and if the JSIs are of the same JSI class or subclass.
+    def jsi_fingerprint
+      {class: jsi_class, jsi_document: jsi_document, jsi_ptr: jsi_ptr}
     end
+    include Util::FingerprintHash
   end
 end