json-schema-generator2 0.0.9

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: bbc6dd023b2df953628f8a813a30af6fe92c74c6da74ac4729c1c5c48a221f9c
+  data.tar.gz: ea00fbd7e22c89010dbab29d9dead34aeeb5fc21fe7dd46d6836448af430b42c
+SHA512:
+  metadata.gz: 821ffb27d6d612d7b78e8b7932080af6c65081f4c01f87331cc554d12977e7fe0ed25dae6765470507316655ff9856d2e1afffaf355f4871e1cd9ca17b999d45
+  data.tar.gz: ef218c2f8e6463c7d8a7a3720692bcbcab77017a59c0ea5759e545575363bcd16beda2e515906ea7d007aec51e494c2ce3f06621a0b26baff0d5451ac0564ff4

data/.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/.gitmodules

@@ -0,0 +1,3 @@
+[submodule "kata"]
+	path = kata
+	url = https://github.com/maxlinc/json-schema-kata.git

data/.travis.yml

@@ -0,0 +1,11 @@
+language: ruby
+rvm:
+- 2.0.0
+- 1.9.3
+- jruby-19mode
+- rbx-2.2.1
+matrix:
+  allow_failures:
+      - rvm: rbx-2.2.1
+before_install:
+  - git submodule update --init --recursive

data/Gemfile

@@ -0,0 +1,8 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in json-schema-generator.gemspec
+gemspec
+
+platforms :rbx do
+  gem 'rubysl'
+end

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Max Lincoln
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,78 @@
+# JSON::SchemaGenerator
+
+JSON::SchemaGenerator tries to save you some time writing [json-schemas](http://json-schema.org/) based on existing data.  I know that there are multiple valid json-schemas that could be generated for any given sample, so I won't generate the exact schema you want.  Our goal is just to make reasonable assumptions that get you close, so you can generate the basic structure and then customize your schema, rather than writing hundreds of lines by hand.
+
+There are [many json-schema validators], but only a few generators.  The best generator I've found is a closed-source web app so you can't embed it.  I couldn't find anything to embed within an open-source Ruby project ([Pacto](https://github.com/thoughtworks/pacto)), so I hacked one together quickly.
+
+My quick, yak-shaving implementaiton wasn't designed for maintainability, but I wrote a test suite that gives me the confidence to rewrite or even port to other languages.  The tests are in [json-schema-kata](https://github.com/maxlinc/json-schema-kata).  I called it a kata for a reason:
+
+> Code Kata is an attempt to bring this element of practice to software development. A kata is an exercise in karate where you repeat a form many, many times, making little improvements in each. The intent behind code kata is similar.
+>
+> Dave Thomas [Code Kata](http://codekata.pragprog.com/2007/01/code_kata_backg.html)
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'json-schema-generator'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install json-schema-generator
+
+## Usage
+
+Command line:
+```sh
+# Usage: json-schema-generator --help
+# Simple example:
+$ json-schema-generator my_sample.json --schema-version draft3
+```
+
+Ruby:
+```ruby
+file = 'my_sample.json' # or any identifier for the description
+JSON::SchemaGenerator.generate file, File.read(file), {:schema_version => 'draft3'}
+```
+
+## Features
+
+JSON::SchemaGenerate has the following features or assumptions for generating "best-guess" schemas:
+
+* **Schema Versions**: Support draft4 of json-schema.
+* **Options**:
+  * **Defaults**: Can generate default values.
+  * **Descriptions**: Can generate a description indicating where the schema came from.
+* **Features/Assumptions**:
+  * **Detecting optional properties:** if you are using arrays (even arrays with complex nested types), the generator will use all available data to figure detect if a field is optional.
+  * **Assume required:** in all other cases, I assume everything I find in the sample is required.  I believe it is better to generate a schema that is too strict than too lenient.  It is easy to review and fix false negatives, by updating the schema to mark those items as optional.  A false positive will go unnoticed and will not point you towards a solution.
+  * **Detect types:** I detect objects, arrays, strings, integers, numbers and booleans.
+
+## Known issues
+
+* Currently assumes the root element is a hash.  Generation will fail for data that has an array as the root.
+
+## Tests
+
+This was a [yak shaving](http://en.wiktionary.org/wiki/yak_shaving) project.  I needed to generate schemas for a ruby project.  I wanted to embed a schema generator within a larger open-source project ([Pacto](https://github.com/thoughtworks/pacto)), and couldn't find one I could use.  So I wrote one.
+
+There are [many json-schema validators], but only a few generators.  The best generator I've found is a closed-source Web app so you can't embed it.  I'm hoping more people will write generators in more languages.  I also hope someone replaces my quick "yak-shaving" implementation
+There are many json-schema validators available in many different programming languages.  However, there are only a few generators.  I hacked this together quickly because I wanted one I could easily use in Ruby.  My implementaiton is yak-shaving, not a well-design and maintainable solution.  So I wanted acceptance tests that are fully decoupled from the implementation - even the programming language - so I can completely re-write my solution or port it to other languages.
+
+The tests are in the json-schema-kata project.  The test runner is ruby, but it makes no assumptions about the implementation language.  Feel free to use my tests to port to any language.
+
+## Contributing
+
+Contributions Welcome!  It's easy to contribute tests to the json-schema-kata project.  You just need to add a sample and what you think the generated schema should look like for each supported json-schema version (draft3, draft4, etc).
+
+Rewrites, ports, tutorials: I called the tests a "kata" because it is a small problem that is good for practicing programming skills, but could be solved many different ways.  Can you implement a generator using functional programming?  An internal DSL?  JSONPath?  Seven languages in seven Weeks?  Try it out, hone your skills, and share your results.
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,15 @@
+require "bundler/gem_tasks"
+
+task :default => :test
+
+task :test do
+  ENV['PATH'] = ENV['PATH'] + File::PATH_SEPARATOR + File.expand_path('bin', Dir.pwd)
+  puts ENV['PATH']
+  Bundler.with_clean_env do
+    Dir.chdir 'kata' do
+      system 'bundle install'
+      system 'bundle exec rake'
+    end
+  end
+  fail 'Tests did not pass!' unless $?.success?
+end

data/bin/json-schema-generator

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require 'json-schema-generator'
+require 'json/schema_generator_cli'
+JSON::SchemaGeneratorCLI.new(ARGV.dup).execute!

data/json-schema-generator.gemspec

@@ -0,0 +1,28 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'json/schema_generator/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "json-schema-generator2"
+  spec.version       = JSON::SchemaGenerator::VERSION
+  spec.authors       = ["Max Lincoln"]
+  spec.email         = ["max@devopsy.com"]
+  spec.description   = %q{A very basic json-schema generator}
+  spec.summary       = %q{A very basic json-schema generator}
+  spec.homepage      = ""
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files`.split($/)
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency "jsonpath"
+
+  spec.add_development_dependency "bundler", "~> 1.3"
+  spec.add_development_dependency "rake"
+  spec.add_development_dependency "rspec"
+  spec.add_development_dependency "json-schema"
+  spec.add_development_dependency "travis"
+end

data/lib/json-schema-generator.rb

@@ -0,0 +1,5 @@
+require 'digest/sha1'
+require 'stringio'
+require 'json'
+require 'json/schema_generator'
+require 'json/schema_generator_cli'

data/lib/json/schema_generator.rb

@@ -0,0 +1,130 @@
+require 'json/schema_generator/statement_group'
+require 'json/schema_generator/brute_force_required_search'
+
+module JSON
+  class SchemaGenerator
+
+    class << self
+      def generate name, data, opts = {}
+        JSON::SchemaGenerator.new(name, opts).generate data
+      end
+    end
+
+    def initialize name, opts = {}
+      @defaults = opts[:defaults]
+      @allow_null = opts[:allow_null]
+
+      @buffer = StringIO.new
+      @name = name
+    end
+
+    def generate raw_data
+      data = JSON.load(raw_data)
+      @brute_search = BruteForceRequiredSearch.new data
+
+      statement_group = StatementGroup.new
+      statement_group.add "\"$schema\": \"http://json-schema.org/draft4/schema#\""
+      statement_group.add "\"description\": \"Generated from json_schema_generator #rusteze\""
+      case data
+      when Array
+        $stop = true
+        create_array(statement_group, data, detect_required(data))
+      else
+        create_hash(statement_group, data, detect_required(data))
+      end
+      @buffer.puts statement_group
+      result
+    end
+
+    protected
+
+    def create_primitive(statement_group, key, value, required_keys)
+      if required_keys.nil?
+        required = true
+      else
+        required = required_keys.include? key
+      end
+
+      type = case value
+      when TrueClass, FalseClass
+        "boolean"
+      when String
+        "string"
+      when Integer, Float
+        "number"
+      else
+        raise "Unknown Primitive Type for #{key}! #{value.class}"
+      end
+
+      if @allow_null
+        statement_group.add "\"type\": #{[type, "null"]}"
+      else
+        statement_group.add "\"type\": \"#{type}\""
+      end
+      # statement_group.add "\"oneOf\": [{\"type\": \"#{type}\"}, {\"type\": \"null\"}]"
+      statement_group.add "\"default\": #{value.inspect}" if @defaults
+    end
+
+    def create_values(key, value, required_keys = nil, in_array = false)
+      statement_group = StatementGroup.new key
+      case value
+      when NilClass
+      when TrueClass, FalseClass, String, Integer, Float
+        create_primitive(statement_group, key, value, required_keys)
+      when Array
+        create_array(statement_group, value, detect_required(value))
+      when Hash
+        if in_array
+          create_hash(statement_group, value, required_keys)
+        else
+          create_hash(statement_group, value, detect_required(value))
+        end
+      else
+        raise "Unknown Type for #{key}! #{value.class}"
+      end
+      statement_group
+    end
+
+    def create_hash(statement_group, data, required_keys)
+      statement_group.add '"type": "object"'
+      required_keys ||= []
+      required_string = required_keys.map(&:inspect).join ', '
+      statement_group.add "\"required\": [#{required_string}]" unless required_keys.empty?
+      statement_group.add create_hash_properties data, required_keys
+      statement_group
+    end
+
+    def create_hash_properties(data, required_keys)
+      statement_group = StatementGroup.new "properties"
+      data.collect do |k,v|
+        @brute_search.push k,v
+        statement_group.add create_values k, v, required_keys
+        @brute_search.pop
+      end
+      statement_group
+    end
+
+    def create_array(statement_group, data, required_keys)
+      statement_group.add '"type": "array"'
+
+      # FIXME - Code assumes that all items in the array have the same structure
+      # Assume lowest common denominator - allow 0 items and unique not required
+        statement_group.add '"minItems": 0'
+
+      # TODO - consider a eq? method for StatementGroup class to evaluate LCD schema from all items in array
+      statement_group.add create_values("items", data.first, required_keys, true)
+
+      statement_group
+    end
+
+    def detect_required(collection)
+      @brute_search.find_required
+    rescue NoMethodError
+      collection.keys if collection.respond_to?(:keys)
+    end
+
+    def result
+      @buffer.string
+    end
+  end
+end

data/lib/json/schema_generator/brute_force_required_search.rb

@@ -0,0 +1,46 @@
+require 'jsonpath'
+
+module JSON
+  class SchemaGenerator
+    class BruteForceRequiredSearch
+      def initialize(data)
+        @data = data.dup
+        @json_path = data.is_a?(Array) ? ['$[*]'] : ['$']
+      end
+
+      def push(key, value)
+        @json_path.push value.is_a?(Array) ? "#{key}[*]" : key
+      end
+
+      def pop
+        @json_path.pop
+      end
+
+      def current_path
+        @json_path.join '.'
+      end
+
+      def search_path search_key
+        current_path.gsub(/\[\*\]$/, "[?(@.#{search_key})]")
+      end
+
+      def required? child_key
+        begin
+          JsonPath.new(search_path(child_key)).on(@data).count == JsonPath.new(current_path).on(@data).count
+        rescue SyntaxError
+          # There are some special characters that can't be handled by JsonPath.
+          # e.g. if child key is OS-DCF:diskConfig
+          false
+        end
+      end
+
+      def child_keys
+        JsonPath.new(current_path).on(@data).map(&:keys).flatten.uniq
+      end
+
+      def find_required
+        child_keys.select {|k| required? k}
+      end
+    end
+  end
+end

data/lib/json/schema_generator/statement_group.rb

@@ -0,0 +1,26 @@
+module JSON
+  class SchemaGenerator
+    class StatementGroup
+      def initialize key = nil
+        @key = key
+        @statements = []
+      end
+
+      def add statement
+        @statements << statement
+      end
+
+      def to_s
+        buffer = StringIO.new
+        if @key.nil?
+          buffer.puts "{"
+        else 
+          buffer.puts "\"#{@key}\": {"
+        end
+        buffer.puts @statements.join ', '
+        buffer.puts "}"
+        buffer.string
+      end
+    end
+  end
+end

data/lib/json/schema_generator/version.rb

@@ -0,0 +1,5 @@
+module JSON
+  class SchemaGenerator
+    VERSION = "0.0.9"
+  end
+end

data/lib/json/schema_generator_cli.rb

@@ -0,0 +1,36 @@
+#!/usr/bin/env ruby
+
+require 'optparse'
+
+class JSON::SchemaGeneratorCLI
+  def initialize(argv, stdin=STDIN, stdout=STDOUT, stderr=STDERR, kernel=Kernel)
+    @argv, @stdin, @stdout, @stderr, @kernel = argv, stdin, stdout, stderr, kernel
+  end
+
+  def execute!
+    default_version = 'draft4'
+    supported_versions = ['draft4']
+
+    options = {
+      :schema_version => default_version,
+      :defaults => false
+    }
+
+
+
+    OptionParser.new do |opts|
+      opts.on("--defaults", "Record default values in the generated schema") { options[:defaults] = true }
+      opts.on("--schema-version draft4", [:draft4],
+        "Version of json-schema to generate (#{supported_versions.join ', '}).  Default: #{default_version}") do |schema_version|
+          options[:schema_version] = schema_version
+        end
+      opts.on("--allow-null", "Includes 'null' as an allow type for all properties in the schema") { options[:allow_null] = true }
+      opts.parse!
+    end
+
+    file = ARGV.shift
+    schema = JSON.parse(JSON::SchemaGenerator.generate file, File.read(file), options)
+    @stdout.puts JSON.pretty_generate schema
+    @kernel.exit(0)
+  end
+end

data/scripts/challenges/generate_defaults

@@ -0,0 +1,3 @@
+#!/bin/bash
+file=$1
+bundle exec json-schema-generator $1 --schema-version draft4 --defaults

data/scripts/challenges/generate_draft3

@@ -0,0 +1,3 @@
+#!/bin/bash
+file=$1
+bundle exec json-schema-generator $1 --schema-version draft3

data/scripts/challenges/generate_draft4

@@ -0,0 +1,3 @@
+#!/bin/bash
+file=$1
+bundle exec json-schema-generator $1

metadata

@@ -0,0 +1,146 @@
+--- !ruby/object:Gem::Specification
+name: json-schema-generator2
+version: !ruby/object:Gem::Version
+  version: 0.0.9
+platform: ruby
+authors:
+- Max Lincoln
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2020-09-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: jsonpath
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: json-schema
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: travis
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: A very basic json-schema generator
+email:
+- max@devopsy.com
+executables:
+- json-schema-generator
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".gitmodules"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/json-schema-generator
+- json-schema-generator.gemspec
+- lib/json-schema-generator.rb
+- lib/json/schema_generator.rb
+- lib/json/schema_generator/brute_force_required_search.rb
+- lib/json/schema_generator/statement_group.rb
+- lib/json/schema_generator/version.rb
+- lib/json/schema_generator_cli.rb
+- scripts/challenges/generate_defaults
+- scripts/challenges/generate_draft3
+- scripts/challenges/generate_draft4
+homepage: ''
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.8
+signing_key:
+specification_version: 4
+summary: A very basic json-schema generator
+test_files: []