doc_repo 0.1.1 → 1.0.0.pre.beta.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 719ad86ec23cfdb739553ff34cea62ea834a529b
-  data.tar.gz: 6ac8ba2b6b0a5b32045f77a5622d6b552a2935ee
+  metadata.gz: 30a5493aa98e1c88a25bc6b485716d6e8aecf14b
+  data.tar.gz: 9ffdcb7925e8e3743dca930bfe03a46669305b4d
 SHA512:
-  metadata.gz: f069ab4f8c94d1cbb1bed98d7025d0d6f80b3eb7748f9f9086ab80eef0907a14540c6c300f68c88624239d14506ab0782494f402b61372e9f7aff52eb619e92e
-  data.tar.gz: fdbb092b9c9cd12fec33ac10a5e8e0c303e3e458df0c16df56784fd78d0d6d6967aae8e7fda69d3c05ac6b5284498c7b1a8ad2f038f8c32e0c57622879b4d211
+  metadata.gz: 0b1b66bdfe4e58c16bd0c173c65795c96ea5aaacb313376a92927a5bc8ca60d0d18032aca18db0259bcdb86400a0bd77d4949f33e5dafbc2e5d1549d9a54bd1b
+  data.tar.gz: 70ab80ee398d1491c22255206f41573a9268fe6adc87113a02269165dcd54973292797e10018c9b0c2ad86a3eb3f256e65f56e7e0a508a566ca6d038527e01c2

data/.gitignore

@@ -18,3 +18,6 @@ mkmf.log
 # Ignore bundler related files
 /bundle/
 *.bundle
+
+# Ignore RSpec runner related files
+/spec/examples.txt

data/.travis.yml

@@ -1,14 +1,14 @@
 language: ruby
 sudo: false
+distro: trusty
 bundler_args: --binstubs --standalone --without documentation debug
 script: bin/rspec
 rvm:
-  - 2.0
-  - 2.1
+  - 2.4     # Always try testing against latest patch release
+  - 2.4.0
+  - 2.4.1
   - ruby-head
-  - rbx-2
 matrix:
   allow_failures:
     - rvm: ruby-head
-    - rvm: rbx-2
   fast_finish: true

data/CHANGELOG.md

@@ -0,0 +1,47 @@
+## v1.0.0-beta.1 / 2017-07-07
+
+[Full Changelog](http://github.com/RadiusNetworks/doc_repo/compare/v0.1.1...v1.0.0-beta.1)
+
+Enhancements:
+
+- Add support for 2.4 (Aaron Kromer, #5)
+- Add new configuration settings (Aaron Kromer, #8)
+  - `cache_options`
+  - `cache_store`
+  - `doc_formats`
+  - `doc_root`
+  - `fallback_ext`
+- Add ability to provide custom error handling through callback to avoid
+  control flow by exception (Aaron Kromer, #8)
+- Provide ability to specify separate `not_found` and `error` handlers with
+  `not_found` falling back to the `error` handler when not set (Aaron Kromer,
+  #8)
+- Support local HTTP cache reducing remote origin requests (Aaron Kromer, #8)
+- Further Reduce remote origin requests through use of `Expires` header with
+  conditional `GET` requests using `If-None-Match` and `If-Modified-Since` when
+  local HTTP cache is configured (Aaron Kromer, #8)
+- Support Rails view caching of `DocRepo::Doc` instances (Aaron Kromer, #8)
+- Support Rails 5.2 recyclable view caches for `DocRepo::Doc` instances (Aaron
+  Kromer, #8)
+- Support Rails conditional `GET` through `fresh_when` and `stale?` for
+  `DocRepo::Doc` instances (Aaron Kromer, #8)
+
+Bug Fixes:
+
+- Add back the `DocRepo.configuration=` writer (Aaron Kromer, #5)
+
+Breaking Changes:
+
+- Drop support for Ruby 2.0, 2.1, and 2.2 (Aaron Kromer, #5)
+- Drop support for Ruby 2.3 (Aaron Kromer, #8)
+- Drop support for inheriting configuration from `ENV` (Aaron Kromer, #8)
+- Drop the following classes (Aaron Kromer, #8)
+  - `DocRepo::BadPageFormat`
+  - `DocRepo::NotFound`
+  - `DocRepo::GithubFile`
+  - `DocRepo::Page`
+  - `DocRepo::Response`
+- `DocRepo#respond_with` is now `DocRepo#request` and the object yielded
+  to the block uses handler blocks (Aaron Kromer, #8)
+- `DocRepo::Repository` requires a `DocRepo::Configuration` on creation (Aaron
+  Kromer, #8)

data/Gemfile

@@ -7,3 +7,8 @@ group :debug do
   gem "pry-nav"
   gem "travis-lint", require: false
 end
+
+group :benchmarks do
+  gem "kalibera", require: false
+  gem "benchmark-ips", require: false
+end

data/README.md

@@ -1,8 +1,12 @@
 # DocRepo
 
-Store your markdown based documentation in a repo but serve it from with in your app.
+Store your markdown based documentation in a repo but serve it from with in
+your app.
 
-This is a little project that will pull raw markdown from the GitHub API and proxy them through your app. This lets you render things in your app, customize the layout and access control -- but lets you update the docs without re-deploying.
+This is a little project that will pull raw markdown from the GitHub API and
+proxy them through your app. This lets you render things in your app, customize
+the layout and access control -- but lets you update the docs without
+re-deploying.
 
 
 ## Installation
@@ -15,30 +19,177 @@ gem 'doc_repo'
 
 And then execute:
 
-    $ bundle
+```console
+$ bundle
+```
 
 Or install it yourself as:
 
-    $ gem install doc_repo
+```console
+$ gem install doc_repo
+```
 
 ## Usage
 
-Create an initializer to configure Doc Repo, in rails this would live in `config/initializers/doc_repo.rb`:
+Initialize the configuration through `DocRepo.configuration`. It's good to
+place this somewhere early in the app startup (such as a Rails initializer
+`config/initializers/doc_repo.rb`):
 
 ```ruby
 DocRepo.configure do |c|
-  # GitHub Orgnization or User:
+  # GitHub Organization or User:
   c.org = "RadiusNetworks"
 
   # GitHub Repo:
-  c.repo = "proximitykit-documentation"
+  c.repo = "doc_repo"
 
-  # Git Branch (Optional):
+  # Git Branch (Optional - default is 'master'):
   c.branch = "master"
 end
 ```
 
-Create a controller to render the documentation pages. In Rails you might use something like this:
+Requests for documents can then be made through `DocRepo.request`:
+
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    # Do something with the document
+  end
+
+  on.redirect do |target|
+    # The asset exists else where and should be requested directly
+  end
+end
+```
+
+### Advanced Configuration
+
+Most functionality in Doc Repo can be configured. The full list of available
+settings is:
+
+```ruby
+DocRepo.configure do |c|
+  # Repo settings
+  c.org = "YourOrg"
+  c.repo = "your_repo"
+  c.branch = "api-v2"               # Default: "master"
+  c.doc_root = "/api-docs/rest"     # Default: "docs"
+
+  # Content settings
+  c.doc_formats = %w[               # Default: %w[ .md .markdown .htm .html ]
+    .md
+    .mark
+    .txt
+  ]
+  c.fallback_ext = ".mark"          # Default: ".md"
+
+  # Cache settings
+  c.cache_store = Rails.cache       # Default: DocRepo::NullCache.instance
+  c.cache_options = {               # Default: {}
+    namespace: :docs,
+    expires_in: 36.hours,
+  }
+end
+```
+
+When a request is made for a URI with an extension not listed in `doc_formats`
+the `redirect` handler will be called without making a remote request.
+
+### Error Handling
+
+When no error handling is configured errors are raised:
+
+```ruby
+def show
+  DocRepo.request(params[:slug]) do |on|
+    on.complete do |doc|
+      # Do something with the document
+    end
+
+    on.redirect do |target|
+      # The asset exists else where and should be requested directly
+    end
+  end
+rescue DocRepo::Error => error
+  # Handle the error
+end
+```
+
+However, errors such as a missing document may be more common and behavior
+handling should be treated differently. An example of this is a Rails app which
+pulls the document name from the URL. When someone mistypes the URL that isn't
+really an internal error.
+
+Doc Repo provides an alternative interface to avoid control flow by exception.
+This interface also allows separating behavior for the common missing document
+from other error cases:
+
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    # Do something with the document
+  end
+
+  on.redirect do |target|
+    # The asset exists else where and should be requested directly
+  end
+
+  on.not_found do |error|
+    # Handle the missing document
+  end
+
+  on.error do |error|
+    # Handle the error
+  end
+end
+```
+
+When the `not_found` handler is left undefined the defined `error` handler will
+be called. When this is not defined the default `raise` behavior is used.
+
+### Caching
+
+By default no caching is enabled. Specifically the default configuration uses a
+null cache which results in all requests being sent to the remote origin
+server. Custom cache stores can configured through the `cache_store`
+configuration setting. In order for Doc Repo to work with the custom cache it
+must implement the following APIs (existing Rails cache stores implement
+these):
+
+  - `fetch(key, options = {}, &block)`
+  - `write(key, value, options = {})`
+
+Any configured `cache_options` are provided directly to the `cache_store` for
+all `fetch` and `write` calls.
+
+When a custom cache store is configured it will be used as an internal local
+HTTP cache. This HTTP cache will prevent remote origin requests when possible.
+This is accomplished by serving content from local cache as long as the cache
+is valid per the cache store. Additionally, Doc Repo supports a basic
+understanding of general HTTP cache through the
+[`Expires`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires)([RFC
+7234](https://tools.ietf.org/html/rfc7234#section-5.3)) header.
+
+When a local HTTP cache has expired according to the `Expires` header, but is
+still valid in `cache_store`, a conditional `GET` request will be made to the
+origin server. Any [`ETag`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag)([RFC
+7232](https://tools.ietf.org/html/rfc7232#section-2.3)) or
+[`Last-Modified`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified)([RFC
+7232](https://tools.ietf.org/html/rfc7232#section-2.2)) headers originally
+provided by the origin server will be sent in the request through
+[`If-None-Match`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match)([RFC
+7232](https://tools.ietf.org/html/rfc7232#section-3.2)) and
+[`If-Modified-Since`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since)([RFC
+7232](https://tools.ietf.org/html/rfc7232#section-3.3)) headers respectively.
+
+Based on the response either the existing cache will be refreshed (i.e. in
+response to a `304 Not Modified`) or replaced (i.e. in response to a `200 OK`).
+This will cause the local HTTP cache to be re-written to the `cache_store`.
+
+### Rails
+
+We suggest creating a controller to render the documentation pages. A simple
+implementation may look like the following:
 
 ```ruby
 class DocsController < ApplicationController
@@ -49,20 +200,135 @@ class DocsController < ApplicationController
   end
 
   def show
-    DocRepo.respond_with(params[:slug]) do |f|
-      # Render the body:
-      f.html {|body| render text: body, layout: "docs" }
+    DocRepo.request(params[:slug]) do |on|
+      on.complete do |target|
+        @doc = doc
+        fresh_when @doc
+      end
 
-      # Redirect to images and assets:
-      f.redirect {|url| redirect_to url }
+      on.redirect do |target|
+        redirect_to target.location, status: target.code
+      end
+
+      on.not_found do |error|
+        logger.warn "Not Found (URI=#{error.uri})"
+        render file: "public/404.html", status: :not_found, layout: false
+      end
     end
-  rescue DocRepo::NotFound
-    raise ActionController::RoutingError.new('Not Found')
   end
+end
+```
+
+#### Rendering and Views
+
+By default all `DocRepo::Doc` instances will generate safe HTML when provided
+to `render` as the following types `:html`, `:plain`, and `:body`:
 
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    # These two lines are equivalent
+    render html: doc.to_html.html_safe
+    render html: doc
+
+    # As are these
+    render plain: doc.to_html.html_safe
+    render plain: doc
+  end
 end
 ```
 
+For those documents which are written in markdown, if you wish to provide a way
+to display the raw markdown you will need to explicitly provide it through
+`DocRepo::Doc#content`:
+
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    respond_to do |format|
+      format.html { render html: doc }
+
+      format.text { render plain: doc.content }
+    end
+  end
+end
+```
+
+Inside of a view you will need to call `to_html`, `content` or `to_text` as
+appropriate:
+
+```erb
+<%== doc.to_html %>
+```
+
+```erb
+<%= doc.to_html.html_safe %>
+```
+
+#### View Caches and Conditional `GET` Support
+
+The above mentioned caching behavior does not hook into the Rails view cache
+nor the conditional `GET` request/response interfaces. However, `DocRepo::Doc`
+instances provided to the `complete` handler do implement the necessary
+interfaces.
+
+You can explicitly define how to handle conditional `GET` through `stale?` or
+`fresh_when`:
+
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    @doc = doc
+    fresh_when strong_etag: doc.cache_key_with_version, last_modified: doc.last_modified
+  end
+end
+```
+
+Alternatively, you can provide the document instance directly:
+
+```ruby
+DocRepo.request(params[:slug]) do |on|
+  on.complete do |doc|
+    @doc = doc
+    fresh_when @doc
+  end
+end
+```
+
+This also applies to view caches:
+
+```erb
+<% cache @doc do %>
+  <%== @doc.to_html %>
+<% end %>
+```
+
+#### Rails 5.1 and Earlier Cache Keys
+
+The gem will attempt to check the Rails version when it is loaded and the
+`Rails` module is defined. When it detects a version prior to 5.2 it will load
+a patch which retains the legacy behavior of `DocRepo::Doc#cache_key`
+containing version information. On theses versions of Rails
+`DocRepo::Doc#cache_key_with_version` will simply be an alias for `cache_key`.
+
+#### Rails 5.2 Recyclable View Caches
+
+Support for this feature is built-in. The default implementation for
+`DocRepo::Doc#cache_key` does not include the version. Additionally,
+`DocRepo::Doc#cache_key_with_version` is already available to provide a
+versioned implementation. This means Rails view caches can be recycled while
+conditional `GET` calls through `fresh_when` and `stale?` continue to behave as
+expected.
+
+While we do not suggest it, if you wish to explicitly retain the legacy
+`cache_key` behavior then you will need to load it through an initializer:
+
+```ruby
+# config/initializers/doc_repo.rb
+require 'doc_repo/rails/legacy_versioned_cache'
+```
+
+
 ## Contributing
 
 1. Fork it

data/benchmarks/digests.rb

@@ -0,0 +1,55 @@
+# Run from the command line: bundle exec ruby benchmarks/reject_extract.rb
+require 'kalibera'
+require 'benchmark/ips'
+require 'digest'
+
+MESSAGE = <<~MESSAGE
+  Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum pulvinar
+  magna metus, vel facilisis sapien mollis sit amet. Pellentesque bibendum mi a
+  mi malesuada dignissim. Donec pharetra purus at finibus pretium. Sed vitae
+  odio lobortis, bibendum mi id, eleifend mauris. Integer varius molestie
+  bibendum.
+MESSAGE
+
+Benchmark.ips do |x|
+  x.config stats: :bootstrap, confidence: 95
+
+  x.report("MD5") do
+    Digest::MD5.hexdigest MESSAGE
+  end
+
+  x.report("RMD160") do
+    Digest::RMD160.hexdigest MESSAGE
+  end
+
+  x.report("SHA1") do
+    Digest::SHA1.hexdigest MESSAGE
+  end
+
+  x.report("SHA2") do
+    Digest::SHA2.hexdigest MESSAGE
+  end
+
+  x.compare!
+end
+
+__END__
+
+Warming up --------------------------------------
+                 MD5    32.125k i/100ms
+              RMD160    21.157k i/100ms
+                SHA1    30.969k i/100ms
+                SHA2    17.655k i/100ms
+Calculating -------------------------------------
+                 MD5    402.545k (± 0.9%) i/s -      2.024M in   5.035449s
+              RMD160    239.013k (± 0.7%) i/s -      1.206M in   5.049813s
+                SHA1    402.324k (± 0.8%) i/s -      2.013M in   5.008906s
+                SHA2    199.182k (± 0.8%) i/s -      1.006M in   5.058252s
+                   with 95.0% confidence
+
+Comparison:
+                 MD5:   402545.1 i/s
+                SHA1:   402323.6 i/s - same-ish: difference falls within error
+              RMD160:   239013.3 i/s - 1.68x  (± 0.02) slower
+                SHA2:   199182.2 i/s - 2.02x  (± 0.03) slower
+                   with 95.0% confidence

data/doc_repo.gemspec

@@ -18,13 +18,13 @@ Gem::Specification.new do |spec|
   spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
   spec.require_paths = ["lib"]
 
-  spec.required_ruby_version = '~> 2.0'
+  spec.required_ruby_version = '~> 2.3'
 
-  spec.add_dependency "rouge", "~> 1.8"
+  spec.add_dependency "rouge", "~> 2.1"
   spec.add_dependency "redcarpet", "~> 3.2"
 
   spec.add_development_dependency "bundler", "~> 1.6"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "rspec", "~> 3.0"
-  spec.add_development_dependency "webmock", "~> 1.0"
+  spec.add_development_dependency "rake", "~> 12.0"
+  spec.add_development_dependency "rspec", "~> 3.6"
+  spec.add_development_dependency "webmock", "~> 3.0"
 end

data/lib/doc_repo/configuration.rb

@@ -1,11 +1,72 @@
+# frozen_string_literal: true
+require 'singleton'
+
 module DocRepo
+  class NullCache
+    include Singleton
+    instance.freeze
+
+    def fetch(name, options = nil)
+      yield
+    end
+
+    def write(name, value, options = nil)
+    end
+  end
+
   class Configuration
-    attr_accessor :org, :repo, :branch
+    Settings = Module.new
+    include Settings
+
+    def self.setting(name, default: nil)
+      attr_writer name
+      Settings.module_exec do
+        attr_reader name
+      end
+      if default
+        define_method(name) do
+          super() || default
+        end
+      end
+    end
+    private_class_method :setting
+
+    setting :branch, default: "master"
+
+    setting :cache_options, default: {}
+
+    setting :cache_store, default: NullCache.instance
+
+    setting :doc_formats, default: %w[
+      .md
+      .markdown
+      .htm
+      .html
+    ].freeze
+
+    setting :doc_root, default: "docs"
+
+    setting :fallback_ext, default: ".md"
+
+    setting :org
+
+    setting :repo
 
     def initialize
-      @org    = ENV['DOC_REPO_ORG']
-      @repo   = ENV['DOC_REPO_REPONAME']
-      @branch = ENV['DOC_REPO_BRANCH'] || "master"
+      yield self if block_given?
+    end
+
+    def inspect
+      settings = to_h.map { |setting, value| "#{setting}=#{value.inspect}" }
+                     .join(", ")
+      "#<#{self.class}:0x%014x #{settings}>" % (object_id << 1)
+    end
+
+    def to_h
+      Settings.instance_methods(_include_super = false)
+              .each_with_object({}) { |setting, hash|
+                hash[setting] = public_send(setting)
+              }
     end
   end
 end

data/lib/doc_repo/doc.rb

@@ -0,0 +1,84 @@
+# frozen_string_literal: true
+require_relative 'converters/markdown_parser'
+require 'digest'
+require 'time'
+
+module DocRepo
+  class Doc
+    include HttpResult
+
+    # @api private
+    module Cache
+      attr_reader :cache_key, :cache_version
+
+      def cache_control
+        http['Cache-Control']
+      end
+
+      def cache_key_with_version
+        "#{cache_key}-#{cache_version}"
+      end
+    end
+    include Cache
+
+    def initialize(uri, http_response)
+      @http = http_response
+      init_result_readers(uri, @http.code)
+      @cache_key = uri.dup.freeze
+
+      # The Github raw server currently provides ETags for content. It's
+      # possible a change in servers/APIs may cause some content to no longer
+      # produce ETags. Additionally, for app cache versioning we have a default
+      # opinion that only the raw content important.
+      #
+      # For these reasons we calculate the cache version based solely on the
+      # raw content, ignoring any provided ETags or lack of one. As this cache
+      # version is meant to be used for general change comparisons we are not
+      # overly concerned with cryptographic level collision prevention or
+      # security. We'd rather have something reasonably fast to keep overhead
+      # down.
+      #
+      # As MD5 and SHA1 are roughly equivalent in speed (see
+      # benchmarks/digests.rb) we choose the latter as it provides a reduction
+      # in collisions at almost no additional cost.
+      @cache_version = Digest::SHA1.hexdigest(@http.body.to_s).freeze
+
+      # NOTE: Not set by Github raw site - we include it for future proofing
+      @last_modified = Time.httpdate(@http['Last-Modified']).freeze rescue nil
+    end
+
+    attr_reader :last_modified
+
+    attr_reader :http
+    private :http
+
+    def content
+      http.body
+    end
+
+    def content_type
+      # NOTE: The Github raw site does not respond with anything other than
+      # `text/plain` for general HTTP errors.
+      http['Content-Type']
+    end
+
+    def success?
+      true
+    end
+
+    def to_html(_options = {})
+      Converters::MarkdownParser.new(
+        extensions: %i[
+          no_intra_emphasis
+          tables
+          fenced_code_blocks
+          autolink
+          strikethrough
+          lax_spacing
+          superscript
+          with_toc_data
+        ]
+      ).convert(content.to_s)
+    end
+  end
+end

data/lib/doc_repo/error.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+module DocRepo
+  Error = Class.new(StandardError)
+
+  class UnhandledAction < Error
+    def initialize(action, msg = nil)
+      super(msg)
+      @action = action.to_s
+    end
+
+    attr_reader :action
+  end
+end