turbostreamer 1.10.0 → 2.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 63f1e21caf6c87b095cbd4528226ebbc0489dc7f58ef6de192ed222b1920863e
-  data.tar.gz: d7e9e7237f6fe49b46c114a39da1b6983db8cdc758a4b0d10cb9ce97d65a7ad4
+  metadata.gz: 0304ed518c7b493ea45ca44b3b3165d88a8f8171b7a3492b86c95a0c7b856617
+  data.tar.gz: e8b9e0d3335c16375794e0de491aa34ef176e06f2d49754e918f28ea492fe305
 SHA512:
-  metadata.gz: 69a4204a0598eb92fae7a08c1385241b0f648998eb5f6762c42bd61c8ded3689078ed1930b06e15383fc5ab744f5e1f8b85b6efd2eda58cbc125831683a8ca30
-  data.tar.gz: e8002ae1e4369b59071dff8a74f3f57d2e51d63c242c86023e174c6fb00b74b9777defa68e5e1241ace35bbf24e4a92d6dd275870155b18e4438d732e5807896
+  metadata.gz: a61cc82094a231be929bd06358d64566364ac587741af0de0764b5d1db4279dfdcb14e7b4244fc249f447d70ca5f75a58dcbc8d72c244fbd02edde1b75f000c2
+  data.tar.gz: b6e7abe601616b9fdd57e51acb9f4c24d34d7571c85c51e1a655c2d89cdc70849f62ef7bfeb8513078439a84ee6db6e5f84a9434cc75d97a5bb8d2008be1f757

data/CHANGELOG.md

@@ -0,0 +1,86 @@
+# Changelog
+
+2.0.0
+-----
+
+Breaking:
+
+* Requires Rails 8.0+ and Ruby 3.3+.
+* `write` is no longer aliased onto `ActionView::OutputBuffer` and
+  `ActionView::StreamingBuffer`. Installing the gem used to add a non-escaping
+  append to every buffer in the application; the encoders now write into
+  `ActionView::TurboBuffer` instead. Code outside TurboStreamer that called
+  `output_buffer.write` was relying on that alias and will need `safe_concat`.
+* `ActionView::JSONStreamingBuffer` is now `ActionView::StreamingTurboBuffer`.
+
+* Optimize internal `extract!` calls to save on memory allocation [PR #25](https://github.com/malomalo/turbostreamer/pull/25)
+* Add `frozen_string_literal` magic comments
+* Remove some old Rails code
+* Add Rails 8.0 & 8.1 to CI; drop support for Rails < 8.0 and Ruby < 3.3
+* Package `LICENSE` and `CHANGELOG.md` with the gem, and fix `spec.files` dropping
+  everything but `README.md` when the gem is built on a shell without brace expansion
+* Layouts now work. A `.json.streamer` layout places the template with
+  `json.yield!`. Layout and template share one builder, so the template writes
+  into the same stream rather than being buffered and spliced. The layout
+  renders first and yields to the template, the reverse of an ERB layout.
+  Applies both to `render stream: true` and to ordinary rendering; previously a
+  streamed layout was resolved and then discarded, and an unstreamed one had no
+  way to place the template's JSON at all.
+* Fix streaming JSON raising `NoMethodError: undefined method 'instrument'`.
+  `AbstractRenderer#instrument` was removed in Rails 6.1, so every streamed
+  render failed -- silently, since `Body#each` rescues and substitutes an error
+  page. It now notifies `render_template.action_view` directly.
+* Stop aliasing `write` onto `ActionView::OutputBuffer` and
+  `ActionView::StreamingBuffer`. The encoders now stream into `TurboStreamer::Buffer`,
+  which wraps the ActionView buffer instead of patching it, so the non-escaping
+  `write` is no longer added to every buffer in the application.
+  `ActionView::JSONStreamingBuffer` moves to `TurboStreamer::StreamingBuffer`.
+
+1.11.0 - 2024-04-29
+-----
+* Fix timestamp precision for Rails [PR #24](https://github.com/malomalo/turbostreamer/pull/24)
+* Fix CI yajl archive download URL [PR #22](https://github.com/malomalo/turbostreamer/pull/22)
+
+1.10.0
+-----
+* Fixed Rails 6.1 & Ruby 3.0 Compatibility
+
+1.9.0
+-----
+* Fixed deprecation of using `Proc.new` to capture block; replaced with `&block`
+
+1.8.0
+-----
+* Make the StreamingRenderer Rails 6 compatible [PR #15](https://github.com/malomalo/turbostreamer/issues/15)
+* Update gemspec to require Ruby 2.5+ [PR #14](https://github.com/malomalo/turbostreamer/issues/14)
+
+1.7.0
+-----
+* Add the ability to set default options for encoders
+* Allow setting the `buffer_size` on the OJ encode
+* Reduce find_template calls [PR #11](https://github.com/malomalo/turbostreamer/pull/1)
+* Don't require a layout to stream template in Rails
+
+1.5.0
+-----
+* Add Rails 6.0 support
+* Drop Rails 4.2 support
+
+1.4.0
+-----
+* Replace deprecated fragment_cache_key for Rails 5.2 support
+
+1.3.0
+-----
+* Bump version and update bundler
+
+1.2.0
+-----
+* Add `TurboStreamer#merge!` to merge a hash or array into the current json stream.
+
+1.1.0
+-----
+* Add `Oj` as an encoder option
+* Add ability to pass encoder as an option to `TurboStreamer#new` (symbol or class)
+* Ability to set default encoder for mime type with `TurboStreamer#set_default_encoder`
+* Add some performance test

data/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2011-2017 David Heinemeier Hansson, 37signals
+Copyright (c) 2017 Jonathan Bracy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -1,17 +1,23 @@
 # TurboStreamer
 
-TurboStreamer gives you a simple DSL for generating JSON that beats massaging giant
-hash structures. This is particularly helpful when the generation process is
-fraught with conditionals and loops.
+[![GitHub Build Status](https://img.shields.io/github/workflow/status/malomalo/turbostreamer/CI?style=flat-square)](https://github.com/malomalo/turbostreamer/actions?query=workflow%3ACI)
 
+[![Gem Version](http://img.shields.io/gem/v/turbostreamer.svg?style=flat-square)](http://badge.fury.io/rb/turbostreamer)
+[![License](https://img.shields.io/github/license/malomalo/turbostreamer.svg?style=flat-square)](http://badge.fury.io/rb/turbostreamer)
+
+TurboStreamer gives you a simple DSL like jBuilder for generating JSON that
+streams directly to a String or IO object.
 
 [Jbuilder](https://github.com/rails/jbuilder) builds a Hash as it renders the
 template and once complete converts the Hash to JSON. TurboStreamer on the other
 hand writes directly to the output as it is rendering the template. Because of
 this some of the magic cannot be done and requires a little more verboseness.
 
-Examples
---------
+Because no time is spent creating a hash caching is also fast. No time is spent
+marshaling and unmarshaling from the cache, instead the string is cached and
+directly inserted into the stream skipping any unmarshaling.
+
+## Examples
 
 ``` ruby
 # app/views/message/show.json.streamer
@@ -31,7 +37,7 @@ json.object! do
   if current_user.admin?
     json.visitors calculate_visitors(@message)
   end
-  
+
   json.tags do
     json.array! do
       @message.tags.each { |tag| json.child! tag }
@@ -66,7 +72,7 @@ This will build the following structure:
   "visitors": 15,
 
   "tags": ['public'],
-  
+
   "comments": [
     { "content": "Hello everyone!", "created_at": "2011-10-29T20:45:28-05:00" },
     { "content": "To you my good sir!", "created_at": "2011-10-29T20:47:28-05:00" }
@@ -151,7 +157,7 @@ json.object! do
       json.url url_for(@message.creator, format: :json)
     end
   end
-  
+
   if current_user.admin?
     json.visitors calculate_visitors(@message)
   end
@@ -185,6 +191,85 @@ json.partial! partial: 'posts/post', collection: @posts, as: :post
 json.comments @post.comments, partial: 'comment/comment', as: :comment
 ```
 
+### Layouts
+
+A `.json.streamer` layout can wrap the template. Call `json.yield!` where the
+template's JSON should go:
+
+```ruby
+# app/views/layouts/application.json.streamer
+json.object! do
+  json.meta do
+    json.object! { json.version 1 }
+  end
+  json.key! :data
+  json.yield!
+end
+
+# app/views/posts/index.json.streamer
+json.array! @posts, :id, :title
+
+# => { "meta": { "version": 1 }, "data": [ { "id": 1, "title": "..." } ] }
+```
+
+The layout and the template share one builder, so the template writes straight
+into the same stream at the point it is yielded — nothing is buffered into a
+string and spliced back in. This works whether or not the response is streamed
+with `render stream: true`; when it is, a large response still arrives in
+chunks.
+
+`json.yield!` goes anywhere a value goes, including inside an array:
+
+```ruby
+json.array! do
+  json.child! { json.object! { json.first true } }
+  json.child! { json.yield! }
+end
+```
+
+A layout that only wraps is just the one call:
+
+```ruby
+json.yield!
+```
+
+A layout may yield more than once, and one that never yields renders without the
+template — both as an ERB layout does. The difference is that ERB replays a
+buffered string, whereas each `json.yield!` renders the template again, so
+anything it does happens again too:
+
+```ruby
+json.array! do
+  json.child! { json.yield! }
+  json.child! { json.yield! }   # the template is rendered a second time
+end
+```
+
+#### Why `json.yield!` and not `yield`
+
+`yield` is a Ruby keyword that returns a value, and the template's JSON is never
+a value here — it is written into the stream at the position the layout has
+reached. Placing it therefore has to go through the builder, and `json.yield!`
+matches the rest of the DSL, where the methods that write something end in `!`:
+`object!`, `array!`, `child!`, `partial!`, `merge!`, `cache!`. A bare `yield` in
+a `.json.streamer` layout raises `LocalJumpError` naming `json.yield!`.
+
+#### Layouts are not ERB layouts
+
+Two differences worth knowing:
+
+* **The layout renders first**, and `json.yield!` renders the template it wraps.
+  An ERB layout is the other way around: the template is rendered up front and
+  the layout concatenates the resulting string. Reversing it is what lets the
+  template write into the layout's builder instead of being encoded to a string
+  and spliced back in — which neither encoder can do into a keyed slot.
+* **There is one yield, and it has no name.** `content_for` / `provide` have no
+  analogue, and a layout cannot ask for content the template defines later. In
+  ERB that works because the layout runs in a Fiber and suspends until the
+  template provides the key; here the layout calls the template directly, so
+  there is nothing to suspend. A JSON document's shape is positional, so one
+  yield in one place is generally what you want.
+
 You can explicitly make TurboStreamer object return null if you want:
 
 ``` ruby

data/lib/action_view/streaming_turbo_buffer.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+# Hands each chunk to a block as it is encoded rather than accumulating it,
+# so StreamingTemplateRenderer can push straight to the client instead of
+# buffering the whole response.
+#
+# Like TurboBuffer, it subclasses its ActionView counterpart and takes over
+# the block that one was writing to, so the inherited methods stay consistent
+# with what has been written.
+class ActionView::StreamingTurboBuffer < ActionView::StreamingBuffer
+
+  def initialize(block)
+    @block = block
+  end
+
+  # The encoders -- and the C extensions behind them, Oj::StreamWriter and
+  # Wankel::StreamEncoder -- write their output with the IO-style `write`,
+  # which ActionView's buffers do not have.
+  def write(value)
+    string = value.to_s
+    @block.call(string)
+    string.bytesize
+  end
+
+end

data/lib/action_view/streaming_turbo_template_renderer.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+# Prepended to StreamingTemplateRenderer by the railtie.
+#
+# ActionView's own streaming renders the *layout* and buffers the template
+# into a string for it, which would defeat the point here -- the whole JSON
+# document would be built in memory before a byte was written. So streamer
+# templates take their own path, writing to the client as the encoder
+# produces bytes.
+module ActionView::StreamingTurboTemplateRenderer
+
+  def render_template(view, template, layout_name = nil, locals = {})
+    return super unless template.handler == TurboStreamer::Handler
+
+    locals ||= {}
+    layout = layout_name && find_layout(layout_name, locals.keys, [formats.first])
+    log_skipped_layout(layout_name) if layout_name && layout.nil?
+
+    ActionView::StreamingTemplateRenderer::Body.new do |buffer|
+      delayed_render_json(buffer, template, layout, view, locals)
+    end
+  end
+
+  private
+
+    def delayed_render_json(buffer, template, layout, view, locals)
+      output = ActionView::StreamingTurboBuffer.new(buffer)
+
+      ActiveSupport::Notifications.instrument(
+        "render_template.action_view",
+        identifier: template.identifier,
+        layout: layout && layout.virtual_path,
+        locals: locals
+      ) do
+        if layout
+          render_json_layout(view, layout, locals, output) do |json|
+            template.render(view, locals.merge(json: json), output)
+          end
+        else
+          template.render(view, locals, output)
+        end
+      end
+    end
+
+end

data/lib/action_view/turbo_buffer.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+  # The encoders -- and the C extensions behind them, Oj::StreamWriter and
+# Wankel::StreamEncoder -- write their output with the IO-style `write`.
+# ActionView's buffers only expose `<<` / `concat` / `safe_concat`, so they
+# have to be adapted before an encoder can stream into one.
+#
+# Subclassing OutputBuffer means this *is* one: Template#render finishes with
+# `result.is_a?(OutputBuffer) ? result.to_s : result`, so target! can hand it
+# back directly instead of unwrapping. It takes over the String the given
+# buffer was writing to rather than keeping one of its own, so the inherited
+# methods all report on the same content and there is nothing to delegate.
+#
+# See StreamingTurboBuffer for the streaming counterpart.
+class ActionView::TurboBuffer < ActionView::OutputBuffer
+
+  # instance_of? rather than is_a?, since this class is itself an
+  # OutputBuffer and must not be wrapped a second time.
+  def self.wrap(buffer)
+    if buffer.instance_of?(ActionView::OutputBuffer)
+      ActionView::TurboBuffer::new(buffer.raw_buffer)
+    elsif buffer.instance_of?(ActionView::StreamingBuffer)
+      ActionView::StreamingTurboBuffer.new(buffer.block)
+    elsif buffer.respond_to?(:write)
+      buffer
+    else
+      raise ArgumentError, "#{buffer.class} has no #write, so an encoder cannot stream into it"
+    end
+  end
+
+  def initialize(raw_buffer)
+    @raw_buffer = raw_buffer
+  end
+
+  # Whatever reaches here is already encoded JSON, so it is appended verbatim
+  # instead of being HTML-escaped -- the same thing the old alias did by
+  # pointing `write` at `safe_concat`.
+  def write(value)
+    string = value.to_s
+    @raw_buffer << string
+    string.bytesize
+  end
+
+end

data/lib/action_view/turbo_template_renderer.rb

@@ -0,0 +1,83 @@
+# frozen_string_literal: true
+
+# Prepended to TemplateRenderer by the railtie.
+module ActionView::TurboTemplateRenderer
+
+  private
+
+    # ActionView renders the template into a string first and then hands that
+    # string to the layout. That works for ERB, where the layout concatenates
+    # text.
+    #
+    # For streamer templates the order is reversed. The layout renders first
+    # and its `yield` renders the template into the layout's own
+    # builder, which is what the streaming renderer does too.
+    def render_template(view, template, layout_name, locals)
+      return super unless template.handler == TurboStreamer::Handler
+
+      # find_layout returns nil when the layout exists in another format but
+      # not this one -- an app with layouts/application.html.erb and no JSON
+      # layout. It only raises MissingTemplate when there is no layout by
+      # that name in any format.
+      layout = layout_name && find_layout(layout_name, locals.keys, [formats.first])
+      if layout.nil?
+        log_skipped_layout(layout_name) if layout_name
+        return super
+      end
+
+      body = ActiveSupport::Notifications.instrument(
+        "render_layout.action_view",
+        identifier: layout.identifier
+      ) do
+        render_json_layout(view, layout, locals) do |json|
+          ActiveSupport::Notifications.instrument(
+            "render_template.action_view",
+            identifier: template.identifier,
+            layout: layout.virtual_path,
+            locals: locals
+          ) do
+            template.render(view, locals.merge(json: json))
+          end
+        end
+      end
+
+      # render_json_layout hands back its buffer; the caller wants a String.
+      build_rendered_template(body.to_s, template)
+    end
+
+    # Renders a layout with a builder of our own, so `json.yield!` has something
+    # to place. Layout and template write to one encoder, so the yielded JSON
+    # lands where it is placed with its commas and nesting intact.
+    #
+    # `render_inner` renders the template the layout wraps, taking the builder
+    # to render into; the buffer, when given, is the streaming one both renders
+    # share. Shared with StreamingTurboTemplateRenderer, which reaches it
+    # through StreamingTemplateRenderer's superclass.
+    #
+    # The block exists only to give a bare `yield` a useful failure. Without one
+    # it would be `no block given (yield)`, which says nothing about layouts.
+    def render_json_layout(view, layout, locals, buffer = nil, &render_inner)
+      # ActionView::TurboBuffer in full: the compact `module
+      # ActionView::TurboTemplateRenderer` above does not nest ActionView
+      # lexically, so a bare TurboBuffer would not resolve.
+      json = TurboStreamer::Template.new(view, output_buffer: buffer || ActionView::TurboBuffer.new(String.new))
+      json.yield_content = render_inner
+
+      layout.render(view, locals.merge(json: json)) do |*|
+        raise ::LocalJumpError, '`yield` is not supported in a .json.streamer layout, use `json.yield!`'
+      end
+
+      json.target!
+    end
+
+    def log_skipped_layout(layout_name)
+      logger = ActionView::Base.logger
+      return unless logger
+
+      logger.debug do
+        "  Skipped layout #{layout_name} -- it does not exist for " \
+        "#{formats.first.inspect}; rendering without a layout"
+      end
+    end
+
+end

data/lib/turbostreamer/dependency_tracker.rb

@@ -1,61 +1,49 @@
-require 'turbostreamer'
-
-dependency_tracker = false
-
-begin
-  require 'action_view'
-  require 'action_view/dependency_tracker'
-  dependency_tracker = ::ActionView::DependencyTracker
-rescue LoadError
-  begin
-    require 'cache_digests'
-    dependency_tracker = ::CacheDigests::DependencyTracker
-  rescue LoadError
-  end
-end
+# frozen_string_literal: true
+
+# TODO: maybe remove reliance on ERBTracker ala: https://github.com/rails/jbuilder/commit/710979958aa012f4ff21800abf5bd50b717df0a6
+require 'action_view'
+require 'action_view/dependency_tracker'
+
+class TurboStreamer
+  module DependencyTrackerMethods
+    # Matches:
+    #   json.partial! "messages/message"
+    #   json.partial!('messages/message')
+    #
+    DIRECT_RENDERS = /
+      \w+\.partial!     # json.partial!
+      \(?\s*            # optional parenthesis
+      (['"])([^'"]+)\1  # quoted value
+    /x
+
+    # Matches:
+    #   json.partial! partial: "comments/comment"
+    #   json.comments @post.comments, partial: "comments/comment", as: :comment
+    #   json.array! @posts, partial: "posts/post", as: :post
+    #   = render partial: "account"
+    #
+    INDIRECT_RENDERS = /
+      (?::partial\s*=>|partial:)  # partial: or :partial =>
+      \s*                         # optional whitespace
+      (['"])([^'"]+)\1            # quoted value
+    /x
+
+    def dependencies
+      direct_dependencies + indirect_dependencies + explicit_dependencies
+    end
+
+    private
 
-if dependency_tracker
-  class TurboStreamer
-    module DependencyTrackerMethods
-      # Matches:
-      #   json.partial! "messages/message"
-      #   json.partial!('messages/message')
-      #
-      DIRECT_RENDERS = /
-        \w+\.partial!     # json.partial!
-        \(?\s*            # optional parenthesis
-        (['"])([^'"]+)\1  # quoted value
-      /x
-
-      # Matches:
-      #   json.partial! partial: "comments/comment"
-      #   json.comments @post.comments, partial: "comments/comment", as: :comment
-      #   json.array! @posts, partial: "posts/post", as: :post
-      #   = render partial: "account"
-      #
-      INDIRECT_RENDERS = /
-        (?::partial\s*=>|partial:)  # partial: or :partial =>
-        \s*                         # optional whitespace
-        (['"])([^'"]+)\1            # quoted value
-      /x
-
-      def dependencies
-        direct_dependencies + indirect_dependencies + explicit_dependencies
-      end
-
-      private
-
-      def direct_dependencies
-        source.scan(DIRECT_RENDERS).map(&:second)
-      end
-
-      def indirect_dependencies
-        source.scan(INDIRECT_RENDERS).map(&:second)
-      end
+    def direct_dependencies
+      source.scan(DIRECT_RENDERS).map(&:second)
     end
-  end
 
-  ::TurboStreamer::DependencyTracker = Class.new(dependency_tracker::ERBTracker)
-  ::TurboStreamer::DependencyTracker.send :include, ::TurboStreamer::DependencyTrackerMethods
-  dependency_tracker.register_tracker :streamer, ::TurboStreamer::DependencyTracker
+    def indirect_dependencies
+      source.scan(INDIRECT_RENDERS).map(&:second)
+    end
+  end
 end
+
+::TurboStreamer::DependencyTracker = Class.new(::ActionView::DependencyTracker::ERBTracker)
+::TurboStreamer::DependencyTracker.send :include, ::TurboStreamer::DependencyTrackerMethods
+::ActionView::DependencyTracker.register_tracker :streamer, ::TurboStreamer::DependencyTracker

data/lib/turbostreamer/encoders/oj.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "oj"
 
 class TurboStreamer

data/lib/turbostreamer/encoders/wankel.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require 'wankel'
 
 class TurboStreamer

data/lib/turbostreamer/errors.rb

@@ -1,11 +1,10 @@
-
+# frozen_string_literal: true
 
 class TurboStreamer
   module Errors
     class MergeError < ::StandardError
       def self.build(updates)
-        message = "Can't merge #{updates.inspect} which isn't Hash or Array"
-        new(message)
+        new("Can't merge #{updates.inspect} which isn't Hash or Array")
       end
     end
   end

data/lib/turbostreamer/handler.rb

@@ -1,7 +1,10 @@
+# frozen_string_literal: true
+
 require "turbostreamer"
 require "active_support/core_ext"
-# This module makes TurboStreamer work with Rails using the template handler API.
 
+# This module makes TurboStreamer work with Rails using the template handler
+# API.
 class TurboStreamer
   class Handler
     
@@ -12,14 +15,10 @@ class TurboStreamer
       true
     end
     
-    # TODO: setting source=nil is for rails 5.x compatability, once unsppored
-    # source can be a required param and
-    # `source = template.source if source.nil?` can be removed
-    def self.call(template, source=nil)
-      source = template.source if source.nil?
+    def self.call(template, source)
       # this juggling is required to keep line numbers right in the error
-      %{__already_defined = defined?(json); json||=TurboStreamer::Template.new(self, output_buffer: output_buffer || ActionView::OutputBuffer.new); #{source}
-        json.target! unless (__already_defined && __already_defined != "method")}
+      %{__parent_json = local_assigns[:json]; json = __parent_json || TurboStreamer::Template.new(self, output_buffer: ActionView::TurboBuffer.wrap(output_buffer)); #{source}
+        json.target! unless __parent_json}
     end
     
   end

data/lib/turbostreamer/key_formatter.rb

@@ -1,33 +1,31 @@
-require 'active_support/core_ext/array'
+# frozen_string_literal: true
 
-class TurboStreamer
-  class KeyFormatter
-    def initialize(*args)
-      @format = {}
-      @cache = {}
+class TurboStreamer::KeyFormatter
+  def initialize(*args)
+    @format = {}
+    @cache = {}
 
-      options = args.extract_options!
-      args.each do |name|
-        @format[name] = []
-      end
-      options.each do |name, paramaters|
-        @format[name] = paramaters
-      end
+    options = args.extract_options!
+    args.each do |name|
+      @format[name] = []
     end
-
-    def initialize_copy(original)
-      @cache = {}
+    options.each do |name, paramaters|
+      @format[name] = paramaters
     end
+  end
+
+  def initialize_copy(original)
+    @cache = {}
+  end
 
-    def format(key)
-      @cache[key] ||= @format.inject(key.to_s) do |result, args|
-        func, args = args
-        if ::Proc === func
-          func.call result, *args
-        else
-          result.send func, *args
-        end
+  def format(key)
+    @cache[key] ||= @format.inject(key.to_s) do |result, args|
+      func, args = args
+      if ::Proc === func
+        func.call result, *args
+      else
+        result.send func, *args
       end
     end
   end
-end
+end

data/lib/turbostreamer/railtie.rb

@@ -1,15 +1,46 @@
-require 'rails/railtie'
-require 'turbostreamer/handler'
-require 'turbostreamer/template'
+# frozen_string_literal: true
+
+# Rails. 8.1 requires this not sure why we have to load it and the railtie doesn't
+require "active_support/core_ext/module/delegation"
 
-require File.expand_path('../../../ext/actionview/buffer', __FILE__)
-require File.expand_path('../../../ext/actionview/streaming_template_renderer', __FILE__)
+require 'rails/railtie'
 
 class TurboStreamer
   class Railtie < ::Rails::Railtie
     initializer :turbostreamer do
       ActiveSupport.on_load :action_view do
+        # Require turbostreamer in here so it's only loaded if needed
+        require 'turbostreamer'
+        require 'action_view/turbo_buffer'
+        require 'action_view/streaming_turbo_buffer'
+        require 'action_view/turbo_template_renderer'
+        require 'action_view/streaming_turbo_template_renderer'
+
+        ActionView::TemplateRenderer.prepend(ActionView::TurboTemplateRenderer)
+        ActionView::StreamingTemplateRenderer.prepend(ActionView::StreamingTurboTemplateRenderer)
+
+        # Register Turbostreamer with Rails
         ActionView::Template.register_template_handler :streamer, TurboStreamer::Handler
+        
+        # Resolve the encoder once, here, the way an app resolves its cache
+        # store at boot. Left unset, default_encoder_for falls through to
+        # get_encoder on every render, and the `require` in there re-scans
+        # $LOAD_PATH each time.
+        encoder = TurboStreamer.default_encoder_for(:json)
+        TurboStreamer.set_default_encoder(:json, encoder)
+
+        # Oj's :rails mode is what escapes HTML entities the way
+        # ActiveSupport::JSON does, so a payload embedded in a <script> tag is
+        # safe. Merge rather than skip when options are already set: an app
+        # setting an unrelated one -- buffer_size, say -- would otherwise lose
+        # the mode along with the escaping, silently. Anything the app set
+        # explicitly still wins.
+        if encoder.name == 'TurboStreamer::OjEncoder'
+          TurboStreamer.set_default_encoder_options(
+            :oj, { mode: :rails }.merge(TurboStreamer.default_encoder_options(:oj))
+          )
+        end
+        
         require 'turbostreamer/dependency_tracker'
       end
     end

data/lib/turbostreamer/template.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require 'turbostreamer'
 
 class TurboStreamer::Template < TurboStreamer
@@ -12,6 +14,10 @@ class TurboStreamer::Template < TurboStreamer
     @context = context
     super(*args, &block)
   end
+
+  # The proc that renders the template this layout wraps, for json.yield! to
+  # place.
+  attr_accessor :yield_content
   
   def partial!(name_or_options, locals = {})
     if name_or_options.class.respond_to?(:model_name) && name_or_options.respond_to?(:to_partial_path)
@@ -35,6 +41,29 @@ class TurboStreamer::Template < TurboStreamer
     end
   end
 
+  # The same thing as a statement rather than a value, for a layout that would
+  # rather write the key itself:
+  #
+  #   json.key! :data
+  #   json.yield!
+  def yield!
+    content = @yield_content
+    # The same thing Ruby says for a `yield` with no block behind it, because
+    # that is what this is.
+    raise ::LocalJumpError, 'no block given (yield)' if content.nil?
+
+    # The template renders through this same builder, so it would otherwise
+    # still see the content and yield straight back into itself. Clear it while
+    # it renders -- a template has nothing to yield -- and put it back, so a
+    # layout can go on to yield again.
+    begin
+      @yield_content = nil
+      content.call(self)
+    ensure
+      @yield_content = content
+    end
+  end
+
   def array!(collection = BLANK, *attributes, &block)
     options = attributes.extract_options!
 
@@ -201,9 +230,6 @@ class TurboStreamer::Template < TurboStreamer
 
     if @context.respond_to?(:combined_fragment_cache_key)
       key = @context.combined_fragment_cache_key(key)
-    elsif @context.respond_to?(:fragment_cache_key)
-      # TODO: remove after droping rails 5.1 support
-      key = @context.fragment_cache_key(key)
     elsif ::Hash === key
       key = url_for(key).split('://', 2).last
     end
@@ -224,17 +250,21 @@ class TurboStreamer::Template < TurboStreamer
     end
   end
 
-  def _partial_options?(options)
-    ::Hash === options && options.key?(:as) && options.key?(:partial)
-  end
-
-  def _is_active_model?(object)
-    object.class.respond_to?(:model_name) && object.respond_to?(:to_partial_path)
-  end
-
+  # The base rule, plus the case only Rails has: a nil collection rendered
+  # through a partial -- `json.comments nil, partial: 'comment/comment', as:
+  # :comment` -- has to reach array! to come out as [] rather than being
+  # treated as a single object to extract from.
+  #
+  # Written out rather than calling super, because child! runs this on every
+  # element and the second dispatch showed up. It stays here rather than moving
+  # into TurboStreamer because partial! is a Template method: in a plain
+  # builder the `:as` clause has nothing to route to, and would only turn
+  # `json.foo nil, as: :x` from a TypeError into [].
   def _eachable_arguments?(value, *args)
-    return true if super
+    return true if value.respond_to?(:each) && !value.is_a?(Hash)
+
     options = args.last
     ::Hash === options && options.key?(:as)
   end
+
 end

data/lib/turbostreamer/version.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class TurboStreamer
-  VERSION = '1.10.0'
+  VERSION = '2.0.0'
 end

data/lib/turbostreamer.rb

@@ -1,9 +1,14 @@
+# frozen_string_literal: true
+
 require 'stringio'
-require 'turbostreamer/key_formatter'
-require 'turbostreamer/errors'
 
 class TurboStreamer
 
+  autoload :Handler, 'turbostreamer/handler'
+  autoload :Template, 'turbostreamer/template'
+  autoload :KeyFormatter, 'turbostreamer/key_formatter'
+  autoload :Errors, 'turbostreamer/errors'
+
   BLANK = ::Object.new
 
   ENCODERS = {
@@ -29,7 +34,7 @@ class TurboStreamer
       @encoder_options = @@encoder_options[options[:encoder]]
     elsif options[:encoder].nil?
       @encoder = TurboStreamer.default_encoder_for(options[:mime] || :json)
-      if encoder_symbol = ENCODERS[options[:mime] || :json].find { |k, v| v == @encoder.name.delete_prefix('TurboStreamer::').delete_suffix('Encoder') }&.first
+      if encoder_symbol = TurboStreamer.encoder_symbol_for(options[:mime] || :json, @encoder)
         @encoder_options = @@encoder_options[encoder_symbol]
       else
         @encoder_options = {}
@@ -94,11 +99,7 @@ class TurboStreamer
   #
   #   { "name": David", "age": 32 }, { "name": Jamie", "age": 31 }
   def extract!(object, *attributes)
-    if ::Hash === object
-      attributes.each{ |key| _set_value key, object.fetch(key) }
-    else
-      attributes.each{ |key| _set_value key, object.public_send(key) }
-    end
+    _extract(object, attributes)
   end
 
   # Turns the current element into an array and iterates over the passed
@@ -164,7 +165,7 @@ class TurboStreamer
     else
       # json.author @post.creator, :name, :email_address
       # { "author": { "name": "David", "email_address": "david@thinking.com" } }
-      object!{ extract!(value, *args) }
+      object!{ _extract(value, args) }
     end
   end
 
@@ -228,22 +229,57 @@ class TurboStreamer
     @@key_formatter = formatter
   end
 
-  def self.set_default_encoder(mime, encoder, default_options={})
-    if encoder.is_a?(Symbol)
-      @@default_encoders[mime] = get_encoder(mime, encoder)
+  # The symbol an encoder is known by, given either the symbol itself or the
+  # class. Options are always keyed by symbol, since that is what a builder has
+  # to look them up with.
+  def self.encoder_symbol_for(mime, encoder)
+    return encoder if encoder.is_a?(Symbol)
+
+    ENCODERS[mime]&.key(encoder.name.delete_prefix('TurboStreamer::').delete_suffix('Encoder'))
+  end
+
+  def self.set_default_encoder(mime, encoder, default_options=nil)
+    @@default_encoders[mime] = if encoder.is_a?(Symbol)
+      get_encoder(mime, encoder)
     else
-      @@default_encoders[mime] = encoder
+      encoder
+    end
+
+    # Keyed by symbol even when handed a class -- keying by the class stored
+    # options a builder would never find, dropping them silently.
+    if default_options
+      @@encoder_options[encoder_symbol_for(mime, encoder) || encoder] = default_options
     end
-    @@encoder_options[encoder] = default_options
   end
-  
+
   def self.set_default_encoder_options(encoder, options)
     @@encoder_options[encoder] = options
   end
 
+  # Does not use [] -- @@encoder_options defaults new keys to {} *and assigns
+  # them*, so reading through it would make has_default_encoder_options? true.
+  def self.default_encoder_options(encoder)
+    @@encoder_options.fetch(encoder, {})
+  end
+
+  def self.has_default_encoder_options?(encoder)
+    @@encoder_options.has_key?(encoder)
+  end
+
+  # Memoized because `require` with a relative feature name re-scans $LOAD_PATH
+  # on every call, stat-ing each entry, even once the file is loaded. This runs
+  # per render whenever no default encoder has been set -- which is the case in
+  # a Rails app, since the railtie only sets encoder options -- and the stat
+  # storm dwarfed the encoding itself.
+  #
+  # A [mime, key] pair always names the same class, so caching it does not
+  # affect set_default_encoder: default_encoder_for still reads @@default_encoders.
   def self.get_encoder(mime, key)
-    require "turbostreamer/encoders/#{key}"
-    Object.const_get("TurboStreamer::#{ENCODERS[mime][key]}Encoder")
+    @encoders ||= {}
+    @encoders[[mime, key]] ||= begin
+      require "turbostreamer/encoders/#{key}"
+      Object.const_get("TurboStreamer::#{ENCODERS[mime][key]}Encoder")
+    end
   end
 
   def self.default_encoder_for(mime)
@@ -317,7 +353,7 @@ class TurboStreamer
     elsif _eachable_arguments?(value, *args)
       _scope{ array!(value, *args) }
     else
-      object!{ extract!(value, *args) }
+      object!{ _extract(value, args) }
     end
 
   end
@@ -326,15 +362,25 @@ class TurboStreamer
   def target!
     @encoder.flush
 
-    if @encoder.output.is_a?(::StringIO)
-      @encoder.output.string
+    output = @encoder.output
+
+    if output.is_a?(::StringIO)
+      output.string
     else
-      @encoder.output
+      output
     end
   end
 
   private
 
+  def _extract(object, attributes)
+    if ::Hash === object
+      attributes.each{ |key| _set_value key, object.fetch(key) }
+    else
+      attributes.each{ |key| _set_value key, object.public_send(key) }
+    end
+  end
+
   def _write(key, value)
     @encoder.key(_key(key))
     @encoder.value(value)

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: turbostreamer
 version: !ruby/object:Gem::Version
-  version: 1.10.0
+  version: 2.0.0
 platform: ruby
 authors:
 - Jon Bracy
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-09-08 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -16,14 +15,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 5.0.0
+        version: 8.0.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 5.0.0
+        version: 8.0.0
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -53,7 +52,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: oj
+  name: minitest-reporters
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -67,7 +66,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: bundler
+  name: oj
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -109,7 +108,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: byebug
+  name: debug
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -151,49 +150,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: analyzer
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: jbuilder
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: rabl
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: multi_json
+  name: railties
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -206,7 +163,9 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description:
+description: |
+  TurboStreamer is a JBuilder-like DSL for building
+  JSON that streams directly to a string or IO
 email:
 - jonbracy@gmail.com
 executables: []
@@ -214,9 +173,13 @@ extensions: []
 extra_rdoc_files:
 - README.md
 files:
+- CHANGELOG.md
+- LICENSE
 - README.md
-- ext/actionview/buffer.rb
-- ext/actionview/streaming_template_renderer.rb
+- lib/action_view/streaming_turbo_buffer.rb
+- lib/action_view/streaming_turbo_template_renderer.rb
+- lib/action_view/turbo_buffer.rb
+- lib/action_view/turbo_template_renderer.rb
 - lib/turbostreamer.rb
 - lib/turbostreamer/dependency_tracker.rb
 - lib/turbostreamer/encoders/oj.rb
@@ -230,8 +193,11 @@ files:
 homepage: https://github.com/malomalo/turbostreamer
 licenses:
 - MIT
-metadata: {}
-post_install_message:
+metadata:
+  source_code_uri: https://github.com/malomalo/turbostreamer
+  bug_tracker_uri: https://github.com/malomalo/turbostreamer/issues
+  changelog_uri: https://github.com/malomalo/turbostreamer/blob/master/CHANGELOG.md
+  rubygems_mfa_required: 'true'
 rdoc_options:
 - "--main"
 - README.md
@@ -241,15 +207,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.5'
+      version: 3.3.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.22
-signing_key:
+rubygems_version: 4.0.11
 specification_version: 4
 summary: Stream JSON via a Builder-style DSL
 test_files: []

data/ext/actionview/buffer.rb

@@ -1,25 +0,0 @@
-module ActionView
-  class OutputBuffer
-    alias :write :safe_concat
-  end
-  
-  class StreamingBuffer #:nodoc:
-    alias :write :safe_concat
-  end
-  
-  class JSONStreamingBuffer #:nodoc:
-    def initialize(block)
-      @block = block
-    end
-
-    def <<(value)
-      @block.call(value.to_s)
-    end
-    alias :write  :<<
-    alias :concat  :<<
-    alias :append= :<<
-    alias :safe_concat :<<
-    alias :safe_append= :<<
-  end
-  
-end

data/ext/actionview/streaming_template_renderer.rb

@@ -1,41 +0,0 @@
-module ActionView
-  class StreamingTemplateRenderer < TemplateRenderer
-    
-    def render_template(view, template, layout_name = nil, locals = {}) #:nodoc:
-      template_supports_streaming = (layout_name && template.supports_streaming?) || template.handler == TurboStreamer::Handler
-      return [super.body] unless layout_name && template_supports_streaming
-
-      locals ||= {}
-      layout   = layout_name && find_layout(layout_name, locals.keys, [formats.first])
-
-      Body.new do |buffer|
-        if template.handler == TurboStreamer::Handler
-          delayed_render_json(buffer, template, layout, view, locals)
-        else
-          delayed_render(buffer, template, layout, view, locals)
-        end
-      end
-    end
-
-    private
-
-      def delayed_render_json(buffer, template, layout, view, locals)
-        # Wrap the given buffer in the StreamingBuffer and pass it to the
-        # underlying template handler. Now, every time something is concatenated
-        # to the buffer, it is not appended to an array, but streamed straight
-        # to the client.
-        output  = ActionView::JSONStreamingBuffer.new(buffer)
-        yielder = lambda { |*name| view._layout_for(*name) }
-      
-        instrument(:template, identifier: template.identifier, layout: layout.try(:virtual_path)) do
-          fiber = Fiber.new do
-            template.render(view, locals, output, &yielder)
-          end
-
-          fiber.resume
-        end
-
-      end
-
-  end
-end