liquid2 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 856722c8ce4f6b647fff6e6c5ce61661845f6f8a51f51a4fe6be13ac21fe0287
-  data.tar.gz: 8da99a1c9e659519eac46600e1e9609cd6e57dc9ac88ad8589b564fbf7861eb5
+  metadata.gz: 396a11c43cb73cebe443927f69b83210d9c1651d9168ad7412726f54470492a7
+  data.tar.gz: ed127fb805a3862d3bde1db35dd88a713b714af3f65c51ba58d0f85fd09052c3
 SHA512:
-  metadata.gz: 40a9180a57d81c6461c056de6bf38e10ab6a95bb177acb59ad43aa0a92a80f44c3070c490afc67f30a12035427098fd78d26915b4336e587a1cd25188d49223a
-  data.tar.gz: e2a572fde0fffb372f37fbe8ad02bb49acaf6d55e2c449f6c3502b49b07996843c4396bbde08135629195d39e10b305e8848b1f17aec75ad0690951dba41f8e9
+  metadata.gz: 5c72bdc10c87b94b3863826cd1e964be714de2277c1c4627480fe0a9f869de977ae9efeb55cbaec573a625e43a46a0758312c36e15697d97934b177c968c078f
+  data.tar.gz: f2c6ec9e5d3d6d0fb86ba2455c0ec8134c8896243c6d8fab859ee3fbbd2c1bb17f2aaf949d591edf96eaf998d5562f7290e1ab9893502764ac9498ddb33aba97

data/CHANGELOG.md

@@ -1,5 +1,19 @@
-## [Unreleased]
+## [0.2.0] - 25-05-12
 
-## [0.1.0] - 2025-03-18
+- Fixed error context info when raising `UndefinedError` from `StrictUndefined`.
+- Fixed parsing of compound expressions given as filter arguments.
+- Fixed sorting of string representations of floats with the `sort_numeric` filter.
+- Fixed the string representation of variable paths with bracket notation and nested paths.
+- Added `Template#docs`, which returns an array of `DocTag` instances used in a template.
+- Added implementations of the `{% macro %}` and `{% call %}` tags.
+- Added `Template#macros`, which returns arrays of `{% macro %}` and `{% call %}` tags used in a template.
+- Added template inheritance tags `{% extends %}` and `{% block %}`.
+- Added block scoped variables with the `{% with %}` tag.
+
+## [0.1.1] - 2025-05-01
+
+- Add `base64` dependency to gemspec.
+
+## [0.1.0] - 2025-05-01
 
 - Initial release

data/LICENSE_SHOPIFY.txt

@@ -1,3 +1,11 @@
+This project is not affiliated with Shopify, but we do reference
+Shopify/liquid frequently and have used code from Shopify/liquid.
+A copy of the Shopify/liquid license is included below.
+
+See https://github.com/Shopify/liquid.
+
+---
+
 Copyright (c) 2005, 2006 Tobias Luetke
 
 Permission is hereby granted, free of charge, to any person obtaining

data/README.md

@@ -4,6 +4,22 @@
 Liquid templates for Ruby, with some extra features.
 </p>
 
+<p align="center">
+  <a href="https://github.com/jg-rp/ruby-liquid2/blob/main/LICENSE.txt">
+    <img alt="GitHub License" src="https://img.shields.io/github/license/jg-rp/ruby-liquid2?style=flat-square">
+  </a>
+  <a href="https://github.com/jg-rp/ruby-liquid2/actions">
+    <img src="https://img.shields.io/github/actions/workflow/status/jg-rp/ruby-liquid2/main.yml?branch=main&label=tests&style=flat-square" alt="Tests">
+  </a>
+  <br>
+  <a href="https://rubygems.org/gems/liquid2">
+    <img alt="Gem Version" src="https://img.shields.io/gem/v/liquid2?style=flat-square">
+  </a>
+  <a href="https://github.com/jg-rp/ruby-liquid2">
+    <img alt="Static Badge" src="https://img.shields.io/badge/Ruby-3.1%20%7C%203.2%20%7C%203.3%20%7C%203.4-CC342D?style=flat-square">
+  </a>
+</p>
+
 ---
 
 **Table of Contents**
@@ -12,11 +28,28 @@ Liquid templates for Ruby, with some extra features.
 - [Example](#example)
 - [Links](#links)
 - [About](#about)
-- [Usage](#usage)
+- [API](#api)
+- [Development](#development)
 
 ## Install
 
-TODO
+Add `'liquid2'` to your Gemfile:
+
+```
+gem 'liquid2', '~> 0.2.0'
+```
+
+Or
+
+```
+gem install liquid2
+```
+
+Or
+
+```
+bundle add liquid2
+```
 
 ## Example
 
@@ -31,13 +64,13 @@ puts template.render("you" => "Liquid")  # Hello, Liquid!
 ## Links
 
 - Change log: https://github.com/jg-rp/ruby-liquid2/blob/main/CHANGELOG.md
-- RubyGems: TODO
+- RubyGems: https://rubygems.org/gems/liquid2
 - Source code: https://github.com/jg-rp/ruby-liquid2
 - Issue tracker: https://github.com/jg-rp/ruby-liquid2/issues
 
 ## About
 
-This project aims to be mostly compatible with [Shopify/liquid](https://github.com/Shopify/liquid), but with fewer quirks and some new features.
+This project's template syntax aims to be mostly compatible with [Shopify/liquid](https://github.com/Shopify/liquid), but with fewer quirks and some new features. The [Ruby API](#api) is quite different.
 
 For those already familiar with Liquid, here's a quick description of the features added in Liquid2. Also see [test/test_compliance.rb](https://github.com/jg-rp/ruby-liquid2/blob/main/test/test_compliance.rb) for a list of [golden tests](https://github.com/jg-rp/golden-liquid) that we skip.
 
@@ -45,7 +78,7 @@ For those already familiar with Liquid, here's a quick description of the featur
 
 #### "Proper" string literal parsing
 
-String literals can contain markup delimiters (`{{`, `}}`, `{%`, `%}`, `{#` and `#}`) and c-like escape sequences without interfering with template parsing. Escape sequences follow JSON string syntax and semantics, with the addition of single quoted strings and the `\'` escape sequence.
+String literals can contain markup delimiters (`{{`, `}}`, `{%`, `%}`, `{#` and `#}`) without interfering with template parsing, and c-like escape sequences. Escape sequences follow JSON string syntax and semantics, with the addition of single quoted strings and the `\'` escape sequence.
 
 ```liquid
 {% assign x = "Hi \uD83D\uDE00!" %}
@@ -117,14 +150,14 @@ In this example, `{% if not user %}` is equivalent to `{% unless user %}`, howev
 
 #### Inline conditional and relational expressions
 
-In most expressions where you'd normally provide a literal (string, integer, float, true, false, nil/null) or variable name/path (foo.bar[0]), you can now use an inline conditional or relational expression.
+In most expressions where you'd normally provide a literal (string, integer, float, `true`, `false`, `nil`/`null`) or variable name/path (`foo.bar[0]`), you can now use an inline conditional or relational expression.
 
 See [Shopify/liquid #1922](https://github.com/Shopify/liquid/pull/1922) and [jg-rp/liquid #175](https://github.com/jg-rp/liquid/pull/175).
 
 These two templates are equivalent.
 
 ```liquid
-{{ user.name || "guest" }}
+{{ user.name or "guest" }}
 ```
 
 ```liquid
@@ -152,7 +185,7 @@ Many built-in filters that operate on arrays now accept lambda expression argume
 
 #### Dedicated comment syntax
 
-Comments surrounded by `{#` and `#}` are enabled by default. Additional `#`'s can be added to comment out blocks of markup that already contain comments, as long as hashes are balanced.
+Text surrounded by `{#` and `#}` are comments. Additional `#`'s can be added to comment out blocks of markup that already contain comments, as long as hashes are balanced.
 
 ```liquid2
 {## comment this out for now
@@ -190,17 +223,379 @@ Here we use `~` to remove the newline after the opening `for` tag, but preserve
 
 Integer and float literals can use scientific notation, like `1.2e3` or `1e-2`.
 
-## Usage
+#### Extra tags and filters
+
+Liquid2 includes implementations of `{% extends %}` and `{% block %}` for template inheritance, `{% with %}` for block scoped variables and `{% macro %}` and `{% call %}` for defining parameterized blocks.
+
+There's also built-in implementations of `sort_numeric` and `json` filters.
+
+## API
+
+### Liquid2.render
+
+`self.render: (String source, ?Hash[String, untyped]? data) -> String`
+
+Parse and render Liquid template _source_ using the default Liquid environment. If _data_ is given, hash keys will be available as template variables with their associated values.
+
+```ruby
+require "liquid2"
+
+puts Liquid2.render("Hello, {{ you }}!", "you" => "World")  # Hello, World!
+```
+
+This is a convenience method equivalent to `Liquid2::DEFAULT_ENVIRONMENT.parse(source).render(data)`.
+
+### Liquid2.parse
+
+`self.parse: (String source, ?globals: Hash[String, untyped]?) -> Template`
+
+Parse or "compile" Liquid template _source_ using the default Liquid environment. The resulting `Liquid2::Template` instance has a `render(data)` method, which can be called multiple times with different data.
+
+```ruby
+require "liquid2"
+
+template = Liquid2.parse("Hello, {{ you }}!")
+puts template.render("you" => "World") # Hello, World!
+puts template.render("you" => "Liquid") # Hello, Liquid!
+```
+
+If the _globals_ keyword argument is given, that data will be _pinned_ to the template and will be available as template variables every time you call `Template#render`. Pinned data will be merged with data passed to `Template#render`, with `render` arguments taking priority over pinned data if there's a name conflict.
+
+`Liquid2.render(source)` is a convenience method equivalent to `Liquid2::DEFAULT_ENVIRONMENT.parse(source)` or `Liquid2::Environment.new.parse(source)`.
+
+### Configure
+
+Both `Liquid2.parse` and `Liquid2.render` are convenience methods that use the default `Liquid2::Environment`. Often you'll want to configure an environment, then load and render template from that.
+
+```ruby
+require "liquid2"
+
+env = Liquid2::Environment.new(loader: Liquid2::CachingFileSystemLoader.new("templates/"))
+template = env.parse("Hello, {{ you }}!")
+template.render("you" => "World") # Hello, World!
+```
+
+Assuming you've configured a template loader, `Environment#get_template(name)`, the `{% render %}` tag and the `{% include %}` tag will use that `Liquid2::Loader` to find, read and parse templates. This example will look for templates in a relative folder on your file system called `templates`.
+
+```ruby
+require "liquid2"
+
+env = Liquid2::Environment.new(loader: Liquid2::CachingFileSystemLoader.new("templates/"))
+template = env.get_template("index.liquid")
+another_template = env.parse("{% render 'index.liquid' %}")
+# ...
+```
 
-TODO
+We'd expect a `Liquid2::LiquidTemplateNotFoundError` if `index.liquid` does not exist in the folder `templates/`.
+
+See [`environment.rb`](https://github.com/jg-rp/ruby-liquid2/blob/main/lib/liquid2/environment.rb) for all `Liquid2::Environment` options. Builtin template loaders are [`HashLoader`](https://github.com/jg-rp/ruby-liquid2/blob/main/lib/liquid2/loader.rb), [`FileSystemLoader`](https://github.com/jg-rp/ruby-liquid2/blob/main/lib/liquid2/loaders/file_system_loader.rb) and `CachingFileSystemLoader`. You are encouraged to implement your own template loaders to read template source text from a database or parse front matter, for example.
+
+#### Tags and filters
+
+All builtin tags and filters are registered with a new `Liquid2::Environment` by default. You can register or remove tags and/or filters using `Environment#register_filter`, `Environment#delete_filter`, `Environment#register_tag` and `Environment#delete_tag`, or override `Environment#setup_tags_and_filters` in an `Environment` subclass.
+
+```ruby
+require "liquid2"
+
+class MyEnv < Liquid2::Environment
+  def setup_tags_and_filters
+    super
+    delete_filter("base64_decode")
+    delete_filter("base64_encode")
+    delete_filter("base64_url_safe_decode")
+    delete_filter("base64_url_safe_encode")
+    register_tag("with", WithTag)
+  end
+end
+
+env = MyEnvironment.new
+# ...
+```
+
+See [`environment.rb`](https://github.com/jg-rp/ruby-liquid2/blob/main/lib/liquid2/environment.rb) for a list of builtin tags and filters, [`lib/liquid2/filters`](https://github.com/jg-rp/ruby-liquid2/tree/main/lib/liquid2/filters) for example filter implementations, and [`lib/liquid/nodes/tags`](https://github.com/jg-rp/ruby-liquid2/tree/main/lib/liquid2/nodes/tags) for example tag implementations.
+
+#### Undefined
+
+The default _undefined_ type is an instance of `Liquid2::Undefined`. It is silently ignored and, when rendered, produces an empty string. Passing `undefined: Liquid2::StrictUndefined` when initializing a `Liquid2::Environment` will cause all uses of an undefined template variable to raise a `Liquid2::UndefinedError`.
+
+```ruby
+require "liquid2"
+
+env = Liquid2::Environment.new(undefined: Liquid2::StrictUndefined)
+template = env.parse("Hello, {{ nosuchthing }}!")
+puts template.render
+#   -> "Hello, {{ nosuchthing }}!":1:10
+#   |
+# 1 | Hello, {{ nosuchthing }}!
+#   |           ^^^^^^^^^^^ "nosuchthing" is undefined
+```
+
+By default, instances of `Liquid2::StrictUndefined` are considered falsy when tested for truthiness, without raising an error.
+
+```ruby
+require "liquid2"
+
+env = Liquid2::Environment.new(undefined: Liquid2::StrictUndefined)
+template = env.parse("Hello, {{ nosuchthing or 'foo' }}!")
+puts template.render # Hello, foo!
+```
+
+Setting `falsy_undefined: false` when initializing a `Liquid2::Environment` will cause instances of `Liquid2::StrictUndefined` to raise an error when tested for truthiness.
+
+There's also `Liquid2::StrictDefaultUndefined`, which behaves like `StrictUndefined` but plays nicely with the `default` filter.
+
+### Static analysis
+
+Instances of `Liquid2::Template` include several methods for statically analyzing the template's syntax tree and reporting tag, filter and variable usage.
+
+`Template#variables` returns an array of variables used in the template. Notice that we get the _root segment_ only, excluding segments that make up a path to a variable.
+
+```ruby
+require "liquid2"
+
+source = <<~LIQUID
+  Hello, {{ you }}!
+  {% assign x = 'foo' | upcase %}
+
+  {% for ch in x %}
+      - {{ ch }}
+  {% endfor %}
+
+  Goodbye, {{ you.first_name | capitalize }} {{ you.last_name }}
+  Goodbye, {{ you.first_name }} {{ you.last_name }}
+LIQUID
+
+template = Liquid2.parse(source)
+p template.variables # ["you", "x", "ch"]
+```
+
+`Template#variable_paths` is similar, but includes all segments for each variable/path.
+
+```ruby
+# ... continued from above
+p template.variable_paths # ["you", "you.first_name", "you.last_name", "x", "ch"]
+```
+
+And `Template#variable_segments` does the same, but returns each variable/path as an array of segments instead of a string.
+
+```ruby
+# ... continued from above
+p template.variable_segments # [["you"], ["you", "first_name"], ["you", "last_name"], ["x"], ["ch"]]
+```
+
+Sometimes you'll only be interested in variables that are not in scope from previous tags (like `assign` and `capture`) or temporary block scope variables (like `forloop`). We call such variables "global" and provide `Template#global_variables`, `Template#global_variable_paths` and `Template#global_variable_segments`.
+
+```ruby
+# ... continued from above
+p template.global_variables # ["you"]
+```
+
+`Template#tag_names` and `Template#filter_names` return an array of tag and filter names used in the template.
+
+```ruby
+# ... continued from above
+p template.filter_names # ["upcase", "capitalize"]
+p template.tag_names # ["assign", "for"]
+```
+
+`Template#macros` returns arrays of `{% macro %}` and `{% call %}` tags found in the template.
+
+```ruby
+require "liquid2"
+
+source = <<~LIQUID
+  {% macro foo, you %}Hello, {{ you }}!{% endmacro -%}
+  {% call foo, 'World' %}
+  {% call bar, 'Liquid' %}
+LIQUID
+
+template = Liquid2.parse(source)
+macro_tags, call_tags = template.macros
+
+p macro_tags.map(&:macro_name).uniq # ["foo"]
+p call_tags.map(&:macro_name).uniq # ["foo", "bar"]
+```
+
+Finally there's `Template#comments` and `Template#docs`, which return instances of comments nodes and `DocTag` nodes, respectively. Each node has a `token` attribute, including a start index, and a `text` attribute, which is the comment or doc text.
+
+```ruby
+require "liquid2"
+
+source = <<~LIQUID
+  {% doc %}
+    Some doc comment
+  {% enddoc %}
+  {% assign x = 42 %}
+
+  {# note y could be nil #}
+  {{ x | plus: y or 7 }}
+LIQUID
+
+template = Liquid2.parse(source)
+p template.docs.map(&:text) # ["\n  Some doc comment\n"]
+p template.comments.map(&:text) # [" note y could be nil "]
+```
+
+### Drops
+
+Our "drop" interface defines the methods used by Liquid2 when non-primitive values appear as variables in templates. Primitive types are strings, integers, floats, `true`, `false`, `nil`/`null`, hashes, arrays and the special `blank` and `empty` objects. All other objects are accessed using the drop interface.
+
+#### To string
+
+`to_s` is called when outputting a drop.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  def to_s
+    "Hi!"
+  end
+end
+
+puts Liquid2.render("{{ thing }}", "thing" => MyDrop.new) # Hi!
+```
+
+#### Item fetching and method calling
+
+We use `[](key)` and `key?(key)` both for fetching items from a collection-like object and/or calling methods. Notice that `baz` is not called in this example.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  INVOCABLE = Set["foo", "bar"]
+
+  def [](key)
+    send(key) if INVOCABLE.member?(key)
+  end
+
+  def key?(key)
+    INVOCABLE.member?(key)
+  end
+
+  def foo = 42
+  def bar = "Hello!"
+  def baz = "not public"
+end
+
+puts Liquid2.render("{{ thing.foo }} {{ thing.baz }}", "thing" => MyDrop.new) # 42
+```
+
+#### Enumerating drops
+
+Any `Enumerable` will work with the `{% for %}` tag and filters that operate on arrays.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  include Enumerable
+
+  def each
+    yield "foo"
+    yield "bar"
+    yield 42
+  end
+end
+
+puts Liquid2.render("{% for x in y %}{{ x }},{% endfor %}", "y" => MyDrop.new) # foo,bar,42,
+```
+
+#### First, last and size
+
+If an object responds to `first`, `last` or `size`, those methods will be called by the `first`, `last` and `size` filters, and the special `.first`, `.last` or `.size` attributes.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  def first
+    42
+  end
+end
+
+puts Liquid2.render("{{ x.first }} {{ x | first }}", "x" => MyDrop.new) # 42 42
+```
+
+#### Lazy slicing
+
+If an object responds to `slice`, Liquid will call it with `start`, `length` and `reversed` arguments when evaluating it as a `for` loop target. `slice` takes priority over `is_a?(Enumerable)` and is intended for lazily loading items when slicing large collections.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  def initialize(*items)
+    @items = items
+  end
+
+  def slice(start, length, reversed)
+    # Pretend items are coming from a database or over a network.
+    array = @items.slice(start || 0, length || @items.length)
+    reversed ? array.reverse! : array
+  end
+end
+
+puts Liquid2.render("{% for x in y offset: 2, limit: 5 %}{{ x }},{% endfor %}", "y" => MyDrop.new) # 2,3,4,5,6,
+```
+
+#### Truthiness, comparisons and path segments
+
+If an object responds to `to_liquid(context)`, `to_liquid` will be called and the result used when testing drops for truthiness, comparing a drop to another value or when using the drop as a variable path segment.
+
+```ruby
+require "liquid2"
+
+class MyDrop
+  # @param context [RenderContext]
+  def to_liquid(_context)
+    "foo"
+  end
+end
+
+puts Liquid2.render("{% if x == 'foo' %}true{% else %}false{% endif %}", "x" => MyDrop.new) # true
+```
 
 ## Development
 
-TODO
+The [golden liquid](https://github.com/jg-rp/golden-liquid) test suite is included in this repository as a Git [submodule](https://git-scm.com/book/en/v2/Git-Tools-Submodules). Clone this project and initialize the submodule with something like:
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+```shell
+$ git clone git@github.com:jg-rp/ruby-liquid2.git
+$ cd ruby-liquid2
+$ git submodule update --init
+```
 
-## Contributing
+We use [Bundler](https://bundler.io/) and [Rake](https://ruby.github.io/rake/). Install development dependencies with:
+
+```
+bundle install
+```
+
+Run tests with:
+
+```
+bundle exec rake test
+```
+
+Lint with:
+
+```
+bundle exec rubocop
+```
+
+And type check with:
+
+```
+bundle exec steep
+```
+
+Run one of the benchmarks with:
+
+```
+bundle exec ruby performance/benchmark.rb
+```
 
 ### Profiling
 
@@ -212,6 +607,14 @@ Dump profile data with `bundle exec ruby performance/profile.rb`, then generate
 bundle exec stackprof --d3-flamegraph .stackprof-cpu-parse.dump > flamegraph-cpu-parse.html
 ```
 
+#### Memory profile
+
+Print memory usage to the terminal.
+
+```
+bundle exec ruby performance/memory_profile.rb
+```
+
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/lib/liquid2/context.rb

@@ -29,8 +29,8 @@ module Liquid2
   # Per render contextual information. A new RenderContext is created automatically
   # every time `Template#render` is called.
   class RenderContext
-    attr_reader :env, :template, :disabled_tags, :globals
-    attr_accessor :interrupts, :tag_namespace
+    attr_reader :env, :disabled_tags, :globals
+    attr_accessor :interrupts, :tag_namespace, :template
 
     BUILT_IN = BuiltIn.new
 
@@ -71,7 +71,6 @@ module Liquid2
 
       # Namespaces are searched from right to left. When a RenderContext is extended, the
       # temporary namespace is pushed to the end of this queue.
-      # TODO: exclude @globals if globals is empty
       @scope = ReadOnlyChainHash.new(@counters, BUILT_IN, @globals, @locals)
 
       # A namespace supporting stateful tags, such as `cycle` and `increment`.
@@ -171,10 +170,16 @@ module Liquid2
       template_ = @template
       @template = template if template
       @scope << namespace
-      yield
-    ensure
-      @template = template_
-      @scope.pop
+      begin
+        yield
+      rescue LiquidError => e
+        e.template_name = template.full_name if template && !e.template_name
+        e.source = template.source if template && !e.source
+        raise
+      ensure
+        @template = template_
+        @scope.pop
+      end
     end
 
     # Copy this render context and add _namespace_ to the new scope.
@@ -206,13 +211,17 @@ module Liquid2
                 ReadOnlyChainHash.new(@globals, namespace)
               end
 
-      self.class.new(template || @template,
-                     globals: scope,
-                     disabled_tags: disabled_tags,
-                     copy_depth: @copy_depth + 1,
-                     parent: self,
-                     loop_carry: loop_carry,
-                     local_namespace_carry: @assign_score)
+      context = self.class.new(template || @template,
+                               globals: scope,
+                               disabled_tags: disabled_tags,
+                               copy_depth: @copy_depth + 1,
+                               parent: self,
+                               loop_carry: loop_carry,
+                               local_namespace_carry: @assign_score)
+
+      # XXX: bit of a hack
+      context.tag_namespace[:extends] = @tag_namespace[:extends]
+      context
     end
 
     # Push a new namespace and forloop for the duration of a block.
@@ -222,10 +231,12 @@ module Liquid2
       raise_for_loop_limit(length: forloop.length)
       @loops << forloop
       @scope << namespace
-      yield
-    ensure
-      @scope.pop
-      @loops.pop
+      begin
+        yield
+      ensure
+        @scope.pop
+        @loops.pop
+      end
     end
 
     # Return the last ForLoop obj if one is available, or an instance of Undefined otherwise.