jektex 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7c8a9820fea56e754c67d39351bba20c1f21cfd91f1ac99a0543aca9f455d4cb
-  data.tar.gz: a0b680d4a6ee949e57b9b79e29054c81884d171cfb0923025ffa36bb646f1a30
+  metadata.gz: 63d33439743b4634d9758d3d466832179f0314a0e45679f414f11e13b04d27e7
+  data.tar.gz: 3b76e1ffe31724de5c1ce32df26fad3e4f6ad9fdd9df482b9ee019aa009ce5c2
 SHA512:
-  metadata.gz: 51f899741e534e393bdef3424e4760c7a7779a8868ef62d00b9f4b90b89bd4d670d985031a2acd14202ecc0245057fb569248687f677bf255a2eefb77f234321
-  data.tar.gz: 92fb84bf319058a0f6b3fa77283e54289e20438b1684b82374a31312de5b944ce1751cd356f09552ae9f1575f337af100c836ef675af8749ef59986ea6c24c6a
+  metadata.gz: 036e9f9683e6235763e47f9761dbc0b97e40b853cfa3059cc63acc6c7d994d3dbee2afeec415d0fb660cf97edd3911d3fb1acbfff3ab2a1c01a76296b64145dd
+  data.tar.gz: 2de454e9296dad57c9701ae361172442f2ed10e86eb5120e8b7ab4f06f75ba5f0b356ad708c9385bf701988bef3a8c6df5ad77a37293fc94361bdf53df5b657c

data/README.md

@@ -3,6 +3,7 @@
 # ![Jektex](https://blackblog.cz/assets/img/projects/jektex.svg)
 A Jekyll plugin for blazing-fast server-side cached LaTeX rendering, with support for macros.
 Enjoy the comfort of LaTeX and Markdown without cluttering your site with bloated JavaScript.
+This project is [endorsed by KaTeX.org](https://katex.org/docs/libs#jekyll).
 
 ## Features
 - Renders LaTeX formulas during Jekyll rendering
@@ -14,6 +15,8 @@ Enjoy the comfort of LaTeX and Markdown without cluttering your site with bloate
 - Is very easy to setup
 - Doesn't interfere with Jekyll workflow and project structure
 - Marks invalid expressions in document, printing its location during rendering
+- Leaves LaTeX inside code blocks and inline code untouched, so you can write about LaTeX
+- Renders formulas inside raw HTML blocks, which kramdown skips
 - Is highly configurable with sensible defaults
 - Makes sure that cache does not contain expression rendered with outdated configuration
 - Supports two major LaTeX notations
@@ -44,7 +47,25 @@ ea commodo consequat.
 _Why Jektex does not use conventional single `$` for inline formulas and double `$$` for
 display mode?  
 This is how [kramdown](https://kramdown.gettalong.org/) (Jekyll's markdown parser) works 
-so I decided to respect this convention. It makes this plugin more consistent and universal._
+so I decided to respect this convention. It makes this plugin more consistent and universal.
+See [this issue](https://github.com/gettalong/kramdown/issues/762) for more context._
+
+**Formulas inside raw HTML blocks**  
+Kramdown does not process markdown inside block-level HTML tags, so it leaves formulas
+like `<div>$$\beta$$</div>` unconverted. Jektex finds these leftover `$$` formulas after
+the conversion and renders them itself. A formula standing alone on its line renders
+in display mode, a formula inside text flow renders inline:
+```html
+<div>The inline formula $$e^{i\theta}$$ sits in text flow.
+
+$$ \left[ \frac{-\hbar^2}{2\mu}\nabla^2 + V(\mathbf{r},t)\right] \Psi(\mathbf{r},t) $$
+</div>
+```
+This applies only to markdown source files and never inside `pre`, `code`, `script`
+and similar tags. You can prevent rendering of a specific formula by escaping it
+as `\$$` or by putting it in a code span. If you prefer kramdown to process the
+content of an HTML block itself, give the tag the
+[`markdown="1"` attribute](https://kramdown.gettalong.org/syntax.html#html-blocks).
 
 
 ### LaTex math mode notation
@@ -73,11 +94,17 @@ ea commodo consequat.
 There is a build in macro for jektex logo. You can use it as `\jektex`.
 
 ### Config
-Jektex si highly configurable via your `_config.yml` file.
+Jektex is highly configurable via your `_config.yml` file.
+Unknown options and invalid values are reported during the build and fall back to their defaults.
 
 **Disabling cache**  
-You can disable caching with `disable_disk_cache = true` in `_config.yml`.
+You can disable caching with the `disable_disk_cache` option.
 Caching is enabled by default.
+This is Jekyll's own option, so unlike the options below it belongs
+at the top level of `_config.yml`, not under the `jektex` key:
+```yaml
+disable_disk_cache: true
+```
 You can find more information on [Jekyll's official website](https://jekyllrb.com/docs/configuration/options/).
 
 **Setting cache location**  
@@ -124,6 +151,15 @@ jektex:
 And yes, you have to escape the backlash (`\`) with another backlash.
 This is due to the [yaml specification](https://yaml.org/).
 
+You can define macros with parameters:
+```yaml
+jektex:
+  macros:
+    - ["\\vec", "\\mathbf{#1}"]
+    - ["\\addBar", "\\bar{#1}"]
+```
+This simulates behaviour of LaTeX `\newcommand`.
+
 **Silencing Jektex output**  
 Jektex periodically informs the user about rendered/cached equations.
 If this is not desired, you can set the `silent` option (`false` by default).
@@ -132,6 +168,33 @@ jektex:
   silent: true
 ```
 
+**KaTeX options**  
+You can pass any [KaTeX rendering option](https://katex.org/docs/options) to the renderer
+through the `katex_options` key. Write the keys exactly as the KaTeX documentation spells them:
+```yaml
+jektex:
+  katex_options:
+    trust: false
+    output: htmlAndMathml
+    strict: warn
+```
+
+The most useful options are:
+
+- `trust` toggles features KaTeX deems potentially unsafe (`false` by default), namely
+  `\url`, `\href`, `\includegraphics`, `\htmlClass`, `\htmlId`, `\htmlStyle` and `\htmlData`.
+- `output: html` halves the size of every rendered formula by omitting the invisible MathML
+  copy. Be aware that the MathML is what screen readers use.
+- `maxExpand` limits macro expansion and protects your build from runaway recursive macros.
+
+Jektex controls `displayMode`, `macros`, `throwOnError` and `globalGroup` itself,
+so these keys are ignored (define your macros with the `macros` option above).
+Values of the documented KaTeX options are checked: an invalid value is reported
+and KaTeX's own default is used instead. Option names jektex does not know
+(for example ones added by newer KaTeX versions) are passed through unchecked.
+Changing any KaTeX option invalidates the cache and the next build re-renders everything once.
+
+
 **Complete examples**  
 Recommended config:
 ```yaml
@@ -139,6 +202,8 @@ jektex:
   cache_dir: ".jektex-cache"
   ignore: ["*.xml"]
   silent: false
+  katex_options:
+    trust: false
   macros:
     - ["\\Q", "\\mathbb{Q}"]
     - ["\\C", "\\mathbb{C}"]
@@ -150,6 +215,7 @@ jektex:
   cache_dir: ".jekyll-cache"
   ignore: []
   silent: false
+  katex_options: {}
   macros: []
 ```
 
@@ -178,9 +244,9 @@ plugins:
 
 and don't forget to add `katex.min.css` to you HTML head:
 ```html
-<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.16.9/dist/katex.min.css" integrity="sha384-MlJdn/WNKDGXveldHDdyRP1R4CTHr3FeuDNfhsLPYrq2t0UBkUdK2jyTnXPEK1NQ" crossorigin="anonymous">
+<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex@0.17.0/dist/katex.min.css" integrity="sha384-vlBdW0r3AcZO/HboRPznQNowvexd3fY8qHOWkBi5q7KGgqJ+F48+DceybYmrVbmB" crossorigin="anonymous">
 ```
-It is much better practice to download the [**css** file](https://cdn.jsdelivr.net/npm/katex@0.16.9/dist/katex.min.css) and load it as an asset from your server directly.
+It is much better practice to download the [**css** file](https://cdn.jsdelivr.net/npm/katex@0.17.0/dist/katex.min.css) and load it as an asset from your server directly.
 You can find more information on [KaTeX's website](https://katex.org/docs/browser.html).
 
 ## Contributions and bug reports

data/lib/jektex/cache.rb

@@ -0,0 +1,107 @@
+require 'digest'
+require 'fileutils'
+
+module Jektex
+  # Disk cache of rendered expressions. The cache file additionally stores
+  # the macro table it was rendered with, so expressions using an edited
+  # macro can be invalidated individually, and a fingerprint of everything
+  # else that affects KaTeX output, so a KaTeX or configuration change
+  # discards the whole cache instead of serving stale HTML.
+  class Cache
+    FORMAT = 2
+
+    attr_reader :updated_global_macros
+
+    # names of macros whose definition differs between the two tables
+    def self.updated_macros_between(current_macros, cached_macros)
+      current = current_macros || Hash.new
+      cached = cached_macros || Hash.new
+      (current.keys | cached.keys).reject { |name| current[name] == cached[name] }
+    end
+
+    def initialize(config, reporter: nil)
+      @config = config
+      @reporter = reporter
+      @entries = Hash.new
+      @updated_global_macros = Array.new
+    end
+
+    def load
+      payload = read_payload
+      if payload
+        @entries = payload["entries"]
+        @updated_global_macros =
+          self.class.updated_macros_between(@config.global_macros, payload["macros"])
+      end
+      return self
+    end
+
+    def fetch(expression, display_mode)
+      return nil if @updated_global_macros.any? { |macro| expression.include?(macro) }
+      return @entries[key_for(expression, display_mode)]
+    end
+
+    def store(expression, display_mode, html)
+      @entries[key_for(expression, display_mode)] = html
+    end
+
+    def size
+      return @entries.size
+    end
+
+    def save
+      return if @config.disable_disk_cache
+      path = @config.path_to_cache_file
+      FileUtils.mkdir_p(File.dirname(path))
+      payload = { "format" => FORMAT,
+                  "fingerprint" => fingerprint,
+                  "macros" => @config.global_macros,
+                  "entries" => @entries }
+      # write to a temporary file first so an interrupted write
+      # can never leave a truncated cache behind
+      temporary_path = path + ".tmp"
+      File.open(temporary_path, "wb") { |file| Marshal.dump(payload, file) }
+      File.rename(temporary_path, path)
+    end
+
+    private
+
+    def read_payload
+      path = @config.path_to_cache_file
+      return nil unless File.exist?(path)
+      payload = File.open(path, "rb") { |file| Marshal.load(file) }
+      unless valid?(payload)
+        @reporter&.info("cache file is invalid and will be rebuilt")
+        return nil
+      end
+      unless payload["fingerprint"] == fingerprint
+        @reporter&.info("cache reset (configuration or KaTeX changed)")
+        return nil
+      end
+      return payload
+    rescue StandardError
+      @reporter&.info("cache file is invalid and will be rebuilt")
+      return nil
+    end
+
+    def valid?(payload)
+      payload.is_a?(Hash) &&
+        payload["format"] == FORMAT &&
+        payload["entries"].is_a?(Hash) &&
+        payload["macros"].is_a?(Hash)
+    end
+
+    def key_for(expression, display_mode)
+      Digest::SHA2.hexdigest(expression) + (display_mode ? ":display" : ":inline")
+    end
+
+    # everything that changes KaTeX output (except macros, which are diffed
+    # individually) must be part of this fingerprint; katex_options covers
+    # all pass-through options, jektex-side additions must be appended
+    def fingerprint
+      @fingerprint ||= [FORMAT,
+                        Digest::SHA2.file(@config.path_to_katex_js).hexdigest[0, 16],
+                        @config.katex_options.sort.inspect].join("|")
+    end
+  end
+end

data/lib/jektex/configuration.rb

@@ -0,0 +1,167 @@
+module Jektex
+  class Config
+    # KaTeX options jektex needs to control itself: displayMode is decided
+    # per expression by its delimiters, macros have their own config key,
+    # throwOnError drives the error reporting pipeline, and globalGroup
+    # would make rendering order-dependent, which breaks caching
+    RESERVED_KATEX_OPTIONS = ["displayMode", "macros", "throwOnError", "globalGroup"].freeze
+
+    KNOWN_OPTIONS = ["cache_dir", "ignore", "silent", "macros", "katex_options"].freeze
+
+    # value checks for the KaTeX options documented at katex.org/docs/options;
+    # keys not listed here are passed to KaTeX unchecked, so options added
+    # by future KaTeX versions keep working without a jektex update
+    KATEX_OPTION_VALIDATIONS = {
+      "output"           => ['"html", "mathml" or "htmlAndMathml"', '"htmlAndMathml"',
+                             ->(value) { ["html", "mathml", "htmlAndMathml"].include?(value) }],
+      "leqno"            => ["true or false", "false", ->(value) { [true, false].include?(value) }],
+      "fleqn"            => ["true or false", "false", ->(value) { [true, false].include?(value) }],
+      "errorColor"       => ["a color string", '"#cc0000"', ->(value) { value.is_a?(String) }],
+      "minRuleThickness" => ["a number", "the KaTeX default", ->(value) { value.is_a?(Numeric) }],
+      "colorIsTextColor" => ["true or false", "false", ->(value) { [true, false].include?(value) }],
+      "maxSize"          => ["a number", "unlimited", ->(value) { value.is_a?(Numeric) }],
+      "maxExpand"        => ["a number", "1000", ->(value) { value.is_a?(Numeric) }],
+      "strict"           => ['true, false, "ignore", "warn" or "error"', '"warn"',
+                             ->(value) { [true, false, "ignore", "warn", "error"].include?(value) }],
+      "trust"            => ["true or false", "false", ->(value) { [true, false].include?(value) }]
+    }.freeze
+
+    attr_reader :path_to_katex_js
+    attr_reader :disable_disk_cache
+    attr_reader :ignore
+    attr_reader :console_indent
+    attr_reader :silent
+    attr_reader :front_matter_tag
+    attr_reader :katex_options
+    attr_reader :global_macros
+    attr_reader :markdown_extensions
+    # problems found while reading the configuration; the plugin prints
+    # them at startup, so this class can stay free of console output
+    attr_reader :warnings
+
+    def initialize(config)
+      @path_to_katex_js = File.join(__dir__, "/katex.min.js")
+
+      @cache_dir = ".jekyll-cache"
+      @cache_file = "jektex-cache.marshal"
+      @disable_disk_cache = false
+
+      @ignore = Array.new
+      @console_indent = " " * 13
+      @silent = false
+
+      @front_matter_tag = "jektex"
+      @katex_options = Hash.new
+
+      @markdown_ext = "markdown,mkdown,mkdn,mkd,md"
+
+      @global_macros = Hash.new
+      @warnings = Array.new
+
+      if config.is_a?(Hash)
+        update_from_jekyll_config(config)
+      end
+      add_jektex_logo_macro
+      @ignore.append("#{@cache_dir}/*")
+      @markdown_extensions = @markdown_ext.to_s.split(",").map { |ext| ".#{ext.strip.downcase}" }
+    end
+
+    def path_to_cache_file
+      return File.join(@cache_dir, @cache_file)
+    end
+
+    def number_of_global_macros
+      return @global_macros.length - 1
+    end
+
+    private
+    def update_from_jekyll_config(config)
+      if config.key?("jektex")
+        jektex_config = config["jektex"]
+        if jektex_config.is_a?(Hash)
+          warn_about_unknown_options(jektex_config)
+          @cache_dir = checked_option(jektex_config, "cache_dir", @cache_dir, "a string") do |value|
+            value.is_a?(String)
+          end
+          @ignore = checked_option(jektex_config, "ignore", @ignore, "a list of file patterns") do |value|
+            value.is_a?(Array) && value.all? { |pattern| pattern.is_a?(String) }
+          end
+          @silent = checked_option(jektex_config, "silent", @silent, "true or false") do |value|
+            value == true || value == false
+          end
+          read_katex_options(jektex_config)
+          read_macros(jektex_config)
+        else
+          @warnings.append('the "jektex" configuration must be a mapping of options, ignoring it')
+        end
+      end
+
+      @disable_disk_cache = config["disable_disk_cache"] if config.has_key?("disable_disk_cache")
+      @markdown_ext = config["markdown_ext"] if config.has_key?("markdown_ext")
+    end
+
+    def warn_about_unknown_options(jektex_config)
+      jektex_config.each_key do |key|
+        next if KNOWN_OPTIONS.include?(key)
+        @warnings.append("unknown option \"#{key}\" " \
+                         "(known options: #{KNOWN_OPTIONS.join(", ")})")
+      end
+    end
+
+    # returns the configured value when the validation block accepts it,
+    # otherwise warns, names the default and returns it
+    def checked_option(jektex_config, name, default, expectation)
+      return default unless jektex_config.key?(name)
+      value = jektex_config[name]
+      return value if yield(value)
+      @warnings.append("option \"#{name}\" must be #{expectation}, " \
+                       "falling back to default: #{default.inspect}")
+      return default
+    end
+
+    def read_katex_options(jektex_config)
+      return unless jektex_config.key?("katex_options")
+      options = jektex_config["katex_options"]
+      unless options.is_a?(Hash)
+        @warnings.append('option "katex_options" must be a mapping of KaTeX options, ' \
+                         "falling back to default: {}")
+        return
+      end
+      options.each_key do |key|
+        if RESERVED_KATEX_OPTIONS.include?(key.to_s)
+          @warnings.append("KaTeX option \"#{key}\" is controlled by jektex and was ignored")
+        end
+      end
+      @katex_options = options.reject { |key, _value| RESERVED_KATEX_OPTIONS.include?(key.to_s) }
+      KATEX_OPTION_VALIDATIONS.each do |name, (expectation, default_description, valid)|
+        next unless @katex_options.key?(name)
+        next if valid.call(@katex_options[name])
+        @warnings.append("KaTeX option \"#{name}\" must be #{expectation}, " \
+                         "falling back to KaTeX default: #{default_description}")
+        @katex_options.delete(name)
+      end
+    end
+
+    def read_macros(jektex_config)
+      return unless jektex_config.has_key?("macros")
+      macro_list = jektex_config["macros"]
+      valid = macro_list.is_a?(Array) && macro_list.all? do |pair|
+        pair.is_a?(Array) && pair.size == 2 && pair.all? { |part| part.is_a?(String) }
+      end
+      unless valid
+        @warnings.append('option "macros" must be a list of [name, definition] pairs of strings, ' \
+                         "falling back to default: no macros")
+        return
+      end
+      for macro_definition in macro_list
+        @global_macros[macro_definition[0]] = macro_definition[1]
+      end
+    end
+
+    def add_jektex_logo_macro
+      @global_macros['\jektex'] =
+        '\text{\raisebox{-0.55ex}{J}\kern{-0.3ex}E\kern{-0.25ex}\raisebox{-0.5ex}{K}\kern{-0.7ex}}\TeX'
+    end
+
+  end
+end

data/lib/jektex/hooks.rb

@@ -0,0 +1,51 @@
+# Jekyll integration. Everything else in this gem is plain Ruby;
+# this file is the only one that touches Jekyll::Hooks.
+
+module Jektex
+  class << self
+    attr_accessor :config, :cache, :renderer, :page_processor, :reporter
+  end
+end
+
+# Fires once per process (for jekyll build and jekyll serve alike),
+# at the end of Site#initialize.
+Jekyll::Hooks.register :site, :after_init do |site|
+  Jektex.config    = Jektex::Config.new(site.config || Hash.new)
+  Jektex.reporter  = Jektex::Reporter.new(Jektex.config)
+  Jektex.config.warnings.each { |warning| Jektex.reporter.warn(warning) }
+  Jektex.cache     = Jektex::Cache.new(Jektex.config, reporter: Jektex.reporter).load
+  Jektex.renderer  = Jektex::Renderer.new(Jektex.config)
+  Jektex.page_processor = Jektex::PageProcessor.new(config: Jektex.config,
+                                           cache: Jektex.cache,
+                                           renderer: Jektex.renderer,
+                                           reporter: Jektex.reporter)
+  Jektex.reporter.macro_summary(Jektex.config.number_of_global_macros,
+                                Jektex.cache.updated_global_macros.size)
+end
+
+# LaTeX notation (\( \) and \[ \]) in raw content, before Liquid/kramdown:
+# expressions are protected behind inert tokens so markdown cannot touch them.
+Jekyll::Hooks.register [:pages, :documents], :pre_render do |page|
+  page.content = Jektex.page_processor.process_content(page)
+end
+
+# After conversion the HTML structure is known: tokens inside code markup
+# are restored to their source text, everything else is rendered — including
+# the \(..\)/\[..\] delimiters kramdown produces from its $$..$$ notation
+# and the $$..$$ kramdown leaves untouched inside raw HTML blocks.
+Jekyll::Hooks.register [:pages, :documents], :post_render do |page|
+  page.output = Jektex.page_processor.process_output(page)
+end
+
+# Fires once per rebuild in watch mode — and also once during
+# Site#initialize BEFORE after_init has run, hence the safe navigation.
+Jekyll::Hooks.register :site, :after_reset do |_site|
+  Jektex.page_processor&.reset_counters
+end
+
+Jekyll::Hooks.register :site, :post_write do |_site|
+  # print stats once more so the error log cannot overwrite them
+  Jektex.reporter.finish(Jektex.page_processor.rendered_count,
+                         Jektex.page_processor.cache_hit_count)
+  Jektex.cache.save
+end