pagefind 1.5.2.1 → 1.5.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 389746e86fa995ad022379eae8e96610fa5eec6457fbbb7973356cded5917d50
-  data.tar.gz: bbeb2782cd10317935d074c399d003eafcdd7804de828e781897ee0f556c9386
+  metadata.gz: 29fc9d08b459ad0bb8185f81b9b66fef23fc4e2616d1dc0472d4aa7de0754a2b
+  data.tar.gz: 89d3be5213c8b96fe48c32d6a6708344acedf2ed370db437c2ad3d988ca0fe90
 SHA512:
-  metadata.gz: 8159f4491ac8240015a080316e9a1ad4ec1e4754e635e961b3546d2e049c7f41d890aad5a9a95804835d28184e4513632faa6bce048bf6d4264790ebfdd0f6dd
-  data.tar.gz: 24a77f5f40ee2bdd73ff2d2eae036843c0ca4040084e6f2548e542caf8bd12d80bf53ba7dbcb31002c9de395ab4d852b474878ac40c0a8517e88e5c57536506f
+  metadata.gz: c7a84061ed37ee139b733df2794ff830e63240ee9ffe1e7084bca1c58c5cb92c65987d995d880e8d2eda6570237dc61356625305bd8e37aca685047c095e6e72
+  data.tar.gz: c70307c54f6fc2b519c276c34e604cf5a4d4b236c1929d3132d0d2892dc4243d09e354e24fc6c0741ed4e72860d46420dafd3b65cb50c949b33970a67356084d

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 # pagefind-ruby changelog
 
+## v1.5.2.2
+
+* Add `Pagefind::Index` and `Pagefind::Service` @bangseongbeom
+* Support `PAGEFIND_EXTENDED_BINARY_PATH` and `PAGEFIND_BINARY_PATH` @bangseongbeom
+
 ## v1.5.2.1
 
 * Ship the pagefind_extended release, as Pagefind's npm package does @bangseongbeom

data/README.md

@@ -4,9 +4,9 @@
 [![Ruby Code Style](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://github.com/standardrb/standard)
 [![Gem Version](https://badge.fury.io/rb/pagefind.svg)](https://badge.fury.io/rb/pagefind)
 
-A self-contained `pagefind` executable, wrapped up in a ruby gem. That's it. Nothing else.
+A self-contained `pagefind` executable, with an indexing API, wrapped up in a ruby gem. That's it. Nothing else.
 
-This gem is based on [tailwindcss-ruby](https://github.com/flavorjones/tailwindcss-ruby). Much of the packaging approach, code, and documentation was adapted from it.
+This gem is based on [tailwindcss-ruby](https://github.com/flavorjones/tailwindcss-ruby). Much of the packaging approach, code, and documentation was adapted from it. The indexing API was adapted from Pagefind's [Python](https://pagefind.app/docs/py-api/) and [NodeJS](https://pagefind.app/docs/node-api/) wrappers.
 
 
 ## Installation
@@ -37,7 +37,7 @@ gem install pagefind
 
 ### Using a local installation of `pagefind`
 
-If you are not able to use the vendored precompiled binaries (for example, if you're on an unsupported platform), you can use a [local installation](https://pagefind.app/docs/installation/) of the `pagefind_extended` or `pagefind` binary by setting an environment variable named `PAGEFIND_INSTALL_DIR` to the directory path containing the binary.
+If you are not able to use the vendored precompiled binaries (for example, if you're on an unsupported platform), you can use a [local installation](https://pagefind.app/docs/installation/) of the `pagefind_extended` or `pagefind` binary by setting an environment variable named `PAGEFIND_EXTENDED_BINARY_PATH` or `PAGEFIND_BINARY_PATH` to the path of the binary, or `PAGEFIND_INSTALL_DIR` to the directory path containing the binary.
 
 For example, if you've [built Pagefind from source](https://pagefind.app/docs/installation/#building-from-source) with `cargo install pagefind` so that the binary is found at `$HOME/.cargo/bin/pagefind`, then you should set your environment variable like so:
 
@@ -99,6 +99,289 @@ Options:
 ```
 
 
+## Indexing API
+
+This gem also provides an interface to the indexing binary as a Ruby library you can require.
+
+There are situations where using this library is beneficial:
+
+- Integrating Pagefind into an existing Ruby project, e.g. writing a plugin for a static site generator that can pass in-memory HTML files to Pagefind.
+  Pagefind can also return the search index in-memory, to be hosted via the dev mode alongside the files.
+- Users looking to index their site and augment that index with extra non-HTML pages can run a standard Pagefind crawl with [`add_directory`](#indexadd_directory) and augment it with [`add_custom_record`](#indexadd_custom_record).
+- Users looking to use Pagefind's engine for searching miscellaneous content such as PDFs or subtitles, where [`add_custom_record`](#indexadd_custom_record) can be used to build the entire index from scratch.
+
+### Example usage
+
+<!-- this example is copied verbatim from test/integration.rb -->
+
+``` ruby
+require "pagefind"
+
+html_content = <<~HTML
+  <html>
+    <body>
+      <main>
+        <h1>Example HTML</h1>
+        <p>This is an example HTML page.</p>
+      </main>
+    </body>
+  </html>
+HTML
+
+Pagefind::Index.open(root_selector: "main", logfile: "index.log", output_path: "./output", verbose: true) do |index|
+  new_file = index.add_html_file(
+    content: html_content,
+    url: "https://example.com",
+    source_path: "other/example.html"
+  )
+  new_record = index.add_custom_record(
+    url: "/elephants/",
+    content: "Some testing content regarding elephants",
+    language: "en",
+    meta: {title: "Elephants"}
+  )
+  new_dir = index.add_directory("./public")
+  pp new_file, new_record, new_dir
+
+  index.get_files.each do |file|
+    puts "#{file[:content].bytesize.to_s.rjust(10)}B #{file[:path]}"
+  end
+end
+```
+
+All methods are synchronous: each one blocks until the native Pagefind binary running in the background responds. Pagefind handles one request at a time, so threads sharing an index or a service take turns.
+
+Responses are returned as hashes with symbol keys.
+
+### Pagefind::Index
+
+`Pagefind::Index` manages a Pagefind index.
+
+`Pagefind::Index.open` yields the index to a block and returns the block's value.
+Entering the block starts a backing Pagefind service and creates an in-memory index in the backing service.
+Exiting the block writes the in-memory index to disk and then shuts down the backing Pagefind service.
+
+``` ruby
+Pagefind::Index.open do |index| # open the index
+  # update the index
+end
+# the index is closed here and files are written to disk.
+```
+
+Each method of `Pagefind::Index` that talks to the backing Pagefind service can raise `Pagefind::ServiceError`.
+If an exception is raised inside `Pagefind::Index.open`'s block, the block exits without writing the index files to disk.
+
+``` ruby
+Pagefind::Index.open do |index| # open the index
+  index.add_directory("./public")
+  raise "not today"
+end
+# the index closes without writing anything to disk
+```
+
+`Pagefind::Index.open` optionally takes keyword arguments that can apply parts of the [Pagefind CLI config](https://pagefind.app/docs/config-options/). The options available at this level are:
+
+``` ruby
+Pagefind::Index.open(
+  root_selector: "main",
+  exclude_selectors: ["nav"],
+  force_language: "en",
+  include_characters: "._",
+  verbose: true,
+  logfile: "index.log",
+  keep_index_url: true,
+  write_playground: true,
+  output_path: "./output"
+) do |index|
+  # ...
+end
+```
+
+See the relevant documentation for these configuration options in the [Configuring the Pagefind CLI](https://pagefind.app/docs/config-options/) documentation.
+
+### index.add_directory
+
+Indexes a directory from disk using the standard Pagefind indexing behaviour.
+This is equivalent to running the Pagefind binary with `--site <dir>`.
+
+``` ruby
+# Index all the HTML files in the public directory
+indexed_dir = index.add_directory("./public")
+page_count = indexed_dir[:page_count] # Integer
+```
+
+If the `path` provided is relative, it will be relative to the current working directory of your Ruby process. Both `String` and `Pathname` are accepted.
+
+``` ruby
+# Index files in a directory matching a given glob pattern.
+indexed_dir = index.add_directory("./public", glob: "**/*.{html}")
+```
+
+Optionally, a custom `glob` can be supplied which controls which files Pagefind will consume within the directory. The default is shown, and the `glob` option can be omitted entirely.
+See [Wax patterns documentation](https://github.com/olson-sean-k/wax#patterns) for more details.
+
+### index.add_html_file
+
+Adds a virtual HTML file to the Pagefind index. Useful for files that don't exist on disk, for example a static site generator that is serving files from memory.
+
+``` ruby
+html_content = <<~HTML
+  <html lang="en"><body>
+    <h1>A Full HTML Document</h1>
+    <p> ... </p>
+  </body></html>
+HTML
+
+# Index a file as if Pagefind was indexing from disk
+new_file = index.add_html_file(
+  content: html_content,
+  source_path: "other/example.html"
+)
+
+# Index HTML content, giving it a specific URL
+new_file = index.add_html_file(
+  content: html_content,
+  url: "https://example.com"
+)
+```
+
+The `source_path` should represent the path of this HTML file if it were to exist on disk. Pagefind will use this path to generate the URL. It should be relative, or absolute to a path within the current working directory.
+
+Instead of `source_path`, a `url` may be supplied to explicitly set the URL of this search result.
+
+The `content` should be the full HTML source, including the outer `<html> </html>` tags. This will be run through Pagefind's standard HTML indexing process, and should contain any required Pagefind attributes to control behaviour.
+
+If successful, a hash is returned containing metadata about the completed indexing.
+
+### index.add_custom_record
+
+Adds a direct record to the Pagefind index.
+Useful for adding non-HTML content to the search results.
+
+``` ruby
+custom_record = index.add_custom_record(
+  url: "/contact/",
+  content: "My raw content to be indexed for search. " \
+    "Will be lightly processed by Pagefind.",
+  language: "en",
+  meta: {
+    title: "Contact",
+    category: "Landing Page"
+  },
+  filters: {tags: ["landing", "company"]},
+  sort: {weight: "20"}
+)
+
+page_word_count = custom_record[:page_word_count] # Integer
+page_url = custom_record[:page_url] # String
+page_meta = custom_record[:page_meta] # Hash[Symbol, String]
+```
+
+The `url`, `content`, and `language` keyword arguments are all required. `language` should be an [ISO 639-1 code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes).
+
+`meta` is optional, and is strictly a flat hash of keys to string values.
+See the [Metadata documentation](https://pagefind.app/docs/metadata/) for semantics.
+
+`filters` is optional, and is strictly a flat hash of keys to arrays of string values.
+See the [Filters documentation](https://pagefind.app/docs/filtering/) for semantics.
+
+`sort` is optional, and is strictly a flat hash of keys to string values.
+See the [Sort documentation](https://pagefind.app/docs/sorts/) for semantics.
+*When Pagefind is processing an index, number-like strings will be sorted numerically rather than alphabetically. As such, the value passed in should be `"20"` and not `20`*
+
+If successful, a hash is returned containing metadata about the completed indexing.
+
+### index.get_files
+
+Gets raw data of all files in the Pagefind index.
+Useful for integrating a Pagefind index into the development mode of a static site generator and hosting these files yourself.
+
+This method returns every file at once, which can be a lot of data.
+
+``` ruby
+index.get_files.each do |file|
+  path = file[:path] # String
+  content = file[:content] # binary String
+  # ...
+end
+```
+
+### index.write_files
+
+Calling `index.write_files` writes the index files to disk, as they would be written when running the standard Pagefind binary directly.
+
+Exiting `Pagefind::Index.open`'s block automatically calls `index.write_files`, so calling this method is not necessary in normal operation.
+
+Calling this method won't prevent files being written when the block exits, which may cause duplicate files to be written.
+If calling this method manually, you probably want to also call `index.delete_index`.
+
+``` ruby
+Pagefind::Index.open(output_path: "./public/pagefind") do |index|
+  # ... add content to index
+
+  # write files to the configured output path for the index:
+  index.write_files
+
+  # write files to a different output path:
+  index.write_files("./custom/pagefind")
+
+  # prevent also writing files when exiting the block:
+  index.delete_index
+end
+```
+
+The output path should contain the path to the desired Pagefind bundle directory. If relative, it is relative to the current working directory of your Ruby process.
+
+### index.delete_index
+
+Deletes the data for the given index from its backing Pagefind service.
+Doesn't affect any written files or data returned by `get_files`.
+
+``` ruby
+index.delete_index
+```
+
+Calling `index.get_files` or `index.write_files` doesn't consume the index, and further modifications can be made. In situations where many indexes are being created, the `delete_index` call helps clear out memory from a shared Pagefind binary service.
+
+Reusing a `Pagefind::Index` object after calling `index.delete_index` will raise `Pagefind::ServiceError`.
+
+Not calling this method is fine — these indexes will be cleaned up when `Pagefind::Index.open`'s block exits, its backing Pagefind service closes, or your Ruby process exits.
+
+### Pagefind::Service
+
+`Pagefind::Service` manages a Pagefind service running in a subprocess.
+
+When `Pagefind::Service.open` is given a block, the backing service starts, is yielded to the block, and shuts down when the block exits. Without a block, it returns the running service, which you should shut down with `close`.
+
+``` ruby
+service = Pagefind::Service.open
+# ...
+service.close
+
+Pagefind::Service.open do |service| # the service launches
+  # ...
+end
+# the service closes
+```
+
+You should invoke `Pagefind::Service` directly when you want to use the same backing service for many indexes. `service.create_index` takes the same options as `Pagefind::Index.open`.
+
+Indexes created this way are not written to disk automatically, so call `write_files` yourself:
+
+``` ruby
+Pagefind::Service.open do |service|
+  default_index = service.create_index
+  other_index = service.create_index(output_path: "./search/nonstandard")
+
+  default_index.add_directory("./a")
+  other_index.add_directory("./b")
+
+  default_index.write_files
+  other_index.write_files
+end
+```
+
+
 ## Troubleshooting
 
 ### `ERROR: Cannot find the pagefind executable` for supported platform

data/lib/pagefind/index.rb

@@ -0,0 +1,162 @@
+# frozen_string_literal: true
+
+module Pagefind
+  # Manages a Pagefind index.
+  #
+  # Index.open yields the index to a block.
+  # Entering the block starts a backing Pagefind service and creates an in-memory index in the backing service.
+  # Exiting the block writes the in-memory index to disk and then shuts down the backing Pagefind service.
+  #
+  # Each method of Index that talks to the backing Pagefind service can raise ServiceError.
+  # If an exception is raised inside Index.open's block, the block exits without writing the index files to disk.
+  #
+  # Index optionally takes configuration options that can apply parts of the [Pagefind CLI config](https://pagefind.app/docs/config-options/). The options available at this level are listed in Index.new.
+  #
+  # See the relevant documentation for these configuration options in the
+  # [Configuring the Pagefind CLI](https://pagefind.app/docs/config-options/) documentation.
+  class Index
+    # Yields a new index to the block and returns the block's value. Takes the same options as
+    # Index.new.
+    #
+    # You should invoke Service directly when you want to use the same backing service for
+    # many indexes. See Service#create_index.
+    #
+    #: [T] (?root_selector: String?, ?exclude_selectors: Array[String]?, ?force_language: String?, ?verbose: bool?, ?logfile: String?, ?keep_index_url: bool?, ?write_playground: bool?, ?include_characters: String?, ?output_path: (String | Pathname)?) { (Index) -> T } -> T
+    def self.open(**config)
+      raise ArgumentError, "#{name}.open requires a block" unless block_given?
+
+      Service.open do |service|
+        index = new(service, **config)
+        result = yield index
+        index.write_files unless index.instance_variable_get(:@deleted)
+        result
+      end
+    end
+
+    # Creates an in-memory index in the backing Pagefind service.
+    #
+    # @rbs service: Service -- The backing Pagefind service to create the index in.
+    # @rbs root_selector: String? -- The root selector to use for the index.
+    #   If not supplied, Pagefind will use the `<html>` tag.
+    # @rbs exclude_selectors: Array[String]? -- Extra element selectors that Pagefind should ignore when indexing.
+    # @rbs force_language: String? -- Ignores any detected languages and creates a single index for the entire site as the
+    #   provided language. Expects an ISO 639-1 code, such as `en` or `pt`.
+    # @rbs verbose: bool? -- Prints extra logging while indexing the site. Only affects the CLI, does not impact
+    #   web-facing search.
+    # @rbs logfile: String? -- A path to a file to log indexing output to in addition to stdout.
+    #   The file will be created if it doesn't exist and overwritten on each run.
+    # @rbs keep_index_url: bool? -- Whether to keep `index.html` at the end of search result paths.
+    #   By default, a file at `animals/cat/index.html` will be given the URL
+    #   `/animals/cat/`. Setting this option to `true` will result in the URL
+    #   `/animals/cat/index.html`.
+    # @rbs write_playground: bool? -- When writing or outputting files, also write the Pagefind playground to /pagefind/playground/.
+    #   Defaults to false, ensuring the playground isn't available on a live site.
+    # @rbs include_characters: String? -- Include these characters when indexing and searching words.
+    #   Useful for sites documenting technical topics such as programming languages.
+    # @rbs output_path: (String | Pathname)? -- The folder to output the search bundle into, relative to the working directory.
+    #   Defaults to `pagefind`.
+    # @rbs return: void
+    def initialize(
+      service,
+      root_selector: nil,
+      exclude_selectors: nil,
+      force_language: nil,
+      verbose: nil,
+      logfile: nil,
+      keep_index_url: nil,
+      write_playground: nil,
+      include_characters: nil,
+      output_path: nil
+    )
+      @service = service
+      @output_path = output_path
+
+      config = {
+        root_selector:,
+        exclude_selectors:,
+        force_language:,
+        verbose:,
+        logfile:,
+        keep_index_url:,
+        write_playground:,
+        include_characters:
+      }
+      result = @service.send_message({type: "NewIndex", config:})
+      @index_id = result[:index_id]
+    end
+
+    # Adds an HTML file to the index.
+    #
+    # @rbs content: String -- The source HTML content of the file to be parsed.
+    # @rbs source_path: String? -- The source path the HTML file would have on disk.
+    #   Must be a relative path, or an absolute path within the current working directory.
+    #   Pagefind will compute the result URL from this path.
+    # @rbs url: String? -- An explicit URL to use, instead of having Pagefind compute the
+    #   URL based on the source_path. If not supplied, source_path must be supplied.
+    # @rbs return: { type: "IndexedFile", page_word_count: Integer, page_url: String, page_meta: Hash[Symbol, String] }
+    def add_html_file(content:, source_path: nil, url: nil)
+      @service.send_message({type: "AddFile", index_id: @index_id, file_path: source_path, url:, file_contents: content})
+    end
+
+    # Adds a direct record to the Pagefind index.
+    #
+    # This method is useful for adding non-HTML content to the search results.
+    #
+    # @rbs content: String -- The raw content of this record.
+    # @rbs url: String -- The output URL of this record. Pagefind will not alter this.
+    # @rbs language: String -- ISO 639-1 code of the language this record is written in.
+    # @rbs meta: Hash[String | Symbol, String]? -- The metadata to attach to this record. Supplying a `title` is highly recommended.
+    # @rbs filters: Hash[String | Symbol, Array[String]]? -- The filters to attach to this record. Filters are used to group records together.
+    # @rbs sort: Hash[String | Symbol, String]? -- The sort keys to attach to this record.
+    # @rbs return: { type: "IndexedFile", page_word_count: Integer, page_url: String, page_meta: Hash[Symbol, String] }
+    def add_custom_record(url:, content:, language:, meta: nil, filters: nil, sort: nil)
+      @service.send_message({type: "AddRecord", index_id: @index_id, url:, content:, language:, meta:, filters:, sort:})
+    end
+
+    # Indexes a directory from disk using the standard Pagefind indexing behaviour.
+    #
+    # This is equivalent to running the Pagefind binary with `--site <dir>`.
+    #
+    # @rbs path: String | Pathname -- The path to the directory to index. If the `path` provided is relative,
+    #   it will be relative to the current working directory of your Ruby process.
+    # @rbs glob: String? -- A glob pattern to filter files in the directory. If not provided, all
+    #   files matching `**/*.{html}` are indexed. For more information on glob patterns,
+    #   see the [Wax patterns documentation](https://github.com/olson-sean-k/wax#patterns).
+    # @rbs return: { type: "IndexedDir", page_count: Integer }
+    def add_directory(path, glob: nil)
+      @service.send_message({type: "AddDir", index_id: @index_id, path: path.to_s, glob:})
+    end
+
+    # Writes the index files to disk.
+    #
+    # If you're using Index.open, there's no need to call this method:
+    # if no error occurred, exiting the block automatically writes the index files to disk.
+    #
+    # @rbs output_path: (String | Pathname)? -- A path to override the configured output path for the index.
+    # @rbs return: { type: "WriteFiles", output_path: String }
+    def write_files(output_path = @output_path)
+      @service.send_message({type: "WriteFiles", index_id: @index_id, output_path: output_path&.to_s})
+    end
+
+    # Gets raw data of all files in the Pagefind index.
+    #
+    # This method emits all files, which can be a lot of data.
+    #
+    # @rbs return: Array[{ path: String, content: String }] -- Each file's `:content` is a binary string.
+    def get_files
+      result = @service.send_message({type: "GetFiles", index_id: @index_id})
+      result[:files].map do |file|
+        {path: file[:path], content: file[:content].unpack1("m0")}
+      end
+    end
+
+    # Deletes the data for the given index from its backing Pagefind service.
+    # Doesn't affect any written files or data returned by #get_files.
+    #
+    #: () -> void
+    def delete_index
+      @service.send_message({type: "DeleteIndex", index_id: @index_id})
+      @deleted = true
+    end
+  end
+end

data/lib/pagefind/service.rb

@@ -0,0 +1,111 @@
+# frozen_string_literal: true
+
+require "json"
+require "open3"
+
+module Pagefind
+  # Raised when the backing Pagefind service reports an error, or when communicating with it fails.
+  class ServiceError < StandardError
+    # The original message that Pagefind failed to parse, if any.
+    attr_reader :original_message #: String?
+
+    #: (?String? message, ?original_message: String?) -> void
+    def initialize(message = nil, original_message: nil)
+      super(message)
+      @original_message = original_message
+    end
+  end
+
+  # Manages a backing Pagefind service, running as a `pagefind --service` process.
+  #
+  # Pagefind handles one request at a time, so threads sharing a service take turns.
+  class Service
+    # Launches a backing Pagefind service. If a block is given, yields the service and shuts it down
+    # when the block exits, returning the block's value.
+    #
+    #: () -> Service
+    #: [T] () { (Service) -> T } -> T
+    def self.open
+      service = new
+      return service unless block_given?
+
+      begin
+        yield service
+      ensure
+        service.close
+      end
+    end
+
+    # Launches a backing Pagefind service. Call #close to shut it down.
+    #
+    #: () -> void
+    def initialize
+      @message_id = 0
+      @mutex = Thread::Mutex.new
+      @stdin, @stdout, @wait_thread = Open3.popen2(Pagefind.executable, "--service")
+      @stdin.binmode
+      @stdout.binmode
+    end
+
+    # Creates an in-memory index in this service. Takes the same options as Index.new.
+    #
+    #: (?root_selector: String?, ?exclude_selectors: Array[String]?, ?force_language: String?, ?verbose: bool?, ?logfile: String?, ?keep_index_url: bool?, ?write_playground: bool?, ?include_characters: String?, ?output_path: (String | Pathname)?) -> Index
+    def create_index(**) = Index.new(self, **)
+
+    # Sends a request to the backing Pagefind service and returns its response.
+    #
+    # @rbs payload: Hash[Symbol, untyped] -- The request payload to send, such as `{type: "GetFiles", index_id: 0}`.
+    # @rbs return: Hash[Symbol, untyped]
+    def send_message(payload)
+      response = nil
+      @mutex.synchronize do
+        raise ServiceError, "Pagefind service is not running" if @stdin.closed?
+
+        begin
+          write_message(message_id: (@message_id += 1), payload:)
+          response = read_message
+        ensure
+          # the response wasn't read, so shut down before the next request reads it by mistake
+          @stdin.close unless response
+        end
+      end
+      result = response[:payload]
+
+      if response[:message_id].nil?
+        raise ServiceError.new("Pagefind service error when parsing a message: #{result[:message]}", original_message: result[:original_message])
+      elsif result[:type] == "Error"
+        raise ServiceError.new(result[:message], original_message: result[:original_message])
+      end
+
+      result
+    end
+
+    # Waits for any request in progress to finish, then shuts down the backing Pagefind service.
+    #
+    #: () -> nil
+    def close
+      @mutex.synchronize do
+        @stdin.close # pagefind exits when its stdin reaches EOF
+        @wait_thread.join
+        @stdout.close
+      end
+    end
+
+    private
+
+    #: (Hash[Symbol, untyped] request) -> void
+    def write_message(request)
+      @stdin.write([JSON.generate(request)].pack("m0"), ",")
+    rescue IOError, SystemCallError => e
+      raise ServiceError, "Failed to send a message to the Pagefind service: #{e.message}"
+    end
+
+    #: () -> Hash[Symbol, untyped]
+    def read_message
+      output = @stdout.gets(",", chomp: true)
+      raise ServiceError, "Pagefind service exited" if output.nil?
+
+      JSON.parse(output.unpack1("m0"), symbolize_names: true)
+    end
+  end
+end

data/lib/pagefind/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Pagefind
-  VERSION = "1.5.2.1"
+  VERSION = "1.5.2.2"
 end

data/lib/pagefind.rb

@@ -2,6 +2,8 @@
 
 require_relative "pagefind/version"
 require_relative "pagefind/upstream"
+require_relative "pagefind/service"
+require_relative "pagefind/index"
 
 module Pagefind
   DEFAULT_DIR = File.expand_path(File.join(__dir__, "..", "exe"))
@@ -25,6 +27,9 @@ module Pagefind
     end
 
     def executable(exe_path: DEFAULT_DIR)
+      binary_path = ENV["PAGEFIND_EXTENDED_BINARY_PATH"] || ENV["PAGEFIND_BINARY_PATH"]
+      return binary_path if binary_path
+
       pagefind_install_dir = ENV["PAGEFIND_INSTALL_DIR"]
       if pagefind_install_dir
         if File.directory?(pagefind_install_dir)

data/rakelib/package.rake

@@ -55,6 +55,7 @@
 #  - rake gem                    # Build all the gem files
 #  - rake package                # Build all the gem files (same as `gem`)
 #  - rake repackage              # Force a rebuild of all the gem files
+#  - rake test                   # Run the tests (downloads the current platform's binary first)
 #
 #  Note also that the binary executables will be lazily downloaded when needed, but you can
 #  explicitly download them with the `rake download` command.
@@ -156,5 +157,10 @@ end
 desc "Download all pagefind binaries"
 task "download" => [:check, *exepaths]
 
+local_exepath = exepaths.find do |exepath|
+  Gem::Platform.match_gem?(Gem::Platform.new(File.basename(File.dirname(exepath))), PAGEFIND_RUBY_GEMSPEC.name)
+end
+task test: local_exepath if local_exepath
+
 CLOBBER.add(exepaths.map { |p| File.dirname(p) })
 CLOBBER.add(archivepaths)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: pagefind
 version: !ruby/object:Gem::Version
-  version: 1.5.2.1
+  version: 1.5.2.2
 platform: ruby
 authors:
 - 방성범 (Bang Seongbeom)
@@ -9,8 +9,8 @@ bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies: []
-description: A self-contained `pagefind` executable, wrapped up in a ruby gem. That's
-  it. Nothing else.
+description: A self-contained `pagefind` executable, with an indexing API, wrapped
+  up in a ruby gem. That's it. Nothing else.
 email:
 - bangseongbeom@gmail.com
 executables:
@@ -27,6 +27,8 @@ files:
 - Rakefile
 - exe/pagefind
 - lib/pagefind.rb
+- lib/pagefind/index.rb
+- lib/pagefind/service.rb
 - lib/pagefind/upstream.rb
 - lib/pagefind/version.rb
 - rakelib/package.rake
@@ -55,5 +57,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.20
 specification_version: 4
-summary: A self-contained `pagefind` executable.
+summary: A self-contained `pagefind` executable, with an indexing API.
 test_files: []