jekyll-documents 0.4.0 → 0.6.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76640be0741d32137905f8281bc4e83e4e895971a9ad39e7f360bccf06ba5eef
-  data.tar.gz: 93ac35f0606b84b474ef6aff260e21c7dab78ce5e4a8f7f80b4b97493755ae8d
+  metadata.gz: be2955da22172cc24f7aed4e3dca0ac968bf131040a36db5fe2ba048a64bca90
+  data.tar.gz: 2b3d0544fa7c86739e2d83cd9228ee24c64f6bc4ff7691211aefbc7f3533f837
 SHA512:
-  metadata.gz: db8fa1ad5fd90e910238d8b3b41427d8eeb8451ba70ca095cd610e26b56db88ae564e32856b6890c8c1e8af7b4a294fa8a9a9e7219fdd157bc7b0d80aa0c4181
-  data.tar.gz: a19a1da88694fd8282f3040a3bdad01ca5627667a116a984d16d916a875a346bb156bb371374b6b0ae241b6cbe2a97c8ecfc8cb40a1728960243ce1359e60e97
+  metadata.gz: 1f2c7c21a5cdd8c5e996379467b2ceb38d69e368aa9f8bdae882a657366911fb1262f66a7c8ff549d848b1304fd90a8e56e9f37efe1b856f572fab550899d3d5
+  data.tar.gz: fce8b514c14788e75405a026dcd6e7eabf25955a12ac16445380f995728cf6f2c65099ab42af18c51c8fb2c1361f3f3bb9cdfa1845a3642e4495e532640a7d16

data/CHANGELOG.md

@@ -2,6 +2,65 @@
 
 ## [Unreleased]
 
+## [0.6.1] - 2026-09-03
+
+### Changed
+- Text extraction cache keys now include the source digest, extraction configuration, Plaintext version, and cache schema version so configuration or extractor changes invalidate stale results
+- Extracted document content now retains searchable metadata for title, category, file type, and date
+- Text cache cleanup now preserves shared digest files and removes orphaned files safely
+- Text extraction now enforces `text_max_bytes` as a UTF-8 byte limit and closes source files after reading
+- PDF extraction now discovers `pdftotext` from `PATH`; added macOS and Debian/Ubuntu setup instructions and CI coverage with `poppler-utils`
+- Configured text cache directories are excluded from Jekyll output regardless of the `extract_text` setting
+- Rake packaging and test tasks no longer perform duplicate work and install only the current package artifact
+- Removed redundant package metadata and duplicate development dependency declarations
+
+### Fixed
+- Replaced use of the private ActiveSupport deprecation API with the public `ActiveSupport.deprecator` API
+- Empty extraction results now fall back to metadata-only searchable content
+- Manifest loading now validates entries, logs filesystem read failures, and resets dirty state after successful saves
+- Updated stale CI, development, cache-cleanup, and package-lock documentation
+
+### Tests
+- Added real PDF extraction coverage and stronger cache-hit/persistence assertions
+- Added tests for cache configuration invalidation, UTF-8 byte truncation, shared cache references, orphan cleanup, and conditional cache exclusion
+
+## [0.6.0] - 2026-08-27
+
+### Added
+- Auto-inject `file_type`, `icon_url`, and `icon_set` into jekyll-client-search's `passthrough_fields` config when `documents` is in the search collections — zero extra config needed for icons in search results
+- Documented field renaming in jekyll-client-search for integration with other search conventions
+- Framework-agnostic CSS file (`assets/css/documents.css`) with icon scaling utility classes (`icon-x1` through `icon-x9`: 16px to 512px) — no dependency on Bulma, Bootstrap, or Tailwind
+- Icons default to `1em` (line-height) so they scale with surrounding text
+
+### Changed
+- Renamed icon CSS class from `file-icon` to `document-file-icon` to avoid collision with Bulma's `.file-icon` (which uses `display: flex` and breaks inline SVG icons onto a separate line)
+- Inline `1em` sizing added to JS-rendered and folder icons as a fallback when the CSS file is not included
+
+## [0.5.0] - 2026-08-27
+
+### Added
+- Text extraction from PDF, DOCX, XLSX, PPTX, ODT, ODS, and ODP files via the optional `plaintext` gem, enabling full-text search of document contents through `jekyll-client-search`
+- `extract_text` configuration option to enable text extraction (disabled by default)
+- `text_max_bytes` configuration option to control extracted text truncation (default 500KB)
+- `text_cache_dir` configuration option for the persistent cache directory (default `.cache/jekyll-documents`)
+- `TextExtractionManifest` class with SHA-256 content-based cache invalidation, atomic writes, sharded text file storage, and automatic cleanup of entries for deleted source files
+- Cache directory excluded from Jekyll build output via a `:site, :after_init` hook
+- 9 integration tests with real DOCX and ODT fixtures validating extraction, caching, manifest persistence, and cleanup
+- `plaintext` as a development dependency for testing extraction
+
+### Changed
+- Minimum Ruby version raised from 3.3 to 3.4 (tested on Ruby 3.4.10)
+- RuboCop target version updated from 3.3 to 3.4
+- Updated development dependencies: `rubocop` ~> 1.90, `rubocop-performance` ~> 1.27, `simplecov` ~> 1.1, `rake` ~> 13.4
+- Updated Gemfile pins: `google-protobuf` ~> 4.36, `sass-embedded` ~> 1.103
+- CI workflow matrix simplified to Ruby 3.4 only (dropped 3.3)
+- `documentation_uri` in gemspec metadata now points to DeepWiki instead of duplicating the GitHub repo URL
+- Removed redundant `homepage_uri` from gemspec metadata (already covered by `spec.homepage`)
+- Suppressed ActiveSupport deprecation warnings from the `plaintext` gem (`String#mb_chars`, deprecated in Rails 8.2)
+
+### Fixed
+- `CHANGELOG.md` file permissions corrected to be world-readable (was `600`, now `644`)
+
 ## [0.4.0] - 2026-08-26
 
 ### Added

data/README.md

@@ -9,7 +9,7 @@
 
 Turn files in `assets/documents/` into browsable document pages.
 
-**Requirements**: Ruby 3.3+ • Jekyll 4.4+
+**Requirements**: Ruby 3.4+ • Jekyll 4.4+
 
 **Features**: Auto-collection • File icons • Categories • Search
 
@@ -120,6 +120,187 @@ documents:
 
 See [configuration.rb](lib/jekyll/documents/configuration.rb) for all options.
 
+## Icon sizing
+
+Icons default to `1em` (line-height) so they scale with surrounding text.
+A framework-agnostic CSS file with fixed-size utility classes is included:
+
+```html
+<link rel="stylesheet" href="{{ '/assets/css/documents.css' | relative_url }}">
+```
+
+| Class | Size |
+| --- | --- |
+| (default) | `1em` — scales with line height |
+| `icon-x1` | 16px |
+| `icon-x2` | 32px |
+| `icon-x3` | 48px |
+| `icon-x4` | 64px |
+| `icon-x5` | 96px |
+| `icon-x6` | 128px |
+| `icon-x7` | 150px |
+| `icon-x8` | 256px |
+| `icon-x9` | 512px |
+
+Usage with the `document_icon` tag:
+
+```liquid
+{% document_icon page class:"document-file-icon icon-x2" %}
+```
+
+Or with any `<img>`:
+
+```html
+<img src="..." class="document-file-icon icon-x3" />
+```
+
+The CSS uses `display: inline-block` and `vertical-align: middle` — no
+dependency on Bulma, Bootstrap, Tailwind, or any other framework.
+
+## Search Integration
+
+Documents are compatible with [jekyll-client-search](https://github.com/gundestrup/jekyll-client-search)
+for client-side search. Each document has `categories` (plural array) and
+searchable `content` baked in at generation time, so search plugins can
+index uploaded files alongside posts.
+
+To include documents in the search index, add `documents` to the
+`collections` list in `_config.yml`:
+
+```yaml
+client_search:
+  collections:
+    - posts
+    - documents
+```
+
+### Icons and file-type metadata in search results
+
+jekyll-documents bakes `file_type`, `icon_url`, and `icon_set` into each
+document's data at generation time. When jekyll-client-search is installed
+and `documents` is in the search collections, these fields are
+**auto-injected** into the search index — no extra configuration needed:
+
+```yaml
+client_search:
+  collections:
+    - posts
+    - documents
+```
+
+This makes jekyll-client-search render:
+
+- `data-file-type`, `data-icon-set` attributes on each result `<article>`
+  (for CSS-based badges and theme-aware styling)
+- An `<img class="client-search-result-icon">` before the title, using the
+  icon from the configured `icon_set` (color, lines, minimal, or ultra-minimal)
+
+The icon automatically matches the `icon_set` configured in your `documents`
+section — no extra configuration needed for theme consistency.
+
+**Field renaming** (for integration with other search conventions):
+
+```yaml
+client_search:
+  passthrough_fields:
+    - file_type: doctype      # rename in the search index
+    - icon_url: thumbnail
+  icon_field: thumbnail
+```
+
+**CSS examples:**
+
+```css
+/* File-type badge */
+.client-search-result[data-file-type="pdf"]::before {
+  content: "PDF";
+  background: #e74c3c; color: white;
+  padding: 0 0.3em; font-size: 0.7em; margin-right: 0.3em;
+}
+
+/* Theme-aware icon sizing */
+.client-search-result[data-icon-set="color"] .client-search-result-icon { width: 2em; }
+.client-search-result[data-icon-set="ultra-minimal"] .client-search-result-icon { width: 1em; }
+```
+
+See the [jekyll-client-search README](https://github.com/gundestrup/jekyll-client-search#customizing-search-results-with-css)
+for all CSS customization options.
+
+## Text Extraction
+
+Extract text from PDF/DOCX/XLSX/PPTX/ODT/ODS/ODP files so search engines
+can index document contents, not just metadata.
+
+### Setup
+
+Add the optional [`plaintext`](https://github.com/planio-gmbh/plaintext) gem
+to your Gemfile:
+
+```ruby
+# Gemfile
+gem "plaintext", group: :jekyll_plugins
+```
+
+Enable extraction in `_config.yml`:
+
+```yaml
+documents:
+  extract_text: true
+```
+
+### How it works
+
+- Extracted text is stored in `doc.content`, which `jekyll-client-search`
+  indexes automatically — no extra configuration needed
+- Text is cached in `.cache/jekyll-documents/` (in your site source,
+  not `.jekyll-cache/`), so it **survives `jekyll clean`**
+- Cache invalidation uses **SHA-256 file digests** — only changed files
+  are re-extracted
+- Stale cache entries for deleted files are cleaned up automatically
+  at the end of each build
+- Falls back to metadata-only content (title, category, file type, date)
+  if the `plaintext` gem is missing or extraction fails
+
+### Configuration
+
+```yaml
+documents:
+  extract_text: true                    # Enable text extraction
+  text_max_bytes: 500000                # Truncate extracted text (default 500KB)
+  text_cache_dir: ".cache/jekyll-documents"  # Cache directory in site source
+```
+
+Add the cache directory to `.gitignore`:
+
+```
+.cache/jekyll-documents/
+```
+
+### CLI tool dependencies
+
+The `plaintext` gem uses the `rubyzip` Ruby gem for Office formats (no CLI
+tools needed). PDF extraction shells out to a system command:
+
+| Format | Tool |
+|--------|------|
+| PDF | `pdftotext` (poppler-utils) |
+| DOCX/PPTX/XLSX | rubyzip (Ruby gem, no CLI needed) |
+| ODT/ODS/ODP | rubyzip (Ruby gem, no CLI needed) |
+
+Install `pdftotext` for PDF extraction:
+
+```bash
+# macOS
+brew install poppler
+
+# Debian/Ubuntu
+sudo apt-get install poppler-utils
+```
+
+On macOS and other non-standard installations, the executable is discovered
+from `PATH`. You can also configure a custom `pdftotext` command through the
+`plaintext` gem configuration.
+
 ## Development
 
 ```bash

data/_includes/category_list.html

@@ -4,7 +4,7 @@
 <ul class="documents-category-list">
   {% for cat in cats %}
     <li>
-      <img src="{{ '/assets/icons/' | append: icon_set | append: '/folder-svgrepo-com.svg' | relative_url }}" alt="Folder" class="folder-icon" />
+      <img src="{{ '/assets/icons/' | append: icon_set | append: '/folder-svgrepo-com.svg' | relative_url }}" alt="Folder" class="folder-icon" style="width:1em;height:1em;vertical-align:middle;" />
       <span class="category-name">{{ cat }}</span>
       <span class="category-count">({{ site.documents | where: "category", cat | size }})</span>
     </li>

data/assets/css/documents.css

@@ -0,0 +1,34 @@
+/*
+ * jekyll-documents — icon sizing utilities
+ *
+ * Framework-agnostic: no dependency on Bulma, Bootstrap, Tailwind, etc.
+ * Icons default to 1em (line-height) so they scale with surrounding text.
+ * Override with .icon-x{N} classes for fixed pixel sizes.
+ *
+ * Usage:
+ *   {% document_icon page class:"document-file-icon icon-x2" %}
+ *   <img src="..." class="document-file-icon icon-x3" />
+ *
+ * Include this stylesheet in your site:
+ *   <link rel="stylesheet" href="{{ '/assets/css/documents.css' | relative_url }}">
+ */
+
+/* Default: icon scales with line height */
+.document-file-icon,
+.folder-icon {
+  width: 1em;
+  height: 1em;
+  vertical-align: middle;
+  display: inline-block;
+}
+
+/* Fixed pixel sizes */
+.icon-x1  { width: 16px;  height: 16px;  }
+.icon-x2  { width: 32px;  height: 32px;  }
+.icon-x3  { width: 48px;  height: 48px;  }
+.icon-x4  { width: 64px;  height: 64px;  }
+.icon-x5  { width: 96px;  height: 96px;  }
+.icon-x6  { width: 128px; height: 128px; }
+.icon-x7  { width: 150px; height: 150px; }
+.icon-x8  { width: 256px; height: 256px; }
+.icon-x9  { width: 512px; height: 512px; }

data/assets/js/documents-search.js

@@ -32,7 +32,7 @@
     if (!iconUrl) return "";
     const altText = `${String(fileType || "file").toUpperCase()} file`;
     const url = escapeHtml(withBaseurl(iconUrl));
-    return `<img src="${url}" alt="${escapeHtml(altText)}" class="file-icon" />`;
+    return `<img src="${url}" alt="${escapeHtml(altText)}" class="document-file-icon" style="width:1em;height:1em;vertical-align:middle;" />`;
   }
 
   function render(matches) {

data/jekyll-documents.gemspec

@@ -20,12 +20,11 @@ Gem::Specification.new do |spec|
     "source_code_uri" => "https://github.com/gundestrup/jekyll-documents",
     "bug_tracker_uri" => "https://github.com/gundestrup/jekyll-documents/issues",
     "changelog_uri" => "https://github.com/gundestrup/jekyll-documents/blob/main/CHANGELOG.md",
-    "documentation_uri" => "https://github.com/gundestrup/jekyll-documents",
-    "homepage_uri" => "https://github.com/gundestrup/jekyll-documents",
+    "documentation_uri" => "https://deepwiki.com/gundestrup/jekyll-documents",
     "rubygems_mfa_required" => "true"
   }
 
-  spec.required_ruby_version = ">= 3.3"
+  spec.required_ruby_version = ">= 3.4"
 
   spec.files         = Dir.glob("{lib,assets,_includes,_layouts}/**/*") +
                        ["README.md", "CHANGELOG.md", "LICENSE", "jekyll-documents.gemspec"]
@@ -34,10 +33,11 @@ Gem::Specification.new do |spec|
   spec.add_dependency "jekyll", ">= 4.4", "< 5.0"
 
   spec.add_development_dependency "bundler-audit", "~> 0.9"
-  spec.add_development_dependency "rake", "~> 13.0"
+  spec.add_development_dependency "plaintext", "~> 0.3"
+  spec.add_development_dependency "rake", "~> 13.4"
   spec.add_development_dependency "rspec", "~> 3.13"
-  spec.add_development_dependency "rubocop", "~> 1.88"
-  spec.add_development_dependency "rubocop-performance", "~> 1.26"
-  spec.add_development_dependency "simplecov", "~> 1.0"
+  spec.add_development_dependency "rubocop", "~> 1.90"
+  spec.add_development_dependency "rubocop-performance", "~> 1.27"
+  spec.add_development_dependency "simplecov", "~> 1.1"
   spec.add_development_dependency "yard", "~> 0.9"
 end

data/lib/jekyll/documents/configuration.rb

@@ -24,6 +24,11 @@ module Jekyll
         "json_index" => true,
         "json_index_path" => "/documents.json",
 
+        # Text extraction for search indexing (requires optional 'plaintext' gem)
+        "extract_text" => false,
+        "text_max_bytes" => 500_000,
+        "text_cache_dir" => ".cache/jekyll-documents",
+
         # Optional category mapping
         "category_map" => {}
       }.freeze

data/lib/jekyll/documents/file_type_icons.rb

@@ -154,11 +154,13 @@ module Jekyll
       # @return [String] HTML img tag
       # @example
       #   file_type_icon_tag('pdf') #=> \
-      #     '<img src="/assets/icons/color/pdf.svg" alt="PDF file" class="file-icon" />'
-      def file_type_icon_tag(file_type, css_class: "file-icon", alt: nil, context: nil)
+      #     '<img src="..." alt="PDF file" class="document-file-icon" ... />'
+      def file_type_icon_tag(file_type, css_class: "document-file-icon", alt: nil, context: nil)
         url = file_type_icon(file_type, context)
         alt_text = alt || "#{file_type.to_s.upcase} file"
-        %(<img src="#{url}" alt="#{alt_text}" class="#{css_class}" />)
+        style = "width:1em;height:1em;vertical-align:middle;"
+        "<img src=\"#{url}\" alt=\"#{alt_text}\" " \
+          "class=\"#{css_class}\" style=\"#{style}\" />"
       end
 
       private

data/lib/jekyll/documents/generator.rb

@@ -10,12 +10,25 @@ module Jekyll
       safe true
       priority :normal
 
+      CONTENT_TYPES = {
+        ".pdf" => "application/pdf",
+        ".docx" => "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+        ".xlsx" => "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+        ".pptx" => "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+        ".odt" => "application/vnd.oasis.opendocument.text",
+        ".ods" => "application/vnd.oasis.opendocument.spreadsheet",
+        ".odp" => "application/vnd.oasis.opendocument.presentation"
+      }.freeze
+
+      private_constant :CONTENT_TYPES
+
       # Generates document collection from files in configured root directory
       # @param site [Jekyll::Site] the Jekyll site instance
       # @return [void]
       def generate(site)
         @site   = site
         @config = Configuration.read(site)
+        @text_manifest = nil
 
         root = File.join(site.source, @config["root"])
         unless Dir.exist?(root)
@@ -25,6 +38,8 @@ module Jekyll
 
         collection = ensure_collection(site, "documents")
 
+        current_paths = []
+
         Dir.glob("#{root}/**/*").each do |path|
           next unless File.file?(path)
 
@@ -36,6 +51,7 @@ module Jekyll
           next unless @config["include_extensions"].include?(ext)
 
           rel_path = path.delete_prefix("#{site.source}/")
+          current_paths << rel_path
           category = infer_category_from(rel_path)
           basename = File.basename(path, ext)
 
@@ -61,10 +77,31 @@ module Jekyll
           bake_document_data(doc, file_info)
           collection.docs << doc
         end
+
+        cleanup_manifest(current_paths) if @config["extract_text"]
+        configure_client_search(site)
       end
 
       private
 
+      # Auto-injects passthrough_fields into client_search config when the
+      # documents collection is indexed. Only acts if client_search is already
+      # configured with +documents+ in its collections list — does nothing if
+      # jekyll-client-search is not installed or not used.
+      def configure_client_search(site)
+        search_config = site.config["client_search"]
+        return unless search_config.is_a?(Hash)
+        return unless Array(search_config["collections"]).include?("documents")
+
+        fields = search_config["passthrough_fields"] || []
+        existing = fields.flat_map { |f| f.is_a?(Hash) ? f.keys : [f.to_s] }
+        %w[file_type icon_url icon_set].each do |field|
+          fields << field unless existing.include?(field)
+        end
+        search_config["passthrough_fields"] = fields
+        search_config["icon_field"] = "icon_url" unless search_config.key?("icon_field")
+      end
+
       # Ensures a collection exists and is configured for output
       # @param site [Jekyll::Site] the Jekyll site instance
       # @param label [String] the collection name
@@ -83,6 +120,89 @@ module Jekyll
         "#{title} #{data['category']} #{file_type} #{date_str}"
       end
 
+      def extract_file_content(info)
+        return nil unless load_plaintext
+
+        content_type = CONTENT_TYPES[info[:ext]]
+        return nil unless content_type
+
+        manifest = text_manifest
+        rel_path = info[:rel_path]
+        digest = ::Digest::SHA256.file(info[:path]).hexdigest
+        metadata = extraction_cache_metadata
+
+        cached = manifest.get(rel_path, digest, metadata)
+        return cached if cached
+
+        configure_pdf_extractor(content_type)
+        text = ActiveSupport.deprecator.silence do
+          File.open(info[:path], "rb") do |file|
+            ::Plaintext::Resolver.new(file, content_type).text
+          end
+        end
+        text = truncate_bytes(text, @config["text_max_bytes"]) if text
+        manifest.set(rel_path, digest, text, metadata) if text
+        text
+      rescue StandardError => e
+        ::Jekyll.logger.warn "jekyll-documents",
+                             "Text extraction failed for #{info[:path]}: #{e.message}"
+        nil
+      end
+
+      def configure_pdf_extractor(content_type)
+        return unless content_type == "application/pdf"
+        return unless ::Plaintext::Configuration["pdftotext"].nil?
+
+        executable = ENV.fetch("PATH", "").split(File::PATH_SEPARATOR)
+                        .map { |directory| File.join(directory, "pdftotext") }
+                        .find { |path| File.executable?(path) }
+        return unless executable
+
+        ::Plaintext::Configuration.config["pdftotext"] =
+          [executable, "-enc", "UTF-8", "__FILE__", "-"]
+      end
+
+      def load_plaintext
+        return true if defined?(::Plaintext)
+
+        require "plaintext"
+        true
+      rescue LoadError
+        ::Jekyll.logger.warn "jekyll-documents",
+                             "extract_text is enabled but the 'plaintext' gem is not installed. " \
+                             "Run: gem install plaintext"
+        false
+      end
+
+      def text_manifest
+        @text_manifest ||= TextExtractionManifest.new(@site, @config["text_cache_dir"])
+      end
+
+      def extraction_cache_metadata
+        {
+          "schema_version" => 1,
+          "text_max_bytes" => @config["text_max_bytes"],
+          "plaintext_version" => if ::Plaintext.const_defined?(:VERSION)
+                                   ::Plaintext::VERSION.to_s
+                                 else
+                                   "unknown"
+                                 end
+        }
+      end
+
+      def truncate_bytes(text, max_bytes)
+        bytes = text.to_s.encode("UTF-8", invalid: :replace, undef: :replace)
+                    .byteslice(0, max_bytes)
+        bytes = bytes.to_s.force_encoding("UTF-8")
+        bytes = bytes.byteslice(0, bytes.bytesize - 1) until bytes.valid_encoding?
+        bytes
+      end
+
+      def cleanup_manifest(current_rel_paths)
+        text_manifest.cleanup_deleted(current_rel_paths)
+        text_manifest.save
+      end
+
       def bake_document_data(doc, info)
         data = doc.data
         category = remap_category(info[:category])
@@ -102,7 +222,17 @@ module Jekyll
         data["permalink"]  = @config["permalink"]
                              .gsub(":category", category.to_s)
                              .gsub(":slug", info[:slug])
-        doc.content = searchable_content(info[:title], data, info[:file_type])
+        metadata_content = searchable_content(info[:title], data, info[:file_type])
+        doc.content = if @config["extract_text"]
+                        extracted_content = extract_file_content(info)
+                        if extracted_content && !extracted_content.empty?
+                          "#{metadata_content} #{extracted_content}"
+                        else
+                          metadata_content
+                        end
+                      else
+                        metadata_content
+                      end
       end
 
       # Creates a virtual source path for the document

data/lib/jekyll/documents/layout_registrar.rb

@@ -57,4 +57,13 @@ end
 
 Jekyll::Hooks.register :site, :after_init do |site|
   Jekyll::Documents::LayoutRegistrar.register(site)
+
+  # Exclude the text extraction cache directory from Jekyll output
+  config = Jekyll::Documents::Configuration.read(site)
+  if config["text_cache_dir"]
+    site.config["exclude"] = Array(site.config["exclude"])
+    unless site.config["exclude"].include?(config["text_cache_dir"])
+      site.config["exclude"] << config["text_cache_dir"]
+    end
+  end
 end

data/lib/jekyll/documents/tags/doc_link.rb

@@ -63,7 +63,9 @@ module Jekyll
 
         url = relative_url(icon_url)
         file_type = doc.data["file_type"].to_s.upcase
-        "<img src=\"#{escape_html(url)}\" alt=\"#{file_type}\" class=\"file-icon doc-link-icon\" />"
+        "<img src=\"#{escape_html(url)}\" alt=\"#{file_type}\" " \
+          "class=\"document-file-icon doc-link-icon\" " \
+          "style=\"width:1em;height:1em;vertical-align:middle;\" />"
       end
 
       def size_html(doc)

data/lib/jekyll/documents/tags/document_icon.rb

@@ -21,10 +21,11 @@ module Jekyll
         url = relative_url(icon_url)
         file_type = document_value(document, "file_type").to_s
         alt = @options["alt"] || "#{file_type.upcase} file"
-        css_class = @options["class"] || "file-icon"
+        css_class = @options["class"] || "document-file-icon"
 
-        %(<img src="#{escape_html(url)}" alt="#{escape_html(alt)}" ) \
-          + %(class="#{escape_html(css_class)}" />)
+        style = "width:1em;height:1em;vertical-align:middle;"
+        "<img src=\"#{escape_html(url)}\" alt=\"#{escape_html(alt)}\" " \
+          "class=\"#{escape_html(css_class)}\" style=\"#{style}\" />"
       end
 
       private

data/lib/jekyll/documents/text_extraction_manifest.rb

@@ -0,0 +1,193 @@
+# frozen_string_literal: true
+
+require "json"
+require "fileutils"
+require "digest"
+
+module Jekyll
+  module Documents
+    module TextExtractionManifestSupport
+      private
+
+      def load_manifest
+        return {} unless File.file?(@manifest_path)
+
+        data = JSON.parse(File.read(@manifest_path, encoding: "UTF-8"))
+        data.is_a?(Hash) ? data.select { |_path, entry| valid_entry?(entry) } : {}
+      rescue JSON::ParserError => e
+        ::Jekyll.logger.warn "jekyll-documents",
+                             "Corrupt text extraction manifest, starting fresh: #{e.message}"
+        {}
+      rescue StandardError => e
+        ::Jekyll.logger.warn "jekyll-documents",
+                             "Text extraction manifest read failed: #{e.message}"
+        {}
+      end
+
+      def valid_entry?(entry)
+        entry.is_a?(Hash) && entry.values_at("digest", "cache_key", "text_file").all?(String)
+      end
+
+      def cache_key(digest, metadata)
+        Digest::SHA256.hexdigest(JSON.generate("digest" => digest, "metadata" => metadata))
+      end
+
+      def text_file_path(text_file)
+        return unless text_file.is_a?(String)
+
+        path = File.expand_path(File.join(@text_dir, text_file))
+        root = File.expand_path(@text_dir)
+        path.start_with?("#{root}#{File::SEPARATOR}") ? path : nil
+      end
+
+      def write_text_file(digest, text)
+        shard = digest[0, 2]
+        subdir = File.join(@text_dir, shard)
+        FileUtils.mkdir_p(subdir)
+        filename = "#{digest}.txt"
+        File.write(File.join(subdir, filename), text, encoding: "UTF-8")
+        File.join(shard, filename)
+      end
+
+      def cleanup_unreferenced_text_files
+        referenced = @manifest.values.filter_map do |entry|
+          text_file_path(entry["text_file"]) if valid_entry?(entry)
+        end.to_set
+        Dir.glob(File.join(@text_dir, "**", "*.txt")).each do |path|
+          File.delete(path) unless referenced.include?(File.expand_path(path))
+        end
+      rescue StandardError
+        nil
+      end
+    end
+
+    # Manages the text extraction manifest — a persistent JSON file that tracks
+    # which documents have had their text extracted, keyed by SHA-256 digest.
+    #
+    # The manifest lives in <site.source>/<cache_dir>/text-extraction-manifest.json
+    # and extracted text is stored in separate files under <cache_dir>/text/
+    # (sharded by the first 2 hex chars of the digest to avoid huge directories).
+    #
+    # This design borrows from jekyll-imgflow's ManifestManager:
+    # - Atomic writes (temp file + rename)
+    # - SHA-256 content digests for cache invalidation
+    # - Cleanup of entries for deleted source files
+    # - Survives `jekyll clean` (stored in site source, not .jekyll-cache/)
+    class TextExtractionManifest
+      include TextExtractionManifestSupport
+
+      MANIFEST_FILENAME = "text-extraction-manifest.json"
+      TEXT_SUBDIR = "text"
+
+      attr_reader :manifest_path, :cache_dir
+
+      # @param site [Jekyll::Site] the Jekyll site instance
+      # @param cache_dir [String] relative path from site source for cache directory
+      def initialize(site, cache_dir)
+        @site = site
+        @cache_dir = cache_dir
+        @cache_root = File.join(site.source, cache_dir)
+        @manifest_path = File.join(@cache_root, MANIFEST_FILENAME)
+        @text_dir = File.join(@cache_root, TEXT_SUBDIR)
+        @manifest = load_manifest
+        @dirty = false
+      end
+
+      # Get cached text for a document if the digest matches.
+      # @param rel_path [String] relative path of the source document
+      # @param digest [String] SHA-256 hex digest of the source file
+      # @return [String, nil] extracted text if cache hit, nil otherwise
+      def get(rel_path, digest, metadata = {})
+        entry = @manifest[rel_path]
+        return nil unless valid_entry?(entry)
+        return nil unless entry["cache_key"] == cache_key(digest, metadata)
+
+        text_file = text_file_path(entry["text_file"])
+        return nil unless text_file && File.file?(text_file)
+
+        File.read(text_file, encoding: "UTF-8")
+      rescue StandardError => e
+        ::Jekyll.logger.warn "jekyll-documents",
+                             "Manifest read failed for #{rel_path}: #{e.message}"
+        nil
+      end
+
+      # Store extracted text for a document.
+      # @param rel_path [String] relative path of the source document
+      # @param digest [String] SHA-256 hex digest of the source file
+      # @param text [String] the extracted text
+      # @return [void]
+      def set(rel_path, digest, text, metadata = {})
+        text_file = write_text_file(digest, text)
+        @manifest[rel_path] = {
+          "digest" => digest,
+          "cache_key" => cache_key(digest, metadata),
+          "text_file" => text_file,
+          "extracted_at" => Time.now.to_i
+        }
+        cleanup_unreferenced_text_files
+        @dirty = true
+      end
+
+      # Remove manifest entries and their text files for source documents
+      # that no longer exist.
+      # @param current_rel_paths [Array<String>] relative paths of current source files
+      # @return [Integer] number of entries removed
+      def cleanup_deleted(current_rel_paths)
+        current_set = current_rel_paths.to_set
+        removed = 0
+
+        @manifest.each_key do |rel_path|
+          next if current_set.include?(rel_path)
+
+          @manifest.delete(rel_path)
+          removed += 1
+          @dirty = true
+        end
+        cleanup_unreferenced_text_files
+
+        removed
+      end
+
+      # Save the manifest to disk if it has changed.
+      # Uses atomic write (temp file + rename) to prevent corruption.
+      # @return [void]
+      def save
+        return unless @dirty
+
+        content = JSON.pretty_generate(@manifest)
+        if File.exist?(@manifest_path) && File.binread(@manifest_path) == content
+          @dirty = false
+          return
+        end
+
+        FileUtils.mkdir_p(@cache_root)
+        temporary_path = "#{@manifest_path}.tmp-#{Process.pid}-#{Thread.current.object_id}"
+        File.open(temporary_path, "wb") do |file|
+          file.write(content)
+          file.flush
+          file.fsync
+        end
+        File.rename(temporary_path, @manifest_path)
+        @dirty = false
+      ensure
+        FileUtils.rm_f(temporary_path) if defined?(temporary_path) && temporary_path
+      end
+
+      # Check if a document is in the manifest with a matching digest.
+      # @param rel_path [String] relative path of the source document
+      # @param digest [String] SHA-256 hex digest of the source file
+      # @return [Boolean]
+      def cached?(rel_path, digest, metadata = {})
+        entry = @manifest[rel_path]
+        !!(valid_entry?(entry) && entry["cache_key"] == cache_key(digest, metadata))
+      end
+
+      # Number of entries in the manifest.
+      # @return [Integer]
+      def size
+        @manifest.size
+      end
+    end
+  end
+end

data/lib/jekyll/documents/version.rb

@@ -2,6 +2,6 @@
 
 module Jekyll
   module Documents
-    VERSION = "0.4.0"
+    VERSION = "0.6.1"
   end
 end

data/lib/jekyll-documents.rb

@@ -7,6 +7,7 @@ require_relative "jekyll/documents/utils"
 require_relative "jekyll/documents/filters"
 require_relative "jekyll/documents/file_type_icons"
 require_relative "jekyll/documents/generator"
+require_relative "jekyll/documents/text_extraction_manifest"
 require_relative "jekyll/documents/assets_generator"
 require_relative "jekyll/documents/json_index_generator"
 require_relative "jekyll/documents/layout_registrar"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: jekyll-documents
 version: !ruby/object:Gem::Version
-  version: 0.4.0
+  version: 0.6.1
 platform: ruby
 authors:
 - Svend Gundestrup
@@ -43,20 +43,34 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0.9'
+- !ruby/object:Gem::Dependency
+  name: plaintext
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.3'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '13.0'
+        version: '13.4'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '13.0'
+        version: '13.4'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -77,42 +91,42 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.88'
+        version: '1.90'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.88'
+        version: '1.90'
 - !ruby/object:Gem::Dependency
   name: rubocop-performance
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.26'
+        version: '1.27'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.26'
+        version: '1.27'
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.1'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: yard
   requirement: !ruby/object:Gem::Requirement
@@ -144,6 +158,7 @@ files:
 - _includes/documents_search.html
 - _includes/latest_documents.html
 - _layouts/document.html
+- assets/css/documents.css
 - assets/icons/color/ai-document-svgrepo-com.svg
 - assets/icons/color/attachment-document-svgrepo-com.svg
 - assets/icons/color/audio-document-svgrepo-com.svg
@@ -255,6 +270,7 @@ files:
 - lib/jekyll/documents/tags/doc_link.rb
 - lib/jekyll/documents/tags/document_icon.rb
 - lib/jekyll/documents/tags/latest_documents.rb
+- lib/jekyll/documents/text_extraction_manifest.rb
 - lib/jekyll/documents/utils.rb
 - lib/jekyll/documents/version.rb
 homepage: https://github.com/gundestrup/jekyll-documents
@@ -264,8 +280,7 @@ metadata:
   source_code_uri: https://github.com/gundestrup/jekyll-documents
   bug_tracker_uri: https://github.com/gundestrup/jekyll-documents/issues
   changelog_uri: https://github.com/gundestrup/jekyll-documents/blob/main/CHANGELOG.md
-  documentation_uri: https://github.com/gundestrup/jekyll-documents
-  homepage_uri: https://github.com/gundestrup/jekyll-documents
+  documentation_uri: https://deepwiki.com/gundestrup/jekyll-documents
   rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths:
@@ -274,7 +289,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '3.3'
+      version: '3.4'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="