jekyll-documents 0.3.3 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b2e673dc309679ea3bc1ce25687bb9fbf52901f27b7b29517c63ef9337291e82
4
- data.tar.gz: f89cef0e3d22c85665b516bd52da7f137b612c62c21f24e22ef33dc7a10170a8
3
+ metadata.gz: 989b1372181a7ca00444d25b5fab29751a077d9818c41cca1fe8c14208050214
4
+ data.tar.gz: 8215c9234df6eba090b83a296acd0c058a73ae3bd16b0d11fd2ce69e539d788e
5
5
  SHA512:
6
- metadata.gz: 956c33415793dd9e285f9e81874b612278c188d0dd39237a1bbdeea7964af1d784655b171d7efbf485b37dfa948c02ade3703b4e82e1b411a90d9c2049cb0f65
7
- data.tar.gz: c6069125baef1f06acbd58124f5993881b2e595a60b02e470fb0a8a3a94a764f62b0bbd8ad53320b325d614590aa494fb469f83674145f1924db945e1afc0e46
6
+ metadata.gz: ba352213f38733179f053f989cfe5f363dee453176f4d301629fa0d5ab34b5d4f3aad443e8f46bc9b4ee9106626d09cbc255544862b12ba05850259a5b8879e9
7
+ data.tar.gz: 911cb909b005316a55d1978127935cf1bbf6da093c5dd0a9b6ec9e66112a9aa88d094e82c318f8c7312450ec970592caf506a62b392b47e3f0f35380be5be9d5
data/CHANGELOG.md CHANGED
@@ -2,6 +2,65 @@
2
2
 
3
3
  ## [Unreleased]
4
4
 
5
+ ## [0.6.0] - 2026-08-27
6
+
7
+ ### Added
8
+ - Auto-inject `file_type`, `icon_url`, and `icon_set` into jekyll-client-search's `passthrough_fields` config when `documents` is in the search collections — zero extra config needed for icons in search results
9
+ - Documented field renaming in jekyll-client-search for integration with other search conventions
10
+ - Framework-agnostic CSS file (`assets/css/documents.css`) with icon scaling utility classes (`icon-x1` through `icon-x9`: 16px to 512px) — no dependency on Bulma, Bootstrap, or Tailwind
11
+ - Icons default to `1em` (line-height) so they scale with surrounding text
12
+
13
+ ### Changed
14
+ - Renamed icon CSS class from `file-icon` to `document-file-icon` to avoid collision with Bulma's `.file-icon` (which uses `display: flex` and breaks inline SVG icons onto a separate line)
15
+ - Inline `1em` sizing added to JS-rendered and folder icons as a fallback when the CSS file is not included
16
+
17
+ ## [0.5.0] - 2026-08-27
18
+
19
+ ### Added
20
+ - Text extraction from PDF, DOCX, XLSX, PPTX, ODT, ODS, and ODP files via the optional `plaintext` gem, enabling full-text search of document contents through `jekyll-client-search`
21
+ - `extract_text` configuration option to enable text extraction (disabled by default)
22
+ - `text_max_bytes` configuration option to control extracted text truncation (default 500KB)
23
+ - `text_cache_dir` configuration option for the persistent cache directory (default `.cache/jekyll-documents`)
24
+ - `TextExtractionManifest` class with SHA-256 content-based cache invalidation, atomic writes, sharded text file storage, and automatic cleanup of entries for deleted source files
25
+ - Cache directory excluded from Jekyll build output via a `:site, :after_init` hook
26
+ - 9 integration tests with real DOCX and ODT fixtures validating extraction, caching, manifest persistence, and cleanup
27
+ - `plaintext` as a development dependency for testing extraction
28
+
29
+ ### Changed
30
+ - Minimum Ruby version raised from 3.3 to 3.4 (tested on Ruby 3.4.10)
31
+ - RuboCop target version updated from 3.3 to 3.4
32
+ - Updated development dependencies: `rubocop` ~> 1.90, `rubocop-performance` ~> 1.27, `simplecov` ~> 1.1, `rake` ~> 13.4
33
+ - Updated Gemfile pins: `google-protobuf` ~> 4.36, `sass-embedded` ~> 1.103
34
+ - CI workflow matrix simplified to Ruby 3.4 only (dropped 3.3)
35
+ - `documentation_uri` in gemspec metadata now points to DeepWiki instead of duplicating the GitHub repo URL
36
+ - Removed redundant `homepage_uri` from gemspec metadata (already covered by `spec.homepage`)
37
+ - Suppressed ActiveSupport deprecation warnings from the `plaintext` gem (`String#mb_chars`, deprecated in Rails 8.2)
38
+
39
+ ### Fixed
40
+ - `CHANGELOG.md` file permissions corrected to be world-readable (was `600`, now `644`)
41
+
42
+ ## [0.4.0] - 2026-08-26
43
+
44
+ ### Added
45
+ - `categories` array baked into each document's data (in addition to the existing singular `category`) for compatibility with search plugins like `jekyll-client-search` that expect the plural Jekyll convention
46
+ - Searchable content string (title, category, file type, date) set as document content so client-side search engines can index uploaded documents
47
+
48
+ ### Changed
49
+ - Refactored `Generator#generate` by extracting `bake_document_data` and `searchable_content` helper methods to keep method length within RuboCop limits
50
+
51
+ ### Changed
52
+ - Simplified release workflow to tag-push trigger (`push: tags: v*`) — no manual `gh release create` needed
53
+ - Switched to RubyGems trusted publishing (`rubygems/release-gem@v1` with OIDC)
54
+ - Centralized version in `version.rb` as single source of truth — removed `version` field from `package.json` (was drifted to 0.3.1)
55
+ - Added `rake version:bump` and `rake version:check_changelog` tasks
56
+ - Added CHANGELOG gate to release workflow (fails if entry missing for the version)
57
+ - Added `npm audit` and `npm outdated` (non-blocking) to CI
58
+ - Replaced `setup_hooks.sh` with `bin/install-hooks.sh` (pre-commit: rubocop only, pre-push: rubocop + rspec)
59
+ - Dropped reek from gemspec, Rakefile, and deleted `.reek.yml` (KISS)
60
+ - Deleted `bump_version.sh`, `release.sh`, `rollback.sh` — replaced by rake tasks and release workflow
61
+ - Renamed `AI_INSTRUCTIONS.md` to `AGENTS.md` (agents.md open convention)
62
+ - Updated `CLAUDE.md` and `.windsurfrules` to point to `AGENTS.md`
63
+
5
64
  ## [0.3.3] - 2026-08-12
6
65
 
7
66
  ### Added
data/README.md CHANGED
@@ -9,7 +9,7 @@
9
9
 
10
10
  Turn files in `assets/documents/` into browsable document pages.
11
11
 
12
- **Requirements**: Ruby 3.3+ • Jekyll 4.4+
12
+ **Requirements**: Ruby 3.4+ • Jekyll 4.4+
13
13
 
14
14
  **Features**: Auto-collection • File icons • Categories • Search
15
15
 
@@ -120,6 +120,173 @@ documents:
120
120
 
121
121
  See [configuration.rb](lib/jekyll/documents/configuration.rb) for all options.
122
122
 
123
+ ## Icon sizing
124
+
125
+ Icons default to `1em` (line-height) so they scale with surrounding text.
126
+ A framework-agnostic CSS file with fixed-size utility classes is included:
127
+
128
+ ```html
129
+ <link rel="stylesheet" href="{{ '/assets/css/documents.css' | relative_url }}">
130
+ ```
131
+
132
+ | Class | Size |
133
+ | --- | --- |
134
+ | (default) | `1em` — scales with line height |
135
+ | `icon-x1` | 16px |
136
+ | `icon-x2` | 32px |
137
+ | `icon-x3` | 48px |
138
+ | `icon-x4` | 64px |
139
+ | `icon-x5` | 96px |
140
+ | `icon-x6` | 128px |
141
+ | `icon-x7` | 150px |
142
+ | `icon-x8` | 256px |
143
+ | `icon-x9` | 512px |
144
+
145
+ Usage with the `document_icon` tag:
146
+
147
+ ```liquid
148
+ {% document_icon page class:"document-file-icon icon-x2" %}
149
+ ```
150
+
151
+ Or with any `<img>`:
152
+
153
+ ```html
154
+ <img src="..." class="document-file-icon icon-x3" />
155
+ ```
156
+
157
+ The CSS uses `display: inline-block` and `vertical-align: middle` — no
158
+ dependency on Bulma, Bootstrap, Tailwind, or any other framework.
159
+
160
+ ## Search Integration
161
+
162
+ Documents are compatible with [jekyll-client-search](https://github.com/gundestrup/jekyll-client-search)
163
+ for client-side search. Each document has `categories` (plural array) and
164
+ searchable `content` baked in at generation time, so search plugins can
165
+ index uploaded files alongside posts.
166
+
167
+ To include documents in the search index, add `documents` to the
168
+ `collections` list in `_config.yml`:
169
+
170
+ ```yaml
171
+ client_search:
172
+ collections:
173
+ - posts
174
+ - documents
175
+ ```
176
+
177
+ ### Icons and file-type metadata in search results
178
+
179
+ jekyll-documents bakes `file_type`, `icon_url`, and `icon_set` into each
180
+ document's data at generation time. When jekyll-client-search is installed
181
+ and `documents` is in the search collections, these fields are
182
+ **auto-injected** into the search index — no extra configuration needed:
183
+
184
+ ```yaml
185
+ client_search:
186
+ collections:
187
+ - posts
188
+ - documents
189
+ ```
190
+
191
+ This makes jekyll-client-search render:
192
+
193
+ - `data-file-type`, `data-icon-set` attributes on each result `<article>`
194
+ (for CSS-based badges and theme-aware styling)
195
+ - An `<img class="client-search-result-icon">` before the title, using the
196
+ icon from the configured `icon_set` (color, lines, minimal, or ultra-minimal)
197
+
198
+ The icon automatically matches the `icon_set` configured in your `documents`
199
+ section — no extra configuration needed for theme consistency.
200
+
201
+ **Field renaming** (for integration with other search conventions):
202
+
203
+ ```yaml
204
+ client_search:
205
+ passthrough_fields:
206
+ - file_type: doctype # rename in the search index
207
+ - icon_url: thumbnail
208
+ icon_field: thumbnail
209
+ ```
210
+
211
+ **CSS examples:**
212
+
213
+ ```css
214
+ /* File-type badge */
215
+ .client-search-result[data-file-type="pdf"]::before {
216
+ content: "PDF";
217
+ background: #e74c3c; color: white;
218
+ padding: 0 0.3em; font-size: 0.7em; margin-right: 0.3em;
219
+ }
220
+
221
+ /* Theme-aware icon sizing */
222
+ .client-search-result[data-icon-set="color"] .client-search-result-icon { width: 2em; }
223
+ .client-search-result[data-icon-set="ultra-minimal"] .client-search-result-icon { width: 1em; }
224
+ ```
225
+
226
+ See the [jekyll-client-search README](https://github.com/gundestrup/jekyll-client-search#customizing-search-results-with-css)
227
+ for all CSS customization options.
228
+
229
+ ## Text Extraction
230
+
231
+ Extract text from PDF/DOCX/XLSX/PPTX/ODT/ODS/ODP files so search engines
232
+ can index document contents, not just metadata.
233
+
234
+ ### Setup
235
+
236
+ Add the optional [`plaintext`](https://github.com/planio-gmbh/plaintext) gem
237
+ to your Gemfile:
238
+
239
+ ```ruby
240
+ # Gemfile
241
+ gem "plaintext", group: :jekyll_plugins
242
+ ```
243
+
244
+ Enable extraction in `_config.yml`:
245
+
246
+ ```yaml
247
+ documents:
248
+ extract_text: true
249
+ ```
250
+
251
+ ### How it works
252
+
253
+ - Extracted text is stored in `doc.content`, which `jekyll-client-search`
254
+ indexes automatically — no extra configuration needed
255
+ - Text is cached in `.cache/jekyll-documents/` (in your site source,
256
+ not `.jekyll-cache/`), so it **survives `jekyll clean`**
257
+ - Cache invalidation uses **SHA-256 file digests** — only changed files
258
+ are re-extracted
259
+ - Stale cache entries for deleted files are cleaned up automatically
260
+ at the start of each build
261
+ - Falls back to metadata-only content (title, category, file type, date)
262
+ if the `plaintext` gem is missing or extraction fails
263
+
264
+ ### Configuration
265
+
266
+ ```yaml
267
+ documents:
268
+ extract_text: true # Enable text extraction
269
+ text_max_bytes: 500000 # Truncate extracted text (default 500KB)
270
+ text_cache_dir: ".cache/jekyll-documents" # Cache directory in site source
271
+ ```
272
+
273
+ Add the cache directory to `.gitignore`:
274
+
275
+ ```
276
+ .cache/jekyll-documents/
277
+ ```
278
+
279
+ ### CLI tool dependencies
280
+
281
+ The `plaintext` gem uses the `rubyzip` Ruby gem for Office formats (no CLI
282
+ tools needed). PDF extraction shells out to a system command:
283
+
284
+ | Format | Tool |
285
+ |--------|------|
286
+ | PDF | `pdftotext` (poppler-utils) |
287
+ | DOCX/PPTX/XLSX | rubyzip (Ruby gem, no CLI needed) |
288
+ | ODT/ODS/ODP | rubyzip (Ruby gem, no CLI needed) |
289
+
123
290
  ## Development
124
291
 
125
292
  ```bash
@@ -139,8 +306,13 @@ See [README.Development.md](README.Development.md) for details.
139
306
  ## Release
140
307
 
141
308
  ```bash
142
- ./bump_version.sh patch
143
- ./release.sh
309
+ bundle exec rake "version:bump[patch]"
310
+ # Edit CHANGELOG.md
311
+ git add lib/jekyll/documents/version.rb CHANGELOG.md
312
+ git commit -m "Release X.Y.Z: summary"
313
+ git tag -a vX.Y.Z -m "Release X.Y.Z"
314
+ git push origin main
315
+ git push origin vX.Y.Z
144
316
  ```
145
317
 
146
318
  ## License
@@ -4,7 +4,7 @@
4
4
  <ul class="documents-category-list">
5
5
  {% for cat in cats %}
6
6
  <li>
7
- <img src="{{ '/assets/icons/' | append: icon_set | append: '/folder-svgrepo-com.svg' | relative_url }}" alt="Folder" class="folder-icon" />
7
+ <img src="{{ '/assets/icons/' | append: icon_set | append: '/folder-svgrepo-com.svg' | relative_url }}" alt="Folder" class="folder-icon" style="width:1em;height:1em;vertical-align:middle;" />
8
8
  <span class="category-name">{{ cat }}</span>
9
9
  <span class="category-count">({{ site.documents | where: "category", cat | size }})</span>
10
10
  </li>
@@ -0,0 +1,34 @@
1
+ /*
2
+ * jekyll-documents — icon sizing utilities
3
+ *
4
+ * Framework-agnostic: no dependency on Bulma, Bootstrap, Tailwind, etc.
5
+ * Icons default to 1em (line-height) so they scale with surrounding text.
6
+ * Override with .icon-x{N} classes for fixed pixel sizes.
7
+ *
8
+ * Usage:
9
+ * {% document_icon page class:"document-file-icon icon-x2" %}
10
+ * <img src="..." class="document-file-icon icon-x3" />
11
+ *
12
+ * Include this stylesheet in your site:
13
+ * <link rel="stylesheet" href="{{ '/assets/css/documents.css' | relative_url }}">
14
+ */
15
+
16
+ /* Default: icon scales with line height */
17
+ .document-file-icon,
18
+ .folder-icon {
19
+ width: 1em;
20
+ height: 1em;
21
+ vertical-align: middle;
22
+ display: inline-block;
23
+ }
24
+
25
+ /* Fixed pixel sizes */
26
+ .icon-x1 { width: 16px; height: 16px; }
27
+ .icon-x2 { width: 32px; height: 32px; }
28
+ .icon-x3 { width: 48px; height: 48px; }
29
+ .icon-x4 { width: 64px; height: 64px; }
30
+ .icon-x5 { width: 96px; height: 96px; }
31
+ .icon-x6 { width: 128px; height: 128px; }
32
+ .icon-x7 { width: 150px; height: 150px; }
33
+ .icon-x8 { width: 256px; height: 256px; }
34
+ .icon-x9 { width: 512px; height: 512px; }
@@ -32,7 +32,7 @@
32
32
  if (!iconUrl) return "";
33
33
  const altText = `${String(fileType || "file").toUpperCase()} file`;
34
34
  const url = escapeHtml(withBaseurl(iconUrl));
35
- return `<img src="${url}" alt="${escapeHtml(altText)}" class="file-icon" />`;
35
+ return `<img src="${url}" alt="${escapeHtml(altText)}" class="document-file-icon" style="width:1em;height:1em;vertical-align:middle;" />`;
36
36
  }
37
37
 
38
38
  function render(matches) {
@@ -20,12 +20,11 @@ Gem::Specification.new do |spec|
20
20
  "source_code_uri" => "https://github.com/gundestrup/jekyll-documents",
21
21
  "bug_tracker_uri" => "https://github.com/gundestrup/jekyll-documents/issues",
22
22
  "changelog_uri" => "https://github.com/gundestrup/jekyll-documents/blob/main/CHANGELOG.md",
23
- "documentation_uri" => "https://github.com/gundestrup/jekyll-documents",
24
- "homepage_uri" => "https://github.com/gundestrup/jekyll-documents",
23
+ "documentation_uri" => "https://deepwiki.com/gundestrup/jekyll-documents",
25
24
  "rubygems_mfa_required" => "true"
26
25
  }
27
26
 
28
- spec.required_ruby_version = ">= 3.3"
27
+ spec.required_ruby_version = ">= 3.4"
29
28
 
30
29
  spec.files = Dir.glob("{lib,assets,_includes,_layouts}/**/*") +
31
30
  ["README.md", "CHANGELOG.md", "LICENSE", "jekyll-documents.gemspec"]
@@ -34,11 +33,11 @@ Gem::Specification.new do |spec|
34
33
  spec.add_dependency "jekyll", ">= 4.4", "< 5.0"
35
34
 
36
35
  spec.add_development_dependency "bundler-audit", "~> 0.9"
37
- spec.add_development_dependency "rake", "~> 13.0"
38
- spec.add_development_dependency "reek", "~> 6.5"
36
+ spec.add_development_dependency "plaintext", "~> 0.3"
37
+ spec.add_development_dependency "rake", "~> 13.4"
39
38
  spec.add_development_dependency "rspec", "~> 3.13"
40
- spec.add_development_dependency "rubocop", "~> 1.88"
41
- spec.add_development_dependency "rubocop-performance", "~> 1.26"
42
- spec.add_development_dependency "simplecov", "~> 1.0"
39
+ spec.add_development_dependency "rubocop", "~> 1.90"
40
+ spec.add_development_dependency "rubocop-performance", "~> 1.27"
41
+ spec.add_development_dependency "simplecov", "~> 1.1"
43
42
  spec.add_development_dependency "yard", "~> 0.9"
44
43
  end
@@ -24,6 +24,11 @@ module Jekyll
24
24
  "json_index" => true,
25
25
  "json_index_path" => "/documents.json",
26
26
 
27
+ # Text extraction for search indexing (requires optional 'plaintext' gem)
28
+ "extract_text" => false,
29
+ "text_max_bytes" => 500_000,
30
+ "text_cache_dir" => ".cache/jekyll-documents",
31
+
27
32
  # Optional category mapping
28
33
  "category_map" => {}
29
34
  }.freeze
@@ -154,11 +154,13 @@ module Jekyll
154
154
  # @return [String] HTML img tag
155
155
  # @example
156
156
  # file_type_icon_tag('pdf') #=> \
157
- # '<img src="/assets/icons/color/pdf.svg" alt="PDF file" class="file-icon" />'
158
- def file_type_icon_tag(file_type, css_class: "file-icon", alt: nil, context: nil)
157
+ # '<img src="..." alt="PDF file" class="document-file-icon" ... />'
158
+ def file_type_icon_tag(file_type, css_class: "document-file-icon", alt: nil, context: nil)
159
159
  url = file_type_icon(file_type, context)
160
160
  alt_text = alt || "#{file_type.to_s.upcase} file"
161
- %(<img src="#{url}" alt="#{alt_text}" class="#{css_class}" />)
161
+ style = "width:1em;height:1em;vertical-align:middle;"
162
+ "<img src=\"#{url}\" alt=\"#{alt_text}\" " \
163
+ "class=\"#{css_class}\" style=\"#{style}\" />"
162
164
  end
163
165
 
164
166
  private
@@ -10,6 +10,18 @@ module Jekyll
10
10
  safe true
11
11
  priority :normal
12
12
 
13
+ CONTENT_TYPES = {
14
+ ".pdf" => "application/pdf",
15
+ ".docx" => "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
16
+ ".xlsx" => "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
17
+ ".pptx" => "application/vnd.openxmlformats-officedocument.presentationml.presentation",
18
+ ".odt" => "application/vnd.oasis.opendocument.text",
19
+ ".ods" => "application/vnd.oasis.opendocument.spreadsheet",
20
+ ".odp" => "application/vnd.oasis.opendocument.presentation"
21
+ }.freeze
22
+
23
+ private_constant :CONTENT_TYPES
24
+
13
25
  # Generates document collection from files in configured root directory
14
26
  # @param site [Jekyll::Site] the Jekyll site instance
15
27
  # @return [void]
@@ -25,6 +37,8 @@ module Jekyll
25
37
 
26
38
  collection = ensure_collection(site, "documents")
27
39
 
40
+ current_paths = []
41
+
28
42
  Dir.glob("#{root}/**/*").each do |path|
29
43
  next unless File.file?(path)
30
44
 
@@ -36,6 +50,7 @@ module Jekyll
36
50
  next unless @config["include_extensions"].include?(ext)
37
51
 
38
52
  rel_path = path.delete_prefix("#{site.source}/")
53
+ current_paths << rel_path
39
54
  category = infer_category_from(rel_path)
40
55
  basename = File.basename(path, ext)
41
56
 
@@ -55,30 +70,37 @@ module Jekyll
55
70
  collection: collection
56
71
  )
57
72
 
58
- data = doc.data
59
- data["layout"] = @config["layout"]
60
- data["title"] = title
61
- data["date"] = date ? date.to_time : File.mtime(path)
62
- data["category"] = remap_category(category)
63
- data["file_url"] = "/#{rel_path}"
64
- data["extension"] = ext
65
- data["file_type"] = file_type
66
- data["icon_set"] = icon_set
67
- data["icon_url"] = Jekyll::Documents::FileTypeIcons.icon_for(file_type, icon_set)
68
- data["file_size"] = File.size(path)
69
- data["slug"] = slug
70
- data["permalink"] = @config["permalink"]
71
- .gsub(":category", data["category"].to_s)
72
- .gsub(":slug", slug)
73
-
74
- doc.content = "Auto-generated document page."
75
-
73
+ file_info = { title: title, date: date, category: category,
74
+ rel_path: rel_path, ext: ext, file_type: file_type,
75
+ icon_set: icon_set, slug: slug, path: path }
76
+ bake_document_data(doc, file_info)
76
77
  collection.docs << doc
77
78
  end
79
+
80
+ cleanup_manifest(current_paths) if @config["extract_text"]
81
+ configure_client_search(site)
78
82
  end
79
83
 
80
84
  private
81
85
 
86
+ # Auto-injects passthrough_fields into client_search config when the
87
+ # documents collection is indexed. Only acts if client_search is already
88
+ # configured with +documents+ in its collections list — does nothing if
89
+ # jekyll-client-search is not installed or not used.
90
+ def configure_client_search(site)
91
+ search_config = site.config["client_search"]
92
+ return unless search_config.is_a?(Hash)
93
+ return unless Array(search_config["collections"]).include?("documents")
94
+
95
+ fields = search_config["passthrough_fields"] || []
96
+ existing = fields.flat_map { |f| f.is_a?(Hash) ? f.keys : [f.to_s] }
97
+ %w[file_type icon_url icon_set].each do |field|
98
+ fields << field unless existing.include?(field)
99
+ end
100
+ search_config["passthrough_fields"] = fields
101
+ search_config["icon_field"] = "icon_url" unless search_config.key?("icon_field")
102
+ end
103
+
82
104
  # Ensures a collection exists and is configured for output
83
105
  # @param site [Jekyll::Site] the Jekyll site instance
84
106
  # @param label [String] the collection name
@@ -92,6 +114,87 @@ module Jekyll
92
114
  site.collections[label]
93
115
  end
94
116
 
117
+ def searchable_content(title, data, file_type)
118
+ date_str = data["date"].strftime("%Y-%m-%d")
119
+ "#{title} #{data['category']} #{file_type} #{date_str}"
120
+ end
121
+
122
+ def extract_file_content(info)
123
+ return nil unless load_plaintext
124
+
125
+ content_type = CONTENT_TYPES[info[:ext]]
126
+ return nil unless content_type
127
+
128
+ manifest = text_manifest
129
+ rel_path = info[:rel_path]
130
+ digest = ::Digest::SHA256.file(info[:path]).hexdigest
131
+
132
+ cached = manifest.get(rel_path, digest)
133
+ return cached if cached
134
+
135
+ # Suppress ActiveSupport deprecation warnings from the plaintext gem
136
+ # (it uses String#mb_chars, deprecated in Rails 8.2).
137
+ # We apply our own truncation below, so the gem's internal limit is redundant.
138
+ text = ActiveSupport::Deprecation._instance.silence do
139
+ ::Plaintext::Resolver.new(File.open(info[:path]), content_type).text
140
+ end
141
+ text = text&.truncate(@config["text_max_bytes"]) if text
142
+ manifest.set(rel_path, digest, text) if text
143
+ text
144
+ rescue StandardError => e
145
+ ::Jekyll.logger.warn "jekyll-documents",
146
+ "Text extraction failed for #{info[:path]}: #{e.message}"
147
+ nil
148
+ end
149
+
150
+ def load_plaintext
151
+ return true if defined?(::Plaintext)
152
+
153
+ require "plaintext"
154
+ true
155
+ rescue LoadError
156
+ ::Jekyll.logger.warn "jekyll-documents",
157
+ "extract_text is enabled but the 'plaintext' gem is not installed. " \
158
+ "Run: gem install plaintext"
159
+ false
160
+ end
161
+
162
+ def text_manifest
163
+ @text_manifest ||= TextExtractionManifest.new(@site, @config["text_cache_dir"])
164
+ end
165
+
166
+ def cleanup_manifest(current_rel_paths)
167
+ text_manifest.cleanup_deleted(current_rel_paths)
168
+ text_manifest.save
169
+ end
170
+
171
+ def bake_document_data(doc, info)
172
+ data = doc.data
173
+ category = remap_category(info[:category])
174
+ data["layout"] = @config["layout"]
175
+ data["title"] = info[:title]
176
+ data["date"] = info[:date] ? info[:date].to_time : File.mtime(info[:path])
177
+ data["category"] = category
178
+ data["categories"] = [category] if category
179
+ data["file_url"] = "/#{info[:rel_path]}"
180
+ data["extension"] = info[:ext]
181
+ data["file_type"] = info[:file_type]
182
+ data["icon_set"] = info[:icon_set]
183
+ icon = FileTypeIcons.icon_for(info[:file_type], info[:icon_set])
184
+ data["icon_url"] = icon
185
+ data["file_size"] = File.size(info[:path])
186
+ data["slug"] = info[:slug]
187
+ data["permalink"] = @config["permalink"]
188
+ .gsub(":category", category.to_s)
189
+ .gsub(":slug", info[:slug])
190
+ doc.content = if @config["extract_text"]
191
+ extract_file_content(info) || searchable_content(info[:title], data,
192
+ info[:file_type])
193
+ else
194
+ searchable_content(info[:title], data, info[:file_type])
195
+ end
196
+ end
197
+
95
198
  # Creates a virtual source path for the document
96
199
  # @param basename [String] the file basename
97
200
  # @param category [String] the document category
@@ -57,4 +57,13 @@ end
57
57
 
58
58
  Jekyll::Hooks.register :site, :after_init do |site|
59
59
  Jekyll::Documents::LayoutRegistrar.register(site)
60
+
61
+ # Exclude the text extraction cache directory from Jekyll output
62
+ config = Jekyll::Documents::Configuration.read(site)
63
+ if config["extract_text"] && config["text_cache_dir"]
64
+ site.config["exclude"] = Array(site.config["exclude"])
65
+ unless site.config["exclude"].include?(config["text_cache_dir"])
66
+ site.config["exclude"] << config["text_cache_dir"]
67
+ end
68
+ end
60
69
  end
@@ -63,7 +63,9 @@ module Jekyll
63
63
 
64
64
  url = relative_url(icon_url)
65
65
  file_type = doc.data["file_type"].to_s.upcase
66
- "<img src=\"#{escape_html(url)}\" alt=\"#{file_type}\" class=\"file-icon doc-link-icon\" />"
66
+ "<img src=\"#{escape_html(url)}\" alt=\"#{file_type}\" " \
67
+ "class=\"document-file-icon doc-link-icon\" " \
68
+ "style=\"width:1em;height:1em;vertical-align:middle;\" />"
67
69
  end
68
70
 
69
71
  def size_html(doc)
@@ -21,10 +21,11 @@ module Jekyll
21
21
  url = relative_url(icon_url)
22
22
  file_type = document_value(document, "file_type").to_s
23
23
  alt = @options["alt"] || "#{file_type.upcase} file"
24
- css_class = @options["class"] || "file-icon"
24
+ css_class = @options["class"] || "document-file-icon"
25
25
 
26
- %(<img src="#{escape_html(url)}" alt="#{escape_html(alt)}" ) \
27
- + %(class="#{escape_html(css_class)}" />)
26
+ style = "width:1em;height:1em;vertical-align:middle;"
27
+ "<img src=\"#{escape_html(url)}\" alt=\"#{escape_html(alt)}\" " \
28
+ "class=\"#{escape_html(css_class)}\" style=\"#{style}\" />"
28
29
  end
29
30
 
30
31
  private
@@ -0,0 +1,169 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "json"
4
+ require "fileutils"
5
+ require "digest"
6
+
7
+ module Jekyll
8
+ module Documents
9
+ # Manages the text extraction manifest — a persistent JSON file that tracks
10
+ # which documents have had their text extracted, keyed by SHA-256 digest.
11
+ #
12
+ # The manifest lives in <site.source>/<cache_dir>/text-extraction-manifest.json
13
+ # and extracted text is stored in separate files under <cache_dir>/text/
14
+ # (sharded by the first 2 hex chars of the digest to avoid huge directories).
15
+ #
16
+ # This design borrows from jekyll-imgflow's ManifestManager:
17
+ # - Atomic writes (temp file + rename)
18
+ # - SHA-256 content digests for cache invalidation
19
+ # - Cleanup of entries for deleted source files
20
+ # - Survives `jekyll clean` (stored in site source, not .jekyll-cache/)
21
+ class TextExtractionManifest
22
+ MANIFEST_FILENAME = "text-extraction-manifest.json"
23
+ TEXT_SUBDIR = "text"
24
+
25
+ attr_reader :manifest_path, :cache_dir
26
+
27
+ # @param site [Jekyll::Site] the Jekyll site instance
28
+ # @param cache_dir [String] relative path from site source for cache directory
29
+ def initialize(site, cache_dir)
30
+ @site = site
31
+ @cache_dir = cache_dir
32
+ @cache_root = File.join(site.source, cache_dir)
33
+ @manifest_path = File.join(@cache_root, MANIFEST_FILENAME)
34
+ @text_dir = File.join(@cache_root, TEXT_SUBDIR)
35
+ @manifest = load_manifest
36
+ @dirty = false
37
+ end
38
+
39
+ # Get cached text for a document if the digest matches.
40
+ # @param rel_path [String] relative path of the source document
41
+ # @param digest [String] SHA-256 hex digest of the source file
42
+ # @return [String, nil] extracted text if cache hit, nil otherwise
43
+ def get(rel_path, digest)
44
+ entry = @manifest[rel_path]
45
+ return nil unless entry
46
+ return nil unless entry["digest"] == digest
47
+
48
+ text_file = text_file_path(entry["text_file"])
49
+ return nil unless File.file?(text_file)
50
+
51
+ File.read(text_file, encoding: "UTF-8")
52
+ rescue StandardError => e
53
+ ::Jekyll.logger.warn "jekyll-documents",
54
+ "Manifest read failed for #{rel_path}: #{e.message}"
55
+ nil
56
+ end
57
+
58
+ # Store extracted text for a document.
59
+ # @param rel_path [String] relative path of the source document
60
+ # @param digest [String] SHA-256 hex digest of the source file
61
+ # @param text [String] the extracted text
62
+ # @return [void]
63
+ def set(rel_path, digest, text)
64
+ text_file = write_text_file(digest, text)
65
+ @manifest[rel_path] = {
66
+ "digest" => digest,
67
+ "text_file" => text_file,
68
+ "extracted_at" => Time.now.to_i
69
+ }
70
+ @dirty = true
71
+ end
72
+
73
+ # Remove manifest entries and their text files for source documents
74
+ # that no longer exist.
75
+ # @param current_rel_paths [Array<String>] relative paths of current source files
76
+ # @return [Integer] number of entries removed
77
+ def cleanup_deleted(current_rel_paths)
78
+ current_set = current_rel_paths.to_set
79
+ removed = 0
80
+
81
+ @manifest.each_key do |rel_path|
82
+ next if current_set.include?(rel_path)
83
+
84
+ entry = @manifest[rel_path]
85
+ delete_text_file(entry["text_file"]) if entry
86
+ @manifest.delete(rel_path)
87
+ removed += 1
88
+ @dirty = true
89
+ end
90
+
91
+ removed
92
+ end
93
+
94
+ # Save the manifest to disk if it has changed.
95
+ # Uses atomic write (temp file + rename) to prevent corruption.
96
+ # @return [void]
97
+ def save
98
+ return unless @dirty
99
+
100
+ content = JSON.pretty_generate(@manifest)
101
+ return if File.exist?(@manifest_path) && File.binread(@manifest_path) == content
102
+
103
+ FileUtils.mkdir_p(@cache_root)
104
+ temporary_path = "#{@manifest_path}.tmp-#{Process.pid}-#{Thread.current.object_id}"
105
+ File.open(temporary_path, "wb") do |file|
106
+ file.write(content)
107
+ file.flush
108
+ file.fsync
109
+ end
110
+ File.rename(temporary_path, @manifest_path)
111
+ ensure
112
+ FileUtils.rm_f(temporary_path) if defined?(temporary_path) && temporary_path
113
+ end
114
+
115
+ # Check if a document is in the manifest with a matching digest.
116
+ # @param rel_path [String] relative path of the source document
117
+ # @param digest [String] SHA-256 hex digest of the source file
118
+ # @return [Boolean]
119
+ def cached?(rel_path, digest)
120
+ entry = @manifest[rel_path]
121
+ !!(entry && entry["digest"] == digest)
122
+ end
123
+
124
+ # Number of entries in the manifest.
125
+ # @return [Integer]
126
+ def size
127
+ @manifest.size
128
+ end
129
+
130
+ private
131
+
132
+ def load_manifest
133
+ return {} unless File.file?(@manifest_path)
134
+
135
+ data = JSON.parse(File.read(@manifest_path, encoding: "UTF-8"))
136
+ data.is_a?(Hash) ? data : {}
137
+ rescue JSON::ParserError => e
138
+ ::Jekyll.logger.warn "jekyll-documents",
139
+ "Corrupt text extraction manifest, starting fresh: #{e.message}"
140
+ {}
141
+ rescue StandardError
142
+ {}
143
+ end
144
+
145
+ def text_file_path(text_file)
146
+ File.join(@text_dir, text_file)
147
+ end
148
+
149
+ def write_text_file(digest, text)
150
+ shard = digest[0, 2]
151
+ subdir = File.join(@text_dir, shard)
152
+ FileUtils.mkdir_p(subdir)
153
+ filename = "#{digest}.txt"
154
+ path = File.join(subdir, filename)
155
+ File.write(path, text, encoding: "UTF-8")
156
+ File.join(shard, filename)
157
+ end
158
+
159
+ def delete_text_file(text_file)
160
+ return unless text_file
161
+
162
+ path = text_file_path(text_file)
163
+ File.delete(path) if File.file?(path)
164
+ rescue StandardError
165
+ nil
166
+ end
167
+ end
168
+ end
169
+ end
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Jekyll
4
4
  module Documents
5
- VERSION = "0.3.3"
5
+ VERSION = "0.6.0"
6
6
  end
7
7
  end
@@ -7,6 +7,7 @@ require_relative "jekyll/documents/utils"
7
7
  require_relative "jekyll/documents/filters"
8
8
  require_relative "jekyll/documents/file_type_icons"
9
9
  require_relative "jekyll/documents/generator"
10
+ require_relative "jekyll/documents/text_extraction_manifest"
10
11
  require_relative "jekyll/documents/assets_generator"
11
12
  require_relative "jekyll/documents/json_index_generator"
12
13
  require_relative "jekyll/documents/layout_registrar"
metadata CHANGED
@@ -1,14 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: jekyll-documents
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.3
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Svend Gundestrup
8
- autorequire:
9
8
  bindir: bin
10
9
  cert_chain: []
11
- date: 2026-08-13 00:00:00.000000000 Z
10
+ date: 1980-01-02 00:00:00.000000000 Z
12
11
  dependencies:
13
12
  - !ruby/object:Gem::Dependency
14
13
  name: jekyll
@@ -45,33 +44,33 @@ dependencies:
45
44
  - !ruby/object:Gem::Version
46
45
  version: '0.9'
47
46
  - !ruby/object:Gem::Dependency
48
- name: rake
47
+ name: plaintext
49
48
  requirement: !ruby/object:Gem::Requirement
50
49
  requirements:
51
50
  - - "~>"
52
51
  - !ruby/object:Gem::Version
53
- version: '13.0'
52
+ version: '0.3'
54
53
  type: :development
55
54
  prerelease: false
56
55
  version_requirements: !ruby/object:Gem::Requirement
57
56
  requirements:
58
57
  - - "~>"
59
58
  - !ruby/object:Gem::Version
60
- version: '13.0'
59
+ version: '0.3'
61
60
  - !ruby/object:Gem::Dependency
62
- name: reek
61
+ name: rake
63
62
  requirement: !ruby/object:Gem::Requirement
64
63
  requirements:
65
64
  - - "~>"
66
65
  - !ruby/object:Gem::Version
67
- version: '6.5'
66
+ version: '13.4'
68
67
  type: :development
69
68
  prerelease: false
70
69
  version_requirements: !ruby/object:Gem::Requirement
71
70
  requirements:
72
71
  - - "~>"
73
72
  - !ruby/object:Gem::Version
74
- version: '6.5'
73
+ version: '13.4'
75
74
  - !ruby/object:Gem::Dependency
76
75
  name: rspec
77
76
  requirement: !ruby/object:Gem::Requirement
@@ -92,42 +91,42 @@ dependencies:
92
91
  requirements:
93
92
  - - "~>"
94
93
  - !ruby/object:Gem::Version
95
- version: '1.88'
94
+ version: '1.90'
96
95
  type: :development
97
96
  prerelease: false
98
97
  version_requirements: !ruby/object:Gem::Requirement
99
98
  requirements:
100
99
  - - "~>"
101
100
  - !ruby/object:Gem::Version
102
- version: '1.88'
101
+ version: '1.90'
103
102
  - !ruby/object:Gem::Dependency
104
103
  name: rubocop-performance
105
104
  requirement: !ruby/object:Gem::Requirement
106
105
  requirements:
107
106
  - - "~>"
108
107
  - !ruby/object:Gem::Version
109
- version: '1.26'
108
+ version: '1.27'
110
109
  type: :development
111
110
  prerelease: false
112
111
  version_requirements: !ruby/object:Gem::Requirement
113
112
  requirements:
114
113
  - - "~>"
115
114
  - !ruby/object:Gem::Version
116
- version: '1.26'
115
+ version: '1.27'
117
116
  - !ruby/object:Gem::Dependency
118
117
  name: simplecov
119
118
  requirement: !ruby/object:Gem::Requirement
120
119
  requirements:
121
120
  - - "~>"
122
121
  - !ruby/object:Gem::Version
123
- version: '1.0'
122
+ version: '1.1'
124
123
  type: :development
125
124
  prerelease: false
126
125
  version_requirements: !ruby/object:Gem::Requirement
127
126
  requirements:
128
127
  - - "~>"
129
128
  - !ruby/object:Gem::Version
130
- version: '1.0'
129
+ version: '1.1'
131
130
  - !ruby/object:Gem::Dependency
132
131
  name: yard
133
132
  requirement: !ruby/object:Gem::Requirement
@@ -159,6 +158,7 @@ files:
159
158
  - _includes/documents_search.html
160
159
  - _includes/latest_documents.html
161
160
  - _layouts/document.html
161
+ - assets/css/documents.css
162
162
  - assets/icons/color/ai-document-svgrepo-com.svg
163
163
  - assets/icons/color/attachment-document-svgrepo-com.svg
164
164
  - assets/icons/color/audio-document-svgrepo-com.svg
@@ -270,6 +270,7 @@ files:
270
270
  - lib/jekyll/documents/tags/doc_link.rb
271
271
  - lib/jekyll/documents/tags/document_icon.rb
272
272
  - lib/jekyll/documents/tags/latest_documents.rb
273
+ - lib/jekyll/documents/text_extraction_manifest.rb
273
274
  - lib/jekyll/documents/utils.rb
274
275
  - lib/jekyll/documents/version.rb
275
276
  homepage: https://github.com/gundestrup/jekyll-documents
@@ -279,10 +280,8 @@ metadata:
279
280
  source_code_uri: https://github.com/gundestrup/jekyll-documents
280
281
  bug_tracker_uri: https://github.com/gundestrup/jekyll-documents/issues
281
282
  changelog_uri: https://github.com/gundestrup/jekyll-documents/blob/main/CHANGELOG.md
282
- documentation_uri: https://github.com/gundestrup/jekyll-documents
283
- homepage_uri: https://github.com/gundestrup/jekyll-documents
283
+ documentation_uri: https://deepwiki.com/gundestrup/jekyll-documents
284
284
  rubygems_mfa_required: 'true'
285
- post_install_message:
286
285
  rdoc_options: []
287
286
  require_paths:
288
287
  - lib
@@ -290,15 +289,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
290
289
  requirements:
291
290
  - - ">="
292
291
  - !ruby/object:Gem::Version
293
- version: '3.3'
292
+ version: '3.4'
294
293
  required_rubygems_version: !ruby/object:Gem::Requirement
295
294
  requirements:
296
295
  - - ">="
297
296
  - !ruby/object:Gem::Version
298
297
  version: '0'
299
298
  requirements: []
300
- rubygems_version: 3.5.22
301
- signing_key:
299
+ rubygems_version: 3.6.9
302
300
  specification_version: 4
303
301
  summary: Auto-generate Jekyll pages for documents (PDF/DOCX/...) with category/date/title
304
302
  parsing.