pikuri-core 0.0.7 → 0.1.0

data/lib/pikuri/extractor.rb

@@ -1,82 +1,54 @@
 # frozen_string_literal: true
 
 module Pikuri
-  # The format→text extraction seam: one registry of extractors that
-  # turn an +IO+ of some recognised format (HTML and plain text out
-  # of the box; PDF / office formats via the pikuri-pdf /
-  # pikuri-extractors plug-in gems) into Markdown-flavoured UTF-8
-  # text, consumed through two front doors:
+  # The format→text extraction seam: one ordered registry of extractors
+  # turning an +IO+ of a recognised format (HTML + plain text in core;
+  # PDF / office formats via the pikuri-pdf / pikuri-extractors gems) into
+  # Markdown-flavoured UTF-8, behind two front doors:
   #
-  # * {.extract} — the whole document as one String. The shape the
-  #   indexing / caching callers want ({Pikuri::VectorDb}'s indexer,
-  #   {Tool::WebScrape}'s URL cache): no windowing, no presentation.
-  # * {.extract_paged} — the LLM-tool shape: the same extraction,
-  #   windowed to a line range with a byte cap, returned as a {Page}
-  #   the caller renders. Backs +Workspace::Read+ and
-  #   +VectorDb::Tools::Read+ (via the {FileType} path wrappers) so
-  #   the offset/limit/byte-cap logic lives in one tested place.
+  # * {.extract} — the whole document as one String (no windowing).
+  # * {.extract_paged} — the same, windowed to a line range with a byte
+  #   cap, as a {Page} the caller renders.
   #
-  # Both front doors — +Tool::Scraper+ dispatching on the HTTP
-  # +Content-Type+ header for the web tools, and {FileType} resolving
-  # local paths — route through this one registry, so both share one
-  # set of format truths and "support a new format" is a registry
-  # entry (pikuri-pdf and pikuri-extractors plug PDF and office
-  # formats in without pikuri-core knowing), not a new special case
-  # in two dispatchers.
+  # +Tool::Scraper+ (dispatching on the HTTP +Content-Type+) and {FileType}
+  # (resolving local paths) both route through this registry, so "support a
+  # new format" is a registry entry, not a new dispatcher arm.
   #
   # == The extractor duck type
   #
-  # Each {.registry} entry implements three methods:
+  # Each {.registry} entry implements:
   #
-  # * +matches?(sample:, content_type:)+ → +Boolean+ — claim the
-  #   content. +sample+ is the leading {FileType::SAMPLE_BYTES} bytes
-  #   (for magic-byte sniffs); +content_type+ is the normalized HTTP
-  #   +Content-Type+ for web content, the {FileType.detect_mime}
-  #   result for local files, and may be +nil+ ("no transport
-  #   metadata — sniff if you can").
-  # * +extract(io)+ → +String+ — the whole document as
-  #   Markdown-flavoured UTF-8 text. Raises {Error} on content the
-  #   extractor claimed but cannot parse (malformed PDF, ...).
-  # * +kind+ → +Symbol+ — a short tag (+:text+ / +:pdf+ / +:html+)
-  #   carried on {Page#kind} so rendering callers can word
-  #   format-specific trailers ("End of PDF", the scanned-image
-  #   hint) without re-sniffing.
+  # * +matches?(sample:, content_type:)+ → +Boolean+ — claim the content.
+  #   +sample+ is the leading {FileType::SAMPLE_BYTES} (magic-byte sniff);
+  #   +content_type+ is the normalized HTTP/{FileType.detect_mime} type, or
+  #   +nil+ ("no metadata — sniff if you can").
+  # * +extract(io)+ → +String+ — whole document as Markdown-flavoured UTF-8.
+  #   Raises {Error} on content it claimed but can't parse.
+  # * +kind+ → +Symbol+ — a short tag (+:text+ / +:pdf+ / +:html+) carried
+  #   on {Page#kind} so callers word format-specific trailers without
+  #   re-sniffing.
   #
-  # plus one *optional* method for formats whose lines can be
-  # produced incrementally:
+  # plus one *optional* method:
   #
-  # * +extract_lines(io)+ → +Enumerator<String>+ — the same content
-  #   as +extract+, as a lazy stream of already-+chomp+ed lines.
-  #   {.extract_paged} prefers this when present and stops consuming
-  #   the moment the window fills, so the rest of the document is
-  #   never parsed (pikuri-pdf's extractor: pdf-reader's page list
-  #   parses on access; {Passthrough}: the IO is read line-by-line).
-  #   The enumerator
-  #   must be consumed while +io+ is still open, and may raise
-  #   {Error} mid-iteration. Extractors that need the whole document
-  #   to produce anything ({HTML}: Readability walks the full DOM —
-  #   true of any subprocess-based extractor too) simply omit it;
-  #   {.extract_paged} then extracts in full and windows the result.
+  # * +extract_lines(io)+ → +Enumerator<String>+ — the same content as a
+  #   lazy stream of +chomp+ed lines. {.extract_paged} prefers it and stops
+  #   the moment the window fills, so the document's tail is never parsed
+  #   (pikuri-pdf parses pages on access). Must be consumed while +io+ is
+  #   open; may raise {Error} mid-iteration. Extractors needing the whole
+  #   document ({HTML}, any subprocess-based one) omit it, and
+  #   {.extract_paged} extracts in full then windows.
   #
-  # Windowing itself (offset / limit / byte cap / line truncation) is
-  # presentation and deliberately lives once in {.extract_paged}, not
-  # per extractor — +extract_lines+ is line *production*, the only
-  # genuinely format-specific half of paging.
+  # Windowing (offset / limit / byte cap / truncation) is presentation and
+  # lives once in {.extract_paged}, never per extractor — +extract_lines+ is
+  # line *production*, the only format-specific half of paging.
   #
   # == Errors
   #
-  # Both failure modes are failures the *caller's* LLM can react to,
-  # so they share one rescuable root:
+  # Both modes are failures the caller's LLM can react to, under one
+  # rescuable root:
   #
-  # * {Unsupported} — nothing in {.registry} claimed the content
-  #   (opaque binary, an unhandled content-type).
-  # * {Error} (the root) — an extractor claimed the content but the
-  #   parse failed (malformed PDF, ...).
-  #
-  # Callers map them to their own conventions:
-  # +Tool::Scraper+ re-raises both as +FetchError+;
-  # {FileType.read_as_text} maps {Unsupported} to the +ArgumentError+
-  # binary refusal and {Error} to a +RuntimeError+ carrying the path.
+  # * {Unsupported} — nothing in {.registry} claimed the content.
+  # * {Error} (the root) — an extractor claimed it but the parse failed.
   module Extractor
     module_function
 
@@ -108,71 +80,49 @@ module Pikuri
     #   {PAGE_MAX_LINE_LENGTH}.
     PAGE_LINE_TRUNCATION_MARKER = "... (line truncated to #{PAGE_MAX_LINE_LENGTH} chars)"
 
-    # One windowed slice of a document, returned by {.extract_paged}.
-    # The caller turns this into an observation; this struct carries
-    # everything a trailer needs without the caller re-reading the
-    # document.
-    #
-    # == Fields
+    # One windowed slice of a document, returned by {.extract_paged},
+    # carrying everything a trailer needs without re-reading the document.
     #
-    # * +lines+ — +Array<String>+, the collected window. Already
-    #   per-line truncated (with {PAGE_LINE_TRUNCATION_MARKER}); *not*
-    #   line-numbered — numbering is presentation the caller adds. For
-    #   a PDF the array includes the +"--- Page N ---"+ marker lines
-    #   pikuri-pdf's extractor emits, which count toward +limit+ / the
-    #   byte cap like any other line.
-    # * +start_line+ — the 1-indexed line number of +lines.first+
-    #   (i.e. the +offset+ the caller asked for). +lines.last+ is at
-    #   +start_line + lines.length - 1+.
-    # * +total_lines+ — total line count of the document when known,
-    #   else +nil+. Known when the read reached EOF, when the format
-    #   was extracted in full (no +extract_lines+ — e.g. HTML), or
-    #   when the lazy stream is cheap enough to count to the end
-    #   (plain text). +nil+ when a lazy stream stopped early — the
-    #   byte cap fired, or a PDF filled the window before its last
-    #   page (counting the rest would mean parsing every page,
-    #   defeating the laziness).
-    # * +more+ — +true+ if content remains past this window (the
-    #   caller should offer +offset = start_line + lines.length+).
-    # * +byte_capped+ — +true+ if the byte cap (not the line limit)
-    #   was the stopping criterion.
-    # * +kind+ — the matched extractor's +kind+ tag (+:text+ /
-    #   +:pdf+ / +:html+); lets the caller word format-specific
-    #   trailers and the empty-document message.
+    # * +lines+ — +Array<String>+, the collected window; already per-line
+    #   truncated (with {PAGE_LINE_TRUNCATION_MARKER}), *not* line-numbered
+    #   (numbering is the caller's presentation). Includes any +"--- Page N
+    #   ---"+ marker lines, which count toward +limit+/the byte cap.
+    # * +start_line+ — 1-indexed line number of +lines.first+ (the requested
+    #   +offset+).
+    # * +total_lines+ — +Integer+ document line count when known, else +nil+.
+    #   Known at EOF, when extracted in full (HTML), or on a cheap-to-count
+    #   stream (text); +nil+ when a lazy stream stopped early (byte cap, or a
+    #   PDF window filled before the last page — counting the rest would
+    #   parse every page).
+    # * +more+ — +true+ if content remains past this window.
+    # * +byte_capped+ — +true+ if the byte cap (not the line limit) stopped it.
+    # * +kind+ — the matched extractor's +kind+ tag.
     #
-    # An empty document yields +lines: []+, +total_lines: 0+; an
-    # +offset+ past EOF yields +lines: []+ with +total_lines+ set to
-    # the real (non-zero) count — the caller distinguishes the two.
+    # Empty document → +lines: []+, +total_lines: 0+; +offset+ past EOF →
+    # +lines: []+ with +total_lines+ at the real count — the caller
+    # distinguishes the two.
     Page = Data.define(:lines, :start_line, :total_lines, :more, :byte_capped, :kind)
 
-    # The extractor registry, consulted in order — first match wins.
-    # Core ships two entries: {HTML} matches on content-type, and
-    # {Passthrough} is the terminal plain-text arm. A gem adding a
-    # format picks its insertion point by the strength of its claim:
-    # a magic-byte sniff that never misfires on text goes at the
-    # *front* so it beats {HTML}'s content-type match even under a
-    # lying header (+registry.unshift(X)+ — pikuri-pdf does this);
-    # a content-type / weaker-sniff claimer inserts before the
-    # terminal entry (+registry.insert(-2, X)+ — pikuri-extractors
-    # does this).
+    # The extractor registry, consulted in order — first match wins. Core
+    # ships {HTML} (content-type) then the terminal {Passthrough} plain-text
+    # arm. A gem picks its insertion point by claim strength: a magic-byte
+    # sniff that never misfires on text goes at the *front* to beat {HTML}
+    # under a lying header (+registry.unshift(X)+ — pikuri-pdf); a
+    # content-type/weaker claimer inserts before the terminal entry
+    # (+registry.insert(-2, X)+ — pikuri-extractors).
     #
-    # @return [Array<#matches?>] mutable, deliberately — this is the
-    #   plug-in seam.
+    # @return [Array<#matches?>] mutable, deliberately — the plug-in seam.
     def registry
       @registry ||= [HTML, Passthrough]
     end
 
-    # Extract the whole document behind +io+ as one Markdown-flavoured
-    # UTF-8 String. May be empty (empty text file, scanned-image PDF
-    # with no extractable text).
+    # Extract the whole document behind +io+ as one Markdown-flavoured UTF-8
+    # String. May be empty (empty text file, scanned-image PDF).
     #
-    # @param io [IO, StringIO] seekable IO positioned at the start of
-    #   the content; this method reads a leading sample for the
-    #   +matches?+ sniff and rewinds before extracting.
-    # @param content_type [String, nil] normalized content-type when
-    #   the transport supplies one (HTTP header, {FileType.detect_mime}
-    #   result); +nil+ when unknown — extractors then rely on their
-    #   byte sniffs.
+    # @param io [IO, StringIO] seekable IO at the content start; reads a
+    #   leading sample for the sniff and rewinds before extracting.
+    # @param content_type [String, nil] normalized content-type when the
+    #   transport supplies one; +nil+ when unknown (extractors sniff).
     # @return [String]
     # @raise [Unsupported] when no registry entry claims the content.
     # @raise [Error] when the matched extractor cannot parse it.
@@ -180,41 +130,34 @@ module Pikuri
       extractor_for(io, content_type).extract(io)
     end
 
-    # Extract +io+ and return a windowed {Page}: the lines from
-    # +offset+ (1-indexed) up to +limit+ of them, stopping early if
-    # +max_bytes+ is reached, with over-long lines truncated at
+    # Extract +io+ into a windowed {Page}: lines from +offset+ (1-indexed) up
+    # to +limit+, stopping early at +max_bytes+, over-long lines truncated at
     # +max_line_length+.
     #
-    # Lazy where the format allows: extractors that implement
-    # +extract_lines+ (plain text; pikuri-pdf's PDF) are consumed
-    # only until the window fills — reading the first window of a
-    # 500-page PDF parses a handful of pages, and the first page of
-    # a gigabyte log never loads it. Extractors without it (HTML) are extracted
-    # in full and then windowed, which is also what makes their
-    # +total_lines+ always exact.
+    # Lazy where the format allows: +extract_lines+ extractors (text,
+    # pikuri-pdf) are consumed only until the window fills — a 500-page PDF's
+    # first window parses a handful of pages. Extractors without it (HTML) are
+    # extracted in full then windowed, which is why their +total_lines+ is
+    # always exact.
     #
-    # @param io [IO, StringIO] seekable IO positioned at the start.
+    # @param io [IO, StringIO] seekable IO at the start.
     # @param content_type [String, nil] as for {.extract}.
-    # @param offset [Integer] 1-indexed first line to include. The
-    #   caller is responsible for validating +offset >= 1+.
-    # @param limit [Integer] maximum lines to collect. Caller
-    #   validates +limit >= 1+.
+    # @param offset [Integer] 1-indexed first line; caller validates +>= 1+.
+    # @param limit [Integer] max lines to collect; caller validates +>= 1+.
     # @param max_bytes [Integer] hard byte cap on collected content.
     # @param max_line_length [Integer] per-line truncation threshold.
-    # @return [Page] the windowed slice.
+    # @return [Page]
     # @raise [Unsupported] when no registry entry claims the content.
     # @raise [Error] when the matched extractor cannot parse it.
     def extract_paged(io, content_type: nil, offset: 1, limit: PAGE_DEFAULT_LIMIT,
                       max_bytes: PAGE_MAX_BYTES, max_line_length: PAGE_MAX_LINE_LENGTH)
       extractor = extractor_for(io, content_type)
       if extractor.respond_to?(:extract_lines)
-        # count_tail is a per-format economics call: once the window
-        # fills, counting the rest of a plain-text stream is a cheap
-        # sequential read (so the trailer can say "of N"), while for
-        # a PDF it would mean parsing every remaining page — exactly
-        # what extract_lines exists to avoid. Plugged-in extractors
-        # (pikuri-pdf's included) get the conservative default (stop
-        # early, total unknown).
+        # count_tail is a per-format economics call: once the window fills,
+        # counting the rest of a text stream is a cheap sequential read (so
+        # the trailer says "of N"), but for a PDF it would parse every
+        # remaining page — what extract_lines exists to avoid. Plugged-in
+        # extractors get the conservative default (stop early, total unknown).
         window(extractor.extract_lines(io),
                offset: offset, limit: limit, max_bytes: max_bytes,
                max_line_length: max_line_length, kind: extractor.kind,
@@ -227,11 +170,10 @@ module Pikuri
       end
     end
 
-    # Find the first registry entry claiming +io+'s content: read the
-    # leading {FileType::SAMPLE_BYTES} for the sniff, rewind, and ask
-    # each extractor in order.
+    # First registry entry claiming +io+'s content: read the leading
+    # {FileType::SAMPLE_BYTES} for the sniff, rewind, ask each in order.
     #
-    # @param io [IO, StringIO] seekable IO positioned at the start.
+    # @param io [IO, StringIO] seekable IO at the start.
     # @param content_type [String, nil]
     # @return [#extract] the matched extractor.
     # @raise [Unsupported] when nothing matches.
@@ -244,15 +186,12 @@ module Pikuri
     end
     private_class_method :extractor_for
 
-    # Collect a {Page} window out of +lines+ (an Array or a lazy
-    # Enumerator of already-+chomp+ed lines). +known_total+ is the
-    # full line count when the caller extracted everything up front
-    # (Array case), +nil+ for a lazy stream — then +total_lines+ is
-    # exact only if the iteration reached EOF: +count_tail+ keeps
-    # the loop counting (without collecting) past the line limit
-    # when consuming the rest of the stream is cheap; without it the
-    # loop breaks and leaves the total unknown. The byte cap always
-    # aborts the count.
+    # Collect a {Page} window from +lines+ (Array or lazy Enumerator of
+    # +chomp+ed lines). +known_total+ is the full count when extracted up
+    # front (Array), +nil+ for a lazy stream — then +total_lines+ is exact
+    # only if iteration reached EOF: +count_tail+ keeps counting past the
+    # limit when the tail is cheap, else the loop breaks and the total stays
+    # unknown. The byte cap always aborts the count.
     #
     # @param lines [Enumerable<String>]
     # @param known_total [Integer, nil]

data/lib/pikuri/file_type.rb

@@ -2,66 +2,37 @@
 
 module Pikuri
   # Magic-byte content sniffing, plus the path-aware front over the
-  # {Extractor} registry. Two responsibilities:
+  # {Extractor} registry. Two core responsibilities:
   #
-  # * {.detect_mime} — recognise a file from its leading bytes. Returns
-  #   a MIME String for formats pikuri knows how to handle specially
-  #   ({+application/pdf+}, the four image formats), or +nil+ for
-  #   "unrecognised — could be text, could be opaque binary; caller
-  #   decides".
-  # * {.binary?} — heuristic text-vs-binary classifier. Independent of
-  #   {.detect_mime}: a file can be both recognised (e.g. PDF) *and*
-  #   binary. {.detect_mime} tells you what the bytes are;
-  #   {.binary?} tells you whether they're safe to render as text.
+  # * {.detect_mime} — recognise a file from its leading bytes. Returns a
+  #   MIME String for formats pikuri handles specially (+application/pdf+,
+  #   the four image formats), or +nil+ for "unrecognised".
+  # * {.binary?} — text-vs-binary heuristic, independent of {.detect_mime}
+  #   (a file can be both recognised, e.g. PDF, *and* binary): what the
+  #   bytes are vs. whether they're safe to render as text.
   #
-  # On top of those sit the two +Pathname+ conveniences,
-  # {.read_as_text} (whole document, the {Pikuri::VectorDb} indexer's
-  # shape) and {.read_as_text_paged} (line-windowed, the Read tools'
-  # shape). Both are thin wrappers: they own the *path-level* refusals
-  # (missing file, directory, image) and the exception mapping, then
-  # hand the opened IO to {Extractor.extract} /
-  # {Extractor.extract_paged} — which format the bytes are and how
-  # they become text is entirely the registry's business, so a gem
-  # plugging a new extractor in extends these wrappers for free.
+  # On top sit two +Pathname+ conveniences, {.read_as_text} (whole document)
+  # and {.read_as_text_paged} (line-windowed). Both own the *path-level*
+  # refusals (missing file, directory, image) and the exception mapping,
+  # then hand the IO to {Extractor}; format-to-text is the registry's
+  # business, so a new extractor extends these wrappers for free.
   #
-  # {.detect_mime} and {.binary?} accept either a +String+ of bytes
-  # (sample taken by the caller) or a +Pathname+ — when given a path,
-  # the module opens the file in binary mode and reads {SAMPLE_BYTES}
-  # for the sniff itself. The Pathname form is the convenience path;
-  # the bytes form is for callers that already have the sample or are
-  # calling both methods on the same file and want to avoid a second
-  # open.
-  #
-  # == Why a separate module
-  #
-  # Without this module, magic-byte tables and the binary heuristic
-  # ended up scattered through whichever tool needed them — first PDF
-  # in {Workspace::Read}, then images alongside it, then a copy of
-  # {.binary?} reached for by {Workspace::Edit}. Collecting the
-  # detection logic here lets {Read} focus on routing
-  # (mime-to-formatter), {Edit} drop its cross-tool reach, and new
-  # tools share one set of magic-byte truths.
+  # {.detect_mime}/{.binary?} take either a +String+ of bytes (caller
+  # sampled) or a +Pathname+ (opened binary, {SAMPLE_BYTES} read).
   #
   # == Deliberate non-goals
   #
-  # * *Not a full MIME database.* The set grows when a pikuri tool
-  #   needs a new format, not speculatively. Keeps the "audit in an
-  #   evening" ceiling honest.
-  # * *No path / extension fallback.* Extensions lie (a renamed
-  #   +.png+ → opaque garbage); magic-byte detection on the actual
-  #   content is the source of truth. Callers that need
-  #   extension-based behaviour can layer it themselves.
-  # * *No convenience predicates* like +image?+ / +pdf?+. Callers do
-  #   +mime == 'application/pdf'+ or +mime&.start_with?('image/')+ —
-  #   one extra character, zero added API surface.
+  # * *Not a full MIME database.* The set grows when a tool needs a format,
+  #   not speculatively.
+  # * *No path/extension fallback.* Extensions lie (a renamed +.png+);
+  #   magic-byte detection on content is the source of truth.
+  # * *No +image?+/+pdf?+ predicates.* Callers do +mime == 'application/pdf'+
+  #   — one character, zero added surface.
   module FileType
     module_function
 
-    # @return [Integer] recommended number of bytes to sample for
-    #   {.detect_mime} and {.binary?}. Big enough to catch every
-    #   prefix pikuri sniffs today (the largest is WebP's 12-byte
-    #   container header) with comfortable slack; small enough that
-    #   reading it off any reasonable filesystem is effectively free.
+    # @return [Integer] bytes to sample for {.detect_mime}/{.binary?}. Covers
+    #   every prefix sniffed today (largest: WebP's 12-byte header) with slack.
     SAMPLE_BYTES = 4096
 
     # @return [Float] fraction of the sample that may be non-printable
@@ -85,15 +56,13 @@ module Pikuri
     #   with this five-byte ASCII sequence per ISO 32000-1 §7.5.2.
     PDF_MAGIC = '%PDF-'
 
-    # Recognise a file from its leading bytes. Returns the MIME type
-    # as a String for formats pikuri handles specially, or +nil+ for
-    # "unrecognised" — callers interpret +nil+ themselves (text,
+    # Recognise a file from its leading bytes. Returns the MIME String for
+    # formats pikuri handles specially, or +nil+ for "unrecognised" (text,
     # opaque binary, ...).
     #
-    # @param input [String, Pathname] the bytes to inspect, or a
-    #   +Pathname+ that this method opens in binary mode and reads up
-    #   to {SAMPLE_BYTES} from. Caller is responsible for verifying the
-    #   path exists; missing-file errors propagate as +Errno::ENOENT+.
+    # @param input [String, Pathname] bytes to inspect, or a +Pathname+
+    #   opened binary and read up to {SAMPLE_BYTES}. Caller verifies the path
+    #   exists; missing-file errors propagate as +Errno::ENOENT+.
     # @return [String, nil]
     def detect_mime(input)
       bytes = sample_of(input)
@@ -109,19 +78,13 @@ module Pikuri
       nil
     end
 
-    # Heuristic text-vs-binary classifier matching opencode's: any
-    # +NUL+ byte forces +true+; otherwise count bytes outside the
-    # printable +\t \n \v \f \r+ + ASCII-32..126 range and ratio
-    # against the sample size. UTF-8 continuation bytes (0x80-0xBF)
-    # are >127 so they sit outside the non-printable ranges and pass
-    # through unflagged, letting UTF-8 text read fine. An empty
-    # sample is treated as not-binary (callers reading an empty file
-    # take the empty-text path).
+    # Text-vs-binary heuristic (opencode's): any +NUL+ forces +true+; else
+    # ratio bytes outside printable (+\t\n\v\f\r+ + ASCII 32..126) against the
+    # sample. UTF-8 continuation bytes (0x80-0xBF) sit above 127 so they pass
+    # unflagged; an empty sample is not-binary.
     #
-    # @param input [String, Pathname] the bytes to inspect, or a
-    #   +Pathname+ that this method opens in binary mode and reads up
-    #   to {SAMPLE_BYTES} from. Caller is responsible for verifying
-    #   the path exists.
+    # @param input [String, Pathname] bytes, or a +Pathname+ opened binary
+    #   and read up to {SAMPLE_BYTES}. Caller verifies the path exists.
     # @return [Boolean]
     def binary?(input)
       bytes = sample_of(input)
@@ -136,37 +99,19 @@ module Pikuri
       non_printable.to_f / bytes.bytesize > BINARY_NONPRINTABLE_THRESHOLD
     end
 
-    # Read +path+ and return its content as plain UTF-8 text, routed
-    # through the {Extractor} registry: anything
-    # unrecognised-but-textual passes through verbatim
-    # ({Extractor::Passthrough}); with pikuri-pdf registered, PDFs
-    # are extracted with +"--- Page N ---"+ markers (a scanned-image
-    # PDF with no extractable text comes back as the empty String, a
-    # deliberate silent skip callers detect by length if they care).
-    #
-    # Refusal cases — all raise rather than returning a sentinel
-    # because the callers are internal pikuri code, not an LLM
-    # tool. The LLM-facing +Workspace::Read+ does its own routing and
-    # returns "Error: ..." observations; that's a separate concern.
-    #
-    # * Path doesn't exist → +Errno::ENOENT+.
-    # * Path is a directory → +ArgumentError+.
-    # * Image (PNG / JPEG / GIF / WebP per {.detect_mime}) →
-    #   +ArgumentError+; images aren't text.
-    # * Content no extractor claims (opaque binary) →
-    #   +ArgumentError+, mapped from {Extractor::Unsupported}.
-    # * Extraction failure (malformed PDF, ...) → +RuntimeError+ with
-    #   the path included, mapped from {Extractor::Error} so callers
-    #   don't need to know any extractor's exception hierarchy.
+    # Read +path+ as plain UTF-8 text through the {Extractor} registry:
+    # unrecognised-but-textual passes through verbatim; with pikuri-pdf
+    # registered, PDFs get +"--- Page N ---"+ markers (a scanned-image PDF
+    # comes back empty — a deliberate silent skip callers detect by length).
+    # Refusals raise (callers are internal pikuri code, not an LLM tool).
     #
     # @param path [Pathname] file to read.
-    # @return [String] UTF-8 text. May be empty (empty text file, or
-    #   scanned-image PDF).
-    # @raise [ArgumentError] if +path+ isn't a +Pathname+, points at
-    #   a directory, is an image, or is binary.
+    # @return [String] UTF-8 text; may be empty (empty file, scanned PDF).
+    # @raise [ArgumentError] if +path+ isn't a +Pathname+, is a directory, an
+    #   image, or opaque binary (the last mapped from {Extractor::Unsupported}).
     # @raise [Errno::ENOENT] if +path+ doesn't exist.
-    # @raise [RuntimeError] on an extraction failure (malformed /
-    #   unsupported PDF, ...).
+    # @raise [RuntimeError] on an extraction failure (mapped from
+    #   {Extractor::Error}, path included).
     def read_as_text(path)
       mime = guard_extractable(path)
       path.open('rb') { |io| Extractor.extract(io, content_type: mime) }
@@ -176,29 +121,21 @@ module Pikuri
       raise "Cannot extract text from #{path}: #{e.message}"
     end
 
-    # Extract +path+ and return a windowed {Extractor::Page}: the
-    # lines from +offset+ (1-indexed) up to +limit+ of them, stopping
-    # early if +max_bytes+ is reached, with over-long lines truncated
-    # at +max_line_length+. Same routing and refusal contract as
-    # {.read_as_text}; the windowing semantics (including the lazy
-    # +extract_lines+ consumption that stops parsing once the window
-    # fills) are {Extractor.extract_paged}'s.
-    # The LLM-facing callers map the exceptions into +"Error: ..."+
-    # observations themselves.
+    # Extract +path+ into a windowed {Extractor::Page} (lines from +offset+ up
+    # to +limit+, byte-capped, long lines truncated). Same routing and refusal
+    # contract as {.read_as_text}; windowing semantics are
+    # {Extractor.extract_paged}'s.
     #
     # @param path [Pathname] file to read.
-    # @param offset [Integer] 1-indexed first line to include. The
-    #   caller is responsible for validating +offset >= 1+.
-    # @param limit [Integer] maximum lines to collect. Caller
-    #   validates +limit >= 1+.
+    # @param offset [Integer] 1-indexed first line; caller validates +>= 1+.
+    # @param limit [Integer] max lines to collect; caller validates +>= 1+.
     # @param max_bytes [Integer] hard byte cap on collected content.
     # @param max_line_length [Integer] per-line truncation threshold.
-    # @return [Extractor::Page] the windowed slice.
-    # @raise [ArgumentError] if +path+ isn't a +Pathname+, is a
-    #   directory, an image, or binary.
+    # @return [Extractor::Page]
+    # @raise [ArgumentError] if +path+ isn't a +Pathname+, is a directory, an
+    #   image, or binary.
     # @raise [Errno::ENOENT] if +path+ doesn't exist.
-    # @raise [RuntimeError] on an extraction failure (malformed /
-    #   unsupported PDF, ...).
+    # @raise [RuntimeError] on an extraction failure.
     def read_as_text_paged(path, offset: 1, limit: Extractor::PAGE_DEFAULT_LIMIT,
                            max_bytes: Extractor::PAGE_MAX_BYTES,
                            max_line_length: Extractor::PAGE_MAX_LINE_LENGTH)
@@ -213,16 +150,13 @@ module Pikuri
       raise "Cannot extract text from #{path}: #{e.message}"
     end
 
-    # The shared path-level refusals for {.read_as_text} /
-    # {.read_as_text_paged}: must be an existing non-directory
-    # +Pathname+, and not an image (images are data for a vision
-    # model, never text). Returns the {.detect_mime} result so the
-    # caller can pass it to the {Extractor} as the content-type hint.
+    # Shared path-level refusals for the read-as-text pair: an existing
+    # non-directory +Pathname+, not an image (images are vision-model data).
+    # Returns the {.detect_mime} result as the content-type hint.
     #
     # @param path [Pathname]
     # @return [String, nil] the sniffed MIME type.
-    # @raise [ArgumentError] on a non-Pathname, a directory, or an
-    #   image.
+    # @raise [ArgumentError] on a non-Pathname, a directory, or an image.
     # @raise [Errno::ENOENT] if +path+ doesn't exist.
     def guard_extractable(path)
       raise ArgumentError, "expected Pathname, got #{path.class}" unless path.is_a?(Pathname)
@@ -236,20 +170,14 @@ module Pikuri
     end
     private_class_method :guard_extractable
 
-    # Coerce an +input+ argument into a bytes String for the sniffs.
-    # +String+ inputs are returned as-is (caller already sampled);
-    # +Pathname+ inputs are opened in binary mode and up to
-    # {SAMPLE_BYTES} are read off the front. Empty files come back
-    # as an empty String — {.binary?} treats that as not-binary and
-    # {.detect_mime} returns +nil+ for it, which is what the
-    # empty-text path wants.
+    # Coerce +input+ into a bytes String for the sniffs: a +String+ is
+    # returned as-is, a +Pathname+ is opened binary and up to {SAMPLE_BYTES}
+    # read. Empty files → empty String (not-binary, +nil+ mime).
     #
     # @param input [String, Pathname]
-    # @return [String] raw bytes (ASCII-8BIT encoding for the path
-    #   case; whatever the caller passed for the bytes case)
-    # @raise [ArgumentError] if +input+ is neither a +String+ nor a
-    #   +Pathname+ — refuses to guess, since a bare String could be
-    #   either a path or actual bytes.
+    # @return [String] raw bytes.
+    # @raise [ArgumentError] if neither — refuses to guess, since a bare
+    #   String could be a path or actual bytes.
     def sample_of(input)
       case input
       when String