docbook 0.1.0

data/docs/features/xinclude/index.adoc

@@ -0,0 +1,119 @@
+---
+layout: default
+title: XInclude Resolution
+parent: Features
+nav_order: 2
+has_children: true
+---
+
+# XInclude Resolution
+
+The DocBook gem provides custom XInclude resolution for composing documents from multiple XML files. It supports the standard XInclude namespace (`http://www.w3.org/2001/XInclude`) with both XML and text parsing modes, plus advanced text filtering via `fragid` schemes.
+
+## Why a Custom Resolver?
+
+The gem implements its own XInclude resolution rather than using Nokogiri's built-in `do_xinclude`. This provides:
+
+- **Iterative resolution** -- Handles nested includes (file A includes file B which includes file C) by re-scanning the document after each resolution
+- **fragid support** -- Text includes support three filtering schemes (`search=`, `line=`, `char=`) for extracting portions of a file
+- **Robust error handling** -- Failed includes are silently skipped, allowing partial document builds
+
+## Namespace
+
+XInclude elements use the standard namespace:
+
+[source,xml]
+----
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <chapter>
+    <xi:include href="intro.xml"/>
+  </chapter>
+</book>
+----
+
+The resolver finds `xi:include` elements using XPath:
+
+[source,ruby]
+----
+XINCLUDE_NS = "http://www.w3.org/2001/XInclude"
+includes = doc.xpath("//xi:include", "xi" => XINCLUDE_NS)
+----
+
+## Iterative Resolution
+
+The resolver processes includes iteratively rather than recursively. After each include is resolved, the document is re-scanned for new `xi:include` elements:
+
+[source,ruby]
+----
+def self.resolve(doc)
+  loop do
+    includes = doc.xpath("//xi:include", "xi" => XINCLUDE_NS)
+    break if includes.empty?
+
+    resolved_any = false
+    includes.each do |inc|
+      # ... resolve each include ...
+      resolved_any = true
+      break  # re-scan after each resolve (tree changes)
+    end
+    break unless resolved_any
+  end
+  doc
+end
+----
+
+This approach handles:
+
+- **Nested includes** -- A resolved file may contain its own `xi:include` elements
+- **Self-referential text includes** -- A file can include portions of itself using `fragid`
+- **Document tree mutations** -- The break-and-rescan pattern avoids stale XPath results
+
+## Parse Modes
+
+Two parse modes are supported, controlled by the `parse` attribute on `xi:include`:
+
+[cols="1,3,3"]
+|===
+| Mode | Attribute | Description
+
+| XML | `parse="xml"` (default) | Reads the referenced file, parses it as XML, and replaces the `xi:include` element with the root element of the included document
+| Text | `parse="text"` | Reads the referenced file as plain text and replaces the `xi:include` element with a text node
+|===
+
+## Base Path Resolution
+
+Include paths are resolved relative to the including document's URL. When using `resolve_string`, the base path is set from the input file:
+
+[source,ruby]
+----
+def self.resolve_string(xml_string, base_path: nil)
+  file_uri = "file://#{File.expand_path(base_path)}"
+  doc = Nokogiri::XML(xml_string, file_uri)
+  resolve(doc)
+end
+----
+
+## CLI Integration
+
+XInclude resolution is enabled by default in the `to-html` command and disabled by default in the `format` command:
+
+[source,bash]
+----
+# XInclude enabled (default for to-html)
+docbook to-html book.xml -o output.html
+
+# Explicitly disable XInclude
+docbook to-html book.xml -o output.html --no-xinclude
+
+# XInclude disabled (default for format)
+docbook format book.xml
+
+# Enable XInclude for format
+docbook format book.xml --xinclude
+----
+
+## Child Pages
+
+- <<xml-includes,XML Includes>> -- Standard XML inclusion with namespace handling
+- <<text-includes,Text Includes>> -- Plain text inclusion for source code and verbatim content
+- <<fragid-schemes,fragid Schemes>> -- Text filtering with `search=`, `line=`, and `char=` schemes

data/docs/features/xinclude/text-includes.adoc

@@ -0,0 +1,123 @@
+---
+layout: default
+title: Text Includes
+parent: XInclude Resolution
+grand_parent: Features
+nav_order: 2
+---
+
+# Text Includes
+
+Text includes (`parse="text"`) read the referenced file as plain text and insert its content as a text node. This is the standard way to include source code, configuration files, or other verbatim text content in DocBook documents.
+
+## Basic Usage
+
+[source,xml]
+----
+<chapter xmlns:xi="http://www.w3.org/2001/XInclude">
+  <title>Installation</title>
+
+  <para>Run the following command to install the gem:</para>
+
+  <screen><xi:include href="examples/install.sh" parse="text"/></screen>
+
+  <para>Then create a configuration file:</para>
+
+  <programlisting language="yaml"><xi:include href="examples/config.yaml" parse="text"/></programlisting>
+</chapter>
+----
+
+The file content is inserted verbatim as a text node, replacing the `xi:include` element. No XML parsing is performed on the included content.
+
+## How It Works
+
+[source,ruby]
+----
+def self.resolve_text_include(doc, inc, file_path, fragid)
+  content = File.read(file_path)
+  if fragid && !fragid.empty?
+    filtered = apply_fragid(content, fragid)
+    content = filtered if filtered
+  end
+  text_node = doc.create_text_node(content)
+  inc.replace(text_node)
+end
+----
+
+The process is straightforward:
+
+. **Read the file** -- The file is read as a plain string with `File.read`
+. **Apply fragid filter** -- If a `fragid` attribute is present, the content is filtered (see <<fragid-schemes,fragid Schemes>>)
+. **Create text node** -- The content is wrapped in a Nokogiri text node
+. **Replace the include** -- The `xi:include` element is replaced with the text node
+
+## Without fragid
+
+When no `fragid` attribute is present, the entire file content is included:
+
+[source,xml]
+----
+<!-- Include an entire source file -->
+<programlisting language="ruby"><xi:include href="lib/docbook.rb" parse="text"/></programlisting>
+----
+
+[source,xml]
+----
+<!-- Include a shell script -->
+<screen><xi:include href="scripts/setup.sh" parse="text"/></screen>
+----
+
+[source,xml]
+----
+<!-- Include a configuration file -->
+<programlisting language="json"><xi:include href="config.json" parse="text"/></programlisting>
+----
+
+## Common Use Cases
+
+### Source Code Listings
+
+Text includes are most commonly used to embed source code files directly in documentation, ensuring the documented code is always synchronized with the actual codebase:
+
+[source,xml]
+----
+<section>
+  <title>The Html Output Class</title>
+  <para>The main output class handles HTML generation:</para>
+  <programlisting language="ruby"><xi:include href="lib/docbook/output/html.rb" parse="text"/></programlisting>
+</section>
+----
+
+### Configuration Examples
+
+[source,xml]
+----
+<section>
+  <title>Default Configuration</title>
+  <programlisting language="yaml"><xi:include href="config/default.yaml" parse="text"/></programlisting>
+</section>
+----
+
+### Log Output and Results
+
+[source,xml]
+----
+<section>
+  <title>Expected Output</title>
+  <screen><xi:include href="test/fixtures/expected-output.txt" parse="text"/></screen>
+</section>
+----
+
+## Combined with fragid
+
+Text includes become significantly more powerful when combined with the `fragid` attribute, which allows extracting specific portions of a file rather than the entire content. This is particularly useful for:
+
+- Extracting specific functions from source files
+- Including only relevant lines from large files
+- Showing a specific section between two markers
+
+See <<fragid-schemes,fragid Schemes>> for detailed documentation of the three filtering schemes:
+
+- `search=` -- Pattern-based extraction (literal or regex)
+- `line=` -- Line number range extraction (RFC 5147)
+- `char=` -- Character offset extraction (RFC 5147)

data/docs/features/xinclude/xml-includes.adoc

@@ -0,0 +1,167 @@
+---
+layout: default
+title: XML Includes
+parent: XInclude Resolution
+grand_parent: Features
+nav_order: 1
+---
+
+# XML Includes
+
+XML includes (`parse="xml"`) are the default XInclude mode. The referenced file is read, parsed as XML, and its root element replaces the `xi:include` element in the including document.
+
+## Basic Usage
+
+The simplest form includes an entire XML file:
+
+[source,xml]
+----
+<!-- book.xml -->
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <title>My Book</title>
+  <xi:include href="chapters/introduction.xml"/>
+  <xi:include href="chapters/advanced.xml"/>
+</book>
+----
+
+[source,xml]
+----
+<!-- chapters/introduction.xml -->
+<chapter xml:id="intro">
+  <title>Introduction</title>
+  <para>This is the introduction chapter.</para>
+</chapter>
+----
+
+After resolution, the `xi:include` element is replaced with the root `<chapter>` element:
+
+[source,xml]
+----
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <title>My Book</title>
+  <chapter xml:id="intro">
+    <title>Introduction</title>
+    <para>This is the introduction chapter.</para>
+  </chapter>
+  <xi:include href="chapters/advanced.xml"/>
+</book>
+----
+
+Note that only the first include is resolved per iteration. The document is re-scanned and the second include is resolved in the next pass.
+
+## How It Works
+
+The resolution process:
+
+. **Read the file** -- The referenced file is read from disk
+. **Parse as XML** -- The file content is parsed with Nokogiri, using the file's full path as the base URL
+. **Handle namespaces** -- If the included root element has a namespace, it is registered on the main document's root element if not already present
+. **Replace the include** -- The `xi:include` element is replaced with the root element of the included document
+
+[source,ruby]
+----
+def self.resolve_xml_include(doc, inc, file_path)
+  included_xml = File.read(file_path)
+  included_doc = Nokogiri::XML(included_xml, "file://#{File.expand_path(file_path)}")
+
+  root = included_doc.root
+  # Ensure namespace is declared in target document
+  if root.namespace
+    ns = root.namespace
+    existing = doc.root.namespace_definitions.find { |n| n.href == ns.href }
+    doc.root.add_namespace_definition(ns.prefix, ns.href) unless existing
+  end
+
+  # Replace the xi:include with the root element itself
+  inc.replace(root.dup)
+end
+----
+
+## Namespace Handling
+
+When the included document uses namespaces (e.g., DocBook namespace `xmlns:db="http://docbook.org/ns/docbook"`), the resolver ensures the namespace is registered on the target document's root element. This prevents namespace conflicts and ensures the merged document is well-formed.
+
+## Base Path Resolution
+
+File paths in `href` attributes are resolved relative to the including document's location:
+
+[source,xml]
+----
+<!-- /project/book.xml -->
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <!-- Resolves to /project/chapters/ch01.xml -->
+  <xi:include href="chapters/ch01.xml"/>
+
+  <!-- Resolves to /project/shared/legal.xml -->
+  <xi:include href="shared/legal.xml"/>
+</book>
+----
+
+The base path is extracted from the document's URL:
+
+[source,ruby]
+----
+def self.base_dir_for(document)
+  url = document.url
+  return nil unless url
+  File.dirname(url.sub(%r{^file://}, ""))
+end
+----
+
+## Nested Includes
+
+Because the resolver works iteratively, nested includes are handled automatically. When a file is included that itself contains `xi:include` elements, those are resolved in subsequent iterations:
+
+[source,xml]
+----
+<!-- book.xml -->
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <xi:include href="part1.xml"/>
+</book>
+
+<!-- part1.xml -->
+<part>
+  <title>Part I</title>
+  <xi:include href="chapters/ch01.xml"/>
+  <xi:include href="chapters/ch02.xml"/>
+</part>
+
+<!-- chapters/ch01.xml -->
+<chapter xml:id="ch01">
+  <title>Chapter 1</title>
+  <para>Content here.</para>
+</chapter>
+----
+
+The resolution order is:
+
+. `book.xml` includes `part1.xml` -- `xi:include` replaced with `<part>` element
+. Re-scan finds two new `xi:include` elements inside the included `<part>`
+. `part1.xml`'s first include (`ch01.xml`) is resolved
+. Re-scan finds one remaining `xi:include`
+. `part1.xml`'s second include (`ch02.xml`) is resolved
+. Re-scan finds no more `xi:include` elements -- done
+
+## Common Patterns
+
+**Book with per-chapter files:**
+
+[source,xml]
+----
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+  <xi:include href="frontmatter.xml"/>
+  <xi:include href="part1.xml"/>
+  <xi:include href="part2.xml"/>
+  <xi:include href="appendices.xml"/>
+</book>
+----
+
+**Shared boilerplate:**
+
+[source,xml]
+----
+<!-- Including a common legal notice in multiple documents -->
+<preface>
+  <xi:include href="../shared/legal-notice.xml"/>
+</preface>
+----

data/docs/getting-started/index.adoc

@@ -0,0 +1,50 @@
+---
+layout: default
+title: Getting Started
+nav_order: 2
+has_children: true
+---
+
+== Getting Started
+
+Welcome to the *Metanorma DocBook* gem -- a Ruby library for parsing *DocBook 5 XML* documents and converting them to rich, interactive HTML output.
+
+DocBook 5 is an OASIS standard XML vocabulary designed for marking up technical documentation, including books, articles, reference pages, and more. The `docbook` gem provides a complete toolkit for working with DocBook XML files: parsing them into Ruby object models, resolving cross-references and XIncludes, and producing self-contained HTML documents or directory-based output ready for deployment.
+
+The gem is built on top of https://github.com/lutaml/lutaml-model[`lutaml-model`], which provides a type-safe, declarative way to define XML-to-Ruby mappings. Every DocBook element -- from `<book>` down to inline elements like `<emphasis>` and `<xref>` -- is represented as a first-class Ruby object with typed attributes.
+
+=== Prerequisites
+
+Before you begin, make sure you have the following:
+
+* *Ruby >= 3.2* installed on your system
+* *Bundler* (optional, but recommended for project integration)
+* Basic familiarity with XML and the DocBook 5 vocabulary
+
+=== What You Will Learn
+
+This section covers everything you need to start using the gem:
+
+* *<<installation,Installation>>* -- Install the gem via RubyGems or Bundler, and verify your setup.
+* *<<quick-start,Quick Start>>* -- Run your first validation, format, and HTML conversion from the command line.
+
+=== About DocBook 5 XML
+
+DocBook 5 is a semantic markup language for technical documentation. Unlike presentation-oriented formats, DocBook describes the *structure* and *meaning* of your content. A DocBook `<book>` contains chapters, sections, paragraphs, code listings, figures, tables, cross-references, and much more. This separation of content from presentation makes DocBook ideal for single-source publishing pipelines.
+
+The `docbook` gem supports 39 root element types, including the most commonly used: `<article>`, `<book>`, `<chapter>`, `<appendix>`, `<section>`, `<preface>`, `<part>`, and `<reference>`.
+
+=== What the Gem Provides
+
+[cols="2,5"]
+|===
+| Feature | Description
+
+| *CLI* | A Thor-based command-line interface with four commands: `validate`, `format`, `to-html`, and `roundtrip`.
+| *Ruby API* | Programmatic access to parsing, XInclude resolution, cross-reference resolution, and HTML generation.
+| *HTML Output* | Two output modes: a self-contained single HTML file (with embedded Base64 images) and a directory mode (with a Vue.js-powered reading experience).
+| *XInclude Support* | Full resolution of XInclude elements including nested includes, text includes, and `text+fragid` extensions.
+| *Cross-Reference Resolution* | Automatic resolution of `xml:id` references to section titles and other targets.
+|===
+
+Dive into <<installation>> to get the gem installed, then head to <<quick-start>> for hands-on examples.

data/docs/getting-started/installation.adoc

@@ -0,0 +1,113 @@
+---
+layout: default
+title: Installation
+parent: Getting Started
+nav_order: 1
+---
+
+== Installation
+
+Install the Metanorma DocBook gem to start parsing and converting DocBook XML files.
+
+=== Requirements
+
+The gem has the following runtime requirements:
+
+[cols="1,4"]
+|===
+| Requirement | Details
+
+| *Ruby* | Version 3.2 or later (`>= 3.2.0`)
+| *Nokogiri* | XML parsing library (installed automatically as a dependency)
+| *lutaml-model* | Declarative XML-to-Ruby object mapping (`~> 0.8.0`, installed automatically)
+| *Thor* | CLI framework (installed automatically)
+| *Liquid* | Template engine for HTML generation (`~> 5.0`, installed automatically)
+| *Marcel* | MIME type detection for image embedding (installed automatically)
+|===
+
+=== Install with RubyGems
+
+The simplest way to install the gem is directly from the command line:
+
+[source,bash]
+----
+$ gem install docbook
+----
+
+This installs the `docbook` command-line tool and all required dependencies.
+
+=== Install with Bundler
+
+For Ruby projects, add the gem to your `Gemfile`:
+
+[source,ruby]
+----
+# Gemfile
+gem "docbook"
+----
+
+Then run Bundler to install:
+
+[source,bash]
+----
+$ bundle install
+----
+
+If you want to track the latest development version from the repository:
+
+[source,ruby]
+----
+# Gemfile
+gem "docbook", github: "metanorma/docbook", branch: "main"
+----
+
+=== Verify the Installation
+
+After installing, confirm that the `docbook` CLI is available:
+
+[source,bash]
+----
+$ docbook --help
+----
+
+You should see output listing the available commands:
+
+[source]
+----
+Commands:
+  docbook format INPUT       # Format/prettify DocBook XML
+  docbook help [COMMAND]     # Describe available commands or one specific command
+  docbook roundtrip INPUT... # Round-trip test DocBook XML files
+  docbook to-html INPUT      # Convert DocBook XML to HTML
+  docbook validate INPUT     # Validate DocBook XML well-formedness
+----
+
+You can also check the gem version from Ruby:
+
+[source,ruby]
+----
+require "docbook"
+puts Docbook::VERSION  # => "0.1.0"
+----
+
+=== Building from Source
+
+To build and install from the source repository:
+
+[source,bash]
+----
+$ git clone https://github.com/metanorma/docbook.git
+$ cd docbook
+$ bundle install
+$ rake install
+----
+
+This builds the gem from the current source and installs it locally. The `docbook` executable will be available in your path.
+
+=== Troubleshooting
+
+*Nokogiri compilation errors:* If Nokogiri fails to build native extensions, ensure you have the required system libraries. On macOS, `xcode-select --install` usually resolves this. On Ubuntu/Debian, install `libxml2-dev` and `libxslt1-dev`.
+
+*Ruby version mismatch:* The gem requires Ruby 3.2 or later. Check your version with `ruby --version`. Use a version manager like `rbenv` or `rvm` if you need to install a newer Ruby.
+
+Once installed, proceed to the <<quick-start>> page to start using the gem.