pdfmd-cli 3.24.19__tar.gz → 3.25.20__tar.gz

{pdfmd_cli-3.24.19/pdfmd_cli.egg-info → pdfmd_cli-3.25.20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pdfmd-cli
-Version: 3.24.19
+Version: 3.25.20
 Summary: One command from Markdown to a good-looking PDF: a Pandoc wrapper with smart defaults, project-file discovery and a multi-engine fallback chain
 Author: Ali Perdekhan
 License-Expression: MIT
@@ -40,7 +40,9 @@ Dynamic: license-file
 [Pandoc](https://pandoc.org) and fills in everything you would otherwise
 have to remember: sensible fonts and margins, the right Markdown dialect,
 your project's metadata/preamble/filter files, and a fallback chain across
-every PDF engine you have installed.
+every PDF engine you have installed. The same document also builds to a
+Word or OpenDocument file that follows its page setup, fonts and LaTeX (see
+[Word and OpenDocument output](#word-and-opendocument-output)).
 
 ```console
 $ pdfmd lecture
@@ -48,6 +50,11 @@ AUTO: READER TITLE MARGIN MONOFONT. Use --verbose to see in full
 OK    lecture.md
 ```
 
+In a folder with a single Markdown file you do not even need the name: `cd` into the folder and run `pdfmd`. It finds
+the file, its metadata, preamble and filters, and writes the PDF beside it (with several Markdown files it asks for
+one, or for `-b` to convert all of them or `-r` to join them). `pdfmd --help` is one short page; `pdfmd --help all`
+is the complete option list and `pdfmd --help tables` (or `source`, `history`, `output` ...) one subject at a time.
+
 <p align="center">
   <img src="https://raw.githubusercontent.com/aliperdehan/pdfmd/main/docs/lecture.png" width="560" alt="lecture.md rendered to PDF">
 </p>
@@ -76,6 +83,13 @@ turns that knowledge into defaults:
 - **It doesn't give up on the first engine.** If `lualatex` fails or isn't
   installed, it tries the next engine, then the next, and tells you why each
   one failed.
+- **It writes real Word files.** `pdfmd report -o report.docx` reads the front matter the PDF reads (paper size,
+  margins, fonts, line spacing) and writes it into the file's page setup and styles, so the Word file is laid out the
+  way the PDF is. With LaTeX installed, what only LaTeX can say in the document comes along: `\ce{...}` chemistry,
+  `\SI{...}{...}` units and equations become native, editable Word text and equations; a house style's own macros can be
+  given a Word recipe; what is left (tikz, chemfig, math Pandoc cannot convert) is drawn by LaTeX in the document's
+  own preamble and embedded as vector pictures. `pdfmd-options: {office: {latex: auto|images|off}}`, `--no-auto
+  officelatex` and `pdfmd --setup` switch it (see [Word and OpenDocument output](#word-and-opendocument-output)).
 - **It handles every script and emoji.** Text the main font cannot draw (Arabic,
   Hebrew, Chinese, Japanese, Korean, Greek, Cyrillic with Kazakh letters, Indic and
   more) is set run by run in an installed font for its script, emoji become colour
@@ -86,6 +100,17 @@ turns that knowledge into defaults:
   `pdfmd mingyun` finds `命運.md`.
 - **It reads PDFs too.** `pdfmd paper.pdf` restores a PDF's own embedded source or
   OCRs the pages to Markdown (see [PDF to Markdown](#pdf-to-markdown)).
+- **Tables can be data.** A `::: {.csv}` block puts a table's data in a CSV file or right in the document (any
+  delimiter, alignment and widths as attributes), and `pdfmd --extract-tables` turns the tables you already have into
+  such blocks after checking that Pandoc reads the same tables back (see [Tables straight from a CSV
+  file](#tables-straight-from-a-csv-file)). A table that is too wide gets its columns sized by their content.
+- **Long code wraps, SVG and web images work.** Code lines wrap at the margin in a LaTeX build, an SVG is converted for
+  LaTeX with whatever is installed, and an image from the web is fetched once and kept.
+- **Finished files are not rewritten.** A `.typ` file goes to Typst and an `.html` file to WeasyPrint or a browser, so
+  the page setup, styles and scripts survive; `--hybrid` makes the PDF open in LibreOffice as an editable document.
+- **It keeps the history.** `pdfmd --history` lists the backups, commits and compiles of a file, `--history-restore`
+  puts a version back (backing up the one it replaces), and the build history can live in a `.hst` file instead of the
+  document (see [Build stamps, history and snapshots](#build-stamps-history-and-snapshots)).
 - **It shows what it did.** Every automatic decision prints an `AUTO` line,
   `-v` shows the exact Pandoc command, and every default can be switched off.
 
@@ -156,8 +181,8 @@ Office files.
 <details>
 <summary>Without pipx</summary>
 
-`pdfmd.py` is a single file that needs Python 3.9+ (with `pdfmd_inkmd/`
-beside it for the no-Pandoc fallback). It also runs
+`pdfmd.py` needs Python 3.9+ and runs straight from a clone, with the folders beside it
+(`pdfmd_inkmd/` for the no-Pandoc fallback, `pdfmd_office/`, `pdfmd_setup/`, `pdfmd_unicode/`). It also runs
 directly, and `pyyaml`/`pypdf` are optional (features that need them are
 skipped with a warning):
 
@@ -199,7 +224,7 @@ the cache, ending with the command that fixes each missing piece:
 $ pdfmd --doctor
 ...
 Reading PDFs (PDF to Markdown):
-OK    batchocr 1.2.4
+OK    batchocr 1.2.5
 MISS  tesseract: not found
 MISS  poppler: not found
 
@@ -219,6 +244,27 @@ pdfmd --completion bash > ~/.local/share/bash-completion/completions/pdfmd
 pdfmd --completion fish > ~/.config/fish/completions/pdfmd.fish
 ```
 
+### An older Pandoc, and a machine with no network
+
+The oldest Pandoc `pdfmd` is tested with is **3.1.3** (what the claude.ai sandbox has); the whole test suite runs against
+it. An older one may work, and the first build with it says so once; `pdfmd --doctor` lists what a given Pandoc does
+without. With 3.1.3: Typst PDFs cannot show a picture `pdfmd` drew itself (before 3.2), a numbered equation in a Word file
+sits in a plain table instead of a borderless one (before 3.4), and `--extract-tables` leaves a table whose caption has
+a `{#tbl:id}` (before 3.8.2: Pandoc reads that as words, so the change could not be checked). A `pandoc-crossref` must be
+the release built for the Pandoc you run; one built for another is not run, with a warning, because it would put `??` where
+the numbers go.
+
+Where `pip install pdfmd-cli` cannot reach the network (a sandbox with no DNS that only takes an upload), two ways:
+
+```sh
+python3 scripts/build_zipapp.py          # on a machine with pdfmd's source: dist/pdfmd.pyz, 1.4 MB, PyYAML and pypdf inside
+python3 pdfmd.pyz report.md              # on the other: needs Python 3.9+, Pandoc and a PDF engine; unpacks itself once
+                                         # into ~/.cache/pdfmd/pyz/ because Pandoc has to open the Lua filters as files
+
+pip download pdfmd-cli -d wheelhouse     # or: a folder of wheels, carried over
+pip install --no-index --find-links wheelhouse pdfmd-cli
+```
+
 ### No Pandoc? The built-in fallback
 
 On a machine with no Pandoc, or Pandoc but no PDF engine, `pdfmd` does not
@@ -322,6 +368,28 @@ letters, no tones, vowels left out where the script leaves them out. Names that
 romanize alike (homophones) are listed as ambiguous, never picked between. The
 same spelling rules find a section by its heading (`doc#tyche`).
 
+### A small editor
+
+```sh
+pdfmd --install tui           # once: prompt_toolkit, about 1 MB
+pdfmd --edit lecture          # nano-like full screen; Ctrl-B saves and builds, Ctrl-P opens the PDF
+```
+
+`pdfmd --edit` (alpha) opens the Markdown file in a plain full-screen editor with the build one key away: Ctrl-S
+saves, Ctrl-B saves and builds (what pdfmd said stays at the bottom), Ctrl-P opens the result, Ctrl-W / F3 find,
+Ctrl-G go to a line, Ctrl-K / Ctrl-U cut and paste a line, Ctrl-Z / Ctrl-Y undo and redo, Ctrl-X leaves (asking once
+when there are unsaved changes); F1 lists them. Options given beside it (`pdfmd --edit lecture -e lualatex`) are used
+for the build, and a name that does not exist yet starts a new file. It is deliberately rough: a base for what comes later.
+
+### In VS Code
+
+`pdfmd --init-vscode` writes a `.vscode/tasks.json` here (build, build and open, watch, extract tables; Terminal > Run
+Build Task). The [`vscode/`](https://github.com/aliperdehan/pdfmd/tree/main/vscode) folder of the repository is a small
+extension over the same command line: build (`Cmd/Ctrl+Alt+B`), build and open, build as Word/HTML/Typst/slides, extract
+tables (all, or chosen ones), file history, compare and restore, restore the source a PDF carries (right-click it). Every
+command it runs is written to the *pdfmd* output panel. Install it with `npx @vscode/vsce package` and
+`code --install-extension pdfmd-vscode-*.vsix`.
+
 ### One section
 
 `pdfmd doc#onlyapart` builds just the section whose heading is "Only a Part"
@@ -445,8 +513,9 @@ LaTeX. For `.docx` and `.odt` pdfmd adds a filter that makes it native where it
 environment or macro behind each (`the tikzpicture environment`, `the macro \irspectrum`), without drawing or writing anything: a
 quick way to see what is left to give a recipe (see the profile below).
 
-`pdfmd-options: {office: {latex: off}}` or `--no-auto officelatex` skips all of it, `office: {labels: off}`
-skips the PDF build (numbers are then counted).
+`pdfmd-options: {office: {latex: auto}}` is the default; `latex: images` draws everything LaTeX says as a picture
+(formulas, `\ce` and units too, instead of making them native); `latex: off` or `--no-auto officelatex` skips all of it
+(Pandoc's own behaviour). `office: {labels: off}` skips the PDF build (numbers are then counted).
 
 The same filter serves the other outputs when LaTeX is the problem: **the soffice fallback** (the Word file above, then LibreOffice: `pdfmd -e soffice report.md` is the `.docx` as LibreOffice draws it) now
 keeps the page size, margins and fonts, shows equations and draws what LaTeX can; a **Typst or WeasyPrint run after LaTeX
@@ -570,12 +639,16 @@ style, a compound library, a shared bibliography and parts.
   The TeX distribution's own packages, fonts and Pandoc are not carried.
 - **Comments.** Notes to yourself do not travel: the attached source loses its
   `<!-- -->` comments, the LaTeX preamble's `%` comments, the text between a
-  `.bib`'s entries and a CSL file's XML comments. Choose per kind:
-  `strip-comments: [preamble, bibliography]` or `{markdown: false}`
-  (`--strip-comments-in`, `--keep-comments-in`). Code blocks, `\verb`, `\url` and
-  verbatim environments are never touched, and a `%` that closes a line keeps a
-  bare `%`, because it swallows the line break. YAML metadata is merged and
-  written again, so its `#` comments are always gone.
+  `.bib`'s entries, a CSL file's XML comments and the `#` comments of YAML
+  files. Choose per kind: `strip-comments: [preamble, bibliography]` or
+  `{markdown: false}` (`--strip-comments-in`, `--keep-comments-in`). Code
+  blocks, `\verb`, `\url` and verbatim environments are never touched, and a
+  `%` that closes a line keeps a bare `%`, because it swallows the line break.
+  With `strip-comments: false` (or `metadata` kept) the attached source gives
+  back your metadata files and the document's front matter byte for byte,
+  comments and quoting included; `--restore` of such a PDF is a full source
+  copy. The data files of `.csv` blocks (`tables/*.csv`) are always stored
+  with the source.
 - **Only the bibliography entries you cite** are attached (and what they
   cross-reference), not a shared file of hundreds; `attach-bibliography: all`
   keeps it whole, and `nocite: '@*'` keeps every entry.
@@ -832,12 +905,31 @@ Measured values:
   <img src="https://raw.githubusercontent.com/aliperdehan/pdfmd/main/docs/results.png" width="480" alt="A CSV file rendered as a table">
 </p>
 
+Or write the data in the document itself, in a code block so that nothing in it is read as Markdown:
+
+````markdown
+::: {.csv align="lrr"}
+```
+Compound,Mass (g),Price
+Water,18.015,1.20
+"Ethanol, dry",46.07,3.45
+```
+:::
+
+: Masses and prices {#tbl:m}
+````
+
 - Cells are read as Pandoc Markdown (`H~2~O`, `$x^2$`, `\ce{...}`, `[@key]`); `reader="gfm"` makes them plain GFM.
 - A caption is the line after the block (`: Caption {#tbl:id}`, or `Table: Caption`), or `caption="Caption"` inside the
   braces, with the identifier as `#tbl:id` there too. A caption in the attributes wins over a line after the block.
-- A relative `file=` is found beside the document (and along the resource path), wherever Pandoc runs.
-- The delimiter is detected from the extension (`.tsv` means tab), or set
-  with `delimiter=";"`.
+- A relative `file=` is found beside the document (and along the resource path), wherever Pandoc runs. With both
+  `file=` and data inside the block, the file is used.
+- The delimiter is `delimiter=";"` or its name (`semicolon`, `tab`, `pipe`, `space`, `colon`, `comma`); without one it
+  comes from the extension (`.tsv` means tab) or from the first line (whichever of `,` `;` tab `|` it holds most of).
+- Columns are aligned with `align="lcr"` (or `align="left,center,right"`), or by a second row in the data made of
+  `---`, `:--`, `:-:`, `--:` as in a pipe table; `align=` wins. If such a row is really data, say `separator="none"`.
+  `widths="5,1,1"` sets relative widths, and a second row with unequal dashes does too. Without either, pdfmd picks
+  widths that suit the contents.
 - The first row is the header unless you add `header="false"`.
 - Large files are capped at 10 rows × 7 columns, so a huge CSV can't
   silently fill 40 pages. `rows=all` or `cols=20` raise the cap. When a
@@ -845,6 +937,118 @@ Measured values:
 
 This works for every output format.
 
+#### Turning tables into CSV blocks, and back
+
+```
+pdfmd --extract-tables report.md      # every table becomes a .csv block reading tables/<name>.csv
+pdfmd --expand-tables report.md       # and back to ordinary tables
+```
+
+`--extract-tables` rewrites the document in place, after copying the old one to the backup folder, and says so. Pipe,
+simple, multiline and grid tables are all read (the ones inside code blocks are not tables). The files are named after
+the captions (`Masses and prices` gives `tables/masses-and-prices.csv`), or `table1`, `table2` by position;
+`--table-names a,b,c` names them, and a list that does not match the number of tables is an error that changes nothing.
+`--table-numbers 1,3-4` extracts just those tables (by position in the document) and leaves the rest; the names then go to
+the chosen ones. In a document in parts the data goes to the report's `tables/` folder and an unnamed table is named after
+its part (`discussion-table1.csv`).
+The caption, the alignment and the widths the document chose stay as attributes of the block, and before anything is
+written Pandoc reads the old and the new document and the tables must come out the same. A table Pandoc cannot write
+as CSV (a list inside a cell, a cell spanning rows, no header row) is left as it is, with the reason; so is every
+`.csv` block already in the document, and if the check fails the whole document is left untouched and the command exits
+with an error. A document with no front matter is built as `gfm`, which reads pipe tables only, so only those are
+touched there. `--tables-inline` puts the data
+inside the document (in a fenced block) instead of in files, `--tables-dir DIR` picks the folder, and `--dry-run` shows
+the change without making it. `--expand-tables` is the reverse, every row included.
+
+A `.csv` block that holds its data inside the document stays as it is, unless you pass `--extract-inline-csv`, which moves
+that data to a file as well. To keep one table or block as it is whatever you run, put `<!-- pdfmd: ignore -->` on the
+line above it (or under its caption); the comment survives comment stripping, because pdfmd's own comments are kept.
+A comment between a table and its caption would stop Pandoc from reading the caption as one, so there is no such
+position.
+
+### Code: wrapping and line numbers
+
+With a LaTeX engine a long line of code wraps at the margin instead of running off the page (a long path or option list
+breaks where the line is full), and `{wrap=false}` on one block, `--no-code-wrap` or `code-wrap: false` in
+`pdfmd-options` lets it run on. Line numbers are off until you ask for them:
+
+````markdown
+``` {.python .numberLines startFrom=10}
+def f(x):
+    return x * 2
+```
+````
+
+`--line-numbers` (or `pdfmd-options: {line-numbers: true}`, or the setting in `pdfmd --setup`) numbers every code block,
+`--line-numbers 5` numbers every fifth line, `{.noNumberLines}` leaves one block out, and `step=5` and `numbersep=8pt`
+adjust a single block.
+
+### Images: SVG and remote ones
+
+LaTeX reads PDF, PNG and JPEG, not SVG. In a LaTeX build pdfmd converts each SVG the document names, once, with
+whatever is installed (`rsvg-convert`, Inkscape, `cairosvg`, `svglib`, or LibreOffice, which most machines have), keeps
+the PDF by the file's content, and uses it where the document wrote the SVG: a Markdown image, a raw
+`\includegraphics{figure.svg}`, a raw `\includesvg{figure}`. A raw `\includegraphics` also gets `graphicx` loaded,
+which Pandoc only does for a Markdown image (a PNG in one used to stop the build with "undefined control sequence").
+A Markdown image from the web is fetched once and kept for a week, so the build does not need the network again, an SVG
+badge works, and a Pandoc built without HTTP support (Homebrew's is) no longer drops it; with no network there is one
+warning and the image's description stays. `--no-auto svg` and `--no-auto remoteimages` turn the two off, and
+`pdfmd --doctor` says which converter will be used.
+
+### Editable in LibreOffice, and one file that carries its data
+
+`--hybrid` makes the PDF a LibreOffice *hybrid PDF*: besides its pages it carries an OpenDocument copy of the document
+(`Original.odt`, or `.odp`, `.ods`), and when LibreOffice opens the PDF it opens that copy as a document you can edit,
+instead of the pages in Draw. An Office file (`pdfmd minutes.docx --hybrid`) is converted by LibreOffice, so the copy is
+the file itself; a Markdown file gets an `.odt` built by pdfmd with its metadata, so what LibreOffice shows is the
+Word-route rendering of the source (not the LaTeX pages). `pdfmd --restore report.pdf` writes the ODF file back out.
+`pdfmd-options: {hybrid: true}` or `pdfmd --setup` makes it the default.
+
+A single Markdown file can carry its data too: `pdfmd report.md --assemble-only --embed-metadata files -o report.one.md`
+folds in the images, CSV tables and included files the text points at (text as it is, anything else as base64), and the
+file builds on its own, in an empty folder; `pdfmd --unpack` writes them back byte for byte. It is left out of the default
+kinds because it can be big (`bundle-max-mb` caps it); for a PDF, `--bundle` stores the same files beside the source.
+
+### HTML, LaTeX, Typst and Word pieces inside Markdown
+
+Markdown can carry raw pieces: `<img>`, `<table>`, `<b>` and `<div>` (HTML), `\ce{...}` or a tikz picture (LaTeX), a
+`` ```{=typst} `` block, Word or OpenDocument XML. Pandoc keeps the ones written in the format it is writing and drops
+every other, silently, so a document that mixes them loses pieces in every build. `pdfmd-options.raw` (or `--raw`) says
+which syntaxes take part in each output family, and carries the foreign ones over:
+
+```yaml
+pdfmd-options:
+  raw:
+    tex:    [tex, html, typst]   # LaTeX builds: PDF through lualatex, xelatex ...
+    typst:  [typst, html, tex]   # Typst builds
+    html:   [html, tex, typst]   # HTML, EPUB, PDF through weasyprint and the other HTML engines (-e html)
+    office: [office, html]       # Word and OpenDocument, and PDF through soffice
+```
+
+```sh
+pdfmd report --raw               # everything for every family
+pdfmd report --raw-for html=tex,typst --raw-for tex=html      # or per family, on the command line
+```
+
+A family takes the syntaxes it lists, in this way: its own syntax stays as written; **HTML** is read by Pandoc's HTML
+reader, so an `<img>` becomes the target's image (an SVG then goes through the image conversion below), a `<table>` a
+real table, `<b>...</b>` bold text and a `<div>` around Markdown a division, even when the opening and the closing tag
+sit in separate pieces; **LaTeX** is read by Pandoc's LaTeX reader (what it does not know, a custom macro, stays LaTeX
+and is lost outside LaTeX; Word builds keep their own LaTeX route, see
+[Word and OpenDocument output](#word-and-opendocument-output)); **Typst** is drawn by `typst compile` as a cropped
+vector picture, a PDF for LaTeX, Typst and Word, an SVG for HTML. A syntax a family does not list is dropped, its own
+included, which is odd but a clear way to say "this output has no HTML in it". A family not mentioned is left to Pandoc
+as it always was, `raw: all` lists everything, and `--no-raw` turns it off for a run. Pictures are kept in pdfmd's cache by
+their content. (An HTML file that is not self-contained points at the cache; use `--self-contained` to carry them.)
+Not yet: LaTeX pieces Pandoc cannot read (tikz, chemfig) as pictures in HTML, HTML drawn as a picture, Word XML as
+anything but Word XML.
+
+**PDF images in HTML.** A browser does not show a PDF in `<img>`, an SVG it does. For an HTML build (a web page, an
+EPUB, a PDF through WeasyPrint) every image that is a PDF, `![fig](plot.pdf)` or `<img src="plot.pdf">`, is converted to
+an SVG with poppler's `pdftocairo` (or `mutool`, `pdf2svg`, `inkscape`) and stays vector; the first page is used and the
+SVG is kept by the PDF's content. It is on by default, since it only changes what would not have shown; `--no-auto
+pdfimages` turns it off. LaTeX and Typst read PDFs themselves, and Word has its own route.
+
 ### Not just Markdown
 
 ```sh
@@ -853,9 +1057,24 @@ pdfmd paper.tex          # compiled directly with a LaTeX engine: reruns until
                          # no .aux/.log clutter (--keep-aux keeps them)
 pdfmd minutes.docx       # Word/PowerPoint/Excel/ODF: converted by LibreOffice
 pdfmd analysis.qmd       # handed to Quarto, so code chunks actually run
-pdfmd page.html          # anything else Pandoc can read (give the extension)
+pdfmd page.html          # built by WeasyPrint, or a browser if the page runs scripts: no Pandoc
+pdfmd note.typ           # compiled by Typst itself: the page setup in the file is kept
+pdfmd notes.rst          # anything else Pandoc can read (give the extension)
 ```
 
+A finished `.typ` or `.html` file goes straight to its own engine, because Pandoc would read it into its own model
+and write it out again, and the page size, the styles and the scripts would not survive the trip. For a page that is
+WeasyPrint first (it is quiet and needs no browser); a page with `<script>` goes to Chrome, Chromium, Edge or Brave
+(headless, printing the page), as does one WeasyPrint cannot draw, and wkhtmltopdf, Prince, pagedjs-cli and LibreOffice
+come after. `-e edge` (or `weasyprint`, `chromium`...) picks one. As with an Office file, such a document gets none of
+pdfmd's defaults, since it is finished; `--apply-defaults` (also in `pdfmd --setup`) gives it the title, author, date,
+language, font, font size, paper and margin of its `metadata.yaml`, ahead of its own settings so those win. A `.tex`
+file that no LaTeX engine can compile is tried through Pandoc, with a warning.
+
+The reader comes from the file's extension, as in Pandoc, and a bare name is looked up as `<name>.md`, so give the
+extension for anything else. The automatic decisions about Markdown (`READER`, `TITLE`, `citeproc`, `crossref` below)
+apply to Markdown files only.
+
 **A plain-text file as Markdown (opt-in).** `pdfmd notes.txt --text-to-markdown` reads a `.txt` file the way a person typed it
 and guesses its structure: headings (capitals, `1.2 Title`, `====` underlines), bullet and numbered lists, tables from
 columns separated by runs of spaces, indented blocks as code, hard-wrapped lines joined into paragraphs. The words never
@@ -871,16 +1090,19 @@ Most of these print an `AUTO` line, and each can be switched off individually.
 
 | `AUTO` kind | When | What happens |
 |---|---|---|
-| `READER` | no YAML front matter | reads the file as GitHub-flavoured Markdown (content-sized table columns, relaxed blank-line rules) |
-| `TITLE` | no front matter, first line is `# Title` | that heading becomes the document title, and the remaining headings move up one level |
+| `READER` | a Markdown file with no YAML front matter | reads the file as GitHub-flavoured Markdown (content-sized table columns, relaxed blank-line rules) |
+| `TITLE` | a Markdown file with no front matter, first line is `# Title` | that heading becomes the document title, and the remaining headings move up one level |
 | `MARGIN` | no margin or geometry set anywhere | 1-inch margins instead of LaTeX's wide defaults |
 | `MAINFONT` | no `mainfont:` and no `-f` | STIX Two Text (Times New Roman if it isn't installed), or a serif that has the letters when the document is mostly in a script STIX Two Text lacks (Kazakh Cyrillic, say); retried with DejaVu Serif if a glyph is still missing. A font you name is never changed except by `fallback: document` |
 | `UNICODE` | text the main font cannot draw (Arabic, Han, Greek with accents, rare symbols...) | sets each run in an installed font for its script (see *Other scripts*), instead of printing boxes |
 | `MONOFONT` | the document contains code | JetBrains Mono for code (Menlo or another installed monospace font if it isn't installed) |
-| `tablewidth` | a wide pipe table | balances column widths so the table fits the page, and leaves narrow tables at their natural width |
+| `tablewidth` | a pipe table with plain `---` separators and long lines | picks the column widths that give the fewest lines within the text width (a column never narrower than its longest word), and gives a table that fits its natural width back; unequal dashes (`--|-----`) are kept, and `table-widths: keep` in `pdfmd-options` leaves every table as written |
+| `svg`, `remoteimages` | an SVG, or an image from the web, in a LaTeX build | the SVG is converted to a PDF (once), the remote image is fetched and kept (`--no-auto svg`, `--no-auto remoteimages`) |
+| `typstdirect`, `htmldirect` | a `.typ` or `.html` file | built by Typst / WeasyPrint or a browser, not through Pandoc (`--no-auto typstdirect` for the old route) |
+| `codewrap` | code blocks in a LaTeX build | long lines wrap at the margin (`--no-code-wrap` or `--no-auto codewrap` for the old behaviour) |
 | `YAML` / `TEX` / `LUA` | project files found | attaches `metadata.yaml`, `preamble.tex`, `<name>.lua` (see below) |
-| `citeproc` | `@key` / `[@key, p. 90]` citations | adds `--citeproc`, so citations and the reference list render from your `bibliography:` without any flag (`--no-citeproc` turns it off) |
-| `crossref` | `@fig:`/`@tbl:` references | adds the `pandoc-crossref` filter, ahead of citeproc |
+| `citeproc` | `@key` / `[@key, p. 90]` citations (as Pandoc reads them: `@key` in a code span or an e-mail address is none) | adds `--citeproc`, so citations and the reference list render from your `bibliography:` without any flag (`--no-citeproc` turns it off) |
+| `crossref` | `@fig:`/`@tbl:` references or `{#fig:...}` labels | adds the `pandoc-crossref` filter, ahead of citeproc |
 | `papersize` | `pagesize: a4` (a common typo) | converts it to Pandoc's real `papersize:` |
 
 `-v` explains each decision and prints the exact command it runs:
@@ -1093,15 +1315,16 @@ One shared metadata file can also be **symlinked** into many folders. A
 relative `bibliography: refs.bib` inside it then finds the `refs.bib`
 next to the file's real copy, so you never need an absolute path.
 
-## Build stamps and snapshots
+## Build stamps, history and snapshots
 
-Both are off by default.
+All are off by default.
 
 - **`--stamp`** keeps a `BUILD NOTES` HTML comment at the end of the `.md`
   up to date. It records when the document was compiled and with which
   `pdfmd` version, plus the versions of any LaTeX packages you name with
   `--stamp-packages`. The comment is invisible in the PDF. `--stamp-mode history` also keeps a list of past compiles.
-- **`--backup`** saves a timestamped copy of the source into `backup/`
+- **`--backup`** saves a timestamped copy of the source into `.backups/` (a hidden folder; an existing `backup/` is kept
+  using, and pdfmd says where the first copy went)
   after every successful build. A copy is skipped when nothing changed,
   and `keep: 30` limits how many are kept.
 
@@ -1110,16 +1333,45 @@ Both can be turned on for a whole folder from `metadata.yaml`:
 ```yaml
 pdfmd-options:
   stamp: true
-  backup: { dir: backup, keep: 30 }
+  backup: { dir: .backups, keep: 30 }
+```
+
+**History in a file instead of in the document.** `--stamp-store file` (or `stamp: {store: file}`) writes each compile
+to `NAME.hst` in the backup folder, and the document itself is not edited by a build. The file is one line per entry,
+newest first, so it reads in any editor, two of them join with `cat`, and `pdfmd --merge-history a.hst b.hst -o all.hst`
+keeps every entry once:
+
+```
+# pdfmd history of report.md -- one entry per line, newest first: WHEN | KIND | TEXT [| key=value ...]
+2026-10-09 11:22:01 | compiled | with nulabreport v1.27.7, pdfmd v3.25.7 | sha=3fa91c0d4b21 out=report.pdf
 ```
 
+`pdfmd --history-to-file report.md` moves the whole `BUILD NOTES` block that pdfmd owns into that file (the compiles as
+entries, anything written by hand inside it as one `note` entry), and `--history-to-notes` builds the block again (both
+back the document up first, `--dry-run` shows). A document may carry several
+`BUILD NOTES` blocks (yours, an AI agent's): pdfmd writes only into the one that already holds its own lines, or the
+only one there is, or starts one of its own when there are several and none is its; the rest is never touched, and a
+line `pdfmd: ignore` in a block keeps pdfmd out of it. The file travels with the document: `--embed-metadata` folds it
+in (`hst`), `--unpack` writes it out, and `--attach-source` stores it in the PDF for `--restore`.
+
+**Going back.** `pdfmd --history report.md` lists every version it can find, newest first: the backups (in `.backups`,
+and in an older `backup/`, `backups/`... beside the document, whatever naming each file has), the git commits that
+changed the file (the repository may be a folder above), and the compiles recorded in its history. In a terminal the
+list is navigable: a number shows what differs, `d 3` the diff, `r 3` restores. Without one, `--history-diff` and
+`--history-restore` take a number, a backup's timestamp (any start of it: `20261003`), a file name, a commit hash,
+`latest`, or `previous` (the newest that is not the file as it is). A restore backs the file up first, so nothing is
+ever lost; it keeps the build notes and history of the file as it is now, and adds a `Restored version ...` entry
+(`stamp: {restored-note: false}` or `--no-restored-note` leave it out; `--dry-run` shows what would change).
+`pdfmd --init-backups report.md` turns all of this on for a document: backups and a history file, set in its front
+matter under an `AUTO GENERATED` comment (`--global` for every document), and a first copy.
+
 Separately, when `pypdf` is installed, every PDF built by `pdfmd` gets two
 hidden metadata keys, `PdfmdVersions` and `PdfmdBuildDate`, which you can
 read with `pdfinfo -meta`. Turn this off with `--no-stamp-pdf-metadata`.
 
 ## Reference
 
-- [`docs/recipes.md`](https://github.com/aliperdehan/pdfmd/blob/main/docs/recipes.md) has short "how do I..." answers: a paper in several scripts, a scanned PDF to Markdown, a folder with a house style, a book from many files.
+- [`docs/recipes.md`](https://github.com/aliperdehan/pdfmd/blob/main/docs/recipes.md) is the task-by-task manual: fifteen chapters of "how do I..." with the commands (the basics, a house style, long documents, tables, code and images, raw HTML/LaTeX/Typst, other inputs and outputs, the PDF that carries its source, history, fonts and scripts, speed, the editor and VS Code, and what to do when something looks wrong), plus an option reference by task.
 - `pdfmd --help` lists every flag.
 - The docstring at the top of [`pdfmd.py`](https://github.com/aliperdehan/pdfmd/blob/main/pdfmd.py) is the full reference
   for each behaviour and its edge cases.