@aiquants/markdown-explorer 0.20.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +73 -0
- package/README.md +13 -7
- package/dist/EditSurface-BesNcSQj.cjs +6 -0
- package/dist/EditSurface-DNjI8pV3.js +975 -0
- package/dist/{ExplorerSplit-DkE46pS0.cjs → ExplorerSplit-B4oNvydP.cjs} +1 -1
- package/dist/{ExplorerSplit-BAYa3WO3.js → ExplorerSplit-GUdzpDPg.js} +1 -1
- package/dist/{MultiLayout-BkhnkQs5.js → MultiLayout-BJ-s6AaD.js} +2 -2
- package/dist/{MultiLayout-DsxYURra.cjs → MultiLayout-DtKChQ_s.cjs} +1 -1
- package/dist/{NativeScrollBar-CcNqDlO6.js → NativeScrollBar-D10fsk-7.js} +1 -1
- package/dist/{NativeScrollBar-B57yWQVG.cjs → NativeScrollBar-DJKlPWvD.cjs} +1 -1
- package/dist/{TreeLayout-DCGqVwt9.cjs → TreeLayout-C1mTmoPp.cjs} +1 -1
- package/dist/{TreeLayout-Cf2PzZEI.js → TreeLayout-DQT5p2vI.js} +2 -2
- package/dist/core/labels.d.cts +9 -27
- package/dist/core/labels.d.ts +9 -27
- package/dist/{index-DiLkl4dC.js → index-0pnuyLUO.js} +1445 -1515
- package/dist/index-T98wklgM.cjs +13 -0
- package/dist/index.cjs +1 -1
- package/dist/index.js +1 -1
- package/dist/{merge-DSfJceoQ.cjs → merge-CTkzzHtn.cjs} +1 -1
- package/dist/{merge-BIXNcrea.js → merge-DauzoA2W.js} +1 -1
- package/docs/behavior/dom-hooks.md +1 -1
- package/docs/behavior/layout.md +2 -1
- package/docs/behavior/localization.md +6 -5
- package/docs/behavior/security.md +17 -7
- package/package.json +5 -5
- package/dist/EditSurface-D1yklbWu.js +0 -833
- package/dist/EditSurface-GxmyUeGE.cjs +0 -3
- package/dist/index-Bdve5EFS.cjs +0 -13
|
@@ -8,9 +8,11 @@ The explorer renders documents that the reader did not write, from storage the r
|
|
|
8
8
|
|
|
9
9
|
- A document path from a URL must be acceptable before any source sees it: non-empty, at most 4096 UTF-16 code units, well-formed (no lone surrogate, which no URL can carry), without control characters (U+0000–U+001F, U+007F) and without backslashes (some filesystems read them as separators). Anything else is answered as `invalid`.
|
|
10
10
|
- Deny rules (`deny`, default `DEFAULT_DENIED_PATH_SEGMENTS`) hide matching entries from every listing — by entry name and by path, so a denied folder is pruned with everything under it — and refuse every document or children request for a matching path with a `forbidden` failure before any source is called.
|
|
11
|
-
- A rule is matched against each path segment (or entry name) at every percent-decoding level up to three decodes, against each level whole and against its parts split again on `/` or `\` (a storage may read either character as part of one name, as a Linux file name, a Drive name or a GCS object name does, or as a separator), and after NFKC normalization and full case folding (upper-casing then lower-casing, which folds `ſ` to `s` and `ı` to `i`). Decoding is lenient: a malformed escape stays as written while the valid ones around it are decoded.
|
|
11
|
+
- A rule is matched against each path segment (or entry name) at every percent-decoding level up to three decodes, against each level whole and against its parts split again on `/` or `\` (a storage may read either character as part of one name, as a Linux file name, a Drive name or a GCS object name does, or as a separator), and after NFKC normalization and full case folding (upper-casing then lower-casing, which folds `ſ` to `s` and `ı` to `i`). Decoding is lenient: a malformed escape stays as written while the valid ones around it are decoded.
|
|
12
|
+
`%2Eenv`, `%252Eenv`, `.%65nv`, `docs%5C.env`, `SECRETS` and `ſecrets` are all denied like `.env` and `secrets`, whatever the storage backend decodes or folds before opening the path.
|
|
12
13
|
- Pattern syntax: a pattern matches one whole segment; `*` matches any run of characters inside it. `deny: false` turns the rules off; a custom array replaces the defaults (spread `DEFAULT_DENIED_PATH_SEGMENTS` to extend them).
|
|
13
|
-
- The defaults cover environment files (`.env*` — every name starting with `.env`, for example `.env~`, `.env-old`, `.envs/`, and also `.envelope.md` — and `*.env` with the names its backups and copies take:
|
|
14
|
+
- The defaults cover environment files (`.env*` — every name starting with `.env`, for example `.env~`, `.env-old`, `.envs/`, and also `.envelope.md` — and `*.env` with the names its backups and copies take:
|
|
15
|
+
`*.env.*`, `*.env~*`, `*.env-*`, `*.env_*` and `*.env#*`, for example `prod.env.bak`, `prod.env.production`, `prod.env.1`, `prod.env~`, `prod.env-old`, `prod.env_backup` and `#prod.env#`; rules have no exceptions, so a document named like `deploy.env.md` is denied too, while names in which `.env` is followed by a letter or digit, such as `prod.environment.md`, or has no dot before it, such as `environment.md` and `my-env.md`, are not),
|
|
14
16
|
version-control metadata (`.git`, `.git-credentials`, `.svn`, `.hg`), `node_modules`, secrets files (`secrets`, `.secrets`, and `secrets.` with `json`, `yml`, `yaml`, `toml`, `ini` or `txt`), the credential folders and files of command-line tools (`.ssh`, `.aws`, `.gnupg`, `.docker`, `.kube`, `.terraform`, `.npmrc`, `.yarnrc`, `.yarnrc.yml`, `.netrc`, `_netrc`, `.pypirc`, `.pgpass`, `.my.cnf`, `.htpasswd`, `.vault-token`),
|
|
15
17
|
keys and keystores (`id_rsa*`, `id_dsa*`, `id_ecdsa*`, `id_ed25519*`, `*.pem`, `*.key`, `*.p8`, `*.p12`, `*.pfx`, `*.ppk`, `*.jks`, `*.keystore`, `*.kdbx`), credential files (`*credentials*.json`, `service-account*.json`), Terraform state and variables (`*.tfstate`, `*.tfstate.*`, `*.tfvars`, `*.tfvars.json`) and shell histories (`.*_history`, `fish_history`).
|
|
16
18
|
- `createFileSystemSource` additionally checks lexical containment before any filesystem call and again after resolving the real path, refuses symbolic links, and never follows a path outside its root. It opens a file without following a link (where the platform has `O_NOFOLLOW`) and reads it only when the opened file is the one the checks saw (the same device and inode; a document read refuses another file as `forbidden`).
|
|
@@ -41,7 +43,8 @@ Markdown HTML returned by a source is sanitized on the server with DOMPurify, in
|
|
|
41
43
|
- `allow`, `allowfullscreen`, `frameborder` and `referrerpolicy` survive only on iframes: `referrerpolicy` only as `no-referrer`, `origin`, `same-origin`, `strict-origin` or `strict-origin-when-cross-origin` (`EMBED_FRAME_REFERRER_POLICIES`), and `allow` reduced to `autoplay`, `clipboard-write`, `encrypted-media`, `fullscreen`, `picture-in-picture` and `web-share`, each without its allowlist (`reduceFrameFeatures`);
|
|
42
44
|
- `target` survives only on `a` and `area`, and a link with any `target` other than `_self` gets `rel="noopener noreferrer"`;
|
|
43
45
|
- `id` attributes that would clobber `document` or form properties (`cookie`, `forms`, `location`, …) are removed — except on headings, which the table of contents needs and which cannot clobber `document`;
|
|
44
|
-
- the markup converters write for GitHub alerts and Zenn's message and details containers survives as written, through DOMPurify's own allow-lists: `div`, `p`, `aside`, `span`, `details` and `summary` with their `class` (and `aria-hidden` on a message's symbol), and an alert's inline SVG icon, `svg` with its `class`, `viewBox`, `version`, `width`, `height` and `aria-hidden` and `path` with its `d` (the README's Documents gives the markup, and its Load the styles the `@aiquants/markdown` stylesheets that draw it).
|
|
46
|
+
- the markup converters write for GitHub alerts and Zenn's message and details containers survives as written, through DOMPurify's own allow-lists: `div`, `p`, `aside`, `span`, `details` and `summary` with their `class` (and `aria-hidden` on a message's symbol), and an alert's inline SVG icon, `svg` with its `class`, `viewBox`, `version`, `width`, `height` and `aria-hidden` and `path` with its `d` (the README's Documents gives the markup, and its Load the styles the `@aiquants/markdown` stylesheets that draw it).
|
|
47
|
+
Inside it, as anywhere else, event handlers, `script`, `foreignObject`, `use` and `animate` are removed and URLs follow the rules above;
|
|
45
48
|
- `data-*` attributes survive only for the Markdown renderer's contract — `data-link-kind`, `data-doc-path`, `data-doc-exists`, `data-footnote-ref` and `data-footnote-backref` on links, `data-asset-src` and `data-asset-exists` on images, `data-footnotes` on sections, `data-code-meta` on code (a ```spreadsheets table's options, an inert value), and the embed placeholder below — so the renderer's iframe plugin trigger (`data-plugin-type`) and the `<div data-plugin-type>` placeholders of `@aiquants/markdown` 5 never reach the browser; SVG and MathML pass through DOMPurify's own allow-lists;
|
|
46
49
|
- the embed placeholder, `data-embed-url` on a `<p>` (the converter's mark of a paragraph whose only content is one absolute `http(s)` link without credentials, holding the link's `href` byte for byte), survives only where `placeholderLinkOf` of `@aiquants/markdown/embeds` accepts it, whatever the provider.
|
|
47
50
|
DOMPurify visits a parent before its children, so the sanitizer records the paragraph's attribute as parsed and, once DOMPurify is done with the attributes of a link directly inside that paragraph, restores it only when the paragraph holds exactly that one link (besides comments, white space, U+00A0 and U+3000) and the parsed attribute value equals the link's `href` as DOMPurify trimmed it.
|
|
@@ -65,15 +68,22 @@ A markdown document that lists more than 10,000 headings (`EXPLORER_MAX_HEADINGS
|
|
|
65
68
|
Sanitizing never runs on the request thread. HTML larger than `sanitizer.maxHtmlBytes` (4 MiB by default) is refused as `too-large` at once; the rest goes to a pool of worker threads (`sanitizer.workers`, 2 by default), shared by every explorer of the process with the same pool settings, with a per-document deadline (`sanitizer.deadlineMs`, 30 s by default): a document that misses it is `too-large`, and its worker is replaced.
|
|
66
69
|
In the worker the nesting limit is enforced while the HTML is parsed, by the parser and in the mode DOMPurify's jsdom uses (parse5, the input as a whole document, scripting disabled), before DOMPurify runs and within the document's deadline.
|
|
67
70
|
The parse stops, and the document is `too-large` without being sanitized, as soon as an element is placed deeper than 512 levels (the depth at which Chromium's HTML parser stops nesting elements; the children of `body` are at depth 1 and a `template`'s contents one level below it), or as soon as the parser has created more elements than the HTML has characters plus `html`, `head` and `body` (only formatting elements the parser reopens over and over can do that; an ordinary document stays far below).
|
|
68
|
-
Before DOMPurify runs, the guard also counts every element, attribute and comment the parser creates and refuses the document (`too-large`) once the count passes (`sanitizer.workerHeapMb` − 64 MiB − `sanitizer.maxHtmlBytes` × 32 bytes) ÷ 4 KiB, which is 81,920 nodes by default, or once it has seen more than 4,096 comments wherever they sit. jsdom inserts each comment the parser attaches to the document itself (before `<html>` or after `</html>`) in time that grows with those already there (4,000 took 0.37 s and 8,000 took 1.4 s in the worker, while 8,000 comments inside the body took 0.14 s),
|
|
69
|
-
|
|
71
|
+
Before DOMPurify runs, the guard also counts every element, attribute and comment the parser creates and refuses the document (`too-large`) once the count passes (`sanitizer.workerHeapMb` − 64 MiB − `sanitizer.maxHtmlBytes` × 32 bytes) ÷ 4 KiB, which is 81,920 nodes by default, or once it has seen more than 4,096 comments wherever they sit. jsdom inserts each comment the parser attaches to the document itself (before `<html>` or after `</html>`) in time that grows with those already there (4,000 took 0.37 s and 8,000 took 1.4 s in the worker, while 8,000 comments inside the body took 0.14 s),
|
|
72
|
+
so the cap keeps that worst case near 0.4 s. The constants are measured: the worker needs about 48 MiB of its own, a parsed node costs at most about 3.4 KB at the sanitizer's peak and a byte of HTML at most about 31 bytes, so a document the guard accepts fits in the worker's heap. The budget does not depend on the worker's heap limit and holds even where a process-wide `--max-old-space-size` overrides it.
|
|
73
|
+
Before parsing, a linear scan follows every `<` or `</` followed by a letter through the tokenizer's tag states, quotes included, and refuses the document once a tag could hold more than 1,024 attributes, duplicates included; it also reads text the tokenizer would treat as text, so it may refuse more, never less.
|
|
74
|
+
While parsing, `html` and `body` may receive at most 1,024 attributes through repeated tags. parse5, jsdom and DOMPurify look each attribute up in its element's list, so one element's attributes cost time that grows with the square of their number: without the cap, 40,000 attributes in one tag take 36 s and 79 repeated `body` tags of 1,000 attributes 90 s in the worker, while 78 tags of 1,024 take 1.8 s.
|
|
75
|
+
The guard also adds up the name and value characters of every attribute the parser creates, including the copies it gives a formatting element it reopens or the adoption agency recreates, and refuses the document once they exceed `sanitizer.maxHtmlBytes` (a document whose elements each appear once never does): Without the budget, 1 MiB of copied `title` makes the worker write 210 million characters; the budget refuses it in about 0.1 s.
|
|
76
|
+
Sanitized HTML longer than 4 × `sanitizer.maxHtmlBytes` UTF-16 code units (16,777,216 by default; ordinary HTML measured at most 2.05 characters per byte) is a cached `too-large`: the worker never posts it and the service refuses it should it arrive, so neither the cache (where the longest outcome weighs 32 MiB of the default 64 MiB) nor a caller ever holds more.
|
|
70
77
|
Every sanitizer worker starts with Node's `resourceLimits`: `workerHeapMb` MiB of old generation (512 by default) and 32 MiB of young generation. A worker whose heap runs out is ended with `ERR_WORKER_OUT_OF_MEMORY`, and its document is answered `too-large` and cached. The young generation is kept small on purpose: with Node's default for a worker (192 MiB), a worker at its limit could abort the whole process instead of ending alone, because one scavenge can promote more than the 16 MiB of leeway Node gives it.
|
|
71
78
|
An element counts at the depth where the parser first places it, so when the parser later moves misnested markup to a shallower place (the adoption agency for misnested formatting tags, a `frameset` replacing `body`), a document whose finished tree is at most 512 levels deep can still be refused; a deeper one is never accepted. DOMPurify's own walk also stops at the first element deeper than 512 levels.
|
|
72
79
|
Each request names its reader's scope. The pool admits at most `sanitizer.jobsPerScope` documents per reader running or waiting and `sanitizer.maxQueuedJobs` waiting in all; a document beyond either is `failed`, logged without the principal and not remembered, so a later request tries again.
|
|
73
|
-
A page sends the document of every panel to the sanitizer at once for one reader, so the admission defaults follow the multi view: `jobsPerScope` defaults to `multiView.maxPanels` + max(1, `workers` − 1) + 3 — the panels, the previous page's documents still running after a reload (a running job is not stopped when its requests leave, and one reader runs at most max(1, `workers` − 1) at once), the two intent prefetches and one spare; 16 with the default 12 panels and two workers. `maxQueuedJobs` defaults to four times `jobsPerScope`, also when it is set.
|
|
80
|
+
A page sends the document of every panel to the sanitizer at once for one reader, so the admission defaults follow the multi view: `jobsPerScope` defaults to `multiView.maxPanels` + max(1, `workers` − 1) + 3 — the panels, the previous page's documents still running after a reload (a running job is not stopped when its requests leave, and one reader runs at most max(1, `workers` − 1) at once), the two intent prefetches and one spare; 16 with the default 12 panels and two workers. `maxQueuedJobs` defaults to four times `jobsPerScope`, also when it is set.
|
|
81
|
+
Neither may be set below `multiView.maxPanels` (a `RangeError` naming both at startup). All anonymous readers share one scope and therefore one admission budget.
|
|
74
82
|
A reader runs at most `workers` − 1 documents at once (one when there is a single worker), and a free worker goes to the reader running the fewest, so one reader cannot starve the others; all anonymous readers share one scope.
|
|
75
83
|
Concurrent requests of one reader for the same input share one job; jobs of different readers are separate, and the first to finish answers the others; a waiting job is dropped when every request waiting on it has left. The first outcome remembered for an input stays: it answers every other job still pending for that input and drops those still waiting, and a duplicate already running finishes on its worker without answering anyone or replacing that outcome, so a document near the deadline or the heap limit cannot flip between `sanitized` and `too-large`.
|
|
76
|
-
Every outcome that describes the HTML (a refusal for nesting, the node budget, the heap or the deadline included) is remembered by the SHA-256 of the input in a cache bounded by count (1024 entries) and by bytes (`sanitizer.cacheBytes`, 64 MiB by default), so a document read by many readers is sanitized — or refused — once. A worker is sent a document only after it has reported that it is ready.
|
|
84
|
+
Every outcome that describes the HTML (a refusal for nesting, the node budget, the heap or the deadline included) is remembered by the SHA-256 of the input in a cache bounded by count (1024 entries) and by bytes (`sanitizer.cacheBytes`, 64 MiB by default), so a document read by many readers is sanitized — or refused — once. A worker is sent a document only after it has reported that it is ready.
|
|
85
|
+
A worker that fails before then (it cannot be started, is not ready within the deadline, or fails or exits while starting) never received the document: the document is answered `failed`, the cause is logged, and the outcome is not cached. A failure while a worker holds a document it was sent, such as an uncaught error or an unexpected exit, is that document's crash and stays a cached `failed`, whatever order Node delivers the worker's error and its messages in. Running out of heap and missing the deadline stay a cached `too-large`.
|
|
86
|
+
Creating the explorer fails when the worker file is missing next to the server module; where DOMPurify cannot run, it refuses instead of returning its input unchanged, and the document fails.
|
|
77
87
|
|
|
78
88
|
Fixed or absolutely positioned markup in a document cannot cover the host or the explorer: the document region is its containing block and clips its painting (in a multi-view panel, the panel's frame clips it). Such markup can still cover the rest of its own document.
|
|
79
89
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@aiquants/markdown-explorer",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.22.0",
|
|
4
4
|
"description": "Markdown document explorer for React Router 8 (framework mode): a source tree with a single-document view and a multi-panel reading view, storage-agnostic server ports, golden-ratio layout and an en / ja UI.",
|
|
5
5
|
"sideEffects": [
|
|
6
6
|
"**/*.css"
|
|
@@ -70,9 +70,9 @@
|
|
|
70
70
|
"peerDependencies": {
|
|
71
71
|
"@aiquants/directory-tree": "^5.0.1",
|
|
72
72
|
"@aiquants/drag-drop-panels": "^0.12.0",
|
|
73
|
-
"@aiquants/markdown": "^
|
|
73
|
+
"@aiquants/markdown": "^11.0.1",
|
|
74
74
|
"@aiquants/resize-panels": "^2.2.0",
|
|
75
|
-
"@aiquants/spreadsheet": "^0.
|
|
75
|
+
"@aiquants/spreadsheet": "^0.16.0",
|
|
76
76
|
"@aiquants/virtualscroll": "^3.22.1",
|
|
77
77
|
"react": "^19.2.7",
|
|
78
78
|
"react-dom": "^19.2.7",
|
|
@@ -81,9 +81,9 @@
|
|
|
81
81
|
"devDependencies": {
|
|
82
82
|
"@aiquants/directory-tree": "5.0.1",
|
|
83
83
|
"@aiquants/drag-drop-panels": "0.12.0",
|
|
84
|
-
"@aiquants/markdown": "
|
|
84
|
+
"@aiquants/markdown": "11.0.1",
|
|
85
85
|
"@aiquants/resize-panels": "2.2.0",
|
|
86
|
-
"@aiquants/spreadsheet": "0.
|
|
86
|
+
"@aiquants/spreadsheet": "0.16.0",
|
|
87
87
|
"@aiquants/virtualscroll": "3.22.1",
|
|
88
88
|
"@heroicons/react": "^2.2.0",
|
|
89
89
|
"@playwright/test": "^1.59.1",
|