tosijs-ui 1.16.0 → 1.16.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +73 -0
- package/dist/doc-system/example-policy.d.ts +43 -3
- package/dist/doc-system/example-policy.js +89 -8
- package/dist/doc-system/highlight.d.ts +5 -1
- package/dist/doc-system/highlight.js +11 -14
- package/dist/doc-system/render.js +10 -1
- package/dist/doc-system/site/check-examples.js +8 -12
- package/dist/doc-system/site/epub.js +6 -5
- package/dist/doc-system/site/index.d.ts +1 -0
- package/dist/doc-system/site/index.js +3 -0
- package/dist/iife.js +90 -90
- package/dist/iife.js.map +15 -11
- package/dist/index.d.ts +1 -0
- package/dist/index.js +1 -0
- package/dist/live-example/component.d.ts +6 -3
- package/dist/live-example/component.js +148 -15
- package/dist/live-example/dialects.d.ts +66 -0
- package/dist/live-example/dialects.js +89 -0
- package/dist/live-example/index.d.ts +2 -0
- package/dist/live-example/index.js +2 -0
- package/dist/live-example/insert-examples.js +51 -21
- package/dist/live-example/save-to-source.js +4 -5
- package/dist/search-field.d.ts +74 -0
- package/dist/search-field.js +742 -0
- package/dist/tag-colors.d.ts +8 -0
- package/dist/tag-colors.js +25 -0
- package/dist/tag-list.d.ts +1 -62
- package/dist/tag-list.js +7 -89
- package/dist/tag.d.ts +63 -0
- package/dist/tag.js +77 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/llms.txt +3 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,78 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.16.1
|
|
4
|
+
|
|
5
|
+
A patch: everything here is additive. Two features, a search field that suggests refinements
|
|
6
|
+
and pluggable live-example dialects, and nothing that changes how existing code behaves.
|
|
7
|
+
`dist/iife.js` grows by 2.9 kB gzip (469.9 → 472.8 kB), which is both features going through
|
|
8
|
+
the batteries-included bundle; the ESM path pays only for what it imports.
|
|
9
|
+
|
|
10
|
+
### `<tosi-search-field>`: a search box that suggests refinements (snowfox, virta #2459)
|
|
11
|
+
|
|
12
|
+
A new component in the spirit of Finder's search field. As you type, it lists **hints** from
|
|
13
|
+
rules you supply: type `doc` and it can offer *name contains "doc"* or *Word documents*.
|
|
14
|
+
Picking one turns it into a **tag** in the query and consumes the text, and you can keep
|
|
15
|
+
typing. `value` is `{ tags, text }`.
|
|
16
|
+
|
|
17
|
+
- A tag is `{ caption, background?, color?, …your data }`, with chips coloured like
|
|
18
|
+
`<tosi-tag-list>`'s.
|
|
19
|
+
- It fires `change` when the tags change, and `action` on Enter when no hint is highlighted.
|
|
20
|
+
- It is a real combobox: focus stays in the field, ↓/↑ move through the hints, Escape closes
|
|
21
|
+
them, and Backspace in an empty field removes the last tag. The hints close only for a
|
|
22
|
+
scroll that moves the field itself, so a table re-filtering on every keystroke (which
|
|
23
|
+
scrolls it) doesn't close them the instant they open, and scrolling past the end of the list
|
|
24
|
+
doesn't scroll the page.
|
|
25
|
+
- A clear (✕) button, shown when there's anything to clear, removes the tags and the text
|
|
26
|
+
(also `clear()`).
|
|
27
|
+
- **As a filter:** give each tag a `test(item)` and the field a `textTest(item, text)`, and
|
|
28
|
+
`field.filter` is an array filter to hand straight to `<tosi-table>.filter`. The docs filter
|
|
29
|
+
the 3,655-row emoji table this way. `hints` and `textTest` can be passed to the element
|
|
30
|
+
creator like any other property; `textTest` defaults to `null`, because tosijs sets a creator
|
|
31
|
+
prop as a property only when its default is not `undefined` (otherwise it was stored as an
|
|
32
|
+
attribute, a stringified function). `filter` returns the same function until the query
|
|
33
|
+
changes, so a table's filter memo holds.
|
|
34
|
+
- Importing it on its own (`tosijs-ui/search-field`) costs 25.5 kB gzip: `<tosi-tag>` moved
|
|
35
|
+
into its own module (still exported from `tag-list`), so it no longer pulls in the menu
|
|
36
|
+
system.
|
|
37
|
+
|
|
38
|
+
### Pluggable dialects for live examples (#184)
|
|
39
|
+
|
|
40
|
+
Live examples can now be written in languages tosijs-ui doesn't know about. `registerDialect`
|
|
41
|
+
from `tosijs-ui/live-example` adds one, and a fence in that language becomes a live example
|
|
42
|
+
like `js`, `tjs` and `ts`: it groups with `html`/`css`/`test` blocks, honours `:static` and the
|
|
43
|
+
site's example policy, and saves back to source. `js`, `tjs` and `ts` are now built-in entries
|
|
44
|
+
of the same registry.
|
|
45
|
+
|
|
46
|
+
A dialect either **transforms** its source to JavaScript that the example runs as usual, or
|
|
47
|
+
**runs** it itself — given the example's preview, its options, the page's modules, an
|
|
48
|
+
`AbortSignal` for when the example re-runs, and `report()` for showing a result. Registering
|
|
49
|
+
over a built-in (`registerDialect('tjs', { transform })`) swaps in your own tjs-lang build.
|
|
50
|
+
This is what tjs-lang asked for to move AJS into doc pages; see the live-example page, "Adding
|
|
51
|
+
a dialect".
|
|
52
|
+
|
|
53
|
+
If your dialect's name is also a syntax-highlighting grammar, call `registerLiveLanguage` from
|
|
54
|
+
`tosijs-ui/site` in your site config too: the build-time highlighter never sees page
|
|
55
|
+
registrations, and would otherwise tokenize the source the example reads.
|
|
56
|
+
|
|
57
|
+
A replaced built-in is what runs in the browser, in dev and on the deployed site alike: the
|
|
58
|
+
build pre-transpiles `tjs` examples with the installed tjs-lang (so readers don't load the
|
|
59
|
+
transpiler), and the page skips that pre-transpiled code once the site registers its own `tjs`.
|
|
60
|
+
The build-time example check still transpiles `tjs` fences with the installed tjs-lang, so an
|
|
61
|
+
override that accepts newer syntax can fail that check; `checkExamples: false` is the escape
|
|
62
|
+
until the site config can say `tjs` is overridden.
|
|
63
|
+
|
|
64
|
+
Fence languages are case-sensitive, as they were in 1.16.0: ```` ```JS ```` is not a live
|
|
65
|
+
example. One visible difference: 1.16.0 also left such a fence unhighlighted, and it is now
|
|
66
|
+
highlighted as ordinary static code, on the page and in the ePub.
|
|
67
|
+
|
|
68
|
+
**Fence options.** Any fence can carry a JSON object after its language,
|
|
69
|
+
`` ```tjs {"runTests": "report"} ``, passed to its dialect as `options`. Malformed options are
|
|
70
|
+
reported in the console with the example they belong to.
|
|
71
|
+
|
|
72
|
+
Two more copies of "which languages run" were found while doing this, a hard-coded selector in
|
|
73
|
+
`insertExamples` and a private fence parser in the example checker, which would have read
|
|
74
|
+
`{"debug":true}` as the mode `true`. Both now use the shared rule.
|
|
75
|
+
|
|
3
76
|
## 1.16.0
|
|
4
77
|
|
|
5
78
|
### `<tosi-md>` sanitizes by default (#179)
|
|
@@ -1,5 +1,37 @@
|
|
|
1
1
|
/** Fence languages that EXECUTE. Everything else is display-only. */
|
|
2
2
|
export declare const EXECUTABLE_LANGS: Set<string>;
|
|
3
|
+
/**
|
|
4
|
+
* Make fences in `lang` live examples, as an example's source block.
|
|
5
|
+
*
|
|
6
|
+
* You normally call `registerDialect` (which calls this). Call it directly only in a BUILD
|
|
7
|
+
* process that must agree with the page: the static highlighter runs at build time, where the
|
|
8
|
+
* page's `registerDialect` calls never happen, so a dialect named like a highlighter grammar
|
|
9
|
+
* (`python`, say) would be tokenized there and the example would read markup instead of code.
|
|
10
|
+
* Registering the name in your site config avoids that.
|
|
11
|
+
*/
|
|
12
|
+
export declare function registerLiveLanguage(lang: string): void;
|
|
13
|
+
/** Is `lang` an example's source language — `js`, `tjs`, `ts`, or a registered dialect? */
|
|
14
|
+
export declare function isDialectLanguage(lang: string): boolean;
|
|
15
|
+
/**
|
|
16
|
+
* THE `language-*` class pattern, and the only place it is written.
|
|
17
|
+
*
|
|
18
|
+
* It was spelled `[A-Za-z0-9_+#-]+` four times in `highlight.ts` and `[\w-]+` once in
|
|
19
|
+
* `epub.ts`, so a `c++` or `c#` fence was read as a language by one and as `c` by the other.
|
|
20
|
+
* `+` and `#` are in the class deliberately: they are real language names. It lives here, not
|
|
21
|
+
* in `highlight.ts`, because live examples need it too and must not import the highlighter's
|
|
22
|
+
* grammar map to get it.
|
|
23
|
+
*/
|
|
24
|
+
export declare const LANGUAGE_NAME = "[A-Za-z0-9_+#-]+";
|
|
25
|
+
/**
|
|
26
|
+
* The fence language of a `<code class="language-…">`, **as written** (case preserved), or `''`.
|
|
27
|
+
*
|
|
28
|
+
* Case matters: fence languages are lowercase (`parseFenceInfo` reads `[a-z]+`), so ```` ```JS ````
|
|
29
|
+
* is not a live example. 1.16.1's first cut matched classes case-insensitively in one place
|
|
30
|
+
* and not the others, and an uppercase fence became live in `insertExamples` but not in
|
|
31
|
+
* save-to-source's scan — its content vanished from the page and every later save landed
|
|
32
|
+
* on the wrong block. Use `langOfClass` (lowercased) only for grammar lookups.
|
|
33
|
+
*/
|
|
34
|
+
export declare function languageOfClass(className: string | null | undefined): string;
|
|
3
35
|
/**
|
|
4
36
|
* - `'auto'` — the six executable languages become live examples (the web default).
|
|
5
37
|
* - `'opt-in'` — only a fence that asks, via `:inline` / `:iframe` / `:ide`.
|
|
@@ -13,24 +45,30 @@ export type ExamplePolicy = 'auto' | 'opt-in' | 'none';
|
|
|
13
45
|
/**
|
|
14
46
|
* Will this fence become a live example?
|
|
15
47
|
*
|
|
16
|
-
* @param lang the fence language
|
|
48
|
+
* @param lang the fence language as written (`js`, `html`, `typescript`, …); case-sensitive
|
|
17
49
|
* @param mode the `:<mode>` suffix, if any (`inline` | `iframe` | `ide` | `static`)
|
|
18
50
|
* @param policy `'auto'` (executables run) or `'opt-in'` (only fences that ask)
|
|
19
51
|
*/
|
|
20
52
|
export declare function isLiveFence(lang: string, mode: string | undefined, policy?: ExamplePolicy): boolean;
|
|
21
53
|
/**
|
|
22
|
-
* THE fence-info parser. `js`, `css#anchor`, `js:iframe`, `ts:ide#demo`, `ts#demo:ide
|
|
54
|
+
* THE fence-info parser. `js`, `css#anchor`, `js:iframe`, `ts:ide#demo`, `ts#demo:ide`,
|
|
55
|
+
* `tjs {"runTests": "report"}`.
|
|
23
56
|
*
|
|
24
57
|
* `:mode` (inline | iframe | ide | static) sets the live example's execution mode; `#id` gives
|
|
25
58
|
* it a stable anchor. `#id` is `[A-Za-z0-9_-]+` and `:mode` is `[a-z]+`, so the two cannot
|
|
26
59
|
* overlap and each is parsed independently, order-free.
|
|
27
60
|
*
|
|
61
|
+
* A JSON object after the head is the example's **options**, passed to its dialect (#184). It
|
|
62
|
+
* is split off FIRST: `{"debug":true}` contains `:true`, which the mode pattern would
|
|
63
|
+
* otherwise read as a mode. Options that are not a JSON object are reported in
|
|
64
|
+
* `optionsError` rather than dropped, so an author sees why their options did nothing.
|
|
65
|
+
*
|
|
28
66
|
* Extracted because a SECOND, worse copy existed in `save-to-source.ts` — a regex that
|
|
29
67
|
* captured the language as `[\w-]*` and therefore stopped dead at the colon, so it could not
|
|
30
68
|
* see `:static` at all. With a `:static` fence in a document, its example ordinals diverged
|
|
31
69
|
* from the ones `insert-examples` assigns, and an edit saved over a DIFFERENT block than the
|
|
32
70
|
* one edited. Silently: the "couldn't locate this example" guard only fires when the ordinal
|
|
33
|
-
* is out of range, and here a group existed at that index
|
|
71
|
+
* is out of range, and here a group existed at that index.
|
|
34
72
|
*
|
|
35
73
|
* This is the same failure the `isLiveFence` docblock above describes, one layer down: the
|
|
36
74
|
* rule was stated twice and the copies disagreed. Parse fence info here or not at all.
|
|
@@ -39,4 +77,6 @@ export declare function parseFenceInfo(info: string): {
|
|
|
39
77
|
lang: string;
|
|
40
78
|
mode?: string;
|
|
41
79
|
id?: string;
|
|
80
|
+
options?: Record<string, unknown>;
|
|
81
|
+
optionsError?: string;
|
|
42
82
|
};
|
|
@@ -25,10 +25,66 @@ export const EXECUTABLE_LANGS = new Set([
|
|
|
25
25
|
'css',
|
|
26
26
|
'test',
|
|
27
27
|
]);
|
|
28
|
+
/**
|
|
29
|
+
* The languages that are an example's SOURCE block (its one executable slot). `js`, `tjs` and
|
|
30
|
+
* `ts` are built in; `registerLiveLanguage` adds more (`registerDialect` in
|
|
31
|
+
* `tosijs-ui/live-example` calls it, so a registered dialect's fences run).
|
|
32
|
+
*
|
|
33
|
+
* Kept here, beside `EXECUTABLE_LANGS`, because the same fence is judged in four places
|
|
34
|
+
* (grouping, save-to-source, the highlighter, the ePub) and they must all see a registered
|
|
35
|
+
* dialect or none of them may.
|
|
36
|
+
*/
|
|
37
|
+
const SOURCE_LANGS = new Set(['js', 'ts', 'tjs']);
|
|
38
|
+
/**
|
|
39
|
+
* Make fences in `lang` live examples, as an example's source block.
|
|
40
|
+
*
|
|
41
|
+
* You normally call `registerDialect` (which calls this). Call it directly only in a BUILD
|
|
42
|
+
* process that must agree with the page: the static highlighter runs at build time, where the
|
|
43
|
+
* page's `registerDialect` calls never happen, so a dialect named like a highlighter grammar
|
|
44
|
+
* (`python`, say) would be tokenized there and the example would read markup instead of code.
|
|
45
|
+
* Registering the name in your site config avoids that.
|
|
46
|
+
*/
|
|
47
|
+
export function registerLiveLanguage(lang) {
|
|
48
|
+
const name = lang.toLowerCase();
|
|
49
|
+
if (!/^[a-z]+$/.test(name)) {
|
|
50
|
+
throw new Error(`registerLiveLanguage: "${lang}" is not a fence language (lowercase letters only, since the fence parser reads \`[a-z]+\`)`);
|
|
51
|
+
}
|
|
52
|
+
if (EXECUTABLE_LANGS.has(name) && !SOURCE_LANGS.has(name)) {
|
|
53
|
+
throw new Error(`registerLiveLanguage: "${name}" is already an example block (html/css/test) and cannot be a dialect`);
|
|
54
|
+
}
|
|
55
|
+
SOURCE_LANGS.add(name);
|
|
56
|
+
}
|
|
57
|
+
/** Is `lang` an example's source language — `js`, `tjs`, `ts`, or a registered dialect? */
|
|
58
|
+
export function isDialectLanguage(lang) {
|
|
59
|
+
return SOURCE_LANGS.has(lang);
|
|
60
|
+
}
|
|
61
|
+
/**
|
|
62
|
+
* THE `language-*` class pattern, and the only place it is written.
|
|
63
|
+
*
|
|
64
|
+
* It was spelled `[A-Za-z0-9_+#-]+` four times in `highlight.ts` and `[\w-]+` once in
|
|
65
|
+
* `epub.ts`, so a `c++` or `c#` fence was read as a language by one and as `c` by the other.
|
|
66
|
+
* `+` and `#` are in the class deliberately: they are real language names. It lives here, not
|
|
67
|
+
* in `highlight.ts`, because live examples need it too and must not import the highlighter's
|
|
68
|
+
* grammar map to get it.
|
|
69
|
+
*/
|
|
70
|
+
export const LANGUAGE_NAME = '[A-Za-z0-9_+#-]+';
|
|
71
|
+
const LANGUAGE_CLASS = new RegExp(`language-(${LANGUAGE_NAME})`);
|
|
72
|
+
/**
|
|
73
|
+
* The fence language of a `<code class="language-…">`, **as written** (case preserved), or `''`.
|
|
74
|
+
*
|
|
75
|
+
* Case matters: fence languages are lowercase (`parseFenceInfo` reads `[a-z]+`), so ```` ```JS ````
|
|
76
|
+
* is not a live example. 1.16.1's first cut matched classes case-insensitively in one place
|
|
77
|
+
* and not the others, and an uppercase fence became live in `insertExamples` but not in
|
|
78
|
+
* save-to-source's scan — its content vanished from the page and every later save landed
|
|
79
|
+
* on the wrong block. Use `langOfClass` (lowercased) only for grammar lookups.
|
|
80
|
+
*/
|
|
81
|
+
export function languageOfClass(className) {
|
|
82
|
+
return String(className ?? '').match(LANGUAGE_CLASS)?.[1] ?? '';
|
|
83
|
+
}
|
|
28
84
|
/**
|
|
29
85
|
* Will this fence become a live example?
|
|
30
86
|
*
|
|
31
|
-
* @param lang the fence language
|
|
87
|
+
* @param lang the fence language as written (`js`, `html`, `typescript`, …); case-sensitive
|
|
32
88
|
* @param mode the `:<mode>` suffix, if any (`inline` | `iframe` | `ide` | `static`)
|
|
33
89
|
* @param policy `'auto'` (executables run) or `'opt-in'` (only fences that ask)
|
|
34
90
|
*/
|
|
@@ -37,7 +93,8 @@ export function isLiveFence(lang, mode, policy = 'auto') {
|
|
|
37
93
|
// costs the reader the highlighting.
|
|
38
94
|
if (policy === 'none')
|
|
39
95
|
return false;
|
|
40
|
-
|
|
96
|
+
// Case-sensitive, like `parseFenceInfo`: see `languageOfClass`.
|
|
97
|
+
if (!EXECUTABLE_LANGS.has(lang) && !SOURCE_LANGS.has(lang))
|
|
41
98
|
return false;
|
|
42
99
|
// `:static` opts out under any policy, so one corpus can target all of them.
|
|
43
100
|
if (mode === 'static')
|
|
@@ -45,27 +102,51 @@ export function isLiveFence(lang, mode, policy = 'auto') {
|
|
|
45
102
|
return policy === 'opt-in' ? mode !== undefined : true;
|
|
46
103
|
}
|
|
47
104
|
/**
|
|
48
|
-
* THE fence-info parser. `js`, `css#anchor`, `js:iframe`, `ts:ide#demo`, `ts#demo:ide
|
|
105
|
+
* THE fence-info parser. `js`, `css#anchor`, `js:iframe`, `ts:ide#demo`, `ts#demo:ide`,
|
|
106
|
+
* `tjs {"runTests": "report"}`.
|
|
49
107
|
*
|
|
50
108
|
* `:mode` (inline | iframe | ide | static) sets the live example's execution mode; `#id` gives
|
|
51
109
|
* it a stable anchor. `#id` is `[A-Za-z0-9_-]+` and `:mode` is `[a-z]+`, so the two cannot
|
|
52
110
|
* overlap and each is parsed independently, order-free.
|
|
53
111
|
*
|
|
112
|
+
* A JSON object after the head is the example's **options**, passed to its dialect (#184). It
|
|
113
|
+
* is split off FIRST: `{"debug":true}` contains `:true`, which the mode pattern would
|
|
114
|
+
* otherwise read as a mode. Options that are not a JSON object are reported in
|
|
115
|
+
* `optionsError` rather than dropped, so an author sees why their options did nothing.
|
|
116
|
+
*
|
|
54
117
|
* Extracted because a SECOND, worse copy existed in `save-to-source.ts` — a regex that
|
|
55
118
|
* captured the language as `[\w-]*` and therefore stopped dead at the colon, so it could not
|
|
56
119
|
* see `:static` at all. With a `:static` fence in a document, its example ordinals diverged
|
|
57
120
|
* from the ones `insert-examples` assigns, and an edit saved over a DIFFERENT block than the
|
|
58
121
|
* one edited. Silently: the "couldn't locate this example" guard only fires when the ordinal
|
|
59
|
-
* is out of range, and here a group existed at that index
|
|
122
|
+
* is out of range, and here a group existed at that index.
|
|
60
123
|
*
|
|
61
124
|
* This is the same failure the `isLiveFence` docblock above describes, one layer down: the
|
|
62
125
|
* rule was stated twice and the copies disagreed. Parse fence info here or not at all.
|
|
63
126
|
*/
|
|
64
127
|
export function parseFenceInfo(info) {
|
|
65
128
|
const text = String(info || '');
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
129
|
+
const brace = text.indexOf('{');
|
|
130
|
+
const head = brace === -1 ? text : text.slice(0, brace);
|
|
131
|
+
const parsed = {
|
|
132
|
+
lang: head.match(/^[a-z]+/)?.[0] ?? '',
|
|
133
|
+
mode: head.match(/:([a-z]+)/)?.[1],
|
|
134
|
+
id: head.match(/#([A-Za-z0-9_-]+)/)?.[1],
|
|
70
135
|
};
|
|
136
|
+
if (brace !== -1) {
|
|
137
|
+
const json = text.slice(brace).trim();
|
|
138
|
+
try {
|
|
139
|
+
const value = JSON.parse(json);
|
|
140
|
+
if (value && typeof value === 'object' && !Array.isArray(value)) {
|
|
141
|
+
parsed.options = value;
|
|
142
|
+
}
|
|
143
|
+
else {
|
|
144
|
+
parsed.optionsError = `fence options must be a JSON object, got ${json}`;
|
|
145
|
+
}
|
|
146
|
+
}
|
|
147
|
+
catch (error) {
|
|
148
|
+
parsed.optionsError = `fence options are not valid JSON (${error.message}): ${json}`;
|
|
149
|
+
}
|
|
150
|
+
}
|
|
151
|
+
return parsed;
|
|
71
152
|
}
|
|
@@ -21,7 +21,11 @@ export declare function registeredGrammars(): string[];
|
|
|
21
21
|
export declare function grammarFor(fenceLang: string): string;
|
|
22
22
|
/** Grammars this build can load — for tests, diagnostics, and the docs. */
|
|
23
23
|
export declare function loadableGrammars(): string[];
|
|
24
|
-
/**
|
|
24
|
+
/**
|
|
25
|
+
* The fence language of a `<code class="language-…">`, lowercased, or `''` — for GRAMMAR
|
|
26
|
+
* lookups. The pattern itself is `languageOfClass` in `example-policy.ts`, the only place it
|
|
27
|
+
* is written; ask `isLiveFence` with the case-preserved `languageOfClass`, not with this.
|
|
28
|
+
*/
|
|
25
29
|
export declare function langOfClass(className: string | null | undefined): string;
|
|
26
30
|
export declare function ensureGrammar(lang: string): Promise<boolean>;
|
|
27
31
|
/**
|
|
@@ -24,7 +24,7 @@ it would deepen exactly the dependency that issue wants to escape. Prism's core
|
|
|
24
24
|
grammars load per language, which also covers the languages a prose or book corpus uses and
|
|
25
25
|
CodeMirror does not bundle (shell, python, rust, json, yaml, diff).
|
|
26
26
|
*/
|
|
27
|
-
import { isLiveFence } from './example-policy.js';
|
|
27
|
+
import { isLiveFence, languageOfClass, LANGUAGE_NAME, } from './example-policy.js';
|
|
28
28
|
/** Languages Prism has built in — no grammar file to load. */
|
|
29
29
|
const BUILTIN = new Set([
|
|
30
30
|
'markup',
|
|
@@ -185,17 +185,12 @@ export function loadableGrammars() {
|
|
|
185
185
|
return Object.keys(GRAMMARS).sort();
|
|
186
186
|
}
|
|
187
187
|
/**
|
|
188
|
-
*
|
|
189
|
-
*
|
|
190
|
-
*
|
|
191
|
-
* `c++` or `c#` fence was read as a language by one and as `c` by the other — the same block
|
|
192
|
-
* classified two ways by two passes over the same document. `+` and `#` are in the class
|
|
193
|
-
* deliberately: they are real language names, not stray punctuation.
|
|
188
|
+
* The fence language of a `<code class="language-…">`, lowercased, or `''` — for GRAMMAR
|
|
189
|
+
* lookups. The pattern itself is `languageOfClass` in `example-policy.ts`, the only place it
|
|
190
|
+
* is written; ask `isLiveFence` with the case-preserved `languageOfClass`, not with this.
|
|
194
191
|
*/
|
|
195
|
-
const LANGUAGE_CLASS = /language-([A-Za-z0-9_+#-]+)/;
|
|
196
|
-
/** The fence language of a `<code class="language-…">`, lowercased, or `''`. */
|
|
197
192
|
export function langOfClass(className) {
|
|
198
|
-
return (
|
|
193
|
+
return languageOfClass(className).toLowerCase();
|
|
199
194
|
}
|
|
200
195
|
let prism;
|
|
201
196
|
let warnedNoPrism = false;
|
|
@@ -449,7 +444,7 @@ export async function highlightBlocks(root, opts = {}) {
|
|
|
449
444
|
.filter((code) => {
|
|
450
445
|
const el = code;
|
|
451
446
|
const mode = el.parentElement?.getAttribute('data-example-mode') ?? undefined;
|
|
452
|
-
return !isLiveFence(
|
|
447
|
+
return !isLiveFence(languageOfClass(el.className), mode, policy);
|
|
453
448
|
});
|
|
454
449
|
if (blocks.length === 0)
|
|
455
450
|
return 0;
|
|
@@ -499,7 +494,9 @@ function decodeEntities(s) {
|
|
|
499
494
|
/** Every fence language present in rendered markdown — what grammars a page needs. */
|
|
500
495
|
export function languagesIn(html) {
|
|
501
496
|
return [
|
|
502
|
-
...new Set([
|
|
497
|
+
...new Set([
|
|
498
|
+
...html.matchAll(new RegExp(`<code class="language-(${LANGUAGE_NAME})"`, 'g')),
|
|
499
|
+
].map((m) => m[1].toLowerCase())),
|
|
503
500
|
];
|
|
504
501
|
}
|
|
505
502
|
/**
|
|
@@ -526,10 +523,10 @@ export async function highlightHtml(html, policy = 'auto') {
|
|
|
526
523
|
example reads its SOURCE out of this element, and tokenizing it hands the example markup
|
|
527
524
|
where it expected code.
|
|
528
525
|
*/
|
|
529
|
-
return html.replace(
|
|
526
|
+
return html.replace(new RegExp(`(<pre([^>]*)>)<code class="language-(${LANGUAGE_NAME})">([\\s\\S]*?)</code>`, 'g'), (whole, preTag, preAttrs, lang, body) => {
|
|
530
527
|
const l = lang.toLowerCase();
|
|
531
528
|
const mode = preAttrs.match(/data-example-mode="([a-z]+)"/)?.[1];
|
|
532
|
-
if (isLiveFence(
|
|
529
|
+
if (isLiveFence(lang, mode, policy))
|
|
533
530
|
return whole;
|
|
534
531
|
if (!available.has(l))
|
|
535
532
|
return whole;
|
|
@@ -29,12 +29,18 @@ docMarked.use({
|
|
|
29
29
|
const id = parsed.id ?? '';
|
|
30
30
|
const mode = parsed.mode ?? '';
|
|
31
31
|
const bake = currentBakes?.get(token.text);
|
|
32
|
-
|
|
32
|
+
// Fence options (#184) ride to the page as JSON in an attribute; a parse error rides
|
|
33
|
+
// too, so the page can say which example it belongs to (the build does not know).
|
|
34
|
+
const options = parsed.options && escapeAttr(JSON.stringify(parsed.options));
|
|
35
|
+
const optionsError = parsed.optionsError && escapeAttr(parsed.optionsError);
|
|
36
|
+
if (!id && !mode && !bake && !options && !optionsError)
|
|
33
37
|
return false; // default rendering — byte-identical
|
|
34
38
|
let html = baseRenderer.code({ ...token, lang });
|
|
35
39
|
const attrs = [
|
|
36
40
|
id && `data-example-id="${id}"`,
|
|
37
41
|
mode && `data-example-mode="${mode}"`,
|
|
42
|
+
options && `data-example-options="${options}"`,
|
|
43
|
+
optionsError && `data-example-options-error="${optionsError}"`,
|
|
38
44
|
]
|
|
39
45
|
.filter(Boolean)
|
|
40
46
|
.join(' ');
|
|
@@ -63,6 +69,9 @@ function slugify(s) {
|
|
|
63
69
|
function escapeHtml(s) {
|
|
64
70
|
return s.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');
|
|
65
71
|
}
|
|
72
|
+
function escapeAttr(s) {
|
|
73
|
+
return escapeHtml(s).replace(/"/g, '"');
|
|
74
|
+
}
|
|
66
75
|
// Per-parse footnote state. `parse()` is synchronous and Marked tokenizes fully
|
|
67
76
|
// before rendering, so a module-level object is safe: preprocess resets it, the
|
|
68
77
|
// tokenizers fill it (defs during the block pass, refs during inline — refs set
|
|
@@ -23,7 +23,7 @@ Build-time only (bun). Never import from browser code.
|
|
|
23
23
|
*/
|
|
24
24
|
import { marked } from 'marked';
|
|
25
25
|
import { rewriteImports, AsyncFunction, loadTransform, transformAvailable, UnsupportedImportError, } from '../../live-example/code-transform.js';
|
|
26
|
-
import { isLiveFence } from '../example-policy.js';
|
|
26
|
+
import { isLiveFence, parseFenceInfo, } from '../example-policy.js';
|
|
27
27
|
// The default live-example context (matches the IIFE globals the pages provide).
|
|
28
28
|
// A project that sets a custom `context` on its <tosi-doc-system> can pass its
|
|
29
29
|
// own keys; these are the tosijs-ui defaults.
|
|
@@ -54,15 +54,11 @@ const EXECUTABLE = new Set(['js', 'tjs', 'ts', 'test']);
|
|
|
54
54
|
// scope is the difference between "once" and "once per rebuild". Lazily created, so a
|
|
55
55
|
// corpus with no `ts` examples never makes one at all.
|
|
56
56
|
let tsTranspiler;
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
}
|
|
61
|
-
|
|
62
|
-
/** The `:<mode>` suffix of a fence info string, if any — `js:static` -> `static`. */
|
|
63
|
-
function modeOf(info) {
|
|
64
|
-
return String(info ?? '').match(/:([a-z]+)/)?.[1];
|
|
65
|
-
}
|
|
57
|
+
/*
|
|
58
|
+
Fence info is parsed by `parseFenceInfo`, the one copy of that grammar. This file had its own
|
|
59
|
+
`dialectOf`/`modeOf` pair; once fences could carry JSON options (#184) `modeOf` would have read
|
|
60
|
+
`{"debug":true}` as the mode `true`.
|
|
61
|
+
*/
|
|
66
62
|
function collectCodeTokens(text) {
|
|
67
63
|
const out = [];
|
|
68
64
|
const walk = (tokens) => {
|
|
@@ -75,9 +71,9 @@ function collectCodeTokens(text) {
|
|
|
75
71
|
*/
|
|
76
72
|
if (t.type === 'code')
|
|
77
73
|
out.push({
|
|
78
|
-
lang:
|
|
74
|
+
lang: parseFenceInfo(t.lang ?? '').lang,
|
|
79
75
|
text: t.text,
|
|
80
|
-
mode:
|
|
76
|
+
mode: parseFenceInfo(t.lang ?? '').mode,
|
|
81
77
|
});
|
|
82
78
|
if (Array.isArray(t.tokens))
|
|
83
79
|
walk(t.tokens);
|
|
@@ -17,12 +17,12 @@ Build-time only (Bun APIs + the `zip` CLI); never import from browser code.
|
|
|
17
17
|
import * as fs from 'fs';
|
|
18
18
|
import * as path from 'path';
|
|
19
19
|
import { renderDocMarkdown } from '../render.js';
|
|
20
|
-
import { highlightHtml
|
|
20
|
+
import { highlightHtml } from '../highlight.js';
|
|
21
21
|
import { buildSlugMap, pathForSlug, slugForPath, withBase } from '../routing.js';
|
|
22
22
|
import { buildNavTree } from '../nav-tree.js';
|
|
23
23
|
import { partitionByBook, DEFAULT_BOOK } from '../book-target.js';
|
|
24
24
|
import { epubVolumeIdentity } from './epub-volumes.js';
|
|
25
|
-
import { isLiveFence } from '../example-policy.js';
|
|
25
|
+
import { isLiveFence, languageOfClass } from '../example-policy.js';
|
|
26
26
|
import { DEFAULT_BOOK_CSS, stripDocMeta, flatten, slugify, } from '../book-html.js';
|
|
27
27
|
import { selectBookDocs } from '../book-manifest.js';
|
|
28
28
|
// Re-exported for back-compat (tosijs-ui/site's public surface + tests).
|
|
@@ -159,9 +159,10 @@ function exampleLangOf(pre, policy) {
|
|
|
159
159
|
}
|
|
160
160
|
}
|
|
161
161
|
const cls = (code && code.getAttribute('class')) || '';
|
|
162
|
-
// `
|
|
163
|
-
// fence was read as `c` here and correctly there, classifying one block two ways.
|
|
164
|
-
|
|
162
|
+
// `languageOfClass` is THE pattern — this file used to spell it `[\w-]+`, so a `c++` / `c#`
|
|
163
|
+
// fence was read as `c` here and correctly there, classifying one block two ways. Case
|
|
164
|
+
// preserved, because the live-fence rule is case-sensitive.
|
|
165
|
+
const lang = languageOfClass(cls);
|
|
165
166
|
if (!lang)
|
|
166
167
|
return null;
|
|
167
168
|
// The `:mode` suffix rides on the <pre> as `data-example-mode` — the same place
|
|
@@ -19,3 +19,4 @@ export { listEpubVolumes, epubVolumeIdentity, renderEpubDownloads, type EpubVolu
|
|
|
19
19
|
export { currentHolder, describeHolder, lockPathFor, lockDecision, isProcessAlive, } from './build-lock.js';
|
|
20
20
|
export type { LockHolder, LockDecision } from './build-lock.js';
|
|
21
21
|
export { registerGrammar, registeredGrammars, highlightHtml, highlightBlocks, grammarFor, } from '../highlight.js';
|
|
22
|
+
export { registerLiveLanguage } from '../example-policy.js';
|
|
@@ -38,3 +38,6 @@ the ePub and print — not just a hydrated browser tab. That matters more than u
|
|
|
38
38
|
wrong token colour is cosmetic on the web and permanent in a printed book.
|
|
39
39
|
*/
|
|
40
40
|
export { registerGrammar, registeredGrammars, highlightHtml, highlightBlocks, grammarFor, } from '../highlight.js';
|
|
41
|
+
// For a site whose registered dialect (#184) shares its name with a highlighter grammar: call
|
|
42
|
+
// this in the site config so the build treats those fences as live examples too.
|
|
43
|
+
export { registerLiveLanguage } from '../example-policy.js';
|