punctilio 3.1.2 → 3.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +61 -61
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
>
|
|
1
|
+
> _punctilio_ (n.): precise observance of formalities.
|
|
2
2
|
|
|
3
3
|
[](https://github.com/alexander-turner/punctilio/actions/workflows/test.yml)
|
|
4
4
|
[](https://github.com/alexander-turner/punctilio/actions/workflows/lint.yml)
|
|
@@ -6,16 +6,16 @@
|
|
|
6
6
|
|
|
7
7
|
Pretty good at making your text pretty. The most feature-complete and reliable English typography package. `punctilio` transforms plain ASCII into typographically correct Unicode, even across HTML element boundaries.
|
|
8
8
|
|
|
9
|
-
**Smart quotes** · **Em/en dashes** · **Ellipses** · **Math symbols** · **Legal symbols** · **Arrows** · **Primes** · **Fractions** · **Superscripts** · **Ligatures** · **Non-breaking spaces** · **HTML-aware** · **Bri’ish, German, and French localisation support**
|
|
9
|
+
**Smart quotes** · **Em/en dashes** · **Ellipses** · **Math symbols** · **Legal symbols** · **Arrows** · **Primes** · **Fractions** · **Superscripts** · **Ligatures** · **Non-breaking spaces** · **HTML-aware** · **Markdown support** · **Bri’ish, German, and French localisation support**
|
|
10
10
|
|
|
11
11
|
```typescript
|
|
12
|
-
import { transform } from
|
|
12
|
+
import { transform } from "punctilio";
|
|
13
13
|
|
|
14
|
-
transform(
|
|
14
|
+
transform(`"It's a beautiful thing, the destruction of words..." -- 1984`);
|
|
15
15
|
// → “It’s a beautiful thing, the destruction of words…”—1984
|
|
16
16
|
```
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
`punctilio` accepts three input formats: text, Markdown, and HTML.
|
|
19
19
|
|
|
20
20
|
```bash
|
|
21
21
|
npm install punctilio
|
|
@@ -25,52 +25,52 @@ npm install punctilio
|
|
|
25
25
|
|
|
26
26
|
As far as I can tell, `punctilio` is the most reliable and feature-complete. I built `punctilio` for [my website](https://turntrout.com/design). I wrote[^wrote] and sharpened the core regexes sporadically over several months, exhaustively testing edge cases. Eventually, I decided to spin off the functionality into its own package.
|
|
27
27
|
|
|
28
|
-
[^wrote]: While Claude is the number one contributor to this repository, that’s because Claude helped me port my existing code and added some features. The core regular expressions (e.g. dashes, quotes, multiplication signs) are human-written and were quite delicate. Those numerous commits don’t show in this repo’s history.
|
|
28
|
+
[^wrote]: While Claude is the number one contributor to this repository, that’s because Claude helped me port my existing code and added some features. The core regular expressions (e.g. dashes, quotes, multiplication signs) are human-written and were quite delicate. Those numerous commits don’t show in this repo’s history.
|
|
29
29
|
|
|
30
30
|
I tested `punctilio` 1.2.9 against [`smartypants`](https://www.npmjs.com/package/smartypants) 0.2.2, [`tipograph`](https://www.npmjs.com/package/tipograph) 0.7.4, [`smartquotes`](https://www.npmjs.com/package/smartquotes) 2.3.2, [`typograf`](https://www.npmjs.com/package/typograf) 7.6.0, and [`retext-smartypants`](https://www.npmjs.com/package/retext-smartypants) 6.2.0.[^python] These other packages have spotty feature coverage and inconsistent impact on text. For example, `smartypants` mishandles quotes after em dashes (though quite hard to see in GitHub’s font) and lacks multiplication sign support.
|
|
31
31
|
|
|
32
32
|
[^python]: The Python libraries I found were closely related to the JavaScript packages. I tested them and found similar scores, so I don’t include separate Python results.
|
|
33
33
|
|
|
34
|
-
|
|
|
35
|
-
|
|
36
|
-
| 5x5 |
|
|
34
|
+
| Input | `smartypants` | `punctilio` |
|
|
35
|
+
| :------------------------------------: | :----------------------------------------: | :---------: |
|
|
36
|
+
| <span class="no-formatting">5x5</span> | <span class="no-formatting">5x5</span> (✗) | 5×5 (✓) |
|
|
37
37
|
|
|
38
38
|
My [`benchmark.mjs`](https://github.com/alexander-turner/punctilio/blob/main/benchmark.mjs) measures how well libraries handle a [wide range of scenarios](https://github.com/alexander-turner/punctilio/blob/main/benchmark_cases.json). The benchmark normalizes stylistic differences (e.g. non-breaking vs regular space, British vs American dash spacing) for fair comparison.
|
|
39
39
|
|
|
40
|
-
|
|
|
41
|
-
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
| `retext-smartypants` | 65 (41%)
|
|
48
|
-
|
|
49
|
-
|
|
|
50
|
-
|
|
51
|
-
|
|
|
52
|
-
|
|
|
53
|
-
|
|
|
54
|
-
|
|
|
55
|
-
|
|
|
56
|
-
|
|
|
57
|
-
|
|
|
58
|
-
|
|
|
59
|
-
|
|
|
60
|
-
|
|
|
61
|
-
|
|
|
62
|
-
|
|
|
63
|
-
|
|
|
64
|
-
|
|
|
65
|
-
| English localization | <span class="no-formatting">American / British</span> |
|
|
66
|
-
|
|
|
67
|
-
|
|
|
68
|
-
|
|
|
40
|
+
| Package | Passed (of 157) |
|
|
41
|
+
| -------------------: | :-------------- |
|
|
42
|
+
| `punctilio` | 155 (99%) |
|
|
43
|
+
| `tipograph` | 91 (58%) |
|
|
44
|
+
| `typograf` | 74 (47%) |
|
|
45
|
+
| `smartquotes` | 72 (46%) |
|
|
46
|
+
| `smartypants` | 68 (43%) |
|
|
47
|
+
| `retext-smartypants` | 65 (41%) |
|
|
48
|
+
|
|
49
|
+
| Feature | Example | `punctilio` | `smartypants` | `tipograph` | `smartquotes` | `typograf` |
|
|
50
|
+
| -------------------: | :---------------------------------------------------: | :---------: | :-----------: | :---------: | :-----------: | :--------: |
|
|
51
|
+
| Smart quotes | <span class="no-formatting">"hello" → “hello”</span> | ✓ | ✓ | ✓ | ✓ | ✓ |
|
|
52
|
+
| Leading apostrophe | <span class="no-formatting">'Twas → ’Twas</span> | ✓ | ✗ | ✗ | ◐ | ✗ |
|
|
53
|
+
| Em dash | <span class="no-formatting">-- → —</span> | ✓ | ✓ | ✗ | ✗ | ✓ |
|
|
54
|
+
| En dash (ranges) | <span class="no-formatting">1-5 → 1–5</span> | ✓ | ✗ | ✓ | ✗ | ✗ |
|
|
55
|
+
| Minus sign | <span class="no-formatting">-5 → −5</span> | ✓ | ✗ | ✓ | ✗ | ✗ |
|
|
56
|
+
| Ellipsis | <span class="no-formatting">... → …</span> | ✓ | ✓ | ✓ | ✗ | ✓ |
|
|
57
|
+
| Multiplication | <span class="no-formatting">5x5 → 5×5</span> | ✓ | ✗ | ✗ | ✗ | ◐ |
|
|
58
|
+
| Math symbols | <span class="no-formatting">!= → ≠</span> | ✓ | ✗ | ◐ | ✗ | ◐ |
|
|
59
|
+
| Legal symbols | <span class="no-formatting">(c) 2004 → © 2004</span> | ✓ | ✗ | ◐ | ✗ | ✓ |
|
|
60
|
+
| Arrows | <span class="no-formatting">-> → →</span> | ✓ | ✗ | ◐ | ✗ | ◐ |
|
|
61
|
+
| Prime marks | <span class="no-formatting">5'10" → 5′10″</span> | ✓ | ✗ | ✓ | ✓ | ✗ |
|
|
62
|
+
| Degrees | <span class="no-formatting">20 C → 20 °C</span> | ✓ | ✗ | ✗ | ✗ | ✓ |
|
|
63
|
+
| Fractions | <span class="no-formatting">1/2 → ½</span> | ✓ | ✗ | ✗ | ✗ | ✓ |
|
|
64
|
+
| Superscripts | <span class="no-formatting">2nd → 2ⁿᵈ</span> | ✓ | ✗ | ✗ | ✗ | ✗ |
|
|
65
|
+
| English localization | <span class="no-formatting">American / British</span> | ✓ | ✗ | ✗ | ✗ | ✗ |
|
|
66
|
+
| Ligatures | <span class="no-formatting">?? → ⁇</span> | ✓ | ✗ | ✓ | ✗ | ✗ |
|
|
67
|
+
| Non-English quotes | <span class=”no-formatting”>„Hallo”</span> | ✓ | ✗ | ✓ | ✗ | ◐ |
|
|
68
|
+
| Non-breaking spaces | <span class="no-formatting">Chapter 1</span> | ✓ | ✗ | ✗ | ✗ | ✓ |
|
|
69
69
|
|
|
70
70
|
### Known limitations of `punctilio`
|
|
71
71
|
|
|
72
|
-
| Pattern
|
|
73
|
-
|
|
72
|
+
| Pattern | Behavior | Notes |
|
|
73
|
+
| :--------------------- | :------------------------- | :------------------------------------------------------------------------------------------- |
|
|
74
74
|
| `'99 but 5' clearance` | `5'` not converted to `5′` | Leading apostrophe is indistinguishable from an opening quote without semantic understanding |
|
|
75
75
|
|
|
76
76
|
## Test suite
|
|
@@ -86,16 +86,16 @@ Other typography libraries take one of two approaches, both with drawbacks.
|
|
|
86
86
|
1. String-based libraries (like [`smartypants`](https://www.npmjs.com/package/smartypants)) transform plain text but are unaware of HTML structure. If you concatenate text from `<em>Wait</em>...`, transform it into `Wait…`, and then try to convert back—you've lost track of where the `</em>` belongs.
|
|
87
87
|
2. AST-based libraries (like [`rehype-retext`](https://github.com/rehypejs/rehype-retext)) process each text node individually, preserving structure but losing cross-node information. A quote that opens inside `<em>"Wait</em>` and closes outside it `..."` spans two text node. Processed independently, the library can't tell whether the final `"` is opening or closing, because it never sees both at once.
|
|
88
88
|
|
|
89
|
-
`punctilio` introduces
|
|
89
|
+
`punctilio` introduces _separation boundaries_ to get the best of both worlds:
|
|
90
90
|
|
|
91
91
|
1. Flatten the parent container's contents to a string, delimiting element boundaries with a private-use Unicode character (`U+E000`) to avoid unintended matches.
|
|
92
92
|
2. Every regex allows (and preserves) these characters, treating them as boundaries of a “permeable membrane” through which contextual information flows. For example, `.U+E000..` still becomes `…U+E000`.
|
|
93
|
-
3. Rehydrate the HTML AST. For all
|
|
93
|
+
3. Rehydrate the HTML AST. For all _k_, set element _k_’s text content to the segment starting at separator occurrence _k_.
|
|
94
94
|
|
|
95
95
|
```typescript
|
|
96
|
-
import { transform, DEFAULT_SEPARATOR } from
|
|
96
|
+
import { transform, DEFAULT_SEPARATOR } from "punctilio";
|
|
97
97
|
|
|
98
|
-
transform(`"Wait${DEFAULT_SEPARATOR}"`)
|
|
98
|
+
transform(`"Wait${DEFAULT_SEPARATOR}"`);
|
|
99
99
|
// → `“Wait”${DEFAULT_SEPARATOR}`
|
|
100
100
|
// The separator doesn’t block the information that this should be an end-quote!
|
|
101
101
|
```
|
|
@@ -103,13 +103,13 @@ transform(`"Wait${DEFAULT_SEPARATOR}"`)
|
|
|
103
103
|
For `rehype` / `unified` pipelines, use the built-in plugin which handles the separator logic automatically:
|
|
104
104
|
|
|
105
105
|
```typescript
|
|
106
|
-
import rehypePunctilio from
|
|
106
|
+
import rehypePunctilio from "punctilio/rehype";
|
|
107
107
|
|
|
108
108
|
unified()
|
|
109
109
|
.use(rehypeParse)
|
|
110
110
|
.use(rehypePunctilio)
|
|
111
111
|
.use(rehypeStringify)
|
|
112
|
-
.process('<p><em>"Wait</em>..." -- she said</p>')
|
|
112
|
+
.process('<p><em>"Wait</em>..." -- she said</p>');
|
|
113
113
|
// → <p><em>“Wait</em>…”—she said</p>
|
|
114
114
|
// The opening quote inside <em> and the closing quote outside it
|
|
115
115
|
// are both resolved correctly across the element boundary.
|
|
@@ -117,26 +117,26 @@ unified()
|
|
|
117
117
|
|
|
118
118
|
For Markdown ASTs via `remark`, use `remarkPunctilio` which applies the same separator technique to preserve inline element boundaries, or use `transformMarkdown` for a simpler Markdown-to-Markdown pipeline.
|
|
119
119
|
|
|
120
|
-
For manual DOM walking or custom transforms, use `transformElement` from `punctilio/rehype`.
|
|
120
|
+
For manual DOM walking or custom transforms, use `transformElement` from `punctilio/rehype`.
|
|
121
121
|
|
|
122
122
|
## Options
|
|
123
123
|
|
|
124
|
-
`punctilio` doesn't enable all transformations by default. Fractions and degrees tend to match too aggressively (perfectly applying the degree transformation requires semantic meaning). Superscript letters and punctuation ligatures have spotty font support. Furthermore, `ligatures = true` can change the meaning of text by collapsing question and exclamation marks.
|
|
124
|
+
`punctilio` doesn't enable all transformations by default. Fractions and degrees tend to match too aggressively (perfectly applying the degree transformation requires semantic meaning). Superscript letters and punctuation ligatures have spotty font support. Furthermore, `ligatures = true` can change the meaning of text by collapsing question and exclamation marks.
|
|
125
125
|
|
|
126
126
|
```typescript
|
|
127
127
|
transform(text, {
|
|
128
|
-
punctuationStyle:
|
|
129
|
-
dashStyle:
|
|
130
|
-
|
|
131
|
-
symbols: true,
|
|
132
|
-
collapseSpaces: true,
|
|
133
|
-
fractions: false,
|
|
134
|
-
degrees: false,
|
|
135
|
-
superscript: false,
|
|
136
|
-
ligatures: false,
|
|
137
|
-
nbsp: true,
|
|
138
|
-
checkIdempotency: true,
|
|
139
|
-
})
|
|
128
|
+
punctuationStyle: "american" | "british" | "german" | "french" | "none", // default: 'american'
|
|
129
|
+
dashStyle: "american" | "british" | "none", // default: 'american'
|
|
130
|
+
|
|
131
|
+
symbols: true, // ellipsis, math, legal, arrows
|
|
132
|
+
collapseSpaces: true, // normalize whitespace
|
|
133
|
+
fractions: false, // 1/2 → ½
|
|
134
|
+
degrees: false, // 20 C → 20 °C
|
|
135
|
+
superscript: false, // 1st → 1ˢᵗ
|
|
136
|
+
ligatures: false, // ??? → ⁇, ?! → ⁈, !? → ⁉, !!! → !
|
|
137
|
+
nbsp: true, // non-breaking spaces (after honorifics, between numbers and units, etc.)
|
|
138
|
+
checkIdempotency: true, // verify transform(transform(x)) === transform(x)
|
|
139
|
+
});
|
|
140
140
|
```
|
|
141
141
|
|
|
142
142
|
- Fully general prime mark conversion (e.g. `5'10"` → `5′10″`) requires semantic understanding to distinguish from closing quotes (e.g. `"Term 1"` should produce closing quotes). `punctilio` counts quotes to heuristically guess whether the matched number at the end of a quote (if not, it requires a prime mark). Other libraries like `tipograph` 0.7.4 use simpler patterns that make more mistakes.
|
|
@@ -148,7 +148,7 @@ transform(text, {
|
|
|
148
148
|
- Spaced en-dashes between words (word – word)
|
|
149
149
|
- The `german` style uses low-9 quotes: „double” (U+201E/U+201C) and ‚single' (U+201A/U+2018).
|
|
150
150
|
- Punctuation outside quotes
|
|
151
|
-
- The `french` style uses guillemets with non-breaking space padding:
|
|
151
|
+
- The `french` style uses guillemets with non-breaking space padding: « Bonjour ».
|
|
152
152
|
- Single quotes remain as curly quotes
|
|
153
153
|
- Punctuation outside quotes
|
|
154
154
|
- Setting either style to `none` skips the entire transform category: `punctuationStyle: 'none'` preserves straight quotes, apostrophes, and prime marks; `dashStyle: 'none'` preserves all hyphens, number ranges, date ranges, and minus signs.
|