polytypo 1.17.1 → 1.17.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. checksums.yaml +4 -4
  2. data/lib/polytypo/data/VERSION +1 -1
  3. data/lib/polytypo/data/fixtures/bg.json +1 -1
  4. data/lib/polytypo/data/fixtures/cs.json +1 -1
  5. data/lib/polytypo/data/fixtures/de-CH.json +1 -1
  6. data/lib/polytypo/data/fixtures/de-DE.json +9 -26
  7. data/lib/polytypo/data/fixtures/el.json +1 -1
  8. data/lib/polytypo/data/fixtures/en-GB.json +172 -1
  9. data/lib/polytypo/data/fixtures/en-US.json +181 -516
  10. data/lib/polytypo/data/fixtures/es.json +1 -1
  11. data/lib/polytypo/data/fixtures/fi.json +1 -1
  12. data/lib/polytypo/data/fixtures/fr-CA.json +1 -1
  13. data/lib/polytypo/data/fixtures/fr.json +48 -37
  14. data/lib/polytypo/data/fixtures/hr.json +1 -1
  15. data/lib/polytypo/data/fixtures/hu.json +1 -1
  16. data/lib/polytypo/data/fixtures/it.json +1 -1
  17. data/lib/polytypo/data/fixtures/locale-resolution.json +1 -1
  18. data/lib/polytypo/data/fixtures/lt.json +1 -1
  19. data/lib/polytypo/data/fixtures/nl.json +1 -1
  20. data/lib/polytypo/data/fixtures/pl.json +1 -1
  21. data/lib/polytypo/data/fixtures/pt-BR.json +1 -1
  22. data/lib/polytypo/data/fixtures/pt-PT.json +1 -1
  23. data/lib/polytypo/data/fixtures/ro.json +1 -1
  24. data/lib/polytypo/data/fixtures/ru.json +1 -1
  25. data/lib/polytypo/data/fixtures/sk.json +1 -1
  26. data/lib/polytypo/data/fixtures/sl.json +1 -1
  27. data/lib/polytypo/data/fixtures/sr.json +1 -1
  28. data/lib/polytypo/data/fixtures/sv.json +1 -1
  29. data/lib/polytypo/data/fixtures/tr.json +1 -1
  30. data/lib/polytypo/data/fixtures/uk.json +1 -1
  31. data/lib/polytypo/data/locales/registry.json +1 -1
  32. data/lib/polytypo/data/rules/modes.md +124 -9
  33. data/lib/polytypo/data/rules/order.json +1 -1
  34. data/lib/polytypo/modes/markdown.rb +109 -5
  35. data/lib/polytypo/modes/runner.rb +13 -4
  36. data/lib/polytypo/modes/spans.rb +45 -3
  37. data/lib/polytypo/version.rb +1 -1
  38. metadata +1 -1
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "es",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "fi",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "fr-CA",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "fr",
4
4
  "cases": [
5
5
  {
@@ -752,9 +752,7 @@
752
752
  "rule": "nbsp",
753
753
  "mode": "markdown",
754
754
  "dialect": "commonmark",
755
- "frontmatterKeys": [
756
- "title"
757
- ],
755
+ "frontmatterKeys": ["title"],
758
756
  "in": "---\ntitle: \"Chapitre 1: le début\"\ndate: \"2026-09-25\"\n---\n\nVoici: le corps.\n",
759
757
  "out": "---\ntitle: \"Chapitre 1 : le début\"\ndate: \"2026-09-25\"\n---\n\nVoici : le corps.\n",
760
758
  "note": "modes.md §3.7.4 with the locale that made §3.7.3 skip the block in the first place: inside a listed quoted scalar a colon is content, so fr inserts its narrow no-break space, while the unlisted date field and the key itself keep every byte. That contrast is the whole argument for a caller-named list."
@@ -863,9 +861,7 @@
863
861
  "id": "fr-yaml-narrow-space-in-quoted",
864
862
  "rule": "nbsp",
865
863
  "mode": "yaml",
866
- "keys": [
867
- "description"
868
- ],
864
+ "keys": ["description"],
869
865
  "in": "description: \"Une note !\"\n",
870
866
  "out": "description: \"Une note !\"\n",
871
867
  "note": "modes.md §3.8.6: a quoted scalar's content is one span, and French spacing applies inside it exactly as in text mode."
@@ -874,9 +870,7 @@
874
870
  "id": "fr-yaml-no-space-across-colon-split",
875
871
  "rule": "nbsp",
876
872
  "mode": "yaml",
877
- "keys": [
878
- "description"
879
- ],
873
+ "keys": ["description"],
880
874
  "in": "description: prose:more prose\n",
881
875
  "out": "description: prose:more prose\n",
882
876
  "note": "modes.md §3.8.6: the `:` is outside every span, so the insertion `nbsp` would make before it lands on a span edge and is discarded by §3.4. This is the accepted cost of the split, and it is what keeps the scalar parseable."
@@ -885,9 +879,7 @@
885
879
  "id": "fr-yaml-guillemets-across-block-lines",
886
880
  "rule": "quotes",
887
881
  "mode": "yaml",
888
- "keys": [
889
- "description"
890
- ],
882
+ "keys": ["description"],
891
883
  "in": "description: |\n Il a dit \"bonjour\n monsieur\" et il est parti\n",
892
884
  "out": "description: |\n Il a dit « bonjour\n monsieur » et il est parti\n",
893
885
  "note": "modes.md §3.2 model C in a locale whose quotation output is guillemets plus U+00A0 rather than curly quotes — structurally different from the en-US case of the same shape."
@@ -953,9 +945,7 @@
953
945
  "id": "fr-yaml-plain-elision",
954
946
  "rule": "apostrophe",
955
947
  "mode": "yaml",
956
- "keys": [
957
- "description"
958
- ],
948
+ "keys": ["description"],
959
949
  "in": "description: l'idee et l'autre\n",
960
950
  "out": "description: l’idee et l’autre\n",
961
951
  "note": "modes.md §3.8.6 in a locale where the apostrophe is elision rather than possession: the same cell as en-us-yaml-plain-apostrophe, and §8 item 4's requirement that a construct be pinned in more than one locale rather than the same English output with the locale field changed."
@@ -964,9 +954,7 @@
964
954
  "id": "fr-yaml-single-quoted-escape-elision-stays",
965
955
  "rule": "apostrophe",
966
956
  "mode": "yaml",
967
- "keys": [
968
- "description"
969
- ],
957
+ "keys": ["description"],
970
958
  "in": "description: 'l''idee et l''autre'\n",
971
959
  "out": "description: 'l''idee et l''autre'\n",
972
960
  "note": "modes.md §3.8.6: in a locale where the apostrophe is elision, the unit is the elision, and it is outside every span. The scalar has nothing else to convert and comes back unchanged — the cost §7.13 measures, pinned so that it is a documented property and not a surprise."
@@ -975,9 +963,7 @@
975
963
  "id": "fr-yaml-single-quoted-escape-split",
976
964
  "rule": "dashes",
977
965
  "mode": "yaml",
978
- "keys": [
979
- "description"
980
- ],
966
+ "keys": ["description"],
981
967
  "in": "description: 'l''idee -- et ''l''autre...'\n",
982
968
  "out": "description: 'l''idee — et ''l''autre…'\n",
983
969
  "note": "modes.md §3.8.6: a spaced dash and an ellipsis interior to a span convert, either side of units; the elisions stay straight."
@@ -986,9 +972,7 @@
986
972
  "id": "fr-yaml-single-quoted-escape-growth-declined",
987
973
  "rule": "dashes",
988
974
  "mode": "yaml",
989
- "keys": [
990
- "description"
991
- ],
975
+ "keys": ["description"],
992
976
  "in": "description: 'a''--''b'\n",
993
977
  "out": "description: 'a''--''b'\n",
994
978
  "note": "modes.md §3.8.6 and §3.4: `fr` is a `-spaced` locale, so the dash would grow from 2 to 3 code points against a unit. The edge-growth rule discards it, as it does against `:` and `#`. The en-US case with the same input is applied, which is what makes this one discriminate."
@@ -997,9 +981,7 @@
997
981
  "id": "fr-yaml-single-quoted-no-escape",
998
982
  "rule": "dashes",
999
983
  "mode": "yaml",
1000
- "keys": [
1001
- "description"
1002
- ],
984
+ "keys": ["description"],
1003
985
  "in": "description: 'deux -- trois'\n",
1004
986
  "out": "description: 'deux — trois'\n",
1005
987
  "note": "modes.md §3.8.6: a single-quoted scalar with nothing doubled inside it is claimed like any other. The style itself never disables the mode — only the doubling does, which is what the neighbouring case pins."
@@ -1018,10 +1000,7 @@
1018
1000
  "rule": "quotes",
1019
1001
  "mode": "markdown",
1020
1002
  "dialect": "commonmark",
1021
- "frontmatterKeys": [
1022
- "title",
1023
- "slug"
1024
- ],
1003
+ "frontmatterKeys": ["title", "slug"],
1025
1004
  "in": "---\ntitle: \"a\rb\"\nslug: c \"d\"\n---\n\nBody \"q\" here.\n",
1026
1005
  "out": "---\ntitle: \"a\rb\"\nslug: c « d »\n---\n\nBody « q » here.\n",
1027
1006
  "note": "modes.md §3.7.4 per-line U+000D bail in a locale whose quotation marks carry their own spacing, so the declined line and the converted one are visibly different constructions rather than the same English output relabelled (§8 item 4)."
@@ -1030,11 +1009,7 @@
1030
1009
  "id": "fr-yaml-lone-cr-costs-one-line",
1031
1010
  "rule": "quotes",
1032
1011
  "mode": "yaml",
1033
- "keys": [
1034
- "title",
1035
- "slug",
1036
- "summary"
1037
- ],
1012
+ "keys": ["title", "slug", "summary"],
1038
1013
  "in": "title: a \"b\"\nslug: c\rd \"e\"\n",
1039
1014
  "out": "title: a « b »\nslug: c\rd \"e\"\n",
1040
1015
  "note": "modes.md §3.8.4 step 9 in a locale whose quotation marks carry their own spacing: the line with the lone U+000D is declined and the line before it converts, so the cost is one line and not the document."
@@ -1078,6 +1053,42 @@
1078
1053
  }
1079
1054
  ],
1080
1055
  "note": "modes.md §3.9.4, accepted cost: the narrow space would be inserted at a span edge next to the hole, which the edge-growth rule (§3.4) discards. Without protect it is inserted."
1056
+ },
1057
+ {
1058
+ "id": "fr-markdown-commonmark-delimiter-space-insertion",
1059
+ "rule": "nbsp",
1060
+ "mode": "markdown",
1061
+ "dialect": "commonmark",
1062
+ "in": "*a.**! et mot !\n",
1063
+ "out": "*a.**! et mot !\n",
1064
+ "note": "modes.md §3.4 delimiter-neighbour rule: a space inserted before the exclamation mark would make the double asterisk a closing delimiter that pairs with the first asterisk; that edit is discarded. The same insertion after a word (mot !) is applied."
1065
+ },
1066
+ {
1067
+ "id": "fr-markdown-commonmark-delimiter-tilde-insertion",
1068
+ "rule": "nbsp",
1069
+ "mode": "markdown",
1070
+ "dialect": "commonmark",
1071
+ "in": "Voir ~25 et « ~25 et «~25 enfin\n",
1072
+ "out": "Voir ~25 et « ~25 et «~25 enfin\n",
1073
+ "note": "modes.md §3.4 delimiter-neighbour rule, tilde as a delimiter: inserting a no-break space between the opening guillemet and a tilde is declined, because the tilde could then open strikethrough; the spaced form (« ~25) is already correct. An accepted cost: the rule tests adjacency to whitespace, not CommonMark's full flanking rules."
1074
+ },
1075
+ {
1076
+ "id": "fr-markdown-commonmark-autolink-literal-terminator-nbsp",
1077
+ "rule": "nbsp",
1078
+ "mode": "markdown",
1079
+ "dialect": "commonmark",
1080
+ "in": "Voir https://example.com/a : oui, et voilà : non\n",
1081
+ "out": "Voir https://example.com/a : oui, et voilà : non\n",
1082
+ "note": "modes.md §3.7.3 terminator guard, accepted cost: a no-break space is not U+0020 and GFM parsers differ on whether it ends a URL, so the space after the URL is left as typed; the one after a word converts."
1083
+ },
1084
+ {
1085
+ "id": "fr-markdown-commonmark-autolink-literal-email-terminator-converts",
1086
+ "rule": "nbsp",
1087
+ "mode": "markdown",
1088
+ "dialect": "commonmark",
1089
+ "in": "Écrire à foo@example.com ; oui\n",
1090
+ "out": "Écrire à foo@example.com ; oui\n",
1091
+ "note": "modes.md §3.7.3 terminator guard: it applies to http, https and www addresses only, so the space after an e-mail address converts as before."
1081
1092
  }
1082
1093
  ]
1083
1094
  }
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "hr",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "hu",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "it",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "cases": [
4
4
  {
5
5
  "id": "exact-en-us",
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "lt",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "nl",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "pl",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "pt-BR",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "pt-PT",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "ro",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "ru",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "sk",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "sl",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "sr",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "sv",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "tr",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "locale": "uk",
4
4
  "cases": [
5
5
  {
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "$comment": "Locale resolution input. Algorithm is specified in spec/rules/locale-resolution.md and is identical in every runtime — never delegate it to a platform locale-negotiation library. \"spec\" here must track spec/VERSION exactly — it is not itself the global version source; scripts/validate-spec.mjs enforces the match.",
4
4
  "locales": [
5
5
  "en-US",
@@ -19,7 +19,7 @@ step 9 with §3.8.5's content-line condition, added in 1.10.0, §3.8.4's carriag
19
19
  continuation line with the bails of §3.8.6 that use it, added in 1.11.0, and §3.8.4's node lines,
20
20
  declined keys, node markers and own-line region, with the changes to steps 1, 3, 4, 5, 6, 7 and 8
21
21
  that go with them, added in 1.11.0, and §3.9 with the amendments it carries to §3.1, §3.5 step 2,
22
- §4, §5, §7 item 7 and §8, added in 1.13.0).
22
+ §4, §5, §7 item 7 and §8, added in 1.13.0, and §3.4's delimiter-neighbour rule with §3.7.3's autolink-literal tail, terminator guard and reference labels, and the amendments they carry to §3.4's deletion paragraph, §5 items 2 and 3, its fifth-part note and testing obligation, and §6, added in 1.17.2).
23
23
 
24
24
  ---
25
25
 
@@ -356,9 +356,13 @@ Together they distinguish exactly the cases that matter:
356
356
  | U+0020 → U+00A0 (`nbsp`) | yes | 1 → 1 | **applied** — U+00A0 is not U+0020 |
357
357
  | insertion of U+202F (`nbsp` N1/N2) | yes | 0 → 1 | **discarded**, as before |
358
358
 
359
- **Deletion at an edge is not restricted**, and does not need to be. A rule can only delete
360
- U+0020 (`spaces`), and a deletion cannot bring into existence a structural token that was not
361
- already there — it can only remove a character from inside a span. The whitespace that _is_
359
+ **Deletion at an edge is not restricted by the edge-growth rule**, and does not need to be. A rule
360
+ can only delete U+0020 (`spaces`), and a deletion cannot bring into existence a structural token
361
+ that was not already there — it can only remove a character from inside a span. **That sentence
362
+ was true of every token the parser had already recognised, and false of one the parser had
363
+ declined to recognise.** An unpaired `_`, `*` or `~` is literal text inside a span, and whether it
364
+ is a delimiter depends on what stands beside it; deleting a space is enough to change that, and so
365
+ is inserting or replacing one. The delimiter-neighbour rule below closes it. The whitespace that _is_
362
366
  structural in Markdown — a line prefix, a list marker's indentation, the two spaces of a hard
363
367
  break — is protected by the −2 line marker (§3.2): it sits beside a `BREAK`, and `spaces`
364
368
  refuses to touch a run bordering one, in every mode.
@@ -371,6 +375,43 @@ fixed point, but it no longer demonstrates anything: `dashes` §3.2 step 2a decl
371
375
  containing an authored U+2013 or U+2014, so that input is now refused a step earlier and never
372
376
  reaches the edge-growth rule. `--` is the live carrier.)
373
377
 
378
+ **The delimiter-neighbour rule (`markdown` only, spec 1.17.2).** In `markdown` mode the three
379
+ code points `DELIM` = U+002A, U+005F, U+007E are literal content wherever the parser found no
380
+ delimiter pair, and CommonMark decides whether a run of them can open or close emphasis from the
381
+ code points on either side of it: whitespace or not. Measured, with the pre-1.17.2 behaviour:
382
+
383
+ | Mode | Input | Locale | Output before 1.17.2 | What changed |
384
+ | ---------- | ------------------------------------ | ------- | ----------------------------- | ---------------------------------------------------------------------------------- |
385
+ | `markdown` | `see booking._ , emails.booking_ here` | `en-GB` | `see booking._, emails.booking_ here` | `spaces` deleted the space; the two underscores now pair as emphasis |
386
+ | `markdown` | `x _ -- y_` | `en-US` | `x _—y_` | `dashes` collapsed `␣--␣`, removing the space beside the underscore; same result |
387
+ | `markdown` | `*a.**!` | `fr` | `*a.**` U+202F `!` | `nbsp` inserted a narrow no-break space before the `!`; the double asterisk now closes and pairs |
388
+
389
+ Write `W` for the set of Unicode whitespace in the CommonMark sense (U+0009, U+000A, U+000C,
390
+ U+000D, U+0020, U+00A0, U+1680, U+2000–U+200A, U+202F, U+205F, U+3000). For an edit replacing
391
+ `cp[p … q]` with `w` of length `r` (an insertion has `q = p − 1`), evaluated on the array the rule
392
+ read:
393
+
394
+ - if `cp[p−1]` is in `DELIM`, let `old = cp[p]` and `new = w[0]` when `r > 0`, otherwise `cp[q+1]`;
395
+ - if `cp[q+1]` is in `DELIM`, let `old = cp[q]` and `new = w[r−1]` when `r > 0`, otherwise `cp[p−1]`.
396
+
397
+ > **The edit is discarded if, on either side, exactly one of `old` and `new` is in `W`.** A marker
398
+ > belongs to neither class, a position outside the array is neither, and the test reads no other
399
+ > code point and no parser state.
400
+
401
+ It applies to `markdown` and to no other mode: `text` has no emphasis, and `html` has no
402
+ delimiters in that sense. It is a function of the array alone, so it fires identically on every
403
+ run (§5 item 3). It is deliberately a test of **adjacency to whitespace**, not a reimplementation
404
+ of CommonMark's flanking rules, and it is therefore stricter than flanking requires: in `fr`,
405
+ `«~25` (U+00AB U+007E …) keeps its tight form although the tilde would flank the same way after a
406
+ no-break space. That is accepted: a delimiter beside a space that a rule would change is rare in
407
+ prose.
408
+
409
+ **Its premise is that no rule swaps punctuation for non-punctuation beside a delimiter.** CommonMark
410
+ 0.31 counts symbols as punctuation, so `(c)` → `©` and `+/-` → `±` keep a delimiter's class, and
411
+ the rule tests whitespace alone. A parser that implements an older definition (punctuation only)
412
+ would classify them differently, and two fixtures (`en-us-markdown-commonmark-delimiter-neighbour-symbols-kept`)
413
+ make such a port fail rather than drift. No rule emits a member of `DELIM`.
414
+
374
415
  ### 3.5 Applying the result
375
416
 
376
417
  1. Extract the span sequence from the source. Record each span's **source offsets**.
@@ -515,6 +556,44 @@ Skipped, exhaustively:
515
556
  unless tables are recognised at all, and §6's `[P: html, markdown]` marking on `dashes`' URL
516
557
  bullet holds **only** because GFM autolink literals make a bare `https://…` a link — in pure
517
558
  CommonMark a bare URL is ordinary text and would be typeset;
559
+ - **the tail of a bare web address (spec 1.17.2).** Wherever the source text of a **maximal span**
560
+ contains `http://`, `https://` or `www.` (ASCII letters in either case, **with no test on the
561
+ character before it**: a parser may read a link directly after a digit), everything from there up to — and **not** including — the first U+0020,
562
+ U+0009, U+000A, U+000D or `<`, or the end of the span, is skipped. The same holds from the
563
+ position where a parser-recognised GFM autolink literal that begins with one of those prefixes
564
+ ends, up to the same point.
565
+ This is a scan of the source and not a question put to a parser, because parsers disagree about
566
+ where such an address may start (`"www.example.com"`: GFM's own tokenizer reads no link there,
567
+ mdast's autolink transform reads one) and where it ends. The rules would otherwise change what
568
+ ends it: a closing `"` that becomes U+201D, an apostrophe that becomes U+2019, or `...` that
569
+ becomes U+2026 is not ASCII punctuation, and the parser reading the _output_ takes it into the
570
+ link destination. That is damage to the link, not to the typography. A quotation mark that cannot
571
+ convert leaves its partner unpaired, so the pair is left as typed rather than half-converted.
572
+ **An e-mail address is not covered**: it ends where the parser says, and no rule changes that.
573
+ A **maximal span** is the run of processable text with no source between its pieces: the pieces a
574
+ parser reports separately (an unpaired `_` or `*` is one) are **merged before this scan and before
575
+ §3.4's delimiter-neighbour rule reads its neighbours**. §7 item 5's 'should' is a 'must' in
576
+ `markdown`, since both depend on it;
577
+ - **the terminator of a bare web address (spec 1.17.2).** A span that begins at the U+0020 or
578
+ U+0009 that ends such an address discards every edit that starts at its first code point, unless
579
+ the replacement begins with U+0020. Collapsing `␣--␣` to a closed-up dash would delete the space
580
+ and let the address run on into the dash and the next word, and a no-break space is not
581
+ guaranteed to end an address in every GFM parser. Edits that keep a U+0020 there are applied:
582
+ `Read https://example.com/a - b` still becomes `Read https://example.com/a – b` in `en-GB`;
583
+ - **the label of a reference (spec 1.17.2).** In `[Smith's book]` and `[Smith's book][]` the link
584
+ text is also the label matched against `[Smith's book]: …`, and that label is skipped (above).
585
+ Converting the text makes it match nothing and the link disappears, so the text of a **shortcut**
586
+ or **collapsed** reference is skipped as well, and so is the alt text of such an image. A reference
587
+ is a bracketed run that the parser formed from a definition: **shortcut** when its source ends at
588
+ the `]` of the text, **collapsed** when `[]` follows it directly, **full** when a label in `[…]`
589
+ follows. Labels match as CommonMark §4.7 matches them, with no locale-dependent case mapping.
590
+ Square brackets that match no definition are not a reference and their text converts; the text of a full
591
+ reference, `[text][ref]`, is prose and converts. The label of a **footnote call** `[^x]` and of a
592
+ **footnote definition** `[^x]:` is skipped for the same reason; the 1.17.1 text above already said
593
+ so for the definition, and the JavaScript engine did not do it (measured on 1.17.1). **Accepted:** text that matched no
594
+ definition in the input and matches one after conversion (`[it's]` with `[it’s]: …` elsewhere)
595
+ becomes a link; closing that direction needs the definitions compared, and it creates a link
596
+ rather than destroying one;
518
597
  - HTML blocks and inline raw HTML, which are handed to the `html` skip list of §3.6 rather than
519
598
  processed as markdown. **Inline HTML arrives as isolated tags with Markdown between them**,
520
599
  not as a tree, so §3.6's subtree rule must be implemented as a **stack of open skipped
@@ -1661,7 +1740,7 @@ they do not carry it over for free either. Write `M` for the whole mode transfor
1661
1740
  indented code. `dashes` emits U+0020 in every `-spaced` locale. The claim was wrong when it
1662
1741
  was written, and it is what hid the defect now repaired in §3.4.
1663
1742
 
1664
- The honest discharge has four parts, and the third is **positional rather than alphabetic** —
1743
+ The honest discharge has four parts (a fifth, for what the parser decides, was added in 1.17.2 and follows them), and the third is **positional rather than alphabetic** —
1665
1744
  the same move as CO-S in `pipeline-idempotency.md` §5.1a, applied to _where_ a character may
1666
1745
  be placed rather than to _which_ characters exist:
1667
1746
 
@@ -1675,11 +1754,22 @@ they do not carry it over for free either. Write `M` for the whole mode transfor
1675
1754
  - **U+0020 is made safe by position, not by inertness.** The edge-growth rule (§3.4) means a
1676
1755
  rule can introduce a code point only at a position **strictly interior** to a span. An
1677
1756
  interior position has span content on both sides, so an emitted U+0020 can never become
1678
- adjacent to a delimiter — delimiters are structural tokens, outside every span — and can
1757
+ adjacent to a **recognised** delimiter — those are structural tokens, outside every span — and can
1679
1758
  never begin a line, because a line terminator is always a −2 marker and a line start is
1680
1759
  therefore a span edge or outside spans entirely. **The characters that decide Markdown
1681
1760
  structure and the positions a rule may write to are disjoint sets.** That is what makes the
1682
1761
  partition stable; no property of U+0020 itself is relied upon.
1762
+ - **A delimiter the parser did not recognise is the exception, and it is closed by §3.4's
1763
+ delimiter-neighbour rule (spec 1.17.2).** The sentence above holds for tokens outside every
1764
+ span. An unpaired `*`, `_` or `~` is _inside_ a span as literal text, and a rule that changes
1765
+ whether whitespace stands beside it can make it a delimiter: this was a false statement until
1766
+ 1.17.2, and §3.4's delimiter-neighbour rule is the repair. The same is true of the end of a bare URL, which only
1767
+ the parser decides, and of a reference label, which only a definition elsewhere decides — §3.7.3
1768
+ skips the tail of an autolink literal and the text of a shortcut or collapsed reference for the
1769
+ same reason. **This list is not claimed to be complete.** A deletion
1770
+ can also create a backslash escape or a character reference from literal text (`foo\ ,`
1771
+ becomes `foo\,` and `AT&amp ;T` becomes `AT&amp;T`): the parse tree keeps its shape and its
1772
+ text changes, which a shape comparison does not see. That class is not closed by 1.17.2.
1683
1773
  - **Deletion** is only ever of a U+0020, and structural whitespace borders a `BREAK` — a real
1684
1774
  one in `text` mode, a −2 marker in the others — where `spaces` refuses to act
1685
1775
  (`spaces.md` §3.2 step 4). A deletion therefore cannot turn `- item` into `-item`, collapse
@@ -1720,7 +1810,9 @@ they do not carry it over for free either. Write `M` for the whole mode transfor
1720
1810
 
1721
1811
  > **The fifth part, normative and binding on future modes: the predicate that decides whether
1722
1812
  > a span exists must not read anything a rule can change.** In `html` and `markdown` the
1723
- > question never arises, because span selection reads only the document's structure. `yaml`'s
1813
+ > question does not arise for the span _predicate_, because span selection reads only the
1814
+ > document's structure. It arose for the span's _neighbourhood_, which a parser also reads: see
1815
+ > the delimiter-neighbour rule and §3.7.3's autolink tail. `yaml`'s
1724
1816
  > first draft decided a plain scalar's processability from **its own text** — it had to contain
1725
1817
  > a space and a letter — and `spaces` can delete that space. One line was enough to falsify the
1726
1818
  > obligation: `ref: ${{ steps.pin.outputs.sha }}` yields one span, comes back as
@@ -1748,8 +1840,10 @@ they do not carry it over for free either. Write `M` for the whole mode transfor
1748
1840
  3. **Redistribution is deterministic.** Guaranteed by §3.4: no edit spans a marker, and any edit
1749
1841
  that would place code points at a span extremity which were not there before — an insertion at
1750
1842
  a boundary, or a replacement longer than what it replaced — is discarded rather than assigned
1751
- to a side. The discard is a pure function of `(p, q, r, s₀, s₁)`, so it happens identically on
1752
- every run: a rule that was declined at an edge once is declined there always, and the output
1843
+ to a side. The discard is a pure function of the array and its span table (since 1.17.2: of
1844
+ `(p, q, r, s₀, s₁)`, of the code points beside the edit for §3.4's delimiter-neighbour rule, and
1845
+ of whether the span begins at the end of a web address for §3.7.3's terminator guard), so it
1846
+ happens identically on every run: a rule that was declined at an edge once is declined there always, and the output
1753
1847
  contains nothing for the next run to reconsider.
1754
1848
 
1755
1849
  Given 1–3, `M(M(x)) = M(x)`.
@@ -1774,6 +1868,21 @@ the single-unit templates cannot reach. §3.8.7 records what each run did and di
1774
1868
  structure and types is blind to a string whose content is damaged, and that an alphabet without
1775
1869
  `---` reported clean twice while §3.4's `r = d` hole was open.
1776
1870
 
1871
+ Since 1.17.2 the `markdown` run must compare, between the input and the output, **the parse
1872
+ tree including link destinations and reference identifiers**, not only the span partition: the
1873
+ defect behind the delimiter-neighbour rule changed the tree while the partition looked the same,
1874
+ and a sweep that compares spans alone cannot see it. Its alphabet must include `*`, `_`, `~`, a
1875
+ space, U+002D, U+0022, U+0027, `.`, `!` and `:`, and its templates must include a **bare URL
1876
+ followed by each of those**, and a reference link with its definition. Run it in locales whose
1877
+ `dashes` collapses a spaced dash (`en-US`), whose `nbsp` inserts before `!` (`fr`), and whose
1878
+ quotes are curly, because each reaches a different rule. The tree to compare is **mdast's, as `remark-parse` with `remark-gfm` produces it** (and, where a
1879
+ runtime has another parser, that parser's as well), not only the tokenizer's own: the tokenizer reads
1880
+ no link in `"www.example.com"`, mdast does, and that is the class §3.7.3's scan closes. Templates
1881
+ must include an address with `_`, `*` and `~` inside it after `"`, `'` and `(`, and an address
1882
+ directly after a digit. **Two classes are carved out because they are accepted, not because
1883
+ they are untested**: a text that matched no definition in the input and matches one in the output
1884
+ (§3.7.3's accepted case), and an e-mail address, whose end only the parser decides.
1885
+
1777
1886
  ---
1778
1887
 
1779
1888
  ## 6. Which §4 claims change
@@ -1827,6 +1936,12 @@ means the claim holds in those modes only.
1827
1936
  No rule's §4 becomes _false_ in a mode. Two become narrower ([P: html, markdown]); none becomes
1828
1937
  rule-local.
1829
1938
 
1939
+ **Since 1.17.2 every rule's reach in `markdown` is narrower in three further places**, none of which
1940
+ makes a §4 statement false: beside an unpaired `*`, `_` or `~` where an edit would change whether
1941
+ whitespace stands next to it (§3.4); in the tail of a bare web address and at the space that ends it
1942
+ (§3.7.3); and in the text of a shortcut or collapsed reference and in footnote labels (§3.7.3). In
1943
+ each, the rule is given fewer chances to fire and the text is left as typed.
1944
+
1830
1945
  ---
1831
1946
 
1832
1947
  ## 7. Open questions
@@ -1,5 +1,5 @@
1
1
  {
2
- "spec": "1.17.1",
2
+ "spec": "1.17.3",
3
3
  "$comment": "Single source of truth for pipeline order. Rules run in ascending `order`. Disabling a rule removes it from the sequence and never reorders the rest. Rule ids are public API (see docs/ARCHITECTURE.md section 5). \"spec\" here must track spec/VERSION exactly — it is not itself the global version source; scripts/validate-spec.mjs enforces the match.",
4
4
  "rules": [
5
5
  {