polytypo 1.9.0 → 1.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/lib/polytypo/data/VERSION +1 -1
- data/lib/polytypo/data/fixtures/cs.json +1 -1
- data/lib/polytypo/data/fixtures/de-CH.json +1 -1
- data/lib/polytypo/data/fixtures/de-DE.json +1 -1
- data/lib/polytypo/data/fixtures/el.json +1 -1
- data/lib/polytypo/data/fixtures/en-GB.json +1 -1
- data/lib/polytypo/data/fixtures/en-US.json +1010 -3
- data/lib/polytypo/data/fixtures/es.json +1 -1
- data/lib/polytypo/data/fixtures/fi.json +1 -1
- data/lib/polytypo/data/fixtures/fr-CA.json +1 -1
- data/lib/polytypo/data/fixtures/fr.json +14 -1
- data/lib/polytypo/data/fixtures/it.json +1 -1
- data/lib/polytypo/data/fixtures/locale-resolution.json +1 -1
- data/lib/polytypo/data/fixtures/nl.json +1 -1
- data/lib/polytypo/data/fixtures/pl.json +1 -1
- data/lib/polytypo/data/fixtures/pt-BR.json +1 -1
- data/lib/polytypo/data/fixtures/pt-PT.json +1 -1
- data/lib/polytypo/data/fixtures/ru.json +1 -1
- data/lib/polytypo/data/fixtures/sk.json +278 -0
- data/lib/polytypo/data/fixtures/sv.json +1 -1
- data/lib/polytypo/data/fixtures/tr.json +1 -1
- data/lib/polytypo/data/fixtures/uk.json +1 -1
- data/lib/polytypo/data/locales/registry.json +3 -2
- data/lib/polytypo/data/locales/sk.json +94 -0
- data/lib/polytypo/data/rules/modes.md +206 -57
- data/lib/polytypo/data/rules/order.json +1 -1
- data/lib/polytypo/modes/yaml.rb +179 -30
- data/lib/polytypo/version.rb +1 -1
- metadata +3 -1
|
@@ -7,14 +7,18 @@ for all five runtimes and is parser-agnostic by construction: `parse5`, `nokogir
|
|
|
7
7
|
`golang.org/x/net/html` and PHP's DOM disagree about almost everything this document does not
|
|
8
8
|
forbid them from doing. `yaml` mode is parser-**free** rather than parser-agnostic, for the
|
|
9
9
|
reason §3.8.1 measures.
|
|
10
|
-
**Spec version:** 1.
|
|
10
|
+
**Spec version:** 1.11.0 (0.1.0 for everything except §3.3's class-membership table rows for
|
|
11
11
|
`nbsp` and `apostrophe`, split in 1.2.0, §3.8, added in 1.3.0, §3.3's note on `quotes`'
|
|
12
12
|
span-boundary elision veto reading the marker as a trigger, added in 1.4.0 — which changes no
|
|
13
13
|
row of the table it follows — §3.3's `CLOSEDELIM` entry, added in 1.5.0, and §3.7.4 with the
|
|
14
14
|
amendments it carries to §3.1's definitions, §3.2's Model C, §3.5 step 3, §3.7.3's frontmatter
|
|
15
15
|
bullet, §3.8.6's accepted-cost paragraph, §5, §6 and §7, added in 1.7.0, §3.7.3a, added in
|
|
16
16
|
1.8.0, and §3.8.6's treatment of `''` in a single-quoted scalar, which replaces the bail it had
|
|
17
|
-
until 1.9.0, with the matching amendments to §3.7.4.1, §3.8.7, §5, §7.13 and §8
|
|
17
|
+
until 1.9.0, with the matching amendments to §3.7.4.1, §3.8.7, §5, §7.13 and §8, and §3.8.4
|
|
18
|
+
step 9 with §3.8.5's content-line condition, added in 1.10.0, §3.8.4's carriage-return region, which replaces step 9's decline of a line carrying a U+000D, added in 1.11.0, §3.8.4 step 7's definition of a
|
|
19
|
+
continuation line with the bails of §3.8.6 that use it, added in 1.11.0, and §3.8.4's node lines,
|
|
20
|
+
declined keys, node markers and own-line region, with the changes to steps 1, 3, 4, 5, 6, 7 and 8
|
|
21
|
+
that go with them, added in 1.11.0).
|
|
18
22
|
|
|
19
23
|
---
|
|
20
24
|
|
|
@@ -719,17 +723,21 @@ already recognises, and the option only changes what happens inside it:
|
|
|
719
723
|
content the scan is given, and a span reaching a declined position is dropped whole rather than
|
|
720
724
|
trimmed.** Three ports reached for the shortcut of substituting the U+000D for another
|
|
721
725
|
character the scan already declines, and it is not equivalent: substitution moves the character
|
|
722
|
-
to a different line and can
|
|
723
|
-
converts both keys
|
|
724
|
-
|
|
726
|
+
to a different line and can change a value run, so `title` / `slug` / a content-final U+000D,
|
|
727
|
+
which converts both keys, would silence `slug` (a substituted character makes that last line
|
|
728
|
+
non-blank, hence a continuation line, §3.8.4 step 7), and a stray U+000D on a key whose value
|
|
729
|
+
run continues onto an indented line converts a key that is not a key. Both shapes are pinned, and both agree with
|
|
725
730
|
what `yaml` mode already does with the same characters — measured in two shipped runtimes.
|
|
726
731
|
**Where this rule and `yaml` mode part is on purpose, and it is one shape:** a content line
|
|
727
732
|
whose terminator is a bare U+000D is clean to `yaml` mode, which strips it, and declined here,
|
|
728
|
-
because the window includes it. So `title` / `slug`
|
|
729
|
-
`yaml` mode and does not here. The wider decline is the direction this section takes everywhere
|
|
733
|
+
because the window includes it. So `title` / `slug` with `slug`'s line ending in a bare U+000D
|
|
734
|
+
converts `slug` in `yaml` mode and does not here (measured, 1.11.0). The wider decline is the direction this section takes everywhere
|
|
730
735
|
else — a miss is invisible, a U+000D inside a span is not — and the cost is conversions lost in
|
|
731
|
-
documents written with line endings from the last century.
|
|
732
|
-
|
|
736
|
+
documents written with line endings from the last century. Since 1.10.0 `yaml` mode declines a
|
|
737
|
+
line containing a lone U+000D itself (§3.8.4's carriage-return region), so the two paths agree on a stray U+000D
|
|
738
|
+
inside a line; they still part on the shape named just above. It does
|
|
739
|
+
so by declining rather than by widening §3.8.4's line model to YAML's or CommonMark's, which
|
|
740
|
+
would change what `yaml` mode converts for every caller;
|
|
733
741
|
- **both delimiter lines stay outside every span**, as does every line terminator, so no edit
|
|
734
742
|
can reach `---` itself and §3.7.3's setext-underline hazard is unreachable;
|
|
735
743
|
- an **unterminated** block is not a block — §3.7.3 already yields no frontmatter construct
|
|
@@ -758,8 +766,8 @@ processed under `frontmatterKeys: ["title"]` — with the cost §7.12 already ac
|
|
|
758
766
|
paths, no globs, so a caller with a machine-read `title` nested somewhere must either take it too
|
|
759
767
|
or name none. Matching is exact, code point for code point, with no case folding.
|
|
760
768
|
|
|
761
|
-
**The scan is §3.8's, unchanged.** §3.8.4 steps 1–9, §3.8.5 and
|
|
762
|
-
verbatim, with `frontmatterKeys` as step 8's key predicate in place of `keys`. Nothing about YAML
|
|
769
|
+
**The scan is §3.8's, unchanged.** §3.8.4 steps 1–9, its carriage-return region, §3.8.5 and
|
|
770
|
+
§3.8.6 apply to the block content verbatim, with `frontmatterKeys` as step 8's key predicate in place of `keys`. Nothing about YAML
|
|
763
771
|
is specified twice: frontmatter **is** YAML, and the argument that made §3.8 a hand-written scan
|
|
764
772
|
rather than a parser call (§3.8.1 — two of the five ecosystems' libraries cannot report an end
|
|
765
773
|
offset) applies here for the same reason and with the same measurements. Every miss §7.11 lists
|
|
@@ -956,8 +964,8 @@ a key matches iff its source text, with trailing U+0020 removed, equals a member
|
|
|
956
964
|
point for code point. No case folding — ARCHITECTURE.md §4.4 forbids locale-dependent case
|
|
957
965
|
operations and Turkish dotless ı is the standing reason. No paths, no globs, no wildcards, no
|
|
958
966
|
nesting: a key named `description` is processable wherever it occurs and at any depth. A **quoted
|
|
959
|
-
key never matches**, because §3.8.4 step 5
|
|
960
|
-
cost, recorded in §7.11.
|
|
967
|
+
key never matches**, because §3.8.4 step 5 reads it as a declined key or as a node line, and neither
|
|
968
|
+
is ever compared; that is an accepted cost, recorded in §7.11.
|
|
961
969
|
|
|
962
970
|
**The option also removes a defect the content test carried**, which is why nothing of it
|
|
963
971
|
survives. A content test is a predicate on the text inside a span, and `spaces` can delete the
|
|
@@ -988,8 +996,10 @@ throw this mode adds is the option throw of §3.8.2, which is about the call, no
|
|
|
988
996
|
|
|
989
997
|
The source is the code-point array of §3.1. A **line** is a maximal run containing no U+000A,
|
|
990
998
|
**and a U+000D immediately before that U+000A is not part of the line** — it is a terminator like
|
|
991
|
-
the U+000A itself, so it lies outside every span and comes back untouched.
|
|
992
|
-
|
|
999
|
+
the U+000A itself, so it lies outside every span and comes back untouched. **The same holds for a
|
|
1000
|
+
U+000D that ends the input with no U+000A after it.** Line terminators are never inside a span,
|
|
1001
|
+
so no rule can create or destroy one — and since 1.10.0 that sentence is true of a lone U+000D as
|
|
1002
|
+
well, by the carriage-return region below and §3.8.5, rather than only of the terminators this paragraph defines. `indent` is the number of leading
|
|
993
1003
|
U+0020 on the line and `i` the index of the first code point after them.
|
|
994
1004
|
|
|
995
1005
|
The carriage-return clause is normative rather than obvious, and it is here because five runtimes
|
|
@@ -999,48 +1009,97 @@ and is unrecognised, so the block yields no spans, while a plain scalar carries
|
|
|
999
1009
|
return **inside** its span and hands it to the rules as content.
|
|
1000
1010
|
|
|
1001
1011
|
> **Normative, and the repair of a defect the first draft shipped: a line consumed by a construct
|
|
1002
|
-
> is never scanned again.**
|
|
1003
|
-
>
|
|
1004
|
-
>
|
|
1012
|
+
> is never scanned again.** Three constructs consume lines: an inline value's continuation lines
|
|
1013
|
+
> (step 7), a **node line** — a line that opens a node without being the value of a key (step 5)
|
|
1014
|
+
> — and a document marker followed by content (step 3). Without them a multi-line quoted scalar,
|
|
1015
|
+
> a flow collection, a block scalar header and a folded plain scalar all leak their continuation
|
|
1016
|
+
> lines back into the scan as if they were mappings. The first draft closed only the first of the
|
|
1017
|
+
> three, and 1.11.0 closed the other two (polytypo/polytypo#64): `- |` followed by ` title: c -- d`
|
|
1018
|
+
> converted text that YAML reads as the content of a block scalar.
|
|
1005
1019
|
|
|
1006
1020
|
Lines are processed in order.
|
|
1007
1021
|
|
|
1008
|
-
1.
|
|
1009
|
-
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1015
|
-
|
|
1016
|
-
|
|
1017
|
-
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
|
-
|
|
1021
|
-
|
|
1022
|
-
|
|
1023
|
-
|
|
1024
|
-
|
|
1025
|
-
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
|
|
1029
|
-
|
|
1030
|
-
|
|
1031
|
-
|
|
1032
|
-
|
|
1022
|
+
1. A line that is empty or all U+0020 yields no spans and absorbs nothing. **A line is blank only
|
|
1023
|
+
if it is empty or all U+0020**, here and wherever this section says blank (step 7, §3.8.5): a
|
|
1024
|
+
line containing U+0009 or U+000D is never blank, and a port must not test blankness with a
|
|
1025
|
+
trim that strips either. **A line containing U+0009 anywhere yields no spans, and neither does
|
|
1026
|
+
anything it absorbs** (1.11.0): a tab makes indentation undecidable, which is what every step
|
|
1027
|
+
below depends on, but it does not stop a scalar the line opens from owning the lines under
|
|
1028
|
+
it, so steps 2 to 7 still decide what the line absorbs and the run is then skipped without
|
|
1029
|
+
spans.
|
|
1030
|
+
2. The line yields no spans and absorbs nothing if `cp[i]` is `#` (comment) or `%` (directive).
|
|
1031
|
+
3. A line is a **document marker** if it begins at `i` with `---` or with `...` and the next code
|
|
1032
|
+
point is U+0020 or the line ends there. A marker followed, after any U+0020, by the end of the
|
|
1033
|
+
line or by `#` yields no spans and absorbs nothing. **A marker in column 0 followed by
|
|
1034
|
+
anything else opens a node** (1.11.0): it yields no spans and absorbs every following line up
|
|
1035
|
+
to, but not including, the next line that begins in column 0 with a document marker, or up to
|
|
1036
|
+
the end of the input. An indented one yields no spans and absorbs nothing, since YAML reads a
|
|
1037
|
+
marker only in column 0. The trailing-content form is declined as well as the bare one: `--- key: value`
|
|
1038
|
+
is a node introduced by a document marker, and reading the marker as part of a key is how the
|
|
1039
|
+
first draft produced a key of `--- key`; `--- |` and `--- "x` are nodes whose lines are not
|
|
1040
|
+
mapping lines.
|
|
1041
|
+
4. **Node markers are consumed, not skipped.** While `cp[i]` is `-`, `?` or `:` and the next code
|
|
1042
|
+
point is U+0020 or the line ends there, advance `i` past the marker and past any U+0020 after
|
|
1043
|
+
it. Repeat — `- - key: v` is legal and nests. If `i` reaches the end of the line, or `cp[i]` is
|
|
1044
|
+
`#`, the line yields no spans and absorbs nothing: a marker with nothing after it opens a
|
|
1045
|
+
nested node whose lines are scanned on their own. The line **has a marker** if `i` moved. `?`
|
|
1046
|
+
and `:` are markers since 1.11.0: `? |` and `: |` open a block scalar exactly as `- |` does.
|
|
1047
|
+
5. **Find the key.** Let `first` be `cp[i]`.
|
|
1048
|
+
- If `first` is `"` or `'`, find the quote that closes it on the line. For `'`, `''` is one
|
|
1049
|
+
unit and does not close it. For `"`, a U+005C and the code point after it are skipped together and
|
|
1050
|
+
never close the quote: nothing is decoded, and no YAML escape ends in a code point that
|
|
1051
|
+
would close it, so `"a\"b": 1` is a quoted key. A quote not closed on the line has none. If a closing quote is found at `c`, and after any U+0020 the next code
|
|
1052
|
+
point is a `:` that is the last code point of the line or is followed by U+0020, the line has
|
|
1053
|
+
a **declined key** `cp[i … c]` with that colon, and the scan continues at step 6. Otherwise
|
|
1054
|
+
the line is a **node line**. Any doubt resolves to "node line", which costs a miss; the other
|
|
1055
|
+
error treats a scalar as a key and lets its continuation lines convert.
|
|
1056
|
+
- If `first` is `{`, `[`, `&`, `*` or `!`, the line is a **node line**.
|
|
1057
|
+
- Otherwise scan `j` upward from `i`.
|
|
1058
|
+
- If `cp[j]` is `#` and `cp[j−1]` is U+0020, a comment begins and the line has no key: it is
|
|
1059
|
+
a node line.
|
|
1060
|
+
- If `cp[j]` is `:` **and** `j` is the last code point of the line or `cp[j+1]` is U+0020,
|
|
1061
|
+
the key is `cp[i … j−1]` with trailing U+0020 removed, and the scan continues at step 6.
|
|
1062
|
+
- **Otherwise `j` advances by one and the scan continues**, including when `cp[j]` is a colon
|
|
1063
|
+
that is not followed by U+0020 or the line end — such a colon is an ordinary character of
|
|
1064
|
+
the key, and `a:b: v` has the key `a:b`. Stating the else-branch is not pedantry: without
|
|
1065
|
+
it the first draft admitted two readings of that line, which is the exact divergence
|
|
1066
|
+
§3.8.1 says the scan exists to prevent.
|
|
1067
|
+
- If the scan reached the end of the line without a key, the line is a node line.
|
|
1068
|
+
- A key containing `"`, `'`, `{`, `[`, `&`, `*`, `!` or `#` anywhere — `a!b: value`,
|
|
1069
|
+
`/users/{id}:` — is a **declined key**. The wider reading is the normative one, and saying
|
|
1070
|
+
so is what stops a second implementer from testing only the first position and getting a
|
|
1071
|
+
different span set. The cost is recorded in §7.11.
|
|
1072
|
+
- **A node line yields no spans and absorbs its value run** (defined at step 7). A node line
|
|
1073
|
+
that has no marker (step 4) and at which `first` is `|`, `>`, `{` or `[`, or is a quote with
|
|
1074
|
+
no closing quote on the line, is **own-line**, and absorbs instead every following
|
|
1075
|
+
line up to, but not including, the first one that is non-empty and does not begin with
|
|
1076
|
+
U+0020 — or, when its indent is 0, up to the next line that begins in column 0 with a
|
|
1077
|
+
document marker or the end of the input (1.11.0). The floor is column 0 for the reason the
|
|
1078
|
+
carriage-return region's is: the scan does not track the key above, and a flow collection, a
|
|
1079
|
+
block scalar header or a quote on a line of its own continues at any depth below it — or at
|
|
1080
|
+
its own, for a block scalar and a flow collection.
|
|
1033
1081
|
6. Let `v` be the index of the first code point after the colon that is not U+0020. **If there is
|
|
1034
1082
|
none, the value is empty**: the following, more-indented lines are a nested node and are
|
|
1035
|
-
scanned on their own. The line yields no spans
|
|
1083
|
+
scanned on their own. The line yields no spans and absorbs nothing, a declined key included
|
|
1084
|
+
(`"k":` and `/users/{id}:` open nested mappings); go to the next line.
|
|
1036
1085
|
7. Otherwise the line carries an **inline value**, and the **value run** — every following line
|
|
1037
1086
|
that is blank or indented more than `indent` — belongs to that value. Those lines are never
|
|
1038
1087
|
scanned as lines of their own. The only spans they can contribute are the ones step 9 takes
|
|
1039
1088
|
from a recognised block scalar; every other step below yields no spans for the whole run.
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1089
|
+
**A line of the run that is not blank is a continuation line; a run of blank lines only has
|
|
1090
|
+
none.** Blank lines after a value's last line are trivia — YAML folds them into a plain scalar
|
|
1091
|
+
only when a more-indented line follows — so a file ending in a blank line, or a blank line
|
|
1092
|
+
between two mapping entries, does not make the value above it a continuation (1.11.0). §3.8.6
|
|
1093
|
+
uses the term. **A declined key absorbs its value run only if its value can continue on the
|
|
1094
|
+
next line** (1.11.0): `cp[v]` is `|`, `>`, `{`, `[`, `&` or `!`, or a quote that has no closing
|
|
1095
|
+
quote on the line (as in step 5) or is followed, after the closing quote and any U+0020, by
|
|
1096
|
+
something other than the end of the line or `#`. Any other value of a declined key — plain, alias, comment, a closed
|
|
1097
|
+
quoted scalar — absorbs nothing, because none of them can be continued by a line that has a
|
|
1098
|
+
key shape, so `- "a": 1` / ` title: c -- d` leaves `title` a sibling key and it converts.
|
|
1099
|
+
8. **The key must be listed.** A declined key is never a member of `keys` (§3.8.2), and neither
|
|
1100
|
+
is a key of step 5 that is not listed: the line and its value run yield no spans. This is the
|
|
1101
|
+
step that makes `run:`, `if:` and `image:` unreachable, and it is checked before the value's
|
|
1102
|
+
form so that an unlisted key costs nothing to decline.
|
|
1044
1103
|
9. `cp[v]` selects the form:
|
|
1045
1104
|
- `&`, `*` or `!` — anchor, alias or tag — and `{` or `[` — a flow collection: no spans.
|
|
1046
1105
|
- `#`: the value is absent and only a comment follows: no spans.
|
|
@@ -1048,6 +1107,28 @@ Lines are processed in order.
|
|
|
1048
1107
|
- `"` or `'`: §3.8.6, quoted scalar.
|
|
1049
1108
|
- anything else: §3.8.6, plain scalar.
|
|
1050
1109
|
|
|
1110
|
+
**The carriage-return region (1.10.0, widened in 1.11.0).** A line containing a U+000D yields no
|
|
1111
|
+
spans, and neither does any later line up to, but not including, the first later line that is
|
|
1112
|
+
non-empty, does not begin with U+0020 and contains no U+000D. Any U+000D a line contains is a lone
|
|
1113
|
+
one, since a terminator's is not part of its line (above), and YAML 1.2.2 §5.4 reads it as a line
|
|
1114
|
+
break that this section's LF-only model does not — so it would otherwise land inside a span and be
|
|
1115
|
+
handed to the rules as content, which the opening of this section forbids in terms. The rule is
|
|
1116
|
+
applied to the spans steps 1–9, §3.8.5 and §3.8.6 produce, whatever those steps did with the lines
|
|
1117
|
+
it covers: it removes spans, never adds one, and does not change which lines any value run
|
|
1118
|
+
consumes. **The floor is column 0, not the line's own indent**, because the LF-only scan and YAML
|
|
1119
|
+
agree on nothing between a lone U+000D and the next line in column 0: a key shape can open inside
|
|
1120
|
+
the line that carries it, deeper than the piece after it; the line can sit inside another key's
|
|
1121
|
+
value run and never be visited; and a comment line has no key at all. In 1.10.0 this was step 9,
|
|
1122
|
+
which declined only a line on which a key was found, and so missed `run: |` / ` echo a` /
|
|
1123
|
+
`<CR> x` / ` title: c -- d`, where `<CR> x` has indent 0, ends the unlisted block's value run,
|
|
1124
|
+
and left `title` to be converted inside what YAML reads as a script (polytypo/polytypo#63). The
|
|
1125
|
+
cost is a miss and no more: in a nested document everything up to the next column-0 line is
|
|
1126
|
+
declined although YAML sees real keys there, and a comment, document marker or empty-valued key
|
|
1127
|
+
carrying a U+000D declines the indented lines under it. A model that floors the region at the
|
|
1128
|
+
carrying line's own indent was rejected: it leaves the first shape above open. So was a floor
|
|
1129
|
+
computed per piece, which agrees with column 0 only on nested documents and adds the places five
|
|
1130
|
+
scanners can part (empty and blank pieces, pieces opening with `- `).
|
|
1131
|
+
|
|
1051
1132
|
#### 3.8.5 Block scalars — one span per content line, and the chomping indicator never enters
|
|
1052
1133
|
|
|
1053
1134
|
The header is `cp[v]` followed by **at most one** chomping indicator (`-` or `+`) and **at most
|
|
@@ -1057,10 +1138,14 @@ spans. **The header is never inside a span.**
|
|
|
1057
1138
|
|
|
1058
1139
|
The block's content is the value run of §3.8.4 step 7. Let `bi` be the indentation of its first
|
|
1059
1140
|
non-blank line, or `indent + n` when the header carried an explicit indentation indicator `n`.
|
|
1060
|
-
**One definition, and
|
|
1141
|
+
**One definition, and four conditions that make the whole block yield no spans** — the block is
|
|
1061
1142
|
consumed either way, never rescanned:
|
|
1062
1143
|
|
|
1063
1144
|
- any line of the run contains U+0009;
|
|
1145
|
+
- any line of the run contains a U+000D (§3.8.4's carriage-return region, 1.10.0) — a lone one, since a terminator's
|
|
1146
|
+
is not part of its line. The whole block goes, as for a tab, rather than the one line: the
|
|
1147
|
+
block's lines are one construct, and a carriage return in the middle of it is evidence that
|
|
1148
|
+
the author's line model is not the one this section reads;
|
|
1064
1149
|
- the run's first non-blank line is indented less than `bi`, which can only happen when an
|
|
1065
1150
|
explicit indicator disagrees with the block as written;
|
|
1066
1151
|
- any later non-blank line of the run is indented less than `bi`.
|
|
@@ -1091,6 +1176,13 @@ that line the value yields no spans, and its continuation lines are consumed by
|
|
|
1091
1176
|
rather than rescanned. After the closing quote only U+0020 and an optional `#` comment may
|
|
1092
1177
|
follow. The span is the content between the quotes, exclusive of both.
|
|
1093
1178
|
|
|
1179
|
+
**A quoted scalar whose value run holds a continuation line (§3.8.4 step 7) also yields no
|
|
1180
|
+
spans.** The scan does not decide what a more-indented non-blank line after a closed scalar is —
|
|
1181
|
+
a stray continuation, a comment, or the sibling key of a sequence item whose `indent` counts the
|
|
1182
|
+
spaces before its `- ` — and it claims only what it has proved. Blank lines alone are not a
|
|
1183
|
+
continuation, so `title: "x"` followed by a blank line is scanned like `title: "x"` alone (1.11.0;
|
|
1184
|
+
until then the blank line silenced it).
|
|
1185
|
+
|
|
1094
1186
|
The offset model of §3.1 requires that source characters and content characters be the same thing,
|
|
1095
1187
|
and two constructs break that:
|
|
1096
1188
|
|
|
@@ -1134,8 +1226,9 @@ colon, and applying the plain-scalar test here is what made the first draft decl
|
|
|
1134
1226
|
**Plain.** The scalar runs to the end of the line, minus a trailing comment — the first `#`
|
|
1135
1227
|
preceded by U+0020, and that U+0020 with it — and minus any remaining trailing U+0020. Then:
|
|
1136
1228
|
|
|
1137
|
-
- **the continuation bail**: if the value run of §3.8.4 step 7
|
|
1138
|
-
multi-line plain scalar and yields no spans.
|
|
1229
|
+
- **the continuation bail**: if the value run of §3.8.4 step 7 holds a continuation line, the
|
|
1230
|
+
scalar is a multi-line plain scalar and yields no spans. Trailing blank lines are not one
|
|
1231
|
+
(1.11.0). Its line folding is a construct this scan does not
|
|
1139
1232
|
claim;
|
|
1140
1233
|
- **compact nesting is not a value**: the scalar yields no spans if it begins with `-` followed by
|
|
1141
1234
|
U+0020 or the line end (`key: - item` opens a sequence), or if it contains a colon followed by
|
|
@@ -1219,6 +1312,21 @@ adjacency veto leaves an adjacent pair alone. The split is therefore specified f
|
|
|
1219
1312
|
§3.8.1 gives, that it needs no mapping and puts the pair out of every rule's reach by construction,
|
|
1220
1313
|
and not because the sweep shows the unsplit alternative to be unsafe at payload length four.
|
|
1221
1314
|
|
|
1315
|
+
**A targeted sweep for U+000D (1.10.0).** It is narrower than the sweeps above on purpose —
|
|
1316
|
+
its alphabet carries neither U+005C, `''` nor `---`, which §5 requires of the idempotency sweeps
|
|
1317
|
+
and which those sweeps still carry — because it checks one property, §3.8.4's, that they never
|
|
1318
|
+
tested: **no span contains a line terminator.** Over documents built from the
|
|
1319
|
+
alphabet `a`, U+0020, U+000D, U+000A, `:`, `#`, `'`, `"`, `-` and `|`, in seven templates — a
|
|
1320
|
+
plain, a double-quoted and a single-quoted scalar, a block scalar, an unterminated final line, a
|
|
1321
|
+
payload ahead of a key, and a payload on the continuation line of a CRLF value — at payload
|
|
1322
|
+
lengths 1–5, **777 770 documents**, the scan's spans were read out and tested directly. Before
|
|
1323
|
+
1.10.0, **95 505 of them (12.3%) had a span containing a U+000D**; `title: a<CR>b -- c` was
|
|
1324
|
+
converted with the carriage return inside the span, and `"aaaa<CR>"` was a quoted scalar whose
|
|
1325
|
+
span ended in one. After, **none**. This check is discriminating by construction, since it
|
|
1326
|
+
failed 95 505 times on the engine it was written against. It reads spans rather than output
|
|
1327
|
+
because no rule emits or deletes a U+000D, so the output never showed it — the guarantee was
|
|
1328
|
+
false and nothing downstream noticed.
|
|
1329
|
+
|
|
1222
1330
|
**Three things it could not find**, each worth naming because each cost something.
|
|
1223
1331
|
|
|
1224
1332
|
- It did not find the workflow damage, because a shell script inside a block scalar is a string
|
|
@@ -1414,7 +1522,10 @@ tests none of this document. In `yaml` the sweep alphabet must include `:`, `#`,
|
|
|
1414
1522
|
as one token**, since those are the characters whose adjacency the argument above turns on and a
|
|
1415
1523
|
three-dash run is not reachable from single dashes at a bounded payload length; and it must
|
|
1416
1524
|
include U+0022 and U+005C, without which the sweep cannot reach a quoted scalar's delimiters at
|
|
1417
|
-
all. Since 1.
|
|
1525
|
+
all. Since 1.10.0 it must also include **U+000D**, and a `yaml` run must test the property §3.8.4
|
|
1526
|
+
states — that no span contains a line terminator — directly on the spans, because a rule that
|
|
1527
|
+
neither emits nor deletes a U+000D leaves the output unchanged whether or not one is inside a
|
|
1528
|
+
span. Since 1.9.0 it must also include **`''` as one token** next to a lone U+0027, and its
|
|
1418
1529
|
single-quoted templates must carry the pair at the start, in the middle and at the end of the
|
|
1419
1530
|
payload, because a unit's behaviour at a span extremity is a different path from its behaviour
|
|
1420
1531
|
between two letters. Since 1.7.0 the `markdown` run must also carry a template with a **frontmatter block and
|
|
@@ -1623,18 +1734,49 @@ rule-local.
|
|
|
1623
1734
|
requires a processable scalar to be the value of a key, and a bare item has none for
|
|
1624
1735
|
`keys` to match. `- key: value` is unaffected;
|
|
1625
1736
|
- **a key containing `"`, `'`, `{`, `[`, `&`, `*`, `!` or `#` anywhere** — `"description"`,
|
|
1626
|
-
but also `a!b` — declined at step 5
|
|
1627
|
-
its keys gets nothing from this mode
|
|
1737
|
+
but also `a!b` — is a declined key at step 5 and never compared, so a document that quotes
|
|
1738
|
+
its keys gets nothing from this mode. Since 1.11.0 the lines under a declined key are still
|
|
1739
|
+
scanned, unless its value can continue (step 7);
|
|
1628
1740
|
- **a plain scalar whose type is resolved from its whole text** — a listed key's value can
|
|
1629
1741
|
change tag without any syntactic character being touched (§5 item 2's note). `1 .` becomes
|
|
1630
1742
|
`1.` and stops being a string. This one is not a miss but an accepted cost, and it is the
|
|
1631
1743
|
only entry here that is not purely in the widening direction;
|
|
1632
1744
|
- **a multi-line plain scalar**, and a quoted scalar or flow collection that does not close
|
|
1633
1745
|
on its opening line: all three are consumed by step 7 and yield no spans;
|
|
1746
|
+
- **a closed quoted scalar followed by a continuation line** (1.11.0) — also consumed by step
|
|
1747
|
+
7. Two valid documents land here and are missed rather than damaged: a more-indented
|
|
1748
|
+
comment line after the scalar, and a sequence item holding a mapping
|
|
1749
|
+
(`- title: "x"` / ` slug: y`), whose second key counts as more indented than the first;
|
|
1750
|
+
- **a node line, and what it absorbs** (1.11.0, polytypo/polytypo#64). A line that opens a
|
|
1751
|
+
node without being a key's value — `- |`, `- "x`, `- [a,`, `? |`, a header on a line of its
|
|
1752
|
+
own — yields no spans and absorbs its value run, so the lines under it are never read as
|
|
1753
|
+
mappings. Misses, all valid documents: a line holding only an anchor or a tag, with or
|
|
1754
|
+
without a comment, over a nested mapping (`- &x` / ` title: a -- b`); a line that starts
|
|
1755
|
+
with an anchor, a tag or a flow collection and holds a key (`- &a k: v` / ` title: a -- b`,
|
|
1756
|
+
`- {a: 1}: v` / ` title: a -- b`), which absorbs the sibling after it; a quoted key whose
|
|
1757
|
+
value can continue (`- "a": |` / ` title: a -- b`), where YAML reads `title` as a sibling
|
|
1758
|
+
and the run absorbs it; an explicit key inside an entry (`- ? a` / ` : b` / ` title: a --
|
|
1759
|
+
b`); and a document marker followed by a node in column 0 (`--- !t` / `title: a -- b`),
|
|
1760
|
+
which absorbs the document. Measured on 5,912 YAML files and Markdown frontmatter blocks
|
|
1761
|
+
under the author's project directories (`en-US`, twelve prose keys), 1.10.0 and 1.11.0
|
|
1762
|
+
change the same lines: none of these costs fires there;
|
|
1763
|
+
- **an own-line node's region** (1.11.0). A flow collection, a quote or a block scalar header
|
|
1764
|
+
on a line of its own absorbs up to the next line in column 0, since the scan does not track
|
|
1765
|
+
the key above it, so `title` in a nested document after such a line (`info:` /
|
|
1766
|
+
` description:` / ` "x` / ` y"` / ` title: a -- b`) is missed; at indent 0 it absorbs
|
|
1767
|
+
the rest of the document, which a flow-collection key (`{a: 1}: v` / `title: a -- b`) also
|
|
1768
|
+
does. The fixtures pin the misses and the column-0 control;
|
|
1769
|
+
- **a tab** (1.11.0): a line containing U+0009 absorbs what it opens and yields no spans for
|
|
1770
|
+
it, where until 1.11.0 the lines under it were scanned;
|
|
1634
1771
|
- **a flow collection** — `tags: [one, two]` and `{a: b}` — even on one line;
|
|
1635
1772
|
- **a quoted scalar containing an escape** — `"say \"hi\""` — where the source spells the
|
|
1636
1773
|
content with more characters than it has;
|
|
1637
|
-
- **a block scalar whose indentation is ambiguous**, per §3.8.5's
|
|
1774
|
+
- **a block scalar whose indentation is ambiguous**, per §3.8.5's bail conditions;
|
|
1775
|
+
- **a line carrying a lone U+000D** (1.10.0), every line after it up to the next non-empty
|
|
1776
|
+
line in column 0 that carries none (1.11.0), and a block scalar with one on any line.
|
|
1777
|
+
Measured on the author's 367-file content directory: no file contains one. It is the price
|
|
1778
|
+
of keeping §3.8.4's line model LF-only while not letting a carriage return into a span, and
|
|
1779
|
+
it falls on documents written with Classic Mac line endings.
|
|
1638
1780
|
|
|
1639
1781
|
12. **`keys` matches a bare name at any depth, and nothing narrower.** `description` is
|
|
1640
1782
|
processable wherever it occurs, which is what makes the option portable — no paths, no
|
|
@@ -1672,7 +1814,14 @@ rule-local.
|
|
|
1672
1814
|
line, so that a stray one inside a single quoted value costs that value and not the block.
|
|
1673
1815
|
The block is still skipped, so nothing machine-read is typeset either way. A lone-U+000D
|
|
1674
1816
|
document is one line to §3.8.4 and therefore loses the whole block, which is the price of
|
|
1675
|
-
not widening that section here.
|
|
1817
|
+
not widening that section here. Since 1.10.0 `yaml` mode makes the same decline for itself
|
|
1818
|
+
(§3.8.4, 1.10.0), per line and consuming the value run, so the two paths no longer differ on
|
|
1819
|
+
a stray U+000D inside a line. One shape of the same character was missed in both until
|
|
1820
|
+
1.11.0: a U+000D at column 0 of a line inside an *unlisted* key's block scalar ended that
|
|
1821
|
+
value run under §3.8.4's LF-only model, so a listed-looking line after it was scanned as a
|
|
1822
|
+
mapping line of its own, which YAML would read as block content and which `yaml` mode then
|
|
1823
|
+
converted (polytypo/polytypo#63). It is closed without widening the line model, by
|
|
1824
|
+
§3.8.4's carriage-return region, at the cost recorded in §7.11.
|
|
1676
1825
|
- **§3.8.6's treatment of `''` in a single-quoted scalar: the cost, before and after 1.9.0.**
|
|
1677
1826
|
Until 1.9.0 a single-quoted scalar containing `''` yielded no spans. An apostrophe inside
|
|
1678
1827
|
such a scalar is spelled `''`, and an apostrophe is exactly what `apostrophe` and `quotes`
|
|
@@ -1728,7 +1877,7 @@ changes.
|
|
|
1728
1877
|
blocks, and the JSX-vs-skipped-element case distinction (`tests/modes/markdown.test.ts`).
|
|
1729
1878
|
3a. **YAML span selection** — the three scalar forms, the `keys` option (a listed key, an
|
|
1730
1879
|
unlisted one, a quoted one, the same name at two depths), block-scalar indentation and
|
|
1731
|
-
chomping, the `:`/`#` split of §3.8.6, the `''` split of §3.8.6 (since 1.9.0), step 7's consumption of an inline value's continuation
|
|
1880
|
+
chomping, the `:`/`#` split of §3.8.6, the `''` split of §3.8.6 (since 1.9.0), a lone U+000D (since 1.10.0), step 7's consumption of an inline value's continuation
|
|
1732
1881
|
lines, and every bail of §3.8.4. `yaml` carries a heavier fixture burden than the other two
|
|
1733
1882
|
for a reason §8's opening argument does not cover: its span selection is **specified rather
|
|
1734
1883
|
than delegated** (§3.8.1), so a fixture is the only thing standing between five hand-written
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"spec": "1.
|
|
2
|
+
"spec": "1.11.0",
|
|
3
3
|
"$comment": "Single source of truth for pipeline order. Rules run in ascending `order`. Disabling a rule removes it from the sequence and never reorders the rest. Rule ids are public API (see docs/ARCHITECTURE.md section 5). \"spec\" here must track spec/VERSION exactly — it is not itself the global version source; scripts/validate-spec.mjs enforces the match.",
|
|
4
4
|
"rules": [
|
|
5
5
|
{
|