marktip 0.6.0__tar.gz → 0.7.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {marktip-0.6.0 → marktip-0.7.1}/CMakeLists.txt +1 -1
- {marktip-0.6.0 → marktip-0.7.1}/PKG-INFO +93 -37
- {marktip-0.6.0 → marktip-0.7.1}/README.md +92 -36
- {marktip-0.6.0 → marktip-0.7.1}/pyproject.toml +1 -1
- {marktip-0.6.0 → marktip-0.7.1}/src/ast.cpp +202 -72
- {marktip-0.6.0 → marktip-0.7.1}/src/ast.h +17 -4
- {marktip-0.6.0 → marktip-0.7.1}/src/errors.h +2 -1
- {marktip-0.6.0 → marktip-0.7.1}/src/marktip.cpp +19 -12
- {marktip-0.6.0 → marktip-0.7.1}/src/parser.cpp +29 -9
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_parse.py +21 -0
- marktip-0.7.1/tests/test_strict.py +745 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_validate.py +1 -1
- marktip-0.6.0/tests/test_strict.py +0 -404
- {marktip-0.6.0 → marktip-0.7.1}/.github/workflows/build-distributions.yml +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/.gitignore +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/LICENSE +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/python/marktip/__init__.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/scripts/benchmark.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/src/parser.h +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/src/serializer.cpp +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/src/serializer.h +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_document.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_errors.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_roundtrip.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_serialize.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/tests/test_uri_policy.py +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/LICENSE.md +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/src/entity.c +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/src/entity.h +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/src/md4c.c +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/src/md4c.h +0 -0
- {marktip-0.6.0 → marktip-0.7.1}/third_party/md4c/upstream_metadata/README.md +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: marktip
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.7.1
|
|
4
4
|
Summary: Fast C++/MD4C Markdown parser and canonical Markdown serializer for Tiptap-style JSON
|
|
5
5
|
Keywords: markdown,tiptap,md4c,parser
|
|
6
6
|
Author: Saneaven (Minjae Kyung)
|
|
@@ -111,46 +111,93 @@ An attr the serializer never reads is dropped without a word, so a `colspan: 2`
|
|
|
111
111
|
|
|
112
112
|
```python
|
|
113
113
|
tm.from_dict(ast).to_markdown() # '| x |\n| --- |' — the colspan is gone
|
|
114
|
-
tm.from_dict(ast, strict=
|
|
114
|
+
tm.from_dict(ast, strict="content") # InvalidNodeError: attr 'colspan' cannot be
|
|
115
115
|
# expressed in markdown: GFM tables have no cell spanning
|
|
116
116
|
```
|
|
117
117
|
|
|
118
|
-
`strict
|
|
119
|
-
It is
|
|
118
|
+
`strict` names what the conversion may not lose, from the walk that is already happening, so a consumer does not have to re-traverse the tree in Python to find out whether the document survives intact.
|
|
119
|
+
It is keyword-only and a `from_dict` option only — the parser emits nothing but attrs it understands, with values in range, so there is nothing on the `from_markdown` path for it to catch.
|
|
120
120
|
|
|
121
|
-
|
|
|
121
|
+
| `strict` | Keeps |
|
|
122
122
|
| --- | --- |
|
|
123
|
-
| `
|
|
124
|
-
| `
|
|
125
|
-
| `
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
123
|
+
| `"off"` (default) | nothing is checked |
|
|
124
|
+
| `"content"` | content and structure. Presentation an editor stamped on is type-checked and dropped |
|
|
125
|
+
| `"exact"` | every attr the JSON carries |
|
|
126
|
+
|
|
127
|
+
Both enabled levels also enforce the ProseMirror same-type self-exclusion rule on each node's marks.
|
|
128
|
+
The standard marks represented by marktip are self-exclusive, so a node cannot carry the same mark type twice — including two `link` marks with different destinations.
|
|
129
|
+
The conflicting mark is rejected with `InvalidNodeError` rather than being serialized into Markdown that reparses with different formatting or structure.
|
|
130
|
+
`strict="off"` skips this check because strict validation is disabled.
|
|
131
|
+
|
|
132
|
+
> **Changed in 0.7.1** — `strict="content"` and `strict="exact"` now reject duplicate same-type marks with `invalid_mark_set`. Markdown parsing collapses nested spans that map to the same self-exclusive mark, so parser output remains valid on the strict read path.
|
|
133
|
+
|
|
134
|
+
The two levels differ over one thing, and it is worth being precise about which:
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
# what a stock Tiptap Link extension submits for a link with only href set
|
|
138
|
+
mark = {"type": "link", "attrs": {"href": "https://a", "target": "_blank",
|
|
139
|
+
"rel": "noopener noreferrer nofollow",
|
|
140
|
+
"class": None, "title": None}}
|
|
141
|
+
|
|
142
|
+
tm.from_dict(ast, strict="content") # fine — target and rel are dropped
|
|
143
|
+
tm.from_dict(ast, strict="exact") # InvalidNodeError: attr 'target' cannot be expressed
|
|
144
|
+
# in markdown: a markdown link carries only a
|
|
145
|
+
# destination and a title
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Dropping that `target` costs the *author* nothing — the editor stamped it on and nobody wrote it.
|
|
149
|
+
Dropping a `colspan` moves every cell after it under a different header, so `"content"` refuses that one too.
|
|
150
|
+
Nothing in the JSON tells the two apart: ProseMirror serializes schema defaults onto every node, so a chosen `target` and a stamped one arrive identical.
|
|
151
|
+
That is why the caller picks, rather than the library guessing — store markdown as the record and `"content"` is the promise you want; keep the JSON and it is `"exact"`.
|
|
152
|
+
|
|
153
|
+
| Node | Accepted attrs | Dropped, no markdown form |
|
|
154
|
+
| --- | --- | --- |
|
|
155
|
+
| `heading` | `level`: 1–6 | |
|
|
156
|
+
| `codeBlock` | `language`, `info`: single-line `str` | |
|
|
157
|
+
| `bulletList`, `taskList` | `tight`: `bool` | |
|
|
158
|
+
| `orderedList` | `start`: 0–999999999, `tight`: `bool` | `type`: `str` |
|
|
159
|
+
| `taskItem` | `checked`: `bool` | |
|
|
160
|
+
| `table` | `colCount`: non-negative `int` | |
|
|
161
|
+
| `tableHeader`, `tableCell` | `align`: `"left"`/`"center"`/`"right"`/`None`, `colspan`: `1`, `rowspan`: `1` | `colwidth`: list of `int` |
|
|
162
|
+
| `image` | `src` **(required)**, `alt`, `title`: `str` | `width`, `height`: `str` or `int` |
|
|
163
|
+
| `htmlBlock`, `htmlInline` | `html`: `str` | |
|
|
164
|
+
| `link` mark | `href` **(required)**, `title`: `str` | `target`, `rel`, `class`: `str` |
|
|
133
165
|
|
|
134
166
|
Every other type takes no attrs at all.
|
|
135
|
-
|
|
167
|
+
A name in neither column is an `unknown_attr` at both levels — that is the rule that stops an `onclick` from vanishing silently.
|
|
168
|
+
`bool` and `int` stay distinct: `{"level": True}` is refused rather than read as `1`.
|
|
169
|
+
Most of these also take `None`, meaning the attr was never set; the paragraph below says which do not.
|
|
170
|
+
|
|
171
|
+
> **Changed in 0.7.0** — `strict` takes a level name instead of a bool, so `strict=True` now raises `TypeError`; `strict="exact"` is what it used to mean.
|
|
172
|
+
> Three things move with the new `"content"` level: the attrs in the right-hand column above are accepted and dropped rather than refused, `href`/`src` are required, and a refused presentation attr reports `unrepresentable` where it used to report `unknown_attr`.
|
|
136
173
|
|
|
137
|
-
> **Changed in 0.6.0** — `None` on the attrs above
|
|
174
|
+
> **Changed in 0.6.0** — `None` on the optional attrs above now means the attr is unset, and the key is dropped on the way in rather than coerced to `""`.
|
|
138
175
|
> Two things move with it: an `image` whose `alt` is `None` *and* which carries content children now renders that child text, and `to_dict()` omits such a key instead of echoing `""`.
|
|
139
176
|
|
|
140
|
-
`None`
|
|
141
|
-
ProseMirror serializes every attr
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
177
|
+
`None` reads as unset wherever Tiptap declares `default: null`, which is most of the schema.
|
|
178
|
+
ProseMirror serializes every attr including defaults, so `None` is what the editor submits for an attr nobody filled in, and an unset attr is accepted already; the markdown is the same either way.
|
|
179
|
+
The exceptions are the settings Tiptap gives a non-null default — `level`, `start`, `tight`, `checked`, `colspan`, `rowspan` — plus `html`, which is marktip's own attr with no Tiptap counterpart.
|
|
180
|
+
Tiptap never emits a `None` for those, so one means the dict was built wrong; the conversion would survive it, and strict reports it because it is a bug rather than a loss.
|
|
181
|
+
`align` is unchanged: `None` was already a *value* there ("no alignment"), not an unset marker.
|
|
182
|
+
|
|
183
|
+
`href` and `src` are the one place a `None` is not "unset" but "missing", and both levels refuse it:
|
|
184
|
+
|
|
185
|
+
```python
|
|
186
|
+
tm.from_dict(link_without_href).to_markdown() # '[a]()' — reads back as a real link
|
|
187
|
+
tm.from_dict(link_without_href, strict="content") # InvalidNodeError: attr 'href' is required
|
|
188
|
+
# for mark type 'link'
|
|
189
|
+
```
|
|
146
190
|
|
|
147
|
-
`
|
|
191
|
+
A link with no destination is not a link, so writing `[a]()` is not a lossy conversion but an invented one, and the empty destination survives into stored Markdown.
|
|
192
|
+
An `href` of `""` is a different case and is accepted: that is what `from_markdown` itself produces for `[a]()`, and it round-trips exactly.
|
|
193
|
+
Rejecting an empty destination is the URI policy's job (`link_relative="reject"`).
|
|
194
|
+
|
|
195
|
+
`colspan`/`rowspan` are *known* attrs whose only accepted values are the no-ops Tiptap's table extension puts on every cell.
|
|
148
196
|
Refusing the names outright would leave strict unusable for the documents it exists to check.
|
|
149
|
-
Tiptap's Link extension is the other direction: its default `target` and `rel` have no markdown form, so they are refused, which is the same rule that stops an `onclick` from vanishing silently.
|
|
150
197
|
|
|
151
|
-
Two attrs are accepted and then ignored
|
|
198
|
+
Two attrs are accepted and then ignored at every level.
|
|
152
199
|
The serializer derives a table's column count from its rows, so `colCount` is never read, and it takes cell alignment from the header row only, so `align` on a body row is never read either.
|
|
153
|
-
|
|
200
|
+
Neither is a loss — the structure they describe is what survives — and `from_markdown` emits both, so refusing them would break re-parsing stored canonical Markdown on the read path, where a write-time refusal is much harder to notice.
|
|
154
201
|
|
|
155
202
|
## Errors
|
|
156
203
|
|
|
@@ -187,18 +234,24 @@ For `from_markdown` it locates the node in the parsed document, since the input
|
|
|
187
234
|
| `disallowed_scheme` | `InvalidNodeError` | a link `href` / image `src` scheme is outside the allowlist |
|
|
188
235
|
| `disallowed_relative_url` | `InvalidNodeError` | a scheme-less `href`/`src` violates the relative policy |
|
|
189
236
|
| `invalid_uri_char` | `InvalidNodeError` | an `href`/`src` contains an ASCII control character |
|
|
190
|
-
| `unknown_attr` | `InvalidNodeError` | `strict`: the attr
|
|
237
|
+
| `unknown_attr` | `InvalidNodeError` | `strict`: the attr name is in neither dialect for that type |
|
|
191
238
|
| `invalid_attr_value` | `InvalidNodeError` | `strict`: the attr's value has the wrong type or is out of range |
|
|
192
|
-
| `unrepresentable` | `InvalidNodeError` | `strict`: the value is well-formed, but
|
|
239
|
+
| `unrepresentable` | `InvalidNodeError` | `strict`: the value is well-formed, but markdown cannot carry it |
|
|
240
|
+
| `missing_attr` | `InvalidNodeError` | `strict`: a required attr is absent or `None` — a `link` with no `href`, an `image` with no `src` |
|
|
241
|
+
| `invalid_mark_set` | `InvalidNodeError` | `strict`: a node carries conflicting marks of the same type |
|
|
193
242
|
| `markdown_max_depth` | `ParseError` | markdown nesting exceeds 2048 |
|
|
194
243
|
| `parse_failed` | `ParseError` | MD4C could not parse the input |
|
|
195
244
|
|
|
196
245
|
`from_markdown` still raises a plain `TypeError` when the argument is not `str`/`bytes` — that is a caller bug, not a malformed document.
|
|
197
|
-
So is a bad
|
|
246
|
+
So is a bad option value: a non-iterable `link_schemes` raises `TypeError`, and `link_schemes=("https://",)`, `link_relative="nope"` or `strict="strict"` raises `ValueError`.
|
|
247
|
+
`strict` takes a level name, so `strict=True` raises `TypeError` rather than quietly meaning something.
|
|
198
248
|
|
|
199
249
|
> **Changed in 0.4.0** — these were previously `ValueError` (`from_dict`) and `RuntimeError` (`from_markdown`).
|
|
200
250
|
> `MarktipError` derives from `Exception` directly, so `except ValueError` no longer catches them.
|
|
201
251
|
|
|
252
|
+
> **Changed in 0.7.0** — `missing_attr` is new, and an attr with no markdown form now reports `unrepresentable` rather than `unknown_attr`, which is left meaning "no schema declares this name".
|
|
253
|
+
> `attrs` values that are not `str`/`int`/`bool` are dropped instead of stringified, so `to_dict()` no longer echoes a `"[1, 2]"` the caller never wrote.
|
|
254
|
+
|
|
202
255
|
> **Changed in 0.5.0** — an ASCII control character in a link `href` or image `src` is now rejected outright, with or without the URI policy.
|
|
203
256
|
> A URI cannot carry one unencoded (`%09` is the encoded form), and leaving it in would defeat the allowlist itself, because browsers strip tab/LF/CR before reading the scheme.
|
|
204
257
|
> This also rejects `[x](<a\tb>)`, which CommonMark otherwise accepts.
|
|
@@ -226,17 +279,19 @@ It guarantees, for the whole tree:
|
|
|
226
279
|
A document that survives `from_dict` always serializes; callers do not need to re-check any of the above.
|
|
227
280
|
`from_markdown` enforces the same two URI rules on what it parses.
|
|
228
281
|
|
|
229
|
-
It deliberately does **not** validate, unless `strict
|
|
282
|
+
It deliberately does **not** validate, unless `strict` asks it to:
|
|
230
283
|
|
|
231
284
|
- **`attrs` names and value types.**
|
|
232
285
|
Keys are free-form.
|
|
233
|
-
`str`, `int`, and `bool` round-trip unchanged;
|
|
234
|
-
The one exception is the attrs [Refusing lossy conversions](#refusing-lossy-conversions) lists as accepting `None`: there the key is dropped instead of coerced, because a `None` on those means the attr was never set.
|
|
286
|
+
`str`, `int`, and `bool` round-trip unchanged; every other value is dropped, `None` included, so `to_dict()` gives back what you passed in minus the keys markdown could not have carried anyway.
|
|
235
287
|
An attr with no markdown form is dropped at serialization — see [Refusing lossy conversions](#refusing-lossy-conversions).
|
|
236
|
-
- **Per-node required attrs
|
|
237
|
-
A `heading` without `level
|
|
238
|
-
`strict`
|
|
288
|
+
- **Per-node required attrs.**
|
|
289
|
+
A `heading` without `level` or a `codeBlock` without `language` is accepted; the serializer substitutes a default rather than failing.
|
|
290
|
+
`strict` adds exactly two: a `link` needs an `href` and an `image` needs a `src`, because the default there is an empty destination that reads back as a real link.
|
|
239
291
|
`{"src": None}` and no `src` key at all are the same thing to every part of the library, the URI policy included.
|
|
292
|
+
- **Mark sets.**
|
|
293
|
+
With strict validation disabled, two marks of the same type may appear on one node and are serialized independently.
|
|
294
|
+
`strict="content"` and `strict="exact"` reject the second mark because marktip's standard ProseMirror/Tiptap marks are self-exclusive.
|
|
240
295
|
- **Content models.**
|
|
241
296
|
Which node may contain which is not checked — a `text` node directly under `doc` is accepted and serialized.
|
|
242
297
|
- **Raw HTML.**
|
|
@@ -261,7 +316,7 @@ except tm.InvalidNodeError as e:
|
|
|
261
316
|
## Defined normalizations
|
|
262
317
|
|
|
263
318
|
Markdown cannot represent every Tiptap document exactly.
|
|
264
|
-
Rather than emitting
|
|
319
|
+
Rather than emitting or retaining a representation that changes on the next conversion, marktip applies these deterministic (and idempotent) normalizations during parsing and serialization:
|
|
265
320
|
|
|
266
321
|
- **Hard breaks in headings** — ATX headings are single-line, so a `hardBreak` (or a literal newline carried over from setext input) inside a heading serializes as a single space: `heading("a", hardBreak, "b")` → `# a b`.
|
|
267
322
|
- **Multi-block table cells** — blocks inside a cell are joined with `<br>` and the cell is flattened to one line; two paragraphs in a cell reparse as one paragraph containing a `hardBreak`.
|
|
@@ -269,6 +324,7 @@ Rather than emitting markdown that reparses into a different structure, marktip
|
|
|
269
324
|
- **Heading levels** — clamped to 1–6 at serialization (`0` → `#`, `7` → `######`).
|
|
270
325
|
- **Ordered list start** — clamped to CommonMark's 0–999999999 (`-5` → `0.`), and the running number stops at that ceiling rather than emitting a 10-digit marker. Only the first number is honoured on reparse, so a repeated ceiling changes nothing.
|
|
271
326
|
- **Code fence info strings** — a `language` is truncated at the first newline, since an info string is a single line; one containing a backtick switches the fence to `~~~`, which carries it losslessly.
|
|
327
|
+
- **Nested same-type marks** — Markdown spans that map to the same self-exclusive mark are collapsed (`*outer _inner_ outer*` becomes one continuous `italic` mark) rather than producing an invalid ProseMirror mark set.
|
|
272
328
|
- **Emphasis boundary whitespace** — whitespace touching an emphasis delimiter has no valid markdown form and is expelled outside the marks (`bold("굵게 ")` → `**굵게** `), cf. prosemirror-markdown's `expelEnclosingWhitespace`.
|
|
273
329
|
- **Adjacent same-family lists** — consecutive lists of the same family alternate markers (`-`/`*`, `1.`/`1)`) so they stay separate lists on reparse instead of merging (which would renumber ordered items or spread task checkboxes).
|
|
274
330
|
|
|
@@ -83,46 +83,93 @@ An attr the serializer never reads is dropped without a word, so a `colspan: 2`
|
|
|
83
83
|
|
|
84
84
|
```python
|
|
85
85
|
tm.from_dict(ast).to_markdown() # '| x |\n| --- |' — the colspan is gone
|
|
86
|
-
tm.from_dict(ast, strict=
|
|
86
|
+
tm.from_dict(ast, strict="content") # InvalidNodeError: attr 'colspan' cannot be
|
|
87
87
|
# expressed in markdown: GFM tables have no cell spanning
|
|
88
88
|
```
|
|
89
89
|
|
|
90
|
-
`strict
|
|
91
|
-
It is
|
|
90
|
+
`strict` names what the conversion may not lose, from the walk that is already happening, so a consumer does not have to re-traverse the tree in Python to find out whether the document survives intact.
|
|
91
|
+
It is keyword-only and a `from_dict` option only — the parser emits nothing but attrs it understands, with values in range, so there is nothing on the `from_markdown` path for it to catch.
|
|
92
92
|
|
|
93
|
-
|
|
|
93
|
+
| `strict` | Keeps |
|
|
94
94
|
| --- | --- |
|
|
95
|
-
| `
|
|
96
|
-
| `
|
|
97
|
-
| `
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
95
|
+
| `"off"` (default) | nothing is checked |
|
|
96
|
+
| `"content"` | content and structure. Presentation an editor stamped on is type-checked and dropped |
|
|
97
|
+
| `"exact"` | every attr the JSON carries |
|
|
98
|
+
|
|
99
|
+
Both enabled levels also enforce the ProseMirror same-type self-exclusion rule on each node's marks.
|
|
100
|
+
The standard marks represented by marktip are self-exclusive, so a node cannot carry the same mark type twice — including two `link` marks with different destinations.
|
|
101
|
+
The conflicting mark is rejected with `InvalidNodeError` rather than being serialized into Markdown that reparses with different formatting or structure.
|
|
102
|
+
`strict="off"` skips this check because strict validation is disabled.
|
|
103
|
+
|
|
104
|
+
> **Changed in 0.7.1** — `strict="content"` and `strict="exact"` now reject duplicate same-type marks with `invalid_mark_set`. Markdown parsing collapses nested spans that map to the same self-exclusive mark, so parser output remains valid on the strict read path.
|
|
105
|
+
|
|
106
|
+
The two levels differ over one thing, and it is worth being precise about which:
|
|
107
|
+
|
|
108
|
+
```python
|
|
109
|
+
# what a stock Tiptap Link extension submits for a link with only href set
|
|
110
|
+
mark = {"type": "link", "attrs": {"href": "https://a", "target": "_blank",
|
|
111
|
+
"rel": "noopener noreferrer nofollow",
|
|
112
|
+
"class": None, "title": None}}
|
|
113
|
+
|
|
114
|
+
tm.from_dict(ast, strict="content") # fine — target and rel are dropped
|
|
115
|
+
tm.from_dict(ast, strict="exact") # InvalidNodeError: attr 'target' cannot be expressed
|
|
116
|
+
# in markdown: a markdown link carries only a
|
|
117
|
+
# destination and a title
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Dropping that `target` costs the *author* nothing — the editor stamped it on and nobody wrote it.
|
|
121
|
+
Dropping a `colspan` moves every cell after it under a different header, so `"content"` refuses that one too.
|
|
122
|
+
Nothing in the JSON tells the two apart: ProseMirror serializes schema defaults onto every node, so a chosen `target` and a stamped one arrive identical.
|
|
123
|
+
That is why the caller picks, rather than the library guessing — store markdown as the record and `"content"` is the promise you want; keep the JSON and it is `"exact"`.
|
|
124
|
+
|
|
125
|
+
| Node | Accepted attrs | Dropped, no markdown form |
|
|
126
|
+
| --- | --- | --- |
|
|
127
|
+
| `heading` | `level`: 1–6 | |
|
|
128
|
+
| `codeBlock` | `language`, `info`: single-line `str` | |
|
|
129
|
+
| `bulletList`, `taskList` | `tight`: `bool` | |
|
|
130
|
+
| `orderedList` | `start`: 0–999999999, `tight`: `bool` | `type`: `str` |
|
|
131
|
+
| `taskItem` | `checked`: `bool` | |
|
|
132
|
+
| `table` | `colCount`: non-negative `int` | |
|
|
133
|
+
| `tableHeader`, `tableCell` | `align`: `"left"`/`"center"`/`"right"`/`None`, `colspan`: `1`, `rowspan`: `1` | `colwidth`: list of `int` |
|
|
134
|
+
| `image` | `src` **(required)**, `alt`, `title`: `str` | `width`, `height`: `str` or `int` |
|
|
135
|
+
| `htmlBlock`, `htmlInline` | `html`: `str` | |
|
|
136
|
+
| `link` mark | `href` **(required)**, `title`: `str` | `target`, `rel`, `class`: `str` |
|
|
105
137
|
|
|
106
138
|
Every other type takes no attrs at all.
|
|
107
|
-
|
|
139
|
+
A name in neither column is an `unknown_attr` at both levels — that is the rule that stops an `onclick` from vanishing silently.
|
|
140
|
+
`bool` and `int` stay distinct: `{"level": True}` is refused rather than read as `1`.
|
|
141
|
+
Most of these also take `None`, meaning the attr was never set; the paragraph below says which do not.
|
|
142
|
+
|
|
143
|
+
> **Changed in 0.7.0** — `strict` takes a level name instead of a bool, so `strict=True` now raises `TypeError`; `strict="exact"` is what it used to mean.
|
|
144
|
+
> Three things move with the new `"content"` level: the attrs in the right-hand column above are accepted and dropped rather than refused, `href`/`src` are required, and a refused presentation attr reports `unrepresentable` where it used to report `unknown_attr`.
|
|
108
145
|
|
|
109
|
-
> **Changed in 0.6.0** — `None` on the attrs above
|
|
146
|
+
> **Changed in 0.6.0** — `None` on the optional attrs above now means the attr is unset, and the key is dropped on the way in rather than coerced to `""`.
|
|
110
147
|
> Two things move with it: an `image` whose `alt` is `None` *and* which carries content children now renders that child text, and `to_dict()` omits such a key instead of echoing `""`.
|
|
111
148
|
|
|
112
|
-
`None`
|
|
113
|
-
ProseMirror serializes every attr
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
149
|
+
`None` reads as unset wherever Tiptap declares `default: null`, which is most of the schema.
|
|
150
|
+
ProseMirror serializes every attr including defaults, so `None` is what the editor submits for an attr nobody filled in, and an unset attr is accepted already; the markdown is the same either way.
|
|
151
|
+
The exceptions are the settings Tiptap gives a non-null default — `level`, `start`, `tight`, `checked`, `colspan`, `rowspan` — plus `html`, which is marktip's own attr with no Tiptap counterpart.
|
|
152
|
+
Tiptap never emits a `None` for those, so one means the dict was built wrong; the conversion would survive it, and strict reports it because it is a bug rather than a loss.
|
|
153
|
+
`align` is unchanged: `None` was already a *value* there ("no alignment"), not an unset marker.
|
|
154
|
+
|
|
155
|
+
`href` and `src` are the one place a `None` is not "unset" but "missing", and both levels refuse it:
|
|
156
|
+
|
|
157
|
+
```python
|
|
158
|
+
tm.from_dict(link_without_href).to_markdown() # '[a]()' — reads back as a real link
|
|
159
|
+
tm.from_dict(link_without_href, strict="content") # InvalidNodeError: attr 'href' is required
|
|
160
|
+
# for mark type 'link'
|
|
161
|
+
```
|
|
118
162
|
|
|
119
|
-
`
|
|
163
|
+
A link with no destination is not a link, so writing `[a]()` is not a lossy conversion but an invented one, and the empty destination survives into stored Markdown.
|
|
164
|
+
An `href` of `""` is a different case and is accepted: that is what `from_markdown` itself produces for `[a]()`, and it round-trips exactly.
|
|
165
|
+
Rejecting an empty destination is the URI policy's job (`link_relative="reject"`).
|
|
166
|
+
|
|
167
|
+
`colspan`/`rowspan` are *known* attrs whose only accepted values are the no-ops Tiptap's table extension puts on every cell.
|
|
120
168
|
Refusing the names outright would leave strict unusable for the documents it exists to check.
|
|
121
|
-
Tiptap's Link extension is the other direction: its default `target` and `rel` have no markdown form, so they are refused, which is the same rule that stops an `onclick` from vanishing silently.
|
|
122
169
|
|
|
123
|
-
Two attrs are accepted and then ignored
|
|
170
|
+
Two attrs are accepted and then ignored at every level.
|
|
124
171
|
The serializer derives a table's column count from its rows, so `colCount` is never read, and it takes cell alignment from the header row only, so `align` on a body row is never read either.
|
|
125
|
-
|
|
172
|
+
Neither is a loss — the structure they describe is what survives — and `from_markdown` emits both, so refusing them would break re-parsing stored canonical Markdown on the read path, where a write-time refusal is much harder to notice.
|
|
126
173
|
|
|
127
174
|
## Errors
|
|
128
175
|
|
|
@@ -159,18 +206,24 @@ For `from_markdown` it locates the node in the parsed document, since the input
|
|
|
159
206
|
| `disallowed_scheme` | `InvalidNodeError` | a link `href` / image `src` scheme is outside the allowlist |
|
|
160
207
|
| `disallowed_relative_url` | `InvalidNodeError` | a scheme-less `href`/`src` violates the relative policy |
|
|
161
208
|
| `invalid_uri_char` | `InvalidNodeError` | an `href`/`src` contains an ASCII control character |
|
|
162
|
-
| `unknown_attr` | `InvalidNodeError` | `strict`: the attr
|
|
209
|
+
| `unknown_attr` | `InvalidNodeError` | `strict`: the attr name is in neither dialect for that type |
|
|
163
210
|
| `invalid_attr_value` | `InvalidNodeError` | `strict`: the attr's value has the wrong type or is out of range |
|
|
164
|
-
| `unrepresentable` | `InvalidNodeError` | `strict`: the value is well-formed, but
|
|
211
|
+
| `unrepresentable` | `InvalidNodeError` | `strict`: the value is well-formed, but markdown cannot carry it |
|
|
212
|
+
| `missing_attr` | `InvalidNodeError` | `strict`: a required attr is absent or `None` — a `link` with no `href`, an `image` with no `src` |
|
|
213
|
+
| `invalid_mark_set` | `InvalidNodeError` | `strict`: a node carries conflicting marks of the same type |
|
|
165
214
|
| `markdown_max_depth` | `ParseError` | markdown nesting exceeds 2048 |
|
|
166
215
|
| `parse_failed` | `ParseError` | MD4C could not parse the input |
|
|
167
216
|
|
|
168
217
|
`from_markdown` still raises a plain `TypeError` when the argument is not `str`/`bytes` — that is a caller bug, not a malformed document.
|
|
169
|
-
So is a bad
|
|
218
|
+
So is a bad option value: a non-iterable `link_schemes` raises `TypeError`, and `link_schemes=("https://",)`, `link_relative="nope"` or `strict="strict"` raises `ValueError`.
|
|
219
|
+
`strict` takes a level name, so `strict=True` raises `TypeError` rather than quietly meaning something.
|
|
170
220
|
|
|
171
221
|
> **Changed in 0.4.0** — these were previously `ValueError` (`from_dict`) and `RuntimeError` (`from_markdown`).
|
|
172
222
|
> `MarktipError` derives from `Exception` directly, so `except ValueError` no longer catches them.
|
|
173
223
|
|
|
224
|
+
> **Changed in 0.7.0** — `missing_attr` is new, and an attr with no markdown form now reports `unrepresentable` rather than `unknown_attr`, which is left meaning "no schema declares this name".
|
|
225
|
+
> `attrs` values that are not `str`/`int`/`bool` are dropped instead of stringified, so `to_dict()` no longer echoes a `"[1, 2]"` the caller never wrote.
|
|
226
|
+
|
|
174
227
|
> **Changed in 0.5.0** — an ASCII control character in a link `href` or image `src` is now rejected outright, with or without the URI policy.
|
|
175
228
|
> A URI cannot carry one unencoded (`%09` is the encoded form), and leaving it in would defeat the allowlist itself, because browsers strip tab/LF/CR before reading the scheme.
|
|
176
229
|
> This also rejects `[x](<a\tb>)`, which CommonMark otherwise accepts.
|
|
@@ -198,17 +251,19 @@ It guarantees, for the whole tree:
|
|
|
198
251
|
A document that survives `from_dict` always serializes; callers do not need to re-check any of the above.
|
|
199
252
|
`from_markdown` enforces the same two URI rules on what it parses.
|
|
200
253
|
|
|
201
|
-
It deliberately does **not** validate, unless `strict
|
|
254
|
+
It deliberately does **not** validate, unless `strict` asks it to:
|
|
202
255
|
|
|
203
256
|
- **`attrs` names and value types.**
|
|
204
257
|
Keys are free-form.
|
|
205
|
-
`str`, `int`, and `bool` round-trip unchanged;
|
|
206
|
-
The one exception is the attrs [Refusing lossy conversions](#refusing-lossy-conversions) lists as accepting `None`: there the key is dropped instead of coerced, because a `None` on those means the attr was never set.
|
|
258
|
+
`str`, `int`, and `bool` round-trip unchanged; every other value is dropped, `None` included, so `to_dict()` gives back what you passed in minus the keys markdown could not have carried anyway.
|
|
207
259
|
An attr with no markdown form is dropped at serialization — see [Refusing lossy conversions](#refusing-lossy-conversions).
|
|
208
|
-
- **Per-node required attrs
|
|
209
|
-
A `heading` without `level
|
|
210
|
-
`strict`
|
|
260
|
+
- **Per-node required attrs.**
|
|
261
|
+
A `heading` without `level` or a `codeBlock` without `language` is accepted; the serializer substitutes a default rather than failing.
|
|
262
|
+
`strict` adds exactly two: a `link` needs an `href` and an `image` needs a `src`, because the default there is an empty destination that reads back as a real link.
|
|
211
263
|
`{"src": None}` and no `src` key at all are the same thing to every part of the library, the URI policy included.
|
|
264
|
+
- **Mark sets.**
|
|
265
|
+
With strict validation disabled, two marks of the same type may appear on one node and are serialized independently.
|
|
266
|
+
`strict="content"` and `strict="exact"` reject the second mark because marktip's standard ProseMirror/Tiptap marks are self-exclusive.
|
|
212
267
|
- **Content models.**
|
|
213
268
|
Which node may contain which is not checked — a `text` node directly under `doc` is accepted and serialized.
|
|
214
269
|
- **Raw HTML.**
|
|
@@ -233,7 +288,7 @@ except tm.InvalidNodeError as e:
|
|
|
233
288
|
## Defined normalizations
|
|
234
289
|
|
|
235
290
|
Markdown cannot represent every Tiptap document exactly.
|
|
236
|
-
Rather than emitting
|
|
291
|
+
Rather than emitting or retaining a representation that changes on the next conversion, marktip applies these deterministic (and idempotent) normalizations during parsing and serialization:
|
|
237
292
|
|
|
238
293
|
- **Hard breaks in headings** — ATX headings are single-line, so a `hardBreak` (or a literal newline carried over from setext input) inside a heading serializes as a single space: `heading("a", hardBreak, "b")` → `# a b`.
|
|
239
294
|
- **Multi-block table cells** — blocks inside a cell are joined with `<br>` and the cell is flattened to one line; two paragraphs in a cell reparse as one paragraph containing a `hardBreak`.
|
|
@@ -241,6 +296,7 @@ Rather than emitting markdown that reparses into a different structure, marktip
|
|
|
241
296
|
- **Heading levels** — clamped to 1–6 at serialization (`0` → `#`, `7` → `######`).
|
|
242
297
|
- **Ordered list start** — clamped to CommonMark's 0–999999999 (`-5` → `0.`), and the running number stops at that ceiling rather than emitting a 10-digit marker. Only the first number is honoured on reparse, so a repeated ceiling changes nothing.
|
|
243
298
|
- **Code fence info strings** — a `language` is truncated at the first newline, since an info string is a single line; one containing a backtick switches the fence to `~~~`, which carries it losslessly.
|
|
299
|
+
- **Nested same-type marks** — Markdown spans that map to the same self-exclusive mark are collapsed (`*outer _inner_ outer*` becomes one continuous `italic` mark) rather than producing an invalid ProseMirror mark set.
|
|
244
300
|
- **Emphasis boundary whitespace** — whitespace touching an emphasis delimiter has no valid markdown form and is expelled outside the marks (`bold("굵게 ")` → `**굵게** `), cf. prosemirror-markdown's `expelEnclosingWhitespace`.
|
|
245
301
|
- **Adjacent same-family lists** — consecutive lists of the same family alternate markers (`-`/`*`, `1.`/`1)`) so they stay separate lists on reparse instead of merging (which would renumber ordered items or spread task checkboxes).
|
|
246
302
|
|
|
@@ -4,7 +4,7 @@ build-backend = "scikit_build_core.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "marktip"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.7.1"
|
|
8
8
|
description = "Fast C++/MD4C Markdown parser and canonical Markdown serializer for Tiptap-style JSON"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.9"
|