wikiparser-node 1.35.0 → 1.35.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -28
- package/bundle/bundle-es8.min.js +29 -29
- package/bundle/bundle-lsp.min.js +34 -34
- package/bundle/bundle.min.js +24 -24
- package/config/default.json +3 -0
- package/config/enwiki.json +3 -0
- package/config/jawiki.json +3 -0
- package/config/minimum.json +1 -0
- package/config/zhwiki.json +3 -0
- package/coverage/badge.svg +1 -1
- package/data/signatures.json +13 -0
- package/dist/addon/attribute.js +13 -12
- package/dist/addon/link.js +3 -3
- package/dist/addon/token.js +4 -4
- package/dist/addon/transclude.js +8 -8
- package/dist/base.d.mts +3 -1
- package/dist/base.d.ts +3 -1
- package/dist/bin/config.js +3 -3
- package/dist/index.js +21 -13
- package/dist/lib/document.js +14 -12
- package/dist/lib/element.d.ts +2 -1
- package/dist/lib/element.js +17 -13
- package/dist/lib/lintConfig.js +7 -6
- package/dist/lib/lsp.js +10 -8
- package/dist/lib/node.js +8 -6
- package/dist/lib/range.js +17 -13
- package/dist/lib/ranges.js +25 -26
- package/dist/lib/text.js +14 -13
- package/dist/lib/title.js +7 -6
- package/dist/parser/braces.js +5 -7
- package/dist/parser/hrAndDoubleUnderscore.js +2 -2
- package/dist/parser/list.js +1 -1
- package/dist/parser/magicLinks.js +2 -1
- package/dist/parser/selector.js +15 -15
- package/dist/src/arg.js +4 -4
- package/dist/src/atom.js +1 -1
- package/dist/src/attribute.js +5 -4
- package/dist/src/attributes.js +10 -7
- package/dist/src/converter.js +4 -4
- package/dist/src/converterRule.js +2 -2
- package/dist/src/heading.js +2 -2
- package/dist/src/imageParameter.js +12 -3
- package/dist/src/index.js +15 -10
- package/dist/src/link/base.js +10 -7
- package/dist/src/link/category.js +3 -3
- package/dist/src/link/file.js +6 -5
- package/dist/src/link/index.js +5 -3
- package/dist/src/multiLine/gallery.js +2 -2
- package/dist/src/nowiki/comment.js +4 -3
- package/dist/src/nowiki/doubleUnderscore.js +1 -1
- package/dist/src/nowiki/listBase.js +2 -2
- package/dist/src/nowiki/noinclude.js +1 -1
- package/dist/src/nowiki/quote.js +3 -3
- package/dist/src/onlyinclude.js +1 -1
- package/dist/src/parameter.js +2 -2
- package/dist/src/table/index.js +9 -8
- package/dist/src/table/td.js +5 -5
- package/dist/src/table/trBase.js +4 -4
- package/dist/src/tag/html.js +9 -9
- package/dist/src/tag/index.js +6 -5
- package/dist/src/tag/tvar.js +4 -3
- package/dist/src/tagPair/index.js +6 -4
- package/dist/src/transclude.js +3 -3
- package/dist/util/debug.js +3 -2
- package/dist/util/diff.js +11 -6
- package/dist/util/selector.js +1 -1
- package/dist/util/string.js +2 -2
- package/extensions/dist/base.js +12 -4
- package/extensions/dist/codejar.js +1 -1
- package/package.json +27 -27
- package/logo.png +0 -0
package/README.md
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
|
-
<div align="center"><img src="logo.png" width="200" alt="WikiParser-Node logo"></div>
|
|
2
|
-
|
|
1
|
+
<div align="center"><img src="https://github.com/bhsd-harry/wikiparser-node/raw/main/logo.png" width="200" alt="WikiParser-Node logo"></div>
|
|
2
|
+
|
|
3
|
+
# WikiParser-Node
|
|
3
4
|
|
|
4
5
|
[](https://www.npmjs.com/package/wikiparser-node)
|
|
5
6
|
[](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/codeql.yml)
|
|
@@ -8,31 +9,31 @@
|
|
|
8
9
|
[](https://app.codacy.com/gh/bhsd-harry/wikiparser-node/dashboard)
|
|
9
10
|

|
|
10
11
|
|
|
11
|
-
|
|
12
|
+
## Other Languages
|
|
12
13
|
|
|
13
14
|
- [简体中文](./README-%28ZH%29.md)
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
## Introduction
|
|
16
17
|
|
|
17
18
|
WikiParser-Node is an offline [Wikitext](https://www.mediawiki.org/wiki/Wikitext) parser developed by Bhsd for the [Node.js](https://nodejs.org/) environment. It can parse almost all wiki syntax and generate an [Abstract Syntax Tree (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) ([Try it online](https://bhsd-harry.github.io/wikiparser-node/#editor)). It also allows for easy querying and modification of the AST, and returns the modified wikitext.
|
|
18
19
|
|
|
19
20
|
Although WikiParser-Node is not originally designed to convert Wikitext to HTML, it provides a limited capability to do so. [Here](https://bhsd-harry.github.io/wikiparser-website/) is a list of example HTML pages rendered using this package.
|
|
20
21
|
|
|
21
|
-
|
|
22
|
+
## Other Versions
|
|
22
23
|
|
|
23
|
-
|
|
24
|
+
### Mini (also known as [WikiLint](https://www.npmjs.com/package/wikilint))
|
|
24
25
|
|
|
25
26
|
This version provides a [CLI](https://en.wikipedia.org/wiki/Command-line_interface), but only retains the parsing functionality and linting functionality. The parsed AST cannot be modified. It powers the [Wikitext LSP](https://www.npmjs.com/package/wikitext-lsp), which provides multiple language services for editors such as [VS Code](https://marketplace.visualstudio.com/items?itemName=Bhsd.vscode-extension-wikiparser), [Sublime Text](https://lsp.sublimetext.io/language_servers/#mediawiki), and [Helix](https://github.com/helix-editor/helix/wiki/Language-Server-Configurations#wikitext).
|
|
26
27
|
|
|
27
28
|
A list of available linting rules can be found [here](https://github.com/bhsd-harry/wikiparser-node/wiki/Rules).
|
|
28
29
|
|
|
29
|
-
|
|
30
|
+
### Browser-compatible
|
|
30
31
|
|
|
31
32
|
A browser-compatible version, which can be used for code highlighting or as a linting plugin in conjunction with editors such as [CodeMirror](https://www.npmjs.com/package/@bhsd/codemirror-mediawiki) and [Monaco](https://www.npmjs.com/package/monaco-wiki) ([Usage example](https://bhsd-harry.github.io/wikiparser-node)). It has been integrated into the MediaWiki official [CodeMirror extension](https://www.mediawiki.org/wiki/Extension:CodeMirror) since Release 1.45.
|
|
32
33
|
|
|
33
|
-
|
|
34
|
+
## Installation
|
|
34
35
|
|
|
35
|
-
|
|
36
|
+
### Node.js
|
|
36
37
|
|
|
37
38
|
Please install the corresponding version as needed (`WikiParser-Node` or `WikiLint`), for example:
|
|
38
39
|
|
|
@@ -46,7 +47,7 @@ or
|
|
|
46
47
|
npm i wikilint
|
|
47
48
|
```
|
|
48
49
|
|
|
49
|
-
|
|
50
|
+
### Browser
|
|
50
51
|
|
|
51
52
|
You can download the code via CDN, for example:
|
|
52
53
|
|
|
@@ -62,9 +63,9 @@ or
|
|
|
62
63
|
|
|
63
64
|
For more browser extensions, please refer to the corresponding [documentation](https://github.com/bhsd-harry/wikiparser-node/wiki/Browser-%28EN%29).
|
|
64
65
|
|
|
65
|
-
|
|
66
|
+
## Usage
|
|
66
67
|
|
|
67
|
-
|
|
68
|
+
### CLI usage
|
|
68
69
|
|
|
69
70
|
For MediaWiki sites with the [CodeMirror extension](https://mediawiki.org/wiki/Extension:CodeMirror) installed, such as different language editions of Wikipedia and other [Wikimedia Foundation-hosted sites](https://meta.wikimedia.org/wiki/Special:SiteMatrix), you can use the following command to obtain the parser configuration:
|
|
70
71
|
|
|
@@ -81,76 +82,76 @@ The generated configuration file will be saved in the `config` directory. You ca
|
|
|
81
82
|
Parser.config = 'jawiki';
|
|
82
83
|
```
|
|
83
84
|
|
|
84
|
-
|
|
85
|
+
### API usage
|
|
85
86
|
|
|
86
87
|
Please refer to the [Wiki](https://github.com/bhsd-harry/wikiparser-node/wiki/Home-%28EN%29).
|
|
87
88
|
|
|
88
|
-
|
|
89
|
+
## Performance
|
|
89
90
|
|
|
90
91
|
A full database dump (`*.xml.bz2`) [scan](https://www.npmjs.com/package/lint-wiki-dumps) of English Wikipedia's ~19 million articles (parsing and linting) on a personal MacBook Air takes about 5 hours.
|
|
91
92
|
|
|
92
|
-
|
|
93
|
+
## Known issues
|
|
93
94
|
|
|
94
|
-
|
|
95
|
+
### Parser
|
|
95
96
|
|
|
96
97
|
1. Memory leaks may occur in rare cases.
|
|
97
98
|
1. Invalid page names with unicode characters are treated like valid ones ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Render%20invalid%20page%20names%20as%20plain%20text%20(T53090))).
|
|
98
99
|
1. Preformatted text with a leading space is only processed by [`Token.prototype.toHtml`](https://github.com/bhsd-harry/wikiparser-node/wiki/Token-%28EN%29#tohtml).
|
|
99
100
|
1. BCP 47 language codes are not supported in language conversion ([Example](https://bhsd-harry.github.io/wikiparser-node/tests.html#Explicit%20definition%20of%20language%20variant%20alternatives%20(BCP%2047%20codes))).
|
|
100
101
|
|
|
101
|
-
|
|
102
|
+
### HTML conversion
|
|
102
103
|
|
|
103
104
|
<details>
|
|
104
|
-
|
|
105
|
+
<summary>Expand</summary>
|
|
105
106
|
|
|
106
|
-
|
|
107
|
+
#### Extension
|
|
107
108
|
|
|
108
109
|
1. Many extensions are not supported, such as `<indicator>` and `<ref>`.
|
|
109
110
|
1. `&` needs to be escaped in `<syntaxhighlight>` ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#XSS%20is%20escaped)).
|
|
110
111
|
|
|
111
|
-
|
|
112
|
+
#### Transclusion
|
|
112
113
|
|
|
113
114
|
1. Some parser functions are not supported.
|
|
114
115
|
1. New lines in `{{localurl:}}` are not handled correctly ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Basic%20%7B%7Blocalurle%3A%7D%7D%20test)).
|
|
115
116
|
|
|
116
|
-
|
|
117
|
+
#### Heading
|
|
117
118
|
|
|
118
119
|
1. The table of contents (TOC) is not supported.
|
|
119
120
|
|
|
120
|
-
|
|
121
|
+
#### HTML tag
|
|
121
122
|
|
|
122
123
|
1. Style sanitization is sometimes different ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#CSS%20safety%20test%20(all%20browsers)%3A%20vertical%20tab%20(T57332%20%2F%20CVE-2013-4567))).
|
|
123
124
|
1. Table fostered content from `<table>` HTML tags ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#6.%20Encapsulate%20foster-parented%20transclusion%20content)).
|
|
124
125
|
|
|
125
|
-
|
|
126
|
+
#### Table
|
|
126
127
|
|
|
127
128
|
1. `<caption>` elements are wrapped in `<tbody>` elements ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Trim%20whitespace%20in%20wikitext%20headings%2C%20list%20items%2C%20table%20captions%2C%20headings%2C%20and%20cells)).
|
|
128
129
|
1. Unclosed HTML tags in the table fostered content ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Fuzz%20testing%3A%20Parser24)).
|
|
129
130
|
1. `<tr>` elements should not be fostered ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Empty%20TR%20followed%20by%20a%20template-generated%20TR)).
|
|
130
131
|
|
|
131
|
-
|
|
132
|
+
#### Link
|
|
132
133
|
|
|
133
134
|
1. Link trail is not supported ([Example](https://bhsd-harry.github.io/wikiparser-node/tests.html#1.%20Interaction%20of%20linktrail%20and%20template%20encapsulation)).
|
|
134
135
|
1. Block elements inside a link should break it into multiple links ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#%3Cpre%3E%20inside%20a%20link)).
|
|
135
136
|
1. Invalid or missing images (Examples [1](https://bhsd-harry.github.io/wikiparser-node/tests.html#Fuzz%20testing%3A%20image%20with%20bogus%20manual%20thumbnail), [2](https://bhsd-harry.github.io/wikiparser-node/tests.html#Gallery%20with%20line%20not%20in%20the%20file%20namespace)).
|
|
136
137
|
1. Link starting with `../` on a subpage ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Parent%20directory%20in%20redirect)).
|
|
137
138
|
|
|
138
|
-
|
|
139
|
+
#### External link
|
|
139
140
|
|
|
140
141
|
1. External images are not supported (Examples [1](http://bhsd-harry.github.io/wikiparser-node/tests.html#External%20image), [2](http://bhsd-harry.github.io/wikiparser-node/tests.html#External%20links%3A%20Clickable%20images)).
|
|
141
142
|
1. No percent-encoding in displayed free external links ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Parsoid%3A%20pipe%20in%20transclusion%20parameter)).
|
|
142
143
|
|
|
143
|
-
|
|
144
|
+
#### Block element
|
|
144
145
|
|
|
145
146
|
1. Incomplete `<p>` wrapping when there are block elements (e.g., [`<pre>`](http://bhsd-harry.github.io/wikiparser-node/tests.html#%3Cpre%3E%20inside%20a%20link), [`<div>`](http://bhsd-harry.github.io/wikiparser-node/tests.html#Templates%3A%20Scopes%20should%20not%20be%20expanded%20unnecessarily) or even [closing tags](http://bhsd-harry.github.io/wikiparser-node/tests.html#Non-word%20characters%20don't%20terminate%20tag%20names%20(T19663%2C%20T42670%2C%20T54022))).
|
|
146
147
|
1. Mixed lists ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Mixed%20Lists%3A%20Test%204)).
|
|
147
148
|
|
|
148
|
-
|
|
149
|
+
#### Language conversion
|
|
149
150
|
|
|
150
151
|
1. Automatic language conversion is not supported.
|
|
151
152
|
1. Support for manual language conversion is minimal ([Example](https://bhsd-harry.github.io/wikiparser-node/tests.html#Explicit%20session-wise%20one-way%20language%20variant%20mapping%20(A%20flag%20and%20-%20flag))).
|
|
152
153
|
|
|
153
|
-
|
|
154
|
+
#### Miscellaneous
|
|
154
155
|
|
|
155
156
|
1. Illegal HTML entities ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#Illegal%20character%20references%20(T106578%2C%20T113194))).
|
|
156
157
|
|