wikiparser-node 1.40.0 → 1.42.0

package/README.md

@@ -5,9 +5,10 @@
 [![npm version](https://badge.fury.io/js/wikiparser-node.svg)](https://www.npmjs.com/package/wikiparser-node)
 [![CodeQL](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/codeql.yml/badge.svg)](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/codeql.yml)
 [![CI](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/node.js.yml/badge.svg)](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/node.js.yml)
+[![NPM downloads](https://img.shields.io/npm/dm/wikiparser-node)](https://www.npmjs.com/package/wikiparser-node)
 [![jsDelivr hits (npm)](https://img.shields.io/jsdelivr/npm/hm/wikiparser-node)](https://www.npmjs.com/package/wikiparser-node)
 [![Codacy Badge](https://app.codacy.com/project/badge/Grade/a2fbe7641031451baca2947ae6d7891f)](https://app.codacy.com/gh/bhsd-harry/wikiparser-node/dashboard)
-![Istanbul coverage](./coverage/badge.svg)
+![Coverage](./coverage/badge.svg)
 
 ## Other Languages
 
@@ -17,11 +18,30 @@
 
 WikiParser-Node is an offline [Wikitext](https://www.mediawiki.org/wiki/Wikitext) parser developed by [Bhsd](https://github.com/bhsd-harry) for the [Node.js](https://nodejs.org/) environment. It can parse almost all [wiki syntax](https://www.mediawiki.org/wiki/Help:Advanced_editing) and generate an [Abstract Syntax Tree (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) ([Try it online](https://bhsd-harry.github.io/wikiparser-node/#editor)). It also allows for easy querying and modification of the AST, and returns the modified Wikitext.
 
-Although WikiParser-Node is not originally designed to convert Wikitext to HTML, it provides a limited capability to do so. [Here](https://bhsd-harry.github.io/wikiparser-website/) is a list of example HTML pages from [MediaWiki.org](https://www.mediawiki.org/) rendered using this package.
+Although WikiParser-Node is not primarily designed to convert Wikitext to HTML, it provides pragmatic HTML rendering for many situations. [Here](https://bhsd-harry.github.io/wikiparser-website/) is a list of example HTML pages from [MediaWiki.org](https://www.mediawiki.org/) rendered using this package.
+
+WikiParser-Node has been extensively tested against the official [MediaWiki PHP parser tests](https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/refs/heads/master/tests/parser/) with ~3,000 test cases, covering various edge cases and peculiarities of Wikitext. These tests are available [here](https://bhsd-harry.github.io/wikiparser-node/tests.html).
+
+## Why WikiParser-Node
+
+- **Round-trip editing for bots and automation**: parse Wikitext into an AST, query and modify nodes, then write back valid Wikitext.
+- **LSP and linting ready for Node.js tooling**: powers [WikiLint](https://www.npmjs.com/package/wikilint) and [Wikitext LSP](https://www.npmjs.com/package/wikitext-lsp).
+- **Browser/editor integration**: works with [CodeMirror](https://www.npmjs.com/package/@bhsd/codemirror-mediawiki), [Monaco](https://www.npmjs.com/package/monaco-wiki), and MediaWiki's official [CodeMirror extension](https://www.mediawiki.org/wiki/Extension:CodeMirror).
+- **Large-scale usage evidence**: [full-dump parsing and linting on English Wikipedia scale](https://lint-wiki-dumps.toolforge.org/) is practical on consumer hardware.
+- **Transparent quality signals**: [CI](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/node.js.yml), [CodeQL](https://github.com/bhsd-harry/wikiparser-node/actions/workflows/codeql.yml), public [parser-test results](https://bhsd-harry.github.io/wikiparser-node/tests.html), and coverage are all visible in this repository.
+
+## Used by
+
+<div align="center">
+<a href="https://www.mediawiki.org/"><img src="https://www.mediawiki.org/static/images/icons/mediawikiwiki.svg" width="50" height="50" alt="MediaWiki"></a>
+<a href="https://helix-editor.com/"><img src="https://helix-editor.com/logo.svg" width="50" height="50" alt="Helix"></a>
+<a href="https://lsp.sublimetext.io/"><img src="https://avatars.githubusercontent.com/u/48095564?s=100" width="50" height="50" alt="LSP for Sublime Text"></a>
+<a href="http://www.qbittorrent.org/"><img src="https://avatars.githubusercontent.com/u/2131270?s=100" width="50" height="50" alt="qBittorrent"></a>
+</div>
 
 ## Other Versions
 
-### Mini (also known as [WikiLint](https://www.npmjs.com/package/wikilint))
+### [WikiLint](https://www.npmjs.com/package/wikilint)
 
 This version provides a [CLI](https://en.wikipedia.org/wiki/Command-line_interface), but only retains the parsing and linting functionality. The parsed AST cannot be modified. It powers the [Wikitext LSP](https://www.npmjs.com/package/wikitext-lsp), which provides multiple language services for editors such as [VS Code](https://marketplace.visualstudio.com/items?itemName=Bhsd.vscode-extension-wikiparser), [Sublime Text](https://lsp.sublimetext.io/language_servers/#mediawiki), and [Helix](https://github.com/helix-editor/helix/wiki/Language-Server-Configurations#wikitext).
 
@@ -31,6 +51,10 @@ A list of available linting rules can be found [here](https://github.com/bhsd-ha
 
 A browser-compatible version, which can be used for code highlighting or as a [LSP](https://microsoft.github.io/language-server-protocol/) plugin in conjunction with editors such as [CodeMirror](https://www.npmjs.com/package/@bhsd/codemirror-mediawiki) and [Monaco](https://www.npmjs.com/package/monaco-wiki) ([Usage example](https://bhsd-harry.github.io/wikiparser-node)). It has been integrated into the MediaWiki official [CodeMirror extension](https://www.mediawiki.org/wiki/Extension:CodeMirror) since Release 1.45.
 
+### [WikiParser-Template](https://www.npmjs.com/package/wikiparser-template)
+
+A lightweight version that only supports parsing and manipulation of templates. This version is designed for use cases where only template processing is needed, such as certain types of bots or web tools (e.g., [GANReviewTool](https://en.wikipedia.org/wiki/User:Novem_Linguae/Scripts/GANReviewTool)) that focus on template manipulation.
+
 ## Installation
 
 ### Node.js
@@ -79,19 +103,42 @@ The generated configuration file will be saved in the [`config` directory](https
 
 ```javascript
 // For example:
-Parser.config = 'frwiki';
+Parser.config = "frwiki";
 ```
 
 ### API usage
 
 Please refer to the [Wiki](https://github.com/bhsd-harry/wikiparser-node/wiki/Home-%28EN%29). In particular, there are some [usage examples](https://github.com/bhsd-harry/wikiparser-node/wiki/Home-%28EN%29#examples) that demonstrate how to use this package to complete various tasks.
 
+#### Round-trip editing quickstart (TypeScript)
+
+```ts
+import Parser from "wikiparser-node";
+import type {TranscludeToken} from "wikiparser-node";
+
+Parser.config = "enwiki";
+const root = Parser.parse("{{Infobox|name=Old}}\nText"),
+	template = root.querySelector<TranscludeToken>("template#Template:Infobox");
+template?.setValue("name", "New");
+const wikitext = String(root);
+assert.strictEqual(wikitext, "{{Infobox|name=New}}\nText");
+```
+
 ## Performance
 
 A full database dump (`*.xml.bz2`) [scan](https://www.npmjs.com/package/lint-wiki-dumps) of English Wikipedia's ~19 million articles (parsing and linting) on a personal MacBook Air takes about 5 hours.
 
+## Best fit
+
+- MediaWiki bot workflows that require robust AST manipulation and round-trip-safe edits.
+- Node.js pipelines for linting and refactoring Wikitext.
+- LSP-based language tooling.
+- Browser-side editing helpers, and gadgets/user scripts that require Wikitext parsing.
+
 ## Known issues
 
+The following limitations are documented for transparency.
+
 ### Parser
 
 1. Memory leaks may occur in rare cases.
@@ -106,8 +153,8 @@ A full database dump (`*.xml.bz2`) [scan](https://www.npmjs.com/package/lint-wik
 
 #### Extension
 
-1. Many extensions are not supported, such as `<indicator>` and `<ref>`.
-1. `&` needs to be escaped in `<syntaxhighlight>` ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#XSS%20is%20escaped)).
+1. Many extensions are not supported.
+1. [Sub-referencing](https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Sub-referencing) is not supported.
 
 #### Transclusion
 
@@ -116,7 +163,7 @@ A full database dump (`*.xml.bz2`) [scan](https://www.npmjs.com/package/lint-wik
 
 #### Heading
 
-1. The table of contents (TOC) is not supported.
+1. RTL support in the table of contents (TOC) ([Example](http://bhsd-harry.github.io/wikiparser-node/tests.html#span%20tags%20with%20directionality%20in%20TOC)).
 
 #### HTML tag
 
@@ -150,6 +197,7 @@ A full database dump (`*.xml.bz2`) [scan](https://www.npmjs.com/package/lint-wik
 
 1. Automatic language conversion is not supported.
 1. Support for manual language conversion is minimal ([Example](https://bhsd-harry.github.io/wikiparser-node/tests.html#Explicit%20session-wise%20one-way%20language%20variant%20mapping%20(A%20flag%20and%20-%20flag))).
+1. BCP 47 language codes are not supported in language conversion ([Example](https://bhsd-harry.github.io/wikiparser-node/tests.html#Explicit%20definition%20of%20language%20variant%20alternatives%20(BCP%2047%20codes))).
 
 #### Miscellaneous