usfm-grammar-web 3.0.0-alpha.1 → 3.0.0-alpha.10

package/README.md

@@ -1,58 +1,211 @@
 # USFM Grammar
 
 ## Description
-USFM Grammar is a JavaScript library for parsing and converting USFM (Unified Standard Format Markers) to/from USJ (Unified Standard JSON) format. This library provides functionalities to parse USFM strings into a syntax tree and convert them into a JSON-like structure (USJ), and vice versa.
+This is the web alternative to the (USFM-Grammar 3.x)[https://www.npmjs.com/package/usfm-grammar] library to be used from HTML, react etc. USFM Grammar is a JavaScript library for parsing and converting USFM (Unified Standard Format Markers) to/from USJ (Unified Standard JSON) format. This library provides functionalities to parse USFM strings into a syntax tree and convert them into a JSON-like structure (USJ), and vice versa.
 
 ## Installation
 You can install USFM Grammar via npm:
 
-```bash
-npm install usfm-grammar
-```
 
 ## Usage
-Here's how you can use USFM Grammar in your JavaScript/TypeScript projects:
+Here's how you can use USFM Grammar in your react projects:
+```bash
+npm install usfm-grammar-web
+```
 
 ```javascript
-const { USFMParser}  = require('usfm-grammar');
+import React, { useState, useEffect } from 'react';
+import { USFMParser, Filter, Validator } from 'usfm-grammar-web';
+
+function App() {
+  const [result, setResult] = useState(null);
+  const [result2, setResult2] = useState(null);
+
+  useEffect(() => {
+    const initParser = async () => {
+      await USFMParser.init("https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.1/tree-sitter-usfm.wasm",
+                            "https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.1/tree-sitter.wasm");
+      await Validator.init("https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter-usfm.wasm",
+                            "https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter.wasm");
+
+    };
+    initParser();
+  }, []);
+
+  const calculateValue = async () => {
+    const usfmParser = new USFMParser('\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 more text');
+    const output = usfmParser.toUSJ();
+    setResult(JSON.stringify(output));
+
+    const usfmParser2 = new USFMParser(null, output) //initialse from USJ
+    const usfm = usfmParser2.usfm;
+    setResult2(usfm);
+  };
+
+  return (
+    <div className="App">
+      <div>
+        <button onClick={calculateValue}>Test</button>
+        <p>USJ: {result}</p>
+        <p>USFM: {result2}</p>
+      </div>
+    </div>
+  );
+}
+
+export default App;
+```
 
-(async () => {
-  await USFMParser.init();
-  const usfmParser = new USFMParser()
-  const output = usfmParser.usfmToUsj('\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 more text')
+It can be used directly in the HTML script tag too. Please ensure its dependencies are also included(xmldom, ajv, xpath).
+
+```html
+<script type="module">
+  import { USFMParser, Filter, Validator } from 'https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.2/dist/bundle.mjs';
+  console.log('Hello world');
+  (async () => {
+  await USFMParser.init("https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter-usfm.wasm",
+                            "https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter.wasm");
+  await Validator.init("https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter-usfm.wasm",
+                            "https://cdn.jsdelivr.net/npm/usfm-grammar-web@3.0.0-alpha.10/tree-sitter.wasm");
+  const usfmParser = new USFMParser('\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 more text')
+  const output = usfmParser.toUSJ()
   console.log({ output })
-  const usfm = usfmParser.usjToUsfm(output)
+
+  const usfmParser2 = new USFMParser(null, output);
+  const usfm = usfmParser2.usfm;
   console.log({ usfm })
-})();
+  })();
+</script>
 ```
 
-If the above import cause issue in an esm project use the below format:
+## API Documentation
+
+
+### Parsing USFM, checking errors
+
 ```javascript
-import pkg from 'usfm-grammar';
-const USFMParser = pkg.USFMParser;
+const USFM = '\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 some more text'
+const usfmParser = new USFMParser(USFM);
+console.log(usfmParser.errors)
+```
+
+### USJ 
+Here's how you can use USFM Grammar in your JavaScript projects to work with the JSON format, USJ:
+
+```javascript
+const USFM = '\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 some more text'
+const usfmParser = new USFMParser(USFM);
+
+const USJ = usfmParser.toUSJ() // USFM to USJ
+console.log(JSON.stringify(USJ, null, 2));
 
+const usfmParser2 = new USFMParser(usfmString=null, fromUsj=USJ) // USJ to USFM
+const usfmGen = usfmParser2.usfm;
+console.log(usfmGen);
 ```
+Working with USJ, also gives options like filtering selected markers, to edit the original USFM content. To understand more about how `exclude_markers`, `include_markers`, `combine_texts`  and `Filter` works refer the section on [filtering on USJ](#filtering-on-usj)
 
-If using from react, please refer the instructions for it [here](../docs/react-usage.md).
+### USX
+
+To work with the XML format, USX:
+```javascript
+import { DOMImplementation, XMLSerializer } from 'xmldom';
+
+const USFM = '\\id GEN\n\\c 1\n\\p\n\\v 1 In the begining..\\v 2 some more text'
+const usfmParser = new USFMParser(USFM);
+
+const usxElem = usfmParser.toUSX() // USFM to USX
+const usxSerializer = new XMLSerializer();
+const usx = usxSerializer.serializeToString(usxElem);
+
+console.log(usx);
+
+const usfmParser2 = new USFMParser(usfmString=null, fromUsj=null, fromUsx=usxElem) // USX to USFM
+const usfmGen = usfmParser2.usfm;
+console.log(usfmGen);
+```
+
+### Autofix and Validation
+Experimental Validation and Autofix feature for USFM:
+```javascript
+import {Validator} from "usfm-grammar";
+
+const wrongUSFM="\\id GEN\n\\c 1\n\\v 1 test verse"
+const checker = new Validator();
+const resp = checker.isValidUSFM(wrongUSFM); // true or false
+console.log(checker.message) // List of errors if present
+
+const editedUSFM = checker.autoFixUSFM(wrongUSFM);
+console.log(checker.message); // Report on autofix attempt 
+
+```
+
+Validation of USJ format:
+```javascript
+const {Validator} = require("usfm-grammar");
+const simpleUSJ = {
+  type: 'USJ',
+  version: '0.3.0',
+  content: [
+    { type: 'book', marker: 'id', code: 'GEN', content: [] },
+    { type: 'chapter', marker: 'c', number: '1', sid: 'GEN 1' },
+    { type: 'para', marker: 'p', content: [
+        {type: 'verse', marker: 'v', number: 1 },
+        "In the begining..",
+        {type: 'verse', marker: 'v', number: 2 }
+      ] }
+  ]
+}
+const checker = new Validator();
+console.log(checker.isValidUSJ(simpleUSJ));
+console.log(checker.message);
+```
+
+### Filtering on USJ
+The filtering on USJ, the JSON output, is a feature incorporated to allow data extraction, markup cleaning etc. The arguments `exclude_markers` and `include_markers` in the methods `USFMParser.toUSJ()` makes this possible. Also the  `USFMParser.toList()`, can accept these inputs and perform similar operations. There is CLI versions also for these arguments to replicate the filtering feature there.
+
+- *excludeMarkers*
+
+  The first  input parameter to `toUSJ()` and `toList` of `USFMParser` class. Defaults to `null`. When proivded, all markers except those listed will be included in the output.
+
+- *includeMarkers*
+
+  The second input parameter to `toUSJ()` and `toList` of `USFMParser` class. Defaults to `null`. When proivded, only those markers listed will be included in the output. `includeMarkers` is applied before applying `excludeMarkers`. 
 
-## API Documentation
 
-### `USFMParser.init()`
-Initializes the USFMParser. This function must be called before creating instances of `USFMParser`. And can take the grammar and the tree-sitter  files (in wasm format) as arguments, that is included in the package.
+- *combineTexts*
 
-### `USFMParser.usfmToUsj(usfmString: string): Object`
-Converts a USFM string to a USJ object.
+   Fourth input parameter to `toUSJ()` and `toList` of `USFMParser` class. Defaults to `true`. After filtering out makers like paragraphs and characters, we are left with texts from within them, if 'text-in-excluded-parent' is also not excluded. These text snippets may come as separate components in the contents list. When this option is `True`, the consequetive text snippets will be concatinated together. The text concatination is done in a puctuation and space aware manner. If users need more control over the space handling or for any other reason, would prefer the texts snippets as different components in the output, this can be set to `False`.
 
-- `usfmString`: The input USFM string.
+- *usfm_grammar.Filter*
 
-Returns: A JSON-like object representing the USJ.
+  This Class provides a set of enums that would be useful in providing in the `excludeMarkers` and `includeMarkers` inputs rather than users listing out individual markers. The class has following options
+  ```
+    BOOK_HEADERS : identification and introduction markers
+    TITLES : section headings and associated markers
+    COMMENTS : comment markers like \rem
+    PARAGRAPHS : paragraph markers like \p, poetry markers, list table markers
+    CHARACTERS : all character level markups like \em, \w, \wj etc and their nested versions with +
+    NOTES : foot note, cross-reference and their content markers
+    STUDY_BIBLE : \esb and \cat
+    BCV : \id, \c and \v
+    TEXT : 'text-in-excluded-parent'
+    ```
+    To inspect which are the markers in each of these options, it could be just printed out, `print(Filter.TITLES)`. These could be used individually or concatinated to get the desired filtering of markers and data:
+    ```javascript
+    output = usfmParser.toUSJ(null, include_markers=Filter.BCV)
+    output = usfmParser.toUSJ(null, include_markers=Filter.BCV+Filter.TEXT)
+    output = usfmParser.toUSJ(exclude_markers=Filter.PARAGRAPHS+Filter.CHARACTERS)
+    ``` 
+- Inner contents of excluded markers
 
-### `USFMParser.usjToUsfm(usjObject: Object): string`
-Converts a USJ object to a USFM string.
+  For markers like `\p` `\q` etc, by excluding them, we only remove them from the heirachy and retain the inner contents like `\v`, text etc that would be coming inside it. But for certain other markers like `\f`, `\x`, `\esb`  etc, if they are excluded their inner contents are also excluded. Following is the set of all markers, who inner contents are discarded if they are mentioned in `excludeMarkers` or not included in `includeMarkers`.
+  ```
+  BOOK_HEADERS, TITLES, COMMENTS, NOTES, STUDY_BIBLE
+  ```
+  :warning: Generally, it is recommended to NOT use both `exclude_markers` and `includeMarkers` together as it could lead to unexpected behavours and data loss. For instance if `include_makers` has `\fk` and `excludeMarkers` has `\f`, the output will not contain `\fk` as all inner contents of `\f` will be discarded.
 
-- `usjObject`: The input USJ object.
 
-Returns: The converted USFM string.
 
 ## Contributing
 Contributions are welcome! If you find any issues or have suggestions for improvements, feel free to open an issue or create a pull request on [GitHub](https://github.com/your-username/usfm-grammar).