tree-sitter-okf 0.2.0__tar.gz

tree_sitter_okf-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 tree-sitter-okf contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tree_sitter_okf-0.2.0/NOTICE

@@ -0,0 +1,27 @@
+tree-sitter-okf
+Copyright (c) 2026 tree-sitter-okf contributors
+Licensed under the MIT License (see LICENSE).
+
+This project includes third-party material:
+
+1. tree-sitter-markdown
+   https://github.com/tree-sitter-grammars/tree-sitter-markdown
+   Copyright (c) 2021 Matthias Deiml
+   Licensed under the MIT License (vendor/tree-sitter-markdown/LICENSE).
+
+   The markdown rules in grammar/block.js, grammar/inline.js and
+   grammar/common.js, the markdown half of src/scanner.c, and the files in
+   vendor/tree-sitter-markdown/ are derived from or copied from this project
+   at commit a0a00f817d02412bd92c54d316f164d827b57b5c.  The changes are
+   listed in vendor/tree-sitter-markdown/MERGE.md.
+
+2. Open Knowledge Format example bundles
+   https://github.com/GoogleCloudPlatform/open-knowledge-format
+   Copyright 2026 Google LLC
+   Licensed under the Apache License, Version 2.0
+   (test/fixtures/bundles/LICENSE.md).
+
+   The files in test/fixtures/bundles/ are unmodified copies of the
+   `bundles/` directory at commit ad30107c31c06aec8a7d5636e0d1058118604e6f,
+   used as test fixtures only.  They are not part of the distributed
+   packages.  test/fixtures/bundles/LOCK.json records each file's SHA-256.

tree_sitter_okf-0.2.0/PKG-INFO

@@ -0,0 +1,281 @@
+Metadata-Version: 2.4
+Name: tree-sitter-okf
+Version: 0.2.0
+Summary: Open Knowledge Format (OKF) grammar for tree-sitter: markdown concept documents with YAML frontmatter
+Author: tree-sitter-okf contributors
+License: MIT
+Project-URL: Homepage, https://github.com/antstanley/tree-sitter-okf
+Keywords: incremental,parsing,tree-sitter,okf
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Compilers
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Provides-Extra: core
+Requires-Dist: tree-sitter~=0.24; extra == "core"
+Dynamic: license-file
+
+# tree-sitter-okf
+
+A [tree-sitter](https://tree-sitter.github.io/) grammar for
+[Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/open-knowledge-format)
+v0.2 documents: markdown with YAML frontmatter, organised into knowledge
+bundles.
+
+**Status: 0.2.0.** The version follows OKF: 0.2.x targets OKF v0.2 (see
+[`docs/releasing.md`](docs/releasing.md)). It implements the whole
+[grammar specification](docs/grammar-spec.md) (milestones M0–M5). See
+[`CHANGELOG.md`](CHANGELOG.md) for what is verified and the known
+limitations.
+
+## Why a grammar of its own?
+
+`.md` files in an OKF bundle are more than markdown:
+
+1. **The document shape is OKF's.** A frontmatter block plus a body, with
+   rules about where frontmatter is and isn't allowed.
+2. **The profile is OKF's.** GFM tables (for `# Schema`), footnotes (for
+   per-claim attribution), fenced and indented code, and cross-links.
+   `tree-sitter-markdown` has no footnotes and gates GFM behind build flags.
+3. **The semantic join crosses the frontmatter/body boundary.** Frontmatter
+   `sources[].id` is joined to footnote labels in the body. Only a single
+   tree can express that in one query.
+
+So this grammar parses the whole document into one tree:
+
+```markdown
+---
+type: Metric
+tags: [finance]
+sources:
+- id: gl
+  resource: /tables/gl
+---
+
+# Gross Margin
+
+Revenue minus COGS.[^gl]
+
+[^gl]: General ledger.
+```
+
+```query
+(source_file
+  frontmatter: (frontmatter
+    (block_mapping
+      (block_mapping_pair key: (plain_scalar) value: (plain_scalar))
+      (block_mapping_pair key: (plain_scalar) value: (flow_sequence (plain_scalar)))
+      (block_mapping_pair key: (plain_scalar)
+        value: (block_sequence
+          (block_sequence_item
+            (block_mapping
+              (block_mapping_pair key: (plain_scalar) value: (plain_scalar))
+              (block_mapping_pair key: (plain_scalar) value: (plain_scalar))))))))
+  body: (body
+    (section
+      (atx_heading (atx_h1_marker) heading_content: (inline))
+      (paragraph (inline (footnote_reference label: (footnote_label))))
+      (footnote_definition label: (footnote_label) (paragraph (inline))))))
+```
+
+* **Frontmatter** is parsed natively as [OKF-YAML](docs/okf-yaml.md), a
+  documented subset of YAML. Anything outside it becomes a
+  `yaml_unsupported` node, never an `ERROR`. A missing closing `---` is a
+  `MISSING "---"`, so tools can say exactly what is wrong.
+* **The body** uses the rules of
+  [tree-sitter-markdown](https://github.com/tree-sitter-grammars/tree-sitter-markdown),
+  with block and inline merged into one grammar. GFM is always on, footnotes
+  are added, and node names match upstream. Existing markdown queries and
+  habits carry over. The deltas are listed in
+  [`vendor/tree-sitter-markdown/MERGE.md`](vendor/tree-sitter-markdown/MERGE.md).
+* **Every node type** is documented in [`docs/node-types.md`](docs/node-types.md).
+
+## Installing
+
+| Ecosystem | Package | Language |
+|---|---|---|
+| Node | `tree-sitter-okf` (npm) | `require('tree-sitter-okf')` |
+| Rust | `tree-sitter-okf` (crates.io) | `tree_sitter_okf::LANGUAGE` |
+| Python | `tree-sitter-okf` (PyPI) | `tree_sitter_okf.language()` |
+| Go | `github.com/antstanley/tree-sitter-okf/bindings/go` | `tree_sitter_okf.Language()` |
+| Swift | `TreeSitterOkf` (SwiftPM) | `tree_sitter_okf()` |
+| C | `make && make install` (`libtree-sitter-okf`, `tree-sitter-okf.pc`) | `tree_sitter_okf()` |
+
+The packages are not published yet. Until they are, install from this
+repository (for example `npm install github:antstanley/tree-sitter-okf`,
+or `pip install git+https://github.com/antstanley/tree-sitter-okf`).
+
+The generated parser uses tree-sitter **ABI 14** (spec D12). That is the
+highest ABI Neovim 0.10 loads, and every current runtime supports it.
+
+```js
+const Parser = require('tree-sitter');
+const OKF = require('tree-sitter-okf');
+
+const parser = new Parser();
+parser.setLanguage(OKF);
+const tree = parser.parse(source);
+console.log(tree.rootNode.toString());
+```
+
+```python
+import tree_sitter, tree_sitter_okf
+
+parser = tree_sitter.Parser(tree_sitter.Language(tree_sitter_okf.language()))
+tree = parser.parse(source.encode())
+```
+
+## Queries
+
+| File | Purpose |
+|---|---|
+| `queries/highlights.scm` | Highlighting, with `@markup.*` capture names (Neovim, Helix) |
+| `queries/injections.scm` | Fenced code by info string, HTML. Frontmatter is native, not injected |
+| `queries/injections-fullyaml.scm` | Opt-in: injects the frontmatter as `yaml` for full-YAML hosts |
+| `queries/locals.scm` | Heading scopes (outlines, folding) |
+| `queries/tags.scm` | Concept definitions and cross-link references |
+| `queries/okf/fields.scm` | One capture per well-known frontmatter key (`@okf.field.type`, …) |
+| `queries/okf/links.scm` | Link classification: URI, bundle-relative, relative, fragment |
+| `queries/okf/sources.scm` | `sources[]` entries and their fields |
+| `queries/okf/citations.scm` | Footnote labels, for the `sources[].id` join |
+| `queries/okf/scalars.scm` | A replaceable typing policy for scalars (integer, timestamp, boolean) |
+| `queries/okf/sections.scm` | Conventional headings and `log.md` date headings |
+| `queries/okf/computation.scm` | The `# Computation` heading → code block pairing |
+
+The OKF queries use only node patterns and `#match?`, so they give the same
+results in every host.
+
+## Host helpers
+
+Some OKF facts need something the parse tree does not have: a filename, a
+clock, the absence of a key, or what a value means. The Node and Python
+bindings ship reference helpers for them. The helpers classify documents,
+compute concept ids, turn frontmatter into a plain value, and report trust
+tier, lifecycle status and staleness. They also join citations to footnotes
+and produce OKF conformance findings.
+
+```js
+const { okf } = require('tree-sitter-okf');
+okf.classify('metrics/gross-margin.md', tree); // 'concept'
+okf.trustTier(tree);                           // 'human-reviewed'
+okf.citations(tree).unresolved;                // footnotes with no source
+okf.conformance('metrics/gross-margin.md', tree);
+```
+
+The helpers are specified in [`docs/host-helpers.md`](docs/host-helpers.md)
+and pinned by shared fixtures, so the two implementations agree.
+
+## Editor setup
+
+OKF documents are `.md` files, and `okf` is a superset of the markdown that
+`tree-sitter-markdown` parses. It is safe to register `okf` for markdown
+buffers, ideally only in OKF bundles (spec D11).
+
+### Neovim (nvim-treesitter)
+
+```lua
+-- nvim-treesitter `master` branch API
+local parsers = require('nvim-treesitter.parsers').get_parser_configs()
+parsers.okf = {
+  install_info = {
+    url = 'https://github.com/antstanley/tree-sitter-okf',
+    files = { 'src/parser.c', 'src/scanner.c' },
+    branch = 'main',
+  },
+}
+vim.treesitter.language.register('okf', 'markdown')
+```
+
+Then run `:TSInstall okf`, and copy `queries/*.scm` into
+`~/.config/nvim/queries/okf/`. nvim-treesitter does not fetch queries for
+parsers it does not ship. To use `okf` only inside bundles, register it from
+an autocommand that checks for the bundle's root `index.md`, instead of
+globally.
+
+### Helix
+
+```toml
+# languages.toml
+[[language]]
+name = "okf"
+scope = "source.okf"
+file-types = ["md"]
+roots = ["index.md"]
+injection-regex = "okf"
+
+[[grammar]]
+name = "okf"
+source = { git = "https://github.com/antstanley/tree-sitter-okf", rev = "<commit sha>" }
+```
+
+Helix needs a commit SHA for `rev`, so use the commit of a release tag.
+
+Then run `hx --grammar fetch && hx --grammar build`, and copy `queries/*.scm`
+into `~/.config/helix/runtime/queries/okf/`.
+
+### Command line
+
+```sh
+npx tree-sitter parse path/to/concept.md
+npx tree-sitter query queries/okf/fields.scm path/to/concept.md
+```
+
+## Development
+
+```sh
+npm install                 # tree-sitter CLI and Node bindings
+npm run generate            # tree-sitter generate --abi 14
+npm test                    # corpus: tree-sitter test
+script/test                 # every suite (below); --bench adds the benchmark
+```
+
+`script/test` needs a Python environment with `tree-sitter`, `PyYAML`,
+`tree-sitter-yaml` and this package installed (`pip install -e .`).
+
+| Suite | What it checks |
+|---|---|
+| `tree-sitter test` | Hand-reviewed trees in `test/corpus/` (frontmatter, body, documents, errors) |
+| `script/diff-upstream` | tree-sitter-markdown's own corpus through this grammar. Every difference is a recorded divergence |
+| `script/diff-yaml` | Every frontmatter scalar value against PyYAML |
+| `script/test-queries` | Query captures on fixture documents, native and full-YAML modes |
+| `script/helpers-fixtures.js` | Host helpers against `test/helpers/cases.json` (Node and Python tests run the same cases) |
+| `script/property-test` | No `ERROR` on fuzzed inputs, lossless leaves, incremental == full parse, CRLF/BOM, linear time |
+| `script/node-types --check` | `docs/node-types.md` covers every node type |
+| `script/verify-vendor` | Vendored markdown sources and the OKF fixture bundles match their SHA-256 locks |
+| `script/bench` | Parse throughput and incremental latency against `bench/baseline.json` |
+
+Test fixtures in `test/fixtures/bundles/` are the four official OKF example
+bundles, pinned by `LOCK.json` and refreshed with `script/sync-fixtures`.
+Upstream markdown is refreshed with `script/revendor` (see `MERGE.md`).
+
+Releases are cut with [changesets](https://github.com/changesets/changesets).
+Run `npx changeset` for every user-facing change. Tagging the merged
+"version packages" PR publishes to npm, crates.io and PyPI by trusted
+publishing (see [`docs/releasing.md`](docs/releasing.md)).
+
+The experimental `[[wiki link]]` and `#tag` dialects are off by default.
+Build them with `OKF_DIALECT_WIKILINK=1 OKF_DIALECT_TAGS=1 npm run generate`.
+
+## Repository layout
+
+```
+grammar.js               assembles the layers below
+grammar/                 block.js, inline.js, common.js (merged markdown), frontmatter.js (OKF-YAML)
+src/scanner.c            the external scanner: markdown block + inline, OKF-YAML frontmatter
+queries/                 editor queries and the OKF query library
+bindings/                C, Go, Node, Python, Rust, Swift (+ host helpers for Node, Python)
+docs/                    grammar spec, OKF-YAML subset, node types, host helpers
+test/                    corpus, fixture bundles, query and helper fixtures
+vendor/tree-sitter-markdown/   pinned upstream sources and MERGE.md
+script/                  test, differential, property, bench and maintenance scripts
+bench/                   benchmark baseline
+```
+
+## License
+
+MIT (see [`LICENSE`](LICENSE)). The markdown rules derive from
+tree-sitter-markdown (MIT). The OKF example bundles used as test fixtures
+are Apache-2.0. See [`NOTICE`](NOTICE).

tree_sitter_okf-0.2.0/README.md

@@ -0,0 +1,261 @@
+# tree-sitter-okf
+
+A [tree-sitter](https://tree-sitter.github.io/) grammar for
+[Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/open-knowledge-format)
+v0.2 documents: markdown with YAML frontmatter, organised into knowledge
+bundles.
+
+**Status: 0.2.0.** The version follows OKF: 0.2.x targets OKF v0.2 (see
+[`docs/releasing.md`](docs/releasing.md)). It implements the whole
+[grammar specification](docs/grammar-spec.md) (milestones M0–M5). See
+[`CHANGELOG.md`](CHANGELOG.md) for what is verified and the known
+limitations.
+
+## Why a grammar of its own?
+
+`.md` files in an OKF bundle are more than markdown:
+
+1. **The document shape is OKF's.** A frontmatter block plus a body, with
+   rules about where frontmatter is and isn't allowed.
+2. **The profile is OKF's.** GFM tables (for `# Schema`), footnotes (for
+   per-claim attribution), fenced and indented code, and cross-links.
+   `tree-sitter-markdown` has no footnotes and gates GFM behind build flags.
+3. **The semantic join crosses the frontmatter/body boundary.** Frontmatter
+   `sources[].id` is joined to footnote labels in the body. Only a single
+   tree can express that in one query.
+
+So this grammar parses the whole document into one tree:
+
+```markdown
+---
+type: Metric
+tags: [finance]
+sources:
+- id: gl
+  resource: /tables/gl
+---
+
+# Gross Margin
+
+Revenue minus COGS.[^gl]
+
+[^gl]: General ledger.
+```
+
+```query
+(source_file
+  frontmatter: (frontmatter
+    (block_mapping
+      (block_mapping_pair key: (plain_scalar) value: (plain_scalar))
+      (block_mapping_pair key: (plain_scalar) value: (flow_sequence (plain_scalar)))
+      (block_mapping_pair key: (plain_scalar)
+        value: (block_sequence
+          (block_sequence_item
+            (block_mapping
+              (block_mapping_pair key: (plain_scalar) value: (plain_scalar))
+              (block_mapping_pair key: (plain_scalar) value: (plain_scalar))))))))
+  body: (body
+    (section
+      (atx_heading (atx_h1_marker) heading_content: (inline))
+      (paragraph (inline (footnote_reference label: (footnote_label))))
+      (footnote_definition label: (footnote_label) (paragraph (inline))))))
+```
+
+* **Frontmatter** is parsed natively as [OKF-YAML](docs/okf-yaml.md), a
+  documented subset of YAML. Anything outside it becomes a
+  `yaml_unsupported` node, never an `ERROR`. A missing closing `---` is a
+  `MISSING "---"`, so tools can say exactly what is wrong.
+* **The body** uses the rules of
+  [tree-sitter-markdown](https://github.com/tree-sitter-grammars/tree-sitter-markdown),
+  with block and inline merged into one grammar. GFM is always on, footnotes
+  are added, and node names match upstream. Existing markdown queries and
+  habits carry over. The deltas are listed in
+  [`vendor/tree-sitter-markdown/MERGE.md`](vendor/tree-sitter-markdown/MERGE.md).
+* **Every node type** is documented in [`docs/node-types.md`](docs/node-types.md).
+
+## Installing
+
+| Ecosystem | Package | Language |
+|---|---|---|
+| Node | `tree-sitter-okf` (npm) | `require('tree-sitter-okf')` |
+| Rust | `tree-sitter-okf` (crates.io) | `tree_sitter_okf::LANGUAGE` |
+| Python | `tree-sitter-okf` (PyPI) | `tree_sitter_okf.language()` |
+| Go | `github.com/antstanley/tree-sitter-okf/bindings/go` | `tree_sitter_okf.Language()` |
+| Swift | `TreeSitterOkf` (SwiftPM) | `tree_sitter_okf()` |
+| C | `make && make install` (`libtree-sitter-okf`, `tree-sitter-okf.pc`) | `tree_sitter_okf()` |
+
+The packages are not published yet. Until they are, install from this
+repository (for example `npm install github:antstanley/tree-sitter-okf`,
+or `pip install git+https://github.com/antstanley/tree-sitter-okf`).
+
+The generated parser uses tree-sitter **ABI 14** (spec D12). That is the
+highest ABI Neovim 0.10 loads, and every current runtime supports it.
+
+```js
+const Parser = require('tree-sitter');
+const OKF = require('tree-sitter-okf');
+
+const parser = new Parser();
+parser.setLanguage(OKF);
+const tree = parser.parse(source);
+console.log(tree.rootNode.toString());
+```
+
+```python
+import tree_sitter, tree_sitter_okf
+
+parser = tree_sitter.Parser(tree_sitter.Language(tree_sitter_okf.language()))
+tree = parser.parse(source.encode())
+```
+
+## Queries
+
+| File | Purpose |
+|---|---|
+| `queries/highlights.scm` | Highlighting, with `@markup.*` capture names (Neovim, Helix) |
+| `queries/injections.scm` | Fenced code by info string, HTML. Frontmatter is native, not injected |
+| `queries/injections-fullyaml.scm` | Opt-in: injects the frontmatter as `yaml` for full-YAML hosts |
+| `queries/locals.scm` | Heading scopes (outlines, folding) |
+| `queries/tags.scm` | Concept definitions and cross-link references |
+| `queries/okf/fields.scm` | One capture per well-known frontmatter key (`@okf.field.type`, …) |
+| `queries/okf/links.scm` | Link classification: URI, bundle-relative, relative, fragment |
+| `queries/okf/sources.scm` | `sources[]` entries and their fields |
+| `queries/okf/citations.scm` | Footnote labels, for the `sources[].id` join |
+| `queries/okf/scalars.scm` | A replaceable typing policy for scalars (integer, timestamp, boolean) |
+| `queries/okf/sections.scm` | Conventional headings and `log.md` date headings |
+| `queries/okf/computation.scm` | The `# Computation` heading → code block pairing |
+
+The OKF queries use only node patterns and `#match?`, so they give the same
+results in every host.
+
+## Host helpers
+
+Some OKF facts need something the parse tree does not have: a filename, a
+clock, the absence of a key, or what a value means. The Node and Python
+bindings ship reference helpers for them. The helpers classify documents,
+compute concept ids, turn frontmatter into a plain value, and report trust
+tier, lifecycle status and staleness. They also join citations to footnotes
+and produce OKF conformance findings.
+
+```js
+const { okf } = require('tree-sitter-okf');
+okf.classify('metrics/gross-margin.md', tree); // 'concept'
+okf.trustTier(tree);                           // 'human-reviewed'
+okf.citations(tree).unresolved;                // footnotes with no source
+okf.conformance('metrics/gross-margin.md', tree);
+```
+
+The helpers are specified in [`docs/host-helpers.md`](docs/host-helpers.md)
+and pinned by shared fixtures, so the two implementations agree.
+
+## Editor setup
+
+OKF documents are `.md` files, and `okf` is a superset of the markdown that
+`tree-sitter-markdown` parses. It is safe to register `okf` for markdown
+buffers, ideally only in OKF bundles (spec D11).
+
+### Neovim (nvim-treesitter)
+
+```lua
+-- nvim-treesitter `master` branch API
+local parsers = require('nvim-treesitter.parsers').get_parser_configs()
+parsers.okf = {
+  install_info = {
+    url = 'https://github.com/antstanley/tree-sitter-okf',
+    files = { 'src/parser.c', 'src/scanner.c' },
+    branch = 'main',
+  },
+}
+vim.treesitter.language.register('okf', 'markdown')
+```
+
+Then run `:TSInstall okf`, and copy `queries/*.scm` into
+`~/.config/nvim/queries/okf/`. nvim-treesitter does not fetch queries for
+parsers it does not ship. To use `okf` only inside bundles, register it from
+an autocommand that checks for the bundle's root `index.md`, instead of
+globally.
+
+### Helix
+
+```toml
+# languages.toml
+[[language]]
+name = "okf"
+scope = "source.okf"
+file-types = ["md"]
+roots = ["index.md"]
+injection-regex = "okf"
+
+[[grammar]]
+name = "okf"
+source = { git = "https://github.com/antstanley/tree-sitter-okf", rev = "<commit sha>" }
+```
+
+Helix needs a commit SHA for `rev`, so use the commit of a release tag.
+
+Then run `hx --grammar fetch && hx --grammar build`, and copy `queries/*.scm`
+into `~/.config/helix/runtime/queries/okf/`.
+
+### Command line
+
+```sh
+npx tree-sitter parse path/to/concept.md
+npx tree-sitter query queries/okf/fields.scm path/to/concept.md
+```
+
+## Development
+
+```sh
+npm install                 # tree-sitter CLI and Node bindings
+npm run generate            # tree-sitter generate --abi 14
+npm test                    # corpus: tree-sitter test
+script/test                 # every suite (below); --bench adds the benchmark
+```
+
+`script/test` needs a Python environment with `tree-sitter`, `PyYAML`,
+`tree-sitter-yaml` and this package installed (`pip install -e .`).
+
+| Suite | What it checks |
+|---|---|
+| `tree-sitter test` | Hand-reviewed trees in `test/corpus/` (frontmatter, body, documents, errors) |
+| `script/diff-upstream` | tree-sitter-markdown's own corpus through this grammar. Every difference is a recorded divergence |
+| `script/diff-yaml` | Every frontmatter scalar value against PyYAML |
+| `script/test-queries` | Query captures on fixture documents, native and full-YAML modes |
+| `script/helpers-fixtures.js` | Host helpers against `test/helpers/cases.json` (Node and Python tests run the same cases) |
+| `script/property-test` | No `ERROR` on fuzzed inputs, lossless leaves, incremental == full parse, CRLF/BOM, linear time |
+| `script/node-types --check` | `docs/node-types.md` covers every node type |
+| `script/verify-vendor` | Vendored markdown sources and the OKF fixture bundles match their SHA-256 locks |
+| `script/bench` | Parse throughput and incremental latency against `bench/baseline.json` |
+
+Test fixtures in `test/fixtures/bundles/` are the four official OKF example
+bundles, pinned by `LOCK.json` and refreshed with `script/sync-fixtures`.
+Upstream markdown is refreshed with `script/revendor` (see `MERGE.md`).
+
+Releases are cut with [changesets](https://github.com/changesets/changesets).
+Run `npx changeset` for every user-facing change. Tagging the merged
+"version packages" PR publishes to npm, crates.io and PyPI by trusted
+publishing (see [`docs/releasing.md`](docs/releasing.md)).
+
+The experimental `[[wiki link]]` and `#tag` dialects are off by default.
+Build them with `OKF_DIALECT_WIKILINK=1 OKF_DIALECT_TAGS=1 npm run generate`.
+
+## Repository layout
+
+```
+grammar.js               assembles the layers below
+grammar/                 block.js, inline.js, common.js (merged markdown), frontmatter.js (OKF-YAML)
+src/scanner.c            the external scanner: markdown block + inline, OKF-YAML frontmatter
+queries/                 editor queries and the OKF query library
+bindings/                C, Go, Node, Python, Rust, Swift (+ host helpers for Node, Python)
+docs/                    grammar spec, OKF-YAML subset, node types, host helpers
+test/                    corpus, fixture bundles, query and helper fixtures
+vendor/tree-sitter-markdown/   pinned upstream sources and MERGE.md
+script/                  test, differential, property, bench and maintenance scripts
+bench/                   benchmark baseline
+```
+
+## License
+
+MIT (see [`LICENSE`](LICENSE)). The markdown rules derive from
+tree-sitter-markdown (MIT). The OKF example bundles used as test fixtures
+are Apache-2.0. See [`NOTICE`](NOTICE).

tree_sitter_okf-0.2.0/bindings/python/tree_sitter_okf/__init__.py

@@ -0,0 +1,40 @@
+"""Open Knowledge Format (OKF) grammar for tree-sitter: markdown concept documents with YAML frontmatter"""
+
+from importlib.resources import files as _files
+
+from ._binding import language
+
+
+def _get_query(name, file):
+    query = _files(f"{__package__}.queries") / file
+    globals()[name] = query.read_text()
+    return globals()[name]
+
+
+def __getattr__(name):
+    if name == "HIGHLIGHTS_QUERY":
+        return _get_query("HIGHLIGHTS_QUERY", "highlights.scm")
+    if name == "INJECTIONS_QUERY":
+        return _get_query("INJECTIONS_QUERY", "injections.scm")
+    if name == "LOCALS_QUERY":
+        return _get_query("LOCALS_QUERY", "locals.scm")
+    if name == "TAGS_QUERY":
+        return _get_query("TAGS_QUERY", "tags.scm")
+
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+__all__ = [
+    "language",
+    "HIGHLIGHTS_QUERY",
+    "INJECTIONS_QUERY",
+    "LOCALS_QUERY",
+    "TAGS_QUERY",
+]
+
+
+def __dir__():
+    return sorted(__all__ + [
+        "__all__", "__builtins__", "__cached__", "__doc__", "__file__",
+        "__loader__", "__name__", "__package__", "__path__", "__spec__",
+    ])

tree_sitter_okf-0.2.0/bindings/python/tree_sitter_okf/__init__.pyi

@@ -0,0 +1,8 @@
+from typing import Final
+
+HIGHLIGHTS_QUERY: Final[str]
+INJECTIONS_QUERY: Final[str]
+LOCALS_QUERY: Final[str]
+TAGS_QUERY: Final[str]
+
+def language() -> object: ...

tree_sitter_okf-0.2.0/bindings/python/tree_sitter_okf/binding.c

@@ -0,0 +1,35 @@
+#include <Python.h>
+
+typedef struct TSLanguage TSLanguage;
+
+TSLanguage *tree_sitter_okf(void);
+
+static PyObject* _binding_language(PyObject *Py_UNUSED(self), PyObject *Py_UNUSED(args)) {
+    return PyCapsule_New(tree_sitter_okf(), "tree_sitter.Language", NULL);
+}
+
+static struct PyModuleDef_Slot slots[] = {
+#ifdef Py_GIL_DISABLED
+    {Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
+    {0, NULL}
+};
+
+static PyMethodDef methods[] = {
+    {"language", _binding_language, METH_NOARGS,
+     "Get the tree-sitter language for this grammar."},
+    {NULL, NULL, 0, NULL}
+};
+
+static struct PyModuleDef module = {
+    .m_base = PyModuleDef_HEAD_INIT,
+    .m_name = "_binding",
+    .m_doc = NULL,
+    .m_size = 0,
+    .m_methods = methods,
+    .m_slots = slots,
+};
+
+PyMODINIT_FUNC PyInit__binding(void) {
+    return PyModuleDef_Init(&module);
+}