plain-parser 0.1.4.dev9__tar.gz → 0.1.4.dev10__tar.gz

plain_parser-0.1.4.dev10/PKG-INFO

@@ -0,0 +1,125 @@
+Metadata-Version: 2.5
+Name: plain-parser
+Version: 0.1.4.dev10
+Summary: Parser for ***plain specification files
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Compilers
+Requires-Python: >=3.11
+Requires-Dist: mistletoe<2,>=1.6.0
+Requires-Dist: python-frontmatter<2,>=1.3.0
+Requires-Dist: python-liquid2<0.5,>=0.4.0
+Requires-Dist: pyyaml<7,>=6
+Provides-Extra: dev
+Requires-Dist: black==26.5.1; extra == 'dev'
+Requires-Dist: flake8==7.3.0; extra == 'dev'
+Requires-Dist: isort==8.0.1; extra == 'dev'
+Requires-Dist: mypy==2.3.0; extra == 'dev'
+Requires-Dist: pytest==9.1.1; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# plain-parser
+
+Python parser for [`***plain`](https://www.plainlang.org/docs/) specification files.
+
+Reads a `.plain` module and its `import` / `requires` chain, resolves Liquid templates,
+validates concepts and linked resources, and returns the marshalled specification tree
+plus the ordered list of functionalities (FRIDs).
+
+## Install
+
+```bash
+pip install plain-parser
+```
+
+## Command line
+
+```
+plain-parser check <file.plain> [--template-dir DIR] [--config-name NAME]
+plain-parser parse <file.plain> [--template-dir DIR] [--config-name NAME]
+```
+
+Both commands parse the module and its `requires` chain, check that every linked resource is an existing
+file (not a directory), and walk the module's functionalities. They stop at the first error.
+
+### `check`
+
+Validates the module. In addition to the steps above it reads every linked file and rejects binary files and
+files holding a large base64 blob.
+
+```
+$ plain-parser check my_spec.plain
+my_spec.plain: OK
+```
+
+Exit `0` when the module is valid. On an error, `Error: …` is written to stderr and the exit code is `1`.
+
+### `parse`
+
+Prints the parse result as one UTF-8 JSON document on stdout. Linked files are located but not read.
+
+```
+$ plain-parser parse my_spec.plain
+{
+  "format_version": 1,
+  "modules": [{"module": "...", "tree": {...}}, ...],
+  "resources": {...}
+}
+```
+
+- `format_version`: `1`.
+- `modules`: every module in the `requires` chain, in build order: deepest ancestor first, the module itself last.
+  Each entry has the module name and its specification tree, in the shape described under *Python* below.
+- `resources`: every linked-resource target in the chain, as written in the spec, mapped to the path it resolves
+  to, relative to the entry module's directory, with `/` as separator: `notes.md` when found next to the module,
+  `../tpl/notes.md` when found in a template directory. The directory part is lexical (symlinks not resolved) and
+  the target is appended as written (`..` kept), so joining the value onto the entry module's directory names the
+  file the parser matched. When no relative path exists (another drive on Windows) the value is the absolute path,
+  still with `/`. Two spellings of one file (`notes.md`, `./notes.md`) are two keys.
+
+Exit `0` on success. On a parser error the JSON is `{"format_version": 1, "type": "<error class>",
+"message": "…"}` and the exit code is `1`.
+
+### Options and exit codes
+
+Modules and `{% include %}` templates are looked up in the `.plain` file's directory, then in `--template-dir`.
+When `--template-dir` is not given, it is read from the `template-dir` key of `config.yaml` (or the file named by
+`--config-name`), found next to the `.plain` file or in the working directory. Finding it in both is a usage
+error. The config file is read even when `--template-dir` is given, so a malformed one fails the run either way.
+Other config keys are ignored.
+
+| Exit code | Meaning |
+|---|---|
+| `0` | Success |
+| `1` | The module, its chain, or a linked resource is invalid |
+| `2` | Usage error, including an ambiguous or malformed config file |
+| `130` | Interrupted |
+
+## Python
+
+```python
+from plain_parser import plain_file_parser
+
+module_name, plain_source_tree, required_modules = plain_file_parser("my_spec.plain", ["."])
+```
+
+`plain_source_tree` is a dict keyed by section (`definitions`, `implementation reqs`,
+`test reqs`, `functional specs`), each a list of `{"markdown": ..., "linked_resources": [...]}`
+entries. `plain_parser.plain_spec` has helpers for walking functionalities
+(`get_frids`, `get_specifications_for_frid`, `collect_linked_resources`).
+
+## Development
+
+```bash
+uv venv --python 3.11 && source .venv/bin/activate
+uv pip install -e ".[dev]"
+pytest
+black . --check && isort . --check-only && flake8 . && mypy src
+```
+
+## License
+
+MIT, see [LICENSE](LICENSE).

plain_parser-0.1.4.dev10/README.md

@@ -0,0 +1,102 @@
+# plain-parser
+
+Python parser for [`***plain`](https://www.plainlang.org/docs/) specification files.
+
+Reads a `.plain` module and its `import` / `requires` chain, resolves Liquid templates,
+validates concepts and linked resources, and returns the marshalled specification tree
+plus the ordered list of functionalities (FRIDs).
+
+## Install
+
+```bash
+pip install plain-parser
+```
+
+## Command line
+
+```
+plain-parser check <file.plain> [--template-dir DIR] [--config-name NAME]
+plain-parser parse <file.plain> [--template-dir DIR] [--config-name NAME]
+```
+
+Both commands parse the module and its `requires` chain, check that every linked resource is an existing
+file (not a directory), and walk the module's functionalities. They stop at the first error.
+
+### `check`
+
+Validates the module. In addition to the steps above it reads every linked file and rejects binary files and
+files holding a large base64 blob.
+
+```
+$ plain-parser check my_spec.plain
+my_spec.plain: OK
+```
+
+Exit `0` when the module is valid. On an error, `Error: …` is written to stderr and the exit code is `1`.
+
+### `parse`
+
+Prints the parse result as one UTF-8 JSON document on stdout. Linked files are located but not read.
+
+```
+$ plain-parser parse my_spec.plain
+{
+  "format_version": 1,
+  "modules": [{"module": "...", "tree": {...}}, ...],
+  "resources": {...}
+}
+```
+
+- `format_version`: `1`.
+- `modules`: every module in the `requires` chain, in build order: deepest ancestor first, the module itself last.
+  Each entry has the module name and its specification tree, in the shape described under *Python* below.
+- `resources`: every linked-resource target in the chain, as written in the spec, mapped to the path it resolves
+  to, relative to the entry module's directory, with `/` as separator: `notes.md` when found next to the module,
+  `../tpl/notes.md` when found in a template directory. The directory part is lexical (symlinks not resolved) and
+  the target is appended as written (`..` kept), so joining the value onto the entry module's directory names the
+  file the parser matched. When no relative path exists (another drive on Windows) the value is the absolute path,
+  still with `/`. Two spellings of one file (`notes.md`, `./notes.md`) are two keys.
+
+Exit `0` on success. On a parser error the JSON is `{"format_version": 1, "type": "<error class>",
+"message": "…"}` and the exit code is `1`.
+
+### Options and exit codes
+
+Modules and `{% include %}` templates are looked up in the `.plain` file's directory, then in `--template-dir`.
+When `--template-dir` is not given, it is read from the `template-dir` key of `config.yaml` (or the file named by
+`--config-name`), found next to the `.plain` file or in the working directory. Finding it in both is a usage
+error. The config file is read even when `--template-dir` is given, so a malformed one fails the run either way.
+Other config keys are ignored.
+
+| Exit code | Meaning |
+|---|---|
+| `0` | Success |
+| `1` | The module, its chain, or a linked resource is invalid |
+| `2` | Usage error, including an ambiguous or malformed config file |
+| `130` | Interrupted |
+
+## Python
+
+```python
+from plain_parser import plain_file_parser
+
+module_name, plain_source_tree, required_modules = plain_file_parser("my_spec.plain", ["."])
+```
+
+`plain_source_tree` is a dict keyed by section (`definitions`, `implementation reqs`,
+`test reqs`, `functional specs`), each a list of `{"markdown": ..., "linked_resources": [...]}`
+entries. `plain_parser.plain_spec` has helpers for walking functionalities
+(`get_frids`, `get_specifications_for_frid`, `collect_linked_resources`).
+
+## Development
+
+```bash
+uv venv --python 3.11 && source .venv/bin/activate
+uv pip install -e ".[dev]"
+pytest
+black . --check && isort . --check-only && flake8 . && mypy src
+```
+
+## License
+
+MIT, see [LICENSE](LICENSE).

plain_parser-0.1.4.dev10/src/plain_parser/cli.py

@@ -0,0 +1,233 @@
+"""Command-line entry point: ``plain-parser {check,parse} <file.plain> [--template-dir DIR] [--config-name NAME]``."""
+
+import argparse
+import json
+import os
+import sys
+from dataclasses import dataclass
+
+import yaml
+
+from plain_parser import loaders, plain_file, plain_spec
+
+DEFAULT_CONFIG_NAME = "config.yaml"
+TEMPLATE_DIR_CONFIG_KEYS = ("template_dir", "template-dir")
+PARSE_FORMAT_VERSION = 1
+
+
+class AmbiguousConfigFileError(Exception):
+    pass
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="plain-parser", description="Tools for ***plain specification files.")
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    check_parser = subparsers.add_parser(
+        "check",
+        help="Validate a .plain module and its requires chain, and read every linked file: "
+        "binary files and large base64 blobs are rejected.",
+    )
+    parse_parser = subparsers.add_parser(
+        "parse",
+        help="Print the parse result of a .plain module and its requires chain as JSON. Linked files are "
+        "located but never read.",
+    )
+    for subparser in (check_parser, parse_parser):
+        subparser.add_argument("plain_file", help="Path to the .plain module.")
+        subparser.add_argument(
+            "--template-dir",
+            help="Directory searched for modules and templates after the .plain file's own directory. "
+            "Overrides the template-dir value of the config file.",
+        )
+        subparser.add_argument(
+            "--config-name",
+            default=DEFAULT_CONFIG_NAME,
+            help="Name of the config file to read template-dir from. Looked up in the .plain file's directory "
+            "and the current working directory. Defaults to %(default)s.",
+        )
+    return parser
+
+
+def resolve_config_file(config_name: str, plain_file_path: str) -> str | None:
+    """The config file next to the .plain file, else the one in the working directory, else None."""
+    plain_file_dir = os.path.dirname(os.path.abspath(plain_file_path))
+    plain_dir_config = os.path.normpath(os.path.join(plain_file_dir, config_name))
+    cwd_config = os.path.normpath(os.path.join(os.getcwd(), config_name))
+
+    in_plain_dir = os.path.exists(plain_dir_config)
+    in_cwd = os.path.exists(cwd_config)
+    if in_plain_dir and in_cwd and plain_dir_config != cwd_config:
+        raise AmbiguousConfigFileError(
+            f"Config file '{config_name}' was found in two locations:\n"
+            f"  - Plain file directory: {plain_file_dir}\n"
+            f"  - Current working directory: {os.getcwd()}\n"
+            f"Remove the config file from one of these locations to resolve the ambiguity."
+        )
+    if in_plain_dir:
+        return plain_dir_config
+    if in_cwd:
+        return cwd_config
+    return None
+
+
+def template_dir_from_config(config_file: str) -> str | None:
+    """The template-dir value of the config file, resolved against the file's directory. Other keys are ignored."""
+    with open(config_file) as f:
+        config = yaml.safe_load(f) or {}
+
+    for key in TEMPLATE_DIR_CONFIG_KEYS:
+        value = config.get(key)
+        if value:
+            value = os.path.expanduser(str(value))
+            return value if os.path.isabs(value) else os.path.join(os.path.dirname(config_file), value)
+    return None
+
+
+@dataclass
+class ParseResult:
+    modules: list[
+        tuple[str, dict]
+    ]  # (module name, plain source tree) for the chain, deepest ancestor first, entry last.
+    # Each link target in the chain, as written, to the path it resolves to, relative to the entry module's dir
+    # with `/` as separator (loaders.relative_resource_path).
+    resources: dict[str, str]
+
+
+def parse_spec(plain_file_path: str, template_dir: str | None, load_resources: bool) -> ParseResult:
+    """Parse the module and its requires chain, resolve its links, and walk the entry's FRIDs; raise on the first error.
+
+    With ``load_resources``, each module's linked files are also read and checked (text only, no large base64 blob) right
+    after that module's links are collected, so a resource error is reported before a FRID-walk error.
+    """
+    # Absolute, so the chain parse and the resource paths below search the same directories.
+    template_dirs = [os.path.dirname(os.path.abspath(plain_file_path))]
+    if template_dir:
+        template_dirs.append(os.path.abspath(template_dir))
+
+    chain = plain_file.parse_module_chain(os.path.basename(plain_file_path), template_dirs)
+
+    # The target joined as written onto its template dir (`..` kept, symlinks not resolved), made relative to the
+    # entry module's dir so the result does not depend on where the project sits on the machine.
+    resources: dict[str, str] = {}
+    for module_name, plain_source_tree in chain:
+        resources_list: list[dict] = []
+        plain_spec.collect_linked_resources(plain_source_tree, resources_list, None, True)
+        if load_resources:
+            loaders.load_linked_resources(template_dirs, resources_list, module_name)
+        for resource in resources_list:
+            target = resource["target"]
+            resource_dir = loaders.find_resource_dir(template_dirs, target)
+            resources[target] = loaders.relative_resource_path(template_dirs[0], resource_dir, target)
+
+    _, top_plain_source_tree = chain[-1]
+    for frid in plain_spec.get_frids(top_plain_source_tree):
+        plain_spec.get_specifications_for_frid(top_plain_source_tree, frid)
+
+    return ParseResult(chain, resources)
+
+
+def check(plain_file_path: str, template_dir: str | None) -> None:
+    """Raise if the module, its requires chain, or its linked resources are invalid."""
+    parse_spec(plain_file_path, template_dir, load_resources=True)
+
+
+def _error_output(type_name: str, message: str) -> dict:
+    return {"format_version": PARSE_FORMAT_VERSION, "type": type_name, "message": message}
+
+
+def _escape(text: str) -> str:
+    """The text with every non-ASCII character as a backslash escape, so it always encodes."""
+    return text.encode("ascii", "backslashreplace").decode("ascii")
+
+
+def _encode_output(output: dict) -> bytes:
+    """The output as UTF-8 JSON; raise if it is not JSON-serialisable or holds text that is not valid UTF-8."""
+    return (json.dumps(output, indent=2, ensure_ascii=False) + "\n").encode("utf-8")
+
+
+def parse(plain_file_path: str, template_dir: str | None) -> int:
+    """Print the parse result, or the parser error, as one UTF-8 JSON document on stdout; return the exit code."""
+    output: dict
+    try:
+        result = parse_spec(plain_file_path, template_dir, load_resources=False)
+        output = {
+            "format_version": PARSE_FORMAT_VERSION,
+            "modules": [{"module": module_name, "tree": tree} for module_name, tree in result.modules],
+            "resources": result.resources,
+        }
+        exit_code = 0
+    except Exception as e:
+        output = _error_output(type(e).__name__, str(e))
+        exit_code = 1
+
+    try:
+        data = _encode_output(output)
+    except UnicodeEncodeError as e:
+        # A file or directory name that is not valid UTF-8, which is allowed on Linux.
+        if exit_code == 1:
+            # A parser error whose message names such a file. Keep its type and escape the name.
+            output = _error_output(output["type"], _escape(output["message"]))
+        else:
+            # A valid result that contains such a name. Report the output line that contains it.
+            line_start = e.object.rfind("\n", 0, e.start) + 1
+            line_end = e.object.find("\n", e.end)
+            line = e.object[line_start : line_end if line_end != -1 else None].strip().rstrip(",")
+            output = _error_output(
+                type(e).__name__, f"The parse result holds text that is not valid UTF-8: {_escape(line)}"
+            )
+        data = _encode_output(output)
+        exit_code = 1
+    except Exception as e:
+        # A value in the result that JSON cannot represent. Report it as a JSON error.
+        data = _encode_output(_error_output(type(e).__name__, _escape(str(e))))
+        exit_code = 1
+
+    # Write the UTF-8 bytes to stdout's byte stream, because sys.stdout encodes text with the system encoding, which
+    # may not be UTF-8.
+    buffer = getattr(sys.stdout, "buffer", None)
+    if buffer is None:
+        # A Python caller replaced stdout with a stream that takes only text.
+        sys.stdout.write(data.decode("utf-8"))
+        return exit_code
+
+    try:
+        sys.stdout.flush()  # Send out anything already written to sys.stdout first.
+        buffer.write(data)
+        buffer.flush()
+    except BrokenPipeError:
+        # The reader closed the pipe early. Send stdout to /dev/null, so Python's flush at exit does not fail again,
+        # and keep the exit code.
+        devnull = os.open(os.devnull, os.O_WRONLY)
+        os.dup2(devnull, sys.stdout.fileno())
+    return exit_code
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+
+    try:
+        # The config file is resolved even when --template-dir is given, so an ambiguous config is always an error.
+        config_template_dir = _template_dir_from_config(parser, args)
+        if args.command == "parse":
+            return parse(args.plain_file, args.template_dir or config_template_dir)
+        check(args.plain_file, args.template_dir or config_template_dir)
+    except KeyboardInterrupt:
+        return 130
+    except Exception as e:
+        print(f"Error: {e}", file=sys.stderr)
+        return 1
+
+    print(f"{args.plain_file}: OK")
+    return 0
+
+
+def _template_dir_from_config(parser: argparse.ArgumentParser, args: argparse.Namespace) -> str | None:
+    try:
+        config_file = resolve_config_file(args.config_name, args.plain_file)
+        return template_dir_from_config(config_file) if config_file is not None else None
+    except AmbiguousConfigFileError as e:
+        parser.error(str(e))
+    except Exception as e:
+        parser.error(f"Error reading config file: {e}")

{plain_parser-0.1.4.dev9 → plain_parser-0.1.4.dev10}/src/plain_parser/loaders.py

@@ -36,20 +36,41 @@ def open_from(dirs, file_name):
     return None
 
 
-def resolve_linked_resource(dirs, file_name):
-    """Return the first existing path for file_name across dirs, or None if not found.
+def find_resource_dir(dirs, file_name):
+    """Return the first dir in which file_name exists, or None if not found.
 
     Mirrors the search order used by open_from so that validating a linked resource and
     loading it can never disagree.
     """
     for dir in dirs:
-        full_file_name = os.path.join(dir, file_name)
-        if os.path.exists(full_file_name):
-            return full_file_name
+        if os.path.exists(os.path.join(dir, file_name)):
+            return dir
 
     return None
 
 
+def resolve_linked_resource(dirs, file_name):
+    """Return the first existing path for file_name across dirs, or None if not found."""
+    resource_dir = find_resource_dir(dirs, file_name)
+    return None if resource_dir is None else os.path.join(resource_dir, file_name)
+
+
+def relative_resource_path(base_dir: str, resource_dir: str, target: str) -> str:
+    """The path of target in resource_dir, relative to base_dir, with `/` as separator.
+
+    The dir part is computed lexically (symlinks not resolved) and the target is appended as written (`..` kept),
+    so joining the result onto base_dir names the same file as os.path.join(resource_dir, target). When no relative
+    path exists (another drive on Windows), the absolute path is returned instead, still with `/`.
+    """
+    try:
+        rel_dir = os.path.relpath(resource_dir, base_dir)
+    except ValueError:
+        return os.path.join(resource_dir, target).replace(os.sep, "/")
+    if rel_dir == os.curdir:
+        return target
+    return rel_dir.replace(os.sep, "/") + "/" + target
+
+
 def load_linked_resources(template_dirs: list[str], resources_list, module_name: str):
     linked_resources = {}
 

plain_parser-0.1.4.dev10/tests/data/cli/parent_link/a/spec/top.plain

@@ -0,0 +1,7 @@
+***implementation reqs***
+
+- A req.
+
+***functional specs***
+
+- A functionality. See [the notes](../shared/notes.md).

plain_parser-0.1.4.dev10/tests/data/cli/parent_link/b/shared/notes.md

@@ -0,0 +1 @@
+notes

plain_parser-0.1.4.dev10/tests/data/cli/resource_before_frid_walk/base.plain

@@ -0,0 +1,8 @@
+***implementation reqs***
+
+- :Implementation: is written in Python.
+- :Implementation: follows [the data](../binary.bin).
+
+***functional specs***
+
+- :Implementation: has an entry point.

plain_parser-0.1.4.dev10/tests/data/cli/resource_before_frid_walk/snippet.plain

@@ -0,0 +1 @@
+the variable {{ name | code_variable }}

plain_parser-0.1.4.dev10/tests/data/cli/resource_before_frid_walk/top.plain

@@ -0,0 +1,14 @@
+---
+requires:
+  - base
+---
+
+***implementation reqs***
+
+- :Implementation: is written in Python.
+
+***functional specs***
+
+- :Implementation: names {% include "snippet.plain", name: "first" %}.
+
+- :Implementation: names {% include "snippet.plain", name: "second" %}.

plain_parser-0.1.4.dev10/tests/data/cli/shared_link_helper.plain

@@ -0,0 +1,7 @@
+***implementation reqs***
+
+- A req.
+
+***functional specs***
+
+- A functionality. See [the notes](notes.md).

plain_parser-0.1.4.dev10/tests/data/cli/shared_link_top.plain

@@ -0,0 +1,12 @@
+---
+requires:
+  - shared_link_helper
+---
+
+***implementation reqs***
+
+- A req.
+
+***functional specs***
+
+- A functionality. Also [the notes](notes.md).

plain_parser-0.1.4.dev10/tests/data/cli/template_link/spec/top.plain

@@ -0,0 +1,7 @@
+***implementation reqs***
+
+- A req.
+
+***functional specs***
+
+- A functionality. See [the notes](notes.md).

plain_parser-0.1.4.dev10/tests/data/cli/template_link/tpl/notes.md

@@ -0,0 +1 @@
+notes