amati 0.1.1__tar.gz → 0.2__tar.gz

amati-0.2/.dockerignore

@@ -0,0 +1 @@
+.venv

amati-0.2/Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.13-slim
+
+ENV PYTHONUNBUFFERED=1
+
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
+
+WORKDIR /app
+COPY pyproject.toml uv.lock README.md ./
+COPY amati/ amati/
+
+RUN uv sync --locked --no-dev
+
+RUN adduser --disabled-password --gecos '' appuser && chown -R appuser /app
+USER appuser
+
+ENTRYPOINT ["uv", "run", "python", "amati/amati.py"]

{amati-0.1.1 → amati-0.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: amati
-Version: 0.1.1
+Version: 0.2
 Summary: Validates that a .yaml or .json file conforms to the OpenAPI Specifications 3.x.
 Project-URL: Homepage, https://github.com/ben-alexander/amati
 Project-URL: Issues, https://github.com/ben-alexander/amati/issues
@@ -15,6 +15,7 @@ Classifier: Topic :: Software Development :: Testing :: Acceptance
 Requires-Python: >=3.13
 Requires-Dist: abnf>=2.3.1
 Requires-Dist: idna>=3.10
+Requires-Dist: jinja2>=3.1.6
 Requires-Dist: jsonpickle>=4.1.1
 Requires-Dist: jsonschema>=4.24.0
 Requires-Dist: pydantic>=2.11.5
@@ -23,26 +24,64 @@ Description-Content-Type: text/markdown
 
 # amati
 
-A programme designed to validate that a file conforms to [OpenAPI Specification](https://spec.openapis.org/oas/v3.1.1.html) (OAS).
-
-Currently a proof of concept.
+amati is designed to validate that a file conforms to the [OpenAPI Specification v3.x](https://spec.openapis.org/) (OAS).
 
 ## Name
 
 amati means to observe in Malay, especially with attention to detail. It's also one of the plurals of beloved or favourite in Italian.
 
+## Usage
+
+```sh
+python amati/amati.py --help
+usage: amati [-h] [-s SPEC] [-cc] [-d DISCOVER] [-l] [-hr]
+
+Tests whether a OpenAPI specification is valid. Will look an openapi.json or openapi.yaml file in the directory that
+amati is called from. If --discover is set will search the directory tree. If the specification does not follow the
+naming recommendation the --spec switch should be used. Creates a file <filename>.errors.json alongside the original
+specification containing a JSON representation of all the errors.
+
+options:
+  -h, --help            show this help message and exit
+  -s, --spec SPEC       The specification to be parsed
+  -cc, --consistency-check
+                        Runs a consistency check between the input specification and the parsed specification
+  -d, --discover DISCOVER
+                        Searches the specified directory tree for openapi.yaml or openapi.json.
+  -l, --local           Store errors local to the caller in a file called <file-name>.errors.json; a .amati/ directory
+                        will be created.
+  -hr, --html-report    Creates an HTML report of the errors, called <file-name>.errors.html, alongside the original
+                        file or in a .amati/ directory if the --local switch is used
+```
+
+A Dockerfile is available on [DockerHub](https://hub.docker.com/r/benale/amati/tags)
+
+To run against a specific specification the location of the specification needs to be mounted in the container.
+
+```sh
+docker run -v "<path-to-mount>:/<mount-name> amati <options>
+```
+
+e.g. 
+
+```sh
+docker run -v /Users/myuser/myrepo:/data amati --spec data/myspec.yaml --hr
+```
+
 ## Architecture
 
 This uses Pydantic, especially the validation, and Typing to construct the entire OAS as a single data type. Passing a dictionary to the top-level data type runs all the validation in the Pydantic models constructing a single set of inherited classes and datatypes that validate that the API specification is accurate.
 
 Where the specification conforms, but relies on implementation-defined behavior (e.g. [data type formats](https://spec.openapis.org/oas/v3.1.1.html#data-type-format)), a warning will be raised.
 
-## Requirements
+## Contributing
+
+### Requirements
 
 * The latest version of [uv](https://docs.astral.sh/uv/)
 * [git 2.49+](https://git-scm.com/downloads/linux)
 
-## Testing and formatting
+### Testing and formatting
 
 This project uses:
 
@@ -54,7 +93,7 @@ This project uses:
 * [Black](https://black.readthedocs.io/en/stable/index.html) for automated formatting
 * [isort](https://pycqa.github.io/isort/) for import sorting
 
-It's expected that there are no errors, no surviving mutants and 100% of the code is reached and executed.
+It's expected that there are no errors and 100% of the code is reached and executed. The strategy for test coverage is based on parsing test specifications and not unit tests.
 
 amati runs tests on external specifications, detailed in `tests/data/.amati.tests.yaml`. To be able to run these tests the appropriate GitHub repos need to be local. Specific revisions of the repos can be downloaded by running
 
@@ -62,13 +101,15 @@ amati runs tests on external specifications, detailed in `tests/data/.amati.test
 python scripts/tests/setup_test_specs.py
 ```
 
-To run everything, from linting, type checking to downloading test specs run:
+To run everything, from linting, type checking to downloading test specs and building and testing the Docker image run:
 
 ```sh
 sh bin/checks.sh
 ```
 
-## Building
+You will need to have Docker installed.
+
+### Building
 
 The project uses a [`pyproject.toml` file](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#writing-pyproject-toml) to determine what to build.
 
@@ -80,6 +121,27 @@ uv venv
 uv sync
 ```
 
+### Docker
+
+A development Docker image is provided, `Dockerfile.dev`, to build:
+
+```sh
+docker build -t amati -f Dockerfile .
+```
+
+and to run against a specific specification the location of the specification needs to be mounted in the container.
+
+```sh
+docker run -v "<path-to-mount>:/<mount-name> amati <options>
+```
+
+This can be tested against a provided specification, from the root directory
+
+```sh
+docker run --detach -v "$(pwd):/data" amati
+```
+
+
 ### Data
 
 There are some scripts to create the data needed by the project, for example, all the possible licences. If the data needs to be refreshed this can be done by running the contents of `/scripts/data`.

amati-0.2/README.md

@@ -0,0 +1,127 @@
+# amati
+
+amati is designed to validate that a file conforms to the [OpenAPI Specification v3.x](https://spec.openapis.org/) (OAS).
+
+## Name
+
+amati means to observe in Malay, especially with attention to detail. It's also one of the plurals of beloved or favourite in Italian.
+
+## Usage
+
+```sh
+python amati/amati.py --help
+usage: amati [-h] [-s SPEC] [-cc] [-d DISCOVER] [-l] [-hr]
+
+Tests whether a OpenAPI specification is valid. Will look an openapi.json or openapi.yaml file in the directory that
+amati is called from. If --discover is set will search the directory tree. If the specification does not follow the
+naming recommendation the --spec switch should be used. Creates a file <filename>.errors.json alongside the original
+specification containing a JSON representation of all the errors.
+
+options:
+  -h, --help            show this help message and exit
+  -s, --spec SPEC       The specification to be parsed
+  -cc, --consistency-check
+                        Runs a consistency check between the input specification and the parsed specification
+  -d, --discover DISCOVER
+                        Searches the specified directory tree for openapi.yaml or openapi.json.
+  -l, --local           Store errors local to the caller in a file called <file-name>.errors.json; a .amati/ directory
+                        will be created.
+  -hr, --html-report    Creates an HTML report of the errors, called <file-name>.errors.html, alongside the original
+                        file or in a .amati/ directory if the --local switch is used
+```
+
+A Dockerfile is available on [DockerHub](https://hub.docker.com/r/benale/amati/tags)
+
+To run against a specific specification the location of the specification needs to be mounted in the container.
+
+```sh
+docker run -v "<path-to-mount>:/<mount-name> amati <options>
+```
+
+e.g. 
+
+```sh
+docker run -v /Users/myuser/myrepo:/data amati --spec data/myspec.yaml --hr
+```
+
+## Architecture
+
+This uses Pydantic, especially the validation, and Typing to construct the entire OAS as a single data type. Passing a dictionary to the top-level data type runs all the validation in the Pydantic models constructing a single set of inherited classes and datatypes that validate that the API specification is accurate.
+
+Where the specification conforms, but relies on implementation-defined behavior (e.g. [data type formats](https://spec.openapis.org/oas/v3.1.1.html#data-type-format)), a warning will be raised.
+
+## Contributing
+
+### Requirements
+
+* The latest version of [uv](https://docs.astral.sh/uv/)
+* [git 2.49+](https://git-scm.com/downloads/linux)
+
+### Testing and formatting
+
+This project uses:
+
+* [Pytest](https://docs.pytest.org/en/stable/) as a testing framework
+* [PyLance](https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance) on strict mode for type checking
+* [Pylint](https://www.pylint.org/) as a linter, using a modified version from [Google's style guide](https://google.github.io/styleguide/pyguide.html)
+* [Hypothesis](https://hypothesis.readthedocs.io/en/latest/index.html) for test data generation
+* [Coverage](https://coverage.readthedocs.io/en/7.6.8/) on both the tests and code for test coverage
+* [Black](https://black.readthedocs.io/en/stable/index.html) for automated formatting
+* [isort](https://pycqa.github.io/isort/) for import sorting
+
+It's expected that there are no errors and 100% of the code is reached and executed. The strategy for test coverage is based on parsing test specifications and not unit tests.
+
+amati runs tests on external specifications, detailed in `tests/data/.amati.tests.yaml`. To be able to run these tests the appropriate GitHub repos need to be local. Specific revisions of the repos can be downloaded by running
+
+```sh
+python scripts/tests/setup_test_specs.py
+```
+
+To run everything, from linting, type checking to downloading test specs and building and testing the Docker image run:
+
+```sh
+sh bin/checks.sh
+```
+
+You will need to have Docker installed.
+
+### Building
+
+The project uses a [`pyproject.toml` file](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#writing-pyproject-toml) to determine what to build.
+
+To install, assuming that [uv](https://docs.astral.sh/uv/) is already installed and initialised
+
+```sh
+uv python install
+uv venv
+uv sync
+```
+
+### Docker
+
+A development Docker image is provided, `Dockerfile.dev`, to build:
+
+```sh
+docker build -t amati -f Dockerfile .
+```
+
+and to run against a specific specification the location of the specification needs to be mounted in the container.
+
+```sh
+docker run -v "<path-to-mount>:/<mount-name> amati <options>
+```
+
+This can be tested against a provided specification, from the root directory
+
+```sh
+docker run --detach -v "$(pwd):/data" amati
+```
+
+
+### Data
+
+There are some scripts to create the data needed by the project, for example, all the possible licences. If the data needs to be refreshed this can be done by running the contents of `/scripts/data`.
+
+
+
+

amati-0.2/TEMPLATE.html

@@ -0,0 +1,116 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>amati</title>
+    <!-- Tailwind CSS CDN -->
+    <script src="https://cdn.tailwindcss.com"></script>
+    <style>
+        body {
+            font-family: "Inter", sans-serif;
+            background-color: #f3f4f6;
+            display: flex;
+            justify-content: center;
+            align-items: flex-start;
+            min-height: 100vh;
+            padding: 2rem;
+            box-sizing: border-box;
+        }
+        .container {
+            background-color: #ffffff;
+            padding: 2rem;
+            border-radius: 1rem;
+            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
+            width: 100%;
+            max-width: 1200px; /* Increased max-width for more columns */
+            overflow-x: auto; /* Added for horizontal scrolling on small screens */
+        }
+        table {
+            width: 100%;
+            border-collapse: collapse;
+        }
+        th, td {
+            padding: 0.75rem;
+            text-align: left;
+            border-bottom: 1px solid #e5e7eb;
+        }
+        th {
+            background-color: #f9fafb;
+            font-weight: 600;
+            color: #4b5563;
+            text-transform: uppercase;
+            font-size: 0.875rem;
+        }
+        tr:hover {
+            background-color: #f3f4f6;
+        }
+        .code-block {
+            background-color: #e0e7ff;
+            padding: 0.25rem 0.5rem;
+            border-radius: 0.375rem;
+            font-family: monospace;
+            color: #4f46e5;
+            white-space: nowrap; /* Prevent line breaks for code */
+        }
+        .error-url {
+            color: #3b82f6;
+            text-decoration: none;
+        }
+        .error-url:hover {
+            text-decoration: underline;
+        }
+    </style>
+</head>
+<body>
+    <div class="container">
+        <h1 class="text-3xl font-bold text-center text-gray-800 mb-6">amati error report</h1>
+
+        {% if errors %}
+            <div class="overflow-x-auto rounded-lg shadow-sm border border-gray-200">
+                <table class="min-w-full divide-y divide-gray-200">
+                    <thead class="bg-gray-50">
+                        <tr>
+                            <th scope="col" class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider rounded-tl-lg">Type</th>
+                            <th scope="col" class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">Location</th>
+                            <th scope="col" class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">Message</th>
+                            <th scope="col" class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">Input</th>
+                            <th scope="col" class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider rounded-tr-lg">URL</th>
+                        </tr>
+                    </thead>
+                    <tbody class="bg-white divide-y divide-gray-200">
+                        {% for item in errors %}
+                            <tr>
+                                <td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">{{ item.type | default('N/A') }}</td>
+                                <td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
+                                    <code class="code-block">
+                                        {# Check if 'loc' exists and is iterable before joining #}
+                                        {% if item.loc is defined and item.loc is iterable %}
+                                            {{ item.loc | join(' -> ') }}
+                                        {% else %}
+                                            N/A
+                                        {% endif %}
+                                    </code>
+                                </td>
+                                <td class="px-6 py-4 text-sm text-gray-500">{{ item.msg | default('N/A') }}</td>
+                                <td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
+                                    <code class="code-block">
+                                        {% if item.input is defined and item.input %}
+                                            {{ item.input | default('N/A') }}
+                                        {% else %}
+                                            N/A
+                                        {% endif %}
+                                    </code>
+                                </td>
+                                <td class="px-6 py-4 whitespace-nowrap text-sm text-blue-600"><a href="{{ item.url | default('#') }}" class="error-url" target="_blank">{{ item.url | default('N/A') }}</a></td>
+                            </tr>
+                        {% endfor %}
+                    </tbody>
+                </table>
+            </div>
+        {% else %}
+            <p class="text-center text-gray-600 mt-8">No data available to display.</p>
+        {% endif %}
+    </div>
+</body>
+</html>

{amati-0.1.1 → amati-0.2}/amati/__init__.py

@@ -13,4 +13,3 @@ __version__ = importlib.metadata.version("amati")
 
 from amati.amati import dispatch, run
 from amati.exceptions import AmatiValueError
-from amati.references import AmatiReferenceException, Reference, References

amati-0.2/amati/_error_handler.py

@@ -0,0 +1,48 @@
+"""
+Handles Pydantic errors and amati logs to provide a consistent view to the user.
+"""
+
+import json
+from typing import cast
+
+from amati.logging import Log
+
+type JSONPrimitive = str | int | float | bool | None
+type JSONArray = list["JSONValue"]
+type JSONObject = dict[str, "JSONValue"]
+type JSONValue = JSONPrimitive | JSONArray | JSONObject
+
+
+def remove_duplicates(data: list[JSONObject]) -> list[JSONObject]:
+    """
+    Remove duplicates by converting each dict to a JSON string for comparison.
+    """
+    seen: set[str] = set()
+    unique_data: list[JSONObject] = []
+
+    for item in data:
+        # Convert to JSON string with sorted keys for consistent hashing
+        item_json = json.dumps(item, sort_keys=True, separators=(",", ":"))
+        if item_json not in seen:
+            seen.add(item_json)
+            unique_data.append(item)
+
+    return unique_data
+
+
+def handle_errors(errors: list[JSONObject] | None, logs: list[Log]) -> list[JSONObject]:
+    """
+    Makes errors and logs consistent for user consumption.
+    """
+
+    result: list[JSONObject] = []
+
+    if errors:
+        result.extend(errors)
+
+    if logs:
+        result.extend(cast(list[JSONObject], logs))
+
+    result = remove_duplicates(result)
+
+    return result

{amati-0.1.1 → amati-0.2}/amati/amati.py

@@ -7,15 +7,16 @@ import json
 import sys
 from pathlib import Path
 
-import jsonpickle
+from jinja2 import Environment, FileSystemLoader
 from pydantic import BaseModel, ValidationError
-from pydantic_core import ErrorDetails
 
 # pylint: disable=wrong-import-position
 
 sys.path.insert(0, str(Path(__file__).parent.parent))
+from amati._error_handler import handle_errors
 from amati._resolve_forward_references import resolve_forward_references
 from amati.file_handler import load_file
+from amati.logging import Log, LogMixin
 
 type JSONPrimitive = str | int | float | bool | None
 type JSONArray = list["JSONValue"]
@@ -23,7 +24,7 @@ type JSONObject = dict[str, "JSONValue"]
 type JSONValue = JSONPrimitive | JSONArray | JSONObject
 
 
-def dispatch(data: JSONObject) -> tuple[BaseModel | None, list[ErrorDetails] | None]:
+def dispatch(data: JSONObject) -> tuple[BaseModel | None, list[JSONObject] | None]:
     """
     Returns the correct model for the passed spec
 
@@ -59,7 +60,7 @@ def dispatch(data: JSONObject) -> tuple[BaseModel | None, list[ErrorDetails] | N
     try:
         model = module.OpenAPIObject(**data)
     except ValidationError as e:
-        return None, e.errors()
+        return None, json.loads(e.json())
 
     return model, None
 
@@ -86,7 +87,12 @@ def check(original: JSONObject, validated: BaseModel) -> bool:
     return original_ == new_
 
 
-def run(file_path: str | Path, consistency_check: bool = False):
+def run(
+    file_path: str | Path,
+    consistency_check: bool = False,
+    local: bool = False,
+    html_report: bool = False,
+):
     """
     Runs the full amati process on a specific specification file.
 
@@ -101,24 +107,56 @@ def run(file_path: str | Path, consistency_check: bool = False):
         consistency_check: Whether or not to verify the output against the input
     """
 
-    data = load_file(file_path)
+    spec = Path(file_path)
 
-    result, errors = dispatch(data)
+    data = load_file(spec)
 
-    if result and consistency_check:
-        if check(data, result):
-            print("Consistency check successful")
-        else:
-            print("Consistency check failed")
+    logs: list[Log] = []
+
+    with LogMixin.context():
+        result, errors = dispatch(data)
+        logs.extend(LogMixin.logs)
+
+    if errors or logs:
+
+        handled_errors: list[JSONObject] = handle_errors(errors, logs)
+
+        file_name = Path(Path(file_path).parts[-1])
+        error_file = file_name.with_suffix(file_name.suffix + ".errors")
+        error_path = spec.parent
 
-    if errors:
-        if not Path(".amati").exists():
-            Path(".amati").mkdir()
+        if local:
+            error_path = Path(".amati")
 
-        error_file = Path(file_path).parts[-1]
+            if not error_path.exists():
+                error_path.mkdir()
 
-        with open(f".amati/{error_file}.json", "w", encoding="utf-8") as f:
-            f.write(jsonpickle.encode(errors, unpicklable=False))  # type: ignore
+        with open(
+            error_path / error_file.with_suffix(error_file.suffix + ".json"),
+            "w",
+            encoding="utf-8",
+        ) as f:
+            f.write(json.dumps(handled_errors))
+
+        if html_report:
+            env = Environment(
+                loader=FileSystemLoader(".")
+            )  # Assumes template is in the same directory
+            template = env.get_template("TEMPLATE.html")
+
+            # Render the template with your data
+            html_output = template.render(errors=handled_errors)
+
+            # Save the output to a file
+            with open(
+                error_path / error_file.with_suffix(error_file.suffix + ".html"),
+                "w",
+                encoding="utf-8",
+            ) as f:
+                f.write(html_output)
+
+    if result and consistency_check:
+        return check(data, result)
 
 
 def discover(discover_dir: str = ".") -> list[Path]:
@@ -170,6 +208,9 @@ if __name__ == "__main__":
         --discover is set will search the directory tree. If the specification
         does not follow the naming recommendation the --spec switch should be
         used.
+        
+        Creates a file <filename>.errors.json alongside the original specification
+        containing a JSON representation of all the errors.
         """,
     )
 
@@ -185,7 +226,8 @@ if __name__ == "__main__":
         "--consistency-check",
         required=False,
         action="store_true",
-        help="Runs a consistency check between the input specification and amati",
+        help="Runs a consistency check between the input specification and the"
+        " parsed specification",
     )
 
     parser.add_argument(
@@ -196,6 +238,25 @@ if __name__ == "__main__":
         help="Searches the specified directory tree for openapi.yaml or openapi.json.",
     )
 
+    parser.add_argument(
+        "-l",
+        "--local",
+        required=False,
+        action="store_true",
+        help="Store errors local to the caller in a file called <file-name>.errors.json"
+        "; a .amati/ directory will be created.",
+    )
+
+    parser.add_argument(
+        "-hr",
+        "--html-report",
+        required=False,
+        action="store_true",
+        help="Creates an HTML report of the errors, called <file-name>.errors.html,"
+        " alongside the original file or in a .amati/ directory if the --local switch"
+        " is used",
+    )
+
     args = parser.parse_args()
 
     if args.spec:
@@ -204,4 +265,9 @@ if __name__ == "__main__":
         specifications = discover(args.discover)
 
     for specification in specifications:
-        run(specification, args.consistency_check)
+        if successful_check := run(
+            specification, args.consistency_check, args.local, args.html_report
+        ):
+            print("Consistency check successful for {specification}")
+        else:
+            print("Consistency check failed for {specification}")

{amati-0.1.1 → amati-0.2}/amati/exceptions.py

@@ -4,8 +4,6 @@ Exceptions, declared here to not put in __init__
 
 from typing import Optional
 
-from amati.references import References
-
 
 class AmatiValueError(ValueError):
     """
@@ -21,6 +19,6 @@ class AmatiValueError(ValueError):
         ValueError
     """
 
-    def __init__(self, message: str, reference: Optional[References] = None):
+    def __init__(self, message: str, reference_uri: Optional[str] = None):
         self.message = message
-        self.reference = reference
+        self.reference_uri = reference_uri

{amati-0.1.1 → amati-0.2}/amati/fields/email.py

@@ -5,14 +5,10 @@ Validates an email according to the RFC5322 ABNF grammar - §3:
 from abnf import ParseError
 from abnf.grammars import rfc5322
 
-from amati import AmatiValueError, Reference
+from amati import AmatiValueError
 from amati.fields import Str as _Str
 
-reference = Reference(
-    title="Internet Message Format",
-    url="https://www.rfc-editor.org/rfc/rfc5322#section-3",
-    section="Syntax",
-)
+reference_uri = "https://www.rfc-editor.org/rfc/rfc5322#section-3"
 
 
 class Email(_Str):
@@ -23,5 +19,5 @@ class Email(_Str):
             rfc5322.Rule("address").parse_all(value)
         except ParseError as e:
             raise AmatiValueError(
-                message=f"{value} is not a valid email address", reference=reference
+                f"{value} is not a valid email address", reference_uri
             ) from e