asyncapi-python 0.2.2__tar.gz → 0.2.4__tar.gz

{asyncapi_python-0.2.2 → asyncapi_python-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: asyncapi-python
-Version: 0.2.2
+Version: 0.2.4
 Summary: Easily generate type-safe and async Python applications from AsyncAPI 3 specifications.
 License: Apache-2.0
 Author: Yaroslav Petrov
@@ -52,6 +52,7 @@ Easily generate type-safe and async Python applications from AsyncAPI 3 specific
 - [x] Supports publish-subscribe pattern
 - [ ] AsyncAPI trait support
 - [ ] Customizable message encoder/decoder
+- [x] Works as a plugin for [pantsbuild](https://pantsbuild.org) (see [instructions](#usage-as-a-pants-plugin) below)
 
 ## Requirements
 
@@ -82,6 +83,102 @@ pip install asyncapi-python[amqp]
 
 You can replace `amqp` with any other supported protocols. For more info, see [Supported Protocols](#supported-protocols--use-cases) section.
 
+### Usage as a Pants plugin
+
+> The following method was tested with pants version 2.23.1.
+> Pleas note that Pants plugin API is still in development and things might break.
+
+This library can act as a plugin for [Pants](https://pantsbuild.org). More specifically, it creates a new target type: `asyncapi_python_service` -- which can be used like:
+
+```python
+# BUILD
+asyncapi_python_service(
+  name="asyncapi_app",
+  service="app.asyncapi.yaml",
+  sources=[
+    "app.asyncapi.yaml", 
+    "lib.asyncapi.yaml", 
+    "commons.*.asyncapi.yaml"
+  ]
+)
+```
+
+This will be generating python module named `asyncapi_app` on `codegen-export` and `export` goals.
+This target can later be used as a dependency of `python_sources`.
+
+```python
+# BUILD
+python_sources(
+  dependencies=[
+    ":asyncapi_app",
+    ":reqs",
+  ],
+)
+
+python_requirements(
+  name="reqs",
+)
+```
+
+Note that this plugin does not do dependency injection, so asyncapi-python must be a dependency
+
+```text
+# requirements.txt
+asyncapi-python[amqp]
+```
+
+### Deploying this plugin into pants monorepo
+
+In order to deploy this plugin into your pants monorepo, create the following structure inside your plugins folder:
+
+```bash
+pants-plugins/
+└── asyncapi_python_plugin
+    ├── BUILD
+    ├── __init__.py
+    ├── register.py
+    └── requirements.txt
+```
+
+`requirements.txt` must contain:
+
+```text
+asyncapi-python
+```
+
+`register.py` should have:
+
+```python
+from asyncapi_python_pants.register import *
+```
+
+`BUILD` must include:
+
+```python
+python_sources(
+    dependencies=[":reqs"],
+)
+
+python_requirements(
+    name="reqs",
+)
+```
+
+`__init__.py` can be empty, but it has to exist.
+
+Finally, add `pants-plugins` to your `PYTHONPATH`, and add the created folder as a backend package:
+
+```toml
+# pants.toml
+backend_packages = [
+    "asyncapi_python_plugin",
+    ...
+]
+pythonpath = ["%(buildroot)s/pants-plugins"]
+```
+
+When everything is done, test if the plugin works by running `pants export-codegen ::`
+
 ## Supported Protocols / Use Cases
 
 Below, you may see the table of protocols and the supported use cases. The tick signs (✅) contain links to the examples for each implemented protocol-use case pair, while the hammer signs (🔨) contain links to the Issues tracking the progress for protocol-use case pairs. The list of protocols and use cases is expected to increase over the progress of development.

{asyncapi_python-0.2.2 → asyncapi_python-0.2.4}/README.md

@@ -25,6 +25,7 @@ Easily generate type-safe and async Python applications from AsyncAPI 3 specific
 - [x] Supports publish-subscribe pattern
 - [ ] AsyncAPI trait support
 - [ ] Customizable message encoder/decoder
+- [x] Works as a plugin for [pantsbuild](https://pantsbuild.org) (see [instructions](#usage-as-a-pants-plugin) below)
 
 ## Requirements
 
@@ -55,6 +56,102 @@ pip install asyncapi-python[amqp]
 
 You can replace `amqp` with any other supported protocols. For more info, see [Supported Protocols](#supported-protocols--use-cases) section.
 
+### Usage as a Pants plugin
+
+> The following method was tested with pants version 2.23.1.
+> Pleas note that Pants plugin API is still in development and things might break.
+
+This library can act as a plugin for [Pants](https://pantsbuild.org). More specifically, it creates a new target type: `asyncapi_python_service` -- which can be used like:
+
+```python
+# BUILD
+asyncapi_python_service(
+  name="asyncapi_app",
+  service="app.asyncapi.yaml",
+  sources=[
+    "app.asyncapi.yaml", 
+    "lib.asyncapi.yaml", 
+    "commons.*.asyncapi.yaml"
+  ]
+)
+```
+
+This will be generating python module named `asyncapi_app` on `codegen-export` and `export` goals.
+This target can later be used as a dependency of `python_sources`.
+
+```python
+# BUILD
+python_sources(
+  dependencies=[
+    ":asyncapi_app",
+    ":reqs",
+  ],
+)
+
+python_requirements(
+  name="reqs",
+)
+```
+
+Note that this plugin does not do dependency injection, so asyncapi-python must be a dependency
+
+```text
+# requirements.txt
+asyncapi-python[amqp]
+```
+
+### Deploying this plugin into pants monorepo
+
+In order to deploy this plugin into your pants monorepo, create the following structure inside your plugins folder:
+
+```bash
+pants-plugins/
+└── asyncapi_python_plugin
+    ├── BUILD
+    ├── __init__.py
+    ├── register.py
+    └── requirements.txt
+```
+
+`requirements.txt` must contain:
+
+```text
+asyncapi-python
+```
+
+`register.py` should have:
+
+```python
+from asyncapi_python_pants.register import *
+```
+
+`BUILD` must include:
+
+```python
+python_sources(
+    dependencies=[":reqs"],
+)
+
+python_requirements(
+    name="reqs",
+)
+```
+
+`__init__.py` can be empty, but it has to exist.
+
+Finally, add `pants-plugins` to your `PYTHONPATH`, and add the created folder as a backend package:
+
+```toml
+# pants.toml
+backend_packages = [
+    "asyncapi_python_plugin",
+    ...
+]
+pythonpath = ["%(buildroot)s/pants-plugins"]
+```
+
+When everything is done, test if the plugin works by running `pants export-codegen ::`
+
 ## Supported Protocols / Use Cases
 
 Below, you may see the table of protocols and the supported use cases. The tick signs (✅) contain links to the examples for each implemented protocol-use case pair, while the hammer signs (🔨) contain links to the Issues tracking the progress for protocol-use case pairs. The list of protocols and use cases is expected to increase over the progress of development.

{asyncapi_python-0.2.2 → asyncapi_python-0.2.4}/pyproject.toml

@@ -1,11 +1,12 @@
 [tool.poetry]
 name = "asyncapi-python"
-version = "0.2.2"
+version = "0.2.4"
 license = "Apache-2.0"
 description = "Easily generate type-safe and async Python applications from AsyncAPI 3 specifications."
 authors = ["Yaroslav Petrov <yaroslav.v.petrov@gmail.com>"]
 readme = "README.md"
 packages = [
+    { include = "asyncapi_python_pants", from = "src" },
     { include = "asyncapi_python_codegen", from = "src" },
     { include = "asyncapi_python", from = "src" },
 ]

{asyncapi_python-0.2.2 → asyncapi_python-0.2.4}/src/asyncapi_python_codegen/document/utils.py

@@ -22,10 +22,10 @@ from collections import defaultdict
 
 
 Reference = Union[None, tuple[Path, tuple[str, ...]]]
-"""A reference type"""
+"""A reference type, maps a document path to a set of references that point to it"""
 
 ReferenceCounter = defaultdict[Reference, set[Reference]]
-"""A reference counter"""
+"""A reference counter, maps each reference to a set of references that point to it"""
 
 
 def populate_jsonschema_defs(schema: Any) -> Any:
@@ -45,10 +45,17 @@ def populate_jsonschema_defs(schema: Any) -> Any:
 
 def _count_references(schema: Any, this: Reference, counter: ReferenceCounter):
     """Recursively constructs back references within the JsonSchema"""
+
+    # List case   
+    if isinstance(schema, list):
+        for v in schema:
+            _count_references(v, this, counter)
+
+    # Dict case
     if not isinstance(schema, dict):
         return
 
-    if "$ref" in schema:
+    if "$ref" in schema: # If dict is $ref object
         ref: Ref[Any] = Ref.model_validate(schema)
         with set_current_doc_path(ref.filepath):
             ref = ref.flatten()
@@ -61,7 +68,7 @@ def _count_references(schema: Any, this: Reference, counter: ReferenceCounter):
         with set_current_doc_path(ref.filepath):
             return _count_references(doc, child, counter)
 
-    for v in schema.values():
+    for v in schema.values(): # Recur
         _count_references(v, this, counter)
 
 
@@ -72,6 +79,12 @@ def _populate_jsonschema_recur(
     ignore_shared: bool = False,
 ) -> Any:
     """Recursively populates JsonSchema $defs object"""
+
+    # List case
+    if isinstance(schema, list):
+        return [_populate_jsonschema_recur(v, counter, shared_schemas, ignore_shared) for v in schema]
+
+    # Dict case
     if not isinstance(schema, dict):
         return schema
 

asyncapi_python-0.2.4/src/asyncapi_python_pants/register.py

@@ -0,0 +1,19 @@
+from pants.engine.rules import collect_rules
+from pants.engine.unions import UnionRule
+from pants.backend.python.util_rules import pex
+from pants.core.goals.resolves import ExportableTool
+
+from .rules import *
+from .targets import *
+
+
+def rules():
+    return [
+        *collect_rules(),
+        *pex.rules(),
+        UnionRule(GenerateSourcesRequest, GeneratePythonFromAsyncapiRequest),
+    ]
+
+
+def target_types():
+    return [AsyncapiServiceTarget]

asyncapi_python-0.2.4/src/asyncapi_python_pants/rules.py

@@ -0,0 +1,93 @@
+from importlib.metadata import version
+from pants.engine.internals.native_engine import (
+    Digest,
+    MergeDigests,
+    RemovePrefix,
+    AddPrefix,
+    Snapshot,
+)
+from pants.core.util_rules.stripped_source_files import StrippedSourceFiles
+from pants.core.util_rules.source_files import SourceFilesRequest
+from pants.engine.target import (
+    GeneratedSources,
+    TransitiveTargets,
+    TransitiveTargetsRequest,
+)
+from pants.engine.rules import rule, Get, MultiGet
+from pants.engine.process import ProcessResult
+from pants.source.source_root import SourceRoot, SourceRootRequest
+from pants.backend.python.target_types import ConsoleScript
+from pants.backend.python.util_rules.interpreter_constraints import (
+    InterpreterConstraints,
+)
+from pants.backend.python.util_rules.pex import (
+    Pex,
+    PexProcess,
+    PexRequest,
+    PexRequirements,
+)
+from .targets import *
+
+
+@rule
+async def generate_python_from_asyncapi(
+    request: GeneratePythonFromAsyncapiRequest,
+) -> GeneratedSources:
+    pex = await Get(
+        Pex,
+        PexRequest(
+            output_filename="asyncapi-python-codegen.pex",
+            internal_only=True,
+            requirements=PexRequirements(
+                [f"asyncapi-python[codegen]=={version('asyncapi-python')}"]
+            ),
+            interpreter_constraints=InterpreterConstraints([">=3.9"]),
+            main=ConsoleScript("asyncapi-python-codegen"),
+        ),
+    )
+    transitive_targets = await Get(
+        TransitiveTargets,
+        TransitiveTargetsRequest([request.protocol_target.address]),
+    )
+    all_sources_stripped = await Get(
+        StrippedSourceFiles,
+        SourceFilesRequest(
+            (tgt.get(AsyncapiSourcesField) for tgt in transitive_targets.closure),
+            for_sources_types=(AsyncapiSourcesField,),
+        ),
+    )
+    input_digest = await Get(
+        Digest,
+        MergeDigests(
+            (
+                all_sources_stripped.snapshot.digest,
+                pex.digest,
+            )
+        ),
+    )
+    output_dir = "_generated_files"
+    module_name = request.protocol_target.address.target_name
+    result = await Get(
+        ProcessResult,
+        PexProcess(
+            pex,
+            argv=[
+                request.protocol_target[AsyncapiServiceField].value or "",
+                f"{output_dir}/{module_name}",
+            ],
+            description=f"Generating Python sources from {request.protocol_target.address}.",
+            output_directories=(output_dir,),
+            input_digest=input_digest,
+        ),
+    )
+    source_root_request = SourceRootRequest.for_target(request.protocol_target)
+    normalized_digest, source_root = await MultiGet(
+        Get(Digest, RemovePrefix(result.output_digest, output_dir)),
+        Get(SourceRoot, SourceRootRequest, source_root_request),
+    )
+    source_root_restored = (
+        await Get(Snapshot, AddPrefix(normalized_digest, source_root.path))
+        if source_root.path != "."
+        else await Get(Snapshot, Digest, normalized_digest)
+    )
+    return GeneratedSources(source_root_restored)

asyncapi_python-0.2.4/src/asyncapi_python_pants/targets.py

@@ -0,0 +1,53 @@
+from pants.engine.target import GenerateSourcesRequest
+from pants.engine.target import (
+    COMMON_TARGET_FIELDS,
+    Dependencies,
+    Target,
+    AsyncFieldMixin,
+    MultipleSourcesField,
+    StringField,
+)
+from pants.backend.python.target_types import PythonSourceField
+from pants.backend.python.target_types import (
+    InterpreterConstraintsField,
+    PythonResolveField,
+)
+
+
+class AsyncapiPythonInterpreterConstraints(InterpreterConstraintsField): ...
+
+
+class AsyncapiPythonResolveField(PythonResolveField): ...
+
+
+class AsyncapiServiceField(StringField):
+    alias = "service"
+
+
+class AsyncapiSourcesField(MultipleSourcesField, AsyncFieldMixin):
+    expected_file_extensions = (".yaml", ".asyncapi.yaml")
+
+
+class AsyncapiDependencies(Dependencies): ...
+
+
+class AsyncapiServiceTarget(Target):
+    alias = "asyncapi_python_service"
+    help = "A single AsyncAPI file."
+    core_fields = (
+        *COMMON_TARGET_FIELDS,
+        AsyncapiDependencies,
+        AsyncapiSourcesField,
+        AsyncapiServiceField,
+        AsyncapiPythonInterpreterConstraints,
+        AsyncapiPythonResolveField,
+    )
+
+
+class InjectAsyncapiDependencies(Target):
+    inject_for = AsyncapiDependencies
+
+
+class GeneratePythonFromAsyncapiRequest(GenerateSourcesRequest):
+    input = AsyncapiSourcesField
+    output = PythonSourceField