pyjelly 0.6.1__tar.gz → 0.6.2__tar.gz

{pyjelly-0.6.1 → pyjelly-0.6.2}/.github/workflows/ci.yml

@@ -35,7 +35,7 @@ jobs:
       matrix:
         conformance-tests: [false]
         version: ['3.9', '3.13']
-        platform: [ubuntu-latest, macos-latest]
+        platform: [ubuntu-latest, macos-latest, windows-latest]
         include:
         - version: '3.10'
           platform: ubuntu-latest
@@ -59,6 +59,16 @@ jobs:
       with:
         submodules: true
 
+    - name: Windows walkaround for broken symlinks
+      if: runner.os == 'Windows'
+      run: |
+        rm -r pyjelly/_proto
+        rm -r tests/conformance_tests/rdf
+        rm -r tests/conformance_tests/grpc
+        cp -r submodules/protobuf/proto pyjelly/_proto
+        cp -r submodules/protobuf/test/rdf tests/conformance_tests/rdf
+        cp -r submodules/protobuf/test/grpc tests/conformance_tests/grpc
+
     - uses: astral-sh/setup-uv@v6
       with:
         enable-cache: true

{pyjelly-0.6.1 → pyjelly-0.6.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyjelly
-Version: 0.6.1
+Version: 0.6.2
 Summary: Jelly-RDF implementation for Python
 Project-URL: Homepage, https://w3id.org/jelly/pyjelly
 Project-URL: Documentation, https://w3id.org/jelly/pyjelly
@@ -44,9 +44,22 @@ Description-Content-Type: text/markdown
 
 ## Features
 
-- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly)
-- **Seamless integration with [rdflib](https://rdflib.readthedocs.io/)**
-- **Stream processing support** for large datasets or streams of many RDF graphs/datasets
+- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly).
+- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** with no third-party dependencies, allowing for:
+    - Serialization and parsing of statements to and from Jelly files.
+    - Parsing and serializing streams of graphs and statements.
+- Precise control over **serialization options, framing and compression**.
+- **Seamless** integration with: 
+    - **[rdflib](https://w3id.org/jelly/pyjelly/dev/getting-started)**
+    - **[RDFLib-Neo4j](https://w3id.org/jelly/pyjelly/dev/rdflib-neo4j-integration)**
+    - **[NetworkX](https://w3id.org/jelly/pyjelly/dev/networkx-integration)**
+- **Stream processing support** for large datasets or streams of all [physical stream types](https://w3id.org/jelly/dev/specification/reference/#physicalstreamtype).
+
+**pyjelly** is useful when dealing with (see [full description](https://w3id.org/jelly/pyjelly/dev/overview/#use-cases)):
+
+- Dumping and loading **large RDF datasets**.
+- **Client-server communication**.
+- Workflows, where **streaming** is required.
 
 ## Getting started
 

{pyjelly-0.6.1 → pyjelly-0.6.2}/README.md

@@ -8,9 +8,22 @@
 
 ## Features
 
-- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly)
-- **Seamless integration with [rdflib](https://rdflib.readthedocs.io/)**
-- **Stream processing support** for large datasets or streams of many RDF graphs/datasets
+- **Fast reading and writing** of RDF knowledge graphs in the [Jelly format](http://w3id.org/jelly).
+- **Standalone [generic API](https://w3id.org/jelly/pyjelly/dev/generic-sink)** with no third-party dependencies, allowing for:
+    - Serialization and parsing of statements to and from Jelly files.
+    - Parsing and serializing streams of graphs and statements.
+- Precise control over **serialization options, framing and compression**.
+- **Seamless** integration with: 
+    - **[rdflib](https://w3id.org/jelly/pyjelly/dev/getting-started)**
+    - **[RDFLib-Neo4j](https://w3id.org/jelly/pyjelly/dev/rdflib-neo4j-integration)**
+    - **[NetworkX](https://w3id.org/jelly/pyjelly/dev/networkx-integration)**
+- **Stream processing support** for large datasets or streams of all [physical stream types](https://w3id.org/jelly/dev/specification/reference/#physicalstreamtype).
+
+**pyjelly** is useful when dealing with (see [full description](https://w3id.org/jelly/pyjelly/dev/overview/#use-cases)):
+
+- Dumping and loading **large RDF datasets**.
+- **Client-server communication**.
+- Workflows, where **streaming** is required.
 
 ## Getting started
 

{pyjelly-0.6.1 → pyjelly-0.6.2}/docs/contributing/index.md

@@ -4,7 +4,9 @@ Hi! This guide explains how to get started with developing pyjelly and contribut
 
 ## Quick start
 
-Clone the project: 
+**System requirements:** while pyjelly works on any system that supports Python, **for development** we recommend using Linux, macOS, or WSL on Windows. We advise against using Windows directly, as it has caused many headaches in the past.
+
+**Clone the project:** 
 
 ```shell
 git clone --recursive git@github.com:Jelly-RDF/pyjelly.git

{pyjelly-0.6.1 → pyjelly-0.6.2}/pyjelly/parse/lookup.py

@@ -26,7 +26,7 @@ class LookupDecoder:
 
     def __init__(self, *, lookup_size: int) -> None:
         if lookup_size > MAX_LOOKUP_SIZE:
-            msg = f"lookup size must be less than {MAX_LOOKUP_SIZE}"
+            msg = f"lookup size cannot be larger than {MAX_LOOKUP_SIZE}"
             raise JellyAssertionError(msg)
         self.lookup_size = lookup_size
         placeholders = (None,) * lookup_size

{pyjelly-0.6.1 → pyjelly-0.6.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pyjelly"
-version = "0.6.1"
+version = "0.6.2"
 readme = "README.md"
 description = "Jelly-RDF implementation for Python"
 authors = [

{pyjelly-0.6.1 → pyjelly-0.6.2}/submodules/protobuf/test/README.md

@@ -3,4 +3,4 @@
 Machine-readable test manifests are in the `manifest.ttl` file of each subdirectory. The manifests follow the vocabulary in the `vocabulary.ttl` file.
 
 - `rdf` – tests for the [Jelly RDF serialization format specification](https://w3id.org/jelly/dev/specification/serialization/) (Jelly-RDF).
-- `grpc` – tests for the [Jelly gRPC streaming protocol specification](https://w3id.org/jelly/dev/specification/protocol/) (Jelly-RDF gRPC).
+- `grpc` – tests for the [Jelly gRPC streaming protocol specification](https://w3id.org/jelly/dev/specification/streaming/) (Jelly-RDF gRPC).

{pyjelly-0.6.1 → pyjelly-0.6.2}/submodules/protobuf/test/rdf/from_jelly/manifest.ttl

@@ -891,22 +891,19 @@ BASE           <https://w3id.org/jelly/dev/tests/rdf/from_jelly/>
     mf:name "Single frame, s_triple_term is missing a p_literal on the first nesting level, p_literal on the deeper levels repeats. Repeated terms are prohibited in quoted triples." ;
     rdft:approval rdft:Proposed ;
     mf:requires jellyt:requirementPhysicalTypeQuads, jellyt:requirementRdfStar, jellyt:requirementGeneralizedRdf ;
-    mf:action <quads_rdf_star_generalized/neg_001/in.jelly> ;
-    mf:result <quads_rdf_star_generalized/neg_001/out_000.nq> .
+    mf:action <quads_rdf_star_generalized/neg_001/in.jelly> .
 
 <quads_rdf_star_generalized/neg_002> a jellyt:TestNegative, jellyt:TestRdfFromJelly ;
     mf:name "Single frame, a quad row only has an empty s_triple_term, quad before has three (3) proper s_triple_terms. Repeated terms are prohibited in quoted triples." ;
     rdft:approval rdft:Proposed ;
     mf:requires jellyt:requirementPhysicalTypeQuads, jellyt:requirementRdfStar, jellyt:requirementGeneralizedRdf ;
-    mf:action <quads_rdf_star_generalized/neg_002/in.jelly> ;
-    mf:result <quads_rdf_star_generalized/neg_002/out_000.nq> .
+    mf:action <quads_rdf_star_generalized/neg_002/in.jelly> .
 
 <quads_rdf_star_generalized/neg_003> a jellyt:TestNegative, jellyt:TestRdfFromJelly ;
     mf:name "Single frame, o_triple_term is missing p_literal on the second nesting level, the same p_literal is used before and after in nested triples and in the quad row before. Repeated terms are prohibited in quoted triples." ;
     rdft:approval rdft:Proposed ;
     mf:requires jellyt:requirementPhysicalTypeQuads, jellyt:requirementRdfStar, jellyt:requirementGeneralizedRdf ;
-    mf:action <quads_rdf_star_generalized/neg_003/in.jelly> ;
-    mf:result <quads_rdf_star_generalized/neg_003/out_000.nq> .
+    mf:action <quads_rdf_star_generalized/neg_003/in.jelly> .
 
 <quads_rdf_star_generalized/pos_001> a jellyt:TestPositive, jellyt:TestRdfFromJelly ;
     mf:name "Single frame. A variety of generalized RDF-star quads with graph labels being BN/IRI/Literal/Default graph. Prefix table disabled." ;

pyjelly-0.6.2/tests/conformance_tests/generate_conformance_report.py

@@ -0,0 +1,330 @@
+from __future__ import annotations
+
+import datetime as dt
+import os
+import re
+import shutil
+import subprocess
+from collections import defaultdict
+from dataclasses import dataclass
+from pathlib import Path
+from typing import cast
+
+import pytest
+from _pytest.nodes import Item
+from _pytest.reports import TestReport
+from rdflib import BNode, Graph, Literal, Namespace, URIRef
+from rdflib.namespace import DCTERMS, RDF, XSD
+
+EARL = Namespace("http://www.w3.org/ns/earl#")
+DOAP = Namespace("http://usefulinc.com/ns/doap#")
+FOAF = Namespace("http://xmlns.com/foaf/0.1/")
+
+ASSERTOR = URIRef("#assertor")
+IMPL = URIRef("#impl")
+DEVELOPER = URIRef("#developer")
+_outcome_map = {
+    "passed": EARL.passed,
+    "failed": EARL.failed,
+    "inapplicable": EARL.inapplicable,
+}
+
+
+@dataclass(frozen=True)
+class Result:
+    """Single test result."""
+
+    name: str
+    outcome: str
+
+
+def get_release_version_date() -> tuple[str, str]:
+    """
+    Get version and date of the current pyjelly release.
+
+    Returns:
+        tag (str): current release version
+        date (str): current release date
+
+    """
+    git = shutil.which("git")
+    if git is None:
+        raise RuntimeError
+    cp = subprocess.run(  # noqa: S603
+        [git, "describe", "--tags", "--abbrev=0"],
+        text=True,
+        check=True,
+        capture_output=True,
+    )
+    tag = cp.stdout.strip()
+    if not re.match(r"^[A-Za-z0-9._-]+$", tag):
+        raise ValueError
+    date = subprocess.check_output(  # noqa: S603
+        [git, "log", "-1", "--format=%ai", tag], text=True
+    ).strip()
+    return tag.strip("v"), date.split(" ")[0]
+
+
+class ConformanceReportPlugin:
+    """
+    Plugin class used during pytest execution for conformance test reporting.
+
+    Pytest Hooks:
+        pytest_runtest_makereport: store results data from each test
+        pytest_sessionfinish: build RDF graph with all test results
+    """
+
+    def __init__(
+        self,
+        path_to_rdflib_report: str,
+        path_to_generic_report: str,
+    ) -> None:
+        """
+        Initialize ConformanceReportPlugin.
+
+        Args:
+            path_to_rdflib_report (str): report path for RDFLib integration
+            path_to_generic_report (str): report path for pyjelly with Generic API
+
+        """
+        self.path_to_rdflib_report = path_to_rdflib_report
+        self.path_to_generic_report = path_to_generic_report
+        self._results: defaultdict[str, list[Result]] = defaultdict(list)
+        self._case_by_id: dict[str, object | None] = defaultdict(list)
+
+    @staticmethod
+    def _outcome_from_report(report: pytest.TestReport) -> str:
+        """
+        Map the report.outcome to string.
+
+        Static method.
+        """
+        if report.outcome == "skipped":
+            return "inapplicable"
+        return report.outcome
+
+    @staticmethod
+    def _initialize_graph(implementation: str) -> Graph:
+        """
+        Initialize the graph, depending on implementation.
+
+        Args:
+            implementation (str): implementation name
+
+        Returns:
+            Graph: the initialized graph
+
+        """
+        pyjelly_v, pyjelly_d = get_release_version_date()
+        g = Graph()
+        g.bind("earl", EARL)
+        g.bind("doap", DOAP)
+        g.bind("foaf", FOAF)
+        g.bind("dc", DCTERMS)
+        g.bind("xsd", XSD)
+
+        # add primaryTopic
+        g.add((URIRef(""), FOAF.primaryTopic, URIRef("#impl")))
+        g.add(
+            (
+                URIRef(""),
+                DCTERMS.issued,
+                Literal(
+                    dt.datetime.now(dt.timezone.utc).isoformat(), datatype=XSD.dateTime
+                ),
+            )
+        )
+        g.add((URIRef(""), FOAF.maker, URIRef("#assertor")))
+
+        # add developer
+
+        # add assertor
+        g.add((ASSERTOR, RDF.type, EARL.Software))
+        g.add((ASSERTOR, RDF.type, EARL.Assertor))
+        g.add((ASSERTOR, FOAF.name, Literal("pyjelly test suite")))
+        g.add(
+            (
+                ASSERTOR,
+                FOAF.homepage,
+                URIRef(
+                    "https://github.com/Jelly-RDF/pyjelly/tree/main/tests/conformance_tests"
+                ),
+            )
+        )
+
+        g.add((DEVELOPER, RDF.type, FOAF.Group))
+        g.add((DEVELOPER, FOAF.name, Literal("pyjelly contributors")))
+        g.add((DEVELOPER, FOAF.homepage, URIRef("https://w3id.org/jelly/pyjelly")))
+
+        if implementation.lower() == "generic":
+            description = "pyjelly (Generic API)"
+            implementation = "pyjelly (Generic API)"
+        elif implementation.lower() == "rdflib":
+            description = "pyjelly integration with RDFLib"
+            implementation = "pyjelly (RDFLib)"
+        else:
+            description = "implementation not found"
+        # add implementation
+        g.add((IMPL, RDF.type, DOAP.Project))
+        g.add((IMPL, RDF.type, DOAP.TestSubject))
+        g.add((IMPL, RDF.type, DOAP.Software))
+        g.add((IMPL, DOAP.name, Literal(implementation)))
+        g.add((IMPL, DOAP.developer, DEVELOPER))
+        g.add((IMPL, DOAP.homepage, URIRef("https://w3id.org/jelly/pyjelly")))
+        g.add((IMPL, DOAP.description, Literal(description, lang="en")))
+        g.add((IMPL, DOAP["programming-language"], Literal("Python")))
+        release = BNode()
+        g.add((IMPL, DOAP.release, release))
+        g.add((release, DOAP.name, Literal("pyjelly")))
+        g.add((release, DOAP.revision, Literal(pyjelly_v)))
+        g.add((release, DCTERMS.created, Literal(pyjelly_d, datatype=XSD.date)))
+        return g
+
+    @staticmethod
+    def _add_test_record_to_graph(
+        g: Graph, outcome: str, test_uri: str, assertor: URIRef, impl: URIRef
+    ) -> Graph:
+        """
+        Add one test record to a graph.
+
+        Args:
+            g (Graph): Graph to add test record to
+            outcome (str): outcome of test
+            test_uri (str): URI of test record
+            assertor (URIRef): URIRef of assertor
+            impl (URIRef): URIRef of implementation
+
+        Returns:
+            Graph: Graph with added test record
+
+        Static method.
+
+        """
+        test_name = "#" + "_".join(test_uri.split("/")[6:])
+        g.add((URIRef(test_name), RDF.type, EARL.Assertion))
+        g.add((URIRef(test_name), EARL.assertedBy, assertor))
+        g.add((URIRef(test_name), EARL.subject, impl))
+        g.add(
+            (
+                URIRef(test_name),
+                EARL.test,
+                URIRef(
+                    f"https://github.com/Jelly-RDF/jelly-protobuf/tree/main/test/{'/'.join(test_uri.split('/')[6:])}"
+                ),
+            )
+        )
+        g.add((URIRef(test_name), EARL.mode, EARL.automatic))
+        result = BNode()
+        g.add((URIRef(test_name), EARL.result, result))
+        g.add((result, RDF.type, EARL.TestResult))
+        g.add(
+            (
+                result,
+                DCTERMS.date,
+                Literal(
+                    dt.datetime.now(dt.timezone.utc).isoformat(), datatype=XSD.dateTime
+                ),
+            )
+        )
+        g.add((result, EARL.outcome, _outcome_map[outcome]))
+        return g
+
+    def pytest_runtest_call(self, item: Item) -> None:
+        case = None
+        if hasattr(item, "callspec"):
+            case = item.callspec.params.get("case")
+        self._case_by_id[item.nodeid] = case
+
+    def pytest_runtest_logreport(self, report: TestReport) -> None:
+        """
+        Pytest hook, called after each test.
+
+        Args:
+            report (TestReport): single test report
+
+        """
+        if report.when != "call":
+            return
+        node_id = report.nodeid
+        case = self._case_by_id.get(node_id)
+        outcome = self._outcome_from_report(report)
+        test_name = report.nodeid.rsplit("::", 1)[-1]
+        uri = getattr(case, "uri", None)
+        self._results[cast(str, uri)].append(
+            Result(
+                name=test_name,
+                outcome=outcome,
+            )
+        )
+
+    def pytest_sessionfinish(self) -> None:
+        """
+        Pytest hook, called once the session finishes.
+
+        Builds RDF Graph using RDFLib and serializes to Turtle.
+        """
+        g_generic = self._initialize_graph(implementation="generic")
+        g_rdflib = self._initialize_graph(implementation="RDFLib")
+
+        for k, v in self._results.items():
+            if len(v) == 1:
+                r = v[0]
+                if (
+                    "test_generic" in r.name
+                    or "test_serializes_generic" in r.name
+                    or "test_serializing_fails_generic" in r.name
+                ):
+                    g_generic = self._add_test_record_to_graph(
+                        g_generic, r.outcome, k, ASSERTOR, IMPL
+                    )
+                    g_rdflib = self._add_test_record_to_graph(
+                        g_rdflib, "inapplicable", k, ASSERTOR, IMPL
+                    )
+                elif (
+                    "test_rdflib" in r.name
+                    or "test_serializes_rdf" in r.name
+                    or "test_serializing_fails_rdf" in r.name
+                ):
+                    g_rdflib = self._add_test_record_to_graph(
+                        g_rdflib, r.outcome, k, ASSERTOR, IMPL
+                    )
+                    g_generic = self._add_test_record_to_graph(
+                        g_generic, "inapplicable", k, ASSERTOR, IMPL
+                    )
+
+            if len(v) > 1:
+                for r in v:
+                    if (
+                        "test_generic" in r.name
+                        or "test_serializes_generic" in r.name
+                        or "test_serializing_fails_generic" in r.name
+                    ):
+                        g_generic = self._add_test_record_to_graph(
+                            g_generic, r.outcome, k, ASSERTOR, IMPL
+                        )
+                    elif (
+                        "test_rdflib" in r.name
+                        or "test_serializes_rdf" in r.name
+                        or "test_serializing_fails_rdf" in r.name
+                    ):
+                        g_rdflib = self._add_test_record_to_graph(
+                            g_rdflib, r.outcome, k, ASSERTOR, IMPL
+                        )
+
+        g_rdflib.serialize(self.path_to_rdflib_report, format="turtle")
+        g_generic.serialize(self.path_to_generic_report, format="turtle")
+
+
+def main() -> int:
+    os.environ["REPORTING_MODE"] = "1"
+    tests_dir = str(Path(__file__).resolve().parent)
+    conformance_report_plugin = ConformanceReportPlugin(
+        path_to_rdflib_report="pyjelly RDFLib.ttl",
+        path_to_generic_report="pyjelly Generic API.ttl",
+    )
+
+    return pytest.main(args=[tests_dir], plugins=[conformance_report_plugin])
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

{pyjelly-0.6.1 → pyjelly-0.6.2}/tests/conformance_tests/test_rdf/test_serialize.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import os
 from dataclasses import dataclass, field
 from pathlib import Path
 from unittest.mock import patch
@@ -22,6 +23,7 @@ from tests.utils.rdf_test_cases import jelly_validate, needs_jelly_cli
 
 # Manifest path for to_jelly tests. This is where all test cases are defined.
 TO_JELLY_MANIFEST = RDF_TO_JELLY_MANIFEST
+REPORTING_MODE = os.getenv("REPORTING_MODE", "0") == "1"
 
 
 @dataclass
@@ -174,12 +176,13 @@ workaround_rdflib_serializes_default_graph_id.start()
 # Collect and categorize test cases for pytest parameterization
 ALL_TO_JELLY_CASES = load_to_jelly_manifest_cases(TO_JELLY_MANIFEST)
 
-# Temporary exclusion of a problematic case, can be replaced with xfail later
-ALL_TO_JELLY_CASES = [
-    case
-    for case in ALL_TO_JELLY_CASES
-    if not ("pos_014" in case.uri and case.category == "rdf11")
-]
+if not REPORTING_MODE:
+    # Temporary exclusion of a problematic case, can be replaced with xfail later
+    ALL_TO_JELLY_CASES = [
+        case
+        for case in ALL_TO_JELLY_CASES
+        if not ("pos_014" in case.uri and case.category == "rdf11")
+    ]
 
 RDF11_POSITIVE_CASES = [
     pytest.param(case, id=case.id)

{pyjelly-0.6.1 → pyjelly-0.6.2}/tests/unit_tests/test_parse/test_lookup_decoder.py

@@ -16,7 +16,7 @@ def test_lookup_size_ok(size: int) -> None:
 def test_max_lookup_size_exceeded(size: int) -> None:
     with pytest.raises(JellyAssertionError) as excinfo:
         LookupDecoder(lookup_size=size)
-    assert str(excinfo.value) == f"lookup size must be less than {MAX_LOOKUP_SIZE}"
+    assert str(excinfo.value) == f"lookup size cannot be larger than {MAX_LOOKUP_SIZE}"
 
 
 def test_decode_zero_error() -> None:

{pyjelly-0.6.1 → pyjelly-0.6.2}/uv.lock

@@ -1409,7 +1409,7 @@ wheels = [
 
 [[package]]
 name = "pyjelly"
-version = "0.6.1"
+version = "0.6.2"
 source = { editable = "." }
 dependencies = [
     { name = "protobuf" },