pyjelly 0.2.3__tar.gz → 0.4.0__tar.gz

{pyjelly-0.2.3 → pyjelly-0.4.0}/.github/workflows/release.yml

@@ -58,7 +58,7 @@ jobs:
         uses: pypa/gh-action-pypi-publish@release/v1
 
       - name: 'Create a GitHub release'
-        uses: ncipollo/release-action@v1.16.0
+        uses: ncipollo/release-action@v1.18.0
         with:
           prerelease: false
           draft: false

{pyjelly-0.2.3 → pyjelly-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyjelly
-Version: 0.2.3
+Version: 0.4.0
 Summary: Jelly-RDF implementation for Python
 Project-URL: Homepage, https://w3id.org/jelly/pyjelly
 Project-URL: Documentation, https://w3id.org/jelly/pyjelly
@@ -28,7 +28,7 @@ Classifier: Topic :: File Formats
 Classifier: Topic :: Software Development :: Libraries
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.9
-Requires-Dist: protobuf>=5.29.3
+Requires-Dist: protobuf>=6.30.0
 Requires-Dist: typing-extensions>=4.12.2
 Provides-Extra: rdflib
 Requires-Dist: rdflib>=7.1.4; extra == 'rdflib'
@@ -50,13 +50,13 @@ Description-Content-Type: text/markdown
 
 ## Getting started
 
-To get started with pyjelly, you can install it via pip:
+Install pyjelly from **[PyPI](https://pypi.org/project/pyjelly/)**:
 
 ```bash
 pip install pyjelly[rdflib]
 ```
 
-To write an RDF graph to a Jelly file, you can use the following code:
+To write an RDF graph to a Jelly file:
 
 ```python
 from rdflib import Graph
@@ -66,7 +66,7 @@ g.parse("http://xmlns.com/foaf/spec/index.rdf")
 g.serialize(destination="foaf.jelly", format="jelly")
 ```
 
-To read a Jelly file and convert it to an rdflib Graph, you can use:
+To read a Jelly file and convert it to an rdflib `Graph`:
 
 ```python
 from rdflib import Graph
@@ -75,19 +75,19 @@ g = Graph()
 g.parse("foaf.jelly", format="jelly")
 ```
 
-**See more examples, [API reference](https://w3id.org/jelly/pyjelly/dev/api), and more in the [documentation](https://w3id.org/jelly/pyjelly).**
+**See [our documentation](https://w3id.org/jelly/pyjelly) for [further examples](https://w3id.org/jelly/pyjelly/dev/getting-started/), a full [API reference](https://w3id.org/jelly/pyjelly/dev/api), and more.**
 
 ## Contributing and support
 
 This project is being actively developed – you can stay tuned by [watching this repository](https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/about-notifications#subscription-options).
 
-You can also join the **[Jelly Discord chat](https://discord.gg/A8sN5XwVa5)** to ask questions about pyjelly and to be up-to-date with the development activities.
+Join the **[Jelly Discord chat](https://discord.gg/A8sN5XwVa5)** to ask questions about pyjelly and to be up-to-date with the development activities.
 
 ### Commercial support
 
 **[NeverBlink](https://neverblink.eu)** provides commercial support services for Jelly, including implementing custom features, system integrations, implementations for new frameworks, benchmarking, and more.
 
-## Contributing
+### Contributing
 
 If you'd like to contribute, check out our [contributing guidelines](CONTRIBUTING.md).
 

{pyjelly-0.2.3 → pyjelly-0.4.0}/README.md

@@ -14,13 +14,13 @@
 
 ## Getting started
 
-To get started with pyjelly, you can install it via pip:
+Install pyjelly from **[PyPI](https://pypi.org/project/pyjelly/)**:
 
 ```bash
 pip install pyjelly[rdflib]
 ```
 
-To write an RDF graph to a Jelly file, you can use the following code:
+To write an RDF graph to a Jelly file:
 
 ```python
 from rdflib import Graph
@@ -30,7 +30,7 @@ g.parse("http://xmlns.com/foaf/spec/index.rdf")
 g.serialize(destination="foaf.jelly", format="jelly")
 ```
 
-To read a Jelly file and convert it to an rdflib Graph, you can use:
+To read a Jelly file and convert it to an rdflib `Graph`:
 
 ```python
 from rdflib import Graph
@@ -39,19 +39,19 @@ g = Graph()
 g.parse("foaf.jelly", format="jelly")
 ```
 
-**See more examples, [API reference](https://w3id.org/jelly/pyjelly/dev/api), and more in the [documentation](https://w3id.org/jelly/pyjelly).**
+**See [our documentation](https://w3id.org/jelly/pyjelly) for [further examples](https://w3id.org/jelly/pyjelly/dev/getting-started/), a full [API reference](https://w3id.org/jelly/pyjelly/dev/api), and more.**
 
 ## Contributing and support
 
 This project is being actively developed – you can stay tuned by [watching this repository](https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/about-notifications#subscription-options).
 
-You can also join the **[Jelly Discord chat](https://discord.gg/A8sN5XwVa5)** to ask questions about pyjelly and to be up-to-date with the development activities.
+Join the **[Jelly Discord chat](https://discord.gg/A8sN5XwVa5)** to ask questions about pyjelly and to be up-to-date with the development activities.
 
 ### Commercial support
 
 **[NeverBlink](https://neverblink.eu)** provides commercial support services for Jelly, including implementing custom features, system integrations, implementations for new frameworks, benchmarking, and more.
 
-## Contributing
+### Contributing
 
 If you'd like to contribute, check out our [contributing guidelines](CONTRIBUTING.md).
 

{pyjelly-0.2.3 → pyjelly-0.4.0}/docs/contributing/index.md

@@ -4,16 +4,33 @@ Hi! This guide explains how to get started with developing pyjelly and contribut
 
 ## Quick start
 
-1. Clone the project with `git clone git@github.com:Jelly-RDF/pyjelly.git`.
+Clone the project: 
 
-2. We use `uv` for package management. If you don't already have it, [install uv](https://github.com/astral-sh/uv).
-    * On Linux this is: `curl -LsSf https://astral.sh/uv/install.sh | sh`
+```shell
+git clone --recursive git@github.com:Jelly-RDF/pyjelly.git
+```
+
+We use `uv` for package management. If you don't already have it, [install uv](https://github.com/astral-sh/uv).
 
-3. Run `uv sync` to install the project.
-    * If you use an IDE, make sure that it uses the Python interpreter from the environment that will be created in the `.venv` directory.
-    * If you get an error about the uv version being incorrect, run `uv self update`
+On Linux this is: 
 
-4. [Activate the environment](https://docs.python.org/3/library/venv.html#how-venvs-work) or use [`uv run` to run commands and code](https://docs.astral.sh/uv/guides/projects/). 
+```shell
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+Install the project:
+
+```shell
+uv sync
+```
+
+!!! note 
+    
+    If you use an IDE, make sure that it uses the Python interpreter from the environment that will be created in the `.venv` directory.
+
+    If you get an error about the uv version being incorrect, run `uv self update`
+
+[Activate the environment](https://docs.python.org/3/library/venv.html#how-venvs-work) or use [`uv run` to run commands and code](https://docs.astral.sh/uv/guides/projects/). 
 
 ## Giving feedback
 
@@ -27,16 +44,22 @@ If you are proposing a feature:
 
 ## Contributing code
 
-1. Every major pull request should be connected to an issue. If you see a problem, first create an issue.
-    * For minor issues (typos, small fixes) make sure you describe your problem well in the PR.
-2. When opening a pull request:
-    * Use a descriptive title.
-    * Reference the related issue in the description.
-3. Please make sure your code passes all the checks:
-    * Tests (`pytest`)
-    * Type safety (`mypy`)
-    * Formatting and linting (`ruff` or via `pre-commit`)
-    This helps us follow best practices and keep the codebase in shape.
+If you've found a major issue (not just a typo) or have a feature request, please [open an issue](https://github.com/Jelly-RDF/pyjelly/issues/new/choose) first. This helps us keep track of what needs to be done and allows others to contribute as well.
+
+**When opening a pull request:**
+
+- Use a descriptive title.
+- Clearly describe what the pull request does and why it is needed.
+- Reference the related issue in the description.
+
+The CI will check if your code passes all checks. You can run these checks locally:
+
+- Tests: `uv run pytest`
+- Type safety: `uv run mypy .`
+- Linter: `uv run ruff check --fix`
+- Formatter: `uv run ruff format`
+
+This helps us follow best practices and keep the codebase in shape.
 
 ## Contributing documentation
 

{pyjelly-0.2.3 → pyjelly-0.4.0}/docs/env.py

@@ -90,7 +90,7 @@ def define_env(env):
     @env.macro
     def git_link(file: str):
         tag = git_tag()
-        return f"https://github.com/Jelly-RDF/jelly-python/blob/{tag}/{file}"
+        return f"https://github.com/Jelly-RDF/pyjelly/blob/{tag}/{file}"
 
     @env.macro
     def proto_version():

pyjelly-0.4.0/docs/examples/rdflib/04_parse_grouped.py

@@ -0,0 +1,17 @@
+import gzip
+import urllib.request
+
+from pyjelly.integrations.rdflib.parse import parse_jelly_grouped
+
+# Dataset: Katrina weather measurements (10k graphs)
+# Documentation: https://w3id.org/riverbench/datasets/lod-katrina/dev
+url = "https://w3id.org/riverbench/datasets/lod-katrina/dev/files/jelly_10K.jelly.gz"
+
+# Load, uncompress .gz file, and pass to Jelly parser, all in a streaming manner
+with (
+    urllib.request.urlopen(url) as response,
+    gzip.open(response) as jelly_stream,
+):
+    graphs = parse_jelly_grouped(jelly_stream)
+    for i, graph in enumerate(graphs):
+        print(f"Graph {i} in the stream has {len(graph)} triples")

pyjelly-0.4.0/docs/examples/rdflib/05_parse_flat.py

@@ -0,0 +1,29 @@
+import gzip
+import urllib.request
+
+from pyjelly.integrations.rdflib.parse import parse_jelly_flat, Triple
+from rdflib import URIRef
+
+# Dataset: OpenStreetMap data for Denmark (first 10k objects)
+# Documentation: https://w3id.org/riverbench/datasets/osm2rdf-denmark/dev
+url = (
+    "https://w3id.org/riverbench/datasets/osm2rdf-denmark/dev/files/jelly_10K.jelly.gz"
+)
+
+# We are looking for city names in the dataset
+predicate_to_look_for = URIRef("https://www.openstreetmap.org/wiki/Key:addr:city")
+city_names = set()
+
+with (
+    urllib.request.urlopen(url) as response,
+    gzip.open(response) as jelly_stream,
+):
+    for event in parse_jelly_flat(jelly_stream):
+        if isinstance(event, Triple):  # we are only interested in triples
+            if event.p == predicate_to_look_for:
+                city_names.add(event.o)
+
+print(f"Found {len(city_names)} unique city names in the dataset.")
+print("10 random city names:")
+for city in list(city_names)[:10]:
+    print(f"- {city}")

pyjelly-0.4.0/docs/examples/rdflib/06_serialize_grouped.py

@@ -0,0 +1,22 @@
+from pyjelly.integrations.rdflib.serialize import grouped_stream_to_file
+
+from rdflib import Graph, Literal, Namespace
+import random
+
+
+def generate_sample_graphs():
+    ex = Namespace("http://example.org/")
+    for _ in range(10):
+        g = Graph()
+        g.add((ex.sensor, ex.temperature, Literal(random.random())))
+        g.add((ex.sensor, ex.humidity, Literal(random.random())))
+        yield g
+
+
+output_file_name = "output.jelly"
+
+print(f"Streaming graphs into {output_file_name}…")
+sample_graphs = generate_sample_graphs()
+with open(output_file_name, "wb") as out_file:
+    grouped_stream_to_file(sample_graphs, out_file)
+print("All done.")

pyjelly-0.4.0/docs/getting-started.md

@@ -0,0 +1,73 @@
+# Getting started
+
+This guide shows how to install pyjelly and prepare your environment for use with RDFLib.
+
+## Installation (with RDFLib)
+
+Install pyjelly from PyPI:
+
+```
+pip install pyjelly[rdflib]
+```
+
+pyjelly requires **Python 3.9** or newer and works on all major platforms (Linux, macOS, Windows).
+
+
+## Usage with RDFLib
+
+Once installed, pyjelly integrates with RDFLib automatically. You can immediately serialize and parse `.jelly` files using the standard RDFLib API.
+
+### Serializing a graph
+
+To serialize a graph to the Jelly format:
+
+{{ code_example('rdflib/01_serialize.py') }}
+
+This creates a [delimited Jelly stream]({{ proto_link("user-guide/#delimited-vs-non-delimited-jelly") }}) using default options.
+
+### Parsing a graph
+
+To load RDF data from a `.jelly` file:
+
+{{ code_example('rdflib/02_parse.py') }}
+
+RDFLib will reconstruct the graph from the Jelly file.
+
+### Parsing a stream of graphs
+
+You can process a Jelly stream as a stream of graphs. A Jelly file consists of "frames" (batches of statements) – we can load each frame as a separate RDFLib graph.
+
+In this example, we use a [dataset of weather measurements](https://w3id.org/riverbench/datasets/lod-katrina/dev), which is an RDF graph stream. We count the number of triples in each graph:
+
+{{ code_example('rdflib/04_parse_grouped.py') }}
+
+Because `parse_jelly_grouped` returns a generator, each iteration receives **one** graph, keeping memory usage bounded to the current frame. So, large datasets and live streams can be processed efficiently.
+
+### Parsing a stream of triples
+
+You can also process a Jelly stream as a flat stream of triples.
+
+In this more complex example, we look through a fragment of Denmark's OpenStreetMap to find all city names:
+
+{{ code_example('rdflib/05_parse_flat.py') }}
+
+`parse_jelly_flat` returns a generator of stream events (i.e., statements parsed). This allows you to efficiently process the file triple-by-triple and build custom aggregations from the stream.
+
+### Serializing a stream of graphs
+
+If you have a generator object containing graphs, you can easily serialize it into the Jelly format, like in the following example: 
+
+{{ code_example('rdflib/06_serialize_grouped.py')}}
+
+This method allows for transmitting logically grouped data, preserving their original division.
+For more precise control over frame serialization you can use [lower-level API](api.md)
+
+### File extension support
+
+You can generally omit the `format="jelly"` parameter if the file ends in `.jelly` – RDFLib will auto-detect the format:
+
+{{ code_example('rdflib/03_parse_autodetect.py') }}
+
+!!! warning 
+
+    Unfortunately, the way this is implemented in RDFLib is a bit wonky, so it will only work if you explicitly import `pyjelly.integrations.rdflib`, or you used `format="jelly"` in the `serialize()` or `parse()` call before.

pyjelly-0.4.0/docs/overview.md

@@ -0,0 +1,38 @@
+## What is Jelly and pyjelly?
+
+**[Jelly]({{ proto_link() }})** is a high-performance serialization format for RDF knowledge graphs and knowledge graph streams. It's designed to be fast, compact, and flexible. 
+
+With Jelly, you can transmit both flat and structured streams of triples, quads, graphs, and datasets. Jelly works well in both batch and real-time settings – including files, sockets, or streaming protocols like Kafka or gRPC.
+
+**pyjelly** is a Python implementation of the Jelly protocol. It provides:
+
+* Full support for reading and writing Jelly-encoded RDF data
+* Seamless integration with [RDFLib](https://rdflib.readthedocs.io/) (*"works just like Turtle"*)
+* Support for all Jelly stream types
+* Tools for working with delimited and non-delimited Jelly streams
+* Fine-grained control over serialization options, compression, and framing
+
+## Overview
+
+### Supported stream types
+
+pyjelly supports all [physical stream types]({{ proto_link("specification/reference/#physicalstreamtype") }}): `TRIPLES`, `QUADS` and `GRAPHS`.
+
+See the full [stream type matrix]({{ proto_link("serialization/#consistency-with-physical-stream-types") }}) for an overview of valid combinations.
+
+### Conformance to the Jelly specification
+
+pyjelly is continuously tested for conformance to [the Jelly specification]({{ proto_link("specification/") }}). While the vast majority of features are implemented, there are a few edge cases left to resolve.
+
+You can track the progress in [the conformance test suite definition]({{ git_link("tests/utils/rdf_test_cases.py") }}).
+
+## Use cases
+
+Use cases for pyjelly include:
+
+- **Client-server communication** – link your client app in Python to the server (e.g., [Apache Jena, RDF4J](https://w3id.org/jelly/jelly-jvm)) with Jelly to reduce latency and improve user experience.
+- **Inter-service communication** – use Jelly to efficiently exchange RDF data between microservices.
+- **Data science workflows** – use Jelly to read and write RDF data in data science pipelines, enabling efficient processing of large datasets. 
+    - pyjelly is fully streaming, so it can handle large datasets without loading everything into memory at once.
+    - We are working on support for pandas and other data science libraries – stay tuned for updates!
+- **Database dumps and bulk loads** – quickly read and write large RDF datasets with Jelly, reducing storage space and improving database maintenance tasks.

{pyjelly-0.2.3 → pyjelly-0.4.0}/mkdocs.yml

@@ -125,4 +125,5 @@ nav:
       - "contributing/index.md"
       - Code of conduct: "contributing/code_of_conduct.md"
   - Changelog: "https://github.com/Jelly-RDF/pyjelly/releases"
+  - Licensing and citation: "https://w3id.org/jelly/dev/licensing"
   - Back to main Jelly docs ↩: "https://w3id.org/jelly/"