xobjects 0.6.5__tar.gz → 0.6.6__tar.gz

xobjects-0.6.6/.github/workflows/black.yml

@@ -0,0 +1,10 @@
+name: Lint
+
+on: [push, pull_request]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: psf/black@26.1.0

xobjects-0.6.6/.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+*.pyc
+*.pyo
+*.swp
+*.egg-info/
+docs/_build
+cov_html
+.vscode
+.pytest_cache
+.coverage
+.idea
+*.c
+**/.DS_Store

xobjects-0.6.6/.pre-commit-config.yaml

@@ -0,0 +1,12 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+    -   id: check-yaml
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+  - repo: https://github.com/psf/black
+    rev: 23.10.1
+    hooks:
+      - id: black
+        language_version: python3 # Should be a command that runs python3.6+

xobjects-0.6.6/.readthedocs.yaml

@@ -0,0 +1,20 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+   configuration: docs/conf.py
+
+# Optionally build your docs in additional formats such as PDF
+formats:
+   - pdf
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+   version: 3.7
+#   install:
+#   - requirements: docs/requirements.txt

xobjects-0.6.6/PKG-INFO

@@ -0,0 +1,404 @@
+Metadata-Version: 2.4
+Name: xobjects
+Version: 0.6.6
+Summary: In-memory serialization and code generator for CPU and GPU
+Home-page: https://xsuite.readthedocs.io/
+Download-URL: https://pypi.python.org/pypi/xobjects
+Author: Riccardo De Maria
+Author-email: Riccardo De Maria <riccardo.de.maria@cern.ch>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://xsuite.readthedocs.io/
+Project-URL: Bug Tracker, https://github.com/xsuite/xsuite/issues
+Project-URL: Documentation, https://xsuite.readthedocs.io/
+Project-URL: Source Code, https://github.com/xsuite/xobjects
+Project-URL: Download, https://pypi.python.org/pypi/xobjects
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: cffi
+Requires-Dist: scipy
+Provides-Extra: tests
+Requires-Dist: pytest; extra == "tests"
+Requires-Dist: pytest-mock; extra == "tests"
+Dynamic: author
+Dynamic: download-url
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: requires-python
+
+# Xobjects
+
+Xobjects is a low-level Python package for defining structured data that can be
+stored in CPU or GPU memory and accessed from generated C APIs. It is used by
+the Xsuite ecosystem as the memory, serialization, and kernel-compilation layer,
+but it can also be used directly when a project needs the same data layout from
+Python and from compiled CPU, CUDA, or OpenCL code.
+
+The main building blocks are:
+
+- contexts, which select the execution and memory backend;
+- buffers, which own memory allocated by a context;
+- xobject data types, which describe the binary layout of scalars, arrays,
+  structures, strings, and references;
+- kernels, which bind C-like source code to typed Python callables.
+
+## Installation
+
+```bash
+python -m pip install xobjects
+```
+
+For local development:
+
+```bash
+git clone https://github.com/xsuite/xobjects.git
+cd xobjects
+python -m pip install -e ".[tests]"
+```
+
+GPU contexts require their corresponding runtime packages and drivers:
+
+- `ContextCpu` uses NumPy arrays and CFFI-compiled CPU kernels.
+- `ContextCupy` uses CuPy arrays and CUDA kernels.
+- `ContextPyopencl` uses PyOpenCL arrays and OpenCL kernels.
+
+OpenMP support for CPU kernels requires an OpenMP-capable compiler and runtime
+to be installed in the environment.
+
+
+## Development
+
+Run the test suite with:
+
+```bash
+pytest tests
+```
+
+Some tests and examples need optional GPU dependencies or a configured GPU
+runtime. CPU-only development can still run the regular test subset that does
+not require those contexts. Which tests are run is specified by the `XOBJECTS_TEST_CONTEXTS` flag. By default all contexts supported by the system are used, but this can be narrowed down, e.g. only the CPU serial and OpenMP tests will be run if `XOBJECTS_TEST_CONTEXTS=ContextCpu;ContextCpu:auto` is set.
+
+## Contributing
+
+The project uses Black for Python formatting and provides a pre-commit
+configuration. Install the pre-commit hooks before committing:
+
+```bash
+pip install pre-commit
+pre-commit install
+```
+
+This will run formatting hooks on every commit.
+
+
+## Contexts and Buffers
+
+A context represents where data lives and where kernels execute. A buffer is a
+memory allocation owned by a context. Xobjects can allocate a new buffer
+implicitly, or they can be placed explicitly in an existing buffer so that
+several objects share one memory range.
+
+```python
+import xobjects as xo
+
+ctx = xo.ContextCpu()
+buffer = ctx.new_buffer(capacity=1024)
+
+class Point(xo.Struct):
+    x = xo.Float64
+    y = xo.Float64
+
+p1 = Point(x=1.0, y=2.0, _context=ctx)   # new buffer on ctx
+p2 = Point(x=3.0, y=4.0, _buffer=buffer) # explicit buffer
+
+print(p1._context)
+print(p2._buffer is buffer)
+```
+
+If no `_context` or `_buffer` is supplied, Xobjects uses the default CPU
+context. Passing `_context` creates a new buffer in that context. Passing
+`_buffer` places the object in that buffer. Passing `_offset` is supported for
+advanced use, but it must be used carefully because overlapping allocations can
+corrupt objects in the same buffer.
+
+Context arrays can be converted to and from NumPy arrays through the context:
+
+```python
+import numpy as np
+import xobjects as xo
+
+ctx = xo.ContextCpu()
+host = np.arange(8, dtype=np.float64)
+
+on_context = ctx.nparray_to_context_array(host)
+back_on_host = ctx.nparray_from_context_array(on_context)
+```
+
+On GPU contexts, array attributes expose NumPy-like GPU arrays. Some NumPy
+operations require an explicit copy back to host memory with
+`nparray_from_context_array`.
+
+`ContextCpu` can compile kernels with OpenMP support when it is created with
+`omp_num_threads` different from `0`:
+
+```python
+import xobjects as xo
+
+ctx = xo.ContextCpu(omp_num_threads="auto") # let OpenMP choose the thread count
+ctx = xo.ContextCpu(omp_num_threads=4)      # use four OpenMP threads
+ctx = xo.ContextCpu(omp_num_threads=0)      # serial CPU execution
+```
+
+## Defining Data
+
+Xobjects data types describe a stable binary layout. The same definition drives
+Python accessors, buffer allocation, serialization helpers, and generated C
+accessors.
+
+### Scalars
+
+Scalar types wrap fixed-width NumPy dtypes and C types:
+
+- floating point: `Float64`, `Float32`;
+- signed integers: `Int64`, `Int32`, `Int16`, `Int8`;
+- unsigned integers: `UInt64`, `UInt32`, `UInt16`, `UInt8`.
+
+```python
+import xobjects as xo
+
+value = xo.Float64(3.5)
+index = xo.Int64(7)
+```
+
+### Arrays
+
+Arrays are declared by indexing an item type:
+
+```python
+import xobjects as xo
+
+Vector = xo.Float64[3]   # fixed-length array
+Series = xo.Float64[:]   # one dynamic dimension
+Matrix = xo.Float64[:, 3]
+
+v = Vector([1.0, 2.0, 3.0])
+s = Series([0.0, 1.0, 2.0, 3.0])
+m = Matrix([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
+```
+
+Arrays can contain scalars, structs, strings, and references. Fixed-size arrays
+have a static footprint; dynamic arrays store size and layout metadata in the
+buffer before their data.
+
+### Structs
+
+`Struct` classes define named fields stored together in one xobject.
+
+```python
+import xobjects as xo
+
+class Particle(xo.Struct):
+    x = xo.Float64
+    px = xo.Float64
+    state = xo.Int64
+
+particle = Particle(x=1.0, px=0.01, state=1)
+particle.x += 0.5
+```
+
+Fields can be scalars, arrays, nested structs, strings, `Ref` values, or
+`UnionRef` values. Dynamic fields are stored through offsets inside the same
+buffer-backed object.
+
+### Hybrid Classes
+
+`HybridClass` is a convenience layer for user-facing Python classes that keep
+regular Python methods and attributes while storing selected fields in an
+underlying xobject. The C-visible fields are declared in `_xofields`.
+
+```python
+import xobjects as xo
+
+class Record(xo.HybridClass):
+    _xofields = {
+        "value": xo.Float64,
+        "samples": xo.Float64[:],
+    }
+
+    def add_offset(self, offset):
+        self.value += offset
+
+record = Record(value=1.5, samples=[0.0, 1.0, 2.0])
+record.add_offset(0.5)
+
+print(record.value)
+print(record._xobject.samples)
+```
+
+The attributes listed in `_xofields` are backed by the same memory as
+`record._xobject`. Array fields are exposed as NumPy or NumPy-like arrays on the
+owning context.
+
+### Strings and References
+
+`String` stores a null-terminated variable-length string. `Ref(T)` stores a
+reference to another object of type `T` in the same buffer. `UnionRef` stores a
+reference that can point to one of several declared xobject types.
+
+References are useful when multiple objects need to share data without copying
+it. They must refer to objects in the same buffer because the stored value is a
+relative offset.
+
+## The C Programming Model
+
+Every xobject type can generate C declarations and accessor functions for its
+buffer layout. This lets kernel code read and write fields without hard-coding
+offsets.
+
+```python
+import xobjects as xo
+
+class Point(xo.Struct):
+    x = xo.Float64
+    y = xo.Float64
+
+print(Point._gen_c_api().source)
+```
+
+The generated API includes functions such as:
+
+```c
+double Point_get_x(const Point obj);
+void Point_set_x(Point obj, double value);
+double* Point_getp_x(Point obj);
+```
+
+For array fields, the generated API also includes length, getter, setter, and
+pointer helpers. For references and unions, it includes helpers that recover the
+referenced object and type information.
+
+Handwritten C code should use the generated accessors instead of assuming a
+particular byte layout. That keeps the source portable across CPU, CUDA, and
+OpenCL backends and across static or dynamic field layouts.
+
+Xobjects provides portability macros through `xobjects/headers/common.h`, for
+example `GPUFUN`, `GPUKERN`, `GPUGLMEM`, and `RESTRICT`.
+
+## Defining Kernels
+
+A kernel is a compiled function registered on a context. Its source is C-like
+code and its Python-side signature is described with `xo.Kernel` and `xo.Arg`.
+Xobjects uses the signature to generate the required type API, compile the
+source for the selected backend, and expose the result through `context.kernels`.
+
+```python
+import numpy as np
+import xobjects as xo
+
+ctx = xo.ContextCpu()
+
+source = r"""
+GPUKERN
+void scale_and_add(const int32_t n,
+                   const double a,
+                   GPUGLMEM const double* x,
+                   GPUGLMEM double* y)
+{
+    VECTORIZE_OVER(i, n)
+        y[i] = a * x[i] + y[i];
+    END_VECTORIZE
+}
+"""
+
+kernels = {
+    "scale_and_add": xo.Kernel(
+        args=[
+            xo.Arg(xo.Int32, name="n"),
+            xo.Arg(xo.Float64, name="a"),
+            xo.Arg(xo.Float64, pointer=True, const=True, name="x"),
+            xo.Arg(xo.Float64, pointer=True, name="y"),
+        ],
+        n_threads="n",
+    )
+}
+
+ctx.add_kernels(
+    sources=[source],
+    kernels=kernels,
+    extra_headers=['#include "xobjects/headers/common.h"'],
+)
+
+x = ctx.nparray_to_context_array(np.arange(5, dtype=np.float64))
+y = ctx.nparray_to_context_array(np.ones(5, dtype=np.float64))
+
+ctx.kernels.scale_and_add(n=len(x), a=2.0, x=x, y=y)
+print(ctx.nparray_from_context_array(y))
+```
+
+Kernels can also take xobject structs or arrays as arguments:
+
+```python
+import xobjects as xo
+
+class Point(xo.Struct):
+    x = xo.Float64
+    y = xo.Float64
+
+source = r"""
+GPUFUN
+double norm2(Point p)
+{
+    return Point_get_x(p) * Point_get_x(p)
+         + Point_get_y(p) * Point_get_y(p);
+}
+"""
+
+kernels = {
+    "norm2": xo.Kernel(
+        args=[xo.Arg(Point, name="p")],
+        ret=xo.Arg(xo.Float64),
+    )
+}
+
+ctx = xo.ContextCpu()
+ctx.add_kernels(
+    sources=[source],
+    kernels=kernels,
+    extra_headers=['#include "xobjects/headers/common.h"'],
+)
+
+point = Point(x=3.0, y=4.0, _context=ctx)
+print(ctx.kernels.norm2(p=point))
+```
+
+For reusable classes, kernel descriptions can be attached to a `Struct` or
+`HybridClass` through `_kernels` and compiled with `compile_kernels()`.
+
+Loops written with `VECTORIZE_OVER` use the execution policy of the selected
+context. On a serial CPU context they become regular `for` loops. On an OpenMP
+CPU context they become OpenMP parallel loops. On GPU contexts they map to the
+backend thread index.
+
+## Moving and Copying
+
+Hybrid objects support moving or copying data across buffers and contexts:
+
+```python
+import xobjects as xo
+
+class Item(xo.HybridClass):
+    _xofields = {
+        "value": xo.Float64,
+    }
+
+cpu_item = Item(value=1.0, _context=xo.ContextCpu())
+
+other_cpu_ctx = xo.ContextCpu()
+other_item = cpu_item.copy(_context=other_cpu_ctx)
+cpu_item.move(_buffer=xo.context_default.new_buffer())
+```
+
+Objects with references have stricter rules because references are buffer-local.
+When using `Ref` or `UnionRef`, keep the referencing object and the referenced
+object in the same buffer.