sharedbox 0.4.1__tar.gz → 0.5.0__tar.gz

sharedbox-0.5.0/.github/workflows/contention.yaml

@@ -0,0 +1,45 @@
+name: Contention
+
+on:
+  pull_request:
+    branches: [ main ]
+    paths:
+      - include/sharedbox/**
+      - src/sharedbox/_native/**
+      - src/sharedbox/benchmarks/contention.py
+      - .github/workflows/contention.yaml
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  contention:
+    name: Contention on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ ubuntu-latest, windows-latest ]
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0
+
+      - uses: astral-sh/setup-uv@v7
+
+      - name: Install the project
+        run: uv sync --frozen --python 3.12
+
+      - name: Run benchbox contention
+        shell: bash
+        run: |
+          cores=$(uv run --no-sync python -c "import os; print(os.cpu_count())")
+          echo "### Contention on ${{ matrix.os }}, $cores logical cores" >> "$GITHUB_STEP_SUMMARY"
+          uv run --no-sync benchbox contention --json contention.json --markdown >> "$GITHUB_STEP_SUMMARY"
+
+      - uses: actions/upload-artifact@v7
+        with:
+          name: contention-${{ matrix.os }}
+          path: contention.json

{sharedbox-0.4.1 → sharedbox-0.5.0}/CHANGELOG.md

@@ -9,6 +9,44 @@ Dates are marked as `DD-MM-YYYY`
 
 ## [Unreleased]
 
+## [0.5.0] - 09-10-2026
+
+### Added
+
+- `SharedBox.read_into`: copies an array field into an array the caller
+  passes and returns it, without allocating a new array.
+- `SharedBox.writing`: a context manager that holds the box's write lock
+  and yields an array field as an array to fill in place.
+- `benchbox contention`: times writes and reads of an `int` while several
+  writer and reader processes share it, for a box, `mp.Value` and
+  `SharedMemory` with a `Lock`.
+
+### Changed
+
+- `SharedBox`: a field named `read_into` raises `TypeError`.
+- `SharedBox`: a field named `writing` raises `TypeError`.
+- `SharedBox`: reading a collection, tuple or record field whose members
+  are `bool`, `int`, `float` or `str` takes less time: about 90 ns less
+  for a list of 16 `float` on Windows.
+- `SharedBox`: reading a field whose value cannot be changed (a frozen
+  dataclass, attrs class or msgspec Struct, a NamedTuple, a tuple, a
+  frozenset, an enum, flag or literal, `complex`, `date`, `time`,
+  `datetime`, `timedelta`, `UUID` or `Decimal`) returns the value the
+  previous read built until the field is written: 40 ns instead of 540 ns
+  for a frozen dataclass of four fields on Windows. A tuple, frozenset or
+  record qualifies only when its members are of these kinds or `bool`,
+  `int`, `float`, `str` or `bytes`.
+- `SharedBox`: a frozen record's `__post_init__`, attrs converters and
+  validators run on the first read after each write instead of on every
+  read.
+
+### Fixed
+
+- `SharedBox.attach`: a box whose header gives a field count other than the
+  class's raises `SchemaMismatchError` instead of `ValueError`.
+
+## [0.4.1] - 07-10-2026
+
 ### Changed
 
 - `SharedBox.update`: takes about 7 to 9 ns less per call on Windows.
@@ -292,7 +330,9 @@ stage.events.nested.connect(lambda path, new, old: print(path, new))
 
 - Initial release
 
-[Unreleased]: https://github.com/jacopoabramo/sharedbox/compare/v0.4.0...HEAD
+[Unreleased]: https://github.com/jacopoabramo/sharedbox/compare/v0.5.0...HEAD
+[0.5.0]: https://github.com/jacopoabramo/sharedbox/compare/v0.4.1...v0.5.0
+[0.4.1]: https://github.com/jacopoabramo/sharedbox/compare/v0.4.0...v0.4.1
 [0.4.0]: https://github.com/jacopoabramo/sharedbox/compare/v0.3.1...v0.4.0
 [0.3.1]: https://github.com/jacopoabramo/sharedbox/compare/v0.3.0...v0.3.1
 [0.3.0]: https://github.com/jacopoabramo/sharedbox/compare/0.2.4...v0.3.0

{sharedbox-0.4.1 → sharedbox-0.5.0}/CLAUDE.md

@@ -33,15 +33,17 @@ sharedbox/
 |   |-- py.typed
 |   |-- benchmarks/            the benchbox command (extra: benchmarks)
 |   |   |-- cli.py             entry point; exits with an install hint without Typer
-|   |   |-- _app.py            Typer commands: ops, roundtrip, size, plot, all
+|   |   |-- _app.py            Typer commands: ops, roundtrip, contention, size, plot, all
 |   |   |-- ops.py             pyperf timings of single operations, against the stdlib
 |   |   |-- roundtrip.py       cross-process round trip percentiles
+|   |   |-- contention.py      throughput and percentiles with several writer and reader processes
 |   |   |-- plot.py            SVG charts of the ops and roundtrip results (matplotlib)
 |   |   |-- size.py            wheel and extension size (standard library only)
 |   |   `-- size_diff.py       wheel size table against main, for CI (standard library only)
 |   `-- _native/
 |       |-- module.cpp         nanobind module: Segment, the Field descriptor, the BoxMethod type
-|       |                      (native update and snapshot) and the error classes
+|       |                      (native update and snapshot), ValueCache (values kept for reuse) and the
+|       |                      error classes
 |       |-- codec.{hpp,cpp}    converts field values to and from their stored bytes; encode_all
 |       |                      checks every value of an update before any is written,
 |       |                      with_record reads the whole record at once
@@ -51,6 +53,7 @@ sharedbox/
 |-- tests/                     pytest; many tests spawn processes
 |   |-- crossproc.py           helpers that run a box in another process
 |   |-- forged_types.py        builds segments with forged description tables
+|   |-- test_benchbox_contention.py  benchbox contention, a short run
 |   |-- test_benchbox_plot.py  benchbox plot; skipped without matplotlib and pyperf
 |   |-- test_capsule.py        __sharedbox_box__, and the C consumer in tests/cpp/consumer/
 |   |-- test_doc_examples.py   runs each docs/examples/*.py script
@@ -78,6 +81,8 @@ sharedbox/
 |-- .github/workflows/check-docs.yaml    builds the site and runs check_xrefs.py
 |-- .github/workflows/publish-docs.yaml  deploys the checked site to GitHub Pages
 |-- .github/workflows/codspeed.yml  benchmarks on CodSpeed
+|-- .github/workflows/contention.yaml  benchbox contention on Linux and Windows, for pull requests
+|                              that touch the native code, and by hand
 |-- CMakeLists.txt             sharedbox::headers, sharedbox::c, extension build
 |-- stubtest-allowlist.txt     stubtest exceptions for nanobind types
 |-- .clang-format              clang-format style for the C and C++ sources
@@ -280,11 +285,12 @@ if __name__ == "__main__":
     Motor.unlink()
 ```
 
-Every read decodes a fresh value from the segment. `update(**values)` writes
-several fields at once; `watch(field)` and `events` report changes from any
-process. C++ and C code take a box through `__sharedbox_box__` or open it by
-name; see `docs/how-to/accept-a-box-in-cpp.md`,
-`docs/how-to/accept-a-box-in-c.md` and
+A read decodes the stored bytes; a value that cannot be changed (a
+`datetime`, or a frozen record, tuple or frozenset holding only such values)
+is reused until its field is written. `update(**values)` writes several
+fields at once; `watch(field)` and `events` report changes from any process.
+C++ and C code take a box through `__sharedbox_box__` or open it by name; see
+`docs/how-to/accept-a-box-in-cpp.md`, `docs/how-to/accept-a-box-in-c.md` and
 `docs/how-to/open-a-box-from-a-program.md`.
 
 ## Docs

{sharedbox-0.4.1 → sharedbox-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sharedbox
-Version: 0.4.1
+Version: 0.5.0
 Summary: Python containers using shared memory.
 Author-Email: Jacopo Abramo <jacopo.abramo@gmail.com>
 License-Expression: Apache-2.0
@@ -34,16 +34,24 @@ Description-Content-Type: text/markdown
 > [!WARNING]
 > This project is a work in progress; be patient or feel free to contribute.
 
-`sharedbox` keeps records in shared memory. Each box is one named segment, and every process that opens it reads and writes the same fields.
+`sharedbox` lets several Python processes share one record, as if they all
+held the same dataclass. You declare the fields once, and every process that
+opens the record reads and writes the same values in shared memory, so no
+process has to send them to another.
 
 ## Installation
 
-It is recommended to install `sharedbox` in a virtual environment; for example using `uv`:
+`sharedbox` needs CPython 3.11 or newer on Windows x64 or Linux x86_64. PyPI
+has compiled wheels, so you don't need a compiler:
 
 ```sh
-uv venv --python 3.11
-.venv\Scripts\activate
-uv pip install sharedbox
+pip install sharedbox
+```
+
+or, in a project that [`uv`](https://docs.astral.sh/uv/) manages:
+
+```sh
+uv add sharedbox
 ```
 
 ## Quick start
@@ -76,14 +84,32 @@ if __name__ == "__main__":
     Motor.unlink()
 ```
 
+The child finds the box by its class alone, and the parent prints the change
+as soon as the child makes it.
+
 ## Documentation
 
-The [documentation site](https://jacopoabramo.github.io/sharedbox) has a
-tutorial, how-to guides, explanations of how a box works, and the API
-reference.
+The [documentation site](https://jacopoabramo.github.io/sharedbox) starts
+with three tutorials, then has how-to guides, explanations of how a
+box works, and the API reference. C and C++ code can use a box too, either
+handed over from Python or opened by name; the
+[C and C++ guides](https://jacopoabramo.github.io/sharedbox/how-to/accept-a-box-in-cpp/)
+show how.
+
+## Development
+
+To change `sharedbox` itself you need `git`, `uv`, CMake 3.30 or newer and a
+C++20 compiler (MSVC or GCC):
+
+```sh
+git clone https://github.com/jacopoabramo/sharedbox.git
+cd sharedbox
+uv sync --dev
+uv run pytest
+```
 
-C++ and C code can take a box from Python or open one by name; see the
-[C and C++ guides](https://jacopoabramo.github.io/sharedbox/how-to/accept-a-box-in-cpp/).
+[How to set up a development environment](https://jacopoabramo.github.io/sharedbox/how-to/set-up-development/)
+has the details.
 
 ## License
 

{sharedbox-0.4.1 → sharedbox-0.5.0}/README.md

@@ -10,16 +10,24 @@
 > [!WARNING]
 > This project is a work in progress; be patient or feel free to contribute.
 
-`sharedbox` keeps records in shared memory. Each box is one named segment, and every process that opens it reads and writes the same fields.
+`sharedbox` lets several Python processes share one record, as if they all
+held the same dataclass. You declare the fields once, and every process that
+opens the record reads and writes the same values in shared memory, so no
+process has to send them to another.
 
 ## Installation
 
-It is recommended to install `sharedbox` in a virtual environment; for example using `uv`:
+`sharedbox` needs CPython 3.11 or newer on Windows x64 or Linux x86_64. PyPI
+has compiled wheels, so you don't need a compiler:
 
 ```sh
-uv venv --python 3.11
-.venv\Scripts\activate
-uv pip install sharedbox
+pip install sharedbox
+```
+
+or, in a project that [`uv`](https://docs.astral.sh/uv/) manages:
+
+```sh
+uv add sharedbox
 ```
 
 ## Quick start
@@ -52,14 +60,32 @@ if __name__ == "__main__":
     Motor.unlink()
 ```
 
+The child finds the box by its class alone, and the parent prints the change
+as soon as the child makes it.
+
 ## Documentation
 
-The [documentation site](https://jacopoabramo.github.io/sharedbox) has a
-tutorial, how-to guides, explanations of how a box works, and the API
-reference.
+The [documentation site](https://jacopoabramo.github.io/sharedbox) starts
+with three tutorials, then has how-to guides, explanations of how a
+box works, and the API reference. C and C++ code can use a box too, either
+handed over from Python or opened by name; the
+[C and C++ guides](https://jacopoabramo.github.io/sharedbox/how-to/accept-a-box-in-cpp/)
+show how.
+
+## Development
+
+To change `sharedbox` itself you need `git`, `uv`, CMake 3.30 or newer and a
+C++20 compiler (MSVC or GCC):
+
+```sh
+git clone https://github.com/jacopoabramo/sharedbox.git
+cd sharedbox
+uv sync --dev
+uv run pytest
+```
 
-C++ and C code can take a box from Python or open one by name; see the
-[C and C++ guides](https://jacopoabramo.github.io/sharedbox/how-to/accept-a-box-in-cpp/).
+[How to set up a development environment](https://jacopoabramo.github.io/sharedbox/how-to/set-up-development/)
+has the details.
 
 ## License
 

{sharedbox-0.4.1 → sharedbox-0.5.0}/benchmarks/test_bench_box.py

@@ -1,5 +1,6 @@
 import os
 from collections.abc import Iterator
+from dataclasses import dataclass
 from itertools import count
 from typing import Annotated
 
@@ -17,6 +18,22 @@ class Record(SharedBox):
     s: Annotated[str, Capacity(32)]
 
 
+class Samples(SharedBox):
+    floats: Annotated[list[float], Capacity(16)]
+
+
+@dataclass(frozen=True)
+class Quad:
+    a: int
+    b: float
+    c: bool
+    d: int
+
+
+class Quads(SharedBox):
+    quad: Quad
+
+
 @pytest.fixture
 def box() -> Iterator[Record]:
     name = f"bench-codspeed-{os.getpid()}-{next(NAMES)}"
@@ -55,3 +72,27 @@ def test_native_set_int(benchmark: BenchmarkFixture, box: Record) -> None:
 
 def test_native_get_int(benchmark: BenchmarkFixture, box: Record) -> None:
     benchmark(box._segment.get, 0)
+
+
+@pytest.fixture
+def samples() -> Iterator[Samples]:
+    name = f"bench-codspeed-{os.getpid()}-{next(NAMES)}"
+    with Samples.create(name, [0.5] * 16) as box:
+        yield box
+    Samples.unlink(name)
+
+
+def test_read_float_list(benchmark: BenchmarkFixture, samples: Samples) -> None:
+    benchmark(getattr, samples, "floats")
+
+
+@pytest.fixture
+def quads() -> Iterator[Quads]:
+    name = f"bench-codspeed-{os.getpid()}-{next(NAMES)}"
+    with Quads.create(name, Quad(1, 1.5, True, 2)) as box:
+        yield box
+    Quads.unlink(name)
+
+
+def test_read_record(benchmark: BenchmarkFixture, quads: Quads) -> None:
+    benchmark(getattr, quads, "quad")

sharedbox-0.5.0/docs/diagrams/style.d2

@@ -0,0 +1,61 @@
+# Shared look of every diagram in the docs. A diagram pulls it in with
+# `...@diagrams/style` on its first line and gives shapes these classes.
+classes: {
+  layer: {
+    style: {
+      border-radius: 10
+      font-size: 18
+      bold: true
+    }
+  }
+  step: {
+    style.border-radius: 6
+  }
+  current: {
+    style: {
+      border-radius: 6
+      stroke-width: 4
+      bold: true
+      shadow: true
+    }
+  }
+  done: {
+    style: {
+      border-radius: 6
+      opacity: 0.4
+    }
+  }
+  failed: {
+    style: {
+      border-radius: 6
+      stroke: "#c62828"
+      stroke-dash: 4
+      font-color: "#c62828"
+    }
+  }
+  process: {
+    shape: hexagon
+  }
+  hardware: {
+    shape: cylinder
+  }
+  file: {
+    shape: page
+  }
+  note: {
+    shape: text
+    style: {
+      italic: true
+      font-size: 14
+    }
+  }
+  gap: {
+    label: ""
+    style.opacity: 0
+  }
+  # a shape a later step brings in: declared on the first board so that every
+  # board keeps the same layout, and shown by a step that changes its class
+  hidden: {
+    style.opacity: 0
+  }
+}

{sharedbox-0.4.1 → sharedbox-0.5.0}/docs/examples/store_arrays.py

@@ -23,6 +23,18 @@ sensor.frame = np.full((4, 6), 7, np.uint8)
 print(int(sensor.frame.sum()))  # 168
 # --8<-- [end:write]
 
+# --8<-- [start:read-into]
+frame = np.empty((4, 6), np.uint8)
+sensor.read_into("frame", frame)
+print(int(frame.sum()))  # 168
+# --8<-- [end:read-into]
+
+# --8<-- [start:writing]
+with sensor.writing("frame") as frame:
+    frame[0, :] = 255
+print(int(sensor.frame[0].sum()))  # 1530
+# --8<-- [end:writing]
+
 # --8<-- [start:any-library]
 raw = sensor.raw
 print(np.from_dlpack(raw))  # [0. 0. 0.]

sharedbox-0.5.0/docs/explanation/checking-a-process-is-alive.md

@@ -0,0 +1,89 @@
+---
+icon: lucide/lightbulb
+---
+
+# Checking a process is alive
+
+Sometimes `sharedbox` has to answer a simple-sounding question: is the
+process recorded in a [segment](glossary.md#segment) still running? The
+answer decides whether a crashed waiter's slot can be freed, and what an
+error message tells you about who holds a name. This page explains how it
+answers, and why the obvious check isn't enough.
+
+## A pid and a start time
+
+The obvious check would be "is there a process with this process id
+(pid)?", but that isn't enough: once a process exits, the operating system
+can give its pid to a new one. So `sharedbox.hpp` names a process by its
+pid and its start time, the way the `psutil` library tells a process from a
+later one with the same pid.[^psutil] On Windows the start
+time is the creation time from `GetProcessTimes`. On Linux it is field 22
+of `/proc/<pid>/stat`.[^proc-stat] `process_alive()` reads the start time
+of whatever process has the pid now and decides like this:
+
+```d2 title="Is the recorded process still running?"
+...@diagrams/style
+direction: down
+pid: "a process has the pid?" {class: step}
+zombie: "on Linux: a zombie?" {
+  class: step
+  tooltip: A zombie, state Z or X in /proc/PID/stat, has exited and keeps its entry only until its parent collects it.
+}
+readable: "start time readable?" {
+  class: step
+  tooltip: It is not when the process belongs to another user or /proc is mounted with hidepid.
+}
+same: "same start time?" {class: step}
+dead: "dead" {class: failed}
+alive: "alive" {class: current}
+pid -> dead: "no"
+pid -> zombie: "yes"
+zombie -> dead: "yes"
+zombie -> readable: "no"
+readable -> alive: "no" {
+  tooltip: The process exists and nothing shows it is a different one, so it counts as alive.
+}
+readable -> same: "yes"
+same -> alive: "yes"
+same -> dead: "no, the pid was reused"
+```
+
+A recorded start time that couldn't be read is treated the same way: the
+process counts as alive.[^proc-stat]
+
+On Linux there's one more catch: containers give their processes their own
+set of pids, called a pid namespace, so the same pid can mean different
+processes in different containers. That's why
+[waiter slots](glossary.md#waiter-slot) and the header's creator fields also
+record the namespace, and a process in another namespace, or one whose
+namespace is unknown, is never judged dead.
+
+## Where the check is used
+
+- Freeing the waiter slots of dead processes, as [Waiting for
+  changes](waiting-for-changes.md#waking-a-process) describes.
+- Naming the creator in
+  [`SegmentExistsError`][sharedbox.SegmentExistsError]. When a create finds
+  the name taken, the extension reads the existing header with
+  `sharedbox::inspect()` and checks the creator it records. The creator
+  writes the header before it publishes the [box](glossary.md#box), so a
+  name that holds no published box is checked the same way. The docstring
+  of `SegmentExistsError` lists the messages that result.
+
+Even when the creator has exited, nothing is removed automatically, because
+other processes may still be using a segment whose creator died. Removing
+it is your call; [How to clean up segments](../how-to/clean-up-segments.md)
+shows how.
+
+## Sources
+
+[^psutil]:
+    psutil (BSD-3-Clause), `Process`: a process is identified by its pid
+    and its creation time, so a reused pid is not mistaken for the same
+    process.
+    <https://github.com/giampaolo/psutil>
+
+[^proc-stat]:
+    Linux manual page `proc_pid_stat(5)`: the process state (field 3) and
+    `starttime` (field 22).
+    <https://man7.org/linux/man-pages/man5/proc_pid_stat.5.html>