dbzero 0.1.12__tar.gz → 0.2.2__tar.gz

dbzero-0.2.2/AGENTS.md

@@ -0,0 +1,41 @@
+# dbzero
+
+dbzero is a state management system for persisting process state without a database. It lets Python processes keep their in-memory state durable across restarts — no separate DB server, schemas, or ORM. The core is C++ with Python bindings. See https://docs.dbzero.io for user-facing documentation.
+
+## Development workflow
+
+### TDD is required
+
+When implementing new features, follow test-driven development:
+
+1. Write a failing test first (Python tests in `python_tests/`, C++ tests under `tests/` / `subprojects/`).
+2. Implement the minimum code to make the test pass.
+3. Refactor while keeping tests green.
+
+All tests must pass before a change is considered complete.
+
+### Building
+
+- Debug build: `./docker/dbzero-build.sh`
+- Release build: `./docker/dbzero-build.sh -r`
+
+### Running tests
+
+- Python tests: `./scripts/run_tests.sh`
+- If any C++ source under the native/core part of the project was modified, also run the C++ test suite (do not rely on the Python tests alone to cover native changes).
+
+Never mark a task done while tests are failing.
+
+## Implementation notes
+
+### MorphingBIndex: address and type can change on mutation
+
+A `MorphingBIndex` does not behave like a typical container. On mutation (`insert`, `erase`) it may morph into a different internal storage variant (itty / array_2..4 / vector / bindex), and the morph can change both its **address** and its **type**.
+
+Consequences for any code that mutates a `MorphingBIndex`:
+
+- Any externally stored `{address, type}` pair referring to the bindex is potentially invalidated after every `insert` or `erase` call. Lookups through a stale pair read pre-mutation storage and return wrong data.
+- A live handle to the bindex remains valid across the mutation and reflects the new storage; prefer re-reading `bindex.getAddress()` / `bindex.getIndexType()` from the handle over trusting any previously captured copy.
+- Destructive shortcuts (destroying and rebuilding the whole bindex, or erasing it entirely from its parent) avoid the issue since no stale reference remains.
+
+When adding a new mutating path that operates on a `MorphingBIndex`, treat re-syncing any externally held `{address, type}` as mandatory, not an optimization. Collection-specific handling (where these pairs live, which paths must re-sync) is documented at the top of the relevant `.cpp` files.

{dbzero-0.1.12 → dbzero-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: dbzero
-Version: 0.1.12
+Version: 0.2.2
 Summary: A state management system for Python 3.x that unifies your applications business logic, data persistence, and caching into a single, efficient layer.
 Author-Email: "DBZero Software sp. z o.o" <info@dbzero.io>
 License:                   GNU LESSER GENERAL PUBLIC LICENSE

{dbzero-0.1.12 → dbzero-0.2.2}/dbzero/setup.py

@@ -10,7 +10,7 @@ if sys.byteorder == 'big':
 
 setup(
     name='dbzero',
-    version='0.1.12',
+    version='0.2.2',
     description='DBZero community edition',
     packages=['dbzero'],
     python_requires='>=3.9',

dbzero-0.2.2/docker/Dockerfile-claude

@@ -0,0 +1,88 @@
+FROM python:3.12-bullseye
+
+ARG NODE_MAJOR=20
+ARG CLAUDE_CODE_VERSION=latest
+ARG DEV_USER=claude
+ARG DEV_UID=1000
+ARG DEV_GID=1000
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV HOME=/home/${DEV_USER}
+ENV CLAUDE_CONFIG_DIR=/home/${DEV_USER}/.claude
+ENV PATH=/home/${DEV_USER}/.local/bin:${PATH}
+
+# Install development dependencies, Node.js 20, and Claude Code.
+RUN apt-get update && apt-get install -y \
+    ca-certificates \
+    cmake \
+    curl \
+    gdb \
+    gettext-base \
+    git \
+    gnupg \
+    jq \
+    less \
+    meson \
+    ninja-build \
+    psmisc \
+    python3-dbg \
+    python3-pip \
+    python3-venv \
+    ripgrep \
+    rsync \
+    screen \
+    unzip \
+    valgrind \
+    && mkdir -p /etc/apt/keyrings \
+    && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key \
+        | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
+    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" \
+        > /etc/apt/sources.list.d/nodesource.list \
+    && apt-get update && apt-get install -y nodejs \
+    && npm install -g "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" \
+    && node "$(npm root -g)/@anthropic-ai/claude-code/install.cjs" \
+    && npm cache clean --force \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN groupadd --gid ${DEV_GID} ${DEV_USER} \
+    && useradd --uid ${DEV_UID} --gid ${DEV_GID} --create-home --shell /bin/bash ${DEV_USER}
+
+# Install Python build tooling and project requirements.
+RUN python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel build
+
+COPY requirements.txt /usr/src/dbzero/
+RUN python3 -m pip install --no-cache-dir --upgrade -r /usr/src/dbzero/requirements.txt
+
+# Seed Claude Code defaults and make them available in interactive shells.
+COPY docker/claude-settings.json /opt/claude/settings.json
+COPY docker/claude-shell-init.sh /usr/local/bin/claude-shell-init
+COPY docker/dbzero-build.sh /usr/local/bin/dbzero-build
+COPY docker/dbzero-build-package.sh /usr/local/bin/dbzero-build-package
+RUN chmod +x /usr/local/bin/claude-shell-init \
+    && chmod +x /usr/local/bin/dbzero-build /usr/local/bin/dbzero-build-package \
+    && mkdir -p ${CLAUDE_CONFIG_DIR} /etc/profile.d \
+    && chmod 700 ${CLAUDE_CONFIG_DIR} \
+    && cp /opt/claude/settings.json ${CLAUDE_CONFIG_DIR}/settings.json \
+    && chmod 600 ${CLAUDE_CONFIG_DIR}/settings.json \
+    && chown -R ${DEV_UID}:${DEV_GID} ${HOME} /opt/claude \
+    && printf '%s\n' \
+        'export CLAUDE_CONFIG_DIR="${CLAUDE_CONFIG_DIR:-$HOME/.claude}"' \
+        > /etc/profile.d/claude-code.sh \
+    && printf '%s\n' \
+        'export CLAUDE_CONFIG_DIR="${CLAUDE_CONFIG_DIR:-$HOME/.claude}"' \
+        >> ${HOME}/.bashrc \
+    && claude --version
+
+RUN ulimit -c unlimited
+
+COPY --chown=${DEV_UID}:${DEV_GID} . /usr/src/dbzero
+WORKDIR /usr/src/dbzero
+
+RUN python3 scripts/generate_meson.py ./src/dbzero/ core
+RUN python3 scripts/generate_meson_tests.py tests/
+
+USER ${DEV_USER}
+
+ENTRYPOINT ["/usr/local/bin/claude-shell-init"]
+CMD ["/bin/bash"]

dbzero-0.2.2/docker/claude-settings.json

@@ -0,0 +1,8 @@
+{
+  "$schema": "https://json.schemastore.org/claude-code-settings.json",
+  "autoUpdatesChannel": "stable",
+  "defaultShell": "bash",
+  "env": {
+    "DISABLE_AUTOUPDATER": "1"
+  }
+}

dbzero-0.2.2/docker/claude-shell-init.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+CLAUDE_CONFIG_DIR="${CLAUDE_CONFIG_DIR:-$HOME/.claude}"
+export CLAUDE_CONFIG_DIR
+
+mkdir -p "$CLAUDE_CONFIG_DIR"
+chmod 700 "$CLAUDE_CONFIG_DIR"
+
+if [[ ! -f "$CLAUDE_CONFIG_DIR/settings.json" && -f /opt/claude/settings.json ]]; then
+    cp /opt/claude/settings.json "$CLAUDE_CONFIG_DIR/settings.json"
+    chmod 600 "$CLAUDE_CONFIG_DIR/settings.json"
+fi
+
+if [[ $# -eq 0 ]]; then
+    exec /bin/bash
+fi
+
+exec "$@"

dbzero-0.2.2/docker/dbzero-build-package.sh

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+show_help() {
+    echo "Build dbzero python package in the ./build directory and optionally install it"
+    echo "Use: dbzero-build-package [options]"
+    echo " -h, --help   Shows this help screen."
+    echo " --install    Install the package locally"
+    exit 0
+}
+
+if [[ ! -f setup.py || ! -d dbzero ]]; then
+    echo "Run dbzero-build-package from the repository root." >&2
+    exit 1
+fi
+
+install_package="false"
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        -h|--help) show_help ;;
+        --install) install_package="true" ; shift ;;
+        --) shift ; break ;;
+        *) break ;;
+    esac
+done
+
+pip_install_args=()
+if python3 -m pip install --help 2>/dev/null | grep -q -- "--break-system-packages"; then
+    pip_install_args+=(--break-system-packages)
+fi
+if [[ "$(id -u)" -ne 0 && -z "${VIRTUAL_ENV:-}" ]]; then
+    pip_install_args+=(--user)
+fi
+
+rm -rf ./.build
+mkdir -p ./.build/dbzero
+cp ./dbzero/* ./.build/dbzero
+cp setup.py ./.build/setup.py
+cp LICENSE ./.build/LICENSE
+cp README.md ./.build/README.md
+
+cd ./.build
+python3 setup.py sdist
+
+if [[ "$install_package" == "true" ]]; then
+    python3 -m pip install "${pip_install_args[@]}" "$(ls ./dist/*.tar.gz | sort | tail -n 1)"
+fi
+
+cd ..
+rm -rf ./.build

dbzero-0.2.2/docker/dbzero-build.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+show_help() {
+    echo "Builds dbzero for the Claude development container"
+    echo "Use: dbzero-build [options]"
+    echo " -h, --help                   Shows this help screen."
+    echo " -j, --jobs                   Threads number. Max by default."
+    echo " -r, --release                Compile as release. Note: debug build is by default."
+    echo " -s, --sanitize               Compile with sanitizers."
+    echo " -p, --prefix                 Install build under the specified prefix."
+    echo " -t, --tests                  Build tests."
+    echo " -e, --disable_debug_exceptions      Disable debug exceptions."
+    echo ""
+    exit 0
+}
+
+if [[ ! -f scripts/generate_meson.py || ! -d dbzero ]]; then
+    echo "Run dbzero-build from the repository root." >&2
+    exit 1
+fi
+
+export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH:-}:/usr/include/python3.9/"
+
+cores=$(grep -c ^processor /proc/cpuinfo)
+build_type="debug"
+sanitizer="false"
+enable_debug_exceptions="true"
+build_tests="false"
+
+default_prefix="/usr/local"
+if [[ "$(id -u)" -ne 0 ]]; then
+    default_prefix="$HOME/.local"
+fi
+install_prefix="${DBZERO_INSTALL_PREFIX:-$default_prefix}"
+
+temp=$(getopt -o hj:rtsep: --long help,jobs:,release,tests,sanitize,disable_debug_exceptions,prefix: -n 'dbzero-build' -- "$@")
+if [[ $? -ne 0 ]]; then
+    exit 1
+fi
+eval set -- "$temp"
+
+while true; do
+    case "$1" in
+        -h|--help) show_help ;;
+        -s|--sanitize) sanitizer="true" ; shift ;;
+        -r|--release) build_type="release" ; shift ;;
+        -t|--tests) build_tests="true" ; shift ;;
+        -e|--disable_debug_exceptions) enable_debug_exceptions="false" ; shift ;;
+        -p|--prefix) install_prefix="$2" ; shift 2 ;;
+        -j|--jobs) cores="$2" ; shift 2 ;;
+        --) shift ; break ;;
+        *) echo "Argument parsing error: $1" >&2 ; exit 1 ;;
+    esac
+done
+
+if [[ "$cores" -lt 1 ]]; then
+    echo "Argument parsing error: Wrong jobs number: $cores" >&2
+    exit 1
+fi
+
+python3 scripts/generate_meson.py ./src/dbzero/ core
+python3 scripts/generate_meson_tests.py tests/
+python3 scripts/generate_meson_dbzero.py dbzero/
+
+mkdir -p build
+
+build_dir="build/debug"
+if [[ "$build_type" == "release" ]]; then
+    build_dir="build/release"
+fi
+
+options=(
+    "-Denable_debug_exceptions=$enable_debug_exceptions"
+    "-Denable_sanitizers=$sanitizer"
+    "-Dbuild_tests=$build_tests"
+)
+
+if [[ -f "$build_dir/meson-private/coredata.dat" ]]; then
+    meson setup --reconfigure --prefix="$install_prefix" --buildtype="$build_type" "${options[@]}" "$build_dir"
+else
+    meson setup --prefix="$install_prefix" --buildtype="$build_type" "${options[@]}" "$build_dir"
+fi
+
+ninja -C "$build_dir" -j "$cores"
+meson install -C "$build_dir"
+
+cd dbzero
+envsubst < dbzero/dbzero.template > dbzero/dbzero.py
+dbzero-build-package --install
+echo "$build_type"

{dbzero-0.1.12 → dbzero-0.2.2}/pyproject.toml

@@ -4,7 +4,7 @@ requires = ['meson-python']
 
 [project]
 name = 'dbzero'
-version = '0.1.12'
+version = '0.2.2'
 description = 'A state management system for Python 3.x that unifies your applications business logic, data persistence, and caching into a single, efficient layer.'
 readme = 'README.md'
 requires-python = '>=3.9'

{dbzero-0.1.12 → dbzero-0.2.2}/python_tests/test_auto_weak_proxy.py

@@ -53,7 +53,6 @@ def test_auto_wrap_expires_when_source_deleted(db0_fixture):
         _ = obj_2.value.value
 
 
-
 def test_auto_wrap_on_cross_prefix_assignment(db0_fixture):
     """Cross-prefix assignment is silently wrapped as weak_proxy by default."""
     px_1 = db0.get_current_prefix().name

dbzero-0.2.2/python_tests/test_cache.py

@@ -0,0 +1,189 @@
+# SPDX-License-Identifier: LGPL-2.1-or-later
+# Copyright (c) 2025 DBZero Software sp. z o.o.
+
+import pytest
+import multiprocessing
+import dbzero as db0
+from random import randint
+from .conftest import DB0_DIR
+from .memo_test_types import MemoTestClass, MemoTestSingleton
+    
+
+def get_string(str_len):
+    return 'a' * str_len    
+    
+
+def rand_array(max_bytes):
+    import random
+    actual_len = random.randint(1, max_bytes)
+    return [1] * (int(actual_len / 8) + 1)
+
+    
+def test_cache_size_can_be_updated_at_runtime(db0_fixture):
+    cache_0 = db0.get_cache_stats()    
+    # create object instances to populate cache
+    buf = []
+    for _ in range(1000):
+        buf.append(MemoTestClass(get_string(1024)))
+    cache_1 = db0.get_cache_stats()    
+    diff_1 = cache_1["size"] - cache_0["size"]
+    # reduce cache size so that only 1/2 of objects can fit
+    db0.set_cache_size(512 * 1024)
+    cache_2 = db0.get_cache_stats()    
+    # make sure cache size / capacity was adjusted with at least 95% accuracy
+    assert abs(1.0 - (512 * 1024) / cache_2["size"]) < 0.05
+    assert abs(1.0 - cache_2["capacity"] / cache_2["size"]) < 0.05
+    
+    
+def test_base_lock_usage_does_not_exceed_limits(db0_fixture):
+    # run this test only in debug mode
+    if 'D' in db0.build_flags():
+        append_count = 200
+        cache_size = 1024 * 1024
+        db0.set_cache_size(cache_size)
+        usage_1 = db0.get_base_lock_usage()[0]
+        for _ in range(append_count):
+            _ = MemoTestClass(rand_array(16384))
+        db0.clear_cache()
+        import gc
+        gc.collect()        
+        usage_2 = db0.get_base_lock_usage()[0]
+        print("usage diff = ", usage_2 - usage_1)
+        assert usage_2 - usage_1 < cache_size * 1.5
+
+
+def test_lang_cache_can_reach_capacity(db0_fixture):
+    buf = db0.list()
+    # python instances are added to lang cache until it reaches capacity
+    initial_capacity = db0.get_lang_cache_stats()["capacity"]    
+    for _ in range(initial_capacity * 2):        
+        buf.append(MemoTestClass(123))
+    # here we call commit to flush from internal buffers (e.g. TagIndex)
+    db0.commit()            
+    # capacity not changed
+    assert db0.get_lang_cache_stats()["capacity"] == initial_capacity
+    # capacity might be exceeded due to indeterministic gc collection by Python
+    assert db0.get_lang_cache_stats()["size"] < initial_capacity * 2
+
+
+# --- Writer process used by the stale-instance repro below.
+# The writer cycles through: create obj, commit, (await reader), delete obj,
+# create a replacement obj (likely reusing the same physical slot), commit.
+def _writer_create_delete_create(prefix_name, req_queue, resp_queue):
+    db0.init(DB0_DIR)
+    db0.open(prefix_name, "rw")
+    # singleton holds a strong reference so we control exactly when the
+    # writer-side instance is dropped
+    root = MemoTestSingleton(None)
+    while True:
+        cmd = req_queue.get()
+        if cmd is None:
+            break
+        action = cmd[0]
+        if action == "create":
+            value = cmd[1]
+            obj = MemoTestClass(value)
+            root.value = obj
+            uuid = db0.uuid(obj)
+            del obj
+            db0.commit()
+            resp_queue.put(uuid)
+        elif action == "delete":
+            # drop the only strong reference and commit so the object
+            # and its slab slot are freed on disk before the next create
+            root.value = None
+            db0.commit()
+            resp_queue.put("ok")
+    db0.close()
+
+
+def test_lang_cache_returns_stale_instance_after_slot_reuse(db0_fixture):
+    """
+    Reproduces a lang-cache coherence bug on read-only clients.
+
+    Scenario: a writer creates obj1, the reader fetches it into its
+    LangCache. The writer then deletes obj1 and creates obj2, which is
+    likely allocated at obj1's recycled physical address. After refresh,
+    the reader fetches obj2 by its new UUID; the client-side LangCache
+    is keyed by (fixture_id, address) - no instance_id - so it returns
+    the stale wrapper that was bound to obj1's (now recycled) slot.
+
+    The test records UUIDs only - it intentionally does not keep Python
+    references to fetched wrappers. Holding a stale wrapper around
+    exposes a separate "stale instance access" class of bugs which is
+    out of scope here; dropping the wrapper after each fetch isolates
+    the LangCache coherence issue. The cache entry itself survives the
+    wrapper being dropped (it holds its own reference via
+    ObjectSharedExtPtr) until something explicitly clears it.
+    """
+    prefix_name = db0.get_current_prefix().name
+    # make sure the prefix exists on disk before the reader opens it read-only
+    db0.commit()
+    db0.close()
+
+    req_queue = multiprocessing.Queue()
+    resp_queue = multiprocessing.Queue()
+    writer = multiprocessing.Process(
+        target=_writer_create_delete_create,
+        args=(prefix_name, req_queue, resp_queue),
+    )
+    writer.start()
+
+    # Record (uuid, expected_value) pairs seen across all iterations so we
+    # can also validate post-loop that fetching any historical uuid either
+    # succeeds with the right value or fails cleanly - never silently
+    # returns a stale/wrong instance.
+    seen = []
+    try:
+        db0.init(DB0_DIR)
+        db0.open(prefix_name, "r")
+        iterations = 50
+        for i in range(iterations):
+            first_value = i
+            second_value = 100000 + i
+
+            # writer: create obj1 (instance 0)
+            req_queue.put(("create", first_value))
+            uuid1 = resp_queue.get(timeout=30)
+
+            db0.refresh()
+            obj1 = db0.fetch(uuid1)
+            assert obj1.value == first_value, \
+                f"iter {i}: unexpected obj1.value={obj1.value}"
+            assert db0.uuid(obj1) == uuid1, \
+                f"iter {i}: uuid1 roundtrip failed"
+            # intentionally drop the wrapper immediately - we only retain
+            # the uuid, not the Python instance
+            del obj1
+
+            # writer: delete obj1 and commit
+            req_queue.put(("delete",))
+            assert resp_queue.get(timeout=30) == "ok"
+
+            # writer: create obj2 (instance 1) - likely reuses obj1's slot
+            req_queue.put(("create", second_value))
+            uuid2 = resp_queue.get(timeout=30)
+            assert uuid1 != uuid2
+
+            db0.refresh()
+            obj2 = db0.fetch(uuid2)
+            # If the LangCache wrongly served the stale entry for obj1's
+            # address, db0.uuid(obj2) will not match the uuid we just
+            # resolved through, and obj2.value may report obj1's value.
+            assert db0.uuid(obj2) == uuid2, (
+                f"iter {i}: uuid mismatch - lang cache served stale wrapper "
+                f"(expected {uuid2}, got {db0.uuid(obj2)})"
+            )
+            assert obj2.value == second_value, (
+                f"iter {i}: lang cache returned stale instance "
+                f"(expected {second_value}, got {obj2.value}); "
+                f"uuid1={uuid1} uuid2={uuid2}"
+            )
+            seen.append((uuid2, second_value))
+            del obj2
+    finally:
+        req_queue.put(None)
+        writer.join(timeout=30)
+        if writer.is_alive():
+            writer.terminate()
+            writer.join()

{dbzero-0.1.12 → dbzero-0.2.2}/python_tests/test_set.py

@@ -658,3 +658,36 @@ def test_db0_set_iterator_type_valid(db0_fixture):
     s = db0.set()
     it = iter(s)
     assert type(type(it)) is type
+
+
+def test_db0_set_remove_with_hash_collision(db0_fixture):
+    # The TUPLE hash in PyHash.cpp is xor-of-element-hashes, so any
+    # two permutations of the same elements collide. That puts them
+    # in the same m_index bucket, which is the only path that exercises
+    # Set::remove's bindex.erase(*it) branch (size > 1).
+    #
+    # If that branch fails to re-sync m_index after the underlying
+    # bindex morphs to a smaller container, the bucket's stale
+    # {address, type} will keep reporting pre-erase data on later
+    # lookups — wrong len(), wrong `in`, stale iteration.
+    a = (1, 2)
+    b = (2, 1)
+
+    s = db0.set()
+    s.add(a)
+    s.add(b)
+    assert len(s) == 2
+    assert a in s
+    assert b in s
+
+    s.remove(a)
+    assert a not in s
+    assert b in s
+    assert len(s) == 1
+    # Iteration should see only b.
+    assert list(s) == [b]
+
+    # Removing the last colliding element must fully clean the bucket.
+    s.remove(b)
+    assert b not in s
+    assert len(s) == 0

{dbzero-0.1.12 → dbzero-0.2.2}/python_tests/test_weak_refs.py

@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: LGPL-2.1-or-later
 # Copyright (c) 2025 DBZero Software sp. z o.o.
 
+import random
+
 import pytest
 import dbzero as db0
 from .memo_test_types import MemoTestPxClass
@@ -178,7 +180,7 @@ def test_long_weak_ref_inside_set(db0_fixture):
     obj_1 = MemoTestPxClass(123, prefix=px_1)
     set_1 = db0.set([db0.weak_proxy(obj_1)])
     for obj in set_1:
-        assert obj == obj_1    
+        assert obj == obj_1
 
 
 def test_long_weak_ref_inside_dict(db0_fixture):