screamer 3.0.2__tar.gz → 3.2.0__tar.gz

{screamer-3.0.2 → screamer-3.2.0}/CHANGELOG.md

@@ -5,6 +5,36 @@ All notable changes to this project are documented in this file.
 Unreleased
 ----------
 
+3.2.0 - 2026-10-14
+------------------
+
+### Changed
+
+* `segment_shuffle`, and `position_test` / `positionTest` with
+  `method="segments"`, never place two runs of the same value next to each
+  other. Before, two such runs could merge into one longer trade, so a
+  shuffled copy could trade less and pay less cost than the real strategy,
+  which pushed the p-value up whenever the backtest had costs: on a long/short
+  signal with no flat stretches the copies made about half the trades. Every
+  copy now changes position as often as the original. Results for
+  `method="segments"` change for any seed; there is no way to get the old
+  shuffle, which was biased.
+
+### Fixed
+
+* The built-in help (`help.json`) for `VPIN`, `Dropna`, `Resample` and `split`
+  is up to date with their documentation pages again.
+
+3.1.0 - 2026-10-14
+------------------
+
+### Changed
+
+* screamer is built with nanobind 3.1.0 (was 2.13.0). Results and timings are
+  unchanged. The local nanobind patch now also fixes a crash when a Python
+  class that subclasses a screamer operator sets a `__module__` that UTF-8
+  cannot encode.
+
 3.0.2 - 2026-10-14
 ------------------
 

{screamer-3.0.2 → screamer-3.2.0}/CMakeLists.txt

@@ -1,5 +1,5 @@
 cmake_minimum_required(VERSION 3.18)
-project(screamer VERSION 3.0.2 LANGUAGES CXX)
+project(screamer VERSION 3.2.0 LANGUAGES CXX)
 
 # Set the default build type to Release for optimizations
 if(NOT CMAKE_BUILD_TYPE)
@@ -47,24 +47,26 @@ set(LOCAL_CACHE_DIR ${CMAKE_SOURCE_DIR}/_local_deps)
 
 # nanobind replaces pybind11 (migration 2026-08). nanobind has STABLE_ABI from
 # Python 3.12+, which collapses the release wheel matrix to one abi3 wheel per
-# platform. v2.13.0 (bumped from v2.9.2 for 3.0.0) fixes crashes on an object
-# whose class has a non-str __module__. See
-# docs/superpowers/plans/2026-08-01-nanobind-migration.md.
+# platform. v3.1.0 (bumped from v2.13.0) brings faster attribute lookups and
+# function dispatch. See docs/superpowers/plans/2026-08-01-nanobind-migration.md.
 #
 # Pinned by commit, with a local patch, cmake/nanobind-surrogate-module.patch,
-# for two crashes of v2.13.0 and nanobind master that are not yet fixed upstream
-# (report both; remove each hunk when its fix lands):
-#  1. nb_type_name() and nb_func_render_signature() tolerate a __module__ that
-#     is not valid UTF-8 (for example '\udc80').
-#  2. make_mt_from_buffer_protocol() (nb_ndarray.cpp) divides view->strides[i]
-#     without a NULL check. A ctypes array and a NumPy datetime64/timedelta64
-#     scalar export a buffer with strides == NULL, so any nb::ndarray argument
-#     given one segfaulted. The patch reads NULL strides as C-contiguous.
-#     screamer also refuses those values before any ndarray cast (TypeError,
-#     cast_double.h is_foreign_value).
+# for crashes of v3.1.0 that are fixed in open upstream PRs (remove each hunk
+# when its fix is released):
+#  1. wjakob/nanobind#1455: an error message holding a str that UTF-8 cannot
+#     encode (a lone surrogate such as '\udc80', from a class's __module__ or an
+#     exception message) segfaulted. Buffer::put_pystr() escapes it instead.
+#     The patch also uses it in nb_type_init() (nb_type.cpp), where subclassing
+#     a bound type with such a __module__ segfaulted; not yet reported upstream.
+#  2. wjakob/nanobind#1454: make_mt_from_buffer_protocol() (nb_ndarray.cpp) read
+#     view->strides[i] without a NULL check. A ctypes array and a NumPy
+#     datetime64/timedelta64 scalar export a buffer with strides == NULL, so any
+#     nb::ndarray argument given one segfaulted. The patch reads NULL strides as
+#     C-contiguous. screamer also refuses those values before any ndarray cast
+#     (TypeError, cast_double.h is_foreign_value).
 # A version bump must re-check that the patch still applies (configure fails
 # loudly when it does not).
-set(SCREAMER_NANOBIND_VERSION "2.13.0")
+set(SCREAMER_NANOBIND_VERSION "3.1.0")
 set(SCREAMER_NANOBIND_PATCH "${CMAKE_SOURCE_DIR}/cmake/nanobind-surrogate-module.patch")
 
 if(EXISTS "${LOCAL_CACHE_DIR}/nanobind")
@@ -92,7 +94,7 @@ else()
     FetchContent_Declare(
         nanobind
         GIT_REPOSITORY      https://github.com/wjakob/nanobind.git
-        GIT_TAG             e2dc00f7a34f935c6cf91948776d59c4709e9fe6  # v2.13.0
+        GIT_TAG             68480a9e6883bb1bf65e1925423321e2a89d07f9  # v3.1.0
         UPDATE_DISCONNECTED ON
         PATCH_COMMAND       "${CMAKE_COMMAND}" -DSRC=<SOURCE_DIR>
                             -DPATCH=${SCREAMER_NANOBIND_PATCH}

{screamer-3.0.2 → screamer-3.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: screamer
-Version: 3.0.2
+Version: 3.2.0
 Summary: Screamingly fast financial technical indicators with C++ performance and Python simplicity, built for both time series analysis and real-time, event-driven streaming.
 Keywords: streaming,indicators,real-time,algorithmic,trading,low-latency
 Author-Email: Thijs van den Berg <thijs@sitmo.com>

{screamer-3.0.2 → screamer-3.2.0}/bindings/bindings_fin.cpp

@@ -74,7 +74,7 @@ namespace nanobind { namespace detail {
 template <> struct type_caster<screamer::PerAsset> {
     NB_TYPE_CASTER(screamer::PerAsset, const_name("float | Sequence[float]"))
 
-    bool from_python(handle src, uint8_t flags, cleanup_list* cleanup) noexcept {
+    bool from_python(handle src, uint32_t flags, cleanup_list* cleanup) noexcept {
         if (!PySequence_Check(src.ptr())) {
             if (screamer::bool_kind(src.ptr())) return false;   // a bool is not a number setting
             const double d = PyFloat_AsDouble(src.ptr());
@@ -1173,7 +1173,8 @@ void init_bindings_fin(nb::module_& m) {
         nb::list out;
         for (const auto& st : sv::kStats)
             out.append(nb::make_tuple(st.name, sv::stat_source_name(st.source), st.column,
-                                      st.larger < 0 ? nb::none() : nb::bool_(st.larger == 1)));
+                                      st.larger < 0 ? nb::object(nb::none())
+                                                     : nb::object(nb::bool_(st.larger == 1))));
         return out;
     });
     m.def("_backtest_report", &backtest_report_py, "values"_a, "periods_per_year"_a.none());

screamer-3.2.0/cmake/nanobind-surrogate-module.patch

@@ -0,0 +1,135 @@
+diff --git a/src/buffer.h b/src/buffer.h
+index f574826..e48a6cd 100644
+--- a/src/buffer.h
++++ b/src/buffer.h
+@@ -47,6 +47,26 @@ public:
+     /// Append a dynamic string
+     void put_dstr(const char *str) { put(str, strlen(str)); }
+ 
++    /// Append a Python 'str'. One that UTF-8 cannot encode (e.g. holding a
++    /// lone surrogate) is appended with backslash escapes instead.
++    void put_pystr(PyObject *o) {
++        Py_ssize_t size;
++        const char *s = PyUnicode_AsUTF8AndSize(o, &size);
++        if (s) {
++            put(s, (size_t) size);
++            return;
++        }
++        PyErr_Clear();
++        PyObject *b = PyUnicode_AsEncodedString(o, "utf-8", "backslashreplace");
++        if (b) {
++            put_dstr(PyBytes_AsString(b));
++            Py_DECREF(b);
++        } else {
++            PyErr_Clear();
++            put("<unencodable str>");
++        }
++    }
++
+     /// Append a single character to the buffer
+     void put(char c) {
+         if (m_cur + 1 >= m_end)
+diff --git a/src/error.cpp b/src/error.cpp
+index bd2f3f0..498a902 100644
+--- a/src/error.cpp
++++ b/src/error.cpp
+@@ -166,14 +166,14 @@ const char *error_what(error_payload *p) noexcept {
+         try {
+             object name = steal(raise_if_null(
+                 PyObject_GetAttrString(exc_type.ptr(), "__name__")));
+-            exc_buf.put_dstr(borrow<str>(name).c_str());
++            exc_buf.put_pystr(name.ptr());
+             exc_buf.put(": ");
+         } catch (...) { PyErr_Clear(); }
+     }
+ 
+     if (exc_value.is_valid()) {
+         try {
+-            exc_buf.put_dstr(str(exc_value).c_str());
++            exc_buf.put_pystr(str(exc_value).ptr());
+         } catch (...) {
+             PyErr_Clear();
+             exc_buf.put("<exception str() failed>");
+diff --git a/src/nb_func.cpp b/src/nb_func.cpp
+index 5a98252..fa71f8b 100644
+--- a/src/nb_func.cpp
++++ b/src/nb_func.cpp
+@@ -574,7 +574,7 @@ nb_func_error_overload(PyObject *self, PyObject *const *args_in,
+     buf.put("\nInvoked with types: ");
+     for (size_t i = 0; i < nargs_in; ++i) {
+         str name = steal<str>(nb_inst_name(args_in[i]));
+-        buf.put_dstr(name.c_str());
++        buf.put_pystr(name.ptr());
+         if (i + 1 < nargs_in)
+             buf.put(", ");
+     }
+@@ -597,7 +597,7 @@ nb_func_error_overload(PyObject *self, PyObject *const *args_in,
+             buf.put_dstr(key_cstr);
+             buf.put(": ");
+             str name = steal<str>(nb_inst_name(value));
+-            buf.put_dstr(name.c_str());
++            buf.put_pystr(name.ptr());
+             buf.put(", ");
+         }
+         buf.rewind(2);
+diff --git a/src/nb_ndarray.cpp b/src/nb_ndarray.cpp
+index 521a725..2fc51fa 100644
+--- a/src/nb_ndarray.cpp
++++ b/src/nb_ndarray.cpp
+@@ -550,14 +550,23 @@ static mt_unique_ptr_t make_mt_from_buffer_protocol(PyObject *o, bool ro) {
+     mt->dltensor.byte_offset = 0UL;
+ 
+     const int64_t itemsize = (int64_t) view->itemsize;
+-    for (int32_t i = 0; i < ndim; ++i) {
+-        int64_t stride = view->strides[i] / itemsize;
+-        if (stride * itemsize != view->strides[i]) {
+-            PyBuffer_Release(view.get());
+-            return mt_unique_ptr;
+-        }
+-        strides[i] = stride;
++    // An exporter may leave 'strides' NULL even though PyBUF_STRIDES was
++    // requested (e.g. ctypes arrays, NumPy scalars). The buffer is then
++    // C-contiguous.
++    int64_t contig_stride = 1;
++    for (int32_t i = ndim - 1; i >= 0; --i) {
+         shape[i] = (int64_t) view->shape[i];
++        if (view->strides) {
++            int64_t stride = view->strides[i] / itemsize;
++            if (stride * itemsize != view->strides[i]) {
++                PyBuffer_Release(view.get());
++                return mt_unique_ptr;
++            }
++            strides[i] = stride;
++        } else {
++            strides[i] = contig_stride;
++            contig_stride *= shape[i];
++        }
+     }
+ 
+     mt->version = {dlpack::major_version, dlpack::minor_version};
+diff --git a/src/nb_type.cpp b/src/nb_type.cpp
+index 04fe8f8..e6893ca 100644
+--- a/src/nb_type.cpp
++++ b/src/nb_type.cpp
+@@ -8,6 +8,7 @@
+ */
+ 
+ #include "nb_internals.h"
++#include "buffer.h"
+ #include "nb_ft.h"
+ 
+ 
+@@ -741,8 +742,12 @@ static NB_NOINLINE int nb_type_init_py(PyTypeObject *self) {
+         return -1;
+     }
+ 
++    // The name may hold a str that UTF-8 cannot encode (from a hostile
++    // '__module__'); put_pystr() escapes it
+     PyObject *name_o = nb_type_name((PyObject *) self);
+-    char *name = strdup_check(PyUnicode_AsUTF8AndSize(name_o, nullptr));
++    Buffer name_buf(64);
++    name_buf.put_pystr(name_o);
++    char *name = strdup_check(name_buf.get());
+     Py_DECREF(name_o);
+ 
+     // Assemble the record locally, it is published as a whole below

{screamer-3.0.2 → screamer-3.2.0}/docs/backtesting_best_practices.md

@@ -174,6 +174,20 @@ The recipe [Is my backtest significant?](recipes/significance-test.md) runs
 `position_test` on a real tape and on a market with built-in momentum, and
 computes `probabilistic_sharpe` for the result.
 
+Which test answers which question:
+
+| Question | Test | Randomizes | Reruns the strategy | Example |
+|---|---|---|---|---|
+| Does the timing beat random timing with the same trades? | `position_test` | when the trades happen | no | [Is my strategy's timing real?](notebooks/20-is-the-timing-real.ipynb) |
+| Does the best of several variants survive the choice? | `position_test` with a `(T, K)` target, `family_p_value` | all variants' timing together | no | [Is my backtest significant?](recipes/significance-test.md) |
+| Does a tuned or fitted strategy beat the same procedure on random markets? | `strategy_test`, `method="bars"` or `"returns"` | the market's bars or returns | yes | [I tuned my parameters. Is the result real?](notebooks/21-is-the-tuned-result-real.ipynb) |
+| Does a feature carry information? | `strategy_test`, `method="shift"` | the features' alignment with the market | yes | [Does my feature carry information?](notebooks/22-does-the-feature-carry-information.ipynb) |
+| Is the Sharpe ratio above zero, or above the best of the variants tried? | `probabilistic_sharpe`, `deflated_sharpe` | nothing (analytic) | no | [Is my backtest significant?](recipes/significance-test.md) |
+
+`position_test` keeps the strategy's exposure in every randomized run, so a
+long-biased strategy in a rising market is tested on its timing only, not on
+the trend it rides.
+
 ## Selection
 
 If you try many variants (parameters, signals, filters) and report the best,

{screamer-3.0.2 → screamer-3.2.0}/docs/functions_fin/position_test.md

@@ -148,6 +148,11 @@ The test checks one fixed target series. If the strategy was fitted, tuned, or
 selected on the same data, that choice also inflates the result, and this test
 does not see it. [`strategy_test`](strategy_test.md) re-runs the whole strategy
 on permuted market data and covers it.
+The notebook
+[I tuned my parameters. Is the result real?](../notebooks/21-is-the-tuned-result-real.ipynb)
+shows the difference on a strategy tuned on a random walk, and
+[Is my strategy's timing real?](../notebooks/20-is-the-timing-real.ipynb) is a
+short worked example of this test.
 
 ## Examples
 

{screamer-3.0.2 → screamer-3.2.0}/docs/functions_fin/segment_shuffle.md

@@ -20,14 +20,20 @@ topics:
 # `segment_shuffle`
 
 Returns `n` copies of a series as a `(T, n)` array. The series is split into
-runs of equal value, and each copy writes the runs back in a random order. For a
-target-position series, each trade and each flat stretch is one run, so every
-copy holds exactly the same trades, with the same sides, sizes, and holding
-periods, at random times.
+runs of equal value, and each copy writes the runs back in a random order in
+which no two neighbouring runs have the same value. For a target-position
+series, each trade and each flat stretch is one run, so every copy holds exactly
+the same trades, with the same sides, sizes, and holding periods, at random
+times, and changes position as often as the original.
+
+Two neighbouring runs of the same value would merge into one longer trade, and
+a copy with fewer trades pays less cost, which would bias a test with costs.
+When the series alternates between two values with nothing in between, such as
+a long/short signal with no flat stretches, the runs can only alternate, so the
+copies keep the alternation and move the run lengths.
 
 The leading NaN warmup stays in place. A NaN inside the series is a run of its
-own, like any other value. Two runs with the same value can end up next to each
-other and then read as one longer run.
+own, like any other value.
 
 The function is batch-only and runs in C++. Each column has its own generator,
 seeded from `seed` and the column number, so the same seed gives the same

{screamer-3.0.2 → screamer-3.2.0}/docs/functions_fin/strategy_test.md

@@ -105,6 +105,12 @@ distributions and scatter plots as shown in
 `periods_per_year` annualizes the ratios. The same `seed` gives the same
 randomized inputs in Python and JavaScript.
 
+Worked examples:
+[I tuned my parameters. Is the result real?](../notebooks/21-is-the-tuned-result-real.ipynb)
+(`method="bars"`) and
+[Does my feature carry information?](../notebooks/22-does-the-feature-carry-information.ipynb)
+(`method="shift"`).
+
 ## Examples
 
 A trend-following strategy on two markets: one whose returns have momentum, and

screamer-3.2.0/docs/notebooks/20-is-the-timing-real.ipynb

@@ -0,0 +1,155 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "0acd5223",
+   "metadata": {},
+   "source": [
+    "# Is my strategy's timing real?\n",
+    "\n",
+    "[`position_test`](../functions_fin/position_test.md) keeps your strategy's\n",
+    "trades and moves them to random times, then reruns each copy through the same\n",
+    "backtest engine. The p-value is the share of random timings that do at least as\n",
+    "well as yours. A small p-value says the timing carries information.\n",
+    "\n",
+    "The data: one-minute bars of six hours of BTC perpetual trades."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ba4e0928",
+   "metadata": {
+    "tags": [
+     "hide-input"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "import matplotlib.pyplot as plt\n",
+    "from screamer import (Resample, EwMean, LogReturn, Sign, FillNa, BacktestOHLCTarget,\n",
+    "                      position_test)\n",
+    "\n",
+    "trades = pd.read_csv(\"data/deribit_btc_perp_6h.csv\")\n",
+    "bars, _ = Resample(freq=60_000, agg=\"ohlc\")(trades[\"price\"].to_numpy(),\n",
+    "                                             trades[\"timestamp\"].to_numpy())\n",
+    "o, h, l, c = bars.T"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a100ffb7",
+   "metadata": {},
+   "source": [
+    "## Test a strategy"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1fbe4c41",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "signal = Sign()(FillNa(0.0)(EwMean(span=10)(LogReturn()(c))))   # follow the recent trend\n",
+    "engine = BacktestOHLCTarget(taker_fee=5e-4)\n",
+    "\n",
+    "res = position_test(engine, signal, o, h, l, c, n=999)\n",
+    "print(f\"Sharpe per bar {res.observed:+.3f}, p = {res.p_value:.3f}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c3796e1a",
+   "metadata": {},
+   "source": [
+    "## Look at every statistic\n",
+    "\n",
+    "The p-value covers one statistic, the Sharpe ratio. `res.null_stats` holds every\n",
+    "[`backtest_report`](../functions_fin/backtest_report.md) statistic of each\n",
+    "random timing, and `res.observed_stats` those of the real run, so you can place\n",
+    "the real run on two statistics at once. Each dot is one random timing; the star\n",
+    "is your strategy."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a115e3b7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "null, real = res.null_stats, res.observed_stats\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(10, 3.5))\n",
+    "for ax, (x, y) in zip(axes, [(\"sharpe\", \"max_drawdown\"), (\"total_pnl\", \"win_rate\")]):\n",
+    "    ax.scatter(null[x], null[y], s=6, color=\"lightsteelblue\", label=\"random timing\")\n",
+    "    ax.scatter(real[x], real[y], s=250, marker=\"*\", color=\"crimson\", label=\"your strategy\")\n",
+    "    ax.set_xlabel(x); ax.set_ylabel(y)\n",
+    "axes[0].legend(fontsize=8)\n",
+    "plt.tight_layout()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8aec7968",
+   "metadata": {},
+   "source": [
+    "The star sits inside the cloud on both views: six hours of data give no\n",
+    "evidence that this trend rule's timing beats chance. The\n",
+    "[significance recipe](../recipes/significance-test.md) runs the same test on a\n",
+    "market where the rule has an edge.\n",
+    "\n",
+    "## Shift or segments\n",
+    "\n",
+    "`method=\"shift\"` (the default) rotates the whole position series by a random\n",
+    "offset, so the trades keep their order and clustering. `method=\"segments\"`\n",
+    "shuffles the order of the trades, which also breaks their clustering. Use it to\n",
+    "check that the result does not hinge on a few trades in one favourable\n",
+    "stretch."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "98ad65c2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for method in (\"shift\", \"segments\"):\n",
+    "    p = position_test(engine, signal, o, h, l, c, n=999, method=method).p_value\n",
+    "    print(f\"{method:8s} p = {p:.3f}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "95682df5",
+   "metadata": {},
+   "source": [
+    "## Adapt it\n",
+    "\n",
+    "- Pass your own target series and engine: `position_test(engine, target, *market)`\n",
+    "  takes the same arguments as `engine(target, *market)`.\n",
+    "- Choose the statistic with `metric=` (`\"total_pnl\"`, `\"sortino\"`,\n",
+    "  `\"max_drawdown\"`, ...) and annualize ratios with `periods_per_year=`.\n",
+    "- Tried several variants? Pass them as one `(T, K)` target and read\n",
+    "  `family_p_value`.\n",
+    "- Tuned the strategy on this data? `position_test` cannot see the tuning; see\n",
+    "  [I tuned my parameters. Is the result real?](21-is-the-tuned-result-real.ipynb)."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}