thermospike 0.1.0__tar.gz

thermospike-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 The thermospike authors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

thermospike-0.1.0/PKG-INFO

@@ -0,0 +1,107 @@
+Metadata-Version: 2.4
+Name: thermospike
+Version: 0.1.0
+Summary: Stochastic spiking neurons that sample from Boltzmann distributions, with a count of the random bits and synaptic events each sample costs.
+Author: The thermospike contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/tlcdv/thermospike
+Project-URL: Source, https://github.com/tlcdv/thermospike
+Project-URL: Issues, https://github.com/tlcdv/thermospike/issues
+Keywords: thermodynamic computing,neuromorphic,spiking neural network,p-bit,neural sampling,boltzmann machine
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Dynamic: license-file
+
+# thermospike
+
+Stochastic spiking neurons that sample from Boltzmann distributions. The library counts what each sample costs in random bits, synaptic events and multiply adds.
+
+thermospike sits between two fields. Thermodynamic and probabilistic computing use physical noise to sample (p-bits, Gibbs samplers). Neuromorphic engineering uses spikes and event driven synapses. Neural sampling theory (Buesing, Bill, Nessler and Maass, 2011) shows that a network of stochastic spiking neurons with a refractory period samples from the same Boltzmann distribution as a p-bit network. thermospike implements both and measures the difference in cost.
+
+## Status
+
+Version 0.1.0. Small networks in pure Python and NumPy. Every sampler is checked against the exact distribution, found by enumerating every state. An FPGA reference design is planned and does not exist yet.
+
+## Install
+
+```
+pip install thermospike
+```
+
+From source:
+
+```
+git clone https://github.com/tlcdv/thermospike
+cd thermospike
+pip install -e ".[test]"
+pytest
+```
+
+## Example
+
+```python
+from thermospike import SpikingSampler, NumpyNoise, and_gate, empirical_distribution, exact_distribution, kl_divergence
+
+model = and_gate()                     # C = A AND B, as an energy function
+clamp = {2: 1}                         # fix the output C to 1 and run the gate backwards
+sampler = SpikingSampler(model, NumpyNoise(seed=0), tau=4, clamp=clamp)
+samples = sampler.sample(20000)
+
+print(samples.mean(axis=0))            # A and B are near 1
+print(kl_divergence(empirical_distribution(samples), exact_distribution(model, clamp=clamp)))
+print(sampler.counters)                # random bits, spikes, synaptic events, multiply adds
+```
+
+## What is in the library
+
+| Part | Contents |
+|---|---|
+| `BoltzmannMachine` | Binary neurons (0 silent, 1 active), symmetric weights, biases |
+| `PBitSampler` | Sequential Gibbs sampling, one p-bit per neuron |
+| `SpikingSampler` | Neural sampling with an absolute refractory period of `tau` steps. With `tau=1` it is the p-bit sampler |
+| `NumpyNoise`, `LFSRNoise` | Software noise, and a 32 bit LFSR model of the noise a small FPGA would use |
+| `Counters` | Updates, random draws, random bits, spikes, state changes, synaptic events, dense multiply adds |
+| `exact_distribution`, `kl_divergence` | Checks against enumeration for up to 20 neurons |
+| `effective_sample_size`, `integrated_autocorrelation_time` | Mixing diagnostics, so costs are compared per independent sample |
+| `and_gate`, `max_cut`, `random_machine` | Example problems |
+
+## First measurement
+
+`python examples/compare_cost.py --neurons 8 --sweeps 40000 --seeds 0 1 2` on one random 8 neuron machine (model seed 0, weight scale 1) gave this table. KL is KL(empirical || exact) in nats. With 256 states and 40000 samples, finite sampling alone gives about 0.003, so every row sits near that floor. ESS is the effective sample size, the worst case over every neuron and the energy. Costs are per effective sample.
+
+| Sampler | KL mean | ESS mean | Random bits per ESS | Synaptic events per ESS | Dense multiply adds per ESS |
+|---|---|---|---|---|---|
+| p-bit, NumPy noise (32 bit draws) | 0.00260 | 27909 | 367.1 | 11.28 | 80.3 |
+| p-bit, 16 bit LFSR | 0.00290 | 29014 | 176.5 | 10.96 | 77.2 |
+| spiking, tau 2 | 0.00293 | 18400 | 336.4 | 14.51 | 73.6 |
+| spiking, tau 4 | 0.00282 | 9675 | 430.9 | 21.80 | 94.3 |
+| spiking, tau 8 | 0.00345 | 4902 | 644.5 | 31.64 | 141.0 |
+
+Per sweep, the spiking sampler draws fewer random numbers, because a refractory neuron draws none. Per effective sample, this saving disappears. The refractory period makes consecutive samples more correlated, and at tau 4 and 8 the spiking sampler costs more than the p-bit sampler on every count. Only tau 2 is close, with 8 percent fewer random bits and multiply adds and 29 percent more synaptic events. The largest saving in this table comes from the noise source. A 16 bit LFSR halves the random bits with no measured loss of quality.
+
+This is one small machine, so these are hypotheses. Event counts are not joules. An energy claim needs a hardware cost for each kind of event.
+
+## Plan
+
+1. Larger sparse networks, more machines per size, and a vectorised sampler.
+2. Spiking variants that keep the per sweep saving without the extra correlation.
+3. An FPGA reference design (Tang Nano 9K) with a host script that checks the hardware samples against the exact distribution.
+4. Comparisons with other libraries on the same problems.
+
+Issues and pull requests are welcome.
+
+## References
+
+- L. Buesing, J. Bill, B. Nessler, W. Maass. Neural Dynamics as Sampling. PLoS Computational Biology, 2011.
+- K. Y. Camsari, R. Faria, B. M. Sutton, S. Datta. Stochastic p-bits for invertible logic. Physical Review X, 2017.
+
+## License
+
+MIT

thermospike-0.1.0/README.md

@@ -0,0 +1,86 @@
+# thermospike
+
+Stochastic spiking neurons that sample from Boltzmann distributions. The library counts what each sample costs in random bits, synaptic events and multiply adds.
+
+thermospike sits between two fields. Thermodynamic and probabilistic computing use physical noise to sample (p-bits, Gibbs samplers). Neuromorphic engineering uses spikes and event driven synapses. Neural sampling theory (Buesing, Bill, Nessler and Maass, 2011) shows that a network of stochastic spiking neurons with a refractory period samples from the same Boltzmann distribution as a p-bit network. thermospike implements both and measures the difference in cost.
+
+## Status
+
+Version 0.1.0. Small networks in pure Python and NumPy. Every sampler is checked against the exact distribution, found by enumerating every state. An FPGA reference design is planned and does not exist yet.
+
+## Install
+
+```
+pip install thermospike
+```
+
+From source:
+
+```
+git clone https://github.com/tlcdv/thermospike
+cd thermospike
+pip install -e ".[test]"
+pytest
+```
+
+## Example
+
+```python
+from thermospike import SpikingSampler, NumpyNoise, and_gate, empirical_distribution, exact_distribution, kl_divergence
+
+model = and_gate()                     # C = A AND B, as an energy function
+clamp = {2: 1}                         # fix the output C to 1 and run the gate backwards
+sampler = SpikingSampler(model, NumpyNoise(seed=0), tau=4, clamp=clamp)
+samples = sampler.sample(20000)
+
+print(samples.mean(axis=0))            # A and B are near 1
+print(kl_divergence(empirical_distribution(samples), exact_distribution(model, clamp=clamp)))
+print(sampler.counters)                # random bits, spikes, synaptic events, multiply adds
+```
+
+## What is in the library
+
+| Part | Contents |
+|---|---|
+| `BoltzmannMachine` | Binary neurons (0 silent, 1 active), symmetric weights, biases |
+| `PBitSampler` | Sequential Gibbs sampling, one p-bit per neuron |
+| `SpikingSampler` | Neural sampling with an absolute refractory period of `tau` steps. With `tau=1` it is the p-bit sampler |
+| `NumpyNoise`, `LFSRNoise` | Software noise, and a 32 bit LFSR model of the noise a small FPGA would use |
+| `Counters` | Updates, random draws, random bits, spikes, state changes, synaptic events, dense multiply adds |
+| `exact_distribution`, `kl_divergence` | Checks against enumeration for up to 20 neurons |
+| `effective_sample_size`, `integrated_autocorrelation_time` | Mixing diagnostics, so costs are compared per independent sample |
+| `and_gate`, `max_cut`, `random_machine` | Example problems |
+
+## First measurement
+
+`python examples/compare_cost.py --neurons 8 --sweeps 40000 --seeds 0 1 2` on one random 8 neuron machine (model seed 0, weight scale 1) gave this table. KL is KL(empirical || exact) in nats. With 256 states and 40000 samples, finite sampling alone gives about 0.003, so every row sits near that floor. ESS is the effective sample size, the worst case over every neuron and the energy. Costs are per effective sample.
+
+| Sampler | KL mean | ESS mean | Random bits per ESS | Synaptic events per ESS | Dense multiply adds per ESS |
+|---|---|---|---|---|---|
+| p-bit, NumPy noise (32 bit draws) | 0.00260 | 27909 | 367.1 | 11.28 | 80.3 |
+| p-bit, 16 bit LFSR | 0.00290 | 29014 | 176.5 | 10.96 | 77.2 |
+| spiking, tau 2 | 0.00293 | 18400 | 336.4 | 14.51 | 73.6 |
+| spiking, tau 4 | 0.00282 | 9675 | 430.9 | 21.80 | 94.3 |
+| spiking, tau 8 | 0.00345 | 4902 | 644.5 | 31.64 | 141.0 |
+
+Per sweep, the spiking sampler draws fewer random numbers, because a refractory neuron draws none. Per effective sample, this saving disappears. The refractory period makes consecutive samples more correlated, and at tau 4 and 8 the spiking sampler costs more than the p-bit sampler on every count. Only tau 2 is close, with 8 percent fewer random bits and multiply adds and 29 percent more synaptic events. The largest saving in this table comes from the noise source. A 16 bit LFSR halves the random bits with no measured loss of quality.
+
+This is one small machine, so these are hypotheses. Event counts are not joules. An energy claim needs a hardware cost for each kind of event.
+
+## Plan
+
+1. Larger sparse networks, more machines per size, and a vectorised sampler.
+2. Spiking variants that keep the per sweep saving without the extra correlation.
+3. An FPGA reference design (Tang Nano 9K) with a host script that checks the hardware samples against the exact distribution.
+4. Comparisons with other libraries on the same problems.
+
+Issues and pull requests are welcome.
+
+## References
+
+- L. Buesing, J. Bill, B. Nessler, W. Maass. Neural Dynamics as Sampling. PLoS Computational Biology, 2011.
+- K. Y. Camsari, R. Faria, B. M. Sutton, S. Datta. Stochastic p-bits for invertible logic. Physical Review X, 2017.
+
+## License
+
+MIT

thermospike-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "thermospike"
+version = "0.1.0"
+description = "Stochastic spiking neurons that sample from Boltzmann distributions, with a count of the random bits and synaptic events each sample costs."
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.10"
+authors = [{ name = "The thermospike contributors" }]
+dependencies = ["numpy>=1.24"]
+keywords = ["thermodynamic computing", "neuromorphic", "spiking neural network", "p-bit", "neural sampling", "boltzmann machine"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering",
+    "Intended Audience :: Science/Research",
+]
+
+[project.urls]
+Homepage = "https://github.com/tlcdv/thermospike"
+Source = "https://github.com/tlcdv/thermospike"
+Issues = "https://github.com/tlcdv/thermospike/issues"
+
+[project.optional-dependencies]
+test = ["pytest>=7"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]

thermospike-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

thermospike-0.1.0/src/thermospike/__init__.py

@@ -0,0 +1,32 @@
+"""thermospike: stochastic spiking neurons that sample, with noise as the resource."""
+
+from .diagnostics import autocorrelation, effective_sample_size, integrated_autocorrelation_time
+from .exact import all_states, empirical_distribution, exact_distribution, kl_divergence, state_index
+from .model import BoltzmannMachine
+from .noise import LFSRNoise, NoiseSource, NumpyNoise
+from .problems import and_gate, cut_size, max_cut, random_machine
+from .samplers import Counters, PBitSampler, SpikingSampler
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "BoltzmannMachine",
+    "Counters",
+    "LFSRNoise",
+    "NoiseSource",
+    "NumpyNoise",
+    "PBitSampler",
+    "SpikingSampler",
+    "all_states",
+    "and_gate",
+    "autocorrelation",
+    "cut_size",
+    "effective_sample_size",
+    "empirical_distribution",
+    "exact_distribution",
+    "integrated_autocorrelation_time",
+    "kl_divergence",
+    "max_cut",
+    "random_machine",
+    "state_index",
+]

thermospike-0.1.0/src/thermospike/diagnostics.py

@@ -0,0 +1,48 @@
+"""Mixing diagnostics. Consecutive samples of a Markov chain are correlated, so
+a fair cost comparison divides the cost by the effective number of samples."""
+
+from typing import List, Optional
+
+import numpy as np
+
+from .model import BoltzmannMachine
+
+
+def autocorrelation(x: np.ndarray) -> np.ndarray:
+    """Normalised autocorrelation of a series at every lag, computed with an FFT."""
+    x = np.asarray(x, dtype=float) - np.mean(x)
+    n = len(x)
+    spectrum = np.fft.rfft(x, 2 * n)
+    acf = np.fft.irfft(spectrum * np.conj(spectrum))[:n]
+    return acf / acf[0]
+
+
+def integrated_autocorrelation_time(x: np.ndarray) -> float:
+    """1 + 2 * sum of autocorrelations, truncated by Geyer's initial positive sequence.
+
+    It is 1 for independent samples. A value of 10 means ten correlated samples
+    carry about as much information as one independent sample.
+    """
+    if np.var(x) == 0:
+        raise ValueError("the series is constant, so its autocorrelation time is undefined")
+    rho = autocorrelation(x)
+    total = 0.0
+    for m in range(len(rho) // 2):
+        pair = rho[2 * m] + rho[2 * m + 1]
+        if pair <= 0:
+            break
+        total += pair
+    return max(1.0, 2.0 * total - 1.0)
+
+
+def effective_sample_size(samples: np.ndarray, model: Optional[BoltzmannMachine] = None) -> float:
+    """Worst case effective sample size over every neuron that changes, and over the energy if a model is given."""
+    series: List[np.ndarray] = [samples[:, k] for k in range(samples.shape[1]) if np.var(samples[:, k]) > 0]
+    if model is not None:
+        energies = np.array([model.energy(z) for z in samples])
+        if np.var(energies) > 0:
+            series.append(energies)
+    if not series:
+        raise ValueError("no series in the samples changes, so nothing can be measured")
+    worst = max(integrated_autocorrelation_time(s) for s in series)
+    return len(samples) / worst

thermospike-0.1.0/src/thermospike/exact.py

@@ -0,0 +1,50 @@
+"""Exact distributions by enumeration, to check samplers on small networks."""
+
+from typing import Dict, Optional
+
+import numpy as np
+
+from .model import BoltzmannMachine
+
+MAX_EXACT_NEURONS = 20
+
+
+def all_states(n: int) -> np.ndarray:
+    """Every binary state of n neurons. Row i is the state with index i."""
+    if n > MAX_EXACT_NEURONS:
+        raise ValueError(f"enumeration is limited to {MAX_EXACT_NEURONS} neurons")
+    index = np.arange(2 ** n)
+    return ((index[:, None] >> np.arange(n)) & 1).astype(np.int8)
+
+
+def state_index(states: np.ndarray) -> np.ndarray:
+    """Index of each state row, with neuron k as bit k."""
+    states = np.atleast_2d(states)
+    return (states.astype(np.int64) << np.arange(states.shape[1])).sum(axis=1)
+
+
+def exact_distribution(model: BoltzmannMachine, beta: float = 1.0,
+                       clamp: Optional[Dict[int, int]] = None) -> np.ndarray:
+    """Probability of every state. States that break a clamp get probability 0."""
+    states = all_states(model.n)
+    energies = np.array([model.energy(z) for z in states])
+    logits = -beta * energies
+    for k, value in (clamp or {}).items():
+        logits[states[:, k] != value] = -np.inf
+    logits -= logits.max()
+    weights = np.exp(logits)
+    return weights / weights.sum()
+
+
+def empirical_distribution(samples: np.ndarray) -> np.ndarray:
+    """Fraction of samples in every state."""
+    counts = np.bincount(state_index(samples), minlength=2 ** samples.shape[1])
+    return counts / counts.sum()
+
+
+def kl_divergence(p: np.ndarray, q: np.ndarray) -> float:
+    """KL(p || q) in nats. Use p for the samples and q for the exact distribution."""
+    support = p > 0
+    if np.any(q[support] == 0):
+        return float("inf")
+    return float(np.sum(p[support] * np.log(p[support] / q[support])))

thermospike-0.1.0/src/thermospike/model.py

@@ -0,0 +1,43 @@
+"""Boltzmann machines over binary neurons.
+
+A state z has one entry per neuron, 1 for active (spiking) and 0 for silent.
+The energy is E(z) = -1/2 z.W.z - b.z and the target distribution is
+p(z) proportional to exp(-beta * E(z)).
+"""
+
+from typing import Sequence
+
+import numpy as np
+
+
+class BoltzmannMachine:
+    """Symmetric weights with a zero diagonal, and one bias per neuron."""
+
+    def __init__(self, weights: Sequence[Sequence[float]], biases: Sequence[float]):
+        w = np.asarray(weights, dtype=float)
+        b = np.asarray(biases, dtype=float)
+        if w.ndim != 2 or w.shape[0] != w.shape[1]:
+            raise ValueError("weights must be a square matrix")
+        if b.shape != (w.shape[0],):
+            raise ValueError("biases must have one entry per neuron")
+        if not np.allclose(w, w.T):
+            raise ValueError("weights must be symmetric")
+        if np.any(np.diag(w) != 0):
+            raise ValueError("weights must have a zero diagonal")
+        self.weights = w
+        self.biases = b
+        # Number of synapses that leave each neuron. An event driven chip pays
+        # this many synaptic updates each time the neuron changes state.
+        self.fan_out = np.count_nonzero(w, axis=0)
+
+    @property
+    def n(self) -> int:
+        return len(self.biases)
+
+    def energy(self, z: np.ndarray) -> float:
+        z = np.asarray(z, dtype=float)
+        return float(-0.5 * z @ self.weights @ z - self.biases @ z)
+
+    def potential(self, z: np.ndarray, k: int) -> float:
+        """Membrane potential of neuron k, which is minus the energy change of switching it on."""
+        return float(self.biases[k] + self.weights[k] @ z)

thermospike-0.1.0/src/thermospike/noise.py

@@ -0,0 +1,69 @@
+"""Noise sources for stochastic neurons.
+
+Every source counts the random bits it gives out, so a sampler can report
+the cost of its randomness next to the quality of its samples.
+"""
+
+from typing import Optional, Protocol
+
+import numpy as np
+
+
+class NoiseSource(Protocol):
+    """A source of uniform numbers on (0, 1) that counts its random bits."""
+
+    bits_per_draw: int
+    bits_used: int
+
+    def uniform(self) -> float:
+        """Return one uniform number on (0, 1)."""
+        ...
+
+
+class NumpyNoise:
+    """Software noise from the NumPy PCG64 generator."""
+
+    def __init__(self, seed: Optional[int] = None, bits_per_draw: int = 32):
+        self._rng = np.random.default_rng(seed)
+        self.bits_per_draw = bits_per_draw
+        self.bits_used = 0
+
+    def uniform(self) -> float:
+        self.bits_used += self.bits_per_draw
+        return float(self._rng.random())
+
+
+# Galois mask for the maximal length polynomial x^32 + x^22 + x^2 + x + 1.
+LFSR32_TAPS = 0x80200003
+
+
+class LFSRNoise:
+    """A 32 bit Galois linear feedback shift register, as a cheap FPGA would use.
+
+    Each draw shifts the register `bits_per_draw` times and reads the output
+    bits as an integer, so the resolution of one draw is 2 ** -bits_per_draw.
+    """
+
+    def __init__(self, seed: int = 1, bits_per_draw: int = 16):
+        state = seed & 0xFFFFFFFF
+        if state == 0:
+            raise ValueError("an LFSR seed must be nonzero in its low 32 bits")
+        if not 1 <= bits_per_draw <= 32:
+            raise ValueError("bits_per_draw must be between 1 and 32")
+        self.state = state
+        self.bits_per_draw = bits_per_draw
+        self.bits_used = 0
+
+    def next_bit(self) -> int:
+        bit = self.state & 1
+        self.state >>= 1
+        if bit:
+            self.state ^= LFSR32_TAPS
+        return bit
+
+    def uniform(self) -> float:
+        value = 0
+        for _ in range(self.bits_per_draw):
+            value = (value << 1) | self.next_bit()
+        self.bits_used += self.bits_per_draw
+        return (value + 0.5) / (1 << self.bits_per_draw)

thermospike-0.1.0/src/thermospike/problems.py

@@ -0,0 +1,40 @@
+"""Small example problems written as Boltzmann machines."""
+
+from typing import Iterable, Optional, Tuple
+
+import numpy as np
+
+from .model import BoltzmannMachine
+
+
+def random_machine(n: int, scale: float = 1.0, seed: Optional[int] = None) -> BoltzmannMachine:
+    """Dense random weights and biases, drawn from a normal distribution with this scale."""
+    rng = np.random.default_rng(seed)
+    upper = np.triu(rng.normal(0.0, scale, (n, n)), k=1)
+    return BoltzmannMachine(upper + upper.T, rng.normal(0.0, scale, n))
+
+
+def and_gate(strength: float = 3.0) -> BoltzmannMachine:
+    """Neurons (A, B, C) with low energy only where C = A AND B.
+
+    The energy is strength * (3C + AB - 2AC - 2BC). It is 0 on the four rows of
+    the truth table and at least `strength` on every other state. Clamp A and B
+    to compute C, or clamp C to 1 to run the gate backwards and find A = B = 1.
+    """
+    s = strength
+    weights = [[0.0, -s, 2 * s],
+               [-s, 0.0, 2 * s],
+               [2 * s, 2 * s, 0.0]]
+    return BoltzmannMachine(weights, [0.0, 0.0, -3 * s])
+
+
+def max_cut(n: int, edges: Iterable[Tuple[int, int]]) -> BoltzmannMachine:
+    """Lowest energy states are the largest cuts of the graph. Energy = minus the cut size."""
+    adjacency = np.zeros((n, n))
+    for i, j in edges:
+        adjacency[i, j] = adjacency[j, i] = 1.0
+    return BoltzmannMachine(-2 * adjacency, adjacency.sum(axis=1))
+
+
+def cut_size(edges: Iterable[Tuple[int, int]], z: np.ndarray) -> int:
+    return sum(int(z[i] != z[j]) for i, j in edges)

thermospike-0.1.0/src/thermospike/samplers.py

@@ -0,0 +1,128 @@
+"""Samplers that run a Boltzmann machine with stochastic neurons.
+
+PBitSampler is sequential Gibbs sampling. Each neuron is a p-bit that turns on
+with probability sigmoid(beta * u), where u is its membrane potential.
+
+SpikingSampler is neural sampling with an absolute refractory period
+(Buesing, Bill, Nessler and Maass, PLoS Comput Biol 2011). A neuron that
+spikes stays active for tau steps. Outside that period it spikes with
+probability sigmoid(beta * u - log(tau)). The active neurons are samples from
+the same Boltzmann distribution, and the neuron draws no random bits while
+it is refractory. With tau = 1 it is the p-bit sampler.
+"""
+
+import math
+from dataclasses import dataclass
+from typing import Dict, Optional
+
+import numpy as np
+
+from .model import BoltzmannMachine
+from .noise import NoiseSource
+
+
+@dataclass
+class Counters:
+    """Work done by a sampler. These are event counts, not joules."""
+
+    updates: int = 0            # neuron update steps
+    random_draws: int = 0       # uniform numbers drawn
+    random_bits: int = 0        # bits behind those draws
+    spikes: int = 0             # p-bit: off to on events. Spiking: every spike, also one that extends an active period
+    state_changes: int = 0      # on to off and off to on events
+    synaptic_events: int = 0    # fan out of every state change (event driven cost)
+    dense_multiply_adds: int = 0  # cost of recomputing every potential from scratch
+
+
+def sigmoid(x: float) -> float:
+    if x >= 0:
+        return 1.0 / (1.0 + math.exp(-x))
+    e = math.exp(x)
+    return e / (1.0 + e)
+
+
+class _Sampler:
+    def __init__(self, model: BoltzmannMachine, noise: NoiseSource, beta: float = 1.0,
+                 clamp: Optional[Dict[int, int]] = None, initial: Optional[np.ndarray] = None):
+        self.model = model
+        self.noise = noise
+        self.beta = beta
+        self.clamp = dict(clamp or {})
+        self.z = np.zeros(model.n) if initial is None else np.asarray(initial, dtype=float).copy()
+        for k, value in self.clamp.items():
+            self.z[k] = value
+        self.free = [k for k in range(model.n) if k not in self.clamp]
+        self.counters = Counters()
+
+    def _draw(self) -> float:
+        bits_before = self.noise.bits_used
+        value = self.noise.uniform()
+        self.counters.random_draws += 1
+        self.counters.random_bits += self.noise.bits_used - bits_before
+        return value
+
+    def _set(self, k: int, value: int) -> None:
+        if self.z[k] != value:
+            self.counters.state_changes += 1
+            self.counters.synaptic_events += int(self.model.fan_out[k])
+            self.z[k] = value
+
+    def _potential(self, k: int) -> float:
+        self.counters.dense_multiply_adds += self.model.n - 1
+        return self.model.potential(self.z, k)
+
+    def _update(self, k: int) -> None:
+        raise NotImplementedError
+
+    def sweep(self) -> None:
+        """Update every free neuron once, in index order."""
+        for k in self.free:
+            self.counters.updates += 1
+            self._update(k)
+
+    def sample(self, n_samples: int, burn_in: int = 100) -> np.ndarray:
+        """Run burn_in sweeps, then return the state after each of n_samples sweeps."""
+        for _ in range(burn_in):
+            self.sweep()
+        out = np.empty((n_samples, self.model.n), dtype=np.int8)
+        for i in range(n_samples):
+            self.sweep()
+            out[i] = self.z
+        return out
+
+
+class PBitSampler(_Sampler):
+    """Sequential Gibbs sampling with p-bit neurons."""
+
+    def _update(self, k: int) -> None:
+        p_on = sigmoid(self.beta * self._potential(k))
+        value = 1 if self._draw() < p_on else 0
+        if value == 1 and self.z[k] == 0:
+            self.counters.spikes += 1
+        self._set(k, value)
+
+
+class SpikingSampler(_Sampler):
+    """Neural sampling with stochastic spiking neurons and a refractory period of tau steps."""
+
+    def __init__(self, model: BoltzmannMachine, noise: NoiseSource, tau: int = 1, **kwargs):
+        if tau < 1:
+            raise ValueError("tau must be at least 1")
+        super().__init__(model, noise, **kwargs)
+        self.tau = tau
+        self.log_tau = math.log(tau)
+        # Steps left in the active period. A neuron is active while this is 1 or more.
+        self.refractory = np.where(self.z == 1, 1, 0)
+
+    def _update(self, k: int) -> None:
+        if self.refractory[k] > 1:
+            self.refractory[k] -= 1
+            return
+        p_spike = sigmoid(self.beta * self._potential(k) - self.log_tau)
+        if self._draw() < p_spike:
+            self.refractory[k] = self.tau
+            self.counters.spikes += 1
+            self._set(k, 1)
+        else:
+            self.refractory[k] = 0
+            self._set(k, 0)

thermospike-0.1.0/src/thermospike.egg-info/PKG-INFO

@@ -0,0 +1,107 @@
+Metadata-Version: 2.4
+Name: thermospike
+Version: 0.1.0
+Summary: Stochastic spiking neurons that sample from Boltzmann distributions, with a count of the random bits and synaptic events each sample costs.
+Author: The thermospike contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/tlcdv/thermospike
+Project-URL: Source, https://github.com/tlcdv/thermospike
+Project-URL: Issues, https://github.com/tlcdv/thermospike/issues
+Keywords: thermodynamic computing,neuromorphic,spiking neural network,p-bit,neural sampling,boltzmann machine
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Dynamic: license-file
+
+# thermospike
+
+Stochastic spiking neurons that sample from Boltzmann distributions. The library counts what each sample costs in random bits, synaptic events and multiply adds.
+
+thermospike sits between two fields. Thermodynamic and probabilistic computing use physical noise to sample (p-bits, Gibbs samplers). Neuromorphic engineering uses spikes and event driven synapses. Neural sampling theory (Buesing, Bill, Nessler and Maass, 2011) shows that a network of stochastic spiking neurons with a refractory period samples from the same Boltzmann distribution as a p-bit network. thermospike implements both and measures the difference in cost.
+
+## Status
+
+Version 0.1.0. Small networks in pure Python and NumPy. Every sampler is checked against the exact distribution, found by enumerating every state. An FPGA reference design is planned and does not exist yet.
+
+## Install
+
+```
+pip install thermospike
+```
+
+From source:
+
+```
+git clone https://github.com/tlcdv/thermospike
+cd thermospike
+pip install -e ".[test]"
+pytest
+```
+
+## Example
+
+```python
+from thermospike import SpikingSampler, NumpyNoise, and_gate, empirical_distribution, exact_distribution, kl_divergence
+
+model = and_gate()                     # C = A AND B, as an energy function
+clamp = {2: 1}                         # fix the output C to 1 and run the gate backwards
+sampler = SpikingSampler(model, NumpyNoise(seed=0), tau=4, clamp=clamp)
+samples = sampler.sample(20000)
+
+print(samples.mean(axis=0))            # A and B are near 1
+print(kl_divergence(empirical_distribution(samples), exact_distribution(model, clamp=clamp)))
+print(sampler.counters)                # random bits, spikes, synaptic events, multiply adds
+```
+
+## What is in the library
+
+| Part | Contents |
+|---|---|
+| `BoltzmannMachine` | Binary neurons (0 silent, 1 active), symmetric weights, biases |
+| `PBitSampler` | Sequential Gibbs sampling, one p-bit per neuron |
+| `SpikingSampler` | Neural sampling with an absolute refractory period of `tau` steps. With `tau=1` it is the p-bit sampler |
+| `NumpyNoise`, `LFSRNoise` | Software noise, and a 32 bit LFSR model of the noise a small FPGA would use |
+| `Counters` | Updates, random draws, random bits, spikes, state changes, synaptic events, dense multiply adds |
+| `exact_distribution`, `kl_divergence` | Checks against enumeration for up to 20 neurons |
+| `effective_sample_size`, `integrated_autocorrelation_time` | Mixing diagnostics, so costs are compared per independent sample |
+| `and_gate`, `max_cut`, `random_machine` | Example problems |
+
+## First measurement
+
+`python examples/compare_cost.py --neurons 8 --sweeps 40000 --seeds 0 1 2` on one random 8 neuron machine (model seed 0, weight scale 1) gave this table. KL is KL(empirical || exact) in nats. With 256 states and 40000 samples, finite sampling alone gives about 0.003, so every row sits near that floor. ESS is the effective sample size, the worst case over every neuron and the energy. Costs are per effective sample.
+
+| Sampler | KL mean | ESS mean | Random bits per ESS | Synaptic events per ESS | Dense multiply adds per ESS |
+|---|---|---|---|---|---|
+| p-bit, NumPy noise (32 bit draws) | 0.00260 | 27909 | 367.1 | 11.28 | 80.3 |
+| p-bit, 16 bit LFSR | 0.00290 | 29014 | 176.5 | 10.96 | 77.2 |
+| spiking, tau 2 | 0.00293 | 18400 | 336.4 | 14.51 | 73.6 |
+| spiking, tau 4 | 0.00282 | 9675 | 430.9 | 21.80 | 94.3 |
+| spiking, tau 8 | 0.00345 | 4902 | 644.5 | 31.64 | 141.0 |
+
+Per sweep, the spiking sampler draws fewer random numbers, because a refractory neuron draws none. Per effective sample, this saving disappears. The refractory period makes consecutive samples more correlated, and at tau 4 and 8 the spiking sampler costs more than the p-bit sampler on every count. Only tau 2 is close, with 8 percent fewer random bits and multiply adds and 29 percent more synaptic events. The largest saving in this table comes from the noise source. A 16 bit LFSR halves the random bits with no measured loss of quality.
+
+This is one small machine, so these are hypotheses. Event counts are not joules. An energy claim needs a hardware cost for each kind of event.
+
+## Plan
+
+1. Larger sparse networks, more machines per size, and a vectorised sampler.
+2. Spiking variants that keep the per sweep saving without the extra correlation.
+3. An FPGA reference design (Tang Nano 9K) with a host script that checks the hardware samples against the exact distribution.
+4. Comparisons with other libraries on the same problems.
+
+Issues and pull requests are welcome.
+
+## References
+
+- L. Buesing, J. Bill, B. Nessler, W. Maass. Neural Dynamics as Sampling. PLoS Computational Biology, 2011.
+- K. Y. Camsari, R. Faria, B. M. Sutton, S. Datta. Stochastic p-bits for invertible logic. Physical Review X, 2017.
+
+## License
+
+MIT

thermospike-0.1.0/src/thermospike.egg-info/SOURCES.txt

@@ -0,0 +1,16 @@
+LICENSE
+README.md
+pyproject.toml
+src/thermospike/__init__.py
+src/thermospike/diagnostics.py
+src/thermospike/exact.py
+src/thermospike/model.py
+src/thermospike/noise.py
+src/thermospike/problems.py
+src/thermospike/samplers.py
+src/thermospike.egg-info/PKG-INFO
+src/thermospike.egg-info/SOURCES.txt
+src/thermospike.egg-info/dependency_links.txt
+src/thermospike.egg-info/requires.txt
+src/thermospike.egg-info/top_level.txt
+tests/test_thermospike.py

thermospike-0.1.0/src/thermospike.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

thermospike-0.1.0/src/thermospike.egg-info/requires.txt

@@ -0,0 +1,4 @@
+numpy>=1.24
+
+[test]
+pytest>=7

thermospike-0.1.0/src/thermospike.egg-info/top_level.txt

@@ -0,0 +1 @@
+thermospike

thermospike-0.1.0/tests/test_thermospike.py

@@ -0,0 +1,170 @@
+import numpy as np
+import pytest
+
+from thermospike import (
+    BoltzmannMachine,
+    LFSRNoise,
+    NumpyNoise,
+    PBitSampler,
+    SpikingSampler,
+    all_states,
+    and_gate,
+    cut_size,
+    effective_sample_size,
+    empirical_distribution,
+    exact_distribution,
+    integrated_autocorrelation_time,
+    kl_divergence,
+    max_cut,
+    random_machine,
+)
+
+# KL(empirical || exact) for 32 states from 40000 correlated samples sits near
+# 0.001 nats from finite sampling alone. A wrong sampler lands far above 0.01.
+KL_LIMIT = 0.01
+
+
+# ------------------------------------------------------------------ noise
+
+
+def test_lfsr_is_deterministic_and_counts_bits():
+    a, b = LFSRNoise(seed=7, bits_per_draw=16), LFSRNoise(seed=7, bits_per_draw=16)
+    assert [a.uniform() for _ in range(50)] == [b.uniform() for _ in range(50)]
+    assert a.bits_used == 50 * 16
+
+
+def test_lfsr_output_is_close_to_uniform():
+    noise = LFSRNoise(seed=12345)
+    values = np.array([noise.uniform() for _ in range(20000)])
+    assert 0 < values.min() and values.max() < 1
+    assert abs(values.mean() - 0.5) < 0.01
+    counts, _ = np.histogram(values, bins=10, range=(0, 1))
+    assert counts.min() > 1800 and counts.max() < 2200
+
+
+def test_lfsr_rejects_zero_seed():
+    with pytest.raises(ValueError):
+        LFSRNoise(seed=0)
+
+
+# ------------------------------------------------------------------ model
+
+
+def test_model_rejects_asymmetric_weights():
+    with pytest.raises(ValueError):
+        BoltzmannMachine([[0, 1], [2, 0]], [0, 0])
+
+
+def test_energy_matches_hand_calculation():
+    model = BoltzmannMachine([[0, 2], [2, 0]], [1, -1])
+    # E = -W01 z0 z1 - b.z
+    assert model.energy([1, 1]) == pytest.approx(-2 - 1 + 1)
+    assert model.energy([1, 0]) == pytest.approx(-1)
+    assert model.potential(np.array([0.0, 1.0]), 0) == pytest.approx(1 + 2)
+
+
+def test_exact_distribution_sums_to_one_and_respects_clamp():
+    model = random_machine(4, seed=0)
+    p = exact_distribution(model, clamp={2: 1})
+    assert p.sum() == pytest.approx(1.0)
+    assert np.all(p[all_states(4)[:, 2] == 0] == 0)
+
+
+# ------------------------------------------------------------------ samplers against exact enumeration
+
+
+@pytest.mark.parametrize("noise", [NumpyNoise(seed=1), LFSRNoise(seed=99)], ids=["numpy", "lfsr"])
+def test_pbit_sampler_matches_exact_distribution(noise):
+    model = random_machine(5, seed=3)
+    samples = PBitSampler(model, noise).sample(40000)
+    kl = kl_divergence(empirical_distribution(samples), exact_distribution(model))
+    assert kl < KL_LIMIT
+
+
+@pytest.mark.parametrize("tau", [2, 4])
+def test_spiking_sampler_matches_exact_distribution(tau):
+    model = random_machine(5, seed=3)
+    samples = SpikingSampler(model, NumpyNoise(seed=2), tau=tau).sample(40000)
+    kl = kl_divergence(empirical_distribution(samples), exact_distribution(model))
+    assert kl < KL_LIMIT
+
+
+def test_spiking_sampler_with_tau_one_is_the_pbit_sampler():
+    model = random_machine(4, seed=5)
+    pbit = PBitSampler(model, NumpyNoise(seed=8)).sample(2000, burn_in=0)
+    spiking = SpikingSampler(model, NumpyNoise(seed=8), tau=1).sample(2000, burn_in=0)
+    assert np.array_equal(pbit, spiking)
+
+
+def test_spiking_sampler_skips_random_draws_while_refractory():
+    model = random_machine(5, seed=3)
+    pbit = PBitSampler(model, NumpyNoise(seed=1))
+    spiking = SpikingSampler(model, NumpyNoise(seed=1), tau=4)
+    pbit.sample(5000)
+    spiking.sample(5000)
+    assert pbit.counters.random_draws == pbit.counters.updates
+    assert spiking.counters.random_draws < spiking.counters.updates
+    assert spiking.counters.random_bits == 32 * spiking.counters.random_draws
+
+
+def test_synaptic_events_follow_state_changes():
+    model = random_machine(5, seed=3)
+    sampler = PBitSampler(model, NumpyNoise(seed=4))
+    sampler.sample(1000)
+    # Dense random weights: every neuron reaches the other four.
+    assert sampler.counters.synaptic_events == 4 * sampler.counters.state_changes
+    assert sampler.counters.dense_multiply_adds == 4 * sampler.counters.updates
+
+
+# ------------------------------------------------------------------ mixing diagnostics
+
+
+def test_autocorrelation_time_of_independent_samples_is_one():
+    x = np.random.default_rng(0).normal(size=20000)
+    assert integrated_autocorrelation_time(x) == pytest.approx(1.0, abs=0.1)
+
+
+def test_autocorrelation_time_of_ar1_series_matches_theory():
+    # For x[t] = phi * x[t-1] + noise the time is (1 + phi) / (1 - phi) = 19 at phi = 0.9.
+    rng = np.random.default_rng(1)
+    x = np.zeros(200000)
+    for t in range(1, len(x)):
+        x[t] = 0.9 * x[t - 1] + rng.normal()
+    assert integrated_autocorrelation_time(x) == pytest.approx(19.0, rel=0.15)
+
+
+def test_constant_series_is_rejected():
+    with pytest.raises(ValueError):
+        integrated_autocorrelation_time(np.ones(100))
+
+
+def test_effective_sample_size_skips_clamped_neurons():
+    model = and_gate()
+    samples = PBitSampler(model, NumpyNoise(seed=3), clamp={2: 1}).sample(5000)
+    ess = effective_sample_size(samples, model)
+    assert 0 < ess <= 5000
+
+
+# ------------------------------------------------------------------ example problems
+
+
+def test_and_gate_forward_and_backward():
+    model = and_gate(strength=3.0)
+    for a in (0, 1):
+        for b in (0, 1):
+            samples = PBitSampler(model, NumpyNoise(seed=a * 2 + b), clamp={0: a, 1: b}).sample(5000)
+            assert samples[:, 2].mean() == pytest.approx(a & b, abs=0.1)
+    clamp = {2: 1}
+    samples = SpikingSampler(model, NumpyNoise(seed=11), tau=3, clamp=clamp).sample(20000)
+    kl = kl_divergence(empirical_distribution(samples), exact_distribution(model, clamp=clamp))
+    assert kl < KL_LIMIT
+    assert np.mean(np.all(samples == 1, axis=1)) > 0.85
+
+
+def test_annealing_finds_the_maximum_cut_of_a_ring():
+    edges = [(i, (i + 1) % 6) for i in range(6)]
+    sampler = PBitSampler(max_cut(6, edges), NumpyNoise(seed=0))
+    for beta in np.linspace(0.1, 5.0, 200):
+        sampler.beta = beta
+        sampler.sweep()
+    assert cut_size(edges, sampler.z) == 6