mixingmatrix 0.2.0__tar.gz

mixingmatrix-0.2.0/CHANGELOG.md

@@ -0,0 +1,192 @@
+# Changelog
+
+## 0.2.0 -- scaling
+
+### OSQP is now a required dependency
+
+It was listed as the optional `[fast]` extra, but the SciPy fallback is roughly
+100x slower -- optional in name only, and a trap for anyone who installed
+without it. `pip install mixingmatrix` now gets it. The SciPy backend remains
+selectable as `qp_backend="scipy"`, where it serves as an independent
+implementation to check OSQP against rather than as a fallback; the fallback
+warning is gone, and a missing OSQP now raises with instructions instead of
+silently degrading.
+
+`solve` now runs at n = 10^4.  What was in the way was memory, not the
+eigendecomposition: the solver stored five dense `n x n` arrays, which is
+3.7 GB at that size.
+
+### The change
+
+* **The scaled dual is exactly low rank.**  `U_new = U + (B - Q - Z_new)` and
+  `Z_new = V - Delta` give `U_new = Delta` identically -- the `U` cancels -- so
+  the dual is the rank-2-to-9 spectral correction from the first iteration
+  onward, and `Z` is `B - Q + U_prev - U`.  Every matrix in the iteration is
+  now `diag + edges + P diag(c) P^T`, held by the new
+  `mixingmatrix.operators.SymOperator`.
+* **A matrix-free ADMM driver** (`_admm_matrix_free`) built on that, selected
+  automatically above `dense_below` nodes.  Memory `O(m + nk)` instead of
+  `O(n^2)`: 3.7 GB to a few megabytes at n = 10^4.  It is the *same iteration*, not
+  an approximation -- `tests/test_large.py` asserts it reproduces the dense
+  driver's iterate to 1e-9.
+* **The certificate is stored in factored form** `(Vp, Yp, Vm, Ym)` rather than
+  as an assembled `G`.  Reduced costs, `tr G`, `q^T G q` and `||G||_*` all
+  become small products in the factors, and `||G||_* = tr(Yp) + tr(Ym)` is now
+  exact rather than an eigendecomposition.
+* **Sparse paths throughout**: `slem`, `is_valid`, `stationary_distribution`,
+  `gossip`, all four baselines, and `Solution.weights`, which is now built
+  straight from the edge weights.  `Solution.to_dense()` refuses above 16384
+  nodes instead of allocating 2 GB.
+
+### The inner QP, which turned out to be the real ceiling at n = 10^4
+
+With the dense arrays gone, profiling put **74 s in OSQP's setup** and 1.8 s in
+every projection at n = 10^4, against 0.17 s for the whole spectral step -- and
+the setup time did not move with the tolerance, which is the signature of
+fill-in rather than iteration count.
+
+* **Split the linear map out of the inner QP.**  `P = 4I + 2 A^T A` is the edge
+  adjacency matrix; introducing `y = A w` leaves a diagonal Hessian and a KKT
+  system with the incidence matrix's sparsity.  Identical solutions to 1e-16.
+  Setup at n = 10^4: 96 s to 10.5 s. FDLA's unconstrained solve moved to the
+  matching saddle system for the same reason.
+* **Polishing off above n = 2000.**  It re-factorises the active-set system and
+  at this size reported "no active set detected" while doubling the solve.
+
+Together: 74 s of setup and 1.8 s per projection became **7.8 s and 0.39 s**,
+roughly 100x more ADMM iterations per second at n = 10^4.
+
+What remains is fill-in, and it belongs to the graph rather than the code: the
+same n = 10^4 solve sets up in 7.8 s on a random 4-regular graph and **0.04 s**
+on a 100x100 grid.  The scaling benchmark now reports both families.
+
+### A spectral step that converges the subspace, not the eigenvectors
+
+The epigraph prox uses `sum_{active} (lambda_i - sign(lambda_i) s) v_i v_i^T`,
+which is unchanged by any rotation within a degenerate group -- it needs the
+span.  ARPACK's convergence test is per eigenpair, so it restarts trying to
+separate eigenvalues that are genuinely equal, and at an SLEM optimum they are:
+a 100x100 grid at n = 10^4 has an active set of 30 packed inside 1e-4, and cost
+1767 products per iteration.
+
+* New `spectral.subspace_extreme`: block subspace iteration with a
+  caller-supplied acceptance test, warm-started across outer iterations.  The
+  ADMM's test is "every pair above the threshold converged, and no inactive
+  Ritz value can reach the threshold given its own residual bound", so a
+  converged return is exactly as rigorous as ARPACK's.  It also applies the
+  operator to `k` vectors per call rather than one.
+* Non-convergence falls back to ARPACK rather than proceeding on an unverified
+  subspace.
+* **Guarded**, because block iteration converges at rate
+  `|lambda_{k+1}/lambda_k|` and an expander's extreme spectrum is flat: three
+  consecutive failures switch it off for the solve.  Unguarded it was 1.7x
+  *worse* on expanders while being 2.25x better on grids; guarded it is never
+  worse and up to 2.25x better.
+
+### Two fixes worth 3x at n = 1000
+
+* The Lanczos window `k` ratcheted up early (when rho is small and half the
+  spectrum is active) and never came back down, sitting at 48 while the active
+  set was 3-5.  Letting it shrink to `2 k_active + 4`: 564 matvecs per
+  iteration to 234, same answer.
+* The ARPACK tolerance was divided by `n`, asking for 1e-8 at n = 1000 and
+  buying only restarts.
+
+### rho did not scale either
+
+`rho` sets the epigraph threshold through `rho * sum_{active}(a_i - s) = 1`, so
+it decides how much of the spectrum is active -- and the active set is exactly
+what the spectral step must compute.  A fixed `rho = 1` therefore got worse as
+graphs grew, and the *first* iterations of a large solve were its most
+expensive.  Eight iterations from a cold start at n = 10^4, random 4-regular:
+
+| rho | active set | per iteration |
+| --- | --- | --- |
+| 1 | 30 | 10996 ms |
+| 10 | 8 | 1980 ms |
+| 100 | 2 | 1184 ms |
+
+Identical SLEM in every row.  Boyd's residual-balancing heuristic does correct
+`rho`, but only every 25 iterations, and at 11 s an iteration a 45-second
+budget never reaches 25 -- the heuristic was outrun by the problem it exists to
+fix.
+
+`rho` now starts at `sqrt(n) / 8` above n = 2000 and stays exactly `1.0` below
+it.  The threshold is empirical: a naive size-scaling *regressed* n = 1000
+(SLEM 0.850490 against 0.848264 at the same budget), because a larger `rho`
+also slows the outer iteration and below a few thousand nodes that costs more
+than the cheaper spectral step buys.  At a 45-second budget, n = 10^4:
+**3 iterations became 91**.
+
+### Honest about what this does and does not buy
+
+Memory is solved; convergence is not.  At n = 10^4 a time-limited solve returns
+a feasible matrix that beats Metropolis-Hastings with a large but rigorous
+certified gap -- not a converged optimum.  `docs/internals.md` and `docs/limits.md` say which method
+to use at which size, and what each one actually delivers.
+
+`method="smoothing"` now raises above 2000 nodes rather than attempting a
+dense eigendecomposition: its softmax gradient weights every eigenvalue, so it
+needs the full spectrum by construction.
+
+## 0.1.0
+
+First release.
+
+### Core
+
+* `solve(G)` -- fastest-mixing Markov chain weights, with `method="auto"`
+  choosing an exact conic solve for small graphs (when CVXPY is installed) and
+  ADMM otherwise.
+* `Solution` with node-label round-tripping, a SLEM **recomputed from the
+  returned matrix** on every access, `support` / `zero_edges`, and a
+  `summary()` that never prints success over a run that hit a limit.
+* `time_limit` returns the best feasible iterate rather than nothing; `callback`
+  can stop a solve and get the same guarantee.
+
+### Certificates
+
+* `Solution.certified_gap` -- a rigorous bound from the dual, valid even for an
+  under-converged solve.
+* `can_remove` / `can_improve` -- single-edge certificates.  Removal is always a
+  proof; addition is a proof only when the reduced cost is nonnegative, and the
+  API is explicit about the asymmetry.
+* `update(sol, G_new)` -- skips the solve entirely for a certified change, warm
+  starts a priced removal (1.5x-17x fewer iterations on the measured families),
+  and falls back to a cold solve when the certificate says a warm start will not
+  help.
+
+### Baselines and evaluation
+
+* `metropolis_hastings`, `max_degree`, `best_constant` (solved in closed form),
+  `lazy_random_walk`.
+* `slem`, `spectral_gap`, `consensus_rounds`, `mixing_time`,
+  `stationary_distribution`, `is_valid`.
+* `compare(G)` -- one table, the optimum against the usual weights, on the
+  user's own graph.
+* `gossip`, `rounds_to_consensus` -- run the chain instead of trusting the
+  eigenvalue.
+
+### Variants
+
+* `allow_negative=True` -- FDLA (Xiao & Boyd 2004).
+* `stationary=pi` -- the fastest reversible chain with a given stationary
+  distribution, through the same parameterisation rather than a second code
+  path.
+* `symmetric=False` raises `NotImplementedError` rather than solving something
+  else.
+
+### Solvers
+
+* `admm` (default) -- exact epigraph prox, dense below n=400 and Lanczos above,
+  with the inner QP tolerance and the ARPACK tolerance both tied to the outer
+  residual.
+* `smoothing` -- accelerated projected gradient on a softmax surrogate;
+  moderate accuracy, no dual variable.
+* `subgradient` -- the honest naive method, reported as `stalled` because it
+  certifies nothing.
+* `cvxpy` -- exact interior-point reference.
+
+### Command line
+
+`mixingmatrix solve|compare|info`, reading five graph formats and writing four.

mixingmatrix-0.2.0/CITATION.cff

@@ -0,0 +1,62 @@
+cff-version: 1.2.0
+title: "mixingmatrix: optimal mixing matrices for graphs"
+message: "If you use this software, please cite it and the papers it implements."
+type: software
+authors:
+  - given-names: Raghuram
+    email: raghuram87@gmail.com
+repository-code: "https://github.com/raghuram87/mixingmatrix"
+license: MIT
+version: 0.2.0
+keywords:
+  - fastest mixing Markov chain
+  - distributed averaging
+  - consensus
+  - gossip
+  - spectral graph theory
+  - semidefinite programming
+references:
+  - type: article
+    title: "Fastest Mixing Markov Chain on a Graph"
+    authors:
+      - family-names: Boyd
+        given-names: Stephen
+      - family-names: Diaconis
+        given-names: Persi
+      - family-names: Xiao
+        given-names: Lin
+    journal: "SIAM Review"
+    volume: 46
+    issue: 4
+    year: 2004
+    doi: 10.1137/S0036144503423264
+  - type: article
+    title: "Fast linear iterations for distributed averaging"
+    authors:
+      - family-names: Xiao
+        given-names: Lin
+      - family-names: Boyd
+        given-names: Stephen
+    journal: "Systems & Control Letters"
+    volume: 53
+    issue: 1
+    year: 2004
+    doi: 10.1016/j.sysconle.2004.02.022
+  - type: article
+    title: "Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers"
+    authors:
+      - family-names: Boyd
+        given-names: Stephen
+      - family-names: Parikh
+        given-names: Neal
+      - family-names: Chu
+        given-names: Eric
+      - family-names: Peleato
+        given-names: Borja
+      - family-names: Eckstein
+        given-names: Jonathan
+    journal: "Foundations and Trends in Machine Learning"
+    volume: 3
+    issue: 1
+    year: 2011
+    doi: 10.1561/2200000016

mixingmatrix-0.2.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,63 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, colour, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behaviour that contributes to a positive environment:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologising to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best for the overall community
+
+Examples of unacceptable behaviour:
+
+* The use of sexualised language or imagery, and sexual attention or advances
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Project maintainers are responsible for clarifying and enforcing our standards
+of acceptable behaviour and will take appropriate and fair corrective action in
+response to any behaviour that they deem inappropriate, threatening, offensive,
+or harmful.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behaviour may be
+reported to the project maintainers at raghuram87@gmail.com. All complaints
+will be reviewed and investigated promptly and fairly.
+
+Project maintainers are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
+
+[homepage]: https://www.contributor-covenant.org

mixingmatrix-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,101 @@
+# Contributing
+
+## Getting set up
+
+```bash
+pip install -e ".[dev,all]"
+pytest
+```
+
+OSQP is a required dependency and is installed with the package.
+
+## Invariants to preserve
+
+1. `Solution.slem` is recomputed from `Solution.weights` on every access, never
+   taken from a solver's own objective. A new solver returns weights and
+   effort; the package scores it.
+2. `status == "optimal"` means the solver met its stopping test. A run that
+   stopped on a limit warns and reports the limit it hit.
+3. `EdgeCertificate.is_proof` is `True` only when the mathematics establishes
+   the claim. `can_improve` is asymmetric by design — see
+   `docs/incremental.md`.
+4. Node labels round-trip. Integer indexing stays inside `GraphIndex`.
+
+## Adding a solver
+
+One function and one decorator:
+
+```python
+from mixingmatrix.solvers.base import SolveOptions, SolverOutput, register_solver
+
+@register_solver("my-method")
+def my_method(problem, opts: SolveOptions) -> SolverOutput:
+    w = ...                       # edge weights, feasible
+    return SolverOutput(w=w, status="optimal", iterations=k)
+```
+
+`problem` is a `MixingProblem`: `to_matrix`, `to_sparse`, `edge_gradient`,
+`objective`, `metropolis_weights`, and the constraint data `A`, `q`, `nonneg`.
+Use `EdgeProjector` for the projection rather than writing your own.
+
+See `docs/internals.md` for how the solver works.
+
+Then import the module in `mixingmatrix/solvers/__init__.py` so the decorator runs.
+`tests/test_solvers.py::test_available_methods_are_all_usable` picks it up
+automatically and will hold you to feasibility.
+
+If your method has no optimality certificate, report `stalled`, not `optimal`,
+and leave `dual_residual` as `nan`.
+
+## Adding a baseline
+
+Put it in `baselines.py`, return a `scipy.sparse.csr_matrix`, and add it to
+`BASELINES`.  `tests/test_evaluate.py::test_baselines_are_feasible` will check
+it across every graph family in `conftest.py`.
+
+## Tests
+
+* `test_solve.py` — core contract, edge cases, known closed-form optima.
+* `test_certificates.py` — each certificate is checked by making the change and
+  re-solving, not just by internal consistency.
+* `test_incremental.py` — the update path agrees with a cold solve.
+* `test_large.py` — the matrix-free driver matches the dense one, and nothing
+  densifies at scale.
+* `test_properties.py` — Hypothesis, for the feasibility invariants.
+* `test_solvers.py`, `test_evaluate.py`, `test_cli.py` — the rest.
+
+Numbers quoted in the README come from `benchmarks/measure.py`.  If you change
+something that moves them, re-run it and update the tables rather than leaving
+a claim that used to be true.
+
+## Style
+
+`ruff check` and `ruff format` with the settings in `pyproject.toml`.  Comments
+explain *why*, not what; the code already says what.
+
+## Cutting a release
+
+1. Update `CHANGELOG.md` and bump the version in `pyproject.toml`,
+   `CITATION.cff` and `src/mixingmatrix/__init__.py`.
+2. Build and validate:
+
+   ```bash
+   python -m build
+   python -m twine check dist/*
+   ```
+
+3. Install the wheel into a clean environment and import it from there, not
+   from the source tree — this is what catches a packaging mistake before it
+   reaches an index where a version can never be replaced.
+4. Rehearse on TestPyPI, install from it, then upload to PyPI.
+5. Tag and release:
+
+   ```bash
+   git tag -a vX.Y.Z -m "mixingmatrix X.Y.Z"
+   git push origin vX.Y.Z
+   gh release create vX.Y.Z --title "mixingmatrix X.Y.Z" --generate-notes
+   ```
+
+If the repository is linked to Zenodo, a DOI is minted from the release
+automatically; `.zenodo.json` controls its metadata. The link must be in place
+*before* the release is created, as Zenodo only sees releases made after it.

mixingmatrix-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Raghuram
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mixingmatrix-0.2.0/MANIFEST.in

@@ -0,0 +1,20 @@
+# The sdist doubles as the archived artefact a journal submission points at, so
+# it carries the documentation, the paper and the community files -- not just
+# the importable code.
+include LICENSE
+include README.md
+include CHANGELOG.md
+include CITATION.cff
+include CONTRIBUTING.md
+include CODE_OF_CONDUCT.md
+include paper.md
+include paper.bib
+recursive-include docs *.md
+recursive-include examples *.py
+recursive-include benchmarks *.py *.md
+recursive-include tests *.py
+
+# Build and test cruft.
+global-exclude *.py[cod]
+global-exclude __pycache__
+prune src/mixingmatrix.egg-info