exactcond 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,86 @@
1
+ Metadata-Version: 2.4
2
+ Name: exactcond
3
+ Version: 0.2.0
4
+ Summary: Exact conditional reference distributions for tables and selected linear models
5
+ Author-email: "M. Theodor Loots" <theo.loots@gmail.com>
6
+ License: GPL-3.0-or-later
7
+ Project-URL: Homepage, https://github.com/mtloots/exactcond
8
+ Requires-Python: >=3.8
9
+ Description-Content-Type: text/markdown
10
+
11
+ # exactcond
12
+
13
+ Exact conditional reference distributions for tables and for linear models whose structure was
14
+ selected from the same data. Pure C back-end, shared byte for byte with the Python package of the
15
+ same name.
16
+
17
+ ## The one idea
18
+
19
+ A nuisance parameter is removed by conditioning on a statistic sufficient for it. What survives
20
+ carries no nuisance, so a reference drawn from it is exact at every sample size, with no appeal to
21
+ an asymptotic approximation and no bootstrap.
22
+
23
+ ## Two conditionings
24
+
25
+ **Both margins fixed.** In an r by c table the row and column totals are sufficient for the
26
+ marginal parameters, so conditioning on both leaves the Fisher-Yates law. It is sampled exactly,
27
+ and by an argument needing no appeal to Patefield's algorithm: the conditional law given both
28
+ margins is precisely the law of the table formed by pairing a fixed vector of row labels with a
29
+ uniformly random permutation of the column labels.
30
+
31
+ **Direction only.** Under a null in which the tested structure does not enter the data-generating
32
+ process, a statistic invariant to the regression coefficients and to the error scale depends on the
33
+ data only through the residual direction, which is uniform on the sphere of the residual space
34
+ whatever those nuisance parameters are. The reference stays exact when the statistic is maximised
35
+ over candidates, and so remains valid after a spatial weight matrix has been selected by fit.
36
+
37
+ ## Why the dispersion statistic
38
+
39
+ For nominal rows against an ordinal column, `stat = "dispersion"` is the between-row dispersion of
40
+ the mean score. Its components are the row mean scores themselves, so where the column score is a
41
+ signed difference the statistic is assembled out of the very contrasts being reported, and a reader
42
+ can follow the arithmetic from the table to the test. A rank test on such a variable answers a
43
+ different question, and answers it on data that are mostly ties.
44
+
45
+ ## Example
46
+
47
+ ```r
48
+ library(exactcond)
49
+ tab <- matrix(c(3, 228, 72, 10, 548, 67, 7, 276, 131), nrow = 3, byrow = TRUE)
50
+ ec_rc_test(tab, v = c(-1, 0, 1), B = 200000, seed = 4207)
51
+ ```
52
+
53
+ ## Notes on honesty
54
+
55
+ `ec_rc_test()` reports the largest null draw alongside the p-value, and says so when the observed
56
+ statistic exceeded every draw: that is a resolution floor, not a p-value, and the two should not be
57
+ written the same way. Every p-value comes with a Wilson interval for the Monte Carlo error in the
58
+ number of draws actually taken.
59
+
60
+ `ec_rc_two()` samples the paired law directly rather than convolving two separately estimated null
61
+ distributions. The convolution needs the observed score to be reachable under no association; for a
62
+ strongly associated table it is not, at any number of draws, and the failure arrives precisely when
63
+ both tables are strongly associated, which is the case in which the correct answer is a large
64
+ p-value.
65
+
66
+ `ec_multi_test()` answers "does ANY of these items differ across the groups" with ONE exact test:
67
+ the sum or the maximum over items of a per-item statistic, referred to the law obtained by permuting
68
+ the group labels across respondents or, when a cluster is given, across clusters that keep their
69
+ members. This is Pesarin's nonparametric combination with the direct and Tippett combining functions: one
70
+ p-value for the whole battery, so nothing to correct for multiplicity, and the per-item statistics
71
+ referred to the same permutations for a closed follow-up in the manner of Westfall and Young. The sum has power against
72
+ many items moving a little, the maximum against one moving a lot, and the cluster permutation is the
73
+ exact treatment of an interviewer or fieldworker effect, at negligible cost when there is none.
74
+
75
+ `ec_diffnet_test()` carries the same idea to a network. A differential network normally rests on a
76
+ Gaussian null; here the edge statistic is unchanged and only its reference is, from a chi-square to
77
+ a permutation of the group labels, which is exact whatever the marginals are and distribution-free
78
+ on ranks. One p-value for the whole network, per-edge p-values from the same permutations, and a
79
+ cluster permutation for interviewer effects. The route is the established one (NetCoMi's `diffnet`,
80
+ the Network Comparison Test, exchangeability blocks); what is here is the implementation.
81
+
82
+ ## Reproducibility
83
+
84
+ Every replicate draws from its own stream, seeded from the base seed and the replicate index, so
85
+ results do not depend on the thread count and are identical across the R and Python fronts.
86
+ `parity_exactcond.sh` checks that agreement to fifteen significant figures.
@@ -0,0 +1,76 @@
1
+ # exactcond
2
+
3
+ Exact conditional reference distributions for tables and for linear models whose structure was
4
+ selected from the same data. Pure C back-end, shared byte for byte with the Python package of the
5
+ same name.
6
+
7
+ ## The one idea
8
+
9
+ A nuisance parameter is removed by conditioning on a statistic sufficient for it. What survives
10
+ carries no nuisance, so a reference drawn from it is exact at every sample size, with no appeal to
11
+ an asymptotic approximation and no bootstrap.
12
+
13
+ ## Two conditionings
14
+
15
+ **Both margins fixed.** In an r by c table the row and column totals are sufficient for the
16
+ marginal parameters, so conditioning on both leaves the Fisher-Yates law. It is sampled exactly,
17
+ and by an argument needing no appeal to Patefield's algorithm: the conditional law given both
18
+ margins is precisely the law of the table formed by pairing a fixed vector of row labels with a
19
+ uniformly random permutation of the column labels.
20
+
21
+ **Direction only.** Under a null in which the tested structure does not enter the data-generating
22
+ process, a statistic invariant to the regression coefficients and to the error scale depends on the
23
+ data only through the residual direction, which is uniform on the sphere of the residual space
24
+ whatever those nuisance parameters are. The reference stays exact when the statistic is maximised
25
+ over candidates, and so remains valid after a spatial weight matrix has been selected by fit.
26
+
27
+ ## Why the dispersion statistic
28
+
29
+ For nominal rows against an ordinal column, `stat = "dispersion"` is the between-row dispersion of
30
+ the mean score. Its components are the row mean scores themselves, so where the column score is a
31
+ signed difference the statistic is assembled out of the very contrasts being reported, and a reader
32
+ can follow the arithmetic from the table to the test. A rank test on such a variable answers a
33
+ different question, and answers it on data that are mostly ties.
34
+
35
+ ## Example
36
+
37
+ ```r
38
+ library(exactcond)
39
+ tab <- matrix(c(3, 228, 72, 10, 548, 67, 7, 276, 131), nrow = 3, byrow = TRUE)
40
+ ec_rc_test(tab, v = c(-1, 0, 1), B = 200000, seed = 4207)
41
+ ```
42
+
43
+ ## Notes on honesty
44
+
45
+ `ec_rc_test()` reports the largest null draw alongside the p-value, and says so when the observed
46
+ statistic exceeded every draw: that is a resolution floor, not a p-value, and the two should not be
47
+ written the same way. Every p-value comes with a Wilson interval for the Monte Carlo error in the
48
+ number of draws actually taken.
49
+
50
+ `ec_rc_two()` samples the paired law directly rather than convolving two separately estimated null
51
+ distributions. The convolution needs the observed score to be reachable under no association; for a
52
+ strongly associated table it is not, at any number of draws, and the failure arrives precisely when
53
+ both tables are strongly associated, which is the case in which the correct answer is a large
54
+ p-value.
55
+
56
+ `ec_multi_test()` answers "does ANY of these items differ across the groups" with ONE exact test:
57
+ the sum or the maximum over items of a per-item statistic, referred to the law obtained by permuting
58
+ the group labels across respondents or, when a cluster is given, across clusters that keep their
59
+ members. This is Pesarin's nonparametric combination with the direct and Tippett combining functions: one
60
+ p-value for the whole battery, so nothing to correct for multiplicity, and the per-item statistics
61
+ referred to the same permutations for a closed follow-up in the manner of Westfall and Young. The sum has power against
62
+ many items moving a little, the maximum against one moving a lot, and the cluster permutation is the
63
+ exact treatment of an interviewer or fieldworker effect, at negligible cost when there is none.
64
+
65
+ `ec_diffnet_test()` carries the same idea to a network. A differential network normally rests on a
66
+ Gaussian null; here the edge statistic is unchanged and only its reference is, from a chi-square to
67
+ a permutation of the group labels, which is exact whatever the marginals are and distribution-free
68
+ on ranks. One p-value for the whole network, per-edge p-values from the same permutations, and a
69
+ cluster permutation for interviewer effects. The route is the established one (NetCoMi's `diffnet`,
70
+ the Network Comparison Test, exchangeability blocks); what is here is the implementation.
71
+
72
+ ## Reproducibility
73
+
74
+ Every replicate draws from its own stream, seeded from the base seed and the replicate index, so
75
+ results do not depend on the thread count and are identical across the R and Python fronts.
76
+ `parity_exactcond.sh` checks that agreement to fifteen significant figures.
@@ -0,0 +1,15 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "exactcond"
7
+ version = "0.2.0"
8
+ description = "Exact conditional reference distributions for tables and selected linear models"
9
+ readme = "README.md"
10
+ requires-python = ">=3.8"
11
+ license = {text = "GPL-3.0-or-later"}
12
+ authors = [{name = "M. Theodor Loots", email = "theo.loots@gmail.com"}]
13
+
14
+ [project.urls]
15
+ Homepage = "https://github.com/mtloots/exactcond"
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,34 @@
1
+ """Build the shared exact-conditional C back-end at INSTALL time, as an Extension.
2
+
3
+ The earlier setup.py subclassed setuptools' build_py and compiled with distutils' new_compiler.
4
+ distutils is gone from Python 3.12 and current setuptools no longer supports that subclassing, so
5
+ pip failed with "type object 'build_py' has no attribute '__getattr__'" on every platform (12 Oct
6
+ 2026). Declaring the sources as an Extension hands the compile to pip, inside build isolation, and
7
+ lets real platform wheels be built, after which users need no compiler at all. Same pattern as
8
+ arcstat-py.
9
+
10
+ init.c is excluded deliberately: it is R's symbol-registration glue and references R headers.
11
+ """
12
+ import glob
13
+ import os
14
+
15
+ from setuptools import Extension, setup
16
+
17
+ # Paths must be RELATIVE to setup.py: setuptools rejects absolute source paths.
18
+ SRC = os.path.join("src", "exactcond")
19
+
20
+ sources = sorted(
21
+ s for s in glob.glob(os.path.join(SRC, "*.c"))
22
+ if os.path.basename(s) != "init.c"
23
+ )
24
+
25
+ # Leading underscore: it carries no PyInit and is loaded with ctypes.CDLL by _core.py.
26
+ ext = Extension(
27
+ name="exactcond._libexactcond",
28
+ sources=sources,
29
+ include_dirs=[SRC],
30
+ libraries=[] if os.name == "nt" else ["m"],
31
+ optional=False,
32
+ )
33
+
34
+ setup(ext_modules=[ext])
@@ -0,0 +1,15 @@
1
+ """exactcond: exact conditional reference distributions for tables and selected linear models.
2
+
3
+ Reference distributions exact at every sample size, obtained by conditioning on a statistic
4
+ sufficient for the nuisance parameter rather than by an asymptotic approximation: the Fisher-Yates
5
+ law of a two-way table given both margins, sampled by random permutation of the column labels; a
6
+ two-table form conditioning further on the total score to cancel a common association parameter;
7
+ and the uniform law of the residual direction on the sphere, which stays exact when the statistic
8
+ is maximised over candidate structures and so remains valid after selection by fit. ctypes binding
9
+ to a shared C back end also used by the R package 'exactcond'. Pure standard library.
10
+ """
11
+ from ._core import (ec_scores, ec_rc_null, ec_rc_test, ec_rc_two, ec_sphere_maxquad, ec_pvalue,
12
+ ec_multi_test, ec_diffnet_test)
13
+ __all__ = ["ec_scores", "ec_rc_null", "ec_rc_test", "ec_rc_two", "ec_sphere_maxquad", "ec_pvalue",
14
+ "ec_multi_test", "ec_diffnet_test"]
15
+ __version__ = "0.2.0"
@@ -0,0 +1,356 @@
1
+ """exactcond: Python (ctypes) binding to the shared C back-end. Pure standard library.
2
+
3
+ The same C sources back the R package 'exactcond'; the parity harness in the repository requires
4
+ every exported quantity to agree byte for byte between the two fronts, which holds because the
5
+ random stream lives in C and is seeded per replicate rather than drawn from either language's
6
+ generator.
7
+
8
+ LOADING has two paths, as in the other packages in this programme. An INSTALLED package carries a
9
+ library that pip built at install time, so no compiler is needed here. A SOURCE CHECKOUT has the
10
+ .c file but no library, so it is compiled on demand into a writable cache directory, which keeps
11
+ the edit-a-C-file-and-re-import loop working without writing into site-packages.
12
+ """
13
+ import ctypes as _ct
14
+ import glob as _glob
15
+ import os as _os
16
+ import sys as _sys
17
+
18
+ _here = _os.path.dirname(_os.path.abspath(__file__))
19
+
20
+
21
+ def _find_built():
22
+ for pat in ("_libexactcond*.so", "_libexactcond*.dylib", "_libexactcond*.pyd",
23
+ "_libexactcond*.dll"):
24
+ hits = sorted(_glob.glob(_os.path.join(_here, pat)))
25
+ if hits:
26
+ return hits[0]
27
+ return None
28
+
29
+
30
+ def _cache_dir():
31
+ if _os.access(_here, _os.W_OK):
32
+ return _here
33
+ base = _os.environ.get("XDG_CACHE_HOME") or _os.path.join(_os.path.expanduser("~"), ".cache")
34
+ d = _os.path.join(base, "exactcond")
35
+ _os.makedirs(d, exist_ok=True)
36
+ return d
37
+
38
+
39
+ def _ext():
40
+ return "dll" if _sys.platform == "win32" else ("dylib" if _sys.platform == "darwin" else "so")
41
+
42
+
43
+ def _compile_from_source():
44
+ import subprocess as _sp
45
+ srcs = sorted(f for f in _glob.glob(_os.path.join(_here, "*.c"))
46
+ if _os.path.basename(f) != "init.c")
47
+ if not srcs:
48
+ return None
49
+ out = _os.path.join(_cache_dir(), "libexactcond." + _ext())
50
+ cc = _os.environ.get("CC") or "cc"
51
+ try:
52
+ _sp.check_call([cc, "-O2", "-fPIC", "-shared", "-I", _here] + srcs + ["-lm", "-o", out])
53
+ except (OSError, _sp.CalledProcessError) as exc:
54
+ raise ImportError(
55
+ "exactcond could not build its C back-end from the bundled sources (%s). In an "
56
+ "installed copy this should never happen, because pip builds the library at install "
57
+ "time. In a source checkout a C compiler is required; set CC if it is not 'cc'." % exc)
58
+ return out
59
+
60
+
61
+ _libpath = _find_built()
62
+ if _libpath is None:
63
+ _cached = _os.path.join(_cache_dir(), "libexactcond." + _ext())
64
+ _libpath = _cached if _os.path.exists(_cached) else _compile_from_source()
65
+ if _libpath is None:
66
+ raise ImportError("exactcond: no compiled back-end and no C sources to build one from.")
67
+
68
+ _lib = _ct.CDLL(_libpath)
69
+ _dp, _ip = _ct.POINTER(_ct.c_double), _ct.POINTER(_ct.c_int)
70
+
71
+ for _f, _sig in (
72
+ ("C_ec_rc_null", [_ip, _ip, _ip, _ip, _ip, _ip, _dp, _dp, _ip, _dp]),
73
+ ("C_ec_rc_test", [_ip, _ip, _ip, _ip, _ip, _dp, _dp, _ip, _dp]),
74
+ ("C_ec_rc_two", [_ip, _ip, _ip, _ip, _ip, _ip, _ip, _dp, _dp, _dp, _ip, _dp]),
75
+ ("C_ec_sphere_maxquad", [_dp, _dp, _ip, _ip, _ip, _ip, _dp]),
76
+ ("C_ec_pvalue", [_dp, _dp, _ip, _dp]),
77
+ ("C_ec_multi", [_ip, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _dp, _dp, _dp]),
78
+ ("C_ec_diffnet", [_dp, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _ip, _dp, _dp, _dp]),
79
+ ):
80
+ getattr(_lib, _f).argtypes = _sig
81
+ getattr(_lib, _f).restype = None
82
+
83
+ _STATS = {"linlin": 0, "dispersion": 1, "pearson": 2, "lr": 3}
84
+
85
+
86
+ def _d(xs):
87
+ xs = list(xs)
88
+ return (_ct.c_double * len(xs))(*[float(x) for x in xs])
89
+
90
+
91
+ def _i(xs):
92
+ xs = list(xs)
93
+ return (_ct.c_int * len(xs))(*[int(x) for x in xs])
94
+
95
+
96
+ def _one(x):
97
+ return _ct.byref(_ct.c_int(int(x)))
98
+
99
+
100
+ def _oned(x):
101
+ return _ct.byref(_ct.c_double(float(x)))
102
+
103
+
104
+ def _flat(x):
105
+ """Accept a nested sequence or a flat one and return (rows, cols, flat row-major list)."""
106
+ rows = list(x)
107
+ if not rows:
108
+ raise ValueError("empty table")
109
+ if not hasattr(rows[0], "__len__"):
110
+ raise ValueError("expected a two-dimensional table")
111
+ c = len(rows[0])
112
+ if any(len(r) != c for r in rows):
113
+ raise ValueError("ragged table")
114
+ return len(rows), c, [int(v) for r in rows for v in r]
115
+
116
+
117
+ def ec_scores(k, centre=False):
118
+ """Integer scores for an ordinal margin.
119
+
120
+ The dispersion and linear-by-linear statistics are unchanged in distribution by an affine
121
+ transformation of the scores, so centring alters the reported statistic but never the p-value.
122
+ """
123
+ s = [float(i) for i in range(int(k))]
124
+ if centre:
125
+ m = sum(s) / len(s)
126
+ s = [v - m for v in s]
127
+ return s
128
+
129
+
130
+ def _scores(u, v, r, c):
131
+ u = ec_scores(r) if u is None else [float(t) for t in u]
132
+ v = ec_scores(c) if v is None else [float(t) for t in v]
133
+ if len(u) != r:
134
+ raise ValueError("u must have one score per row, so length %d" % r)
135
+ if len(v) != c:
136
+ raise ValueError("v must have one score per column, so length %d" % c)
137
+ return u, v
138
+
139
+
140
+ def ec_rc_null(rows, cols, stat="dispersion", u=None, v=None, B=20000, seed=1):
141
+ """Draws of a table statistic from the law conditional on BOTH margins."""
142
+ rows = [int(t) for t in rows]
143
+ cols = [int(t) for t in cols]
144
+ if sum(rows) != sum(cols):
145
+ raise ValueError("the row and column totals must agree")
146
+ uu, vv = _scores(u, v, len(rows), len(cols))
147
+ out = (_ct.c_double * int(B))()
148
+ _lib.C_ec_rc_null(_i(rows), _i(cols), _one(len(rows)), _one(len(cols)), _one(B),
149
+ _one(_STATS[stat]), _d(uu), _d(vv), _one(seed), out)
150
+ return list(out)
151
+
152
+
153
+ def ec_rc_test(x, stat="dispersion", u=None, v=None, B=20000, seed=1):
154
+ """Exact conditional test in a two-way table, conditioning on both margins.
155
+
156
+ Returns a dict with the observed statistic, the exact p-value and its mid-p companion, a
157
+ Wilson interval reflecting the Monte Carlo error in B draws, the largest null draw, and the
158
+ row mean scores, which for the dispersion statistic ARE the contrasts being reported.
159
+ """
160
+ r, c, flat = _flat(x)
161
+ if any(t < 0 for t in flat):
162
+ raise ValueError("x must hold non-negative counts")
163
+ uu, vv = _scores(u, v, r, c)
164
+ out = (_ct.c_double * 6)()
165
+ _lib.C_ec_rc_test(_i(flat), _one(r), _one(c), _one(B), _one(_STATS[stat]),
166
+ _d(uu), _d(vv), _one(seed), out)
167
+ n = [sum(flat[i * c:(i + 1) * c]) for i in range(r)]
168
+ means = [sum(vv[j] * flat[i * c + j] for j in range(c)) / n[i] if n[i] else float("nan")
169
+ for i in range(r)]
170
+ return {"statistic": out[0], "p_value": out[1], "midp": out[2], "mc_lower": out[3],
171
+ "mc_upper": out[4], "max_null": out[5], "row_means": means, "n": n,
172
+ "B": int(B), "stat": stat}
173
+
174
+
175
+ def ec_rc_two(t1, t2, u=None, v=None, B=20000, burn=20000, thin=20, alpha=0.05, seed=1):
176
+ """Exact conditional test that two tables share a linear-by-linear association parameter.
177
+
178
+ Under the null of a common parameter the joint density carries it only through the total score,
179
+ which the conditioning fixes, so it leaves the density altogether. The law is sampled by moves
180
+ making a two by two swap in each table with equal and opposite score changes, starting at the
181
+ observed pair -- which is what keeps it usable when both tables are strongly associated, the
182
+ case in which convolving two separately estimated null distributions fails outright.
183
+ """
184
+ r, c, f1 = _flat(t1)
185
+ r2, c2, f2 = _flat(t2)
186
+ if (r, c) != (r2, c2):
187
+ raise ValueError("the two tables must have the same dimension")
188
+ if r < 2 or c < 2:
189
+ raise ValueError("both tables must have at least two rows and two columns")
190
+ uu, vv = _scores(u, v, r, c)
191
+ if any(abs(t - round(t)) > 1e-9 for t in uu + vv):
192
+ raise ValueError("ec_rc_two needs integer scores, so the total score lies on a lattice")
193
+ out = (_ct.c_double * 5)()
194
+ _lib.C_ec_rc_two(_i(f1), _i(f2), _one(r), _one(c), _one(B), _one(burn), _one(thin),
195
+ _d(uu), _d(vv), _oned(alpha), _one(seed), out)
196
+ if out[0] < 0:
197
+ raise RuntimeError("the sampler could not be initialised on these tables")
198
+ return {"midp": out[0], "size": out[1], "T1": out[2], "S": out[3], "accept": out[4],
199
+ "B": int(B), "burn": int(burn), "thin": int(thin), "alpha": float(alpha)}
200
+
201
+
202
+ def ec_sphere_maxquad(M, A, B=20000, seed=1):
203
+ """Exact null of a quadratic form maximised over candidate structures.
204
+
205
+ M is the residual maker; A is one symmetric matrix or a list of them. Each draw takes the
206
+ maximum INSIDE the draw, which is what makes the reference valid after a structure has been
207
+ chosen by fit.
208
+ """
209
+ rows = [list(r) for r in M]
210
+ n = len(rows)
211
+ if any(len(r) != n for r in rows):
212
+ raise ValueError("M must be square")
213
+ if not hasattr(A[0][0], "__len__"):
214
+ A = [A]
215
+ flat = []
216
+ for a in A:
217
+ ar = [list(r) for r in a]
218
+ if len(ar) != n or any(len(r) != n for r in ar):
219
+ raise ValueError("every candidate must be n by n")
220
+ for i in range(n):
221
+ for j in range(n):
222
+ flat.append(float(ar[i][j]))
223
+ Mf = [float(rows[i][j]) for i in range(n) for j in range(n)]
224
+ out = (_ct.c_double * int(B))()
225
+ _lib.C_ec_sphere_maxquad(_d(Mf), _d(flat), _one(n), _one(len(A)), _one(B), _one(seed), out)
226
+ return list(out)
227
+
228
+
229
+ def ec_multi_test(x, group, cluster=None, stat="pearson", B=20000, seed=1):
230
+ """One exact test for a battery of categorical items across groups.
231
+
232
+ x is a sequence of items (each a sequence of integer codes from zero, or from one, over the
233
+ respondents) or a sequence of respondent rows; group gives each respondent's group; cluster,
234
+ if given, each respondent's cluster, every cluster inside one group, in which case the labels
235
+ are permuted across clusters. The sum and the maximum over items of the per-item statistic
236
+ (Pearson on the group-by-level table, or the dispersion of group mean scores) are each
237
+ referred to the same permutations, giving one p-value for the battery, and the per-item
238
+ statistics are referred to those permutations too, for a closed follow-up.
239
+ """
240
+ rows = [list(r) for r in x]
241
+ n_g = len(list(group))
242
+ if len(rows) == n_g and (not rows or hasattr(rows[0], "__len__")) and len(rows[0]) != n_g:
243
+ items = [[int(r[j]) for r in rows] for j in range(len(rows[0]))] # respondent rows
244
+ else:
245
+ items = [[int(v) for v in r] for r in rows] # item columns
246
+ n = len(items[0]); m = len(items)
247
+ if any(len(c) != n for c in items) or n_g != n:
248
+ raise ValueError("every item and the group vector must have one entry per respondent")
249
+ items = [[v - 1 for v in c] if min(c) == 1 else c for c in items]
250
+ if any(min(c) < 0 for c in items):
251
+ raise ValueError("items must be coded from zero or one")
252
+ k = [max(c) + 1 for c in items]
253
+ levels = sorted(set(group)); grp = [levels.index(v) for v in group]; g = len(levels)
254
+ if g < 2:
255
+ raise ValueError("group must have at least two levels")
256
+ if cluster is None:
257
+ cl = [0] * n; ncl = 0
258
+ else:
259
+ cluster = list(cluster)
260
+ if len(cluster) != n:
261
+ raise ValueError("cluster must have one entry per respondent")
262
+ clev = sorted(set(cluster)); cl = [clev.index(v) for v in cluster]; ncl = len(clev)
263
+ seen = {}
264
+ for c, a in zip(cl, grp):
265
+ if seen.setdefault(c, a) != a:
266
+ raise ValueError("every cluster must lie inside one group")
267
+ which = {"pearson": 0, "dispersion": 1}[stat]
268
+ out = (_ct.c_double * 12)(); ist = (_ct.c_double * m)(); ip = (_ct.c_double * m)()
269
+ _lib.C_ec_multi(_i([v for c in items for v in c]), _one(n), _one(m), _one(g), _i(grp), _i(k),
270
+ _i(cl), _one(ncl), _one(which), _one(B), _one(seed), out, ist, ip)
271
+ def part(i, j, mx): # obs at i; p, midp, lo, hi from j
272
+ return {"statistic": out[i], "p_value": out[j], "midp": out[j + 1],
273
+ "mc_lower": out[j + 2], "mc_upper": out[j + 3], "max_null": mx}
274
+ return {"sum": part(0, 2, out[10]), "max": part(1, 6, out[11]),
275
+ "items": [{"statistic": ist[j], "p_value": ip[j]} for j in range(m)],
276
+ "n": [grp.count(a) for a in range(g)], "clusters": ncl, "B": int(B), "stat": stat,
277
+ "exchangeable": "clusters" if ncl > 0 else "respondents"}
278
+
279
+
280
+ def _ranks(v):
281
+ """Average ranks, ties shared, which is what makes the correlation Spearman's."""
282
+ order = sorted(range(len(v)), key=lambda i: v[i])
283
+ out = [0.0] * len(v)
284
+ i = 0
285
+ while i < len(order):
286
+ j = i
287
+ while j + 1 < len(order) and v[order[j + 1]] == v[order[i]]:
288
+ j += 1
289
+ r = (i + j) / 2.0 + 1.0
290
+ for t in range(i, j + 1):
291
+ out[order[t]] = r
292
+ i = j + 1
293
+ return out
294
+
295
+
296
+ def ec_diffnet_test(x, group, cluster=None, rank=True, B=20000, seed=1):
297
+ """A differential network without a Gaussian assumption.
298
+
299
+ x is a sequence of variables (each a sequence over the respondents) or of respondent rows.
300
+ The edge statistic is the heterogeneity of the Fisher-transformed correlations across groups,
301
+ the same quantity the Gaussian theory uses, referred to the law obtained by permuting the group
302
+ labels rather than to a chi-square; with rank=True the correlations are Spearman's and the
303
+ procedure is distribution-free. Returns the sum and the maximum over edges, each with one
304
+ p-value for the whole network, and the per-edge statistics and p-values from those same
305
+ permutations for a closed follow-up.
306
+ """
307
+ rows = [list(r) for r in x]
308
+ n_g = len(list(group))
309
+ if len(rows) == n_g and rows and hasattr(rows[0], "__len__") and len(rows[0]) != n_g:
310
+ cols = [[float(r[j]) for r in rows] for j in range(len(rows[0]))]
311
+ else:
312
+ cols = [[float(v) for v in r] for r in rows]
313
+ p = len(cols); n = len(cols[0])
314
+ if p < 2:
315
+ raise ValueError("x must hold at least two variables")
316
+ if any(len(c) != n for c in cols) or n_g != n:
317
+ raise ValueError("every variable and the group vector must have one entry per respondent")
318
+ if rank:
319
+ cols = [_ranks(c) for c in cols]
320
+ levels = sorted(set(group)); grp = [levels.index(v) for v in group]; g = len(levels)
321
+ if g < 2:
322
+ raise ValueError("group must have at least two levels")
323
+ if min(grp.count(a) for a in range(g)) < 4:
324
+ raise ValueError("every group needs at least four respondents")
325
+ if cluster is None:
326
+ cl = [0] * n; ncl = 0
327
+ else:
328
+ cluster = list(cluster)
329
+ if len(cluster) != n:
330
+ raise ValueError("cluster must have one entry per respondent")
331
+ clev = sorted(set(cluster)); cl = [clev.index(v) for v in cluster]; ncl = len(clev)
332
+ seen = {}
333
+ for c, a in zip(cl, grp):
334
+ if seen.setdefault(c, a) != a:
335
+ raise ValueError("every cluster must lie inside one group")
336
+ m = p * (p - 1) // 2
337
+ out = (_ct.c_double * 12)(); est = (_ct.c_double * m)(); ep = (_ct.c_double * m)()
338
+ _lib.C_ec_diffnet(_d([v for c in cols for v in c]), _one(n), _one(p), _one(g), _i(grp),
339
+ _i(cl), _one(ncl), _one(B), _one(seed), out, est, ep)
340
+ def part(i, j, mx):
341
+ return {"statistic": out[i], "p_value": out[j], "midp": out[j + 1],
342
+ "mc_lower": out[j + 2], "mc_upper": out[j + 3], "max_null": mx}
343
+ pairs = [(a, b) for a in range(p) for b in range(a + 1, p)]
344
+ return {"sum": part(0, 2, out[10]), "max": part(1, 6, out[11]),
345
+ "edges": [{"from": a, "to": b, "statistic": est[e], "p_value": ep[e]}
346
+ for e, (a, b) in enumerate(pairs)],
347
+ "n": [grp.count(a) for a in range(g)], "clusters": ncl, "B": int(B),
348
+ "ranked": bool(rank), "exchangeable": "clusters" if ncl > 0 else "respondents"}
349
+
350
+
351
+ def ec_pvalue(obs, null):
352
+ """Tail probabilities from an observed value and a sample from the null."""
353
+ null = [float(t) for t in null]
354
+ out = (_ct.c_double * 4)()
355
+ _lib.C_ec_pvalue(_oned(obs), _d(null), _one(len(null)), out)
356
+ return {"p_value": out[0], "midp": out[1], "mc_lower": out[2], "mc_upper": out[3]}