privpy 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. privpy-0.1.0/DESIGN.md +135 -0
  2. privpy-0.1.0/MANIFEST.in +5 -0
  3. privpy-0.1.0/PKG-INFO +186 -0
  4. privpy-0.1.0/README.md +159 -0
  5. privpy-0.1.0/RELEASING.md +58 -0
  6. privpy-0.1.0/TESTING.md +60 -0
  7. privpy-0.1.0/VALIDATION.md +43 -0
  8. privpy-0.1.0/examples/orders.json +6 -0
  9. privpy-0.1.0/examples/orders.py +53 -0
  10. privpy-0.1.0/pyproject.toml +28 -0
  11. privpy-0.1.0/requirements-test.txt +3 -0
  12. privpy-0.1.0/scripts/check_dist.py +91 -0
  13. privpy-0.1.0/scripts/check_release.py +16 -0
  14. privpy-0.1.0/setup.cfg +4 -0
  15. privpy-0.1.0/setup.py +55 -0
  16. privpy-0.1.0/src/privpy/__init__.py +15 -0
  17. privpy-0.1.0/src/privpy/_compiler.py +203 -0
  18. privpy-0.1.0/src/privpy/_core.py +207 -0
  19. privpy-0.1.0/src/privpy/_native.py +82 -0
  20. privpy-0.1.0/src/privpy/_version.py +1 -0
  21. privpy-0.1.0/src/privpy/_wire.py +69 -0
  22. privpy-0.1.0/src/privpy/build.py +49 -0
  23. privpy-0.1.0/src/privpy/errors.py +37 -0
  24. privpy-0.1.0/src/privpy/intrinsics.py +23 -0
  25. privpy-0.1.0/src/privpy/native/runtime.cpp +951 -0
  26. privpy-0.1.0/src/privpy.egg-info/PKG-INFO +186 -0
  27. privpy-0.1.0/src/privpy.egg-info/SOURCES.txt +40 -0
  28. privpy-0.1.0/src/privpy.egg-info/dependency_links.txt +1 -0
  29. privpy-0.1.0/src/privpy.egg-info/requires.txt +4 -0
  30. privpy-0.1.0/src/privpy.egg-info/top_level.txt +1 -0
  31. privpy-0.1.0/tests/__init__.py +13 -0
  32. privpy-0.1.0/tests/csv_oracle.py +16 -0
  33. privpy-0.1.0/tests/fixtures.py +247 -0
  34. privpy-0.1.0/tests/helpers.py +26 -0
  35. privpy-0.1.0/tests/strategies.py +36 -0
  36. privpy-0.1.0/tests/test_api.py +406 -0
  37. privpy-0.1.0/tests/test_compiler.py +210 -0
  38. privpy-0.1.0/tests/test_native.py +225 -0
  39. privpy-0.1.0/tests/test_packaging.py +90 -0
  40. privpy-0.1.0/tests/test_properties.py +288 -0
  41. privpy-0.1.0/tests/test_regressions.py +33 -0
  42. privpy-0.1.0/tests/test_stateful.py +84 -0
privpy-0.1.0/DESIGN.md ADDED
@@ -0,0 +1,135 @@
1
+ # privpy — final API draft
2
+
3
+ A region owns private data. Protected functions compute on it. Explicit export authorizes disclosure to ordinary Python or an exact local file.
4
+
5
+ This is an API and execution design. It does not, by itself, establish hardware memory isolation.
6
+
7
+ ## Public API
8
+
9
+ ```python
10
+ with PrivateRegion() as region:
11
+ private_result = protected_function(public_input)
12
+ public_result = region.export(private_result)
13
+ region.export(private_result, to=Path("/absolute/output.csv"), transform=as_csv)
14
+ ```
15
+
16
+ ```python
17
+ @private_function
18
+ def protected_function(...):
19
+ ...
20
+
21
+ region.export(value, *, to=None, transform=None)
22
+ ```
23
+
24
+ There is no named policy, general-purpose load method, manual run method, format argument, or destination alias.
25
+
26
+ ## Region
27
+
28
+ The context manager owns private objects and their lifetime and establishes the active region. The surrounding Python block executes normally. Decorated functions execute in the protected runtime.
29
+
30
+ Every private reference belongs to exactly one region. References from another region are rejected. Closing the region, including on exception, invalidates all its references. Exported copies and files survive independently.
31
+
32
+ Ordinary Python arguments can be copied into computations. Their original copies remain public. Private source readers retrieve and parse their inputs inside the runtime; their source must itself be protected if confidentiality before retrieval matters.
33
+
34
+ ## Protected functions
35
+
36
+ Decorated functions accept supported public arguments and private references. Their implementation is compiled or interpreted by the protected runtime, never executed as an ordinary Python callback with plaintext inputs.
37
+
38
+ Calls use ordinary syntax and automatically dispatch to the active region. Calls between protected functions remain inside. A call without an active region fails.
39
+
40
+ All intermediate and returned objects remain private: scalars, strings, collections, arrays and supported dataframes. Comparisons, lengths, schemas and aggregates receive no automatic exemption. Branches and loops can use private conditions inside the runtime. Unsupported operations fail; there is no plaintext fallback to host Python.
41
+
42
+ Outside the runtime, results are opaque references. They can be passed to protected functions or exported. They cannot expose contents through conversion, iteration, indexing, truth testing, attribute access or serialization. Their representation is a fixed placeholder.
43
+
44
+ ## Export
45
+
46
+ `export` is the application's deliberate authorization to disclose. It checks mechanics, not whether the data is semantically safe to reveal. The application developer is trusted to choose outputs. Deliberately exporting an original secret is permitted.
47
+
48
+ Arguments:
49
+
50
+ | Argument | Meaning |
51
+ | --- | --- |
52
+ | `value` | A private reference owned by the active region |
53
+ | `transform` | Optional protected function accepting one value and returning a private result |
54
+ | `to=None` | Return an independent supported Python value |
55
+ | `to=Path(...)` | Write bytes directly to that absolute local file; return `None` |
56
+
57
+ The optional transform runs inside the region. Its result remains private until export. With no transform, the value must already be suitable for its destination. File destinations require bytes; there is no inferred serialization, encoding or encryption. Existing files are not overwritten. Aliases, relative paths, URLs, file objects and callbacks are not supported as destinations in the initial API.
58
+
59
+ ```python
60
+ region.export(value, to=path, transform=as_csv)
61
+ ```
62
+
63
+ is equivalent to:
64
+
65
+ ```python
66
+ encoded = as_csv(value)
67
+ region.export(encoded, to=path)
68
+ ```
69
+
70
+ Saved plaintext is disclosed. Confidential persistence uses a protected transform that serializes and encrypts to bytes, plus a protected reader that reverses it later. Key management and a secure encryption integration must exist before that feature can be claimed. File extensions never select these behaviors.
71
+
72
+ ## Target dataframe example
73
+
74
+ The following illustrates the eventual supported-integration API, not unrestricted pandas or SDK compatibility:
75
+
76
+ ```python
77
+ @private_function
78
+ def read_orders():
79
+ token = secrets.get("payments-token")
80
+ records = payments.fetch_orders(token)
81
+ return DataFrame(records)
82
+
83
+ @private_function
84
+ def summarize(orders):
85
+ paid = orders[orders["status"] == "paid"]
86
+ return paid.groupby("country", as_index=False)["amount"].sum()
87
+
88
+ @private_function
89
+ def as_csv(frame):
90
+ return frame.to_csv(index=False).encode("utf-8")
91
+
92
+ with PrivateRegion() as region:
93
+ totals = summarize(read_orders())
94
+ region.export(totals, to=Path("/reports/monthly.csv"), transform=as_csv)
95
+ ```
96
+
97
+ Source retrieval, credential handling and library operations require supported runtime integrations. Neither a decorator nor moving a pointer makes an arbitrary Python SDK safe to execute on private data.
98
+
99
+ ## Execution sketch
100
+
101
+ ```mermaid
102
+ flowchart TD
103
+ H[Ordinary Python: public inputs and private references]
104
+ R[Native runtime: region-owned objects and compiled functions]
105
+ S[Supported source readers]
106
+ T[Optional protected transform]
107
+ E[Explicit export: validate ownership and destination]
108
+ F[Absolute local file: exact bytes]
109
+ H -->|Protected function call| R
110
+ S --> R
111
+ R -->|Opaque reference| H
112
+ R --> T
113
+ T --> E
114
+ R -->|No transform| E
115
+ E -->|Independent ordinary value| H
116
+ E -->|Direct native write| F
117
+ ```
118
+
119
+ ## Initial implementation boundary
120
+
121
+ The first module uses a native C++ runtime with a restricted Python-syntax interpreter. Python compiles function source into a data-only representation. Private function evaluation, local variables, source parsing, results and file-output bytes live in native storage. The Python interface receives handles until explicit export.
122
+
123
+ Initial supported data: bounded integers, finite floats, booleans, null, UTF-8 strings, bytes, lists and string-keyed records. Initial computations include assignments, expressions, comparisons, branches, loops and calls between protected functions. File readers and JSON/CSV byte encoders provide a concrete end-to-end vertical slice.
124
+
125
+ Full dataframe engines, real secrets-manager SDKs, HTTP clients, encryption, arbitrary Python modules, asynchronous functions and custom Python classes are future integrations, not implicit compatibility promises.
126
+
127
+ The prototype targets accidental disclosure through supported language operations. It does not protect against hostile code with unrestricted native-memory access, a debugger, a compromised runtime, process dumps, timing/termination/error side channels or an application developer deliberately exporting secrets. Native resource release is not a guarantee of complete zeroization of allocator memory, registers or operating-system copies. Normal Python handles do not hold the plaintext object graph.
128
+
129
+ A production backend would need security review, memory-management hardening, further resource limits and a defined threat model. A strong enclave or process-isolation backend can use the same public API, but is not implemented by this prototype.
130
+
131
+ ## Background
132
+
133
+ - [Python context-manager semantics](https://docs.python.org/3/reference/compound_stmts.html#the-with-statement)
134
+ - [Native memory access through Python](https://docs.python.org/3/library/ctypes.html)
135
+ - [Information flow, including exceptional control flow](https://www.cs.cornell.edu/jif/doc/jif-3.3.0/label_checking.html)
@@ -0,0 +1,5 @@
1
+ include README.md DESIGN.md TESTING.md VALIDATION.md RELEASING.md requirements-test.txt
2
+ recursive-include src/privpy/native *.cpp
3
+ recursive-include tests *.py
4
+ recursive-include examples *.py *.json
5
+ recursive-include scripts *.py
privpy-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,186 @@
1
+ Metadata-Version: 2.4
2
+ Name: privpy
3
+ Version: 0.1.0
4
+ Summary: Experimental native private computation regions for Python
5
+ Home-page: https://github.com/technoyoda/privpy
6
+ Project-URL: Source, https://github.com/technoyoda/privpy
7
+ Project-URL: Issues, https://github.com/technoyoda/privpy/issues
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Programming Language :: Python :: 3 :: Only
11
+ Classifier: Programming Language :: C++
12
+ Classifier: Operating System :: MacOS
13
+ Classifier: Operating System :: POSIX :: Linux
14
+ Requires-Python: >=3.9
15
+ Description-Content-Type: text/markdown
16
+ Provides-Extra: test
17
+ Requires-Dist: hypothesis<7,>=6.100; extra == "test"
18
+ Requires-Dist: coverage<8,>=7; extra == "test"
19
+ Dynamic: classifier
20
+ Dynamic: description
21
+ Dynamic: description-content-type
22
+ Dynamic: home-page
23
+ Dynamic: project-url
24
+ Dynamic: provides-extra
25
+ Dynamic: requires-python
26
+ Dynamic: summary
27
+
28
+ # privpy
29
+
30
+ **Private computations, native-owned values, explicit export.**
31
+
32
+ Experimental same-process Python module. The runtime is C++17; the entire test suite is Python, using `unittest` and Hypothesis.
33
+
34
+ ## Install from PyPI
35
+
36
+ Install the latest release into your Python environment:
37
+
38
+ ```sh
39
+ python -m pip install privpy
40
+ ```
41
+
42
+ To upgrade an existing installation:
43
+
44
+ ```sh
45
+ python -m pip install --upgrade privpy
46
+ ```
47
+
48
+ Prebuilt wheels include the native runtime for CPython 3.9–3.14 on macOS 14+ and glibc 2.28+ Linux, on Intel and ARM. These wheels do not require a local C++ compiler. If pip selects a source archive instead, the source-build requirements below apply. Windows and PyPy are not supported yet.
49
+
50
+ ## Example
51
+
52
+ Save protected functions in a Python file so their source can be inspected.
53
+
54
+ ```python
55
+ from pathlib import Path
56
+ from privpy import PrivateRegion, private_function
57
+ from privpy.intrinsics import read_json, json_bytes
58
+
59
+ @private_function
60
+ def read_records(path):
61
+ return read_json(path)
62
+
63
+ @private_function
64
+ def total_paid(records):
65
+ total = 0
66
+ for row in records:
67
+ if row["status"] == "paid":
68
+ total += row["amount"]
69
+ return total
70
+
71
+ @private_function
72
+ def encode_total(total):
73
+ return json_bytes({"total": total})
74
+
75
+ with PrivateRegion() as region:
76
+ records = read_records("/absolute/orders.json")
77
+ total = total_paid(records)
78
+ print(total) # <PrivateRef>
79
+
80
+ # Deliberate disclosure: the caller chooses what gets exported.
81
+ public_total = region.export(total)
82
+ region.export(total, to=Path("/absolute/new-report.json"), transform=encode_total)
83
+ ```
84
+
85
+ There is no `policy`, `region.run`, universal `load`, `format`, or destination alias. The context manager establishes the active region. Decorated functions execute in a native interpreter; their original Python bodies are never invoked with private inputs.
86
+
87
+ ## Build and run
88
+
89
+ Requires Python 3.9+ and macOS or Linux. Building from source needs a C++17 compiler with floating-point `std::to_chars` support. Release wheels target macOS 14+ and glibc 2.28+ Linux. The pinned development environment uses Python 3.12.
90
+
91
+ ```sh
92
+ python -m venv .venv
93
+ source .venv/bin/activate
94
+ python -m pip install -r requirements-test.txt
95
+ PYTHONPATH=src python -m privpy.build
96
+ PYTHONPATH=src python examples/orders.py
97
+ PYTHONPATH=src python examples/orders.py --output /absolute/new-report.csv
98
+ ```
99
+
100
+ If a virtual environment already exists, activate and reuse it.
101
+
102
+ `python -m pip install .` builds the native library during installation. There are no Python runtime dependencies. Source-checkout use only needs `PYTHONPATH=src` and the explicit build command.
103
+
104
+ ## Implemented contract
105
+
106
+ - A region owns native values. Outside Python receives a `PrivateRef` containing an owner reference and integer handle.
107
+ - Public arguments are copied in; their original Python copies remain public.
108
+ - Native source readers retrieve regular files without routing plaintext through Python. They do not make a public source file secret.
109
+ - Private calls remain native. Outside callbacks, dynamic imports and arbitrary Python libraries are rejected.
110
+ - Every result stays private, including booleans, lengths and aggregates.
111
+ - Export is deliberate disclosure. It checks mechanics, not whether the contents are semantically safe to reveal.
112
+ - `export(value)` returns an independent supported Python value.
113
+ - `export(value, to=Path(...))` requires bytes and writes them directly from native storage.
114
+ - An optional decorated transform runs before export. There is no implicit formatting, encoding or encryption.
115
+ - Destinations must be absolute. Existing files and symlinks are never replaced. Symlink components are rejected; use an already-canonical directory path. Complete output is published atomically from a mode-0600 temporary file in the same directory.
116
+ - Closing a region invalidates all its references. Exported values and files survive.
117
+
118
+ ## Supported subset
119
+
120
+ Data: null, booleans, signed 64-bit integers, finite binary64 floats, Unicode strings, bytes, lists and string-keyed dictionaries.
121
+
122
+ Syntax: local assignment, single-level subscript updates, unpacking, arithmetic, comparisons, boolean expressions, indexing, conditional expressions, `if`, `for`, `while`, loop `else`, `break`, `continue`, return, one-generator list comprehensions and calls between private functions.
123
+
124
+ Builtins: `len`, `bool`, scalar `str`, `abs`, `sum`, `min`, `max`, `range`.
125
+
126
+ Methods: dictionary `get`, `keys`, `values`, `items`; string UTF-8 `encode`.
127
+
128
+ Source and encoding operations:
129
+
130
+ ```python
131
+ from privpy.intrinsics import (
132
+ read_json, read_text, read_bytes, json_bytes, csv_bytes, utf8_bytes,
133
+ )
134
+ ```
135
+
136
+ Prototype semantics:
137
+
138
+ - CSV uses minimal quoting with comma separators and CRLF record endings. NUL characters are preserved and do not by themselves force quoting.
139
+ - Collection updates use value semantics; updating one binding does not mutate another reference.
140
+ - Dictionaries iterate in sorted key order, and their view methods return lists.
141
+ - Integers are bounded. Overflow raises a sanitized execution error.
142
+ - Floor division and modulo currently accept integers only.
143
+ - Mixed integer/float comparisons reject integers outside binary64's exact integer range.
144
+ - Functions require inspectable source. Closures, defaults, variadic/keyword-only parameters and keyword arguments inside protected calls are unsupported. Named arguments at the public call boundary work.
145
+ - Limits bound input/value sizes, collections, native nesting, recursion and interpreter steps. `PrivateRegion(max_steps=...)` adjusts a per-call resource limit.
146
+ - Unsupported syntax fails explicitly, even in unreachable branches.
147
+
148
+ Full dataframe engines, secrets-manager SDKs, network clients, encryption, arbitrary third-party libraries, custom classes and asynchronous private functions are not implemented. The example demonstrates record processing without claiming pandas compatibility. [DESIGN.md](DESIGN.md) contains the target API and architecture sketch.
149
+
150
+ ## Security boundary
151
+
152
+ This prototype prevents accidental disclosure through supported operations. Native values are not a hidden Python object graph. Source parsing and file export do not route contents through Python callbacks.
153
+
154
+ It is **not a hostile-code sandbox or a production-grade secrets facility**. Native-memory access, debuggers, the operating system, compromised native code and a developer deliberately exporting secrets remain outside the guarantee. Timing, exception occurrence, termination, file sizes and I/O side effects can reveal information. Native release does not guarantee full zeroization.
155
+
156
+ Source/file operations run with the process's filesystem rights. The application developer is trusted. There is intentionally no independent policy engine.
157
+
158
+ ## Tests
159
+
160
+ ```sh
161
+ PYTHONPATH=src python -m unittest discover -s tests -t . -v
162
+ PYTHONPATH=src python -m coverage run -m unittest discover -s tests -t . -q
163
+ python -m coverage report
164
+ PRIVPY_HYPOTHESIS_PROFILE=ci PYTHONPATH=src python -m unittest discover -s tests -t . -q
165
+ ```
166
+
167
+ Hypothesis provides generated data, shrinking, a persistent regression database, generated programs and a state machine for lifetime/ownership sequences.
168
+
169
+ Profiles: `dev` uses up to 200 examples per property, `ci` up to 1,000 with deterministic generation, and `stress` up to 10,000. Finite input spaces may be exhausted earlier. Failures include minimized examples and reproduction instructions. `PRIVPY_TEST_STATE` selects the local database directory.
170
+
171
+ Run the same **Python** suite against an UndefinedBehaviorSanitizer build:
172
+
173
+ ```sh
174
+ PYTHONPATH=src python -m privpy.build --sanitize --output /absolute/work/runtime-sanitized.dylib
175
+ PRIVPY_NATIVE=/absolute/work/runtime-sanitized.dylib PYTHONPATH=src python -m unittest discover -s tests -t . -q
176
+ ```
177
+
178
+ On Linux use a `.so` output path. Test cases remain Python; the native library is rebuilt with instrumentation.
179
+
180
+ See [TESTING.md](TESTING.md) for extending the suite.
181
+
182
+ ## Builds and PyPI
183
+
184
+ GitHub Actions runs tests, native sanitizer checks, source builds, and wheels for CPython 3.9–3.14 on Linux and macOS, each on Intel and ARM. Published GitHub releases trigger verified PyPI uploads through Trusted Publishing.
185
+
186
+ See [RELEASING.md](RELEASING.md) for the exact PyPI publisher fields and release steps, and [VALIDATION.md](VALIDATION.md) for local verification results.
privpy-0.1.0/README.md ADDED
@@ -0,0 +1,159 @@
1
+ # privpy
2
+
3
+ **Private computations, native-owned values, explicit export.**
4
+
5
+ Experimental same-process Python module. The runtime is C++17; the entire test suite is Python, using `unittest` and Hypothesis.
6
+
7
+ ## Install from PyPI
8
+
9
+ Install the latest release into your Python environment:
10
+
11
+ ```sh
12
+ python -m pip install privpy
13
+ ```
14
+
15
+ To upgrade an existing installation:
16
+
17
+ ```sh
18
+ python -m pip install --upgrade privpy
19
+ ```
20
+
21
+ Prebuilt wheels include the native runtime for CPython 3.9–3.14 on macOS 14+ and glibc 2.28+ Linux, on Intel and ARM. These wheels do not require a local C++ compiler. If pip selects a source archive instead, the source-build requirements below apply. Windows and PyPy are not supported yet.
22
+
23
+ ## Example
24
+
25
+ Save protected functions in a Python file so their source can be inspected.
26
+
27
+ ```python
28
+ from pathlib import Path
29
+ from privpy import PrivateRegion, private_function
30
+ from privpy.intrinsics import read_json, json_bytes
31
+
32
+ @private_function
33
+ def read_records(path):
34
+ return read_json(path)
35
+
36
+ @private_function
37
+ def total_paid(records):
38
+ total = 0
39
+ for row in records:
40
+ if row["status"] == "paid":
41
+ total += row["amount"]
42
+ return total
43
+
44
+ @private_function
45
+ def encode_total(total):
46
+ return json_bytes({"total": total})
47
+
48
+ with PrivateRegion() as region:
49
+ records = read_records("/absolute/orders.json")
50
+ total = total_paid(records)
51
+ print(total) # <PrivateRef>
52
+
53
+ # Deliberate disclosure: the caller chooses what gets exported.
54
+ public_total = region.export(total)
55
+ region.export(total, to=Path("/absolute/new-report.json"), transform=encode_total)
56
+ ```
57
+
58
+ There is no `policy`, `region.run`, universal `load`, `format`, or destination alias. The context manager establishes the active region. Decorated functions execute in a native interpreter; their original Python bodies are never invoked with private inputs.
59
+
60
+ ## Build and run
61
+
62
+ Requires Python 3.9+ and macOS or Linux. Building from source needs a C++17 compiler with floating-point `std::to_chars` support. Release wheels target macOS 14+ and glibc 2.28+ Linux. The pinned development environment uses Python 3.12.
63
+
64
+ ```sh
65
+ python -m venv .venv
66
+ source .venv/bin/activate
67
+ python -m pip install -r requirements-test.txt
68
+ PYTHONPATH=src python -m privpy.build
69
+ PYTHONPATH=src python examples/orders.py
70
+ PYTHONPATH=src python examples/orders.py --output /absolute/new-report.csv
71
+ ```
72
+
73
+ If a virtual environment already exists, activate and reuse it.
74
+
75
+ `python -m pip install .` builds the native library during installation. There are no Python runtime dependencies. Source-checkout use only needs `PYTHONPATH=src` and the explicit build command.
76
+
77
+ ## Implemented contract
78
+
79
+ - A region owns native values. Outside Python receives a `PrivateRef` containing an owner reference and integer handle.
80
+ - Public arguments are copied in; their original Python copies remain public.
81
+ - Native source readers retrieve regular files without routing plaintext through Python. They do not make a public source file secret.
82
+ - Private calls remain native. Outside callbacks, dynamic imports and arbitrary Python libraries are rejected.
83
+ - Every result stays private, including booleans, lengths and aggregates.
84
+ - Export is deliberate disclosure. It checks mechanics, not whether the contents are semantically safe to reveal.
85
+ - `export(value)` returns an independent supported Python value.
86
+ - `export(value, to=Path(...))` requires bytes and writes them directly from native storage.
87
+ - An optional decorated transform runs before export. There is no implicit formatting, encoding or encryption.
88
+ - Destinations must be absolute. Existing files and symlinks are never replaced. Symlink components are rejected; use an already-canonical directory path. Complete output is published atomically from a mode-0600 temporary file in the same directory.
89
+ - Closing a region invalidates all its references. Exported values and files survive.
90
+
91
+ ## Supported subset
92
+
93
+ Data: null, booleans, signed 64-bit integers, finite binary64 floats, Unicode strings, bytes, lists and string-keyed dictionaries.
94
+
95
+ Syntax: local assignment, single-level subscript updates, unpacking, arithmetic, comparisons, boolean expressions, indexing, conditional expressions, `if`, `for`, `while`, loop `else`, `break`, `continue`, return, one-generator list comprehensions and calls between private functions.
96
+
97
+ Builtins: `len`, `bool`, scalar `str`, `abs`, `sum`, `min`, `max`, `range`.
98
+
99
+ Methods: dictionary `get`, `keys`, `values`, `items`; string UTF-8 `encode`.
100
+
101
+ Source and encoding operations:
102
+
103
+ ```python
104
+ from privpy.intrinsics import (
105
+ read_json, read_text, read_bytes, json_bytes, csv_bytes, utf8_bytes,
106
+ )
107
+ ```
108
+
109
+ Prototype semantics:
110
+
111
+ - CSV uses minimal quoting with comma separators and CRLF record endings. NUL characters are preserved and do not by themselves force quoting.
112
+ - Collection updates use value semantics; updating one binding does not mutate another reference.
113
+ - Dictionaries iterate in sorted key order, and their view methods return lists.
114
+ - Integers are bounded. Overflow raises a sanitized execution error.
115
+ - Floor division and modulo currently accept integers only.
116
+ - Mixed integer/float comparisons reject integers outside binary64's exact integer range.
117
+ - Functions require inspectable source. Closures, defaults, variadic/keyword-only parameters and keyword arguments inside protected calls are unsupported. Named arguments at the public call boundary work.
118
+ - Limits bound input/value sizes, collections, native nesting, recursion and interpreter steps. `PrivateRegion(max_steps=...)` adjusts a per-call resource limit.
119
+ - Unsupported syntax fails explicitly, even in unreachable branches.
120
+
121
+ Full dataframe engines, secrets-manager SDKs, network clients, encryption, arbitrary third-party libraries, custom classes and asynchronous private functions are not implemented. The example demonstrates record processing without claiming pandas compatibility. [DESIGN.md](DESIGN.md) contains the target API and architecture sketch.
122
+
123
+ ## Security boundary
124
+
125
+ This prototype prevents accidental disclosure through supported operations. Native values are not a hidden Python object graph. Source parsing and file export do not route contents through Python callbacks.
126
+
127
+ It is **not a hostile-code sandbox or a production-grade secrets facility**. Native-memory access, debuggers, the operating system, compromised native code and a developer deliberately exporting secrets remain outside the guarantee. Timing, exception occurrence, termination, file sizes and I/O side effects can reveal information. Native release does not guarantee full zeroization.
128
+
129
+ Source/file operations run with the process's filesystem rights. The application developer is trusted. There is intentionally no independent policy engine.
130
+
131
+ ## Tests
132
+
133
+ ```sh
134
+ PYTHONPATH=src python -m unittest discover -s tests -t . -v
135
+ PYTHONPATH=src python -m coverage run -m unittest discover -s tests -t . -q
136
+ python -m coverage report
137
+ PRIVPY_HYPOTHESIS_PROFILE=ci PYTHONPATH=src python -m unittest discover -s tests -t . -q
138
+ ```
139
+
140
+ Hypothesis provides generated data, shrinking, a persistent regression database, generated programs and a state machine for lifetime/ownership sequences.
141
+
142
+ Profiles: `dev` uses up to 200 examples per property, `ci` up to 1,000 with deterministic generation, and `stress` up to 10,000. Finite input spaces may be exhausted earlier. Failures include minimized examples and reproduction instructions. `PRIVPY_TEST_STATE` selects the local database directory.
143
+
144
+ Run the same **Python** suite against an UndefinedBehaviorSanitizer build:
145
+
146
+ ```sh
147
+ PYTHONPATH=src python -m privpy.build --sanitize --output /absolute/work/runtime-sanitized.dylib
148
+ PRIVPY_NATIVE=/absolute/work/runtime-sanitized.dylib PYTHONPATH=src python -m unittest discover -s tests -t . -q
149
+ ```
150
+
151
+ On Linux use a `.so` output path. Test cases remain Python; the native library is rebuilt with instrumentation.
152
+
153
+ See [TESTING.md](TESTING.md) for extending the suite.
154
+
155
+ ## Builds and PyPI
156
+
157
+ GitHub Actions runs tests, native sanitizer checks, source builds, and wheels for CPython 3.9–3.14 on Linux and macOS, each on Intel and ARM. Published GitHub releases trigger verified PyPI uploads through Trusted Publishing.
158
+
159
+ See [RELEASING.md](RELEASING.md) for the exact PyPI publisher fields and release steps, and [VALIDATION.md](VALIDATION.md) for local verification results.
@@ -0,0 +1,58 @@
1
+ # Building and publishing privpy
2
+
3
+ Repository: https://github.com/technoyoda/privpy.
4
+ Distribution and import name: `privpy`.
5
+
6
+ ## PyPI setup
7
+
8
+ Configure a GitHub **Trusted Publisher** with these exact fields:
9
+
10
+ | PyPI field | Value |
11
+ | --- | --- |
12
+ | PyPI project name | `privpy` |
13
+ | Owner | `technoyoda` |
14
+ | Repository | `privpy` |
15
+ | Workflow filename | `release.yml` |
16
+ | Environment | `pypi` |
17
+
18
+ For a new project, use PyPI's [pending publisher form](https://pypi.org/manage/account/publishing/). For a project you already own, add the publisher in its Publishing settings. PyPI explains both [new-project setup](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) and [existing-project setup](https://docs.pypi.org/trusted-publishers/adding-a-publisher/).
19
+
20
+ Create the `pypi` environment in GitHub under repository **Settings → Environments**, and configure its release-tag restrictions as appropriate. If you add required reviewers, the publish job will wait for their approval. No PyPI API token or GitHub secret is needed: the publisher uses GitHub OIDC. [PyPI publishing documentation](https://docs.pypi.org/trusted-publishers/using-a-publisher/)
21
+
22
+ ## What the workflows do
23
+
24
+ - **Tests** (`tests.yml`): Linux and macOS on Python 3.12 and 3.14, the full property-based suite with the 1,000-example CI profile, Python coverage, and a second run against UndefinedBehaviorSanitizer.
25
+ - **Build distributions** (`build.yml`): a tested source archive and 24 installed-and-tested wheels for CPython 3.9–3.14, on Linux x86_64/AArch64 and macOS Intel/Apple Silicon. Wheels are built from that source archive. The final bundle must pass metadata, version, native-library, and complete-matrix checks.
26
+ - **Publish to PyPI** (`release.yml`): on a **published GitHub release**, require the tag to match the source version, rerun Tests and Build distributions, then publish their validated artifacts through the `pypi` environment.
27
+
28
+ Pushes to `main` and pull requests run Tests and Build distributions. Both also support manual runs. Neither of those events publishes to PyPI. The publishing job has OIDC permission; build and test jobs have read-only repository permissions. Publishing runs in a separate job without checking out or executing project source.
29
+
30
+ Published GitHub prereleases also trigger publishing; use a PEP 440 prerelease version such as `0.2.0rc1` and matching tag `v0.2.0rc1`. A GitHub prerelease checkbox alone does not turn `0.2.0` into a PyPI prerelease.
31
+
32
+ Wheels target glibc 2.28+ Linux and macOS 14+. Windows, musl Linux, PyPy and free-threaded Python are not part of this matrix. Older Python wheel tests resolve a compatible Hypothesis version; the main CI suite uses `requirements-test.txt` pins.
33
+
34
+ The publisher emits PyPI attestations. [PyPI attestation documentation](https://docs.pypi.org/attestations/producing-attestations/)
35
+
36
+ ## Make a release
37
+
38
+ 1. Change `src/privpy/_version.py`. It is the single version source.
39
+ 2. Commit and push the changes; check the Tests and Build distributions runs.
40
+ 3. Create a tag exactly matching that version, for example `v0.1.0`, on the desired commit.
41
+ 4. Publish a GitHub release for that tag.
42
+ 5. Watch Publish to PyPI. Downloadable build artifacts are retained in its Actions run.
43
+
44
+ Only the publish step uploads to PyPI. PyPI versions cannot be overwritten. A partial upload fails visibly; investigate which files arrived before deciding how to recover. The workflow does not silently skip existing distributions.
45
+
46
+ ## Local packaging check
47
+
48
+ Run inside the project with a virtual environment:
49
+
50
+ ```sh
51
+ python -m pip install build twine
52
+ python -m build
53
+ python -m twine check --strict dist/*
54
+ python scripts/check_dist.py dist
55
+ RELEASE_TAG=v0.1.0 python scripts/check_release.py
56
+ ```
57
+
58
+ `python -m build` builds the wheel from the newly produced source archive. Building from source needs a C++17 compiler with floating-point `std::to_chars` support. macOS release builds set `MACOSX_DEPLOYMENT_TARGET=14.0`. To reproduce the full matrix, use the Build distributions workflow; local cibuildwheel requires additional platform tooling. [cibuildwheel documentation](https://cibuildwheel.pypa.io/en/stable/setup/)
@@ -0,0 +1,60 @@
1
+ # Test contract and growth plan
2
+
3
+ All test code is Python. The suite uses `unittest` and real Hypothesis property-based tests, rather than replacing properties with fixed random loops.
4
+
5
+ ## Categories
6
+
7
+ | Suite | Contract |
8
+ | --- | --- |
9
+ | `test_api.py` | Lifetime, ownership, task/thread contexts, opacity, export, destination behavior |
10
+ | `test_compiler.py` | Allowed syntax, rejected escape operations, bindings, control flow, no callbacks |
11
+ | `test_properties.py` | Nested-value round trips, Python differential oracles, generated expressions, transforms, serialization and I/O |
12
+ | `test_stateful.py` | Generated create, compute, copy, export, nest, close and reopen sequences |
13
+ | `test_native.py` | Malformed parser/ABI inputs, Unicode, numeric boundaries, resource limits, concurrent publication |
14
+ | `test_regressions.py` | Minimized float, CSV and scoping failures retained independently of the example database |
15
+ | `test_packaging.py` | Release-tag properties, malformed/incomplete distribution rejection, metadata and native-library requirements |
16
+
17
+ ## Properties
18
+
19
+ - Export of `identity(x)` preserves supported values.
20
+ - Safe-range arithmetic and comparisons agree with Python; overflow is rejected.
21
+ - Generated expression trees agree with an independent Python oracle.
22
+ - Unicode indexing, length and encoding agree with Python.
23
+ - CSV encoding agrees with `csv.writer` for documented input types.
24
+ - The CSV oracle uses an escape character absent from the generated input to avoid older CPython's incidental NUL quoting. NUL values remain covered, including deterministic regressions.
25
+ - JSON reading and encoding agree with Python for the strict supported subset.
26
+ - Inline transforms equal separate protected transformation followed by export.
27
+ - File export writes exact bytes; paths never infer formats.
28
+ - Existing files remain unchanged under repeated/concurrent exports and symlink attempts.
29
+ - Exported mutable objects are independent copies.
30
+ - Reference representations are independent of contents.
31
+ - Lifetime and ownership hold under generated operation sequences.
32
+ - Unsupported callbacks fail before executing host code on private values.
33
+ - Malformed native requests produce handled failures rather than crashes.
34
+
35
+ ## Profiles and reproducibility
36
+
37
+ `dev`: up to 200 examples per property, persistent shrinking database.
38
+
39
+ `ci`: up to 1,000 examples per property, deterministic generation for a fixed Hypothesis version.
40
+
41
+ `stress`: up to 10,000 examples per property, persistent shrinking database.
42
+
43
+ Pinned versions are in `requirements-test.txt`. Hypothesis reports minimized failures and reproduction blobs. Preserve important discovered cases as deterministic regressions too; the local database is not the only record of a bug.
44
+
45
+ ## Adding a feature
46
+
47
+ 1. Specify its semantics and unsupported cases.
48
+ 2. Add a Python oracle or invariant independent of the native implementation.
49
+ 3. Extend strategies with valid and invalid inputs.
50
+ 4. Add state-machine transitions when lifetime or ownership changes.
51
+ 5. Preserve minimized bug regressions.
52
+ 6. Run normal and sanitizer suites.
53
+
54
+ Coverage helps find missing behavior; it is not proof of security. Python coverage does not measure C++. Native coverage needs separate instrumentation and reporting. Sanitizers do not prove absence of bugs or side channels.
55
+
56
+ The Python coverage report enforces a 90% floor. CI configuration exercises macOS and Linux; only the local platform results recorded in `VALIDATION.md` have actually been run during initial development.
57
+
58
+ ## Exclusions
59
+
60
+ Tests do not claim hardware isolation, resistance to hostile native memory access, constant-time execution, full zeroization, unrestricted Python/pandas compatibility, or semantic safety of deliberately exported values.
@@ -0,0 +1,43 @@
1
+ # Initial validation
2
+
3
+ Validated on macOS ARM64 with Python 3.12.14 and Apple Clang 21.0.0. All test code is Python. The package version is 0.1.0.
4
+
5
+ ## Results
6
+
7
+ | Check | Result |
8
+ | --- | --- |
9
+ | Full Python suite | 211 tests passed |
10
+ | Property coverage | 51 Hypothesis properties plus a rule-based state machine |
11
+ | Python runtime coverage | 96% combined statement/branch coverage; 90% enforced floor |
12
+ | CI profile + AddressSanitizer + UndefinedBehaviorSanitizer | 211 tests passed; no sanitizer diagnostics |
13
+ | Native line coverage | 93.80% |
14
+ | Native branch coverage | 79.33% |
15
+ | Native function coverage | 100% of 85 functions |
16
+ | GitHub Actions static validation | actionlint 1.7.12 passed |
17
+ | cibuildwheel configuration | 24 intended Python/platform build identifiers verified |
18
+ | Local wheel and source archive | Built successfully; strict Twine metadata checks passed |
19
+ | Release guards | Tag matching, missing native runtime, wrong versions and incomplete bundles tested |
20
+
21
+ The native coverage values come from the final CI-profile sanitizer run. AddressSanitizer leak detection was disabled; these results do not establish leak freedom. Python coverage excludes the compiler-invocation build helper and does not measure C++.
22
+
23
+ The default Hypothesis profile exercises up to 200 examples per property; the CI run uses up to 1,000. Finite strategy domains can be exhausted earlier. The state machine generates region creation, computation, export, nesting, closure and invalid-reference sequences.
24
+
25
+ ## Failures caught during development
26
+
27
+ - Valid subnormal floating-point input was rejected by the original number parser.
28
+ - Binary wire encoding incorrectly applied a plaintext limit to its expanded representation.
29
+ - Float rendering differed from Python at notation boundaries.
30
+ - CSV needed quotes around a single empty field.
31
+ - Comprehension variable scope incorrectly affected later name resolution.
32
+ - Repeated structural nesting and shared-value expansion needed explicit limits.
33
+ - A generated unknown-method test allowed Python keywords; the strategy now generates legal identifiers.
34
+
35
+ Runtime regressions are preserved as deterministic cases alongside generated tests. The release suite also rejects incomplete or incorrectly identified package bundles.
36
+
37
+ ## Scope
38
+
39
+ Local execution verifies macOS ARM64 and Python 3.12. The Linux, Intel macOS, and other Python jobs are configured in GitHub Actions but have not run remotely during this initial local implementation. Publishing requires the repository changes to be pushed and the PyPI Trusted Publisher to be configured. No PyPI upload was performed.
40
+
41
+ The commit candidates were checked for local home/workspace paths, personal Git identity, email addresses, credential patterns, generated artifacts and logs. Local build outputs, virtual environments, profiling files and test databases are excluded. Repository author configuration uses a neutral identity. The explicitly chosen public GitHub repository is retained in package metadata.
42
+
43
+ These checks do not establish a production security boundary. The limits described in README.md and DESIGN.md still apply, including hostile native code, side channels and lack of guaranteed zeroization.
@@ -0,0 +1,6 @@
1
+ [
2
+ {"country": "US", "amount": 1200, "status": "paid"},
3
+ {"country": "GB", "amount": 700, "status": "paid"},
4
+ {"country": "US", "amount": 300, "status": "paid"},
5
+ {"country": "GB", "amount": 900, "status": "pending"}
6
+ ]