syft-restrict 0.1.0__tar.gz

syft_restrict-0.1.0/.gitignore

@@ -0,0 +1,200 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+credentials/
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# python stuff
+*__pycache__*
+*.pyc
+*.egg-info/
+*.swp
+*.swo
+*.ipynb_checkpoints*
+**.pytest_cache/
+**/pip-wheel-metadata
+
+# pipenv
+Pipfile.lock
+
+# poetry
+poetry.lock
+
+# pdm
+.pdm.toml
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+.idea/
+
+# VS Code
+.vscode/
+
+# macOS
+.DS_Store
+
+# Google Drive credentials
+credentials.json
+client_secret*.json
+token.pickle
+token.json
+*creds.json
+
+# SyftBox related
+~/SyftBox/
+SyftBox/
+
+# Temporary files
+*.tmp
+*.bak
+*.swp
+*~
+
+# Claude AI settings
+.claude/*
+!.claude/settings.json
+CLAUDE.md
+
+# CI test outputs
+test_outputs/
+test_suite_output.log
+
+
+# Notebooks
+notebooks/e2e/sales_mock.csv
+notebooks/e2e/sales_private.csv
+notebooks/e2e/readme.mdpackages/syft-bg/docs/sync-service.md
+packages/syft-bg/docs/sync-review-notes.md
+notebooks/ai_audit/internal/data/pdfs
+notebooks/ai_audit/internal/data/manifest.csv
+notebooks/ai_audit/internal/data/readme.md
+
+.agents/
+AGENTS.md
+
+# Local JupyterLab state (jupyter MCP server runs from repo root)
+.jupyter/
+.jupyter_ystore.db

syft_restrict-0.1.0/PKG-INFO

@@ -0,0 +1,109 @@
+Metadata-Version: 2.4
+Name: syft-restrict
+Version: 0.1.0
+Summary: Verify + obfuscate: prove JAX/Flax inference code only does math (no data theft) while hiding the model architecture
+Author-email: OpenMined <info@openmined.org>
+License: Apache-2.0
+Requires-Python: >=3.10
+Requires-Dist: pydantic>=2
+Description-Content-Type: text/markdown
+
+# syft-restrict
+
+A **static analyzer** for Python source files that **default-denies** dynamic
+constructs in a **private** region, typically the model logic that must be kept private, while allowing the **public** region to be reviewed by the data owner in an **obfuscated** copy.
+
+## Overview
+
+`syft-restrict` splits a Python file into two regions:
+
+- **Private** — the lines the user marks as private, typically the model logic that
+  must not be readable. Mark them with `# syft-restrict: obfuscate-start` /
+  `# syft-restrict: obfuscate-end` or `# syft-restrict: hide-start` / `# syft-restrict: hide-end` comments in the source.
+- **Public** — every other line outside the marked ranges: imports, data loading, wrappers. It is
+  never checked or obfuscated. The recipient of the obfuscated file can read it directly.
+
+Given that split, `syft-restrict` executes the following two steps:
+
+- **The verifier** statically walks the private region and analyzes it against a default-deny
+  policy. Every construct must be explicitly allowed, or verification fails
+  and reports each offending line.
+- **The obfuscator** runs after a clean verify. It creates an obfuscated copy by rewriting the private
+  region — renaming identifiers, blanking constants, replacing lines with `■■■■■■■■` — so the logic can't be read
+  back out, while the public region is copied through untouched.
+
+The **verifier** is inspired by
+[**RestrictedPython**](https://github.com/zopefoundation/RestrictedPython), but
+differs in a few ways:
+
+- **It analyzes — it doesn't run.** It never executes the source. Only the static checks decide pass/fail.
+- The public code may **import allowlisted libraries** and call into them.
+  Imports are not allowed in private code, and any library call must be
+  **explicitly allow-listed**.
+- **Dynamic Python lives in the public part.** Anything dynamic the author needs
+  (file/tokenizer loading, the generation loop, wrappers around library methods)
+  must be written in the **public** region — where it can be reviewed directly — and
+  may be **called by** the private part.
+
+## Usage
+
+Mark the private region in the source with comments:
+
+```python
+# syft-restrict: obfuscate-start
+def attention(x):
+    ...
+# syft-restrict: obfuscate-end
+
+MODEL_ID = "gemma-2b"  # syft-restrict: hide
+```
+
+_See [docs/verify.md](docs/verify.md#public-vs-private) for the full marker syntax (single-line markers, nesting rules, and error cases)._
+
+Then run:
+
+```python
+import syft_restrict as restrict
+
+result = restrict.run(
+    "gemma_inference.py",
+    allow_functions=["jax.*", "flax.linen.*"],  # functions callable BY NAME (path-resolved)
+    allow_operators=["arithmetic", "indexing", "comparison"],  # operators allowed ON A VALUE
+)
+# On success: writes gemma_inference.obfuscated.py and returns result.certificate.
+# On a policy violation: raises PolicyViolation naming each offending line.
+# If the file has no markers and obfuscate=/hide= are both omitted: raises MarkerError.
+```
+
+`obfuscate=`/`hide=` still work as an explicit escape hatch — pass either one (even an
+empty list) and marker scanning is skipped entirely in favor of your own 1-based line
+ranges:
+
+```python
+result = restrict.run(
+    "gemma_inference.py",
+    obfuscate=[[22, 93], [99, 280]],  # 1-based ranges: identifiers renamed, constants blanked
+    hide=[],                          # 1-based ranges: whole line replaced with ■■■■■■■■
+    allow_functions=["jax.*", "flax.linen.*"],
+    allow_operators=["arithmetic", "indexing", "comparison"],
+)
+```
+
+This is how **[examples/gemma_inference.py](examples/gemma_inference.py)** generates
+**[examples/gemma_inference.obfuscated.py](examples/gemma_inference.obfuscated.py)** —
+see [examples/generate.py](examples/generate.py).
+
+The same model marked up with comment blocks instead of numeric ranges lives in
+**[examples/gemma_inference_marked.py](examples/gemma_inference_marked.py)** /
+[examples/generate_marked.py](examples/generate_marked.py).
+
+If syft-restrict was successfully executed on the true original file, the obfuscated file can be safely shared without exposing the private section.
+
+Pass `strict=False` to `run` to get a `RunResult` with `.ok` / `.violations`
+instead of an exception.
+
+## Documentation
+
+- [docs/verify.md](docs/verify.md) — how verification works and what private code may do (allow side).
+- [docs/blacklist.md](docs/blacklist.md) — default-deny catalog and violation codes (deny side).
+- [docs/code-layout.md](docs/code-layout.md) — source modules and test layout.

syft_restrict-0.1.0/README.md

@@ -0,0 +1,99 @@
+# syft-restrict
+
+A **static analyzer** for Python source files that **default-denies** dynamic
+constructs in a **private** region, typically the model logic that must be kept private, while allowing the **public** region to be reviewed by the data owner in an **obfuscated** copy.
+
+## Overview
+
+`syft-restrict` splits a Python file into two regions:
+
+- **Private** — the lines the user marks as private, typically the model logic that
+  must not be readable. Mark them with `# syft-restrict: obfuscate-start` /
+  `# syft-restrict: obfuscate-end` or `# syft-restrict: hide-start` / `# syft-restrict: hide-end` comments in the source.
+- **Public** — every other line outside the marked ranges: imports, data loading, wrappers. It is
+  never checked or obfuscated. The recipient of the obfuscated file can read it directly.
+
+Given that split, `syft-restrict` executes the following two steps:
+
+- **The verifier** statically walks the private region and analyzes it against a default-deny
+  policy. Every construct must be explicitly allowed, or verification fails
+  and reports each offending line.
+- **The obfuscator** runs after a clean verify. It creates an obfuscated copy by rewriting the private
+  region — renaming identifiers, blanking constants, replacing lines with `■■■■■■■■` — so the logic can't be read
+  back out, while the public region is copied through untouched.
+
+The **verifier** is inspired by
+[**RestrictedPython**](https://github.com/zopefoundation/RestrictedPython), but
+differs in a few ways:
+
+- **It analyzes — it doesn't run.** It never executes the source. Only the static checks decide pass/fail.
+- The public code may **import allowlisted libraries** and call into them.
+  Imports are not allowed in private code, and any library call must be
+  **explicitly allow-listed**.
+- **Dynamic Python lives in the public part.** Anything dynamic the author needs
+  (file/tokenizer loading, the generation loop, wrappers around library methods)
+  must be written in the **public** region — where it can be reviewed directly — and
+  may be **called by** the private part.
+
+## Usage
+
+Mark the private region in the source with comments:
+
+```python
+# syft-restrict: obfuscate-start
+def attention(x):
+    ...
+# syft-restrict: obfuscate-end
+
+MODEL_ID = "gemma-2b"  # syft-restrict: hide
+```
+
+_See [docs/verify.md](docs/verify.md#public-vs-private) for the full marker syntax (single-line markers, nesting rules, and error cases)._
+
+Then run:
+
+```python
+import syft_restrict as restrict
+
+result = restrict.run(
+    "gemma_inference.py",
+    allow_functions=["jax.*", "flax.linen.*"],  # functions callable BY NAME (path-resolved)
+    allow_operators=["arithmetic", "indexing", "comparison"],  # operators allowed ON A VALUE
+)
+# On success: writes gemma_inference.obfuscated.py and returns result.certificate.
+# On a policy violation: raises PolicyViolation naming each offending line.
+# If the file has no markers and obfuscate=/hide= are both omitted: raises MarkerError.
+```
+
+`obfuscate=`/`hide=` still work as an explicit escape hatch — pass either one (even an
+empty list) and marker scanning is skipped entirely in favor of your own 1-based line
+ranges:
+
+```python
+result = restrict.run(
+    "gemma_inference.py",
+    obfuscate=[[22, 93], [99, 280]],  # 1-based ranges: identifiers renamed, constants blanked
+    hide=[],                          # 1-based ranges: whole line replaced with ■■■■■■■■
+    allow_functions=["jax.*", "flax.linen.*"],
+    allow_operators=["arithmetic", "indexing", "comparison"],
+)
+```
+
+This is how **[examples/gemma_inference.py](examples/gemma_inference.py)** generates
+**[examples/gemma_inference.obfuscated.py](examples/gemma_inference.obfuscated.py)** —
+see [examples/generate.py](examples/generate.py).
+
+The same model marked up with comment blocks instead of numeric ranges lives in
+**[examples/gemma_inference_marked.py](examples/gemma_inference_marked.py)** /
+[examples/generate_marked.py](examples/generate_marked.py).
+
+If syft-restrict was successfully executed on the true original file, the obfuscated file can be safely shared without exposing the private section.
+
+Pass `strict=False` to `run` to get a `RunResult` with `.ok` / `.violations`
+instead of an exception.
+
+## Documentation
+
+- [docs/verify.md](docs/verify.md) — how verification works and what private code may do (allow side).
+- [docs/blacklist.md](docs/blacklist.md) — default-deny catalog and violation codes (deny side).
+- [docs/code-layout.md](docs/code-layout.md) — source modules and test layout.

syft_restrict-0.1.0/docs/blacklist.md

@@ -0,0 +1,257 @@
+# What is not allowed
+
+This document lists everything the checker rejects in the **private** region.
+
+- The model is **default-deny**. Tables below name common rejections with clear
+  codes.
+- Anything not listed as allowed in [verify.md](verify.md) is also
+  rejected.
+- Each entry includes the **violation code** returned by `verify()`.
+
+---
+
+## Index of violation codes
+
+| Code                | Meaning                                                                  |
+| ------------------- | ------------------------------------------------------------------------ |
+| `node-type`         | Syntax not on the allow list (walrus, `match`, …)                        |
+| `banned-name`       | Reference or call to a banned builtin (`open`, `eval`, …)                |
+| `banned-construct`  | Forbidden statement/expression form (import, `with`, f-string, …)        |
+| `call-not-allowed`  | Library path not allow-listed (or hit by `disallow_functions`)           |
+| `call-unresolved`   | Call target not provably safe                                            |
+| `method-on-value`   | Named method on an unknown value (`x.reshape`)                           |
+| `attr-on-value`     | Attribute read/write on an unknown value, or bad `self` chain            |
+| `attr-not-allowed`  | Library attribute path not allow-listed                                  |
+| `dunder-attr`       | Dunder attribute (`.__class__`, …)                                       |
+| `dunder-name`       | Bare dunder name (`__class__`)                                           |
+| `dunder-def`        | Defining a forbidden magic method                                        |
+| `class-keyword`     | e.g. `metaclass=`                                                        |
+| `class-base`        | Base class not allow-listed                                              |
+| `reserved-name`     | Rebinding a trusted name (`jnp`, public wrapper, private def, `self`, …) |
+| `operator-disabled` | Operator used without enabling its group in `allow_operators`            |
+
+---
+
+## Forbidden constructs
+
+These constructs are banned outright in private code.
+
+Code: **`banned-construct`**.
+
+| Construct             | Example                             | Why                                                    |
+| --------------------- | ----------------------------------- | ------------------------------------------------------ |
+| Import                | `import os`, `from os import path`  | Imports belong in public code                          |
+| `with`                | `with open(f) as g: ...`            | Runs enter/exit hooks; often I/O                       |
+| `try` / `raise`       | `try: ... finally: ...`             | Exception tricks / host surfaces                       |
+| `global` / `nonlocal` | `global x`                          | Escape local naming rules                              |
+| `del`                 | `del x`                             | Can remove names the policy relies on                  |
+| `assert`              | `assert cond`                       | Disappears under `python -O`                           |
+| Async                 | `async def`, `await`, …             | Out of scope for pure inference                        |
+| Generators            | `yield`, `yield from`               | Suspended execution                                    |
+| F-strings             | `f"hi"`, `f"{x}"`                   | Interpolation runs formatting with no normal call site |
+| Decorators            | `@property`, `@nn.compact`, `@evil` | Run code when the def/class is reached                 |
+
+```python
+# rejected
+import os
+y = f"value={x}"
+with ctx() as g:
+    pass
+
+@nn.compact
+def __call__(self, x): ...
+```
+
+### Unknown / future syntax
+
+Any syntax not on the allow list is rejected.
+
+Code: **`node-type`**.
+
+| Example                | Notes            |
+| ---------------------- | ---------------- |
+| `y = (z := 1)`         | Walrus           |
+| `match x: case 1: ...` | Pattern matching |
+
+New Python syntax stays blocked until reviewed and explicitly allowed.
+
+---
+
+## Banned builtins
+
+These names may never be **used** (i.e, loaded) in private code, even if they are
+**not called**.
+
+Code: **`banned-name`**.
+
+### Dynamic code / reflection / I/O
+
+- `eval`
+- `exec`
+- `compile`
+- `__import__`
+- `getattr`
+- `setattr`
+- `delattr`
+- `hasattr`
+- `vars`
+- `globals`
+- `locals`
+- `dir`
+- `open`
+- `input`
+- `breakpoint`
+- `memoryview`
+- `type`
+- `__build_class__`
+- `print`
+
+### Formatting / buffer builtins
+
+Same idea as calling dunders on a value (`x.__repr__()`), spelled as a bare call:
+
+- `repr`
+- `str`
+- `ascii`
+- `format`
+- `bytes`
+
+> [!WARNING] > `bytes(x)` can dump raw buffer contents. `print` is a stdout channel. F-strings are banned
+> separately as constructs (above), including with no `{...}` at all.
+
+```python
+# all rejected (banned-name)
+f = open
+d = {"o": open}
+def run(op=open): ...
+open("/etc/passwd")          # call reports banned-name once
+y = [v for v in open(path)]  # passive positions still checked
+```
+
+---
+
+## Calls and attributes
+
+| What                                         | Example                                    | Code               |
+| -------------------------------------------- | ------------------------------------------ | ------------------ |
+| Library path not allowed                     | `np.dot(a, b)`                             | `call-not-allowed` |
+| Disallow list hits an otherwise-allowed path | `jnp.save(...)` under `disallow_functions` | `call-not-allowed` |
+| Call target not proven safe                  | `fn(x)` where `fn` is a parameter          | `call-unresolved`  |
+| Named method on a value                      | `x.reshape(8, -1)`, `items.append(1)`      | `method-on-value`  |
+| Attribute on a value                         | `x.shape`, `x.T`, `obj.send = data`        | `attr-on-value`    |
+| Deep `self` chain                            | `self.a.b`, `self.sub.evil(...)`           | `attr-on-value`    |
+| Unsafe `self.<name>(...)`                    | stashed `open` on `self` in `setup`        | `attr-on-value`    |
+| Dunder attribute                             | `obj.__class__`, `self.__dict__`           | `dunder-attr`      |
+| Bare dunder name                             | `c = __class__`                            | `dunder-name`      |
+| Library attr not allowed                     | `np.pi` when numpy is not allowed          | `attr-not-allowed` |
+
+```python
+# rejected
+a = x.reshape(8, -1)     # method-on-value
+b = x.shape              # attr-on-value
+c = obj.__class__        # dunder-attr
+
+def apply(fn, x):
+    return fn(x)         # call-unresolved
+
+```
+
+> [!IMPORTANT] > **`call-unresolved` is default-deny for callees.** It catches dangerous callables that never
+> mention a banned builtin by name (parameters, opaque locals, `d["k"](x)`).
+
+Only single-level `self.<name>` / `cls.<name>` is special-cased. Rules for when `self.x(...)` is
+safe are in [verify.md](verify.md#self-and-flax-style-modules).
+
+---
+
+## Classes and definitions
+
+| What                   | Example                               | Code            |
+| ---------------------- | ------------------------------------- | --------------- |
+| Class keywords         | `class M(nn.Module, metaclass=Meta)`  | `class-keyword` |
+| Bad base class         | `class M(SomeLib)`, `class M(object)` | `class-base`    |
+| Forbidden magic method | `def __getattr__`, `def __reduce__`   | `dunder-def`    |
+
+Allowed hooks only: `setup`, `__call__`.
+
+Decorators are banned outright as a [forbidden construct](#forbidden-constructs).
+
+```python
+# rejected
+class M(object):    # class-base
+    def __getattr__(self, name):    # dunder-def
+        return None
+```
+
+---
+
+## Reserved names
+
+Rebinding a name the checker trusts reports **`reserved-name`**.
+
+| Reserved             | Example rebind                                | Why                                    |
+| -------------------- | --------------------------------------------- | -------------------------------------- |
+| Import aliases       | `jnp = make_evil()`                           | Would lie about resolved library paths |
+| Public wrappers      | nested `def transpose` shadowing a public one | Defeats the reviewed wrapper           |
+| Private defs/classes | `helper = evil` after `def helper`            | Bare calls trust the def by name       |
+| Safe builtins        | `list = None`                                 | Bare calls trust `list(...)` by name   |
+| `self` / `cls`       | `self = x`, nested `def f(self):`             | `self.*` is trusted by spelling        |
+
+Applies to assignment, `for` / comprehension targets, parameters, and nested defs.
+Private def names are also protected against rebind in **public** glue between private ranges.
+
+```python
+# rejected
+import jax.numpy as jnp
+jnp = 1     # reserved-name
+
+def helper(x):
+    return x
+def f(evil):
+    helper = evil      # reserved-name
+    return helper(1)
+```
+
+---
+
+## Operator bundles
+
+If a group of operators is not in `allow_operators`, using any of its operators
+reports **`operator-disabled`**.
+
+| Bundle       | If missing, these fail |
+| ------------ | ---------------------- |
+| `arithmetic` | `a + b`, `-a`, …       |
+| `comparison` | `a < b`, `a and b`, …  |
+| `indexing`   | `x[0]`, `x[1:3]`, …    |
+
+---
+
+## Optional `disallow_functions`
+
+There is no built-in library denylist. Everything not explicitly allow-listed is
+rejected. Safety comes from **`allow_functions`**.
+
+When you use a broad allow (`jax.*`), pass `disallow_functions=[...]` for a hard floor. Hits report
+**`call-not-allowed`**.
+
+Useful patterns under broad JAX/Flax allows:
+
+- Host / debug / experimental: `jax.experimental.*`, `jax.debug.*`, `jax.pure_callback`, `*.io_callback`, `*.host_callback*`
+- FFI / interop: `jax.dlpack.*`, `jax.ffi*`
+- Array ↔ disk: `jax.numpy.save`, `load`, `tofile`, `fromfile`, `memmap`, …
+- Checkpointing: `flax.serialization.*`, `flax.training.checkpoints.*`, `orbax.*`
+
+```python
+# public import, private use — still resolved and checked
+from jax.numpy import save as persist
+persist(x, "out.npz")   # call-not-allowed if disallow includes jax.numpy.save
+```
+
+---
+
+## See also
+
+- [verify.md](verify.md) — how checking works and what is allowed
+- `tests/verify/test_disallowed.py` — disallowed tests
+- `tests/verify/test_bypasses.py` — multi-step attack regressions

syft_restrict-0.1.0/docs/code-layout.md

@@ -0,0 +1,23 @@
+# Code layout
+
+Map of `src/syft_restrict/`:
+
+| Module          | Role                                                                  |
+| --------------- | --------------------------------------------------------------------- |
+| `astutil.py`    | Line ranges, import scan, small syntax helpers. No policy.            |
+| `policy.py`     | Allow lists, safe/banned builtins, hook list, `Policy`.               |
+| `verifier.py`   | Static checker: walks private lines default-deny, reports violations. |
+| `obfuscator.py` | After a clean verify: rename private identifiers, blank constants.    |
+| `runner.py`     | `run()` — verify → obfuscate → certificate.                           |
+| `errors.py`     | `RestrictError`, `PolicyViolation`.                                   |
+| `__init__.py`   | Public exports.                                                       |
+
+Tests under `tests/verify/`:
+
+| File                      | Role                                                            |
+| ------------------------- | --------------------------------------------------------------- |
+| `test_whitelist.py`       | Green path (aligns with [verify.md](verify.md))                 |
+| `test_whitelisted_lib.py` | Library paths + operator bundles                                |
+| `test_disallowed.py`      | Default-deny catalog (aligns with [blacklist.md](blacklist.md)) |
+| `test_bypasses.py`        | Multi-step escape regressions                                   |
+| `test_ranges.py`          | Invalid private ranges                                          |