nadata 0.1.0__tar.gz

nadata-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Nathan Cerisara
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

nadata-0.1.0/PKG-INFO

@@ -0,0 +1,191 @@
+Metadata-Version: 2.4
+Name: nadata
+Version: 0.1.0
+Summary: A declarative data schema and object-graph modeling framework
+License-Expression: MIT
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: mypy>=1.8; extra == "dev"
+Dynamic: license-file
+
+# Nadata
+
+[![CI](https://github.com/nath54/Nadata/actions/workflows/ci.yml/badge.svg)](https://github.com/nath54/Nadata/actions/workflows/ci.yml)
+
+**Nadata** is a declarative data schema and object-graph framework with its own
+language. You describe your data model in `.nastruct` schema files, populate it
+in `.nadata` data files, and get a typed, queryable, persistent object graph —
+usable from the command line or as an embedded Python library.
+
+```text
+# structs.nastruct — the model
+class Human {
+    name: Str;
+    birthday: DateTime = DateTime();
+
+    @age(now: DateTime = DateTime.now()) -> DateTime {
+        return now - self.birthday;
+    }
+}
+
+# data.nadata — the data
+tom = Human(name="Tom", birthday=DateTime(d=12, m=4, y=2003));
+
+print("Tom is an adult:", tom.age(DateTime.now()).year >= 18);
+Array[Human] adults = Human.where((h: Human) -> h.age(DateTime.now()).year >= 18);
+```
+
+## Features
+
+- **Typed schema language** — structs, inheritance, attributes with defaults,
+  `const` bind-once attributes, methods and computed properties, standalone
+  functions; `import "...";` shares schema libraries across projects
+- **String library and hash indexes** — typed string members (`split`,
+  `join`, `replace`, `contains`, ...), and opt-in per-attribute hash
+  indexes (`index(Human, "email")` + `get_by(Human, "email", x)`) with
+  automatic insert / update / removal maintenance — O(1) equality
+  lookups instead of full scans
+- **Rich type system** — sized integers/floats (`Int32`, `UFloat16`, ...),
+  `Str`, `Bool`, `DateTime`, `Color`, `Array[T]`, `Dict[K, V]`, `Set[T]`,
+  `Queue[T]` / `PriorityQueue[T, P]`, sparse spatial grids
+  (`GridSquare2D[T]`, `GridHex2D[T]`, `GridCube3D[T]`), shape-checked
+  `Tensor[T, d1, ...]` with symbolic dims bound to `const` attributes,
+  entity references; indexing (`arr[0]`,
+  `dict["k"]`) and string concatenation
+- **Error handling** — `raise "message";` and `try { ... } catch (err) { ... }`
+  in scripts and method bodies; runtime errors (indexing, I/O builtins)
+  are catchable too, while `return` / `break` / `continue` pass through
+- **Declarative data files** — forward and circular references, script
+  statements, `print`, chained polymorphic queries
+- **Runtime introspection** — `elt.id`, `type(x)`, `elt.list_attr()` /
+  `list_methods()`, schema-constrained `getattr`/`setattr`, and database-wide
+  `schema_classes()` / `schema_functions()`
+- **Persistence** — whole-database dumps, dirty-only delta appends, and
+  removal tombstones to a single `.ndb` file
+- **Bounded memory** — disk-backed lazy rows (~50 bytes/row index), a
+  bounded LRU decode cache with clean-only eviction, `pin`/`unpin`,
+  `flush`/`clear_cache`, and `remove_from_db` from scripts — stream
+  millions of rows while the model stays hot
+- **Autograd and training** — tape-based reverse-mode differentiation on Float
+  tensors (arithmetic, `matmul`, reductions, shape ops, `gather`, slices,
+  `relu`), seeded data partitioning for train/valid/test splits, and built-in
+  `SGD` — train models end-to-end in `.nadata` scripts
+- **Sessions & identity** — a project folder is a session: `#init` seeds once
+  per world, `#auto` syncs every open, `#manual` scripts take parameters
+  (`#manual(lr: Float = 0.05)`), and `nid = "name"` gives instances stable,
+  updatable identity (`get_by_nid`, `nids`, `first`/`last`) — schema files
+  re-merge on every open, and schema evolution is reconciled against
+  persisted rows (removed attributes are dropped; type changes are
+  reported, never fatal). I/O builtins (`read_json`, `write_csv`,
+  `file_hash`, ...) resolve against the project root, with
+  `get_setting`/`set_setting` freshness markers
+- **Interactive console** — `nadata console <path>` REPL over any loaded
+  database, with persistent state and expression echoing
+- **Python API** — load, query, mutate, execute raw source, persist
+- **Zero dependencies** — pure Python 3.10+
+
+## Installation
+
+```bash
+git clone <repository-url>
+cd Nadata
+pip install -e .
+pip install -e .[dev]   # optional: pytest + mypy for development
+```
+
+## Quick start
+
+```bash
+# Run a project (schema + data) and execute its scripts
+nadata run examples/ex0_simple
+
+# Print a summary of a project or a saved .ndb database
+nadata info examples/ex0_simple
+nadata info store.ndb
+
+# Persist a project to a single database file
+nadata dump examples/ex0_simple store.ndb
+
+# Explore it interactively (works on folders and .ndb files)
+nadata console examples/ex0_simple
+nadata console store.ndb
+```
+
+Or from Python:
+
+```python
+import nadata
+
+app = nadata.Nadata.load("./examples/ex0_simple")
+
+print(app.get_attribute("tom", "birthplace"))   # Paris
+print(app.count("Human"))                        # 3
+
+app.dump("store.ndb")                            # persist
+```
+
+## Documentation
+
+| Document | Contents |
+|---|---|
+| [docs/tutorial.md](docs/tutorial.md) | Step-by-step tutorial: your first project |
+| [docs/syntax.md](docs/syntax.md) | Full language reference: rules and syntax |
+| [docs/architecture.md](docs/architecture.md) | How the engine works, stage by stage |
+| [docs/api.md](docs/api.md) | Python API and CLI reference |
+| [docs/persistence.md](docs/persistence.md) | The `.ndb` format, dumps and incremental saves |
+| [docs/benchmarks.md](docs/benchmarks.md) | Engine stage profiling, synthetic benchmarks, SQLite comparison |
+| [docs/development.md](docs/development.md) | Code style, testing, contributing |
+
+## Examples
+
+The `examples/` folder contains self-contained projects, from basics to
+edge cases:
+
+| Example | Demonstrates |
+|---|---|
+| `ex0_simple` | Family graph: inheritance, methods, forward refs, queries |
+| `ex1_primitives` | Every sized integer/float type and boundary values |
+| `ex2_inheritance` | Circular references, polymorphic queries |
+| `ex3_collections` | Nested arrays, dictionaries, `Color`, `DateTime` math |
+| `ex4_expressions` | `if`/`else`, boolean operators, local declarations |
+| `ex5_edge_cases` | Parser edge cases: comments, spacing, empty constructs |
+| `ex6_loops_collections` | Loops (`for-in`, C-style, `while`), fuel guards, collections |
+| `ex7_queues_sets` | FIFO / priority queues, typed sets, membership probes |
+| `ex8_grids_tensors` | Sparse spatial grids (square/hex/cube), shape-checked tensors |
+| `ex9_constants` | `const` bind-once attributes, symbolic tensor dims |
+| `ex10_cnn` | Conv1d/conv2d, pooling, and a basic convolutional neural network |
+| `ex11_transformer` | A minimal transformer block: attention, LayerNorm, feed-forward, `topk` sampling |
+| `ex12_optimizer_checkpoint` | Training a gelu network with Adam, checkpointing optimizer state to a sidecar file, and resuming from it |
+
+## Editor support
+
+Syntax highlighting for `.nastruct` and `.nadata` files ships as a
+VS Code / VSCodium extension in
+[`editors/vscode-nadata`](editors/vscode-nadata). `.vsix` packages are
+not stored in git — build one and install it:
+
+```bash
+cd editors/vscode-nadata
+npx -y @vscode/vsce package --allow-missing-repository   # -> nadata-support-<version>.vsix
+code --install-extension nadata-support-<version>.vsix
+```
+
+Packaged releases attach the current `.vsix` to the matching git tag.
+
+A sync test (`tests/test_grammar.py`) keeps the grammar aligned with the
+language: it fails whenever a builtin type, keyword, or scope is missing.
+
+## Development
+
+```bash
+python -m pytest tests/ -q        # run the test suite
+python -m mypy nadata/               # type check (strict)
+python -m scripts.inspect_parse examples/ex0_simple   # inspect the parse IR
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

nadata-0.1.0/README.md

@@ -0,0 +1,178 @@
+# Nadata
+
+[![CI](https://github.com/nath54/Nadata/actions/workflows/ci.yml/badge.svg)](https://github.com/nath54/Nadata/actions/workflows/ci.yml)
+
+**Nadata** is a declarative data schema and object-graph framework with its own
+language. You describe your data model in `.nastruct` schema files, populate it
+in `.nadata` data files, and get a typed, queryable, persistent object graph —
+usable from the command line or as an embedded Python library.
+
+```text
+# structs.nastruct — the model
+class Human {
+    name: Str;
+    birthday: DateTime = DateTime();
+
+    @age(now: DateTime = DateTime.now()) -> DateTime {
+        return now - self.birthday;
+    }
+}
+
+# data.nadata — the data
+tom = Human(name="Tom", birthday=DateTime(d=12, m=4, y=2003));
+
+print("Tom is an adult:", tom.age(DateTime.now()).year >= 18);
+Array[Human] adults = Human.where((h: Human) -> h.age(DateTime.now()).year >= 18);
+```
+
+## Features
+
+- **Typed schema language** — structs, inheritance, attributes with defaults,
+  `const` bind-once attributes, methods and computed properties, standalone
+  functions; `import "...";` shares schema libraries across projects
+- **String library and hash indexes** — typed string members (`split`,
+  `join`, `replace`, `contains`, ...), and opt-in per-attribute hash
+  indexes (`index(Human, "email")` + `get_by(Human, "email", x)`) with
+  automatic insert / update / removal maintenance — O(1) equality
+  lookups instead of full scans
+- **Rich type system** — sized integers/floats (`Int32`, `UFloat16`, ...),
+  `Str`, `Bool`, `DateTime`, `Color`, `Array[T]`, `Dict[K, V]`, `Set[T]`,
+  `Queue[T]` / `PriorityQueue[T, P]`, sparse spatial grids
+  (`GridSquare2D[T]`, `GridHex2D[T]`, `GridCube3D[T]`), shape-checked
+  `Tensor[T, d1, ...]` with symbolic dims bound to `const` attributes,
+  entity references; indexing (`arr[0]`,
+  `dict["k"]`) and string concatenation
+- **Error handling** — `raise "message";` and `try { ... } catch (err) { ... }`
+  in scripts and method bodies; runtime errors (indexing, I/O builtins)
+  are catchable too, while `return` / `break` / `continue` pass through
+- **Declarative data files** — forward and circular references, script
+  statements, `print`, chained polymorphic queries
+- **Runtime introspection** — `elt.id`, `type(x)`, `elt.list_attr()` /
+  `list_methods()`, schema-constrained `getattr`/`setattr`, and database-wide
+  `schema_classes()` / `schema_functions()`
+- **Persistence** — whole-database dumps, dirty-only delta appends, and
+  removal tombstones to a single `.ndb` file
+- **Bounded memory** — disk-backed lazy rows (~50 bytes/row index), a
+  bounded LRU decode cache with clean-only eviction, `pin`/`unpin`,
+  `flush`/`clear_cache`, and `remove_from_db` from scripts — stream
+  millions of rows while the model stays hot
+- **Autograd and training** — tape-based reverse-mode differentiation on Float
+  tensors (arithmetic, `matmul`, reductions, shape ops, `gather`, slices,
+  `relu`), seeded data partitioning for train/valid/test splits, and built-in
+  `SGD` — train models end-to-end in `.nadata` scripts
+- **Sessions & identity** — a project folder is a session: `#init` seeds once
+  per world, `#auto` syncs every open, `#manual` scripts take parameters
+  (`#manual(lr: Float = 0.05)`), and `nid = "name"` gives instances stable,
+  updatable identity (`get_by_nid`, `nids`, `first`/`last`) — schema files
+  re-merge on every open, and schema evolution is reconciled against
+  persisted rows (removed attributes are dropped; type changes are
+  reported, never fatal). I/O builtins (`read_json`, `write_csv`,
+  `file_hash`, ...) resolve against the project root, with
+  `get_setting`/`set_setting` freshness markers
+- **Interactive console** — `nadata console <path>` REPL over any loaded
+  database, with persistent state and expression echoing
+- **Python API** — load, query, mutate, execute raw source, persist
+- **Zero dependencies** — pure Python 3.10+
+
+## Installation
+
+```bash
+git clone <repository-url>
+cd Nadata
+pip install -e .
+pip install -e .[dev]   # optional: pytest + mypy for development
+```
+
+## Quick start
+
+```bash
+# Run a project (schema + data) and execute its scripts
+nadata run examples/ex0_simple
+
+# Print a summary of a project or a saved .ndb database
+nadata info examples/ex0_simple
+nadata info store.ndb
+
+# Persist a project to a single database file
+nadata dump examples/ex0_simple store.ndb
+
+# Explore it interactively (works on folders and .ndb files)
+nadata console examples/ex0_simple
+nadata console store.ndb
+```
+
+Or from Python:
+
+```python
+import nadata
+
+app = nadata.Nadata.load("./examples/ex0_simple")
+
+print(app.get_attribute("tom", "birthplace"))   # Paris
+print(app.count("Human"))                        # 3
+
+app.dump("store.ndb")                            # persist
+```
+
+## Documentation
+
+| Document | Contents |
+|---|---|
+| [docs/tutorial.md](docs/tutorial.md) | Step-by-step tutorial: your first project |
+| [docs/syntax.md](docs/syntax.md) | Full language reference: rules and syntax |
+| [docs/architecture.md](docs/architecture.md) | How the engine works, stage by stage |
+| [docs/api.md](docs/api.md) | Python API and CLI reference |
+| [docs/persistence.md](docs/persistence.md) | The `.ndb` format, dumps and incremental saves |
+| [docs/benchmarks.md](docs/benchmarks.md) | Engine stage profiling, synthetic benchmarks, SQLite comparison |
+| [docs/development.md](docs/development.md) | Code style, testing, contributing |
+
+## Examples
+
+The `examples/` folder contains self-contained projects, from basics to
+edge cases:
+
+| Example | Demonstrates |
+|---|---|
+| `ex0_simple` | Family graph: inheritance, methods, forward refs, queries |
+| `ex1_primitives` | Every sized integer/float type and boundary values |
+| `ex2_inheritance` | Circular references, polymorphic queries |
+| `ex3_collections` | Nested arrays, dictionaries, `Color`, `DateTime` math |
+| `ex4_expressions` | `if`/`else`, boolean operators, local declarations |
+| `ex5_edge_cases` | Parser edge cases: comments, spacing, empty constructs |
+| `ex6_loops_collections` | Loops (`for-in`, C-style, `while`), fuel guards, collections |
+| `ex7_queues_sets` | FIFO / priority queues, typed sets, membership probes |
+| `ex8_grids_tensors` | Sparse spatial grids (square/hex/cube), shape-checked tensors |
+| `ex9_constants` | `const` bind-once attributes, symbolic tensor dims |
+| `ex10_cnn` | Conv1d/conv2d, pooling, and a basic convolutional neural network |
+| `ex11_transformer` | A minimal transformer block: attention, LayerNorm, feed-forward, `topk` sampling |
+| `ex12_optimizer_checkpoint` | Training a gelu network with Adam, checkpointing optimizer state to a sidecar file, and resuming from it |
+
+## Editor support
+
+Syntax highlighting for `.nastruct` and `.nadata` files ships as a
+VS Code / VSCodium extension in
+[`editors/vscode-nadata`](editors/vscode-nadata). `.vsix` packages are
+not stored in git — build one and install it:
+
+```bash
+cd editors/vscode-nadata
+npx -y @vscode/vsce package --allow-missing-repository   # -> nadata-support-<version>.vsix
+code --install-extension nadata-support-<version>.vsix
+```
+
+Packaged releases attach the current `.vsix` to the matching git tag.
+
+A sync test (`tests/test_grammar.py`) keeps the grammar aligned with the
+language: it fails whenever a builtin type, keyword, or scope is missing.
+
+## Development
+
+```bash
+python -m pytest tests/ -q        # run the test suite
+python -m mypy nadata/               # type check (strict)
+python -m scripts.inspect_parse examples/ex0_simple   # inspect the parse IR
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

nadata-0.1.0/nadata/__init__.py

@@ -0,0 +1,15 @@
+"""
+Nadata package entry point.
+
+Re-exports the high level Nadata facade and the CLI main function so the
+library can be imported (import nadata) and the console script installed
+(nadata = nadata.cli:main) from a single package.
+"""
+
+# Import Modules
+from nadata.api import Nadata
+from nadata.cli import main
+
+__all__ = ["Nadata", "main", "__version__"]
+
+__version__ = "0.1.0"