jsonatapy 2.1.4__tar.gz → 2.1.6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- jsonatapy-2.1.6/.serena/.gitignore +2 -0
- jsonatapy-2.1.6/.serena/memories/architecture.md +86 -0
- jsonatapy-2.1.6/.serena/memories/conventions.md +32 -0
- jsonatapy-2.1.6/.serena/memories/core.md +39 -0
- jsonatapy-2.1.6/.serena/memories/memory_maintenance.md +33 -0
- jsonatapy-2.1.6/.serena/memories/suggested_commands.md +40 -0
- jsonatapy-2.1.6/.serena/memories/task_completion.md +23 -0
- jsonatapy-2.1.6/.serena/memories/tech_stack.md +19 -0
- jsonatapy-2.1.6/.serena/project.yml +160 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/Cargo.lock +258 -119
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/Cargo.toml +4 -3
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/PKG-INFO +15 -14
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/README.md +14 -13
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/pyproject.toml +1 -1
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/ast.rs +66 -11
- jsonatapy-2.1.6/src/ast_transform.rs +1986 -0
- jsonatapy-2.1.6/src/datetime.rs +1873 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/evaluator.rs +1647 -321
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/functions.rs +1 -1
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/lib.rs +111 -98
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/parser.rs +201 -9
- jsonatapy-2.1.6/src/signature.rs +768 -0
- jsonatapy-2.1.6/tests/datetime_picture_suite.rs +173 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/tests/integration_test.rs +173 -2
- jsonatapy-2.1.6/tests/parent_and_focus_binding_suite.rs +186 -0
- jsonatapy-2.1.4/src/datetime.rs +0 -442
- jsonatapy-2.1.4/src/signature.rs +0 -436
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/.gitignore +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/.gitmodules +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/CHANGELOG.md +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/LICENSE +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/benches/evaluator_bench.rs +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/examples/evaluator_demo.rs +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/examples/parser_demo.rs +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/python/jsonatapy/__init__.py +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/python/jsonatapy/py.typed +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/compiler.rs +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/parser/README.md +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/value.rs +0 -0
- {jsonatapy-2.1.4 → jsonatapy-2.1.6}/src/vm.rs +0 -0
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
## Evaluator/compiler architecture (non-obvious internals)
|
|
2
|
+
|
|
3
|
+
### JValue (src/value.rs)
|
|
4
|
+
```rust
|
|
5
|
+
pub enum JValue {
|
|
6
|
+
Null, Bool(bool), Number(f64),
|
|
7
|
+
String(Rc<str>), Array(Rc<Vec<JValue>>), Object(Rc<IndexMap<String, JValue>>),
|
|
8
|
+
Undefined, Lambda { lambda_id, params, name, signature },
|
|
9
|
+
Builtin { name }, Regex { pattern, flags },
|
|
10
|
+
#[cfg(feature = "python")] LazyPyDict(Rc<lazy::LazyPyDict>),
|
|
11
|
+
}
|
|
12
|
+
```
|
|
13
|
+
- O(1) clone for every variant (Rc bump for heap types).
|
|
14
|
+
- Constructors: `JValue::string(..)`, `::array(..)`, `::lambda(..)`, `::builtin(..)`.
|
|
15
|
+
- Gotchas: `Rc<str>` doesn't satisfy `Borrow<String>` (use `&**s` for HashMap lookups);
|
|
16
|
+
`Rc<Vec>` can't `.into_iter()` (use `.iter()` or `.to_vec()`/`Rc::make_mut()` to mutate).
|
|
17
|
+
|
|
18
|
+
### Null vs Undefined
|
|
19
|
+
JSONata distinguishes explicit `null` from "not present" `Undefined`; the tree-walker
|
|
20
|
+
historically produced `Null` for almost all "field not found" cases (a pre-existing bug fixed by
|
|
21
|
+
migrating ~20 hand-written call sites in evaluator.rs — fast paths, multi-step loops,
|
|
22
|
+
out-of-range array index, `apply_stages`, etc. — plus ~20 more downstream sites that assumed
|
|
23
|
+
"Null means missing": `signature.rs::validate_and_coerce`, builtin arg guards, arithmetic
|
|
24
|
+
explicit-null branches, `??`, `$not`/`$string`/`$append`/`$spread`). Reference pattern for
|
|
25
|
+
correct "present vs missing" semantics: `compiled_field_step` in evaluator.rs — Object: field-or-
|
|
26
|
+
Undefined; Array: map+flatten+skip-Undefined; other: Undefined. Exception: `AstNode::Variable`
|
|
27
|
+
(unbound `$var`) deliberately still returns `Null`, not `Undefined` — only path/field "not found"
|
|
28
|
+
semantics were migrated, not unbound-variable semantics. If tree-walker code decides
|
|
29
|
+
present-vs-missing and a value isn't propagating as expected, suspect a leftover pre-migration
|
|
30
|
+
site (grep `unwrap_or(JValue::Null)` / `JValue::Null =>` near field-access code).
|
|
31
|
+
|
|
32
|
+
### Lambda representation
|
|
33
|
+
- Lambdas live in the scope stack's `lambdas: HashMap<String, StoredLambda>`; `JValue::Lambda`
|
|
34
|
+
only carries a `lambda_id` (+ params/name/signature) referencing that map.
|
|
35
|
+
- `lambda_id` MUST come from `Evaluator`'s monotonic `next_lambda_id`/`fresh_lambda_id()` counter,
|
|
36
|
+
never a pointer-derived id (`format!("{:p}", ast_node)`) — an AST node's address is fixed per
|
|
37
|
+
source site but each evaluation of a recursive/repeated lambda creates a new closure instance;
|
|
38
|
+
pointer-derived ids alias distinct instances in the scope map, causing ~0.5%-frequency
|
|
39
|
+
intermittent wrong-closure bugs (spurious T2010 / U1001 depth-limit errors). This was a real,
|
|
40
|
+
hard-to-reproduce bug (issue #35) — reproducible only in a tight pure-Rust loop with a fresh
|
|
41
|
+
`Evaluator` per iteration.
|
|
42
|
+
- `collect_lambda_ids()` transitively follows a `StoredLambda`'s captured_env.
|
|
43
|
+
- `:=` with a lambda RHS stores only in the lambdas map, not in regular bindings.
|
|
44
|
+
|
|
45
|
+
### Compilation pipeline (bytecode fast path)
|
|
46
|
+
`evaluator::try_compile_expr(ast)` → `CompiledExpr` (IR) → `compiler::BytecodeCompiler::compile`
|
|
47
|
+
→ `BytecodeProgram` (flat `Vec<Instr>`) → `peephole()` folds (`PushData+GetField` →
|
|
48
|
+
`GetDataField`, `GetVar+GetField` → `GetVarField`, elides `Not+Not`) → `vm::Vm::new(bc).run(...)`.
|
|
49
|
+
`JsonataExpression` (lib.rs) caches `bytecode: OnceCell<Option<BytecodeProgram>>`; all 4 evaluate
|
|
50
|
+
entrypoints try the VM first and fall back to the tree-walker.
|
|
51
|
+
|
|
52
|
+
Key correctness rules baked into the compiler/VM (violate these and specific test-suite groups
|
|
53
|
+
regress):
|
|
54
|
+
- And/Or compile to jump-based short-circuit code, not a binary instruction — required for
|
|
55
|
+
exprs like `foo = '' or $number(foo) = 0` where evaluating rhs unconditionally would be wrong.
|
|
56
|
+
- Arithmetic/comparison ops carry compile-time `ExplicitNull` flags (`Add(lhs_en, rhs_en)`,
|
|
57
|
+
`CmpLt(...)`) for correct T2002/T2010 errors.
|
|
58
|
+
- `get_field_cached` flattens one level of nested arrays when mapping (matches `compiled_field_step`)
|
|
59
|
+
— required for `flattening/case001`.
|
|
60
|
+
- `MakeArray` skips undefined and flattens one level; but a "complex" `ArrayConstruct` (elements
|
|
61
|
+
marked `is_nested=true`) must NOT flatten and falls back to `EvalFallback`.
|
|
62
|
+
- `FieldPath` with filter predicates always falls back to `eval_compiled` (tree-walker) — filters
|
|
63
|
+
aren't compiled.
|
|
64
|
+
- Numeric predicates like `[0]`, `[1]` in a path are INDEX access, not boolean filters — must
|
|
65
|
+
fall back to the tree-walker in `try_compile_path`.
|
|
66
|
+
- Peephole 2-instruction folds must remap BOTH consumed instructions' jump targets to the same
|
|
67
|
+
new address, or jumps land mid-fold.
|
|
68
|
+
|
|
69
|
+
### Lambda compilation (Phase 3)
|
|
70
|
+
`StoredLambda.compiled_body: Option<CompiledExpr>`, compiled at definition time via
|
|
71
|
+
`try_compile_expr_with_allowed_vars`. Fast path in `invoke_stored_lambda` skips scope push/pop
|
|
72
|
+
when: body is compiled AND no signature AND not a thunk AND no captured functions. HOF fast
|
|
73
|
+
paths (`$map`/`$filter`/`$reduce`) handle both inline lambdas and stored-lambda variables.
|
|
74
|
+
`compiled_body` is deliberately set to `None` (forcing tree-walker) for: transform body,
|
|
75
|
+
ChainPipe composition, TCO thunks, partial-application body markers — these are known-uncompiled
|
|
76
|
+
sites, not bugs.
|
|
77
|
+
|
|
78
|
+
### LazyPyDict (Phase 5, Python-only perf path)
|
|
79
|
+
`JValue::LazyPyDict(Rc<LazyPyDict>)` (src/lazy.rs) lets field access on a Python dict skip full
|
|
80
|
+
materialization into `JValue::Object`; `to_object()` must skip `Undefined` entries when
|
|
81
|
+
materializing (absent fields must not appear as explicit nulls). Every code path that switches
|
|
82
|
+
on `JValue` object/array shape (sort keys, `$keys`/`$spread`/`$lookup`/`$merge`/`$sort`,
|
|
83
|
+
`$each`/`$sift`, the `@` tuple binding, `apply_transform_deep`) needs an explicit `LazyPyDict` arm
|
|
84
|
+
or a materialize-first step — it's easy to add a new object-shaped builtin and forget this.
|
|
85
|
+
Benefits sparse field-access patterns (2–5x speedup measured); dense full-object access can
|
|
86
|
+
slightly regress vs. eager materialization — don't assume it's a strict win everywhere.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
## Codebase conventions
|
|
2
|
+
|
|
3
|
+
- **Mirror the JS reference structurally**: module names and function names should track
|
|
4
|
+
`jsonata-js`'s `src/*.js` (parser.js → parser.rs, functions.js → functions.rs, etc.) so
|
|
5
|
+
upstream diffs are easy to port. Don't restructure Rust modules away from this mapping
|
|
6
|
+
without strong reason.
|
|
7
|
+
- **Two execution paths, one semantics**: tree-walking `Evaluator` (evaluator.rs) is the
|
|
8
|
+
reference/fallback implementation; `compiler.rs`/`vm.rs` is a bytecode fast path that falls
|
|
9
|
+
back to the tree-walker (`EvalFallback`) for anything not compilable. New JSONata semantics
|
|
10
|
+
should be correct in the tree-walker first (it's the oracle used for compatibility testing),
|
|
11
|
+
then optionally fast-pathed into the compiler/VM — see `mem:architecture` for the split's
|
|
12
|
+
concrete rules (short-circuit and/or, explicit-null flags, array flattening, etc.).
|
|
13
|
+
- **`JValue` (src/value.rs) is the value type**, not `serde_json::Value` — `Rc`-wrapped
|
|
14
|
+
variants for O(1) clone (`String(Rc<str>)`, `Array(Rc<Vec<JValue>>)`, `Object(Rc<IndexMap<...>>)`).
|
|
15
|
+
Lambdas/builtins/regexes are first-class `JValue` variants, not tagged JSON objects.
|
|
16
|
+
`serde_json::Value` is only used at I/O boundaries (JSON string parsing, test fixtures).
|
|
17
|
+
See `mem:architecture` for the full variant list and common gotchas (`Rc<str>` vs
|
|
18
|
+
`Borrow<String>`, `Rc<Vec>` iteration/mutation).
|
|
19
|
+
- **Null vs Undefined are semantically distinct** (JSONata quirk carried over from JS) — do not
|
|
20
|
+
conflate them when writing new "field not found" or "value missing" logic; see
|
|
21
|
+
`mem:architecture` for the correct reference pattern and history of a subtle prior bug here.
|
|
22
|
+
- **Rust style**: 2021 edition, `cargo fmt` + `cargo clippy -- -D warnings` (zero warnings,
|
|
23
|
+
non-negotiable), `Result<T, E>` for fallible ops, rustdoc `///` on public APIs, comments should
|
|
24
|
+
explain *deviations from JS behavior*, not restate what code does.
|
|
25
|
+
- **Python style**: PEP 8 via ruff, type hints required on public APIs (mypy strict), ruff
|
|
26
|
+
double-quote/100-col formatting — no black despite CLAUDE.MD mentioning it (ruff format is
|
|
27
|
+
what's actually configured in pyproject.toml).
|
|
28
|
+
- **Test-driven against the JS suite**: the JSONata-js reference suite
|
|
29
|
+
(`tests/python/test_reference_suite.py` over the `tests/jsonata-js` submodule) is the
|
|
30
|
+
compatibility oracle. Any evaluator/compiler change should be checked against the FULL
|
|
31
|
+
1258-case suite, not just a hand-picked subset — subtle regressions have historically hidden
|
|
32
|
+
in low-frequency edge cases (e.g. lambda id aliasing at ~0.5% call frequency).
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
## jsonatapy: source map
|
|
2
|
+
|
|
3
|
+
Rust-based Python extension implementing JSONata (JSON query/transform language), targeting
|
|
4
|
+
100% compatibility with the reference `jsonata-js` v2.1.0 test suite (1258/1258 passing).
|
|
5
|
+
Two published artifacts from one codebase: `jsonata-core` (crates.io, pure Rust) and
|
|
6
|
+
`jsonatapy` (PyPI, PyO3 wrapper). See root `CLAUDE.MD` for full project charter/design goals —
|
|
7
|
+
this memory only covers what CLAUDE.MD omits or what changes faster than that doc.
|
|
8
|
+
|
|
9
|
+
### Layout
|
|
10
|
+
- `src/` — Rust crate (`jsonata-core`). Core files: `ast.rs`, `parser.rs` (+ `src/parser/README.md`),
|
|
11
|
+
`evaluator.rs` (~8700 lines, tree-walking evaluator — the reference/fallback path),
|
|
12
|
+
`compiler.rs` + `vm.rs` (bytecode compiler + VM — the fast path, falls back to the
|
|
13
|
+
tree-walker for anything not compilable), `value.rs` (`JValue`), `functions.rs`, `datetime.rs`,
|
|
14
|
+
`signature.rs`, `lib.rs` (PyO3 Python boundary).
|
|
15
|
+
- `python/jsonatapy/` — thin Python package; `__init__.py` re-exports the compiled `_jsonatapy`
|
|
16
|
+
extension module and adds optional orjson-backed `json_dumps`/`json_loads`.
|
|
17
|
+
- `tests/jsonata-js/` — git submodule, the upstream JS reference suite (source of the 1258 cases).
|
|
18
|
+
- `tests/python/test_reference_suite.py` — runs the submodule's cases against the compiled extension.
|
|
19
|
+
- `tests/integration_test.rs` — Rust-side integration tests (separate from inline `#[cfg(test)]`).
|
|
20
|
+
- `benchmarks/` — cross-language perf comparisons (Python/JS/Rust) driving the "faster than JS" claim.
|
|
21
|
+
- `scripts/setup-uv.sh` — canonical from-scratch dev bootstrap (installs uv+Rust, builds, runs both test suites).
|
|
22
|
+
|
|
23
|
+
### Further memories
|
|
24
|
+
- `mem:tech_stack` — toolchain, package managers, version pins.
|
|
25
|
+
- `mem:suggested_commands` — day-to-day dev/build/test commands.
|
|
26
|
+
- `mem:conventions` — code style and architectural conventions specific to this codebase.
|
|
27
|
+
- `mem:task_completion` — what to run before considering a change done.
|
|
28
|
+
- `mem:architecture` — JValue/evaluator/compiler pipeline internals, non-obvious invariants.
|
|
29
|
+
|
|
30
|
+
### Non-obvious invariants
|
|
31
|
+
- Two independent execution paths must stay behaviorally identical: the tree-walking
|
|
32
|
+
`Evaluator` (evaluator.rs) and the bytecode `Vm` (vm.rs/compiler.rs). The VM is preferred when
|
|
33
|
+
compilable; anything it can't compile falls back to the tree-walker. Changes to evaluation
|
|
34
|
+
semantics generally need to be made in both, or the fallback path silently diverges.
|
|
35
|
+
- Repo root has multiple stale/duplicate Python venvs (`.venv`, `.venv-wsl`, `.venv.bak`) — don't
|
|
36
|
+
assume one is canonical without checking; `uv` manages its own env from `pyproject.toml`/`uv.lock`.
|
|
37
|
+
- Serena's language auto-detection for this project defaults to `typescript` (misled by the
|
|
38
|
+
`tests/jsonata-js` submodule / `node_modules`) unless `.serena/project.yml`'s `languages:` list
|
|
39
|
+
is corrected to `[rust, python]` — check this after any Serena project reset.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Memory Maintenance
|
|
2
|
+
|
|
3
|
+
## Discovery Model
|
|
4
|
+
|
|
5
|
+
- Core principle: progressive discovery through references, building a graph of memories.
|
|
6
|
+
- Initially, agents are provided with the list of all memories (names only).
|
|
7
|
+
- Agents should read `mem:core` as the top-level entry point (graph root).
|
|
8
|
+
This memory should contain references to other memories covering major project domains.
|
|
9
|
+
The referenced memories shall, in turn, shall contain references to even more specific memories, and so on.
|
|
10
|
+
The depth of the graph shall depend on the project complexity.
|
|
11
|
+
- Use topics/folders to group related memories in order to make the content structure explicit.
|
|
12
|
+
Folders can mirror project structure (e.g. modules like frontend/backend) or topics like debugging, architecture, etc.
|
|
13
|
+
- Memory references must use a mem: prefix inside backticks, e.g. `mem:frontend/core`.
|
|
14
|
+
The surrounding text should clearly indicate when to read the memory/which content to expect.
|
|
15
|
+
The text should provide more precise guidance than the memory name alone,
|
|
16
|
+
i.e. avoid a reference like "frontend debugging: `mem:frontend/debugging` and instead make clear which aspects of frontend debugging are covered.
|
|
17
|
+
- Memories themselves should not contain information about when to read them; this is the responsibility of the referring memory.
|
|
18
|
+
|
|
19
|
+
## Style
|
|
20
|
+
|
|
21
|
+
Dense agent notes, not prose docs. Prefer invariants, terse bullets.
|
|
22
|
+
Avoid obvious context, rationale, and examples unless they prevent likely mistakes.
|
|
23
|
+
Keep guidance durable and generalizable, not task-local.
|
|
24
|
+
|
|
25
|
+
## Add/update threshold
|
|
26
|
+
|
|
27
|
+
Add or update memories only with stable, non-obvious project conventions that avoid complex rediscovery in the future.
|
|
28
|
+
Do not add: quick-read facts; generic language/framework knowledge; one-off task notes; volatile line-level details; behavior likely to change soon.
|
|
29
|
+
|
|
30
|
+
## Maintenance Actions
|
|
31
|
+
|
|
32
|
+
- Renaming memories: References are updated automatically if handled via Serena's memory rename tool.
|
|
33
|
+
- Checking for stale memories (e.g. after deletion): Call `serena memories check` for a report.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
## Suggested commands (Linux/WSL)
|
|
2
|
+
|
|
3
|
+
### Build
|
|
4
|
+
```bash
|
|
5
|
+
maturin develop --release # build Rust ext + install into active venv (release, use for perf-sensitive testing)
|
|
6
|
+
maturin develop # debug build, faster to compile, use for quick iteration
|
|
7
|
+
maturin build --release # produce a wheel in dist/
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
### Test
|
|
11
|
+
```bash
|
|
12
|
+
cargo test # Rust unit + tests/integration_test.rs
|
|
13
|
+
uv run pytest tests/python/test_reference_suite.py # 1258 JSONata-js reference cases (primary compat gate)
|
|
14
|
+
uv run pytest tests/python/ -v # full Python suite
|
|
15
|
+
uv run pytest tests/python/test_functions.py::test_string_functions -v # single test
|
|
16
|
+
```
|
|
17
|
+
Reference suite depends on the `tests/jsonata-js` git submodule being initialized:
|
|
18
|
+
`git submodule update --init --recursive`.
|
|
19
|
+
|
|
20
|
+
### Lint / format (zero-warnings policy on Rust side)
|
|
21
|
+
```bash
|
|
22
|
+
cargo fmt
|
|
23
|
+
cargo clippy -- -D warnings
|
|
24
|
+
uv run ruff check python/ tests/
|
|
25
|
+
uv run ruff format python/ tests/
|
|
26
|
+
uv run mypy python/
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### Benchmark
|
|
30
|
+
```bash
|
|
31
|
+
cargo bench # Criterion, uses `bench` feature
|
|
32
|
+
uv run python benchmarks/python/... # cross-language comparisons (see benchmarks/ for entrypoints)
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
### Linux/WSL-specific notes
|
|
36
|
+
- Standard GNU coreutils apply (no BSD `find`/`sed` quirks).
|
|
37
|
+
- Compiled extension `.so` files live directly in `python/jsonatapy/` (`_jsonatapy.cpython-3XX-x86_64-linux-gnu.so`)
|
|
38
|
+
after `maturin develop` — one per Python minor version tested.
|
|
39
|
+
- Repo has leftover `.venv`, `.venv-wsl`, `.venv.bak` dirs; prefer `uv run` which resolves its
|
|
40
|
+
own environment rather than assuming a particular venv is active.
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
## Definition of done
|
|
2
|
+
|
|
3
|
+
Run all of these before considering a change complete; all are zero-tolerance (project charter
|
|
4
|
+
says 100% test compatibility and zero-clippy-warnings are non-negotiable):
|
|
5
|
+
|
|
6
|
+
```bash
|
|
7
|
+
cargo fmt --check # or `cargo fmt` to auto-fix, then verify
|
|
8
|
+
cargo clippy -- -D warnings
|
|
9
|
+
cargo test
|
|
10
|
+
maturin develop --release # rebuild extension so Python tests exercise the new Rust code
|
|
11
|
+
uv run pytest tests/python/test_reference_suite.py # full 1258-case JS compat suite
|
|
12
|
+
uv run pytest tests/python/ -v # rest of the Python suite
|
|
13
|
+
uv run ruff check python/ tests/
|
|
14
|
+
uv run mypy python/
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
If the change touches evaluator/compiler semantics, confirm both execution paths agree:
|
|
18
|
+
the reference suite should pass under the default (VM-preferred) path AND when the VM/compiler
|
|
19
|
+
fast path is bypassed (forcing the tree-walker) — see `mem:architecture` for how prior migrations
|
|
20
|
+
verified this dual-path parity.
|
|
21
|
+
|
|
22
|
+
If the change is user-facing (new function, new signature, changed error), update
|
|
23
|
+
`CHANGELOG.md` per Keep a Changelog format (per CLAUDE.MD).
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
## Tech stack
|
|
2
|
+
|
|
3
|
+
- **Rust**: edition 2021, `rust-version = "1.70"` (stable channel only, no nightly features per
|
|
4
|
+
CLAUDE.MD). Crate name `jsonata-core` (lib name `jsonata_core`), cdylib+rlib.
|
|
5
|
+
- **Key Rust deps**: `pyo3` 0.29 (optional, feature `python`), `serde`/`serde_json` (preserve_order),
|
|
6
|
+
`simd-json` 0.17 (optional, feature `simd`, default-on — SIMD JSON parsing fast path),
|
|
7
|
+
`indexmap` (ordered maps for JSONata objects), `regex`, `chrono`, `thiserror`, `stacker`
|
|
8
|
+
(stack growth for deep recursion), `rand`.
|
|
9
|
+
- **Cargo features**: `default = ["simd"]`, `simd` (dep:simd-json), `python` (dep:pyo3,
|
|
10
|
+
enables the PyO3 boundary), `bench` (exposes `_bench` facade for Criterion).
|
|
11
|
+
- **Python**: >=3.10, published wheels for 3.10–3.14. Build backend: **maturin** (module name
|
|
12
|
+
`jsonatapy._jsonatapy`, python-source = `python`).
|
|
13
|
+
- **Package manager**: **uv** (uv.lock present) — prefer `uv run <cmd>` over bare `pip`/`python`.
|
|
14
|
+
- **Python dev tooling**: ruff (lint+format, line-length 100, double quotes, target py310),
|
|
15
|
+
mypy (strict mode), pytest (+pytest-cov, pytest-xdist for parallel runs).
|
|
16
|
+
- **Benchmarking**: Criterion (Rust, `cargo bench`, `evaluator_bench`), plus Python-side
|
|
17
|
+
benchmarks comparing against `jsonata` (JS) and `jsonata-rs`.
|
|
18
|
+
- **Docs**: mkdocs + mkdocs-material (`docs/`, `mkdocs.yml`), not Sphinx despite CLAUDE.MD
|
|
19
|
+
mentioning Sphinx — mkdocs is what's actually wired up in pyproject.toml `[project.optional-dependencies.docs]`.
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# the name by which the project can be referenced within Serena/when chatting with the LLM.
|
|
2
|
+
project_name: "jsonatapy"
|
|
3
|
+
|
|
4
|
+
# list of languages for which language servers are started (LSP backend only); choose from:
|
|
5
|
+
# ada al angular ansible bash
|
|
6
|
+
# bsl clojure cpp cpp_ccls crystal
|
|
7
|
+
# csharp csharp_omnisharp cue dart elixir
|
|
8
|
+
# elm erlang fortran fsharp gdscript
|
|
9
|
+
# go groovy haskell haxe hlsl
|
|
10
|
+
# html java json julia kotlin
|
|
11
|
+
# latex lean4 lua luau markdown
|
|
12
|
+
# matlab msl nix ocaml pascal
|
|
13
|
+
# perl php php_phpactor php_phpantom powershell
|
|
14
|
+
# python python_jedi python_pyrefly python_ty r
|
|
15
|
+
# rego ruby ruby_solargraph rust scala
|
|
16
|
+
# scss solidity svelte swift systemverilog
|
|
17
|
+
# terraform toml typescript typescript_vts vue
|
|
18
|
+
# yaml zig
|
|
19
|
+
# (This list may be outdated; generated with scripts/print_language_list.py;
|
|
20
|
+
# For the current list, see values of Language enum here:
|
|
21
|
+
# https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
|
|
22
|
+
# For some languages, there are alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
|
|
23
|
+
# Note:
|
|
24
|
+
# - For C, use cpp
|
|
25
|
+
# - For JavaScript, use typescript
|
|
26
|
+
# - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
|
|
27
|
+
# - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
|
|
28
|
+
# - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
|
|
29
|
+
# - For Free Pascal/Lazarus, use pascal
|
|
30
|
+
# Special requirements:
|
|
31
|
+
# Some languages require additional setup/installations.
|
|
32
|
+
# See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
|
|
33
|
+
# When using multiple languages, the first language server that supports a given file will be used for that file.
|
|
34
|
+
# The first language is the default language and the respective language server will be used as a fallback.
|
|
35
|
+
# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
|
|
36
|
+
languages:
|
|
37
|
+
- rust
|
|
38
|
+
- python
|
|
39
|
+
|
|
40
|
+
# the encoding used by text files in the project
|
|
41
|
+
# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
|
|
42
|
+
encoding: "utf-8"
|
|
43
|
+
|
|
44
|
+
# optional shell command to run before the language backend (LSP or JetBrains) is initialised.
|
|
45
|
+
# the command runs in the project root directory and is only executed if the project is trusted
|
|
46
|
+
# (see trusted_project_path_patterns in the global configuration).
|
|
47
|
+
# serena waits for the command to exit: a non-zero exit code is logged as an error but does not
|
|
48
|
+
# abort activation. a per-project timeout (activation_command_timeout, default 180s) is the safety
|
|
49
|
+
# backstop for non-terminating commands; on expiry the process is killed and activation continues.
|
|
50
|
+
# example: activation_command: "npx nx run-many -t build"
|
|
51
|
+
activation_command:
|
|
52
|
+
|
|
53
|
+
# maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
|
|
54
|
+
# must be a positive number.
|
|
55
|
+
activation_command_timeout: 180
|
|
56
|
+
|
|
57
|
+
# line ending convention to use when writing source files.
|
|
58
|
+
# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default)
|
|
59
|
+
# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings.
|
|
60
|
+
line_ending:
|
|
61
|
+
|
|
62
|
+
# The language backend to use for this project.
|
|
63
|
+
# If not set, the global setting from serena_config.yml is used.
|
|
64
|
+
# Valid values: LSP, JetBrains
|
|
65
|
+
# Note: the backend is fixed at startup. If a project with a different backend
|
|
66
|
+
# is activated post-init, an error will be returned.
|
|
67
|
+
language_backend:
|
|
68
|
+
|
|
69
|
+
# whether to use project's .gitignore files to ignore files
|
|
70
|
+
ignore_all_files_in_gitignore: true
|
|
71
|
+
|
|
72
|
+
# advanced configuration option allowing to configure language server-specific options.
|
|
73
|
+
# Maps the language key to the options.
|
|
74
|
+
# The settings are considered only if the project is trusted (see global configuration to define trusted projects).
|
|
75
|
+
# See https://oraios.github.io/serena/02-usage/050_configuration.html#language-server-specific-settings
|
|
76
|
+
ls_specific_settings: {}
|
|
77
|
+
|
|
78
|
+
# list of workspace folder paths (LSP backend only).
|
|
79
|
+
# These folders will be used to build up Serena's symbol index.
|
|
80
|
+
# Paths must be within the project root and should thus be relative to the project root.
|
|
81
|
+
# Furthermore, the paths should not be filtered by ignore settings.
|
|
82
|
+
# Default setting: The entire project root folder (".") is considered.
|
|
83
|
+
# In (large) monorepos, this can be used to index only subfolders of the project root, e.g.
|
|
84
|
+
# ls_workspace_folders:
|
|
85
|
+
# - "./subproject1"
|
|
86
|
+
# - "./subproject2"
|
|
87
|
+
ls_workspace_folders: ["."]
|
|
88
|
+
|
|
89
|
+
# list of additional workspace folder paths for cross-package reference support.
|
|
90
|
+
# Paths can be absolute or relative to the project root.
|
|
91
|
+
# Each folder is registered as an LSP workspace folder, enabling language servers to discover
|
|
92
|
+
# symbols and references across package boundaries, but these folders are not indexed by Serena,
|
|
93
|
+
# i.e. the respective symbols will not be found using Serena's symbol search tools.
|
|
94
|
+
# Example:
|
|
95
|
+
# additional_workspace_folders:
|
|
96
|
+
# - ../sibling-package
|
|
97
|
+
# - ../shared-lib
|
|
98
|
+
ls_additional_workspace_folders: []
|
|
99
|
+
|
|
100
|
+
# list of additional paths to ignore in this project.
|
|
101
|
+
# Same syntax as gitignore, so you can use * and **.
|
|
102
|
+
# Note: global ignored_paths from serena_config.yml are also applied additively.
|
|
103
|
+
ignored_paths: []
|
|
104
|
+
|
|
105
|
+
# whether the project is in read-only mode
|
|
106
|
+
# If set to true, all editing tools will be disabled and attempts to use them will result in an error
|
|
107
|
+
# Added on 2025-04-18
|
|
108
|
+
read_only: false
|
|
109
|
+
|
|
110
|
+
# list of tool names to exclude.
|
|
111
|
+
# This extends the existing exclusions (e.g. from the global configuration)
|
|
112
|
+
# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
|
|
113
|
+
excluded_tools: []
|
|
114
|
+
|
|
115
|
+
# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default).
|
|
116
|
+
# This extends the existing inclusions (e.g. from the global configuration).
|
|
117
|
+
# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
|
|
118
|
+
included_optional_tools: []
|
|
119
|
+
|
|
120
|
+
# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools.
|
|
121
|
+
# This cannot be combined with non-empty excluded_tools or included_optional_tools.
|
|
122
|
+
# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
|
|
123
|
+
fixed_tools: []
|
|
124
|
+
|
|
125
|
+
# list of mode names that are to be activated by default, overriding the setting in the global configuration.
|
|
126
|
+
# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
|
|
127
|
+
# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
|
|
128
|
+
# Otherwise, this overrides the setting from the global configuration (serena_config.yml).
|
|
129
|
+
# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
|
|
130
|
+
# for this project.
|
|
131
|
+
# This setting can, in turn, be overridden by CLI parameters (--mode).
|
|
132
|
+
# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
|
|
133
|
+
default_modes:
|
|
134
|
+
|
|
135
|
+
# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
|
|
136
|
+
# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
|
|
137
|
+
# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
|
|
138
|
+
added_modes:
|
|
139
|
+
|
|
140
|
+
# initial prompt for the project. It will always be given to the LLM upon activating the project
|
|
141
|
+
# (contrary to the memories, which are loaded on demand).
|
|
142
|
+
initial_prompt: ""
|
|
143
|
+
|
|
144
|
+
# time budget (seconds) per tool call for the retrieval of additional symbol information
|
|
145
|
+
# such as docstrings or parameter information.
|
|
146
|
+
# This overrides the corresponding setting in the global configuration; see the documentation there.
|
|
147
|
+
# If null or missing, use the setting from the global configuration.
|
|
148
|
+
symbol_info_budget:
|
|
149
|
+
|
|
150
|
+
# list of regex patterns which, when matched, mark a memory entry as read‑only.
|
|
151
|
+
# Extends the list from the global configuration, merging the two lists.
|
|
152
|
+
read_only_memory_patterns: []
|
|
153
|
+
|
|
154
|
+
# list of regex patterns for memories to completely ignore.
|
|
155
|
+
# Matching memories will not appear in list_memories or activate_project output
|
|
156
|
+
# and cannot be accessed via read_memory or write_memory.
|
|
157
|
+
# To access ignored memory files, use the read_file tool on the raw file path.
|
|
158
|
+
# Extends the list from the global configuration, merging the two lists.
|
|
159
|
+
# Example: ["_archive/.*", "_episodes/.*"]
|
|
160
|
+
ignored_memory_patterns: []
|