natal-core 0.1.0.dev1__tar.gz → 0.1.0.dev2__tar.gz

natal_core-0.1.0.dev2/.github/copilot-instructions.md

@@ -0,0 +1,55 @@
+# Copilot Workspace Instructions for NATAL Core
+
+Apply these rules for all tasks in this repository.
+
+## Rule Priority
+
+Use this priority order when rules overlap:
+1. docstring_spec.md
+2. quality_checks_spec.md
+3. docstring_spec_cn.md
+4. quality_checks_spec_cn.md
+5. AGENTS.md
+
+## Mandatory Validation
+
+Before finalizing any code change, run and pass:
+- pytest
+- pyright
+- ruff check src demos
+
+If public exports changed, also run:
+- python scripts/generate_init_pyi.py
+
+## Docstring Rules
+
+- Follow Google style section names only.
+- Keep docstring content in English.
+- Ensure typed parameters, returns, and attributes.
+
+## Testing Rules
+
+- Add or update tests for non-trivial behavior changes.
+- Include boundary and error-path coverage when relevant.
+- Prefer pytest-collected tests over script-style smoke tests.
+- Keep tests deterministic.
+
+## Type and Lint Rules
+
+- Avoid new implicit Any in modified code.
+- Do not add # type: ignore unless necessary and justified.
+- Do not add broad Ruff ignore rules.
+
+## Response Requirements
+
+For implementation tasks, include:
+- Changed files summary
+- Behavior impact summary
+- Validation commands executed
+- Residual risks or follow-up items if any
+
+## Working Style
+
+- Keep changes small and focused.
+- Preserve existing behavior unless the task explicitly asks for behavior changes.
+- Prefer minimal-scope refactors.

{natal_core-0.1.0.dev1 → natal_core-0.1.0.dev2}/.gitignore

@@ -24,3 +24,5 @@ site/*
 .vscode/*
 !.vscode/settings.json
 !.vscode/extensions.json
+
+.trae/*

natal_core-0.1.0.dev2/.python-version

@@ -0,0 +1 @@
+3.12

natal_core-0.1.0.dev2/.readthedocs.yaml

@@ -0,0 +1,15 @@
+version: 2
+
+build:
+  os: ubuntu-24.04
+  tools:
+    python: "3.12"
+
+python:
+  install:
+    - method: pip
+      path: .
+    - requirements: docs/requirements.txt
+
+mkdocs:
+  configuration: mkdocs.en.yml

natal_core-0.1.0.dev2/.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+  "files.trimTrailingWhitespace": true,
+  "python.analysis.autoImportCompletions": true,
+  "C_Cpp_Runner.msvcBatchPath": ""
+}

natal_core-0.1.0.dev2/AGENTS.md

@@ -0,0 +1,63 @@
+# NATAL Core Agent Rules
+
+This file defines default instructions for AI Agents working in this repository.
+These rules are always-on at repository scope.
+
+## 1) Source of truth
+
+Follow these documents in descending priority:
+1. docstring_spec.md
+2. quality_checks_spec.md
+3. docstring_spec_cn.md (Chinese explanation)
+4. quality_checks_spec_cn.md (Chinese explanation)
+
+If rules conflict, prefer the English specification files.
+
+## 2) Mandatory quality gates before proposing completion
+
+For code changes, run and pass all of the following:
+- pytest
+- pyright
+- ruff check src demos
+
+Activate the repository virtual environment before running `pyright` so it uses
+the project-installed dependencies and configuration.
+
+After those commands complete, review `docs/` for any needed documentation updates and apply them before finalizing the change.
+
+If API exports changed, also run:
+- python scripts/generate_init_pyi.py
+
+## 3) Docstring requirements
+
+- Use Google style sections only.
+- Keep docstring text in English.
+- Ensure all parameters/returns/attributes are explicitly typed (annotation preferred).
+- Do not invent section headers.
+
+## 4) Testing requirements
+
+- Add or update tests for all non-trivial behavior changes.
+- Cover happy path, boundaries, and error paths when relevant.
+- Keep tests deterministic (fixed seeds for random behavior).
+- Prefer pytest-collected tests over script-style smoke tests.
+
+## 5) Typing and linting requirements
+
+- Do not introduce new implicit Any in modified code.
+- Do not add # type: ignore unless absolutely necessary with a clear reason.
+- Do not add broad Ruff ignores; keep exceptions minimal and documented.
+
+## 6) Change note requirements (human and AI)
+
+Every change summary must include:
+- Files changed
+- Behavior changes
+- Validation commands executed
+- Residual risks or follow-up items (if any)
+
+## 7) Working mode for this repository
+
+- Repository is mainly solo-maintained with occasional external contributions.
+- Keep changes small and focused.
+- Lightweight review is acceptable, but quality gates are never optional.

natal_core-0.1.0.dev2/CHANGELOG.md

@@ -0,0 +1,109 @@
+# Changelog
+
+## 🎉 2026.4.4.b - v0.0.0-beta released!
+- Refactor sampling method naming from 'use_dirichlet_sampling' to 'use_continuous_sampling'
+
+## 2026.4.4
+- feat(spatial): Add support for sparse CSR adjacency matrices and extend migration strategy (heterogenic kernels / hybrid)
+- feat(genetic_presets): Add ToxinAntidoteDrive preset and corresponding tests
+
+## 2026.4.3
+implemented spatial population with stable numba cache & parallel computation
+
+## 2026.3.28.c
+add type annotations
+
+## 2026.3.28.b
+add tests
+
+## 2026.3.28
+`Species.from_dict` (and any repeated child-structure creation) crashed with `ValueError: Child structure '…' already exists` on a second call with the same name, even though `GeneticStructure.__new__` and `_single_register` already silently return cached instances. The fix makes the entire singleton chain consistent.
+
+### Changes
+
+- **`ChildStructureRegistry.add`** — return `_storage[name]` instead of raising when the name already exists. `add` is now idempotent, matching `__new__`/`_single_register` semantics.
+
+```python
+# Before: second call raises
+sp1 = Species.from_dict("Mosquito", {"chr1": {"loc": ["WT", "Dr"]}})
+sp2 = Species.from_dict("Mosquito", {"chr1": {"loc": ["WT", "Dr"]}})
+# → ValueError: Child structure 'chr1' already exists.
+
+# After: same cached instance returned
+assert sp1 is sp2  # ✓
+```
+
+- **`tests/conftest.py`** — removed all cache-clearing fixtures (`_GLOBAL_STRUCTURE_CACHE`, `GeneticEntity._instance_cache`, `Genotype._cache`). Clearing entity caches while live `Locus._entities` sets retained old references caused `EntityRegistry` (identity-based deduplication) to register fresh objects as duplicates, producing spurious extra alleles. The singleton chain is now coherent end-to-end without manual clearing.
+
+- **`tests/test_genetic_structures.py`** — two regression tests covering idempotent `from_dict` (same instance returned) and no chromosome duplication on repeated calls.
+
+- **`.gitignore`** — removed `tests/` and `test_*` entries that were suppressing test file tracking.
+
+## 2026.3.27
+- 提升数值稳定性：修复了连续化抽样中的数值稳定性问题，当 $n \le 1$ 时退化为确定性
+- 初步重构文档和 docstring
+- 部分公用代码迁移到其他模块
+
+## 2026.3.26.e
+- 优化 UI 字体大小和绘图描点策略
+
+## 2026.3.26.d
+- 修复了 algorithms 中未正确处理配子储存与替换的问题
+- 为 algorithms 增加概率范围限制和异常报错
+- 为 algorithms 支持完整的 mating rates
+- 增加 numba 随机种子设置方法
+- 优化 UI 界面
+
+## 2026.3.26.c
+- 重命名 `hook/` -> `hooks/`，并同步更新相关导入路径
+- 重命名 `kernel/` -> `kernels/`，并同步更新相关导入路径
+
+## 2026.3.26.b
+- 移除 `simulation_kernels.py` 中未使用的 `run/run_tick` 旧入口，统一走 hook codegen 生成的 wrapper
+- 将 kernel wrapper codegen 从 `hook/compiler.py` 拆分到 `natal/kernel/codegen.py`，并将模板外置到 `src/natal/kernel/templates/kernel_wrappers.py.tmpl`
+- 将核心 `simulation_kernels` 实现迁移到 `natal/kernel/simulation_kernels.py`，保留 `natal/simulation_kernels.py` 兼容转发
+- 包结构重命名：`natal/hook` -> `natal/hooks`，`natal/kernel` -> `natal/kernels`，并同步更新内部导入与模板引用
+
+## 2026.3.26
+- 修复 `HomingDrive` 中 maternal deposition 总是有效的 bug
+- 增加 spatial model 的一些模块
+
+## 2026.3.25
+- 继续根据类型检查进行了一些修改，当前版本可用
+- 修改 UI title 和 favicon
+
+## 2026.3.24
+- 根据类型检查进行了一些修改，当前版本可用
+
+## 2026.3.23
+- 创建 webui
+- 部分概念重命名：`IndexCore` -> `IndexRegistry`，`recipe` -> `preset`
+- 恢复 `AgeStructuredPopulation` 的生存率、初始状态构建逻辑
+- 修复了 `gamete_allele_conversion.py` 中 `sex_filter` 无效的问题（现在可以识别字符串和数字表示的性别，且无效的表示会报错）
+- 优化文档
+
+## 2026.3.21
+- 重构 hook，优化 hook 缓存
+
+## 2026.3.20
+- 移除 `jitclass`，`PopulationState`, `DiscretePopulationState` 和 `PopulationConfig` 全部改为用 `NamedTuple`
+- 优化 `njit` 缓存
+- 增加编译提示信息
+
+## 2026.3.18.b
+- Hook 执行内核拆分：将 CSR 执行逻辑抽为外部函数 `execute_csr_event_arrays`，统一以 `HookProgram` 作为数据载体
+- 新增 `HookProgram` 与 `build_hook_program` 作为非 jitclass 路径的数据接口（第一阶段）
+- 完全移除 `HookRegistry` 符号，核心内核统一走 `HookProgram + execute_csr_event_program`
+- 优化 Declarative `when` 表达式的解析，支持 `and`、`or`、`not` 等逻辑运算符，增强表达能力
+- 为 `Op.scale` / `Op.subtract` / `Op.sample` / `Op.set_count` 增加抽样逻辑
+
+## 2026.3.18
+- 完善 HomingModificationDrive 逻辑
+- 修复了配子产生的默认逻辑，此前会产生各种 gamete label 的配子（且每一种归一化），现在只产生默认 label（"default" 或 index 0）
+- 修复了 hooks 的 bug（响应式 hook 被注册但未被使用）
+- 临时移除 `algorithms.fertilize_with_mating_genotype` 中对 `P` 的归一化，以支持致死效应的模拟（需后续检查正确性）
+
+## 2026.3.14
+- 优化种群初始化方式
+- 加入高级转换规则和 recipe 支持
+- 引入基因型 pattern 解析器

{natal_core-0.1.0.dev1 → natal_core-0.1.0.dev2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: natal-core
-Version: 0.1.0.dev1
+Version: 0.1.0.dev2
 Summary: Numba-Accelerated Toolkit for Analysis of Lifecycle-based population genetic dynamics
 Project-URL: Homepage, https://github.com/jyzhu-pointless/natal-core
 Project-URL: Documentation, https://natal-core.readthedocs.io/en/latest/
@@ -33,9 +33,11 @@ Requires-Dist: pytest>=7.0.0; extra == 'test'
 Description-Content-Type: text/markdown
 
 # ⚡️ NATAL Core
+
 **N**umba-**A**ccelerated **T**oolkit for **A**nalysis of **L**ifecycles.
 
-[![PyPI](https://img.shields.io/pypi/v/natal-core.svg)](https://pypi.org/project/natal-core/)
+[![GitHub](https://img.shields.io/github/v/release/jyzhu-pointless/natal-core?label=GitHub&color=purple)](https://github.com/jyzhu-pointless/natal-core/releases/latest)
+[![PyPI](https://img.shields.io/pypi/v/natal-core.svg?label=PyPI&color=yellow)](https://pypi.org/project/natal-core/)
 [![Python](https://img.shields.io/badge/Python-3.9+-blue.svg)](https://www.python.org/downloads/)
 [![NumPy](https://img.shields.io/badge/NumPy-2.0.0+-green.svg)](https://numpy.org/)
 [![Numba](https://img.shields.io/badge/Numba-0.60.0+-orange.svg)](https://numba.pydata.org/)

{natal_core-0.1.0.dev1 → natal_core-0.1.0.dev2}/README.md

@@ -1,7 +1,9 @@
 # ⚡️ NATAL Core
+
 **N**umba-**A**ccelerated **T**oolkit for **A**nalysis of **L**ifecycles.
 
-[![PyPI](https://img.shields.io/pypi/v/natal-core.svg)](https://pypi.org/project/natal-core/)
+[![GitHub](https://img.shields.io/github/v/release/jyzhu-pointless/natal-core?label=GitHub&color=purple)](https://github.com/jyzhu-pointless/natal-core/releases/latest)
+[![PyPI](https://img.shields.io/pypi/v/natal-core.svg?label=PyPI&color=yellow)](https://pypi.org/project/natal-core/)
 [![Python](https://img.shields.io/badge/Python-3.9+-blue.svg)](https://www.python.org/downloads/)
 [![NumPy](https://img.shields.io/badge/NumPy-2.0.0+-green.svg)](https://numpy.org/)
 [![Numba](https://img.shields.io/badge/Numba-0.60.0+-orange.svg)](https://numba.pydata.org/)

natal_core-0.1.0.dev2/TODO.md

@@ -0,0 +1,20 @@
+# TODO
+
+- 复用 `state_translation` 模块的翻译函数到 WebUI
+- `Species.from_dict` 构建后，应当支持指定性染色体
+- 检查性染色体的遗传逻辑是否正确（遗传张量雌雄 index 应该不同）
+
+## initialization / finish 现状
+
+```txt
+事件定义里仍有 initialization、finish（以及 first/early/late）。
+base_population.py (line 51)
+types.py (line 124)
+kernel 加速路径目前只执行 first/early/late（CSR+chain）。
+simulation_kernels.py (line 382)
+finish 是 Python 层触发（run 结束或 finish_simulation()），不在 kernel 事件链里。
+age_structured_population.py (line 878)
+discrete_generation_population.py (line 233)
+base_population.py (line 801)
+initialization 目前也在 Python 事件体系里，不在 kernel 执行路径。
+```

natal_core-0.1.0.dev2/benchmarks/README.md

@@ -0,0 +1,100 @@
+# Benchmarking & Profiling Scripts
+
+This directory contains performance analysis and optimization scripts for natal-core.
+
+## Scripts Overview
+
+### Profiling Scripts
+
+- **profile_spatial_hex_run.py**: Profile spatial.run(5) with 100x100 demes using cProfile
+  ```bash
+  python benchmarks/profile_spatial_hex_run.py
+  ```
+  Identifies Python-layer hotspots and cumulative function call patterns.
+
+- **profile_single_deme_repro.py**: Profile single-deme reproduction (older approach, may need updates)
+  ```bash
+  python benchmarks/profile_single_deme_repro.py
+  ```
+
+- **profile_repro_direct.py**: Direct kernel profiling using low-level SimulationKernels API
+  ```bash
+  python benchmarks/profile_repro_direct.py
+  ```
+
+### Benchmarking Scripts
+
+- **benchmark_kernel_overhead.py**: Compare state management overhead vs kernel execution time
+  ```bash
+  python benchmarks/benchmark_kernel_overhead.py
+  ```
+  Measures cost of stacking/unstacking arrays and compares with pure kernel time.
+
+- **benchmark_stages.py**: Per-tick performance analysis
+  ```bash
+  python benchmarks/benchmark_stages.py
+  ```
+  Measures individual tick execution cost across multiple runs.
+
+## Usage Tips
+
+### Running from Project Root
+All scripts use PYTHONPATH-relative imports, so run from project root:
+```bash
+cd /Users/pointless/Desktop/work/natal-core
+python benchmarks/profile_spatial_hex_run.py
+```
+
+### Interpreting Results
+
+**cProfile Output**:
+- Look for functions with high `cumtime` and high `ncalls`
+- `tottime` shows time spent in function itself
+- `cumtime` includes time in called functions
+
+**Benchmark Output**:
+- Average ± std dev gives performance stability indication
+- Variance > 10% suggests system-level noise or thermal effects
+- Single best/worst runs often excluded from analysis
+
+### Key Metrics from Last Run
+- **Spatial.run(5) total**: ~2.74-2.88s (5 runs)
+- **Per-tick cost**: ~0.574s ± 0.053s
+- **Python overhead**: 1.4% (state management)
+- **Numba kernel time**: 98.6% (bottleneck)
+
+## Performance Analysis Results
+
+📊 See `OPTIMIZATION_REPORT.md` in project root for detailed findings.
+
+### Recent Optimization
+Copy removal in spatial kernels (run_spatial_reproduction, run_spatial_survival, run_spatial_aging) achieved:
+- 29% speedup in state stack/unstack (3.74s → 2.64s)
+- 56% faster pytest suite (15.83s → 6.89s)
+- 1-2% overall demo improvement
+
+### Bottleneck Identified
+98.6% of time spent in @njit Numba code:
+- Not visible in cProfile (Numba functions appear as single call)
+- Dominated by algorithmic cost (mating, fertilization, offspring generation)
+- Further optimization requires:
+  - Numba algorithm tuning
+  - Parallelization flags adjustment
+  - Memory layout optimization
+
+## Future Optimization Directions
+
+1. **Numba Algorithm Level**
+   - Profile mating probability matrix computation
+   - Optimize offspring probability tensor construction
+   - Reduce allocations in inner loops
+
+2. **Parallelization**
+   - Test different n_jobs values
+   - Verify prange scheduling is optimal
+   - Check thread synchronization overhead
+
+3. **State Management** (higher risk)
+   - Cache frequently accessed arrays
+   - Consider mutable state structure (refactor from NamedTuple)
+   - Batch state updates

natal_core-0.1.0.dev2/benchmarks/benchmark_kernel_overhead.py

@@ -0,0 +1,126 @@
+# type:ignore
+"""
+Compare execution time of run_spatial_fn directly to identify if time is in Numba or Python.
+"""
+
+import sys
+from pathlib import Path
+
+# Ensure project root is in path for imports
+project_root = Path(__file__).parent.parent
+if str(project_root) not in sys.path:
+    sys.path.insert(0, str(project_root))
+
+import time
+import numpy as np
+from demos.spatial_hex import build_hex_spatial_population
+
+
+def benchmark_runs():
+    """Run 3 iterations of spatial.run(5) and measure times."""
+    times = []
+
+    for run_idx in range(3):
+        spatial = build_hex_spatial_population()
+        spatial.run(1)  # warmup
+
+        start = time.perf_counter()
+        spatial.run(5)
+        elapsed = time.perf_counter() - start
+        times.append(elapsed)
+
+        print(f"Run {run_idx + 1}: {elapsed:.4f}s")
+
+    print(f"\nAverage: {np.mean(times):.4f}s (±{np.std(times):.4f}s)")
+    return times
+
+
+def benchmark_kernel_time():
+    """Measure just the Numba kernel call without state management."""
+    spatial = build_hex_spatial_population()
+    spatial.run(1)  # warmup
+
+    # Manually reconstruct what run() does, but time only the kernel call
+    n_steps = 5
+
+    # Setup
+    hooks = spatial._demes[0].get_compiled_event_hooks()
+    run_fn = hooks.run_spatial_fn
+    registry = hooks.registry
+    config = spatial._shared_config()
+
+    # Method 1: Measure including state stack/unstack
+    print("=" * 60)
+    print("Method 1: Including state stack/unstack")
+    start = time.perf_counter()
+    for _ in range(3):
+        ind_all, sperm_all = spatial._stack_deme_state_arrays()
+        final_state_tuple, _history, was_stopped = run_fn(
+            ind_count_all=ind_all,
+            sperm_store_all=sperm_all,
+            config=config,
+            registry=registry,
+            tick=int(spatial._tick),
+            n_ticks=int(n_steps),
+            adjacency=spatial._adjacency,
+            migration_mode=spatial._migration_mode_code,
+            topology_rows=int(spatial._topology.rows),
+            topology_cols=int(spatial._topology.cols),
+            topology_wrap=bool(spatial._topology.wrap),
+            migration_kernel=spatial._migration_kernel_array(),
+            kernel_include_center=bool(spatial._kernel_include_center),
+            migration_rate=float(spatial._migration_rate),
+        )
+        spatial._apply_stacked_state(final_state_tuple[0], final_state_tuple[1], int(final_state_tuple[2]))
+    elapsed = time.perf_counter() - start
+    print(f"  3 runs (with stack/unstack): {elapsed:.4f}s ({elapsed/3:.4f}s per run)")
+
+    # Reset for next test
+    spatial = build_hex_spatial_population()
+    spatial.run(1)  # warmup
+    hooks = spatial._demes[0].get_compiled_event_hooks()
+    run_fn = hooks.run_spatial_fn
+    registry = hooks.registry
+    config = spatial._shared_config()
+
+    # Method 2: Measure with pre-stacked arrays
+    print("=" * 60)
+    print("Method 2: Pre-stacked arrays (one allocation)")
+    start = time.perf_counter()
+
+    # Stack once
+    ind_all, sperm_all = spatial._stack_deme_state_arrays()
+
+    # Time 3 runs with same arrays
+    t0 = time.perf_counter()
+    for _ in range(3):
+        final_state_tuple, _history, was_stopped = run_fn(
+            ind_count_all=ind_all,
+            sperm_store_all=sperm_all,
+            config=config,
+            registry=registry,
+            tick=0,
+            n_ticks=5,
+            adjacency=spatial._adjacency,
+            migration_mode=spatial._migration_mode_code,
+            topology_rows=int(spatial._topology.rows),
+            topology_cols=int(spatial._topology.cols),
+            topology_wrap=bool(spatial._topology.wrap),
+            migration_kernel=spatial._migration_kernel_array(),
+            kernel_include_center=bool(spatial._kernel_include_center),
+            migration_rate=float(spatial._migration_rate),
+        )
+        # Don't apply state for this benchmark
+
+    elapsed = time.perf_counter() - t0
+    print(f"  3 runs (pre-stacked, no unstack): {elapsed:.4f}s ({elapsed/3:.4f}s per run)")
+
+
+if __name__ == "__main__":
+    print("=" * 60)
+    print("BENCHMARK: spatial.run(5) with 100x100 demes (3 runs)")
+    print("=" * 60)
+    benchmark_runs()
+
+    print("\n")
+    benchmark_kernel_time()

natal_core-0.1.0.dev2/benchmarks/benchmark_stages.py

@@ -0,0 +1,44 @@
+# type:ignore
+"""
+Micro-benchmark to measure individual tick costs in spatial simulation.
+"""
+
+import sys
+from pathlib import Path
+
+# Ensure project root is in path for imports
+project_root = Path(__file__).parent.parent
+if str(project_root) not in sys.path:
+    sys.path.insert(0, str(project_root))
+
+import time
+import numpy as np
+from demos.spatial_hex import build_hex_spatial_population
+
+
+def benchmark_individual_stages():
+    """Measure cost of individual ticks."""
+    spatial = build_hex_spatial_population()
+    spatial.run(1)  # warmup
+
+    print("=" * 60)
+    print("Per-tick Performance Analysis")
+    print("=" * 60)
+
+    times = []
+    for run_idx in range(3):
+        spatial = build_hex_spatial_population()
+        spatial.run(1)  # warmup
+
+        start = time.perf_counter()
+        spatial.run(1)   # measure 1 tick
+        elapsed = time.perf_counter() - start
+        times.append(elapsed)
+
+        print(f"Run {run_idx + 1}, 1 tick: {elapsed:.4f}s")
+
+    print(f"\nAverage per tick: {np.mean(times):.4f}s (±{np.std(times):.4f}s)")
+
+
+if __name__ == "__main__":
+    benchmark_individual_stages()