pyxel-config-core 0.7.0__tar.gz

pyxel_config_core-0.7.0/CHANGELOG.md

@@ -0,0 +1,361 @@
+# Changelog
+
+All notable changes to pyxel-config-core are recorded here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses
+[Semantic Versioning](https://semver.org/).
+
+Releases up to 0.5.0 were published to PyPI before the repository tagged its
+releases, so they link to PyPI. From 0.6.0 on, each release is published from a
+`core-v` tag.
+
+## [Unreleased]
+
+## [0.7.0] - 2026-11-06
+
+### Security
+
+- The deep check's safety gate now refuses every file Pyxel opens or looks for
+  while it loads. `charge_to_volt`'s `value` and `from_capacitance`, plain or
+  under `sensitive` and `reference`, are read as images when they are text, so
+  a config could make the machine send a request to a URL, and the answer said
+  which local files exist. Any text in a detector's characteristics is now
+  refused as a file, along with a calibration's `target_data_path` and
+  `weights_from_file`. A new drift guard reads Pyxel's source and fails when a
+  release adds a place that opens or evaluates something during a load.
+- An APD converter's `lambda` may no longer ask for unbounded arithmetic.
+  `lambda x: 9**9**9**9` never returned, and `lambda x: 1 << 10**11` asked for
+  about 12.5 GB. A power now takes a plain exponent of at most 64 and a base
+  with no power in it, and shifts and bit operations are refused.
+- The deep check runs `pyxel.load()` in a process of its own, with a time limit
+  (120 seconds; `--deep-timeout` or `deep_timeout=` changes it). A load still
+  running then is stopped and reported as an error. Without trust, the process
+  also has a limit on its memory where the system allows one, and it never
+  imports modules from the current folder.
+- Text past 20,000 characters is not read as Python. On Python 3.10 a chain of
+  some 100,000 operators in `times`, `values` or an APD `function` ended the
+  whole process with a segmentation fault, where a config now gets a refusal
+  that says why.
+
+### Added
+
+- `--deep-timeout SECONDS` on `validate`, and `deep_timeout=` in the library.
+- More of what Pyxel refuses as it loads a config is refused without running
+  it: readout `times` that are zero, not increasing, empty, or not below
+  `start_time`; a custom observation with no file of values; and a
+  `charge_to_volt` value outside Pyxel's range.
+- What stops a run once it has started is refused too: a pipeline with no step
+  to run, a `save_data_to_file` bucket Pyxel cannot save, an observation
+  parameter whose key or values Pyxel cannot use, a sweep whose parameters are
+  all disabled, and a model missing what it reads from the detector (such as
+  `dark_current` with no temperature).
+- A focal plane's `scene_generation` steps are checked: their keys and their
+  `func`. So is a plane's geometry, where `height: 0` ends in a singular matrix
+  at run, and its readout, with the same rules as a flat config's.
+- A `func` with no dot, with a trailing dot, or under a misspelled `pyxel.model.`
+  is refused, on flat configs and on tiles alike, since Pyxel cannot import it.
+
+### Changed
+
+- `validate` reads a readout's `times` the way Pyxel does, so a plane accepts
+  `numpy.linspace(1, 3, 3)`, `times_from_file` and `exposure: {}`, all of which
+  Pyxel runs.
+- A step that is not a known model answers to the schema's generic branch. A
+  user's own model with `enabled: maybe` gets one finding instead of 21, and
+  `func: 123` gets one that names `func`.
+- `-n` on `search` must be a positive number; 0 and negative numbers exit 2.
+- `pyxel-config-mcp` without the `mcp` extra explains the fix, as
+  `pyxel-config mcp` does, rather than showing a traceback.
+
+### Fixed
+
+- Text Pyxel's reader cannot read is a finding at its line rather than an
+  internal error: `2026-13-45`, `!!int abc`, `0b_` and the like. So are a `row`
+  of hundreds of digits, a key holding a slash or written as both `1` and `"1"`,
+  a tile named `"²"`, a non-text stage in `fix`, and `range()` of a huge number
+  in `--deep`.
+- `fix --json` works on a config holding `!!binary`, and `fix` on an unreadable
+  encoding reports it and exits as `validate` does.
+- `validate --deep --strict` with no Pyxel installed exits 3, as the README says.
+- A reader closing the pipe early on Windows no longer turns into exit 120.
+- `params` takes the full path that `list_models` hands out.
+- `explain` gives a next step on every finding, run rules and missing required
+  fields included.
+- `fix` gives the same suggestion on every run, and never suggests the same name
+  it just refused. A `func` of 20,000 characters no longer takes seconds.
+- A merging map shallower than its anchor is no longer reported as a duplicate
+  key.
+
+## [0.6.1] - 2026-10-14
+
+### Added
+
+- A documentation site at https://pyxel-config-lab-ede25c.gitlab.io/docs/:
+  installation, a quickstart, guides to the command line, the library, the MCP
+  server and semantic search, how validation works, and an API reference read
+  from the docstrings. Its guide pages include sections of the README, so the
+  two cannot say different things, and the quickstart's printed output is a
+  test (`tests/test_docs_quickstart.py`).
+- NumPy-style docstrings on every public function and result type.
+- A `docs` extra with what builds the site.
+- A pipeline step without `func` is refused: Pyxel builds every step from it,
+  a disabled one included, although its schema leaves it optional.
+- A focal-plane tile with more than one detector block is refused, as Pyxel's
+  tile loader refuses it, and only Pyxel's four detector keys count as a
+  detector block.
+- `model_parameters(model, stage=None)`, `pyxel-config params` and the
+  `get_pyxel_model_parameters` MCP tool cover every model the bundled schema
+  accepts (`glow`, the CMOS noise models, `load_detector` and `save_detector`
+  among the 19 that had nothing). Each argument carries its type, `required`,
+  `has_default`, Pyxel's own `default` and the `example` value Pyxel's example
+  configs use. `stage` picks between the two stages a name such as
+  `simple_collection` exists in.
+
+### Changed
+
+- The package's Documentation link points at the documentation site instead
+  of the README.
+- Text that cannot be read as a config (a YAML syntax error, a key written
+  twice, an alias bomb, nesting hundreds of levels deep, text that is not
+  UTF-8) is a finding in the `validate` and `explain` reports, with exit code
+  1, rather than stopping the run with exit code 3. One such file no longer
+  leaves a SARIF or Code Quality report empty. A file that cannot be opened is
+  listed in the report too, and the run still exits with 3.
+- A file saved as UTF-16 or UTF-32 (as Windows PowerShell 5.1's `>` writes
+  it) is an error, since Pyxel reads a config file as UTF-8 only. Its content
+  is still checked.
+- `ParameterInfo.default` is Pyxel's own default rather than the catalogue's
+  example value, which is now `ParameterInfo.example`. A step that leaves an
+  argument out gets the default; writing the example overrides it (for
+  `simple_conversion`'s `quantum_efficiency`, the detector's own value).
+  Types are spelled as Python spells them: `float`, `list[int] | null`.
+- A number written as text, such as `1e3`, is reported only where text is
+  not what the field takes: a step's `name` or an output folder is left
+  alone, while a number field, or one that takes a number or a file name
+  (`charge_to_volt`'s `value`), still is.
+- `fix` refuses `-w` and `-o FILE` together (exit code 2) instead of quietly
+  repairing in place. With `-o` or to stdout, a config with nothing to repair
+  is written exactly as it was, comments included.
+
+### Fixed
+
+- `.nan` and `.inf` in a detector field Pyxel bounds (temperature, quantum
+  efficiency, full well capacity, the sizes) are refused, as Pyxel's own range
+  checks refuse them.
+- A top-level key that is not text (`yes:`, `1:`, `~:`) no longer crashes
+  `validate_config`, `explain_config` or `autofix`.
+- SARIF and Code Quality reports read each file once: a report with 3,000
+  findings took over four minutes and now takes well under a second. A
+  finding inside a mapping merged with `<<: *anchor` lands on its own line,
+  and a file outside the working directory is a `file:` URI in SARIF.
+- The deep check hands Pyxel the file's text exactly as written, comments
+  included, and the deep check and `fix` write files as bytes: a config saved
+  with Windows line endings no longer arrives with every line break doubled.
+- A semantic search tried before the index is built no longer makes the
+  index look built to `doctor` and to every later search.
+- The semantic search guide installs `"pyxel-config-core[rag]"` from PyPI
+  rather than an editable install that only works in a clone, and the README
+  says which engine each tutorial search uses.
+
+### Security
+
+- The deep check's safety gate refuses binary data (YAML `!!binary`) anywhere
+  in a config: Pyxel unpickles an APD converter function given that way, which
+  can run any code. Without trust it also refuses the files Pyxel opens while
+  it loads (`times_from_file`, a custom observation's `from_file`, an APD
+  converter's `filename`, URLs included) and a value range of more than a
+  million values. `--trust-config` (or `trusted=True`) lifts all three for a
+  config you wrote yourself; on the MCP server they apply whenever it was
+  started with `--allow-deep`.
+
+## [0.6.0] - 2026-10-03
+
+### Added
+
+- `validate_config(config, deep=True)` hands the YAML to `pyxel.load()` when
+  Pyxel is installed and reports anything the loader rejects, which catches
+  what the JSON schema cannot express (readout times out of order, for
+  example).
+  `ValidationResult.deep_checked` says whether the loader actually ran. The
+  same switch is `--deep` on the `validate` and `explain` commands and
+  `deep=true` on the `validate_pyxel_config` and `explain_pyxel_config` MCP
+  tools.
+- Loading runs part of a config: Pyxel evaluates text such as a readout's
+  `times` as Python and imports each model's module. Before it loads, the deep
+  check reads every such value and refuses to run the config unless each is
+  plain data or a known harmless call (`pyxel_config_core/expressions.py`).
+  `--trust-config` (or `trusted=True`) skips that reading for a config you
+  wrote yourself. On the MCP server the deep check is off unless the server is
+  started with `pyxel-config mcp --allow-deep`.
+- The deep check names the Pyxel that judged the config:
+  `ValidationResult.deep_pyxel_version` and `Explanation.deep_pyxel_version`
+  carry the installed release whose loader ran, the summary says "and loads
+  in Pyxel 3.0.2", and the CLI's report and JSON say the same. Warnings
+  Pyxel raises while loading (those Python shows a user, not its own
+  deprecation notices) are reported as warning findings.
+- `--deep` that cannot run (Pyxel missing, or installed but failing to import,
+  with the real import error named) exits with 3, "could not run", instead of
+  passing with 0. Any unexpected error also exits with 3, never with 1, which
+  means "the config has errors".
+- `explain_config` returns an `Explanation`: `valid`, a one-sentence `summary`
+  ("This CCD exposure config has 2 problems in the charge_generation stage"),
+  and the diagnostics, each carrying a concrete `next_step` (the model name to
+  rename to, the accepted types, the section to add, the release that renamed
+  a field).
+- `pyxel_config_core/config_shape.py`, the Python port of the browser app's
+  `configShape.js`: it names the running mode and detector a config uses,
+  reports a missing or duplicated one, and offers a did-you-mean for a
+  misspelled top-level key (`ccd_detecor`, `exposre`). The root-level anyOf
+  chorus ("'ccd_detector' is a required property", four times) is gone.
+- `--format {text,json,sarif,codequality}` on `validate` and `explain`.
+  `--json` remains as the short spelling of `--format json`. SARIF 2.1.0 and
+  GitLab Code Quality reports carry the line of the offending key, found by
+  walking the YAML.
+- `pyxel_config_core/data/BUNDLE.json`, a manifest naming the Pyxel release
+  the bundled schema was generated from and the schema's SHA-256.
+  `schema_version()` exposes it, `ValidationResult.pyxel_version` and
+  `Explanation.pyxel_version` carry it on every verdict, and `doctor` prints
+  it alongside whether the shipped schema still matches the manifest.
+- `Diagnostic.category` (`schema`, `func`, `shape`, `advisory`, `pyxel-load`),
+  which is the SARIF rule id and the Code Quality check name.
+- `search_pyxel_tutorials` takes an explicit `semantic: bool = False`; it was
+  documented as keyword search but silently used the vector index when one
+  existed.
+- `py.typed`, `[tool.ruff]` and `[tool.mypy]` configuration, a `dev` extra,
+  and Python 3.13 in the classifiers.
+- A `.pre-commit-hooks.yaml` at the repo root exposing `pyxel-config-validate`.
+- `scripts/refresh_bundle.py` at the repo root: the one reproducible command
+  for upgrading every bundled artifact to a new Pyxel release, with `--check`
+  for CI.
+
+- Run rules (`run_rules.py`): what Pyxel 3.0.2 refuses at load or stops on at
+  run time although its schema accepts it, reported as errors in the new
+  `pyxel-rule` category. A readout with both `times` and `times_from_file`, a
+  calibration with both `weights` and `weights_from_file`, an observation with
+  no parameters, `simple_measurement` with neither a `gain` nor a detector
+  `charge_to_volt`, `simple_amplifier` without `pre_amplification`, and
+  `simple_adc` without its ADC settings. The browser applies the same rules,
+  and CI runs a fixture for each through the real Pyxel.
+- A run rule for whole numbers written as decimals where Pyxel refuses one:
+  a detector's `row` or `col` and a CCD's `image_row` or `overscan_row` stop
+  `pyxel.load()` when written `128.0`, and an exposure's `pipeline_seed`
+  stops the run. Fields Pyxel takes a decimal in (`adc_bit_resolution`, a
+  wavelength's `resolution`) are left alone.
+- A run rule for a model whose own code refuses the config's detector
+  class: 19 of Pyxel 3.0.2's models check the class and raise a TypeError
+  on any other (`cdm` on a CMOS detector, `apd_gain` on a CCD). Each was
+  found by calling the model on every class, and CI repeats that against
+  the installed Pyxel. Models Pyxel's metadata restricts but whose code runs
+  on any detector are left alone.
+- Focal plane arrays, which the Config Lab exports and the bundled schema
+  has no branch for, are validated instead of refused as a config missing
+  its detector and pipeline (`pyxel_config_core/fpa.py`). The plane is
+  checked against Pyxel's focal-plane loader as the browser checks it (one
+  running mode, exposure only; a physical geometry with one scale above 0;
+  tile IDs 1..N; positions as mappings with an angle of 0, 90, 180 or 270;
+  a plane-wide scene; nothing beside `focal_plane_array`), and each tile is
+  validated against the schema and the run rules as the flat config Pyxel
+  builds it from. Every finding names its tile and its path in the plane; a
+  finding in the plane's running mode is said once. `fix` repairs model
+  names inside tiles, `explain` says which tiles hold the problems, and the
+  deep check reads every tile and the plane-wide scene before Pyxel loads
+  the plane. The pre-commit hook and the MCP tools follow.
+
+### Changed
+
+- Schema errors are grouped by path. The anyOf alternatives of a single field
+  collapse into one line naming every accepted type ("'x' is not one of the
+  accepted types: number, object, null"), at most eight paths are listed, and
+  a closing diagnostic counts what was left out. Previously the first five
+  unique (path, message) pairs were shown and the rest silently dropped.
+- A pipeline step whose `func` is known is validated against that model's own
+  definition, so a mistyped argument reports as itself instead of alongside
+  the required arguments of every other model in the stage.
+- The stale-detector heuristic in the advisory warnings now looks for any
+  `*_detector` key rather than requiring a `geometry` block under it.
+- `search` exits 4 on zero hits with `--json` too; it was 0.
+- `rag.semantic_search` reuses one embedder and one Qdrant client per index
+  directory instead of building both on every call.
+- The bundled `tests/galaxy_config.yaml` fixture is valid under Pyxel 3.0.2
+  (`charge_to_volt` instead of the 2.x `charge_to_volt_conversion`).
+
+### Deprecated
+
+- The `pyxel-config-index` console script. It prints a notice and forwards
+  to `pyxel-config index`, which is the same operation.
+
+### Fixed
+
+- A model of the user's own (a `func` outside `pyxel.`, which Pyxel's schema
+  accepts through its generic ModelFunction branch) is no longer reported as
+  unknown, and `fix` no longer rewrites it into the built-in it resembles. A
+  typo under `pyxel.` is still caught.
+- `pyxel-config fix --json -w` and `fix --json -o FILE` never wrote the
+  repaired config; the JSON branch returned before the write.
+- `autofix` on a misspelled stage key moved the steps to the owning stage but
+  left the empty misspelled key behind, so the "fixed" config still failed
+  with "Additional properties are not allowed".
+- Text that cannot be read as a config (a YAML syntax error, or a document
+  that is not a mapping) made `validate_config`, `explain_config` and
+  `autofix` raise, so an MCP client got a bare tool failure with no reason.
+  Each now returns a finding in the new `yaml` category that says what is
+  wrong and on which line.
+- A few hundred bytes of nested YAML aliases load into a structure that
+  walks as tens of millions of values, and validating it kept the CLI, the
+  pre-commit hook or the MCP server busy for minutes. Text that expands past
+  its own length (or 100,000 values, whichever is larger) is now refused
+  before anything walks it (`pyxel_config_core/yaml_read.py`); no config
+  without aliases can reach that.
+- A reader that closed the pipe early (`pyxel-config validate x.yaml | head`)
+  turned a failing check into exit 0. The verdict's exit code now stands.
+- A config saved by Windows PowerShell 5.1's `>` or `Out-File` (UTF-16 with a
+  byte order mark) crashed the command line with exit 1. Files and stdin are
+  decoded by their byte order mark (UTF-8, UTF-16 or UTF-32), and other text
+  that is not UTF-8 is refused with exit 3, naming the byte that failed.
+- `fix -w` wrote back a config with nothing to repair, which dropped its
+  comments and formatting. Such a config is now left exactly as it is.
+- `fix -o -` and `tutorial --raw` wrote through the console's code page, so on
+  Windows a redirect turned characters outside it into question marks. Both
+  write UTF-8.
+- A key written twice in one mapping (a second `pipeline:` block, say) was
+  accepted, while Pyxel keeps only the last. The reader now refuses it and
+  names both lines; keys a merge (`<<`) brings in may still be overridden.
+- `fix` reported no remaining problems for any config without a root
+  `pipeline`, so it exited 0 on a config missing its pipeline. It now
+  reports the full verdict after its repairs, whatever the config holds.
+- SARIF and Code Quality reports named each file as typed on the command
+  line, so a report written on Windows carried backslashes and pointed at
+  nothing on a CI runner, and stdin was "-". Files are now named relative to
+  the working directory with forward slashes (a `file:` URI in SARIF when
+  outside it, percent-encoded in SARIF either way), and stdin is "stdin".
+- The `rag` extra allowed qdrant-client 1.7, but semantic search calls
+  `query_points` (added in 1.10), so with 1.7 to 1.9 every search fell back
+  to keywords with only "AttributeError" as the reason. The floor is now
+  1.10, an older client is named with the command that fixes it, and a
+  degraded result carries the cause's message.
+- The pre-commit hook checked every YAML file by default, so a repository's
+  own `.gitlab-ci.yml` (or a GitHub workflow) failed the commit. It now
+  leaves the common CI and tooling YAML files alone.
+- The sdist shipped the test suite without the three configs it reads, so
+  its tests stopped at collection. A MANIFEST.in now adds them and the
+  conftest; the cross-language tests skip outside the repository.
+- The bundled schema and tutorials reached the wheel only because setuptools
+  guessed at their folders, with a warning that a later release may stop
+  guessing. Both folders are now declared packages, and a test holds every
+  file in the package to the declared list.
+- The README's GitLab CI recipe ran the plain check first, so a failing
+  verdict stopped the job before the Code Quality report was written. The
+  report now comes first and is uploaded with `when: always`.
+
+## [0.5.0] - 2026-09-19
+
+### Added
+
+- The `pyxel-config` command line: `validate`, `explain`, `fix`, `models`,
+  `params`, `tutorial`, `search`, `index`, `mcp`, `doctor` and `banner`, with
+  the exit code as the verdict and stdout kept clean for pipes.
+
+[Unreleased]: https://gitlab.com/dobybaxter127/pyxel-config-lab/-/compare/core-v0.7.0...main
+[0.7.0]: https://gitlab.com/dobybaxter127/pyxel-config-lab/-/tags/core-v0.7.0
+[0.6.1]: https://gitlab.com/dobybaxter127/pyxel-config-lab/-/tags/core-v0.6.1
+[0.6.0]: https://gitlab.com/dobybaxter127/pyxel-config-lab/-/tags/core-v0.6.0
+[0.5.0]: https://pypi.org/project/pyxel-config-core/0.5.0/

pyxel_config_core-0.7.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Doby Baxter
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pyxel_config_core-0.7.0/MANIFEST.in

@@ -0,0 +1,7 @@
+# What the sdist carries beyond the package and its data: the test suite with
+# the configs it reads and the conftest that points it at this copy, so the
+# tests run from an unpacked sdist. The cross-language fixtures live at the
+# repository root; the tests that need them skip outside it.
+include conftest.py
+include CHANGELOG.md
+recursive-include tests *.py *.yaml