sydost-tracekit 0.0.1__tar.gz

sydost_tracekit-0.0.1/CHANGELOG.md

@@ -0,0 +1,30 @@
+# Changelog
+
+All notable changes of sydost-tracekit are documented here. The project follows
+[Semantic Versioning](https://semver.org).
+
+## [0.0.1] — 2026-11-11
+
+First release: reference memory traces of virtual machines and the accuracy of the
+accumulated-deficit estimate from samples (A. Ferubko, O. Kazakov, BGITU).
+
+* Recorder: the tick after a sample is the first one not behind by a whole period; an overdue
+  tick is run once, a tick is never sampled twice.
+* `synthetic`: scenarios A, B, C reproduce the prototype arrays bit for bit; held-out scenario D.
+
+* `recorder`: the guest recorder, Python standard library only, one file; drift-free
+  schedule, late and skipped tick accounting, CPU share of the recorder.
+* `collector`: host collector of libvirt memory statistics with the snapshot age
+  (optional extra `kvm`, imported lazily).
+* `Trace`, `load_trace`, `save_trace`, `TraceFormatError`: the trace format with validation;
+  `recordings_to_trace` and `demand` (balloon-aware) for recordings.
+* `clock`: host and guest clock alignment (Cristian's algorithm) and drift in ppm.
+* `integral`: deficit, its estimate from samples, `positive_part_integral` (exact, crossing
+  points), `sampling_study`.
+* `bounds`: statement 4.1 (`hold_error_bound`), the refined hold bound
+  (`hold_error_bounds_refined`), statement 4.2 (`linear_error_bound`), `estimate_constants`,
+  `plan_sampling` (the sampling rate for a target error).
+* `replay`, `tune`: controller replay with fixed polling, Pareto front and selection under
+  `P(θ) ≤ α` and `D ≤ D_max`.
+* Command line `sydost-tracekit record | integrate | bound | tune`.
+* Documentation in English and Russian, seven examples, tests with coverage.

sydost_tracekit-0.0.1/CITATION.cff

@@ -0,0 +1,36 @@
+cff-version: 1.2.0
+message: "If you use this software, please cite it as below."
+type: software
+title: "sydost-tracekit: reference memory traces of virtual machines and accuracy of accumulated-deficit estimates from samples"
+abstract: >-
+  A Python library for recording reference memory traces of virtual machines (a
+  stdlib-only guest recorder and a libvirt host collector), for estimating the
+  accumulated memory deficit from samples with explicit error bounds (hold and
+  linear reconstruction, refined hold bound, exact integral of the positive part),
+  for choosing a sampling rate for a target error, and for tuning the monitoring
+  configuration by a Pareto front.
+version: 0.0.1
+date-released: 2026-11-11
+license: MIT
+repository-code: "https://github.com/89605502155/sydost-tracekit"
+url: "https://github.com/89605502155/sydost-tracekit"
+keywords:
+  - memory trace
+  - virtualization
+  - KVM
+  - libvirt
+  - sampling
+  - accumulated deficit
+  - error bound
+  - monitoring
+authors:
+  - family-names: Ferubko
+    given-names: Andrey
+    email: ferubko1999@yandex.ru
+    orcid: "https://orcid.org/0009-0006-5625-137X"
+    affiliation: "Bryansk State Engineering-Technological University (BGITU)"
+  - family-names: Kazakov
+    given-names: Oleg
+    email: kazakov@bgitu.ru
+    orcid: "https://orcid.org/0000-0001-9665-8138"
+    affiliation: "Bryansk State Engineering-Technological University (BGITU)"

sydost_tracekit-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Andrey Ferubko, Oleg Kazakov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

sydost_tracekit-0.0.1/PKG-INFO

@@ -0,0 +1,141 @@
+Metadata-Version: 2.4
+Name: sydost-tracekit
+Version: 0.0.1
+Summary: Reference memory traces of virtual machines and accuracy of accumulated-deficit estimates from samples: stdlib guest recorder, libvirt collector, error bounds, tuning by Pareto front
+Keywords: memory-trace,virtualization,kvm,libvirt,sampling,integral,deficit,error-bound,monitoring,tuning
+Author: Andrey Ferubko, Oleg Kazakov
+Author-email: Andrey Ferubko <ferubko1999@yandex.ru>, Oleg Kazakov <kazakov@bgitu.ru>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: System :: Monitoring
+Classifier: Typing :: Typed
+Requires-Dist: numpy>=1.26
+Requires-Dist: libvirt-python>=8.0 ; extra == 'kvm'
+Requires-Python: >=3.11
+Project-URL: Homepage, https://github.com/89605502155/sydost-tracekit
+Project-URL: Documentation, https://github.com/89605502155/sydost-tracekit/tree/main/docs
+Project-URL: Repository, https://github.com/89605502155/sydost-tracekit
+Project-URL: Issues, https://github.com/89605502155/sydost-tracekit/issues
+Project-URL: Changelog, https://github.com/89605502155/sydost-tracekit/blob/main/CHANGELOG.md
+Provides-Extra: kvm
+Description-Content-Type: text/markdown
+
+# sydost-tracekit
+
+[Русская версия](README.ru.md) · [Documentation](docs/en/quickstart.md) · [Changelog](CHANGELOG.md)
+
+`sydost-tracekit` records reference memory traces of virtual machines, estimates the
+accumulated memory deficit from samples taken at a coarser step, bounds the error of
+that estimate, and tunes the monitoring configuration on the traces.
+
+The accumulated deficit `D = ∫ (d − a)⁺ dt` is an integral. Samples of the demand held
+for a step ℓ define a simple function, so the estimate converges to `D` as ℓ → 0.
+The library gives explicit error bounds: `ℓ·V(d)` for holding the samples (first order)
+and `ℓ²·(T·M/12 + K·L/3)` for linear interpolation (second order).
+
+Developed at Bryansk State Engineering-Technological University (BGITU) by Andrey Ferubko
+([ORCID](https://orcid.org/0009-0006-5625-137X)) and Oleg Kazakov
+([ORCID](https://orcid.org/0000-0001-9665-8138)).
+
+## Install
+
+```sh
+pip install sydost-tracekit           # analysis, bounds, tuning, CLI
+pip install "sydost-tracekit[kvm]"    # adds libvirt-python for the host collector
+uv add sydost-tracekit
+```
+
+The guest recorder needs only the Python standard library: copy the single file
+`src/sydost_tracekit/recorder.py` into the guest (see the deployment guide).
+
+## Quickstart
+
+```python
+from sydost_tracekit import deficit_integral, estimate_from_samples, simulate, synthetic_trace
+
+tr = synthetic_trace("A", T=600.0, dt=0.1, seed=7)  # reference trace, step 0.1 s
+run = simulate(tr, interval_s=4.0)  # allocation of a controller polled every 4 s
+D = deficit_integral(tr.d, run.alloc, tr.dt)
+est = estimate_from_samples(tr.t[::20], tr.d[:, ::20], run.alloc, tr.dt, "linear")
+print(f"D = {D:.1f} MiB*s, linear estimate from 2 s samples = {est:.1f} MiB*s")
+```
+
+## Command line
+
+```sh
+sydost-tracekit record --hz 10 --duration 600 --out trace_vm1.csv   # run the recorder here
+sydost-tracekit integrate ref.csv --alloc alloc.csv --steps 0.5,1,2 # estimate D from samples
+sydost-tracekit bound ref.csv --steps 1 --target 1e4               # bounds and sampling rate
+sydost-tracekit tune train.csv --intervals 1,2,4,8 --alpha 0.5 --dmax 10
+```
+
+## Features
+
+- **Sampling rate for a target error.** `plan_sampling` gives the method and the step
+  (rate) for which the a-priori bound does not exceed the target, from the constants of
+  a reference trace.
+- **Refined hold bound.** `hold_error_bounds_refined` counts only the variation that
+  matters for the deficit (the rises above the allocation and the falls of the held
+  value above it). It is valid and tighter than `ℓ·V(d)`; the examples show it.
+- **Exact integral of the positive part** of a piecewise linear signal via the crossing
+  points (`positive_part_integral`); a step allocation is a repeated knot.
+- **Guest recorder** in the standard library only: drift-free schedule, late and skipped
+  tick accounting, CPU share of the recorder itself.
+- **Host–guest clock alignment**: offset from request–response exchanges (Cristian's
+  algorithm, the error is at most half of the round trip) and linear drift in ppm.
+- **Trace format** with validation (uniform grid, finite values, unique names).
+- **Tuner**: sweep of configurations, Pareto front, selection under `P(θ) ≤ α` and
+  `D ≤ D_max`.
+
+## How it differs from generic tools
+
+| Need | Generic tools | sydost-tracekit |
+|---|---|---|
+| Integral of a sampled series | `scipy.integrate.trapezoid`, `pandas` resampling | the integral of the **deficit** `(d − a)⁺` with a known allocation, and the exact crossing-point formula |
+| Error of the integral | none: no bound for a sampled deficit | explicit bounds `ℓ·V(d)`, `ℓ²(T·M/12 + K·L/3)` and the refined hold bound |
+| "What sampling rate do I need?" | rule of thumb | the step for a target absolute error, from the trace |
+| Collection | Prometheus, node_exporter, collectd: sample every 15–60 s by default | a stdlib guest recorder at 10 Hz with overhead accounting, and a libvirt host collector |
+| Tuning | manual thresholds | Pareto front over monitoring cost, selection under constraints |
+
+## Documentation
+
+- English: [quickstart](docs/en/quickstart.md), [trace format](docs/en/trace-format.md),
+  [bounds and theory](docs/en/bounds.md), [recorder deployment](docs/en/recorder-deployment.md),
+  [API](docs/en/api.md), [CLI](docs/en/cli.md).
+- Русский: [README.ru.md](README.ru.md), [docs/ru](docs/ru).
+
+Runnable scripts are in [examples](examples/); they finish in a few seconds.
+
+## Development
+
+```sh
+uv sync
+uv run ruff check . && uv run ruff format --check .
+uv run mypy
+uv run pytest
+```
+
+Enable the commit-message rule once per clone: `git config core.hooksPath scripts/git-hooks`.
+Heavy experiments (minutes of recording at 100 Hz) are in [experiments](experiments/), not run by the tests.
+
+## Citation
+
+See [CITATION.cff](CITATION.cff). Authors: Andrey Ferubko (ferubko1999@yandex.ru) and
+Oleg Kazakov (kazakov@bgitu.ru), Bryansk State Engineering-Technological University (BGITU).
+
+## License
+
+MIT, see [LICENSE](LICENSE). Copyright (c) 2026 Andrey Ferubko, Oleg Kazakov.

sydost_tracekit-0.0.1/README.md

@@ -0,0 +1,106 @@
+# sydost-tracekit
+
+[Русская версия](README.ru.md) · [Documentation](docs/en/quickstart.md) · [Changelog](CHANGELOG.md)
+
+`sydost-tracekit` records reference memory traces of virtual machines, estimates the
+accumulated memory deficit from samples taken at a coarser step, bounds the error of
+that estimate, and tunes the monitoring configuration on the traces.
+
+The accumulated deficit `D = ∫ (d − a)⁺ dt` is an integral. Samples of the demand held
+for a step ℓ define a simple function, so the estimate converges to `D` as ℓ → 0.
+The library gives explicit error bounds: `ℓ·V(d)` for holding the samples (first order)
+and `ℓ²·(T·M/12 + K·L/3)` for linear interpolation (second order).
+
+Developed at Bryansk State Engineering-Technological University (BGITU) by Andrey Ferubko
+([ORCID](https://orcid.org/0009-0006-5625-137X)) and Oleg Kazakov
+([ORCID](https://orcid.org/0000-0001-9665-8138)).
+
+## Install
+
+```sh
+pip install sydost-tracekit           # analysis, bounds, tuning, CLI
+pip install "sydost-tracekit[kvm]"    # adds libvirt-python for the host collector
+uv add sydost-tracekit
+```
+
+The guest recorder needs only the Python standard library: copy the single file
+`src/sydost_tracekit/recorder.py` into the guest (see the deployment guide).
+
+## Quickstart
+
+```python
+from sydost_tracekit import deficit_integral, estimate_from_samples, simulate, synthetic_trace
+
+tr = synthetic_trace("A", T=600.0, dt=0.1, seed=7)  # reference trace, step 0.1 s
+run = simulate(tr, interval_s=4.0)  # allocation of a controller polled every 4 s
+D = deficit_integral(tr.d, run.alloc, tr.dt)
+est = estimate_from_samples(tr.t[::20], tr.d[:, ::20], run.alloc, tr.dt, "linear")
+print(f"D = {D:.1f} MiB*s, linear estimate from 2 s samples = {est:.1f} MiB*s")
+```
+
+## Command line
+
+```sh
+sydost-tracekit record --hz 10 --duration 600 --out trace_vm1.csv   # run the recorder here
+sydost-tracekit integrate ref.csv --alloc alloc.csv --steps 0.5,1,2 # estimate D from samples
+sydost-tracekit bound ref.csv --steps 1 --target 1e4               # bounds and sampling rate
+sydost-tracekit tune train.csv --intervals 1,2,4,8 --alpha 0.5 --dmax 10
+```
+
+## Features
+
+- **Sampling rate for a target error.** `plan_sampling` gives the method and the step
+  (rate) for which the a-priori bound does not exceed the target, from the constants of
+  a reference trace.
+- **Refined hold bound.** `hold_error_bounds_refined` counts only the variation that
+  matters for the deficit (the rises above the allocation and the falls of the held
+  value above it). It is valid and tighter than `ℓ·V(d)`; the examples show it.
+- **Exact integral of the positive part** of a piecewise linear signal via the crossing
+  points (`positive_part_integral`); a step allocation is a repeated knot.
+- **Guest recorder** in the standard library only: drift-free schedule, late and skipped
+  tick accounting, CPU share of the recorder itself.
+- **Host–guest clock alignment**: offset from request–response exchanges (Cristian's
+  algorithm, the error is at most half of the round trip) and linear drift in ppm.
+- **Trace format** with validation (uniform grid, finite values, unique names).
+- **Tuner**: sweep of configurations, Pareto front, selection under `P(θ) ≤ α` and
+  `D ≤ D_max`.
+
+## How it differs from generic tools
+
+| Need | Generic tools | sydost-tracekit |
+|---|---|---|
+| Integral of a sampled series | `scipy.integrate.trapezoid`, `pandas` resampling | the integral of the **deficit** `(d − a)⁺` with a known allocation, and the exact crossing-point formula |
+| Error of the integral | none: no bound for a sampled deficit | explicit bounds `ℓ·V(d)`, `ℓ²(T·M/12 + K·L/3)` and the refined hold bound |
+| "What sampling rate do I need?" | rule of thumb | the step for a target absolute error, from the trace |
+| Collection | Prometheus, node_exporter, collectd: sample every 15–60 s by default | a stdlib guest recorder at 10 Hz with overhead accounting, and a libvirt host collector |
+| Tuning | manual thresholds | Pareto front over monitoring cost, selection under constraints |
+
+## Documentation
+
+- English: [quickstart](docs/en/quickstart.md), [trace format](docs/en/trace-format.md),
+  [bounds and theory](docs/en/bounds.md), [recorder deployment](docs/en/recorder-deployment.md),
+  [API](docs/en/api.md), [CLI](docs/en/cli.md).
+- Русский: [README.ru.md](README.ru.md), [docs/ru](docs/ru).
+
+Runnable scripts are in [examples](examples/); they finish in a few seconds.
+
+## Development
+
+```sh
+uv sync
+uv run ruff check . && uv run ruff format --check .
+uv run mypy
+uv run pytest
+```
+
+Enable the commit-message rule once per clone: `git config core.hooksPath scripts/git-hooks`.
+Heavy experiments (minutes of recording at 100 Hz) are in [experiments](experiments/), not run by the tests.
+
+## Citation
+
+See [CITATION.cff](CITATION.cff). Authors: Andrey Ferubko (ferubko1999@yandex.ru) and
+Oleg Kazakov (kazakov@bgitu.ru), Bryansk State Engineering-Technological University (BGITU).
+
+## License
+
+MIT, see [LICENSE](LICENSE). Copyright (c) 2026 Andrey Ferubko, Oleg Kazakov.

sydost_tracekit-0.0.1/README.ru.md

@@ -0,0 +1,105 @@
+# sydost-tracekit
+
+[English version](README.md) · [Документация](docs/ru/quickstart.md) · [История версий](CHANGELOG.md)
+
+`sydost-tracekit` записывает эталонные трассы памяти виртуальных машин, оценивает накопленный
+дефицит памяти по отсчётам с более грубым шагом, даёт границы погрешности этой оценки и
+настраивает конфигурацию мониторинга по трассам.
+
+Накопленный дефицит `D = ∫ (d − a)⁺ dt` — интеграл. Отсчёты потребности, удержанные на шаге ℓ,
+задают простую функцию, поэтому оценка сходится к `D` при ℓ → 0. Библиотека даёт явные границы
+погрешности: `ℓ·V(d)` для удержания (первый порядок) и `ℓ²·(T·M/12 + K·L/3)` для линейной
+интерполяции (второй порядок).
+
+Разработано в Брянском государственном инженерно-технологическом университете (БГИТУ)
+Андреем Ферубко ([ORCID](https://orcid.org/0009-0006-5625-137X)) и Олегом Казаковым
+([ORCID](https://orcid.org/0000-0001-9665-8138)).
+
+## Установка
+
+```sh
+pip install sydost-tracekit           # анализ, границы, настройка, командная строка
+pip install "sydost-tracekit[kvm]"    # добавляет libvirt-python для сборщика на хосте
+uv add sydost-tracekit
+```
+
+Гостевой регистратор требует только стандартной библиотеки Python: скопируйте один файл
+`src/sydost_tracekit/recorder.py` в гостевую ВМ (см. руководство по развёртыванию).
+
+## Быстрый старт
+
+```python
+from sydost_tracekit import deficit_integral, estimate_from_samples, simulate, synthetic_trace
+
+tr = synthetic_trace("A", T=600.0, dt=0.1, seed=7)  # эталонная трасса, шаг 0,1 с
+run = simulate(tr, interval_s=4.0)  # выделение контроллера, опрашиваемого раз в 4 с
+D = deficit_integral(tr.d, run.alloc, tr.dt)
+est = estimate_from_samples(tr.t[::20], tr.d[:, ::20], run.alloc, tr.dt, "linear")
+print(f"D = {D:.1f} MiB*s, линейная оценка по отсчётам раз в 2 с = {est:.1f} MiB*s")
+```
+
+## Командная строка
+
+```sh
+sydost-tracekit record --hz 10 --duration 600 --out trace_vm1.csv   # запустить регистратор здесь
+sydost-tracekit integrate ref.csv --alloc alloc.csv --steps 0.5,1,2 # оценить D по отсчётам
+sydost-tracekit bound ref.csv --steps 1 --target 1e4               # границы и частота опроса
+sydost-tracekit tune train.csv --intervals 1,2,4,8 --alpha 0.5 --dmax 10
+```
+
+## Возможности
+
+- **Частота опроса для заданной погрешности.** `plan_sampling` даёт метод и шаг (частоту),
+  при которых априорная граница не превышает цель, по константам эталонной трассы.
+- **Уточнённая граница для удержания.** `hold_error_bounds_refined` учитывает только вариацию,
+  существенную для дефицита (подъёмы потребности над выделением и спады удерживаемого значения
+  над ним). Она верна и уже, чем `ℓ·V(d)`; примеры это показывают.
+- **Точный интеграл положительной части** кусочно-линейного сигнала по точкам пересечения
+  (`positive_part_integral`); ступенчатое выделение — повторённый узел.
+- **Гостевой регистратор** только на стандартной библиотеке: расписание без дрейфа, учёт
+  опозданий и пропущенных тиков, доля CPU самого регистратора.
+- **Совмещение часов хоста и гостя**: смещение по обменам «запрос–ответ» (алгоритм Кристиана,
+  погрешность не больше половины времени обмена) и линейный дрейф в ppm.
+- **Формат трасс** с проверкой (равномерная сетка, конечные значения, уникальные имена).
+- **Настройка**: перебор конфигураций, фронт Парето, выбор при ограничениях `P(θ) ≤ α` и
+  `D ≤ D_max`.
+
+## Чем отличается от общих инструментов
+
+| Задача | Общие инструменты | sydost-tracekit |
+|---|---|---|
+| Интеграл по отсчётам | `scipy.integrate.trapezoid`, пересэмплирование в `pandas` | интеграл **дефицита** `(d − a)⁺` при известном выделении и точная формула по точкам пересечения |
+| Погрешность интеграла | нет: границы для дискретного дефицита не дают | явные границы `ℓ·V(d)`, `ℓ²(T·M/12 + K·L/3)` и уточнённая граница для удержания |
+| «Какая частота опроса нужна?» | эмпирическое правило | шаг для заданной абсолютной погрешности по трассе |
+| Сбор | Prometheus, node_exporter, collectd: по умолчанию опрос раз в 15–60 с | регистратор в госте на 10 Гц с учётом накладных расходов и сборщик libvirt на хосте |
+| Настройка | ручные пороги | фронт Парето по стоимости мониторинга, выбор при ограничениях |
+
+## Документация
+
+- Русский: [быстрый старт](docs/ru/quickstart.md), [формат трасс](docs/ru/trace-format.md),
+  [границы и теория](docs/ru/bounds.md), [развёртывание регистратора](docs/ru/recorder-deployment.md),
+  [API](docs/ru/api.md), [командная строка](docs/ru/cli.md).
+- English: [README.md](README.md), [docs/en](docs/en).
+
+Готовые скрипты лежат в [examples](examples/); каждый выполняется за несколько секунд.
+
+## Разработка
+
+```sh
+uv sync
+uv run ruff check . && uv run ruff format --check .
+uv run mypy
+uv run pytest
+```
+
+Правило сообщений коммитов включается один раз на клон: `git config core.hooksPath scripts/git-hooks`.
+Тяжёлые эксперименты (записи минутами на 100 Гц) лежат в [experiments](experiments/) и тестами не запускаются.
+
+## Цитирование
+
+См. [CITATION.cff](CITATION.cff). Авторы: Андрей Ферубко (ferubko1999@yandex.ru) и
+Олег Казаков (kazakov@bgitu.ru), Брянский государственный инженерно-технологический университет (БГИТУ).
+
+## Лицензия
+
+MIT, см. [LICENSE](LICENSE). Copyright (c) 2026 Andrey Ferubko, Oleg Kazakov.

sydost_tracekit-0.0.1/docs/en/api.md

@@ -0,0 +1,21 @@
+# API
+
+The public names are exported from `sydost_tracekit`. Each function has a numpy-style
+docstring with runnable examples.
+
+| Group | Names | Purpose |
+|---|---|---|
+| Traces | `Trace`, `load_trace`, `save_trace`, `TraceFormatError` | trace on a uniform grid, CSV format |
+| Recordings | `read_recorder_csv`, `demand`, `recordings_to_trace` | recorder output to traces |
+| Clocks | `ClockSample`, `estimate_offset`, `guest_to_host`, `fit_drift` | host–guest alignment |
+| Reconstruction | `hold`, `linear` | samples to a function of time |
+| Integral | `deficit`, `deficit_integral`, `total_variation`, `positive_part_integral`, `estimate_from_samples`, `sampling_study` | deficit and its estimate |
+| Bounds | `hold_error_bound`, `hold_error_bounds_refined`, `linear_error_bound`, `estimate_constants`, `Constants`, `required_step`, `plan_sampling`, `Plan` | a-priori and refined bounds, sampling plan |
+| Replay | `ReserveController`, `simulate`, `RunResult` | controller replay with fixed polling |
+| Tuning | `sweep`, `pareto_front`, `select` | cost–quality front and selection |
+| Synthetic | `synthetic_trace`, `VmProfile`, `SCENARIOS` | scenarios A, B, C (identical to the prototype for the same seed); D held out |
+| Guest | `sydost_tracekit.recorder.record`, `main` | stdlib recorder (also the file copied to guests) |
+| Host | `sydost_tracekit.collector.collect`, `connect` | libvirt collector (extra `kvm`) |
+
+Units: time in seconds, memory in MiB, the accumulated deficit in MiB·s (the CLI prints GiB·s
+where the name says so).

sydost_tracekit-0.0.1/docs/en/bounds.md

@@ -0,0 +1,100 @@
+# Bounds and theory
+
+## The estimate from samples
+
+Let `a` be the allocation (known exactly from the host journal), and `d` the demand.
+The deficit is `f = (d − a)⁺` and `D = ∫ f dt`. Samples of `d` at step `ℓ`, held
+until the next sample, define a simple function `d_ℓ`. The estimate is
+
+    D_ℓ = ∫ (d_ℓ − a)⁺ dt.
+
+As `ℓ → 0`, `d_ℓ → d` almost everywhere, and by the Lebesgue dominated convergence theorem
+`D_ℓ → D` (the integrands are bounded by the maximum memory plus the reserve).
+
+## Statement 4.1: hold
+
+    |D_ℓ − D| ≤ ℓ · V(d),
+
+where `V(d)` is the total variation of the demand. It follows from `|x⁺ − y⁺| ≤ |x − y|`.
+The bound is valid and of first order in `ℓ`, but it counts variation at all times.
+
+```python
+import numpy as np
+from sydost_tracekit import hold_error_bound
+
+print(hold_error_bound(np.array([[0.0, 2.0, 1.0]]), 0.5))
+```
+
+## Refined hold bound
+
+Let `x_k` be the held value at the fine point `k`. The error of a point is nonzero only if
+the demand or the held value is above the allocation. Pointwise,
+
+    −(d_k − max(x_k, a_k))⁺  ≤  (estimate − exact at k)  ≤  (x_k − max(d_k, a_k))⁺,
+
+and summing over the fine grid with step `dt` gives
+
+    −dt Σ (d_k − max(x_k, a_k))⁺  ≤  D_ℓ − D  ≤  dt Σ (x_k − max(d_k, a_k))⁺.
+
+The lower bound counts only the rises of the demand above the allocation, the upper bound
+only the falls of the held value above it. It is never looser than `ℓ·V(d)`, and it is much
+tighter when the demand stays below the allocation.
+
+```python
+from sydost_tracekit import hold_error_bounds_refined, simulate, synthetic_trace
+
+tr = synthetic_trace("A", T=600.0, dt=0.1, seed=7)
+alloc = simulate(tr, interval_s=4.0).alloc
+lo, hi = hold_error_bounds_refined(tr.d, alloc, tr.dt, ell=4.0)
+print(f"refined: [{lo:.1f}, {hi:.1f}] MiB*s")
+```
+
+## Statement 4.2: linear interpolation
+
+If `d` is `L`-Lipschitz with `K` kinks and `|d''| ≤ M` away from the kinks, then
+
+    |D_lin − D| ≤ ℓ² · (T·M/12 + K·L/3).
+
+On a smooth step the interpolation error is at most `M·u·(ℓ − u)/2` (u is the time from the
+start of the step), and its integral is `M·ℓ³/12`. On a step with a kink the error is at most
+`2L·u·(ℓ − u)/ℓ`, and its integral `L·ℓ²/3`. Summing over the steps gives the statement.
+The standard error estimate of linear interpolation is the reference for the derivation.
+The constants are estimated from a reference trace by `estimate_constants`; for a noisy trace
+`M` and `K` are large and the bound is loose.
+
+## Exact integral of the positive part
+
+On each segment of a piecewise linear signal, `g = x − a` is linear. The integral of its
+positive part follows from the crossing point with zero, so the integral of `(x − a)⁺` is
+exact:
+
+```python
+import numpy as np
+from sydost_tracekit import positive_part_integral
+
+t = np.array([0.0, 1.0, 2.0])
+x = np.array([0.0, 4.0, 0.0])
+a = np.array([2.0, 2.0, 2.0])
+print(positive_part_integral(t, x, a))
+```
+
+## What sampling rate is needed?
+
+For an admissible absolute error `E` (MiB·s), the admissible step is the largest `ℓ` for which
+the bound of the method does not exceed `E`: `ℓ = E / V` for hold, `ℓ = sqrt(E / C)` for
+linear, with `C = Σ (T·M/12 + K·L/3)` over the machines. `plan_sampling` picks the method with
+the larger step, that is, the lower rate.
+
+```python
+from sydost_tracekit import estimate_constants, plan_sampling, synthetic_trace
+
+tr = synthetic_trace("B", T=600.0, dt=0.1, seed=7)
+plan = plan_sampling(1e4, estimate_constants(tr.d, tr.dt))
+print(plan.method, round(plan.step_s, 3), round(plan.rate_hz, 3))
+```
+
+## Scope
+
+The bounds are for the reconstruction of the demand. They assume that the allocation is
+known exactly, and that the balloon is instant. Synthetic traces are for debugging; the
+conclusions of a study must rest on traces of a real system.

sydost_tracekit-0.0.1/docs/en/cli.md

@@ -0,0 +1,44 @@
+# Command line
+
+`sydost-tracekit` has four commands. Each has `--help`.
+
+## record
+
+Runs the guest recorder on this machine. It writes the CSV and the JSON summary.
+
+```sh
+sydost-tracekit record --hz 10 --duration 600 --out trace_vm1.csv
+```
+
+## integrate
+
+Estimates the deficit from samples of a reference trace for several steps and methods.
+The allocation file has the same format and grid as the reference.
+
+```sh
+sydost-tracekit integrate ref.csv --alloc alloc.csv --steps 0.5,1,2 --methods hold,linear
+```
+
+The output gives the exact `D` and the relative error in percent for each step and method.
+
+## bound
+
+Prints the constants of the reference trace (`V`, `L`, `M`, `K`), the linear bound and the
+hold bound `ℓ·V` for the steps given, and, with `--target`, the method, step and rate that
+meet an absolute error in MiB·s.
+
+```sh
+sydost-tracekit bound ref.csv --steps 1,2 --target 10000
+```
+
+## tune
+
+Replays fixed polling intervals on the trace, prints the cost (samples per machine-hour),
+`P(θ)`, `D`, the Pareto flag, and the selected interval under `P ≤ α` and `D ≤ D_max`.
+
+```sh
+sydost-tracekit tune train.csv --intervals 1,2,4,8 --theta 64 --alpha 0.5 --dmax 10
+```
+
+`--theta` is the deficit threshold in MiB. `--dmax` is in GiB·s. Without `--dmax` there is no
+limit on `D`. If no interval satisfies the constraints, the command says so.

sydost_tracekit-0.0.1/docs/en/quickstart.md

@@ -0,0 +1,29 @@
+# Quickstart
+
+Install the package, then estimate the accumulated deficit of a trace from samples.
+
+```sh
+pip install sydost-tracekit
+```
+
+The first example takes a synthetic trace (scenario A, four machines), replays a
+controller polled every 4 seconds, and estimates the deficit from samples every 2 seconds:
+
+```python
+from sydost_tracekit import deficit_integral, estimate_from_samples, simulate, synthetic_trace
+
+tr = synthetic_trace("A", T=600.0, dt=0.1, seed=7)  # reference trace, step 0.1 s
+run = simulate(tr, interval_s=4.0)  # allocation of a controller polled every 4 s
+D = deficit_integral(tr.d, run.alloc, tr.dt)
+est = estimate_from_samples(tr.t[::20], tr.d[:, ::20], run.alloc, tr.dt, "linear")
+print(f"D = {D:.1f} MiB*s, linear estimate from 2 s samples = {est:.1f} MiB*s")
+```
+
+Next steps:
+
+- [Trace format](trace-format.md): how the demand of a machine is stored.
+- [Bounds](bounds.md): how large the error of the estimate can be, and which sampling rate is needed.
+- [Recorder deployment](recorder-deployment.md): how to run the guest recorder.
+- [CLI](cli.md) and [API](api.md).
+
+The examples in [examples](../../examples/) show each function on a short script.