visin 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- visin-0.1.0/.gitignore +13 -0
- visin-0.1.0/CHANGELOG.md +51 -0
- visin-0.1.0/CONTRIBUTING.md +100 -0
- visin-0.1.0/LICENSE +21 -0
- visin-0.1.0/Makefile +57 -0
- visin-0.1.0/PKG-INFO +147 -0
- visin-0.1.0/README.md +86 -0
- visin-0.1.0/SECURITY.md +22 -0
- visin-0.1.0/docs/architecture.md +91 -0
- visin-0.1.0/docs/changelog.md +1 -0
- visin-0.1.0/docs/contributing.md +1 -0
- visin-0.1.0/docs/getting-started.md +79 -0
- visin-0.1.0/docs/guides/cli.md +56 -0
- visin-0.1.0/docs/guides/configuration.md +62 -0
- visin-0.1.0/docs/guides/frameworks.md +54 -0
- visin-0.1.0/docs/guides/offline.md +68 -0
- visin-0.1.0/docs/guides/reading.md +45 -0
- visin-0.1.0/docs/guides/reporting.md +169 -0
- visin-0.1.0/docs/index.md +43 -0
- visin-0.1.0/docs/reference/api.md +5 -0
- visin-0.1.0/docs/reference/errors.md +9 -0
- visin-0.1.0/docs/reference/index.md +18 -0
- visin-0.1.0/docs/reference/integrations.md +11 -0
- visin-0.1.0/docs/reference/offline.md +9 -0
- visin-0.1.0/docs/reference/run.md +7 -0
- visin-0.1.0/docs/reference/system.md +7 -0
- visin-0.1.0/mkdocs.yml +81 -0
- visin-0.1.0/pyproject.toml +137 -0
- visin-0.1.0/scripts/release.py +254 -0
- visin-0.1.0/src/visin/__init__.py +42 -0
- visin-0.1.0/src/visin/__main__.py +5 -0
- visin-0.1.0/src/visin/_internal/__init__.py +9 -0
- visin-0.1.0/src/visin/_internal/config.py +112 -0
- visin-0.1.0/src/visin/_internal/console.py +31 -0
- visin-0.1.0/src/visin/_internal/reports.py +187 -0
- visin-0.1.0/src/visin/_internal/sender.py +143 -0
- visin-0.1.0/src/visin/_internal/serialize.py +85 -0
- visin-0.1.0/src/visin/_internal/spool.py +306 -0
- visin-0.1.0/src/visin/_internal/transport.py +303 -0
- visin-0.1.0/src/visin/_version.py +1 -0
- visin-0.1.0/src/visin/api.py +206 -0
- visin-0.1.0/src/visin/cli.py +289 -0
- visin-0.1.0/src/visin/errors.py +40 -0
- visin-0.1.0/src/visin/integrations/__init__.py +5 -0
- visin-0.1.0/src/visin/integrations/_metrics.py +49 -0
- visin-0.1.0/src/visin/integrations/keras.py +91 -0
- visin-0.1.0/src/visin/integrations/lightning.py +122 -0
- visin-0.1.0/src/visin/offline.py +56 -0
- visin-0.1.0/src/visin/py.typed +0 -0
- visin-0.1.0/src/visin/run.py +1094 -0
- visin-0.1.0/src/visin/system.py +293 -0
- visin-0.1.0/tests/README.md +76 -0
- visin-0.1.0/tests/conftest.py +60 -0
- visin-0.1.0/tests/contract/test_openapi.py +175 -0
- visin-0.1.0/tests/fakes.py +109 -0
- visin-0.1.0/tests/integrations/test_callbacks.py +238 -0
- visin-0.1.0/tests/internal/test_config.py +69 -0
- visin-0.1.0/tests/internal/test_reports.py +88 -0
- visin-0.1.0/tests/internal/test_sender.py +82 -0
- visin-0.1.0/tests/internal/test_serialize.py +106 -0
- visin-0.1.0/tests/internal/test_spool.py +228 -0
- visin-0.1.0/tests/internal/test_transport.py +211 -0
- visin-0.1.0/tests/test_api.py +114 -0
- visin-0.1.0/tests/test_cli.py +128 -0
- visin-0.1.0/tests/test_offline.py +82 -0
- visin-0.1.0/tests/test_run.py +966 -0
- visin-0.1.0/tests/test_system.py +132 -0
- visin-0.1.0/tests/tooling/test_release.py +113 -0
visin-0.1.0/.gitignore
ADDED
visin-0.1.0/CHANGELOG.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this package are recorded here. The format follows
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
|
|
5
|
+
[Semantic Versioning](https://semver.org/spec/v2.0.0.html). Until 1.0, a minor version may change
|
|
6
|
+
the public API; a patch version never does.
|
|
7
|
+
|
|
8
|
+
The release workflow writes each version's entry from the conventional commits since the previous
|
|
9
|
+
one, together with anything written by hand under Unreleased.
|
|
10
|
+
|
|
11
|
+
## [Unreleased]
|
|
12
|
+
|
|
13
|
+
## [0.1.0] - 2026-09-26
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
|
|
17
|
+
- `visin.init()`, the one call most scripts need: registers a run, or attaches to the one
|
|
18
|
+
`VISIN_TRAINING_UUID` names, and reports nothing when no server is configured.
|
|
19
|
+
- `Run.log_epoch`, `log_test_results`, `log_benchmark`, `log_config`, `upload_visualization` and
|
|
20
|
+
`update`, sent from a background thread in the order they were made.
|
|
21
|
+
- `train=` and `val=` on `log_epoch`, for the two curves Visin's charts read.
|
|
22
|
+
- Safe retries: epochs and test results carry their own UUIDs, and a POST without one is repeated
|
|
23
|
+
only when it provably never reached the server.
|
|
24
|
+
- Store and forward: reports wait on disk while Visin is unreachable and are sent when it answers.
|
|
25
|
+
`VISIN_MODE=offline` keeps every report on disk for a machine with no route to Visin.
|
|
26
|
+
- `visin sync` and `visin.sync()`, which send kept reports, resume where they stopped, and never
|
|
27
|
+
send a benchmark or config twice.
|
|
28
|
+
- `visin check`, which tests the connection and the token, and with `--write` creates and deletes
|
|
29
|
+
a test run.
|
|
30
|
+
- `visin runs` and `visin.Api`, for reading projects, runs, epochs, test results and benchmarks,
|
|
31
|
+
with `Api.epochs_frame` for a pandas DataFrame.
|
|
32
|
+
- `system_info()` for benchmarks and `system_metrics()` for the run's System tab, including GPUs
|
|
33
|
+
through PyTorch or `nvidia-smi`.
|
|
34
|
+
- Keras and PyTorch Lightning callbacks.
|
|
35
|
+
- Conversion of NumPy and PyTorch scalars; NaN and infinity are sent as gaps.
|
|
36
|
+
- Only rank zero of a distributed job reports.
|
|
37
|
+
- A crashed or terminated run still sends its epochs and is marked failed.
|
|
38
|
+
- `Run.attach(..., mark_status=False)`, for a test or benchmark script that reports into a
|
|
39
|
+
training's run without changing its status.
|
|
40
|
+
- `epoch_uuid=` on `log_test_results`, `log_benchmark` and `upload_visualization`, to name an epoch
|
|
41
|
+
directly, such as the one a checkpoint's file name records.
|
|
42
|
+
- `Run.resumed`, and a resumed run is marked running again while it trains.
|
|
43
|
+
- A warning when an epoch logged again is kept at its first recorded values.
|
|
44
|
+
- `visin.enable_console_logging()`.
|
|
45
|
+
|
|
46
|
+
### Fixed
|
|
47
|
+
|
|
48
|
+
- github release flow
|
|
49
|
+
|
|
50
|
+
[Unreleased]: https://github.com/visin-platform/visin-py/compare/v0.1.0...HEAD
|
|
51
|
+
[0.1.0]: https://github.com/visin-platform/visin-py/releases/tag/v0.1.0
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Thanks for helping. This page covers setting up, the checks a change has to pass, and how a release
|
|
4
|
+
happens.
|
|
5
|
+
|
|
6
|
+
## Set up
|
|
7
|
+
|
|
8
|
+
```sh
|
|
9
|
+
git clone https://github.com/visin-platform/visin-py
|
|
10
|
+
cd visin-py
|
|
11
|
+
make install # .venv with every tool, plus the pre-commit hooks
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
`make help` lists every target. Each one runs the same command CI runs.
|
|
15
|
+
|
|
16
|
+
## Before you open a pull request
|
|
17
|
+
|
|
18
|
+
```sh
|
|
19
|
+
make check
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
This runs, in order:
|
|
23
|
+
|
|
24
|
+
| Target | What it checks |
|
|
25
|
+
| --- | --- |
|
|
26
|
+
| `make lint` | `ruff check` and `ruff format --check`. `make format` fixes most findings. |
|
|
27
|
+
| `make typecheck` | `mypy --strict` over the package. |
|
|
28
|
+
| `make coverage` | The whole test suite, failing if coverage drops below the floor in `pyproject.toml`. |
|
|
29
|
+
| `make build` | Builds the sdist and wheel, and checks their metadata with `twine check --strict`. |
|
|
30
|
+
| `make docs` | Builds the documentation site with `mkdocs build --strict`, so a broken link or reference fails. |
|
|
31
|
+
|
|
32
|
+
`make contract` checks every request the package sends against Visin's API spec. It reads the spec
|
|
33
|
+
from `VISIN_OPENAPI`, or from a Visin checkout next to this one at `../visin`. CI runs it against
|
|
34
|
+
Visin's main branch.
|
|
35
|
+
|
|
36
|
+
The coverage floor sits just below current coverage. Raise it when you add tests, and never lower it
|
|
37
|
+
to make a change pass.
|
|
38
|
+
|
|
39
|
+
## Where things are
|
|
40
|
+
|
|
41
|
+
The package's layout, and the reasoning behind its main decisions, are in
|
|
42
|
+
[How it is built](https://visin-platform.github.io/visin-py/architecture/). The tests mirror the package, and
|
|
43
|
+
[tests/README.md](https://github.com/visin-platform/visin-py/blob/main/tests/README.md) says what each file covers.
|
|
44
|
+
|
|
45
|
+
## Writing a change
|
|
46
|
+
|
|
47
|
+
- **Keep the loop safe.** Nothing in `Run` may raise into a training loop unless `strict=True` is
|
|
48
|
+
set. Report problems through `_handle`.
|
|
49
|
+
- **New request fields** must exist in Visin's API. The contract test fails on a field the server
|
|
50
|
+
would strip.
|
|
51
|
+
- **Support Python 3.9.** CI tests 3.9 to 3.14. `from __future__ import annotations` makes the
|
|
52
|
+
newer annotation syntax safe. Runtime expressions such as `isinstance(x, int | str)` are not.
|
|
53
|
+
- **No new runtime dependencies** without a strong reason. The package goes into any training image,
|
|
54
|
+
and `requests` is its only dependency. Optional features belong behind an extra.
|
|
55
|
+
- **Test what the change promises**, through the public API where you can, using the fakes in
|
|
56
|
+
`tests/fakes.py`. No test may touch the network or `~/.visin`; `tests/conftest.py` enforces this.
|
|
57
|
+
- **Documentation:** a user-facing change updates the relevant page under `docs/`, and docstrings
|
|
58
|
+
feed the API reference.
|
|
59
|
+
|
|
60
|
+
## Commit messages
|
|
61
|
+
|
|
62
|
+
Commits follow [Conventional Commits](https://www.conventionalcommits.org/), because the release
|
|
63
|
+
workflow builds the changelog and picks the version from them:
|
|
64
|
+
|
|
65
|
+
| Prefix | Changelog section | Version bump |
|
|
66
|
+
| --- | --- | --- |
|
|
67
|
+
| `feat:` | Added | minor |
|
|
68
|
+
| `fix:` | Fixed | patch |
|
|
69
|
+
| `perf:` | Changed | patch |
|
|
70
|
+
| `feat!:`, or a `BREAKING CHANGE:` footer | Breaking changes | major (minor before 1.0) |
|
|
71
|
+
| `docs:`, `test:`, `ci:`, `chore:`, `refactor:`, `build:`, `style:` | not listed | patch, when released |
|
|
72
|
+
|
|
73
|
+
For anything a user should know that a commit subject cannot say, add a line under `## [Unreleased]`
|
|
74
|
+
in `CHANGELOG.md`. The release keeps it.
|
|
75
|
+
|
|
76
|
+
## Releasing
|
|
77
|
+
|
|
78
|
+
A maintainer runs the **Release** workflow from the Actions tab, and that one run does everything:
|
|
79
|
+
|
|
80
|
+
1. runs the full CI suite;
|
|
81
|
+
2. works out the version from the commits since the last tag, or uses the one given
|
|
82
|
+
(`patch`, `minor`, `major` or `1.4.0`);
|
|
83
|
+
3. writes `src/visin/_version.py` and the `CHANGELOG.md` entry, commits `chore(release): X.Y.Z`
|
|
84
|
+
and tags `vX.Y.Z`;
|
|
85
|
+
4. builds, and publishes to PyPI through trusted publishing, so no API token is stored anywhere;
|
|
86
|
+
5. creates the GitHub release with the changelog entry and the built files attached;
|
|
87
|
+
6. publishes the documentation.
|
|
88
|
+
|
|
89
|
+
Choose `testpypi` as the target to rehearse this. That builds and publishes to TestPyPI, and commits,
|
|
90
|
+
tags and releases nothing.
|
|
91
|
+
|
|
92
|
+
`python scripts/release.py --dry-run` shows locally what the next release would be.
|
|
93
|
+
|
|
94
|
+
### One-time setup
|
|
95
|
+
|
|
96
|
+
- On PyPI and TestPyPI, add a trusted publisher for `visin-platform/visin-py`, workflow `release.yml`,
|
|
97
|
+
with the environments `pypi` and `testpypi`.
|
|
98
|
+
- In the repository settings, create those two environments. Adding required reviewers to `pypi`
|
|
99
|
+
makes every release wait for approval.
|
|
100
|
+
- Under **Pages**, set the source to **GitHub Actions**.
|
visin-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Toomas Tahves
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
visin-0.1.0/Makefile
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Every check CI runs, runnable here. `make check` is the one to run before a PR.
|
|
2
|
+
#
|
|
3
|
+
# Uses .venv when it exists (make install creates it), the PATH otherwise, which
|
|
4
|
+
# is how CI runs the same targets.
|
|
5
|
+
|
|
6
|
+
VENV ?= .venv
|
|
7
|
+
BIN := $(if $(wildcard $(VENV)/bin/python),$(VENV)/bin/,)
|
|
8
|
+
PYTHON ?= python3
|
|
9
|
+
|
|
10
|
+
.DEFAULT_GOAL := help
|
|
11
|
+
.PHONY: help install lint format typecheck test coverage contract build docs docs-serve check clean
|
|
12
|
+
|
|
13
|
+
help: ## List the targets
|
|
14
|
+
@grep -E '^[a-z-]+:.*## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*## "}; {printf " make %-11s %s\n", $$1, $$2}'
|
|
15
|
+
|
|
16
|
+
install: ## Create .venv with every dev tool, and install the git hooks
|
|
17
|
+
$(PYTHON) -m venv $(VENV)
|
|
18
|
+
$(VENV)/bin/pip install --upgrade pip
|
|
19
|
+
$(VENV)/bin/pip install -e '.[dev]'
|
|
20
|
+
$(VENV)/bin/pre-commit install
|
|
21
|
+
|
|
22
|
+
lint: ## Lint and check formatting (ruff)
|
|
23
|
+
$(BIN)ruff check .
|
|
24
|
+
$(BIN)ruff format --check .
|
|
25
|
+
|
|
26
|
+
format: ## Fix formatting and the lint findings ruff can fix
|
|
27
|
+
$(BIN)ruff format .
|
|
28
|
+
$(BIN)ruff check --fix .
|
|
29
|
+
|
|
30
|
+
typecheck: ## Type-check the package (mypy, strict)
|
|
31
|
+
$(BIN)mypy
|
|
32
|
+
|
|
33
|
+
test: ## Run the tests
|
|
34
|
+
$(BIN)pytest
|
|
35
|
+
|
|
36
|
+
coverage: ## Run the tests with coverage; fails under the floor in pyproject.toml
|
|
37
|
+
$(BIN)pytest --cov --cov-report=term-missing --cov-report=xml
|
|
38
|
+
|
|
39
|
+
contract: ## Check every request against Visin's API spec (needs VISIN_OPENAPI or ../visin)
|
|
40
|
+
VISIN_REQUIRE_CONTRACT=1 $(BIN)pytest tests/contract
|
|
41
|
+
|
|
42
|
+
build: ## Build the sdist and wheel, and check their metadata
|
|
43
|
+
rm -rf dist
|
|
44
|
+
$(BIN)python -m build
|
|
45
|
+
$(BIN)twine check --strict dist/*
|
|
46
|
+
|
|
47
|
+
docs: ## Build the documentation site, failing on any warning
|
|
48
|
+
$(BIN)mkdocs build --strict
|
|
49
|
+
|
|
50
|
+
docs-serve: ## Serve the documentation with live reload
|
|
51
|
+
$(BIN)mkdocs serve
|
|
52
|
+
|
|
53
|
+
check: lint typecheck coverage build docs ## Everything CI checks
|
|
54
|
+
@echo "all checks passed"
|
|
55
|
+
|
|
56
|
+
clean: ## Remove build, test and docs output
|
|
57
|
+
rm -rf dist site .coverage coverage.xml .pytest_cache .mypy_cache .ruff_cache
|
visin-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: visin
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Report training runs, epochs, test results, benchmarks and prediction frames to Visin, and read them back.
|
|
5
|
+
Project-URL: Homepage, https://github.com/visin-platform/visin-py
|
|
6
|
+
Project-URL: Source, https://github.com/visin-platform/visin-py
|
|
7
|
+
Project-URL: Issues, https://github.com/visin-platform/visin-py/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/visin-platform/visin-py/blob/main/CHANGELOG.md
|
|
9
|
+
Project-URL: Documentation, https://github.com/visin-platform/visin-py#readme
|
|
10
|
+
Author: Toomas Tahves
|
|
11
|
+
License-Expression: MIT
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Keywords: computer-vision,experiment-tracking,machine-learning,mlops,visin
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Intended Audience :: Science/Research
|
|
17
|
+
Classifier: Operating System :: OS Independent
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
24
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
25
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
26
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
27
|
+
Classifier: Topic :: Scientific/Engineering :: Image Recognition
|
|
28
|
+
Classifier: Typing :: Typed
|
|
29
|
+
Requires-Python: >=3.9
|
|
30
|
+
Requires-Dist: requests>=2.25
|
|
31
|
+
Provides-Extra: dev
|
|
32
|
+
Requires-Dist: build>=1.0; extra == 'dev'
|
|
33
|
+
Requires-Dist: coverage>=7.10; extra == 'dev'
|
|
34
|
+
Requires-Dist: jsonschema>=4.18; extra == 'dev'
|
|
35
|
+
Requires-Dist: mkdocs-material>=9.5; extra == 'dev'
|
|
36
|
+
Requires-Dist: mkdocs<2,>=1.6; extra == 'dev'
|
|
37
|
+
Requires-Dist: mkdocstrings[python]>=0.26; extra == 'dev'
|
|
38
|
+
Requires-Dist: mypy>=1.10; extra == 'dev'
|
|
39
|
+
Requires-Dist: pandas>=1.3; extra == 'dev'
|
|
40
|
+
Requires-Dist: pre-commit>=3.5; extra == 'dev'
|
|
41
|
+
Requires-Dist: pytest-cov>=5; extra == 'dev'
|
|
42
|
+
Requires-Dist: pytest>=7; extra == 'dev'
|
|
43
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
44
|
+
Requires-Dist: twine>=6.1; extra == 'dev'
|
|
45
|
+
Requires-Dist: types-requests; extra == 'dev'
|
|
46
|
+
Provides-Extra: docs
|
|
47
|
+
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
|
|
48
|
+
Requires-Dist: mkdocs<2,>=1.6; extra == 'docs'
|
|
49
|
+
Requires-Dist: mkdocstrings[python]>=0.26; extra == 'docs'
|
|
50
|
+
Provides-Extra: pandas
|
|
51
|
+
Requires-Dist: pandas>=1.3; extra == 'pandas'
|
|
52
|
+
Provides-Extra: system
|
|
53
|
+
Requires-Dist: psutil>=5.8; extra == 'system'
|
|
54
|
+
Provides-Extra: test
|
|
55
|
+
Requires-Dist: coverage>=7.10; extra == 'test'
|
|
56
|
+
Requires-Dist: jsonschema>=4.18; extra == 'test'
|
|
57
|
+
Requires-Dist: pandas>=1.3; extra == 'test'
|
|
58
|
+
Requires-Dist: pytest-cov>=5; extra == 'test'
|
|
59
|
+
Requires-Dist: pytest>=7; extra == 'test'
|
|
60
|
+
Description-Content-Type: text/markdown
|
|
61
|
+
|
|
62
|
+
# visin
|
|
63
|
+
|
|
64
|
+
[](https://github.com/visin-platform/visin-py/actions/workflows/ci.yml)
|
|
65
|
+
[](https://pypi.org/project/visin/)
|
|
66
|
+
[](https://pypi.org/project/visin/)
|
|
67
|
+
[](https://github.com/visin-platform/visin-py/blob/main/LICENSE)
|
|
68
|
+
|
|
69
|
+
Send training runs to [Visin](https://github.com/visin-platform/visin): epochs, test results,
|
|
70
|
+
benchmarks, prediction frames and configs. Read them back for analysis.
|
|
71
|
+
|
|
72
|
+
```sh
|
|
73
|
+
pip install visin
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
```python
|
|
77
|
+
import visin
|
|
78
|
+
|
|
79
|
+
with visin.init("unet baseline", project="road-seg") as run:
|
|
80
|
+
for epoch in range(1, 21):
|
|
81
|
+
train_loss, val_loss, val_iou = train_one_epoch()
|
|
82
|
+
run.log_epoch(epoch, train={"loss": train_loss}, val={"loss": val_loss, "mean_iou": val_iou})
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Point it at your Visin with two environment variables. Without them, the same script runs and
|
|
86
|
+
reports nothing.
|
|
87
|
+
|
|
88
|
+
```sh
|
|
89
|
+
export VISIN_URL=https://vision-api.example.com
|
|
90
|
+
export VISIN_TOKEN=… # a project token, from the project's Settings → API Tokens
|
|
91
|
+
visin check --write # confirm this machine can report
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## What it does for you
|
|
95
|
+
|
|
96
|
+
- **Never stops your training.** Reports go out from a background thread, and failures are logged,
|
|
97
|
+
never raised.
|
|
98
|
+
- **Retries safely.** Epochs and test results carry their own ids, so a retried request is never
|
|
99
|
+
stored twice.
|
|
100
|
+
- **Survives the network.** Reports wait on disk while Visin is unreachable, and are sent when it
|
|
101
|
+
answers. With `VISIN_MODE=offline`, a compute node with no route to Visin keeps everything for
|
|
102
|
+
`visin sync`.
|
|
103
|
+
- **Keeps crashed runs.** A run that crashes, or that the scheduler terminates, still sends its
|
|
104
|
+
epochs and is marked failed.
|
|
105
|
+
- **Takes metrics as they are.** NumPy and PyTorch scalars are converted, and NaN becomes a gap in
|
|
106
|
+
the chart.
|
|
107
|
+
- **Reports once in distributed jobs.** Only rank zero reports.
|
|
108
|
+
|
|
109
|
+
## More than epochs
|
|
110
|
+
|
|
111
|
+
```python
|
|
112
|
+
run.log_test_results(12, {"day": {"car": {"iou": 0.74}}, "night": {"car": {"iou": 0.61}}})
|
|
113
|
+
run.log_benchmark({"device": "cuda", "batch_size": 1, "fps": 119.0}) # machine details filled in
|
|
114
|
+
run.upload_visualization(12, "renders/overlay_0001.png", kind="overlay")
|
|
115
|
+
run.log_config(args) # argparse, dataclass, pydantic, Hydra…
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## Frameworks
|
|
119
|
+
|
|
120
|
+
```python
|
|
121
|
+
from visin.integrations.lightning import VisinCallback # or visin.integrations.keras
|
|
122
|
+
|
|
123
|
+
trainer = L.Trainer(callbacks=[VisinCallback(name="segformer b2", project="road-seg")])
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Reading runs back
|
|
127
|
+
|
|
128
|
+
```python
|
|
129
|
+
api = visin.Api()
|
|
130
|
+
for run in api.trainings(project="road-seg", status="completed"):
|
|
131
|
+
frame = api.epochs_frame(run) # pip install 'visin[pandas]'
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
## Documentation
|
|
135
|
+
|
|
136
|
+
**[visin-platform.github.io/visin-py](https://visin-platform.github.io/visin-py/)** has the
|
|
137
|
+
getting-started guide, guides to offline use and each framework, and the API reference.
|
|
138
|
+
|
|
139
|
+
## Contributing
|
|
140
|
+
|
|
141
|
+
See [CONTRIBUTING.md](https://github.com/visin-platform/visin-py/blob/main/CONTRIBUTING.md). In
|
|
142
|
+
short: `make install`, then `make check` before a pull request. Changes are listed in the
|
|
143
|
+
[changelog](https://github.com/visin-platform/visin-py/blob/main/CHANGELOG.md).
|
|
144
|
+
|
|
145
|
+
## License
|
|
146
|
+
|
|
147
|
+
MIT
|
visin-0.1.0/README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# visin
|
|
2
|
+
|
|
3
|
+
[](https://github.com/visin-platform/visin-py/actions/workflows/ci.yml)
|
|
4
|
+
[](https://pypi.org/project/visin/)
|
|
5
|
+
[](https://pypi.org/project/visin/)
|
|
6
|
+
[](https://github.com/visin-platform/visin-py/blob/main/LICENSE)
|
|
7
|
+
|
|
8
|
+
Send training runs to [Visin](https://github.com/visin-platform/visin): epochs, test results,
|
|
9
|
+
benchmarks, prediction frames and configs. Read them back for analysis.
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
pip install visin
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
```python
|
|
16
|
+
import visin
|
|
17
|
+
|
|
18
|
+
with visin.init("unet baseline", project="road-seg") as run:
|
|
19
|
+
for epoch in range(1, 21):
|
|
20
|
+
train_loss, val_loss, val_iou = train_one_epoch()
|
|
21
|
+
run.log_epoch(epoch, train={"loss": train_loss}, val={"loss": val_loss, "mean_iou": val_iou})
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Point it at your Visin with two environment variables. Without them, the same script runs and
|
|
25
|
+
reports nothing.
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
export VISIN_URL=https://vision-api.example.com
|
|
29
|
+
export VISIN_TOKEN=… # a project token, from the project's Settings → API Tokens
|
|
30
|
+
visin check --write # confirm this machine can report
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## What it does for you
|
|
34
|
+
|
|
35
|
+
- **Never stops your training.** Reports go out from a background thread, and failures are logged,
|
|
36
|
+
never raised.
|
|
37
|
+
- **Retries safely.** Epochs and test results carry their own ids, so a retried request is never
|
|
38
|
+
stored twice.
|
|
39
|
+
- **Survives the network.** Reports wait on disk while Visin is unreachable, and are sent when it
|
|
40
|
+
answers. With `VISIN_MODE=offline`, a compute node with no route to Visin keeps everything for
|
|
41
|
+
`visin sync`.
|
|
42
|
+
- **Keeps crashed runs.** A run that crashes, or that the scheduler terminates, still sends its
|
|
43
|
+
epochs and is marked failed.
|
|
44
|
+
- **Takes metrics as they are.** NumPy and PyTorch scalars are converted, and NaN becomes a gap in
|
|
45
|
+
the chart.
|
|
46
|
+
- **Reports once in distributed jobs.** Only rank zero reports.
|
|
47
|
+
|
|
48
|
+
## More than epochs
|
|
49
|
+
|
|
50
|
+
```python
|
|
51
|
+
run.log_test_results(12, {"day": {"car": {"iou": 0.74}}, "night": {"car": {"iou": 0.61}}})
|
|
52
|
+
run.log_benchmark({"device": "cuda", "batch_size": 1, "fps": 119.0}) # machine details filled in
|
|
53
|
+
run.upload_visualization(12, "renders/overlay_0001.png", kind="overlay")
|
|
54
|
+
run.log_config(args) # argparse, dataclass, pydantic, Hydra…
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Frameworks
|
|
58
|
+
|
|
59
|
+
```python
|
|
60
|
+
from visin.integrations.lightning import VisinCallback # or visin.integrations.keras
|
|
61
|
+
|
|
62
|
+
trainer = L.Trainer(callbacks=[VisinCallback(name="segformer b2", project="road-seg")])
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Reading runs back
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
api = visin.Api()
|
|
69
|
+
for run in api.trainings(project="road-seg", status="completed"):
|
|
70
|
+
frame = api.epochs_frame(run) # pip install 'visin[pandas]'
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
## Documentation
|
|
74
|
+
|
|
75
|
+
**[visin-platform.github.io/visin-py](https://visin-platform.github.io/visin-py/)** has the
|
|
76
|
+
getting-started guide, guides to offline use and each framework, and the API reference.
|
|
77
|
+
|
|
78
|
+
## Contributing
|
|
79
|
+
|
|
80
|
+
See [CONTRIBUTING.md](https://github.com/visin-platform/visin-py/blob/main/CONTRIBUTING.md). In
|
|
81
|
+
short: `make install`, then `make check` before a pull request. Changes are listed in the
|
|
82
|
+
[changelog](https://github.com/visin-platform/visin-py/blob/main/CHANGELOG.md).
|
|
83
|
+
|
|
84
|
+
## License
|
|
85
|
+
|
|
86
|
+
MIT
|
visin-0.1.0/SECURITY.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
## Reporting a vulnerability
|
|
4
|
+
|
|
5
|
+
Please don't open a public issue. Report it privately through GitHub's
|
|
6
|
+
[security advisories](https://github.com/visin-platform/visin-py/security/advisories/new) for this
|
|
7
|
+
repository. Include what you found, how to reproduce it, and which version you tested.
|
|
8
|
+
|
|
9
|
+
You will get a reply within a week. A fix is released as a patch version, and the advisory is
|
|
10
|
+
published with it.
|
|
11
|
+
|
|
12
|
+
## Supported versions
|
|
13
|
+
|
|
14
|
+
Fixes go into the latest release. Upgrading is the fix for older versions.
|
|
15
|
+
|
|
16
|
+
## What this package does with your credentials
|
|
17
|
+
|
|
18
|
+
- The token is sent only as an `Authorization: Bearer` header, and only to the configured `VISIN_URL`.
|
|
19
|
+
- File uploads go to a signed URL from a separate HTTP session, which never carries the token.
|
|
20
|
+
- The token is never written to disk. Reports kept for `visin sync` hold request bodies only, and
|
|
21
|
+
`sync` sends them with the credentials configured when it runs.
|
|
22
|
+
- TLS verification is on unless `VISIN_VERIFY_SSL=0` is set.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# How it is built
|
|
2
|
+
|
|
3
|
+
This page is for people working on the package. It covers where each thing lives and why it works
|
|
4
|
+
the way it does.
|
|
5
|
+
|
|
6
|
+
## The files
|
|
7
|
+
|
|
8
|
+
```text
|
|
9
|
+
src/visin/
|
|
10
|
+
├── __init__.py the public API: everything a user imports comes from here
|
|
11
|
+
├── run.py Run and init: everything a training script writes
|
|
12
|
+
├── api.py Api: reading runs back
|
|
13
|
+
├── offline.py sync() and pending(): sending reports kept on disk
|
|
14
|
+
├── system.py system_info() and system_metrics()
|
|
15
|
+
├── errors.py VisinError and its subclasses
|
|
16
|
+
├── cli.py, __main__.py the `visin` command
|
|
17
|
+
├── integrations/
|
|
18
|
+
│ ├── keras.py Keras callback
|
|
19
|
+
│ ├── lightning.py PyTorch Lightning callback
|
|
20
|
+
│ └── _metrics.py sorting flat metric names into train/val/test
|
|
21
|
+
├── _internal/ machinery; not public, may change in any release
|
|
22
|
+
│ ├── config.py settings from the environment and arguments
|
|
23
|
+
│ ├── transport.py HTTP: retries, response unwrapping, signed uploads
|
|
24
|
+
│ ├── serialize.py NumPy/tensors/NaN into JSON Visin accepts
|
|
25
|
+
│ ├── reports.py what each kind of report is, and how it is delivered
|
|
26
|
+
│ ├── sender.py the background thread reports are sent from
|
|
27
|
+
│ └── spool.py reports on disk: the file format and syncing it
|
|
28
|
+
└── _version.py the version, and the only place it is written
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The rule of thumb: a module at the top level is something a user touches, and `_internal` is how it
|
|
32
|
+
works.
|
|
33
|
+
|
|
34
|
+
## One report, from call to server
|
|
35
|
+
|
|
36
|
+
```text
|
|
37
|
+
run.log_epoch(...) run.py: validate, convert to JSON, build an op
|
|
38
|
+
│ {"op": "epoch", "body": {...}}
|
|
39
|
+
▼
|
|
40
|
+
Sender queue ──► background thread sender.py: one thread, so order is kept
|
|
41
|
+
│
|
|
42
|
+
▼
|
|
43
|
+
deliver(client, op) reports.py: POST /epochs/upload; a 409 counts as delivered
|
|
44
|
+
│
|
|
45
|
+
├─ delivered ──────────────► done
|
|
46
|
+
├─ refused (4xx, 500) ─────► logged and counted; retrying would not help
|
|
47
|
+
└─ unreachable (5xx/down) ─► spool.py: appended to ~/.visin/runs/<uuid>.jsonl,
|
|
48
|
+
sent by the run once Visin answers, or by `visin sync`
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Every write is first an **op**: a small JSON dict naming its kind and carrying its request body.
|
|
52
|
+
Because an op is data, the same op can be sent at once, written to disk, and sent from disk a week
|
|
53
|
+
later by `visin sync`, and all three go through the one `deliver` function. There is a single code
|
|
54
|
+
path for sending.
|
|
55
|
+
|
|
56
|
+
## Decisions worth knowing
|
|
57
|
+
|
|
58
|
+
**Nothing raises into the training loop.** A metrics backend is not worth a training job. Failures
|
|
59
|
+
are logged and counted, and `finish` summarises them. `strict=True` exists for tests.
|
|
60
|
+
|
|
61
|
+
**Retries depend on the request, not on its method.** `transport.py` retries on its own terms
|
|
62
|
+
instead of leaving it to urllib3, because a POST is only safe to repeat if it carries its own id.
|
|
63
|
+
Runs, epochs and test results do: epoch UUIDs are `uuid5(run, epoch)`, and a repeat is answered 409.
|
|
64
|
+
Benchmarks and configs do not, so they are retried only when the request provably never reached
|
|
65
|
+
the server: the connection failed, or the rate limiter or auth refused it before any handler ran.
|
|
66
|
+
500 is never retried, as Visin documents.
|
|
67
|
+
|
|
68
|
+
**One sender thread.** Writes depend on each other: a run is created before its epochs, and an
|
|
69
|
+
epoch before its test results. Keeping them in order costs less than recovering when they arrive
|
|
70
|
+
out of order.
|
|
71
|
+
|
|
72
|
+
**The run is created synchronously.** `init` returns only once the server has the run, or once it
|
|
73
|
+
is known to be unreachable and the creation has been kept on disk. So an epoch can never arrive
|
|
74
|
+
before its run.
|
|
75
|
+
|
|
76
|
+
**Files are copied at the call.** Scripts render each epoch's frames under the same names, so the
|
|
77
|
+
original may hold the next epoch's image by the time a queued upload runs.
|
|
78
|
+
|
|
79
|
+
**SIGTERM becomes SystemExit**, but only if nobody else handles it. Otherwise a scheduler's
|
|
80
|
+
termination skips `atexit`, and the run loses its queued epochs and stays "running" forever.
|
|
81
|
+
|
|
82
|
+
**No default server.** Unset means disabled. A hard-coded address once sent a copied script's runs
|
|
83
|
+
to somebody else's Visin.
|
|
84
|
+
|
|
85
|
+
## The API contract
|
|
86
|
+
|
|
87
|
+
`tests/contract/test_openapi.py` runs every public entry point against a fake server and checks each
|
|
88
|
+
request against Visin's published OpenAPI spec. A request that fails its schema, hits a path that no
|
|
89
|
+
longer exists, or carries a field the server would silently strip, fails the test. CI runs it
|
|
90
|
+
against Visin's main branch weekly, so an API change that would break this package shows up without
|
|
91
|
+
anyone having to remember it.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
--8<-- "CHANGELOG.md"
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
--8<-- "CONTRIBUTING.md"
|