pytrosna 0.1.0__tar.gz

pytrosna-0.1.0/CHANGELOG.md

@@ -0,0 +1,28 @@
+# Changelog
+
+All notable changes of pytrosna are documented here. The project follows
+[Semantic Versioning](https://semver.org).
+
+## 0.1.0 — 2026-10-09
+
+First release: a pure-Python implementation of the Trosna file format,
+version 1.0.
+
+* Reading and writing of `.trosna` files, byte-compatible with the reference
+  Rust implementation (the golden file is reproduced byte for byte).
+* All encodings of the format (plain, delta-bitpack, delta-of-delta, RLE,
+  Gorilla XOR, dictionary) with adaptive per-segment selection, and the
+  codecs none, LZ4 and Zstandard.
+* Editing: inserts, updates, deletions of points and ranges, interval
+  annotations, file and device metadata; atomic, SHA-256 hash-chained
+  commits; rollback; time travel (`as_of`) and differences between versions.
+* Crash safety: recovery of unfinished files, refusal to truncate files
+  damaged in the middle, `verify`, `recover` and `compact`.
+* High-level API (`pytrosna.open`, `read_*`, `write`, `create`, transactions)
+  with adapters for pandas, Polars and PyArrow; a NumPy-only low-level API
+  (`Writer`, `Reader`, `Batch`).
+* The `pytrosna` command-line tool (CSV import and export, inspection,
+  editing, history, maintenance).
+* Documentation in English and Russian; about 400 tests including
+  property-based, model-based, crash-simulation, corruption and
+  interoperability tests.

pytrosna-0.1.0/LICENSE-APACHE

@@ -0,0 +1,176 @@
+                              Apache License
+                        Version 2.0, January 2004
+                     http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+   "License" shall mean the terms and conditions for use, reproduction,
+   and distribution as defined by Sections 1 through 9 of this document.
+
+   "Licensor" shall mean the copyright owner or entity authorized by
+   the copyright owner that is granting the License.
+
+   "Legal Entity" shall mean the union of the acting entity and all
+   other entities that control, are controlled by, or are under common
+   control with that entity. For the purposes of this definition,
+   "control" means (i) the power, direct or indirect, to cause the
+   direction or management of such entity, whether by contract or
+   otherwise, or (ii) ownership of fifty percent (50%) or more of the
+   outstanding shares, or (iii) beneficial ownership of such entity.
+
+   "You" (or "Your") shall mean an individual or Legal Entity
+   exercising permissions granted by this License.
+
+   "Source" form shall mean the preferred form for making modifications,
+   including but not limited to software source code, documentation
+   source, and configuration files.
+
+   "Object" form shall mean any form resulting from mechanical
+   transformation or translation of a Source form, including but
+   not limited to compiled object code, generated documentation,
+   and conversions to other media types.
+
+   "Work" shall mean the work of authorship, whether in Source or
+   Object form, made available under the License, as indicated by a
+   copyright notice that is included in or attached to the work
+   (an example is provided in the Appendix below).
+
+   "Derivative Works" shall mean any work, whether in Source or Object
+   form, that is based on (or derived from) the Work and for which the
+   editorial revisions, annotations, elaborations, or other modifications
+   represent, as a whole, an original work of authorship. For the purposes
+   of this License, Derivative Works shall not include works that remain
+   separable from, or merely link (or bind by name) to the interfaces of,
+   the Work and Derivative Works thereof.
+
+   "Contribution" shall mean any work of authorship, including
+   the original version of the Work and any modifications or additions
+   to that Work or Derivative Works thereof, that is intentionally
+   submitted to Licensor for inclusion in the Work by the copyright owner
+   or by an individual or Legal Entity authorized to submit on behalf of
+   the copyright owner. For the purposes of this definition, "submitted"
+   means any form of electronic, verbal, or written communication sent
+   to the Licensor or its representatives, including but not limited to
+   communication on electronic mailing lists, source code control systems,
+   and issue tracking systems that are managed by, or on behalf of, the
+   Licensor for the purpose of discussing and improving the Work, but
+   excluding communication that is conspicuously marked or otherwise
+   designated in writing by the copyright owner as "Not a Contribution."
+
+   "Contributor" shall mean Licensor and any individual or Legal Entity
+   on behalf of whom a Contribution has been received by Licensor and
+   subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   copyright license to reproduce, prepare Derivative Works of,
+   publicly display, publicly perform, sublicense, and distribute the
+   Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   (except as stated in this section) patent license to make, have made,
+   use, offer to sell, sell, import, and otherwise transfer the Work,
+   where such license applies only to those patent claims licensable
+   by such Contributor that are necessarily infringed by their
+   Contribution(s) alone or by combination of their Contribution(s)
+   with the Work to which such Contribution(s) was submitted. If You
+   institute patent litigation against any entity (including a
+   cross-claim or counterclaim in a lawsuit) alleging that the Work
+   or a Contribution incorporated within the Work constitutes direct
+   or contributory patent infringement, then any patent licenses
+   granted to You under this License for that Work shall terminate
+   as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+   Work or Derivative Works thereof in any medium, with or without
+   modifications, and in Source or Object form, provided that You
+   meet the following conditions:
+
+   (a) You must give any other recipients of the Work or
+       Derivative Works a copy of this License; and
+
+   (b) You must cause any modified files to carry prominent notices
+       stating that You changed the files; and
+
+   (c) You must retain, in the Source form of any Derivative Works
+       that You distribute, all copyright, patent, trademark, and
+       attribution notices from the Source form of the Work,
+       excluding those notices that do not pertain to any part of
+       the Derivative Works; and
+
+   (d) If the Work includes a "NOTICE" text file as part of its
+       distribution, then any Derivative Works that You distribute must
+       include a readable copy of the attribution notices contained
+       within such NOTICE file, excluding those notices that do not
+       pertain to any part of the Derivative Works, in at least one
+       of the following places: within a NOTICE text file distributed
+       as part of the Derivative Works; within the Source form or
+       documentation, if provided along with the Derivative Works; or,
+       within a display generated by the Derivative Works, if and
+       wherever such third-party notices normally appear. The contents
+       of the NOTICE file are for informational purposes only and
+       do not modify the License. You may add Your own attribution
+       notices within Derivative Works that You distribute, alongside
+       or as an addendum to the NOTICE text from the Work, provided
+       that such additional attribution notices cannot be construed
+       as modifying the License.
+
+   You may add Your own copyright statement to Your modifications and
+   may provide additional or different license terms and conditions
+   for use, reproduction, or distribution of Your modifications, or
+   for any such Derivative Works as a whole, provided Your use,
+   reproduction, and distribution of the Work otherwise complies with
+   the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+   Notwithstanding the above, nothing herein shall supersede or modify
+   the terms of any separate license agreement you may have executed
+   with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+   names, trademarks, service marks, or product names of the Licensor,
+   except as required for reasonable and customary use in describing the
+   origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+   agreed to in writing, Licensor provides the Work (and each
+   Contributor provides its Contributions) on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+   implied, including, without limitation, any warranties or conditions
+   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+   PARTICULAR PURPOSE. You are solely responsible for determining the
+   appropriateness of using or redistributing the Work and assume any
+   risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+   whether in tort (including negligence), contract, or otherwise,
+   unless required by applicable law (such as deliberate and grossly
+   negligent acts) or agreed to in writing, shall any Contributor be
+   liable to You for damages, including any direct, indirect, special,
+   incidental, or consequential damages of any character arising as a
+   result of this License or out of the use or inability to use the
+   Work (including but not limited to damages for loss of goodwill,
+   work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses), even if such Contributor
+   has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+   the Work or Derivative Works thereof, You may choose to offer,
+   and charge a fee for, acceptance of support, warranty, indemnity,
+   or other liability obligations and/or rights consistent with this
+   License. However, in accepting such obligations, You may act only
+   on Your own behalf and on Your sole responsibility, not on behalf
+   of any other Contributor, and only if You agree to indemnify,
+   defend, and hold each Contributor harmless for any liability
+   incurred by, or claims asserted against, such Contributor by reason
+   of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS

pytrosna-0.1.0/LICENSE-MIT

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Andrey Ferubko
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pytrosna-0.1.0/PKG-INFO

@@ -0,0 +1,267 @@
+Metadata-Version: 2.4
+Name: pytrosna
+Version: 0.1.0
+Summary: Read, write and edit Trosna time-series files (.trosna) in pure Python: columnar, compressed, versioned, with pandas, Polars and PyArrow adapters
+Keywords: time-series,file-format,trosna,columnar,compression,gorilla,versioning,pandas,polars,arrow
+Author: Andrey Ferubko
+Author-email: Andrey Ferubko <ferubko1999@yandex.ru>
+License-Expression: MIT OR Apache-2.0
+License-File: LICENSE-APACHE
+License-File: LICENSE-MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: System :: Archiving :: Compression
+Classifier: Typing :: Typed
+Requires-Dist: crc32c>=2.4
+Requires-Dist: lz4>=4.3
+Requires-Dist: numpy>=1.26
+Requires-Dist: tzdata ; sys_platform == 'win32'
+Requires-Dist: zstandard>=0.22 ; python_full_version < '3.14'
+Requires-Dist: pyarrow>=15 ; extra == 'all'
+Requires-Dist: pandas>=2.1 ; extra == 'all'
+Requires-Dist: polars>=1.0 ; extra == 'all'
+Requires-Dist: pyarrow>=15 ; extra == 'arrow'
+Requires-Dist: pandas>=2.1 ; extra == 'pandas'
+Requires-Dist: polars>=1.0 ; extra == 'polars'
+Requires-Python: >=3.11
+Project-URL: Homepage, https://github.com/89605502155/pytrosna
+Project-URL: Documentation, https://github.com/89605502155/pytrosna/tree/main/docs
+Project-URL: Repository, https://github.com/89605502155/pytrosna
+Project-URL: Issues, https://github.com/89605502155/pytrosna/issues
+Project-URL: Format specification, https://github.com/89605502155/trosna-file/blob/main/docs/SPEC.md
+Project-URL: Rust implementation, https://github.com/89605502155/trosna-file
+Provides-Extra: all
+Provides-Extra: arrow
+Provides-Extra: pandas
+Provides-Extra: polars
+Description-Content-Type: text/markdown
+
+# pytrosna
+
+**Read, write and edit Trosna time-series files (`.trosna`) in pure Python.**
+
+[Русская версия](https://github.com/89605502155/pytrosna/blob/main/README.ru.md) ·
+[User guide](https://github.com/89605502155/pytrosna/blob/main/docs/guide.md) ·
+[API reference](https://github.com/89605502155/pytrosna/blob/main/docs/api.md) ·
+[Examples](https://github.com/89605502155/pytrosna/tree/main/examples) ·
+[Format specification](https://github.com/89605502155/trosna-file/blob/main/docs/SPEC.md)
+
+Trosna is a file format for time series. A `.trosna` file is as easy to
+create and read as CSV, but stores data the way time-series databases do:
+column by column, with encodings designed for time stamps and measurements,
+and with a verifiable history of every change.
+
+`pytrosna` is a pure-Python implementation of format version 1.0, fully
+compatible with the reference Rust implementation
+([trosna-file](https://github.com/89605502155/trosna-file)): files written by
+either one are read, edited and verified by the other. The writer uses the
+same algorithms, down to the choice of encodings, and reproduces the golden
+file of the Rust implementation byte for byte. (Where a segment is
+compressed with Zstandard, the two libraries may produce different but
+equivalent compressed bytes.)
+
+* **Compact.** Every column segment is encoded with the smallest of
+  delta-of-delta, TS_2DIFF-style bit packing, Gorilla XOR, run-length and
+  dictionary encodings, then compressed with Zstandard or LZ4. A sensor
+  series takes **1.5 bytes per point** (CSV: 16.9, Parquet + zstd: 2.6).
+* **Editable, with history.** Insert, change and delete points anywhere on
+  the time axis and label time intervals. Every change is an atomic commit;
+  every earlier version stays readable (`as_of=`). Commits form a SHA-256 hash
+  chain, so the history is tamper-evident.
+* **Crash-safe.** The file is an append-only log of checksummed frames. After
+  a crash it reads as of its last commit; the next writer removes the
+  unfinished tail.
+* **Several devices per file**, each a table with its own columns, time unit
+  (s, ms, us, ns) and time zone.
+* **Works with the Python data stack.** Read into and write from pandas,
+  Polars and PyArrow, or use NumPy-backed batches without any of them.
+* **Pure Python.** No compiler needed: the dependencies are NumPy, `lz4`,
+  `crc32c` and, before Python 3.14, `zstandard`.
+
+## Installation
+
+```sh
+pip install pytrosna                 # NumPy-backed core
+pip install "pytrosna[pandas]"       # + pandas
+pip install "pytrosna[polars]"       # + Polars
+pip install "pytrosna[arrow]"        # + PyArrow
+pip install "pytrosna[all]"          # everything
+
+uv add "pytrosna[all]"               # with uv
+```
+
+Python 3.11 or newer is required.
+
+## Quick start
+
+```python
+import pandas as pd
+import pytrosna
+
+df = pd.DataFrame(
+    {
+        "time": pd.date_range("2026-10-08 10:00", periods=5, freq="10s", tz="Europe/Moscow"),
+        "temperature": [21.5, 21.6, 21.7, 21.6, 21.8],
+        "humidity": [40, 41, 41, 42, 42],
+    }
+)
+
+pytrosna.write("room.trosna", df, device="room1")  # create the file (one commit)
+
+pytrosna.read_pandas("room.trosna")  # everything, as pandas
+# a time range and a column, as Polars:
+pytrosna.read_polars("room.trosna", start="2026-10-08 10:00:10", columns=["temperature"])
+pytrosna.read_arrow("room.trosna")  # a pyarrow.Table
+pytrosna.read("room.trosna")  # a NumPy-backed pytrosna.Batch
+```
+
+Times can be `datetime`, `pandas.Timestamp`, `numpy.datetime64`, ISO 8601 text
+or raw integers. Text without a UTC offset is a wall-clock time of the
+device's time zone.
+
+## Editing, annotations and time travel
+
+```python
+f = pytrosna.open("room.trosna")
+
+with f.edit(message="sensor check", author="me") as tx:  # one atomic commit
+    tx.update("room1", "2026-10-08 10:00:20", temperature=21.65)
+    tx.delete("room1", "2026-10-08 10:00:10")
+    tx.insert("room1", "2026-10-08 10:01:00", temperature=22.0)
+    tx.annotate("room1", "2026-10-08 10:00:15", "2026-10-08 10:00:25", "door open")
+# if the block raises, nothing is written
+
+f.read_pandas()  # the current data
+f.read_pandas(as_of=1)  # the data as first imported
+f.diff(1).points  # what changed since commit 1
+f.annotations()  # labelled intervals
+f.commits()  # the hash-chained history
+f.verify().ok  # checksums, hash chain, decoding of all data
+f.compact("small.trosna")  # a copy without history that links to it
+```
+
+Appending is just as simple:
+
+```python
+more_rows = df.assign(time=df["time"] + pd.Timedelta("1h"))
+pytrosna.write("room.trosna", more_rows, device="room1", mode="a")
+```
+
+## Low-level API
+
+`Writer` and `Reader` work with raw integer time stamps and NumPy arrays and
+need no other libraries:
+
+```python
+import numpy as np
+from pytrosna import DeviceSchema, Reader, Writer
+
+with Writer.create("vms.trosna") as w:  # closing commits
+    w.create_device(DeviceSchema.build("vm01", "ms", {"cpu": "float64", "ram_mb": "int64"}))
+    w.write(
+        "vm01",
+        {
+            "time": np.arange(0, 10_000, 1000),
+            "cpu": np.random.rand(10),
+            "ram_mb": np.full(10, 2048),
+        },
+    )
+    w.commit(message="first measurements")
+    w.update("vm01", 3000, cpu=0.99)
+    w.delete_range("vm01", 7000, 9000)
+
+with Reader("vms.trosna") as r:
+    batch = r.query("vm01").columns(["cpu"]).time_range(0, 5000).collect()
+    batch.time, batch["cpu"].values  # NumPy arrays
+    r.query("vm01").as_of(1).count()  # 10 points in version 1
+```
+
+## Command line
+
+The package installs a `pytrosna` command:
+
+```sh
+pytrosna convert room.csv room.trosna --device room1    # CSV → Trosna (types are detected)
+pytrosna info room.trosna                               # devices, history, storage per column
+pytrosna cat room.trosna --from "2026-10-08 10:00" --format table
+pytrosna update room.trosna --time "2026-10-08 10:00:20" temperature=21.65 -m "sensor check"
+pytrosna annotate room.trosna --start "2026-10-08 10:00:15" --end "2026-10-08 10:00:25" --label "door open"
+pytrosna log room.trosna
+pytrosna diff room.trosna --from 1
+pytrosna verify room.trosna
+pytrosna convert room.trosna back.csv                   # Trosna → CSV
+```
+
+`pytrosna --help` lists all commands: `info`, `cat`, `convert`, `insert`,
+`update`, `delete`, `annotate`, `unannotate`, `annotations`, `log`, `diff`,
+`verify`, `recover`, `compact`.
+
+## Size and speed
+
+One million readings of a slowly varying sensor (values rounded to 0.01, one
+per second), with Zstandard:
+
+| Format                      | Bytes per point |
+|-----------------------------|-----------------|
+| **Trosna (pytrosna)**       | **1.49**        |
+| Arrow IPC (Feather), zstd   | 2.18            |
+| Parquet, zstd               | 2.56            |
+| CSV, gzip                   | 3.14            |
+| CSV                         | 16.90           |
+
+pytrosna writes about 0.8 and reads about 0.6 million points per second on a
+laptop: its encoders and decoders are vectorized with NumPy where the format
+allows it. When speed matters more than having no compiled dependencies, the
+Rust implementation is faster; the files are the same.
+
+## Compatibility and testing
+
+* Format version 1.0, as specified in
+  [SPEC.md](https://github.com/89605502155/trosna-file/blob/main/docs/SPEC.md).
+* The writer reproduces the golden file of the Rust implementation **byte for
+  byte**, and the test suite (about 400 tests, 98 % line coverage) includes
+  interoperability tests that run the reference `trosna` tool on files written
+  by pytrosna and vice versa. The code examples of this README and of the
+  guides are executed by the tests as well.
+* Property-based tests compare every encoder with a reference implementation
+  written directly from the specification; a model-based test checks the
+  editing semantics at every version; files are truncated at every byte
+  offset and corrupted with random bit flips to check recovery and error
+  handling.
+
+## Development
+
+```sh
+uv sync                       # create the environment with all extras and tools
+uv run pytest                 # the test suite (TROSNA_CLI=/path/to/trosna enables interop tests)
+uv run ruff check . && uv run ruff format --check .
+uv run mypy
+uv build                      # sdist and wheel in dist/
+```
+
+Publishing to PyPI is described in
+[docs/publishing.md](https://github.com/89605502155/pytrosna/blob/main/docs/publishing.md).
+
+## Citation
+
+If you use Trosna in research, please cite the review that motivated it:
+
+> Ferubko A. O., Kazakov O. D. Review of data structures and file formats for
+> storage and transfer of time series with regard to their support in the
+> Python and Rust ecosystems. 2026.
+
+## License
+
+Licensed under either of [Apache License, Version 2.0](LICENSE-APACHE) or
+[MIT license](LICENSE-MIT) at your option.