unitarrow 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (5) hide show
  1. unitarrow-0.0.1/.gitignore +21 -0
  2. unitarrow-0.0.1/PKG-INFO +142 -0
  3. unitarrow-0.0.1/README.md +133 -0
  4. unitarrow-0.0.1/pyproject.toml +17 -0
  5. unitarrow-0.0.1/src/unitarrow/__init__.py +6 -0
unitarrow-0.0.1/.gitignore ADDED
@@ -0,0 +1,21 @@
1
+ # Generated by Cargo
2
+ # will have compiled files and executables
3
+ debug
4
+ target
5
+
6
+ # These are backup files generated by rustfmt
7
+ **/*.rs.bk
8
+
9
+ # MSVC Windows builds of rustc generate these, which store debugging information
10
+ *.pdb
11
+
12
+ # Generated by cargo mutants
13
+ # Contains mutation testing data
14
+ **/mutants.out*/
15
+
16
+ # RustRover
17
+ # JetBrains specific template is maintained in a separate JetBrains.gitignore that can
18
+ # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
19
+ # and can be added to the global gitignore or merged into this file. For a more nuclear
20
+ # option (not recommended) you can uncomment the following to ignore the entire idea folder.
21
+ #.idea/
unitarrow-0.0.1/PKG-INFO ADDED
@@ -0,0 +1,142 @@
1
+ Metadata-Version: 2.4
2
+ Name: unitarrow
3
+ Version: 0.0.1
4
+ Summary: Physical units and quantity semantics for Apache Arrow columns (name reservation; spec in development)
5
+ Project-URL: Repository, https://github.com/unitarrow/unitarrow
6
+ License-Expression: BSD-3-Clause
7
+ Requires-Python: >=3.9
8
+ Description-Content-Type: text/markdown
9
+
10
+ # UnitArrow
11
+
12
+ **Physical units and quantity semantics for Apache Arrow columns.**
13
+ Column-level, cross-language, and wire-native: a unit annotates a whole
14
+ column as Arrow extension metadata, survives IPC / Flight / Parquet, and
15
+ converts as a metadata-level operation — never a per-element object.
16
+
17
+ > **Status: pre-release.** This repository reserves the `unitarrow`
18
+ > name while the specification is finalized and the first implementation
19
+ > lands. The spec draft, roadmap, and reference implementation will be
20
+ > published here. UnitArrow is a community extension-type specification
21
+ > in the spirit of GeoArrow; it is not affiliated with or endorsed by
22
+ > the Apache Arrow project or the ASF.
23
+
24
+ ## Why
25
+
26
+ Every existing approach to units dies at a boundary. In-memory unit
27
+ libraries (pint, astropy, unyt) get the algebra right and lose
28
+ everything at serialization — their units are properties of runtime
29
+ objects. File-format conventions (netCDF/CF, HDF5 attributes) survive
30
+ storage but carry *dead* metadata: no checking, no propagation, no
31
+ algebra. Compile-time unit systems are safest and can't cross a wire at
32
+ all.
33
+
34
+ UnitArrow puts units where all of those layers already meet: the Arrow
35
+ interchange layer. A tagged column keeps its unit across processes,
36
+ languages, and the network; a consumer that has never heard of
37
+ UnitArrow sees a plain float column with human-readable metadata beside
38
+ it. Adoption never needs to be coordinated: producers can tag before
39
+ their consumers understand units, consumers can upgrade before their
40
+ producers do, and each side gains value on its own schedule.
41
+
42
+ ### Where existing tools stop
43
+
44
+ | Approach | What it gets right | Where it stops |
45
+ |---|---|---|
46
+ | pint / astropy.units / unyt / Unitful.jl | Correct unit algebra, array-level units, mature registries | Units are runtime-object properties: they do not survive serialization. As of pint-pandas 0.8 / pandas 3.0 / pyarrow 25, `pa.Table.from_pandas` on a pint column doesn't even degrade — it raises `ArrowTypeError`. Adoption is all-or-nothing: everything becomes a Quantity, and pandas operations that don't silently fall back to object dtype fail outright. |
47
+ | netCDF/CF, HDF5 attributes, FITS `BUNIT` | Units survive storage; CF's standard-name vocabulary is decades of accumulated wisdom | The metadata is *dead*: no algebra, no checking, no propagation through compute — and each convention is welded to one container format. |
48
+ | F# units of measure, mp-units, uom | Compile-time safety, zero runtime cost | Locked inside one language's type checker; cannot cross a wire, an FFI boundary, or into dynamic languages at all. |
49
+ | SQL engines / dataframe libraries | Where the computation actually happens | No unit concept whatsoever; column metadata is stripped by the first query. |
50
+
51
+ None of this is a criticism of the algebra in these tools — pint's is
52
+ excellent, and CF's vocabulary is a direct input to UnitArrow's. Their
53
+ shared limitation is *placement*: each lives in one layer and dies at
54
+ that layer's edge. UnitArrow's job is the layer none of them can reach
55
+ — units as a property of the *data interchange* itself — and adapters
56
+ to and from pint are a natural part of the roadmap, so in-process pint
57
+ users gain wire survival rather than losing anything.
58
+
59
+ ## Adopt gradually — the whole design bends around this
60
+
61
+ UnitArrow is *gradually typed*, the way TypeScript is: untagged columns
62
+ are always legal, checking is a dial rather than a wall, and each step
63
+ is independently useful. You can stop at any step and keep everything
64
+ it gave you.
65
+
66
+ ### Step 1 — Tag, ship, read (zero-cost drop-in)
67
+
68
+ Convert a pandas or polars dataframe to Arrow, attach units, and ship
69
+ it. Read a UnitArrow table back into pandas with units intact.
70
+ Conversion is one scalar rescale driven by column metadata. No code
71
+ changes anywhere else; untouched pipelines pass tagged tables through
72
+ bit-identical.
73
+
74
+ ```python
75
+ # API sketch — subject to change before 1.0
76
+ import unitarrow as ua
77
+
78
+ table = ua.tag(df, {"power": "MW", "gas": "GBtu"}) # pandas / polars / pyarrow in
79
+ table = ua.convert(table, {"gas": "MWh"}) # metadata-level rescale
80
+ df2 = ua.to_pandas(table) # units survive the round trip
81
+ ```
82
+
83
+ The same table read in a browser via arrow-js exposes its units from
84
+ field metadata — client-side unit toggling with no server round trip.
85
+
86
+ ### Step 2 — Checked compute (IR + WebAssembly)
87
+
88
+ Opt columns into checking and computation becomes dimensionally sound
89
+ *before* it runs: multiplying composes units, adding requires
90
+ commensurability, and summing an instantaneous MW column into "energy"
91
+ is an error — the only path from MW rows to MWh is an explicit,
92
+ period-aware `integrate`. A small typed expression IR lowers to your
93
+ engine (arrow-rs, polars, DuckDB) and to WebAssembly, so the same
94
+ checked operations — convert, integrate, differentiate, aggregate —
95
+ run in the browser over arrow-js buffers.
96
+
97
+ ### Step 3 — End-to-end validated data products
98
+
99
+ The final step of the journey: units verified from UnitArrow sources
100
+ all the way to UnitArrow published outputs. Tables finalize with
101
+ `publish()` — checked in strict mode, pinned to a registry version, and
102
+ sealed so consumers can verify who published a table and that it hasn't
103
+ changed. Full derivation chains (including external tools like LP
104
+ solvers) become attestable and reproducible. This trust layer is
105
+ specified separately (the Provenance Companion) and is entirely
106
+ optional — Steps 1 and 2 never require it.
107
+
108
+ ## Design principles
109
+
110
+ - **Zero-cost drop-in.** Tagging is O(columns) and touches no data
111
+ bytes. Untagged tables are silent everywhere.
112
+ - **Graceful degradation.** Unaware consumers read the storage type plus
113
+ legible metadata. Nothing is ever gated on adoption.
114
+ - **Semantics as data.** Units, dimensions, display forms, and quantity
115
+ vocabularies live in a versioned registry, federated by namespace —
116
+ domains extend it without permission.
117
+ - **One implementation.** A single Rust core (`unitarrow-core`) does all
118
+ parsing, dimension algebra, and conversion, exposed to Python
119
+ (`unitarrow-py`, via the Arrow PyCapsule interface — no pyarrow
120
+ dependency), TypeScript, and WASM (`unitarrow-wasm`). A conformance
121
+ suite of golden files keeps every binding identical.
122
+
123
+ ## Packages
124
+
125
+ | Package | Registry | Status |
126
+ |---|---|---|
127
+ | `unitarrow` / `unitarrow-core` / `unitarrow-py` / `unitarrow-wasm` | crates.io | reserved |
128
+ | `unitarrow` | PyPI | reserved |
129
+ | `unitarrow` | npm | reserved |
130
+
131
+ ## Roadmap
132
+
133
+ Specification first, then `unitarrow-core`, then the Python and
134
+ browser codecs (Step 1), then checked semantics and the IR (Step 2),
135
+ then the Provenance Companion (Step 3). The full spec and milestone
136
+ roadmap will be published in this repository.
137
+
138
+ ---
139
+
140
+ *UnitArrow began in energy-systems research — where BTU, MWh, and
141
+ per-unit quantities collide daily — but nothing in it is
142
+ energy-specific.*
unitarrow-0.0.1/README.md ADDED
@@ -0,0 +1,133 @@
1
+ # UnitArrow
2
+
3
+ **Physical units and quantity semantics for Apache Arrow columns.**
4
+ Column-level, cross-language, and wire-native: a unit annotates a whole
5
+ column as Arrow extension metadata, survives IPC / Flight / Parquet, and
6
+ converts as a metadata-level operation — never a per-element object.
7
+
8
+ > **Status: pre-release.** This repository reserves the `unitarrow`
9
+ > name while the specification is finalized and the first implementation
10
+ > lands. The spec draft, roadmap, and reference implementation will be
11
+ > published here. UnitArrow is a community extension-type specification
12
+ > in the spirit of GeoArrow; it is not affiliated with or endorsed by
13
+ > the Apache Arrow project or the ASF.
14
+
15
+ ## Why
16
+
17
+ Every existing approach to units dies at a boundary. In-memory unit
18
+ libraries (pint, astropy, unyt) get the algebra right and lose
19
+ everything at serialization — their units are properties of runtime
20
+ objects. File-format conventions (netCDF/CF, HDF5 attributes) survive
21
+ storage but carry *dead* metadata: no checking, no propagation, no
22
+ algebra. Compile-time unit systems are safest and can't cross a wire at
23
+ all.
24
+
25
+ UnitArrow puts units where all of those layers already meet: the Arrow
26
+ interchange layer. A tagged column keeps its unit across processes,
27
+ languages, and the network; a consumer that has never heard of
28
+ UnitArrow sees a plain float column with human-readable metadata beside
29
+ it. Adoption never needs to be coordinated: producers can tag before
30
+ their consumers understand units, consumers can upgrade before their
31
+ producers do, and each side gains value on its own schedule.
32
+
33
+ ### Where existing tools stop
34
+
35
+ | Approach | What it gets right | Where it stops |
36
+ |---|---|---|
37
+ | pint / astropy.units / unyt / Unitful.jl | Correct unit algebra, array-level units, mature registries | Units are runtime-object properties: they do not survive serialization. As of pint-pandas 0.8 / pandas 3.0 / pyarrow 25, `pa.Table.from_pandas` on a pint column doesn't even degrade — it raises `ArrowTypeError`. Adoption is all-or-nothing: everything becomes a Quantity, and pandas operations that don't silently fall back to object dtype fail outright. |
38
+ | netCDF/CF, HDF5 attributes, FITS `BUNIT` | Units survive storage; CF's standard-name vocabulary is decades of accumulated wisdom | The metadata is *dead*: no algebra, no checking, no propagation through compute — and each convention is welded to one container format. |
39
+ | F# units of measure, mp-units, uom | Compile-time safety, zero runtime cost | Locked inside one language's type checker; cannot cross a wire, an FFI boundary, or into dynamic languages at all. |
40
+ | SQL engines / dataframe libraries | Where the computation actually happens | No unit concept whatsoever; column metadata is stripped by the first query. |
41
+
42
+ None of this is a criticism of the algebra in these tools — pint's is
43
+ excellent, and CF's vocabulary is a direct input to UnitArrow's. Their
44
+ shared limitation is *placement*: each lives in one layer and dies at
45
+ that layer's edge. UnitArrow's job is the layer none of them can reach
46
+ — units as a property of the *data interchange* itself — and adapters
47
+ to and from pint are a natural part of the roadmap, so in-process pint
48
+ users gain wire survival rather than losing anything.
49
+
50
+ ## Adopt gradually — the whole design bends around this
51
+
52
+ UnitArrow is *gradually typed*, the way TypeScript is: untagged columns
53
+ are always legal, checking is a dial rather than a wall, and each step
54
+ is independently useful. You can stop at any step and keep everything
55
+ it gave you.
56
+
57
+ ### Step 1 — Tag, ship, read (zero-cost drop-in)
58
+
59
+ Convert a pandas or polars dataframe to Arrow, attach units, and ship
60
+ it. Read a UnitArrow table back into pandas with units intact.
61
+ Conversion is one scalar rescale driven by column metadata. No code
62
+ changes anywhere else; untouched pipelines pass tagged tables through
63
+ bit-identical.
64
+
65
+ ```python
66
+ # API sketch — subject to change before 1.0
67
+ import unitarrow as ua
68
+
69
+ table = ua.tag(df, {"power": "MW", "gas": "GBtu"}) # pandas / polars / pyarrow in
70
+ table = ua.convert(table, {"gas": "MWh"}) # metadata-level rescale
71
+ df2 = ua.to_pandas(table) # units survive the round trip
72
+ ```
73
+
74
+ The same table read in a browser via arrow-js exposes its units from
75
+ field metadata — client-side unit toggling with no server round trip.
76
+
77
+ ### Step 2 — Checked compute (IR + WebAssembly)
78
+
79
+ Opt columns into checking and computation becomes dimensionally sound
80
+ *before* it runs: multiplying composes units, adding requires
81
+ commensurability, and summing an instantaneous MW column into "energy"
82
+ is an error — the only path from MW rows to MWh is an explicit,
83
+ period-aware `integrate`. A small typed expression IR lowers to your
84
+ engine (arrow-rs, polars, DuckDB) and to WebAssembly, so the same
85
+ checked operations — convert, integrate, differentiate, aggregate —
86
+ run in the browser over arrow-js buffers.
87
+
88
+ ### Step 3 — End-to-end validated data products
89
+
90
+ The final step of the journey: units verified from UnitArrow sources
91
+ all the way to UnitArrow published outputs. Tables finalize with
92
+ `publish()` — checked in strict mode, pinned to a registry version, and
93
+ sealed so consumers can verify who published a table and that it hasn't
94
+ changed. Full derivation chains (including external tools like LP
95
+ solvers) become attestable and reproducible. This trust layer is
96
+ specified separately (the Provenance Companion) and is entirely
97
+ optional — Steps 1 and 2 never require it.
98
+
99
+ ## Design principles
100
+
101
+ - **Zero-cost drop-in.** Tagging is O(columns) and touches no data
102
+ bytes. Untagged tables are silent everywhere.
103
+ - **Graceful degradation.** Unaware consumers read the storage type plus
104
+ legible metadata. Nothing is ever gated on adoption.
105
+ - **Semantics as data.** Units, dimensions, display forms, and quantity
106
+ vocabularies live in a versioned registry, federated by namespace —
107
+ domains extend it without permission.
108
+ - **One implementation.** A single Rust core (`unitarrow-core`) does all
109
+ parsing, dimension algebra, and conversion, exposed to Python
110
+ (`unitarrow-py`, via the Arrow PyCapsule interface — no pyarrow
111
+ dependency), TypeScript, and WASM (`unitarrow-wasm`). A conformance
112
+ suite of golden files keeps every binding identical.
113
+
114
+ ## Packages
115
+
116
+ | Package | Registry | Status |
117
+ |---|---|---|
118
+ | `unitarrow` / `unitarrow-core` / `unitarrow-py` / `unitarrow-wasm` | crates.io | reserved |
119
+ | `unitarrow` | PyPI | reserved |
120
+ | `unitarrow` | npm | reserved |
121
+
122
+ ## Roadmap
123
+
124
+ Specification first, then `unitarrow-core`, then the Python and
125
+ browser codecs (Step 1), then checked semantics and the IR (Step 2),
126
+ then the Provenance Companion (Step 3). The full spec and milestone
127
+ roadmap will be published in this repository.
128
+
129
+ ---
130
+
131
+ *UnitArrow began in energy-systems research — where BTU, MWh, and
132
+ per-unit quantities collide daily — but nothing in it is
133
+ energy-specific.*
unitarrow-0.0.1/pyproject.toml ADDED
@@ -0,0 +1,17 @@
1
+ [build-system]
2
+ requires = ["hatchling"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "unitarrow"
7
+ version = "0.0.1"
8
+ description = "Physical units and quantity semantics for Apache Arrow columns (name reservation; spec in development)"
9
+ readme = "README.md"
10
+ license = "BSD-3-Clause"
11
+ requires-python = ">=3.9"
12
+
13
+ [project.urls]
14
+ Repository = "https://github.com/unitarrow/unitarrow"
15
+
16
+ [tool.hatch.build.targets.wheel]
17
+ packages = ["src/unitarrow"]
unitarrow-0.0.1/src/unitarrow/__init__.py ADDED
@@ -0,0 +1,6 @@
1
+ """UnitArrow: physical units and quantity semantics for Apache Arrow columns.
2
+
3
+ Name reservation - the specification is in development.
4
+ See https://github.com/unitarrow/unitarrow
5
+ """
6
+ __version__ = "0.0.1"