dataiter 1.3.1__tar.gz → 2.0__tar.gz

{dataiter-1.3.1 → dataiter-2.0}/.gitignore

@@ -1,19 +1,17 @@
-*.lprof
-*.prof
-.claude
+.claude/settings.local.json
 .env
 .envrc
 .pytest_cache
+.venv
 __pycache__
-benchmark-head.py
 benchmark-versions.csv
 benchmark-versions.ods
 build
-dataiter.egg-info
 dist
 doc/_build
 doc/comparison/comparison.html
 test.py
+tmp
 tmp.csv
+uv.lock
 validation/*.csv
-venv

{dataiter-1.3.1 → dataiter-2.0}/PKG-INFO

@@ -1,21 +1,22 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.5
 Name: dataiter
-Version: 1.3.1
+Version: 2.0
 Summary: Simple, light-weight data frames for Python
 Project-URL: Homepage, https://github.com/otsaloma/dataiter
+Project-URL: Documentation, https://dataiter.readthedocs.io/
+Project-URL: Changelog, https://github.com/otsaloma/dataiter/blob/master/NEWS.md
 Author-email: Osmo Salomaa <otsaloma@iki.fi>
-License: MIT
+License-Expression: MIT
 License-File: AUTHORS.md
 License-File: COPYING
-Requires-Python: >=3.11.0
+Requires-Python: >=3.12.0
 Requires-Dist: attd>=0.3
-Requires-Dist: numpy<3.0,>=2.4.0
-Requires-Dist: pyarrow>=2.0
-Requires-Dist: wcwidth>=0.1
+Requires-Dist: numpy<3.0,>=2.5.0
+Requires-Dist: pyarrow>=26.0
+Requires-Dist: wcwidth>=0.3.0
 Description-Content-Type: text/markdown
 
-Simple, Light-Weight Data Frames for Python
-===========================================
+# Simple, Light-Weight Data Frames for Python
 
 [![PyPI](https://img.shields.io/pypi/v/dataiter.svg)](https://pypi.org/project/dataiter)
 [![Downloads](https://pepy.tech/badge/dataiter/month)](https://pepy.tech/project/dataiter)
@@ -25,8 +26,9 @@ Dataiter's **`DataFrame`** is a class for tabular data similar to R's
 is under the hood a dictionary of NumPy arrays and thus capable of fast
 vectorized operations. You can consider it to be a light-weight
 alternative to Pandas with a simple and consistent API. Performance-wise
-Dataiter relies on NumPy and Numba and is likely to be at best
-comparable to Pandas.
+Dataiter relies on NumPy and PyArrow and is close to the level of
+Pandas. Currently the notable slow part is string data and operations
+(Dataiter's NumPy StringDType being slower than Pandas' Arrow strings).
 
 ## Installation
 
@@ -36,18 +38,8 @@ pip install -U dataiter
 
 # Latest development version
 pip install -U git+https://github.com/otsaloma/dataiter
-
-# Numba (optional)
-pip install -U numba
 ```
 
-Recommended NumPy version is currently >= 2.4.0 due to various
-StringDType fixes that have landed in NumPy 2.2.1 and 2.4.0.
-
-Dataiter optionally uses **Numba** to speed up certain operations. If
-you have Numba installed, Dataiter will use it automatically. It's
-currently not a hard dependency, so you need to install it separately.
-
 ## Quick Start
 
 ```python
@@ -81,12 +73,23 @@ the differences and similarities in common operations.
 
 https://dataiter.readthedocs.io/en/stable/comparison.html
 
-## Development
+### For Coding Agents
+
+Dataiter's API is small enough that the complete documentation fits in
+about 50k tokens. Coding agents can read all of it in one go and know
+the whole API before writing any code. This differs from bigger
+packages, like Pandas and Polars, which must fight against outdated
+training data and introduce their huge API piecemeal, hoping the agent
+knows what to ask for. For best results, have your agent download
+`llms-full.txt` and read it in full, rather than fetch it with a tool
+that summarizes web pages. Each documentation page is also available as
+Markdown by replacing `.html` with `.md` in the URL.
 
-To install a virtualenv for development, use
+- https://dataiter.readthedocs.io/en/stable/llms-full.txt
+- https://dataiter.readthedocs.io/en/stable/data-frame.md
 
-    make venv
+## Development
 
-or, for a specific Python version
+Install uv from https://docs.astral.sh/uv/ and then run
 
-    make PYTHON=python3.X venv
+    uv sync

{dataiter-1.3.1 → dataiter-2.0}/README.md

@@ -1,5 +1,4 @@
-Simple, Light-Weight Data Frames for Python
-===========================================
+# Simple, Light-Weight Data Frames for Python
 
 [![PyPI](https://img.shields.io/pypi/v/dataiter.svg)](https://pypi.org/project/dataiter)
 [![Downloads](https://pepy.tech/badge/dataiter/month)](https://pepy.tech/project/dataiter)
@@ -9,8 +8,9 @@ Dataiter's **`DataFrame`** is a class for tabular data similar to R's
 is under the hood a dictionary of NumPy arrays and thus capable of fast
 vectorized operations. You can consider it to be a light-weight
 alternative to Pandas with a simple and consistent API. Performance-wise
-Dataiter relies on NumPy and Numba and is likely to be at best
-comparable to Pandas.
+Dataiter relies on NumPy and PyArrow and is close to the level of
+Pandas. Currently the notable slow part is string data and operations
+(Dataiter's NumPy StringDType being slower than Pandas' Arrow strings).
 
 ## Installation
 
@@ -20,18 +20,8 @@ pip install -U dataiter
 
 # Latest development version
 pip install -U git+https://github.com/otsaloma/dataiter
-
-# Numba (optional)
-pip install -U numba
 ```
 
-Recommended NumPy version is currently >= 2.4.0 due to various
-StringDType fixes that have landed in NumPy 2.2.1 and 2.4.0.
-
-Dataiter optionally uses **Numba** to speed up certain operations. If
-you have Numba installed, Dataiter will use it automatically. It's
-currently not a hard dependency, so you need to install it separately.
-
 ## Quick Start
 
 ```python
@@ -65,12 +55,23 @@ the differences and similarities in common operations.
 
 https://dataiter.readthedocs.io/en/stable/comparison.html
 
-## Development
+### For Coding Agents
+
+Dataiter's API is small enough that the complete documentation fits in
+about 50k tokens. Coding agents can read all of it in one go and know
+the whole API before writing any code. This differs from bigger
+packages, like Pandas and Polars, which must fight against outdated
+training data and introduce their huge API piecemeal, hoping the agent
+knows what to ask for. For best results, have your agent download
+`llms-full.txt` and read it in full, rather than fetch it with a tool
+that summarizes web pages. Each documentation page is also available as
+Markdown by replacing `.html` with `.md` in the URL.
 
-To install a virtualenv for development, use
+- https://dataiter.readthedocs.io/en/stable/llms-full.txt
+- https://dataiter.readthedocs.io/en/stable/data-frame.md
 
-    make venv
+## Development
 
-or, for a specific Python version
+Install uv from https://docs.astral.sh/uv/ and then run
 
-    make PYTHON=python3.X venv
+    uv sync

dataiter-2.0/dataiter/__init__.py

@@ -0,0 +1,109 @@
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2020 Osmo Salomaa
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+__version__ = "2.0"
+
+#: Default amount of elements to return from :meth:`Vector.head`,
+#: :meth:`Vector.sample` and :meth:`Vector.tail`.
+DEFAULT_PEEK_ELEMENTS = 10
+
+#: Default amount of items to return from :meth:`ListOfDicts.head`,
+#: :meth:`ListOfDicts.sample` and :meth:`ListOfDicts.tail`.
+DEFAULT_PEEK_ITEMS = 3
+
+#: Default amount of rows to return from :meth:`DataFrame.head`,
+#: :meth:`DataFrame.sample` and :meth:`DataFrame.tail`.
+DEFAULT_PEEK_ROWS = 10
+
+#: Amount of significant digits to print floats with, all values of a
+#: column or vector using the decimals that the largest value gets.
+PRINT_FLOAT_PRECISION = 6
+
+#: Maximum amount of elements to show when printing a :class:`Vector`.
+PRINT_MAX_ELEMENTS = 100
+
+#: Maximum amount of items to show when printing a :class:`ListOfDicts`.
+PRINT_MAX_ITEMS = 10
+
+#: Maximum amount of rows to show when printing a :class:`DataFrame`.
+PRINT_MAX_ROWS = 20
+
+#: Maximum amount of columns to wrap print output to. This is only a fallback
+#: in case Python's ``shutil.get_terminal_size`` fails to detect the width of
+#: your terminal. By default the detected full width is used.
+PRINT_MAX_WIDTH = 80
+
+#: Thousand separator to use when printing numbers. By default this is blank,
+#: meaning no thousand separators are rendered.
+PRINT_THOUSAND_SEPARATOR = ""
+
+#: Maximum width to truncate string columns to in :class:`DataFrame` print
+#: output. When this is exceeded, strings will be cut and an ellipsis (``…``)
+#: rendered at the cut point.
+PRINT_TRUNCATE_WIDTH = 36
+
+from dataiter import dtypes
+from dataiter.vector import Vector
+from dataiter.vector import of_type
+from dataiter.data_frame import DataFrame
+from dataiter.data_frame import DataFrameColumn
+from dataiter.geojson import GeoJSON
+from dataiter.list_of_dicts import ListOfDicts
+from dataiter import dt
+from dataiter import regex
+
+from dataiter.aggregate import all
+from dataiter.aggregate import any
+from dataiter.aggregate import count
+from dataiter.aggregate import count_unique
+from dataiter.aggregate import first
+from dataiter.aggregate import last
+from dataiter.aggregate import max
+from dataiter.aggregate import mean
+from dataiter.aggregate import median
+from dataiter.aggregate import min
+from dataiter.aggregate import mode
+from dataiter.aggregate import nth
+from dataiter.aggregate import prod
+from dataiter.aggregate import quantile
+from dataiter.aggregate import std
+from dataiter.aggregate import sum
+from dataiter.aggregate import var
+
+from dataiter.io import read_csv
+from dataiter.io import read_geojson
+from dataiter.io import read_json
+from dataiter.io import read_jsonl
+from dataiter.io import read_npz
+from dataiter.io import read_parquet
+
+from dataiter import util
+
+#: Special value to refer to the next element/item.
+#: Used by ``replace_na`` to replace missing values with
+#: the next non-missing value, i.e. to fill backward.
+NEXT = util.Sentinel("NEXT")
+
+#: Special value to refer to the previous element/item.
+#: Used by ``replace_na`` to replace missing values with
+#: the previous non-missing value, i.e. to fill forward.
+PREVIOUS = util.Sentinel("PREVIOUS")