arrowsql 0.1.0__tar.gz

arrowsql-0.1.0/.cargo/config.toml

@@ -0,0 +1,7 @@
+# Size flag for the release cdylib on x86_64 glibc Linux (lld, the default linker
+# since Rust 1.90): identical-code folding.
+# `-z pack-relative-relocs` (DT_RELR) is NOT set here: it needs glibc >= 2.36 at
+# load time and would break manylinux_2_28 wheels. scripts/build.sh --release and
+# scripts/wheel_size.sh add it for local builds (scripts/release_rustflags.sh).
+[target.x86_64-unknown-linux-gnu]
+rustflags = ["-C", "link-arg=-Wl,--icf=all"]

arrowsql-0.1.0/.gitattributes

@@ -0,0 +1,6 @@
+# GitHub language statistics (linguist).
+# tests/upstream is Microsoft's mssql-python test suite, copied in unchanged.
+tests/upstream/** linguist-vendored
+# vendor/mssql-tds is the TDS client this driver is built on, carried with local
+# patches (vendor/mssql-tds/VENDOR.md): count it as this repository's Rust.
+vendor/mssql-tds/** linguist-vendored=false

arrowsql-0.1.0/.gitignore

@@ -0,0 +1,15 @@
+.venv/
+.venv-upstream/
+.env.local
+data/
+target/
+*.so
+__pycache__/
+*.xml
+*.log
+.worktrees/
+.codegraph/
+reviews/*.log
+.claude/worktrees/
+.overlay-upstream/
+.tls-test/

arrowsql-0.1.0/CHANGELOG.md

@@ -0,0 +1,83 @@
+# Changelog
+
+All notable changes to this project. Format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
+[SemVer](https://semver.org/) once 1.0 is reached (0.x may break).
+
+Version numbers in this project are deliberately three different things:
+
+| what | value | meaning |
+|---|---|---|
+| distribution `arrowsql` | 0.1.0 | this project's release number (`pyproject.toml`) |
+| `arrowsql.__version__` | 1.16.0 | the upstream `mssql-python` release the Python layer is vendored from; kept so code that checks the upstream version keeps working |
+| `arrowsql.ddbc_bindings.__version__` | 1.0.0 | version of the Rust binding contract (`docs/CONTRACT.md`) |
+
+## [Unreleased]
+
+### Added
+
+- UTF-8 transfer for Arrow reads: `read_arrow(..., utf8_strings=True)` and the new
+  `cursor.execute_arrow(sql, params=None, *, utf8_strings=True, batch_size=...)` describe the
+  result and cast nvarchar/nchar/ntext columns to a UTF-8 collated varchar on the server (query
+  wrapping is built in Rust: `ddbc_bindings.utf8_wrap_query`). About half the string bytes on
+  the wire; same Arrow result. Needs SQL Server 2019+/Azure SQL; falls back to the original
+  query on ORDER BY, parameters, duplicate/empty column names, describe failures, a small
+  string share of the row, or when the first 256 rows are not at least 10% smaller in UTF-8 (CJK
+  grows). `utf8_strings=False` opts out.
+- Bulk copy writes each DECIMAL cell at the shortest legal magnitude width (4/8/12/16 bytes)
+  instead of the width implied by the column precision. SQL Server accepts this; 1M rows of nine
+  `decimal(38,2)` columns with small values upload 55 MB instead of 163 MB. `ARROWSQL_DECIMAL_FIXED=1`
+  restores precision-width cells.
+- The project is called **arrowsql** (distribution and import name `arrowsql`,
+  repository github.com/bmsuisse/arrowsql). A `mssql_python` alias package keeps
+  `import mssql_python` working: its modules are the `arrowsql` modules.
+- PyO3 extension `arrowsql.ddbc_bindings` over Microsoft's `mssql-tds`
+  (vendored, with local patches, in `vendor/mssql-tds`): no ODBC driver, same
+  Python API as upstream mssql-python 1.16.0.
+- Native Arrow: `cursor.arrow()`, `arrow_batch()`, `arrow_reader()`,
+  `fetchall_arrow()`, `fetchmany_arrow()`, `arrow_stream()`, `read_arrow()`
+  (partitioned parallel reads).
+- Agent mode (`arrowsql.agent`, `python -m arrowsql.agent`, console script
+  `arrowsql-agent`), ported from arrowbricks: `query()` / `query_cursor()` return
+  a compact typed TSV (or Markdown) preview with a strict byte ceiling, row and
+  cell limits, an optional head/tail sample, a footer that never claims an
+  unknown total, `-- rows_affected` / `-- message` lines and every result set;
+  failures are JSON with a category, the server's own message and a recovery
+  hint (SQL Server error number, then SQLSTATE). Rendering is Rust
+  (`ddbc_bindings.AgentRender`) over the Arrow sink; `sql_variant` is now
+  readable through it.
+- `read_arrow_stream()`: partitioned parallel read as a bounded, unordered Arrow
+  stream (`__arrow_c_stream__`; native `ArrowMergeStream` fed by one connection
+  per worker) so DuckDB / deltalake overlap with the read and memory stays
+  bounded. `close()` or an early-stopping consumer cancels the statements and
+  closes the worker connections; the first worker error reaches the consumer.
+  `read_arrow()` is now built on it. `partition_bounds=` (and
+  `partition_nullable=`) take typed boundaries (int, date/datetime, Decimal,
+  str) as disjoint half-open slices plus a NULL slice, bound as parameters,
+  with no MIN/MAX or size probe.
+- Native bulk copy (rows and Arrow), `write_arrow()` parallel loads.
+- Native asyncio (`arrowsql.aio`) with a built-in pool (Unix only).
+- `sqlstate` and `native_error` on every exception
+  ([upstream #581](https://github.com/microsoft/mssql-python/issues/581));
+  bulk copy on the cursor's own connection, so `#temp` tables work
+  ([#492](https://github.com/microsoft/mssql-python/issues/492)).
+- Type stubs for the port-specific API (`ddbc_bindings.pyi`, RUST-PORT section of
+  `mssql_python.pyi`).
+- CI (`.github/workflows/ci.yml`) and a wheel/sdist build workflow
+  (`wheels.yml`, artifacts only, no publishing).
+- Project metadata: SPDX license, classifiers, URLs; CONTRIBUTING.
+
+### Changed
+
+- `-z pack-relative-relocs` moved out of `.cargo/config.toml` into the local
+  release scripts, so CI manylinux_2_28 wheels do not require glibc 2.36.
+- The `async-preview` extra pins `mssql-python-rs==0.4.0` (was 0.3.0).
+- The sdist now carries `vendor/mssql-tds`.
+
+### Known gaps
+
+- Pool checkout does not probe an idle connection with a timeout (upstream
+  GH #854), and `set_attr(SQL_ATTR_CONNECTION_TIMEOUT)` is not carried across
+  pool reuse. See `tests/upstream_excluded.txt`.
+- Always Encrypted, `ActiveDirectoryPassword` and NTLM are not supported.
+- Only Linux x86_64 has been exercised against a database.

arrowsql-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,45 @@
+# Contributing
+
+## Setup
+
+You need Rust (stable, 1.90+), Python 3.10+ and a SQL Server 2022 you can write to
+(`mcr.microsoft.com/mssql/server:2022-latest` in docker or podman is fine).
+
+```bash
+uv venv -p 3.13 .venv && VIRTUAL_ENV=.venv uv pip install -r requirements-dev.txt
+scripts/build.sh                 # dev build into .venv; --release for benchmarks
+export DB_CONNECTION_STRING="Server=localhost,1433;Database=master;Uid=sa;Pwd=...;Encrypt=yes;TrustServerCertificate=yes"
+```
+
+`scripts/env.sh` and `scripts/run_tests.py` are conveniences for the maintainers'
+setup (a `.env.local` next to the main checkout, one private database per run);
+anyone else can just set `DB_CONNECTION_STRING`.
+
+## Tests
+
+```bash
+python -m pytest -q --timeout 120 tests/common_issues tests/port
+python scripts/check_upstream_baseline.py     # vendored upstream suite vs tests/upstream_passing_baseline.txt
+cargo test --lib
+```
+
+Scope and rules for the three suites: [docs/TEST_SCOPE.md](docs/TEST_SCOPE.md). A new
+behaviour needs a test in `tests/common_issues`. If you exclude an upstream test,
+add it to `tests/upstream_excluded.txt` with a one-line reason.
+
+## Code
+
+- `cargo clippy --all-targets` should not add warnings; run `cargo fmt` on files you touch.
+- Edits to the vendored Python layer (`python/arrowsql`) and to `vendor/mssql-tds`
+  are marked `RUST-PORT` and listed in `docs/PYTHON_LAYER_CHANGES.md` /
+  `vendor/mssql-tds/VENDOR.md`. Keep upstream's lines unchanged where you can.
+- New binding symbols go into `docs/CONTRACT.md` and `python/arrowsql/ddbc_bindings.pyi`.
+- Performance claims need a run of `benchmarks/run.py` (see `benchmarks/README.md`);
+  do not hand-edit `benchmarks/RESULTS.md` or the README benchmark tables.
+- Do not commit local paths, host names or credentials.
+
+## Pull requests
+
+Describe what changed and how you tested it, update `CHANGELOG.md`, and keep commits
+focused. Releases and PyPI publishing are done by the maintainers
+(`docs/PACKAGING.md`).