rowproof 0.1.0__tar.gz → 0.1.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. {rowproof-0.1.0 → rowproof-0.1.2}/PKG-INFO +13 -1
  2. {rowproof-0.1.0 → rowproof-0.1.2}/README.md +12 -0
  3. {rowproof-0.1.0 → rowproof-0.1.2}/pyproject.toml +1 -1
  4. rowproof-0.1.2/rowproof/__init__.py +1 -0
  5. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/cli/main.py +21 -12
  6. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/errors.py +26 -0
  7. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/hashdiff.py +11 -1
  8. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/PKG-INFO +13 -1
  9. rowproof-0.1.0/rowproof/__init__.py +0 -1
  10. {rowproof-0.1.0 → rowproof-0.1.2}/LICENSE +0 -0
  11. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/cli/__init__.py +0 -0
  12. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/cli/render.py +0 -0
  13. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/cli/spec.py +0 -0
  14. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/config/__init__.py +0 -0
  15. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/__init__.py +0 -0
  16. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/_chwire.py +0 -0
  17. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/_pgwire.py +0 -0
  18. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/_sfwire.py +0 -0
  19. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/base.py +0 -0
  20. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/clickhouse.py +0 -0
  21. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/postgres.py +0 -0
  22. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/connectors/snowflake.py +0 -0
  23. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/__init__.py +0 -0
  24. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/column_matching.py +0 -0
  25. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/joindiff.py +0 -0
  26. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/models.py +0 -0
  27. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/normalisation.py +0 -0
  28. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/core/segmentation.py +0 -0
  29. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/report/__init__.py +0 -0
  30. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof/report/html.py +0 -0
  31. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/SOURCES.txt +0 -0
  32. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/dependency_links.txt +0 -0
  33. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/entry_points.txt +0 -0
  34. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/requires.txt +0 -0
  35. {rowproof-0.1.0 → rowproof-0.1.2}/rowproof.egg-info/top_level.txt +0 -0
  36. {rowproof-0.1.0 → rowproof-0.1.2}/setup.cfg +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rowproof
3
- Version: 0.1.0
3
+ Version: 0.1.2
4
4
  Summary: Prove a database migration or replication didn't lose, add or change a single row.
5
5
  License: Apache-2.0
6
6
  Requires-Python: >=3.11
@@ -87,6 +87,18 @@ public.orders postgres:postgres/public.orders -> clickhouse:default/orders
87
87
  | ClickHouse | `clickhouse-connect` | |
88
88
  | Snowflake | `snowflake-connector-python` | Optional extra: `rowproof[snowflake]` |
89
89
 
90
+ ### Snowflake: known limitations
91
+
92
+ Snowflake support works, but it is the least battle-tested of the three engines. It has been tested against a single trial account with small tables; Postgres and ClickHouse have had far more. Know these before you rely on it:
93
+
94
+ - **Login is username and password only.** The connection string has no place for a warehouse, role, SSO, MFA, or key-pair login. Accounts that require SSO or key-pair authentication won't work yet, and queries run on the user's *default* warehouse (a user with none set will fail).
95
+ - **Column names are case-sensitive.** Snowflake stores unquoted names in upper case, so `--key id` won't match a column stored as `ID`. Pass `--key ID`, or create the table with quoted lower-case names. The error message tells you when this is the problem.
96
+ - **Primary keys are often not declared** on Snowflake tables (Snowflake doesn't enforce them). If auto-detection finds none, pass `--key`.
97
+ - **JSON columns are compared as plain text.** `VARIANT`, `ARRAY` and `OBJECT` use rule UNK-1: two JSON values that mean the same thing but are formatted differently will show as different, and a JSON `null` is not distinguished from an SQL `NULL`.
98
+ - **Special characters in passwords must be percent-encoded** in the connection string (`@` becomes `%40`).
99
+ - **It uses your warehouse's credits.** Every query runs on your Snowflake warehouse. Start with a small table or `--sample`.
100
+ - **No Snowflake benchmark yet.** The 100M-row benchmark below is Postgres to ClickHouse only.
101
+
90
102
  ## Supported types (canonical comparison rules)
91
103
 
92
104
  Every cross-engine type difference is governed by one of these published rules, and every difference rowproof reports cites the rule ID that applied — never a vague "values differ."
@@ -65,6 +65,18 @@ public.orders postgres:postgres/public.orders -> clickhouse:default/orders
65
65
  | ClickHouse | `clickhouse-connect` | |
66
66
  | Snowflake | `snowflake-connector-python` | Optional extra: `rowproof[snowflake]` |
67
67
 
68
+ ### Snowflake: known limitations
69
+
70
+ Snowflake support works, but it is the least battle-tested of the three engines. It has been tested against a single trial account with small tables; Postgres and ClickHouse have had far more. Know these before you rely on it:
71
+
72
+ - **Login is username and password only.** The connection string has no place for a warehouse, role, SSO, MFA, or key-pair login. Accounts that require SSO or key-pair authentication won't work yet, and queries run on the user's *default* warehouse (a user with none set will fail).
73
+ - **Column names are case-sensitive.** Snowflake stores unquoted names in upper case, so `--key id` won't match a column stored as `ID`. Pass `--key ID`, or create the table with quoted lower-case names. The error message tells you when this is the problem.
74
+ - **Primary keys are often not declared** on Snowflake tables (Snowflake doesn't enforce them). If auto-detection finds none, pass `--key`.
75
+ - **JSON columns are compared as plain text.** `VARIANT`, `ARRAY` and `OBJECT` use rule UNK-1: two JSON values that mean the same thing but are formatted differently will show as different, and a JSON `null` is not distinguished from an SQL `NULL`.
76
+ - **Special characters in passwords must be percent-encoded** in the connection string (`@` becomes `%40`).
77
+ - **It uses your warehouse's credits.** Every query runs on your Snowflake warehouse. Start with a small table or `--sample`.
78
+ - **No Snowflake benchmark yet.** The 100M-row benchmark below is Postgres to ClickHouse only.
79
+
68
80
  ## Supported types (canonical comparison rules)
69
81
 
70
82
  Every cross-engine type difference is governed by one of these published rules, and every difference rowproof reports cites the rule ID that applied — never a vague "values differ."
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "rowproof"
3
- version = "0.1.0"
3
+ version = "0.1.2"
4
4
  description = "Prove a database migration or replication didn't lose, add or change a single row."
5
5
  readme = "README.md"
6
6
  requires-python = ">=3.11"
@@ -0,0 +1 @@
1
+ __version__ = "0.1.2"
@@ -135,23 +135,26 @@ def _parse_column_map(value: str | None) -> dict | None:
135
135
  return result or None
136
136
 
137
137
 
138
+ def _hook_sql_log(connector, sql_log: list[str] | None) -> None:
139
+ """spec §8.2: JSON output carries every generated SQL statement too —
140
+ reuse the exact same on_query hook --verbose already installs
141
+ (chaining onto it, so --verbose logging to stderr keeps working
142
+ unchanged) rather than a second SQL-capturing mechanism. `list.append`
143
+ is atomic in CPython, so pooled worker threads can share one log.
144
+ """
145
+ if sql_log is None or not hasattr(connector, "on_query"):
146
+ return
147
+ previous = connector.on_query
148
+ connector.on_query = lambda sql, _prev=previous: (_prev(sql) if _prev else None, sql_log.append(sql))
149
+
150
+
138
151
  def _connect_pair(args, sql_log: list[str] | None = None):
139
152
  src_spec = parse_source_spec(args.source)
140
153
  tgt_spec = parse_source_spec(args.target)
141
154
  source = _make_connector(src_spec.engine, args.verbose)
142
155
  target = _make_connector(tgt_spec.engine, args.verbose)
143
- if sql_log is not None:
144
- for connector in (source, target):
145
- if hasattr(connector, "on_query"):
146
- previous = connector.on_query
147
- # spec §8.2: JSON output carries every generated SQL
148
- # statement too — reuse the exact same on_query hook
149
- # --verbose already installs (chaining onto it, so
150
- # --verbose logging to stderr keeps working unchanged)
151
- # rather than a second SQL-capturing mechanism.
152
- connector.on_query = (
153
- lambda sql, _prev=previous: (_prev(sql) if _prev else None, sql_log.append(sql))
154
- )
156
+ _hook_sql_log(source, sql_log)
157
+ _hook_sql_log(target, sql_log)
155
158
  _connect_with_retry(source, src_spec.connect_dsn)
156
159
  _connect_with_retry(target, tgt_spec.connect_dsn)
157
160
 
@@ -168,6 +171,12 @@ def _connect_pair(args, sql_log: list[str] | None = None):
168
171
  if threads > 1:
169
172
  source_pool = [_make_connector(src_spec.engine, args.verbose) for _ in range(threads)]
170
173
  target_pool = [_make_connector(tgt_spec.engine, args.verbose) for _ in range(threads)]
174
+ # The pooled connectors run the per-segment hash queries -- the
175
+ # actual proof of a match -- so they must feed the same SQL log as
176
+ # the main connectors, or the JSON/HTML "Reproduce" output would
177
+ # silently omit exactly the queries a reviewer wants to re-run.
178
+ for connector in (*source_pool, *target_pool):
179
+ _hook_sql_log(connector, sql_log)
171
180
  for connector in source_pool:
172
181
  _connect_with_retry(connector, src_spec.connect_dsn)
173
182
  for connector in target_pool:
@@ -20,6 +20,32 @@ class NoPrimaryKeyError(RowProofError):
20
20
  self.table_name = table_name
21
21
 
22
22
 
23
+ class TableNotFoundError(RowProofError):
24
+ def __init__(self, table_name: str):
25
+ super().__init__(
26
+ f"Table '{table_name}' not found, or it has no visible columns "
27
+ "(check the name, the schema, and that this login can read it)."
28
+ )
29
+ self.table_name = table_name
30
+
31
+
32
+ class KeyColumnNotFoundError(RowProofError):
33
+ def __init__(self, table_name: str, column: str, available: list[str]):
34
+ hint = ""
35
+ by_lower = {c.lower(): c for c in available}
36
+ if column.lower() in by_lower and by_lower[column.lower()] != column:
37
+ hint = (
38
+ f" Did you mean '{by_lower[column.lower()]}'? Column names are "
39
+ "case-sensitive here (Snowflake stores unquoted names in upper case)."
40
+ )
41
+ super().__init__(
42
+ f"Key column '{column}' not found in '{table_name}'.{hint} "
43
+ f"Available columns: {', '.join(available)}."
44
+ )
45
+ self.table_name = table_name
46
+ self.column = column
47
+
48
+
23
49
  class NonUniqueKeyError(RowProofError):
24
50
  def __init__(self, table_name: str, key_columns: list[str], duplicate_example: tuple):
25
51
  example = ", ".join(f"{c}={v!r}" for c, v in zip(key_columns, duplicate_example))
@@ -19,7 +19,12 @@ from dataclasses import dataclass
19
19
 
20
20
  from rowproof.connectors.base import Connector
21
21
  from rowproof.core.column_matching import match_columns
22
- from rowproof.core.errors import NonUniqueKeyError, NoPrimaryKeyError
22
+ from rowproof.core.errors import (
23
+ KeyColumnNotFoundError,
24
+ NonUniqueKeyError,
25
+ NoPrimaryKeyError,
26
+ TableNotFoundError,
27
+ )
23
28
  from rowproof.core.models import (
24
29
  Algorithm,
25
30
  Column,
@@ -105,7 +110,12 @@ def _build_plan(
105
110
  where: str | None = None,
106
111
  ) -> _Plan:
107
112
  cols_by_name = _resolve_columns(connector, table)
113
+ if not cols_by_name:
114
+ raise TableNotFoundError(table.qualified_name)
108
115
  resolved_key = _resolve_key_columns(connector, table, key_columns)
116
+ for key_col in resolved_key:
117
+ if key_col not in cols_by_name:
118
+ raise KeyColumnNotFoundError(table.qualified_name, key_col, list(cols_by_name))
109
119
 
110
120
  if columns:
111
121
  selected = list(columns)
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: rowproof
3
- Version: 0.1.0
3
+ Version: 0.1.2
4
4
  Summary: Prove a database migration or replication didn't lose, add or change a single row.
5
5
  License: Apache-2.0
6
6
  Requires-Python: >=3.11
@@ -87,6 +87,18 @@ public.orders postgres:postgres/public.orders -> clickhouse:default/orders
87
87
  | ClickHouse | `clickhouse-connect` | |
88
88
  | Snowflake | `snowflake-connector-python` | Optional extra: `rowproof[snowflake]` |
89
89
 
90
+ ### Snowflake: known limitations
91
+
92
+ Snowflake support works, but it is the least battle-tested of the three engines. It has been tested against a single trial account with small tables; Postgres and ClickHouse have had far more. Know these before you rely on it:
93
+
94
+ - **Login is username and password only.** The connection string has no place for a warehouse, role, SSO, MFA, or key-pair login. Accounts that require SSO or key-pair authentication won't work yet, and queries run on the user's *default* warehouse (a user with none set will fail).
95
+ - **Column names are case-sensitive.** Snowflake stores unquoted names in upper case, so `--key id` won't match a column stored as `ID`. Pass `--key ID`, or create the table with quoted lower-case names. The error message tells you when this is the problem.
96
+ - **Primary keys are often not declared** on Snowflake tables (Snowflake doesn't enforce them). If auto-detection finds none, pass `--key`.
97
+ - **JSON columns are compared as plain text.** `VARIANT`, `ARRAY` and `OBJECT` use rule UNK-1: two JSON values that mean the same thing but are formatted differently will show as different, and a JSON `null` is not distinguished from an SQL `NULL`.
98
+ - **Special characters in passwords must be percent-encoded** in the connection string (`@` becomes `%40`).
99
+ - **It uses your warehouse's credits.** Every query runs on your Snowflake warehouse. Start with a small table or `--sample`.
100
+ - **No Snowflake benchmark yet.** The 100M-row benchmark below is Postgres to ClickHouse only.
101
+
90
102
  ## Supported types (canonical comparison rules)
91
103
 
92
104
  Every cross-engine type difference is governed by one of these published rules, and every difference rowproof reports cites the rule ID that applied — never a vague "values differ."
@@ -1 +0,0 @@
1
- __version__ = "0.1.0"
File without changes
File without changes
File without changes