SQLDbWrpr 5.1.0__tar.gz → 5.2.0__tar.gz

{sqldbwrpr-5.1.0 → sqldbwrpr-5.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: SQLDbWrpr
-Version: 5.1.0
+Version: 5.2.0
 Summary: Wrapper for importing CSV and Text files into MySQL and Postgress
 License-Expression: MIT
 License-File: LICENSE.txt
@@ -65,7 +65,7 @@ When both are supplied, `p_db_structure` takes precedence for backward compatibi
   - Import CSV data from files or in-memory rows, including single-volume and numbered multi-volume files.
   - Export full tables or custom SQL query results to CSV, including optional multi-volume exports.
 - **Database Support**: Includes MySQL and PostgreSQL wrappers with dialect-specific SQL rendering.
-- **User and Permission Management**: Create MySQL users and grant database rights.
+- **User and Permission Management**: Create and delete MySQL users or PostgreSQL login roles, and grant backend-specific database or table rights.
 - **Batch Processing**: Configure import batch sizes for larger CSV loads.
 
 ### Project Structure
@@ -83,6 +83,17 @@ When both are supplied, `p_db_structure` takes precedence for backward compatibi
 pip install SqlDbWrpr
 ```
 
+### Running the Tests
+
+The focused database tests use temporary MySQL 8.0 and PostgreSQL 16 Docker containers. Ensure Docker is running, then install the development dependencies and run the test suite:
+
+```powershell
+poetry install --with dev
+poetry run pytest
+```
+
+The test fixtures wait for each database to become ready and remove the temporary containers after the test session.
+
 ### Quick Start With A Legacy Structure
 
 ```python
@@ -167,6 +178,115 @@ db = PostgreSQL(
 db.import_csv("Users", p_csv_db=[("Username",), ("alice",)], p_vol_type="Single")
 ```
 
+## Function Reference
+
+This section documents the callable API in `src/sqldbwrpr/sqldbwrpr.py`.
+
+### `SchemaSourceError`
+
+- `SchemaSourceError(ValueError)`: Raised when no usable schema source is provided to initialize a wrapper (`p_db_structure`, `p_sqlalchemy_metadata`, or `p_sqlalchemy_base.metadata`).
+
+### `SQLDbWrpr` (base wrapper)
+
+- `__init__(p_host_name, p_user_name, p_password, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_db_port, p_ssl_ca, p_ssl_key, p_ssl_cert)`
+  - Initializes shared wrapper state, resolves schema source, configures import/export behavior, and precomputes char vs non-char field maps.
+  - Schema resolution order: explicit `p_db_structure` first, then SQLAlchemy metadata, else raises `SchemaSourceError`.
+- `close()`
+  - Closes the active DB connection if present.
+- `build_column_sql(p_field_name, p_field_type, p_field_params, p_field_comment)`
+  - Builds the column fragment for `CREATE TABLE` in the active dialect (base implementation is MySQL-oriented).
+  - Applies `AUTO_INCREMENT`, `UNSIGNED`, `NOT NULL`, `ZEROFILL`, defaults, and comments based on field parameters.
+- `build_index_sql(p_table_name, p_idx_name, p_idx_fields, p_unique=False)`
+  - Builds an index clause with sort directions from legacy index metadata.
+- `build_insert_sql(p_table_name, p_header, p_replace=False)`
+  - Builds an `INSERT` or `REPLACE` statement with backend parameter placeholders.
+- `param_placeholder()`
+  - Returns the DB-API parameter token used by execution (`%s`).
+- `quote_identifier(p_identifier)`
+  - Quotes SQL identifiers when a backend quote character is configured; otherwise returns plain identifier text.
+- `quote_identifier_list(p_identifiers)`
+  - Quotes and joins a list of identifiers using commas.
+- `render_default_sql(p_field_type, p_default_value)`
+  - Renders a `DEFAULT` clause, quoting string-like defaults for char/varchar types.
+- `render_field_type(p_field_type, p_field_params)`
+  - Renders legacy type arrays into SQL type declarations (e.g., `varchar(n)`, `decimal(p,s)`).
+- `create_db()`
+  - Base database recreation flow: drops an existing DB and creates/selects a fresh one.
+- `create_tables()`
+  - Validates legacy schema metadata and creates tables, keys, indexes, and constraints in dependency order.
+  - Splits table creation and post-create operations where backend rules require deferred statements.
+- `create_users(p_admin_user, p_new_users)`
+  - Creates MySQL users if they do not already exist.
+- `delete_users(p_admin_user, p_del_users)`
+  - Drops MySQL users that exist in `mysql.user`.
+- `_err_broken_rec(p_sql_str, p_csv_db_slice)`
+  - Retry helper used during imports to isolate/log failing rows and stop on first unrecoverable record.
+- `export_to_csv(p_csv_path, p_table_name, p_delimiter="|", p_strip_chars="", p__vol_size=0, p_sql_query="")`
+  - Exports table/query results to CSV in single-file or multi-volume mode.
+  - Multi-volume mode chunks output by record count and generates numbered files.
+- `get_db_field_types()`
+  - Populates per-table lists of character vs non-character columns for type-sensitive import handling.
+- `grant_rights(p_admin_user, p_user_rights)`
+  - Grants MySQL rights and corresponding grant-option privileges according to supplied rights tuples.
+- `import_csv(p_table_name, p_csv_file_name="", p_key="", p_header="", p_del_head=False, p_csv_db="", p_csv_corr_str_file_name="", p_vol_type="Multi", p_verbose=False, p_replace=False)`
+  - Imports CSV data from file(s) or in-memory rows.
+  - Supports single- and multi-volume input, optional header override/removal, correction-string preprocessing, type conversion, date normalization, and batched insert/replace.
+- `import_and_split_csv(p_split_struct, p_data, p_header="", p_insert_header=False, p_verbose=False, p_debug=False)`
+  - Splits an input dataset into multiple destination table payloads using declarative field mappings and transform commands, then imports each generated dataset.
+- `from_sqlalchemy_metadata(p_sqlalchemy_metadata)` (`@classmethod`)
+  - Converts SQLAlchemy `MetaData` (sorted tables) to the legacy `db_structure` dictionary.
+- `resolve_db_structure(p_db_structure=None, p_sqlalchemy_base=None, p_sqlalchemy_metadata=None)` (`@staticmethod`)
+  - Central schema source resolver used by constructors; raises `SchemaSourceError` when none are provided.
+- `_action_to_legacy_code(p_action)` (`@staticmethod`)
+  - Maps SQLAlchemy FK action text (e.g., `CASCADE`, `SET NULL`) to legacy action codes (`C`, `N`, etc.).
+- `_build_default_field_params()` (`@staticmethod`)
+  - Produces the baseline legacy field-parameter block used during SQLAlchemy conversion.
+- `_column_type_to_legacy(p_column)` (`@staticmethod`)
+  - Converts a SQLAlchemy column type object into the legacy type-array format.
+- `_column_to_legacy_field(p_column)` (`@staticmethod`)
+  - Converts one SQLAlchemy column into a legacy field definition, including nullability, autoincrement, defaults, and comments.
+- `_set_foreign_keys(p_table, p_table_structure)` (`@staticmethod`)
+  - Writes legacy foreign-key metadata onto converted table fields.
+- `_set_indexes(p_table, p_table_structure)` (`@staticmethod`)
+  - Writes legacy index metadata for converted table fields.
+- `_set_primary_key(p_table, p_table_structure)` (`@staticmethod`)
+  - Marks primary-key columns in converted legacy table definitions.
+- `_table_to_legacy_structure(p_table)` (`@staticmethod`)
+  - Converts an entire SQLAlchemy table into a legacy table structure and enriches it with keys/indexes.
+- `_print_err_msg(p_err, p_msg="")` (`@staticmethod`)
+  - Formats and prints database error details before termination paths.
+
+### `MySQL(SQLDbWrpr)`
+
+- `__init__(p_host_name, p_user_name, p_password, p_user_rights, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_admin_username, p_admin_user_password, p_db_port, **kwargs)`
+  - Opens a MySQL connection and cursor, optionally recreates database/tables, or selects the configured DB.
+  - Includes fallback logic to create and grant rights to missing users when valid admin credentials and rights data are supplied.
+
+### `PostgreSQL(SQLDbWrpr)`
+
+- `__init__(p_host_name, p_user_name, p_password, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_db_port, p_maintenance_db, **kwargs)`
+  - Configures PostgreSQL-specific behavior (quoted identifiers, non-inline indexes), opens connection, and optionally recreates DB/tables.
+- `build_column_sql(p_field_name, p_field_type, p_field_params, p_field_comment)`
+  - PostgreSQL-specific column SQL renderer; handles non-AI `NOT NULL` and defaults.
+- `build_index_sql(p_table_name, p_idx_name, p_idx_fields, p_unique=False)`
+  - Builds PostgreSQL `CREATE INDEX`/`CREATE UNIQUE INDEX` statements.
+- `build_insert_sql(p_table_name, p_header, p_replace=False)`
+  - Builds PostgreSQL `INSERT`; when `p_replace=True`, builds `ON CONFLICT` upsert/no-op behavior from primary-key metadata.
+- `create_db()`
+  - PostgreSQL DB recreation flow via `pg_database` lookup, optional forced drop, create, and reconnect.
+- `create_users(p_admin_user, p_new_users)`
+  - Creates PostgreSQL login roles that do not already exist, using the supplied passwords.
+- `delete_users(p_admin_user, p_del_users)`
+  - Drops PostgreSQL roles that currently exist.
+- `grant_rights(p_admin_user, p_user_rights)`
+  - Grants database-level rights when the table target is `*`; otherwise grants table-level rights. PostgreSQL ignores the host field in each rights entry.
+- `render_default_sql(p_field_type, p_default_value)`
+  - Renders PostgreSQL defaults with proper escaping for string values.
+- `render_field_type(p_field_type, p_field_params)`
+  - Maps legacy types to PostgreSQL types, including `SERIAL`/`BIGSERIAL` for autoincrement fields.
+- `_connect(p_db_name)`
+  - Internal connector helper returning a `psycopg` connection for the requested database.
+
 ______________________________________________________________________
 
 ## Updating ReleaseNotes Instructions

{sqldbwrpr-5.1.0 → sqldbwrpr-5.2.0}/README.md

@@ -29,7 +29,7 @@ When both are supplied, `p_db_structure` takes precedence for backward compatibi
   - Import CSV data from files or in-memory rows, including single-volume and numbered multi-volume files.
   - Export full tables or custom SQL query results to CSV, including optional multi-volume exports.
 - **Database Support**: Includes MySQL and PostgreSQL wrappers with dialect-specific SQL rendering.
-- **User and Permission Management**: Create MySQL users and grant database rights.
+- **User and Permission Management**: Create and delete MySQL users or PostgreSQL login roles, and grant backend-specific database or table rights.
 - **Batch Processing**: Configure import batch sizes for larger CSV loads.
 
 ### Project Structure
@@ -47,6 +47,17 @@ When both are supplied, `p_db_structure` takes precedence for backward compatibi
 pip install SqlDbWrpr
 ```
 
+### Running the Tests
+
+The focused database tests use temporary MySQL 8.0 and PostgreSQL 16 Docker containers. Ensure Docker is running, then install the development dependencies and run the test suite:
+
+```powershell
+poetry install --with dev
+poetry run pytest
+```
+
+The test fixtures wait for each database to become ready and remove the temporary containers after the test session.
+
 ### Quick Start With A Legacy Structure
 
 ```python
@@ -131,6 +142,115 @@ db = PostgreSQL(
 db.import_csv("Users", p_csv_db=[("Username",), ("alice",)], p_vol_type="Single")
 ```
 
+## Function Reference
+
+This section documents the callable API in `src/sqldbwrpr/sqldbwrpr.py`.
+
+### `SchemaSourceError`
+
+- `SchemaSourceError(ValueError)`: Raised when no usable schema source is provided to initialize a wrapper (`p_db_structure`, `p_sqlalchemy_metadata`, or `p_sqlalchemy_base.metadata`).
+
+### `SQLDbWrpr` (base wrapper)
+
+- `__init__(p_host_name, p_user_name, p_password, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_db_port, p_ssl_ca, p_ssl_key, p_ssl_cert)`
+  - Initializes shared wrapper state, resolves schema source, configures import/export behavior, and precomputes char vs non-char field maps.
+  - Schema resolution order: explicit `p_db_structure` first, then SQLAlchemy metadata, else raises `SchemaSourceError`.
+- `close()`
+  - Closes the active DB connection if present.
+- `build_column_sql(p_field_name, p_field_type, p_field_params, p_field_comment)`
+  - Builds the column fragment for `CREATE TABLE` in the active dialect (base implementation is MySQL-oriented).
+  - Applies `AUTO_INCREMENT`, `UNSIGNED`, `NOT NULL`, `ZEROFILL`, defaults, and comments based on field parameters.
+- `build_index_sql(p_table_name, p_idx_name, p_idx_fields, p_unique=False)`
+  - Builds an index clause with sort directions from legacy index metadata.
+- `build_insert_sql(p_table_name, p_header, p_replace=False)`
+  - Builds an `INSERT` or `REPLACE` statement with backend parameter placeholders.
+- `param_placeholder()`
+  - Returns the DB-API parameter token used by execution (`%s`).
+- `quote_identifier(p_identifier)`
+  - Quotes SQL identifiers when a backend quote character is configured; otherwise returns plain identifier text.
+- `quote_identifier_list(p_identifiers)`
+  - Quotes and joins a list of identifiers using commas.
+- `render_default_sql(p_field_type, p_default_value)`
+  - Renders a `DEFAULT` clause, quoting string-like defaults for char/varchar types.
+- `render_field_type(p_field_type, p_field_params)`
+  - Renders legacy type arrays into SQL type declarations (e.g., `varchar(n)`, `decimal(p,s)`).
+- `create_db()`
+  - Base database recreation flow: drops an existing DB and creates/selects a fresh one.
+- `create_tables()`
+  - Validates legacy schema metadata and creates tables, keys, indexes, and constraints in dependency order.
+  - Splits table creation and post-create operations where backend rules require deferred statements.
+- `create_users(p_admin_user, p_new_users)`
+  - Creates MySQL users if they do not already exist.
+- `delete_users(p_admin_user, p_del_users)`
+  - Drops MySQL users that exist in `mysql.user`.
+- `_err_broken_rec(p_sql_str, p_csv_db_slice)`
+  - Retry helper used during imports to isolate/log failing rows and stop on first unrecoverable record.
+- `export_to_csv(p_csv_path, p_table_name, p_delimiter="|", p_strip_chars="", p__vol_size=0, p_sql_query="")`
+  - Exports table/query results to CSV in single-file or multi-volume mode.
+  - Multi-volume mode chunks output by record count and generates numbered files.
+- `get_db_field_types()`
+  - Populates per-table lists of character vs non-character columns for type-sensitive import handling.
+- `grant_rights(p_admin_user, p_user_rights)`
+  - Grants MySQL rights and corresponding grant-option privileges according to supplied rights tuples.
+- `import_csv(p_table_name, p_csv_file_name="", p_key="", p_header="", p_del_head=False, p_csv_db="", p_csv_corr_str_file_name="", p_vol_type="Multi", p_verbose=False, p_replace=False)`
+  - Imports CSV data from file(s) or in-memory rows.
+  - Supports single- and multi-volume input, optional header override/removal, correction-string preprocessing, type conversion, date normalization, and batched insert/replace.
+- `import_and_split_csv(p_split_struct, p_data, p_header="", p_insert_header=False, p_verbose=False, p_debug=False)`
+  - Splits an input dataset into multiple destination table payloads using declarative field mappings and transform commands, then imports each generated dataset.
+- `from_sqlalchemy_metadata(p_sqlalchemy_metadata)` (`@classmethod`)
+  - Converts SQLAlchemy `MetaData` (sorted tables) to the legacy `db_structure` dictionary.
+- `resolve_db_structure(p_db_structure=None, p_sqlalchemy_base=None, p_sqlalchemy_metadata=None)` (`@staticmethod`)
+  - Central schema source resolver used by constructors; raises `SchemaSourceError` when none are provided.
+- `_action_to_legacy_code(p_action)` (`@staticmethod`)
+  - Maps SQLAlchemy FK action text (e.g., `CASCADE`, `SET NULL`) to legacy action codes (`C`, `N`, etc.).
+- `_build_default_field_params()` (`@staticmethod`)
+  - Produces the baseline legacy field-parameter block used during SQLAlchemy conversion.
+- `_column_type_to_legacy(p_column)` (`@staticmethod`)
+  - Converts a SQLAlchemy column type object into the legacy type-array format.
+- `_column_to_legacy_field(p_column)` (`@staticmethod`)
+  - Converts one SQLAlchemy column into a legacy field definition, including nullability, autoincrement, defaults, and comments.
+- `_set_foreign_keys(p_table, p_table_structure)` (`@staticmethod`)
+  - Writes legacy foreign-key metadata onto converted table fields.
+- `_set_indexes(p_table, p_table_structure)` (`@staticmethod`)
+  - Writes legacy index metadata for converted table fields.
+- `_set_primary_key(p_table, p_table_structure)` (`@staticmethod`)
+  - Marks primary-key columns in converted legacy table definitions.
+- `_table_to_legacy_structure(p_table)` (`@staticmethod`)
+  - Converts an entire SQLAlchemy table into a legacy table structure and enriches it with keys/indexes.
+- `_print_err_msg(p_err, p_msg="")` (`@staticmethod`)
+  - Formats and prints database error details before termination paths.
+
+### `MySQL(SQLDbWrpr)`
+
+- `__init__(p_host_name, p_user_name, p_password, p_user_rights, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_admin_username, p_admin_user_password, p_db_port, **kwargs)`
+  - Opens a MySQL connection and cursor, optionally recreates database/tables, or selects the configured DB.
+  - Includes fallback logic to create and grant rights to missing users when valid admin credentials and rights data are supplied.
+
+### `PostgreSQL(SQLDbWrpr)`
+
+- `__init__(p_host_name, p_user_name, p_password, p_recreate_db, p_db_name, p_db_structure, p_sqlalchemy_base, p_sqlalchemy_metadata, p_batch_size, p_bar_len, p_msg_width, p_verbose, p_db_port, p_maintenance_db, **kwargs)`
+  - Configures PostgreSQL-specific behavior (quoted identifiers, non-inline indexes), opens connection, and optionally recreates DB/tables.
+- `build_column_sql(p_field_name, p_field_type, p_field_params, p_field_comment)`
+  - PostgreSQL-specific column SQL renderer; handles non-AI `NOT NULL` and defaults.
+- `build_index_sql(p_table_name, p_idx_name, p_idx_fields, p_unique=False)`
+  - Builds PostgreSQL `CREATE INDEX`/`CREATE UNIQUE INDEX` statements.
+- `build_insert_sql(p_table_name, p_header, p_replace=False)`
+  - Builds PostgreSQL `INSERT`; when `p_replace=True`, builds `ON CONFLICT` upsert/no-op behavior from primary-key metadata.
+- `create_db()`
+  - PostgreSQL DB recreation flow via `pg_database` lookup, optional forced drop, create, and reconnect.
+- `create_users(p_admin_user, p_new_users)`
+  - Creates PostgreSQL login roles that do not already exist, using the supplied passwords.
+- `delete_users(p_admin_user, p_del_users)`
+  - Drops PostgreSQL roles that currently exist.
+- `grant_rights(p_admin_user, p_user_rights)`
+  - Grants database-level rights when the table target is `*`; otherwise grants table-level rights. PostgreSQL ignores the host field in each rights entry.
+- `render_default_sql(p_field_type, p_default_value)`
+  - Renders PostgreSQL defaults with proper escaping for string values.
+- `render_field_type(p_field_type, p_field_params)`
+  - Maps legacy types to PostgreSQL types, including `SERIAL`/`BIGSERIAL` for autoincrement fields.
+- `_connect(p_db_name)`
+  - Internal connector helper returning a `psycopg` connection for the requested database.
+
 ______________________________________________________________________
 
 ## Updating ReleaseNotes Instructions

{sqldbwrpr-5.1.0 → sqldbwrpr-5.2.0}/pyproject.toml

@@ -6,7 +6,7 @@
 
 [project]
     name = 'SQLDbWrpr'
-    version = '5.1.0'
+    version = '5.2.0'
     description = 'Wrapper for importing CSV and Text files into MySQL and Postgress'
     authors = [
         { name = "Hendrik du Toit", email = "hendrik@brightedge.co.za" }

{sqldbwrpr-5.1.0 → sqldbwrpr-5.2.0}/src/sqldbwrpr/sqldbwrpr.py

@@ -17,13 +17,7 @@ import mysql.connector
 import psycopg
 from beetools import msg as bm
 from mysql.connector import errorcode
-
-# from pathlib import Path
-
-
-# _PROJ_DESC = __doc__.split("\n")[0]
-# _PROJ_PATH = Path(__file__)
-# _PROJ_NAME = _PROJ_PATH.stem
+from psycopg import sql as psycopg_sql
 
 
 class SchemaSourceError(ValueError):
@@ -52,7 +46,7 @@ class SQLDbWrpr:
         p_ssl_key=None,
         p_ssl_cert=None,
     ):
-        """Initialize common wrapper state and resolve the database schema.
+        """Initialise a common wrapper state and resolve the database schema.
 
         A legacy `p_db_structure` dictionary takes precedence. If it is not
         supplied, SQLAlchemy metadata is read from `p_sqlalchemy_metadata` or
@@ -101,6 +95,7 @@ class SQLDbWrpr:
         self.inline_indexes = True
         self.msg_width = p_msg_width
         self.non_char_fields = {}
+        self.owns_connection = True
         self._password = p_password
         self.re_create_db = p_recreate_db
         self.silent = p_verbose
@@ -110,10 +105,16 @@ class SQLDbWrpr:
         self.get_db_field_types()
         self.db_port = p_db_port
 
+    def _execute_import_rows(self, p_table_name, p_header, p_rows, p_replace):
+        """Execute one prepared import batch without managing its transaction."""
+        self.cur.executemany(self.build_insert_sql(p_table_name, p_header, p_replace), p_rows)
+
     def close(self):
-        """Close the active database connection."""
-        if self.conn:
+        """Close owned connections, or just this wrapper's cursor on a borrowed connection."""
+        if self.conn and self.owns_connection:
             self.conn.close()
+        elif self.cur:
+            self.cur.close()
 
     def build_column_sql(self, p_field_name, p_field_type, p_field_params, p_field_comment):
         """Build SQL for one column in a CREATE TABLE statement."""
@@ -548,40 +549,28 @@ class SQLDbWrpr:
         return success
 
     def create_users(self, p_admin_user, p_new_users):
-        """Create MySQL users that do not already exist."""
-        c_user_name = 0
-        self.cur.execute("SELECT User, Host FROM mysql.user")
-        curr_users = self.cur.fetchall()
-        for user in p_new_users:
-            if not user[c_user_name] in curr_users:
-                try:
-                    self.cur.execute(
-                        "CREATE USER IF NOT EXISTS '{}'@'{}' IDENTIFIED BY '{}'".format(
-                            user[0], self.host_name, user[1]
-                        )
-                    )
-                except self.db_error as err:
-                    self._print_err_msg(err, "Could not create user")
-                    self.close()
-                    sys.exit()
-            self.conn.commit()
-        self.success = True
+        """Create database users.
+
+        Concrete database backends must implement this because user management
+        SQL differs between databases.
+        """
+        raise NotImplementedError(f"{self.__class__.__name__} must implement create_users()")
 
     def delete_users(self, p_admin_user, p_del_users):
-        c_user_name = 0
-        # c_password = 1
-        c_host = 2
-        self.cur.execute("SELECT User FROM mysql.user")
-        curr_users = [x[0] for x in self.cur.fetchall()]
-        for user in p_del_users:
-            if user[c_user_name] in curr_users:
-                try:
-                    self.cur.execute(f"DROP USER '{user[c_user_name]}'@'{user[c_host]}'")
-                except self.db_error as err:
-                    self._print_err_msg(err, "Could not delete user")
-                    self.close()
-                    sys.exit()
-        self.success = True
+        """Delete database users.
+
+        Concrete database backends must implement this because user management
+        SQL differs between databases.
+        """
+        raise NotImplementedError(f"{self.__class__.__name__} must implement delete_users()")
+
+    def grant_rights(self, p_admin_user, p_user_rights):
+        """Grant database rights.
+
+        Concrete database backends must implement this because privilege syntax
+        differs between databases.
+        """
+        raise NotImplementedError(f"{self.__class__.__name__} must implement grant_rights()")
 
     def _err_broken_rec(self, p_sql_str, p_csv_db_slice):
         """Write broken record to logger"""
@@ -609,11 +598,11 @@ class SQLDbWrpr:
         """Export a table to a csv file
 
         Parameters
-        - p_csv_path         - Path name of the file to be exported
-        - p_table_name = ''  - Table name to export
-        - p_delimeter = '|'  - Field delimiter to use
+        - p_csv_path - Path name of the file to be exported
+        - p_table_name = '' - Table name to export
+        - p_delimeter = '|' - Field delimiter to use
         - p_strip_chars = '' - characters to strip from text
-        - p__vol_size = 0    - Create a multiple volume export. p__vol_size is
+        - p__vol_size = 0 - Create a multiple-volume export. p__vol_size is
                              the number of records per file.  0 will create
                              only one volume.
         """
@@ -768,41 +757,6 @@ class SQLDbWrpr:
                 else:
                     self.non_char_fields[p_table_name].append(field)
 
-    def grant_rights(self, p_admin_user, p_user_rights):
-        """Grant configured MySQL rights to users."""
-        c_user_name = 0
-        # c_password = 1
-        c_host = 1
-        c_db = 2
-        c_table = 3
-        c_rights = 4
-        # success = True
-        for right in p_user_rights:
-            try:
-                sql_str = "GRANT {} ON {}.{} TO '{}'@'{}'".format(
-                    ",".join(right[c_rights:]),
-                    right[c_db],
-                    right[c_table],
-                    right[c_user_name],
-                    right[c_host],
-                )
-                self.cur.execute(sql_str)
-                self.conn.commit()
-                sql_str = "GRANT {} ON {}.{} TO '{}'@'{}' WITH GRANT OPTION".format(
-                    ",".join(right[c_rights:]),
-                    right[c_db],
-                    right[c_table],
-                    right[c_user_name],
-                    right[c_host],
-                )
-                self.cur.execute(sql_str)
-                self.conn.commit()
-            except self.db_error as err:
-                self._print_err_msg(err)
-                self.close()
-                sys.exit()
-        self.success = True
-
     def import_csv(
         self,
         p_table_name,
@@ -815,6 +769,7 @@ class SQLDbWrpr:
         p_vol_type="Multi",
         p_verbose=False,
         p_replace=False,
+        p_manage_transaction=True,
     ):
         """Import CSV rows into a database table.
 
@@ -843,7 +798,11 @@ class SQLDbWrpr:
         - p_replace = False
           - False - INSERT into database
           - True - REPLACE into database
+        - p_manage_transaction = True
+          - False leaves commits and rollbacks to the caller and propagates database errors.
         """
+        if not p_manage_transaction and self.conn.autocommit:
+            raise ValueError("Caller-managed imports require autocommit=False")
 
         def import_volume(p_csv_db, p_header, p_verbose):
             """Prepare and write one in-memory CSV volume to the active table."""
@@ -958,6 +917,12 @@ class SQLDbWrpr:
 
             def write_to_table(p_csv_db):
                 """Write prepared rows to the destination table."""
+                if not p_manage_transaction:
+                    for start in range(1, len(p_csv_db), self.batch_size):
+                        self._execute_import_rows(
+                            p_table_name, header, p_csv_db[start : start + self.batch_size], p_replace
+                        )
+                    return
                 i = 1
                 j = 0  # In case batch size is more than all records
                 list_len = len(p_csv_db)
@@ -1085,6 +1050,7 @@ class SQLDbWrpr:
         p_insert_header=False,
         p_verbose=False,
         p_debug=False,
+        p_manage_transaction=True,
     ):
         """Split CSV rows into one or more destination table imports.
 
@@ -1247,6 +1213,7 @@ class SQLDbWrpr:
                 p_header=new_header,
                 p_verbose=p_verbose,
                 p_replace=p_split_struct[seq]["Replace"],
+                p_manage_transaction=p_manage_transaction,
             )
 
     @classmethod
@@ -1532,6 +1499,77 @@ class MySQL(SQLDbWrpr):
         self.success = True
         pass
 
+    def create_users(self, p_admin_user, p_new_users):
+        """Create MySQL users that do not already exist."""
+        c_user_name = 0
+        self.cur.execute("SELECT User, Host FROM mysql.user")
+        curr_users = self.cur.fetchall()
+        for user in p_new_users:
+            user_key = (user[c_user_name], self.host_name)
+            if user_key not in curr_users:
+                try:
+                    self.cur.execute(
+                        "CREATE USER IF NOT EXISTS '{}'@'{}' IDENTIFIED BY '{}'".format(
+                            user[0], self.host_name, user[1]
+                        )
+                    )
+                except self.db_error as err:
+                    self._print_err_msg(err, "Could not create user")
+                    self.close()
+                    sys.exit()
+                self.conn.commit()
+        self.success = True
+
+    def delete_users(self, p_admin_user, p_del_users):
+        """Delete MySQL users that exist."""
+        c_user_name = 0
+        c_host = 2
+        self.cur.execute("SELECT CAST(User AS CHAR) FROM mysql.user")
+        curr_users = [x[0] for x in self.cur.fetchall()]
+        for user in p_del_users:
+            if user[c_user_name] in curr_users:
+                try:
+                    self.cur.execute(f"DROP USER '{user[c_user_name]}'@'{user[c_host]}'")
+                except self.db_error as err:
+                    self._print_err_msg(err, "Could not delete user")
+                    self.close()
+                    sys.exit()
+                self.conn.commit()
+        self.success = True
+
+    def grant_rights(self, p_admin_user, p_user_rights):
+        """Grant configured MySQL rights to users."""
+        c_user_name = 0
+        c_host = 1
+        c_db = 2
+        c_table = 3
+        c_rights = 4
+        for right in p_user_rights:
+            try:
+                sql_str = "GRANT {} ON {}.{} TO '{}'@'{}'".format(
+                    ",".join(right[c_rights:]),
+                    right[c_db],
+                    right[c_table],
+                    right[c_user_name],
+                    right[c_host],
+                )
+                self.cur.execute(sql_str)
+                self.conn.commit()
+                sql_str = "GRANT {} ON {}.{} TO '{}'@'{}' WITH GRANT OPTION".format(
+                    ",".join(right[c_rights:]),
+                    right[c_db],
+                    right[c_table],
+                    right[c_user_name],
+                    right[c_host],
+                )
+                self.cur.execute(sql_str)
+                self.conn.commit()
+            except self.db_error as err:
+                self._print_err_msg(err)
+                self.close()
+                sys.exit()
+        self.success = True
+
 
 class PostgreSQL(SQLDbWrpr):
     """Wrapper for PostgreSQL databases."""
@@ -1552,9 +1590,19 @@ class PostgreSQL(SQLDbWrpr):
         p_verbose=False,
         p_db_port="5432",
         p_maintenance_db="postgres",
+        p_connection=None,
         **kwargs,
     ):
-        """Create a PostgreSQL wrapper and optionally recreate the target database."""
+        """Create a PostgreSQL wrapper, optionally borrowing a caller-owned DB-API connection.
+
+        A borrowed connection is never closed or reconfigured. Use
+        p_manage_transaction=False on imports to keep its transaction caller-owned.
+        Database recreation is unavailable on borrowed connections.
+        """
+        if p_connection is not None and p_recreate_db:
+            raise ValueError("Cannot recreate a database on a borrowed connection")
+        if p_batch_size < 1:
+            raise ValueError("Batch size must be positive")
         super().__init__(
             p_host_name=p_host_name,
             p_user_name=p_user_name,
@@ -1576,14 +1624,27 @@ class PostgreSQL(SQLDbWrpr):
         self.maintenance_db = p_maintenance_db
         self._connection_kwargs = kwargs
         connect_db = self.maintenance_db if self.re_create_db else self.db_name
-        self.conn = self._connect(connect_db)
-        self.conn.autocommit = True
+        self.owns_connection = p_connection is None
+        self.conn = self._connect(connect_db) if self.owns_connection else p_connection
+        if self.owns_connection:
+            self.conn.autocommit = True
         self.cur = self.conn.cursor()
         if self.re_create_db:
             if self.create_db():
                 self.create_tables()
         self.success = True
 
+    def _execute_import_rows(self, p_table_name, p_header, p_rows, p_replace):
+        """Use batched VALUES for psycopg2; retain native executemany for other drivers."""
+        if type(self.conn).__module__.split(".")[0] == "psycopg2":
+            # Borrowing this driver implies the host already has psycopg2 installed.
+            from psycopg2.extras import execute_values
+
+            statement = self.build_insert_sql(p_table_name, p_header, p_replace, p_multi_row=True)
+            execute_values(self.cur, statement, p_rows, page_size=self.batch_size)
+        else:
+            super()._execute_import_rows(p_table_name, p_header, p_rows, p_replace)
+
     def build_column_sql(self, p_field_name, p_field_type, p_field_params, p_field_comment):
         """Build PostgreSQL SQL for one column in a CREATE TABLE statement."""
         sql_str = f"{self.quote_identifier(p_field_name)} {self.render_field_type(p_field_type, p_field_params)}"
@@ -1606,12 +1667,13 @@ class PostgreSQL(SQLDbWrpr):
             ",".join(index_fields),
         )
 
-    def build_insert_sql(self, p_table_name, p_header, p_replace=False):
+    def build_insert_sql(self, p_table_name, p_header, p_replace=False, p_multi_row=False):
         """Build a PostgreSQL INSERT or upsert statement."""
-        insert_sql = "INSERT INTO {} ({}) VALUES ({})".format(
+        values_sql = "%s" if p_multi_row else f"({','.join(self.param_placeholder() for _ in p_header)})"
+        insert_sql = "INSERT INTO {} ({}) VALUES {}".format(
             self.quote_identifier(p_table_name),
             ",".join([self.quote_identifier(str(x)) for x in p_header]),
-            ",".join([self.param_placeholder() for x in range(len(p_header))]),
+            values_sql,
         )
         if not p_replace:
             return insert_sql
@@ -1685,6 +1747,32 @@ class PostgreSQL(SQLDbWrpr):
             return f"DECIMAL({str(p_field_type[1])}, {str(p_field_type[2])})"
         return type_map.get(field_type, field_type.upper())
 
+    def update_rows(self, p_table_name, p_header, p_rows, p_key="id"):
+        """Update supplied columns in bounded batches; the caller owns commit and rollback.
+
+        p_header must contain p_key and at least one update column. Unlike an
+        upsert, this never creates missing records or supplies defaults.
+        """
+        if p_key not in p_header or len(set(p_header)) != len(p_header) or len(p_header) < 2:
+            raise ValueError("Updates require a unique header containing the key and update columns")
+        if self.conn.autocommit:
+            raise ValueError("Caller-managed updates require autocommit=False")
+        fields = [field for field in p_header if field != p_key]
+        indexes = [p_header.index(field) for field in [*fields, p_key]]
+        assignments = ",".join(f"{self.quote_identifier(field)} = %s" for field in fields)
+        statement = (
+            f"UPDATE {self.quote_identifier(p_table_name)} SET {assignments} "
+            f"WHERE {self.quote_identifier(p_key)} = %s"
+        )
+        for start in range(0, len(p_rows), self.batch_size):
+            rows = [tuple(row[index] for index in indexes) for row in p_rows[start : start + self.batch_size]]
+            if type(self.conn).__module__.split(".")[0] == "psycopg2":
+                from psycopg2.extras import execute_batch
+
+                execute_batch(self.cur, statement, rows, page_size=self.batch_size)
+            else:
+                self.cur.executemany(statement, rows)
+
     def _connect(self, p_db_name):
         """Open a PostgreSQL connection."""
         return psycopg.connect(
@@ -1695,3 +1783,81 @@ class PostgreSQL(SQLDbWrpr):
             port=self.db_port,
             **self._connection_kwargs,
         )
+
+    def create_users(self, p_admin_user, p_new_users):
+        """Create PostgreSQL login roles that do not already exist."""
+        c_user_name = 0
+        c_password = 1
+
+        self.cur.execute("SELECT rolname FROM pg_roles WHERE rolcanlogin = true")
+        curr_users = [row[0] for row in self.cur.fetchall()]
+
+        for user in p_new_users:
+            user_name = user[c_user_name]
+            password = user[c_password]
+
+            if user_name not in curr_users:
+                try:
+                    self.cur.execute(
+                        psycopg_sql.SQL("CREATE USER {} WITH PASSWORD {}").format(
+                            psycopg_sql.Identifier(user_name),
+                            psycopg_sql.Literal(password),
+                        )
+                    )
+                except self.db_error as err:
+                    self._print_err_msg(err, "Could not create user")
+                    self.close()
+                    sys.exit()
+        self.success = True
+
+    def delete_users(self, p_admin_user, p_del_users):
+        """Delete PostgreSQL users that exist."""
+        c_user_name = 0
+
+        self.cur.execute("SELECT rolname FROM pg_roles")
+        curr_users = [row[0] for row in self.cur.fetchall()]
+
+        for user in p_del_users:
+            user_name = user[c_user_name]
+
+            if user_name in curr_users:
+                try:
+                    self.cur.execute(f"DROP USER {self.quote_identifier(user_name)}")
+                except self.db_error as err:
+                    self._print_err_msg(err, "Could not delete user")
+                    self.close()
+                    sys.exit()
+        self.success = True
+
+    def grant_rights(self, p_admin_user, p_user_rights):
+        """Grant configured PostgreSQL rights to users.
+
+        Expected right format:
+        [user_name, host, database, table, right1, right2, ...]
+        The host value is ignored because PostgreSQL does not grant privileges
+        by user/host pair like MySQL.
+        """
+        c_user_name = 0
+        c_db = 2
+        c_table = 3
+        c_rights = 4
+
+        for right in p_user_rights:
+            try:
+                rights_sql = ",".join(right[c_rights:])
+                user_sql = self.quote_identifier(right[c_user_name])
+                db_sql = self.quote_identifier(right[c_db])
+                table_sql = self.quote_identifier(right[c_table])
+
+                if right[c_table] == "*":
+                    sql_str = f"GRANT {rights_sql} ON DATABASE {db_sql} TO {user_sql}"
+                else:
+                    sql_str = f"GRANT {rights_sql} ON TABLE {table_sql} TO {user_sql}"
+
+                self.cur.execute(sql_str)
+                self.conn.commit()
+            except self.db_error as err:
+                self._print_err_msg(err)
+                self.close()
+                sys.exit()
+        self.success = True