gitdata-lib 0.0.17__tar.gz → 0.0.19__tar.gz

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/CHANGELOG.md

@@ -6,6 +6,19 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
 
 ## [Unreleased]
 
+## [v0.0.19] - 2026-09-28
+- add direct `gitdata ls` and `gitdata scan` support for Oracle services and tables
+- show actionable CLI errors for Oracle connection, authentication, service, table, and query failures
+- redact inline URL passwords from debug output and escape terminal control characters in CLI listings
+- show actionable CLI errors for MariaDB/MySQL connection, authentication, and database-selection failures instead of tracebacks
+
+## [v0.0.18] - 2026-09-28
+- support and test Python 3.9 through 3.12
+- replace removed `collections` aliases with `collections.abc`
+- update PostgreSQL and CLI dependencies for Python 3.12
+- register explicit SQLite date and timestamp adapters and converters
+- run pytest-based CI and local test workflows across supported Python versions and database integrations
+
 ## [v0.0.17] - 2026-09-28
 - add direct `gitdata ls` and streaming `gitdata scan` for CSV, SQLite, and MariaDB refs without digestion or storage
 - allow `gitdata ls` on a MariaDB server ref to list databases visible to the user

gitdata_lib-0.0.19/PKG-INFO

@@ -0,0 +1,248 @@
+Metadata-Version: 2.4
+Name: gitdata-lib
+Version: 0.0.19
+Summary: Data extraction and analysis library
+Home-page: https://github.com/gitdata/gitdata-lib
+Author: DSI Labs
+Author-email: support@gitdata.com
+Classifier: Development Status :: 1 - Planning
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Database :: Front-Ends
+Requires-Python: >=3.9,<3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: python-decouple>=3.4
+Requires-Dist: Unipath>=1.1
+Requires-Dist: PyMySQL==0.10.1
+Requires-Dist: psycopg2-binary==2.9.10
+Requires-Dist: python-dotenv
+Requires-Dist: requests
+Requires-Dist: docopt-ng<1,>=0.9
+Requires-Dist: faker
+Requires-Dist: cryptography
+Provides-Extra: oracle
+Requires-Dist: oracledb<4,>=1.3; extra == "oracle"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# GitData
+
+GitData gives you one set of commands for exploring files and databases.
+List what is inside a source with `gitdata ls`, then preview its data with
+`gitdata scan`.
+
+## Install
+
+GitData supports Python 3.9 through 3.12.
+
+```bash
+python3 -m pip install gitdata-lib
+```
+
+Oracle support is optional:
+
+```bash
+python3 -m pip install 'gitdata-lib[oracle]'
+```
+
+## Explore a data source
+
+The same two commands work across CSV files, SQLite databases,
+MariaDB/MySQL databases, and Oracle databases:
+
+```bash
+gitdata ls <source>
+gitdata scan <source>
+```
+
+Use `-l` for a detailed listing, `-a` to include system objects, and `--limit`
+to choose how many rows to preview.
+
+## CSV
+
+List the columns in a CSV file:
+
+```bash
+gitdata ls examples/locations-headered.csv
+```
+
+Preview its rows:
+
+```bash
+gitdata scan --limit 10 examples/locations-headered.csv
+```
+
+## SQLite
+
+Start with the database file, then move into a table or column:
+
+```bash
+# List tables
+gitdata ls examples/inspection.sqlite
+
+# Show table details
+gitdata ls -l examples/inspection.sqlite/customers
+
+# Preview rows
+gitdata scan --limit 5 examples/inspection.sqlite/customers
+
+# Examine one column
+gitdata scan --limit 5 examples/inspection.sqlite/customers.email
+```
+
+SQLite references follow this shape:
+
+```text
+path/to/database.sqlite
+path/to/database.sqlite/table
+path/to/database.sqlite/table.column
+```
+
+## MariaDB and MySQL
+
+Keep passwords in environment variables instead of putting them in database
+references:
+
+```bash
+export GITDATA_DB_PASSWORD='your-password'
+
+SERVER='mysql://user@db.example.com?password_env=GITDATA_DB_PASSWORD'
+DATABASE='mysql://user@db.example.com/sales?password_env=GITDATA_DB_PASSWORD'
+TABLE='mysql://user@db.example.com/sales.orders?password_env=GITDATA_DB_PASSWORD'
+```
+
+Explore the server, database, and table with the same commands:
+
+```bash
+# List databases
+gitdata ls "$SERVER"
+
+# List tables
+gitdata ls "$DATABASE"
+
+# Show columns and indexes
+gitdata ls -l "$TABLE"
+
+# Preview rows
+gitdata scan --limit 10 "$TABLE"
+```
+
+MariaDB/MySQL references follow this shape:
+
+```text
+mysql://user@host
+mysql://user@host/database
+mysql://user@host/database.table
+```
+
+If the username is omitted, GitData uses the current operating-system username.
+Use `gitdata ls -a` when you also want to see system databases or tables.
+
+### Try the included MariaDB example
+
+The repository includes `examples/mariadb.sql`. Load it into a disposable
+MariaDB container:
+
+```bash
+docker run --name gitdata-example-mariadb \
+  -e MARIADB_ROOT_PASSWORD=example \
+  -p 3307:3306 \
+  -d mariadb:10.7
+
+until docker exec gitdata-example-mariadb \
+  mariadb-admin ping -h 127.0.0.1 -uroot -pexample --silent
+do
+  sleep 1
+done
+
+docker exec -i gitdata-example-mariadb \
+  mariadb -uroot -pexample < examples/mariadb.sql
+
+export GITDATA_EXAMPLE_PASSWORD=example
+gitdata ls 'mysql://root@127.0.0.1:3307/gitdata_example?password_env=GITDATA_EXAMPLE_PASSWORD'
+gitdata scan --limit 5 'mysql://root@127.0.0.1:3307/gitdata_example.customers?password_env=GITDATA_EXAMPLE_PASSWORD'
+```
+
+Remove the example when finished:
+
+```bash
+docker rm -f gitdata-example-mariadb
+```
+
+## Command reference
+
+```text
+gitdata ls <source>          list databases, tables, or columns
+gitdata ls -l <source>       show a detailed listing
+gitdata ls -a <source>       include system objects
+gitdata scan <source>        preview and profile data
+gitdata scan --limit N ...   preview at most N rows
+```
+
+Run a command with `--help` for all options:
+
+```bash
+gitdata ls --help
+gitdata scan --help
+```
+
+## Oracle
+
+Install the optional Oracle driver, then keep the password in an environment
+variable:
+
+```bash
+python3 -m pip install 'gitdata-lib[oracle]'
+export ORACLE_PASSWORD='your-password'
+
+DATABASE='oracle://app@db.example.com:1521/FREEPDB1?password_env=ORACLE_PASSWORD'
+TABLE="${DATABASE}#orders"
+```
+
+Explore the service and its tables with the same commands:
+
+```bash
+# List tables owned by the connected user
+gitdata ls "$DATABASE"
+
+# Show table counts and sizes
+gitdata ls -l "$DATABASE"
+
+# Show column details
+gitdata ls -l "$TABLE"
+
+# Preview rows
+gitdata scan --limit 10 "$TABLE"
+```
+
+Oracle references follow this shape:
+
+```text
+oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE
+oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE#table
+```
+
+Inline passwords are not accepted. Ordinary Oracle table and column names are
+shown in lowercase. Quoted identifiers are shown with their double quotes so
+names that differ only by Oracle quoting remain distinct; use the displayed
+spelling after `#` when selecting one (the quotes may be URL-encoded as `%22`).

gitdata_lib-0.0.19/README.md

@@ -0,0 +1,201 @@
+# GitData
+
+GitData gives you one set of commands for exploring files and databases.
+List what is inside a source with `gitdata ls`, then preview its data with
+`gitdata scan`.
+
+## Install
+
+GitData supports Python 3.9 through 3.12.
+
+```bash
+python3 -m pip install gitdata-lib
+```
+
+Oracle support is optional:
+
+```bash
+python3 -m pip install 'gitdata-lib[oracle]'
+```
+
+## Explore a data source
+
+The same two commands work across CSV files, SQLite databases,
+MariaDB/MySQL databases, and Oracle databases:
+
+```bash
+gitdata ls <source>
+gitdata scan <source>
+```
+
+Use `-l` for a detailed listing, `-a` to include system objects, and `--limit`
+to choose how many rows to preview.
+
+## CSV
+
+List the columns in a CSV file:
+
+```bash
+gitdata ls examples/locations-headered.csv
+```
+
+Preview its rows:
+
+```bash
+gitdata scan --limit 10 examples/locations-headered.csv
+```
+
+## SQLite
+
+Start with the database file, then move into a table or column:
+
+```bash
+# List tables
+gitdata ls examples/inspection.sqlite
+
+# Show table details
+gitdata ls -l examples/inspection.sqlite/customers
+
+# Preview rows
+gitdata scan --limit 5 examples/inspection.sqlite/customers
+
+# Examine one column
+gitdata scan --limit 5 examples/inspection.sqlite/customers.email
+```
+
+SQLite references follow this shape:
+
+```text
+path/to/database.sqlite
+path/to/database.sqlite/table
+path/to/database.sqlite/table.column
+```
+
+## MariaDB and MySQL
+
+Keep passwords in environment variables instead of putting them in database
+references:
+
+```bash
+export GITDATA_DB_PASSWORD='your-password'
+
+SERVER='mysql://user@db.example.com?password_env=GITDATA_DB_PASSWORD'
+DATABASE='mysql://user@db.example.com/sales?password_env=GITDATA_DB_PASSWORD'
+TABLE='mysql://user@db.example.com/sales.orders?password_env=GITDATA_DB_PASSWORD'
+```
+
+Explore the server, database, and table with the same commands:
+
+```bash
+# List databases
+gitdata ls "$SERVER"
+
+# List tables
+gitdata ls "$DATABASE"
+
+# Show columns and indexes
+gitdata ls -l "$TABLE"
+
+# Preview rows
+gitdata scan --limit 10 "$TABLE"
+```
+
+MariaDB/MySQL references follow this shape:
+
+```text
+mysql://user@host
+mysql://user@host/database
+mysql://user@host/database.table
+```
+
+If the username is omitted, GitData uses the current operating-system username.
+Use `gitdata ls -a` when you also want to see system databases or tables.
+
+### Try the included MariaDB example
+
+The repository includes `examples/mariadb.sql`. Load it into a disposable
+MariaDB container:
+
+```bash
+docker run --name gitdata-example-mariadb \
+  -e MARIADB_ROOT_PASSWORD=example \
+  -p 3307:3306 \
+  -d mariadb:10.7
+
+until docker exec gitdata-example-mariadb \
+  mariadb-admin ping -h 127.0.0.1 -uroot -pexample --silent
+do
+  sleep 1
+done
+
+docker exec -i gitdata-example-mariadb \
+  mariadb -uroot -pexample < examples/mariadb.sql
+
+export GITDATA_EXAMPLE_PASSWORD=example
+gitdata ls 'mysql://root@127.0.0.1:3307/gitdata_example?password_env=GITDATA_EXAMPLE_PASSWORD'
+gitdata scan --limit 5 'mysql://root@127.0.0.1:3307/gitdata_example.customers?password_env=GITDATA_EXAMPLE_PASSWORD'
+```
+
+Remove the example when finished:
+
+```bash
+docker rm -f gitdata-example-mariadb
+```
+
+## Command reference
+
+```text
+gitdata ls <source>          list databases, tables, or columns
+gitdata ls -l <source>       show a detailed listing
+gitdata ls -a <source>       include system objects
+gitdata scan <source>        preview and profile data
+gitdata scan --limit N ...   preview at most N rows
+```
+
+Run a command with `--help` for all options:
+
+```bash
+gitdata ls --help
+gitdata scan --help
+```
+
+## Oracle
+
+Install the optional Oracle driver, then keep the password in an environment
+variable:
+
+```bash
+python3 -m pip install 'gitdata-lib[oracle]'
+export ORACLE_PASSWORD='your-password'
+
+DATABASE='oracle://app@db.example.com:1521/FREEPDB1?password_env=ORACLE_PASSWORD'
+TABLE="${DATABASE}#orders"
+```
+
+Explore the service and its tables with the same commands:
+
+```bash
+# List tables owned by the connected user
+gitdata ls "$DATABASE"
+
+# Show table counts and sizes
+gitdata ls -l "$DATABASE"
+
+# Show column details
+gitdata ls -l "$TABLE"
+
+# Preview rows
+gitdata scan --limit 10 "$TABLE"
+```
+
+Oracle references follow this shape:
+
+```text
+oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE
+oracle://user@host:port/service?password_env=ENVIRONMENT_VARIABLE#table
+```
+
+Inline passwords are not accepted. Ordinary Oracle table and column names are
+shown in lowercase. Quoted identifiers are shown with their double quotes so
+names that differ only by Oracle quoting remain distinct; use the displayed
+spelling after `#` when selecting one (the quotes may be URL-encoded as `%22`).

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/gitdata/__init__.py

@@ -1,4 +1,4 @@
-"""
+r"""
    ____ _ _   ____        _
   / ___(_) |_|  _ \  __ _| |_ __ _
  | |  _| | __| | | |/ _` | __/ _` |

gitdata_lib-0.0.19/gitdata/__version__.py

@@ -0,0 +1 @@
+__version__ = '0.0.19'

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/gitdata/cli/__init__.py

@@ -24,12 +24,62 @@ import sys
 from docopt import docopt
 
 import gitdata
+from gitdata.connectors.common import ConnectorError
 from gitdata.utils import trim
 
 
 root_logger = logging.getLogger()
 
 
+def _redact_inline_url_password(value):
+    scheme_end = value.find('://')
+    if scheme_end < 1:
+        return value
+    authority_start = scheme_end + 3
+    separators = [
+        position for position in (
+            value.find('/', authority_start),
+            value.find('?', authority_start),
+            value.find('#', authority_start),
+        )
+        if position >= 0
+    ]
+    authority_end = min(separators) if separators else len(value)
+    authority = value[authority_start:authority_end]
+    at = authority.rfind('@')
+    colon = authority.find(':')
+    if at < 0 or colon < 0 or colon > at:
+        return value
+    redacted = authority[:colon + 1] + '***' + authority[at:]
+    return value[:authority_start] + redacted + value[authority_end:]
+
+
+def _redact_debug_value(value):
+    """Remove URL passwords before rendering parsed debug arguments."""
+    if isinstance(value, str):
+        return _redact_inline_url_password(value)
+    if isinstance(value, list):
+        return [_redact_debug_value(item) for item in value]
+    if isinstance(value, tuple):
+        return tuple(_redact_debug_value(item) for item in value)
+    if isinstance(value, dict):
+        return {
+            key: _redact_debug_value(item)
+            for key, item in value.items()
+        }
+    return value
+
+
+def terminal_text(value):
+    """Render text without terminal control characters."""
+    return ''.join(
+        character
+        if character.isprintable()
+        else character.encode('unicode_escape').decode('ascii')
+        for character in str(value)
+    )
+
+
 def print_help(doc):
     """Print help text"""
     print(trim(doc))
@@ -53,6 +103,9 @@ def main():
     except KeyboardInterrupt:
         print('\nInterrupted.', file=sys.stderr)
         raise SystemExit(130)
+    except ConnectorError as error:
+        print('gitdata: {}'.format(error), file=sys.stderr)
+        raise SystemExit(1)
 
 
 def _main():
@@ -69,7 +122,7 @@ def _main():
     )
 
     if args['--debug']:
-        print(args)
+        print(_redact_debug_value(args))
         root_logger.setLevel(logging.DEBUG)
 
     argv = [args['<command>']] + args['<args>']

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/gitdata/cli/gitdata_ls.py

@@ -7,6 +7,7 @@ options:
     -h, --help      show help
 """
 
+from gitdata.cli import terminal_text
 from gitdata.connectors import fetch
 from gitdata.inspection import list_items
 
@@ -35,7 +36,7 @@ def _display(value):
         return '-'
     if isinstance(value, int):
         return '{:,}'.format(value)
-    return str(value)
+    return terminal_text(value)
 
 
 def _row(values, widths):
@@ -81,4 +82,4 @@ def ls_to_console(args):
         _print_verbose(root, items)
         return
     for item in items:
-        print(item['name'])
+        print(terminal_text(item['name']))

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/gitdata/cli/gitdata_scan.py

@@ -12,6 +12,7 @@ from decimal import Decimal
 import shutil
 import sys
 
+from gitdata.cli import terminal_text
 from gitdata.connectors import fetch
 from gitdata.inspection import scan
 
@@ -31,7 +32,12 @@ def _title(report):
     )
     row_label = 'row' if row_label_count == 1 else 'rows'
     return '{}: {:,} {}, {} {}{}'.format(
-        report['name'], columns, column_label, rows, row_label, sampled
+        terminal_text(report['name']),
+        columns,
+        column_label,
+        rows,
+        row_label,
+        sampled,
     )
 
 
@@ -48,6 +54,7 @@ def _format_value(value):
         text = '{:%Y-%m-%d}'.format(value)
     else:
         text = str(value)
+    text = terminal_text(text)
     if len(text) > 12:
         return text[:12] + '...'
     return text
@@ -81,7 +88,7 @@ def _column_widths():
 
 
 def _cell(value, width, right=False):
-    text = str(value)
+    text = terminal_text(value)
     if len(text) > width:
         half = round(width / 2)
         text = (text[:half - 2] + ' ... ' + text[-half + 3:])[:width]

{gitdata_lib-0.0.17 → gitdata_lib-0.0.19}/gitdata/connectors/common.py

@@ -98,6 +98,10 @@ def explore(node, destination):
 #                 yield a, b
 
 
+class ConnectorError(Exception):
+    """Raised when a Connector cannot complete an expected operation."""
+
+
 class UnsupportedRef(ValueError):
     """Raised when no Connector can resolve a ref."""