fullscript.spellbook 2.2.0__tar.gz → 3.0.1__tar.gz

fullscript_spellbook-3.0.1/AGENTS.md

@@ -0,0 +1,107 @@
+# Working on Spellbook
+
+Notes for anyone - person or coding agent - changing this package. This file covers the
+things that are not obvious from reading the code.
+
+## What this package is
+
+A thin, friendly wrapper over Snowflake, MySQL, Google Sheets, Google Drive and some
+statistics helpers, published publicly to PyPI as `fullscript.spellbook`.
+
+**It is a public package.** Never commit Fullscript hostnames, database names, RDS instance
+identifiers, usernames or internal URLs - not in code, examples, docs or tests. Use
+placeholders (`${MYSQL_HOST}`, `your_rds_instance_name`) instead.
+
+## Running the tests
+
+```bash
+pip install -e '.[test]'
+pytest
+```
+
+Tests mock the database and `subprocess`, so they need no network and no credentials.
+`tests/test_db_magic.py` covers connections and auth, `tests/test_utils.py` covers config
+loading. Testing against a real database needs AWS SSO, which no automated run can do.
+
+## Versioning
+
+The version is **not** in a file. `setuptools_scm` derives it from git tags, so releasing
+means tagging:
+
+```bash
+git tag 3.0.0 && git push origin main --tags
+```
+
+Pushing a tag triggers `.gitlab-ci.yml`, which publishes to both the internal registry and
+public PyPI. Do not tag unless you intend to publish.
+
+## Conventions worth keeping
+
+**Errors raise, they do not return empty.** Every failure path raises a subclass of
+`SpellbookError` from `spellbook/exceptions.py`. Returning `pd.DataFrame()` or `None` on
+failure is the thing this package deliberately moved away from: a caller cannot tell
+"matched no rows" from "never ran", which produces confidently wrong analysis. An empty
+DataFrame must only ever mean zero rows.
+
+**Decorative output goes to stderr, never stdout.** The wizard ASCII art is charm, not
+data. `utils.show_wizard()` honours `SPELLBOOK_QUIET=1`; set that when calling Spellbook
+from a script or agent. Use `logging` for diagnostics rather than `print`.
+
+**Function signatures are `(db_name, query)`, in that order.** `execute_mysql` once took
+`(query, dbname)`, which silently broke every `run_sql` call against MySQL. Keep the order
+and the `db_name` spelling consistent across every function.
+
+**Bind parameters in catalogue queries.** `list_tables` and `describe_table` pass values
+through SQLAlchemy parameters so a `%` or a quote in user input cannot reshape the query.
+`preview_table` cannot bind a table name, so it validates it against a strict identifier
+pattern instead.
+
+## Traps that have cost time before
+
+**Validate the database being used, not the whole config.** The entire `databases` section
+is re-parsed on every lookup, so any check that runs during parsing applies to databases the
+caller never asked for. Release 3.0.0 raised on the first unset `${VAR}` anywhere in the
+section, which meant an unrelated Snowflake entry with no key passphrase blocked every MySQL
+query. `parse_env_variables` now substitutes permissively and records unresolved names under
+`_unresolved_env`; `db_magic._validate_db_config` turns those into an error for the selected
+database only, and only for settings in `REQUIRED_FIELDS`. Settings that are legitimately
+empty - a Snowflake passphrase, `ssl_ca`, `port`, `schema` - must stay out of that list.
+
+**Config file precedence.** `utils.find_config_file()` prefers `spellbook_config.yaml` in
+the *current working directory* over the one in your home directory. Running anything from
+this repository picks up the placeholder example config, not your real one. If a connection
+fails with odd credentials, check which file was loaded.
+
+**Two MySQL drivers.** Password auth uses `mysql+mysqlconnector`; AWS RDS IAM auth uses
+`mysql+pymysql`, because only pymysql completes the IAM token handshake. They are not
+interchangeable:
+
+- pymysql substitutes parameters whenever `args is not None`, and pandas passes an empty
+  tuple, so a literal `%` in `like '%value%'` is read as a format character and raises.
+  mysql-connector guards on `if params:` instead. `db_magic._iam_cursor_class()` normalises
+  empty parameters to `None` so both drivers behave the same. Do not remove it without a
+  test that runs a `like '%...%'` query.
+- An empty `ssl` dict makes TLS merely *preferred* in pymysql. The dict passed must be
+  truthy for TLS to be required.
+
+**RDS IAM specifics.** Tokens expire in about 15 minutes, so they are fetched per
+connection and never cached. `rx rds password` takes the *RDS instance* name, which usually
+differs from the config alias - hence the optional `rds_db_name` field. The token is URL
+shaped and must be `quote_plus`-escaped into the connection string.
+
+**`iam_auth` is internal only.** It shells out to `rx`, which is Fullscript internal
+tooling, so the feature cannot work for public users of this package. Keep password auth
+working and keep the docs honest about the limitation.
+
+## Repository layout
+
+| Path | Purpose |
+| --- | --- |
+| `spellbook/db_magic.py` | Snowflake and MySQL connections, queries, schema helpers |
+| `spellbook/exceptions.py` | The `SpellbookError` hierarchy |
+| `spellbook/utils.py` | Config loading, env substitution, wizards, SQL file loading |
+| `spellbook/sheet_magic.py` | Google Sheets |
+| `spellbook/drive_magic.py` | Google Drive |
+| `spellbook/stats_magic.py` | Statistical tests |
+| `spellbook_config.yaml` | **Example** config, published - placeholders only |
+| `documentation/` | User-facing docs per module |

fullscript_spellbook-3.0.1/CLAUDE.md

@@ -0,0 +1 @@
+See [AGENTS.md](AGENTS.md) for how to work on this package.

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fullscript.spellbook
-Version: 2.2.0
+Version: 3.0.1
 Summary: A simple package to make your life easier.
 Author-email: Meinhard Mare <meinhard.mare@fullscript.com>, Thomas Edwards <thomas.edwards@fullscript.com>, Gui Chagas <gui.chagas@fullscript.com>, David King <david.king@fullscript.com>, Karsten Melchers <karsten.melchers@fullscript.com>
 License: MIT

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/documentation/db_magic.md

@@ -10,6 +10,48 @@ A magical wrapper to perform work with databases.
 from spellbook import db_magic
 ```
 
+### Error Handling
+
+Every failure raises an exception. An empty DataFrame always means "the query matched no
+rows" and never "something went wrong", so results can be trusted without extra checks.
+
+```python
+from spellbook import db_magic
+from spellbook.exceptions import SpellbookError, QueryError, DatabaseNotFoundError
+
+try:
+    df = db_magic.get_data('reporting_db', 'select * from users limit 10')
+except DatabaseNotFoundError:
+    ...  # the name is not in spellbook_config.yaml
+except QueryError:
+    ...  # the database rejected the query
+except SpellbookError:
+    ...  # anything Spellbook raises
+```
+
+| Exception | Raised when |
+| --- | --- |
+| `SpellbookError` | Base class for all of the below |
+| `ConfigError` | The config file is missing, malformed, or references an unset `${VAR}` |
+| `DatabaseNotFoundError` | No database in the config matches the requested name |
+| `AuthenticationError` | Credentials could not be obtained or were rejected |
+| `ConnectionFailedError` | The database could not be reached |
+| `QueryError` | The query was rejected or failed while running |
+
+> **Upgrading from 2.x:** these functions used to print a message and return an empty
+> DataFrame or `None` on failure. Code that inspected `df.empty` to detect *errors* now
+> needs a `try`/`except`. Code that used `df.empty` to detect *zero rows* is unaffected.
+
+### Quiet Mode
+
+Spellbook prints a wizard for company on each query. It goes to stderr, never stdout, so it
+cannot contaminate results. Set `SPELLBOOK_QUIET=1` to turn it off - recommended for
+scripts, scheduled jobs and anything driven by an AI agent.
+
+```bash
+export SPELLBOOK_QUIET=1
+```
+
 ### MySQL Authentication
 
 MySQL connections support two authentication methods, chosen per database in `spellbook_config.yaml`.
@@ -159,3 +201,54 @@ db_magic.run_sql("mysql_db", update_query)
 
 print("SQL executed successfully!")
 ```
+
+### Exploring a database
+
+These helpers answer "what is in here?" without hand-writing `information_schema` queries.
+They are useful interactively, and they are what an AI agent needs before it can write a
+sensible query.
+
+##### list_databases()
+
+Lists the databases in your configuration file, with no credentials included.
+
+```python
+db_magic.list_databases()
+#            name     type        database  iam_auth
+# 0  reporting_db    mysql  mysql_database      True
+```
+
+##### list_tables(db_name, pattern=None)
+
+Lists the tables in a database, optionally filtered by a case-insensitive substring.
+
+```python
+db_magic.list_tables("mysql_db")
+db_magic.list_tables("mysql_db", pattern="order")
+```
+
+The pattern is passed as a bound parameter, so `%` and quotes in it are treated as literal
+text rather than changing the query.
+
+##### describe_table(db_name, table)
+
+Returns one row per column, with its type, nullability and default. Raises `QueryError` if
+the table does not exist.
+
+```python
+db_magic.describe_table("mysql_db", "users")
+#   column_name data_type is_nullable column_default
+# 0          id    bigint          NO           None
+# 1        name   varchar         YES           None
+```
+
+##### preview_table(db_name, table, limit=10)
+
+Reads the first few rows of a table.
+
+```python
+db_magic.preview_table("mysql_db", "users", limit=5)
+```
+
+`table` must be a plain identifier - letters, digits, underscores, dots or dollar signs -
+because a table name cannot be passed as a bound parameter.

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/fullscript.spellbook.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fullscript.spellbook
-Version: 2.2.0
+Version: 3.0.1
 Summary: A simple package to make your life easier.
 Author-email: Meinhard Mare <meinhard.mare@fullscript.com>, Thomas Edwards <thomas.edwards@fullscript.com>, Gui Chagas <gui.chagas@fullscript.com>, David King <david.king@fullscript.com>, Karsten Melchers <karsten.melchers@fullscript.com>
 License: MIT

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/fullscript.spellbook.egg-info/SOURCES.txt

@@ -1,6 +1,8 @@
 .gitignore
 .gitlab-ci.yml
 .python-version
+AGENTS.md
+CLAUDE.md
 LICENSE
 MANIFEST.in
 README.md
@@ -22,6 +24,8 @@ fullscript.spellbook.egg-info/top_level.txt
 spellbook/__init__.py
 spellbook/db_magic.py
 spellbook/drive_magic.py
+spellbook/exceptions.py
+spellbook/py.typed
 spellbook/sheet_magic.py
 spellbook/stats_magic.py
 spellbook/utils.py
@@ -38,4 +42,5 @@ spellbook/wizards/wizard6.txt
 spellbook/wizards/wizard7.txt
 spellbook/wizards/wizard8.txt
 spellbook/wizards/wizard9.txt
-tests/test_db_magic.py
+tests/test_db_magic.py
+tests/test_utils.py

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/fullscript.spellbook.egg-info/scm_file_list.json

@@ -3,6 +3,8 @@
     ".gitignore",
     ".gitlab-ci.yml",
     ".python-version",
+    "AGENTS.md",
+    "CLAUDE.md",
     "LICENSE",
     "MANIFEST.in",
     "README.md",
@@ -16,6 +18,8 @@
     "spellbook/__init__.py",
     "spellbook/db_magic.py",
     "spellbook/drive_magic.py",
+    "spellbook/exceptions.py",
+    "spellbook/py.typed",
     "spellbook/sheet_magic.py",
     "spellbook/stats_magic.py",
     "spellbook/utils.py",
@@ -33,6 +37,7 @@
     "spellbook/wizards/wizard8.txt",
     "spellbook/wizards/wizard9.txt",
     "spellbook_config.yaml",
-    "tests/test_db_magic.py"
+    "tests/test_db_magic.py",
+    "tests/test_utils.py"
   ]
 }

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/fullscript.spellbook.egg-info/scm_version.json

@@ -1,7 +1,7 @@
 {
-  "tag": "2.2.0",
+  "tag": "3.0.1",
   "distance": 0,
-  "node": "gb79ed51cfa3c9db792124b375e8edb615250590b",
+  "node": "g4c7e282904d3f4b9436cecd6265dbc228d36c8de",
   "dirty": false,
   "branch": "HEAD",
   "node_date": "2026-10-22"

{fullscript_spellbook-2.2.0 → fullscript_spellbook-3.0.1}/pyproject.toml

@@ -146,6 +146,6 @@ where = ["."]
 include = ["spellbook"]
 
 [tool.setuptools.package-data]
-"spellbook" = ["wizards/*.txt"]
+"spellbook" = ["wizards/*.txt", "py.typed"]
 
 [tool.setuptools_scm]

fullscript_spellbook-3.0.1/spellbook/__init__.py

@@ -0,0 +1,19 @@
+"""Spellbook: connect to databases, spreadsheets and Drive, and run analyses."""
+
+from spellbook.exceptions import (
+    AuthenticationError,
+    ConfigError,
+    ConnectionFailedError,
+    DatabaseNotFoundError,
+    QueryError,
+    SpellbookError,
+)
+
+__all__ = [
+    'SpellbookError',
+    'ConfigError',
+    'DatabaseNotFoundError',
+    'AuthenticationError',
+    'ConnectionFailedError',
+    'QueryError',
+]