tabletalk 0.4.0__tar.gz → 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {tabletalk-0.4.0 → tabletalk-0.6.0}/.github/workflows/db_provider_tests.yml +15 -2
- {tabletalk-0.4.0 → tabletalk-0.6.0}/.github/workflows/publish.yml +13 -5
- {tabletalk-0.4.0 → tabletalk-0.6.0}/.gitignore +9 -0
- tabletalk-0.6.0/COMMERCIAL-LICENSE.md +12 -0
- tabletalk-0.6.0/LICENSE +133 -0
- tabletalk-0.6.0/LICENSE_HISTORY.md +9 -0
- tabletalk-0.6.0/PKG-INFO +194 -0
- tabletalk-0.6.0/README.md +161 -0
- tabletalk-0.6.0/docs/architecture.md +53 -0
- tabletalk-0.6.0/docs/commands.md +33 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/docs/conf.py +5 -5
- tabletalk-0.6.0/docs/configuration.md +80 -0
- tabletalk-0.6.0/docs/contributing.md +35 -0
- tabletalk-0.6.0/docs/evals.md +94 -0
- tabletalk-0.6.0/docs/getting-started.md +87 -0
- tabletalk-0.6.0/docs/index.md +18 -0
- tabletalk-0.6.0/docs/release.md +56 -0
- tabletalk-0.6.0/docs/research.md +49 -0
- tabletalk-0.6.0/examples/dbt-analytics/README.md +149 -0
- tabletalk-0.6.0/examples/dbt-analytics/agents/revenue.yaml +19 -0
- tabletalk-0.6.0/examples/dbt-analytics/dbt_project.yml +10 -0
- tabletalk-0.6.0/examples/dbt-analytics/evals/revenue.yaml +62 -0
- tabletalk-0.6.0/examples/dbt-analytics/imports/finance-recognized-revenue.sql +5 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/marts/fct_orders.sql +16 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/other/duplicate_orders.sql +2 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/schema.yml +79 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/sources.yml +40 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/staging/disabled_orders.sql +2 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/staging/ephemeral_order_keys.sql +2 -0
- tabletalk-0.6.0/examples/dbt-analytics/models/staging/stg_orders.sql +15 -0
- tabletalk-0.6.0/examples/dbt-analytics/package-lock.yml +4 -0
- tabletalk-0.6.0/examples/dbt-analytics/packages/shared_models/dbt_project.yml +4 -0
- tabletalk-0.6.0/examples/dbt-analytics/packages/shared_models/models/schema.yml +8 -0
- tabletalk-0.6.0/examples/dbt-analytics/packages/shared_models/models/shared_calendar.sql +1 -0
- tabletalk-0.6.0/examples/dbt-analytics/packages.yml +2 -0
- tabletalk-0.6.0/examples/dbt-analytics/profiles.yml +8 -0
- tabletalk-0.6.0/examples/dbt-analytics/references/july-recognized-revenue.sql +5 -0
- tabletalk-0.6.0/examples/dbt-analytics/seeds/raw_orders.csv +6 -0
- tabletalk-0.6.0/examples/dbt-analytics/semantics/revenue.yaml +26 -0
- tabletalk-0.6.0/examples/dbt-analytics/tabletalk.yaml +12 -0
- tabletalk-0.6.0/examples/dbt-analytics/target/catalog.json +1 -0
- tabletalk-0.6.0/examples/dbt-analytics/target/manifest.json +1 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/pyproject.toml +9 -10
- tabletalk-0.6.0/tabletalk/__init__.py +22 -0
- tabletalk-0.6.0/tabletalk/agents.py +298 -0
- tabletalk-0.6.0/tabletalk/authoring.py +106 -0
- tabletalk-0.6.0/tabletalk/cli.py +1460 -0
- tabletalk-0.6.0/tabletalk/connections.py +166 -0
- tabletalk-0.6.0/tabletalk/diagnostics.py +311 -0
- tabletalk-0.6.0/tabletalk/evals/__init__.py +765 -0
- tabletalk-0.6.0/tabletalk/executors.py +29 -0
- tabletalk-0.6.0/tabletalk/factories.py +114 -0
- tabletalk-0.6.0/tabletalk/imports.py +230 -0
- tabletalk-0.6.0/tabletalk/inference.py +462 -0
- tabletalk-0.6.0/tabletalk/interfaces.py +84 -0
- tabletalk-0.6.0/tabletalk/manifest.py +587 -0
- tabletalk-0.6.0/tabletalk/metrics.py +441 -0
- tabletalk-0.6.0/tabletalk/migrations.py +76 -0
- tabletalk-0.6.0/tabletalk/project.py +228 -0
- tabletalk-0.6.0/tabletalk/proposal_service.py +1062 -0
- tabletalk-0.6.0/tabletalk/proposals.py +375 -0
- tabletalk-0.6.0/tabletalk/providers/duckdb_provider.py +26 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/tabletalk/providers/openai_provider.py +78 -48
- tabletalk-0.6.0/tabletalk/providers/snowflake_provider.py +46 -0
- tabletalk-0.6.0/tabletalk/providers/sqlite_provider.py +30 -0
- tabletalk-0.6.0/tabletalk/reports.py +266 -0
- tabletalk-0.6.0/tabletalk/runtime/__init__.py +678 -0
- tabletalk-0.6.0/tabletalk/schemas.py +72 -0
- tabletalk-0.6.0/tabletalk/semantic_analysis.py +366 -0
- tabletalk-0.6.0/tabletalk/semantic_guard.py +164 -0
- tabletalk-0.6.0/tabletalk/semantic_inference.py +93 -0
- tabletalk-0.6.0/tabletalk/semantics.py +246 -0
- tabletalk-0.6.0/tabletalk/tests/test_connections.py +108 -0
- tabletalk-0.6.0/tabletalk/tests/test_dbt_native.py +1269 -0
- tabletalk-0.6.0/tabletalk/tests/test_domain.py +235 -0
- tabletalk-0.6.0/tabletalk/tests/test_executor.py +25 -0
- tabletalk-0.6.0/tabletalk/tests/test_live_ollama.py +40 -0
- tabletalk-0.6.0/tabletalk/tests/test_metrics.py +142 -0
- tabletalk-0.6.0/tabletalk/tests/test_proposal_service.py +447 -0
- tabletalk-0.6.0/tabletalk/tests/test_reports.py +290 -0
- tabletalk-0.6.0/tabletalk/tests/test_semantic_analysis.py +161 -0
- tabletalk-0.6.0/tabletalk/tests/test_semantic_guard.py +142 -0
- tabletalk-0.6.0/tabletalk/traces.py +241 -0
- tabletalk-0.6.0/tabletalk/validation.py +199 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/uv.lock +913 -93
- tabletalk-0.4.0/LICENSE +0 -157
- tabletalk-0.4.0/PKG-INFO +0 -272
- tabletalk-0.4.0/README.md +0 -242
- tabletalk-0.4.0/contexts/default_context.yaml +0 -13
- tabletalk-0.4.0/docs/api-reference.md +0 -621
- tabletalk-0.4.0/docs/architecture.md +0 -41
- tabletalk-0.4.0/docs/commands.md +0 -75
- tabletalk-0.4.0/docs/concepts.md +0 -45
- tabletalk-0.4.0/docs/configuration.md +0 -71
- tabletalk-0.4.0/docs/contexts.md +0 -35
- tabletalk-0.4.0/docs/contributing.md +0 -182
- tabletalk-0.4.0/docs/databases.md +0 -38
- tabletalk-0.4.0/docs/dbt-integration.md +0 -21
- tabletalk-0.4.0/docs/evals.md +0 -64
- tabletalk-0.4.0/docs/getting-started.md +0 -46
- tabletalk-0.4.0/docs/index.md +0 -53
- tabletalk-0.4.0/docs/llm-providers.md +0 -203
- tabletalk-0.4.0/docs/profiles.md +0 -20
- tabletalk-0.4.0/docs/refactor/current-state-audit.md +0 -109
- tabletalk-0.4.0/docs/refactor/deletion-plan.md +0 -25
- tabletalk-0.4.0/docs/refactor/evaluation-strategy.md +0 -24
- tabletalk-0.4.0/docs/refactor/licensing-audit.md +0 -45
- tabletalk-0.4.0/docs/refactor/market-positioning.md +0 -42
- tabletalk-0.4.0/docs/refactor/migration-plan.md +0 -27
- tabletalk-0.4.0/docs/refactor/product-thesis.md +0 -65
- tabletalk-0.4.0/docs/refactor/progress-audit.md +0 -47
- tabletalk-0.4.0/docs/refactor/reliability-model.md +0 -40
- tabletalk-0.4.0/docs/refactor/target-architecture.md +0 -52
- tabletalk-0.4.0/docs/refactor/user-journey.md +0 -40
- tabletalk-0.4.0/docs/safe-mode.md +0 -146
- tabletalk-0.4.0/docs/web-ui.md +0 -103
- tabletalk-0.4.0/examples/duckdb-analytics/README.md +0 -19
- tabletalk-0.4.0/examples/duckdb-analytics/agents/revenue.yaml +0 -51
- tabletalk-0.4.0/examples/duckdb-analytics/evals/fixtures/data.sql +0 -14
- tabletalk-0.4.0/examples/duckdb-analytics/evals/fixtures/schema.sql +0 -19
- tabletalk-0.4.0/examples/duckdb-analytics/evals/revenue.yaml +0 -59
- tabletalk-0.4.0/examples/duckdb-analytics/seed.py +0 -14
- tabletalk-0.4.0/examples/duckdb-analytics/tabletalk.yaml +0 -17
- tabletalk-0.4.0/examples/snowflake-production/README.md +0 -27
- tabletalk-0.4.0/examples/snowflake-production/agents/sales.yaml +0 -37
- tabletalk-0.4.0/examples/snowflake-production/evals/fixtures/data.sql +0 -5
- tabletalk-0.4.0/examples/snowflake-production/evals/fixtures/schema.sql +0 -14
- tabletalk-0.4.0/examples/snowflake-production/evals/sales.yaml +0 -36
- tabletalk-0.4.0/examples/snowflake-production/tabletalk.yaml +0 -27
- tabletalk-0.4.0/examples/sqlite-starter/README.md +0 -25
- tabletalk-0.4.0/examples/sqlite-starter/agents/sales.yaml +0 -37
- tabletalk-0.4.0/examples/sqlite-starter/evals/fixtures/data.sql +0 -5
- tabletalk-0.4.0/examples/sqlite-starter/evals/fixtures/schema.sql +0 -6
- tabletalk-0.4.0/examples/sqlite-starter/evals/sales.yaml +0 -53
- tabletalk-0.4.0/examples/sqlite-starter/tabletalk.yaml +0 -17
- tabletalk-0.4.0/tabletalk/__init__.py +0 -44
- tabletalk-0.4.0/tabletalk/agents.py +0 -284
- tabletalk-0.4.0/tabletalk/app.py +0 -227
- tabletalk-0.4.0/tabletalk/cli.py +0 -918
- tabletalk-0.4.0/tabletalk/compiler/__init__.py +0 -23
- tabletalk-0.4.0/tabletalk/compiler/artifact.py +0 -101
- tabletalk-0.4.0/tabletalk/compiler/compiler.py +0 -384
- tabletalk-0.4.0/tabletalk/compiler/plan.py +0 -52
- tabletalk-0.4.0/tabletalk/dbt_manifest.py +0 -208
- tabletalk-0.4.0/tabletalk/domain/__init__.py +0 -47
- tabletalk-0.4.0/tabletalk/domain/errors.py +0 -119
- tabletalk-0.4.0/tabletalk/domain/models.py +0 -141
- tabletalk-0.4.0/tabletalk/domain/serialization.py +0 -39
- tabletalk-0.4.0/tabletalk/evals/__init__.py +0 -32
- tabletalk-0.4.0/tabletalk/evals/loader.py +0 -323
- tabletalk-0.4.0/tabletalk/evals/metrics.py +0 -555
- tabletalk-0.4.0/tabletalk/evals/models.py +0 -138
- tabletalk-0.4.0/tabletalk/evals/receipts.py +0 -127
- tabletalk-0.4.0/tabletalk/evals/reporters.py +0 -70
- tabletalk-0.4.0/tabletalk/evals/runner.py +0 -316
- tabletalk-0.4.0/tabletalk/evals/sql_analysis.py +0 -81
- tabletalk-0.4.0/tabletalk/factories.py +0 -195
- tabletalk-0.4.0/tabletalk/interfaces.py +0 -491
- tabletalk-0.4.0/tabletalk/project.py +0 -679
- tabletalk-0.4.0/tabletalk/providers/duckdb_provider.py +0 -143
- tabletalk-0.4.0/tabletalk/providers/snowflake_provider.py +0 -180
- tabletalk-0.4.0/tabletalk/providers/sqlite_provider.py +0 -98
- tabletalk-0.4.0/tabletalk/runtime/__init__.py +0 -6
- tabletalk-0.4.0/tabletalk/runtime/sql_validation.py +0 -272
- tabletalk-0.4.0/tabletalk/runtime/structured.py +0 -983
- tabletalk-0.4.0/tabletalk/static/app.js +0 -265
- tabletalk-0.4.0/tabletalk/static/index.html +0 -129
- tabletalk-0.4.0/tabletalk/static/logo.jpg +0 -0
- tabletalk-0.4.0/tabletalk/static/styles.css +0 -134
- tabletalk-0.4.0/tabletalk/structured_output.py +0 -96
- tabletalk-0.4.0/tabletalk/tests/conftest.py +0 -569
- tabletalk-0.4.0/tabletalk/tests/providers/__init__.py +0 -0
- tabletalk-0.4.0/tabletalk/tests/providers/test_duckdb.py +0 -382
- tabletalk-0.4.0/tabletalk/tests/providers/test_snowflake.py +0 -71
- tabletalk-0.4.0/tabletalk/tests/providers/test_sqlite.py +0 -369
- tabletalk-0.4.0/tabletalk/tests/test_agent_evals.py +0 -280
- tabletalk-0.4.0/tabletalk/tests/test_agents.py +0 -219
- tabletalk-0.4.0/tabletalk/tests/test_app.py +0 -299
- tabletalk-0.4.0/tabletalk/tests/test_ask.py +0 -174
- tabletalk-0.4.0/tabletalk/tests/test_cli.py +0 -281
- tabletalk-0.4.0/tabletalk/tests/test_compiler.py +0 -129
- tabletalk-0.4.0/tabletalk/tests/test_dbt_manifest.py +0 -109
- tabletalk-0.4.0/tabletalk/tests/test_domain.py +0 -135
- tabletalk-0.4.0/tabletalk/tests/test_eval_receipts.py +0 -122
- tabletalk-0.4.0/tabletalk/tests/test_evals.py +0 -195
- tabletalk-0.4.0/tabletalk/tests/test_factories.py +0 -219
- tabletalk-0.4.0/tabletalk/tests/test_interfaces.py +0 -142
- tabletalk-0.4.0/tabletalk/tests/test_live_ollama.py +0 -78
- tabletalk-0.4.0/tabletalk/tests/test_plan.py +0 -32
- tabletalk-0.4.0/tabletalk/tests/test_project.py +0 -385
- tabletalk-0.4.0/tabletalk/tests/test_sql_validation.py +0 -148
- tabletalk-0.4.0/tabletalk/tests/test_structured_output.py +0 -63
- tabletalk-0.4.0/tabletalk/tests/test_structured_runtime.py +0 -552
- tabletalk-0.4.0/tabletalk/tests/test_utils.py +0 -69
- tabletalk-0.4.0/tabletalk/utils.py +0 -202
- tabletalk-0.4.0/tabletalk.yaml +0 -20
- {tabletalk-0.4.0 → tabletalk-0.6.0}/.pre-commit-config.yaml +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/.python-version +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/docs/Makefile +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/docs/make.bat +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/tabletalk/main.py +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/tabletalk/providers/__init__.py +0 -0
- {tabletalk-0.4.0 → tabletalk-0.6.0}/tabletalk/tests/__init__.py +0 -0
|
@@ -75,8 +75,21 @@ jobs:
|
|
|
75
75
|
with ZipFile(wheel) as archive:
|
|
76
76
|
names = archive.namelist()
|
|
77
77
|
assert not any("/tests/" in name for name in names)
|
|
78
|
-
assert not any("
|
|
79
|
-
assert "tabletalk/
|
|
78
|
+
assert not any("tabletalk/static/" in name for name in names)
|
|
79
|
+
assert "tabletalk/app.py" not in names
|
|
80
|
+
for required in (
|
|
81
|
+
"tabletalk/agents.py",
|
|
82
|
+
"tabletalk/evals/__init__.py",
|
|
83
|
+
"tabletalk/manifest.py",
|
|
84
|
+
"tabletalk/project.py",
|
|
85
|
+
"tabletalk/runtime/__init__.py",
|
|
86
|
+
"tabletalk/validation.py",
|
|
87
|
+
):
|
|
88
|
+
assert required in names, f"missing wheel module: {required}"
|
|
89
|
+
for license_name in ("LICENSE", "LICENSE_HISTORY.md", "COMMERCIAL-LICENSE.md"):
|
|
90
|
+
assert any(name.endswith("licenses/" + license_name) for name in names), (
|
|
91
|
+
f"missing wheel license file: {license_name}"
|
|
92
|
+
)
|
|
80
93
|
PY
|
|
81
94
|
uv venv .package-test
|
|
82
95
|
uv pip install --python .package-test/bin/python dist/*.whl
|
|
@@ -46,13 +46,21 @@ jobs:
|
|
|
46
46
|
with ZipFile(wheel) as archive:
|
|
47
47
|
names = archive.namelist()
|
|
48
48
|
assert not any("/tests/" in name for name in names)
|
|
49
|
-
assert not any("
|
|
49
|
+
assert not any("tabletalk/static/" in name for name in names)
|
|
50
|
+
assert "tabletalk/app.py" not in names
|
|
50
51
|
for required in (
|
|
51
|
-
"tabletalk/
|
|
52
|
-
"tabletalk/
|
|
53
|
-
"tabletalk/
|
|
52
|
+
"tabletalk/agents.py",
|
|
53
|
+
"tabletalk/evals/__init__.py",
|
|
54
|
+
"tabletalk/manifest.py",
|
|
55
|
+
"tabletalk/project.py",
|
|
56
|
+
"tabletalk/runtime/__init__.py",
|
|
57
|
+
"tabletalk/validation.py",
|
|
54
58
|
):
|
|
55
|
-
assert required in names, f"missing wheel
|
|
59
|
+
assert required in names, f"missing wheel module: {required}"
|
|
60
|
+
for license_name in ("LICENSE", "LICENSE_HISTORY.md", "COMMERCIAL-LICENSE.md"):
|
|
61
|
+
assert any(name.endswith("licenses/" + license_name) for name in names), (
|
|
62
|
+
f"missing wheel license file: {license_name}"
|
|
63
|
+
)
|
|
56
64
|
PY
|
|
57
65
|
|
|
58
66
|
- name: Verify wheel installation
|
|
@@ -71,6 +71,7 @@ examples/ecommerce/evals/fixtures/*.duckdb
|
|
|
71
71
|
__pycache__
|
|
72
72
|
|
|
73
73
|
# TableTalk generated output and per-user runtime state
|
|
74
|
+
.tabletalk/
|
|
74
75
|
/examples/*/manifest/
|
|
75
76
|
.tabletalk_history.jsonl
|
|
76
77
|
.tabletalk_favorites.json
|
|
@@ -81,6 +82,8 @@ __pycache__
|
|
|
81
82
|
|
|
82
83
|
# dbt generated and per-user state
|
|
83
84
|
.user.yml
|
|
85
|
+
dbt_packages/
|
|
86
|
+
logs/
|
|
84
87
|
/examples/*/dbt_project/dbt_packages/
|
|
85
88
|
/examples/*/dbt_project/logs/
|
|
86
89
|
|
|
@@ -145,6 +148,12 @@ docs/_build/
|
|
|
145
148
|
.pybuilder/
|
|
146
149
|
target/
|
|
147
150
|
|
|
151
|
+
# The reference project's manifest is a dbt-generated acceptance fixture.
|
|
152
|
+
!examples/dbt-analytics/target/
|
|
153
|
+
examples/dbt-analytics/target/*
|
|
154
|
+
!examples/dbt-analytics/target/manifest.json
|
|
155
|
+
!examples/dbt-analytics/target/catalog.json
|
|
156
|
+
|
|
148
157
|
# Jupyter Notebook
|
|
149
158
|
.ipynb_checkpoints
|
|
150
159
|
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Commercial licensing
|
|
2
|
+
|
|
3
|
+
The source code in this repository is available under the
|
|
4
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). That license permits the
|
|
5
|
+
personal and noncommercial uses described in its terms. It does not grant
|
|
6
|
+
commercial-use rights.
|
|
7
|
+
|
|
8
|
+
William Bates offers implementation, integration, semantic onboarding, and
|
|
9
|
+
custom eval-suite development. Commercial use requires separate written
|
|
10
|
+
permission. Contact [William Bates](mailto:wbates@palanbates.com).
|
|
11
|
+
|
|
12
|
+
This notice is not itself a commercial license or an offer of specific terms.
|
tabletalk-0.6.0/LICENSE
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
# PolyForm Noncommercial License 1.0.0
|
|
2
|
+
|
|
3
|
+
<https://polyformproject.org/licenses/noncommercial/1.0.0>
|
|
4
|
+
|
|
5
|
+
## Acceptance
|
|
6
|
+
|
|
7
|
+
In order to get any license under these terms, you must agree
|
|
8
|
+
to them as both strict obligations and conditions to all
|
|
9
|
+
your licenses.
|
|
10
|
+
|
|
11
|
+
## Copyright License
|
|
12
|
+
|
|
13
|
+
The licensor grants you a copyright license for the
|
|
14
|
+
software to do everything you might do with the software
|
|
15
|
+
that would otherwise infringe the licensor's copyright
|
|
16
|
+
in it for any permitted purpose. However, you may
|
|
17
|
+
only distribute the software according to [Distribution
|
|
18
|
+
License](#distribution-license) and make changes or new works
|
|
19
|
+
based on the software according to [Changes and New Works
|
|
20
|
+
License](#changes-and-new-works-license).
|
|
21
|
+
|
|
22
|
+
## Distribution License
|
|
23
|
+
|
|
24
|
+
The licensor grants you an additional copyright license
|
|
25
|
+
to distribute copies of the software. Your license
|
|
26
|
+
to distribute covers distributing the software with
|
|
27
|
+
changes and new works permitted by [Changes and New Works
|
|
28
|
+
License](#changes-and-new-works-license).
|
|
29
|
+
|
|
30
|
+
## Notices
|
|
31
|
+
|
|
32
|
+
You must ensure that anyone who gets a copy of any part of
|
|
33
|
+
the software from you also gets a copy of these terms or the
|
|
34
|
+
URL for them above, as well as copies of any plain-text lines
|
|
35
|
+
beginning with `Required Notice:` that the licensor provided
|
|
36
|
+
with the software. For example:
|
|
37
|
+
|
|
38
|
+
> Required Notice: Copyright Yoyodyne, Inc. (http://example.com)
|
|
39
|
+
|
|
40
|
+
## Changes and New Works License
|
|
41
|
+
|
|
42
|
+
The licensor grants you an additional copyright license to
|
|
43
|
+
make changes and new works based on the software for any
|
|
44
|
+
permitted purpose.
|
|
45
|
+
|
|
46
|
+
## Patent License
|
|
47
|
+
|
|
48
|
+
The licensor grants you a patent license for the software that
|
|
49
|
+
covers patent claims the licensor can license, or becomes able
|
|
50
|
+
to license, that you would infringe by using the software.
|
|
51
|
+
|
|
52
|
+
## Noncommercial Purposes
|
|
53
|
+
|
|
54
|
+
Any noncommercial purpose is a permitted purpose.
|
|
55
|
+
|
|
56
|
+
## Personal Uses
|
|
57
|
+
|
|
58
|
+
Personal use for research, experiment, and testing for
|
|
59
|
+
the benefit of public knowledge, personal study, private
|
|
60
|
+
entertainment, hobby projects, amateur pursuits, or religious
|
|
61
|
+
observance, without any anticipated commercial application,
|
|
62
|
+
is use for a permitted purpose.
|
|
63
|
+
|
|
64
|
+
## Noncommercial Organizations
|
|
65
|
+
|
|
66
|
+
Use by any charitable organization, educational institution,
|
|
67
|
+
public research organization, public safety or health
|
|
68
|
+
organization, environmental protection organization,
|
|
69
|
+
or government institution is use for a permitted purpose
|
|
70
|
+
regardless of the source of funding or obligations resulting
|
|
71
|
+
from the funding.
|
|
72
|
+
|
|
73
|
+
## Fair Use
|
|
74
|
+
|
|
75
|
+
You may have "fair use" rights for the software under the
|
|
76
|
+
law. These terms do not limit them.
|
|
77
|
+
|
|
78
|
+
## No Other Rights
|
|
79
|
+
|
|
80
|
+
These terms do not allow you to sublicense or transfer any of
|
|
81
|
+
your licenses to anyone else, or prevent the licensor from
|
|
82
|
+
granting licenses to anyone else. These terms do not imply
|
|
83
|
+
any other licenses.
|
|
84
|
+
|
|
85
|
+
## Patent Defense
|
|
86
|
+
|
|
87
|
+
If you make any written claim that the software infringes or
|
|
88
|
+
contributes to infringement of any patent, your patent license
|
|
89
|
+
for the software granted under these terms ends immediately. If
|
|
90
|
+
your company makes such a claim, your patent license ends
|
|
91
|
+
immediately for work on behalf of your company.
|
|
92
|
+
|
|
93
|
+
## Violations
|
|
94
|
+
|
|
95
|
+
The first time you are notified in writing that you have
|
|
96
|
+
violated any of these terms, or done anything with the software
|
|
97
|
+
not covered by your licenses, your licenses can nonetheless
|
|
98
|
+
continue if you come into full compliance with these terms,
|
|
99
|
+
and take practical steps to correct past violations, within
|
|
100
|
+
32 days of receiving notice. Otherwise, all your licenses
|
|
101
|
+
end immediately.
|
|
102
|
+
|
|
103
|
+
## No Liability
|
|
104
|
+
|
|
105
|
+
**As far as the law allows, the software comes as is, without
|
|
106
|
+
any warranty or condition, and the licensor will not be liable
|
|
107
|
+
to you for any damages arising out of these terms or the use
|
|
108
|
+
or nature of the software, under any kind of legal claim.**
|
|
109
|
+
|
|
110
|
+
## Definitions
|
|
111
|
+
|
|
112
|
+
The **licensor** is the individual or entity offering these
|
|
113
|
+
terms, and the **software** is the software the licensor makes
|
|
114
|
+
available under these terms.
|
|
115
|
+
|
|
116
|
+
**You** refers to the individual or entity agreeing to these
|
|
117
|
+
terms.
|
|
118
|
+
|
|
119
|
+
**Your company** is any legal entity, sole proprietorship,
|
|
120
|
+
or other kind of organization that you work for, plus all
|
|
121
|
+
organizations that have control over, are under the control of,
|
|
122
|
+
or are under common control with that organization. **Control**
|
|
123
|
+
means ownership of substantially all the assets of an entity,
|
|
124
|
+
or the power to direct its management and policies by vote,
|
|
125
|
+
contract, or otherwise. Control can be direct or indirect.
|
|
126
|
+
|
|
127
|
+
**Your licenses** are all the licenses granted to you for the
|
|
128
|
+
software under these terms.
|
|
129
|
+
|
|
130
|
+
**Use** means anything you do with the software requiring one
|
|
131
|
+
of your licenses.
|
|
132
|
+
|
|
133
|
+
Required Notice: Copyright 2026 William Bates.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# License history
|
|
2
|
+
|
|
3
|
+
Earlier TableTalk revisions and releases were offered under Creative Commons
|
|
4
|
+
Attribution-NonCommercial-NoDerivatives 4.0 terms. Those grants remain in
|
|
5
|
+
effect for the revisions to which they applied.
|
|
6
|
+
|
|
7
|
+
Revisions committed on or after August 9, 2026 are offered under the
|
|
8
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). Commercial use of those
|
|
9
|
+
revisions requires a separate written agreement.
|
tabletalk-0.6.0/PKG-INFO
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: tabletalk
|
|
3
|
+
Version: 0.6.0
|
|
4
|
+
Summary: dbt-native development, evaluation, and observability for trusted data agents
|
|
5
|
+
Project-URL: Homepage, https://github.com/wtbates99/tabletalk
|
|
6
|
+
Project-URL: Repository, https://github.com/wtbates99/tabletalk
|
|
7
|
+
Project-URL: Commercial License, https://github.com/wtbates99/tabletalk/blob/main/COMMERCIAL-LICENSE.md
|
|
8
|
+
Author-email: William Bates <wbates@palanbates.com>
|
|
9
|
+
License-Expression: PolyForm-Noncommercial-1.0.0
|
|
10
|
+
License-File: COMMERCIAL-LICENSE.md
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
License-File: LICENSE_HISTORY.md
|
|
13
|
+
Keywords: data-agents,dbt,evaluation,llm,observability,sql
|
|
14
|
+
Classifier: Development Status :: 3 - Alpha
|
|
15
|
+
Classifier: License :: Other/Proprietary License
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Requires-Python: >=3.10
|
|
20
|
+
Requires-Dist: click>=8.0
|
|
21
|
+
Requires-Dist: openai>=1.0.0
|
|
22
|
+
Requires-Dist: pyyaml>=6.0
|
|
23
|
+
Requires-Dist: rich>=13.0
|
|
24
|
+
Requires-Dist: sqlglot>=25.0
|
|
25
|
+
Provides-Extra: all
|
|
26
|
+
Requires-Dist: duckdb>=0.9; extra == 'all'
|
|
27
|
+
Requires-Dist: snowflake-connector-python>=3.0; extra == 'all'
|
|
28
|
+
Provides-Extra: duckdb
|
|
29
|
+
Requires-Dist: duckdb>=0.9; extra == 'duckdb'
|
|
30
|
+
Provides-Extra: snowflake
|
|
31
|
+
Requires-Dist: snowflake-connector-python>=3.0; extra == 'snowflake'
|
|
32
|
+
Description-Content-Type: text/markdown
|
|
33
|
+
|
|
34
|
+
# TableTalk
|
|
35
|
+
|
|
36
|
+
**Mount a dbt project. Review proposed agents and evaluations. Prove that your data agents are safe, grounded, and correct.**
|
|
37
|
+
|
|
38
|
+
[](https://github.com/wtbates99/tabletalk/actions/workflows/db_provider_tests.yml)
|
|
39
|
+
[](https://www.python.org/)
|
|
40
|
+
[](https://pypi.org/project/tabletalk/)
|
|
41
|
+
[](LICENSE)
|
|
42
|
+
|
|
43
|
+
TableTalk is the dbt-native development, evaluation, and observability layer for
|
|
44
|
+
trusted data agents. It uses the analytical knowledge that a company already
|
|
45
|
+
maintains in dbt. It creates reproducible agents and defensible evaluation
|
|
46
|
+
evidence. It does not require a second large semantic configuration system.
|
|
47
|
+
|
|
48
|
+
TableTalk is not a general agent framework. It is not a BI dashboard. It does
|
|
49
|
+
not replace dbt.
|
|
50
|
+
|
|
51
|
+
## What TableTalk proves
|
|
52
|
+
|
|
53
|
+
- `manifest.json` defines the queryable resources.
|
|
54
|
+
- Agent files contain dbt selectors and small TableTalk policies.
|
|
55
|
+
- Every agent and eval starts as a review-only proposal.
|
|
56
|
+
- Approval binds the exact proposal, manifest, base files, and resource bytes.
|
|
57
|
+
- Trusted semantic statements record source evidence.
|
|
58
|
+
- Inferred statements remain untrusted until a user confirms them.
|
|
59
|
+
- SQL validation runs before every read-only warehouse query.
|
|
60
|
+
- Live questions and evals use the same runtime.
|
|
61
|
+
- Approved reference queries provide changing-data ground truth.
|
|
62
|
+
- Inference artifacts support strict replay with no model request.
|
|
63
|
+
- Reports include hard checks, trial metrics, uncertainty, paired comparisons,
|
|
64
|
+
and failure ownership.
|
|
65
|
+
|
|
66
|
+
## Install
|
|
67
|
+
|
|
68
|
+
```console
|
|
69
|
+
pip install tabletalk
|
|
70
|
+
pip install "tabletalk[duckdb]"
|
|
71
|
+
pip install "tabletalk[snowflake]"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
TableTalk requires Python 3.10 or newer. SQLite uses the Python standard
|
|
75
|
+
library. The initial warehouse adapters are SQLite, DuckDB, and Snowflake.
|
|
76
|
+
|
|
77
|
+
## Start from an existing dbt project
|
|
78
|
+
|
|
79
|
+
```console
|
|
80
|
+
cd my-dbt-project
|
|
81
|
+
dbt parse
|
|
82
|
+
dbt docs generate
|
|
83
|
+
tabletalk init
|
|
84
|
+
tabletalk propose
|
|
85
|
+
tabletalk approve
|
|
86
|
+
tabletalk eval run
|
|
87
|
+
tabletalk ask revenue "What was recognized revenue last month?"
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
`tabletalk propose` analyzes dbt groups, tags, packages, paths, owners, access,
|
|
91
|
+
metrics, semantic models, tests, constraints, descriptions, and lineage. It
|
|
92
|
+
creates at most three coherent proposals. It prefers curated marts. It shows
|
|
93
|
+
semantic gaps and exact file diffs. It does not write project resources.
|
|
94
|
+
|
|
95
|
+
The approval command requires the exact proposal digest for automated use:
|
|
96
|
+
|
|
97
|
+
```console
|
|
98
|
+
tabletalk approve prp_EXAMPLE \
|
|
99
|
+
--yes \
|
|
100
|
+
--digest sha256:EXAMPLE \
|
|
101
|
+
--actor "$USER"
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Approval fails if the manifest or a reviewed base file changed. TableTalk runs
|
|
105
|
+
approved evals immediately.
|
|
106
|
+
|
|
107
|
+
## Acquire missing meaning
|
|
108
|
+
|
|
109
|
+
Provide a targeted answer:
|
|
110
|
+
|
|
111
|
+
```console
|
|
112
|
+
tabletalk propose \
|
|
113
|
+
--answer "time_basis:model.analytics.orders.booked_revenue=booked_date"
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Preview an organizational query as evidence:
|
|
117
|
+
|
|
118
|
+
```console
|
|
119
|
+
tabletalk propose --sql-import finance/revenue.sql
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Request optional model hypotheses:
|
|
123
|
+
|
|
124
|
+
```console
|
|
125
|
+
tabletalk propose --infer
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
TableTalk marks every model hypothesis as inferred. An inferred statement cannot
|
|
129
|
+
support an answer. A user must confirm its meaning through a new proposal.
|
|
130
|
+
|
|
131
|
+
## Evaluate and replay
|
|
132
|
+
|
|
133
|
+
```console
|
|
134
|
+
tabletalk eval run revenue --trials 3 --workers 2
|
|
135
|
+
tabletalk eval run revenue --trials 3 --cache-policy replay
|
|
136
|
+
tabletalk eval run revenue --baseline revenue-v1
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Each run writes a concise terminal result and three detailed artifacts:
|
|
140
|
+
|
|
141
|
+
- versioned JSON for tools
|
|
142
|
+
- JUnit XML for CI
|
|
143
|
+
- Markdown for a pull request.
|
|
144
|
+
|
|
145
|
+
The report identifies the model, agent, manifest, semantic knowledge, prompt,
|
|
146
|
+
runtime, warehouse, and inference fingerprint. It reports pass rate, a Wilson
|
|
147
|
+
confidence interval, latency, tokens, configured cost, flaky outcomes, and
|
|
148
|
+
deterministic failure diagnoses.
|
|
149
|
+
|
|
150
|
+
## Security and retention
|
|
151
|
+
|
|
152
|
+
- Use a warehouse identity with physical read-only permissions.
|
|
153
|
+
- Treat connector permissions as the final security boundary.
|
|
154
|
+
- TableTalk rejects writes, unsafe reads, unknown resources, unknown columns,
|
|
155
|
+
unsafe joins, excessive limits, and unapproved sensitive data.
|
|
156
|
+
- TableTalk never treats lineage as proof that a join is correct.
|
|
157
|
+
- TableTalk does not store credentials in traces, reports, or fingerprints.
|
|
158
|
+
- Inference artifacts hash prompts by default. Set `persist_prompts: true` only
|
|
159
|
+
when exact raw-prompt retention is acceptable.
|
|
160
|
+
- Traces and reports retain bounded evidence rows. Exact evidence is required
|
|
161
|
+
for deterministic replay and claim validation.
|
|
162
|
+
- TableTalk does not delete artifacts automatically. The operator must apply a
|
|
163
|
+
retention policy to `.tabletalk/`.
|
|
164
|
+
|
|
165
|
+
See the [commerce example](examples/dbt-analytics/README.md), the
|
|
166
|
+
[getting-started guide](docs/getting-started.md), and the
|
|
167
|
+
[research evidence](docs/research.md).
|
|
168
|
+
|
|
169
|
+
## Development
|
|
170
|
+
|
|
171
|
+
```console
|
|
172
|
+
uv sync --locked --all-groups
|
|
173
|
+
uv run pytest -q
|
|
174
|
+
uv run ruff check tabletalk
|
|
175
|
+
uv run mypy tabletalk
|
|
176
|
+
uv build
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
The live Ollama smoke test is opt-in. It uses only `gemma4:31b-cloud`:
|
|
180
|
+
|
|
181
|
+
```console
|
|
182
|
+
TABLETALK_LIVE_OLLAMA=1 uv run pytest -m live_ollama
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
## License and services
|
|
186
|
+
|
|
187
|
+
TableTalk is source-available under the
|
|
188
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). The license permits the uses
|
|
189
|
+
that its text defines. Commercial use requires separate written permission.
|
|
190
|
+
|
|
191
|
+
William Bates offers implementation, integration, semantic onboarding, and
|
|
192
|
+
custom eval-suite development. See the
|
|
193
|
+
[commercial notice](COMMERCIAL-LICENSE.md). Earlier releases keep their earlier
|
|
194
|
+
license grants. See [LICENSE_HISTORY.md](LICENSE_HISTORY.md).
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# TableTalk
|
|
2
|
+
|
|
3
|
+
**Mount a dbt project. Review proposed agents and evaluations. Prove that your data agents are safe, grounded, and correct.**
|
|
4
|
+
|
|
5
|
+
[](https://github.com/wtbates99/tabletalk/actions/workflows/db_provider_tests.yml)
|
|
6
|
+
[](https://www.python.org/)
|
|
7
|
+
[](https://pypi.org/project/tabletalk/)
|
|
8
|
+
[](LICENSE)
|
|
9
|
+
|
|
10
|
+
TableTalk is the dbt-native development, evaluation, and observability layer for
|
|
11
|
+
trusted data agents. It uses the analytical knowledge that a company already
|
|
12
|
+
maintains in dbt. It creates reproducible agents and defensible evaluation
|
|
13
|
+
evidence. It does not require a second large semantic configuration system.
|
|
14
|
+
|
|
15
|
+
TableTalk is not a general agent framework. It is not a BI dashboard. It does
|
|
16
|
+
not replace dbt.
|
|
17
|
+
|
|
18
|
+
## What TableTalk proves
|
|
19
|
+
|
|
20
|
+
- `manifest.json` defines the queryable resources.
|
|
21
|
+
- Agent files contain dbt selectors and small TableTalk policies.
|
|
22
|
+
- Every agent and eval starts as a review-only proposal.
|
|
23
|
+
- Approval binds the exact proposal, manifest, base files, and resource bytes.
|
|
24
|
+
- Trusted semantic statements record source evidence.
|
|
25
|
+
- Inferred statements remain untrusted until a user confirms them.
|
|
26
|
+
- SQL validation runs before every read-only warehouse query.
|
|
27
|
+
- Live questions and evals use the same runtime.
|
|
28
|
+
- Approved reference queries provide changing-data ground truth.
|
|
29
|
+
- Inference artifacts support strict replay with no model request.
|
|
30
|
+
- Reports include hard checks, trial metrics, uncertainty, paired comparisons,
|
|
31
|
+
and failure ownership.
|
|
32
|
+
|
|
33
|
+
## Install
|
|
34
|
+
|
|
35
|
+
```console
|
|
36
|
+
pip install tabletalk
|
|
37
|
+
pip install "tabletalk[duckdb]"
|
|
38
|
+
pip install "tabletalk[snowflake]"
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
TableTalk requires Python 3.10 or newer. SQLite uses the Python standard
|
|
42
|
+
library. The initial warehouse adapters are SQLite, DuckDB, and Snowflake.
|
|
43
|
+
|
|
44
|
+
## Start from an existing dbt project
|
|
45
|
+
|
|
46
|
+
```console
|
|
47
|
+
cd my-dbt-project
|
|
48
|
+
dbt parse
|
|
49
|
+
dbt docs generate
|
|
50
|
+
tabletalk init
|
|
51
|
+
tabletalk propose
|
|
52
|
+
tabletalk approve
|
|
53
|
+
tabletalk eval run
|
|
54
|
+
tabletalk ask revenue "What was recognized revenue last month?"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
`tabletalk propose` analyzes dbt groups, tags, packages, paths, owners, access,
|
|
58
|
+
metrics, semantic models, tests, constraints, descriptions, and lineage. It
|
|
59
|
+
creates at most three coherent proposals. It prefers curated marts. It shows
|
|
60
|
+
semantic gaps and exact file diffs. It does not write project resources.
|
|
61
|
+
|
|
62
|
+
The approval command requires the exact proposal digest for automated use:
|
|
63
|
+
|
|
64
|
+
```console
|
|
65
|
+
tabletalk approve prp_EXAMPLE \
|
|
66
|
+
--yes \
|
|
67
|
+
--digest sha256:EXAMPLE \
|
|
68
|
+
--actor "$USER"
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Approval fails if the manifest or a reviewed base file changed. TableTalk runs
|
|
72
|
+
approved evals immediately.
|
|
73
|
+
|
|
74
|
+
## Acquire missing meaning
|
|
75
|
+
|
|
76
|
+
Provide a targeted answer:
|
|
77
|
+
|
|
78
|
+
```console
|
|
79
|
+
tabletalk propose \
|
|
80
|
+
--answer "time_basis:model.analytics.orders.booked_revenue=booked_date"
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Preview an organizational query as evidence:
|
|
84
|
+
|
|
85
|
+
```console
|
|
86
|
+
tabletalk propose --sql-import finance/revenue.sql
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Request optional model hypotheses:
|
|
90
|
+
|
|
91
|
+
```console
|
|
92
|
+
tabletalk propose --infer
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
TableTalk marks every model hypothesis as inferred. An inferred statement cannot
|
|
96
|
+
support an answer. A user must confirm its meaning through a new proposal.
|
|
97
|
+
|
|
98
|
+
## Evaluate and replay
|
|
99
|
+
|
|
100
|
+
```console
|
|
101
|
+
tabletalk eval run revenue --trials 3 --workers 2
|
|
102
|
+
tabletalk eval run revenue --trials 3 --cache-policy replay
|
|
103
|
+
tabletalk eval run revenue --baseline revenue-v1
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Each run writes a concise terminal result and three detailed artifacts:
|
|
107
|
+
|
|
108
|
+
- versioned JSON for tools
|
|
109
|
+
- JUnit XML for CI
|
|
110
|
+
- Markdown for a pull request.
|
|
111
|
+
|
|
112
|
+
The report identifies the model, agent, manifest, semantic knowledge, prompt,
|
|
113
|
+
runtime, warehouse, and inference fingerprint. It reports pass rate, a Wilson
|
|
114
|
+
confidence interval, latency, tokens, configured cost, flaky outcomes, and
|
|
115
|
+
deterministic failure diagnoses.
|
|
116
|
+
|
|
117
|
+
## Security and retention
|
|
118
|
+
|
|
119
|
+
- Use a warehouse identity with physical read-only permissions.
|
|
120
|
+
- Treat connector permissions as the final security boundary.
|
|
121
|
+
- TableTalk rejects writes, unsafe reads, unknown resources, unknown columns,
|
|
122
|
+
unsafe joins, excessive limits, and unapproved sensitive data.
|
|
123
|
+
- TableTalk never treats lineage as proof that a join is correct.
|
|
124
|
+
- TableTalk does not store credentials in traces, reports, or fingerprints.
|
|
125
|
+
- Inference artifacts hash prompts by default. Set `persist_prompts: true` only
|
|
126
|
+
when exact raw-prompt retention is acceptable.
|
|
127
|
+
- Traces and reports retain bounded evidence rows. Exact evidence is required
|
|
128
|
+
for deterministic replay and claim validation.
|
|
129
|
+
- TableTalk does not delete artifacts automatically. The operator must apply a
|
|
130
|
+
retention policy to `.tabletalk/`.
|
|
131
|
+
|
|
132
|
+
See the [commerce example](examples/dbt-analytics/README.md), the
|
|
133
|
+
[getting-started guide](docs/getting-started.md), and the
|
|
134
|
+
[research evidence](docs/research.md).
|
|
135
|
+
|
|
136
|
+
## Development
|
|
137
|
+
|
|
138
|
+
```console
|
|
139
|
+
uv sync --locked --all-groups
|
|
140
|
+
uv run pytest -q
|
|
141
|
+
uv run ruff check tabletalk
|
|
142
|
+
uv run mypy tabletalk
|
|
143
|
+
uv build
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
The live Ollama smoke test is opt-in. It uses only `gemma4:31b-cloud`:
|
|
147
|
+
|
|
148
|
+
```console
|
|
149
|
+
TABLETALK_LIVE_OLLAMA=1 uv run pytest -m live_ollama
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
## License and services
|
|
153
|
+
|
|
154
|
+
TableTalk is source-available under the
|
|
155
|
+
[PolyForm Noncommercial License 1.0.0](LICENSE). The license permits the uses
|
|
156
|
+
that its text defines. Commercial use requires separate written permission.
|
|
157
|
+
|
|
158
|
+
William Bates offers implementation, integration, semantic onboarding, and
|
|
159
|
+
custom eval-suite development. See the
|
|
160
|
+
[commercial notice](COMMERCIAL-LICENSE.md). Earlier releases keep their earlier
|
|
161
|
+
license grants. See [LICENSE_HISTORY.md](LICENSE_HISTORY.md).
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Architecture and safety boundaries
|
|
2
|
+
|
|
3
|
+
The live and eval paths use one runtime:
|
|
4
|
+
|
|
5
|
+
```text
|
|
6
|
+
dbt artifacts -> semantic analysis -> proposal -> exact approval
|
|
7
|
+
|
|
|
8
|
+
v
|
|
9
|
+
question -> resolved agent -> trusted meaning -> model inference
|
|
10
|
+
-> SQL AST validation -> read-only connector -> evidence rows
|
|
11
|
+
-> evidence-linked answer -> hard checks -> result artifact
|
|
12
|
+
-> deterministic metrics -> diagnosis -> reports
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## Boundaries
|
|
16
|
+
|
|
17
|
+
| Boundary | Module | Rule |
|
|
18
|
+
|---|---|---|
|
|
19
|
+
| dbt ingestion | `manifest.py` | `manifest.json` alone can add queryable resources. |
|
|
20
|
+
| semantic analysis | `semantic_analysis.py` | Analysis labels trusted, confirmed, inferred, missing, and conflicting meaning. |
|
|
21
|
+
| proposal | `proposal_service.py` | Generation is deterministic unless the user requests `--infer`. |
|
|
22
|
+
| approval | `proposals.py` | Approval binds exact content, manifest digest, base digest, actor, and dependencies. |
|
|
23
|
+
| inference | `runtime` and `inference.py` | Inference records are separate from metric computation. |
|
|
24
|
+
| SQL safety | `validation.py` | The parsed AST determines actual resources and columns. |
|
|
25
|
+
| execution | `connections.py` | The connector opens or governs a read-only target. |
|
|
26
|
+
| evaluation | `evals` | Eval cases call the same runtime as live questions. |
|
|
27
|
+
| metrics | `metrics.py` | Metrics use saved trial results and paired keys. |
|
|
28
|
+
| diagnosis | `diagnostics.py` | Deterministic checks map to stable failure codes and owners. |
|
|
29
|
+
| reports | `reports.py` | JSON, Markdown, and JUnit use one report object. |
|
|
30
|
+
|
|
31
|
+
## Safety rules
|
|
32
|
+
|
|
33
|
+
Validation rejects multiple statements, writes, commands, external-read
|
|
34
|
+
functions, unknown or ambiguous relations, unknown columns, joins without a
|
|
35
|
+
condition, excessive limits, and unapproved sensitive data.
|
|
36
|
+
|
|
37
|
+
Lineage proves dependency. It does not prove join correctness. Join evidence
|
|
38
|
+
must come from constraints, relationship tests, explicit metadata, or approved
|
|
39
|
+
eval behavior.
|
|
40
|
+
|
|
41
|
+
The connector permission is the final security boundary. SQL validation adds a
|
|
42
|
+
second deterministic boundary.
|
|
43
|
+
|
|
44
|
+
## Reproducibility
|
|
45
|
+
|
|
46
|
+
An inference artifact has a content address and a complete configuration
|
|
47
|
+
fingerprint. The cache index points from one fingerprint to one immutable
|
|
48
|
+
artifact. The artifact stores a trace or one typed non-answer outcome. Strict
|
|
49
|
+
replay makes no model request.
|
|
50
|
+
|
|
51
|
+
The local executor uses bounded threads and preserves task order. Evaluation
|
|
52
|
+
semantics do not depend on the executor. A future distributed executor can
|
|
53
|
+
implement the same small interface. Apache Spark is not a core dependency.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Commands
|
|
2
|
+
|
|
3
|
+
| Command | Result |
|
|
4
|
+
|---|---|
|
|
5
|
+
| `tabletalk init` | Mount a parsed dbt project and create small project directories. |
|
|
6
|
+
| `tabletalk migrate [PATH]` | Add current schema identifiers to editable YAML resources. |
|
|
7
|
+
| `tabletalk propose` | Analyze semantics and create at most three review-only proposals. |
|
|
8
|
+
| `tabletalk proposal show [ID]` | Show exact proposal bytes, evidence, and diff. |
|
|
9
|
+
| `tabletalk proposal edit ID FILE --from PATH` | Create a new immutable proposal from an edited file. |
|
|
10
|
+
| `tabletalk proposal reject [ID] --reason TEXT` | Record a proposal rejection. |
|
|
11
|
+
| `tabletalk approve [ID]` | Approve exact proposal bytes and run written evals. |
|
|
12
|
+
| `tabletalk agent create` | Create a review-only agent proposal from explicit selectors. |
|
|
13
|
+
| `tabletalk agent list` | List approved agents and resolved model counts. |
|
|
14
|
+
| `tabletalk agent show NAME` | Show one approved agent and its resolved dbt scope. |
|
|
15
|
+
| `tabletalk eval create NAME` | Execute reviewed SQL and create a review-only eval proposal. |
|
|
16
|
+
| `tabletalk eval run [NAME]` | Run eval trials and write JSON, Markdown, and JUnit reports. |
|
|
17
|
+
| `tabletalk baseline approve NAME --from REPORT` | Approve exact prior trial results as a baseline. |
|
|
18
|
+
| `tabletalk baseline show NAME` | Show an approved baseline. |
|
|
19
|
+
| `tabletalk ask NAME QUESTION` | Run the shared runtime and show answer provenance. |
|
|
20
|
+
| `tabletalk doctor` | Validate artifacts, target access, selectors, and eval coverage. |
|
|
21
|
+
|
|
22
|
+
`tabletalk propose` accepts `--sql-import`, `--bi-import`, `--history-import`,
|
|
23
|
+
`--answer GAP=VALUE`, and `--infer`.
|
|
24
|
+
|
|
25
|
+
`tabletalk eval run` accepts `--case`, `--trials`, `--workers`, `--baseline`, and
|
|
26
|
+
`--cache-policy`. Cache policies are `normal`, `read-only`, `write-only`,
|
|
27
|
+
`replay`, and `disabled`.
|
|
28
|
+
|
|
29
|
+
`replay` fails on a cache miss. It never calls a model.
|
|
30
|
+
|
|
31
|
+
Exit code 3 means that one or more eval hard checks failed. Exit code 4 means
|
|
32
|
+
that a resource or approval failed validation. Exit code 1 means that an
|
|
33
|
+
operational step failed.
|