mainsequence 6.0.52__tar.gz → 7.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- mainsequence-7.0.0/MANIFEST.in +1 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/PKG-INFO +9 -6
- {mainsequence-6.0.52 → mainsequence-7.0.0}/README.md +8 -5
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/AGENTS.md +5 -5
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/a2a_sdk_execution/SKILL.md +3 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/application_surfaces/api_surfaces/SKILL.md +1 -1
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/data_access/exploration/SKILL.md +27 -27
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/data_publishing/meta_tables/SKILL.md +14 -14
- {mainsequence-6.0.52/agent_scaffold/skills/data_publishing/data_nodes → mainsequence-7.0.0/agent_scaffold/skills/data_publishing/time_index_table_updates}/SKILL.md +72 -67
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/maintenance/bug_auditor/SKILL.md +2 -2
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/maintenance/project-maintenance/SKILL.md +1 -1
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/platform_operations/access_control_and_sharing/SKILL.md +7 -7
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/platform_operations/orchestration_and_releases/SKILL.md +4 -4
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/sdk_project_execution/SKILL.md +7 -7
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/api.py +91 -91
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/cli.py +273 -356
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/agent_runtime_models.py +7 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/base.py +3 -4
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/dtype_codec.py +3 -1
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/metatables/core.py +192 -182
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/models_foundry.py +29 -17
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/utils.py +1 -1
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/__init__.py +8 -4
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/hashing.py +12 -9
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/sqlalchemy_contracts.py +5 -10
- mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_refs.py +303 -0
- {mainsequence-6.0.52/mainsequence/meta_tables/data_nodes → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates}/__init__.py +4 -4
- mainsequence-6.0.52/mainsequence/meta_tables/data_nodes/build_operations.py → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates/configuration.py +159 -41
- mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates/dependencies.py +9 -0
- mainsequence-6.0.52/mainsequence/meta_tables/data_nodes/persist_managers.py → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates/managers.py +219 -320
- {mainsequence-6.0.52/mainsequence/meta_tables/data_nodes → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates}/namespacing.py +4 -4
- mainsequence-6.0.52/mainsequence/meta_tables/data_nodes/run_operations.py → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates/runner.py +152 -182
- mainsequence-6.0.52/mainsequence/meta_tables/data_nodes/data_nodes.py → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates/updaters.py +173 -509
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/PKG-INFO +9 -6
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/SOURCES.txt +19 -15
- {mainsequence-6.0.52 → mainsequence-7.0.0}/pyproject.toml +1 -1
- mainsequence-7.0.0/tests/fixtures/time_index_table_update_hash_v2_golden.json +141 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_canonical_api_route_contract.py +8 -16
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_cli.py +231 -181
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_client.py +19 -18
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_duckdb_interface_dimensions.py +6 -6
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_environment_response_contracts.py +5 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_filter_normalization.py +190 -33
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_meta_tables_client_models.py +17 -32
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_meta_tables_sqlalchemy_contracts.py +21 -13
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_pod_project_resolution.py +22 -26
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_project_runtime_context.py +22 -6
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_source_table_configuration.py +0 -1
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_sqlite_interface_dimensions.py +4 -6
- mainsequence-7.0.0/tests/test_time_index_table_access_dimension_audit.py +263 -0
- mainsequence-6.0.52/tests/test_data_node_storage_dimension_queries.py → mainsequence-7.0.0/tests/test_time_index_table_dimension_queries.py +16 -17
- mainsequence-6.0.52/tests/test_build_operations_hashing.py → mainsequence-7.0.0/tests/test_time_index_table_update_configuration.py +61 -61
- mainsequence-6.0.52/tests/test_run_configuration.py → mainsequence-7.0.0/tests/test_time_index_table_update_configuration_client.py +236 -232
- mainsequence-6.0.52/tests/test_data_node_update_flow.py → mainsequence-7.0.0/tests/test_time_index_table_update_flow.py +46 -36
- mainsequence-7.0.0/tests/test_time_index_table_update_hard_cut.py +357 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_update_runner_uid_runtime.py +123 -114
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_update_statistics.py +5 -5
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_update_uid_guards.py +14 -8
- mainsequence-6.0.52/mainsequence/meta_tables/data_nodes/models.py +0 -33
- mainsequence-6.0.52/tests/test_data_access_mixin_dimension_audit.py +0 -234
- {mainsequence-6.0.52 → mainsequence-7.0.0}/LICENSE +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/data_publishing/meta_table_migrations/SKILL.md +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/__main__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/bootstrap.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/browser_auth.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/config.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/docker_utils.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/doctor.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/local_ops.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/migrations.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/model_filters.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/project_status.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/pydantic_cli.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/sdk_utils.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/ssh_utils.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/cli/ui.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/client.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/compute_validation.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/data_sources_interfaces/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/data_sources_interfaces/duckdb.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/data_sources_interfaces/local_paths.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/data_sources_interfaces/sqlite.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/exceptions.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/metatables/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/models_helpers.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/models_user.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/client/observability.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/defaults.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/instrumentation/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/instrumentation/utils.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/logconf.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/__main__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/compiled_sql/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/compiled_sql/v1.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/future_registry.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/alembic.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/env.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/provider.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/registry.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/scaffold.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/templates/__init__.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/templates/env.py.mako +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/migrations/templates/script.py.mako +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/pydantic_metadata.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/meta_tables/schema_names.py +0 -0
- {mainsequence-6.0.52/mainsequence/meta_tables/data_nodes → mainsequence-7.0.0/mainsequence/meta_tables/time_index_table_updates}/utils.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/project_context.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/project_skills.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/runtime_flags.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence/scaffold_skills.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/dependency_links.txt +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/entry_points.txt +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/requires.txt +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/mainsequence.egg-info/top_level.txt +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/setup.cfg +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_auth_precedence.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_cli_browser_auth.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_cli_migrations.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_dependency_extras.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_environment_resource_context.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_instrumentation.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_local_ops.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_logconf.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_meta_table_migrations.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_models_user_request_bound_auth.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_notification_uid_contract.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_observability_client.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_project_batch_jobs_from_file.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_project_branch_contract.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_project_skills.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_scaffold_skills.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_scheduler_client_model.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_schema_names.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_secret_client_model.py +0 -0
- {mainsequence-6.0.52 → mainsequence-7.0.0}/tests/test_ssh_utils.py +0 -0
|
@@ -0,0 +1 @@
|
|
|
1
|
+
include tests/fixtures/time_index_table_update_hash_v2_golden.json
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: mainsequence
|
|
3
|
-
Version:
|
|
3
|
+
Version: 7.0.0
|
|
4
4
|
Summary: Main Sequence SDK
|
|
5
5
|
Author-email: Main Sequence GmbH <dev@main-sequence.io>
|
|
6
6
|
License-Expression: LicenseRef-MainSequence-Proprietary
|
|
@@ -55,7 +55,7 @@ The Main Sequence Python SDK is the client and development toolkit for the Main
|
|
|
55
55
|
|
|
56
56
|
The Main Sequence platform allows you to:
|
|
57
57
|
|
|
58
|
-
1. rapidly build and deploy data products
|
|
58
|
+
1. rapidly build and deploy data products with normalized `MetaTable` contracts and `TimeIndexTableUpdater` producer workflows
|
|
59
59
|
2. rapidly deploy RBAC-enabled dashboards on the platform
|
|
60
60
|
3. rapidly deploy agents using the Google Agent SDK
|
|
61
61
|
|
|
@@ -69,6 +69,7 @@ The key idea is that you can focus on development and deployment, while the plat
|
|
|
69
69
|
- Documentation: [Documentation Site](https://mainsequence-sdk.github.io/mainsequence-sdk/)
|
|
70
70
|
- Security policy: [SECURITY.md](SECURITY.md)
|
|
71
71
|
- Release history: [CHANGELOG.md](CHANGELOG.md)
|
|
72
|
+
- Major-version migration: [6.x to 7.0 hard cut](docs/migrations/v7-time-index-table-updater-hard-cut.md)
|
|
72
73
|
|
|
73
74
|
## What this repository contains
|
|
74
75
|
|
|
@@ -76,8 +77,8 @@ This repository contains the SDK and the documentation used to build and operate
|
|
|
76
77
|
|
|
77
78
|
Main package areas:
|
|
78
79
|
|
|
79
|
-
- `mainsequence.meta_tables`: `MetaTable`s, `
|
|
80
|
-
- `mainsequence.client`: API client models for projects, jobs,
|
|
80
|
+
- `mainsequence.meta_tables`: `MetaTable`s, `TimeIndexTableUpdater`s, update workflows, and persistence
|
|
81
|
+
- `mainsequence.client`: API client models for projects, jobs, time-index tables, sharing, and platform resources
|
|
81
82
|
- `mainsequence.cli`: the `mainsequence` command-line interface
|
|
82
83
|
|
|
83
84
|
Repository areas:
|
|
@@ -98,11 +99,13 @@ Recommended entry points:
|
|
|
98
99
|
- Beginner tutorial:
|
|
99
100
|
- [MainSequence SDK tutorial Project](https://github.com/mainsequence-projects/mainsequence-sdk-tutorial)
|
|
100
101
|
- Knowledge:
|
|
101
|
-
- [
|
|
102
|
+
- [Time-Index Table Updaters](docs/knowledge/time_index_table_updates.md)
|
|
102
103
|
- [MetaTables](docs/knowledge/meta_tables/index.md)
|
|
103
104
|
- [Constants and Secrets](docs/knowledge/infrastructure/constants_and_secrets.md)
|
|
104
105
|
- [Scheduling Jobs](docs/knowledge/infrastructure/scheduling_jobs.md)
|
|
105
106
|
- [Streamlit Dashboards](docs/knowledge/dashboards/streamlit/index.md)
|
|
107
|
+
- Migrations:
|
|
108
|
+
- [6.x to 7.0: Time-Index Table Updater hard cut](docs/migrations/v7-time-index-table-updater-hard-cut.md)
|
|
106
109
|
- CLI:
|
|
107
110
|
- [CLI Overview](docs/cli/index.md)
|
|
108
111
|
|
|
@@ -149,7 +152,7 @@ mainsequence project build_local_venv --path .
|
|
|
149
152
|
From there, the normal learning path is:
|
|
150
153
|
|
|
151
154
|
1. model your first canonical table with a backend-managed `MetaTable`
|
|
152
|
-
2. create your first `
|
|
155
|
+
2. create your first `TimeIndexTableUpdater` as an opinionated MetaTable-backed update workflow
|
|
153
156
|
3. understand sharing and RBAC for published tables
|
|
154
157
|
4. add an API or another application surface
|
|
155
158
|
5. schedule jobs
|
|
@@ -13,7 +13,7 @@ The Main Sequence Python SDK is the client and development toolkit for the Main
|
|
|
13
13
|
|
|
14
14
|
The Main Sequence platform allows you to:
|
|
15
15
|
|
|
16
|
-
1. rapidly build and deploy data products
|
|
16
|
+
1. rapidly build and deploy data products with normalized `MetaTable` contracts and `TimeIndexTableUpdater` producer workflows
|
|
17
17
|
2. rapidly deploy RBAC-enabled dashboards on the platform
|
|
18
18
|
3. rapidly deploy agents using the Google Agent SDK
|
|
19
19
|
|
|
@@ -27,6 +27,7 @@ The key idea is that you can focus on development and deployment, while the plat
|
|
|
27
27
|
- Documentation: [Documentation Site](https://mainsequence-sdk.github.io/mainsequence-sdk/)
|
|
28
28
|
- Security policy: [SECURITY.md](SECURITY.md)
|
|
29
29
|
- Release history: [CHANGELOG.md](CHANGELOG.md)
|
|
30
|
+
- Major-version migration: [6.x to 7.0 hard cut](docs/migrations/v7-time-index-table-updater-hard-cut.md)
|
|
30
31
|
|
|
31
32
|
## What this repository contains
|
|
32
33
|
|
|
@@ -34,8 +35,8 @@ This repository contains the SDK and the documentation used to build and operate
|
|
|
34
35
|
|
|
35
36
|
Main package areas:
|
|
36
37
|
|
|
37
|
-
- `mainsequence.meta_tables`: `MetaTable`s, `
|
|
38
|
-
- `mainsequence.client`: API client models for projects, jobs,
|
|
38
|
+
- `mainsequence.meta_tables`: `MetaTable`s, `TimeIndexTableUpdater`s, update workflows, and persistence
|
|
39
|
+
- `mainsequence.client`: API client models for projects, jobs, time-index tables, sharing, and platform resources
|
|
39
40
|
- `mainsequence.cli`: the `mainsequence` command-line interface
|
|
40
41
|
|
|
41
42
|
Repository areas:
|
|
@@ -56,11 +57,13 @@ Recommended entry points:
|
|
|
56
57
|
- Beginner tutorial:
|
|
57
58
|
- [MainSequence SDK tutorial Project](https://github.com/mainsequence-projects/mainsequence-sdk-tutorial)
|
|
58
59
|
- Knowledge:
|
|
59
|
-
- [
|
|
60
|
+
- [Time-Index Table Updaters](docs/knowledge/time_index_table_updates.md)
|
|
60
61
|
- [MetaTables](docs/knowledge/meta_tables/index.md)
|
|
61
62
|
- [Constants and Secrets](docs/knowledge/infrastructure/constants_and_secrets.md)
|
|
62
63
|
- [Scheduling Jobs](docs/knowledge/infrastructure/scheduling_jobs.md)
|
|
63
64
|
- [Streamlit Dashboards](docs/knowledge/dashboards/streamlit/index.md)
|
|
65
|
+
- Migrations:
|
|
66
|
+
- [6.x to 7.0: Time-Index Table Updater hard cut](docs/migrations/v7-time-index-table-updater-hard-cut.md)
|
|
64
67
|
- CLI:
|
|
65
68
|
- [CLI Overview](docs/cli/index.md)
|
|
66
69
|
|
|
@@ -107,7 +110,7 @@ mainsequence project build_local_venv --path .
|
|
|
107
110
|
From there, the normal learning path is:
|
|
108
111
|
|
|
109
112
|
1. model your first canonical table with a backend-managed `MetaTable`
|
|
110
|
-
2. create your first `
|
|
113
|
+
2. create your first `TimeIndexTableUpdater` as an opinionated MetaTable-backed update workflow
|
|
111
114
|
3. understand sharing and RBAC for published tables
|
|
112
115
|
4. add an API or another application surface
|
|
113
116
|
5. schedule jobs
|
|
@@ -62,7 +62,7 @@ CLI/SDK operations, and validation steps.
|
|
|
62
62
|
Core responsibilities:
|
|
63
63
|
|
|
64
64
|
- translate user intent into the correct Main Sequence implementation path:
|
|
65
|
-
- for data publishing and data pipelines, use `
|
|
65
|
+
- for data publishing and data pipelines, use `TimeIndexTableUpdater`s and `MetaTable`s
|
|
66
66
|
- for APIs serving the Command Center frontend, use the contract-authoritative
|
|
67
67
|
FastAPI release workflow
|
|
68
68
|
- for visualization, confirm the delivery target with the user:
|
|
@@ -80,7 +80,7 @@ Core responsibilities:
|
|
|
80
80
|
|
|
81
81
|
Typical outcomes include:
|
|
82
82
|
|
|
83
|
-
- build a `
|
|
83
|
+
- build a `TimeIndexTableUpdater` to publish a data pipeline
|
|
84
84
|
- build a `MetaTable` to record operational or application data
|
|
85
85
|
- build and release a `FastAPI` API whose Command Center contract-bearing
|
|
86
86
|
responses conform to a recorded revision of the canonical Command Center SDK
|
|
@@ -140,7 +140,7 @@ Delegation rules:
|
|
|
140
140
|
|
|
141
141
|
## Main Sequence Source-Of-Truth Rule
|
|
142
142
|
|
|
143
|
-
For any task involving Main Sequence code, CLI usage,
|
|
143
|
+
For any task involving Main Sequence code, CLI usage, TimeIndexTableUpdaters, orchestration, jobs, dashboards,
|
|
144
144
|
agents, releases, artifacts, RBAC, or platform
|
|
145
145
|
validation, always consult the latest relevant Main Sequence documentation before acting.
|
|
146
146
|
|
|
@@ -198,8 +198,8 @@ Typical routing:
|
|
|
198
198
|
`.agents/skills/mainsequence/project_to_agent/SKILL.md`
|
|
199
199
|
- project status audits, blocker analysis, failure classification, and upstream SDK assessment:
|
|
200
200
|
`.agents/skills/mainsequence/maintenance/bug_auditor/SKILL.md`
|
|
201
|
-
-
|
|
202
|
-
`.agents/skills/mainsequence/data_publishing/
|
|
201
|
+
- TimeIndexTableUpdaters, updates, identifiers, schema, metadata:
|
|
202
|
+
`.agents/skills/mainsequence/data_publishing/time_index_table_updates/SKILL.md`
|
|
203
203
|
- MetaTables, SQLAlchemy contracts, backend-managed registration, and governed operations:
|
|
204
204
|
`.agents/skills/mainsequence/data_publishing/meta_tables/SKILL.md`
|
|
205
205
|
- platform data discovery, published table search, and object identification before implementation:
|
|
@@ -71,6 +71,9 @@ mainsequence agent session get_or_create \
|
|
|
71
71
|
```
|
|
72
72
|
|
|
73
73
|
Use the returned session `uid` as the target `AgentSession` UID.
|
|
74
|
+
The returned `runtime_capabilities` map is a read-only advertisement from the
|
|
75
|
+
session harness. Inspect it when capability-aware behavior is required; never
|
|
76
|
+
construct, send, or override it from client code.
|
|
74
77
|
|
|
75
78
|
If this A2A request originates from an existing caller session, include the
|
|
76
79
|
parent session UID:
|
|
@@ -88,7 +88,7 @@ Record:
|
|
|
88
88
|
administrative, webhook, external integration, or backend-to-backend
|
|
89
89
|
- the exact contract ID and role for each request or response body
|
|
90
90
|
- route paths, HTTP methods, authentication expectations, and error semantics
|
|
91
|
-
- upstream `
|
|
91
|
+
- upstream `TimeIndexTableRef`, MetaTable, service, or external data dependencies
|
|
92
92
|
- whether the API uses backend transport or a contract-defined direct
|
|
93
93
|
development transport
|
|
94
94
|
|
{mainsequence-6.0.52 → mainsequence-7.0.0}/agent_scaffold/skills/data_access/exploration/SKILL.md
RENAMED
|
@@ -1,27 +1,27 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: mainsequence-data-exploration
|
|
3
|
-
description: Use this skill when the task is to discover what `
|
|
3
|
+
description: Use this skill when the task is to discover what `TimeIndexMetaTable` or `MetaTable` data already exists on the Main Sequence platform before implementation starts.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Main Sequence Data Exploration
|
|
7
7
|
|
|
8
8
|
## Overview
|
|
9
9
|
|
|
10
|
-
Use this skill when the task is about discovering what `
|
|
10
|
+
Use this skill when the task is about discovering what `TimeIndexMetaTable` or `MetaTable` data is already available on the platform before implementation starts.
|
|
11
11
|
|
|
12
12
|
This skill is for discovery only.
|
|
13
13
|
|
|
14
14
|
It helps answer questions such as:
|
|
15
15
|
|
|
16
16
|
- what published data already exists on this platform
|
|
17
|
-
- whether the right surface is a `
|
|
17
|
+
- whether the right storage surface is a `TimeIndexMetaTable` or `MetaTable`
|
|
18
18
|
- which identifier, storage UID, or object UID should be used
|
|
19
19
|
- what metadata is available for those objects
|
|
20
20
|
- what still needs to be clarified before implementation starts
|
|
21
21
|
|
|
22
22
|
## This Skill Can Do
|
|
23
23
|
|
|
24
|
-
- discover published
|
|
24
|
+
- discover published time-index tables through the CLI
|
|
25
25
|
- inspect published table identifiers, storage UIDs, and metadata
|
|
26
26
|
- search for tables by keyword, description, or column name
|
|
27
27
|
- inspect organization-visible table identifiers
|
|
@@ -32,7 +32,7 @@ It helps answer questions such as:
|
|
|
32
32
|
|
|
33
33
|
This skill must not claim ownership of:
|
|
34
34
|
|
|
35
|
-
- producing or modifying `
|
|
35
|
+
- producing or modifying `TimeIndexTableUpdater` pipelines
|
|
36
36
|
- designing `MetaTable` registration or operation behavior
|
|
37
37
|
- deciding code-level read patterns for discovered datasets
|
|
38
38
|
- building APIs or dashboards
|
|
@@ -45,8 +45,8 @@ This skill discovers and identifies data. It does not publish, redesign, or defi
|
|
|
45
45
|
|
|
46
46
|
- project bootstrap, scaffolding, and routing:
|
|
47
47
|
`.agents/skills/mainsequence/project_design/SKILL.md`
|
|
48
|
-
- `
|
|
49
|
-
`.agents/skills/mainsequence/data_publishing/
|
|
48
|
+
- `TimeIndexTableUpdater` creation or modification:
|
|
49
|
+
`.agents/skills/mainsequence/data_publishing/time_index_table_updates/SKILL.md`
|
|
50
50
|
- `MetaTable` registration or operation work:
|
|
51
51
|
`.agents/skills/mainsequence/data_publishing/meta_tables/SKILL.md`
|
|
52
52
|
- FastAPI providers serving the Command Center frontend:
|
|
@@ -62,7 +62,7 @@ Streamlit dashboard implementation is app-owned project code, not a Main Sequenc
|
|
|
62
62
|
|
|
63
63
|
1. `AGENTS.md`
|
|
64
64
|
2. `docs/cli/index.md`
|
|
65
|
-
3. `docs/knowledge/
|
|
65
|
+
3. `docs/knowledge/time_index_table_updates.md`
|
|
66
66
|
4. `docs/knowledge/meta_tables/index.md` when the exploration touches row-oriented storage
|
|
67
67
|
|
|
68
68
|
## Inputs This Skill Needs
|
|
@@ -79,7 +79,7 @@ Before exploring, collect or infer:
|
|
|
79
79
|
|
|
80
80
|
For every non-trivial exploration task, decide:
|
|
81
81
|
|
|
82
|
-
1. Is the
|
|
82
|
+
1. Is the storage surface a `TimeIndexMetaTable` or `MetaTable`?
|
|
83
83
|
2. Do we know the identifier already, or do we need to search first?
|
|
84
84
|
3. What metadata or object details are needed to complete the discovery?
|
|
85
85
|
4. Which implementation skill should receive the handoff after discovery?
|
|
@@ -97,17 +97,17 @@ Discovery priority:
|
|
|
97
97
|
- use column lookup only when the user is specifically searching for a schema or column name
|
|
98
98
|
- use detail only after you have identified the candidate object you want to inspect
|
|
99
99
|
|
|
100
|
-
Semantic description discovery for published
|
|
100
|
+
Semantic description discovery for published time-index tables searches
|
|
101
101
|
`TimeIndexMetaTable` metadata rows through:
|
|
102
102
|
|
|
103
103
|
- `GET /api/v1/time-index-meta-tables/description-search/?q=<text>`
|
|
104
104
|
|
|
105
|
-
This is the
|
|
105
|
+
This is the time-index table discovery surface because published output tables are
|
|
106
106
|
represented by `TimeIndexMetaTable` metadata.
|
|
107
107
|
|
|
108
108
|
Typical semantic discovery commands:
|
|
109
109
|
|
|
110
|
-
- `mainsequence
|
|
110
|
+
- `mainsequence time-index-table search "<keyword>"`
|
|
111
111
|
|
|
112
112
|
Optional ranking knobs for description discovery:
|
|
113
113
|
|
|
@@ -149,10 +149,10 @@ profile-to-table identity and must match the parent `TimeIndexMetaTable.uid`.
|
|
|
149
149
|
Structured filtering is a different path. Use it only when you already know the
|
|
150
150
|
field to filter by:
|
|
151
151
|
|
|
152
|
-
- `mainsequence
|
|
153
|
-
- `mainsequence
|
|
154
|
-
- `mainsequence
|
|
155
|
-
- `mainsequence
|
|
152
|
+
- `mainsequence time-index-table list`
|
|
153
|
+
- `mainsequence time-index-table list --show-filters`
|
|
154
|
+
- `mainsequence time-index-table list --filter KEY=VALUE`
|
|
155
|
+
- `mainsequence time-index-table detail <TIME_INDEX_META_TABLE_UID>`
|
|
156
156
|
|
|
157
157
|
Collection listing is scoped to one Organization Environment. Inside a
|
|
158
158
|
registered project checkout, the CLI derives that scope from the process-frozen
|
|
@@ -163,15 +163,15 @@ context while unrelated local development remains available.
|
|
|
163
163
|
Column lookup is also a different path. Use it only for schema-name discovery,
|
|
164
164
|
not as the default dataset discovery flow:
|
|
165
165
|
|
|
166
|
-
- `mainsequence
|
|
166
|
+
- `mainsequence time-index-table search "<column-name>" --mode column`
|
|
167
167
|
|
|
168
168
|
Raw SQL inspection is only for targeted exploration after you have a UID:
|
|
169
169
|
|
|
170
170
|
- `mainsequence meta-table run_query <META_TABLE_UID> "SELECT * FROM public.some_table LIMIT 100"`
|
|
171
|
-
- `mainsequence
|
|
171
|
+
- `mainsequence time-index-table run_query <TIME_INDEX_META_TABLE_UID> "SELECT * FROM public.some_table LIMIT 100"`
|
|
172
172
|
|
|
173
|
-
Use the MetaTable command for `MetaTable` UIDs. Use the
|
|
174
|
-
for `
|
|
173
|
+
Use the MetaTable command for `MetaTable` UIDs. Use the time-index-table command only
|
|
174
|
+
for `TimeIndexMetaTable` UIDs. Do not confuse the two surfaces.
|
|
175
175
|
|
|
176
176
|
For direct SDK/backend usage, MetaTable raw SQL is sent as a JSON string body,
|
|
177
177
|
not as an object. Do not send `{ "sql": "SELECT ..." }`.
|
|
@@ -182,12 +182,12 @@ Use the CLI or SDK client to identify what exists and collect the relevant ident
|
|
|
182
182
|
|
|
183
183
|
Do not define code-level read patterns here.
|
|
184
184
|
|
|
185
|
-
For
|
|
185
|
+
For time-index table discovery specifically:
|
|
186
186
|
|
|
187
|
-
- start with `mainsequence
|
|
188
|
-
- use `mainsequence
|
|
189
|
-
- use `mainsequence
|
|
190
|
-
- use `mainsequence
|
|
187
|
+
- start with `mainsequence time-index-table search`
|
|
188
|
+
- use `mainsequence time-index-table list` only when you need broader enumeration or structured filters
|
|
189
|
+
- use `mainsequence time-index-table search "<column-name>" --mode column` only for schema-name lookup
|
|
190
|
+
- use `mainsequence time-index-table detail` only after search or list identified the target storage
|
|
191
191
|
|
|
192
192
|
### 3. Report platform objects with evidence
|
|
193
193
|
|
|
@@ -205,7 +205,7 @@ Once the discovery is complete, hand off to the correct implementation skill.
|
|
|
205
205
|
|
|
206
206
|
Examples:
|
|
207
207
|
|
|
208
|
-
- if the task is about publishing or modifying time-series data, hand off to `.agents/skills/mainsequence/data_publishing/
|
|
208
|
+
- if the task is about publishing or modifying time-series data, hand off to `.agents/skills/mainsequence/data_publishing/time_index_table_updates/SKILL.md`
|
|
209
209
|
- if the task is about row-oriented operational data, hand off to `.agents/skills/mainsequence/data_publishing/meta_tables/SKILL.md`
|
|
210
210
|
- if the task is about APIs or dashboards, hand off to the relevant surface skill after discovery is complete
|
|
211
211
|
|
|
@@ -214,7 +214,7 @@ Examples:
|
|
|
214
214
|
When reviewing an exploration result, look for:
|
|
215
215
|
|
|
216
216
|
- a discovery claim without CLI evidence
|
|
217
|
-
- confusion between `
|
|
217
|
+
- confusion between updater behavior, `TimeIndexMetaTable`, and `MetaTable`
|
|
218
218
|
- jumping into implementation work before discovery is complete
|
|
219
219
|
- claiming implementation conclusions from a discovery-only pass
|
|
220
220
|
|
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: mainsequence-meta-tables
|
|
3
|
-
description: Use this skill when the task is about defining, querying, or reviewing Main Sequence MetaTables. This skill owns SQLAlchemy table contracts, backend-managed table authoring, external table registration, governed compiled SQL operations, foreign keys, indexes, naming, cadence, and validation rules. For Alembic migration lifecycle work, use the mainsequence-metatable-migrations skill. It does not own
|
|
3
|
+
description: Use this skill when the task is about defining, querying, or reviewing Main Sequence MetaTables. This skill owns SQLAlchemy table contracts, backend-managed table authoring, external table registration, governed compiled SQL operations, foreign keys, indexes, naming, cadence, and validation rules. For Alembic migration lifecycle work, use the mainsequence-metatable-migrations skill. It does not own TimeIndexTableUpdater producers, API route contracts, scheduling, releases, or sharing policy.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Main Sequence MetaTables
|
|
7
7
|
|
|
8
8
|
## Overview
|
|
9
9
|
|
|
10
|
-
Use this skill when the task changes row-oriented project tables that are not naturally time-
|
|
10
|
+
Use this skill when the task changes row-oriented project tables that are not naturally time-indexed.
|
|
11
11
|
|
|
12
12
|
This skill is for schema-driven application tables registered through TS Manager as `MetaTable` resources.
|
|
13
13
|
|
|
@@ -21,13 +21,13 @@ This skill is for schema-driven application tables registered through TS Manager
|
|
|
21
21
|
- design governed compiled SQL read and write operations
|
|
22
22
|
- route provider-based Alembic contract evolution to the MetaTable migration skill
|
|
23
23
|
- review table contracts for physical-name, namespace, and identifier issues
|
|
24
|
-
- review whether a task should be a `MetaTable` or a `
|
|
24
|
+
- review whether a task should be a `MetaTable` or a `TimeIndexTableUpdater`
|
|
25
25
|
|
|
26
26
|
## This Skill Must Not Claim
|
|
27
27
|
|
|
28
28
|
This skill must not claim ownership of:
|
|
29
29
|
|
|
30
|
-
-
|
|
30
|
+
- TimeIndexTableUpdater producer contracts
|
|
31
31
|
- Command Center FastAPI wire contracts
|
|
32
32
|
- workspace payloads
|
|
33
33
|
- job scheduling, image pinning, or releases
|
|
@@ -41,8 +41,8 @@ If the user is still in the discovery process and does not yet know what data ex
|
|
|
41
41
|
`.agents/skills/mainsequence/data_access/exploration/SKILL.md`
|
|
42
42
|
- MetaTable migrations:
|
|
43
43
|
`.agents/skills/mainsequence/data_publishing/meta_table_migrations/SKILL.md`
|
|
44
|
-
-
|
|
45
|
-
`.agents/skills/mainsequence/data_publishing/
|
|
44
|
+
- TimeIndexTableUpdaters:
|
|
45
|
+
`.agents/skills/mainsequence/data_publishing/time_index_table_updates/SKILL.md`
|
|
46
46
|
- Command Center-serving FastAPI providers:
|
|
47
47
|
`.agents/skills/mainsequence/application_surfaces/api_surfaces/SKILL.md`
|
|
48
48
|
- Jobs, images, resources, and releases:
|
|
@@ -84,7 +84,7 @@ If ownership of the physical table lifecycle is unclear, stop before choosing a
|
|
|
84
84
|
|
|
85
85
|
For every non-trivial task, decide:
|
|
86
86
|
|
|
87
|
-
1. Is this table really row-oriented, or should it be a
|
|
87
|
+
1. Is this table really row-oriented, or should it be a TimeIndexTableUpdater?
|
|
88
88
|
2. What is the business key?
|
|
89
89
|
3. Who owns the catalog boundary: `platform_managed` or `external_registered`?
|
|
90
90
|
4. Who owns physical DDL: `backend_managed`, `alembic_managed`, or `external_registered`?
|
|
@@ -236,7 +236,7 @@ selected `AlembicMetaTableMigration.metatable_models` list and let
|
|
|
236
236
|
and bind them.
|
|
237
237
|
|
|
238
238
|
For platform-managed migration registration, the data source is resolved from
|
|
239
|
-
the active Main Sequence project/session, the same way
|
|
239
|
+
the active Main Sequence project/session, the same way TimeIndexTableUpdater does. Do not
|
|
240
240
|
require or thread a `data_source_uid` through normal platform-managed example
|
|
241
241
|
code.
|
|
242
242
|
|
|
@@ -389,14 +389,14 @@ Only use physical table names returned by registered `MetaTable` objects when co
|
|
|
389
389
|
|
|
390
390
|
Do not hardcode platform-managed physical names manually.
|
|
391
391
|
|
|
392
|
-
### 7.
|
|
392
|
+
### 7. Updater output-table deletes use the time-index-table tail-delete API
|
|
393
393
|
|
|
394
|
-
For `PlatformTimeIndexMetaTable`
|
|
394
|
+
For `PlatformTimeIndexMetaTable` output tables populated by TimeIndexTableUpdaters, do not design raw
|
|
395
395
|
SQL delete operations or compiled SQL delete operations for rollback, repair, or
|
|
396
|
-
stream cleanup. Route that work to the
|
|
396
|
+
stream cleanup. Route that work to the TimeIndexTableUpdater skill and use
|
|
397
397
|
`TimeIndexMetaTable.delete_after_date(...)`.
|
|
398
398
|
|
|
399
|
-
The
|
|
399
|
+
The TimeIndexTableUpdater delete path is:
|
|
400
400
|
|
|
401
401
|
```text
|
|
402
402
|
POST /api/v1/time-index-meta-tables/<table_uid>/delete-after-date/
|
|
@@ -426,9 +426,9 @@ When reviewing an existing MetaTable workflow, look for:
|
|
|
426
426
|
- migration work that asks users to define backend payloads, artifact rows, or SDK request objects
|
|
427
427
|
- compiled SQL operations without complete table scope
|
|
428
428
|
- raw SQL that hardcodes stale physical names
|
|
429
|
-
- raw SQL or compiled SQL deletes against
|
|
429
|
+
- raw SQL or compiled SQL deletes against TimeIndexTableUpdater-owned
|
|
430
430
|
`PlatformTimeIndexMetaTable` storage instead of `delete_after_date(...)`
|
|
431
|
-
- a table that should really be modeled as a
|
|
431
|
+
- a table that should really be modeled as a TimeIndexTableUpdater instead
|
|
432
432
|
|
|
433
433
|
## Validation Checklist
|
|
434
434
|
|