oblako 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- oblako-0.1.0/.cfn-guard.rules +52 -0
- oblako-0.1.0/.github/workflows/ci.yml +213 -0
- oblako-0.1.0/.github/workflows/docs.yml +61 -0
- oblako-0.1.0/.github/workflows/redshift-image.yml +67 -0
- oblako-0.1.0/.github/workflows/release.yml +58 -0
- oblako-0.1.0/.gitignore +53 -0
- oblako-0.1.0/.pre-commit-config.yaml +30 -0
- oblako-0.1.0/CHANGELOG.md +140 -0
- oblako-0.1.0/CONTRIBUTING.md +44 -0
- oblako-0.1.0/LICENSE +202 -0
- oblako-0.1.0/Makefile +62 -0
- oblako-0.1.0/NOTICE +11 -0
- oblako-0.1.0/PKG-INFO +205 -0
- oblako-0.1.0/README.md +133 -0
- oblako-0.1.0/SECURITY.md +18 -0
- oblako-0.1.0/THIRD_PARTY_LICENSES.md +92 -0
- oblako-0.1.0/docker-compose.yml +173 -0
- oblako-0.1.0/docs/_static/custom.css +54 -0
- oblako-0.1.0/docs/_static/diagrams/access.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/access.svg +32 -0
- oblako-0.1.0/docs/_static/diagrams/ai-ml.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/ai-ml.svg +83 -0
- oblako-0.1.0/docs/_static/diagrams/caddy.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/caddy.svg +46 -0
- oblako-0.1.0/docs/_static/diagrams/containers.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/containers.svg +64 -0
- oblako-0.1.0/docs/_static/diagrams/data.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/data.svg +107 -0
- oblako-0.1.0/docs/_static/diagrams/routing.png +0 -0
- oblako-0.1.0/docs/_static/diagrams/routing.svg +33 -0
- oblako-0.1.0/docs/_static/oblako_almostly_logo.png +0 -0
- oblako-0.1.0/docs/api.md +84 -0
- oblako-0.1.0/docs/architecture.md +123 -0
- oblako-0.1.0/docs/conf.py +50 -0
- oblako-0.1.0/docs/dashboard.md +48 -0
- oblako-0.1.0/docs/index.md +64 -0
- oblako-0.1.0/docs/license.md +37 -0
- oblako-0.1.0/docs/quickstart.md +147 -0
- oblako-0.1.0/docs/runtimes.md +76 -0
- oblako-0.1.0/docs/services.md +605 -0
- oblako-0.1.0/examples/demo-notebooks/README.md +19 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/01_byoc_training.ipynb +214 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/02_endpoint_deployment.ipynb +266 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/03_sagemaker_pipeline.ipynb +280 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/README.md +43 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/container/Dockerfile +13 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/container/src/main.py +19 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/container/src/serve.py +79 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/container/src/train.py +74 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/creditdata.py +106 -0
- oblako-0.1.0/examples/demo-notebooks/credit-scoring/prep.py +75 -0
- oblako-0.1.0/examples/demo-notebooks/feast/README.md +32 -0
- oblako-0.1.0/examples/demo-notebooks/feast/feature_store_on_redshift.ipynb +509 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/01_kinesis_transaction_stream.ipynb +190 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/02_linear_learner_training.ipynb +263 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/README.md +51 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/container/Dockerfile +10 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/container/src/main.py +15 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/container/src/train.py +51 -0
- oblako-0.1.0/examples/demo-notebooks/fraud/frauddata.py +47 -0
- oblako-0.1.0/examples/python/agentcore/agent.py +42 -0
- oblako-0.1.0/examples/python/athena/daily_sales.py +173 -0
- oblako-0.1.0/examples/python/bedrock/batch.py +97 -0
- oblako-0.1.0/examples/python/bedrock/invoke.py +61 -0
- oblako-0.1.0/examples/python/cloudformation/deploy.py +103 -0
- oblako-0.1.0/examples/python/dynamodb/vector_search.py +101 -0
- oblako-0.1.0/examples/python/ecs/Dockerfile +8 -0
- oblako-0.1.0/examples/python/ecs/README.md +62 -0
- oblako-0.1.0/examples/python/ecs/app.py +94 -0
- oblako-0.1.0/examples/python/ecs/deploy.py +97 -0
- oblako-0.1.0/examples/python/ecs/infra/ecs-fargate-alb.yaml +143 -0
- oblako-0.1.0/examples/python/ecs/requirements.txt +3 -0
- oblako-0.1.0/examples/python/opensearch/rag.py +139 -0
- oblako-0.1.0/examples/python/platform_tour.py +37 -0
- oblako-0.1.0/examples/python/rds/aurora.py +85 -0
- oblako-0.1.0/examples/python/redshift/data_api.py +100 -0
- oblako-0.1.0/examples/python/redshift/ml.py +123 -0
- oblako-0.1.0/examples/python/redshift/scoring.py +109 -0
- oblako-0.1.0/examples/python/redshift/text2sql.py +197 -0
- oblako-0.1.0/examples/python/s3/model_store.py +66 -0
- oblako-0.1.0/examples/python/s3/presigned_upload.py +72 -0
- oblako-0.1.0/examples/python/sagemaker/local.py +41 -0
- oblako-0.1.0/examples/python/sagemaker/mlflow_image/Dockerfile +12 -0
- oblako-0.1.0/examples/python/sagemaker/mlflow_image/train.py +88 -0
- oblako-0.1.0/examples/python/sagemaker/mme_image/Dockerfile +9 -0
- oblako-0.1.0/examples/python/sagemaker/mme_image/serve.py +74 -0
- oblako-0.1.0/examples/python/sagemaker/monitor_image/Dockerfile +9 -0
- oblako-0.1.0/examples/python/sagemaker/monitor_image/process.py +62 -0
- oblako-0.1.0/examples/python/sagemaker/pipeline/pipeline_local.py +144 -0
- oblako-0.1.0/examples/python/sagemaker/pipeline/scripts/double.py +9 -0
- oblako-0.1.0/examples/python/sagemaker/pipeline/scripts/prepare.py +8 -0
- oblako-0.1.0/examples/python/sagemaker/process_image/Dockerfile +7 -0
- oblako-0.1.0/examples/python/sagemaker/process_image/process.py +34 -0
- oblako-0.1.0/examples/python/sagemaker/serve_image/Dockerfile +10 -0
- oblako-0.1.0/examples/python/sagemaker/serve_image/serve.py +52 -0
- oblako-0.1.0/examples/python/sagemaker/train_image/Dockerfile +9 -0
- oblako-0.1.0/examples/python/sagemaker/train_image/train.py +54 -0
- oblako-0.1.0/examples/python/sagemaker/tune_image/Dockerfile +9 -0
- oblako-0.1.0/examples/python/sagemaker/tune_image/train.py +68 -0
- oblako-0.1.0/examples/python/sam/README.md +140 -0
- oblako-0.1.0/examples/python/sam/app.py +122 -0
- oblako-0.1.0/examples/python/sam/event.json +1 -0
- oblako-0.1.0/examples/python/sam/pyproject.toml +9 -0
- oblako-0.1.0/examples/python/sam/template.yaml +34 -0
- oblako-0.1.0/examples/python/sam/uv.lock +23 -0
- oblako-0.1.0/examples/python/stepfunctions/asl/bedrock-prompt-chaining.asl.json +58 -0
- oblako-0.1.0/examples/python/stepfunctions/asl/sagemaker-hpo-transform.asl.json +63 -0
- oblako-0.1.0/examples/python/stepfunctions/asl/sagemaker-preprocess-train.asl.json +45 -0
- oblako-0.1.0/examples/python/stepfunctions/asl/sagemaker-train-transform.asl.json +47 -0
- oblako-0.1.0/examples/python/stepfunctions/run_workflows.py +130 -0
- oblako-0.1.0/examples/python/stepfunctions/workflow.py +113 -0
- oblako-0.1.0/oblako/__init__.py +3 -0
- oblako-0.1.0/oblako/banner.py +64 -0
- oblako-0.1.0/oblako/cli.py +530 -0
- oblako-0.1.0/oblako/config.py +68 -0
- oblako-0.1.0/oblako/dashboard/__init__.py +1 -0
- oblako-0.1.0/oblako/dashboard/api.py +1912 -0
- oblako-0.1.0/oblako/dashboard/frontend/.gitignore +25 -0
- oblako-0.1.0/oblako/dashboard/frontend/README.md +16 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_AMS-Regular-BQhdFMY1.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_AMS-Regular-DMm9YOAa.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_AMS-Regular-DRggAlZN.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Bold-ATXxdsX0.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Bold-BEiXGLvX.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Bold-Dq_IR9rO.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Regular-CTRA-rTL.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Regular-Di6jR-x-.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Caligraphic-Regular-wX97UBjC.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Bold-BdnERNNW.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Bold-BsDP51OF.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Bold-CL6g_b3V.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Regular-CB_wures.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Regular-CTYiF6lA.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Fraktur-Regular-Dxdc4cR9.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Bold-Cx986IdX.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Bold-Jm3AIy58.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Bold-waoOVXN0.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-BoldItalic-DxDJ3AOS.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-BoldItalic-DzxPMmG6.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-BoldItalic-SpSLRI95.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Italic-3WenGoN9.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Italic-BMLOBm91.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Italic-NWA7e6Wa.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Regular-B22Nviop.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Regular-Dr94JaBh.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Main-Regular-ypZvNtVU.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-BoldItalic-B3XSjfu4.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-BoldItalic-CZnvNsCZ.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-BoldItalic-iY-2wyZ7.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-Italic-DA0__PXp.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-Italic-flOr_0UB.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Math-Italic-t53AETM-.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Bold-CFMepnvq.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Bold-D1sUS0GD.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Bold-DbIhKOiC.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Italic-C3H0VqGB.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Italic-DN2j7dab.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Italic-YYjJ1zSn.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Regular-BNo7hRIc.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Regular-CS6fqUqJ.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_SansSerif-Regular-DDBCnlJ7.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Script-Regular-C5JkGWo-.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Script-Regular-D3wIWfF6.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Script-Regular-D5yQViql.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size1-Regular-C195tn64.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size1-Regular-Dbsnue_I.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size1-Regular-mCD8mA8B.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size2-Regular-B7gKUWhC.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size2-Regular-Dy4dx90m.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size2-Regular-oD1tc_U0.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size3-Regular-CTq5MqoE.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size3-Regular-DgpXs0kz.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size4-Regular-BF-4gkZK.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size4-Regular-DWFBv043.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Size4-Regular-Dl5lxZxV.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Typewriter-Regular-C0xS9mPB.woff +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Typewriter-Regular-CO6r4hn1.woff2 +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/KaTeX_Typewriter-Regular-D3Ib7_Hf.ttf +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/duckdb-browser-DQUTRvrj.js +2 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/index-CIhx4uB3.js +401 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/assets/index-CLqonV80.css +1 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/favicon.svg +1 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/icons.svg +24 -0
- oblako-0.1.0/oblako/dashboard/frontend/dist/index.html +14 -0
- oblako-0.1.0/oblako/dashboard/frontend/eslint.config.js +29 -0
- oblako-0.1.0/oblako/dashboard/frontend/index.html +13 -0
- oblako-0.1.0/oblako/dashboard/frontend/package-lock.json +5544 -0
- oblako-0.1.0/oblako/dashboard/frontend/package.json +40 -0
- oblako-0.1.0/oblako/dashboard/frontend/public/favicon.svg +1 -0
- oblako-0.1.0/oblako/dashboard/frontend/public/icons.svg +24 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/App.css +184 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/App.jsx +123 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/assets/hero.png +0 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/assets/react.svg +1 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/assets/vite.svg +1 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/components/CodeEditor.jsx +43 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/components/FlowGraph.jsx +103 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/components/SqlEditor.jsx +6 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/components/useDuckDB.js +52 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/index.css +119 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/main.jsx +10 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/AppConfigPage.jsx +543 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/AthenaPage.jsx +156 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/BedrockPage.jsx +114 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/CloudFormationPage.jsx +150 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/DynamoDBPage.jsx +99 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/Ec2Page.jsx +144 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/GluePage.jsx +389 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/IamPage.jsx +195 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/KinesisPage.jsx +143 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/LambdaPage.jsx +528 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/NotebookPage.jsx +290 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/RdsPage.jsx +143 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/RedshiftPage.jsx +224 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/S3Page.jsx +99 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/SageMakerPage.jsx +370 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/ServicesPage.jsx +86 -0
- oblako-0.1.0/oblako/dashboard/frontend/src/pages/StepFunctionsPage.jsx +222 -0
- oblako-0.1.0/oblako/dashboard/frontend/vite.config.js +7 -0
- oblako-0.1.0/oblako/engines/__init__.py +4 -0
- oblako-0.1.0/oblako/engines/agentcore/__init__.py +108 -0
- oblako-0.1.0/oblako/engines/appconfig/__init__.py +85 -0
- oblako-0.1.0/oblako/engines/appconfig/agent.py +176 -0
- oblako-0.1.0/oblako/engines/appconfig/app.py +280 -0
- oblako-0.1.0/oblako/engines/appconfig/engine.py +302 -0
- oblako-0.1.0/oblako/engines/appconfig/rule_evaluator.py +305 -0
- oblako-0.1.0/oblako/engines/athena/__init__.py +85 -0
- oblako-0.1.0/oblako/engines/athena/app.py +394 -0
- oblako-0.1.0/oblako/engines/athena/catalogs.py +44 -0
- oblako-0.1.0/oblako/engines/athena/ctas.py +131 -0
- oblako-0.1.0/oblako/engines/athena/workgroups.py +167 -0
- oblako-0.1.0/oblako/engines/bedrock/__init__.py +1 -0
- oblako-0.1.0/oblako/engines/bedrock/adapter.py +299 -0
- oblako-0.1.0/oblako/engines/bedrock/backends.py +211 -0
- oblako-0.1.0/oblako/engines/bedrock/eventstream.py +45 -0
- oblako-0.1.0/oblako/engines/bedrock/foundation_models.py +190 -0
- oblako-0.1.0/oblako/engines/bedrock/guardrails.py +174 -0
- oblako-0.1.0/oblako/engines/bedrock/jobs.py +197 -0
- oblako-0.1.0/oblako/engines/bedrock/models.py +142 -0
- oblako-0.1.0/oblako/engines/bedrock/ollama_client.py +75 -0
- oblako-0.1.0/oblako/engines/bedrock_runtime/__init__.py +63 -0
- oblako-0.1.0/oblako/engines/bedrock_runtime/app.py +388 -0
- oblako-0.1.0/oblako/engines/cloudformation/__init__.py +80 -0
- oblako-0.1.0/oblako/engines/cloudformation/app.py +361 -0
- oblako-0.1.0/oblako/engines/cloudformation/engine.py +601 -0
- oblako-0.1.0/oblako/engines/cloudformation/providers.py +645 -0
- oblako-0.1.0/oblako/engines/cloudformation/transform.py +113 -0
- oblako-0.1.0/oblako/engines/dynamodb_vectors/__init__.py +92 -0
- oblako-0.1.0/oblako/engines/dynamodb_vectors/app.py +685 -0
- oblako-0.1.0/oblako/engines/ecs_control/__init__.py +76 -0
- oblako-0.1.0/oblako/engines/ecs_control/app.py +136 -0
- oblako-0.1.0/oblako/engines/ecs_metadata/__init__.py +149 -0
- oblako-0.1.0/oblako/engines/eventbridge/__init__.py +84 -0
- oblako-0.1.0/oblako/engines/eventbridge/app.py +503 -0
- oblako-0.1.0/oblako/engines/firehose/__init__.py +90 -0
- oblako-0.1.0/oblako/engines/firehose/app.py +829 -0
- oblako-0.1.0/oblako/engines/glue_catalog/__init__.py +788 -0
- oblako-0.1.0/oblako/engines/glue_catalog/crawlers.py +942 -0
- oblako-0.1.0/oblako/engines/glue_catalog/expression.py +182 -0
- oblako-0.1.0/oblako/engines/glue_catalog/jobs.py +297 -0
- oblako-0.1.0/oblako/engines/glue_catalog/store.py +287 -0
- oblako-0.1.0/oblako/engines/glue_catalog/workflows.py +647 -0
- oblako-0.1.0/oblako/engines/host.py +133 -0
- oblako-0.1.0/oblako/engines/iam/__init__.py +5 -0
- oblako-0.1.0/oblako/engines/iam/evaluator.py +98 -0
- oblako-0.1.0/oblako/engines/identity.py +68 -0
- oblako-0.1.0/oblako/engines/lambda_shim/__init__.py +142 -0
- oblako-0.1.0/oblako/engines/mlflow/Dockerfile +27 -0
- oblako-0.1.0/oblako/engines/mlflow/__init__.py +9 -0
- oblako-0.1.0/oblako/engines/mwaa/__init__.py +81 -0
- oblako-0.1.0/oblako/engines/mwaa/app.py +176 -0
- oblako-0.1.0/oblako/engines/mwaa/environments.py +694 -0
- oblako-0.1.0/oblako/engines/rds_control/__init__.py +85 -0
- oblako-0.1.0/oblako/engines/rds_control/app.py +277 -0
- oblako-0.1.0/oblako/engines/rds_control/instances.py +409 -0
- oblako-0.1.0/oblako/engines/rds_data/__init__.py +67 -0
- oblako-0.1.0/oblako/engines/rds_data/app.py +147 -0
- oblako-0.1.0/oblako/engines/rds_data/executor.py +288 -0
- oblako-0.1.0/oblako/engines/redshift_control/__init__.py +85 -0
- oblako-0.1.0/oblako/engines/redshift_control/app.py +269 -0
- oblako-0.1.0/oblako/engines/redshift_control/clusters.py +356 -0
- oblako-0.1.0/oblako/engines/redshift_control/credentials.py +135 -0
- oblako-0.1.0/oblako/engines/redshift_control/serverless.py +377 -0
- oblako-0.1.0/oblako/engines/redshift_data/__init__.py +71 -0
- oblako-0.1.0/oblako/engines/redshift_data/app.py +264 -0
- oblako-0.1.0/oblako/engines/redshift_data/executor.py +373 -0
- oblako-0.1.0/oblako/engines/redshift_ml/__init__.py +85 -0
- oblako-0.1.0/oblako/engines/s3_ext/__init__.py +738 -0
- oblako-0.1.0/oblako/engines/s3_ext/inventory.py +218 -0
- oblako-0.1.0/oblako/engines/s3_ext/notifications.py +308 -0
- oblako-0.1.0/oblako/engines/s3tables/__init__.py +485 -0
- oblako-0.1.0/oblako/engines/s3tables/iceberg.py +185 -0
- oblako-0.1.0/oblako/engines/s3vectors/__init__.py +447 -0
- oblako-0.1.0/oblako/engines/sagemaker/__init__.py +74 -0
- oblako-0.1.0/oblako/engines/sagemaker/app.py +649 -0
- oblako-0.1.0/oblako/engines/sagemaker/executor.py +1997 -0
- oblako-0.1.0/oblako/engines/sagemaker/stubs.py +113 -0
- oblako-0.1.0/oblako/images/notebook/Dockerfile +23 -0
- oblako-0.1.0/oblako/images/notebook/aws-config +6 -0
- oblako-0.1.0/oblako/images/redshift/Dockerfile +91 -0
- oblako-0.1.0/oblako/images/redshift/entrypoint.sh +84 -0
- oblako-0.1.0/oblako/images/redshift/ext/Makefile +4 -0
- oblako-0.1.0/oblako/images/redshift/ext/oblako_redshift.c +80 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/00_extensions.sql +14 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/01_system_tables.sql +32 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/02_functions.sql +56 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/03_date_functions.sql +137 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/04_template1_date_functions.sql +5 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/05_catalog_views.sql +207 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/06_propagate_catalog_views.sql +16 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/07_copy_unload.sql +41 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/08_super.sql +72 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/09_redshift_ml.sql +97 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/10_propagate_redshift_ml.sql +7 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/11_integer_avg.sql +117 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/12_password_auth.sh +11 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/13_iceberg.sql +271 -0
- oblako-0.1.0/oblako/images/redshift/initdb.d/99_system_catalog.sql +172 -0
- oblako-0.1.0/oblako/images/redshift/ml_train/Dockerfile +10 -0
- oblako-0.1.0/oblako/images/redshift/ml_train/train.py +300 -0
- oblako-0.1.0/oblako/images/redshift/proxy/copy_unload.py +895 -0
- oblako-0.1.0/oblako/images/redshift/proxy/datepart.py +76 -0
- oblako-0.1.0/oblako/images/redshift/proxy/iceberg_tables.py +1477 -0
- oblako-0.1.0/oblako/images/redshift/proxy/integer_avg.py +86 -0
- oblako-0.1.0/oblako/images/redshift/proxy/listagg.py +130 -0
- oblako-0.1.0/oblako/images/redshift/proxy/pivot_unpivot.py +127 -0
- oblako-0.1.0/oblako/images/redshift/proxy/redshift_ml.py +966 -0
- oblako-0.1.0/oblako/images/redshift/proxy/redshift_proxy.py +800 -0
- oblako-0.1.0/oblako/images/redshift/proxy/super_nav.py +329 -0
- oblako-0.1.0/oblako/images/redshift-cluster/000_citus.sql +3 -0
- oblako-0.1.0/oblako/images/redshift-cluster/13_svv_table_info.sql +80 -0
- oblako-0.1.0/oblako/images/redshift-cluster/Dockerfile +55 -0
- oblako-0.1.0/oblako/images/redshift-cluster/cluster-entrypoint.sh +46 -0
- oblako-0.1.0/oblako/notebook.py +299 -0
- oblako-0.1.0/oblako/ports.py +89 -0
- oblako-0.1.0/oblako/profile.py +160 -0
- oblako-0.1.0/oblako/services/__init__.py +39 -0
- oblako-0.1.0/oblako/services/appconfig.py +62 -0
- oblako-0.1.0/oblako/services/awslambda.py +106 -0
- oblako-0.1.0/oblako/services/backends.py +730 -0
- oblako-0.1.0/oblako/services/base.py +145 -0
- oblako-0.1.0/oblako/services/bedrock.py +139 -0
- oblako-0.1.0/oblako/services/boto.py +79 -0
- oblako-0.1.0/oblako/services/caddy.py +102 -0
- oblako-0.1.0/oblako/services/cloudformation.py +50 -0
- oblako-0.1.0/oblako/services/dynamodb.py +67 -0
- oblako-0.1.0/oblako/services/ec2.py +210 -0
- oblako-0.1.0/oblako/services/ecs.py +640 -0
- oblako-0.1.0/oblako/services/elbv2.py +256 -0
- oblako-0.1.0/oblako/services/glue.py +260 -0
- oblako-0.1.0/oblako/services/glue_assets/S3ProxyCompatInterceptor.java +89 -0
- oblako-0.1.0/oblako/services/glue_catalog.py +40 -0
- oblako-0.1.0/oblako/services/iam.py +130 -0
- oblako-0.1.0/oblako/services/iceberg.py +108 -0
- oblako-0.1.0/oblako/services/kinesis.py +55 -0
- oblako-0.1.0/oblako/services/mlflow.py +170 -0
- oblako-0.1.0/oblako/services/moto.py +76 -0
- oblako-0.1.0/oblako/services/opensearch.py +72 -0
- oblako-0.1.0/oblako/services/platform.py +150 -0
- oblako-0.1.0/oblako/services/rds.py +240 -0
- oblako-0.1.0/oblako/services/redshift.py +502 -0
- oblako-0.1.0/oblako/services/s3proxy.py +267 -0
- oblako-0.1.0/oblako/services/sagemaker.py +460 -0
- oblako-0.1.0/oblako/services/sfn_templates.py +454 -0
- oblako-0.1.0/oblako/services/stepfunctions.py +102 -0
- oblako-0.1.0/oblako/services/trino.py +156 -0
- oblako-0.1.0/pyproject.toml +209 -0
- oblako-0.1.0/tests/__init__.py +0 -0
- oblako-0.1.0/tests/appconfig/__init__.py +0 -0
- oblako-0.1.0/tests/appconfig/test_appconfig.py +148 -0
- oblako-0.1.0/tests/awslambda/__init__.py +0 -0
- oblako-0.1.0/tests/awslambda/test_lambda.py +152 -0
- oblako-0.1.0/tests/bedrock/__init__.py +0 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_adapter.py +128 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_backends.py +135 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_control.py +135 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_guardrails.py +83 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_openrouter_live.py +32 -0
- oblako-0.1.0/tests/bedrock/test_bedrock_runtime.py +99 -0
- oblako-0.1.0/tests/bedrock/test_ollama.py +65 -0
- oblako-0.1.0/tests/cloudformation/__init__.py +0 -0
- oblako-0.1.0/tests/cloudformation/test_cloudformation.py +864 -0
- oblako-0.1.0/tests/conftest.py +90 -0
- oblako-0.1.0/tests/dynamodb/test_dynamodb_vectors.py +354 -0
- oblako-0.1.0/tests/ecs/__init__.py +0 -0
- oblako-0.1.0/tests/ecs/test_ecs.py +210 -0
- oblako-0.1.0/tests/ecs/test_ecs_control.py +158 -0
- oblako-0.1.0/tests/ecs/test_fargate_sizes.py +50 -0
- oblako-0.1.0/tests/iam/__init__.py +0 -0
- oblako-0.1.0/tests/iam/test_iam_evaluator.py +88 -0
- oblako-0.1.0/tests/mwaa/__init__.py +0 -0
- oblako-0.1.0/tests/mwaa/test_mwaa_api.py +100 -0
- oblako-0.1.0/tests/mwaa/test_mwaa_environment.py +123 -0
- oblako-0.1.0/tests/notebook/__init__.py +0 -0
- oblako-0.1.0/tests/notebook/test_notebook.py +67 -0
- oblako-0.1.0/tests/opensearch/__init__.py +0 -0
- oblako-0.1.0/tests/opensearch/test_opensearch.py +87 -0
- oblako-0.1.0/tests/ports.py +10 -0
- oblako-0.1.0/tests/rds/__init__.py +0 -0
- oblako-0.1.0/tests/rds/test_rds.py +230 -0
- oblako-0.1.0/tests/rds/test_rds_data.py +220 -0
- oblako-0.1.0/tests/rds/test_rds_replication.py +186 -0
- oblako-0.1.0/tests/redshift/__init__.py +0 -0
- oblako-0.1.0/tests/redshift/test_redshift.py +87 -0
- oblako-0.1.0/tests/redshift/test_redshift_avg.py +79 -0
- oblako-0.1.0/tests/redshift/test_redshift_avg_unit.py +31 -0
- oblako-0.1.0/tests/redshift/test_redshift_awswrangler.py +256 -0
- oblako-0.1.0/tests/redshift/test_redshift_catalog.py +92 -0
- oblako-0.1.0/tests/redshift/test_redshift_cluster.py +237 -0
- oblako-0.1.0/tests/redshift/test_redshift_copy_unload.py +448 -0
- oblako-0.1.0/tests/redshift/test_redshift_copy_unload_unit.py +303 -0
- oblako-0.1.0/tests/redshift/test_redshift_data.py +137 -0
- oblako-0.1.0/tests/redshift/test_redshift_datepart_unit.py +51 -0
- oblako-0.1.0/tests/redshift/test_redshift_functions.py +170 -0
- oblako-0.1.0/tests/redshift/test_redshift_iceberg.py +488 -0
- oblako-0.1.0/tests/redshift/test_redshift_iceberg_unit.py +158 -0
- oblako-0.1.0/tests/redshift/test_redshift_listagg.py +81 -0
- oblako-0.1.0/tests/redshift/test_redshift_listagg_unit.py +70 -0
- oblako-0.1.0/tests/redshift/test_redshift_ml.py +318 -0
- oblako-0.1.0/tests/redshift/test_redshift_ml_unit.py +283 -0
- oblako-0.1.0/tests/redshift/test_redshift_multinode.py +164 -0
- oblako-0.1.0/tests/redshift/test_redshift_pivot.py +87 -0
- oblako-0.1.0/tests/redshift/test_redshift_pivot_unit.py +55 -0
- oblako-0.1.0/tests/redshift/test_redshift_proxy.py +124 -0
- oblako-0.1.0/tests/redshift/test_redshift_proxy_unit.py +326 -0
- oblako-0.1.0/tests/redshift/test_redshift_psycopg3.py +129 -0
- oblako-0.1.0/tests/redshift/test_redshift_redtape.py +219 -0
- oblako-0.1.0/tests/redshift/test_redshift_serverless.py +218 -0
- oblako-0.1.0/tests/redshift/test_redshift_sqlalchemy.py +192 -0
- oblako-0.1.0/tests/redshift/test_redshift_super.py +148 -0
- oblako-0.1.0/tests/redshift/test_redshift_super_unit.py +136 -0
- oblako-0.1.0/tests/redshift/test_redshift_trust.py +20 -0
- oblako-0.1.0/tests/s3/__init__.py +0 -0
- oblako-0.1.0/tests/s3/test_s3_addressing.py +86 -0
- oblako-0.1.0/tests/s3/test_s3_extensions.py +414 -0
- oblako-0.1.0/tests/s3/test_s3proxy.py +98 -0
- oblako-0.1.0/tests/s3tables/test_s3tables.py +311 -0
- oblako-0.1.0/tests/s3vectors/test_s3vectors.py +176 -0
- oblako-0.1.0/tests/sagemaker/__init__.py +0 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker.py +54 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_async.py +174 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_async_sns.py +98 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_control.py +127 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_endpoint.py +151 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_featurestore.py +166 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_metadata.py +127 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_mlflow.py +140 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_mme.py +153 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_monitor.py +251 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_processing.py +112 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_registry.py +105 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_sdk_v3.py +170 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_stubs.py +41 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_transform.py +145 -0
- oblako-0.1.0/tests/sagemaker/test_sagemaker_tuning.py +167 -0
- oblako-0.1.0/tests/sagemaker/test_studio_domain.py +64 -0
- oblako-0.1.0/tests/services/__init__.py +0 -0
- oblako-0.1.0/tests/services/test_athena.py +81 -0
- oblako-0.1.0/tests/services/test_athena_s3tables.py +99 -0
- oblako-0.1.0/tests/services/test_athena_unit.py +191 -0
- oblako-0.1.0/tests/services/test_aws_namespacing.py +17 -0
- oblako-0.1.0/tests/services/test_backends.py +195 -0
- oblako-0.1.0/tests/services/test_caddy.py +27 -0
- oblako-0.1.0/tests/services/test_config.py +36 -0
- oblako-0.1.0/tests/services/test_ec2.py +65 -0
- oblako-0.1.0/tests/services/test_engine_host.py +71 -0
- oblako-0.1.0/tests/services/test_engine_identity.py +58 -0
- oblako-0.1.0/tests/services/test_eventbridge.py +280 -0
- oblako-0.1.0/tests/services/test_firehose.py +409 -0
- oblako-0.1.0/tests/services/test_firehose_unit.py +110 -0
- oblako-0.1.0/tests/services/test_glue.py +15 -0
- oblako-0.1.0/tests/services/test_glue_athena.py +154 -0
- oblako-0.1.0/tests/services/test_glue_catalog.py +518 -0
- oblako-0.1.0/tests/services/test_glue_crawlers.py +251 -0
- oblako-0.1.0/tests/services/test_glue_jobs.py +113 -0
- oblako-0.1.0/tests/services/test_glue_output.py +68 -0
- oblako-0.1.0/tests/services/test_glue_workflow.py +65 -0
- oblako-0.1.0/tests/services/test_glue_workflows.py +266 -0
- oblako-0.1.0/tests/services/test_iceberg.py +22 -0
- oblako-0.1.0/tests/services/test_k8s_backend.py +72 -0
- oblako-0.1.0/tests/services/test_kinesis.py +24 -0
- oblako-0.1.0/tests/services/test_kinesis_streams.py +46 -0
- oblako-0.1.0/tests/services/test_mlflow.py +55 -0
- oblako-0.1.0/tests/services/test_moto_backed_services.py +137 -0
- oblako-0.1.0/tests/services/test_rds_control.py +66 -0
- oblako-0.1.0/tests/services/test_redshift_control.py +64 -0
- oblako-0.1.0/tests/services/test_redshift_tls.py +100 -0
- oblako-0.1.0/tests/services/test_s3_ext_unit.py +215 -0
- oblako-0.1.0/tests/services/test_services.py +254 -0
- oblako-0.1.0/tests/services/test_trino.py +30 -0
- oblako-0.1.0/tests/stepfunctions/__init__.py +0 -0
- oblako-0.1.0/tests/stepfunctions/test_sfn_templates.py +72 -0
- oblako-0.1.0/tests/stepfunctions/test_stepfunctions.py +169 -0
- oblako-0.1.0/tests/test_banner.py +37 -0
- oblako-0.1.0/tests/test_cli_up.py +113 -0
- oblako-0.1.0/tests/test_port_overrides.py +56 -0
- oblako-0.1.0/tests/test_profile.py +97 -0
- oblako-0.1.0/typings/catboost.pyi +17 -0
- oblako-0.1.0/typings/joblib.pyi +6 -0
- oblako-0.1.0/uv.lock +7669 -0
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
#
|
|
2
|
+
# oblako's CloudFormation conventions for the example stacks under examples/.
|
|
3
|
+
# Not a compliance pack: each rule encodes how an oblako example is meant to run
|
|
4
|
+
# against the local control plane (moto) and tear down cleanly, so a failure means
|
|
5
|
+
# a template drifted from that, not a policy opinion.
|
|
6
|
+
#
|
|
7
|
+
|
|
8
|
+
# ECS runs on Fargate. oblako's local ECS backend runs the container directly;
|
|
9
|
+
# there is no EC2 capacity to launch onto.
|
|
10
|
+
let ecs_services = Resources.*[ Type == 'AWS::ECS::Service' ]
|
|
11
|
+
rule ECS_IS_FARGATE when %ecs_services !empty {
|
|
12
|
+
%ecs_services.Properties.LaunchType == 'FARGATE'
|
|
13
|
+
<< Violation: oblako runs ECS on Fargate; set LaunchType to FARGATE. >>
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
# Lambda / SAM functions pin a runtime oblako's local Lambda image supports (the
|
|
17
|
+
# shogo82148 AL2023 images), so `sam local` and the local Lambda runner agree.
|
|
18
|
+
let sam_functions = Resources.*[ Type == 'AWS::Serverless::Function' ]
|
|
19
|
+
rule LAMBDA_SUPPORTED_RUNTIME when %sam_functions !empty {
|
|
20
|
+
%sam_functions.Properties.Runtime IN ['python3.11', 'python3.12', 'python3.13', 'nodejs20.x', 'nodejs22.x']
|
|
21
|
+
<< Violation: pin a runtime oblako's local Lambda image supports (e.g. python3.12). >>
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
# Log groups keep a bounded retention, so repeated local runs do not pile up.
|
|
25
|
+
let log_groups = Resources.*[ Type == 'AWS::Logs::LogGroup' ]
|
|
26
|
+
rule LOG_GROUPS_HAVE_RETENTION when %log_groups !empty {
|
|
27
|
+
%log_groups.Properties.RetentionInDays exists
|
|
28
|
+
<< Violation: set RetentionInDays on log groups so local runs stay tidy. >>
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
# Teardown leaves nothing behind. Retainable resources delete with the stack, so
|
|
32
|
+
# oblako's local control plane does not keep orphans between runs.
|
|
33
|
+
let buckets = Resources.*[ Type == 'AWS::S3::Bucket' ]
|
|
34
|
+
rule S3_DELETES_WITH_THE_STACK when %buckets !empty {
|
|
35
|
+
%buckets.DeletionPolicy == 'Delete'
|
|
36
|
+
%buckets.UpdateReplacePolicy == 'Delete'
|
|
37
|
+
<< Violation: set DeletionPolicy and UpdateReplacePolicy to Delete on buckets. >>
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
rule S3_PUBLIC_ACCESS_BLOCKED when %buckets !empty {
|
|
41
|
+
%buckets.Properties.PublicAccessBlockConfiguration.BlockPublicAcls == true
|
|
42
|
+
%buckets.Properties.PublicAccessBlockConfiguration.BlockPublicPolicy == true
|
|
43
|
+
%buckets.Properties.PublicAccessBlockConfiguration.IgnorePublicAcls == true
|
|
44
|
+
%buckets.Properties.PublicAccessBlockConfiguration.RestrictPublicBuckets == true
|
|
45
|
+
<< Violation: buckets block all public access. >>
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
let ecr_repos = Resources.*[ Type == 'AWS::ECR::Repository' ]
|
|
49
|
+
rule ECR_EMPTY_ON_DELETE when %ecr_repos !empty {
|
|
50
|
+
%ecr_repos.Properties.EmptyOnDelete == true
|
|
51
|
+
<< Violation: set EmptyOnDelete so a repo with images does not block stack deletion. >>
|
|
52
|
+
}
|
|
@@ -0,0 +1,213 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
# Pull requests only: a merge to main brings the commits its pull request already
|
|
4
|
+
# tested. "Run workflow" (workflow_dispatch) runs every job on demand.
|
|
5
|
+
on:
|
|
6
|
+
pull_request:
|
|
7
|
+
workflow_dispatch:
|
|
8
|
+
|
|
9
|
+
# A newer push to the same pull request cancels the run it supersedes.
|
|
10
|
+
concurrency:
|
|
11
|
+
group: ci-${{ github.event.pull_request.number || github.ref }}
|
|
12
|
+
cancel-in-progress: true
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
# Which heavy jobs this change needs: each runs only when its code changed
|
|
16
|
+
# (or on a manual run), so a docs or example change costs only lint + unit.
|
|
17
|
+
# A change to this workflow runs no heavy job by itself; check it with
|
|
18
|
+
# "Run workflow".
|
|
19
|
+
changes:
|
|
20
|
+
name: changed areas
|
|
21
|
+
runs-on: ubuntu-latest
|
|
22
|
+
# paths-filter lists a pull request's files through the API
|
|
23
|
+
permissions:
|
|
24
|
+
contents: read
|
|
25
|
+
pull-requests: read
|
|
26
|
+
outputs:
|
|
27
|
+
integration: ${{ steps.filter.outputs.integration }}
|
|
28
|
+
cluster: ${{ steps.filter.outputs.cluster }}
|
|
29
|
+
kubernetes: ${{ steps.filter.outputs.kubernetes }}
|
|
30
|
+
steps:
|
|
31
|
+
- uses: actions/checkout@v4
|
|
32
|
+
- uses: dorny/paths-filter@v3
|
|
33
|
+
id: filter
|
|
34
|
+
with:
|
|
35
|
+
filters: |
|
|
36
|
+
integration:
|
|
37
|
+
- "oblako/**"
|
|
38
|
+
- "tests/**"
|
|
39
|
+
- "pyproject.toml"
|
|
40
|
+
- "uv.lock"
|
|
41
|
+
- "docker-compose.yml"
|
|
42
|
+
cluster:
|
|
43
|
+
- "oblako/images/redshift/**"
|
|
44
|
+
- "oblako/images/redshift-cluster/**"
|
|
45
|
+
- "oblako/engines/redshift_control/**"
|
|
46
|
+
- "oblako/engines/redshift_data/**"
|
|
47
|
+
- "tests/redshift/test_redshift_multinode.py"
|
|
48
|
+
kubernetes:
|
|
49
|
+
- "oblako/services/backends.py"
|
|
50
|
+
- "tests/services/test_k8s_backend.py"
|
|
51
|
+
|
|
52
|
+
unit:
|
|
53
|
+
name: lint + unit tests
|
|
54
|
+
runs-on: ubuntu-latest
|
|
55
|
+
steps:
|
|
56
|
+
- uses: actions/checkout@v4
|
|
57
|
+
|
|
58
|
+
- name: Install uv
|
|
59
|
+
uses: astral-sh/setup-uv@v5
|
|
60
|
+
with:
|
|
61
|
+
enable-cache: true
|
|
62
|
+
|
|
63
|
+
- name: Lint (ruff)
|
|
64
|
+
run: uv run --python 3.12 --extra dev ruff check oblako tests
|
|
65
|
+
|
|
66
|
+
# Docker-free subset: anything not auto-marked `integration` or `kubernetes`
|
|
67
|
+
# by tests/conftest.py is picked up — new tests Just Work without ci.yml edits.
|
|
68
|
+
- name: Unit tests
|
|
69
|
+
run: >
|
|
70
|
+
uv run --python 3.12 --extra dev python -m pytest -v
|
|
71
|
+
-m "not integration and not kubernetes and not cluster"
|
|
72
|
+
|
|
73
|
+
integration:
|
|
74
|
+
name: integration tests (Docker services)
|
|
75
|
+
needs: changes
|
|
76
|
+
if: needs.changes.outputs.integration == 'true' || github.event_name == 'workflow_dispatch'
|
|
77
|
+
runs-on: ubuntu-latest
|
|
78
|
+
steps:
|
|
79
|
+
- uses: actions/checkout@v4
|
|
80
|
+
|
|
81
|
+
# The sagemaker extra pulls in torch + CUDA wheels (via sagemaker-serve),
|
|
82
|
+
# several GB, and the tests then build training images; without room the
|
|
83
|
+
# image builds fail in pip. The runner image ships toolchains this job
|
|
84
|
+
# never uses, so drop them first.
|
|
85
|
+
- name: Free disk space
|
|
86
|
+
run: |
|
|
87
|
+
df -h /
|
|
88
|
+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc \
|
|
89
|
+
/usr/local/.ghcup /opt/hostedtoolcache/CodeQL
|
|
90
|
+
docker image prune -af
|
|
91
|
+
df -h /
|
|
92
|
+
|
|
93
|
+
- name: Install uv
|
|
94
|
+
uses: astral-sh/setup-uv@v5
|
|
95
|
+
with:
|
|
96
|
+
enable-cache: true
|
|
97
|
+
|
|
98
|
+
# Build the redshift image from source and tag it as the pull target, so
|
|
99
|
+
# integration tests run against the code under change (RedshiftService uses
|
|
100
|
+
# the local image when present, no pull). Closes the publish-lag gap where
|
|
101
|
+
# an image change lands before the registry republishes.
|
|
102
|
+
- name: Build redshift image
|
|
103
|
+
run: docker build -t public.ecr.aws/oblako/redshift-local:16 oblako/images/redshift
|
|
104
|
+
|
|
105
|
+
# Start the oblako services and wait for readiness (up to 180s each, for
|
|
106
|
+
# image pulls + DB/JVM init; OpenSearch is the slow one). Trino also starts
|
|
107
|
+
# the Glue engine, its awsdatacatalog metastore. Ollama/Bedrock
|
|
108
|
+
# is not run here: it needs large model downloads. The sagemaker extra
|
|
109
|
+
# (the v3 SDK) lets the SageMaker SDK tests drive its local modes.
|
|
110
|
+
- name: Start services
|
|
111
|
+
run: >
|
|
112
|
+
uv run --python 3.12 --extra dev --extra sagemaker python -c "from oblako.services import Oblako as O; import sys;
|
|
113
|
+
o = O(); svcs = [o.s3, o.dynamodb, o.kinesis, o.moto, o.stepfunctions, o.redshift, o.rds, o.opensearch, o.iceberg, o.trino];
|
|
114
|
+
[s.start() for s in svcs];
|
|
115
|
+
r = {s.name: s.wait_ready(timeout=180) for s in svcs};
|
|
116
|
+
print('ready:', r); sys.exit(0 if all(r.values()) else 1)"
|
|
117
|
+
|
|
118
|
+
# Trust oblako's Redshift TLS cert in the test environment, as a reader does,
|
|
119
|
+
# so redshift_connector connects with sslmode=verify-ca.
|
|
120
|
+
- name: Trust the Redshift certificate
|
|
121
|
+
run: uv run --python 3.12 --extra dev --extra sagemaker oblako trust
|
|
122
|
+
|
|
123
|
+
# MySQL (test_rds_data mysql cases) isn't started, so those skip; the Step
|
|
124
|
+
# Functions tests use Pass-only state machines (no Lambda).
|
|
125
|
+
- name: Integration tests
|
|
126
|
+
run: >
|
|
127
|
+
uv run --python 3.12 --extra dev --extra sagemaker python -m pytest -v
|
|
128
|
+
-m integration
|
|
129
|
+
|
|
130
|
+
- name: Disk usage after tests
|
|
131
|
+
if: always()
|
|
132
|
+
run: df -h /
|
|
133
|
+
|
|
134
|
+
redshift-cluster:
|
|
135
|
+
name: redshift MPP cluster (Citus)
|
|
136
|
+
needs: changes
|
|
137
|
+
if: needs.changes.outputs.cluster == 'true' || github.event_name == 'workflow_dispatch'
|
|
138
|
+
runs-on: ubuntu-latest
|
|
139
|
+
steps:
|
|
140
|
+
- uses: actions/checkout@v4
|
|
141
|
+
|
|
142
|
+
- name: Install uv
|
|
143
|
+
uses: astral-sh/setup-uv@v5
|
|
144
|
+
with:
|
|
145
|
+
enable-cache: true
|
|
146
|
+
|
|
147
|
+
# The cluster image builds FROM the single-node image (COPY --from), so
|
|
148
|
+
# build + tag that first; buildkit then resolves it from the local store.
|
|
149
|
+
- name: Build redshift image
|
|
150
|
+
run: docker build -t public.ecr.aws/oblako/redshift-local:16 oblako/images/redshift
|
|
151
|
+
|
|
152
|
+
# Coordinator + 2 workers, named explicitly so compose starts only the
|
|
153
|
+
# cluster (a bare `--profile cluster up` would also start the single-node
|
|
154
|
+
# `redshift` service, colliding on 5439).
|
|
155
|
+
- name: Build + start the Citus cluster
|
|
156
|
+
run: >
|
|
157
|
+
docker compose --profile cluster up -d --build
|
|
158
|
+
redshift-coordinator redshift-w1 redshift-w2
|
|
159
|
+
|
|
160
|
+
- name: Wait for the workers to register
|
|
161
|
+
run: |
|
|
162
|
+
for _ in $(seq 1 60); do
|
|
163
|
+
n=$(docker compose exec -T redshift-coordinator \
|
|
164
|
+
psql -U oblako -d oblako -h 127.0.0.1 -p 5433 -tAc \
|
|
165
|
+
"SELECT count(*) FROM pg_dist_node WHERE isactive" 2>/dev/null \
|
|
166
|
+
| tr -d '[:space:]')
|
|
167
|
+
echo "active Citus nodes: ${n:-0}"
|
|
168
|
+
[ "${n:-0}" -ge 3 ] && exit 0
|
|
169
|
+
sleep 3
|
|
170
|
+
done
|
|
171
|
+
echo "cluster did not form"; docker compose logs redshift-coordinator | tail -40; exit 1
|
|
172
|
+
|
|
173
|
+
- name: Trust the Redshift certificate
|
|
174
|
+
run: uv run --python 3.12 --extra dev oblako trust
|
|
175
|
+
|
|
176
|
+
- name: Cluster tests
|
|
177
|
+
run: >
|
|
178
|
+
uv run --python 3.12 --extra dev python -m pytest -v
|
|
179
|
+
tests/redshift/test_redshift_cluster.py
|
|
180
|
+
|
|
181
|
+
# A multi-node cluster created through the Redshift API (moto behind the
|
|
182
|
+
# redshift-control proxy) runs from the image compose just built.
|
|
183
|
+
- name: Start moto
|
|
184
|
+
run: >
|
|
185
|
+
uv run --python 3.12 --extra dev python -c "from oblako.services import Oblako as O;
|
|
186
|
+
m = O().moto; m.start(); import sys; sys.exit(0 if m.wait_ready(timeout=120) else 1)"
|
|
187
|
+
|
|
188
|
+
- name: Multi-node cluster through the Redshift API
|
|
189
|
+
run: >
|
|
190
|
+
uv run --python 3.12 --extra dev python -m pytest -v -m cluster
|
|
191
|
+
|
|
192
|
+
kubernetes:
|
|
193
|
+
name: kubernetes backend (kind)
|
|
194
|
+
needs: changes
|
|
195
|
+
if: needs.changes.outputs.kubernetes == 'true' || github.event_name == 'workflow_dispatch'
|
|
196
|
+
runs-on: ubuntu-latest
|
|
197
|
+
steps:
|
|
198
|
+
- uses: actions/checkout@v4
|
|
199
|
+
|
|
200
|
+
- name: Install uv
|
|
201
|
+
uses: astral-sh/setup-uv@v5
|
|
202
|
+
with:
|
|
203
|
+
enable-cache: true
|
|
204
|
+
|
|
205
|
+
# Provision a real cluster so the Kubernetes backend's run/port-forward path
|
|
206
|
+
# (not just the manifest builder) is exercised end to end.
|
|
207
|
+
- name: Create kind cluster
|
|
208
|
+
uses: helm/kind-action@v1
|
|
209
|
+
|
|
210
|
+
- name: Kubernetes backend test
|
|
211
|
+
run: >
|
|
212
|
+
uv run --python 3.12 --extra dev python -m pytest -v
|
|
213
|
+
-m kubernetes
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
name: docs
|
|
2
|
+
|
|
3
|
+
# Build the Sphinx site and publish it to S3 + CloudFront
|
|
4
|
+
# (https://oblako-sdk.almostly.ai/). Authenticates via GitHub OIDC to the
|
|
5
|
+
# least-privilege role `oblako-docs-deploy` (no static AWS keys).
|
|
6
|
+
|
|
7
|
+
on:
|
|
8
|
+
push:
|
|
9
|
+
branches: [main]
|
|
10
|
+
paths:
|
|
11
|
+
- "docs/**"
|
|
12
|
+
- "oblako/services/**" # api.md autodocs the service classes
|
|
13
|
+
- ".github/workflows/docs.yml"
|
|
14
|
+
workflow_dispatch:
|
|
15
|
+
|
|
16
|
+
permissions:
|
|
17
|
+
contents: read
|
|
18
|
+
id-token: write # required for GitHub OIDC -> AWS
|
|
19
|
+
|
|
20
|
+
concurrency:
|
|
21
|
+
group: docs
|
|
22
|
+
cancel-in-progress: true
|
|
23
|
+
|
|
24
|
+
env:
|
|
25
|
+
AWS_REGION: us-east-1
|
|
26
|
+
S3_TARGET: s3://oblako-public/docs/
|
|
27
|
+
DISTRIBUTION_ID: E9NGABBAAK9CF
|
|
28
|
+
|
|
29
|
+
jobs:
|
|
30
|
+
build-and-deploy:
|
|
31
|
+
runs-on: ubuntu-latest
|
|
32
|
+
steps:
|
|
33
|
+
- uses: actions/checkout@v4
|
|
34
|
+
|
|
35
|
+
- uses: actions/setup-python@v5
|
|
36
|
+
with:
|
|
37
|
+
python-version: "3.12"
|
|
38
|
+
|
|
39
|
+
- name: Install docs dependencies
|
|
40
|
+
run: pip install -e ".[docs]"
|
|
41
|
+
|
|
42
|
+
- name: Build the Sphinx site
|
|
43
|
+
run: sphinx-build -b html -W --keep-going docs docs/_build/html
|
|
44
|
+
|
|
45
|
+
- name: Configure AWS credentials (OIDC)
|
|
46
|
+
uses: aws-actions/configure-aws-credentials@v4
|
|
47
|
+
with:
|
|
48
|
+
# Role ARN kept in a repo secret so the account ID isn't in the
|
|
49
|
+
# public source (and is masked in Actions logs). Value:
|
|
50
|
+
# arn:aws:iam::<account-id>:role/oblako-docs-deploy
|
|
51
|
+
role-to-assume: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
|
|
52
|
+
aws-region: ${{ env.AWS_REGION }}
|
|
53
|
+
|
|
54
|
+
- name: Sync to S3
|
|
55
|
+
run: aws s3 sync docs/_build/html "$S3_TARGET" --delete
|
|
56
|
+
|
|
57
|
+
- name: Invalidate CloudFront
|
|
58
|
+
run: |
|
|
59
|
+
aws cloudfront create-invalidation \
|
|
60
|
+
--distribution-id "$DISTRIBUTION_ID" \
|
|
61
|
+
--paths "/*"
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
name: redshift-image
|
|
2
|
+
|
|
3
|
+
# Build + publish the oblako Redshift image whenever its source changes, so
|
|
4
|
+
# `oblako up` / docker-compose can just pull it. Published to two registries:
|
|
5
|
+
# - Docker Hub: deburky/redshift-local (discoverable, human-facing)
|
|
6
|
+
# - ECR Public: public.ecr.aws/oblako/redshift-local (no pull rate limits)
|
|
7
|
+
#
|
|
8
|
+
# Needs repo secrets:
|
|
9
|
+
# DOCKERHUB_USERNAME = deburky
|
|
10
|
+
# DOCKERHUB_TOKEN = a Docker Hub access token (PAT) with Read/Write
|
|
11
|
+
# AWS_ECR_PUBLIC_ROLE_ARN = arn of the oblako-ecr-public-publish OIDC role
|
|
12
|
+
# (kept in a secret so the account id isn't in source)
|
|
13
|
+
on:
|
|
14
|
+
push:
|
|
15
|
+
branches: [main]
|
|
16
|
+
paths: ["oblako/images/redshift/**", ".github/workflows/redshift-image.yml"]
|
|
17
|
+
schedule:
|
|
18
|
+
- cron: "0 6 1 * *" # monthly (1st, 06:00 UTC): pick up postgres:16 base-image CVE fixes
|
|
19
|
+
workflow_dispatch:
|
|
20
|
+
|
|
21
|
+
permissions:
|
|
22
|
+
id-token: write # for GitHub OIDC -> the ECR Public publish role
|
|
23
|
+
contents: read
|
|
24
|
+
|
|
25
|
+
env:
|
|
26
|
+
ECR_PUBLIC_REGISTRY: public.ecr.aws/oblako
|
|
27
|
+
|
|
28
|
+
jobs:
|
|
29
|
+
publish:
|
|
30
|
+
runs-on: ubuntu-latest
|
|
31
|
+
steps:
|
|
32
|
+
- uses: actions/checkout@v4
|
|
33
|
+
|
|
34
|
+
- uses: docker/setup-qemu-action@v3 # emulation for the arm64 build leg
|
|
35
|
+
- uses: docker/setup-buildx-action@v3
|
|
36
|
+
|
|
37
|
+
- name: Log in to Docker Hub
|
|
38
|
+
uses: docker/login-action@v3
|
|
39
|
+
with:
|
|
40
|
+
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
41
|
+
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
42
|
+
|
|
43
|
+
- name: Configure AWS credentials (OIDC)
|
|
44
|
+
uses: aws-actions/configure-aws-credentials@v4
|
|
45
|
+
with:
|
|
46
|
+
role-to-assume: ${{ secrets.AWS_ECR_PUBLIC_ROLE_ARN }}
|
|
47
|
+
aws-region: us-east-1 # ECR Public's control plane is us-east-1 only
|
|
48
|
+
|
|
49
|
+
- name: Log in to ECR Public
|
|
50
|
+
uses: aws-actions/amazon-ecr-login@v2
|
|
51
|
+
with:
|
|
52
|
+
registry-type: public
|
|
53
|
+
|
|
54
|
+
- name: Build and push (multi-arch, both registries)
|
|
55
|
+
uses: docker/build-push-action@v6
|
|
56
|
+
with:
|
|
57
|
+
context: oblako/images/redshift
|
|
58
|
+
platforms: linux/amd64,linux/arm64
|
|
59
|
+
push: true
|
|
60
|
+
# no SBOM/provenance attestation manifests (the "unknown · 0 MB" rows)
|
|
61
|
+
provenance: false
|
|
62
|
+
sbom: false
|
|
63
|
+
tags: |
|
|
64
|
+
deburky/redshift-local:16
|
|
65
|
+
deburky/redshift-local:latest
|
|
66
|
+
${{ env.ECR_PUBLIC_REGISTRY }}/redshift-local:16
|
|
67
|
+
${{ env.ECR_PUBLIC_REGISTRY }}/redshift-local:latest
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# Publish oblako to PyPI when a version tag is pushed (git tag v0.1.0 && git push
|
|
2
|
+
# origin v0.1.0). Trusted publishing: PyPI trusts this workflow through GitHub's
|
|
3
|
+
# OIDC token, so no API token is stored anywhere. The publish job runs in the
|
|
4
|
+
# `pypi` environment, which can require a reviewer's approval before it starts.
|
|
5
|
+
name: release
|
|
6
|
+
|
|
7
|
+
on:
|
|
8
|
+
push:
|
|
9
|
+
tags: ["v*"]
|
|
10
|
+
|
|
11
|
+
permissions:
|
|
12
|
+
contents: read
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
build:
|
|
16
|
+
name: build and check
|
|
17
|
+
runs-on: ubuntu-latest
|
|
18
|
+
steps:
|
|
19
|
+
- uses: actions/checkout@v4
|
|
20
|
+
|
|
21
|
+
- name: Install uv
|
|
22
|
+
uses: astral-sh/setup-uv@v5
|
|
23
|
+
|
|
24
|
+
- name: The tag matches the package version
|
|
25
|
+
run: |
|
|
26
|
+
version=$(uv run --no-project python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
|
|
27
|
+
if [ "v${version}" != "${GITHUB_REF_NAME}" ]; then
|
|
28
|
+
echo "tag ${GITHUB_REF_NAME} does not match pyproject.toml version ${version}"
|
|
29
|
+
exit 1
|
|
30
|
+
fi
|
|
31
|
+
|
|
32
|
+
- name: Build the wheel and the source archive
|
|
33
|
+
run: uv build
|
|
34
|
+
|
|
35
|
+
- name: Check the distributions
|
|
36
|
+
run: uvx twine check --strict dist/*
|
|
37
|
+
|
|
38
|
+
- uses: actions/upload-artifact@v4
|
|
39
|
+
with:
|
|
40
|
+
name: dist
|
|
41
|
+
path: dist/
|
|
42
|
+
|
|
43
|
+
publish:
|
|
44
|
+
name: publish to PyPI
|
|
45
|
+
needs: build
|
|
46
|
+
runs-on: ubuntu-latest
|
|
47
|
+
environment:
|
|
48
|
+
name: pypi
|
|
49
|
+
url: https://pypi.org/project/oblako/
|
|
50
|
+
permissions:
|
|
51
|
+
id-token: write # trusted publishing
|
|
52
|
+
steps:
|
|
53
|
+
- uses: actions/download-artifact@v4
|
|
54
|
+
with:
|
|
55
|
+
name: dist
|
|
56
|
+
path: dist/
|
|
57
|
+
|
|
58
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
oblako-0.1.0/.gitignore
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
build/
|
|
6
|
+
/dist/
|
|
7
|
+
.eggs/
|
|
8
|
+
|
|
9
|
+
# Virtual envs
|
|
10
|
+
.venv/
|
|
11
|
+
venv/
|
|
12
|
+
.python-version
|
|
13
|
+
|
|
14
|
+
# Tooling caches
|
|
15
|
+
.pytest_cache/
|
|
16
|
+
.ruff_cache/
|
|
17
|
+
.mypy_cache/
|
|
18
|
+
.coverage
|
|
19
|
+
htmlcov/
|
|
20
|
+
|
|
21
|
+
# Node (dashboard UI) — note: oblako/dashboard/frontend/dist is committed (served by the app)
|
|
22
|
+
node_modules/
|
|
23
|
+
|
|
24
|
+
# Editors / OS
|
|
25
|
+
.vscode/
|
|
26
|
+
.idea/
|
|
27
|
+
.DS_Store
|
|
28
|
+
# macOS custom-folder-icon metadata (stray Finder droppings)
|
|
29
|
+
Icon-package/
|
|
30
|
+
**/Icon-package/
|
|
31
|
+
|
|
32
|
+
# SAM build artifacts
|
|
33
|
+
.aws-sam/
|
|
34
|
+
|
|
35
|
+
# Temp
|
|
36
|
+
.tmp/
|
|
37
|
+
|
|
38
|
+
# Local env files
|
|
39
|
+
.env
|
|
40
|
+
.env.*
|
|
41
|
+
|
|
42
|
+
# `oblako notebook` launch artifacts (generated per launch in the working dir)
|
|
43
|
+
.oblako-aws-config
|
|
44
|
+
oblako-welcome.ipynb
|
|
45
|
+
|
|
46
|
+
# Jupyter
|
|
47
|
+
.ipynb_checkpoints/
|
|
48
|
+
|
|
49
|
+
# SageMaker SDK local-mode working dirs (ModelTrainer local_container_root)
|
|
50
|
+
.sm-local/
|
|
51
|
+
|
|
52
|
+
# Sphinx output (CI builds it for the docs site)
|
|
53
|
+
docs/_build/
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Run on demand: uvx pre-commit run --all-files
|
|
2
|
+
# Install the git hooks: uvx pre-commit install --hook-type pre-commit --hook-type commit-msg
|
|
3
|
+
repos:
|
|
4
|
+
- repo: https://github.com/commitizen-tools/commitizen
|
|
5
|
+
rev: v4.16.4
|
|
6
|
+
hooks:
|
|
7
|
+
- id: commitizen # validate the message format: Service(+|~|-): description
|
|
8
|
+
stages: [commit-msg]
|
|
9
|
+
|
|
10
|
+
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
11
|
+
rev: v0.15.14
|
|
12
|
+
hooks:
|
|
13
|
+
- id: ruff-check # lint (autofix); covers oblako/, tests/, examples/, notebooks
|
|
14
|
+
args: [--fix]
|
|
15
|
+
- id: ruff-format # formatter; keeps boxed section-divider comments intact
|
|
16
|
+
|
|
17
|
+
- repo: https://github.com/PyCQA/pydocstyle
|
|
18
|
+
rev: 6.3.0
|
|
19
|
+
hooks:
|
|
20
|
+
- id: pydocstyle
|
|
21
|
+
# scope to the package; tests/examples/dashboard aren't docstring-checked
|
|
22
|
+
files: ^oblako/.*\.py$
|
|
23
|
+
exclude: ^oblako/dashboard/
|
|
24
|
+
|
|
25
|
+
- repo: https://github.com/aws-cloudformation/cfn-lint
|
|
26
|
+
rev: v1.47.1
|
|
27
|
+
hooks:
|
|
28
|
+
- id: cfn-lint
|
|
29
|
+
# the tracked CloudFormation/SAM templates (build artifacts are gitignored)
|
|
30
|
+
files: ^examples/python/(sam/template|ecs/infra/ecs-fargate-alb)\.yaml$
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
Commits follow `Service(<+|~|->): description`, where `+` = **Added**, `~` =
|
|
9
|
+
**Changed**, `-` = **Removed**. Running `cz bump` turns those commits into the
|
|
10
|
+
versioned entries below.
|
|
11
|
+
|
|
12
|
+
## Unreleased
|
|
13
|
+
|
|
14
|
+
## v0.1.0 (2026-10-05)
|
|
15
|
+
|
|
16
|
+
The first release.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **Glue**: triggers and workflows: on-demand, scheduled and conditional triggers, workflow runs that follow the graph
|
|
21
|
+
- **Glue**: crawlers and classifiers: S3 targets crawled into tables with partitions and inferred schemas
|
|
22
|
+
- **Examples**: the Feast notebook runs on a provisioned cluster or a Redshift Serverless workgroup
|
|
23
|
+
- **EventBridge**: Redshift scheduled queries as on AWS: cron schedules, Serverless targets, batches, WithEvent; the proxy is oblako's EventBridge endpoint
|
|
24
|
+
- **Redshift**: GetClusterCredentials issues real temporary credentials; every awswrangler.redshift function is tested
|
|
25
|
+
- **Glue**: connections: Create, Get, GetConnections, Update, Delete and BatchDelete, so awswrangler's connect(connection=...) works
|
|
26
|
+
- **Redshift**: UNLOAD's PARTITION BY, MANIFEST, JSON and compression, with files named as Redshift names them
|
|
27
|
+
- **Redshift**: awsdatacatalog.<db>.<table> names, Redshift's auto-mounted Data Catalog; MERGE from a distributed source on Citus
|
|
28
|
+
- **Redshift**: ALTER TABLE on Iceberg tables: columns, type widening, compression, partition evolution
|
|
29
|
+
- **Redshift**: MERGE into Iceberg tables; Iceberg DDL answers with a command, not a status row
|
|
30
|
+
- **Redshift**: Apache Iceberg tables: CREATE EXTERNAL SCHEMA, CREATE TABLE ... USING ICEBERG, INSERT, UPDATE, DELETE
|
|
31
|
+
- **Release**: publish oblako to PyPI from a version tag, through trusted publishing
|
|
32
|
+
- **CLI**: the oblako banner, in block letters and oblako blue
|
|
33
|
+
- **CLI**: oblako configure writes an AWS profile whose endpoints are oblako's
|
|
34
|
+
- **MWAA**: environments run AWS's own Airflow image, with DAGs from S3 and InvokeRestApi
|
|
35
|
+
- **Redshift**: Serverless namespaces and workgroups, in the API, the Data API and CloudFormation (#52)
|
|
36
|
+
- **Redshift**: multi-node clusters from CreateCluster, on a native multi-arch Citus image (#48)
|
|
37
|
+
- **RDS**: a real PostgreSQL per DB instance, with read replicas and logical replication (#45)
|
|
38
|
+
- **Glue**: the Glue job API, so jobs are created and run with plain boto3 (#42)
|
|
39
|
+
- **Athena**: query S3 Tables through s3tablescatalog/<bucket>, as on AWS (#41)
|
|
40
|
+
- **S3Tables**: serve the S3 Tables Iceberg REST endpoint, so PyIceberg configured for AWS runs unchanged (#40)
|
|
41
|
+
- **Glue**: Parquet and Iceberg tables in the Data Catalog, queried by Athena (#38)
|
|
42
|
+
- **S3**: bucket policies and event notifications to Lambda, SQS, SNS and EventBridge (#37)
|
|
43
|
+
- **S3**: object and bucket tagging, and S3 Inventory, in front of S3Proxy (#36)
|
|
44
|
+
- **CLI**: oblako up starts the in-process API engines, like s3vectors (#35)
|
|
45
|
+
- **redshift**: accept PASSWORD DISABLE on CREATE / ALTER USER (#30)
|
|
46
|
+
- **redshift**: access management as code via redtape compat (#28)
|
|
47
|
+
- **docs**: refresh SDK landing + architecture with diagrams (#24)
|
|
48
|
+
- **redshift**: COPY from JSON (auto / noshred / jsonpaths) + compression (#23)
|
|
49
|
+
- **services**: local S3 Vectors + S3 Tables (#22)
|
|
50
|
+
- **services**: surface ECS + EKS control plane + ECS task metadata endpoint (#20)
|
|
51
|
+
- **SageMaker**: multi-model endpoints (MME)
|
|
52
|
+
- **EventBridge**: fire Redshift Data targets + scheduled rules
|
|
53
|
+
- **Bedrock**: Guardrails (create/get/list/delete + ApplyGuardrail)
|
|
54
|
+
- **Athena**: boto3 athena API executed via the Trino engine
|
|
55
|
+
- **Firehose**: local Kinesis Data Firehose (DirectPut -> S3)
|
|
56
|
+
- **services**: surface moto-backed services (Secrets Manager, SSM, STS, KMS, Logs, ECR, EventBridge)
|
|
57
|
+
- **SageMaker**: Model Registry (package groups + versioned packages + approval)
|
|
58
|
+
- **Bedrock**: streaming (InvokeModelWithResponseStream + ConverseStream)
|
|
59
|
+
- **SageMaker**: async-inference SNS notifications (NotificationConfig)
|
|
60
|
+
- **SageMaker**: Asynchronous Inference (InvokeEndpointAsync)
|
|
61
|
+
- **DynamoDB**: native vector search (SearchVectors) over DynamoDB Local
|
|
62
|
+
- **Bedrock**: real embeddings via invoke_model on embed models
|
|
63
|
+
- **Bedrock**: expand foundation-model catalog to what oblako can serve
|
|
64
|
+
- **SageMaker**: Model Monitor - Data Capture + monitoring schedules
|
|
65
|
+
- **SageMaker**: local Feature Store (online KV + offline S3 Parquet)
|
|
66
|
+
- **SageMaker**: serverless-MLflow training example + endpoint variant echo
|
|
67
|
+
- **SageMaker**: resource CRUD, tags, Stop*, and Studio domains/profiles
|
|
68
|
+
- **SageMaker**: Automatic Model Tuning (HPO) via Syne Tune -> AMT (increment)
|
|
69
|
+
- **SageMaker**: local pipeline example + training Environment passthrough
|
|
70
|
+
- **SageMaker**: local processing jobs (create_processing_job)
|
|
71
|
+
- **SageMaker**: account-free client stubs + local_gpu passthrough
|
|
72
|
+
- **SageMaker**: local batch transform (create_transform_job)
|
|
73
|
+
- **SageMaker**: local real-time endpoints + invoke_endpoint
|
|
74
|
+
- **SageMaker**: local control-plane engine: create_training_job runs in Docker
|
|
75
|
+
- **Redshift**: COPY/UNLOAD<->S3 bridge, SUPER/PartiQL, LISTAGG, PIVOT/UNPIVOT (#18)
|
|
76
|
+
- **Redshift**: auto-distribute DISTKEY/DISTSTYLE on the cluster (increment #2) (#16)
|
|
77
|
+
- **Redshift**: MPP cluster variant on Citus (increment #1) (#15)
|
|
78
|
+
- **Redshift**: sqlalchemy-redshift catalog reflection (unblocks Alembic) (#14)
|
|
79
|
+
- **Commitizen**: Service(+/~/-) commit format and Keep a Changelog (#12)
|
|
80
|
+
|
|
81
|
+
### Changed
|
|
82
|
+
|
|
83
|
+
- **SageMaker**: local pipelines read S3Prefix inputs under when_required; model package versions are never reused
|
|
84
|
+
- **Redshift**: each oblako certificate has its own subject, and oblako trust drops stale ones
|
|
85
|
+
- **Redshift**: oblako trust survives a redshift-connector reinstall; oblako trust --remove undoes it
|
|
86
|
+
- **Examples**: the Feast notebook uses oblako's DynamoDB endpoint, so apply re-runs; outputs refreshed
|
|
87
|
+
- **Redshift**: enable_case_sensitive_identifier exists, off as on Redshift, so awswrangler's to_sql(add_new_columns=True) works
|
|
88
|
+
- **Redshift**: selected SUPER items are named after the path's last key, cast or not, as Redshift Serverless names them
|
|
89
|
+
- **Glue**: a job's S3 output is the data files only, as AWS Glue 5.0 writes it, with no directory markers
|
|
90
|
+
- **Redshift**: a SUPER value selected through dot navigation comes back as JSON text, named after its last step, as Redshift sends it
|
|
91
|
+
- **CLI**: oblako configure won't drop profile settings a newer oblako wrote, unless --force
|
|
92
|
+
- **CloudFormation**: AWS::IAM::Role gets its inline Policies and ManagedPolicyArns, and deletes cleanly
|
|
93
|
+
- **S3**: CreateBucket on a bucket you already own succeeds in us-east-1, as on S3; other Regions keep BucketAlreadyOwnedByYou
|
|
94
|
+
- **Redshift**: UNLOAD into a non-empty prefix fails unless ALLOWOVERWRITE or CLEANPATH, and files are named after the prefix as written
|
|
95
|
+
- **Redshift**: VARCHAR(MAX) is VARCHAR(65535), as Redshift stores and reflects it, so a longer value is refused
|
|
96
|
+
- **Redshift**: oblako trust trusts this machine's certificate and any running compose container's, not one or the other
|
|
97
|
+
- **Redshift**: Iceberg tables behave as Redshift Serverless does: NOT NULL, plain VARCHAR, Redshift's errors and SHOW TABLE
|
|
98
|
+
- **Examples**: every demo notebook re-run against oblako; the fraud stream notebook generates its transactions, so it runs from a fresh clone
|
|
99
|
+
- **Services**: KinesisService is exported from oblako.services, like the other services
|
|
100
|
+
- **Examples**: the OpenSearch RAG example runs on current opensearch-py, Bedrock batch asks factual prompts; cfn-lint checks the SAM and ECS templates again
|
|
101
|
+
- **Examples**: the credit-risk demo notebooks are now credit-scoring
|
|
102
|
+
- **Docs**: the architecture diagrams carry the new oblako logo
|
|
103
|
+
- **Redshift**: Redshift's storage clauses are stripped only from the CREATE TABLE statements of a multi-statement query
|
|
104
|
+
- **Redshift**: docker compose's Redshift containers make their own TLS certificate, and oblako trust reads it while they run
|
|
105
|
+
- **Repo**: untrack the docs build, drop local paths from notebook outputs, add SECURITY.md and CONTRIBUTING.md
|
|
106
|
+
- **Redshift**: each machine makes its own TLS certificate; the image no longer ships a shared key
|
|
107
|
+
- **Docs**: version the README logo URLs, so GitHub's image cache fetches the new logo
|
|
108
|
+
- **Services**: containers publish their ports on this machine only
|
|
109
|
+
- **CLI**: the dashboard listens on this machine only; OBLAKO_PORT_<NAME> moves a service off a taken port
|
|
110
|
+
- **Redshift**: system objects live in pg_catalog, so SQLAlchemy and Alembic see only user tables
|
|
111
|
+
- **Docs**: new oblako logo in the README and the docs sidebar
|
|
112
|
+
- **ECS**: inject task secrets from SSM and Secrets Manager; refuse invalid Fargate sizes
|
|
113
|
+
- **ECS**: plain boto3 RunTask runs the task as a real container
|
|
114
|
+
- **CLI**: the oblako profile sends services it does not list to moto, never to AWS
|
|
115
|
+
- **CloudFormation**: default port 8017, off OpenSearch Dashboards' 5601
|
|
116
|
+
- **CloudFormation**: sam deploy and sam delete run end to end; stacks persist and roll back
|
|
117
|
+
- **Redshift**: Serverless tags, and namespaces without an admin user
|
|
118
|
+
- **S3**: virtual-hosted addressing, UTF-8 keys, and S3 Control's tag API for buckets
|
|
119
|
+
- **MWAA**: report MWAA's defaults and drop null fields, as a real environment does
|
|
120
|
+
- **Firehose**: a CreateDeliveryStream that fails on its Kinesis source leaves nothing behind (#51)
|
|
121
|
+
- **CLI**: oblako up fails when a service is not ready, with --timeout (#49)
|
|
122
|
+
- **Firehose**: name S3 objects, convert to Parquet and COPY into Redshift as AWS does (#47)
|
|
123
|
+
- **DynamoDB**: vector search follows the released API, and tables take tags (#46)
|
|
124
|
+
- **RDS**: pgvector in the engine, and Data API decimals as strings, as on AWS (#44)
|
|
125
|
+
- **Redshift**: AVG of an integer column returns BIGINT, as on Redshift (#43)
|
|
126
|
+
- **Kinesis**: start from the CLI and with oblako up, and fix the crash on start (#39)
|
|
127
|
+
- **S3**: pin S3Proxy 4.1.1 so paginating partitioned keys terminates (#34)
|
|
128
|
+
- **SageMaker**: align with SDK v3, plus in-engine Redshift ML and engine port checks (#32)
|
|
129
|
+
- **docs**: redtape's plan is not reliably empty when converged (#31)
|
|
130
|
+
- **redshift**: render group ACLs the way Redshift does, so redtape converges (#29)
|
|
131
|
+
- **docs**: uniform diagram width so fonts render consistently (#27)
|
|
132
|
+
- **docs**: enlarge diagram text for readability on the docs site (#26)
|
|
133
|
+
- **redshift**: fix flaky Citus SORTKEY-index test (#25)
|
|
134
|
+
- **docs**: reflect new services + license(+): Apache-2.0 open-core (#21)
|
|
135
|
+
- **Firehose**: Kinesis-stream source + Redshift destination
|
|
136
|
+
- **DynamoDB**: match AWS SearchVectors score semantics (COSINE distance)
|
|
137
|
+
- **DynamoDB**: match real AWS SearchVectors wire shape (parity fix)
|
|
138
|
+
- **SageMaker**: run training containers directly (v3-ready, no SDK dependency)
|
|
139
|
+
- **Redshift**: SORTKEY -> index + transactional-DDL auto-distribute (polish) (#17)
|
|
140
|
+
- **Redshift**: bake a fixed TLS cert so pins survive down -v / clones (#13)
|