dlthub-init 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- dlthub_init-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md +28 -0
- dlthub_init-0.1.0/.github/ISSUE_TEMPLATE/config.yml +1 -0
- dlthub_init-0.1.0/.github/ISSUE_TEMPLATE/feature_request.md +18 -0
- dlthub_init-0.1.0/.github/PULL_REQUEST_TEMPLATE.md +20 -0
- dlthub_init-0.1.0/.github/workflows/ci.yml +97 -0
- dlthub_init-0.1.0/.gitignore +12 -0
- dlthub_init-0.1.0/CHANGELOG.md +16 -0
- dlthub_init-0.1.0/CONTRIBUTING.md +67 -0
- dlthub_init-0.1.0/LICENSE +92 -0
- dlthub_init-0.1.0/Makefile +128 -0
- dlthub_init-0.1.0/PKG-INFO +171 -0
- dlthub_init-0.1.0/README.md +60 -0
- dlthub_init-0.1.0/pyproject.toml +82 -0
- dlthub_init-0.1.0/scripts/generate_skills.py +98 -0
- dlthub_init-0.1.0/scripts/update_skills.py +65 -0
- dlthub_init-0.1.0/skills/.gitkeep +0 -0
- dlthub_init-0.1.0/skills/dlthub-router/SKILL.md +49 -0
- dlthub_init-0.1.0/skills/improve-skills/SKILL.md +87 -0
- dlthub_init-0.1.0/skills/setup-secrets/SKILL.md +120 -0
- dlthub_init-0.1.0/skills/setup-secrets/cli-reference.md +69 -0
- dlthub_init-0.1.0/src/dlthub_init/__init__.py +1 -0
- dlthub_init-0.1.0/src/dlthub_init/__main__.py +10 -0
- dlthub_init-0.1.0/src/dlthub_init/cli.py +170 -0
- dlthub_init-0.1.0/src/dlthub_init/collisions.py +83 -0
- dlthub_init-0.1.0/src/dlthub_init/config.py +5 -0
- dlthub_init-0.1.0/src/dlthub_init/display.py +108 -0
- dlthub_init-0.1.0/src/dlthub_init/errors.py +27 -0
- dlthub_init-0.1.0/src/dlthub_init/prompts.py +33 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffold.py +96 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/.dlt/.workspace +0 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/.dlt/config.toml +5 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/.dlt/secrets.toml +1 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/.gitignore +203 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/pyproject.toml +19 -0
- dlthub_init-0.1.0/src/dlthub_init/scaffolds/minimal_workspace/uv.lock +3045 -0
- dlthub_init-0.1.0/src/dlthub_init/skills.py +75 -0
- dlthub_init-0.1.0/src/dlthub_init/strings.py +86 -0
- dlthub_init-0.1.0/src/dlthub_init/uv.py +120 -0
- dlthub_init-0.1.0/tests/__init__.py +0 -0
- dlthub_init-0.1.0/tests/test_cli.py +66 -0
- dlthub_init-0.1.0/tests/test_collisions.py +112 -0
- dlthub_init-0.1.0/tests/test_display.py +67 -0
- dlthub_init-0.1.0/tests/test_prompts.py +30 -0
- dlthub_init-0.1.0/tests/test_scaffold.py +97 -0
- dlthub_init-0.1.0/tests/test_scripts.py +101 -0
- dlthub_init-0.1.0/tests/test_skills.py +109 -0
- dlthub_init-0.1.0/tests/test_uv.py +51 -0
- dlthub_init-0.1.0/tests_integration/__init__.py +0 -0
- dlthub_init-0.1.0/tests_integration/test_e2e_workspace.py +26 -0
- dlthub_init-0.1.0/uv.lock +478 -0
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Bug report
|
|
3
|
+
about: Report something that isn't working as expected
|
|
4
|
+
title: 'fix: '
|
|
5
|
+
labels: bug
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Description
|
|
9
|
+
|
|
10
|
+
<!-- What went wrong? A clear, concise summary. -->
|
|
11
|
+
|
|
12
|
+
## Steps to reproduce
|
|
13
|
+
|
|
14
|
+
<!-- The exact command(s) and inputs. Include a snippet/output if relevant. -->
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
uvx dlthub-init <project-dir>
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Expected behavior
|
|
21
|
+
|
|
22
|
+
<!-- What you expected to happen instead. -->
|
|
23
|
+
|
|
24
|
+
## Environment
|
|
25
|
+
|
|
26
|
+
- `dlthub-init` version:
|
|
27
|
+
- OS:
|
|
28
|
+
- Python version:
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
blank_issues_enabled: false
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Feature request
|
|
3
|
+
about: Suggest an improvement or new capability
|
|
4
|
+
title: 'feat: '
|
|
5
|
+
labels: enhancement
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
<!-- What's the motivation? The gap or friction you're hitting. -->
|
|
11
|
+
|
|
12
|
+
## Proposed behavior
|
|
13
|
+
|
|
14
|
+
<!-- What should happen instead. Describe the desired UX/flow. -->
|
|
15
|
+
|
|
16
|
+
## Alternatives considered
|
|
17
|
+
|
|
18
|
+
<!-- Other approaches you thought about, if any. Delete if not applicable. -->
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
Title: use Conventional Commits — type(scope): description
|
|
3
|
+
types: feat | fix | chore | bug | improvement
|
|
4
|
+
scope (optional, kebab-case): cli, scaffold, collisions, minimal-workspace, ...
|
|
5
|
+
e.g. fix(collisions): never overwrite an existing secrets.toml
|
|
6
|
+
-->
|
|
7
|
+
|
|
8
|
+
## Overview
|
|
9
|
+
|
|
10
|
+
<!-- 1–2 sentences: what this PR does and why. Link any issue/thread it resolves, e.g. "Resolves #6". -->
|
|
11
|
+
|
|
12
|
+
## Changes
|
|
13
|
+
|
|
14
|
+
<!-- Bullets of what changed. For larger PRs, group by area with a bold lead, e.g. **`cli.py`**: ... -->
|
|
15
|
+
-
|
|
16
|
+
|
|
17
|
+
## Tests
|
|
18
|
+
|
|
19
|
+
<!-- How this was verified: new/updated tests, or manual steps. Delete this section if not applicable. -->
|
|
20
|
+
-
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
pull_request:
|
|
5
|
+
push:
|
|
6
|
+
branches:
|
|
7
|
+
- main
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
# POSIX runners: Ubuntu + macOS. GNU make is available, so `make ci` runs
|
|
11
|
+
# the full chain (lint, unit tests, e2e, lock checks, build).
|
|
12
|
+
posix:
|
|
13
|
+
name: ${{ matrix.os }}
|
|
14
|
+
runs-on: ${{ matrix.os }}
|
|
15
|
+
strategy:
|
|
16
|
+
fail-fast: false
|
|
17
|
+
matrix:
|
|
18
|
+
os: [ubuntu-latest, macos-latest]
|
|
19
|
+
|
|
20
|
+
steps:
|
|
21
|
+
- name: Checkout
|
|
22
|
+
uses: actions/checkout@v6
|
|
23
|
+
|
|
24
|
+
- name: Install uv
|
|
25
|
+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
|
26
|
+
with:
|
|
27
|
+
python-version: "3.13"
|
|
28
|
+
enable-cache: true
|
|
29
|
+
|
|
30
|
+
- name: Install dependencies
|
|
31
|
+
run: uv sync --extra dev --frozen
|
|
32
|
+
|
|
33
|
+
- name: Lint + type-check
|
|
34
|
+
run: make lint-ci
|
|
35
|
+
|
|
36
|
+
- name: Unit tests
|
|
37
|
+
run: make test
|
|
38
|
+
|
|
39
|
+
- name: Integration tests
|
|
40
|
+
run: make test-integration
|
|
41
|
+
|
|
42
|
+
- name: Build wheel
|
|
43
|
+
run: make build
|
|
44
|
+
|
|
45
|
+
# Windows runner: GNU make isn't standard on windows-latest. Invoke each
|
|
46
|
+
# check directly via uv so the workflow stays portable.
|
|
47
|
+
windows:
|
|
48
|
+
name: windows-latest
|
|
49
|
+
runs-on: windows-latest
|
|
50
|
+
|
|
51
|
+
steps:
|
|
52
|
+
- name: Checkout
|
|
53
|
+
uses: actions/checkout@v6
|
|
54
|
+
|
|
55
|
+
- name: Install uv
|
|
56
|
+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
|
57
|
+
with:
|
|
58
|
+
python-version: "3.13"
|
|
59
|
+
enable-cache: true
|
|
60
|
+
|
|
61
|
+
- name: Install dependencies
|
|
62
|
+
run: uv sync --extra dev --frozen
|
|
63
|
+
|
|
64
|
+
- name: Lint + type-check
|
|
65
|
+
run: |
|
|
66
|
+
uv run ruff format --check src tests tests_integration
|
|
67
|
+
uv run ruff check src tests tests_integration
|
|
68
|
+
uv run mypy src tests tests_integration
|
|
69
|
+
|
|
70
|
+
- name: Unit tests
|
|
71
|
+
run: uv run python -m unittest discover -s tests -t .
|
|
72
|
+
|
|
73
|
+
- name: Integration tests
|
|
74
|
+
run: uv run python -m unittest discover -s tests_integration -t .
|
|
75
|
+
|
|
76
|
+
- name: Build wheel
|
|
77
|
+
run: uv build
|
|
78
|
+
|
|
79
|
+
lock-check:
|
|
80
|
+
name: uv.lock drift
|
|
81
|
+
runs-on: ubuntu-latest
|
|
82
|
+
|
|
83
|
+
steps:
|
|
84
|
+
- name: Checkout
|
|
85
|
+
uses: actions/checkout@v6
|
|
86
|
+
|
|
87
|
+
- name: Install uv
|
|
88
|
+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
|
89
|
+
with:
|
|
90
|
+
python-version: "3.13"
|
|
91
|
+
enable-cache: true
|
|
92
|
+
|
|
93
|
+
- name: Verify lockfile is in sync with pyproject.toml
|
|
94
|
+
run: make lock-check
|
|
95
|
+
|
|
96
|
+
- name: Verify bundled workspace lockfile is in sync with its pyproject.toml
|
|
97
|
+
run: make scaffold-lock-check
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- Initial release: `uvx dlthub-init [dir]` scaffolds a dltHub workspace into a new or existing directory and optionally creates the virtual environment.
|
|
12
|
+
- Non-destructive by default: safe to run in an existing directory. The run stops only if `.dlt/.workspace` already exists (the directory is already a workspace, exit code `2`); any other existing file is left untouched and reported as skipped. `uv.lock` is written only alongside a freshly created `pyproject.toml`. Flags: `--force`, `--merge`, `--no-pyproject`, `--no-gitignore`, `--no-sync`, `--yes`.
|
|
13
|
+
- Bundled minimal workspace with loosened dependency ranges and a committed `uv.lock` for reproducible installs.
|
|
14
|
+
- `uv sync` runs by default behind an interactive confirmation; a failed sync degrades to a warning so the scaffold still succeeds.
|
|
15
|
+
- `make generate-skills` / `update-skills` / `check-skills` populate the root `skills/` directory from the dltHub AI workbench's `init` toolkit, pinned to a `WORKBENCH_REF` commit (with a CI drift guard), mirroring `dlthub-start`'s AI-generation flow.
|
|
16
|
+
- Scaffolded workspaces receive the bundled skills: copied into `.agents/skills/` (the canonical location most agents read) and linked into `.claude/skills/` via relative symlinks (copied on Windows or when symlinks are unavailable). The skills ship in the wheel from the single-source `skills/` directory.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Contributing to dlthub-init
|
|
2
|
+
|
|
3
|
+
## Setup
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
make dev
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
## Checks
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
make fl # format + lint-fix
|
|
13
|
+
make lint # ruff + mypy (no writes)
|
|
14
|
+
make test # unit tests
|
|
15
|
+
make ci # everything CI runs
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## Layout
|
|
19
|
+
|
|
20
|
+
- `src/dlthub_init/` — the CLI. `cli.py` orchestrates the flow; `scaffold.py`
|
|
21
|
+
enumerates and writes the payload; `collisions.py` implements the
|
|
22
|
+
non-destructive write policy.
|
|
23
|
+
- `src/dlthub_init/scaffolds/minimal_workspace/` — the bundled workspace copied
|
|
24
|
+
into the user's directory.
|
|
25
|
+
- `tests/` — fast unit tests. `tests_integration/` — slow end-to-end tests that
|
|
26
|
+
invoke the real CLI and `uv sync`.
|
|
27
|
+
|
|
28
|
+
## The bundled workspace
|
|
29
|
+
|
|
30
|
+
The scaffold's `pyproject.toml` uses loose dependency ranges; the committed
|
|
31
|
+
`uv.lock` pins exact versions for reproducible installs. After changing the
|
|
32
|
+
scaffold's dependencies, refresh the lock and commit it:
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
make scaffold-lock-upgrade
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Skills
|
|
39
|
+
|
|
40
|
+
The root `skills/` directory is generated from the dltHub AI workbench's `init`
|
|
41
|
+
toolkit, pinned to a commit. Don't hand-edit `skills/` — change the skills
|
|
42
|
+
upstream, then re-pull.
|
|
43
|
+
|
|
44
|
+
Knobs in `scripts/generate_skills.py`:
|
|
45
|
+
- `WORKBENCH_BRANCH` — the branch `update-skills` tracks.
|
|
46
|
+
- `WORKBENCH_REF_SHORT` — short commit SHA (the one GitHub shows); override by hand.
|
|
47
|
+
- `WORKBENCH_REF` — full SHA, written automatically from the short one; never hand-edit.
|
|
48
|
+
- `SKILL_TOOLKITS` — which toolkits' skills to include.
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
make update-skills # track WORKBENCH_BRANCH's latest commit
|
|
52
|
+
make update-skills REF=<branch|sha> # pin a specific branch tip or commit
|
|
53
|
+
make generate-skills # rebuild at the pinned short SHA (auto-fills the full SHA)
|
|
54
|
+
make check-skills # CI drift guard
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
To pin a commit by hand, set `WORKBENCH_REF_SHORT` to a short SHA and run
|
|
58
|
+
`make generate-skills` — it resolves and writes the full `WORKBENCH_REF`.
|
|
59
|
+
|
|
60
|
+
One-off testing without touching the committed pins (env overrides):
|
|
61
|
+
`DLTHUB_WORKBENCH_REPO` (repo URL or local path), `DLTHUB_WORKBENCH_REF`
|
|
62
|
+
(branch/sha to build from), `DLTHUB_SKILL_TOOLKITS` (comma-separated toolkits).
|
|
63
|
+
|
|
64
|
+
## Code style
|
|
65
|
+
|
|
66
|
+
Write self-explanatory code. Do not add comments that narrate what the code
|
|
67
|
+
does; reserve comments for the non-obvious *why*.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
dltHub License
|
|
2
|
+
|
|
3
|
+
Copyright 2026 ScaleVector, GmbH. All rights reserved.
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
## Definitions
|
|
7
|
+
|
|
8
|
+
"Agreement" means the agreement between ScaleVector, GmbH and its affiliates
|
|
9
|
+
(collectively, "dltHub") and you governing the use of dltHub Services, including
|
|
10
|
+
dltHub Platform, as those terms are defined in the applicable dltHub Terms of Use
|
|
11
|
+
available at: https://dlthub.com/terms.
|
|
12
|
+
|
|
13
|
+
"dltHub Services" means dltHub Platform and any other products or services offered
|
|
14
|
+
by dltHub, as defined in the Agreement.
|
|
15
|
+
|
|
16
|
+
"Derivative Works" means any work that is based on, incorporates, or is derived
|
|
17
|
+
from the Licensed Materials, including but not limited to code generated with the
|
|
18
|
+
assistance of the Licensed Materials, modified copies, adaptations, and projects
|
|
19
|
+
into which the Licensed Materials is installed or integrated.
|
|
20
|
+
|
|
21
|
+
"Licensed Materials" means the source code, object code, data, and/or other works,
|
|
22
|
+
including educational materials, to which this license applies.
|
|
23
|
+
|
|
24
|
+
## Scope of Use
|
|
25
|
+
|
|
26
|
+
You may view, use, copy, modify, publish, and distribute the Licensed Materials
|
|
27
|
+
solely in connection with dltHub Services under a governing Agreement. Any use of
|
|
28
|
+
the Licensed Materials outside of dltHub Services is not permitted under this
|
|
29
|
+
license.
|
|
30
|
+
|
|
31
|
+
Without limiting the foregoing, the following are examples of permitted use:
|
|
32
|
+
|
|
33
|
+
1. Using toolkits and skills provided by dltHub to develop, build, test,
|
|
34
|
+
and deploy data pipelines on dltHub Platform.
|
|
35
|
+
2. Copying and modifying toolkit code for use exclusively with dltHub Services and on
|
|
36
|
+
dltHub Services.
|
|
37
|
+
3. Using the Licensed Materials to generate code intended for deployment on dltHub
|
|
38
|
+
Platform.
|
|
39
|
+
|
|
40
|
+
The following are examples of uses that are NOT permitted under this license:
|
|
41
|
+
|
|
42
|
+
1. Using toolkits and skills provided by dltHub to generate code or pipelines for
|
|
43
|
+
deployment on a third-party runtime platform or orchestration service that is not
|
|
44
|
+
part of dltHub Services.
|
|
45
|
+
2. Modifying toolkits to replace, bypass, or substitute components that are part of
|
|
46
|
+
dltHub services in order to operate within a third-party development environment.
|
|
47
|
+
3. Incorporating the Licensed Materials or Derivative Works into any product or
|
|
48
|
+
service that competes with the Licensed Materials, or any other dltHub Services.
|
|
49
|
+
4. Providing the Licensed Materials to third parties as part of a hosted or managed
|
|
50
|
+
service that replicates the functionality of dltHub Services.
|
|
51
|
+
|
|
52
|
+
Use beyond the scope of this license requires a separate license agreement
|
|
53
|
+
negotiated with dltHub on a case-by-case basis.
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
## Redistribution
|
|
57
|
+
|
|
58
|
+
Redistribution of the Licensed Materials, with or without modification, is
|
|
59
|
+
permitted provided that this license notice is included in all copies or substantial
|
|
60
|
+
portions of the Licensed Materials. All recipients are bound by the same scope of
|
|
61
|
+
use restrictions set forth in this license.
|
|
62
|
+
|
|
63
|
+
## Intellectual Property
|
|
64
|
+
|
|
65
|
+
dltHub retains all right, title, and interest in and to the Licensed Materials as
|
|
66
|
+
originally published.
|
|
67
|
+
|
|
68
|
+
dltHub waives all intellectual property claims over Derivative Works, provided that:
|
|
69
|
+
|
|
70
|
+
1. The use of the Licensed Materials complies with the scope of use set forth in
|
|
71
|
+
this license, and
|
|
72
|
+
2. This license notice is retained in all copies or substantial portions of the
|
|
73
|
+
Licensed Materials contained within the Derivative Works.
|
|
74
|
+
|
|
75
|
+
This waiver extends to the Licensed Materials itself as incorporated into Derivative
|
|
76
|
+
Works — you are free to use, modify, and distribute it within your Derivative Works
|
|
77
|
+
without further restriction, subject only to the scope of use and redistribution
|
|
78
|
+
terms above.
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
## Disclaimer and Limitation of Liability
|
|
82
|
+
|
|
83
|
+
THE LICENSED MATERIALS ARE PROVIDED "AS IS" WITH ALL FAULTS. DLTHUB AND ITS
|
|
84
|
+
LICENSORS DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, STATUTORY, OR
|
|
85
|
+
OTHERWISE, INCLUDING WITHOUT LIMITATION THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
|
86
|
+
FOR A PARTICULAR PURPOSE, TITLE, AND NONINFRINGEMENT. DLTHUB AND ITS LICENSORS'
|
|
87
|
+
TOTAL AGGREGATE LIABILITY RELATING TO OR ARISING OUT OF YOUR USE OF OR DLTHUB'S
|
|
88
|
+
PROVISIONING OF THE LICENSED MATERIALS SHALL BE LIMITED TO ONE THOUSAND DOLLARS
|
|
89
|
+
($1,000). IN NO EVENT SHALL DLTHUB OR ITS LICENSORS BE LIABLE FOR ANY LOST PROFITS,
|
|
90
|
+
LOSS OF REVENUE, LOST BUSINESS, COSTS OF COVER OR ANY INCIDENTAL, SPECIAL,
|
|
91
|
+
EXEMPLARY OR PUNITIVE DAMAGES OF ANY KIND OR NATURE IN CONNECTION WITH THIS LICENSE
|
|
92
|
+
OR THE LICENSED MATERIALS.
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
.DEFAULT_GOAL := help
|
|
2
|
+
|
|
3
|
+
.PHONY: help dev lint lint-fix format format-check fl lint-ci test test-integration build clean-dist publish ci lock-upgrade lock-check scaffold-lock-upgrade scaffold-lock-check generate-skills update-skills check-skills workspace workspace-init
|
|
4
|
+
|
|
5
|
+
PYTHON_SOURCES := src tests tests_integration scripts
|
|
6
|
+
SCAFFOLD_DIR ?= src/dlthub_init/scaffolds/minimal_workspace
|
|
7
|
+
|
|
8
|
+
help: ## Show this help message
|
|
9
|
+
@echo "Available targets:"
|
|
10
|
+
@grep -E '^[a-zA-Z0-9_.-]+:.*?## .*$$' $(MAKEFILE_LIST) \
|
|
11
|
+
| awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-22s\033[0m %s\n", $$1, $$2}'
|
|
12
|
+
|
|
13
|
+
dev: ## Install dev dependencies
|
|
14
|
+
uv sync --extra dev
|
|
15
|
+
|
|
16
|
+
lint: ## Lint with ruff and type-check with mypy
|
|
17
|
+
uv run ruff check $(PYTHON_SOURCES)
|
|
18
|
+
uv run mypy $(PYTHON_SOURCES)
|
|
19
|
+
|
|
20
|
+
lint-fix: ## Lint and autofix with ruff, type-check with mypy
|
|
21
|
+
uv run ruff check --fix $(PYTHON_SOURCES)
|
|
22
|
+
uv run mypy $(PYTHON_SOURCES)
|
|
23
|
+
|
|
24
|
+
format: ## Format with ruff
|
|
25
|
+
uv run ruff format $(PYTHON_SOURCES)
|
|
26
|
+
|
|
27
|
+
format-check: ## Check formatting with ruff (no writes)
|
|
28
|
+
uv run ruff format --check $(PYTHON_SOURCES)
|
|
29
|
+
|
|
30
|
+
fl: format lint-fix ## Format and lint-fix in one shot
|
|
31
|
+
|
|
32
|
+
lint-ci: format-check lint ## CI lint workflow (format-check then lint)
|
|
33
|
+
|
|
34
|
+
test: ## Run unit tests (fast)
|
|
35
|
+
uv run python -m unittest discover -s tests -t .
|
|
36
|
+
|
|
37
|
+
test-integration: ## Run e2e integration tests (slow; invokes real CLI + uv sync)
|
|
38
|
+
uv run python -m unittest discover -s tests_integration -t .
|
|
39
|
+
|
|
40
|
+
#
|
|
41
|
+
# Manual test workspaces (under examples/, which is gitignored)
|
|
42
|
+
#
|
|
43
|
+
|
|
44
|
+
WORKSPACE_DIR ?= examples/my-workspace
|
|
45
|
+
|
|
46
|
+
workspace: dev ## Scaffold a fresh ./$(WORKSPACE_DIR) for eyeballing output (pre-deletes; pass ARGS="--no-sync")
|
|
47
|
+
@case "$(WORKSPACE_DIR)" in *..*|"") echo "invalid WORKSPACE_DIR: $(WORKSPACE_DIR)"; exit 1;; esac
|
|
48
|
+
rm -rf -- "$(WORKSPACE_DIR)"
|
|
49
|
+
uv run dlthub-init "$(WORKSPACE_DIR)" $(ARGS)
|
|
50
|
+
|
|
51
|
+
WORKSPACE_INIT_DIR ?= examples/init-workspace
|
|
52
|
+
|
|
53
|
+
workspace-init: dev ## Init in place: make empty ./$(WORKSPACE_INIT_DIR), cd in, run the CLI with no positional (pass ARGS="--no-sync")
|
|
54
|
+
@case "$(WORKSPACE_INIT_DIR)" in *..*|"") echo "invalid WORKSPACE_INIT_DIR: $(WORKSPACE_INIT_DIR)"; exit 1;; esac
|
|
55
|
+
rm -rf -- "$(WORKSPACE_INIT_DIR)"
|
|
56
|
+
mkdir -p -- "$(WORKSPACE_INIT_DIR)"
|
|
57
|
+
cd "$(WORKSPACE_INIT_DIR)" && "$(CURDIR)/.venv/bin/dlthub-init" $(ARGS)
|
|
58
|
+
|
|
59
|
+
build: dev ## Build the package wheel
|
|
60
|
+
uv build
|
|
61
|
+
|
|
62
|
+
clean-dist: ## Remove dist/ directory
|
|
63
|
+
-@rm -r dist/
|
|
64
|
+
|
|
65
|
+
publish: clean-dist build ## Build and publish dlthub-init to PyPI
|
|
66
|
+
ls -l dist/
|
|
67
|
+
@bash -c 'read -s -p "Enter PyPI API token: " PYPI_API_TOKEN; echo; \
|
|
68
|
+
uv publish --token "$$PYPI_API_TOKEN"'
|
|
69
|
+
|
|
70
|
+
lock-upgrade: ## Upgrade the root uv.lock to the latest deps pyproject.toml allows (PKG=<name> to bump one); review the diff and commit
|
|
71
|
+
uv lock $(if $(PKG),--upgrade-package $(PKG),--upgrade)
|
|
72
|
+
|
|
73
|
+
lock-check: ## CI guard: fail if the root uv.lock is out of sync with pyproject.toml
|
|
74
|
+
@echo "lock-check: checking uv.lock against pyproject.toml…"; \
|
|
75
|
+
log="$$(mktemp)"; \
|
|
76
|
+
if uv lock --check >"$$log" 2>&1; then \
|
|
77
|
+
rm -f "$$log"; \
|
|
78
|
+
echo "lock-check: OK — uv.lock is in sync with pyproject.toml."; \
|
|
79
|
+
else \
|
|
80
|
+
echo "lock-check: FAILED — run 'make lock-upgrade' and commit. uv output:"; \
|
|
81
|
+
cat "$$log"; rm -f "$$log"; \
|
|
82
|
+
exit 1; \
|
|
83
|
+
fi
|
|
84
|
+
|
|
85
|
+
scaffold-lock-upgrade: ## Re-resolve the bundled workspace uv.lock (PKG=<name> to bump one); review the diff and commit
|
|
86
|
+
uv lock $(if $(PKG),--upgrade-package $(PKG),--upgrade) --project $(SCAFFOLD_DIR)
|
|
87
|
+
|
|
88
|
+
scaffold-lock-check: ## CI guard: fail if the bundled workspace uv.lock is out of sync with its pyproject
|
|
89
|
+
@echo "scaffold-lock-check: checking $(SCAFFOLD_DIR)/uv.lock against its pyproject.toml…"; \
|
|
90
|
+
log="$$(mktemp)"; \
|
|
91
|
+
if uv lock --check --project $(SCAFFOLD_DIR) >"$$log" 2>&1; then \
|
|
92
|
+
rm -f "$$log"; \
|
|
93
|
+
echo "scaffold-lock-check: OK — uv.lock is in sync with pyproject.toml."; \
|
|
94
|
+
else \
|
|
95
|
+
echo "scaffold-lock-check: FAILED — run 'make scaffold-lock-upgrade' and commit. uv output:"; \
|
|
96
|
+
cat "$$log"; rm -f "$$log"; \
|
|
97
|
+
exit 1; \
|
|
98
|
+
fi
|
|
99
|
+
|
|
100
|
+
#
|
|
101
|
+
# Skills (pulled from the dltHub AI workbench)
|
|
102
|
+
#
|
|
103
|
+
|
|
104
|
+
generate-skills: ## Populate skills/ from the dltHub AI workbench at the pinned WORKBENCH_REF
|
|
105
|
+
uv run python scripts/generate_skills.py
|
|
106
|
+
|
|
107
|
+
update-skills: ## Bump the workbench ref (REF=<sha>, or latest) and regenerate skills/
|
|
108
|
+
uv run python scripts/update_skills.py $(REF)
|
|
109
|
+
|
|
110
|
+
check-skills: ## CI guard: fail if skills/ differs from the generated output
|
|
111
|
+
@echo "check-skills: regenerating skills/ (output hidden unless it fails)…"; \
|
|
112
|
+
log="$$(mktemp)"; \
|
|
113
|
+
if ! uv run python scripts/generate_skills.py >"$$log" 2>&1; then \
|
|
114
|
+
echo "check-skills: generate-skills failed — its output:"; \
|
|
115
|
+
cat "$$log"; rm -f "$$log"; exit 1; \
|
|
116
|
+
fi; \
|
|
117
|
+
rm -f "$$log"; \
|
|
118
|
+
changed="$$(git status --porcelain -- skills)"; \
|
|
119
|
+
if [ -z "$$changed" ]; then \
|
|
120
|
+
echo "check-skills: OK — skills/ is up to date."; \
|
|
121
|
+
else \
|
|
122
|
+
echo "check-skills: FAILED — skills/ differs from 'make generate-skills'; regenerate and commit:"; \
|
|
123
|
+
printf '%s\n' "$$changed" | sed 's/^/ /'; \
|
|
124
|
+
git --no-pager diff -- skills; \
|
|
125
|
+
exit 1; \
|
|
126
|
+
fi
|
|
127
|
+
|
|
128
|
+
ci: lint-ci test test-integration lock-check scaffold-lock-check check-skills build ## Run all CI checks locally
|
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dlthub-init
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Scaffold a dltHub workspace into a new or existing directory.
|
|
5
|
+
Project-URL: Homepage, https://github.com/dlt-hub/dlthub-init
|
|
6
|
+
Project-URL: Repository, https://github.com/dlt-hub/dlthub-init
|
|
7
|
+
Project-URL: Issues, https://github.com/dlt-hub/dlthub-init/issues
|
|
8
|
+
Author: dltHub
|
|
9
|
+
License: dltHub License
|
|
10
|
+
|
|
11
|
+
Copyright 2026 ScaleVector, GmbH. All rights reserved.
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
## Definitions
|
|
15
|
+
|
|
16
|
+
"Agreement" means the agreement between ScaleVector, GmbH and its affiliates
|
|
17
|
+
(collectively, "dltHub") and you governing the use of dltHub Services, including
|
|
18
|
+
dltHub Platform, as those terms are defined in the applicable dltHub Terms of Use
|
|
19
|
+
available at: https://dlthub.com/terms.
|
|
20
|
+
|
|
21
|
+
"dltHub Services" means dltHub Platform and any other products or services offered
|
|
22
|
+
by dltHub, as defined in the Agreement.
|
|
23
|
+
|
|
24
|
+
"Derivative Works" means any work that is based on, incorporates, or is derived
|
|
25
|
+
from the Licensed Materials, including but not limited to code generated with the
|
|
26
|
+
assistance of the Licensed Materials, modified copies, adaptations, and projects
|
|
27
|
+
into which the Licensed Materials is installed or integrated.
|
|
28
|
+
|
|
29
|
+
"Licensed Materials" means the source code, object code, data, and/or other works,
|
|
30
|
+
including educational materials, to which this license applies.
|
|
31
|
+
|
|
32
|
+
## Scope of Use
|
|
33
|
+
|
|
34
|
+
You may view, use, copy, modify, publish, and distribute the Licensed Materials
|
|
35
|
+
solely in connection with dltHub Services under a governing Agreement. Any use of
|
|
36
|
+
the Licensed Materials outside of dltHub Services is not permitted under this
|
|
37
|
+
license.
|
|
38
|
+
|
|
39
|
+
Without limiting the foregoing, the following are examples of permitted use:
|
|
40
|
+
|
|
41
|
+
1. Using toolkits and skills provided by dltHub to develop, build, test,
|
|
42
|
+
and deploy data pipelines on dltHub Platform.
|
|
43
|
+
2. Copying and modifying toolkit code for use exclusively with dltHub Services and on
|
|
44
|
+
dltHub Services.
|
|
45
|
+
3. Using the Licensed Materials to generate code intended for deployment on dltHub
|
|
46
|
+
Platform.
|
|
47
|
+
|
|
48
|
+
The following are examples of uses that are NOT permitted under this license:
|
|
49
|
+
|
|
50
|
+
1. Using toolkits and skills provided by dltHub to generate code or pipelines for
|
|
51
|
+
deployment on a third-party runtime platform or orchestration service that is not
|
|
52
|
+
part of dltHub Services.
|
|
53
|
+
2. Modifying toolkits to replace, bypass, or substitute components that are part of
|
|
54
|
+
dltHub services in order to operate within a third-party development environment.
|
|
55
|
+
3. Incorporating the Licensed Materials or Derivative Works into any product or
|
|
56
|
+
service that competes with the Licensed Materials, or any other dltHub Services.
|
|
57
|
+
4. Providing the Licensed Materials to third parties as part of a hosted or managed
|
|
58
|
+
service that replicates the functionality of dltHub Services.
|
|
59
|
+
|
|
60
|
+
Use beyond the scope of this license requires a separate license agreement
|
|
61
|
+
negotiated with dltHub on a case-by-case basis.
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
## Redistribution
|
|
65
|
+
|
|
66
|
+
Redistribution of the Licensed Materials, with or without modification, is
|
|
67
|
+
permitted provided that this license notice is included in all copies or substantial
|
|
68
|
+
portions of the Licensed Materials. All recipients are bound by the same scope of
|
|
69
|
+
use restrictions set forth in this license.
|
|
70
|
+
|
|
71
|
+
## Intellectual Property
|
|
72
|
+
|
|
73
|
+
dltHub retains all right, title, and interest in and to the Licensed Materials as
|
|
74
|
+
originally published.
|
|
75
|
+
|
|
76
|
+
dltHub waives all intellectual property claims over Derivative Works, provided that:
|
|
77
|
+
|
|
78
|
+
1. The use of the Licensed Materials complies with the scope of use set forth in
|
|
79
|
+
this license, and
|
|
80
|
+
2. This license notice is retained in all copies or substantial portions of the
|
|
81
|
+
Licensed Materials contained within the Derivative Works.
|
|
82
|
+
|
|
83
|
+
This waiver extends to the Licensed Materials itself as incorporated into Derivative
|
|
84
|
+
Works — you are free to use, modify, and distribute it within your Derivative Works
|
|
85
|
+
without further restriction, subject only to the scope of use and redistribution
|
|
86
|
+
terms above.
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
## Disclaimer and Limitation of Liability
|
|
90
|
+
|
|
91
|
+
THE LICENSED MATERIALS ARE PROVIDED "AS IS" WITH ALL FAULTS. DLTHUB AND ITS
|
|
92
|
+
LICENSORS DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, STATUTORY, OR
|
|
93
|
+
OTHERWISE, INCLUDING WITHOUT LIMITATION THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
|
94
|
+
FOR A PARTICULAR PURPOSE, TITLE, AND NONINFRINGEMENT. DLTHUB AND ITS LICENSORS'
|
|
95
|
+
TOTAL AGGREGATE LIABILITY RELATING TO OR ARISING OUT OF YOUR USE OF OR DLTHUB'S
|
|
96
|
+
PROVISIONING OF THE LICENSED MATERIALS SHALL BE LIMITED TO ONE THOUSAND DOLLARS
|
|
97
|
+
($1,000). IN NO EVENT SHALL DLTHUB OR ITS LICENSORS BE LIABLE FOR ANY LOST PROFITS,
|
|
98
|
+
LOSS OF REVENUE, LOST BUSINESS, COSTS OF COVER OR ANY INCIDENTAL, SPECIAL,
|
|
99
|
+
EXEMPLARY OR PUNITIVE DAMAGES OF ANY KIND OR NATURE IN CONNECTION WITH THIS LICENSE
|
|
100
|
+
OR THE LICENSED MATERIALS.
|
|
101
|
+
License-File: LICENSE
|
|
102
|
+
Classifier: Development Status :: 4 - Beta
|
|
103
|
+
Requires-Python: >=3.10
|
|
104
|
+
Requires-Dist: beaupy>=3.9
|
|
105
|
+
Requires-Dist: rich>=13.7
|
|
106
|
+
Provides-Extra: dev
|
|
107
|
+
Requires-Dist: mypy>=1.10; extra == 'dev'
|
|
108
|
+
Requires-Dist: pytest>=8; extra == 'dev'
|
|
109
|
+
Requires-Dist: ruff>=0.6; extra == 'dev'
|
|
110
|
+
Description-Content-Type: text/markdown
|
|
111
|
+
|
|
112
|
+
# dlthub-init (beta)
|
|
113
|
+
|
|
114
|
+
Scaffold a [dltHub](https://dlthub.com) workspace into a new or existing directory.
|
|
115
|
+
|
|
116
|
+
`dlthub-init` lays down the files of a minimal dltHub workspace — `pyproject.toml`,
|
|
117
|
+
a locked `uv.lock`, and the `.dlt/` configuration — and optionally creates the
|
|
118
|
+
virtual environment. That is all it does: it does not log in, run a pipeline, or
|
|
119
|
+
install agent toolkits.
|
|
120
|
+
|
|
121
|
+
Unlike `dlthub-start`, it is safe to run inside an existing repository. It is
|
|
122
|
+
non-destructive by default: it never overwrites your files, and it stops before
|
|
123
|
+
writing anything if a generated path would collide with something already there.
|
|
124
|
+
|
|
125
|
+
## Usage
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
uvx dlthub-init # initialize the current directory
|
|
129
|
+
uvx dlthub-init my-workspace # initialize ./my-workspace
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
After scaffolding, `dlthub-init` offers to create a virtual environment and
|
|
133
|
+
install dependencies with `uv`.
|
|
134
|
+
|
|
135
|
+
### Options
|
|
136
|
+
|
|
137
|
+
| Flag | Effect |
|
|
138
|
+
| --- | --- |
|
|
139
|
+
| `--no-sync` | Scaffold files only; do not create a venv or install dependencies. |
|
|
140
|
+
| `--force` | Overwrite existing generated files (never secrets). |
|
|
141
|
+
| `--merge` | Append missing entries to an existing `.gitignore` instead of skipping it. |
|
|
142
|
+
| `--no-pyproject` | Skip `pyproject.toml`. |
|
|
143
|
+
| `--no-gitignore` | Skip `.gitignore`. |
|
|
144
|
+
| `--verbose`, `-v` | Stream output from `uv`. |
|
|
145
|
+
|
|
146
|
+
## Collision behavior
|
|
147
|
+
|
|
148
|
+
`dlthub-init` is safe to run in an existing directory:
|
|
149
|
+
|
|
150
|
+
- Missing files are **created**.
|
|
151
|
+
- The run stops only if `.dlt/.workspace` already exists — the directory is
|
|
152
|
+
already a dltHub workspace (override with `--force`).
|
|
153
|
+
- Otherwise existing files are **left alone** and reported as skipped: your
|
|
154
|
+
`pyproject.toml`, `.dlt/config.toml`, and `.dlt/secrets.toml` are never
|
|
155
|
+
overwritten, and `.gitignore` is skipped (or **merged** with `--merge`).
|
|
156
|
+
- `uv.lock` is written only when `pyproject.toml` is created, so the lockfile
|
|
157
|
+
always matches the workspace's dependencies.
|
|
158
|
+
- `--force` overwrites the generated files (never secrets).
|
|
159
|
+
|
|
160
|
+
## Development
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
make dev # install dev dependencies
|
|
164
|
+
make lint # ruff + mypy
|
|
165
|
+
make test # unit tests
|
|
166
|
+
make ci # full local CI
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The bundled workspace lives in `src/dlthub_init/scaffolds/minimal_workspace`.
|
|
170
|
+
After editing its `pyproject.toml`, run `make scaffold-lock-upgrade` to refresh
|
|
171
|
+
the committed `uv.lock`.
|