dagster 1.11.6__tar.gz → 1.12.3__tar.gz

dagster-1.12.3/PKG-INFO

@@ -0,0 +1,275 @@
+Metadata-Version: 2.4
+Name: dagster
+Version: 1.12.3
+Summary: Dagster is an orchestration platform for the development, production, and observation of data assets.
+Author: Dagster Labs
+Author-email: hello@dagsterlabs.com
+License: Apache-2.0
+Project-URL: Homepage, https://dagster.io
+Project-URL: GitHub, https://github.com/dagster-io/dagster
+Project-URL: Documentation, https://docs.dagster.io
+Project-URL: Changelog, https://github.com/dagster-io/dagster/releases
+Project-URL: Issue Tracker, https://github.com/dagster-io/dagster/issues
+Project-URL: Twitter, https://twitter.com/dagster
+Project-URL: YouTube, https://www.youtube.com/@dagsterio
+Project-URL: Slack, https://dagster.io/slack
+Project-URL: Blog, https://dagster.io/blog
+Project-URL: Newsletter, https://dagster.io/newsletter-signup
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Environment :: Web Environment
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Topic :: System :: Monitoring
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10,<3.14
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click<9.0,>=5.0
+Requires-Dist: coloredlogs<=14.0,>=6.1
+Requires-Dist: Jinja2
+Requires-Dist: alembic!=1.11.0,!=1.6.3,!=1.7.0,>=1.2.1
+Requires-Dist: grpcio>=1.44.0
+Requires-Dist: grpcio-health-checking>=1.44.0
+Requires-Dist: protobuf<7,>=3.20.0; python_version < "3.11"
+Requires-Dist: protobuf<7,>=4; python_version >= "3.11"
+Requires-Dist: python-dotenv
+Requires-Dist: pytz
+Requires-Dist: requests
+Requires-Dist: setuptools
+Requires-Dist: six
+Requires-Dist: tabulate
+Requires-Dist: tomli<3
+Requires-Dist: tqdm<5
+Requires-Dist: tzdata
+Requires-Dist: structlog
+Requires-Dist: sqlalchemy<3,>=1.0
+Requires-Dist: toposort>=1.0
+Requires-Dist: watchdog<7,>=0.8.3
+Requires-Dist: psutil>=1.0; platform_system == "Windows"
+Requires-Dist: pywin32!=226; platform_system == "Windows"
+Requires-Dist: docstring-parser
+Requires-Dist: universal_pathlib; python_version < "3.12"
+Requires-Dist: universal_pathlib>=0.2.0; python_version >= "3.12"
+Requires-Dist: rich
+Requires-Dist: filelock
+Requires-Dist: dagster-pipes==1.12.3
+Requires-Dist: dagster-shared==1.12.3
+Requires-Dist: antlr4-python3-runtime
+Provides-Extra: docker
+Requires-Dist: docker; extra == "docker"
+Provides-Extra: test
+Requires-Dist: buildkite-test-collector; extra == "test"
+Requires-Dist: docker; extra == "test"
+Requires-Dist: grpcio-tools>=1.44.0; extra == "test"
+Requires-Dist: mypy-protobuf; extra == "test"
+Requires-Dist: objgraph; extra == "test"
+Requires-Dist: pytest-cov==5.0.0; extra == "test"
+Requires-Dist: pytest-mock==3.14.0; extra == "test"
+Requires-Dist: pytest-xdist==3.6.1; extra == "test"
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-asyncio; extra == "test"
+Requires-Dist: pytest-timeout; extra == "test"
+Requires-Dist: responses<=0.23.1; extra == "test"
+Requires-Dist: syrupy>=4.0.0; extra == "test"
+Requires-Dist: tox>=4; extra == "test"
+Requires-Dist: morefs[asynclocal]; extra == "test"
+Requires-Dist: fsspec<2024.5.0; extra == "test"
+Requires-Dist: rapidfuzz; extra == "test"
+Requires-Dist: flaky; extra == "test"
+Requires-Dist: psutil; extra == "test"
+Requires-Dist: ruff==0.11.5; extra == "test"
+Provides-Extra: test-components
+Requires-Dist: tomlkit; extra == "test-components"
+Requires-Dist: jsonschema; extra == "test-components"
+Requires-Dist: pandas; extra == "test-components"
+Requires-Dist: duckdb; extra == "test-components"
+Provides-Extra: mypy
+Requires-Dist: mypy==1.8.0; extra == "mypy"
+Provides-Extra: pyright
+Requires-Dist: pyright==1.1.379; extra == "pyright"
+Requires-Dist: pandas-stubs; extra == "pyright"
+Requires-Dist: types-backports; extra == "pyright"
+Requires-Dist: types-certifi; extra == "pyright"
+Requires-Dist: types-chardet; extra == "pyright"
+Requires-Dist: types-cryptography; extra == "pyright"
+Requires-Dist: types-mock; extra == "pyright"
+Requires-Dist: types-paramiko; extra == "pyright"
+Requires-Dist: types-pyOpenSSL; extra == "pyright"
+Requires-Dist: types-python-dateutil~=2.9.0.20240316; extra == "pyright"
+Requires-Dist: types-PyYAML; extra == "pyright"
+Requires-Dist: types-pytz; extra == "pyright"
+Requires-Dist: types-requests; extra == "pyright"
+Requires-Dist: types-simplejson; extra == "pyright"
+Requires-Dist: types-six; extra == "pyright"
+Requires-Dist: types-tabulate; extra == "pyright"
+Requires-Dist: types-tzlocal; extra == "pyright"
+Requires-Dist: types-toml; extra == "pyright"
+Provides-Extra: ruff
+Requires-Dist: ruff==0.11.5; extra == "ruff"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: license
+Dynamic: license-file
+Dynamic: project-url
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+<div align="center">
+  <!-- Note: Do not try adding the dark mode version here with the `picture` element, it will break formatting in PyPI -->
+  <a target="_blank" href="https://dagster.io" style="background:none">
+    <img alt="dagster logo" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/dagster-readme-header.svg" width="auto" height="100%">
+  </a>
+  <a target="_blank" href="https://github.com/dagster-io/dagster" style="background:none">
+    <img src="https://img.shields.io/github/stars/dagster-io/dagster?labelColor=4F43DD&color=163B36&logo=github">
+  </a>
+  <a target="_blank" href="https://github.com/dagster-io/dagster/blob/master/LICENSE" style="background:none">
+    <img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg?label=license&labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
+    <img src="https://img.shields.io/pypi/v/dagster?labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
+    <img src="https://img.shields.io/pypi/pyversions/dagster?labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://twitter.com/dagster" style="background:none">
+    <img src="https://img.shields.io/badge/twitter-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=twitter" />
+  </a>
+  <a target="_blank" href="https://dagster.io/slack" style="background:none">
+    <img src="https://img.shields.io/badge/slack-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=slack" />
+  </a>
+  <a target="_blank" href="https://linkedin.com/showcase/dagster" style="background:none">
+    <img src="https://img.shields.io/badge/linkedin-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=linkedin" />
+  </a>
+</div>
+
+**Dagster is a cloud-native data pipeline orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.**
+
+It is designed for **developing and maintaining data assets**, such as tables, data sets, machine learning models, and reports.
+
+With Dagster, you declare—as Python functions—the data assets that you want to build. Dagster then helps you run your functions at the right time and keep your assets up-to-date.
+
+Here is an example of a graph of three assets defined in Python:
+
+```python
+import dagster as dg
+import pandas as pd
+
+from sklearn.linear_model import LinearRegression
+
+@dg.asset
+def country_populations() -> pd.DataFrame:
+    df = pd.read_html("https://tinyurl.com/mry64ebh")[0]
+    df.columns = ["country", "pop2022", "pop2023", "change", "continent", "region"]
+    df["change"] = df["change"].str.rstrip("%").astype("float")
+    return df
+
+@dg.asset
+def continent_change_model(country_populations: pd.DataFrame) -> LinearRegression:
+    data = country_populations.dropna(subset=["change"])
+    return LinearRegression().fit(pd.get_dummies(data[["continent"]]), data["change"])
+
+@dg.asset
+def continent_stats(country_populations: pd.DataFrame, continent_change_model: LinearRegression) -> pd.DataFrame:
+    result = country_populations.groupby("continent").sum()
+    result["pop_change_factor"] = continent_change_model.coef_
+    return result
+```
+
+The graph loaded into Dagster's web UI:
+
+<p align="center">
+  <img width="100%" alt="An example asset graph as rendered in the Dagster UI" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/example-lineage.png">
+</p>
+
+Dagster is built to be used at every stage of the data development lifecycle - local development, unit tests, integration tests, staging environments, all the way up to production.
+
+## Quick Start:
+
+If you're new to Dagster, we recommend checking out the [docs](https://docs.dagster.io) or following the hands-on [tutorial](https://docs.dagster.io/etl-pipeline-tutorial/).
+
+Dagster is available on PyPI and officially supports Python 3.9 through Python 3.13.
+
+```bash
+pip install dagster dagster-webserver
+```
+
+This installs two packages:
+
+- `dagster`: The core programming model.
+- `dagster-webserver`: The server that hosts Dagster's web UI for developing and operating Dagster jobs and assets.
+
+## Documentation
+
+You can find the full Dagster documentation [here](https://docs.dagster.io), including the [Quickstart guide](https://docs.dagster.io/getting-started/quickstart).
+
+<hr/>
+
+## Key Features:
+
+  <p align="center">
+    <img width="100%" alt="image" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/key-features-cards.svg">
+  </p>
+
+### Dagster as a productivity platform
+
+Identify the key assets you need to create using a declarative approach, or you can focus on running basic tasks. Embrace CI/CD best practices from the get-go: build reusable components, spot data quality issues, and flag bugs early.
+
+### Dagster as a robust orchestration engine
+
+Put your pipelines into production with a robust multi-tenant, multi-tool engine that scales technically and organizationally.
+
+### Dagster as a unified control plane
+
+Maintain control over your data as the complexity scales. Centralize your metadata in one tool with built-in observability, diagnostics, cataloging, and lineage. Spot any issues and identify performance improvement opportunities.
+
+<hr />
+
+## Master the Modern Data Stack with integrations
+
+Dagster provides a growing library of integrations for today’s most popular data tools. Integrate with the tools you already use, and deploy to your infrastructure.
+
+<br/>
+<p align="center">
+    <a target="_blank" href="https://dagster.io/integrations" style="background:none">
+        <img width="100%" alt="image" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/integrations-bar-for-readme.png">
+    </a>
+</p>
+
+## Community
+
+Connect with thousands of other data practitioners building with Dagster. Share knowledge, get help,
+and contribute to the open-source project. To see featured material and upcoming events, check out
+our [Dagster Community](https://dagster.io/community) page.
+
+Join our community here:
+
+- 🌟 [Star us on GitHub](https://github.com/dagster-io/dagster)
+- 📥 [Subscribe to our Newsletter](https://dagster.io/newsletter-signup)
+- 🐦 [Follow us on Twitter](https://twitter.com/dagster)
+- 🕴️ [Follow us on LinkedIn](https://www.linkedin.com/company/dagsterlabs/)
+- 📺 [Subscribe to our YouTube channel](https://www.youtube.com/@dagsterio)
+- 📚 [Read our blog posts](https://dagster.io/blog)
+- 👋 [Join us on Slack](https://dagster.io/slack)
+- 🗃 [Browse Slack archives](https://discuss.dagster.io)
+- ✏️ [Start a GitHub Discussion](https://github.com/dagster-io/dagster/discussions)
+
+## Contributing
+
+For details on contributing or running the project for development, check out our [contributing
+guide](https://docs.dagster.io/about/contributing).
+
+## License
+
+Dagster is [Apache 2.0 licensed](https://github.com/dagster-io/dagster/blob/master/LICENSE).

dagster-1.12.3/README.md

@@ -0,0 +1,147 @@
+<div align="center">
+  <!-- Note: Do not try adding the dark mode version here with the `picture` element, it will break formatting in PyPI -->
+  <a target="_blank" href="https://dagster.io" style="background:none">
+    <img alt="dagster logo" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/dagster-readme-header.svg" width="auto" height="100%">
+  </a>
+  <a target="_blank" href="https://github.com/dagster-io/dagster" style="background:none">
+    <img src="https://img.shields.io/github/stars/dagster-io/dagster?labelColor=4F43DD&color=163B36&logo=github">
+  </a>
+  <a target="_blank" href="https://github.com/dagster-io/dagster/blob/master/LICENSE" style="background:none">
+    <img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg?label=license&labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
+    <img src="https://img.shields.io/pypi/v/dagster?labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
+    <img src="https://img.shields.io/pypi/pyversions/dagster?labelColor=4F43DD&color=163B36">
+  </a>
+  <a target="_blank" href="https://twitter.com/dagster" style="background:none">
+    <img src="https://img.shields.io/badge/twitter-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=twitter" />
+  </a>
+  <a target="_blank" href="https://dagster.io/slack" style="background:none">
+    <img src="https://img.shields.io/badge/slack-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=slack" />
+  </a>
+  <a target="_blank" href="https://linkedin.com/showcase/dagster" style="background:none">
+    <img src="https://img.shields.io/badge/linkedin-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=linkedin" />
+  </a>
+</div>
+
+**Dagster is a cloud-native data pipeline orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.**
+
+It is designed for **developing and maintaining data assets**, such as tables, data sets, machine learning models, and reports.
+
+With Dagster, you declare—as Python functions—the data assets that you want to build. Dagster then helps you run your functions at the right time and keep your assets up-to-date.
+
+Here is an example of a graph of three assets defined in Python:
+
+```python
+import dagster as dg
+import pandas as pd
+
+from sklearn.linear_model import LinearRegression
+
+@dg.asset
+def country_populations() -> pd.DataFrame:
+    df = pd.read_html("https://tinyurl.com/mry64ebh")[0]
+    df.columns = ["country", "pop2022", "pop2023", "change", "continent", "region"]
+    df["change"] = df["change"].str.rstrip("%").astype("float")
+    return df
+
+@dg.asset
+def continent_change_model(country_populations: pd.DataFrame) -> LinearRegression:
+    data = country_populations.dropna(subset=["change"])
+    return LinearRegression().fit(pd.get_dummies(data[["continent"]]), data["change"])
+
+@dg.asset
+def continent_stats(country_populations: pd.DataFrame, continent_change_model: LinearRegression) -> pd.DataFrame:
+    result = country_populations.groupby("continent").sum()
+    result["pop_change_factor"] = continent_change_model.coef_
+    return result
+```
+
+The graph loaded into Dagster's web UI:
+
+<p align="center">
+  <img width="100%" alt="An example asset graph as rendered in the Dagster UI" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/example-lineage.png">
+</p>
+
+Dagster is built to be used at every stage of the data development lifecycle - local development, unit tests, integration tests, staging environments, all the way up to production.
+
+## Quick Start:
+
+If you're new to Dagster, we recommend checking out the [docs](https://docs.dagster.io) or following the hands-on [tutorial](https://docs.dagster.io/etl-pipeline-tutorial/).
+
+Dagster is available on PyPI and officially supports Python 3.9 through Python 3.13.
+
+```bash
+pip install dagster dagster-webserver
+```
+
+This installs two packages:
+
+- `dagster`: The core programming model.
+- `dagster-webserver`: The server that hosts Dagster's web UI for developing and operating Dagster jobs and assets.
+
+## Documentation
+
+You can find the full Dagster documentation [here](https://docs.dagster.io), including the [Quickstart guide](https://docs.dagster.io/getting-started/quickstart).
+
+<hr/>
+
+## Key Features:
+
+  <p align="center">
+    <img width="100%" alt="image" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/key-features-cards.svg">
+  </p>
+
+### Dagster as a productivity platform
+
+Identify the key assets you need to create using a declarative approach, or you can focus on running basic tasks. Embrace CI/CD best practices from the get-go: build reusable components, spot data quality issues, and flag bugs early.
+
+### Dagster as a robust orchestration engine
+
+Put your pipelines into production with a robust multi-tenant, multi-tool engine that scales technically and organizationally.
+
+### Dagster as a unified control plane
+
+Maintain control over your data as the complexity scales. Centralize your metadata in one tool with built-in observability, diagnostics, cataloging, and lineage. Spot any issues and identify performance improvement opportunities.
+
+<hr />
+
+## Master the Modern Data Stack with integrations
+
+Dagster provides a growing library of integrations for today’s most popular data tools. Integrate with the tools you already use, and deploy to your infrastructure.
+
+<br/>
+<p align="center">
+    <a target="_blank" href="https://dagster.io/integrations" style="background:none">
+        <img width="100%" alt="image" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/integrations-bar-for-readme.png">
+    </a>
+</p>
+
+## Community
+
+Connect with thousands of other data practitioners building with Dagster. Share knowledge, get help,
+and contribute to the open-source project. To see featured material and upcoming events, check out
+our [Dagster Community](https://dagster.io/community) page.
+
+Join our community here:
+
+- 🌟 [Star us on GitHub](https://github.com/dagster-io/dagster)
+- 📥 [Subscribe to our Newsletter](https://dagster.io/newsletter-signup)
+- 🐦 [Follow us on Twitter](https://twitter.com/dagster)
+- 🕴️ [Follow us on LinkedIn](https://www.linkedin.com/company/dagsterlabs/)
+- 📺 [Subscribe to our YouTube channel](https://www.youtube.com/@dagsterio)
+- 📚 [Read our blog posts](https://dagster.io/blog)
+- 👋 [Join us on Slack](https://dagster.io/slack)
+- 🗃 [Browse Slack archives](https://discuss.dagster.io)
+- ✏️ [Start a GitHub Discussion](https://github.com/dagster-io/dagster/discussions)
+
+## Contributing
+
+For details on contributing or running the project for development, check out our [contributing
+guide](https://docs.dagster.io/about/contributing).
+
+## License
+
+Dagster is [Apache 2.0 licensed](https://github.com/dagster-io/dagster/blob/master/LICENSE).