dbos 0.18.0__tar.gz → 0.19.0__tar.gz

{dbos-0.18.0 → dbos-0.19.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: dbos
-Version: 0.18.0
+Version: 0.19.0
 Summary: Ultra-lightweight durable execution in Python
 Author-Email: "DBOS, Inc." <contact@dbos.dev>
 License: MIT
@@ -28,14 +28,14 @@ Description-Content-Type: text/markdown
 
 <div align="center">
 
-# DBOS Transact: Ultra-Lightweight Durable Execution
+# DBOS Transact: A Lightweight Durable Execution Library Built on Postgres
 
 #### [Documentation](https://docs.dbos.dev/) &nbsp;&nbsp;•&nbsp;&nbsp;  [Examples](https://docs.dbos.dev/examples) &nbsp;&nbsp;•&nbsp;&nbsp; [Github](https://github.com/dbos-inc) &nbsp;&nbsp;•&nbsp;&nbsp; [Discord](https://discord.com/invite/jsmC6pXGgX)
 </div>
 
 ---
 
-DBOS Transact is a Python library providing **ultra-lightweight durable execution**.
+DBOS Transact is a Python library for **ultra-lightweight durable execution**.
 For example:
 
 ```python
@@ -55,18 +55,23 @@ def workflow()
 
 Durable execution means your program is **resilient to any failure**.
 If it is ever interrupted or crashes, all your workflows will automatically resume from the last completed step.
-If you want to see durable execution in action, check out [this demo app](https://demo-widget-store.cloud.dbos.dev/) (source code [here](https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/widget-store)).
-No matter how many times you try to crash it, it always resumes from exactly where it left off!
+Durable execution helps solve many common problems:
 
-Under the hood, DBOS Transact works by storing your program's execution state (which workflows are currently executing and which steps they've completed) in a Postgres database.
-So all you need to use it is a Postgres database to connect to&mdash;there's no need for a "workflow server."
-This approach is also incredibly fast, for example [25x faster than AWS Step Functions](https://www.dbos.dev/blog/dbos-vs-aws-step-functions-benchmark).
+- Orchestrating long-running or business-critical workflows so they seamlessly recover from any failure.
+- Running reliable background jobs with no timeouts.
+- Processing incoming events (e.g. from Kafka) exactly once.
+- Running a fault-tolerant distributed task queue.
+- Running a reliable cron scheduler.
+- Operating an AI agent, or anything that connects to an unreliable or non-deterministic API.
 
-Some more cool features include:
+What’s unique about DBOS's implementation of durable execution is that it’s implemented in a **lightweight library** that’s **totally backed by Postgres**.
+To use DBOS, just `pip install` it and annotate your program with DBOS decorators.
+Under the hood, those decorators store your program's execution state (which workflows are currently executing and which steps they've completed) in a Postgres database.
+If your program crashes or is interrupted, they automatically recover its workflows from their stored state.
+So all you need to use DBOS is Postgres&mdash;there are no other dependencies you have to manage, no separate workflow server.
 
-- Scheduled jobs&mdash;run your workflows exactly-once per time interval.
-- Exactly-once event processing&mdash;use workflows to process incoming events (for example, from a Kafka topic) exactly-once.
-- Observability&mdash;all workflows automatically emit [OpenTelemetry](https://opentelemetry.io/) traces.
+One big advantage of this approach is that you can add DBOS to **any** Python application&mdash;**it’s just a library**.
+You can use DBOS to add reliable background jobs or cron scheduling or queues to your app with no external dependencies except Postgres.
 
 ## Getting Started
 
@@ -77,7 +82,7 @@ pip install dbos
 dbos init --config
 ```
 
-Then, try it out with this simple program (requires Postgres):
+Then, try it out with this simple program:
 
 ```python
 from fastapi import FastAPI
@@ -107,14 +112,14 @@ def fastapi_endpoint():
     dbos_workflow()
 ```
 
-Save the program into `main.py`, edit `dbos-config.yaml` to configure your Postgres connection settings, and start it with `fastapi run`.
+Save the program into `main.py` and start it with `fastapi run`.
 Visit `localhost:8000` in your browser to start the workflow.
 When prompted, press `Control + \` to force quit your application.
 It should crash midway through the workflow, having completed step one but not step two.
 Then, restart your app with `fastapi run`.
 It should resume the workflow from where it left off, completing step two without re-executing step one.
 
-To learn how to build more complex workflows, see our [programming guide](https://docs.dbos.dev/python/programming-guide) or [examples](https://docs.dbos.dev/examples).
+To learn how to build more complex workflows, see the [programming guide](https://docs.dbos.dev/python/programming-guide) or [examples](https://docs.dbos.dev/examples).
 
 ## Documentation
 
@@ -125,7 +130,7 @@ To learn how to build more complex workflows, see our [programming guide](https:
 
 - [**AI-Powered Slackbot**](https://docs.dbos.dev/python/examples/rag-slackbot) &mdash; A Slackbot that answers questions about previous Slack conversations, using DBOS to durably orchestrate its RAG pipeline.
 - [**Widget Store**](https://docs.dbos.dev/python/examples/widget-store) &mdash; An online storefront that uses DBOS durable workflows to be resilient to any failure.
-- [**Earthquake Tracker**](https://docs.dbos.dev/python/examples/earthquake-tracker) &mdash; A real-time earthquake dashboard that uses DBOS to stream data from the USGS into Postgres, then visualizes it with Streamlit.
+- [**Scheduled Reminders**](https://docs.dbos.dev/python/examples/scheduled-reminders) &mdash; In just three lines of code, schedule an email to send days, weeks, or months in the future.
 
 More examples [here](https://docs.dbos.dev/examples)!
 

{dbos-0.18.0 → dbos-0.19.0}/README.md

@@ -1,14 +1,14 @@
 
 <div align="center">
 
-# DBOS Transact: Ultra-Lightweight Durable Execution
+# DBOS Transact: A Lightweight Durable Execution Library Built on Postgres
 
 #### [Documentation](https://docs.dbos.dev/) &nbsp;&nbsp;•&nbsp;&nbsp;  [Examples](https://docs.dbos.dev/examples) &nbsp;&nbsp;•&nbsp;&nbsp; [Github](https://github.com/dbos-inc) &nbsp;&nbsp;•&nbsp;&nbsp; [Discord](https://discord.com/invite/jsmC6pXGgX)
 </div>
 
 ---
 
-DBOS Transact is a Python library providing **ultra-lightweight durable execution**.
+DBOS Transact is a Python library for **ultra-lightweight durable execution**.
 For example:
 
 ```python
@@ -28,18 +28,23 @@ def workflow()
 
 Durable execution means your program is **resilient to any failure**.
 If it is ever interrupted or crashes, all your workflows will automatically resume from the last completed step.
-If you want to see durable execution in action, check out [this demo app](https://demo-widget-store.cloud.dbos.dev/) (source code [here](https://github.com/dbos-inc/dbos-demo-apps/tree/main/python/widget-store)).
-No matter how many times you try to crash it, it always resumes from exactly where it left off!
+Durable execution helps solve many common problems:
 
-Under the hood, DBOS Transact works by storing your program's execution state (which workflows are currently executing and which steps they've completed) in a Postgres database.
-So all you need to use it is a Postgres database to connect to&mdash;there's no need for a "workflow server."
-This approach is also incredibly fast, for example [25x faster than AWS Step Functions](https://www.dbos.dev/blog/dbos-vs-aws-step-functions-benchmark).
+- Orchestrating long-running or business-critical workflows so they seamlessly recover from any failure.
+- Running reliable background jobs with no timeouts.
+- Processing incoming events (e.g. from Kafka) exactly once.
+- Running a fault-tolerant distributed task queue.
+- Running a reliable cron scheduler.
+- Operating an AI agent, or anything that connects to an unreliable or non-deterministic API.
 
-Some more cool features include:
+What’s unique about DBOS's implementation of durable execution is that it’s implemented in a **lightweight library** that’s **totally backed by Postgres**.
+To use DBOS, just `pip install` it and annotate your program with DBOS decorators.
+Under the hood, those decorators store your program's execution state (which workflows are currently executing and which steps they've completed) in a Postgres database.
+If your program crashes or is interrupted, they automatically recover its workflows from their stored state.
+So all you need to use DBOS is Postgres&mdash;there are no other dependencies you have to manage, no separate workflow server.
 
-- Scheduled jobs&mdash;run your workflows exactly-once per time interval.
-- Exactly-once event processing&mdash;use workflows to process incoming events (for example, from a Kafka topic) exactly-once.
-- Observability&mdash;all workflows automatically emit [OpenTelemetry](https://opentelemetry.io/) traces.
+One big advantage of this approach is that you can add DBOS to **any** Python application&mdash;**it’s just a library**.
+You can use DBOS to add reliable background jobs or cron scheduling or queues to your app with no external dependencies except Postgres.
 
 ## Getting Started
 
@@ -50,7 +55,7 @@ pip install dbos
 dbos init --config
 ```
 
-Then, try it out with this simple program (requires Postgres):
+Then, try it out with this simple program:
 
 ```python
 from fastapi import FastAPI
@@ -80,14 +85,14 @@ def fastapi_endpoint():
     dbos_workflow()
 ```
 
-Save the program into `main.py`, edit `dbos-config.yaml` to configure your Postgres connection settings, and start it with `fastapi run`.
+Save the program into `main.py` and start it with `fastapi run`.
 Visit `localhost:8000` in your browser to start the workflow.
 When prompted, press `Control + \` to force quit your application.
 It should crash midway through the workflow, having completed step one but not step two.
 Then, restart your app with `fastapi run`.
 It should resume the workflow from where it left off, completing step two without re-executing step one.
 
-To learn how to build more complex workflows, see our [programming guide](https://docs.dbos.dev/python/programming-guide) or [examples](https://docs.dbos.dev/examples).
+To learn how to build more complex workflows, see the [programming guide](https://docs.dbos.dev/python/programming-guide) or [examples](https://docs.dbos.dev/examples).
 
 ## Documentation
 
@@ -98,7 +103,7 @@ To learn how to build more complex workflows, see our [programming guide](https:
 
 - [**AI-Powered Slackbot**](https://docs.dbos.dev/python/examples/rag-slackbot) &mdash; A Slackbot that answers questions about previous Slack conversations, using DBOS to durably orchestrate its RAG pipeline.
 - [**Widget Store**](https://docs.dbos.dev/python/examples/widget-store) &mdash; An online storefront that uses DBOS durable workflows to be resilient to any failure.
-- [**Earthquake Tracker**](https://docs.dbos.dev/python/examples/earthquake-tracker) &mdash; A real-time earthquake dashboard that uses DBOS to stream data from the USGS into Postgres, then visualizes it with Streamlit.
+- [**Scheduled Reminders**](https://docs.dbos.dev/python/examples/scheduled-reminders) &mdash; In just three lines of code, schedule an email to send days, weeks, or months in the future.
 
 More examples [here](https://docs.dbos.dev/examples)!
 

{dbos-0.18.0 → dbos-0.19.0}/dbos/_context.py

@@ -57,6 +57,7 @@ class DBOSContext:
         self.request: Optional["Request"] = None
 
         self.id_assigned_for_next_workflow: str = ""
+        self.is_within_set_workflow_id_block: bool = False
 
         self.parent_workflow_id: str = ""
         self.parent_workflow_fid: int = -1
@@ -78,6 +79,7 @@ class DBOSContext:
         rv.logger = self.logger
         rv.id_assigned_for_next_workflow = self.id_assigned_for_next_workflow
         self.id_assigned_for_next_workflow = ""
+        rv.is_within_set_workflow_id_block = self.is_within_set_workflow_id_block
         rv.parent_workflow_id = self.workflow_id
         rv.parent_workflow_fid = self.function_id
         rv.in_recovery = self.in_recovery
@@ -95,6 +97,10 @@ class DBOSContext:
         if len(self.id_assigned_for_next_workflow) > 0:
             wfid = self.id_assigned_for_next_workflow
         else:
+            if self.is_within_set_workflow_id_block:
+                self.logger.warning(
+                    f"Multiple workflows started in the same SetWorkflowID block. Only the first workflow is assigned the specified workflow ID; subsequent workflows will use a generated workflow ID."
+                )
             wfid = str(uuid.uuid4())
         return wfid
 
@@ -286,7 +292,7 @@ class DBOSContextSwap:
 
 class SetWorkflowID:
     """
-    Set the workflow ID to be used for the enclosed workflow invocation.
+    Set the workflow ID to be used for the enclosed workflow invocation. Note: Only the first workflow will be started with the specified workflow ID within a `with SetWorkflowID` block.
 
     Typical Usage
         ```
@@ -311,7 +317,9 @@ class SetWorkflowID:
         if ctx is None:
             self.created_ctx = True
             _set_local_dbos_context(DBOSContext())
-        assert_current_dbos_context().id_assigned_for_next_workflow = self.wfid
+        ctx = assert_current_dbos_context()
+        ctx.id_assigned_for_next_workflow = self.wfid
+        ctx.is_within_set_workflow_id_block = True
         return self
 
     def __exit__(
@@ -321,6 +329,7 @@ class SetWorkflowID:
         traceback: Optional[TracebackType],
     ) -> Literal[False]:
         # Code to clean up the basic context if we created it
+        assert_current_dbos_context().is_within_set_workflow_id_block = False
         if self.created_ctx:
             _clear_local_dbos_context()
         return False  # Did not handle

{dbos-0.18.0 → dbos-0.19.0}/dbos/_core.py

@@ -84,7 +84,7 @@ if TYPE_CHECKING:
         IsolationLevel,
     )
 
-from sqlalchemy.exc import DBAPIError
+from sqlalchemy.exc import DBAPIError, InvalidRequestError
 
 P = ParamSpec("P")  # A generic type for workflow parameters
 R = TypeVar("R", covariant=True)  # A generic type for workflow return values
@@ -180,21 +180,24 @@ def _init_workflow(
     if class_name is not None:
         inputs = {"args": inputs["args"][1:], "kwargs": inputs["kwargs"]}
 
+    wf_status = status["status"]
     if temp_wf_type != "transaction" or queue is not None:
         # Synchronously record the status and inputs for workflows and single-step workflows
         # We also have to do this for single-step workflows because of the foreign key constraint on the operation outputs table
         # TODO: Make this transactional (and with the queue step below)
-        dbos._sys_db.update_workflow_status(
+        wf_status = dbos._sys_db.update_workflow_status(
             status, False, ctx.in_recovery, max_recovery_attempts=max_recovery_attempts
         )
+        # TODO: Modify the inputs if they were changed by `update_workflow_inputs`
         dbos._sys_db.update_workflow_inputs(wfid, _serialization.serialize_args(inputs))
     else:
         # Buffer the inputs for single-transaction workflows, but don't buffer the status
         dbos._sys_db.buffer_workflow_inputs(wfid, _serialization.serialize_args(inputs))
 
-    if queue is not None:
+    if queue is not None and wf_status == WorkflowStatusString.ENQUEUED.value:
         dbos._sys_db.enqueue(wfid, queue)
 
+    status["status"] = wf_status
     return status
 
 
@@ -413,7 +416,16 @@ def start_workflow(
         max_recovery_attempts=fi.max_recovery_attempts,
     )
 
-    if not execute_workflow:
+    wf_status = status["status"]
+
+    if (
+        not execute_workflow
+        or wf_status == WorkflowStatusString.ERROR.value
+        or wf_status == WorkflowStatusString.SUCCESS.value
+    ):
+        dbos.logger.debug(
+            f"Workflow {new_wf_id} already completed with status {wf_status}. Directly returning a workflow handle."
+        )
         return WorkflowHandlePolling(new_wf_id, dbos)
 
     if fself is not None:
@@ -486,7 +498,7 @@ def workflow_wrapper(
                 temp_wf_type=get_temp_workflow_type(func),
                 max_recovery_attempts=max_recovery_attempts,
             )
-
+            # TODO: maybe modify the parameters if they've been changed by `_init_workflow`
             dbos.logger.debug(
                 f"Running workflow, id: {ctx.workflow_id}, name: {get_dbos_func_name(func)}"
             )
@@ -545,6 +557,7 @@ def decorate_transaction(
                     max_retry_wait_seconds = 2.0
                     while True:
                         has_recorded_error = False
+                        txn_error: Optional[Exception] = None
                         try:
                             with session.begin():
                                 # This must be the first statement in the transaction!
@@ -608,15 +621,24 @@ def decorate_transaction(
                                     max_retry_wait_seconds,
                                 )
                                 continue
+                            txn_error = dbapi_error
+                            raise
+                        except InvalidRequestError as invalid_request_error:
+                            dbos.logger.error(
+                                f"InvalidRequestError in transaction {func.__qualname__} \033[1m Hint: Do not call commit() or rollback() within a DBOS transaction.\033[0m"
+                            )
+                            txn_error = invalid_request_error
                             raise
                         except Exception as error:
+                            txn_error = error
+                            raise
+                        finally:
                             # Don't record the error if it was already recorded
-                            if not has_recorded_error:
+                            if txn_error and not has_recorded_error:
                                 txn_output["error"] = (
-                                    _serialization.serialize_exception(error)
+                                    _serialization.serialize_exception(txn_error)
                                 )
                                 dbos._app_db.record_transaction_error(txn_output)
-                            raise
             return output
 
         if inspect.iscoroutinefunction(func):

{dbos-0.18.0 → dbos-0.19.0}/dbos/_db_wizard.py

@@ -1,5 +1,7 @@
+import json
+import os
 import time
-from typing import TYPE_CHECKING, Optional
+from typing import TYPE_CHECKING, Optional, TypedDict
 
 import docker  # type: ignore
 import typer
@@ -15,8 +17,18 @@ from ._cloudutils.databases import choose_database, get_user_db_credentials
 from ._error import DBOSInitializationError
 from ._logger import dbos_logger
 
+DB_CONNECTION_PATH = os.path.join(".dbos", "db_connection")
 
-def db_connect(config: "ConfigFile", config_file_path: str) -> "ConfigFile":
+
+class DatabaseConnection(TypedDict):
+    hostname: Optional[str]
+    port: Optional[int]
+    username: Optional[str]
+    password: Optional[str]
+    local_suffix: Optional[bool]
+
+
+def db_wizard(config: "ConfigFile", config_file_path: str) -> "ConfigFile":
     # 1. Check the connectivity to the database. Return if successful. If cannot connect, continue to the following steps.
     db_connection_error = _check_db_connectivity(config)
     if db_connection_error is None:
@@ -82,17 +94,20 @@ def db_connect(config: "ConfigFile", config_file_path: str) -> "ConfigFile":
                 f"Could not connect to the database. Exception: {db_connection_error}"
             )
 
-    # 6. Save the config to the config file and return the updated config.
-    # TODO: make the config file prettier
-    with open(config_file_path, "w") as file:
-        file.write(yaml.dump(config))
-
+    # 6. Save the config to the database connection file
+    updated_connection = DatabaseConnection(
+        hostname=config["database"]["hostname"],
+        port=config["database"]["port"],
+        username=config["database"]["username"],
+        password=config["database"]["password"],
+        local_suffix=config["database"]["local_suffix"],
+    )
+    save_db_connection(updated_connection)
     return config
 
 
 def _start_docker_postgres(config: "ConfigFile") -> bool:
     print("Starting a Postgres Docker container...")
-    config["database"]["password"] = "dbos"
     client = docker.from_env()
     pg_data = "/var/lib/postgresql/data"
     container_name = "dbos-db"
@@ -122,7 +137,7 @@ def _start_docker_postgres(config: "ConfigFile") -> bool:
                 continue
             print("[green]Postgres Docker container started successfully![/green]")
             break
-        except Exception as e:
+        except:
             attempts -= 1
             time.sleep(1)
 
@@ -151,7 +166,7 @@ def _check_db_connectivity(config: "ConfigFile") -> Optional[Exception]:
         host=config["database"]["hostname"],
         port=config["database"]["port"],
         database="postgres",
-        query={"connect_timeout": "2"},
+        query={"connect_timeout": "1"},
     )
     postgres_db_engine = create_engine(postgres_db_url)
     try:
@@ -168,3 +183,26 @@ def _check_db_connectivity(config: "ConfigFile") -> Optional[Exception]:
         postgres_db_engine.dispose()
 
     return None
+
+
+def load_db_connection() -> DatabaseConnection:
+    try:
+        with open(DB_CONNECTION_PATH, "r") as f:
+            data = json.load(f)
+            return DatabaseConnection(
+                hostname=data.get("hostname", None),
+                port=data.get("port", None),
+                username=data.get("username", None),
+                password=data.get("password", None),
+                local_suffix=data.get("local_suffix", None),
+            )
+    except:
+        return DatabaseConnection(
+            hostname=None, port=None, username=None, password=None, local_suffix=None
+        )
+
+
+def save_db_connection(connection: DatabaseConnection) -> None:
+    os.makedirs(".dbos", exist_ok=True)
+    with open(DB_CONNECTION_PATH, "w") as f:
+        json.dump(connection, f)

{dbos-0.18.0 → dbos-0.19.0}/dbos/_dbos.py

@@ -83,7 +83,7 @@ from ._context import (
 )
 from ._dbos_config import ConfigFile, load_config, set_env_vars
 from ._error import DBOSException, DBOSNonExistentWorkflowError
-from ._logger import add_otlp_to_all_loggers, dbos_logger, init_logger
+from ._logger import add_otlp_to_all_loggers, dbos_logger
 from ._sys_db import SystemDatabase
 
 # Most DBOS functions are just any callable F, so decorators / wrappers work on F

{dbos-0.18.0 → dbos-0.19.0}/dbos/_dbos_config.py

@@ -6,12 +6,15 @@ from typing import Any, Dict, List, Optional, TypedDict, cast
 
 import yaml
 from jsonschema import ValidationError, validate
+from rich import print
 from sqlalchemy import URL
 
-from ._db_wizard import db_connect
+from ._db_wizard import db_wizard, load_db_connection
 from ._error import DBOSInitializationError
 from ._logger import config_logger, dbos_logger, init_logger
 
+DBOS_CONFIG_PATH = "dbos-config.yaml"
+
 
 class RuntimeConfig(TypedDict, total=False):
     start: List[str]
@@ -23,7 +26,7 @@ class DatabaseConfig(TypedDict, total=False):
     hostname: str
     port: int
     username: str
-    password: Optional[str]
+    password: str
     connectionTimeoutMillis: Optional[int]
     app_db_name: str
     sys_db_name: Optional[str]
@@ -93,7 +96,7 @@ def _substitute_env_vars(content: str) -> str:
     return re.sub(regex, replace_func, content)
 
 
-def get_dbos_database_url(config_file_path: str = "dbos-config.yaml") -> str:
+def get_dbos_database_url(config_file_path: str = DBOS_CONFIG_PATH) -> str:
     """
     Retrieve application database URL from configuration `.yaml` file.
 
@@ -119,7 +122,9 @@ def get_dbos_database_url(config_file_path: str = "dbos-config.yaml") -> str:
     return db_url.render_as_string(hide_password=False)
 
 
-def load_config(config_file_path: str = "dbos-config.yaml") -> ConfigFile:
+def load_config(
+    config_file_path: str = DBOS_CONFIG_PATH, *, use_db_wizard: bool = True
+) -> ConfigFile:
     """
     Load the DBOS `ConfigFile` from the specified path (typically `dbos-config.yaml`).
 
@@ -151,6 +156,9 @@ def load_config(config_file_path: str = "dbos-config.yaml") -> ConfigFile:
     except ValidationError as e:
         raise DBOSInitializationError(f"Validation error: {e}")
 
+    if "database" not in data:
+        data["database"] = {}
+
     if "name" not in data:
         raise DBOSInitializationError(
             f"dbos-config.yaml must specify an application name"
@@ -169,8 +177,6 @@ def load_config(config_file_path: str = "dbos-config.yaml") -> ConfigFile:
     if "runtimeConfig" not in data or "start" not in data["runtimeConfig"]:
         raise DBOSInitializationError(f"dbos-config.yaml must specify a start command")
 
-    data = cast(ConfigFile, data)
-
     if not _is_valid_app_name(data["name"]):
         raise DBOSInitializationError(
             f'Invalid app name {data["name"]}.  App names must be between 3 and 30 characters long and contain only lowercase letters, numbers, dashes, and underscores.'
@@ -179,10 +185,49 @@ def load_config(config_file_path: str = "dbos-config.yaml") -> ConfigFile:
     if "app_db_name" not in data["database"]:
         data["database"]["app_db_name"] = _app_name_to_db_name(data["name"])
 
+    # Load the DB connection file. Use its values for missing fields from dbos-config.yaml. Use defaults otherwise.
+    data = cast(ConfigFile, data)
+    db_connection = load_db_connection()
+    if data["database"].get("hostname"):
+        print(
+            "[bold blue]Loading database connection parameters from dbos-config.yaml[/bold blue]"
+        )
+    elif db_connection.get("hostname"):
+        print(
+            "[bold blue]Loading database connection parameters from .dbos/db_connection[/bold blue]"
+        )
+    else:
+        print(
+            "[bold blue]Using default database connection parameters (localhost)[/bold blue]"
+        )
+
+    data["database"]["hostname"] = (
+        data["database"].get("hostname") or db_connection.get("hostname") or "localhost"
+    )
+    data["database"]["port"] = (
+        data["database"].get("port") or db_connection.get("port") or 5432
+    )
+    data["database"]["username"] = (
+        data["database"].get("username") or db_connection.get("username") or "postgres"
+    )
+    data["database"]["password"] = (
+        data["database"].get("password")
+        or db_connection.get("password")
+        or os.environ.get("PGPASSWORD")
+        or "dbos"
+    )
+    data["database"]["local_suffix"] = (
+        data["database"].get("local_suffix")
+        or db_connection.get("local_suffix")
+        or False
+    )
+
+    # Configure the DBOS logger
     config_logger(data)
 
     # Check the connectivity to the database and make sure it's properly configured
-    data = db_connect(data, config_file_path)
+    if use_db_wizard:
+        data = db_wizard(data, config_file_path)
 
     if "local_suffix" in data["database"] and data["database"]["local_suffix"]:
         data["database"]["app_db_name"] = f"{data['database']['app_db_name']}_local"

{dbos-0.18.0 → dbos-0.19.0}/dbos/_error.py

@@ -35,6 +35,7 @@ class DBOSErrorCode(Enum):
     DeadLetterQueueError = 6
     MaxStepRetriesExceeded = 7
     NotAuthorized = 8
+    ConflictingWorkflowError = 9
 
 
 class DBOSWorkflowConflictIDError(DBOSException):
@@ -47,6 +48,16 @@ class DBOSWorkflowConflictIDError(DBOSException):
         )
 
 
+class DBOSConflictingWorkflowError(DBOSException):
+    """Exception raised different workflows started with the same workflow ID."""
+
+    def __init__(self, workflow_id: str, message: Optional[str] = None):
+        super().__init__(
+            f"Conflicting workflow invocation with the same ID ({workflow_id}): {message}",
+            dbos_error_code=DBOSErrorCode.ConflictingWorkflowError.value,
+        )
+
+
 class DBOSRecoveryError(DBOSException):
     """Exception raised when a workflow recovery fails."""
 

{dbos-0.18.0 → dbos-0.19.0}/dbos/_kafka.py

@@ -1,3 +1,4 @@
+import re
 import threading
 from typing import TYPE_CHECKING, Any, Callable, NoReturn
 
@@ -19,6 +20,14 @@ _kafka_queue: Queue
 _in_order_kafka_queues: dict[str, Queue] = {}
 
 
+def safe_group_name(method_name: str, topics: list[str]) -> str:
+    safe_group_id = "-".join(
+        re.sub(r"[^a-zA-Z0-9\-]", "", str(r)) for r in [method_name, *topics]
+    )
+
+    return f"dbos-kafka-group-{safe_group_id}"[:255]
+
+
 def _kafka_consumer_loop(
     func: _KafkaConsumerWorkflow,
     config: dict[str, Any],
@@ -34,6 +43,12 @@ def _kafka_consumer_loop(
     if "auto.offset.reset" not in config:
         config["auto.offset.reset"] = "earliest"
 
+    if config.get("group.id") is None:
+        config["group.id"] = safe_group_name(func.__qualname__, topics)
+        dbos_logger.warning(
+            f"Consumer group ID not found. Using generated group.id {config['group.id']}"
+        )
+
     consumer = Consumer(config)
     try:
         consumer.subscribe(topics)
@@ -71,8 +86,9 @@ def _kafka_consumer_loop(
                     topic=cmsg.topic(),
                     value=cmsg.value(),
                 )
+                groupID = config.get("group.id")
                 with SetWorkflowID(
-                    f"kafka-unique-id-{msg.topic}-{msg.partition}-{msg.offset}"
+                    f"kafka-unique-id-{msg.topic}-{msg.partition}-{groupID}-{msg.offset}"
                 ):
                     if in_order:
                         assert msg.topic is not None

dbos-0.19.0/dbos/_migrations/versions/04ca4f231047_workflow_queues_executor_id.py

@@ -0,0 +1,34 @@
+"""workflow_queues_executor_id
+
+Revision ID: 04ca4f231047
+Revises: d76646551a6c
+Create Date: 2025-01-15 15:05:08.043190
+
+"""
+
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision: str = "04ca4f231047"
+down_revision: Union[str, None] = "d76646551a6c"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "workflow_queue",
+        sa.Column(
+            "executor_id",
+            sa.Text(),
+            nullable=True,
+        ),
+        schema="dbos",
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("workflow_queue", "executor_id", schema="dbos")