flagsmith-common 3.10.1__tar.gz → 3.12.0__tar.gz

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flagsmith-common
-Version: 3.10.1
+Version: 3.12.0
 Summary: Flagsmith's common library
 Author: Matthew Elwell, Gagan Trivedi, Kim Gustyr, Zach Aysan, Francesco Lo Franco, Rodrigo López Dato, Evandro Myller, Wadii Zaim
 License-Expression: BSD-3-Clause
@@ -32,6 +32,7 @@ Requires-Dist: flagsmith-flag-engine>6 ; extra == 'flagsmith-schemas'
 Requires-Dist: inflection ; extra == 'otel'
 Requires-Dist: opentelemetry-api>=1.25,<2 ; extra == 'otel'
 Requires-Dist: opentelemetry-sdk>=1.25,<2 ; extra == 'otel'
+Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.25,<2 ; extra == 'otel'
 Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.25,<2 ; extra == 'otel'
 Requires-Dist: sentry-sdk>=2.0.0,<3.0.0 ; extra == 'otel'
 Requires-Dist: structlog>=24.4,<26 ; extra == 'otel'
@@ -39,6 +40,7 @@ Requires-Dist: typing-extensions ; extra == 'otel'
 Requires-Dist: backoff>=2.2.1,<3.0.0 ; extra == 'task-processor'
 Requires-Dist: django>4,<6 ; extra == 'task-processor'
 Requires-Dist: django-health-check ; extra == 'task-processor'
+Requires-Dist: environs<16 ; extra == 'task-processor'
 Requires-Dist: opentelemetry-api>=1.25,<2 ; extra == 'task-processor'
 Requires-Dist: prometheus-client>=0.0.16 ; extra == 'task-processor'
 Requires-Dist: pyfakefs>=6,<7 ; extra == 'test-tools'
@@ -95,6 +97,33 @@ This enables the `route` label for Prometheus HTTP metrics.
 
 5. To enable the `/metrics` endpoint, set the `PROMETHEUS_ENABLED` setting to `True`.
 
+### Startup commands
+
+`common.core` registers composite startup verbs on the `flagsmith` command, so a container entrypoint can sequence startup itself rather than via a shell script:
+
+- `flagsmith serve` — wait for the database, then start the API.
+- `flagsmith migrate` — migrate each configured database, then create the cache table. A bare invocation runs the full sequence; `flagsmith migrate <app> <name>` defers to Django for targeted migrations.
+- `flagsmith run-task-processor` — migrate, wait for migrations to be applied, then start the Task Processor.
+- `flagsmith migrate-and-serve` — migrate, run any configured startup commands, then start the API.
+
+The verbs are configured through these settings:
+
+| Setting | Default | Purpose |
+| --- | --- | --- |
+| `FLAGSMITH_MIGRATE_DATABASES` | `["default"]` | Database aliases the `migrate` step applies, in order. |
+| `FLAGSMITH_WAIT_FOR_MIGRATIONS_DATABASES` | `["default"]` | Database aliases `run-task-processor` waits on before starting. |
+| `FLAGSMITH_STARTUP_COMMANDS` | `[]` | Management commands run, in order, between migrate and serve in `migrate-and-serve`. |
+
+…and these environment variables:
+
+| Environment variable | Default | Purpose |
+| --- | --- | --- |
+| `SKIP_WAIT_FOR_DB` | unset | When set, skip waiting for the database. |
+| `TASK_PROCESSOR_NUM_THREADS` | `5` | Number of worker threads. |
+| `TASK_PROCESSOR_SLEEP_INTERVAL_MS` | `500` | Millis each worker waits before checking for new tasks (falls back to `TASK_PROCESSOR_SLEEP_INTERVAL`). |
+| `TASK_PROCESSOR_GRACE_PERIOD_MS` | `20000` | Millis before a running task is considered stuck. |
+| `TASK_PROCESSOR_QUEUE_POP_SIZE` | `10` | Tasks each worker pops from the queue per cycle. |
+
 ### Pre-commit hooks
 
 This repo provides a [`flagsmith-lint-tests`](.pre-commit-hooks.yaml) hook that enforces test conventions:
@@ -166,7 +195,8 @@ OTel instrumentation is opt-in, controlled by environment variables:
 
 | Variable                          | Description                                                                                                           | Default         |
 | --------------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------- |
-| `OTEL_EXPORTER_OTLP_ENDPOINT`     | Base OTLP endpoint (e.g. `http://collector:4318`). If unset, no OTel setup occurs.                                    | _(disabled)_    |
+| `OTEL_EXPORTER_OTLP_ENDPOINT`     | Base OTLP endpoint (e.g. `http://collector:4318` for HTTP, `http://collector:4317` for gRPC). If unset, no OTel setup occurs. | _(disabled)_    |
+| `OTEL_EXPORTER_OTLP_PROTOCOL`     | OTLP transport: `grpc` or `http/protobuf` (default).                                                                  | `http/protobuf` |
 | `OTEL_SERVICE_NAME`               | The `service.name` resource attribute. Defaults to `flagsmith-task-processor` when running the task processor.         | `flagsmith-api` |
 | `OTEL_TRACING_EXCLUDED_URL_PATHS` | Comma-separated URL paths to exclude from tracing (e.g. `health/liveness,health/readiness`).                          | _(none)_        |
 
@@ -176,7 +206,7 @@ Standard `OTEL_*` env vars (e.g. `OTEL_RESOURCE_ATTRIBUTES`, `OTEL_EXPORTER_OTLP
 
 When `OTEL_EXPORTER_OTLP_ENDPOINT` is set, `ensure_cli_env()` sets up:
 
-- **Tracing**: `TracerProvider` with OTLP/HTTP span export, W3C `TraceContext` + `Baggage` propagation, and auto-instrumentation for:
+- **Tracing**: `TracerProvider` with OTLP span export (HTTP or gRPC, per `OTEL_EXPORTER_OTLP_PROTOCOL`), W3C `TraceContext` + `Baggage` propagation, and auto-instrumentation for:
   - **Django** (`DjangoInstrumentor`): creates a root span per HTTP request with span names formatted as `{METHOD} {route_template}` (e.g. `GET /api/v1/projects/{pk}/`).
   - **psycopg2** (`Psycopg2Instrumentor`): creates child spans for each SQL query with `db.system`, `db.statement`, and `db.name` attributes. SQL commenter is enabled, adding trace context as SQL comments for database-side correlation.
   - **Redis** (`RedisInstrumentor`): creates child spans for each Redis command with `db.system` and `db.statement` attributes.

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/README.md

@@ -35,6 +35,33 @@ This enables the `route` label for Prometheus HTTP metrics.
 
 5. To enable the `/metrics` endpoint, set the `PROMETHEUS_ENABLED` setting to `True`.
 
+### Startup commands
+
+`common.core` registers composite startup verbs on the `flagsmith` command, so a container entrypoint can sequence startup itself rather than via a shell script:
+
+- `flagsmith serve` — wait for the database, then start the API.
+- `flagsmith migrate` — migrate each configured database, then create the cache table. A bare invocation runs the full sequence; `flagsmith migrate <app> <name>` defers to Django for targeted migrations.
+- `flagsmith run-task-processor` — migrate, wait for migrations to be applied, then start the Task Processor.
+- `flagsmith migrate-and-serve` — migrate, run any configured startup commands, then start the API.
+
+The verbs are configured through these settings:
+
+| Setting | Default | Purpose |
+| --- | --- | --- |
+| `FLAGSMITH_MIGRATE_DATABASES` | `["default"]` | Database aliases the `migrate` step applies, in order. |
+| `FLAGSMITH_WAIT_FOR_MIGRATIONS_DATABASES` | `["default"]` | Database aliases `run-task-processor` waits on before starting. |
+| `FLAGSMITH_STARTUP_COMMANDS` | `[]` | Management commands run, in order, between migrate and serve in `migrate-and-serve`. |
+
+…and these environment variables:
+
+| Environment variable | Default | Purpose |
+| --- | --- | --- |
+| `SKIP_WAIT_FOR_DB` | unset | When set, skip waiting for the database. |
+| `TASK_PROCESSOR_NUM_THREADS` | `5` | Number of worker threads. |
+| `TASK_PROCESSOR_SLEEP_INTERVAL_MS` | `500` | Millis each worker waits before checking for new tasks (falls back to `TASK_PROCESSOR_SLEEP_INTERVAL`). |
+| `TASK_PROCESSOR_GRACE_PERIOD_MS` | `20000` | Millis before a running task is considered stuck. |
+| `TASK_PROCESSOR_QUEUE_POP_SIZE` | `10` | Tasks each worker pops from the queue per cycle. |
+
 ### Pre-commit hooks
 
 This repo provides a [`flagsmith-lint-tests`](.pre-commit-hooks.yaml) hook that enforces test conventions:
@@ -106,7 +133,8 @@ OTel instrumentation is opt-in, controlled by environment variables:
 
 | Variable                          | Description                                                                                                           | Default         |
 | --------------------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------- |
-| `OTEL_EXPORTER_OTLP_ENDPOINT`     | Base OTLP endpoint (e.g. `http://collector:4318`). If unset, no OTel setup occurs.                                    | _(disabled)_    |
+| `OTEL_EXPORTER_OTLP_ENDPOINT`     | Base OTLP endpoint (e.g. `http://collector:4318` for HTTP, `http://collector:4317` for gRPC). If unset, no OTel setup occurs. | _(disabled)_    |
+| `OTEL_EXPORTER_OTLP_PROTOCOL`     | OTLP transport: `grpc` or `http/protobuf` (default).                                                                  | `http/protobuf` |
 | `OTEL_SERVICE_NAME`               | The `service.name` resource attribute. Defaults to `flagsmith-task-processor` when running the task processor.         | `flagsmith-api` |
 | `OTEL_TRACING_EXCLUDED_URL_PATHS` | Comma-separated URL paths to exclude from tracing (e.g. `health/liveness,health/readiness`).                          | _(none)_        |
 
@@ -116,7 +144,7 @@ Standard `OTEL_*` env vars (e.g. `OTEL_RESOURCE_ATTRIBUTES`, `OTEL_EXPORTER_OTLP
 
 When `OTEL_EXPORTER_OTLP_ENDPOINT` is set, `ensure_cli_env()` sets up:
 
-- **Tracing**: `TracerProvider` with OTLP/HTTP span export, W3C `TraceContext` + `Baggage` propagation, and auto-instrumentation for:
+- **Tracing**: `TracerProvider` with OTLP span export (HTTP or gRPC, per `OTEL_EXPORTER_OTLP_PROTOCOL`), W3C `TraceContext` + `Baggage` propagation, and auto-instrumentation for:
   - **Django** (`DjangoInstrumentor`): creates a root span per HTTP request with span names formatted as `{METHOD} {route_template}` (e.g. `GET /api/v1/projects/{pk}/`).
   - **psycopg2** (`Psycopg2Instrumentor`): creates child spans for each SQL query with `db.system`, `db.statement`, and `db.name` attributes. SQL commenter is enabled, adding trace context as SQL comments for database-side correlation.
   - **Redis** (`RedisInstrumentor`): creates child spans for each Redis command with `db.system` and `db.statement` attributes.

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "flagsmith-common"
-version = "3.10.1"
+version = "3.12.0"
 description = "Flagsmith's common library"
 requires-python = ">=3.11,<4.0"
 dependencies = []
@@ -29,6 +29,7 @@ optional-dependencies = { test-tools = [
     "inflection",
     "opentelemetry-api (>=1.25,<2)",
     "opentelemetry-sdk (>=1.25,<2)",
+    "opentelemetry-exporter-otlp-proto-grpc (>=1.25,<2)",
     "opentelemetry-exporter-otlp-proto-http (>=1.25,<2)",
     "sentry-sdk (>=2.0.0,<3.0.0)",
     "structlog (>=24.4,<26)",
@@ -37,6 +38,7 @@ optional-dependencies = { test-tools = [
     "backoff (>=2.2.1,<3.0.0)",
     "django (>4,<6)",
     "django-health-check",
+    "environs (<16)",
     "opentelemetry-api (>=1.25,<2)",
     "prometheus-client (>=0.0.16)",
 ], flagsmith-schemas = [
@@ -96,7 +98,7 @@ dev = [
     "pytest-freezegun (>=0.4.2, <1.0.0)",
     "pytest-httpserver (>=1.1.3, <2.0.0)",
     "pytest-mock (>=3.14.0, <4.0.0)",
-    "setuptools (>=78.1.1, <79.0.0)",
+    "setuptools (>=83.0.0, <83.1.0)",
     "types-python-dateutil (>=2.9.0.20250516, <3.0.0)",
     "types-simplejson (>=3.20.0.20250326, <4.0.0)",
 ]

flagsmith_common-3.12.0/src/common/core/cli/run.py

@@ -0,0 +1,79 @@
+"""Composite startup verbs for the `flagsmith` entrypoint.
+
+These mirror the sequencing that Core API's `run-docker.sh` performed in
+shell, but run in a single process so the Django boot cost is paid once.
+"""
+
+import os
+import shlex
+
+from django.conf import settings
+from django.core.management import (
+    execute_from_command_line as django_execute_from_command_line,
+)
+
+WAIT_FOR_MIGRATIONS_TIMEOUT_SECONDS = 30
+
+
+def _wait_for_db(
+    *,
+    wait_for_migrations: bool = False,
+    database: str = "default",
+    wait_for: int | None = None,
+) -> None:
+    if os.environ.get("SKIP_WAIT_FOR_DB"):
+        return
+    args = ["waitfordb", "--database", database]
+    if wait_for is not None:
+        args += ["--waitfor", str(wait_for)]
+    if wait_for_migrations:
+        args.append("--migrations")
+    django_execute_from_command_line(["flagsmith", *args])
+
+
+def _migrate() -> None:
+    _wait_for_db()
+    databases: list[str] = getattr(settings, "FLAGSMITH_MIGRATE_DATABASES", ["default"])
+    for database in databases:
+        django_execute_from_command_line(
+            ["flagsmith", "migrate", "--database", database]
+        )
+    django_execute_from_command_line(["flagsmith", "createcachetable"])
+
+
+def migrate(argv: list[str], *, prog: str) -> None:
+    """Migrate the configured databases, then create the cache table."""
+    if argv:
+        django_execute_from_command_line(["flagsmith", "migrate", *argv])
+        return
+    _migrate()
+
+
+def serve(argv: list[str], *, prog: str) -> None:
+    """Wait for the database, then start the API server."""
+    _wait_for_db()
+    django_execute_from_command_line(["flagsmith", "start", "api", *argv])
+
+
+def run_task_processor(argv: list[str], *, prog: str) -> None:
+    """Migrate, wait for migrations to be applied, then start the task processor."""
+    _migrate()
+    databases: list[str] = getattr(
+        settings, "FLAGSMITH_WAIT_FOR_MIGRATIONS_DATABASES", ["default"]
+    )
+    for database in databases:
+        _wait_for_db(
+            wait_for_migrations=True,
+            database=database,
+            wait_for=WAIT_FOR_MIGRATIONS_TIMEOUT_SECONDS,
+        )
+    django_execute_from_command_line(["flagsmith", "start", "task-processor", *argv])
+
+
+def migrate_and_serve(argv: list[str], *, prog: str) -> None:
+    """Migrate, run any configured startup commands, then start the API server."""
+    _migrate()
+    startup_commands: list[str] = getattr(settings, "FLAGSMITH_STARTUP_COMMANDS", [])
+    for command in startup_commands:
+        django_execute_from_command_line(["flagsmith", *shlex.split(command)])
+    django_execute_from_command_line(["flagsmith", "start", "api", *argv])

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/src/common/core/constants.py

@@ -1,3 +1,7 @@
+from common.core.types import OtlpProtocol
+
 DEFAULT_PROMETHEUS_MULTIPROC_DIR_NAME = "flagsmith-prometheus"
 
 LOGGING_DEFAULT_ROOT_LOG_LEVEL = "WARNING"
+
+DEFAULT_OTLP_PROTOCOL: OtlpProtocol = "http/protobuf"

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/src/common/core/main.py

@@ -8,10 +8,12 @@ from tempfile import mkdtemp
 from django.core.management import (
     execute_from_command_line as django_execute_from_command_line,
 )
-from environs import Env
+from environs import Env, validate
 
-from common.core.cli import healthcheck
+from common.core.cli import healthcheck, run
+from common.core.constants import DEFAULT_OTLP_PROTOCOL
 from common.core.logging import setup_logging
+from common.core.types import OtlpProtocol
 from common.gunicorn.processors import make_gunicorn_access_processor
 
 env = Env()
@@ -63,6 +65,7 @@ def ensure_cli_env() -> typing.Generator[None, None, None]:
             build_otel_log_provider,
             build_tracer_provider,
             make_structlog_otel_processor,
+            resolve_otlp_export_endpoints,
             setup_tracing,
         )
 
@@ -72,17 +75,31 @@ def ensure_cli_env() -> typing.Generator[None, None, None]:
             else "flagsmith-api"
         )
         service_name = env.str("OTEL_SERVICE_NAME", default_service_name)
+        otel_protocol = typing.cast(
+            OtlpProtocol,
+            env.str(
+                "OTEL_EXPORTER_OTLP_PROTOCOL",
+                DEFAULT_OTLP_PROTOCOL,
+                validate=validate.OneOf(typing.get_args(OtlpProtocol)),
+            ),
+        )
+        traces_endpoint, logs_endpoint = resolve_otlp_export_endpoints(
+            otel_endpoint,
+            otel_protocol,
+        )
         log_provider = build_otel_log_provider(
-            endpoint=f"{otel_endpoint}/v1/logs",
+            endpoint=logs_endpoint,
             service_name=service_name,
+            protocol=otel_protocol,
         )
         otel_processors = [
             add_otel_trace_context,
             make_structlog_otel_processor(log_provider),
         ]
         tracer_provider = build_tracer_provider(
-            endpoint=f"{otel_endpoint}/v1/traces",
+            endpoint=traces_endpoint,
             service_name=service_name,
+            protocol=otel_protocol,
         )
         excluded_urls = env.str("OTEL_TRACING_EXCLUDED_URL_PATHS", None)
         ctx.enter_context(setup_tracing(tracer_provider, excluded_urls=excluded_urls))
@@ -118,6 +135,11 @@ def execute_from_command_line(argv: list[str]) -> None:
             # Backwards compatibility for task-processor health checks
             # See https://github.com/Flagsmith/flagsmith-task-processor/issues/24
             "checktaskprocessorthreadhealth": healthcheck.main,
+            # Composite startup verbs, formerly run-docker.sh
+            "serve": run.serve,
+            "migrate": run.migrate,
+            "run-task-processor": run.run_task_processor,
+            "migrate-and-serve": run.migrate_and_serve,
         }[subcommand]
     except (IndexError, KeyError):
         logger.info("Invoking Django")

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/src/common/core/otel.py

@@ -11,26 +11,37 @@ from opentelemetry import baggage, trace
 from opentelemetry import context as otel_context
 from opentelemetry._logs import SeverityNumber
 from opentelemetry.baggage.propagation import W3CBaggagePropagator
+from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
+    OTLPLogExporter as GrpcOTLPLogExporter,
+)
+from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
+    OTLPSpanExporter as GrpcOTLPSpanExporter,
+)
 from opentelemetry.exporter.otlp.proto.http._log_exporter import (
-    OTLPLogExporter,
+    OTLPLogExporter as HttpOTLPLogExporter,
 )
 from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
-    OTLPSpanExporter,
+    OTLPSpanExporter as HttpOTLPSpanExporter,
 )
 from opentelemetry.propagate import set_global_textmap
 from opentelemetry.propagators.composite import CompositePropagator
 from opentelemetry.propagators.textmap import TextMapPropagator
 from opentelemetry.sdk._logs import LoggerProvider
-from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
+from opentelemetry.sdk._logs.export import (
+    BatchLogRecordProcessor,
+    LogRecordExporter,
+)
 from opentelemetry.sdk.resources import Resource
 from opentelemetry.sdk.trace import TracerProvider
-from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.sdk.trace.export import BatchSpanProcessor, SpanExporter
 from opentelemetry.trace.propagation.tracecontext import (
     TraceContextTextMapPropagator,
 )
 from opentelemetry.util.types import AnyValue, Attributes
 from structlog.typing import EventDict, Processor
 
+from common.core.types import OtlpProtocol
+
 _SEVERITY_MAP: dict[str, SeverityNumber] = {
     "debug": SeverityNumber.DEBUG,
     "info": SeverityNumber.INFO,
@@ -51,6 +62,17 @@ _RESERVED_KEYS = frozenset(
 )
 
 
+def resolve_otlp_export_endpoints(
+    base_endpoint: str,
+    protocol: OtlpProtocol,
+) -> tuple[str, str]:
+    """Resolve trace and log exporter endpoints from a base OTLP endpoint."""
+    normalised_base = base_endpoint.rstrip("/")
+    if protocol == "grpc":
+        return normalised_base, normalised_base
+    return f"{normalised_base}/v1/traces", f"{normalised_base}/v1/logs"
+
+
 def get_otel_event_name(*, logger_name: str | None, body: str) -> str:
     """Build the event name that reaches OTel from a structlog `(logger, event)`.
 
@@ -153,20 +175,38 @@ def map_value_to_otel_value(value: object) -> str | int | float | bool:
     return json.dumps(value, default=str)
 
 
-def build_otel_log_provider(*, endpoint: str, service_name: str) -> LoggerProvider:
-    """Create and configure an OTel LoggerProvider with OTLP/HTTP export."""
+def build_otel_log_provider(
+    *,
+    endpoint: str,
+    service_name: str,
+    protocol: OtlpProtocol,
+) -> LoggerProvider:
+    """Create and configure an OTel LoggerProvider with OTLP export."""
     resource = Resource.create({"service.name": service_name})
     provider = LoggerProvider(resource=resource)
-    exporter = OTLPLogExporter(endpoint=endpoint)
+    exporter: LogRecordExporter
+    if protocol == "grpc":
+        exporter = GrpcOTLPLogExporter(endpoint=endpoint)
+    else:
+        exporter = HttpOTLPLogExporter(endpoint=endpoint)
     provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
     return provider
 
 
-def build_tracer_provider(*, endpoint: str, service_name: str) -> TracerProvider:
-    """Create a TracerProvider with OTLP/HTTP export."""
+def build_tracer_provider(
+    *,
+    endpoint: str,
+    service_name: str,
+    protocol: OtlpProtocol,
+) -> TracerProvider:
+    """Create a TracerProvider with OTLP export."""
     resource = Resource.create({"service.name": service_name})
     tracer_provider = TracerProvider(resource=resource)
-    span_exporter = OTLPSpanExporter(endpoint=endpoint)
+    span_exporter: SpanExporter
+    if protocol == "grpc":
+        span_exporter = GrpcOTLPSpanExporter(endpoint=endpoint)
+    else:
+        span_exporter = HttpOTLPSpanExporter(endpoint=endpoint)
     tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
     return tracer_provider
 

flagsmith_common-3.12.0/src/common/core/types.py

@@ -0,0 +1,3 @@
+from typing import Literal
+
+OtlpProtocol = Literal["grpc", "http/protobuf"]

flagsmith_common-3.12.0/src/task_processor/constants.py

@@ -0,0 +1,4 @@
+DEFAULT_TASK_PROCESSOR_NUM_THREADS: int = 5
+DEFAULT_TASK_PROCESSOR_SLEEP_INTERVAL_MS: int = 500
+DEFAULT_TASK_PROCESSOR_GRACE_PERIOD_MS: int = 20000
+DEFAULT_TASK_PROCESSOR_QUEUE_POP_SIZE: int = 10

{flagsmith_common-3.10.1 → flagsmith_common-3.12.0}/src/task_processor/utils.py

@@ -1,14 +1,25 @@
 import argparse
 import inspect
 import logging
+import os
 from contextlib import contextmanager
 from typing import Any, Generator
 
+from environs import Env
+
+from task_processor.constants import (
+    DEFAULT_TASK_PROCESSOR_GRACE_PERIOD_MS,
+    DEFAULT_TASK_PROCESSOR_NUM_THREADS,
+    DEFAULT_TASK_PROCESSOR_QUEUE_POP_SIZE,
+    DEFAULT_TASK_PROCESSOR_SLEEP_INTERVAL_MS,
+)
 from task_processor.threads import TaskRunnerCoordinator
 from task_processor.types import TaskCallable, TaskProcessorConfig
 
 logger = logging.getLogger(__name__)
 
+env = Env()
+
 
 def get_task_identifier_from_function(
     function: TaskCallable[Any],
@@ -24,25 +35,41 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
         "--numthreads",
         type=int,
         help="Number of worker threads to run.",
-        default=5,
+        default=env.int(
+            "TASK_PROCESSOR_NUM_THREADS",
+            default=DEFAULT_TASK_PROCESSOR_NUM_THREADS,
+        ),
     )
     parser.add_argument(
         "--sleepintervalms",
         type=int,
         help="Number of millis each worker waits before checking for new tasks",
-        default=2000,
+        default=(
+            env.int("TASK_PROCESSOR_SLEEP_INTERVAL_MS")
+            if "TASK_PROCESSOR_SLEEP_INTERVAL_MS" in os.environ
+            else env.int(
+                "TASK_PROCESSOR_SLEEP_INTERVAL",
+                default=DEFAULT_TASK_PROCESSOR_SLEEP_INTERVAL_MS,
+            )
+        ),
     )
     parser.add_argument(
         "--graceperiodms",
         type=int,
         help="Number of millis before running task is considered 'stuck'.",
-        default=20000,
+        default=env.int(
+            "TASK_PROCESSOR_GRACE_PERIOD_MS",
+            default=DEFAULT_TASK_PROCESSOR_GRACE_PERIOD_MS,
+        ),
     )
     parser.add_argument(
         "--queuepopsize",
         type=int,
         help="Number of tasks each worker will pop from the queue on each cycle.",
-        default=10,
+        default=env.int(
+            "TASK_PROCESSOR_QUEUE_POP_SIZE",
+            default=DEFAULT_TASK_PROCESSOR_QUEUE_POP_SIZE,
+        ),
     )