kiba-core 0.5.3.dev70__tar.gz → 0.5.3.dev72__tar.gz

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/CHANGELOG.md

@@ -8,6 +8,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ## [Unreleased]
 
 ### Added
+- [MAJOR] Updated `RestEthClient` to raise `RpcServerException` (a `ServerException` keeping the original status code and message, with the original exception as its cause) when the node responds with a 5xx, so callers can tell node failures apart from other server errors; `RequesterTimeoutException` is still raised unchanged. Code catching `InternalServerErrorException` (or `BadGatewayException`, `ServiceUnavailableException`, …) around `RestEthClient` calls must catch `RpcServerException` instead
+- [MINOR] Added `RestEthClient.get_code`
+- [MINOR] Added optional `maxConcurrentRequestCount` and `semaphore` to `RestEthClient` to limit concurrent requests; pass the same `semaphore` to several clients to limit them together (it takes precedence over `maxConcurrentRequestCount`)
+- [MINOR] Added `EthClientRegistry` to register regular, archive and paymaster `RestEthClient`s per chain and pick the archive client for dates older than `archiveThresholdDays` (default 15)
+- [PATCH] Fixed log records from `logging.info`, `warning`, `error`, `critical`, `debug`, `exception`, `stat` and `api` recording their location as `core/logging.py` instead of the calling code; this was only visible in `init_logging`'s text format, since the JSON and basic formats don't output the location
+- [MINOR] Updated `init_logging`, `init_json_logging` and `init_basic_logging` to set the loggers of the third-party libraries core uses (`DEFAULT_EXTERNAL_LOGGER_NAMES`: httpx2, httpcore2, sqlalchemy, aiosqlite, asyncpg, botocore, aiobotocore, azure, apscheduler, web3) to `WARNING`; pass `extraLoggerNames` to quieten more loggers on top of these, or `loggerNames` to replace them (`loggerNames=[]` keeps the previous behaviour). `init_external_loggers` takes the same arguments and now defaults to the same list
 - [MINOR] Added `core.cron` (install with the `cron` extra): `CronJob` schedules (interval with jitter, or crontab) that declare a `jobType`, `CronJobProcessor`s that declare the `jobTypes` they process, and `CronExecutor`, which takes a list of processors, routes each job to the processor for its type (failing at startup if a job type has no processor or more than one), runs jobs on an APScheduler scheduler, logs each run as a `CRON` api log with its status and duration, and keeps running when a job fails; also added `ApiRequestCronJob` with `ApiRequestCronJobProcessor` (needs the `requester` extra) for jobs that call a URL
 - [MAJOR] Updated `Requester` to raise `RequesterTimeoutException` (a `GatewayTimeoutException`, 504) instead of `httpx` timeout exceptions; it carries the method, the URL without credentials or query string, the timeout, how long the request ran and the `httpx` timeout type, and keeps the original exception as its cause. Code catching `httpx.TimeoutException` (or `ReadTimeout`, `ConnectTimeout`, …) around `Requester` calls must catch `RequesterTimeoutException` instead
 - [PATCH] Updated `MessageQueueProcessor` failure notifications to show the exception type before its message, so failures from exceptions without text (e.g. `httpx` timeouts) are still identifiable

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kiba-core
-Version: 0.5.3.dev70
+Version: 0.5.3.dev72
 Summary: Kiba Labs' python utilities
 Project-URL: repository, https://github.com/kibalabs/core-py
 Requires-Python: <4,>=3.12

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/core/logging.py

@@ -120,36 +120,43 @@ def init_logger(logger: Logger, loggingLevel: int, handler: StreamHandler) -> No
     logger.setLevel(level=loggingLevel)
 
 
-def init_external_loggers(loggerNames: Collection[str], loggingLevel: int = logging.WARNING) -> None:
-    for loggerName in loggerNames:
+# NOTE(krishan711): third-party libraries pulled in by core's extras; their info-level logs drown out the app's own
+DEFAULT_EXTERNAL_LOGGER_NAMES = ('httpx2', 'httpcore2', 'sqlalchemy', 'aiosqlite', 'asyncpg', 'botocore', 'aiobotocore', 'azure', 'apscheduler', 'web3')
+
+
+def init_external_loggers(loggerNames: Collection[str] | None = None, extraLoggerNames: Collection[str] | None = None, loggingLevel: int = logging.WARNING) -> None:
+    for loggerName in {*(loggerNames if loggerNames is not None else DEFAULT_EXTERNAL_LOGGER_NAMES), *(extraLoggerNames or [])}:
         logging.getLogger(loggerName).setLevel(loggingLevel)
 
 
-def init_logging(name: str, version: str, environment: str, showDebug: bool = False, requestIdHolder: RequestIdHolder | None = None) -> None:
+def init_logging(name: str, version: str, environment: str, showDebug: bool = False, requestIdHolder: RequestIdHolder | None = None, loggerNames: Collection[str] | None = None, extraLoggerNames: Collection[str] | None = None) -> None:
     loggingLevel = logging.DEBUG if showDebug else logging.INFO
     for logFormat in ALL_LOGGER_FORMATS:
         logger = logging.getLogger(name=logFormat.loggerType)
         handler = StreamHandler(stream=sys.stdout)
         handler.setFormatter(fmt=KibaLoggingFormatter(logFormat=logFormat, name=name, version=version, environment=environment, requestIdHolder=requestIdHolder))
         init_logger(logger=logger, loggingLevel=loggingLevel, handler=handler)
+    init_external_loggers(loggerNames=loggerNames, extraLoggerNames=extraLoggerNames)
 
 
-def init_json_logging(name: str, version: str, environment: str, showDebug: bool = False, requestIdHolder: RequestIdHolder | None = None) -> None:
+def init_json_logging(name: str, version: str, environment: str, showDebug: bool = False, requestIdHolder: RequestIdHolder | None = None, loggerNames: Collection[str] | None = None, extraLoggerNames: Collection[str] | None = None) -> None:
     loggingLevel = logging.DEBUG if showDebug else logging.INFO
     for logFormat in ALL_LOGGER_FORMATS:
         logger = logging.getLogger(name=logFormat.loggerType)
         handler = StreamHandler(stream=sys.stdout)
         handler.setFormatter(fmt=KibaJsonLoggingFormatter(logFormat=logFormat, name=name, version=version, environment=environment, requestIdHolder=requestIdHolder))
         init_logger(logger=logger, loggingLevel=loggingLevel, handler=handler)
+    init_external_loggers(loggerNames=loggerNames, extraLoggerNames=extraLoggerNames)
 
 
-def init_basic_logging(showDebug: bool = False) -> None:
+def init_basic_logging(showDebug: bool = False, loggerNames: Collection[str] | None = None, extraLoggerNames: Collection[str] | None = None) -> None:
     loggingLevel = logging.DEBUG if showDebug else logging.INFO
     for logFormat in ALL_LOGGER_FORMATS:
         logger = logging.getLogger(name=logFormat.loggerType)
         handler = StreamHandler(stream=sys.stdout)
         handler.setFormatter(fmt=Formatter(fmt=logFormat.loggerFormat))
         init_logger(logger=logger, loggingLevel=loggingLevel, handler=handler)
+    init_external_loggers(loggerNames=loggerNames, extraLoggerNames=extraLoggerNames)
 
 
 def _serialize_numeric_value(value: Union[float, None]) -> str:
@@ -168,7 +175,8 @@ def stat(name: str, key: str, value: float = 1) -> None:
         nameValue = _serialize_string_value(value=str(name))
         keyValue = _serialize_string_value(value=str(key))
         statValue = _serialize_numeric_value(value=value)
-        STAT_LOGGER.log(level=logging.INFO, msg='', extra={'statName': nameValue, 'statKey': keyValue, 'statValue': statValue})
+        # NOTE(krishan711): stacklevel=2 skips this function so records point at the caller
+        STAT_LOGGER.log(level=logging.INFO, msg='', extra={'statName': nameValue, 'statKey': keyValue, 'statValue': statValue}, stacklevel=2)
 
 
 # TODO(krishan711): make pathPattern mandatory in next major release
@@ -184,6 +192,7 @@ def api(action: str, path: str, query: str, pathPattern: str | None = None, resp
             level=logging.INFO,
             msg='',
             extra={'apiAction': actionString, 'apiPath': pathString, 'apiPathPattern': pathPatternString, 'apiQuery': queryString, 'apiResponse': responseString or '', 'apiDuration': durationString or ''},
+            stacklevel=2,
         )
 
 
@@ -197,9 +206,10 @@ INFO = logging.INFO
 DEBUG = logging.DEBUG
 
 
-def _log(level: int, msg: str, *args: Any, **kwargs: Any) -> None:  # type: ignore[explicit-any]
+def _log(level: int, msg: str, *args: Any, stacklevel: int = 1, **kwargs: Any) -> None:  # type: ignore[explicit-any]
     if ROOT_LOGGER.isEnabledFor(level=level):
-        ROOT_LOGGER._log(level=level, msg=msg, args=args, **kwargs)  # noqa: SLF001
+        # NOTE(krishan711): +2 skips this function and the public wrapper (info, error, ...) so records point at the caller
+        ROOT_LOGGER._log(level=level, msg=msg, args=args, stacklevel=stacklevel + 2, **kwargs)  # noqa: SLF001
 
 
 def critical(msg: str, *args: Any, **kwargs: Any) -> None:  # type: ignore[explicit-any]

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/core/web3/eth_client.py

@@ -1,4 +1,5 @@
 import asyncio
+import contextlib
 import typing
 from typing import Any
 
@@ -24,8 +25,11 @@ from core.exceptions import BadRequestException
 from core.exceptions import ClientException
 from core.exceptions import KibaException
 from core.exceptions import NotFoundException
+from core.exceptions import ServerException
 from core.exceptions import TooManyRequestsException
 from core.requester import Requester
+from core.requester.requester import RequesterTimeoutException
+from core.requester.requester import ResponseException
 from core.util import chain_util
 from core.util import json_util
 from core.util.typing_util import JsonObject
@@ -61,6 +65,11 @@ class TransactionFailedException(KibaException):
         return output
 
 
+class RpcServerException(ServerException):
+    def __init__(self, message: str | None = None, statusCode: int = 500) -> None:
+        super().__init__(message=message or 'RPC Server Error', statusCode=statusCode)
+
+
 class EthClientInterface:
     def __init__(self, web3Connection: Web3, chainId: int, isTestnet: bool = False) -> None:
         self.w3 = web3Connection
@@ -297,12 +306,26 @@ class Web3EthClient(EthClientInterface):
 
 class RestEthClient(EthClientInterface):
     # NOTE(krishan711): find docs at https://eth.wiki/json-rpc/API
-    def __init__(self, url: str, requester: Requester, chainId: int, isTestnet: bool = False, shouldBackoffRetryOnRateLimit: bool = True, retryLimit: int = 10) -> None:
+    def __init__(
+        self,
+        url: str,
+        requester: Requester,
+        chainId: int,
+        isTestnet: bool = False,
+        shouldBackoffRetryOnRateLimit: bool = True,
+        retryLimit: int = 10,
+        maxConcurrentRequestCount: int | None = None,
+        semaphore: asyncio.Semaphore | None = None,
+    ) -> None:
         super().__init__(web3Connection=Web3(), chainId=chainId, isTestnet=isTestnet)
         self.url = url
         self.requester = requester
         self.shouldBackoffRetryOnRateLimit = shouldBackoffRetryOnRateLimit
         self.retryLimit = retryLimit
+        # NOTE(krishan711): pass the same semaphore to several clients to cap their combined concurrent requests, or set maxConcurrentRequestCount to cap just this client
+        if semaphore is None and maxConcurrentRequestCount is not None:
+            semaphore = asyncio.Semaphore(maxConcurrentRequestCount)
+        self.semaphore = semaphore
         self.w3 = Web3()
 
     @staticmethod
@@ -310,39 +333,45 @@ class RestEthClient(EthClientInterface):
         return int(value, 16)
 
     async def _make_request(self, method: str, params: ListAny | None = None) -> DictStrAny:
-        retryCount = 0
-        initialBackoffSeconds = 1.0
-        responseDict: DictStrAny = {}
-        while True:
-            try:
-                response = await self.requester.post_json(
-                    url=self.url,
-                    dataDict={'jsonrpc': '2.0', 'method': method, 'params': params or [], 'id': None},
-                    timeout=10,
-                )
-                responseDict = response.json()
-                if responseDict.get('error'):
-                    errorMessage = responseDict['error'].get('message') or responseDict['error'].get('details') or json_util.dumps(responseDict['error'])
-                    raise BadRequestException(message=errorMessage)
-            except ClientException as exception:
-                # NOTE(krishan711): this is just here to debug 429s from coinbase, remove when done
-                logging.info(f'caught exception on _make_request: {exception!s}')
-                exceptionMessage = exception.message or ''
-                if (
-                    isinstance(exception, TooManyRequestsException)
-                    # NOTE(krishan711): sometimes coinbase returns an error message with 200 status
-                    or 'over rate limit' in exceptionMessage
-                    or '429 Too Many Requests' in exceptionMessage
-                ):
-                    if not self.shouldBackoffRetryOnRateLimit or retryCount >= self.retryLimit:
+        async with self.semaphore or contextlib.nullcontext():
+            retryCount = 0
+            initialBackoffSeconds = 1.0
+            responseDict: DictStrAny = {}
+            while True:
+                try:
+                    response = await self.requester.post_json(
+                        url=self.url,
+                        dataDict={'jsonrpc': '2.0', 'method': method, 'params': params or [], 'id': None},
+                        timeout=10,
+                    )
+                    responseDict = response.json()
+                    if responseDict.get('error'):
+                        errorMessage = responseDict['error'].get('message') or responseDict['error'].get('details') or json_util.dumps(responseDict['error'])
+                        raise BadRequestException(message=errorMessage)
+                except ClientException as exception:
+                    # NOTE(krishan711): this is just here to debug 429s from coinbase, remove when done
+                    logging.info(f'caught exception on _make_request: {exception!s}')
+                    exceptionMessage = exception.message or ''
+                    if (
+                        isinstance(exception, TooManyRequestsException)
+                        # NOTE(krishan711): sometimes coinbase returns an error message with 200 status
+                        or 'over rate limit' in exceptionMessage
+                        or '429 Too Many Requests' in exceptionMessage
+                    ):
+                        if not self.shouldBackoffRetryOnRateLimit or retryCount >= self.retryLimit:
+                            raise
+                        retryCount += 1
+                        exponentialBackoffSeconds = initialBackoffSeconds * (2 ** (retryCount - 1))
+                        logging.info(f'Retrying {method} after {exponentialBackoffSeconds} seconds due to rate limit exceeded: {exception!s}')
+                        await asyncio.sleep(exponentialBackoffSeconds)
+                        continue
+                    raise
+                except (ServerException, ResponseException) as exception:
+                    # NOTE(krishan711): timeouts are raised from the requester rather than the node so they keep their own type
+                    if isinstance(exception, RequesterTimeoutException) or exception.statusCode < 500:  # noqa: PLR2004
                         raise
-                    retryCount += 1
-                    exponentialBackoffSeconds = initialBackoffSeconds * (2 ** (retryCount - 1))
-                    logging.info(f'Retrying {method} after {exponentialBackoffSeconds} seconds due to rate limit exceeded: {exception!s}')
-                    await asyncio.sleep(exponentialBackoffSeconds)
-                    continue
-                raise
-            return responseDict
+                    raise RpcServerException(message=exception.message, statusCode=exception.statusCode) from exception
+                return responseDict
 
     async def get_latest_block_number(self) -> int:
         response = await self._make_request(method='eth_blockNumber')
@@ -394,6 +423,10 @@ class RestEthClient(EthClientInterface):
             raise NotFoundException
         return typing.cast(TxReceipt, method_formatters.PYTHONIC_RESULT_FORMATTERS[RPC.eth_getTransactionReceipt](response['result']))
 
+    async def get_code(self, address: str, blockNumber: int | None = None) -> str:
+        response = await self._make_request(method='eth_getCode', params=[address, hex(blockNumber) if blockNumber is not None else 'latest'])
+        return str(response.get('result', '0x'))
+
     async def get_log_entries(
         self,
         topics: list[str] | None = None,
@@ -481,3 +514,4 @@ class RestEthClient(EthClientInterface):
             transactionData = '0x' + transactionData
         response = await self._make_request(method='eth_sendRawTransaction', params=[transactionData])
         return typing.cast(str, response['result'])
+

kiba_core-0.5.3.dev72/core/web3/eth_client_registry.py

@@ -0,0 +1,46 @@
+import datetime
+
+from core.exceptions import InternalServerErrorException
+from core.util import date_util
+from core.web3.eth_client import RestEthClient
+
+
+class EthClientRegistry:
+    def __init__(self, archiveThresholdDays: int = 15) -> None:
+        self.archiveThresholdDays = archiveThresholdDays
+        self.ethClientsByChainId: dict[int, RestEthClient] = {}
+        self.archiveEthClientsByChainId: dict[int, RestEthClient] = {}
+        self.paymasterEthClientsByChainId: dict[int, RestEthClient] = {}
+
+    def register_client(self, client: RestEthClient) -> None:
+        self.ethClientsByChainId[client.chainId] = client
+
+    def register_archive_client(self, client: RestEthClient) -> None:
+        self.archiveEthClientsByChainId[client.chainId] = client
+
+    def register_paymaster_client(self, client: RestEthClient) -> None:
+        self.paymasterEthClientsByChainId[client.chainId] = client
+
+    def get_client(self, chainId: int, date: datetime.date | None = None) -> RestEthClient:
+        # NOTE(krishan711): regular nodes may have pruned state older than the threshold so those reads go to the archive node
+        dateDiff = date_util.calculate_diff_days(startDate=date_util.datetime_from_date(date=date), endDate=date_util.datetime_from_now()) if date is not None else 0
+        if dateDiff > self.archiveThresholdDays:
+            return self.get_archive_client(chainId=chainId)
+        return self.get_regular_client(chainId=chainId)
+
+    def get_paymaster_client(self, chainId: int) -> RestEthClient:
+        if chainId not in self.paymasterEthClientsByChainId:
+            raise InternalServerErrorException(f'Chain {chainId} does not have a paymaster client')
+        return self.paymasterEthClientsByChainId[chainId]
+
+    def get_archive_client(self, chainId: int) -> RestEthClient:
+        if chainId not in self.archiveEthClientsByChainId:
+            raise InternalServerErrorException(f'Chain {chainId} does not have an archive client')
+        return self.archiveEthClientsByChainId[chainId]
+
+    def get_regular_client(self, chainId: int) -> RestEthClient:
+        if chainId in self.ethClientsByChainId:
+            return self.ethClientsByChainId[chainId]
+        if chainId in self.archiveEthClientsByChainId:
+            return self.archiveEthClientsByChainId[chainId]
+        raise InternalServerErrorException(f'Chain {chainId} does not have a regular client or an archive client')

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/kiba_core.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: kiba-core
-Version: 0.5.3.dev70
+Version: 0.5.3.dev72
 Summary: Kiba Labs' python utilities
 Project-URL: repository, https://github.com/kibalabs/core-py
 Requires-Python: <4,>=3.12

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/kiba_core.egg-info/SOURCES.txt

@@ -88,6 +88,7 @@ core/util/url_util.py
 core/util/value_holder.py
 core/web3/__init__.py
 core/web3/eth_client.py
+core/web3/eth_client_registry.py
 core/web3/multicall3.py
 kiba_core.egg-info/PKG-INFO
 kiba_core.egg-info/SOURCES.txt
@@ -97,6 +98,7 @@ kiba_core.egg-info/top_level.txt
 tests/__init__.py
 tests/test_database.py
 tests/test_exceptions.py
+tests/test_logging.py
 tests/api/test_api_response.py
 tests/api/test_authorizer.py
 tests/api/test_exception_handling_middleware.py
@@ -131,4 +133,5 @@ tests/util/test_process_util.py
 tests/util/test_string_util.py
 tests/util/test_url_util.py
 tests/web3/__init__.py
+tests/web3/test_eth_client_registry.py
 tests/web3/test_rest_eth_client.py

{kiba_core-0.5.3.dev70 → kiba_core-0.5.3.dev72}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "kiba-core"
-version = "0.5.3.dev70"
+version = "0.5.3.dev72"
 description = "Kiba Labs' python utilities"
 readme = "README.md"
 requires-python = ">=3.12, <4"

kiba_core-0.5.3.dev72/tests/test_logging.py

@@ -0,0 +1,259 @@
+import datetime
+import json
+import logging as pythonLogging
+from collections.abc import Iterator
+
+import pytest
+
+from core import logging
+from core.exceptions import NotFoundException
+from core.util.value_holder import RequestIdHolder
+
+APP_LOGGER_NAME = 'test-app-library'
+CORE_LOGGER_NAMES = [logFormat.loggerType for logFormat in logging.ALL_LOGGER_FORMATS]
+
+
+@pytest.fixture(autouse=True)
+def restore_logging_state() -> Iterator[None]:
+    externalLoggerNames = [*logging.DEFAULT_EXTERNAL_LOGGER_NAMES, APP_LOGGER_NAME]
+    originalExternalLevels = {loggerName: pythonLogging.getLogger(loggerName).level for loggerName in externalLoggerNames}
+    originalCoreState = {loggerName: (pythonLogging.getLogger(loggerName).handlers, pythonLogging.getLogger(loggerName).level, pythonLogging.getLogger(loggerName).propagate) for loggerName in CORE_LOGGER_NAMES}
+    for loggerName in externalLoggerNames:
+        pythonLogging.getLogger(loggerName).setLevel(pythonLogging.NOTSET)
+    yield
+    for loggerName, level in originalExternalLevels.items():
+        pythonLogging.getLogger(loggerName).setLevel(level)
+    for loggerName, (handlers, level, propagate) in originalCoreState.items():
+        logger = pythonLogging.getLogger(loggerName)
+        logger.handlers, logger.propagate = handlers, propagate
+        logger.setLevel(level)
+
+
+def _levels(loggerNames: list[str]) -> list[int]:
+    return [pythonLogging.getLogger(loggerName).level for loggerName in loggerNames]
+
+
+def _lines(capsys: pytest.CaptureFixture[str]) -> list[str]:
+    return capsys.readouterr().out.splitlines()
+
+
+def _json_lines(capsys: pytest.CaptureFixture[str]) -> list[dict[str, object]]:
+    return [json.loads(line) for line in _lines(capsys)]
+
+
+class TestExternalLoggers:
+    def test_init_quietens_core_external_loggers_by_default(self) -> None:
+        logging.init_basic_logging()
+        assert _levels(['httpx2', 'sqlalchemy', APP_LOGGER_NAME]) == [pythonLogging.WARNING, pythonLogging.WARNING, pythonLogging.NOTSET]
+
+    def test_extra_logger_names_are_added_to_the_defaults(self) -> None:
+        logging.init_basic_logging(extraLoggerNames=[APP_LOGGER_NAME])
+        assert _levels(['httpx2', APP_LOGGER_NAME]) == [pythonLogging.WARNING, pythonLogging.WARNING]
+
+    def test_logger_names_replace_the_defaults(self) -> None:
+        logging.init_basic_logging(loggerNames=[APP_LOGGER_NAME])
+        assert _levels(['httpx2', APP_LOGGER_NAME]) == [pythonLogging.NOTSET, pythonLogging.WARNING]
+
+    def test_every_init_function_applies_the_logger_names(self) -> None:
+        logging.init_logging(name='app', version='v1', environment='test', extraLoggerNames=[APP_LOGGER_NAME])
+        assert _levels(['httpx2', APP_LOGGER_NAME]) == [pythonLogging.WARNING, pythonLogging.WARNING]
+        pythonLogging.getLogger(APP_LOGGER_NAME).setLevel(pythonLogging.NOTSET)
+        logging.init_json_logging(name='app', version='v1', environment='test', loggerNames=[APP_LOGGER_NAME])
+        assert pythonLogging.getLogger(APP_LOGGER_NAME).level == pythonLogging.WARNING
+
+    def test_init_external_loggers_uses_the_given_level(self) -> None:
+        logging.init_external_loggers(extraLoggerNames=[APP_LOGGER_NAME], loggingLevel=pythonLogging.ERROR)
+        assert _levels(['httpx2', APP_LOGGER_NAME]) == [pythonLogging.ERROR, pythonLogging.ERROR]
+
+    def test_external_logger_level_applies_to_child_loggers(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        pythonLogging.getLogger('sqlalchemy.engine').info('noisy query log')
+        pythonLogging.getLogger('sqlalchemy.engine').warning('important warning')
+        assert _lines(capsys) == ['important warning']
+
+
+class TestBasicLogging:
+    def test_logs_info_and_above_but_not_debug(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.debug('debug line')
+        logging.info('info line')
+        logging.warning('warning line')
+        logging.error('error line')
+        logging.critical('critical line')
+        assert _lines(capsys) == ['info line', 'warning line', 'error line', 'critical line']
+
+    def test_show_debug_includes_debug(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging(showDebug=True)
+        logging.debug('debug line')
+        assert _lines(capsys) == ['debug line']
+
+    def test_messages_are_formatted_with_args(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.info('hello %s, you are %d', 'kiba', 3)
+        assert _lines(capsys) == ['hello kiba, you are 3']
+
+    def test_core_loggers_do_not_propagate_to_avoid_duplicate_output(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.init_basic_logging()
+        logging.stat(name='events', key='created')
+        assert _lines(capsys) == ['events:created:1']
+
+
+class TestStat:
+    def test_logs_name_key_and_value(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.stat(name='events', key='created', value=2.5)
+        assert _lines(capsys) == ['events:created:2.5']
+
+    def test_escapes_separators_and_rounds_values(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.stat(name='db:query', key='user:read', value=1.23456789)
+        logging.stat(name='whole', key='number', value=3.0)
+        assert _lines(capsys) == ['db__query:user__read:1.234568', 'whole:number:3']
+
+    def test_nothing_is_logged_when_the_stat_logger_is_disabled(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.STAT_LOGGER.setLevel(pythonLogging.WARNING)
+        logging.stat(name='events', key='created')
+        assert _lines(capsys) == []
+
+
+class TestApi:
+    def test_logs_action_path_query_response_and_duration(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.api(action='GET', path='/v1/users', query='limit=10', response=200, duration=0.25)
+        assert _lines(capsys) == ['GET:/v1/users:limit=10:200:0.25']
+
+    def test_missing_response_and_duration_are_empty(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.api(action='CRON', path='job', query='')
+        assert _lines(capsys) == ['CRON:job:::']
+
+    def test_escapes_separators(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.api(action='MESSAGE', path='a:b', query='x=1:2')
+        assert _lines(capsys) == ['MESSAGE:a__b:x=1__2::']
+
+    def test_nothing_is_logged_when_the_api_logger_is_disabled(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.API_LOGGER.setLevel(pythonLogging.WARNING)
+        logging.api(action='GET', path='/', query='')
+        assert _lines(capsys) == []
+
+
+class TestException:
+    def test_logs_kiba_exception_message_with_traceback(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        try:
+            raise NotFoundException(message='user not found')
+        except NotFoundException as exception:
+            logging.exception(exception)
+        lines = _lines(capsys)
+        assert lines[0] == 'user not found'
+        assert lines[1] == 'Traceback (most recent call last):'
+        assert lines[-1].endswith("message='user not found')")
+
+    def test_logs_plain_exception_text(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        try:
+            raise ValueError('bad value')
+        except ValueError as exception:
+            logging.exception(exception)
+        lines = _lines(capsys)
+        assert (lines[0], lines[-1]) == ('bad value', 'ValueError: bad value')
+
+    def test_logs_string_message_without_traceback_when_disabled(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_basic_logging()
+        logging.exception('something failed', exc_info=False)
+        assert _lines(capsys) == ['something failed']
+
+
+class TestTextLogging:
+    def test_includes_service_details_request_id_level_and_location(self, capsys: pytest.CaptureFixture[str]) -> None:
+        requestIdHolder = RequestIdHolder()
+        requestIdHolder.set_value(value='req-123')
+        logging.init_logging(name='app', version='v1', environment='test', requestIdHolder=requestIdHolder)
+        logging.warning('hello')
+        parts = _lines(capsys)[0].split(' - ')
+        datetime.datetime.strptime(parts[0], '%Y-%m-%dT%H:%M:%S.%f')  # noqa: DTZ007
+        assert parts[1:8] == ['KIBA_1', 'app', 'v1', 'test', 'req-123', 'WARNING', 'root']
+        assert parts[8].startswith('tests.test_logging.py:test_includes_service_details_request_id_level_and_location:')
+        assert parts[9] == 'hello'
+
+    def test_stat_api_and_exception_report_the_caller_location(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_logging(name='app', version='v1', environment='test')
+        logging.stat(name='events', key='created')
+        logging.api(action='GET', path='/', query='')
+        logging.exception('failed', exc_info=False)
+        locations = [line.split(' - ')[8].rsplit(':', 1)[0] for line in _lines(capsys)]
+        assert locations == ['tests.test_logging.py:test_stat_api_and_exception_report_the_caller_location'] * 3
+
+    def test_uses_each_loggers_own_format(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_logging(name='app', version='v1', environment='test')
+        logging.stat(name='events', key='created')
+        logging.api(action='GET', path='/', query='', response=200)
+        statParts, apiParts = (line.split(' - ') for line in _lines(capsys))
+        assert (statParts[1], statParts[-1]) == ('KIBA_STAT_1', 'events:created:1')
+        assert (apiParts[1], apiParts[-1]) == ('KIBA_API_1', 'GET:/::200:')
+
+    def test_request_id_is_empty_without_a_holder(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_logging(name='app', version='v1', environment='test')
+        logging.info('hello')
+        assert _lines(capsys)[0].split(' - ')[5] == ''
+
+    def test_show_debug_includes_debug(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_logging(name='app', version='v1', environment='test')
+        logging.debug('hidden')
+        logging.init_logging(name='app', version='v1', environment='test', showDebug=True)
+        logging.debug('shown')
+        assert [line.split(' - ')[-1] for line in _lines(capsys)] == ['shown']
+
+
+class TestJsonLogging:
+    def test_writes_one_json_object_per_log_with_service_details(self, capsys: pytest.CaptureFixture[str]) -> None:
+        requestIdHolder = RequestIdHolder()
+        requestIdHolder.set_value(value='req-123')
+        logging.init_json_logging(name='app', version='v1', environment='test', requestIdHolder=requestIdHolder)
+        logging.info('hello %s', 'kiba')
+        record = _json_lines(capsys)[0]
+        loggedDate = datetime.datetime.strptime(str(record.pop('date')), '%Y-%m-%dT%H:%M:%S.%f').replace(tzinfo=datetime.UTC)
+        assert abs((datetime.datetime.now(tz=datetime.UTC) - loggedDate).total_seconds()) < 60  # noqa: PLR2004
+        # NOTE(krishan711): the json formatter logs the unformatted msg, not msg % args
+        assert record == {'message': 'hello %s', 'level': 'INFO', 'logger': 'root', 'format': 'KIBA_1', 'name': 'app', 'version': 'v1', 'environment': 'test', 'requestId': 'req-123'}
+
+    def test_request_id_is_null_without_a_holder(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_json_logging(name='app', version='v1', environment='test')
+        logging.info('hello')
+        assert _json_lines(capsys)[0]['requestId'] is None
+
+    def test_exceptions_log_the_traceback_as_the_message(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_json_logging(name='app', version='v1', environment='test')
+        try:
+            raise ValueError('bad value')
+        except ValueError as exception:
+            logging.exception(exception)
+        record = _json_lines(capsys)[0]
+        assert (record['level'], str(record['message']).splitlines()[0], str(record['message']).splitlines()[-1]) == ('ERROR', 'Traceback (most recent call last):', 'ValueError: bad value')
+
+    def test_stat_fields_are_typed(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_json_logging(name='app', version='v1', environment='test')
+        logging.stat(name='events', key='created', value=2.5)
+        record = _json_lines(capsys)[0]
+        assert {key: record[key] for key in ('format', 'message', 'statName', 'statKey', 'statValue')} == {'format': 'KIBA_STAT_1', 'message': None, 'statName': 'events', 'statKey': 'created', 'statValue': 2.5}
+
+    def test_api_fields_are_typed_and_missing_values_are_null(self, capsys: pytest.CaptureFixture[str]) -> None:
+        logging.init_json_logging(name='app', version='v1', environment='test')
+        logging.api(action='GET', path='/v1/users/123', pathPattern='/v1/users/{userId}', query='limit=10', response=200, duration=0.25)
+        logging.api(action='CRON', path='job', query='')
+        fields = ('format', 'apiAction', 'apiPath', 'apiPathPattern', 'apiQuery', 'apiResponse', 'apiDuration')
+        completeRecord, startRecord = ({key: record[key] for key in fields} for record in _json_lines(capsys))
+        assert completeRecord == {'format': 'KIBA_API_1', 'apiAction': 'GET', 'apiPath': '/v1/users/123', 'apiPathPattern': '/v1/users/{userId}', 'apiQuery': 'limit=10', 'apiResponse': 200, 'apiDuration': 0.25}
+        assert startRecord == {'format': 'KIBA_API_1', 'apiAction': 'CRON', 'apiPath': 'job', 'apiPathPattern': 'job', 'apiQuery': None, 'apiResponse': None, 'apiDuration': None}
+
+
+class TestJsonFieldParsers:
+    def test_empty_values_parse_to_none(self) -> None:
+        assert (logging.json_parse_string_value(''), logging.json_parse_int_value(''), logging.json_parse_float_value('')) == (None, None, None)
+
+    def test_values_parse_to_their_types(self) -> None:
+        assert (logging.json_parse_string_value('abc'), logging.json_parse_int_value('42'), logging.json_parse_float_value('1.5')) == ('abc', 42, 1.5)