mt5cli 0.5.2__tar.gz → 0.6.0__tar.gz

{mt5cli-0.5.2 → mt5cli-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mt5cli
-Version: 0.5.2
+Version: 0.6.0
 Summary: Command-line tool for MetaTrader 5
 Project-URL: Repository, https://github.com/dceoy/mt5cli.git
 Author-email: dceoy <dceoy@users.noreply.github.com>
@@ -160,7 +160,24 @@ update_history_with_config(
 - **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
 - **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
 - **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
-- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping.
+- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
+- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
+
+```python
+from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
+
+rates = collect_latest_closed_rates_by_granularity(
+    [AccountSpec(symbols=["EURUSD", "GBPUSD"], login=12345)],
+    ["M1", "H1"],
+    count=500,
+    retry_count=3,
+)
+eurusd_m1 = rates["EURUSD", "M1"]  # closed bars only
+```
+
+- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
+- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`/`sqlite3.Error` and let the caller decide logging.
+- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
 - **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `Mt5CliClient` that shuts down on exit.
 - **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
 - **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.

{mt5cli-0.5.2 → mt5cli-0.6.0}/README.md

@@ -136,7 +136,24 @@ update_history_with_config(
 - **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
 - **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
 - **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
-- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping.
+- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
+- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
+
+```python
+from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
+
+rates = collect_latest_closed_rates_by_granularity(
+    [AccountSpec(symbols=["EURUSD", "GBPUSD"], login=12345)],
+    ["M1", "H1"],
+    count=500,
+    retry_count=3,
+)
+eurusd_m1 = rates["EURUSD", "M1"]  # closed bars only
+```
+
+- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
+- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`/`sqlite3.Error` and let the caller decide logging.
+- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
 - **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `Mt5CliClient` that shuts down on exit.
 - **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
 - **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.

{mt5cli-0.5.2 → mt5cli-0.6.0}/docs/api/history.md

@@ -215,3 +215,15 @@ frame = series["EURUSD", 1]  # keyed by (symbol, integer timeframe)
   requires existing managed `rate_*` compatibility views and raises
   `ValueError` when they are missing. Duplicate `(symbol, timeframe)` targets
   are rejected.
+- `load_rate_series_by_granularity()` is a thin wrapper that builds the targets,
+  loads the series, and rekeys the result by granularity name to avoid
+  converting integer timeframes downstream:
+
+  ```python
+  from mt5cli import load_rate_series_by_granularity
+
+  series = load_rate_series_by_granularity(
+      "history.db", ["EURUSD"], ["M1", "H1"], count=1000
+  )
+  frame = series["EURUSD", "M1"]  # keyed by (symbol | None, granularity_name)
+  ```

mt5cli-0.6.0/docs/api/sdk.md

@@ -0,0 +1,103 @@
+# SDK Module
+
+::: mt5cli.sdk
+
+## Resilient multi-account orchestration
+
+The SDK ships strategy-agnostic helpers for building long-running collectors on
+top of the read-only client. None of them depend on a particular trading
+application.
+
+### Retrying transient rate collection
+
+`collect_latest_rates_for_accounts_with_retries()` wraps
+`collect_latest_rates_for_accounts()` with bounded exponential backoff. Only
+`pdmt5.Mt5TradingError` and `pdmt5.Mt5RuntimeError` are retried; the final
+failure is re-raised once `retry_count` is exhausted.
+
+```python
+from mt5cli import AccountSpec, collect_latest_rates_for_accounts_with_retries
+
+accounts = [AccountSpec(symbols=["EURUSD"], login=12345)]
+rates = collect_latest_rates_for_accounts_with_retries(
+    accounts,
+    ["M1", "H1"],
+    count=500,
+    retry_count=3,
+    backoff_base=2,  # sleeps 2s, 4s, 8s between attempts
+)
+```
+
+### Latest closed rate bars
+
+MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
+row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
+drops that row with `drop_forming_rate_bar()`, and validates each series is
+non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
+prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
+
+```python
+from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
+
+rates = collect_latest_closed_rates_by_granularity(
+    [AccountSpec(symbols=["EURUSD"], login=12345)],
+    ["M1", "H1"],
+    count=500,
+    retry_count=3,
+)
+closed_m1 = rates["EURUSD", "M1"]
+```
+
+### Resolving credentials and `${ENV_VAR}` placeholders
+
+`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
+values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders, keeping
+secrets out of plan/config files. A missing environment variable raises
+`ValueError`.
+
+```python
+import os
+
+from mt5cli import AccountSpec, resolve_account_specs
+
+os.environ["MT5_LOGIN"] = "12345"
+os.environ["MT5_PASSWORD"] = "secret"
+accounts = [
+    AccountSpec(symbols=["EURUSD"], login="${MT5_LOGIN}", password="${MT5_PASSWORD}")
+]
+
+resolved = resolve_account_specs(accounts, server="Broker-Demo")
+# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
+```
+
+### Throttled incremental history updates
+
+`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
+between successful runs (using a monotonic clock), so an application loop can
+call it every iteration without over-fetching.
+
+```python
+from pdmt5 import Mt5Config, Mt5DataClient
+
+from mt5cli import Dataset, ThrottledHistoryUpdater
+
+updater = ThrottledHistoryUpdater(
+    output="history.db",
+    datasets={Dataset.rates},
+    timeframes=["M1"],
+    interval_seconds=60,  # <= 0 updates on every call
+)
+
+client = Mt5DataClient(config=Mt5Config(login=12345))
+client.initialize_and_login_mt5()
+try:
+    while True:
+        updater.update(client, ["EURUSD", "GBPUSD"])  # no-op until 60s elapse
+        # ... do other work; break when shutting down ...
+finally:
+    client.shutdown()
+```
+
+By default `Mt5TradingError`, `Mt5RuntimeError`, and `sqlite3.Error` propagate so
+the caller controls logging; pass `suppress_errors=True` to swallow them and
+return `False` without advancing the throttle.

{mt5cli-0.5.2 → mt5cli-0.6.0}/mt5cli/__init__.py

@@ -6,8 +6,10 @@ from .history import (
     RateTarget,
     build_rate_targets,
     build_rate_view_name,
+    drop_forming_rate_bar,
     load_rate_data,
     load_rate_data_from_connection,
+    load_rate_series_by_granularity,
     load_rate_series_from_sqlite,
     resolve_history_datasets,
     resolve_history_tick_flags,
@@ -19,11 +21,15 @@ from .history import (
 from .sdk import (
     AccountSpec,
     Mt5CliClient,
+    ThrottledHistoryUpdater,
     account_info,
     build_config,
     collect_history,
+    collect_latest_closed_rates_by_granularity,
+    collect_latest_closed_rates_for_accounts,
     collect_latest_rates,
     collect_latest_rates_for_accounts,
+    collect_latest_rates_for_accounts_with_retries,
     copy_rates_from,
     copy_rates_from_pos,
     copy_rates_range,
@@ -42,6 +48,9 @@ from .sdk import (
     positions,
     recent_history_deals,
     recent_ticks,
+    resolve_account_spec,
+    resolve_account_specs,
+    substitute_env_placeholders,
     symbol_info,
     symbol_info_tick,
     symbols,
@@ -75,19 +84,24 @@ __all__ = [
     "IfExists",
     "Mt5CliClient",
     "RateTarget",
+    "ThrottledHistoryUpdater",
     "account_info",
     "build_config",
     "build_rate_targets",
     "build_rate_view_name",
     "collect_history",
+    "collect_latest_closed_rates_by_granularity",
+    "collect_latest_closed_rates_for_accounts",
     "collect_latest_rates",
     "collect_latest_rates_for_accounts",
+    "collect_latest_rates_for_accounts_with_retries",
     "copy_rates_from",
     "copy_rates_from_pos",
     "copy_rates_range",
     "copy_ticks_from",
     "copy_ticks_range",
     "detect_format",
+    "drop_forming_rate_bar",
     "export_dataframe",
     "export_dataframe_to_sqlite",
     "history_deals",
@@ -96,6 +110,7 @@ __all__ = [
     "latest_rates",
     "load_rate_data",
     "load_rate_data_from_connection",
+    "load_rate_series_by_granularity",
     "load_rate_series_from_sqlite",
     "market_book",
     "minimum_margins",
@@ -110,12 +125,15 @@ __all__ = [
     "positions",
     "recent_history_deals",
     "recent_ticks",
+    "resolve_account_spec",
+    "resolve_account_specs",
     "resolve_history_datasets",
     "resolve_history_tick_flags",
     "resolve_history_timeframes",
     "resolve_rate_tables",
     "resolve_rate_view_name",
     "resolve_rate_view_names",
+    "substitute_env_placeholders",
     "symbol_info",
     "symbol_info_tick",
     "symbols",

{mt5cli-0.5.2 → mt5cli-0.6.0}/mt5cli/history.py

@@ -106,6 +106,23 @@ def resolve_granularity_name(timeframe: int) -> str:
     return str(timeframe)
 
 
+def drop_forming_rate_bar(df_rate: pd.DataFrame) -> pd.DataFrame:
+    """Return closed bars from chronologically ordered MT5 rate data.
+
+    MetaTrader 5 ``copy_rates_from_pos(start_pos=0)`` includes the still-forming
+    current bar as the last row. Slice it off so downstream logic only sees
+    completed bars. Empty frames and single-row frames return empty results.
+
+    Args:
+        df_rate: Rate data ordered oldest-to-newest with the forming bar last.
+
+    Returns:
+        A new DataFrame with all rows except the last. Index and columns are
+        preserved. The input frame is not modified.
+    """
+    return df_rate.iloc[:-1].copy()
+
+
 def build_rate_view_name(
     *,
     symbol: str,
@@ -706,6 +723,55 @@ def load_rate_series_from_sqlite(
             conn.close()
 
 
+def load_rate_series_by_granularity(
+    conn_or_path: SqliteConnOrPath,
+    symbols: Sequence[str],
+    granularities: Sequence[int | str],
+    count: int,
+    *,
+    explicit_tables: Sequence[str] | None = None,
+    allow_missing_symbol: bool = False,
+) -> dict[tuple[str | None, str], pd.DataFrame]:
+    """Load rate series keyed by symbol and string granularity name.
+
+    Builds targets with :func:`build_rate_targets` and loads them with
+    :func:`load_rate_series_from_sqlite`, then rekeys the result by granularity
+    name (for example ``M1``) instead of the integer timeframe to reduce
+    downstream boilerplate.
+
+    Args:
+        conn_or_path: SQLite database path or open connection.
+        symbols: MT5 symbol names. May be empty when ``allow_missing_symbol``.
+        granularities: MT5 timeframes as integers or names (for example ``M1``).
+        count: Number of most recent rows to load per series.
+        explicit_tables: Optional explicit table or view names matching the
+            built targets in row-major order. Required when symbols are omitted.
+        allow_missing_symbol: When True and ``symbols`` is empty, build targets
+            with ``symbol=None`` for each granularity instead of raising.
+
+    Returns:
+        Mapping keyed by ``(symbol | None, granularity_name)`` to each rate
+        DataFrame. Propagates ``ValueError`` (via :func:`build_rate_targets` and
+        :func:`load_rate_series_from_sqlite`) when inputs are empty or invalid,
+        table resolution fails, or duplicate targets are present.
+    """
+    targets = build_rate_targets(
+        symbols,
+        granularities,
+        allow_missing_symbol=allow_missing_symbol,
+    )
+    series = load_rate_series_from_sqlite(
+        conn_or_path,
+        targets,
+        count,
+        explicit_tables=explicit_tables,
+    )
+    return {
+        (symbol, resolve_granularity_name(timeframe)): frame
+        for (symbol, timeframe), frame in series.items()
+    }
+
+
 def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
     """Return existing SQLite columns for a table."""
     quoted_table = quote_sqlite_identifier(table)