mt5cli 0.4.1__tar.gz → 0.4.3__tar.gz

{mt5cli-0.4.1 → mt5cli-0.4.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mt5cli
-Version: 0.4.1
+Version: 0.4.3
 Summary: Command-line tool for MetaTrader 5
 Project-URL: Repository, https://github.com/dceoy/mt5cli.git
 Author-email: dceoy <dceoy@users.noreply.github.com>
@@ -81,6 +81,7 @@ python -m mt5cli -o account.csv account-info
 | `rates-range`      | Export rates for a date range                                                                                |
 | `ticks-from`       | Export ticks from a start date                                                                               |
 | `ticks-range`      | Export ticks for a date range                                                                                |
+| `ticks-recent`     | Export ticks from a recent trailing window                                                                   |
 | `account-info`     | Export account information                                                                                   |
 | `terminal-info`    | Export terminal information                                                                                  |
 | `version`          | Export MetaTrader 5 version information                                                                      |
@@ -88,6 +89,7 @@ python -m mt5cli -o account.csv account-info
 | `symbols`          | Export symbol list                                                                                           |
 | `symbol-info`      | Export symbol details                                                                                        |
 | `symbol-info-tick` | Export the last tick for a symbol                                                                            |
+| `minimum-margins`  | Export minimum-volume buy and sell margin requirements                                                       |
 | `market-book`      | Export market depth (order book)                                                                             |
 | `orders`           | Export active orders                                                                                         |
 | `positions`        | Export open positions                                                                                        |
@@ -151,6 +153,9 @@ update_history_with_config(
 - **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
 - **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
 - **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
+- **Rate view resolution**: use `mt5cli.history.resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases.
+- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
+- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
 
 ## Requirements
 

{mt5cli-0.4.1 → mt5cli-0.4.3}/README.md

@@ -57,6 +57,7 @@ python -m mt5cli -o account.csv account-info
 | `rates-range`      | Export rates for a date range                                                                                |
 | `ticks-from`       | Export ticks from a start date                                                                               |
 | `ticks-range`      | Export ticks for a date range                                                                                |
+| `ticks-recent`     | Export ticks from a recent trailing window                                                                   |
 | `account-info`     | Export account information                                                                                   |
 | `terminal-info`    | Export terminal information                                                                                  |
 | `version`          | Export MetaTrader 5 version information                                                                      |
@@ -64,6 +65,7 @@ python -m mt5cli -o account.csv account-info
 | `symbols`          | Export symbol list                                                                                           |
 | `symbol-info`      | Export symbol details                                                                                        |
 | `symbol-info-tick` | Export the last tick for a symbol                                                                            |
+| `minimum-margins`  | Export minimum-volume buy and sell margin requirements                                                       |
 | `market-book`      | Export market depth (order book)                                                                             |
 | `orders`           | Export active orders                                                                                         |
 | `positions`        | Export open positions                                                                                        |
@@ -127,6 +129,9 @@ update_history_with_config(
 - **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
 - **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
 - **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
+- **Rate view resolution**: use `mt5cli.history.resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases.
+- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
+- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
 
 ## Requirements
 

mt5cli-0.4.1/docs/api/sqlite_history.md → mt5cli-0.4.3/docs/api/history.md

@@ -1,6 +1,6 @@
-# SQLite History Module
+# History Collection (SQLite)
 
-::: mt5cli.sqlite_history
+::: mt5cli.history
 
 ## `collect-history` schema
 
@@ -129,3 +129,38 @@ when required columns are missing.
 The `update_history` SDK path uses the same base tables and optional
 `cash_events` / `positions_reconstructed` views. It additionally maintains
 `rate_<symbol>__<timeframe>` compatibility views when `create_rate_views=True`.
+
+### Rate view resolution
+
+Downstream tools can resolve mt5cli-managed compatibility view names from an
+existing SQLite history database without creating files or guessing legacy
+naming schemes:
+
+```python
+from pathlib import Path
+
+from mt5cli.history import resolve_rate_view_name, resolve_rate_view_names
+
+# Single symbol and granularity
+view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
+
+# Batch resolution in row-major order
+views = resolve_rate_view_names(
+    Path("history.db"),
+    ["EURUSD", "GBPUSD"],
+    ["M1", "H1"],
+)
+```
+
+Resolution rules:
+
+- Returns `rate_<symbol>__<timeframe>` when a symbol stores one timeframe.
+- Returns `rate_<symbol>__<granularity>_<timeframe>` when multiple timeframes
+  are stored for the same symbol.
+- When multiple naming candidates apply, prefers an existing managed
+  `rate_*__*` view from the candidate list.
+- Falls back to single-timeframe naming when the database path is missing or
+  `rates` metadata is unavailable.
+- Pass `require_existing=True` to raise `ValueError` instead of returning a
+  best-guess name when the database or view is missing.
+- Accepts either a SQLite path or an open `sqlite3.Connection`.

{mt5cli-0.4.1 → mt5cli-0.4.3}/docs/api/index.md

@@ -18,6 +18,10 @@ Utility module providing constants, enums, Click parameter types, and helper fun
 
 Programmatic SDK for read-only MetaTrader 5 data collection. Returns pandas DataFrames and provides `collect_history` for SQLite bulk collection.
 
+### [History Collection (SQLite)](history.md)
+
+SQLite storage helpers for the `collect-history` command schema, incremental updates, deduplication, indexes, and optional views.
+
 ## Architecture Overview
 
 The package follows a simple architecture built on top of pdmt5:
@@ -61,12 +65,18 @@ from datetime import UTC, datetime
 from pathlib import Path
 
 from mt5cli import (
+    Dataset,
+    IfExists,
     Mt5CliClient,
     collect_history,
     copy_rates_range,
     detect_format,
     export_dataframe,
+    export_dataframe_to_sqlite,
+    minimum_margins,
+    recent_ticks,
 )
+from mt5cli.history import resolve_rate_view_name
 
 # Fetch rates programmatically
 rates = copy_rates_range(
@@ -82,6 +92,20 @@ fmt = detect_format(Path("output.parquet"))  # Returns "parquet"
 # Export a DataFrame
 export_dataframe(rates, Path("output.csv"), "csv")
 
+# Append to SQLite with deduplication
+export_dataframe_to_sqlite(
+    rates,
+    Path("history.db"),
+    "rates",
+    if_exists=IfExists.APPEND,
+    deduplicate_on=("symbol", "timeframe", "time"),
+)
+
+# Resolve rate compatibility views and fetch recent ticks
+view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
+ticks = recent_ticks("EURUSD", seconds=300)
+margins = minimum_margins("EURUSD")
+
 # Collect history into SQLite
 collect_history(
     Path("history.db"),

{mt5cli-0.4.1 → mt5cli-0.4.3}/docs/index.md

@@ -22,13 +22,22 @@ pip install mt5cli
 
 ## Programmatic usage / SDK usage
 
-mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` when you need to persist results.
+mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` or `export_dataframe_to_sqlite` when you need to persist results.
 
 ```python
 from datetime import UTC, datetime
 from pathlib import Path
 
-from mt5cli import Mt5CliClient, collect_history, copy_rates_range, export_dataframe
+from mt5cli import (
+    Mt5CliClient,
+    collect_history,
+    copy_rates_range,
+    export_dataframe,
+    export_dataframe_to_sqlite,
+    minimum_margins,
+    recent_ticks,
+)
+from mt5cli.history import resolve_rate_view_name
 
 # One-off fetch with module-level helpers
 rates = copy_rates_range(
@@ -39,6 +48,13 @@ rates = copy_rates_range(
 )
 export_dataframe(rates, Path("rates.csv"), "csv")
 
+# Resolve SQLite rate compatibility views for downstream tools
+view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
+
+# Recent tick window and minimum margin summary
+ticks = recent_ticks("EURUSD", seconds=300)
+margins = minimum_margins("EURUSD")
+
 # Reuse one MT5 connection for multiple calls
 with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
     account = client.account_info()
@@ -92,10 +108,11 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
 
 ### Ticks
 
-| Command       | Description                    |
-| ------------- | ------------------------------ |
-| `ticks-from`  | Export ticks from a start date |
-| `ticks-range` | Export ticks for a date range  |
+| Command        | Description                         |
+| -------------- | ----------------------------------- |
+| `ticks-from`   | Export ticks from a start date      |
+| `ticks-range`  | Export ticks for a date range       |
+| `ticks-recent` | Export ticks from a trailing window |
 
 ### Information
 
@@ -108,6 +125,7 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
 | `symbols`          | Export symbol list                      |
 | `symbol-info`      | Export symbol details                   |
 | `symbol-info-tick` | Export the last tick for a symbol       |
+| `minimum-margins`  | Export minimum-volume margin summary    |
 | `market-book`      | Export market depth (order book)        |
 
 ### Trading
@@ -152,7 +170,7 @@ mt5cli -o history.db collect-history \
 
 History orders and deals are fetched per symbol and concatenated, so the symbol filter is applied consistently across all datasets. The `cash_events` view is derived from symbol-filtered `history_deals`, so account-level cash events with empty or non-matching symbols may be excluded. The `positions_reconstructed` view excludes positions with no closing deal, uses volume-weighted open/close prices, and reports reversal deals (`DEAL_ENTRY_INOUT`) via `volume_reversal` / `reversal_count`.
 
-See the [SQLite History schema diagram](api/sqlite_history.md#entity-relationship-diagram) for a sample ER layout of the resulting database.
+See the [History schema diagram](api/history.md#entity-relationship-diagram) for a sample ER layout of the resulting database.
 
 ## Global Options
 

{mt5cli-0.4.1 → mt5cli-0.4.3}/mkdocs.yml

@@ -58,7 +58,7 @@ nav:
       - Overview: api/index.md
       - CLI: api/cli.md
       - SDK: api/sdk.md
-      - SQLite History: api/sqlite_history.md
+      - History Collection (SQLite): api/history.md
       - Utils: api/utils.md
 
 markdown_extensions:

{mt5cli-0.4.1 → mt5cli-0.4.3}/mt5cli/__init__.py

@@ -16,8 +16,10 @@ from .sdk import (
     history_orders,
     last_error,
     market_book,
+    minimum_margins,
     orders,
     positions,
+    recent_ticks,
     symbol_info,
     symbol_info_tick,
     symbols,
@@ -28,7 +30,13 @@ from .sdk import (
 from .sdk import (
     version as mt5_version,
 )
-from .utils import Dataset, IfExists, detect_format, export_dataframe
+from .utils import (
+    Dataset,
+    IfExists,
+    detect_format,
+    export_dataframe,
+    export_dataframe_to_sqlite,
+)
 
 __version__ = version(__package__) if __package__ else None
 
@@ -46,13 +54,16 @@ __all__ = [
     "copy_ticks_range",
     "detect_format",
     "export_dataframe",
+    "export_dataframe_to_sqlite",
     "history_deals",
     "history_orders",
     "last_error",
     "market_book",
+    "minimum_margins",
     "mt5_version",
     "orders",
     "positions",
+    "recent_ticks",
     "symbol_info",
     "symbol_info_tick",
     "symbols",

{mt5cli-0.4.1 → mt5cli-0.4.3}/mt5cli/cli.py

@@ -300,6 +300,44 @@ def ticks_range(
     )
 
 
+@app.command()
+def ticks_recent(
+    ctx: typer.Context,
+    symbol: Annotated[str, typer.Option(help="Symbol name.")],
+    seconds: Annotated[
+        float,
+        typer.Option(help="Lookback window in seconds."),
+    ],
+    date_to: Annotated[
+        datetime | None,
+        typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
+    ] = None,
+    count: Annotated[
+        int,
+        typer.Option(help="Maximum number of ticks to return."),
+    ] = 10000,
+    flags: Annotated[
+        int,
+        typer.Option(
+            click_type=TICK_FLAGS_TYPE,
+            help="Tick flags (ALL, INFO, TRADE, or integer).",
+        ),
+    ] = 1,
+) -> None:
+    """Export ticks from a recent time window."""
+    client = _sdk_client(ctx)
+    _execute_export(
+        ctx,
+        lambda: client.recent_ticks(
+            symbol,
+            seconds,
+            date_to=date_to,
+            count=count,
+            flags=flags,
+        ),
+    )
+
+
 @app.command()
 def account_info(ctx: typer.Context) -> None:
     """Export account information."""
@@ -335,6 +373,16 @@ def symbol_info(
     _execute_export(ctx, lambda: client.symbol_info(symbol))
 
 
+@app.command()
+def minimum_margins(
+    ctx: typer.Context,
+    symbol: Annotated[str, typer.Option(help="Symbol name.")],
+) -> None:
+    """Export minimum-volume buy and sell margin requirements."""
+    client = _sdk_client(ctx)
+    _execute_export(ctx, lambda: client.minimum_margins(symbol))
+
+
 @app.command()
 def orders(
     ctx: typer.Context,

mt5cli-0.4.1/mt5cli/sqlite_history.py → mt5cli-0.4.3/mt5cli/history.py

@@ -1,10 +1,11 @@
-"""SQLite helpers for incremental MT5 history collection."""
+"""SQLite storage helpers for the ``collect-history`` incremental data pipeline."""
 
 from __future__ import annotations
 
 import logging
 import sqlite3
 from datetime import UTC, datetime
+from pathlib import Path
 from typing import TYPE_CHECKING, Literal
 
 import pandas as pd
@@ -122,6 +123,230 @@ def build_rate_view_name(
     return f"rate_{symbol}__{granularity}_{timeframe}"
 
 
+SqliteConnOrPath = sqlite3.Connection | Path | str
+
+
+def _open_history_connection(
+    conn_or_path: SqliteConnOrPath,
+) -> tuple[sqlite3.Connection | None, bool]:
+    """Open a read-only SQLite connection when given a path.
+
+    Returns:
+        A connection and whether the caller should close it. When the path does
+        not exist, returns ``(None, False)`` without creating a database file.
+    """
+    if isinstance(conn_or_path, sqlite3.Connection):
+        return conn_or_path, False
+    path = Path(conn_or_path)
+    if not path.exists():
+        return None, False
+    conn = sqlite3.connect(f"{path.resolve().as_uri()}?mode=ro", uri=True)
+    return conn, True
+
+
+def _load_rates_timeframe_counts(conn: sqlite3.Connection) -> dict[str, int] | None:
+    """Return distinct timeframe counts per symbol from the normalized rates table."""
+    columns = get_table_columns(conn, Dataset.rates.table_name)
+    if not {"symbol", "timeframe"}.issubset(columns):
+        return None
+    rows = conn.execute(
+        "SELECT symbol, COUNT(DISTINCT timeframe) FROM rates GROUP BY symbol",
+    ).fetchall()
+    return {str(symbol): int(count) for symbol, count in rows}
+
+
+def _load_existing_rate_views(conn: sqlite3.Connection) -> set[str]:
+    """Return mt5cli-managed ``rate_*__*`` compatibility view names."""
+    rows = conn.execute(
+        "SELECT name FROM sqlite_master WHERE type = 'view' AND name GLOB 'rate_*__*'",
+    ).fetchall()
+    return {str(row[0]) for row in rows}
+
+
+def _rate_view_name_candidates(
+    *,
+    symbol: str,
+    granularity: str,
+    granularity_count: int,
+    timeframe: int,
+) -> list[str]:
+    """Return candidate view names in preference order."""
+    single = build_rate_view_name(
+        symbol=symbol,
+        granularity=granularity,
+        granularity_count=1,
+        timeframe=timeframe,
+    )
+    if granularity_count <= 1:
+        return [single]
+    multi = build_rate_view_name(
+        symbol=symbol,
+        granularity=granularity,
+        granularity_count=granularity_count,
+        timeframe=timeframe,
+    )
+    return [multi, single]
+
+
+def _resolve_rate_view_name_from_context(
+    *,
+    symbol: str,
+    timeframe: int,
+    granularity_name: str,
+    timeframe_counts: dict[str, int] | None,
+    existing_views: set[str],
+    require_existing: bool = False,
+) -> str:
+    """Resolve one rate view name using preloaded SQLite metadata.
+
+    Returns:
+        Preferred mt5cli-managed rate compatibility view name.
+
+    Raises:
+        ValueError: If ``require_existing`` is True and no managed view exists.
+    """
+    if timeframe_counts is None or symbol not in timeframe_counts:
+        candidates = [
+            build_rate_view_name(
+                symbol=symbol,
+                granularity=granularity_name,
+                granularity_count=1,
+                timeframe=timeframe,
+            ),
+            build_rate_view_name(
+                symbol=symbol,
+                granularity=granularity_name,
+                granularity_count=2,
+                timeframe=timeframe,
+            ),
+        ]
+    else:
+        candidates = _rate_view_name_candidates(
+            symbol=symbol,
+            granularity=granularity_name,
+            granularity_count=timeframe_counts[symbol],
+            timeframe=timeframe,
+        )
+    for candidate in candidates:
+        if candidate in existing_views:
+            return candidate
+    if require_existing:
+        msg = (
+            f"No rate compatibility view exists for symbol {symbol!r} "
+            f"and granularity {granularity_name!r}; "
+            f"candidates: {', '.join(candidates)}."
+        )
+        raise ValueError(msg)
+    return candidates[0]
+
+
+def resolve_rate_view_name(
+    conn_or_path: SqliteConnOrPath,
+    symbol: str,
+    granularity: str,
+    *,
+    require_existing: bool = False,
+) -> str:
+    """Resolve the mt5cli-managed rate compatibility view name.
+
+    Args:
+        conn_or_path: SQLite database path or open connection.
+        symbol: Symbol stored in the normalized ``rates`` table.
+        granularity: Timeframe name (for example ``M1``) or integer string.
+        require_existing: When True, require the database and a managed view to exist.
+
+    Returns:
+        View name such as ``rate_EURUSD__1`` or ``rate_EURUSD__M1_1``.
+
+    Raises:
+        ValueError: If ``require_existing`` is True and the database or view is missing.
+    """
+    timeframe = parse_timeframe(granularity)
+    granularity_name = resolve_granularity_name(timeframe)
+    conn, should_close = _open_history_connection(conn_or_path)
+    try:
+        if conn is None:
+            if require_existing:
+                path = (
+                    conn_or_path
+                    if isinstance(conn_or_path, (Path, str))
+                    else "database"
+                )
+                msg = f"SQLite database not found: {path}"
+                raise ValueError(msg)
+            return build_rate_view_name(
+                symbol=symbol,
+                granularity=granularity_name,
+                granularity_count=1,
+                timeframe=timeframe,
+            )
+        return _resolve_rate_view_name_from_context(
+            symbol=symbol,
+            timeframe=timeframe,
+            granularity_name=granularity_name,
+            timeframe_counts=_load_rates_timeframe_counts(conn),
+            existing_views=_load_existing_rate_views(conn),
+            require_existing=require_existing,
+        )
+    finally:
+        if should_close and conn is not None:
+            conn.close()
+
+
+def resolve_rate_view_names(
+    conn_or_path: SqliteConnOrPath,
+    symbols: Sequence[str],
+    granularities: Sequence[str],
+    *,
+    require_existing: bool = False,
+) -> list[str]:
+    """Resolve rate compatibility view names for symbol and granularity pairs.
+
+    Args:
+        conn_or_path: SQLite database path or open connection.
+        symbols: Symbols stored in the normalized ``rates`` table.
+        granularities: Timeframe names (for example ``M1``) or integer strings.
+        require_existing: When True, require the database and managed views to exist.
+
+    Returns:
+        View names in row-major order: every ``granularity`` for the first
+        symbol, then every granularity for the next symbol, and so on.
+    """
+    conn, should_close = _open_history_connection(conn_or_path)
+    try:
+        if conn is None:
+            return [
+                resolve_rate_view_name(
+                    conn_or_path,
+                    symbol,
+                    granularity,
+                    require_existing=require_existing,
+                )
+                for symbol in symbols
+                for granularity in granularities
+            ]
+        timeframe_counts = _load_rates_timeframe_counts(conn)
+        existing_views = _load_existing_rate_views(conn)
+        resolved: list[str] = []
+        for symbol in symbols:
+            for granularity in granularities:
+                timeframe = parse_timeframe(granularity)
+                resolved.append(
+                    _resolve_rate_view_name_from_context(
+                        symbol=symbol,
+                        timeframe=timeframe,
+                        granularity_name=resolve_granularity_name(timeframe),
+                        timeframe_counts=timeframe_counts,
+                        existing_views=existing_views,
+                        require_existing=require_existing,
+                    ),
+                )
+        return resolved
+    finally:
+        if should_close and conn is not None:
+            conn.close()
+
+
 def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
     """Return existing SQLite columns for a table."""
     rows = conn.execute(f"PRAGMA table_info({table})").fetchall()