ib-connect 0.2.0__tar.gz

ib_connect-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mats Bengtsson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ib_connect-0.2.0/PKG-INFO

@@ -0,0 +1,151 @@
+Metadata-Version: 2.4
+Name: ib_connect
+Version: 0.2.0
+Summary: Modular Interactive Brokers integration toolkit — query contracts, download historical data, and upload to TimescaleDB.
+Author-email: Mats Bengtsson <mats.bengtsson@outlook.com>
+License: MIT
+Project-URL: Homepage, https://github.com/alacazar/ib-connect
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: ib_insync
+Requires-Dist: pandas
+Requires-Dist: psycopg2-binary
+Requires-Dist: watchdog
+Requires-Dist: flask
+Dynamic: license-file
+
+# IB Connect
+
+A modular Python toolkit for Interactive Brokers integration — query contracts, download historical market data, and upload to a TimescaleDB database.
+
+## Architecture
+
+```
+ib_connect/
+├── shared/           # IB connection management (shared by all skills)
+├── skills/
+│   ├── ib_query/     # CLI: qualify and inspect contracts
+│   ├── ib_download/  # CLI + service: async historical data downloader
+│   └── data_upload/  # Service: watch folder → PostgreSQL/TimescaleDB
+└── gui/              # Flask web UI
+```
+
+## Prerequisites
+
+- Python 3.9+
+- [IB Gateway](https://www.interactivebrokers.com/en/trading/ibgateway-stable.php) or TWS running with API connections enabled
+  - Paper trading port: `7497` (default)
+  - Live trading port: `7496`
+- For `data_upload` and `gui`: PostgreSQL with [TimescaleDB](https://www.timescale.com/)
+
+## Installation
+
+```bash
+git clone https://github.com/alacazar/ib-connect.git
+cd ib_connect
+pip install -e .
+```
+
+The `ib-query` and `ib-download` CLI commands are registered as entry points after installation.
+
+## Configuration
+
+The `ib_download` and `gui` skills require a `config.json`. Copy the provided examples and customize:
+
+```bash
+cp ib_connect/skills/ib_download/config.example.json ib_connect/skills/ib_download/config.json
+cp ib_connect/gui/config.example.json ib_connect/gui/config.json
+```
+
+For `data_upload`, set the `PG_URI` environment variable:
+
+```bash
+export PG_URI="postgresql://user:password@localhost:5432/dbname"
+```
+
+## Skills
+
+### ib-query
+
+Query and qualify contract details from IB.
+
+```bash
+ib-query --symbol AAPL --sec-type STK
+```
+
+Output:
+```json
+{"conId": 265598, "symbol": "AAPL", "secType": "STK", "exchange": "SMART", "currency": "USD", "fullName": "APPLE INC"}
+```
+
+See [ib_connect/skills/ib_query/SKILL.md](ib_connect/skills/ib_query/SKILL.md) for all options.
+
+---
+
+### ib-download
+
+Downloads historical OHLCV data from IB and writes CSV files. Consists of two parts:
+
+**`download_service.py` — the background service**
+
+IB enforces strict pacing limits: it throttles connections that issue too many data requests too quickly, and it only allows one active connection per client ID. Running a dedicated background service solves both problems — it owns a single, persistent IB connection and processes jobs one at a time, sleeping 20 seconds between data chunks to stay within IB's limits. Any number of callers can submit jobs without worrying about pacing or connection conflicts.
+
+Start it once and keep it running (e.g. in `screen` or `tmux`, or as a system service):
+
+```bash
+python ib_connect/skills/ib_download/download_service.py
+```
+
+Long date ranges are split automatically into chunks sized to IB's per-request limits. Progress and errors are logged to `download_service.log` alongside the script. A PID lock file prevents duplicate instances.
+
+**`ib-download` — the CLI client**
+
+Submits a job to the service's SQLite queue and returns a `job_key` immediately. The caller can check status later without staying connected.
+
+```bash
+# Submit a job
+ib-download -c 265598 -s 2023-01-01 -e 2023-12-31 -b "1 min"
+
+# Check status
+ib-download --status -k <job_key>
+```
+
+On completion the service can notify an agent via webhook (configure `webhook_url` and `token` in `config.json`).
+
+See [ib_connect/skills/ib_download/SKILL.md](ib_connect/skills/ib_download/SKILL.md) for all options including batch mode and output format.
+
+---
+
+### data-upload
+
+A file-watching service that validates and uploads contract JSON files and OHLCV CSV files to PostgreSQL/TimescaleDB. Drop files into the configured `input_folder` and they are processed automatically — contracts go to `finance.contracts`, OHLCV data to `finance.ohlcv_1m` or `finance.ohlcv_1d`. Processed files are moved to `processed/`; failures go to `errors/`.
+
+```bash
+python ib_connect/skills/data_upload/data_upload.py
+```
+
+Expected filename pattern for OHLCV: `<symbol>.<conid>.<barsize>.csv`.
+
+See [ib_connect/skills/data_upload/SKILL.md](ib_connect/skills/data_upload/SKILL.md) for schema details.
+
+---
+
+### GUI
+
+A Flask web interface that ties the toolkit together. From the browser you can:
+
+- **Query contracts** — search by symbol and security type, inspect the qualified contract details returned by IB
+- **Save contracts** — export selected contracts as JSON to the `data_upload` input folder, so they are picked up and stored in PostgreSQL automatically
+- **Submit download jobs** — configure date range and bar size per contract and enqueue jobs with the download service
+- **Monitor jobs** — check status and cancel pending jobs
+
+```bash
+python ib_connect/gui/app.py
+```
+
+Opens on `http://127.0.0.1:5000` by default (configurable in `gui/config.json`).
+
+## License
+
+[MIT](LICENSE)

ib_connect-0.2.0/README.md

@@ -0,0 +1,134 @@
+# IB Connect
+
+A modular Python toolkit for Interactive Brokers integration — query contracts, download historical market data, and upload to a TimescaleDB database.
+
+## Architecture
+
+```
+ib_connect/
+├── shared/           # IB connection management (shared by all skills)
+├── skills/
+│   ├── ib_query/     # CLI: qualify and inspect contracts
+│   ├── ib_download/  # CLI + service: async historical data downloader
+│   └── data_upload/  # Service: watch folder → PostgreSQL/TimescaleDB
+└── gui/              # Flask web UI
+```
+
+## Prerequisites
+
+- Python 3.9+
+- [IB Gateway](https://www.interactivebrokers.com/en/trading/ibgateway-stable.php) or TWS running with API connections enabled
+  - Paper trading port: `7497` (default)
+  - Live trading port: `7496`
+- For `data_upload` and `gui`: PostgreSQL with [TimescaleDB](https://www.timescale.com/)
+
+## Installation
+
+```bash
+git clone https://github.com/alacazar/ib-connect.git
+cd ib_connect
+pip install -e .
+```
+
+The `ib-query` and `ib-download` CLI commands are registered as entry points after installation.
+
+## Configuration
+
+The `ib_download` and `gui` skills require a `config.json`. Copy the provided examples and customize:
+
+```bash
+cp ib_connect/skills/ib_download/config.example.json ib_connect/skills/ib_download/config.json
+cp ib_connect/gui/config.example.json ib_connect/gui/config.json
+```
+
+For `data_upload`, set the `PG_URI` environment variable:
+
+```bash
+export PG_URI="postgresql://user:password@localhost:5432/dbname"
+```
+
+## Skills
+
+### ib-query
+
+Query and qualify contract details from IB.
+
+```bash
+ib-query --symbol AAPL --sec-type STK
+```
+
+Output:
+```json
+{"conId": 265598, "symbol": "AAPL", "secType": "STK", "exchange": "SMART", "currency": "USD", "fullName": "APPLE INC"}
+```
+
+See [ib_connect/skills/ib_query/SKILL.md](ib_connect/skills/ib_query/SKILL.md) for all options.
+
+---
+
+### ib-download
+
+Downloads historical OHLCV data from IB and writes CSV files. Consists of two parts:
+
+**`download_service.py` — the background service**
+
+IB enforces strict pacing limits: it throttles connections that issue too many data requests too quickly, and it only allows one active connection per client ID. Running a dedicated background service solves both problems — it owns a single, persistent IB connection and processes jobs one at a time, sleeping 20 seconds between data chunks to stay within IB's limits. Any number of callers can submit jobs without worrying about pacing or connection conflicts.
+
+Start it once and keep it running (e.g. in `screen` or `tmux`, or as a system service):
+
+```bash
+python ib_connect/skills/ib_download/download_service.py
+```
+
+Long date ranges are split automatically into chunks sized to IB's per-request limits. Progress and errors are logged to `download_service.log` alongside the script. A PID lock file prevents duplicate instances.
+
+**`ib-download` — the CLI client**
+
+Submits a job to the service's SQLite queue and returns a `job_key` immediately. The caller can check status later without staying connected.
+
+```bash
+# Submit a job
+ib-download -c 265598 -s 2023-01-01 -e 2023-12-31 -b "1 min"
+
+# Check status
+ib-download --status -k <job_key>
+```
+
+On completion the service can notify an agent via webhook (configure `webhook_url` and `token` in `config.json`).
+
+See [ib_connect/skills/ib_download/SKILL.md](ib_connect/skills/ib_download/SKILL.md) for all options including batch mode and output format.
+
+---
+
+### data-upload
+
+A file-watching service that validates and uploads contract JSON files and OHLCV CSV files to PostgreSQL/TimescaleDB. Drop files into the configured `input_folder` and they are processed automatically — contracts go to `finance.contracts`, OHLCV data to `finance.ohlcv_1m` or `finance.ohlcv_1d`. Processed files are moved to `processed/`; failures go to `errors/`.
+
+```bash
+python ib_connect/skills/data_upload/data_upload.py
+```
+
+Expected filename pattern for OHLCV: `<symbol>.<conid>.<barsize>.csv`.
+
+See [ib_connect/skills/data_upload/SKILL.md](ib_connect/skills/data_upload/SKILL.md) for schema details.
+
+---
+
+### GUI
+
+A Flask web interface that ties the toolkit together. From the browser you can:
+
+- **Query contracts** — search by symbol and security type, inspect the qualified contract details returned by IB
+- **Save contracts** — export selected contracts as JSON to the `data_upload` input folder, so they are picked up and stored in PostgreSQL automatically
+- **Submit download jobs** — configure date range and bar size per contract and enqueue jobs with the download service
+- **Monitor jobs** — check status and cancel pending jobs
+
+```bash
+python ib_connect/gui/app.py
+```
+
+Opens on `http://127.0.0.1:5000` by default (configurable in `gui/config.json`).
+
+## License
+
+[MIT](LICENSE)

ib_connect-0.2.0/ib_connect/__init__.py

@@ -0,0 +1 @@
+# IB Connect Package

ib_connect-0.2.0/ib_connect/gui/__init__.py

@@ -0,0 +1 @@
+# GUI Package

ib_connect-0.2.0/ib_connect/gui/app.py

@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+"""
+IB Data Downloader GUI - Flask Web App
+"""
+
+from flask import Flask, render_template, request, jsonify
+import subprocess
+import json
+import os
+import sys
+import asyncio
+import logging
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime
+
+
+from ib_connect.skills.ib_query.query import query_ib
+from ib_connect.skills.ib_download.job_queue import JobQueue
+
+app = Flask(__name__)
+logging.getLogger('werkzeug').setLevel(logging.ERROR)
+
+# Config
+CONFIG_FILE = 'config.json'
+DEFAULT_CONFIG = {
+    'contracts_folder': './data/contracts',
+    'ib_host': '127.0.0.1',
+    'ib_port': 7497,
+    'ib_client_id': 77
+}
+
+def load_config():
+    if os.path.exists(CONFIG_FILE):
+        with open(CONFIG_FILE, 'r') as f:
+            return json.load(f)
+    return DEFAULT_CONFIG
+
+def save_config(config):
+    with open(CONFIG_FILE, 'w') as f:
+        json.dump(config, f, indent=2)
+
+config = load_config()
+
+@app.route('/')
+def index():
+    return render_template('index.html')
+
+@app.route('/query', methods=['POST'])
+def query_contracts():
+    data = request.json
+    # Include configured IB params to avoid conflicts
+    data['host'] = config.get('ib_host', '127.0.0.1')
+    data['port'] = config.get('ib_port', 7497)
+    data['client_id'] = config.get('ib_client_id', 1)
+    def run_query():
+        return asyncio.run(query_ib(data))
+
+    try:
+        with ThreadPoolExecutor() as executor:
+            future = executor.submit(run_query)
+            result = future.result()
+        if isinstance(result, dict) and 'error' in result:
+            return jsonify({'success': False, 'error': result['error']})
+        contracts = result if isinstance(result, list) else [result] if result else []
+        contracts = [c for c in contracts if c is not None]  # Filter out nulls
+        return jsonify({'success': True, 'contracts': contracts})
+    except Exception as e:
+        return jsonify({'success': False, 'error': str(e)})
+
+@app.route('/save_contracts', methods=['POST'])
+def save_contracts():
+    data = request.json
+    selected = data.get('selected', [])
+    # Add defaults for missing required fields
+    for contract in selected:
+        if 'time_zone_id' not in contract or not contract['time_zone_id']:
+            contract['time_zone_id'] = 'US/Eastern'
+        if 'min_tick' not in contract or not contract['min_tick']:
+            contract['min_tick'] = 0.01
+        if 'tick_value' not in contract or not contract['tick_value']:
+            contract['tick_value'] = contract.get('min_tick', 0.01) * contract.get('multiplier', 1)
+        if 'multiplier' not in contract or not contract['multiplier']:
+            contract['multiplier'] = 1
+    # Omit empty strings and zero values, keep None for db defaults
+    cleaned_selected = []
+    for contract in selected:
+        cleaned = {k: v for k, v in contract.items() if v not in ('', 0)}
+        cleaned_selected.append(cleaned)
+    folder = config['contracts_folder']
+    os.makedirs(folder, exist_ok=True)
+    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
+    file_path = os.path.join(folder, f'selected_contracts_{timestamp}.json')
+    with open(file_path, 'w') as f:
+        json.dump(cleaned_selected, f, indent=2)
+    return jsonify({'success': True, 'file': file_path})
+
+@app.route('/download', methods=['POST'])
+def download_data():
+    data = request.json
+    contracts = data.get('contracts', [])
+    downloads_folder = config.get('downloads_folder', './data/downloads')
+    job_queue_db = config.get('job_queue_db', './jobs.db')
+    queue = JobQueue(job_queue_db)
+    jobs = []
+    for contract in contracts:
+        params = {
+            'conid': contract['conid'],
+            'start': contract['start_date'],
+            'end': contract['end_date'],
+            'bar_size': contract['bar_size'],
+            'show': contract.get('show', 'TRADES'),
+            'msg': f"Download {contract['symbol']} data ({contract.get('show', 'TRADES')})"
+        }
+        try:
+            job_key = queue.submit_job(params)
+            jobs.append({'contract': contract['symbol'], 'job_key': job_key, 'status': 'submitted'})
+        except Exception as e:
+            jobs.append({'contract': contract['symbol'], 'error': str(e)})
+    return jsonify({'jobs': jobs})
+
+@app.route('/job_status/<job_key>')
+def job_status(job_key):
+    try:
+        job_queue_db = config['job_queue_db']
+        queue = JobQueue(job_queue_db)
+        status_data = queue.get_status(job_key)
+        if status_data['status'] == 'not_found':
+            return jsonify({'status': 'error', 'details': 'Job not found'})
+        details = f"Status: {status_data.get('status', 'unknown')}"
+        if status_data.get('message'):
+            details += f" - {status_data['message']}"
+        if status_data.get('error'):
+            details += f" - Error: {status_data['error']}"
+        return jsonify({'status': 'ok', 'details': details})
+    except Exception as e:
+        return jsonify({'status': 'error', 'details': str(e)})
+
+@app.route('/cancel_job/<job_key>', methods=['POST'])
+def cancel_job(job_key):
+    try:
+        job_queue_db = config['job_queue_db']
+        queue = JobQueue(job_queue_db)
+        status_data = queue.get_status(job_key)
+        if status_data['status'] == 'pending':
+            queue.remove_job(job_key)
+            return jsonify({'success': True})
+        else:
+            return jsonify({'success': False, 'error': 'Job not pending'})
+    except Exception as e:
+        return jsonify({'success': False, 'error': str(e)})
+
+if __name__ == '__main__':
+    app.run(debug=True, host=config.get('host', '127.0.0.1'), port=config.get('port', 5000))

ib_connect-0.2.0/ib_connect/shared/__init__.py

@@ -0,0 +1 @@
+# Shared utilities for IB Connect

ib_connect-0.2.0/ib_connect/shared/ib_connection.py

@@ -0,0 +1,34 @@
+from ib_insync import IB
+
+class IBConnection:
+    """
+    Connection management for Interactive Brokers using ib_insync.
+    Provides a connected IB instance that can be used as a context manager.
+    """
+
+    @staticmethod
+    def connect(host='127.0.0.1', port=7497, clientId=1, timeout=4.0, readonly=False, account=''):
+        """
+        Establish a connection to IB and return the connected IB instance (which is a context manager).
+        
+        Args:
+            host (str): IB Gateway/TWS host. Default: '127.0.0.1'
+            port (int): IB Gateway/TWS port. Default: 7497
+            clientId (int): Client ID for the connection. Default: 1
+            timeout (float): Connection timeout in seconds. Default: 4.0
+            readonly (bool): Read-only connection. Default: False
+            account (str): Account to use (optional). Default: ''
+        
+        Returns:
+            IB: Connected IB instance (supports context manager: with ib: ...)
+        
+        Raises:
+            ConnectionError: If connection fails.
+        """
+        ib = IB()
+        try:
+            ib.connect(host=host, port=port, clientId=clientId, timeout=timeout, readonly=readonly, account=account)
+        except Exception as e:
+            raise ConnectionError(f"Failed to connect to Interactive Brokers: {e}") from e
+        
+        return ib

ib_connect-0.2.0/ib_connect/skills/__init__.py

@@ -0,0 +1 @@
+# Skills for IB Connect

ib_connect-0.2.0/ib_connect/skills/data_upload/__init__.py

@@ -0,0 +1 @@
+# Data Upload skill