exness-data-preprocess 0.7.1__tar.gz → 0.8.0__tar.gz

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/.cz.toml

@@ -1,6 +1,6 @@
 [tool.commitizen]
 name = "cz_conventional_commits"
-version = "0.7.1"
+version = "0.7.2"
 tag_format = "v$major.$minor.$patch"
 update_changelog_on_bump = true
 version_files = [

exness_data_preprocess-0.8.0/.lint-relative-paths-results.txt

@@ -0,0 +1,62 @@
+🔍 Scanning for non-repo-relative paths in: /Users/terryli/eon/exness-data-preprocess
+   Violations: paths not starting with / or http(s)
+
+❌ Found 141 violation(s):
+
+/Users/terryli/eon/exness-data-preprocess/README.md:112:- **Column Definitions**: See [`schema.py`](src/exness_data_preprocess/schema.py) - Single source of truth
+/Users/terryli/eon/exness-data-preprocess/README.md:113:- **Comprehensive Reference**: See [`DATABASE_SCHEMA.md`](docs/DATABASE_SCHEMA.md) - Query examples and usage patterns
+/Users/terryli/eon/exness-data-preprocess/README.md:401:- Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md)
+/Users/terryli/eon/exness-data-preprocess/README.md:409:- SSoT: [`docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
+/Users/terryli/eon/exness-data-preprocess/README.md:410:- Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)
+/Users/terryli/eon/exness-data-preprocess/README.md:417:- SSoT: [`docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
+/Users/terryli/eon/exness-data-preprocess/README.md:419:**Release Notes**: See [`CHANGELOG.md`](CHANGELOG.md) for complete v0.5.0 details
+/Users/terryli/eon/exness-data-preprocess/README.md:485:**[📚 Complete Documentation Hub](docs/README.md)** - Organized guide from beginner to advanced (72+ documents)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:7:**Full Documentation**: [`README.md`](README.md) - Installation, usage, API reference
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:15:- **[README.md](README.md)** - Installation, API reference, usage examples
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:16:- **[examples/basic_usage.py](examples/basic_usage.py)** - Download, query, coverage operations
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:17:- **[examples/batch_processing.py](examples/batch_processing.py)** - Multi-instrument parallel processing
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:21:- **[docs/README.md](docs/README.md)** - Architecture, planning, research findings
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:22:- **[docs/MODULE_ARCHITECTURE.md](docs/MODULE_ARCHITECTURE.md)** - Complete module documentation with SLOs (v1.7.0)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:23:- **[docs/DATABASE_SCHEMA.md](docs/DATABASE_SCHEMA.md)** - Complete database schema with self-documentation
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:24:- **[docs/RESEARCH_PATTERNS.md](docs/RESEARCH_PATTERNS.md)** - Research lifecycle and tool selection (v1.0.0)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:25:- **[docs/UNIFIED_DUCKDB_PLAN_v2.md](docs/UNIFIED_DUCKDB_PLAN_v2.md)** - v2.0.0 architecture specification
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:26:- **[docs/EXNESS_DATA_SOURCES.md](docs/EXNESS_DATA_SOURCES.md)** - Data source variants and URLs
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:27:- **[Makefile](Makefile)** - Module introspection commands (module-stats, module-complexity, module-deps)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:31:- **[docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)** - SSoT plan v2.0.0 (2.2x speedup)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:32:- **[docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)** - SSoT plan v2.0.0 (complete coverage)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:33:- **[docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md](docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md)** - Phase 1 validation
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:34:- **[docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)** - Phase 2 validation
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:40:- **[CONTRIBUTING.md](CONTRIBUTING.md)** - Contribution guidelines
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:55:**Complete Specification**: [`docs/UNIFIED_DUCKDB_PLAN_v2.md`](docs/UNIFIED_DUCKDB_PLAN_v2.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:76:**Complete Details**: [`docs/MODULE_ARCHITECTURE.md`](docs/MODULE_ARCHITECTURE.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:83:  - SSoT: [`docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:89:  - SSoT: [`docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:90:  - Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:96:  - SSoT: [`docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:109:**Complete Schema**: [`docs/DATABASE_SCHEMA.md`](docs/DATABASE_SCHEMA.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:118:**Complete Guide**: [`docs/EXNESS_DATA_SOURCES.md`](docs/EXNESS_DATA_SOURCES.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:128:**Complete Specification**: [`docs/RESEARCH_PATTERNS.md`](docs/RESEARCH_PATTERNS.md)
+/Users/terryli/eon/exness-data-preprocess/CLAUDE.md:196:**Complete Development Guide**: [`README.md`](README.md) (Installation section)
+/Users/terryli/eon/exness-data-preprocess/tests/README.md:30:- **New**: Phase7 13-column (v1.2.0) OHLC - See [`../src/exness_data_preprocess/schema.py`](../src/exness_data_preprocess/schema.py)
+/Users/terryli/eon/exness-data-preprocess/docs/EXNESS_DATA_SOURCES.md:310:**Full Specification**: [`docs/research/eurusd-zero-spread-deviations/data/plan/phase7_bid_ohlc_construction_v1.1.0.md`](research/eurusd-zero-spread-deviations/data/plan/phase7_bid_ohlc_construction_v1.1.0.md)
+/Users/terryli/eon/exness-data-preprocess/docs/EXNESS_DATA_SOURCES.md:386:- **Phase7 Methodology**: [`docs/research/eurusd-zero-spread-deviations/01-methodology.md`](research/eurusd-zero-spread-deviations/01-methodology.md)
+/Users/terryli/eon/exness-data-preprocess/docs/EXNESS_DATA_SOURCES.md:387:- **Zero-Spread Research**: [`docs/research/eurusd-zero-spread-deviations/README.md`](research/eurusd-zero-spread-deviations/README.md)
+/Users/terryli/eon/exness-data-preprocess/docs/EXNESS_DATA_SOURCES.md:391:- **Current Processor**: [`src/exness_data_preprocess/processor.py`](../src/exness_data_preprocess/processor.py) (needs refactoring)
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:5:**Related**: [`README.md`](../README.md) - Architecture overview
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:43:- See [`docs/validation/ARCHITECTURE_AUDIT_2025-10-17.md`](validation/ARCHITECTURE_AUDIT_2025-10-17.md) for complete audit findings
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:964:See [`Makefile`](../Makefile) for implementation.
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1076:- **[README.md](../README.md)** - User-facing API reference
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1077:- **[docs/README.md](README.md)** - Documentation hub
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1078:- **[docs/DATABASE_SCHEMA.md](DATABASE_SCHEMA.md)** - Complete database schema
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1079:- **[docs/UNIFIED_DUCKDB_PLAN_v2.md](UNIFIED_DUCKDB_PLAN_v2.md)** - v2.0.0 architecture specification
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1080:- **[Makefile](../Makefile)** - Module introspection commands
+/Users/terryli/eon/exness-data-preprocess/docs/MODULE_ARCHITECTURE.md:1082:- **[docs/validation/ARCHITECTURE_AUDIT_2025-10-17.md](validation/ARCHITECTURE_AUDIT_2025-10-17.md)** - Complete audit findings and corrections
+/Users/terryli/eon/exness-data-preprocess/docs/DATABASE_SCHEMA.md:244:**Column Definitions**: See [`../src/exness_data_preprocess/schema.py`](../src/exness_data_preprocess/schema.py) for complete column definitions with types and descriptions.
+/Users/terryli/eon/exness-data-preprocess/docs/DATABASE_SCHEMA.md:258:**Architecture**: Exchange Registry Pattern (v1.6.0) - Session columns dynamically generated from centralized EXCHANGES dict in [`exchanges.py`](../src/exness_data_preprocess/exchanges.py)
+
+... and 91 more (showing first 50)
+
+📝 Fix: Convert relative paths to repo-relative paths starting with /
+   WRONG: [Link](../docs/foo.md)
+   WRONG: [Link](docs/foo.md)
+   RIGHT: [Link](/docs/foo.md)

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/CHANGELOG.md

@@ -1,3 +1,10 @@
+# [0.8.0](https://github.com/terrylica/exness-data-preprocess/compare/v0.7.2...v0.8.0) (2025-12-10)
+
+
+### Features
+
+* add ClickHouse backend for cloud deployment ([c9b51a3](https://github.com/terrylica/exness-data-preprocess/commit/c9b51a3b374eea74375cc86afdd8cb9391fe73e8))
+
 # Changelog
 
 All notable changes to exness-data-preprocess will be documented in this file.
@@ -5,6 +12,37 @@ All notable changes to exness-data-preprocess will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+### 💅 Code Style
+
+- Format markdown files with prettier Standardized formatting for CHANGELOG.md.
+
+### 📚 Documentation
+
+- Move test plan to docs/plans/ - Move tests/archive/E2E_TESTING_PLAN_v1.0.0.md -> docs/plans/ - Remove empty tests/archive/ directory - Consistent with docs/ organization strategy Impact: Planning docs consolidated in docs/plans/ Risk: Low (internal doc, no external references)
+
+- Archive release notes to docs/releases/ - Move RELEASE_NOTES.md, RELEASE_NOTES_SHORT.md -> docs/releases/ - No references in README.md, CONTRIBUTING.md, or CLAUDE.md - GitHub Releases is SSoT for release information Impact: -2 root files, cleaner project root Risk: Low (generated files, no external references)
+
+- Remove redundant DOCUMENTATION.md - Remove DOCUMENTATION.md (content duplicates docs/README.md) - Update README.md link to point to docs/README.md - Both files serve as navigation hubs with identical purpose Impact: -1 root file, single documentation entry point Risk: Low (historical CHANGELOG references preserved, README updated)
+
+- Organize phase plans into docs/phases/ - Move PHASE2*.yaml, PHASE3*.yaml, PHASE4\*.yaml -> docs/phases/ - Update 4 references in CLAUDE.md - Consolidate SSoT planning files in dedicated subdirectory Impact: -3 docs/ root files, improved organization Risk: Low (all references updated, git mv preserves history)
+
+- Move GitHub/PyPI setup to docs/setup/ - Move GITHUB_PYPI_SETUP.md -> docs/setup/ - No active references in README.md or CONTRIBUTING.md - Historical CHANGELOG references preserved Impact: -1 root file, infrastructure docs consolidated Risk: Low (no external links, CHANGELOG preserved)
+
+- Update repository refactoring SSoT to v2.0.0 - Mark plan as Completed - Record 3 implementation discoveries - Document actual execution metrics (6 commits, ~25 minutes) - Track impact: 50% clutter reduction (30 -> 15 items) Findings: - README.md had active reference to DOCUMENTATION.md (updated) - Phase YAML files referenced in 7 locations (all updated) - GITHUB_PYPI_SETUP.md only in historical CHANGELOG (safe move) Zero regressions: All references updated, git history preserved
+
+- Archive obsolete implementation-plan.yaml (2024-11-24) - Move implementation-plan.yaml -> archive/implementation-plan-2024-11-24.yaml - Remove references from docs/README.md - File showed 24 tests (obsolete, current: 48 tests) - Status: completed (no longer active planning doc) Impact: Cleaner docs/ root (-1 file) Risk: Low (archived with timestamp for historical reference)
+
+- Remove outdated planning-index.yaml - Remove planning-index.yaml (last updated 2024-11-24) - Remove entire 'Implementation Plans' section from docs/README.md - Index referenced archived implementation-plan.yaml - Research plans have own README files (no central index needed) Impact: -1 docs/ root file, -1 obsolete section in docs/README.md Risk: None (outdated index with no active purpose)
+
+- Update refactoring SSoT plans to completion status - REPOSITORY_REFACTORING_PLAN.yaml: v2.0.0 -> v3.0.0 - PRIORITY3_CLEANUP_PLAN.yaml: v1.0.0 -> v2.0.0 (Completed) Combined Priority 1-3 Impact: - Root MD files: 10 -> 4 (-60%) - docs/ root files: 12 -> 8 (-33%) - Total clutter: 30 -> 12 items (-60%) Total: 11 commits, ~40 minutes, zero regressions
+
+### 🧰 Maintenance
+
+- Untrack lychee cache files - Remove .lychee-results.json, .lychee-results.txt, .lycheecache from tracking - Files remain gitignored via .gitignore rules - Add REPOSITORY_REFACTORING_PLAN.yaml SSoT (v1.0.0) Impact: Cleaner git status, prevent cache pollution Risk: None (cache files, no external references)
+
+- Remove redundant AGENTS.md developer guidelines - Content duplicates CONTRIBUTING.md - Misleading name (not AI agent configuration) - No active references found Impact: -1 root file, cleaner project root Risk: None (zero references, content preserved in CONTRIBUTING.md)
+
+- Remove redundant AUTHORS.md - Git log is SSoT for contributor history - pyproject.toml defines author (Eon Labs) and maintainer (Terry Li) - No active references found Impact: -1 root file Risk: None (contributor info preserved in pyproject.toml and git log)
 
 ### 💅 Code Style
 

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/CLAUDE.md

@@ -28,8 +28,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ### Optimization Plans (v1.7.0)
 
-- **[docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml](docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)** - SSoT plan v2.0.0 (2.2x speedup)
-- **[docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml](docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)** - SSoT plan v2.0.0 (complete coverage)
+- **[docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)** - SSoT plan v2.0.0 (2.2x speedup)
+- **[docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)** - SSoT plan v2.0.0 (complete coverage)
 - **[docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md](docs/validation/SPIKE_TEST_RESULTS_PHASE1_2025-10-18.md)** - Phase 1 validation
 - **[docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)** - Phase 2 validation
 
@@ -56,7 +56,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ### Module Pattern (v1.7.0)
 
-- **Facade orchestrator** (processor.py) coordinating 7 specialized modules
+- **Facade orchestrator** (processor.py) coordinating 6 module instances + 1 static utility
 - **Separation of concerns** with single-responsibility modules
 - **SLO-based design** (Availability, Correctness, Observability, Maintainability)
 - **Off-the-shelf libraries** (httpx, pandas, DuckDB, exchange_calendars)
@@ -86,14 +86,14 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
   - Pre-compute trading minutes, vectorized .isin() lookup
   - Measured: 5.99s → 2.69s for 302K bars across 10 exchanges
   - Combined Phase 1+2: **~16x total speedup** (8.05s → 0.50s)
-  - SSoT: [`docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
+  - SSoT: [`docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
   - Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)
 
 - **SQL Gap Detection** (Phase 3): Complete coverage + 46% LOC reduction
   - SQL EXCEPT query detects ALL gaps (internal + edges)
   - Bug fix: Now detects internal gaps missed by Python MIN/MAX
   - Measured: 41 → 42 gaps detected (spike test validation)
-  - SSoT: [`docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
+  - SSoT: [`docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
 
 **Pattern**: Spike test first (validate theory) → Implement → Iterative SSoT plan updates
 

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: exness-data-preprocess
-Version: 0.7.1
+Version: 0.8.0
 Summary: Professional Exness forex tick data preprocessing with optimal compression (Parquet Zstd-22) and DuckDB OHLC generation. Provides efficient storage (9% smaller than ZIP) with lossless precision and direct queryability.
 Project-URL: Homepage, https://github.com/terrylica/exness-data-preprocess
 Project-URL: Documentation, https://github.com/terrylica/exness-data-preprocess#readme
@@ -11,7 +11,6 @@ Project-URL: Changelog, https://github.com/terrylica/exness-data-preprocess/blob
 Author-email: Eon Labs <terry@eonlabs.com>
 Maintainer-email: Terry Li <terry@eonlabs.com>
 License: MIT
-License-File: AUTHORS.md
 License-File: LICENSE
 Keywords: backtesting,compression,data-engineering,duckdb,efficient-storage,eurusd,exness,financial-data,forex,forex-data,lossless,microstructure,ohlc,pandas,parquet,preprocessing,pyarrow,quantitative-finance,queryable,tick-data,time-series,zstd
 Classifier: Development Status :: 3 - Alpha
@@ -20,15 +19,14 @@ Classifier: Intended Audience :: Financial and Insurance Industry
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Office/Business :: Financial :: Investment
 Classifier: Topic :: Scientific/Engineering :: Information Analysis
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Python: >=3.9
+Requires-Python: >=3.11
+Requires-Dist: clickhouse-connect>=0.7.0
 Requires-Dist: duckdb>=0.9.0
 Requires-Dist: exchange-calendars>=4.5.6
 Requires-Dist: httpx>=0.25.0
@@ -456,7 +454,7 @@ Data is sourced from Exness's public tick data repository:
 - Vectorized approach: 2.69s (302K bars, 10 exchanges)
 - Combined Phase 1+2: ~16x total speedup (8.05s → 0.50s)
 - Implementation: Pre-compute trading minutes, vectorized `.isin()` lookup
-- SSoT: [`docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
+- SSoT: [`docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
 - Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)
 
 **SQL Gap Detection** - Complete coverage with 46% code reduction:
@@ -464,7 +462,7 @@ Data is sourced from Exness's public tick data repository:
 - Bug fix: Python approach missed internal gaps (41 detected vs 42 actual)
 - SQL EXCEPT operator detects ALL gaps (before + within + after existing data)
 - Code reduced from 62 lines to 34 lines (46% reduction)
-- SSoT: [`docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
+- SSoT: [`docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
 
 **Release Notes**: See [`CHANGELOG.md`](CHANGELOG.md) for complete v0.5.0 details
 
@@ -532,7 +530,7 @@ Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
 
 ## Additional Documentation
 
-**[📚 Complete Documentation Hub](DOCUMENTATION.md)** - Organized guide from beginner to advanced (72+ documents)
+**[📚 Complete Documentation Hub](docs/README.md)** - Organized guide from beginner to advanced (72+ documents)
 
 - **Basic Usage Examples**: See `examples/basic_usage.py`
 - **Batch Processing**: See `examples/batch_processing.py`

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/README.md

@@ -406,7 +406,7 @@ Data is sourced from Exness's public tick data repository:
 - Vectorized approach: 2.69s (302K bars, 10 exchanges)
 - Combined Phase 1+2: ~16x total speedup (8.05s → 0.50s)
 - Implementation: Pre-compute trading minutes, vectorized `.isin()` lookup
-- SSoT: [`docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
+- SSoT: [`docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml`](docs/phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml)
 - Validation: [`docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md`](docs/validation/SPIKE_TEST_RESULTS_PHASE2_2025-10-18.md)
 
 **SQL Gap Detection** - Complete coverage with 46% code reduction:
@@ -414,7 +414,7 @@ Data is sourced from Exness's public tick data repository:
 - Bug fix: Python approach missed internal gaps (41 detected vs 42 actual)
 - SQL EXCEPT operator detects ALL gaps (before + within + after existing data)
 - Code reduced from 62 lines to 34 lines (46% reduction)
-- SSoT: [`docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
+- SSoT: [`docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml`](docs/phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml)
 
 **Release Notes**: See [`CHANGELOG.md`](CHANGELOG.md) for complete v0.5.0 details
 
@@ -482,7 +482,7 @@ Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
 
 ## Additional Documentation
 
-**[📚 Complete Documentation Hub](DOCUMENTATION.md)** - Organized guide from beginner to advanced (72+ documents)
+**[📚 Complete Documentation Hub](docs/README.md)** - Organized guide from beginner to advanced (72+ documents)
 
 - **Basic Usage Examples**: See `examples/basic_usage.py`
 - **Batch Processing**: See `examples/batch_processing.py`

{exness_data_preprocess-0.7.1 → exness_data_preprocess-0.8.0}/docs/MODULE_ARCHITECTURE.md

@@ -26,7 +26,7 @@
   - Bug fix: Now detects internal gaps missed by Python MIN/MAX
 - All optimizations: Backward compatible (no API changes)
 - Validation: All 48 tests pass, spike tests validate measured results
-- SSoT Plans: PHASE2_SESSION_VECTORIZATION_PLAN.yaml, PHASE3_SQL_GAP_DETECTION_PLAN.yaml
+- SSoT Plans: phases/PHASE2_SESSION_VECTORIZATION_PLAN.yaml, phases/PHASE3_SQL_GAP_DETECTION_PLAN.yaml
 
 ### v1.3.1 (2025-10-17) - Documentation Accuracy Update
 

exness_data_preprocess-0.8.0/docs/PRIORITY3_CLEANUP_PLAN.yaml

@@ -0,0 +1,386 @@
+openapi: 3.1.0
+info:
+  title: Priority 3 Repository Cleanup Plan
+  version: 2.0.0
+  description: |
+    Single Source of Truth for Priority 3 optional cleanup (deeper file review).
+
+    Intent: Remove obsolete and redundant files after thorough audit.
+    Constraint: Must verify no active references before removal.
+
+    Scope:
+      - Remove developer guideline duplicates (AGENTS.md)
+      - Remove redundant contributor tracking (AUTHORS.md)
+      - Archive or remove obsolete planning files (implementation-plan.yaml, planning-index.yaml)
+      - Preserve cliff configs (serve different audiences)
+
+    Status: Completed
+
+  contact:
+    name: exness-data-preprocess
+    url: https://github.com/terrylica/exness-data-preprocess
+
+paths:
+  /audit:
+    summary: File audit findings
+
+components:
+  schemas:
+    SLOs:
+      type: object
+      required:
+        - availability
+        - correctness
+        - observability
+        - maintainability
+      properties:
+        availability:
+          type: string
+          description: |
+            No active functionality depends on removed files.
+            Documentation references updated before removal.
+            Git history preserved for removed files.
+          example: "100% - Only remove files with zero active dependencies"
+
+        correctness:
+          type: string
+          description: |
+            Verify no references via grep before removal.
+            Update docs/README.md links atomically.
+            Preserve git history with git rm (not system rm).
+            Raise on unexpected references.
+          example: "Atomic reference updates, git rm for history"
+
+        observability:
+          type: string
+          description: |
+            Document rationale for each removal decision.
+            Track removed files in SSoT plan.
+            Conventional commit messages.
+          example: "Documented rationale, conventional commits"
+
+        maintainability:
+          type: string
+          description: |
+            Use git rm (native operation, not custom scripts).
+            SSoT plan tracks all decisions.
+            No promotional language in documentation.
+          example: "Git native operations, factual documentation"
+
+    AuditFindings:
+      type: object
+      description: File-by-file audit results
+      properties:
+        agents_md:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "AGENTS.md"
+            size:
+              type: string
+              example: "25 lines"
+            content:
+              type: string
+              example: "Developer guidelines (build, test, coding style, commits, data handling)"
+            redundancy:
+              type: string
+              example: "Duplicates CONTRIBUTING.md content"
+            references:
+              type: integer
+              example: 0
+              description: "No active references found via grep"
+            decision:
+              type: string
+              enum: [remove, keep, archive]
+              example: "remove"
+            rationale:
+              type: string
+              example: "Content duplicates CONTRIBUTING.md, no references, misleading name (not AI agent config)"
+
+        authors_md:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "AUTHORS.md"
+            size:
+              type: string
+              example: "25 lines"
+            content:
+              type: string
+              example: "Terry Li (author), Eon Labs (organization), acknowledgments"
+            redundancy:
+              type: string
+              example: "Duplicates pyproject.toml author/maintainer fields and git log"
+            git_contributors:
+              type: array
+              items:
+                type: string
+              example: ["terrylica <terry@eonlabs.com>"]
+            pyproject_info:
+              type: object
+              properties:
+                authors:
+                  type: array
+                  items:
+                    type: string
+                  example: ["Eon Labs <terry@eonlabs.com>"]
+                maintainers:
+                  type: array
+                  items:
+                    type: string
+                  example: ["Terry Li <terry@eonlabs.com>"]
+            references:
+              type: integer
+              example: 0
+            decision:
+              type: string
+              enum: [remove, keep, archive]
+              example: "remove"
+            rationale:
+              type: string
+              example: "Git log and pyproject.toml are SSoT for contributor info, no references"
+
+        cliff_release_notes_toml:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "cliff-release-notes.toml"
+            size:
+              type: string
+              example: "132 lines"
+            purpose:
+              type: string
+              example: "GitHub release notes generation (user-friendly, skips docs/chore/test)"
+            comparison:
+              type: object
+              properties:
+                cliff_toml:
+                  type: string
+                  example: "145 lines - CHANGELOG.md generation (detailed, includes all commits)"
+                difference:
+                  type: string
+                  example: "Different audiences: releases (users) vs changelog (developers)"
+            decision:
+              type: string
+              enum: [remove, keep, merge]
+              example: "keep"
+            rationale:
+              type: string
+              example: "Serves different audience (GitHub release users vs CHANGELOG.md readers)"
+
+        implementation_plan_yaml:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "docs/implementation-plan.yaml"
+            size:
+              type: string
+              example: "37 lines"
+            content:
+              type: string
+              example: "Pytest execution plan from 2024-11-24"
+            status:
+              type: string
+              example: "completed (shows 24 tests passed, current: 48 tests)"
+            references:
+              type: array
+              items:
+                type: string
+              example:
+                - "docs/README.md (1 reference)"
+                - "docs/planning-index.yaml (1 reference)"
+                - "docs/plans/DOCUMENTATION_AUDIT_2025-10-15.md (2 references)"
+            decision:
+              type: string
+              enum: [remove, keep, archive]
+              example: "archive"
+            rationale:
+              type: string
+              example: "Obsolete (6 months old, 24 vs 48 tests). Archive to docs/archive/ and update references."
+
+        planning_index_yaml:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "docs/planning-index.yaml"
+            size:
+              type: string
+              example: "33 lines"
+            content:
+              type: string
+              example: "Index of planning documents (last updated 2024-11-24)"
+            status:
+              type: string
+              example: "Points to research plans and implementation-plan.yaml"
+            references:
+              type: array
+              items:
+                type: string
+              example:
+                - "docs/README.md (1 reference)"
+                - "docs/plans/DOCUMENTATION_AUDIT_2025-10-15.md (2 references)"
+            decision:
+              type: string
+              enum: [remove, keep, update]
+              example: "remove"
+            rationale:
+              type: string
+              example: "Outdated index (6 months), references obsolete implementation-plan.yaml. Research plans have own README files."
+
+    RemovalActions:
+      type: object
+      description: Concrete removal steps
+      properties:
+        action_remove_agents_md:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "AGENTS.md"
+            command:
+              type: string
+              example: "git rm AGENTS.md"
+            references_to_update:
+              type: array
+              items:
+                type: string
+              example: []
+            commit_message:
+              type: string
+              example: "chore: remove redundant AGENTS.md developer guidelines"
+            risk:
+              type: string
+              example: "None - no references, content duplicates CONTRIBUTING.md"
+
+        action_remove_authors_md:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "AUTHORS.md"
+            command:
+              type: string
+              example: "git rm AUTHORS.md"
+            references_to_update:
+              type: array
+              items:
+                type: string
+              example: []
+            commit_message:
+              type: string
+              example: "chore: remove redundant AUTHORS.md"
+            risk:
+              type: string
+              example: "None - pyproject.toml and git log are SSoT"
+
+        action_archive_implementation_plan:
+          type: object
+          properties:
+            source:
+              type: string
+              example: "docs/implementation-plan.yaml"
+            destination:
+              type: string
+              example: "docs/archive/implementation-plan-2024-11-24.yaml"
+            commands:
+              type: array
+              items:
+                type: string
+              example:
+                - "git mv docs/implementation-plan.yaml docs/archive/implementation-plan-2024-11-24.yaml"
+            references_to_update:
+              type: array
+              items:
+                type: string
+              example:
+                - "docs/README.md (remove reference or update to archive location)"
+                - "docs/planning-index.yaml (remove if deleting this file)"
+            commit_message:
+              type: string
+              example: "docs: archive obsolete implementation-plan.yaml (2024-11-24)"
+            risk:
+              type: string
+              example: "Low - referenced in docs/README.md, must update"
+
+        action_remove_planning_index:
+          type: object
+          properties:
+            file:
+              type: string
+              example: "docs/planning-index.yaml"
+            command:
+              type: string
+              example: "git rm docs/planning-index.yaml"
+            references_to_update:
+              type: array
+              items:
+                type: string
+              example:
+                - "docs/README.md (remove reference)"
+            commit_message:
+              type: string
+              example: "docs: remove outdated planning-index.yaml"
+            risk:
+              type: string
+              example: "Medium - referenced in docs/README.md, must update"
+
+    DiscoveredNuances:
+      type: object
+      description: Implementation learnings
+      properties:
+        version:
+          type: string
+          example: "1.0.0"
+        discoveries:
+          type: array
+          items:
+            type: object
+            properties:
+              timestamp:
+                type: string
+                format: date-time
+              finding:
+                type: string
+              impact:
+                type: string
+          example: []
+
+  x-implementation-findings:
+    version: "2.0.0"
+    status: "Completed"
+    notes: |
+      Implementation: 2025-10-28
+      Estimated time: ~15 minutes
+      Actual time: ~15 minutes
+
+      Files audited: 5
+        - AGENTS.md -> removed (369bea2)
+        - AUTHORS.md -> removed (3f23eaa)
+        - cliff-release-notes.toml -> kept (serves different audience than cliff.toml)
+        - docs/implementation-plan.yaml -> archived (4b3219b)
+        - docs/planning-index.yaml -> removed (bddfb98)
+
+      Actions executed: 4 commits
+        - Remove AGENTS.md (25 lines, duplicated CONTRIBUTING.md)
+        - Remove AUTHORS.md (25 lines, duplicated pyproject.toml + git log)
+        - Archive implementation-plan.yaml to archive/implementation-plan-2024-11-24.yaml
+        - Remove planning-index.yaml and entire section from docs/README.md
+
+      References updated:
+        - docs/README.md (removed "Implementation Plans" section with 2 references)
+
+      Impact:
+        - Root directory: 6 -> 4 MD files (-33%)
+        - docs/ root: 10 -> 8 files (-20%)
+        - Combined with Priority 1-2: 60% total clutter reduction
+
+      Validation:
+        - Zero references to removed files
+        - Git history preserved with git rm
+        - Conventional commit messages
+        - All 48 tests still passing