sqa 0.0.24 → 0.0.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (180) hide show
  1. checksums.yaml +4 -4
  2. data/.goose/memory/development.txt +3 -0
  3. data/.semver +6 -0
  4. data/ARCHITECTURE.md +648 -0
  5. data/CHANGELOG.md +82 -0
  6. data/CLAUDE.md +653 -0
  7. data/COMMITS.md +196 -0
  8. data/DATAFRAME_ARCHITECTURE_REVIEW.md +421 -0
  9. data/NEXT-STEPS.md +154 -0
  10. data/README.md +812 -262
  11. data/TASKS.md +358 -0
  12. data/TEST_RESULTS.md +140 -0
  13. data/TODO.md +42 -0
  14. data/_notes.txt +25 -0
  15. data/bin/sqa-console +11 -0
  16. data/data/talk_talk.json +103284 -0
  17. data/develop_summary.md +313 -0
  18. data/docs/advanced/backtesting.md +206 -0
  19. data/docs/advanced/ensemble.md +68 -0
  20. data/docs/advanced/fpop.md +153 -0
  21. data/docs/advanced/index.md +112 -0
  22. data/docs/advanced/multi-timeframe.md +67 -0
  23. data/docs/advanced/pattern-matcher.md +75 -0
  24. data/docs/advanced/portfolio-optimizer.md +79 -0
  25. data/docs/advanced/portfolio.md +166 -0
  26. data/docs/advanced/risk-management.md +210 -0
  27. data/docs/advanced/strategy-generator.md +158 -0
  28. data/docs/advanced/streaming.md +209 -0
  29. data/docs/ai_and_ml.md +80 -0
  30. data/docs/api/dataframe.md +1115 -0
  31. data/docs/api/index.md +126 -0
  32. data/docs/assets/css/custom.css +88 -0
  33. data/docs/assets/js/mathjax.js +18 -0
  34. data/docs/concepts/index.md +68 -0
  35. data/docs/contributing/index.md +60 -0
  36. data/docs/data-sources/index.md +66 -0
  37. data/docs/data_frame.md +317 -97
  38. data/docs/factors_that_impact_price.md +26 -0
  39. data/docs/finviz.md +11 -0
  40. data/docs/fx_pro_bit.md +25 -0
  41. data/docs/genetic_programming.md +104 -0
  42. data/docs/getting-started/index.md +123 -0
  43. data/docs/getting-started/installation.md +229 -0
  44. data/docs/getting-started/quick-start.md +244 -0
  45. data/docs/i_gotta_an_idea.md +22 -0
  46. data/docs/index.md +163 -0
  47. data/docs/indicators/index.md +97 -0
  48. data/docs/indicators.md +110 -24
  49. data/docs/options.md +8 -0
  50. data/docs/strategies/bollinger-bands.md +146 -0
  51. data/docs/strategies/consensus.md +64 -0
  52. data/docs/strategies/custom.md +310 -0
  53. data/docs/strategies/ema.md +53 -0
  54. data/docs/strategies/index.md +92 -0
  55. data/docs/strategies/kbs.md +164 -0
  56. data/docs/strategies/macd.md +96 -0
  57. data/docs/strategies/market-profile.md +54 -0
  58. data/docs/strategies/mean-reversion.md +58 -0
  59. data/docs/strategies/rsi.md +95 -0
  60. data/docs/strategies/sma.md +55 -0
  61. data/docs/strategies/stochastic.md +63 -0
  62. data/docs/strategies/volume-breakout.md +54 -0
  63. data/docs/tags.md +7 -0
  64. data/docs/true_strength_index.md +46 -0
  65. data/docs/weighted_moving_average.md +48 -0
  66. data/examples/README.md +354 -0
  67. data/examples/advanced_features_example.rb +350 -0
  68. data/examples/fpop_analysis_example.rb +191 -0
  69. data/examples/genetic_programming_example.rb +148 -0
  70. data/examples/kbs_strategy_example.rb +208 -0
  71. data/examples/pattern_context_example.rb +300 -0
  72. data/examples/rails_app/Gemfile +34 -0
  73. data/examples/rails_app/README.md +416 -0
  74. data/examples/rails_app/app/assets/javascripts/application.js +107 -0
  75. data/examples/rails_app/app/assets/stylesheets/application.css +659 -0
  76. data/examples/rails_app/app/controllers/analysis_controller.rb +11 -0
  77. data/examples/rails_app/app/controllers/api/v1/stocks_controller.rb +227 -0
  78. data/examples/rails_app/app/controllers/application_controller.rb +22 -0
  79. data/examples/rails_app/app/controllers/backtest_controller.rb +11 -0
  80. data/examples/rails_app/app/controllers/dashboard_controller.rb +21 -0
  81. data/examples/rails_app/app/controllers/portfolio_controller.rb +7 -0
  82. data/examples/rails_app/app/views/analysis/show.html.erb +209 -0
  83. data/examples/rails_app/app/views/backtest/show.html.erb +171 -0
  84. data/examples/rails_app/app/views/dashboard/index.html.erb +118 -0
  85. data/examples/rails_app/app/views/dashboard/show.html.erb +408 -0
  86. data/examples/rails_app/app/views/errors/show.html.erb +17 -0
  87. data/examples/rails_app/app/views/layouts/application.html.erb +60 -0
  88. data/examples/rails_app/app/views/portfolio/index.html.erb +33 -0
  89. data/examples/rails_app/bin/rails +6 -0
  90. data/examples/rails_app/config/application.rb +45 -0
  91. data/examples/rails_app/config/boot.rb +5 -0
  92. data/examples/rails_app/config/database.yml +18 -0
  93. data/examples/rails_app/config/environment.rb +11 -0
  94. data/examples/rails_app/config/routes.rb +26 -0
  95. data/examples/rails_app/config.ru +8 -0
  96. data/examples/realtime_stream_example.rb +274 -0
  97. data/examples/sinatra_app/Gemfile +22 -0
  98. data/examples/sinatra_app/QUICKSTART.md +159 -0
  99. data/examples/sinatra_app/README.md +461 -0
  100. data/examples/sinatra_app/app.rb +344 -0
  101. data/examples/sinatra_app/config.ru +5 -0
  102. data/examples/sinatra_app/public/css/style.css +659 -0
  103. data/examples/sinatra_app/public/js/app.js +107 -0
  104. data/examples/sinatra_app/views/analyze.erb +306 -0
  105. data/examples/sinatra_app/views/backtest.erb +325 -0
  106. data/examples/sinatra_app/views/dashboard.erb +419 -0
  107. data/examples/sinatra_app/views/error.erb +58 -0
  108. data/examples/sinatra_app/views/index.erb +118 -0
  109. data/examples/sinatra_app/views/layout.erb +61 -0
  110. data/examples/sinatra_app/views/portfolio.erb +43 -0
  111. data/examples/strategy_generator_example.rb +346 -0
  112. data/hsa_portfolio.csv +11 -0
  113. data/justfile +0 -0
  114. data/lib/api/alpha_vantage_api.rb +462 -0
  115. data/lib/sqa/backtest.rb +329 -0
  116. data/lib/sqa/data_frame/alpha_vantage.rb +43 -65
  117. data/lib/sqa/data_frame/data.rb +92 -0
  118. data/lib/sqa/data_frame/yahoo_finance.rb +35 -43
  119. data/lib/sqa/data_frame.rb +148 -243
  120. data/lib/sqa/ensemble.rb +359 -0
  121. data/lib/sqa/fpop.rb +199 -0
  122. data/lib/sqa/gp.rb +259 -0
  123. data/lib/sqa/indicator.rb +5 -8
  124. data/lib/sqa/init.rb +15 -8
  125. data/lib/sqa/market_regime.rb +240 -0
  126. data/lib/sqa/multi_timeframe.rb +379 -0
  127. data/lib/sqa/pattern_matcher.rb +497 -0
  128. data/lib/sqa/portfolio.rb +260 -6
  129. data/lib/sqa/portfolio_optimizer.rb +377 -0
  130. data/lib/sqa/risk_manager.rb +442 -0
  131. data/lib/sqa/seasonal_analyzer.rb +209 -0
  132. data/lib/sqa/sector_analyzer.rb +300 -0
  133. data/lib/sqa/stock.rb +67 -125
  134. data/lib/sqa/strategy/bollinger_bands.rb +42 -0
  135. data/lib/sqa/strategy/consensus.rb +5 -2
  136. data/lib/sqa/strategy/kbs_strategy.rb +470 -0
  137. data/lib/sqa/strategy/macd.rb +46 -0
  138. data/lib/sqa/strategy/mp.rb +1 -1
  139. data/lib/sqa/strategy/stochastic.rb +60 -0
  140. data/lib/sqa/strategy/volume_breakout.rb +57 -0
  141. data/lib/sqa/strategy.rb +5 -0
  142. data/lib/sqa/strategy_generator.rb +947 -0
  143. data/lib/sqa/stream.rb +361 -0
  144. data/lib/sqa/version.rb +1 -7
  145. data/lib/sqa.rb +23 -16
  146. data/main.just +81 -0
  147. data/mkdocs.yml +288 -0
  148. data/trace.log +0 -0
  149. metadata +261 -51
  150. data/bin/sqa +0 -6
  151. data/lib/patches/dry-cli.rb +0 -228
  152. data/lib/sqa/activity.rb +0 -10
  153. data/lib/sqa/cli.rb +0 -62
  154. data/lib/sqa/commands/analysis.rb +0 -309
  155. data/lib/sqa/commands/base.rb +0 -139
  156. data/lib/sqa/commands/web.rb +0 -199
  157. data/lib/sqa/commands.rb +0 -22
  158. data/lib/sqa/constants.rb +0 -23
  159. data/lib/sqa/indicator/average_true_range.rb +0 -33
  160. data/lib/sqa/indicator/bollinger_bands.rb +0 -28
  161. data/lib/sqa/indicator/candlestick_pattern_recognizer.rb +0 -60
  162. data/lib/sqa/indicator/donchian_channel.rb +0 -29
  163. data/lib/sqa/indicator/double_top_bottom_pattern.rb +0 -34
  164. data/lib/sqa/indicator/elliott_wave_theory.rb +0 -57
  165. data/lib/sqa/indicator/exponential_moving_average.rb +0 -25
  166. data/lib/sqa/indicator/exponential_moving_average_trend.rb +0 -36
  167. data/lib/sqa/indicator/fibonacci_retracement.rb +0 -23
  168. data/lib/sqa/indicator/head_and_shoulders_pattern.rb +0 -26
  169. data/lib/sqa/indicator/market_profile.rb +0 -32
  170. data/lib/sqa/indicator/mean_reversion.rb +0 -37
  171. data/lib/sqa/indicator/momentum.rb +0 -28
  172. data/lib/sqa/indicator/moving_average_convergence_divergence.rb +0 -29
  173. data/lib/sqa/indicator/peaks_and_valleys.rb +0 -29
  174. data/lib/sqa/indicator/predict_next_value.rb +0 -202
  175. data/lib/sqa/indicator/relative_strength_index.rb +0 -47
  176. data/lib/sqa/indicator/simple_moving_average.rb +0 -24
  177. data/lib/sqa/indicator/simple_moving_average_trend.rb +0 -32
  178. data/lib/sqa/indicator/stochastic_oscillator.rb +0 -68
  179. data/lib/sqa/indicator/true_range.rb +0 -39
  180. data/lib/sqa/trade.rb +0 -26
data/README.md CHANGED
@@ -2,422 +2,972 @@
2
2
 
3
3
  [![Badges?](https://img.shields.io/badge/Badge-We%20don't%20need%20no%20stinkin'%20badges!-red)](https://www.youtube.com/watch?v=VqomZQMZQCQ)
4
4
 
5
- This is a very simplistic set of tools for running technical analysis (quantitative and qualitative) on a stock portfolio. Simplistic means it is not reliable nor intended for any kind of mission-critical financial use. Think of it as a training tool. I do. Its helping me understand why I need professional help from people who know what they are doing.
6
-
7
- The BUY/SELL signals that it generates should not be taken seriously. **DO NOT USE** this library when real money is at stake. If you lose your shirt playing in the stock market don't come crying to me. I think playing in the market is like playing in the street. You are going to get run over.
8
-
9
- <!-- TODO: Consider these gems ...
10
-
11
- This one is most likely out of date:
12
- yahoofinance-ruby: This gem provides a simple Ruby interface to Yahoo Finance's historical quote data. If you're looking to add more data sources to your project, this could be a useful addition. yahoofinance-ruby
13
-
14
- Worth Looking at:
15
- ruby-technical-analysis: This gem provides various technical analysis calculations. It includes over 60 technical analysis functions and indicators like RSI, EMA, SMA, Bollinger Bands, MACD, and more. ruby-technical-analysis
16
-
17
- Maybe later if I want to add an ability to make a live trade from within the SQA context...
18
- ib-ruby: This gem provides a Ruby interface to Interactive Brokers' Trader Workstation (TWS) API, allowing you to build algorithmic trading strategies in Ruby. ib-ruby
19
-
20
- Definitely looking for a plotting package.
21
- plottable: If you're looking to add more visualization capabilities to your project, this gem could be useful. It provides a simple and flexible API for creating plots and charts in Ruby. plottable
22
-
23
- Currently using CSV files; maybe switch to an RDBMS ...
24
- activerecord-import: If you're dealing with large amounts of data, this gem could help improve performance. It adds methods to ActiveRecord for bulk inserting data into the database. activerecord-import
25
-
26
- Could spawn off separate agents for each stock within a portfolio for analysis ...
27
- parallel: This gem can help improve performance by allowing you to run code in parallel. This could be useful if you're running complex calculations on large datasets. parallel
28
-
29
- -->
30
-
5
+ A Ruby library for technical analysis of stock market data. This is a simplistic set of tools for quantitative and qualitative stock analysis designed for **educational purposes only**.
31
6
 
7
+ **⚠️ WARNING:** This is a learning tool, not production software. Do NOT use this library when real money is at stake. The BUY/SELL signals it generates should not be taken seriously. If you lose your shirt playing in the stock market, don't come crying to me. Playing in the market is like playing in the street - you're going to get run over.
32
8
 
33
9
  <!-- Tocer[start]: Auto-generated, don't remove. -->
34
10
 
35
11
  ## Table of Contents
36
12
 
37
- - [This is a Work in Progress](#this-is-a-work-in-progress)
38
- - [Recent Changes](#recent-changes)
13
+ - [Features](#features)
39
14
  - [Installation](#installation)
40
- - [Semantic Versioning](#semantic-versioning)
15
+ - [Configuration](#configuration)
16
+ - [Data Directory](#data-directory)
17
+ - [Alpha Vantage API Key](#alpha-vantage-api-key)
41
18
  - [Usage](#usage)
42
- - [Setup a Config File](#setup-a-config-file)
43
- - [AlphaVantage](#alphavantage)
44
- - [TradingView](#tradingview)
19
+ - [Working with Stocks](#working-with-stocks)
20
+ - [DataFrame Operations](#dataframe-operations)
21
+ - [Technical Indicators](#technical-indicators)
22
+ - [Trading Strategies](#trading-strategies)
23
+ - [Statistics](#statistics)
24
+ - [Interactive Console](#interactive-console)
25
+ - [Architecture](#architecture)
26
+ - [Data Sources](#data-sources)
27
+ - [Alpha Vantage](#alpha-vantage)
45
28
  - [Yahoo Finance](#yahoo-finance)
46
- - [Playing in IRB](#playing-in-irb)
47
- - [With Stocks and Indicators](#with-stocks-and-indicators)
48
- - [Get Historical Prices](#get-historical-prices)
49
- - [Playing in the IRB - Setup](#playing-in-the-irb---setup)
50
- - [Playing in the IRB - Statistics](#playing-in-the-irb---statistics)
51
- - [Playing in the IRB - Indicators](#playing-in-the-irb---indicators)
52
- - [Playing in the IRB - Strategies](#playing-in-the-irb---strategies)
53
- - [Included Program Examples](#included-program-examples)
54
- - [Analysis](#analysis)
55
- - [Web](#web)
56
- - [Predicted FAQ](#predicted-faq)
57
- - [Other Similar Projects](#other-similar-projects)
58
29
  - [Contributing](#contributing)
59
30
  - [License](#license)
60
31
 
61
32
  <!-- Tocer[finish]: Auto-generated, don't remove. -->
62
33
 
63
- ## This is a Work in Progress
34
+ ## Features
35
+
36
+ ### Core Capabilities
37
+ - **High-Performance DataFrames** - Polars-based data structures for time series financial data
38
+ - **150+ Technical Indicators** - Via the `sqa-tai` gem (TA-Lib wrapper)
39
+ - **Trading Strategies** - Framework for building and testing trading strategies
40
+ - **Multiple Data Sources** - Alpha Vantage and Yahoo Finance adapters
41
+ - **Stock Management** - Track stocks with historical prices and company metadata
42
+ - **Statistical Analysis** - Comprehensive statistics on price data
43
+ - **Ticker Validation** - Validate stock symbols against market exchanges
44
+ - **Interactive Console** - IRB console for experimentation (`sqa-console`)
45
+
46
+ ### Advanced Features
47
+ - **Portfolio Management** - Track positions, trades, P&L with commission support
48
+ - **Backtesting Framework** - Simulate trading strategies with comprehensive performance metrics
49
+ - **Strategy Generator** - Reverse-engineer profitable trades to discover indicator patterns
50
+ - **Genetic Programming** - Evolve optimal strategy parameters through natural selection
51
+ - **Knowledge-Based Strategy (KBS)** - RETE-based forward-chaining inference for complex trading rules
52
+ - **Real-Time Streaming** - Event-driven live price processing with on-the-fly signal generation
53
+ - **FPL Analysis** - Future Period Loss/Profit analysis with risk metrics and directional classification
54
+ - **Market Regime Detection** - Identify bull/bear/sideways markets with volatility and strength metrics
55
+ - **Seasonal Analysis** - Discover calendar-dependent patterns (monthly/quarterly performance)
56
+ - **Sector Analysis** - Multi-stock pattern discovery with KBS blackboards per sector
57
+ - **Walk-Forward Validation** - Time-series cross-validation to prevent pattern overfitting
58
+ - **Pattern Context System** - Context-aware patterns that know when/where they're valid
64
59
 
65
- I am experimenting with different gems to support various functionality. Sometimes they do not work out well. For example I've gone through two different gems to implement the data frame capability. Neither did what I wanted so I ended up creating my own data frame class based upon the old tried and true [Hashie](https://github.com/intridea/hashie) library.
60
+ ## Installation
66
61
 
67
- ## Recent Changes
62
+ Install the gem:
68
63
 
69
- * 0.0.24 - Replaced tty-option with dry-cli
64
+ ```bash
65
+ gem install sqa
66
+ ```
70
67
 
71
- ## Installation
68
+ Or add to your `Gemfile`:
72
69
 
73
- Install the gem and add to the application's Gemfile by executing:
70
+ ```ruby
71
+ gem 'sqa'
72
+ ```
74
73
 
75
- bundle add sqa
74
+ Then run:
76
75
 
77
- If bundler is not being used to manage dependencies, install the gem by executing:
76
+ ```bash
77
+ bundle install
78
+ ```
78
79
 
79
- gem install sqa
80
+ ## Configuration
80
81
 
81
- ## Semantic Versioning
82
+ ### Data Directory
82
83
 
83
- `sqa` uses the [sem_version](https://github.com/canton7/sem_version) gem to provide a semantic version object. Sure its old; but, so am I. Doesn't mean we can't do the job.
84
+ SQA stores historical price data and metadata in a data directory. By default, it uses `~/sqa_data/`:
84
85
 
85
86
  ```ruby
86
- # On the command line:
87
- sqa --version
87
+ require 'sqa'
88
88
 
89
- # or inside your program:
89
+ # Initialize with default configuration
90
+ SQA.init
90
91
 
91
- SQA.version # returns SemVersion object
92
- exit(1) unless SQA.version >= SemVersion("1.0.0")
92
+ # Or specify a custom data directory
93
+ SQA::Config.new(data_dir: '~/Documents/my_stock_data')
94
+ ```
95
+
96
+ Create your data directory:
93
97
 
94
- # Okay, you're right, you could put that kind of version
95
- # constraint in a Gemfile and let bundler handle the
96
- # dependencies for you.
98
+ ```bash
99
+ mkdir ~/sqa_data
97
100
  ```
98
101
 
102
+ ### Alpha Vantage API Key
103
+
104
+ SQA uses Alpha Vantage for stock data. You'll need a free API key from [https://www.alphavantage.co/](https://www.alphavantage.co/)
105
+
106
+ Set the environment variable:
107
+
108
+ ```bash
109
+ export AV_API_KEY="your_api_key_here"
110
+ # or
111
+ export ALPHAVANTAGE_API_KEY="your_api_key_here"
112
+ ```
113
+
114
+ The free tier allows:
115
+ - 5 API calls per minute
116
+ - 100 API calls per day
117
+
99
118
  ## Usage
100
119
 
101
- **Do not use!** but its okay to play with.
120
+ ### Working with Stocks
102
121
 
103
- `SQA` can be used from the command line or as a library in your own application.
122
+ ```ruby
123
+ require 'sqa'
104
124
 
105
- `SQA` has a command line component.
125
+ # Initialize SQA
126
+ SQA.init
106
127
 
107
- ```plaintext
108
- $ sqa --help
109
- Stock Quantitative Analysis (SQA)
128
+ # Create or load a stock (automatically fetches/updates data from Alpha Vantage)
129
+ aapl = SQA::Stock.new(ticker: 'AAPL', source: :alpha_vantage)
130
+ #=> aapl with 1207 data points from 2019-01-02 to 2023-10-17
110
131
 
111
- Usage: sqa [analysis|web] [OPTIONS]
132
+ # Access the DataFrame
133
+ aapl.df
134
+ #=> SQA::DataFrame (wraps Polars::DataFrame)
112
135
 
113
- A collection of things
136
+ # Get column names
137
+ aapl.df.columns
138
+ #=> ["timestamp", "open_price", "high_price", "low_price", "close_price", "adj_close_price", "volume"]
114
139
 
115
- Options:
116
- -c, --config string Path to the config file
117
- --data-dir string Set the directory for the SQA data
118
- -d, --debug Turn on debugging output
119
- --dump-config path_to_file Dump the current configuration
120
- -h, --help Print usage
121
- -l, --log_level string Set the log level (debug, info, warn, error,
122
- fatal)
123
- -p, --portfolio string Set the filename of the portfolio
124
- -t, --trades string Set the filename into which trades are
125
- stored
126
- -v, --verbose Print verbosely
127
- --version Print version
140
+ # Access price data
141
+ aapl.df["adj_close_price"]
142
+ #=> Polars::Series with all adjusted closing prices
128
143
 
129
- Examples:
130
- sqa -c ~/.sqa.yml -p portfolio.csv -t trades.csv --data-dir ~/sqa_data
144
+ # Get recent prices
145
+ prices = aapl.df["adj_close_price"].to_a
146
+ prices.last(5)
147
+ #=> [179.8, 180.71, 178.85, 178.72, 177.15]
131
148
 
132
- Optional Command Available:
133
- analysis - Provide an Analysis of a Portfolio
134
- web - Run a web server
149
+ # Access company metadata
150
+ aapl.name
151
+ #=> "Apple Inc."
135
152
 
136
- WARNING: This is a toy, a play thing, not intended for serious use.
137
- ```
153
+ aapl.exchange
154
+ #=> "NASDAQ"
138
155
 
139
- More about the two included programs `analysis` and `web` later.
156
+ aapl.overview
157
+ #=> Hash with detailed company information
158
+ ```
140
159
 
141
- ### Setup a Config File
160
+ ### DataFrame Operations
142
161
 
143
- You will need to create a directory to store the `sqa` data and a configuration file. You can start by doing this:
162
+ SQA::DataFrame is a wrapper around the high-performance Polars DataFrame:
144
163
 
145
164
  ```ruby
146
- gem install sqa
147
- mkdir ~/Documents/sqa_data
148
- sqa --data-dir ~/Documents/sqa_data --dump-config ~/.sqa.yml
149
- ```
165
+ require 'sqa'
166
+
167
+ # Create from hash
168
+ data = {
169
+ timestamp: ['2023-01-01', '2023-01-02', '2023-01-03'],
170
+ price: [100.0, 101.5, 99.8]
171
+ }
172
+ df = SQA::DataFrame.new(data)
150
173
 
151
- By default `SQA` looks for a configuration file named `.sqa.yml` in the current directory. If it does not find one there it looks in the home directory. You can use the `--config` CLI option to specify a path to your custom config file name.
174
+ # Load from CSV
175
+ df = SQA::DataFrame.load(source: 'aapl.csv')
176
+
177
+ # Basic operations
178
+ df.size # Number of rows
179
+ df.ncols # Number of columns
180
+ df.to_h # Convert to hash
181
+ df.to_csv('output.csv') # Save to CSV
182
+
183
+ # Access underlying Polars DataFrame
184
+ df.data # Polars::DataFrame
185
+ ```
152
186
 
153
- You can have multiple configurations and multiple data directories.
187
+ ### Technical Indicators
154
188
 
155
- You can also have one data directory and multiple portfolio and trades files within that single directory.
189
+ All technical indicators are provided by the [`sqa-tai`](https://github.com/MadBomber/sqa-tai) gem, which wraps the industry-standard TA-Lib library:
156
190
 
157
- ### AlphaVantage
191
+ ```ruby
192
+ require 'sqa'
158
193
 
159
- `SQA` makes use of the `AlphaVantage` API to get some stock related information. You will need an API key in order to use this functionality. They have a free rate limited API key which allows 5 accesses per second; total of 100 accesses in a day. If you are doing more than that you are not playing an ought to purchase one of there serious API key plans.
194
+ prices = [100, 102, 105, 103, 107, 110, 108, 112, 115, 113]
160
195
 
161
- [https://www.alphavantage.co/](https://www.alphavantage.co/)
196
+ # Simple Moving Average
197
+ sma = SQAI.sma(prices, period: 5)
198
+ #=> [104.0, 105.4, 106.6, 108.0, 110.4, 111.6]
162
199
 
163
- Put your API key in the system environment variable `AV_API_KEY`
200
+ # Relative Strength Index
201
+ rsi = SQAI.rsi(prices, period: 14)
202
+ #=> [70.5, 68.2, ...]
164
203
 
165
- <!-- TODO: why is it not part of the configuration? ought to be. -->
204
+ # Exponential Moving Average
205
+ ema = SQAI.ema(prices, period: 5)
206
+ #=> [...]
166
207
 
208
+ # Bollinger Bands (returns upper, middle, lower)
209
+ upper, middle, lower = SQAI.bbands(prices, period: 5)
167
210
 
168
- ### TradingView
211
+ # MACD
212
+ macd, signal, histogram = SQAI.macd(prices, fast_period: 12, slow_period: 26, signal_period: 9)
169
213
 
170
- **Not currently implemented** but seriously considering using [TradingView](http://tradingview.com) capabilities. If I do choose to use these capabilities you will need your own API key or account or something.
214
+ # Many more indicators available!
215
+ # See: https://github.com/MadBomber/sqa-tai
216
+ ```
171
217
 
172
- Put your API key in the system environment variable `TV_API_KEY`
218
+ **Available Indicators:** SMA, EMA, WMA, RSI, MACD, Bollinger Bands, Stochastic, ADX, ATR, CCI, Williams %R, ROC, Momentum, and 140+ more via TA-Lib.
173
219
 
174
- <!-- TODO: why is it not part of the configuration? ought to be. -->
220
+ ### Trading Strategies
175
221
 
176
- I kinda like their Terms of Use. I've started crafting an [SQA Terms of Use](docs/terms_of_use.md) document based on theirs. I specifically like sections 7 and 8.
222
+ Build and test trading strategies:
177
223
 
178
- ### Yahoo Finance
224
+ ```ruby
225
+ require 'sqa'
226
+ require 'ostruct'
179
227
 
180
- The finance.yahoo.com website no longer supports an API. To get information from this website you must scrape it. That is how historical / receint stock prices are being obtained in the DataFrame::YahooFinance class. I do not like doing this and do not necessarily recommend using YahooFinance as a source for stock price data.
228
+ # Load strategies
229
+ strategies = SQA::Strategy.new
230
+ strategies.auto_load # Loads all built-in strategies
231
+
232
+ # Add specific strategies
233
+ strategies.add SQA::Strategy::RSI
234
+ strategies.add SQA::Strategy::SMA
235
+ strategies.add SQA::Strategy::EMA
236
+
237
+ # Prepare data for strategy execution
238
+ prices = aapl.df["adj_close_price"].to_a
239
+ rsi_value = SQAI.rsi(prices, period: 14).last
240
+
241
+ vector = OpenStruct.new
242
+ vector.rsi = rsi_value
243
+ vector.prices = prices
244
+
245
+ # Execute strategies
246
+ results = strategies.execute(vector)
247
+ #=> [:hold, :buy, :hold] # One result per strategy
248
+
249
+ # Built-in strategies:
250
+ # - SQA::Strategy::RSI - Based on Relative Strength Index
251
+ # - SQA::Strategy::SMA - Simple Moving Average crossover
252
+ # - SQA::Strategy::EMA - Exponential Moving Average crossover
253
+ # - SQA::Strategy::MACD - MACD crossover strategy
254
+ # - SQA::Strategy::BollingerBands - Bollinger Bands bounce strategy
255
+ # - SQA::Strategy::Stochastic - Stochastic oscillator strategy
256
+ # - SQA::Strategy::VolumeBreakout - Volume-based breakout strategy
257
+ # - SQA::Strategy::MR - Mean Reversion
258
+ # - SQA::Strategy::MP - Market Profile
259
+ # - SQA::Strategy::KBS - Knowledge-based RETE strategy (advanced)
260
+ # - SQA::Strategy::Random - Random signal generator (for testing)
261
+ # - SQA::Strategy::Consensus - Combines multiple strategies
262
+ ```
181
263
 
182
- In the SQA::Stock class the default source is AlphaVantage which has its own limitations.
264
+ ### Statistics
183
265
 
184
- This is not to say that finance.yahoo.com has no use. In fact is is the perfect place to go to manually download CSV files of historical stock price data. When you do that through your browser, the CSV file will be placed in your default downloads directory.
266
+ Comprehensive statistical analysis on price data (via `lite-statistics` gem):
185
267
 
186
- To use this downed file within the SQA environment it must be moved into your `SQA.`data_dir` with a filename that is all lowercase. The filename must be the stock's symbol with a `.csv` extension. For example if you downloaded the entire historical stock price data for Apple Computer (AAPL) the filename in the SQA.data_dir should be "aapl.csv"
268
+ ```ruby
269
+ require 'sqa'
187
270
 
188
- You can manually go to the Yahoo Finance website at [https://finance.yahoo.com/quote/AAPL/history?p=AAPL](https://finance.yahoo.com/quote/AAPL/history?p=AAPL)
271
+ prices = [179.8, 180.71, 178.85, 178.72, 177.15]
272
+
273
+ stats = prices.summary
274
+ #=> {
275
+ # max: 180.71,
276
+ # min: 177.15,
277
+ # mean: 179.046,
278
+ # median: 178.85,
279
+ # mode: nil,
280
+ # range: 3.56,
281
+ # sample_standard_deviation: 1.19,
282
+ # sample_variance: 1.42,
283
+ # ...
284
+ # }
285
+ ```
189
286
 
287
+ ### Portfolio Management
190
288
 
191
- ## Playing in IRB
289
+ Track positions, trades, and P&L:
192
290
 
193
- You can play around in IRB with the SQA framework.
291
+ ```ruby
292
+ require 'sqa'
194
293
 
294
+ # Create portfolio with initial cash and commission
295
+ portfolio = SQA::Portfolio.new(initial_cash: 10_000.0, commission: 1.0)
195
296
 
196
- ### With Stocks and Indicators
297
+ # Buy stock
298
+ portfolio.buy('AAPL', shares: 10, price: 150.0, date: Date.today)
197
299
 
198
- You will need some CSV files. If you ask for a stock to which you have not existing historical price data in a CSV file, `SQA` can use either Alpha Vantage or Yahoo Finance to get some data. I like Alpha Vantage better because it has a well defined and documented API. Yahoo Finance on the other hand does not. You can manually download historical stock price data from Yahoo Finance into you `sqa data directory`
300
+ # Sell stock
301
+ portfolio.sell('AAPL', shares: 5, price: 160.0, date: Date.today)
199
302
 
200
- Historical price data is kept in the `SQA.data_dir` in a CSV file whose name is all lowercase. If you download the CSV file for the stock symbol "AAPL" it should be saved in you `SQA.data_dir` as `aapl.csv`
303
+ # Check portfolio value
304
+ current_prices = { 'AAPL' => 165.0 }
305
+ portfolio.value(current_prices)
306
+ #=> Total portfolio value
201
307
 
202
- #### Get Historical Prices
308
+ # Calculate profit/loss
309
+ portfolio.profit_loss(current_prices)
310
+ #=> P&L amount
203
311
 
204
- Go to [https::/finance.yahoo.com](https::/finance.yahoo.com) and down some historical price data for your favorite stocks. Put those CSV files into the `SQA.data_dir`.
312
+ # View summary
313
+ portfolio.summary
314
+ #=> { initial_cash: 10000.0, cash: 8248.0, positions: 1, trades: 2, ... }
205
315
 
206
- You may need to create a `portfolio.csv` file or you may not.
316
+ # Save trades to CSV
317
+ portfolio.save_to_csv('my_trades.csv')
207
318
 
208
- <!-- TODO: Add a section on how to create a portfolio fCSV file -->
319
+ # Load from CSV
320
+ portfolio.load_from_csv('my_trades.csv')
321
+ ```
209
322
 
210
- The CSV files will be named by the stock's ticker symbol as lower case. For example: `aapl.csv`
323
+ ### Backtesting
211
324
 
212
- ### Playing in the IRB - Setup
325
+ Simulate trading strategies against historical data:
213
326
 
214
327
  ```ruby
215
328
  require 'sqa'
216
- require 'sqa/cli'
217
329
 
218
- # TODO: Is this still true after the dry-cli integration?
219
- #
220
- # You can pass a set of CLI options in a String
221
- SQA.init "-c ~/.sqa.yml"
330
+ SQA.init
331
+ stock = SQA::Stock.new(ticker: 'AAPL')
332
+
333
+ # Create backtest
334
+ backtest = SQA::Backtest.new(
335
+ stock: stock,
336
+ strategy: SQA::Strategy::RSI,
337
+ initial_capital: 10_000.0,
338
+ commission: 1.0
339
+ )
340
+
341
+ # Run backtest
342
+ results = backtest.run
343
+
344
+ # View comprehensive metrics
345
+ puts "Total Return: #{results.total_return.round(2)}%"
346
+ puts "Annualized Return: #{results.annualized_return.round(2)}%"
347
+ puts "Sharpe Ratio: #{results.sharpe_ratio.round(2)}"
348
+ puts "Max Drawdown: #{results.max_drawdown.round(2)}%"
349
+ puts "Total Trades: #{results.total_trades}"
350
+ puts "Win Rate: #{results.win_rate.round(2)}%"
351
+ puts "Profit Factor: #{results.profit_factor.round(2)}"
352
+
353
+ # Access equity curve for charting
354
+ results.equity_curve #=> Array of portfolio values over time
355
+ ```
356
+
357
+ ### Strategy Generator
222
358
 
223
- # If you have an API key for AlphaVantage you can create a new
224
- # CSV file, or update an existing one. Your key MUST be
225
- # in the system environment variable AV_API_KEY
226
- aapl = SQA::Stock.new(ticker: 'aapl', source: :alpha_vantage)
227
- #=> aapl with 1207 data points from 2019-01-02 to 2023-10-17
359
+ Reverse-engineer profitable trades to discover winning patterns:
360
+
361
+ ```ruby
362
+ require 'sqa'
363
+
364
+ SQA.init
365
+ stock = SQA::Stock.new(ticker: 'AAPL')
366
+
367
+ # Create strategy generator
368
+ generator = SQA::StrategyGenerator.new(
369
+ stock: stock,
370
+ min_gain_percent: 10.0, # Find trades with ≥10% gain
371
+ holding_period: (5..20) # Within 5-20 days
372
+ )
373
+
374
+ # Discover patterns
375
+ patterns = generator.discover_patterns(min_pattern_frequency: 3)
376
+
377
+ # Print discovered patterns
378
+ generator.print_patterns(max_patterns: 10)
379
+ #=> Pattern #1:
380
+ #=> Frequency: 15 occurrences
381
+ #=> Average Gain: 12.5%
382
+ #=> Average Holding: 8.3 days
383
+ #=> Conditions:
384
+ #=> - rsi: oversold
385
+ #=> - macd_crossover: bullish
386
+ #=> - volume: high
387
+
388
+ # Generate strategy from top pattern
389
+ strategy = generator.generate_strategy(pattern_index: 0, strategy_type: :class)
390
+
391
+ # Backtest the discovered strategy
392
+ backtest = SQA::Backtest.new(stock: stock, strategy: strategy)
393
+ results = backtest.run
394
+
395
+ # Export patterns to CSV
396
+ generator.export_patterns('/tmp/patterns.csv')
228
397
  ```
229
398
 
230
- `aapl.df` is the data frame. It is implemented as a Hashie::Mash obect -- a Hash or Arrays.
399
+ ### Genetic Programming
400
+
401
+ Evolve optimal strategy parameters through natural selection:
231
402
 
232
403
  ```ruby
233
- aapl.df.keys
234
- #=> [:timestamp, :open_price, :high_price, :low_price, :close_price, :adj_close_price, :volume]
404
+ require 'sqa'
235
405
 
236
- aapl.df.adj_close_price.last(5)
237
- #=> [179.8, 180.71, 178.85, 178.72, 177.15]
406
+ SQA.init
407
+ stock = SQA::Stock.new(ticker: 'AAPL')
408
+
409
+ # Create genetic program
410
+ gp = SQA::GeneticProgram.new(
411
+ stock: stock,
412
+ population_size: 50,
413
+ generations: 100,
414
+ mutation_rate: 0.15
415
+ )
416
+
417
+ # Define parameter space to explore
418
+ gp.define_genes(
419
+ rsi_period: (7..30).to_a,
420
+ buy_threshold: (20..40).to_a,
421
+ sell_threshold: (60..80).to_a
422
+ )
423
+
424
+ # Define fitness function (backtest performance)
425
+ gp.fitness do |genes|
426
+ # Create strategy with these genes
427
+ strategy = create_rsi_strategy(genes)
428
+
429
+ # Backtest and return total return as fitness
430
+ backtest = SQA::Backtest.new(stock: stock, strategy: strategy)
431
+ backtest.run.total_return
432
+ end
433
+
434
+ # Evolve!
435
+ best = gp.evolve
436
+
437
+ puts "Best Parameters:"
438
+ puts " RSI Period: #{best.genes[:rsi_period]}"
439
+ puts " Buy Threshold: #{best.genes[:buy_threshold]}"
440
+ puts " Sell Threshold: #{best.genes[:sell_threshold]}"
441
+ puts " Fitness: #{best.fitness.round(2)}%"
442
+
443
+ # View evolution history
444
+ gp.history.each do |gen|
445
+ puts "Gen #{gen[:generation]}: Best=#{gen[:best_fitness].round(2)}%"
446
+ end
238
447
  ```
239
448
 
240
- `aapl.data` is basic static data, company name, industry etc. It is also implemented as a Hassie::Mash object but is primary treated as a plain hash object.
449
+ ### Knowledge-Based Strategy (KBS)
450
+
451
+ Build sophisticated rule-based systems with RETE forward chaining:
241
452
 
242
453
  ```ruby
243
- aapl.data.keys
244
- #=> [:ticker, :source, :indicators, :overview]
454
+ require 'sqa'
245
455
 
246
- aapl.data.source
247
- => "alpha_vantage"
456
+ # Create KBS strategy
457
+ strategy = SQA::Strategy::KBS.new(load_defaults: false)
458
+
459
+ # Define custom trading rules
460
+ strategy.add_rule :golden_opportunity do
461
+ desc "Perfect storm: Multiple bullish indicators align"
462
+
463
+ # Multiple conditions
464
+ on :rsi, { level: :oversold }
465
+ on :macd, { crossover: :bullish }
466
+ on :stochastic, { zone: :oversold, crossover: :bullish }
467
+ on :trend, { short_term: :up, strength: :strong }
468
+ on :volume, { level: :high }
469
+
470
+ # Negation: Don't buy if overbought elsewhere
471
+ without :rsi, { level: :overbought }
472
+
473
+ # Action
474
+ then do
475
+ assert(:signal, { action: :buy, confidence: :high })
476
+ end
477
+ end
478
+
479
+ # Execute strategy
480
+ signal = strategy.execute(vector)
481
+ #=> :buy or :sell or :hold
482
+
483
+ # Use with backtesting
484
+ backtest = SQA::Backtest.new(
485
+ stock: stock,
486
+ strategy: SQA::Strategy::KBS,
487
+ initial_capital: 10_000.0
488
+ )
489
+ results = backtest.run
490
+ ```
248
491
 
249
- aapl.data.overciew.keys
250
- => [:symbol, :asset_type, :name, :description, :cik, :exchange, :currency, :country, :sector, :industry, :address, :fiscal_year_end, :latest_quarter, :market_capitalization, :ebitda, :pe_ratio, :peg_ratio, :book_value, :dividend_per_share, :dividend_yield, :eps, :revenue_per_share_ttm, :profit_margin, :operating_margin_ttm, :return_on_assets_ttm, :return_on_equity_ttm, :revenue_ttm, :gross_profit_ttm, :diluted_epsttm, :quarterly_earnings_growth_yoy, :quarterly_revenue_growth_yoy, :analyst_target_price, :trailing_pe, :forward_pe, :price_to_sales_ratio_ttm, :price_to_book_ratio, :ev_to_revenue, :ev_to_ebitda, :beta, :"52_week_high", :"52_week_low", :"50_day_moving_average", :"200_day_moving_average", :shares_outstanding, :dividend_date, :ex_dividend_date]
492
+ ### Real-Time Streaming
251
493
 
494
+ Process live stock prices and generate on-the-fly trading signals:
495
+
496
+ ```ruby
497
+ require 'sqa'
498
+
499
+ # Create stream processor
500
+ stream = SQA::Stream.new(
501
+ ticker: 'AAPL',
502
+ window_size: 100,
503
+ strategies: [
504
+ SQA::Strategy::RSI,
505
+ SQA::Strategy::MACD,
506
+ SQA::Strategy::BollingerBands
507
+ ]
508
+ )
509
+
510
+ # Register signal callback
511
+ stream.on_signal do |signal, data|
512
+ puts "🔔 SIGNAL: #{signal.upcase}"
513
+ puts " Price: $#{data[:price]}"
514
+ puts " Consensus: #{data[:strategies_vote]}"
515
+
516
+ # Execute trade, send alert, etc.
517
+ execute_trade(signal, data) if signal != :hold
518
+ end
519
+
520
+ # Register update callback (optional)
521
+ stream.on_update do |data|
522
+ puts "📊 Price update: $#{data[:price]}"
523
+ end
524
+
525
+ # Feed live data (from WebSocket, API, etc.)
526
+ stream.update(
527
+ price: 150.25,
528
+ volume: 1_000_000,
529
+ timestamp: Time.now
530
+ )
531
+
532
+ # Access real-time indicators
533
+ rsi = stream.indicator(:rsi, period: 14)
534
+ sma = stream.indicator(:sma, period: 20)
535
+
536
+ # View stream statistics
537
+ stream.stats
538
+ #=> { ticker: 'AAPL', updates: 125, current_price: 150.25, ... }
252
539
  ```
253
540
 
254
- ### Playing in the IRB - Statistics
541
+ ### FPL Analysis (Future Period Loss/Profit)
255
542
 
256
- Basic statistics are available on all of the SQA::DataFrame arrays.
543
+ Analyze potential future price movements with risk metrics:
257
544
 
258
545
  ```ruby
259
- require 'amazing_print' # to get the ap command
260
-
261
- # Look at some summary stats on the last 5 days of
262
- # adjusted closing pricess of AAPL
263
- ap aapl.df.adj_close_price.last(5).summary
264
- {
265
- :frequencies => {
266
- 179.8 => 1,
267
- 180.71 => 1,
268
- 178.85 => 1,
269
- 178.72 => 1,
270
- 177.15 => 1
271
- },
272
- :max => 180.71,
273
- :mean => 179.046,
274
- :median => 178.85,
275
- :midrange => 178.93,
276
- :min => 177.15,
277
- :mode => nil,
278
- :proportions => {
279
- 179.8 => 0.2,
280
- 180.71 => 0.2,
281
- 178.85 => 0.2,
282
- 178.72 => 0.2,
283
- 177.15 => 0.2
284
- },
285
- :quartile1 => 178.85,
286
- :quartile2 => 179.8,
287
- :quartile3 => 180.71,
288
- :range => 3.5600000000000023,
289
- :size => 5,
290
- :sum => 895.23,
291
- :sample_coefficient_of_variation => 0.006644656242680533,
292
- :sample_kurtosis => 2.089087404921432,
293
- :sample_size => 5,
294
- :sample_skewness => -0.2163861377512453,
295
- :sample_standard_deviation => 1.1896991216269788,
296
- :sample_standard_error => 0.532049621745943,
297
- :sample_variance => 1.415384000000005,
298
- :sample_zscores => {
299
- 179.8 => 0.6337736880639895,
300
- 180.71 => 1.3986729667618856,
301
- 178.85 => -0.16474753695031497,
302
- 178.72 => -0.2740188624785824,
303
- 177.15 => -1.5936802553969298
304
- }
305
- }
306
- #=> nil
546
+ require 'sqa'
547
+
548
+ stock = SQA::Stock.new(ticker: 'AAPL')
549
+
550
+ # Calculate FPL for 14-day future period
551
+ fpl_data = stock.df.fpl(fpop: 14)
552
+ # Returns: [[min_delta, max_delta], ...] for each price point
553
+
554
+ # Full analysis with risk metrics
555
+ analysis = stock.df.fpl_analysis(fpop: 14)
556
+
557
+ analysis.first
558
+ #=> {
559
+ # min_delta: -5.2, # Worst loss percentage
560
+ # max_delta: 8.7, # Best gain percentage
561
+ # risk: 13.9, # Volatility (max - min)
562
+ # direction: :UNCERTAIN,# :UP, :DOWN, :UNCERTAIN, :FLAT
563
+ # magnitude: 1.75, # Average expected movement
564
+ # interpretation: "UNCERTAIN: 1.75% (±6.95% risk)"
565
+ #}
566
+
567
+ # Filter high-quality opportunities
568
+ good_indices = SQA::FPOP.filter_by_quality(
569
+ analysis,
570
+ min_magnitude: 5.0, # Minimum average movement
571
+ max_risk: 10.0, # Maximum acceptable volatility
572
+ directions: [:UP] # Only bullish patterns
573
+ )
574
+
575
+ # Use with Strategy Generator for risk-aware pattern discovery
576
+ generator = SQA::StrategyGenerator.new(
577
+ stock: stock,
578
+ min_gain_percent: 10.0,
579
+ fpop: 10,
580
+ max_fpl_risk: 20.0, # NEW: Filter high-volatility points
581
+ required_fpl_directions: [:UP] # NEW: Only bullish patterns
582
+ )
307
583
  ```
308
584
 
309
- ### Playing in the IRB - Indicators
585
+ **Key Benefits:**
586
+ - Captures both upside AND downside potential
587
+ - Measures volatility/risk during future period
588
+ - Classifies directional bias (UP/DOWN/UNCERTAIN/FLAT)
589
+ - Enables risk-adjusted opportunity filtering
310
590
 
311
- The SQA::Indicator class methods use Arrays not the DataFrame
312
- Here is an example:
591
+ ### Market Regime Detection
313
592
 
593
+ Identify and adapt to changing market conditions:
314
594
 
315
595
  ```ruby
316
- prices = aapl.df.adj_close_price
317
- period = 14 # size of the window in prices to analyze
596
+ # Detect current market regime
597
+ regime = SQA::MarketRegime.detect(stock)
598
+ #=> {
599
+ # type: :bull, # :bull, :bear, :sideways
600
+ # volatility: :low, # :low, :medium, :high
601
+ # strength: :strong, # :weak, :moderate, :strong
602
+ # lookback_days: 60,
603
+ # detected_at: 2024-11-08
604
+ #}
605
+
606
+ # Analyze regime history
607
+ regimes = SQA::MarketRegime.detect_history(stock)
608
+ #=> [
609
+ # { type: :bull, start_index: 0, end_index: 120, duration: 120 },
610
+ # { type: :bear, start_index: 121, end_index: 200, duration: 79 },
611
+ # ...
612
+ #]
613
+
614
+ # Split data by regime
615
+ regime_data = SQA::MarketRegime.split_by_regime(stock)
616
+ #=> {
617
+ # bull: [{ prices: [...], start_index: 0, end_index: 120 }, ...],
618
+ # bear: [...],
619
+ # sideways: [...]
620
+ #}
621
+
622
+ # Use regime-specific strategies
623
+ current_regime = SQA::MarketRegime.detect(stock)
624
+ strategy = case current_regime[:type]
625
+ when :bull
626
+ SQA::Strategy::MomentumBreakout
627
+ when :bear
628
+ SQA::Strategy::MeanReversion
629
+ when :sideways
630
+ SQA::Strategy::BollingerBands
631
+ end
632
+ ```
633
+
634
+ **Key Benefits:**
635
+ - Different strategies for different market conditions
636
+ - Avoid using bull market strategies in bear markets
637
+ - Detect regime changes before they impact performance
638
+
639
+ ### Seasonal Analysis
318
640
 
319
- rsi = SQA::Indicator.rsi(prices, period)
320
- #=> {:rsi=>63.46652828230407, :trend=>:normal}
641
+ Discover calendar-dependent patterns:
642
+
643
+ ```ruby
644
+ # Analyze seasonal performance
645
+ seasonal = SQA::SeasonalAnalyzer.analyze(stock)
646
+ #=> {
647
+ # best_months: [10, 11, 12], # October, November, December
648
+ # worst_months: [5, 6, 7], # May, June, July
649
+ # best_quarters: [4, 1], # Q4 and Q1
650
+ # worst_quarters: [2, 3],
651
+ # has_seasonal_pattern: true,
652
+ # monthly_returns: { ... },
653
+ # quarterly_returns: { ... }
654
+ #}
655
+
656
+ # Filter data for Q4 only (holiday shopping season)
657
+ q4_data = SQA::SeasonalAnalyzer.filter_by_quarters(stock, [4])
658
+ #=> { indices: [...], dates: [...], prices: [...] }
659
+
660
+ # Filter for specific months
661
+ dec_data = SQA::SeasonalAnalyzer.filter_by_months(stock, [12])
662
+
663
+ # Get context for specific date
664
+ context = SQA::SeasonalAnalyzer.context_for_date(Date.new(2024, 12, 15))
665
+ #=> {
666
+ # month: 12,
667
+ # quarter: 4,
668
+ # month_name: "December",
669
+ # quarter_name: "Q4",
670
+ # is_year_end: true,
671
+ # is_holiday_season: true,
672
+ # is_earnings_season: false
673
+ #}
674
+
675
+ # Discover patterns only for best months
676
+ generator = SQA::StrategyGenerator.new(stock: stock)
677
+ patterns = generator.discover_context_aware_patterns(
678
+ analyze_seasonal: true
679
+ )
680
+
681
+ patterns.each do |pattern|
682
+ puts pattern.context.valid_months # [10, 11, 12] for Q4 patterns
683
+ end
321
684
  ```
322
685
 
323
- ### Playing in the IRB - Strategies
686
+ **Real-World Examples:**
687
+ - Retail stocks: Q4 holiday shopping boost
688
+ - Tax prep stocks: Q1/Q4 seasonal surge
689
+ - Energy stocks: Summer driving season
690
+ - Agriculture stocks: Planting/harvest cycles
691
+
692
+ ### Sector Analysis with KBS
324
693
 
325
- The strategies work off of an Object that contains the information required to make its recommendation. Building on the previous Ruby snippet ...
694
+ Analyze patterns across related stocks using KBS blackboards:
326
695
 
327
696
  ```ruby
328
- require 'ostruct'
329
- vector = OpenStruct.new
330
- vector.rsi = rsi
331
-
332
- # Load some trading strategies
333
- ss = SWA::Strategy.new
334
- ss.auto_load # loads everything in the lib/sqa/strategy folder
335
-
336
- # Select some strategies to execute
337
- ss.add SWA::Strategy::Random # 3-sided coin flip
338
- ss.add SQA::Strategy::RSI
339
-
340
- # This is an Array with each "trade" method
341
- # that is defined in each strategy added
342
- ap ss.strategies #=>∑
343
- [
344
- [0] SQA::Strategy::Random#trade(vector),
345
- [1] SQA::Strategy::RSI#trade(vector)
697
+ # Create sector analyzer (separate blackboard per sector)
698
+ analyzer = SQA::SectorAnalyzer.new
699
+
700
+ # Add stocks to technology sector
701
+ analyzer.add_stock('AAPL', sector: :technology)
702
+ analyzer.add_stock('MSFT', sector: :technology)
703
+ analyzer.add_stock('GOOGL', sector: :technology)
704
+
705
+ # Detect sector-wide regime
706
+ tech_stocks = [
707
+ SQA::Stock.new(ticker: 'AAPL'),
708
+ SQA::Stock.new(ticker: 'MSFT'),
709
+ SQA::Stock.new(ticker: 'GOOGL')
346
710
  ]
347
711
 
348
- # Execute those strategies
349
- results = ss.execute(vector)
350
- #=> [:hold, :hold]
712
+ sector_regime = analyzer.detect_sector_regime(:technology, tech_stocks)
713
+ #=> {
714
+ # sector: :technology,
715
+ # consensus_regime: :bull, # Majority vote
716
+ # sector_strength: 85.0, # % of stocks bullish
717
+ # stock_regimes: [...]
718
+ #}
719
+
720
+ # Discover patterns across entire sector
721
+ sector_patterns = analyzer.discover_sector_patterns(
722
+ :technology,
723
+ tech_stocks,
724
+ min_gain_percent: 10.0,
725
+ fpop: 10
726
+ )
727
+
728
+ # Patterns that work for MULTIPLE stocks in sector
729
+ sector_patterns.each do |sp|
730
+ puts "Pattern found in: #{sp[:stocks].join(', ')}"
731
+ puts "Avg gain: #{sp[:avg_gain].round(2)}%"
732
+ puts "Conditions: #{sp[:conditions]}"
733
+ end
734
+
735
+ # Query sector blackboard
736
+ facts = analyzer.query_sector(:technology, :sector_pattern)
737
+
738
+ # Print sector summary
739
+ analyzer.print_sector_summary(:technology)
740
+ ```
741
+
742
+ **Available Sectors:**
743
+ `:technology`, `:finance`, `:healthcare`, `:energy`, `:consumer`, `:industrial`, `:materials`, `:utilities`, `:real_estate`, `:communications`
744
+
745
+ **Key Benefits:**
746
+ - Leverage "stocks move together" in same sector
747
+ - Find patterns valid across multiple related stocks
748
+ - Persistent KBS blackboard tracks sector knowledge
749
+ - Cross-stock pattern validation
750
+
751
+ ### Walk-Forward Validation
752
+
753
+ Prevent overfitting with time-series cross-validation:
754
+
755
+ ```ruby
756
+ generator = SQA::StrategyGenerator.new(
757
+ stock: stock,
758
+ min_gain_percent: 10.0,
759
+ fpop: 10
760
+ )
761
+
762
+ # Run walk-forward validation
763
+ results = generator.walk_forward_validate(
764
+ train_size: 250, # 1 year training window
765
+ test_size: 60, # 3 months testing window
766
+ step_size: 30 # Step forward 1 month each iteration
767
+ )
768
+
769
+ # Only patterns that work out-of-sample
770
+ validated_patterns = results[:validated_patterns]
771
+
772
+ # Detailed results for each iteration
773
+ results[:validation_results].each do |r|
774
+ puts "Train: #{r[:train_period]}"
775
+ puts "Test: #{r[:test_period]}"
776
+ puts "Return: #{r[:test_return]}%"
777
+ puts "Sharpe: #{r[:test_sharpe]}"
778
+ end
779
+
780
+ # Statistics
781
+ all_returns = results[:validation_results].map { |r| r[:test_return] }
782
+ avg_return = all_returns.sum / all_returns.size
783
+ puts "Average out-of-sample return: #{avg_return.round(2)}%"
784
+ ```
785
+
786
+ **How It Works:**
787
+ 1. Split data into train/test windows
788
+ 2. Discover patterns on training data
789
+ 3. Test patterns on future (unseen) data
790
+ 4. Roll forward and repeat
791
+ 5. Only keep patterns that work out-of-sample
792
+
793
+ **Prevents:**
794
+ - Curve-fitting to historical noise
795
+ - Patterns that only worked in the past
796
+ - Overconfidence in backtest results
797
+
798
+ ### Pattern Context System
799
+
800
+ Context-aware patterns that know when/where they're valid:
801
+
802
+ ```ruby
803
+ # Discover patterns with full context
804
+ generator = SQA::StrategyGenerator.new(stock: stock, min_gain_percent: 10.0)
805
+
806
+ patterns = generator.discover_context_aware_patterns(
807
+ analyze_regime: true,
808
+ analyze_seasonal: true,
809
+ sector: :technology
810
+ )
811
+
812
+ # Each pattern has rich context metadata
813
+ pattern = patterns.first
814
+ pattern.context.market_regime #=> :bull
815
+ pattern.context.volatility_regime #=> :low
816
+ pattern.context.valid_months #=> [10, 11, 12]
817
+ pattern.context.valid_quarters #=> [4, 1]
818
+ pattern.context.sector #=> :technology
819
+ pattern.context.discovered_period #=> "2020-01-01 to 2022-12-31"
820
+
821
+ # Runtime validation - check if pattern applies NOW
822
+ valid = pattern.context.valid_for?(
823
+ date: Date.today,
824
+ regime: :bull,
825
+ sector: :technology
826
+ )
827
+
828
+ if valid
829
+ # Execute strategy - context matches current conditions
830
+ signal = strategy.trade(vector)
831
+ else
832
+ # Skip - pattern not valid in current context
833
+ puts "Pattern not valid for current conditions"
834
+ end
835
+
836
+ # Pattern summary includes context
837
+ puts pattern.to_s
838
+ #=> "Pattern(conditions=3, freq=12, gain=11.5%, success=75.0%) [bull months:10,11,12 technology]"
351
839
  ```
352
840
 
353
- `results` is an Array with an entry for each strategy executed. The entries are either :buy, :sell or :hold.
841
+ **Context Metadata:**
842
+ - **Market regime**: Bull/bear/sideways classification
843
+ - **Seasonality**: Valid months/quarters
844
+ - **Sector**: Industry classification
845
+ - **Volatility regime**: Low/medium/high
846
+ - **Discovery period**: When pattern was found
847
+ - **Stability score**: Consistency over time
354
848
 
355
- Currently the strategies are executed sequentially so the results can easily be mapped back to which strategy produced which result. In the future that will change so that the strategies are executed concurrently. When that change is introduced the entries in the `results` object will change -- most likely to an Array of Hashes.
849
+ **Key Benefits:**
850
+ - Patterns aren't universal - they're context-specific
851
+ - Avoid using patterns in wrong market conditions
852
+ - Seasonal awareness (Q4 retail patterns, etc.)
853
+ - Sector-specific strategies
854
+ - Runtime validation before execution
356
855
 
357
- Any specific strategy may not work on every stock. Using the historical data, it is possible to see which strategy works better for a specific stock. **Of course the statistical motto is that historical performance is not a fail-proof indicator for future performance.**
856
+ ## Interactive Console
358
857
 
359
- The strategies that come with the `SQA::Strategy` class are examples only. Its expected that you will come up with your own. If you do, consider sharing them.
858
+ Launch an interactive Ruby console with SQA loaded:
360
859
 
361
- ## Included Program Examples
860
+ ```bash
861
+ sqa-console
862
+ ```
362
863
 
363
- <!-- TODO: What is the name of these things? From the help text they are called commands. They are treated in the code base as a class under the SQA module. Its easy to change the CLI help text to call these programs rather than commands. -->
864
+ This opens IRB with the SQA library pre-loaded, allowing you to experiment interactively:
364
865
 
365
- There are at least two included programs that make use of the `SQA` library.
866
+ ```ruby
867
+ # Already loaded: require 'sqa'
366
868
 
367
- <!-- TODO: These kinds of things need to be in their own repository. -->
869
+ SQA.init
870
+ stock = SQA::Stock.new(ticker: 'MSFT')
871
+ prices = stock.df["close_price"].to_a
872
+ sma = SQAI.sma(prices, period: 20)
873
+ ```
368
874
 
369
- <!-- TODO: Need to complete the API for invoking one of these "programs" from the sqa cli. -->
875
+ ## Examples
370
876
 
371
- ### Analysis
877
+ The `examples/` directory contains comprehensive demonstrations of advanced features:
372
878
 
373
- Does an analysis of a portfollio of stocks.
879
+ - **`genetic_programming_example.rb`** - Evolve RSI parameters through natural selection
880
+ - **`kbs_strategy_example.rb`** - Build rule-based trading systems with RETE
881
+ - **`realtime_stream_example.rb`** - Process live price streams with callbacks
882
+ - **`strategy_generator_example.rb`** - Mine profitable patterns from history
374
883
 
375
- ### Web
884
+ Run examples:
885
+ ```bash
886
+ ruby examples/genetic_programming_example.rb
887
+ ruby examples/strategy_generator_example.rb
888
+ ```
376
889
 
377
- Provides a browser-based interface to some stuff.
890
+ See `examples/README.md` for detailed documentation and integration patterns.
378
891
 
379
- ## Predicted FAQ
892
+ ## Architecture
380
893
 
381
- What is the purpose of the SQA project? The madbomber/sqa project is a set of tools for running technical analysis on a stock portfolio. It's intended as a learning tool to help understand stock market analysis. It's not intended for real-world trading or financial use.
894
+ **Core Components:**
382
895
 
383
- How do I install and use SQA? You can install the gem by running gem install sqa or adding gem 'sqa' to your Gemfile and running bundle install. The project can be used from the command line or as a library in your own application. More details can be found in the README.md file.
896
+ - **`SQA::DataFrame`** - High-performance Polars-based data container for time series
897
+ - **`SQA::Stock`** - Represents a stock with price history and metadata
898
+ - **`SQA::Ticker`** - Stock symbol validation and lookup
899
+ - **`SQA::Strategy`** - Trading strategy execution framework
900
+ - **`SQA::Config`** - Configuration management
901
+ - **`SQAI`** - Alias for `SQA::TAI` (technical indicators from sqa-tai gem)
384
902
 
385
- What technical indicators does SQA support? The project includes a variety of technical indicators such as Average True Range, Bollinger Bands, Candlestick Patterns, Donchian Channel, Double Top Double Bottom Pattern, Exponential Moving Average, Fibonacci Retracement, Head and Shoulders Pattern, Elliott Wave Theory, Market Profile Analysis, Mean Reversion, Momentum, Moving Average Convergence Divergence, Peaks and Valleys, Predict Next Value, Relative Strength Index, Simple Moving Average, and Stochastic Oscillator.
903
+ **Advanced Components:**
386
904
 
387
- What data sources does SQA support? The project supports data from AlphaVantage and Yahoo Finance. You will need an API key for AlphaVantage to use this functionality.
905
+ - **`SQA::Portfolio`** - Position and trade tracking with P&L calculations
906
+ - **`SQA::Backtest`** - Strategy simulation with comprehensive metrics
907
+ - **`SQA::StrategyGenerator`** - Pattern mining from profitable historical trades
908
+ - **`SQA::GeneticProgram`** - Evolutionary algorithm for parameter optimization
909
+ - **`SQA::Strategy::KBS`** - RETE-based forward-chaining inference engine
910
+ - **`SQA::Stream`** - Real-time price stream processor with event callbacks
388
911
 
389
- What are the risks of using SQA for real-world trading? The madbomber/sqa project is intended as a learning tool and is not reliable for any kind of mission-critical financial use. The BUY/SELL signals that it generates should not be taken seriously. If you lose money in the stock market as a result of using this library, the project is not responsible.
912
+ **Data Flow:**
390
913
 
391
- Can I contribute to the SQA project? Yes, contributions are welcome. You can submit bug reports and pull requests on the GitHub repository.
914
+ 1. Create `SQA::Stock` with ticker symbol
915
+ 2. Stock fetches data from Alpha Vantage or Yahoo Finance
916
+ 3. Data stored in Polars-based `SQA::DataFrame`
917
+ 4. Apply technical indicators via `SQAI` / `SQA::TAI`
918
+ 5. Execute trading strategies to generate signals
919
+ 6. Analyze results with statistical functions
392
920
 
393
- What license does SQA use? The project is available as open source under the terms of the MIT License.
921
+ **Design Patterns:**
394
922
 
395
- What is the current state of the project? The project is a work in progress. The developer is experimenting with different gems to support various functionalities. Some features may not be fully implemented or may need further refinement.
923
+ - Plugin architecture for indicators and strategies
924
+ - Data source abstraction (Alpha Vantage, Yahoo Finance)
925
+ - Delegation to Polars for DataFrame operations
926
+ - Configuration hierarchy: defaults < environment variables < config file
396
927
 
928
+ ## Data Sources
397
929
 
398
- ## Other Similar Projects
930
+ ### Alpha Vantage
399
931
 
400
- There are several other (prehaps more mature) projects and libraries that provide similar capabilities to the SQA. Here are a few examples:
932
+ **Recommended data source** with a well-documented API.
401
933
 
402
- TA-Lib (Technical Analysis Library): This is a popular open-source software library that provides tools for technical analysis of financial markets. It includes over 150 functions for various types of analysis including pattern recognition, moving averages, oscillators, and more. It's available in several programming languages including Python, Java, Perl, and more. TA-Lib
934
+ - **URL:** [https://www.alphavantage.co/](https://www.alphavantage.co/)
935
+ - **API Key:** Required (free tier available)
936
+ - **Environment Variable:** `AV_API_KEY` or `ALPHAVANTAGE_API_KEY`
937
+ - **Rate Limits:** 5 calls/minute, 100 calls/day (free tier)
403
938
 
404
- Pandas TA: This is a Python library that provides comprehensive functionalities for technical analysis. It's an extension of the popular data analysis library Pandas and includes a wide range of financial indicators. Pandas TA
939
+ ```ruby
940
+ stock = SQA::Stock.new(ticker: 'GOOGL', source: :alpha_vantage)
941
+ ```
405
942
 
406
- Backtrader: This is a Python library for backtesting trading strategies. It supports a wide range of trading concepts including trading calendars, multiple data feeds, and order execution types. Backtrader
943
+ ### Yahoo Finance
407
944
 
408
- PyAlgoTrade: This is another Python library for backtesting stock trading strategies. It supports Yahoo! Finance, Google Finance, and others as data sources, and has a focus on simplicity and flexibility. PyAlgoTrade
945
+ **No API available** - uses web scraping for historical data.
409
946
 
410
- QuantLib: This is a comprehensive software framework for quantitative finance, it provides tools for many aspects of quantitative finance including trading and risk management, financial instruments, mathematics, numerical methods, and model calibration. QuantLib
947
+ - **URL:** [https://finance.yahoo.com/](https://finance.yahoo.com/)
948
+ - **Manual Download:** Download CSV files and place in `SQA.data_dir`
949
+ - **Filename Format:** `ticker.csv` (lowercase), e.g., `aapl.csv`
411
950
 
412
- zipline: This is a Python library for algorithmic trading. It allows strategy testing and supports live-trading and backtesting, and includes a number of financial computations to aid in trading decisions. zipline
951
+ To manually download:
952
+ 1. Visit [https://finance.yahoo.com/quote/AAPL/history?p=AAPL](https://finance.yahoo.com/quote/AAPL/history?p=AAPL)
953
+ 2. Download historical data as CSV
954
+ 3. Move to your data directory as `aapl.csv`
413
955
 
956
+ ```ruby
957
+ stock = SQA::Stock.new(ticker: 'AAPL', source: :yahoo_finance)
958
+ ```
414
959
 
415
960
  ## Contributing
416
961
 
417
- I can always use some help on this stuff. Got an idea for a new indicator or strategy? Want to improve the math? Make the signals better? Let's collaborate!
962
+ Contributions are welcome! Got an idea for a new indicator or strategy? Want to improve the math or signals?
418
963
 
419
- Bug reports and pull requests are welcome on GitHub at https://github.com/MadBomber/sqa.
964
+ - **Bug reports and pull requests:** [https://github.com/MadBomber/sqa](https://github.com/MadBomber/sqa)
965
+ - **Technical indicators:** Contribute to [sqa-tai](https://github.com/MadBomber/sqa-tai)
420
966
 
421
967
  ## License
422
968
 
423
969
  The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
970
+
971
+ ---
972
+
973
+ **Remember:** This is an educational tool. Historical performance is not an indicator of future results. Never use this for real trading decisions.