sqa 0.0.24 → 0.0.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.goose/memory/development.txt +3 -0
- data/.semver +6 -0
- data/ARCHITECTURE.md +648 -0
- data/CHANGELOG.md +82 -0
- data/CLAUDE.md +653 -0
- data/COMMITS.md +196 -0
- data/DATAFRAME_ARCHITECTURE_REVIEW.md +421 -0
- data/NEXT-STEPS.md +154 -0
- data/README.md +812 -262
- data/TASKS.md +358 -0
- data/TEST_RESULTS.md +140 -0
- data/TODO.md +42 -0
- data/_notes.txt +25 -0
- data/bin/sqa-console +11 -0
- data/data/talk_talk.json +103284 -0
- data/develop_summary.md +313 -0
- data/docs/advanced/backtesting.md +206 -0
- data/docs/advanced/ensemble.md +68 -0
- data/docs/advanced/fpop.md +153 -0
- data/docs/advanced/index.md +112 -0
- data/docs/advanced/multi-timeframe.md +67 -0
- data/docs/advanced/pattern-matcher.md +75 -0
- data/docs/advanced/portfolio-optimizer.md +79 -0
- data/docs/advanced/portfolio.md +166 -0
- data/docs/advanced/risk-management.md +210 -0
- data/docs/advanced/strategy-generator.md +158 -0
- data/docs/advanced/streaming.md +209 -0
- data/docs/ai_and_ml.md +80 -0
- data/docs/api/dataframe.md +1115 -0
- data/docs/api/index.md +126 -0
- data/docs/assets/css/custom.css +88 -0
- data/docs/assets/js/mathjax.js +18 -0
- data/docs/concepts/index.md +68 -0
- data/docs/contributing/index.md +60 -0
- data/docs/data-sources/index.md +66 -0
- data/docs/data_frame.md +317 -97
- data/docs/factors_that_impact_price.md +26 -0
- data/docs/finviz.md +11 -0
- data/docs/fx_pro_bit.md +25 -0
- data/docs/genetic_programming.md +104 -0
- data/docs/getting-started/index.md +123 -0
- data/docs/getting-started/installation.md +229 -0
- data/docs/getting-started/quick-start.md +244 -0
- data/docs/i_gotta_an_idea.md +22 -0
- data/docs/index.md +163 -0
- data/docs/indicators/index.md +97 -0
- data/docs/indicators.md +110 -24
- data/docs/options.md +8 -0
- data/docs/strategies/bollinger-bands.md +146 -0
- data/docs/strategies/consensus.md +64 -0
- data/docs/strategies/custom.md +310 -0
- data/docs/strategies/ema.md +53 -0
- data/docs/strategies/index.md +92 -0
- data/docs/strategies/kbs.md +164 -0
- data/docs/strategies/macd.md +96 -0
- data/docs/strategies/market-profile.md +54 -0
- data/docs/strategies/mean-reversion.md +58 -0
- data/docs/strategies/rsi.md +95 -0
- data/docs/strategies/sma.md +55 -0
- data/docs/strategies/stochastic.md +63 -0
- data/docs/strategies/volume-breakout.md +54 -0
- data/docs/tags.md +7 -0
- data/docs/true_strength_index.md +46 -0
- data/docs/weighted_moving_average.md +48 -0
- data/examples/README.md +354 -0
- data/examples/advanced_features_example.rb +350 -0
- data/examples/fpop_analysis_example.rb +191 -0
- data/examples/genetic_programming_example.rb +148 -0
- data/examples/kbs_strategy_example.rb +208 -0
- data/examples/pattern_context_example.rb +300 -0
- data/examples/rails_app/Gemfile +34 -0
- data/examples/rails_app/README.md +416 -0
- data/examples/rails_app/app/assets/javascripts/application.js +107 -0
- data/examples/rails_app/app/assets/stylesheets/application.css +659 -0
- data/examples/rails_app/app/controllers/analysis_controller.rb +11 -0
- data/examples/rails_app/app/controllers/api/v1/stocks_controller.rb +227 -0
- data/examples/rails_app/app/controllers/application_controller.rb +22 -0
- data/examples/rails_app/app/controllers/backtest_controller.rb +11 -0
- data/examples/rails_app/app/controllers/dashboard_controller.rb +21 -0
- data/examples/rails_app/app/controllers/portfolio_controller.rb +7 -0
- data/examples/rails_app/app/views/analysis/show.html.erb +209 -0
- data/examples/rails_app/app/views/backtest/show.html.erb +171 -0
- data/examples/rails_app/app/views/dashboard/index.html.erb +118 -0
- data/examples/rails_app/app/views/dashboard/show.html.erb +408 -0
- data/examples/rails_app/app/views/errors/show.html.erb +17 -0
- data/examples/rails_app/app/views/layouts/application.html.erb +60 -0
- data/examples/rails_app/app/views/portfolio/index.html.erb +33 -0
- data/examples/rails_app/bin/rails +6 -0
- data/examples/rails_app/config/application.rb +45 -0
- data/examples/rails_app/config/boot.rb +5 -0
- data/examples/rails_app/config/database.yml +18 -0
- data/examples/rails_app/config/environment.rb +11 -0
- data/examples/rails_app/config/routes.rb +26 -0
- data/examples/rails_app/config.ru +8 -0
- data/examples/realtime_stream_example.rb +274 -0
- data/examples/sinatra_app/Gemfile +22 -0
- data/examples/sinatra_app/QUICKSTART.md +159 -0
- data/examples/sinatra_app/README.md +461 -0
- data/examples/sinatra_app/app.rb +344 -0
- data/examples/sinatra_app/config.ru +5 -0
- data/examples/sinatra_app/public/css/style.css +659 -0
- data/examples/sinatra_app/public/js/app.js +107 -0
- data/examples/sinatra_app/views/analyze.erb +306 -0
- data/examples/sinatra_app/views/backtest.erb +325 -0
- data/examples/sinatra_app/views/dashboard.erb +419 -0
- data/examples/sinatra_app/views/error.erb +58 -0
- data/examples/sinatra_app/views/index.erb +118 -0
- data/examples/sinatra_app/views/layout.erb +61 -0
- data/examples/sinatra_app/views/portfolio.erb +43 -0
- data/examples/strategy_generator_example.rb +346 -0
- data/hsa_portfolio.csv +11 -0
- data/justfile +0 -0
- data/lib/api/alpha_vantage_api.rb +462 -0
- data/lib/sqa/backtest.rb +329 -0
- data/lib/sqa/data_frame/alpha_vantage.rb +43 -65
- data/lib/sqa/data_frame/data.rb +92 -0
- data/lib/sqa/data_frame/yahoo_finance.rb +35 -43
- data/lib/sqa/data_frame.rb +148 -243
- data/lib/sqa/ensemble.rb +359 -0
- data/lib/sqa/fpop.rb +199 -0
- data/lib/sqa/gp.rb +259 -0
- data/lib/sqa/indicator.rb +5 -8
- data/lib/sqa/init.rb +15 -8
- data/lib/sqa/market_regime.rb +240 -0
- data/lib/sqa/multi_timeframe.rb +379 -0
- data/lib/sqa/pattern_matcher.rb +497 -0
- data/lib/sqa/portfolio.rb +260 -6
- data/lib/sqa/portfolio_optimizer.rb +377 -0
- data/lib/sqa/risk_manager.rb +442 -0
- data/lib/sqa/seasonal_analyzer.rb +209 -0
- data/lib/sqa/sector_analyzer.rb +300 -0
- data/lib/sqa/stock.rb +67 -125
- data/lib/sqa/strategy/bollinger_bands.rb +42 -0
- data/lib/sqa/strategy/consensus.rb +5 -2
- data/lib/sqa/strategy/kbs_strategy.rb +470 -0
- data/lib/sqa/strategy/macd.rb +46 -0
- data/lib/sqa/strategy/mp.rb +1 -1
- data/lib/sqa/strategy/stochastic.rb +60 -0
- data/lib/sqa/strategy/volume_breakout.rb +57 -0
- data/lib/sqa/strategy.rb +5 -0
- data/lib/sqa/strategy_generator.rb +947 -0
- data/lib/sqa/stream.rb +361 -0
- data/lib/sqa/version.rb +1 -7
- data/lib/sqa.rb +23 -16
- data/main.just +81 -0
- data/mkdocs.yml +288 -0
- data/trace.log +0 -0
- metadata +261 -51
- data/bin/sqa +0 -6
- data/lib/patches/dry-cli.rb +0 -228
- data/lib/sqa/activity.rb +0 -10
- data/lib/sqa/cli.rb +0 -62
- data/lib/sqa/commands/analysis.rb +0 -309
- data/lib/sqa/commands/base.rb +0 -139
- data/lib/sqa/commands/web.rb +0 -199
- data/lib/sqa/commands.rb +0 -22
- data/lib/sqa/constants.rb +0 -23
- data/lib/sqa/indicator/average_true_range.rb +0 -33
- data/lib/sqa/indicator/bollinger_bands.rb +0 -28
- data/lib/sqa/indicator/candlestick_pattern_recognizer.rb +0 -60
- data/lib/sqa/indicator/donchian_channel.rb +0 -29
- data/lib/sqa/indicator/double_top_bottom_pattern.rb +0 -34
- data/lib/sqa/indicator/elliott_wave_theory.rb +0 -57
- data/lib/sqa/indicator/exponential_moving_average.rb +0 -25
- data/lib/sqa/indicator/exponential_moving_average_trend.rb +0 -36
- data/lib/sqa/indicator/fibonacci_retracement.rb +0 -23
- data/lib/sqa/indicator/head_and_shoulders_pattern.rb +0 -26
- data/lib/sqa/indicator/market_profile.rb +0 -32
- data/lib/sqa/indicator/mean_reversion.rb +0 -37
- data/lib/sqa/indicator/momentum.rb +0 -28
- data/lib/sqa/indicator/moving_average_convergence_divergence.rb +0 -29
- data/lib/sqa/indicator/peaks_and_valleys.rb +0 -29
- data/lib/sqa/indicator/predict_next_value.rb +0 -202
- data/lib/sqa/indicator/relative_strength_index.rb +0 -47
- data/lib/sqa/indicator/simple_moving_average.rb +0 -24
- data/lib/sqa/indicator/simple_moving_average_trend.rb +0 -32
- data/lib/sqa/indicator/stochastic_oscillator.rb +0 -68
- data/lib/sqa/indicator/true_range.rb +0 -39
- data/lib/sqa/trade.rb +0 -26
data/docs/data_frame.md
CHANGED
|
@@ -1,164 +1,384 @@
|
|
|
1
|
-
# DataFrame
|
|
1
|
+
# DataFrame Documentation
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Overview
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The `SQA::DataFrame` class is a high-performance wrapper around the Polars DataFrame library, specifically designed for time series financial data manipulation. Polars is a Rust-backed library that provides blazingly fast operations on columnar data.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Architecture
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
* https://github.com/SciRuby
|
|
9
|
+
The DataFrame system consists of two main components:
|
|
11
10
|
|
|
12
|
-
|
|
11
|
+
### 1. SQA::DataFrame
|
|
13
12
|
|
|
14
|
-
|
|
13
|
+
The main DataFrame class that wraps Polars::DataFrame with SQA-specific convenience methods.
|
|
15
14
|
|
|
16
|
-
|
|
15
|
+
**Location**: `lib/sqa/data_frame.rb`
|
|
17
16
|
|
|
18
|
-
|
|
17
|
+
**Key Features**:
|
|
18
|
+
- Wraps Polars::DataFrame for high-performance operations
|
|
19
|
+
- Column-based vectorized operations (avoid row iterations)
|
|
20
|
+
- CSV and JSON import/export
|
|
21
|
+
- Automatic column renaming and transformations
|
|
22
|
+
- FPL (Future Period Loss/Profit) analysis convenience methods
|
|
23
|
+
- Method delegation to underlying Polars DataFrame
|
|
19
24
|
|
|
20
|
-
|
|
25
|
+
### 2. SQA::DataFrame::Data
|
|
21
26
|
|
|
22
|
-
|
|
27
|
+
A metadata storage class for stock information, separate from the price/volume data.
|
|
28
|
+
|
|
29
|
+
**Location**: `lib/sqa/data_frame/data.rb`
|
|
30
|
+
|
|
31
|
+
**Attributes**:
|
|
32
|
+
- `ticker` - Stock symbol (e.g., 'AAPL', 'MSFT')
|
|
33
|
+
- `name` - Company name
|
|
34
|
+
- `exchange` - Exchange symbol (NASDAQ, NYSE, etc.)
|
|
35
|
+
- `source` - Data source (`:alpha_vantage`, `:yahoo_finance`)
|
|
36
|
+
- `indicators` - Technical indicators configuration hash
|
|
37
|
+
- `overview` - Company overview data from Alpha Vantage
|
|
38
|
+
|
|
39
|
+
**Key Features**:
|
|
40
|
+
- Dual initialization: from hash (JSON) or keyword arguments
|
|
41
|
+
- JSON serialization with `to_json`
|
|
42
|
+
- Used by `SQA::Stock` to persist metadata in `~/sqa_data/ticker.json`
|
|
43
|
+
- All attributes are read/write accessible
|
|
44
|
+
|
|
45
|
+
## Creating DataFrames
|
|
46
|
+
|
|
47
|
+
### From Data Sources
|
|
23
48
|
|
|
24
49
|
```ruby
|
|
25
|
-
|
|
50
|
+
# Using Alpha Vantage
|
|
51
|
+
stock = SQA::Stock.new(ticker: 'AAPL')
|
|
52
|
+
df = stock.df # SQA::DataFrame instance
|
|
26
53
|
|
|
27
|
-
#
|
|
28
|
-
|
|
29
|
-
df =
|
|
54
|
+
# Using Yahoo Finance
|
|
55
|
+
stock = SQA::Stock.new(ticker: 'MSFT', source: :yahoo_finance)
|
|
56
|
+
df = stock.df
|
|
30
57
|
```
|
|
31
58
|
|
|
32
|
-
|
|
59
|
+
### From CSV File
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
# Load CSV file (Polars-compatible)
|
|
63
|
+
df = SQA::DataFrame.from_csv_file('path/to/stock_data.csv')
|
|
33
64
|
|
|
34
|
-
|
|
65
|
+
# The underlying Polars DataFrame is accessible via .data
|
|
66
|
+
polars_df = df.data
|
|
67
|
+
```
|
|
35
68
|
|
|
36
|
-
|
|
69
|
+
### From JSON File
|
|
37
70
|
|
|
38
71
|
```ruby
|
|
39
|
-
|
|
40
|
-
|
|
72
|
+
# Load JSON array of hashes
|
|
73
|
+
df = SQA::DataFrame.from_json_file('path/to/stock_data.json')
|
|
41
74
|
```
|
|
42
75
|
|
|
43
|
-
|
|
76
|
+
### From Array of Hashes
|
|
44
77
|
|
|
45
78
|
```ruby
|
|
46
|
-
|
|
47
|
-
|
|
79
|
+
data = [
|
|
80
|
+
{ date: '2024-01-01', close: 150.0, volume: 1_000_000 },
|
|
81
|
+
{ date: '2024-01-02', close: 152.0, volume: 1_200_000 }
|
|
82
|
+
]
|
|
48
83
|
|
|
84
|
+
df = SQA::DataFrame.from_aofh(data)
|
|
49
85
|
```
|
|
50
86
|
|
|
51
|
-
|
|
87
|
+
## Working with DataFrames
|
|
88
|
+
|
|
89
|
+
### Column Access
|
|
52
90
|
|
|
53
91
|
```ruby
|
|
54
|
-
|
|
55
|
-
|
|
92
|
+
# Get column names
|
|
93
|
+
df.columns
|
|
94
|
+
# => ["timestamp", "open", "high", "low", "close", "adj_close_price", "volume"]
|
|
95
|
+
|
|
96
|
+
# Access a column (returns Polars::Series)
|
|
97
|
+
close_prices = df["close"]
|
|
98
|
+
|
|
99
|
+
# Convert to Ruby array
|
|
100
|
+
prices_array = df["adj_close_price"].to_a
|
|
56
101
|
```
|
|
57
102
|
|
|
58
|
-
|
|
103
|
+
### Basic Operations
|
|
104
|
+
|
|
105
|
+
```ruby
|
|
106
|
+
# Get dimensions
|
|
107
|
+
df.size # Number of rows (also: df.nrows, df.length)
|
|
108
|
+
df.ncols # Number of columns
|
|
109
|
+
|
|
110
|
+
# Get first/last rows
|
|
111
|
+
df.head(10) # First 10 rows
|
|
112
|
+
df.tail(10) # Last 10 rows
|
|
113
|
+
|
|
114
|
+
# Column statistics (via Polars)
|
|
115
|
+
df.data["close"].mean
|
|
116
|
+
df.data["volume"].sum
|
|
117
|
+
df.data["close"].min
|
|
118
|
+
df.data["close"].max
|
|
119
|
+
```
|
|
59
120
|
|
|
60
|
-
|
|
121
|
+
### Data Transformation
|
|
61
122
|
|
|
62
123
|
```ruby
|
|
63
|
-
|
|
64
|
-
|
|
124
|
+
# Rename columns
|
|
125
|
+
mapping = { 'Close' => :close, 'Volume' => :volume }
|
|
126
|
+
df.rename_columns!(mapping)
|
|
127
|
+
|
|
128
|
+
# Apply transformers
|
|
129
|
+
transformers = {
|
|
130
|
+
date: ->(val) { Date.parse(val) },
|
|
131
|
+
volume: ->(val) { val.to_i }
|
|
132
|
+
}
|
|
133
|
+
df.apply_transformers!(transformers)
|
|
134
|
+
```
|
|
65
135
|
|
|
66
|
-
|
|
67
|
-
#=> {}
|
|
136
|
+
### Appending DataFrames
|
|
68
137
|
|
|
69
|
-
|
|
138
|
+
```ruby
|
|
139
|
+
# Combine two DataFrames
|
|
140
|
+
df1.append!(df2) # Modifies df1 in place
|
|
141
|
+
df1.concat!(df2) # Alias for append!
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
### Export
|
|
145
|
+
|
|
146
|
+
```ruby
|
|
147
|
+
# To CSV
|
|
148
|
+
df.to_csv('output.csv')
|
|
149
|
+
df.write_csv('output.csv') # Alias
|
|
70
150
|
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
{
|
|
74
|
-
|
|
75
|
-
"High"=>:high,
|
|
76
|
-
"Low"=>:low,
|
|
77
|
-
"Close"=>:close,
|
|
78
|
-
"Adj Close"=>:adj_close,
|
|
79
|
-
"Volume"=>:volume}
|
|
151
|
+
# To Hash
|
|
152
|
+
hash = df.to_h
|
|
153
|
+
# => { close: [150.0, 152.0, ...], volume: [1_000_000, 1_200_000, ...] }
|
|
154
|
+
```
|
|
80
155
|
|
|
156
|
+
## FPL Analysis Methods
|
|
81
157
|
|
|
82
|
-
|
|
83
|
-
#=> #<Daru::Index(7): {date, open, high, low, close, adj_close, volume}>
|
|
158
|
+
The DataFrame includes convenience methods for Future Period Loss/Profit analysis:
|
|
84
159
|
|
|
85
|
-
|
|
86
|
-
#=> #<Daru::Index(7): {date, open, high, low, close, adj_close, volume}>
|
|
160
|
+
### Basic FPL Calculation
|
|
87
161
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
volume
|
|
93
|
-
10741 45377800
|
|
94
|
-
10742 37283200
|
|
95
|
-
10743 47471900
|
|
96
|
-
10744 47460200
|
|
97
|
-
10745 48291400
|
|
98
|
-
10746 38824100
|
|
99
|
-
10747 35175100
|
|
100
|
-
10748 50389300
|
|
101
|
-
10749 61235200
|
|
102
|
-
10750 115799700
|
|
103
|
-
10751 97576100
|
|
104
|
-
10752 67823000
|
|
105
|
-
10753 60378500
|
|
106
|
-
10754 54628800
|
|
162
|
+
```ruby
|
|
163
|
+
# Calculate min/max future deltas for each point
|
|
164
|
+
fpl_data = df.fpl(column: "adj_close_price", fpop: 10)
|
|
165
|
+
# => [[min_delta, max_delta], [min_delta, max_delta], ...]
|
|
107
166
|
```
|
|
108
167
|
|
|
168
|
+
### Comprehensive FPL Analysis
|
|
109
169
|
|
|
170
|
+
```ruby
|
|
171
|
+
# Get detailed analysis with risk metrics
|
|
172
|
+
analysis = df.fpl_analysis(column: "adj_close_price", fpop: 10)
|
|
173
|
+
# => [
|
|
174
|
+
# {
|
|
175
|
+
# min_delta: -2.5,
|
|
176
|
+
# max_delta: 5.3,
|
|
177
|
+
# magnitude: 3.9,
|
|
178
|
+
# risk: 7.8,
|
|
179
|
+
# direction: :UP
|
|
180
|
+
# },
|
|
181
|
+
# ...
|
|
182
|
+
# ]
|
|
183
|
+
```
|
|
110
184
|
|
|
185
|
+
See [FPL Analysis Documentation](advanced/fpop.md) for more details.
|
|
111
186
|
|
|
187
|
+
## Working with Stock Metadata
|
|
112
188
|
|
|
113
|
-
|
|
189
|
+
### Creating Metadata
|
|
190
|
+
|
|
191
|
+
```ruby
|
|
192
|
+
# From keyword arguments
|
|
193
|
+
data = SQA::DataFrame::Data.new(
|
|
194
|
+
ticker: 'AAPL',
|
|
195
|
+
source: :alpha_vantage,
|
|
196
|
+
indicators: { rsi: 14, sma: [20, 50] }
|
|
197
|
+
)
|
|
198
|
+
|
|
199
|
+
# From hash (JSON deserialization)
|
|
200
|
+
json_data = JSON.parse(File.read('aapl.json'))
|
|
201
|
+
data = SQA::DataFrame::Data.new(json_data)
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### Accessing and Modifying Metadata
|
|
205
|
+
|
|
206
|
+
```ruby
|
|
207
|
+
# Read attributes
|
|
208
|
+
data.ticker # => 'AAPL'
|
|
209
|
+
data.source # => :alpha_vantage
|
|
210
|
+
data.indicators # => { rsi: 14, sma: [20, 50] }
|
|
211
|
+
|
|
212
|
+
# Write attributes
|
|
213
|
+
data.name = 'Apple Inc.'
|
|
214
|
+
data.exchange = 'NASDAQ'
|
|
215
|
+
data.overview = {
|
|
216
|
+
'company' => 'Apple Inc.',
|
|
217
|
+
'sector' => 'Technology',
|
|
218
|
+
'market_cap' => 2_800_000_000_000
|
|
219
|
+
}
|
|
220
|
+
```
|
|
114
221
|
|
|
115
|
-
|
|
222
|
+
### Persistence
|
|
116
223
|
|
|
117
224
|
```ruby
|
|
118
|
-
|
|
119
|
-
|
|
225
|
+
# Serialize to JSON
|
|
226
|
+
json_string = data.to_json
|
|
120
227
|
|
|
121
|
-
#
|
|
122
|
-
|
|
123
|
-
#=> [177.970001, 196.449997]
|
|
228
|
+
# Save to file (typically done by SQA::Stock)
|
|
229
|
+
File.write('aapl.json', data.to_json)
|
|
124
230
|
|
|
125
|
-
#
|
|
126
|
-
|
|
231
|
+
# Load from file
|
|
232
|
+
json_data = JSON.parse(File.read('aapl.json'))
|
|
233
|
+
data = SQA::DataFrame::Data.new(json_data)
|
|
127
234
|
```
|
|
128
|
-
<pre>
|
|
129
|
-
= Adj Close
|
|
130
|
-
n :14
|
|
131
|
-
non-missing:14
|
|
132
|
-
median: 192.66500100000002
|
|
133
|
-
mean: 188.7521
|
|
134
|
-
std.dev.: 7.4488
|
|
135
|
-
std.err.: 1.9908
|
|
136
|
-
skew: -0.4783
|
|
137
|
-
kurtosis: -1.7267
|
|
138
|
-
</pre>
|
|
139
235
|
|
|
140
|
-
##
|
|
236
|
+
## Performance Tips
|
|
237
|
+
|
|
238
|
+
1. **Use Column Operations**: Always prefer Polars column operations over Ruby loops
|
|
239
|
+
```ruby
|
|
240
|
+
# GOOD: Vectorized operation
|
|
241
|
+
df.data["close"].mean
|
|
242
|
+
|
|
243
|
+
# BAD: Ruby loop
|
|
244
|
+
df["close"].to_a.sum / df.size.to_f
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
2. **Access Underlying Polars**: Use `df.data` for direct Polars operations
|
|
248
|
+
```ruby
|
|
249
|
+
# Direct Polars access
|
|
250
|
+
filtered = df.data.filter(df.data["volume"] > 1_000_000)
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
3. **Avoid Unnecessary Array Conversions**: Only convert to arrays when needed
|
|
254
|
+
```ruby
|
|
255
|
+
# Only convert when passing to external functions
|
|
256
|
+
prices = df["adj_close_price"].to_a
|
|
257
|
+
rsi = SQAI.rsi(prices, period: 14)
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
4. **Batch Operations**: Combine operations when possible
|
|
261
|
+
```ruby
|
|
262
|
+
# Instead of multiple separate operations
|
|
263
|
+
df.apply_transformers!(transformers)
|
|
264
|
+
df.rename_columns!(mapping)
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
## Data Sources
|
|
268
|
+
|
|
269
|
+
### Alpha Vantage
|
|
270
|
+
|
|
271
|
+
**Location**: `lib/sqa/data_frame/alpha_vantage.rb`
|
|
141
272
|
|
|
142
273
|
```ruby
|
|
143
|
-
|
|
274
|
+
SQA::DataFrame::AlphaVantage.recent('AAPL', full: true)
|
|
144
275
|
```
|
|
145
|
-
The best time to buy a stock is subjective and can vary depending on individual goals, investment strategies, and risk tolerance. However, there are a few general principles to consider:
|
|
146
276
|
|
|
147
|
-
|
|
277
|
+
**Requirements**:
|
|
278
|
+
- Environment variable: `AV_API_KEY` or `ALPHAVANTAGE_API_KEY`
|
|
279
|
+
- Rate limiting: 5 calls/minute (free tier)
|
|
148
280
|
|
|
149
|
-
|
|
281
|
+
### Yahoo Finance
|
|
150
282
|
|
|
151
|
-
|
|
283
|
+
**Location**: `lib/sqa/data_frame/yahoo_finance.rb`
|
|
152
284
|
|
|
153
|
-
|
|
285
|
+
```ruby
|
|
286
|
+
SQA::DataFrame::YahooFinance.recent('AAPL', full: true)
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
**Features**:
|
|
290
|
+
- No API key required
|
|
291
|
+
- Web scraping based (less reliable)
|
|
292
|
+
- Good for testing and fallback
|
|
293
|
+
|
|
294
|
+
## Adding New Data Sources
|
|
154
295
|
|
|
155
|
-
|
|
296
|
+
To add a new data source adapter:
|
|
156
297
|
|
|
157
|
-
|
|
298
|
+
1. Create `lib/sqa/data_frame/my_source.rb`
|
|
299
|
+
2. Define class `SQA::DataFrame::MySource`
|
|
300
|
+
3. Implement `self.recent(ticker, **options)` method
|
|
301
|
+
4. Return data in Polars-compatible format
|
|
302
|
+
5. Add column mapping if needed
|
|
158
303
|
|
|
304
|
+
**Example**:
|
|
159
305
|
|
|
160
306
|
```ruby
|
|
161
|
-
|
|
307
|
+
class SQA::DataFrame::MySource
|
|
308
|
+
TRANSFORMERS = {
|
|
309
|
+
timestamp: ->(val) { Date.parse(val) },
|
|
310
|
+
volume: ->(val) { val.to_i }
|
|
311
|
+
}
|
|
312
|
+
|
|
313
|
+
def self.recent(ticker, **options)
|
|
314
|
+
# Fetch data from your source
|
|
315
|
+
raw_data = fetch_from_api(ticker)
|
|
316
|
+
|
|
317
|
+
# Convert to Polars-compatible format
|
|
318
|
+
SQA::DataFrame.new(raw_data, transformers: TRANSFORMERS)
|
|
319
|
+
end
|
|
320
|
+
end
|
|
162
321
|
```
|
|
163
|
-
Consulting the magic eight ball cluster.... The future looks cloudy. You should have bought it 14 days ago when I told you it was on its way up! Do you ever listen to me? No! I slave over these numbers night and day. I consult the best magic eight ball sources available. What do I get for my efforts? Nothing!
|
|
164
322
|
|
|
323
|
+
## Common Gotchas
|
|
324
|
+
|
|
325
|
+
1. **DataFrame vs Polars**:
|
|
326
|
+
- `df` is `SQA::DataFrame`
|
|
327
|
+
- `df.data` is `Polars::DataFrame`
|
|
328
|
+
|
|
329
|
+
2. **Column Names**:
|
|
330
|
+
- Column names are strings, not symbols
|
|
331
|
+
- Use `df["close"]` not `df[:close]`
|
|
332
|
+
|
|
333
|
+
3. **Method Delegation**:
|
|
334
|
+
- Unknown methods are delegated to Polars::DataFrame
|
|
335
|
+
- Check Polars docs for advanced operations
|
|
336
|
+
|
|
337
|
+
4. **Indicators Need Arrays**:
|
|
338
|
+
- Extract data with `.to_a` before passing to SQAI/TAI functions
|
|
339
|
+
- Example: `prices = df["close"].to_a`
|
|
340
|
+
|
|
341
|
+
## Related Documentation
|
|
342
|
+
|
|
343
|
+
- [FPL Analysis](advanced/fpop.md) - Future Period Loss/Profit utilities
|
|
344
|
+
- [Stock Class](api/stock.md) - Using DataFrames with Stock objects
|
|
345
|
+
- [Indicators](indicators/index.md) - Technical indicator integration
|
|
346
|
+
- [Polars Documentation](https://pola-rs.github.io/polars-book/) - Underlying library
|
|
347
|
+
|
|
348
|
+
## Example: Complete Workflow
|
|
349
|
+
|
|
350
|
+
```ruby
|
|
351
|
+
require 'sqa'
|
|
352
|
+
|
|
353
|
+
SQA.init
|
|
354
|
+
|
|
355
|
+
# Load stock data
|
|
356
|
+
stock = SQA::Stock.new(ticker: 'AAPL')
|
|
357
|
+
|
|
358
|
+
# Access DataFrame
|
|
359
|
+
df = stock.df
|
|
360
|
+
|
|
361
|
+
# Get price array for indicators
|
|
362
|
+
prices = df["adj_close_price"].to_a
|
|
363
|
+
|
|
364
|
+
# Calculate technical indicators
|
|
365
|
+
sma_20 = SQAI.sma(prices, period: 20)
|
|
366
|
+
rsi_14 = SQAI.rsi(prices, period: 14)
|
|
367
|
+
|
|
368
|
+
# FPL analysis
|
|
369
|
+
fpl_analysis = df.fpl_analysis(fpop: 10)
|
|
370
|
+
high_quality = SQA::FPOP.filter_by_quality(
|
|
371
|
+
fpl_analysis,
|
|
372
|
+
min_magnitude: 5.0,
|
|
373
|
+
max_risk: 25.0
|
|
374
|
+
)
|
|
375
|
+
|
|
376
|
+
# Access stock metadata
|
|
377
|
+
puts "Ticker: #{stock.ticker}"
|
|
378
|
+
puts "Exchange: #{stock.exchange}"
|
|
379
|
+
puts "Source: #{stock.source}"
|
|
380
|
+
|
|
381
|
+
# Export data
|
|
382
|
+
df.to_csv("aapl_prices.csv")
|
|
383
|
+
File.write("aapl_metadata.json", stock.data.to_json)
|
|
384
|
+
```
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
## Factors that Impact Price
|
|
2
|
+
|
|
3
|
+
By considering these factors, investors can better understand the potential price movements of stocks and make informed investment decisions.
|
|
4
|
+
|
|
5
|
+
This breakdown approximates how heavily each factor might typically weigh on stock prices across a varied portfolio. However, in specific scenarios, the impact of each factor could be dramatically different. For instance, in times of global financial crisis, economic indicators and market sentiment might dramatically outweigh company-specific performance, or during a technological breakthrough, industry trends could be the dominant factor.
|
|
6
|
+
|
|
7
|
+
Ranked most impactful to least impactful.
|
|
8
|
+
|
|
9
|
+
1. **Company Performance**: 35%
|
|
10
|
+
- The financial health and growth prospects of a company are crucial. Positive earnings reports, strong revenue growth, and effective management often directly translate to increased stock prices.
|
|
11
|
+
|
|
12
|
+
2. **Market Sentiment**: 25%
|
|
13
|
+
- This is a broad factor that encompasses the psychological aspects of the market. Positive market sentiment can drive prices up rapidly, sometimes irrespective of underlying fundamentals.
|
|
14
|
+
|
|
15
|
+
3. **Economic Indicators**: 15%
|
|
16
|
+
- Broad economic trends greatly influence investor confidence and expectations. Robust economic growth, for example, tends to buoy most stocks, particularly in sectors closely tied to economic cycles.
|
|
17
|
+
|
|
18
|
+
4. **Interest Rates**: 10%
|
|
19
|
+
- Changes in interest rates by central banks can shift investor preferences between stocks and bonds, thus having a significant impact on stock prices.
|
|
20
|
+
|
|
21
|
+
5. **Geopolitical Events**: 10%
|
|
22
|
+
- These events can create considerable market volatility and can impact specific sectors more intensely (e.g., energy stocks during oil crises).
|
|
23
|
+
|
|
24
|
+
6. **Industry Trends**: 5%
|
|
25
|
+
- While these can be highly impactful on specific stocks or sectors, their broader impact on stock prices across the market tends to be more contained compared to the other factors unless the trend sparks a sector-wide shift.
|
|
26
|
+
|
data/docs/finviz.md
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# Financial Vizualization (finviz)
|
|
2
|
+
|
|
3
|
+
The original finviz unofficial gem has been untouched for 3 years. I just forked it in the hope that I can bring it forward to current versions of its dependencies.
|
|
4
|
+
|
|
5
|
+
https;//finviz.com
|
|
6
|
+
|
|
7
|
+
## AAPL weekly
|
|
8
|
+
|
|
9
|
+
The default is daily.
|
|
10
|
+
|
|
11
|
+
https://finviz.com/quote.ashx?t=AAPL&p=w
|
data/docs/fx_pro_bit.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# FXProBot
|
|
2
|
+
|
|
3
|
+
FXProBot is a new trading software by Avenix Fzco, a major fintech firm located in Dubai that specializes in automated trading. This advanced robot has been built with the sole aim of improving the experience of both established and novice traders by using sophisticated algorithms and strong risk management approaches. The goal of FXProBot is to change how investors interact with the market; it offers an efficient and effortless platform for forex traders.
|
|
4
|
+
|
|
5
|
+
## Main Features of FXProBot
|
|
6
|
+
|
|
7
|
+
Impulse Detection and Analysis – One of the key attributes of this bot is its well-crafted impulse detection and analysis process. Before initiating any trade, this forex robot scans market impulses, searching for ones that may benefit traders. In instances such as when Doji candles appear or ambiguous market behavior occurs, this feature helps avoid common mistakes.
|
|
8
|
+
|
|
9
|
+
**Trend Conformity** – This feature comes into play when market impulses are aligned with broader market trends. Trade entry accuracy is enhanced by FXProBot’s built-in indicators. In this way, it increases the chances for better trading opportunities, since FXProBot trades in line with prevailing market trends.
|
|
10
|
+
|
|
11
|
+
**Robust Risk Management** – Successful trading necessitates good risk management, which FXProBot demonstrates excellently. Each trade executed by this forex robot incorporates predefined levels for Stop Loss (SL) and Take Profit (TP). This strong framework used for risk management promotes minimal losses, thus safeguarding profits and investments from instability.
|
|
12
|
+
|
|
13
|
+
**Automated Trading** – Trading has become simpler thanks to automated trading offered by FXProBot. This Expert Advisor (EA) uses complex algorithms, ensuring accuracy as it removes emotions and psychological stress from trading decisions. This means that traders can focus on their strategies while FXProBot takes care of execution.
|
|
14
|
+
|
|
15
|
+
**Customizable Settings** – In order to accommodate different trading styles, FXProBot is highly customizable. This allows users to modify the EA’s risk level, trade sizes, and select between numerous indicators available, thereby fine-tuning it to fit any individual preferences.
|
|
16
|
+
|
|
17
|
+
**Backtesting and Optimization** – Backtesting as well as optimization are offered by FXProBot. Traders are able to put their strategies to the test through extensive historical data prior to applying them in live markets. These techniques help improve how trades should be executed, thereby increasing the chances of success.
|
|
18
|
+
|
|
19
|
+
**Regular Updates and Comprehensive Support** – Avenix Fzco constantly aims at improving its creations; FXProBot is regularly updated with market insights and new algorithms for better strategy applications towards changing market conditions. Also, within all stages of one’s trading journey with FXProBot, he or she will have a dedicated support team right from installation up to troubleshooting.
|
|
20
|
+
|
|
21
|
+
**Educational Resources** – In addition, FXProBot provides a wide range of educational resources such as tutorials, webinars, and articles on basic forex concepts and advanced trading methods. These resources enable one to make informed decisions while participating in forex-related activities.
|
|
22
|
+
|
|
23
|
+
## About Avenix Fzco
|
|
24
|
+
|
|
25
|
+
Avenix Fzco, a fintech pioneer in Dubai, UAE, is focused on creating high-level trading software for forex traders. The company’s most recent release, FXProBot, shows its dedication to precision, risk management, and intelligent operations. For more details about FXProBot and how it can improve the trading experience, please visit https://fxprobot.com/.
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
### Overview of Genetic Programming (GP)
|
|
2
|
+
|
|
3
|
+
Genetic Programming (GP) is an evolutionary algorithm-inspired methodology used to evolve programs or models, particularly to solve optimization and search problems. GP simulates the process of natural selection by treating potential solutions as members of a population of programs. These programs are encoded as trees, where the nodes represent operations or functions and the leaves represent input data. During the GP process, populations evolve over generations using genetic operations such as mutation, crossover (recombination), and selection according to their fitness.
|
|
4
|
+
|
|
5
|
+
#### definitions
|
|
6
|
+
##### Single Signal with Varying Parameters
|
|
7
|
+
|
|
8
|
+
prices: historical price data for a stock (Array of Hash)
|
|
9
|
+
pww: past window width (Integer)
|
|
10
|
+
fww: future window width (Integer)
|
|
11
|
+
gain: (Float)
|
|
12
|
+
loss: (Float)
|
|
13
|
+
indicator: historical signal values (Float)
|
|
14
|
+
signal: buy/sell/hold (Symbol)
|
|
15
|
+
x: today - instance in time (Integer)
|
|
16
|
+
|
|
17
|
+
Different fitness functions for buy/sell/hold
|
|
18
|
+
|
|
19
|
+
pwi = (x-pww .. x)
|
|
20
|
+
fwi = (x+1 .. x+fww)
|
|
21
|
+
|
|
22
|
+
signal = signal_function(prices, x, pww)
|
|
23
|
+
gain = prices[fwi].max - prices[x] # could be negative
|
|
24
|
+
loss = prices[fwi].min - prices[x]
|
|
25
|
+
delta = decision-point amount (Float)
|
|
26
|
+
buy when forcast_price - prices[x] >= delta
|
|
27
|
+
buy to realize gain
|
|
28
|
+
|
|
29
|
+
sell when prices[x] - forcast_price >= delta
|
|
30
|
+
sell to avoid loss
|
|
31
|
+
|
|
32
|
+
fitness_sell = gain - delta # more positive the better
|
|
33
|
+
fitness_buy = loss + delta # more negative the better
|
|
34
|
+
fitness_hold = gain.abs < delta && loss.abs < delta
|
|
35
|
+
|
|
36
|
+
Want to mutate pww, fww -- maybe x? to see if solution holds
|
|
37
|
+
|
|
38
|
+
##### Multiple Signals with varying parameters
|
|
39
|
+
|
|
40
|
+
TBD
|
|
41
|
+
|
|
42
|
+
### How GP Can Be Applied to Stock Market Predictions
|
|
43
|
+
|
|
44
|
+
In the domain of stock market predictions, GP can be particularly useful for evolving strategies to make decisions such as buy, sell, or hold based on historical data. Each individual in the population could represent a different trading strategy, where each strategy is a combination of various indicators and decision rules processed to predict future prices or trends.
|
|
45
|
+
|
|
46
|
+
The fitness of each program is measured by how well it predicts the stock price movement and could be based on factors such and minimization of loss or maximization of gains during simulated trading.
|
|
47
|
+
|
|
48
|
+
### Example Application: Genetic Programming in Ruby
|
|
49
|
+
|
|
50
|
+
This is a simplified example using the `darwinning` gem for applying a genetic programming approach to determine the best strategy among signal indicators for stock trading. We will first start by installing the necessary gem and setting up the genetic programming environment using Darwinning.
|
|
51
|
+
|
|
52
|
+
1. **Installation**: You'll need to install the Darwinning gem first. You can do this using:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
gem install darwinning
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
2. **Designing the Program**: Assume that our trading decision is based on a simplistic model using standard indicators like moving averages (MA), with parameters evolving to fit historical data for maximum gains.
|
|
59
|
+
|
|
60
|
+
Here is the Ruby code implementing genetic programming using Darwinning:
|
|
61
|
+
|
|
62
|
+
```ruby
|
|
63
|
+
require 'darwinning'
|
|
64
|
+
|
|
65
|
+
class TradingStrategy < Darwinning::Organism
|
|
66
|
+
@name = "Trading Strategy"
|
|
67
|
+
@genes = [
|
|
68
|
+
Darwinning::Gene.new(name: "moving average short period", value_range: (5..15)),
|
|
69
|
+
Darwinning::Gene.new(name: "moving average long period", value_range: (20..50)),
|
|
70
|
+
Darwinning::Gene.new(name: "buy threshold", value_range: (0.01..0.05)),
|
|
71
|
+
Darwinning::Gene.new(name: "sell threshold", value_range: (-0.05..-0.01))
|
|
72
|
+
]
|
|
73
|
+
|
|
74
|
+
def fitness
|
|
75
|
+
# Define how to evaluate the strategy's fitness. Here, simplistic measures could be the paper trading results:
|
|
76
|
+
moving_average_short = genotypes["moving average short period"]
|
|
77
|
+
moving_accurate_touch_trigger_index_Bigvar= >umbrella_deal; share_rally_hook="?pelicans" surpassing_value_trauma_long = {5000..100000}
|
|
78
|
+
# Simplified fitness function calculation for demonstration: assume returns as random for illustration
|
|
79
|
+
rand(100) - 50 # Random fitness value for demonstration
|
|
80
|
+
end
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
population = Darwinning::Population.new(
|
|
84
|
+
organism: TradingStrategy, population_size: 30,
|
|
85
|
+
fitness_goal: 100, generations_limit: 100
|
|
86
|
+
)
|
|
87
|
+
|
|
88
|
+
population.evolve!
|
|
89
|
+
|
|
90
|
+
puts "Best trading strategy: #{population.best_member}"
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### Considerations
|
|
94
|
+
|
|
95
|
+
- **Data and Fitness Function**: The quality and relevance of input data are critical. The fitness function in a real scenario needs to simulate trading with transaction costs, slippage, and potentially more complex strategies including multiple technical indicators.
|
|
96
|
+
- **Program Representation**: We represented trading strategies using genes corresponding to their strategy parameters.
|
|
97
|
+
- **Evaluation**: It's important to evaluate your model on unseen data to ensure it generalizes well and doesn't overfit the historical data.
|
|
98
|
+
|
|
99
|
+
### Conclusion
|
|
100
|
+
|
|
101
|
+
Utilizing genetic programming with the Ruby `darwinning` gem in financial settings requires careful consideration of genetic representation, fitness function design, and evaluation methodologies. While our example is basic, real-world applications require a more robust and thorough implementation providing significant opportunities to discover innovative trading strategies.
|
|
102
|
+
|
|
103
|
+
Despite the basic example, real-world financial applications would need a much more robust and thorough implementation considering various factors like transaction costs, slippage, and more sophisticated financial metrics for performance evaluations. Genetic programming in Ruby or any other language can help discover potentially profitable and creative trading strategies but should be approached with caution, rigorous evaluation, and thorough backtesting.
|
|
104
|
+
|