ffiec_as_dataframe 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.ruby-version +1 -0
- data/CHANGELOG.md +5 -0
- data/Gemfile +2 -0
- data/README.md +28 -14
- data/Rakefile +9 -1
- data/ffiec_as_dataframe.gemspec +5 -2
- data/lib/ffiec_as_dataframe/call_report.rb +1 -3
- data/lib/ffiec_as_dataframe/version.rb +1 -1
- data/lib/ffiec_as_dataframe.rb +1 -2
- metadata +37 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 702d1c8707d355b1cfadc1b21351d6c8bcbb031133e90341b1f21e0120e7504a
|
|
4
|
+
data.tar.gz: 5677540fc41514b4e2f1114782483ce31378d2938e4a2a4a04da40d410ec8702
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a8a9694420725321973d9293305a9fce3e1288e9e12d7542735991a2af7d83205bae718721e65e47bd3a1a80653cd4e9759485fefe7f21dd103f4e34febe37c4
|
|
7
|
+
data.tar.gz: 16fd2f38704e2151aca67f3377efec0647cd690715248545a254a2a29aaedd8015f6c422b8a2a4e03e3fe55f49a8c69facc03312eb7d68931eeffdaa10f05119
|
data/.gitignore
CHANGED
data/.ruby-version
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
3.3.0
|
data/CHANGELOG.md
CHANGED
data/Gemfile
CHANGED
data/README.md
CHANGED
|
@@ -1,9 +1,14 @@
|
|
|
1
|
-
#
|
|
1
|
+
# FfiecAsDataframe
|
|
2
2
|
|
|
3
|
-
Up
|
|
3
|
+
Up-to-date call report data access for Ruby, using Polars dataframes.
|
|
4
4
|
|
|
5
5
|
This package will fetch call report data (from US banks) from the Federal Financial Institutions Examination Council (FFIEC), and return the results as a Polars Dataframe.
|
|
6
6
|
|
|
7
|
+
## Requirements
|
|
8
|
+
|
|
9
|
+
- Ruby >= 3.3
|
|
10
|
+
- polars-df 0.27.1
|
|
11
|
+
|
|
7
12
|
## Installation
|
|
8
13
|
|
|
9
14
|
Add this line to your application's Gemfile:
|
|
@@ -24,12 +29,12 @@ Or install it yourself as:
|
|
|
24
29
|
|
|
25
30
|
With no arguments provided in the initialization call, the gem will fetch the most recent call report data, and return it as a Polars dataframe. Note that most of the column names correspond to 8 character abbreviations that are documented at https://www.federalreserve.gov/apps/mdrm/data-dictionary . As requirements have changed over time, the number of columns will change with each release of the call report data.
|
|
26
31
|
|
|
27
|
-
```
|
|
28
|
-
3.
|
|
32
|
+
```ruby
|
|
33
|
+
3.3.x :003 > x = FfiecAsDataframe::CallReport.new.fetch
|
|
29
34
|
=>
|
|
30
35
|
shape: (4_594, 3_817)
|
|
31
36
|
...
|
|
32
|
-
3.
|
|
37
|
+
3.3.x :004 > x
|
|
33
38
|
=>
|
|
34
39
|
shape: (4_594, 3_817)
|
|
35
40
|
┌─────────┬─────────────────────────┬────────────────────┬───────────────────┬───┬──────────┬──────────┬──────────┬──────────┐
|
|
@@ -53,16 +58,16 @@ shape: (4_594, 3_817)
|
|
|
53
58
|
|
|
54
59
|
A specific vintage of call report data can be fetched by specifying the "dt" argument in the initialization call.
|
|
55
60
|
|
|
56
|
-
```
|
|
57
|
-
3.
|
|
61
|
+
```ruby
|
|
62
|
+
3.3.x :005 > x = FfiecAsDataframe::CallReport.new(dt= '2022-12-31'.to_date).fetch
|
|
58
63
|
=>
|
|
59
64
|
shape: (4_756, 3_992)
|
|
60
65
|
...
|
|
61
|
-
3.
|
|
66
|
+
3.3.x :006 > x = FfiecAsDataframe::CallReport.new(dt= '2023-12-31'.to_date).fetch
|
|
62
67
|
=>
|
|
63
68
|
shape: (4_641, 3_923)
|
|
64
69
|
...
|
|
65
|
-
3.
|
|
70
|
+
3.3.x :007 > x
|
|
66
71
|
=>
|
|
67
72
|
shape: (4_641, 3_923)
|
|
68
73
|
┌─────────┬─────────────────────────┬────────────────────┬───────────────────┬───┬──────────┬──────────┬──────────┬──────────┐
|
|
@@ -86,12 +91,12 @@ shape: (4_641, 3_923)
|
|
|
86
91
|
|
|
87
92
|
Additionally, a single table (which may be contained in one or more parts) may be fetched using the "tbl" argument to the "new" call. Note that, as in the following example, the "tbl" argument must contain the entire table mnemonic; here we fetched only the "RC" table, despite there being many other tables that also have a mnemonic that starts with "RC".
|
|
88
93
|
|
|
89
|
-
```
|
|
90
|
-
3.
|
|
94
|
+
```ruby
|
|
95
|
+
3.3.x :009 > x = FfiecAsDataframe::CallReport.new(dt= '2022-12-31'.to_date, tbl = 'RC').fetch
|
|
91
96
|
=>
|
|
92
97
|
shape: (4_757, 80)
|
|
93
98
|
...
|
|
94
|
-
3.
|
|
99
|
+
3.3.x :010 > x
|
|
95
100
|
=>
|
|
96
101
|
shape: (4_757, 80)
|
|
97
102
|
┌─────────┬──────────────────────────┬─────────────────────┬────────────────────┬───┬──────────────────┬─────────────────┬─────────────────────────────────┬──────────────────┐
|
|
@@ -116,6 +121,15 @@ shape: (4_757, 80)
|
|
|
116
121
|
|
|
117
122
|
Note that this package uses Selenium in order to run javascript on the FFIEC site before downloading the entire call report. If you have issues running Selenium (which will invoke the generation of a "headless" Chrome session), you may want to include an "options" parameter in the creation of the CallReport object and include "options={headless: false}".
|
|
118
123
|
|
|
124
|
+
## Testing
|
|
125
|
+
|
|
126
|
+
This gem uses minitest for testing. Run tests with:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
bundle exec rake test
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Tests mock Selenium/Chrome interactions to avoid external dependencies. No GitHub Actions or CI workflows are configured.
|
|
119
133
|
|
|
120
134
|
## Development
|
|
121
135
|
|
|
@@ -125,7 +139,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
|
|
|
125
139
|
|
|
126
140
|
## Contributing
|
|
127
141
|
|
|
128
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/
|
|
142
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/bmck/ffiec_as_dataframe. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/bmck/ffiec_as_dataframe/blob/main/CODE_OF_CONDUCT.md).
|
|
129
143
|
|
|
130
144
|
## License
|
|
131
145
|
|
|
@@ -133,4 +147,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
|
|
|
133
147
|
|
|
134
148
|
## Code of Conduct
|
|
135
149
|
|
|
136
|
-
Everyone interacting in the
|
|
150
|
+
Everyone interacting in the FfiecAsDataframe project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/bmck/ffiec_as_dataframe/blob/main/CODE_OF_CONDUCT.md).
|
data/Rakefile
CHANGED
|
@@ -1,4 +1,12 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "bundler/gem_tasks"
|
|
4
|
-
|
|
4
|
+
require "rake/testtask"
|
|
5
|
+
|
|
6
|
+
Rake::TestTask.new(:test) do |t|
|
|
7
|
+
t.libs << "test"
|
|
8
|
+
t.libs << "lib"
|
|
9
|
+
t.test_files = FileList["test/**/*_test.rb"]
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
task default: :test
|
data/ffiec_as_dataframe.gemspec
CHANGED
|
@@ -12,7 +12,7 @@ Gem::Specification.new do |spec|
|
|
|
12
12
|
spec.description = "Call report data from FFIEC"
|
|
13
13
|
spec.homepage = "https://github.com/bmck/ffiec_as_dataframe"
|
|
14
14
|
spec.license = "MIT"
|
|
15
|
-
spec.required_ruby_version = Gem::Requirement.new(">=
|
|
15
|
+
spec.required_ruby_version = Gem::Requirement.new(">= 3.3")
|
|
16
16
|
|
|
17
17
|
spec.metadata["homepage_uri"] = spec.homepage
|
|
18
18
|
spec.metadata["source_code_uri"] = "https://github.com/bmck/ffiec_as_dataframe"
|
|
@@ -28,8 +28,11 @@ Gem::Specification.new do |spec|
|
|
|
28
28
|
spec.require_paths = ["lib"]
|
|
29
29
|
|
|
30
30
|
# Uncomment to register a new dependency of your gem
|
|
31
|
-
spec.add_dependency 'polars-df'
|
|
31
|
+
spec.add_dependency 'polars-df', '0.27.1'
|
|
32
32
|
spec.add_dependency 'rubyzip'
|
|
33
33
|
spec.add_dependency 'activesupport', '~> 6', '<7.0.8'
|
|
34
34
|
spec.add_dependency 'selenium-webdriver'
|
|
35
|
+
|
|
36
|
+
spec.add_development_dependency 'minitest', '~> 5.0'
|
|
37
|
+
spec.add_development_dependency 'rake', '~> 13.0'
|
|
35
38
|
end
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
require 'set'
|
|
1
2
|
require 'polars-df'
|
|
2
3
|
require 'csv'
|
|
3
4
|
require 'zip'
|
|
@@ -18,9 +19,7 @@ module FfiecAsDataframe
|
|
|
18
19
|
end
|
|
19
20
|
|
|
20
21
|
def fetch
|
|
21
|
-
prefix = 'All_Reports_'
|
|
22
22
|
dte = dt.end_of_quarter
|
|
23
|
-
# dir = prefix + date.year.to_s + format('%02d', date.month.to_s) + date.mday.to_s
|
|
24
23
|
df = nil
|
|
25
24
|
|
|
26
25
|
Dir.mktmpdir(nil, Dir.tmpdir) do |d|
|
|
@@ -33,7 +32,6 @@ module FfiecAsDataframe
|
|
|
33
32
|
sleep(0.2)
|
|
34
33
|
driver.find_elements(css: 'select#ListBox1 option').detect{|o| o.text == 'Call Reports -- Single Period'}.click()
|
|
35
34
|
|
|
36
|
-
dte_found = false
|
|
37
35
|
dte = dte.strftime("%m/%d/%Y").to_s
|
|
38
36
|
|
|
39
37
|
if (opt = driver.find_elements(css: 'select#DatesDropDownList option').detect{ |o| o.text == dte }).present?
|
data/lib/ffiec_as_dataframe.rb
CHANGED
metadata
CHANGED
|
@@ -1,29 +1,29 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: ffiec_as_dataframe
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Bill McKinnon
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date:
|
|
11
|
+
date: 2026-08-28 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: polars-df
|
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
|
16
16
|
requirements:
|
|
17
|
-
- -
|
|
17
|
+
- - '='
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version:
|
|
19
|
+
version: 0.27.1
|
|
20
20
|
type: :runtime
|
|
21
21
|
prerelease: false
|
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
|
23
23
|
requirements:
|
|
24
|
-
- -
|
|
24
|
+
- - '='
|
|
25
25
|
- !ruby/object:Gem::Version
|
|
26
|
-
version:
|
|
26
|
+
version: 0.27.1
|
|
27
27
|
- !ruby/object:Gem::Dependency
|
|
28
28
|
name: rubyzip
|
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -72,6 +72,34 @@ dependencies:
|
|
|
72
72
|
- - ">="
|
|
73
73
|
- !ruby/object:Gem::Version
|
|
74
74
|
version: '0'
|
|
75
|
+
- !ruby/object:Gem::Dependency
|
|
76
|
+
name: minitest
|
|
77
|
+
requirement: !ruby/object:Gem::Requirement
|
|
78
|
+
requirements:
|
|
79
|
+
- - "~>"
|
|
80
|
+
- !ruby/object:Gem::Version
|
|
81
|
+
version: '5.0'
|
|
82
|
+
type: :development
|
|
83
|
+
prerelease: false
|
|
84
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
85
|
+
requirements:
|
|
86
|
+
- - "~>"
|
|
87
|
+
- !ruby/object:Gem::Version
|
|
88
|
+
version: '5.0'
|
|
89
|
+
- !ruby/object:Gem::Dependency
|
|
90
|
+
name: rake
|
|
91
|
+
requirement: !ruby/object:Gem::Requirement
|
|
92
|
+
requirements:
|
|
93
|
+
- - "~>"
|
|
94
|
+
- !ruby/object:Gem::Version
|
|
95
|
+
version: '13.0'
|
|
96
|
+
type: :development
|
|
97
|
+
prerelease: false
|
|
98
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
99
|
+
requirements:
|
|
100
|
+
- - "~>"
|
|
101
|
+
- !ruby/object:Gem::Version
|
|
102
|
+
version: '13.0'
|
|
75
103
|
description: Call report data from FFIEC
|
|
76
104
|
email:
|
|
77
105
|
- bill.mckinnon@bmck.org
|
|
@@ -80,6 +108,7 @@ extensions: []
|
|
|
80
108
|
extra_rdoc_files: []
|
|
81
109
|
files:
|
|
82
110
|
- ".gitignore"
|
|
111
|
+
- ".ruby-version"
|
|
83
112
|
- ".travis.yml"
|
|
84
113
|
- CHANGELOG.md
|
|
85
114
|
- CODE_OF_CONDUCT.md
|
|
@@ -108,14 +137,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
108
137
|
requirements:
|
|
109
138
|
- - ">="
|
|
110
139
|
- !ruby/object:Gem::Version
|
|
111
|
-
version:
|
|
140
|
+
version: '3.3'
|
|
112
141
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
113
142
|
requirements:
|
|
114
143
|
- - ">="
|
|
115
144
|
- !ruby/object:Gem::Version
|
|
116
145
|
version: '0'
|
|
117
146
|
requirements: []
|
|
118
|
-
rubygems_version: 3.
|
|
147
|
+
rubygems_version: 3.5.22
|
|
119
148
|
signing_key:
|
|
120
149
|
specification_version: 4
|
|
121
150
|
summary: Call report data from FFIEC
|