financedatabase 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- FinanceDatabase-0.1.0/FinanceDatabase.egg-info/PKG-INFO +173 -0
- FinanceDatabase-0.1.0/FinanceDatabase.egg-info/SOURCES.txt +9 -0
- FinanceDatabase-0.1.0/FinanceDatabase.egg-info/dependency_links.txt +1 -0
- FinanceDatabase-0.1.0/FinanceDatabase.egg-info/top_level.txt +1 -0
- FinanceDatabase-0.1.0/PKG-INFO +173 -0
- FinanceDatabase-0.1.0/README.md +155 -0
- FinanceDatabase-0.1.0/Searcher/__init__.py +10 -0
- FinanceDatabase-0.1.0/Searcher/json_options.py +33 -0
- FinanceDatabase-0.1.0/Searcher/json_picker.py +181 -0
- FinanceDatabase-0.1.0/setup.cfg +4 -0
- FinanceDatabase-0.1.0/setup.py +27 -0
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: FinanceDatabase
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: This is a database of 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures, Options, Currencies, Cryptocurrencies and Money Markets.
|
|
5
|
+
Home-page: https://github.com/JerBouma/FinanceDatabase
|
|
6
|
+
Author: JerBouma
|
|
7
|
+
Author-email: jer.bouma@gmail.com
|
|
8
|
+
License: MIT
|
|
9
|
+
Description: # Finance Database
|
|
10
|
+
As a private investor, the sheer amount of information that can be found on the internet is rather daunting. Trying to
|
|
11
|
+
understand what type of companies or ETFs are available is incredibly challenging with there being millions of
|
|
12
|
+
companies amd derivatives available on the market. Sure, the most traded companies and ETFs can quickly be found
|
|
13
|
+
simply because they are known to the public (for example, Microsoft, Tesla, S&P500 ETF or an All-World ETF). However,
|
|
14
|
+
what else is out there is often unknown.
|
|
15
|
+
|
|
16
|
+
**This database tries to solve that**. It features 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures,
|
|
17
|
+
Options, Currencies, Cryptocurrencies and Money Markets. It therefore allows you to obtain a broad overview of sectors,
|
|
18
|
+
industries, types of investments and much more.
|
|
19
|
+
|
|
20
|
+
The aim of this database is explicitly _not_ to provide up-to-date fundamentals or stock data as those can be obtained
|
|
21
|
+
with ease (with the help of this database) by using [FundamentalAnalysis](https://github.com/JerBouma/FundamentalAnalysis)
|
|
22
|
+
or [yfinance](https://github.com/ranaroussi/yfinance). Instead, it gives insights into the products that exist in each
|
|
23
|
+
country, industry and sector and gives the most essential information about each product. With this information, you
|
|
24
|
+
can analyse specific areas of the financial world and/or find that product that is hard to find. See for examples
|
|
25
|
+
on how you can combine this database and the earlier mentioned packages the section [Examples](#Examples).
|
|
26
|
+
|
|
27
|
+
Some key statistics of the database:
|
|
28
|
+
|
|
29
|
+
| Product | Quantity | Sectors | Industries | Countries | Exchanges |
|
|
30
|
+
| ----------------- | --------- | --------- | ------------- | --------- | --------- |
|
|
31
|
+
| Equities | 84.091 | 16 | 262 | 109 | 79 |
|
|
32
|
+
| ETFs | 15.892 | 268* | 88* | 100** | 44 |
|
|
33
|
+
| Funds | 34.947 | 857* | 416* | 100** | 25 |
|
|
34
|
+
|
|
35
|
+
| Product | Quantity | Exchanges |
|
|
36
|
+
| ----------------- | --------- | --------- |
|
|
37
|
+
| Indices | 24.548 | 62 |
|
|
38
|
+
| Currencies | 2.529 | 2 |
|
|
39
|
+
| Cryptocurrencies | 3.624 | 1 |
|
|
40
|
+
| Options | 13.819 | 1 |
|
|
41
|
+
| Futures | 1.173 | 7 |
|
|
42
|
+
| Money Markets | 1.384 | 2 |
|
|
43
|
+
|
|
44
|
+
\* These numbers refer to families (iShares, Vanguard) and categories (World Stock, Real Estate) respectively.
|
|
45
|
+
\** This is an estimation. Obtaining the country distribution can only be done by collecting data on the underlying
|
|
46
|
+
or by manual search.
|
|
47
|
+
|
|
48
|
+
## Usage
|
|
49
|
+
To access the database you can download the entire repository but I strongly recommend making use of the package
|
|
50
|
+
closely attached to the database. It allows you to select specific json files as well as search through collected
|
|
51
|
+
data with a specific query.
|
|
52
|
+
|
|
53
|
+
You can install the package with the following steps:
|
|
54
|
+
1. `pip install FinanceDatabase`
|
|
55
|
+
- Alternatively, download the 'Searcher' directory.
|
|
56
|
+
2. (within Python) `import FinanceDatabase as fd`
|
|
57
|
+
|
|
58
|
+
The package has the following functions:
|
|
59
|
+
- `show_options(product)` - gives all available options from the below functions per product (i.e. Equities, Funds)
|
|
60
|
+
which then can be used to collect the data.
|
|
61
|
+
- `select_cryptocurrencies(cryptocurrency=None)` - with no input gives all cryptocurrencies, with input gives
|
|
62
|
+
the cryptocurrency of choice.
|
|
63
|
+
- `select_currencies(currency=None)` - with no input gives all currencies, with input gives
|
|
64
|
+
the currency of choice.
|
|
65
|
+
- `select_etfs(category=None)` - with no input gives all etfs, with input gives all etfs of a
|
|
66
|
+
specific category.
|
|
67
|
+
- `select_equities(country=None, sector=None, industry=None)` - with no input gives all equities, with input
|
|
68
|
+
gives all equities of a country, sector, industry or a combination of the three.
|
|
69
|
+
- `select_funds(category=None)` - with no input gives all funds, with input gives all funds of a
|
|
70
|
+
specific category.
|
|
71
|
+
- `def select_indices(market=None)` - with no input gives all indices, with input gives all funds of a
|
|
72
|
+
specific market which usually refers to indices in a specific country (like de_market gives DAX).
|
|
73
|
+
- `select_other(product)` - gives either all Futures, all Moneymarkets or all Options.
|
|
74
|
+
- `search_products(database, query, new_database=None)` - with input from the above functions, this function searching
|
|
75
|
+
for specific values in the summary of the product (i.e. the query 'sustainable')
|
|
76
|
+
|
|
77
|
+
## Examples
|
|
78
|
+
This section gives a few examples of the possibilities with this package. These are merely a few of the things you
|
|
79
|
+
can do with the package and it only uses yfinance. **As you can obtain a wide range of symbols, pretty much any package
|
|
80
|
+
that requires symbols should work.**
|
|
81
|
+
|
|
82
|
+
### United States' Airlines
|
|
83
|
+
If I wish to obtain all companies within the United States listed under 'Airlines' I can write the
|
|
84
|
+
following code:
|
|
85
|
+
````
|
|
86
|
+
import FinanceDatabase as fd
|
|
87
|
+
|
|
88
|
+
airlines_us = fd.select_equities(country='United States', sector='Industrials', industry='Airlines')
|
|
89
|
+
````
|
|
90
|
+
Then, I can use packages like [yfinance](https://github.com/ranaroussi/yfinance) to quickly collect data from
|
|
91
|
+
Yahoo Finance for each symbol in the industry like this:
|
|
92
|
+
````
|
|
93
|
+
from yfinance.utils import get_json
|
|
94
|
+
from yfinance import download
|
|
95
|
+
|
|
96
|
+
airlines_us_fundamentals = {}
|
|
97
|
+
for symbol in airlines_us:
|
|
98
|
+
airlines_us_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
99
|
+
|
|
100
|
+
airlines_us_stock_data = download(list(airlines_us))
|
|
101
|
+
````
|
|
102
|
+
With a few lines of code, I have collected all data from a specific industry within the United States. From here on
|
|
103
|
+
you can compare pretty much any key statistic, fundamental data and stock data. For example, let's plot a simple bar
|
|
104
|
+
chart that gives insights in the Quick Ratios (indicator of the overall financial strength or weakness of a company):
|
|
105
|
+
````
|
|
106
|
+
import matplotlib.pyplot as plt
|
|
107
|
+
|
|
108
|
+
for symbol in airlines_us_fundamentals:
|
|
109
|
+
quick_ratio = airlines_us_fundamentals[symbol]['financialData']['quickRatio']
|
|
110
|
+
long_name = airlines_us_fundamentals[symbol]['quoteType']['longName']
|
|
111
|
+
|
|
112
|
+
if quick_ratio is None:
|
|
113
|
+
continue
|
|
114
|
+
|
|
115
|
+
plt.barh(long_name, quick_ratio)
|
|
116
|
+
|
|
117
|
+
plt.tight_layout()
|
|
118
|
+
plt.show()
|
|
119
|
+
``````
|
|
120
|
+
Which results in the graph displayed below (as of the 3rd of February 2021). From this graph you could identify
|
|
121
|
+
companies that currently lack enough assets to cover their liabilities (quick ratio < 1) and those that do have enough
|
|
122
|
+
assets (quick ratio > 1). Both too low and too high could make you wonder whether the company adequately manages
|
|
123
|
+
its assets.
|
|
124
|
+
|
|
125
|
+

|
|
126
|
+
|
|
127
|
+
### Semiconductors ETFs
|
|
128
|
+
If I wish to obtain all ETFs that have something to do with 'semiconductors' I can use the search function which works
|
|
129
|
+
as follows:
|
|
130
|
+
````
|
|
131
|
+
import FinanceDatabase as fd
|
|
132
|
+
|
|
133
|
+
all_etfs = fd.select_etfs()
|
|
134
|
+
semiconductor_etfs = search_products(all_etfs, 'semiconductor')
|
|
135
|
+
````
|
|
136
|
+
The variable semiconductor_etfs returns all etfs that have the word 'semiconductor' in their summary which usually also
|
|
137
|
+
corresponds to the fact they are targeted around semiconductors. Next, I collect data:
|
|
138
|
+
````
|
|
139
|
+
semiconductor_etfs_fundamentals = {}
|
|
140
|
+
for symbol in semiconductor_etfs:
|
|
141
|
+
semiconductor_etfs_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
142
|
+
````
|
|
143
|
+
And lastly, I have a look at the YTD returns (as of the 3rd of February 2021) of each ETF to understand which ones
|
|
144
|
+
might be worthwile to invest in:
|
|
145
|
+
````
|
|
146
|
+
for symbol in semiconductor_etfs_fundamentals:
|
|
147
|
+
ytd_return = semiconductor_etfs_fundamentals[symbol]['fundPerformance']['trailingReturns']['ytd']
|
|
148
|
+
long_name = semiconductor_etfs_fundamentals[symbol]['quoteType']['longName']
|
|
149
|
+
|
|
150
|
+
if ytd_return is None:
|
|
151
|
+
continue
|
|
152
|
+
|
|
153
|
+
plt.barh(long_name, ytd_return)
|
|
154
|
+
|
|
155
|
+
plt.tight_layout()
|
|
156
|
+
plt.xticks([-1, -0.5, 0, 0.5, 1], ['-100%', '-50%', '0%', '50%', '100%'])
|
|
157
|
+
plt.show()
|
|
158
|
+
````
|
|
159
|
+
This results in the following graph which gives _some_ insights in the available semiconductor ETFs. Then with the large amount of fundamentals data you can
|
|
160
|
+
figure out how each ETF differs and what might be worthwile to invest in.
|
|
161
|
+
|
|
162
|
+

|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
Keywords: finance,database,financials
|
|
166
|
+
Platform: UNKNOWN
|
|
167
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
168
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
|
169
|
+
Classifier: Topic :: Office/Business :: Financial :: Investment
|
|
170
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
171
|
+
Classifier: Operating System :: OS Independent
|
|
172
|
+
Classifier: Programming Language :: Python :: 3
|
|
173
|
+
Description-Content-Type: text/markdown
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
Searcher
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: FinanceDatabase
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: This is a database of 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures, Options, Currencies, Cryptocurrencies and Money Markets.
|
|
5
|
+
Home-page: https://github.com/JerBouma/FinanceDatabase
|
|
6
|
+
Author: JerBouma
|
|
7
|
+
Author-email: jer.bouma@gmail.com
|
|
8
|
+
License: MIT
|
|
9
|
+
Description: # Finance Database
|
|
10
|
+
As a private investor, the sheer amount of information that can be found on the internet is rather daunting. Trying to
|
|
11
|
+
understand what type of companies or ETFs are available is incredibly challenging with there being millions of
|
|
12
|
+
companies amd derivatives available on the market. Sure, the most traded companies and ETFs can quickly be found
|
|
13
|
+
simply because they are known to the public (for example, Microsoft, Tesla, S&P500 ETF or an All-World ETF). However,
|
|
14
|
+
what else is out there is often unknown.
|
|
15
|
+
|
|
16
|
+
**This database tries to solve that**. It features 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures,
|
|
17
|
+
Options, Currencies, Cryptocurrencies and Money Markets. It therefore allows you to obtain a broad overview of sectors,
|
|
18
|
+
industries, types of investments and much more.
|
|
19
|
+
|
|
20
|
+
The aim of this database is explicitly _not_ to provide up-to-date fundamentals or stock data as those can be obtained
|
|
21
|
+
with ease (with the help of this database) by using [FundamentalAnalysis](https://github.com/JerBouma/FundamentalAnalysis)
|
|
22
|
+
or [yfinance](https://github.com/ranaroussi/yfinance). Instead, it gives insights into the products that exist in each
|
|
23
|
+
country, industry and sector and gives the most essential information about each product. With this information, you
|
|
24
|
+
can analyse specific areas of the financial world and/or find that product that is hard to find. See for examples
|
|
25
|
+
on how you can combine this database and the earlier mentioned packages the section [Examples](#Examples).
|
|
26
|
+
|
|
27
|
+
Some key statistics of the database:
|
|
28
|
+
|
|
29
|
+
| Product | Quantity | Sectors | Industries | Countries | Exchanges |
|
|
30
|
+
| ----------------- | --------- | --------- | ------------- | --------- | --------- |
|
|
31
|
+
| Equities | 84.091 | 16 | 262 | 109 | 79 |
|
|
32
|
+
| ETFs | 15.892 | 268* | 88* | 100** | 44 |
|
|
33
|
+
| Funds | 34.947 | 857* | 416* | 100** | 25 |
|
|
34
|
+
|
|
35
|
+
| Product | Quantity | Exchanges |
|
|
36
|
+
| ----------------- | --------- | --------- |
|
|
37
|
+
| Indices | 24.548 | 62 |
|
|
38
|
+
| Currencies | 2.529 | 2 |
|
|
39
|
+
| Cryptocurrencies | 3.624 | 1 |
|
|
40
|
+
| Options | 13.819 | 1 |
|
|
41
|
+
| Futures | 1.173 | 7 |
|
|
42
|
+
| Money Markets | 1.384 | 2 |
|
|
43
|
+
|
|
44
|
+
\* These numbers refer to families (iShares, Vanguard) and categories (World Stock, Real Estate) respectively.
|
|
45
|
+
\** This is an estimation. Obtaining the country distribution can only be done by collecting data on the underlying
|
|
46
|
+
or by manual search.
|
|
47
|
+
|
|
48
|
+
## Usage
|
|
49
|
+
To access the database you can download the entire repository but I strongly recommend making use of the package
|
|
50
|
+
closely attached to the database. It allows you to select specific json files as well as search through collected
|
|
51
|
+
data with a specific query.
|
|
52
|
+
|
|
53
|
+
You can install the package with the following steps:
|
|
54
|
+
1. `pip install FinanceDatabase`
|
|
55
|
+
- Alternatively, download the 'Searcher' directory.
|
|
56
|
+
2. (within Python) `import FinanceDatabase as fd`
|
|
57
|
+
|
|
58
|
+
The package has the following functions:
|
|
59
|
+
- `show_options(product)` - gives all available options from the below functions per product (i.e. Equities, Funds)
|
|
60
|
+
which then can be used to collect the data.
|
|
61
|
+
- `select_cryptocurrencies(cryptocurrency=None)` - with no input gives all cryptocurrencies, with input gives
|
|
62
|
+
the cryptocurrency of choice.
|
|
63
|
+
- `select_currencies(currency=None)` - with no input gives all currencies, with input gives
|
|
64
|
+
the currency of choice.
|
|
65
|
+
- `select_etfs(category=None)` - with no input gives all etfs, with input gives all etfs of a
|
|
66
|
+
specific category.
|
|
67
|
+
- `select_equities(country=None, sector=None, industry=None)` - with no input gives all equities, with input
|
|
68
|
+
gives all equities of a country, sector, industry or a combination of the three.
|
|
69
|
+
- `select_funds(category=None)` - with no input gives all funds, with input gives all funds of a
|
|
70
|
+
specific category.
|
|
71
|
+
- `def select_indices(market=None)` - with no input gives all indices, with input gives all funds of a
|
|
72
|
+
specific market which usually refers to indices in a specific country (like de_market gives DAX).
|
|
73
|
+
- `select_other(product)` - gives either all Futures, all Moneymarkets or all Options.
|
|
74
|
+
- `search_products(database, query, new_database=None)` - with input from the above functions, this function searching
|
|
75
|
+
for specific values in the summary of the product (i.e. the query 'sustainable')
|
|
76
|
+
|
|
77
|
+
## Examples
|
|
78
|
+
This section gives a few examples of the possibilities with this package. These are merely a few of the things you
|
|
79
|
+
can do with the package and it only uses yfinance. **As you can obtain a wide range of symbols, pretty much any package
|
|
80
|
+
that requires symbols should work.**
|
|
81
|
+
|
|
82
|
+
### United States' Airlines
|
|
83
|
+
If I wish to obtain all companies within the United States listed under 'Airlines' I can write the
|
|
84
|
+
following code:
|
|
85
|
+
````
|
|
86
|
+
import FinanceDatabase as fd
|
|
87
|
+
|
|
88
|
+
airlines_us = fd.select_equities(country='United States', sector='Industrials', industry='Airlines')
|
|
89
|
+
````
|
|
90
|
+
Then, I can use packages like [yfinance](https://github.com/ranaroussi/yfinance) to quickly collect data from
|
|
91
|
+
Yahoo Finance for each symbol in the industry like this:
|
|
92
|
+
````
|
|
93
|
+
from yfinance.utils import get_json
|
|
94
|
+
from yfinance import download
|
|
95
|
+
|
|
96
|
+
airlines_us_fundamentals = {}
|
|
97
|
+
for symbol in airlines_us:
|
|
98
|
+
airlines_us_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
99
|
+
|
|
100
|
+
airlines_us_stock_data = download(list(airlines_us))
|
|
101
|
+
````
|
|
102
|
+
With a few lines of code, I have collected all data from a specific industry within the United States. From here on
|
|
103
|
+
you can compare pretty much any key statistic, fundamental data and stock data. For example, let's plot a simple bar
|
|
104
|
+
chart that gives insights in the Quick Ratios (indicator of the overall financial strength or weakness of a company):
|
|
105
|
+
````
|
|
106
|
+
import matplotlib.pyplot as plt
|
|
107
|
+
|
|
108
|
+
for symbol in airlines_us_fundamentals:
|
|
109
|
+
quick_ratio = airlines_us_fundamentals[symbol]['financialData']['quickRatio']
|
|
110
|
+
long_name = airlines_us_fundamentals[symbol]['quoteType']['longName']
|
|
111
|
+
|
|
112
|
+
if quick_ratio is None:
|
|
113
|
+
continue
|
|
114
|
+
|
|
115
|
+
plt.barh(long_name, quick_ratio)
|
|
116
|
+
|
|
117
|
+
plt.tight_layout()
|
|
118
|
+
plt.show()
|
|
119
|
+
``````
|
|
120
|
+
Which results in the graph displayed below (as of the 3rd of February 2021). From this graph you could identify
|
|
121
|
+
companies that currently lack enough assets to cover their liabilities (quick ratio < 1) and those that do have enough
|
|
122
|
+
assets (quick ratio > 1). Both too low and too high could make you wonder whether the company adequately manages
|
|
123
|
+
its assets.
|
|
124
|
+
|
|
125
|
+

|
|
126
|
+
|
|
127
|
+
### Semiconductors ETFs
|
|
128
|
+
If I wish to obtain all ETFs that have something to do with 'semiconductors' I can use the search function which works
|
|
129
|
+
as follows:
|
|
130
|
+
````
|
|
131
|
+
import FinanceDatabase as fd
|
|
132
|
+
|
|
133
|
+
all_etfs = fd.select_etfs()
|
|
134
|
+
semiconductor_etfs = search_products(all_etfs, 'semiconductor')
|
|
135
|
+
````
|
|
136
|
+
The variable semiconductor_etfs returns all etfs that have the word 'semiconductor' in their summary which usually also
|
|
137
|
+
corresponds to the fact they are targeted around semiconductors. Next, I collect data:
|
|
138
|
+
````
|
|
139
|
+
semiconductor_etfs_fundamentals = {}
|
|
140
|
+
for symbol in semiconductor_etfs:
|
|
141
|
+
semiconductor_etfs_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
142
|
+
````
|
|
143
|
+
And lastly, I have a look at the YTD returns (as of the 3rd of February 2021) of each ETF to understand which ones
|
|
144
|
+
might be worthwile to invest in:
|
|
145
|
+
````
|
|
146
|
+
for symbol in semiconductor_etfs_fundamentals:
|
|
147
|
+
ytd_return = semiconductor_etfs_fundamentals[symbol]['fundPerformance']['trailingReturns']['ytd']
|
|
148
|
+
long_name = semiconductor_etfs_fundamentals[symbol]['quoteType']['longName']
|
|
149
|
+
|
|
150
|
+
if ytd_return is None:
|
|
151
|
+
continue
|
|
152
|
+
|
|
153
|
+
plt.barh(long_name, ytd_return)
|
|
154
|
+
|
|
155
|
+
plt.tight_layout()
|
|
156
|
+
plt.xticks([-1, -0.5, 0, 0.5, 1], ['-100%', '-50%', '0%', '50%', '100%'])
|
|
157
|
+
plt.show()
|
|
158
|
+
````
|
|
159
|
+
This results in the following graph which gives _some_ insights in the available semiconductor ETFs. Then with the large amount of fundamentals data you can
|
|
160
|
+
figure out how each ETF differs and what might be worthwile to invest in.
|
|
161
|
+
|
|
162
|
+

|
|
163
|
+
|
|
164
|
+
|
|
165
|
+
Keywords: finance,database,financials
|
|
166
|
+
Platform: UNKNOWN
|
|
167
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
168
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
|
169
|
+
Classifier: Topic :: Office/Business :: Financial :: Investment
|
|
170
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
171
|
+
Classifier: Operating System :: OS Independent
|
|
172
|
+
Classifier: Programming Language :: Python :: 3
|
|
173
|
+
Description-Content-Type: text/markdown
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
# Finance Database
|
|
2
|
+
As a private investor, the sheer amount of information that can be found on the internet is rather daunting. Trying to
|
|
3
|
+
understand what type of companies or ETFs are available is incredibly challenging with there being millions of
|
|
4
|
+
companies amd derivatives available on the market. Sure, the most traded companies and ETFs can quickly be found
|
|
5
|
+
simply because they are known to the public (for example, Microsoft, Tesla, S&P500 ETF or an All-World ETF). However,
|
|
6
|
+
what else is out there is often unknown.
|
|
7
|
+
|
|
8
|
+
**This database tries to solve that**. It features 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures,
|
|
9
|
+
Options, Currencies, Cryptocurrencies and Money Markets. It therefore allows you to obtain a broad overview of sectors,
|
|
10
|
+
industries, types of investments and much more.
|
|
11
|
+
|
|
12
|
+
The aim of this database is explicitly _not_ to provide up-to-date fundamentals or stock data as those can be obtained
|
|
13
|
+
with ease (with the help of this database) by using [FundamentalAnalysis](https://github.com/JerBouma/FundamentalAnalysis)
|
|
14
|
+
or [yfinance](https://github.com/ranaroussi/yfinance). Instead, it gives insights into the products that exist in each
|
|
15
|
+
country, industry and sector and gives the most essential information about each product. With this information, you
|
|
16
|
+
can analyse specific areas of the financial world and/or find that product that is hard to find. See for examples
|
|
17
|
+
on how you can combine this database and the earlier mentioned packages the section [Examples](#Examples).
|
|
18
|
+
|
|
19
|
+
Some key statistics of the database:
|
|
20
|
+
|
|
21
|
+
| Product | Quantity | Sectors | Industries | Countries | Exchanges |
|
|
22
|
+
| ----------------- | --------- | --------- | ------------- | --------- | --------- |
|
|
23
|
+
| Equities | 84.091 | 16 | 262 | 109 | 79 |
|
|
24
|
+
| ETFs | 15.892 | 268* | 88* | 100** | 44 |
|
|
25
|
+
| Funds | 34.947 | 857* | 416* | 100** | 25 |
|
|
26
|
+
|
|
27
|
+
| Product | Quantity | Exchanges |
|
|
28
|
+
| ----------------- | --------- | --------- |
|
|
29
|
+
| Indices | 24.548 | 62 |
|
|
30
|
+
| Currencies | 2.529 | 2 |
|
|
31
|
+
| Cryptocurrencies | 3.624 | 1 |
|
|
32
|
+
| Options | 13.819 | 1 |
|
|
33
|
+
| Futures | 1.173 | 7 |
|
|
34
|
+
| Money Markets | 1.384 | 2 |
|
|
35
|
+
|
|
36
|
+
\* These numbers refer to families (iShares, Vanguard) and categories (World Stock, Real Estate) respectively.
|
|
37
|
+
\** This is an estimation. Obtaining the country distribution can only be done by collecting data on the underlying
|
|
38
|
+
or by manual search.
|
|
39
|
+
|
|
40
|
+
## Usage
|
|
41
|
+
To access the database you can download the entire repository but I strongly recommend making use of the package
|
|
42
|
+
closely attached to the database. It allows you to select specific json files as well as search through collected
|
|
43
|
+
data with a specific query.
|
|
44
|
+
|
|
45
|
+
You can install the package with the following steps:
|
|
46
|
+
1. `pip install FinanceDatabase`
|
|
47
|
+
- Alternatively, download the 'Searcher' directory.
|
|
48
|
+
2. (within Python) `import FinanceDatabase as fd`
|
|
49
|
+
|
|
50
|
+
The package has the following functions:
|
|
51
|
+
- `show_options(product)` - gives all available options from the below functions per product (i.e. Equities, Funds)
|
|
52
|
+
which then can be used to collect the data.
|
|
53
|
+
- `select_cryptocurrencies(cryptocurrency=None)` - with no input gives all cryptocurrencies, with input gives
|
|
54
|
+
the cryptocurrency of choice.
|
|
55
|
+
- `select_currencies(currency=None)` - with no input gives all currencies, with input gives
|
|
56
|
+
the currency of choice.
|
|
57
|
+
- `select_etfs(category=None)` - with no input gives all etfs, with input gives all etfs of a
|
|
58
|
+
specific category.
|
|
59
|
+
- `select_equities(country=None, sector=None, industry=None)` - with no input gives all equities, with input
|
|
60
|
+
gives all equities of a country, sector, industry or a combination of the three.
|
|
61
|
+
- `select_funds(category=None)` - with no input gives all funds, with input gives all funds of a
|
|
62
|
+
specific category.
|
|
63
|
+
- `def select_indices(market=None)` - with no input gives all indices, with input gives all funds of a
|
|
64
|
+
specific market which usually refers to indices in a specific country (like de_market gives DAX).
|
|
65
|
+
- `select_other(product)` - gives either all Futures, all Moneymarkets or all Options.
|
|
66
|
+
- `search_products(database, query, new_database=None)` - with input from the above functions, this function searching
|
|
67
|
+
for specific values in the summary of the product (i.e. the query 'sustainable')
|
|
68
|
+
|
|
69
|
+
## Examples
|
|
70
|
+
This section gives a few examples of the possibilities with this package. These are merely a few of the things you
|
|
71
|
+
can do with the package and it only uses yfinance. **As you can obtain a wide range of symbols, pretty much any package
|
|
72
|
+
that requires symbols should work.**
|
|
73
|
+
|
|
74
|
+
### United States' Airlines
|
|
75
|
+
If I wish to obtain all companies within the United States listed under 'Airlines' I can write the
|
|
76
|
+
following code:
|
|
77
|
+
````
|
|
78
|
+
import FinanceDatabase as fd
|
|
79
|
+
|
|
80
|
+
airlines_us = fd.select_equities(country='United States', sector='Industrials', industry='Airlines')
|
|
81
|
+
````
|
|
82
|
+
Then, I can use packages like [yfinance](https://github.com/ranaroussi/yfinance) to quickly collect data from
|
|
83
|
+
Yahoo Finance for each symbol in the industry like this:
|
|
84
|
+
````
|
|
85
|
+
from yfinance.utils import get_json
|
|
86
|
+
from yfinance import download
|
|
87
|
+
|
|
88
|
+
airlines_us_fundamentals = {}
|
|
89
|
+
for symbol in airlines_us:
|
|
90
|
+
airlines_us_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
91
|
+
|
|
92
|
+
airlines_us_stock_data = download(list(airlines_us))
|
|
93
|
+
````
|
|
94
|
+
With a few lines of code, I have collected all data from a specific industry within the United States. From here on
|
|
95
|
+
you can compare pretty much any key statistic, fundamental data and stock data. For example, let's plot a simple bar
|
|
96
|
+
chart that gives insights in the Quick Ratios (indicator of the overall financial strength or weakness of a company):
|
|
97
|
+
````
|
|
98
|
+
import matplotlib.pyplot as plt
|
|
99
|
+
|
|
100
|
+
for symbol in airlines_us_fundamentals:
|
|
101
|
+
quick_ratio = airlines_us_fundamentals[symbol]['financialData']['quickRatio']
|
|
102
|
+
long_name = airlines_us_fundamentals[symbol]['quoteType']['longName']
|
|
103
|
+
|
|
104
|
+
if quick_ratio is None:
|
|
105
|
+
continue
|
|
106
|
+
|
|
107
|
+
plt.barh(long_name, quick_ratio)
|
|
108
|
+
|
|
109
|
+
plt.tight_layout()
|
|
110
|
+
plt.show()
|
|
111
|
+
``````
|
|
112
|
+
Which results in the graph displayed below (as of the 3rd of February 2021). From this graph you could identify
|
|
113
|
+
companies that currently lack enough assets to cover their liabilities (quick ratio < 1) and those that do have enough
|
|
114
|
+
assets (quick ratio > 1). Both too low and too high could make you wonder whether the company adequately manages
|
|
115
|
+
its assets.
|
|
116
|
+
|
|
117
|
+

|
|
118
|
+
|
|
119
|
+
### Semiconductors ETFs
|
|
120
|
+
If I wish to obtain all ETFs that have something to do with 'semiconductors' I can use the search function which works
|
|
121
|
+
as follows:
|
|
122
|
+
````
|
|
123
|
+
import FinanceDatabase as fd
|
|
124
|
+
|
|
125
|
+
all_etfs = fd.select_etfs()
|
|
126
|
+
semiconductor_etfs = search_products(all_etfs, 'semiconductor')
|
|
127
|
+
````
|
|
128
|
+
The variable semiconductor_etfs returns all etfs that have the word 'semiconductor' in their summary which usually also
|
|
129
|
+
corresponds to the fact they are targeted around semiconductors. Next, I collect data:
|
|
130
|
+
````
|
|
131
|
+
semiconductor_etfs_fundamentals = {}
|
|
132
|
+
for symbol in semiconductor_etfs:
|
|
133
|
+
semiconductor_etfs_fundamentals[symbol] = get_json("https://finance.yahoo.com/quote/" + symbol)
|
|
134
|
+
````
|
|
135
|
+
And lastly, I have a look at the YTD returns (as of the 3rd of February 2021) of each ETF to understand which ones
|
|
136
|
+
might be worthwile to invest in:
|
|
137
|
+
````
|
|
138
|
+
for symbol in semiconductor_etfs_fundamentals:
|
|
139
|
+
ytd_return = semiconductor_etfs_fundamentals[symbol]['fundPerformance']['trailingReturns']['ytd']
|
|
140
|
+
long_name = semiconductor_etfs_fundamentals[symbol]['quoteType']['longName']
|
|
141
|
+
|
|
142
|
+
if ytd_return is None:
|
|
143
|
+
continue
|
|
144
|
+
|
|
145
|
+
plt.barh(long_name, ytd_return)
|
|
146
|
+
|
|
147
|
+
plt.tight_layout()
|
|
148
|
+
plt.xticks([-1, -0.5, 0, 0.5, 1], ['-100%', '-50%', '0%', '50%', '100%'])
|
|
149
|
+
plt.show()
|
|
150
|
+
````
|
|
151
|
+
This results in the following graph which gives _some_ insights in the available semiconductor ETFs. Then with the large amount of fundamentals data you can
|
|
152
|
+
figure out how each ETF differs and what might be worthwile to invest in.
|
|
153
|
+
|
|
154
|
+

|
|
155
|
+
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Modules
|
|
2
|
+
from .json_picker import select_cryptocurrencies
|
|
3
|
+
from .json_picker import select_currencies
|
|
4
|
+
from .json_picker import select_etfs
|
|
5
|
+
from .json_picker import select_equities
|
|
6
|
+
from .json_picker import select_funds
|
|
7
|
+
from .json_picker import select_indices
|
|
8
|
+
from .json_picker import select_other
|
|
9
|
+
from .json_options import show_options
|
|
10
|
+
from .json_options import search_products
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import requests
|
|
2
|
+
import json
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
def show_options(product):
|
|
6
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
7
|
+
"Database/Categories/")
|
|
8
|
+
|
|
9
|
+
if product.lower() not in ['cryptocurrencies', 'currencies', 'equities', 'etfs', 'funds']:
|
|
10
|
+
raise ValueError((product.lower() + " is not an available options. Please choose either 'cryptocurrencies', "
|
|
11
|
+
"'currencies', 'equities', 'etfs' or 'funds'"))
|
|
12
|
+
try:
|
|
13
|
+
json_file = URL + product.lower() + '_options.json'
|
|
14
|
+
request = requests.get(json_file)
|
|
15
|
+
json_data = json.loads(request.text)
|
|
16
|
+
except json.decoder.JSONDecodeError:
|
|
17
|
+
raise ValueError("Not able to find the options for " + product.lower())
|
|
18
|
+
|
|
19
|
+
return json_data
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
def search_products(database, query, new_database=None):
|
|
23
|
+
if new_database is None:
|
|
24
|
+
new_database = {}
|
|
25
|
+
|
|
26
|
+
for symbol in database:
|
|
27
|
+
try:
|
|
28
|
+
if query in database[symbol]['summary']:
|
|
29
|
+
new_database[symbol] = database[symbol]
|
|
30
|
+
except (TypeError, KeyError):
|
|
31
|
+
continue
|
|
32
|
+
|
|
33
|
+
return new_database
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
import requests
|
|
2
|
+
import json
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
def select_cryptocurrencies(cryptocurrency=None):
|
|
6
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
7
|
+
"Database/Cryptocurrencies/")
|
|
8
|
+
|
|
9
|
+
if cryptocurrency:
|
|
10
|
+
try:
|
|
11
|
+
json_file = URL + cryptocurrency + '.json'
|
|
12
|
+
request = requests.get(json_file)
|
|
13
|
+
json_data = json.loads(request.text)
|
|
14
|
+
except json.decoder.JSONDecodeError:
|
|
15
|
+
raise ValueError("Not able to find any data for {}".format(cryptocurrency))
|
|
16
|
+
else:
|
|
17
|
+
try:
|
|
18
|
+
json_file = URL + "_Cryptocurrencies.json"
|
|
19
|
+
request = requests.get(json_file)
|
|
20
|
+
json_data = json.loads(request.text)
|
|
21
|
+
except json.decoder.JSONDecodeError:
|
|
22
|
+
raise ValueError("Not able to find any data.")
|
|
23
|
+
|
|
24
|
+
return json_data
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def select_currencies(currency=None):
|
|
28
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
29
|
+
"Database/Currencies/")
|
|
30
|
+
|
|
31
|
+
if currency:
|
|
32
|
+
try:
|
|
33
|
+
json_file = URL + currency + '.json'
|
|
34
|
+
request = requests.get(json_file)
|
|
35
|
+
json_data = json.loads(request.text)
|
|
36
|
+
except json.decoder.JSONDecodeError:
|
|
37
|
+
raise ValueError("Not able to find any data for {}".format(currency))
|
|
38
|
+
else:
|
|
39
|
+
try:
|
|
40
|
+
json_file = URL + "_Currencies.json"
|
|
41
|
+
request = requests.get(json_file)
|
|
42
|
+
json_data = json.loads(request.text)
|
|
43
|
+
except json.decoder.JSONDecodeError:
|
|
44
|
+
raise ValueError("Not able to find any data.")
|
|
45
|
+
|
|
46
|
+
return json_data
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def select_etfs(category=None):
|
|
50
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
51
|
+
"Database/ETFs/")
|
|
52
|
+
|
|
53
|
+
if category:
|
|
54
|
+
category = category.replace(' ', '%20')
|
|
55
|
+
try:
|
|
56
|
+
json_file = URL + category + '.json'
|
|
57
|
+
request = requests.get(json_file)
|
|
58
|
+
json_data = json.loads(request.text)
|
|
59
|
+
except json.decoder.JSONDecodeError:
|
|
60
|
+
raise ValueError("Not able to find any data for {}".format(category))
|
|
61
|
+
else:
|
|
62
|
+
try:
|
|
63
|
+
json_file = URL + "_ETFs.json"
|
|
64
|
+
request = requests.get(json_file)
|
|
65
|
+
json_data = json.loads(request.text)
|
|
66
|
+
except json.decoder.JSONDecodeError:
|
|
67
|
+
raise ValueError("Not able to find any data.")
|
|
68
|
+
|
|
69
|
+
return json_data
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
def select_equities(country=None, sector=None, industry=None):
|
|
73
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
74
|
+
"Database/Equities/")
|
|
75
|
+
|
|
76
|
+
if country and sector and industry:
|
|
77
|
+
country = country.replace(' ', '%20')
|
|
78
|
+
sector = sector.replace(' ', '%20')
|
|
79
|
+
industry = industry.replace(' ', '%20')
|
|
80
|
+
|
|
81
|
+
try:
|
|
82
|
+
json_file = URL + 'Countries/' + country + '/' + sector + '/' + industry + ".json"
|
|
83
|
+
request = requests.get(json_file)
|
|
84
|
+
json_data = json.loads(request.text)
|
|
85
|
+
except json.decoder.JSONDecodeError:
|
|
86
|
+
raise ValueError("Not able to find any data with the combination of Country ({}), "
|
|
87
|
+
"Sector ({}), Industry ({}).".format(country, sector, industry))
|
|
88
|
+
elif sector and industry:
|
|
89
|
+
sector = sector.replace(' ', '%20')
|
|
90
|
+
industry = industry.replace(' ', '%20')
|
|
91
|
+
|
|
92
|
+
try:
|
|
93
|
+
json_file = URL + 'Sectors/' + sector + '/' + industry + ".json"
|
|
94
|
+
request = requests.get(json_file)
|
|
95
|
+
json_data = json.loads(request.text)
|
|
96
|
+
except json.decoder.JSONDecodeError:
|
|
97
|
+
raise ValueError("Not able to find any data with the combination of, "
|
|
98
|
+
"Sector ({}), Industry ({}).".format(sector, industry))
|
|
99
|
+
elif country:
|
|
100
|
+
country = country.replace(' ', '%20')
|
|
101
|
+
|
|
102
|
+
try:
|
|
103
|
+
json_file = URL + 'Countries/' + country + ".json"
|
|
104
|
+
request = requests.get(json_file)
|
|
105
|
+
json_data = json.loads(request.text)
|
|
106
|
+
except json.decoder.JSONDecodeError:
|
|
107
|
+
raise ValueError("Not able to find any data with the combination of "
|
|
108
|
+
"Country ({}).".format(country))
|
|
109
|
+
elif industry or (country and industry):
|
|
110
|
+
raise ValueError("Only selecting industry or country and industry results in no data.")
|
|
111
|
+
else:
|
|
112
|
+
try:
|
|
113
|
+
json_file = URL + "Equities.json"
|
|
114
|
+
request = requests.get(json_file)
|
|
115
|
+
json_data = json.loads(request.text)
|
|
116
|
+
except json.decoder.JSONDecodeError:
|
|
117
|
+
raise ValueError("Not able to find any data.")
|
|
118
|
+
|
|
119
|
+
return json_data
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
def select_funds(category=None):
|
|
123
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
124
|
+
"Database/Funds/")
|
|
125
|
+
|
|
126
|
+
if category:
|
|
127
|
+
category = category.replace(' ', '%20')
|
|
128
|
+
try:
|
|
129
|
+
json_file = URL + category + '.json'
|
|
130
|
+
request = requests.get(json_file)
|
|
131
|
+
json_data = json.loads(request.text)
|
|
132
|
+
except json.decoder.JSONDecodeError:
|
|
133
|
+
raise ValueError("Not able to find any data for {}".format(category))
|
|
134
|
+
else:
|
|
135
|
+
try:
|
|
136
|
+
json_file = URL + "_Funds.json"
|
|
137
|
+
request = requests.get(json_file)
|
|
138
|
+
json_data = json.loads(request.text)
|
|
139
|
+
except json.decoder.JSONDecodeError:
|
|
140
|
+
raise ValueError("Not able to find any data.")
|
|
141
|
+
|
|
142
|
+
return json_data
|
|
143
|
+
|
|
144
|
+
|
|
145
|
+
def select_indices(market=None):
|
|
146
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
147
|
+
"Database/Indices/")
|
|
148
|
+
|
|
149
|
+
if market:
|
|
150
|
+
try:
|
|
151
|
+
json_file = URL + market + '.json'
|
|
152
|
+
request = requests.get(json_file)
|
|
153
|
+
json_data = json.loads(request.text)
|
|
154
|
+
except json.decoder.JSONDecodeError:
|
|
155
|
+
raise ValueError("Not able to find any data for {}".format(market))
|
|
156
|
+
else:
|
|
157
|
+
try:
|
|
158
|
+
json_file = URL + "_Indices.json"
|
|
159
|
+
request = requests.get(json_file)
|
|
160
|
+
json_data = json.loads(request.text)
|
|
161
|
+
except json.decoder.JSONDecodeError:
|
|
162
|
+
raise ValueError("Not able to find any data.")
|
|
163
|
+
|
|
164
|
+
return json_data
|
|
165
|
+
|
|
166
|
+
|
|
167
|
+
def select_other(product):
|
|
168
|
+
URL = ("https://raw.githubusercontent.com/JerBouma/FinanceDatabase/master/"
|
|
169
|
+
"Database/Other/")
|
|
170
|
+
|
|
171
|
+
if product not in ['Futures', 'Moneymarkets', 'Options']:
|
|
172
|
+
raise ValueError("{} is not available. Only 'Futures', 'Moneymarkets' and 'Options'.")
|
|
173
|
+
|
|
174
|
+
try:
|
|
175
|
+
json_file = URL + product + ".json"
|
|
176
|
+
request = requests.get(json_file)
|
|
177
|
+
json_data = json.loads(request.text)
|
|
178
|
+
except json.decoder.JSONDecodeError:
|
|
179
|
+
raise ValueError("Not able to find any data with type {}.".format(product))
|
|
180
|
+
|
|
181
|
+
return json_data
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
import setuptools
|
|
2
|
+
|
|
3
|
+
with open("README.md", "r") as fh:
|
|
4
|
+
long_description = fh.read()
|
|
5
|
+
|
|
6
|
+
setuptools.setup(
|
|
7
|
+
name="FinanceDatabase",
|
|
8
|
+
packages=["Searcher"],
|
|
9
|
+
version="0.1.0",
|
|
10
|
+
license="MIT",
|
|
11
|
+
description="This is a database of 180.000+ symbols containing Equities, ETFs, Funds, Indices, Futures, "
|
|
12
|
+
"Options, Currencies, Cryptocurrencies and Money Markets.",
|
|
13
|
+
author="JerBouma",
|
|
14
|
+
author_email="jer.bouma@gmail.com",
|
|
15
|
+
url="https://github.com/JerBouma/FinanceDatabase",
|
|
16
|
+
long_description=long_description,
|
|
17
|
+
long_description_content_type="text/markdown",
|
|
18
|
+
keywords=["finance", "database", "financials"],
|
|
19
|
+
classifiers=[
|
|
20
|
+
"Development Status :: 5 - Production/Stable",
|
|
21
|
+
"Intended Audience :: Financial and Insurance Industry",
|
|
22
|
+
"Topic :: Office/Business :: Financial :: Investment",
|
|
23
|
+
"License :: OSI Approved :: MIT License",
|
|
24
|
+
"Operating System :: OS Independent",
|
|
25
|
+
"Programming Language :: Python :: 3"
|
|
26
|
+
],
|
|
27
|
+
)
|