my-markdown-library 0.1.0

data/F24Textbook_MD/10-finance/value-interest.md

@@ -0,0 +1,60 @@
+---
+title: value-interest
+type: textbook
+source_path: content/10-finance/value-interest.ipynb
+chapter: 10
+---
+
+# Present Value, Future Value, and Interest Rates
+
+## The Time Value of Money
+
+An important concept that is the basis for most of finance is the *time value of money*: money now is worth more than money in the future. This makes sense; you would rather have \$100 now than later. But what if I owed you money and I really wanted to postpone the payment? In order to compensate you for your bias toward having money as soon as possible, I would need to pay more than I owed. Otherwise, you might not tolerate a delayed payment. This idea of an extra payment to address time concerns is called *interest*. There is also a dimension of risk as well. If you had reason to doubt my ability to repay you in the future, you might charge me more interest to compensate for this risk.
+
+```{admonition} Definition
+**Interest** is payment at a particular rate for the use of money or for delaying the repayment of money. Interest is typically set at a specific rate and that rate fluctuates based on the amount of time between borrowing and repayment, the risk of default, and other factors determined by the lender.
+```
+
+## Interest
+
+Interest is at the heart of loans, fixed income securities (think bonds), and financial economics in general. We are familiar with the bank account: you give a certain amount of money to a financial institution, and they compensate you for allowing them to use your money to invest in other assets. What they pay you for keeping your money is interest, and is normally quantified as a percentage of what you deposit with them - an interest rate. Thus, for each \$1 deposited at the bank, you will receive $r$ dollars in interest, where $r$ is the interest rate quoted by the bank. Note that $r$ takes the form of a decimal value: 0.05, for instance, and not 5%.
+
+Interest can be paid out in different time intervals - usually monthly, quarterly, yearly or continuously. Also, if you earn some interest in one year, in the next year you will not only earn interest on the initial amount you deposited, but also on the amount you earned the year before. This reflects the idea of *compounding interest*. We are able to determine how much \$1 will be worth in $t$ years, when compounding $n$ times per year at an interest rate of $r$.
+
+$$\begin{aligned}
+\text{Value of 1 dollar in } t \text{ years} &= 1 \times \left(1 + \dfrac{r}{n} \right) \times \left(1 + \dfrac{r}{n} \right) \times \cdots \times \left(1 + \dfrac{r}{n} \right) \\
+&= 1 \left(1 + \dfrac{r}{n} \right)^{nt}
+\end{aligned}$$
+
+Take a look at the table below for an example of the effects of compounding.
+
+|                        | Bank 1: 5\% annual compounding | Bank 2: 5\% semi-annual compounding                       |
+|------------------------|--------------------------------|-----------------------------------------------------------|
+| January 2016           | \$100                          | \$100                                                     |
+| July 2016              |                                | $100 \left(1 + \dfrac{0.05}{2} \right) =$ \$102.50        |
+| January 2017           | $100 (1 + 0.05) =$ \$105       | $102.50 \left(1 + \dfrac{0.05}{2} \right) =$ \$105.0625   |
+| July 2017              |                                | $105.0625 \left(1 + \dfrac{0.05}{2} \right) =$ \$107.69   |
+| January 2018           | $105 (1 + 0.05) =$ \$110.25    | $107.69 \left(1 + \dfrac{0.05}{2} \right) =$ \$110.38     |
+| Total Percent Change   | 10.25%                         | 10.38%                                                    |
+
+Notice that instead of a $5\% \cdot 2 = 10\%$ increase, you end up receiving a 10.25% or 10.38% increase depending on the rate of compounding. This is because the interest you received in the first compounding period (in bank 1’s case, a year, in bank 2’s case, half a year) is added onto your initial deposit, and this new deposit is used for calculating interest in the next period. Thus, even a small amount of money can grow quickly under interest rate compounding.
+
+## Present Value, Future Value, and the Discount Factor
+
+An important related concept is the idea of present and future value (which are effectively opposites). We have already discussed future value above. A \$100 deposit at bank 1 above has a future value of \$110.25 after 2 years. Conversely, an important question frequently asked in finance is the following:
+
+> Given an amount of money in the future, what is its fair value today? 
+
+In this example, what is the present value of \$110.25 at bank 1 two years in the future? Well, from the table above, \$100! This idea of present value is essential to the pricing of assets. In general, an asset's price is the present value of all expected future payments.
+
+$$\begin{aligned}
+\text{FV of 1 dollar} &= 1 \times \left(1 + \dfrac{r}{n} \right)^{nt} \\
+\text{PV of 1 dollar} &= \dfrac{1}{\left(1 + \dfrac{r}{n} \right)^{nt}}
+\end{aligned}$$
+
+We call $\dfrac{1}{(1 + \frac{r}{n})^{nt}}$ a *discount factor*. It discounts the value of \$1 from the future into today. This ties in with the time value of money. Since a dollar today is worth more than a dollar tomorrow, in order for you to be indifferent between receiving money today or tomorrow, the money you would receive tomorrow has to be discounted into the present by some amount that depends on the interest rate.
+
+$$
+\text{PV} = \text{DF} \cdot \text{FV}
+$$
+

data/F24Textbook_MD/11-econometrics/index.md

@@ -0,0 +1,16 @@
+---
+title: index
+type: textbook
+source_path: content/11-econometrics/index.md
+chapter: 11
+---
+
+# Econometrics
+
+One way to think about Econometrics is that it is the closest economists can usually get to doing experiments in the same way experiments are done in medicine and other fields involving people. The idealized experiment is one where people are randomly assigned to either the treatment or control group such that the participants *and* the researchers don't know which group each person is in.
+
+By randomly assigning a large sample of participants to two different groups, we can be fairly confident that on average the two groups are identical in their attributes, apart from the treatment that only one group receives. By having a double-blind experiment, we can be confident that the participants and researchers aren't conciously or subconciously (placebo effect) affecting the results of the treatment group. Because of this setup, we can look at the difference in outcomes of the two groups and be fairly confident that any differences are due to the treatment and nothing else.
+
+But it's usually impossible or unethical to perform an ideal experiment in economics. Imagine trying to examine the effect of years of schooling on future earnings. Can you force people into a treatment group with more years of schooling and a control group with less? Is it sufficient to just collect a sample of people and compare the earnings of people with high schooling to those with low schooling?
+
+How can we answer the above question and others like it? Econometrics. Econometrics is a vast field and today we will just focus on the basics of something called regression.

data/F24Textbook_MD/11-econometrics/multivariable.md

@@ -0,0 +1,218 @@
+---
+title: multivariable
+type: textbook
+source_path: content/11-econometrics/multivariable.ipynb
+chapter: 11
+---
+
+# Multivariable Regression and Bias
+
+```python
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+import statsmodels.api as sm
+import warnings
+warnings.simplefilter("ignore")
+from mpl_toolkits.mplot3d import Axes3D
+from matplotlib import patches
+from datascience import *
+%matplotlib inline
+from sklearn.metrics import mean_squared_error
+from ipywidgets import interact, interactive, fixed
+import ipywidgets as widgets
+
+nlsy = Table.read_table('nlsy_cleaned_small.csv')
+nlsy = nlsy.with_column("college", nlsy.column("college").astype(float))
+nlsy = nlsy.with_column("AFQT", nlsy.column("AFQT").astype(float))
+```
+
+Our procedure earlier showed that we expect to see roughly a 70% increase in earnings in people who went to college vs. people who did not go to college. Does this imply that your decision to go to college was worthwhile, and now you can expect to have roughly 70% higher earnings compared to the version of you who did not go to college?
+
+Let's go back to our discussion of experiments from earlier. In an ideal experiment, we would want a good sample of people who are about to graduate high school, and then randomly assign them to either a treatment group that gets sent to college, and a control group that does not. If you are in the treatment group you *must* go to college, and if you are in the control group you *cannot* go to college. Since the decision to go to college in this case is completely random, we can safely assume that the treatment and control groups are on average identical in attributes, apart from college attendance. We can therefore compare their log earnings in the future to see the effect of going to college.
+
+Clearly this experiment is impossible to perform. We cannot randomly assign people to go to college. What's different between this ideal experiment and our regression from earlier? What's the issue with comparing the differences in log earnings for people in our sample who happened to go to college and those who did not?
+
+In our sample, the treatment (went to college) and control (did not go to college) groups are not identical in every way except for college! People aren't randomly assigned college, they *choose* to go to college. The factors that cause someone to go to college are complex and lead to differences between people who chose to go to college and those who did not. When we perform regression on the variable `college`, since the groups in the sample are different, not only are we capturing the effect of going to college, but we are also capturing the effects of everything else that is different about the two groups that also affects earnings.
+
+Imagine another variable that captures the wealth of your family. College can be very expensive, so it might be the case that the wealthier your family is, the more likely you are to go to college. Also, it's easy to imagine that the wealthier your family is, the wealthier you are likely to be in the future. This implies that the group of people in the sample who went to college will tend to be wealthier than the group that did not. Also, the group of people who went to college is expected to earn more not necessarily because they went to college, but simply because they are wealthier.
+
+Therefore, when we do regression to measure the differences in earnings between people who went to college and those who did not, we also capture differences in earnings between people who grew up wealthier and those who did not. Because of this, we *over-estimate* the effect of going to college. $\hat{\beta}$ captures the average observed benefit of going to college *and* being wealthier, but we're only interested in college. This is called *omitted variable bias*. Family wealth is an omitted variable in this regression and it's causing our results to be biased.
+
+Let's think of another example of omitted variable bias. In the NLSY dataset, there is a variable called `AFQT`. AFQT is a score on a particular standardized test that all people in the sample took. Let's use AFQT as a proxy measurement for the abstract idea of academic capability. While a standardized test is by no means the sole indication of someone's ability or intelligence, let's ignore that very complicated issue for now and assume that AFQT does an O.K. job at capturing this ability variable that is otherwise very difficult to measure.
+
+Is there omitted variable bias from AFQT? Almost certainly. It seems fair to believe that people who choose to go to college are on average more academically-capable, and it also seems fair to say that on average we expect more capable people to earn more. Therefore, $\hat{\beta}$ above might be capturing the effects of being more capable, along with the effects of going to college.
+
+How can we fix this issue? Multivariable regression.
+
+## Multivariable Regression
+
+So far we have only been regressing outcome variable $y$ onto one explanatory variable $x$. To find the regression line, we choose $\hat{\alpha}$ and $\hat{\beta}$ that minimize the mean squared error. But what if we believe that $y$ is actually determined by two variables, $x_1$ and $x_2$? Specifically, what if the "true" model is
+
+$$
+y = \alpha + \beta_1 x_{1} + \beta_2 x_{2} + \epsilon
+$$
+
+and we would like to estimate
+
+$$
+\hat{y} = \hat{\alpha} + \hat{\beta}_1 x_{1} + \hat{\beta}_2 x_{2}
+$$
+
+Now our challenge is choosing $\hat{\alpha}$, $\hat{\beta}_1$, *and* $\hat{\beta}_2$ that minimize the mean squared error. To this end, we will use the `minimize` function to minimize a function `to_minimize` that takes in model parameters and returns the model's RMSE.
+
+```python
+def rmse(target, pred):
+    return np.sqrt(mean_squared_error(target, pred)) 
+
+def to_minimize(intercept, beta_1, beta_2):
+    predictions = intercept + beta_1 * nlsy.column('college') + beta_2 * nlsy.column("AFQT")
+    actual = nlsy.column('log_earn_1999')
+    return rmse(predictions, actual)
+
+minimize(to_minimize)
+```
+
+We see that `minimize` return an $\hat{\alpha}$ of 9.956, a $\hat{\beta}_1$ of 0.430, and a $\hat{\beta}_2$ of 0.008. Let's perform the regression using `statsmodels` and see what we get.
+
+```python
+y = nlsy.select('log_earn_1999').values
+X = nlsy.select('college', 'AFQT').values
+
+model = sm.OLS(y, sm.add_constant(X)).fit()
+model.summary()
+```
+
+Here $\hat{\beta}_1$ is 0.43, compared to 0.72 from the earlier biased (single variable) regression. That's huge! This implies that when we control for a person's ability (i.e. we get rid of that source of bias), we only see that on average going to college is associated with a 43% increase in earnings instead of 72%. Furthermore, looking at the 95% confidence interval for $\hat{\beta}_2$, we see that it does not contain 0, which would imply that AFQT score has a strong non-zero association with earnings.
+
+These observations validate our claim that AFQT was probably an omitted variable causing $\hat{\beta}_1$ to be biased. When interpreting $\hat{\beta}$ coefficients, you should always be mindful of potential sources of bias that could make your coefficients misleading and not useful from an econometric context.
+
+**Note:** A linear regression model makes predictions for $y$ which we've been calling $\hat{y}$. If you imagine that you are the model and are tasked with predicting people's earnings, you will almost certainly want more than just their years of schooling to make an accurate prediction. The more relevant variables you are given, the better predictions you are likely to make compared to just using one variable. The Variable needs to be relevant though; the day of the week a person was born on is probably not useful in predicting earnings. This is just another way of thinking about the usefulness of multivariable regression.
+
+### Visualizing Multivariable Regression
+
+The 3D plots below show us our variables of interest and the regression plane from two different angles.
+
+```python
+ax = plt.figure(figsize=(8,8)).add_subplot(111, projection='3d')
+ax.scatter(nlsy.column("AFQT"), 
+           nlsy.column("college"), 
+           nlsy.column('log_earn_1999'))
+plt.xlabel("AFQT", fontsize=14)
+plt.ylabel("college", fontsize=14)
+ax.set_zlabel("log_earn_1999", fontsize=14)
+plt.title("Data Points", pad=30, size=20);
+```
+
+Scatter plot in 3D showing data points related to AFQT, college, and log_earn_1999.
+
+```python
+X,Y = np.meshgrid(np.arange(0,100,1), np.arange(0,1,0.01))
+Z = 0.0084 * X + 0.4301 * Y + 9.9563
+ax = plt.figure(figsize=(8,8)).add_subplot(111, projection='3d')
+ax.scatter(nlsy.column("AFQT"), 
+           nlsy.column("college"), 
+           nlsy.column('log_earn_1999'))
+ax.plot_surface(X, Y, Z, alpha=0.5)
+plt.xlabel("AFQT", fontsize=14)
+plt.ylabel("college", fontsize=14)
+ax.set_zlabel("log_earn_1999", fontsize=14)
+plt.title("Data Points + Regression Plane", pad=30, size=20);
+```
+
+Scatter plot in 3D with a visible regression plane. Data points are related to AFQT, college, and log_earn_1999. Regression plane represented by a surface.
+
+```python
+X,Y = np.meshgrid(np.arange(0,1,0.01), np.arange(0,100,1))
+Z = 0.4301 * X + 0.0084 * Y + 9.9563
+ax = plt.figure(figsize=(8,8)).add_subplot(111, projection='3d')
+ax.scatter(nlsy.column("college"), 
+           nlsy.column("AFQT"), 
+           nlsy.column('log_earn_1999'))
+ax.plot_surface(X, Y, Z, alpha=0.1)
+plt.ylabel("AFQT", fontsize=14)
+plt.xlabel("college", fontsize=14)
+ax.set_zlabel("log_earn_1999", fontsize=14)
+plt.title("Data Points + Regression Plane", pad=30, size=20);
+```
+
+Scatter plot in 3D with a rotated view of data points related to college, AFQT, and log_earn_1999. Regression plane represented by a semi-transparent surface.
+
+The regression plane, instead of the regression line, represents the values for log earnings that the regression model would predict for any given college and AFQT input. It's a plane now because there are two possible inputs, as opposed to one.
+
+## Colinearity and Dummy Variables
+
+When we do regression onto the variable `college`, why don't we also include a variable that measures not going to college? In other words, why don't we regress on college and the opposite of college so that way we can get an estimate of the average earnings of college-goers and non-college-goers. Why do we do this roundabout way of using the intercept term and a difference in means?
+
+Imagine we have a dataset with a variable for college, a variable for not going to college, and the intercept term. The issue with this is that there is now redundant information.
+
+Let's look at just one element of the sample. Let's say this person went to college, so this person's features are the following:
+* College = 1
+* Not College = 0
+* Intercept term = 1
+
+Clearly there is a redundancy here; you can guess one of the variables from the others. More specifically, by redundancy we mean that *one variable can be written as a linear combination of the other variables*. In fact, there are three different combinations:
+* Intercept = College + Not College
+* Not College = Intercept - College
+* College = Intercept - Not College
+
+These equalities aren't just true for this one person; they actually hold true for any possible person in the sample. This is because of the way we defined "college" and "not college". You can't simultaneously be in both, and so adding them together you get 1, which is just the intercept term.
+
+In general, we have redundancy whenever we have *mutually exclusive* and *exhaustive* dummy variables in combination with an intercept term.
+* Mutually exclusive: You cannot be in more than one dummy variable.
+* Exhaustive: You must be in at least one dummy variable.
+
+You can see that "college" and "not college" satisfy these conditions. So why is this redundancy an issue? It becomes ambiguous what the values for $\hat{\alpha}$, $\hat{\beta}_1$, and $\hat{\beta}_2$ should be in the model where we include all three terms:
+
+$$
+\text{log earnings} = \hat{\alpha} + \hat{\beta}_1 \text{college} + \hat{\beta}_2 \text{not college}
+$$
+
+Consider a case where we expect people who went to college to have log earnings of 10 and those who did not go to college to have log earnings of 8. What values for $\hat{\beta}$ and $\hat{\alpha}$ make sense?
+
+* $\hat{\beta}_1 = 10$
+* $\hat{\beta}_2 = 8$
+* $\hat{\alpha} = 0$
+
+make sense. These are valid values for $\hat{\beta}$ and $\hat{\alpha}$ that satisfy the condition above. To see why, consider a person with college:
+
+$$\begin{aligned}
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \cdot 1 + \hat{\beta}_2 \cdot 0 \\
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \\
+\text{log earnings} &= 0 + 10 = 10
+\end{aligned}$$
+
+and a person without college:
+
+$$\begin{aligned}
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \cdot 0 + \hat{\beta}_2 \cdot 1 \\
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_2 \\
+\text{log earnings} &= 0 + 8 = 8
+\end{aligned}$$
+
+* $\hat{\beta}_1 = 2$
+* $\hat{\beta}_2 = 0$
+* $\hat{\alpha} = 8$
+
+also make sense. To see why, consider a person with college:
+
+$$\begin{aligned}
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \cdot 1 + \hat{\beta}_2 \cdot 0 \\
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \\
+\text{log earnings} &= 8 + 2 = 10
+\end{aligned}$$
+
+and a person without college:
+
+$$\begin{aligned}
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_1 \cdot 0 + \hat{\beta}_2 \cdot 1 \\
+\text{log earnings} &= \hat{\alpha} + \hat{\beta}_2 \\
+\text{log earnings} &= 8 + 0 = 8
+\end{aligned}$$
+
+It turns out, there are actually infinitely many solutions for $\hat{\beta}$ that satisfy the condition where people who went to college have mean log earnings of 10 and people who did not go to college have mean log earnings of 8. This holds true for all situations where you regress on a constant and a set of mutually exclusive and exhaustive dummies. There is no unique solution for $\hat{\beta}$, which is a problem for econometricians who want unique and interpretable coefficients.
+
+In fact, there is mathematical justification for this as well. At some point in the math involved in performing regression, having redundant variables causes a division by 0. This is particularly upsetting for your computer, and it will complain.
+
+So how do we avoid this problem? We deliberately exclude one of the variables. It can technically either be one of the dummy variables or the intercept term, but we usually really want to have an intercept term present in our regression for other reasons. So we usually get rid of one of the dummy variables. Notice that we implicitly did this earlier. We did not include "not college" in our first regression.
+

data/F24Textbook_MD/11-econometrics/reading-econ-papers.md

@@ -0,0 +1,25 @@
+---
+title: reading-econ-papers
+type: textbook
+source_path: content/11-econometrics/reading-econ-papers.ipynb
+chapter: 11
+---
+
+# Reading Economics Papers
+
+In upper division economics courses, you'll often read economics papers that utilize ordinary least squares to conduct regression. Now that we have familiarized ourselves with multi-variate regression, let's familiarize ourselves with reading the results of economics papers!
+
+Let's consider an existing empirical study conducted by David Card {cite}`11reading-econ-papers`, a Nobel Prize winning professor at UC Berkeley, that regresses income on education:
+
+![](https://i.imgur.com/FPLII4s.png)
+
+Every column here is from a different regression: the first column predicts the log hourly earnings from years of education, the fifth column predicts the log annual earnings from years of education, and so on. For now, let's focus on the first column, which states the linear regression as follows: 
+
+$$
+\ln{(\text{hourly earnings})_i} = \alpha + \beta \cdot (\text{years of schooling})_i + \varepsilon_i
+$$
+
+From the table, the education coefficient is 0.100, with a (0.001) underneath it. This means that our $\beta$ value is equal to 0.100. What does the (0.001) mean? It is the standard error, which is essentially a measure of our uncertainty. From Data 8, the standard error is most similar to the standard deviation of sample means, which is a measure of the spread in the population mean. Similarly, the the standard error here is a measure of the spread in the population coefficient. We can use the standard error to construct a confidence interval of the actual coefficient: a 95% confidence interval is between 2 standard errors above and below the reported value.
+
+The effects of schooling on income is captured by the education coefficient term: 0.100. This means that an increase in 1 unit (year) of education is correlated with a log hourly earnings by 0.1. This approximately corresponds to a 10% increase in wages per year of schooling.
+