my-markdown-library 0.1.0

data/F24Textbook_MD/00-intro/index.md

@@ -0,0 +1,292 @@
+---
+title: index
+type: textbook
+source_path: content/00-intro/index.ipynb
+chapter: 0
+---
+
+# Introduction
+
+## The Financial Benefits of Your Major
+
+```python
+import ipywidgets as widgets
+from datascience import *
+import numpy as np
+from ipywidgets import interact, interactive, fixed, interact_manual
+import ipywidgets as widgets
+from IPython.display import display
+import pandas as pd
+%matplotlib inline 
+import matplotlib.pyplot as plt
+plt.style.use('fivethirtyeight')
+```
+
+```python
+import plotly.graph_objects as go
+from plotly.offline import plot
+from IPython.display import display, HTML
+```
+
+Welcome to Data 88E: *Economics Models*! This class will explore the intersection of Data Science and Economics. 
+Specifically, we will utilize methods and techniques in data science to examine both basic and upper-division Economics concepts.
+Throughout the course, we will consider a variety of economic problems both on the macro and micro level. 
+
+In the first demo of the course, we hope to give you a sense of the types problems you can expect to explore this semester by considering a problem that may be of personal relevance to you: the post-graduate incomes of different majors at Cal.
+
+We will be using various visualization techniques to analyze the median incomes of different majors at UC Berkeley, in addition to the median incomes of those same majors at other colleges.
+If you forgot, the median income is the "middle" value: if you sorted all the individual incomes of a major in ascending order, the median would be the value that's exactly in the middle. The median is also called the 50th percentile -- at the median, exactly 50% of the individuals have an income lower than the median.
+
+Do not be concerned if you don't understand the code below: this entire exercise is purely a demo to motivate many profound concepts in Economics. 
+If you're interested, you may choose to come back to this demo at the end of the course and consider all the different techniques utilized in it - it'd be a great way of reflecting upon how much you've learnt!
+
+## Data Collection
+
+Before we can use data science to tackle any issue, we must–well–obtain data (kind of mind-boggling, I know).
+Recall that we want to examine the median incomes of different majors at UC Berkeley as well as the median incomes of those same majors at other colleges.
+The term 'other colleges' is a fairly general one, and in this case we shall consider the average median incomes of those majors at alll other colleges in the United States.
+
+In order to obtain a dataset, you can either collect it yourself (via surveys, questionnaires, etc.) or you can use datasets that others have gathered for you.
+In this demo, we are combining 3 different datasets:
+- The median income for each major at Cal was obtained from [Cal's 2024 First Destination survey](https://calviz.berkeley.edu/t/SAPublic/views/FirstDestinations2024-2/Employment).
+- The median income for each major overall was obtained from surveys conducted by the American Community Survey (ACS) from 2010 to 2012, a very popular data source for Economics Research! In the survey, ACS essentially calls college graduates and asked them their income as well as what they majored in at college. (As a side note, FiveThirtyEight later published this article:[The Economic Guide to Picking a College Major](https://fivethirtyeight.com/features/the-economic-guide-to-picking-a-college-major/) using the results of the survey.) In this project, we will be using a modified version of the ACS survey - we will only be looking at the respondents who are 28 or younger. Can you think of why we would do this?
+- The longitudinal data on long-run outcomes of UC Berkeley alumni was obtained from the [University of California webpage](https://www.universityofcalifornia.edu/infocenter/berkeley-outcomes). We will use this dataset later for a slightly different analysis.
+
+Take a moment to consider the ways in which the 3 different datasets were created. 
+Is it fair to draw direct comparisons between the datasets? What would be some potential issues and how could the differences in our datasets affect our analysis?
+
+
+### Mean vs Median
+Before proceeding further, it is important to consider why we are choosing to look at the median, and not the average, income.
+In order to answer this question, let us think about what the *distribution* of incomes for a population would look like. 
+Most likely, you would see a high amount of incomes around or slightly below the mean, with a few massive outlier incomes above the mean.
+For example, consider a theatre major who becomes a star on Broadway - while they'd be doing absolutely fantastic in their career, they are not representative of the average theatre graduate from Berkeley and would likely pull the average income way up.
+For this reason, using the median is more *robust*: it gives us a better idea of what the typical graduate for any major can generally expect to earn.
+
+Now we'll load in all the data.
+Take a look at the tables for each dataset.
+Note that `P25th` referes to the 25th percentile of incomes (the income level at which exactly 25% of incomes are lower) and `P75th` refers to the 75th percentile of incomes (the income level at which exactly 75% of incomes are lower).
+You may not know what all the different columns in the tables mean. That's okay! 
+As data scientists, we often encounter a lot of irrelevant data that we will discard later.
+
+```python
+# Load in table of all majors' median incomes at Cal
+cal_income = Table.read_table("cal_income.csv")
+cal_income.show(10)
+```
+
+```python
+# Load in table of all other universities' average major median incomes
+other_income = Table.read_table("recent-grads.csv") 
+other_income.show(10)
+```
+
+To make direct comparisons across majors, we combined all the tables above into a single one for us to use below.
+
+```python
+majors = Table.read_table("cal_vs_all.csv")
+majors.show(10)
+```
+
+Our combined table above dropped the columns in above tables that we didn't need to conduct our exploration. 
+It has a column `Median Income Difference`: this column is the Berkeley median income minus the overall median income for each major. 
+It gives us a sense of the value of Cal over the average university: the difference is the additional income we recieve from obtaining a Cal degree.
+
+Before moving forward, take a second to consider how well the above tables would match with each other.
+For example, Electrical Engineering and Computer Science (EECS) is a popular major at Berkeley. However, the `majors` dataset didn't have a direct equivalent for it.
+Instead, the `majors` dataset had Electrical Engineering, Electrical Engineering Technologies and Computer Engineering as separate majors. 
+Since in theory students in EECS focus more on computer engineering, we chose to use the computer engineering data for drawing comparions in our final, combined table.
+However, there's room for ambiguity here and that is another potential flaw in our exploration!
+
+The below graph displays all the median salaries for all the majors in our dataset side by side. Feel free to look at the values for a few seconds - do you find anything interesting?
+[Following image is a bar chart of Median Incomes for Selected Majors]
+
+```python
+def display_major_table(major):
+    selected = majors.where("Major", are.contained_in(major))
+    selected.show()
+    
+    ind = np.arange(selected.num_rows) 
+    width = 0.35
+
+    fig, ax = plt.subplots()
+    rects1 = ax.bar(ind - width/2, selected.column("Cal Median"), width,
+                    label='Cal Median', color = "navy")
+    rects2 = ax.bar(ind + width/2, selected.column("Overall Median"), width,
+                    label='Overall Median', color = "goldenrod")
+    
+    ax.set_ylabel('Income')
+    ax.set_xticks(ind)
+    ax.set_xticklabels(selected.column("Major"))
+    ax.set_title("Median Incomes for Selected Majors at Cal (Blue) and Other Universities (Gold)", fontsize= 13)
+    plt.xticks(rotation=90)
+
+
+dropdown_majors = widgets.SelectMultiple(
+    options=majors.column("Major"),
+    description="Major",
+    disabled=False
+)
+widgets.interact(display_major_table, major=dropdown_majors)
+```
+
+```python
+mjs = majors.column("Major")[::-1]
+cal = majors.column("Cal Median")[::-1]
+ovr = majors.column("Overall Median")[::-1]
+
+fig = go.Figure()
+fig.add_trace(go.Bar(name="Cal Median", y=mjs, x=cal, orientation="h"))
+fig.add_trace(go.Bar(name="Overall Median", y=mjs, x=ovr, orientation="h"))
+fig.update_layout(
+    title="Median Incomes for Selected Majors at Cal (Blue) and Other Universities (Red)",
+    yaxis=dict(title=dict(text="Major")),
+    height=1200,
+)
+plot(fig, filename="fig0.html", auto_open=False)
+
+display(HTML("fig0.html"))
+```
+
+## The most lucrative major
+
+Let's imagine that Belfort is a freshman at Berkeley. 
+Belfort is a very unique individual - he is someone who would be equally good at all majors and enjoys all majors equally.
+He also believes that money is the most important thing in the world.
+These assumptions are vastly oversimplifcations and not the case for anyone in real life; but in Economics you'll find that we end up making significant simplifications to abstract away all the potential complications!
+
+Since Belfort is equally good at and happy with all majors at Cal, the major he chooses is purely dependent on how much money he can earn after college with that major.
+Therefore, he will choose the major with the highest median income.
+
+Let us sort our table by the `Cal Median` column in descending order to see which major that would be.
+
+```python
+majors.sort('Cal Median',descending=True).show(10)
+```
+
+However, what if instead of just wanting the maximum amount of cash after college, Belfort was super proud of getting into Berkeley and instead wanted to maximise the amount of additional benefit he recieves from attending Berkeley over other universities?
+Let's sort our table by the `Median Income Difference` column in descending order to see which major would give Belfort the highest additional income from going to Berkeley.  
+
+Do any values in either of our sorted table surprise you? Why might that be?
+
+```python
+majors.sort("Median Income Difference",descending=True).show(10)
+```
+
+Let's now do the opposite and sort our table by the `Median Income Difference` column in ascending order.
+
+```python
+majors.sort("Median Income Difference").show(10)
+```
+
+As you can see in the table, the median income difference for both nuclear engineering and astrophysics is negative. 
+This is rather peculiar, especially since Cal has great programs for both those majors.
+Why would Cal graduates earn less income than the graduates of the average college for those 2 majors? 
+
+*Hint*: consider what each of our datasets describes. Is it an apples-to-apples comparison?
+
+The answer is because the dataset with the median incomes for all colleges actually described recent graduates (people who are 28 and younger, rather than just fresh college graduates.
+It is likely that most people who graduate with Nuclear Engineering and Astrophysics degrees in other colleges tend to stay in the industry for their fields and keep accumulating experience. This would explain why a dataset that includes data from 28 year olds would have a higher median income than fresh Cal graduates in the same field.
+
+### Generalization and Causal Inference
+
+While Belfort would be equally good at and enjoy all majors equally, the implications for him may not generalize for anyone else. 
+
+If you decide to major in EECS, it does not mean that you will have more income post-grad than that from majoring in anything else. For example, someone who is a fantastic artist but not very good at computer science would probably make far more money majoring in art than in EECS. In addition, just because EECS has the highest median income overall doesn't mean that any individual is guaranteed a high median income if they decide to major in EECS. Overall, our results are not completely *generalizable* to others.
+
+The causal effect of majoring in EECS on incomes may also be overstated. Consider the fact that those who major in EECS tend to come from families whose parents are already in working in tech. As a result, these students will likely also find better jobs post-grad. This is what we call a confounding variable -- it is positively correlated with our treatment variable of majoring in EECS. If we do not observe this confounding factor, our analysis would likely overstate the effect of an EECS degree on post-grad incomes.
+
+### A brief discourse on utility 
+
+From seeing our results, you may be wondering: *why doesn't everyone just major in EECS then*? 
+
+For Belfort, money is the only thing in the world that derives happiness. 
+However, this is once again an oversimplication for the vast majority of people in the real world.
+People derive utility from sources other than just their bank account; often, we choose relatively lower-paying jobs in order to derive more happiness in our lives.
+
+**Utility** is a measure of 'satisfaction' or 'happiness' when we consume a good, while **disutility** is a measure of 'unsatisfaction' or 'harm' when we consume a *bad*. In economics, one assumption we will always make is that people will always seek to maximize their utility and minimize their disutility. 
+
+However, we encounter *diminishing marginal utility* as we consume more and more of a good. For example, the utility of the first cookie you eat is likely much higher than the 20th cookie you eat. Similarly, we may encounter *increasing marginal disutility* as we consume more and more of a bad. For example, the disutility of the first hour doing something you dread is probably not as bad as the 10th hour of doing something you dread.
+
+For many of us, money provides a source of utility, while working or the studying required to get there may provide a source of disutility. This presents a tradeoff: for some, the disutility from studying EECS or working as a programmer greatly outweights the utility from the money. Perhaps another job in a different field requires significantly lower disutility to achieve, without much impact on the utility of the income associated with it. This phenomenon would explain why not everyone decides to major in EECS!
+
+## Incomes Over Time: Computer Science vs Economics 
+
+So far our exploration of the data has considered a rather general timeframe of "post graduation", but perhaps it would be more insightful if we dove in and took a look at how incomes differed across various ages. We can do so using the table below that consists of data collected by the [UC ClioMetric History Project](http://uccliometric.org/). 
+By compiling a database of digital student transcripts of all UC undergraduates and then linking California wages to individuals in the transcript database, 
+they were able to produce comprehensive dashboards that visualize demographics, major choices, and long-run incomes of each UC campus’s alumni over the past 70 years. 
+
+
+You can learn more about how they collected the data in [Incomes Over Time ](https://www.universityofcalifornia.edu/infocenter/long-run-methodology), but for now we will just focus on a snapshot of the data. 
+In particular, we are interested in exploring how percentile incomes compare between Cal alumni who majored in computer science versus those who majored in economics. Run the following cell to check it out.
+
+```python
+cs_vs_econ=Table.read_table("cs_econ_by_age.csv")
+cs_vs_econ.show(10)
+```
+
+Here we have a table where each row corresponds to a certain age. 
+It begins with age 23 (typically how old people are when getting their first job out of college), and goes up until age 62 (when people retire from the workforce). 
+
+Let's graph our data using some line plots: line plots are especially useful when trying to visualize trends that change over time. 
+
+Run the following cell to visualize the data. The yellow lines correspond to CS incomes, while the blue lines correspond to Econ incomes. 
+
+[Following image is a graph of "Comparing Incomes of Computer Science Majors vs Economics Majors"]
+
+```python
+from labellines import labelLines
+df = cs_vs_econ.to_df()
+df.plot("Age",color=["#8A3B00","#E03B00","#DD31E6","#BB689C","navy","mediumblue","royalblue","#5A83A4"])
+plt.ylabel("Annual Income in Dollars")
+plt.title("Comparing Incomes of Computer Science Majors vs Economics Majors");
+
+#add labels
+# plt.text(62.5, 65000, 'Econ 25th Percentile')
+# plt.text(62.5, 100000, 'CS 25th Percentile')
+# plt.text(62.5, 120000, 'Econ Median')
+
+xvals = [55, 55, 55, 55, 55, 55, 55, 55]
+lines = plt.gca().get_lines()
+labelLines(lines, align=True, xvals=xvals)
+
+fig = plt.gcf()
+fig.set_size_inches(15, 11)
+```
+
+Let's take a look at how the incomes compare at each percentile. The most notable trend is probably the fact that for the most the part, CS majors typically make more money than Econ majors.
+We especially observe this trend in the 25th and 50th percentiles, and this follows accordingly with our exploration earlier in the notebook that students who majored in L&S Computer Science on average had a higher median income than pretty much every other major.
+
+The graph above helps explain why CS is one of the most popular majors here at Berkeley. 
+Software engineering and related disciplines have a huge demand for fresh talent, so that individuals entering these fields are duly compensated. This in and of itself is a big appeal of students wanting to major in computer science. 
+
+Right off the bat out of college, fresh CS graduates are looking at a six figure income: even the 25th percentile of CS majors are making over $103,000 by the time they are 25. At any given percentile level, CS graduates outearn their Economics counterparts immediately post-graduation.
+
+### Intertemporal Effects
+
+However, we don't just choose our majors based off of the immediate post-graduation income. Intuitively, we would want to maximize our total lifetime income we make. This idea is called *intertermporal utility maximization*: Economists often take into account one's utility across all possible time periods.
+
+Considering lifetime income, which major would be better? Well, it depends: if you are at the bottom 25th or 50th percentile, CS is better. This suggests that the typical Economics major will likely never outearn the the typical CS major.
+
+However, we observe that Economics majors at higher percentiles ultimately overtake CS majors in earnings. Intuitively, this should make sense. Perhaps top Economics majors are more likely to become successful executives later on in their careers while make more money than top CS majors, who continue to stay as software engineers or become engineering managers.
+
+### Risk
+We observe that Economics majors have a much larger spread in their lifetime earnings than CS majors: top Economics graduates make significantly more than bottom Economics graduates, compared to top CS and bottom CS graduates.
+
+This brings to the light the notion of risk. Majoring in CS carries relatively lower variance: whether you are 
+a top notch developer in the 75th percentile or a subpar developer in the 25th percentile, you will probably make a solid living. Since most CS majors typically end up pursuing similar careers in the realms of software development, there is less **variance** in the occupations. There is also less **risk**: most CS majors can expect to land a software engineering job and continue working as one twenty years into their career.
+
+The same story is less true for Economics majors: a top-notch Economics major will significantly out-earn a subpar Economics major. This is partly due to the vastly differing occupation types Economics majors become: analysts, bankers, consultants, just to name a few (Economics is a very versatile degree, afterall). Twenty years down the line, the top Economics majors will have higher ceilings perhaps with a lot of opportunity for leadership or executive positions. There is a lot more **variance** in the types of jobs and thus salaries that Economics majors will attain. In addition, there is a lot more **risk**: not every Economics major will become a top executive, and most end up with more 'average' jobs that earn less than that of similar percentile CS majors.
+
+This means that if you know you are at the top regardless of your major, you will tend to ultimately earn more income as an Economics major than a CS major. But a caveat: it's impossible to guarantee that you will be in the top, no matter how hard you work. Often, life has uncertainties and involves a great deal of luck to become successful.
+
+The table and graph above only considered the percentile incomes of CS and Economics majors, but if you're interested in drawing more comparisons, check out [Long Run outcomes for UC Berkeley Alumni](https://www.universityofcalifornia.edu/infocenter/berkeley-outcomes). You can compare between all types of popular majors at UC Berkeley, and even toggle around with more features like gender, ethnicity, or even specific courses.
+
+## An Afterword
+
+In this demo, we've covered a series of fundamental Economics ideas such as income, utility, and risk. We've also gone over a series of more statistical concepts that are also at the heart of empirical Economics such as causal inference, generalization, and unobserved variables. We did this all using data science techniques and visualizations, while being cognizant of potential sources of error. We will be revisiting a lot of these themes in later parts of the course.
+
+From this exercise you may have noticed that pursuing either Economics, Data Science, or both are great options in terms of post-graduation incomes. This means you've probably chosen wisely to have ended up in this class. 
+
+Welcome to Data 88E!
+

data/F24Textbook_MD/01-demand/01-demand.md

@@ -0,0 +1,152 @@
+---
+title: 01-demand
+type: textbook
+source_path: content/01-demand/01-demand.ipynb
+chapter: 1
+---
+
+```python
+from datascience import *
+
+import sympy
+import matplotlib.pyplot as plt
+import matplotlib as mpl
+import matplotlib.patches as patches
+# plt.style.use('seaborn-muted')
+%matplotlib inline
+
+from IPython.display import display
+import numpy as np
+import pandas as pd
+solve = lambda x,y: sympy.solve(x-y)[0] if len(sympy.solve(x-y))==1 else "Not Single Solution"
+import warnings
+warnings.filterwarnings('ignore')
+```
+
+# Demand Curves
+
+In this chapter, we will explore one of the most foundational yet important concepts in economics: demand curves. The demand curve shows the graphical relationship between the price of a good or service and the quantity demanded for it over a given period of time. 
+In other words, it shows the quantity of goods or services consumers are willing to buy at each market price. 
+The quantity of goods or services demanded or supplied can be modeled as a function of price, as in:  
+
+$$\text{Quantity} = f(\text{Price})$$
+
+Notably, the curve is downwards sloping because of the law of demand, which states that *as the price of a good or service increases, the quantity demanded for it decreases, assuming all other factors are held constant*. 
+
+This should make intuitive sense: as prices increase, fewer people are willing to pay the higher price for the same good. On the other hand, as prices decrease, more people are willing to pay the lower price for the same good. Hence, the quantity demanded of a good or service has an inverse relationship with the price. 
+
+For now, let's assume that the relationship is somewhat linear and can be described as 
+
+$$\text{Quantity}_{d}=-a \cdot \text{Price}_{d} + b$$
+
+We can interpret the equation above as follows: *as the price of a unit increases by 1, there is an $a$ unit decrease in the quantity demanded.* For example, $\text{Quantity}_{d}=-2 \cdot \text{Price}_{d} + 3$ would suggest that a price increase by 1 would decrease overall quantity demanded in the market by 2. 
+
+Price can also be measured as function of quantity to denote demand. In this case, we use an inverse demand function, as it is the inverse function of the demand function above. Since price is a function of quantity, 
+
+$$\text{Price} = f(\text{Quantity})$$
+
+As we are solving for the inverse of the previous demand function, the inverse demand function for the example above is 
+
+$$\text{Price}_{d}=-\frac{1}{2} \cdot \text{Quantity}_{d} + \dfrac{3}{2}$$
+
+## Movement and the Demand Curve
+
+### Shifts in the Demand Curve
+The demand curve can shift in or out based on exogenous events that occur outside of the market. 
+Some factors other than a change in price of the good/service could be changes in:
+
+*  buyer's income
+*  consumer preferences
+*  expectation of future price/supply/demand
+*  changes in the price of related goods
+
+If any of these changes occur and causes the demand for the selected good/service to decrease, then the curve shifts to the left as less of the good or service will be demanded at every price. Similarly, if any of these changes causes the demand for the selected good/service to increase, the curve would shift to the right. This signifies that more of the good or service will be demanded at every price. For example, consumers' incomes decreased during the 2008 recession, thus decreasing overall buying power and shifting the demand curve leftwards; a left shift in the demand curve suggests that consumers would purchase fewer quantities of goods at every price.
+
+```python
+xs = np.linspace(-10, 20, 100)
+ys = -3 * xs + 8
+ys2 = -3 * xs + 3
+
+arx = 5
+ary = -8
+ardx = -1
+ardy = 0
+
+fig = plt.figure(figsize=[7,7])
+plt.plot(xs, ys, label="Original Demand Curve", linewidth=2)
+plt.plot(xs, ys2, label="Shifted Demand Curve", linestyle="dashed", linewidth=2, color="#049348")
+plt.arrow(arx, ary, ardx, ardy, head_width=0.8, head_length=0.5, length_includes_head=True)
+plt.xticks([])
+plt.yticks([])
+plt.xlabel("Quantity", size=16)
+plt.ylabel("Price", size=16)
+plt.xlim([-0.5, 10.5])
+plt.ylim([-25, 10])
+plt.legend()
+fig.savefig('fig-01-demand-orig-shifted-demand.png', bbox_inches='tight')
+plt.close(fig)
+plt.show()
+```
+
+```{figure} fig-01-demand-orig-shifted-demand.png
+---
+name: orig-shifted-demand.png
+alt: Graph of shifting demand curve
+---
+Fig. Graph of a shifting demand curve
+```
+
+### Movements along the Demand Curve
+
+Above, we looked at when exogenous events affect the demand curve. Another concept is movements along the demand curve, which would be considered endogenous events. In movements along the demand curve, changes in price affect the quantity demanded of the good or service. This assumes ceteris paribus, which means holding all other factors constant. This phenomenom is called the Movement of the Demand Curve. With a change in price, the quantity demanded for the good or service can shift the quantity demanded either upward or downward on the demand curve. For example, consider the shift on the graph below from quantity $q_1$ at price $p_1$ to quantity $q_2$ at price $p_2$.
+
+[Following image is a graph of a shift along a demand curve]
+
+```python
+xs = np.linspace(-10, 20, 100)
+ys = -3 * xs + 8
+
+q_1, p_1 = 1, 5
+q_2, p_2 = 8, -16
+
+fig = plt.figure(figsize=[7,7])
+plt.plot(xs, ys, color="#1F578E")
+plt.scatter([q_1, q_2], [p_1, p_2], s=200, color="g", zorder=15)
+plt.arrow(q_1, p_1, q_2 - q_1, p_2 - p_1, color="#CB7432", width=.1, head_length=1.5, head_width=0.5, length_includes_head=True, zorder=-1)
+
+# (q_1, p_1)
+plt.vlines(q_1, -1000, p_1, linestyle="dashed", color="#1F578E")
+plt.hlines(p_1, -1000, q_1, linestyle="dashed", color="#1F578E")
+
+# (q_2, p_2)
+plt.vlines(q_2, -1000, p_2, linestyle="dashed", color="#1F578E")
+plt.hlines(p_2, -1000, q_2, linestyle="dashed", color="#1F578E")
+
+plt.xticks([q_1, q_2], [r"$q_1$", r"$q_2$"], size=14)
+plt.yticks([p_1, p_2], [r"$p_1$", r"$p_2$"], size=14)
+plt.xlabel("Quantity", size=16)
+plt.ylabel("Price", size=16)
+plt.xlim([-0.5, 10.5])
+plt.ylim([-25, 10])
+fig.savefig('fig-01-demand-q1-p1-q2-p2.png', bbox_inches='tight')
+plt.close(fig)
+```
+
+```{figure} fig-01-demand-q1-p1-q2-p2.png
+---
+name: fig-01-demand-q1-p1-q2-p2.png
+alt: Graph of demand curve illustrating change in quantity purchased(q1) at price one(p1) to quanitity purchased(q2) at price  two (p2)
+---
+Fig. Graph of a shifting demand curve as prices changes -- all other factors constant
+```
+
+## Income and Substitution Effect
+
+Changes in the price of a good also leads to 2 effects, which can help explain the law of demand:
+
+- **Income Effect**: Examines how the change in price of the good affects income, which then affects the quantity demanded of a good or service. If the price of a good increases, it would require the consumer to spend more of their income on the good. This dissuades consumers from purchasing more of the good, decreasing the quantity demanded. If the price of a good decreases, consumers would spend less money to receive the same good. This increases the quantity demanded for the good, because consumers would have more income remaining to purchase additional units, given the increase in the amount of purchasing power required to obtain the good.
+- **Substitution Effect**: Examines how the change in price of the good affects its demand relative to other goods and services. If the price of a good increases, then consumers might look at similar goods that function in the same way or yield an equivalent amount of utility as the original good. Consumers are effectively shifting or substituting away from the relatively more expensive good to a cheaper alternative, thereby decreasing the quantity demanded for the original good. The converse is also true: if the price of a good decreases, then consumers that currently purchase other similar goods might start consuming this good instead, because it would cost less money for them to obtain.
+
+
+For example, if the price of gas increases, the higher price for gas might encourage consumers to look at purchasing more efficient cars, such as electric or hybrid cars, thus decreasing the demand for gas. This is the subsitution effect. If the consumer stays with their original car, then they would have less disposable income after purchasing the now-more-expensive gas, so they might purchase less gas. This is the income effect.
+

data/F24Textbook_MD/01-demand/02-example.md

@@ -0,0 +1,131 @@
+---
+title: 02-example
+type: textbook
+source_path: content/01-demand/02-example.ipynb
+chapter: 1
+---
+
+```python
+# HIDDEN
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+import warnings
+warnings.simplefilter("ignore")
+from matplotlib import patches
+from datascience import *
+%matplotlib inline
+from ipywidgets import interact, interactive, fixed
+import ipywidgets as widgets
+```
+
+# An Empirical Demand Curve
+
+Let's examine some historical data on non-organic avocado prices and sales volumes in San Francisco from 2015 to 2018. The original dataset is taken from Kaggle and can be found here:[Kaggle Dataset on Avocado Prices](https://www.kaggle.com/neuromusic/avocado-prices).
+
+```python
+avocados = Table.read_table("avocados.csv") # is it avocados or avocadoes?
+avocados
+```
+
+## Visualizing the Relationship between Price and Quantity
+
+To construct the demand curve, let's first see what the relationship between price and quantity is. We would expect to see a downward-sloping line between price and quantity; if a product's price increases, consumers will purchase less, and if a product's price decreases, then consumers will purchase more. 
+
+To find this, we will create a scatterplot and draw a regression line (by setting `fit_line = True` in the `tbl.scatter` call) between the points. Regression lines are helpful because they consolidate all the datapoints into a single line, helping us better understand the relationship between the two variables. 
+
+[Following image is a scatter plot of a demand curve for Avocados]
+
+```python
+avocados.scatter("Total Volume", "Average Price", fit_line = True, width = 7, height = 7)
+plt.title("Demand Curve for Avocados", fontsize = 16);
+```
+
+The visualization shows a negative relationship between quantity and price, which is exactly what we expected! As we've discussed, as the price increases, fewer consumers will purchase avocados, so the quantity demanded will decrease. This corresponds to a leftward movement along the demand curve. Alternatively, as the price decreases, the quantity sold will increase because consumers want to maximize their purchasing power and buy more avocados; this is shown by a rightward movement along the curve.
+
+
+As a quick refresher, scatterplots can show positive, negative, or neutral correlations among two variables: 
+- If two variables have a positive correlation, then as one variable increases, the other increases too. 
+- If two variables have a negative correlation, then as one variable increases, the other decreases. 
+- If two variables have a neutral correlation, then if one varible increases, the other variable is unaffected. 
+
+Note that scatterplots do not show or prove causation between two variables-- it is up to the data scientists to prove any causation.
+
+## Fitting a Linear Demand Curve
+
+We will now quantify our demand curve using NumPy's [`np.polyfit` function](https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html). `np.polyfit` returns an array of size 2, where the first element is the slope and the second is the $y$-intercept.
+
+It takes 3 parameters:
+- array of x-coordinates
+- array of y-coordinates
+- degree of polynomial 
+
+Because we are looking for a **linear** function to serve as the demand curve, we will use 1 for the degree of polynomial. 
+
+The general template for the demand curve is $y = mx + b$, where $m$ is the slope and $b$ is $y$-intercept.
+
+### Demand with Price as a Function of Quantity
+
+First, we will fit a demand curve expressed in terms of price as a function of quantity. This aligns with the axes of supply and demand curves, in which the quantity is on the x-axis and price is on the y-axis:
+
+$$P(Q) = m\cdot Q + b$$
+
+```python
+slope, intercept = np.polyfit(avocados.column("Total Volume"), avocados.column("Average Price"), 1)
+print("The slope is:", slope)
+print("The intercept is:", intercept)
+```
+
+Thus, our demand curve is $P(Q) = -0.00000109Q+ 2.2495$; The slope is -0.00000109 and $y$-intercept is 2.2495. This means that as the quantity demanded increases by 1 unit (in this case, 1 avocado), we would expect to see price to decrease by 0.00000109 units.
+
+We can plot this line on a graph. Notice that it is the same line as the one when we indicated `fit_line=True` above.
+
+[Following image is a scatter plot of a demand curve for Avocados]
+
+```python
+plt.scatter(avocados.column("Total Volume"), avocados.column("Average Price"))
+quantities = np.arange(400000, 1600000, 1000)
+predicted_prices = slope * quantities + intercept
+plt.plot(quantities, predicted_prices, color = 'red', label = "Demand curve")
+plt.xlabel("Quantity")
+plt.ylabel("Price")
+plt.legend();
+```
+
+### Demand with Quantity as a Function of Price
+
+Our interpretation of the demand curve and its slope above was probably not quite intuitive: changes in quantity demanded likely do not trigger changes in price, but instead it is the other way around. In addition, the slope was tiny: the marginal increase of one additional avocado sold had very little effect from the change in price. 
+
+Thus, it is more intuitive to think the effect a one dollar change in price has on the quantity demanded, and to flip our axes:
+
+$$D(P) = Q(P) = m\cdot P + b$$
+
+In this course, we will write $Q(P)$ and $D(P)$ fairly interchangeably when referencing demand.
+
+One key thing to remember: our axes are flipped for this demand curve! If you want to plot it, note that the left hand side (dependent variable) is actually the x-axis variable, while the independent variable is the y-axis variable.
+
+Fitting our data using this function, we get:
+
+```python
+slope, intercept = np.polyfit(avocados.column("Average Price"), avocados.column("Total Volume"), 1)
+print("The slope is:", slope)
+print("The intercept is:", intercept)
+```
+
+Here, our demand curve is roughly $Q(P) = -476413P+ 1446952$; the slope is -476413 and $y$-intercept is 1446952. This means that as the price increases by 1 unit (in this case, \$1), we would expect to see quantity demanded to decrease by 476413 units (in this case, 476413 avocados). 
+
+Note that this demand curve is not the same as the previous demand curve! It is not simply the inverse of the previous demand curve.
+
+Plotting this line on a graph, we see a slightly different demand curve: can you see what is different between the two?
+
+[Following image is a scatter plot and fit line of a demand curve for Avocados]
+
+```python
+plt.scatter(avocados.column("Total Volume"), avocados.column("Average Price"))
+prices = np.arange(0.2, 2.3, 0.01)
+predicted_quantities = slope * prices + intercept
+plt.plot(predicted_quantities, prices, color = 'red', label = "Demand curve")
+plt.xlabel("Quantity")
+plt.ylabel("Price")
+plt.legend();
+```