# Advanced Parametric-Interactive Plots

There are times where `iplot` is not sufficient for our needs. Maybe we need to use specific plot commands not included in the basic backends. Maybe we needs to create more plots connected to the same parameters. Maybe we need to use more complex widgets.

In such cases we will have to use [holoviz's panel](https://panel.holoviz.org/index.html). It is impossible to learn everything `panel` has to offer with a single example, therefore the Reader is encouraged to explore the project website for demos and tutorials.

The following example will focus on SymPy, but the same technique can be used with any other scientific library.

For the purpose of these tutorials, we are going to use Bokeh as the plotting library.

```python
from sympy import *
init_printing(use_latex=True)
from spb import get_plot_data, plot
from spb.backends.bokeh import BB
import numpy as np
import panel as pn
pn.extension()
```

## Example 1

Let's suppose we have just computed a symbolic transfer function. We'd like to plot the Bode and Nyquist plots.

```python
kp, t, z, o = symbols("k_P, tau, zeta, omega")
G = kp / (I**2 * t**2 * o**2 + 2 * z * t * o * I + 1)
G
```

$\omega$ (the pulsation) is the discretization variable. The parameters are:
* $k_{p}$: proportional gain;
* $\tau$: response time;
* $\zeta$: damping coefficient.

First, we are going to define the widgets that will compose the GUI. For a list of available widgets consult [the following documentation page](https://panel.holoviz.org/user_guide/Widgets.html#types-of-widgets).

In particular, we'd like three float sliders for the aforementioned parameters, and a button to switch from the Bode plot to Nyquist plot. To quickly create the sliders, we will use the `create_widgets` function, which let us use a syntax similar to the definition of parameters in the `iplot` function.

```python
from spb.interactive import create_widgets

sliders = create_widgets({
    kp: (1, 0, 3, 50, "Gain"),
    t: (1, 0, 3, 50, "Response Time"),
    z: (0.2, 0, 1, 50, "Damping Coefficient")
})

# use pn.widgets to create more complex widgets or for maximum customization
plot_type = pn.widgets.RadioButtonGroup(
    name="Plot Type", options=["Bode", "Nyquist"], button_type='success')

# create empty figures
fig1 = plot(xlabel="[rad / s]", ylabel="Amplitude [dB]", xscale="log",
            backend=BB, show=False).fig
fig2 = plot(xlabel="[rad / s]", ylabel="Phase [rad]", xscale="log",
            backend=BB, show=False).fig
fig3 = plot(xlabel="Re(G)", ylabel="Im(G)", backend=BB, show=False).fig
```

Note that figures are also widgets. Here, we used the `plot` function to quickly initialize the figures (note the keyword argument `show=False`).

Now, let's create an interactive data series that will be later used to generate the numerical data. To do so, we call the `get_plot_data` function, providing a `params` dictionary mapping the parameters to default numerical values, and set `get_series=True`. If we set `get_series=False` (which is the default value), then numerical data will be returned instead of the data series.

```python
s = get_plot_data(G, (o, 1e-03, 1e02), params={
        kp: sliders[kp].value,
        t: sliders[t].value,
        z: sliders[z].value,
    }, n=1000, xscale="log", get_series=True)
```

Since Bode plots uses a logarithm x-axis, we also specified `xscale="log"` to use a logarithm spacing in the discretization points.

Now, we need to create a function that will be called whenever we move the sliders. This function will either add or update the data on the figures.

```python
@pn.depends(sliders[kp], sliders[t], sliders[z], plot_type)
def update(kpval, tval, zval, ptval):
    # this step is mandatory: it informs the series of the
    # availability of new parameters
    s.update_data({
        kp: kpval,
        t: tval,
        z: zval,
    })
    x, y = s.get_data()

    if ptval == "Bode":
        if len(fig1.renderers) == 0:
            fig1.line(x, 20 * np.log10(abs(y)), line_width=2)
            fig2.line(x, np.angle(y), line_width=2)
        else:
            fig1.renderers[0].data_source.data.update({'y': 20 * np.log10(abs(y))})
            fig2.renderers[0].data_source.data.update({'y': np.angle(y)})
        return pn.Column(
                pn.pane.Bokeh(fig1, height=250),
                pn.pane.Bokeh(fig2, height=250))
    else:
        if len(fig3.renderers) == 0:
            fig3.line(np.real(y), np.imag(y), line_width=2)
            fig3.line(np.real(y), -np.imag(y), line_width=2)
        else:
            fig3.renderers[0].data_source.data.update({'y': np.imag(y)})
            fig3.renderers[1].data_source.data.update({'y': -np.imag(y)})
        return pn.Column(pn.pane.Bokeh(fig3, height=500))
```

With `@pn.depends(sliders[kp], sliders[t], sliders[z], plot_type)` we are explicitely asking for this function to be executed whenever we move the sliders or click the buttons. Note that `update` will receive the values of the specified widgets.

Next, we update the data series with the new parameters and extract the numerical data from the symbolic transfer function:
```python
    s.update_data({
        kp: kpval,
        t: tval,
        z: zval,
    })
    x, y = s.get_data()
```
Note that we have passed in a dictionary of parameters, similarly to what we would do if we were using `iplot`.

The last thing to note is that the function returns the objects to be updated: in our case, it will return the figures. The Bode plot is going to use 2 figures, therefore the function returns 2 vertically aligned figures. Nyquist plot will only use one figure.

Finally, we need to create the overall layout. Here, we'll use a left-column containing the sliders and button, and a right-column containing the plots:

```python
pn.Row(
    pn.Column(sliders[kp], sliders[t], sliders[z], plot_type),
    update
)
```

## Example 2

In this example we are going to explore a Non-Circular Planetary Drive. The inspiration comes from the following resource, where we can also find useful references:

>  Erik Mahieu "Noncircular Planetary Drive"
>
> http://demonstrations.wolfram.com/NoncircularPlanetaryDrive/
>
> Wolfram Demonstrations Project
>
> Published: January 8 2014

Let's see if we are able to create something similar with this module.

Let the angular motion of the driven gear be:

$$
\Phi (t) = - \left(r t + p_{1} \sin{\left(c r t\right)} + p_{2} \sin{\left(2 c r t\right)}\right)
$$

where:

* $t$ is the time;
* $c$ is the number of lobes;
* $r$ is the velocity ratio;
* $p_{1}, \, p_{2}$ are displacement function parameters.

The time-derivative of $\Phi$ is a transfer function, that is the ratio of the driven and driving angular velocities:

$$
\eta = \frac{d \Phi}{d t} = \frac{\omega_{2}}{\omega_{1}}
$$

Let the angular velocity of the driving gear be $\omega_{1} = 1$. With a pure rolling condition, the radii of the gears at the contact point are given by:

$$
r_{1} = \frac{\eta}{1 + \eta} \qquad r_{2} = \frac{r_{1}}{\eta}
$$

Let's create an interactive model to study these relationships. We are going to wrap everything into a function, `NCPD`, which can be saved into a Python file and later be called from any Jupyter notebook. The following code is well documented, so let's explore it and then run it.

```python
from spb.interactive import create_widgets
import param

def NCPD():
    # symbolic computations
    p1, p2, t, r, c = symbols("p1, p2, t, r, c")
    # driven angular motion
    phi = - (r * t + p1 * sin(c * r * t) + p2 * sin(2 * c * r * t))
    # transfer function
    phip = phi.diff(t)
    # profile of the driver gear
    r1 = phip / (1 + phip)
    # profile of the driven gear
    r2 = -r1 / phip

    # default parameter values
    params = { p1: 0.035, p2: 0.005, r:2, c:3 }

    # Create interactive data series: they will receive an updated
    # params dictionary each time a widget is modified.
    # Note: get_series=True returns a data series.
    #       get_series=False returns the numerical data.

    # line plot of the driven angular motion (function of a single
    # variable, t).
    s1 = get_plot_data(phi, (t, 0, 2 * pi),
                       get_series=True, pt="pinter", params=params)
    # line plot of the transfer function (function of a single
    # variable, t).
    s2 = get_plot_data(phip, (t, 0, 2 * pi),
                       get_series=True, pt="pinter", params=params)
    # line plot of the driver gear (function of a single variable, t).
    # Note that we are generating polar data.
    s3 = get_plot_data(r1, (t, 0, 2 * pi), polar=True,
                       get_series=True, pt="pinter", params=params)
    # line plot of the driven gear (function of a single variable, t).
    # Here, the function is polar wrt to the driven angular motion, Phi,
    # which is symbolically computed.
    s4 = get_plot_data(r2 * cos(phi), r2 * sin(phi), (t, 0, 2 * pi),
                       get_series=True, pt="pinter", params=params)
    # Numerical rotation matrix about the z-axis.
    Rz = lambda k: np.array([[np.cos(k), np.sin(k)], [-np.sin(k), np.cos(k)]])

    # manually create widgets: this allows for full customization
    from bokeh.models.formatters import PrintfTickFormatter
    # by default, float sliders only shows numbers up to the second
    # decimal place. Use PrintfTickFormatter to customize the printed value.
    p1s = pn.widgets.FloatSlider(name="p1", start=-0.035, end=0.035, value=params[p1],
                                step=0.001, format=PrintfTickFormatter(format='%.3f'))
    p2s = pn.widgets.FloatSlider(name="p2", start=-0.02, end=0.02, value=params[p2],
                                step=0.001, format=PrintfTickFormatter(format='%.3f'))
    ts = pn.widgets.FloatSlider(name="Time, t", start=0, end=2 * np.pi, value=0,
                                step=0.05)
    rs = pn.widgets.IntSlider(name="Speed ratio, r", start=2, end=5, value=params[r])
    c_btns = pn.widgets.RadioButtonGroup(
        name='Number of lobes, c',
        options=[1, 2, 3, 4, 5],
        value=params[c],
        button_type='primary')

    # layout the control widgets
    widgets_col = pn.Column(
        ts,
        rs,
        # NOTE: some panel's widgets have a bug: they do not display the
        # name. Hence, we need to add a custom label to let the user
        # know what the widget is representing.
        pn.pane.HTML("<div>Number of lobes, c</div>"),
        c_btns,
        p1s,
        p2s
    )

    # create two empty figures
    fig1 = plot(backend=BB, xlabel="t", ylabel="Phi",
                title="Driven Angular Motion + Transfer Function", show=False).fig
    fig2 = plot(backend=BB, xlabel="x", ylabel="y",
                title="Rolling Curves",
                aspect="equal", show=False).fig

    # customize the first figure: two y-axis, the left one with
    # blue color, the right one with red color
    from bokeh.models import Range1d, LinearAxis
    color1, color2 = "blue", "red"
    fig1.yaxis.axis_line_color = color1
    fig1.yaxis.major_label_text_color = color1
    fig1.yaxis.major_tick_line_color = color1
    fig1.yaxis.minor_tick_line_color = color1
    fig1.yaxis.axis_label_text_color = color1

    fig1.extra_y_ranges = {'phip': Range1d(start=0, end=1)}
    fig1.add_layout(
        LinearAxis(
            y_range_name='phip',
            axis_label='d(Phi)/dt',
            axis_line_color = color2,
            major_label_text_color = color2,
            major_tick_line_color = color2,
            minor_tick_line_color = color2,
            axis_label_text_color = color2
        ),
        'right')

    # update the data series with new parameters and generate
    # new data. this function reduces code repetition.
    def get_data(series, d):
        series.update_data(d)
        return series.get_data()

    # callback function.
    @pn.depends(ts, rs, c_btns, p1s, p2s)
    def update(tval, rval, cval, p1val, p2val):
        # substitution dictionary
        d = {p1: p1val, p2: p2val, r: rval, c: cval}
        # updated values
        x1, y1 = get_data(s1, d)
        source1 = {"xs": x1, "ys": y1}
        x2, y2 = get_data(s2, d)
        source2 = {"xs": x2, "ys": y2}
        x3, y3 = get_data(s3, d)
        source3 = {"xs": x3, "ys": y3}
        x4, y4, _ = get_data(s4, d)

        # apply the rotations to the driven gear
        d[t] = tval
        # NOTE: the following two evaluations are slow because they are
        # computed with SymPy. Alternatively, we could create two lambda
        # functions outside of this update function to speed things up.
        angle = float(phi.evalf(subs=d))
        phipval = float(phip.evalf(subs=d))
        x4, y4 = np.matmul(Rz(angle), np.array([x4, y4]))
        x4 += 1
        x4, y4 = np.matmul(Rz(tval), np.array([x4, y4]))
        source4 = {"xs": x4, "ys": y4}
        # points on fig1 to visualize the current location
        source5 = {"xs": [tval], "ys": [angle]}
        source6 = {"xs": [tval], "ys": [phipval]}

        if len(fig1.renderers) == 0:
            # add data to the figures
            fig1.line("xs", "ys", source=source1, color=color1)
            fig1.line("xs", "ys", source=source2, color=color2, y_range_name="phip")
            fig2.line("xs", "ys", source=source3, color=color1)
            fig2.line("xs", "ys", source=source4, color=color2)
            # add two dots representing the current time
            fig1.circle("xs", "ys", source=source5, color="black")
            fig1.circle("xs", "ys", source=source6, color="black", y_range_name="phip")
        else:
            # update data
            fig1.renderers[0].data_source.data.update(source1)
            fig1.renderers[1].data_source.data.update(source2)
            fig1.renderers[2].data_source.data.update(source5)
            fig1.renderers[3].data_source.data.update(source6)
            fig2.renderers[0].data_source.data.update(source3)
            fig2.renderers[1].data_source.data.update(source4)

        # vertocally center the data of fig1, leaving 5% of whitespace on
        # top and bottom
        m1, M1 = min(y1), max(y1)
        offset1 = abs(M1 - m1) * 0.05
        m2, M2 = min(y2), max(y2)
        offset2 = abs(M2 - m2) * 0.05
        fig1.y_range.update(start=m1-offset1, end=M1+offset1)
        fig1.extra_y_ranges['phip'].update(start=m2-offset2, end=M2+offset2)

        # layout for the figures
        return pn.Column(
            pn.pane.Bokeh(fig1, height=200),
            pn.pane.Bokeh(fig2),
            width=500
        )

    # return the overall layout
    return pn.Row(
        widgets_col,
        update
    )
```

Let's play with the model:

```python
NCPD()
```

As we can see, as soon as we move any slider, an update will be executed. What if our computation takes a long time? In these occasions, it might be better to execute an update only when the click is released from the slider: this will improve user experience as there won't be significant lags. To achieve this behaviour, we have to change the following line of code:

``` python
@pn.depends(ts, rs, c_btns, p1s, p2s)
```

to something like the following:

``` python
@pn.depends(ts.param.value_throttled, rs, c_btns, p1s, p2s)
```

Obviously, we should attach `.param.value_throttled` to all sliders. This is left to the Reader as an exercise.

Finally, a couple of observations:

1. If we are building complicated applications with several widgets and plots, we can again capture the output ``panel`` object and launch a server by calling the ``show()`` method. For example, ``NCPD().show()`` will open the interactive application on a new browser window. In doing so, we can make better use of all the available space.
2. The creation of these interactive widgets is a trial and error procedure. We will have to execute the same code blocks over and over again, generating new figures, new widgets, ... Memory consumption constantly goes up, so it is a good idea to keep an eye on our system resource monitor. If the browser starts lagging, or memory consumption is too high, try to close the browser, close Jupyter server and starts over.

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 6: Pandas and Statsmodels

## Background

In their article "The New Kaldor Facts: Ideas, Institutions, Population, and Human Capital" in the January 2010 issue of the *American Economic Journal: Macroeconomics*, Charles Jones and Paul Romer document six empirical features about macroeconomics that are yet to be adequately explained by a general theory of economic growth (link to article: [https://www.jstor.org/stable/25760291](https://www.jstor.org/stable/25760291)). Among these facts is the observation that total factor productivity is positively correlated with a country's GDP per worker.

Recall the Cobb-Douglas production function with human capital:

\begin{align}
Y & = A K^{\alpha}\left(hL\right)^{1-\alpha}, \label{eqn:production}
\end{align}

where $Y$ denotes the production of final goods and services, $K$ is the stock of physical capital, $L$ is the labor force (either number of workers or number of worker hours), $h$ is human capital per worker, and $A$ is *total factor productivity* or TFP which represents all other factors that might affect production that are not captured by $K$, $L$, or $h$.

For most countries, we can measure $Y$, $K$, $L$ and $h$. Measurements may not be perfect; for example, we often associate $h$ with years of schooling in a country even though, strictly speaking, on-the-job experience and training also contribute to human capital accumulation. Regardless, the point is that the production function *implies* a value for TFP:

\begin{align}
A & = \frac{Y}{K^{\alpha}(hL)^{1-\alpha}}
\end{align}

TFP is like a residual in a linear regression: it is the part of production that is not explained by observable inputs for production. Remarkably, Jones and Romer show that TFP explains a lot of the variation in GDP per worker across countries. This fact is captured in Figure 1.

Figure 1. TFP and GDP per capita across countries  relative to the US. Source: Jones and Romer (2010)

In this Notebook, we will replicate Figure 1 and use a linear regression procedure to estimate the relationship between TFP and GDP per worker. The data required for the exercise is named `cross_country_production.csv` and is available here: https://github.com/letsgoexploring/econ126/raw/master/Data/Csv/cross_country_production.csv
The file contains production data in 2014 for 144 countries. In addition to the name and country code, the file contains production data for 144 countries in 2014:

| Name               | Description               | Units                       |
|--------------------|---------------------------|-----------------------------|
| `gdp`              | real GDP                  | Millions of 2001 US dollars |
| `labor`            | Average number of workers | Millions                    |
| `physical_capital` | Physical capital stock    | Millions of 2001 US dollars |
| `human_capital`    | Human capital per worker  | Index (unitless)            |

We can use the data to compute TFP for each country in the sample and then to replicate Figure 1.

## Replicate Figure from Jones and Romer (2010)

Import the production data. Add new columns containing GDP per worker, physical capital per worker, and TFP. Construct three scatter plots:
1. GDP per worker against physical capital per worker
2. GDP per worker against human capital per worker
3. TFP against GDP per worker

```python
# Import data into a DataFrame called 'df'. Use the leftmost column as the index
df = pd.read_csv('https://github.com/letsgoexploring/econ126/raw/master/Data/Csv/cross_country_production.csv',index_col=0)

# Print the first five rows of the imported data
df.head()
```

      country
      gdp
      labor
      human_capital
      physical_capital

      country_code

      AGO
      Angola
      205371.265625
      8.343885
      1.445598
      7.960541e+05

      ALB
      Albania
      31419.962891
      0.883679
      2.930766
      1.127928e+05

      ARE
      United Arab Emirates
      618051.187500
      5.938807
      2.730336
      2.429822e+06

      ARG
      Argentina
      859908.500000
      18.098450
      2.945663
      2.148251e+06

      ARM
      Armenia
      27410.783203
      1.201936
      3.122178
      4.400342e+04

```python
# Add a column to df called 'gdp_pw' equal to GDP divided by labor divided by 1000 so units are in thousands of $
df['gdp_pw'] = df['gdp']/df['labor']/1000

# Add a column to df called 'physical_capital_pw' equal to physical capital divided by labor divided by 1000 so units are in thousands of $
df['physical_capital_pw'] = df['physical_capital']/df['labor']/1000

# Set alpha to 1/3
alpha =1/3

# Add a column to df called 'tfp' equal to TFP computed using the production function
df['tfp'] = df['gdp']/df['physical_capital']**alpha/(df['labor']*df['human_capital'])**(1-alpha)

# Sort the rows of df so that the GDP per worker column is in ascending order
df = df.sort_values('gdp_pw')
```

```python
# Construct a scatter plot of real GDP per worker against human capital per worker with:
#    1. All countries as blue circles, size 75, opacity (alpha) 0.25, label: 'All countries'
#    2. USA as red square, size 75, opacity (alpha) 0.5, label='USA'
#    3. Add a legend in the upper left corner with scatterpoints=1
#    4. y-axis limits: [-10,250]
#    5. x-axis label: 'Human capital per worker\n(Index)'
#    6. y-axis label: 'Real GDP per worker\n(Thousands of 2001 $)'
#    7. Add a grid if you'd like

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(df['human_capital'],df['gdp_pw'],s=75,alpha=0.25,label='All countries')
ax.scatter(df['human_capital'].loc['USA'],df['gdp_pw'].loc['USA'],marker='s',c='r',s=75,alpha=0.5,label='USA')
ax.legend(scatterpoints=1,loc='upper left')
ax.set_ylim([-10,250])
ax.set_xlabel('Human capital per worker\n(Index)')
ax.set_ylabel('Real GDP per worker\n(Thousands of 2001 $)')
ax.grid()
```

```python
# Construct a scatter plot of real GDP per worker against physical capital per worker with:
#    1. All countries as blue circles, size 75, opacity (alpha) 0.25, label: 'All countries'
#    2. USA as red square, size 75, opacity (alpha) 0.5, label='USA'
#    3. Add a legend in the upper left corner with scatterpoints=1
#    4. y-axis limits: [-10,250]
#    5. x-axis label: 'Physical capital per worker\n(Thousands of 2001 $)'
#    6. y-axis label: 'Real GDP per worker\n(Thousands of 2001 $)'
#    7. Add a grid if you'd like

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(df['physical_capital_pw'],df['gdp_pw'],s=75,alpha=0.25,label='All countries')
ax.scatter(df['physical_capital_pw'].loc['USA'],df['gdp_pw'].loc['USA'],marker='s',c='r',s=75,alpha=0.5,label='USA')
ax.legend(scatterpoints=1,loc='upper left')
ax.set_ylim([-10,250])
ax.set_xlabel('Physical capital per worker\n(Thousands of 2001 $)')
ax.set_ylabel('Real GDP capital per worker\n(Thousands of 2001 $)')
ax.grid()
```

```python
# Construct a scatter plot of TFP against real GDP per worker with:
#    1. All countries as blue circles, size 75, opacity (alpha) 0.25, label: 'All countries'
#    2. USA as red square, size 75, opacity (alpha) 0.5, label='USA'
#    3. Add a legend in the upper left corner with scatterpoints=1
#    4. x-axis limits: [-10,250]
#    4. y-axis limits: [-10,1600]
#    6. x-axis label: 'Real GDP capital per worker\n(Thousands of 2001 $)'
#    7. y-axis label: 'TFP (Unitless)'
#    8. Add a grid if you'd like

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(df['gdp_pw'],df['tfp'],s=75,alpha=0.25,label='All countries')
ax.scatter(df['gdp_pw'].loc['USA'],df['tfp'].loc['USA'],marker='s',c='r',s=75,alpha=0.5,label='USA')
ax.legend(scatterpoints=1,loc='upper left')
ax.set_xlim([-10,250])
ax.set_ylim([-10,1600])
ax.set_xlabel('Real GDP per worker\n(Thousands of 2001 $)')
ax.set_ylabel('TFP (Unitless)')
ax.grid()
```

The strong positive relationship between GDP per worker and TFP means that *even after accounting for their lower levels of human capital per worker and physical capital per worker*, workers in lower-income countries are less productive because they use what human and physical capital the *do* have less efficiently than workers in higher-income countries. And since TFP isn't directly observable, we still don't know exactly why.

## Estimate Relationship Between TFP and GDP per Worker

The Statsmodels module is an excellent tool for a broad range of statistical applications. Learn more about Statsmodels here: https://www.statsmodels.org/stable/index.html. We can use the module to estimate the relationship between TFP and GDP per worker in our data.

Since the scatter plot reveals some curvature in the relationship between TFP and GDP per worker, let's specifiy the linear regression model with the variables in logs:

\begin{align}
\log \text{TFP}_i & = \beta_0 + \beta_1 \log \text{GDP}_i + \epsilon_i
\end{align}

Once we've obtained the estimated coefficients $\hat{\beta}_0$ and $\hat{\beta}_1$, we can compute predicted values for TFP $\widehat{TFP}_i$ using by setting $\epsilon_i=0$ and solving the regression equation for TFP:

\begin{align}
\widehat{TFP}_i & = \exp\left(\hat{\beta}_0 + \hat{\beta}_1 \log \text{GDP}_i\right)
\end{align}

```python
# Import the Statsmodels module
import statsmodels.api as sm
```

```python
# Create a variable called 'y' that will be to the dependent variable in the regression
y = np.log(df['tfp'])

# Create a variable called 'x' that will be to the independent variable in the regression
x = np.log(df['gdp_pw'])

# Use the function sm.add_constant() to add a constant to x
x = sm.add_constant(x)

# Print the first five rows of x
print(x.head())
```

                  const    gdp_pw
    country_code
    CAF             1.0  0.315622
    BDI             1.0  0.611815
    NER             1.0  0.893346
    MWI             1.0  0.913269
    LBR             1.0  0.960224

```python
# Create an OLS model using Statsmodels
model = sm.OLS(y,x)

# Fit the OLS model to data
results = model.fit()

# Print the summary2() method of results to print summary of regression results:
print(results.summary2())
```

                     Results: Ordinary least squares
    =================================================================
    Model:              OLS              Adj. R-squared:     0.905
    Dependent Variable: tfp              AIC:                -78.3129
    Date:               2021-01-19 12:57 BIC:                -72.3732
    No. Observations:   144              Log-Likelihood:     41.156
    Df Model:           1                F-statistic:        1359.
    Df Residuals:       142              Prob (F-statistic): 1.36e-74
    R-squared:          0.905            Scale:              0.033524
    -------------------------------------------------------------------
                 Coef.    Std.Err.      t      P>|t|    [0.025   0.975]
    -------------------------------------------------------------------
    const        4.1585     0.0463   89.8804   0.0000   4.0670   4.2499
    gdp_pw       0.4936     0.0134   36.8661   0.0000   0.4671   0.5200
    -----------------------------------------------------------------
    Omnibus:              6.608        Durbin-Watson:           1.890
    Prob(Omnibus):        0.037        Jarque-Bera (JB):        6.185
    Skew:                 0.460        Prob(JB):                0.045
    Kurtosis:             3.430        Condition No.:           11
    =================================================================

Estimated coefficients are stored in the `params` attribute of `results`. Residuals of the regression are stored in the attribute `resid`. And predicted values of the dependent variable are stored in `fittedvalues`.

```python
# Print estimated coefficients of linear regression
print(results.params)
```

    const     4.158469
    gdp_pw    0.493573
    dtype: float64

```python
# Print R-squared of the regression
print(results.rsquared)
```

    0.9054032510077362

**Question**

1. What is the interpretation of the coefficient on log GDP per worker?
2. What is the R-squared of the regression? What does it mean?

**Answer**

1. A one percent increase in GDP per worker increases TFP by one half of a percent.
2. The R-squared of the regression is 0.905 and means that GDP per worker explains over 90 percent of the variation in TFP.

```python
# Print the first five rows of residuals.
print(results.resid.head())
```

    country_code
    CAF   -0.368663
    BDI    0.253711
    NER   -0.036212
    MWI   -0.222537
    LBR   -0.093016
    dtype: float64

```python
# Print the mean of the residuals of the regression.
print(results.resid.mean())
```

    2.917419392487217e-15

```python
# Create a variable called 'yhat' equal to the fitted values of y:
yhat = results.fittedvalues

# Print the first five rows of yhat
print(yhat.head())
```

    country_code
    CAF    4.314252
    BDI    4.460445
    NER    4.599401
    MWI    4.609234
    LBR    4.632410
    dtype: float64

Recall that we can use estimated parameters can be used to compute out-of-sample predictions:

\begin{align}
\widehat{TFP}_i & = \exp\left(\hat{\beta}_0 + \hat{\beta}_1 \log \text{GDP}_i\right)
\end{align}

```python
# Create variable called 'gdp_values' that is an array from 0.1 to 300
gdp_values = np.arange(0.1,300,0.001)

# Create variable called 'beta0' equal to the constant of the regression
beta0 = results.params['const']

# Create variable called 'beta1' equal to the coefficient on log GDP per worker from the regression
beta1 = results.params['gdp_pw']

# Create a variable called 'ahat' that is equal to the predicted values of TFP corresponding to gdp_values
ahat = np.exp(beta0 + beta1*np.log(gdp_values))
```

```python
# Recreate the scatter plot of TFP against real GDP per worker with:
#    1. All features and settings from the previous plot
#    2. A plot of ahat against gdp_values; linewidth: 3, opacity: 0.5, label: 'OLS', color: magenta (c='m')

fig = plt.figure()
ax = fig.add_subplot(1,1,1)
ax.scatter(df['gdp_pw'],df['tfp'],s=75,alpha=0.25,label='All countries')
ax.scatter(df['gdp_pw'].loc['USA'],df['tfp'].loc['USA'],marker='s',c='r',s=75,alpha=0.5,label='USA')
ax.plot(gdp_values,ahat,c='m',lw=3,alpha=0.5,label='OLS')
ax.legend(scatterpoints=1,loc='upper left')
ax.set_xlim([-10,250])
ax.set_ylim([-10,1600])
ax.set_xlabel('Real GDP per worker\n(Thousands of 2001 $)')
ax.set_ylabel('TFP (Unitless)')
ax.grid()
```


# Kullback-Leibler divergence

This reading will review the definition of the Kullback-Leibler (or KL) divergence, look at some of its important properties, see how it can be computed in practice with TensorFlow Probability.

```python
import tensorflow as tf
import tensorflow_probability as tfp
tfd = tfp.distributions
print("TF version:", tf.__version__)
print("TFP version:", tfp.__version__)

# Additional packages for the reading
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.patches import Ellipse
```

    TF version: 2.6.0
    TFP version: 0.14.1

## Introduction

As you have already seen, the KL divergence is used in variational inference to score the dissimilarity between two distributions. In this reading, we will examine KL divergence more closely. We will see the definition of the KL divergence and some important properties, as well as how it can be computed using `tfd.kl_divergence` and Monte Carlo estimation.

## Definition of the Kullback-Leibler divergence

Given two probability density or mass functions $q(x)$ and $p(x)$, the Kullback-Leibler divergence between them is defined as

\begin{equation}
    D_{KL}\big[q \ || \ p\big] =\begin{cases}
                                        \text{E}_{X\sim q}\big[
                                           \log q(X) - \log p(X)\big] &\text{if } p(x) = 0 \implies q(x) = 0,\\
                                        \infty &\text{otherwise.}
                                       \end{cases}
\end{equation}

The condition $p(x) = 0 \implies q(x) = 0$ - _absolute continuity_ - ensures that the $\log$ in the expectation is well-defined for all $x$ in the support of $q$.

As was mentioned, the KL divergence is a score for the disagreement of two distributions in their placement of probability mass. A smaller score indicates a greater degree of agreement.

## Properties

The Kullback-Leibler divergence is asymmetric. In general,

\begin{equation}
    D_{KL}\big[q \ || \ p\big] \neq D_{KL}\big[p \ || \ q \big]
\end{equation}

In variational inference, $q$ is the approximating distribution, while $p$ is the distribution being approximated. The other KL divergence - $D_{KL}[p \ || \ q  ]$ - is also sometimes used as a loss function, for reasons that will become clear later in this reading.

### Gibbs' inequality

A crucial property of the KL divergence is that for all $q$ and $p$,

\begin{equation}
    D_{KL}\big[q \ || \ p\big] \geq 0,
\end{equation}

with equality if and only if $q(x) = p(x)$ almost everywhere. This property is very useful when we are trying to learn a $q$ that is similar to a $p$: if $D_{KL}[q \ || \ p] = 0$, then we know that $q$ is identical to $p$.

### What causes KL divergence to increase?

As an example, take $q(x)$ and $p(x)$ to be probability mass functions, and let $\mathcal{X}$ be $q$'s support. Provided $q$ is absolutely continuous with respect to $p$, we have

\begin{equation}
    D_{KL}\big[q \ || \ p\big] = \sum_{x \in \mathcal{X}} q(x) \log \frac{q(x)}{p(x)}.
\end{equation}

Values of $x$ that $p$ assigns mass to but $q$ does not do not feature in this sum. Superficially, this may suggest that divergence is not increased if $q$ fails to place mass where $p$ does. However, $q$ is a probability mass function, so will inevitably place more mass than $p$ at some other value(s) of $x$. At those other locations, $\log q(x)/p(x) > 0$, so the divergence is increased.

On the other hand, if $q$ places probability mass where $p$ does not, then $D_{KL}\big[q \ || \ p\big]$ is $+\infty$ - the KL divergence severely penalizes $q$ for locating probability mass where $p$ does not!

From this combination of effects, we can conclude that

\begin{equation}
    \text{support}(q) \subseteq \text{support}(p) \implies D_{KL}\big[ q \ || \ p \big] < \infty,
\end{equation}

while

\begin{equation}
    \text{support}(p) \subset \text{support}(q) \implies D_{KL}\big[ q \ || \ p \big] = \infty
\end{equation}

Consequently, the KL divergence favours distributions $q$ that have a support contained in the target distribution's (i.e. $p$'s).

The diagram below illustrates how the KL divergence is affected by the support of two bivariate density functions $q$ and $p$. The hatched regions indicate the support of either function.

```python
_, axs = plt.subplots(1, 2, sharex=True, sharey=True, figsize=(11, 5))

delta = 45.0  # degrees

q_ell_inf = Ellipse((0, 0), 2, 1.5, 45, ec='blue', fc='none',
                    alpha=0.5, label='q(x)', hatch='/')
q_ell_fin = Ellipse((0, 0), 0.5, 0.75, 45, ec='blue', fc='none',
                    alpha=0.5, label='q(x)', hatch='/')
p_ell_inf = Ellipse((0, 0), 1, 1, 45, ec='red', fc='none',
                    alpha=0.5, label='p(x)', hatch='\\')
p_ell_fin = Ellipse((0, 0), 1, 1, 45, ec='red', fc='none',
                    alpha=0.5, label='p(x)', hatch='\\')

# KL divergence is infinite
for ell in [q_ell_inf, p_ell_inf]:
    axs[0].add_artist(ell)
axs[0].legend([q_ell_inf, p_ell_inf], ['Support of q', 'Support of p'], loc='lower right')
axs[0].get_xaxis().set_ticks([])
axs[0].get_yaxis().set_ticks([])

# KL divergence is finite
for ell in [q_ell_fin, p_ell_fin]:
    axs[1].add_artist(ell)
axs[1].legend([q_ell_fin, p_ell_fin], ['Support of q', 'Support of p'], loc='lower right')
axs[1].get_xaxis().set_ticks([])
axs[1].get_yaxis().set_ticks([])

axs[0].set_title(r'$D_{KL}[q \ || \ p] = +\infty$')
axs[1].set_title(r'$D_{KL}[q \ || \ p]$ is finite but non-zero')
plt.xlim(-1, 1)
plt.ylim(-1, 1);
```

## Computing KL divergence in TensorFlow

For some choices of $q$ and $p$, the KL divergence can be evaluated to a closed-form expression.

`tfd.kl_divergence` computes the KL divergence between two distributions analytically, provided the divergence in question has been implemented in the TensorFlow Probability library.

Below is an example that uses `tfd.kl_divergence` to compute $D_{KL}\big[q \ || \ p \big]$ when $q$ and $p$ are univariate normal distributions.

```python
# Simple example

mu_q = 0.
sigma_q = 1.
mu_p = 0.
sigma_p = 0.5
distribution_q = tfd.Normal(loc=mu_q, scale=sigma_q)
distribution_p = tfd.Normal(loc=mu_p, scale=sigma_p)

tfd.kl_divergence(distribution_q, distribution_p) # D_{KL}[q || p]
```

    <tf.Tensor: shape=(), dtype=float32, numpy=0.8068528>

Let's check this value. The KL divergence between two univariate normal distributions can be derived directly from the definition of the KL divergence as

\begin{equation}
    D_{KL}\big[ q \ || \ p\big] = \frac{1}{2}\bigg(\frac{\sigma_q^2}{\sigma_p^2} + \frac{(\mu_q - \mu_p)^2}{\sigma_p^2}  + 2\log \frac{\sigma_p}{\sigma_q} - 1\bigg)
\end{equation}

The value of this function should be equal to that returned by `kl_divergence(distribution_q, distribution_p)`.

```python
# Analytical expression for KL divergence between two univariate Normals

0.5*( (sigma_q/sigma_p)**2 + ((mu_q - mu_p)/sigma_p)**2 + 2*np.log(sigma_p/sigma_q) - 1)
```

    0.8068528194400546

Sure enough, it is.

If a batch of distributions is passed to `kl_divergence`, then a batch of divergences will be returned. `kl_divergence` also supports broadcasting.

```python
# Batch example with broadcasting

distributions_q = tfd.Normal(loc=[0., 1.], scale=1.)
distribution_p = tfd.Normal(loc=0., scale=0.5)
```

```python
# Notice the batch_shape

distributions_q
```

    <tfp.distributions.Normal 'Normal' batch_shape=[2] event_shape=[] dtype=float32>

```python
# [D_{KL}[q_1 || p], D_{KL}[q_2 || p]

tfd.kl_divergence(distributions_q, distribution_p)
```

    <tf.Tensor: shape=(2,), dtype=float32, numpy=array([0.8068528, 2.8068528], dtype=float32)>

`kl_divergence` provides a convenient way of computing the KL divergence for many TensorFlow distributions. As a rule of thumb, it will evaluate successfully provided you pass in two distributions of the same parametric family.

```python
# An example with another distribution

beta_q = tfd.Beta(concentration1=12, concentration0=3)
beta_p = tfd.Beta(concentration1=9, concentration0=3)

tfd.kl_divergence(beta_q, beta_p)
```

    WARNING:tensorflow:@custom_gradient grad_fn has 'variables' in signature, but no ResourceVariables were used on the forward pass.

    <tf.Tensor: shape=(), dtype=float32, numpy=0.09615421>

```python
# An example with a multivariate distribution

cov_q = np.array([[1., 0.5], [0.5, 1.]])
cov_p = np.array([[1., 0.], [0., 1.]])
mvtnormal_q = tfd.MultivariateNormalTriL(loc=[0., 0.], scale_tril=tf.linalg.cholesky(cov_q))
mvtnormal_p = tfd.MultivariateNormalTriL(loc=[0., 0.], scale_tril=tf.linalg.cholesky(cov_p))

tfd.kl_divergence(mvtnormal_q, mvtnormal_p)
```

    <tf.Tensor: shape=(), dtype=float64, numpy=0.14384103622589053>

To see a complete list of distributions for which a KL method is defined, refer to `help(tfd.kl_divergence)`.

If you pass `kl_divergence` a pair distributions for which a KL divergence method is not implemented, an error will be raised:

```python
# uniform_q and beta_p are both uniform distributions with support [0, 1]

uniform_q = tfd.Uniform(low=0., high=1.)
beta_p = tfd.Beta(concentration1=0., concentration0=0.)
```

```python
# kl_divergence has no method for computing their divergence

try:
    tfd.kl_divergence(uniform_q, beta_p)
except Exception as e:
    print(e)
```

    No KL(distribution_a || distribution_b) registered for distribution_a type Uniform and distribution_b type Beta

### When `kl_divergence` fails

If you do not have a closed-form expression for your KL divergence, and it is not implemented in `tfd.kl_divergence`, then you can make a Monte Carlo estimate of it. Simply sample $n$ values $x_1, \ldots, x_n$ from $q$, then evaluate the estimate

\begin{equation}
    \frac{1}{n}\sum_{i=1}^n \log\big[q(x_i)\big] - \log\big[p(x_i)\big]
\end{equation}

In general, the Monte Carlo estimator is unbiased and its variance is inversely proportional to $n$.

To show how the variance of the Monte Carlo estimator varies with $n$, let's attempt to estimate $D_{KL}\big[q \ | \ p\big]$ when $q$ and $p$ are univariate normal distributions. We'll make many estimates for several values of $n$, then plot their absolute error as a function of $n$.

We'll start by evaluating the exact value $D_{KL}\big[q \ | \ p\big]$ using `kl_divergence`. Bear in mind that the Monte Carlo estimate will only be useful in situations where this not possible!

```python
# Evaluate the exact KL divergence

distribution_q = tfd.Normal(loc=0., scale=1.)
distribution_p = tfd.Normal(loc=0., scale=0.5)

exact_kl_divergence = tfd.kl_divergence(distribution_q, distribution_p).numpy() # D_{KL}[q || p]
exact_kl_divergence
```

    0.8068528

Next, we'll define a function for making a Monte Carlo estimate for a given $q$, $p$, and $n$.

```python
# Function to estimate the KL divergence with Monte Carlo samples

def monte_carlo_estimate_of_kl_divergence(n, q_sampler, q_density, p_density):
    '''
        Computes a Monte Carlo estimate of D_{KL}[q || p] using
        n samples from q_sampler.

        q_sampler is a function that receives a positive integer
        and returns as many samples from q.

        Given samples x_1, ..., x_n from q_sampler, the Monte Carlo
        estimate is

            \frac{1}{n}\sum_{i=1}^n \log(q(x_i)) - \log(p(x_i))

        where q and p are density/mass functions.
    '''
    x = q_sampler(n)
    KL_estimate = np.mean(np.log(q_density(x)) - np.log(p_density(x)))
    return(KL_estimate)
```

The code below shows how this function can be used to make a single estimate.

```python
# Single MC estimate

n = 1000 # number of samples used in MC estimate
q_sampler = distribution_q.sample
q_density = distribution_q.prob
p_density = distribution_p.prob

monte_carlo_estimate_of_kl_divergence(n, q_sampler, q_density, p_density)
```

    0.7580066

To see how the estimator's variance decreases with increasing $n$, let's evaluate a few hundred estimates for each point in a grid of $n$ values.

```python
# Create a grid of 8 points

n_grid = 10**np.arange(1, 8)
samples_per_grid_point = 100  # Number of MC estimates to make for each value of n
```

```python
# Array to store results

kl_estimates = np.zeros(shape=[samples_per_grid_point, len(n_grid), 2])
```

```python
# Make 100 MC estimates for each value of n, store the results in kl_estimates

for sample_num in range(samples_per_grid_point):
    for grid_num, n in enumerate(n_grid):
        kl_estimates[sample_num, grid_num, 0] = n
        kl_estimates[sample_num, grid_num, 1] = monte_carlo_estimate_of_kl_divergence(n,
                                                            q_sampler, q_density, p_density)
```

```python
# Compute RMSE of estimates (this is approximately equal to the standard deviation of the MC estimator)

rmse_of_kl_estimates = np.sqrt(np.mean((kl_estimates[:, :, 1] -
                                            exact_kl_divergence)**2, axis=0))
```

```python
# Compute absolute error of the MC estimates

abs_error_of_kl_estimates = abs(kl_estimates[:, :, 1].flatten() - exact_kl_divergence)
```

```python
# Plot the results

_, ax = plt.subplots(1, 1, figsize=(15, 5))
plt.xlabel(r'Number of samples in Monte Carlo estimate, $n$')
ax.scatter(kl_estimates[:, :, 0],
           abs_error_of_kl_estimates,
           marker='.', color='red',
           alpha=0.1, label='Absolute error of Monte Carlo estimates')
ax.plot(n_grid, rmse_of_kl_estimates, color='k', label='RMSE of Monte Carlo estimates')
ax.set_xscale('log'); ax.set_yscale('log'); ax.set_ylim([1e-6, 10])
ax.legend();
```

You can see that the gradient of the estimates' RMSE, an estimate of the MC estimator's standard devation, with respect to $n$ is $-\frac{1}{2}$. This is unsurprising: the estimator's variance is inversely proportional to $n$, so its log standard deviation is a linear function of $\log n$ with gradient $-\frac{1}{2}$. As $n$ increases, the Monte Carlo estimates approach the exact value of the KL divergence.

### Summary

You should now feel confident about how the Kullback-Leibler divergence is motivated and defined, what its key properties and why it is used in variational inference, and how it can be computed or estimated in TensorFlow.

### Further reading and resources

* TensorFlow documentation on `tfd.kl_divergence`: https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/kl_divergence

## Appendix

#### Information gain, relative entropy, and Bayesian inference

This section provides further context for the Kullback-Leibler divergence. It is not essential, but it will give you a more complete understanding of what the divergence measures.

The Kullback-Leibler divergence has its origins in information theory. The Shannon entropy, defined as

\begin{equation}
    H(P) := E_{X \sim P(x)}[-\log P(X) ]
\end{equation}

is the greatest lower bound on the average number of nats ($\log 2$ nats are equal to $1$ bit) required to losslessly encode an observation sampled from from $P(x)$. This is an informal statement of a result known as the _source coding theorem_. $-\log P(x)$ is the number of bits used to encode $x$ in the lossless encoding scheme.

Say that a lossless compression algorithm instead encodes observations using a scheme that would be optimal for distribution $Q(x)$. Then the average number of of bits required to encode an observation sampled from $P(x)$ would be

\begin{equation}
    H(P, Q) := E_{X \sim P(x)}[-\log Q(X)]
\end{equation}

This is quantity is referred to as the _cross-entropy_ between $P$ and $Q$. Since $H(P)$ is the minimum average information for encoding observations from $P(x)$ by definition, it follows that $H(P, Q) \geq H(P)$.

The Kullback-Leibler divergence is defined as the average additional information required to encode observations from $P(x)$ using an optimal code for $Q(x)$:

\begin{align}
    D_{KL}(P \ || \ Q) &:= E_{X \sim P(x)}[-\log Q(X)] - E_{X \sim P(x)}[-\log P(X)] \\
                       &= H(P, Q) - H(P)
\end{align}

The KL divergence therefore tells us how inefficient the optimal coding scheme for $Q$ is when applied to data source $P$.

That KL divergence is the difference between a cross-entropy and a Shannon entropy sheds light on why the KL divergence has another moniker - _relative entropy_.

Alternatively, we might consider encoding observations in the context of Bayesian inference. Let $P(y)$ be the prior and $P(y|x)$ be the posterior. Then the Kullback-Leibler divergence

\begin{equation}
    D_{KL}(P(y|x) \ || \ P(y)) = E_{Y \sim P(y|x)}[-\log P(Y)] - E_{Y \sim P(y|x)}[-\log P(Y|x)]
\end{equation}

is the average number of bits saved if observations are encoded using an optimal code for the posterior rather than the prior. In this sense, the KL divergence tells us how much information is gained by conditioning on $X$.


# Pricing with Rational Inattnetion: Sims (2011)

This notebook replicates [Sims (2011)](http://sims.princeton.edu/yftp/RIMP/handbookChapterRI2.pdf) from the Handbook of Monetary Economics using the methods and the [solver](https://github.com/choongryulyang/dynamic_multivariate_RI) from [Afrouzi and Yang (2019)](http://www.afrouzi.com/dynamic_inattention.pdf) that was adapted for Julia by [Miguel Acosta](https://www.acostamiguel.com/home).

## Contents
* [Setup](#setup)
* [Initialization](#param)
* [Solution](#solution)
    * [Benchmark Parameterization](#benchmark)
    * [Lower Cost of Attention: $\omega = 0.1$](#lowomega)
    * [Other Discount Factors: $\beta \in \{0,1\}$](#betas)
* [Impulse Response Functions](#figures)
    * [Benchmark Parameterization](#fig_benchmark)
    * [Lower Cost of Attention: $\omega = 0.1$](#fig_lowomega)
    * [Other Discount Factors: $\beta \in \{0,1\}$](#fig_betas)
* [Extensions](#extensions)
    * [Transition Dynamics of Attention](#trip)
    * [Impulse Response Functions with Information Treatment](#trip_irfs)

## Setup

The problem in [Sims (2011)](http://sims.princeton.edu/yftp/RIMP/handbookChapterRI2.pdf), as it appears on page 21, with slight change of notation,
\begin{align}
	          & \min_{\{\Sigma_{t|t}\succeq 0\}} \mathbb{E}_0\left[\sum_{t=0}^\infty\
	\beta^t \left(tr(\Sigma_{t|t}\mathbf{H}\mathbf{H}')+\omega\log\left(\frac{|\Sigma_{t|t-1}|}{|\Sigma_{t|t}|}\right)\right)\right] \\
	s.t.\quad &
	\Sigma_{t+1|t}=\mathbf{A}\Sigma_{t|t}\mathbf{A}'+\mathbf{Q}\mathbf{Q}'\\
	          & \Sigma_{t|t-1}-\Sigma_{t|t} \text{ positive semi-definite}
\end{align}

where
\begin{align}
	\mathbf{H} = \left[\begin{array}{c} 1 \\ 1\end{array}\right],
    \quad
    \mathbf{A} = \left[\begin{array}{cc}
							0.95 & 0\\
							0 & 0.4\\
					   \end{array}\right],
	\quad
	\mathbf{Q} = \left[\begin{array}{cc}
							\sqrt{0.0975} & 0\\
							0 			& \sqrt{0.86}\\
					   \end{array}\right]
\end{align}
We have renamed the parameters so that the problem directly maps to a D.R.I.P. Otherwise, the problem is the same.

## Initialization
Include the solver:

```julia
include("DRIP.jl");
```

Import supporting packages and aux. commands:

```julia
using Printf, Plots, LaTeXStrings, BenchmarkTools; pyplot();
Base.show(io::IO, f::Float64) = @printf io "%1.3f" f #rounds up floats for display
```

Set parameters:

```julia
## Primitives of D.R.I.P.
β = 0.9;
ω = 1.0;
A = [0.95 0.0; 0.0 0.4];
Q = [√0.0975 0.0; 0.0 √0.86];
H = [1.0; 1.0];
```

## Solution and Performance
### Benchmark Parameterization
Solve and display the optimal posterior covariance matrix:

```julia
sol_bp = solve_drip(ω,β,A,Q,H);
display(sol_bp.Σ_p)
```

    2×2 Array{Float64,2}:
      0.359  -0.177
     -0.177   0.795

Performance for random values of $\omega\in [0,2]$:

```julia
@benchmark solve_drip(ω,β,A,Q,H) setup = (ω = 2*rand())
```

    BenchmarkTools.Trial:
      memory estimate:  363.28 KiB
      allocs estimate:  2145
      --------------
      minimum time:     269.828 μs (0.00% GC)
      median time:      427.627 μs (0.00% GC)
      mean time:        482.297 μs (10.37% GC)
      maximum time:     8.532 ms (93.33% GC)
      --------------
      samples:          10000
      evals/sample:     1

Performance for random values of $\beta\in[0,1]$:

```julia
@benchmark solve_drip(ω,β,A,Q,H) setup = (β = rand())
```

    BenchmarkTools.Trial:
      memory estimate:  366.36 KiB
      allocs estimate:  2152
      --------------
      minimum time:     317.774 μs (0.00% GC)
      median time:      480.493 μs (0.00% GC)
      mean time:        539.690 μs (9.93% GC)
      maximum time:     8.687 ms (94.01% GC)
      --------------
      samples:          9222
      evals/sample:     1

### Lower Cost of Attention: $\omega = 0.1$
Solve and display the optimal posterior covariance matrix:

```julia
sol_lω = solve_drip(0.1,β,A,Q,H);
display(sol_lω.Σ_p)
```

    2×2 Array{Float64,2}:
      0.320  -0.304
     -0.304   0.386

### Different Discount Factors: $\beta \in \{0,1\}$
Solve the model for $\beta=0$ and $\beta=1$ to compare with the benchmark value of $\beta=0.9$:

```julia
sol_lβ = solve_drip(ω,0,A,Q,H); display(sol_lβ.Σ_p);
```

    2×2 Array{Float64,2}:
      0.495  -0.152
     -0.152   0.809

```julia
sol_hβ = solve_drip(ω,1,A,Q,H); display(sol_hβ.Σ_p);
```

    2×2 Array{Float64,2}:
      0.338  -0.178
     -0.178   0.800

## Impulse Response Functions
### Benchmark Parameterization
Get the IRFs:

```julia
T = 25; #length of IRFs
irfs_bp = dripirfs(sol_bp,T);
```

Plot IRFs:

```julia
p1 = plot(1:T, [irfs_bp.x[1,1,:], irfs_bp.a[1,1,:]],
    title             = L"IRFs to Slow-Moving Shock ($\rho = 0.95$)",
    label             = ["Shock" "Price"],
    color             = [:darkgray :black],
    marker            = [:circle :square],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p2 = plot(1:T, [irfs_bp.x[2,2,:], irfs_bp.a[1,2,:]],
    title             = L"IRFs to Fast-Moving Shock ($\rho = 0.4$)",
    label             = ["Shock" "Price"],
    color             = [:darkgray :black],
    marker            = [:circle :square],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p = plot(p1,p2,
    layout     = (2,1),
    xlabel     = "Time",
    lw         = 2,
    xticks     = (1:2:T),
    xlim       = (0,T+1),
    fontfamily = "serif",
    legend     = :topright,
    legendfont = font(12),
    tickfont   = font(12),
    size       = (900,550),
    framestyle = :box)
```

### Lower Cost of Attention: $\omega=0.1$
Get the IRFs:

```julia
T = 25; #length of IRFs
irfs_lω = dripirfs(sol_lω,T);
```

Plot IRFs:

```julia
p1 = plot(1:T, [irfs_lω.x[1,1,:], irfs_lω.a[1,1,:]],
    title             = L"IRFs to Slow-Moving Shock ($\rho = 0.95$)",
    label             = ["Shock" "Price"],
    color             = [:darkgray :black],
    marker            = [:circle :square],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p2 = plot(1:T, [irfs_lω.x[2,2,:], irfs_lω.a[1,2,:]],
    title             = L"IRFs to Fast-Moving Shock ($\rho = 0.4$)",
    label             = ["Shock" "Price"],
    color             = [:darkgray :black],
    marker            = [:circle :square],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p = plot(p1,p2,
    layout     = (2,1),
    xlabel     = "Time",
    lw         = 2,
    xticks     = (1:2:T),
    xlim       = (0,T+1),
    fontfamily = "serif",
    legend     = :topright,
    legendfont = font(12),
    tickfont   = font(12),
    size       = (900,550),
    framestyle = :box)
```

### Other Discount Factors: $\beta\in\{0,1\}$
Get the IRFs:

```julia
T = 25; #length of IRFs
irfs_lβ = dripirfs(sol_lβ,T);
irfs_hβ = dripirfs(sol_hβ,T);
```

Plot IRFs:

```julia
p1 = plot(1:T, [irfs_bp.x[1,1,:],irfs_hβ.a[1,1,:], irfs_lβ.a[1,1,:]],
    title             = L"IRFs to Slow-Moving Shock ($\rho = 0.95$)",
    label             = ["Shock" L"Price ($\beta=1$)" L"Price ($\beta=0$)"],
    color             = [:darkgray :black :gray50],
    marker            = [:circle :square :utriangle],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p2 = plot(1:T, [irfs_bp.x[2,2,:],irfs_hβ.a[1,2,:], irfs_lβ.a[1,2,:]],
    title             = L"IRFs to Fast-Moving Shock ($\rho = 0.4$)",
    label             = ["Shock" L"Priceblack ($\beta=1$)" L"Price ($\beta=0$)"],
    color             = [:darkgray :black :gray50],
    marker            = [:circle :square :utriangle],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p = plot(p1,p2,
    layout     = (2,1),
    xlabel     = "Time",
    lw         = 2,
    xticks     = (1:2:T),
    xlim       = (0,T+1),
    fontfamily = "serif",
    legend     = :topright,
    legendfont = font(12),
    tickfont   = font(12),
    size       = (900,550),
    framestyle = :box)
```

## Extensions

### Transition Dynamics of Attention

In this section, we solve for the transition dynamics of the optimal posterior covariance matrix starting from an initial prior that is different from the steady state prior.

For instance let us consider a case where the firm is at the steady state of the rational inattention problem at time 0, with prior covariance matrix $\bar{\Sigma}_{-1}$, and it receives a one time treatment with a perfectly informative signal about its optimal price:

$$s_0 = \mathbf{H}'\vec{x}_0$$

Then, the new prior covariance matrix is

$$ \Sigma_0 = \bar{\Sigma}_{-1}-\frac{\bar{\Sigma}_{-1}\mathbf{H}\mathbf{H}'\bar{\Sigma}_{-1}}{\mathbf{H}'\bar{\Sigma}_{-1}\mathbf{H}} $$
Let's see how the initial prior is different from the steady state prior:

```julia
Σ_1     = sol_bp.Σ_1;                      # steady state prior
Σ0      = Σ_1 - (Σ_1*H*H'*Σ_1)/(H'*Σ_1*H); # prior after treatment
display(Σ_1); display(Σ0);                 # display prior before and after treatment
```

    2×2 Array{Float64,2}:
      0.422  -0.067
     -0.067   0.987

    2×2 Array{Float64,2}:
      0.323  -0.323
     -0.323   0.323

#### Solve for the transition dynamics
The function `solve_trip` solves for the transition dynamics automatically given the initial signal. Start by initializing the initial signal:

```julia
s0 = Signal(H,0.0);
```

Solve for the transition dynamics given $s_0$:

```julia
Tss     = 15; # guess for time until convergence
bp_trip = solve_trip(sol_bp, s0; T = Tss);
```

Performance for solving the transition dynamics for a random signal:

```julia
@benchmark solve_trip(sol_bp, S; T = 30) setup = (S = Signal(rand(2),0.0))
```

    BenchmarkTools.Trial:
      memory estimate:  837.84 KiB
      allocs estimate:  6256
      --------------
      minimum time:     951.164 μs (0.00% GC)
      median time:      1.232 ms (0.00% GC)
      mean time:        1.401 ms (9.80% GC)
      maximum time:     9.963 ms (85.15% GC)
      --------------
      samples:          3563
      evals/sample:     1

#### Plot Transition Path of Eigenvalues

Plot the marginal values of information. In this problem the state is two dimensional. At any time, for every orthogonalized dimension, the agent weighs the **marginal value** of acquiring information in that dimension against the **marginal cost** of attention which is the parameter $\omega$.**The number of signals that the agent acquires at any time is the number of marginal values that are larger than $\omega$.**

```julia
p = plot(0:Tss-1,[bp_trip.Ds[1,1:Tss],bp_trip.Ds[2,1:Tss],bp_trip.P.ω*ones(Tss,1)],
    label             = ["Low marginal value dim." "High marginal value dim." "Marginal cost of attention"],
    size              = (900,275),
    title             = "Marginal Value of Information",
    xlabel            = "Time",
    color             = [:darkgray :black :black],
    line              = [:solid :solid :dash],
    marker            = [:circle :square :none],
    markercolor       = false,
    markerstrokecolor = :match,
    markersize        = 6,
    xlim              = (-1,Tss),
    xticks            = 0:2:Tss-1,
    legend            = :outertopright,
    fontfamily        = "serif",
    framestyle        = :box)
```

### Impulse Response Functions with Information Treatment
Get the IRFs in the transition path after treatment:

```julia
T = 30; #length of IRFs

tirfs_bp = dripirfs(sol_bp,T,s0); # irfs with treatment
irfs_bp  = dripirfs(sol_bp,T);    # irfs in the Ss (without treatment)
```

Plot IRFs:

```julia
p1 = plot(1:T, [irfs_bp.x[1,1,:], tirfs_bp.a[1,1,:], irfs_bp.a[1,1,:]],
    title             = L"IRFs to Slow-Moving Shock ($\rho = 0.95$)",
    label             = ["Shock" "Price (w/ treatment)" "Price (w/o treatment)"],
    color             = [:darkgray :black :gray80],
    marker            = [:circle :square :utriangle],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p2 = plot(1:T, [tirfs_bp.x[2,2,:], tirfs_bp.a[1,2,:], irfs_bp.a[1,2,:]],
    title             = L"IRFs to Fast-Moving Shock ($\rho = 0.4$)",
    label             = ["Shock" "Price (w/ treatment)" "Price (w/o treatment)"],
    color             = [:darkgray :black :gray80],
    marker            = [:circle :square :utriangle],
    markerstrokecolor = :match,
    markercolor       = false,
    markersize        = 6)
p = plot(p1,p2,
    layout     = (2,1),
    xlabel     = "Time",
    lw         = 2,
    xticks     = (1:2:T),
    xlim       = (0,T+1),
    fontfamily = "serif",
    legend     = :topright,
    legendfont = font(12),
    tickfont   = font(12),
    size       = (900,550),
    framestyle = :box)
```


# The Cournot model

> **Note the following:**
> 1. This is *not* meant to be an example of an actual **model analysis project**, just an example of how to structure such a project.
> 1. Remember the general advice on structuring and commenting your code from [lecture 5](https://numeconcopenhagen.netlify.com/lectures/Workflow_and_debugging).
> 1. Remember this [guide](https://www.markdownguide.org/basic-syntax/) on markdown and (a bit of) latex.
> 1. Turn on automatic numbering by clicking on the small icon on top of the table of contents in the left sidebar.
> 1. The `modelproject.py` file includes a function which could be used multiple times in this notebook.

In this model project, we consider the Cournot model, known from microeconomics. It is a model of industrial organization. In the model are two frims that compete on quantities. We start off by considering a symmetric game, where the firms have identical cost functions. The game is solved using backwards induction, which means that we start of by maximizing the profit of the market follower, defined as firm 2, whereafter we maximize the profit of firm 1.

The project first solve the model analytically, whereafter the model is solved using a numerical model

Imports and set magics:

```python
import numpy as np
from scipy import optimize
import sympy as sm
import pandas as pd
import matplotlib.pyplot as plt

# autoreload modules when code is run
%load_ext autoreload
%autoreload 2

# local modules
import modelproject
```

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

Defining symbols that are used during the project.

```python
q1 = sm.symbols('q_1') # The quantities of firm 1
q2 = sm.symbols('q_2') # The quantities of firm 2
P = sm.symbols('P') # The price that the firms charge for their products
a = sm.symbols('a') # A part of the inverse demand function
c1 = sm.symbols('c_1') # The production cost of firm 1
c2 = sm.symbols('c_2') # The production cost of firm 2
optq2 = sm.symbols('q_2^*') # The optimal quantity of firm 1
optq1 = sm.symbols('q_1^*') # The optimal quantity of firm 2
```

For later use, we start of by defining the price function, also known as the inverse demand function.

```python
def price(a, q1,q2):
    return a-q1 -q2
```

```python
price(a, q1,q2)
```

$\displaystyle a - q_{1} - q_{2}$

Now we define the two profit functions, notice that in the profits depend on the amount the other firm produces, further it can be seen that the profit functions are identical.

```python
def profit1(q1):
    return (a-q1-q2) * q1 - c1 * q1

def profit2(q2):
    return (a-q1-q2) * q2 - c2 * q2
```

```python
profit2(q2)
```

$\displaystyle - c_{2} q_{2} + q_{2} \left(a - q_{1} - q_{2}\right)$

```python
profit1(q1)
```

$\displaystyle - c_{1} q_{1} + q_{1} \left(a - q_{1} - q_{2}\right)$

We start off considering the second firm , and maximize the profit of firm 2 with respect to the quantity of firm 2.

```python
maxprofit2 = sm.diff(profit2(q2), q2)
maxprofit2
```

$\displaystyle a - c_{2} - q_{1} - 2 q_{2}$

And then we solve for q2, as we then have the best response for firm 2 to the quantity that firm 1 produces.

```python
optq2 = sm.solve(maxprofit2,q2)
optq2[0]
```

$\displaystyle \frac{a}{2} - \frac{c_{2}}{2} - \frac{q_{1}}{2}$

```python
brq2 = a/2 -c2/2 + q1/2
```

We will now do the same for the for firm 1.

```python
sm.diff( q1*(a-(q2)-q1)-c1*q1 ,q1)
```

$\displaystyle a - c_{1} - 2 q_{1} - q_{2}$

We now solve for the quantity of firm 1, where it can be seen that there are symmetry between the two firms.

```python
solve = sm.solve(a-c1-2*q1-q2,q1)
solve[0]
```

$\displaystyle \frac{a}{2} - \frac{c_{1}}{2} - \frac{q_{2}}{2}$

We now insert the quantity of firm 1 into the best response of firm 2, and solve for q2, which gives the optimal amount of q2.

```python
solve2 = sm.solve(a/2 - c2/2 - (a/2 - c1/2 - q2/2)/2-q2,q2)
solve2[0]
```

$\displaystyle \frac{a}{3} + \frac{c_{1}}{3} - \frac{2 c_{2}}{3}$

As we know that the two firms are symmetric, we now know the optimal quantity of firm 1 is similar to the optimal quantity of firm 2.

```python
q2 = a/3 - 2*c1/3 +c2/3
```

```python
q1 = a/3 - 2*c2/3 +c1/3
```

Thus we have now fonund the optimal quantities of each firm.

We can now compare the profits of the two firms. In order to do this, we define some values for a and the marginal cost of each firm.

```python
a=100
c1=10
c2=10
```

We now define the quantities that each firm produce, given the defined values above.

```python
q1 = 100/3 +10/3 - (2*10)/3
```

```python
q2 = 100/3 +10/3 - (2*10)/3
```

Given the quantities, we can now see what the price is, how much each firm will produce and the profit of each firm.

```python
print(f'The market price is {round(price(a,q1,q2),1)}')
print(f'The profit of firm 1 is {round(profit1(q1),1)}')
print(f'The quantitiy of firm 1 is {round(q1,1)}')
print(f'The profit of firm 2 is {round(profit2(q2),1)}')
print(f'The quantitiy of firm 2 is {round(q2,1)}')
```

    The market price is 40.0
    The profit of firm 1 is 900.0
    The quantitiy of firm 1 is 30.0
    The profit of firm 2 is 900.0
    The quantitiy of firm 2 is 30.0

## Numerical solution

In this section we solve the cournot model by using a numerical model.

```python
N = 999 # Number of exogenous drawn quantities for firm 2
q2_vector = np.linspace(0,90,N) # Quantities actually getting drawn, thus we draw between quantities for q2 between 0 and 90, and draw 999 values between 0 and 90.
```

```python
# defining empty vectors
q2_try= []
q1_BR = []
pi1_BR = []

for i in q2_vector:
    q2 = i # sets the value of q2 equal to i
    q1_guess = 0 # Initial guess for q1
    objective_function = lambda q1: -profit1(q1) # We want to maximize the profit of firm 1, and have "-", as the function minimizes, we find the profits of firm 1 given the values of firm 2, and find the optimal quantity of q1
    res = optimize.minimize(objective_function, q1_guess, method='BFGS')
    q2_try.append(i) # Append the value of q2
    q1_BR.append(res.x[0]) # Append the best respone quantity
```

```python
df=pd.DataFrame(q2_try)
df.columns=['q2']
df['q1'] = q1_BR
df['Price'] = a - df['q1'] - df['q2'] # Calculate the price given the quantities of firm 1 and firm 2
df['Profit_1'] = np.transpose(df['Price'] * df['q1']) - c2 * df['q1'] # Calculate the profit of firm 1
df['Profit_2'] = np.transpose(df['Price'] * df['q2']) - c2 * df['q2'] # Calculate the profit of firm 2

df = df.round(1)
df.head(1000)
```

      q2
      q1
      Price
      Profit_1
      Profit_2

      0
      0.0
      45.0
      55.0
      2025.0
      0.0

      1
      0.1
      45.0
      55.0
      2020.9
      4.1

      2
      0.2
      44.9
      54.9
      2016.9
      8.1

      3
      0.3
      44.9
      54.9
      2012.8
      12.1

      4
      0.4
      44.8
      54.8
      2008.8
      16.2

      ...
      ...
      ...
      ...
      ...
      ...

      994
      89.6
      0.2
      10.2
      0.0
      16.2

      995
      89.7
      0.1
      10.1
      0.0
      12.1

      996
      89.8
      0.1
      10.1
      0.0
      8.1

      997
      89.9
      0.0
      10.0
      0.0
      4.1

      998
      90.0
      0.0
      10.0
      0.0
      0.0

999 rows × 5 columns

```python
q1_exo=df['q2'].copy() # Copying exogenous drawn quantities from firm 1 to firm 2
q2_BR=df['q1'].copy() # Copying best response from firm 2 to firm 1
```

```python
# Plots the responses of each firm, for different values of the other quantity.

plt.plot(q1_BR[0:999], q1_exo[0:999], label='R1(q2)') # Blue line
plt.plot(df['q2'][0:999],df['q1'][0:999], label='R2(q1)') # Orange line
plt.plot([30],[30],'ro') # Red dot
plt.xlabel('q1')
plt.ylabel('q2')
plt.title('Response functions')
plt.legend()
plt.show()
```

Where it can be seen that the larger the quantity is for firm 1, the lower will the quantity of firm 2 will be. The red dot represents the optimal quantity found in the analytical solution. ,

```python
for i in range(N):
    if df['q2'][i] == df['q1'][i]: # Finds the value where the quantity of firm 1 and firm 2 intersect
        profits1 = df['Profit_1'][i] # Define the profits of firm 1, given the intersect
        profits2 = df['Profit_2'][i]
        price = df['Price'][i] # Defines the price given the quantities in the intersect
        optimal_quantity1 = df['q1'][i] # Defining the quantities given the intersect
        optimal_quantity2 = df['q2'][i]
```

We can now check that the quantity and profits in the Numerical solution compared to the analytical solution.

```python
print(f'The market price is {round(price,1)}')
print(f'The profit of firm 1 is {round(profits1,1)}')
print(f'The quantitiy of firm 1 is {round(optimal_quantity1,1)}')
print(f'The profit of firm 2 is {round(profits2,1)}')
print(f'The quantitiy of firm 2 is {round(optimal_quantity2,1)}')
```

    The market price is 40.0
    The profit of firm 1 is 899.1
    The quantitiy of firm 1 is 30.0
    The profit of firm 2 is 900.5
    The quantitiy of firm 2 is 30.0

Thus it can be seen that the optimal quantity is the same as in the analytical, the price is also the same, but the profits have a few rounding errors, which causes the profit of firm 1 to be lower than the profit of firm 2.

# Further analysis

We now introduce a third firm into the Cournot model, and solve this by using the same approach as in the numerical solution.

```python
q3 = sm.symbols('q_3') # Define the quantity of firm 3
c3 = sm.symbols('c_3') # Define the cost of firm 3
```

```python
# Redefining profits, as the profits for firm 1 and 2 also depends on the quantity of firm 3.
def profit1(q1):
    return (a-(q1+q2+q3)) * q1 - c1 * q1

def profit2(q2):
    return (a-(q1+q2+q3)) * q2 - c2 * q2

def profit3(q1,q2,q3):
    return (a-(q1+q2+q3)) * q3 - c3 * q3
```

```python
profit3(q1,q2,q3)
```

$\displaystyle - c_{3} q_{3} + q_{3} \left(- q_{1} - q_{3} + 10.0\right)$

```python
N = 999 # Number of exogenous drawn quantities
q2_vector = np.linspace(0,90,N) # Quantities actually getting drawn, thus we draw between quantities for q2 between 0 and 90, and draw 999 values between these two different values.
```

```python
c3 = 10 # We define the cost for firm 3 to be similar to the costs of firm 1 and firm 2.
```

```python
# Defining empty vectors
q2_try= []
q3_try= []
q1_BR = []
pi1_BR = []

for i in q2_vector:
    q2 = i # Both q2 and q3 are set to be equal to i.
    q3 = i
    q1_guess = 0
    objective_function = lambda q1: -profit1(q1)
    res = optimize.minimize(objective_function, q1_guess, method='BFGS')
    q2_try.append(i) # Append q2
    q3_try.append(i) # Append q3
    q1_BR.append(res.x[0]) # Append the best response q1
```

```python
df=pd.DataFrame(q2_try)
df.columns=['q2']
df['q1'] = q1_BR
df['q3'] = q3_try
df['Price'] = a - df['q1'] - df['q2'] - df['q3']
df['Profit_1'] = np.transpose(df['Price'] * df['q1']) - c1 * df['q1'] # Calculate profits of firm 1
df['Profit_2'] = np.transpose(df['Price'] * df['q2']) - c2 * df['q2'] # Calculate profits of firm 2
df['Profit_3'] = np.transpose(df['Price'] * df['q3']) - c3 * df['q3'] # Calculate profits of firm 3
df = df.round(1)
df.head(10)
```

      q2
      q1
      q3
      Price
      Profit_1
      Profit_2
      Profit_3

      0
      0.0
      45.0
      0.0
      55.0
      2025.0
      0.0
      0.0

      1
      0.1
      45.0
      0.1
      54.9
      2016.9
      4.0
      4.0

      2
      0.2
      44.9
      0.2
      54.7
      2008.8
      8.1
      8.1

      3
      0.3
      44.9
      0.3
      54.6
      2000.7
      12.1
      12.1

      4
      0.4
      44.8
      0.4
      54.5
      1992.6
      16.0
      16.0

      5
      0.5
      44.8
      0.5
      54.3
      1984.6
      20.0
      20.0

      6
      0.5
      44.7
      0.5
      54.2
      1976.5
      23.9
      23.9

      7
      0.6
      44.7
      0.6
      54.1
      1968.5
      27.8
      27.8

      8
      0.7
      44.6
      0.7
      53.9
      1960.5
      31.7
      31.7

      9
      0.8
      44.6
      0.8
      53.8
      1952.4
      35.5
      35.5

```python
for i in range(N): # Finding equilibria quantities, profits and price
    if df['q2'][i] == df['q1'][i] == df['q3'][i]:
        profits1 = df['Profit_1'][i]
        profits2 = df['Profit_2'][i]
        profits3 = df['Profit_3'][i]
        price1 = df['Price'][i]
        optimal_quantity1 = df['q1'][i]
        optimal_quantity2 = df['q2'][i]
        optimal_quantity3 = df['q3'][i]
```

We can now check the profits of the three firms, where it can be seen that there is a minor error in the model, showing that the profits of the firms are not perfect equal, we can check the profits of firm 3, by using the defined function.

```python
print(f'The market price is {round(price1,1)}')
print(f'The profit of firm 1 is {round(profits1,1)}')
print(f'The quantitiy of firm 1 is {round(optimal_quantity1,1)}')
print(f'The profit of firm 2 is {round(profits2,1)}')
print(f'The quantitiy of firm 2 is {round(optimal_quantity2,1)}')
print(f'The profit of firm 3 is {round(profits3,1)}')
print(f'The quantitiy of firm 3 is {round(optimal_quantity3,1)}')
```

    The market price is 32.5
    The profit of firm 1 is 504.2
    The quantitiy of firm 1 is 22.5
    The profit of firm 2 is 506.2
    The quantitiy of firm 2 is 22.5
    The profit of firm 3 is 506.2
    The quantitiy of firm 3 is 22.5

Where it can be seen that the model shows the same profits for firm 2 and firm 3, while the profit for firm 1 is just a bit lower. We can check the profit of firms, using the profit of firm 3, and insereting that each firm will produce the quantity of 22.5

```python
profit3(22.5,22.5,22.5)
```

    506.25

Thus it can be concluded that the profit of each firm should be 506.25 given that each firm produce a quantity of 22.5

# Conclusion

From the models it can be seen that by introducing a thrid firm into the Cournot model, the combined quantity increase and the price decreases. A third firm will decrease the profit of each firm from 900 to 506.25.


```python
import numpy
from msppy.msp import MSLP
from msppy.solver import SDDP, Extensive
from msppy.evaluation import Evaluation
import matplotlib.pyplot as plt
import seaborn
seaborn.set_style('darkgrid')
```

American put option pricing
=====================
This tutorial deals with the American put option pricing.

Introduction
----------------
In American option, early exercise is preferable when the continuation (time) value shrinks below intrinsic value. The value of the option at each time step is thus given by the maximum of the intrinsic value and continuation value. By virtue of dynamic equation, the value at each time $i$ given by,
\begin{equation}
    V_i(S_i) = \max\big\{(K-S_i)_+,\mathbb{E}_i^Q[\exp(-r\Delta t) V_{i+1}(S_{i+1})|S_i]\big\}
\end{equation}
Where $K$ is the strike price, $\Delta t$ is the time step, $\mathbb{E}_i^Q$ is the risk neutral measure and $r$ is the interest rate

Binomial tree model
---------------------------
Suppose spot price is 36, strike price is 40, volatility is 0.2, interest rate is 6%, expiration is 1 year, time step is 0.02.

```python
S = 36
K = 40
sigma = 0.2
r = 0.06
T = 50
step = 0.02
u = numpy.exp(sigma*numpy.sqrt(step))
d = 1/u
p = (numpy.exp(r*step)-d)/(u-d)
```

Solution
----------

```python
put = MSLP(T=T+1, discount=numpy.exp(-r*step), ctg=1)
coef = [-u,-d]
for t in range(T+1):
    m = put[t]
    s_now, s_past = m.addStateVar()
    y = m.addVar(obj=1,name="y")
    if t > 0:
        m.addConstr(s_now + s_past == 0, uncertainty={s_past: coef})
        m.set_probability([p,1-p])
    else:
        m.addConstr(s_now == S)
    if t < T:
        m.addConstr(m.getVarByName("y") + m.alpha*numpy.exp(-r*step) >= K-m.states[0])
    else:
        m.addConstr(m.getVarByName("y") >= K-m.states[0])
sddp = SDDP(put)
sddp.solve(max_iterations=200, n_processes=3, n_steps=3, logToConsole=0)
```

    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only
    Academic license - for non-commercial use only

Evolution of bound.

```python
plt.plot(sddp.db)
```

More sophicated model can be considered.

```python

```


# IHA1 - Assignment

Welcome to the first individual home assignment!

This assignment consists of two parts:
 * Python and NumPy exercises
 * Build a deep neural network for forward propagation

The focus of this assignment is for you to gain practical knowledge with implementing forward propagation of deep neural networks without using any deep learning framework. You will also gain practical knowledge in two of Python's scientific libraries [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) and [Matplotlib](https://matplotlib.org/devdocs/index.html).

Skeleton code is provided for most tasks and every part you are expected to implement is marked with **TODO**

We expect you to search and learn by yourself any commands you think are useful for these tasks. Don't limit yourself to only what was taught in CL1. Use the help function, [stackoverflow](https://stackoverflow.com/), google, the [python documentation](https://docs.python.org/3.5/library/index.html) and the [NumPy](https://docs.scipy.org/doc/numpy-1.13.0/index.html) documentation to your advantage.

**IMPORTANT NOTE**: The tests available are not exhaustive, meaning that if you pass a test you have avoided the most common mistakes, but it is still not guaranteed that you solution is 100% correct.

Lets start by importing the necessary libraries below

```python
import numpy as np
import matplotlib.pyplot as plt
from utils.tests.iha1Tests import *
```

## 1. Lists and arrays introduction
First, we will warm up with a Python exercise and few NumPy exercises

### 1.1 List comprehensions
Examine the code snippet provided below

```python
myList = []
for i in range(25):
    if i % 2 == 0:
        myList.append(i**2)

print(myList)
```

    [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]

This is not a very "[pythonic](http://docs.python-guide.org/en/latest/writing/style/)" way of writing. Lets re-write the code above using a [list comprehension](https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions). The result will be less code, more readable and elegant. Your solution should be able to fit into one line of code.

```python
myList = None # TODO
print(myList)
```

    None

```python
# sample output from cell above for reference
```

    [0, 4, 16, 36, 64, 100, 144, 196, 256, 324, 400, 484, 576]

### 1.2 Numpy array vs numpy vectors
Run the cell below to create a numpy array.

```python
myArr = np.array([1, 9, 25, 49, 81, 121, 169, 225, 289, 361, 441, 529])
print(myArr)
print(myArr.shape)
```

One of the core features of numpy is to efficiently perform linear algebra operations.
There are two types of one-dimensional representations in numpy: arrays of shape (x,) and vectors of shape (x,1)

The the above result indicates that **myArr** is an array of 12 elements with shape (12,).

Numpy's arrays and vectors both have the type of `numpy.ndarray` but have in some cases different characteristics and it is important to separate the two types because it will save a lot of debugging time later on. Read more about numpy shapes [here](https://stackoverflow.com/a/22074424)

Run the code below to see how the transpose operation behaves differently between an array and vector

```python
# print the shape of an array and the shape of a transposed array
print('myArr is an array of shape:')
print(myArr.shape)
print('The transpose of myArr has the shape:')
print(myArr.T.shape)

# print the shape of a vector and the transpose of a vector
myVec = myArr.reshape(12,1)
print('myVec is a vector of shape:')
print(myVec.shape)
print('The transpose of myVec has the shape:')
print(myVec.T.shape)
```

### 1.3 Numpy exercises
Now run the cell below to create the numpy array `numbers` and then complete the exercises sequentially

```python
numbers = np.arange(24)
print(numbers)
```

```python
# TODO: reshape numbers into a 6x4 matrix
print(numbers)
```

```python
# sample output from cell above for reference
```

    [[ 0  1  2  3]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22 23]]

```python
# test case
test_numpy_reshape(numbers)
```

```python
# TODO: set the element of the last row of the last column to zero
# Hint: Try what happends when indices are negative
print(numbers)
```

```python
# sample output from cell above for reference
```

    [[ 0  1  2  3]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]]

```python
# test case
test_numpy_neg_ix(numbers)
```

```python
# TODO: set every element of the 0th row to 0
print(numbers)
```

```python
# sample output from cell above for reference
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]]

```python
# test case
test_numpy_row_ix(numbers)
```

```python
# TODO: append a 1x4 row vector of zeros to `numbers`,
# resulting in a 7x4 matrix where the new row of zeros is the last row
# Hint: A new matrix must be created in the procedure. Numpy arrays are not dynamic.
print(numbers)
print(numbers.shape)
```

```python
# sample output from cell above for reference
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22  0]
     [ 0  0  0  0]]
    (7, 4)

```python
# test case
test_numpy_append_row(numbers)
```

```python
# TODO: set all elements above 10 to the value 1
print(numbers)
```

```python
# sample output from cell above for reference
```

    [[ 0  0  0  0]
     [ 4  5  6  7]
     [ 8  9 10  1]
     [ 1  1  1  1]
     [ 1  1  1  1]
     [ 1  1  1  0]
     [ 0  0  0  0]]

```python
# test case
test_numpy_bool_matrix(numbers)
```

```python
# TODO: compute the sum of every row and replace `numbers` with the answer
# `numbers` will be a (7,) array as a result
print(numbers.shape)
print(numbers)
```

```python
# sample output from cell above for reference
```

    (7,)
    [ 0 22 28  4  4  3  0]

```python
# test case
test_numpy_sum(numbers)
```

## 2 Building your deep neural network
It is time to start implementing your first feed-forward neural network. In this lab you will only focus on implementing the forward propagation procedure.

When using a neural network, you can not forward propagate the entire dataset at once. Therefore, you divide the dataset into a number of sets/parts called batches. A batch will make up for the first dimension of every input to a layer and the notation `(BATCH_SIZE, NUM_FEATURES)` simply means the dimension of a batch of samples where every sample has `NUM_FEATURES` features

### 2.1 activation functions
You will start by defining a few activation functions that are later needed by the neural network.

#### 2.1.1 ReLU
The neural network will use the ReLU activation function in every layer except for the last. ReLU does element-wise comparison of the input matrix. For example, if the input is `X`, and `X[i,j] == 2` and `X[k,l] == -1`, then after applying ReLU, `X[i,j] == 2` and `X[k,l] == 0` should be true.

The formula for implementing ReLU for a single neuron $i$ is:
\begin{equation}
relu(z_i) =
    \begin{cases}
      0, & \text{if}\ z_i \leq 0 \\
      z_i, & \text{otherwise}
    \end{cases}
\end{equation}

Now implement `relu` in vectorized form

```python
def relu(z):
    """ Implement the ReLU activation function

    Arguments:
    z - the input of the activation function. Has a type of `numpy.ndarray`

    Returns:
    a - the output of the activation function. Has a type of numpy.ndarray and the same shape as `z`
    """

    a = None # TODO

    return a
```

```python
# test case
test_relu(relu)
```

#### 2.1.2 Sigmoid
The sigmoid activation function is common for binary classification. This is because it squashes its input to the range [0,1].
Implement the activation function `sigmoid` using the formula:
\begin{equation}
    \sigma(z) = \frac{1}{1 + e^{-z}}
\end{equation}

```python
def sigmoid(z):
    """ Implement the sigmoid activation function

    Arguments:
    z - the input of the activation function. Has a type of `numpy.ndarray`

    Returns:
    a - the output of the activation function. Has a type of `numpy.ndarray` and the same shape as `z`
    """

    a = None # TODO

    return a
```

```python
# test case
test_sigmoid(sigmoid)
```

#### 2.1.3 Visualization
Make a plot using matplotlib to visualize the activation functions between the input interval [-3,3]. The plot should have the following properties
 * one plot should contain a visualization of both `ReLU` and `sigmoid`
 * x-axis: range of values between [-3,3], **hint**: np.linspace
 * y-axis: the value of the activation functions at a given input `x`
 * a legend explaining which line represents which activation function

```python
# TODO: make a plot of ReLU and sigmoid values in the interval [-3,3]
```

```python
# sample output from cell above for reference
```

#### 2.1.4 Softmax
You will use the softmax activation function / classifier as the final layer of your neural network later in the assignment. Implement `softmax` according the the formula below. The subtraction of the maximum value is there solely to avoid overflows in a practical implementation.
\begin{equation}
softmax(z_i) = \frac{e^{z_i - max(\mathbf{z})}}{ \sum^j e^{z_j - max(\mathbf{z})}}
\end{equation}

```python
def softmax(z):
    """ Implement the softmax activation function

    Arguments:
    z - the input of the activation function, shape (BATCH_SIZE, FEATURES) and type `numpy.ndarray`

    Returns:
    a - the output of the activation function, shape (BATCH_SIZE, FEATURES) and type umpy.ndarray
    """

    a = None # TODO

    return a
```

```python
# test case
test_softmax(softmax)
```

### 2.2 Initialize weights
You will implement a helper function that takes the shape of a layer as input, and returns an initialized weight matrix $\mathbf{W}$ and bias vector $\mathbf{b}$ as output. $\mathbf{W}$ should be sampled from a normal distribution of mean 0 and standard deviation 2, and $\mathbf{b}$ should be initialized to all zeros.

```python
def initialize_weights(layer_shape):
    """ Implement initialization of the weight matrix and biases

    Arguments:
    layer_shape - a tuple of length 2, type (int, int), that determines the dimensions of the weight matrix: (input_dim, output_dim)

    Returns:
    w - a weight matrix with dimensions of `layer_shape`, (input_dim, output_dim), that is normally distributed with
        properties mu = 0, stddev = 2. Has a type of `numpy.ndarray`
    b - a vector of initialized biases with shape (1,output_dim), all of value zero. Has a type of `numpy.ndarray`
    """
    w = None # TODO
    b = None # TODO

    return w, b
```

```python
# test case
test_initialize_weights(initialize_weights)
```

### 2.3 Feed-forward neural network layer module
To build a feed-forward neural network of arbitrary depth you are going to define a neural network layer as a module that can be used to stack upon eachother.

Your task is to complete the `Layer` class by following the descriptions in the comments.

Recall the formula for forward propagation of an arbitrary layer $l$:

\begin{equation}
\mathbf{a}^{[l]} = g(\mathbf{z}^{[l]}) = g(\mathbf{a}^{[l-1]}\mathbf{w}^{[l]} +\mathbf{b}^{[l]})
\end{equation}

$g$ is the activation function given by `activation_fn`, which can be relu, sigmoid or softmax.

```python
class Layer:
    """
    TODO: Build a class called Layer that satisfies the descriptions of the methods
    Make sure to utilize the helper functions you implemented before
    """

    def __init__(self, input_dim, output_dim, activation_fn=relu):
        """
        Arguments:
        input_dim - the number of inputs of the layer. type int
        output_dim - the number of outputs of the layer. type int
        activation_fn - a reference to the activation function to use. Should be `relu` as a default
                        possible values are the `relu`, `sigmoid` and `softmax` functions you implemented earlier.
                        Has the type `function`

        Attributes:
        w - the weight matrix of the layer, should be initialized with `initialize_weights`
            and has the shape (INPUT_FEATURES, OUTPUT_FEATURES) and type `numpy.ndarray`
        b - the bias vector of the layer, should be initialized with `initialize_weights`
            and has the shape (1, OUTPUT_FEATURES) and type `numpy.ndarray`
        activation_fn - a reference to the activation function to use.
                        Has the type `function`
        """
        self.w, self.b = None, None # TODO

        self.activation_fn = None # TODO

    def forward_prop(self, a_prev):
        """ Implement the forward propagation module of the neural network layer
        Should use whatever activation function that `activation_fn` references to

        Arguments:
        a_prev - the input to the layer, which may be the data `X`, or the output from the previous layer.
            a_prev has the shape of (BATCH_SIZE, INPUT_FEATURES) and the type `numpy.ndarray`

        Returns:
        a - the output of the layer when performing forward propagation. Has the type `numpy.ndarray`
        """

        a = None # TODO

        return a
```

```python
# test case, be sure that you pass the previous activation function tests before running this test
test_layer(Layer, relu, sigmoid, softmax)
```

### 2.4 Logistic regression
Binary logistic regression is a classifier where classification is performed by applying the sigmoid activation function to a linear combination of input values. You will now try out your neural network layer by utilizing it as a linear combination of input values and apply the sigmoid activation function to classify a simple problem.

The cell below defines a dataset of 5 points of either class `0` or class `1`. Your assignment is to:
1. Create an instance of a `Layer` with sigmoid activation function
2. Manually tune the weights `w` and `b` of your layer

You can use `test_logistic` to visually inspect how your classifier is performing.

```python
# Run this cell to create the dataset
X_s = np.array([[1, 2],
               [5, 3],
               [8, 8],
               [7, 5],
               [3, 6]])
Y_s = np.array([0,0,1,0,1])

test_logistic(X_s, Y_s)
```

```python
# create an instance of layer
l = Layer(2,1,sigmoid)

# TODO: manually tune weights
l.w = None
l.b = None

# testing your choice of weights with this function
test_logistic(X_s,Y_s,l,sigmoid)
```

```python
# sample output from cell above for reference
```

### 2.5 Feed-forward neural network
Now define the actual neural network class. It is an L-layer neural network, meaning that the number of layers and neurons in each layer is specified as input by the user. Once again, you will only focus on implementing the forward propagation part.

Read the descriptions in the comments and complete the todos

```python
class NeuralNetwork:
    """
    TODO: Implement an L-layer neural network class by utilizing the Layer module defined above
    Each layer should use `relu` activation function, except for the output layer, which should use `softmax`
    """

    def __init__(self, input_n, layer_dims):
        """
        Arguments:
        input_n    - the number of inputs to the network. Should be the same as the length of a data sample
                     Has type int
        layer_dims - a python list or tuple of the number of neurons in each layer. Layer `l` should have a weight matrix
                     with the shape (`layer_dims[l-1]`, `layer_dims[l]`).
                     `layer_dims[-1]` is the dimension of the output layer.
                     Layer 1 should have the dimensions (`input_n`, `layer_dims[0]`).
                     len(layer_dims) is the depth of the neural network
        Attributes:
        input_n - the number of inputs to the network. Has type int
        layers  - a python list of each layer in the network. Each layer should use the `relu` activation function,
                  except for the last layer, which should use `softmax`.
                  Has type `list` containing layers of type `Layer`
        """

        self.input_n = None # TODO
        self.layers = None # TODO

    def forward_prop(self, x):
        """
        Implement the forward propagation procedure through the entire network, from input to output.
        You will now connect each layer's forward propagation function into a chain of layer-wise forward propagations.

        Arguments:
        x - the input data, which has the shape (BATCH_SIZE, NUM_FEATURES) and type `numpy.ndarray`

        Returns:
        a - the output of the last layer after forward propagating through the every layer in `layers`.
            Should have the dimension (BATCH_SIZE, layers[-1].w.shape[1]) and type `numpy.ndarray`
        """
        a = None # TODO

        return a
```

```python
# test case
test_neuralnetwork(NeuralNetwork)
```

## 3 Making predictions with a neural network
In practice, its common to load weights to your neural network that has already been trained.
In this section, you will create an instance of your neural network, load trained weights from disk, and perform predictions.

### 3.1 Load weights from disk
Create an instance of `NeuralNetwork` with input size $28 \times 28 = 784$, two hidden layers of size 100 and an output layer of size 10. Thereafter, load the weights contained in `./utils/ann_weights.npz` to your network.

```python
ann = None # TODO: create instance of ann

# load weights
weights = np.load('./utils/ann_weights.npz')
for l in range(len(ann.layers)):
    ann.layers[l].w = weights['w' + str(l)]
    ann.layers[l].b = weights['b' + str(l)]
```

### 3.2 Prediction
Now, implement the function `predict_and_correct` which does the following:
1. Load `./utils/test_data.npz` from disk
2. Extract test data `X` and `Y` from file
2. Perform for every pair of data:
    a. plot the image `x`
    b. make a prediction using your neural network by forward propagating and picking the most probable class
    c. check wether the prediction is correct (compare with the ground truth number `y`)
    d. print the predicted label and wether it was correct or not

```python
def predict_and_correct(ann):
    """ Load test data from file and predict using your neural network.
    Make a prediction for ever data sample and print it along with wether it was a correct prediction or not

    Arguments:
    ann - the neural network to use for prediction. Has type `NeuralNetwork`

    Returns: # for test case purposes
    A `numpy.ndarray` of predicted classes (integers [0-9]) with shape (11,)
    """
    data = np.load('./utils/test_data.npz_old')
    X, cls = data['X'], data['Y']

    cls_preds = None # TODO: make a predicted number for every image in X

    for i in range(len(X)):
        plt.imshow(X[i].reshape(28,28), cmap='gray')
        plt.show()
        correct = cls_preds[i] == cls[i]
        print('The prediction was {0}, it was {1}!'.format(cls_preds[i], 'correct' if correct else 'incorrect'))

    return cls_preds

cls_pred = predict_and_correct(ann)
```

```python
# final test case
test_predict_and_correct_answer(cls_pred)
```

```python
# sample output from cell above for reference
```

## Congratulations!
You have successfully implemented a neural network from scratch using only NumPy!