```python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
import sympy
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets
```

# 3. Exchange economy

Consider an **exchange economy** with

1. 3 goods, $(x_1,x_2,x_3)$
2. $N$ consumers indexed by \\( j \in \{1,2,\dots,N\} \\)
3. Preferences are Cobb-Douglas with log-normally distributed coefficients

    $$ \begin{eqnarray*}
    u^{j}(x_{1},x_{2},x_{3}) &=&
    \left(x_{1}^{\beta_{1}^{j}}x_{2}^{\beta_{2}^{j}}x_{3}^{\beta_{3}^{j}}\right)^{\gamma}\\
     &  & \,\,\,\beta_{i}^{j}=\frac{\alpha_{i}^{j}}{\alpha_{1}^{j}+\alpha_{2}^{j}+\alpha_{3}^{j}} \\
     &  & \,\,\,\boldsymbol{\alpha}^{j}=(\alpha_{1}^{j},\alpha_{2}^{j},\alpha_{3}^{j}) \\
     &  & \,\,\,\log(\boldsymbol{\alpha}^j) \sim \mathcal{N}(\mu,\Sigma) \\
    \end{eqnarray*} $$

4. Endowments are exponentially distributed,

$$
\begin{eqnarray*}
\boldsymbol{e}^{j} &=& (e_{1}^{j},e_{2}^{j},e_{3}^{j}) \\
 &  & e_i^j \sim f, f(z;\zeta) =  1/\zeta \exp(-z/\zeta)
\end{eqnarray*}
$$

Let $p_3 = 1$ be the **numeraire**. The implied **demand functions** are:

$$
\begin{eqnarray*}
x_{i}^{\star j}(p_{1},p_{2},\boldsymbol{e}^{j})&=&\beta^{j}_i\frac{I^j}{p_{i}} \\
\end{eqnarray*}
$$

where consumer $j$'s income is

$$I^j = p_1 e_1^j + p_2 e_2^j +p_3 e_3^j$$

The **parameters** and **random preferences and endowments** are given by:

```python
# a. parameters
N = 50000
mu = np.array([3,2,1])
Sigma = np.array([[0.25, 0, 0], [0, 0.25, 0], [0, 0, 0.25]])
gamma = 0.8
zeta = 1

# b. random draws
seed = 1986
np.random.seed(seed)

# preferences
alphas = np.exp(np.random.multivariate_normal(mu, Sigma, size=N))
betas = alphas/np.reshape(np.sum(alphas,axis=1),(N,1))

# endowments
e1 = np.random.exponential(zeta,size=N)
e2 = np.random.exponential(zeta,size=N)
e3 = np.random.exponential(zeta,size=N)
```

**Question 1:** Plot the histograms of the budget shares for each good across agents.

To plot the budget shares, we define each individual budget share $\beta_i$ from betas, and plot these in a figure with all betas. Afterwards we create an interactive figure to choose each betadistribution individually

```python
#First we define beta one, two and three (Index starts with 0)
beta_1 = betas[:,0]
beta_2 = betas[:,1]
beta_3 = betas[:,2]
```

```python
#Secondly we define a DataFrame with each beta
Budget_share = pd.DataFrame([beta_1,beta_2,beta_3])

#And transpose
Budget_share = pd.DataFrame(Budget_share).transpose()

#Rename the coloumns
Budget_share = pd.DataFrame(Budget_share).rename(columns={0:"beta_1",1:"beta_2",2:"beta_3"})
```

```python
#Plotting the beta's against each other
Budget_share.plot.hist(bins=300)
```

```python
#Define a functions that creates the figure for each beta.
def budget_share(beta_value):
    pd.DataFrame(Budget_share).hist(column=beta_value,bins=100)
    plt.xlabel('Budget share value')
    plt.ylabel('Number of persons')
    plt.title(f'Budget share across agents for {str(beta_value)}')
    plt.grid(True)
    return plt.show()
```

```python
#Create an interactive figure that shows the distribution for each beta (Beware of the changeing axis)
widgets.interact(budget_share,
                 beta_value = widgets.Dropdown(options=["beta_1","beta_2","beta_3"]))
```

    interactive(children=(Dropdown(description='beta_value', options=('beta_1', 'beta_2', 'beta_3'), value='beta_1…

    <function __main__.budget_share(beta_value)>

#### Comments for the histograms:

It appears that the agents have a preference for good one, as beta_1 has the highest expected value, even though it also has a large spread. Good three is generally not preferred with a very low expected value and low spread. Finally good two has a low expected value, but also a relatively large spread.

Consider the **excess demand functions:**

$z_i(p_1,p_2) = \sum_{j=1}^N x_{i}^{\star j}(p_{1},p_{2},\boldsymbol{e}^{j}) - e_i^j$

**Question 2:** Plot the excess demand functions.

To plot the excess demand functions, we first define the demand functions for each good, and thereafter create a function for each individual excess demand. Finally these are plotted. (The DataFrame method was not as suitable for this task, so we use numpy grids)

```python
#First we define the demand functions
def demand_func1(beta_1,p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,0]*I/p1
def demand_func2(beta_2,p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,1]*I/p2
def demand_func3(beta_3, p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,2]*I/1
```

```python
#Secondly, we can define the excess demand functions
def excess_demand1(betas,p1,p2,e1,e2,e3):
    demand1 = np.sum(demand_func1(betas,p1,p2,e1,e2,e3))
    supply1 = np.sum(e1)
    excess_demand = demand1-supply1
    return excess_demand

def excess_demand2(betas,p1,p2,e1,e2,e3):
    demand2 = np.sum(demand_func2(betas,p1,p2,e1,e2,e3))
    supply2 = np.sum(e2)
    excess_demand  = demand2-supply2
    return excess_demand

def excess_demand3(betas,p1,p2,e1,e2,e3):
    demand3 = np.sum(demand_func3(betas,p1,p2,e1,e2,e3))
    supply3 = np.sum(e3)
    excess_demand = demand3-supply3
    return excess_demand
```

```python
#Thirdly, we define the prices. First by defineing the numpy arrays for each price, thereafter create empty grids and insert them by mesh and enumerate.
size = 100
p1_gen = np.linspace(1,10,size)
p2_gen = np.linspace(1,10,size)

p1_empty_grid = np.empty((size,size))
p2_empty_grid = np.empty((size,size))

p1_grid, p2_grid = np.meshgrid(p1_gen,p2_gen)
for i, p1 in enumerate(p1_gen):
    for j, p2 in enumerate(p2_gen):
        p1_empty_grid[i,j] = excess_demand1(betas,p1,p2,e1,e2,e3)
        p2_empty_grid[i,j] = excess_demand2(betas,p1,p2,e1,e2,e3)
```

```python
#Now that we have the data, we just need to plot it
#Plotting for excess good 1
fig = plt.figure(dpi=100, figsize=(12,5))
axis1 = fig.add_subplot(121, projection = '3d')
fig1 = axis1.plot_surface(p1_grid, p2_grid, p1_empty_grid, cmap=plt.cm.inferno)
axis1.set_title('Excess demand for good 1')
axis1.set_xlabel("p1")
axis1.set_ylabel("p2")
axis1.invert_yaxis()

#And the excess for good 2
axis2 = fig.add_subplot(122, projection = '3d')
fig2 = axis2.plot_surface(p1_grid, p2_grid, p2_empty_grid, cmap=plt.cm.inferno)
axis2.set_title('Excess demand for good 2')
axis2.set_xlabel("p1")
axis2.set_ylabel("p2")
axis2.invert_yaxis()
```

#### Conclusions for excess demand plots:
First of all, its important to beware of the change in axis labels on the z axis. Furthermore, the opvious difference between the figures is the direction of the plot. Where the excess demand of good 1 increases with an increase in p1 and a decrease in p2, the excess demand of good 2 increases when p1 decreases and p2 increases. This makes sense, as the excess demand follows the goods own price for both goods.

**Question 3:** Find the Walras-equilibrium prices, $(p_1,p_2)$, where both excess demands are (approximately) zero, e.g. by using the following tâtonnement process:

1. Guess on $p_1 > 0$, $p_2 > 0$ and choose tolerance $\epsilon > 0$ and adjustment aggressivity parameter, $\kappa > 0$.
2. Calculate $z_1(p_1,p_2)$ and $z_2(p_1,p_2)$.
3. If $|z_1| < \epsilon$ and $|z_2| < \epsilon$ then stop.
4. Else set $p_1 = p_1 + \kappa \frac{z_1}{N}$ and $p_2 = p_2 + \kappa \frac{z_2}{N}$ and return to step 2.

The solution for this question, and the following two, relies heavily on the code from problemset 2. We assume the Walras-equilibrium is found where demand for good three is zero.

```python
#As most parameters and the demand and excess demand functions are already set up, we start by defineing the equilibirum function as in problemset 2
def find_equilibrium(alphas,p1,p2,e1,e2,e3,kappa=0.5,eps=1e-8,maxiter=500):

    t = 0
    while True:
        # a. step 1: define excess demand
        Z1 = excess_demand1(alphas,p1,p2,e1,e2,e3)
        Z2 = excess_demand2(alphas,p1,p2,e1,e2,e3)

        # b: step 2: Define when to stop?
        if  np.abs(Z1) < eps and np.abs(Z2) < eps and t >= maxiter:
            print(f'{t:3d}: p1 = {p1:12.8f} -> excess demand -> {Z1:14.8f} and p2 = {p2:18.8f} -> excess demand {Z2:14.8f}')
            break
        # c. step 3: update p1 and p2
        p1 = p1 + kappa*Z1/alphas.size
        p2 = p2 + kappa*Z2/alphas.size

        # d. step 4: make function return values for both p1 and p2 with modified iteration printout
        if t < 5 or t%500 == 0:
            print(f'{t:3d}: p1 = {p1:12.8f} -> excess demand -> {Z1:14.8f} and p2 = {p2:18.8f} -> excess demand {Z2:14.8f}')
        elif t == 5:
            print('   ...')
        t += 1

# e. call find equilibrium function with defined parameters
p1 = 1
p2 = 1
kappa = 0.5
eps = 1e-8
p1 = find_equilibrium(alphas,p1,p2,e1,e2,e3,kappa=kappa,eps=eps)

```

      0: p1 =   1.15434354 -> excess demand -> 46303.06210182 and p2 =         0.96327349 -> excess demand -11017.95327569
      1: p1 =   1.27671409 -> excess demand -> 36711.16402205 and p2 =         0.93670601 -> excess demand -7970.24375230
      2: p1 =   1.37944193 -> excess demand -> 30818.35277116 and p2 =         0.91847489 -> excess demand -5469.33650090
      3: p1 =   1.46868506 -> excess demand -> 26772.93962492 and p2 =         0.90711916 -> excess demand -3406.71837664
      4: p1 =   1.54806962 -> excess demand -> 23815.36779893 and p2 =         0.90137981 -> excess demand -1721.80611963
       ...
    500: p1 =   5.93547693 -> excess demand ->   555.09286573 and p2 =         2.40971084 -> excess demand   207.27799461
    1000: p1 =   6.37712322 -> excess demand ->   105.23505785 and p2 =         2.57456027 -> excess demand    39.26553955
    1500: p1 =   6.46578148 -> excess demand ->    22.31026660 and p2 =         2.60763836 -> excess demand     8.32331914
    2000: p1 =   6.48478182 -> excess demand ->     4.83263398 and p2 =         2.61472674 -> excess demand     1.80286534
    2500: p1 =   6.48890693 -> excess demand ->     1.05158964 and p2 =         2.61626565 -> excess demand     0.39230425
    3000: p1 =   6.48980501 -> excess demand ->     0.22905431 and p2 =         2.61660069 -> excess demand     0.08545050
    3500: p1 =   6.49000065 -> excess demand ->     0.04990272 and p2 =         2.61667367 -> excess demand     0.01861659
    4000: p1 =   6.49004327 -> excess demand ->     0.01087252 and p2 =         2.61668957 -> excess demand     0.00405608
    4500: p1 =   6.49005256 -> excess demand ->     0.00236887 and p2 =         2.61669304 -> excess demand     0.00088372
    5000: p1 =   6.49005458 -> excess demand ->     0.00051612 and p2 =         2.61669379 -> excess demand     0.00019254
    5500: p1 =   6.49005502 -> excess demand ->     0.00011245 and p2 =         2.61669395 -> excess demand     0.00004195
    6000: p1 =   6.49005512 -> excess demand ->     0.00002450 and p2 =         2.61669399 -> excess demand     0.00000914
    6500: p1 =   6.49005514 -> excess demand ->     0.00000534 and p2 =         2.61669400 -> excess demand     0.00000199
    7000: p1 =   6.49005514 -> excess demand ->     0.00000116 and p2 =         2.61669400 -> excess demand     0.00000043
    7500: p1 =   6.49005515 -> excess demand ->     0.00000025 and p2 =         2.61669400 -> excess demand     0.00000009
    8000: p1 =   6.49005515 -> excess demand ->     0.00000006 and p2 =         2.61669400 -> excess demand     0.00000002
    8500: p1 =   6.49005515 -> excess demand ->     0.00000001 and p2 =         2.61669400 -> excess demand     0.00000000
    8561: p1 =   6.49005515 -> excess demand ->     0.00000001 and p2 =         2.61669400 -> excess demand     0.00000000

#### Conclusions for equilibrium prices:
With the demand for good 3 being zero, we get a Walras-equilibrium where **p1 = approx. 6.49 and p2 = approx 2.62**

**Question 4:** Plot the distribution of utility in the Walras-equilibrium and calculate its mean and variance.

To plot the distribution of utility in the Walras-equilibrium, we generate the function for utility as seen in the start of the task, and use the prices found in Q3 to find the different distributions. <br/>

We do this by defining the demand functions for each good, multiply with the beta values for each good, and return it all through the generated utility function. Finally we plot the results with the DataFrame method as Q1. We thereafter use numpy to find mean, median, variance, and max.

```python
#First generate the demandfunctions and the beta functions, afterwards we
def Walras_equilibrium_utilityfunction(betas,p1,p2,e1,e2,e3,gamma):
    Demand1 = demand_func1(beta_1,p1,p2,e1,e2,e3)
    Demand2 = demand_func2(beta_1,p1,p2,e1,e2,e3)
    Demand3 = demand_func3(beta_1,p1,p2,e1,e2,e3)

    Beta1 = np.array([x[0] for x in betas])
    Beta2 = np.array([x[1] for x in betas])
    Beta3 = np.array([x[2] for x in betas])

    return ((Demand1*Beta1)*(Demand2*Beta2)*(Demand3*Beta3))**gamma
```

```python
#Secondly we define our prices from Q3, run them through our Walras-Utilityfunction and plot the results using pandas.
p1 = 6.490
p2 = 2.617
Walras_data_func = Walras_equilibrium_utilityfunction(betas,p1,p2,e1,e2,e3,gamma)
Walras_data = pd.DataFrame(Walras_data_func).rename(columns={0:'Nr. of Agents'})

pd.DataFrame(Walras_data).plot(kind="hist",title="Distribution of Walras utility",grid=True,bins=10, cmap=plt.cm.summer)
plt.axis([0,6,0,50000])

pd.DataFrame(Walras_data).plot(kind="hist",title="Distribution of Walras utility zoom 1",grid=True,bins=40, cmap=plt.cm.autumn)
plt.axis([0,1,0,50000])

pd.DataFrame(Walras_data).plot(kind="hist",title="Distribution of Walras utility zoom 2",grid=True,bins=200, cmap=plt.cm.winter)
plt.axis([0,0.15,0,50000])

pd.DataFrame(Walras_data).plot(kind="box",title="Boxplot of the distribution of Walras utility", grid=True, cmap=plt.cm.spring)
plt.axis([0,2,0,6])
```

```python
#Finally we simply calculate mean and variance, and max, using numpy
print("The mean is ", np.mean(Walras_data_func))
print("The median is", np.median(Walras_data_func))
print("The variance is", np.var(Walras_data_func))
print("The highest observation is ", np.max(Walras_data_func))
```

    The mean is  0.06535541853385282
    The median is 0.014873998664104271
    The variance is 0.03134762216019003
    The highest observation is  5.897938370865469

#### Conclusions on Q4:
We find that the vast majority of the Walras utilities gater in the low end of the distribution with a rather low variance. The highest observed utility is around 5.89, which is much higher than the mean. As the median is lower than the mean, this indicates right skewedness, which is also clear from the boxplot, or lack of one, due to frequent outliers in the high end og the values.

**Question 5:** Find the Walras-equilibrium prices if instead all endowments were distributed equally. Discuss the implied changes in the distribution of utility. Does the value of $\gamma$ play a role for your conclusions?

To find the Walras-equilibrium prices for equally distributed endownments, the endownments are redefined as the mean of their individual values, and put through the same process as Q2, Q3 and Q4. Finally, we create a interactive figure to show the effects of changes in gamma on the distribution of utility.

```python
#Redefine each endownment using numpy taking the mean and using np.zeros so an array is generated insted of a single value
e1 = np.mean(e1)+np.zeros(N)
e2 = np.mean(e2)+np.zeros(N)
e3 = np.mean(e3)+np.zeros(N)
```

```python
#Same procedure as Q2,Q3 and Q4:
#First we define the demand functions
def demand_func1(beta_1,p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,0]*I/p1
def demand_func2(beta_2,p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,1]*I/p2
def demand_func3(beta_3, p1,p2,e1,e2,e3):
    I = p1*e1+p2*e2+e3
    return betas[:,2]*I/1
```

```python
#Secondly, we can define the excess demand functions
def excess_demand1(betas,p1,p2,e1,e2,e3):
    demand1 = np.sum(demand_func1(betas,p1,p2,e1,e2,e3))
    supply1 = np.sum(e1)
    excess_demand = demand1-supply1
    return excess_demand

def excess_demand2(betas,p1,p2,e1,e2,e3):
    demand2 = np.sum(demand_func2(betas,p1,p2,e1,e2,e3))
    supply2 = np.sum(e2)
    excess_demand  = demand2-supply2
    return excess_demand

def excess_demand3(betas,p1,p2,e1,e2,e3):
    demand3 = np.sum(demand_func3(betas,p1,p2,e1,e2,e3))
    supply3 = np.sum(e3)
    excess_demand = demand3-supply3
    return excess_demand
```

```python
#As most parameters and the demand and excess demand functions are already set up, we start by defineing the equilibirum function as in problemset 2
def find_equilibrium(alphas,p1,p2,e1,e2,e3,kappa=0.5,eps=1e-8,maxiter=500):

    t = 0
    while True:
        # a. step 1: define excess demand
        Z1 = excess_demand1(alphas,p1,p2,e1,e2,e3)
        Z2 = excess_demand2(alphas,p1,p2,e1,e2,e3)

        # b: step 2: Define when to stop?
        if  np.abs(Z1) < eps and np.abs(Z2) < eps and t >= maxiter:
            print(f'{t:3d}: p1 = {p1:12.8f} -> excess demand -> {Z1:14.8f} and p2 = {p2:18.8f} -> excess demand {Z2:14.8f}')
            break
        # c. step 3: update p1 and p2
        p1 = p1 + kappa*Z1/alphas.size
        p2 = p2 + kappa*Z2/alphas.size

        # d. step 4: make function return values for both p1 and p2 with modified iteration printout
        if t < 5 or t%500 == 0:
            print(f'{t:3d}: p1 = {p1:12.8f} -> excess demand -> {Z1:14.8f} and p2 = {p2:18.8f} -> excess demand {Z2:14.8f}')
        elif t == 5:
            print('   ...')
        t += 1

# e. call find equilibrium function with defined parameters
p1 = 1
p2 = 1
kappa = 0.5
eps = 1e-8
p1 = find_equilibrium(alphas,p1,p2,e1,e2,e3,kappa=kappa,eps=eps)

```

      0: p1 =   1.15443484 -> excess demand -> 46330.45322633 and p2 =         0.96321076 -> excess demand -11036.77294234
      1: p1 =   1.27685472 -> excess demand -> 36725.96313815 and p2 =         0.93659869 -> excess demand -7983.62083163
      2: p1 =   1.37961171 -> excess demand -> 30827.09694193 and p2 =         0.91833778 -> excess demand -5478.27168623
      3: p1 =   1.46887242 -> excess demand -> 26778.21126920 and p2 =         0.90696448 -> excess demand -3411.99200640
      4: p1 =   1.54826733 -> excess demand -> 23818.47424070 and p2 =         0.90121737 -> excess demand -1724.13206651
       ...
    500: p1 =   5.93315437 -> excess demand ->   554.03181223 and p2 =         2.41062773 -> excess demand   207.16645183
    1000: p1 =   6.37364924 -> excess demand ->   104.82980595 and p2 =         2.57527414 -> excess demand    39.16838681
    1500: p1 =   6.46189659 -> excess demand ->    22.17738553 and p2 =         2.60824444 -> excess demand     8.28518999
    2000: p1 =   6.48076821 -> excess demand ->     4.79334756 and p2 =         2.61529454 -> excess demand     1.79068332
    2500: p1 =   6.48485636 -> excess demand ->     1.04073422 and p2 =         2.61682178 -> excess demand     0.38879177
    3000: p1 =   6.48574442 -> excess demand ->     0.22618678 and p2 =         2.61715353 -> excess demand     0.08449750
    3500: p1 =   6.48593744 -> excess demand ->     0.04916853 and p2 =         2.61722564 -> excess demand     0.01836808
    4000: p1 =   6.48597940 -> excess demand ->     0.01068876 and p2 =         2.61724132 -> excess demand     0.00399304
    4500: p1 =   6.48598853 -> excess demand ->     0.00232366 and p2 =         2.61724472 -> excess demand     0.00086806
    5000: p1 =   6.48599051 -> excess demand ->     0.00050515 and p2 =         2.61724547 -> excess demand     0.00018871
    5500: p1 =   6.48599094 -> excess demand ->     0.00010982 and p2 =         2.61724563 -> excess demand     0.00004102
    6000: p1 =   6.48599103 -> excess demand ->     0.00002387 and p2 =         2.61724566 -> excess demand     0.00000892
    6500: p1 =   6.48599105 -> excess demand ->     0.00000519 and p2 =         2.61724567 -> excess demand     0.00000194
    7000: p1 =   6.48599106 -> excess demand ->     0.00000113 and p2 =         2.61724567 -> excess demand     0.00000042
    7500: p1 =   6.48599106 -> excess demand ->     0.00000025 and p2 =         2.61724567 -> excess demand     0.00000009
    8000: p1 =   6.48599106 -> excess demand ->     0.00000005 and p2 =         2.61724567 -> excess demand     0.00000002
    8500: p1 =   6.48599106 -> excess demand ->     0.00000001 and p2 =         2.61724567 -> excess demand     0.00000000
    8549: p1 =   6.48599106 -> excess demand ->     0.00000001 and p2 =         2.61724567 -> excess demand     0.00000000

The new prices are **p1 = 6.486 and p2 = 2.617**

```python
#Now generate the new demandfunctions and the beta functions
def Walras_equalendownment_equilibrium_utilityfunction(betas,p1,p2,e1,e2,e3,gamma):
    Demand1 = demand_func1(betas,p1,p2,e1,e2,e3)
    Demand2 = demand_func2(betas,p1,p2,e1,e2,e3)
    Demand3 = demand_func3(betas,p1,p2,e1,e2,e3)

    Beta1 = np.array([x[0] for x in betas])
    Beta2 = np.array([x[1] for x in betas])
    Beta3 = np.array([x[2] for x in betas])

    return ((Demand1*Beta1)*(Demand2*Beta2)*(Demand3*Beta3))**gamma
```

```python
#Finally we define our prices from Q3, run them through our Walras-Utilityfunction and plot the results using pandas.
p1 = 6.486
p2 = 2.617
Walras_data_func = Walras_equalendownment_equilibrium_utilityfunction(betas,p1,p2,e1,e2,e3,gamma)
Walras_data = pd.DataFrame(Walras_data_func).rename(columns={0:'Nr. of Agents'})

pd.DataFrame(Walras_data).plot(kind="hist",title="Distribution of Walras utility",grid=True,bins=200, cmap=plt.cm.summer)
plt.axis([0,0.14,0,800])

pd.DataFrame(Walras_data).plot(kind="hist",title="Distribution of Walras utility zoom 1",grid=True,bins=200, cmap=plt.cm.autumn)
plt.axis([0,0.06,0,800])

pd.DataFrame(Walras_data).plot(kind="box",title="Boxplot of the distribution of Walras utility", cmap=plt.cm.spring)
plt.axis([0,2,0,0.15])
```

```python
#Again we simply calculate mean, median, variance, and max, using numpy
print("The mean is ", np.mean(Walras_data_func))
print("The median is", np.median(Walras_data_func))
print("The varianse is", np.var(Walras_data_func))
print("The highest observation is ", np.max(Walras_data_func))
```

    The mean is  0.03440934151059249
    The median is 0.026899972248576272
    The varianse is 0.000759513584718288
    The highest observation is  0.1363094967261889

```python
#And to add a little extra interpretation of the value of gamma, create a interactive figure to see the difference
def Walras_equalendow_slider_func(betas,p1,p2,e1,e2,e3,gamma):

    Demand1 = demand_func1(betas,p1,p2,e1,e2,e3)
    Demand2 = demand_func2(betas,p1,p2,e1,e2,e3)
    Demand3 = demand_func3(betas,p1,p2,e1,e2,e3)

    Beta1 = np.array([x[0] for x in betas])
    Beta2 = np.array([x[1] for x in betas])
    Beta3 = np.array([x[2] for x in betas])

    final = ((Demand1*Beta1)*(Demand2*Beta2)*(Demand3*Beta3))**gamma
    final = pd.DataFrame(final).rename(columns={0:'Nr. of Agents'})
    pd.DataFrame(final).plot(kind="hist",title="Distribution of Walras utility given different gammas",grid=True, cmap=plt.cm.ocean)

    return plt.show()

widgets.interact(Walras_equalendow_slider_func,
                 betas=fixed(betas),
                 p1 = fixed(p1),
                 p2 = fixed(p2),
                 e1 = fixed(e1),
                 e2 = fixed(e2),
                 e3 = fixed(e3),
                 gamma=(0,1.5,0.1))
```

    interactive(children=(FloatSlider(value=0.7000000000000001, description='gamma', max=1.5), Output()), _dom_cla…

    <function __main__.Walras_equalendow_slider_func(betas, p1, p2, e1, e2, e3, gamma)>

#### Conclusions for Q5:

As can we seen from both the new figures, and calculations, the mean and median are both lower, but the right skewedness is still present. There are much fewer outliers, and the ones that excist are closer to the other values. In other words, the variance is much lower, and the highest observation is also much lower. As can be interpreted from the interactive figure, gamma in fact has an effet on both mean and variance. Low gamma values, has the highest means, with 0 having a mean of one. High gamma values decrease the mean, which can be seen when moving the slider.

```python

```


# MCMC integration test

Here, we'll test performance of the nGP model on sample data.

```python
import numpy as np
import ngptools as ngp
import scipy.stats as stats
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
sns.set_style('darkgrid')

np.random.seed(12355)  # set random seed
```

### Load data

```python
data = np.loadtxt('data/blocks_sampled.csv', delimiter=',')
tobs = data[:, 0]
y = data[:, 1].reshape(-1, 1)
Nt = y.size
```

### Define some useful variables

```python
delta = np.diff(np.concatenate([tobs, [1.1]]))
Np, Nm, Nr = 1, 3, 3
dims = Np, Nm, Nr
```

```python
plt.figure(figsize=(12, 8))
plt.scatter(tobs, y)
pass
```

## Pick priors and starting values for parameters

```python
sigU = 1e3
sigA = 5
sigeps = 2.
sigmu = 4.
sigalpha = 4.
a, b = 1., 1.
```

## Algorithm

The algorithm is as follows:

1. Given the various $\sigma^2$ parameters, perform an exact draw from the posterior for the state vector
    $(U, DU, A)$.
1. Given the state vector, draw
    $$
    \sigma^2_\epsilon \sim \text{inv-Ga}\left(
    a + \frac{N_t}{2},
    b + \frac{1}{2} \sum_t (y_t - U(t))^2
    \right)
    $$
1. Sample $\sigma^2_U$ and $\sigma^2_A$:

    1. Draw a sample of the state vector, $(U^*, DU^*, A^*)$, from the approximate model.
    1. Conditioned on this approximate state vector, sample proposed variance parameters as
        $$\begin{align}
        \sigma^2 &\sim \text{inv-Ga}\left(a + \frac{N_t}{2}, b + \frac{1}{2} R^2 \right) \\
        R^2_U &= \sum_j \delta_j \left[\frac{DU^*(t_{j+1}) - DU^*(t_j)}{\delta_j} - A^*(t_j) \right]^2 \\
        R^2_A &= \sum_j \left[\frac{(A^*(t_{j+1}) - A^*(t_j))^2}{\delta_j}\right]
        \end{align}$$
    1. Accept these proposals with probability $\mathrm{min}(\pi, 1)$ with
        $$
        \pi \equiv \prod_j \frac{\mathcal{N}(\theta_{j + 1} - G_j\theta_j|0, W^*_j)}
        {\mathcal{N}(\theta_{j + 1} - G_j\theta_j|0, W_j)}
        \frac{\mathcal{N}(\tilde{H}^\top (\theta^*_{j + 1} - \tilde{G}_j\theta^*_j)|0, \tilde{W}_j)}
        {\mathcal{N}(\tilde{H}^\top (\theta^*_{j + 1} - \tilde{G}_j\theta^*_j)|0, \tilde{W}^*_j)}
        $$
        (Think of this like $\frac{P(x')}{P(x)}\frac{g(x)}{g(x')}$, with $P$ the posterior and $g$ the proposal
        distribution, and recall that what we are proposing is $\sigma^2|\theta^*$).

### Useful functions:

```python
def draw_state(sigeps, sigU, sigA, approx=False):
    return ngp.sample(y, 1, dims, delta, sigeps, sigU, sigA, sigmu, sigalpha, approx)[0]
```

```python
def draw_sig_eps(U):
    a_eff = a + (Nt / 2)
    b_eff = b + 0.5 * np.sum((y.ravel() - U)**2)
    return np.sqrt(stats.invgamma.rvs(a=a_eff, scale=b_eff))
```

```python
def calc_acceptance_prob(theta, theta_star, G, Gtilde, W, Wstar, Wtilde, Wtilde_star, Htilde):
    Nt = theta.shape[0]
    v = np.empty_like(theta)
    v_star = np.empty((theta.shape[0], Htilde.shape[-1]))

    for t in range(v.shape[0] - 1):
        v[t] = theta[t + 1] - G[t].dot(theta[t])
        v_star[t] = Htilde.T.dot(theta_star[t + 1] - Gtilde[t].dot(theta_star[t]))

    pi = 0
    for t in range(Nt):
        pi += stats.multivariate_normal.logpdf(x=v[t], mean=None, cov=Wstar[t])
        pi -= stats.multivariate_normal.logpdf(x=v[t], mean=None, cov=W[t])
        pi += stats.multivariate_normal.logpdf(x=v_star[t], mean=None, cov=Wtilde[t])
        pi -= stats.multivariate_normal.logpdf(x=v_star[t], mean=None, cov=Wtilde_star[t])

    return np.min([np.exp(pi), 1])
```

```python
def draw_sigs(theta, sigeps, sigU, sigA):
    Nt = theta.shape[0]

    # draw state from approximate model
    theta_star = draw_state(sigeps, sigU, sigA, approx=True)

    Ustar = theta_star[:, 0]
    DUstar = theta_star[:, 1]
    Astar = theta_star[:, 2]

    # calculate sufficient statistics
    R2U = delta[:-1].dot(((np.diff(DUstar) / delta[:-1]) - Astar[:-1]) ** 2)
    R2A = np.sum(np.diff(Astar)**2 / delta[:-1])

    # draw proposals
    a_eff = a + (Nt / 2)
    sigU_star = np.sqrt(stats.invgamma.rvs(a=a_eff, scale=b + 0.5 * R2U))
    sigA_star = np.sqrt(stats.invgamma.rvs(a=a_eff, scale=b + 0.5 * R2A))

    assert(sigU_star > 0)
    assert(sigA_star > 0)

    _, _, G, H, W, _, _ = ngp._assemble_matrices(dims, delta, sigeps, sigU,
                                                 sigA, sigmu, sigalpha, approx=False)
    _, _, Gtilde, Htilde, Wtilde, _, _ = ngp._assemble_matrices(dims, delta, sigeps, sigU,
                                                 sigA, sigmu, sigalpha, approx=True)
    _, _, _, _, Wstar, _, _ = ngp._assemble_matrices(dims, delta, sigeps, sigU_star,
                                                 sigA_star, sigmu, sigalpha, approx=False)
    _, _, _, _, Wtilde_star, _, _ = ngp._assemble_matrices(dims, delta, sigeps, sigU_star,
                                                 sigA_star, sigmu, sigalpha, approx=True)

    # get acceptance probability
    acc_prob = calc_acceptance_prob(theta, theta_star, G, Gtilde, W, Wstar, Wtilde, Wtilde_star, Htilde)

    if np.random.rand() < acc_prob:
        return sigU_star, sigA_star
    else:
        return sigU, sigA
```

```python
from IPython import display
plt.figure(figsize=(12, 8))

Niter = 1500
doplot = True

# initialize
th = np.empty((Niter + 1, Nt, Nm))
sig = np.empty((Niter + 1, 3))
sig[0] = sigeps, sigU, sigA
th[0] = np.nan

# draw samples
for idx in range(Niter):
    theta = draw_state(sigeps, sigU, sigA)
    sigeps = draw_sig_eps(theta[:, 0])
    sigU, sigA = draw_sigs(theta, sigeps, sigU, sigA)

    th[idx + 1] = theta
    sig[idx + 1] = sigeps, sigU, sigA

    # update plot
    if doplot:
        display.display(plt.gcf())
        plt.clf()
        plt.scatter(tobs, y)
        plt.plot(tobs, theta[:, 0])
        plt.xlim(0, 1)
        plt.ylim(-6, 10)
        plt.title("Iteration {}".format(idx))
        display.clear_output(wait=True)
```

```python
sigeps, sigU, sigA
```

    (0.78319878456071945, 609.92783986867062, 0.89549061219769255)

```python
plt.figure(figsize=(12, 8))
trans = 5 / Niter

plt.plot(tobs, th[500:, :, 0].T, color=sns.color_palette()[0], alpha=trans)
plt.scatter(tobs, y)
plt.xlim(0, 1)
pass
```

```python
th.shape
```

    (1501, 128, 3)

```python

```


# Watching Paint Dry

The composited _color_ of the **paint** must not change during drying. The
optical blending function is used with this constraint to compensate for the new
dry layer $C_{d}^{\prime}$, when some volume $\delta_{\alpha}$ is removed from
the wet layer.

$$
C_{d}^{\prime} = \frac{\alpha \times C_{w} + (1 - \delta) \times C_d - a^{\prime}C_w}{1 - \alpha^\prime}, \alpha^\prime = \alpha - \delta_\alpha
$$

The dry layer of the canvas uses a relative height field to allow for unlimited
volume of paint to be added, with a constraint only on the relative change in
height between texels.

```python
from IPython.display import Markdown
Markdown(r"""# Non-Homogeneous Poisson Process Likelihood Function
$$
\frac{\left(Λ_{0,T}\left(t\right)\right)^k e^{\left(Λ_{0,T}\left(t\right)\right)}}{k!}
$$""")
```

# Non-Homogeneous Poisson Process Likelihood Function
$$
\frac{\left(Λ_{0,T}\left(t\right)\right)^k e^{\left(Λ_{0,T}\left(t\right)\right)}}{k!}
$$

```python
from IPython.display import Latex
display(Latex(r"\LaTeX"))
```

\LaTeX

```python
from IPython.display import Latex
Latex(r"""\begin{eqnarray}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = & \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = & 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = & \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = & 0
\end{eqnarray}""")
```

\begin{eqnarray}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = & \frac{4\pi}{c}\vec{\mathbf{j}} \\
\nabla \cdot \vec{\mathbf{E}} & = & 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = & \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = & 0
\end{eqnarray}

https://github.com/nteract/nteract/issues/2434

$\hat{y}_{i}=\hat\beta_{0}+\hat\beta_{1}x_{i}$

$\sum\limits_{i=1}^nx_i$

https://github.com/nteract/nteract/issues/2012

$$
\begin{align}
a &= 1 \\
b &= 2
\end{align}
$$

https://github.com/nteract/nteract/issues/1557

| Age | Number of Cars owned | Owns House | Number of Children| Marital status (s/m/d/w) | Owns a dog | Bought a Boat|
|-|-|-|-|-|-|-|
|66|1|y|2|w|n|y|
|52|2|y|3|m|n|y|
|22|0|n|0|m|y|n|
|25|1|n|1|s|n|n|
|44|0|n|2|d|y|n|
|39|1|y|2|m|y|n|
|26|1|n|2|s|n|n|
|40|3|y|1|m|y|n|
|53|2|y|2|d|n|y|
|64|2|y|3|d|n|n|
|58|2|y|2|m|y|y|
|33|1|n|1|s|n|n|

~~hey~~

```python
import this

print('hey')
```

1. Thing

1. This is a numbered list
2. And within it
  * There's a bulleted list which should be indented
3. It's not, and totally should
4. Hopefully this gets fixed


```python
import numpy as np
import matplotlib.pyplot as plt
import scipy.sparse as sps
```

# training set
Our training set is a simple Gaussian distribution

```python
def gaussian_pdf(num_bit, mu, sigma):
    '''get gaussian distribution function'''
    x = np.arange(2**num_bit)
    pl = 1. / np.sqrt(2 * np.pi * sigma**2) * \
        np.exp(-(x - mu)**2 / (2. * sigma**2))
    return pl/pl.sum()

num_bit = 6
pg = gaussian_pdf(num_bit, mu=2**5-0.5, sigma=2**4)
plt.plot(pg)
plt.show()
```

# Build Circuits
## Building Blocks
Define matrix representations of operations using scipy sparse matrices.
They are grouped to become a layer in a circuit, this layer can be `ArbitraryRotation` or `CNOTEntangler`.
Which are used as our basic building blocks of our `Born Machines`.

```python
######  Pauli Matrices  ########
I2 = sps.eye(2).tocsr()
sx = sps.csr_matrix([[0,1],[1,0.]])
sy = sps.csr_matrix([[0,-1j],[1j,0.]])
sz = sps.csr_matrix([[1,0],[0,-1.]])

p0 = (sz + I2) / 2   #projection operator to |0><0|
p1 = (-sz + I2) / 2

# single bit rotation matrices
def _ri(si, theta):
    return np.cos(theta/2.)*I2 - 1j*np.sin(theta/2.)*si

def rot(t1, t2, t3):
    '''
    a general rotation gate rz(t3)rx(r2)rz(t1).

    Args:
        t1, t2, t3 (float): three angles.

    Returns:
        2x2 csr_matrix: rotation matrix.
    '''
    return _ri(sz, t3).dot(_ri(sx, t2)).dot(_ri(sz, t1))

# multiple bit construction
def CNOT(ibit, jbit, n):
    '''
    CNOT gate

    Args:
        ibit (int): control bit.
        jbit (int): controled bit.
        n (int): total number of qubits.

    Returns:
        4x4 csr_matrix: CNOT matrix.
    '''
    res = _([p0, I2], [ibit, jbit], n)
    res = res + _([p1, sx], [ibit, jbit], n)
    return res

def _(ops, locs, n):
    '''
    Compile operators into specific Hilbert space.

    notice the last bit is the high end bit!

    Args:
        ops (list): list of single bit operators.
        locs (list): list of positions.
        n (int): total number of bits.

    Returns:
        csr_matrix: resulting matrix.
    '''
    if np.ndim(locs) == 0:
        locs = [locs]
    if not isinstance(ops, (list, tuple)):
        ops = [ops]
    locs = np.asarray(locs)
    locs = n - locs
    order = np.argsort(locs)
    locs = np.concatenate([[0], locs[order], [n + 1]])
    return _wrap_identity([ops[i] for i in order], np.diff(locs) - 1)

def _wrap_identity(data_list, num_bit_list):
    if len(num_bit_list) != len(data_list) + 1:
        raise Exception()

    res = sps.eye(2**num_bit_list[0])
    for data, nbit in zip(data_list, num_bit_list[1:]):
        res = sps.kron(res, data)
        res = sps.kron(res, sps.eye(2**nbit, dtype='complex128'))
    return res

def initial_wf(num_bit):
    '''initial wave function |00...0>.'''
    wf = np.zeros(2**num_bit, dtype='complex128')
    wf[0] = 1.
    return wf
```

```python
class ArbitraryRotation(object):
    '''Arbitrary rotation gate'''
    def __init__(self, num_bit):
        self.num_bit = num_bit
        # mask is used to filter out some irrelevant (marked False) parameters
        self.mask = np.array([True] * (3*num_bit), dtype='bool')

    @property
    def num_param(self):
        return self.mask.sum()

    def tocsr(self, theta_list):
        '''transform this block to a sequence of csr_matrices.'''
        theta_list_ = np.zeros(3*self.num_bit)
        theta_list_[self.mask] = theta_list
        rots = [rot(*ths) for ths in theta_list_.reshape([self.num_bit,3])]
        res = [_([r], [i], self.num_bit) for i,r in enumerate(rots)]
        return res

class CNOTEntangler(object):
    '''
    CNOT Entangler Layer.

    Args:
        pairs (list): a list of tuples to represent connections.
    '''
    def __init__(self, num_bit, pairs):
        self.num_bit = num_bit
        self.pairs = pairs

    @property
    def num_param(self):
        return 0

    def tocsr(self, theta_list):
        '''transform this block to a sequence of csr_matrices.'''
        i, j = self.pairs[0]
        res = CNOT(i, j, self.num_bit)
        for i, j in self.pairs[1:]:
            res = CNOT(i,j,self.num_bit).dot(res)
        res.eliminate_zeros()
        return [res]
```

# Circuit
A circuit is consist of queued layers, here this data structure is `BlockQueue`. A `BlockQueue` uses `theta_last` and `memo` attributes to cache sparse operators, this will accelerate simulation significantly.

```python
class BlockQueue(list):
    '''
    Block Queue that keep track of theta_list changing history, for fast update.

    Attributes:
        theta_last (1darray): the cached circuit parameters.
        memo (list): cached sparse matrices for layers.
    '''
    def __init__(self, *args):
        list.__init__(self, *args)
        self.theta_last = None
        self.memo = None

    def __call__(self, qureg, theta_list):
        '''
        Apply operations on vector basis: qureg. This operation changes vector inplace.

        Args:
            qureg (1darray): vector basis.
            theta_list (1darray): circuit parameters:
        '''
        # cache? if theta_list change <= 1 parameters, then don't touch memory.
        remember = self.theta_last is None or (abs(self.theta_last-theta_list)>1e-12).sum() > 1

        mats = []
        theta_last = self.theta_last
        if remember:
            self.theta_last = theta_list.copy()

        qureg_ = qureg
        for iblock, block in enumerate(self):
            # generate or use a block matrix
            num_param = block.num_param
            theta_i, theta_list = np.split(theta_list, [num_param])
            if theta_last is not None:
                theta_o, theta_last = np.split(theta_last, [num_param])
            if self.memo is not None and (num_param==0 or np.abs(theta_i-theta_o).max()<1e-12):
                # use data cached in memory
                mat = self.memo[iblock]
            else:
                if self.memo is not None and not remember:
                    # update the changed gate, but not touching memory.
                    mat = _rot_tocsr_update1(block, self.memo[iblock], theta_o, theta_i)
                else:
                    # regenerate one
                    mat = block.tocsr(theta_i)
            for mat_i in mat:
                qureg_ = mat_i.dot(qureg_)
            mats.append(mat)

        if remember:
            # cache data
            self.memo = mats
        # update register
        qureg[...] = qureg_
        np.testing.assert_(len(theta_list)==0)

    @property
    def num_bit(self):
        return self[0].num_bit

    @property
    def num_param(self):
        return sum([b.num_param for b in self])

def _rot_tocsr_update1(layer, old, theta_old, theta_new):
    '''
    rotation layer csr_matrices update method.

    Args:
        layer (ArbitraryRotation): rotation layer.
        old (csr_matrix): old matrices.
        theta_old (1darray): old parameters.
        theta_new (1darray): new parameters.

    Returns:
        list of csr_matrix: new rotation matrices after the theta changed.
    '''
    idiff_param = np.where(abs(theta_old-theta_new)>1e-12)[0].item()
    idiff = np.where(layer.mask)[0][idiff_param]

    # get rotation parameters
    isite = idiff//3
    theta_list_ = np.zeros(3*layer.num_bit)
    theta_list_[layer.mask] = theta_new

    new = old[:]
    new[isite] = _(rot(*theta_list_[isite*3:isite*3+3]), isite, layer.num_bit)
    return new
```

```python
def get_nn_pairs(num_bit):
    '''get nearest neighbor pairs.'''
    res = []
    for inth in range(2):
        for i in range(inth, num_bit, 2):
            res = res + [(i, i_ % num_bit) for i_ in range(i + 1, i + 2)]
    return res

def get_demo_circuit(num_bit, depth, pairs):
    '''Get the circuit used for demo'''
    blocks = []
    # build circuit
    for idepth in range(depth+1):
        blocks.append(ArbitraryRotation(num_bit))
        if idepth!=depth:
            blocks.append(CNOTEntangler(num_bit, pairs))

    # set leading and trailing Rz to disabled
    blocks[0].mask[::3] = False
    blocks[-1].mask[2::3] = False
    return BlockQueue(blocks)
```

```python
# let's see how this circuit look like
depth = 6
pairs = get_nn_pairs(num_bit)
circuit = get_demo_circuit(num_bit, depth, pairs)
print(circuit)
```

    [<__main__.ArbitraryRotation object at 0x7f45778f8668>, <__main__.CNOTEntangler object at 0x7f45778f86d8>, <__main__.ArbitraryRotation object at 0x7f45778f8710>, <__main__.CNOTEntangler object at 0x7f45778f8748>, <__main__.ArbitraryRotation object at 0x7f45778f8780>, <__main__.CNOTEntangler object at 0x7f45778f87b8>, <__main__.ArbitraryRotation object at 0x7f45778f87f0>, <__main__.CNOTEntangler object at 0x7f45778f8828>, <__main__.ArbitraryRotation object at 0x7f45778f8860>, <__main__.CNOTEntangler object at 0x7f45778f8898>, <__main__.ArbitraryRotation object at 0x7f45778f88d0>, <__main__.CNOTEntangler object at 0x7f45778f8908>, <__main__.ArbitraryRotation object at 0x7f45778f8940>]

```python
# let's see how this circuit works
# rotating the last bit along x axis by pi/2.
theta_list = np.zeros(circuit.num_param)
theta_list[-1] = np.pi/2.

wf = initial_wf(num_bit)
circuit(wf, theta_list)

# now you see the last bit is the high end bit!
print(wf)
```

    [0.70710678+0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        -0.70710678j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j         0.        +0.j         0.        +0.j
     0.        +0.j        ]

# MMD Loss Function
As a reminder
\begin{align}
    \mathcal{L} =&  \left\|\sum_{x} p_\theta(x) \phi(x)- \sum_{x} \pi(x) \phi(x)  \right\|^2 \\
    =&\langle K(x,y)\rangle_{x\sim p_\theta, y\sim p_\theta}-2\langle K(x,y)\rangle_{x\sim p_\theta,y\sim \pi}+\langle K(x, y)\rangle_{x\sim \pi,y\sim \pi}
\end{align}

```python
class RBFMMD2(object):
    '''
    MMD^2 with RBF (Gaussian) kernel.

    Args:
        sigma_list (list): a list of bandwidths.
        basis (1darray): defininng space.

    Attributes:
        K (2darray): full kernel matrix, notice the Hilbert is countable.
    '''
    def __init__(self, sigma_list, basis):
        self.sigma_list = sigma_list
        self.basis = basis
        self.K = mix_rbf_kernel(basis, basis, self.sigma_list)

    def __call__(self, px, py):
        '''
        Args:
            px (1darray, default=None): probability for data set x, used only when self.is_exact==True.
            py (1darray, default=None): same as px, but for data set y.

        Returns:
            float: loss.
        '''
        pxy = px-py
        return self.kernel_expect(pxy, pxy)

    def kernel_expect(self, px, py):
        '''
        expectation value of kernel function.

        Args:
            px (1darray): the first PDF.
            py (1darray): the second PDF.

        Returns:
            float: kernel expectation.
        '''
        return px.dot(self.K).dot(py)

def mix_rbf_kernel(x, y, sigma_list):
    '''
    multi-RBF kernel.

    Args:
        x (1darray|2darray): the collection of samples A.
        x (1darray|2darray): the collection of samples B.
        sigma_list (list): a list of bandwidths.

    Returns:
        2darray: kernel matrix.
    '''
    ndim = x.ndim
    if ndim == 1:
        exponent = np.abs(x[:, None] - y[None, :])**2
    elif ndim == 2:
        exponent = ((x[:, None, :] - y[None, :, :])**2).sum(axis=2)
    else:
        raise
    K = 0.0
    for sigma in sigma_list:
        gamma = 1.0 / (2 * sigma)
        K = K + np.exp(-gamma * exponent)
    return K
```

```python
hndim = 2**num_bit
# mmd loss
mmd = RBFMMD2(sigma_list=[0.25,4], basis=np.arange(2**num_bit))

# when exact match, MMD loss should be 0, e.g. two gaussians
print('Exact Match -> MMD = %.4f'%mmd(pg, pg))
# when not match, loss is not zero
prand = np.random.random(len(pg))
print('Not Match -> MMD = %.4f'%mmd(prand/prand.sum(), pg))
```

    Exact Match -> MMD = 0.0000
    Not Match -> MMD = 0.0223

# Build The Gradient training framework for Born Machine
The gradient of MMD loss is
\begin{eqnarray}
        \frac{\partial \mathcal{L}}{\partial \theta_l^i} &=&\langle{K(x,y)}\rangle_{x\sim p_{\theta^+}, y\sim p_\theta}-\langle {K(x,y)}\rangle_{x\sim p_{\theta^-},y\sim p_\theta}\\
        &-&\langle K(x,y)\rangle_{x\sim p_{\theta^+},y\sim \pi}+\langle {K(x,y)}\rangle_{x\sim p_{\theta^-},y\sim \pi}.
\end{eqnarray}

```python
class QCBM(object):
    '''
    Quantum Circuit Born Machine framework,

    Args:
        circuit (BlockQueue): the circuit architechture.
        mmd (RBFMMD2): maximum mean discrepancy.
        p_data (1darray): data probability distribution in computation basis.
        batch_size (int|None): introducing sampling error, None for no sampling error.
    '''
    def __init__(self, circuit, mmd, p_data, batch_size=None):
        self.circuit = circuit
        self.mmd = mmd
        self.p_data = p_data
        self.batch_size = batch_size

    @property
    def depth(self):
        '''defined by the number of entanglers'''
        return (len(self.circuit)-1)//2

    def pdf(self, theta_list):
        '''
        get probability distribution function.

        Args:
            theta_list (1darray): circuit parameters.

        Returns:
            1darray: probability distribution function.
        '''
        wf = initial_wf(self.circuit.num_bit)
        self.circuit(wf, theta_list)
        pl = np.abs(wf)**2
        # introducing sampling error
        if self.batch_size is not None:
            pl = prob_from_sample(sample_from_prob(np.arange(len(pl)), pl, self.batch_size),
                    len(pl))
        return pl

    def mmd_loss(self, theta_list):
        '''get the loss'''
        # get and cahe probability distritbution of Born Machine
        self._prob = self.pdf(theta_list)
        # use wave function to get mmd loss
        return self.mmd(self._prob, self.p_data)

    def gradient(self, theta_list):
        '''
        cheat and get gradient.
        '''
        prob = self.pdf(theta_list)
        grad = []
        for i in range(len(theta_list)):
            # pi/2 phase
            theta_list[i] += np.pi/2.
            prob_pos = self.pdf(theta_list)
            # -pi/2 phase
            theta_list[i] -= np.pi
            prob_neg = self.pdf(theta_list)
            # recover
            theta_list[i] += np.pi/2.

            grad_pos = self.mmd.kernel_expect(prob, prob_pos) - self.mmd.kernel_expect(prob, prob_neg)
            grad_neg = self.mmd.kernel_expect(self.p_data, prob_pos) - self.mmd.kernel_expect(self.p_data, prob_neg)
            grad.append(grad_pos - grad_neg)
        return np.array(grad)

    def gradient_numerical(self, theta_list, delta=1e-2):
        '''
        numerical differenciation.
        '''
        grad = []
        for i in range(len(theta_list)):
            theta_list[i] += delta/2.
            loss_pos = self.mmd_loss(theta_list)
            theta_list[i] -= delta
            loss_neg = self.mmd_loss(theta_list)
            theta_list[i] += delta/2.

            grad_i = (loss_pos - loss_neg)/delta
            grad.append(grad_i)
        return np.array(grad)

def sample_from_prob(x, pl, num_sample):
    '''
    sample x ~ pl.
    '''
    pl = 1. / pl.sum() * pl
    indices = np.arange(len(x))
    res = np.random.choice(indices, num_sample, p=pl)
    return np.array([x[r] for r in res])

def prob_from_sample(dataset, hndim):
    '''
    emperical probability from data.
    '''
    p_data = np.bincount(dataset, minlength=hndim)
    p_data = p_data / float(np.sum(p_data))
    return p_data
```

```python
# Born Machine
bm = QCBM(circuit, mmd, pg, batch_size=None)  # exact version
```

```python
theta_list = np.random.random(bm.circuit.num_param)*2*np.pi
print('MMD loss for Initial Circuit = %.4f'%bm.mmd_loss(theta_list))
```

    MMD loss for Initial Circuit = 0.0440

```python
# gradient test
# these two gradients should match.
g1 = bm.gradient(theta_list)
print(g1)
g2 = bm.gradient_numerical(theta_list)
print(g2)
```

    [ 0.00128546  0.00291719  0.00723749 -0.00212555 -0.00346394 -0.02016816
     -0.00328782 -0.00331141 -0.00163575  0.00344951 -0.00569198  0.00640275
     -0.01087395  0.02290944  0.01064659  0.00080618  0.00987338 -0.00198704
     -0.00293285 -0.0020108   0.01406099  0.01219367 -0.0071598  -0.00827463
     -0.00176098 -0.00610079 -0.01336093 -0.00128863 -0.01534401  0.00430679
      0.00263345  0.00169888 -0.00967233 -0.00678989  0.00091382  0.01800348
     -0.00441233 -0.01295716  0.00563317 -0.00734346 -0.00115724  0.01418793
      0.01608993 -0.00843482  0.01623441  0.00693906 -0.00473672  0.0004105
     -0.00503149 -0.00539715  0.00390338  0.00445442 -0.00231173  0.00106615
     -0.00127588 -0.01838359 -0.00504569  0.01889813 -0.01501975 -0.01206399
      0.01178966  0.00371226 -0.00705478  0.02368941 -0.02082921  0.00739589
     -0.0021577  -0.01477283  0.00244763 -0.00204842  0.00652822 -0.00984969
      0.02341545 -0.0020058   0.02421392  0.0110414   0.00014127  0.01062275
     -0.00643447 -0.01108307 -0.00313949 -0.02724891 -0.01278688  0.02732687
      0.01152765  0.01059827 -0.01195812 -0.00862762  0.00084189  0.00983612
      0.00131466 -0.00928871  0.00178275  0.01016269 -0.00956343 -0.00202157
      0.00352308 -0.002335    0.00360078  0.00273339 -0.00488223  0.00591172
      0.00146351  0.00280222  0.00017502 -0.0029901   0.00436573 -0.0085621
     -0.00592689 -0.01484778 -0.00706659  0.00452008  0.00867807  0.00718175]
    [ 0.00128542  0.00291719  0.00723746 -0.00212556 -0.00346392 -0.02016801
     -0.00328784 -0.00331133 -0.00163568  0.00344948 -0.00569199  0.00640267
     -0.01087391  0.02290925  0.01064655  0.00080618  0.00987343 -0.00198707
     -0.00293278 -0.00201083  0.0140608   0.01219353 -0.00715968 -0.00827451
     -0.00176097 -0.00610067 -0.01336084 -0.00128863 -0.01534388  0.00430675
      0.00263338  0.00169886 -0.00967222 -0.00678978  0.00091391  0.01800338
     -0.00441234 -0.01295712  0.00563318 -0.0073434  -0.00115716  0.01418772
      0.01608972 -0.00843474  0.01623432  0.00693902 -0.00473666  0.0004105
     -0.00503148 -0.00539708  0.00390345  0.00445426 -0.00231182  0.00106609
     -0.00127601 -0.0183835  -0.00504567  0.01889794 -0.01501969 -0.01206383
      0.01178963  0.00371229 -0.00705485  0.02368918 -0.0208291   0.00739584
     -0.00215777 -0.01477268  0.00244772 -0.00204841  0.00652816 -0.00984968
      0.02341519 -0.00200574  0.02421365  0.01104129  0.00014131  0.01062264
     -0.00643433 -0.0110831  -0.00313947 -0.02724875 -0.01278672  0.02732672
      0.01152756  0.01059819 -0.01195809 -0.00862755  0.00084186  0.00983597
      0.00131462 -0.00928857  0.00178273  0.01016249 -0.00956329 -0.00202151
      0.00352312 -0.00233488  0.00360066  0.00273341 -0.00488221  0.00591174
      0.00146349  0.00280217  0.00017502 -0.00299004  0.0043657  -0.00856198
     -0.00592664 -0.01484749 -0.00706653  0.00451995  0.008678    0.00718163]

# Training
In this basic introduction, I prepaired classical training method like `Adam` and `L-BFGS-B` (as well as other methods supported by scipy).

```python
  def train(bm, theta_list, method, max_iter=1000, step_rate=0.1):
    '''
    train a Born Machine.

    Args:
        bm (QCBM): quantum circuit born machine training strategy.
        theta_list (1darray): initial parameters.
        method ('Adam'|'L-BFGS-B'):
            * L-BFGS-B: efficient, but not noise tolerant.
            * Adam: noise tolerant.
        max_iter (int): maximum allowed number of iterations.
        step_rate (float): learning rate for Adam optimizer.

    Returns:
        (float, 1darray): final loss and parameters.
    '''
    step = [0]
    def callback(x, *args, **kwargs):
        step[0] += 1
        print('step = %d, loss = %s'%(step[0], bm.mmd_loss(x)))

    theta_list = np.array(theta_list)
    if method == 'Adam':
        try:
            from climin import Adam
        except:
            !pip install git+https://github.com/BRML/climin.git
            from climin import Adam
        optimizer = Adam(wrt=theta_list, fprime=bm.gradient,step_rate=step_rate)
        for info in optimizer:
            callback(theta_list)
            if step[0] == max_iter:
                break
        return bm.mmd_loss(theta_list), theta_list
    else:
        from scipy.optimize import minimize
        res = minimize(bm.mmd_loss, x0=theta_list,
                       method=method, jac = bm.gradient, tol=1e-12,
                       options={'maxiter': max_iter, 'disp': 0, 'gtol':1e-10, 'ftol':0},
                       callback=callback,
                       )
        return res.fun, res.x
```

```python
# random initial parameter
theta_list = np.random.random(bm.circuit.num_param)*2*np.pi

loss, theta_list = train(bm, theta_list, 'L-BFGS-B', max_iter=20)
pl = bm.pdf(theta_list)

# display training result
plt.plot(bm.p_data)
plt.plot(pl)
plt.legend(['Data', 'Quantum Circuit Born Machine'])
plt.show()
```

```python

```


```python
from sympy import *
from sympy import init_printing; init_printing(use_latex = 'mathjax')
import numpy as np

#n = int(input('Qué número de valores de energía desea aproximar?'))
n = 4

l, m, hbar = symbols('l m hbar', real = True, constant = True)
var('x,W')

H = ones(n,n)
S = ones(n,n)
U = ones(n,n)
CC = ones(n,n)

#F = [sympify(input('Ingrese la función {0}: '.format(i+1))) for i in range(n)]
F = [x*(l - x),(x**2)*((l - x)**2),x*(l - x)*((l/2)-x),(x**2)*((l - x)**2)*((l/2)-x)]

c = ones(n,n)
for i in range(n):
    for j in range(n):
        c[i,j] = sympify('c%d%d' %(j+1,i+1))

for j in range(1,n+1):                                                             #loop para llenar la matriz H
    for k in range(1,n+1):
        I = ((-hbar**2)/(2*m))
        integrando = I*(F[j-1])*diff(F[k-1], x, 2)
        A = integrate(integrando, (x, 0, l))
        integrandos = (F[j-1])*(F[k-1])
        B = integrate(integrandos, (x, 0, l))
        H[j-1,k-1] *= A
        S[j-1,k-1] *= B
        U[j-1,k-1] *= (H[j-1,k-1] -W*S[j-1,k-1])

E = U.det()
EE = solve(E,W)
#truco para ordenar los W
for i in range(0,n):
    EE[i] = EE[i]*((l**2)/I)
EE.sort()
for i in range(0,n):
    EE[i] = EE[i]*(I/(l**2))

cc = Matrix(c)

for j in range(n):
    for i in range(n):
        C = U*cc.col(j)
        CC[i,j] *= C[i].subs(W, EE[j])
F = []
for m in range(n):
    D = solve(CC.col(m),cc)
    F.append(list(D.items()))

F = Matrix(F)
F
```

$$\left[\begin{matrix}\left ( c_{12}, \quad 0\right ) & \left ( c_{11}, \quad 0\right ) & \left ( c_{13}, \quad - \frac{\sqrt{5} c_{14}}{11} l^{2} + \frac{c_{14} l^{2}}{33}\right )\\\left ( c_{23}, \quad 0\right ) & \left ( c_{21}, \quad - \frac{\sqrt{133} c_{22}}{21} l^{2} + \frac{c_{22} l^{2}}{3}\right ) & \left ( c_{24}, \quad 0\right )\\\left ( c_{32}, \quad 0\right ) & \left ( c_{31}, \quad 0\right ) & \left ( c_{33}, \quad \frac{c_{34} l^{2}}{33} + \frac{\sqrt{5} c_{34}}{11} l^{2}\right )\\\left ( c_{44}, \quad 0\right ) & \left ( c_{41}, \quad \frac{c_{42} l^{2}}{3} + \frac{\sqrt{133} c_{42}}{21} l^{2}\right ) & \left ( c_{43}, \quad 0\right )\end{matrix}\right]$$


# Neuromatch Academy: Week 1, Day 3, Tutorial 2
# Model Fitting: Linear regression with MLE

**Content creators**: Pierre-Étienne Fiquet, Anqi Wu, Alex Hyafil  with help from Byron Galbraith

**Content reviewers**: Lina Teichmann, Madineh Sarvestani, Patrick Mineault, Ella Batty, Michael Waskom

---
#Tutorial Objectives

This is Tutorial 2 of a series on fitting models to data. We start with simple linear regression, using least squares optimization (Tutorial 1) and Maximum Likelihood Estimation (Tutorial 2). We will use bootstrapping to build confidence intervals around the inferred linear model parameters (Tutorial 3). We'll finish our exploration of regression models by generalizing to multiple linear regression and polynomial regression (Tutorial 4). We end by learning how to choose between these various models. We discuss the bias-variance trade-off (Tutorial 5) and Cross Validation for model selection (Tutorial 6).

In this tutorial, we will use a different approach to fit linear models that incorporates the random 'noise' in our data.
- Learn about probability distributions and probabilistic models
- Learn how to calculate the likelihood of our model parameters
- Learn how to implement the maximum likelihood estimator, to find the model parameter with the maximum likelihood

---
# Setup

```python
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
#@title Helper Functions
def plot_density_image(x, y, theta, sigma=1, ax=None):
  """ Plots probability distribution of y given x, theta, and sigma

  Args:

    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta (float): Slope parameter
    sigma (float): standard deviation of Gaussian noise

  """

  # plot the probability density of p(y|x,theta)
  if ax is None:
    fig, ax = plt.subplots()

  xmin, xmax = np.floor(np.min(x)), np.ceil(np.max(x))
  ymin, ymax = np.floor(np.min(y)), np.ceil(np.max(y))
  xx = np.linspace(xmin, xmax, 50)
  yy = np.linspace(ymin, ymax, 50)

  surface = np.zeros((len(yy), len(xx)))
  for i, x_i in enumerate(xx):
    surface[:, i] = stats.norm(theta * x_i, sigma).pdf(yy)

  ax.set(xlabel='x', ylabel='y')

  return ax.imshow(surface, origin='lower', aspect='auto', vmin=0, vmax=None,
            cmap=plt.get_cmap('Wistia'),
            extent=[xmin, xmax, ymin, ymax])

def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.

    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta_hat (float): An estimate of the slope parameter.

  Returns:
    float: The mean squared error of the data with the estimated parameter.
  """
  theta_hat = (x.T @ y) / (x.T @ x)
  return theta_hat
```

---
# Section 1: Maximum Likelihood Estimation (MLE)

```python
#@title Video 1: Maximum Likelihood Estimation
from IPython.display import YouTubeVideo
video = YouTubeVideo(id="8mpNmzLKNfU", width=854, height=480, fs=1)
print("Video available at https://youtube.com/watch?v=" + video.id)
video
```

    Video available at https://youtube.com/watch?v=8mpNmzLKNfU

In the previous tutorial we made the assumption that the data was drawn from a linear relationship with noise added, and found an effective approach for estimating model parameters based on minimizing the mean squared error.

In that case we treated the noise as simply a nuisance, but what if we factored it directly into our model?

Recall our linear model:

\begin{align}
y = \theta x + \epsilon.
\end{align}

The noise component $\epsilon$ is often modeled as a random variable drawn from a Gaussian distribution (also called the normal distribution).

The Gaussian distribution is described by its [probability density function](https://en.wikipedia.org/wiki/Probability_density_function) (pdf)
\begin{align}
\mathcal{N}(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}
\end{align}

and is dependent on two parameters: the mean $\mu$ and the variance $\sigma^2$. We often consider the noise signal to be Gaussian "white noise", with zero mean and unit variance:

\begin{align}
\epsilon \sim \mathcal{N}(0, 1).
\end{align}

## Interactive Demo: Gaussian Distribution Explorer

Use the explorer widget below to see how varying the $\mu$ and $\sigma$ parameters change the location and shape of the samples.

```python
#@title

#@markdown Make sure you execute this cell to enable the widget!

@widgets.interact(mu=widgets.FloatSlider(0.0, min=-2.0, max=2.0),
                  sigma=widgets.FloatSlider(1.0, min=0.5, max=2.0))
def plot_normal_dist(mu=0, sigma=1):

  # Generate pdf & samples from normal distribution with mu/sigma
  rv = stats.norm(mu, sigma)
  x = np.linspace(-5, 5, 100)
  y = rv.pdf(x)
  samples = rv.rvs(1000)

  # Plot
  fig, ax = plt.subplots()
  ax.hist(samples, 20, density=True, color='g', histtype='stepfilled', alpha=0.8,
          label='histogram')
  ax.plot(x, y, color='orange', linewidth=3, label='pdf')
  ax.vlines(mu, 0, rv.pdf(mu), color='y', linewidth=3, label='$\mu$')
  ax.vlines([mu-sigma, mu+sigma], 0, rv.pdf([mu-sigma, mu+sigma]), colors='red',
            color='b', linewidth=3, label='$\sigma$')
  ax.set(xlabel='x', ylabel='probability density', xlim=[-5, 5], ylim=[0, 1.0])
  ax.legend()
```

    interactive(children=(FloatSlider(value=0.0, description='mu', max=2.0, min=-2.0), FloatSlider(value=1.0, desc…

## Section 1.1: Probabilistic Models

Now that we have a model of our noise component $\epsilon$ as random variable, how do we incorporate this back into our original linear model from before? Consider again our simplified model $y = \theta x + \epsilon$ where the noise has zero mean and unit variance $\epsilon \sim \mathcal{N}(0, 1)$. We can now also treat $y$ as a random variable drawn from a Gaussian distribution where $\mu = \theta x$ and $\sigma^2 = 1$:

\begin{align}
y \sim \mathcal{N}(\theta x, 1)
\end{align}

which is to say that the probability of observing $y$ given $x$ and parameter $\theta$ is
\begin{align}
p(y|x,\theta) = \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}(y-\theta x)^2}
\end{align}

---

Let's revisit our original sample dataset where the true underlying model has $\theta = 1.2$.

```python
# @title

# @markdown Execute this cell to generate some simulated data

# setting a fixed seed to our random number generator ensures we will always
# get the same psuedorandom number sequence

np.random.seed(121)
theta = 1.2
n_samples = 30
x = 10 * np.random.rand(n_samples) # sample from a uniform distribution over [0,10)
noise = np.random.randn(n_samples) # sample from a standard normal distribution
y = theta * x + noise
```

This time we can plot the density of $p(y|x,\theta=1.2)$ and see how $p(y)$ changes for different values of $x$.

```python
#@title
#@markdown Execute this cell to visualize p(y|x, theta=1.2)

fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(10, 4))

# Invokes helper function to generate density image plots from data and parameters
im = plot_density_image(x, y, 1.2, ax=ax1)
plt.colorbar(im, ax=ax1)
ax1.axvline(8, color='k')
ax1.set(title=r'p(y | x, $\theta$=1.2)')

# Plot pdf for given x
ylim = ax1.get_ylim()
yy = np.linspace(ylim[0], ylim[1], 50)
ax2.plot(yy, stats.norm(theta * 8, 1).pdf(yy), color='orange', linewidth=2)
ax2.set(
    title=r'p(y|x=8, $\theta$=1.2)',
    xlabel='y',
    ylabel='probability density');
```

## Section 1.2: Likelihood Estimation

Now that we have our probabilistic model, we turn back to our original challenge of finding a good estimate for $\theta$ that fits our data. Given the inherent uncertainty when dealing in probabilities, we talk about the [likelihood](https://en.wikipedia.org/wiki/Likelihood_function) that some estimate $\hat \theta$ fits our data. The likelihood function $\mathcal{L(\theta)}$ is equal to the probability density function parameterized by that $\theta$:

\begin{align}
\mathcal{L}(\theta|x,y) = p(y|x,\theta) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{1}{2\sigma^2}(y-\theta x)^2}
\end{align}

### Exercise 1: Likelihood Function

In this exercise you will implement the likelihood function $\mathcal{L}(\theta|x,y)$ for our linear model where $\sigma = 1$.

After implementing this function, we can produce probabilities that our estimate $\hat{\theta}$ generated the provided observations. We will try with one of the samples from our dataset.

TIP: Use `np.exp` and `np.sqrt` for the exponential and square root functions, respectively.

```python
def likelihood(theta_hat, x, y):
  """The likelihood function for a linear model with noise sampled from a
    Gaussian distribution with zero mean and unit variance.

  Args:
    theta_hat (float): An estimate of the slope parameter.
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.

  Returns:
    ndarray: the likelihood values for the theta_hat estimate
  """
  sigma = 1
  ##############################################################################
  ## TODO for students: implement the likelihood function
  # Fill out function and remove
#   raise NotImplementedError("Student exercise: implement the likelihood function")
  ##############################################################################

  # Compute Gaussian likelihood
  pdf = 1 / np.sqrt(2 * np.pi * sigma**2) * np.exp(-(y - theta_hat * x)**2 / (2 * sigma**2))

  return pdf

# Uncomment below to test your function
print(likelihood(1.0, x[1], y[1]))
```

    0.11344443599846923

We should see that $\mathcal{L}(\theta=1.0|x=2.1,y=3.7) \approx 0.11$. So far so good, but how does this tell us how this estimate is better than any others?

When dealing with a set of data points, as we are with our dataset, we are concerned with their joint probability -- the likelihood that all data points are explained by our parameterization. Since we have assumed that the noise affects each output independently, we can factorize the likelihood, and write:

\begin{align}
\mathcal{L}(\theta|X,Y) = \prod_{i=1}^N \mathcal{L}(\theta|x_i,y_i),
\end{align}

where we have $N$ data points $X = \{x_1,...,x_N\}$ and $Y = \{y_1,...,y_N\}$.

In practice, such a product can be numerically unstable. Indeed multiplying small values together can lead to [underflow](https://en.wikipedia.org/wiki/Arithmetic_underflow), the situation in which the digital representation of floating point number reaches its limit. This problem can be circumvented by taking the logarithm of the likelihood because the logarithm transforms products into sums:

\begin{align}
\operatorname{log}\mathcal{L}(\theta|X,Y) = \sum_{i=1}^N \operatorname{log}\mathcal{L}(\theta|x_i,y_i)
\end{align}

We can take the sum of the log of the output of our `likelihood` method applied to the full dataset to get a better idea of how different $\hat\theta$ compare. We can also plot the different distribution densities over our dataset and see how they line up qualitatively.

```python
#@title
#@markdown Execute this cell to visualize different distribution densities
theta_hats = [0.5, 1.0, 2.2]
fig, axes = plt.subplots(ncols=3, figsize=(16, 4))
for theta_hat, ax in zip(theta_hats, axes):
  ll = np.sum(np.log(likelihood(theta_hat, x, y)))  # log likelihood
  im = plot_density_image(x, y, theta_hat, ax=ax)
  ax.scatter(x, y)
  ax.set(title=fr'$\hat{{\theta}}$ = {theta_hat}, log likelihood: {ll:.2f}')
plt.colorbar(im, ax=ax);
```

Using the log likelihood calculation, we see that $\mathcal{L}(\theta=1.0) > \mathcal{L}(\theta=0.5) > \mathcal{L}(\theta=2.2)$.

This is great: now we have a way to compare estimators based on likelihood. But like with the MSE approach, we want an analytic solution to find the best estimator. In this case, we want to find the estimator that maximizes the likelihood.

## Section 1.3: Finding the Maximum Likelihood Estimator

We want to find the parameter value $\hat\theta$ that makes our data set most likely:

\begin{align}
\hat{\theta}_{\textrm{MLE}} = \underset{\theta}{\operatorname{argmax}} \mathcal{L}(\theta|X,Y)
\end{align}

We discussed how taking the logarithm of the likelihood helps with numerical stability, the good thing is that it does so without changing the parameter value that maximizes the likelihood. Indeed, the $\textrm{log}()$ function is *monotonically increasing*, which means that it preserves the order of its inputs. So we have:

\begin{align}
\hat{\theta}_{\textrm{MLE}} = \underset{\theta}{\operatorname{argmax}} \sum_{i=1}^m \textrm{log} \mathcal{L}(\theta|x_i,y_i)
\end{align}

Now substituting our specific likelihood function and taking its logarithm, we get:
\begin{align}
\hat{\theta}_{\textrm{MLE}} = \underset{\theta}{\operatorname{argmax}} [-\frac{N}{2} \operatorname{log} 2\pi\sigma^2 - \frac{1}{2\sigma^2}\sum_{i=1}^N (y_i-\theta x_i)^2].
\end{align}

Note that maximizing the log likelihood is the same as minimizing the negative log likelihood (in practice optimization routines are developed to solve minimization not maximization problems). Because of the convexity of this objective function, we can take the derivative of our negative log likelihhood, set it to 0, and solve - just like our solution to minimizing MSE.

\begin{align}
\frac{\partial\operatorname{log}\mathcal{L}(\theta|x,y)}{\partial\theta}=\frac{1}{\sigma^2}\sum_{i=1}^N(y_i-\theta x_i)x_i = 0
\end{align}

This looks remarkably like the equation we had to solve for the optimal MSE estimator, and, in fact, we arrive to the exact same solution!

\begin{align}
\hat{\theta}_{\textrm{MLE}} = \hat{\theta}_{\textrm{MSE}} = \frac{\sum_{i=1}^N x_i y_i}{\sum_{i=1}^N x_i^2}
\end{align}

```python
# Compute theta_hat_MLE
theta_hat_mle = (x @ y) / (x @ x)

```

```python
#@title
#@markdown Execute this cell to visualize density with theta_hat_mle

# Plot the resulting distribution density
fig, ax = plt.subplots()
ll = np.sum(np.log(likelihood(theta_hat_mle, x, y))) # log likelihood
im = plot_density_image(x, y, theta_hat_mle, ax=ax)
plt.colorbar(im, ax=ax);
ax.scatter(x, y)
ax.set(title=fr'$\hat{{\theta}}$ = {theta_hat_mle:.2f}, log likelihood: {ll:.2f}');
```

---
# Summary

- Likelihood vs probability
    - $\mathcal{L}(\theta|x, y) = p(y|\theta, x)$
    - $p(y|\theta, x)$ -> "probability of observing the response $y$ given parameter $\theta$ and input $x$"
    - $\mathcal{L}(\theta|x, y)$ -> "likelihood model that parameters $\theta$ produced response $y$ from input $x$"
- Log-likelihood maximization
    - We take the $\textrm{log}$ of the likelihood function for computational convenience
    - The parameters $\theta$ that maximize $\textrm{log}\mathcal{L}(\theta|x, y)$ are the model parameters that maximize the probability of observing the data.
- **Key point**:
  - the log-likelihood is a flexible cost function, and is often used to find model parameters that best fit the data.

---
# Appendix

We can also see $\mathrm{p}(\mathrm{y} | \mathrm{x}, \theta)$ as a function of $x$. This is the stimulus likelihood function, and it is useful in case we want to decode the input $x$ from observed responses $y$. This is what is relevant from the point of view of a neuron that does not have access to the outside world and tries to infer what's out there from the responses of other neurons!


# Digit Recognition Task
If this is your first time running a notebook - welcome!! Notebooks are awesome because they let us play around and experiment
with code with near-instant feedback. Some pointers:
1. To execute a cell, click on it and hit SHIFT-Enter
2. Once something is executed, the variables are in memory - inspect them!

## Getting Started
This first cell imports the necessary libraries so we can get started:

```python
import torch
import numpy as np
from PIL import Image
import torch.nn as nn
import torch.onnx as onnx
import torch.nn.functional as F
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
```

 ## Data!
 Without data we really can't do anythin with machine learning. At this point we have our sharp question **can we predict a digit given a 28x28 vector of numbers?**

 Once that is all squared away we need to take a look at our data. The next cell is a helper function that visualizes the the digits (a sanity check).

```python
def draw_digits(digits):
    fig, axes = plt.subplots(6, 20, figsize=(18, 7),
                            subplot_kw={'xticks':[], 'yticks':[]},
                            gridspec_kw=dict(hspace=0.1, wspace=0.1))
    for i, ax in enumerate(axes.flat):
        X, y = digits[i]
        ax.imshow(255 - X.reshape(28,28) * 255, cmap='gray')
        ax.set_title('{:.0f}'.format(torch.argmax(y).item()))
```

The next cell downloads the standard digit dataset (called MNIST). The `transform` and `target_transform` parts of this call add some conversion steps to make the data more suitable for the models we will try.

```python
digits = datasets.MNIST('data', train=True, download=True,
                        transform=transforms.Compose([
                            transforms.ToTensor(),
                            transforms.Lambda(lambda x: x.reshape(28*28))
                        ]),
                        target_transform=transforms.Compose([
                            transforms.Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, y, 1))
                        ])
                     )
```

Here is our sanity check!

```python
draw_digits(digits)
```

Feel free to take a look at the data by printing out `x` and/or `y`. They really are just numbers! A couple of things that ight seem strange when you print them out:
1. "I thought you said the images were 784 sized vectors of 0-255???" - They were! We just normalized the vectors by dividing by 255 so the new range is 0-1 (it makes the numbers more tidy)
2. "I thought the `y` was a numerical answer?? Instead it's a 10 sized vector!" - Yes - this is called a one-hot encoding of the answer. Now there's a `1` in the index of the right answer. Again, this makes the math work a lot better for the models we will be creating.

```python
x, y = digits[0]
print(x)
print(y)
```

    tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0118,
            0.0706, 0.0706, 0.0706, 0.4941, 0.5333, 0.6863, 0.1020, 0.6510, 1.0000,
            0.9686, 0.4980, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1176, 0.1412, 0.3686, 0.6039,
            0.6667, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.8824, 0.6745, 0.9922,
            0.9490, 0.7647, 0.2510, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1922, 0.9333, 0.9922, 0.9922,
            0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9843, 0.3647, 0.3216,
            0.3216, 0.2196, 0.1529, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.8588, 0.9922,
            0.9922, 0.9922, 0.9922, 0.9922, 0.7765, 0.7137, 0.9686, 0.9451, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3137,
            0.6118, 0.4196, 0.9922, 0.9922, 0.8039, 0.0431, 0.0000, 0.1686, 0.6039,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0549, 0.0039, 0.6039, 0.9922, 0.3529, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.5451, 0.9922, 0.7451, 0.0078, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0431, 0.7451, 0.9922, 0.2745,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1373, 0.9451,
            0.8824, 0.6275, 0.4235, 0.0039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.3176, 0.9412, 0.9922, 0.9922, 0.4667, 0.0980, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.1765, 0.7294, 0.9922, 0.9922, 0.5882, 0.1059, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0627, 0.3647, 0.9882, 0.9922, 0.7333,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.9765, 0.9922,
            0.9765, 0.2510, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1804, 0.5098, 0.7176, 0.9922,
            0.9922, 0.8118, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.1529, 0.5804, 0.8980, 0.9922, 0.9922,
            0.9922, 0.9804, 0.7137, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0941, 0.4471, 0.8667, 0.9922, 0.9922, 0.9922,
            0.9922, 0.7882, 0.3059, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0902, 0.2588, 0.8353, 0.9922, 0.9922, 0.9922, 0.9922,
            0.7765, 0.3176, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0706, 0.6706, 0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.7647,
            0.3137, 0.0353, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.2157, 0.6745, 0.8863, 0.9922, 0.9922, 0.9922, 0.9922, 0.9569, 0.5216,
            0.0431, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.5333, 0.9922, 0.9922, 0.9922, 0.8314, 0.5294, 0.5176, 0.0627,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,
            0.0000])
    tensor([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.])

# Choosing Models
Now that we have some data it's time to start picking models we think might work. This is where the science part of data-science comes in: we guess and then check if our assumptions were right. Imagine models like water pipes that have to distribute water to 10 different hoses depending on 784 knobs. These 784 knobs represent the individual pixels in the digit and the 10 hoses at the end represent the actual number (or at least the index of the one with the most water coming out of it). Our job now is to pick the plumbing in between.

The next three cells represent three different constructions in an increasingly more complex order:

1. The first is a simple linear model,
2. The second is a 3 layer Neural Network,
3. and the last is a full convolutional neural network

While it is out of the scope of this tutorial to fully explain how they work, just imagine they are basically plumbing with internal knobs that have to be tuned to produce the right water pressure at the end to push the most water out of the right
index. As you go down each cell the plumbing and corresponding internal knobs just get more complicated.

```python
class SimpleLinear(nn.Module):
    def __init__(self):
        super(SimpleLinear, self).__init__()
        self.layer1 = nn.Linear(28*28, 10)

    def forward(self, x):
        x = self.layer1(x)
        return F.softmax(x, dim=1)
```

```python
class NeuralNework(nn.Module):
    def __init__(self):
        super(NeuralNework, self).__init__()
        self.layer1 = nn.Linear(28*28, 512)
        self.layer2 = nn.Linear(512, 512)
        self.output = nn.Linear(512, 10)

    def forward(self, x):
        x = F.relu(self.layer1(x))
        x = F.relu(self.layer2(x))
        x = self.output(x)
        return F.softmax(x, dim=1)
```

```python
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = x.view(-1, 1, 28, 28)
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.softmax(x, dim=1)
```

# Optimizing Model Parameters
Now that we have some models it's time to optimize the internal parameters to see if it can do a good job at recognizing digits! It turns out there are some parameters that we can give the optimization algorithm to tune how it trains - these are called hyper-parameters. That's what the two variables represent below:

```python
learning_rate = 1e-3
batch_size = 64
epochs = 5
```

The `learning_rate` basically specifies how fast the algorithm will learn the model parameters. Right now you're probably thinking "let's set it to fifty million #amirite?" The best analogy for why this is a bad idea is golf. I'm a terrible golfist (is that right?) so I don't really know anything - but pretend you are trying to sink a shot (again sorry) but can only hit the ball the same distance every time. Easy right? Hit it the exact length from where you are to the hole! Done! Now pretend you don't know where the hole is but just know the general direction. Now the distance you choose actually matters. If it is too long a distance you'll miss the hole, and then when you hit it back you'll overshoot again. If the distance is too small then it will take forever to get there but for sure you'll eventually get it in. Basically you have to guess what the right distance per shot should be and then try it out. That is basically what the learning rate does for finding the "hole in one" for the right parameters (ok, I'm done with the golf stuff).

Below there are three things that make this all work:
1. **The Model** - this is the function we're making that takes in the digit vector and should return the right number
2. **The Cost Function** (sometimes called the loss function). I know I promised I was done with golf but I lied. Remember how I said in our screwy golf game you knew the general direction of the hole? The cost function tells us the distance to the hole - when it's zero we're there! In actual scientific terms, the cost function tells us how bad the model is at getting the right answer. As we take shots you should see the cost function decreasing. If this does not happen then something is wrong. At this point I would change the shot distance (or `learning_rate`) to something smaller and try again. If that doesn't work maybe change the model!
3. **The Optimizer** - this part is the bit that actually changes the model parameters. It has a sense for the direction we should be shooting and updates all of the internal numbers inside the model to find the best internal knobs to predict the right digits. In this case I am using the Binary Cross Entropy cost function because, well, I know it works. There are a ton of different cost functions you can choose from that fit a variety of different scenarios.

```python
# where to run
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print('Using {} device'.format(device))

# selected model
model = SimpleLinear().to(device)
#model = NeuralNework().to(device)
#model = CNN().to(device)
print(model)

# cost function used to determine best parameters
cost = torch.nn.BCELoss()

# used to create optimal parameters
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
```

    Using cuda device
    SimpleLinear(
      (layer1): Linear(in_features=784, out_features=10, bias=True)
    )

I forgot to mention the whole `cuda` thing. GPUs speed this whole process up because this is basically all a big Matrix multiplcation problem in a `for` loop. PyTorch is great because you basically need to tell the model where to run (either on the CPU or using CUDA - which is a platform for moving computations to the GPU).

Now for the learning part! The `dataloader`'s job is to iterate through the entire dataset (in this case 60,000 examples of digits and their corresponding label) but to take chunks of size `batch_size` of the data to process. This is another hyperparameter that needs to be chose. `epochs` is the number of times we want to loop through the dataset in its entirety (again something we choose based upon how our experiment goes).

A word on how long this takes - it takes a while.

```python
# loader
dataloader = DataLoader(digits, batch_size=batch_size, num_workers=0, pin_memory=True)

# golfing time!
for t in range(epochs):
    print('Epoch {}'.format(t))
    print('-------------------------------')
    # go through the entire dataset once
    for batch, (X, Y) in enumerate(dataloader):
        X, Y = X.to(device), Y.to(device)
        # zero out gradient
        optimizer.zero_grad()
        # make a prediction on this batch!
        pred = model(X)
        # how bad is it?
        loss = cost(pred, Y)
        # compute gradients
        loss.backward()
        # update parameters
        optimizer.step()

        if batch % 100 == 0:
            print('loss: {:>10f}  [{:>5d}/{:>5d}]'.format(loss.item(), batch * len(X), len(dataloader.dataset)))

    print('loss: {:>10f}  [{:>5d}/{:>5d}]\n'.format(loss.item(), len(dataloader.dataset), len(dataloader.dataset)))

print('Done!')
```

    Epoch 0
    -------------------------------
    loss:   0.324714  [    0/60000]
    loss:   0.120360  [ 6400/60000]
    loss:   0.094450  [12800/60000]
    loss:   0.082783  [19200/60000]
    loss:   0.063817  [25600/60000]
    loss:   0.072431  [32000/60000]
    loss:   0.051376  [38400/60000]
    loss:   0.082728  [44800/60000]
    loss:   0.066302  [51200/60000]
    loss:   0.073625  [57600/60000]
    loss:   0.028484  [60000/60000]

    Epoch 1
    -------------------------------
    loss:   0.052062  [    0/60000]
    loss:   0.052166  [ 6400/60000]
    loss:   0.045498  [12800/60000]
    loss:   0.063747  [19200/60000]
    loss:   0.042184  [25600/60000]
    loss:   0.063963  [32000/60000]
    loss:   0.036549  [38400/60000]
    loss:   0.073531  [44800/60000]
    loss:   0.055800  [51200/60000]
    loss:   0.073686  [57600/60000]
    loss:   0.020631  [60000/60000]

    Epoch 2
    -------------------------------
    loss:   0.038994  [    0/60000]
    loss:   0.047847  [ 6400/60000]
    loss:   0.036883  [12800/60000]
    loss:   0.056841  [19200/60000]
    loss:   0.038674  [25600/60000]
    loss:   0.061276  [32000/60000]
    loss:   0.032280  [38400/60000]
    loss:   0.070644  [44800/60000]
    loss:   0.052389  [51200/60000]
    loss:   0.074087  [57600/60000]
    loss:   0.017948  [60000/60000]

    Epoch 3
    -------------------------------
    loss:   0.032573  [    0/60000]
    loss:   0.046425  [ 6400/60000]
    loss:   0.033100  [12800/60000]
    loss:   0.052219  [19200/60000]
    loss:   0.038146  [25600/60000]
    loss:   0.059625  [32000/60000]
    loss:   0.030246  [38400/60000]
    loss:   0.069072  [44800/60000]
    loss:   0.050445  [51200/60000]
    loss:   0.074007  [57600/60000]
    loss:   0.016581  [60000/60000]

    Epoch 4
    -------------------------------
    loss:   0.029027  [    0/60000]
    loss:   0.045549  [ 6400/60000]
    loss:   0.031056  [12800/60000]
    loss:   0.049083  [19200/60000]
    loss:   0.038555  [25600/60000]
    loss:   0.058437  [32000/60000]
    loss:   0.029133  [38400/60000]
    loss:   0.068016  [44800/60000]
    loss:   0.049105  [51200/60000]
    loss:   0.073704  [57600/60000]
    loss:   0.015751  [60000/60000]

    Done!

So what the heck did this actually do? Great question. I will explain this `SimpleLinear` model since its the on we ran first. We are basically learning one matrix and one vector. That's it (it's really anti-climacttic if you ask me). Let me show you why that actually does anything:

\begin{align}
prediction = W \cdot x + b
\end{align}

\begin{align}
\begin{bmatrix}
\hat{y}_1 \\ \vdots \\ \hat{y}_{10}
\end{bmatrix}_{10 \times 1} =
\begin{bmatrix}
w_{1,1} &  \ldots &  w_{1,784} \\
\vdots    & \ddots &  \\
w_{10,1}  & \ldots & w_{10, 784} \\
\end{bmatrix}_{10 \times 784}
\cdot
\begin{bmatrix}x_1 \\ x_2 \\ \vdots \\ x_{783} \\ x_{784}\end{bmatrix}_{784 \times 1}
 +
\begin{bmatrix}
b_1 \\ \vdots \\ b_{10}
\end{bmatrix}_{10 \times 1}
\end{align}

The matrix `W`and the vector `b` is what the `SimpleLinear` model learns. The output is a 10 by 1 matrix whose largest value is the index of the thing number we want to predict. Take a look at what the algorithm learned for the two variables (as well as the sizes):

```python
for p in model.parameters():
    print(p.shape)
    print(p)
```

    torch.Size([10, 784])
    Parameter containing:
    tensor([[ 0.0272,  0.0349, -0.0094,  ..., -0.0113,  0.0137,  0.0301],
            [-0.0065, -0.0143, -0.0217,  ..., -0.0345, -0.0186, -0.0323],
            [-0.0179,  0.0171, -0.0058,  ..., -0.0124, -0.0091,  0.0289],
            ...,
            [ 0.0106, -0.0054, -0.0265,  ..., -0.0170, -0.0061,  0.0227],
            [ 0.0180, -0.0323,  0.0236,  ..., -0.0254,  0.0279,  0.0049],
            [ 0.0107, -0.0218,  0.0175,  ...,  0.0058, -0.0234, -0.0088]],
           device='cuda:0', requires_grad=True)
    torch.Size([10])
    Parameter containing:
    tensor([-0.2599,  0.4166,  0.0458, -0.2472,  0.1208,  0.6905, -0.0332,  0.3545,
            -0.7617, -0.1192], device='cuda:0', requires_grad=True)

# Is it working???
The best way to figure out if it is working is to test the model on data the learning process hasn't used. Luckily we have such a dataset (it is basically a held out section of the data we already have used). I'm loading it all up the same way as befor and printing them out to show you that they're different.

```python
test_digits = datasets.MNIST('data', train=False, download=True,
                        transform=transforms.Compose([
                            transforms.ToTensor(),
                            transforms.Lambda(lambda x: x.reshape(28*28))
                        ]),
                        target_transform=transforms.Compose([
                            transforms.Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(0, y, 1))
                        ])
                     )
draw_digits(test_digits)
```

Let's test it on the first digit above (the 7).

```python
x, y = test_digits[0]
x = x.to(device).view(1, 28*28)
```

We'll tell the model we are using it for evaluation (sometimes called inference) and pass in our previously unseen digit.

```python
model.eval()
with torch.no_grad():
    pred = model(x)
    pred = pred.to('cpu').detach()[0]

print(pred)
```

    tensor([9.6284e-06, 1.5173e-10, 1.4951e-05, 1.7654e-03, 4.8513e-07, 1.5701e-05,
            2.4185e-09, 9.9783e-01, 1.8910e-05, 3.4010e-04])

Now let's see if the predicted digit matches the actual digit:

```python
pred.argmax(0), y.argmax(0)
```

    (tensor(7), tensor(7))

Let's see how well we do over *all* of the test data!

```python
# data loader for test digits
test_dataloader = DataLoader(test_digits, batch_size=64, num_workers=0, pin_memory=True)

# set model to evaluation mode
model.eval()
test_loss = 0
correct = 0

# loop!
with torch.no_grad():
    for batch, (X, Y) in enumerate(dataloader):
        X, Y = X.to(device), Y.to(device)
        pred = model(X)

        test_loss += cost(pred, Y).item()
        correct += (pred.argmax(1) == Y.argmax(1)).type(torch.float).sum().item()

test_loss /= len(dataloader.dataset)
correct /= len(dataloader.dataset)
print('\nTest Error:')
print('acc: {:>0.1f}%, avg loss: {:>8f}'.format(100*correct, test_loss))
```

    Test Error:
    acc: 92.6%, avg loss: 0.000700

# Saving the Model
Every framework is different - in this case PyTorch let's us save the model (which you remember is just a big matrix `W` and a vector `b`) to an internal format as well as to the ONNX format. These can then be loaded up as an asset to a program that is executed every time you need to recognize a digit!

```python
# create dummy variable to traverse graph
x = torch.randint(255, (1, 28*28), dtype=torch.float).to(device) / 255
onnx.export(model, x, 'model.onnx')
print('Saved onnx model to model.onnx')

# saving PyTorch Model Dictionary
torch.save(model.state_dict(), 'model.pth')
print('Saved PyTorch Model to model.pth')
```

    Saved onnx model to model.onnx
    Saved PyTorch Model to model.pth

# Play!
Now that you've gone through the whole process, please go back up to play around! Try changing:
1. The actual model! The other models are almost identical with the exception that they learn additional Matrices (W's and b's) that the images pass through to get the final answer.
2. The hyperparameters like `learning_rate`, `batch_size`, and `epoch`. Does it make things better or worse? 92% is ok, does any other combination of model's and hyperparamters fare better?

## Final Thoughts
Would love your feedback! Was this helpful? Any parts confusing? Drop me a line!

```python

```