# The Model Project (BCG group)
##  OLG Model

This Model Project was created in the concept of a second take home assignment for the course Introduction to Programming and Numerical Analysis. The goal of the project is for the students to become familiar with the implementation of an economic model into python (involving the symbolic and numerical solution of the interested model).

It is structured as follows:

1. Setup
2. Symbolic solution of the Household's and Firm's problems
3. Symbolic and numerical solution for the aggregate level of the model
4. Implementation of the Bisection method at the steady state level of the model

# 1. Setup

```python
#importing the necessary packages

import numpy as np
import sympy as sm

import matplotlib.pyplot as plt
import ipywidgets as widgets
```

**General assumptions:**
1. Closed economy with no government and no unemployment
2. One final good is consumed or invested to produce additional output
3. All markets are competitive
4. Firms and households have perfect foresight about all economic variables

# 2. Symbolic solution of the Household's (i) and Firm's (ii) problems

**(i) Household's problem**

It tries to maximize its utility, given by the function:

\\[ \max_{c_{1,t},c_{2,t+1}}U = ln{(c_{1,t})} + \beta ln{(c_{2,t+1})}\\]

\\[ \text{ s.t. } \\]

\\[z_{t+1} = w_t - c_{1,t} \\]

\\[c_{2,t+1} = z_{t+1}(1 + r_{t+1})\\]

where:

1. $c_{1,t}$ consumption at period $t$
* $c_{2,t+1}$ consumption at period $t+1$
* $w_t$ wage
* $r_{t+1}$ interest rate
* $\beta$ is the subjective discount factor, 0 < $\beta$ < 1
* $z_{t+1}$ is the financial assets held at the beginning of period $t+1$ by the households born at time $t$

**Note:** It is assumed that each household borns and dies with no assets $\alpha_{t}$ = 0, $\alpha_{t+2}$ = 0

```python
# Activating pretty printing
sm.init_printing(use_unicode=True)
```

```python
# Defining symbolically the necessary variables
c1 = sm.symbols('c_1_t')
c2 = sm.symbols('c_2_t+1')
beta = sm.symbols('beta')
z = sm.symbols('z_t+1')
w = sm.symbols('w_t') # wage
r1 = sm.symbols('r_t+1')
u = sm.symbols('U_t') #Utility
L = sm.symbols('L') #Langrange
omega = sm.symbols('omega') #The Langrangian multiplier

# Checking that our variables have the proper format
c1,c2, beta, w, z, r1, L, omega
```

```python
# Defining the utility function
def utility(c1, c2, beta):
    return (sm.log(c1) + beta*sm.log(c2))
```

Calculating the **Lifetime Budget Constraint** that will be used at the Langranian

```python
#Defining the two budget constraints symbolically
budget_constraint_1 = sm.Eq(z, w - c1)
budget_constraint_2 = sm.Eq(c2, z*(1+r1))
budget_constraint_1, budget_constraint_2
```

Solve in **three steps**:

1. **Solve** for $z_{t+1}$ from the budget constraint 1
2. **Substitute** in budget constraint 2
3. **Solve** for $w_t$

**Step 1: Solve** for $z_{t+1}$ from the budget constraint 1

```python
z_from_constraint = sm.solve(budget_constraint_1, z)
z_from_constraint
```

**Step 2: Substitute** in budget constraint 2

```python
z_subs_constraint = budget_constraint_2.subs(z, z_from_constraint[0])
z_subs_constraint
```

**Step 3: Solve** for $w_t$

```python
lifetime_budget_constraint = sm.solve(z_subs_constraint, w)
lifetime_budget_constraint
```

So, the households try to maximize their utility under the **Lifetime Budget Constraint**

\\[ \max_{c_{1,t},c_{2,t+1}}U = ln{(c_{1,t})} + \beta ln{(c_{2,t+1})}\\]

\\[ \text{ s.t. } \\]

\\[c_{1,t} + \frac {c_{2,t+1}}{(1 + r_{t+1})} = w_{t}\\]

```python
# Defining the Lifetime Budget Constraint symbolically
budget_constraint = w - c1 - c2/(1+r1)
budget_constraint
```

```python
#Constructing the Langrangian
Langrangian = utility(c1, c2, beta) + omega * budget_constraint
Langrangian
```

Solve in **six steps**:

1. **Calculate** the two FOCs wrt. $c_{1,t}$ and $c_{2,t+1}$
2. **Isolate** the first FOC for $\omega$
3. **Substitute** in the second FOC
4. **Isolate** the second FOC for $c_{1,t}$
5. **Substitute** into the Lifetime Budget Constraint
6. **Solve** for $c_{2,t+1}$ to find the consumption in period 2
7. **Substitute** $c_{2,t+1}$ in Lifetime Budget Constraint
8. **Solve** for $c_{1,t}$ to find the consumption in period 1
9. **Substitute** in the first budget constraint ($z_{t+1} = w_t - c_{1,t}$)
10. **Solve** for $z_{t+1}$ to find the assets held by the household at period 2

**Step 1: Calculate** the two FOCs wrt. $c_{1,t}$ and $c_{2,t+1}$

```python
foc1 = sm.diff(Langrangian, c1)
foc2 = sm.diff(Langrangian, c2)
foc1, foc2
```

**Step 2: Isolate** the first FOC for $\omega$

```python
foc_1 = sm.solve(foc1, omega)
foc_1
```

**Step 3: Substitute** in the second FOC

```python
foc_2 = foc2.subs(omega, foc_1[0])
foc_2
```

**Step 4: Isolate** the second FOC for $c_{1,t}$

```python
AB = sm.solve(foc_2, c1)
AB
```

**Step 5: Substitute** into the Lifetime Budget Constraint

```python
ABC = budget_constraint.subs(c1, AB[0])
ABC
```

**Step 6: Solve** for $c_{2,t+1}$ to find the consumption in period 2

```python
consumption_2 = sm.solve(ABC,c2)
consumption_2
```

**Step 7: Substitute** $c_{2,t+1}$ in Lifetime Budget Constraint

```python
ABCD = budget_constraint.subs(c2, consumption_2[0])
ABCD
```

**Step 8: Solve** for $c_{1,t}$ to find the consumption in period 1

```python
consumption_1 = sm.solve(ABCD, c1)
consumption_1
```

**Step 9: Substitute** in the first budget constraint ($z_{t+1} = w_t - c_{1,t}$)

```python
ABCDE = budget_constraint_1.subs(c1, consumption_1[0])
ABCDE
```

**Step 10: Solve** for $z_{t+1}$ to find the assets held by the household at period 2

```python
Assets = sm.solve(ABCDE, z)
Assets
```

**Conclusion:** After solving the Household's problem we found that:

\\[c_{1,t} = \frac {w_t}{1+\beta}\\],
\\[c_{2,t+1} = (1+r_{t+1}) \frac {1}{1+(\frac{1}{\beta})} w_{t}\\]
\\[z_{t+1} = \frac {1}{1+(\frac{1}{\beta})} w_{t}\\]

**(ii) Firm's Problem**

The **production function** is a **Cobb-Douglas** with the format of:

\\[ Y_{t} = K_{t}^{\alpha}(A_{t}L_{t})^{1-\alpha}\\]

where:

1. $Y_t = F(K_t,A_tL_t)$ is the produced output

* $K_t$ is the capital

* $L_t$ is the labor force, growing exogenously at rate $n$,\\[L_{t+1} = {(1-n)} L_{t}\\]

* $A_t$ is the index of technological progress, growing exogenously at rate of $g$,\\[A_{t+1} = {(1-g)} A_{t}\\]

* $a$ is the share of physical capital and 0 < $a$ < 1

**Note 1:** It is assumed that capital ($K_{t}$) depreciates at rate $\delta$ per period, too.

**Note 2:** It is more convenient to write $Y_{t}$ and $K_{t}$ per effective worker, so \\[y_{t} = \frac {Y_{t}}{A_{t}L_{t}}\\] and \\[k_{t} = \frac {K_{t}}{A_{t}L_{t}}\\]

**Note 3:** The firm is trying to maximize its profits under perfect competition and no-adjustment costs. So, in the equillibrium firms want to equalise both **Marginal Products $(MPK_t, MPL_t)$** with their rental rates **($r_{t} + \delta$, $w_{t}$)**, respectively.

**FOC's** of the production fuction

\\[ MPK_{t} : {\alpha}k_{t}^{\alpha-1} = r_{t}+{\delta}\\]

\\[ MPL_{t} : {1-\alpha}A_{t}k_{t}^{\alpha} = w_{t}\\]

```python
# Defining symbolically the necessary variables
alpha = sm.symbols('alpha') # share of physical capital
k = sm.symbols ('k_t')
r = sm.symbols('r_t') # interest rate
delta = sm.symbols('delta') # depreciation rate of capital per period
A = sm.symbols('A_t') # technological index
A1 = sm.symbols('A_{t+1}')
g = sm.symbols ('g') #rate of technological growth
L0 = sm.symbols('L_{t}')
L1 = sm.symbols('L_{t+1}')
n = sm.symbols('n') #growth of labor force

# Checking that our variables have the proper format
alpha, k, r, delta, A, A1, g, L0, L1, n
```

First we define the equations for the **technological** and the **labor's force** growth and solve them wrt to period t for later use:

```python
tech = sm.Eq((1+g)*A, A1)
labor = sm.Eq((1+n)*L0, L1)
tech, labor
```

```python
tech0 = sm.solve(tech, A)
labor0 = sm.solve(labor, L0)
tech0, labor0
```

Then we define the two FOCs from the firms maximization and equalize them with their respective rental rates

```python
# Defining the two FOC's and setting them equal with their rental rates
MPK = sm.Eq((alpha*k**(alpha-1)), r+delta)
MPL = sm.Eq (((1-alpha)*A*k**alpha), w)
MPK, MPL
```

**Note:** We will solve the MPL wrt. to w, for later use

```python
Wage = sm.solve(MPL, w)
Wage
```

**Conclusion:** From the firm's maximization problem we calculated the wage:

\\[ w_{t} = A_{t} (\frac{\delta + r_{t}}{\alpha})^\frac{1}{\alpha -1} {(1-\alpha)}\\]

# 3. Symbolic (iii) and analytical (iv) solution for the aggregate level of the model

**(iii) Symbolic solution for the aggregate level of the model**

**Note:** the crucial aggregation is the one of physical capital , which we do it at time t+1 just for convience.

```python
# Defining symbolically the necessary variables
Ks = sm.symbols('K_{t+1}^{s}') #aggregate supply of physical capital at period t+1
Kd = sm.symbols('K_{t+1}^{s}')#aggregate demand of physical capital at period t+1
K = sm.symbols('K_{t+1}')
g1 = sm.symbols('g^{*}')
k1 = sm.symbols('k_{t+1}')

# Checking that our variables have the proper format
Ks, Kd, K, g1, k1
```

Solve in **seven steps**:

1. **Define** the capital supply $K_{t+1}^{s}$
2. **Substitute** for $z_{t+1}$ as calculated in the Household's problem
3. **Substitute** for $w_{t}$ as calculated in the Firm's problem
4. **Substitute** $A_{t}$ with $A_{t+1}$
5. **Substitute** $L_{t}$ with $L_{t+1}$
6. **Substitute** $(1+n)(1+g)$ with $g^{*}$
7. **Equalize** $K_{t+1}^{s} = K_{t+1}^{d} = K_{t+1}$

**Step 1: Define** the capital supply $K_{t+1}^{s}$

```python
capital_supply = sm.Eq(L0*z ,Ks)
capital_supply
```

**Step 2: Substitute** for $z_{t+1}$ as calculated in the Household's problem

```python
H = capital_supply.subs(z, Assets[0])
H
```

**Note:** the wage ($w_{t}$) must be compatible with the demand by firms at period t. So,

\\[ K_{t+1}^{s} = K_{t+1}^{d} = K_{t+1}\\]

**Step 3: Substitute** for $w_{t}$ as calculated in the Firm's problem

```python
I = H.subs(w, Wage[0])
I
```

**Step 4: Substitute** $A_{t}$ with $A_{t+1}$

```python
J = I.subs(A, tech0[0])
J
```

**Step 5: Substitute** $L_{t}$ with $L_{t+1}$

```python
M = J.subs(L0, labor0[0])
M
```

**Step 6: Substitute** $(1+n)(1+g)$ with $(1+g^{*})$

```python
N = M.subs(((1+g)*(1+n)), (g1+1))
N
```

**Step 7: Equalize** $K_{t+1}^{s} = K_{t+1}^{d} = K_{t+1}$

```python
P = N.subs(Ks, K)
P
```

If we divide both sides of the equation with $A_{t+1}L_{t+1}$ then we will obtain the **Key Transition Equation:**

\\[ k_{t+1} = \frac {(1-\alpha)}{(1+g^{*})(1+ \frac{1}{\beta})} k_{t}^\alpha\\]

where

\\[ (1+g^{*}) = (1+g)(1+n)\\]

and $g^{*}$ denotes the per period growth rate of effective workers (AL)

```python
sm.init_printing(use_unicode=False)
```

**(iv) Analytical solution for the aggregate level of the model**

```python
g1 = lambda n, g: (1+g)*(1+n)
solution = lambda k, alpha = 0.4 , g = 0.01, n = 0.06, beta = 0.8: (1-alpha)*k**alpha / ((1+g1(n,g))*(1+1/beta))
y = lambda x: x # this will be used to plot a 45o degree line at the first plot
```

```python
k_values = np.linspace(0, 0.04, 10000)
x = np.linspace(0, 0.04, 10000)
print(solution(k_values))
```

    [0.         0.00089272 0.00117795 ... 0.03553545 0.03553687 0.03553829]

```python
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

ax.plot(k_values, solution(k_values))
ax.plot(x, y(x))

plt.xlabel('$k_{t}$')
plt.ylabel('$k_{t+1}$')
plt.title('Key Transition Equation')

plt.show()
```

**Conclusion:** From the plot we can conclude that the evolution of capital per effective worker is given by a concave function. The intersection of the Key Transition Equation and the 45o degree line presents the value of the steady state ($k_{t+1} = k_{t} = k^{*}$), which in our case (with the given values to the parameters) is approximately $k^{*}$ = 0.03 .

# 4. Implementation of the Bisection method at the steady state level of the model (numerical solution)

The **Key Transition Equation** describes the evolution of the capital per effective worker over two consecutive periods. If we are interested in the analysis of steady states, then we can guess that capital per effective worker must be constant, since the production is identical with the one in the neoclassical growth model. By setting $k_{t+1} = k_{t} = k^{*}$ and substituting into the Key Transition Equation, it becomes:

\\[ k^{*} = \frac{(1-\alpha)}{(1+g^{*})(1+ \frac{1}{\beta})}^\frac{1}{1-\alpha}\\]

First the above mentioned equation is defined as a python function, after bringing all the elements to one side. Then insert it in the Bisection method and print the result (number of iterations needed, the numerical solution and the value of the Key Transition Equation with the numerical solution which approximately close to zero); finally an interactive plot, that depicts the range in which Bisection method searches for the numerical solution of the model, and the conclusion of the implementation of that method are presented.

```python
f = lambda k, alpha=0.4, n=0.01, g=0.06, beta=0.8: k - ((1-alpha)/((1+g1(n,g))*(1+1/beta)))** (1/(1-alpha))
```

```python
def bisection(f,a,b,max_iter=500,tol=1e-6,full_info=False):
    """ bisection

    Args:

        f (function): function
        a (float): left bound
        b (float): right bound
        tol (float): tolerance on solution

    Returns:

        m (float): root

    """

    # test inputs
    if f(a)*f(b) >= 0:
        print("bisection method fails.")
        return None

    # step 1: initialize
    _a = a
    _b = b
    a = np.zeros(max_iter)
    b = np.zeros(max_iter)
    m = np.zeros(max_iter)
    fm = np.zeros(max_iter)
    a[0] = _a
    b[0] = _b

    # step 2-4: main
    i = 0
    while i < max_iter:

        # step 2: midpoint and associated value
        m[i] = (a[i]+b[i])/2
        fm[i] = f(m[i])

        # step 3: determine sub-interval
        if abs(fm[i]) < tol:
            break
        elif f(a[i])*fm[i] < 0:
            a[i+1] = a[i]
            b[i+1] = m[i]
        elif f(b[i])*fm[i] < 0:
            a[i+1] = m[i]
            b[i+1] = b[i]
        else:
            print("bisection method fails.")
            return None

        i += 1

    if full_info:
        return m,i,a,b,fm
    else:
        return m[i],i
```

```python
m,i = bisection(f,-10,10)
print(i,m,f(m))
```

    18 0.03284454345703125 6.825429116411041e-07

```python
def plot_bisection(f, a, b, xmin=-10, xmax=10, xn=100):

    # a. find root and return all information
    m, max_iter, a, b, fm = bisection(f, a, b, full_info=True)

    # b. compute function on grid
    xvec = np.linspace(xmin, xmax, xn)
    fxvec = f(xvec)

    # c. figure
    def _figure(i):
        fig = plt.figure(dpi=100)
        ax = fig.add_subplot(1,1,1)

        ax.plot(xvec,fxvec) # on grid
        ax.plot(m[i],fm[i],'o',color='black',label='current') # mid
        ax.plot([a[i],b[i]],[fm[i],fm[i]],'--',color='black',label='range') # range
        ax.axvline(a[i],ls='--',color='black')
        ax.axvline(b[i],ls='--',color='black')

        ax.legend(loc='lower right')
        ax.grid(True)
        ax.set_ylim([fxvec[0],fxvec[-1]])

    widgets.interact(_figure,
        i=widgets.IntSlider(description="iterations", min=0, max=max_iter, step=1, value=0));

plot_bisection(f, -10, 10)
```

    interactive(children=(IntSlider(value=0, description='iterations', max=18), Output()), _dom_classes=('widget-i…

**Conclusion:** Using the bisection method to calculate the capital per effective worker for the steady state, we found that our model needs 18 iterations to reach the steady state and $k^{*}$ = 0.032844.

```python

```


```python

import numpy as np
import sympy as sympy
import control as co
# We tell python to treat "s" as a SymPy symbol
# We have to make it rational!
s = sympy.Symbol('s', rational=True)
# An example Transfer function
K = (s+3)/(3*s**3 + 2*s**2 + 1*s + 1)
'''
Here is where it gets dirty.
We convert the expression to a Polynomial object in SymPy
and then we extract coefficients, but that list of coefficients
we will get, will be full of sympy.core.numbers, so
it's going to be numbers but sympy's numbers. That's why then we
go and each one of them into a float
'''
# We get the numerator and denomiator
num_list, den_list =  [[float(i) for i in sympy.Poly(i, s).all_coeffs()] for i in K.as_numer_denom()];
# Now num_list and den_list are simple float lists and we put them into
# TransferFunction() function from control module
trf = co.TransferFunction(num_list, den_list)

```

```python
trf
```

$$\frac{s + 3}{3 s^3 + 2 s^2 + s + 1}$$

```python
import sympy as sp
s = sp.Symbol('s', rational=True)
eq=1/(2*s**2 + 3*s + 4)
top, bot = [[float(i) for i in sp.Poly(i, s).all_coeffs()] for i in eq.as_numer_denom()]
import control as co
co.TransferFunction(top, bot)
```

$$\frac{1}{2 s^2 + 3 s + 4}$$

```python

```


# Sparse Relaxations to Binary Random Variables

In this notebook we present sparse relaxations to binary random variables. These are based on relaxing random outcomes from $\{0, 1\}$ to $[0, 1]$ while giving $\{0\}$ and $\{1\}$ discrete treatment. This enables a differentiable reparameterisation that is an unbiased alternative to the biased straight-through estimator.

```python
%load_ext autoreload
%autoreload 2
```

Helper code to plot
* samples
* pdf
* cdf
for distributions over $(\text{lower}, \text{upper}) \supset (0, 1)$

```python
def plot(dist, n=10000, bins=40, step=0.001, lower=-0.2, upper=1.2, masses=[], vlines=[0., 1.], color='blue', title='summary'):

    plt.subplots_adjust(wspace=0.5, hspace=0.8)
    plt.suptitle(title)

    # samples
    plt.subplot(1, 3, 1)
    x = dist.sample(sample_shape=torch.Size([n])).reshape(-1).numpy()
    _ = plt.hist(x, bins, density=False, color=color)
    for value in vlines:
        plt.axvline(value, color='black', linestyle='dashed', linewidth=1)
    plt.xlim(left=lower)
    plt.xlim(right=upper)
    plt.title("samples")

    # pdf
    plt.subplot(1, 3, 2)
    x = np.arange(lower + step, upper, step)
    y = np.exp(dist.log_prob(torch.from_numpy(x).float()).numpy())
    plt.plot(x, y, color=color, linestyle='', marker='.', markersize=0.2)
    for value in vlines:
        plt.axvline(value, color='black', linestyle='dashed', linewidth=1)
    plt.xlim(left=lower)
    plt.xlim(right=upper)
    if masses:
        y_masses = np.exp(dist.log_prob(torch.from_numpy(np.array(masses)).float()).numpy())
        plt.bar(masses, y_masses, width=0.05, align='center', color=color)
#        plt.bar(masses, [dist.p0.numpy(), dist.p1.numpy()], width=0.05, align='center', color=color)
    plt.title("pdf")

    # cdf
    plt.subplot(1, 3, 3)
    y = dist.cdf(torch.from_numpy(x).float()).numpy()
    plt.plot(x, y, color=color, linestyle='', marker='.', markersize=0.2)
    for value in vlines:
        plt.axvline(value, color='black', linestyle='dashed', linewidth=1)
    plt.xlim(left=lower)
    plt.xlim(right=upper)
    if masses:
        y_masses = dist.cdf(torch.from_numpy(np.array(masses)).float()).numpy()
        plt.plot(masses, y_masses, color=color, linestyle='', marker='x', markersize=5)
    plt.title("cdf")

```

```python
import numpy as np
import torch
from matplotlib import pyplot as plt
from torch.distributions import Uniform
from probabll.distributions import Kumaraswamy, Stretched, Truncated01, Rectified01, MixtureD01C01
```

# Stretch and Rectify

Let's start with a Kumaraswamy distribution

```python
a, b = 0.5, 0.5
K = Kumaraswamy(a=torch.full([1], a), b=torch.full([1], b))
```

```python
plot(K, title=f"Kuma({a}, {b})")
```

We can then stretch it to a support larger than $(0, 1)$

```python
SK = Stretched(K)
```

```python
plot(SK, title=f"Stretched({a}, {b})")
```

We can also truncate this stretched distribution back to $(0, 1)$, this requires renormalising the truncated curve to obtain a proper pdf, which we do.

```python
TSK = Truncated01(SK)
```

```python
plot(TSK, title=f"Truncated({a}, {b})")
```

Instead of truncating, we can *rectify* the stretched distribution using a hard-sigmoid. This is quite different. Truncation simply disregards samples outside $(0, 1)$, whereas rectification maps those samples to either $0$ or $1$. The result is a distribution function that mixes a delta function at $0$, another at $1$, and a pdf over $(0, 1)$. The weight of $\delta(x)$ is given by how much of the stretched curve covers the negative support. Similarly, the weight of $\delta(1-x)$ is given by how much of the stretched curve covers the support beyond $1$.
Finally, the continuous distribution in the middle, i.e. over $(0, 1)$, is a density obtained by renormalising the truncated curve (see why we needed that?!).

```python
RSK = Rectified01(SK)
```

```python
plot(RSK, masses=[0., 1.], title=f"Rectified01({a}, {b})")
```

This abstract is very powerful! For as long as we know the cdf and the inverse cdf of a distribution that includes $(0, 1)$ in its support, we can obtain rectified distributions as presented above.

Torch has efficient and differentiable algorithms for the Gaussian cdf and inverse cdf, thus we can use a standard Normal, for example, as base for rectification.

```python
from torch.distributions import Normal
```

If we truncated to $(0, 1)$ a Gaussian centered at $0.5$ with a large variance, we will see that the pdf is close to uniform.

```python
plot(Truncated01(Normal(torch.full([1], 0.5), torch.full([1], 10))), title="T-Normal(0.5, 10)")
```

Let's rectify $\mathcal N(0.5, 10)$ and see what happens.

```python
RN = Rectified01(Normal(torch.full([1], 0.5), torch.full([1], 10)))
plot(RN, masses=[0., 1.], title="Rectified-Gaussian(0.5, 10)")
```

This is very close to assigning $0.5$ to each point mass with some negligible mass roughly uniformly spread over $(0,1)$. That looks like a cool prior, doesn't it?

Here is another way to get similar distribution, without the complicated Gaussian cdf and icdf.

```python
RU = Rectified01(Uniform(torch.full([1], -1.), torch.full([1], 2.)))
plot(RU, masses=[0., 1.], title="Rectified-Uniform(-1, 2.)")
```

and note that in this case the point masses depend on how far we push the support of the Uniform. We could even have the masses differ in importance.

# Direct Parameterisation

Another way to obtain sparse relaxations is to directly parameterise a mixture. Something like:

\begin{align}
    p(x|a, f) &= a_0 \delta(x) + a_1 \delta(1-x) + \underbrace{(1 - a_0 - a_1)}_{\overset{\text{def}}{=} a_c} f(x) \mathbb 1_{(0,1)}(x)
\end{align}

where

* $f(x)$ is a *properly normalised* density over $(0, 1)$
* $\mathbb 1_{(0,1)}(x)$ is the indicator function

Note that *rectified distributions* are members of this family too, howerver, in that case $a_0$, $a_1$, $a_c$, and $f(x)$ all depend on how mass distributes over some support $(l, u) \supset (0, 1)$ as governed by a base density $\pi(x)$. In that case we have

\begin{align}
a_0 &= \int_{-\infty}^0 \pi(x) \text{d}x \\
a_1 &= \int_{1}^\infty \pi(x) \text{d}x \\
a_c &= \int_{0}^1 \pi(x) \text{d}x \\
f(x) &= \frac{\pi(x)\mathbb 1_{(0,1)}(x)}{a_c}
\end{align}

Let's start by mixing deltas with uniform and controlling the weights of the deltas to get the shape we want:

```python
MU = MixtureD01C01(
    cont=Uniform(torch.zeros(1), torch.ones(1)),
    probs=torch.from_numpy(np.array([0.2, 0.3, 0.5])).float()
)
plot(MU, masses=[0., 1.], title="M-Uniform")
```

We can also mix with a Kumaraswamy. One big difference here is that the Kumaraswamy has shapes that promote getting arbitrarily close to $0$ or $1$ without ever touching those values (note how the pdf will attain large values close to $0$ and $1$ in the example below).

```python
MK = MixtureD01C01(
    cont=Kumaraswamy(a=torch.full([1], 0.5), b=torch.full([1], 0.5)),
    probs=torch.from_numpy(np.array([0.2, 0.3, 0.5])).float()
)
plot(MK, masses=[0., 1.], title="M-Kuma")
```

# KL Divergence

Let's consider two distributions in this family of sparse relaxations to binary distributions

\begin{align}
p(x|a, f) &= a_0 \delta(x) + a_1 \delta(1-x) + \underbrace{(1 - a_0 - a_1)}_{\overset{\text{def}}{=} a_c} f(x) \mathbb 1_{(0,1)}(x) \\
q(x|b, g) &= b_0 \delta(x) + b_1 \delta(1-x) + \underbrace{(1 - b_0 - b_1)}_{\overset{\text{def}}{=} b_c} g(x) \mathbb 1_{(0, 1)}(x)
\end{align}

and compute the KL between them.

Then
\begin{align}
    &\text{KL}(p(x|a, f) || q(x|b, g)) \\
    &= a_0 \log \frac{a_0}{b_0} \\
    &+ a_1 \log \frac{a_1}{b_1} \\
    &+ a_c \log \frac{a_c}{b_c} \\
    &+ a_c \int_0^1 f(x) \log \frac{f(x)}{g(x)}  \text{d}x
\end{align}

We have some helper code to debug KL/entropy implementations. The idea is to compare to simpler MC estimates.

```python
from probabll.distributions.utils import mc_kl
```

When training our models, we generally estimate KL with a single sample, but we do so in mini-batches full of training instances and over many epochs. Thus I am going to get a sample mean of the single-sample estimate based on $1000$ simulations.

```python
mc_kl(MK, MU, 100000), np.mean([MK.kl(MU).numpy() for _ in range(1000)])
```

    (tensor([0.1069]), inf)

Also, we could have exploited the fact that we know the Kumaraswamy entropy:

```python
mc_kl(MK, MU, 100000), np.mean([MK.kl(MU, exact_entropy=True).numpy() for _ in range(1000)])
```

    (tensor([0.1089]), 0.113705635)

Recall I pointed out that rectified distributions are in the same family, we can estimate KL for them too:

```python
mc_kl(RSK, RN, 100000), np.mean([RSK.kl(RN).numpy() for _ in range(1000)])
```

    (tensor([1.4282]), 1.4279145)

```python
mc_kl(RSK, RU, 100000), np.mean([RSK.kl(RU).numpy() for _ in range(1000)])
```

    (tensor([0.2061]), 0.20784779)

And we can even mix and match ;)

```python
mc_kl(RSK, MK, 100000), np.mean([RSK.kl(MK).numpy() for _ in range(1000)])
```

    (tensor([0.0914]), 0.09901883)

```python
mc_kl(RSK, MU, 100000), np.mean([RSK.kl(MU).numpy() for _ in range(1000)])
```

    (tensor([0.0510]), 0.050071854)

```python

```


```python
from sympy import *

kwargs = {}
kwargs["mat_str"] = "bmatrix"
kwargs["mat_delim"] = ""

l, EI, M = var("l, EI, M")

l2 = l*l
l3 = l*l*l

pprint("\nVar. 1:")
K = EI/l3
K *= Matrix(
[
[  4*l2 ,  -6*l ,  2*l2 ,   6*l ,  0    ,   0    ],
[ -6*l  ,  12   , -6*l  , -12   ,  0    ,   0    ],
[  2*l2 ,  -6*l ,  8*l2 ,   0   ,  2*l2 ,   6*l  ],
[  6*l  , -12   ,  0    ,  24   , -6*l  , -12    ],
[  0    ,   0   ,  2*l2 ,  -6*l ,  4*l2 ,   6*l  ],
[  0    ,   0   ,  6*l  , -12   ,  6*l  ,  12    ],
]
)
pprint(latex(K / (EI/l3),**kwargs))

p1,w1,p2,w2,p3,w3 = var("psi,w1,psi2,w2,psi3,w3")
M1,F1,M2,F2,M3,F3 = var("M1,F1,M2,F2,M3,F3")
u = Matrix([0,0,p2,w2,0,0])
f = Matrix([M1,F1,-M,0,M3,F3])

unknowns = [M1,F1,p2,w2,M3,F3]

eq = Eq(K*u , f)
sol = solve(eq, unknowns)
pprint(sol)

pprint("\nVar. 2:")
K = EI/l3
K *= Matrix(
[
[  4*l2 ,  -6*l ,  2*l2 ,   6*l  ],
[ -6*l  ,  12   , -6*l  , -12    ],
[  2*l2 ,  -6*l ,  4*l2 ,   6*l  ],
[  6*l  , -12   ,  6*l  ,  12    ],
]
)
# pprint("\nK:")
# pprint(latex(K / (EI/l3),**kwargs))

p1,w1,p2,w2 = var("psi1,w1,psi2,w2")
M1,F1,M2,F2 = var("M1,F1,M2,F2")
u = Matrix([p1,0,0,0])
f = Matrix([-M/2,F1,M2,F2])

unknowns = [p1,F1,M2,F2]

eq = Eq(K*u , f)
sol = solve(eq, unknowns)
pprint(sol)

# Var. 1:
# ⎧    3⋅M      -3⋅M       -M       -M       -M⋅l        ⎫
# ⎨F₁: ───, F₃: ─────, M₁: ───, M₃: ───, ψ₂: ─────, w₂: 0⎬
# ⎩    4⋅l       4⋅l        4        4        8⋅EI       ⎭
#
# Var. 2:
# ⎧    3⋅M      -3⋅M       -M       -M⋅l ⎫
# ⎨F₁: ───, F₂: ─────, M₂: ───, ψ₁: ─────⎬
# ⎩    4⋅l       4⋅l        4        8⋅EI⎭

```


# Демонстрация. Методы научного программирования на примере библиотеки символьных вычислений SymPy.

На основе [урока от Jülich Supercomputing Centre](https://docs.jupyter-jsc.fz-juelich.de/github/FZJ-JSC/jupyter-jsc-notebooks/blob/master/001-Jupyter/001-Tutorials/001-Basic-Tutorials/001-IPython-Kernel/SymPy.ipynb).

Добавляйте ячейки исходного кода программы:

```python
from IPython.display import display

from sympy.interactive import printing
printing.init_printing()

from __future__ import division
import sympy as sym
from sympy import *
x, y, z = symbols("x y z")
k, m, n = symbols("k m n", integer=True)
f, g, h = map(Function, 'fgh')
```

Добавляйте подзаголовки

И свои комментарии:

Исполняемые фрагменты кода:

Алгебра

```python
eq = ((x+y)**2 * (x+1))
eq
```

```python
expand(eq)
```

```python
a = 1/x + (x*sin(x) - 1)/x
a
```

```python
simplify(a)
```

```python
eq = Eq(x**3 + 2*x**2 + 4*x + 8, 0)
eq
```

```python
solve(eq, x)
```

```python
a, b = symbols('a b')
Sum(6*n**2 + 2**n, (n, a, b))
```

Мат. анализ

```python
limit((sin(x)-x)/x**3, x, 0)
```

```python
(1/cos(x)).series(x, 0, 6)
```

```python
diff(cos(x**2)**2 / (1+x), x)
```

```python
integrate(x**2 * cos(x), (x, 0, pi/2))
```

```python
eqn = Eq(Derivative(f(x),x,x) + 9*f(x), 1)
display(eqn)
dsolve(eqn, f(x))
```

#Научная визуализация

На примере разложения в ряд тейлора.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
```

```python
def plot_taylor_approximations(func, x0=None, orders=(2, 4), xrange=(0,1), yrange=None, npts=200):
    """Plot the Taylor series approximations to a function at various orders.

    Parameters
    ----------
    func : a sympy function
    x0 : float
      Origin of the Taylor series expansion.  If not given, x0=xrange[0].
    orders : list
      List of integers with the orders of Taylor series to show.  Default is (2, 4).
    xrange : 2-tuple or array.
      Either an (xmin, xmax) tuple indicating the x range for the plot (default is (0, 1)),
      or the actual array of values to use.
    yrange : 2-tuple
      (ymin, ymax) tuple indicating the y range for the plot.  If not given,
      the full range of values will be automatically used.
    npts : int
      Number of points to sample the x range with.  Default is 200.
    """
    if not callable(func):
        raise ValueError('func must be callable')
    if isinstance(xrange, (list, tuple)):
        x = np.linspace(float(xrange[0]), float(xrange[1]), npts)
    else:
        x = xrange
    if x0 is None: x0 = x[0]
    xs = sym.Symbol('x')
    # Make a numpy-callable form of the original function for plotting
    fx = func(xs)
    f = sym.lambdify(xs, fx, modules=['numpy'])
    # We could use latex(fx) instead of str(), but matploblib gets confused
    # with some of the (valid) latex constructs sympy emits.  So we play it safe.
    plt.plot(x, f(x), label=str(fx), lw=2)
    # Build the Taylor approximations, plotting as we go
    apps = {}
    for order in orders:
        app = fx.series(xs, x0, n=order).removeO()
        apps[order] = app
        # Must be careful here: if the approximation is a constant, we can't
        # blindly use lambdify as it won't do the right thing.  In that case,
        # evaluate the number as a float and fill the y array with that value.
        if isinstance(app, sym.numbers.Number):
            y = np.zeros_like(x)
            y.fill(app.evalf())
        else:
            fa = sym.lambdify(xs, app, modules=['numpy'])
            y = fa(x)
        tex = sym.latex(app).replace('$', '')
        plt.plot(x, y, label=r'$n=%s:\, %s$' % (order, tex) )

    # Plot refinements
    if yrange is not None:
        plt.ylim(*yrange)
    plt.grid()
    plt.legend(loc='best').get_frame().set_alpha(0.8)
```

Определив функцию, строим графики:

```python
plot_taylor_approximations(sin, 0, [2, 4, 6], (0, 2*pi), (-2,2))
```

```python
plot_taylor_approximations(cos, 0, [2, 4, 6], (0, 2*pi), (-2,2))
```

```python
plot_taylor_approximations(lambda x: 1/cos(x), 0, [2,4,6], (0, 2*pi), (-5,5))
```


# Moving Average

Moving average is a series of averages points between many fixed subsets of a dataset.

## Simple Moving Average
The unweighted mean of the previous k data-points.

\begin{aligned}
S M A_{k} &=\frac{p_{n-k+1}+p_{n-k+2} \cdots+p_{n}}{k} \\
&=\frac{1}{k} \sum_{i=n-k+1}^{n} p_{i}
\end{aligned}

## Cumulative Moving Average
The data arrive in an ordered datum stream, and the user would like to get the average of all of the data up until the current datum
\begin{aligned}
C M A_{n+1} &=\frac{x_{n+1}+n \cdot C M A_{n}}{n+1} \\
&=\frac{x_{n+1}+(n+1-1) \cdot C M A_{n}}{n+1} \\
&=\frac{(n+1) \cdot C M A_{n}+x_{n+1}-C M A_{n}}{n+1} \\
&=C M A_{n}+\frac{x_{n+1}-C M A_{n}}{n+1}
\end{aligned}

Where :
\begin{equation}
C M A_{n}=\frac{x_{1}+\cdots+x_{n}}{n}
\end{equation}

## Weighted Moving Average

The average that has multiplying factors to give different weights to data at different positions in the sample window. Mathematically, the weighted moving average is the convolution of the data with a fixed weighting function.

\begin{equation}
\mathrm{WMA}_{M}=\frac{n p_{M}+(n-1) p_{M-1}+\cdots+2 p_{((M-n)+2)}+p_{((M-n)+1)}}{n+(n-1)+\cdots+2+1}
\end{equation}

## Exponential Moving Average

The first-order infinite impulse response filter that applies weighting factors which decrease exponentially. The weighting for each older datum decreases exponentially, never reaching zero. The graph at right shows an example of the weight decrease.

$$
\mathrm{EMA}_{\text {today }}=\frac{p_{1}+(1-\alpha) p_{2}+(1-\alpha)^{2} p_{3}+(1-\alpha)^{3} p_{4}+\cdots}{1+(1-\alpha)+(1-\alpha)^{2}+(1-\alpha)^{3}+\cdots}
$$
since $1 / \alpha=1+(1-\alpha)+(1-\alpha)^{2}+\cdots$.

# Momentum
Momentum investing is a system of buying stocks or other securities that have had high returns over the past three to twelve months, and selling those that have had poor returns over the same period.

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('classic')

pd.plotting.register_matplotlib_converters()
%matplotlib inline
```

# Discussion: Week 4

## Exercise: The Perpetual Inventory Method

The stock of physical capital is a fundamental component of many macroeconomic models but measuring, and therefore acquiring data on, the value of a country's capital stock is hard. The challenge arise because it is hard to estimate the value of something that is not changing ownership in a transaction. What is value of a high-rise apartment building that has had the same owner for 50 years?

One way to overcome this challenge is to use the **perpetual inventory method**. The perpetual inventory method is based on the law of motion of the capital stock:

\begin{align}
K_{t+1} & = I_t + (1-\delta) K_t
\end{align}

where $K_t$ denotes physical capital, $I_t$ denotes investment (purchases of new capital), and $\delta$ denotes the rate of depreciation. Give $K_0$ and data on investment $I_0, I_1, \ldots,$, the the capital stock in periods $1, 2, \ldots$ can be computed iteratively. To obtain $K_0$, the method assumes that the economy is in  a steady state in the initial year. Recall that in the steady state:

\begin{align}
I & = \delta K
\end{align}

So, set $K_0 = \delta^{-1} I_0$.

Steps:

1. Obtain real investment data (https://fred.stlouisfed.org/series/GPDICA)
2. Select a value for the depreciation rate $\delta$. 0.1 is a standard choice.
3. Initialize an array of zeros (or an empty array) to store capital values
4. Set the first value of the captial array equal to the $\delta^{-1} I_0$
5. Iterate over the values in the investment data to compute the remaining values of capital.

Read Timothy Kehoe's notes (http://users.econ.umn.edu/~tkehoe/classes/GrowthAccountingNotes.pdf) for more background on the perpetual inventory method if you're interested.

```python
# Import real investment data from FRED into a DataFrame called 'data' (PROVIDED)
data = pd.read_csv('https://fred.stlouisfed.org/data/GPDICA.txt',sep='\s+',skiprows=13,index_col=0,parse_dates=True)

# Set the columns attribute of data to ['Investment']

# Set the index.name attribute of data to 'Date' (i.e., data.index.name)

# Create a variable called 'delta' that stores the value for delta as 0.1

# Create a variable called 'capital' equal to an array of zeros with the same length as the variable 'data'

# Set the initial value of the 'capital' variable equal to the initial value of investment divided by delta

# Iterate (i.e., use a for loop) over t in range(len(capital)-1) to construct values in the capital array

# Add a 'Capital' column to data

# Divide data by 1000 so that the values are in units are in trillions of $ (instead of billions)

# Print the first 5 rows of data

```

Now construct a figure that contains a plot of capital in the top panel and investment in the bottom panel.

1. The figure should have dimensions $12\times 8$.
2. Y-axis labels for investment and capital should both be "trillions of chained dollars"
3. Each axis should have an appropriate title.

```python
# Create figure

# Construct the plot of capital

# Construct the plot of investment

# Use fig.tight_layout() to adjust whitespace

```

**Question**

1. Why does the computed capital series fluctuate less than the investment data?
2. Given that there are about 325 million people in the US, does the computed value in the last year look large, small, or about right on a per capita basis?

**Answer**

1.

2.


# Constructing Operators and States

Part #1

## Load necessary modules

Import statements like these need to be at the top of every notebook you create.

```python
import numpy as np                # Needed to create arrays (vectors and matrices).
import qutip as qt                # the QuTiP library.
import matplotlib.pyplot as plt   # Plotting tools.
%matplotlib inline
```

## Constructing Operators and States for Two-Level Systems

Start by generating the Pauli spin operators and identity operator for a two-level system. From your favorite QM textbook:

$$
\begin{equation}
\sigma_{x} = \begin{bmatrix}
0 & 1 \\
1 & 0
\end{bmatrix},
\ \
\sigma_{y} = \begin{bmatrix}
0 & -i \\
i & 0
\end{bmatrix},
\ \
\sigma_{z} = \begin{bmatrix}
1 & 0 \\
0 & -1
\end{bmatrix},
\ \
I = \begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}
\end{equation}
$$

```python
qt.sigmax()
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 1.0\\1.0 & 0.0\\\end{array}\right)\end{equation*}

```python
qt.sigmay()
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & -1.0j\\1.0j & 0.0\\\end{array}\right)\end{equation*}

```python
qt.sigmaz()
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}1.0 & 0.0\\0.0 & -1.0\\\end{array}\right)\end{equation*}

```python
qt.identity(2) # Identity works for any dimension, not just 2.
```

Quantum object: dims = [[2], [2]], shape = (2, 2), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}1.0 & 0.0\\0.0 & 1.0\\\end{array}\right)\end{equation*}

### What implicit assumption did we make when writing down these operators?

We have implicity choosen the z-axis as our preferred basis (representation).  What do the corresponding state vectors look like?

```python
up = qt.basis(2,0) # First number is Hilbert dimension, second labels the state
up
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}1.0\\0.0\\\end{array}\right)\end{equation*}

If I don't know what the inputs are, then it is always possible to ask for help.

```python
qt.basis?
```

```python
down = qt.basis(2,1)
down
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\1.0\\\end{array}\right)\end{equation*}

Check the action of $\sigma_{z}$ on the states:

```python
qt.sigmaz() * up
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}1.0\\0.0\\\end{array}\right)\end{equation*}

```python
qt.sigmaz() * down
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\-1.0\\\end{array}\right)\end{equation*}

Of course, given a basis for our Hilbert space, we can form any other state via a linear combination of basis vectors:
$$
\begin{equation}
|\Psi\rangle = a|\uparrow\rangle+b|\downarrow\rangle
\end{equation}
$$

```python
psi = 5*up - 1j*down # Complex numbers in Python written as e.g. 1+3j.
psi
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}5.0\\-1.0j\\\end{array}\right)\end{equation*}

State is not properly normalized.  Can be easily fixed.

```python
psi.unit()
```

Quantum object: dims = [[2], [1]], shape = (2, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.981\\-0.196j\\\end{array}\right)\end{equation*}

What's going on here?  See lecture slides for description of the Quantum Object class.

Part #2

## Constructing Operators and States for Oscillators

Unlike two-level systems, oscillators live in a formally infinite Hilbert space.  Therefore we must be careful as to where we truncate the state space for simulation.

#### The truncation of state space can lead to misunderstandings and error, if not accounted for.

A good choice of basis, and the one used by QuTiP is the usual number state $|n\rangle$ (Fock) basis.

```python
N = 5 # Must always specify how large you want the Hilbert space to be.
psi = qt.basis(N,0)
psi
```

Quantum object: dims = [[5], [1]], shape = (5, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}1.0\\0.0\\0.0\\0.0\\0.0\\\end{array}\right)\end{equation*}

This is the ground state $|0\rangle$ of the number state basis. To get the $|1\rangle$ state simply do:

```python
qt.basis(N,1)
```

Quantum object: dims = [[5], [1]], shape = (5, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\1.0\\0.0\\0.0\\0.0\\\end{array}\right)\end{equation*}

What this shows is that for a given Hilbert size $N$, the corresponding basis vectors go from $n=0 \rightarrow N-1$.

We know that to move up and down the number state basis we need creation and annihiation operators.

```python
a = qt.destroy(N)
a
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = False\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 1.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.414 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 1.732 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 2.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

To build the creation operator we could do `create(N)`, but lets us a `Qobj` method instead:

```python
a.dag() # The 'dag' method creates the dagger (adjoint) of an operator.
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = False\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\1.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 1.414 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.732 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 2.0 & 0.0\\\end{array}\right)\end{equation*}

```python
a.dag()*psi
```

Quantum object: dims = [[5], [1]], shape = (5, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\1.0\\0.0\\0.0\\0.0\\\end{array}\right)\end{equation*}

```python
a.dag()**2 * psi
```

Quantum object: dims = [[5], [1]], shape = (5, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\0.0\\1.414\\0.0\\0.0\\\end{array}\right)\end{equation*}

We can also build operators such as the number operator:

```python
a.dag()*a
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 1.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 2.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 3.000 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 4.0\\\end{array}\right)\end{equation*}

or we could have just used the builtin number operator function:

```python
qt.num(N)
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 1.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 2.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 3.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 4.0\\\end{array}\right)\end{equation*}

## Building Density Matrices

QuTiP focuses on open quantum systesm, i.e. systems that interact with an environment.  This demands the use of density matrices, rather than state vectors, as the interaction with the environment, in general, produces a mixed state.

Typically, we prepare our system in a given pure state.  The corresponding density matrix would then just be formed by the outer product $|\Psi\rangle\langle \Psi|$.

```python
psi = qt.basis(N,2)
psi
```

Quantum object: dims = [[5], [1]], shape = (5, 1), type = ket\begin{equation*}\left(\begin{array}{*{11}c}0.0\\0.0\\1.0\\0.0\\0.0\\\end{array}\right)\end{equation*}

```python
psi*psi.dag()
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

We could also use the built in function `ket2dm()` to do the product for us:

```python
qt.ket2dm(psi)
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

QuTiP has a collection of built in density operators that are commonly used:

```python
qt.fock_dm(N,2)
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 1.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.0\\\end{array}\right)\end{equation*}

```python
qt.coherent_dm(N,alpha=1) #alpha specifies the coherent state amplitude
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.368 & 0.368 & 0.261 & 0.146 & 0.088\\0.368 & 0.368 & 0.261 & 0.146 & 0.088\\0.261 & 0.261 & 0.185 & 0.104 & 0.063\\0.146 & 0.146 & 0.104 & 0.058 & 0.035\\0.088 & 0.088 & 0.063 & 0.035 & 0.021\\\end{array}\right)\end{equation*}

The coherent states represent one example of a situation where the state vector or density matrix is a non-sparse in the Fock basis.  This is because they are generated using a displacement operator using `expm`.

```python
qt.thermal_dm(N,1) #`1` indicates the average number of particles in the thermal state
```

Quantum object: dims = [[5], [5]], shape = (5, 5), type = oper, isherm = True\begin{equation*}\left(\begin{array}{*{11}c}0.516 & 0.0 & 0.0 & 0.0 & 0.0\\0.0 & 0.258 & 0.0 & 0.0 & 0.0\\0.0 & 0.0 & 0.129 & 0.0 & 0.0\\0.0 & 0.0 & 0.0 & 0.065 & 0.0\\0.0 & 0.0 & 0.0 & 0.0 & 0.032\\\end{array}\right)\end{equation*}

These are just simple examples of density matrices.  The more interesting ones comes from doing open-system time-evolution.


# Quantum Walk Search Algorithm

Quantum walks are the quantum equivalence of a classical Markov chain and have become key in many quantum algorithms. In this section, we implement a quantum walk search algorithm that finds marked elements in a graph. The algorithm has a quadratic speedup compared to its classical counterpart.

## 1.Classical Markov chains
A Markov chain is a stochastic process often used to model real-life processes. It consists of states and associated transition probabilities, which describe the probabilities to move between the states in each time step. In discrete-time Markov chains, which we work with here, the time steps are discrete. Markov chains satisfy the Markov property, which means that the next step of the process only depends on the current and not any of the previous steps. A Markov chain has an associated transition matrix, P, which describes the probability to move between each of the states. We show an example of a Markov chain and its associated transition matrix $P$ below.
\begin{equation}
P=
\begin{pmatrix}
0.1 & 0.3 & 0.3\\
0.1 & 0.1 & 0.2 \\
0.8 & 0.6 & 0.5
\end{pmatrix}
\label{eq:matrix_example}
\end{equation}

Given a transition matrix $P$, we can obtain the probability distribution after $t$ time steps by $P^t$.

## 2.Quantum walks
Quantum walks are the quantum equivalent of the classical Markov chain. Due to superposition, a quantum walk will take all possible paths simultaneously until we measure the circuit. Due to quantum interference, some state will cancel out. This makes quantum walk algorithms faster than classical ones, since we can design them in such way that wrong answers quickly cancells out. There are two common models for quantum walks, coined quantum walks and Szegedy quantum walks, which are equivalent under certain circumstances. A coined quantum walk is a walk on the vertices of the graph while Szegedy quantum walk takes place on the edges. Before we show how we can implement a quantum walk, we will introduce both models.

 $\newcommand{\ket}[1]{\left|{#1}\right\rangle}$
 $\newcommand{\bra}[1]{\left\langle{#1}\right|}$
### 2.A Coined quantum walks
A simple example of a coined quantum walk is a walk on the infinite integer line. In this case, we represent the walker's position with an integer, $\{\ket{j} : j \in  \mathbb{Z} \}$, since the walker can walk all integers in $\mathbb{Z}$. A coin decides how the walker should move. On the integer line, the walker can go either left or right. Therefore, the coin's computational basis is $\{\ket{0}, \ket{1}\}$, and we move the walker in one direction if the coin is $\ket{0}$ and in the other direction if the coin is $\ket{1}$.

A coined quantum is a walk on the nodes in a graph, and we refer to the nodes as states. The walker can move between states that are connected with an edge. In the coin model, we have two quantum states and two operators. The first state is the position state, which represents the walker's position. For the walk above, this is an integer since the walker can be anywhere on the integer line. The other state is the coin state. The coin state decides how the walker should move in the next step. We can represent both the coin state and the position state by vectors in Hilbert space. If we can express the coin state by a vector in $\mathcal{H}_C$ and the position state with a vector in $\mathcal{H}_P$, we can express the quantum space for the entire walker as $\mathcal{H} = \mathcal{H}_C \otimes \mathcal{H}_P$.

As we mentioned, the model also has two operators; the coin operator, $C$, and the shift operator, $S$. The coin operator acts on $\mathcal{H}_C$ during each time step and puts the walker in superposition, so it walks all possible paths simultaneously. For the walk on the integer line, this means that it moves both to the left and right in each time step. There are different coin operators, but the most common ones are the Hadamard coin and the Grover coin. A Hadamard coin is a Hadamard gate that puts the walker in an equal superposition:

\begin{equation}
    H = \frac{1}{\sqrt{2}}
\begin{bmatrix}
1 & 1 \\
1 & -1
\end{bmatrix}
\end{equation}

A Grover coin is the Grover diffusion operator from Grover's algorithm. We define it as

\begin{equation}
    G =
    \begin{bmatrix}
\frac{2}{n} -1 & \frac{2}{n} & \ldots & \frac{2}{n}\\
\frac{2}{n} & \frac{2}{n} - 1  & \ldots & \frac{2}{n} \\
\vdots & \vdots & \ddots & \vdots \\
\frac{2}{n} & \frac{2}{n} & \ldots & \frac{2}{n} -1
\end{bmatrix}
\end{equation}

Like the Hadamard coin, a Grover coin puts the walker in superposition. However, it behaves a bit differently. If we apply a Grover coin to a walker in position $\ket{000}$, we obtain the state vector probabilities shown in the figure below. As we can see, the coin does not put the walker in an equal superposition as the Hadamard coin does. Instead, $\ket{000}$ has a much larger probability than the other states.

The other operator in the model, the shift operator, acts on $\mathcal{H}_P$ and moves the walker to the next position. For the walk on the integer line, the shift operator moves the walker to the left of the coin is $\ket{0}$ and to the right if the coin is $\ket{1}$:

\begin{equation}
    S \ket{0}\ket{j} = \ket{0}\ket{j+1}
\end{equation}

\begin{equation}
    S \ket{1}\ket{j} = \ket{1}\ket{j-1}
\end{equation}

With the shift operator defined as above, we can represent one step of the coined quantum as the unitary operator $U$ given by
\begin{equation}
    U = SC,
\end{equation}

where C is the coin operator. For the quantum walk on the integer line we use the Hadamard coin, but C can be either Hadamard coin, Grover coin or any other coin operator.

We can also look several steps ahead. We can express the quantum state $\ket{\psi}$ after $t$ time steps as
\begin{equation}
    \ket{\psi (t)} = U^t \ket{\psi(0)},
\end{equation}
where $\ket{\psi(0)}$ is the initial state and U is the operator for one step of the walk [1].

Coined quantum walks are most suitable for regular graphs, graphs where all nodes have the same number of neighbors [2]. An alternative quantum walk model that is more convenient for non-regular graphs is the Szegedy model that we will look at next.

### 2.B Szedgedy quantum walk
While a coined walk is a walk on the graph's nodes, a Szegedy walk is a walk on the edges on a bipartite double-cover of the original graph. The double-cover graph is a graph with twice as many vertices as the original graph. Two nodes in the bipartite double-cover graph are connected with an edge if and only if the nodes are also connected in the original graph. To create this model, we start with the transition probability matrix P for the classical walk. As described in section 1, we represent a classical discrete-time random walk by a transition matrix $P$. For any $N$-vertex graph with $N \times N$ transition matrix $P$, we can define the corresponding discrete-time quantum walk as a unitary operation on the Hilbert space $\mathcal{H}^N \otimes \mathcal{H}^N$. Let $P_{jk}$ define the probability that we make a transition from state $j$ to $k$. Before we define the walk, we define the normalized states

\begin{equation}
    \ket{\psi_j} := \sum_{k=1}^N \sqrt{P_{kj}} \ket{j,k}, \; j=1,...,N
\end{equation}

and the projection onto ${\ket{\psi_j}}:j=1,...,N$

\begin{equation}
    \Pi := \sum_{j=1}^N \ket{\psi_j} \bra{\psi_j}
\label{eq:sz_pi}
\end{equation}

We also introduce the shift operator S:

\begin{equation}
    S := \sum_{j,k=1}^N \ket{j,k} \bra{k,j}
\label{eq:sz_s}
\end{equation}

With $S$ and $\Pi$ defined as above we can introduce one step of the discrete-time quantum walk:

\begin{equation}
    U := S(2 \Pi - 1),
\label{eq:sz_op}
\end{equation}

where $(2 \Pi - 1)$ is the reflection operator. We also define $t$ steps of the walk as $U^t$ [2].

### 2.C Equivalence of coined and Szegedy quantum walks
It is known that a coined walk with Grover coin is equivalent to Szegedy quantum walk. For more details, we refer to this article [3] by Thomas G. Wong, where he also shows the equivalence between the operators in the two models.

## 3. Example: Implementing quantum walk on a hypercube
A hypercube is the $n$-dimensional counterpart of the $3$-dimensional cube. All nodes have degree $n$, and the hypercube has a total of $N=2^n$ nodes. We can represent the nodes in a hypercube graph by $n$-tuples of binary numbers. The binary representation of the neighbors of a node will differ by only one binary number. For example, in the 4-dimensional hypercube, the neighbors to $0000$ are $0001$, $0010$, $0100$, and $1000$. Thus, a node is connected to all nodes to which the Hamming distance is 1. The edges are also labeled. Two neighboring nodes that differ in the a:th bit are connected by the edge labeled $a$.

The Hilbert space representing a coined quantum walk on the hypercube is $\mathcal{H} = \mathcal{H}^n \otimes \mathcal{H}^{2^n}$, where $\mathcal{H}^n$ denotes the coin space and $\mathcal{H}^{2^n}$ the walker's position. The computational basis is

\begin{equation}
    \big\{ \ket{a,\vec{v}}, 0 \leq a \leq n-1, \vec{v} \in  \{(00...00), (00...01), ....., (11...11 )\} \big\}.
\end{equation}

The value of the coin computational basis $a$, which is associated with edge $a$, decides where the walker should move. If the $a=0$, the walker will go to the node where the first binary value differs from the current node. If $a=1$, the walker will move to the node in which the second value differs from the current, et cetera.  Let $\vec{e}_a$ be an n-tuple where all binary values, except the value with index $a$, are $0$. Then the shift operator $S$ moves the walker from the state $\ket{a} \ket{\vec{v}}$ to $\ket{\vec{v} \oplus \vec{e}_a}$:

\begin{equation}
    S \ket{a} \ket{\vec{v}} = \ket{a} \ket{\vec{v} \oplus \vec{e}_a}.
\end{equation}

We use the Grover coin, $G$, for this walk. Thus, the evolution operator is

\begin{equation}
    U = SG.
\end{equation}

We will now show how we can implement a quantum walk on a 4-dimensional hypercube. We need to implement the coin operator and the shift operator. We start by importing all necessary libraries from Qiskit.

```python
# Importing standard Qiskit libraries
from qiskit import QuantumCircuit, execute, Aer, IBMQ, QuantumRegister, ClassicalRegister
from qiskit.compiler import transpile, assemble
from qiskit.tools.jupyter import *
from qiskit.visualization import *
from qiskit.circuit.library import QFT
from numpy import pi
from qiskit.quantum_info import Statevector
from matplotlib import pyplot as plt
import numpy as np
# Loading your IBM Q account(s)
provider = IBMQ.load_account()
```

The circuit will have 6 qubits, 4 that represents the position and 2 that represents the coin. As we mentioned previously, the coin is a Grover coin, which is the diffuser in Grover's algorithm. We start by implementing this.

```python
one_step_circuit = QuantumCircuit(6, name=' ONE STEP')
# Coin operator
one_step_circuit.h([4,5])
one_step_circuit.z([4,5])
one_step_circuit.cz(4,5)
one_step_circuit.h([4,5])
one_step_circuit.draw()
```

Now, let us implement the shift operator. We know that the walker can only move to neighboring nodes, and all neighboring nodes differ by only one bit. We want to move the walker according to the coin, and we move the walker by applying a NOT gate to one of the node qubits. If the coin is in state $\ket{11}$, we move the walker to the state in which the first node qubit differ. If the coin is $\ket{10}$ or $\ket{01}$, the walker moves to the state where the second and third qubit, respectively, differ. Finally, if the Grover coin is $\ket{00}$, we flip the fourth qubit. We implement this with CCNOT- and NOT gates after the Grover coin. Together, they are one step of a quantum walk on a 4-dimensional hypercube.

```python
# Shift operator function for 4d-hypercube
def shift_operator(circuit):
    for i in range(0,4):
        circuit.x(4)
        if i%2==0:
            circuit.x(5)
        circuit.ccx(4,5,i)

shift_operator(one_step_circuit)

one_step_gate = one_step_circuit.to_instruction()
one_step_circuit.draw()
```

#### 4.Quantum walk search algorithm
We will now implement a quantum walk search algorithm that finds a marked vertex in a graph. First, we describe the algorithm, then we go through its implementation. The quantum walk search algorithm solves the problem of finding marked vertices in a graph by a quantum walk. That is, we mark some set of vertices $|M|$, start at an arbitrary node in the graph and move according to the walk until we find the marked nodes. The basis states in the quantum walk search algorithm have two registers, one corresponding to the current node and the other corresponding to the previous node. That is, the basis states corresponds to the edges in the graph. We denote the quantum walk based on the classical Markov chain with transition matrix $P$ by the unitary operation $W(P)$ on $\mathcal{H}$. We also define $\ket{p_x} = \sum_y \sqrt{P_{xy}}\ket{y}$ as the uniform superposition over the node $x$'s neighbors. Let $\ket{x}\ket{y}$ be a basis state. We define the basis state $\ket{x}\ket{y}$ as ''good'' if $x$ is a marked node. Otherwise, we refer to it as ''bad''. We now introduce ''good'' and ''bad'' states:

\begin{equation}
    \ket{G} = \frac{1}{\sqrt{|M|}} \sum_{x \in M} \ket{x} \ket{p_x}, \;
    \ket{B} = \frac{1}{\sqrt{N-|M|}} \sum_{x \notin M} \ket{x} \ket{p_x},
\end{equation}

which are the superpositions over good and bad basis states. Next, let us define $\epsilon = |M|/N$ and $\theta = \arcsin(\sqrt{\epsilon})$.

In short, the algorithm consists of three steps:

1. Set up the initial state $\ket{U} = \frac{1}{\sqrt{N}} \sum_{x} \ket{x} \ket{p_x} = \sin{\theta} \ket{G} + \cos{\theta} \ket{B}$, a uniform superposition over all edges
2. Repeat $O(1/\sqrt{\epsilon})$ times:

    (a) Reflect through $\ket{B}$

    (b) Reflect through $\ket{U}$

3. Do a measurement in the computational basis

We can easily implement step $1$ with Hadamard gates and the reflection through $\ket{B}$ with a phase oracle that shifts the phase of $x$ if $x$ is in the first register, and leaves the circuit unchanged otherwise.

Step 2(b) is equivalent to finding a unitary $R(P)$ that performs the following mapping:
\begin{align}
\label{eq:mapping_1}
    \ket{U} &\mapsto \ket{U}, \: \text{and} \\
    \ket{\psi} &\mapsto -\ket{\psi}, \: \forall \ket{\psi} \text{in the span of eigenvectors of $W(P)$ that are orthogonal to $\ket{U}$}
\label{eq:mapping_2}
\end{align}

To find this operator we apply phase estimation on $W(P)$. Above we defined $W(P)$ as the evolution operator for the random walk. As we saw in section 2.A, this is a unitary operator. From this follows that the eigenvalues of $W(P)$ have norm $1$. Because of this, we can write the eigenvalues of $W(P)$ on the form $e^{\pm 2i\theta_j}$. The unitary $W(P)$ has one eigenvector with corresponding eigenvalue $1$, which is $\ket{U}$. This is given by $\theta_1=0$. $R(P)$ will find this vector $\ket{U}$ by adding a register with auxiliary qubits and perform phase estimation with precision $O(1/\sqrt{\delta})$, where $\delta$ is the spectral gap of $P$. To do this, we need to apply $W(P)$ $O(1/\sqrt{\delta})$ times. Let $\ket{w}$ be an eigenvector of $W(P)$ with eigenvalue $e^{\pm 2i\theta_j}$. Assume that $\tilde{\theta_j}$ is the best approximation to $\theta_j$ given by the phase estimation. The operation $R(P)$ that performs the mappings in for $\ket{w}$ in step 2(b) is given by [4]

\begin{equation}
    \ket{w} \ket{0} \mapsto \ket{w} \ket{\tilde{\theta_j}} \mapsto (-1)^{|\tilde{\theta_j} \neq 0|} \ket{w} \ket{\tilde{\theta_j}} \mapsto (-1)^{|\tilde{\theta_j} \neq 0|} \ket{w} \ket{0}
\end{equation}

## 5.Example: Quantum walk search on 4-dimensional hypercube

The quantum walk search algorithm makes it possible to find a marked set of nodes in $O(1/\sqrt{\epsilon})$ steps, $\epsilon =  |M|/N$, where $M$ is the number of marked nodes and $N$ is the total number of nodes. This algorithm is originally used with Szegedy quantum walks, where we use two node registers to represent the quantum state. However, a coined walk with a Grover coin is equivalent to a Szegedy quantum walk and since implementations of coined walks are less complex in general, we choose to implement the algorithm with a coined walk. We will use the 4-dimensional hypercube that we implemented in section 3.

In short, we will implement the algorithm as follows. We achieve step 1, a uniform superposition over all edges, by applying Hadamard gates to the node qubits as well as the coin qubits. For step 2(a), we implement a phase oracle. Step 2(b) is implemented by a phase estimation on one step of the quantum walk on the hypercube followed by marking all quantum states where $\theta \neq 0$. We do this by rotating an auxiliary qubit. In the last part of this step, we reverse the phase estimation. The number of theta qubits depends on the precision of $\theta$.

Below, we implement the quantum walk search algorithm on the 4-dimensional hypercube.

For this algorithm we will need to use the inverse of the one step gate implemented earlier. We get this by using the built in circuit function inverse().

```python
one_step_circuit.inverse().draw()
```

The inversed one step gate will be used to reverse the phase estimation later. We need to make controlled gates from both the one step gate that we implemented in section 3 and its inverse. We will later use them depending on the value of the control qubit.

```python
# Make controlled gates
inv_cont_one_step = one_step_circuit.inverse().control()
inv_cont_one_step_gate = inv_cont_one_step.to_instruction()
cont_one_step = one_step_circuit.control()
cont_one_step_gate = cont_one_step.to_instruction()
```

Both the controlled one step gate and the controlled inversed one step gate will be used in the phase estimation. Another thing we will use in the phase estimation is the Quantum Fourier Transform. Qiskit has a function, QFT, which implements the Quantum Fourier Transform. The phase estimation uses the inverse Quantum Fourier Transform but we also will need to use the ordinary QFT to reverse the phase estimation.

```python
inv_qft_gate = QFT(4, inverse=True).to_instruction()
qft_gate = QFT(4, inverse=False).to_instruction()

QFT(4, inverse=True).decompose().draw("mpl")
```

Before we implement the phase estimation, we implement a phase oracle that marks the states 1011 and 1111. Then, we make it a circuit. This is step 2(a) of the algorithm.

```python
phase_circuit =  QuantumCircuit(6, name=' phase oracle ')
# Mark 1011
phase_circuit.x(2)
phase_circuit.h(3)
phase_circuit.mct([0,1,2], 3)
phase_circuit.h(3)
phase_circuit.x(2)
# Mark 1111
phase_circuit.h(3)
phase_circuit.mct([0,1,2],3)
phase_circuit.h(3)
phase_oracle_gate = phase_circuit.to_instruction()
# Phase oracle circuit
phase_oracle_circuit =  QuantumCircuit(11, name=' PHASE ORACLE CIRCUIT ')
phase_oracle_circuit.append(phase_oracle_gate, [4,5,6,7,8,9])
phase_circuit.draw()
```

We will now implement a gate that rotates an auxiliary qubit if the other qubits are non-zero. We will use this gate in the phase estimation, where it will rotate the auxiliary qubit if $\theta \neq 0$.

```python
# Mark q_4 if the other qubits are non-zero
mark_auxiliary_circuit = QuantumCircuit(5, name=' mark auxiliary ')
mark_auxiliary_circuit.x([0,1,2,3,4])
mark_auxiliary_circuit.mct([0,1,2,3], 4)
mark_auxiliary_circuit.z(4)
mark_auxiliary_circuit.mct([0,1,2,3], 4)
mark_auxiliary_circuit.x([0,1,2,3,4])

mark_auxiliary_gate = mark_auxiliary_circuit.to_instruction()
mark_auxiliary_circuit.draw()
```

Now, we will implement step 2(b) of the algorithm. This step consists of a phase estimation one step of the quantum walk followed by an auxiliary qubit that we rotate if $\theta \neq 0$. For this, we use the mark_auxiliary_gate that we just created.  Thereafter, we reverse the phase estimation.

```python
# Phase estimation
phase_estimation_circuit = QuantumCircuit(11, name=' phase estimation ')
phase_estimation_circuit.h([0,1,2,3])
for i in range(0,4):
    stop = 2**i
    for j in range(0,stop):
        phase_estimation_circuit.append(cont_one_step, [i,4,5,6,7,8,9])

# Inverse fourier transform
phase_estimation_circuit.append(inv_qft_gate, [0,1,2,3])

# Mark all angles theta that are not 0 with an auxiliary qubit
phase_estimation_circuit.append(mark_auxiliary_gate, [0,1,2,3,10])

# Reverse phase estimation
phase_estimation_circuit.append(qft_gate, [0,1,2,3])

for i in range(3,-1,-1):
    stop = 2**i
    for j in range(0,stop):
        phase_estimation_circuit.append(inv_cont_one_step, [i,4,5,6,7,8,9])
phase_estimation_circuit.barrier(range(0,10))
phase_estimation_circuit.h([0,1,2,3])

# Make phase estimation gate
phase_estimation_gate = phase_estimation_circuit.to_instruction()
phase_estimation_circuit.draw()
```

Now we implement the whole quantum walk search algorithm using the gates we made previously. We start by applying Hadamard gates to node and coin qubits, which is step 1 in the algorithm. Thereafter, we iteratively apply the phase oracle gate and the phase estimation gate (steps 2(a) and 2(b)). We need $O(1/\sqrt{\epsilon})$ iterations as stated in the description of the algorithm in section 4. Lastly, we measure the node qubits.

```python
# Implementation of the full quantum walk search algorithm
theta_q = QuantumRegister(4, 'theta')
node_q = QuantumRegister(4, 'node')
coin_q = QuantumRegister(2, 'coin')
auxiliary_q = QuantumRegister(1, 'auxiliary')
creg_c2 = ClassicalRegister(4, 'c')
circuit = QuantumCircuit(theta_q, node_q, coin_q, auxiliary_q, creg_c2)
# Apply Hadamard gates to the qubits that represent the nodes and the coin
circuit.h([4,5,6,7,8,9])
iterations = 2

for i in range(0,iterations):
    circuit.append(phase_oracle_gate, [4,5,6,7,8,9])
    circuit.append(phase_estimation_gate, [0,1,2,3,4,5,6,7,8,9,10])

circuit.measure(node_q[0], creg_c2[0])
circuit.measure(node_q[1], creg_c2[1])
circuit.measure(node_q[2], creg_c2[2])
circuit.measure(node_q[3], creg_c2[3])
circuit.draw()
```

Finally we run the implementation on the qasm simulator. We see that the circuit collapse to the marked states a clear majority of the times.

```python
backend = Aer.get_backend('qasm_simulator')
job = execute( circuit, backend, shots=1024 )
hist = job.result().get_counts()
plot_histogram( hist )
```

## 6. References

1. Renato Portugal. Quantum Walks and Search Algorithms. New York, NY: Springer New York, 2013
2. Markus G. Kuhn.Some Introductory Notes on Quantum Computing. Apr. 2000
3. Thomas G. Wong. “Equivalence of Szegedy’s and coined quantum walks”. In: Quantum InformationProcessing 16.9 (July 2017). ISSN: 1573-1332. DOI:10.1007/s11128-017-1667-y. URL:http://dx.doi.org/10.1007/s11128-017-1667-y.37
4. Ronald de Wolf. Quantum Computing: Lecture Notes. 2021. arXiv:1907.09415 [quant-ph]

```python
import qiskit.tools.jupyter
%qiskit_version_table
```

    /usr/local/anaconda3/envs/terra-unstable/lib/python3.9/site-packages/qiskit/aqua/__init__.py:86: DeprecationWarning: The package qiskit.aqua is deprecated. It was moved/refactored to qiskit-terra For more information see <https://github.com/Qiskit/qiskit-aqua/blob/main/README.md#migration-guide>
      warn_package('aqua', 'qiskit-terra')

Version Information

Qiskit Software	Version

qiskit-terra	0.19.0

qiskit-aer	0.9.1

qiskit-ignis	0.6.0

qiskit-ibmq-provider	0.17.0

qiskit-aqua	0.9.5

qiskit	0.31.0

System information

Python version	3.9.7

Python compiler	Clang 10.0.0

Python build	default, Sep 16 2021 08:50:36

OS	Darwin

CPUs	8

Memory (Gb)	32.0

Fri Nov 05 15:38:53 2021 GMT


# Neural Network Representation

In the first two sections of the course we focused on the important Machine Learning foundations of Linear Regression and Logistic Regression. Both are very important data analysis techniques that have been used extensively in both statistics and machine learning for decades. They are powerful, they are (relatively) easy to understand, and they can be used with a surprisingly large number of problem types. But they aren't the end of the story for us, they are just the beginning. This week we will look at how the idea of linear and logistic functions form the building blocks of Deep Neural Networks.

## Preliminaries
We begin by reintroducing some functions from our Logistic Regression and Linear Regression notes.

```python
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

def standardize(x):
    col_means = np.mean(x,axis=0)
    col_std = np.std(x,axis=0)
    return (x - col_means) / col_std , col_means, col_std

def h_linear(theta,X):
    z = np.matmul(theta,X)
    return z

def h_logistic(theta,X):
    z = np.matmul(theta,X)
    return 1 / (1 + np.exp(-z))

def cost_logistic(theta,xi,yi):
    # note - this function calculate the cost for all variations of theta. A vector is returned.
    z = np.matmul(theta,xi)
    h = 1 / (1 + np.exp(-z))
    if yi == 1:
        return -np.log(h)
    elif yi == 0:
        return -np.log(1-h)
    else: raise ValueError('yi was neither 0 nor 1: ' + str(yi))

def cost_linear(theta,xi,yi):
    # note - this function calculate the cost for all variations of theta. A vector is returned.
    z = np.matmul(theta,xi)
    h = 1 / (1 + np.exp(-z))
    rv = ((h - yi)**2) / 2
    return rv

def J(theta,x,y,f):
    s = 0
    m = len(x)
    for i in range(0,m):
        v = f(theta,x[i],y[i])
        s+= v
    return s / m

def gd(theta_in,min_delta_in,a_in,X_in,Y_in,h,maxiterations=-1,cost=cost_logistic):
    m = len(Y_in) # number of training examples
    _ , n = np.shape(X_in) # number of features

    # define a list used to store the costs - these do not have to be calculated but are useful for debugging
    c = []

    # Create a theta variable which we will use to keep track of the current value of theta
    theta = theta_in

    # Create a list which we will use for storing theta results during optimization
    thetas = np.array(theta_in,ndmin=2)

    # Create a counter which is used for limiting the number of iterations used if maxiterations has been defined
    k = 0

    # Begin main search loop
    while maxiterations is -1 or k < maxiterations:
        k+=1
        c.append(J(theta_in,X_in,Y_in,cost))

        temp = np.copy(theta)
        for j in range(0,n):
            # making calculations for jth parameter
            s = 0
            for i in range(0,m):
                s+=  (h(temp.T,np.ravel(X_in[i,:])) - Y_in[i])  * X_in[i,j]
            theta[j] = temp[j] - a_in * (1/m) * s
        thetas = np.vstack((thetas,theta))

        progressing = False
        for j in range(0,n):
            if(abs(temp[j] - theta[j]) > min_delta_in):
                progressing = progressing | True
        if not progressing: break
    return thetas, c
```

## Motivation: From Linear to Non-Linear Model Learning
Both Logistic Regression and Linear Regression are essentially linear models. That is, the hypotheses that they create are based on linear combinations of inputs. When we know that the structure we are looking for in data is more complicated than a linear model we can cheat in Linear and Logistic Regression by introducing extra non-linear features which can be fed into the linear classifier. This is exactly what we did when we introduced higher order features in our datasets in examples for Linear Regression and Logistic Regression.

The problem with this approach is that it does not scale. In order to achieve complex non-linear decision boundaries in a classification problem we might have to add many additional features. If we are introducing quadratic or other higher order polynomial variants this can lead to an exponential number of features.

Neural Networks allow us to build non-linear models without having to manual encode all of these additional higher order features from our base features. The network itself looks at building these higher-order features as part of its own training process. The advantage of this method is that we don't have to guess what higher order features are useful - the neural network will only create higher order features that are shown to have a beneficial effect.

The disadvantage to this neural network approach is computational cost. Neural Networks are orders of magnitude more expensive to train than a logistic regression classifier - even if that classifier has had many extra manually created features added through data expansion. From the mid 1990s to the early 2010s this high computational cost meant that neural networks were not practical problem solvers. However, faster CPUs and speed-ups made possible by GPUs have allowed Neural Networks to finally realize their promise.

## Logistic Units
Neural Networks are architectures of simple processing units called neurons which are inspired by neurons in the brain but which are really just simple computational units which calculate a function of their input. Taken individually these neurons cannot achieve much, but when integrated into a complex architecture and trained with a powerful learning algorithm, they can build complex non-linear models over our data.

In practice many different types of neurons can be used to implement a neural network. Here we will begin by focusing on one of the most important of the these, the logistic unit or logistic neuron.

A logistic unit is based on the logistic classifier as reviewed last week. In simple terms, the logistic unit makes a binary classification decision based on the application of the logistic function to the negative scalar product of an input vector times some parameters. This is formalized in our familiar function below:

\begin{equation}
h_{\theta}(x) = \frac{1}{1 + x^{-\theta x}}
\end{equation}

or

\begin{equation}
h_{\theta}(x) = \frac{1}{1 + e^{-z}}
\end{equation}

where z, the logit, is defined as:

\begin{equation}
z = \theta x
\end{equation}

In the above, $\theta$ are our model parameters, $x$ is our input data vector, and $h$ is the output of our function which is commonly referred to as the hypothesis. $z$ is the linear product of our parameters and input values, which as we have mentioned is often referred to as the **logit**.

We interpret the logistic function as follows:

\begin{equation}
h_{\theta}(x) = P(Y=1 \;|\; x_{i},\theta)
\end{equation}

meaning that we must apply a threshold to $h$ to get our classification decision.

As mentioned we can have many different types of neurons. The most significant difference between neuron types is the output or **activation function**. In the above the activation function is the logistic function, but a linear activation function can for example be used to produce a continuous real valued output from a neuron.

For example we can define and apply a simple linear classifier over two input variables. In the example below our target $Y$ indicates whether a part for a truck was found to be faulty following a vehicle inspection. Feature $x1$ indicates the number of years since the part was replaced and $x2$ indicates the distance the truck has traveled since the last inspection in kilometers.

```python
# Define the target values
Y =  [0,0,0,0,0,1,0,1,1,1,1,1,1]

# Define the 2 input features in a instance wise fashion
X = np.array([[0.3,50000],
              [3.2,100000],
              [0.4,170000],
              [0.8,120000],
              [2,150000],
              [1.2,250000],
              [1.0,160000],
              [2,210000],
              [2,300000],
              [2,400000],
              [3,300000],
              [3,340000],
              [1,350000]
             ])

# standardize our input data
X, means, stds = standardize(X)

# Define the function parameters - these were figured out using gradient descent (not shown here)
theta = [3.6,2.6,11]

# initialize values for features x1 and x2
x1_range = np.arange(0,4,0.1)
x2_range = np.arange(0.0,500000,1000.0)

x1s, x2s = np.meshgrid(x1_range, x2_range) # remember - plt.surface likes this meshgrid output
mesh_rows, mesh_cols = x1s.shape

# construct 1D arrays for x1, x2 and x0
x1 = np.ravel(x1s)
x2 = np.ravel(x2s)
x = np.vstack((x1,x2)).T

# rescale our data based on our normalization parameters
x =  (x-means)/stds

# define the new matrix X based around our standardized vector for X previously obtained
x_0 = np.array([1]*len(x2))
x = np.hstack((x_0[:, np.newaxis],x))

# perform calculations
z = h_logistic(theta,x.T)

# reshape result to have the same organization as result of meshgrid
zs = np.reshape(z,(mesh_rows,mesh_cols))
x1s = np.reshape(x[:,1],(mesh_rows,mesh_cols))
x2s = np.reshape(x[:,2],(mesh_rows,mesh_cols))

for l in range(len(Y)):
    if Y[l] == 0:
        plt.plot(X[l,0],X[l,1],'o',c='r')
    elif Y[l] == 1:
        plt.plot(X[l,0],X[l,1],'o',c='b')
    else: print(Y[l])

plt.contour(x1s, x2s, zs, 1)
plt.show()
```

A logistic neuron is simply a different architectural perspective on the logistic function. Structurally an artificial neuron has the following components:
 * A set of inputs x - these are the values that are fed into the neuron
 * A set of weights w - each weight $w_{i}$ can be interpreted as the relative strength (positive or negative) that we assign to an individual input value $x_{i}$
 * A cell body which computes the output for this neuron - the cell body can compute different types of outputs. For our purposes here we assume the output is a logistic function over the inputs x and the weights w.
 * An output which holds the computed logistic function

We can illustrate this structure with respect to our faulty truck part classifier with the figure below.

<!-- neurons.png -->

As previously described, in logistic regression we appended a vector of 1s to our input data array X in order to facilitate a vectorized implementation of linear functions. The same is accomplished in the architecture of the neuron through the addition of the **bias unit** which:
 1. is always assumed to have a value = 1
 2. Is usually referenced as the $0^{th}$ input unit to the neuron.

In the example above we draw in the bias unit and weights explicitly, but it is important to note that in drawings we usually omit the bias unit and often the values for weights. Thus the example above would more normally be drawn as shown below.

<!-- neurons2.png -->

## Combining Logistic Units
Logistic units can be combined to implement more complex functions. We can examine this by looking at the design of individual binary operations, and then the combination of these units to achieve more sophisticated operations.

### AND Units

Consider first the construction of the simple AND operator defined over two variables x1 and x2. The desired logic for an AND operation is defined as follows:

x1 | x2 | x1 AND x2
--- | --- | :---:
0 |	0 |	0
0 |	1 |	0
1 |	0 |	0
1 | 1 | 1

where x1, x2 and are result are all binary values.

It is relatively easy to find the parameters for a logistic unit which will result in computing an AND function. Such a unit (illustrated fully with bias units and weights) is illustrated below.

<!-- and.png -->

Noting that the value for $x0 = 1$ and $x1,x2 \in \{0,1\}$, we see that the parameter on the bias unit pushes the scalar product of inputs and parameters to a minimum value of -30. The only way this scalar sum of inputs and weights can go above 1 is if both the inputs x1 and x2 are 1. This is summarized in the table below:

x1 | x2 | $\theta x$ | g($\theta$ x)
--- | --- | :---: | :---:
0 |	0 |	-30 | 0
0 |	1 |	-10 | 0
1 |	0 |	-10 | 0
1 | 1 | 10  | 1

Note that the values of g are actually rounded a little bit. You can see the actual values below and tweak the parameters below.

```python
def h_log(z):
    return 1 / (1 + np.exp(-z))

def calc(x1,x2,theta0,theta1,theta2):
    x0 = 1
    z = x0*theta0 + x1*theta1 + x2*theta2
    g = h_log(z)
    return z, g

# parameters for AND approximation
theta0 = -30
theta1 = 20
theta2 = 20

z, g = calc(0,0,theta0,theta1,theta2)
print(str(z) + " " + str(g))

z, g = calc(0,1,theta0,theta1,theta2)
print(str(z) + " " + str(g))

z, g = calc(1,0,theta0,theta1,theta2)
print(str(z) + " " + str(g))

z, g = calc(1,1,theta0,theta1,theta2)
print(str(z) + " " + str(g))
```

    -30 9.357622968839299e-14
    -10 4.5397868702434395e-05
    -10 4.5397868702434395e-05
    10 0.9999546021312976

### OR Units

Similarly the OR operator can be defined over two binary variables $x1$ and $x2$. The logic for the OR operator is summarized below:

x1 | x2 | x1 OR x2
--- | --- | :---:
0 |	0 |	0
0 |	1 |	1
1 |	0 |	1
1 | 1 | 1

By choosing appropriate weights for a logistic unit we can once again achieve the desired logic as illustrated below.

<!-- or.png -->

Here the bias unit only partly pushes the scalar product of inputs and parameters below 1. In the case of either or both x1 and x2 being set to 1, the total scalar product will be greater than 0 with the result being that the logistic function has a total output 1. This can be illustrated in the table below.

x1 | x2 | $\theta x$ | g($\theta$ x)
--- | --- | :---: | :---:
0 |	0 |	-1 | 0
0 |	1 |	1  | 1
1 |	0 |	1  | 1
1 | 1 | 3  | 1

### NOT Units

The final atomic unit we will consider is the NOT unit. The NOT unit takes only one explicit input $x1$ and returns the negation of the value on that input as summarized below:

x1 | NOT x1
--- | :---:
0 |	1
1 |	0

A logistic unit which simulates the behaviour of a NOT operation when applied to binary values is illustrated below.

<!-- not.png -->

Here when x1 is set the 0, the effect of the bias unit with its weight will be to force the scalar product $\theta x$ to be greater than 0. However if $x1=1$ the strong negative value of the weight on $x2$ will lead the scalar product $\theta x$ to be less than 0. This is summarized in the table below.

x1  | $\theta x$ | g($\theta$ x)
--- | :---:      | :---:
0   |     1      |        1
1   |    -1      |        0

### XNOR Units

Given the AND, OR, and NOT units defined above we can implement more sophisticated functtions based on those units. To illustrate let us consider the XNOR operation which is defined with the following truth table:

x1 | x2 | x1 XNOR x2
--- | --- | :---:
0 |	0 |	1
0 |	1 |	0
1 |	0 |	0
1 | 1 | 1

XNOR requires both values $x1$ and $x2$ to have the same inputs. The XNOR operation may be defined in terms of our 3 fundamental operations as follows:

\begin{equation}
x1\; XNOR\; x2 = (x1\; AND\; x2)\; OR\; ((NOT\; x1)\; AND\; (NOT\; x2))
\end{equation}

By allowing the outputs of some units to act as inputs to another units we can implement the above equation with logistic units as follows:

<!-- xnor.png -->

The point of all of this is simply this. We can use logistic units to build all types of simple binary functions, and by layering them on top of each other we can build more complex functions. Constructing a XNOR unit is just a basic example. By having more and more layers of units we can build up even more complex functions if we want to -- this can be complex enough to understand words, images and the like. The example above though is handcrafted, so lets move beyond this handcrafted approach.

## The Architecture of a Neural Network

The network of logistic units used above to define the XNOR operation captures many of the principles of standard neural networks. In this section we discuss the standard basic form of a Neural Network, which is the Feedforward Network. We discuss the Feedforward Network specifically where the network is built exclusively out of logistic units. Networks built out of many other unit types are possible and we will address them later.

### Layers of the Network

The Feedforward Neural Network consists of at least three layers of artificial neurons as illustrated below.

<!-- ffnn.png -->

In the Feedforward architecture we usually talk about three distinct layer types, i.e., the input layer, the hidden layers, and the output layer. During processing an input layer is clamped to an input data vector, i.e., $x$. The output layer meanwhile captures the result of our complete model, i.e., $Y$. One or more hidden layers can be found between the input and output layers. The hidden layers encode higher order features which operate over the input data which transform the data prior to the final classification at the output layer.

Each node in layer $j$ is connected to each node in layer $j-1$ through connections which we refer to as weights. The exception to this is layer 0 which is directly clamped to input data as noted above. In practice weights are simply the parameters we assumed in linear and logistic regression.

### Multi-class Classification

In the example above we have only one neuron in the output layer but there can be many more. For example in the larger network below thee are three output units.

<!-- ffnn2 -->

This means that we can build 3 binary classifiers in parallel. This gives us our normal way of handling multi-class classification problems. For a target, e.g., animal type, with 3 classes, e.g., dog, cat and hamster, we create a binary output unit for each of those 3 types. In training the network on a multi-class classification problem we expect only one output unit to be active at a time. During classification we ideally want only one output unit to be active, but in practice most will have some activity and we select the output node that has the highest activity as our hypothesised class. This is referred to as the **One versus All** method.

### Bias Units

Each layer (with the exception of the output layer) also has a bias unit associated with the layer. We include the bias unit in the network below for illustration, and note that bias unit in layer $j$ is not connected to units in unit $j-1$. The is because the value of the bias unit is fixed to 1.

<!-- ffnn3 -->

### Deep Networks

Deep Feedforward Neural Networks are simply defined as Feedforward Netowrks that have more than one hidden layer. For example even this simple model below would be technically referred to as a Deep Network:

<!-- dnn.png -->

Although more typically our networks will in practice have many more nodes at each layer and a number of hidden layers.

<!-- dnn2.png -->

## Notational Conventions

When working through descriptions of neural network one of the most challenging problems is keeping track of the notation - particularly with respect to indexing through multiple layers of the network and multiple training and testing examples.

### Notation for Data
As with any other supervised machine learning method, our model is trained with a set of m input-output tuples:

\begin{equation}
Data_{Training} = \{ (x^{0},y^{0}),(x^{1},y^{1}),(x^{2},y^{2})\dots (x^{m-1},y^{m-1}) \}
\end{equation}

Each tuple is an individual training case which we index with a superscript.

For each individual training case we assume vectorized inputs and outputs. The length of an input vector must be equal to the number of units on the input layer (not counting the bias unit). Similarly the length of an output vector must be equal to the number of units on the output layer of the network. Thus for a network with 3 input units and 2 output units:

 * $x^{i} \in \{0,1\}^{3\times1}$
 * $y^{i} \in \{0,1\}^{2\times1}$

### Notation for Layers
A neural network is built out of a number of layers. A layer typically includes both the nodes and a set of associated weights. In indexing and describing the layers of an individual network we use the following notation:
 * $L$ - the total number of layers in the network
 * $s_{l}$ - the number of nodes the the $l^{th}$ layer of the network - excluding bias units.

In many sources our first layer is indexed as layer 1 rather than layer 0. However this only makes the programming of these networks more cumbersome. Here we will stick with standard computer science convention of indexing from 0. Hence the input layer is layer 0 and our output layer is layer $L-1$.

#### Notation for Nodes

Each layer includes a set of activation units. We rarely index the units themselves. Instead we index the values produced at those units. We refer to the output of a node as its activation value. For the first layer the activation value for a given unit is simply the corresponding input value from the data. For all other layers the activation value for a unit is its computed output. For logistic units, this is the logistic function applied to the scalar product of the values of the weights and activations of the $j-1$th layer.

We denote activation values with the character $a$. A superscript on $a$ is used to index layers. A subscript is used to index the activation values for an individual unit within that layer. To illustrate:
 * $a^{0}$ refers to the vector of activation values for our input layer. This will correspond to our input vector $x$
 * $a^{L-1}$ refers to the vector of activation values for our output layer. This will be our output vector $y$.
 * $a^{1}$ refers to the vector of activation values on our (first) hidden layer.
 * $a^{1}_{0}$ refers to the activation value of the bias unit on our (first) hidden layer. This will always be 1.
 * $a^{1}_{3}$ refers to the activation value of the 3rd (non-bias) unit in the (first) hidden layer as illustrated below.

<!-- activation.png -->

As well as indexing the activation values of the layer, it is often useful to index the partial result $\theta X$. We refer to this value as $z$ and use the same indexing conventions as with the activation values. It is worth noting though that no values for $z$ exist for the input layer as our activation values for that layer come directly from the input vector $x$ rather than through an application of the logistic function.

#### Notation for Weights

For our current purposes the weights associated with layer $j$ are the weights which follow the outputs from the nodes rather than the weights which feed into nodes. Therefore all layers in the network apart from the output layer have a set of weights. (It is worth noting as an aside here that in implementations we often reverse this notation and have weights associated with their target rather than their source layer.)

Weights are denoted with the character $\theta$. A superscript on $\theta$ is used to index layers. A subscript is used to index individual weights within a layer of weights. That subscript is based on a tuple where the first index refers to the source of the weight and the second index refers to the target.

 * $\theta^{0}$ refers to the weights emerging from the input layer
 * $\theta^{L-2}$ refers to the layer of weights feeding in to the output layer
 * $\theta^{1}$ refers to the weights emerging from the (first) hidden layer
 * $\theta^{1}_{0,1}$ refers to the weight from the bias unit in the (first) hidden layer to the first unit in the following layer
 * $\theta^{1}_{5,2}$ refers to the weight highlighted in the figure below.

<!-- weights.png -->

Note that the target index should never be equal to 0 since bias units are never updated. Note also that the output layer does not have a set of weights defined, therefore $\theta^{L-1}$ and $\theta^{L}$ are both invalid indices.

## The Feedforward Algorithm

The feedfoward algorithm is a parallelized (within a layer) and sequentialized (down through the layers) extension of applying individual activation functions. In our case below the activation functions are all logistic functions but it is important to remember that other possibilities exist.

To illustrate the use of the feedforward algorithm let us assume a network with the topology illustrated below.

<!-- example.png -->

Here:
 * L = 4
 * $S_{0} = 3$, $S_{1} = 3$, $S_{2} = 3$, $S_{3} = 1$

We also assume a single input X. Since $S_{0} = 3$ then the length of the individual input vector is also 3. We also assume a randomly initialized set of weights.

The feedforward algorithm itself can be implemented simply as a sequence of logistic function executions as illustrated in the example below:

```python
# Define the input vector to our network
X = np.array([[0.5,1.1,1.6]])
X = X.T

# Define the weights for all layers
Theta1 = np.random.rand(4,3)
Theta2 = np.random.rand(4,3)
Theta3 = np.random.rand(4,1)

# construct the activation values for layer 1 from X and a bias value
a1 = np.vstack(([1.0],X))

# calculate z for the first hidden layer
z2 = np.matmul(Theta1.T,a1)
# calculate a for the first hidden layer
a2 = np.vstack(([1.0],1 / (1 + np.exp(-z2))))

# calculate z for the second hidden layer
z3 = np.matmul(Theta2.T,a2)
# calculate a for the second hidden layer
a3 = np.vstack(([1.0],1 / (1 + np.exp(-z3))))

# calculate z for the output layer
z4 = np.matmul(Theta3.T,a3)
# calculate a for the output layer
a4 = 1 / (1 + np.exp(-z4))

print(a4)
```

    [[0.91887512]]

Note the following about the implementation above:
 * When we initialize weights we must remember that weights must also be initialized for weights coming from (but not to) bias units.
 * When we construct the activation values for a given layer we add the bias unit value to the layer at that point
 * No bias unit is added to the output layer when we compute the final set of activation functions

Given a set of appropriate parameters we can rework the code above with some arrays for storage and a loop over layers which gives us a more extensible implementation.

```python
# Define the input vector to our network
X = np.array([[0.5,1.1,1.6]])
X = X.T

# Define the number of (visible) units in each layer
S = [3,3,3,1]

# Define empty arrays for theta, activation and z values
theta = [None] * len(S)
a =  [None] * len(S)
z =  [None] * len(S)

# Initialise the weights for all layers
for idx, val in enumerate(S):
    # we don't have weights for the final layer
    if idx == (len(S)-1):
        break
    theta[idx] = np.random.rand( S[idx]+1,S[idx+1])

# Implement the Feedforward Algorithm
for idx, val in enumerate(S):
    if idx == 0:
        # construct the activation values for layer 1 from X and a bias value
        a[idx] = np.vstack(([1.0],X))
    elif idx == (len(S)-1):
        # calculate z for the output layer
        z[idx] = np.matmul(theta[idx-1].T,a[idx-1])
        # calculate a for the output layer
        a[idx] = 1 / (1 + np.exp(-z[idx]))
    else:
        # calculate z for a hidden layer
        z[idx] = np.matmul(theta[idx-1].T,a[idx-1])
        # calculate a for a hidden layer
        a[idx] = np.vstack(([1.0],1 / (1 + np.exp(-z[idx]))))
print(a4)
```

    [[0.91887512]]

## Tutorial Assignments

1. Create a Jupyter notebook called “week 04 work”.
For each of the items below complete the work it its own cell. We will refer to the contents of the cell as a “programme” just because it is a natural way of talking.  The contents of a single cell are not a programme in the normal sense of the word of course.
2. Write a programme to print out “Hello network”
Write a programme that creates a python array with the integers 0 to 9 in it, and then prints them
3. Write a programme that creates a numpy vector with the integers 0 to 9 in it and then try to print the vector
4. Write a programme to print out the cube of each element in the array created in item 3. Try doing this with a loop.
5. Write a programme to print out the cube of each element in the vector created in item 4. Try doing this with vector notation.
6. Write a function name my_fun that takes two arguments as input. The function should assume that the first item passed into the function is a vector and the second item is a scalar. The function should return a new vector which is the vector raised to the power of the scalar. Test the function with your own selection of vectors and scalars.
7. Write a function named my_fun2 that takes as input a single numpy vector and returns the sum of the squares of that vector. Test with input [1,1,1,1] and [2,2,2,2].
8. Write a function that takes as input a vector and a scalar. The function should return the sum of the square of the difference between each individual item and the scalar. Test with your own data.
9. Write a function that takes a vector as input and returns the log of each item in the vector as another vector. Test with your own data. Look in particular at the results you get for inputs such as 0, 0.5, 1, 1.5.
10. Test the function created in point 10 with a sequence of numbers 0.1, 0.2, 0.3, …. 2.0. Use matplotlib to plot the data with a line plot where the input numbers are the X axis and the returned values give the Y axis.
11. Using your own intuitions and trial and error, give parameters Theta0 and Theta1 for the line below. Theta0 is c and Theta1 is m in the traditional highschool equation y=mx+c
12. Use matplotlib to draw two distinct lines in 3D that intersect.

<!-- graph.png -->

Complete Modules/Deep Learning/week 4/Neural Network Essentials.ipynb