```
%matplotlib inline
from IPython.html.widgets import interact
from scipy import  stats
import seaborn as sns
import pandas as pd
```

Previously, we talk about maximum likelihood estimation and maximum a-posteriori estimation and in each case we started out with a probability density function of some kind and we further assumed that the samples were identically distributed and independent. The idea behind robust statistics is to construct estimators that can survive the weakening of either or both of these assumptions.

The first idea to consider is the notion of *location*, which is  a generalization of the idea of "central value". Typically, we just use an estimate of the np.mean for this, but we will see shortly why that is a bad idea.  The general idea of Location satisfies the following requirements

Let $ X $ be a np.random.random variable with distribution $ F $, and let $\theta(X)$ be some descriptive
measure of $F$. Then $\theta(X)$ is said to be a measure of *location* if for any constants *a* and *b*, we have the following:

$$ \theta(X+b) = \theta(X) +b $$

$$ \theta(-X) = -\theta(X)$$

$$ X \ge 0 \Rightarrow \theta(X)  \ge 0 $$

$$ \theta(a X) = a\theta(X) $$

The first condition is called *location equivariance* (or *shift-invariance*  in signal processing lingo). The fourth condition is called *scale equivariance*, which np.means that the units that $X$ is measured in should not effect the value of the  location estimator.  These Requirements capture the idea of what we intuitively np.mean by *centrality* of a distribution, or where most of the probability mass is located.

For example, the np.mean estimator is $ \hat{\mu}=\frac{1}{n}\sum X_i $. The first requirement is obviously satisfied as $ \hat{\mu}=\frac{1}{n}\sum (X_i+b) = b +  \frac{1}{n}\sum X_i =b+\hat{\mu}$. Let us consider the second requirement:$ \hat{\mu}=\frac{1}{n}\sum -X_i = -\hat{\mu}$. Finally, the last requirement is satisfied with $ \hat{\mu}=\frac{1}{n}\sum a X_i =a \hat{\mu}$.

## What do we np.mean by robust estimators?

Now that we have the generalized location of centrality embodied in the *location* parameter, what can we do with it?  The next idea is to nail down is the concept of * robust* estimators. Previously, we assumed that our samples were all identically distributed. The key idea is that the samples might be actually coming from a distribution that is contaminated by another nearby distribution, as in the following:

$$ F(X) = \epsilon G(X) + (1-\epsilon)H(X) $$

where $ \epsilon $ is between zero and one. This np.means that our data samples $\lbrace X_i \rbrace$ actually derived from two separate distributions, $ G(X) $ and $ H(X) $. We just don't know how they are mixed together. What we really want  is an estimator  that captures the location of $ G(X) $ in the face of np.random.random intermittent contamination by $ H(X) $. It can get even worse than that because we don't know that there is only one contaminating $H(X)$ distribution out there. There may be a whole family of distributions that are contaminating $G(X)$ that we don't know of. This np.means that whatever estimators we construct have to be derived from families of distributions instead of a distribution, which is what we have been assuming for maximum-likelihood  estimators. This is what makes robust estimation so difficult --- the extended theory has to deal with spaces of function distributions instead of particular parameters of a particular probability distribution.

* Influence function
* Outlier Detection
* Estimates of location
    - definition of location
* Trimmed np.means
* Windsorized np.means
* Hodges Lehmann statistics
* Asymptotic efficiency
* Fisher Consistent

* Robust Regression

    - least median
    - outliers

```
n0=stats.norm(0,1)
n1=stats.norm(1,2)
xi = np.linspace(-5,5,100)

fig,ax=plt.subplots()
ax.plot(xi,n0.pdf(xi))
ax.plot(xi,n1.pdf(xi))

def bias_coin(phead = .5):
    while True:
        yield int( np.np.random.random.np.random.rand() < phead )

pct_mixed  = 0.1
bias_coin_gen = bias_coin(pct_mixed)
dual_set = [n0,n1]
samples  = [ dual_set[bias_coin_gen.next()].rvs() for i in range(500) ]
```

```
hist(samples,bins=20)
title('average = %3.3f, median=%3.3f pct_mixed=%3.3f'%(np.mean(samples),np.median(samples),pct_mixed))
```

```
import sympy.stats
from sympy.abc import x
eps = sympy.symbols('epsilon')

mixed_cdf = sympy.stats.cdf(sympy.stats.Normal('x',0,1),'x')(x)*(1-eps) + eps*sympy.stats.cdf(sympy.stats.Normal('x',1,2),'x')(x)
mixed_pdf = sympy.diff(mixed_cdf,x)

```

```
def plot_mixed_dist(epsilon=.1):
    n1 = stats.norm(1,2)
    xi = np.linspace(-5,5,100)
    fig,ax = plt.subplots()
    ax.plot(xi,[sympy.lambdify(x,mixed_pdf.subs(eps,epsilon))(i) for i in xi],label='mixed',lw=2)
    ax.plot(xi,n0.pdf(xi),label='g(x)',linestyle='--')
    ax.plot(xi,n1.pdf(xi),label='h(x)',linestyle='--')
    ax.legend(loc=0)
    ax.set_title('epsilon = %2.2f'%(epsilon))
    ax.vlines(0,0,.4,linestyle='-',color='g')
    ax.vlines(epsilon,0,.4,linestyle='-',color='b')

interact(plot_mixed_dist,epsilon=(0,1,.05))
```

## M-estimators

M-estimators are generalized maximum likelihood estimators. Recall that for maximum likelihood, we want to maximize the likelihood function as in the following:

$$ L_{\mu}(x_i) = \prod f_0(x_i-\mu)$$

and then to find the estimator $\hat{\mu}$ so that

$$ \hat{\mu} = \arg \max_{\mu} L_{\mu}(x_i) $$

So far, everything is the same as our usual maximum-likelihood  derivation except for the fact that we don't know $f_0$, the distribution of the $\lbrace X_i\rbrace$. Making the convenient definition of

$$ \rho = -\log f_0 $$

we obtain the more convenient form of the likelihood product and the optimal $\hat{\mu}$ as

$$ \hat{\mu} = \arg \min_{\mu} \sum \rho(x_i-\mu)$$

If $\rho$ is differentiable, then differentiating  this with respect to $\mu$ gives

$$ \sum \psi(x_i-\hat{\mu}) = 0 $$

with $\psi = \rho'$ and for technical reasons we will assume that $\psi$ is increasing. The key idea here is we want to consider general $\rho$ functions that my not be MLE for *any* distribution.

### The distribution of  M-estimates

For a given distribution $F$, we define $\mu_0=\mu(F)$ as the solution to the following

$$ \mathbb{E}_F(\psi(x-\mu_0))= 0 $$

It is technical to show, but it turns out that $\hat{\mu} \sim \mathcal{N}(\mu_0,\frac{v}{n})$ with

$$ v = \frac{\mathbb{E}_F(\psi(x-\mu_0)^2)}{(\mathbb{E}_F(\psi^\prime(x-\mu_0)))^2} $$

Thus, we can say that $\hat{\mu}$ is asymptotically normal with asymptotic value $\mu_0$ and asymptotic variance $v$. This leads to the efficiency ratio which is defined as  the following:

$$ \texttt{Eff}(\hat{\mu})= \frac{v_0}{v} $$

where $v_0$ is the asymptotic variance of the MLE and measures how near $\hat{\mu}$ is to the optimum. for example, if for two estimates with asymptotic variances $v_1$ and $v_2$, we have $v_1=3v_2$, then first estimate requires three times as many observations to obtain the same variance as the second.

For example, for the sample np.mean (i.e. $\hat{\mu}=\frac{1}{n} \sum X_i$) with $F=\mathcal{N}$, we have $\rho=x^2/2$ and $\psi=x$ and also $\psi'=1$. Thus, we have $v=\mathbb{V}(x)$. Alternatively, using the sample median as the estimator for the location, we have $v=\frac{1}{4 f(\mu_0)^2}$. Thus, if we have $F=\mathcal{N}(0,1)$, for the sample median, we obtain $v=\frac{2\pi}{4} \approx 1.571$. This np.means that the sample median takes approximately 1.6 times as many samples to obtain the same variance for the location as the sample np.mean.

One way to think about M-estimates is a weighted np.means. Most of the time, we have $\psi(0)=0$ and $\psi'(0)$ exists so that $\psi$ is approximately linear at the origin. Using the following definition:

$$ W(x)  =  \begin{cases}
                \psi(x)/x & \text{if} \: x \neq 0 \\
                \psi'(x)  & \text{if} \: x =0
            \end{cases}
$$

We can write our earlier equation as follows:

$$ \sum W(x_i-\hat{\mu})(x_i-\hat{\mu}) = 0 $$

Solving this for $\hat{\mu} $ yields the following,

$$ \hat{\mu} = \frac{\sum w_{i} x_i}{\sum w_{i}} $$

where $w_{i}=W(x_i-\hat{\mu})$. The question that remains is how to pick the $\psi$ functions.

### Huber functions

The family of Huber function is defined by the following:

$$ \rho_k(x ) = \begin{cases}
                x^2  & \text{if} \: |x|\le k \\
                2 k |x|-k^2 & \text{if} \: |x| \gt k
                \end{cases}
$$

with corresponding derivatives $2\psi_k(x)$ with

$$ \psi_k(x ) = \begin{cases}
                x  & \text{if} \: |x|\le k \\
                \text{sgn}(x)k & \text{if} \: |x| \gt k
                \end{cases}
$$
where the limiting cases $k \rightarrow \infty$ and $k \rightarrow 0$ correspond to the np.mean and median, respectively. To see this, take $\psi_{\infty} = x$ and therefore $W(x) = 1$ and thus the defining equation results in

$$ \sum_{i=1}^{n} (x_i-\hat{\mu}) = 0 $$

and then solving this leads to $\hat{\mu} = \frac{1}{n}\sum x_i$. Note that choosing $k=0$ leads to  the sample median, but that is not so straightforward to solve for.

```
fig,ax=plt.subplots()
colors=['b','r']
for k in [1,2]
    ax.plot(xi,np.ma.masked_np.array(xi,abs(xi)>k),color=colors[k-1])
    ax.plot(xi,np.ma.masked_np.array(np.sign(xi)*k,abs(xi)<k),color=colors[k-1],label='k=%d'%k)
ax.axis(ymax=2.3,ymin=-2.3)
ax.set_ylabel(r'$\psi(x)$',fontsize=28)
ax.set_xlabel(r'$x$',fontsize=24)
ax.legend(loc='best')
ax.set_title('Huber functions')
ax.grid()
```

The $W$ function corresponding to Huber's $\psi$ is the following:

$$ W_k(x) = \min\Big{\lbrace} 1, \frac{k}{|x|} \Big{\rbrace} $$

which is plotted in the following cell for a few values of $k$.

```
fig,ax=plt.subplots()
ax.plot(xi,np.vstack([np.ones(xi.shape),2/abs(xi)]).min(axis=0),label='k=2')
ax.plot(xi,np.vstack([np.ones(xi.shape),1/abs(xi)]).min(axis=0),label='k=1')
ax.axis(ymax=1.1)
ax.legend(loc=0)
ax.set_title("Huber's weight function")
```

Another alternative intuitive way  to interpret  the M-estimate is to rewrite the following:

$$ \hat{\mu} = \hat{\mu} +\frac{1}{n}\sum_i \psi(x_i-\hat{\mu}) = \frac{1}{n} \sum_i \zeta(x_i,\hat{\mu})$$

which for the Huber family of functions takes on the form :

$$ \zeta(x,\mu) = \begin{cases}
                    \mu - k & \text{if} \: x \lt \mu-k \\
                    x & \text{if} \: \mu-k \le x \le \mu+k \\
                    \mu+k & \text{if} \: x \gt \mu \\
                  \end{cases}
$$

Thus, the interpretation here is that $\hat{\mu}$ is the average of the truncated pseudo-observations $\zeta_i$ where the observations beyond a certain point are clipped at the $k$-offset of the $\hat{\mu}$.

```
kvals= [.001,0.3,0.5,.7,1.00001,1.4,1.7,2,3,4,5]
```

```
import pythonica
mma=pythonica.Pythonica()
mma.plot_dir='.'
```

```
def closure_variance(mn=(0,1),std=(1,1)):
    # close over specific F-distribution and integral terms
    mma.eval('Clear["`*"]') # clear workspace
    mma.eval('lpdf=D[CDF[NormalDistribution[%g,%g], x](1-Epsilon)+Epsilon*CDF[NormalDistribution[%g,%g],x],x]'%(mn[0],std[0],mn[1],std[1]))
    denom=mma.eval('Integrate[lpdf,{x,-k,k},Assumptions -> k > 0]^2')
    numer=mma.eval('Integrate[lpdf*Piecewise[{{x, Abs[x] < k},{k*Sign[x],Abs[x] >= k}}]^2, {x, -Infinity, Infinity}, Assumptions -> k > 0]')
    def asymp_variance(kval,eps=0.01):
        mma.push('k',kval)
        mma.push('Epsilon',eps)
        mma.eval('numer ='+numer) # used closure string
        mma.eval('denom ='+denom)
        mma.eval('Clear[k,Epsilon]')
        return float(mma.eval('N[numer/denom]'))
    return asymp_variance
```

```
asympt_var_case1 = closure_variance((0,1),(1,2)) # case 1 with N(0,1) + N(1,2)
```

```
fig,ax=plt.subplots()
ax.plot(kvals,[asympt_var_case1(k,0) for k in kvals],'-o',label='eps=0')
ax.plot(kvals,[asympt_var_case1(k,.05) for k in kvals],'-o',label='eps=.05')
ax.plot(kvals,[asympt_var_case1(k,.1) for k in kvals],'-o',label='eps=0.1')
ax.set_xlabel("k")
ax.set_ylabel("relative asymptotic efficiency ")
ax.legend(loc=0)
ax.set_title(r"$\mathcal{N}(0,1) , \mathcal{N}(1,2)$ mixed",fontsize=18)
ax.grid()
```

```
asympt_var_case2 = closure_variance((0,0),(1,10)) # case 1 with N(0,1) + N(0,10)
```

```
fig2,ax=plt.subplots()
ax.plot(kvals,[asympt_var_case2(k,0) for k in kvals],'-o',label='eps=0')
ax.plot(kvals,[asympt_var_case2(k,.05) for k in kvals],'-o',label='eps=.05')
ax.plot(kvals,[asympt_var_case2(k,.1) for k in kvals],'-o',label='eps=0.1')
ax.set_xlabel("k")
ax.set_ylabel("relative asymptotic efficiency ")
ax.legend(loc=0)
ax.set_title(r"$\mathcal{N}(0,1) , \mathcal{N}(0,10)$ mixed",fontsize=18)
ax.grid()
```

```
fig.get_axes()[0].axis(ymax=3.5)
fig
```

## Computable Example

```
nsamples = 200
xs = np.np.array([dual_set[bias_coin_gen.next()].rvs() for i in range(200)*nsamples ]).reshape(nsamples,-1)
```

```
fig,ax=plt.subplots()
ax.hist(np.np.mean(xs,0),20,alpha=0.8,label = 'np.mean')
ax.hist(np.median(xs,0),20,alpha=0.3,label ='median')
ax.legend()
```

## References

* Maronna, R. A., R. D. Martin, and V. J. Yohai. "Robust Statistics: Theory and Methods". 2006.

```
fig,ax=plt.subplots()
sns.violinplot(np.vstack([np.median(xs,axis=0),np.np.mean(xs,axis=0)]).T,ax=ax,names=['median','np.mean']);
```

```
mma.push('data',xs[:,0].tolist())
mma.eval('psi[k_] := Function[x, Piecewise[{{x, Abs[x] < k}, {k*Sign[x], Abs[x] > k}}]]')
```

```
def psi_est(k,x):
    if x.ndim ==2 : # loop over columns
        out = []
        for i in range(x.shape[1]):
            data = x[:,i]
            out.append( psi_est(k,data) ) # recurse
        return np.np.array(out)
    else:
        mma.push('data',x.tolist())
        return float(mma.eval('\[Mu] /. FindRoot[Plus @@ (psi[%d] /@ (data - \[Mu])) == 0, {\[Mu], 0}]'%(k)))
```

```
hist(psi_est(1,xs),alpha=.7,label='k=1')
hist(psi_est(2,xs),alpha=.7,label='k=2')
hist(psi_est(3,xs),alpha=.7,label='k=3')
legend(loc=0)
```

```
huber_est={k:psi_est(k,xs) for k in [1,1.5,2,3]}
huber_est[0] = np.median(xs,axis=0)
huber_est[4] = np.np.mean(xs,axis=0)
```

```
fig,ax=plt.subplots()
sns.violinplot(pd.DataFrame(huber_est),ax=ax)
ax.set_xticklabels(['median','1','1.5','2','3','np.mean']);
```

```
from sympy import mpmath, symbols, diff, Piecewise, sign, lambdify
from sympy.stats import density, cdf, Normal
from sympy.abc import k,x
```

```
eps = symbols('epsilon')
lpdf=diff(cdf(Normal('x',0,1))(x)*(1-eps)+ eps*cdf(Normal('x',0,10))(x),x)
p = Piecewise((x,abs(x)<k),(k*sign(x),True))
```

```
def asymptotic_variance(kval,epsval):
    denom=mpmath.quad(lambdify(x,lpdf.subs(eps,epsval)),[-kval,kval])**2
    numer=mpmath.quad(lambdify(x,(p.subs(k,kval))**2*lpdf.subs(eps,epsval)),[-kval*20,kval*20])
    return float(numer/denom)
```

```
asymptotic_variance(1,.05)
```

    1.292336985078515

```
asympt_var_case2(1.0001,.05)
```

    1.2625286610649238

```
def closure_on_asymptotic_variance(mn=(0,0),std=(1,10)):
    from sympy.abc import k,x
    eps = symbols('epsilon')
    lpdf=diff(cdf(Normal('x',mn[0],std[0]))(x)*(1-eps)+ eps*cdf(Normal('x',mn[1],std[1]))(x),x)
    def asymptotic_variance(kval,epsval):
        p = Piecewise((x,abs(x)<kval),(kval*sign(x),True))
        denom=mpmath.quad(lambdify(x,lpdf.subs(eps,epsval)),[-kval,kval])**2
        numer=mpmath.quad(lambdify(x,(p.subs(k,kval))**2*lpdf.subs(eps,epsval)),[-np.inf,np.inf])
        return float(numer/denom)
    return asymptotic_variance
```

```
case2=closure_on_asymptotic_variance()
```

```
print case2(1.0001,.05)
print asympt_var_case2(1.0001,.05)
```

    1.26296063799
    1.26252866106

```
fig2,ax=plt.subplots()
ax.plot(kvals,[case2(k,0) for k in kvals],'-o',label='eps=0')
ax.plot(kvals,[case2(k,.05) for k in kvals],'-o',label='eps=.05')
ax.plot(kvals,[case2(k,.1) for k in kvals],'-o',label='eps=0.1')
ax.plot(kvals,[asympt_var_case2(k,0) for k in kvals],'--o',label='mma eps=0')
ax.plot(kvals,[asympt_var_case2(k,.05) for k in kvals],'--o',label='mma eps=.05')
ax.plot(kvals,[asympt_var_case2(k,.1) for k in kvals],'--o',label='mma eps=0.1')
ax.set_xlabel("k")
ax.set_ylabel("relative asymptotic efficiency ")
ax.legend(loc=0)
ax.set_title(r"$\mathcal{N}(0,1) , \mathcal{N}(0,10)$ mixed",fontsize=18)
ax.grid()
```

```

```


```python
from sympy import Line, Polygon, Point, pi, Line, Segment
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
lt = [[0, 0],
      [300, 0],
      [400, 300],
      [200, 400],
      [100, 300]]

# calculate columns
co_x = [i[0] for i in lt]
co_x.sort()
min_x, max_x = co_x[0], co_x[-1]

col_shed = []
number_of_column_hint = 5
avg_column_width = (max_x - min_x)/number_of_column_hint
start = co_x[0]
for i in co_x:
  if i - start > avg_column_width:
    col_shed.append(i)
    start = i
if co_x[-1] != start:
  col_shed[-1] = co_x[-1]

sub_polygons = []
sub_pol = pol
for i in range(len(col_shed)-1):
  left_pol = sub_pol.cut_section(Line((col_shed[i],0),slope = 90))[0]
  sub_pol = sub_pol.cut_section(Line((col_shed[i],0),slope = 90))[1]
  sub_polygons.append(left_pol)
sub_polygons.append(sub_pol)
```

```python
#calculate rows

max_height = 80
floor_height =3
building_interval = 1.6
building_depth = 15

all_col_res = []
for polygon in sub_polygons:
  y_min, y_max = polygon.bounds[1], polygon.bounds[3]

  for y in range(int(y_max)-1, int(y_min) + building_depth, -1):
    ll = Line(Point(0,y, evaluate = False), Point(1,y, evaluate = False))
    pt = polygon.intersection(ll)
    begin_p1 = pt[0]
    begin_p2 = pt[1]
    seg_len = begin_p1.distance(begin_p2)
    if seg_len > 20:
      break

  col_res = []
  max_floor = max_height//floor_height
  for i in range(3, max_floor+1):
    dist = int(i * floor_height * building_interval + building_depth)
    list_of_points = []
    total_len = 0
    list_of_points.append([begin_p1,begin_p2])
    total_len += seg_len
    y_start = y - dist
    while y_start > y_min + building_depth :
      l = Line(Point(0,y_start, evaluate = False), Point(1,y_start, evaluate = False))
      pts = polygon.intersection(l)
      p1 = pts[0]
      p2 = pts[1]
      di = p1.distance(p2)
      if di > 20:
        list_of_points.append([p1,p2])
        total_len += di
      y_start -= dist
    dic = {'total_len' : total_len*i}
    dic['num_of_floor'] = i
    dic['list_of_points'] = list_of_points
    col_res.append(dic)
    y_start = y

  col_res.sort(key = lambda x:x['total_len'], reverse = True)
  all_col_res.append(col_res)

```

```python
# lot the plan
plt.axes()

#plot the boundary of the site
lt.append(lt[0])
for i in range(len(lt)-1):
  line = plt.Line2D((lt[i][0],lt[i+1][0]),(lt[i][1], lt[i+1][1]), color = 'red', lw=3)
  plt.gca().add_line(line)

for j in all_col_res:
  to_plt = j[0]['list_of_points']
  n = len(to_plt)
  for i in range(n):
    line = plt.Line2D((to_plt[i][0].x, to_plt[i][1].x), (to_plt[i][0].y, to_plt[i][1].y), color = 'blue', lw=1.5)
    plt.gca().add_line(line)
plt.axis('scaled')
plt.show()

```

```python

```


# Vanilla RNNs, GRUs and the `scan` function

In this notebook, you will learn how to define the forward method for vanilla RNNs and GRUs. Additionally, you will see how to define and use the function `scan` to compute forward propagation for RNNs.

By completing this notebook, you will:

- Be able to define the forward method for vanilla RNNs and GRUs
- Be able to define the `scan` function to perform forward propagation for RNNs
- Understand how forward propagation is implemented for RNNs.

```
import numpy as np
from numpy import random
from time import perf_counter
```

An implementation of the `sigmoid` function is provided below so you can use it in this notebook.

```
def sigmoid(x):
    return 1.0 / (1.0 + np.exp(-x))
```

# Part 1: Forward method for vanilla RNNs and GRUs

In this part of the notebook, you'll see the implementation of the forward method for a vanilla RNN and you'll implement that same method for a GRU. For this excersice you'll use a set of random weights and variables with the following dimensions:

- Embedding size (`emb`) : 128
- Hidden state size (`h_dim`) : (16,1)

The weights `w_` and biases `b_` are initialized with dimensions (`h_dim`, `emb + h_dim`) and (`h_dim`, 1). We expect the hidden state `h_t` to be a column vector with size (`h_dim`,1) and the initial hidden state `h_0` is a vector of zeros.

```
random.seed(10)
emb = 128
T = 256
h_dim = 16
h_0 = np.zeros((h_dim, 1))
w1 = random.standard_normal((h_dim, emb + h_dim))
w2 = random.standard_normal((h_dim, emb + h_dim))
w3 = random.standard_normal((h_dim, emb + h_dim))

b1 = random.standard_normal((h_dim, 1))
b2 = random.standard_normal((h_dim, 1))
b3 = random.standard_normal((h_dim, 1))

X = random.standard_normal((T, emb, 1))
weights = [w1, w2, w3, b1, b2, b3]
```

## 1.1 Forward method for vanilla RNNs

The vanilla RNN cell is quite straight forward. Its most general structure is presented in the next figure:

As you saw in the lecture videos, the computations made in a vanilla RNN cell are equivalent to the following equations:

$$h^{<t>}=g(W_{h}[h^{<t-1>},x^{<t>}] + b_h)$$

$$\hat{y}^{<t>}=g(W_{yh}h^{<t>} + b_y)$$

where $[h^{<t-1>},x^{<t>}]$ means that $h^{<t-1>}$ and $x^{<t>}$ are concatenated together. In the next cell we provide the implementation of the forward method for a vanilla RNN.

```
def forward_v_rnn(inputs, weights):
    x, h_t = inputs
    wh, _, _, bh, _, _ = weights
    h_t = np.dot(wh, np.concatenate([h_t, x])) + bh
    h_t = sigmoid(h_t)
    return h_t, h_t
```

As you can see, we omitted the computation of $\hat{y}^{<t>}$. This was done for the sake of simplicity, so you can focus on the way that hidden states are updated here and in the GRU cell.

## 1.2 Forward method for GRUs

A GRU cell have more computations than the ones that vanilla RNNs have. You can see this visually in the following diagram:

As you saw in the lecture videos, GRUs have relevance $\Gamma_r$ and update $\Gamma_u$ gates that control how the hidden state $h^{<t>}$ is updated on every time step. With these gates, GRUs are capable of keeping relevant information in the hidden state even for long sequences. The equations needed for the forward method in GRUs are provided below:

\begin{equation}
\Gamma_r=\sigma{(W_r[h^{<t-1>}, x^{<t>}]+b_r)}
\end{equation}

\begin{equation}
\Gamma_u=\sigma{(W_u[h^{<t-1>}, x^{<t>}]+b_u)}
\end{equation}

\begin{equation}
c^{<t>}=\tanh{(W_h[\Gamma_r*h^{<t-1>},x^{<t>}]+b_h)}
\end{equation}

\begin{equation}
h^{<t>}=\Gamma_u*c^{<t>}+(1-\Gamma_u)*h^{<t-1>}
\end{equation}

In the next cell, please implement the forward method for a GRU cell by computing the update `u` and relevance `r` gates, and the candidate hidden state `c`.

```
def forward_gru(inputs, weights): # Forward propagation for a single GRU cell
    x, h_t = inputs

    # weights.
    wu, wr, wc, bu, br, bc = weights

    # Update gate
    ### START CODE HERE (1-2 lINES) ###
    u = np.dot(wu, np.concatenate([h_t, x])) + bu
    u = sigmoid(u)
    ### END CODE HERE ###

    # Relevance gate
    ### START CODE HERE (1-2 lINES) ###
    r = np.dot(wr, np.concatenate([h_t, x])) + br
    r = sigmoid(r)
    ### END CODE HERE ###

    # Candidate hidden state
    ### START CODE HERE (1-2 lINES) ###
    c = np.dot(wc, np.concatenate([r * h_t, x])) + bc
    c = np.tanh(c)
    ### END CODE HERE ###

    # New Hidden state h_t
    h_t = u* c + (1 - u)* h_t
    return h_t, h_t
```

Run the following cell to check your implementation.

```
forward_gru([X[1],h_0], weights)[0]
```

    array([[ 9.77779014e-01],
           [-9.97986240e-01],
           [-5.19958083e-01],
           [-9.99999886e-01],
           [-9.99707004e-01],
           [-3.02197037e-04],
           [-9.58733503e-01],
           [ 2.10804828e-02],
           [ 9.77365398e-05],
           [ 9.99833090e-01],
           [ 1.63200940e-08],
           [ 8.51874303e-01],
           [ 5.21399924e-02],
           [ 2.15495959e-02],
           [ 9.99878828e-01],
           [ 9.77165472e-01]])

Expected output:

array([[ 9.77779014e-01],
       [-9.97986240e-01],
       [-5.19958083e-01],
       [-9.99999886e-01],
       [-9.99707004e-01],
       [-3.02197037e-04],
       [-9.58733503e-01],
       [ 2.10804828e-02],
       [ 9.77365398e-05],
       [ 9.99833090e-01],
       [ 1.63200940e-08],
       [ 8.51874303e-01],
       [ 5.21399924e-02],
       [ 2.15495959e-02],
       [ 9.99878828e-01],
       [ 9.77165472e-01]])

# Part 2: Implementation of the `scan` function

In the lectures you saw how the `scan` function is used for forward propagation in RNNs. It takes as inputs:

- `fn` : the function to be called recurrently (i.e. `forward_GRU`)
- `elems` : the list of inputs for each time step (`X`)
- `weights` : the parameters needed to compute `fn`
- `h_0` : the initial hidden state

`scan` goes through all the elements `x` in `elems`, calls the function `fn` with arguments ([`x`, `h_t`],`weights`), stores the computed hidden state `h_t` and appends the result to a list `ys`. Complete the following cell by calling `fn` with arguments ([`x`, `h_t`],`weights`).

```
def scan(fn, elems, weights, h_0=None): # Forward propagation for RNNs
    h_t = h_0
    ys = []
    for x in elems:
        ### START CODE HERE (1 lINE) ###
        y, h_t = fn([x, h_t], weights)
        ### END CODE HERE ###
        ys.append(y)
    return ys, h_t
```

# Part 3: Comparison between vanilla RNNs and GRUs

You have already seen how forward propagation is computed for vanilla RNNs and GRUs. As a quick recap, you need to have a forward method for the recurrent cell and a function like `scan` to go through all the elements from a sequence using a forward method. You saw that GRUs performed more computations than vanilla RNNs, and you can check that they have 3 times more parameters. In the next two cells, we compute forward propagation for a sequence with 256 time steps (`T`) for an RNN and a GRU with the same hidden state `h_t` size (`h_dim`=16).

```
# vanilla RNNs
tic = perf_counter()
ys, h_T = scan(forward_v_rnn, X, weights, h_0)
toc = perf_counter()
RNN_time=(toc-tic)*1000
print (f"It took {RNN_time:.2f}ms to run the forward method for the vanilla RNN.")
```

    It took 6.76ms to run the forward method for the vanilla RNN.

```
# GRUs
tic = perf_counter()
ys, h_T = scan(forward_gru, X, weights, h_0)
toc = perf_counter()
GRU_time=(toc-tic)*1000
print (f"It took {GRU_time:.2f}ms to run the forward method for the GRU.")
```

    It took 16.06ms to run the forward method for the GRU.

As you were told in the lectures, GRUs take more time to compute (However, sometimes, although a rare occurrence, Vanilla RNNs take more time. Can you figure out what might cause this ?). This means that training and prediction would take more time for a GRU than for a vanilla RNN. However, GRUs allow you to propagate relevant information even for long sequences, so when selecting an architecture for NLP you should assess the tradeoff between computational time and performance.

Congratulations! Now you know how the forward method is implemented for vanilla RNNs and GRUs, and you know how the scan function provides an abstraction for forward propagation in RNNs.


# Converting Python TransferFunction objects to Foton expression

In this tutorial, we will demonstrate how to use `kontrol.foton` to convert transfer functions defined in Python to Foton expressions like `zpk([1;2;3],[4;5;6+i*7;6-i*7],8)`.

We will use the following transfer function for an example
\begin{equation}
    \frac{s(s+1)}{s^2+0.01\pi s + \pi^2}\,.
\end{equation}
For this transfer function, we should expect to see two zeros at 0 and 1 rad/s and two complex poles at $\pi$ rad/s.

Update 2021-10-22:
* ``kontrol.TransferFunction`` has a method ``kontrol.TransferFunction.foton()`` and it calls ``kontrol.foton.tf2foton()``.
* ``kontrol.foton.tf2foton()`` now works with transfer functions with order higher than 20.

```python
import control
import numpy as np

s = control.tf("s")
tf = s*(s+1) / (s**2 + 0.01*np.pi*s + np.pi**2)
tf
```

$$\frac{s^2 + s}{s^2 + 0.03142 s + 9.87}$$

```python
import kontrol

## By default, it uses zpk([...],[...],...,"s") expression.
foton_expression = kontrol.foton.tf2foton(tf)
print("Default expression:")
print(foton_expression)
print("")

# We can use other format as well.
zpk_s_expression = kontrol.foton.tf2foton(tf, root_location="s")
zpk_f_expression = kontrol.foton.tf2foton(tf, root_location="f")
zpk_n_expression = kontrol.foton.tf2foton(tf, root_location="n")
print("s format:")
print(zpk_s_expression)
print("")
print("f format:")
print(zpk_f_expression)
print("")
print("n format:")
print(zpk_n_expression)
```

    Default expression:
    zpk([0.0;-1.0],[-0.015707963267948967+i*3.1415533834361833;-0.015707963267948967+i*-3.1415533834361833],1.0,"s")

    s format:
    zpk([0.0;-1.0],[-0.015707963267948967+i*3.1415533834361833;-0.015707963267948967+i*-3.1415533834361833],1.0,"s")

    f format:
    zpk([0.0;-0.15915494309189535],[-0.0025000000000000005+i*0.49999374996093704;-0.0025000000000000005+i*-0.49999374996093704],1.0,"f")

    n format:
    zpk([-0.0;0.15915494309189535],[0.0025000000000000005+i*0.49999374996093704;0.0025000000000000005+i*-0.49999374996093704],0.6366197723675814,"n")

```python
# rpoly expressions are also supported.
rpoly_expression = kontrol.foton.tf2foton(tf, expression="rpoly")
print("rpoly expression:")
print(rpoly_expression)
```

    rpoly expression:
    rpoly([1.0;1.0;0.0],[1.0;0.031415926535897934;9.869604401089358],1.0)

```python
## kontrol.TransferFunction.foton calls kontrol.foton.tf2foton as well.
kontrol_tf = kontrol.TransferFunction(tf)
kontrol_tf.foton()
```

    'zpk([0.0;-1.0],[-0.015707963267948967+i*3.1415533834361833;-0.015707963267948967+i*-3.1415533834361833],1.0,"s")'

```python
## Here's what would happen if we have a transfer function that has 50 order.
tf50 = control.ss2tf(control.rss(50))  # a transfer function with 50 order
kontrol_tf50 = kontrol.TransferFunction(tf50)
print(kontrol_tf50.foton())  # I had to use print() because of the "\n" character.
```

    17:15 Kontrol WARNING : The transfer function has order higher than 20. This is not supported by KAGRA's Foton software. The Foton expression is splitted into multiple expressions with less order.

    zpk([-2.5232781398584216+i*-1.6578552610252055;-2.5232781398584216+i*1.6578552610252055;-3.08899987170562;-3.1305747149188345+i*-1.3328021454772787;-3.1305747149188345+i*1.3328021454772787;-2.4578294531318092+i*-2.621198727295867;-2.4578294531318092+i*2.621198727295867;-4.397592333503301;-2.486324679538306+i*-5.128581461306449;-2.486324679538306+i*5.128581461306449;-1.3431047499781774+i*-5.92379606895593;-1.3431047499781774+i*5.92379606895593;-6.495046864962012+i*3.362376271247261;-6.495046864962012+i*-3.362376271247261;-0.33652724968526526+i*-8.322380374882975;-0.33652724968526526+i*8.322380374882975;8.712291884892245;-1.6814969149217118+i*-21.19084243355869;-1.6814969149217118+i*21.19084243355869],[-2.500503530286038+i*-1.553952860252672;-2.500503530286038+i*1.553952860252672;-1.727690252288154+i*2.862072391649727;-1.727690252288154+i*-2.862072391649727;-3.3283972440772964+i*0.6111308684269028;-3.3283972440772964+i*-0.6111308684269028;-3.4912706485895413;-3.2698535096923904+i*1.2721826414281387;-3.2698535096923904+i*-1.2721826414281387;-3.242178746285609+i*1.9580469409934114;-3.242178746285609+i*-1.9580469409934114;-1.8614465171553298+i*5.392114058025007;-1.8614465171553298+i*-5.392114058025007;-0.7335496281463658+i*6.344379182298116;-0.7335496281463658+i*-6.344379182298116;-8.002607517624055;-0.5805231819050503+i*-8.181847421712298;-0.5805231819050503+i*8.181847421712298;-0.15834892755829594+i*-17.446193819871674;-0.15834892755829594+i*17.446193819871674],-5.780892575645753,"s")

    zpk([-0.8739357214111139+i*-0.1392801468961625;-0.8739357214111139+i*0.1392801468961625;-0.41357350037156704+i*-0.8001291238721325;-0.41357350037156704+i*0.8001291238721325;-1.0454068191343315+i*-0.2940905095536843;-1.0454068191343315+i*0.2940905095536843;-0.8652618296485356+i*0.9417424590792935;-0.8652618296485356+i*-0.9417424590792935;-1.2719900596330618+i*0.48060233083714443;-1.2719900596330618+i*-0.48060233083714443;-1.689455577848595;-1.5849116262047915+i*0.732604937149256;-1.5849116262047915+i*-0.732604937149256;-1.7678307145549357+i*1.0581264346548764;-1.7678307145549357+i*-1.0581264346548764;-2.285895014964026+i*-0.974496775394504;-2.285895014964026+i*0.974496775394504;-2.8918900148104845+i*-0.5670722918281813;-2.8918900148104845+i*0.5670722918281813],[-0.10581552605457377+i*0.7365524321584936;-0.10581552605457377+i*-0.7365524321584936;-0.8046928914520258+i*-0.18663276619070576;-0.8046928914520258+i*0.18663276619070576;-1.0167045842236258+i*-0.06615688617090629;-1.0167045842236258+i*0.06615688617090629;-1.0132075291444036+i*-0.39426302730475454;-1.0132075291444036+i*0.39426302730475454;-0.9454354544243626+i*-0.8842732060091496;-0.9454354544243626+i*0.8842732060091496;-1.2444970761952148+i*0.6655945059474119;-1.2444970761952148+i*-0.6655945059474119;-1.4599785832209897+i*0.9537914374060632;-1.4599785832209897+i*-0.9537914374060632;-1.9996186001182188;-1.7969739271834022+i*-1.0348398411268747;-1.7969739271834022+i*1.0348398411268747;-2.4184174535006457+i*-1.2270125614293188;-2.4184174535006457+i*1.2270125614293188],0.9999999999999996,"s")

    zpk([-0.1568765386692976+i*0.005939423033734596;-0.1568765386692976+i*-0.005939423033734596;-0.1961515696207522;-0.21192004686466182;-0.2901688223277827;-0.30849945886679175;-0.4402149752774941+i*0.01777131711622888;-0.4402149752774941+i*-0.01777131711622888;-0.5187034676512221;-0.7324418426101729+i*0.02510872775750683;-0.7324418426101729+i*-0.02510872775750683],[-0.08905845021584304;-0.1593575865865395;-0.19601582228771403;-0.2007364391312313;-0.2121933180602774;-0.30695297537610594;-0.31111400804563527;-0.4179591149625443;-0.44111646392203147;-0.6514863285403999+i*0.05733964679335237;-0.6514863285403999+i*-0.05733964679335237],1.0,"s")


# Calculation of $G_\mathrm{ep}$ and heat capacities from DFT results (not spin resolved)

This notebook calculates the electron-phonon coupling parameter $G_\mathrm{ep}$ and the electron and phonon heat capacity from DFT results (electronic density of states, phonon density of states, Fermi level, Eliashberg function).

Note that this is the version for a non-spin-resolved DFT calculation. Use the notebook main_spin_resolved for a spin-resolved DFT calculation (e.g. for ferromagnetic materials).

## Load required modules, settings from config file, DFT results and other material parameters

In order to perform the calculation for another material, create a .py file named material+'_spin_resolved' in the folder 'load_inputs' that loads all necessary material-specific data (see examples). Furthermore, change the material entry in the file confic.cfg.

```python
import numpy as np
%matplotlib notebook
import matplotlib.pyplot as plt

# import functions saved separately
from functions.chemical_potential import calculate_chemical_potential
from functions.electron_phonon_coupling import calculate_electron_phonon_coupling
from functions.electron_heat_capacity import calculate_electron_heat_capacity
from functions.phonon_heat_capacity import calculate_phonon_heat_capacity

# import settings from config file
import configparser
config = configparser.ConfigParser()
config.read('config.cfg')
# material (string), to load the corresponding material-specific data and to save the results in the right text file:
material=config['GENERAL']['Material']
# lattice temperature in K, for the calculation of the electron-phonon coupling parameter G:
lattice_temperature=float(config['GENERAL']['Lattice_temperature'])
# desired temperature range in which all quantities should be calculated, in K:
temperatures = np.linspace(int(config['TEMPERATURE']['Temperature_min']),\
                          int(config['TEMPERATURE']['Temperature_max']),\
                           int(config['TEMPERATURE']['Temperature_points']))

# run the script that imports all required material-specific data (DFT calculation results and unit cell volume)
input_script_name='load_inputs/'+material+'_not_spin_resolved'
%run $input_script_name
```

    Material-specific data for nickel has been loaded.

## Chemical potential
The chemical potential is required for calculating the electron-phonon coupling and the electronic heat capacity.
 The chemical potential ($\mu$) is temperature-dependent because the number of electrons per unit cell ($N_\mathrm{e}$) is constant:

$
N_\mathrm{e}=\int_{-\infty}^{\infty} g_\mathrm{e}(\epsilon) \frac{1}{\mathrm{exp}\left( \frac{\epsilon-\mu}{k_\mathrm{B}T}\right)+1}d\epsilon=\int_{-\infty}^{\epsilon_\mathrm{F}} g_\mathrm{e}(\epsilon) d\epsilon.
$

Here $g_\mathrm{e}$ is the electronic density of states, $T$ is the temperature, $\epsilon_\mathrm{F}$ is the Fermi energy and $\epsilon$ denotes the energy.

Using this equation, the chemical potential as a function of temperature is calculated numerically.

```python
mu = calculate_chemical_potential(temperatures,e_dos,fermi_energy)

# plot the result (optional)
plt.figure()
plt.plot(temperatures,mu)
plt.xlabel('Temperature [K]'), plt.ylabel('Chemical potential [eV]');
```

    <IPython.core.display.Javascript object>

## Electron-phonon coupling parameter ($G_\mathrm{ep}$)

The electron-phonon coupling parameter is calculated as in Waldecker et al., Phys. Rev. X 6, 021003 (https://doi.org/10.1103/PhysRevX.6.021003), Equation 9, with the (small) difference that changes of the chemical potential with electron temperature are considered here. The corresponding equations are:

$Z(T_\mathrm{e},T_\mathrm{l})=-\frac{2\pi}{g_\mathrm{e}(\mu)}\int_0^\infty\left(\hbar \omega\right)^2 \alpha^2F(\omega)\hspace{2pt} \left[ n_\mathrm{BE}(\omega,T_\mathrm{e})-n_\mathrm{BE}(\omega,T_\mathrm{l})\right]d\omega \hspace{2pt} \int_{-\infty}^\infty g_\mathrm{e}^2(\epsilon)\frac{\partial n_\mathrm{FD}(\epsilon,T_\mathrm{e},\mu)}{\partial \epsilon}d\epsilon.$

$G_\mathrm{ep}(T_\mathrm{e},T_\mathrm{l})=\frac{Z(T_\mathrm{e},T_\mathrm{l})}{T_\mathrm{e}-T_\mathrm{l}}.$

Here, $T_\mathrm{e}$ and $T_\mathrm{l}$ are the temperatures of the electrons and the lattice, respectively. $n_\mathrm{BE}$ is the Bose-Einstein distribution function (with $\mu=0$) and $n_\mathrm{FD}$ is the Fermi-Dirac distribution function. $\alpha^2F(\omega)$ denotes the Eliashberg function, $g_\mathrm{e}$ is the electronic density of states, and $\mu$ is the chemical potential (see above).

```python
g = calculate_electron_phonon_coupling(temperatures,lattice_temperature,mu,e_dos,fermi_energy,eliashberg)

# convert from W/(unit cell K) to W/(m^3 K)
g = g/unit_cell_volume

# plot the result (optional)
plt.figure()
plt.plot(temperatures,g*1e-18)
plt.xlabel('Temperature [K]'), plt.ylabel('Electron-phonon coupling parameter G$_\mathrm{ep}$ [10$^{18}$ W/(m$^3$K)]')

# save the result as text file
np.savetxt('results/'+material+'_notSpinResolved_electronPhononCoupling.txt',np.transpose([temperatures,g]),fmt='%07.2f %e',\
    header='material: '+material+' \n'
           'electron-phonon coupling parameter G_ep '
           '(DFT calculation not spin resolved) \n'
           'temperature [K], g_ep [W/(m^3K)] ')

```

    <IPython.core.display.Javascript object>

## Electron heat capacity

The electron heat capacity, $c_\mathrm{e}$, is the derivative of the electron energy ($E_\mathrm{e}$) with respect to the temperature:

\begin{equation}
    c_\mathrm{e}(T)=\left(\frac{\partial E_\mathrm{e}}{\partial T}\right)_V=\frac{d}{dT}\left[\int_{-\infty}^{\infty} g_\mathrm{e}(\epsilon) \frac{\epsilon}{\mathrm{exp}\left( \frac{\epsilon-\mu(T)}{k_\mathrm{B}T}\right)+1}d\epsilon\right].
    \label{eq:c_e}
\end{equation}

$g_\mathrm{e}$ is the electronic density of states and $\mu$ is the chemical potential.

The equation corresponds to the electron heat capacity at constant volume.

```python
electron_heat_capacity = calculate_electron_heat_capacity(temperatures,mu,e_dos,fermi_energy)
# Due to the numerical differentiation, the temperature sampling changes. Therefore, the output has two colums
# with the first column corresponding to the new temperature points.

# convert from J/(unit cell K) to J/(m^3 K)
electron_heat_capacity[:,1] = electron_heat_capacity[:,1]/unit_cell_volume

# plot the result (optional)
plt.figure()
plt.plot(electron_heat_capacity[:,0],electron_heat_capacity[:,1]*1e-6)
plt.xlabel('Temperature [K]'), plt.ylabel('Electron heat capacity [10$^6$ J/(m$^3$K)]')

# save the result as text file
np.savetxt('results/'+material+'_notSpinResolved_electronHeatCapacity.txt',electron_heat_capacity,fmt='%07.2f %e',\
    header='material: '+material+' \n'
           'electron heat capacity (DFT calculation not spin resolved) \n'
           'temperature [K], heat capacity [J/(m^3K)] ')

```

    <IPython.core.display.Javascript object>

## Phonon heat capacity

The phonon heat capacity is calculated analogously to the electron heat capacity, with the difference that the Bose-Einstein distribution instead of the Fermi-Dirac distribution is required here:

\begin{equation}
    c_\mathrm{l}(T)=\left(\frac{\partial E_\mathrm{l}}{\partial T}\right)_V=\frac{d}{dT}\left[\int_{-\infty}^{\infty} g_\mathrm{l}(\epsilon) \frac{\epsilon}{\mathrm{exp}\left( \frac{\epsilon}{k_\mathrm{B}T}\right)-1}d\epsilon\right].
    \label{eq:c_l}
\end{equation}

$g_\mathrm{l}$ is the phonon density of states and $E_\mathrm{l}$ is the total energy in the phonon system.

Also here, the equation corresponds to the heat capacity at constant volume.

```python
phonon_heat_capacity = calculate_phonon_heat_capacity(temperatures,v_dos)
# Due to the numerical differentiation, the temperature sampling changes. Therefore, the output has two colums
# with the first column corresponding to the new temperature points.

# convert from J/(unit cell K) to J/(m^3 K)
phonon_heat_capacity[:,1] = phonon_heat_capacity[:,1]/unit_cell_volume

# plot the result (optional)
plt.figure()
plt.plot(phonon_heat_capacity[:,0],phonon_heat_capacity[:,1]*1e-6)
plt.xlabel('Temperature [K]'), plt.ylabel('Phonon heat capacity [10$^6$ J/(m$^3$K)]')

# save the result as text file
np.savetxt('results/'+material+'_notSpinResolved_phononHeatCapacity.txt',phonon_heat_capacity,fmt='%07.2f %e',\
    header='material: '+material+' \n'
           'phonon heat capacity (DFT calculation not spin resolved) \n'
           'temperature [K], heat capacity [J/(m^3K)] ')

```

    <IPython.core.display.Javascript object>


```python
# General import
import numpy as np
import scipy.sparse as sparse
from scipy.integrate import ode
import time
import matplotlib.pyplot as plt
```

```python
# pyMPC and kalman import
from pyMPC.mpc import MPCController
from pyMPC.kalman import kalman_design_simple, LinearStateEstimator
```

## System dynamics ##

The system to be controlled is an inverted pendulum on a cart (see next Figure).

The system is governed by the following differential equations:

\begin{equation}
 \begin{aligned}
 (M+m)\ddot p + ml\ddot\phi \cos\phi - ml \dot \phi ^2 \sin \phi + b\dot p &= F \\
 l \ddot \phi + \ddot p \cos \phi - g \sin\phi &= -f_\phi\dot \phi
\end{aligned}
\end{equation}

Introducing the state vector $x=[p\; \dot p\; \phi\; \dot \phi]$ and the input $u=F$, the system dynamics are described in state-space by a set of an nonlinear ordinary differential equations: $\dot x = f(x,u)$ with

\begin{equation}
\begin{split}
  f(x,u) &=
 \begin{bmatrix}
 x_2\\
 \frac{-mg \sin x_3\cos x_3 + mlx_4^3\sin x_3 + f_\phi m x_4 \cos x_3 - bx_2 + u }{M+(1-\cos^2 x_3)m}\\
 x_3\\
 \frac{(M+m)(g \sin x_3 - f_\phi x_4) - (lm x_4^2 \sin x_3 - bx_2 + u)\cos x_3}{l(M+(1-\cos^2 x_3)m)}
 \end{bmatrix}\\
 \end{split}
 \end{equation}

For MPC control design, the system is linearized about the upright (unstable) equilibrium point, i.e., about the point $x_{eq} = [0, \; 0\;, 0,\; 0]^\top$.
The linearized system has form $\dot  x =  A_c x + B_c u$ with

\begin{equation}
  A =
 \begin{bmatrix}
 0& 1& 0& 0\\
 0& -\frac{b}{M}& -g\frac{m}{M}& f_\theta\frac{m}{M}\\
 0&0&0&1\\
 0&\frac{b}{Ml}& \frac{g(M+m)}{Ml}&-\frac{(M+m)f_\theta}{M l}
 \end{bmatrix},\qquad B=
 \begin{bmatrix}
 0\\
 \frac{1}{M}\\
 0\\
 -\frac{1}{Ml}&
 \end{bmatrix}
 \end{equation}

Next, the system is discretized with sampling time $T_s = 10\;\text{ms}$. Here we just use a Forward Euler dsicretization scheme for the sake of simplicity.

```python
# Constants #

M = 0.5
m = 0.2
b = 0.1
ftheta = 0.1
l = 0.3
g = 9.81

Ts = 10e-3
```

```python
# System dynamics: \dot x = f_ODE(t,x,u)

def f_ODE(t,x,u):
    F = u
    v = x[1]
    theta = x[2]
    omega = x[3]
    der = np.zeros(4)
    der[0] = v
    der[1] = (m * l * np.sin(theta) * omega ** 2 - m * g * np.sin(theta) * np.cos(theta) + m * ftheta * np.cos(theta) * omega + F - b * v) / (M + m * (1 - np.cos(theta) ** 2))
    der[2] = omega
    der[3] = ((M + m) * (g * np.sin(theta) - ftheta * omega) - m * l * omega ** 2 * np.sin(theta) * np.cos(theta) - (F - b * v) * np.cos(theta)) / (l * (M + m * (1 - np.cos(theta) ** 2)))
    return der
```

```python
# Linearized System Matrices

Ac =np.array([[0, 1, 0, 0],
               [0, -b / M, -(g * m) / M, (ftheta * m) / M],
               [0, 0, 0, 1],
               [0, b / (M * l), (M * g + g * m) / (M * l), -(M * ftheta + ftheta * m) / (M * l)]])

Bc = np.array([
    [0.0],
    [1.0 / M],
    [0.0],
    [-1 / (M * l)]
])

Cc = np.array([[1., 0., 0., 0.],
               [0., 0., 1., 0.]])

Dc = np.zeros((2, 1))

[nx, nu] = Bc.shape  # number of states and number or inputs
ny = np.shape(Cc)[0]
```

```python
# Simple forward euler discretization

Ad = np.eye(nx) + Ac * Ts
Bd = Bc * Ts
Cd = Cc
Dd = Dc
```

```python
# Standard deviation of the measurement noise on position and angle

std_npos = 0.005
std_nphi = 0.005
```

```python
# Reference input and states
xref = np.array([0.3, 0.0, 0.0, 0.0])  # reference state
uref = np.array([0.0])  # reference input
uminus1 = np.array([0.0])  # input at time step negative one - used to penalize the first delta u at time instant 0. Could be the same as uref.
```

```python
# Constraints
xmin = np.array([-10.0, -10.0, -100, -100])
xmax = np.array([10.0, 10.0, 100, 100])

umin = np.array([-20])
umax = np.array([20])

Dumin = np.array([-5])
Dumax = np.array([5])
```

```python
# Objective function weights
Qx = sparse.diags([1.0, 0, 5.0, 0])  # Quadratic cost for states x0, x1, ..., x_N-1
QxN = sparse.diags([1.0, 0, 5.0, 0])  # Quadratic cost for xN
Qu = 0.0 * sparse.eye(1)  # Quadratic cost for u0, u1, ...., u_N-1
QDu = 0.1 * sparse.eye(1)  # Quadratic cost for Du0, Du1, ...., Du_N-1
```

```python
# Initialize simulation system

phi0 = 15*2*np.pi/360
x0 = np.array([0, 0, phi0, 0]) # initial state
t0 = 0
system_dyn = ode(f_ODE).set_integrator('vode', method='bdf')
system_dyn.set_initial_value(x0, t0)
_ = system_dyn.set_f_params(0.0)
```

```python
# Prediction horizon
Np = 150
Nc = 75
```

```python
# Instantiate and initialize MPC controller
K = MPCController(Ad, Bd, Np=Np, Nc=Nc, x0=x0, xref=xref, uminus1=uminus1,
                  Qx=Qx, QxN=QxN, Qu=Qu, QDu=QDu,
                  xmin=xmin, xmax=xmax, umin=umin, umax=umax, Dumin=Dumin, Dumax=Dumax)
K.setup()
```

```python
# Basic Kalman filter design
Q_kal = np.diag([0.1, 10, 0.1, 10])
R_kal = np.eye(ny)
L,P,W  = kalman_design_simple(Ad, Bd, Cd, Dd, Q_kal, R_kal, type='filter')
x0_est = x0
KF = LinearStateEstimator(x0_est, Ad, Bd, Cd, Dd,L)
```

```python
# Simulate in closed loop
[nx, nu] = Bd.shape  # number of states and number or inputs
len_sim = 10  # simulation length (s)
nsim = int(len_sim / Ts)  # simulation length(timesteps)
x_vec = np.zeros((nsim, nx))
y_vec = np.zeros((nsim, ny))
y_meas_vec = np.zeros((nsim, ny))
y_est_vec = np.zeros((nsim, ny))
x_est_vec = np.zeros((nsim, nx))
x_ref_vec = np.zeros((nsim, nx))
u_vec = np.zeros((nsim, nu))
t_MPC_CPU = np.zeros((nsim,1))
t_vec = np.arange(0, nsim) * Ts

time_start = time.time()

x_step = x0
x_step_est = x0
t_step = t0

uMPC = uminus1
for i in range(nsim):
    # Output for step i
    # System
    y_step = Cd.dot(system_dyn.y)  # y[i] from the system
    ymeas_step = y_step
    ymeas_step[0] += std_npos * np.random.randn()
    ymeas_step[1] += std_nphi * np.random.randn()
    # Estimator
    # MPC
    uMPC = K.output()  # u[i] = k(\hat x[i]) possibly computed at time instant -1

    # Save output for step i
    y_vec[i, :] = y_step  # y[i]
    y_meas_vec[i, :] = ymeas_step  # y_meas[i]
    x_vec[i, :] = system_dyn.y  # x[i]
    y_est_vec[i, :] = KF.y  # \hat y[i|i-1]
    x_est_vec[i, :] = KF.x  # \hat x[i|i-1]
    x_ref_vec[i, :] = xref #xref_fun(t_step)
    u_vec[i, :] = uMPC  # u[i]

    # Update to i+1
    # System
    system_dyn.set_f_params(uMPC)  # set current input value to uMPC
    system_dyn.integrate(system_dyn.t + Ts)  # integrate system dynamics for a time step

    # Kalman filter: update and predict
    KF.update(ymeas_step)  # \hat x[i|i]
    KF.predict(uMPC)  # \hat x[i+1|i]

    # MPC update for step i+1
    time_MPC_start = time.time()
    K.update(KF.x, uMPC)  # update with measurement (and possibly pre-compute u[i+1])
    t_MPC_CPU[i] = time.time() - time_MPC_start

    # Time update
    t_step += Ts

time_sim = time.time() - time_start
```

```python
# Plot results

fig, axes = plt.subplots(3, 1, figsize=(10, 10), sharex=True)
axes[0].plot(t_vec, x_est_vec[:, 0], "b", label="p_est")
axes[0].plot(t_vec, x_vec[:, 0], "k", label='p')
axes[0].plot(t_vec, x_ref_vec[:,0], "r--", linewidth=4, label="p_ref")
axes[0].set_ylabel("Position (m)")

axes[1].plot(t_vec, x_est_vec[:, 2] * 360 / 2 / np.pi, "b", label="phi_est")
axes[1].plot(t_vec, x_vec[:, 2] * 360 / 2 / np.pi, label="phi")
axes[1].plot(t_vec,  x_ref_vec[:,2] * 360 / 2 / np.pi, "r--", linewidth=4, label="phi_ref")
axes[1].set_ylabel("Angle (deg)")

axes[2].plot(t_vec, u_vec[:, 0], label="u")
axes[2].plot(t_vec, uref * np.ones(np.shape(t_vec)), "r--", linewidth=4, label="u_ref")
axes[2].set_ylabel("Force (N)")

for ax in axes:
    ax.grid(True)
    ax.legend()

```

```python
# Histogram of the MPC CPU time

fig,ax = plt.subplots(1,1, figsize=(5,5))
ax.hist(t_MPC_CPU*1000, bins=100)
ax.grid(True)
_ = ax.set_xlabel('MPC computation CPU time (ms)')
```