```python
from __future__ import division # use so 1/2 = 0.5, etc.
%pylab inline
import sk_dsp_comm.sigsys as ss
import scipy.signal as signal
from IPython.display import Image, SVG
```

```python
pylab.rcParams['savefig.dpi'] = 100 # default 72
%config InlineBackend.figure_formats=['svg'] # SVG inline viewing
```

# Introduction to Python and the Jupyter Notebook

```python
t = arange(-4,4,.01)
x = cos(2*pi*t)
plot(t,x)

grid()
```

# Rectangle and Triangle Pulses Defined
Before showing more examples, consider some familiar signal primitives in your signals and systems background.

To see these defined in the text see in particular Appendix F.5 (p.727) in the table of Fourier transform pairs.

**Rectangle**
\begin{align}
	\Pi\Big(\frac{t}{\tau}\Big) &= \begin{cases}
		1, & |t| \leq \tau/2 \\
		0, & \text{otherwise}
	\end{cases}
\end{align}
**Triangle**
\begin{align}
	\Lambda\Big(\frac{t}{\tau}\Big) &= \begin{cases}
		1-|t/\tau|, & |t|\leq \tau \\
		0, & \text{otherwise}
	\end{cases}
\end{align}
To more readily play with these function represent them numerically in Python. The module `ss.py` has some waveform primitives to help.

```python
t = arange(-5,5,.01)
x_rect = ss.rect(t-3,2)
x_tri = ss.tri(t+2,1.5)
subplot(211)
plot(t,x_rect)
grid()
ylabel(r'$\Pi((t-3)/2)$');
subplot(212)
plot(t,x_tri)
grid()
xlabel(r'Time (s)')
ylabel(r'$\Lambda((t+2)/1.5)$');
tight_layout()
```

* Consider an interactive version of the above:

```python
# Make an interactive version of the above
from ipywidgets import interact, interactive

def pulses_plot(D1,D2,W1,W2):
    t = arange(-5,5,.01)
    x_rect = ss.rect(t-D1,W1)
    x_tri = ss.tri(t-D2,W2)
    subplot(211)
    plot(t,x_rect)
    grid()
    ylabel(r'$\Pi((t-3)/2)$');
    subplot(212)
    plot(t,x_tri)
    grid()
    xlabel(r'Time (s)')
    ylabel(r'$\Lambda((t+2)/1.5)$');
    tight_layout()

interactive_plot = interactive(pulses_plot,D1 = (-3,3,.5), D2 = (-3,3,.5), W1 = (0.5,2,.25), W2 = (0.5,2,.25));
output = interactive_plot.children[-1]
output.layout.height = '350px'
interactive_plot
```

## More Signal Plotting
The basic pulse shapes (primitives) defined in the module `ssd.py` are very useful for working Text 2.13a &d, but there are also times when you need a custom piecewise function.

### Simple Cases:
Consider plotting

* $x_1(t) = \sin(2\pi\cdot 5t) \Pi((t-2)/2)$ for $0\leq t \leq 10$
* $x_2(t) = \sum_{n=-\infty}^\infty = \Pi((t-5n)/1)$ for $-10 \leq t \leq 10$

```python
t1 = arange(0,10+.01,.01) # arange stops one step size less than the upper limit
x1 = sin(2*pi*5*t1)* ss.rect(t1-2,2)
subplot(211)
plot(t1,x1)
xlabel(r'Time (s)')
ylabel(r'$x_1(t)$')
grid()
t2 = arange(-10,10,.01)
# Tweak mod() to take on negative values
x2 = ss.rect(mod(t2+2.5,5)-2.5,1)
subplot(212)
plot(t2,x2)
xlabel(r'Time (s)')
ylabel(r'$x_2(t)$')
grid()
tight_layout()
```

### Custom Piecewise:
A custom piecewise function is a direct and to the point way of getting a more complex function plotted. Consider plotting:
\begin{align}
    x_3(t) = \begin{cases}
        1 + t^2, & 0\leq t \leq 3 \\
        \cos(2\pi\cdot5\cdot t) & 3 < t \leq 5 \\
        0, & \text{otherwise}
    \end{cases}
\end{align}
for $-2\leq t \leq 6$.

```python
def x3_func(t):
    """
    Create a piecewise function for plotting x3
    """
    x3 = zeros_like(t)
    for k,tk in enumerate(t):
        if tk >= 0 and tk <= 3:
            x3[k] = 1 + tk**2
        elif tk > 3 and tk <= 5:
            x3[k] = cos(2*pi*5*tk)
    return x3
```

```python
t3 = arange(-2,6+.01,.01)
x3 = x3_func(t3)
plot(t3,x3)
xlabel(r'Time (s)')
ylabel(r'$x_3(t)$')
xlim([-2,6])
grid()
```

```python
26/2
```

# Energy and Power Signals

The general definitions are:
\begin{align}
		E &\overset{\Delta}{=} \lim_{T\rightarrow\infty} \int_{-T}^T |x(t)|^2\, dt = \int_{-\infty}^\infty |x(t)|^2\, dt \\
		P &\overset{\Delta}{=} \lim_{T\rightarrow\infty}\frac{1}{2T} \int_{-T}^T |x(t)|^2\, dt
\end{align}
For the case of a periodic signal, you can take the definition of $P$ above and reduce the calculation down to
\begin{align}
    P = \frac{1}{T} \int_{t_0}^{t_0+T} |x(t)|^2\, dt
\end{align}
where $t_0$ can be any convenient value.

Consider the waveform of Text problem 2.14b
\begin{align}
    x_2(t) = \sum_{n=-\infty}^\infty \Lambda\Big(\frac{t-3n}{2}\Big)
\end{align}

You can create an approximation to the waveform over a finite number of periods by doing a little programming:

```python
def periodic_tri(t,tau,T,N):
    """
    Approximate x2(t) by running the sum index from -N to +N.
    The period is set by T and tau is the tri pulse width
    parameter (base width is 2*tau).

    Mark Wickert January 2015
    """
    x = zeros_like(t)
    for n in arange(-N,N+1):
        x += ss.tri(t-T*n,tau)
    return x
```

```python
t = arange(-10,10,.001)
x = periodic_tri(t,2,6,10)
plot(t,x)
plot(t,abs(x)**2)
grid()
#xlim([-5,5])
xlabel(r'Time (s)')
ylabel(r'$x_2(t)$ and $x_2^2(t)$');
```

For the power calculation create a time array that runs over exactly one period. Below is the case for the original problem statement.

```python
T0 = 6
tp = arange(-T0/2,T0/2+.001,.001)
xp = periodic_tri(tp,2,T0,5)
plot(tp,xp)
plot(tp,abs(xp)**2)
legend((r'$x(t)$', r'$|x(t)|^2$'),loc='best',shadow=True)
grid();
xlim([-T0/2,T0/2])
xlabel(r'Time (s)')
ylabel(r'$x_2(t)$ and $x_2^2(t)$');
```

A simple numerical approximation to the integral
\begin{align}
    P = \frac{1}{T}\int_0^T |x_b(t)|^2\, dt
\end{align}
is shown below:

```python
#Power calculation
Px2 = (1/T0)*sum(xp**2)*.001 # rectangular partitions for integral
print('Power estimate via numerical integration: %2.4f W' % Px2)
```

## Power in the Sum of Two Sinusoids

The problem is what is the power in the signal
\begin{align}
    x(t) = A_1 \cos(\omega_1 t +\phi_1) + A_2 \cos(\omega_2 t + \phi_2),\ -\infty < t < \infty
\end{align}
Since we are not certain that $x(t)$ is periodic, the power calculation requires that we form
\begin{align}
    P_x = \lim_{T\rightarrow\infty} \frac{1}{T} \int_{-T/2}^{T/2} |x(t)|^2\, dt = \langle |x(t)|^2\rangle
\end{align}
* Rather that just jumping in and making a mess, consider first the expansion of $|x(t)|^2 = x^2(t)$:
\begin{align}
    x^2(t) &= \frac{A_1^2}{2}\big[1+\cos(2\omega_1 t + \phi_1)\big] + \frac{A_2^2}{2}\big[1+\cos(2\omega_2 t + \phi_2)\big] \\
    &\quad + 2\frac{A_1 A_2}{2}\Big\{\cos[(\omega_1 + \omega_2)t + (\phi_1+\phi_2)\big] + \cos[(\omega_1 - \omega_2)t + (\phi_1-\phi_2)\big]\Big\}
\end{align}
* The time average operator is linear, so we consider $\langle\ \ \rangle$ operating on each term of the above independently
* For $\omega_1 \neq \omega_2$, the first two terms yield $A_1^2/2$ and $A_2^2/2$ respectively
* The last term requires some thinking, but as long as $\omega_1 \neq \omega_2$ the times average of $\cos[(\omega_1 + \omega_2)t + (\phi_1+\phi_2)]$ and $\cos[(\omega_1 - \omega_2)t + (\phi_1-\phi_2)$], the two terms respectively are each zero!
* Finally,
\begin{align}
    P_x = \frac{A_1^2}{2} + \frac{A_2^2}{2}
\end{align}
* When the frequencies are equal, then you can combine the terms using trig identities (recall the phasor addition formula from ECE 2610
\begin{align}
    x(t) = A\cos(\omega t + \phi)
\end{align}
where $\omega = \omega_1 = \omega_2$ and
\begin{align}
    Ae^{j\phi} = A_1e^{j\phi_1} + A_2 e^{j\phi_2}
\end{align}

```python
t = arange(-10,10,.001)
x1 = 4*cos(2*pi*10*t)
x2 = 3*cos(2*pi*3.45*t+pi/9)
plot(t,x1)
plot(t,x2)
plot(t,x1+x2)
grid()
xlabel(r'Time (s)')
ylabel(r'Amplitude')
legend((r'$x_1(t)$', r'$x_2(t)$', r'$x_1(t)+x_2(t)$'),loc='best',shadow=True)
xlim([-.1,.1]);
```

```python
print('Power calculations: %3.2f, %3.2f, %3.2f' \
      % (var(x1),var(x2),var(x1+x2)))
```

```python
print('Theory: %3.2f, %3.2f, %3.2f' \
      % (4**2/2,3**2/2,4**2/2+3**2/2))
```

# Fourier Series and Line Spectra Plotting

Being able to easily plot the line spectra of periodic signals will hopefully enhance your understanding. The module `ss.py` contains the function `ss.line_spectra()` for this purpose. The function assumes that the Fourier coefficients, $X_n$ are available for a real signal $x(t)$. The function plots line spectra as:
* The two-sided magnitude spectra
* The two-sided magnitude spectra in dB with an adjustable floor level in dB
* The two-sided phase spectra in radians
* The one-sided line spectra corresponding to the three cases listed immediately above
Examples are given below for the case of a simple pulse train and then for a trapezoidal pulse train. IN the case of the trapezoidal pulse train the underlying Fourier coefficients are obtained numerically using the FFT as described in the course notes.

A fundamental requirement in using `ss.line_spectra()` is to beable to supply the coefficients starting with the DC term coefficient $X_0$ and moving up to the $N$th harmonic. Before plotting the pulse train line spectra I first describe a *helper* function for visualizing the pulse train waveform.

## Pulse Train

```python
def pulse_train(Np,fs,tau,t0):
    """
    Generate a discrete-time approximation to a continuous-time
    pulse train signal. Amplitude values are [0,1]. Scale and offset
    later if needed.

    Inputs
    ------
     Np = number of periods to generate
     fs = samples per period
    tau = duty cycle
     t0 = pulse delay time relative to first rising edge at t = 0

    Return
    ------
    t = time axis array
    x = waveform

    Mark Wickert, January 2015
    """
    t = arange(0,Np*fs+1,1)/fs #time is normalized to make period T0 = 1.0
    x = zeros_like(t)
    # Using a brute force approach, just fill x with the sample values
    for k,tk in enumerate(t):
        if mod(tk-t0,1) <= tau and mod(tk-t0,1) >= 0:
            x[k] = 1
    return t,x
```

```python
tau = 1/8; fs = 8*16; t0 = 0 # note t0 = tau/2
subplot(211)
t,x = pulse_train(4,fs,tau,t0)
plot(t,x) # Just a plot of xa(t)
ylim([-.1,1.1])
grid()
ylabel(r'$x_a(t)$')
title(r'Pulse Train Signal: (top) $x_a(t)$, (bot) $x_b(t) = 1-x_a(t)$');
subplot(212)
t,x = pulse_train(4,fs,tau,t0)
plot(t,1-x) # Note here y(t) = 1 - x(t), a special case of
ylim([-.1,1.1]) # y(t) = A + B*x(t) in the notes
grid()
xlabel(r'Time ($t/T_0$)')
ylabel(r'$x_b(t)$');
```

## Example: Pulse Train Line Spectra

For the case of pulse train having the initial pulse starting at $t=0$, i.e.,
\begin{align}
    x(t) = \sum_{k=-\infty}^\infty A\cdot \Pi\left(\frac{t-\tau/2-kT_0}{\tau}\right),
\end{align}
the Fourier coefficient are given by
\begin{align}
    X_n = A\cdot\frac{\tau}{T_0}\cdot\text{sinc}(nf_0\tau)\cdot\exp(-j2\pi n f_0t_0)
\end{align}
where $f_0 = 1/T_0$ is the fundamental frequency and here $t_0 = \tau/2$.

Line spectra plotting is shown below for this case. If the pulse train should be shifted in time to some other orientation, then the phase plot will change, as the included $\exp(j2\pi n f_0 t_0)$ term will be different.

**Note:** The pulse train function define above is slightly different from the pulse train defined in the book and shown in mathematical form as $x(t)$ just above in this cell. The function `pulse_train()` has the first pulse starting exactly at $t=0$. To move the pule train right or left on the time axis, you can use the function parameter `t0`.

```python
n = arange(0,25+1) # Get 0 through 25 harmonics
tau = 0.125; f0 = 1; A = 1;
Xn = A*tau*f0*sinc(n*f0*tau)*exp(-1j*2*pi*n*f0*tau/2)
# Xn = -Xn # Convert the coefficients from xa(t) t0 xb(t)
# Xn[0] += 1
figure(figsize=(6,2))
f = n # Assume a fundamental frequency of 1 Hz so f = n
ss.line_spectra(f,Xn,mode='mag',sides=2,fsize=(6,2))
xlim([-25,25]);
#ylim([-50,10])
figure(figsize=(6,2))
ss.line_spectra(f,Xn,mode='phase',fsize=(6,2))
xlim([-25,25]);
```

## Example: Trapezoidal Pulse

Consider the line spectra of a finite rise and fall time pulse train is of practical interest. The function `trap_pulse()` allows you first visualize one period of the trapezoidal pulse train, and then use this waveform in obtaining numerically the Fourier coefficients of this signal. PLotting the corresponding line spectra follows.

A point to be main is that by slowing down the edges (rise time/fall time) of the pulse train the amplitude of the harmonics falls off more rapidly. When considering the clock speed in todays PCs this can be a good thing as harmonic emission is an issue.

```python
def trap_pulse(N,tau,tr):
    """
    xp = trap_pulse(N,tau,tr)

    Mark Wickert, January 2015
    """
    n = arange(0,N)
    t = n/N
    xp = zeros(len(t))
    # Assume tr and tf are equal
    T1 = tau + tr
    # Create one period of the trapezoidal pulse waveform
    for k in n:
        if t[k] <= tr:
            xp[k] = t[k]/tr
        elif (t[k] > tr and t[k] <= tau):
            xp[k] = 1
        elif (t[k] > tau and t[k] < T1):
            xp[k] = -t[k]/tr + 1 + tau/tr;
        else:
            xp[k] = 0
    return xp, t
```

Let $\tau = 1/8$ and $t_r = 1/20$:

```python
# tau = 1/8, tr = 1/20
N = 1024
xp,t = trap_pulse(N,1/8,1/20)
Xp = fft.fft(xp)
figure(figsize=(6,2))
plot(t,xp)
grid()
title(r'Spectra of Finite Risetime Pulse Train: $\tau = 1/8$ $t_r = 1/20$')
ylabel(r'$x(t)$')
xlabel('Time (s)')
f = arange(0,N/2)
ss.line_spectra(f[0:25],Xp[0:25]/N,'magdB',floor_dB=-80,fsize=(6,2))
ylabel(r'$|X_n| = |X(f_n)|$ (dB)');
#% tau = 1/8, tr = 1/10
xp,t = trap_pulse(N,1/8,1/10)
Xp = fft.fft(xp)
figure(figsize=(6,2))
plot(t,xp)
grid()
title(r'Spectra of Finite Risetime Pulse Train: $\tau = 1/8$ $t_r = 1/10$')
ylabel(r'$x(t)$')
xlabel('Time (s)')
ss.line_spectra(f[0:25],Xp[0:25]/N,'magdB',floor_dB=-80,fsize=(6,2))
ylabel(r'$|X_n| = |X(f_n)|$ (dB)');
```

With the edge speed slowed down it is clear that the harmonics drop off faster.

# Fourier Transforms

The Fourier transfrom definition is:
\begin{align}
    X(f) &= \int_{-\infty}^\infty x(t)\ e^{-j2\pi ft}\, dt \\
    x(t) &= \int_{-\infty}^\infty X(f)\, e^{j2\pi ft}\, df
\end{align}

A numerical approximation to the Fourier transform is possible using the FFT, or more conveniently using the function `freqz()` from the package `scipy.signal`. A helper function to abstract some of the digital signal processing details is `f, X = FT_approx(x,dt,Nfft)`. The function is now part of `sigsys.py` with name change to `ft_approx()`:

```python
def FT_approx(x,t,Nfft):
    '''
    Approximate the Fourier transform of a finite duration
    signal using scipy.signal.freqz()

    Inputs
    ------
       x = input signal array
       t = time array used to create x(t)
    Nfft = the number of frdquency domain points used to
           approximate X(f) on the interval [fs/2,fs/2], where
           fs = 1/Dt. Dt being the time spacing in array t

    Return
    ------
    f = frequency axis array in Hz
    X = the Fourier transform approximation (complex)

    Mark Wickert, January 2015
    '''
    fs = 1/(t[1] - t[0])
    t0 = (t[-1]+t[0])/2 # time delay at center
    N0 = len(t)/2 # FFT center in samples
    f = arange(-1/2,1/2,1/Nfft)
    w, X = signal.freqz(x,1,2*pi*f)
    X /= fs # account for dt = 1/fs in integral
    X *= exp(-1j*2*pi*f*fs*t0)# time interval correction
    X *= exp(1j*2*pi*f*N0)# FFT time interval is [0,Nfft-1]
    F = f*fs
    return F, X
```

## Example: Rectangular Pulse

As as simple starting point example, consider $x(t) = \Pi(t\tau)$. The well known result for the Fourier transfrom (FT) is:
\begin{align}
    X(f) = \mathcal{F}\left\{\Pi\left(\frac{t}{\tau}\right)\right\} = \tau\,\text{sinc}(f\tau)
\end{align}
We now use the above defined `FT_approx()` to obtain a numerical approximation to the FT of the rectangular pulse.

**Tips:**
* Make sure the signal is well contained on the time interval used to generate $x(t)$
* Make sure the sampling rate, one over the sample spacing, is adequate to represent the signal spectrum
* From sampling theory, the reange of frequencies represented by the spectrum estimate will be $f_s/2 \leq f < f_s/2$

```python
fs = 100 # sampling rate in Hz
tau = 1
t = arange(-5,5,1/fs)
x0 = ss.rect(t-.5,tau)
figure(figsize=(6,5))
subplot(311)
plot(t,x0)
grid()
ylim([-0.1,1.1])
xlim([-2,2])
title(r'Exact Waveform')
xlabel(r'Time (s)')
ylabel(r'$x_0(t)$');

# FT Exact Plot
fe = arange(-10,10,.01)
X0e = tau*sinc(fe*tau)
subplot(312)
plot(fe,abs(X0e))
#plot(f,angle(X0))
grid()
xlim([-10,10])
title(r'Exact Spectrum Magnitude')
xlabel(r'Frequency (Hz)')
ylabel(r'$|X_0e(f)|$');

# FT Approximation Plot
f,X0 = ss.ft_approx(x0,t,4096)
subplot(313)
plot(f,abs(X0))
#plot(f,angle(X0))
grid()
xlim([-10,10])
title(r'Approximation Spectrum Magnitude')
xlabel(r'Frequency (Hz)')
ylabel(r'$|X_0(f)|$');
tight_layout()
```

## Example: Text Problem 2.31a Drill Down

In this problem you are given
\begin{align}
    x_1(t) = \Pi\left(\frac{t+1/2}{1}\right) - \Pi\left(\frac{t-1/2}{1}\right)
\end{align}
The Fourier transfrom of this signal can be found using *linearity* and the *time delay* theorems.
\begin{align}
    X_1(f) &= \mathcal{F}\left\{\Pi\left(\frac{t+1/2}{1}\right) - \Pi\left(\frac{t-1/2}{1}\right)\right\} \\
    &= \text{sinc}(f)\cdot\left[e^{j2\pi f\cdot 1/2} - e^{-j2\pi f\cdot 1/2}\right]\times\frac{2j}{2j} \\
    &= 2j\ \text{sinc}(f)\cdot\sin(\pi f)
\end{align}

```python
fs = 100
t = arange(-5,5,1/fs)
x1 = ss.rect(t+1/2,1)-ss.rect(t-1/2,1)
subplot(211)
plot(t,x1)
grid()
ylim([-1.1,1.1])
xlim([-2,2])
xlabel(r'Time (s)')
ylabel(r'$x_1(t)$');
fe = arange(-10,10,.01)
X1e = 2*1j*sinc(fe)*sin(pi*fe)
f,X1 = ss.ft_approx(x1,t,4096)
subplot(212)
plot(f,abs(X1))
plot(fe,abs(X1e))
#plot(f,angle(X1))
legend((r'Num Approx',r'Exact'),loc='best')
grid()
xlim([-10,10])
xlabel(r'Frequency (Hz)')
ylabel(r'$|X_1(f)|$');
tight_layout()
```

* Notice the numerical approximation and exact spectral plots overlay one another

## Example: Modulation Theorem

Consider the modulation theorem, which is extremely important to communications theory:
\begin{align}
    y(t) &= x(t)\cdot\cos(2\pi f_0 t) \\
    Y(f) &= \frac{1}{2}\left[X(f-f_0) + X(f+f_0)\right]
\end{align}
Here we will use a triangle pulse for $x(t)$:

```python
fs = 100 # sampling rate in Hz
tau = 1
t = arange(-5,5,1/fs)
x3 = ss.tri(t,tau)
y = x3*cos(2*pi*10*t)
subplot(211)
plot(t,x3)
plot(t,y)
grid()
ylim([-1.1,1.1])
xlim([-2,2])
legend((r'$x_3(t)$', r'$y(t)$'),loc='lower right',shadow=True)
title(r'Time Domain: $x_3(t)$ and $y(t)=x_3(t)\cos(2\pi\cdot 5\cdot t)$')
xlabel(r'Time (s)')
ylabel(r'$y(t)$');
f,Y = ss.ft_approx(y,t,4096)
subplot(212)
plot(f,abs(Y))
#plot(f,angle(X0))
grid()
title(r'Frequency Domain: $Y(f)$')
xlim([-15,15])
xlabel(r'Frequency (Hz)')
ylabel(r'$|Y(f)|$');
tight_layout()
```

## Example: Representing a Bandlimited Signal

We know that in theory a bandlimited signal can only be generated from a signal having infinite duration. Specifically, a signal with rectangular spectrum has Fourier transfrom pair:
\begin{align}
    x(t) = 2W\text{sinc}(2Wt) \overset{\mathcal{F}}{\Leftrightarrow} \Pi\left(\frac{f}{2W}\right) = X(f)
\end{align}
In a simulation we expect to have troubles modeling the finite duration aspects of the signal.

```python
fs = 100 # sampling rate in Hz
W = 5
t = arange(-5,5,1/fs)
x4 = 2*W*sinc(2*W*t)
figure(figsize=(6,2))
plot(t,x4)
grid()
#ylim([-1.1,1.1])
xlim([-2,2])
title(r'Time Domain: $x_4(t),\ W = 5$ Hz')
xlabel(r'Time (s)')
ylabel(r'$x_4(t)$');
f,X4 = ss.ft_approx(x4,t,4096)
figure(figsize=(6,2))
plot(f,abs(X4))
grid()
title(r'Frequency Domain: $X_4(f)$')
xlim([-10,10])
xlabel(r'Frequency (Hz)')
ylabel(r'$|X_4(f)|$');
figure(figsize=(6,2))
plot(f,20*log10(abs(X4)))
grid()
title(r'Frequency Domain: $X_4(f)$ in dB')
ylim([-50,5])
xlim([-10,10])
xlabel(r'Frequency (Hz)')
ylabel(r'$|X_4(f)|$ (dB)');
```

**Note:** The dB version (last plot) reveals that the first sidelobes of the spectrum are only down ~21dB. Increasing the length of the time window will not help. The spectral side lobes will become more tightly packed, but the first sidelobe will still be down only 21dB. With other pulse shapes in the time domain, i.e., note simply a truncted $\text{sinc}()$ function reduced sidelobes can be obtained.

# Convolution

* The convolution of two signals $x_1(t)$ and $x_2(t)$ is defined as
\begin{align}
	x(t) &= x_1(t)\ast x_2(t) = \int_{-\infty}^\infty x_1(\lambda)x_2(t-\lambda)\, d\lambda \\
	&= x_2(t)\ast x_1(t) = \int_{-\infty}^\infty x_2(\lambda)x_1(t-\lambda)\, d\lambda
\end{align}
* A special convolution case is $\delta(t-t_0)$
\begin{align}
		\delta(t-t_0)\ast x(t) &= \int_{-\infty}^\infty \delta(\lambda-t_0)x(t-\lambda)\, d\lambda \\
		&= x(t-\lambda)\big|_{\lambda=t_0} = x(t-t_0)
\end{align}
You can experiment with the convolution integral numerically using `ssd.conv_integral()` found in the module `ssd.py`.

```python
t = arange(-2,2.001,.001)
p1 = ss.rect(t,1)
p2 = ss.rect(t,3)
y,ty = ss.conv_integral(p1,t,p2,t)
plot(ty,y)
ylim([-.01,1.01])
grid()
xlabel(r'Time (s)')
ylabel(r'$y(t)$');
```

For convolutions involving semi-infinite signals, such as $u(t)$, you can tell `ssd.conv_integral()` about this via the optional extent argument. See the function help using
```python
ss.conv_integral?
```

```python
# Consider a pulse convolved with an exponential ('r' type extent)
tx = arange(-1,8,.01)
x = ss.rect(tx-2,4) # pulse starts at t = 0
h = 4*exp(-4*tx)*ss.step(tx)
y,ty = ss.conv_integral(x,tx,h,tx,extent=('f','r')) # note extents set
plot(ty,y) # expect a pulse charge and discharge waveform
grid()
title(r'$\Pi((t-2)/4)\ast 4 e^{-4t} u(t)$')
xlabel(r'Time (s)')
ylabel(r'$y(t)$');
```

# Spectrum of PN Sequence (exact)
The cell below is a copy of the earlier pulse train line spectra example. Use this as a template to create the solution to the PN code problem of HW 3.

```python
n = arange(0,25+1) # Get 0 through 25 harmonics
tau = 0.125; f0 = 1; A = 1;
Xn = A*tau*f0*sinc(n*f0*tau)*exp(-1j*2*pi*n*f0*tau/2)
# Xn = -Xn # Convert the coefficients from xa(t) t0 xb(t)
# Xn[0] += 1
figure(figsize=(6,2))
f = n # Assume a fundamental frequency of 1 Hz so f = n
ss.line_spectra(f,Xn,mode='mag',sides=2,fsize=(6,2))
xlim([-25,25]);
#ylim([-50,10])
figure(figsize=(6,2))
ss.line_spectra(f,Xn,mode='phase',fsize=(6,2))
xlim([-25,25]);
```

# Spectrum of PN Sequence (approx)
The code below approximates the PSD of the PN code using a numerical approximation to the Fourier coefficients, $X_n$. This development may be useful for the lab, as you can esily change the waveform level without having to rework the theory.

The approach taken here to create one period of the PN waveform at 10 samples per bit. The line containing the function `ss.upsample()` converts the bit sequence into a waveform by upsampling and filtering with a rectangular pulse shape (`ones(10)`). The function `ss.fs_coeff()` numerically calculates the $X_n$'s. To plot the PSD from the Fourier coefficients we use
$$
    S_x(f) = \sum_{n=-\infty}^\infty |X_n|^2 \delta(f-nf_0)
$$
where $f_0$ in this case is $1/(MT_0)$ with $T_0$ beging the bit period and $M$ the code period in bits.

```python
x_PN4 = ss.m_seq(4)
x = signal.lfilter(ones(10),1,ss.upsample(x_PN4,10))
t = arange(0,len(x))/10
figure(figsize=(6,2))
plot(t,x);
title(r'Time Domain and PSD of $M=15$ PN Code with $T = 1$')
xlabel(r'Time (s)')
ylabel(r'x(t)')
axis([0,15,-0.1,1.1]);
grid()
# 10 samples/bit so 150 samples/period
# harmonics spaced by 1/(15*T) = 1/15
Xk,fk = ss.fs_coeff(x,45,1/15)
ss.line_spectra(fk,Xk,'magdB',lwidth=2.0,floor_dB=-50,fsize=(6,2))
xlim([-3,3])
ylabel(r'$|X_n| = |X(f_n)|$ (dB)');
```

```python
# Line spacing
1/15
```

```python
import sk_dsp_comm.digitalcom as dc
y_PN5_bits = ss.PN_gen(10000,5)
# Convert to waveform level shifted to +/-1 amplitude
y = 2*signal.lfilter(ones(10),1,ss.upsample(y_PN5_bits,10))-1
# Find the time averaged autocorrelation function normalized
# to have a peak amplitude of 1
Ry,tau = dc.xcorr(y,y,400)
# We know Ry is real so strip small imag parts from FFT-based calc
Ry = Ry.real
```

```python
fs = 10
t = arange(len(y))/fs
plot(t[:500],y[:500])
title(r'PN Waveform for 5 Stages (Period $2^5 -1 = 31$ bits)')
ylabel(r'Amplitude')
xlabel(r'Bits (10 samples/bit)')
grid();
```

```python
tau_s = tau/10
figure(figsize=(6,2))
plot(tau_s,Ry)
title(r'Autocorrelation and PSD Estimates for $M=31$ with $T = 1$')
xlabel(r'Autocorrelation Lag $\tau$ (s)')
ylabel(r'$R_y(\tau)$')
grid();
figure(figsize=(6,2))
psd(y,2**12,10)
xlabel(r'Frequency (Hz)')
ylabel(r'$S_y(f)$ (dB)')
#xlim([0,.002]);
ylim([-30,20]);
```

### Lab Tip: PN Generator Coded at the Bit Level
In Lab 2 of ECE 4670 a C/C++ version of a PN generator is implemented to run the ARM `mbed` LPC 1768 microcontroller (https://www.sparkfun.com/products/9564). At the heart of this code is:
```C
// Globals defined as unsigned int
tap1 -= 1;
tap2 -= 1;
mask1 = 0x1 << (tap1);
mask2 = 0x1 << (tap2);
bit = 0x0;
sync = 0x0;

void gen_PN() {
    my_pin5 = bit;
    my_pin6 = synch_bit;
    led2 = bit;
    led3 = synch_bit;
    if (clk_state == 0x1)
    {
    // Advance m-sequence generator by one bit
    // XOR tap1 and tap2 SR values and feedback to input
        fb = ((sr & mask1)>> tap1) ^ ((sr & mask2) >> tap2);
        sr = (sr << 1) + fb;
        bit = sr & 0x1;
        // Use random number generator in place of m-sequence bits
        if (DIP_sw4)
        {
            bit = rand_int() & 0x1;
        }
        clk_state = 0x0;
        // See if all 1's condition exists in SR
        if ((sr & synch) == synch) {
            synch_bit = 0x1;
        }
        else
        {
            synch_bit = 0x0;
        }
    }
    else
    {
        if (DIP_sw1) bit = !bit;
        clk_state = 0x1;
    }
}
```
The data type is `unsigned int`, which on the mbed is `uint16_t`, an unsigned 16-bit integer. A single unsigned integer is used as a 16-bit shift register with the LSB, furthest bit to the right, used to represent the first register stage. The shift register is advanced using a left shift `<<` bitwise operation. We can code this Python almost directly, as shown below.

```python
class bitwise_PN(object):
    """
    Implement a PN generator using bitwise manipulation for
    the shift register. The LSB holds b0 and bits are shifted left.
              +----+----+----+----+----+----+----+
         sr = |bM-1| .. |bM-k| .. | b2 | b1 | b0 |
              +----+----+----+----+----+----+----+
                 |         |
    Feedback:(tap1-1)   (tap2-1)    Shift left using <<

    Mark Wickert February 2017
    """
    def __init__(self,tap1,tap2,Nstage,sr_initialize):
        """
        Initialize the PN generator object
        """
        self.tap1 = tap1 - 1
        self.tap2 = tap2 - 1
        self.mask1 = 0x1 << (tap1 - 1) # to select bit of interest
        self.mask2 = 0x1 << (tap2 - 1) # to select bit of interest
        self.Nstage = Nstage
        self.period = 2**Nstage - 1
        self.sr = sr_initialize
        self.bit = 0
        self.sync_bit = 0

    def clock_PN(self):
        '''
        Method to advance m-sequence generator by one bit
        XOR tap1 and tap2 SR values and feedback to input
        '''
        fb = ((self.sr & self.mask1)>> self.tap1) ^ \
            ((self.sr & self.mask2) >> self.tap2)
        self.sr = (self.sr << 1) + fb
        self.sr = self.sr & self.period # set MSBs > Nstage to 0
        self.bit = self.sr & 0x1 # output LSB from SR
        # See if all 1's condition exits in SR, if so output a synch pulse
        if ((self.sr & self.period) == self.period):
            self.sync_bit = 0x1
        else:
            self.sync_bit = 0x0
        print('output = %d, sr contents = %s, sync bit = %d' \
             % (self.bit, binary(self.sr, self.Nstage), self.sync_bit))
```

```python
# A simple binary format display function which shows
# leading zeros to a fixed bit width
def binary(num, length=8):
    return format(num, '#0{}b'.format(length + 2))
```

```python
PN1 = bitwise_PN(10,7,10,0x1)
```

```python
PN1.clock_PN()
```

```python
# sr initial condition
sr = 0b1
```

```python
Nout = 20
x_out = zeros(Nout)
s_out = zeros(Nout)
PN1 = bitwise_PN(3,2,3,0x1)
for k in range(Nout):
    PN1.clock_PN()
    x_out[k] = PN1.bit
    s_out[k] = PN1.sync_bit
```

```python
stem(x_out)
stem(0.2*s_out,markerfmt = 'ro')
ylim([0,1.1])
```

## Cross Correlation and Signal Delay

The idea of the autocorrelation function can be extended to the cross correlation, that is the correlation or likeness between two signals, say $x(t)$ and $y(t)$. Define
\begin{align}
    R_{xy}(\tau) = \langle x(t)y(t+\tau)\rangle = \lim_{T\rightarrow\infty} \frac{1}{2T}\int_{-T}^T x(t)y(t+\tau)\, dt
\end{align}
Consider a simulation example using `dc.xcorr(x,t,lags)`:

```python
import sk_dsp_comm.digitalcom as dc
x_PN4_bits = ss.PN_gen(10000,6)
# Convert to waveform level shifted to +/-1 amplitude
x_s = 2*signal.lfilter(ones(10),1,ss.upsample(x_PN4_bits,10))-1
# Form a delayed version of x_S
T_D = 35 # 35 sample delay
y_s = signal.lfilter(concatenate((zeros(T_D),array([1]))),1,x_s)
figure(figsize=(6,2))
plot(x_s[:200])
plot(y_s[:200])
ylim([-1.1,1.1])
title(r'Delayed and Undelayed Signals for $T_D = 35$ Samples')
xlabel(r'Samples (10/PN bit)')
ylabel(r'$x_s(t)$ and $y_s(t)$')
grid();
# Find the time averaged autocorrelation function normalized
# to have a peak amplitude of 1
Ryx,tau = dc.xcorr(y_s,x_s,200) #note order change
# We know Ryx is real
Ryx = Ryx.real
tau_s = tau/10
figure(figsize=(6,2))
plot(tau_s,Ryx)
title(r'Cross Correlation for $M=4$ with $T = 1$ and Delay 35 Samples')
xlabel(r'Autocorrelation Lag $\tau$ (s)')
ylabel(r'$R_{yx}(\tau)$')
grid();
```

# Spectral Containment Bandwidth (text problem 2.55)

In text problem 2.55 you are asked to find the 90% energy contain bandwidth of a signal $x_i(t)$. Specifically you are to find the lowpass or one-sided bandwidth of a baseband signal such that 90% of the total signal energy is contained in the bandwidth, $B_{90}$. You obtain $B_{90}$ by solving the following equation
\begin{align}
    0.9 = \frac{0.9 E_\text{total}}{E_\text{total}} = \frac{\int_{-B_{90}}^{B_{90}} G(f) df}{\int_{-\infty}^\infty G(f) df} = \frac{2\int_0^{B_{90}} G(f) df}{2\int_0^\infty G(f) df} = \frac{\int_0^{B_{90}} G(f) df}{\int_0^\infty G(f) df},
\end{align}
where $G(f) = |X_i(f)|^2$ is the energy spectral density of $x_i(t)$.

For parts (c) and (d) the problem states you need to perform numerical integration.

## Example:

In an exalier example found in this notebook I found the Fourier transform of
\begin{align}
    x(t) = \Pi\left(\frac{t-\tau/2}{\tau}\right) - \Pi\left(\frac{t+\tau/2}{\tau}\right)
\end{align}
to be
\begin{align}
    X(f) &= 2j\ \text{sinc}(f\tau)\cdot\sin(\pi f\tau)
\end{align}
Note I have modified the problem to now have pulse width $\tau$ to better match the homework problem where $\tau$ is a variable.

The energy spectral density is
\begin{align}
    G(f) = 4\, \text{sinc}^2(f\tau)\cdot\sin^2(\pi f\tau)
\end{align}

I convenient way to numerically integrate $G(f)$ is using simple reactangular partitions, but making sure that $\Delta f$ is small relative to the changes in $G(f)$. Since you do not know what the value of $\tau$ you consider a *normalized frequency* variable $f_n = f\tau$ in the analysis. The rest of the steps are:

1. Sweep $G(f_n)$ using an array `fn` running from zero to $f_n$ large enough to insure that $G(f_n)$ is very small relative to it largest value. In Python this is just filling an array, `Gn` with the functional values.
2. Form a new array which contains the cumulative sum of the values in `Gn`, say `Gn_cumsum = cumsum(Gn)`. Aso form the sum of the array values, i.e., `Gn_tot = sum(Gn)`
3. Plot the ratio of ``Gn_cumsum/Gn_sum` versus `fn`. The curve should start at zero and climb to one as $f_n$ becomes large. The value of $f_n$ where the curve crosses through 0.9 is the 90% containment bandwidth.

**Note:** You might notice that $\Delta f$, which is needed in the rectangular integration formula was never used. Why? In the calculation of the cumulative sum and the calculation of the total, both should include $\Delta f$, hence in the ration the values cancel out. Nice!

```python
fn = arange(0,10,.001)
Gn = 4*sinc(fn)**2 * sin(pi*fn)**2
Gn_cumsum = cumsum(Gn)
Gn_tot = sum(Gn)
plot(fn,Gn_cumsum/Gn_tot)
grid()
xlabel('Normalized Frequency $f\tau$')
ylabel('Fractional Power Containment');
```

```python
fn_idx = find(abs(Gn_cumsum/Gn_tot - 0.9)< 0.0005)
fn_idx
```

```python
print('The normalized 90 percent containment bandwidth is %2.2f Hz-s.' \
      % fn[1448])
```

# Filter Analysis

To facilitate the performance analysis of both discrete-time and continuous-time filters, the functions `freqz_resp()` and `freqs_resp()` are available (definitions below, respectively). With these functions you can quickly move from *z*-domain or *s*-domain rational system function coefficients to visualization of the filter frequency response
* Magnitude
* Magnitude in dB
* Phase in radians
* Group delay in samples or seconds (digital filter)
* Group delay in seconds (analog filter)

```python
def freqz_resp(b,a=[1],mode = 'dB',fs=1.0,Npts = 1024,fsize=(6,4)):
    """
    A method for displaying digital filter frequency response magnitude,
    phase, and group delay. A plot is produced using matplotlib

    freq_resp(self,mode = 'dB',Npts = 1024)

    A method for displaying the filter frequency response magnitude,
    phase, and group delay. A plot is produced using matplotlib

    freqs_resp(b,a=[1],Dmin=1,Dmax=5,mode = 'dB',Npts = 1024,fsize=(6,4))

        b = ndarray of numerator coefficients
        a = ndarray of denominator coefficents
     Dmin = start frequency as 10**Dmin
     Dmax = stop frequency as 10**Dmax
     mode = display mode: 'dB' magnitude, 'phase' in radians, or
            'groupdelay_s' in samples and 'groupdelay_t' in sec,
            all versus frequency in Hz
     Npts = number of points to plot; defult is 1024
    fsize = figure size; defult is (6,4) inches

    Mark Wickert, January 2015
    """
    f = np.arange(0,Npts)/(2.0*Npts)
    w,H = signal.freqz(b,a,2*np.pi*f)
    plt.figure(figsize=fsize)
    if mode.lower() == 'db':
        plt.plot(f*fs,20*np.log10(np.abs(H)))
        plt.xlabel('Frequency (Hz)')
        plt.ylabel('Gain (dB)')
        plt.title('Frequency Response - Magnitude')

    elif mode.lower() == 'phase':
        plt.plot(f*fs,np.angle(H))
        plt.xlabel('Frequency (Hz)')
        plt.ylabel('Phase (rad)')
        plt.title('Frequency Response - Phase')

    elif (mode.lower() == 'groupdelay_s') or (mode.lower() == 'groupdelay_t'):
        """
        Notes
        -----

        Since this calculation involves finding the derivative of the
        phase response, care must be taken at phase wrapping points
        and when the phase jumps by +/-pi, which occurs when the
        amplitude response changes sign. Since the amplitude response
        is zero when the sign changes, the jumps do not alter the group
        delay results.
        """
        theta = np.unwrap(np.angle(H))
        # Since theta for an FIR filter is likely to have many pi phase
        # jumps too, we unwrap a second time 2*theta and divide by 2
        theta2 = np.unwrap(2*theta)/2.
        theta_dif = np.diff(theta2)
        f_diff = np.diff(f)
        Tg = -np.diff(theta2)/np.diff(w)
        max_Tg = np.max(Tg)
        #print(max_Tg)
        if mode.lower() == 'groupdelay_t':
            max_Tg /= fs
            plt.plot(f[:-1]*fs,Tg/fs)
            plt.ylim([0,1.2*max_Tg])
        else:
            plt.plot(f[:-1]*fs,Tg)
            plt.ylim([0,1.2*max_Tg])
        plt.xlabel('Frequency (Hz)')
        if mode.lower() == 'groupdelay_t':
            plt.ylabel('Group Delay (s)')
        else:
            plt.ylabel('Group Delay (samples)')
        plt.title('Frequency Response - Group Delay')
    else:
        s1 = 'Error, mode must be "dB", "phase, '
        s2 = '"groupdelay_s", or "groupdelay_t"'
        print(s1 + s2)
```

```python
def freqs_resp(b,a=[1],Dmin=1,Dmax=5,mode = 'dB',Npts = 1024,fsize=(6,4)):
    """
    A method for displaying analog filter frequency response magnitude,
    phase, and group delay. A plot is produced using matplotlib

    freqs_resp(b,a=[1],Dmin=1,Dmax=5,mode='dB',Npts=1024,fsize=(6,4))

        b = ndarray of numerator coefficients
        a = ndarray of denominator coefficents
     Dmin = start frequency as 10**Dmin
     Dmax = stop frequency as 10**Dmax
     mode = display mode: 'dB' magnitude, 'phase' in radians, or
            'groupdelay', all versus log frequency in Hz
     Npts = number of points to plot; defult is 1024
    fsize = figure size; defult is (6,4) inches

    Mark Wickert, January 2015
    """
    f = np.logspace(Dmin,Dmax,Npts)
    w,H = signal.freqs(b,a,2*np.pi*f)
    plt.figure(figsize=fsize)
    if mode.lower() == 'db':
        plt.semilogx(f,20*np.log10(np.abs(H)))
        plt.xlabel('Frequency (Hz)')
        plt.ylabel('Gain (dB)')
        plt.title('Frequency Response - Magnitude')

    elif mode.lower() == 'phase':
        plt.semilogx(f,np.angle(H))
        plt.xlabel('Frequency (Hz)')
        plt.ylabel('Phase (rad)')
        plt.title('Frequency Response - Phase')

    elif mode.lower() == 'groupdelay':
        """
        Notes
        -----

        See freqz_resp() for calculation details.
        """
        theta = np.unwrap(np.angle(H))
        # Since theta for an FIR filter is likely to have many pi phase
        # jumps too, we unwrap a second time 2*theta and divide by 2
        theta2 = np.unwrap(2*theta)/2.
        theta_dif = np.diff(theta2)
        f_diff = np.diff(f)
        Tg = -np.diff(theta2)/np.diff(w)
        max_Tg = np.max(Tg)
        #print(max_Tg)
        plt.semilogx(f[:-1],Tg)
        plt.ylim([0,1.2*max_Tg])
        plt.xlabel('Frequency (Hz)')
        plt.ylabel('Group Delay (s)')
        plt.title('Frequency Response - Group Delay')
    else:
        print('Error, mode must be "dB" or "phase or "groupdelay"')
```

## Example: Discrete-Time Chebyshev Type I Bandpass Filter

```python
import sk_dsp_comm.iir_design_helper as iird
import sk_dsp_comm.fir_design_helper as fird
```

```python
b1,a1,sos1 = iird.IIR_bpf(200,250,300,350,0.1,60.0,1000,'butter')
b2,a2,sos2 = iird.IIR_bpf(200,250,300,350,0.1,60.0,1000,'cheby1')
```

```python
figure()
iird.freqz_resp_cas_list([sos1,sos2],'dB',1000)
ylim([-70,0])
grid();
figure()
iird.freqz_resp_cas_list([sos1,sos2],'groupdelay_t',1000)
grid();
figure()
iird.sos_zplane(sos2)
```

```python
b,a = signal.cheby1(5,.1,2*array([250,300])/1000,btype='bandpass')
```

```python
freqz_resp(b,a,mode='dB',fs=1000,fsize=(6,2))
grid()
ylim([-80,5]);
xlim([100,400]);
freqz_resp(b,a,mode='groupdelay_s',fs=1000,fsize=(6,2))
grid()
xlim([100,400]);
```

## Example: Continuous-Time Bessel Bandpass Filter

```python
bc,ac = signal.bessel(7,2*pi*array([10.0,50.0])*1e6,btype='bandpass',analog=True)
```

```python
freqs_resp(bc,ac,6,9,mode='dB',fsize=(6,2))
grid()
ylim([-80,5]);
freqs_resp(bc,ac,6,9,mode='groupdelay',fsize=(6,2))
grid()
```

## Second-Order Butterworth Lowpass Response

Consider a 3rd-order analog Butterworth is the $s$-domain having transfer function $H(s)$. Using the `scipy.signal` function `butter()` we find the coefficients to the rational transfer function of the form:
\begin{align}
    H(s) = \frac{\sum_{n=0}^M b_n s^n}{\sum_{n=0}^N a_n s^n}
\end{align}

```python
b3,a3 = signal.butter(3,2*pi*1,analog=True)
freqs_resp(b3,a3,-1,2,mode='dB',fsize=(6,2))
grid()
ylim([-80,5]);
freqs_resp(b3,a3,-1,2,mode='groupdelay',fsize=(6,2))
grid()
```

### Obtaining the Step Response via Simulation

Time domain simulation of continuous time system can be performed using the `signal.lsim()` function. You have to make sure the time step is sufficiently small relative to the filter bandwidth.

```python
t = arange(0,2,.0001)
xs = ss.step(t)
tout,ys,x_state = signal.lsim((b3,a3),xs,t)
plot(t,ys)
title(r'Third-Order Butterworth Step Response for $f_3 = 1$ Hz')
ylabel(r'Ste Response')
xlabel(r'Time (s)')
grid();
```

docs/source/nb_examples/Continuous-Time Signals and Systems using sigsys.ipynb


Lambda School Data Science, Unit 2: Predictive Modeling

# Regression & Classification, Module 2

## Objectives
- Go from simple regression (1 feature) to multiple regression (2+ features)
- Use regression metrics: MAE (Mean Absolute Error), RMSE (Root Mean Squared Error), and $R^2$ Score
- Understand how ordinary least squares regression minimizes the sum of squared errors
- Get and plot coefficients
- Explain why overfitting is a problem. Do train/test split

## Setup

```python
# If you're in Colab...
import os, sys
in_colab = 'google.colab' in sys.modules

if in_colab:
    # Install required python packages:
    # pandas-profiling, version >= 2.0
    # plotly, version >= 4.0
    !pip install --upgrade pandas-profiling plotly

    # Pull files from Github repo
    os.chdir('/content')
    !git init .
    !git remote add origin https://github.com/LambdaSchool/DS-Unit-2-Regression-Classification.git
    !git pull origin master

    # Change into directory for module
    os.chdir('module2')
```

```python
# Ignore this Numpy warning when using Plotly Express:
# FutureWarning: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead.
import warnings
warnings.filterwarnings(action='ignore', category=FutureWarning, module='numpy')
```

```python
# Use these functions later

%matplotlib inline
from matplotlib.patches import Rectangle
import matplotlib.pyplot as plt
import numpy as np
import plotly.express as px
import plotly.graph_objs as go
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression

def regression_3d(df, x, y, z, **kwargs):
    """
    Visualize linear regression in 3D: 2 features + 1 target

    df : Pandas DataFrame
    x : string, feature 1 column in df
    y : string, feature 2 column in df
    z : string, target column in df
    """

    # Plot data
    fig = px.scatter_3d(df, x, y, z, **kwargs)

    # Fit Linear Regression
    features = [x, y]
    target = z
    model = LinearRegression()
    model.fit(df[features], df[target])

    # Define grid of four points in the feature space
    xmin, xmax = df[x].min(), df[x].max()
    ymin, ymax = df[y].min(), df[y].max()
    coords = [[xmin, ymin],
              [xmin, ymax],
              [xmax, ymin],
              [xmax, ymax]]

    # Make predictions for the grid
    Z = model.predict(coords).reshape((2,2), order='F')

    # Plot predictions as a 3D surface (plane)
    fig.add_trace(go.Surface(x=[xmin,xmax], y=[ymin,ymax], z=Z))

    return fig

def regression_residuals(df, feature, target, m, b):
    """
    Visualize linear regression, with residual errors,
    in 2D: 1 feature + 1 target.

    Use the m & b parameters to "fit the model" manually.

    df : Pandas DataFrame
    feature : string, feature column in df
    target : string, target column in df
    m : numeric, slope for linear equation
    b : numeric, intercept for linear requation
    """

    # Plot data
    df.plot.scatter(feature, target)

    # Make predictions
    x = df[feature]
    y = df[target]
    y_pred = m*x + b

    # Plot predictions
    plt.plot(x, y_pred)

    # Plot residual errors
    for x, y1, y2 in zip(x, y, y_pred):
        plt.plot((x, x), (y1, y2), color='grey')

    # Print regression metrics
    mae = mean_absolute_error(y, y_pred)
    r2 = r2_score(y, y_pred)
    print('Mean Absolute Error:', mae)
    print('R^2:', r2)

def regression_squared_errors(df, feature, target, m, b):
    """
    Visualize linear regression, with squared errors,
    in 2D: 1 feature + 1 target.

    Use the m & b parameters to "fit the model" manually.

    df : Pandas DataFrame
    feature : string, feature column in df
    target : string, target column in df
    m : numeric, slope for linear equation
    b : numeric, intercept for linear requation
    """

    # Plot data
    fig = plt.figure(figsize=(7,7))
    ax = plt.axes()
    df.plot.scatter(feature, target, ax=ax)

    # Make predictions
    x = df[feature]
    y = df[target]
    y_pred = m*x + b

    # Plot predictions
    ax.plot(x, y_pred)

    # Plot squared errors
    xmin, xmax = ax.get_xlim()
    ymin, ymax = ax.get_ylim()
    scale = (xmax-xmin)/(ymax-ymin)
    for x, y1, y2 in zip(x, y, y_pred):
        bottom_left = (x, min(y1, y2))
        height = abs(y1 - y2)
        width = height * scale
        ax.add_patch(Rectangle(xy=bottom_left, width=width, height=height, alpha=0.1))

    # Print regression metrics
    mse = mean_squared_error(y, y_pred)
    rmse = np.sqrt(mse)
    mae = mean_absolute_error(y, y_pred)
    r2 = r2_score(y, y_pred)
    print('Mean Squared Error:', mse)
    print('Root Mean Squared Error:', rmse)
    print('Mean Absolute Error:', mae)
    print('R^2:', r2)

# Credit: Jake VanderPlas, Python Data Science Handbook, Chapter 5.3
# https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation.html#Validation-curves-in-Scikit-Learn
def PolynomialRegression(degree=2, **kwargs):
    return make_pipeline(PolynomialFeatures(degree),
                         LinearRegression(**kwargs))
```

# Predict Elections! 🗳

```python
import pandas as pd
import plotly.express as px

df = pd.read_csv('../data/bread_peace_voting.csv')
px.scatter(
    df,
    x='Average Recent Growth in Personal Incomes',
    y='Incumbent Party Vote Share',
    text='Year',
    title='US Presidential Elections, 1952-2016',
    trendline='ols',
)
```

#### Douglas Hibbs, [Background Information on the ‘Bread and Peace’ Model of Voting in Postwar US Presidential Elections](https://douglas-hibbs.com/background-information-on-bread-and-peace-voting-in-us-presidential-elections/)

> Aggregate two-party vote shares going to candidates of the party holding the presidency during the postwar era are well explained by just two fundamental determinants:

> (1) Positively by weighted-average growth of per capita real disposable personal income over the term.
> (2) Negatively by cumulative US military fatalities (scaled to population) owing to unprovoked, hostile deployments of American armed forces in foreign wars.

#### Data sources
- 1952-2012: Douglas Hibbs, [2014 lecture at Deakin University Melbourne](http://www.douglas-hibbs.com/HibbsArticles/HIBBS-PRESVOTE-SLIDES-MELBOURNE-Part1-2014-02-26.pdf), Slide 40
- 2016, Vote Share: [The American Presidency Project](https://www.presidency.ucsb.edu/statistics/elections)
- 2016, Recent Growth in Personal Incomes: [The 2016 election economy: the "Bread and Peace" model final forecast](https://angrybearblog.com/2016/11/the-2016-election-economy-the-bread-and-peace-model-final-forecast.html)
- 2016, US Military Fatalities: Assumption that Afghanistan War fatalities in 2012-16 occured at the same rate as 2008-12

> Fatalities denotes the cumulative number of American military fatalities per millions of US population the in Korea, Vietnam, Iraq and Afghanistan wars during the presidential terms preceding the 1952, 1964, 1968, 1976 and 2004, 2008 and 2012 elections. —[Hibbs](http://www.douglas-hibbs.com/HibbsArticles/HIBBS-PRESVOTE-SLIDES-MELBOURNE-Part1-2014-02-26.pdf), Slide 33

```python

```

## Linear Algebra

The same result that is found by minimizing the sum of the squared errors can be also found through a linear algebra process known as the "Least Squares Solution:"

\begin{align}
\hat{\beta} = (X^{T}X)^{-1}X^{T}y
\end{align}

Before we can work with this equation in its linear algebra form we have to understand how to set up the matrices that are involved in this equation.

### The $\beta$ vector

The $\beta$ vector represents all the parameters that we are trying to estimate, our $y$ vector and $X$ matrix values are full of data from our dataset. The $\beta$ vector holds the variables that we are solving for: $\beta_0$ and $\beta_1$

Now that we have all of the necessary parts we can set them up in the following equation:

\begin{align}
y = X \beta + \epsilon
\end{align}

Since our $\epsilon$ value represents **random** error we can assume that it will equal zero on average.

\begin{align}
y = X \beta
\end{align}

The objective now is to isolate the $\beta$ matrix. We can do this by pre-multiplying both sides by "X transpose" $X^{T}$.

\begin{align}
X^{T}y =  X^{T}X \beta
\end{align}

Since anything times its transpose will result in a square matrix, if that matrix is then an invertible matrix, then we should be able to multiply both sides by its inverse to remove it from the right hand side. (We'll talk tomorrow about situations that could lead to $X^{T}X$ not being invertible.)

\begin{align}
(X^{T}X)^{-1}X^{T}y =  (X^{T}X)^{-1}X^{T}X \beta
\end{align}

Since any matrix multiplied by its inverse results in the identity matrix, and anything multiplied by the identity matrix is itself, we are left with only $\beta$ on the right hand side:

\begin{align}
(X^{T}X)^{-1}X^{T}y = \hat{\beta}
\end{align}

We will now call it "beta hat" $\hat{\beta}$ because it now represents our estimated values for $\beta_0$ and $\beta_1$

### Lets calculate our $\beta$ coefficients with numpy!

```python
# This is NOT an objective you'll be tested on. It's just a demo.

# X is a matrix. Add constant for the intercept.
from statsmodels.api import add_constant
X = add_constant(df[feature].values)
print('X')
print(X)

# y is a column vector
y = df[target].values[:, np.newaxis]
print('y')
print(y)

# Least squares solution in code
X_transpose = X.T
X_transpose_X = X_transpose @ X
X_transpose_X_inverse = np.linalg.inv(X_transpose_X)
X_transpose_y = X_transpose @ y
beta_hat = X_transpose_X_inverse @ X_transpose_y

print('Beta Hat')
print(beta_hat)
```

```python

```

## Overfitting Demo

```python
# Read New York City property sales data, from first 4 months of 2019.
# Dataset has 23040 rows, 21 columns.
df = pd.read_csv('../data/NYC_Citywide_Rolling_Calendar_Sales.csv')
assert df.shape == (23040, 21)

# Change column names. Replace spaces with underscores
df.columns = [col.replace(' ', '_') for col in df]

# Remove symbols from SALE_PRICE string, convert to integer
df['SALE_PRICE'] = (
    df['SALE_PRICE']
    .str.replace('$','')
    .str.replace('-','')
    .str.replace(',','')
    .astype(int)
)

# Keep subset of rows:
# Tribeca neighborhood, Condos - Elevator Apartments,
# 1 unit, sale price more than $1, less than $35 million
mask = (
    (df['NEIGHBORHOOD'].str.contains('TRIBECA')) &
    (df['BUILDING_CLASS_CATEGORY'] == '13 CONDOS - ELEVATOR APARTMENTS') &
    (df['TOTAL_UNITS'] == 1) &
    (df['SALE_PRICE'] > 0) &
    (df['SALE_PRICE'] < 35000000)
)
df = df[mask]

# Data now has 90 rows, 21 columns
assert df.shape == (90, 21)

# Convert SALE_DATE to datetime
df['SALE_DATE'] = pd.to_datetime(df['SALE_DATE'], infer_datetime_format=True)

from ipywidgets import interact
import pandas as pd
from sklearn.linear_model import LinearRegression

# Read New York City property sales data, from first 4 months of 2019.
# Dataset has 23040 rows, 21 columns.
df = pd.read_csv('../data/NYC_Citywide_Rolling_Calendar_Sales.csv')
assert df.shape == (23040, 21)

# Change column names. Replace spaces with underscores
df.columns = [col.replace(' ', '_') for col in df]

# Remove symbols from SALE_PRICE string, convert to integer
df['SALE_PRICE'] = (
    df['SALE_PRICE']
    .str.replace('$','')
    .str.replace('-','')
    .str.replace(',','')
    .astype(int)
)

# Keep subset of rows:
# Tribeca neighborhood, Condos - Elevator Apartments,
# 1 unit, sale price more than $1, less than $35 million
mask = (
    (df['NEIGHBORHOOD'].str.contains('TRIBECA')) &
    (df['BUILDING_CLASS_CATEGORY'] == '13 CONDOS - ELEVATOR APARTMENTS') &
    (df['TOTAL_UNITS'] == 1) &
    (df['SALE_PRICE'] > 0) &
    (df['SALE_PRICE'] < 35000000)
)
df = df[mask]

# Data now has 90 rows, 21 columns
assert df.shape == (90, 21)

# Convert SALE_DATE to datetime
df['SALE_DATE'] = pd.to_datetime(df['SALE_DATE'], infer_datetime_format=True)

# Arrange X features matrix & y target vector
features = ['GROSS_SQUARE_FEET']
target = 'SALE_PRICE'
X = df[features]
y = df[target]
```

```python
from IPython.display import display, HTML
from sklearn.model_selection import train_test_split

# Train/Test Split
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=11)

# Repeatedly fit increasingly complex models, and keep track of the scores
polynomial_degrees = range(1, 10, 2)
train_r2s = []
test_r2s = []

for degree in polynomial_degrees:
    model = PolynomialRegression(degree)
    display(HTML(f'Polynomial degree={degree}'))

    model.fit(X_train, y_train)
    train_r2 = model.score(X_train, y_train)
    test_r2 = model.score(X_test, y_test)
    display(HTML(f'<b style="color: blue">Train R2 {train_r2:.2f}</b>'))
    display(HTML(f'<b style="color: red">Test R2 {test_r2:.2f}</b>'))

    plt.scatter(X_train, y_train, color='blue', alpha=0.5)
    plt.scatter(X_test, y_test, color='red', alpha=0.5)
    plt.xlabel(features)
    plt.ylabel(target)

    x_domain = np.linspace(X.min(), X.max())
    curve = model.predict(x_domain)
    plt.plot(x_domain, curve, color='blue')
    plt.show()
    display(HTML('<hr/>'))

    train_r2s.append(train_r2)
    test_r2s.append(test_r2)

display(HTML('Validation Curve'))
plt.plot(polynomial_degrees, train_r2s, color='blue', label='Train')
plt.plot(polynomial_degrees, test_r2s, color='red', label='Test')
plt.xlabel('Model Complexity (Polynomial Degree)')
plt.ylabel('R^2 Score')
plt.legend()
plt.show()
```

#### Jake VanderPlas, [_Python Data Science Handbook,_ Chapter 5.3](https://jakevdp.github.io/PythonDataScienceHandbook/05.03-hyperparameters-and-model-validation.html#The-Bias-variance-trade-off)


# The Basic OLG model

The OLG model is one of two main analytical frameworks for analyzing the basic intertemporal choice, consumption vs saving.

OLG is set to capture potential interaction of different generations of individuals in the marketplace hence, providing a tractable alternative to the infinite-horizon economic models

Imports and set magics:

```python
import numpy as np
from scipy import optimize
import sympy as sm
from sympy import lambdify
import matplotlib.pyplot as plt
import math as m
from scipy import stats as st
from scipy.optimize import fsolve
import time
#As we are using a lot of theory in this project, we want the equations to be presented well, why we import the init_session
from sympy import init_session
init_session()

# autoreload modules when code is run
%load_ext autoreload
%autoreload 2

# local modules
import modelproject
```

    IPython console for SymPy 1.5.1 (Python 3.7.6-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at https://docs.sympy.org/1.5.1/

    The autoreload extension is already loaded. To reload it, use:
      %reload_ext autoreload

# Model description

In this model, we consider a static economy with an infinite number og households, $i \in N$
In addition, population grows at a constant rate,

$L_{t}=L_{t-1}(1+n)$

Time, t, is discrete and runs to infinity and individuals born at time t live for the period $t$ til $t+1$, meaning the individuals are grouped in two generations, young and old.

We assume a general, separable utility function for the individuals given by

$U_t=u(c_{1t})+u(c_{2t+1})(1/(1+\beta))$

$\beta$ is added as a discount factor of the time-horizon.

Production side is given by competitive firms and a CRS aggregate production function meaning we have the following production criterias
$Y_{t}=F(K_{t},L_{t})$

$r_{t}=f'(k_{t})$

$w_{t} = f(k_{t})-k_{t}f'(k_{t})$

$\bullet$ NOTE: $k_{t}=K_{t}/L_{t}$

The savings by an individual of a generation is determined as a solution to

$max U_{t}$ s.t.

$w_{t}=s_{t}+c_{1t}$

$c_{2t+1}=s_{t}(1+r_{t+1})$

Where $c_{1t}$ is consumption of the individual born at t when young at date t and $c_{2t+1}$ consumption when old at date t+1. In this model, the setup is built such that the old individuals will rent their savings of time t as capital to firms at  t+1. Therefore they will eventually receive a gross rate of return given by $(1+r_{t})$. The second consumption constraint incorporates the fact that the individual will consume all his life-earnings. Hence, the lifetime constraint is given by

$w_{t}=c_{1+t}+c_{2t+1}/(1+r_{t+1})$

# Solution of the household problem

The solution is found by setting up the Lagrangian

$L(c_{1t},c_{2t+1},\lambda)= u(c_{1t})+u(c_{2t+1})(1/1+\beta)+\lambda[w_{t}-c_{1t}-(c_{2t+1}/(1+r_{t+1}))]$

FOCs:

$u'(c_{1t})=\lambda$

$(1/(1+\beta))u'(c_{t2+1})= \lambda/(1+r_{t+1})$

combining the above to find the Euler equation:

$(1+r_{t+1})/(1+\beta)u'(c_{2t+1}) = u'(c_{1t})$

The interpretation of Euler is that if the consumer gives up one unit of consumption in the young state, the marginal cost will be given by $u'(c_{1t})$ and the marginal benefit will be $(1+r_{t+1})$ units of consumption in next period, when old.

Next, we want to find the **optimal savings path**:

This is done by substituting budget constrain into the Euler eq. above. Hence, we find

$(1+r_{t+1})/(1+\beta)u'(c_{2t+1}) = u'(c_{1t})$ $\rightarrow$

$(1+r_{t+1})/(1+\beta)u'((1+r_{t+1})s_{t})=u'(w_{t} - s_{t})$

where optimal savings are a function of wage and the interest rate.

Knowing the decision of individuals, we can now examine the **Law of Motion** by aggregating the economy as follows:

As we are working with two periods, we know that the aggregate stock of capital at $t+1$ must equal the total savings of the economy at the time $t$ as well as the non-depreciated capital stock which is carried from $t$ and the 'dissavings' of same period.

This means, the aggregate capital level at time $t+1$ is given by the following:

$K_{t+1}=S_{t}+(1-\delta)K_{t}-K_{t}$ $\rightarrow$

$K_{t+1}=S_{t}-\delta K_{t}$

Remember, $k_{t} \equiv K_{t}/L_{t}$. Hence,

$k_{t+1}(1+n) = s_{t}-\delta k_{t}$

In short, inserting the findings above into the capital accumulation and for simplicity setting $\delta =0$ we now have the law of motion per worker in the entire economy defined by

$k_{t+1} (1+n)=s(f(k_{t})-f'(k_{t})k_{t}, f'(k_{t+1}))$

{note: the intuition above is found at http://web.econ.ku.dk/okocg/MAT-OEK/Mak%C3%98k2/Mak%C3%98k2-2016/Forel%C3%A6sninger/Ch%203-2016-1.pdf as well as in our lecture slides from the Macroeconomics course at Copenhagen University}

Now, we wish to solve the model using **sympy**

```python
#First, defining the setting, here the variables we want to use

#Households definitions
beta = sm.symbols('beta')
c1 = sm.symbols('c_{1t}')
c2 = sm.symbols('c_{2t+1}')
lmda = sm.symbols('lambda')
#Savings definitions
r = sm.symbols('r_{t+1}')
w = sm.symbols('w_t')
#Capital stock definitions
s = sm.symbols('s_t')
kt = sm.symbols('k_t')
k1 = sm.symbols('k_{t+1}')
delta = sm.symbols('delta')
n = sm.symbols('n')

```

Now, we want to define the utility function. In this case, we use the log utility

```python
#We start by setting a standard utility, then we specify the form afterwards

u_std = sm.Function('u')

def log_u(c):
    return sm.log(c)

def U(c1, c2, beta, u):
    return u(c1) + 1 / (1 + beta) * u(c2)
#Print
U(c1, c2, beta, u = u_std)
```

Now, the budget constraints are set up to calculate the intertemporal consumption constraint

```python
#Defining budget constraints
c_1budget = sm.Eq(w, c1 + s)
c_2budget = sm.Eq(c2, (1+r)*s)

#Printing the budget constraints
c_1budget, c_2budget

```

Next, we define the intertemporal consumption constraints

```python
#Intertemporal consumption constraint defined
ib = c_1budget.subs(s, sm.solve(c_2budget, s)[0])
#Print intertemporal budget constraint
ib
```

Setting up the Lagrangian, deriving FOCs

```python
#Lagrangian
l = U(c1,c2,beta, u = u_std) - lmda *  (ib.rhs - w)

#Print Lagrangian
l

#Deriving FOCs
dc1    = sm.diff(l, c1)
dc2    = sm.diff(l, c2)
dlmda = sm.diff(l, lmda)
#Print FOCs
dc1, dc2, dlmda
```

Now we find the **Euler Equation**

```python
#Defining the Euler Equation
def euler_equation(dc1, dc2, u, c1, lmda):
    x = dc2.subs(lmda, sm.solve(dc1, lmda)[0])

    euler_eq = sm.Eq(sm.Derivative(u(c1)),
                     sm.solve(x, sm.Derivative(u(c1)))[0]
                    )
    return euler_eq

#Calling make_euler_equation to calculate the Euler eq
euler = euler_equation(dc1, dc2, u = u_std, c1 = c1, lmda = lmda)

#Printing result
euler

euler = euler.subs(c1, w - s).subs(c2, c_2budget.rhs)
#Printing result
euler

```

$\displaystyle \left. \frac{d}{d c_{1t}} u{\left(c_{1t} \right)} \right|_{\substack{ c_{1t}=- s_{t} + w_{t} }} = \frac{\left(r_{t+1} + 1\right) \left. \frac{d}{d c_{2t+1}} u{\left(c_{2t+1} \right)} \right|_{\substack{ c_{2t+1}=s_{t} \left(r_{t+1} + 1\right) }}}{\beta + 1}$

Euler implicitly determines the value of $s_{t}$.

Now we want to determine the development in capital.

```python
#First we set the functional form of $u_{t}$ using the $log$ function as this enables for further examination.
euler = euler.replace(u_std, log_u).doit()
#Print
euler

```

Now we define the values and setting for the **firms** in the economy.
To do this, we explore the aggregate production function, normalizing this by using the per capita function as this enables us to better calculate expressions for **interest rates** and **wages** in equilibrium.

```python
#Production function
Y = sm.symbols('Y_t')
K = sm.symbols('K_t')
L = sm.symbols('L_t')
#including y_t and alpha
y = sm.symbols('y_t')
alpha = sm.symbols('alpha')
#deriving the production function by using the values above

prod_f = sm.Eq(Y,K**alpha * L**(1-alpha))

#Normalizing
normprod_f = sm.Eq(y, kt**alpha)
#Print
normprod_f
```

```python
ir = sm.Eq(r, sm.Derivative(normprod_f.rhs, kt)).doit()
w_t = sm.Eq(w, normprod_f.rhs - kt*sm.Derivative(normprod_f.rhs, kt)).doit()

#Print results of interest rate and wage
ir, w_t
```

Solving for capital period t+1(capital evolution path)

```python
k_1 = sm.Eq(k1, 1/(1+n)* (s - delta*kt) )
#Print
k_1
```

We now derive the **transition path** as follows

```python
k_sav = sm.solve(euler, s)[0].subs(w, w_t.rhs)
# Substituting
if r in k_sav.atoms():
    k_sav = k_sav.subs(r, ir.rhs)

t_path = k_1.subs(s, k_sav)
#Print
t_path
```

To make a visualization of the transition path, we now specify a number of parameters and define a function of the transition path

```python
#Defining the transition path
transition_curve = sm.lambdify((kt, beta, delta, n, alpha,r ), t_path.rhs)
def transition_c(kt, beta, delta,alpha, n, r = 0):
    return transition_curve(kt, beta, delta, alpha, n, r)
#Defining equilibrium
eq = sm.lambdify((beta, delta, alpha, n),sm.solve(t_path.subs(k1, kt), kt)[0])
def eqm(beta, delta,alpha, n,):
    return eq(beta, delta, alpha, n)

```

Letting alpha vary to show how this affects the transition path, plotting the graph

```python
#Defining variables
ex = np.linspace(0,3,1000)
_b = 0.05
_d = 0.02
_n = 0.2
#Plotting figure
plt.figure(figsize=(14,7))
#Plot 45 degree line
plt.plot(ex, ex, color = 'blue')
#Defining moving function
for _a in np.linspace(0.01, 0.2,5):
    sol = [transition_c(kt = x, alpha = _a, beta = _b, delta = _d, n = _n) for x in ex]
    ks = eqm(alpha = _a, beta = _b, n = _n, delta = _d)
    plt.plot(ex, sol, color = 'red', alpha = 1)
    plt.annotate(f'$\\alpha$={round(_a,2)}', xy= (ks + 0.01, ks - 0.01))
#Standard settings
plt.xlabel('$k_t$', size =15)
plt.ylabel('$k_{t+1}$', size=15)
plt.title('Transition curves\n', size = 20)
plt.xlim(0,0.4)
plt.ylim(0,0.4)

```

For a lower alpha, the higher is the steady state. The reason behind this, that a higher alpha the more productive is the capital and therefor a higher demand for capital per worker. This means that economy reach the steady state faster. A plausible value would be 1/3 on capital and 1-1/3 n labour. As we can see from our law of motion equation then a higher n (higher population) reduce the capital accumulation and thereby have a lower steady state.

To further characterize the equilibrium paths, we construct the transition diagram in the plane $(k_{t},k_{t+1})$ examining the convergence towards **steady state**

```python
#Setting variable values
_r = 0.05
_d = 0.05
_a = 0.5

#Defining k steady state
ksteady = 0.0002
ex = range(8)
ex2 = np.linspace(0,1,500)
#Transition
out = list()
for _ in ex:
    ksteady = transition_c(kt = ksteady, alpha = _a, beta = _b, delta = _d, n = _n)
    out.append(ksteady)

```

Now we plot the figure in order to illustrate the convergence towards steady state

```python
res = [transition_c(kt = x, alpha = _a, beta = _b, delta = _d, n = _n) for x in ex2]
#Plotting figure
plt.figure(figsize=(14,7))
#Plot 45 degree line
plt.plot(ex, ex, color = 'blue')
#Plotting the steady state convergence line
plt.plot(ex2, res, color = 'red', alpha = 1)
plt.step(out[:-1], out[1:], where = 'post', color = 'green', linestyle = '--', alpha = 0.8)
plt.scatter(out[:-1], out[1:], color = 'red', alpha = 0.8)
#Standard settings
plt.xlabel('$k_t$', size = 15)
plt.ylabel('$k_{t+1}$', size = 15)
plt.xlim(0,0.2)
plt.ylim(0,0.2)
plt.title('Convergence towards steady state\n', size = 20)

```

    Text(0.5, 1.0, 'Convergence towards steady state\n')

Outside the steady-state $k_{t}$ does not grow at a constant rate. Hence, the economy will in time approach the balanced growth path as seen on the graph.

# Extended OLG model

Now we add a government to our model to solve an example of the model using random choosen values. The government collects taxes on both capital and labour. Taxes are given by $\tau$.

We now consider the household problem

$max$           $u(c_{2t+1})+\beta u(c_{1t})$

s.t.

$c_{2t+1}+s=(1-\tau)w+t_{2t+1}$

$c_{1t}=R+t_{1t}$

where $R$ is the after-tax gross interest rate which is given by

$R=(1+(1-\tau)r)$

Instead of having to substitute and solve for an equation of one unknown, as we did in the beginning, we now want to use the **Gauss-Seidel** method for solving the problem.

This mdel is an iterative technique for solving a square system of $n$ linear equations with an unknown vaiable, $x$:

$Ax = b$

Which is defined by the iteration:

$Lx^{k+1} =b-Ux^{k}$

Where the variable $x^{k}$ defines the $k^{th}$ approximation of x as well as $x^{k+1}$ is the $k+1$ iteration of x. The matrix, $A$, is decomposed into the component $L$ and a strictly upper triangular component denoted U, meaning $A = L+U$.

The Gauss-Seidel method will solve the lhs of the expression of
$Lx^{k+1} =b-Ux^{k}$ which can be written as:

$x^{k+1} =L^{-1} (b-Ux^{k})$

However, as L is of triangular form, the elements of the variable $x^{k+1}$ can be computed sequentially using the forward substitution given by:

$x_i^{k+1} = 1/a_{ii}(b_{i}- \sum_{j=1}^{i-1} a_{ij}x_{j}^{k+1} - \sum_{j=i+1}^{n} a_{ij}x_{j}^{k} ), i=1,...,n$

Using this method, we start with an initial guess of the kapital in time 1t, denoted $K_{guess}$.
Next, we solve for the parameters $q, w and R$

$q = \alpha AK^{\alpha-1}_{1t} L^{1-a}$

$w=(1-\alpha)K^{\alpha}_{1t} L^{1-\alpha}$

$R=1+(1-\tau)(q-\delta)$

We then solve for $s*$ which is the optimal savings of the household, given by

$s* =N_{2t+1} ((\beta R((1-\tau)w+t_{2t+1})-t_{1t})/((1+\beta)R))$

Next, we aggregate over all households of the economy to find $K_{new}$

$K_{new}=N_{2t+1} \bullet s*$

Lastly, we calculate errors and update capital as follows

$K_{guess} = \lambda K_{new} +(1-\lambda)K_{guess}$ where $0< \lambda <1$ is an updating parameter.

{note: the intuition behind this model is found at https://www.sciencedirect.com/topics/engineering/gauss-seidel-method}

```python
#Set parameters
A_t     = 1
alpha   = 0.33
beta    = 0.8
delta   = 0.0
L_t     = 1
N_2     = 1.0
N_1     = 1.0
tau_L   = 0.22
tau_K   = 0.16
t_2     = 0.0
t_1     = 0.0
#

#Initial guess of capital
error       = 100
guess_lmda  = 0.4
K_guess     = 0.4
iter        = 1
#Iterating to find the values
while (iter<300) or (error>0.001):
    # Now we want to solve for q and w by
    q = alpha*A_t*K_guess**(alpha-1)
    w = (1-alpha)*A_t*K_guess**alpha
    R = 1 + (1-tau_K)*(q - delta)
    K_new = N_2* (beta*R*((1-tau_L)*w + t_2) - t_1)/((1+beta)*R)
    # Calculate discrepancy between old and new capital stock
    error = abs(K_guess-K_new)/K_guess
    # Update capital stock
    K_guess    = guess_lmda*K_new + (1-guess_lmda)*K_guess
    iter = iter +1

#Results
Ks = K_new
qs = q
Rs = R
rs = qs - delta
ws = w
Ys = A_t*Ks**alpha*L_t**(1-alpha)

```

Now we want to find solutions for the entire economy

```python
#Household optimal consumption per period
ss = Ks/N_2
c2s= (1-tau_L)*ws + t_2 - ss
c1s= Rs*ss + t_1

# Residual consumption of the government
Gs = N_2*tau_L*ws + N_1*tau_K*rs*ss

# Finally, aggregate the consumption of the household
Cs = N_2*c2s + N_1*c1s

# To ensure, the above holds, we run a fast check on the condition of the goods market and resoure constraint
ARC = Ys - delta*Ks - Cs - Gs

print("The results using the Gauss-Seidel method are as follows")

print("K* = {:6.4f}".format(Ks))
print("q* = {:6.4f}".format(qs))
print("r* = {:6.4f}".format(rs))
print("R* = {:6.4f}".format(Rs))
print("w* = {:6.4f}".format(ws))
print("Y* = {:6.4f}".format(Ys))
print("-------------------------")
print("ARC = {:6.4f}".format(ARC))
print("Number of iterations = " +str(iter))
print("-------------------------")
print("Optimal consumption of the household is:")
print("s* = ", ss)
print("c1* = ", c1s)
print("c2* = ", c2s)
print("Whereas the residual consumption of the government is:")
print("G* = ", Gs)
print("And the aggregate consumption of the household is:")
print("C* = ", Cs)

```

    The results using the Gauss-Seidel method are as follows
    K* = 0.1132
    q* = 1.4208
    r* = 1.4208
    R* = 2.1935
    w* = 0.3264
    Y* = 0.4872
    -------------------------
    ARC = 0.0000
    Number of iterations = 300
    -------------------------
    Optimal consumption of the household is:
    s* =  0.1131649073144145
    c1* =  0.24822222207139938
    c2* =  0.14145613414301814
    Whereas the residual consumption of the government is:
    G* =  0.09754139399115577
    And the aggregate consumption of the household is:
    C* =  0.3896783562144175


# Обратные связи в контуре управления

Для рассмотренных в предыдущих лекциях регуляторов требуется оценивать состояние объекта управления. Для построения таких оценок  необходимо реализовать обратные связи в контуре управления. На практике для этого используются специальные устройства: датчики.

# Случайные величины

Случайная величина -- это переменная, значениее которой определяется в результате эксперимента, подверженного влиянию случайных факторов. Случайные величины характеризуются функция плотности вероятности
\begin{equation}
p(a \leq \xi \leq b) = \int_{a}^{b} p(\xi) \,d\xi
\end{equation}
которая определяет вероятность попадания значения $\xi$ в интервал $[a \quad b]$.

Математическим ожиданием случайной величины называется
\begin{equation}
\mathbb{E}[\xi] = \int_{-\infty}^{\infty} \xi \cdot p(\xi) \,d\xi
\end{equation}

Дисперсия случайной величины
\begin{equation}
\mathbb{D}[\xi] = \mathbb{E}[\left(\xi - \mathbb{E}[\xi]\right)^2]
\end{equation}

Ковариация двух случайных величин
\begin{equation}
    \Sigma[\xi_1, \xi_2] = \mathbb{E}[(\xi_1 - \mathbb{E}[\xi_1]) (\xi_2 - \mathbb{E}[\xi_2])]
\end{equation}

```python
# [ПРИМЕР 1] Измерения случайной величины
import numpy as np

xi = np.random.random()
print(xi)
```

    0.030119762146224804

```python
# [ПРИМЕР 2] Распределение случайной величины
import numpy as np
import matplotlib.pyplot as plt

np.random.seed(200)
N = 999
xi = np.random.random(N)

# plot xi
fig1 = plt.figure(figsize=(10,5))
ax1 = fig1.add_subplot(1,1,1)

ax1.set_title("Random variable xi")
ax1.plot(range(N), xi, color = 'b')
ax1.set_ylabel(r'xi')
ax1.set_xlabel(r'n')
ax1.grid(True)
```

```python
# [ПРИМЕР 3] Нормальное распределение
import numpy as np
import scipy.stats as st
import matplotlib.pyplot as plt

import numpy as np
x = np.random.normal(3, 1, 100000)

_, bins, _ = plt.hist(x, 50, density = True, alpha = 0.5)

mu, sigma = st.norm.fit(x)
best_fit_line = st.norm.pdf(bins, mu, sigma)

plt.plot(bins, best_fit_line)
```

```python
# [ПРИМЕР 4] Математическое ожидание и дисперсия
import numpy as np
import matplotlib.pyplot as plt

#np.random.seed(200)
N = 999
xi = np.random.normal(0, 1, N)
e = np.mean(xi)
print("Expected value: ", (e))
d = np.mean((xi - e)**2)
print("Variance: ", (d))

# plot xi
fig1 = plt.figure(figsize=(10,5))
ax1 = fig1.add_subplot(1,1,1)

ax1.set_title("Random variable xi")
ax1.plot(range(N), xi, color = 'b')
ax1.set_ylabel(r'xi')
ax1.set_xlabel(r'n')
ax1.set_ylim([-4 * d, 4 * d])
ax1.grid(True)
```

```python
mean = [0, 0]
covariance_mat = [[1., -0.5], [-0.5, 1.]]
x, y = np.random.multivariate_normal(mean, covariance_mat, 10000000).T

plt.figure(figsize = (3, 3 ))
plt.hist2d(x, y, bins=(1000, 1000), cmap = plt.cm.jet)
plt.subplots_adjust(bottom = 0, top = 1, left = 0, right = 1)
plt.xlim(-5, 5)
plt.ylim(-5, 5)
plt.show()
```

# Фильтр Калмана (Линейные системы)

Модель системы (процесса):
$$x_k = A_k \cdot x_{k-1} + B_k \cdot u_k + w_k,$$

где $w_k$~$N(0, Q_k)$ - нормально распределённый случайный процесс, характеризующийся нулевым математическим ожиданием и матрицей ковариации $Q_k$.

Модель наблюдений (измерений):
$$y_k = H_k \cdot x_k + v_k,$$

где $v_k$ - нормально распределённый случайный процесс, характеризующийся нулевым математическим ожиданием и матрицей ковариации $R_k$.

Требуется получить оценку вектора состояния системы $\hat{x}_k$, зная аналогичную оценку на предыдущем шаге ($\hat{x}_{k-1}$), выход системы (вектор измерений) $y_k$ и вектор управляющих параметров $u_k$.

## Прогноз

Прогноз (экстраполяция) вектора состояния с помощью модели процесса:
$$\overline{x}_k = A_k \cdot \hat{x}_{k-1} + B_k \cdot u_k ,$$

Прогноз матрицы ковариации ошибок:
$$\overline{P}_k = A_k\cdot \hat{P}_{k-1}\cdot A_k^T + Q_k$$

## Коррекция

Вычисление коэффициента усиления Калмана:
$$K_k = \overline{P}_k \cdot H_{k}^T \cdot \left(H_k\cdot \overline{P}_k\cdot H_k^T + R_k\right)^{-1}$$

Оценка матрицы ковариации ошибки:
$$\hat{P}_k = \left(I - K_k\cdot H_k\right)\cdot \overline{P}_k$$

Оценка вектора состояния:
$$\hat{x}_k = \overline{x}_k + K_k\cdot\left(y_k - H_k\cdot\overline{x}_k\right)$$

# Пример

Рассмотрим задачу о движении точки по прямой под действием случайных (постоянных внутри каждого такта управления) ускорений.

Вектор состояния системы включает в себя координату $x$ и скорость $v$, то есть $x_k = [x \quad v]^T$. Уравнения движения:

$$x_k = A_k \cdot x_{k-1} + G_k \cdot a_k,$$
где
$$A_k =
\begin{pmatrix}
1 & \Delta t \\
0 & 1
\end{pmatrix}, \quad
G_k = \begin{pmatrix}
0.5 \Delta t^2 \\
\Delta t
\end{pmatrix}.
$$

Матрица ковариации шума процесса:
$$Q = G\cdot G^T \cdot \sigma_a^2,$$
где $\sigma_a$ характеризует случайное распределение ускорений.

На каждом такте процесса измеряется координата точки. Таким образом, матрица наблюдения
$$H =
\begin{pmatrix}
1 & 0
\end{pmatrix},$$

а модель наблюдений
$$y_k = H \cdot x_k + v_k,$$

где $v_k$ - нормально распределённый шум измерений ($\sigma_m$). Матрица ковариации шума измерений:
$R = \left[ \sigma_m^2 \right].$

```python
import numpy as np
import matplotlib.pyplot as plt
from random import normalvariate

class RealWorld:
    def __init__(self, sigma_acc, sigma_meas, dt):
        self.time = 0.0
        self.time_step = dt
        self.position = 0.0
        self.sigma_acc = sigma_acc
        self.velocity = 0.1
        self.measurement = None

        # шум измерений
        self.sigma_meas = sigma_meas

    def measure(self):
        if self.measurement == None:
            self.measurement = self.position + normalvariate(0, self.sigma_meas)

        return self.measurement

    def step(self):
        self.time += self.time_step
        acceleration = normalvariate(0, self.sigma_acc)
        self.position += self.velocity * self.time_step + 0.5 * acceleration * self.time_step**2
        self.velocity += acceleration * self.time_step

        self.measurement = None
```

# Пример

Рассмотрим задачу о движении точки по прямой под действием случайных (постоянных внутри каждого такта управления) ускорений.

Вектор состояния системы включает в себя координату $x$ и скорость $v$, то есть $x_k = [x \quad v]^T$. Уравнения движения:

$$x_k = A_k \cdot x_{k-1} + G_k \cdot a_k,$$
где
$$A_k =
\begin{pmatrix}
1 & \Delta t \\
0 & 1
\end{pmatrix}, \quad
G_k = \begin{pmatrix}
0.5 \Delta t^2 \\
\Delta t
\end{pmatrix}.
$$

Матрица ковариации шума процесса:
$$Q = G\cdot G^T \cdot \sigma_a^2,$$
где $\sigma_a$ характеризует случайное распределение ускорений.

На каждом такте процесса измеряется координата точки. Таким образом, матрица наблюдения
$$H =
\begin{pmatrix}
1 & 0
\end{pmatrix},$$

а модель наблюдений
$$y_k = H \cdot x_k + v_k,$$

где $v_k$ - нормально распределённый шум измерений ($\sigma_m$). Матрица ковариации шума измерений:
$R = \left[ \sigma_m^2 \right].$

```python
sigma_a = 0.01 # нормально распределённые ускорения

# дисперсия шума измерений
sigma_measurement = 10.

world = RealWorld(sigma_a, sigma_measurement, 0.5)

#оператор эволюции
A = np.array([[1., world.time_step],[0., 1.]])
G = np.array([0.5 * world.time_step**2, world.time_step])

# шум процесса
Q = np.outer(G, G) * sigma_a**2

# матрица ковариации ошибки
position_uncertainty = 1.
velocity_uncertainty = 1.
P = np.array([[position_uncertainty, 0.],[0., velocity_uncertainty]])

#модель наблюдений, измеряем только положение
H = np.array([1., 0.])

# дисперсия шума измерений
R = np.array([sigma_measurement**2])
```

```python
episode_len = 1000
data = np.zeros((6, episode_len))

for i in range(episode_len):
    world.step()
    measurement = world.measure()

    if i == 0: # первое измерение
        x_est = np.array([measurement, 0.])
    elif i == 1: # второе измерение
        x_est = np.array([measurement, ( measurement - data[4, i-1] ) / world.time_step])
    else: # если i >=2 начинаем применять модель
        ##################################################################
        # прогноз
        vel_est = data[5, i-1]
        pos_est = data[4, i-1] + vel_est * world.time_step
        x_pred = np.array([pos_est, vel_est])

        # прогноз матрицы ковариации ошибки
        P_pred = A.dot(P).dot(A.T) + Q

        ##################################################################
        # Коррекция
        K = P_pred.dot(H.T) / (H.dot(P_pred).dot(H.T) + R)
        P = (np.eye(2) - K.dot(H)).dot(P_pred)

        x_est = x_pred + K.dot(measurement - H.dot(x_pred))

    data[:, i] = np.array([world.time, world.position, world.velocity, measurement, x_est[0], x_est[1]])

# plot
fig1 = plt.figure(figsize=(16,8))
ax1 = fig1.add_subplot(1,2,1)
ax2 = fig1.add_subplot(1,2,2)

# r
ax1.set_title("position")
ax1.plot(data[0, :], data[3, :], 'k', label = 'pos_mes')
ax1.plot(data[0, :], data[1, :], 'r', label = 'pos_world')
#ax1.plot(data[0, :], data[4, :]-data[1, :], 'g', label = 'pos_est')

ax1.set_ylabel(r'r')
ax1.set_xlabel(r't, [s]')
ax1.grid(True)
ax1.legend()

# v
ax2.set_title("velocity")
ax2.plot(data[0, :], data[2, :], 'r', label = 'v')
#ax2.plot(data[0, :], data[5, :], 'g', label = 'v_est')

ax2.set_ylabel(r'v')
ax2.set_xlabel(r't, [s]')
ax2.grid(True)
ax2.legend()

fig2 = plt.figure(figsize=(16,8))
ax3 = fig2.add_subplot(1,2,1)
ax4 = fig2.add_subplot(1,2,2)

# r
ax3.set_title("position")
ax3.plot(data[0, :], data[1, :], 'r', label = 'pos_world')
ax3.plot(data[0, :], data[4, :], 'g', label = 'pos_est')

ax3.set_ylabel(r'r')
ax3.set_xlabel(r't, [s]')
ax3.grid(True)
ax3.legend()

# v
ax4.set_title("velocity")
ax4.plot(data[0, :], data[2, :], 'r', label = 'v')
ax4.plot(data[0, :], data[5, :], 'g', label = 'v_est')

ax4.set_ylabel(r'v')
ax4.set_xlabel(r't, [s]')
ax4.grid(True)
ax4.legend()
```

# Модель измерений
Рассмотрим движение системы
\begin{equation}
\dot{x} = f(x).
\end{equation}

Вектор измерений $z$ зависит от состояния системы, а также содержит случайную компоненту
\begin{equation}
z(x) = h(x) + \xi.
\end{equation}
Функция $h(x)$ связывает состояние системы с измерением датчика.

Например,
если датчик GNNS-приемника в одномерной задаче движения тележки по рельсам
смещен от центра тележки $x$ на расстояние $r$,
можно записать
\begin{equation}
h(x) = x + r.
\end{equation}

# Расширенный фильтр Калмана (Extended Kalman Filter)

Расширенный фильтр Калмана подразумевает, как правило, нелиинейную модель системы (процесса):

\begin{equation}
\dot{x} = f(x) + w(t).
\end{equation}

Шум $w$ имеет нормальное распределение, нулевое математическое ожидание и матрицу ковариации $Q$.

Модель наблюдений также может описываться нелинейным уравнением
\begin{equation}
y = h(x) + v(t),
\end{equation}
где $v$ имеет нормальное распределение, нулевое математическое ожидание и матрицу ковариации $R$. Однако, как правило, считается, что измерения обрабатываются фильтром периодически с частотой такта управления, поэтому модель записывают в видее соотношенийми между вектором состояния на момент получения измерений $x_k = x(t_k)$ и набором измерений $y_k = y(t_k)$:
\begin{equation}
y_k = h(x_k) + v_k.
\end{equation}

Алгоритм снова выполняется в два этапа - прогноз и коррекция.

## Прогноз

Прогноз (экстраполяция) вектора состояния с помощью нелинейной модели процесса:
$$\overline{x}_k = \hat{x}_{k-1} + \int_{t_{k-1}}^{t_k} f(x)dt,$$

Прогноз матрицы ковариации ошибок:
\begin{equation}
\overline{P}_k = \Phi_k \cdot \hat{P}_{k-1} \cdot \Phi_k^T + Q,
\end{equation}
где
\begin{equation}
\Phi_k = I + F\cdot \Delta t = I +\frac{\partial f(x)}{\partial x}\cdot(t_k -t_{k-1})
\end{equation}

## Коррекция

Здесь отличие от линейного алгоритма в необходимости линеаризовать модель наблюдений, чтобы получить матрицу $H$:
\begin{equation}
H_k = \frac{\partial h(x}{\partial x}.
\end{equation}

Вычисление коэффициента усиления Калмана:
$$K_k = \overline{P}_k \cdot H_{k}^T \cdot \left(H_k\cdot \overline{P}_k\cdot H_k^T + R_k\right)^{-1}$$

Оценка матрицы ковариации ошибки:
$$\hat{P}_k = \left(I - K_k\cdot H_k\right)\cdot \overline{P}_k$$

Оценка вектора состояния:
$$\hat{x}_k = \overline{x}_k + K_k\cdot\left(y_k - h(\overline{x}_k)\right)$$

# Пример: Вращение твёрдого тела с неподвижным центром масс

Ориентация твердого тела описывается кватернионом $q$, задающим положение связанной с телом системы координат относительно некоторой неподвижной системы координат. Говорят, что кватернион $q^{\mathrm{BI}}$ задает ориентацию некоторого базиса (B) относительно некоторого другого базиса (I), если представление любого вектора $\mathbf{v}$ в этих системах координат определяется соотношением:
$$\mathbf{v}^{\mathrm{B}} = q^{\mathrm{BI}}\circ\mathbf{v}^{\mathrm{I}}\circ \tilde{q}^{\mathrm{BI}}$$

Кинематические уравнения твердого тела записываются как:

\begin{equation}\label{eq:quat}\tag{1}
\dot{q} = \frac{1}{2}q\circ \boldsymbol{\omega},
\end{equation}

где $q$ - кватернион ориентации тела, $\boldsymbol{\omega}$ - угловая скорость тела в рпоекциях на связанные с телом оси.

Модель движения твёрдого тела с неодвижной точкой дополняется динамическими уравнениями Эйлера

\begin{equation}\label{eq:euler}\tag{2}
\mathbf{J}\cdot \dot{\boldsymbol{\omega}} + \boldsymbol{\omega} \times \mathbf{J}\cdot \boldsymbol{\omega} = \mathbf{T},
\end{equation}

где $\mathbf{J}$ - тензор инерции тела, $\mathbf{T}$ - главный момнет сил, действующих на тело.

Таким образом, вектор состояния состояит из 4х компонент кватерниона ориентации и 3х компонент вектора угловой скорости. А модель процесса состоит из уравнений \eqref{eq:quat} и \eqref{eq:euler}.

```python

```

Lecture Jupyter Notebooks/Lecture 8 Kalman Filter.ipynb


### Basic use of the GIPL model

Before you begin, install:

```conda install -c conda-forge pymt pymt_gipl seaborn```

**GIPL** (Geophysical Institute Permafrost Laboratory) is an implicit finite difference one-dimensional heat flow numerical model.

\begin{equation}
\frac{\partial H(x, t)}{\partial \tau}=\frac{\partial}{\partial x}\left(k(x, t) \frac{\partial t(x, \tau)}{\partial x}\right)
\end{equation}

The model uses fine vertical resolution grid which preserves the latent-heat effects in the phase transition zone, even under conditions of rapid or abrupt changes in the temperature fields. It includes upper boundary condition (usually air temperature), constant geothermal heat flux at the lower boundary, and initial temperature distribution with depth. The other inputs are snow depth, snow thermal conductivity, etc. The core output is temperature distributions at different depths.

**References**

Marchenko, S., Romanovsky, V., & Tipenko, G. (2008, June). Numerical modeling of spatial permafrost dynamics in Alaska. In Proceedings of the ninth international conference on permafrost (Vol. 29, pp. 1125-1130). Institute of Northern Engineering, University of Alaska Fairbanks.

Jafarov, E. E., Marchenko, S. S., and Romanovsky, V. E.: Numerical modeling of permafrost dynamics in Alaska using a high spatial resolution dataset, The Cryosphere, 6, 613-624, https://doi.org/10.5194/tc-6-613-2012, 2012.

----------------
Some libs should be imported.

**pymt.models** is required.

**others** are used for implementing and plotting

```python
import pymt.models
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import matplotlib.colors as mcolors
from matplotlib.colors import LinearSegmentedColormap
sns.set(style='whitegrid', font_scale= 1.2)
```

**Import the GIPL model**

```python
gipl = pymt.models.GIPL()
gipl.name
```

Call the setup method to provide a default GIPL configuration file.

```python
defaults = gipl.setup('.')
print(defaults)
```

**Initialze the model**

```python
gipl.initialize('gipl_config.cfg')
```

**Get the soil depth in the model**

It is used for plotting.

```python
# List input and output variable names.
print(gipl.output_var_names)
print(gipl.input_var_names)

# Get soil depth: [unit: m]
depth = gipl.get_grid_z(2)
n_depth = int(len(depth))
```

**Get the length of forcing data**

```python
# Get the length of forcing data
ntime = int(gipl.end_time)

# Define a variable to store soil temperature through the time period

tsoil = np.zeros((n_depth, ntime)) * np.nan

print('Final soil temperatures will be ', tsoil.shape)
```

**Run the model and plot the results**

```python
fig = plt.figure(figsize=[10,4.5])
ax = fig.add_subplot(1,2,1)
plt.ylim([15,0])
plt.xlim([-20,20])
plt.xlabel('Soil Temperature ($^oC$)')
plt.ylabel('Depth (m)')
plt.plot([0,0],[15,0],'k--')

for i in np.arange(int(ntime)):
    gipl.update() # Update Once
    tsoil[:,i] = gipl.get_value('soil__temperature') # save all temperature to a matrix
    plt.plot(tsoil[depth>=0,i], depth[depth>=0],color = [0.7,0.7,0.7], alpha = 0.1) # plot result of each time

plt.plot(tsoil[depth>=0,:].max(axis=1) , depth[depth>=0], 'r', linewidth = 2, label = 'Max' ) # Max
plt.plot(tsoil[depth>=0,:].min(axis=1) , depth[depth>=0], 'b', linewidth = 2, label = 'Min' ) # Min
plt.plot(tsoil[depth>=0,:].mean(axis=1), depth[depth>=0], 'k', linewidth = 2, label = 'Mean') # Mean
plt.legend()
plt.title('ALT='+str(depth[depth>=0][np.argmin(np.abs(tsoil[depth>=0,:].max(axis=1)))])+'m')

ax2 = fig.add_subplot(1,2,2)
ax2.plot(tsoil[40,:], color = 'k', alpha = 0.7)
plt.title('Soil Surface (0.0m)')
```

```python
fig = plt.figure(figsize=[9,4])
divnorm = mcolors.TwoSlopeNorm(vmin=-25., vcenter=0., vmax=10)
plt.contourf(np.arange(ntime), depth, tsoil, np.linspace(-25,10,15),
             norm = divnorm,
             cmap="RdBu_r", extend = 'both')

plt.ylim([5,0])
cb = plt.colorbar()
plt.xlabel('Day')
plt.ylabel('Depth (m)')
cb.ax.set_ylabel('Soil Temperature ($^oC$)')

plt.contour(np.arange(ntime), depth, tsoil, [0]) # ZERO
```

----------
### Try to simulate a warming (10%) of air temperature on the permafrost

```python
gipl.initialize('gipl_config.cfg')
fig = plt.figure(figsize=[5,8])
plt.ylim([15,0])
plt.xlim([-20,20])
plt.xlabel('Soil Temperature ($^oC$)')
plt.ylabel('Depth (m)')
plt.plot([0,0],[15,0],'k--')

for i in np.arange(int(ntime)):
    tair_raw = gipl.get_value('land_surface_air__temperature')
    gipl.set_value('land_surface_air__temperature', tair_raw + abs(tair_raw)*0.1)
    gipl.update()
    tsoil[:,i] = gipl.get_value('soil__temperature')
    plt.plot(tsoil[depth>=0,i], depth[depth>=0],color = [0.7,0.7,0.7], alpha = 0.1)

plt.plot(tsoil[depth>=0,:].max(axis=1), depth[depth>=0], 'r', linewidth = 2, label = 'Max')
plt.plot(tsoil[depth>=0,:].min(axis=1), depth[depth>=0], 'b', linewidth = 2, label = 'Min')
plt.plot(tsoil[depth>=0,:].mean(axis=1), depth[depth>=0], 'k', linewidth = 2, label = 'Mean')
plt.legend()
```

```python

```