# Stability Analysis in 2D
Stability Analysis is used to identify the change of a system of equations by plotting it using numerical techniques. It helps us identify when the variables in the system are 0 or otherwise.

### Libraries
Some of the libraries we need for Stability Analysis

Sympy: Stands for Symbolic Python, provides functionality for performing algebra and calculus.

Numpy: Offers Numeric computation

Matplotlib: Plotting and Charting library

Scipy: Scientific Computing Library

ipywidgets: Helps us develop a dashboard-like user experience. This library has no mathematical purpose here

```python
from sympy import symbols, solve, lambdify, sympify, dsolve, Eq, solveset, linear_eq_to_matrix, nonlinsolve, Matrix, diff, sqrt
import numpy as np
from matplotlib import pyplot as plt
from matplotlib.pyplot import quiver
from scipy import integrate

from ipywidgets import widgets
from IPython.display import clear_output, display, HTML
```

## Runge Kutta 4

Our implementation of the Runge-Kutta 4 algorithm. It is an iterative and non analytical method for solving equations.

```python
def rk4( f, x0, y0, h, xn ):
    n = int((xn - x0)/h)
    x = np.array( [ x0 ] * n, dtype=float )
    y = np.array( [ y0 ] * n, dtype=float )
    for i in range( n-1 ):
        k1 = h * f( x[i], y[i] )
        k2 = h * f( x[i] + 0.5 * h, y[i] + 0.5 * k1 )
        k3 = h * f( x[i] + 0.5 * h, y[i] + 0.5 * k2 )
        k4 = h * f( x[i] + h, y[i] + k3 )

        x[i+1] = x[i] + h
        y[i+1] = y[i] + ( k1 + 2.0 * ( k2 + k3 ) + k4 ) / 6.0

    return y, x

```

## Analytical Solution

This function plots the analytical solution of the equation. This is the same job as done in the RK4 above, except it is analytical.

```python
def plotEquation (eq, solvar, plotvar, inputrange):
    '''
        Given a sympy equation object `eq`, this function return a series of values,
        in the range of `inputrange` such that the equation is solved for
        `solvar` and the input is for `plotvar`
    '''
    eq = Eq(eq, 0)
    sol = solve(eq, solvar)
    vals = []
    for s in sol:
        f = lambdify((plotvar), sol)
        vals.append([f(i) for i in inputrange])

    return vals
```

## Finding Fixed Points
This function finds input values for which both the provided equations are zero. These are fixed points - points were both systems have zero rate of change.

```python
def findFixedPoints(f, g, xrange, yrange):
    return [(x, y) for x in xrange for y in yrange if f(x, y) == 0 and g(x, y) == 0]
```

## Phase Potrait Function
This function finds a sequence of values for the equations. It helps us see it change over time and against each other.

```python
def system(initialx, initialy, func1, func2, delta, timerange):
    x = [initialx]
    y = [initialy]

    for i in range(timerange):
        x.append(x[i] + func1(x[i], y[i]) * delta)
        y.append(y[i] + func2(x[i], y[i]) * delta)

    return x, y
```

## Eigen Value Finder

```python
def displayEigenValues(solutions, jacobian):
    op = '<br /> <br /> <br /> <h2>Eigen Values</h2> <br /> '
    op = op + '''<table style="font-size:20px">
                     <th>
                         <td>Stable Point</td>
                         <td>Eigen Values</td>
                         <td>Type of Stability Point</td>
                     </th>'''
    for s in solutions:
        eqmat = jacobian.subs([ (x, s[0]), (y, s[1]) ])
        ev = list(eqmat.eigenvals().keys())

        if ev[0].is_real:
            if ev[0] > 0 and ev[1] > 0:
                t = 'Unstable Node'
            elif ev[0] < 0 and ev[1] < 0:
                t = 'Stable Node'
            elif (ev[0] < 0 and ev[1] > 0) or (ev[0] > 0 and ev[1] < 0):
                t = 'Saddle Point'
        else:
            if ev[0].args[0] > 0:
                t = 'Unstable Focus'
            if ev[0].args[0] < 0:
                t = 'Stable Focus'

        op = op + '''<tr>
                            <td></td>
                            <td>%s</td>
                            <td>%s <br /> %s</td>
                            <td>%s</td>
                        </tr>''' % (str(s), ev[0], ev[1], t)
    op = op + '</table>'
```

## Mother Function
The mother of all functions uses the functions above and more to deliver the finished 'Stability Analysis Dashboard'

```python
def plots(expr1, expr2, x, y, startx, starty, stepsize, numsteps):
    '''
        expr1: sympy expression - non linear ode
        expr2: sympy expression - non linear ode
        x: sympy variable used in expr1 and/or expr2
        y: sympy variable used in expr1 and/or expr2
        startx: for the runge-kutta4, the x variable needs a start value
        starty: for the runge-kutta4, the y variable needs a start value
        stepsize: step size for runge-kutta4
        numsteps: the number of steps runge-kutta4 should take
    '''
    # converting the equations to functions
    f1 = lambdify((x, y), expr1)
    f2 = lambdify((x, y), expr2)

    # phase space
    if not(startx == '' or starty == '' or stepsize == '' or numsteps == ''):
        sx, sy = system(int(startx), int(starty), f1, f2, float(stepsize), int(numsteps))
        fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10,10))
        ax1.plot(sx, label='X in Time')
        ax1.plot(sy, label='Y in Time')
        ax2.plot(sx, sy, label='X against Y')
        ax1.legend()
        ax2.legend()

    # compute ranges to work in
    yrange = np.linspace(y_range_slider.value[0], y_range_slider.value[1])
    xrange = np.linspace(x_range_slider.value[0], x_range_slider.value[1])

    # a plot for the quiver plot and nullclines
    fig, ax = plt.subplots(figsize=(20,20))
    ax.set_xlim([min(xrange), max(xrange)])
    ax.set_ylim([min(yrange), max(yrange)])

    # plotting the 4 nullclines from range-kutta4
    rkx, rky = rk4( f1, min(xrange), -1000, .25, max(xrange) )
    ax.scatter(rkx, rky, marker='+', label='Solution with Runge-Kutta 4', s=150)

    rky, rkx = rk4( f1, min(xrange), -1000, .25, max(xrange) )
    ax.scatter(rkx, rky, marker='+', label='Solution with Runge-Kutta 4', s=150)

    rkx, rky = rk4( f2, min(yrange), 0, .25, max(yrange) )
    ax.scatter(rkx, rky, marker='+', label='Solution with Runge-Kutta 4', s=150)

    rky, rkx = rk4( f2, min(yrange), 0, .25, max(yrange) )
    ax.scatter(rkx, rky, marker='+', label='Solution with Runge-Kutta 4', s=150)

    # compute quivers
    f1_val = [[f1(i, j) for i in xrange] for j in yrange];
    f2_val = [[f2(i, j) for i in xrange] for j in yrange];

    # plot quiver plot
    q = ax.quiver(xrange, yrange, f1_val, f2_val, alpha=.5)

    # solve analytically using sympy
    solutions = solve((Eq(expr1, 0), Eq(expr2, 0)), x, y)

    # compute jacobian and eigen values
    equationMatrix = Matrix([ expr1, expr2 ])
    varMat = Matrix([ x, y ])
    jacobian = equationMatrix.jacobian(varMat)
    display(HTML('''<h2>Jacobian Matrix</h2> <br /> The Jacobian matrix has single order derivatives
                    of a System of Equations.</h2>'''))
    display(jacobian)

    display(HTML( displayEigenValues(solutions, jacobian) ))

    # plot the analytical solution
    try:
        [ax.plot(yrange, nc, c='b', alpha=.7, label='Analytical Solution') for nc in plotEquation(expr1, y, x, yrange)]
        [ax.plot(nc, yrange, c='b', alpha=.7, label='Analytical Solution') for nc in plotEquation(expr1, x, y, xrange)]
        [ax.plot(nc, xrange, c='r', alpha=.7, label='Analytical Solution') for nc in plotEquation(expr2, x, y, xrange)]
        [ax.plot(xrange, nc, c='r', alpha=.7, label='Analytical Solution') for nc in plotEquation(expr2, y, x, yrange)]
    except:
        print('Some nullcline values are complex')

    # plot the roots
    try:
        [ax.scatter(i[0], i[1], marker='x', label='Stable Point', s=150) for i in solutions]
    except:
        print('Some roots are complex')

    # a legend for our plot
    display(HTML('''<br /> <br /> <br /> <h2>Phase Potrait</h2> <br />
                 Phase Potrait shows the change in the two quantities with respect to each other
                 and idependantly.<br />'''))

    display(HTML('''<h2>Quiver Plot and Null Clines</h2><br />
                    A quiver plot shows the direction the system is moving in at each co-ordinate. <br />
                    Null Clines are lines where one of the two variables are zero. The Points
                    where Null Clines intersect are where the system is stable because both variables are
                    zero at that point.<br />
                    Null Clines can be obtained analytically (using calculus) and numerically.'''))

    fig.show()

```

## Specify your equation
The you can type your equation.
Use `**` for powers
`*` for Multiplication
In `symbols`, specify the variables used.

```python
x, y = symbols('x y')

# expr1 = 2*x - y + 3*(x**2 - y**2) + 2*x*y
expr1 = 2*x - x**2 - x*y
# expr1 = 5 - y + 5
# expr1 = 0.04*x**2 + 5*x + 140 - y - 0

display(expr1)

# expr2 = 0.02*(.2*x+y)
# expr2 = x - 3*y - 3*(x**2-y**2) + 3*x*y
expr2 = -y + x*y
# expr2 = 3*(4*x-5*y)
display(expr2)
```

$\displaystyle - x^{2} - x y + 2 x$

$\displaystyle x y - y$

# Select range of x and y values and click the plot button

The code below prepares our dashboard.

```python
def displayWidgets():
    display(y_range_slider)
    display(x_range_slider)

    display(startx)
    display(starty)
    display(stepsize)
    display(stepcount)

    display(renderButton)
```

```python

display(HTML('<h3>Select ranges for the two variables.</h3>'))

y_range_slider = widgets.IntRangeSlider(min=-1000, max=1000)

x_range_slider = widgets.IntRangeSlider(min=-1000, max=1000)

display(HTML('<h3>Input for RK4</h3> <br /> Optimum values depend on the equation and can be obtained by trial and error.'))

startx = widgets.Text(placeholder='Start Value of X for RK4', disabled=False)
starty = widgets.Text(placeholder='Start Value of Y for RK4', disabled=False)
stepsize = widgets.Text(placeholder='Step Size for RK4', disabled=False)
stepcount = widgets.Text(placeholder='Step Count for RK4', disabled=False)

renderButton = widgets.Button(description='Plot')

displayWidgets()

@renderButton.on_click
def renderButtonOnClick(b):
    clear_output()

    displayWidgets()

    plots(expr1, expr2, x, y, startx.value, starty.value, stepsize.value, stepcount.value)
```


# Finite Elements Lab 1 Worksheet

```python
from IPython.core.display import HTML
css_file = 'https://raw.githubusercontent.com/ngcm/training-public/master/ipython_notebook_styles/ngcmstyle.css'
HTML(url=css_file)
```

<link href='http://fonts.googleapis.com/css?family=Open+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 1000px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1200px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Open Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Arvo', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Arvo', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Arvo', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Arvo', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Arvo', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>

## 1d problem

The temperature $T$ of a bar of length $1$ at equilibrium satisfies

$$
  \partial_{xx} T + f(x) = 0.
$$

The heat source is given by $f(x)$. We'll hold the temperature fixed at the right edge by setting $T(1) = 0$. We'll allow heat to conduct through the left edge by setting $\partial_x T(0) = 0$.

Dealing with the second derivative is a problem, so we want to remove some derivatives from the formulation. Introducing a *weighting* function $w(x)$. Multiply the equation by $w$ and integrate by parts to get

$$
  \left[ \partial_x T(x) w(x) \right]_0^1 - \int_0^1 \text{d}x \, \partial_x T(x) \partial_x w(x) = - \int_0^1 \text{d}x \, f(x) w(x).
$$

Putting in the boundary conditions gives

$$
  \int_0^1 \text{d}x \, \partial_x T(x) \partial_x w(x) = \int_0^1 \text{d}x \, w(x) f(x).
$$

This is the **weak form**. It can be proved that solutions of the *strong* form (with two derivatives) are also solutions of the weak form, and vice versa.

### Function representation

To get an approximate solution we split the domain into intervals, or elements. To start we'll choose $[0, 1/2]$ and $[1/2, 1]$, which means there are nodes at $\{ 0, 1/2, 1 \}$. We want to represent all unknown functions ($T, w$) on these elements. We'll introduce some *shape* or *basis* functions $N_A(x)$ and write, for example,

$$
  T(x) = \sum_A T_A N_A(x).
$$

Here $A$ is a counter associated with the nodes, so here $A = \{ 0, 1, 2 \}$: the nodes are labelled $x_A$. The shape function $N_A(x)$ is $1$ at the associated node, and $0$ at all other nodes. If we choose the shape functions to be piecewise linear then the three shape functions therefore look like

$$
\begin{align}
  N_0(x) &= \begin{cases} 1 - 2 x, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  N_1(x) &= \begin{cases} 2 x, & 0 \le x \le 1/2 \\ 2 - 2 x, & 1/2 \le x \le 1, \end{cases}, \\
  N_2(x) &= \begin{cases} 0, & 0 \le x \le 1/2 \\ 2 x - 1, & 1/2 \le x \le 1. \end{cases}
\end{align}
$$

```python
%matplotlib inline
import numpy
from matplotlib import pyplot
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
rcParams['figure.figsize'] = (12,6)
```

```python
x = numpy.linspace(0, 1, 1000)
N0 = numpy.where(x <= 0.5, 1-2*x, numpy.zeros_like(x))
N1 = numpy.where(x <= 0.5, 2*x, 2-2*x)
N2 = numpy.where(x <= 0.5, numpy.zeros_like(x), 2*x-1)
pyplot.figure()
pyplot.plot(x, N0, label=r"$N_0$")
pyplot.plot(x, N1, label=r"$N_1$")
pyplot.plot(x, N2, label=r"$N_2$")
pyplot.xlabel(r"$x$")
pyplot.ylim(-0.1, 1.1)
pyplot.legend();
```

With this definition of the shape functions, the coefficients are given exactly as (e.g.) $T_A = T(x_A)$. **NOTE**: because we have imposed the boundary condition $T(1) = 0$ we must have that $T_2 \equiv 0$, so we do not need to calculate $T_2$.

We can now write out the weak form of the problem as

$$
  \sum_A \sum_B T_A w_B \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x) = \sum_B w_B \int_0^1 \text{d}x \, N_A(x) f(x).
$$

This has to be true for *any* choice of weight function, so for *any* coefficients $w_B$. This is true only if

$$
  \sum_A T_A \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x) = \int_0^1 \text{d}x \, N_A(x) f(x).
$$

This can be written as a matrix problem:

$$
  K {\bf T} = {\bf F}.
$$

The coefficients of the **stiffness matrix** $K$ are given by

$$
  K_{AB} = \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x).
$$

The coefficients of the **force vector** $F$ are given by

$$
  F_{A} = \int_0^1 \text{d}x \, N_A(x) f(x).
$$

### Tasks

1. Compute (analytically) the coefficients of the stiffness matrix given the shape functions above.
2. Compute (analytically) the coefficients of the force vector when $f(x) = x$.
3. Solve (numerically) for $T_A$ and plot the resulting solution for $T(x)$. Compare against the exact solution $T(x) = (1 - x^3) / 6)$.

```python

```

## The element viewpoint

This works surprisingly well, but it's clearly going to get very complex when the number of elements (or nodes) gets large as we try to improve accuracy. To work systematically with large numbers of elements, we need to consider one at a time.

First, note that in any element there's only two shape functions that are not zero: those that correspond to the nodes at the edge of the element. We'll call these shape functions $N_1$ and $N_2$.

Second, note that by doing a coordinate transformation, we can take *any* element from the interval $[ x_A, x_{A+1} ]$ to the interval $[ \xi_1, \xi_2 ]$ where we fix (by convention) $\xi_1 = -1$ and $\xi_2 = 1$. We have

$$
\begin{align}
  \xi(x) &= \frac{2 x - x_A - x_{A+1}}{x_{A+1} - x_{A}}, \\
  x(\xi) &= \frac{(x_{A+1} - x_{A}) \xi + x_A + x_{A+1}}{2}.
\end{align}
$$

We can now write the shape functions in terms of the *reference coordinates* $\xi$ as

$$
  N_a(\xi) = \tfrac{1}{2} \left( 1 + \xi_a \xi \right), \qquad a = 1, 2.
$$

As above we needed the derivatives of the shape function when computing coefficients, we note that

$$
  \partial_{\xi} N_a = \frac{(-1)^a}{2}, \qquad a = 1, 2.
$$

We will also need to change coefficients of vectors which will require a Jacobian, which needs derivatives of the coordinate transformation. This needs

$$
\begin{align}
  \partial_x \xi &= \frac{2}{x_{A+1} - x_{A}}, \\
  \partial_{\xi} x &= \frac{(x_{A+1} - x_{A})}{2}.
\end{align}
$$

With this information we can compute the stiffness matrix $k^e_{ab}$ for the single element $e$. This gives

$$
\begin{align}
  k^e_{ab} &= \int_{x_A}^{x_{A+1}} \text{d}x \, \partial_x N_a \partial_x N_b \\
  &= \int_{-1}^{+1} \text{d}\xi \, \partial_{\xi} x \, \partial_{x} N_a \partial_{x} N_b \\
  &= \int_{-1}^{+1} \text{d}\xi \, \left(\partial_{\xi} x\right)^{-1} \, \partial_{\xi} N_a \partial_{\xi} N_b \\
  &= \frac{(-1)^{a+b}}{x_{A+1} - x_A}.
\end{align}
$$

This is incredibly useful: there's no need to do any integrals at all.

The element force vector is given by

$$
\begin{align}
  f^e_{a} &= \int_{x_A}^{x_{A+1}} \text{d}x \, N_a f(x) + \text{boundary terms}
  & = \int_{x_A}^{x_{A+1}} \text{d}x \, N_a f(x).
\end{align}
$$

This is specific to our choice of boundary conditions: normally there would be extra terms.

To evaluate this it's convenient to write $f(x) = \sum_a f_a N_a$ within the element, where $f_a = f(x(\xi_a))$. With the simple shape functions we have here, we get

$$
\begin{equation}
  f^e_{a} = \frac{x_{A+1} - x_{A}}{6} \begin{pmatrix} 2 f_1 + f_2 \\ f_1 + 2 f_2 \end{pmatrix}.
\end{equation}
$$

### Linking elements to equations

In the end we want to construct a linear (matrix) equation to give us the solution $T_A$ at all nodes $A$ where it isn't enforced by the boundary conditions (which in the problem above is all nodes except the right-hand boundary). We note that each *interior* node is linked to two elements, so contributions from the element matrix may affect more than one equation. To keep track of this, we construct the **location matrix** or **location array** $LM$ which, given the node number $a = 1, 2$ and the element number $e$ returns the associated equation number.

In our case we have

$$
  A = LM(a, e) = \begin{cases} e & a = 1 \\ e+1 & a = 2 \end{cases}
$$

for $e = 0, 1, \dots, N_{\text{elements}}-1$. For the final element we have to change things, as the final node is not to be included. Here we'll set the equation number $A$ to be $-1$.

```python
Nelements = 4

LM = numpy.zeros((2, Nelements))
for e in range(Nelements):
    LM[0, e] = e
    LM[1, e] = e+1
LM[1, -1] = -1 # Right hand node of final element is not considered thanks to BC.
```

Now the global stiffness matrix and force vector can be assembled: for each element $e$ we construct $k^e_{ab}$ and add:

$$
\begin{align}
  K_{LM(a, e) \, LM(b, e)} &= K_{LM(a, e) \, LM(b, e)} + k^e_{ab}, & a, b &= 1, 2, \\
  f_{LM(a, e)} &= f_{LM(a, e)} + f^e_a, & a &= 1, 2.
\end{align}
$$

### Algorithm

This gives our full algorithm:

1. Set number of elements $N_{\text{elements}}$.
2. Set node locations $x_A, A = 0, \dots, N_{\text{elements}}$.
3. Set up the location matrix $LM$.
4. Set up arrays of zeros for the global stiffness matrix (size $N_{\text{elements}} \times N_{\text{elements}}$) and force vector (size $N_{\text{elements}}$).
5. For each element:

    1. Form the element stiffness matrix $k^e_{ab}$.
    2. Form the element force vector $f^e_a$.
    3. Add the contributions to the global stiffness matrix and force vector

6. Solve $K {\bf T} = {\bf F}$.

### Task

1. Write a finite element solver that takes the number of elements and the function $f(x)$, forms a uniformly spaced grid, and solves for $T$.
2. Check that it returns the same result as above when applied to two elements with $f(x) = x$.
3. Apply it to the case with $f(x) = x^2$ with the same boundary conditions: the exact solution is $(1 - x^4) / 12$. Compute the error in the 2-norm and show how it converges.

```python

```

FEEG6016 Simulation and Modelling/07-Finite-Elements-Lab-1.ipynb


# **2.1 Pull-out of elastic fiber from rigid matrix**

[](https://moodle.rwth-aachen.de/mod/page/view.php?id=551807)

# The simplest possible pull-out model

An analytical solution of the pull-out problem is obtained by

1. Integrate the differential equilibrium equation relating the shear flow with the change of the normal force
$A_\mathrm{f} \mathrm{d}\sigma_\mathrm{f}$ in the fiber on an infinitesimal element $\mathrm{d}x$
\begin{align}\dfrac{\mathrm{d}\sigma_\mathrm{f}}{\mathrm{d}x} = \dfrac{p\bar{\tau}}{A_\mathrm{f}}\end{align}
2. Substitute the result into the elastic constitutive law of the reinforcement
\begin{align}\varepsilon_\mathrm{f} = \dfrac{\sigma_\mathrm{f}}{E_\mathrm{f}}\end{align}
3. Substitute the result into the kinematic relation stating that the pull-out displacement is equal to the integral of fiber strain along the debonded length
\begin{align}u_\mathrm{f} = \int_{a}^0 \varepsilon_\mathrm{f} \, \mathrm{d}x\end{align}
4. Identify the integration constants by applying boundary conditions: equilibrium at loaded end and compatibility and smoothness at the end of the debonded zone $x = a$

These step deliver the pull-out curve as a square root function
\begin{align}
P = \sqrt{p \bar{\tau} E_\mathrm{f} A_\mathrm{f} w}
\end{align}

## Graphical summary of the model derivation

# Model application
Let us utilize the the derived model to simulate the test results of the RILEM pull-out test

| Symbol | Unit |Description  |
|:- |:- |:- |
| $E_\mathrm{f}$ | MPa | Young's modulus of reinforcement |
| $\bar{\tau}$ | MPa | Bond stress |
| $A_\mathrm{f}$ | mm$^2$ | Cross-sectional area of reinforcement |
| $p$ | mm | Perimeter of contact between concrete and reinforcement |

### **Observation**

 - The measured displament at the loaded and unloaded end are different
 - Their difference increases with increasing bond length $L_\mathrm{b}$
 - The shape of the pull-out curve has a shape of a square root function

### **Question**

 - Can the above derived model describe the debonding process correctly?

# Look inside the specimen using the model

The parameters of the above experiment are specified as follows

```python
ds = 16
A_f = (ds/2)**2 * 3.14 # mm^2 - reinforcement area
L_b = 5 * ds # mm - bond length
E_f = 210000 # MPa - reinforcement stiffness
p_b = 3.14 * ds # mm - bond perimeter
w_max = 0.12 # mm - maximum displacement
```

**Construct the model:** To study the model behavior import the class `PO_ELF_RLM`, construct it with the defined parameters  and run the `interact` method

```python
%matplotlib widget
from pull_out import PO_ELF_RLM
po = PO_ELF_RLM(E_f=E_f, L_b=L_b, p=p_b, A_f=A_f, w_max=w_max)
```

**Remark:** that the length $L_b$ is not the end of the bond zone. It only measures the slip at the position $x = L_\mathrm{b}$ from the loadedend. However, the debonding process can continue beyond this length.

```python
po.interact()
```

    VBox(children=(HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black…

## Let's learn from the model

Exercise the relation between $P$ and $\tau(x)$ and between $w$ and $\varepsilon(x)$.

 1. What is the meaning of the green area?
 2. What is the meaning of the red area?
 3. What is the meaning of the slope of the green curve?
 4. Is it possible to reproduce the shown RILEM test response using this "frictional" model?
 4. What is the role of debonded length $a$ in view of general non-linear simulation?
 5. When does the pull-out fail?
 5. What happends with $a$ upon unloading?

      1.3 Elastic stiffness of the composite

    2.2 Classification of pullout configurations

```python

```


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Homework 5

**Instructions:** Complete the notebook below. Download the completed notebook in HTML format. Upload assignment using Canvas.

**Due:** Feb. 20 at **12:30pm.**

## Exercise: AR(1) Process Simulation

For each of the AR(1) processes defined below:

1. Compute $y$ for $201$ periods (i.e., $t=0,\ldots, 200$).
2. Construct a well-labeled plot of $y$ for each simulation.
2. Print the mean and standard deviation of the stimulated process.

You may use the function from the Notebook for Lecture 10, write your own, or take a more brute force approach.

```python
# Define a function for computing the first-order difference equation. CELL NOT PROVIDED
def ar1_sim(rho,sigma,y0,T,):
    '''Function for simulating an AR(1) process

        y[t] = rho*y[t-1] + epsilon[t]

    Args:
        rho (float):    Autoregressive coefficient on y[t-1]
        sigma (float):  Standard deviation of epsilon
        y0 (float):     Initial value of the process
        T (int):        Number of periods to simulate including initial period

     Returns:
         NumPy ndarray
     '''

    y = np.zeros(T)
    epsilon = np.random.normal(scale=sigma,size=T)
    y[0] = y0
    for t in range(T-1):
        y[t+1] = rho*y[t]+epsilon[t+1]

    return y

# Create variable 'T' equal to number of simulation periods
T = 201

# Set NumPy RNG seed to 126 st that simulations look the same each time I run the Notebook
np.random.seed(126)
```

1. $y_{t} = 0.33 y_{t-1} + \epsilon_t, \; \; \; \; \epsilon \sim \mathcal{N}(0,1), \; \; \; \; y_0 = 0$

```python
rho=0.33
sigma = 1
y0 = 0

y = ar1_sim(rho,sigma,y0,T)

plt.plot(y,lw=3)
plt.title('AR(1): $\\rho='+str(rho)+'$ and $\\sigma='+str(sigma)+'$')
plt.xlabel('$t$',fontsize=15)
plt.ylabel('$y_t$',fontsize=15)
plt.grid()
```

```python
# print mean of y
print('mean of y:              ',round(y.mean(),4))

# Print standard deviation of y
print('standard deviation of y: ',round(y.std(),4))
```

    mean of y:               -0.0743
    standard deviation of y:  1.0322

2. $y_{t} = y_{t-1} + \epsilon_t, \; \; \; \; \epsilon \sim \mathcal{N}(0,1), \; \; \; \; y_0 = 0$

```python
rho=1
sigma = 1
y0 = 0

y = ar1_sim(rho,sigma,y0,T)

plt.plot(y,lw=3)
plt.title('AR(1): $\\rho='+str(rho)+'$ and $\\sigma='+str(sigma)+'$')
plt.xlabel('$t$',fontsize=15)
plt.ylabel('$y_t$',fontsize=15)
plt.grid()
```

```python
# print mean of y
print('mean of y:              ',round(y.mean(),4))

# Print standard deviation of y
print('standard deviation of y:',round(y.std(),4))
```

    mean of y:               -3.0308
    standard deviation of y: 3.8289

3. $y_{t} = -0.95 y_{t-1} + \epsilon_t, \; \; \; \; \epsilon \sim \mathcal{N}(0,1), \; \; \; \; y_0 = 0$

```python
rho=-0.95
sigma = 1
y0 = 0

y = ar1_sim(rho,sigma,y0,T)

plt.plot(y,lw=3)
plt.title('AR(1): $\\rho='+str(rho)+'$ and $\\sigma='+str(sigma)+'$')
plt.xlabel('$t$',fontsize=15)
plt.ylabel('$y_t$',fontsize=15)
plt.grid()
```

```python
# print mean of y
print('mean of y:              ',round(y.mean(),4))

# Print standard deviation of y
print('standard deviation of y: ',round(y.std(),4))
```

    mean of y:               -0.0131
    standard deviation of y:  3.1956

4. $y_{t} = 0.75 y_{t-1} + \epsilon_t , \; \; \; \; \epsilon \sim \mathcal{N}(0,4), \; \; \; \; y_0 = 20$

```python
rho=0.75
sigma = 2
y0 = 20

y = ar1_sim(rho,sigma,y0,T)

plt.plot(y,lw=3)
plt.title('AR(1): $\\rho='+str(rho)+'$ and $\\sigma='+str(sigma)+'$')
plt.xlabel('$t$',fontsize=15)
plt.ylabel('$y_t$',fontsize=15)
plt.grid()
```

```python
# print mean of y
print('mean of y:              ',round(y.mean(),4))

# Print standard deviation of y
print('standard deviation of y:',round(y.std(),4))
```

    mean of y:               0.8346
    standard deviation of y: 3.6313

5. $y_{t} = 0.9 y_{t-1} + \epsilon_t , \; \; \; \; \epsilon \sim \mathcal{N}(0,3), \; \; \; \; y_0 = 0$

```python
rho=0.9
sigma = np.sqrt(3)
y0 = 0

y = ar1_sim(rho,sigma,y0,T)

plt.plot(y,lw=3)
plt.title('AR(1): $\\rho='+str(rho)+'$ and $\\sigma='+str(round(sigma,4))+'$')
plt.xlabel('$t$',fontsize=15)
plt.ylabel('$y_t$',fontsize=15)
plt.grid()
```

```python
# print mean of y
print('mean of y:              ',round(y.mean(),4))

# Print standard deviation of y
print('standard deviation of y: ',round(y.std(),4))
```

    mean of y:               2.8525
    standard deviation of y:  4.5849

## Exercise: Stochastic Growth

We've seen in class that total factory productivity (TFP) in the US fluctuates over the business cycle around a long-term growth trend. That is, total factor productivity is characterized by *stochastic growth*. In this problem, you will simulate a model of TFP with stochastic growth.

### Background

Suppose that the trend component of TFP $A^{trend}$ grows smoothly at the constant rate $g$:

\begin{align}
A^{trend}_t & = (1+g)A_{t-1}^{trend}, \tag{1}
\end{align}

where $A_0$ is given. The log-deviations of actual TFP $A$ from trend TFP are determined according to an AR(1) process:

\begin{align}
\log\left(A_t/A^{trend}_t\right) & = \rho \log \left(A_{t-1} / A^{trend}_{t-1}\right) + \epsilon_t, \tag{2}
\end{align}

where $A_0$ is given and $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$. Solve quation (2) for $A_t$ to get an expression for $A_t$ in terms of $A_{t-1}$, $A^{trend}_{t}$, $A^{trend}_{t-1}$, and $\epsilon_t$:

\begin{align}
A_t & = A^{trend}_t \displaystyle e^{\rho \log \left(A_{t-1} / A^{trend}_{t-1}\right) + \epsilon_t} \tag{3}
\end{align}

### Simulation

To simulate this model:

1. Specify values for $A_0$ and $A^{trend}_0$.
2. For $t = 0, 1, \ldots... T$, compute $A^{trend}_{t+1}$ using equation (1) and then compute $A_{t+1}$ using equation (3)

Use the following prameter values for this simulation

| $A^{trend}_0$ | $A_0$ | $g$  | $\rho$ | $\sigma$ | $T$ |
|---------------|-------|------|--------|----------|-----|
| 1             | 1     | 0.02 | 0.7    | 0.025    | 101 |

In the follwing cell, simulate $A^{trend}$ and $A$ using the parameter values provided.

```python
# Simulate the stochastic growth model of TFP
T = 101
g = 0.02
rho = 0.7
sigma = 0.025

epsilon = np.random.normal(scale=sigma,size=T)
a_trend = np.ones(T)*1
a = np.ones(T)*1

for t in range(T-1):
    a_trend[t+1] = (1+g)*a_trend[t]
    a[t+1] = a_trend[t+1]*np.exp(rho*np.log(a[t]/a_trend[t])+epsilon[t])
```

Contruct a well-labeled plot of the simulated values of $A$ and $A^{trend}$

```python
# Construct a plot of simulated TFP with its trend with:
#    1. Actual line: blue with lw=1, alpha=0.7, label = 'actual'
#    2. Trend line:  red with lw=3, alpha=0.25, label = 'trend'
plt.plot(a_trend,'r',lw=3,alpha=0.25,label='Trend')
plt.plot(a,label='Actual')
plt.title('Simulated TFP with Stochastic Growth')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5))
plt.grid()
```

Contruct a well-labeled plot of the simulated values of $\log \left(A/A^{trend}\right)$

```python
# Construct a plot of simulated log deviation of TFP from trend
plt.plot(np.log(a/a_trend),lw=3)
plt.title('Simulated $\log A_t/A^{trend}$')
plt.grid()
```

Compute the mean and standard deviation of $\log \left(A/A^{trend}\right)$

```python
# print mean of log(A/A trend)
print('mean of log(A/A trend):              ',round((np.log(a/a_trend)).mean(),4))

# Print standard deviation of log(A/A trend)
print('standard deviation of log(A/A trend): ',round((np.log(a/a_trend)).std(),4))
```

    mean of log(A/A trend):               -0.0192
    standard deviation of log(A/A trend):  0.03

## Exercise: Questions for Prescott (1983) Reading

The following questions are about Edward Prescott's 1986 article "Theory  Ahead of Business Cycle Measurement" from the Fall 1986 issue of the Minneapolis Fed's *Quarterly Review* (link to article: [https://www.minneapolisfed.org/research/qr/qr1042.pdf](https://www.minneapolisfed.org/research/qr/qr1042.pdf)). The article is a summary of the research agenda of the author, his main collaborator, Finn Kydland, and others. That agenda entailed incorporating stochastic growth of TFP into the neoclassical growth model (essentially a Solow model but with the saving rate determined as a consequence a of utility maximization problem) as a way of modeling business cycle fluctuations. This line of research would later be called *real business cycle* theory.

The below questions are specifically about:
* Pages 9-11 (including the table on page 12)
* The final paragraph on page 21

**Question:** On page 10, Prescott writes that the "models constructed within this theoretical framework are necessarily highly abstract. Consequently they are necessarily false." What does Kydland mean by this? In what sense are abstract models necessarily false?

**Answer**

All theoretical models are approximations of reality and therefore will not relfect the complexity of the natural environment. But that doesn't mean that abstract models can't be useful. In fact, abstraction helps us understand complicated phenomena by emphasizing only the most relevant aspects of the issue. <!-- answer -->

**Question:** On page 10, Prescott lists two reasons for why he doesn't like the term *business cycle*. What is the first reason and what does it mean?

**Answer**

The first reason that Prescott doesn't like the term *business cyle* is that it implies that short-term fluctuations are the result of economic forces that are *independent* of long-term growth. In Prescott's view, short-term fluctuations and long-term growth *are a consequence of the same process*: stochastic TFP growth. That is, TFP grows, but at a random and unpredictable rate.<!-- answer -->

**Question:** Table 1 on page 12 shows that household consumption measures in the US fluctuate less than investment measures. What is the intuition for why consumption varies less than investment?

**Answer**

People like to smooth their consumption over time as much as they can. When houseohlds have higher-than-average income, they save most for the future and so investment rises more than consumption when incomes are high. And when household incomes are lower-than-average, households primarily reduce saving and investment falls more than consumption when incomes are high.<!-- answer -->

**Question:** In the last paragraph of page 21, to what does Prescott attribute economic fluctuations? Can you think of any reasons why this statement was, and still is, controversial?

**Answer**

"Economic fluctuations are the optimal responses to uncertainty in the rate of technological change." The statement is controversial for several reasons. <!-- answer -->
* First, the argument attributes *all* economic fluctuations to TFP variation and disregards other plausible candidates like, for example, exogenous variation in monetary policy. <!-- answer -->
* Second, Prescott's claim that fluctuations are *optimal* implies that the economy is *always* doing the best it can with what it has: When unemployment is high, it's because TFP fell relative to trend and households decided to take more leisure. <!-- answer -->
* Third, the statement implies that, even in principle, policymakers cannot improve the welfare of the public by stabilizing the business cycle. <!-- answer -->


# Spectral Estimation of Random Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Introduction

In the preceding sections various statistical measures have been introduced to characterize random processes and signals. For instance, the probability density function (PDF) $p_x(\theta)$, the mean value $\mu_x$, the auto-correlation function (ACF) $\varphi_{xx}[\kappa]$ and its Fourier transformation, the power spectral density (PSD) $\Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. For many random processes whose internal structure is known, these measures can be derived in closed-form. However, for practical random signals measures of interest have to be estimated from a limited number of samples. These estimated quantities can e.g. be used to fit a parametric model of the random process or as parameters in algorithms.

### Problem Statement

The estimation of the spectral properties of a random signal is of special interest for spectral analysis. The discrete Fourier transform (DFT) of a random signal is also random. It is not very well suited to gain insights into the average spectral structure of a random signal. We aim at estimating the PSD $\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ of a wide-sense stationary (WSS) and ergodic process from a limited number of samples. This is known as [*spectral (density) estimation*](https://en.wikipedia.org/wiki/Spectral_density_estimation). Many techniques have been developed for this purpose. They can be classified into

1. non-parametric and
2. parametric

techniques. Non-parametric techniques estimate the PSD of the random signal without assuming any particular structure for the generating random process. In contrary, parametric techniques assume that the generating random process can be modeled by a few parameters. Their aim is to estimate these parameters in order to characterize the spectral properties of the random signal.

### Evaluation

Various measures have been introduced in order to evaluate the performance of a particular estimation technique. The estimate $\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ can be regarded as a random signal itself. The performance of an estimator is therefore evaluated in a statistical sense. For the PSD, the following metrics are of interest

#### Bias

The [bias of an estimator](https://en.wikipedia.org/wiki/Estimator#Bias)

\begin{equation}
b_{\hat{\Phi}_{xx}}
= E\{ \hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) - \Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})\}
= E\{ \hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) \} - \Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})
\end{equation}

quantifies the difference between the estimated $\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$ and the true $\Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$. An estimator is
* biased if $b_{\hat{\Phi}_{xx}} \neq 0$ and
* bias-free if $b_{\hat{\Phi}_{xx}} = 0$.

#### Variance

The [variance of an estimator](https://en.wikipedia.org/wiki/Estimator#Variance)

\begin{equation}
\sigma^2_{\hat{\Phi}_{xx}} = E \left\{ \left(\hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) - E\{ \hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})\} \right)^2 \right\}
\end{equation}

quantifies its quadratic deviation from its mean value $E\{ \hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})\}$.

#### Consistency

A [consistent estimator](https://en.wikipedia.org/wiki/Estimator#Consistency) is an estimator for which the following conditions hold for a large number $N$ of samples:

1. the estimator is unbiased
    $$ \lim_{N \to \infty} b_{\hat{\Phi}_{xx}} = 0 $$

2. its variance converges towards zero
    $$ \lim_{N \to \infty} \sigma^2_{\hat{\Phi}_{xx}} = 0 $$

3. it converges in probability to the true $\Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})$
    $$ \lim_{N \to \infty} \Pr \left\{ | \hat{\Phi}_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega}) -  \Phi_{xx}(\mathrm{e}^{\,\mathrm{j}\,\Omega})| > \alpha \right\} = 0$$
    where $\alpha > 0$ denotes a (small) constant.

The last condition ensures that a consistent estimator does not generate outliers. Consistency is a desired property of an estimator. It ensures that if the number of samples $N$ increases towards infinity, the resulting estimates converges towards the true PSD.

#### Example - Discrete Fourier transform of sample functions

The following example computes and plots the magnitude of the discrete Fourier transform (DFT) $|X_n[\mu]|$ of an ensemble of random signals $x_n[k]$. In the plot, each color denotes one sample function and $\Omega[\mu] = \frac{2 \pi}{N} \mu$ where $N$ denotes the length of the sample function. The magnitude spectra are plotted as continuous signals for ease of illustration.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sig

N = 256  # number of samples
M = 5  # number of sample functions

# generate random signal
np.random.seed(1)
x = np.random.normal(size=(M, N))
h = sig.firwin2(N, [0, .5, .52, .55, .57, 1], [0, 0, 1, 1, 0, 0])
x = [np.convolve(xi, h, mode='same') for xi in x]

# DFT of signal
X = np.fft.rfft(x, axis=1)
Om = np.linspace(0, np.pi, X.shape[1])

# plot signal and its spectrum
plt.figure(figsize=(10,4))
plt.plot(Om, np.abs(X.T))
plt.title('Magnitude spectrum')
plt.xlabel(r'$\Omega[\mu]$')
plt.ylabel(r'$|X[\mu]|$')
plt.axis([0, np.pi, 0, 30]);
```

**Exercise**

* Increase the number `N` of samples. What changes? What does not change with respect to the evaluation criteria introduced above?
* Is the DFT of a single sample function a consistent estimator for the spectral properties of a random process?

Increasing the number of samples does only lead to an increase in the number of discrete frequencies $\mu$. The amplitude of the fluctuations (variance) of the spectra within $1.5 < \Omega < 1.8$ is not decreased when increasing the number of samples. The DFT of a single sample function is hence not a consistent estimator since at least the second condition is violated.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


## Lecture 3

### Numpy

## Covered topics:

* `numpy` - a package for storing and working with N-dimensional arrays of data.
* Why Python's `list` object is not adequate.
  * Dealing with high-dimensional data tricky.
  * Does not have 'maths' notation, e.g. $x + y$
  * Speed.
* Slicing.
* Examples:
  * Mandelbrot set.
  * Markov chains.
  * Calculating $\pi$ via random sampling.

Working with numbers is at the core of most activities in scientific and engineering computations. The topic is so important that there entire libraries dedicated to working with numbers. `numpy` is the most important and widely used package for dealing with numerical data in Python.

We have already seen Python's `list` data structure. We used it for storing a list (sometimes called an array) of data of arbitrary type. We also know how to *slice* the list to get out particular elements we are interested in, and *iterate* over the list.

*Reminder:*

```python
mylist = [0, 3, 4, 5, 'test']
print(mylist[0:3])
print(mylist[-1])
for l in mylist:
    print(l)
```

    [0, 3, 4]
    test
    0
    3
    4
    5
    test

### 'Problems' with Python's `list`.

1. Dealing with high-dimension data is tricky.

\begin{equation}
A = \begin{bmatrix}
0 & 1 & 2 \\
3 & 4 & 5 \\
6 & 7 & 8
\end{bmatrix}
\end{equation}

Idea: A list of lists! Let's put each row in its own `list` and then put the rows in another list.

```python
A = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
```

*Exercises*:
1. Select the first row.
2. Select the first number of the second row.
3. Print the first column. (Not so easy).

```python
A = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]

print(A[0])
print(A[1][0])
for row in A:
    print(row[0])
```

    [0, 1, 2]
    3
    0
    3
    6

2. `list` does not conform to 'standard' mathematical conventions.

$$ x = [1, 2, 3]$$
$$ y = [4, 5, 6]$$
$$ x + y = \; ? $$

```python
x = [1, 2, 3]
y = [4, 5, 6]
x + y
```

    [1, 2, 3, 4, 5, 6]

## Something better: `numpy`

```python
import numpy
import numpy as np # np is now an alias (shorthand) for numpy
np.lookfor('create array')
```

    Search results for 'create array'
    ---------------------------------
    numpy.array
        Create an array.
    numpy.memmap
        Create a memory-map to an array stored in a *binary* file on disk.
    numpy.diagflat
        Create a two-dimensional array with the flattened input as a diagonal.
    numpy.fromiter
        Create a new 1-dimensional array from an iterable object.
    numpy.partition
        Return a partitioned copy of an array.
    numpy.ctypeslib.as_array
        Create a numpy array from a ctypes array or POINTER.
    numpy.ma.diagflat
        Create a two-dimensional array with the flattened input as a diagonal.
    numpy.ma.make_mask
        Create a boolean mask from an array.
    numpy.lib.Arrayterator
        Buffered iterator for big arrays.
    numpy.ctypeslib.as_ctypes
        Create and return a ctypes object from a numpy array.  Actually
    numpy.ma.mrecords.fromarrays
        Creates a mrecarray from a (flat) list of masked arrays.
    numpy.ma.mvoid.__new__
        Create a new masked array from scratch.
    numpy.ma.MaskedArray.__new__
        Create a new masked array from scratch.
    numpy.ma.mrecords.fromtextfile
        Creates a mrecarray from data stored in the file `filename`.
    numpy.asarray
        Convert the input to an array.
    numpy.ndarray
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.recarray
        Construct an ndarray that allows field access using attributes.
    numpy.chararray
        chararray(shape, itemsize=1, unicode=False, buffer=None, offset=0,
    numpy.exp
        Calculate the exponential of all elements in the input array.
    numpy.pad
        Pad an array.
    numpy.asanyarray
        Convert the input to an ndarray, but pass ndarray subclasses through.
    numpy.cbrt
        Return the cube-root of an array, element-wise.
    numpy.copy
        Return an array copy of the given object.
    numpy.diag
        Extract a diagonal or construct a diagonal array.
    numpy.exp2
        Calculate `2**p` for all `p` in the input array.
    numpy.fmax
        Element-wise maximum of array elements.
    numpy.fmin
        Element-wise minimum of array elements.
    numpy.load
        Load arrays or pickled objects from ``.npy``, ``.npz`` or pickled files.
    numpy.modf
        Return the fractional and integral parts of an array, element-wise.
    numpy.rint
        Round elements of the array to the nearest integer.
    numpy.sort
        Return a sorted copy of an array.
    numpy.sqrt
        Return the non-negative square-root of an array, element-wise.
    numpy.array_equiv
        Returns True if input arrays are shape consistent and all elements equal.
    numpy.dtype
        Create a data type object.
    numpy.expm1
        Calculate ``exp(x) - 1`` for all elements in the array.
    numpy.isnan
        Test element-wise for NaN and return result as a boolean array.
    numpy.isnat
        Test element-wise for NaT (not a time) and return result as a boolean array.
    numpy.log10
        Return the base 10 logarithm of the input array, element-wise.
    numpy.log1p
        Return the natural logarithm of one plus the input array, element-wise.
    numpy.power
        First array elements raised to powers from second array, element-wise.
    numpy.ufunc
        Functions that operate element by element on whole arrays.
    numpy.choose
        Construct an array from an index array and a set of arrays to choose from.
    numpy.nditer
        Efficient multi-dimensional iterator object to iterate over arrays.
    numpy.maximum
        Element-wise maximum of array elements.
    numpy.minimum
        Element-wise minimum of array elements.
    numpy.swapaxes
        Interchange two axes of an array.
    numpy.full_like
        Return a full array with the same shape and type as a given array.
    numpy.ones_like
        Return an array of ones with the same shape and type as a given array.
    numpy.bitwise_or
        Compute the bit-wise OR of two arrays element-wise.
    numpy.empty_like
        Return a new array with the same shape and type as a given array.
    numpy.zeros_like
        Return an array of zeros with the same shape and type as a given array.
    numpy.asarray_chkfinite
        Convert the input to an array, checking for NaNs or Infs.
    numpy.bitwise_and
        Compute the bit-wise AND of two arrays element-wise.
    numpy.bitwise_xor
        Compute the bit-wise XOR of two arrays element-wise.
    numpy.float_power
        First array elements raised to powers from second array, element-wise.
    numpy.ma.exp
        Calculate the exponential of all elements in the input array.
    numpy.diag_indices
        Return the indices to access the main diagonal of an array.
    numpy.nested_iters
        Create nditers for use in nested loops
    numpy.ma.sqrt
        Return the non-negative square-root of an array, element-wise.
    numpy.ma.log10
        Return the base 10 logarithm of the input array, element-wise.
    numpy.chararray.tolist
        a.tolist()
    numpy.put_along_axis
        Put values into the destination array by matching 1d index and data slices.
    numpy.ma.choose
        Use an index array to construct a new array from a set of choices.
    numpy.ma.maximum
        Element-wise maximum of array elements.
    numpy.ma.minimum
        Element-wise minimum of array elements.
    numpy.ma.mrecords.MaskedRecords.__new__
        Create a new masked array from scratch.
    numpy.savez_compressed
        Save several arrays into a single file in compressed ``.npz`` format.
    numpy.matlib.rand
        Return a matrix of random values with given shape.
    numpy.datetime_as_string
        Convert an array of datetimes into an array of strings.
    numpy.ma.bitwise_or
        Compute the bit-wise OR of two arrays element-wise.
    numpy.ma.bitwise_and
        Compute the bit-wise AND of two arrays element-wise.
    numpy.ma.bitwise_xor
        Compute the bit-wise XOR of two arrays element-wise.
    numpy.ma.make_mask_none
        Return a boolean mask of the given shape, filled with False.
    numpy.ma.tests.test_subclassing.MSubArray.__new__
        Create a new masked array from scratch.
    numpy.core._multiarray_umath.clip
        Clip (limit) the values in an array.
    numpy.core.tests.test_overrides._new_duck_type_and_implements
        Create a duck array type and implements functions.
    numpy.ma.tests.test_subclassing.SubMaskedArray.__new__
        Create a new masked array from scratch.
    numpy.ma.mrecords.fromrecords
        Creates a MaskedRecords from a list of records.
    numpy.core._multiarray_umath.empty_like
        Return a new array with the same shape and type as a given array.
    numpy.f2py.tests.test_array_from_pyobj.Array.has_shared_memory
        Check that created array shares data with input array.
    numpy.core._dtype._construction_repr
        Creates a string repr of the dtype, excluding the 'dtype()' part
    numpy.abs
        Calculate the absolute value element-wise.
    numpy.add
        Add arguments element-wise.
    numpy.cos
        Cosine element-wise.
    numpy.log
        Natural logarithm, element-wise.
    numpy.mod
        Return element-wise remainder of division.
    numpy.lib.recfunctions.require_fields
        Casts a structured array to a new dtype using assignment by field-name.
    numpy.sin
        Trigonometric sine, element-wise.
    numpy.tan
        Compute tangent element-wise.
    numpy.ceil
        Return the ceiling of the input, element-wise.
    numpy.conj
        Return the complex conjugate, element-wise.
    numpy.cosh
        Hyperbolic cosine, element-wise.
    numpy.fabs
        Compute the absolute values element-wise.
    numpy.fmod
        Return the element-wise remainder of division.
    numpy.less
        Return the truth value of (x1 < x2) element-wise.
    numpy.log2
        Base-2 logarithm of `x`.
    numpy.sign
        Returns an element-wise indication of the sign of a number.
    numpy.sinh
        Hyperbolic sine, element-wise.
    numpy.tanh
        Compute hyperbolic tangent element-wise.
    numpy.equal
        Return (x1 == x2) element-wise.
    numpy.core._multiarray_umath.datetime_as_string
        Convert an array of datetimes into an array of strings.
    numpy.floor
        Return the floor of the input, element-wise.
    numpy.frexp
        Decompose the elements of x into mantissa and twos exponent.
    numpy.hypot
        Given the "legs" of a right triangle, return its hypotenuse.
    numpy.isinf
        Test element-wise for positive or negative infinity.
    numpy.ldexp
        Returns x1 * 2**x2, element-wise.
    numpy.trunc
        Return the truncated value of the input, element-wise.
    numpy.arccos
        Trigonometric inverse cosine, element-wise.
    numpy.arcsin
        Inverse sine, element-wise.
    numpy.arctan
        Trigonometric inverse tangent, element-wise.
    numpy.around
        Evenly round to the given number of decimals.
    numpy.divide
        Returns a true division of the inputs, element-wise.
    numpy.divmod
        Return element-wise quotient and remainder simultaneously.
    numpy.source
        Print or write to a file the source code for a NumPy object.
    numpy.square
        Return the element-wise square of the input.
    numpy.arccosh
        Inverse hyperbolic cosine, element-wise.
    numpy.arcsinh
        Inverse hyperbolic sine element-wise.
    numpy.arctan2
        Element-wise arc tangent of ``x1/x2`` choosing the quadrant correctly.
    numpy.arctanh
        Inverse hyperbolic tangent element-wise.
    numpy.deg2rad
        Convert angles from degrees to radians.
    numpy.degrees
        Convert angles from radians to degrees.
    numpy.greater
        Return the truth value of (x1 > x2) element-wise.
    numpy.rad2deg
        Convert angles from radians to degrees.
    numpy.radians
        Convert angles from degrees to radians.
    numpy.signbit
        Returns element-wise True where signbit is set (less than zero).
    numpy.spacing
        Return the distance between x and the nearest adjacent number.
    numpy.copysign
        Change the sign of x1 to that of x2, element-wise.
    numpy.diagonal
        Return specified diagonals.
    numpy.isfinite
        Test element-wise for finiteness (not infinity or not Not a Number).
    numpy.multiply
        Multiply arguments element-wise.
    numpy.negative
        Numerical negative, element-wise.
    numpy.subtract
        Subtract arguments, element-wise.
    numpy.heaviside
        Compute the Heaviside step function.
    numpy.logaddexp
        Logarithm of the sum of exponentiations of the inputs.
    numpy.nextafter
        Return the next floating-point value after x1 towards x2, element-wise.
    numpy.not_equal
        Return (x1 != x2) element-wise.
    numpy.left_shift
        Shift the bits of an integer to the left.
    numpy.less_equal
        Return the truth value of (x1 =< x2) element-wise.
    numpy.logaddexp2
        Logarithm of the sum of exponentiations of the inputs in base-2.
    numpy.logical_or
        Compute the truth value of x1 OR x2 element-wise.
    numpy.nan_to_num
        Replace NaN with zero and infinity with large finite numbers (default
    numpy.reciprocal
        Return the reciprocal of the argument, element-wise.
    numpy.bitwise_not
        Compute bit-wise inversion, or bit-wise NOT, element-wise.
    numpy.einsum_path
        Evaluates the lowest cost contraction order for an einsum expression by
    numpy.histogram2d
        Compute the bi-dimensional histogram of two data samples.
    numpy.logical_and
        Compute the truth value of x1 AND x2 element-wise.
    numpy.logical_not
        Compute the truth value of NOT x element-wise.
    numpy.logical_xor
        Compute the truth value of x1 XOR x2, element-wise.
    numpy.right_shift
        Shift the bits of an integer to the right.
    numpy.ma.abs
        Calculate the absolute value element-wise.
    numpy.ma.add
        Add arguments element-wise.
    numpy.ma.cos
        Cosine element-wise.
    numpy.ma.log
        Natural logarithm, element-wise.
    numpy.ma.mod
        Return element-wise remainder of division.
    numpy.ma.sin
        Trigonometric sine, element-wise.
    numpy.ma.tan
        Compute tangent element-wise.
    numpy.floor_divide
        Return the largest integer smaller or equal to the division of the inputs.
    numpy.ma.ceil
        Return the ceiling of the input, element-wise.
    numpy.ma.cosh
        Hyperbolic cosine, element-wise.
    numpy.fft.ifft
        Compute the one-dimensional inverse discrete Fourier Transform.
    numpy.ma.fabs
        Compute the absolute values element-wise.
    numpy.ma.fmod
        Return the element-wise remainder of division.
    numpy.ma.less
        Return the truth value of (x1 < x2) element-wise.
    numpy.ma.log2
        Base-2 logarithm of `x`.
    numpy.ma.sinh
        Hyperbolic sine, element-wise.
    numpy.ma.tanh
        Compute hyperbolic tangent element-wise.
    numpy.greater_equal
        Return the truth value of (x1 >= x2) element-wise.
    numpy.fft.ifftn
        Compute the N-dimensional inverse discrete Fourier Transform.
    numpy.ma.equal
        Return (x1 == x2) element-wise.
    numpy.ma.floor
        Return the floor of the input, element-wise.
    numpy.ma.hypot
        Given the "legs" of a right triangle, return its hypotenuse.
    numpy.busdaycalendar
        A business day calendar object that efficiently stores information
    numpy.ma.arccos
        Trigonometric inverse cosine, element-wise.
    numpy.ma.arcsin
        Inverse sine, element-wise.
    numpy.ma.arctan
        Trigonometric inverse tangent, element-wise.
    numpy.ma.divide
        Returns a true division of the inputs, element-wise.
    numpy.lib.recfunctions.unstructured_to_structured
        Converts and n-D unstructured array into an (n-1)-D structured array.
    numpy.ma.arccosh
        Inverse hyperbolic cosine, element-wise.
    numpy.ma.arcsinh
        Inverse hyperbolic sine element-wise.
    numpy.ma.arctan2
        Element-wise arc tangent of ``x1/x2`` choosing the quadrant correctly.
    numpy.ma.arctanh
        Inverse hyperbolic tangent element-wise.
    numpy.ma.greater
        Return the truth value of (x1 > x2) element-wise.
    numpy.ma.multiply
        Multiply arguments element-wise.
    numpy.ma.negative
        Numerical negative, element-wise.
    numpy.ma.subtract
        Subtract arguments, element-wise.
    numpy.ma.tests.test_subclassing.SubArray
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.ma.conjugate
        Return the complex conjugate, element-wise.
    numpy.ma.not_equal
        Return (x1 != x2) element-wise.
    numpy.ma.remainder
        Return element-wise remainder of division.
    numpy.ma.empty_like
        empty_like(prototype, dtype=None, order='K', subok=True, shape=None)
    numpy.ma.less_equal
        Return the truth value of (x1 =< x2) element-wise.
    numpy.ma.logical_or
        Compute the truth value of x1 OR x2 element-wise.
    numpy.ma.logical_and
        Compute the truth value of x1 AND x2 element-wise.
    numpy.ma.logical_not
        Compute the truth value of NOT x element-wise.
    numpy.ma.logical_xor
        Compute the truth value of x1 XOR x2, element-wise.
    numpy.ma.true_divide
        Returns a true division of the inputs, element-wise.
    numpy.ma.floor_divide
        Return the largest integer smaller or equal to the division of the inputs.
    numpy.ma.greater_equal
        Return the truth value of (x1 >= x2) element-wise.
    numpy.core.tests.test_function_base.PhysicalQuantity2
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.lib.tests.test_stride_tricks.SimpleSubClass
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.core.tests.test_numeric.TestKeepdims.sub_array
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.ma.tests.test_subclassing.ComplicatedSubArray
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.lib.tests.test_stride_tricks.VerySimpleSubClass
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.core.tests.test_multiarray.TestArrayPriority.Foo
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.core.tests.test_multiarray.TestArrayPriority.Bar
        ndarray(shape, dtype=float, buffer=None, offset=0,
    numpy.testing._gen_alignment_data
        generator producing data with different alignment and offsets
    numpy.random.RandomState.rand
        Random values in a given shape.

```python
x = np.array([1, 2, 3])
y = np.array([4, 5, 6])
# f-strings, Python 3.6 and above
print(f"x: {x}")
print(f"y: {y}")
```

    x: [1 2 3]
    y: [4 5 6]

```python
x + y
```

    array([5, 7, 9])

3. Speed

*Exercise*: Write a function `numpy_add` that takes two numpy arrays, loops over the entries, and computes the sum entry-by-entry and places the result into a third numpy array.

Tips:

* `def` - define a function
* `return x` - return `x` from the function
* `len(a)` - get the length of an array.
* `np.zeros(n)` - make a numpy array of length `n`
* `for i in range(0, len(a)): print(i)` - counting indices $0, ..., \mathrm{len}(a) - 1$

```python
def numpy_add(a, b):
    assert(len(a) == len(b))
    c = np.zeros(len(a))
    for i in range(0, len(a)):
        c[i] = a[i] + b[i]
    return c

a = np.ones(1000000)
b = np.ones(1000000)
```

```python
%time c = numpy_add(a, b)
```

    CPU times: user 886 ms, sys: 6.25 ms, total: 892 ms
    Wall time: 908 ms

```python
%time c = a + b
```

    CPU times: user 8.84 ms, sys: 2.78 ms, total: 11.6 ms
    Wall time: 10.8 ms

## Key points:
* Avoid writing 'tight' loops in Python.
  * Tight loop - looping over thousands of entries of data and performing a single operation on each piece of data.
* Think about how algorithms can be written without tight loops.
* Always prefer functionality in numpy that can act on many pieces of data in one call.

*Example*: Plot the function $\sin(x)$.

```python
x = np.linspace(0.0, 2.0*np.pi, num=50)
y = np.sin(x) # Apply sin function to all of the entries of x
print(f"x: {x}")
print(f"y: {y}")
```

    x: [0.         0.12822827 0.25645654 0.38468481 0.51291309 0.64114136
     0.76936963 0.8975979  1.02582617 1.15405444 1.28228272 1.41051099
     1.53873926 1.66696753 1.7951958  1.92342407 2.05165235 2.17988062
     2.30810889 2.43633716 2.56456543 2.6927937  2.82102197 2.94925025
     3.07747852 3.20570679 3.33393506 3.46216333 3.5903916  3.71861988
     3.84684815 3.97507642 4.10330469 4.23153296 4.35976123 4.48798951
     4.61621778 4.74444605 4.87267432 5.00090259 5.12913086 5.25735913
     5.38558741 5.51381568 5.64204395 5.77027222 5.89850049 6.02672876
     6.15495704 6.28318531]
    y: [ 0.00000000e+00  1.27877162e-01  2.53654584e-01  3.75267005e-01
      4.90717552e-01  5.98110530e-01  6.95682551e-01  7.81831482e-01
      8.55142763e-01  9.14412623e-01  9.58667853e-01  9.87181783e-01
      9.99486216e-01  9.95379113e-01  9.74927912e-01  9.38468422e-01
      8.86599306e-01  8.20172255e-01  7.40277997e-01  6.48228395e-01
      5.45534901e-01  4.33883739e-01  3.15108218e-01  1.91158629e-01
      6.40702200e-02 -6.40702200e-02 -1.91158629e-01 -3.15108218e-01
     -4.33883739e-01 -5.45534901e-01 -6.48228395e-01 -7.40277997e-01
     -8.20172255e-01 -8.86599306e-01 -9.38468422e-01 -9.74927912e-01
     -9.95379113e-01 -9.99486216e-01 -9.87181783e-01 -9.58667853e-01
     -9.14412623e-01 -8.55142763e-01 -7.81831482e-01 -6.95682551e-01
     -5.98110530e-01 -4.90717552e-01 -3.75267005e-01 -2.53654584e-01
     -1.27877162e-01 -2.44929360e-16]

```python
%matplotlib inline
import matplotlib.pyplot as plt
plt.xlabel(r"$x$")
plt.ylabel(r"$\sin(x)$")
plt.plot(x, y, 'o-')
```

## Slicing

*Slicing* allows us to select one or more than one elements of an existing array.

```python
a = np.arange(5)
print(a)
```

    [0 1 2 3 4]

## Selecting single elements of an array

* Select the first element

```python
a[0]
```

    0

* Select the third element (remember, index starts from zero!)

```python
a[2]
```

    2

* Select the fifth element (remember, index starts from zero!)

```python
a[4]
```

    4

* Select the last element, without knowing the length of the array.

```python
a[-1]
```

    4

## Selecting multiple elements of an array

* Select the first and second elements of the array.
* syntax: `[start:end]` exclusive of `end`
  * default start: `0`
  * default end: `N`

```python
print(a[0:len(a)])
print(a[:])
```

    [0 1 2 3 4]
    [0 1 2 3 4]

* First to end

```python
a[0:5]
```

    array([0, 1, 2, 3, 4])

```python
a[0:]
```

    array([0, 1, 2, 3, 4])

* Exercise: Select the second, third and fourth elements

```python
a[1:4]
```

    array([1, 2, 3])

* Exercise: Select everything.

```python
a[:]
```

    array([0, 1, 2, 3, 4])

## Advanced slicing

* Select every second element.
* Syntax: `[start:end:stride]`
  * default start: `0` for `stride > 0` and `-1` for `stride < 0`
  * default end: `N` for `stride > 0` and `- N - 1` for `stride < 0`
  * default stride: `1`

Full details: https://numpy.org/doc/stable/reference/arrays.indexing.html

```python
a[::2]
```

    array([0, 2, 4])

```python
a[::2]
```

    array([0, 2, 4])

Let's make a longer array.

```python
b = np.arange(0, 10)
print(b)
```

    [0 1 2 3 4 5 6 7 8 9]

* Exercise: Select every third entry starting with the second entry.

```python
b[1::3]
```

    array([1, 4, 7])

* *Exercise*: Starting with the last entry, select every third entry working backwards through the array. (Hint: Try playing with a `stride` of `-1`).

```python
b[-1::-3]
```

    array([9, 6, 3, 0])

* *Exercise*: Starting with the second to last entry, select every third entry working backwards through the array.

```python
b[-2::-3]
```

    array([8, 5, 2])

## N-dimensional indexing
* Slices in each axis are seperated by commas.
* `[start0:stop0:step0, start1:stop1:step1]`
  * Default: `[,]` = `[::,::]`

```python
c = np.arange(0, 32).reshape(8, 4)
print(c)
```

    [[ 0  1  2  3]
     [ 4  5  6  7]
     [ 8  9 10 11]
     [12 13 14 15]
     [16 17 18 19]
     [20 21 22 23]
     [24 25 26 27]
     [28 29 30 31]]

* Select the element on the third row in the fourth column.

```python
c[2, 3] # [axis0, axis1]
```

    11

* *Exercise*: From every second row, select every element.
  * Select every second row.

```python
c[::2, ::]
```

    array([[ 0,  1,  2,  3],
           [ 8,  9, 10, 11],
           [16, 17, 18, 19],
           [24, 25, 26, 27]])

```python
c[::2, ]
```

    array([[ 0,  1,  2,  3],
           [ 8,  9, 10, 11],
           [16, 17, 18, 19],
           [24, 25, 26, 27]])

```python
c[::2]
```

    array([[ 0,  1,  2,  3],
           [ 8,  9, 10, 11],
           [16, 17, 18, 19],
           [24, 25, 26, 27]])

* Exercise: Select the elements in every third row, starting with the second row, and in every second column.

```python
c[1::3, ::2]
```

    array([[ 4,  6],
           [16, 18],
           [28, 30]])

## Slicing based on integers
* Select the elements at (1, 1), (2, 2) and (3, 3).

```python
c[(1, 2, 3), (1, 2, 3)]
```

    array([ 5, 10, 15])

* Exercise: Select the elements at (5, 3) and (1, 2).

```python
c[[5, 1], [3, 2]] # Can use lists, ndarrays or tuples.
```

    array([23,  6])

## Slicing based on booleans

```python
c < 5
```

    array([[ True,  True,  True,  True],
           [ True, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False]])

```python
c < 5
```

    array([[ True,  True,  True,  True],
           [ True, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False],
           [False, False, False, False]])

## Mini-project: Mandelbrot set

#### Key Skills:

* Complex numbers
* Creating a grid of points in the complex plane
* Iteration
* Avoiding tight loops with operations on `numpy` arrays
* Boolean masking
* Basic plot

The Mandelbrot set is the set of complex numbers $c$ that remain bounded under repeated application of the function:
$$z_0 = c \quad z_{n + 1} = z_{n}^2 + c$$

### A reminder of complex numbers

$$z = x + j y$$

where $j = \sqrt{-1}$ and $x = \Re(z)$ and $y = \Im(z)$ are the real and complex parts of $z$.

```python
z = 3.0 + 1j*4.0
print("Real part of z: {}".format(np.real(z)))
print("Complex part of z: {}".format(np.complex(z)))
print("z + z = {}".format(z + z))
print("z**2 = {}".format(z**2))
```

    Real part of z: 3.0
    Complex part of z: (3+4j)
    z + z = (6+8j)
    z**2 = (-7+24j)

### Step 1: Create a grid of points $c$ in the complex plane

1. Tool: `np.linspace`. Create 300 points between -2 and 1 and place the result in `x`.
2. Tool: `np.linspace`. Create 300 points between -1.5 and 1.5 and place the result in `y`.
3. Tool: `x[:, np.newaxis]`, `y[np.newaxis, :]`. What does this do?

```python
x = np.linspace(-2, 1, 300)
y = np.linspace(-1.5, 1.5, 300)
c = x[:, np.newaxis] + 1j*y[np.newaxis, :]
```

### Step 2: Apply the function

The Mandelbrot set is the set of complex numbers $c$ that remain bounded under repeated application of the function:
$$z_0 = c \quad z_{n + 1} = z_{n}^2 + c$$

1. Tool: `for i in range(0, 50):`
2. `numpy` element-wise operations `z**2` `+` `c`
3. Making a copy of c `np.zeros_like(c)`

```python
num_iterations = 50
z = np.zeros_like(c)
for _ in range(0, num_iterations):
    z = z**2 + c
```

### Step 3: Select the $c$ "that remain bounded"

1. Tool: Boolean operation `z < bound`

```python
bound = 100.0
mandelbrot_set = z < bound
```

### Step 4: Plot the points $c$ that are in the Mandelbrot set

1. Tool: `plt.imshow`.

```python
import matplotlib.pyplot as plt
plt.imshow(mandelbrot_set.T, extent=[-2, 1, -1.5, 1.5])
plt.gray()
plt.xlabel("$\Re(c)$")
plt.ylabel("$\Im(c)$")
plt.show()
```

## Mini-project: Markov Chain

Key Skills:
* Calculating a transition matrix
* Defining a transition matrix in `numpy`
* Matrix-vector multiplication

Wikipedia: A *Markov chain* is a *stochastic model* describing a sequence of possible events in which the *probability of each event* depends only on the state attained in the previous event.

Imagine you are standing in a five room house.

*Drawing of floorplan of house*

The doors between the rooms of the house are locked.

Problem: Every hour the doors of the house are unlocked, and *you must* move to another room in the house. You have no preference over which room you want to be in next, so you move randomly between the rooms. **Which room will you spend the most time in?**

Now imagine you are in room 1. You could move to rooms 2, 4 and 5 (three rooms). You cannot move to room 3.

Denote $m_{ij}$ the *probability* of moving between room $j$ and room $i$.

What are $m_{21}$, $m_{41}$ and $m_{51}$? What are $m_{11}$ and $m_{31}$?

$$m_{21} = m_{41} = m_{51} = 1/3$$
$$m_{11} = m_{31} = 0$$

${m}_{1} = \begin{bmatrix}  m_{11} \\ m_{21} \\ m_{31} \\ m_{41}  \\m_{51} \end{bmatrix} = \begin{bmatrix}  0 \\ 1/3 \\ 0 \\ 1/3  \\ 1/3 \end{bmatrix} $

*Exercise*: Write down $m_1$, $m_2$, $m_3$, $m_4$, $m_5$.

The transition matrix $M$ has entries $m_{ij}$.

*Exercise*: Write the transition matrix $M$ on paper.

*Exercise*: Write the transition matrix $M$ in a `numpy` array.

$$M = \begin{bmatrix}
0 & 1/3 & 0 & 1/3 & 1/4 \\
1/3 & 0 & 1/3 & 0 & 1/4 \\
0 & 1/3 & 0 & 1/3 & 1/4 \\
1/3 & 0 & 1/3 & 0 & 1/4 \\
1/3 & 1/3 & 1/3 & 1/3 & 0 \\
\end{bmatrix}$$

```python
M = np.array([[0, 1/3, 0, 1/3, 1/4],
              [1/3, 0, 1/3, 0, 1/4],
              [0, 1/3, 0, 1/3, 1/4],
              [1/3, 0, 1/3, 0, 1/4],
              [1/3, 1/3, 1/3, 1/3, 0]])
print(M)
```

    [[0.         0.33333333 0.         0.33333333 0.25      ]
     [0.33333333 0.         0.33333333 0.         0.25      ]
     [0.         0.33333333 0.         0.33333333 0.25      ]
     [0.33333333 0.         0.33333333 0.         0.25      ]
     [0.33333333 0.33333333 0.33333333 0.33333333 0.        ]]

Question: Which room will you spend the most time in?

Let's assume we are certain we start in room 1.

$$ p_0 = \begin{bmatrix} 1.0 & 0.0 & 0.0 & 0.0 & 0.0 \end{bmatrix} $$

```python
p_0 = np.array([1.0, 0.0, 0.0, 0.0, 0.0])
```

Where will we end up next?

$$ p_1 = M p_0$$

```python
p_1 = M@p_0
print(p_1)
```

    [0.         0.33333333 0.         0.33333333 0.33333333]

```python
p_2 = M@p_1
print(p_2)
```

    [0.30555556 0.08333333 0.30555556 0.08333333 0.22222222]

```python
p_3 = M@p_2
print(p_3)
```

    [0.11111111 0.25925926 0.11111111 0.25925926 0.25925926]

```python
p = np.array([1.0, 0.0, 0.0, 0.0, 0.0])
for i in range(0, 50):
    p = M@p
print(p)
```

    [0.1875 0.1875 0.1875 0.1875 0.25  ]

*Question*: What happens if we are not sure where we start?

```python
p = np.array([0.25, 0.25, 0.0, 0.25, 0.25])
for i in range(0, 50):
    p = M@p
print(p)
```

    [0.1875 0.1875 0.1875 0.1875 0.25  ]

## Mini-project: Calculating $\pi$ with Monte Carlo

Key skills:
* Simulating random numbers
* Boolean operations
* Slicing with strides
* Counting
* Algorithms with no tight-loops

Imagine you are blind-folded and you have an infinite number of darts to throw at the following square board:

*Draw square with quarter circle*

How can we calculate the value of $\pi$?

*Monte Carlo* (random simulation) provides a powerful answer to this problem.

Steps:

1. Randomly throw (many) darts into the square (random simulation).
2. Calculate the ratio $r$ of the number of darts that fall inside the circle/total number thrown.
3. $\pi = 4r$

### Step 1: Simulate from uniform distribution

We want to 'simulate' the idea of throwing darts randomly at a square board.

Computers can *generate* (pseudo-)random numbers.

1. Tool: `np.random.uniform(size=())` to simulate N draws of uniformly distributed random numbers on the board.

Uniformly distributed - equally likely to be at any value between $[0, 1]$.

```python
N = 100000
darts = np.random.uniform(size=(N, 2))
plt.axis('equal')
plt.xlabel('x')
plt.ylabel('y')
plt.plot(darts[::100, 0], darts[::100, 1], 'x')
```

### Step 2: Calculate the squared distance from the point to the origin.

1. Tool: Slicing to get all x and all y coordinates.
2. Tool: `**2` to square entries of an array.

```python
dist = darts[:, 0]**2 + darts[:, 1]**2
```

### Step 3: Which darts landed inside the circle?

1. Tool: Boolean selection.

```python
inside_outside = dist < 1.0
print(inside_outside)
```

    [ True  True  True ...  True False  True]

### Step 4: Count the number of `True` values.

1. Tool: Use `np.count_nonzero`.

```python
np.lookfor('count')
```

    Search results for 'count'
    --------------------------
    numpy.bincount
        Count number of occurrences of each value in array of non-negative ints.
    numpy.busday_count
        Counts the number of valid days between `begindates` and
    numpy.nanmax
        Return the maximum of an array or maximum along an axis, ignoring any
    numpy.nanmin
        Return minimum of an array or minimum along an axis, ignoring any NaNs.
    numpy.count_nonzero
        Counts the number of non-zero values in the array ``a``.
    numpy.ma.count
        Count the non-masked elements of the array along the given axis.
    numpy.fromfile
        Construct an array from data in a text or binary file.
    numpy.fromiter
        Create a new 1-dimensional array from an iterable object.
    numpy.nancumsum
        Return the cumulative sum of array elements over a given axis treating Not a
    numpy.bytes0.count
        Return the number of non-overlapping occurrences of subsection sub in
    numpy.frombuffer
        Interpret a buffer as a 1-dimensional array.
    numpy.fromstring
        A new 1-D array initialized from text data in a string.
    numpy.nancumprod
        Return the cumulative product of array elements over a given axis treating Not a
    numpy.str0.count
        Return the number of non-overlapping occurrences of substring sub in
    numpy.unpackbits
        Unpacks elements of a uint8 array into a binary-valued output array.
    numpy.ma.count_masked
        Count the number of masked elements along the given axis.
    numpy.bytes0.replace
        Return a copy with all occurrences of substring old replaced by new.
    numpy.str0.replace
        Return a copy with all occurrences of substring old replaced by new.
    numpy.ma.frombuffer
        Interpret a buffer as a 1-dimensional array.
    numpy.random.Philox
        Container for the Philox (4x64) pseudo-random number generator.
    numpy.ma.MaskedArray.count
        Count the non-masked elements of the array along the given axis.
    numpy.testing._assert_valid_refcount
        Check that ufuncs don't mishandle refcount of object `1`.
    numpy.core._multiarray_umath.bincount
        Count number of occurrences of each value in array of non-negative ints.
    numpy.core._multiarray_umath.busday_count
        Counts the number of valid days between `begindates` and
    numpy.all
        Test whether all array elements along a given axis evaluate to True.
    numpy.any
        Test whether any array element along a given axis evaluates to True.
    numpy.core._multiarray_umath.unpackbits
        Unpacks elements of a uint8 array into a binary-valued output array.
    numpy.npv
        Returns the NPV (Net Present Value) of a cash flow series.
    numpy.ptp
        Range of values (maximum - minimum) along an axis.
    numpy.sin
        Trigonometric sine, element-wise.
    numpy.sum
        Sum of array elements over a given axis.
    numpy.flip
        Reverse the order of elements in an array along the given axis.
    numpy.ipmt
        Compute the interest portion of a payment.
    numpy.prod
        Return the product of array elements over a given axis.
    numpy.size
        Return the number of elements along a given axis.
    numpy.angle
        Return the angle of the complex argument.
    numpy.ravel
        Return a contiguous flattened array.
    numpy.fliplr
        Flip array in the left/right direction.
    numpy.flipud
        Flip array in the up/down direction.
    numpy.select
        Return an array drawn from elements in choicelist, depending on conditions.
    numpy.seterr
        Set how floating-point errors are handled.
    numpy.unique
        Find the unique elements of an array.
    numpy.average
        Compute the weighted average along the specified axis.
    numpy.nonzero
        Return the indices of the elements that are non-zero.
    numpy.reshape
        Gives a new shape to an array without changing its data.
    numpy.char.count
        Returns an array with the number of non-overlapping occurrences of
    numpy.convolve
        Returns the discrete, linear convolution of two one-dimensional sequences.
    numpy.digitize
        Return the indices of the bins to which each value in input array belongs.
    numpy.errstate
        errstate(**kwargs)
    numpy.gradient
        Return the gradient of an N-dimensional array.
    numpy.chararray
        chararray(shape, itemsize=1, unicode=False, buffer=None, offset=0,
    numpy.histogram
        Compute the histogram of a set of data.
    numpy.is_busday
        Calculates which of the given dates are valid days, and which are not.
    numpy.seterrcall
        Set the floating-point error callback function or log object.
    numpy.einsum_path
        Evaluates the lowest cost contraction order for an einsum expression by
    numpy.histogram2d
        Compute the bi-dimensional histogram of two data samples.
    numpy.histogramdd
        Compute the multidimensional histogram of some data.
    numpy.result_type
        result_type(*arrays_and_dtypes)
    numpy.ma.dot
        Return the dot product of two arrays.
    numpy.ma.sin
        Trigonometric sine, element-wise.
    numpy.fft.hfft
        Compute the FFT of a signal that has Hermitian symmetry, i.e., a real
    numpy.ma.diag
        Extract a diagonal or construct a diagonal array.
    numpy.busday_offset
        First adjusts the date to fall on a valid day according to
    numpy.chararray.count
        Returns an array with the number of non-overlapping occurrences of
    numpy.datetime_data
        Get information about the step size of a date or time type.
    numpy.ma.size
        Return the number of elements along a given axis.
    numpy.bytes0.hex
        Create a str of hexadecimal numbers from a bytes object.
    numpy.ma.angle
        Return the angle of the complex argument.
    numpy.fft.irfft
        Compute the inverse of the n-point DFT for real input.
    numpy.busdaycalendar
        A business day calendar object that efficiently stores information
    numpy.ma.ravel
        Returns a 1D version of self, as a view.
    numpy.fft.irfftn
        Compute the inverse of the N-dimensional FFT of real input.
    numpy.char.replace
        For each element in `a`, return a copy of the string with all
    numpy.ma.average
        Return the weighted average of array over the given axis.
    numpy.ma.ediff1d
        Compute the differences between consecutive elements of an array.
    numpy.ma.nonzero
        nonzero(self)
    numpy.matrix.ravel
        Return a flattened matrix.
    numpy.random.SFC64
        BitGenerator for Chris Doty-Humphrey's Small Fast Chaotic PRNG.
    numpy.ufunc.reduce
        Reduces `a`'s dimension by one, by applying ufunc along one axis.
    numpy.histogram_bin_edges
        Function to calculate only the edges of the bins used by the `histogram`
    numpy.chararray.resize
        Change shape and size of array in-place.
    numpy.linalg.matrix_rank
        Return matrix rank of array using SVD method
    numpy.core.tests.test_dtype.TestStructuredObjectRefcounting
        These tests cover various uses of complicated structured types which
    numpy.ma.MaskedArray.ravel
        Returns a 1D version of self, as a view.
    numpy.ma.MaskedArray.nonzero
        Return the indices of unmasked elements that are not zero.
    numpy.random.Generator.zipf
        Draw samples from a Zipf distribution.
    numpy.random.Generator.gamma
        Draw samples from a Gamma distribution.
    numpy.random.Generator.power
        Draws samples in [0, 1] from a power distribution with positive
    numpy.random.Generator.gumbel
        Draw samples from a Gumbel distribution.
    numpy.random.Generator.normal
        Draw random samples from a normal (Gaussian) distribution.
    numpy.random.Generator.pareto
        Draw samples from a Pareto II or Lomax distribution with
    numpy.random.RandomState.zipf
        Draw samples from a Zipf distribution.
    numpy.random.Generator.laplace
        Draw samples from the Laplace or double exponential distribution with
    numpy.random.Generator.poisson
        Draw samples from a Poisson distribution.
    numpy.random.Generator.uniform
        Draw samples from a uniform distribution.
    numpy.random.Generator.weibull
        Draw samples from a Weibull distribution.
    numpy.random.RandomState.gamma
        Draw samples from a Gamma distribution.
    numpy.random.RandomState.power
        Draws samples in [0, 1] from a power distribution with positive
    numpy.random.Generator.binomial
        Draw samples from a binomial distribution.
    numpy.random.Generator.logistic
        Draw samples from a logistic distribution.
    numpy.random.RandomState.gumbel
        Draw samples from a Gumbel distribution.
    numpy.random.RandomState.normal
        Draw random samples from a normal (Gaussian) distribution.
    numpy.random.RandomState.pareto
        Draw samples from a Pareto II or Lomax distribution with
    numpy.random.Generator.lognormal
        Draw samples from a log-normal distribution.
    numpy.random.Generator.logseries
        Draw samples from a logarithmic series distribution.
    numpy.random.RandomState.laplace
        Draw samples from the Laplace or double exponential distribution with
    numpy.random.RandomState.poisson
        Draw samples from a Poisson distribution.
    numpy.random.RandomState.uniform
        Draw samples from a uniform distribution.
    numpy.random.RandomState.weibull
        Draw samples from a Weibull distribution.
    numpy.random.RandomState.binomial
        Draw samples from a binomial distribution.
    numpy.random.RandomState.logistic
        Draw samples from a logistic distribution.
    numpy.random.Generator.multinomial
        Draw samples from a multinomial distribution.
    numpy.random.RandomState.lognormal
        Draw samples from a log-normal distribution.
    numpy.random.RandomState.logseries
        Draw samples from a logarithmic series distribution.
    numpy.core._multiarray_umath.is_busday
        Calculates which of the given dates are valid days, and which are not.
    numpy.random.RandomState.multinomial
        Draw samples from a multinomial distribution.
    numpy.core._multiarray_umath.result_type
        result_type(*arrays_and_dtypes)
    numpy.random.Generator.standard_gamma
        Draw samples from a standard Gamma distribution.
    numpy.core._multiarray_umath.busday_offset
        First adjusts the date to fall on a valid day according to
    numpy.random.RandomState.standard_gamma
        Draw samples from a standard Gamma distribution.
    numpy.random.RandomState.random_integers
        Random integers of type `np.int_` between `low` and `high`, inclusive.
    numpy.lib.recfunctions.structured_to_unstructured
        Converts and n-D structured array into an (n+1)-D unstructured array.
    numpy.lib.recfunctions.unstructured_to_structured
        Converts and n-D unstructured array into an (n-1)-D structured array.
    numpy.random.Generator.multivariate_hypergeometric
        multivariate_hypergeometric(colors, nsample, size=None,
    numpy.core.tests.test_dtype.iter_struct_object_dtypes
        Iterates over a few complex dtypes and object pattern which
    numpy.testing._private.noseclasses.KnownFailure
        Plugin that installs a KNOWNFAIL error class for the
    numpy.core.tests.test_dtype.TestStructuredObjectRefcounting.test_structured_object_indexing
        Structured object reference counting for advanced indexing.
    numpy.core.tests.test_dtype.TestStructuredObjectRefcounting.test_structured_object_item_setting
        Structured object reference counting for simple item setting
    numpy.core.tests.test_dtype.TestStructuredObjectRefcounting.test_structured_object_create_delete
        Structured object reference counting in creation and deletion
    numpy.fft.tests.test_fft.TestFFT1D.assertCountEqual
        Asserts that two iterables have the same elements, the same number of
    numpy.core.tests.test_dtype.TestStructuredObjectRefcounting.test_structured_object_take_and_repeat
        Structured object reference counting for specialized functions.

2. Found tool: np.count_nonzero

```python
num_inside = np.count_nonzero(inside_outside)
print(num_inside)
```

    78599

### Step 5. Calculate $\pi$!

```python
estimate_pi = 4.0*num_inside/N
print(estimate_pi)
print(np.pi)
print(np.abs(np.pi - estimate_pi))
```

    3.14396
    3.141592653589793
    0.00236734641020675