# Multilateration in 2D: IoT/LoRaWAN Mass Surveillance
"*Multilateration (MLAT) is a surveillance technique based on the measurement of the difference in distance to two stations at known locations by broadcast signals at known times.*" - [en.wikipedia.org/wiki/Multilateration](http://en.wikipedia.org/wiki/Multilateration)

## Abstract
A single motivated individual with several hundred USD and a hobbyist level of competency in electronics and programming would be able to set up a mass surveillance system to track individual LoRa IoT devices on the scale of a small city, provided that the geography of the city is accommodating to the placement of receiving devices (e.g. surrounded by hills).

## Locating the source of a signal transmission - intuition

When a signal is transmitted by a radio device it propagates through the atmosphere at approximately the speed of light - that is, approximately $v = 3 \text{e} 8ms^{-1}$.

If this signal is received by two receivers (towers), the time difference of arrival ($\text{TDOA}$) between the first and second tower can be determined. $\text{TDOA} = t_1 - t_0$ where $t_0$ is the time the signal is received first and $t_1$ is the time the signal is received second. The towers require synchronized high accuracy clocks, which can be achieved in practice with a GPS clock on each tower.

The below animation shows a signal propagating from a transmitter "Tx" and being received by two towers at different times, allowing the $\text{TDOA}$ to be calculated.

This $\text{TDOA}$ can be used to determine the *difference* in distance that the transmitter is located from the towers. For example, assume that the transmitter is on a circle of radius $d$ metres from tower 1, where the signal was received first. Then it must also be on a circle of radius $d+v \times \text{TDOA}$ metres from tower 0, where the signal was received second. The device must then lie at the intersection of the two circles, giving two possible locations (or one if the circles only touch, or none if the circles do not touch).

By iterating over a range of values for $d$ and at each iteration finding the intersection of the two circles, a locus of possible transmitter locations can be determined. This produces a hyperbolic curve on which the transmitter lies. Notably, it is not required to know *when* the signal was first transmitted - as you would with trilateration - and so no communication with the transmitter is required beyond simply identifying the signal.

The below animation shows circles of increasing radius around the two towers and the resulting hyperbolic locus of intersections as $d$ is increased. The circle around tower 1 has a radius of $r_1=d$, and the circle around tower 0 has a radius of $r_0=d+\text{TDOA} \times v$ (noting $\text{TDOA} \times v$ is constant and $\text{TDOA}$ comes from the previous animation).

With $n$ towers, this process can be repeated between the tower that first received the message and every other tower to produce $n-1$ loci. In general, with three towers the device location can be narrowed down to at least two positions and with four towers to exactly one position. There are some cases, depending on the relative geometry of the towers and transmitter and the error in the timestamping, where this is not the case.

The accuracy of the determined location will depend on the accuracy of the timestamping, the effects of diffraction and obstacles on path length, the relative geometry of the transmitter and towers, etc. The approximation of the problem to two dimensions will also introduce error.

## Analysis
Multilateration was just explained in an intuitive manner. However, it is convenient to have a set of mathematical expressions that describe the system if the location of the transmitter is to be found. This analysis is inspired heavily by [André Andersen's](http://blog.andersen.im/2012/07/signal-emitter-positioning-using-multilateration) similar work.

Consider, in Euclidean $\mathbb{R}^{2}$ space, a transmitter located at $\vec{x}$ whose transmission at time $t_0$ propagates at speed $v \ ms^{-1}$ and is received by a set of $n$ towers. Let the tower that receives this transmission first be at location $\vec{p}_c$ with receive time $t_c$. Let the remaining $n-1$ towers be located at $\vec{p}_i$ with transmission receive times $t_i$.

For the first tower, $\vec{p}_c$, we can say that the distance between the transmitter $\vec{x}$ and the tower is equal to the transmission propagation speed multiplied by the time of flight.
$$
\|\vec{x} - \vec{p}_c\| = v(t_c-t_0)
$$

We can make a similar statement for each of the other towers.
$$
\begin{align}
\|\vec{x} - \vec{p}_i\| &= v(t_i-t_0) \\
&= v(t_i - t_c + t_c - t_0) \\
&= v(t_i - t_c) + v(t_c - t_0) \\
\end{align}
$$

Combining these two expressions we obtain the following, where the only unknown is $\vec{x}$.
$$
\|\vec{x} - \vec{p}_c\| = \|\vec{x} - \vec{p}_i\| - v(t_i - t_c)
$$

Expanding this out we obtain a set of $n-1$ expressions, where there are $n$ towers in total. Subscript $x$ and $y$ denote the x and y components of a vector respectively.
$$
\begin{align}
&\|\vec{x} - \vec{p}_c\| - \|\vec{x} - \vec{p}_i\| + v(t_i - t_c) &= 0 &, \quad i = 1, ..., n-1 \\
\Rightarrow \ & \sqrt{(\vec{x}_x - \vec{p}_{c,x})^2 + (\vec{x}_y - \vec{p}_{c,y})^2}
              - \sqrt{(\vec{x}_x - \vec{p}_{i,x})^2 + (\vec{x}_y - \vec{p}_{i,y})^2}
              + v(t_i - t_c) &= 0 &, \quad i = 1, ..., n-1 \\
\end{align}
$$

Each of these equations represents a single hyperbola. These hyperbolas can be plotted individually, producing the same graphs shown above by taking circle intersections, or they can be solved by finding a value of $\vec{x}$ that minimizes their error.

## Implications: IoT and LoRaWAN

Multilateration can be used to locate the source of a transmission if
1. the transmission can be received in several locations, and
2. the receive time can be accurately recorded at each location.

Furthermore, if one transmitter's signal contains a persistent and unencrypted transmitter ID then it will be possible to track the location of that device.

One emerging IoT technology, LoRa/LoRaWAN, allows these requirements to be satisfied cheaply with hobbyist equipment while also transmitting a pseudo-persistent unencrypted device ID. Every device in a LoRaWAN network is given a 32-bit device address - akin to an IPv4 or IPv6 address - that uniquely identifies it on a network. This address is not globally unique, but is unique within a single LoRaWAN network. This address is assigned by the network operator and can be re-assigned at any time, though typically the same address is used for a substantial length of time, making the address pseudo-persistent - again akin to an IPv4 or IPv6 address. In cases where the device is activated by personalization, the device will always use the same address.

Sections 4.3.1.6, 4.3.3, and 4.4.2 of the LoRaWAN [specification](./lorawantm_specification_-v1.1.pdf) specify the encryption of uplink messages (messages sent from an IoT device to the network). Notably, the device address (DevAddr) is never encrypted; in fact it is required that the address be known (unencrypted) in order to decrypt the encrypted portions of the message. Any person listening for LoRa transmissions will receive these messages and will be able to read the device address.

LoRa transmissions can typically travel at least 1km, and distances of up to at least 15km can be attained in real-world conditions. As a result, only a low density of receivers need be installed in order to receive all messages over a large geographical area. For example the city of Wellington, New Zealand, is surrounded by hills. A handful of receivers atop, or partway up, some of the hills would be sufficient to cover the entire city. To cover the main portion of the city with three or four overlapping receivers, as required for multilateration, would not be difficult.

LoRa receivers with GPS are cheap - the Dragino LoRa/GPS Shield for Arduino comes with a LoRa module and a GPS module with 10ns timing accuracy and typically sells for about 50 USD. This device is additionally easy to configure and use, with an abundance of resources available online.

## Summary
Given these three points - LoRa messages contain a pseudo-persistent unique ID, LoRa messages can be received over large distances, and LoRa messages can be received and accurately timestamped with cheap hobbyist equipment - there is a very real possibility to create a passive mass surveillance network that tracks individual LoRa devices. This is a severe privacy concern for applications such as vehicle, person (e.g. elderly), and goods tracking.

## This notebook

This notebook presents a small python script that produces a set of loci of possible transmitter locations given a set of towers with locations and signal receive times. Additionally, the code used to create the animations is given.

```python
from multilaterate import get_loci
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import least_squares
```

```python
# How many towers. All towers recieve the transmission.
num_towers = 4

# Metre length of a square containing the transmitting
# device, centred around (x, y) = (0, 0). Device will be randomly placed
# in this area.
tx_square_side = 5e3

# Metre length of a square containing the towers,
# centred around (x, y) = (0, 0). towers will be randomly placed
# in this area.
rx_square_side = 25e3

# Speed of transmission propogation. Generally equal to speed of
# light for radio signals.
v = 3e8

# Time at which transmission is performed. Really just useful to
# make sure the code is using relative times rather than depending on one
# of the receive times being zero.
t_0 = 2.5

# Metre increments to radii of circles when generating locus of
# circle intersection.
delta_d = int(100)

# Max distance a transmission will be from the tower that first
# received the transmission. This puts an upper bound on the radii of the
# circle, thus limiting the size of the locus to be near the towers.
max_d = int(20e3)

# Standard deviation of noise added to the
# receive times at the towers. Mean is zero.
rec_time_noise_stdd = 1e-6

# Whether to plot circles that would be
# used if performing trilateration. These are circles that are centred
# on the towers and touch the transmitter site.
plot_trilateration_circles = False

# Whether to plot a straight line
# between every pair of towers. This is useful for visualising the
# hyperbolic loci focal points.
plot_lines_between_towers = False
```

```python
# Generate towers with x and y coordinates.
# for tower i: x, y = towers[i][0], towers[i][1]
towers = (np.random.rand(num_towers, 2)-0.5) * rx_square_side
print('towers:\n', towers)

# location of transmitting device with tx[0] being x and tx[1] being y.
tx = (np.random.rand(2)-0.5) * tx_square_side
print('tx:', tx)

# Distances from each tower to the transmitting device,
# simply triangle hypotenuse.
# distances[i] is distance from tower i to transmitter.
distances = np.array([ ( (x[0]-tx[0])**2 + (x[1]-tx[1])**2 )**0.5
                       for x in towers])
print('distances:', distances)

# Time at which each tower receives the transmission.
rec_times = distances/v + t_0
# Add noise to receive times
rec_times += np.random.normal(loc=0, scale=rec_time_noise_stdd,
                              size=num_towers)
print('rec_times:', rec_times)

# Get the loci.
loci = get_loci(rec_times, towers, v, delta_d, max_d)

# Plot towers and transmission location.
fig, ax = plt.subplots(figsize=(5,5))
max_width = max(tx_square_side, rx_square_side)/2
ax.set_ylim((max_width*-1, max_width))
ax.set_xlim((max_width*-1, max_width))
for i in range(towers.shape[0]):
    x = towers[i][0]
    y = towers[i][1]
    ax.scatter(x, y)
    ax.annotate('Tower '+str(i), (x, y))
ax.scatter(tx[0], tx[1])
ax.annotate('Tx', (tx[0], tx[1]))

# Iterate over every unique combination of towers and plot nifty stuff.
for i in range(num_towers):
    if(plot_trilateration_circles):
        # Circle from tower i to tx site
        circle1 = (towers[i][0], towers[i][1], distances[i])
        circle = plt.Circle((circle1[0], circle1[1]),
                            radius=circle1[2], fill=False)
        ax.add_artist(circle)
    for j in range(i+1, num_towers):
        if(plot_lines_between_towers):
            # Line between towers
            ax.plot((towers[i][0], towers[j][0]),
                    (towers[i][1], towers[j][1]))

for locus in loci:
    ax.plot(locus[0], locus[1])
plt.show()

```

```python
# plot the same hyperbola using the derived expressions.

fig, ax = plt.subplots(figsize=(5,5))
max_width = max(tx_square_side, rx_square_side)/2
ax.set_ylim((max_width*-1, max_width))
ax.set_xlim((max_width*-1, max_width))

c = np.argmin(rec_times)  # Tower that received first.
p_c = towers[c]
t_c = rec_times[c]

x = np.linspace(towers[i][0] - 50000, towers[i][0] + 50000, 100)
y = np.linspace(towers[i][1] - 50000, towers[i][1] + 50000, 100)
x, y = np.meshgrid(x, y)

for i in range(num_towers):
    if i == c:
        continue

    p_i = towers[i]
    t_i = rec_times[i]

    plt.contour(
        x, y,
        (
           np.sqrt((x-p_c[0])**2 + (y-p_c[1])**2)
         - np.sqrt((x-p_i[0])**2 + (y-p_i[1])**2)
         + v*(t_i - t_c)
        ),
        [0])
```

```python
# Solve the location of the transmitter.

c = np.argmin(rec_times)
p_c = np.expand_dims(towers[c], axis=0)
t_c = rec_times[c]

# Remove the c tower to allow for vectorization.
all_p_i = np.delete(towers, c, axis=0)
all_t_i = np.delete(rec_times, c, axis=0)

def eval_solution(x):
    """ x is 2 element array of x, y of the transmitter"""
    return (
          np.linalg.norm(x - p_c, axis=1)
        - np.linalg.norm(x - all_p_i, axis=1)
        + v*(all_t_i - t_c)
    )

# Initial guess.
x_init = [0, 0]

# Find a value of x such that eval_solution is minimized.
# Remember the receive times have error added to them: rec_time_noise_stdd.
res = least_squares(eval_solution, x_init)

print(f"Actual emitter location:    ({tx[0]:.1f}, {tx[1]:.1f}) ")
print(f"Calculated emitter locaion: ({res.x[0]:.1f}, {res.x[1]:.1f})")
print(f"Error in metres:            {np.linalg.norm(tx-res.x):.1f}")

# And now plot the solution.
fig, ax = plt.subplots(figsize=(5,5))
max_width = max(tx_square_side, rx_square_side)/2
ax.set_ylim((max_width*-1, max_width))
ax.set_xlim((max_width*-1, max_width))

for locus in loci:
    ax.plot(locus[0], locus[1])

ax.scatter(tx[0], tx[1], color='red')
ax.annotate('Actual', (tx[0], tx[1]))

ax.scatter(res.x[0], res.x[1], color='blue')
ax.annotate('Solved', (res.x[0], res.x[1]))

plt.show()

```

## Animations
Below cells generate the animations used above. Some variable re-use.

```python
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.animation import FuncAnimation
from IPython.display import HTML, Image
from multilaterate import get_locus

# For animation GIF output:
# brew install imagemagick
# brew works on mac, different for different OS I guess.

# If using to_html5_video:
'''
plt.rcParams['animation.ffmpeg_path'] = '/usr/local/Cellar/' \
                                        'ffmpeg/4.0.2/bin/ffmpeg'
'''

towers = np.array([[5e3, 13e3], [16e3, 5e3], [5e3, 7e3],
                   [12.5e3, 1e3]])
tx = np.array([15e3, 8e3])
distances = np.array([ ( (x[0]-tx[0])**2 + (x[1]-tx[1])**2 )**0.5
                       for x in towers])
rec_times = distances/v

v = 3e8
delta_d = 10
```

```python
## Create a visualisation for TDOA

# Plot towers and transmission location.
fig, ax = plt.subplots(figsize=(5,5))
ax.set_ylim((0, 20e3))
ax.set_xlim((0, 20e3))
for i in range(2):
    x = towers[i][0]
    y = towers[i][1]
    ax.scatter(x, y)
    ax.annotate('Tower '+str(i), (x, y))
ax.scatter(tx[0], tx[1])
ax.annotate('Tx', (tx[0], tx[1]))

circle = plt.Circle((tx[0], tx[1]),
                    radius=1, fill=False)
ax.add_artist(circle)

# Annotations, to be updated during animation
cur_time = ax.annotate('t = 0', (1e3, 19e3))
t0 = ax.annotate('Tower 0 received at t = ', (1e3, 18e3))
t1 = ax.annotate('Tower 1 received at t = ', (1e3, 17e3))
TDOA_ann = ax.annotate('TDOA = ', (1e3, 16e3))

v_vec = ax.arrow(tx[0], tx[1], 0, 1e3,
                 head_width=500, head_length=500, fc='k', ec='k')
v_ann = ax.annotate('v = {:.0E} m/s'.format(v), (tx[0], tx[1]+1e3))

n_frames = 300
max_seconds = 50e-6

t0_rec = 0
t1_rec = 0
TDOA = 0

def animate(i):
    global t0_rec, t1_rec, TDOA, v_vec, v_ann

    t = i/n_frames * max_seconds
    radius = v * t

    v_vec.remove()
    v_vec = ax.arrow(tx[0], tx[1]+radius, 0, 1e3,
                 head_width=500, head_length=500, fc='k', ec='k')
    v_ann.set_position((tx[0] - 0.5e3, tx[1]+radius+1.5e3))

    circle.radius = radius

    cur_time.set_text('t = {:.2E} s'.format(t))

    if(t >= rec_times[0] and t0_rec == 0):
        t0_rec = t
        t0.set_text(t0.get_text() + '{:.2E} s'.format(t0_rec))
    if(t >= rec_times[1] and t1_rec == 0):
        t1_rec = t
        t1.set_text(t1.get_text() + '{:.2E} s'.format(t1_rec))
    if(t0_rec != 0 and t1_rec != 0 and TDOA == 0):
        TDOA = abs(t1_rec-t0_rec)
        TDOA_ann.set_text(TDOA_ann.get_text() \
                          + '{:.2E} s'.format(TDOA))

anim = FuncAnimation(fig, animate,
                     frames=n_frames, interval=16.6, blit=False)

# HTML(anim.to_html5_video()) # doesn't play nice with github markdown
anim.save('animations/tdoa.gif', writer='imagemagick', fps=60)
plt.close()
Image(url='animations/tdoa.gif')

```

```python
## Create a visualisation for locus

TDOA_dist = v*TDOA

# Plot towers and transmission location.
fig, ax = plt.subplots(figsize=(5,5))
ax.set_ylim((0, 20e3))
ax.set_xlim((0, 20e3))
for i in range(2):
    x = towers[i][0]
    y = towers[i][1]
    ax.scatter(x, y)
    ax.annotate('Tower '+str(i), (x, y))
ax.scatter(tx[0], tx[1])
ax.annotate('Tx', (tx[0], tx[1]))

circle0 = plt.Circle((towers[0][0], towers[0][1]),
                    radius=1e3, fill=False)
circle1 = plt.Circle((towers[1][0], towers[1][1]),
                    radius=1e3, fill=False)
ax.add_artist(circle0)
ax.add_artist(circle1)

# Annotations, to be updated during animation
cur_d_ann = ax.annotate('d = 0', (1e3, 19e3))
r0_ann = ax.annotate('Tower 0 radius r0 = ', (1e3, 18e3))
r1_ann = ax.annotate('Tower 1 radius r1 = ', (1e3, 17e3))

n_frames = 300
max_d = 10e3

locus_plot, = ax.plot([], [], marker=None)

def animate(i):
    # global t0_rec, t1_rec, TDOA, v_vec, v_ann

    d = i/n_frames * max_d

    circle0.radius = d + TDOA_dist
    circle1.radius = d

    cur_d_ann.set_text('d = {:.2E} m'.format(d))
    r0_ann.set_text('Tower 0 radius r0 = {:.2E} m'.format(d
                                                + TDOA_dist))
    r1_ann.set_text('Tower 1 radius r1 = {:.2E} m'.format(d))

    locus = get_locus(tower_1 = (towers[0][0], towers[0][1]),
                      tower_2 = (towers[1][0], towers[1][1]),
                      time_1 = t0_rec, time_2 = t1_rec,
                      v = v, delta_d = delta_d, max_d=d)
    locus_plot.set_xdata(locus[0])
    locus_plot.set_ydata(locus[1])

anim = FuncAnimation(fig, animate,
                     frames=n_frames, interval=16.6, blit=False)

# HTML(anim.to_html5_video()) # doesn't play nice with github markdown
anim.save('animations/locus.gif', writer='imagemagick', fps=60)
plt.close()
Image(url='animations/locus.gif')
```

```python
## Create a visualisation for loci

# Plot towers and transmission location.
fig, ax = plt.subplots(figsize=(5,5))
ax.set_ylim((0, 20e3))
ax.set_xlim((0, 20e3))

for i in range(towers.shape[0]):
    x = towers[i][0]
    y = towers[i][1]
    ax.scatter(x, y)
    ax.annotate('Tower '+str(i), (x, y))
ax.scatter(tx[0], tx[1])
ax.annotate('Tx', (tx[0], tx[1]))

circles = []
locus_plots = []
for i in range(towers.shape[0]):
    circle = plt.Circle((towers[i][0], towers[i][1]),
                         radius=0, fill=False)
    ax.add_artist(circle)
    circles.append(circle)

    locus_plot, = ax.plot([], [], marker=None)
    locus_plots.append(locus_plot)

# Annotations, to be updated during animation
cur_d_ann = ax.annotate('d = 0', (1e3, 19e3))

n_frames = 600
max_d = 10e3

def animate(i):
    # global t0_rec, t1_rec, TDOA, v_vec, v_ann

    d = i/n_frames * max_d
    cur_d_ann.set_text('d = {:.2E} m'.format(d))

    first_tower = int(np.argmin(rec_times))
    circles[first_tower].radius = d

    for j in [x for x in range(towers.shape[0]) if x!= first_tower]:
        # print('tower', str(first_tower), 'to', str(j))
        locus = get_locus(tower_1=(towers[first_tower][0],
                                   towers[first_tower][1]),
                          tower_2=(towers[j][0], towers[j][1]),
                          time_1=rec_times[first_tower],
                          time_2=rec_times[j],
                          v=v, delta_d=delta_d, max_d=d)
        locus_plots[j].set_xdata(locus[0])
        locus_plots[j].set_ydata(locus[1])

        TDOA_j = v * (rec_times[j] - rec_times[first_tower])
        circles[j].radius = d + TDOA_j

anim = FuncAnimation(fig, animate,
                     frames=n_frames, interval=16.6, blit=False)

# HTML(anim.to_html5_video()) # doesn't play nice with github markdown
anim.save('animations/loci.gif', writer='imagemagick', fps=60)
plt.close()
Image(url='animations/loci.gif')
```


Contenido bajo licencia Creative Commons BY 4.0 y código bajo licencia MIT. © Juan Gómez y Nicolás Guarín-Zapata 2020. Este material es parte del curso Modelación Computacional en el programa de Ingeniería Civil de la Universidad EAFIT.

# Integración numerica

## Introducción

Discutiremos de manera breve la definición de cuadratura. Posteriormente nos concentraremos en las cuadraturas Gaussianas que por su eficiencia y facilidad de sistematización son de amplio uso en ingeniería y física. Para estas cubriremos su desarrollo general y su implementación en Python. Los detalles de la cuadratura Gaussiana y su implementación se discutirán por medio de un ejemplo.

**Al completar este notebook usted debería estar en la capacidad de:**

* Identificar una cuadratura como una formula de evaluar integrales numéricamente.

* Identificar la relación entre la función a integrar y el tipo de esquema requerido para su evaluación.

* Evaluar integrales numéricamente usando cuadraturas Gaussianas.

## Cuadraturas

Una cuadratura es una formula para la evaluación numerica de integrales de la forma general:

$$I=\int\limits_{V(\vec{x})} f(\vec{x}) \mathrm{d}V(\vec{x}) \approx\sum_{i=1}^{N} f(\vec{x}_i)w_i\, .$$

Note que esta expresión corresponde a la evaluación de la función $f(x)$ en $N$ puntos de coordenadas $x_i$ multiplicados por $N$ factores $w_i$. Los factores se denominan **pesos** o factores de ponderación ya que se encargan de ponderar la contribución de cada término $f(x_i)$ a $I$ y tienen una interpretación similar al diferencial $\mathrm{d}V$. Incluso, estos últimos son los que se encargarían de aportar las unidades pertinentes a la integral (aproximada).

### Ejemplo: regla del trapecio

Una cuadratura con la cual estamos familiarizados es la regla del trapecio dada por:

$$I=\int\limits_a^b f(x) \mathrm{d}x \approx \frac{h}{2}[f(a) + f(b)]\, ,$$

en donde $h = b - a$. En esta expresión podemos reconocer los factores de ponderación $w_1 = h/2$, $w_2 = h/2$ y los puntos de evaluación $x_1 = a$ y $x_2 = b$.

Por ejemplo, consideremos la siguiente integral:

$$I = \int\limits_{-1}^{+1} (x^3 + 4x^2 - 10) \mathrm{d}x \approx 1.0\cdot f(-1) + 1.0\cdot f(+1) = -12\, .$$

### Cuadraturas Gaussianas

Una de las cuadraturas mas poderosas encontradas en la practica son las denominadas cuadraturas [Gaussianas](https://en.wikipedia.org/wiki/Gaussian_quadrature). En estas, los factores de ponderación $w_i$ y los puntos de evaluación $x_i$ son seleccionados de manera que se obtenga la mejor aproximación (mínimo error) de la manera más efectiva (mínimo número de puntos de evaluación). El ser formuladas usando un proceso de ajuste de $2 N$ parámetros correspondientes a los $N$ pesos y a los $N$ puntos de evaluación permiten integrar de manera exacta funciones polinomiales de orden a lo sumo $2 N - 1$.

La principal desventaja de las cuadraturas Gaussianas es el hecho de que en estas los puntos de evaluación se encuentran especificados en términos de coordenadas en el rango fijo entre $x=-1$ y $x=+1$ lo cual obliga a que sea necesario realizar una transformación previa o cambio de variable.

Para evitar confusiones en la notación denotemos el espacio en el que se indican las coordenadas de las cuadraturas Gaussianas mediante la letra $r$, de manera que el cambio de variables se expresa como:

$$I = \int\limits_{x=a}^{x=b} f(x) \mathrm{d}x \equiv \int\limits_{r=-1}^{r=+1}F(r) \mathrm{d}r\, .$$

Nótese que el cambio de variables implica:

* Relacionar $x$ y $r$ lo que podemos escribir de forma general como $x = x(r)$ y $r = r(x)$.

* Expresar $f(x)$ en términos de la nueva variable de acuerdo con $F(r) = f[x(r)]$.

* Expresar $\mathrm{d}x$ en términos de $\mathrm{d}r$.

### Cuadratura de 2 puntos

Considere el caso de una cuadratura de 2 puntos, es decir $N =2$. En este caso los factores de ponderación y puntos de evaluación se especifican en la siguiente tabla:

| $r$                 | $w$   |
|---------------------|-------|
| $\frac{-\sqrt3}{3}$ | $1.0$ |
| $\frac{+\sqrt3}{3}$ | $1.0$ |

Para realizar el cambio de variables asumamos que la relación entre las variables independientes $x$ y $r$ es lineal de manera que:

$$x(r) = \frac{1}{2}(a + b) + \frac{r}{2}(b - a) \equiv \frac{1}{2}(a + b) + \frac{h}{2}r\, ,$$

y por lo tanto:

$$\mathrm{d}x=\frac{h}{2}\mathrm{d}r\, .$$

Esto que produce la siguiente equivalencia entre las integrales en los 2 espacios:

$$I = \int\limits_{x=a}^{x=b} f(x) \mathrm{d}x \equiv \int\limits_{r=-1}^{r=+1} f[ x(r)]\frac{h}{2} \mathrm{d}r\, .$$

Ahora, la integral formulada en el espacio de $r$ es fácilmente evaluable mediante las coordenadas y pesos de la tabla.

Consultar los factores y puntos de integración para una cuadratura Gaussiana de 4 puntos.

## Solución en Python

En los bloques de código que se presentan a continuación se implementa la cuadratura Gaussiana de 2 puntos para calcular la integral:

$$
I=\int_{x = -1}^{x = +1}(x^3+4x^2-10)\operatorname dx
$$

Adicionar comentarios a cada uno de los bloques de código que se presentan a continuación.

```python
%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym
```

En el espacio encerrado entre comillas en cada una de las siguientes subrutinas indique el significado de cada uno de los parámetros y su tipo de dato.

```python
def gpoints2():
    """Cuadratura de Gauss de 2 puntos"""
    xw = np.zeros([2])
    xp = np.zeros([2])
    xw[:] = 1.0
    xp[0] = -0.577350269189626
    xp[1] = 0.577350269189626

    return xw, xp
```

```python
def transform(a, b, r):
    """

    """
    h = b-a
    xr = (a + b)/2.0 + h*r/2.0

    return xr, h
```

```python
def myfun(x):
    """

    """
    fx = x**3 + 4*x**2 - 10

    return fx
```

Adicione comentarios al código de integración.

```python
ngpts = 2
a = -1.0
b = +1.0
integral = 0.0
xw, xp = gpoints2()
for i in range(0, ngpts):
    xr, h = transform(a, b, xp[i])
    fx = myfun(xr)
    integral = integral + fx*h/2.0*xw[i]
print(integral)
```

    -17.333333333333332

**Preguntas:**

1. Modificar el código anterior para calcular la integral con una cuadratura de 3 puntos.

2. Repetir el cálculo de la integral anterior si ahora los límites de integración son $a =0$ y $b=2$.

3. Usando la cuadratura Gaussiana calcular la siguiente integral:

$$I=\int\limits_{x=3.0}^{x=6.0} \mathrm{d}x$$

4. ¿Cómo sería la generalización de la cuadratura Gaussiana sobre un cuadrilátero?

## Glosario de términos

**Cuadratura:** Formula de integración numerica compuesta por un conjunto de puntos de evaluación y factores de ponderación.

**Punto de integración:** Punto de evaluación de la función a integrar mediante una cuadratura numérica.

**Punto de Gauss:** Punto de integración en una cuadratura Gaussiana.

**Factor de ponderación:** Constante que pondera la contribución de la función a la integral cuando esta es evaluada en un punto de integración determinado.

## Actividad para la clase

La figura muestra el problema de una cuña de semi-ángulo interno $\phi=\frac\pi4$ y lado $\ell = 10.0$ sometida a tracciones en las superficies inclinadas de magnitud $S = 1.0$.

Considerando que la relaciónes deformación-desplazamiento y tensión-deformación están dadas por:

\begin{align}
\varepsilon_{xx} &= \frac{\partial u}{\partial x}\, ,\\
\varepsilon_{yy} &= \frac{\partial v}{\partial y}\, ,\\
\varepsilon_{xy} &= \frac{1}{2}\left(\frac{\partial u}{\partial y}
                  + \frac{\partial v}{\partial x}\right)\, ,\\
\sigma_{xx} &= \frac E{1 + \nu}\varepsilon_{xx} + \frac{\nu E}{(1+\nu)(1-2\nu)}(\varepsilon_{xx} + \varepsilon_{yy})\, ,\\
\sigma_{yy} &= \frac E{1+\nu}\varepsilon_{yy} + \frac{\nu E}{(1+\nu)(1-2\nu)}(\varepsilon_{xx} + \varepsilon_{yy})\, ,\\
\sigma_{xy} &= \frac{E}{2(1 + \nu)} \varepsilon_{xy}\, ,
\end{align}

se pide:

1. Calcular la energía de deformación del sistema dada por:

$$I = \frac{1}{2}\int\limits_S (\sigma_{xx}\varepsilon_{xx} + \sigma_{yy}\varepsilon_{yy}
                                + 2\sigma_{xy}\varepsilon_{xy})\mathrm{d}S\, ,$$

asumiendo que los desplazamientos en los puntos izquierdo y derecho están dados por

$$\vec{u}_\text{izq} = -2.0 \hat{\imath}\, ,$$

y

$$\vec{u}_\text{der} = +2.0\hat{\imath}\, ,$$

mientras que los de los puntos superior e inferior corresponden a

$$\vec{u}_\text{sup} = -2.0 \hat{\jmath}\, ,$$

y

$$\vec{u}_\text{inf}=+2.0\hat{\jmath}\, .$$

2. Verifique que su resultado es correcto comparando con la solución analítica del problema.

## Formato del notebook

La siguiente celda cambia el formato del Notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>


Homework 4

Yutong Dai yutongd3@illinois.edu

## 3.6
a) According to the graph below, the optimal value for the objective function is $-\frac{11}{3}$, which is attained at $(\frac{1}{3},\frac{4}{3})$.

```python
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0,1.2,100)
fig, ax = plt.subplots()
ax.plot(x, 1 + x, 'b-', markersize=2, label="x2=1+x1")
ax.plot(x, 2-(2/3)*x, 'b-', markersize=2, label="x2=2-2/3*x1")
ax.plot(x, np.zeros_like(x)+4/3, 'b-', markersize=2, label="x2=4/3")
ax.plot(x, -(3/2)*x - (1/2) * (-11/3), "g-.", label="optimal")
plt.plot(3/5,8/5,'ro',markersize=4, label='(3/5,8/5)')
plt.plot(1/3,4/3,'ko',markersize=4, label='(1/3,4/3)')
plt.plot(1,4/3,'co',markersize=4, label='(1,4/3)')
ax.grid(True, which='both')
ax.axhline(y=0, color='k')
ax.axvline(x=0, color='k')
plt.legend()
plt.xlabel('x1')
plt.ylabel('x2')
plt.show()
```

b)

Rewrite the problem in the standard form,

$$
\begin{align}
& \max \quad -3x_1  - 2x_2\\
& s.t \quad -x_1 + x_2 + x_3 = 1\\
& \qquad 2x_1 + 3x_2 + x_4 = 6\\
& \qquad x_2 -x_5 = \frac{4}{3} \\
& \qquad x_1,...,x_5\geq 0
\end{align}
$$

Then we partition the matrix $A$ as

$$
A=
\begin{bmatrix}
-1 & 1 & 1 & || & 0 & 0 \\
2  & 3 & 0 & || & 1 & 0\\
0  & 1 & 0 & || & 0 & -1
\end{bmatrix},
$$

So $B=
\begin{bmatrix}
-1 & 1 & 1 \\
2  & 3 & 0 \\
0  & 1 & 0 \\
\end{bmatrix}$,
$N=
\begin{bmatrix}
 0 & 0 \\
 1 & 0\\
 0 & -1
\end{bmatrix}$, $C_B^T=(-3,-2,0)$, $C_N^T=(0,0)$, $b^T=(1,6,4/3)$.

So the **basic feasible solution** is $x_0^T=[(B^{-1}b)^T,0^T]=(1, 4/3, 2/3, 0, 0)$.
And the tableau form is given below.

|   | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 0 | 0 | 0 |1.5  | 2.5 |  17/3|
| $x_1$| 0| 1 | 0 | 0 |  0.5  | 1.5 |  1|
| $x_2$| 0| 0 | 1 | 0 |   0 | -1 | 4/3 |
| $x_3$| 0| 0 | 0 | 1 |  0.5  | 2.5 |  2/3|

```python
B = np.array([[-1,1,1],[2,3,0],[0,1,0]])
N = np.array([[0,0],[1,0],[0,-1]])
CB = np.array([[-3],[-2],[0]])
CN = np.array([[0],[0]])
b = np.array([[1],[6],[4/3]])
print("Current Cost:{}".format(np.dot(CB.T,np.linalg.inv(B).dot(b))))
print("Current BFS:{}".format(np.linalg.inv(B).dot(b).T))
print("Cost Reduction:{}".format(CN.T-np.dot(CB.T,np.linalg.inv(B)).dot(N)))
print("B.inv * N:\n{}".format(np.linalg.inv(B).dot(N)))
```

    Current Cost:[[-5.66666667]]
    Current BFS:[[1.         1.33333333 0.66666667]]
    Cost Reduction:[[1.5 2.5]]
    B.inv * N:
    [[ 0.5  1.5]
     [ 0.  -1. ]
     [ 0.5  2.5]]

c) Choose $x_4$ as the pivot column.
$\delta=\min\{1/0.5, (2/3)/(0.5)\}=4/3$, then the tableau becomes to

|   | $z$ | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$ | RHS |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $z$ | -1 | 0 | 0 | -3 |0  | -5 |  11/3|
| $x_1$| 0| 1 | 0 | -1 |  0  | -1 |  1/3|
| $x_2$| 0| 0 | 1 | 0 |   0 | -1 | 4/3 |
| $x_4$| 0| 0 | 0 | 2 |  1  | 5 |  4/3|

So the current solution is $(1/3, 4/3, 0, 4/3, 0)$ and the base matrix is
$B=
\begin{bmatrix}
-1 & 1 &  0  \\
2  & 3 & 1 \\
0  & 1 &  0
\end{bmatrix}$.

Since all the cost is negative, the solution is optimal for the maximization problem.

d)
In order to draw the requirements space, we need to reformulate this problem.

$$
\begin{align}
& \max \quad -3x_1  - 2x_2' - \frac{8}{3}\\
& s.t \quad -x_1 + x_2' + x_3 = -\frac{1}{3}\\
& \qquad 2x_1 + 3x_2' + x_4 = 2\\
& \qquad x_1,x_2',x_3,x_4\geq 0
\end{align}
$$

i) According to the discussion with the TA, the possible bases are defined as 2 by 2 matrix.

They are

$$
\begin{align}
& B_1 =
\begin{bmatrix}
-1 & 1 \\
2  & 3
\end{bmatrix},
B_2 =
\begin{bmatrix}
-1 & 1 \\
2  & 0
\end{bmatrix},
B_3 =
\begin{bmatrix}
-1 & 0 \\
2  & 1
\end{bmatrix}\\
& B_4 =
\begin{bmatrix}
1 & 1\\
3 & 0
\end{bmatrix},
B_5 =
\begin{bmatrix}
1 & 0\\
3 & 1
\end{bmatrix},
B_6 =
\begin{bmatrix}
1 & 0\\
0 & 1
\end{bmatrix}
\end{align}
$$

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
f, ((ax1, ax2, ax3), (ax4, ax5, ax6)) = plt.subplots(2, 3, figsize=(9,6))
ax1.plot([-1,0], [2,0])
ax1.plot([1,0], [3,0])
ax1.set_title('B1')
ax2.plot([-1,0], [2,0])
ax2.plot([1,0], [0,0])
ax2.set_title('B2')
ax3.plot([-1,0], [2,0])
ax3.plot([0,0], [1,0])
ax3.set_title('B3')

ax4.plot([1,0], [3,0])
ax4.plot([1,0], [0,0])
ax4.set_title('B4')
ax5.plot([1,0], [3,0])
ax5.plot([0,0], [1,0])
ax5.set_title('B5')
ax6.plot([1,0], [0,0])
ax6.plot([0,0], [1,0])
ax6.set_title('B6')
plt.show()

```

ii)

Possible feasible bases are given as

$$
B_1 =
\begin{bmatrix}
-1 & 1 \\
2  & 3
\end{bmatrix},
B_2 =
\begin{bmatrix}
-1 & 1 \\
2  & 0
\end{bmatrix},
B_3=
\begin{bmatrix}
-1 & 0 \\
2  & 1
\end{bmatrix}.
$$

```python
import numpy as np
A = np.array([[-1,1,1,0],[2,3,0,1]])
b = np.array([[-1/3], [2]])
for i in range(3):
    for j in range(i+1,4):
            a = A[:,[i,j]]
            #print(i,j, np.linalg.inv(a).dot(b))
```

e)

* The basic matrix $B_1$ corresponds to $(3/5, 8/5)$.
* The basic matrix $B_2$ corresponds to $(1, 4/3)$.
* The basic matrix $B_3$ correspond to $(1/3, 4/3)$.

## 3.9
By adding a slack variable $x_7$, we can change the maximization problem to the standard form. $p_1,...,p_7$ are 7 basic feasible solutions.

|   | $x_1$ | $x_2$ | $x_3$ | $x_4$ | $x_5$  | $x_6$ | $x_7$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
| $p_1$ | 20 | 0 | 0 | 0 | 0 | 0 | 0 |
| $p_2$ | 0 | 10 | 0 | 0 | 0 | 0 | 0 |
| $p_3$ | 0 | 0 | 20 | 0 | 0 | 0 | 0 |
| $p_4$ | 0 | 0 | 0 | 20 | 0 | 0 | 0 |
| $p_5$ | 0 | 0 | 0 | 0 | 20 | 0 | 0 |
| $p_6$ | 0 | 0 | 0 | 0 | 0 | 15 | 0 |
| $p_7$ | 0 | 0 | 0 | 0 | 0 | 0 | 60 |

Clearly, the optimal value is 100, which attains at the point $p_5$.

**General Methodology:**

Find $j^*\in \arg\max \{\frac{c_j}{a_j}\}$. Then the $j^*$(s) indicate(s) the optimal solution, where we set the  $j^*$-th coordinate to $b/a_j$ while keep the rest coordinates 0.

If $c_j <0 \text{ or } c_j > 0, a_j\leq 0$, then just skip calculating $\frac{c_j}{a_j}$, since those variables won't become basic variables.

## 3.13
a) The objective value will increase by $3\times 7 = 21$.

b) ~~**No**. If have some $j$ such that $c_j-z_j > 0$, then we can let $x_j$ enter the basic variables and kick out one from the basic variables, while still keep increasing the objective function value.~~

**Yes.** If for some $j$, $x_j$ is not a basic variable and the current BFS is degenerated, then  $c_j-z_j > 0$ can be possible.

c) ~~**Yes**. Suppose $x_0$ is a BFS, then $c^T(x_0+ \lambda d) \geq c^Tx_0,\forall \lambda \geq 0$. Notice that $x_0+ \lambda d$ is still in the feasible set, so the problem is unbounded.~~

**No**.If $X$ is an empty, then it is not necessarily true.

d) **No**. If all the BFS of $X$ are non-degenerated, then the statement is true.

e) **No**. When the optimal solution is degenerated, we stay at the same BFS after one iteration.

f) **False**. A point can lead to multiple different bases. So $B_1$ and $B_2$ are not necessarily adjacent.

g) **True**. Consider the case that cost $c$ is $0$, then every feasible solution is optimal.

h) The tight upper bound on the feasible bases is $m+1$ while the tight upper bound on the BFS is $m+1$.

i) **False**. Consider a convex cone defined by $n+1$ hyper-planes, then it can have $n+1$ extreme directions. For example, the figure below has more than 3 extreme directions. (Source:Google Image.)

j) ~~**True**. Since the degree of the degeneracy is $1$, we know that there are $r=n-m+1$ planes from $x\geq 0$ are binding and there are $q=m-(r-(n-m))=m-1$ basic variables are positive. Each possible choice of a basis $B$ that includes the columns of these $q$ positive variables represents $\bar x$(#). So there are $C_{n-(m-1)}^{m-(m-1)}=n-m+1$ bases associate with this $\bar x$.~~

~~Note, to see the statement (#), notice that $B \bar x=b$, so $b$ is the linear combination of columns in $B$ that corresponds to positive basic variables.~~

This is false. Above statement is flawed in *Each possible choice of a basis $B$ that includes the columns of these $q$ positive variables represents $\bar x$*. This is incorrect, as columns of these $q$ positive variables can identical to one the rest, which makes $B$ not full rank.

Consider

$$
\begin{cases}
& x_1 -x_3 +x_4 =1\\
& x_2 -x_3 = 0\\
& x_1,x_2,x_3,x_4 \geq 0
\end{cases}
$$

$\bar x = (1, 0, 0, 0)$ only corresponds to two bases.

## 3.42
We can reformulate the problem as following

$$
\begin{align}
& \min \quad z\\
& s.t \quad -2x_1 - x_2 + x_3 \leq z\\
& \qquad 2x_1 + x_2 + 4x_3 \leq 6\\
& \qquad x_1 + 4x_2 - x_3 \leq 4\\
& \qquad x_1,x_2,x_3\geq 0
\end{align}
$$

* Step 1: Eliminate $x_1$

$$
\begin{cases}
& x_1 \geq -\frac{1}{2}z -\frac{1}{2}x_2  + \frac{1}{2}x_3 \\
& x_1 \leq 3 -\frac{1}{2}x_2 -2x_3\\
& x_1 \leq 4 - 4x_2 + x_3\\
& x_1,x_2,x_3 \geq 0
\end{cases}
\Rightarrow
\begin{cases}
& 3 -\frac{1}{2}x_2 -2x_3 \geq x_1 \geq -\frac{1}{2}z -\frac{1}{2}x_2  + \frac{1}{2}x_3 \\
& 4 - 4x_2 + x_3\geq x_1 \geq -\frac{1}{2}z -\frac{1}{2}x_2  + \frac{1}{2}x_3 \\
& 3 -\frac{1}{2}x_2 -2x_3 \geq x_1 \geq 0 \\
& 4 - 4x_2 + x_3\geq x_1 \geq 0\\
& x_2,x_3 \geq 0
\end{cases}
\Rightarrow
\begin{cases}
& x_2 \leq 6 - 4x_3 \\
& x_2 \leq 1 + \frac{1}{4}x_3\\
& x_2 \leq \frac{8}{7} + \frac{1}{7}z + \frac{1}{7}x_3\\
& x_3 \leq \frac{1}{5}z + \frac{6}{5}\\
& x_2,x_3 \geq 0
\end{cases}
$$

* Step 2: Eliminate $x_2$

$$
\begin{cases}
& x_2 \leq 6 - 4x_3 \\
& x_2 \leq 1 + \frac{1}{4}x_3\\
& x_2 \leq \frac{8}{7} + \frac{1}{7}z + \frac{1}{7}x_3\\
& x_3 \leq \frac{1}{5}z + \frac{6}{5}\\
& x_2,x_3 \geq 0
\end{cases}
\Rightarrow
\begin{cases}
& 0 \leq x_2 \leq 6 - 4x_3 \\
& 0 \leq x_2 \leq 1 + \frac{1}{4}x_3\\
& 0 \leq x_2 \leq \frac{8}{7} + \frac{1}{7}z + \frac{1}{7}x_3\\
& x_3 \leq \frac{1}{5}z + \frac{6}{5}\\
& x_3 \geq 0
\end{cases}
\Rightarrow
\begin{cases}
& x_3 \leq \frac{3}{2} \\
& x_3 \geq -8 - z\\
& x_3 \leq \frac{1}{5}z + \frac{6}{5}\\
& x_3 \geq 0
\end{cases}
$$

* Step 3: Eliminate $x_3$

$$
\begin{cases}
& x_3 \leq \frac{3}{2} \\
& x_3 \geq -8 - z\\
& x_3 \leq \frac{1}{5}z + \frac{6}{5}\\
& x_3 \geq 0
\end{cases}
\Rightarrow
\begin{cases}
& \frac{1}{5}z + \frac{6}{5} \geq x_3 \geq -8 - z\\
& \frac{1}{5}z + \frac{6}{5} \geq x_3 \geq 0 \\
& \frac{3}{2} \geq x_3 \geq -8 - z\\
& \frac{3}{2}\geq x_3 \geq 0 \\
\end{cases}
\Rightarrow
z\geq -6
$$

So the optimal value for the original LP is **z=6**.

## Exercise 3.18

a) **False**. The cost of every consecutive BFS visited by the simplex is strictly less than the cost of the previous one. otherwise the algorithms is supposed to terminate. To see this,  in every iteration, the algorithm is moving along a direction $d_j$ with positive distance given by the minimal ratio test, where $j$ satisfies $(c_j-z_j)< 0, j \in N$.

b) **True**. Assume $x_r$ is leaving the basic variables and $x_k$ enters. Then previous $c_r-z_r$ changes from zero to $-\frac{c_r-z_r}{y_{k,r}}$, which is positive. So $x_r$ can no longer enter the basic variables.

c) **False**. Look at the example p.114 of Chapter 3 of the BT's book. The problem is,

$$
\begin{align}
& \min x_5 + x_6 + x_7 + x_8 \\
& s.t \quad  x_1 + 2x_2+ 3x_3 +x_5 = 3\\
& \qquad -x_1 + 2x_2 + 6x_3 + x_6= 2 \\
& \qquad 4x_2 + 9x_3+ x_7 = 5\\
& \qquad 3x_3 + x_4 + x_8 = 1\\
& \qquad x_1,...,x_8 \geq 0
\end{align}
$$

Let's begin with $(0, 0, 0, 0, 3, 2, 5, 1)$.

* In the first iteration, $x_4$ enteres and the BFS is $(0, 0, 0, 1, 3, 2, 5, 0 )$.
* In the very next iteration, $x_4$ leaves and the BFS is $(0, 0, 1/3, 0, 2, 0, 2, 0)$.

d) **False**. When the optimal solution is not unique, the statement is false. Consider the following problem

$$
\begin{align}
& \min x_1\\
& s.t \quad  x_2 - x_3 = 1\\
& \qquad x_2 + x_4 = 2\\
& \qquad x_1,x_2,x_3,x_4\geq 0
\end{align}
$$

Consider
$B_1 =
\begin{bmatrix}
1 & -1 \\
1  & 0
\end{bmatrix}$, the BFS is $(0, 2, 1, 0)$ and non-degenerated. Also, consider
$B_2 =
\begin{bmatrix}
1 & 0 \\
1  & 1
\end{bmatrix}$, the BFS is $(0, 1, 0, 1)$ and non-degenerated.

e) ~~**False**. Consider the cost $c=0$, then any feasible solution is optimal.~~

In general this is true. But for simplex method, it can only explore the BFS, hence at most $m$ positive components.

## Exercise 3.19
Since the problem only asks for some parameters values, so the answer below by no means will be comprehensive. They are just some feasible choices.

a) In order to guarantee the optimality, $\beta=0$. To make the case simple, we can also assume $\delta>0$ and $\delta+\frac{2}{3}\gamma=0$ to guarantee the optimality of the BFS by a iteration of the simplex method. In sum, a possible solution would be

$$
(\alpha, \beta, \gamma, \delta, \eta) = (1, 0, -3, 2, 1).
$$

b) For the LP to be unbounded, we need to set $\delta,\alpha,\gamma$ to be negative. A possible combination would be

$$
(\alpha, \beta, \gamma, \delta, \eta) = (-1, 1, -1, -1, 1).
$$

c) As long we can move current BFS by a positive distance, we are done. This requires $\beta, \delta>0$.A possible combination would be

$$
(\alpha, \beta, \gamma, \delta, \eta) = (1, 3, -3, 3, 2).
$$

## Exercise 3.20

a) $\beta \geq 0$, $\alpha,\gamma,\delta,\eta,\zeta\in R$.

b) $\beta < 0$, $\alpha\geq 0$, $\gamma,\delta,\eta,\zeta\in R$. In this case, no direction will increase $x_2$ to a non-negative number.

c) We need $\beta \geq 0$ to guarantee the feasibility. Then as long as at least one of the $\gamma,\delta,\zeta\in R$ is negative, then the current basis is not optimal. Also, set $\eta \in R$.

d) We need $\beta \geq 0$ to guarantee the feasibility. To make the optimal is unbounded, we need to make the one $y_j\leq 0$ with only one iteration.

After trying different pivots, I discover the following ranges work. $\alpha <0,\gamma<0$, $\beta,  \delta,\zeta \geq 0$, $\eta > \frac{4}{3}$, and $2\frac{\gamma}{\eta} + \delta <0$.

The last constraints guarantee the negative reduced cost for $x_4$ after one iteration of the simplex method.

e) To make the statement hold, $\gamma <0$, $\eta > 0$ and $\frac{2}{\eta}<\frac{3}{2}$. In sum, $\beta\geq 0, \gamma < 0, \eta > \frac{4}{3}$, $\alpha, \delta,\zeta \in R$.

f) This happens when there is a degeneracy, that is to say, $\beta=0$. In sum, $\beta=0, \zeta < 0$, $\alpha, \gamma, \eta, \delta \in R$.

## Exercise 3.27

(a) Notice that, if $x\geq 0$ and $x\neq 0$, there exist a $\alpha>0$, such that $\alpha x=y + z$. To see this, define $\alpha=\frac{1}{\min_i \{x_i|x_i>0\}}$ and $y^T=(I(x_1>0), ..., I(x_n>0))$, where $I(.)$ is a indicator function.
Then $\alpha x=y + z$, where $z\geq 0$. Since $Ax=0$, so we have $A(y+z)=0$. To conclude, the problem can be formulated as

$$
\begin{align}
& \max\qquad 1_n^T y\\
& s.t.\quad A(y+z)=0\\
& \qquad y_i \leq 1, i=1,2,...,n\\
& \qquad y, z \geq 0
\end{align}
$$

(b)

$$
\begin{align}
& \max\qquad 1_n^T y\\
& s.t.\quad A(y+z)=\alpha b\\
& \qquad y_i \leq 1, i=1,2,...,n\\
& \qquad y, z \geq 0 \\
& \qquad\alpha \geq 1
\end{align}
$$

**Remark**: The reason I set $\alpha\geq 1$ instead of using $\alpha\geq 0$, is only to scale the coordinates of $x$, which are smaller than 1 but greater than 0.

## Exercise 3.31

a) As shown below, $(b_1,b_2)$ is supposed to be enclosed by the four boundaries such that $P(b_1,b_2)$ is non-empty.

```python
import matplotlib.pyplot as plt
import numpy as np
fig, ax = plt.subplots()
x, y = np.array([2,3,1,1]), np.array([1,2,1,3])
for i in range(4):
    ax.plot(x[i],y[i], 'bo', markersize=5)
    ax.annotate("A{}".format(i+1), (x[i]+0.01,y[i]+0.01))
ax.plot(x[[3, 2]],y[[3, 2]], 'r-', markersize=3)
ax.plot(x[[2, 0]],y[[2, 0]], 'r-', markersize=3)
ax.plot(x[[0, 1]],y[[0, 1]], 'r-', markersize=3)
ax.plot(x[[1, 3]],y[[1, 3]], 'r-', markersize=3)
plt.show()
```

b) To have degeneracy, we must set at least one basic variable to zero. So the  $(b_1,b_2)$ has to lie on the boundary of the set, namely on the four segments, whose endpoints are ($A_i,A_j$) respectively.

c)
Denote $\hat A_1=(2,1,1)^T,\hat A_2=(3,2,1)^T,\hat A_3=(1,1,1)^T,\hat A_4=(1,3,1)^T$ and $B_1=(\hat A_1, \hat A_2,\hat A_3)$, $B_2=(\hat A_1, \hat A_2,\hat A_4)$, $B_3=(\hat A_1, \hat A_3,\hat A_4)$, $B_4=(\hat A_2, \hat A_3,\hat A_4)$.

* If $(b_1,b_2)$ lies on ($A_3,A_1$), then the bases are $B_1, B_3$.

* If $(b_1,b_2)$ lies on ($A_1,A_2$), then the bases are $B_1, B_2$.

* If $(b_1,b_2)$ lies on ($A_2,A_4$), then the bases are $B_2, B_4$.

* If $(b_1,b_2)$ lies on ($A_4,A_3$), then the bases are $B_3, B_4$.

d)

* See below for $S_1 = \{(b_1,b_2) | B_1 \text{ is optimal }\}$

* See below for $S_2 = \{(b_1,b_2) | B_2 \text{ is optimal }\}$

* See below for $S_3 = \{(b_1,b_2) | B_3 \text{ is optimal }\}$

* See below for $S_4 = \{(b_1,b_2) | B_4 \text{ is optimal }\} = \emptyset$

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
f, ((ax0, ax1), (ax2, ax3)) = plt.subplots(2, 2, figsize=(6,6))
ax0.plot([2,3],[1,2], 'r-', markersize=3)
ax0.plot([2,9/5],[1, 7/5], 'r-', markersize=3)
ax0.plot([9/5, 3],[7/5, 2], 'r-', markersize=3)
ax0.set_title('S_1')

ax1.plot([1,3],[3,2], 'r-', markersize=3)
ax1.plot([3,9/5],[2, 7/5], 'r-', markersize=3)
ax1.plot([9/5, 1],[7/5, 3], 'r-', markersize=3)
ax1.set_title('S_2')

ax2.plot(x[[3, 2]],y[[3, 2]], 'r-', markersize=3)
ax2.plot([1,9/5],[1, 7/5], 'r-', markersize=3)
ax2.plot([9/5, 1],[7/5, 3], 'r-', markersize=3)
ax2.set_title('S_3')

ax3.set_title('S_4')
plt.show()
```

e) According to the second graph below, if $(b_1, b_2)=(2,1)$, then BFS is set to $(1,0,0,0)$, we got the optimal.

f) Based on the first figure below, we know that $(b_1, b_2)$ can be present by any three columns of $A$.

At first iteration, we pivot on $x_2$, then we end up with the basic variables $x_1,x_3,x_4$. From the second figure below, we know it's the optimal solution.

```python
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits import mplot3d
fig, ax = plt.subplots()
x, y = np.array([2,3,1,1]), np.array([1,2,1,3])
for i in range(4):
    ax.plot(x[i],y[i], 'bo', markersize=5)
    ax.annotate("A{}".format(i+1), (x[i]+0.01,y[i]+0.01))
ax.plot(x[[3, 2]],y[[3, 2]], 'r-', markersize=3)
ax.plot(x[[2, 0]],y[[2, 0]], 'r-', markersize=3)
ax.plot(x[[0, 1]],y[[0, 1]], 'r-', markersize=3)
ax.plot(x[[1, 3]],y[[1, 3]], 'r-', markersize=3)
ax.plot(x[[1, 3]],y[[1, 2]], 'r-.', markersize=3)
ax.plot(x[[0, 3]],y[[0, 3]], 'r-.', markersize=3)
ax.plot(9/5, 7/5, 'ko', markersize=5)
plt.show()
```

```python
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits import mplot3d
fig = plt.figure()
ax = plt.axes(projection='3d')
x, y, z = np.array([2,3,1,1]), np.array([1,2,1,3]), np.array([1,3,2,2])
z0 = np.array([0, 0, 0, 0])
ax.scatter3D(x, y, z, c="blue", s=30)
for i in range(4):
    #ax.plot([x[i],x[i]],[y[i], y[i]],[z[i],z0[i]], "r-")
    for j in range(i+1, 4):
        ax.plot(x[[i,j]], y[[i,j]], z[[i,j]],"k-")
ax.plot([9/5, 9/5], [7/5, 7/5], [0, 3],"r-")
ax.set_xlim(0,3), ax.set_ylim(0,3), ax.set_zlim(0,3)
ax.set_xlabel('X'), ax.set_ylabel('Y'), ax.set_zlabel('Z')
plt.show()
```

```python
# c = [1, 3, 2, 2]
# A = [[2,3,1,1], [1, 2,1,3], [1,1,1,1]]
# b = [9/5, 7/5, 1]
# x1_bounds = (0, None)
# x2_bounds = (0, None)
# x3_bounds = (0, None)
# x4_bounds = (0, None)
# from scipy.optimize import linprog
# res = linprog(c, A_eq=A, b_eq=b, bounds=(x1_bounds, x2_bounds,x3_bounds,x4_bounds), options={"disp": True})
```

    Optimization terminated successfully.
             Current function value: 1.200000
             Iterations: 5

```python
import numpy as np
A = np.array([[2,3,1,1], [1, 2,1,3], [1,1,1,1]])
B = A [:,[0,2,3]]
```

```python
np.linalg.inv(B)
```

    array([[ 1. ,  0. , -1. ],
           [-1. , -0.5,  2.5],
           [-0. ,  0.5, -0.5]])

```python

```


# メモ

これだけ！線形代数
石井俊全 いしいとしあき 秀和システム

を読む。

```
# 連立一次方程式
%%latex
\displaystyle
\begin{array}{rcl}
    y &=& 4 x - 3 \\
    2 x + 3 y - 1 &=& 0
\end{array}
```

\displaystyle
\begin{array}{rcl}
y &=& 4 x - 3 \\
2 x + 3 y - 1 &=& 0
\end{array}

```
# 行列
%%latex
\displaystyle
\begin{pmatrix}
    3 & 4 \\
    1 & 2
\end{pmatrix}
~とか~
\begin{pmatrix}
    1 & 2 & 7 \\
    3 & 5 & 6
\end{pmatrix}
```

\displaystyle
\begin{pmatrix}
    3 & 4 \\
    1 & 2
\end{pmatrix}
~とか~
\begin{pmatrix}
    1 & 2 & 7 \\
    3 & 5 & 6
\end{pmatrix}

```
# 行列式
%%latex
\displaystyle
ちなみに~
\begin{pmatrix}
    3 & 4 \\
    1 & 2
\end{pmatrix}
~の行列式は~2~になる。
```

\displaystyle
ちなみに~
\begin{pmatrix}
    3 & 4 \\
    1 & 2
\end{pmatrix}
~の行列式は~2~になる。

```
# sympy によるベクトル表示
from sympy import *
init_printing()
a = MatrixSymbol('a',2,1)
display (a)
a = ([[3],[-1]])
display (Matrix(a))
```

```
# ベクトルの内積
%%latex
\bm{a}\cdot\bm{b}=|\bm{a}||\bm{b}| \cos \theta
```

\bm{a}\cdot\bm{b}=|\bm{a}||\bm{b}| \cos \theta

```
# 3元連立1次方程式 p.89
%%latex
\displaystyle
\left \{
\begin{array}{rrrr}
    3 x + & y+& 2z=& 9 \\
    x - & 2 y + & + z =& 8\\
    -2x-& y+& z=& -3
\end{array}
\right . \\
を解く。
```

\displaystyle
\left \{
\begin{array}{rrrr}
    3 x + & y+& 2z=& 9 \\
    x - & 2 y + & + z =& 8\\
    -2x-& y+& z=& -3
\end{array}
\right . \\
を解く。

```
from sympy import *
init_printing()
A = Matrix([[3,1,2],[1,-2,1],[-2,-1,1]])
x,y,z = symbols('x,y,z')
p = Matrix([[x],[y],[z]])
b = Matrix([[9],[8],[-3]])
display(Eq(A*p, b))
print()
display(solve(A*p-b,x,y,z))
```


# Tarea Nro. 3 - LinAlg + Sympy

- Nombre y apellido: Renato Balcázar
- Fecha:             01-12-2020

### Producto punto

1. Use un producto punto y la lista de compras de la tabla 9.4 para determinar su cuenta total en la tienda.

```python
import numpy as np

products = np.array([2,1,2,5,1])
prices = np.array([3.5, 1.25, 4.25, 1.55, 3.15])

result = products.dot(prices)
result

```

    27.65

### Multiplicación matricial

2. Con un calorímetro de bomba se realizó una serie de experimentos. En cada experimento se usó una cantidad diferente de agua. Calcule la capacidad calorífica total para el calorímetro en cada uno de los experimentos, mediante multiplicación matricial, los datos de la tabla 9.8 y la información acerca de la capacidad calorífica que sigue a la tabla.

```python
propierties = np.array([
    [ 110, 250, 10],
    [ 100, 250, 10],
    [ 101, 250, 10],
    [98.6, 250, 10],
    [99.4, 250, 10]
])

capacity_elements = np.array([
    [0.45],
    [4.20],
    [0.90]
])

res = np.dot(propierties, capacity_elements)
res

```

    array([[1108.5 ],
           [1104.  ],
           [1104.45],
           [1103.37],
           [1103.73]])

### Determinantes e inversos

3. Recuerde que no todas las matrices tienen inverso. Una matriz es singular (es decir: no tiene inverso) si su determinante es igual a 0 (es decir, |A| = 0). Use la función determinante para probar si cada una de las siguientes matrices tiene inverso:

Si existe un inverso, calcúlelo.

```python
A = np.array([
    [2, -1],
    [4,  5]
])

B = np.array([
    [4, 2],
    [2, 1]
])

C = np.array([
    [ 2,  0,  0],
    [ 1,  2,  2],
    [ 5, -4,  0]
])

if (np.linalg.det(A) != 0):
    print("\nInversa de A: ")
    print(np.linalg.inv(A))
else:
    print("\nA no posee una inversa porque su determinante es 0")

if (np.linalg.det(B) != 0):
    print("\nInversa de B: ")
    print(np.linalg.inv(B))
else:
    print("\nB no posee una inversa porque su determinante es 0")

if (np.linalg.det(C) != 0):
    print("\nInversa de C: ")
    print(np.linalg.inv(C))
else:
    print("\nC no posee una inversa porque su determinante es 0")

```

    Inversa de A:
    [[ 0.35714286  0.07142857]
     [-0.28571429  0.14285714]]

    B no posee una inversa porque su determinante es 0

    Inversa de C:
    [[ 0.5    0.     0.   ]
     [ 0.625  0.    -0.25 ]
     [-0.875  0.5    0.25 ]]

### Resolución de sistemas ecuaciones lineales

4. Resuelva el siguiente sistema de ecuaciones

```python
coef = np.array([
    [ 3,  4,  2, -1,  1,  7,  1],
    [ 2, -2,  3, -4,  5,  2,  8],
    [ 1,  2,  3,  1,  2,  4,  6],
    [ 5, 10,  4,  3,  9, -2,  1],
    [ 3,  2, -2, -4, -5, -6,  7],
    [-2,  9,  1,  3, -3,  5,  1],
    [ 1, -2, -8,  4,  2,  4,  5]
])
results = np.array([
    42,
    32,
    12,
    -5,
    10,
    18,
    17
])

respuesta = np.linalg.solve(coef, results)
respuesta

```

    array([-0.18899493,  2.54589061, -3.28057396, -6.75778176,  1.32124449,
            4.31944831,  0.62940585])

### Cálculo

5. La capacidad calorífica Cp de un gas se puede modelar con la ecuación empírica

$$C_p = a + bT + cT^2+dT^3$$

donde a, b, c y d son constantes empíricas y T es la temperatura en grados Kelvin. El cambio en entalpía (una medida de energía) conforme el gas se caliente de T1 a T2  es la integral de esta ecuación con respecto a T:

$$\bigtriangleup h = \int_{T_{1}}^{T_{2}} C_{p} \,dT  $$

Encuentre el cambio en entalpía del oxígeno gaseoso conforme se calienta de 300 K a 1000 K. Los valores de a, b, c y d para el oxígeno son

$$
\begin{equation}
\begin{split}
a & = 25.48 \\
b & = 1.520 x 10^{-2} \\
c & = -0.7155 x 10^{-5} \\
d & = 1.312 x 10^{-9}
\end{split}
\end{equation}
$$

```python
from sympy import *
t = Symbol('t')

t_1 = 300
t_2 = 1000

a = 25.48
b = 1.520*(10**(-2))
c = -0.7155*(10**(-5))
d = 1.312*(10**(-9))

capacity = a + b*t + c*(t**2) + d*(t**3)

h_variation = integrate(capacity, (t, t_1, t_2))
h_variation

```

$\displaystyle 22756.7382$


### Sympy, General

```python
import sympy as sp

sp.sqrt(8)
```

$\displaystyle 2 \sqrt{2}$

```python
r1 = sp.Rational(4, 5)
r2 = sp.Rational(5, 4)
print(r1 + r2)
print(r1 / r2)
```

    41/20
    16/25

```python
x, y = sp.symbols("x y")
e1 = y + 2 * x
e2 = 2 * x + y
print(e1)
print(e2)
print(e1 - x)
print(e2 + 1)
print(x * e1)
print(sp.expand(x * e1))
print(sp.factor(x * e1))
```

    2*x + y
    2*x + y
    x + y
    2*x + y + 1
    x*(2*x + y)
    2*x**2 + x*y
    x*(2*x + y)

### Derivatives

```python
import sympy as sp
import numpy as np
from scipy.misc import derivative

x = sp.Symbol('x')
print(sp.diff(3*x**2+1, x))

def f(x):
    return 3*x**2 + 1

def d(x):
    return derivative(f, x)

print(derivative(f, 2.0))

%matplotlib inline
import matplotlib.pyplot as plt

y = np.linspace(-3, 3)
print(y)
plt.plot(y, f(y))
plt.plot(y, d(y))
```

### Integrals

```python
print(sp.integrate(3.0*x**2 + 1, x))

from scipy.integrate import quad
def f(x):
    return 3.0 * x**2 + 1

i = quad(f, 0, 2)
print(i[0])
```

    1.0*x**3 + 1.0*x
    10.000000000000002

```python

```


```python
%matplotlib inline
```

```python
import numpy as np
import numpy.polynomial.polynomial as p
import matplotlib.pyplot as plt

from turtle import *
from itertools import groupby
import re
from Crypto.Util import number
from sympy.ntheory import factorint
import time
```

# Basic Algebra Exercise
## Functions, Polynomials, Complex Numbers. Applications of Abstract Algebra

### Problem 1. Polynomial Interpolation
We know that if we have a set of $n$ data points with coordinates $(x_1; y_1), (x_2; y_2), \dots, (x_n; y_n)$, we can try to figure out what function may have generated these points.

Please note that **our assumptions about the data** will lead us to choosing one function over another. This means that our results are as good as our data and assumptions. Therefore, it's extremely important that we write down our assumptions (which sometimes can be difficult as we sometimes don't realize we're making them). It will be better for our readers if they know what those assumptions and models are.

In this case, we'll state two assumptions:
1. The points in our dataset are generated by a polynomial function
2. The points are very precise, there is absolutely no error in them. This means that the function should pass **through every point**

This method is called *polynomial interpolation* (*"polynomial"* captures assumption 1 and *"interpolation"* captures assumption 2).

It can be proved (look at [Wikipedia](https://en.wikipedia.org/wiki/Polynomial_interpolation) for example) that if we have $n$ data points, there is only one polynomial of degree $n-1$ which passes through them. In "math speak": "the vector spaces of $n$ points and polynomials of degree $n-1$ are isomorphic (there exists a bijection mapping one to the other)".

There are a lot of ways to do interpolation. We can also write the function ourselves if we want but this requires quite a lot more knowledge than we already covered in this course. So we'll use a function which does this for us. `numpy.polyfit()` is one such function. It accepts three main parameters (there are others as well, but they are optional): a list of $x$ coordinates, a list of $y$ coordinates, and a polynomial degree.

Let's say we have these points:
```python
points = np.array([(0, 0), (1, 0.8), (2, 0.9), (3, 0.1), (4, -0.8), (5, -1.0)])
```

First, we need to "extract" the coordinates:
```python
x = points[:, 0]
y = points[:, 1]
```

Then, we need to calculate the interpolating polynomial. For the degree, we'll set $n-1$:
```python
coefficients = np.polyfit(x, y, len(points) - 1)
poly = np.poly1d(coefficients)
```

After that, we need to plot the function. To do this, we'll create a range of $x$ values and evaluate the polynomial at each value:
```python
plot_x = np.linspace(np.min(x), np.max(x), 1000)
plot_y = poly(plot_x)
```

Finally, we need to plot the result. We'll plot both the fitting polynomial curve (using `plt.plot()`) and the points (using `plt.scatter`). It's also nice to have different colors to make the line stand out from the points.
```python
plt.plot(plot_x, plot_y, c = "green")
plt.scatter(x, y)
plt.xlabel("x")
plt.ylabel("y")
plt.show()
```
Don't forget to label the axes!

Your task now is to wrap the code in a function. It should accept a list of points, the polynomial degree, min and max value of $x$ used for plotting. We'll use this function to try some other cases.

```python
def interpolate_polynomial(points, degree, min_x, max_x):
    """
    Interpolates a polynomial of the specified degree through the given points and plots it
    points - a list of points (x, y) to plot
    degree - the polynomial degree
    min_x, max_x - range of x values used to plot the interpolating polynomial
    """
    x = points[:, 0]
    y = points[:, 1]

    coefficients = np.polyfit(x, y, degree)
    poly = np.poly1d(coefficients)

    plot_x = np.linspace(min_x, max_x, 1000)
    plot_y = poly(plot_x)

    plt.plot(plot_x, plot_y, c = "green")
    plt.scatter(x, y)
    plt.xlabel("x")
    plt.ylabel("y")
    plt.title("Degree=" + str(degree))
    plt.show()
```

```python
points = np.array([(0, 0), (1, 0.8), (2, 0.9), (3, 0.1), (4, -0.8), (5, -1.0)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

We see this is a very nice fit. This is expected, of course. Let's try to expand our view a little. Let's try to plot other values of $x$, further than the original ones. This is **extrapolation**.

```python
interpolate_polynomial(points, len(points) - 1, -5, 10)
```

Hmmm... it seems our polynomial goes a little wild outside the original range. This is to show how **extrapolation can be quite dangerous**.

Let's try a lower polynomial degree now. We used 4, how about 3, 2 and 1?
**Note:** We can add titles to every plot so that we know what exactly we're doing. The title may be passed as an additional parameter to our function.

```python
interpolate_polynomial(points, 3, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 2, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

We see the fitting curves (or line in the last case) struggle more and more and they don't pass through every point. This breaks our assumptions but it can be very useful.

Okay, one more thing. How about increasing the degree? Let's try 5, 7 and 10. Python might complain a little, just ignore it, everything is fine... sort of :).

```python
interpolate_polynomial(points, 5, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 7, np.min(points[:, 0]), np.max(points[:, 0]))
interpolate_polynomial(points, 10, np.min(points[:, 0]), np.max(points[:, 0]))
```

Those graphs look pretty much the same. But that's the point exactly. I'm being quite sneaky here. Let's try to expand our view once again and see what our results really look like.

```python
interpolate_polynomial(points, 5, -10, 10)
interpolate_polynomial(points, 7, -10, 10)
interpolate_polynomial(points, 10, -10, 10)
```

Now we see there are very wild differences. Even though the first two plots look quite similar, look at the $y$ values - they're quite different.

So, these are the dangers of interpolation. Use a too high degree, and you get "the polynomial wiggle". These are all meant to represent **the same** data points but they look insanely different. Here's one more comparison.

```python
interpolate_polynomial(points, len(points) - 1, -2, 7)
interpolate_polynomial(points, len(points) + 1, -2, 7)
```

Now we can see what big difference even a small change in degree can make. This is why we have to choose our interpolating functions very carefully. Generally, a lower degree means a simpler function, which is to be preferred. See [Occam's razor](https://en.wikipedia.org/wiki/Occam%27s_razor).

And also, **we need to be very careful about our assumptions**.

```python
points = np.array([(-5, 0.03846), (-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882), (5, 0.03846)])
interpolate_polynomial(points, len(points) - 1, np.min(points[:, 0]), np.max(points[:, 0]))
```

This one definitely looks strange. Even stranger, if we remove the outermost points... ($x = \pm 5$), we get this

```python
points = np.array([(-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882)])
interpolate_polynomial(points, len(points - 1), np.min(points[:, 0]), np.max(points[:, 0]))
```

This is because the generating function is not a polynomial. It's actually:
$$ y = \frac{1}{1 + x^2} $$

Plot the polynomial interpolation and the real generating function **on the same plot**. You may need to modify the original plotting function or just copy its contents.

```python
def plot_function(points, min_x, max_x):
        x = points[:, 0]
        y = points[:, 1]

        plot_x = np.linspace(min_x, max_x, 1000)
        plot_y = [1 / (1 + x**2) for x in plot_x]

        plt.plot(plot_x, plot_y, c = "green")
        plt.scatter(x, y)
        plt.xlabel("x")
        plt.ylabel("y")
        plt.show()
```

```python
points = np.array([(-5, 0.03846), (-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882), (5, 0.03846)])
plot_function(points, np.min(points[:, 0]), np.max(points[:, 0]))
```

```python
points = np.array([(-4, 0.05882), (-3, 0.1), (-2, 0.2), (-1, 0.5), (0, 1), (1, 0.5), (2, 0.2), (3, 0.1), (4, 0.05882)])
plot_function(points, np.min(points[:, 0]), np.max(points[:, 0]))
```

### Problem 2. Complex Numbers as Vectors
We saw that a complex number $z = a + bi$ is equivalent to (and therefore can be represented as) the ordered tuple $(a; b)$, which can be plotted in a 2D space. So, complex numbers and 2D points are equivalent. What is more, we can draw a vector from the origin of the coordinate plane to our point. This is called a point's **radius-vector**.

Let's try plotting complex numbers as radius vectors. Don't forget to label the real and imaginary axes. Also, move the axes to the origin. Hint: These are called "spines"; you'll need to move 2 of them to the origin and remove the other 2 completely. Hint 2: You already did this in the previous lab.

We can use `plt.quiver()` to plot the vector. It can behave a bit strangely, so we'll need to set the scale of the vectors to be the same as the scale on the graph axes:
```python
plt.quiver(0, 0, z.real, z.imag, angles = "xy", scale_units = "xy", scale = 1)
```

Other than that, the main parameters are: $x_{begin}$, $y_{begin}$, $x_{length}$, $y_{length}$ in that order.

Now, set the aspect ratio of the axes to be equal. Also, add grid lines. Set the axis numbers (called ticks) to be something like `range(-3, 4)` for now.
```python
plt.xticks(range(-3, 4))
plt.yticks(range(-3, 4))

```

If you wish to, you can be a bit more clever with the tick marks. Find the minimal and maximal $x$ and $y$ values and set the ticks according to them. It's a good practice not to jam the plot too much, so leave a little bit of space. That is, if the actual x-range is $[-2; 2]$, set the plotting to be $[-2.5; 2.5]$ for example. Otherwise, the vector heads (arrows) will be "jammed" into a corner or side of the plot.

```python
def plot_complex_number(z):
    """
    Plots the complex number z as a radius vector in the 2D space
    """
    ax = plt.gca()
    ax.spines["bottom"].set_position("zero")
    ax.spines["left"].set_position("zero")
    ax.spines["top"].set_visible(False)
    ax.spines["right"].set_visible(False)

    plt.quiver(0, 0, z.real, z.imag, angles = "xy", scale_units = "xy", scale = 1)
    plt.xlabel("Re")
    plt.ylabel("Im")

    plt.xticks(range(-3, 4))
    plt.yticks(range(-3, 4))

    plt.show()

plot_complex_number(2 + 3j)
```

How about many numbers? We'll need to get a little bit more creative. First, we need to create a 2D array, each element of which will be a 4-element array: `[0, 0, z.real, z.imag]`. Next, `plt.quiver()` can accept a range of values. Look at [this StackOverflow post](https://stackoverflow.com/questions/12265234/how-to-plot-2d-math-vectors-with-matplotlib) for details and adapt your code.

```python
def plot_complex_numbers(numbers, colors):
    """
    Plots the given complex numbers as radius vectors in the 2D space
    """
    ax = plt.gca()
    ax.spines["bottom"].set_position("zero")
    ax.spines["left"].set_position("zero")
    ax.spines["top"].set_visible(False)
    ax.spines["right"].set_visible(False)

    nums_twod = np.array([[z.real, z.imag] for z in numbers])

    U, V = zip(*nums_twod)

    plt.quiver(0, 0, U, V, angles = "xy", scale_units = "xy", scale = 1, color = colors)

    plt.xlabel("Re")
    plt.ylabel("Im")

    plt.xticks(range(-4, 5))
    plt.yticks(range(-4, 5))

    plt.show()

plot_complex_numbers([2 + 3j, -2 - 1j, -3, 2j], ["green", "red", "blue", "orange"])
```

Now let's see what the operations look like. Let's add two numbers and plot the result.

```python
z1 = 2 + 3j
z2 = 1 - 1j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

We can see that adding the complex numbers is equivalent to adding vectors (remember the "parallelogram rule"). As special cases, let's try adding pure real and pure imaginary numbers:

```python
z1 = 2 + 3j
z2 = 2 + 0j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

```python
z1 = 2 + 3j
z2 = 0 + 2j
plot_complex_numbers([z1, z2, z1 + z2], ["red", "blue", "green"])
```

How about multiplication? First we know that multiplying by 1 gives us the same vector and mulpiplying by -1 gives us the reversed version of the same vector. How about multiplication by $\pm i$?

```python
z = 2 + 3j
plot_complex_numbers([z, z * 1], ["red", "blue"])
plot_complex_numbers([z, z * -1], ["red", "blue"])
plot_complex_numbers([z, z * 1j], ["red", "blue"])
plot_complex_numbers([z, z * -1j], ["red", "blue"])
```

So, multiplication by $i$ is equivalent to 90-degree rotation. We can actually see the following equivalence relationships between multiplying numbers and rotation about the origin:

| Real | Imaginary | Result rotation |
|------|-----------|-----------------|
| 1    | 0         | $0^\circ$       |
| 0    | 1         | $90^\circ$      |
| -1   | 0         | $180^\circ$     |
| 0    | -1        | $270^\circ$     |

Once again, we see the power of abstraction and algebra in practice. We know that complex numbers and 2D vectors are equivalent. Now we see something more: addition and multiplication are equivalent to translation (movement) and rotation!

Let's test the multiplication some more. We can see the resulting vector is the sum of the original vectors, but *scaled and rotated*:

```python
z1 = 2 + 3j
z2 = 1 - 2j
plot_complex_numbers([z1, z2, z1 * z2], ["red", "blue", "green"])
```

### Problem 3. Recursion and Fractals

> "To understand recursion, you first need to understand recursion."

There are three main parts to a recursive function:
1. Bottom - when the recursion should finish
2. Operation - some meaningful thing to do
3. Recursive call - calling the same function
4. Clean-up - returning all data to its previous state (this reverses the effect of the operation)

Let's do one of the most famous recursion examples. And I'm not talking about Fibonacci here. Let's draw a tree using recursive functions.

The figure we're going to draw is called a **fractal**. It's self-similar, which means that if you zoom in on a part of it, it will look the same. You can see fractals everywhere in nature, with broccoli being one of the prime examples. Have a look:

First, we need to specify the recursive part. In order to draw a tree, we need to draw a line of a given length (which will be the current branch), and then draw two more lines to the left and right. By "left" and "right", we should mean "rotation by a specified angle".

So, this is how to draw a branch: draw a line and prepare to draw two more branches to the left and right. This is going to be our recursive call.

To make things prettier, more natural-looking (and have a natural end to our recursion), let's draw each "sub-branch" a little shorter. If the branch becomes too short, it won't have "child branches". This will be the bottom of our recursion.

There's one more important part of recursion, and this is **"the clean-up"**. After we did something in the recursive calls, it's very important to return the state of everything as it was **before** we did anything. In this case, after we draw a branch, we go back to our starting position.

Let's first import the most import-ant (no pun intended...) Python drawing library: `turtle`! In order to make things easier, we'll import all methods directly.
```python
from turtle import *
```

You can look up the docs about turtle if you're more interested. The basic things we're going to use are going forward and backward by a specified number of pixels and turning left and right by a specified angle (in degrees).

Let's now define our recursive function:
```python
def draw_branch(branch_length, angle):
    if branch_length > 5:
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle)
        left(2 * angle)
        draw_branch(branch_length - 15, angle)
        right(angle)
        backward(branch_length)
```

And let's call it:
```python
draw_branch(100, 20)
```

We need to start the tree not at the middle, but toward the bottom of the screen, so we need to make a few more adjustments. We can wrap the setup in another function and call it. Let's start one trunk length below the center (the trunk length is the length of the longest line).

```python
def draw_tree(trunk_length, angle):
    speed("fastest")
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle)
```

Note that the graphics will show in a separate window. Also note that sometimes you might get bugs. If you do, go to Kernel > Restart.

```python
def draw_branch(branch_length, angle):
    if branch_length > 5:
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle)
        left(2 * angle)
        draw_branch(branch_length - 15, angle)
        right(angle)
        backward(branch_length)
```

```python
def draw_tree(trunk_length, angle):
    speed("fastest")
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle)
```

Experiment with different lengths and angles. Especially interesting angles are $30^\circ$, $45^\circ$, $60^\circ$ and $90^\circ$.

```python
draw_tree(100, 30)
```

```python
draw_tree(100, 45)
```

```python
draw_tree(100, 90)
```

Now modify the original function a little. Draw the lines with different thickness. Provide the trunk thickness at the initial call. Similar to how branches go shorter, they should also go thinner.

```python
def draw_branch(branch_length, angle, thickness):
    if branch_length > 5:
        width(thickness)
        forward(branch_length)
        right(angle)
        draw_branch(branch_length - 15, angle, abs(thickness - 1))
        left(2 * angle)
        draw_branch(branch_length - 15, angle, abs(thickness - 1))
        right(angle)
        backward(branch_length)
```

```python
def draw_tree(trunk_length, angle, thickness):
    speed("fastest")
    left(90)
    up()
    backward(trunk_length)
    down()
    draw_branch(trunk_length, angle, thickness)
```

```python
draw_tree(100, 20, 5)
```

#### * Optional problem
Try to draw another kind of fractal graphic using recursion and the `turtle` library. Two very popular examples are the "Koch snowflake" and the "Sierpinski triangle". You can also modify the original tree algorithm to create more natural-looking trees. You can, for example, play with angles, number of branches, lengths, and widths. The Internet has a lot of ideas about this :). Hint: Look up **"L-systems"**.

### Problem 4. Run-length Encoding
One application of algebra and basic math can be **compression**. This is a way to save data in less space than it originally takes. The most basic form of compression is called [run-length encoding](https://en.wikipedia.org/wiki/Run-length_encoding).

Write a function that encodes a given text. Write another one that decodes.

We can see that RLE is not very useful in the general case. But it can be extremely useful if we have very few symbols. An example of this can be DNA and protein sequences. DNA code, for example, has only 4 characters.

Test your encoding and decoding functions on a DNA sequence (you can look up some on the Internet). Measure how much your data is compressed relative to the original.

```python
def encode(text):
    """
    Returns the run-length encoded version of the text
    (numbers after symbols, length = 1 is skipped)
    """
    splitted = list(text)

    grouped = [list(j) for i, j in groupby(splitted)]

    encoded = ''

    for group in grouped:
        encoded += group[0]
        encoded += str(len(group)) if len(group) > 1 else ''

    return encoded

def decode(text):
    """
    Decodes the text using run-length encoding
    """

    return re.sub(r'(\D)(\d*)', lambda m: m.group(1) * int(m.group(2)) if m.group(2) != '' else m.group(1), text)
```

```python
# Tests
# Test that the functions work on their own
assert encode("AABCCCDEEEE") == "A2BC3DE4"
assert decode("A2BC3DE4") == "AABCCCDEEEE"

# Test that the functions really invert each other
assert decode(encode("AABCCCDEEEE")) == "AABCCCDEEEE"
assert encode(decode("A2BC3DE4")) == "A2BC3DE4"
```

### Problem 5. Function Invertibility and Cryptography
As we already saw, some functions are able to be inverted. That is, if we know the output, we can see what input generated it directly. This is true if the function is **one-to-one correspondence** (bijection).

However, not all functions are created the same. Some functions are easy to compute but their inverses are extremely difficult. A very important example is **number factorization**. It's relatively easy (computationally) to multiply numbers but factoring them is quite difficult. Let's run an experiment.

We'll need a function to generate random n-bit numbers. One such number can be found in the `Crypto` package
```python
from Crypto.Util import number
random_integer = number.getRandomNBitInteger(n_bits)
```

We could, of course, write our factorization by hand but we'll use `sympy`
```python
from sympy.ntheory import factorint
factorint(1032969399047817906432668079951) # {3: 2, 79: 1, 36779: 1, 7776252885493: 1, 5079811103: 1}
```

This function returns a `dict` where the keys are the factors, and the values - how many times they should be multiplied.

We'll also need a tool to accurately measure performance. Have a look at [this one](https://docs.python.org/3/library/time.html#time.time) for example.

Specity a sequence of bit lengths, in increasing order. For example, you might choose something like `[10, 20, 25, 30, 32, 33, 35, 38, 40]`. Depending on your computer's abilities you can go as high as you want. For each bit length, generate a number. See how much time it takes to factor it. Then see how much time it takes to multiply the factors. Be careful how you measure these. You shouldn't include the number generation (or any other external functions) in your timing.

In order to have better accuracy, don't do this once per bit length. Do it, for example, five times, and average the results.

Plot all multiplication and factorization times as a function of the number of bits. You should see that factorization is much, much slower. If you don't see this, just try larger numbers :D.

```python
n_bits = [10, 20, 25, 30, 32, 33, 35, 38, 40, 64, 128]
f_times = []
mul_times = []

factorized = []
```

```python
def test_factorization():
    for bit in n_bits:
        random_integer = number.getRandomNBitInteger(bit)

        start = time.time()
        factorized_int = factorint(random_integer)
        end = time.time()
        f_times.append(end - start)

        factorized.append(factorized_int)
```

```python
def test_multiply():
    product = 1

    for factors in factorized:
        start = time.time()
        for factor, tm in factors.items():
            product *= factor ** tm

        end = time.time()
        mul_times.append(end - start)
```

```python
test_factorization()
test_multiply()
```

```python
plt.plot(n_bits, f_times, c = 'red', label = "Factorization")
plt.plot(n_bits, mul_times, c = 'green', label = "Multiplication")
plt.legend()
plt.title("Factorization vs Multiplication")
plt.show()
```

### * Problem 6. Diffie - Hellman Simulation
As we already saw, there are functions which are very easy to compute in the "forward" direction but really difficult (computationally) to invert (that is, determine the input from the output). There is a special case: the function may have a hidden "trap door". If you know where that door is, you can invert the function easily. This statement is at the core of modern cryptography.

Look up **Diffie - Hellman key exchange** (here's a [video](https://www.youtube.com/watch?v=cM4mNVUBtHk) on that but feel free to use anything else you might find useful).

Simulate the algorithm you just saw. Generate large enough numbers so the difference is noticeable (say, factoring takes 10-15 seconds). Simulate both participants in the key exchange. Simulate an eavesdropper.

First, make sure after both participants run the algotihm, they have *the same key* (they generate the same number).

Second, see how long it takes for them to exchange keys.

Third, see how long it takes the eavesdropper to arrive at the correct shared secret.

You should be able to see **the power of cryptography**. In this case, it's not that the function is irreversible. It can be reversed, but it takes a really long time (and with more bits, we're talking billions of years). However, if you know something else (this is called a **trap door**), the function becomes relatively easy to invert.

```python
# Write your code here
```

### ** Problem 7. The Galois Field in Cryptography
Research about the uses of the Galois field. What are its properties? How can it be used in cryptography? Write a simple cryptosystem based on the field.

You can use the following questions to facilitate your research:
* What is a field?
* What is GF(2)? Why is it an algebraic field?
* What is perfect secrecy? How does it relate to the participants in the conversation, and to the outside eavesdropper?
* What is symmetrical encryption?
* How to encrypt one-bit messages?
* How to extend the one-bit encryption system to many buts?
* Why is the system decryptable? How do the participants decrypt the encrypted messages?
* Why isn't the eavesdropper able to decrypt?
* What is a one-time pad?
    * How does the one-time pad achieve perfect secrecy?
* What happens if we try to use a one-time pad many times?
    * Provide an example where you break the "many-time pad" security
* What are some current enterprise-grade applications of encryption over GF(2)?
* Implement a cryptosystem based on GF(2). Show correctness on various test cases

### ** Problem 8. Huffman Compression Algorithm
Examine and implement the **Huffman algorithm** for compressing data. It's based on information theory and probiability theory. Document your findings and provide your implementation.

This algorithm is used for **lossless compression**: compressing data without loss of quality. You can use the following checklist:

* What is the difference betwenn lossless and lossy compression?
* When can we get away with lossy compression?
* What is entropy?
* How are Huffman trees constructed?
    * Provide a few examples
* How can we get back the uncompressed data from the Huffman tree?
* How and where are Huffman trees stored?
* Implement the algorithm. Add any other formulas / assumptions / etc. you might need.
* Test the algorithm. A good meaure would be percentage compression: $$\frac{\text{compressed}}{\text{uncompressed}} * 100\%$$
* How well does Huffman's algorithm perform compared to other compression algorithms (e.g. LZ77)?


# DATA 5600: Introduction to Regression and Machine Learning for Analytics

## __Computational Thinking, Statistical Thinking, and First Steps with the Monte Carlo Method__

Author:      Tyler J. Brough

Updated: September 22, 2021

---

```python
import numpy as np
from scipy import stats
import seaborn as sns
import matplotlib.pyplot as plt
from typing import List

sns.set_style('darkgrid')
plt.rcParams['figure.figsize'] = [10, 5]
```

```python
students = ['Tyler Clayson', 'Marshall Day', 'Liz Giles', 'Connor Hoiland', 'Josie Hull',
            'Brian Kissmer', 'Andrew Martin', 'Jacob Needham', 'Tori Rallison', 'Emily Rice',
            'Chris Simiskey', 'Wes Smith', 'Andrew Walker', 'Austin Warr', 'Connor Waterman']

def draw_a_student(students: List[str]):
    the_student = np.random.choice(students)
    print(f"{the_student}, come on down!")
```

```python
draw_a_student(students)
```

    Jacob Needham, come on down!

---

### __Example 1:__ The Binomial Distribution

#### __Preliminary Bits__

__Bernoulli Trials:__ A set of $n$ independent binary variables in which the $j$th observation is either a 'success' or 'failure', with probability of success, $\theta$, being the same for all trials.

#### The __Binomial distribution__

The distribution of the number of _successes_, $X$, in a series of $n$-independent ___Bernoulli trials___ where the probability of success at each trial is $\theta$ and the probability of failure is $1 - \theta$. Specifically the distribution is given by

$$
\large{Pr(X = x) = \frac{n!}{x!(n-x)!} \theta^{x}(1 - \theta)^{n-x}, \quad x = 1, 2, \ldots, n}
$$

The mean, variance, skewness and kurtosis of the distribution are as follows:

$$
\begin{align}
\mbox{mean} &= n\theta \\
& \\
\mbox{variance} &= n\theta(1 - \theta) \\
& \\
\mbox{skewness} &= ((1 - \theta) - \theta) / (n\theta(1-\theta))^{\frac{1}{2}} \\
& \\
\mbox{kurtosis} &= 3 - \frac{6}{n} + \frac{1}{n\theta(1 - \theta)} \\
\end{align}
$$

```python
np.random.binomial(n=10, p=0.5, size=1)
```

    array([3])

```python
M = 10_000
N = 50
theta = 0.5

heads = np.zeros(M)

## the main simulation loop
for i in range(M):
    heads[i] = np.random.binomial(n=N, p=theta)

(np.mean(heads), np.std(heads, ddof=1))
```

    (24.9841, 3.585238148750837)

```python
N * theta
```

    25.0

```python
np.sqrt(N * theta * (1. - theta))
```

    3.5355339059327378

```python
plt.hist(heads, bins=100);
```

__NB:__ We can actually vectorize this simulation quite simply as the following:

```python
more_heads = np.random.binomial(n=N, p=theta, size=M)
```

```python
plt.hist(more_heads, bins=25);
```

__Q:__ What's the difference?

##### __Standarizing the Results__

```python
z_scores = (more_heads - np.mean(more_heads)) / np.std(more_heads, ddof=1)
plt.hist(z_scores, bins=100);
```

```python
(np.mean(z_scores), np.std(z_scores, ddof=1))
```

    (4.092726157978177e-16, 1.0)

---

### __Example 2:__ The Poisson Distribution

#### The __Poisson Distribution__

The probability distribution of the number of occurrences, $X$, or some random event, in an interval of time or space. Given by

$$
\large{Pr(X = x) = \frac{e^{-\lambda} \lambda^{x}}{x!}, \quad x = 0, 1, \ldots}
$$

The mean and variance of the distribution are both $\lambda$. The skewness of the distribution is $1 / \sqrt{\lambda}$ and its kurtosis is $3 + 1 / \lambda$. The distribution is positively skewed for all values of $\lambda$.

```python
np.random.poisson?
```

    Docstring:
    poisson(lam=1.0, size=None)

    Draw samples from a Poisson distribution.

    The Poisson distribution is the limit of the binomial distribution
    for large N.

    .. note::
        New code should use the ``poisson`` method of a ``default_rng()``
        instance instead; please see the :ref:`random-quick-start`.

    Parameters
    ----------
    lam : float or array_like of floats
        Expectation of interval, must be >= 0. A sequence of expectation
        intervals must be broadcastable over the requested size.
    size : int or tuple of ints, optional
        Output shape.  If the given shape is, e.g., ``(m, n, k)``, then
        ``m * n * k`` samples are drawn.  If size is ``None`` (default),
        a single value is returned if ``lam`` is a scalar. Otherwise,
        ``np.array(lam).size`` samples are drawn.

    Returns
    -------
    out : ndarray or scalar
        Drawn samples from the parameterized Poisson distribution.

    See Also
    --------
    Generator.poisson: which should be used for new code.

    Notes
    -----
    The Poisson distribution

    .. math:: f(k; \lambda)=\frac{\lambda^k e^{-\lambda}}{k!}

    For events with an expected separation :math:`\lambda` the Poisson
    distribution :math:`f(k; \lambda)` describes the probability of
    :math:`k` events occurring within the observed
    interval :math:`\lambda`.

    Because the output is limited to the range of the C int64 type, a
    ValueError is raised when `lam` is within 10 sigma of the maximum
    representable value.

    References
    ----------
    .. [1] Weisstein, Eric W. "Poisson Distribution."
           From MathWorld--A Wolfram Web Resource.
           http://mathworld.wolfram.com/PoissonDistribution.html
    .. [2] Wikipedia, "Poisson distribution",
           https://en.wikipedia.org/wiki/Poisson_distribution

    Examples
    --------
    Draw samples from the distribution:

    >>> import numpy as np
    >>> s = np.random.poisson(5, 10000)

    Display histogram of the sample:

    >>> import matplotlib.pyplot as plt
    >>> count, bins, ignored = plt.hist(s, 14, density=True)
    >>> plt.show()

    Draw each 100 values for lambda 100 and 500:

    >>> s = np.random.poisson(lam=(100., 500.), size=(100, 2))
    Type:      builtin_function_or_method

```python
np.random.poisson(lam=4.5, size=10)
```

    array([7, 6, 7, 5, 5, 4, 5, 9, 1, 4])

```python
draw_a_student(students)
```

    Liz Giles, come on down!

```python
M = 10_000
N = 100
lam = 2

avg_counts = np.zeros(M)
for i in range(M):
    counts = np.random.poisson(lam=lam, size = N)
    avg_counts[i] = np.mean(counts)

```

```python
avg_counts[:10]
```

    array([1.81, 1.88, 1.87, 1.98, 2.02, 2.02, 1.93, 2.08, 1.81, 2.05])

---

```python
plt.hist(avg_counts, bins=25);
```

```python
z_scores = (avg_counts-lam)/np.sqrt(lam)
plt.hist(z_scores, bins=50);
```

### __Example 3:__ The Exponential Distribution

```python

```

#### __Preliminary Bits__

__Poisson Process:__ A point process with independent increments at constant intensity, say $\lambda$. The count after time $t$ has therefore a _Poisson distribution_ with mean $\lambda t$.

__Point Process:__ A stochastic process concerned with the occurrence of events at points of time or space determined by some chance mechanism.

#### The __Exponential Distribution__

The probability distribution $f(x)$ given by

$$
\large{f(x) = \lambda e^{-\lambda x}, \quad x > 0}
$$

The mean, variance, skewness, and kurtosis of the distribution are as follows:

$$
\begin{align}
\mbox{mean} &= 1 / \lambda \\
& \\
\mbox{variance} &= 1 / \lambda^{2} \\
& \\
\mbox{skewness} &= 2 \\
& \\
\mbox{kurtosis} &= 9 \\
& \\
\end{align}
$$

__NB:__ The distribution of intervals between independent consecutive random events, i.e. those following a ___Poisson process___.

__NB:__ `numpy` uses a different parameterization from the one above:

$$
f(x) = \frac{1}{\beta} e^{-\frac{1}{\beta}x}, \quad \mbox{which just substitutes} \quad \lambda = \frac{1}{\beta}
$$

```python
np.random.exponential?
```

```python
np.random.exponential(scale = 4, size=10)
```

```python
draw_a_student(students)
```

```python

```

```python

```

```python

```

---

### __A Simple Linear Regression: The Slope Coefficient__

Consider the following simple linear regression:

$$
\large{y_{i} = a + b x_{i} + \varepsilon_{t}}
$$

We are interested in the OLS estimate of the $b$ coefficient, $\hat{b}$, and its sampling distribution.

```python

```