```python
from IPython.core.display import HTML
from IPython.display import Image
HTML("""
<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>
""")
```

<style>
.output_png {
    display: table-cell;
    text-align: center;
    vertical-align: middle;
}
</style>

# *Circuitos Elétricos I - Semana 4*

### Problema 1

Determine o circuito equivalente de Thévenin ($v_{th}$, $R_{th}$) do ponto de vista dos terminais $(a,b)$ do circuito abaixo.

a) Determine a $v_{th}$ pelo método das tensões de nó.\
b) Determine a corrente de curto-circuito $i_{cc}$ pelo método das tensões de nó.\
c) Determine a $v_{th}$ via superposição.\
d) Determine a $R_{th}$ via resistência equivalente.\
e) Determine $v_{th}$, $R_{th}$ via transformação de fontes.

Simulação do circuito: https://tinyurl.com/y4kwdr8z

```python
import sympy as sp
from utils import symdisp
```

```python
# define as N variáveis desconhecidas
v1, v2, v3, v4 = sp.symbols('v1, v2, v3, v4')

# define os sistema de N equações
eq1 = sp.Eq()
eq2 = sp.Eq()
eq3 = sp.Eq()

# resolve o sistema
soluc = sp.solve((eq1, eq2, eq3), dict=True)
soluc = soluc[0]

v1 = soluc[v1]
v2 = soluc[v2]
v4 = soluc[v4]
v3 = 2

print('Solução do sistema:')
symdisp('v_1 = ', round(v1, 2), ' V')
symdisp('v_2 = ', round(v2, 2), ' V')
symdisp('v_3 = ', round(v3, 2), ' V')
symdisp('v_4 = ', round(v4, 2), ' V')
```

Jupyter notebooks/Circuitos Eletricos I - Semana 4.2.ipynb


# Acoustics

```python
%matplotlib inline
```

```python
%config InlineBackend.figure_format = 'svg'
import numpy as np
from exact_solvers import acoustics, acoustics_demos
from IPython.display import IFrame
```

In this chapter we consider our first *system* of hyperbolic conservation laws.  We study the acoustics equations that were introduced briefly in [Introduction.ipynb](Introduction.ipynb).  We first describe the physical context of this system and then investigate its characteristic structure and the solution to the Riemann problem.  This system is described in more detail in Chapter 3 of (LeVeque 2002).

If you wish to examine the Python code for this chapter, please see:

 - [exact_solvers/acoustics.py](exact_solvers/acoustics.py)
 - [exact_solvers/acoustics_demos.py](exact_solvers/acoustics_demos.py)

## Physical setting
The linear acoustic equations describe the propagation of small perturbations in a fluid, such as sound waves.  In [Advection.ipynb](Advection.ipynb) we derived the one-dimensional continuity equation, which describes mass conservation:
\begin{align} \label{Ac:continuity}
    \rho_t + (\rho u)_x & = 0.
\end{align}
For more realistic fluid models, we need another equation that determines the velocity $u$.  This typically takes the form of a conservation law for momentum $\rho u$.  Momentum, like density, is transported by fluid motion with corresponding flux $\rho u$.  Additionally, any difference in pressure will also lead to a flux of momentum that is proportional to the pressure difference.  Thus the momentum equation takes the form
\begin{align} \label{Ac:mom_cons}
(\rho u)_t + (\rho u^2 + P(\rho))_x & = 0,
\end{align}
where we assumed the system obeys the equation of state $P(\rho)$, where the pressure $P$ is only a function of the density. More general equations of state will be consider, along with fully nonlinear fluid motions in [Euler.ipynb](Euler.ipynb) and [Euler_Tammann.ipynb](Euler_Tammann.ipynb).  Linear acoustics equations focus on the behavior of small perturbations in the system above.

In order to derive acoustics linear equations, note equations (\ref{Ac:continuity}) and (\ref{Ac:mom_cons}) together form a hyperbolic system $q_t+f(q)_x=0$ with
\begin{align*}
q & = \begin{bmatrix} \rho \\ \rho u \end{bmatrix} &
f(q) & = \begin{bmatrix} \rho u \\ \rho u^2 + P(\rho) \end{bmatrix}
\end{align*}

We will further need to make use of the quasilinear form of a given hyperbolic system (which will be used throughout this book):
$$q_t + f'(q) q_x = 0,$$
where $f'(q)$ denotes the Jacobian of the flux $f$ with respect to the conserved variables $q$.  In the present system, as is often the case, $f$ is most naturally written in terms of so-called primitive variables (in this case $\rho$ and $u$) rather than in terms of the conserved variables $q$.  In order to find the flux Jacobian (and thus the quasilinear form), we first write $f$ in terms of the conserved variables $(q_1,q_2) = (\rho, \rho u)$:
\begin{align}
f(q) & = \begin{bmatrix} q_2 \\ q_2^2/q_1 + P(q_1) \end{bmatrix}.
\end{align}

Now we can differentiate to find the flux Jacobian:
\begin{align*}
f'(q) & = \begin{bmatrix} \partial f_1/\partial q_1 & \partial f_1/\partial q_2 \\
                          \partial f_2/\partial q_1 & \partial f_2/\partial q_2 \end{bmatrix} \\
      & = \begin{bmatrix} 0 & 1 \\ -q_2^2/q_1^2 + P'(q_1) & 2q_2/q_1 \end{bmatrix} \\
      & = \begin{bmatrix} 0 & 1 \\ P'(\rho)-u^2 & 2u \end{bmatrix}.
\end{align*}

Thus small perturbations to an ambient fluid state $\rho_0, u_0$ evolve according to the linearized equations $q_t + f'(q_0) q_x = 0$, or more explicitly
\begin{align*}
\rho_t + (\rho u)_x & = 0 \\
(\rho u)_t + (P'(\rho_0)-u_0^2)\rho_x + 2u_0(\rho u)_x & = 0.
\end{align*}
As we are only interested in small perturbations of equation (\ref{Ac:mom_cons}), we expand the perturbations $\rho-\rho_0$ and $\rho u - \rho_0 u_0$ as functions of a small parameter $\epsilon$, and then we discard terms of order $\epsilon^2$ and higher. This results in the linear hyperbolic system
\begin{align*}
p_t + u_0 p_x + P'(\rho_0) u_x & = 0 \\
u_t + \frac{1}{\rho_0} p_x + u_0 u_x & = 0,
\end{align*}
where $p(x,t)$ is the pressure as a function of $x$ and $t$. If the ambient fluid is at rest (i.e. $u_0=0$) and the pressure is directly proportional to the density, then this simplifies to
\begin{align} \label{Ac:main}
 \left[ \begin{array}{c}
p \\
u
\end{array} \right]_t +  \underbrace{\left[ \begin{array}{cc}
0 & K_0 \\
1/\rho_0 & 0  \\
\end{array} \right]}_{\mathbf{A}}
\left[ \begin{array}{c}
p \\
u \end{array} \right]_x = 0,
\end{align}
where $K_0=\rho_0 P'(\rho_0)$ is referred to as the bulk modulus of compressibility. The system of equations (\ref{Ac:main}) is called the linear acoustics equations.

For the rest of this chapter we work with (\ref{Ac:main}) and let $q=[p,u]^T$.  Then we can write (\ref{Ac:main}) as $q_t + A q_x = 0$.  For simplicity, we also drop the subscripts on $K, \rho$.  Direct calculation reveals that the eigenvectors of $A$ are
\begin{align}
\lambda_1 = -c, \qquad \lambda_2 = c
\end{align}

where $c=\sqrt{{K}/{\rho}}$ will correspond to the speed of sound in a medium with a given density and bulk modulu. The corresponding right eigenvectors are given by
\begin{align*}
r_1 = \begin{bmatrix}\begin{array}{c}-Z\\1\end{array}\end{bmatrix}, \qquad r_2 = \begin{bmatrix}\begin{array}{c}Z\\1\end{array}\end{bmatrix},
\end{align*}

where $Z=\rho c$ is called the acoustic impedance. Defining $R = [r_1 r_2]$ and $\Lambda = diag(\lambda_1, \lambda_2)$, we have $AR = R\Lambda$, or $A = R \Lambda R^{-1}$.  Substituting this into (\ref{Ac:main}) yields
\begin{align*}
q_t + A q_x & = 0 \\
q_t + R \Lambda R^{-1} q_x & = 0 \\
R^{-1}q_t + \Lambda R^{-1} q_x & = 0 \\
w_t + \Lambda w_x & = 0,
\end{align*}
where we have introduced the *characteristic variables* $w=R^{-1}q$.  The last system above is simply a pair of decoupled advection equations for $w_1$ and $w_2$, with velocities $\lambda_1$ and $\lambda_2$; a system we already know how to solve. Thus we see that the eigenvalues of $A$ are the velocities at which information propagates in the solution.

## Solution by characteristics

The discussion above suggests a strategy for solving the Cauchy problem:

1. Decompose the initial data $(p(x,0), u(x,0))$ into characteristic variables $w(x,0)=(w_1^0(x),w_2^0(x,0))$ using the relation $w = R^{-1}q$.
2. Evolve the characteristic variables: $w_p(x,t) = w_p^0(x-\lambda_p t)$.
3. Transform back to the physical variables: $q = Rw$.

The first step in this process amounts to expressing the vector $q$ in the basis given by $r_1, r_2$.  Solving the system $Rw=q$ yields
\begin{align*}
q = w_1 r_1 + w_2 r_2,
\end{align*}
where
\begin{align*}
w_1 = \frac{- p + Z u}{2Z}, \ \ \ \ \ \
w_2 = \frac{ p + Z u}{2Z}.
\end{align*}

We visualize this below, where the first plot shows the two eigenvectors, and the second plot shows how $q$ can be expressed as a linear combination of the two eigenvectors, $r_1$ and $r_2$.  In the live notebook you can adjust the left and right states or the material parameters to see how this affects the construction of the Riemann solution.

```python
acoustics_demos.decompose_q_interactive()
```

In the second and third steps, we evolve the characteristic variables $w$ and then transform back to the original variables. Let's assume the acoustics equations have a bump in pressure with zero velocity as the initial condition. We visualize this below, where the time evolution in the characteristic variables is shown in the first plot, and the time evolution back in one of the original variables (velocity) is shown in the second plot.

```python
acoustics_demos.char_solution_interactive()
```

Notice how in the characteristic variables $w$ (plotted on the left), each part of the solution simply advects (translates) since each of the characteristics variables simply obeys an uncoupled advection equation. However, back in the original variables the behavior is more complex and cannot be explained simply by advection.

## The Riemann problem

Now that we know how to solve the Cauchy problem, solution of the Riemann problem is merely a special case.  We have the special initial data
\begin{align*}
q(x,0) = \begin{cases}
q_\ell & \text{if   } x \le 0, \\
q_r & \text{if   } x > 0.
\end{cases}
\end{align*}
We can proceed as before, by decomposing into characteristic components, advecting, and then transforming back.  But since we know the solution will be constant almost everywhere, it's even simpler to just decompose the jump $\Delta q = q_r - q_\ell$ in terms of the characteristic variables, and advect the two resulting jumps $\Delta w_1$ and $\Delta w_2$:
\begin{align*}
\Delta q = \Delta w_1 r_1 + \Delta w_2 r_2,
\end{align*}
Since $R\Delta w = \Delta q$, we have
\begin{align*}
\Delta w_1 = \frac{-\Delta p + Z\Delta u}{2Z}, \ \ \ \ \ \
\Delta w_2 = \frac{\Delta p + Z\Delta u}{2Z}.
\end{align*}

Thus the solution has the structure depicted below.

The three constant states are related by the jumps:
\begin{align}
q_m = q_\ell + \Delta w_1 r_1 = q_r - \Delta w_2 r_2.
\label{eq:acussol}
\end{align}
Note that the form of the eigenvectors shows that, in any propagating discontinuity, the jump in $p$ is $\pm Z$ times the jump in $u$.  More generally, the eigenvectors of the coefficient matrix of a linear hyperbolic system reveal the relation between jumps in the different components of $q$ across the wave propagating with speed given by its corresponding eigenvalue.  For acoustics, the impedance is the physical parameter that determines this relation.

### A simple solution
Here we provide some very simple initial data, and we call the linear Riemann solver. This will output the three states $q_l$, $q_m$ and $q_r$, and the speeds of the two waves.

```python
# Initial data for Riemann problem
rho = 0.5               # density
bulk = 2.            # bulk modulus
ql = np.array([3,2])   # Left state
qr = np.array([3,-2])  # Right state
# Calculated parameters
c = np.sqrt(bulk/rho)  # calculate sound speed
Z = np.sqrt(bulk*rho)  # calculate impedance
print("With density rho = %g,  bulk modulus K = %g" \
      % (rho,bulk))
print("We compute: sound speed c = %g, impedance Z = %g \n" \
      % (c,Z))
```

```python
# Call and print Riemann solution
states, speeds, reval = \
    acoustics.exact_riemann_solution(ql ,qr, [rho, bulk])

print("The states ql, qm and qr are: ")
print(states, "\n")
print("The left and right wave speeds are:")
print(speeds)
```

As we already showed in the previous section, one way to understand the solution to a linear system like acoustics equations is by looking at the phase plane. We can show the structure of this solution in the $p-u$ phase plane. We show the two initial conditions of the Riemann problem $q_l$ and $q_r$ as points in the phase space; the lines crossing this points correspond to the eigenvectors, $r_1$ and $r_2$.

The solution of the middle state $q_m$ is simply the intersection of the line generated by $r_1$ passing through $q_l$ and the line generated by $r_2$ passing through $q_r$. The structure of this solution becomes evident from equation (\ref{eq:acussol}). Note there is another intersection between these lines (dashed), these correspond to connecting $q_l$ to $q_m$ with the $r_2$ eigenvector, which is not the correct solution (see equation (\ref{eq:acussol})).

In the live notebook, the cell below allows you to interactively adjust the initial conditions the material parameters as well as the plot range, so that you can explore how the structure of the solution in the phase plane is affected by these quantities.

```python
acoustics_demos.interactive_phase_plane(ql,qr,rho,bulk)
```

Note that the eigenvectors are given in terms of the impedance $Z$, which depends on the density $\rho$
and the bulk modulus $K$. Therefore, when $\rho$ and $K$ are modified the eigenvectors change and consequently the slope of the lines changes as well.

## Examples
We will use the exact solver in [exact_solvers/acoustics.py](exact_solvers/acoustics.py) and the functions in [exact_solvers/acoustics_demos.py](exact_solvers/acoustics_demos.py) to plot interactive solutions for different interesting examples.

### Shock tube

If there is a jump in pressure and the velocity is $0$ in both initial states (the shock tube problem) then the resulting Riemann solution consists of pressure jumps of equal magnitude propagating in both directions, with equal and opposite jumps in velocity. This is called the shock tube problem since it emulates what would happen inside a shock tube, where the air is static and a separate chamber at the end of the tube is pressurized to a high pressure and then released. This generates a high-pressure wave propagating towards the low-pressure region, while a lower pressure wave is propagating back into the chamber. We will later see that if we use more complex model, like Euler equations, these waves will correspond to a shock and a rarefaction, respectively.

```python
ql = np.array([5,0])
qr = np.array([1,0])
rho = 1.0
bulk = 4.0
acoustics_demos.riemann_plot_pplane(ql,qr,rho,bulk)
```

We can also observe the structure of the solution in the phase plane. In the second plot, we show the structure of the solution in the phase plane.

### Reflection from a wall

As another example, suppose the pressure is initially the same in the left and right states, while the velocities are non-zero with $u_r = -u_\ell > 0$.  Particles are converging from both sides and if the initial states have this symmetry, then the result is a middle state $q_m$ in which the velocity is 0 (and the pressure is higher than on either side).

```python
ql = np.array([2,1])
qr = np.array([2,-1])
rho = 1.0
bulk = 1.5
acoustics_demos.riemann_plot_pplane(ql,qr,rho,bulk)
```

We again show the Riemann solution at a given time along with the solution in the phase plane for comparison. Note the symmetry of the solution is also observed in the phase plane solution.

If one discards half the solution for $x>0$ (or for $x<0$), one can view this as a solution to a problem with fluid streaming at a constant velocity toward a solid wall. The result is an acoustic wave that moves away from the wall, and the fluid behind the shock has been decelerated to velocity 0, i.e. it is stationary at the wall.

This type of Riemann solution is critical when imposing solid wall boundary conditions in a numerical method. If ghost cells are introduced outside the domain and the state in the ghost cell set by reflecting the interior solution with the symmetry seen here (equal pressure, negated velocity), then the solution to the Riemann problem at the cell interfaces yields a solution that satisfies the desired boundary conditions.

Note it is possible to extend the Riemann problem solution for acoustic equations to cases where there are different materials on the left and right side. This is useful to solve the acoustic wave propagation across interfaces or heterogeneous media.

### Interactive phase plane with solution at fixed time

For a more general exploration of the solution to the acoustics equation, we now show an interactive solution of the acoustics equations. The initial condition, $q_l$ and $q_r$, can be modified by dragging and dropping the points in the phase plane plot (in the notebook version, or on [this webpage](http://www.clawpack.org/riemann_book/html/phase_plane_acoustics_small.html)). The solution at some fixed time will be modified accordingly, by showing the three values corresponding values for the pressure and the velocity.

```python
IFrame(src='html/phase_plane_acoustics_small_notitle.html',
       width=980, height=340)
```

### Bump initial condition
In this example, we use the original bump example described in the beginning of this chapter. The initial condition is simply a bump in the pressure, while the velocity equals to zero. Additionally, we set walls at $x=-2$ and $x=2$, so the wave is fully reflected back, and we can see how it interacts with itself. This animation is produced by running a numerical method, and can be viewed in the interactive notebook or on [this webpage](http://www.clawpack.org/riemann_book/html/acoustics_bump_animation.html).

```python
acoustics_demos.bump_animation(numframes = 50)
```


```python
%reload_ext autoreload
%autoreload 2
import pandas as pd
import warnings
warnings.filterwarnings("ignore", category=RuntimeWarning);
warnings.filterwarnings("ignore", category=DeprecationWarning);
#Need to fix deprecation warning
warnings.filterwarnings("ignore");
warnings.filterwarnings('ignore')
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
%config InlineBackend.figure_format = 'retina'

```

```python
from hmmlearn.hmm import GaussianHMM
from hmm import GaussianFSHMM
```

## HMM and FSHMM testing

#### Generate synthetic data

We generate two features from an HMM following the parameters from the paper, and three features that are random noise.

```python
startprob = np.array([0.4, 0.6])

transmat = np.array([[0.75, 0.25],
                     [0.4, 0.6]])

means = np.array([[10.0,  20.0],
                  [30.0, 60.0]])

covars = 25.0 * np.tile(np.identity(2), (2, 1, 1))
model = GaussianHMM(n_components=2, covariance_type="diag")

model.startprob_ = startprob
model.transmat_ = transmat
model.means_ = means
model._covars_ = covars
```

```python
X, Y = model.sample(500)
```

```python
x1 = np.random.normal(0, 1, 500)
x2 = np.random.normal(0, 1, 500)
x3 = np.random.normal(0, 1, 500)
```

```python
df = pd.DataFrame(X)
df['3'] = x1
df['4'] = x2
df['5'] = x3
```

```python
sns.set(style="whitegrid", palette="tab10")
df.plot(figsize=(16,9));
```

```python
# To make the plot interactive using cufflinks uncomment the following
# import plotly
# import cufflinks as cf
# from plotly.offline import init_notebook_mode, plot, iplot
# import colorlover as cl

# plotly.offline.init_notebook_mode(connected=False)
# cf.set_config_file(offline=True, world_readable=False, theme='white')
# df.iplot()
```

### FSHMM

#### Test from paper

```python
%reload_ext autoreload
%autoreload 2
from hmm import GaussianFSHMM
```

```python
model_new_new = GaussianFSHMM(k=50, n_components=2, algorithm='viterbi', covariance_type="diag", init_params='m', n_iter=1000)
```

```python
model_new_new.transmat_ = np.array([[0.5, 0.5], [0.5, 0.5]])
model_new_new.startprob_ = np.array([0.5, 0.5])
model_new_new._covars_ = 4.0 * np.tile(np.identity(5), (2, 1, 1))
```

```python
df_norm = (df-df.mean())/df.std()
df_scaled = (df-df.min())/(df.max()-df.min())
```

```python
# df_norm.iplot()
# df_scaled.iplot()
```

```python
model_new_new.fit(df_norm)
```

    GaussianFSHMM(k=None)

#### Feature saliencies of all features. First two saliencies are almost one and the last three features (added noise) are close to zero.

```python
rho_df = pd.DataFrame(model_new_new.rho_, index=['rho1', 'rho2', 'rho3', 'rho4', 'rho5'], columns=['Saliency']).T.round(3)
rho_df
```

      rho1
      rho2
      rho3
      rho4
      rho5

      Saliency
      0.982
      0.957
      0.0
      0.0
      0.035

```python
rho_df.T.plot.bar(figsize=(11,6));
```

```python
model_new_new.monitor_.converged
```

    True

```python
model_new_new.monitor_
```

    ConvergenceMonitor(history=[-5441.17401988895, -5441.1740198853095],
              iter=1000, n_iter=1000, tol=1e-09, verbose=False)

#### Transition matrix

The transition matrix from the original HMM was:

$$\begin{pmatrix}
           0.75 & 0.25\\
           0.4, & 0.6
    \end{pmatrix}$$

```python
pd.DataFrame(model_new_new.transmat_).round(3)
```

      0
      1

      0
      0.73
      0.27

      1
      0.37
      0.63

#### Parameter estimates for $\mu$
The means for the two-state HMM were:

$$\begin{align}
    S0_{\mu} =
        \begin{pmatrix}
            10\\
           20
    \end{pmatrix}
    \qquad
    S1_{\mu} =
        \begin{pmatrix}
            30\\
           60
    \end{pmatrix}
    \nonumber
  \end{align}$$

and the other three time series had a mean of 0.

```python
pd.DataFrame((model_new_new.means_)*np.array(df.std())+np.array(df.mean()), columns=['mu1', 'mu2', 'mu3', 'mu4', 'mu5'], index=['S0', 'S1']).round(3)
```

      mu1
      mu2
      mu3
      mu4
      mu5

      S0
      10.347
      20.180
      -1.094
      -1.006
      -1.293

      S1
      30.139
      60.239
      0.999
      0.980
      1.017

#### Parameter estimates for $\sigma$

The variances for the two-state HMM were:

$$\begin{align}
    S0_{\sigma^2} =
        \begin{pmatrix}
            25\\
           25
    \end{pmatrix}
    \qquad
    S1_{\sigma^2} =
        \begin{pmatrix}
            25\\
           25
    \end{pmatrix}
    \nonumber
  \end{align}$$

therefore the std for each component and each state was 5, and the other three time series had a std of 1.

```python
pd.DataFrame(np.sqrt((model_new_new._covars_)*np.array(df.std())+np.array(df.mean())), columns=['sigma1', 'sigma2', 'sigma3', 'sigma4', 'sigma5'], index=['S0', 'S1']).round(3)
```

      sigma1
      sigma2
      sigma3
      sigma4
      sigma5

      S0
      4.694
      6.235
      0.549
      0.564
      0.974

      S1
      4.690
      6.241
      0.549
      0.564
      0.492

#### Parameter estimates for $\epsilon$, the last three values (the irrelevant features) are close to zero, which is the true value.

```python
pd.DataFrame(model_new_new.epsilon_, index=['eps1', 'eps2', 'eps3', 'eps4', 'eps5'], columns=['MAP']).T.round(3)
```

      eps1
      eps2
      eps3
      eps4
      eps5

      MAP
      -0.012
      0.097
      -0.0
      0.0
      0.014

#### Parameter estimates for $\tau$, the last three features are close to one, which is the true value.

```python
pd.DataFrame(model_new_new.tau_, index=['tau1', 'tau2', 'tau3', 'tau4', 'tau5'], columns=['MAP']).T.round(3)
```

      tau1
      tau2
      tau3
      tau4
      tau5

      MAP
      0.653
      0.703
      0.997
      0.997
      0.966

```python
# HTML('''''')
```

```python

```

```python

```


# Table of Contents

1  ALGO1 : Introduction à l'algorithmique

2  Cours Magistral 9

2.1  Documentation de scipy.opt.linprog

2.2  Fonction de débogage

2.3  Premier exemple

2.4  Second exemple

2.4.1  Essayons avec différentes méthodes de résolution :

2.4.1.1  Avec la méthode du simplexe

2.4.1.2  Avec la méthode du point intérieur (un autre algorithme)

2.5  Bonus : implémentation manuelle de l'algorithme du simplexe

2.5.1  Algorithme

2.5.2  Conversion en forme standard

2.5.3  Utilitaires pour les matrices

2.5.4  L'algorithme du simplexe

2.5.5  Un premier exemple

2.5.6  Tests

2.6  Conclusion

# [ALGO1 : Introduction à l'algorithmique](https://perso.crans.org/besson/teach/info1_algo1_2019/)

- [Page du cours](https://perso.crans.org/besson/teach/info1_algo1_2019/) : https://perso.crans.org/besson/teach/info1_algo1_2019/
- Magistère d'Informatique de Rennes - ENS Rennes - Année 2019/2020
- Intervenants :
  + Cours : [Lilian Besson](https://perso.crans.org/besson/)
  + Travaux dirigés : [Raphaël Truffet](http://perso.eleves.ens-rennes.fr/people/Raphael.Truffet/)
- Références :
  + [Open Data Structures](http://opendatastructures.org/ods-python.pdf)

# Cours Magistral 9

- Ce cours traite de programmation linéaire.

- On va illustrer deux programmes linéaires résolus avec la fonction `scipy.optimize.linprog`.

```python
import numpy as np

import matplotlib.pyplot as plt

import scipy.optimize as opt
```

## Documentation de `scipy.opt.linprog`

```python
print("\n".join(opt.linprog.__doc__.split("\n")[:31]))
```

        Linear programming: minimize a linear objective function subject to linear
        equality and inequality constraints.

        Linear programming solves problems of the following form:

        .. math::

            \min_x \ & c^T x \\
            \mbox{such that} \ & A_{ub} x \leq b_{ub},\\
            & A_{eq} x = b_{eq},\\
            & l \leq x \leq u ,

        where :math:`x` is a vector of decision variables; :math:`c`,
        :math:`b_{ub}`, :math:`b_{eq}`, :math:`l`, and :math:`u` are vectors; and
        :math:`A_{ub}` and :math:`A_{eq}` are matrices.

        Informally, that's:

        minimize::

            c @ x

        such that::

            A_ub @ x <= b_ub
            A_eq @ x == b_eq
            lb <= x <= ub

        Note that by default ``lb = 0`` and ``ub = None`` unless specified with
        ``bounds``.

La programmation linéaire résout des problèmes de la forme suivante :

$$
\begin{align}
\min_x \ & c^T x \\
\mbox{such that} \ & A_{ub} x \leq b_{ub},\\
& A_{eq} x = b_{eq},\\
& l \leq x \leq u ,
\end{align}
$$

avec $x$ un vecteur de variables de décisions ; $c$, $b_{ub}$, $b_{eq}$, $l$, et $u$ sont des vecteurs ; $A_{ub}$ et $A_{eq}$ sont des matrices.

## Fonction de débogage

```python
def make_callback():
    list_of_x, list_of_fun = [], []
    def debug_callback(opt_res):
        print("\nA new optimization step gave:")
        print(f"Current solution x = {opt_res.x}")
        list_of_x.append(opt_res.x)
        print(f"Current value of c @ x = {opt_res.fun}")
        list_of_fun.append(opt_res.fun)
        print(f"Success? = {opt_res.success}")
        print(f"The (nominally positive) values of the slack, b_ub - A_ub @ x. = {opt_res.slack}")
        print(f"The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = {opt_res.con}")
        print(f"Algorithm in phase {opt_res.phase}")
        print(f"Algorithm in iteration number {opt_res.nit}")
        status = {
            0: "Optimization proceeding nominally.",
            1: "Iteration limit reached.",
            2: "Problem appears to be infeasible.",
            3: "Problem appears to be unbounded.",
            4: "Numerical difficulties encountered.",
        }
        print(f"Algorithm status {status[opt_res.status]}")
        if opt_res.message: print(f"Algorithm message: {opt_res.message}")
    return list_of_x, list_of_fun, debug_callback
```

## Premier exemple

On va suivre l'exemple détaillé en cours :

- Variables
  + $x$ nombre de tables fabriquées par semaine,
  + $y$ nombre de chaises fabriquées par semaine.

- Objectif :
  + maximiser $30x + 10y$.

- Contraintes :
  + heures de travail : $6x+3y \leq 36$,
  + demande : $y \geq 3x$,
  + stockage : $x + y/4 \leq 4$,
  + positivité : $x \geq 0$,
  + positivité : $y \geq 0$.

Mise sous la forme requise par la fonction `linprog`, cela va donner :

$$
\begin{align}
\min_{[x, y]} \ & [-30, -10]^T [x, y] \\
\mbox{such that} \ &
    [
        [6, 3], [3, -1], [1, 1/4]
    ]
    [x, y]
= [6x + 3y, 3x - y, x + y/4] \leq [36, 0, 4],\\
& [0, 0] \leq [x, y] \leq [+\infty,+\infty] ,
\end{align}
$$

Et donc avec Python cela sera :

```python
c = np.array([-30, -10])

A_ub = np.array([[6, 3], [3, -1], [1, 1/4]])
b_ub = np.array([36, 0, 4])

A_eq = None
b_eq = None

# all variables are bound to be in (0, +inf)
bounds = (0, None)
```

Objectif :

```python
import sympy
x, y = sympy.var('x y')
```

```python
c.T @ [x, y]
```

$\displaystyle - 30 x - 10 y$

Contraintes d'inéqualités :

```python
A_ub @ [x, y]
```

    array([6.0*x + 3.0*y, 3.0*x - 1.0*y, 1.0*x + 0.25*y], dtype=object)

```python
b_ub
```

    array([36,  0,  4])

Essayons avec différentes méthodes de résolution :

```python
list_of_x, list_of_fun, debug_callback = make_callback()

opt.linprog(c,
            A_ub=A_ub, b_ub=b_ub,
            A_eq=A_eq, b_eq=b_eq,
            bounds=bounds,
            method="simplex",
            callback=debug_callback,
)
```

    A new optimization step gave:
    Current solution x = [0. 0.]
    Current value of c @ x = 0.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [36.  0.  4.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 0
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [0. 0.]
    Current value of c @ x = 0.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [36.  0.  4.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 1
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [2.28571429 6.85714286]
    Current value of c @ x = -137.14285714285717
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [1.71428571e+00 8.88178420e-16 0.00000000e+00]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 2
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [2.28571429 6.85714286]
    Current value of c @ x = -137.14285714285717
    Success? = True
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [1.71428571e+00 8.88178420e-16 0.00000000e+00]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 3
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [2.28571429 6.85714286]
    Current value of c @ x = -137.14285714285717
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [1.71428571e+00 8.88178420e-16 0.00000000e+00]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 2
    Algorithm in iteration number 3
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [2. 8.]
    Current value of c @ x = -140.0
    Success? = True
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [7.10542736e-15 2.00000000e+00 0.00000000e+00]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 2
    Algorithm in iteration number 4
    Algorithm status Optimization proceeding nominally.

         con: array([], dtype=float64)
         fun: -140.0
     message: 'Optimization terminated successfully.'
         nit: 4
       slack: array([7.10542736e-15, 2.00000000e+00, 0.00000000e+00])
      status: 0
     success: True
           x: array([2., 8.])

```python
x_opt, y_opt = _.x
```

La solution obtenue est donc $x = 2$ et $y = 8$, qui donnerait un profit maximal de $+140 €$ par semaine en respectant toutes les contraintes.

Pour obtenir une solution entière, on a rien à faire ici.

Si la solution optimale était par exemple $2.23$ et $6.43$, on pourrait essayer $x = 2, 3$ et $y = 6, 7$, ie. on arrondit en dessous et au dessus, et on prend la solution qui satisfait les contraintes et maximise l'objectif :

```python
x_opt, y_opt
```

    (2.000000000000001, 7.9999999999999964)

```python
import itertools
```

```python
sol = None
min_obj = float("+inf")

for (x, y) in itertools.product(
        [int(np.floor(x_opt)), int(np.ceil(x_opt))],
        [int(np.floor(y_opt)), int(np.ceil(y_opt))],
    ):
    obj = c.T @ [x, y]
    ctr = (A_ub @ [x, y]) <= b_ub
    print(f"Pour (x, y) = {x, y}, l'objectif vaut {obj}, la contrainte vaut {ctr}")
    if np.all(ctr) and obj < min_obj:
        min_obj = obj
        sol = [x, y]

print(f"==> Donc on utilise la solution entière optimale = {sol}")
```

    Pour (x, y) = (2, 7), l'objectif vaut -130, la contrainte vaut [ True  True  True]
    Pour (x, y) = (2, 8), l'objectif vaut -140, la contrainte vaut [ True  True  True]
    Pour (x, y) = (3, 7), l'objectif vaut -160, la contrainte vaut [False False False]
    Pour (x, y) = (3, 8), l'objectif vaut -170, la contrainte vaut [False False False]
    ==> Donc on utilise la solution entière optimale = [2, 8]

La solution entière optimale à ce premier problème est donc de fabriquer $x=2$ tables et $y=8$ chaises chaque semaine.

## Second exemple

Mise sous la forme requise par la fonction `linprog`, cela va donner :

$$
\begin{align}
\min_{[x, y]} \ & [4, 3]^T [x, y] \\
\mbox{such that} \ &
    [
        [-1, 4], [1, 1], [3, -1]
    ]
    [x, y]
= [-x + 4y, x + y, 3x - y] \leq [16, 9, 15],\\
& [0, 0] \leq [x, y] \leq [+\infty,+\infty] ,
\end{align}
$$

Et donc avec Python cela sera :

```python
c = np.array([-4, -3])

A_ub = np.array([[-1, 4], [1, 1], [3, -1]])
b_ub = np.array([16, 9, 15])

A_eq = None
b_eq = None

# all variables are bound to be in (0, +inf)
bounds = (0, None)
```

Objectif :

```python
import sympy
x, y = sympy.var('x y')
```

```python
c.T @ [x, y]
```

$\displaystyle - 4 x - 3 y$

Contraintes d'inéqualités :

```python
A_ub @ [x, y]
```

    array([-x + 4*y, x + y, 3*x - y], dtype=object)

```python
b_ub
```

    array([16,  9, 15])

### Essayons avec différentes méthodes de résolution :

#### Avec la méthode du simplexe

```python
list_of_x, list_of_fun, debug_callback = make_callback()

opt.linprog(c,
            A_ub=A_ub, b_ub=b_ub,
            A_eq=A_eq, b_eq=b_eq,
            bounds=bounds,
            method="simplex",
            callback=debug_callback,
)
```

    A new optimization step gave:
    Current solution x = [0. 0.]
    Current value of c @ x = 0.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [16.  9. 15.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 0
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [0. 4.]
    Current value of c @ x = -12.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [ 0.  5. 19.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 1
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [4. 5.]
    Current value of c @ x = -31.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [0. 0. 8.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 2
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [4. 5.]
    Current value of c @ x = -31.0
    Success? = True
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [0. 0. 8.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 3
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [4. 5.]
    Current value of c @ x = -31.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [0. 0. 8.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 2
    Algorithm in iteration number 3
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [6. 3.]
    Current value of c @ x = -33.0
    Success? = True
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [10.  0.  0.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 2
    Algorithm in iteration number 4
    Algorithm status Optimization proceeding nominally.

         con: array([], dtype=float64)
         fun: -33.0
     message: 'Optimization terminated successfully.'
         nit: 4
       slack: array([10.,  0.,  0.])
      status: 0
     success: True
           x: array([6., 3.])

```python
plt.figure(figsize=(10, 7))
plt.title("Valeur de l'objectif étape par étape (méthode simplexe)")
plt.plot(list_of_fun, "ro-", lw=3, ms=14)
plt.show()
```

```python
plt.figure(figsize=(10, 7))
plt.title("Position des points étape par étape (méthode simplexe)")
list_of_X, list_of_Y = [x for (x,y) in list_of_x], [y for (x,y) in list_of_x]
# plt.plot(list_of_X, list_of_Y, 'bo-')
plt.plot(list_of_X, 'bo-', label="Valeur de x", lw=3, ms=14)
plt.plot(list_of_Y, 'gd-', label="Valeur de y", lw=3, ms=14)
plt.legend()
plt.show()
```

#### Avec la méthode du point intérieur (un autre algorithme)

Cet autre algorithme est plus récent, plus technique, et il fonctionne généralement mieux : plus rapide, plus stable numérique.

```python
list_of_x, list_of_fun, debug_callback = make_callback()

opt.linprog(c,
            A_ub=A_ub, b_ub=b_ub,
            A_eq=A_eq, b_eq=b_eq,
            bounds=bounds,
            method="interior-point",
            callback=debug_callback,
)
```

    A new optimization step gave:
    Current solution x = [1. 1.]
    Current value of c @ x = -7.0
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [13.  7. 13.]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 0
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [4.01262108 3.38515509]
    Current value of c @ x = -26.205949599949
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [6.47200073 1.60222383 6.34729184]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 1
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [5.9137431  2.95777926]
    Current value of c @ x = -32.52831014613114
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [10.08262607  0.12847765  0.21654997]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 2
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [5.9999917  2.99999607]
    Current value of c @ x = -32.999954993936036
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [1.00000074e+01 1.22338576e-05 2.09841077e-05]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 3
    Algorithm status Optimization proceeding nominally.

    A new optimization step gave:
    Current solution x = [6. 3.]
    Current value of c @ x = -32.9999999977497
    Success? = False
    The (nominally positive) values of the slack, b_ub - A_ub @ x. = [1.00000000e+01 6.11692030e-10 1.04921938e-09]
    The (nominally zero) residuals of the equality constraints, b_eq - A_eq @ x. = []
    Algorithm in phase 1
    Algorithm in iteration number 4
    Algorithm status Optimization proceeding nominally.

         con: array([], dtype=float64)
         fun: -32.9999999977497
     message: 'Optimization terminated successfully.'
         nit: 4
       slack: array([1.00000000e+01, 6.11692030e-10, 1.04921938e-09])
      status: 0
     success: True
           x: array([6., 3.])

```python
plt.figure(figsize=(10, 7))
plt.title("Valeur de l'objectif étape par étape (méthode point intérieur)")
plt.plot(list_of_fun, "ro-", lw=3, ms=14)
plt.show()
```

```python
plt.figure(figsize=(10, 7))
plt.title("Position des points étape par étape (méthode point intérieur)")
list_of_X, list_of_Y = [x for (x,y) in list_of_x], [y for (x,y) in list_of_x]
# plt.plot(list_of_X, list_of_Y, 'bo-')
plt.plot(list_of_X, 'bo-', label="Valeur de x", lw=3, ms=14)
plt.plot(list_of_Y, 'gd-', label="Valeur de y", lw=3, ms=14)
plt.legend()
plt.show()
```

----
## Bonus : implémentation manuelle de l'algorithme du simplexe

- Refs : [ce poste de blogue](http://jeremykun.com/2014/12/01/linear-programming-and-the-simplex-algorithm/), [ces notes de cours](http://jeffe.cs.illinois.edu/teaching/algorithms/notes/I-simplex.pdf), et [cet autre poste de blogue](https://medium.com/@jacob.d.moore1/coding-the-simplex-algorithm-from-scratch-using-python-and-numpy-93e3813e6e70).
- Code source venant de : https://github.com/j2kun/simplex-algorithm/

### Algorithme

```python
import numpy as np
import heapq
```

```python
def identity(numRows, numCols, val=1, rowStart=0):
    """ Return a rectangular identity matrix with the specified diagonal entiries, possibly starting in the middle.
    """
    # return val * np.ones((numRows, numCols))
    return [
        [
            (val if i == j else 0)
            for j in range(numCols)
        ]
        for i in range(rowStart, numRows)
    ]
```

### Conversion en forme standard

```python
def standardForm(cost,
                 greaterThans=None, gtThreshold=None,
                 lessThans=None, ltThreshold=None,
                 equalities=None, eqThreshold=None,
                 maximization=True):
    """
       standardForm: [float], [[float]], [float], [[float]], [float], [[float]], [float] -> [float], [[float]], [float]
       Convert a linear program in general form to the standard form for the
       simplex algorithm.  The inputs are assumed to have the correct dimensions: cost
       is a length n list, greaterThans is an n-by-m matrix, gtThreshold is a vector
       of length m, with the same pattern holding for the remaining inputs. No
       dimension errors are caught, and we assume there are no unrestricted variables.
    """
    newVars = 0
    numRows = 0
    if gtThreshold:
        newVars += len(gtThreshold)
        numRows += len(gtThreshold)
    if ltThreshold:
        newVars += len(ltThreshold)
        numRows += len(ltThreshold)
    if eqThreshold:
        numRows += len(eqThreshold)

    if not maximization:
        cost = [-x for x in cost]

    if newVars == 0:
        return cost, equalities, eqThreshold

    newCost = list(cost) + ([0] * newVars)

    constraints = [ ]
    threshold   = [ ]

    oldConstraints = [(greaterThans, gtThreshold, -1), (lessThans, ltThreshold, 1),
                     (equalities, eqThreshold, 0)]

    offset = 0
    for constraintList, oldThreshold, coefficient in oldConstraints:
        constraints += [c + r for c, r in zip(constraintList,
             identity(numRows, newVars, coefficient, offset))]

        threshold += oldThreshold
        offset += len(oldThreshold)

    return newCost, constraints, threshold
```

### Utilitaires pour les matrices

```python
def dot(a, b):
    return sum(x*y for x, y in zip(a, b))
```

```python
def column(A, j):
    return [row[j] for row in A]
```

```python
def transpose(A):
    return [column(A, j) for j in range(len(A[0]))]
```

```python
def isPivotCol(col):
    return (len([c for c in col if c == 0]) == len(col) - 1) and sum(col) == 1

def variableValueForPivotColumn(tableau, column):
    pivotRow = [i for (i, x) in enumerate(column) if x == 1][0]
    return tableau[pivotRow][-1]
```

```python
# assume the last m columns of A are the slack variables; the initial basis is
# the set of slack variables
def initialTableau(c, A, b):
    tableau = [row[:] + [x] for row, x in zip(A, b)]
    tableau.append([ci for ci in c] + [0])
    return tableau
```

```python
def primalSolution(tableau):
    # the pivot columns denote which variables are used
    columns = transpose(tableau)
    indices = [j for j, col in enumerate(columns[:-1]) if isPivotCol(col)]
    return [(colIndex, variableValueForPivotColumn(tableau, columns[colIndex]))
            for colIndex in indices]
```

```python
def objectiveValue(tableau):
    return -(tableau[-1][-1])
```

```python
def canImprove(tableau):
    lastRow = tableau[-1]
    return any(x > 0 for x in lastRow[:-1])
```

```python
# this can be slightly faster
def moreThanOneMin(L):
    if len(L) <= 1:
        return False

    x, y = heapq.nsmallest(2, L, key=lambda x: x[1])
    return x == y
```

```python
def findPivotIndex(tableau):
    # pick minimum positive index of the last row
    column_choices = [(i, x) for (i, x) in enumerate(tableau[-1][:-1]) if x > 0]
    column = min(column_choices, key=lambda a: a[1])[0]

    # check if unbounded
    if all(row[column] <= 0 for row in tableau):
        raise Exception('Linear program is unbounded.')

    # check for degeneracy: more than one minimizer of the quotient
    quotients = [
        (i, r[-1] / r[column])
        for i, r in enumerate(tableau[:-1])
        if r[column] > 0
    ]

    if moreThanOneMin(quotients):
        raise Exception('Linear program is degenerate.')

    # pick row index minimizing the quotient
    row = min(quotients, key=lambda x: x[1])[0]

    return row, column
```

```python
def pivotAbout(tableau, pivot):
    i, j = pivot

    pivotDenom = tableau[i][j]
    tableau[i] = [x / pivotDenom for x in tableau[i]]

    for k,row in enumerate(tableau):
        if k != i:
            pivotRowMultiple = [y * tableau[k][j] for y in tableau[i]]
            tableau[k] = [x - y for x,y in zip(tableau[k], pivotRowMultiple)]
```

### L'algorithme du simplexe

```python
def simplex(c, A, b):
    """
    simplex: c: [float], A: [[float]], b: [float] -> [float], float
    Solve the given standard-form linear program:
        max <c, x>
        s.t. Ax = b
             x >= 0
    Providing the optimal solution x* and the value of the objective function.
    """
    tableau = initialTableau(c, A, b)
    print("Initial tableau:")
    for row in tableau:
        print(row)
    print()

    while canImprove(tableau):
        pivot = findPivotIndex(tableau)
        print("Next pivot index is={}\n".format(pivot))
        pivotAbout(tableau, pivot)
        print("Tableau after pivot:")
        for row in tableau:
            print(row)
        print()

    return tableau, primalSolution(tableau), objectiveValue(tableau)
```

### Un premier exemple

```python
c = [300, 250, 450]
A = [[15, 20, 25], [35, 60, 60], [20, 30, 25], [0, 250, 0]]
b = [1200, 3000, 1500, 500]

# add slack variables by hand
A[0] += [1, 0, 0, 0]
A[1] += [0, 1, 0, 0]
A[2] += [0, 0, 1, 0]
A[3] += [0, 0, 0, -1]
c += [0, 0, 0, 0]

t, s, v = simplex(c, A, b)
print(s)
print(v)
```

    Initial tableau:
    [15, 20, 25, 1, 0, 0, 0, 1200]
    [35, 60, 60, 0, 1, 0, 0, 3000]
    [20, 30, 25, 0, 0, 1, 0, 1500]
    [0, 250, 0, 0, 0, 0, -1, 500]
    [300, 250, 450, 0, 0, 0, 0, 0]

    Next pivot index is=(3, 1)

    Tableau after pivot:
    [15.0, 0.0, 25.0, 1.0, 0.0, 0.0, 0.08, 1160.0]
    [35.0, 0.0, 60.0, 0.0, 1.0, 0.0, 0.24, 2880.0]
    [20.0, 0.0, 25.0, 0.0, 0.0, 1.0, 0.12, 1440.0]
    [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, -0.004, 2.0]
    [300.0, 0.0, 450.0, 0.0, 0.0, 0.0, 1.0, -500.0]

    Next pivot index is=(1, 6)

    Tableau after pivot:
    [3.333333333333332, 0.0, 5.0, 1.0, -0.33333333333333337, 0.0, 0.0, 200.0]
    [145.83333333333334, 0.0, 250.0, 0.0, 4.166666666666667, 0.0, 1.0, 12000.0]
    [2.5, 0.0, -5.0, 0.0, -0.5, 1.0, 0.0, 0.0]
    [0.5833333333333334, 1.0, 1.0, 0.0, 0.01666666666666667, 0.0, 0.0, 50.0]
    [154.16666666666666, 0.0, 200.0, 0.0, -4.166666666666667, 0.0, 0.0, -12500.0]

    Next pivot index is=(2, 0)

    Tableau after pivot:
    [0.0, 0.0, 11.666666666666664, 1.0, 0.33333333333333315, -1.333333333333333, 0.0, 200.0]
    [0.0, 0.0, 541.6666666666667, 0.0, 33.333333333333336, -58.33333333333334, 1.0, 12000.0]
    [1.0, 0.0, -2.0, 0.0, -0.2, 0.4, 0.0, 0.0]
    [0.0, 1.0, 2.166666666666667, 0.0, 0.13333333333333336, -0.23333333333333336, 0.0, 50.0]
    [0.0, 0.0, 508.3333333333333, 0.0, 26.666666666666664, -61.666666666666664, 0.0, -12500.0]

    Next pivot index is=(1, 4)

    Tableau after pivot:
    [0.0, 0.0, 6.250000000000001, 1.0, 0.0, -0.75, -0.009999999999999993, 80.00000000000007]
    [0.0, 0.0, 16.25, 0.0, 1.0, -1.7500000000000002, 0.03, 360.0]
    [1.0, 0.0, 1.25, 0.0, 0.0, 0.04999999999999993, 0.006, 72.0]
    [0.0, 1.0, 0.0, 0.0, 0.0, 5.551115123125783e-17, -0.004000000000000001, 1.999999999999993]
    [0.0, 0.0, 75.0, 0.0, 0.0, -14.999999999999993, -0.7999999999999999, -22100.0]

    Next pivot index is=(0, 2)

    Tableau after pivot:
    [0.0, 0.0, 1.0, 0.15999999999999998, 0.0, -0.11999999999999998, -0.0015999999999999988, 12.80000000000001]
    [0.0, 0.0, 0.0, -2.5999999999999996, 1.0, 0.1999999999999995, 0.05599999999999998, 151.99999999999986]
    [1.0, 0.0, 0.0, -0.19999999999999996, 0.0, 0.1999999999999999, 0.007999999999999998, 55.999999999999986]
    [0.0, 1.0, 0.0, 0.0, 0.0, 5.551115123125783e-17, -0.004000000000000001, 1.999999999999993]
    [0.0, 0.0, 0.0, -11.999999999999998, 0.0, -5.999999999999995, -0.68, -23060.0]

    [(0, 55.999999999999986), (1, 1.999999999999993), (2, 12.80000000000001), (4, 151.99999999999986)]
    23060.0

Et pou comparer avec la réponse donnée par `scipy.optimize.linprog` :

```python
opt_res = opt.linprog(-np.array(c), A_ub=A, b_ub=b, method="simplex")
opt_res
```

         con: array([], dtype=float64)
         fun: -23400.0
     message: 'Optimization terminated successfully.'
         nit: 5
       slack: array([  0.,   0.,   0., 500.])
      status: 0
     success: True
           x: array([ 60.,   0.,  12.,   0., 180.,   0.,   0.])

La solution optimale trouvée par `scipy.optimize.linprog` est meilleure que celle trouvée par notre algorithme.

```python
v, - opt_res.fun
```

    (23060.0, 23400.0)

Notre implémentation donne une solution :

```python
s
```

    [(0, 55.999999999999986),
     (1, 1.999999999999993),
     (2, 12.80000000000001),
     (4, 151.99999999999986)]

Qui s'interprète comme étant assez proche de la solution trouvée par `scipy.optimize.linprog`.

```python
opt_res.x
```

    array([ 60.,   0.,  12.,   0., 180.,   0.,   0.])

```python
s2 = np.array([56, 2, 12, 0, 152, 0, 0])
s2
```

    array([ 56,   2,  12,   0, 152,   0,   0])

```python
np.linalg.norm(opt_res.x - s2) / np.linalg.norm(opt_res.x)
```

    0.1491454186681393

C'est une différence relativement faible…

### Tests

```python
def test(expected, actual):
    e, a = np.array(expected), np.array(actual)
    if not np.isclose(np.linalg.norm(e - a), 0):
        import sys, traceback
        (filename, lineno, container, code) = traceback.extract_stack()[-2]
        print("Test: {} failed on line {} in file {}.\nExpected {} but got {}\n".format((code, lineno, filename, expected, actual)))
```

```python
def testFromPost():
    cost = [1, 1, 1]
    gts = [[0, 1, 4]]
    gtB = [10]
    lts = [[3, -2, 0]]
    ltB = [7]
    eqs = [[1, 1, 0]]
    eqB = [2]

    expectedCost = [1,1,1,0,0]
    expectedConstraints = [[0,1,4,-1,0], [3,-2,0,0,1], [1,1,0,0,0]]
    expectedThresholds = [10,7,2]
    c, cs, ts = standardForm(cost, gts, gtB, lts, ltB, eqs, eqB)
    test(expectedCost, c)
    test(expectedConstraints, cs)
    test(expectedThresholds, ts)

    A_ub = np.array([
        [0, 1, 4],
        [-3, 2, 0]
    ])
    b_ub = np.array([10, -7])
    opt_res = opt.linprog(-np.array(cost),
                          A_eq=np.array(eqs), b_eq=np.array(eqB),
                          A_ub=np.array(A_ub), b_ub=np.array(b_ub),
                          method="simplex")
    print("Expected cost", expectedCost)
    print("scipy.optimize.linprog gives a solution =", opt_res.x)
```

```python
testFromPost()
```

    Expected cost [1, 1, 1, 0, 0]
    scipy.optimize.linprog gives a solution = [2.  0.  2.5]

Un second test :

```python
def test2():
    cost = [1, 1, 1]
    lts = [[3, -2, 0]]
    ltB = [7]
    eqs = [[1, 1, 0]]
    eqB = [2]

    expectedCost = [1, 1, 1, 0]
    expectedConstraints = [[3, -2, 0, 1], [1 ,1 ,0 ,0]]
    expectedThresholds = [7, 2]
    test((expectedCost, expectedConstraints, expectedThresholds),
         standardForm(cost, lessThans=lts, ltThreshold=ltB, equalities=eqs, eqThreshold=eqB))
```

```python
test2()
```

Un dernier test :

```python
def test3():
    cost = [1, 1, 1]
    eqs = [[1, 1, 0], [2, 2, 2]]
    eqB = [2, 5]

    expectedCost = [1, 1, 1]
    expectedConstraints = [[3, -2, 0], [1, 1, 0]]
    expectedThresholds = [2, 5]
    test((expectedCost, expectedConstraints, expectedThresholds),
         standardForm(cost, equalities=eqs, eqThreshold=eqB))
```

```python
test3()
```

Ca suffit pour ces exemples.

## Conclusion

C'est bon pour aujourd'hui !


# Tutorial 1: Compiling arbitrary single-particle basis rotations in linear depth
This is the first of several tutorial explaining key concepts in OpenFermion-Cirq. These tutorials build on one another and should be studied in order. In this tutorial we will discuss the compilation of circuits for implementing arbitrary rotations of the single-particle basis of an electronic structure simulation. As an example, we show how one can use these methods to simulate the evolution of an arbitrary non-interacting fermion model.

## Background

### Second quantized fermionic operators

In order to represent fermionic systems on a quantum computer one must first discretize space. Usually, one expands the many-body wavefunction in a basis of spin-orbitals $\varphi_p = \varphi_p(r)$ which are single-particle basis functions. For reasons of spatial efficiency, all NISQ (and even most error-corrected) algorithms for simulating fermionic systems focus on representing operators in second-quantization. Second-quantized operators are expressed using the fermionic creation and annihilation operators, $a^\dagger_p$ and $a_p$. The action of $a^\dagger_p$ is to excite a fermion in spin-orbital $\varphi_p$ and the action of $a_p$ is to annihilate a fermion from spin-orbital $\varphi_p$. Specifically, if electron $i$ is represented in a space of spin-orbitals $\{\varphi_p(r_i)\}$ then $a^\dagger_p$ and $a_p$ are related to Slater determinants through the equivalence,
$$
\langle r_0 \cdots r_{\eta-1} | a^\dagger_{0} \cdots a^\dagger_{\eta-1} | \varnothing\rangle \equiv \sqrt{\frac{1}{\eta!}}
\begin{vmatrix}
\varphi_{0}\left(r_0\right) & \varphi_{1}\left( r_0\right) & \cdots & \varphi_{\eta-1} \left( r_0\right) \\
\varphi_{0}\left(r_1\right) & \varphi_{1}\left( r_1\right) & \cdots & \varphi_{\eta-1} \left( r_1\right) \\
\vdots & \vdots & \ddots & \vdots\\
\varphi_{0}\left(r_{\eta-1}\right) & \varphi_{1}\left(r_{\eta-1}\right) & \cdots & \varphi_{\eta-1} \left(r_{\eta-1}\right) \end{vmatrix}
$$
where $\eta$ is the number of electrons in the system, $|\varnothing \rangle$ is the Fermi vacuum and $\varphi_p(r)=\langle r|\varphi_p \rangle$ are the single-particle orbitals that define the basis. By using a basis of Slater determinants, we ensure antisymmetry in the encoded state.

### Rotations of the single-particle basis

Very often in electronic structure calculations one would like to rotate the single-particle basis. That is, one would like to generate new orbitals that are formed from a linear combination of the old orbitals. Any particle-conserving rotation of the single-particle basis can be expressed as
$$
\tilde{\varphi}_p = \sum_{q} \varphi_q u_{pq}
\quad
\tilde{a}^\dagger_p = \sum_{q} a^\dagger_q u_{pq}
\quad
\tilde{a}_p = \sum_{q} a_q u_{pq}^*
$$
where $\tilde{\varphi}_p$, $\tilde{a}^\dagger_p$, and $\tilde{a}^\dagger_p$ correspond to spin-orbitals and operators in the rotated basis and $u$ is an $N\times N$ unitary matrix. From the Thouless theorem, this single-particle rotation
is equivalent to applying the $2^N \times 2^N$ operator
$$
  U(u) = \exp\left(\sum_{pq} \left[\log u \right]_{pq} \left(a^\dagger_p a_q - a^\dagger_q a_p\right)\right)
$$
where $\left[\log u\right]_{pq}$ is the $(p, q)$ element of the matrix $\log u$.

There are many reasons that one might be interested in performing such basis rotations. For instance, one might be interested in preparing the Hartree-Fock (mean-field) state of a chemical system, by rotating from some initial orbitals (e.g. atomic orbitals or plane waves) into the molecular orbitals of the system. Alternatively, one might be interested in rotating from a basis where certain operators are diagonal (e.g. the kinetic operator is diagonal in the plane wave basis) to a basis where certain other operators are diagonal (e.g. the Coulomb operator is diagonal in the position basis). Thus, it is a very useful thing to be able to apply circuits corresponding to $U(u)$ on a quantum computer in low depth.

### Compiling linear depth circuits to rotate the orbital basis

OpenFermion-Cirq prominently features routines for implementing the linear depth / linear connectivity basis transformations described in [Phys. Rev. Lett. 120, 110501](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.120.110501). While we will not discuss this functionality here, we also support routines for compiling the more general form of these transformations which do not conserve particle-number, known as a Bogoliubov transfomation, using routines described in [Phys. Rev. Applied 9, 044036](https://journals.aps.org/prapplied/abstract/10.1103/PhysRevApplied.9.044036). We will not discuss the details of how these methods are implemented here and instead refer readers to those papers. All that one needs in order to compile the circuit $U(u)$ using OpenFermion-Cirq is the $N \times N$ matrix $u$, which we refer to in documentation as the "basis_transformation_matrix". Note that if one intends to apply this matrix to a computational basis state with only $\eta$ electrons, then one can reduce the number of gates required by instead supplying the $\eta \times N$ rectangular matrix that characterizes the rotation of the occupied orbitals only. OpenFermion-Cirq will automatically take advantage of this symmetry.

## OpenFermion-Cirq example implementation: exact evolution under tight binding models
In this example will show how basis transforms can be used to implement exact evolution under a random Hermitian one-body fermionic operator
\begin{equation}
H = \sum_{pq} T_{pq} a^\dagger_p a_q.
\end{equation}
That is, we will compile a circuit to implement $e^{-i H t}$ for some time $t$. Of course, this is a tractable problem classically but we discuss it here since it is often useful as a subroutine for more complex quantum simulations. To accomplish this evolution, we will use basis transformations. Suppose that $u$ is the basis transformation matrix that diagonalizes $T$. Then, we could implement $e^{-i H t}$ by implementing $U(u)^\dagger (\prod_{k} e^{-i \lambda_k Z_k}) U(u)$ where $\lambda_k$ are the eigenvalues of $T$.

Below, we initialize the T matrix characterizing $H$ and then obtain the eigenvalues $\lambda_k$ and eigenvectors $u_k$ of $T$. We print out the OpenFermion FermionOperator representation of $T$.

```python
import openfermion
import numpy

# Set the number of qubits in our example.
n_qubits = 3
simulation_time = 1.
random_seed = 8317

# Generate the random one-body operator.
T = openfermion.random_hermitian_matrix(n_qubits, seed=random_seed)

# Diagonalize T and obtain basis transformation matrix (aka "u").
eigenvalues, eigenvectors = numpy.linalg.eigh(T)
basis_transformation_matrix = eigenvectors.transpose()

# Print out familiar OpenFermion "FermionOperator" form of H.
H = openfermion.FermionOperator()
for p in range(n_qubits):
    for q in range(n_qubits):
        term = ((p, 1), (q, 0))
        H += openfermion.FermionOperator(term, T[p, q])
print(H)
```

    (0.5367212624097257+0j) [0^ 0] +
    (-0.26033703159240107+3.3259173741375454j) [0^ 1] +
    (1.3433603748462144+1.544987250567917j) [0^ 2] +
    (-0.26033703159240107-3.3259173741375454j) [1^ 0] +
    (-2.9143303700812435+0j) [1^ 1] +
    (-1.52843836446248+1.3527486791390022j) [1^ 2] +
    (1.3433603748462144-1.544987250567917j) [2^ 0] +
    (-1.52843836446248-1.3527486791390022j) [2^ 1] +
    (2.261633626116526+0j) [2^ 2]

Now we're ready to make a circuit! First we will use OpenFermion-Cirq to generate the basis transform $U(u)$ from the basis transformation matrix $u$ by calling the Bogoliubov transform function (named as such because this function can also handle non-particle conserving basis transformations). Then, we'll apply local $Z$ rotations to phase by the eigenvalues, then we'll apply the inverse transformation. That will finish the circuit. We're just going to print out the first rotation to keep things easy-to-read, but feel free to play around with the notebook.

```python
import openfermioncirq
import cirq

# Initialize the qubit register.
qubits = cirq.LineQubit.range(n_qubits)

# Start circuit with the inverse basis rotation, print out this step.
inverse_basis_rotation = cirq.inverse(openfermioncirq.bogoliubov_transform(qubits, basis_transformation_matrix))
circuit = cirq.Circuit(inverse_basis_rotation)
print(circuit)

# Add diagonal phase rotations to circuit.
for k, eigenvalue in enumerate(eigenvalues):
    phase = -eigenvalue * simulation_time
    circuit.append(cirq.Rz(rads=phase).on(qubits[k]))

# Finally, restore basis.
basis_rotation = openfermioncirq.bogoliubov_transform(qubits, basis_transformation_matrix)
circuit.append(basis_rotation)
```

    0: ────────────────────────────────────YXXY────────Rz(-0.0π)──────────────
                                           │
    1: ─────────────YXXY────────Z^0.522────#2^-0.656───YXXY────────Rz(0.0π)───
                    │                                  │
    2: ───Z^0.762───#2^-0.249───Z^-0.519───────────────#2^-0.479───Rz(-π)─────

Finally, we can check whether our circuit applied to a random initial state with the exact result. Print out the fidelity with the exact result.

```python
# Initialize a random initial state.
initial_state = openfermion.haar_random_vector(
    2 ** n_qubits, random_seed).astype(numpy.complex64)

# Numerically compute the correct circuit output.
import scipy
hamiltonian_sparse = openfermion.get_sparse_operator(H)
exact_state = scipy.sparse.linalg.expm_multiply(
    -1j * simulation_time * hamiltonian_sparse, initial_state)

# Use Cirq simulator to apply circuit.
simulator = cirq.Simulator()
result = simulator.simulate(circuit, qubit_order=qubits,
                            initial_state=initial_state)
simulated_state = result.final_state

# Print final fidelity.
fidelity = abs(numpy.dot(simulated_state, numpy.conjugate(exact_state)))**2
print(fidelity)
```

    1.0000000960717732

Thus, we see that the circuit correctly effects the intended evolution. We can now use Cirq's compiler to output the circuit using gates native to near-term devices, and then optimize those circuits. We'll output in QASM 2.0 just to demonstrate that functionality.

```python
xmon_circuit = cirq.google.optimized_for_xmon(circuit)
print(xmon_circuit.to_qasm())
```

    // Generated from Cirq v0.6.0

    OPENQASM 2.0;
    include "qelib1.inc";

    // Qubits: [0, 1, 2]
    qreg q[3];

    u2(pi*-0.3412628078, pi*0.3412628078) q[2];
    u2(pi*-0.5794122432, pi*0.5794122432) q[1];
    u2(pi*0.0897056074, pi*-0.0897056074) q[0];
    cz q[1],q[2];
    u3(pi*-0.1242949803, pi*0.6587371922, pi*-0.6587371922) q[2];
    u3(pi*-0.1242949803, pi*1.4205877568, pi*-1.4205877568) q[1];
    cz q[1],q[2];
    u3(pi*-0.3154309963, pi*0.1587371922, pi*-0.1587371922) q[2];
    u3(pi*-0.8528172267, pi*-0.0794122432, pi*0.0794122432) q[1];
    cz q[0],q[1];
    u3(pi*-0.328242091, pi*0.4205877568, pi*-0.4205877568) q[1];
    u3(pi*-0.328242091, pi*0.0897056074, pi*-0.0897056074) q[0];
    cz q[0],q[1];
    u3(pi*-0.9772799497, pi*0.9205877568, pi*-0.9205877568) q[1];
    u3(pi*-0.0380943193, pi*0.5897056074, pi*-0.5897056074) q[0];
    cz q[1],q[2];
    u3(pi*-0.2326621647, pi*-0.3412628078, pi*0.3412628078) q[2];
    u3(pi*-0.2326621647, pi*0.4205877568, pi*-0.4205877568) q[1];
    cz q[1],q[2];
    u3(pi*-0.336801226, pi*1.1587371922, pi*-1.1587371922) q[2];
    u3(pi*-0.7711683715, pi*0.9205877568, pi*-0.9205877568) q[1];
    cz q[0],q[1];
    u3(pi*-0.328242091, pi*0.4205877568, pi*-0.4205877568) q[1];
    u3(pi*-0.328242091, pi*0.0897056074, pi*-0.0897056074) q[0];
    cz q[0],q[1];
    u3(pi*-0.9123845954, pi*-0.0794122432, pi*0.0794122432) q[1];
    u2(pi*-0.9102943926, pi*0.9102943926) q[0];
    rz(pi*-0.1553859346) q[0];
    cz q[1],q[2];
    u3(pi*-0.1242949803, pi*-0.3412628078, pi*0.3412628078) q[2];
    u3(pi*-0.1242949803, pi*0.4205877568, pi*-0.4205877568) q[1];
    cz q[1],q[2];
    u2(pi*-1.3412628078, pi*1.3412628078) q[2];
    u2(pi*0.4205877568, pi*-0.4205877568) q[1];
    rz(pi*1.0794122432) q[1];
    rz(pi*-0.9205877568) q[2];


```python
%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import math
from numba import jit, njit, vectorize
```

##What is Numba?
Numba is a **just-in-time**, **type-specializing**, **function compiler** for accelerating **numerically-focused** Python. That's a long list, so let's break down those terms:

+ **function compiler**: Numba compiles Python functions, not entire applications, and not parts of functions. Numba does not replace your Python interpreter, but is just another Python module that can turn a function into a (usually) faster function.
+ **type-specializing**: Numba speeds up your function by generating a specialized implementation for the specific data types you are using. Python functions are designed to operate on generic data types, which makes them very flexible, but also very slow. In practice, you only will call a function with a small number of argument types, so Numba will generate a fast implementation for each set of types.
+ **just-in-time**: Numba translates functions when they are first called. This ensures the compiler knows what argument types you will be using. This also allows Numba to be used interactively in a Jupyter notebook just as easily as a traditional application.
+ **numerically-focused**: Currently, Numba is focused on numerical data types, like int, float, and complex. There is very limited string processing support, and many string use cases are not going to work well on the GPU. To get best results with Numba, you will likely be using NumPy arrays.

### Problem 1 - A First Numba Function

**1a)** To start our exploration of Numba's features, let's write a python function to add two numbers. We'll creatively name it `add`:

```python
def add(x, y):
# add code here
```

Now, test the function, first with two scalar integers:

```python
# add code here
```

**1b)** With Numpy, we can use our function to add not just scalars, but vectors as well. Using your favorite array creation routine, create two integer arrays with ten elements each, called `a` and `b`, and use your `add` function to add them.

```python
# add code here
```

Okay, so our function can add things. Now, let's use Numba's `jit` function to create a Numba version of our addition function:

```python
numba_add = jit(add)
```

More commonly, you will use `jit` as a decorator, by adding `@jit` to the line above your function definition, but the above version shows you that at heart, `@jit` is just a python function that takes other functions as its argument!

**1c)**  By default, a Numba function saves the original python version of the function in the variable `py_func`. Check that the original python version gives you the same answer as the Numba version.

```python
# add code here
```

#### Profiling
A central feature of parallel programming, Numba, and writing efficient code more generally is **profiling**, or understanding how long various pieces of your program take to run. Profiling tools are becoming ever more sophisticated, but for today we're going to stick with the tried-and-true method of timing things. An easy way to do this in python is using the `%timeit` magic function. Let's try it out on our addition function:

```python
%timeit add(1,2)
```

What's going on here? `%timeit` is running our function many times, and then reporting the average time it takes to run. This is generally a better approach than timing a single function execution, because it accounts for random events that may cause any given run to perform poorly.

**1d)** Compare the time it takes to run your function with scalar vs array arguments, then your function vs python's add function (the standard ''+'' operator).

```python
# add code here

```

So, scalars are faster than arrays (makes sense), and python's addition function is better than ours (seems reasonable). Now, let's see how fast our pre-compiled Numba addition function is.

```python
# add code here
```

Hold on - our new pre-compiled function is running even slower than the original python version! What's going on here?

### Problem 2 - A Better Numba Function

(This problem borrowed from [seibert's 2018 gtc numba tutorial](https://github.com/ContinuumIO/gtc2018-numba).)

As we saw in the first example, Numba isn't going to speed up everything. Generally, Numba will help you most in circumstances where python's line-by-line interperability and lack of type casting is slowing it down. We can use a slightly more complicated function to demonstrate this. The following is a function to calculate the hypotenuse of two numbers, that has been carefully designed to compensate for the computer's finite precision representation of numbers (check out https://en.wikipedia.org/wiki/Hypot for more info).

**2a)** Use the `@jit` decorator to generate a Numba version of this function.

```python
# add code here
def hypotenuse(x, y):
  x = abs(x);
  y = abs(y);
  t = min(x, y);
  x = max(x, y);
  t = t / x;
  return x * math.sqrt(1+t*t)
```

**2b)** Use the `%timeit` function to determine whether the Numba version of the hyptonenuse function is better than the original Python implementation.

```python
# add code here
```

**2c)** Numba functions can call other functions, provided they are also Numba functions. Below is a function that loops through two numpy arrays and puts their sum into an output array. Modify the following function to calculate the hypotenuse instead.

```python
@njit # this is an alias for @jit(nopython=True)
def ex_func(x, y, out):
  for i in range(x.shape[0]):
    out[i] = x[i] + y[i] # change this line
```

```python
in1 = np.arange(10, dtype=np.float64)
in2 = 2 * in1 + 1
out = np.empty_like(in1)

print('in1:', in1)
print('in2:', in2)

ex_func(in1, in2, out)

print('out:', out)
```

```python
# This test will fail until you fix the ex1 function
np.testing.assert_almost_equal(out, np.hypot(in1, in2))
```

### Problem 3 - Fun with Fractals
Now that we've got the basics of the Numba `jit` decorator down, let's have a little fun. A classic example problem in parallel programming is the calculation of a fractal, because a large fraction of the work can be done in parallel. Below is some code that calculates whether a number is a member of the Julia set, and then computes the set on a discrete domain to calculate a fractal.

**3a)** Modify the code below to use Numba and test how much faster it is than the original python implementation.

```python
# add code here
def julia(x, y, max_iters):
    """
    Given the real and imaginary parts of a complex number,
    determine if it is a candidate for membership in the Julia
    set given a fixed number of iterations.
    """
    i = 0
    c = complex(-0.8, 0.156)
    a = complex(x,y)
    for i in range(max_iters):
        a = a*a + c
        if (a.real*a.real + a.imag*a.imag) > 1000:
            return 0
    return 255
```

```python
# add code here
def create_fractal(min_x, max_x, min_y, max_y, image, iters):
    height = image.shape[0]
    width = image.shape[1]

    pixel_size_x = (max_x - min_x) / width
    pixel_size_y = (max_y - min_y) / height
    for x in range(width):
        real = min_x + x * pixel_size_x
        for y in range(height):
            imag = min_y + y * pixel_size_y
            color = julia(real, imag, iters)
            image[y, x] = color

    return image
```

```python
image = np.zeros((500, 750), dtype=np.uint8)
%timeit create_fractal(-2.0, 2.0, -1.0, 1.0, image, 200)
```

Want to see what you made? Run the following code to plot the image. Feel free to pick your favorite matplotlib color map :)

```python
plt.imshow(image)
plt.viridis()
plt.show()
```

**3b)** There is more than one type of fractal in the world, however! Below is a function that determines membership in the Mandelbrot set. Modify the function below using Numba, then modify the code above to produce a new pretty picture.

```python
# add code here
def mandel(x, y, max_iters):
    """
    Given the real and imaginary parts of a complex number,
    determine if it is a candidate for membership in the Mandelbrot
    set given a fixed number of iterations.
    """
    i = 0
    c = complex(x,y)
    z = 0.0j
    for i in range(max_iters):
        z = z*z + c
        if (z.real*z.real + z.imag*z.imag) >= 4:
            return i
    return 255
```

```python
# add code here
```

```python
image = np.zeros((500, 750), dtype=np.uint8)
create_fractal(-2.0, 2.0, -1.0, 1.0, image, 200)
plt.imshow(image)
plt.viridis()
plt.show()
```

### Problem 4 - Typing

Much of the power of Numba comes from its ability to compile a specific version of a python function based on the **data types** of its arguments. The data type describes what kind of variables the function uses, and in Numpy and Numba, pre-defined type names are based on the kind and size of the number in memory. You can find the type of a variable (or array) using numpy's `dtype` object. For example, let's see what type our `a` array is.

```python
a.dtype
```

This tells us the array contains integers, and each integer has been assigned 64bits in memory (or equivalently, 8 bytes). Most python functions are defined to work on arbitrary types, so that if you use the + operator, for example, you can add integers, floats, complex numbers, or even strings! However, this flexibility comes at a cost, performance-wise. Numba, on the other hand, compiles each function based on the types of its arguments, and infers the type of the result. You can see this if you run the `inspect_types` function on a numba function:

```python
numba_add.inspect_types()
```

**4a)** Numba has inferred the types for this function based on how we've used it. Try out your `numba_add` function with two floating point numbers, then re-inspect the types of the Numba function. Are they the same?

```python
#Add code here
```

So far we have been using what Numba refers to as "lazy" (or "call-time") decoration. Basically, we've been letting Numba do the work of figuring out how we're using the function and inferring the types for us. Alternatively, if we know how we are going to use a given function, we can use "eager" (or "compile-time") decoration. To do this, we make use of the `vectorize` decorator. For example, if we want to make an integer-only version of our addition function, we could write:

```python
@vectorize(['int64(int64, int64)'], target='cpu')
def add_ufunc(x, y):
  return x + y
```

#### Numpy Universal Functions
You'll notice a couple of new things here. In the first set of brackets, we have specified both the argument types of the function (those are inside the parentheses), as well as the return type of the function. This is just making explicit what Numba was previously inferring on our behalf. In second set of brackets you'll see that we have specified a 'target' architechture for the function. The default is `cpu`, which means that Numba is optimizing the function to your specific machine. Other options include `parallel`, which allows you to take advantage of multicore processors, and `cuda`, which we'll be discussing more tomorrow. You'll also notice that we called this a 'ufunc', which is short for [Universal Function](https://docs.scipy.org/doc/numpy/reference/ufuncs.html). In brief, universal functions are numpy functions that operate on ndarrays in element by element fashion. So if we pass two vectors to our `add_func`, they will be added together and a return vector of the same shape will be returned.

```python
%timeit add_ufunc(a,b)
```

**4b)** Try your ufunc out with a new target, 'parallel'. How does the speed compare? What if the array size is much larger?

```python
# add code here
```

### Problem 5 - Direct Summation
(This problem borrowed in its entirety from [gforsyth's scipy17 numba tutorial](https://github.com/gforsyth/numba_tutorial_scipy2017).)

Many physical problems require the evaluation of all pairwise interactions of a large number of particles, so-called N-body problems. These problems arise in molecular dynamics, astrodynamics and electromagnetics among others.

Their pairwise interactions can be expressed as:

\begin{equation}
f_i = \sum_{j=1}^n{P \left(\boldsymbol{x}_i, \boldsymbol{x}_j \right)w_j} \ \ \ \text{for } i=1,2,...,n
\end{equation}

*  where subscripts $i$,  $j$ respectively denote *target* and *source*
*  $f_i$ can be a *potential* (or *force*) at target point $i$
*  $w_j$ is the *source weight*
*  $\boldsymbol{x}_i, \boldsymbol{x}_j$ are the *spatial positions* of particles
*  $P \left(\boldsymbol{x}_i, \boldsymbol{x}_j \right)$ is the *interaction kernel*.

In order to evalute the potential $f_i$ at a target point $i$, we have to loop over each source particle $j$. Since there are $n$ target points $i$, this 'brute-force' approach costs $\mathcal{O} \left(n^2 \right)$ operations.

One possible approach in this kind of problem is to define a few classes, say `Point` and `Particle` and then loop over the objects and perform the necessary point-to-point calculations.

```python
class Point():
    """
    Arguments:
        domain: the domain of random generated coordinates x,y,z,
                default=1.0

    Attributes:
        x, y, z: coordinates of the point
    """
    def __init__(self, domain=1.0):
        self.x = domain * np.random.random()
        self.y = domain * np.random.random()
        self.z = domain * np.random.random()

    def distance(self, other):
        return ((self.x - other.x)**2 +
                (self.y - other.y)**2 +
                (self.z - other.z)**2)**.5
```

```python
class Particle(Point):
    """
    Attributes:
        m: mass of the particle
        phi: the potential of the particle
    """

    def __init__(self, domain=1.0, m=1.0):
        Point.__init__(self, domain)
        self.m = m
        self.phi = 0.
```

Next, we define a function to calculate the particle interaction via direct summation:

```python
def direct_sum(particles):
    """
    Calculate the potential at each particle
    using direct summation method.

    Arguments:
        particles: the list of particles

    """
    for i, target in enumerate(particles):
        for source in (particles[:i] + particles[i+1:]):
            r = target.distance(source)
            target.phi += source.m / r
```

All that's left is to create a list of random particles with assigned masses:

```python
n = 1000
particles = [Particle(m = 1 / n) for i in range(n)]
```

**5a)** Run the direct summation code and determine how long it takes with 10, 100, 1000 particles. Is there a relationship?

```python
# add code here
```

Add answer here.

**How do we use Numba on this problem?** There is a subtle issue here - Numba doesn't support jitting native Python classes.  There is a `jit_class` structure in Numba but it's still in early development. But we'd like to have attributes for readable programming. The solution is to build NumPy custom dtypes.

```python
particle_dtype = np.dtype({'names':['x','y','z','m','phi'],
                             'formats':[np.double,
                                        np.double,
                                        np.double,
                                        np.double,
                                        np.double]})
```

```python
myarray = np.ones(3, dtype=particle_dtype)
```

```python
myarray
```

You can access an individual "attribute" like this:

```python
myarray[0]['x'] = 2.0
```

**5b)** Write a `jit` function `create_n_random_particles` that takes the arguments `n` (number of particles), `m` (mass of every particle) and a domain within which to generate a random number (as in the class above).
It should create an array with `n` elements and `dtype=particle_dtype` and then return that array.

For each particle, the mass should be initialized to the value of `m` and the potential `phi` initialized to zero.

*Hint: You will probably want to loop over the number of particles within the function to assign attributes.*

```python
# add code here
```

Now we'll create our array of particles using the new function.

```python
particles = create_n_random_particles(1000, .001, 1)
```

```python
particles[:3]
```

We don't have a `distance` method anymore, so we need to write a function to take care of that.

**5c)** Write a `jit` function `distance` to calculate the distance between two particles of dtype `particle_dtype`.

```python
# add code here
```

```python
distance(particles[0], particles[1])
```

```python
%%timeit
distance(particles[0], particles[1])
```

**5d)** Modify the `direct_sum` function above to instead work on a NumPy array of particles.  Loop over each element in the array and calculate its total potential. Time the result and compare it to your previous version of this function.

```python
# add code here
```