###### Content provided under a Creative Commons Attribution license, CC-BY 4.0; code under MIT license. (c)2014 Lorena A. Barba, Olivier Mesnard. Thanks: NSF for support via CAREER award #1149784.

##### Version 0.4 -- April 2015

# Source panel method

We are now getting close to the finish line with *AeroPython*! Our first few lessons introduced the fundamental flow solutions of potential flow, and we quickly learned that using our superposition powers we could get some useful results in aerodynamics.

The superposition of a [doublet](03_Lesson03_doublet.ipynb) and a free stream gave the flow around a circular cylinder, and we learned about the *D'Alembert paradox*: the result of zero drag for potential flow around a cylinder. Adding a [vortex](06_Lesson06_vortexLift.ipynb) at the center of the cylinder, we learned about lift and the *Kutta-Joukowski theorem* stating that lift is proporional to circulation: $L=\rho U \Gamma$. A most important result!

Adding together fundamental solutions of potential flow and seeing what we get when interpreting a dividing streamline as a solid body is often called an *indirect method*. This method goes all the way back to Rankine in 1871! But its applicability is limited because we can't stipulate a geometry and find the flow associated to it.

In [Lesson 9](09_Lesson09_flowOverCylinder.ipynb), we learned that it is possible to stipulate first the geometry, and then solve for the source strengths on a panel discretization of the body that makes the flow tangent at the boundary. This is called a *direct method* and it took off in the 1960s with the work of Hess and Smith at Douglas Aircraft Company.

A set of panels (line segments in 2D) can represent the surface of any solid body immersed in a potential flow by making the source-sheet strengths such that the normal velocity at each panel is equal to zero. This is a very powerful idea! But you should realize that all the panel strengths are coupled to each other, which is why we end up with a linear system of equations.

For an arbitrary geometry, we need to build a set of panels according to some points that define  the geometry. In this lesson, we will read from a file a geometry definition corresponding to a **NACA0012 airfoil**, create a set of panels, and solve for the source-sheet strengths to get flow around the airfoil.

*Make sure you have studied [Lesson 9](09_Lesson09_flowOverCylinder.ipynb) carefully before proceeding!* We will not repeat the full mathematical formulation in this notebook, so refer back as needed.

First, load our favorite Python libraries, and the `integrate` module from SciPy:

```python
import os
import math
import numpy
from scipy import integrate
from matplotlib import pyplot
# display the figures in the Notebook
%matplotlib inline
```

Next, we read the body geometry from a file using the NumPy function [`loadtxt()`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html). The file comes from the [Airfoil Tools](http://airfoiltools.com/airfoil/details?airfoil=n0012-il) website and it contains a set of coordinates for the standard NACA0012 symmetric profile. We saved the file in the `resources` folder and load it from our local copy.

The geometry points get loaded into one NumPy array, so we separate the data into two arrays: `x,y` (for better code readability). The subsequent code will plot the geometry of the airfoil.

```python
# read of the geometry from a data file
naca_filepath = os.path.join('resources', 'naca0012.dat')
with open (naca_filepath, 'r') as file_name:
    x, y = numpy.loadtxt(file_name, dtype=float, delimiter='\t', unpack=True)

# plot the geometry
width = 10
pyplot.figure(figsize=(width, width))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.plot(x, y, color='k', linestyle='-', linewidth=2)
pyplot.axis('scaled', adjustable='box')
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(-0.1, 0.1);
```

## Discretization into panels

Like in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb), we will create a discretization of the body geometry into panels (line segments in 2D). A panel's attributes are: its starting point, end point and mid-point, its length and its orientation. See the following figure for the nomenclature used in the code and equations below.

We can modify the `Panel` class from our previous notebook slightly, to work better for our study of flow over an airfoil. The only difference is that we identify points on the top or bottom surfaces with the words `upper` and `lower`, which is only used later for plotting results with different colors for the top and bottom surfaces of the profile.

```python
class Panel:
    """
    Contains information related to a panel.
    """
    def __init__(self, xa, ya, xb, yb):
        """
        Initializes the panel.

        Sets the end-points and calculates the center, length,
        and angle (with the x-axis) of the panel.
        Defines if the panel is on the lower or upper surface of the geometry.
        Initializes the source-sheet strength, tangential velocity,
        and pressure coefficient to zero.

        Parameters
        ----------
        xa: float
            x-coordinate of the first end-point.
        ya: float
            y-coordinate of the first end-point.
        xb: float
            x-coordinate of the second end-point.
        yb: float
            y-coordinate of the second end-point.
        """
        self.xa, self.ya = xa, ya
        self.xb, self.yb = xb, yb

        self.xc, self.yc = (xa+xb)/2, (ya+yb)/2       # control-point (center-point)
        self.length = math.sqrt((xb-xa)**2+(yb-ya)**2)     # length of the panel

        # orientation of the panel (angle between x-axis and panel's normal)
        if xb-xa <= 0.:
            self.beta = math.acos((yb-ya)/self.length)
        elif xb-xa > 0.:
            self.beta = math.pi + math.acos(-(yb-ya)/self.length)

        # location of the panel
        if self.beta <= math.pi:
            self.loc = 'upper'
        else:
            self.loc = 'lower'

        self.sigma = 0.                             # source strength
        self.vt = 0.                                # tangential velocity
        self.cp = 0.                                # pressure coefficient
```

For the circular cylinder, the discretization into panels was really easy. This is the part that gets more complicated when you want to compute the flow around a general geometry, while the solution part is effectively the same as in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb).

The function below will create the panels from the geometry data that was read from a file. It is better to have small panels near the leading-edge and the trailing edge, where the curvature is large. One method to get a non uniform distribution around the airfoil is to first discretize a circle with diameter equal to the airfoil's chord, with the leading edge and trailing edge touching the circle at a node, as shown in the following sketch.

Then, we store the $x$-coordinates of the circle points, `x_circle`, which will also be the $x$-coordinates of the panel nodes, `x`, and project the $y$-coordinates of the circle points onto the airfoil by interpolation. We end up with a node distribution on the airfoil that is refined near the leading edge and the trailing edge. It will look like this:

With the discretization method just described, the function `define_panels()` returns an array of objects, each an instance of the class `Panel` and containing all information about a panel, given the desired number of panels and the set of body coordinates.

A few remarks about the implementation of the function `define_panels()`:

* we just need to compute the $x$-coordinates of the circle (`x_circle`) since the $y$-coordinates of the panel nodes will be computed by interpolation;
* we create a circle with `N+1` points, but the first and last points coincide;
* we extend our NumPy arrays by adding an extra value that is equal to the first one; thus we don't have to do anything special with the value `x[i+1]` in the different loops;
* the *while*-loop is used to find two consecutive points, (`x[I]`,`y[I]`) and (`x[I+1]`,`y[I+1]`), on the foil such that the interval [`x[I]`,`x[I+1]`] contains the value `x_ends[i]`; we use the keyword `break` to get out of the loop;
* once the two points have been identified, the value `y_ends[i]` is computed by interpolation.

```python
def define_panels(x, y, N=40):
    """
    Discretizes the geometry into panels using the 'cosine' method.

    Parameters
    ----------
    x: 1D array of floats
        x-coordinate of the points defining the geometry.
    y: 1D array of floats
        y-coordinate of the points defining the geometry.
    N: integer, optional
        Number of panels;
        default: 40.

    Returns
    -------
    panels: 1D Numpy array of Panel objects
        The discretization of the geometry into panels.
    """
    R = (x.max()-x.min())/2                                    # radius of the circle
    x_center = (x.max()+x.min())/2                             # x-coord of the center
    x_circle = x_center + R*numpy.cos(numpy.linspace(0, 2*math.pi, N+1))  # x-coord of the circle points

    x_ends = numpy.copy(x_circle)      # projection of the x-coord on the surface
    y_ends = numpy.empty_like(x_ends)  # initialization of the y-coord Numpy array

    x, y = numpy.append(x, x[0]), numpy.append(y, y[0])    # extend arrays using numpy.append

    # computes the y-coordinate of end-points
    I = 0
    for i in range(N):
        while I < len(x)-1:
            if (x[I] <= x_ends[i] <= x[I+1]) or (x[I+1] <= x_ends[i] <= x[I]):
                break
            else:
                I += 1
        a = (y[I+1]-y[I])/(x[I+1]-x[I])
        b = y[I+1] - a*x[I+1]
        y_ends[i] = a*x_ends[i] + b
    y_ends[N] = y_ends[0]

    panels = numpy.empty(N, dtype=object)
    for i in range(N):
        panels[i] = Panel(x_ends[i], y_ends[i], x_ends[i+1], y_ends[i+1])

    return panels
```

Now we can use this function, calling it with a desired number of panels whenever we execute the cell below. We also plot the resulting geometry.

```python
N = 40                            # number of panels
panels = define_panels(x, y, N)   # discretizes of the geometry into panels

# plot the geometry and the panels
width = 10
pyplot.figure(figsize=(width, width))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.plot(x, y, color='k', linestyle='-', linewidth=2)
pyplot.plot(numpy.append([panel.xa for panel in panels], panels[0].xa),
            numpy.append([panel.ya for panel in panels], panels[0].ya),
            linestyle='-', linewidth=1, marker='o', markersize=6, color='#CD2305')
pyplot.axis('scaled', adjustable='box')
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(-0.1, 0.1);
```

## Freestream conditions

The NACA0012 airfoil will be immersed in a uniform flow with velocity $U_\infty$ and an angle of attack $\alpha=0$. Even though it may seem like overkill to create a class for the freestream, we'll do it anyway. When creating a class, one is expecting to also create several intances of its objects. Here, we just have one freestream, so why define a class? Well, it makes the code more readable and does not block the programer from using the variable names `u_inf` and `alpha` for something else outside of the class.
Also, every time we need the freestream condition as input to a function, we will just have to pass the object as an argument and not all the attributes of the freestream.

```python
class Freestream:
    """
    Freestream conditions.
    """
    def __init__(self, u_inf=1.0, alpha=0.0):
        """
        Sets the freestream speed and angle (with the x-axis).

        Parameters
        ----------
        u_inf: float, optional
            Freestream speed;
            default: 1.0.
        alpha: float, optional
            Angle of attack in degrees;
            default: 0.0.
        """
        self.u_inf = u_inf
        self.alpha = alpha*math.pi/180          # degrees --> radians
```

```python
# define and creates the object freestream
u_inf = 1.0                                # freestream spee
alpha = 0.0                                # angle of attack (in degrees)
freestream = Freestream(u_inf, alpha)      # instantiation of the object freestream
```

## Flow tangency boundary condition

Enforcing the flow-tangency condition on each *control point* approximately makes the body geometry correspond to a dividing streamline (and the approximation improves if we represented the body with more and more panels). So, for each panel $i$, we make $u_n=0$ at $(x_{c_i},y_{c_i})$, which leads to the equation derived in the previous lesson:

\begin{equation}
u_{n_i} = \frac{\partial}{\partial n_i}\left\lbrace \phi\left(x_{c_i},y_{c_i}\right) \right\rbrace = 0
\end{equation}

i.e.

\begin{equation}
0 = U_\infty \cos\beta_i + \frac{\sigma_i}{2} + \sum_{j=1,j\neq i}^{N_p} \frac{\sigma_j}{2\pi} \int \frac{
\left(x_{c_i}-x_j(s_j)\right) \cos\beta_i
+ \left(y_{c_i}-y_j(s_j)\right) \sin\beta_i
}
{\left(x_{c_i}-x_j(s)\right)^2 + \left(y_{c_i}-y_j(s)\right)^2} {\rm d}s_j
\end{equation}

In the equation above, we calculate the derivative of the potential in the normal direction to enforce the flow tangency condition on each panel. But later, we will have to calculate the derivative in the tangential direction to compute the surface pressure coefficient. And, when we are interested in plotting the velocity field onto a mesh, we will have to calculate the derivative in the $x$- and $y$-direction.

Therefore the function below is similar to the one implemented in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb) to obtain the integrals along each panel, but we've generalized it to adapt to the direction of derivation (by means of two new arguments, `dxdz` and `dydz`, which respectively represent the value of $\frac{\partial x_{c_i}}{\partial z_i}$ and $\frac{\partial y_{c_i}}{\partial z_i}$, $z_i$ being the desired direction).

Moreover, the function is also more general in the sense of allowing any evaluation point, not just a control point on a panel (the argument `p_i` has been replaced by the coordinates `x` and `y` of the control-point, and `p_j` has been replaced with `panel`).

```python
def integral(x, y, panel, dxdz, dydz):
    """
    Evaluates the contribution of a panel at one point.

    Parameters
    ----------
    x: float
        x-coordinate of the target point.
    y: float
        y-coordinate of the target point.
    panel: Panel object
        Source panel which contribution is evaluated.
    dxdz: float
        Derivative of x in the z-direction.
    dydz: float
        Derivative of y in the z-direction.

    Returns
    -------
    Integral over the panel of the influence at the given target point.
    """
    def integrand(s):
        return ( ((x - (panel.xa - math.sin(panel.beta)*s))*dxdz
                  +(y - (panel.ya + math.cos(panel.beta)*s))*dydz)
                / ((x - (panel.xa - math.sin(panel.beta)*s))**2
                   +(y - (panel.ya + math.cos(panel.beta)*s))**2) )
    return integrate.quad(integrand, 0.0, panel.length)[0]
```

## Building the linear system

Here, we build and solve the linear system of equations of the form

\begin{equation}[A][\sigma] = [b].\end{equation}

In building the matrix, below, we call the `integral()` function with the correct values for the last parameters: $\cos \beta_i$ and $\sin\beta_i$, corresponding to a derivative in the normal direction.

Finally, we use `linalg.solve()` from NumPy to solve the system and find the strength of each panel.

```python
def build_matrix(panels):
    """
    Builds the source matrix.

    Parameters
    ----------
    panels: 1D array of Panel object
        The source panels.

    Returns
    -------
    A: 2D Numpy array of floats
        The source matrix (NxN matrix; N is the number of panels).
    """
    N = len(panels)
    A = numpy.empty((N, N), dtype=float)
    numpy.fill_diagonal(A, 0.5)

    for i, p_i in enumerate(panels):
        for j, p_j in enumerate(panels):
            if i != j:
                A[i,j] = 0.5/math.pi*integral(p_i.xc, p_i.yc, p_j, math.cos(p_i.beta), math.sin(p_i.beta))

    return A

def build_rhs(panels, freestream):
    """
    Builds the RHS of the linear system.

    Parameters
    ----------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.

    Returns
    -------
    b: 1D Numpy array of floats
        RHS of the linear system.
    """
    b = numpy.empty(len(panels), dtype=float)

    for i, panel in enumerate(panels):
        b[i] = -freestream.u_inf * math.cos(freestream.alpha - panel.beta)

    return b
```

```python
A = build_matrix(panels)                    # compute the singularity matrix
b = build_rhs(panels, freestream)           # compute the freestream RHS
```

```python
# solve the linear system
sigma = numpy.linalg.solve(A, b)

for i, panel in enumerate(panels):
    panel.sigma = sigma[i]
```

## Surface pressure coefficient

From Bernoulli's equation, the pressure coefficient on the $i$-th panel is

\begin{equation}C_{p_i} = 1-\left(\frac{u_{t_i}}{U_\infty}\right)^2
\end{equation}

where $u_{t_i}$ is the tangential component of the velocity at the center point of the $i$-th panel,

\begin{equation}
u_{t_i} = -U_\infty \sin\beta_i + \sum_{j=1}^{N_p} \frac{\sigma_j}{2\pi} \int \frac{
\left(x_{c_i}-x_j(s_j)\right) \frac{\partial x_{c_i}}{\partial t_i}
+ \left(y_{c_i}-y_j(s_j)\right) \frac{\partial y_{c_i}}{\partial t_i}
}
{\left(x_{c_i}-x_j(s)\right)^2 + \left(y_{c_i}-y_j(s)\right)^2} {\rm d}s_j
\end{equation}

with

\begin{equation}
\frac{\partial x_{c_i}}{\partial t_i} = -\sin\beta_i \quad\text{and} \quad \frac{\partial y_{c_i}}{\partial t_i} = \cos\beta_i
\end{equation}

Notice that below we call the function `integral()` with different arguments: $-\sin\beta_i$ and $\cos\beta_i$ to get the derivation in the tangential direction.

```python
def get_tangential_velocity(panels, freestream):
    """
    Computes the tangential velocity on the surface of the panels.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    """
    N = len(panels)
    A = numpy.empty((N, N), dtype=float)
    numpy.fill_diagonal(A, 0.0)

    for i, p_i in enumerate(panels):
        for j, p_j in enumerate(panels):
            if i != j:
                A[i,j] = 0.5/math.pi*integral(p_i.xc, p_i.yc, p_j, -math.sin(p_i.beta), math.cos(p_i.beta))

    b = freestream.u_inf * numpy.sin([freestream.alpha - panel.beta for panel in panels])

    sigma = numpy.array([panel.sigma for panel in panels])

    vt = numpy.dot(A, sigma) + b

    for i, panel in enumerate(panels):
        panel.vt = vt[i]
```

```python
# compute the tangential velocity at the center-point of each panel
get_tangential_velocity(panels, freestream)
```

```python
def get_pressure_coefficient(panels, freestream):
    """
    Computes the surface pressure coefficients on the panels.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    """
    for panel in panels:
        panel.cp = 1.0 - (panel.vt/freestream.u_inf)**2
```

```python
# computes the surface pressure coefficients
get_pressure_coefficient(panels, freestream)
```

### Theoretical solution

There is a classical method to obtain the theoretical characteristics of airfoils, known as *Theodorsen's method*. It uses the Joukowski transformation but is able to deal with any airfoil by an additional transformation between a "near circle" and a circle. The method is hairy indeed! But the resulting values of pressure coefficient are provided for some airfoils in table form in the 1945 [NACA Report No.824](http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930090976.pdf), available from the NASA web server (see p. 71).

The values of $(u/U_{\infty})^2$ are given for several stations along the chord length. We transcribed them here, saving them into an array:

```python
voverVsquared=numpy.array([0, 0.64, 1.01, 1.241, 1.378, 1.402, 1.411, 1.411, 1.399, 1.378, 1.35, 1.288, 1.228, 1.166, 1.109, 1.044, 0.956, 0.906, 0])
print(voverVsquared)
```

    [ 0.     0.64   1.01   1.241  1.378  1.402  1.411  1.411  1.399  1.378
      1.35   1.288  1.228  1.166  1.109  1.044  0.956  0.906  0.   ]

```python
xtheo=numpy.array([0, 0.5, 1.25, 2.5, 5.0, 7.5, 10, 15, 20, 25, 30, 40, 50, 60, 70, 80, 90, 95, 100])
xtheo = xtheo/100
print(xtheo)
```

    [ 0.      0.005   0.0125  0.025   0.05    0.075   0.1     0.15    0.2     0.25
      0.3     0.4     0.5     0.6     0.7     0.8     0.9     0.95    1.    ]

### And plot the result!

We will use the values from the NACA report (also given in the book by Abbot and von Doenhoff, ["Theory of Wing Sections,"](http://books.google.com/books/about/Theory_of_Wing_Sections_Including_a_Summ.html?id=DPZYUGNyuboC) 1949) to visually compare the pressure distribution with the result of our source panel method. Let's see how it looks!

```python
# plot the surface pressure coefficient
pyplot.figure(figsize=(10, 6))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('$C_p$', fontsize=16)
pyplot.plot([panel.xc for panel in panels if panel.loc == 'upper'],
            [panel.cp for panel in panels if panel.loc == 'upper'],
            label='upper',
            color='r', linewidth=1, marker='x', markersize=8)
pyplot.plot([panel.xc for panel in panels if panel.loc == 'lower'],
            [panel.cp for panel in panels if panel.loc == 'lower'],
            label='lower',
            color='b', linewidth=0, marker='d', markersize=6)
pyplot.plot(xtheo, 1-voverVsquared,
            label='theoretical',
            color='k', linestyle='--',linewidth=2)
pyplot.legend(loc='best', prop={'size':14})
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(1.0, -0.6)
pyplot.title('Number of panels : %d' % N);
```

That looks pretty good! The only place where the panel method doesn't quite match the tabulated data from Theordorsen's method is at the trailing edge. But note that the flow-tangency boundary condition in the panel method is applied at the control point of the panel (not at the endpoints), so this discrepancy is not surprising.

##### Accuracy check

For a closed body, the sum of all the source strengths must be zero. If not, it means the body would be adding or absorbing mass from the flow! Therfore, we should have

$$\sum_{j=1}^{N} \sigma_j l_j = 0$$

where $l_j$ is the length of the $j^{\text{th}}$ panel.

With this, we can get a get an idea of the accuracy of the source panel method.

```python
# calculate the accuracy
accuracy = sum([panel.sigma*panel.length for panel in panels])
print('--> sum of source/sink strengths: {}'.format(accuracy))
```

    --> sum of source/sink strengths: 0.004617031175283089

## Streamlines onto a mesh grid

To get a streamline plot, we have to create a mesh (like we've done in all *AeroPython* lessons!) and compute the velocity field onto it. Knowing the strength of every panel, we find the $x$-component of the velocity by taking derivative of the velocity potential in the $x$-direction, and the $y$-component by taking derivative in the $y$-direction:

$$u\left(x,y\right) = \frac{\partial}{\partial x}\left\lbrace \phi\left(x,y\right) \right\rbrace$$

$$v\left(x,y\right) = \frac{\partial}{\partial y}\left\lbrace \phi\left(x,y\right) \right\rbrace$$

Notice that here we call the funcion `integral()` with $1,0$ as the final arguments when calculating the derivatives in the $x$-direction, and $0,1$ for the derivatives in th $y$-direction.

In addition, we use the function `numpy.vectorize()` (as we did in [Lesson 8](08_Lesson08_sourceSheet.ipynb)) to avoid the nested loops over the domain.

```python
def get_velocity_field(panels, freestream, X, Y):
    """
    Computes the velocity field on a given 2D mesh.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    X: 2D Numpy array of floats
        x-coordinates of the mesh points.
    Y: 2D Numpy array of floats
        y-coordinate of the mesh points.

    Returns
    -------
    u: 2D Numpy array of floats
        x-component of the velocity vector field.
    v: 2D Numpy array of floats
        y-component of the velocity vector field.
    """
    # freestream contribution
    u = freestream.u_inf * math.cos(freestream.alpha) * numpy.ones_like(X, dtype=float)
    v = freestream.u_inf * math.sin(freestream.alpha) * numpy.ones_like(X, dtype=float)
    # add the contribution from each source (superposition powers!!!)
    vec_intregral = numpy.vectorize(integral)
    for panel in panels:
        u += panel.sigma / (2.0 * math.pi) * vec_intregral(X, Y, panel, 1, 0)
        v += panel.sigma / (2.0 * math.pi) * vec_intregral(X, Y, panel, 0, 1)

    return u, v
```

```python
# define a mesh grid
nx, ny = 20, 20  # number of points in the x and y directions
x_start, x_end = -1.0, 2.0
y_start, y_end = -0.3, 0.3
X, Y = numpy.meshgrid(numpy.linspace(x_start, x_end, nx),
                      numpy.linspace(y_start, y_end, ny))

# compute the velocity field on the mesh grid
u, v = get_velocity_field(panels, freestream, X, Y)
```

```python
# plot the velocity field
width = 10
pyplot.figure(figsize=(width, width))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.streamplot(X, Y, u, v,
                  density=1, linewidth=1, arrowsize=1, arrowstyle='->')
pyplot.fill([panel.xc for panel in panels],
            [panel.yc for panel in panels],
            color='k', linestyle='solid', linewidth=2, zorder=2)
pyplot.axis('scaled', adjustable='box')
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.title('Streamlines around a NACA 0012 airfoil (AoA = ${}^o$)'.format(alpha), fontsize=16);
```

We can now calculate the pressure coefficient. In Lesson 9, we computed the pressure coefficient on the surface of the circular cylinder. That was useful because we have an analytical solution for the surface pressure on a cylinder in potential flow. For an airfoil, we are interested to see how the pressure looks all around it, and we make a contour plot in the flow domain.

```python
# compute the pressure field
cp = 1.0 - (u**2+v**2)/freestream.u_inf**2

# plot the pressure field
width = 10
pyplot.figure(figsize=(width, width))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
contf = pyplot.contourf(X, Y, cp,
                        levels=numpy.linspace(-2.0, 1.0, 100), extend='both')
cbar = pyplot.colorbar(contf,
                       orientation='horizontal',
                       shrink=0.5, pad = 0.1,
                       ticks=[-2.0, -1.0, 0.0, 1.0])
cbar.set_label('$C_p$', fontsize=16)
pyplot.fill([panel.xc for panel in panels],
            [panel.yc for panel in panels],
            color='k', linestyle='solid', linewidth=2, zorder=2)
pyplot.axis('scaled', adjustable='box')
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.title('Contour of pressure field', fontsize=16);
```

### Final words

We've learned to use a source-sheet to represent any solid body: first a [circular cylinder](09_Lesson09_flowOverCylinder.ipynb) (which we knew we could get by superposing a doublet and a freestream), and now an airfoil.

But what is the feature of airfoils that makes them interesting? Well, the fact that we can use them to generate lift and make things that fly, of course! But what do we need to generate lift? Think, think ... what is it?

## References

1. [Airfoil Tools](http://airfoiltools.com/index), website providing airfoil data.
1. Ira Herbert Abbott, Albert Edward Von Doenhoff and Louis S. Stivers, Jr. (1945), "Summary of Airfoil Data," NACA Report No.824, [PDF on the NASA web server](http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930090976.pdf) (see p. 71)
1. Ira Herbert Abbott, Albert Edward Von Doenhoff, "Theory of Wing Sections, Including a Summary of Airfoil Data" (1949), Dover Press.

A further reference on Theodorsen's method is:

* Roland Schinzinger, Patricio A. A. Laura (1991), "Conformal Mapping: Methods and Applications." Dover edition in 2003. [Read on Google Books](https://books.google.com/books?id=qe-7AQAAQBAJ&lpg=PA128&ots=wbg0jLlqq5&dq=method%20theodorsen&pg=PA128#v=onepage&q=%22method%20of%20theodorsen%20and%20garrick%22&f=false)

---

###### Please ignore the cell below. It just loads our style for the notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('../styles/custom.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Alegreya Sans', sans-serif;
    font-style:regular;
    font-weight: 200;
    font-size: 50pt;
    line-height: 100%;
    color:#CD2305;
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Fenix', serif;
    font-size: 22pt;
    line-height: 100%;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Fenix', serif;
    margin-top:12px;
	font-size: 16pt;
    margin-bottom: 3px;
    font-style: regular;
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Fenix', serif;
    font-size: 2pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: #CD2305;
    font-style: italic;
    margin-bottom: .5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'Source Code Pro', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


Floating point errors in the solution of the quadratic equation
===============================================================
In this notebook I attempt to derive formula for the errors in
the solution of the quadratic equation $A x^2 + B x + C = 0$.
I respect two kind of errors: Propagated uncertainty in $A, B,$
and $C$. And roundoff errors in the solution formula due to
floating point computations.

```
import scipy
import numpy as np
import sympy
#from sympy import *
from IPython.display import display, Math, Latex
def prettylatex(x):
    display(Math(sympy.latex(x)))
import sys
import os
import interval
import math
```

```
def itostring(i):
    assert len(i) == 1
    return u'{}+/-{} ⊂ {}'.format(i.midpoint[0].inf,0.5*(i[0].sup-i[0].inf),i)

def eps_half_interval_around(x, eps):
    return interval.interval([x*(1-0.5*eps), x*(1+0.5*eps)])
```

```
def gamma_t(dtype):
    # Get machine epsilon. (Isn't the machine epsilon actually half of that? ... Well not that it's a big deal ...)
    eps = np.finfo(np.float16).eps.astype(np.float64)
    # Then make the gamma function as defined in PBRT pg.216
    def gamma(n):
        return n*eps/(1-n*eps)
    return gamma

def typedlike(x, t):
    """ return x cast to same type as t. t.dtype must give the numpy type."""
    return np.array(x, dtype=t.dtype)
```

```
def errorformula1(A, B, C, rA, rB, rC):
    xi = typedlike(1 if B < 0 else -1,A)
    one = typedlike(1,A)
    four = typedlike(4, A)
    two  = typedlike(2, A)
    three = typedlike(3, A)
    eps  = np.finfo(A.dtype).eps
    Q = B*B - A*C
    sQ = np.sqrt(Q)
    E1 = eps*(np.abs((B - sQ*xi)/A) + np.abs(sQ/A/two) + np.abs(C/sQ) + np.abs(B*B/four/A/sQ) + np.abs(sQ/A/four))
    E2 = rA*np.abs(A*(xi*C/A/sQ + (B-xi*sQ)/(two*A*A))) + rB/two*np.abs(B/A*(B*xi/sQ-one)) + rC*np.abs(C/sQ)
    return E1+E2

def errorformula2(A, B, C, rA, rB, rC):
    xi = typedlike(1 if B < 0 else -1,A)
    one = typedlike(1,A)
    four = typedlike(4, A)
    two  = typedlike(2, A)
    three = typedlike(3, A)
    eps  = np.finfo(A.dtype).eps
    Q = B*B - A*C
    sQ = np.sqrt(Q)
    E1_den1 = B - xi*sQ
    E1_den2 = E1_den1*E1_den1
    E1 = eps*(four*np.abs(C/E1_den1) + two*np.abs(C*sQ/E1_den2) + np.abs(four*A*C*C/sQ/E1_den2) + np.abs(B*B*C/sQ/E1_den2) + np.abs(C*sQ/E1_den2))
    E2 = four*rA*np.abs(A*C*C/sQ/E1_den2) + two*rB*np.abs((B*C*(B*xi-sQ))/sQ/E1_den2) + rC*np.abs(C*(four*A*C*xi/sQ/E1_den2 - two/E1_den1))
    return E1+E2
```

```
sqrt = np.sqrt

def quadratic(A, B, C):
    Q = B*B - typedlike(4, A)*A*C
    sQ = sqrt(Q)
    sol1 = (-B + sQ)/(typedlike(2, A)*A)
    sol2 = (-B - sQ)/(typedlike(2, A)*A)
    return sol1,sol2

def better_quadratic(A, B, C, rA, rB, rC):
    # from PBRT pg. 1080
    D = B*B - typedlike(4, A)*A*C
    sD = sqrt(D)
    Q = -B+sD if B<0 else -B-sD
    t0 = Q/2/A
    t1 = 2*C/Q
    e0 = errorformula1(A, B, C, rA, rB, rC)
    e1 = errorformula2(A, B, C, rA, rB, rC)
    if t0<t1:
        t0, t1, e0, e1 = t1, t0, e1, e0
    return t0, t1, e0, e1

# TODO: implement improved solution precision method
#def quadratic_interval(A, B, C):
#     Q = B*B - 4*A*C
#     sQ = interval.imath.sqrt(Q)
#     sol1 = (-B + sQ)/(2*A)
#     sol2 = (-B - sQ)/(2*A)
    # return sol1, sol2
```

```
def print_quadratic_solution_and_float_errors(A, B, C, rA, rB, rC):
    print "A={}, B={}, C={}".format(A, B, C)
    print "========================"
    eps = float(np.finfo(np.float16).eps)
    #print "Float16 Eps: ", eps
    print "Float64 solution:"
    A64 = np.float64(A)
    B64 = np.float64(B)
    C64 = np.float64(C)
    A16, B16, C16 = np.float16(A),np.float16(B),np.float16(C)
    t0_64,t1_64 = quadratic(A64,B64,C64)
    print t0_64, t1_64
    print "Residual:"
    print (A*t0_64*t0_64 + B*t0_64 + C), (A*t1_64*t1_64 + B*t1_64 + C)
    print "---------------------"
    print "Float16 solution:"
    t0_16, t1_16 = quadratic(A16, B16, C16)
    print t0_16, t1_16
    print "Residual:"
    print (A*t0_16*t0_16 + B*t0_16 + C), (A*t1_16*t1_16 + B*t1_16 + C)
    print "---------------------"
    print "Improved Float16 Solution"
    t0_better_16, t1_better_16, e0_better_16, e1_better_16 = better_quadratic(A16, B16, C16, np.float16(eps*rA), np.float16(eps*rB), np.float16(eps*rC))
    print t0_better_16, t1_better_16
    print "Residual:"
    print (A*t0_better_16*t0_better_16 + B*t0_better_16 + C), (A*t1_better_16*t1_better_16 + B*t1_better_16 + C)
    print "---------------------"
    print "Error Estimates"
    rA = rA*eps
    rB = rB*eps
    rC = rC*eps
#     isol1, isol2 = quadratic_interval(eps_half_interval_around(A, rA), eps_half_interval_around(B, rB), eps_half_interval_around(C, rC))
#     def fmt(i, x):
#         return "[{:0.5f}, {:0.5f}]".format(i[0].inf-x, i[0].sup-x)
#     print "Solution 1 bounds around true solution:", fmt(isol1, t0_64)
#     print "Solution 2 bounds around true solution:", fmt(isol2, t1_64)
    assert type(e0_better_16) == np.float16
    assert type(e1_better_16) == np.float16
    print "Analytic bound of solution 1: +/-", e0_better_16
    print "Analytic bound of solution 2: +/-", e1_better_16
    print "Actual errors of improved solution:", (t0_better_16-t0_64), (t1_better_16-t1_64)
    print ""
```

```
print_quadratic_solution_and_float_errors(1, 100, 1, 1, 1, 1)
#print_quadratic_solution_and_float_errors(1, -100, 1, 1, 1, 1)
#print_quadratic_solution_and_float_errors(3, 210, 3002 , 1, 1, 1)
#print_quadratic_solution_and_float_errors(3, -210, 3002 , 1, 1, 1)
#print_quadratic_solution_and_float_errors(1, -100, 0.001 , 1, 1, 1)
#print_quadratic_solution_and_float_errors(3, 100, 3, 100, 0, 0)
#print_quadratic_solution_and_float_errors(3, 100, 3, 0, 100, 0)
#print_quadratic_solution_and_float_errors(3, 100, 3, 0, 0, 100)
#print_quadratic_solution_and_float_errors(3, 100, 3*(1+np.finfo(np.float16).eps*100), 0, 0, 100)
```

    A=1, B=100, C=1
    ========================
    Float64 solution:
    -0.010001000200048793 -99.98999899979995
    Residual:
    1.2212453270876722e-13 0.0
    ---------------------
    Float16 solution:
    0.0 -100.0
    Residual:
    1.0 1.0
    ---------------------
    Improved Float16 Solution
    -0.01 -100.0
    Residual:
    9.999999999998899e-05 1.0
    ---------------------
    Error Estimates
    Analytic bound of solution 1: +/- 4.89e-05
    Analytic bound of solution 2: +/- 0.4883
    Actual errors of improved solution: 1.0002000487927681e-06 -0.010001000200048793

```
A, B, C, rA, rB, rC, Q_, e1, e2, e3, e4, e5, e6, S_, em, xi = sympy.symbols('A B C rA rB rC Q e1 e2 e3 e4 e5 e6 S e_m xi')
```

```
roundoff_rel_errs = [e1, e2, e3, e4, e5, e6]
```

```
Q = B*B - 4*A*C
# There are two types of solutions. xi switches the sign, as in the improved solution formula.
S1_exact = (-B + xi*sympy.sqrt(Q))/(2*A)
S2_exact = 2*C/(-B + xi*sympy.sqrt(Q))
prettylatex(S1_exact)
prettylatex(S2_exact)
```

$$\frac{1}{2 A} \left(- B + \xi \sqrt{- 4 A C + B^{2}}\right)$$

$$\frac{2 C}{- B + \xi \sqrt{- 4 A C + B^{2}}}$$

```
# Represent solutions with floating point errors.
# e1  ... e6 are relative roundoff errors after arithmetic operations.
# Their values depend on the next closest floating point value, and is generally unkown.
Q = B*B - 4*A*C
EQ = (B*B*(1+e3) - 4*A*C*(1+e2))*(1+e1)
S1 = (-B + xi*sympy.sqrt(EQ)*(1+e4))/(2*A)*(1+e5)*(1+e6)
S2 = 2*C/(-B + xi*sympy.sqrt(EQ)*(1+e4))*(1+e5)*(1+e6)
prettylatex(S1)
prettylatex(S2)
```

$$\frac{1}{2 A} \left(- B + \xi \sqrt{\left(e_{1} + 1\right) \left(- 4 A C \left(e_{2} + 1\right) + B^{2} \left(e_{3} + 1\right)\right)} \left(e_{4} + 1\right)\right) \left(e_{5} + 1\right) \left(e_{6} + 1\right)$$

$$\frac{2 C \left(e_{5} + 1\right) \left(e_{6} + 1\right)}{- B + \xi \sqrt{\left(e_{1} + 1\right) \left(- 4 A C \left(e_{2} + 1\right) + B^{2} \left(e_{3} + 1\right)\right)} \left(e_{4} + 1\right)}$$

```
# Do simple error propagation by linearization:
# Taylor series expansion to first order around true coefficients and e1..e6=0.
# That is  F(A + A*rA, ..., e1, ....) \approx F(A, ..., 0, ...) + |dF/dA|*A*rA + .... + |dF/de1|*e1 + ....
derivs = {}
for s in [A, B, C] + roundoff_rel_errs:
    d = sympy.diff(S1,s).subs({e1:0, e2:0, e3:0, e4:0, e5:0, e6:0}).subs(Q , Q_)
    derivs[s] = d
    prettylatex(sympy.Eq(sympy.Derivative(S_, s) ,d))
```

$$\frac{d}{d A} S = - \frac{C \xi}{A \sqrt{Q}} - \frac{1}{2 A^{2}} \left(- B + \sqrt{Q} \xi\right)$$

$$\frac{d}{d B} S = \frac{\frac{B \xi}{\sqrt{Q}} - 1}{2 A}$$

$$\frac{d}{d C} S = - \frac{\xi}{\sqrt{Q}}$$

$$\frac{d}{d e_{1}} S = \frac{\xi}{2 A \sqrt{Q}} \left(- 2 A C + \frac{B^{2}}{2}\right)$$

$$\frac{d}{d e_{2}} S = - \frac{C \xi}{\sqrt{Q}}$$

$$\frac{d}{d e_{3}} S = \frac{B^{2} \xi}{4 A \sqrt{Q}}$$

$$\frac{d}{d e_{4}} S = \frac{\sqrt{Q} \xi}{2 A}$$

$$\frac{d}{d e_{5}} S = \frac{1}{2 A} \left(- B + \sqrt{Q} \xi\right)$$

$$\frac{d}{d e_{6}} S = \frac{1}{2 A} \left(- B + \sqrt{Q} \xi\right)$$

```
# Then estimate each error term by derivative times uncertainty magnitude.
# EStimate e1...e6 as machine epsilon.
roundoff_total_err = 0
for s in roundoff_rel_errs:
    term = derivs[s]
    roundoff_total_err += sympy.Abs(sympy.ratsimp(term)).subs(Q , Q_)
roundoff_total_err = (sympy.simplify(roundoff_total_err))*em
prettylatex(roundoff_total_err)
```

$$e_{m} \left(\left|{\frac{1}{A} \left(B - \sqrt{Q} \xi\right)}\right| + \frac{\left|{\frac{\sqrt{Q} \xi}{A}}\right|}{2} + \left|{\frac{C \xi}{\sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{B^{2} \xi}{A \sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{\xi}{A \sqrt{Q}} \left(4 A C - B^{2}\right)}\right|\right)$$

```
# Take supplied errors for A,B, and C.
propagated_error = 0
for s, rs in [(A, rA), (B, rB), (C, rC)]:
    term = derivs[s]
    err = sympy.Abs((term))*rs
    propagated_error += err
prettylatex(propagated_error)
```

$$rA \left|{- \frac{C \xi}{A \sqrt{Q}} - \frac{1}{2 A^{2}} \left(- B + \sqrt{Q} \xi\right)}\right| + \frac{rB}{2} \left|{\frac{1}{A} \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}\right| + rC \left|{\frac{\xi}{\sqrt{Q}}}\right|$$

```
# In total ...
total_error1 = roundoff_total_err + propagated_error
prettylatex(sympy.Eq(sympy.symbols('ERR1'), total_error1))
```

$$ERR_{1} = e_{m} \left(\left|{\frac{1}{A} \left(B - \sqrt{Q} \xi\right)}\right| + \frac{\left|{\frac{\sqrt{Q} \xi}{A}}\right|}{2} + \left|{\frac{C \xi}{\sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{B^{2} \xi}{A \sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{\xi}{A \sqrt{Q}} \left(4 A C - B^{2}\right)}\right|\right) + rA \left|{- \frac{C \xi}{A \sqrt{Q}} - \frac{1}{2 A^{2}} \left(- B + \sqrt{Q} \xi\right)}\right| + \frac{rB}{2} \left|{\frac{1}{A} \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}\right| + rC \left|{\frac{\xi}{\sqrt{Q}}}\right|$$

```
prettylatex(total_error1.subs(B-xi*sympy.sqrt(Q_), sympy.symbols('G')).subs(Q, Q_))
```

$$e_{m} \left(\left|{\frac{G}{A}}\right| + \frac{3}{4} \left|{\frac{\sqrt{Q} \xi}{A}}\right| + \left|{\frac{C \xi}{\sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{B^{2} \xi}{A \sqrt{Q}}}\right|\right) + rA \left|{\frac{C \xi}{A \sqrt{Q}} - \frac{G}{2 A^{2}}}\right| + \frac{rB}{2} \left|{\frac{1}{A} \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}\right| + rC \left|{\frac{\xi}{\sqrt{Q}}}\right|$$

```
# Now the same for solution type 2
derivs = {}
for s in [A, B, C] + roundoff_rel_errs:
    d = sympy.diff(S2,s).subs({e1:0, e2:0, e3:0, e4:0, e5:0, e6:0}).subs(Q , Q_)
    derivs[s] = d
roundoff_total_err = 0
for s in roundoff_rel_errs:
    term = derivs[s]
    roundoff_total_err += sympy.Abs((term)).subs(Q , Q_)
roundoff_total_err = (sympy.simplify(roundoff_total_err))*em
propagated_error = 0
for s, rs in [(A, rA), (B, rB), (C, rC)]:
    term = derivs[s]
    err = sympy.Abs((term))*rs
    propagated_error += err
total_error2 = roundoff_total_err + propagated_error
prettylatex(sympy.Eq(sympy.symbols('ERR2'),total_error2))
```

$$ERR_{2} = e_{m} \left(4 \left|{\frac{C}{B - \sqrt{Q} \xi}}\right| + 2 \left|{\frac{C \sqrt{Q} \xi}{\left(B - \sqrt{Q} \xi\right)^{2}}}\right| + 4 \left|{\frac{A C^{2} \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + \left|{\frac{B^{2} C \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + \left|{\frac{C \xi \left(4 A C - B^{2}\right)}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right|\right) + 4 rA \left|{\frac{C^{2} \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + 2 rB \left|{\frac{C \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}{\left(B - \sqrt{Q} \xi\right)^{2}}}\right| + rC \left|{\frac{4 A C \xi}{\sqrt{Q} \left(- B + \sqrt{Q} \xi\right)^{2}} + \frac{2}{- B + \sqrt{Q} \xi}}\right|$$

```
prettylatex(total_error2.subs(Q, Q_).subs(B-xi*sympy.sqrt(Q_), sympy.symbols('G')))
```

$$e_{m} \left(4 \left|{\frac{C}{G}}\right| + 3 \left|{\frac{C \xi}{G^{2}} \sqrt{Q}}\right| + 4 \left|{\frac{A C^{2} \xi}{G^{2} \sqrt{Q}}}\right| + \left|{\frac{B^{2} C \xi}{G^{2} \sqrt{Q}}}\right|\right) + 4 rA \left|{\frac{C^{2} \xi}{G^{2} \sqrt{Q}}}\right| + 2 rB \left|{\frac{C}{G^{2}} \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}\right| + rC \left|{\frac{4 A C \xi}{G^{2} \sqrt{Q}} - \frac{2}{G}}\right|$$

```
# Let's try. However, note that all calculations are done in default floating point format.
# But I take the low precision eps of float16. Normally the error calculations themselves would be subject to roundoff error.
def print_absolute_errors(A_val, B_val, C_val, eA, eB, eC, computation_eps, val_type = None):
    if val_type is not None:
        A_val = val_type(A_val)
        B_val = val_type(B_val)
        C_val = val_type(C_val)
        eA = val_type(eA)
        eB = val_type(eB)
        eC = val_type(eC)
    #eps = np.finfo(np.float16).eps
    d = {A : A_val, B : B_val, C : C_val, em : computation_eps}
    d[xi] = 1 if d[B]<0 else -1
    de = d.copy()
    de.update({ rA: eA, rB : eB, rC: eC})
    S1_val = S1_exact.subs(d).evalf()
    S2_val = S2_exact.subs(d).evalf()
    e1_val = total_error1.subs(Q_ , Q).subs(d).evalf()
    e2_val = total_error2.subs(Q_ , Q).subs(d).evalf()
    #if S1_val < S2_val:
    #    S1_val, S2_val, e1_val, e2_val = S2_val, S1_val, e2_val, e1_val
    print "Sol = ", S1_val, S2_val
    print "Err1 = ", e1_val, "(", total_error1.subs(Q_ , Q).subs(de).evalf() ,")"
    print "Err2 = ", e2_val, "(", total_error2.subs(Q_ , Q).subs(de).evalf() ,")"

print_absolute_errors(1, 100, 1, 0, 0, 0.1, np.finfo(np.float32).eps, val_type = np.float32)
```

    Sol =  -99.9899902343750 -0.0100010018795729
    Err1 =  100.0*rA + 1.0001*rB + 0.010002*rC + 3.57604003511369e-5 ( 0.00103596039116383 )
    Err2 =  1.0004e-6*rA + 0.00010003*rB + 0.010002*rC + 3.57675622453257e-9 ( 0.00100020365789533 )

```
# Compute the relative error.
rel_err1 = (total_error1 / sympy.Abs(S1_exact.subs(Q,Q_))).subs(Q, Q_)
prettylatex(rel_err1)
```

$$\frac{2}{\left|{\frac{1}{A} \left(B - \sqrt{Q} \xi\right)}\right|} \left(e_{m} \left(\left|{\frac{1}{A} \left(B - \sqrt{Q} \xi\right)}\right| + \frac{3}{4} \left|{\frac{\sqrt{Q} \xi}{A}}\right| + \left|{\frac{C \xi}{\sqrt{Q}}}\right| + \frac{1}{4} \left|{\frac{B^{2} \xi}{A \sqrt{Q}}}\right|\right) + rA \left|{- \frac{C \xi}{A \sqrt{Q}} - \frac{1}{2 A^{2}} \left(- B + \sqrt{Q} \xi\right)}\right| + \frac{rB}{2} \left|{\frac{1}{A} \left(\frac{B \xi}{\sqrt{Q}} - 1\right)}\right| + rC \left|{\frac{\xi}{\sqrt{Q}}}\right|\right)$$

```
# Relative error for type 2 solution.
rel_err2 = (total_error2 / sympy.Abs(S2_exact.subs(Q,Q_)))
rel_err2 = sympy.simplify(rel_err2)
# Sympy appears to have problems canceling absolute terms :-(
prettylatex(rel_err2)
```

$$\frac{1}{\left|{\frac{C}{B - \sqrt{Q} \xi}}\right|} \left(\frac{e_{m}}{2} \left(4 \left|{\frac{C}{B - \sqrt{Q} \xi}}\right| + 2 \left|{\frac{C \sqrt{Q} \xi}{\left(B - \sqrt{Q} \xi\right)^{2}}}\right| + 4 \left|{\frac{A C^{2} \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + \left|{\frac{B^{2} C \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + \left|{\frac{C \xi \left(4 A C - B^{2}\right)}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right|\right) + 2 rA \left|{\frac{C^{2} \xi}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + rB \left|{\frac{C \left(B \xi - \sqrt{Q}\right)}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right| + rC \left|{\frac{2 A C \xi - \sqrt{Q} \left(B - \sqrt{Q} \xi\right)}{\sqrt{Q} \left(B - \sqrt{Q} \xi\right)^{2}}}\right|\right)$$

```
# Study relative errors
eps64 = np.finfo(np.float64).eps
eps32 = np.finfo(np.float32).eps
def compute_error_propagation_coefficients(A_val, B_val, C_val, computation_eps = eps64):
    """ Compute linear factors in the propagation of errors when computation is done using 'computation_eps'. Relative input errors rA,rB,rC are
        measured in machine epsilons (float32). The printed formula approximates the relative error w.r.t the true solution in units of
        epsilon of float32. """
    dABC = {A : A_val, B : B_val, C : C_val, em : computation_eps}
    dABC[xi] = 1 if d[B]<0 else -1
    print "Solutions: ", S1_exact.subs(Q_, Q).subs(dABC).evalf(), S2_exact.subs(Q_, Q).subs(dABC).evalf()
    rel_error1_in_eps32 = (rel_err1/eps32).subs(Q_ , Q).subs(dABC).subs({rA: A*rA*eps32, rB : B*rB*eps32, rC : C*rC*eps32}).evalf()
    rel_error2_in_eps32 = (rel_err2/eps32).subs(Q_ , Q).subs(dABC).subs({rA: A*rA*eps32, rB : B*rB*eps32, rC : C*rC*eps32}).evalf()
    print "Solution Type 1: {},\nSolution Type 2: {}".format(rel_error1_in_eps32, rel_error2_in_eps32)

# Worst catastrophic cancelation in the BB - 4AC term!
compute_error_propagation_coefficients(1, 2.*(1+eps32),  1)
```

    Solutions:  -1.00048840047384 -0.999511837944738
    Solution Type 1: 1024.50009155273*A*rA + 1023.99996948242*B*rB + 1023.50009155273*C*rC + 3.81656206993623e-6,
    Solution Type 2: 1023.50009155273*A*rA + 1023.99996948242*B*rB + 1024.50009155273*C*rC + 3.81656206993623e-6

```
# Same here. Float32 calculations.
compute_error_propagation_coefficients(1, 2.*(1+eps32),  1, computation_eps = eps32)
```

    Solutions:  -1.00048840047384 -0.999511837944738
    Solution Type 1: 1024.50009155273*A*rA + 1023.99996948242*B*rB + 1023.50009155273*C*rC + 2049.00115919127,
    Solution Type 2: 1023.50009155273*A*rA + 1023.99996948242*B*rB + 1024.50009155273*C*rC + 2049.00115919127

```
# Benign cases
compute_error_propagation_coefficients(1, 3,  1, computation_eps = eps64)
compute_error_propagation_coefficients(1, 3, 1, computation_eps = eps32)
```

    Solutions:  -2.61803398874989 -0.381966011250105
    Solution Type 1: 1.17082043397669*A*rA + 0.447213605681647*B*rB + 0.170820383068248*C*rC + 5.95253473659895e-9,
    Solution Type 2: 0.170820383068248*A*rA + 0.44721356884395*B*rB + 1.1708203234636*C*rC + 5.95253473659895e-9
    Solutions:  -2.61803398874989 -0.381966011250105
    Solution Type 1: 1.17082043397669*A*rA + 0.447213605681647*B*rB + 0.170820383068248*C*rC + 3.19574283420307,
    Solution Type 2: 0.170820383068248*A*rA + 0.44721356884395*B*rB + 1.1708203234636*C*rC + 3.19574283420307

```
# Catastrophic cancelation in -B + sqrt(....) prevented by improved solution formula.
compute_error_propagation_coefficients(1, 100, 1, computation_eps = eps64)
compute_error_propagation_coefficients(1, 100, 1, computation_eps = eps32)
```

    Solutions:  -99.9899989998000 -0.0100010002000500
    Solution Type 1: 1.00010005767486*A*rA + 0.0100020003020874*B*rB + 0.000100030004041248*C*rC + 5.58812178674155e-9,
    Solution Type 2: 0.000100030004041248*A*rA + 0.0100020000040344*B*rB + 1.0000999703994*C*rC + 5.58812178674155e-9
    Solutions:  -99.9899989998000 -0.0100010002000500
    Solution Type 1: 1.00010005767486*A*rA + 0.0100020003020874*B*rB + 0.000100030004041248*C*rC + 3.00009995911915,
    Solution Type 2: 0.000100030004041248*A*rA + 0.0100020000040344*B*rB + 1.0000999703994*C*rC + 3.00010009534472

```

```


# Gaussian Processes

In this exercise, you will implement Gaussian process regression and apply it to a toy and a real dataset. We use the notation used in the paper "Rasmussen (2005). Gaussian Processes in Machine Learning" linked on ISIS.

Let us first draw a training set $X = (x_1,\dots,x_n)$ and a test set $X_\star = (x^\star_1,\dots,x^\star_m)$ from a $d$-dimensional input distribution. The Gaussian Process is a model under which the real-valued outputs $\mathbf{f} = (f_1,\dots,f_n)$ and $\mathbf{f}_\star = (f^\star_1,\dots,f^\star_m)$ associated to $X$ and $X_\star$ follow the Gaussian distribution:

\begin{equation*}
\left[
\begin{array}{c}\mathbf{f}\\
\mathbf{f}_\star\end{array}
\right]
\sim
\mathcal{N}
\left(
\left[
\begin{array}{c}
\boldsymbol{0}\\
\boldsymbol{0}
\end{array}
\right]
,
\left[
\begin{array}{cc}
\Sigma & \Sigma_\star\\
\Sigma_\star^\top & \Sigma_{\star\star}
\end{array}
\right]
\right)
\end{equation*}

where

\begin{align*}
\Sigma &= k(X,X)+\sigma^2 I\\
\Sigma_\star &= k(X,X_\star)\\
\Sigma_{\star\star} &= k(X_\star,X_\star)+\sigma^2 I
\end{align*}

and where $k(\cdot,\cdot)$ is the Gaussian kernel function. (The kernel function is implemented in `utils.py`.) Predicting the output for new data points $X_\star$ is achieved by conditioning the joint probability distribution on the training set. Such conditional distribution called posterior distribution can be written as:

\begin{equation}
\mathbf{f}_\star | \mathbf{f} \sim \mathcal{N} (
\underbrace{\Sigma_\star^\top \Sigma^{-1} \mathbf{f}}_{\boldsymbol{\mu}_\star}
~,~
\underbrace{\Sigma_{\star\star} - \Sigma_\star^\top \Sigma^{-1} \Sigma_\star}_{C_\star}
)
\end{equation}

Having inferred the posterior distribution, the log-likelihood of observing for the inputs $X_\star$ the outputs $\mathbf{y}_\star$ is given by evaluating the distribution $\mathbf{f}_\star | \mathbf{f}$ at $\mathbf{y}_\star$:

\begin{equation}
\log p(\mathbf{y}_\star | \mathbf{f}) = -\frac1{2} (\mathbf{y}_\star - \boldsymbol{\mu}_\star)^\top C^{-1}_\star (\mathbf{y}_\star - \boldsymbol{\mu}_\star) - \frac1{2}\log|C_\star| - \frac{m}{2}\log2\pi
\end{equation}

where $|\cdot|$ is the determinant. Note that the likelihood of the data given this posterior distribution can be measured both for the training data and the test data.

## Part 1: Implementing a Gaussian Process (30 P)

**Tasks:**

* **Create a class `GP_Regressor` that implements a Gaussian process regressor and has the following three methods:**

 * **`def __init__(self,Xtrain,Ytrain,width,noise):`** Initialize a Gaussian process with noise parameter $\sigma$ and width parameter $w$. The variable `Xtrain` is a two-dimensional array where each row is one data point from the training set. The Variable `Ytrain` is a vector containing the associated targets. The function must also precompute the matrix $\Sigma^{-1}$ for subsequent use by the method `predict()` and `loglikelihood()`.

 * **`def predict(self,Xtest):`** For the test set $X_\star$ of $m$ points received as parameter, return the mean vector of size $m$ and covariance matrix of size $m \times m$ of the corresponding output, that is, return the parameters $(\boldsymbol{\mu}_\star,C_\star)$ of the Gaussian distribution $\mathbf{f}_\star | \mathbf{f}$.

 * **`def loglikelihood(self,Xtest,Ytest):`** For a data set $X_\star$ of $m$ test points received as first parameter, return the loglikelihood of observing the outputs $\mathbf{y}_\star$ received as second parameter.

```python
# --------------------------
import utils
import numpy as np
import math

class GP_Regressor:
        def __init__(self,Xtrain,Ytrain,width,noise):
            self.width=width
            self.noise=noise

            self.X=Xtrain
            self.Y=Ytrain

            self.kXX=utils.gaussianKernel(self.X,self.X,self.width)+self.noise**2*numpy.identity(len(self.X))
            self.kXXI=numpy.linalg.inv(self.kXX)

        def predict(self,Xtest):
            self.kZZ=utils.gaussianKernel(Xtest,Xtest,self.width)+self.noise**2*numpy.identity(len(Xtest))
            self.kXZ=utils.gaussianKernel(self.X,Xtest,self.width)
            self.kZX=self.kXZ.T
            mean=self.kZX.dot(self.kXXI.dot(self.Y))
            cov=self.kZZ-self.kZX.dot(self.kXXI).dot(self.kXZ)
            return mean,cov

        def loglikelihood(self,Xtest,Ytest):
            mean,cov=self.predict(Xtest)
            cov1=np.linalg.inv(cov)
            q=-0.5*cov1.dot(Ytest-mean).dot(Ytest-mean)
            q1=-0.5*np.linalg.slogdet(cov)[1]
            q2=-0.5*Xtest.shape[0]*numpy.log(2*numpy.pi)
            return q+q1+q2
 #----------------------------
```

* **Test your implementation by running the code below (it visualizes the mean and variance of the prediction at every location of the input space) and compares the behavior of the Gaussian process for various noise parameters $\sigma$ and width parameters $w$.**

```python
import utils,datasets,numpy
import matplotlib.pyplot as plt
%matplotlib inline

# Open the toy data
Xtrain,Ytrain,Xtest,Ytest = utils.split(*datasets.toy())

# Create an analysis distribution
Xrange = numpy.arange(-3.5,3.51,0.025)[:,numpy.newaxis]

f = plt.figure(figsize=(18,15))

# Loop over several parameters:
for i,noise in enumerate([2.5,0.5,0.1]):
    for j,width in enumerate([0.1,0.5,2.5]):

        # Create Gaussian process regressor object
        gp = GP_Regressor(Xtrain,Ytrain,width,noise)

        # Compute the predicted mean and variance for test data
        mean,cov = gp.predict(Xrange)
        var = cov.diagonal()

        # Compute the log-likelihood of training and test data
        lltrain = gp.loglikelihood(Xtrain,Ytrain)
        lltest  = gp.loglikelihood(Xtest ,Ytest )

        # Plot the data
        p = f.add_subplot(3,3,3*i+j+1)
        p.set_title('noise=%.1f width=%.1f lltrain=%.1f, lltest=%.1f'%(noise,width,lltrain,lltest))
        p.set_xlabel('x')
        p.set_ylabel('y')
        p.scatter(Xtrain,Ytrain,color='green',marker='x') # training data
        p.scatter(Xtest,Ytest,color='green',marker='o')   # test data
        p.plot(Xrange,mean,color='blue')                  # GP mean
        p.plot(Xrange,mean+var**.5,color='red')           # GP mean + std
        p.plot(Xrange,mean-var**.5,color='red')           # GP mean - std
        p.set_xlim(-3.5,3.5)
        p.set_ylim(-4,4)
```

## Part 2: Application to the Yacht Hydrodynamics Data Set (20 P)

In the second part, we would like to apply the Gaussian process regressor that you have implemented to a real dataset: the Yacht Hydrodynamics Data Set available on the UCI repository at the webpage http://archive.ics.uci.edu/ml/datasets/Yacht+Hydrodynamics. As stated on the web page, the input variables for this regression problem are:

1. Longitudinal position of the center of buoyancy
2. Prismatic coefficient
3. Length-displacement ratio
4. Beam-draught ratio
5. Length-beam ratio
6. Froude number

and we would like to predict from these variables the residuary resistance per unit weight of displacement (last column in the file `yacht_hydrodynamics.data`).

**Tasks:**

* **Load the data using `datasets.yacht()` and partition the data between training and test set using the function `utils.split()`. Standardize the data (center and rescale) so that each dimension of the training data and the labels have mean 0 and standard deviation 1 over the training set.**

* **Train several Gaussian processes on the regression task using various combinations of width and noise parameters.**

* **Draw two contour plots where the training and test log-likelihood are plotted as a function of the noise and width parameters. Choose suitable ranges of parameters so that the best parameter combination for the test set is in the plot. Use the same ranges and contour levels for the training and test plots. The contour levels can be chosen linearly spaced between e.g. 50 and the maximum log-likelihood value**

```python
# --------------------------
# TODO: Replace by your code
# --------------------------

from matplotlib import pyplot as plt

Xtrain,Ytrain,Xtest,Ytest = utils.split(*datasets.yacht())

xmean, xstd = Xtrain.mean(axis=0),Xtrain.std(axis=0)
ymean, ystd = Ytrain.mean(), Ytrain.std()

Xtrain = (Xtrain-xmean) / xstd
Ytrain = (Ytrain-ymean) / ystd
Xtest = (Xtest -xmean) / xstd
Ytest = (Ytest-ymean) / ystd

noises = np.linspace(0.005,0.04,num=24)
widths = np.linspace(0.05,2.0,num=24)

PX,PY=numpy.meshgrid(widths,noises)
lltrain = np.zeros([len(noises),len(widths)])
lltest = np.zeros([len(noises),len(widths)])

for i,noise in enumerate(noises):
    for j,width in enumerate(widths):
        gp = GP_Regressor(Xtrain,Ytrain,width,noise)
        lltrain[i,j] = gp.loglikelihood(Xtrain,Ytrain)
        lltest[i,j] = gp.loglikelihood(Xtest,Ytest)
m=max(lltrain.max(),lltest.max())

f=plt.figure(figsize=(12,6))

p=f.add_subplot(1, 2, 1)
p.set_title('logP(train | GP posterior)')
p.set_xlabel('width')
p.set_ylabel('noise')
CS=p.contour(PX,PY,lltrain,levels=numpy.arange(50,m,20))
p.clabel(CS,inline=1,fontsize=10)

p=f.add_subplot(1, 2, 2)
p.set_xlabel('width')
p.set_ylabel('noise')
p.set_title('logP(test | GP posterior)')
CS=p.contour(PX,PY,lltest,levels=numpy.arange(50,m,20))
p.clabel(CS,inline=1,fontsize=10)
```

```python

```

Ex09 - Gaussian Process/sheet09-programming.ipynb


# Steam equilibrating with liquid water

Accident scenario: steam leaks into a rigid, insulated tank that is partially filled with water.

The steam and liquid water are not initially at thermal equilibrium, though they are at the same pressure. The steam is at temperature $T_{s,1}$ = 600 C and pressure $P_1$ = 20 MPa.
The liquid water is initially at $T_{\text{liq},1}$ = 40 C and pressure $P_1$ = 20 MPa.
The total volume of the tank is $V$ = 10 m$^3$ and the volume of the liquid water initially in the tank is $V_{\text{liq},1} = 1 \; \text{m}^3$.

Eventually, the contents of the tank reach a uniform temperature and pressure, $T_2$ and $P_2$. The tank is well-insulated and rigid.

**Problem:** Determine the final temperature and pressure of the water in the tank.

```python
# load necessary modules

# Numpy adds some useful numerical types and functions
import numpy as np

# Cantera will handle thermodynamic properties
import cantera as ct

# Pint gives us some helpful unit conversion
from pint import UnitRegistry
ureg = UnitRegistry()
Q_ = ureg.Quantity
```

First, what are the initial states of the steam and liquid?

```python
steam = ct.Water()
temp_steam1 = Q_(600, 'degC')
pres1 = Q_(20, 'MPa')
steam.TP = temp_steam1.to('K').magnitude, pres1.to('Pa').magnitude
steam()
```

      water:

           temperature          873.15  K
              pressure           2e+07  Pa
               density         55.0118  kg/m^3
      mean mol. weight          18.016  amu
        vapor fraction               1

                              1 kg            1 kmol
                           -----------      ------------
              enthalpy    -1.24332e+07        -2.24e+08     J
       internal energy    -1.27968e+07       -2.305e+08     J
               entropy         10024.8        1.806e+05     J/K
        Gibbs function    -2.11864e+07       -3.817e+08     J
     heat capacity c_p         2808.18        5.059e+04     J/K
     heat capacity c_v         1964.45        3.539e+04     J/K

```python
liquid = ct.Water()
temp_liquid1 = Q_(40, 'degC')
liquid.TP = temp_liquid1.to('K').magnitude, pres1.to('Pa').magnitude
liquid()
```

      water:

           temperature          313.15  K
              pressure           2e+07  Pa
               density         1000.81  kg/m^3
      mean mol. weight          18.016  amu
        vapor fraction               0

                              1 kg            1 kmol
                           -----------      ------------
              enthalpy    -1.57857e+07       -2.844e+08     J
       internal energy    -1.58056e+07       -2.848e+08     J
               entropy         4084.54        7.359e+04     J/K
        Gibbs function    -1.70647e+07       -3.074e+08     J
     heat capacity c_p         4128.68        7.438e+04     J/K
     heat capacity c_v         4011.51        7.227e+04     J/K

State $s,1$ is in the superheated vapor region (vapor fraction: 1) and state $\text{liq},1$ is in the compressed liquid region (vapor fraction: 0).

We can calculate the mass of liquid water in the tank, and then determine the volume and mass of steam:

$$
m_{\text{liq},1} = \frac{V_{\text{liq},1}}{v_{\text{liq},1}} \\
V_{s,1} = V_{\text{tank}} - V_{\text{liq},1} \\
m_{s,1} = \frac{V_{s,1}}{v_{s,1}}
$$

```python
vol_tank = Q_(10, 'meter^3')
vol_liquid1 = Q_(1, 'meter^3')

mass_liquid1 = vol_liquid1 / Q_(liquid.v, 'm^3/kg')
print(f'Mass of liquid at state 1: {mass_liquid1: .2f}')

vol_steam1 = vol_tank - vol_liquid1
mass_steam1 = vol_steam1 / Q_(steam.v, 'm^3/kg')
print(f'Mass of steam at state 1: {mass_steam1: .2f}')

mass_1 = mass_liquid1 + mass_steam1
print(f'Total mass of system: {mass_1: .2f}')
```

    Mass of liquid at state 1: 1000.81 kilogram
    Mass of steam at state 1: 495.11 kilogram
    Total mass of system: 1495.92 kilogram

Do a mass balance on the tank for going from state 1 to state 2 (after equilibrium):
\begin{equation}
\frac{dm}{dt} = m_2 - m_1 = 0
\end{equation}
and so $m_2 = m_1$, meaning the mass of water in the tank does not change.
We can then find the specific volume of water in the tank:
\begin{equation}
v_2 = \frac{V_{\text{tank}}}{m_2}
\end{equation}

The tank experiences no heat or work, and does not exhibit any bulk changes in kinetic or potential energy; we can do an energy balance on the tank for the process of going from state 1 to state 2:

$$
Q - W = \Delta KE + \Delta PE + \Delta U \\
\rightarrow 0 = U_2 - U_1 \\
0 = u_2 m_2 - \left( u_{\text{liq},1} m_{\text{liq},1} + u_{s,1} m_{s,1} \right)
$$

Therefore, we can find the specific internal energy of the final state:
\begin{equation}
u_2 = \frac{ u_{\text{liq},1} m_{\text{liq},1} + u_{s,1} m_{s,1} }{m_2}
\end{equation}

```python
mass_2 = mass_1

spec_vol2 = vol_tank / mass_2
print(f'Specific volume of state 2: {spec_vol2: .2e}')

int_energy2 = (
    Q_(liquid.u, 'J/kg')*mass_liquid1 + Q_(steam.u, 'J/kg')*mass_steam1
    ) / mass_2
int_energy2.ito('kilojoule/kg')
print(f'Internal energy of state 2: {int_energy2: .2f}')

water_equilibrium = ct.Water()
water_equilibrium.UV = int_energy2.to('J/kg').magnitude, spec_vol2.to('m^3/kg').magnitude

water_equilibrium()
```

    Specific volume of state 2: 6.68e-03 meter ** 3 / kilogram
    Internal energy of state 2: -14809.79 kilojoule / kilogram

      water:

           temperature         510.436  K
              pressure     3.18519e+06  Pa
               density         149.592  kg/m^3
      mean mol. weight          18.016  amu
        vapor fraction       0.0887074

                              1 kg            1 kmol
                           -----------      ------------
              enthalpy    -1.47885e+07       -2.664e+08     J
       internal energy    -1.48098e+07       -2.668e+08     J
               entropy         6505.97        1.172e+05     J/K
        Gibbs function    -1.81094e+07       -3.263e+08     J
     heat capacity c_p    inf              inf              J/K
     heat capacity c_v         6766.85        1.219e+05     J/K

So, at equilibrium, the tank contains a mixture of saturated liquid and vapor, with temperature and pressure:

```python
final_temperature = Q_(water_equilibrium.T, 'K')
final_pressure = Q_(water_equilibrium.P, 'Pa')

print(f'Final temperature: {final_temperature: .2f}')
print(f'Final pressure: {final_pressure.to(ureg.MPa): .2f}')
```

    Final temperature: 510.44 kelvin
    Final pressure: 3.19 megapascal


$\newcommand{\mb}[1]{\mathbf{ #1 }}$
$\newcommand{\bs}[1]{\boldsymbol{ #1 }}$
$\newcommand{\bb}[1]{\mathbb{ #1 }}$

$\newcommand{\R}{\bb{R}}$

$\newcommand{\ip}[2]{\left\langle #1, #2 \right\rangle}$
$\newcommand{\norm}[1]{\left\Vert #1 \right\Vert}$

$\newcommand{\der}[2]{\frac{\mathrm{d} #1 }{\mathrm{d} #2 }}$
$\newcommand{\derp}[2]{\frac{\partial #1 }{\partial #2 }}$

# Finite Dimensional Koopman Bilinear System

Consider a nonlinear dynamical system that allows an exact finite dimensional Koopman canonical transform such that the control-affine dynamics can be transformed to a bilinear system. Consider the dynamical system

\begin{equation}
\mb{\dot{x}}=\mb{f}_0(\mb x) + \mb f_1 ( \mb x) u_1 + \mb f_2(\mb x) u_2,
\end{equation}

where we for this example choose $\mb f_0, \mb f_1$ as follows:

\begin{equation}
\mb f_0(\mb x) = \begin{bmatrix} x_3 \\ x_4 \\ \lambda x_3 \\ \mu x_4 + (2 \lambda - \mu) c x_3^2 \end{bmatrix}, \qquad
\mb f_1(\mb x) = \begin{bmatrix} 0 \\ 0 \\ 1 \\ 0 \end{bmatrix}, \qquad
\mb f_2(\mb x) = \begin{bmatrix} 0 \\ 0 \\ 0 \\ x_1+1 \end{bmatrix},
\end{equation}

and $\lambda, \mu, c \in \mathbb{R}$ are scalar parameters of the system. Setting $ \mb x = [q_1 \, q_2 \, \dot{q_1} \, \dot{q_2}]^T$,
these equations of motion can be expressed as robotic dynamics of the form $\mb{D}(\mb{q})\ddot{\mb{q}} + \mb{C}(\mb{q}, \dot{\mb{q}})\dot{\mb{q}} + \mb{G}(\mb{q}) = \mb{B}\mb{u}$, where $\mb D$ is the inertia matrix, $\mb C$ is the matrix of Coriolis terms, $\mb G$ is the matrix of gravitational terms, and $\mb B$ is the static actuation matrix. Rewriting $\mb f_0, \mb f_1, \mb f_2$ in terms of $\mb D, \mb C, \mb G,$ and $\mb B$ yield

\begin{equation}
\mb D(\mb q) = \begin{bmatrix} 1 & 0\\ 0 &  \frac{1}{q_1+1} \end{bmatrix},
\qquad \mb C(\mb q, \mb{\dot{q}}) = -\begin{bmatrix} \lambda & 0 \\ \frac{1}{q_1 + 1}(2 \lambda - \mu) c \dot{q}_1 & \frac{1}{q_1 +1} \mu \end{bmatrix}, \qquad
\mb G(\mb q) = \begin{bmatrix} 0 \\ 0 \end{bmatrix}
\qquad \mb B = \begin{bmatrix}1 & 0 \\ 0 & 1 \end{bmatrix},
\end{equation}

```python
import numpy as np
import scipy as sc
import sys
sys.path.append('../../../')
```

# Define experiment parameters

```python
from core.dynamics import RoboticDynamics

class FiniteDimKoopSys(RoboticDynamics):
    def __init__(self, lambd, mu, c):
        RoboticDynamics.__init__(self, 2, 2)
        self.params = lambd, mu, c

    def D(self, q):
        return np.array([[1, 0],[0, (q[0]+1)**(-1)]])

    def C(self, q, q_dot):
        labmd, mu, c = self.params
        return -np.array([[lambd, 0], [(q[0]+1)**(-1)*(2*lambd - mu)*c*q_dot[0], (q[0]+1)**(-1)*mu]])

    def G(self, q):
        return np.array([0, 0])

    def B(self, q):
        return np.array([[1, 0], [0, 1]])

class FiniteDimKoopSysDiscrete(FiniteDimKoopSys):
    def __init__(self, lambd, mu, c, dt):
        FiniteDimKoopSys.__init__(self, lambd, mu, c)
        self.dt = dt

    def eval_dot(self, x, u, t):
        return x + self.dt*self.drift(x, t) + self.dt*np.dot(self.act(x, t),u)

    def get_linearization(self, x0, x1, u0, t):
        lambd, mu, c = self.params
        A_lin = np.eye(self.n) + self.dt*np.array([[0, 0, 1, 0],
                                            [0, 0, 0, 1],
                                            [0, 0, lambd, 0],
                                            [u0[1], 0, 2*(2*lambd-mu)*c*x0[2], mu]])

        B_lin = self.dt*np.array([[0, 0],
                          [0, 0],
                          [1, 0],
                          [0, x0[0]+1]])

        if x1 is None:
            x1 = A_lin@x0 + B_lin@u0

        f_d = self.eval_dot(x0,u0,t)
        r_lin = f_d - x1

        return A_lin, B_lin, r_lin

n, m = 4, 2
lambd, mu, c = .3, .2, -.5
finite_dim_koop_sys = FiniteDimKoopSys(lambd, mu, c)
```

```python
from koopman_core.dynamics import LinearLiftedDynamics

A_lin = np.array([[0, 0, 1, 0],
                  [0, 0, 0, 1],
                  [0, 0, lambd, 0],
                  [0, 0, 0, mu]])
B_lin = np.array([[0, 0],
                  [0, 0],
                  [1, 0],
                  [0, 1]])
dt = 1e-2
linearized_sys = LinearLiftedDynamics(A_lin, B_lin, np.eye(n), lambda x: x)

# MPC parameters:
umax = np.array([7.5, 7.5])
umin = -umax
xmax = np.array([6., 6., 5., 5.])
xmin = -xmax
Q_mpc = 1*sc.sparse.diags([1, 1, 1e-1, 1e-1])
QN_mpc = 1e4*sc.sparse.eye(n)
R_mpc = sc.sparse.eye(m)
traj_duration = 2
traj_length = int(traj_duration/dt)

# Design trajectory:
x0 = np.array([-2., -2., 0., 0.])
set_pt = np.array([5., 3., 0., 0.])
term_constraint = False

# Define initial solution for SQP algorithm:
x_init = np.linspace(x0, set_pt, int(traj_length)+1)
u_init = np.zeros((m,traj_length)).T
```

# Construct Koopman bilinear form of the system

As a result of the careful construction of this system, there exists a Koopman canonical transform, $\mb z = T(\mb x)$ that exactly transforms the control-affine dynamics into a bilinear system. Consider the transformation:

\begin{equation}
    T(\mb q, \mb{\dot{q}}) = \begin{bmatrix}
    \phi_1(\mb q, \mb{\dot{q}})\\
    \phi_2(\mb q, \mb{\dot{q}})\\
    \phi_3(\mb q, \mb{\dot{q}})\\
    \phi_4(\mb q, \mb{\dot{q}})\\
    \phi_5(\mb q, \mb{\dot{q}})\\
    \phi_6(\mb q, \mb{\dot{q}})\\
    \end{bmatrix}
    = \begin{bmatrix}
    1\\
    q_1 - \frac{1}{\lambda}\dot{q}_1\\
    q_2 - \frac{1}{\mu} \dot{q}_2 + \frac{(2 \lambda - \mu)c}{2\lambda \mu} \dot{q}_1^2\\
    \dot{q}_1\\
    \dot{q}_2 - c \dot{q}_1^2\\
    \dot{q}_1^2\\
    \end{bmatrix},
\end{equation}

where $\phi_1, \phi_2, \phi_3, \phi_4, \phi_5, \phi_6$ are eigenfunctions of the Koopman operator associated with the drift
vector field $\mb f_0$. The matrix with the eigenvalue associated with the i-th eigenfunction on the i-th diagonal
element is $F=\text{diag}(0, 0, \lambda, \mu, 2 \lambda, 0)$. Then, to reformulate the dynamics we have:

\begin{equation}
L_{\mb f_1} T(\mb q, \mb{\dot{q}}) = \begin{bmatrix} 0\\ -\frac{1}{\lambda}\\ \frac{(2\lambda - \mu)c}{\lambda \mu}\dot{q}_1\\ 1 \\ -2c\dot{q}_1 \\ 2\dot{q_1} \end{bmatrix}, \qquad
L_{\mb f_2} T(\mb q, \mb{\dot{q}}) = \begin{bmatrix} 0 \\ 0\\ -\frac{1}{\mu}(q_1 + 1)\\0 \\ q_1 + 1 \\ 0 \end{bmatrix}
\end{equation}

and the dynamics can be equivalently transformed to a bilinear form $\mb{\dot{z}} = F \mb z + G_1\mb z u_1 + G_2\mb z u_2$ with

\begin{equation}
F = \begin{bmatrix}
0 &0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0\\
0 &0 & 0 & 0 & 0 & 0\\
0 &0 & 0 & \lambda & 0 & 0\\
0 &0 & 0 & 0 & \mu & 0 \\
0 &0 & 0 & 0 & 0 & 2 \lambda\\
\end{bmatrix}, \qquad
G_1 = \begin{bmatrix}
0 & 0 & 0 & 0 & 0 & 0\\
-\frac{1}{\lambda}& 0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & \frac{(2\lambda - \mu)c}{\lambda \mu} & 0 & 0\\
1 & 0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & -2c & 0 & 0\\
0 & 0 & 0 & 2 & 0 & 0\\
\end{bmatrix}
, \qquad
G_2 = \begin{bmatrix}
0 & 0 & 0 & 0 & 0 & 0\\
0 & 0 &  0 & 0 & 0 & 0\\
 -\frac{1}{\mu} & -\frac{1}{\mu} & 0 &  -\frac{1}{\lambda \mu} & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0\\
1 & 1 & 0 & \frac{1}{\lambda} & 0 & 0\\
0 & 0 & 0 & 0 & 0 & 0\\
\end{bmatrix}
\end{equation}

```python
from koopman_core.dynamics import BilinearLiftedDynamics

F = np.array([[0, 0, 0, 0, 0, 0],
              [0, 0, 0, 0, 0, 0],
              [0, 0, 0, 0, 0, 0],
              [0, 0, 0, lambd, 0, 0],
              [0, 0, 0, 0, mu, 0],
              [0, 0, 0, 0, 0, 2*lambd]])
G_1 = np.array([[0, 0, 0, 0, 0, 0],
                [-1/lambd, 0, 0, 0, 0, 0],
                [0, 0, 0, (2*lambd - mu)*c/(lambd*mu), 0, 0],
                [1, 0, 0, 0, 0, 0],
                [0, 0, 0, -2*c, 0, 0],
                [0, 0, 0, 2, 0, 0]])
G_2 = np.array([[0, 0, 0, 0, 0, 0],
                [0, 0, 0, 0, 0, 0],
                [-1/mu, -1/mu, 0, -1/(lambd*mu), 0, 0],
                [0, 0, 0, 0, 0, 0],
                [1, 1, 0, 1/lambd, 0, 0],
                [0, 0, 0, 0, 0, 0]])
G = [G_1, G_2]
phi_fun = lambda x: np.array([1,
                            x[0,0] - 1/lambd*x[0,2],
                            x[0,1] - 1/mu*x[0,3] + (2*lambd - mu)*c/(2*lambd*mu)*x[0,2]**2,
                            x[0,2],
                            x[0,3] - c*x[0,2]**2,
                            x[0,2]**2])
phi_fun_grad = lambda x: np.array([[0., 0., 0., 0.]
                                   [1., 0., -1/lambd, 0.],
                                   [0., 1., (2*lambd-mu)*c/(lambd*mu)*x[0,2], -1/mu],
                                   [0., 0., 1., 0.],
                                   [0., 0., -2*c*x[0,2], 1.],
                                   [0., 0., 2*x[0,2], 0.]])

C_q = np.array([[0, 1, 0, 1/lambd, 0, 0],
                [0, 0, 1, 0, 1/mu, c/mu - ((2*lambd - mu)*c)/(2*lambd*mu)]])
C_qdot = np.array([[0, 0, 0, 1, 0, 0],
                   [0, 0, 0, 0, 1, c]])
C = np.concatenate((C_q, C_qdot), axis=0)
n_koop = 6

koop_bilinear_sys = BilinearLiftedDynamics(n_koop, m, F, G, C, phi_fun)
```

#### Verify that the Koopman Bilinear Form model and control-affine model are equivalent

To verify that the models are in fact equivalent we sample a random control sequence and simulate both models forward with the same control sequence. As can be seen below, the resulting trajectories are equivalent.

```python
from matplotlib.pyplot import subplots, tight_layout, show
from koopman_core.controllers.openloop_controller import OpenLoopController

ol_controller = OpenLoopController(finite_dim_koop_sys, np.random.normal(size=(2,200)).T, np.arange(0,200)*dt)
z0 = phi_fun(x0.reshape((1,-1)))
xs, us = finite_dim_koop_sys.simulate(x0, ol_controller, np.arange(0,201)*dt)
zs_koop, us_koop = koop_bilinear_sys.simulate(z0, ol_controller, np.arange(0,201)*dt)
xs_koop = np.dot(C, zs_koop.T)

_, axs = subplots(3, 2, figsize=(15, 6))
ylabels = ['$q_1$', '$q_2$', '$\\dot{q}_1$', '$\\dot{q}_2$', '$u_1$', '$u_2$']

for ax, data_fb, data_koop, ylabel in zip(axs.flatten(), np.vstack((xs[:-1,:].T, us.T)), np.vstack((xs_koop[:,:-1], us_koop.T)), ylabels):
    ax.plot(np.arange(0,200)*dt, data_fb, linewidth=3, label='True system')
    ax.plot(np.arange(0,200)*dt, data_koop, linewidth=3, label='KCT')
    ax.set_ylabel(ylabel, fontsize=16)
    ax.grid()
    ax.set_xlabel('$t$ (sec)', fontsize=16)
    ax.legend()

tight_layout()
show()
```

# Design model predictive controllers

```python
solver_settings = {}
solver_settings['gen_embedded_ctrl'] = False
solver_settings['warm_start'] = True
solver_settings['polish'] = True
solver_settings['polish_refine_iter'] = 3
solver_settings['scaling'] = True
solver_settings['adaptive_rho'] = True
solver_settings['check_termination'] = 25
solver_settings['max_iter'] = 4000
solver_settings['eps_abs'] = 1e-6
solver_settings['eps_rel'] = 1e-6
solver_settings['eps_prim_inf'] = 1e-4
solver_settings['eps_dual_inf'] = 1e-4
solver_settings['linsys_solver'] = 'qdldl'
```

#### Linear model predictive controller

```python
from koopman_core.controllers import MPCController

controller_lmpc = MPCController(linearized_sys, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt, terminal_constraint=term_constraint, add_slack=True)
```

#### Bilinear model predictive controller

```python
from koopman_core.controllers import NonlinearMPCControllerNb, BilinearMPCControllerNb

# Define rough discrete-time model:
A_d = np.eye(n_koop) + koop_bilinear_sys.A*dt
B_d = [b*dt for b in koop_bilinear_sys.B]
kbf_d = BilinearLiftedDynamics(n_koop, m, A_d, B_d, C, koop_bilinear_sys.basis, continuous_mdl=False, dt=dt)

# Define kbf controller:
controller_knmpc = BilinearMPCControllerNb(kbf_d, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt, solver_settings, terminal_constraint=term_constraint)
z0 = phi_fun(x0.reshape((1,-1)))
z_init = np.array([phi_fun(x.reshape((1,-1))) for x in x_init])
controller_knmpc.construct_controller(z_init, u_init)
```

#### Nonlinear model predictive controller

```python
finite_dim_koop_sys_d = FiniteDimKoopSysDiscrete(lambd, mu, c, dt)
controller_nmpc = NonlinearMPCControllerNb(finite_dim_koop_sys_d, traj_length, dt, umin, umax, xmin, xmax, Q_mpc, R_mpc, QN_mpc, set_pt, solver_settings, terminal_constraint=term_constraint)
controller_nmpc.construct_controller(x_init, u_init)
```

# Evaluate performance of controllers for trajectory generation

#### Design trajectories with the constructed MPCs

```python
max_iter = 50

controller_lmpc.eval(x0, 0)
xr_lmpc = controller_lmpc.parse_result()
ur_lmpc = controller_lmpc.get_control_prediction()

controller_knmpc.solve_to_convergence(z0, 0., z_init, u_init, max_iter=max_iter)
xr_knmpc = koop_bilinear_sys.C@controller_knmpc.get_state_prediction().T
ur_knmpc = controller_knmpc.get_control_prediction().T

controller_nmpc.solve_to_convergence(x0, 0., x_init, u_init, max_iter=max_iter)
xr_nmpc = controller_nmpc.get_state_prediction().T
ur_nmpc = controller_nmpc.get_control_prediction().T
```

    /home/carlfolkestad/anaconda3/envs/bedmd_mpc/lib/python3.7/site-packages/numba/core/dispatcher.py:241: UserWarning: Numba extension module 'numba_scipy' failed to load due to 'ValueError(No function '__pyx_fuse_0pdtr' found in __pyx_capi__ of 'scipy.special.cython_special')'.
      entrypoints.init_all()
    /home/carlfolkestad/Coding_projects/koopman-learning-and-control/koopman_core/controllers/bilinear_mpc_controller_numba.py:14: NumbaPerformanceWarning: np.dot() is faster on contiguous arrays, called on (array(float64, 1d, C), array(float64, 2d, A))
      r_vec[i, :] = np.dot(z_init[i, :], A_reshaped[i*nx:(i+1)*nx, :])
    /home/carlfolkestad/Coding_projects/koopman-learning-and-control/koopman_core/controllers/bilinear_mpc_controller_numba.py:9: NumbaPerformanceWarning: np.dot() is faster on contiguous arrays, called on (array(float64, 2d, A), array(float64, 2d, F))
      B_lst_flat = (np.dot(z_init[:-1, :], B_arr)).flatten()
    /home/carlfolkestad/Coding_projects/koopman-learning-and-control/koopman_core/controllers/nonlinear_mpc_controller_numba.py:225: RuntimeWarning: divide by zero encountered in double_scalars
      while (iter == 0 or np.linalg.norm(u_prev - self.cur_u) / np.linalg.norm(u_prev) > eps) and iter < max_iter:

#### Simulate designed trajectories open-loop

```python
from koopman_core.controllers import OpenLoopController
t_eval = np.arange(0,traj_length+1)*dt
ol_controller_lmpc = OpenLoopController(finite_dim_koop_sys, ur_lmpc.T, t_eval[:-1])
xs_lmpc, us_lmpc = finite_dim_koop_sys.simulate(x0, ol_controller_lmpc, t_eval)
xs_lmpc, us_lmpc = xs_lmpc.T, us_lmpc.T

ol_controller_knmpc = OpenLoopController(finite_dim_koop_sys, ur_knmpc.T, t_eval[:-1])
xs_knmpc, us_knmpc = finite_dim_koop_sys.simulate(x0, ol_controller_knmpc, t_eval)
xs_knmpc, us_knmpc = xs_knmpc.T, us_knmpc.T

ol_controller_nmpc = OpenLoopController(finite_dim_koop_sys, ur_nmpc.T, t_eval[:-1])
xs_nmpc, us_nmpc = finite_dim_koop_sys.simulate(x0, ol_controller_nmpc, t_eval)
xs_nmpc, us_nmpc = xs_nmpc.T, us_nmpc.T
```

#### Plot/analyze the results

```python
import matplotlib.pyplot as plt
from matplotlib.ticker import MaxNLocator
from tabulate import tabulate

plot_inds = [0, 2, 1, 3, 0, 1]
subplot_inds = [1, 2, 4, 5, 3, 6]
labels = ['$x_1$ (m)', '$x_2$ (m)', '$\\dot{x}_1$ (m/s)','$\\dot{x}_2$ (m/s)','$F_1$ (N)','$F_2$ (N)']
colors = ['tab:blue', 'tab:orange', 'tab:brown']

plt.figure(figsize=(15,8))
plt.suptitle('Trajectory designed with model predictive controllers\nsolid lines - designed trajectory | dashed lines - open loop simulated trajectory | black dotted lines - state/actuation bounds')
for ii in range(6):
    ind = plot_inds[ii]
    if ii < 4:
        plt.subplot(2,3,subplot_inds[ii])
        plt.plot(t_eval, xr_lmpc[ind,:], colors[0], label='Linear MPC')
        plt.plot(t_eval, xr_knmpc[ind, :], colors[1], label='Bilinear MPC')
        plt.plot(t_eval, xr_nmpc[ind,:], colors[2], label='Nonlinear MPC')

        plt.plot(t_eval, xs_lmpc[ind,:], '--', color=colors[0])
        plt.plot(t_eval, xs_knmpc[ind, :], '--', color=colors[1])
        plt.plot(t_eval, xs_nmpc[ind,:], '--', color=colors[2])

        plt.plot([0, 2.], [xmax[ind], xmax[ind]], ':k')
        plt.plot([0, 2.], [xmin[ind], xmin[ind]], ':k')
        plt.scatter(t_eval[0], x0[ind], color='g')
        plt.scatter(t_eval[-1], set_pt[ind], color='r')
        plt.ylabel(labels[ind])
        plt.ylim(xmin[ind]-0.1,xmax[ind]+0.1)
    else:
        ax = plt.subplot(2,3,subplot_inds[ii])
        plt.plot(t_eval[:-1],ur_lmpc[ind,:], color=colors[0], label='Linear MPC')
        plt.plot(t_eval[:-1], ur_knmpc[ind, :], color=colors[1], label='Bilinear MPC')
        plt.plot(t_eval[:-1],ur_nmpc[ind,:], color=colors[2], label='Nonlinear MPC')
        plt.plot([0, 2.], [umax[ind], umax[ind]], ':k')
        plt.plot([0, 2.], [umin[ind], umin[ind]], ':k')
        plt.ylabel(labels[ii])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))
        if subplot_inds[ii]==3:
            plt.legend(loc='upper right', bbox_to_anchor=(0.975, 0.95), frameon=False)
    if subplot_inds[ii] == 4 or subplot_inds[ii] == 5 or subplot_inds[ii] == 6:
        plt.xlabel('Time (sec)')

plt.show()

cost_ref_lmpc = (xr_lmpc[:,-1]-set_pt).T@QN_mpc@(xr_lmpc[:,-1]-set_pt) + np.sum(np.diag(ur_lmpc.T@R_mpc@ur_lmpc))
cost_ref_knmpc = (xr_knmpc[:,-1]-set_pt).T@QN_mpc@(xr_knmpc[:,-1]-set_pt) + np.sum(np.diag(ur_knmpc.T@R_mpc@ur_knmpc))
cost_ref_nmpc = (xr_nmpc[:,-1]-set_pt).T@QN_mpc@(xr_nmpc[:,-1]-set_pt) + np.sum(np.diag(ur_nmpc.T@R_mpc@ur_nmpc))

cost_ol_lmpc = (xs_lmpc[:,-1]-set_pt).T@QN_mpc@(xs_lmpc[:,-1]-set_pt) + np.sum(np.diag(ur_lmpc.T@R_mpc@ur_lmpc))
cost_ol_knmpc = (xs_knmpc[:,-1]-set_pt).T@QN_mpc@(xs_knmpc[:,-1]-set_pt) + np.sum(np.diag(ur_knmpc.T@R_mpc@ur_knmpc))
cost_ol_nmpc = (xs_nmpc[:,-1]-set_pt).T@QN_mpc@(xs_nmpc[:,-1]-set_pt) + np.sum(np.diag(ur_nmpc.T@R_mpc@ur_nmpc))

print('Quadratic cost (normalized by NMPC cost) of designed and realized trajectories designed with each MPC:\n')
print(tabulate([['Linear MPC', cost_ref_lmpc/cost_ref_nmpc, cost_ol_lmpc/cost_ol_nmpc],
                ['Koopman bilinear MPC', cost_ref_knmpc/cost_ref_nmpc, cost_ol_knmpc/cost_ol_nmpc],
                ['Nonlinear MPC', 1, 1]],
               headers=['Designed trajectory', 'Realized trajectory']))
```

#### Study evolution of the solution after each iteration of the SQP-algorithm

```python
n_iter = min(len(controller_nmpc.x_iter),len(controller_knmpc.x_iter))

# Calculate cost after each iteration:
iter_cost_nmpc, iter_cost_knmpc = [], []

ol_controller_init = OpenLoopController(finite_dim_koop_sys, u_init, t_eval[:-1])
xs_init, _ = finite_dim_koop_sys.simulate(x0, ol_controller_init, t_eval)
xs_init, us_init = xs_init.T, u_init.T
init_cost = (xs_init[:,-1]-set_pt).T@QN_mpc@(xs_init[:,-1]-set_pt) + np.sum(np.diag(us_init.T@R_mpc@us_init))
iter_cost_nmpc = [init_cost]
iter_cost_knmpc = [init_cost]

for ii in range(n_iter):
    ur_knmpc_iter = controller_knmpc.u_iter[ii].T
    ol_controller_knmpc_iter = OpenLoopController(finite_dim_koop_sys, ur_knmpc_iter, t_eval[:-1])
    xs_knmpc_iter, _ = finite_dim_koop_sys.simulate(x0, ol_controller_knmpc_iter, t_eval)
    xs_knmpc_iter, us_knmpc_iter = xs_knmpc_iter.T, ur_knmpc_iter.T
    iter_cost_knmpc.append((xs_knmpc_iter[:,-1]-set_pt).T@QN_mpc@(xs_knmpc_iter[:,-1]-set_pt) + np.sum(np.diag(us_knmpc_iter.T@R_mpc@us_knmpc_iter)))

    ur_nmpc_iter = controller_nmpc.u_iter[ii].T
    ol_controller_nmpc_iter = OpenLoopController(finite_dim_koop_sys, ur_nmpc_iter, t_eval[:-1])
    xs_nmpc_iter, _ = finite_dim_koop_sys.simulate(x0, ol_controller_nmpc_iter, t_eval)
    xs_nmpc_iter, us_nmpc_iter = xs_nmpc_iter.T, ur_nmpc_iter.T
    iter_cost_nmpc.append((xs_nmpc_iter[:,-1]-set_pt).T@QN_mpc@(xs_nmpc_iter[:,-1]-set_pt) + np.sum(np.diag(us_nmpc_iter.T@R_mpc@us_nmpc_iter)))

plt.figure(figsize=(15,8))
plt.suptitle('Control solution after each iteration of the SQP-algorithm for NMPC and K-NMPC')
ax1 = plt.subplot(2,3,1)
ax1.plot(t_eval[:-1],u_init[:, 0])
ax2 = plt.subplot(2,3,2)
ax2.plot(t_eval[:-1],u_init[:, 1], label='Iteration 0')
ax3 = plt.subplot(2,3,4)
ax3.plot(t_eval[:-1],u_init[:, 0])
ax4 = plt.subplot(2,3,5)
ax4.plot(t_eval[:-1],u_init[:, 0])

for it in range(n_iter):
    ax1.plot(t_eval[:-1],controller_nmpc.u_iter[it][0,:])
    ax2.plot(t_eval[:-1],controller_nmpc.u_iter[it][1,:], label='Iteration '+str(it+1))
    ax3.plot(t_eval[:-1],controller_knmpc.u_iter[it][0,:])
    ax4.plot(t_eval[:-1],controller_knmpc.u_iter[it][1,:])

ax5 = plt.subplot(1,3,3)
ax5.plot(np.arange(1,n_iter), iter_cost_nmpc[1:n_iter]/iter_cost_nmpc[-1], 'tab:cyan', label='NMPC')
ax5.plot(np.arange(1,n_iter), iter_cost_knmpc[1:n_iter]/iter_cost_nmpc[-1], 'k', label='K-NMPC')

ax1.set_title('Control action sequence at each iteration\n$u_1$')
ax1.set_ylabel('NMPC\nControl value')
ax2.set_title('$u_2$')
ax2.legend(ncol=2, loc='upper right', frameon=False)
ax3.set_ylabel('K-NMPC\nControl value')
ax3.set_xlabel('Time (sec)')
ax4.set_xlabel('Time (sec)')
ax5.set_title('Realized cost after each iteration\n')
ax5.set_xlabel('Iteration')
ax5.set_ylabel('Cost (normalized)')
ax5.legend(loc='upper right', frameon=False)

plt.show()

print('Solution statistics\n')
print(tabulate([['Nonlinear MPC', len(controller_nmpc.x_iter), np.mean(controller_nmpc.comp_time), np.std(controller_nmpc.comp_time), sum(controller_nmpc.comp_time)],
                ['Koopman bilinear MPC', len(controller_knmpc.x_iter), np.mean(controller_knmpc.comp_time), np.std(controller_knmpc.comp_time), sum(controller_knmpc.comp_time)]],
               headers=['Number of SQP\niterations','Mean comp. time per\niteration (secs)', 'Std comp. time per\niteration (secs)', 'Total comp.\ntime (secs)']))
```

# Evaluate performance of controllers for closed-loop control

#### Design finite horizon controllers

```python
Q_mpc_cl = 1e2*sc.sparse.diags([1, 1, 1e-1, 1e-1])
QN_mpc_cl= Q_mpc_cl
R_mpc_cl = sc.sparse.eye(m)
traj_duration = 0.5
N_cl = int(traj_duration/dt)
max_iter_sqp = 100

solver_settings_cl = solver_settings
```

```python
controller_lmpc_cl = MPCController(linearized_sys, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt, terminal_constraint=False, add_slack=True)
controller_knmpc_cl = BilinearMPCControllerNb(kbf_d, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt, solver_settings_cl, terminal_constraint=False, add_slack=True)
controller_knmpc_cl.construct_controller(controller_knmpc.cur_z[:N_cl+1,:], controller_knmpc.cur_u[:N_cl,:])

controller_nmpc_cl = NonlinearMPCControllerNb(finite_dim_koop_sys_d, N_cl, dt, umin, umax, xmin, xmax, Q_mpc_cl, R_mpc_cl, QN_mpc_cl, set_pt, solver_settings_cl, terminal_constraint=False, add_slack=True)
controller_nmpc_cl.construct_controller(controller_nmpc.cur_z[:N_cl+1,:], controller_nmpc.cur_u[:N_cl,:])
```

```python
controller_knmpc_cl.solve_to_convergence(z0, 0., controller_knmpc.cur_z[:N_cl+1,:], controller_knmpc.cur_u[:N_cl,:], max_iter=max_iter_sqp)
controller_knmpc_cl.eval(x0, 0.)
controller_nmpc_cl.solve_to_convergence(x0, 0., controller_nmpc.cur_z[:N_cl+1,:], controller_nmpc.cur_u[:N_cl,:], max_iter=max_iter_sqp)
controller_nmpc_cl.eval(x0, 0.)
controller_knmpc_cl.comp_time, controller_knmpc_cl.prep_time, controller_knmpc_cl.qp_time,  = [], [], []
controller_nmpc_cl.comp_time, controller_nmpc_cl.prep_time, controller_nmpc_cl.qp_time,  = [], [], []
```

```python
solver_settings_cl['polish'] = False
solver_settings_cl['check_termination'] = 5
solver_settings_cl['max_iter'] = 5
solver_settings_cl['eps_abs'] = 1e-2
solver_settings_cl['eps_rel'] = 1e-2
solver_settings['eps_prim_inf'] = 1e-3
solver_settings['eps_dual_inf'] = 1e-3
controller_nmpc_cl.update_solver_settings(solver_settings_cl)
controller_knmpc_cl.update_solver_settings(solver_settings_cl)
```

#### Simulate designed trajectories closed-loop

```python
xs_lmpc_cl, us_lmpc_cl = finite_dim_koop_sys.simulate(x0, controller_lmpc_cl, t_eval)
xs_lmpc_cl, us_lmpc_cl = xs_lmpc_cl.T, us_lmpc_cl.T

controller_knmpc_cl.comp_time = []
xs_knmpc_cl, us_knmpc_cl = finite_dim_koop_sys.simulate(x0, controller_knmpc_cl, t_eval)
xs_knmpc_cl, us_knmpc_cl = xs_knmpc_cl.T, us_knmpc_cl.T

controller_nmpc_cl.comp_time = []
xs_nmpc_cl, us_nmpc_cl = finite_dim_koop_sys.simulate(x0, controller_nmpc_cl, t_eval)
xs_nmpc_cl, us_nmpc_cl = xs_nmpc_cl.T, us_nmpc_cl.T
```

#### Plot/analyze the results

```python
plot_inds = [0, 2, 1, 3, 0, 1]
subplot_inds = [1, 2, 4, 5, 3, 6]
labels = ['$x_1$ (m)', '$x_2$ (m)', '$\\dot{x}_1$ (m/s)','$\\dot{x}_2$ (m/s)','$F_1$ (N)','$F_2$ (N)']
colors = ['tab:blue', 'tab:orange', 'tab:brown']

plt.figure(figsize=(15,8))
plt.suptitle('Closed loop control with model predictive controllers\n black dotted lines - state/actuation bounds')
for ii in range(6):
    ind = plot_inds[ii]
    if ii < 4:
        plt.subplot(2,3,subplot_inds[ii])
        plt.plot(t_eval, xs_lmpc_cl[ind,:], colors[0], label='Linear MPC')
        plt.plot(t_eval, xs_knmpc_cl[ind, :], colors[1], label='Bilinear MPC')
        plt.plot(t_eval, xs_nmpc_cl[ind,:], colors[2], label='Nonlinear MPC')

        plt.plot([0, 2.], [xmax[ind], xmax[ind]], ':k')
        plt.plot([0, 2.], [xmin[ind], xmin[ind]], ':k')
        plt.scatter(t_eval[0], x0[ind], color='g')
        plt.scatter(t_eval[-1], set_pt[ind], color='r')
        plt.ylabel(labels[ind])
        #plt.ylim(xmin[ind]-0.1,xmax[ind]+0.1)
    else:
        ax = plt.subplot(2,3,subplot_inds[ii])
        plt.plot(t_eval[:-1],us_lmpc_cl[ind,:], color=colors[0], label='Linear MPC')
        plt.plot(t_eval[:-1], us_knmpc_cl[ind, :], color=colors[1], label='Bilinear MPC')
        plt.plot(t_eval[:-1],us_nmpc_cl[ind,:], color=colors[2], label='Nonlinear MPC')
        plt.plot([0, 2.], [umax[ind], umax[ind]], ':k')
        plt.plot([0, 2.], [umin[ind], umin[ind]], ':k')
        plt.ylabel(labels[ii])
        ax.yaxis.set_major_locator(MaxNLocator(integer=True))
        if subplot_inds[ii]==3:
            plt.legend(loc='upper right', bbox_to_anchor=(0.975, 0.95), frameon=False)
    if subplot_inds[ii] == 4 or subplot_inds[ii] == 5 or subplot_inds[ii] == 6:
        plt.xlabel('Time (sec)')

plt.show()

cost_cl_lmpc = np.sum(np.diag((xs_lmpc_cl[:,:-1]-set_pt.reshape(-1,1)).T@Q_mpc@(xs_lmpc_cl[:,:-1]-set_pt.reshape(-1,1)))) + (xs_lmpc_cl[:,-1]-set_pt).T@QN_mpc@(xs_lmpc_cl[:,-1]-set_pt) + np.sum(np.diag(us_lmpc_cl.T@R_mpc@us_lmpc_cl))
cost_cl_knmpc = np.sum(np.diag((xs_knmpc_cl[:,:-1]-set_pt.reshape(-1,1)).T@Q_mpc@(xs_knmpc_cl[:,:-1]-set_pt.reshape(-1,1)))) + (xs_knmpc_cl[:,-1]-set_pt).T@QN_mpc@(xs_knmpc_cl[:,-1]-set_pt) + np.sum(np.diag(us_knmpc_cl.T@R_mpc@us_knmpc_cl))
cost_cl_nmpc = np.sum(np.diag((xs_nmpc_cl[:,:-1]-set_pt.reshape(-1,1)).T@Q_mpc@(xs_nmpc_cl[:,:-1]-set_pt.reshape(-1,1)))) + (xs_nmpc_cl[:,-1]-set_pt).T@QN_mpc@(xs_nmpc_cl[:,-1]-set_pt) + np.sum(np.diag(us_nmpc_cl.T@R_mpc@us_nmpc_cl))

print('\nSolution statistics:\n')
print(tabulate([['Linear MPC', cost_cl_lmpc/cost_cl_nmpc, np.mean(controller_lmpc_cl.comp_time), np.std(controller_lmpc_cl.comp_time)],
                ['Koopman bilinear MPC', cost_cl_knmpc/cost_cl_nmpc, np.mean(controller_knmpc_cl.comp_time), np.std(controller_knmpc_cl.comp_time)],
                ['Nonlinear MPC', cost_cl_nmpc/cost_cl_nmpc, np.mean(controller_nmpc_cl.comp_time), np.std(controller_nmpc_cl.comp_time)]],
               headers=['Quadratic cost\n(normalized)', 'Mean comp. time per\niteration (secs)', 'Std comp. time per\niteration (secs)']))
```

```python
print('\nSolution time profiling:\n')
print(tabulate([['NMPC', np.mean(controller_nmpc_cl.comp_time), np.mean(controller_nmpc_cl.prep_time), np.mean(controller_nmpc_cl.qp_time)],
                ['Koopman bilinear MPC', np.mean(controller_knmpc_cl.comp_time), np.mean(controller_knmpc_cl.prep_time), np.mean(controller_knmpc_cl.qp_time)]],
               headers=['Total comp time', 'setup time', 'qp solve time' ]))
```

    Solution time profiling:

                            Total comp time    setup time    qp solve time
    --------------------  -----------------  ------------  ---------------
    NMPC                        0.00425506    0.00386536       0.0003897
    Koopman bilinear MPC        0.000850148   0.000302155      0.000547993

```python

```