# Quadratically constrained quadratic programming and its applications in portfolio optimization

# Correct Rendering of this notebook

This notebook makes use of the `latex_envs` Jupyter extension for equations and references.  If the LaTeX is not rendering properly in your local installation of Jupyter , it may be because you have not installed this extension.  Details at https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/latex_envs/README.html

The notebook is also not rendered well by GitHub so if you are reading it from there, you may prefer the [pdf version instead](./static/portfolio_optimization_qcqp.pdf).

# Installing the NAG library and running this notebook

This notebook depends on the NAG library for Python to run. Please read the instructions in the [Readme.md](https://github.com/numericalalgorithmsgroup/NAGPythonExamples/blob/master/local_optimization/Readme.md#install) file to download, install and obtain a licence for the library.

Instruction on how to run the notebook can be found [here](https://github.com/numericalalgorithmsgroup/NAGPythonExamples/blob/master/local_optimization/Readme.md#jupyter).

# Introduction

Quadratically constrained quadratic programming (QCQP) is a type of optimization problem in which both the objective function and the constraints involve quadratic functions. A general QCQP problem has the following form
\begin{equation}
\begin{array}{ll}
\underset{x\in\Re^n}{\mbox{minimize}} &  \frac{1}{2}x^TP_0x+q_0^Tx+r_0\\[0.6ex]
\mbox{subject to} & \frac{1}{2}x^TP_ix+q_i^Tx+r_i\leq0,\quad i=1,\ldots,p.
\end{array}
\end{equation}
It appears in applications such as modern portfolio theory, machine learning, engineering and control. Convex QCQP is usually handled through conic optimization, or, more precisely, second-order cone programming (SOCP) due to its computational efficiency and ability to detect infeasibility. However, using SOCP to solve convex QCQP is nontrivial task which requires extra amount of effort to transform problem data and add auxiliary variables. In this notebook, we are going to demonstrate how to use the *NAG Optimization Modelling Suite* in the NAG Library to define and solve QCQP in portfolio optimization.

# Data Preparation

We consider daily prices for the 30 stocks in the DJIA from March 2018 to March 2019. In practice, the estimation of the mean return $r$ and covariance $V$ is often a nontrivial task. In this notebook, we estimate those entities using simple sample estimates.

```python
# Import necessary libraries
import pickle as pkl
import numpy as np
import matplotlib.pyplot as plt
```

```python
# Load stock price data from stock_price.plk
# Stock_price: dict = ['close_price': [data], 'date_index': [data]]
stock_price = stock_price = pkl.load(open('./data/stock_price.pkl', 'rb'))
close_price = stock_price['close_price']
date_index = stock_price['date_index']
```

```python
# Size of data, m: number of observations, n: number of stocks
m = len(date_index)
n = len(close_price)
```

```python
# Extract stock closing prices to a numpy array
data = np.zeros(shape=(m, n))
i = 0
for stock in close_price:
    data[:,i] = close_price[stock]
    plt.plot(np.arange(m), data[:,i])
    i += 1
# Plot closing prices
plt.xlabel('Time (days)')
plt.ylabel('Closing price ($)')
plt.show()
```

For each stock $i$, we first estimate the $j$th daily relative return as $$relative~return_{i,j} = \frac{closing~price_{i,j+1}-closing~price_{i,j}}{closing~price_{i,j}}.$$

```python
# Relative return
rel_rtn = np.zeros(shape=(m-1, n))
for j in range(m-1):
    rel_rtn[j,:] = np.divide(data[j+1,:] - data[j,:], data[j,:])
# Plot relative return
for i in range(n):
    plt.plot(np.arange(m-1),rel_rtn[:,i])
plt.xlabel('Time (days)')
plt.ylabel('Relative return')
plt.show()
```

Simply take arithmetic mean of each column of relative return to get mean return $r$ for each stock, followed by estimating covariance $V$ using numpy.

```python
# Mean return
r = np.zeros(n)
r = rel_rtn.sum(axis=0)
r = r / (m-1)
# Covariance matrix
V = np.cov(rel_rtn.T)
```

# Classic Mean-Variance Model
## Efficient Frontier

One of the major goals of portfolio management is to achieve a certain level of return under a specific risk measurement. Here we demonstrate how to use NAG Library to build efficient frontier by solving classical Markowitz model with long-only constraint (meaning, buy to hold and short selling is not allowed):
\begin{equation}\label{MV_model}
\begin{array}{ll}
\underset{x\in\Re^n}{\mbox{minimize}} & -r^Tx+\mu x^TVx\\[0.6ex]
\mbox{subject to} & e^Tx = 1,\\[0.6ex]
     & x\geq0,
\end{array}
\end{equation}
where $e\in\Re^n$ is vector of all ones and $\mu$ is a scalar controling trade-off between return and risk. Note one could build the efficient frontier by varying $\mu$ from $0$ to a certain value.

```python
# Import the NAG Library
from naginterfaces.base import utils
from naginterfaces.library import opt
from naginterfaces.library import lapackeig
# Import necessary math libraries
from scipy.sparse import coo_matrix
import math as mt
import warnings as wn
```

```python
# Input for quadratic objective
# Sparsity pattern of upper triangular V
irowq, icolq = np.nonzero(np.triu(V))
v_val = V[irowq, icolq]
# Convert to 1-based
irowq = irowq + 1
icolq = icolq + 1
# Sparsity pattern of r, which is actually dense in this application
idxr = np.arange(1, n+1)

# Input for linear constraint: e'x = 1
irowa = np.full(n, 1, dtype=int)
icola = np.arange(1, n+1)
a = np.full(n, 1.0, dtype=float)
bl = np.full(1, 1.0, dtype=float)
bu = np.full(1, 1.0, dtype=float)

# Input for bound constraint: x >= 0
blx = np.zeros(n)
bux = np.full(n, 1.e20, float)
```

The input data is ready, we can easily build the efficient frontier as follows.

```python
# Set step for mu
step = 2001

# Initialize output data: absolute risk and return
ab_risk = np.empty(0, float)
ab_rtn = np.empty(0, float)

for mu in np.linspace(0.0, 2000.0, step):
    # Create problem handle
    handle = opt.handle_init(n)

    # Set quadratic objective function
    # In qcqp standard form q should be 2*mu*V
    q = 2.0 * mu * v_val
    idqc = -1
    opt.handle_set_qconstr(handle, 0.0, idqc, idxr, -r, irowq, icolq, q)

    # Set linear constraint e'x = 1
    opt.handle_set_linconstr(handle, bl, bu, irowa, icola, a)

    # Set bound constraint
    opt.handle_set_simplebounds(handle, blx, bux)

    # Set options
    for option in [
            'Print Options = NO',
            'Print Level = 1',
            'Print File = -1',
            'SOCP Scaling = A'
    ]:
        opt.handle_opt_set(handle, option)

    # Call socp interior point solver
    # Mute warnings and do not count results from warnings
    wn.simplefilter('error', utils.NagAlgorithmicWarning)
    try:
        slt = opt.handle_solve_socp_ipm(handle)

        # Compute risk and return from the portfolio
        ab_risk = np.append(ab_risk, mt.sqrt(slt.x[0:n].dot(V.dot(slt.x[0:n]))))
        ab_rtn = np.append(ab_rtn, r.dot(slt.x[0:n]))
    except utils.NagAlgorithmicWarning:
        pass

    # Destroy the handle:
    opt.handle_free(handle)
```

```python
# plot the result
plt.plot(ab_risk*100.0, ab_rtn*100.0)
plt.ylabel('Total Expected Return (%)')
plt.xlabel('Absolute Risk (%)')
plt.show()
```

## Maximizing the Sharpe ratio

The Sharpe ratio is defined as the ratio of return of portfolio and standard deviation of the portfolio's excess return. It is usually used to measure the efficiency of a portfolio. Find the most efficient portfolio is equivalent to solve the following optimization problem.
\begin{equation}\label{sr_model}
\begin{array}{ll}
\underset{x\in\Re^n}{\mbox{minimize}} & \frac{\sqrt{x^TVx}}{r^Tx}\\[0.6ex]
\mbox{subject to} & e^Tx = 1,\\[0.6ex]
     & x\geq0.
\end{array}
\end{equation}
By replacing $x$ with $\frac{y}{\lambda}, \lambda\gt0$, model (\ref{sr_model}) is equivalent to
\begin{equation}\label{sr_model_eq}
\begin{array}{ll}
\underset{y\in\Re^n, \lambda\in\Re}{\mbox{minimize}} & y^TVy\\[0.6ex]
\mbox{subject to} & e^Ty = \lambda,\\[0.6ex]
     & r^Ty=1, \\
     & y\geq0, \\
     & \lambda\geq0.
\end{array}
\end{equation}
Problem (\ref{sr_model_eq}) is similar to problem (\ref{MV_model}) in the sense that they both have a quadratic objective function and linear constraints.

```python
# Input for linear constraint: e'y = lambda
irowa = np.full(n+1, 1, dtype=int)
icola = np.arange(1, n+2)
a = np.append(np.full(n, 1.0, dtype=float), -1.0)
bl = np.zeros(1)
bu = np.zeros(1)

# Inpute for linear constraint: r'y = 1
irowa = np.append(irowa, np.full(n, 2, dtype=int))
icola = np.append(icola, np.arange(1, n+1))
a = np.append(a, r)
bl = np.append(bl, 1.0)
bu = np.append(bu, 1.0)

# Input for bound constraint: x >= 0
blx = np.zeros(n+1)
bux = np.full(n+1, 1.e20, float)
```

Now we can call the NAG SOCP solver as follows.

```python
# Create problem handle
handle = opt.handle_init(n+1)

# Set quadratic objective function
# In qcqp standard form q should be 2*V
q = 2.0 * v_val
idqc = -1
opt.handle_set_qconstr(handle, 0.0, idqc, irowq=irowq, icolq=icolq, q=q)

# Set linear constraints
opt.handle_set_linconstr(handle, bl, bu, irowa, icola, a)

# Set bound constraint
opt.handle_set_simplebounds(handle, blx, bux)

# Set options
for option in [
        'Print Options = NO',
        'Print Level = 1',
        'Print File = -1',
        'SOCP Scaling = A'
]:
    opt.handle_opt_set(handle, option)

# Call socp interior point solver
slt = opt.handle_solve_socp_ipm(handle)

sr_risk = mt.sqrt(slt.x[0:n].dot(V.dot(slt.x[0:n])))/slt.x[n]
sr_rtn = r.dot(slt.x[0:n])/slt.x[n]
sr_x = slt.x[0:n]/slt.x[n]

# Destroy the handle:
opt.handle_free(handle)
```

```python
# plot result.
plt.plot(ab_risk*100.0, ab_rtn*100.0, label='Efficient frontier')
plt.plot([sr_risk*100], [sr_rtn*100], 'rs', label='Portfolio with maximum Sharpe ratio')
plt.plot([sr_risk*100, 0.0], [sr_rtn*100, 0.0], 'r-', label='Capital market line')
plt.axis([min(ab_risk*100), max(ab_risk*100), min(ab_rtn*100), max(ab_rtn*100)])
plt.ylabel('Total Expected Return (%)')
plt.xlabel('Absolute Risk (%)')
plt.legend()
plt.show()
```

# Portfolio optimization with tracking-error constraint

To avoid taking unnecessary risk when beating a benchmark, the investors commonly impose a limit on the volatility of the deviation of the active portfolio from the benchmark, which is also known as tracking-error volatility (TEV) \cite{J03}. The model to build efficient frontier in excess-return space is
\begin{equation}\label{er_tev}
\begin{array}{ll}
\underset{x\in\Re^n}{\mbox{maximize}} & r^Tx\\
\mbox{subject to} & e^Tx = 0,\\
     & x^TVx\leq tev,
\end{array}
\end{equation}
where $tev$ is a limit on the track-error. Roll \cite{R92} noted that problem (\ref{er_tev}) is totally independent of the benchmark and leads to the unpalatable result that the active portfolio has systematically higher risk than the benchmark and is not optimal. Therefore, in this section we solve a more advanced model by taking absolute risk into account as follows.
\begin{equation}\label{tev_model}
\begin{array}{ll}
\underset{x\in\Re^n}{\mbox{minimize}} & -r^Tx+\mu (x+b)^TV(x+b)\\
\mbox{subject to} & e^Tx = 0,\\
     & x^TVx\leq tev,\\
     & x+b\geq0,
\end{array}
\end{equation}
where $b$ is a benchmark portfolio. In this demonstration, it is generated synthetically. Note here we use the same covariance matrix $V$ for tev and absolute risk measurement for demonstration purpose. In practice one could use different covariance matrices from different markets.

```python
# Generate a benchmark portfolio from efficient portfolio that maximiz the Sharpe ratio
# Perturb x
b = sr_x + 1.e-1
# Normalize b
b = b/sum(b)

# Set limit on tracking-error
tev = 0.000002

# Compute risk and return at the benchmark
b_risk = mt.sqrt(b.dot(V.dot(b)))
b_rtn = r.dot(b)
```

```python
# Input for linear constraint: e'x = 0
irowa = np.full(n, 1, dtype=int)
icola = np.arange(1, n+1)
a = np.full(n, 1.0, dtype=float)
bl = np.zeros(1)
bu = np.zeros(1)

# Input for bound constraint: x >= -b
blx = -b
bux = np.full(n, 1.e20, float)
```

```python
# Initialize output data: TEV risk and return
tev_risk = np.empty(0, float)
tev_rtn = np.empty(0, float)

for mu in np.linspace(0.0, 2000.0, step):
    # Create problem handle
    handle = opt.handle_init(n)

    # Set quadratic objective function
    # In qcqp standard form q should be 2*mu*V
    q = 2.0 * mu * v_val
    r_mu = 2.0*mu*V.dot(b)-r
    idqc = -1
    opt.handle_set_qconstr(handle, 0.0, idqc, idxr, r_mu, irowq, icolq, q)

    # Set quadratic constraint
    # In qcqp standard form q should be 2*V
    q = 2.0 * v_val
    idqc = 0
    opt.handle_set_qconstr(handle, -tev, idqc, irowq=irowq, icolq=icolq, q=q)

    # Set linear constraint e'x = 1
    opt.handle_set_linconstr(handle, bl, bu, irowa, icola, a)

    # Set bound constraint
    opt.handle_set_simplebounds(handle, blx, bux)

    # Set options
    for option in [
            'Print Options = NO',
            'Print Level = 1',
            'Print File = -1',
            'SOCP Scaling = A'
    ]:
        opt.handle_opt_set(handle, option)

    # Call socp interior point solver
    # Mute warnings and do not count results from warnings
    wn.simplefilter('error', utils.NagAlgorithmicWarning)
    try:
        slt = opt.handle_solve_socp_ipm(handle)

#       Compute risk and return from the portfolio
        tev_risk = np.append(tev_risk, mt.sqrt((slt.x[0:n]+b).dot(V.dot(slt.x[0:n]+b))))
        tev_rtn = np.append(tev_rtn, r.dot(slt.x[0:n]+b))
    except utils.NagAlgorithmicWarning:
        pass

    # Destroy the handle:
    opt.handle_free(handle)
```

```python
# Plot the result
plt.figure(figsize=(7.5, 5.5))
plt.plot(ab_risk*100.0, ab_rtn*100.0, label='Classic efficient frontier')
plt.plot([sr_risk*100], [sr_rtn*100], 'rs', label='Portfolio with maximum Sharpe ratio')
plt.plot([sr_risk*100, 0.0], [sr_rtn*100, 0.0], 'r-', label='Capital market line')
plt.plot(b_risk*100, b_rtn*100, 'r*', label='Benchmark portfolio')
plt.plot(tev_risk*100.0, tev_rtn*100.0, 'seagreen', label='Efficient frontier with tev constraint')

plt.axis([min(ab_risk*100), max(ab_risk*100), min(tev_rtn*100), max(ab_rtn*100)])
plt.ylabel('Total Expected Return (%)')
plt.xlabel('Absolute Risk (%)')
plt.legend()
plt.show()
```

# Conclusion

In this notebook, we demonstrated how to use NAG Library to solve various quadratic models in portfolio optimization. Conic optimization is usually a good choice to solve convex QCQP. It is worth pointing out that the versatility of SOCP is not just limited to the QCQP models mentioned here. It covers a lot more problems and constraints. For example, DeMiguel et al. \cite{DGNU09} discussed portfolio optimization with norm constraint, which can be easily transformed into an SOCP problem. We refer readers to the NAG Library documentation \cite{NAGDOC} on SOCP solver and \cite{AG03, LVBL98} for more details.

# References

[1] Jorion Philippe, ``_Portfolio optimization with tracking-error constraints_'', Financial Analysts Journal, vol. 59, number 5, pp. 70--82,  2003.

[2] Roll Richard, ``_A mean/variance analysis of tracking error_'', The Journal of Portfolio Management, vol. 18, number 4, pp. 13--22,  1992.

[3] DeMiguel Victor, Garlappi Lorenzo, Nogales Francisco J et al., ``_A generalized approach to portfolio optimization: Improving performance by constraining portfolio norms_'', Management science, vol. 55, number 5, pp. 798--812,  2009.

[4] Numerical Algorithms Group, ``_NAG documentation_'',  2019.  [online](https://www.nag.com/numeric/fl/nagdoc_latest/html/frontmatter/manconts.html)

[5] Alizadeh Farid and Goldfarb Donald, ``_Second-order cone programming_'', Mathematical programming, vol. 95, number 1, pp. 3--51,  2003.

[6] Lobo Miguel Sousa, Vandenberghe Lieven, Boyd Stephen et al., ``_Applications of second-order cone programming_'', Linear algebra and its applications, vol. 284, number 1-3, pp. 193--228,  1998.

```python

```


```python
from decodes.core import *
from decodes.io.jupyter_out import JupyterOut
import math

out = JupyterOut.unit_square( )
```

# Transformation Mathematics

We are familiar with a set of operations in CAD designated by verbs, such as "Move”, “Mirror”, “Rotate”, and “Scale”, and that ***act upon a geometric object to produce the same kind of object, only transformed***.

Operations such as these are termed ***transformations or transforms***. After an object has undergone a transformation, we can observe that certain properties of the object are altered while others are preserved.

Mathematicians employ a number of terms (such as ***congruency, isometry, similarity, and affinity***) to classify transformations by the features they preserve and those they distort.

Consider, for example, the axonometric projection transformation, which projects geometry onto a plane in such a way that parallel lines are mapped onto parallel lines, thereby maintaining parallelism.

Mathematically speaking, we say that ***a transformation of a space onto itself is a rule which assigns to every point $P$ in the space another point $P^*$ in the space***.

The simple reflection that was constructed by compass and straightedge is an example of a transformation of the plane (or any point on the plane) onto itself (in that all points end up somewhere else on the plane).

Notice that, in the case of the mirror reflection, ***the kinds of things that come out after being reflected are the same kinds of things that go in***; namely, a reflected line is still a line with the same length, a reflected circle is still a circle with the same radius, and a reflected curve is still the same curve with all of the same geometric properties.

There are other transformations that do not preserve geometric features in the same way. Consider the ***circle inversion transformation***, which can be expressed as the function below.

\begin{align}
T(x,y) = (\frac{R^2x}{x^2+y^2},\frac{R^2y}{x^2+y^2})
\end{align}

The geometric properties that are preserved here may be more difficult to discern.

Knowing that the graphic shows the inversion of points on a hexagonal grid, we can understand that while the linearity of lines are not preserved, the angle between two lines or curves is preserved.

How and under what circumstances different classes of geometric features are preserved is an important and distinguishing property of transformations.

Even though a transformation is formally defined as any function that takes a point and gives a point back in return, we will find it beneficial to narrow this definition to include only those transformations that may be represented by a particularly useful mathematical construct: ***the matrix***.

A matrix is ***a structure for organizing sets of values in rows and columns, such that
these values may be operated upon by a set of algebraic rules***.

Matrix algebra underlies much of geometric computation. The expenditure of just a bit of effort in mastering the fundamentals of this potentially imposing mathematical construct will yield a wealth of insight in return.

## Matrix Fundamentals

A brief account of the matrix will serve to ground our understanding of how transformations work in computer graphics in general, and will offer a basis for the implementation of transformations in the Decod.es library in particular.

For this we will need a grasp of the basic notation for writing matrices, and a working understanding of how they are used to perform operations.

In this section, we:
* Detail the relevant notational conventions
* Present the algebra of matrices
* Demonstrate their basic operation on a simple example of transforming 2d vectors

### Matrix Notation

A mathematical matrix is much like its namesake in code: a two-dimensional array that organizes values into regular rows and columns.

An ***m x n matrix*** (read as “*m by n*”) and denoted throughout this chapter as $(m \times n)$, is an arrangement of elements into ***m rows*** and ***n columns***. Any matrix for which the number of rows and the number of columns are the same may be termed a square matrix.

By convention, the notation for a generic element contained within a matrix is $c_{ij}$, with the subscript index $i$ indicating the containing row, and the index $j$ the containing column.

Note that the conventional ordering of the indices of a matrix is the reverse of the `(x,y)` convention that we are accustomed to in describing horizontal and vertical positions.

Also, positions are numbered starting at the top left, and the indexing starts with `(1,1)`, not with `(0,0)` as we have become accustomed to in code.

\begin{bmatrix}
    c_{11} & c_{12} & c_{13} & c_{14} \\
    c_{21} & c_{22} & c_{23} & c_{24} \\
    c_{31} & c_{32} & c_{33} & c_{34}
\end{bmatrix}

### Matrix Algebra

With a grasp of the notation conventionally used to describe matrices, we are ready to review the rules by which they may be combined and manipulated.

Three of the basic operations we are able to perform on matrices - addition, subtraction, and scalar multiplication - work exactly the same for matrices as they do for vectors, proceeding by operating on one set of matching components at a time.

#### Matrix Addition

Matrix addition and subtraction works ***component-wise***, matching components at the same indices of each matrix. This procedure requires that each matrix exhibits the same number of rows and columns.

\begin{align}
\begin{bmatrix} 2 & -1 \\ 3 & 0 \end{bmatrix} +
\begin{bmatrix} -1 & 5 \\ 0 & 10 \end{bmatrix} =
\begin{bmatrix} 1 & 4 \\ 3 & 10 \end{bmatrix}
\end{align}

#### Matrix-Scalar Multiplication

Scalar multiplication matches the given scalar to each of the components of the matrix.

\begin{align}
3
\begin{bmatrix} 1 & -1 \\ -2 & 1 \\ 0 & 2 \end{bmatrix} =
\begin{bmatrix} 3 & -3 \\ -6 & 3 \\ 0 & 6 \end{bmatrix}
\end{align}

#### Matrix-Matrix Multiplication

Matrices may be multiplied together to form another matrix, but the convention for doing so is more involved.

Here, the components are formed by pairing rows of the first matrix with columns of the second and performing a “dot product” of the components.

This convention imposes a rule on the shapes of the two matrices being multiplied: the number of columns in the first must match the number of rows of the second, such that a $(m \times p)$ matrix can only multiply a $(p \times n)$ matrix. The result of this multiplication is a $(m \times n)$ matrix that takes its number of rows from the first matrix, and its number of columns from the second.

In summary:

\begin{align}
(m \times p)(p \times n) = (m \times n)
\end{align}

Each entry ***matches a row from the first matrix with a column from the second***, and is calculated by a dot product operation, as shown below.

### Matrices and Vectors

Points and Vecs may be represented by matrices, such that a two-dimensional vector may be expressed as a $(1 \times 2)$ or, more often ***a $(2 \times 1)$ matrix***. Seen in this way, we can multiply a matrix by a vector only so long as the dimensions are compatible. A square matrix $M$, can then multiply a vector $\vec{x} = (x,y)$ in the following way:

\begin{align}
M\vec{x} =
\begin{bmatrix} c_{11} & c_{12} \\ c_{21} & c_{22} \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix} =
\begin{bmatrix} c_{11}x + c_{12}y \\ c_{21}x + c_{22}y \end{bmatrix}
\end{align}

A square matrix multiplied by a vector in $\mathbb{R}^2$ yields another vector in $\mathbb{R}^2$.

We can legitimately say that ***$M$ maps one set of points onto a corresponding set of points***. This is the very definition of a transformation. It is no exaggeration to say that the implications of this are profound.

Consider that we have demonstrated that ***a compact and versatile mathematical form is capable of describing a high-level operation***. So armed, we need not think of any geometric operation, such as the rotation of a set of objects about an axis, merely as a command in software. Instead, we now have ***a mathematical instrument that captures this action precisely, compactly, and in a format that is completely independent*** from any software platform.

The ramifications of this discovery are indeed far-reaching, and extend well beyond the two-dimensional planar transformations captured by the square matrix demonstrated above.

In summary.

When a matrix $M$ multiplies a vector $\vec{x}$, it has the effect of transforming this vector into a new vector $M\vec{x}$.

Substituting points for vectors, any $(2 \times 2)$ matrix can then be seen as a ***planar transformation*** that maps any point in the plane to another point in the plane.

Similarly, a $(3 \times 3)$ matrix specifies a ***spatial transformation*** which maps a point from one location in space to another.

We require two more insights before we are in good position for implementation.

* We need a deeper understanding of the nature of ***a special class of transformations*** that represents the basic building blocks critical to many operations relevant to visual design.
* To aggregate these basic elements into more complex operations requires ***a method for expressing transformations into coherent sequences***.

#### Examples of Matrix-Vector Multiplication

Before moving on, it will be worth our time to consider the specific cases outlined in a nearby table that demonstrate what happens to a generic vector when multiplied by a variety of fixed square matrices. These examples will help us to associate some familiar actions with matrices that produce them.

##### Scaling Matrix
$ M = \begin{bmatrix} s & 0 \\ 0 & s \end{bmatrix} $

This matrix scales vectors by a uniform scaling factor as can be seen by multiplying the matrix by a vector, expanded out below. The vector is stretched for values of $s$ greater than one, and contracts for values less than one. For negative values of $s$, the transformed vector is both scaled and flipped across the origin.

\begin{align}
\begin{bmatrix} s & 0 \\ 0 & s \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix} =
\begin{bmatrix} sx \\ sy \end{bmatrix} =
s\begin{bmatrix} x \\ y \end{bmatrix}
\end{align}

##### Rotation Matrix
$ M = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} $

We can see what transformation this matrix represents by looking at how it acts on specific vectors. Multiplying this matrix by a vector rotates the vector by ninety degrees counterclockwise about the origin:
* $(1,0)$ is transformed to $(0,1)$
* $(0,1)$ is transformed to $(-1,0)$
* $(- 1,0)$ is transformed to $(0, -1)$.

\begin{align}
\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix} =
\begin{bmatrix} (0x) + (-1y) \\ (1x) + (0y) \end{bmatrix} =
\begin{bmatrix} -y \\ x \end{bmatrix}
\end{align}

##### Mirror Matrix
$ M = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} $

This matrix transforms the vector $(x,y)$ to $(y,x)$ which is the vector mirrored across the line $y = x$.

\begin{align}
\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix} =
\begin{bmatrix} (0x) + (1y) \\ (1x) + (0y) \end{bmatrix} =
\begin{bmatrix} y \\ x \end{bmatrix}
\end{align}

##### Projection Matrix
$ M = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} $

This matrix maps $(x,y)$ to $(x,0)$, its projection onto the x-axis.

\begin{align}
\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix} =
\begin{bmatrix} (1x) + (0y) \\ (0x) + (0y) \end{bmatrix} =
\begin{bmatrix} x \\ 0 \end{bmatrix}
\end{align}

## Matrix Transformations

The kind of transformations that can be described by matrices are very special, and just three categories of matrix transformations are prevalent in computer graphics: ***linear, affine, and projective transformations***.

***Linear transformations*** represent the most constrained category, and include elemental transforms such as rotation, scaling, shearing, and reflection. A discussion of these will comprise the bulk of this section.

A closely related category are the ***affine transformations***. These are ***"almost" linear***, as they can be expressed as the combination of a linear transformation and a translation vector. While pairing a matrix with a vector can be useful, an even more compact representation of an affine transformation as a matrix can be achieved by ***elevating the dimension*** of the matrix.

Finally, we have the ***projective transformations*** that include orthographic projection and perspectival projection.

### Linear Transformations

To discuss the unique features of linear transformations, we will first establish the relationship between linear transformations and matrix transformations.

To do so, we denote transformations that act on a vector by multiplication of a matrix as $T(\vec{x}) = M\vec{x}$.

Matrices such as this share a number of properties in common for any choice of matrix $M$. Crucially, the following two properties hold true:

* The transformation of the sum of any two vectors is equal to the sum of their individual transformations. In other words, $T(\vec{x} + \vec{y}) = T(\vec{x}) + T(\vec{y})$ for any vectors $\vec{x}$ and $\vec{y}$.
* The transformation of the product of a scalar and a vector is equal to the product of the scalar and the transformation of the vector. In other words, $T(c\vec{x}) = cT(\vec{x})$ for any vector $\vec{x}$ and scalar $c$.

Any transformation that satisfies these two properties is called a ***linear transformation***.

Linearity yields a remarkable number of useful consequences. Among these, three are particularly relevant for our purposes: two that concern the preservation of geometric features, and one that allows us to predict the action of a transformation simply by examining the values held by particular components of it.

* Linear transformations map straight lines to straight lines.
* Linear transformations preserve parallelism
* If we know how a linear transformation acts for ***each vector in a basis***, then we can predict how it will transform ***every point and vector in that space***.

\begin{align}
T(\vec{x}) = T(x,y) = xT(1,0) + yT(0,1) = xT(\vec{e_{1}}) + yT(\vec{e_{2}})
\end{align}

This last property of linear transformations allows us to quickly read off the action of any given matrix, and enables us to write matrices with properties that we can easily control.

Take, for example, the following matrix: An examination of the components here reveals how the standard basis vectors are transformed, and from this, we are able to extrapolate a pattern of behavior that can be applied more generally.

\begin{align}
\begin{bmatrix} 1 & 0 \\ 0 & 2 \end{bmatrix}
\end{align}

The basis vector $\vec{e_{1}} = (1,0)$ is unchanged by the transformation, but that $\vec{e_{2}} = (0,1)$ is stretched to twice its length. This is a scale 1d.

\begin{align}
\begin{bmatrix} 1 & 1 \\ 0 & 2 \end{bmatrix}
\end{align}

The vector $\vec{e_{1}} = (1,0)$ is again fixed, so the x-axis remains unchanged, but $\vec{e_{2}} = (0,1)$ is shifted to the line $y = 2x$. This is a shear.

\begin{align}
\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}
\end{align}

This does precisely nothing.

Mathematicians gave this one a compelling name: the identity transformation or $I$.

\begin{align}
T(\vec{x}) =
xT(\vec{e_{1}}) + yT(\vec{e_{2}}) =
\begin{bmatrix} T(\vec{e_{1}}) T(\vec{e_{2}}) \end{bmatrix} \vec{x}
\end{align}

Not only is every matrix transformation a linear one, but every linear transformation can be represented by a matrix. With this in mind, we can now assemble a library of useful linear transformations.

#### Selected Linear Transformations in the Plane

##### Rotation
$ \begin{bmatrix} cos\theta & -sin\theta \\ sin\theta & cos\theta \end{bmatrix} $

Building upon the earlier example representing a rotation by ninety degrees, the above matrix shows a transformation that rotates a vector by an arbitrary angle counter-clockwise about the origin. We’ve seen that all we need in order to construct this matrix is to understand how basis vectors are transformed.

Working with the standard basis, we can show that rotating $\vec{e_{1}} = (1,0)$ by $\theta$ counterclockwise will result in the vector $(cos\theta, sin\theta)$. Similarly, $\vec{e_{2}} = (0,1)$  transforms to $(-sin\theta, cos\theta)$. Putting these transformed basis vectors as columns, we arrive at the nearby matrix.

##### Orthogonal Projection
$ \begin{bmatrix}
cos^2\theta & cos\theta \ sin\theta  \\
cos\theta \ sin\theta  & sin^2\theta
\end{bmatrix} $

Given a line through the origin rotated at an angle $\theta$ counterclockwise from the horizontal, we may construct a matrix representing the transformation of the normal projection onto this line. The orthogonal projection of a point onto this line is equivalent to the nearest point on the line.

To see how the standard basis vectors are transformed, we will make use of the formula for the projected vector derived using the dot product. Since a unit vector along the projection line is given by $\vec{u} = (cos\theta, sin\theta)$, the projected vector for $\vec{u_{1}}$ onto this line is given by

\begin{align}
(\vec{e_{1}} \bullet \vec{u}) \ \vec{u} =
cos\theta \ (cos\theta, sin\theta) =
(\ cos^2\theta, \ cos\theta sin\theta \ )
\end{align}

Similarly, the projected vector for $\vec{e_{2}} = (0,1)$ is $( \ sin\theta, \ cos\theta sin^2\theta \ )$.

##### Mirror
$ \begin{bmatrix}
2 \ cos^2\theta-1 & 2 \ cos\theta \ sin\theta  \\
2 \ cos\theta \ sin\theta  & 2 \ sin^2\theta-1
\end{bmatrix} $

Given a line as constructed above, we may express a general mirror transformation across this line in terms of the projection vectors by simple vector subtraction, as given by

\begin{align}
\vec{p_{mirror}} = \vec{p_{near}} + ( \vec{p_{near}} - \vec{p}) =
2\vec{p_{near}} - \vec{p}
\end{align}

The reflection across this line of $\vec{e_{1}} = (1,0)$ is thus given by $2(cos^2 \theta, cos\theta sin\theta) - (1,0)$ and the mirror of $\vec{e_{1}} = (0,1)$ is given by $2(sin \theta, cos\theta sin^2\theta) - (0,1)$. From these, we arrive at the general mirror transformation above.

***What's missing?***

Examining what we have covered thus far, we may note the conspicuous absence of what is perhaps the most basic of all transformations: translation. Although basic, this transformation is actually not a linear transformation.

Expressing translation as displacement by a fixed vector, $T(\vec{x}) = \vec{x} + \vec{b}$, we see that the first condition of linearity is violated by any nonzero translation vector $\vec{b}$.

\begin{align}
T(\vec{x} + \vec{y}) =
\vec{x}+\vec{y}+\vec{b} \neq
T(\vec{x}) + T(\vec{y})
\end{align}

It appears the that ***translation is not able to be represented using a square matrix***. To account for a wider range of transformations that include translations using matrices requires elevating the size of the matrices employed. We'll get to that in a bit.

### The Algebra of Transformations in Sequence

Some transformations are better described as a sequence of operations, broken down into an ordered list of more basic transformations.

The order of operations at work here matters.

One great advantage of the matrix form is that ***the cumulative effect of the application of a sequence of transformations is equivalent to the application of the ordered product of this sequence***. In other words, we can capture a series of transformations in a single matrix.

Of critical importance here is the order in which this multiplication is done:

Successive application of transformations represented by matrices translates to multiplying matrices in ***right-to-left order***.

\begin{align}
\begin{bmatrix} \frac{1}{\sqrt{2}} & \frac{-1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix}
\begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix} =
\begin{bmatrix} \frac{-1}{\sqrt{2}} & \frac{-1}{\sqrt{2}} \\ \frac{-1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix}
\end{align}

\begin{align}
\begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} \frac{1}{\sqrt{2}} & \frac{-1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix} =
\begin{bmatrix} \frac{-1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix}
\end{align}

## From Math to Code

The matrices used to express transformations in three-dimensional space are not $(3 \times 3)$, as we would expect from the discussion so far. Rather, they are $(4 \times 4)$. In deconstructing the rationale behind this, we address the question of how to represent the translation transformation.

The easiest way to include translation into the mix of linear transformations is simply to combine a linear transformation with a translation vector: $T(\vec{x}) = Mx + \vec{b}$. In fact, this broader set of transformations make up the class of ***affine transformations***.

Every linear transformation is affine but not the other way around.

Tacking a vector on a matrix to deal with affine transformations is fantastic, but it ruins the purity of the matrix format. This is no good when considering how to move from math to code. There is an alternative.

We can employ a system of coordinates called ***homogeneous coordinates***. By
employing these, it is possible to use a $(4 \times 4)$ matrix to describe not
only affine transformations in a three-dimensional space, but also many other useful transforms.

### The Elevated Matrix

The dominant technique in computer graphics is to elevate the square matrix to have an added dimension on each side.

Therefore, $(3 \times 3)$ matrices are used for transformations in two dimensions and $(4 \times 4)$ are used for transformations in three dimensions.

This unified representation both accounts for ***translation***, and accommodates the larger class of ***projective transformations*** which includes perspective projection.

Since matrix multiplication only works if the two matrices involved have compatible shapes, this technique also require vectors and points that exhibit a modified structure. To be compatible with elevated matrices, our points and vectors must be granted an extra coordinate.

Points in homogeneous coordinates are interchangeable with Cartesian points so long as $w = 1$, while vectors in homogeneous coordinates maintain a $w = 0$.

We can learn alot by simply familiarizing ourselves with the relationship between certain patterns of component values in a $(4 \times 4)$ matrix and the spatial transformations that result.

First, the long awaited ***translation transformation***.

\begin{align}
    \begin{bmatrix}
        1 & 0 & 0 & b_{x} \\
        0 & 1 & 0 & b_{y} \\
        0 & 0 & 1 & b_{z} \\
        0 & 0 & 0 & 1 \\
    \end{bmatrix}
    \begin{bmatrix} x \\ y \\ z \\ 1 \end{bmatrix} =
    \begin{bmatrix} x + b_{x} \\ y + b_{y} \\ z + b_{z} \\ 1 \end{bmatrix}
\end{align}

This translation matrix applied to points moves them, but applied to a vector in homogeneous coordinates $(x,y,z,0)$, leaves the vector unchanged.

```python

```

107 - Transformations and Intersections/242 - Transformation Mathematics.ipynb


# Demo - 3D Poisson's equation

**Mikael Mortensen** (email: `mikaem@math.uio.no`), Department of Mathematics, University of Oslo.

Date: **April 13, 2018**

**Summary.** This is a demonstration of how the Python module [shenfun](https://github.com/spectralDNS/shenfun) can be used to solve a 3D Poisson
equation in a 3D tensor product domain that has homogeneous Dirichlet boundary
conditions in one direction and periodicity in the
remaining two. The solver described runs with MPI without any further
considerations required from the user. Spectral convergence, as shown in [Figure](#fig:3d:ct0), is demonstrated.
The demo is implemented in
a single Python file [dirichlet_poisson3D.py](https://github.com/spectralDNS/shenfun/blob/master/demo/dirichlet_poisson3D.py), and the numerical method is is described in more detail by J. Shen [[shen1]](#shen1) and [[shen95]](#shen95).

Convergence of 3D Poisson solvers for both Legendre and Chebyshev modified basis function.

## Poisson's equation

Poisson's equation is given as

$$
\begin{equation}
\nabla^2 u(\boldsymbol{x}) = f(\boldsymbol{x}) \quad \text{for }\, \boldsymbol{x}=(x, y, z) \in \Omega, \label{eq:3d:poisson} \tag{1}
\end{equation}
$$

$$
\begin{equation}
u(\pm 1 ,y, z) =0,
\label{_auto1} \tag{2}
\end{equation}
$$

$$
\begin{equation}
u(x, 2\pi, z) = u(x, 0, z),
\label{_auto2} \tag{3}
\end{equation}
$$

$$
\begin{equation}
u(x, y, 2\pi) = u(x, y, 0),
\label{_auto3} \tag{4}
\end{equation}
$$

where $u(\boldsymbol{x})$ is the solution and $f(\boldsymbol{x})$ is a function. The domain
$\Omega = [-1, 1]\times [0, 2\pi]^2$.

To solve Eq. ([1](#eq:3d:poisson)) with the Galerkin method we need smooth basis
functions, $v(\boldsymbol{x})$, that live
in the Hilbert space $H^1(\Omega)$ and that satisfy the given boundary
conditions. To this end we will use one basis function for the $x$-direction,
$\mathcal{X}(x)$,
one for the $y$-direction, $\mathcal{Y}(y)$, and one for the $z$-direction,
$\mathcal{Z}(z)$. And
then we create three-dimensional basis functions like

$$
v(x, y, z) = \mathcal{X}(x) \mathcal{Y}(y) \mathcal{Z} (z).
$$

The basis functions $\mathcal{Y}(y)$ and $\mathcal{Z}(z)$ are chosen as Fourier exponentials, since these
functions are periodic. Likewise, the basis functions $\mathcal{X}(x)$ are chosen as
modified Legendre or Chebyshev polynomials, using $\phi_l(x)$ to refer to either
one

$$
\begin{equation}
\mathcal{X}_l(x) = \phi_l(x) - \phi_{l+2}(x), \forall \, l \in \boldsymbol{l}^{N_0},
\label{_auto4} \tag{5}
\end{equation}
$$

$$
\begin{equation}
\mathcal{Y}_m(y) =  e^{\imath m y}, \forall \, m \in \boldsymbol{m}^{N_1},
\label{_auto5} \tag{6}
\end{equation}
$$

$$
\begin{equation}
\mathcal{Z}_n(z) = e^{\imath n z}, \forall \, n \in \boldsymbol{n}^{N_2},
\label{_auto6} \tag{7}
\end{equation}
$$

where the size of the discretized problem is $\boldsymbol{N} = (N_0, N_1, N_2)$,
$\boldsymbol{l}^{N_0} = (0, 1, \ldots, N_0-3)$, $\boldsymbol{m}^{N_1} =
(-N_1/2, -N_1/2+1, \ldots, N_1/2-1)$ and $\boldsymbol{n}^{N_2} = (-N_2/2, -N_2/2+1,
\ldots, N_2/2-1)$. However, due to [Hermitian symmetry](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.fft.rfft.html#numpy.fft.rfft), we only store $N_2/2+1$
wavenumbers in the $z$-direction, such that $\boldsymbol{n}^{N_2} = (0, 1, \ldots,
N_2/2)$. We refer to the Cartesian wavenumber mesh on vector form as $\boldsymbol{k}$:

$$
\boldsymbol{k} = \{(l, m, n)\, | \,(l, m, n)  \in \boldsymbol{l}^{N_0} \times \boldsymbol{m}^{N_1} \times \boldsymbol{n}^{N_2}\}.
$$

We have the one-dimensional spaces

$$
\begin{equation}
V^{N_0} = \text{span}\{ \mathcal{X}_l \}_{l\in\boldsymbol{l}^{N_0}},
\label{_auto7} \tag{8}
\end{equation}
$$

$$
\begin{equation}
V^{N_1} = \text{span}\{ \mathcal{Y}_m \}_{m\in\boldsymbol{m}^{N_1}},
\label{_auto8} \tag{9}
\end{equation}
$$

$$
\begin{equation}
V^{N_2} = \text{span}\{ \mathcal{Z}_n \}_{n\in\boldsymbol{n}^{N_2}},
\label{_auto9} \tag{10}
\end{equation}
$$

and from these we create a tensor product space $W^{\boldsymbol{N}}(\boldsymbol{x})$

$$
\begin{equation}
W^{\boldsymbol{N}}(\boldsymbol{x}) = V^{N_0}(x) \otimes V^{N_1}(y) \otimes V^{N_2}(z).
\label{_auto10} \tag{11}
\end{equation}
$$

And then we look for discrete solutions $u \in W^{\boldsymbol{N}}$ like

$$
\begin{equation}
u(\boldsymbol{x}) = \sum_{l\in \boldsymbol{l}^{N_0}} \sum_{m\in \boldsymbol{m}^{N_1}}\sum_{n\in
\boldsymbol{n}^{N_2}}\hat{u}_{lmn} \mathcal{X}_l(x) \mathcal{Y}_m(y) \mathcal{Z}_n(z), \label{eq:3d:u} \tag{12}
\end{equation}
$$

$$
\begin{equation}
 = \sum_{\boldsymbol{\textsf{k}} \in \boldsymbol{k}}\hat{u}_{\boldsymbol{\textsf{k}}} v_{\boldsymbol{\textsf{k}}}(\boldsymbol{x}),
\label{_auto11} \tag{13}
\end{equation}
$$

where $\hat{u}_{lmn}$ are components of the expansion coefficients for $u$ and
the second form, $\{\hat{u}_{\boldsymbol{\textsf{k}}}\}_{\boldsymbol{\textsf{k}}\in\boldsymbol{k}}$, is a shorter,
simplified notation, with sans-serif $\boldsymbol{\textsf{k}}=(l, m, n)$.
The expansion coefficients are the unknowns in the spectral Galerkin method.

We now formulate a variational problem using the Galerkin method: Find $u \in
W^{\boldsymbol{N}}$ such that

$$
\begin{equation}
\int_{\Omega} \nabla^2 u \, \overline{v} \, w\, \boldsymbol{dx} = \int_{\Omega} f \,
\overline{v}\, w\, \boldsymbol{dx} \quad
\forall v \, \in \, W^{\boldsymbol{N}}. \label{eq:3d:varform} \tag{14}
\end{equation}
$$

Here $\boldsymbol{dx}=dxdydz$, and the overline represents a complex conjugate, which is needed here because
the Fourier exponentials are complex functions.
The weighted integrals, weighted by $w(\boldsymbol{x})$, are called inner products, and a common notation is

$$
\begin{equation}
\int_{\Omega} u \, \overline{v} \, w\, \boldsymbol{dx} = \langle u, v\rangle _w.
\label{_auto12} \tag{15}
\end{equation}
$$

The integral can either be computed exactly, or with quadrature. The advantage
of the latter is that it is generally faster, and that non-linear terms may be
computed just as quickly as linear. For a linear problem, it does not make much of a difference, if any at all. Approximating the integral with quadrature, we obtain

$$
\begin{align*}
\int_{\Omega} u \, \overline{v} \, w\, \boldsymbol{dx} &\approx \langle u, v
\rangle_w^{\boldsymbol{N}},  \\
&\approx \sum_{i=0}^{N_0-1} \sum_{j=0}^{N_1-1}\sum_{k=0}^{N_2-1} u(x_i, y_j, z_k) \overline{v}(x_i, y_j, z_k) w(x_i, y_j, z_k),
\end{align*}
$$

where $w(\boldsymbol{x})$ now are the quadrature weights. The quadrature points
$\{x_i\}_{i=0}^{N_0-1}$ are specific to the chosen basis, and even within basis there
are two different choices based on which quadrature rule is selected, either
Gauss or Gauss-Lobatto. The quadrature points for the Fourier bases are the
uniform $\{y_j\}_{j=0}^{N_1-1}=2\pi j / N_1$ and $\{z_k\}_{k=0}^{N_2-1} = 2 \pi
k/N_2$.

Inserting for test function ([12](#eq:3d:u)) and trialfunction
$v_{pqr} = \mathcal{X}_{p} \mathcal{Y}_q \mathcal{Z}_r$ on the
left hand side of ([14](#eq:3d:varform)), we get

$$
\begin{align*}
\langle \nabla^2u, v \rangle_w^{\boldsymbol{N}} &= \left\langle \nabla^2\sum_{l\in \boldsymbol{l}^{N_0}}
\sum_{m\in \boldsymbol{m}^{N_1}}\sum_{n\in \boldsymbol{n}^{N_2}}\hat{u}_{lmn}
\mathcal{X}_{l} \mathcal{Y}_m \mathcal{Z}_n,
\mathcal{X}_{p} \mathcal{Y}_q \mathcal{Z}_r \right\rangle_w^{\boldsymbol{N}}, \\
    &= \left[\left(\mathcal{X}_l^{''}, \mathcal{X}_p \right)_w^N - (m^2+n^2)\left(\mathcal{X}_l, \mathcal{X}_p \right)_w^N  \right]\delta_{mq} \delta_{nr} \hat{u}_{lmn}, \\
    &= \left( a_{pl} - (m^2 + n^2)b_{pl}\right) \hat{u}_{lqr},
\end{align*}
$$

where the notation $(\cdot, \cdot)_w^{N_0}$

$$
\begin{equation}
b_{pl} = \left( \mathcal{X}_l, \mathcal{X}_p \right)_w^{N_0} = \sum_{i=0}^{N_0-1} \mathcal{X}_l(x_i)
\mathcal{X}_p(x_i) w(x_i),
\label{_auto13} \tag{16}
\end{equation}
$$

is used to represent an $L_2$ inner product along only the first, nonperiodic,
direction. The delta functions above come from integrating over the two periodic
directions, where we use constant weight functions $w=1/(2\pi)$ in the
inner products

$$
\begin{equation}
\int_0^{2\pi} \mathcal{Y}_m(y) \overline{\mathcal{Y}}_q(y) \frac{1}{2\pi} dy = \delta_{mq}, \label{eq:delta0} \tag{17}
\end{equation}
$$

$$
\begin{equation}
\int_0^{2\pi} \mathcal{Z}_n(z) \overline{\mathcal{Z}}_r(z) \frac{1}{2\pi} dz = \delta_{nr}. \label{eq:delta1} \tag{18}
\end{equation}
$$

The Kronecker delta-function $\delta_{ij}$ is one for $i=j$ and
zero otherwise.

The right hand side of Eq. ([14](#eq:3d:varform)) is computed as

$$
\begin{equation}
\tilde{f}_{pqr} = \left\langle f, \mathcal{X}_{p}
\mathcal{Y}_q \mathcal{Z}_r  \right \rangle_w^{\boldsymbol{N}},
\label{_auto14} \tag{19}
\end{equation}
$$

where a tilde is used because this is not a complete transform of the function
$f$, but only an inner product.

The linear system of equations to solve for the expansion coefficients can now
be found as follows

$$
\begin{equation}
\left(a_{lj} - (m^2+n^2)b_{lj}\right) \hat{u}_{jmn} =
\tilde{f}_{lmn}\quad \forall \, (l,m,n) \in \boldsymbol{k}. \label{eq:AB} \tag{20}
\end{equation}
$$

Now, when $\hat{\boldsymbol{u}} = \{\hat{u}_{\boldsymbol{\textsf{k}}}\}_{\boldsymbol{\textsf{k}} \in \boldsymbol{k}}$ is
found by solving this linear system over the
entire computational mesh, it may be
transformed to real space $u(\boldsymbol{x})$ using ([12](#eq:3d:u)). Note that the matrices
$A \in \mathbb{R}^{N_0-3 \times N_0-3}$ and $B \in \mathbb{R}^{N_0-3 \times N_0-3}$
differ for Legendre or Chebyshev bases, but
for either case they have a
special structure that allows for a solution to be found very efficiently
in the order of $\mathcal{O}(N_0-3)$ operations given $m$ and $n$, see
[[shen1]](#shen1) and [[shen95]](#shen95). Fast solvers for ([20](#eq:AB)) are implemented in `shenfun` for both bases.

### Method of manufactured solutions

In this demo we will use the method of manufactured
solutions to demonstrate spectral accuracy of the `shenfun` bases. To
this end we choose a smooth analytical function that satisfies the given boundary
conditions:

$$
\begin{equation}
u_e(x, y, z) = \left(\cos(4x) + \sin(2y) + \sin(4z)\right)(1-x^2). \label{eq:3d:u_e} \tag{21}
\end{equation}
$$

Sending $u_e$ through the Laplace operator, we obtain the right hand side

$$
\begin{equation}
 \nabla^2 u_e(x,y,z) = -16(1 - x^2) \cos(4 x) + 16 x \sin(4 x) - 2 \cos(4 x)
                  - (1-x^2)(4 \sin(2y) + 16\sin(4z)).  \label{eq:3d:solution} \tag{22}
\end{equation}
$$

Now, setting $f_e(\boldsymbol{x}) = \nabla^2 u_e(\boldsymbol{x})$ and solving for $\nabla^2
u(\boldsymbol{x}) = f_e(\boldsymbol{x})$, we can compare the numerical solution $u(\boldsymbol{x})$ with
the analytical solution $u_e(\boldsymbol{x})$ and compute error norms.

## Implementation

### Preamble

We will solve the Poisson problem using the [shenfun](https://github.com/spectralDNS/shenfun) Python module. The first thing needed
is then to import some of this module's functionality
plus some other helper modules, like [Numpy](https://numpy.org) and [Sympy](https://sympy.org):

```
from sympy import symbols, cos, sin, exp, lambdify
import numpy as np
from shenfun.tensorproductspace import TensorProductSpace
from shenfun import inner, div, grad, TestFunction, TrialFunction, Function, \
    project, Dx, FunctionSpace, comm, Array, chebyshev, dx, la
```

We use `Sympy` for the manufactured solution and `Numpy` for testing. MPI for
Python (`mpi4py`) is required for running the solver with MPI.

### Manufactured solution

The exact solution $u_e(x, y, z)$ and the right hand side $f_e(x, y, z)$ are created using `Sympy` as follows

```
x, y, z = symbols("x,y,z")
ue = (cos(4*x) + sin(2*y) + sin(4*z))*(1-x**2)
fe = ue.diff(x, 2) + ue.diff(y, 2) + ue.diff(z, 2)
```

These solutions are now valid for a continuous domain. The next step is thus to
discretize, using the computational mesh

$$
(x_i, y_j, z_k)\, \forall \, (i, j, k) \in [0, 1,\ldots, N_0-1] \times [0, 1, \ldots, N_1-1] \times [0, 1, \ldots, N_2-1]
$$

and a finite number of basis functions.

Note that it is not mandatory to use `Sympy` for the manufactured solution. Since the
solution is known ([22](#eq:3d:solution)), we could just as well simply use `Numpy`
to compute $f_e$. However, with `Sympy` it is much
easier to experiment and quickly change the solution.

### Discretization and MPI

We create three function spaces with given size, one for each dimension of the problem.
From these three spaces a [TensorProductSpace](https://shenfun.readthedocs.io/en/latest/shenfun.html#shenfun.tensorproductspace.TensorProductSpace) is created.

```
# Size of discretization
N = [14, 15, 16]

SD = FunctionSpace(N[0], 'Chebyshev', bc=(0, 0))
#SD = FunctionSpace(N[0], 'Legendre', bc=(0, 0))
K1 = FunctionSpace(N[1], 'Fourier', dtype='D')
K2 = FunctionSpace(N[2], 'Fourier', dtype='d')
T = TensorProductSpace(comm, (SD, K1, K2), axes=(0, 1, 2))
X = T.local_mesh()
```

Note that we can either choose a Legendre or a Chebyshev basis for the
nonperiodic direction. The
[TensorProductSpace](https://shenfun.readthedocs.io/en/latest/shenfun.html#shenfun.tensorproductspace.TensorProductSpace) class takes an MPI communicator as first argument and the
computational mesh is distributed internally using the `pencil` method. The
`T.local_mesh` method returns the mesh local to each processor. The `axes`
keyword determines the order of transforms going back and forth between real and
spectral space. With `axes=(0, 1, 2)` and a forward transform (from real space
to spectral, i.e., from $u$ to $\hat{u}$) axis 2 is transformed first and then 1
and 0, respectively.

The manufactured solution is created with Dirichlet boundary conditions in the
$x$-direction, and for this reason `SD` is the first space in `T`. We could just
as well have put the nonperiodic direction along either $y$- or $z$-direction,
though, but this would then require that the order of the transformed axes be
changed as well. For example, putting the Dirichlet direction along $y$, we
would need to create the tensorproductspace as

```Python
        T = TensorProductSpace(comm, (K1, SD, K2), axes=(1, 0, 2))
```

such that the Dirichlet direction is the last to be transformed. The reason for
this is that only the Dirichlet direction leads to matrices that need to be
inverted (or solved). And for this we need the entire data array along the Dirichlet
direction to be local to the processor. If the `SD` basis is the last to be
transformed, then the data will be aligned in this direction, whereas the other
two directions may both, or just one of them, be distributed.

Note that `X` is a list containing local values of the arrays $\{x_i\}_{i=0}^{N_0-1}$,
$\{y_j\}_{j=0}^{N_1-0}$ and $\{z_k\}_{k=0}^{N_2-1}$.
Now, it's not possible to run a jupyter notebook with more than one process,
but we can imagine running [the complete solver](https://github.com/spectralDNS/shenfun/blob/master/demo/dirichlet_poisson3D.py)
with 4 procesors and a processor mesh of shape $2\times 2$.
We would then get the following local slices for
each processor in spectral space

```Python
        print(comm.Get_rank(), T.local_slice())
        3 [slice(0, 14, None), slice(8, 15, None), slice(5, 9, None)]
        1 [slice(0, 14, None), slice(0, 8, None), slice(5, 9, None)]
        2 [slice(0, 14, None), slice(8, 15, None), slice(0, 5, None)]
        0 [slice(0, 14, None), slice(0, 8, None), slice(0, 5, None)]
```

where the global shape is $\boldsymbol{N}=(14, 15, 9)$ after taking advantage of
Hermitian symmetry in the $z$-direction. So, all processors have the complete first dimension available locally, as they
should. Furthermore, processor three owns the slices from $8:15$ and $5:9$ along
axes $y$ and $z$, respectively. Processor 2 owns slices $0:8$ and $0:5$ etc. In
real space the mesh is distributed differently. First of all the global mesh
shape is $\boldsymbol{N}=(14, 15, 16)$, and it is distributed along the first two
dimensions. The local slices can be inspected as

```Python
        print(comm.Get_rank(), T.local_slice(False))
        0 [slice(0, 7, None), slice(0, 8, None), slice(0, 16, None)]
        1 [slice(0, 7, None), slice(8, 15, None), slice(0, 16, None)]
        2 [slice(7, 14, None), slice(0, 8, None), slice(0, 16, None)]
        3 [slice(7, 14, None), slice(8, 15, None), slice(0, 16, None)]
```

Since two directions are distributed, both in spectral and real space, we say
that we have a two-dimensional decomposition (here a $2\times 2$ shaped
processor mesh) and the
MPI distribution is of type *pencil*. It is also possible to choose a *slab*
decomposition, where only one dimension of the array is distributed. This choice
needs to be made when creating the tensorproductspace as

```Python
        T = TensorProductSpace(comm, (SD, K1, K2), axes=(0, 1, 2), slab=True)
```

which would lead to a mesh that is distributed along $x$-direction in real space
and $y$-direction in spectral space. The local slices would then be

        print(comm.Get_rank(), T.local_slice()) # spectral space
        1 [slice(0, 14, None), slice(4, 8, None), slice(0, 9, None)]
        2 [slice(0, 14, None), slice(8, 12, None), slice(0, 9, None)]
        0 [slice(0, 14, None), slice(0, 4, None), slice(0, 9, None)]
        3 [slice(0, 14, None), slice(12, 15, None), slice(0, 9, None)]
        print(comm.Get_rank(), T.local_slice(False)) # real space
        3 [slice(11, 14, None), slice(0, 15, None), slice(0, 16, None)]
        0 [slice(0, 4, None), slice(0, 15, None), slice(0, 16, None)]
        2 [slice(8, 11, None), slice(0, 15, None), slice(0, 16, None)]
        1 [slice(4, 8, None), slice(0, 15, None), slice(0, 16, None)]

Note that the *slab* decomposition is usually the fastest choice. However, the maximum
number of processors with *slab* is $\min \{N_0, N_1\}$, whereas a *pencil*
approach can be used with up to $\min \{N_1(N_2/2+1), N_0 N_1\}$ processors.

### Variational formulation

The variational problem ([14](#eq:3d:varform)) can be assembled using `shenfun`'s
form language, which is perhaps surprisingly similar to FEniCS.

```
u = TrialFunction(T)
v = TestFunction(T)
# Get f on quad points
fj = Array(T, buffer=fe)
# Compute right hand side of Poisson equation
f_hat = inner(v, fj)
# Get left hand side of Poisson equation
matrices = inner(v, div(grad(u)))
```

The Laplacian operator is recognized as `div(grad)`. The `matrices` object is a
list of two tensor product matrices, stored as instances of the class [TPMatrix](https://shenfun.readthedocs.io/en/latest/shenfun.html#shenfun.matrixbase.TPMatrix).
The two tensor product matrices represents

$$
a_{pl} \delta_{mq} \delta_{nr}\, \text{ and }\, -(m^2 + n^2)b_{pl} \delta_{mq} \delta_{nr}
$$

from Eqs. ([20](#eq:AB)), ([17](#eq:delta0)) and ([18](#eq:delta1)).
The second matrix ($-(m^2 + n^2)b_{pl} \delta_{mq} \delta_{nr}$) has an
attribute `scale` that is equal to $-(m^2+n^2)$.
This `scale` is stored as a numpy array of shape $(1, 15, 9)$, representing the set
$\{-(m^2+n^2): (m, n) \in \boldsymbol{m}^{N_1} \times \boldsymbol{n}^{N_2}\}$. Note that $\boldsymbol{n}^{N_2}$ is stored
simply as an array of length $N_2/2+1$ (here 9), since the transform in direction $z$
takes a real signal and transforms it taking advantage of Hermitian symmetry,
see [rfft](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.fft.rfft.html).

### Solve linear equations

Finally, solve linear equation system and transform solution from spectral
$\hat{u}_{\boldsymbol{\textsf{k}}}$ vector to the real space $u(\boldsymbol{x})$ and then check how the solution corresponds with the exact solution $u_e$.

```
# Create Helmholtz linear algebra solver
Solver = chebyshev.la.Helmholtz
#Solver = la.SolverGeneric1ND # For Legendre
H = Solver(matrices)

# Solve and transform to real space
u_hat = Function(T)           # Solution spectral space
u_hat = H(u_hat, f_hat)       # Solve
uq = T.backward(u_hat)

# Compare with analytical solution
uj = Array(T, buffer=ue)
error = comm.reduce(np.linalg.norm(uj-uq)**2)
if comm.Get_rank() == 0:
    print("Error=%2.16e" %(np.sqrt(error)))
```

### Convergence test

To do a convergence test we will now create a function `main`, that takes the
number of quadrature points as parameter, and prints out
the error.

```
def main(N, family='Chebyshev'):
    Solver = chebyshev.la.Helmholtz if family.lower() == 'chebyshev' else la.SolverGeneric1ND
    SD = FunctionSpace(N, family=family, bc=(0, 0))
    K1 = FunctionSpace(N, family='F', dtype='D')
    K2 = FunctionSpace(N, family='F', dtype='d')
    T = TensorProductSpace(comm, (SD, K1, K2), axes=(0, 1, 2))

    u = TrialFunction(T)
    v = TestFunction(T)

    # Get f on quad points
    fj = Array(T, buffer=fe)

    # Compute right hand side of Poisson's equation
    f_hat = Function(T)
    f_hat = inner(v, fj, output_array=f_hat)
    if family == 'legendre':
        f_hat *= -1.

    # Get left hand side of Poisson equation
    if family.lower() == 'chebyshev':
        matrices = inner(v, div(grad(u)))
    else:
        matrices = inner(grad(v), grad(u))

    # Create Helmholtz linear algebra solver
    H = Solver(matrices)

    # Solve and transform to real space
    u_hat = Function(T)           # Solution spectral space
    u_hat = H(f_hat, u_hat)       # Solve

    uj = Array(T)
    uj = u_hat.backward(uj)

    # Compare with analytical solution
    ua = Array(T, buffer=ue)
    #l2_error = np.linalg.norm(uj-ua)
    L2_error = np.sqrt(dx((uj-ua)**2))
    return L2_error
```

For example, we find the error of a Chebyshev discretization
using 12 quadrature points as

```
main(12, 'Chebyshev')
```

To get the convergence we call `main` for a list
of $N=[12, 16, \ldots, 48]$, and collect the errornorms in
arrays to be plotted. The error can be plotted using
[matplotlib](https://matplotlib.org), and the generated
figure is also shown in this demos summary.

```
%matplotlib inline

import matplotlib.pyplot as plt

N = range(12, 50, 4)
error = {}
for basis in ('legendre', 'chebyshev'):
    error[basis] = []
    for i in range(len(N)):
        errN = main(N[i], basis)
        error[basis].append(errN)

plt.figure(figsize=(6, 4))
for basis, col in zip(('legendre', 'chebyshev'), ('r', 'b')):
    plt.semilogy(N, error[basis], col, linewidth=2)
plt.title('Convergence of Poisson solvers 1D')
plt.xlabel('N')
plt.ylabel('Error norm')
plt.legend(('Legendre', 'Chebyshev'))
plt.show()
```

The spectral convergence is evident and we can see that
after $N=24$ roundoff errors dominate as the errornorm trails off around $10^{-14}$.

## Complete solver

A complete solver, that can use either Legendre or Chebyshev bases, chosen as a
command-line argument, can also be found [here](https://github.com/spectralDNS/shenfun/blob/master/demo/dirichlet_poisson3D.py).

1.  **J. Shen**.
    Efficient Spectral-Galerkin Method I. Direct Solvers of Second- and Fourth-Order Equations Using Legendre Polynomials,
    *SIAM Journal on Scientific Computing*,
    15(6),
    pp. 1489-1505,
    [doi: 10.1137/0915089](http://dx.doi.org/10.1137/0915089),
    1994.

2.  **J. Shen**.
    Efficient Spectral-Galerkin Method II. Direct Solvers of Second- and Fourth-Order Equations Using Chebyshev Polynomials,
    *SIAM Journal on Scientific Computing*,
    16(1),
    pp. 74-87,
    [doi: 10.1137/0916006](http://dx.doi.org/10.1137/0916006),
    1995.


# Tutoriel MID Contrôle
<u>Objectif</u> : Illustrer les propriétés de la MID sous la forme contrôle.

Concrètement, nous considérons un système qui a un controle par une loi de commande retardée (expression mathématique) et l'on prend compte et tirons avantage de ce retard dans le calcul des gains de la loi de commande retardée pour stabiliser le système.

<u> Informations importantes </u> :

- Stabilité : Lorsque l'on a l'équation décrivant un système dans l'espace de Laplace (transformée de Laplace), la stabilité est traduite par l'ensemble des racines (complexes) de l'équation ayant leurs parties réelles négatives. On dit que les racines sont à gauche (de l'axe des ordonnées).
- Transformée de Laplace (juste pour info) : Dans notre cas, la partie la plus importante dans la transformée de Laplace est la dérivation, c'est ce qui nous permet de passer d'une équation différentielle à une équation polynomiale. L'idée très grossière est qu'une dérivée d'ordre $n$ dans l'espace "classique"/"temporel" correspond à une multiplication par un terme $s^n$ dans l'espace de Laplace (fréquentiel). Donc, par exemple, $4\times k \times y''(t)\rightarrow 4ks^2y(s)$

On traite une équation différentielle du type :
$$
\frac{d^n y(t)}{dt^n} + \sum\limits_{i=0}^{n-1}a_i\frac{d^i y(t)}{dt^i} + \sum\limits_{j=0}^m \alpha_j\frac{d^j y(t-\tau)}{dt^j}=0
$$

- $n, m\in\mathbb{N}, n>m$
- $a_i, \alpha_j \in \mathbb{R}$
- $s\in\mathbb{C}$,
- $\tau \in \mathbb{R}$

Par une transformée de Laplace :
$$
Q(s) = s^n+a_{n-1}s^{n-1}+...+a_0 + \left[\alpha_ms^m+...+\alpha_0\right]e^{-s\tau}
$$

Cette équation est polynomiale et non différentielle ce qui la rend beaucoup plus simple à traiter.

Cette fois-ci, on considère que l'on connait les paramètres du systèmes $a_i$ ainsi que l'un des hyperparamètres $s_0$ ou $\tau$. Ainsi, nous allons utiliser la MID pour trouver les $\alpha_j$ et l'hyperparamètre manquant.

Le code suivant a été développé sur la base du pseudo-code de la MID Générique, décrit par Franck, puis adapté en version interactive.

Version : 1.1 (22/04/2021) - test pull

```python
from IPython.core.display import display, HTML
display(HTML("<style>div.output_scroll { height: 44em; }</style>"))
%matplotlib notebook
import numpy as np
import sympy as sp
import cxroots as cx
import matplotlib.pyplot as plt
import ipywidgets as widgets
from IPython.display import display
import matplotlib.cm as cm

def compute_admissibilite(n,m,value_s0,value_tau):

    s = sp.symbols('s')  # define variable s for our problem to be solved
    tau = sp.symbols('tau')  # define variable tau : delay

    a = sp.symbols(["a{:d}".format(i) for i in range(n)], real=True)
    alpha = sp.symbols(["alpha{:d}".format(i) for i in range(m + 1)], real=True)
    avalue = [1, 1]

    Polynomial = s**n + np.array(a).dot([s**i for i in range(n)]) # Revient à faire s^n + a_{n-1}^{n-1}...
    Delayed = np.array(alpha).dot([s**i for i in range(m+1)])*sp.exp(-s*tau) # Revient à faire
    #b^m*s^m + b_{m-1}^{m-1}...
    Q = Polynomial + Delayed

    SysDerivatif = [Q]
    for i in range(m+1):
        DerniereDerivee = SysDerivatif[-1]
        SysDerivatif.append(DerniereDerivee.diff(s)) # Dérivée par rapport à s

    sol = sp.linsolve(SysDerivatif[:-1], alpha).args[0] # Solveur selon les alpha et les a

    finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)}) #remplace les coeffs
    finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, avalue)})
    solS0 = finaleq.subs({tau : value_tau})
    solS0 = sp.solve(solS0)
    solS0eval = [i.evalf() for i in solS0]

    computedS0 = solS0[1]
    alpha_num = sol.subs({asymb: aval for asymb, aval in zip(a, avalue)})
    alpha_num = alpha_num.subs({s : computedS0})
    alpha_num = alpha_num.subs({tau : value_tau})
    alpha_num_eval = [i.evalf() for i in alpha_num]
    alpha_sens = alpha_num_eval

    finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)}) #remplace les coeffs
    finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, avalue)})
    solTau = finaleq.subs({s : value_s0})
    solTau = sp.solve(solTau)

    computedTau = solTau[0]
    alpha_num = sol.subs({asymb: aval for asymb, aval in zip(a, avalue)})
    alpha_num = alpha_num.subs({tau : computedTau})
    alpha_num = alpha_num.subs({s : value_s0})
    alpha_num_eval = [i.evalf() for i in alpha_num]

    polyAdm = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)})
    polyAdm = polyAdm.subs({asymb: aval for asymb, aval in zip(a, avalue)})
    polyAdm = sp.simplify(polyAdm)

    s0range = np.arange(-10, 0, 0.01)
    taurange = np.arange(0, 10, 0.01)

    func = sp.lambdify([s, tau], polyAdm)

    return s0range,taurange,polyAdm,s,tau,Q,avalue,alpha_sens,SysDerivatif,alpha,a

def getRoots(m, Q, dominancy, delay, avalue, alphavalue, xwindow, ywindow,s,a,alpha,tau):
    derivees = [Q, Q.diff(s)]
    for i in range(len(derivees)) :
        derivees[i] = derivees[i].subs({ai: ai_num for ai, ai_num in zip(a, avalue)})
        derivees[i] = derivees[i].subs({alphai: alphai_num for alphai, alphai_num in zip(alpha, alphavalue)})
        derivees[i] = derivees[i].subs({tau : delay})
    func = [sp.lambdify(s, i) for i in derivees]
    rect = cx.Rectangle(xwindow, ywindow)
    root_count = rect.count_roots(func[0])
    roots = rect.roots(func[0], func[1], rootErrTol=1e-5, absTol=1e-5, M=m+2)
    xroot, yroot = np.real(roots[0]), np.imag(roots[0])
    return xroot, yroot

def solve_tau_connu(tau_val, acoef, Q,m,s,SysDerivatif,alpha,a,tau):
    sys = [Q]
    for i in range(m+1):
        DerniereDerivee = sys[-1]
        sys.append(DerniereDerivee.diff(s)) # Dérivée par rapport à s
    sol = sp.linsolve(SysDerivatif[:-1], alpha).args[0]
    finaleq = SysDerivatif[-1].subs({alph : alphacoef for alph, alphacoef in zip(alpha, sol)})
    finaleq = finaleq.subs({asymb: aval for asymb, aval in zip(a, acoef)})
    solS0 = finaleq.subs({tau : tau_val})
    solS0 = sp.solve(solS0)
    solS0eval = [i.evalf() for i in solS0]
    try :
        solution = max([i for i in solS0eval if i<0])
    except Exception :
        traceback.print_exc()
    return solution

def compute_sensibilite(value_tau,Q,m,s,SysDerivatif,alpha,a,tau):
    tau_nominal = value_tau
    step = 1e-2
    nbIt = 10
    values = [0] + [-step * i for i in range(1, nbIt + 1)] + [step * i for i in range(1, nbIt + 1)]
    values.sort()
    tau_sens = []
    s0_sens = []
    for value in values :
        tau_sens.append(tau_nominal + value)
        s0_sens.append(solve_tau_connu(tau_sens[-1], [2, 1], Q,m,s,SysDerivatif,alpha,a,tau))

    sensIterations = len(tau_sens)
    bList = np.linspace(start=255, stop=0, num=sensIterations//2) #Blue to black
    rList = np.linspace(start=0, stop=255, num=sensIterations//2) #black to red
    if len(bList)+len(rList)==sensIterations-1 :
        zerosToFill = [0] * (sensIterations//2 + 1)
    else :
        zerosToFill = [0] * (sensIterations//2 )
    bList = [*bList, *zerosToFill]
    rList = [*zerosToFill, *rList]
    colorArray = []
    for i in range(len(tau_sens)):
        colorArray.append([rList[i], 0, bList[i]])
    colorArray = np.array(colorArray)

    return tau_sens,s0_sens,colorArray
```

<style>div.output_scroll { height: 44em; }</style>

```python
output_admissibilite = widgets.Output()
output_equation = widgets.Output()
output_sensibilite = widgets.Output()

slider_n = widgets.IntSlider(min=0,max=2,step=1,description='n :',value=2)
slider_m = widgets.IntSlider(min=0,max=3,step=1,description='m :',value=1)
FloatText_s0 = widgets.BoundedFloatText(value=-1.5,min=-2.0,max=0.0,step=0.1,description='s0:',disabled=False)
FloatText_tau = widgets.BoundedFloatText(value=0.73,min=0.50,max=1.00,step=0.01,description='tau:',disabled=False)

def common_slider(n,m,value_s0,value_tau):
    output_admissibilite.clear_output()
    output_equation.clear_output()
    output_sensibilite.clear_output()

    with output_admissibilite:

        # additional calculations for the plot
        s0range,taurange,polyAdm,s,tau,Q,avalue,alpha_sens,SysDerivatif,alpha,a = compute_admissibilite(n,m,value_s0,value_tau)

        s0range = np.arange(-10, 0, 0.01)
        taurange = np.arange(0, 10, 0.01)

        func = sp.lambdify([s, tau], polyAdm)

        fig, ax = plt.subplots()
        X, Y = np.meshgrid(s0range, taurange)
        z = func(X, Y)
        CS = ax.contour(X, Y, z, [0])
        ax.grid()
        plt.xlabel(r"$s_0$")
        plt.ylabel(r"$\tau$")
        plt.title("Admissibility plot")

    with output_equation :
        print ("Eq = ",polyAdm)

    with output_sensibilite :

        # additional calculations for the plot
        tau_sens,s0_sens,colorArray = compute_sensibilite(value_tau,Q,m,s,SysDerivatif,alpha,a,tau)

        scaler = lambda x : [i / 255 for i in x]

        plt.figure()
        for i in range(len(tau_sens)):
            xroot, yroot = getRoots(1, Q, s0_sens[i], tau_sens[i],
                                    avalue, alpha_sens, [-5, 5], [-5, 5],s,a,alpha,tau)
            plt.scatter(xroot, yroot, c=[scaler(colorArray[i])])

        plt.grid()
        plt.xlabel(r"$Re(s)$")
        plt.ylabel(r"$Im(s)$")
        tau_sens.sort()
        plt.title(r"Sensitivity plot, $\tau \in $"+f"[{tau_sens[0]}, {tau_sens[-1]}]")

def slider_n_eventhandler(change):
    common_slider(change.new, slider_m.value, FloatText_s0.value, FloatText_tau.value)
def slider_m_eventhandler(change):
    common_slider(slider_n.value, change.new, FloatText_s0.value, FloatText_tau.value)
def FloatText_s0_eventhandler(change):
    common_slider(slider_n.value, slider_m.value, change.new, FloatText_tau.value)
def FloatText_tau_eventhandler(change):
    common_slider(slider_n.value, slider_m.value, FloatText_s0.value, change.new)

slider_n.observe(slider_n_eventhandler, names='value')
slider_m.observe(slider_m_eventhandler, names='value')
FloatText_s0.observe(FloatText_s0_eventhandler, names='value')
FloatText_tau.observe(FloatText_tau_eventhandler, names='value')

input_widgets = widgets.HBox([slider_n, slider_m, FloatText_s0, FloatText_tau])

tab = widgets.Tab([output_admissibilite, output_equation, output_sensibilite])
tab.set_title(0, 'Plot Admissibility ')
tab.set_title(1, 'Output equation')
tab.set_title(2, 'Plot sensitivity')

dashboard = widgets.VBox([widgets.Label('Insert degree of polynomial n, degree of delay polynomial m, s0 and tau :'),input_widgets, tab])
```

```python
display(dashboard)
```

    VBox(children=(Label(value='Insert degree of polynomial n, degree of delay polynomial m, s0 and tau :'), HBox(…


```python
import matplotlib.pyplot as plt
from mpl_toolkits import mplot3d
import numpy as np
import sympy as sym
```

```python
def f(X, n, q, b, c):
    Z = np.zeros(len(X))
    for i in range(len(X)):
        for j in range(int(n)):
            for k in range(int(n)):
                Z[i] += q[j][k]*X[i][j]*X[i][k]

        for j in range(int(n)):
            Z[i] += b[j]*X[i][j]
    return Z
```

```python
def f_mesh(X, Y, q, b, c):
    Z = np.zeros(len(X))
    Z = q[0][0]*X*X + q[0][1]*X*Y + q[1][0]*Y*X + q[1][1]*Y*Y + b[0]*X + b[1]*Y + c
    return Z
```

```python
def f2(X, Y, n, q, b, c):
    Z = q[0][0]*X*X + q[0][1]*X*Y + q[1][0]*Y*X + q[1][1]*Y*Y + b[0]*X + b[1]*Y + int(c[0])
    return Z
```

```python
def adam(X_new, X1, Y1, Z1, n, q, b, c, alpha=0.1, beta1=0.9, beta2=0.99, eps=0.000000001, precision=0.0001, max_iter=200):
    X_old = np.zeros((1, 2))
    X_new = np.zeros((1, 2))
    dfr = np.zeros((1, 2))

    X_new[0][0] = 4.9
    X_new[0][1] = 4.9
    i = 0
    Xs = np.zeros((max_iter, 2))
    Ys = np.zeros(max_iter)

    V = np.zeros((max_iter + 1, 2))
    S = np.zeros((max_iter + 1, 2))

    V_corr = np.zeros((1, 2))
    S_corr = np.zeros((1, 2))

    x, y = sym.symbols('x y')
    df1 = sym.diff(f2(x, y, n, q, b, c), x)
    df2 = sym.diff(f2(x, y, n, q, b, c), y)

    while np.all(abs(X_new - X_old)) > precision and max_iter > i:
        Xs[i] = X_new
        Ys[i] = f2 (X_new[0][0], X_new[0][1], n, q, b, c)

        X_old = X_new

        dfr[0][0] = df1.evalf(subs={x: X_old[0][0], y: X_old[0][1]})
        dfr[0][1] = df2.evalf(subs={x: X_old[0][0], y: X_old[0][1]})

        i += 1
        V[i] = beta1 * V[i-1] + (1 - beta1) * dfr # momentum
        S[i] = beta2 * S[i-1] + (1 - beta2) * np.power(dfr, 2) # rmsprop

        V_corr = V[i]/(1 - np.power(beta1, i))
        S_corr = S[i]/(1 - np.power(beta2, i))

        X_new = X_new - alpha * V_corr / (np.sqrt(S_corr) + eps)

        alpha *= 0.99
        # print("V: {}, S:{}, V_corr: {}, S_corr: {}, dfr: {}, X_new: {}".format(V[i], S[i], V_corr, S_corr, dfr, X_new))

    print("Finished with {} step".format(i))
    if (i < max_iter):
        Xs[i] = X_new
        Ys[i] = f2(X_new[0][0], X_new[0][1], n, q, b, c)

        for j in range(max_iter - 1, i, -1):
            Xs = np.delete(Xs, j, axis=0)
            Ys = np.delete(Ys, j, axis=0)
    return Xs, Ys

```

```python
X1 = np.arange(-5, 5, 0.1)
Y1 = np.arange(-5, 5, 0.1)
Z1 = np.zeros(len(X1))

X_new = np.zeros((100,2))

for i in range(len(X1)):
    X_new[i][0] = X1[i]
    X_new[i][1] = Y1[i]

Z1 = f(X_new, n, q, b, c)

x_list, y_list = adam(X_new, X1, Y1, Z1, n, q, b, c)
```

    X_new:  [[4.8 4.8]]
    X_new:  [[4.70105385 4.70105385]]
    X_new:  [[4.60318665 4.60318665]]
    X_new:  [[4.50642288 4.50642288]]
    X_new:  [[4.41078632 4.41078632]]
    X_new:  [[4.31629993 4.31629993]]
    X_new:  [[4.22298578 4.22298578]]
    X_new:  [[4.13086498 4.13086498]]
    X_new:  [[4.03995758 4.03995758]]
    X_new:  [[3.95028253 3.95028253]]
    X_new:  [[3.86185756 3.86185756]]
    X_new:  [[3.77469921 3.77469921]]
    X_new:  [[3.68882268 3.68882268]]
    X_new:  [[3.60424188 3.60424188]]
    X_new:  [[3.52096934 3.52096934]]
    X_new:  [[3.43901618 3.43901618]]
    X_new:  [[3.35839212 3.35839212]]
    X_new:  [[3.27910545 3.27910545]]
    X_new:  [[3.20116304 3.20116304]]
    X_new:  [[3.12457031 3.12457031]]
    X_new:  [[3.04933126 3.04933126]]
    X_new:  [[2.97544849 2.97544849]]
    X_new:  [[2.90292319 2.90292319]]
    X_new:  [[2.8317552 2.8317552]]
    X_new:  [[2.76194304 2.76194304]]
    X_new:  [[2.69348389 2.69348389]]
    X_new:  [[2.6263737 2.6263737]]
    X_new:  [[2.56060719 2.56060719]]
    X_new:  [[2.49617791 2.49617791]]
    X_new:  [[2.43307828 2.43307828]]
    X_new:  [[2.37129965 2.37129965]]
    X_new:  [[2.31083234 2.31083234]]
    X_new:  [[2.25166569 2.25166569]]
    X_new:  [[2.19378815 2.19378815]]
    X_new:  [[2.13718728 2.13718728]]
    X_new:  [[2.08184985 2.08184985]]
    X_new:  [[2.02776189 2.02776188]]
    X_new:  [[1.97490869 1.97490869]]
    X_new:  [[1.92327494 1.92327494]]
    X_new:  [[1.87284472 1.87284472]]
    X_new:  [[1.82360158 1.82360158]]
    X_new:  [[1.77552857 1.77552857]]
    X_new:  [[1.72860832 1.72860832]]
    X_new:  [[1.68282305 1.68282305]]
    X_new:  [[1.63815465 1.63815465]]
    X_new:  [[1.59458472 1.59458472]]
    X_new:  [[1.55209459 1.55209459]]
    X_new:  [[1.51066538 1.51066538]]
    X_new:  [[1.47027803 1.47027803]]
    X_new:  [[1.43091336 1.43091336]]
    X_new:  [[1.39255207 1.39255207]]
    X_new:  [[1.3551748 1.3551748]]
    X_new:  [[1.31876215 1.31876215]]
    X_new:  [[1.28329473 1.28329473]]
    X_new:  [[1.24875315 1.24875315]]
    X_new:  [[1.21511808 1.21511808]]
    X_new:  [[1.18237028 1.18237028]]
    X_new:  [[1.15049057 1.15049057]]
    X_new:  [[1.11945992 1.11945992]]
    X_new:  [[1.08925943 1.08925943]]
    X_new:  [[1.05987036 1.05987036]]
    X_new:  [[1.03127413 1.03127413]]
    X_new:  [[1.00345237 1.00345237]]
    X_new:  [[0.9763869 0.9763869]]
    X_new:  [[0.95005976 0.95005976]]
    X_new:  [[0.92445323 0.92445323]]
    X_new:  [[0.89954981 0.89954981]]
    X_new:  [[0.87533226 0.87533226]]
    X_new:  [[0.8517836 0.8517836]]
    X_new:  [[0.8288871 0.8288871]]
    X_new:  [[0.80662633 0.80662633]]
    X_new:  [[0.78498511 0.78498511]]
    X_new:  [[0.76394754 0.76394754]]
    X_new:  [[0.74349803 0.74349803]]
    X_new:  [[0.72362126 0.72362126]]
    X_new:  [[0.70430221 0.70430221]]
    X_new:  [[0.68552613 0.68552613]]
    X_new:  [[0.66727861 0.6672786 ]]
    X_new:  [[0.64954548 0.64954548]]
    X_new:  [[0.63231291 0.63231291]]
    X_new:  [[0.61556734 0.61556734]]
    X_new:  [[0.59929552 0.59929552]]
    X_new:  [[0.58348447 0.58348447]]
    X_new:  [[0.56812152 0.56812152]]
    X_new:  [[0.5531943 0.5531943]]
    X_new:  [[0.53869069 0.53869069]]
    X_new:  [[0.52459888 0.52459888]]
    X_new:  [[0.51090735 0.51090735]]
    X_new:  [[0.49760483 0.49760483]]
    X_new:  [[0.48468034 0.48468034]]
    X_new:  [[0.47212318 0.47212318]]
    X_new:  [[0.4599229 0.4599229]]
    X_new:  [[0.44806933 0.44806933]]
    X_new:  [[0.43655255 0.43655255]]
    X_new:  [[0.4253629 0.4253629]]
    X_new:  [[0.41449096 0.41449096]]
    X_new:  [[0.40392757 0.40392757]]
    X_new:  [[0.39366382 0.39366382]]
    X_new:  [[0.38369101 0.38369101]]
    X_new:  [[0.3740007 0.3740007]]
    X_new:  [[0.36458466 0.36458466]]
    X_new:  [[0.3554349 0.3554349]]
    X_new:  [[0.34654364 0.34654364]]
    X_new:  [[0.33790332 0.33790332]]
    X_new:  [[0.32950658 0.32950658]]
    X_new:  [[0.32134628 0.32134628]]
    X_new:  [[0.31341547 0.31341547]]
    X_new:  [[0.30570741 0.30570741]]
    X_new:  [[0.29821553 0.29821553]]
    X_new:  [[0.29093347 0.29093347]]
    X_new:  [[0.28385504 0.28385504]]
    X_new:  [[0.27697422 0.27697422]]
    X_new:  [[0.27028519 0.27028519]]
    X_new:  [[0.26378227 0.26378227]]
    X_new:  [[0.25745997 0.25745997]]
    X_new:  [[0.25131295 0.25131295]]
    X_new:  [[0.24533602 0.24533602]]
    X_new:  [[0.23952416 0.23952416]]
    X_new:  [[0.23387248 0.23387248]]
    X_new:  [[0.22837624 0.22837624]]
    X_new:  [[0.22303086 0.22303086]]
    X_new:  [[0.21783187 0.21783187]]
    X_new:  [[0.21277495 0.21277495]]
    X_new:  [[0.20785589 0.20785589]]
    X_new:  [[0.20307064 0.20307064]]
    X_new:  [[0.19841524 0.19841524]]
    X_new:  [[0.19388587 0.19388587]]
    X_new:  [[0.1894788 0.1894788]]
    X_new:  [[0.18519045 0.18519045]]
    X_new:  [[0.18101732 0.18101732]]
    X_new:  [[0.17695601 0.17695601]]
    X_new:  [[0.17300326 0.17300326]]
    X_new:  [[0.16915587 0.16915587]]
    X_new:  [[0.16541077 0.16541077]]
    X_new:  [[0.16176495 0.16176495]]
    X_new:  [[0.15821551 0.15821551]]
    X_new:  [[0.15475965 0.15475965]]
    X_new:  [[0.15139463 0.15139463]]
    X_new:  [[0.14811782 0.14811782]]
    X_new:  [[0.14492665 0.14492665]]
    X_new:  [[0.14181862 0.14181862]]
    X_new:  [[0.13879135 0.13879135]]
    X_new:  [[0.13584248 0.13584248]]
    X_new:  [[0.13296977 0.13296977]]
    X_new:  [[0.130171 0.130171]]
    X_new:  [[0.12744406 0.12744406]]
    X_new:  [[0.12478688 0.12478688]]
    X_new:  [[0.12219747 0.12219747]]
    X_new:  [[0.11967387 0.11967387]]
    X_new:  [[0.11721423 0.11721423]]
    X_new:  [[0.11481671 0.11481671]]
    X_new:  [[0.11247954 0.11247954]]
    X_new:  [[0.11020102 0.11020102]]
    X_new:  [[0.10797948 0.10797948]]
    X_new:  [[0.10581332 0.10581332]]
    X_new:  [[0.10370097 0.10370097]]
    X_new:  [[0.10164092 0.10164092]]
    X_new:  [[0.0996317 0.0996317]]
    X_new:  [[0.0976719 0.0976719]]
    X_new:  [[0.09576012 0.09576012]]
    X_new:  [[0.09389503 0.09389503]]
    X_new:  [[0.09207534 0.09207534]]
    X_new:  [[0.09029978 0.09029978]]
    X_new:  [[0.08856714 0.08856714]]
    X_new:  [[0.08687623 0.08687623]]
    X_new:  [[0.0852259 0.0852259]]
    X_new:  [[0.08361504 0.08361504]]
    X_new:  [[0.08204256 0.08204256]]
    X_new:  [[0.08050743 0.08050743]]
    X_new:  [[0.07900861 0.07900861]]
    X_new:  [[0.07754514 0.07754514]]
    X_new:  [[0.07611604 0.07611604]]
    X_new:  [[0.07472039 0.07472039]]
    X_new:  [[0.0733573 0.0733573]]
    X_new:  [[0.07202587 0.07202587]]
    X_new:  [[0.07072528 0.07072528]]
    X_new:  [[0.06945468 0.06945468]]
    X_new:  [[0.06821329 0.06821329]]
    X_new:  [[0.06700033 0.06700033]]
    X_new:  [[0.06581504 0.06581504]]
    X_new:  [[0.06465669 0.06465669]]
    X_new:  [[0.06352458 0.06352458]]
    X_new:  [[0.06241802 0.06241802]]
    X_new:  [[0.06133633 0.06133633]]
    X_new:  [[0.06027886 0.06027886]]
    X_new:  [[0.059245 0.059245]]
    X_new:  [[0.05823411 0.05823411]]
    X_new:  [[0.05724562 0.05724562]]
    X_new:  [[0.05627894 0.05627894]]
    X_new:  [[0.05533351 0.05533351]]
    X_new:  [[0.05440878 0.05440878]]
    X_new:  [[0.05350423 0.05350423]]
    X_new:  [[0.05261934 0.05261934]]
    X_new:  [[0.05175362 0.05175362]]
    X_new:  [[0.05090657 0.05090657]]
    X_new:  [[0.05007773 0.05007773]]
    X_new:  [[0.04926663 0.04926663]]
    X_new:  [[0.04847284 0.04847284]]
    X_new:  [[0.04769592 0.04769592]]
    X_new:  [[0.04693544 0.04693544]]
    Finished with 200 step

```python
n = input("Enter power of your function: ")
q = []
for i in range(int(n)):
    q.append(input("Enter the function's coefficient matrix q's row q[{}]: ".format(i)).split())
b = input("Enter the function's coefficient b matrix: ").split()
c = input("Enter the function's constant c: ")

for i in range(int(n)):
    q[i] = list(map(float, q[i]))
b = list(map(float, b))
c = list(map(float, c))
```

    Enter power of your function: 2
    Enter the function's coefficient matrix q's row q[0]: 1 0
    Enter the function's coefficient matrix q's row q[1]: 0 2
    Enter the function's coefficient b matrix: 0 0
    Enter the function's constant c: 0

```python
X1, Y1 = np.meshgrid(X1, Y1)
Z1 = f_mesh(X1, Y1, q, b, c)

X, Y = zip(*x_list)
Z = y_list

ax = plt.subplots(nrows=1, ncols=1, figsize=(10,10))
cs = plt.contour(X1, Y1, Z1)
plt.clabel(cs, inline=1, fontsize=10)
colors = ['b', 'g', 'm', 'c', 'orange']
for j in range(1, len(X)):
    ax[1].annotate('', xy=(X[j], Y[j]), xytext=(X[j-1], Y[j-1]),
                arrowprops={'arrowstyle': '->', 'color': 'r', 'lw': 1},
                va='center', ha='center')
ax[1].scatter(X, Y, s=40, lw=0)
ax[1].set_xlabel('X')
ax[1].set_ylabel('Y')
ax[1].set_title('Minimizing function')
```

```python
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_surface(X1, Y1, Z1, rstride=1, cstride=1,
                cmap='viridis', edgecolor='none')
ax.set_title('surface')
plt.show()
```

```python

```