# Exercise session nº 2
---
# Sonic Hedgehog Signaling Gradient Readout in the Vertebrate Neural Tube

__*Sacha Ichbiah, 31/01/22, ENS Paris*__

This subject is extracted from :
> N. Balaskas et Al., *Gene Regulatory Logic for Reading the Sonic Hedgehog Signaling Gradient in the Vertebrate Neural Tube*, Cell, 2012.
> https://doi.org/10.1016/j.cell.2011.10.047

Secreted signals, known as morphogens, provide the positional information that organizes gene expression and cellular differentiation in many developing tissues. Several morphogens gradients can be observed on biological systems during development. However, the way the cells integrate the signal is often complicated and organism dependent. A simple model introduced by __Lewis Wolpert__, called the __French flag model__, postulates that the different cell fates adopted is the result of the crossing of thresholds of morphogens concentrations.

In the vertebrate neural tube, Sonic Hedgehog (Shh) acts as a morphogen to control the pattern of neuronal subtype specification. However, it was not clear how Shh gradient were interpreted by the cell to induce different cell fates. This article shows that a spatially and temporally changing gradient of Shh signaling is interpreted by the regulatory logic of a downstream transcriptional network. The design of the network, which links three transcription factors to Shh signaling, is responsible for differential spatial and temporal gene expression. In addition, the network renders cells insensitive to fluctuations in signaling and confers hysteresis - memory of the signal. The morphogen interpretation is an emergent property of the architecture of a transcriptional network that provides robustness and reliability to tissue patterning.

During this session, we will model the gene regulation network using differential equations and observe the behaviour induced by the architecture of this network.

---

## I - Temporal evolution of the GRN :

The relations between these proteins can be modeled as such :

$
\begin{align}
\frac{dP}{dt} &=\frac{\alpha}{1 + (\frac{N}{N_{critP}})^{h_1} +  (\frac{O}{O_{critP}})^{h_2}  } - k_1P \newline
\frac{dO}{dt} &=\frac{\beta G}{1 + G} \frac{1}{1 + (\frac{N}{N_{critO}})^{h_3}  } - k_2O \newline
\frac{dN}{dt} &=\frac{\gamma G}{1 + G} \frac{1}{1 + (\frac{O}{O_{critN}})^{h_4} + (\frac{P}{P_{critN}})^{h_5}  } - k_3N \newline
\end{align}
$

#### **Question 1 :**
> Discretize these differential equations with a forward euler-scheme

__*Correction*__ :

$
\begin{align}
P(t+dt) &= P(t)(1-k_1dt) + dt \frac{\alpha}{1 + (\frac{N(t)}{N_{critP}})^{h_1} +  (\frac{O(t)}{O_{critP}})^{h_2}  } \newline
O(t+dt) &= O(t)(1-k_2dt) + dt \frac{\beta G}{1 + G} \frac{1}{1 + (\frac{N(t)}{N_{critO}})^{h_3}  }   \newline
N(t+dt) &= N(t)(1-k_3dt) + dt \frac{\gamma G}{1 + G} \frac{1}{1 + (\frac{O(t)}{O_{critN}})^{h_4} + (\frac{P(t)}{P_{critN}})^{h_5}  } \newline
\end{align}
$

#### **Question 2 :**
> Integrate these equations for the given parameters for $t \in [0,50]$. Try with G $\in [1,2,3,4,5]$ What do you observe ?

__*Correction*__ :

The steady states values of O,P,N changes with G.

```python
alpha = 3
beta = 5
gamma = 5
h1 = 6
h2 = 2
h3 = 5
h4 = 1
h5 = 1
k1 = 1
k2 = 1
k3 = 1
OcritP = 1
NcritP = 1
NcritO = 1
PcritN = 1
n = 1
m = 1
```

```python
import numpy as np
import matplotlib.pyplot as plt
tfinal = 50
npoints = 1000
timepoints = np.linspace(0,tfinal,npoints)
G=3
fig,ax =plt.subplots(1,5,figsize=(25,4))

for i,G in enumerate([1,2,3,4,5]) :
    P=np.zeros(npoints)
    O=np.zeros(npoints)
    N=np.zeros(npoints)

    dt = timepoints[1]-timepoints[0]

    for t in range(npoints):
        if t==0 :
            P[t]=3
            continue

        P[t] = P[t-1]*(1-k1*dt) + dt* (alpha)/( 1 + N[t-1]**h1 + O[t-1]**h2  )
        O[t] = O[t-1]*(1-k2*dt) + dt* (beta*G)/(1+G)/(1+N[t-1]**h3)
        N[t] = N[t-1]*(1-k3*dt) + dt* (gamma*G)/(1+G)/(1 + O[t-1]**h4 + P[t-1]**h5)

    ax[i].plot(timepoints,P,color='tab:blue',label='P')
    ax[i].plot(timepoints,O,color='tab:red',label='O')
    ax[i].plot(timepoints,N,color='tab:green',label='N')
    ax[i].set_title("Temporal profiles")
    ax[i].set_xlabel("t")
    ax[i].set_xlim(0,50)
    ax[i].set_ylabel("P,O,N levels")
    ax[i].legend()
```

## II - Phase diagram of the GRN :

#### **Question 3 :**
> Plot the phase diagram of the steady states values of P,O and D for G $\in G_{vals}$. Then plot the phase diagram for $\alpha = 0, \beta=0 \text{ and } (\alpha,\beta)=0$

```python
from tqdm import tqdm
n_gvals = 100
Gvals = np.linspace(0,5,n_gvals)
Ps = np.zeros(n_gvals)
Os = np.zeros(n_gvals)
Ns = np.zeros(n_gvals)

alpha = 3
beta = 5
gamma = 5
h1 = 6
h2 = 2
h3 = 5
h4 = 1
h5 = 1
k1 = 1
k2 = 1
k3 = 1
OcritP = 1
NcritP = 1
NcritO = 1
PcritN = 1
n = 1
m = 1

P=np.zeros(n_gvals)
O=np.zeros(npoints)
N=np.zeros(npoints)

dt = timepoints[1]-timepoints[0]

fig,ax =plt.subplots(1,4,figsize=(20,3))

dictparams ={"Alpha 3, beta 5":(3,5),"Alpha 3, beta 0":(3,0),"Alpha 0, beta 5":(0,5),"Alpha 0, beta 0":(0,0)}
for n,key in enumerate(tqdm(dictparams.keys())) :
    alpha,beta = dictparams[key]
    for i,G in enumerate(Gvals) :
        tfinal = 50
        npoints = 10000
        timepoints = np.linspace(0,tfinal,npoints)

        P=np.zeros(npoints)
        O=np.zeros(npoints)
        N=np.zeros(npoints)

        for t in range(npoints):
            if t==0 :
                P[t]=3
                continue

            P[t] = P[t-1]*(1-k1*dt) + dt* (alpha)/( 1 + N[t-1]**h1 + O[t-1]**h2  )
            O[t] = O[t-1]*(1-k2*dt) + dt* (beta*G)/(1+G)/(1+N[t-1]**h3)
            N[t] = N[t-1]*(1-k3*dt) + dt* (gamma*G)/(1+G)/(1 + O[t-1]**h4 + P[t-1]**h5)

        Ps[i]=P[-1]
        Os[i]=O[-1]
        Ns[i]=N[-1]

    ax[n].plot(Gvals,Ps,color='tab:blue',label='P')
    ax[n].plot(Gvals,Os,color='tab:red',label='O')
    ax[n].plot(Gvals,Ns,color='tab:green',label='N')
    ax[n].set_title("Numerical Simulations "+key)
    ax[n].set_xlabel("G")
    ax[n].set_ylabel("P,O,N levels")
    ax[n].legend()
```

## III - Noise buffering by the GRN :
#### **Question 4 :**
> Compare the temporal evolution with $G = 5$ to the one with $G \mapsto \mathcal{N}(5,1)$, a normal law of mean 5 and of variance 1.  What are the effects of the noise on the proteins concentrations ?

```python
alpha = 3
beta = 5
gamma = 5
h1 = 6
h2 = 2
h3 = 5
h4 = 1
h5 = 1
k1 = 1
k2 = 1
k3 = 1
OcritP = 1
NcritP = 1
NcritO = 1
PcritN = 1
n = 1
m = 1

G_mean = 5
G_std = 1

import numpy as np
import matplotlib.pyplot as plt
tfinal = 20
npoints = 1000
timepoints = np.linspace(0,tfinal,npoints)
P=np.zeros(npoints)
O=np.zeros(npoints)
N=np.zeros(npoints)

dt = timepoints[1]-timepoints[0]

for t in range(npoints):
    if t==0 :
        P[t]=3
        continue

    G = G_mean

    P[t] = P[t-1]*(1-k1*dt) + dt* (alpha)/( 1 + N[t-1]**h1 + O[t-1]**h2  )
    O[t] = O[t-1]*(1-k2*dt) + dt* (beta*G)/(1+G)/(1+N[t-1]**h3)
    N[t] = N[t-1]*(1-k3*dt) + dt* (gamma*G)/(1+G)/(1 + O[t-1]**h4 + P[t-1]**h5)

plt.plot(timepoints,P,color='tab:blue',label='P')
plt.plot(timepoints,O,color='tab:red',label='O')
plt.plot(timepoints,N,color='tab:green',label='N')
plt.title("Temporal profiles")
plt.xlabel("t")
plt.xlim(0,20)
plt.ylabel("P,O,N levels")
plt.legend()
Gsnoise = np.zeros(npoints)

for t in range(npoints):
    if t==0 :
        P[t]=3
        continue

    G = np.random.normal(G_mean,G_std*2)
    Gsnoise[t]=G

    P[t] = P[t-1]*(1-k1*dt) + dt* (alpha)/( 1 + N[t-1]**h1 + O[t-1]**h2  )
    O[t] = O[t-1]*(1-k2*dt) + dt* (beta*G)/(1+G)/(1+N[t-1]**h3)
    N[t] = N[t-1]*(1-k3*dt) + dt* (gamma*G)/(1+G)/(1 + O[t-1]**h4 + P[t-1]**h5)

plt.plot(timepoints,P,color='tab:blue',label='P with noise',linestyle = '--')
plt.plot(timepoints,O,color='tab:red',label='O with noise',linestyle ='--')
plt.plot(timepoints,N,color='tab:green',label='N with noise',linestyle = '--')
plt.title("Temporal profiles of proteins concentrations")
plt.xlabel("t")
plt.xlim(0,20)
plt.ylabel("P,O,N levels")
plt.legend()

plt.figure()
plt.plot(timepoints,Gsnoise,label = 'Noisy G',color='k')
plt.axhline(y=5,label = "Constant G", color = 'r')
plt.title("Temporal profile of G")
plt.legend()
```

## IV - Hysteresis
#### **Question 5 :**
> From the steady state at G = 0, make G evolve slowly such that the steady state is reached at each variation of G. Then when G = 5, make G decrease back to zero. what do you observe ?

```python
n_gvals = 100
Gvals = np.hstack((np.linspace(0,5,n_gvals)[:-1],np.linspace(5,0,n_gvals)))
Ps = np.zeros(n_gvals*2-1)
Os = np.zeros(n_gvals*2-1)
Ns = np.zeros(n_gvals*2-1)

alpha = 3
beta = 5
gamma = 5
h1 = 6
h2 = 2
h3 = 5
h4 = 1
h5 = 1
k1 = 1
k2 = 1
k3 = 1
OcritP = 1
NcritP = 1
NcritO = 1
PcritN = 1
n = 1
m = 1

dt = timepoints[1]-timepoints[0]

for i,G in enumerate(Gvals) :
    tfinal = 50
    npoints = 100
    timepoints = np.linspace(0,tfinal,npoints)

    P=np.zeros(npoints)
    O=np.zeros(npoints)
    N=np.zeros(npoints)

    for t in range(npoints):
        if t==0 :
            if i == 0:
                P[t]=3
            else :
                P[t]=Ps[i-1]
            continue

        P[t] = P[t-1]*(1-k1*dt) + dt* (alpha)/( 1 + N[t-1]**h1 + O[t-1]**h2  )
        O[t] = O[t-1]*(1-k2*dt) + dt* (beta*G)/(1+G)/(1+N[t-1]**h3)
        N[t] = N[t-1]*(1-k3*dt) + dt* (gamma*G)/(1+G)/(1 + O[t-1]**h4 + P[t-1]**h5)

    Ps[i]=P[-1]
    Os[i]=O[-1]
    Ns[i]=N[-1]

fig,axs = plt.subplots(1,3,figsize=(15,3))
axs[0].plot(Gvals,Ps,color='tab:blue',label='P')
axs[0].set_xlabel("G")
axs[0].set_ylabel("P levels")

axs[1].plot(Gvals,Os,color='tab:red',label='O')
axs[1].set_xlabel("G")
axs[1].set_ylabel("O levels")

axs[2].plot(Gvals,Ns,color='tab:green',label='N')
axs[2].set_xlabel("G")
axs[2].set_ylabel("N levels")
```

# Conclusion

The proposed model provides evidence that Shh morphogen interpretation in the neural tube is a property of the downstream GNN. Cells transform the extracellular gradient of Shh into a dynamic profile of intracellular Gli activity that engages a transcriptional circuit, the regulatory logic of which is responsible for the generation of the characteristic temporal and spatial patterns of gene expression. This mechanism offers a powerful strategy to achieve the characteristic precision and robustness of morphogen-mediated pattern formation.

In the following of the article, the authors introduce another model, extending the preceding one by taking into account a forth protein, Gli. It allows them to reproduce the behaviours observed in their in-vivo experiments. On top of explaining the variable spatial patterns of gene expression, the networks confers both robustness and hysteresis of protein production. The insensitivity of the circuit to transient changes in the level of signaling provides a way to achieve reliable patterning in spite of the inherent noisiness of development. The study highlights the information-processing power of transcriptional networks and the simplicity and adaptability of this mechanism suggest that it is likely to be relevant for the control of patterning of tissues other than the neural tube.

The gene regulatory networks encountered in biological systems are often too complicated to construct them heuristically. Eric H. Davidson pionnered the use of bioinformatical tools to automatically infer gene regulation networks from genomic and transcriptomic data. It gives an engineer view of the genes relationships during developments, showing the complexity of the mechanisms involved.

> Gene regulation network for endomesoderm specification made with biotapestry : http://www.biotapestry.org. Don't try to understand this graph !

_Additional references :_
> Michael Akam, *Making stripes ineleganty*, Nature, 1989. https://doi.org/10.1038/341282a0

> Eric H. Davidson, *Emerging properties of animal gene regulatory networks*, Nature, 2010. https://doi.org/10.1038/nature09645

> Eric H. Davidson, Samuel Levine *Properties of developmental gene regulatory networks*, PNAS, 2008 https://doi.org/10.1073/pnas.0806007105

> Lewis Wolpert, *Positional information and the spatial pattern of cellular differentiation*, Journal of Theoretical Biology, 1969. https://doi.org/10.1016/S0022-5193(69)80016-0


# Overview and Examples

A brief description of what linearsolve is for followed by examples.

## What linearsolve Does

linearsolve defines a class - linearsolve.model - with several functions for approximating, solving, and simulating dynamic stochastic general equilibrium (DSGE) models. The equilibrium conditions for most DSGE models can be expressed as a vector function $f$:

\begin{align}
f(E_t X_{t+1}, X_t, \epsilon_{t+1}) = 0,
\end{align}

where 0 is an $n\times 1$ vector of zeros, $X_t$ is an $n\times 1$ vector of endogenous variables, and $\epsilon_{t+1}$ is an $m\times 1$ vector of exogenous structural shocks to the model. $E_tX_{t+1}$ denotes the expecation of the $t+1$ endogenous variables based on the information available to agents in the model as of time period $t$.

linearsolve.model has methods for computing linear and log-linear approximations of the model given above and methods for solving and simulating the linear model.

## Example 1: Quickly Simulate an RBC Model

Here I demonstrate how how relatively straightforward it is to appoximate, solve, and simulate a DSGE model using `linearsolve`. In the example that follows, I describe the procedure more carefully.

\begin{align}
C_t^{-\sigma} & = \beta E_t \left[C_{t+1}^{-\sigma}(\alpha A_{t+1} K_{t+1}^{\alpha-1} + 1 - \delta)\right]\\
C_t + K_{t+1} & = A_t K_t^{\alpha} + (1-\delta)K_t\\
\log A_{t+1} & = \rho_a \log A_{t} + \epsilon_{t+1}
\end{align}

In the block of code that immediately follows, I input the model, solve for the steady state, compute the log-linear approximation of the equilibirum conditions, and compute some impulse responses following a shock to technology $A_t$.

```python
# Import numpy, pandas, linearsolve, matplotlib.pyplot
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline

# Input model parameters
parameters = pd.Series(dtype=float)
parameters['alpha']  = .35
parameters['beta']  = 0.99
parameters['delta']   = 0.025
parameters['rhoa'] = .9
parameters['sigma'] = 1.5
parameters['A'] = 1

# Funtion that evaluates the equilibrium conditions
def equations(variables_forward,variables_current,parameters):

    # Parameters
    p = parameters

    # Variables
    fwd = variables_forward
    cur = variables_current

    # Household Euler equation
    euler_eqn = p.beta*fwd.c**-p.sigma*(p.alpha*cur.a*fwd.k**(p.alpha-1)+1-p.delta) - cur.c**-p.sigma

    # Goods market clearing
    market_clearing = cur.c + fwd.k - (1-p.delta)*cur.k - cur.a*cur.k**p.alpha

    # Exogenous technology
    technology_proc = p.rhoa*np.log(cur.a) - np.log(fwd.a)

    # Stack equilibrium conditions into a numpy array
    return np.array([
            euler_eqn,
            market_clearing,
            technology_proc
        ])

# Initialize the model
model = ls.model(equations = equations,
                 n_states=2,
                 n_exo_states = 1,
                 var_names=['a','k','c'],
                 parameters = parameters)

# Compute the steady state numerically
guess = [1,1,1]
model.compute_ss(guess)

# Find the log-linear approximation around the non-stochastic steady state and solve
model.approximate_and_solve()

# Compute impulse responses and plot
model.impulse(T=41,t0=5,shocks=None)

fig = plt.figure(figsize=(12,4))
ax1 =fig.add_subplot(1,2,1)
model.irs['e_a'][['a','k','c']].plot(lw='5',alpha=0.5,grid=True,ax=ax1).legend(loc='upper right',ncol=3)
ax2 =fig.add_subplot(1,2,2)
model.irs['e_a'][['e_a','a']].plot(lw='5',alpha=0.5,grid=True,ax=ax2).legend(loc='upper right',ncol=2)
```

## Example 2: An RBC Model with More Details

Consider the equilibrium conditions for a basic RBC model without labor:

\begin{align}
C_t^{-\sigma} & = \beta E_t \left[C_{t+1}^{-\sigma}(\alpha A_{t+1} K_{t+1}^{\alpha-1} + 1 - \delta)\right]\\
Y_t & = A_t K_t^{\alpha}\\
I_t & = K_{t+1} - (1-\delta)K_t\\
Y_t & = C_t + I_t\\
\log A_t & = \rho_a \log A_{t-1} + \epsilon_t
\end{align}

In the nonstochastic steady state, we have:

\begin{align}
K & = \left(\frac{\alpha A}{1/\beta+\delta-1}\right)^{\frac{1}{1-\alpha}}\\
Y & = AK^{\alpha}\\
I & = \delta K\\
C & = Y - I
\end{align}

Given values for the parameters $\beta$, $\sigma$, $\alpha$, $\delta$, and $A$, steady state values of capital,  output, investment, and consumption are easily computed.

### Initializing the model in `linearsolve`

To initialize the model, we need to first set the model's parameters. We do this by creating a Pandas Series variable called `parameters`:

```python
# Input model parameters
parameters = pd.Series(dtype=float)
parameters['alpha']  = .35
parameters['beta']  = 0.99
parameters['delta']   = 0.025
parameters['rhoa'] = .9
parameters['sigma'] = 1.5
parameters['A'] = 1
```

Next, we need to define a function that returns the equilibrium conditions of the model. The function will take as inputs two vectors: one vector of "current" variables and another of "forward-looking" or one-period-ahead variables. The function will return an array that represents the equilibirum conditions of the model. We'll enter each equation with all variables moved to one side of the equals sign. For example, here's how we'll enter the produciton fucntion:

`production_function = technology_current*capital_current**alpha - output_curent`

Here the variable `production_function` stores the production function equation set equal to zero. We can enter the equations in almost any way we want. For example, we could also have entered the production function this way:

`production_function = 1 - output_curent/technology_current/capital_current**alpha`

One more thing to consider: the natural log in the equation describing the evolution of total factor productivity will create problems for the solution routine later on. So rewrite the equation as:

\begin{align}
A_{t+1} & =  A_{t}^{\rho_a}e^{\epsilon_{t+1}}\\
\end{align}

So the complete system of equations that we enter into the program looks like:

\begin{align}
C_t^{-\sigma} & = \beta E_t \left[C_{t+1}^{-\sigma}(\alpha Y_{t+1} /K_{t+1}+ 1 - \delta)\right]\\
Y_t & = A_t K_t^{\alpha}\\
I_t & = K_{t+1} - (1-\delta)K_t\\
Y_t & = C_t + I_t\\
A_{t+1} & =  A_{t}^{\rho_a}e^{\epsilon_{t+1}}
\end{align}

Now let's define the function that returns the equilibrium conditions:

```python
# Define function to compute equilibrium conditions
def equations(variables_forward,variables_current,parameters):

    # Parameters
    p = parameters

    # Variables
    fwd = variables_forward
    cur = variables_current

    # Household Euler equation
    euler_eqn = p.beta*fwd.c**-p.sigma*(p.alpha*fwd.y/fwd.k+1-p.delta) - cur.c**-p.sigma

    # Production function
    production_fuction =  cur.a*cur.k**p.alpha - cur.y

    # Capital evolution
    capital_evolution = fwd.k - (1-p.delta)*cur.k - cur.i

    # Goods market clearing
    market_clearing = cur.c + cur.i - cur.y

    # Exogenous technology
    technology_proc = cur.a**p.rhoa- fwd.a

    # Stack equilibrium conditions into a numpy array
    return np.array([
            euler_eqn,
            production_fuction,
            capital_evolution,
            market_clearing,
            technology_proc
        ])
```

Notice that inside the function we have to define the variables of the model form the elements of the input vectors `variables_forward` and `variables_current`.

## Initializing the model

To initialize the model, we need to specify the total number of state variables in the model, the number of state variables with exogenous shocks, the names of the endogenous variables, and the parameters of the model.

It is *essential* that the variable names are ordered in the following way: First the names of the endogenous variables with the state variables with exogenous shocks, then the state variables without shocks, and finally the control variables. Ordering within the groups doesn't matter.

```python
# Initialize the model
rbc = ls.model(equations = equations,
               n_states=2,
               n_exo_states=1,
               var_names=['a','k','c','y','i'],
               parameters=parameters)
```

### Steady state

Next, we need to compute the nonstochastic steady state of the model. The `.compute_ss()` method can be used to compute the steady state numerically. The method's default is to use scipy's `fsolve()` function, but other scipy root-finding functions can be used: `root`, `broyden1`, and `broyden2`. The optional argument `options` lets the user pass keywords directly to the optimization function. Check out the documentation for Scipy's nonlinear solvers here: http://docs.scipy.org/doc/scipy/reference/optimize.html

```python
# Compute the steady state numerically
guess = [1,1,1,1,1]
rbc.compute_ss(guess)
print(rbc.ss)
```

    a     1.000000
    k    34.398226
    c     2.589794
    y     3.449750
    i     0.859956
    dtype: float64

Note that the steady state is returned as a Pandas Series. Alternatively, you could compute the steady state directly and then sent the `rbc.ss` attribute:

```python
# Steady state solution
p = parameters
K = (p.alpha*p.A/(1/p.beta+p.delta-1))**(1/(1-p.alpha))
C = p.A*K**p.alpha - p.delta*K
Y = p.A*K**p.alpha
I = Y - C

rbc.set_ss([p.A,K,C,Y,I])
print(rbc.ss)
```

    a     1.000000
    k    34.398226
    c     2.589794
    y     3.449750
    i     0.859956
    dtype: float64

### Log-linearization and solution

Now we use the `.log_linear()` method to find the log-linear appxoximation to the model's equilibrium conditions. That is, we'll transform the nonlinear model into a linear model in which all variables are expressed as log-deviations from the steady state. Specifically, we'll compute the matrices $A$ and $B$ that satisfy:

\begin{align}
A E_t\left[ x_{t+1} \right] & = B x_t + \left[ \begin{array}{c} \epsilon_{t+1} \\ 0 \end{array} \right],
\end{align}

where the vector $x_{t}$ denotes the log deviation of the endogenous variables from their steady state values.

```python
# Find the log-linear approximation around the non-stochastic steady state
rbc.log_linear_approximation()

print('The matrix A:\n\n',np.around(rbc.a,4),'\n\n')
print('The matrix B:\n\n',np.around(rbc.b,4))
```

    The matrix A:

     [[ 0.00000e+00 -8.30000e-03 -3.59900e-01  8.30000e-03  0.00000e+00]
     [ 0.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00]
     [ 0.00000e+00  3.43982e+01  0.00000e+00  0.00000e+00  0.00000e+00]
     [ 0.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00]
     [-1.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00  0.00000e+00]]

    The matrix B:

     [[-0.     -0.     -0.3599 -0.     -0.    ]
     [-3.4497 -1.2074 -0.      3.4497 -0.    ]
     [-0.     33.5383 -0.     -0.      0.86  ]
     [-0.     -0.     -2.5898  3.4497 -0.86  ]
     [-0.9    -0.     -0.     -0.     -0.    ]]

Finally, we need to obtain the *solution* to the log-linearized model. The solution is a pair of matrices $F$ and $P$ that specify:

1. The current values of the non-state variables $u_{t}$ as a linear function of the previous values of the state variables $s_t$.
1. The future values of the state variables $s_{t+1}$ as a linear function of the previous values of the state variables $s_t$ and the future realisation of the exogenous shock process $\epsilon_{t+1}$.

\begin{align}
u_t  &  = Fs_t\\
s_{t+1} & = Ps_t + \epsilon_{t+1}.
\end{align}

We use the `.klein()` method to find the solution.

```python
# Solve the model
rbc.solve_klein(rbc.a,rbc.b)

# Display the output
print('The matrix F:\n\n',np.around(rbc.f,4),'\n\n')
print('The matrix P:\n\n',np.around(rbc.p,4))
```

    The matrix F:

     [[ 0.2297  0.513 ]
     [ 1.      0.35  ]
     [ 3.3197 -0.1408]]

    The matrix P:

     [[0.9    0.    ]
     [0.083  0.9715]]

### Impulse responses

One the model is solved, use the `.impulse()` method to compute impulse responses to exogenous shocks to the state. The method creates the `.irs` attribute which is a dictionary with keys equal to the names of the exogenous shocks and the values are Pandas DataFrames with the computed impulse respones. You can supply your own values for the shocks, but the default is 0.01 for each exogenous shock.

```python
# Compute impulse responses and plot
rbc.impulse(T=41,t0=1,shocks=None,percent=True)

print('Impulse responses to a 0.01 unit shock to A:\n\n',rbc.irs['e_a'].head())
```

    Impulse responses to a 0.01 unit shock to A:

        e_a      a         k         c         y         i
    0  0.0  0.000  0.000000  0.000000  0.000000  0.000000
    1  1.0  1.000  0.000000  0.229718  1.000000  3.319739
    2  0.0  0.900  0.082993  0.249318  0.929048  2.976083
    3  0.0  0.810  0.155321  0.265744  0.864362  2.667127
    4  0.0  0.729  0.218116  0.279348  0.805341  2.389390

Plotting is easy.

```python
rbc.irs['e_a'][['a','k','c','y','i']].plot(lw='5',alpha=0.5,grid=True).legend(loc='upper right',ncol=2)
rbc.irs['e_a'][['e_a','a']].plot(lw='5',alpha=0.5,grid=True).legend(loc='upper right',ncol=2)
```

## Stochastic simulation

Creating a stochastic simulation of the model is straightforward with the `.stoch_sim()` method. In the following example, I create a 151 period (including t=0) simulation by first simulating the model for 251 periods and then dropping the first 100 values. The standard deviation of the shock to $A_t$ is set to 0.00763. The seed for the numpy random number generator is set to 0.

```python
rbc.stoch_sim(T=121,drop_first=100,cov_mat=np.array([0.00763**2]),seed=0,percent=True)
rbc.simulated[['k','c','y','i']].plot(lw='5',alpha=0.5,grid=True).legend(loc='upper right',ncol=4)
rbc.simulated[['a']].plot(lw='5',alpha=0.5,grid=True).legend(ncol=4)
rbc.simulated['e_a'].plot(lw='5',alpha=0.5,grid=True).legend(ncol=4)
```

## Example 3: A New-Keynesian Model

Consider the new-Keynesian business cycle model from Walsh (2017), chapter 8 expressed in log-linear terms:

\begin{align}
y_t & = E_ty_{t+1} - \sigma^{-1} (i_t - E_t\pi_{t+1}) + g_t\\
\pi_t & = \beta  E_t\pi_{t+1} + \kappa  y_t + u_t\\
i_t & = \phi_x  y_t + \phi_{\pi}  \pi_t + v_t\\
r_t & = i_t - E_t\pi_{t+1}\\
g_{t+1} & = \rho_g g_{t} + \epsilon_{t+1}^g\\
u_{t+1} & = \rho_u u_{t} + \epsilon_{t+1}^u\\
v_{t+1} & = \rho_v v_{t} + \epsilon_{t+1}^v
\end{align}

where $y_t$ is the output gap (log-deviation of output from the natural rate), $\pi_t$ is the quarterly rate of inflation between $t-1$ and $t$, $i_t$ is the nominal interest rate on funds moving between period $t$ and $t+1$, $r_t$ is the real interest rate, $g_t$ is the exogenous component of demand, $u_t$ is an exogenous component of inflation, and $v_t$ is the exogenous component of monetary policy.

Since the model is already log-linear, there is no need to approximate the equilibrium conditions. We'll still use the `.log_linear` method to find the matrices $A$ and $B$, but we'll have to set the `islinear` option to `True` to avoid generating an error.

### Inintialize model and solve

```python
# Input model parameters
beta = 0.99
sigma= 1
eta  = 1
omega= 0.8
kappa= (sigma+eta)*(1-omega)*(1-beta*omega)/omega

rhor = 0.9
phipi= 1.5
phiy = 0

rhog = 0.5
rhou = 0.5
rhov = 0.9

Sigma = 0.001*np.eye(3)

# Store parameters
parameters = pd.Series({
    'beta':beta,
    'sigma':sigma,
    'eta':eta,
    'omega':omega,
    'kappa':kappa,
    'rhor':rhor,
    'phipi':phipi,
    'phiy':phiy,
    'rhog':rhog,
    'rhou':rhou,
    'rhov':rhov
})

# Define function that computes equilibrium conditions
def equations(variables_forward,variables_current,parameters):

    # Parameters
    p = parameters

    # Variables
    fwd = variables_forward
    cur = variables_current

    # Exogenous demand
    g_proc =  p.rhog*cur.g - fwd.g

    # Exogenous inflation
    u_proc =  p.rhou*cur.u - fwd.u

    # Exogenous monetary policy
    v_proc =  p.rhov*cur.v - fwd.v

    # Euler equation
    euler_eqn = fwd.y -1/p.sigma*(cur.i-fwd.pi) + fwd.g - cur.y

    # NK Phillips curve evolution
    phillips_curve = p.beta*fwd.pi + p.kappa*cur.y + fwd.u - cur.pi

    # interest rate rule
    interest_rule = p.phiy*cur.y+p.phipi*cur.pi + fwd.v - cur.i

    # Fisher equation
    fisher_eqn = cur.i - fwd.pi - cur.r

    # Stack equilibrium conditions into a numpy array
    return np.array([
            g_proc,
            u_proc,
            v_proc,
            euler_eqn,
            phillips_curve,
            interest_rule,
            fisher_eqn
        ])

# Initialize the nk model
nk = ls.model(equations=equations,
              n_states=3,
              n_exo_states = 3,
              var_names=['g','u','v','i','r','y','pi'],
              parameters=parameters)

# Set the steady state of the nk model
nk.set_ss([0,0,0,0,0,0,0])

# Find the log-linear approximation around the non-stochastic steady state
nk.linear_approximation()

# Solve the nk model
nk.solve_klein(nk.a,nk.b)
```

### Compute impulse responses and plot

Compute impulse responses of the endogenous variables to a one percent shock to each exogenous variable.

```python
# Compute impulse responses
nk.impulse(T=11,t0=1,shocks=None)

# Create the figure and axes
fig = plt.figure(figsize=(12,12))
ax1 = fig.add_subplot(3,1,1)
ax2 = fig.add_subplot(3,1,2)
ax3 = fig.add_subplot(3,1,3)

# Plot commands
nk.irs['e_g'][['g','y','i','pi','r']].plot(lw='5',alpha=0.5,grid=True,title='Demand shock',ax=ax1).legend(loc='upper right',ncol=5)
nk.irs['e_u'][['u','y','i','pi','r']].plot(lw='5',alpha=0.5,grid=True,title='Inflation shock',ax=ax2).legend(loc='upper right',ncol=5)
nk.irs['e_v'][['v','y','i','pi','r']].plot(lw='5',alpha=0.5,grid=True,title='Interest rate shock',ax=ax3).legend(loc='upper right',ncol=5)
```

### Construct a stochastic simulation and plot

Contruct a 151 period stochastic simulation by first siumlating the model for 251 periods and then dropping the first 100 values. The seed for the numpy random number generator is set to 0.

```python
# Compute stochastic simulation
nk.stoch_sim(T=151,drop_first=100,cov_mat=Sigma,seed=0)

# Create the figure and axes
fig = plt.figure(figsize=(12,8))
ax1 = fig.add_subplot(2,1,1)
ax2 = fig.add_subplot(2,1,2)

# Plot commands
nk.simulated[['y','i','pi','r']].plot(lw='5',alpha=0.5,grid=True,title='Output, inflation, and interest rates',ax=ax1).legend(ncol=4)
nk.simulated[['g','u','v']].plot(lw='5',alpha=0.5,grid=True,title='Exogenous demand, inflation, and policy',ax=ax2).legend(ncol=4,loc='lower right')
```

```python
# Plot simulated exogenous shocks
nk.simulated[['e_g','g']].plot(lw='5',alpha=0.5,grid=True).legend(ncol=2)
nk.simulated[['e_u','u']].plot(lw='5',alpha=0.5,grid=True).legend(ncol=2)
nk.simulated[['e_v','v']].plot(lw='5',alpha=0.5,grid=True).legend(ncol=2)
```


# The Efficient Frontier of Optimal Portfolio Transactions

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) showed that for each value of risk aversion there is a unique optimal execution strategy. The optimal strategy is obtained by minimizing the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

where $E(x)$ is the **Expected Shortfall**, $V(x)$ is the **Variance of the Shortfall**, and $\lambda$ corresponds to the trader’s risk aversion. The expected shortfall and variance of the optimal trading strategy are given by:

In this notebook, we will learn how to visualize and interpret these equations.

# The Expected Shortfall

As we saw in the previous notebook, even if we use the same trading list, we are not guaranteed to always get the same implementation shortfall due to the random fluctuations in the stock price. This is why we had to reframe the problem of finding the optimal strategy in terms of the average implementation shortfall and the variance of the implementation shortfall. We call the average implementation shortfall, the expected shortfall $E(x)$, and the variance of the implementation shortfall $V(x)$. So, whenever we talk about the expected shortfall we are really talking about the average implementation shortfall. Therefore, we can think of the expected shortfall as follows. Given a single trading list, the expected shortfall will be the value of the average implementation shortfall if we were to implement this trade list in the stock market many times.

To see this, in the code below we implement the same trade list on 50,000 trading simulations. We call each trading simulation an episode. Each episode will consist of different random fluctuations in stock price. For each episode we will compute the corresponding implemented shortfall. After all the 50,000 trading simulations have been carried out we calculate the average implementation shortfall and the variance of the implemented shortfalls. We can then compare these values with the values given by the equations for $E(x)$ and $V(x)$ from the Almgren and Chriss model.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the liquidation time
l_time = 60

# Set the number of trades
n_trades = 60

# Set trader's risk aversion
t_risk = 1e-6

# Set the number of episodes to run the simulation
episodes = 200

utils.get_av_std(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, trs = episodes)

# Get the AC Optimal strategy for the given parameters
ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

# Extreme Trading Strategies

Because some investors may be willing to take more risk than others, when looking for the optimal strategy we have to consider a wide range of risk values, ranging from those traders that want to take zero risk to those who want to take as much risk as possible. Let's take a look at these two extreme cases. We will define the **Minimum Variance** strategy as that one followed by a trader that wants to take zero risk and the **Minimum Impact** strategy at that one followed by a trader that wants to take as much risk as possible. Let's take a look at the values of $E(x)$ and $V(x)$ for these extreme trading strategies. The `utils.get_min_param()` uses the above equations for $E(x)$ and $V(x)$, along with the parameters from the trading environment to calculate the expected shortfall and standard deviation (the square root of the variance) for these strategies. We'll start by looking at the Minimum Impact strategy.

```python
import utils

# Get the minimum impact and minimum variance strategies
minimum_impact, minimum_variance = utils.get_min_param()
```

### Minimum Impact Strategy

This trading strategy will be taken by trader that has no regard for risk. In the Almgren and Chriss model this will correspond to having the trader's risk aversion set to $\lambda = 0$. In this case the trader will sell the shares at a constant rate over a long period of time. By doing so, he will minimize market impact, but will be at risk of losing a lot of money due to the large variance. Hence, this strategy will yield the lowest possible expected shortfall and the highest possible variance, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of \$197,000 dollars but has a very big standard deviation of over 3 million dollars.

```python
minimum_impact
```

AC Optimal Strategy for Minimum Impact

  Number of Days to Sell All the Shares:	     250	       Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                1,284,394.9	   Expected Shortfall:	                $197,000.00

  Utility:	                               $197,000.00	   Standard Deviation of Shortfall:	  $3,453,707.55

### Minimum Variance Strategy

This trading strategy will be taken by trader that wants to take zero risk, regardless of transaction costs. In the Almgren and Chriss model this will correspond to having a variance of $V(x) = 0$. In this case, the trader would prefer to sell the all his shares immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. This strategy will yield the smallest possible variance, $V(x) = 0$, and the highest possible expected shortfall, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of over 2.5 million dollars but has a standard deviation equal of zero.

```python
minimum_variance
```

AC Optimal Strategy for Minimum Variance

  Number of Days to Sell All the Shares:	       1	         Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                     0.2	        Expected Shortfall:	               $2,562,500.00

  Utility:	                               $2,562,500.00	   Standard Deviation of Shortfall:	      $0.00

# The Efficient Frontier

The goal of Almgren and Chriss was to find the optimal strategies that lie between these two extremes. In their paper, they showed how to compute the trade list that minimizes the expected shortfall for a wide range of risk values. In their model, Almgren and Chriss used the parameter $\lambda$ to measure a trader's risk-aversion. The value of $\lambda$ tells us how much a trader is willing to penalize the variance of the shortfall, $V(X)$, relative to expected shortfall, $E(X)$. They showed that for each value of $\lambda$ there is a uniquely determined optimal execution strategy. We define the **Efficient Frontier** to be the set of all these optimal trading strategies. That is, the efficient frontier is the set that contains the optimal trading strategy for each value of $\lambda$.

The efficient frontier is often visualized by plotting $(x,y)$ pairs for a wide range of $\lambda$ values, where the $x$-coordinate is given by the equation of the expected shortfall, $E(X)$, and the $y$-coordinate is given by the equation of the variance of the shortfall, $V(X)$. Therefore, for a given a set of parameters, the curve defined by the efficient frontier represents the set of optimal trading strategies that give the lowest expected shortfall for a defined level of risk.

In the code below, we plot the efficient frontier for $\lambda$ values in the range $(10^{-7}, 10^{-4})$, using the default parameters in our trading environment. Each point of the frontier represents a distinct strategy for optimally liquidating the same number of stocks. A risk-averse trader, who wishes to sell quickly to reduce exposure to stock price volatility, despite the trading costs incurred in doing so, will likely choose a value of $\lambda = 10^{-4}$. On the other hand, a trader
who likes risk, who wishes to postpones selling, will likely choose a value of $\lambda = 10^{-7}$. In the code, you can choose a particular value of $\lambda$ to see the expected shortfall and level of variance corresponding to that particular value of trader's risk aversion.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Plot the efficient frontier for the default values. The plot points out the expected shortfall and variance of the
# optimal strategy for the given the trader's risk aversion. Valid range for the trader's risk aversion (1e-7, 1e-4).
utils.plot_efficient_frontier(tr_risk = 1e-6)
```

```python

```


# Solution for compulsary exercise 2 - TKT4196 2021

```python
import numpy as np
import scipy as sp
from scipy.stats import multivariate_normal as mvn
import matplotlib.pyplot as plt
import scipy.stats
import time
import sympy as sym

fontsizes=18
plt.rcParams.update({'font.size': fontsizes})
plt.rcParams.update({"font.family": "serif"})
plt.rcParams.update({"mathtext.fontset" : "cm"})
plt.rcParams.update({'font.serif': 'Times New Roman'})
plt.close('all')
```

Save all basic values given, awaiting transformation to u-space

```python
# =============================================================================
# INPUT

mu_dt = 1.12
V_dt = 0.32
std_dt = mu_dt * V_dt

mu_df = 1.1
V_df= 0.2
std_df = mu_df * V_df

mu_N1 = 40
V_N1 = 0.3
std_N1 = mu_N1 * V_N1

mu_N2 = 60
V_N2 = 0.2
std_N2 = mu_N2 * V_N2

mu_G1 = 400000                                    #N
V_G1 = 0.1
std_G1 = mu_G1 * V_G1                             #N

mu_Q = 500000                                     #N
V_Q = 0.35
std_Q = mu_Q * V_Q                                #N

l1 = 5000                                         #mm

rho_rc = 25*1e-6                                  #N/mm^3

A = lambda d: np.pi/4 * d**2                      # Defining function for area of the pile [mm2]
C = lambda d: np.pi * d                           # Defining function for circumference of the pile [mm]
G2 = lambda h,d: A(d) * h * rho_rc                # Defining function for self weight of the pile [N]

k1 = 0.01                                         # constant in front of N_1 in c_1 [N/mm2]
k2 = 0.005                                        # constant in front of N_2 in c_2 [N/mm2]
```

Function calculating parameters from moments as defined in Exercise 3:

```python
def addparams(X):
    moments = X['moments']
    if X['type'] == 'Normal':
        theta = X['moments']
    if X['type'] == 'Lognormal':
        sigma_ln = np.sqrt(np.log(moments[1]**2/moments[0]**2+1))
        mu_ln = np.log(moments[0])-0.5*sigma_ln**2
        theta = [mu_ln,sigma_ln]
    if X['type'] == 'Gumbel':
        a = np.pi/np.sqrt(6)/moments[1]
        b = moments[0] - 0.5772/a
        theta = [a,b]
    X['theta']=theta
```

### Transformed input
We use dictionaries here to allow type to be defined and utilised

```python
#TRANSFORMED INPUT
G1 = {}
G1['type'] = "Gumbel"
G1['moments'] = [mu_G1,std_G1]
addparams(G1)

Q = {}
Q['type'] = "Gumbel"
Q['moments'] = [mu_Q,std_Q]
addparams(Q)

N1 = {}
N1['type'] = "Lognormal"
N1['moments'] = [mu_N1,std_N1]
addparams(N1)

N2 = {}
N2['type'] = "Lognormal"
N2['moments'] = [mu_N2,std_N2]
addparams(N2)

dt = {}
dt['type'] = "Normal"
dt['moments'] = [mu_dt,std_dt]
addparams(dt)

df = {}
df['type'] = "Normal"
df['moments'] = [mu_df,std_df]
addparams(df)
```

Limit state function in the X-space

$$ g_X(R_t,R_s,G_1,G_2,Q) = (R_t + R_s) - (G_1 + G_2 + Q) $$

substituting in the random variables:

$$ g_X(\delta_t, \delta_f, N_1, N_2, G_1, Q_1) = \delta_t \, A \cdot 10 \, k_2 \, N_2 + \delta_f \, C \cdot 0.4 \left(k_1 \, l_1 \, N_1 + k_2 \, (h-l_1) \, N_2\right) - (G_1 + A \, \rho_{rc} \, h + Q)$$

where:

$A$: Cross sectional area

$C$: Cross sectional circumference

```python
gX = lambda x,h,d: x[0]*A(d)*10*k2*x[3]+x[1]*C(d)*0.4*(k1*x[2]*l1+(h-l1)*k2*x[3])-(x[4]+A(d)*rho_rc*h+x[5])
```

Recall that we turn our limit state function from a function of $x$, to a function of $u$

$$g=g(x)=g(x(u)) \Rightarrow g=g(u)$$

To do this mapping, we need the to know the function $x(u)$

We recall that

$$x = F_{X_i}^{-1}(\Phi(u))$$

Note that that $F_{x_i}$ depends on the distribution type of variable $X_i$, whilst $\Phi$ is always the same.

The compendium covers the distribution types which we represent variables with in this course, we program their $x(u)$:

```python
def x2u(X):
    #X is here a dictionary, which describes the type and stores its parameters
    if X['type'] == 'Normal':
        # If X is normal distributed, use Eq. (3.8) (isolated x)
        x = lambda u: u*X['theta'][1] + X['theta'][0]
        # X['theta'][1] = sigma
        # X['theta'][0] = mu
    if X['type'] == 'Lognormal':
        # If X is lognormal distributed, use Eq. (3.26)
        x = lambda u: np.exp(X['theta'][1]*u+X['theta'][0])
        # X['theta'][1] = sigma_L
        # X['theta'][0] = mu_L
    if X['type'] == 'Gumbel':
        # If X is Gumbel distributed, use Eq. (3.27)
        x = lambda u: X['theta'][1]-1/X['theta'][0]*np.log(-np.log(sp.stats.norm.cdf(u)))
        # X['theta'][1] = b
        # X['theta'][0] = a
    return x
```

We want to differentiate the limit state with respects to $u$ in our iteration. We can do this by utilizing the chain rule:

$$\frac{\partial g}{\partial u} = \frac{\partial g}{ \partial x}\frac{\partial x}{\partial u}$$

$$ \frac{\partial x}{\partial u} = \frac{\partial}{\partial u} \left( F_{x}^{-1}(\Phi(u) \right) $$

```python
def dxdufun(X):
    if X['type'] == "Normal":
        dxdu = lambda u: X['theta'][1]
    if X['type'] == "Lognormal":
        dxdu = lambda u: X['theta'][1] * np.exp(X['theta'][0]+X['theta'][1] * u)
    if X['type'] == "Gumbel":
        dxdu = lambda u: - sp.stats.norm.pdf(u)/(X['theta'][0] * sp.stats.norm.cdf(u)*np.log(sp.stats.norm.cdf(u)))
    return dxdu
```

We create a list of the transformed functions for each variable to make the code easier to look at.

```python
U = [x2u(dt), x2u(df), x2u(N1), x2u(N2), x2u(G1), x2u(Q)]
```

We can now write our LSF as a function of u:

```python
gU = lambda u,h,d: gX([U[0](u[0]),
                   U[1](u[1]),
                   U[2](u[2]),
                   U[3](u[3]),
                   U[4](u[4]),
                   U[5](u[5])],h,d)
```

Next, we define the function that gives us the next, improved $\alpha$-vector

```python
def alpha_next(u,h,d):
    dgdu = np.zeros(6)
    dgdu[0] = ( A(d)*10*k2*U[3](u[3])*dxdufun(dt)(u[0]) )
    dgdu[1] = ( C(d)*0.4*(k1*U[2](u[2])*l1+(h-l1)*k2*U[3](u[3]))*dxdufun(df)(u[1]) )
    dgdu[2] = ( C(d)*0.4*k1*l1*U[1](u[1])*dxdufun(N1)(u[2]) )
    dgdu[3] = ( (A(d)*10*k2*U[0](u[0])+k2*C(d)*0.4*(h-l1)*U[1](u[1]))*dxdufun(N2)(u[3]) )
    dgdu[4] = ( -dxdufun(G1)(u[4]) )
    dgdu[5] = ( -dxdufun(Q)(u[5]) )

    k = np.sqrt(sum(dgdu**2))                                  #Normalisation factor, makes ||alpha|| = 1

    alpha = -dgdu/k

    return alpha
```

Recall that this does not correspond exactly to eq. (3.10) as this equation requires us to insert the previous values of $\beta$ and $\mathrm{\alpha}$, in other words, a new  $\mathrm{\alpha}$ is not found until we do this. This means that we need a starting value, $\mathrm{\alpha}_0$

```python
alpha0 = np.array([-1/np.sqrt(6),-1/np.sqrt(6),-1/np.sqrt(6),1/np.sqrt(6),1/np.sqrt(6),1/np.sqrt(6)])
```

Now we need a way to solve $g_u(\beta \cdot \mathbf{\alpha}) = 0$ with respect to $\beta$. This can be done in Python by utilizing `fsolve`

```python
from scipy.optimize import fsolve
```

Since `fsolve` is a numerical solver it needs a starting guess for $\beta$. The logical guess would be the previous $\beta$ we calculated. This also means we need a starting value for $\beta$, $\beta_0$:

```python
beta0=4
```

```python
def beta_next(alpha, beta_prev,h,d):
    equation = lambda beta: gU(beta*alpha,h,d)
    beta_new = fsolve(equation, x0=beta_prev) #Finds beta that gives equtaion = 0
    return beta_new
```

```python
def FORM_tol(h,d,alpha_start, beta_start, tol=1e-5, itreturn=False):
    alphas = [alpha_start]
    beta1 = beta_next(alpha_start, beta0, h, d)
    betas = [beta_start, beta1]                          #Need at least two values of beta for the next line to work, first beta is therefore calculated manually
    while abs((betas[-1] - betas[-2])/betas[-1]) >= tol:
        alpha_new = alpha_next(betas[-1]*alphas[-1],h,d) #calculates new, improved alpha, using the last calculated values (by using [-1])
        alphas.append(alpha_new)                         #adds the new, improved alpha to the list of alphas
        beta_new = beta_next(alphas[-1],betas[-1],h,d)   #calculates new, improved beta
        betas.append(beta_new)                           #adds the new, improved beta to the list of betas
    if itreturn:
        return alphas[-1], betas[-1], len(alphas)-1      #returns the final values, and number of iterations
    else:
        return alphas[-1], betas[-1]                     #returns only the final values
```

### Monte Carlo simulation
We use Monte Carlo as a way to test the quality of our results

```python
def MCS(h,d,n_sim):

    n_sim = int(n_sim)

    # Defining arrays with realisations for each randomly distributed variable

    df_mcs = np.random.normal(loc=mu_df, scale=std_df, size=n_sim)
    dt_mcs = np.random.normal(loc=mu_dt, scale=std_dt, size=n_sim)
    N1_mcs = np.random.lognormal(N1['theta'][0], N1['theta'][1], size=n_sim)
    N2_mcs = np.random.lognormal(N2['theta'][0], N2['theta'][1], size=n_sim)
    G1_mcs = np.random.gumbel(G1['theta'][1], 1/G1['theta'][0], size=n_sim)
    Q_mcs =  np.random.gumbel(Q['theta'][1], 1/Q['theta'][0], size=n_sim)

    l2 = h - l1

    c1 = k1 * N1_mcs
    c2 = k2 * N2_mcs

    f1 = 0.4 * c1
    f2 = 0.4 * c2

    Rs1 = df_mcs * C(d) * l1 * f1
    Rs2 = df_mcs * C(d) * l2 * f2
    Rs = Rs1 + Rs2                                      # Collect all random variables that contribute to shaft resistance
    Rt = dt_mcs * A(d) * 10 * c2                        # Collect all random variables that contribute to tip resistance

    g = Rt + Rs - (G1_mcs + G2(h,d) + Q_mcs)            # LSF for MCS
    fails = 0
    for el in g:
        if el <= 0:
            fails+=1

    pf = fails/n_sim
    beta = -sp.stats.norm.ppf(pf)

    return beta, pf
```

## Task 1

We test our FORM algortithm by comparing the results with Monte Carlo simulation.

We choose two combinations,

- $h = 6000 \text{ mm}, \phi = 800 \text{ mm}$
- $h = 8000 \text{ mm}, \phi = 700 \text{ mm}$

```python
alpha1, beta1 = FORM_tol(6000,800,alpha_start=alpha0, beta_start=beta0)
beta1_mcs = MCS(6000, 800, 1e6)[0]

alpha2, beta2 = FORM_tol(8000,700,alpha_start=alpha0, beta_start=beta0)
beta2_mcs = MCS(8000, 700, 1e6)[0]

print(u'h = 6000 mm, \u03C6 = 800 mm:')
print(u'\u03B2-FORM = %.5f' % beta1)
print(u'\u03B2-MCS = %.5f' % beta1_mcs)
print()
print(u'h = 8000 mm, \u03C6 = 700 mm:')
print(u'\u03B2-FORM = %.5f' % beta2)
print(u'\u03B2-MCS = %.5f' % beta2_mcs)
```

    h = 6000 mm, φ = 800 mm:
    β-FORM = 3.91698
    β-MCS = 3.87188

    h = 8000 mm, φ = 700 mm:
    β-FORM = 4.03157
    β-MCS = 3.96304

## Task 2

```python
f1 = lambda h,d: FORM_tol(h, d, alpha_start = alpha0, beta_start = beta0)[1] #Only returns the beta-value
```

```python
beta = f1(10000,600)
pf = sp.stats.norm.cdf(-beta)

print('Reliability index = %.5f' % beta)
print('Probability of failure = %.3e' % pf)
```

    Reliability index = 3.93919
    Probability of failure = 4.088e-05

## Task 3

```python
n=20                                                # Resolution of the plot

h_dum = np.linspace(5000, 20000,n)                  # Dummy arrays used for plotting
d_dum = np.linspace(200, 1000,n)

hh, dd = np.meshgrid(h_dum,d_dum)

beta_dum = np.zeros_like(hh)

for i in range(n):
    for j in range(n):
        beta_dum[i,j]=f1(hh[i,j],dd[i,j])
```

```python
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax1 = fig.add_subplot(111, projection='3d')
ax1.plot_surface(hh/1000,dd/1000, beta_dum,cmap=plt.cm.coolwarm)

# Enhance the plot
ax1.set_xlim(h_dum[0]/1000,h_dum[-1]/1000)
ax1.set_ylim(d_dum[0]/1000,d_dum[-1]/1000)

ax1.set_xlabel(r'$h$ [m]',fontsize=15)
ax1.set_ylabel(r'$\phi$ [m]',fontsize=15)
ax1.set_title(r'$\beta(h,\phi)$',fontsize=20)

plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
ax1.zaxis.set_tick_params(labelsize=12)

plt.tight_layout()
plt.show()
```

## Task 4

```python
# Optimization problem
# Cost Inputs: =================================================

C_0 = 50e3                                               # Fixed construction cost [NOK]
c_pile = 1e-5                                            # Cost of pile pr mm3 [NOK/mm3]
N_F = 5                                                  # Expected fatalities given failure
i_e = 0.03                                               # Interest rate
i_s = 0.025                                              # Societal interest rate
T_sl = 50                                                # Service live [years]
SWTP = 32.1e6                                            # NOK
H = 20*C_0                                               # Failure cost
D = 2*C_0                                                # Demolition cost
w = 1/T_sl                                               # Obsolence rate
```

```python
n=20                                                     # Again, resolution of plots

h_dum = np.linspace(5e3, 20e3,n)
d_dum = np.linspace(0.2e3, 1e3,n)

hh, dd = np.meshgrid(h_dum,d_dum)

PF_1 = lambda h, d: scipy.stats.norm.cdf(-f1(h,d))       # Annual probability of failure

pf_1 = np.zeros_like(hh)
C_1 = np.zeros_like(hh)

V = lambda h,d: A(d)*h                                   # Volume of pile given height and diameter

for i in range(n):
    for j in range(n):
        pf_1[i,j] = PF_1(hh[i,j], dd[i,j])
        C_1[i,j] = V(hh[i,j], dd[i,j]) * c_pile
```

```python
C_c = C_0 + C_1                                          # Construction cost
EC_f =  (C_c + H)* pf_1/i_e                              # Infite renewal expected failure cost
EC_obs = (C_c + D)*w/i_e                                 # Infinte renewal expected obsolence cost
EC_tot = C_c + EC_f + EC_obs                             # Excpected total cost
```

```python
from matplotlib import ticker

def plotZ(Z,Tittel,pos):
    ax1 = fig.add_subplot(pos,projection='3d')
    ax1.plot_surface(hh/1000,dd/1000,Z,cmap=plt.cm.coolwarm)
    ax1.set_xlim(h_dum[0]/1000,h_dum[-1]/1000)
    ax1.set_ylim(d_dum[0]/1000,d_dum[-1]/1000)
    ax1.set_xlabel('h [m]',fontsize=15)
    ax1.set_ylabel('d [m]',fontsize=15)
    ax1.set_zlabel("NOK",rotation=90,labelpad=5,fontsize=15)
    ax1.set_title(Tittel,fontsize=20)
    formatter = ticker.ScalarFormatter(useMathText=True)
    formatter.set_scientific(True)
    formatter.set_powerlimits((-1,1))
    ax1.zaxis.set_major_formatter(formatter)
    ax1.xaxis.set_tick_params(labelsize=12)
    ax1.yaxis.set_tick_params(labelsize=12)
    ax1.zaxis.set_tick_params(labelsize=12)
```

```python
fig = plt.figure(figsize=(10,10))
plotZ(C_c,'Construction costs',221)
plotZ(EC_obs,"Obsolescence cost",222)
plotZ(EC_f,"Expected cost of failure ",223)
plotZ(EC_tot,"Total expected cost",224)
plt.tight_layout()
fig.subplots_adjust(wspace=.1,hspace=.2)
plt.show()
```

## Task 5

```python
n=50

d_opt = 700

h_dum = np.linspace(5e3, 10e3,n+1)

C_1 = lambda h: c_pile * V(h, d_opt)

C_c = lambda h: C_0 + C_1(h)

EC_obs = lambda h: (C_c(h) + D)*w/i_e

EC_f = lambda h: (C_c(h)+H) * PF_1(h, d_opt)*1/i_e

f2 = lambda h: float(C_c(h) + EC_obs(h) + EC_f(h) )

fig = plt.figure(figsize=(10,5))
ax = fig.add_subplot(111)
ax.plot(h_dum/1000,np.vectorize(f2)(h_dum),'k')
ax.set_xlabel(r'$h$ [m]')
ax.set_xlim(h_dum[0]/1000,h_dum[-1]/1000)
ax.set_ylabel(r'E[$C_{tot}]$ [NOK]')
ax.grid(which='major')
ax.minorticks_on()
ax.grid(which='minor', alpha=0.2)
fig.tight_layout()
plt.show()
```

```python
res=scipy.optimize.minimize(f2,6e3,method='nelder-mead',options={'xatol': 1e-2})
h_opt=float(res.x)
ECtot_opt = float(res.fun)
print(res)
print('\nOptimal height: %.0f mm \nOptimal expected total costs: %.0f NOK' % (h_opt, ECtot_opt))
```

     final_simplex: (array([[5801.11083984],
           [5801.10168457]]), array([193676.69550286, 193676.69550306]))
               fun: 193676.6955028569
           message: 'Optimization terminated successfully.'
              nfev: 34
               nit: 17
            status: 0
           success: True
                 x: array([5801.11083984])

    Optimal height: 5801 mm
    Optimal expected total costs: 193677 NOK

```python
def hjelpelinjer(ax, x,y, c='b',label=None):
    ax.plot([x,x],[0,y],(c + '--'))
    ax.plot([0,x],[y,y],(c + '--'))
    ax.plot(x,y,(c + 'o'),label=label)

fig = plt.figure(figsize=(5,5))
ax = fig.add_subplot(111)
ax.plot(h_dum/1000,np.vectorize(f2)(h_dum),'k')
hjelpelinjer(ax, h_opt/1000,f2(h_opt))
ax.set_xlabel(r'$h$ [m]')
ax.set_xlim(5,7)
ax.set_ylabel(r'E[$C_{tot}]$ [NOK]',fontsize=20)
ax.set_ylim(ECtot_opt*0.95, ECtot_opt*1.05)
ax.grid(which='major')
ax.minorticks_on()
ax.grid(which='minor', alpha=0.2)
fig.tight_layout()
plt.show()
```

## Task 6

```python
h_dum = np.linspace(5000,20000,50)

C1 = c_pile*np.pi/4*d_opt**2
K1 = C1*(i_s + w)/(SWTP*N_F)

#Numerical differentation:
dh = 1
dPdh = lambda h: (PF_1(h+dh, d_opt) - PF_1(h-dh, d_opt)) / (2*dh)

#Finding the height, h_min, which gives -dPdh = K1
equation = lambda h: -dPdh(h) - K1

h_min = float(fsolve(equation, 12000))

fig = plt.figure(figsize=(10,10))

ax1 = fig.add_subplot(211)
ax1.plot(h_dum,-np.vectorize(dPdh)(h_dum),'k',label=r'$-\frac{\mathrm{d}P_{f,1}}{\mathrm{d}h}$')
ax1.plot([h_dum[0],h_dum[-1]],[K1,K1],'r--',label='$K_1$')
ax1.plot(h_min,K1,'ro')
ax1.set_xlabel('$h$ [mm]')
ax1.set_ylabel('[mm$^{-1}$]')
ax1.set_xlim(h_dum[0],h_dum[-1])
ax1.set_ylim(K1*0.1, K1*20)
ax1.minorticks_on()
ax1.legend()

ax2 = fig.add_subplot(212)
ax2.plot(h_dum, np.vectorize(f2)(h_dum),'k')
ax2.axvspan(h_min,h_dum[-1],facecolor='g',alpha=0.1,label='Acceptable region')
ax2.set_xlim(h_dum[0],h_dum[-1])
ax2.set_ylim(f2(h_min)*0.8,f2(h_min)*1.2)
ax2.set_xlabel('$h$ [mm]')
ax2.set_ylabel(r'E[$C_{tot}]$ [NOK]')
hjelpelinjer(ax2, h_opt, f2(h_opt),'b')
hjelpelinjer(ax2, h_min, f2(h_min),'g')
ax2.minorticks_on()
ax2.legend(loc='lower right')

fig.tight_layout()
plt.show()

if h_min <= h_opt:
    res = 'acceptable'
else:
    res = 'not acceptable'

print('Minimal accepted height: %.0f mm' % h_min)
print('The optimal design is %s.' %res)
```

```python

```


# Demo - Eigenvalues on the Möbius strip

**Mikael Mortensen** (email: `mikaem@math.uio.no`), Department of Mathematics, University of Oslo.

Date: **October 15, 2020**

**Summary.** This is a demonstration of how the Python module [shenfun](https://github.com/spectralDNS/shenfun) can be used to
compute eigenvalues and vectors of the Laplace-Beltrami
operator on a Möbius strip. The absolute value of the eigenvector
corresponding to the eigth smallest eigenvalue $\lambda=8.054788196$
is shown in the figure below, read on to see how it was computed.

## The Möbius strip

A Möbius strip is the simplest non-orientable surface embedded in
$\mathbb{R}^3$. There are several realizations possible, and we
will here consider the one parametrized by [[Kalvoda2020]](#Kalvoda2020)

$$
\begin{equation}
    x(\theta, t) = \left(R-t\cos\frac{\theta}{2R}\right) \cos \frac{\theta}{R},
\label{_auto1} \tag{1}
\end{equation}
$$

$$
\begin{equation}
    y(\theta, t) = \left(R-t\cos\frac{\theta}{2R}\right) \sin \frac{\theta}{R},
\label{_auto2} \tag{2}
\end{equation}
$$

$$
\begin{equation}
    z(\theta, t) = -t \sin \frac{\theta}{2 R},
\label{_auto3} \tag{3}
\end{equation}
$$

where $R$ is the main radius of the strip. $\theta$ is the parameter that determines
the angle for moving around the strip, like the angle of a circle.
For one trip around the strip move $\theta$ from $\theta_0$ to $\theta_0+2\pi R$.
A function in Cartesian coordinates $u(\mathbf{x})$ for $\mathbf{x} \in \mathbb{R}^3$
is mapped to computational coordinates as $u(\mathbf{x}) = \tilde{u}(\theta, t)$.
By moving once around the strip a function can be seen to be twisted periodic,
such that

$$
\begin{equation}
    \tilde{u}(\theta_0, t) = \tilde{u}(\theta_0 + 2\pi R, -t),
\label{_auto4} \tag{4}
\end{equation}
$$

$$
\begin{equation}
    \frac{\partial \tilde{u}}{\partial \theta}(\theta_0, t) = \frac{\partial \tilde{u}}{\partial \theta}(\theta_0 + 2\pi R, -t).
\label{_auto5} \tag{5}
\end{equation}
$$

The twisted condition does not lend itself easily to a regular tensor
product basis. On the other hand, by moving twice around the strip,
regular periodic boundary conditions

$$
\begin{equation}
    \tilde{u}(\theta_0, t) = \tilde{u}(\theta_0 + 4\pi R, t),
\label{_auto6} \tag{6}
\end{equation}
$$

will apply [[Kalvoda2020]](#Kalvoda2020), and we can discretize using Fourier
exponentials in the $\theta$-direction. Since the reference domain of Fourier
exponentials is $[-\pi, \pi]$ we define $\varphi = \theta /(2 R)$, such that
the computational domain is $(\varphi, t) \in \mathbb{I}^2 = [-\pi, \pi] \times [-1, 1]$,
with the parametrization

$$
\begin{equation}
    x(\varphi, t) = \left(R-t\cos {\varphi}\right) \cos {2 \varphi},
\label{_auto7} \tag{7}
\end{equation}
$$

$$
\begin{equation}
    y(\varphi, t) = \left(R-t\cos {\varphi}\right) \sin {2 \varphi},
\label{_auto8} \tag{8}
\end{equation}
$$

$$
\begin{equation}
    z(\varphi, t) = -t \sin {\varphi}.
\label{_auto9} \tag{9}
\end{equation}
$$

Note that the reference domain corresponds to $(\theta, t) \in = [-2\pi R, 2\pi R] \times [-1, 1]$.
It is also trivial to adjust the $t$-domain with an affine map for a more narrow or
wider strip, but this added complexity is not discussed here. One can simply
choose the width of the strip below when choosing function space for the
$t$-direction.

For the $t$-direction, a mixed Legendre,
$\psi_{i} = L_{i} - L_{i+2}$, or Chebyshev, $\psi_{i} = T_{i} - T_{i+2}$,
basis can be used and we obtain a regular tensor product basis.
The Legendre basis leads to more sparse matrices, and will be chosen
as default, but Chebyshev also works just fine.
Note that the same problem is
solved by Kalvoda et al. [[Kalvoda2020]](#Kalvoda2020) using a tensor product
basis with Fourier exponentials for the $\theta$-direction and a mix
of cosines and sines for the $t$-direction. Kalvoda et al. cannot
make use of tensor product matrices and integrates using a
two-dimensional quadrature scheme over the entire domain, leading
to a dense matrix. We will here only use 1D quadrature and
tensor products to get the coefficient matrix.

## The Laplace Beltrami operator

We consider the eigenvalue problem of the
Laplace Beltrami operator by solving

$$
\begin{equation}
    -\nabla^2 u(\mathbf{x}) = \lambda u(\mathbf{x}), \quad \text{ for } \mathbf{x} \in \Omega,
\label{_auto10} \tag{10}
\end{equation}
$$

where $u$ is the solution, $\lambda$ the eigenvalues and $\Omega$ the Möbius
strip. We consider only homogeneous Dirichlet boundary conditions on the boundary.

To solve this problem with the spectral Galerkin method we
choose an appropriate space $V$ and find $u\in V$ such that

$$
\int_{\Omega} -\nabla^2 u \, v^* \omega d\sigma = \int_{\Omega} \lambda u v^* \omega d\sigma, \quad \forall v \in V
$$

where $v^*$ is the complex conjugate of the test function $v$.

With shenfun it is enough to operate in general coordinates as above
and let the curvilinear mathematics all happen under the hood.
However, for this example it is interesting to see what the
Laplace-Beltrami operator looks like in computational coordinates.
This is quite a bit of work by hand, and the starting point
is the position vector

$$
\begin{equation}
\mathbf{r} = x(\varphi, t) \mathbf{i} + y(\varphi, t) \mathbf{j} + z(\varphi, t) \mathbf{k}.
\label{_auto11} \tag{11}
\end{equation}
$$

From the position vector we compute the he covariant
basis vectors $\mathbf{b}_i = \partial \mathbf{r} / \partial X^{i}$,
where $\mathbf{X} = \{X^{i}\}_{i\in(1,2)} = (\varphi, t)$,
and get the covariant metric tensor $g_{ij}=\mathbf{b}_i \cdot \mathbf{b}_j$
and its determinant $g = \text{det}\{g_{ij}\}$. We get

$$
\begin{equation}
    {g} = (2R-2t\cos \varphi)^2+t^2.
\label{_auto12} \tag{12}
\end{equation}
$$

and

$$
g_{ij} =
\begin{pmatrix}
  g & 0 \\
  0 & 1
\end{pmatrix}
$$

Likewise, the contravariant metric tensor $g^{ij}$ is the
inverse of the covariant

$$
g^{ij} =
\begin{pmatrix}
  \frac{1}{g} & 0 \\
  0 & 1
\end{pmatrix}
$$

Please note that all these metrics and other terms are computed
under the hood by shenfun, and a user does not normally have to worry
about these.

It can be shown that the Laplace-Beltrami operator in curvilinear coordinates
is given as

$$
\begin{equation}
\nabla^2 \tilde{u} = \frac{1}{\sqrt{g}}\frac{\partial}{\partial X^{i}}\left( g^{ij}\sqrt{g} \frac{\partial \tilde{u}}{\partial X^{j}}\right),
\label{_auto13} \tag{13}
\end{equation}
$$

with summation on repeated indices.
Using this and the surface element $d\sigma=\sqrt{g} d\varphi dt$, the
variational form in computational coordinates becomes

$$
\begin{equation}
    -\int_{\mathbb{I}^2}\frac{\partial}{\partial X^{i}}\left( g^{ij}\sqrt{g} \frac{\partial \tilde{u}}{\partial X^{j}}\right) \, \tilde{v}^*\, \tilde{\omega} d\varphi dt = \int_{\mathbb{I}^2} \lambda \tilde{u} \tilde{v}^*\, \tilde{\omega}\sqrt{g} d\varphi dt,
\label{_auto14} \tag{14}
\end{equation}
$$

Summing and expanding some derivatives, we get

$$
\begin{align*}
    -\int_{\mathbb{I}^2}  \Big\{ \frac{1}{\sqrt{g}}\frac{\partial^2 \tilde{u}}{\partial \varphi^2 } &+\frac{4 t \sin \varphi \left(t \cos {\varphi} -R \right) }{g^{\frac{3}{2}}}\frac{\partial \tilde{u}}{\partial \varphi  } \notag \\
    & +\sqrt{g}\frac{\partial^2 \tilde{u}}{\partial t^2 }
    +\frac{4 \cos {\varphi} (t \cos \varphi -R) + t}{ \sqrt{g}}\frac{\partial  \tilde{u}}{\partial t  }  \Big\} \tilde{v}^* \tilde{\omega} d\varphi dt \notag \\
    &= \int_{\mathbb{I}^2} \lambda \tilde{u} \tilde{v}^* \tilde{\omega} \sqrt{g} d\varphi dt,
\end{align*}
$$

Note that the variational problem contains unseparable variable coefficients,
like $1/\sqrt{g}$ and $\sqrt{g}$ and as such cannot
easily be solved using efficient tensor product algebra.
However, note that both $g$ and $g^2$ are separable (i.e., they can
be written as products of simpler functions separated by the arguments
$g(\varphi, t) = \sum_k g_1^k(\varphi)g_2^k(t)$),
and using the unconventional weight $\tilde{\omega} = g^{3/2}$ we get

$$
\begin{align*}
    -\int_{\mathbb{I}^2}  \Big\{ g \frac{\partial^2 \tilde{u}}{\partial \varphi^2 } &+ {4 t \sin \varphi \left(t \cos {\varphi} -R \right) } \frac{\partial \tilde{u}}{\partial \varphi  } \notag \\
    & + g^2 \frac{\partial^2 \tilde{u}}{\partial t^2 }
    +g \left({4 \cos {\varphi} (t \cos \varphi -R) + t} \right) \frac{\partial  \tilde{u}}{\partial t  }  \Big\} \tilde{v}^* d\varphi dt \notag \\
    &= \int_{\mathbb{I}^2} \lambda g^{2} \tilde{u} \tilde{v}^* d\varphi dt,
\end{align*}
$$

where the left hand side can be assembled using tensor product matrices,
where no single 1D matrix has more than 9 diagonals.

Fortunately we do not have to do all this by hand since we have a
software that automatically assembles such matrices for us.
Now let's see how this problem can be handled with shenfun.

## Implementation

First we need to create function spaces for each direction $\varphi$
and $t$, and then a tensor product space from the two. We use the
parametrization given above and shenfun will then automatically
differentiate to create basis functions and metrics, like $g$.
One important factor, though. Sympy's [simplify](https://docs.sympy.org/latest/modules/simplify/simplify.html)
will sometimes have problems finding the best possible simplification
of a term, like $g$. And in this particular case we need
to discourage the use of powers, or else sympy will end up with a
much more complicated $g$ than we get below.

```
from shenfun import *
import sympy as sp
from IPython.display import Math, Latex, display
from scipy.sparse.linalg import eigs
config['basisvectors'] = 'covariant'

phi, t = psi = sp.symbols('x,y', real=True)

RR = sp.Rational(132, 20)/sp.pi # Same as Kalvoda et al
#RR = 2
# Use a symbolic R first, then later substitute for the value in RR
R = sp.Symbol('R', real=True, positive=True)
rv = ((R-t*sp.cos(phi))*sp.cos(2*phi),
      (R-t*sp.cos(phi))*sp.sin(2*phi),
       -t*sp.sin(phi))

def discourage_powers(expr):
    POW = sp.Symbol('POW')
    count = sp.count_ops(expr, visual=True)
    count = count.replace(POW, 100)
    count = count.replace(sp.Symbol, type(sp.S.One))
    return count

N = (80, 40)
B0 = FunctionSpace(N[0], 'F', domain=(-np.pi, np.pi), dtype='D')
B1 = FunctionSpace(N[1], 'L', bc=(0, 0), domain=(-0.75, 0.75)) # Use same domain as Kalvoda et al
T = TensorProductSpace(comm, (B0, B1), coordinates=(psi, rv, True, (), discourage_powers), axes=(1, 0))

u = TrialFunction(T)
v = TestFunction(T)
```

Note the `discourage_powers` function. Try to turn it off (if
you are watching this in an interactive setting) and see what happens
to, e.g., `T.coors.sg`, which is $\sqrt{g}$.

We can now check to see what the Laplace-Beltrami operator
looks like when computed with shenfun:

```
g = sp.Function('g')(phi, t)
replace = [(T.coors.sg**2, g)]
Math((div(grad(u))).tolatex(funcname='f', symbol_names={phi:'\\varphi', t:'t'}, replace=replace))
```

Not surprisingly, this is the same (except not multiplied by $\sqrt{g}$) as what
is seen in the variational form above.

At this point we replace the symbol R with a number  in order to
assemble floating point matrices. The number R can be changed
above as the variable `RR`. For now it is set to use a value used also
by Kalvoda et al., such that we can doublecheck our eigenvalues.
We multiply forms with
$g^{3/2}$ before assembling to get separable variational forms,
leading to sparse tensor product matrices.

```
T.coors.subs(R, RR)
M = inner(v*T.coors.sg**3, -div(grad(u)))
B = inner(v*T.coors.sg**3, u)
```

Here `M` and `B` are lists of instances of the tensor product
matrix class [TPMatrix](https://shenfun.readthedocs.io/en/latest/shenfun.html#shenfun.matrixbase.TPMatrix).

```
print(f'Number of matrices for M = {len(M)} and B = {len(B)}')
```

So quite a few matrices, but they are all sparse. We solve by
using a Kronecker product solver [Solver2D](https://shenfun.readthedocs.io/en/latest/shenfun.html#shenfun.la.Solver2D) that flattens the
tensor product matrices and solution vector C-style.

```
mm = la.Solver2D(M)
bb = la.Solver2D(B)
Mc = mm.mat.copy()
Bc = bb.mat.copy()
```

Finally, solve the eigenvalue problem using a sparse eigenvalue solver
from scipy, which is wrapping ARPACK. Note that ARPACK is better at
finding large than small eigenvalues and for this reason we use a shift-inverted
version, see [https://docs.scipy.org/doc/scipy/reference/tutorial/arpack.html](https://docs.scipy.org/doc/scipy/reference/tutorial/arpack.html).

```
f = eigs(Mc, k=40, M=Bc, which='LM', sigma=0)
```

We have now found all eigenvalues on the Möbius strip with two rotations. So
some of the eigenvalues/eigenvectors will not have the correct twisted
periodic boundary conditions. To get only the correct eigenvalues, we filter a little
bit, checking the boundary:

```
u_hat = Function(T)
eigvals = []
for i in range(f[1].shape[1]):
    u_hat[:, :-2] = np.reshape(f[1][:, i], T.dims())
    tt = u_hat.eval(np.array([[-np.pi, 0, -np.pi, 0], [0.5, -0.5, 0.65, -0.65]]))
    dt = Dx(u_hat, 0, 1).eval(np.array([[-np.pi, 0, -np.pi, 0], [0.5, -0.5, 0.65, -0.65]]))
    # Check for twisted periodic
    if abs(tt[0]-tt[1]+tt[2]-tt[3]) < 1e-7 and abs(dt[0]-dt[1]+dt[2]-dt[3]) < 1e-8:
        eigvals.append((i, f[0][i].real))
        print(f'Twisted eigenvalue {len(eigvals):2d} {i:2d} {f[0][i].real:2.12e} Error {np.linalg.norm(Mc*f[1][:, i] - f[0][i]*Bc*f[1][:, i]):2.4e}')
```

These are the lowest true eigenvalues of the Möbius strip.
We note that they are very similar to Table 2 in [[Kalvoda2020]](#Kalvoda2020). We can now
plot the eigenvectors using, e.g., mayavi or plotly.
Choose the eigenvalue number first and then the rest follows

```
l = 7
u_hat[:, :-2] = np.reshape(f[1][:, eigvals[l][0]], T.dims())
u_hat2 = u_hat.refine([2*N[0], 2*N[1]])
N0 = u_hat2.function_space().shape(False)[0]//2+1
fig = surf3D(u_hat2, slices=(slice(0, N0), slice(None)))
fig.show()
```

Or make subplot of several of the eigenvectors:

```
import plotly
from plotly.subplots import make_subplots
import plotly.graph_objects as go
rows = 3
cols = 2
fig = make_subplots(rows=rows, cols=cols, start_cell="top-left", specs=[[dict(type='surface')]*cols]*rows,
                    subplot_titles=(f'$\lambda_1={eigvals[0][1]:2.5f}$', f'$\lambda_3={eigvals[2][1]:2.5f}$',
                                    f'$\lambda_5={eigvals[4][1]:2.5f}$', f'$\lambda_7={eigvals[6][1]:2.5f}$',
                                    f'$\lambda_9={eigvals[8][1]:2.5f}$', f'$\lambda_{{11}}={eigvals[10][1]:2.5f}$'))
N0 = T.shape(False)[0]//2+1 # Remember, data is for two rounds around the strip, we need only 1
x, y, z = T.local_cartesian_mesh()
x, y, z = x[:N0], y[:N0], z[:N0]
d = {'visible': False, 'showgrid': False, 'zeroline': False}
for l in range(rows*cols):
    u_hat[:, :-2] = np.reshape(f[1][:, eigvals[2*l][0]], T.dims())
    s = go.Surface(x=x, y=y, z=z, surfacecolor=abs(u_hat.backward()[:N0]),
                   colorscale=plotly.colors.sequential.Jet,
                   showscale=False)
    fig.add_trace(s, row=l//cols+1, col=l%cols+1)
    scene = 'scene' if l == 0 else f'scene{l+1}'
    fig.update_layout({scene: {'xaxis': d, 'yaxis': d, 'zaxis': d, 'camera': {'eye': dict(x=0.85, y=0.85, z=0.85)}}})
fig.show()
```

Finally, plot the sparsity pattern of the coefficient matrix. You
need to zoom in order to see the pattern properly and for
this reason we use plotly instead of the more convenient
matplotlib [spy](https://matplotlib.org/3.3.1/api/_as_gen/matplotlib.pyplot.spy.html)
function (`%matplotlib notebook` does not work in an
interactive jupyterlab and as such a jupyter book session).
If you are curious, then please change basis to Chebyshev and
note the difference from Legendre. Otherwise, results should be very
much alike.

```
import plotly.express as px
z = np.where(abs(mm.mat.toarray()) > 1e-6, 0, 1).astype(np.bool)
fig = px.imshow(z, binary_string=True)
fig.show()
```

1.  **T. Kalvoda, D. Krejcirik and K. Zahradová**.
    Effective Quantum Dynamics on the Möbius Strip,
    *Journal of Physics A: Mathematical and Theoretical*,
    53(37),
    pp. 375201,
    [doi: 10.1088/1751-8121/ab8b3a](http://dx.doi.org/10.1088/1751-8121/ab8b3a),
    2020.


# ヤコビアン

```python
import sympy as sy
from sympy.printing.numpy import NumPyPrinter
from sympy import julia_code
from sympy.utilities.codegen import codegen
import tqdm
import os
from pathlib import Path

from kinematics import Global
```

```python
N=5
kinem = Global(N)
kinem.set_J_v_simple()
```

    computing kinematics...
    Matrix([[q_large[0, 0]], [q_large[1, 0]], [q_large[2, 0]]])
    Matrix([[q_large[3, 0]], [q_large[4, 0]], [q_large[5, 0]]])
    Matrix([[q_large[6, 0]], [q_large[7, 0]], [q_large[8, 0]]])
    Matrix([[q_large[9, 0]], [q_large[10, 0]], [q_large[11, 0]]])
    Matrix([[q_large[12, 0]], [q_large[13, 0]], [q_large[14, 0]]])
    done computing kinematics!

ヤコビアンを時間微分

コード生成

```python
cwd = str(Path().resolve())
base = cwd + "/derived"
xi = sy.Symbol("xi")
```

```python
# # Cコード生成
# cwd = str(Path().resolve())
# base = cwd + "/derived"
# dir_name = base + '/c_src/'
# os.makedirs(dir_name, exist_ok=True)

# xi = sy.Symbol("xi")

# def gen_c(f, name, dir_name):
#     [(c_name, c_code), (h_name, c_header)] = codegen(
#         name_expr=(name, f),
#         language="C",
#         project= name + "project",
#         to_files=False
#     )

#     f = open(dir_name + c_name, 'w')
#     f.write(c_code)
#     f.close()

#     f = open(dir_name + h_name, 'w')
#     f.write(c_header)
#     f.close()

# os.makedirs(dir_name + "Phi_s/", exist_ok=True)
# for i, Phi in enumerate(kinem.Phi_s):
#     gen_c(Phi.subs(kinem.xi_large[i, 0], xi), "Phi_"+str(i), dir_name + "Phi_s/")

# os.makedirs(dir_name + "J_s/", exist_ok=True)
# for i, J in enumerate(kinem.J_v_simples):
#     gen_c(J.subs(kinem.xi_large[i, 0], xi), "J_"+str(i), dir_name + "J_s/")

```

```python
# # pythonコード生成
# dir_name = base + '/python_src'
# os.makedirs(dir_name, exist_ok=True)

# numpy_word = "import numpy\ndef f(q_large, xi):\n    return "

# # for i, P in enumerate(kinem.Phi_s):
# #     name = dir_name + "/Phi_" + str(i) + ".py"
# #     f = open(name, 'w')
# #     f.write(numpy_word)
# #     f.write(NumPyPrinter().doprint(P.subs(kinem.xi_large[i, 0], xi)))
# #     f.close()

# for i, J in enumerate(kinem.J_v_simples):
#     name = dir_name + "/J_" + str(i) + ".py"
#     f = open(name, 'w')
#     f.write(numpy_word)
#     f.write(NumPyPrinter().doprint(J.subs(kinem.xi_large[i, 0], xi)))
#     f.close()

```

```python
# # juliaコード生成
# dir_name = base + '/julia_src'
# os.makedirs(dir_name + "/Phi", exist_ok=True)
# os.makedirs(dir_name + "/J", exist_ok=True)

# julia_word = "function f(q_large::Matrix{T}, xi_large::Matrix{T}) where T\n    "

# for i, P in enumerate(kinem.Phi_s):
#     name = dir_name + "/Phi" + "/Phi_" + str(i) + ".jl"
#     f = open(name, 'w')
#     f.write("module " + "Phi_" + str(i) + "\n")
#     f.write(julia_word)
#     f.write(julia_code(P))
#     f.write("\nend\nend")
#     f.close()

# for i, J in enumerate(kinem.J_v_simples):
#     name = dir_name + "/J" + "/J_" + str(i) + ".jl"
#     f = open(name, 'w')
#     f.write("module " + "J_" + str(i) + "\n")
#     f.write(julia_word)
#     f.write(julia_code(J))
#     f.write("\nend\nend")
#     f.close()
```


# CME 193 - Lecture 5 - SciPy

SciPy is a library built on top of NumPy that contains functionality common in scientific computing. Here is a list of subpackages it contains:

- cluster:	Clustering algorithms
- constants:	Physical and mathematical constants
- fftpack:	Fast Fourier Transform routines
- **integrate:	Integration and ordinary differential equation solvers**
- interpolate:	Interpolation and smoothing splines
- io:	Input and Output
- **linalg:	Linear algebra**
- ndimage:	N-dimensional image processing
- odr:	Orthogonal distance regression
- **optimize:	Optimization and root-finding routines**
- signal:	Signal processing
- sparse:	Sparse matrices and associated routines
- spatial:	Spatial data structures and algorithms
- special:	Special functions
- stats:	Statistical distributions and functions

Today we'll "tour" the ones in **bold**.

# Solving linear systems of equations
By far the most common operation in scientific computing is solving a system of equations, specified by $Ax = b$. For example:

$$
\begin{align}
3x + 2y + 5z &=  2 \\
x - y + z &= 4 \\
6y  + z &= -1
\end{align}
$$

```python
import numpy as np
import scipy.linalg as linalg
```

```python
a = np.array([[3, 2, 5], [1, -1, 1], [0, 5, 1]])
```

```python
b = np.array([2, 4, -1])
```

```python
x = linalg.solve(a, b)
x
```

    array([14.6,  1.6, -9. ])

```python
a @ x
```

    array([ 2.,  4., -1.])

```python
# A matrix with more rows than columns (more equations than unknowns)
a = np.array([[1, 2, 1],
              [1, 1, 2],
              [2, 1, 1],
              [1, 1, 1]])
b = np.array([4, 3, 5, 4])
```

```python
linalg.solve(a, b)
```

```python
linalg.lstsq(a, b)  # Finds the x that minimizes ||Ax - b||
```

    (array([2.05263158, 1.05263158, 0.05263158]),
     0.8421052631578952,
     3,
     array([4.35889894, 1.        , 1.        ]))

```python
x = linalg.lstsq(a, b)[0]
a @ x  # this is close to b
```

    array([4.21052632, 3.21052632, 5.21052632, 3.15789474])

```python
b
```

    array([4, 3, 5, 4])

```python
linalg.norm(a @ x - b)**2
```

    0.842105263157895

# Optimization using `scipy.optimize`

Often in scientific computing you'll want to find the minimum of a function. For example, consider the following function.

```python
import matplotlib.pyplot as plt

def f(x):
    return x**4 - x**2

x = np.linspace(-1.2, 1.2, 100)
y = f(x)

plt.plot(x, y)
plt.grid()
plt.xlabel("$x$", fontsize=16)
plt.ylabel("$x^4 - x^2$", fontsize=16)
plt.show()
```

```python
import scipy.optimize as opt
```

The simplest way to minimize a function is using [`scipy.optimize.minimize`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html).

```python
result = opt.minimize(f, x0=0, method='Nelder-Mead')
result
```

     final_simplex: (array([[0.707125 ],
           [0.7070625]]), array([-0.25, -0.25]))
               fun: -0.2499999993361326
           message: 'Optimization terminated successfully.'
              nfev: 48
               nit: 24
            status: 0
           success: True
                 x: array([0.707125])

```python
plt.plot(x, y)
plt.plot(result.x, f(result.x), "ro")

plt.grid()
plt.xlabel("$x$", fontsize=16)
plt.ylabel("$x^4 - x^2$", fontsize=16)
plt.show()
```

```python
def g(z):
    x, y = z
    return np.sin(x + 2) * np.sin(y + 2)

result = opt.minimize(g, x0=np.array([0, 1]))
result
```

          fun: -0.9999999999743093
     hess_inv: array([[ 1.03270707, -0.01102124],
           [-0.01102124,  1.0037377 ]])
          jac: array([ 6.79492950e-06, -2.29477882e-06])
      message: 'Optimization terminated successfully.'
         nfev: 32
          nit: 6
         njev: 8
       status: 0
      success: True
            x: array([-0.42919689,  2.71238668])

```python
# def f(x):
#     return x ** 2 + x ** 4

f = lambda x: x **2 + x ** 4
opt.minimize(f, x0=0.5)
```

          fun: 6.326558280167952e-17
     hess_inv: array([[0.49999901]])
          jac: array([-1.00677177e-09])
      message: 'Optimization terminated successfully.'
         nfev: 15
          nit: 3
         njev: 5
       status: 0
      success: True
            x: array([-7.95396648e-09])

```python
opt.minimize(lambda x: x ** 2 + x ** 4, x0=0.5)
```

          fun: 6.326558280167952e-17
     hess_inv: array([[0.49999901]])
          jac: array([-1.00677177e-09])
      message: 'Optimization terminated successfully.'
         nfev: 15
          nit: 3
         njev: 5
       status: 0
      success: True
            x: array([-7.95396648e-09])

```python
g = lambda x, y: x ** 2 + y * x ** 4
```

```python
g(10, 5)
```

    50100

## Linear programming
A very common case is linear programming (LP). These are optimization problems that can be written in the form

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & c^{T}x  \\
\text{subject to} \;\; & A_{ub}x \leq b_{ub} \\
& A_{eq}x = b_{eq}
\end{split}
\end{equation}
$$

Here, we are finding the vector $x$ that minimizes the dot product $c^T x$, where $c$ is some fixed vector, out of all $x$ that satisfy $A_{ub}x \leq b_{ub}$ and $A_{eq}x = b_{eq}$, where $A_{ub}$ and $A_{eq}$ are matrices and $b_{ub}$ and $b_{eq}$ are vectors.

## Exercise 6
Using Google and reading documentation are important parts of programming. `scipy.optimize` comes with specialized functions for solving linear programming problems.

Figure out how to solve LPs using `scipy.optimize`, and solve the following LP:
$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & x_1 + 2 x_2  \\
\text{subject to} \;\; & x_1 \leq 1 \\
& 5 x_1 + x_2 \geq 0
\end{split}
\end{equation}
$$

Note that the problem is equivalent to

$$
\begin{equation}
\begin{split}
\text{minimize} \;\; & c^T x  \\
\text{subject to} \;\; & A_{ub}x \leq b_{ub} \\
& A_{eq}x = b_{eq}
\end{split}
\end{equation}
$$
where
$$
c = \begin{pmatrix} 1 \\ 2 \end{pmatrix} \qquad
A_{ub} = \begin{pmatrix} 1 & 0 \\ -5 & -1 \end{pmatrix}, \qquad
b_{ub} = \begin{pmatrix} 1 \\ 0 \end{pmatrix}, \qquad
A_{eq} = 0, \qquad
b_{eq} = 0.
$$

We can see this because
$$
 \begin{pmatrix} 1 & 0 \\ -5 & -1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} x_1 \\ -5x_1 - x_2 \end{pmatrix}
$$

```python
# YOUR CODE HERE
c = np.array([1, 2])
A_ub = np.array([[1, 0], [-5, -1]])
b_ub = np.array([1, 0])
opt.linprog(c, A_ub=A_ub, b_ub=b_ub, bounds=(None, None))
```

         con: array([], dtype=float64)
         fun: -9.0
     message: 'Optimization terminated successfully.'
         nit: 2
       slack: array([0., 0.])
      status: 0
     success: True
           x: array([ 1., -5.])

## Curve fitting

```python
def f(x, a, b):
    return a * np.sin(b * x)

x = np.linspace(-2, 2, 30)
y = f(x, 1.3, 3) + 0.3 * np.random.randn(30)
plt.plot(x, y, 'ro')
plt.show()
```

```python
(a, b), cov = opt.curve_fit(f, x, y)
a, b
```

    (1.2708917080168607, 2.9696448413642798)

```python
x_fine = np.linspace(-2, 2, 200)
plt.plot(x_fine, f(x_fine, a, b))
plt.plot(x, y, 'ro')
plt.show()
```

# Numerical integration using `scipy.integrate`

```python
import scipy.integrate as integrate
import numpy as np
```

Here's how to compute $$
\int_0^\pi \sin(x) \,e^{-x} \,dx
$$

```python
def f(x):
    return np.sin(x) * np.exp(-x)
```

```python
f(1.5)
```

    0.22257121610821853

```python
integrate.quad(f, 0, np.pi)
```

    (0.5216069591318861, 5.791000558329331e-15)

```python
def f_antiderivative(x):
    return -np.exp(-x) * (np.sin(x) + np.cos(x)) / 2
```

```python
f_antiderivative(np.pi) - f_antiderivative(0)
```

    0.5216069591318861

```python
integrate.quad(f, 0, np.inf)
```

    (0.5000000000000002, 1.4875911973447031e-08)

```python
f_antiderivative(10000) - f_antiderivative(0)
```

    0.5

## Physical simulation
SciPy can solve ODEs (ordinary differential equations). Let's simulate a throwing a ball in two dimensions. The ball is described as a function of time by four functions: $x(t), y(t), v_x(t), v_y(t)$, which are governed by the ODE system:

$$
    \frac{dx}{dt} = v_x, \qquad
    \frac{dy}{dt} = v_y, \qquad
    \frac{dv_x}{dt} = 0, \qquad
    \frac{dv_y}{dt} = -g.
$$

We can think of this in vectorized form as:

$$
    \frac{d}{dt} \begin{pmatrix} x \\ y \\ v_x \\ v_y \end{pmatrix} = \begin{pmatrix} v_x \\ v_y \\ 0 \\ -g \end{pmatrix}
$$

```python
def dzdt(z, t):
    x, y, vx, vy = z
    g = 1
    return np.array([vx, vy, 0, -g])
```

```python
np.linspace(0, 50, 20)
```

    array([ 0.        ,  2.63157895,  5.26315789,  7.89473684, 10.52631579,
           13.15789474, 15.78947368, 18.42105263, 21.05263158, 23.68421053,
           26.31578947, 28.94736842, 31.57894737, 34.21052632, 36.84210526,
           39.47368421, 42.10526316, 44.73684211, 47.36842105, 50.        ])

```python
import matplotlib.pyplot as plt
```

```python
x, y = 0, 0
vx, vy = 1, 20
z = np.array([x, y, vx, vy])
t = np.linspace(0, 50, 20)

result = integrate.odeint(dzdt, z, t)

plt.scatter(result[:, 0], result[:, 1])
plt.show()
```

## Exercise 7
We can add drag with the following slight modification to the ODE, which adds a force with direction opposing the current velocity, and with magnitude proportional to the velocity squared.

$$
    \frac{d}{dt} \begin{pmatrix} x \\ y \\ v_x \\ v_y \end{pmatrix} = \begin{pmatrix} v_x \\ v_y \\ -\alpha v_x \sqrt{v_x^2 + v_y^2} \\ -g -\alpha v_y \sqrt{v_x^2 + v_y^2} \end{pmatrix}
$$

Implement this with $\alpha = 0.001$, plot the resulting trajectory, and compare to the dragless case.

```python
def dzdt_drag(z, t):
    x, y, vx, vy = z
    g = 1
    alpha = 0.01
    v = np.sqrt(vx**2 + vy**2)
    return np.array([vx, vy, -alpha * vx * v, -g - alpha * vy * v])
```

```python
result_drag = integrate.odeint(dzdt_drag, z, t)

plt.scatter(result[:, 0], result[:, 1])
plt.scatter(result_drag[:, 0], result_drag[:, 1])
plt.show()
```


# Software profesional en Acústica 2020-21 (M2i)

*This notebook was adapted from Chapter 1 of [The FEniCS Tutorial Volume I](https://fenicsproject.org/pub/tutorial/sphinx1/) by Hans Petter Langtangen and Anders Logg, released under CC Attribution 4.0 license. It has been created by Xiangmin Jiao (University of Stony Brook University) and it is available in the repository [Unifem/FEniCS-note](https://github.com/unifem/fenics-notes).*

# The equations of vibrations in fluid-structure interaction (displacement formulation)

Vibrations in linear elasticity is the study of how solid objects are responding to a mechanical vibration and become
internally stressed due to prescribed time-harmonic loading conditions. It is an important problem
in modern engineering. Its corresponding PDE is a generalization of the
Helmholtz equation, and it is among one of the most popular PDEs in
engineering. We now study its variational formulation of a fluid-structure interaction and how to solve
this problem using FEniCS in 2D.

## PDE problem

The time-harmonic equation governing vibrations of a fluid-structure problem involving a elastic solid structure $\Omega_S$ and a fluid domain $\Omega_F$ can be written as

\begin{align}
\label{ftut-elast-varform-equilibrium}\tag{1}
&-\omega^2\rho_S\boldsymbol{u}_S -\boldsymbol{\nabla}\cdot\boldsymbol{\sigma} = \boldsymbol{0}\hbox{ in }\Omega_S,\\
&-\omega^2\rho_F\boldsymbol{u}_F -\rho_Fc^2\nabla\mathrm{div}\boldsymbol{u}_F = \boldsymbol{0}\hbox{ in }\Omega_F,
\label{ftut-fluid-varform-equilibrium}\tag{2}
\end{align}

where $\boldsymbol{\sigma}$ is the *stress tensor*, $c$ is the sound speed in the fluid domain, $\boldsymbol{u}_S$ and $\boldsymbol{u}_F$ are the displacement fields in the solid anf fluid domain,
$\rho_S$ and $\rho_F$ are the *mass density* of the solid and fluid domain, and $\omega$ the angular frequency. For isotropic materials, the stress tensor is further related to the deformation by
the following two equations:
\begin{align}
\boldsymbol{\sigma} &= \lambda\,\hbox{tr}\,(\boldsymbol{\varepsilon}) \boldsymbol{I} + 2\mu\boldsymbol{\varepsilon},
\label{ftut-elast-varform-stresstrain}\tag{3}\\
\boldsymbol{\varepsilon} &= \frac{1}{2}\left(\boldsymbol{\nabla} \boldsymbol{u}_S + (\boldsymbol{\nabla} \boldsymbol{u}_S)^{\top}\right),
\label{ftut-elast-varform-strainu}\tag{4}
\end{align}
where $\boldsymbol{\varepsilon}$ is the *symmetric strain-rate tensor* (symmetric gradient),
and $\boldsymbol{u}$ is the *displacement vector field*, $\boldsymbol{I}$ denotes the *identity tensor*,
$\mathrm{tr}$ denotes the *trace operator* on a tensor, and $\lambda$ and $\mu$
are material properties known as *Lamé's elasticity parameters*.

We can combine (\ref{ftut-elast-varform-stresstrain}) and
(\ref{ftut-elast-varform-strainu}) to obtain
\begin{equation}
\label{ftut-elast-varform-stressu}\tag{5}
\boldsymbol{\boldsymbol{\sigma}} = \lambda(\boldsymbol{\nabla}\cdot \boldsymbol{u}_S)\boldsymbol{I} + \mu(\boldsymbol{\nabla} \boldsymbol{u}_S + (\boldsymbol{\nabla} \boldsymbol{u}_S)^{\top})
\end{equation}

Note that
(\ref{ftut-elast-varform-equilibrium})-(\ref{ftut-elast-varform-strainu})
can easily be transformed to a single vector PDE for $\boldsymbol{u}_S$, which is the
governing PDE for the unknown $\boldsymbol{u}_S$ (Navier's equation).  In the
derivation of the variational formulation, however, it is convenient
to keep the equations split as above.

## Variational formulation

The variational formulation of
([1](#mjx-eqn-ftut-elast-varform-equilibrium))-([4](#mjx-eqn-ftut-elast-varform-strainu))
consists of forming the inner product of
([1](#mjx-eqn-ftut-elast-varform-equilibrium))-([2](#mjx-eqn-ftut-fluid-varform-equilibrium)) and two *vector* test functions
$(\boldsymbol{v}_F,\boldsymbol{v}_S)\in \hat{V}$, where $\hat{V}$ is a vector-valued test function space in $\Omega_F$ and $\Omega_S$ such that
\begin{align}
\boldsymbol{v}_F\cdot \mathbf{n} = \boldsymbol{v}_S\cdot \mathbf{n},\\
\boldsymbol{\sigma}(\boldsymbol{u}_S)\mathbf{n} = -\rho_F c^2\mathrm{div}\boldsymbol{u}_{F}\mathbf{n},
\end{align}

on the coupling boundary $\Gamma_I=\partial\Omega_F\cap\partial\Omega_F$. So, integrating over the domain $\Omega_F\cup\Omega_S$ and taking into account the symmetry of the tensor of elasticity, it holds

\begin{align}
-\omega^2\int_{\Omega_S} \rho_S\boldsymbol{u}_S\cdot \boldsymbol{v}_S\ \mathrm{d}\boldsymbol{x}
 + \int_{\Omega_S} \boldsymbol{\sigma}(\boldsymbol{u}_S) : \boldsymbol{\epsilon}(\boldsymbol{v}_S)\ \mathrm{d}\boldsymbol{x}
 -\omega^2\int_{\Omega_F} \rho_F\boldsymbol{u}_F\cdot \boldsymbol{v}_F\ \mathrm{d}\boldsymbol{x}
 + \int_{\Omega_F} \rho_F c^2\mathrm{div}\boldsymbol{v}_F\,\mathrm{div}\boldsymbol{v}_F \mathrm{d}\boldsymbol{x}
 = \int_{\Gamma_T} \boldsymbol{T}\cdot \boldsymbol{v}_S\ \mathrm{d}\boldsymbol{s}
\label{ftut-elast-varform-sigma_inner_gradv}\tag{8}
\end{align}

for all $(\boldsymbol{v}_F,\boldsymbol{v}_S)\in \hat{V}$ such that $\boldsymbol{u}_S=\mathbf{0}$ on the campled boundary $\Gamma_C=\{\mathbf{x}\in\partial\Omega_{S}: x_0=0\}$ and the traction boundary $\Gamma_T=\{\mathbf{x}\in\partial\Omega_{S}: x_0=L\}$. In addition,
$\boldsymbol{\epsilon}(\boldsymbol{v})$ is the symmetric part of $\boldsymbol{\nabla} \boldsymbol{v}$.

### Enforcing boundary conditions

Now let us consider how to enforce boundary conditions.
For Dirichlet boundaries, we will enforce boundary-conditions strongly.
For these points, no test functions are associated with the Dirichlet nodes.

For traction boundary conditions, we will enforce the boundary condition
weakly using the variational form ([8](#mjx-eqn-ftut-elast-varform-sigma_inner_gradv)).
Similar to the Helmholtz equation, we require their corresponding test
functions $\boldsymbol{v}_S$ vanish along $\Gamma_C$.
Then, the boundary integral above has no effects for points on
$\partial\Omega_S\setminus\Gamma_T$.

### Summary of variational form
In summary, the variational problem is to find $\boldsymbol{u}$ in a vector function space $\hat{V}$ such that
\begin{equation}
a((\boldsymbol{u}_F,\boldsymbol{u}_S),(\boldsymbol{v}_F,\boldsymbol{v}_S)) = L((\boldsymbol{v}_F,\boldsymbol{v}_S))\quad\forall (\boldsymbol{v}_F,\boldsymbol{v}_S)\in\hat{V},
\end{equation}
where
\begin{align}
a((\boldsymbol{u}_F,\boldsymbol{u}_S),(\boldsymbol{v}_F,\boldsymbol{v}_S)) &= -\omega^2\int_{\Omega_S} \rho_S\boldsymbol{u}_S\cdot \boldsymbol{v}_S\ \mathrm{d}\boldsymbol{x}
 + \int_{\Omega_S} \boldsymbol{\sigma}(\boldsymbol{u}_S) : \boldsymbol{\epsilon}(\boldsymbol{v}_S)\ \mathrm{d}\boldsymbol{x}
 -\omega^2\int_{\Omega_F} \rho_F\boldsymbol{u}_F\cdot \boldsymbol{v}_F\ \mathrm{d}\boldsymbol{x}
 + \int_{\Omega_F} \rho_F c^2\mathrm{div}\boldsymbol{v}_F\,\mathrm{div}\boldsymbol{v}_F \mathrm{d}\boldsymbol{x}
\end{align}
and
\begin{equation}
\boldsymbol{\sigma}(\boldsymbol{u}_S) = \lambda(\boldsymbol{\nabla}\cdot \boldsymbol{u}_S)\boldsymbol{I} + \mu(\boldsymbol{\nabla} \boldsymbol{u}_S + (\boldsymbol{\nabla} \boldsymbol{u}_S)^{\top}).\\
\end{equation}

## FEniCS implementation

To demonstrate the implementation, we will model a clamped beam deformed under a time-harmonic surface force on the opposite free cross-section surface. This can be modeled by setting $\boldsymbol{T}=(0,0,1)$ on that boundary $\Gamma_T$. The solid structure is box-shaped with length and width $L$, whereas the fluid domain is an interior box-shaped domain with length and width $W<L$. We
set $\boldsymbol{u}=(0,0,0)$ at the clamped end, $x=0$. The rest of the lateral boundary is
traction free; that is, we set $\boldsymbol{T} = 0$. Therefore,
$$L((\boldsymbol{v}_F,\boldsymbol{v}_S)) = \int_{\Gamma_T} \boldsymbol{T}\cdot \boldsymbol{v}_S \mathrm{d}\boldsymbol{s}$$
for this problem.

### Import packages

We start by importing fenics and enforcing matplotlib. In addition, we import `mshr` for mesh generation.

```python
import numpy as np
from dolfin import *
from mshr import *
import matplotlib.pyplot as plt
%matplotlib inline
```

### Generate the mesh and function spaces

Our action startes by generating meshes and defining function spaces.

```python
# Create mesh and define function space
length = 1; width = 1.
length_fluid = 0.6; width_fluid = 0.6

# Mesh
N = 8 # use positive integer values
mesh = RectangleMesh(Point(0, 0), Point(length, width), N*10, N*10, "right/left")
V = VectorFunctionSpace(mesh, 'P', 2)
```

To define the partition of the computational domain attending to the fluid and solid subdomains, each triangle of the finite element mesh is marked with a number flag or *marker*:

```python
# Initialize subdomain and boundary markers
tol = 1e-5
fluid_domain = CompiledSubDomain('(fabs(x[0]-0.5) < L0/2. + tol) and (fabs(x[1]-0.5) < L1/2. + tol)', L0=length_fluid, L1=width_fluid, tol=tol)

# Initialize mesh function for boundary
domain_markers = MeshFunction('size_t', mesh, mesh.topology().dim())
domain_markers.set_all(1)
fluid_domain.mark(domain_markers, 2)

# Define the boundary measure with the boundary markers
dx = Measure('dx', domain=mesh, subdomain_data=domain_markers)

plot(domain_markers)
plt.show()
```

### Define the variational problem

The primary unknown is now a vector field $\boldsymbol{u}$ and not a scalar field,
so we need to work with a vector function space. We will use
piecewise-linear basis functions for all the components.

```python
u = TrialFunction(V)
v = TestFunction(V)
```

With `u = TrialFunction(V)` we get `u` as a vector-valued finite element
function with three components for this 2D problem.

Next, we define the stress tensor and $a$. The gradient and divergence operators
now have a prefix `nabla_`.
This is not strictly necessary in the present problem, but is
recommended in general for vector PDEs arising from continuum mechanics,
if you interpret $\boldsymbol{\nabla}$ as a vector in the PDE notation. See
the notes on `grad(u)` vs. `nabla_grad(u)` below.

```python
from ufl import nabla_div

# Define strain and stress
def epsilon(u):
    return 0.5*(nabla_grad(u) + nabla_grad(u).T)
    #return sym(nabla_grad(u))

# Frequency
omega = 2*np.pi*1.0

# Physical constants for the solid
rho_fluid = 0.2
c = 1.

# Physical constants for the solid
rho = 1.
beta = 1.25
lambda_ = beta
mu = 1
d = u.geometric_dimension()  # space dimension

def sigma(u):
    return lambda_*nabla_div(u)*Identity(d) + 2*mu*epsilon(u)

# Define a
a = -omega**2*rho*inner(u, v)*dx(1) + inner(sigma(u), epsilon(v))*dx(1) -omega**2*rho_fluid*inner(u, v)*dx(2) + rho_fluid*c**2*div(u)*div(v)*dx(2)
```

To define the partition of the boundary in the computational domain, each face of the finite element mesh is marked with a number flag or *marker*:

```python
# Initialize subdomain and boundary markers
tol = 1e-5
clamped_boundary = CompiledSubDomain('on_boundary and near(x[0],L,tol)', L=0., tol=tol)
traction_boundary = CompiledSubDomain('on_boundary and near(x[0],L,tol)', L=length, tol=tol)

# Initialize mesh function for boundary
boundary_markers = MeshFunction('size_t', mesh, mesh.topology().dim() - 1)
boundary_markers.set_all(0)
clamped_boundary.mark(boundary_markers, 1)
traction_boundary.mark(boundary_markers, 2)

# Define the boundary measure with the boundary markers
ds = Measure('ds', domain=mesh, subdomain_data=boundary_markers)
```

To define $L$, $\boldsymbol{f}=(0, 0, 1)$ is a constant vector, instead of a scalar.
Such a vector constant is specified as `Constant(0, 0, 1)` in FEniCS.

```python
# Define L
f = Constant((0, 1))
L = dot(f, v)*ds(2)
```

### Define boundary conditions

We only specify the Dirichlet boundary condition. For the boundary condition
$u=(0, 0, 0)$, we must set a vector value to zero, not just a scalar.
We specify the vector constant as `Constant((0, 0, 0))`.

```python
# Define boundary condition
bc = DirichletBC(V, Constant((0, 0)), boundary_markers, 1)
```

### Solve the variational problem

Finally, we can solve the problem.

```python
# Compute solution
u = Function(V)
solve(a == L, u, bc)
```

### Plot the solution

Any component of the solution can be computed or even the modulus of the vector-field solution

```python
# Plot the Finite Element approximation
def plot_solution(u):
    '''plot solution of FEM-based simulation'''
    fig = plt.figure(figsize=(10,10))
    fig = plot(u)
    plt.xlabel(r'$x$ / m')
    plt.ylabel(r'$y$ / m')
    plt.colorbar(fig, fraction=0.038, pad=0.04);

# Plot vector field
plot_solution(u)

# Compute magnitude of displacement
u_magnitude = sqrt(dot(u, u))
Q = FunctionSpace(mesh, 'P', 2)
u_magnitude = project(u_magnitude, Q)
plot_solution(u_magnitude)
```

```python
# Plot second component of the solution
plot_solution(u[1])
```

```python
# Plot first component of the solution
plot_solution(u[0])
```