# MFS based MD Simulation

### A Sound scattering simulator based on the method of fundamental solutions

#### Contributors:
*Nicholas St. Clair, Physics (Kleckner Lab), University of California, Merced*

*Arnold Kim, Applied Mathematics, University of California, Merced*

*Dustin Kleckner, Physics, University of California, Merced*

#### Description:
The following notebook makes use of the method of fundamental solutions to compute the sound scattered from multiple sound hard scattering bodies. It determines the forces resultant from said scattering events, some of which are non-linear interactions brought about by interference of acoustic fields. The force data from the MFS scattering code is then fed into a Runge-Kutta based step integrator to produce a molecular dynamics type simulation of particle dynamics.

# 1. Relevant Libraries and Functions

```python
#Import relevant python libraries, as well as MFS master class
import numpy as np
import scipy.special as sp
import time
import matplotlib.pyplot as plt
from mfs import mfs as MFS
import csv
from scipy.integrate import solve_ivp
```

```python
'''A function to assign random initial positions to a group of particles,
the x & y coordinates are both drawn from a Gaussian distribution, whose spread relative to
the particle size can be specified. A few if/else statements ensure that none of the particles
are initially overlapping.'''

def init_config(a,Np,σ=3):
    #σ: spread in distribution size
    #a: particle radius
    #Np: Number of particles for which to definie initial locations

    #X,Y are arrays that contain all of the coordinates of the particles in the config
    #Initialize the coordinates of the first particle of config in X,Y
    X = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
    Y = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
    #X_t,Y_t are temporary coordinates of the next particle to be added to the config
    X_t = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
    Y_t = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
    #ch is a checker used to indicate overlapping particles
    ch = 0

    #hold this loop open until coordinate arrays contain positions for Np particles
    while len(X) < Np:
        for i in range(len(X)):
            #Check if the next particle to be added is overlapping with any of the current particles
            if np.sqrt((X[i] - X_t)**2 + (Y[i] - Y_t)**2) < (2.0 * a):
                #if so add 1 to the checker variable
                ch += 1
            else:
                #otherwise do nothing
                ch += 0
        #if the checker variable is zero, it means the temporary particle position does not
        #overlap with any of the current particles
        if ch == 0:
            #so we add the temporary position to the existing array of particle locations
            X = np.append(X,X_t)
            Y = np.append(Y,Y_t)
        #Otherwise if the checker variable is nonzero, we have an overlap
        elif ch > 0:
            #so we define a new set of temporary coordinates to be tested
            X_t = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
            Y_t = np.random.normal(loc = 0.0, scale = σ * a, size = 1 )
            #and reset the checker variable
            ch = 0
    #Once we have a satisfying array of particle coordinates (size of array = number of desired particles,
    #and none of the particles overlap, return that array)
    return(X,Y)
```

## 2. Base differential equation
The following cell defines a function which takes a 1 X (6*Np) array (called M below) containing current position and velocity info and returns an array of the same shape with info updated according to the current velocity and force data (Np: Particle Number). The function also takes particle radius (a), particle mass (m), and dynamic viscosity of host fluid ($\mu$), in order to determine the Stokes Drag force on a moving particle.

*Unfortunately the integrator only accepts 1D arrays...so the differential equation to be solved must take the following form:*

\begin{equation}
     \begin{bmatrix}
         x_{i,t+\Delta t}\\
         y_{i,t+\Delta t}\\
         z_{i,t+\Delta t}\\
         v_{i,t+\Delta t}^{x}\\
         v_{i,t+\Delta t}^{y}\\
         v_{i,t+\Delta t}^{z}\\
         \vdots
     \end{bmatrix} \  = \
     \begin{bmatrix}
         v_{i,t}^{x}\\
         v_{i,t}^{y}\\
         v_{i,t}^{z}\\
         a_{i,t}^{x}\\
         a_{i,t}^{y}\\
         a_{i,t}^{z}\\
         \vdots
     \end{bmatrix}
     \Delta t, \ \ \ i=1,2,...Np
\end{equation}

The MFS class is responsible for determing the acoustic forces resultant on all particles, as well as any contact forces between pairs of particles that come too close. This in invoked by calling the force() and contact() methods from the MFS class. An instance of the scatter class is passed to the solver, this instance only need be called once before starting the solver.

The method of determining an acoustic radiation force on one or multiple particles is detailed in the Jupyter notebooks titled: "Acoustic Radiation Force on a single sound hard sphere", and "Acoustic forces between two and three spheres". The user is referred to those notebooks for a comprehensive understanding of the Method of Fundamental Solutions for determining the acoustically induced forces on sound-hard, spherical, scattering bodies.

The contact forces computed between pairs of particles are determined according to a power law scheme, whereby the force between two particles goes as the separation distance between those particles raised to some power:

\begin{equation}
        \vec{F}_{contact} = C*F_{0} \left[ \frac{\alpha - (\delta /d)}{\alpha - 1} \right]^{\beta} \hat{\delta}
\end{equation}

In the above $\alpha$ determines the "cutoff distance", i.e. the location within which to begin calculating the contact force. $\delta$ is the interparticle separation distance, d is the particle diameter, $\beta$ controls the sharpness of the force as it goes from zero to it's max value, and $C*F_{0}$ is the max value of the force, with $C$ ultimately controlling that value. $F_{0}$ is the reference acoustic force experienced by a perfect sound absorbing sphere within an external pressure field. The reference acoustic force is simply the acoustic energy density contained in the external field, multiplied by the particle volume, and is defined as:

\begin{equation}
    F_{0} = \frac{P_{0}^{2}}{4\rho_{0}c_{0}^2}\frac{4}{3}\pi a^{3}
\end{equation}

where $P_{0}^{2}$ is the pressure amplitude of the external field, $\rho_{0}$ is the mass density of the host medium, and $c_{0}$ is the sound speed in the host medium.

```python
def Binding(t, M, Np, a, m, μ, scat):
    #store current position and velocity data to separate arrays
    X = M.reshape(Np,6)[:,:3]
    V = M.reshape(Np,6)[:,3:6]
    #feed the class the current position data and use to compute forces
    scat.solve(X)
    Ft = scat.force() + scat.contact()
    #Determine current acceleration data (including viscous contributions)
    A = (Ft - 6*np.pi*μ*a*V)/m
    #Add acceleration due to gravity in the z-direction
    A[:,2] -= 9.8
    #zip together current velocity and acceleration data to update particle kinematics
    Out = np.array(list(zip(V,A))).flatten()
    return Out;
```

## 3. Runge-Kutta based solver

The following cell defines initial conditions, as well as any global parameters necessary to set up the simulation.
It then feeds the initial conditions to the solver to be run for a predetermined amount of time.

```python
#time range of simulation
t_0 = 0
t_f = 10

#parameters for MFS class, as well as damping parameters used in differential equation
#MFS number (number of source points)
N = 92
#quadrature number (actual number of integration pointns is 2*Nq^2)
Nq = 8
# acoustic wavenumber (1/m)
k = 733
#wave speed (m/s)
c = 343
#angular frequency (rad/s)
ω = k*c
#size parameter
ka = 1.466
#particle radius (m)
a = ka / k
#mass density of air (kg/m^3)
ρ = 1.225
#incident field (velocity potential) amplitude (m^2 / s)
phi_a = 4e2 / (ρ * ω)
#particle mass density (kg/m^3)
ρp = 10
#particle mass (kg)
m = ((4 * np.pi / 3) * a ** 3) * ρp
#dynamic viscosity of air (kg/(m*s))
μ = 1.81e-5
#z level of group (m), generally set to be a stable levitation plane
zl = -0.0012542461099942367
#number of particles
Np = 5
#variance in initial particle configuration spacing (in terms of 'a', the particle radius)
σ = 3

#Loop through number of desired trials
num_trials = 100
for n in range(num_trials):

    #set particle locations in initial config
    pos = np.empty((Np,3))
    pos[:,0], pos[:,1] = init_config(a,Np,σ)
    pos[:,2] = np.full(Np,zl)

    #set particle velocities in initial config (generally nil)
    vel = np.zeros((Np,3))
    ICs = np.empty((Np,6))

    #load particle locations and velocities into an initial condition array, fed to the integrator
    ICs[:, :3] = pos
    ICs[:, 3:] = vel

    #call MFS scatter class with relevant parameters

    scat = MFS.Scatter(N=N, k=k, a=a, phi_a = phi_a, rho = ρ, source_depth=0.5)
    scat.incoming_planewaves([1, 1j], [(0, 0, 1), (0, 0, -1)])
    #run the simulation
    nsol = solve_ivp(lambda t,M: Binding(t, M, Np, a, m, μ, scat), [t_0,t_f], ICs.flatten(), max_step = 0.05 )

    #uncomment the lines below in order to save the simulation data from each of the trials.

    #specifiy path of folder within which to save simulation data for each trial
    path = 'MD_Data_raw/'

    #specify a unique file name for each trial run, and include size parameter in the file name
    filename = f'MD_Data_ka={ka}_({n}).csv'

    #write trial data to the unique file using the python csv class
    with open(path+filename, mode = 'w') as csvfile:
        writer = csv.writer(csvfile, delimiter = ',')
        #write the time data as first row
        writer.writerow(nsol.t)

        #loop over number of time steps in simulation
        for i in range(len(nsol.t)):
            #write the kinematic data from each time step
            writer.writerow(nsol.y[:,i])

    #A separate notebook is used to analyze the data, said notebook is also included in the github repo.
```

```python

```

Jupyter Notebooks/MFS -- Molecular Dynamics Simulation -- Data creation.ipynb


# MATHEMATICAL EPIDEMIOLOGY PRESENTATION

```python
import matplotlib.pyplot as plt
from scipy.integrate import solve_ivp
import numpy as np
import sympy as sp
import scipy.linalg as sl
from sympy.interactive import printing
printing.init_printing(use_latex=True)
```

```python
#Defining the symbols.
z_t,s_t,i_t= sp.symbols('z_t,s_t,i_t')
a_1,b_1,c_1,a_2,b_2,c_2,d_2=sp.symbols('a_1,b_1,c_1,a_2,b_2,c_2,d_2')
```

```python
#Equations representing our system
dot_z_t=-z_t+z_t*s_t-z_t*i_t
display(dot_z_t)
dot_s_t=s_t-z_t*s_t-0.5*s_t*i_t+i_t
display(dot_s_t)
dot_i_t=i_t-z_t*i_t+0.5*s_t*i_t-i_t
display(dot_i_t)
```

```python
#Creating equations from the expressions above.
zEq = sp.Eq(dot_z_t,0)
display(zEq)
sEq = sp.Eq(dot_s_t,0)
display(sEq)
iEq = sp.Eq(dot_i_t,0)
display(iEq)
```

```python
#Determining the equilibrium points.
EqPnts = sp.solve( (zEq,sEq,iEq), z_t,s_t,i_t )
print(EqPnts)
```

    [(0.0, 0.0, 0.0), (0.250000000000000, 0.500000000000000, -0.500000000000000), (1.00000000000000, 1.00000000000000, 0.0), (1.00000000000000, 2.00000000000000, 1.00000000000000)]

```python
#Linearization of the system.
Matrix=sp.Matrix([dot_z_t,dot_s_t,dot_i_t])
Matrix
```

```python
Mat=sp.Matrix([z_t,s_t,i_t])
Mat
```

```python
JacobianMat=Matrix.jacobian(Mat)
JacobianMat
```

```python
EV1=JacobianMat.eigenvals()
#display(EV1)
```

# Evaluating Jacobian Matrix at point (1,2,1)

```python
p=EqPnts[3][0]
q=EqPnts[3][1]
r=EqPnts[3][2]
Jacsub=JacobianMat.subs([(z_t,p),(s_t,q),(i_t,r)])
Jacsub
```

```python
eigenvalues=Jacsub.eigenvals()
eigenvalues
```

# Evaluating Jacobian Matrix at point (1,1,0)

```python
p=EqPnts[2][0]
q=EqPnts[2][1]
r=EqPnts[2][2]
Jacsub=JacobianMat.subs([(z_t,p),(s_t,q),(i_t,r)])
Jacsub
```

```python
eigenvalues=Jacsub.eigenvals()
eigenvalues
```

# Our equation without replacing the constants

```python
#Equations representing our system
Dot_z_t=-a_1*z_t+b_1*z_t*s_t-c_1*z_t*i_t
display(Dot_z_t)
Dot_s_t=a_2*s_t-b_2*z_t*s_t-c_2*s_t*i_t+d_2*i_t
display(Dot_s_t)
Dot_i_t=a_2*i_t-b_2*z_t*i_t+c_2*s_t*i_t-d_2*i_t
display(Dot_i_t)
```

```python
#Creating equations from the expressions above.
zEQ = sp.Eq(Dot_z_t,0)
display(zEQ)
sEQ = sp.Eq(Dot_s_t,0)
display(sEQ)
iEQ = sp.Eq(Dot_i_t,0)
display(iEQ)
```

```python
#Linearization of the system.
Matrix1=sp.Matrix([Dot_z_t,Dot_s_t,Dot_i_t])
Matrix1
```

```python
JacobianMat1=Matrix1.jacobian(Mat)
JacobianMat1
```

```python
p=EqPnts[3][0]
q=EqPnts[3][1]
r=EqPnts[3][2]
Jacsub1=JacobianMat1.subs([(z_t,p),(s_t,q),(i_t,r)])
Jacsub1
```

```python
eigenvalues=Jacsub.eigenvals()
eigenvalues
```

# Plotting the Graph.

```python
#Stating the initial conditions given.
z_t=0.9
s_t=1.9
i_t=0.8
#Stating the parameter values given
a_1=1;b_1=1;c_1=1;a_2=1;b_2=1;c_2=0.5;d_2=1

#Definig a function of the system given that will return the values of each compartment
#def system(t,I):
 #   Dot_z_t=-a_1*I[0]+b_1*I[0]*I[1]-c_1*I[0]*I[2]
  #  Dot_s_t=a_2*I[1]-b_2*I[0]*I[1]-c_2*I[1]*I[2]+d_2*I[2]
   # Dot_i_t=a_2*I[2]-b_2*I[0]*I[2]+c_2*I[1]*I[2]-d_2*I[2]
    #return [Dot_z_t,Dot_s_t,Dot_i_t]

def system(t,I):
    Dot_z_t=-a_1*I[0]+b_1*I[0]*I[1]
    Dot_s_t=a_2*I[1]-b_2*I[0]*I[1]
    return [Dot_z_t,Dot_s_t]
```

```python
#Solution = solve_ivp(system,(0,20),[z_t,s_t,i_t],max_step = 1)
Solution = solve_ivp(system,(0,20),[z_t,s_t],max_step = 1)
#Solution
```

```python
#Plotting on the same graph the solution curves of the system.
time = Solution.t   #From our definition above.
Values = Solution.y  #The array of our solution values.
z_t = Values[0]; s_t = Values[1]; #i_t = Values[2];
#plt.plot(time,z_t,"red",label="predator")
#plt.plot(time,s_t,"Blue",label="health prey")
#plt.plot(time,i_t,"Orange",label="infected prey")
#xmin, xmax, ymin, ymax = axis([0, 20, 0, 2])
#ax.set(xlim=(0, 20), ylim=(0, 2))
plt.plot(time,s_t,z_t)
plt.title("Our solution Lotka-Volterra system.")
plt.xlabel("Time Interval")
plt.ylabel("$prey-predator$ Values")
plt.legend()
plt.ylim(0.,2)
```

```python
#Stating the initial conditions given.
z_t=0.9
s_t=1.9
i_t=0.8
#Stating the parameter values given
a_1=1;b_1=1;c_1=1;a_2=1;b_2=1;c_2=0.5;d_2=1

#Definig a function of the system given that will return the values of each compartment
def system(t,I):
    Dot_z_t=-a_1*I[0]+b_1*I[0]*I[1]-c_1*I[0]*I[2]
    Dot_s_t=a_2*I[1]-b_2*I[0]*I[1]-c_2*I[1]*I[2]+d_2*I[2]
    Dot_i_t=a_2*I[2]-b_2*I[0]*I[2]+c_2*I[1]*I[2]-d_2*I[2]
    return [Dot_z_t,Dot_s_t,Dot_i_t]

```

```python
Solution = solve_ivp(system,(0,20),[z_t,s_t,i_t],max_step = 1)
#Solution = solve_ivp(system,(0,20),[z_t,s_t],max_step = 1)
#Solution
```

```python
#Plotting on the same graph the solution curves of the system.
time = Solution.t   #From our definition above.
Values = Solution.y  #The array of our solution values.
z_t = Values[0]; s_t = Values[1]; i_t = Values[2];
plt.plot(time,z_t,"red",label="predator")
plt.plot(time,s_t,"Blue",label="health prey")
#plt.plot(time,i_t,"green",label="infected prey")
#plt.plot(time,s_t,z_t)
#plt.plot(time,z_t,label="Lokta")
plt.plot(time,s_t+i_t,label="Lokta")
plt.title("Our solution curves for the system.")
plt.xlabel("Time Interval")
plt.ylabel("$z_t,s_t,i_t$ Values")
plt.ylim(0.9,2)
plt.legend()
```

```python
#Plotting on the same graph the solution curves of the system.
time = Solution.t   #From our definition above.
Values = Solution.y  #The array of our solution values.
z_t = Values[0]; s_t = Values[1]; i_t = Values[2];
plt.plot(time,z_t,"red",label="predator")
plt.plot(time,s_t,"Blue",label="health prey")
plt.plot(time,i_t,"green",label="infected prey")
#plt.plot(time,s_t,z_t)
#plt.plot(time,z_t+i_t,label="Lokta")
#plt.plot(time,s_t,label="Lokta")
plt.title("Our solution curves for the system.")
plt.xlabel("Time Interval")
plt.ylabel("$z_t,s_t,i_t$ Values")
plt.ylim(0,2)
plt.legend()
```

```python
plt.plot(z_t,(s_t+i_t))

```

```python
plt.plot(s_t,z_t)
```

```python
plt.plot(time,z_t)
```

```python
plt.plot(time,s_t)
```

```python
plt.plot(time,s_t,z_t)
```

```python

```

Winnie Jepkonga Kulei ME Presentation..ipynb


```python
from sympy import *
init_printing()
x = Symbol('x')
```

```python
f0 = 2 * (x - 2) ** 2 + 5*(x -2) + 1
```

```python
solve(f0, x)
```

```python

```


```python
# Spacetime algebra. [1, -1, -1, -1] signature

# Make SymPy available to this program:
import sympy
from sympy import *

# Make GAlgebra available to this program:
from galgebra.ga import *
from galgebra.mv import *
from galgebra.printer import Fmt, GaPrinter, Format
    # Fmt:       sets the way that a multivector's basis expansion is output.
    # GaPrinter: makes GA output a little more readable.
    # Format:    turns on latex printer.
from galgebra.gprinter import gFormat, gprint
gFormat()
```

```python
txyz = (t, x, y, z) = symbols('t x y z', real=True)
stcoords = (t,x,y,z) = symbols('t x y z')
st = Ga('\mathbf{e}', g=[1, -1, -1, -1], coords=stcoords)
(et, ex, ey, ez) = st.mv()
```

```python
st.mv('X', 'vector')
```

```python

```


# Jupyter notebook example

## Simple plots

Loading the necessary modules (maybe _numpy_ is superceeded by _scipy_)

```python
import numpy as npy
import scipy as scy
# import sympy as spy
# import timeit
```

The sine function, $\sin(t)$, and its Fourier-transformation.
$$f_{max}=\frac1{2\Delta t},\quad \Delta f=\frac1{t_{max}}$$

```python
# tmax, t = 200/4, scy.linspace(0,tmax,201)
t = [ i/4 for i in range(201) ]
y = [ npy.sin(t[i]/10*2*npy.pi) for i in range(201) ]
Y = scy.fft(y)
Y = abs(Y[:101])*2/200
f = scy.linspace(0,1/t[1]/2,101)
```

### Plotting the graphs

```python
%matplotlib inline
import matplotlib.pyplot as plt
```

```python
plt.figure(1,figsize=(10,4))
plt.subplot(121)
plt.plot(t,y,'g.')
plt.title('$\sin(t)$')
plt.xlabel('$t$[s]')
plt.grid('on')

plt.subplot(122)
plt.plot(f,Y,'b')
plt.title('FFT')
plt.xlabel(r'$\nu$[Hz]')
plt.grid('on')

plt.show()
```

```html

```

## Fázissík módszer
### Elsőrendű differenciálegyenletek
$$\begin{array}{rcl}
\dot x &=& (a^2x^2+y^2-1)y\\
\dot y &=& (x^2+a^2y^2-1)x
\end{array}
$$
### Nullvonalak és fázisportré

```python
# %matplotlib inline
from pylab import *
```

```python
a = 2
x, y = meshgrid(arange(-1.5, 1.5, 0.1), arange(-1.5, 1.5, 0.1))
fxy = -((a*x)**2+y**2-1)*y
gxy = (x**2+(a*y)**2-1)*x
streamplot(x, y, fxy, gxy)
# f(x,y)=0 görbék: ellipszis+egyenes
contour(x,y,fxy,1,colors="red")
# g(x,y)=0 görbék: ellipszis+egyenes
contour(x,y,gxy,1,colors="green")
grid(); show()
```

## Benchmarking

```python
import time
import timeit
```

```python
# benchmark from https://devblogs.nvidia.com/drop-in-acceleration-gnu-octave/
N = 8192
# def f(N):
A = scy.single(scy.rand(N,N))
B = scy.single(scy.rand(N,N))
#    return A * B
#tic = time.clock()
#print(tic)
#C = A * B
etime = timeit.timeit('A*B',globals=globals(),number=1)
#toc = time.clock()
#print(toc)
elapsedTime = etime #toc-tic
# elapsedTime = etime(clock(), start);
print("Elapsed time: {0:.2g} sec (".format(elapsedTime))
gFlops = 2*N*N*N/(elapsedTime * 1e+9)
# disp(gFlops);
print("{0:.2f} GFlops)\n".format(gFlops))
```

    Elapsed time: 0.64 sec (
    1713.92 GFlops)

```python

```


# 1. 2D Linear Diffusion

We consider the 1d linear Convection equation, under a constant velocity

$$
\partial_t u = \nu \nabla^2 u
$$

```python
# needed imports
from numpy import zeros, ones, linspace, zeros_like
from matplotlib.pyplot import plot, contourf, show, colorbar
%matplotlib inline
```

```python
# Initial condition
import numpy as np
u0 = lambda x,y: np.exp(-(x-.3)**2/.05**2)*np.exp(-(y-.3)**2/.05**2)

ts = linspace(0., 1., 401)
x,y = np.meshgrid(ts,ts)
u = u0(x,y)
```

```python
contourf(x,y, u); colorbar() ; show()
```

### Time scheme

$$\frac{u^{n+1}-u^n}{\Delta t} - \nu \nabla^2 u^{n+1} = 0 $$

$$ \left(I - \nu \Delta t \nabla^2 \right) u^{n+1} = u^n $$

### Weak formulation

$$
\langle v, u^{n+1} \rangle + \nu \Delta t ~ \langle \nabla v, \nabla u^{n+1} \rangle = \langle v, u^n \rangle
$$

expending $u^n$ over the fem basis, we get the linear system

$$A U^{n+1} = M U^n$$

where

$$
M_{ij} = \langle b_i, b_j \rangle
$$

$$
A_{ij} = \langle b_i, b_j \rangle + \nu \Delta t ~ \langle \nabla b_i,  \nabla b_j \rangle
$$

## Abstract Model using SymPDE

```python
from sympde.core import Constant
from sympde.expr import BilinearForm, LinearForm, integral
from sympde.topology import ScalarFunctionSpace, Square, element_of, grad, dot
```

```python
# ... abstract model
domain = Square()

V = ScalarFunctionSpace('V', domain)

x,y = domain.coordinates

u,v = [element_of(V, name=i) for i in ['u', 'v']]

nu = Constant('nu')
dt = Constant('dt')

# bilinear form
expr = v*u + nu*dt* dot(grad(v), grad(u))
a = BilinearForm((u,v), integral(domain , expr))

# bilinear form for the mass matrix
expr = u*v
m  = BilinearForm((u,v), integral(domain , expr))

# linear form for initial condition
from sympy import exp
expr = exp(-(x-.3)**2/.05**2)*exp(-(y-.3)**2/.05**2)*v
l = LinearForm(v, integral(domain, expr))
```

## Discretization using Psydac

```python
from psydac.api.discretization import discretize
```

```python
nu = 0.3   # viscosity
T = 0.02  # T final time
dt = 0.001
niter = int(T / dt)

degree = [3,3]    # spline degree
ncells = [64,64]   # number of elements
```

```python
# Create computational domain from topological domain
domain_h = discretize(domain, ncells=ncells, comm=None)

# Discrete spaces
Vh = discretize(V, domain_h, degree=degree)

# Discretize the bilinear forms
ah = discretize(a, domain_h, [Vh, Vh])
mh = discretize(m, domain_h, [Vh, Vh])

# Discretize the linear form for the initial condition
lh = discretize(l, domain_h, Vh)
```

```python
# assemble matrices and convert them to scipy
M = mh.assemble().tosparse()
A = ah.assemble(nu=nu, dt=dt).tosparse()

# assemble the rhs and convert it to numpy array
rhs = lh.assemble().toarray()
```

```python
from scipy.sparse.linalg import cg, gmres
```

```python
# L2 projection of the initial condition
un, status = cg(M, rhs, tol=1.e-8, maxiter=5000)
```

```python
from simplines import plot_field_2d

nbasis = [W.nbasis for W in Vh.spaces]
plot_field_2d(Vh.knots, Vh.degree, un.reshape(nbasis)) ; colorbar() ; show()
```

```python
for i in range(0, niter):
    b = M.dot(un)
    un, status = gmres(A, b, tol=1.e-8, maxiter=5000)
```

```python
nbasis = [W.nbasis for W in Vh.spaces]
plot_field_2d(Vh.knots, Vh.degree, un.reshape(nbasis)) ; colorbar() ; show()
```

```python

```

```python

```


# In-class exercise 9: Deep Learning (Part 1B)
In this notebook we will gain some hands-on experience with backpropagation

```python
import numpy as np
import matplotlib.pyplot as plt
import time

%matplotlib inline

from sklearn.datasets import load_digits
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import label_binarize
```

```python
X, y = load_digits(return_X_y=True)
# Convert labels into one-hot format
Y = label_binarize(y, classes=np.unique(y))
K = Y.shape[1]  # number of classes
D = X.shape[1]  # number of features

np.random.seed(123)
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.3)
```

```python
# Dataset consists of 1797 samples. Each sample is represented with a 64-dimensional vector and belongs to one of 10 classes
X.shape, Y.shape
```

    ((1797, 64), (1797, 10))

# 1. Simple backpropagation example

Addition of two vectors

```python
class Add:
    def forward(self, x, y):
        return x + y

    def backward(self, d_out):
        d_x = d_out
        d_y = d_out
        return d_x, d_y
```

Elementwise multiplication of two vectors

```python
class Multiply:
    def forward(self, x, y):
        self.cache = (x, y)
        return x * y

    def backward(self, d_out):
        x, y = self.cache
        d_x = d_out * y
        d_y = d_out * x
        return d_x, d_y
```

Sum of a vector

```python
class Sum:
    def forward(self, x):
        self.cache = x
        return np.sum(x)

    def backward(self, d_out):
        x = self.cache
        d_x = d_out * np.ones_like(x)
        return d_x
```

### Dot product of two vectors as composition of multiplication and summation

```python
x = np.arange(1, 5, dtype=np.float32)
y = np.arange(-1, 3, dtype=np.float32)
```

```python
mult = Multiply()
vec_sum = Sum()

w = mult.forward(x, y)
z = vec_sum.forward(w)

d_w = vec_sum.backward(1.0)  #因为是导数的最开始，所以没有dout了，乘以1就是自己
d_x, d_y = mult.backward(d_w)
```

```python
w, z, d_x, d_y
```

    (array([-1.,  0.,  3.,  8.], dtype=float32),
     10.0,
     array([-1.,  0.,  1.,  2.], dtype=float32),
     array([1., 2., 3., 4.], dtype=float32))

### Dot product of two vectors as one operation

Dot product of two vectors

```python
class DotProduct:
    def forward(self, x, y):
        self.cache = x, y
        return np.dot(x, y)

    def backward(self, d_out):
        x, y = self.cache
        d_x = d_out * y
        d_y = d_out * x
        return d_x, d_y
```

```python
x = np.arange(1, 5, dtype=np.float32)
y = np.arange(-1, 3, dtype=np.float32)
```

```python
dp = DotProduct()
z = dp.forward(x, y)
d_x, d_y = dp.backward(1.0)
```

```python
z, d_x, d_y
```

    (10.0,
     array([-1.,  0.,  1.,  2.], dtype=float32),
     array([1., 2., 3., 4.], dtype=float32))

**Lessons:**
1. By implementing `forward` and `backward` method we can compute gradients of an arbitrary composition of functions
2. Use `cache` to store values that will be needed in the backward pass
3. Multiple operations can be combined into a single module (i.e. function)
4. Pass `1.0` as `d_out` for the terminal node in our computational graph

# 2. Multi-class logistic regression (without backprop)

Multi-class logistic regression model

Data:
* Data matrix $\mathbf{X} \in \mathbb{R}^{N \times D}$.
* Target labels in one-hot format $\mathbf{Y} \in \mathbb{R}^{N \times K}$.
$Y_{nk} = 1$ if sample $n$ belongs to class $k$, $Y_{nk} = 0$ otherwise.

Model parameters:
* Weight matrix $\mathbf{W} \in \mathbb{R}^{D \times K}$.
* Bias vector $\mathbf{b} \in \mathbb{R}^{K}$.

Making predictions with the model:
* Logits
$$\mathbf{a}_n = \mathbf{W x}_n + \mathbf{b}$$
* Denote the matrix of logits as
$$\mathbf{A} = \mathbf{XW} +  \mathbf{1}_N \mathbf{b}^T \in \mathbb{R}^{N \times K}$$
* Convert logits to probabilities using softmax function
$$p(Y_{nk} = 1 \mid \mathbf{x}_n, \mathbf{W}, \mathbf{b}) = \frac{\exp(A_{nk})}{\sum_{c = 1}^{K} \exp(A_{nc})}$$

Negative log-likelihood

\begin{align}
-\log p(\mathbf{Y} \mid \mathbf{X}, \mathbf{W}, \mathbf{b}) &= - \frac{1}{N}\sum_{n=1}^{N} \sum_{k=1}^{K} Y_{nk} \log p(Y_{nk} = 1 \mid \mathbf{x}_n, \mathbf{W}, \mathbf{b})\\
&= \frac{1}{N} \sum_{n=1}^{N} \sum_{k=1}^{K} Y_{nk} \left(-A_{nk} + \log \left( \sum_{c=1}^{C} \exp(A_{nc}) \right) \right)\\
%&= \frac{1}{N} \sum_{n=1}^{N} \left(\sum_{k=1}^{K} -Y_{nk} A_{nk} \right) + \log \left( \sum_{c=1}^{C} \exp(A_{nc}) \right)
\end{align}

```python
from scipy.special import softmax
```

```python
def predict(X, W, b):
    """Generate predictions for a multi-class logistic regression model.

    Args:
        X: data matrix, shape (N, D)
        W: weight matrix, shape (D, K)
        b: bias vector, shape (K)

    Returns:
        Y_pred: Predicted class probabilities, shape (N, K).
            Y_pred[n, k] = probability that sample n belongs to class k.
    """
    return softmax(X @ W + b, axis=1)
```

Negative log-likelihood of multiclass logistic regression

```python
def nll_loss(X, W, b, Y):
    """Compute negative log-likelihood of a logistic regression model.

    Also known as categorical cross entropy loss.

    Args:
        X: data matrix, shape (N, D)
        W: weight matrix, shape (D, K)
        b: bias vector, shape (K)
        Y: true labels in one-hot format, shape (N, K)

    Returns:
        loss: loss of the logistic regression model, shape ()
    """
    N = X.shape[0]
    logits = X @ W + b

    logits_shifted = logits - logits.max(axis=1, keepdims=True)
    log_sum_exp = np.log(np.sum(np.exp(logits_shifted), axis=1, keepdims=True))
    log_probs = logits_shifted - log_sum_exp
    loss = -np.sum(Y * log_probs) / N
    return loss
```

```python
def nll_grad(X, W, b, Y):
    """Compute gradient of the NLL loss w.r.t. W and b.

    Args:
        X: data matrix, shape (N, D)
        W: weight matrix, shape (D, K)
        b: bias vector, shape (K)
        Y: true labels in one-hot format, shape (N, K)

    Returns:
        d_W: gradient of the los w.r.t. W, shape (D, K)
        d_b: gradient of the los w.r.t. b, shape (K)
    """
    N = X.shape[0]
    probas = softmax(X @ W + b, axis=1) - Y
    d_W = X.T @ probas / N
    d_b = probas.sum(axis=0) / N
    return d_W, d_b
```

```python
# Initialize learnable model parameters
W = np.zeros([D, K])
b = np.zeros([K])
```

```python
# Specify optimization parameters
learning_rate = 1e-2
max_epochs = 301
report_frequency = 25
```

```python
for epoch in range(max_epochs):
    # Compute train loss
    loss = nll_loss(X_train, W, b, Y_train)

    if epoch % report_frequency == 0:
        print(f'Epoch {epoch:4d}, loss = {loss:.4f}')

    # Perform the update
    d_W, d_b = nll_grad(X_train, W, b, Y_train)

    W = W - learning_rate * d_W
    b = b - learning_rate * d_b
```

    Epoch    0, loss = 2.3026
    Epoch   25, loss = 0.3411
    Epoch   50, loss = 0.2293
    Epoch   75, loss = 0.1862
    Epoch  100, loss = 0.1610
    Epoch  125, loss = 0.1440
    Epoch  150, loss = 0.1314
    Epoch  175, loss = 0.1215
    Epoch  200, loss = 0.1136
    Epoch  225, loss = 0.1069
    Epoch  250, loss = 0.1013
    Epoch  275, loss = 0.0964
    Epoch  300, loss = 0.0921

```python
loss_test = nll_loss(X_test, W, b, Y_test)
y_pred_test = predict(X_test, W, b).argmax(axis=1)
y_test = Y_test.argmax(axis=1)
acc_test = accuracy_score(y_test, y_pred_test)
print(f'loss_test = {loss_test:.4f}, accuracy_test = {acc_test:.3f}')
```

    loss_test = 0.1511, accuracy_test = 0.956

# 3. Multi-class logistic regression (with backprop)

```python
import nn
```

```python
class LogisticRegression:
    """Logistic regression model.

    Gradients are computed with backpropagation.
    """
    def __init__(self, num_features, num_classes, learning_rate=1e-2):
        self.learning_rate = learning_rate

        # Initialize the model parameters
        self.params = {
            'W': np.zeros([num_features, num_classes]),
            'b': np.zeros([num_classes])
        }

        # Define layers
        self.affine = nn.Affine()
        self.cross_entropy = nn.CategoricalCrossEntropy()

    def predict(self, X):
        """Generate predictions for one minibatch.

        Args:
            X: data matrix, shape (N, D)

        Returns:
            Y_pred: predicted class probabilities, shape (N, D)
            Y_pred[n, k] = probability that sample n belongs to class k
        """
        logits = self.affine.forward(X,
                                     self.params['W'],
                                     self.params['b'])
        return softmax(logits, axis=1)

    def step(self, X, Y):
        """Perform one step of gradient descent on the minibatch of data."""
        # Forward pass - compute the loss on training data
        logits = self.affine.forward(X,
                                     self.params['W'],
                                     self.params['b'])
        loss = self.cross_entropy.forward(logits, Y)

        # Backward pass - compute the gradients of loss w.r.t. all the model parameters
        grads = {}
        d_logits, _ = self.cross_entropy.backward()
        _, grads['W'], grads['b'] = self.affine.backward(d_logits)

        # Apply the gradients
        for p in self.params:
            self.params[p] = self.params[p] - self.learning_rate * grads[p]
        return loss
```

```python
# Specify optimization parameters
learning_rate = 1e-2
max_epochs = 301
report_frequency = 25
```

```python
log_reg = LogisticRegression(num_features=D, num_classes=K, learning_rate=learning_rate)
```

```python
for epoch in range(max_epochs):
    loss = log_reg.step(X_train, Y_train)

    if epoch % report_frequency == 0:
        print(f'Epoch {epoch:4d}, loss = {loss:.4f}')
```

    Epoch    0, loss = 2.3026
    Epoch   25, loss = 0.3411
    Epoch   50, loss = 0.2293
    Epoch   75, loss = 0.1862
    Epoch  100, loss = 0.1610
    Epoch  125, loss = 0.1440
    Epoch  150, loss = 0.1314
    Epoch  175, loss = 0.1215
    Epoch  200, loss = 0.1136
    Epoch  225, loss = 0.1069
    Epoch  250, loss = 0.1013
    Epoch  275, loss = 0.0964
    Epoch  300, loss = 0.0921

```python
y_pred_test = log_reg.predict(X_test).argmax(axis=1)
loss_test = log_reg.step(X_test, Y_test)
y_test = Y_test.argmax(axis=1)
acc_test = accuracy_score(y_test, y_pred_test)
print(f'loss_test = {loss_test:.4f}, accuracy_test = {acc_test:.3f}')
```

    loss_test = 0.1511, accuracy_test = 0.956


```python

```

```python
import sympy as sm
i,j,k,l,m,n = sm.symbols('i j k l m n',cls=sm.Idx)
# Minor matrix
M1 = sm.MatrixSymbol('A',m,n)
M2 = sm.MatrixSymbol('B',m,n)
T = sm.Matrix([[1,2,3],[2,2,5],[3,3,2]])
T.det()

A = sm.Matrix(M1.subs([(m,3),(n,3)]))
B = sm.Matrix(M2.subs([(m,2),(n,2)]))

# minor matrix
A.minorMatrix(0,0)
B.minorMatrix(0,0)
A.minor(0,0)
B.minor(0,0)

# cofactor matrix
B.cofactorMatrix()
A.cofactorMatrix()

# aejoin matrix
A.adjugate()
B.adjugate()

# determinat
A.det()
B.det()

# inverse
(1/A.det()*(A*A.cofactorMatrix().T))
(1/A.det())*(A.adjugate())
```

$\displaystyle \left[\begin{matrix}\frac{A_{1, 1} A_{2, 2} - A_{1, 2} A_{2, 1}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{- A_{0, 1} A_{2, 2} + A_{0, 2} A_{2, 1}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{A_{0, 1} A_{1, 2} - A_{0, 2} A_{1, 1}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}}\\\frac{- A_{1, 0} A_{2, 2} + A_{1, 2} A_{2, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{A_{0, 0} A_{2, 2} - A_{0, 2} A_{2, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{- A_{0, 0} A_{1, 2} + A_{0, 2} A_{1, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}}\\\frac{A_{1, 0} A_{2, 1} - A_{1, 1} A_{2, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{- A_{0, 0} A_{2, 1} + A_{0, 1} A_{2, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}} & \frac{A_{0, 0} A_{1, 1} - A_{0, 1} A_{1, 0}}{A_{0, 0} A_{1, 1} A_{2, 2} - A_{0, 0} A_{1, 2} A_{2, 1} - A_{0, 1} A_{1, 0} A_{2, 2} + A_{0, 1} A_{1, 2} A_{2, 0} + A_{0, 2} A_{1, 0} A_{2, 1} - A_{0, 2} A_{1, 1} A_{2, 0}}\end{matrix}\right]$

```python
A.adjugate()
```

$\displaystyle \left[\begin{matrix}A_{1, 1} A_{2, 2} - A_{1, 2} A_{2, 1} & - A_{0, 1} A_{2, 2} + A_{0, 2} A_{2, 1} & A_{0, 1} A_{1, 2} - A_{0, 2} A_{1, 1}\\- A_{1, 0} A_{2, 2} + A_{1, 2} A_{2, 0} & A_{0, 0} A_{2, 2} - A_{0, 2} A_{2, 0} & - A_{0, 0} A_{1, 2} + A_{0, 2} A_{1, 0}\\A_{1, 0} A_{2, 1} - A_{1, 1} A_{2, 0} & - A_{0, 0} A_{2, 1} + A_{0, 1} A_{2, 0} & A_{0, 0} A_{1, 1} - A_{0, 1} A_{1, 0}\end{matrix}\right]$

```python
sm.Sum(A[i,j]*A[j,k],(j,0,2))
A.cofactor(0,0)
```

$\displaystyle A_{1, 1} A_{2, 2} - A_{1, 2} A_{2, 1}$

```python
# minor (소행열 식)
T.minor(0,1)
```

$\displaystyle -11$

```python
# cofactor (여인수)
T.cofactor(0,1)
```

$\displaystyle 11$

```python
# cofactor matrix(여인수 행열)
T.cofactorMatrix()/T.det()
```

$\displaystyle \left[\begin{matrix}-1 & 1 & 0\\\frac{5}{11} & - \frac{7}{11} & \frac{3}{11}\\\frac{4}{11} & \frac{1}{11} & - \frac{2}{11}\end{matrix}\right]$

# inverse
> ### $
AA^{-1} = I, \quad A^{-1} = B \\
A\vec{x} = \vec{k}
\\ \Rightarrow (A^{-1}A)\vec{x} = A^{-1}\vec{k}
\\ \therefore \vec{x} = A^{-1}\vec{k}
$
> ### 1차원 : 길이로 나눈다
>> ### $ A = \Big[ a_{11} \Big], x = [x_{11}], k = [k_{11}]
\\ \quad \because A^{-1} = (a_{11})^{-1} = frac{1}{a_{11}} \\
\mapsto Ax = b \to x_{11} = \frac{k_{11}}{a_{11}}$
> ### 2차원 : ()으로 나눈다
>> ### $
A = \left[ \begin{array}{}
a_{11} & a_{12} \\ a_{21} & a_{22}
\end{array}\right]
= \left[ \begin{array}{}
a & b \\ c &  d
\end{array}\right]
\\ A\vec{x} = \vec{k} \Rightarrow
\left[ \begin{array}{}
a & b \\ c & d
\end{array}\right]
\left[ \begin{array}{}
x \\ y
\end{array}\right]
=
\left[ \begin{array}{}
u \\ v
\end{array}\right]
$

```python
# adjoin marix(수반행열) = cofactor matrix.transpose()
# ad - bc
T.adjugate()
a+bi c+di
[ab] [cd] [real imag]
ad - bc
```

$\displaystyle \left[\begin{matrix}-11 & 5 & 4\\11 & -7 & 1\\0 & 3 & -2\end{matrix}\right]$

```python
T.adjugate()/T.det()
```

$\displaystyle \left[\begin{matrix}-1 & \frac{5}{11} & \frac{4}{11}\\1 & - \frac{7}{11} & \frac{1}{11}\\0 & \frac{3}{11} & - \frac{2}{11}\end{matrix}\right]$

```python
T.det()
```

$\displaystyle 0$

```python

```


# Quadratic Programs

## Introduction

In this tutorial, we briefly introduce how to build optimization problems using Qiskit's optimization module.
Qiskit introduces the `QuadraticProgram` class to make a model of an optimization problem.
More precisely, it deals with quadratically constrained quadratic programs given as follows:

$$
\begin{align}
\text{minimize}\quad& x^\top Q_0 x + c^\top x\\
\text{subject to}\quad& A x \leq b\\
& x^\top Q_i x + a_i^\top x \leq r_i, \quad 1,\dots,i,\dots,q\\
& l_i \leq x_i \leq u_i, \quad 1,\dots,i,\dots,n,
\end{align}
$$

where the $Q_i$ are $n \times n$ matrices, $A$ is a $m \times n$ matrix , $x$, and $c$ are $n$-dimensional vectors, $b$ is an $m$-dimensional vector, and where $x$ can defined as binary, integer, or continuous variables.
In addition to "$\leq$" constraints 'QuadraticProgram' also supports "$\geq$" and "$=$".

## Loading a `Quadratic Program` from an LP file

As setup, you need to import the following module.

```python
from qiskit_optimization import QuadraticProgram
from qiskit_optimization.translators import from_docplex_mp
```

You start with an empty model. How to add variables and constraints to a model is explained in the section "Directly constructing a `QuadraticProgram`".

Qiskit's optimization module supports the conversion from Docplex model. You can easily make a model of an optimization problem with Docplex.
You can find the documentation of Docplex at https://ibmdecisionoptimization.github.io/docplex-doc/mp/index.html

You can load a Docplex model to `QuadraticProgram` by using `from_docplex_mp` function.

## Loading a `QuadraticProgram` from a docplex model

```python
# Make a Docplex model
from docplex.mp.model import Model

mdl = Model('docplex model')
x = mdl.binary_var('x')
y = mdl.integer_var(lb=-1, ub=5, name='y')
mdl.minimize(x + 2 * y)
mdl.add_constraint(x - y == 3)
mdl.add_constraint((x + y) * (x - y) <= 1)
print(mdl.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: docplex model

    Minimize
     obj: x + 2 y
    Subject To
     c1: x - y = 3
     qc1: [ x^2 - y^2 ] <= 1

    Bounds
     0 <= x <= 1
     -1 <= y <= 5

    Binaries
     x

    Generals
     y
    End

```python
# load from a Docplex model
mod = from_docplex_mp(mdl)
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: docplex model

    Minimize
     obj: x + 2 y
    Subject To
     c0: x - y = 3
     q0: [ x^2 - y^2 ] <= 1

    Bounds
     0 <= x <= 1
     -1 <= y <= 5

    Binaries
     x

    Generals
     y
    End

## Directly constructing a `QuadraticProgram`

We then explain how to make model of an optimization problem directly using `QuadraticProgram`.
Let's start from an empty model.

```python
# make an empty problem
mod = QuadraticProgram('my problem')
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj:
    Subject To

    Bounds
    End

The `QuadraticProgram` supports three types of variables:
- Binary variable
- Integer variable
- Continuous variable

When you add variables, you can specify names, types, lower bounds and upper bounds.

When you display your problem as LP format,
`Binaries` denotes binary variables and `Generals` denotes integer variables.
If variables are not included in either `Binaries` or `Generals`, such variables are continuous ones with default lower bound = 0 and upper bound = infinity.
Note that you cannot use 'e' or 'E' as the first character of names due to the [specification of LP format](https://www.ibm.com/support/knowledgecenter/SSSA5P_12.7.1/ilog.odms.cplex.help/CPLEX/FileFormats/topics/LP_VariableNames.html).

```python
# Add variables
mod.binary_var(name='x')
mod.integer_var(name='y', lowerbound=-1, upperbound=5)
mod.continuous_var(name='z', lowerbound=-1, upperbound=5)
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj:
    Subject To

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

You can set the objective function by invoking `QuadraticProgram.minimize` or `QuadraticProgram.maximize`.
You can add a constant term as well as linear and quadratic objective function by specifying linear and quadratic terms with either list, matrix or dictionary.

Note that in the LP format the quadratic part has to be scaled by a factor $1/2$.
Thus, when printing as LP format, the quadratic part is first multiplied by 2 and then divided by 2 again.

For quadratic programs, there are 3 pieces that have to be specified: a constant (offset), a linear term ($c^{T}x$), and a quadratic term ($x^{T}Qx$).

The cell below shows how to declare an objective function using a dictionary. For the linear term, keys in the dictionary correspond to variable names, and the corresponding values are the coefficients. For the quadratic term, keys in the dictionary correspond to the two variables being multiplied, and the values are again the coefficients.

```python
# Add objective function using dictionaries
mod.minimize(constant=3, linear={'x': 1}, quadratic={('x', 'y'): 2, ('z', 'z'): -1})
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: x + [ 4 x*y - 2 z^2 ]/2 + 3
    Subject To

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

Another way to specify the quadratic program is using arrays. For the linear term, the array corresponds to the vector $c$ in the mathematical formulation. For the quadratic term, the array corresponds to the matrix $Q$. Note that the ordering of the variables ($x$ in the mathematical formulation) is the order in which the variables were originally declared in the QuadraticProgram object.

```python
# Add objective function using lists/arrays
mod.minimize(constant=3, linear=[1,0,0], quadratic=[[0,1,0],[1,0,0],[0,0,-1]])
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: x + [ 4 x*y - 2 z^2 ]/2 + 3
    Subject To

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

You can access the constant, the linear term, and the quadratic term by looking at `Quadratic.objective.{constant, linear, quadratic}`, respectively.
As for linear and quadratic terms, you can get a dense matrix (`to_array`), a sparse matrix (`coefficients`), and a dictionary (`to_dict`).
For dictionaries, you can specify whether to use variable indices or names as keys.
Note that the quadratic terms are stored in a compressed way, e.g., `{('x', 'y'): 1, ('y', 'x'): 2}` is stored as `{('x', 'y'): 3}`.
You can get the quadratic term as a symmetric matrix by calling `to_array(symmetric=True)` or `to_dict(symmetric=True)`.
If you call `to_dict(name=True)`, you can get a dictionary whose keys are pairs of variable names.

```python
print('constant:\t\t\t', mod.objective.constant)
print('linear dict:\t\t\t', mod.objective.linear.to_dict())
print('linear array:\t\t\t', mod.objective.linear.to_array())
print('linear array as sparse matrix:\n', mod.objective.linear.coefficients, '\n')
print('quadratic dict w/ index:\t', mod.objective.quadratic.to_dict())
print('quadratic dict w/ name:\t\t', mod.objective.quadratic.to_dict(use_name=True))
print('symmetric quadratic dict w/ name:\t', mod.objective.quadratic.to_dict(use_name=True, symmetric=True))
print('quadratic matrix:\n', mod.objective.quadratic.to_array(),'\n')
print('symmetric quadratic matrix:\n', mod.objective.quadratic.to_array(symmetric=True),'\n')
print('quadratic matrix as sparse matrix:\n', mod.objective.quadratic.coefficients)
```

    constant:			 3
    linear dict:			 {0: 1}
    linear array:			 [1 0 0]
    linear array as sparse matrix:
       (0, 0)	1

    quadratic dict w/ index:	 {(0, 1): 2, (2, 2): -1}
    quadratic dict w/ name:		 {('x', 'y'): 2, ('z', 'z'): -1}
    symmetric quadratic dict w/ name:	 {('y', 'x'): 1, ('x', 'y'): 1, ('z', 'z'): -1}
    quadratic matrix:
     [[ 0  2  0]
     [ 0  0  0]
     [ 0  0 -1]]

    symmetric quadratic matrix:
     [[ 0  1  0]
     [ 1  0  0]
     [ 0  0 -1]]

    quadratic matrix as sparse matrix:
       (0, 1)	2
      (2, 2)	-1

## Adding/removing linear and quadratic constraints

You can add linear constraints by setting name, linear expression, sense and right-hand-side value (rhs).
You can use senses 'EQ', 'LE', and 'GE' as Docplex supports.

```python
# Add linear constraints
mod.linear_constraint(linear={'x': 1, 'y': 2}, sense='==', rhs=3, name='lin_eq')
mod.linear_constraint(linear={'x': 1, 'y': 2}, sense='<=', rhs=3, name='lin_leq')
mod.linear_constraint(linear={'x': 1, 'y': 2}, sense='>=', rhs=3, name='lin_geq')
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: x + [ 4 x*y - 2 z^2 ]/2 + 3
    Subject To
     lin_eq: x + 2 y = 3
     lin_leq: x + 2 y <= 3
     lin_geq: x + 2 y >= 3

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

You can add quadratic constraints as well as objective function and linear constraints.

```python
# Add quadratic constraints
mod.quadratic_constraint(linear={'x': 1, 'y': 1}, quadratic={('x', 'x'): 1, ('y', 'z'): -1}, sense='==', rhs=1, name='quad_eq')
mod.quadratic_constraint(linear={'x': 1, 'y': 1}, quadratic={('x', 'x'): 1, ('y', 'z'): -1}, sense='<=', rhs=1, name='quad_leq')
mod.quadratic_constraint(linear={'x': 1, 'y': 1}, quadratic={('x', 'x'): 1, ('y', 'z'): -1}, sense='>=', rhs=1, name='quad_geq')
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: x + [ 4 x*y - 2 z^2 ]/2 + 3
    Subject To
     lin_eq: x + 2 y = 3
     lin_leq: x + 2 y <= 3
     lin_geq: x + 2 y >= 3
     quad_eq: [ x^2 - y*z ] + x + y = 1
     quad_leq: [ x^2 - y*z ] + x + y <= 1
     quad_geq: [ x^2 - y*z ] + x + y >= 1

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

You can access linear and quadratic terms of linear and quadratic constraints as in the same way as the objective function.

```python
lin_geq = mod.get_linear_constraint('lin_geq')
print('lin_geq:', lin_geq.linear.to_dict(use_name=True), lin_geq.sense, lin_geq.rhs)
quad_geq = mod.get_quadratic_constraint('quad_geq')
print('quad_geq:', quad_geq.linear.to_dict(use_name=True), quad_geq.quadratic.to_dict(use_name=True), quad_geq.sense, lin_geq.rhs)
```

    lin_geq: {'x': 1.0, 'y': 2.0} ConstraintSense.GE 3
    quad_geq: {'x': 1.0, 'y': 1.0} {('x', 'x'): 1.0, ('y', 'z'): -1.0} ConstraintSense.GE 3

You can also remove linear/quadratic constraints by `remove_linear_constraint` and `remove_quadratic_constraint`.

```python
# Remove constraints
mod.remove_linear_constraint('lin_eq')
mod.remove_quadratic_constraint('quad_leq')
print(mod.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: x + [ 4 x*y - 2 z^2 ]/2 + 3
    Subject To
     lin_leq: x + 2 y <= 3
     lin_geq: x + 2 y >= 3
     quad_eq: [ x^2 - y*z ] + x + y = 1
     quad_geq: [ x^2 - y*z ] + x + y >= 1

    Bounds
     0 <= x <= 1
     -1 <= y <= 5
     -1 <= z <= 5

    Binaries
     x

    Generals
     y
    End

You can substitute some of variables with constants or other variables.
More precisely, `QuadraticProgram` has a method `substitute_variables(constants=..., variables=...)` to deal with the following two cases.
- $x \leftarrow c$: when `constants` have a dictionary `{x: c}`.
- $x \leftarrow c y$: when `variables` have a dictionary `{x: (y, c)}`.

## Substituting Variables

```python
sub = mod.substitute_variables(constants={'x': 0}, variables={'y': ('z', -1)})
print(sub.export_as_lp_string())
```

    \ This file has been generated by DOcplex
    \ ENCODING=ISO-8859-1
    \Problem name: my problem

    Minimize
     obj: [ - 2 z^2 ]/2 + 3
    Subject To
     lin_leq: - 2 z <= 3
     lin_geq: - 2 z >= 3
     quad_eq: [ z^2 ] - z = 1
     quad_geq: [ z^2 ] - z >= 1

    Bounds
     -1 <= z <= 1
    End

If the resulting problem is infeasible due to lower bounds or upper bounds, the methods returns the status `Status.INFEASIBLE`.
We try to replace variable `x` with -1, but -1 is out of range of `x` (0 <= `x` <= 1). So, it returns `Status.INFEASIBLE`.

```python
sub = mod.substitute_variables(constants={'x': -1})
print(sub.status)
```

    Infeasible substitution for variable: x

    QuadraticProgramStatus.INFEASIBLE

You cannot substitute variables multiple times.
The method raises an error in such a case.

```python
from qiskit_optimization import QiskitOptimizationError
try:
    sub = mod.substitute_variables(constants={'x': -1}, variables={'y': ('x', 1)})
except QiskitOptimizationError as e:
    print('Error: {}'.format(e))
```

    Error: 'Cannot substitute by variable that gets substituted itself: y <- x 1'

```python
import qiskit.tools.jupyter
%qiskit_version_table
%qiskit_copyright
```

Version Information

Qiskit Software	Version

Qiskit	None

Terra	0.17.0.dev0+3c103ae

Aer	0.8.0

Ignis	0.6.0.dev0+6103f99

Aqua	0.9.0.dev0+0058183

IBM Q Provider	0.12.0.dev0+8f3168b

System information

Python	3.7.9 (default, Aug 31 2020, 07:22:35)
[Clang 10.0.0 ]

OS	Darwin

CPUs	2

Memory (Gb)	12.0

Fri Mar 05 12:02:48 2021 EST

This code is a part of Qiskit

© Copyright IBM 2017, 2021.

This code is licensed under the Apache License, Version 2.0. You may
obtain a copy of this license in the LICENSE.txt file in the root directory
 of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
Any modifications or derivative works of this code must retain this
copyright notice, and modified files need to carry a notice indicating
that they have been altered from the originals.

```python

```


# Vergleichsspannung

 $$\mathbf{u}(x_1,x_2,x_3)= \begin{pmatrix}u_1(x_1,x_2,x_3)\\u_2(x_1,x_2,x_3)\\u_3(x_1,x_2,x_3) \end{pmatrix}$$

 $$\mathbf{E} = \frac{1}{2}\left(\nabla \mathbf{u}+(\nabla \mathbf{u})^T\right)$$

 $$\mathbf{\Sigma}  =2\mu \mathbf{E}+\lambda \text{tr}(\mathbf{E})\mathbf{I} $$

S. 182 in [Issler et. al, 2003].

$$\mathbf{\Sigma}  =\begin{pmatrix}\sigma_{11}&&\sigma_{12}&&\sigma_{13}\\
                                   \sigma_{21}&&\sigma_{22}&&\sigma_{23}\\
                                   \sigma_{31}&&\sigma_{31}&&\sigma_{33}
                                 \end{pmatrix} $$

Wie bekommxt man an einer Stelle $\mathbf{x}\in\mathbb{R}^3$ einen Spannungswert, den man farbcodieren und mit anderen Spannungswerten (z.B. Streckgrenze) vergleichen kann? Z.B. durch die von Mises Vergleichsspannung:

$$\sigma_{v}=\frac{1}{\sqrt{2}}\sqrt{(\sigma_{11}-\sigma_{22})^2+(\sigma_{22}-\sigma_{33})^2+(\sigma_{33}-\sigma_{11})^2+6\cdot(\sigma_{12}^2+\sigma_{23}^2+\sigma_{31}^2)}$$

Sind die Hauptspannungen $\sigma_{I},\sigma_{II},\sigma_{III}$ bekannt, dann ist die von Mises Vergleichsspannung:

$$\sigma_{v}=\frac{1}{\sqrt{2}}\sqrt{(\sigma_{I}-\sigma_{II})^2+(\sigma_{II}-\sigma_{III})^2+(\sigma_{III}-\sigma_{I})^2}$$.

Die die Hauptspannungen $\sigma_{I},\sigma_{II},\sigma_{III}$ sind die Eigenwerte der Spannungsmatrix.

Im Folgenden ein Beispiel. Auf der Seite [sympy live](https://live.sympy.org/) können Sie die Eigenwerte von Matrizen symbolsich berechnen.

```python
from sympy import *
import numpy as np
from numpy import sqrt
from matplotlib import pylab as plt
import plotly.express as px
import plotly.graph_objects as go
```

```python
Sigma=Matrix(([20,0,0],[0,30,40],[0,40,-30]))
Sigma
```

$\displaystyle \left[\begin{matrix}20 & 0 & 0\\0 & 30 & 40\\0 & 40 & -30\end{matrix}\right]$

```python
Sigma.eigenvals()
```

    {20: 1, -50: 1, 50: 1}

```python
sigmaI=20
sigmaII=-50
sigmaIII=50
```

```python
summe=(sigmaI-sigmaII)**2+(sigmaII-sigmaIII)**2+(sigmaIII-sigmaI)**2
sigma_v=(1/np.sqrt(2.))*np.sqrt(summe)
```

```python
sigma_v
```

    88.88194417315589

```python
Spannung=Matrix(([-30,0,-60  ],[0,30,-60],[-60,-60,0]))
```

```python
Spannung
```

$\displaystyle \left[\begin{matrix}-30 & 0 & -60\\0 & 30 & -60\\-60 & -60 & 0\end{matrix}\right]$

```python
Spannung.eigenvals()
```

    {-90: 1, 90: 1, 0: 1}

```python
sigmaI=-90
sigmaII=0
sigmaIII=90
```

```python
summe=(sigmaI-sigmaII)**2+(sigmaII-sigmaIII)**2+(sigmaIII-sigmaI)**2
sigma_v=(1/np.sqrt(2.))*np.sqrt(summe)
sigma_v
```

    155.88457268119893

```python
Spannung2=Matrix(([20,40,-20  ],[40,20,-20],[-20,-20,-10]))
Spannung2
```

$\displaystyle \left[\begin{matrix}20 & 40 & -20\\40 & 20 & -20\\-20 & -20 & -10\end{matrix}\right]$

```python
Spannung2.eigenvals()
```

    {70: 1, -20: 2}

```python
sigmaI=70
sigmaII=-20
sigmaIII=-20

summe=(sigmaI-sigmaII)**2+(sigmaII-sigmaIII)**2+(sigmaIII-sigmaI)**2
sigma_v=(1/np.sqrt(2.))*np.sqrt(summe)
sigma_v
```

    90.0

```python
Spannung3=Matrix(([2,5,3  ],[5,1,4],[3,4,3]))

Spannung3
```

$\displaystyle \left[\begin{matrix}2 & 5 & 3\\5 & 1 & 4\\3 & 4 & 3\end{matrix}\right]$

```python
Spannung3.eigenvals()
```

    {10: 1, -2 - sqrt(3): 1, -2 + sqrt(3): 1}

```python
list(Spannung3.eigenvals())
```

    [10, -2 - sqrt(3), -2 + sqrt(3)]

```python
Spannung3.charpoly()
```

$\displaystyle \operatorname{PurePoly}{\left( \lambda^{3} - 6 \lambda^{2} - 39 \lambda - 10, \lambda, domain=\mathbb{Z} \right)}$

```python
sigmaI=10
sigmaII=-2 - np.sqrt(3)
sigmaIII= -2 + np.sqrt(3)

summe=(sigmaI-sigmaII)**2+(sigmaII-sigmaIII)**2+(sigmaIII-sigmaI)**2
sigma_v=(1/np.sqrt(2.))*np.sqrt(summe)
sigma_v
```

    12.369316876852979

```python
lamda = symbols('lamda')
```

```python
Spannung3.charpoly(lamda).as_expr()
```

$\displaystyle \lambda^{3} - 6 \lambda^{2} - 39 \lambda - 10$

```python
poly = lambdify(lamda, Spannung3.charpoly(lamda).as_expr())
```

```python
#plt.plot(np.linspace(-15, 15), poly(np.linspace(-15, 15)))
fig=px.line(x=np.linspace(-15, 15),y=poly(np.linspace(-15, 15)))
fig.add_trace(go.Scatter(x=np.asfarray(list(Spannung3.eigenvals())),
                         y=poly(np.asfarray(list(Spannung3.eigenvals()))),
                         mode='markers',))
fig.show()
```

```python
poly(10)
```

```python
factor(Spannung3.charpoly(lamda).as_expr())
```

```python
np.asfarray(list(Spannung3.eigenvals()))
```

```python
poly(np.asfarray(list(Spannung3.eigenvals())))
```

```python
poly(1)
```

```python

```


## Lecture topic 5:

## Ordinary and partial differential equations

## Part 2

```python
from lecture_utils import *
```

### Topics of Part 2

1. Continuation with integrators
    - Leapfrog
    - Verlet

2. Partial differential equations

### Repetition: Leapfrog method

Scheme comparing RK2 and leapfrog (Figure adapted from "Computational Physics" by Marc Newman)

- RK2:
$$
 \begin {align}
 x\left(t+\frac{1}{2}h\right) &= x(t) + \frac{1}{2}hf(x(t),t)\\
   x(t+h) &= x(t) + hf\left(x\left(t+\frac{1}{2}h\right),t+\frac{1}{2}h\right)
 \end{align}
$$

- Leapfrog

\begin{align}
   x\left(t+h\right) &= x(t) + hf\left(x\left(t+\frac{1}{2}h\right),t+\frac{1}{2}h\right)\\
   x\left(t+\frac{3}{2}h\right) &= x\left(t+\frac{1}{2}h\right) + hf(x(t+h),t+h)\\
\end{align}

#### When/why would one use Leapfrog instead of RK?

- RK4 more accurate, but not time-reversal symmetric
- time-reversal symmetric behavior important for energy conservation
- energy conservation important for many problems in physics, for example:
  - nonlinear pendulum
  - planet orbiting a star
  - molecular dynamics (computer simulation of movement of atoms and molecules)

### Time reversal and energy conservation

- forward and backward solution should be identical
- forward means we have a positive interval $h$
- backwards means we have a negative interval $-h$

### Forward and backward calculation with Leapfrog

Equations for backward calculation ($h \rightarrow -h$):

$$
\begin{align}
   x\left(t-h\right) &= x(t) - hf\left(x\left(t-\frac{1}{2}h\right),t-\frac{1}{2}h\right)\\
   x\left(t-\frac{3}{2}h\right) &= x\left(t-\frac{1}{2}h\right) - hf(x(t-h),t-h)
\end{align}
$$

Let's now start the backward calculation from $t+\frac{3}{2}h$, i.e., $t\rightarrow t+\frac{3}{2}h$

$$
\begin{align}
   x\left(t+\frac{1}{2}h\right) &= x\left(t + \frac{3}{2}h\right) - hf(x\left(t+h),t+h\right)\\
   x(t) &= x(t+h) - hf\left(x\left(t+\frac{1}{2}h\right),t+\frac{1}{2}h\right)
\end{align}
$$

The Leapfrog method is time-reversal symmetric. Time reversal symmetry means that if start from a certain time and go backwards in time we can exactly retrace all steps of the forward solution. Let's start our backward calculation at $t + \frac{3}{2}h$ as shown in the figure. We perform the step between the midpoints from $t+\frac{3}{2}h$ to $t+\frac{1}{2}h$ and the step between the full integer points from $t+h$ to $t$. The same steps are performed in the forward algorithm, just reversed. The mathematical operations are the same, just reversed and the last two equations are identical to the forward equations on slide 3.

### Forward and backward calculation with RK2

Equations for backward calculation ($h \rightarrow -h$):

$$
 \begin {align}
   x\left(t-\frac{1}{2}h\right) &= x(t) - \frac{1}{2}hf(x,t)\\
   x(t-h) &= x(t) - hf\left(x\left(t-\frac{1}{2}h\right),t-\frac{1}{2}h\right)
 \end{align}
$$

Let's start the backward calculation from $t+h$, i.e., $t\rightarrow t+h$

$$
 \begin {align}
   x\left(t+\frac{1}{2}h\right) &= x(t+h) - \frac{1}{2}hf(x(t+h),t+h)\\
   x(t) &= x(t+h)- hf\left(x\left(t+\frac{1}{2}h\right),t+\frac{1}{2}h\right)
 \end{align}
$$

RK2 is not time-reversal. Let's start the backward calculation from $t+h$. We perform a midpoint step from $t+h$ to $t+\frac{1}{2}h$ and a full step form $t+h$ to $h$. However, in the forward algorithm we don't perform a midpoint step from $t+\frac{1}{2}h$ to $t+h$. It is easy to show that the last two equations are not identical to the RK2 forward equations on slide 3.

### Example: Nonlinear pendulum

$$
\frac{\mathrm{d}^2\theta}{\mathrm{d}t^2} = - \frac{g}{l}\sin(\theta)
$$

Transformation into two first-order equations

$$
\begin{align}
 \frac{\mathrm{d}\theta}{\mathrm{d}t}  = \omega, \qquad
 \frac{\mathrm{d}\omega}{\mathrm{d}t} = - \frac{g}{l}\sin(\theta)
\end{align}
$$

The motion of the pendulum is time-reversal symmetric. The motion the pendulum makes in a single period, is exactly the same backwards as it is forwards. The length of the arm is 10 cm and m =1 kg. The potential energy of the pendulum is $V = mgl(1-\cos(\theta))$ and the kinetic energy is $T= 0.5ml^2(\mathrm{d}\theta/\mathrm{d}t)^2$.

```python
from numpy import sin, cos, pi, array, arange
from matplotlib import pyplot as plt

""""Definition of parameters and initial conditions"""
g = 9.81
l = 0.1        # Lenght of arm is 10 cm
a = 0.0        # start time
b = 10.0       # end time
h = 0.001      # time step

start_theta_degree = 10
start_theta =  pi*start_theta_degree/180

tpoints = arange(a,b,h)

"""Right-hand side of differential equations"""
def f(r):
    theta = r[0]
    omega = r[1]
    ftheta = omega
    fomega = -(g/l)*sin(theta)
    return array([ftheta,fomega],float)

""" Potential energy """
def V(theta):
    return g*l*(1-cos(theta))

""" Kinetic energy """
def T(omega):
    return 0.5*l*l*omega*omega

```

```python
""" RK2 integration for pendulum """
def rk2_integration(tpoints,start_theta):
    #Initial condition: r[0] = start_theta ;  and r[1] = omega = 0
    r = array([start_theta,0.0],float)
    thetapoints = []
    Vpoints = []
    Tpoints = []
    Epoints =[]

    for t in tpoints:
        thetapoints.append(r[0]*180/pi)
        Vpoints.append(V(r[0]))
        Tpoints.append(T(r[1]))
        Epoints.append(V(r[0])+T(r[1]))
        k1 = h*f(r)
        k2 = h*f(r+0.5*k1)
        r += k2
    return thetapoints, Epoints

""" Leapfrog integration for pendulum """
def leapfrog_integration(tpoints,start_theta):
    thetapoints = []
    Vpoints = []
    Tpoints = []
    Epoints =[]

    r1 = array([start_theta,0.0],float) # Initial value for point 1
    r2 = r1 + 0.5*h*f(r1)               # Initial value for point 2 = midpoint

    for t in tpoints:
        thetapoints.append(r1[0]*180/pi)
        Vpoints.append(V(r1[0]))
        Tpoints.append(T(r1[1]))
        Epoints.append(V(r1[0])+T(r1[1]))
        r1 += h*f(r2)
        r2 += h*f(r1)
    return thetapoints, Epoints
```

```python
"""Solve problem with RK2"""
thetapointsRK2,EpointsRK2 = rk2_integration(tpoints,start_theta)

"""Solve problem with Leapfrog"""
thetapointsLF,EpointsLF = leapfrog_integration(tpoints,start_theta)

"""Plot theta with respect to time"""
plt.rc('font', size=16)

xstart = a
xend = b

plt.figure(1)
plt.xlim(xstart,xend)
plt.ylim(-start_theta_degree-10,start_theta_degree+10)
#plt.ylim(-200,200)
plt.ylabel('Angle displacement theta')
plt.plot(tpoints,thetapointsRK2,label='RK2',linewidth=1.0)
plt.legend()
plt.figure(2)
plt.xlim(xstart,xend)
plt.ylim(-start_theta_degree-10,start_theta_degree+10)
plt.ylabel('Angle displacement theta')
plt.xlabel('Time')
plt.plot(tpoints,thetapointsLF,label='Leapfrog',linewidth=1.0)
plt.legend()
plt.show()

```

```python
"""Plot total energy with respect to time"""
plt.rc('font', size=16)

xstart = a
xend = b

plt.figure(1)
plt.xlim(xstart,xend)
plt.ylabel('Total energy')
plt.plot(tpoints,EpointsRK2,label='RK2',linewidth=1.0)
plt.legend()
plt.figure(2)
plt.xlim(xstart,xend)
plt.ylabel('Total energy')
plt.xlabel('Time')
plt.plot(tpoints,EpointsLF,label='LF',linewidth=1.0)
plt.plot(tpoints,EpointsRK2,label='RK2',linewidth=1.0)
plt.legend()
plt.show()
```

Now the settings are $t_{end}=b =10$s and $h = 0.001$. Try the following things
-  decrease $h$ by an order of magnitude
-  set $h$ to 0.01 and $b$ to 4000s.

Especially the last point gives some interesting insights. Watch the videos to get some more explanations.

The solution with our numerical solvers is only approximate, which means the total energy of the system is only approximately constant. With RK2 we see clearly a drift in the energy. The leapfrog algorithm conserves energy at the end of a full swing of the pendulum, i.e., at the beginning and the end of the swing the energy will be the same.The energy fluctuates during the course of the swing though, i.e., it is not conserved during fractions of a period. However, at the end of the swing, it will return to the correct value. The leapfrog method is thus useful for solving the equations of motion of energy conserving physical systems. If we wait long enough with RK2, the energy will drift, also with RK4, i.e., the pendulum might stop swinging or the planet might fall out of orbit into the star.

### Verlet method

- specialized method to solve ODEs of the form

\begin{align}
 \frac{\mathrm{d}^2x}{\mathrm{d}t^2} = f(x,t)
\end{align}

- most important example: Newton's equation of motion

\begin{equation}
 \frac{\mathrm{d}^2\mathbf{r}_i}{\mathrm{d}t^2} = \frac{\mathbf{F}_i}{m_i}
\end{equation}

- Verlet is a variant of Leapfrog method

In the case of molecular dynamics (MD), the force on atom $i$ will depend on the positions of all other atoms in the system. The interaction potentials are also in the case of classical MD non-linear functions of $\mathbf{r}$. In the case of ab initio MD the forces are obtained from quantum mechanics, i.e., the corresponding equations of motion must be always solved numerically.

### Derivation of Verlet method

We transform

\begin{align}
 \frac{\mathrm{d}^2x}{\mathrm{d}t^2} = f(x,t)
\end{align}

to 2 first-order differential equations

\begin{equation}
 \frac{\mathrm{d}x}{\mathrm{d}t} = v, \qquad \frac{\mathrm{d}v}{\mathrm{d}t} = f(x,t)
\end{equation}

Using Leapfrog we would define a vector $\mathbf{r} = (x,v)$ and combine the two equations to a single vector equation

\begin{equation}
 \frac{\mathrm{d}\mathbf{r}}{\mathrm{d}t} = \mathbf{f}(\mathbf{r},t)
\end{equation}

With Leapfrog, the explicit expression to solve for $\mathbf{r}$ are

- Full step

\begin{align}
 x(t+h) &= x(t) + hv\left(t+\frac{1}{2}h\right)\\
 v(t+h) &= v(t) + h f\left(x\left(t+\frac{1}{2}h\right),t+\frac{1}{2}h\right)
\end{align}

- Midpoint step
\begin{align}
  x\left(t+\frac{3}{2}h\right) &= x\left(t+\frac{1}{2}h\right) + hv(t+h)\\
  v\left(t+\frac{3}{2}h\right) &= v\left(t+\frac{1}{2}h\right) + hf(x(t+h),t+h)
\end{align}

We can derive a full solution to the problem by only using

\begin{align}
  x(t+h) &= x(t) + hv\left(t+\frac{1}{2}h\right)\\
  v\left(t+\frac{3}{2}h\right) &= v\left(t+\frac{1}{2}h\right) + hf(x(t+h),t+h)
\end{align}

The initial value for $v\left(t+\frac{1}{2}h\right)$ can be obtained from Euler's method with a step size of $\frac{1}{2}h$

\begin{equation}
 v\left(t + \frac{1}{2}h\right) = v(t) + \frac{1}{2}hf(x(t),t)
\end{equation}

We never need to calculate $v$ at any integer point or $x$ at half integers. This is an improvement over leapfrog where we would solve all four equations, i.e., we have to do only half the work compared to leapfrog.
This simplification only works for differential equations that have the specific form  $ \frac{\mathrm{d}^2x}{\mathrm{d}t^2} = f(x,t)$. For these type of ODEs, the right-hand side of the first equation ($ \frac{\mathrm{d}x}{\mathrm{d}t} =v$) depends on $v$, but not on $x$. The right-hand side of the second equation ($\frac{\mathrm{d}v}{\mathrm{d}t} = f(x,t)$) depends on $x$, but not on $v$. However, solving the equations of motion are ODEs of this form and they are a pretty common problem in physics.

There is a small problem so far: We know $v$ at half-integer points and $x$ at full integer points, i.e., we never know both quantities at the same time. This is problematic if we want to calculate the potential, kinetic and total energy of the system because then we have to know $x$ and $v$ at the same time.

To make sure that we know also know $v$ at the integer points, we perform an additional half step. Let's assume we would know $v(t+h)$ then could perform a half-step with step size $-\frac{1}{2}h$ using Euler's method

\begin{equation}
 v\left(t+\frac{1}{2}h\right) = v(t+h) - \frac{1}{2}hf(x(t+h),t+h)
\end{equation}

Rearranging yields:
\begin{equation}
 v(t+h) = v\left(t+\frac{1}{2}h\right) + \frac{1}{2}hf(x(t+h),t+h)
\end{equation}

In combination with the equations on the previous slide, this give us the Verlet method.

### Working equations for the Verlet algorithm

Start:

\begin{equation}
 v\left(t + \frac{1}{2}h\right) = v(t) + \frac{1}{2}hf(x(t),t)
\end{equation}

Then iterate:

\begin{align}
 x(t+h) &= x(t) + hv\left(t+\frac{1}{2}h\right)\\
      k &= hf(x(t+h),t+h)\\
 v(t+h) &= v\left(t+\frac{1}{2}h\right) + \frac{1}{2}k\\
v\left(t+\frac{3}{2}h\right) &=  v\left(t+\frac{1}{2}h\right) + k
\end{align}

We are given initial values of $x$ and $v$ at some time $t$. We start by calculating the $v$ at the $t+\frac{1}{2}h$.  The subsequent values of $x$ and $v$ are then derived by applying the set of equations above.

### Verlet working equations for simultaneous differential equations

Let's assume we have an equation of the form

\begin{align}
 \frac{\mathrm{d}^2\mathbf{r}}{\mathrm{d}t^2} = \mathbf{f}(\mathbf{r},t)
\end{align}

wher $\mathbf{r}=(x,y,...)$ is a vector. The Verlet working equations then transform to

Start:

\begin{equation}
 \mathbf{v}\left(t + \frac{1}{2}h\right) = \mathbf{v}(t) + \frac{1}{2}h\mathbf{f}(\mathbf{r}(t),t)
\end{equation}

Iterate:

\begin{align}
 \mathbf{r}(t+h) &= \mathbf{r}(t) + h\mathbf{v}\left(t+\frac{1}{2}h\right)\\
 \mathbf{k} &= h\mathbf{f}(\mathbf{r}(t+h),t+h)\\
 \mathbf{v}(t+h) &= \mathbf{v}\left(t+\frac{1}{2}h\right) + \frac{1}{2}\mathbf{k}\\
 \mathbf{v}\left(t+\frac{3}{2}h\right) &=  \mathbf{v}\left(t+\frac{1}{2}h\right) + \mathbf{k}
\end{align}

When solving equations of motion, we usually have simulateneous second-order differential equations with position vector $\mathbf{r}=(x,y,z)$, i.e., three simultaneous second-order differential equations, which can be transformed to 6 simulatenous first-order equations. If we want to solve the equation of motions for $n$ interacting particles, we have $6n$ simulatenous euqations to solve.

There are different flavors of the Verlet method. What we have discussed here is often called the Velocity Verlet algorithm.

### Error propagation with the (Velocity) Verlet method

- Verlet conserves, since it is a variant of leapfrog, also the energy
- error for single step is $\mathcal{O}(n^3)$
- accumulated error is $\mathcal{O}(n^2)$

 ### Error propagation with the Verlet method

 Example: Earth (blue) +  moon (gray) orbiting around sun (yellow)

- Equations of motion solved by the (Velocity) Verlet method
- yields right behavior
- Video by Miguel Caro (Advanced Statistical Physics course at Aalto), see also   https://youtu.be/KQAP90SWtiQ

```python
play_VelocityVerlet()
```

### Summary: Methods for solving ODEs and error

| method | single step error | accumulated error |
| --- | --- | --- |
| Euler | $\mathcal{O}(n^2)$ | $\mathcal{O}(n)$ |
| RK2   | $\mathcal{O}(n^3)$ | $\mathcal{O}(n^2)$ |
| RK4        | $\mathcal{O}(n^5)$ | $\mathcal{O}(n^4)$ |
| Leapfrog   | $\mathcal{O}(n^3)$ | $\mathcal{O}(n^2)$ |
| (Velocity) Verlet   | $\mathcal{O}(n^3)$ | $\mathcal{O}(n^2)$ |

Leapfrog and Velocity Verlet are in addition time-reversal symmetric and conserve, e.g., energy.

Lesson learned: it is important to choose a sensible integration scheme for the ODEs together with a proper step size

## Partial differential equations

Many problems in physics are partial differential equations, e.g.,

- Laplace and Poisson equations
- Maxwell's equations
- Schrödinger equation
- wave equation
- diffusion equation

$\rightarrow$ solving them is usually computationally more demanding than the ODE case

### Types of problems

1. boundary problems
2. initial value problems

- $\rightarrow$ initial value problems are typically harder to solve for partial differential equations
- we will learn one method for each

For ODEs we discussed only initial value problems, meaning that we are solving differential equations given the initial values of the variables. This is the most common from for differential equations in physics. However, there are also boundary value problems.

For instance, let's consider the example of a ball thrown into the air. We could specify two initial conditions:  the height of the ball at $t=0$ and its initial upward velocity. Another possibilty is to formulate this example as boundary value problem. We could specify our two conditions as initial and end condition instead. We could specify that the ball has the height $x(t=0)=0$ and $x(t_1)=0$, where $t_1$ is a later time, i.e., we specify the time when the ball is thrown and when it lands.

Initial value problems are generally easier to solve for ODEs. For partial differential equations, the opposite is true.

### Relaxation method for boundary value problems

(Figure from "Computational Physics" by Marc Newman

To introduce the method we will look at a simple electrostatics problem: an empty box, which as conducting wall, all of which are grounded to 0 V except for the wall at the top, which is at some other voltage $V$. The (very) small gaps between the top wall and the others are intended to show that they are insulated from one another.

Goal: determine value of the electrostatic potential at points within the box

This also a boundary value problem: we want to describe the behavior of a variable in space and we are given some constraints on the variable around that space.

We can find the electrostatic potential $\phi$ inside the box by solving the 2D Laplace equation

\begin{equation}
 \frac{\partial^2\phi}{\partial x^2} + \frac{\partial^2\phi}{\partial y^2} = 0
\end{equation}

with the boundary conditions that $\phi =V$ on the top wall and $\phi =0$ on the other walls.

Procedure:
  - use method of finite difference to express the second derivatives of $\phi$
      $\rightarrow$ see lecture topic 2
  - use relaxation method to solve the obtained set of linear simultaneous equations
      $\rightarrow$ see lecture topic 3

### Repetition: Finite differences for second derivatives

We calculate the first derivates first using the central difference method for derivatives at $x+h/2$ and $x-h/2$.

\begin{equation}
 f'(x+h/2) \approx \frac{f(x+h)-f(x)}{h} \qquad f'(x-h/2) \approx \frac{f(x)-f(x-h)}{h}
\end{equation}

Now we applyt the central difference method again for the second derivative

\begin{align}
 f'' &\approx  \frac{f'(x+h/2)-f'(x-h/2)}{h}\\
     & = \frac{f(x+h)-2f(x)+f(x-h)}{h^2}
\end{align}

### Apply the finite difference method to the Laplace equation

(Figure from "Computational Physics" by Marc Newman

- divide the space in the box into grid points with spacing $a$ as shown in the figure.
- put points also on interior and boundaries of that space

Calculate now 2nd derivatives:

\begin{align}
 \frac{\partial^2\phi}{\partial x^2} &=\frac{\phi(x+a,y) + \phi(x-a,y) -2\phi(x,y)}{a^2}\\
 \frac{\partial^2\phi}{\partial y^2} &=\frac{\phi(x,y+a) + \phi(x,y-a) -2\phi(x,y)}{a^2}
\end{align}

The Laplace equation in 2D is now:

\begin{equation}
\frac{\partial^2\phi}{\partial x^2} +  \frac{\partial^2\phi}{\partial y^2} = \frac{\phi(x+a,y) + \phi(x-a,y) +\phi(x,y+a) + \phi(x,y-a)-4\phi(x,y)}{a^2} = 0
\end{equation}

We basically add the values of $\phi$ at all the grid points adjacent to $(x,y)$ and subtract 4x the value at $(x,y)$ and then divide by $a^2$.

We need to solve

\begin{equation}
 \phi(x+a,y) + \phi(x-a,y) +\phi(x,y+a) + \phi(x,y-a)-4\phi(x,y) = 0
\end{equation}

- we have one equation like this for every grid point $(x,y)$
- the solution to the entire set gives us $\phi(x,y)$ at every grid point

$\rightarrow$ large set of linear simultaneous equations

$\rightarrow$ method of choice is here the relaxation method (lecture topic 3)

This linear set of equations could be solved, in principle, with Gaussian elimination or LU decomposition. In this case, using the relaxation method is a better choice because it is computationally cheaper. We introduced the relaxation method for non-linear equations, but they -of course- also be applied to linear equations.

### Application of the relaxation method to our electrostatics problem

Let's first rearrange

\begin{equation}
\phi(x,y) =\frac{1}{4} (\phi(x+a,y) + \phi(x-a,y) +\phi(x,y+a) + \phi(x,y-a))
\end{equation}

Procedure:
- fix $\phi(x,y)$ at the boundaries of the system
- guess some initial values for $\phi(x,y)$, can be bad, doesn't matter
- calculate new values $\phi'$ we the guessed initial values
- repeat until convergence reached

$\rightarrow$ this is also known as  Jacobi method

### Solution of the 2D Laplace equation

Let's now solve our electrostatics problem with the Jacobi method assuming that the box is 1 m long each side, $V =  1$ volt and the grid spacing $a = 1cm$. We hve 100 grid points or 101 if we count the points at both beginning and end.

```python
from numpy import empty,zeros,max
from matplotlib import pyplot as plt

#Constants
M = 100         # Grid squares on a side
V = 1.0         # Voltage at top wall
target = 1e-6   # Target accuracy

# Create arrays to hold potential values
phi = zeros([M+1,M+1],float)
phi[0,:] = V
phiprime = empty([M+1,M+1],float)

# Main loop
delta = 1.0
while delta>target:

    # Calculate new values of the potential
    for i in range(M+1):
        for j in range(M+1):
            if i==0 or i==M or j==0 or j==M:
                phiprime[i,j] = phi[i,j]
            else:
                phiprime[i,j] = (phi[i+1,j] + phi[i-1,j] \
                                 + phi[i,j+1] + phi[i,j-1])/4

    # Calculate maximum difference from old values
    delta = max(abs(phi-phiprime))

    # Swap the two arrays around
    phi,phiprime = phiprime,phi
    # Alternative to swapping
    #phi  = phiprime
    #phiprime = empty([M+1,M+1],float)

# Make a plot
plt.imshow(phi)

```

The produced figure shows that there is a region of high electric potential around the top wall of the box, as expected, and low  potential aroudn the other three walls. Note that the program will run for a while.

A few notes about the program:
- if point $i,j$ at boundary, then set to start values; the values at the boundaries never change
- if points $i,j$ not at boundary, calculate $\phi'$

### Things to note about the Jacobi method

Accuracy:
- only approximate since we use finite differences for derivatives
- small target accuracy won't fix this
- higher-order derivative approximation necessary to improve accuracy

Accessible points
- the calculation give the value of $\phi$ only at the grid points and not elsewhere
- in between values: interpolation schemes possible

 Division of space
- boundaries around space may not always be square
- can be difficult to divide space with square grid $\rightarrow$ grid points don't fall on boundaries

### Other, faster methods for boundary value problems

- overrelaxation
- Gauss-Seidel method

### Initial value problems

- starting point of variable known
- goal: prediction of future variation as function of time

Example:  Diffusion equation
\begin{equation}
    \frac{\partial \phi}{\partial t} = D\frac{\partial^2\phi}{\partial x^2}
\end{equation}

- use space-time grid? 🤔
- we have boundary conditions in the spatial dimension ($x$), but not in time dimension
$\rightarrow$ relaxation method breaks down because we don't know what value to use for time-like end of the grid

We have now (compared to the 2D Laplace equation), the independent variables $x$ and $t$, instead of $x$ and $y$ One might think that we can proceed as before and create a space-time grid, then write the derivatives in finite difference form and get a set of simultaneous equations that can be solved by the relaxation method. This doesn't work because we have only boundary conditions in the spatial direction $x$. For the time dimension we have an initial conditions. We know where the value starts, but not where it ends. The relaxation method breaks thus down because we don't know what value of $\phi$ to use for time-like end of the grid.

### FTCS method

- short for forward-time centered-space method
- method to solve initial value problems for partial differential equations

Procedure:
 - divide spatial dimension $x$ into a grid of points with spacing $a$
 - calculate second derivative with respect to $x$ with finite differences
   $\rightarrow$ simultaneous ordinary differential equations are obtained
 - use the Euler method to solve them

For the ODEs we arrived at the conclusion that we shouldn't use Euler's method due to the poor accuracy. Why do we use it here? Estimating the second derivative with finite differences is not very accurate. There is no point in using a very accurate solver. Euler's method gives errros which are comparable to the error introduced by the finite difference approach.

### Example: diffusion equation

\begin{equation}
    \frac{\partial \phi}{\partial t} = D\frac{\partial^2\phi}{\partial x^2}
\end{equation}

Take second derivatives with finite differences

\begin{align}
 \frac{\partial^2\phi}{\partial x^2} =\frac{\phi(x+a,y) + \phi(x-a,y) -2\phi(x,y)}{a^2}
\end{align}

and insert in diffusion equation

\begin{equation}
    \frac{\partial \phi}{\partial t} = \frac{D}{a^2}\left[\phi(x+a,y) + \phi(x-a,y) -2\phi(x,y)\right]
\end{equation}

We can think of the value of $\phi$ at the different grid points as separate variables
$\rightarrow$ we have a set of simultaneous ODEs now.

We have the ODE

\begin{equation}
 \frac{\mathrm{d}\phi}{\mathrm{d}t} = f(\phi,t)
\end{equation}

where $f(\phi,t)$ is the right-hand side of the last equation. Solving with Euler yields

\begin{equation}
 \phi(x,t+h) = \phi(x,t) + h\frac{D}{a^2}\left[\phi(x+a,y) + \phi(x-a,y) -2\phi(x,y)\right]
\end{equation}

If we know $\phi$ at every grid pint $x$ hat some time $t$, then we get from this equation the value of each grid point at time $t+h$.

#### Example: Solving the heat equation with FTCS

We have a steel container, which is 1 cm thick and is initially at a uniform temperature of 20 degree Celsius everywhere. The container is placed in a bath of cold water at 0 degree celsius and filled with hot water at 50 degree Celsius.

Assumptions: container is arbitrarily wide. Neither cold not hot water change temperature.

We have to solve the 1D diffusion equation for the temperature $T$

\begin{equation}
 \frac{\partial T}{\partial t} = D \frac{\partial^2 T}{\partial x^2}
\end{equation}

The $x$-axis is divided into 100 equal points (101 in total counting boundaries). The first and last point have fixes temperatures at 50 degree Celsius and 0 degree Celsius, respectively. Intermediate points are initially at 20 degree Celsius. The diffusion coefficient is $D =  4.25\times10^{-6}\text{m}^2\text{s}^{-1}$.

Task: Plot of temperatures profile at times $t = 0.01, 0.1, 0.4, 1, 10$ s.

```python
from numpy import empty
from matplotlib import pyplot as plt
import time

# Constants
L = 0.01      # Thickness of steel in meters
D = 4.25e-6   # Thermal diffusivity
N = 100       # Number of divisions in grid
a = L/N       # Grid spacing
h = 1e-4      # Time-step
epsilon = h/1000

Tlo = 0.0     # Low temperature in Celcius
Tmid = 20.0   # Intermediate temperature in Celcius
Thi = 50.0    # Hi temperature in Celcius

t1 = 0.01
t2 = 0.1
t3 = 0.4
t4 = 1.0
t5 = 10.0
tend = t5 + epsilon

# Create arrays
T = empty(N+1,float)
T[0] = Thi
T[N] = Tlo
T[1:N] = Tmid
Tp = empty(N+1,float)
Tp[0] = Thi
Tp[N] = Tlo

# Main loop
t = 0.0
c = h*D/(a*a)

start = time.time()
while t<tend:

    # Calculate the new values of T

    #for i in range(1,N):  ## Loop is the slow alternative
    #    Tp[i] = T[i] + c*(T[i+1]+T[i-1]-2*T[i])

    Tp[1:N] = T[1:N] + c* (T[2:N+1]+T[0:N-1]-2*T[1:N])
    T,Tp = Tp,T
    t += h

    # Make plots at the given times
    if abs(t-t1)<epsilon:
        plt.plot(T,label='t1')
    if abs(t-t2)<epsilon:
        plt.plot(T,label='t2')
    if abs(t-t3)<epsilon:
        plt.plot(T,label='t3')
    if abs(t-t4)<epsilon:
        plt.plot(T,label='t4')
    if abs(t-t5)<epsilon:
        plt.plot(T,label='t5')

end = time.time()
print(end - start)
plt.xlabel("x")
plt.ylabel("T in degree Celsius")
plt.legend()

```

The temperatue profile starts off with a large region of the system at 20 degree Celsius and a much smaller region of hotter and colder temperatures. The profile becomes smoother over time, til we have a straight line in the end.

Comments:
- do not compare directly `t == t1`. They could be unequal due to numerical rounding errors; calculating the absolut value of the difference is better

- use arrays instead of loop to calculate Tp; it's faster.

### Application of FTCS to wave equation

The 1D wave equation is

\begin{equation}
 \frac{\partial^2\phi}{\partial x^2} - \frac{1}{v^2}\frac{\partial^2\phi}{\partial t^2} =0
\end{equation}

Your task in exercise 5.6 is to apply the FTCS method to the wave equation.This is an interesting case considering numercial stability.

### Numerical stability of FTCS

- von Neumann stability analysis yields for the diffusion equation that the solution will be stable if

\begin{equation}
 h\leq \frac{a^2}{2D}
\end{equation}

- for the wave equation there exists no such expression.

Lecture 5 - Differential equations/lecture_topic5_differential_eq_part2.ipynb


```python
# Imports needed for this exercice
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import hcipy as hc #
```

In this homework we will reproduce numerically the results from Labeyrie et al. (1970)
https://ui.adsabs.harvard.edu/abs/1970A%26A.....6...85L/abstract

First we will review how the Fourier Transform scales in a multiples and single dish telescopes based on discussions from the lecture. Our notations are:

- $\mathbf{x}$ dimensions in units of length at the location the astrophysical scene $\mathcal{A}$

- $z$ distance from astrophysical scene to observatory

-  $\mathbf{u}$ dimensions in units of length at plane of observatory $\mathcal{O}$

- $D$ and $f$ are size of telescope aperture and focal length

- $\mathbf{\xi}$ dimensions in units of length at the telescope focus $\mathcal{F}$

We first start with the propagation from the location of astrophysical scene to the plane of observatory, assuming a coherent spatially distributed electrical field (never really happens but let's pretend):

\begin{equation}
E_{\mathcal{O}}(\mathbf{u}) = \frac{1}{\pi \lambda z} \int E_{\mathcal{A}}(\mathbf{x}) e^{- i \frac{2 \pi}{\lambda z} \mathbf{x} \mathbf{u} } d\mathbf{x}
\end{equation}

Instead of integarting over the physical distance at the location the astrophysical scene, we use the angular size of the astrophysical scene $\mathbf{\alpha}_{z} = \mathbf{x}/z$ and we find:

\begin{equation}
E_{\mathcal{O}}(\mathbf{u}) = FT[E_{\mathcal{A}}(\mathbf{\alpha}_{z})] (\frac{\mathbf{u}}{\lambda})
\end{equation}

If the incoherent case we assume that we have a sum of delta functions and very similar algebra yeilds:

\begin{equation}
I_{\mathcal{O}}(\mathbf{u}) = | FT[I_{\mathcal{A}}(\mathbf{\alpha_{z}})] (\frac{\mathbf{u}}{\lambda}) |
\end{equation}

This is where we find the analogy between the two planes in radio astronomy or interferometry: the plane of the sky, in units of observable angle $\mathbf{\alpha}$ and the plane of the observatory u-v plane, in units of "baseline over wavelength" $\frac{\mathbf{u}}{\lambda}$.

The relationship between the plane of observatory and the telescope focus is:

\begin{equation}
E_{\mathcal{F}}(\mathbf{\xi}) = \frac{1}{\pi \lambda f} \int E_{\mathcal{O}}(\mathbf{u}) e^{- i \frac{2 \pi}{\lambda f} \mathbf{u} \mathbf{\xi} } d\mathbf{u}
\end{equation}

We now use the angular size of the focal plane  $\mathbf{\alpha}_{f} = \mathbf{\xi}/f$ and find:

\begin{equation}
E_{\mathcal{F}}(\mathbf{\alpha}_{f}) = FT[E_{\mathcal{O}}(\mathbf{u})] (\frac{\mathbf{\alpha}_{f}}{\lambda})
\end{equation}

Using the expression for the plane of the observatory we derived before we find:

\begin{equation}
E_{\mathcal{F}}(\mathbf{\alpha}_{f}) = FT \left( FT[E_{\mathcal{A}}(\mathbf{\alpha}_{z})] (\frac{\mathbf{u}}{\lambda})\right) (\frac{\mathbf{\alpha}_{f}}{\lambda})
\end{equation}

In virture of the fourier scaling theorem the $\lambda$ s cancel out and we find that in angular units the image of the sky is at the focus of the telescope (with a sign flip if the telesocpe is only one lens):

\begin{equation}
I_{\mathcal{F}}(\mathbf{\alpha}_{f}) = I_{\mathcal{A}}(-\mathbf{\alpha}_{Z})
\end{equation}

However when we want to deal with instrumental (or atmospheric) artifacts seen through a single dish astronomical telescope we usually use slightly different notations:

\begin{equation}
E_{\mathcal{F}}(\mathbf{\xi}) = \frac{1}{\pi \lambda f} \int E_{\mathcal{O}}(\mathbf{u}) e^{- i 2 \pi \lambda f \frac{\mathbf{u}}{D} \frac{\mathbf{\xi} D}{\lambda f} } d\mathbf{u}
\end{equation}

If we define the dimensions in the plane of the observatory (also called telescope pupil) in units of telescope diameter $\mathcal{v} = \frac{\mathcal{u}}{D}$ and the dimensions in the focal plane is units of the angular resolution of the telescope $\mathbf{\eta} = \frac{\mathbf{\xi} D}{\lambda f} $ then we are working with a very nice set of unitless dimensions:

\begin{equation}
E_{\mathcal{F}}(\mathbf{\eta}) = \frac{1}{\pi \lambda f} \int E_{\mathcal{O}}(D \mathbf{v}) e^{- i 2 \pi \mathbf{v} \mathbf{\eta}} d\mathbf{v}
\end{equation}

These are the dimensions used in HCIPy when making the focal grid. One can choose to work in units of angular resolution (and to not specify anything in the make_focal_grid function but make sure that pupil_grid diameter is one) or physical units and specify eveyrthing (which is what we will do below).

# Make the image of a binary star

The goal is to become familiar with the way physical light is propagated in an instrument. The machine is fourier transforms of course, but the dimensions/scalings can be tricky

1) The following example shows how to simjulate the Point Spread Function of a 5 m telescope, with a 71.5 m focal length, at 750 nm, all the way to the 30 th airy ring with a sampling of 2 pixels per airy ring. Repeat the same thing using generic coordinates (eg units of angular resolution).

```python
Npup = 256
D_tel = 5
f = 71.5
wavelength =  750e-9
pupil_grid = hc.field.make_pupil_grid(Npup, diameter=D_tel)
circular_aperture_object =  hc.aperture.circular_aperture(D_tel, center=[0,0]) # We use the center option to create the apertures
telescope_pupil = circular_aperture_object(pupil_grid)
plt.figure()
im = hc.imshow_field(telescope_pupil)
plt.colorbar()
plt.show()
wavefront = hc.Wavefront(telescope_pupil,wavelength)
focal_grid = hc.make_focal_grid(q=2, num_airy=30,pupil_diameter=D_tel, focal_length=f, reference_wavelength=wavelength)
prop = hc.FraunhoferPropagator(pupil_grid, focal_grid,focal_length=f)
focal_image = prop.forward(wavefront)
plt.figure()
hc.imshow_field(np.log10(focal_image.intensity / focal_image.intensity.max()))
plt.xlabel('Focal plane distance [$mu$ m ]')
plt.ylabel('Focal plane distance [$mu$ m]')
plt.colorbar()
plt.show()
```

2) Using this, create a simulated image of a binary system with a second star at $5 \lambda /D$ (using in generic coordinates). The tricky part is to get the right slope for the phase ramp before the Fourier transform. You will know you will get it right when the second star lands on the 5 th Airy ring.

3) Same exercise but this time in physical coordinates

# Make the image of a binary star behind telescope turbulence

This is how one case define the main charateristic of atmospheric turbulence. If you can curious you can read one of the great classics
https://www.sciencedirect.com/science/article/abs/pii/S007966380870204X?via%3Dihub

```python
seeing = 1.4 # arcsec @ 500nm (convention)
outer_scale = 40 # meter
tau0 = 0.005 # seconds
fried_parameter = hc.seeing_to_fried_parameter(seeing)
Cn_squared = hc.Cn_squared_from_fried_parameter(fried_parameter, 500e-9)
velocity = 0.314 * fried_parameter / tau0
print('r0   = {0:.1f}cm'.format(fried_parameter * 100))
print('L0   = {0:.1f}m'.format(outer_scale))
print('tau0 = {0:.1f}ms'.format(tau0 * 1000))
print('v    = {0:.1f}m/s'.format(velocity))
```

    r0   = 7.2cm
    L0   = 40.0m
    tau0 = 5.0ms
    v    = 4.5m/s

4) $r_0$ is the equivalent size of a telescope if observations were diffraction limited. Could resolve our hypothetical binary star at 5 $\lambda/D$ on a 5 m telescope?

This is how we generaty synthetic phase screens representtaive of Kolmogorov  turbulence in one layer of the atmosphere. We work in physical units here.

```python
Npup = 256
D_tel = 5
f = 71.5
wavelength =  750e-9
pupil_grid = hc.field.make_pupil_grid(Npup, diameter=D_tel)
circular_aperture_object =  hc.aperture.circular_aperture(D_tel, center=[0,0]) # We use the center option to create the apertures
telescope_pupil = circular_aperture_object(pupil_grid)
wavefront = hc.Wavefront(telescope_pupil,wavelength)
focal_grid = hc.make_focal_grid(q=2, num_airy=30,pupil_diameter=D_tel, focal_length=f, reference_wavelength=wavelength)
prop = hc.FraunhoferPropagator(pupil_grid, focal_grid,focal_length=f)
layer = hc.InfiniteAtmosphericLayer(pupil_grid, Cn_squared, outer_scale, velocity)
```

```python
phase_screen_phase = layer.phase_for(wavelength) # in radian
hc.imshow_field(phase_screen_phase, cmap='RdBu')
plt.colorbar()
```

This is how we make  the layer moves across the telescope aperture

```python
layer.t = 0
phase_screen_phase = layer.phase_for(wavelength) # in radian
plt.figure()
hc.imshow_field(phase_screen_phase, cmap='RdBu')
plt.colorbar()
layer.t = 0.1
phase_screen_phase = layer.phase_for(wavelength) # in radian
plt.figure()
hc.imshow_field(phase_screen_phase, cmap='RdBu')
plt.colorbar()
layer.t = 0.2
phase_screen_phase = layer.phase_for(wavelength) # in radian
plt.figure()
hc.imshow_field(phase_screen_phase, cmap='RdBu')
plt.colorbar()
layer.t = 0.3
phase_screen_phase = layer.phase_for(wavelength) # in radian
plt.figure()
hc.imshow_field(phase_screen_phase, cmap='RdBu')
plt.colorbar()
```

## Combine results from previous sections to demonstrate how speckles imaging works

5) Based on all that was done so far. Reproduce the results from Labeyrie et al. 1970. For debuging purposes you can first ignore the atmosphere and simply fourier transform the results from question 3 (or two) to get a feel for what is going on. There is little guidance by design. We will go over it in class.

6) Explore brightness ratio of binary, serpations, atmopsphrice conditions and averaging of images and discuss how some of these various paramters qualitively change the SNR of a detction.