# Time Evolution: Split Operator Method

### Category: Prerequisites

### Prerequisites: Quantum Mechanics

When cleaning my apartment, sometimes I just grab the nearest dirty thing to me and try to do something to it.  But that is not the most efficient way to get things done.  If I'm planning, I'll first dedicate my attention to one problem, like putting clothing away, then rotate my attention to something else, like dirty dishes.  I can keep focused on just one task and do it well.  Each problem I solve optimally in shorter intervals instead of tackling everything at once.

That same principle applies to solving partial differential equations.  [1] called this principle one of the big ideas of numerical computation.  In numerics, we call it Strang splitting.

We will be applying Strang splitting to solve the Schrondinger equation, but people use the same idea to a variety of problems, like ones with different timescales, length scales, or physical processes.  We will be using it to seperate out terms diagonal in position space from terms diagonal in momentum space.

We can describe a class of general partial differential equations by
\begin{equation}
\frac{\partial}{\partial t}y = L_1(y,t)+L_2(y,t)
\end{equation}

Over a small time step, the following approximation holds
\begin{equation}
y(\delta t)= e^{L_1(y,0) \delta t+L_2(y,0) \delta t}y(0)
\end{equation}

For Strang splitting, instead of applying both operators together, we break them up into two.  I'll discuss non-commutivity later.
\begin{equation}
y(\delta t)= e^{L_1 (y,0) \delta t} e^{L_2(y,0) \delta t} y(0) = U_1 U_2 y(0)
\end{equation}
$U_1$ and $U_2$ are evolution operators. We can define
\begin{equation}
\tilde{y(0)}= U_2 y(0)
\end{equation}
so that
\begin{equation}
y (\delta t) = U_1 \tilde{y}(0)
\end{equation}

### Applying to Quantum Mechanics
Now let's take a look at the Schrodinger Equation:
\begin{equation}
i \hbar \frac{\partial}{\partial t} | \Psi \rangle = \mathcal{H} | \Psi \rangle
=
\left[ \frac{\hat{p}^2}{2m} + V(x) \right] | \Psi \rangle
=
\left[ \mathcal{H}_p + \mathcal{H}_x \right] | \Psi \rangle
\end{equation}
The Hamiltonian gets seperated into position terms and momentum terms.  For ease, let's define our unitary evolution operators,
\begin{equation}
U_p(\delta t)=e^{-\frac{i}{\hbar}\mathcal{H}_p \delta t}
\;\;\;\;\;
U_x (\delta t)= e^{-\frac{i}{\hbar}\mathcal{H}_x \delta t}
\end{equation}

I mentioned earlier that I would discuss non-communitvity.  We need to do that now.  We can't simply seperate the evolution operator for the full Hamiltonian into two parts, because we would introduce terms proportional to the commutator.
\begin{equation}
e^{A+B}=e^{A}e^{B}e^{[A,B]}
\end{equation}
$e^{A+B}$ expanded has terms that look like $AB$ and $BA$, whereas $e^{A}e^{B}$ only has terms that look like $AB$.  We lose the symmetry of the expression.  We can gain back an order of accuracy by symmetrizing our formula, calculating a time step by
\begin{equation}
|\Psi (\delta t) \rangle =
U_x (\delta t/2) U_p (\delta t) U_x (\delta t/2)
\end{equation}
But the next step will the start with $U_x (\delta t/2)$ !

\begin{equation}
|\Psi (2 \delta t) \rangle = \left(
U_x (\frac{\delta t}{2}) U_p (\delta t) U_x (\frac{\delta t}{2})\right)\left( U_x (\frac{\delta t}{2}) U_p (\delta t) U_x (\frac{\delta t}{2}) \right)
\end{equation}
\begin{equation}
= U_x (\frac{\delta t}{2}) U_p (\delta t) U_x (\delta t)  U_p (\delta t) U_x (\frac{\delta t}{2})
\end{equation}

All we need to do to add an order of accuracy is start the simulation with $U_x(\delta t/2)$ and end it with $U_x(\delta t/2)$, leaving everything else the same.  Pretty remarkable you can get that much of an improvement for that little work.  Once we apply this to a bunch of time steps, we get
\begin{equation}
U_x (\frac{\delta t}{2}) U_p (\delta t) \left( \prod_{n-1} U_x(\delta t) U_p (\delta t) \right) U_x (\frac{\delta t}{2}).
\end{equation}
We have to apply a few operators before starting the loop. Between the loop and a measurement, we have to apply an additional operator.

In the spatial domain, the momentum operator involves derivatives and is rather icky.  But in the momentum domain, we only have to multiply by $k^2/2m$.  Thanks to some nicely optimized libraries, we can just transform into the momentum domain with `fft`, solve the momentum problem there, and transform back with `ifft`.

## Rabi Oscillations

To demonstrate time evolution in a simple system with interesting physics, I chose to apply the split operator to Rabi Oscillations between two harmonic oscillators.

To get an idea of what will happen, we will use a qualitative model of two states weakly coupled to each other by a parameter $\epsilon$.  If we have the two minima sufficifiently seperated from each other, tunneling will happen slowly and will not significantly affect the shape of the eigenfunctions and their energies $E_0$.  Instead of of solving for the shape of the wavefunction, we solve a two-state Hamiltonian that looks like this,

\begin{equation}i \hbar \frac{\partial}{\partial t}
\begin{bmatrix}
| \phi_r \rangle  \\
| \phi_l \rangle
\end{bmatrix}
= \begin{bmatrix}
E_0  & \epsilon \\
ϵ & E_0 \\
\end{bmatrix}
\begin{bmatrix}
| \phi_r \rangle \\
| \phi_l \rangle
\end{bmatrix}
\end{equation}
The eigenvalues and corresponding eigenvectors of the matrix are,
\begin{equation}
\lambda_1 = E_0 + \epsilon \;\;\;\;\;\;
\lambda_2 = E_0 - \epsilon
\end{equation}

\begin{equation}
\vec{v}_{1} = \begin{bmatrix}
| \phi_r \rangle \\
| \phi_l \rangle
\end{bmatrix}
\;\;\;\;\;
\vec{v}_2 = \begin{bmatrix}
| \phi_r \rangle \\
- |\phi_l \rangle
\end{bmatrix}
\end{equation}

If a wavefunction starts purely in the right state, we want to choose a combination of our eigenvectors that sets the left state to zero at $t=0$.  The resulting wavefunction will evolve as,
\begin{equation}
| \Psi (t) \rangle = \frac{1}{\sqrt{2}}e^{-\frac{i}{\hbar} E_0 t } \left(
e^{-\frac{i}{\hbar} \epsilon t} \begin{bmatrix}
| \phi_r \rangle \\
| \phi_l \rangle
\end{bmatrix}
+ e^{\frac{i}{\hbar} \epsilon t} \begin{bmatrix}
| \phi_r \rangle \\
-| \phi_l \rangle
\end{bmatrix}
\right)
\end{equation}

\begin{equation}
= \sqrt{2} e^{-\frac{i}{\hbar} E_0 t } \begin{bmatrix}
\cos \frac{\epsilon t}{\hbar} | \phi_r \rangle \\
i \sin \frac{\epsilon t}{\hbar} | \phi_l \rangle
\end{bmatrix}
\end{equation}

Thus this simple phenomological model shows us how we can expect the wavefunction to move back and forth between the two wells in a cyclical manner, with a period proportional to the rate of tunneling.

#### Packages
For the first time, I'm using `Plots.jl`, with `PlotlyJS` as the backend for `Plots`.  I had to coax my computer a bit to get the packages to work, but the errors were specific to my computer.  I choose to switch as I believe the `PlotlyJS` output will provide a better expierence for those viewing the GitHub-Pages static site, though it might cause more trouble for anyone using the jupyter notebooks.  If you are having trouble plotting yourself, I recommend just switching back to whatever package is easiest for you.

I used `Plots` to generate a gif for me directly, but I found my old method of generating a file of .png files and then using `ffmpeg` from the command line much fast and easier.

```julia
using Plots
using FFTW
gr()
```

    ┌ Info: Recompiling stale cache file /home/shaula/.julia/compiled/v1.1/FFTW/PvIn2.ji for FFTW [7a1cc6ca-52ef-59f5-83cd-3a7055c09341]
    └ @ Base loading.jl:1184

    Plots.GRBackend()

## Input Parameters

```julia
# Set Time Parameters
t0=0
tf=40000
dt=.1

# Set Space Grid Parameters
dx=.1
xmax=8
# xmin will be -xmax.  Making the situation symmetric

# How far seperated are the potential minima
seperation=6;
# minima at seperation/2 and -seperation/2

# How often we measure occupation and view the state
nmeasure=1000;
```

## Automatic Evaluation Parameters

Given the parameters above, we can calculate the following variables that we will use in the code.

Note: `k` Gave me a bit of a headache. The algorithm depends quite a bit on the conventions `fft` decides to use ;(
Currently, I'm using odd `N`.  You'll have to change the formula if you use even `N`.

```julia
t=collect(t0:dt:tf)
x=collect(-xmax:dx:xmax)

nt=length(t)
N=length(x)

k = [ collect(0:((N-1)/2)) ; collect(-(N-1)/2:-1) ] *2*π/(N*dx);

occupation=zeros(Complex{Float64},floor(Int,nt/nmeasure),2);
```

## The Potentials and Evolution Operators

```julia
Vx=.5*(abs.(x).-seperation/2).^2;
Vk=k.^2/2

Uxh=exp.(-im*Vx*dt/2);
Ux=exp.(-im*Vx*dt);
Uf=exp.(-im*Vk*dt);
"potentials and evolvers defined"
```

    "potentials and evolvers defined"

```julia
plot(x, Vx)

plot!(xlabel="x", ylabel="V",
plot_title="Double Well Potential")
```

## The Unperturbed Wavefunctions

The ground state for a harmonic oscillator is a Gaussian
\begin{equation}
\langle x | \phi \rangle= \phi (x) = \frac{1}{\pi^{1/4}} e^{-\frac{x^2}{2}}
\end{equation}
We assume $\omega = \hbar = m = 1$ for sake of convenience.

```julia
ϕ(x)=π^(-.25)*exp(-x.^2/2)
```

    ϕ (generic function with 1 method)

```julia
ϕl=ϕ.(x.+seperation/2);
ϕr=ϕ.(x.-seperation/2);
Ψ0=convert(Array{Complex{Float64},1},ϕl);
```

```julia
plot(x,ϕl,label="ϕl")
plot!(x,ϕr,label="ϕr")

plot!(xlabel="x", ylabel="ϕ",
plot_title="Left and Right Wavefunctions")
```

## FFT's

This algorithm runs a large number of Fast Fourier Transforms and Inverse Fast Fourier Transforms.  To speed the process, we can tell the computer to spend some time, in the beginning, allocating the right amount of space and optimizing the routine for the particular size and type of array we are passing it.

The next cell does this, by using `plan_fft` and `plan_ifft` to generate objects that can act on our arrays as operators.

```julia
ft=plan_fft(Ψ0);
Ψf=ft*Ψ0;
ift=plan_ifft(Ψf);
```

## Occupancy of each state

To measure the occupancy of the total wavefunction in either the left or right well groundstate, I compute the value
\begin{equation}
c_r=\langle \Psi | \phi_r \rangle  = \int \Psi^* (x) \phi_r(x) dx \;\;\;\;\;\;\; p_r=c_r^2
\end{equation}
\begin{equation}
c_l = \langle \Psi | \phi_l \rangle = \int \Psi^* (x) \phi_l (x) dx\;\;\;\;\;\;\; p_l = c_l^2
\end{equation}
The probability of being in the state is the coefficient squared.

Though in theory these values will always be real, numerical errors introduce some errors, and Julia will assume that the answer is complex.  Therefore, we need to apply `abs` to make the numbers `Float64` instead of `Complex{Float64}`.

```julia
nmeas=1000
c=zeros(Float64,floor(Int,nt/nmeas),2);
```

```julia
# Uncomment the # lines to generate a gif.  Note: It takes a long time

Ψ=Ψ0;
jj=1

# The operators we have to start off with
Ψ=Ψ.*Uxh

Psif=ft*Ψ
Psif=Psif.*Uf
Ψ=ift*Psif

#@gif for ii in 1:nt
for ii in 1:nt

    Ψ=Ψ.*Ux

    Psif=ft*Ψ
    Psif=Psif.*Uf
    Ψ=ift*Psif

    if ii%nmeas == 0

        # Every time we measure, we have to finish with a U_x half time step
        Ψt=Ψ.*Uxh

        c[jj,1]=abs(sum( conj(Ψt).*ϕl )) *dx
        c[jj,2]=abs(sum( conj(Ψt).*ϕr )) *dx

        jj+=1
    end

    #plot(x[21:141],Vx[21:141]/6, label="Vx scaled")
    #plot!(x,abs(conj(Ψt).*Ψt), label="Wavefunction")
    #plot!(xlabel="x", ylabel="Ψ",
    #    plot_title="Wavefunction evolution")

end
#end every nmeas

Ψ=Ψ.*Uxh;
```

```julia
plot(c[:,1].^2,label="Left Prob")
plot!(c[:,2].^2, label="Right Prob")
plot!(xlabel="x",ylabel="Probability",
    plot_title="Rabi Oscillations for a Double Harmonic Oscillator")
```

PyPlot generated png's strung together:

Plots generated gif:

The faster one can perform Fourier Transforms, the faster one can perform this algorithm.  Therefore, scientists, such as [2], will use multiple cores or GPU's.

In addition to real time evolution, algorithms like this can determine the ground state of an arbitrary system by imaginary time evolution.  Soon, I will take the work covered here and look at this aspect of the algorithm.

[1] Glowinski, Roland, Stanley J. Osher, and Wotao Yin, eds. Splitting Methods in Communication, Imaging, Science, and Engineering. Springer, 2017.

[2] Heiko Bauke and Christoph H. Keitel. Accelerating the Fourier split operator method via graphics processing unit. Computer Physics Communications, 182(12):2454–2463 (2011)

```julia

```


# Elemento Spring

## Fundamento teórico

El elemento *Spring* es un elemento finito unidimensional donde
las coordenadas locales y globales coinciden. Cada elemento spring tiene dos
nodos como se muestra en la figura siguiete. Sea la rigidez del
resorte la denotada por $k$, en este caso la matriz de rigidez del elemento está
dada por:

\begin{equation}
K_{(e)} = \begin{bmatrix}
k & -k \\
-k & k
\end{bmatrix}
\end{equation}

Obviamente la matriz de rigidez para un elemento *spring* es de $2\,x\,2$, dado que
este tiene dos grados de libertada, uno en cada nodo. Consecuentemente para un
sistema de elementos *spring* con $n$ nodos, el tamaño de la matriz global de
rigidez $K$ será de $n\,x\,n$. La matriz global de rigidez se obtiene ensamblando
los matrices de rigidez por elemento $K_{(i)}$ para $i=1,2,...,n$, utilizando el método
directo de la rigidez.

Una vez que la matriz global de rigidez $K$ es obtenida  se tiene un sistema de ecuaciones
de la forma:

\begin{equation}
[K]\{U\} = \{F\}
\end{equation}

Donde $U$ es el vector global de desplazamientos nodales y $F$ es el vector global de
fuerzas nodales.

El sistema de ecuaciones resultantes se puede simplificar aplicando las condiciones
de frontera o restricciones de desplazamiento, quedando generalmente un sistema
de menor dimensión el cuál está determinado y puede resolverse utilizando métodos
de álgebra lineal´, quedando una posible solución como:

\begin{equation}
\overline{U} = \overline{K}^{-1}\, \overline{F}
\end{equation}

Donde $\overline{U}, \overline{K} \,\, y \,\, \overline{F}$ corresponden a las variables descritas
anteriormente, después de aplicar las condiciones de frontera correspondientes.

## Un ejemplo resuelto en NuSA

**Ejemplo 1.** Para el ensamble mostrado en la figura siguiente, calcular
a) la matriz global de rigidez  b) los desplazamientos de los nodos 3 y 4  c) las fuerzas
de reacción en los nodos 1 y 2, y  d) las fuerzas en cada elemento. Una fuerza de 5000 lb
es aplicada en el nodo 4 en la dirección $x$, las constantes de rigidez para cada resorte
se muestran en la figura. Los nodos 1 y 2 están fijos.

Los pasos para solucionar el problema utilizando NuSA se resumen en la siguiente
lista:

1. Importar las librerías a utilizar
2. Definir constantes o datos de entrada
1. Crear un modelo del tipo correspondiente
1. Crear nodos y elementos
1. Agregar los nodos y elementos al modelo
1. Indicar las cargas y condiciones de frontera
1. Resolver el modelo
1. Consultar los datos de salida requeridos

Siguiendo la metodología anterior, vamos a ir *desmenuzando* cada uno de los
puntos expuestos.

#### Importar las librerías a utilizar

Se importan los módulos `core`, `model` y `element`, que contienen
todas las clases necesarias para crear y solucionar el modelo de elemento finito.

```
from nusa.core import *
from nusa.model import *
from nusa.element import *
```

#### Definir constantes o datos de entrada

En esta paso se crean variables con datos a utilizar en el resto del procedimiento, las
cuales pueden ser fuerzas nodales aplicadas, desplazamientos preescritos, o bien constantes
mecánicas del material.

Para nuestro ejemplo se definen la fuerza $P$ aplicada en el nodo 4 y las constantes de rigidez
para cada resorte.

```
# Definiendo constantes
P = 5000.0
k1, k2, k3 = 1000, 2000, 3000
```

#### Crear un modelo de tipo correspondiente

Para este caso se crea un modelo instanciando un objeto de la clase `SpringModel`.

```
m1 = SpringModel("2D Model")
```

Como puede notarse, el único argumento de entrada es un nombre para el modelo, mismo que no es necesario.

#### Crear nodos y elementos

```
# Nodos
n1 = Node((0,0))
n2 = Node((0,0))
n3 = Node((0,0))
n4 = Node((0,0))
# Elementos
e1 = Spring((n1,n3),k1)
e2 = Spring((n3,n4),k2)
e3 = Spring((n4,n2),k3)
```

#### Agregar los nodos y elementos al modelo

```
for nd in (n1,n2,n3,n4):
    m1.addNode(nd)
for el in (e1,e2,e3):
    m1.addElement(el)
```

#### Indicar las cargas y condiciones de frontera

```
m1.addForce(n4,(P,))
m1.addConstraint(n1,ux=0)
m1.addConstraint(n2,ux=0)
```

#### Resolver el modelo

```
m1.solve()
```
#### Consultar los datos de salida requeridos

```
# a) Matriz global
print "a) Matriz global:\n {0}".format(m1.KG)
# b) Desplazamiento en los nodos 3 y 4
print "\nb) Desplazamientos de nodos 3 y 4"
print "UX3: {0}".format(n3.ux)
print "UX4: {0}".format(n4.ux)
# c) Fuerzas de reacción en los nodos 1 y 2
print "\nc) Fuerzas nodales en 1 y 2"
print "FX1: {0}".format(n1.fx)
print "FX2: {0}".format(n2.fx)
# d) Fuerzas en cada resorte
print "\nd) Fuerzas en elementos"
print "FE1:\n {0}".format(e1.fx)
print "FE2:\n {0}".format(e2.fx)
print "FE3:\n {0}".format(e3.fx)
```

Podemos, entonces, ejecutar el script resultante.

```python
from nusa import *

"""
Logan, D. (2007). A first course in the finite element analysis.
Example 2.1, pp. 42.
"""
P = 5000.0
k1, k2, k3 = 1000, 2000, 3000
# Model
m1 = SpringModel("2D Model")
# Nodes
n1 = Node((0,0))
n2 = Node((0,0))
n3 = Node((0,0))
n4 = Node((0,0))
# Elements
e1 = Spring((n1,n3),k1)
e2 = Spring((n3,n4),k2)
e3 = Spring((n4,n2),k3)

# Add elements
for nd in (n1,n2,n3,n4):
    m1.add_node(nd)
for el in (e1,e2,e3):
    m1.add_element(el)

m1.add_force(n4,(P,))
m1.add_constraint(n1,ux=0)
m1.add_constraint(n2,ux=0)
m1.solve()

# a) Matriz global
print("a) Matriz global:\n {0}".format(m1.KG))
# b) Desplazamiento en los nodos 3 y 4
print("\nb) Desplazamientos de nodos 3 y 4")
print("UX3: {0}".format(n3.ux))
print("UX4: {0}".format(n4.ux))
# c) Fuerzas de reacción en los nodos 1 y 2
print("\nc) Fuerzas nodales en 1 y 2")
print("FX1: {0}".format(n1.fx))
print("FX2: {0}".format(n2.fx))
# d) Fuerzas en cada resorte
print("\nd) Fuerzas en elementos")
print("FE1:\n {0}".format(e1.fx))
print("FE2:\n {0}".format(e2.fx))
print("FE3:\n {0}".format(e3.fx))
```

    a) Matriz global:
     [[ 1000.     0. -1000.     0.]
     [    0.  3000.     0. -3000.]
     [-1000.     0.  3000. -2000.]
     [    0. -3000. -2000.  5000.]]

    b) Desplazamientos de nodos 3 y 4
    UX3: 0.9090909090909092
    UX4: 1.3636363636363638

    c) Fuerzas nodales en 1 y 2
    FX1: -909.0909090909091
    FX2: -4090.9090909090914

    d) Fuerzas en elementos
    FE1:
     [[-909.09090909]
     [ 909.09090909]]
    FE2:
     [[-909.09090909]
     [ 909.09090909]]
    FE3:
     [[ 4090.90909091]
     [-4090.90909091]]

```python

```


# Lecture 18: Numerical Solutions to the Diffusion Equation
## (Implicit Methods)

### Sections

* [Introduction](#Introduction)
* [Learning Goals](#Learning-Goals)
* [On Your Own](#On-Your-Own)
* [In Class](#In-Class)
    * [Revisiting the Discrete Version of Fick's Law](#Revisiting-the-Discrete-Version-of-Fick's-Law)
    * [A Linear System for Diffusion](#A-Linear-System-for-Diffusion)
    * [An Implicit Numerical Solution](#An-Implicit-Numerical-Solution)
    * [Deconstruction of the Solution Scheme](#Deconstruction-of-the-Solution-Scheme)
* [Homework](#Homework)
* [Summary](#Summary)
* [Looking Ahead](#Looking-Ahead)
* [Reading Assignments and Practice](#Reading-Assignments-and-Practice)

Possible future improvement:  Crank-Nicholson.

### Introduction
----

This lecture introduces the implicit scheme for solving the diffusion equation. Examine the descritization for our explicit scheme that we covered in the previous lecture:

$$
\frac{u_{i,\, j+1} - u_{i,\, j}}{\Delta t} = D \frac{u_{i - 1,\, j} - 2 u_{i,\, j} + u_{i + 1,\, j}}{\Delta x^2}
$$

This expression uses a _forward difference_ in time where we are subtracting the value of our dependent variable at time index $j$ from the value of our dependent variable at time-index $j+1$.  If, instead, we perform a backward difference (replace $j$ with $j-1$) we will be subtracting our dependent variable at $j-1$ from the value at the index $j$.  For example:

$$
\frac{u_{i,\, j} - u_{i,\, j-1}}{\Delta t} = D \frac{u_{i - 1,\, j} - 2 u_{i,\, j} + u_{i + 1,\, j}}{\Delta x^2}
$$

Attempting to repeat our previous algebraic manipulations we find that the solution to this equation is in terms of three unknown quantities at the index $j$.  These quantities depend on indices $i-1$, $i$ and $i+1$ and our solution is only known to the index $j-1$.  This seems like a complication that cannot be resolved however, examination of all the resulting equations in our grid will revel that this is a linear system that can be solved with the inclusion of the boundary conditions.

The point of this lecture is to develop your understanding for how the use of matrices and linear algebra can be used to solve this problem.  The system of equations and the method for solving the equations is known as an "implicit method".  There is a good discussion in Numerical Recipes by Teukolsky, et al. to provide a foundation for these methods.

[Top of Page](#Sections)

### Learning Goals
----

* Re-develop the descretizaton of Fick's law such that the solution scheme is implicit
* Write the method as a linear system
* Incorporate boundary conditions
* Develop a solution strategy using linear algebra and Numpy or SciPy as appropriate.

[Top of Page](#Sections)

### On Your Own
----

Suggestions for improvement of this section:

* Develop numpy methods for linear algebra (e.g. creating matrices efficiently)
* Matrix operations relevant to LA.
* Solve a simple linear system.

[Top of Page](#Sections)

### In Class
----

* Re-derive the discrete form of Fick's law.
* Examine the structure of the resulting matrix.
* Write a solver.

### Revisiting the Discrete Version of Fick's Law

We start with a re-statement of Fick's second law in finite difference form that uses a FORWARD difference in time:

$$
\frac{u_{i,\, j+1} - u_{i,\, j}}{\Delta t} = D \frac{u_{i - 1,\, j} - 2 u_{i,\, j} + u_{i + 1,\, j}}{\Delta x^2}
$$

This choice of time differencing led to the EXPLICIT scheme.  This time, we choose a BACKWARD difference in time as follows:

$$
\frac{u_{i,\, j} - u_{i,\, j-1}}{\Delta t} = D \frac{u_{i - 1,\, j} - 2 u_{i,\, j} + u_{i + 1,\, j}}{\Delta x^2}
$$

This choice leads to a set of linear equations.  To illustrate how this develops we will write the equation above for a grid of eight points that represent the quantity of diffusing substance.  See the next figure.

In the above figure we represent a grid of two dimensions - this grid is identical to the explicit finite difference grid.  The main difference between the explicit and implicit method is the way we fill the grid to arrive at our solution.  In the spatial dimension we have 8 columns (the "$i$" index) and in the time dimension we show only three rows (the "$j$" index).  The sizes of the grid in the two dimensions are arbitrary.  Keep the following in mind:

* The solution is known to the $j-1$ index.
* The unknowns are the $j$ indcies.

Algebraiclly rearranging this differencing scheme, we can write down:

$$
u_{i,\, j-1} = \frac{\Delta t D}{\Delta x^2} \left( - u_{i - 1,\, j} + 2 u_{i,\, j} - u_{i + 1,\, j} \right) + u_{i,\, j}
$$

one additional re-arrangment (substitute $\beta$ for the factor containing the diffusion coefficient) and we get:

$$
- \beta u_{i - 1,\, j} + (1 + 2 \beta) u_{i,\, j} - \beta u_{i + 1,\, j} = u_{i,\, j-1}
$$

We include "ghost cells" in grey above to enforce the boundary conditions.  We can use fixed value (setting the ghost cells to a particular number) or fixed flux (setting the value of the ghost cell based on a pair of interior cells) to produce a linear system with an equal number of unknowns and equations.

### A Linear System for Diffusion

We begin as usual by importing SymPy into the namespace and using `init_session` to define some helpful symbols.  We also define a pair of symbols $U_{LHS}$ and $U_{RHS}$ that will be used to define values in the ghost cells.

```python
import sympy as sp
sp.init_session(quiet=True)
var('U_LHS U_RHS')
```

We define the symbols we want to use in our linear system.  For this demonstration, I don't add the time index but I keep my subscripts consistent with the figure above.

```python
var('dt dx beta u1:7 b1:7')
```

In this cell we create the square matrix holding the coefficients that multiply the unknown quantities.  Note the structure of the matrix.  It is a _tridiagonal_ matrix.  The function in NumPy is very compact, in SymPy not so much.  So I apologize for the syntax in the SymPy/Python code below, but the more compact version can be difficult to read:

```python
hpad = ones(0, 1); vpad = ones(1, 0)
mainDiag = 2*beta+1; offDiag = -beta

M = (sp.diag(vpad, offDiag, offDiag, offDiag, offDiag, offDiag, hpad)+ \
     sp.diag(hpad, offDiag, offDiag, offDiag, offDiag, offDiag, vpad)+ \
     sp.diag(mainDiag,mainDiag,mainDiag,mainDiag,mainDiag,mainDiag))

M
```

Here is our vector of unknown quantities.  We know the solution to the $j-1$ time step.  All of these symbols represent the value of our field (e.g. concentration, temperature, etc.) at the $j$'th time step.

```python
xmatrix = sp.Matrix([u1,u2,u3,u4,u5,u6])
xmatrix
```

If we've got everything correct, this matrix product will reproduce the discrete diffusion equation outlined above.  You'll note that the boundary equations are not formed correctly.  For reference, here is the discrete form:

$$
- \beta u_{i - 1,\, j} + (1 + 2 \beta) u_{i,\, j} - \beta u_{i + 1,\, j} = u_{i,\, j-1}
$$

```python
M*xmatrix
```

It should start to become clear that we can write this linear system (of a tridiagonal matrix and a column vector of unknowns) as a matrix equation:

$$
M \cdot \overline{x} = \overline{b}
$$

Where M is the square matrix, x is the vector of unknown quantities and b is the last known value of the system variables (the $u_{i,j}$ are the unknowns, the $j-1$ are the last known values).  There is still some work to be done before we can use linear algebra to get the solution.  We need to implement the boundary conditions.

### Fixed Value Boundary Conditions

Start with the form at the interior of the grid:

$$
- \beta u_{i - 1,\, j} + (1 + 2 \beta) u_{i,\, j} - \beta u_{i + 1,\, j} = u_{i,\, j-1}
$$

To get the form correct at the top and bottom of this solution vector we need to imagine adding "ghost cells" to the boundaries of our domain at $i=0$ and $i=7$.  Using the above expression, let $i = 1$:

$$
- \beta u_{0,\, j} + (1 + 2 \beta) u_{1,\, j} - \beta u_{2,\, j} = u_{1,\, j-1}
$$

If we have fixed value boundary conditions, we then know the value of $u_0$.  This is the boundary condition of our simulation.  We will call this value $U_{LHS}$, substitute $U_{LHS} = u_0$ and move the known quantities to the RHS of the equation:

$$
(1 + 2 \beta) u_{1,\, j} - \beta u_{2,\, j} = u_{1,\, j-1} + \beta U_{LHS}
$$

### Fixed Flux Boundary Conditions

If we have fixed flux boundary conditions we can write the flux as a central difference on the cell $u_1$ that uses the "ghost" point at $u_0$:

$$
\frac{u_{2,\, j} - u_{0,\, j}}{2 \Delta x} = F
$$

Proceeding as before with $i=1$:

$$
- \beta u_{0,\, j} + (1 + 2 \beta) u_{1,\, j} - \beta u_{2,\, j} = u_{1,\, j-1}
$$

This time we know the relationship of $u_0$ to the other unknowns due to the specification of the defined flux boundary condition.  Solving for $u_0$ we get:

$$
u_{0,\, j} = u_{2,\, j} - {2 \Delta x} F
$$

Substituting this into our expression that includes the ghost cell gives us:

$$
- \beta (u_{2,\, j} - {2 \Delta x} F) + (1 + 2 \beta) u_{1,\, j} - \beta u_{2,\, j} = u_{1,\, j-1}
$$

Simplifying:

$$
(1 + 2 \beta) u_{1,\, j} - 2 \beta u_{2,\, j} = u_{1,\, j-1} - \beta 2 \Delta x F
$$

So in this case we have to modify the matrix $M$ entries AND the solution vector $b$ recalling that the $j-1$ index is the known solution.

We have now recovered the form of the equation in the dot product $M \cdot x$ and the form of this equation is telling us that we need to modify the solution vector $b$ with information about the boundary conditions before we find the inverse of the matrix and compute the new solution vector.

Modifying the $b$ matrix with the known ghost cell values for the fixed value boundary conditions we get:

```python
bmatrix = sp.Matrix([(b1+beta*U_LHS),b2,b3,b4,b5,(b6+beta*U_RHS)])
bmatrix
```

So the full form of our system is therefore:

$$
\left[\begin{matrix}2 \beta + 1 & - \beta & 0 & 0 & 0 & 0\\- \beta & 2 \beta + 1 & - \beta & 0 & 0 & 0\\0 & - \beta & 2 \beta + 1 & - \beta & 0 & 0\\0 & 0 & - \beta & 2 \beta + 1 & - \beta & 0\\0 & 0 & 0 & - \beta & 2 \beta + 1 & - \beta\\0 & 0 & 0 & 0 & - \beta & 2 \beta + 1\end{matrix}\right] \cdot \left[\begin{matrix}u_{1}\\u_{2}\\u_{3}\\u_{4}\\u_{5}\\u_{6}\end{matrix}\right] = \left[\begin{matrix}U_{LHS} \beta + b_{1}\\b_{2}\\b_{3}\\b_{4}\\b_{5}\\U_{RHS} \beta + b_{6}\end{matrix}\right]
$$

`SymPy` can evaluate the LHS for us.

```python
sp.Eq(M*xmatrix,bmatrix)
```

All that remains is to solve the above linear system.  Instead of using `SymPy`, we will use some tools in a different Python library.

[Top of Page](#Sections)

### An Implicit Numerical Solution

General setup in this section:

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
```

Simulation parameters:

```python
numberOfPoints = 100
lengthOfDomain = 1.0
dx = lengthOfDomain/numberOfPoints
xPoints = np.linspace(0.0, lengthOfDomain, numberOfPoints)
initialCondition = np.sin(xPoints*np.pi/lengthOfDomain)
```

A simple function to plot the initial condition:

```python
def plotIC():
    fig = plt.figure()
    axes = fig.add_axes([0.1, 0.1, 0.8, 0.8])
    axes.plot(xPoints, initialCondition, 'ro')
    axes.set_xlabel('Distance $x$')
    axes.set_ylabel('Concentration of Stuff $c(x,t)$')
    axes.set_title('Initial Conditions');
```

```python
plotIC()
```

It is worth noting that these schemes are unconditionally stable - so any choice of time step will produce a solution.  The accuracy of the solution does depend on this choice, though.

```python
diffusionCoefficient = 10.0
dt = dx**2/(diffusionCoefficient)
numberOfIterations = 1000
```

We create two solution vectors rather than one whole array to hold all of our solution.  This is not particular to the implicit method, but it demonstrates another technique for saving memory and speeding up the calculation.  We will fill these matrices and swap them (move data from `new` into `old` and overwrite `new`) at each time step.

```python
newConcentration = np.zeros((numberOfPoints), dtype='float32')
oldConcentration = np.zeros((numberOfPoints), dtype='float32')
```

First, some syntax:

```python
['h','h','h']*3
```

The matrix has to be square.  It should have the same dimensions as the nubmer of points in the system.  The following code snippet was inspired by [this](http://stackoverflow.com/questions/5842903/block-tridiagonal-matrix-python) post.

```python
def tridiag(a, b, c, k1=-1, k2=0, k3=1):
    # Here we use Numpy addition to make the job easier.
    return np.diag(a, k1) + np.diag(b, k2) + np.diag(c, k3)

a = [-dt*diffusionCoefficient/dx/dx]*(numberOfPoints-1)
b = [2*dt*diffusionCoefficient/dx/dx+1]*(numberOfPoints)
c = [-dt*diffusionCoefficient/dx/dx]*(numberOfPoints-1)

A = tridiag(a, b, c)
```

```python
A
```

We first need to prime the arrays by copying the initial condition into `oldConcentration`.  Afterwards it will be enough to swap pointers (a variable that points to a memory location).

```python
np.copyto(oldConcentration,initialCondition)
```

[Top of Page](#Sections)

### Deconstruction of the Solution Scheme

In spite of the small chunk of code a few cells below, there is a lot going on.  Let us dissect it.  In bullet points:

* Before the first solution step we enforce the boundary conditions.  Our choice of matrix means that we are using "fixed value" boundary conditions.  So we need to modify the `b` vector accordingly.  The indexing notation of Numpy that permits us to find the first (`[0]`) and last cell (`[-1]`) of an array is very helpful here.

```python
    oldConcentration[0] = oldConcentration[0] + uLHS*dt*diffusionCoefficient/dx/dx
    oldConcentration[-1] = oldConcentration[-1] + uRHS*dt*diffusionCoefficient/dx/dx

```

Recall:

$$
\left[\begin{matrix}2 \beta + 1 & - \beta & 0 & 0 & 0 & 0\\- \beta & 2 \beta + 1 & - \beta & 0 & 0 & 0\\0 & - \beta & 2 \beta + 1 & - \beta & 0 & 0\\0 & 0 & - \beta & 2 \beta + 1 & - \beta & 0\\0 & 0 & 0 & - \beta & 2 \beta + 1 & - \beta\\0 & 0 & 0 & 0 & - \beta & 2 \beta + 1\end{matrix}\right] \cdot \left[\begin{matrix}u_{1}\\u_{2}\\u_{3}\\u_{4}\\u_{5}\\u_{6}\end{matrix}\right] = \left[\begin{matrix}U_{LHS} \beta + b_{1}\\b_{2}\\b_{3}\\b_{4}\\b_{5}\\U_{RHS} \beta + b_{6}\end{matrix}\right]
$$

* Solving the system involves using the built in `NumPy` functions to invert the matrix.  What is returned is the solution vector.  Please note that I'm using an internal `Numpy` (an optimized function!) function to COPY the results of the linear algebra solution into the `newConcentration` vector.

```python
    np.copyto(newConcentration,np.linalg.solve(A,oldConcentration))
```

* Rather than storing ALL the data, we instead store just the current and the old concentrations.  There are efficiencies in doing this, but if we want the older values, we need to store them on disk or in memory.
* Tuple unpacking in Python leads to the `A,B=B,A` syntax below.  This switches the references to the arrays.  This is important for efficiency - you don't want to move any data if you don't have to.  If you are running big calculations then moving that data around is a waste of time/resources.  Better to just swap references.

```python
    oldConcentration, newConcentration = newConcentration, oldConcentration
```

* Repeat the process and after a specified number of iterations, plot the results.

```python
uLHS = 0.0
uRHS = 0.0
numIterations = 200

for i in range(numIterations):
    # enforce boundary conditions
    oldConcentration[0] = oldConcentration[0] + uLHS*dt*diffusionCoefficient/dx/dx
    oldConcentration[-1] = oldConcentration[-1] + uRHS*dt*diffusionCoefficient/dx/dx
    # solve the system
    np.copyto(newConcentration,np.linalg.solve(A,oldConcentration))
    # swap pointers
    oldConcentration, newConcentration = newConcentration, oldConcentration

# plot the results
fig2 = plt.figure()
axes = fig2.add_axes([0.1, 0.1, 0.8, 0.8])
axes.plot(xPoints, newConcentration, 'ro')
axes.set_ylim(0,1)
axes.set_xlabel('Distance $x$')
axes.set_ylabel('Concentration of Stuff $c(x,t)$')
axes.set_title('Solution');
```

[Top of Page](#Sections)

### Homework
----

* Solve the diffusion couple problem
* Compare to the analytical solution
* Describe the differences between them (in words and with some plots, maybe)
* Examine the error in terms of truncation versus roundoff error.

[Top of Page](#Sections)

### Looking Ahead
----

TBA

[Top of Page](#Sections)

### Reading Assignments and Practice
----

TBA

[Top of Page](#Sections)