# 13. Уравнение фильтрации и его решения - построение с использованием python и sympy

Материалы курсов "Исследования скважин и пластов" и "Гидродинамические исследования скважин" в РГУ нефти и газа имени И.М.Губкина.

версия 0.1 от 08.11.2021

Хабибуллин Ринат 2021 г.

---

[Открыть на google colab](https://colab.research.google.com/github/khabibullinra/welltest_examples/blob/master/jupyter/13_sympy_solutions_manipulations.ipynb)    (требуется логин google для работы)

Системы компьютерной алгебры могут заметно облегчить работу со сложными математическими выражениями, в частности в решениями уравнения фильтрации. В данной тетрадки соберем примеры работы с системой [sympy с использованием python](https://www.sympy.org/en/index.html). Эта система аналогична коммерческим системам [Wolfram Mathematica](https://www.wolfram.com/mathematica/) или [Maple](https://www.maplesoft.com/products/Maple/), и хотя не обладает столь же широкими возможностями, зато открыта и бесплатна и во многом способна помочь решать инженерные задачи.

Для установки системы надо выполнить команду `pip install sympy`

```python
# чтобы работало на колабе надо установить пакет для численного обратного преобразования Лапласа
!pip install anaflow
```

    Requirement already satisfied: anaflow in c:\rnt\welltest_examples\env\lib\site-packages (1.0.1)
    Requirement already satisfied: pentapy>=1.1.0 in c:\rnt\welltest_examples\env\lib\site-packages (from anaflow) (1.1.2)
    Requirement already satisfied: scipy>=1.1.0 in c:\rnt\welltest_examples\env\lib\site-packages (from anaflow) (1.7.1)
    Requirement already satisfied: numpy>=1.14.5 in c:\rnt\welltest_examples\env\lib\site-packages (from anaflow) (1.20.3)

```python
import sympy as sp

# для построения численных решений можно использовать два варианта - mpmath и numpy в связке с scipy
import numpy as np
import mpmath as mp
# для построения графиков когда надо, используем matplotlib
import matplotlib.pyplot as plt
# для численного обратного преобразования Лапласа с использованием numpy и scipy функций
import anaflow
```

# Содержание

* [13.1 Уравнение фильтрации](#13.1)
* [13.2 Общее решение модифицированного уравнения Бесселя](#13.2)
* [13.3 Частное решение для бесконечного пласта и конечного радиуса скважины](#13.3)
* [13.4 Построение численных решений на основе аналитического  с использованием `mpmath`](#13.4)
* [13.5 Построение численных решений на основе аналитического  с использованием `numpy` и `scipy`](#13.5)
* [13.6 Частное решение для конечного круглого пласта с постоянным давлением на контуре и конечного радиуса скважины.](#13.6)
    * [13.6.1. Построение решения с использованием `sympy.dsolve`](#13.6.1)
* [13.7 Частное решение для конечного круглого пласта с замкнутой границей и конечного радиуса скважины.](#13.7)

## 13.1 Уравнение фильтрации

Рассматривается уравнение фильтрации в безразмерных переменных

$$
\frac{\partial p_D}{ \partial t_D} = \frac{1}{r_D}\left[ \frac{ \partial{}}{ \partial{r_D} }\left( r_D \dfrac{\partial p_D}{ \partial r_D} \right) \right]
\tag{13.1.1}
$$

где введены следующие безразмерные переменные
* $r_D$ - безразмерное расстояние от центра скважины
* $t_D$ - безразмерное время
* $p_D$ - безразмерное давление

Соответствующие определения безразмерных переменных

$$ r_D = \frac{r}{r_w} \tag{13.1.2}$$
$$ t_D = \frac{0.00036 kt}{\phi \mu c_t r_w^2}  \tag{13.1.3}$$
$$ p_D = \frac{kh}{ 18.41 q_s B \mu} \left( p_i - p \right)  \tag{13.1.4} $$

где в свою очередь
* $q_s$ - дебит скважины на поверхности, приведенный к нормальным условиям, м3/сут
* $\phi$ - пористость, доли единиц
* $\mu$ - вязкость нефти в пласте, сП
* $B$ - объемный коэффициент нефти, м3/м3
* $p_i$ - начальное давление в пласте, атм
* $p$ - давление забойное, атм
* $c_t$ - общая сжимаемость системы в пласте, 1/атм
* $k$ - проницаемость, мД
* $t$ -  время, час
* $r$ - расстояние от центра скважины, м
* $r_w$ - радиус скважины, м

Решение такого уравнение может быть получено с использованием преобразования Лапласа.

$$
L \left [ f(t) \right] = \tilde{f}(u) = \int_{0}^{\infty}f(t)e^{-ut}dt
\tag{13.1.4}
$$

где $u$ параметр пространства Лапласа соответствующий времени.

Тогда уравнение в пространстве Лапласа преобразуется к виду:

$$
u \tilde{p}_D  =  \dfrac{1}{r_D} \left[\dfrac{d}{d r_D} \left(r_D \dfrac{d{\tilde{p}_D}}{d r_D} \right) \right]
\tag{13.1.5}
\label{13.1.5}
$$

Уравнение \eqref{13.1.5} известно как   модифицированное уравнение Бесселя .
где
* $u$ - переменная пространства Лапласа, соответствующая времени
* $\tilde{p}_D(u, r_D)$ - изображение давления в пространстве Лапласа

С уравнением \eqref{13.1.5} можно начать работать с использованием системы компьютерной алгебры [`sympy`](https://www.sympy.org/en/index.html).

```python
# определим переменные с которыми будем работать

r_d = sp.symbols('r_d', real=True, positive=True)
p_d = sp.symbols('p_d', cls=sp.Function, real=True, positive=True)
u = sp.symbols('u',real=True, positive=True)
C1, C2 = sp.symbols('C1 C2')
```

```python
# определим уравнение фильтрации в пространстве Лапласа
diff_eq = sp.Eq(u * p_d(r_d),
                1 / r_d * (sp.diff(r_d * sp.diff(p_d(r_d), r_d) , r_d)) )
diff_eq
```

$\displaystyle u \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{r_{d} \frac{d^{2}}{d r_{d}^{2}} \operatorname{p_{d}}{\left(r_{d} \right)} + \frac{d}{d r_{d}} \operatorname{p_{d}}{\left(r_{d} \right)}}{r_{d}}$

## 13.2 Общее решение модифицированного уравнения Бесселя

```python
# попробуем решить уравнение
soln = sp.dsolve(diff_eq, p_d(r_d))
soln
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = C_{1} I_{0}\left(r_{d} \sqrt{u}\right) + C_{2} Y_{0}\left(i r_{d} \sqrt{u}\right)$

```python
# проверим, что решение полученное dsolve удовлетворяет уравнению
sp.checkodesol(diff_eq, soln)
```

    (True, 0)

Уравнение Бесселя легко решается,
$$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = C_{1} I_{0}\left(r_{d} \sqrt{u}\right) + C_{2} Y_{0}\left(i r_{d} \sqrt{u}\right) \tag{13.2.1}
\label{13.2.1}$$

хотя решение \eqref{13.2.1} по виду и немного отличается от того, что обычно приводится в книгах. Вместо мофицированной функции Бесселя второго рода $K_0(x)$ решение выражается через функцию Бесселя второго рода $Y_0(ix)$ для мнимого арумента.

[Известны выражения](https://en.wikipedia.org/wiki/Bessel_function#Modified_Bessel_functions:_I%CE%B1,_K%CE%B1) связывающие функции Бесселя первого $J_\alpha(iz)$ и второго рода $Y_\alpha(iz)$ для мнимых аргументов с модифицированными функциями Бесселя первого $I_\alpha(z)$  и второго рода $K_\alpha(z)$.

$$ J_\alpha(iz) = e^{\frac{\alpha\pi i}{2}} I_\alpha(z) \tag{13.2.2} $$
$$ Y_\alpha(iz) = e^{\frac{(\alpha+1)\pi i}{2}}I_\alpha(z) - \frac{2}{\pi}e^{-\frac{\alpha\pi i}{2}}K_\alpha(z) \tag{13.2.3} \label{13.2.3}$$

преобразуем \eqref{13.2.3}

$$ Y_0(iz) = e^{\frac{\pi i}{2}}I_0(z) - \dfrac{2}{\pi}K_0(z) \tag{13.2.4}$$

Учитывая эти выражения можно убедиться что выражение вида

$$
\tilde{p}_D(u, r_D) = A(u) K_0(r_D \sqrt u) + B(u) I_0(r_D \sqrt u)
\tag{13.2.5}
\label{13.2.5}
$$

которое можно найти в книгах также будет являться решение уравнения фильтрации, что можно проверить командой `sympy.checkodesol`

```python
# зададим в явном виде решение с использованием K_0 и I_0
A, B = sp.symbols('A B')
soln2 = sp.Eq(p_d(r_d) , A * sp.besselk(0, r_d * sp.sqrt(u)) + B * sp.besseli(0, r_d * sp.sqrt(u)))
soln2
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = A K_{0}\left(r_{d} \sqrt{u}\right) + B I_{0}\left(r_{d} \sqrt{u}\right)$

```python
sp.pprint(soln2)
```

    p_d(r_d) = A⋅besselk(0, r_d⋅√u) + B⋅besseli(0, r_d⋅√u)

```python
# проверим, что это решение также удовлетворяет исходному уравнению
sp.checkodesol(diff_eq, soln2)
```

    (True, 0)

Дальше покажем как можно работать с решением (13.2.5) для построения частных решений

## 13.3 Частное решение для бесконечного пласта и конечного радиуса скважины.

Частное решение строится за счет поиска параметров $A$ и $B$ для уравнения (13.2.5) так, чтобы они удовлетворяли граничным условиям.

1. Граничное условия на бесконечности

$$\lim_{r_D \to \infty} p_D(r_D, t_D) = 0  \tag{13.3.1} \label{13.3.1}$$

в пространстве Лапласа  \eqref{13.3.1} преобразуется в следующее

$$\lim_{r_D \to \infty} \tilde{p}_D(r_D, u) = 0 \tag{13.3.2} \label{13.3.2}$$

2. Граничное условие на скважине

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ \partial p_D(r_D, t_D)}{\partial r_D} \right] = -1  \tag{13.3.3} \label{13.3.3}$$

в пространстве Лапласа  \eqref{13.3.3} с учетом выражения (12.9) преобразуется к

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ d \tilde{p}_D}{d r_D} \right] = -\dfrac{1}{u} \tag{13.3.4} \label{13.3.4}$$

Условие на бесконечности трудно записать в `sympy` в явном виде. Но можно проверить пределы выражения (13.2.5) при разных значениях параметров  $A$ и $B$

```python
sp.limit(soln2.rhs, r_d, sp.oo)
```

$\displaystyle \lim_{r_{d} \to \infty}\left(A K_{0}\left(r_{d} \sqrt{u}\right) + B I_{0}\left(r_{d} \sqrt{u}\right)\right)$

Предел при призвольных  $A$ и $B$ не определен.

Положим $A=0$. Это делается командой `.subs(A, 0)` для исследуемого выражения

```python
sp.limit(soln2.rhs.subs(A, 0), r_d, sp.oo)
```

$\displaystyle \lim_{r_{d} \to \infty}\left(B I_{0}\left(r_{d} \sqrt{u}\right)\right)$

Получаем, что этот предел также не определен. Положим $B=0$.

```python
sp.limit(soln2.rhs.subs(B, 0), r_d, sp.oo)
```

$\displaystyle 0$

Получаем, что при $B=0$  выражение $\displaystyle \lim_{r_{d} \to \infty}\left(A K_{0}\left(r_{d} \sqrt{u}\right) + B I_{0}\left(r_{d} \sqrt{u}\right)\right) = 0$

Откуда можно сделать вывод, что для приведенных граничных условий $B=0$.

```python
soln3 = soln2.subs(B, 0)
soln3
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = A K_{0}\left(r_{d} \sqrt{u}\right)$

Найдем констранту $A$ из второго граничного условия (13.3.4)

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ d \tilde{p}_D}{d r_D} \right] = -\dfrac{1}{u} \tag{13.3.4} \label{13.3.4}$$

```python
bc2 = sp.Eq(r_d * soln3.rhs.diff(r_d), -1/u)
bc2
```

$\displaystyle - A r_{d} \sqrt{u} K_{1}\left(r_{d} \sqrt{u}\right) = - \frac{1}{u}$

```python
r_wd = sp.symbols('r_wd', real=True, positive=True)
bc2_sol = sp.solve(bc2.subs(r_d, r_wd), A)
bc2_sol
```

    [1/(r_wd*u**(3/2)*besselk(1, r_wd*sqrt(u)))]

```python
soln4 = soln3.subs(A, bc2_sol[0])
soln4
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{K_{0}\left(r_{d} \sqrt{u}\right)}{r_{wd} u^{\frac{3}{2}} K_{1}\left(r_{wd} \sqrt{u}\right)}$

При $r_{wd} = 1$ получим

```python
soln5 = soln4.subs(r_wd, 1)
soln5
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} K_{1}\left(\sqrt{u}\right)}$

Получили решение аналогичное решению (12.23). При этом часть преобразований при получении решений была сделана с использованием sympy.

$$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} K_{1}\left(\sqrt{u}\right)} \tag{13.3.5} $$

## 13.4 Построение численных решений на основе аналитического  с использованием `mpmath`

Полученное решение можно использовать для отображения. Оно легко копируется в формате LaTeX и встравляется в блоки описания. Его можно использовать для дальнейших аналитических преобразований. Его можно также использовать для построения численных решений.

Покажем как решение можно построить с использованием mpmath и реализации численного обратного преобразования mpmath

```python
# преобразование sympy выражения в lambda функцию с использованием mpmath
soln5_mpmath = sp.lambdify([u, r_d], soln5.rhs, modules = ['mpmath'])

# после преобразования lambda функцию можно использовать для проведения расчетов как обычныую функцию
```

```python
# пример расчета для одного значения, чтобы убедиться, что все работает
soln5_mpmath(u=1, r_d=1)
```

    mpf('0.69948393559377231')

```python
# определим функцию для расчета обратного преобразования Лапласа
# отдельная функция удобно чтобы передавать r_d второй параметр
def soln5_inv_lapl(t_d=1000, r_d=1):
    sol_lap = lambda u: soln5_mpmath(u=u, r_d=r_d)
    return mp.invertlaplace(sol_lap, t_d)
```

```python
# пример расчета безразмерного давления с использованием mpmath
soln5_inv_lapl(t_d=1000, r_d=1)
```

    mpf('3.8605905955786238')

```python
# нарисуем график зависимости безразмерного давления от безразмерного времени
# поскольку считает относительно медленно ограничим количество точек для расчета
mp.plot(soln5_inv_lapl,[1,100], points=20)
```

```python
# нарисуем график зависимости безразмерного давления от безразмерного расстояния
# поскольку считает относительно медленно ограничим количество точек для расчета
td=1000
mp.plot(lambda rd: soln5_inv_lapl(td,rd),[1,100], points=20)
```

## 13.5 Построение численных решений на основе аналитического  с использованием `numpy` и `scipy`

Использованием `mpmath` удобно для проверки расчетов, так как эта библиотека позволяет выбрать разные методы расчета и задать нужную точность. Но расчеты при этом проводятся медленно. Для более быстрых расчетов можно использовать расчетные функции `numpy` и `scipy`, а для обратного преобразования Лапласа библиотеку `anaflow`

```python
# преобразование sympy выражения в lambda функцию с использованием numpy
soln5_numpy = sp.lambdify([u, r_d], soln5.rhs, modules = ['numpy', 'scipy'])

# после преобразования lambda функцию можно использовать для проведения расчетов как обычныую функцию
```

```python
# пример расчета для одного значения, чтобы убедиться, что все работает
soln5_numpy(u=1, r_d=1)
```

    0.6994839355937723

```python
# определим функцию для расчета обратного преобразования Лапласа
# отдельная функция удобно чтобы передавать r_d второй параметр
def soln5_inv_lapl_numpy(t_d=1000, r_d=1):
    sol_lap = anaflow.get_lap_inv(soln5_numpy,  r_d=r_d)
    return sol_lap(t_d)
```

```python
# пример расчета для одного значения, чтобы убедиться, что все работает
soln5_inv_lapl_numpy(1000)
```

    array([3.86059079])

Построим графики с использованием сгенерированного решения. В отличии от `mpmath` решения считает быстрее, поэтому можно построить сразу много графиков.

```python
# зададим диапазоны изменения параметров для построения графиков
td_arr = np.logspace(1,5,100)
rd_arr = np.logspace(1,3,100)

# построим графики
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=[15,5])
for ri in np.linspace(1,200,10):
    ax1.plot(td_arr, soln5_inv_lapl_numpy(td_arr, r_d=ri), label = f'$r_d={ri:.0f}$')
for ti in np.linspace(1e4, 1e7,10):
    ax2.plot(rd_arr, np.squeeze( [soln5_inv_lapl_numpy(ti, r_d=ri) for ri in rd_arr ] ), label = f'$t_d={ti:.0f}$')
# наведем немного красоты
ax1.set_xlabel('$t_d$')
ax1.set_ylabel('$p_d$')
ax1.legend()
ax2.set_xlabel('$r_d$')
ax2.set_ylabel('$p_d$')
ax2.legend()
plt.show()
```

## 13.6 Частное решение для конечного круглого пласта с постоянным давлением на контуре и конечного радиуса скважины.

Частное решение строится за счет поиска параметров $A$ и $B$ для уравнения (13.2.5) так, чтобы они удовлетворяли граничным условиям.

1. Граничное условия на внешнем контуре с постоянным давлением

$$ p_D(r_e, t_D) = 0  \tag{13.6.1} \label{13.6.1}$$

в пространстве Лапласа  \eqref{13.6.1} преобразуется в следующее

$$ \tilde{p}_D(r_e, u) = 0 \tag{13.6.2} \label{13.6.2}$$

2. Граничное условие на скважине

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ \partial p_D(r_D, t_D)}{\partial r_D} \right] = -1  \tag{13.6.3} \label{13.6.3}$$

в пространстве Лапласа  \eqref{13.6.3} с учетом выражения (12.9) преобразуется к

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ d \tilde{p}_D}{d r_D} \right] = -\dfrac{1}{u} \tag{13.6.4} \label{13.6.4}$$

```python
soln2
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = A K_{0}\left(r_{d} \sqrt{u}\right) + B I_{0}\left(r_{d} \sqrt{u}\right)$

```python
r_e = sp.symbols('r_e', positive=True)
eq1 = sp.Eq(soln2.subs(r_d, r_e).rhs,0)
eq1
```

$\displaystyle A K_{0}\left(r_{e} \sqrt{u}\right) + B I_{0}\left(r_{e} \sqrt{u}\right) = 0$

```python
eq2 = sp.Eq(r_d * soln2.rhs.diff(r_d), -1/u).subs(r_d, 1)
eq2
```

$\displaystyle - A \sqrt{u} K_{1}\left(\sqrt{u}\right) + B \sqrt{u} I_{1}\left(\sqrt{u}\right) = - \frac{1}{u}$

```python
sol_AB = sp.solve([eq1,eq2], [A,B])
sol_AB
```

    {A: besseli(0, r_e*sqrt(u))/(u**(3/2)*besseli(0, r_e*sqrt(u))*besselk(1, sqrt(u)) + u**(3/2)*besseli(1, sqrt(u))*besselk(0, r_e*sqrt(u))),
     B: -besselk(0, r_e*sqrt(u))/(u**(3/2)*besseli(0, r_e*sqrt(u))*besselk(1, sqrt(u)) + u**(3/2)*besseli(1, sqrt(u))*besselk(0, r_e*sqrt(u)))}

```python
sol_re_const_p = soln2.subs(A, sol_AB[A]).subs(B, sol_AB[B]).simplify()
sol_re_const_p
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{- I_{0}\left(r_{d} \sqrt{u}\right) K_{0}\left(r_{e} \sqrt{u}\right) + I_{0}\left(r_{e} \sqrt{u}\right) K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} \left(I_{0}\left(r_{e} \sqrt{u}\right) K_{1}\left(\sqrt{u}\right) + I_{1}\left(\sqrt{u}\right) K_{0}\left(r_{e} \sqrt{u}\right)\right)}$

Таким образом получено решение для постоянного давления на круговой границе вокруг скважины
$$
\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{- I_{0}\left(r_{d} \sqrt{u}\right) K_{0}\left(r_{e} \sqrt{u}\right) + I_{0}\left(r_{e} \sqrt{u}\right) K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} \left(I_{0}\left(r_{e} \sqrt{u}\right) K_{1}\left(\sqrt{u}\right) + I_{1}\left(\sqrt{u}\right) K_{0}\left(r_{e} \sqrt{u}\right)\right)} \tag{13.6.5}
$$

Ипользуя подход показанный выше можно построить графики зависимости безразмерного давления от времени и расстояния для этого решения с использованием numpy и scipy функций

```python
# преобразование sympy выражения в lambda функцию с использованием numpy
soln_re_const_p_numpy = sp.lambdify([u, r_d, r_e], sol_re_const_p.rhs, modules = ['numpy', 'scipy'])

# после преобразования lambda функцию можно использовать для проведения расчетов как обычныую функцию

# определим функцию для расчета обратного преобразования Лапласа
# отдельная функция удобно чтобы передавать r_d второй параметр
def soln_re_const_p_inv_lapl_numpy(t_d=1000, r_d=1, r_e=100):
    sol_lap = anaflow.get_lap_inv(soln_re_const_p_numpy,  r_d=r_d, r_e=r_e)
    return sol_lap(t_d)

# зададим диапазоны изменения параметров для построения графиков
td_arr = np.logspace(1,5,100)
rd_arr = np.logspace(1,2,100)
re=100
# построим графики
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=[15,5])
for ri in np.linspace(1,100,10):
    ax1.plot(td_arr, soln_re_const_p_inv_lapl_numpy(td_arr, r_d=ri, r_e=re), label = f'$r_d={ri:.0f}$')
for ti in np.linspace(1e2, 1e4,10):
    ax2.plot(rd_arr, np.squeeze( [soln_re_const_p_inv_lapl_numpy(ti, r_d=ri, r_e=re) for ri in rd_arr ] ), label = f'$t_d={ti:.0f}$')
# наведем немного красоты
ax1.set_xlabel('$t_d$')
ax1.set_ylabel('$p_d$')
ax1.legend()
ax2.set_xlabel('$r_d$')
ax2.set_ylabel('$p_d$')
ax2.legend()
plt.show()
```

### 13.6.1. Построение решения с использованием `sympy.dsolve`

Отметим, что в некоторых случаях можно построить частные решения для ОДУ напряму с использованием `sympy`

Вспомним с каким уравнением мы работаем

```python
diff_eq
```

$\displaystyle u \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{r_{d} \frac{d^{2}}{d r_{d}^{2}} \operatorname{p_{d}}{\left(r_{d} \right)} + \frac{d}{d r_{d}} \operatorname{p_{d}}{\left(r_{d} \right)}}{r_{d}}$

Построим снова общее решение, чтобы убедиться, что на текущем этапе все работает

```python
soln = sp.dsolve(diff_eq, p_d(r_d))
soln
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = C_{1} I_{0}\left(r_{d} \sqrt{u}\right) + C_{2} Y_{0}\left(i r_{d} \sqrt{u}\right)$

Построим частное решение сразу задав в `sympy.dsolve` граничные условия на искомую функцию

```python
soln_sympy = sp.dsolve(diff_eq, p_d(r_d),
                       ics={p_d(r_d).subs(r_d, r_e):0,
                            p_d(r_d).diff(r_d).subs(r_d,1):-1/u}).simplify()
soln_sympy
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{i \left(I_{0}\left(r_{d} \sqrt{u}\right) Y_{0}\left(i r_{e} \sqrt{u}\right) - I_{0}\left(r_{e} \sqrt{u}\right) Y_{0}\left(i r_{d} \sqrt{u}\right)\right)}{u^{\frac{3}{2}} \left(I_{0}\left(r_{e} \sqrt{u}\right) Y_{1}\left(i \sqrt{u}\right) - i I_{1}\left(\sqrt{u}\right) Y_{0}\left(i r_{e} \sqrt{u}\right)\right)}$

Решение построено, хотя оно выражается через $Y_0$ вместо $K_0$ в решении (13.6.5) и в нем в явном виде присутствует $i$ мнимая единица.

Это решение также можно использовать для проведения расчетов и построения графиков. Для того, чтобы подавить предупреждения об игнорировании мнимой части - при проведении обратного преобразования Лапласа оставим только реальную часть в явном виде.

Также ограничим диапазон отображения графика зависимости давления от времени - чтобы не показывать неверные значения получаемых в ходе расчета для малых времен.

```python
# преобразование sympy выражения в lambda функцию с использованием numpy
soln_re_const_p_numpy = sp.lambdify([u, r_d, r_e], soln_sympy.rhs, modules = ['numpy', 'scipy'])

# после преобразования lambda функцию можно использовать для проведения расчетов как обычныую функцию

# определим функцию для расчета обратного преобразования Лапласа
# отдельная функция удобно чтобы передавать r_d второй параметр
def soln_re_const_p_inv_lapl_numpy(t_d=1000, r_d=1, r_e=100):
    sol_lap = anaflow.get_lap_inv(soln_re_const_p_numpy,  r_d=r_d, r_e=r_e)
    return sol_lap(t_d).real

# зададим диапазоны изменения параметров для построения графиков
td_arr = np.logspace(1,5,100)
rd_arr = np.logspace(1,2,100)
re=100
# построим графики
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=[15,5])
for ri in np.linspace(1,100,10):
    ax1.plot(td_arr, soln_re_const_p_inv_lapl_numpy(td_arr, r_d=ri, r_e=re), label = f'$r_d={ri:.0f}$')
for ti in np.linspace(1e2, 1e4,10):
    ax2.plot(rd_arr, np.squeeze( [soln_re_const_p_inv_lapl_numpy(ti, r_d=ri, r_e=re) for ri in rd_arr ] ), label = f'$t_d={ti:.0f}$')
# наведем немного красоты
ax1.set_xlabel('$t_d$')
ax1.set_ylabel('$p_d$')
ax1.set_ylim(0,5)
ax1.legend()
ax2.set_xlabel('$r_d$')
ax2.set_ylabel('$p_d$')
ax2.legend()
plt.show()
```

Графики построились, хотя и пришлось подавлять предупреждения об игнорировании мнимой части и для одного из значений расстояния на расчет развалился. Первый вариант построения решения с использованием $K_0$ выглядит надежнее.

## 13.7 Частное решение для конечного круглого пласта с замкнутой границей и конечного радиуса скважины.

Частное решение строится за счет поиска параметров $A$ и $B$ для уравнения (13.2.5) так, чтобы они удовлетворяли граничным условиям.

1. Граничное условия на внешнем контуре с замкнутой границей

$$\left( \frac{d p_D}{d r_d}\right)_{r_d=r_e} = 0  \tag{13.6.1} \label{13.7.1}$$

в пространстве Лапласа  \eqref{13.7.1} преобразуется в следующее

$$\left( \frac{ \tilde{p}_D(r_d, u)}{d r_d}\right)_{r_d=r_e} = 0 \tag{13.6.2} \label{13.7.2}$$

2. Граничное условие на скважине

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ \partial p_D(r_D, t_D)}{\partial r_D} \right] = -1  \tag{13.6.3} \label{13.7.3}$$

в пространстве Лапласа  \eqref{13.7.3} с учетом выражения (12.9) преобразуется к

$$\lim_{r_D \to r_{wD}} \left[ r_D \dfrac{ d \tilde{p}_D}{d r_D} \right] = -\dfrac{1}{u} \tag{13.6.4} \label{13.7.4}$$

```python
r_e = sp.symbols('r_e', positive=True)
eq1 = sp.Eq(soln2.rhs.diff(r_d).subs(r_d, r_e),0)
eq1
```

$\displaystyle - A \sqrt{u} K_{1}\left(r_{e} \sqrt{u}\right) + B \sqrt{u} I_{1}\left(r_{e} \sqrt{u}\right) = 0$

```python
eq2 = sp.Eq(r_d * soln2.rhs.diff(r_d), -1/u).subs(r_d, 1)
eq2
```

$\displaystyle - A \sqrt{u} K_{1}\left(\sqrt{u}\right) + B \sqrt{u} I_{1}\left(\sqrt{u}\right) = - \frac{1}{u}$

```python
sol_AB = sp.solve([eq1,eq2], [A,B])
sol_AB
```

    {A: besseli(1, r_e*sqrt(u))/(-u**(3/2)*besseli(1, sqrt(u))*besselk(1, r_e*sqrt(u)) + u**(3/2)*besseli(1, r_e*sqrt(u))*besselk(1, sqrt(u))),
     B: besselk(1, r_e*sqrt(u))/(-u**(3/2)*besseli(1, sqrt(u))*besselk(1, r_e*sqrt(u)) + u**(3/2)*besseli(1, r_e*sqrt(u))*besselk(1, sqrt(u)))}

```python
sol_re_close = soln2.subs(A, sol_AB[A]).subs(B, sol_AB[B]).simplify()
sol_re_close
```

$\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{I_{0}\left(r_{d} \sqrt{u}\right) K_{1}\left(r_{e} \sqrt{u}\right) + I_{1}\left(r_{e} \sqrt{u}\right) K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} \left(- I_{1}\left(\sqrt{u}\right) K_{1}\left(r_{e} \sqrt{u}\right) + I_{1}\left(r_{e} \sqrt{u}\right) K_{1}\left(\sqrt{u}\right)\right)}$

Таким образом получено решение для замкнутой круговой границы вокруг скважины
$$
\displaystyle \operatorname{p_{d}}{\left(r_{d} \right)} = \frac{I_{0}\left(r_{d} \sqrt{u}\right) K_{1}\left(r_{e} \sqrt{u}\right) + I_{1}\left(r_{e} \sqrt{u}\right) K_{0}\left(r_{d} \sqrt{u}\right)}{u^{\frac{3}{2}} \left(- I_{1}\left(\sqrt{u}\right) K_{1}\left(r_{e} \sqrt{u}\right) + I_{1}\left(r_{e} \sqrt{u}\right) K_{1}\left(\sqrt{u}\right)\right)} \tag{13.7.5}
$$

Ипользуя подход показанный выше можно построить графики зависимости безразмерного давления от времени и расстояния для этого решения с использованием numpy и scipy функций

```python
# преобразование sympy выражения в lambda функцию с использованием numpy
soln_re_close_numpy = sp.lambdify([u, r_d, r_e], sol_re_close.rhs, modules = ['numpy', 'scipy'])

# после преобразования lambda функцию можно использовать для проведения расчетов как обычныую функцию

# определим функцию для расчета обратного преобразования Лапласа
# отдельная функция удобно чтобы передавать r_d второй параметр
def soln_re_close_inv_lapl_numpy(t_d=1000, r_d=1, r_e=100):
    sol_lap = anaflow.get_lap_inv(soln_re_close_numpy,  r_d=r_d, r_e=r_e)
    return sol_lap(t_d)

# зададим диапазоны изменения параметров для построения графиков
td_arr = np.logspace(1,5,100)
rd_arr = np.logspace(1,2,100)
re=100
# построим графики
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=[15,5])
for ri in np.linspace(1,100,10):
    ax1.plot(td_arr, soln_re_close_inv_lapl_numpy(td_arr, r_d=ri, r_e=re), label = f'$r_d={ri:.0f}$')
for ti in np.linspace(1e2, 1e4,10):
    ax2.plot(rd_arr, np.squeeze( [soln_re_close_inv_lapl_numpy(ti, r_d=ri, r_e=re) for ri in rd_arr ] ), label = f'$t_d={ti:.0f}$')
# наведем немного красоты
ax1.set_xlabel('$t_d$')
ax1.set_ylabel('$p_d$')
ax1.legend()
ax2.set_xlabel('$r_d$')
ax2.set_ylabel('$p_d$')
ax2.legend()
plt.show()
```


```python
# This notebook demonstrates gradient descent methods with fixed step size and backtracking line search.
# It was prepared by Vlad Kobzar (vk283@nyu.edu) for the Machine Learning course at NYU's Center for
# Data Science
# https://davidrosenberg.github.io/ml2017/
# February 5, 2017

# To install matplotlib, use e.g. pip install matplotlib

# To display animations, we use ffmpeg. To check whether this package has been installed on your
# computer run:
#      import matplotlib.animation as animation
#      print(animation.writers.list())
# If that shows ffmpeg, they installed it.
#
# Otherwise, it can be installed using, e.g., homebrew.
#     brew install ffmpeg
# To install homebrew on MAC, run
#     /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```

```python
import numpy as np
import sympy as sp
from numpy import linalg

# evaluate_gradient returns a numpy array representing the gradient
# of a function evaluated at a point.
def evaluate_gradient(f_str, point, variables=['x', 'y']):
    var = []
    grad = []
    for i in variables:
        var.append(sp.symbols(i))
    if len(point) != len(variables):
        raise ValueError('Length of points doesnt match the number of variables')
    for i in range(len(var)):
        diff_func = sp.lambdify(variables, sp.diff(sp.sympify(f_str), var[i]))
        grad.append(diff_func(*point))
    return np.array(grad)

# fixed_step_gd returns a list of numpy 2D arrays representing
# the steps of a gradient descent minimization algorithm with
# a fixed step size with respect to a function of two variables.
def fixed_step_gd(f_str, initial_vector, step_size,
                              maxstep = 12, precision=0.2):
    x=initial_vector
    steps=[x]
    grad = np.array ([0, 2*precision])
    i=0
    while linalg.norm(grad,2)> precision and (i< maxstep):
        grad=-evaluate_gradient(f_str, [x[0],x[1]])
        x = x +step_size*grad
        i+=1
        steps.append(x)
    return steps

# backtracking_line_search returns a list of numpy 2D arrays
# representing the steps of a gradient descent minimization
# with a backtracking line search for a function of two variables.
# The parameters alpha must be in (0...0.5) and beta in (0...1).
# See, e.g., Boyd, Algorithm 9.2.
def backtracking_line_search_gd(f_str, initial_vector,
                              maxstep = 12, precision=0.2, alpha = 0.1, beta =0.7):
    x = initial_vector
    steps=[initial_vector]
    grad=np.array([0, 2*precision])
    i=0
    f_lambda = sp.lambdify(['x', 'y'], f_str, "numpy")
    #This loop calculates the coordinates of each step.
    while (linalg.norm(grad,2) > precision) and (i< maxstep):
        grad=-evaluate_gradient(func_str, [x[0],x[1]])
        t=1
        i+=1
        j=0
        funcnextstep=f_lambda(x[0]+ t*grad[0],x[1]+ t*grad[1])
        linearextrapolation = f_lambda(x[0], x[1])+alpha*t*np.dot(evaluate_gradient(f_str, x), grad)
        #This loop calculates the size of each step
        while (funcnextstep> linearextrapolation) and (i< maxbacktrackstep):
            t=beta*t
            j += 1
            funcnextstep=f_lambda(x[0]+ t*grad[0],x[1]+ t*grad[1])
            linearextrapolation = f_lambda(x[0], x[1])+alpha*t*np.dot(evaluate_gradient(f_str, x), grad)
        x=x+t*grad
        steps.append(x)
    return steps
```

```python
# init and animate are called by animation at initialization
# and each frame of the animation
def init():
    global  points, backtracking_points
    pts.set_data ([],[])
    sct.set_data([],[])
    backtracking_pts.set_data ([],[])
    backtracking_sct.set_data([],[])
    return pts, sct, backtracking_pts, backtracking_sct

def animate(i):
    global  points, backtracking_points
    if i <len (points):
        pts.set_data([item[0] for item in points[0:i]], [item[1] for item in points[0:i]])
        sct.set_data([item[0] for item in points[0:(i)]], [item[1] for item in points[0:(i)]])
    if i <len (backtracking_points):
        backtracking_pts.set_data([item[0] for item in backtracking_points[0:i]],
                                    [item[1] for item in backtracking_points[0:i]])
        backtracking_sct.set_data([item[0] for item in backtracking_points[0:(i)]],
                                [item[1] for item in backtracking_points[0:(i)]])
    return pts, sct, backtracking_pts, backtracking_sct
```

```python
# The following parameters are set by the user
x_init=np.array([-1,1]) # initial search point
xmin = x_init[0]-.5     # boundaries of the displayed domain
xmax = x_init[0]+4      # +1.5 Use the commented constants are
ymin = x_init[1]-.5     # -2   used to generate
ymax = x_init[1]+4      # +.5  figure 9.3 from Boyd & Vandenberge, Convex Optimization
grid_size = 40
precision = 0.2         #gradient l2 norm threshold for the stopping condition

func_str = '3*(.4*x)**2 - (.4*x)*y +0.5*y**2-(.4*x)-3*y' #The objective function (of x and y)

# Use the below to generate figure 9.3 from Boyd
# func_str =  'exp(x+3*y-0.1)+ exp(x-3*y-0.1)+exp(-x-0.1)'#Fixed GD step size
# Fixed step search doesn't work, for example, for the Boyd example. To turn it off,
# maxgdstep should be set to 1.

gdstep = 0.5            # Size of a fixed step
maxbacktrackstep=25     # Maximum number of steps for the backtracking line search
maxgdstep=25            # Maximum number of steps for the fixed step search

# The following code generates the animation
import matplotlib.pyplot as plt
from matplotlib import animation, rc
from IPython.display import HTML
rc('animation', html='html5')

# The following code generates the search steps and countour lines
func_sym=sp.sympify(func_str)
func_lambda = sp.lambdify(['x', 'y'], func_str, "numpy")
points = fixed_step_gd (func_str, x_init, gdstep, maxgdstep, precision)
backtracking_points= backtracking_line_search_gd(func_str, x_init, maxbacktrackstep,  precision)
x = np.linspace(xmin, xmax, grid_size)
y = np.linspace(ymin, ymax, grid_size)
xx, yy = np.meshgrid(x, y)
z = func_lambda(x[None,:],y[:,None] )
levels = np.array(sorted([func_lambda(p[0],p[1]) for p in backtracking_points]))

# The following code plots the search steps and countour lines
fig = plt.figure()
ax = fig.add_subplot(111)
plt.gca().set_aspect('equal', adjustable='box')
cp = plt.contour(xx, yy,z, levels = levels)
plt.title('Gradient Descent \n f(x,y)=%s' %func_str, fontsize=7)
plt.xlabel('x ')
plt.ylabel('y ')

pts, = ax.plot([],[], color='red',
               label='Fixed step size:  %.2f \n in %d steps' % (gdstep, (len(points)-1)))
sct,  = ax.plot([], 'ro', markersize=3)
backtracking_pts, = ax.plot([],[], color='black',
                            label='Backtracking line search\n in %d steps' % (len(backtracking_points)-1))
backtracking_sct, = ax.plot([], 'ko', markersize=3)
plt.legend([pts,backtracking_pts], [pts.get_label(), backtracking_pts.get_label()], loc='best', fontsize=7)

anim = animation.FuncAnimation(fig, animate, frames= max([len(points),
                               len (backtracking_points)]), interval=500, blit=True, init_func=init)
HTML(anim.to_html5_video())
```

```python

```

Notebooks/GD with Fixed Step and Backtracking/gd_fixed_and_backtracking.ipynb


Exercise material of the MSc-level course **Numerical Methods in Geotechnical Engineering**.
Held at Technische Universität Bergakademie Freiberg.

Comments to:

*Prof. Dr. Thomas Nagel
Chair of Soil Mechanics and Foundation Engineering
Geotechnical Institute
Technische Universität Bergakademie Freiberg.*

https://tu-freiberg.de/en/soilmechanics

```python
import numpy as np
import matplotlib.pyplot as plt

#Some plot settings
plt.style.use('seaborn-deep')
plt.rcParams['lines.linewidth']= 2.0
plt.rcParams['lines.color']= 'black'
plt.rcParams['legend.frameon']=True
plt.rcParams['font.family'] = 'serif'
plt.rcParams['legend.fontsize']=14
plt.rcParams['font.size'] = 14
plt.rcParams['axes.spines.right'] = False
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.left'] = True
plt.rcParams['axes.spines.bottom'] = True
plt.rcParams['axes.axisbelow'] = True
plt.rcParams['figure.figsize'] = (12, 6)
```

# Exercise 7 - 1D Consolidation, Terzaghi

## Governing differential equation

Neglecting lateral displacements and fluid flow

$$
	\epsilon_{xx} = \epsilon_{yy} = 0 \quad \text{and} \quad q_x = q_y = 0
$$

the volume balance simplifies to

$$
	\dot{\epsilon}_{zz} = \frac{\partial q_z}{\partial z}
$$

Now we substitute the constitutive law (linear elasticity) of the solid and Darcy's law for the fluid motion into the volume balance and get

$$
    \dot{\sigma}_{zz}^\text{eff} = E_\text{S} \dot{\epsilon}_{zz} \quad \text{and} \quad \frac{\partial q_z}{\partial z} = - \frac{k}{\mu_\text{F}} \frac{\partial^2 p}{\partial z^2} \quad \text{ergibt} \quad
		 \frac{\dot{\sigma}_{zz}^\text{eff}}{E_\text{S}} = - \frac{k}{\mu_\text{F}} \frac{\partial^2 p}{\partial z^2}
$$

Assuming constant total stress (constant external loads) we find with the use of the effective stress principle

$$
	\sigma_{zz} = \sigma_{zz}^\text{eff} + p = \text{const} \quad \rightarrow \quad \dot{\sigma}_{zz} = 0 = \dot{\sigma}_{zz}^\text{eff} + \dot{p}
$$

Substitution yields the partial, linear and homogeneous differential equation

$$
	\dot{p} = \frac{E_\text{S} k}{\mu_\text{F}} p_{,zz} = \frac{E_\text{S} K_\text{D}}{\gamma_\text{F}} p_{,zz} = c_\text{v} p_{,zz} \quad \text{with } K_\text{D} = \frac{k\rho_\text{F} g}{\mu_\text{F}}
$$

$[k] = \text{m}^2$: intrinsic permeability

$[K_\text{D}] = \text{m s}^{-1}$: hydraulic conductivity

$[c_\text{v}] = \text{m}^2\text{s}^{-1}$: coefficient of consolidation

## Weak form

The pore pressure can have (the essential/Dirichlet) boundary conditions in the form:

$$
    p = \bar{p}\ \forall z \in \partial \Omega_\mathrm{D}
$$

We now introduce a test function $\eta$ which vanishes where the pore pressure is given

$$
    \eta = 0\ \forall z \in \partial \Omega_\mathrm{D}
$$

and construct the weak form (using integration by parts):

\begin{align}
   0 &= \int \limits_0^H \eta \left[\dot{p} - c_\text{v} p_{,zz} \right] \text{d}z
   \\
   &= \int \limits_0^H \left[\eta \dot{p} - \left( \eta c_\text{v} p_{,z} \right)_{,z} + \eta_{,z} c_\text{v} p_{,z} \right] \, \text{d}z
   \\
   &= \int \limits_0^H \left[\eta \dot{p} + \eta_{,z} c_\text{v} p_{,z} \right] \, \text{d}z + \left[ \eta E_\text{S} q_z \right]^H_0
\end{align}

where the natural/Neumann boundary conditions have appeared.

## Finite elements in 1D

We have a soil column of height $H$ on top of the bed rock at $z=0$.

We first create an element class. An element knows the number of nodes it has, their IDs in the global node vector, and the coordinates of its nodes. Linear elements have 2 nodes and 2 quadrature points, quadratic elements 3 nodes and 3 quadrature points. The natural coordinates of the element run from -1 to 1, and the quadrature points and weights are directly taken from Numpy.

```python
#element class
class line_element():#local coordinates go from -1 to 1
    #takes number of nodes, global nodal coordinates, global node ids
    def __init__(self, nnodes=2, ncoords=[0.,1.], nids=[0,1]):
        self.__nnodes = nnodes
        if (len(ncoords) != self.__nnodes):
            raise Exception("Number of coordinates does not match number \
                            of nodes of element (%i vs of %i)" %(self.__nnodes,len(ncoords)))
        else:
            self.__coords = np.array(ncoords)

        self.__natural_coords = (self.__coords-self.__coords[0])/(self.__coords[-1]-self.__coords[0])*2. - 1.

        if (len(nids) != self.__nnodes):
            raise Exception("Number of node IDs does not match number \
                            of nodes of element (%i vs of %i)" %(self.__nnodes,len(nids)))
        else:
            self.__global_ids = np.array(nids)
        self.__quad_degree = self.__nnodes
        self.__quad_points, self.__quad_weights = np.polynomial.legendre.leggauss(self.__quad_degree)

```

Next, we wish to generate a one-dimensional mesh by specifying the length of a line, the number of elements into which the mesh is to be split, and the number of nodes per element.

```python
def number_of_nodes(nelems,nodes_per_elem):
    return nelems*nodes_per_elem - (nelems - 1)

def generate_mesh(domain_length,nelems,nodes_per_elem):
    nn = number_of_nodes(nelems,nodes_per_elem)
    #coordinate vector of global nodes
    global_nodal_coordinates = np.linspace(0.,domain_length,nn)
    global_solution = np.array([0.]*nn)

    #generate elements
    element_vector = []
    for i in range(nelems):
        node_start = (nodes_per_elem-1)*i
        element_vector.append(
            line_element(nodes_per_elem,
                         global_nodal_coordinates[node_start:node_start+nodes_per_elem],
                         list(range(node_start,node_start+nodes_per_elem))))

    return global_nodal_coordinates, element_vector, global_solution
```

## Shape functions in 1D

As in exercise 06, we allow linear and higher-order shape functions.

```python
#N
def shape_function(element_order,xi):
    if (element_order == 2): #-1,1
            return np.array([(1.-xi)/2., (1.+xi)/2.])
    elif (element_order == 3): #-1, 0, 1
            return np.array([(xi - 1.)*xi/2., (1-xi)*(1+xi), (1+xi)*xi/2.])
#dN_dxi
def dshape_function_dxi(element_order,xi):
    if (element_order == 2): #-1,1
        return np.array([-0.5*xi/xi, 0.5*xi/xi])  #xi only later for plotting dimensions
    elif (element_order == 3):#-1,0,1
        return np.array([xi - 0.5,-2.*xi,xi + 0.5])

#dz_dxi
def element_jacobian(element,xi):
    element_order = element._line_element__nnodes
    Jacobian = 0.
    Jacobian += dshape_function_dxi(element_order,xi).dot(element._line_element__coords)
    return Jacobian

#dN_dz
def grad_shape_function(element,xi):
    element_order = element._line_element__nnodes
    Jac = element_jacobian(element,xi)
    return dshape_function_dxi(element_order,xi)/Jac

```

## Time and space discretization, Picard iterations

Using a backward Euler approach for simplicity we find the time-discrete weak form as ($p_{n+1} \equiv p$)

$$
    0 = \int \limits_0^H \left[\eta \frac{p - p_n}{\Delta t} + \eta_{,z} c_\text{v} p_{,z} \right] \, \text{d}z + \left[ \eta E_\text{S} q_z \right]^H_0
$$

Introducing standard FE approximations

$$
    p \approx N_i \hat{p}_i, \quad \eta \approx N_i \hat{\eta}_i, \quad \frac{\partial p}{\partial z} \approx \nabla  N_i \hat{p}_i, \quad \frac{\partial \eta}{\partial z} \approx \nabla N_i \hat{\eta}_i
$$

This yields

$$
\begin{align}
 0 &= \int \limits_0^H \left[ N_i \hat{\eta}_i \frac{N_k \hat{p}_k - N_k \hat{p}_{n,k}}{\Delta t} + \nabla N_i \hat{\eta}_i c_\text{v} \nabla N_k \hat{p}_k \right] \, \text{d}z + \left[ N_i \hat{\eta}_i E_\text{S} q_z \right]^H_0
 \end{align}
$$

Now we bring all quantities associated with the unknown pressure to the left-hand side (LHS) and all known quantities to the RHS:

$$
    \hat{\eta}_i \int \limits_0^H \left[ N_i \frac{1}{\Delta t} N_k + \nabla N_i c_\text{v} \nabla  N_k \right] \, \text{d}z\ \hat{p}_k =  \hat{\eta}_i \left[ N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=H} \delta_{n_\text{n}i} - N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=0} \delta_{i0} \right] +  \hat{\eta}_i \int \limits_0^H N_i \frac{1}{\Delta t} N_k \, \text{d}z\ \hat{p}_{n,k}
$$

can be simplified by realizing that the nodal test function values are arbitrary and thus

$$
    \int \limits_0^H \left[ N_i \frac{1}{\Delta t} N_k + \nabla N_i c_\text{v} \nabla N_k \right] \, \text{d}z\ \hat{p}_k = N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=H} \delta_{n_\text{n}i} - N_{n_\text{n}} E_\text{S} \bar{q}_z|_{z=0} \delta_{i0} +  \int \limits_0^H N_i \frac{1}{\Delta t} N_k \, \text{d}z\ \hat{p}_{n,k}
$$

which leaves us with $n_\text{n}$ equations for the $n_\text{n}$ unknown nodal pressures $\hat{p}_k$. If any coefficients in the above are taken as pressure-dependent, the system could be solved repeatedly usind Picard iterations. For strong non-linearities, a Newton linearization would typically be used.

### Question:

How would the equation differ for an explicit time integration scheme (forward Euler)?

What we require now is the local assembler to calculate the stiffness matrix and the local right-hand side. Local integration is performed by Gauss quadrature:

$$
    \int \limits_{-1}^1 f(\xi)\,\text{d}\xi \approx \sum \limits_{i=1}^{n_\text{gp}} f(\xi_i) w_i
$$

## Local assember

```python
def Stiffness(z):
    E0 = 5.e6 #Pa
    return E0

def Conductivity(z):
    kf = 1.e-9#m/s
    return kf

def SpecificWeight(z):
    g = 9.81 #m/s²
    rhow = 1000. #kg/m³
    return g*rhow

def ConsolidationCoeff(z):#m²/s
    return Stiffness(z)*Conductivity(z)/SpecificWeight(z)
```

```python
def local_assembler(elem,dt,prev_sol,mass_lumping=False):
    element_order = elem._line_element__nnodes
    K_loc = np.zeros((element_order,element_order))
    M_loc = np.zeros((element_order,element_order))
    b_loc = np.zeros(element_order)
    z_nodes = elem._line_element__coords
    for i in range(elem._line_element__quad_degree):
        #local integration point coordinate
        xi = elem._line_element__quad_points[i]
        #shape function
        N = shape_function(element_order,xi)
        #gradient of shape function
        dN_dX = grad_shape_function(elem,xi)
        #determinant of Jacobian
        detJ = np.abs(element_jacobian(elem,xi))
        #integration weight
        w = elem._line_element__quad_weights[i]

        #global integration point coordinate (for spatially varying properties)
        z_glob = np.dot(N,z_nodes)
        #evaluation of local material/structural properties
        E = Stiffness(z_glob)
        #evaluation of local body force
        CV = ConsolidationCoeff(z_glob)

        #assembly of local stiffness matrix
        M_loc = np.outer(N,N) / dt
        if (mass_lumping):
            M_loc = np.diag(M_loc.sum(0)) #diagonal of column sum

        K_loc += (np.outer(dN_dX,dN_dX) * CV + M_loc)* w * detJ

        #assembly of local RHS
        p_prev = np.dot(N,prev_sol)#pressure in integration point
        b_loc += N * p_prev/dt * w * detJ
    return K_loc,b_loc
```

## Global assembly

Now we can construct the global matrix system $\mathbf{K}\mathbf{u} = \mathbf{f}$ or $\mathbf{A}\mathbf{x}=\mathbf{b}$ (see lecture script).

```python
def global_assembler(nodes,elements,solution,dt,mass_lumping=False):
    K_glob = np.zeros((len(nodes),len(nodes)))
    b_glob = np.zeros(len(nodes))
    for i,elem in enumerate(elements):
        start_id = elem._line_element__global_ids[0]
        end_id = elem._line_element__global_ids[-1]
        K_i, b_i = local_assembler(elem,dt,solution[start_id:end_id+1],mass_lumping)
        K_glob[start_id:end_id+1,start_id:end_id+1] += K_i
        b_glob[start_id:end_id+1] += b_i
    return K_glob, b_glob
```

## Application of boundary conditions

First we apply flux boundary conditions

```python
def apply_Neumann_bc(b_glob,node_id,value):
    b_glob[node_id] += value
    return b_glob
```

Then we apply Dirichlet bc

```python
def apply_Dirichlet_bc(K_glob,b_glob,node_id,value):
    K_glob[node_id,:] = 0.# = K_glob[:,node_id] = 0.
    K_glob[node_id,node_id] = 1.
    b_glob[node_id] = value
    return K_glob, b_glob
```

## Application of initial conditions

Since we're dealing with a time-dependent problem (rate problem) we require initial conditions for the pore pressure, i.e. $p_0 = p(t=0)\ \forall\ z$. Due to the very specific assumptions in deriving this consolidation equation, the initial pressure is given by the (suddenly) applied load:

$$
    p_0 = \sigma_{zz} = \text{const.}
$$

```python
def apply_initial_conditions(solution,sig_v):
    solution *= 0.
    solution += sig_v
    return
```

## Time loop and problem solution

We now establish the time loop and in each time step perform the global assembly, apply a vanishing traction on the top and constrain the displacement at the bottom to zero.

```python
def time_loop(dt,nodes,elements,solution,mass_lumping=False):
    #Startwerte
    t_end = 366*24*60*60 #s
    absolute_tolerance = 1.e-6
    max_iter = 100
    iteration_counter = np.array([0])
    apply_initial_conditions(solution,200.e3)
    y = [solution] #create a list that will hold the solution vectors at all time points
    times = np.array([0.])
    #
    while times[-1]+dt < t_end: #repeat the loop as long as the final time step is below the end point
        times = np.append(times,times[-1]+dt) #here define the next time point as the previous time point plus the time increment dt
        y_old = y[-1] #Starting value for recursive update
        i = 0
        #
        while True:
            K, f = global_assembler(nodes,elements,y[-1],dt,mass_lumping)
            #f = apply_Neumann_bc(f,len(nodes)-1,0)
            K, f = apply_Dirichlet_bc(K, f, len(nodes)-1, 0.)#free draining top
            solution = np.linalg.solve(K,f)
            i += 1
            if (np.abs(np.linalg.norm(solution) - np.linalg.norm(y_old)) < absolute_tolerance or i > max_iter): #if change is below tolerance, stop iterations
                break
            y_old = solution #preparation of next recursion

        y.append(solution) #append the new found solution to the solution vector
        iteration_counter = np.append(iteration_counter,i) #store how much iterations this time step took to converge
    return times, y,iteration_counter
```

```python
#spatial discretization
H = 10.
nel = 20
n_per_el = 3
nodes,elements,solution=generate_mesh(H,nel,n_per_el)
```

```python
times, sols, iters = time_loop(24*60*60,nodes,elements,solution)
```

```python
plt.xlabel('$z$ / m')
plt.ylabel('$p$ / kPa')
plt.title('Finite element solution')
plt.plot(nodes, sols[0]/1.e3, marker='o', label='$t = %i$ d' %(times[0]/60/60))
plt.plot(nodes, sols[1]/1.e3, marker='o', label='$t = %i$ d' %(times[1]/60/60/24))
plt.plot(nodes, sols[10]/1.e3, marker='o', label='$t = %i$ d' %(times[10]/60/60/24))
plt.plot(nodes, sols[30]/1.e3, marker='o', label='$t = %i$ d' %(times[30]/60/60/24))
plt.plot(nodes, sols[200]/1.e3, marker='o', label='$t = %i$ d' %(times[200]/60/60/24))
plt.plot(nodes, sols[-1]/1.e3, marker='o', label='$t = %i$ d' %(times[-1]/60/60/24))
plt.legend();
```

```python
#from matplotlib import animation
#from IPython.display import HTML

# First set up the figure, the axis, and the plot element we want to animate
#fig, ax = plt.subplots();
#ax.set_xlim(( 0, 11));
#ax.set_ylim((0, 210));

#line, = ax.plot([], [], lw=2);

# initialization function: plot the background of each frame
#def init():
#    line.set_data(nodes, sols[0]/1e3)
#    return (line,)

# animation function. This is called sequentially
#def animate(i):
    #x = nodes
    #y = sols[i]/1e3
    #line.set_data(x, y)
    #return (line,)

# call the animator. blit=True means only re-draw the parts that have changed.
#anim = animation.FuncAnimation(fig, animate, init_func=init,
#                               frames=len(sols)-1, interval=20, blit=True);
#HTML(anim.to_html5_video())
```

### Convergence study

Let's do a simple convergence study.

```python
dts = [12.*3600,24.*3600,48.*3600]
nels = [10,20,40]
H = 10.

fig, ax = plt.subplots(nrows=3,ncols=3,figsize=(18,18))

for i,dt in enumerate(dts):
    for j,nel in enumerate(nels):
        #print("Running (n,dt) combination ", dt, nel)
        number_of_elements = nel
        nodes_per_element = 2
        nodes,elements,solution=generate_mesh(H,number_of_elements,nodes_per_element)
        times, sols, iters = time_loop(dt,nodes,elements,solution)
        nodes,elements,solution=generate_mesh(H,number_of_elements,nodes_per_element)
        times_lumped, sols_lumped, iters_lumped = time_loop(dt,nodes,elements,solution,True)
        #
        ax[i][j].plot(nodes, sols[0]/1.e3, marker='o', color='green', label='$t = %i$ d' %(times[0]/60/60/24))
        ax[i][j].plot(nodes, sols[1]/1.e3, marker='o', color='red', label='$t = %i$ d' %(times[1]/60/60/24))
        ax[i][j].plot(nodes, sols[10]/1.e3, marker='o', color='blue', label='$t = %i$ d' %(times[10]/60/60/24))
        #
        ax[i][j].plot(nodes, sols_lumped[0]/1.e3, marker='d', ls=':', color='green')
        ax[i][j].plot(nodes, sols_lumped[1]/1.e3, marker='d', ls=':', color='red')
        ax[i][j].plot(nodes, sols_lumped[10]/1.e3, marker='d', ls=':', color='blue')
        #
        ax[i][j].set_xlabel('$z$ / m')
        ax[i][j].set_ylabel('$p$ / kPa')
        ax[i][j].set_title('dt = %i h, nel = %i' %(dt/3600,nel))
        ax[i][j].legend(fontsize=10)
fig.tight_layout()
```

We observe that time step size and discretization size act together, and that unsuitable choices can lead to unphysical oscillations.

## Interfaces

This type of behaviour can not only be observed near boundaries but also at material interfaces. Let's assume that a rigid permeable layer acts as an overburden on top of a softer low-permeability layer.

```python
def Stiffness(z):
    if z < H/2:
        E0 = 5.e6 #Pa
    else:
        E0 = 5e8 #Pa
    return E0

def Conductivity(z):
    if z < H/2:
        kf = 1.e-10#m/s
    else:
        kf = 1e-8#m/s
    return kf
```

```python
dts = [12.*3600,24.*3600,48.*3600]
nels = [10,20,40]
H = 10.

fig, ax = plt.subplots(nrows=3,ncols=3,figsize=(18,18))

for i,dt in enumerate(dts):
    for j,nel in enumerate(nels):
        #print("Running (n,dt) combination ", dt, nel)
        number_of_elements = nel
        nodes_per_element = 2
        nodes,elements,solution=generate_mesh(H,number_of_elements,nodes_per_element)
        times, sols, iters = time_loop(dt,nodes,elements,solution)
        nodes,elements,solution=generate_mesh(H,number_of_elements,nodes_per_element)
        times_lumped, sols_lumped, iters_lumped = time_loop(dt,nodes,elements,solution,True)
        #
        ax[i][j].plot(nodes, sols[0]/1.e3, marker='o', color='green', label='$t = %i$ d' %(times[0]/60/60/24))
        ax[i][j].plot(nodes, sols[1]/1.e3, marker='o', color='red', label='$t = %i$ d' %(times[1]/60/60/24))
        ax[i][j].plot(nodes, sols[10]/1.e3, marker='o', color='blue', label='$t = %i$ d' %(times[10]/60/60/24))
        #
        ax[i][j].plot(nodes, sols_lumped[0]/1.e3, marker='d', ls=':', color='green')
        ax[i][j].plot(nodes, sols_lumped[1]/1.e3, marker='d', ls=':', color='red')
        ax[i][j].plot(nodes, sols_lumped[10]/1.e3, marker='d', ls=':', color='blue')
        #
        ax[i][j].set_xlabel('$z$ / m')
        ax[i][j].set_ylabel('$p$ / kPa')
        ax[i][j].set_title('dt = %i h, nel = %i' %(dt/3600,nel))
        ax[i][j].text(1.5,100,'$C_\\mathrm{V} = %.2e$ m/s²' %ConsolidationCoeff(1.5))
        ax[i][j].text(6.5,100,'$C_\\mathrm{V} = %.2e$ m/s²' %ConsolidationCoeff(6.5))
        ax[i][j].legend(fontsize=10)
fig.tight_layout()
```


# Introduction to dynamic AFM simulations
###### Content under Creative Commons Attribution license CC-BY 4.0 version,
#### Enrique A. López-Guerra.

Purpose of the notebook: show an application of numerical methods to simulate the dynamics of a probe in atomic force microscopy.

Requirements to take the best advantage of this notebook: knowing the fundamentals of  [Harmonic Oscillators](http://en.wikipedia.org/wiki/Harmonic_oscillsator) in clasical mechanics and [Fundamentals of Vibrations](http://en.wikipedia.org/wiki/Vibration).

## Introduction

Since the atomic force microscope (AFM) was invented in 1986 it has become one of the main tools to study matter at the micro and nanoscale. This powerful tool is so versatile that it can be used to study a wide variety of materials, ranging from stiff inorganic surfaces to soft biological samples.

In its early stages the AFM was used in permanent contact with the sample (the probe is dragged over the sample during the whole operation), which brought about important drawbacks, such as rapid probe wear and often sample damage, but these obstacles have been overcome with the development of dynamic techniques.

In this Jupyter notebook, we will focus on the operation of the probe in dynamic mode.

```python
from __future__ import division, print_function, absolute_import, unicode_literals
import os
import numpy
import matplotlib.pyplot as plt
%matplotlib inline
from IPython.display import Image
```

```python
path = os.getcwd()
fig1 = path + '/Fig1.jpg'
Image(filename=fig1)
```

#### Figure 1.   Schematics of the setup of a atomic force microscope (Adapted from reference 6)

In AFM the interacting probe is in general a rectangular cantilever (please check the image above that shows the AFM setup where you will be able to see the probe!).
Probably the most used dynamic technique in AFM is the Tapping Mode. In this method the probe taps a surface in intermittent contact fashion. The purpose of tapping the probe over the surface instead of dragging it is to reduce frictional forces that may cause damage of soft samples and wear of the tip. Besides with the tapping mode we can get more information about the sample!  HOW???

In Tapping Mode AFM the cantilever is shaken to oscillate up and down at a specific frequency (most of the time shaken at its natural frequency). Then the deflection of the tip is measured at that frequency to get information about the sample. Besides acquiring the topography of the sample, the phase lag between the excitation and the response of the cantilever can be related to compositional material properties!
In other words one can simultaneously get information about how the surface looks and also get compositional mapping of the surface!  THAT SOUNDS POWERFUL!!!

```python
fig2 = path + '/Fig2DHO.jpg'
Image(filename=fig2)
```

#### Figure 2. Schematics of a damped harmonic oscillator without tip-sample interactions

## Analytical Solution

The motion of the probe can be derived using [Euler-Bernoulli's](http://en.wikipedia.org/wiki/Euler%E2%80%93Bernoulli_beam_theory) equation. However that equation has partial derivatives (it depends on time and space) because it deals with finding the position of each point of the beam in a certain time, which cant make the problem too expensive computationally for our purposes. In our case, we have the advantage that we are only concerned about the position of the tip (which is the only part of the probe that will interact with the sample). As a consequence many researchers in AFM have successfully made approximations using a simple mass point model approximation [see ref. 2] like the one in figure 2 (with of course the addition of tip sample forces!  We will see more about this later).

First we will study the system of figure 2 AS IS (without addition of tip-sample force term), WHY?  Because we want to get an analytical solution to get a reference of how our integration schemes are working, and the addition of tip sample forces to our equation will prevent the acquisition of straightforward analytical solutions :(

Then, the equation of motion of the damped harmonic oscillator of figure 2, which is DRIVEN COSINUSOIDALLY (remember that we are exciting our probe during the scanning process) is:

$$\begin{equation}
m \frac{d^2z}{dt^2} = - k z - \frac{m\omega_0}{Q}\frac{dz}{dt} + F_0\cos(\omega t)
\end{equation}$$
where k is the stiffness of the cantilever, z is the vertical position of the tip with respect to the cantilever base position, Q is the quality factor (which is related to the damping of the system), $F_0$ is the driving force amplitude, $\omega_0$ is the resonance frequency of the oscillator, and $\omega$ is the frequency of the oscillating force.

The analytical solution of the above ODE is composed by a transient term and a steady state term. We are only interested in the steady state part because during the scanning process it is assumed that the probe has achieved that state.

The steady state solution is given by:
$$\begin{equation}
A\cos (\omega t - \phi)
\end{equation}$$

where A is the steady state amplitude of the oscillation response, which depends on the cantilever parameters and the driving parameters, as can be seen in the following relation:
$$\begin{equation}
A = \frac{F_0/m}{\sqrt{(\omega_0^2-\omega^2)^2+(\frac{\omega\omega_0}{Q})^2}}
\end{equation}$$

and $\phi$ is given by:
$$\begin{equation}
\phi = \arctan \big( \frac{\omega\omega_0/Q}{\omega_0^2 - \omega^2} \big)
\end{equation}$$

Let's first name the variables that we are going to use. Because we are dealing with a damped harmonic oscillator model we have to include variables such as: spring stiffness, resonance frequency, quality factor (related to damping coefficient), target oscillation amplitude, etc.

```python
k = 10.
fo = 45000
wo = 2.0*numpy.pi*fo
Q = 25.

period = 1./fo
m = k/(wo**2)
Ao = 60.e-9
Fd = k*Ao/Q

spp = 28. # time steps per period
dt = period/spp #Intentionally chosen to be quite big
#you can decrease dt by increasing the number of steps per period

simultime = 100.*period
N = int(simultime/dt)

#Analytical solution

time_an = numpy.linspace(0,simultime,N)  #time array for the analytical solution
z_an = numpy.zeros(N)  #position array for the analytical solution

#Driving force amplitude this gives us 60nm of amp response (A_target*k/Q)
Fo_an = 24.0e-9

A_an = Fo_an*Q/k  #when driven at resonance A is simply Fo*Q/k
phi = numpy.pi/2  #when driven at resonance the phase is pi/2

z_an[:] = A_an*numpy.cos(wo*time_an[:] - phi) #this gets the analytical solution

#slicing the array to include only steady state (only the last 10 periods)
z_an_steady = z_an[int(90.*period/dt):]
time_an_steady = time_an[int(90.*period/dt):]

plt.title('Plot 1    Analytical Steady State Solution of Eq 1', fontsize=20)
plt.xlabel('time, ms', fontsize=18)
plt.ylabel('z_Analytical, nm', fontsize=18)
plt.plot(time_an_steady*1e3, z_an_steady*1e9, 'b--')

```

### Approximating through Euler's method

If we perform a Taylor series expansion of $z_{n+1}$ around $z_{n}$ we get:

$$z_{n+1} = z_{n} + \Delta t\frac{dz}{dt}\big|_n + {\mathcal O}(\Delta t^2)$$

The Euler formula neglects terms in the order of two or higher, ending up as:

$$\begin{equation}
z_{n+1} = z_{n} + \Delta t\frac{dz}{dt}\big|_n
\end{equation}$$

It can be easily seen that the truncation error of the Euler algorithm is in the order of ${\mathcal O}(\Delta t^2)$.

This is a second order ODE, but we can convert it to a system of two coupled 1st order differential equations. To do it we will define $\frac{dz}{dt} = v$. Then equation (1) will be decomposed as:
$$\begin{equation}
\frac{dz}{dt} = v
\end{equation}$$

$$\begin{equation}
\frac{dv}{dt} = -kz-\frac{m\omega_0}{Q}+F_o\cos(\omega t)
\end{equation}$$

These coupled equations will be used during Euler's aproximation and also during our integration using Runge Kutta 4 method.

```python
t= numpy.linspace(0,simultime,N) #time grid for Euler method

#Initializing variables for Euler
vdot_E = numpy.zeros(N)
v_E = numpy.zeros(N)
z_E = numpy.zeros(N)

#Initial conditions
z_E[0]= 0.0
v_E[0]=0.0

for i in range (N-1):
    vdot_E[i] =( ( -k*z_E[i] - (m*wo/Q)*(v_E[i])   +\
                    Fd*numpy.cos(wo*t[i]) ) / m)  #Equation 7
    v_E[i+1] = v_E[i] + dt*vdot_E[i]  #Based on equation 5
    z_E[i+1] = z_E[i] + v_E[i]*dt  #Equation 5

plt.title('Plot 2  Eulers approximation of Equation1', fontsize=20);
plt.plot(t*1e3,z_E*1e9);
plt.xlabel('time, s', fontsize=18);
plt.ylabel('z_Euler, nm', fontsize=18);

```

This looks totally unphysical! We were expecting to have a steady state oscillation of 60 nm and we got a huge oscillation that keeps growing. Can it be due to the scheme? The timestep that we have chosen is quite big with respect to the oscillation period. We have intentionally set it to ONLY 28 time steps per period (That could be the reason why the scheme can't capture the physics of the problem). That's quite discouraging. However the timestep is quite big and it really gets better as you decrease the time step. Try it! Reduce the time step and see how the numerical solution acquires an amplitude of 60 nm as the analytical one. At this point we can't state anything about accuracy before doing an analysis of error (we will make this soon). But first, let's try to analyze if another more efficient scheme can capture the physics of our damped harmonic oscillator even with this large time step.

### Let's try to get more accurate... Verlet Algorithm

This is a very popular algorithm widely used in molecular dynamics simulations. Its popularity has been related to high stability when compared to the simple Euler method, it is also very simple to implement and accurate as we will see soon!  Verlet integration can be seen as using the central difference approximation to the second derivative. Consider the Taylor expansion of $z_{n+1}$ and $z_{n-1}$ around $z_n$:

$$\begin{equation}
z_{n+1} = z_n + \Delta t \frac{dz}{dt}\big|_n + \frac{\Delta t^2}{2} \frac{d^2 z}{d t^2}\big|_n + \frac{\Delta t^3}{6} \frac{d^3 z}{d t^3}\big|_n + {\mathcal O}(\Delta t^4)
\end{equation}$$

$$\begin{equation}
z_{n-1} = z_n - \Delta t \frac{dz}{dt}\big|_n + \frac{\Delta t^2}{2} \frac{d^2 z}{dt^2}\big|_n - \frac{\Delta t^3}{6}
\frac{d^3 z}{d t^3}\big|_n + {\mathcal O}(\Delta t^4)
\end{equation}$$

Adding up these two expansions and solving for $z_{n+1}$ we get:

$$z_{n+1}= 2z_{n} - z_{n-1} + \frac{d^2 z}{d t^2} \Delta t^2\big|_n + {\mathcal O}(\Delta t^4) $$

Verlet algorithm neglects terms on the order of 4 or higher, ending up with:

$$\begin{equation}
z_{n+1}= 2z_{n} - z_{n-1} + \frac{d^2 z}{d t^2} \Delta t^2\big|_n
\end{equation}$$

This looks nice; it seems that the straightforward calculation of the second derivative will give us good results. BUT have you seen that we also need the value of the first derivative (velocity) to put it into the equation of motion that we are integrating (see equation 1). YES, that's a main drawback of this scheme and therefore it's mainly used in applications where the equation to be integrated doesn't have first derivative. But don't panic we will see what can we do...

What about subtracting equations 8 and 9 and then solving for $\frac{dz}{dt}\big|_n$:
$$
\frac{dz}{dt}\big|_n =  \frac{z_{n+1} - z_{n-1}}{2\Delta t} + {\mathcal O}(\Delta t^2)
$$
If we neglect terms on the order of 2 or higher we can calculate velocity:
$$\begin{equation}
\frac{dz}{dt}\big|_n =  \frac{z_{n+1} - z_{n-1}}{2\Delta t}
\end{equation}$$

This way of calculating velocity is pretty common in Verlet integration in applications where velocity is not explicit in the equation of motion. However for our purposes of solving equation 1 (where first derivative is explicitly present) it seems that we will lose accuracy because of the velocity, we will discuss more about this soon after...

Have you noticed that we need a value $z_{n-1}$? Does it sound familiar?  YES!  This is not a self-starting method. As a result we will have to overcome the issue by setting the initial conditions of the first step using Euler approximation. This is a bit annoying, but a couple of extra lines of code won't kill you :)

```python
time_V = numpy.linspace(0,simultime,N)

#Initializing variables for Verlet
zdoubledot_V = numpy.zeros(N)
zdot_V = numpy.zeros(N)
z_V = numpy.zeros(N)

#Initial conditions Verlet. Look how we use Euler for the first step approximation!
z_V[0] = 0.0
zdot_V[0] = 0.0
zdoubledot_V[0] = (   ( -k*z_V[0] - (m*wo/Q)*zdot_V[0] +\
                            Fd*numpy.cos(wo*t[0]) )  ) / m
zdot_V[1] = zdot_V[0] + zdoubledot_V[0]*dt
z_V[1] = z_V[0] + zdot_V[0]*dt
zdoubledot_V[1] = (   ( -k*z_V[1] - (m*wo/Q)*zdot_V[1] +\
                            Fd*numpy.cos(wo*t[1]) )  ) / m

#VERLET ALGORITHM

for i in range(2,N):
    z_V[i] = 2*z_V[i-1] - z_V[i-2] + zdoubledot_V[i-1]*dt**2 #Eq 10
    zdot_V[i] = (z_V[i]-z_V[i-2])/(2.0*dt) #Eq 11
    zdoubledot_V[i] = (   ( -k*z_V[i] - (m*wo/Q)*zdot_V[i] +\
                               Fd*numpy.cos(wo*t[i]) )  ) / m #from eq 1

plt.title('Plot 3  Verlet approximation of Equation1', fontsize=20);
plt.xlabel('time, ms', fontsize=18);
plt.ylabel('z_Verlet, nm', fontsize=18);
plt.plot(time_V*1e3, z_V*1e9, 'g-');
plt.ylim(-65,65);

```

It WAS ABLE to capture the physics! Even with the big time step that we use with Euler scheme!

As you can see, and as we previously discussed the harmonic response is composed of a transient and a steady part. We are only concerned about the steady-state, since it is assumed that the probe achieves steady state motion during the imaging process. Therefore, we are going to slice our array in order to show only the last 10 oscillations, and we will see if it resembles the analytical solution.

```python
#Slicing the full response vector to get the steady state response
z_steady_V = z_V[int(90*period/dt):]
time_steady_V = time_V[int(90*period/dt):]

plt.title('Plot 3  Verlet approx. of steady state sol. of Eq 1', fontsize=20);
plt.xlabel('time, ms', fontsize=18);
plt.ylabel('z_Verlet, nm', fontsize=18);
plt.plot(time_steady_V*1e3, z_steady_V*1e9, 'g-');
plt.ylim(-65,65);
plt.show();

```

### Let's use now one of the most popular schemes...     The Runge Kutta 4!

The Runge Kutta 4 (RK4) method is very popular for the solution of ODEs. This method is designed to solve 1st order differential equations. We have converted our 2nd order ODE to a system of two coupled 1st order ODEs when we implemented the Euler scheme (equations 5 and 6). And we will  have to use these equations for the RK4 algorithm.

In order to clearly see the RK4 implementation we are going to put equations 5 and 6 in the following form:
$$\begin{equation}
\frac{dz}{dt}=v \Rightarrow f1(t,z,v)
\end{equation}$$
$$\begin{equation}
\frac{dv}{dt} = -kz-\frac{m\omega_0}{Q}+F_ocos(\omega t) \Rightarrow f2(t,z,v)
\end{equation}$$

It can be clearly seen that we have two coupled equations f1 and f2 and both depend in t, z, and v.

The RK4 equations for our special case where we have two coupled equations, are the following:
$$\begin{equation}
k_1 = f1(t_i, z_i, v_i)
\end{equation}$$
$$\begin{equation}
m_1 = f2(t_i, z_i, v_i)
\end{equation}$$
$$\begin{equation}
k_2 = f1(t_i +1/2\Delta t, z_i + 1/2k_1\Delta t, v_i + 1/2m_1\Delta t)
\end{equation}$$
$$\begin{equation}
m_2 = f2(t_i +1/2\Delta t, z_i + 1/2k_1\Delta t, v_i + 1/2m_1\Delta t)
\end{equation}$$
$$\begin{equation}
k_3 = f1(t_i +1/2\Delta t, z_i + k_2\Delta t, v_i + 1/2m_2\Delta t)
\end{equation}$$
$$\begin{equation}
m_3 = f2(t_i +1/2\Delta t, z_i + 1/2k_2\Delta t, v_i + 1/2m_2\Delta t)
\end{equation}$$
$$\begin{equation}
k_4 = f1(t_i + \Delta t, z_i + k_3\Delta t, v_i + m_3\Delta t)
\end{equation}$$
$$\begin{equation}
k_4 = f2(t_i + \Delta t, z_i + k_3\Delta t, v_i + m_3\Delta t)
\end{equation}$$

$$\begin{equation}
f1_{n+1} = f1_n + \Delta t/6(k_1+2k_2+2k_3+k_4)
\end{equation}$$
$$\begin{equation}
f2_{n+1} = f2_n + \Delta t/6(m_1+2m_2+2m_3+m_4)
\end{equation}$$

Please notice how k values and m values are used sequentially, since it is crucial in the implementation of the method!

```python
#Definition of v, z, vectors
vdot_RK4 = numpy.zeros(N)
v_RK4 = numpy.zeros(N)
z_RK4 = numpy.zeros(N)
k1v_RK4 = numpy.zeros(N)
k2v_RK4 = numpy.zeros(N)
k3v_RK4 = numpy.zeros(N)
k4v_RK4 = numpy.zeros(N)

k1z_RK4 = numpy.zeros(N)
k2z_RK4 = numpy.zeros(N)
k3z_RK4 = numpy.zeros(N)
k4z_RK4 = numpy.zeros(N)

#calculation of velocities RK4

#INITIAL CONDITIONS
v_RK4[0] = 0
z_RK4[0] = 0

for i in range (1,N):
       #RK4
        k1z_RK4[i] = v_RK4[i-1]   #k1 Equation 14
        k1v_RK4[i] = ((   ( -k*z_RK4[i-1] - (m*wo/Q)*v_RK4[i-1] + \
                           Fd*numpy.cos(wo*t[i-1]) )  ) / m )   #m1 Equation 15

        k2z_RK4[i] = ((v_RK4[i-1])+k1v_RK4[i]/2.*dt) #k2 Equation 16
        k2v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k1z_RK4[i]/2.*dt) - (m*wo/Q)*\
                           (v_RK4[i-1] +k1v_RK4[i]/2.*dt) + Fd*\
                           numpy.cos(wo*(t[i-1] + dt/2.)) )  ) / m ) #m2 Eq 17

        k3z_RK4[i] = ((v_RK4[i-1])+k2v_RK4[i]/2.*dt) #k3, Equation 18
        k3v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k2z_RK4[i]/2.*dt) - (m*wo/Q)*\
                            (v_RK4[i-1] +k2v_RK4[i]/2.*dt) + Fd*\
                            numpy.cos(wo*(t[i-1] + dt/2.)) )  ) / m ) #m3, Eq 19

        k4z_RK4[i] = ((v_RK4[i-1])+k3v_RK4[i]*dt) #k4, Equation 20
        k4v_RK4[i] = ((   ( -k*(z_RK4[i-1] + k3z_RK4[i]*dt) - (m*wo/Q)*\
                           (v_RK4[i-1] + k3v_RK4[i]*dt) + Fd*\
                           numpy.cos(wo*(t[i-1] + dt)) )  ) / m )#m4, Eq 21

        #Calculation of velocity, Equation 23
        v_RK4[i] = v_RK4[i-1] + 1./6*dt*(k1v_RK4[i] + 2.*k2v_RK4[i] +\
                        2.*k3v_RK4[i] + k4v_RK4[i] )
        #calculation of position, Equation 22
        z_RK4 [i] = z_RK4[i-1] + 1./6*dt*(k1z_RK4[i] + 2.*k2z_RK4[i] +\
                        2.*k3z_RK4[i] + k4z_RK4[i] )

#slicing array to get steady state
z_steady_RK4 = z_RK4[int(90.*period/dt):]
time_steady_RK4 = t[int(90.*period/dt):]

plt.title('Plot 3  RK4 approx. of steady state sol. of Eq 1', fontsize=20);
plt.xlabel('time, ms', fontsize=18);
plt.ylabel('z_RK4, nm', fontsize=18);
plt.plot(time_steady_RK4 *1e3, z_steady_RK4*1e9, 'r-');
plt.ylim(-65,65);
plt.show();
```

### Error Analysis

Let's plot together our solutions using the different schemes along with our analytical reference.

```python
plt.title('Plot 4  Schemes comparison with analytical sol.', fontsize=20);
plt.plot(time_an_steady*1e3, z_an_steady*1e9, 'b--' );
plt.plot(time_steady_V*1e3, z_steady_V*1e9, 'g-' );
plt.plot(time_steady_RK4*1e3, z_steady_RK4*1e9, 'r-');
plt.xlim(2.0, 2.06);
plt.legend(['Analytical solution', 'Verlet method', 'Runge Kutta 4']);
plt.xlabel('time, ms', fontsize=18);
plt.ylabel('z_position, nm', fontsize=18);

```

It was pointless to include Euler in the last plot because it was not following the physics at all for this given time step. REMEMBER that Euler can give fair approximations, but you MUST decrease the time step in this particular case if you want to see the sinusoidal trajectory!
It seems our different schemes are giving different quality in approximating the solution. However it's hard to conclude something strong based on this qualitative observations. In order to state something stronger we have to perform further error analysis. We will do this at the end of the notebook after the references and will choose L1 norm for this purpose (You can find more information about this [L1](http://en.wikipedia.org/wiki/Taxicab_geometry) ).

As we can see Runge Kutta 4 converges faster than Verlet for the range of time steps studied. And the difference between both is near one order of magnitude. One additional advantage with Runge Kutta 4 is that the method is very stable, even with big time steps (eg. 10 time steps per period) the method is able to catch up the physics of the oscillation, something where Verlet is not so good at.

### Let's add a sample and oscillate our probe over it

It is very common in the field of probe microscopy to model the tip sample interactions through DMT contact mechanics.
DMT stands for Derjaguin, Muller and Toporov who were the scientists that developed the model (see ref 1). This model uses Hertz contact mechanics (see ref 2) with the addition of long range tip-sample interactions. These long range tip-sample interactions are ascribed to intermolecular interactions between the atoms of the tip and the upper atoms of the surface, and include mainly the contribution of van de Waals forces and Pauli repulsion from electronic clouds when the atoms of the tip meet closely the atoms of the surface. Figure 2 displays a force vs distance curve (FD curve) where it is shown how the forces between the tip and the sample behave with respect to the separation. It can be seen that at positive distances the tip starts "feeling" attraction from the tip (from the contribution of van der Waals forces) where the slope of the curve is positive and at some minimum distance ($a_0$) the tip starts experiencing repulsive interactions arising from electronic cloud repulsion (area where the slope of the curve is negative and the forces are negative). At lower distances, an area known as "contact area" arises and it is characterized by a negative slope and an emerging positive force.

```python
fig3 = path + '/Fig3FDcurve.jpg'
Image(filename=fig3)
```

### Figure 3.   Force vs Distance profile depicting tip-sample interactions in AFM (Adapted from reference 6)

In Hertz contact mechanics, one central aspect is to consider that the contact area increases as the sphere is pressed against an elastic surface, and this increase of the contact area "modulates" the effective stiffness of the sample. This concept is represented in figure 4 where the sample is depicted as comprised by a series of springs that are activated as the tip goes deeper into the sample. In other words, the deeper the sample goes, the larger the contact area and therefore more springs are activated (see more about this on reference 5).

```python
fig4 = path + '/Fig4Hertzspring.jpg'
Image(filename= fig4)
```

### Figure 4.  Conceptual representation of Hertz contact mechanics

This concept is represented mathematically by a non-linear spring whose elastic coefficient is a function of the contact area which at the same time depends on the sample indentation ( k(d)  ).
$$F_{ts} = k(d)d$$
where
$$k(d) = 4/3E*\sqrt{Rd}$$
being $\sqrt{Rd}$ the contact area when a sphere of radius R indents a half-space to depth d.
$E*$ is the effective Young's modulus of the tip-sample interaction.

The long range attractive forces are derived using Hamaker's equation (see reference 4): $if$ $d > a_0$
$$F_{ts} = \frac{-HR}{6d^2}$$

where H is the Hamaker constant, R the tip radius and d the tip sample distance. $a_0$ is defined as the intermolecular distance and normally is chosen to be 0.2 nm.

In summary the equations that we will include in our code to take care of the tip sample interactions are the following:

$$\begin{equation}
Fts_{DMT} = \begin{cases} \frac{-HR}{6d^2} \quad \quad d \leq{a_0}\\ \\
\frac{-HR}{6d^2} + 4/3E*R^{1/2}d^{3/2}   \quad \quad d> a_0 \end{cases}
\end{equation}$$

where the effective Young's modulus E* is defined by:
$$\begin{equation}
1/E* = \frac{1-\nu^2}{E_t}+\frac{1-\nu^2}{E_s}
\end{equation}$$
where $E_t$ and $E_s$ are the tip and sample Young's modulus respectively. $\nu_t$ and $\nu_s$ are tip and sample Poisson ratios, respectively.

##### Enough theory, Let's make our code!

Now we will have to solve equation (1) but with the addition of tip-sample interactions which are described by equation (5). So we have a second order non-linear ODE which is no longer analytically straightforward:

$$\begin{equation}
m \frac{d^2z}{dt^2} = - k z - \frac{m\omega_0}{Q}\frac{dz}{dt} + F_0 cos(\omega t) + Fts_{DMT}
\end{equation}$$

Therefore we have to use numerical methods to solve it. RK4 has shown to be more accurate to solve equation (1) among the methods reviewed in the previous section of the notebook, and therefore it is going to be the chosen method to solve equation (6).

Now we have to declare all the variables related to the tip-sample forces. Since we are modeling our tip-sample forces using Hertz contact mechanics with addition of long range Van der Waals forces we have to define the Young's modulus of the tip and sample, the diameter of the tip of our probe, Poisson ratio, etc.

```python
#DMT parameters  (Hertz contact mechanics with long range Van der Waals forces added
a=0.2e-9 #intermolecular parameter
H=6.4e-20 #hamaker constant of sample
R=20e-9 #tip radius of the cantilever
Es=70e6 #elastic modulus of sample
Et=130e9 #elastic modulus of the tip
vt=0.3 #Poisson coefficient for tip
vs=0.3 #Poisson coefficient for sample
E_star= 1/((1-pow(vt,2))/Et+(1-pow(vs,2))/Es)  #Effective Young Modulus
```

Now let's declare the timestep, the simulation time and let's oscillate our probe!

```python
#IMPORTANT distance where you place the probe above the sample
z_base = 40.e-9

spp = 280. # time steps per period
dt = period/spp

simultime = 100.*period
N = int(simultime/dt)
t = numpy.linspace(0,simultime,N)

#Initializing variables for RK4
v_RK4 = numpy.zeros(N)
z_RK4 = numpy.zeros(N)
k1v_RK4 = numpy.zeros(N)
k2v_RK4 = numpy.zeros(N)
k3v_RK4 = numpy.zeros(N)
k4v_RK4 = numpy.zeros(N)

k1z_RK4 = numpy.zeros(N)
k2z_RK4 = numpy.zeros(N)
k3z_RK4 = numpy.zeros(N)
k4z_RK4 = numpy.zeros(N)

TipPos = numpy.zeros(N)
Fts = numpy.zeros(N)
Fcos = numpy.zeros(N)

for i in range(1,N):
   #RK4
    k1z_RK4[i] = v_RK4[i-1]   #k1 Equation 14
    k1v_RK4[i] = ((   ( -k*z_RK4[i-1] - (m*wo/Q)*v_RK4[i-1] + \
                       Fd*numpy.cos(wo*t[i-1]) +Fts[i-1])  ) / m )   #m1 Equation 15

    k2z_RK4[i] = ((v_RK4[i-1])+k1v_RK4[i]/2.*dt) #k2 Equation 16
    k2v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k1z_RK4[i]/2.*dt) - (m*wo/Q)*\
                       (v_RK4[i-1] +k1v_RK4[i]/2.*dt) + Fd*\
                       numpy.cos(wo*(t[i-1] + dt/2.))  +Fts[i-1])  ) / m )  #m2 Eq 17

    k3z_RK4[i] = ((v_RK4[i-1])+k2v_RK4[i]/2.*dt) #k3, Equation 18
    k3v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k2z_RK4[i]/2.*dt) - (m*wo/Q)*\
                        (v_RK4[i-1] +k2v_RK4[i]/2.*dt) + Fd*\
                        numpy.cos(wo*(t[i-1] + dt/2.))  +Fts[i-1])  ) / m ) #m3, Eq19

    k4z_RK4[i] = ((v_RK4[i-1])+k3v_RK4[i]*dt) #k4, Equation 20
    k4v_RK4[i] = ((   ( -k*(z_RK4[i-1] + k3z_RK4[i]*dt) - (m*wo/Q)*\
                       (v_RK4[i-1] + k3v_RK4[i]*dt) + Fd*\
                       numpy.cos(wo*(t[i-1] + dt))  +Fts[i-1])  ) / m )#m4, Eq 21

    #Calculation of velocity, Equation 23
    v_RK4[i] = v_RK4[i-1] + 1./6*dt*(k1v_RK4[i] + 2.*k2v_RK4[i] +\
                    2.*k3v_RK4[i] + k4v_RK4[i] )
    #calculation of position, Equation 22
    z_RK4 [i] = z_RK4[i-1] + 1./6*dt*(k1z_RK4[i] + 2.*k2z_RK4[i] +\
                     2.*k3z_RK4[i] + k4z_RK4[i] )

    TipPos[i] = z_base + z_RK4[i] #Adding base position to z position

    #calculation of DMT force

    if  TipPos[i] > a: #this defines the attractive regime
        Fts[i] = -H*R/(6*(TipPos[i])**2)
    else:  #this defines the repulsive regime
        Fts[i] = -H*R/(6*a**2)+4./3*E_star*numpy.sqrt(R)*(a-TipPos[i])**1.5

    Fcos[i] = Fd*numpy.cos(wo*t[i])  #Driving force (this will be helpful to plot the driving force)

#Slicing arrays to get steady state
TipPos_steady = TipPos[int(95*period/dt):]
t_steady = t[int(95*period/dt):]
Fcos_steady = Fcos[int(95*period/dt):]
Fts_steady = Fts[int(95*period/dt):]

plt.figure(1)
fig, ax1 = plt.subplots()
ax2 = ax1.twinx()
ax1.plot(t_steady*1e3,TipPos_steady*1e9, 'g-')
ax2.plot(t_steady*1e3, Fcos_steady*1e9, 'b-')
ax1.set_xlabel('Time,s')
ax1.set_ylabel('Tip position (nm)', color='g')
ax2.set_ylabel('Drive Force (nm)', color='b')
plt.title('Plot 7   Tip response and driving force', fontsize = 20)

```

```python
plt.figure(2)
plt.title('Plot 8  Force-Distance curve', fontsize=20)
plt.plot(TipPos*1e9, Fts*1e9, 'b--' )
plt.xlabel('Tip Position, nm', fontsize=18)
plt.ylabel('Force, nN', fontsize=18)
plt.xlim(-20, 30)
```

Check that we have two sinusoidals. The one in green (the output) is the response signal of the tip (the tip trajectory in time) while the blue one (the input) is the cosinusoidal driving force that we are using to excite the tip. When the tip is excited in free air (without tip sample interactions) the phase lag between the output and the input is 90 degrees. You can test that with the previous code by only changing the position of the base to a high-enough position that it does not interact with the sample. However in the above plot the phase lag is less than 90 degrees. Interestingly the phase can give relative information about the material properties of the sample. There is a well-developed theory of this in tapping mode AFM and it's called phase spectroscopy. If you are interested in this topic you can read reference 1.
Also look at the above plot and see that the response amplitude is no longer 60 nm as we initially set (in this case is near 45 nm!). It means that we have experienced a significant amplitude reduction due to the tip sample interactions.
Besides with the data acquired we are able to plot a Force-curve as the one shown in Figure 3. It shows the attractive and repulsive interactions of our probe with the surface.

We have arrived to the end of the notebook. I hope you have found it interesting and helpful!

REFERENCES

1. Garcı́a, Ricardo, and Ruben Perez. "Dynamic atomic force microscopy methods." Surface science reports 47.6 (2002): 197-301.
2. B. V. Derjaguin, V. M. Muller, and Y. P. Toporov, J. Colloid
Interface Sci. 53, 314  (1975)
3. Hertz, H. R., 1882, Ueber die Beruehrung elastischer Koerper (On Contact Between Elastic Bodies), in Gesammelte Werke (Collected Works), Vol. 1, Leipzig, Germany, 1895.
4. Van Oss, Carel J., Manoj K. Chaudhury, and Robert J. Good. "Interfacial Lifshitz-van der Waals and polar interactions in macroscopic systems." Chemical Reviews 88.6 (1988): 927-941.
5. Enrique A. López-Guerra, and Santiago D. Solares. "Modeling viscoelasticity through spring–dashpot models in intermittent-contact atomic force microscopy." Beilstein journal of nanotechnology 5, no. 1 (2014): 2149-2163.
6. Enrique A. López-Guerra, and Santiago D. Solares, "El microscopio de Fuerza Atómica: Metodos y Aplicaciones." Revista UVG (2013) No. 28, 14-23.

### OPTIONAL: Further error analysis based in norm L1

```python
print('This cell takes a while to compute')

"""ERROR ANALYSIS EULER, VERLET AND RK4"""

# time-increment array
dt_values = numpy.array([8.0e-7, 2.0e-7, 0.5e-7, 1e-8, 0.1e-8])

# array that will contain solution of each grid
z_values_E = numpy.zeros_like(dt_values, dtype=numpy.ndarray)
z_values_V = numpy.zeros_like(dt_values, dtype=numpy.ndarray)
z_values_RK4 = numpy.zeros_like(dt_values, dtype=numpy.ndarray)
z_values_an = numpy.zeros_like(dt_values, dtype=numpy.ndarray)

for n, dt in enumerate(dt_values):
    simultime = 100*period
    timestep = dt
    N = int(simultime/dt)
    t = numpy.linspace(0.0, simultime, N)

    #Initializing variables for Verlet
    zdoubledot_V = numpy.zeros(N)
    zdot_V = numpy.zeros(N)
    z_V = numpy.zeros(N)

    #Initializing variables for RK4
    vdot_RK4 = numpy.zeros(N)
    v_RK4 = numpy.zeros(N)
    z_RK4 = numpy.zeros(N)
    k1v_RK4 = numpy.zeros(N)
    k2v_RK4 = numpy.zeros(N)
    k3v_RK4 = numpy.zeros(N)
    k4v_RK4 = numpy.zeros(N)

    k1z_RK4 = numpy.zeros(N)
    k2z_RK4 = numpy.zeros(N)
    k3z_RK4 = numpy.zeros(N)
    k4z_RK4 = numpy.zeros(N)

    #Initial conditions Verlet (started with Euler approximation)
    z_V[0] = 0.0
    zdot_V[0] = 0.0
    zdoubledot_V[0] = (   ( -k*z_V[0] - (m*wo/Q)*zdot_V[0] + \
                           Fd*numpy.cos(wo*t[0]) )  ) / m
    zdot_V[1] = zdot_V[0] + zdoubledot_V[0]*timestep**2
    z_V[1] = z_V[0] + zdot_V[0]*dt
    zdoubledot_V[1] = (   ( -k*z_V[1] - (m*wo/Q)*zdot_V[1] + \
                           Fd*numpy.cos(wo*t[1]) )  ) / m

    #Initial conditions Runge Kutta
    v_RK4[1] = 0
    z_RK4[1] = 0

    #Initialization variables for Analytical solution
    z_an = numpy.zeros(N)

    # time loop
    for i in range(2,N):

        #Verlet
        z_V[i] = 2*z_V[i-1] - z_V[i-2] + zdoubledot_V[i-1]*dt**2 #Eq 10
        zdot_V[i] = (z_V[i]-z_V[i-2])/(2.0*dt) #Eq 11
        zdoubledot_V[i] = (   ( -k*z_V[i] - (m*wo/Q)*zdot_V[i] +\
                                   Fd*numpy.cos(wo*t[i]) )  ) / m #from eq 1

        #RK4
        k1z_RK4[i] = v_RK4[i-1]   #k1 Equation 14
        k1v_RK4[i] = ((   ( -k*z_RK4[i-1] - (m*wo/Q)*v_RK4[i-1] + \
                           Fd*numpy.cos(wo*t[i-1]) )  ) / m )   #m1 Equation 15

        k2z_RK4[i] = ((v_RK4[i-1])+k1v_RK4[i]/2.*dt) #k2 Equation 16
        k2v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k1z_RK4[i]/2.*dt) - (m*wo/Q)*\
                           (v_RK4[i-1] +k1v_RK4[i]/2.*dt) + Fd*\
                           numpy.cos(wo*(t[i-1] + dt/2.)) )  ) / m )  #m2 Eq 17

        k3z_RK4[i] = ((v_RK4[i-1])+k2v_RK4[i]/2.*dt) #k3, Equation 18
        k3v_RK4[i] = ((   ( -k*(z_RK4[i-1]+ k2z_RK4[i]/2.*dt) - (m*wo/Q)*\
                            (v_RK4[i-1] +k2v_RK4[i]/2.*dt) + Fd*\
                            numpy.cos(wo*(t[i-1] + dt/2.)) )  ) / m ) #m3, Eq 19

        k4z_RK4[i] = ((v_RK4[i-1])+k3v_RK4[i]*dt) #k4, Equation 20
        k4v_RK4[i] = ((   ( -k*(z_RK4[i-1] + k3z_RK4[i]*dt) - (m*wo/Q)*\
                           (v_RK4[i-1] + k3v_RK4[i]*dt) + Fd*\
                           numpy.cos(wo*(t[i-1] + dt)) )  ) / m )#m4, Equation 21

        #Calculation of velocity, Equation 23
        v_RK4[i] = v_RK4[i-1] + 1./6*dt*(k1v_RK4[i] + 2.*k2v_RK4[i] +\
                        2.*k3v_RK4[i] + k4v_RK4[i] )
        #calculation of position, Equation 22
        z_RK4 [i] = z_RK4[i-1] + 1./6*dt*(k1z_RK4[i] + 2.*k2z_RK4[i] +\
                         2.*k3z_RK4[i] + k4z_RK4[i] )

        #Analytical solution
        A_an = Fo_an*Q/k  #when driven at resonance A is simply Fo*Q/k
        phi = numpy.pi/2  #when driven at resonance the phase is pi/2
        z_an[i] = A_an*numpy.cos(wo*t[i] - phi)  #Analytical solution eq. 1

    #Slicing the full response vector to get the steady state response
    z_steady_V = z_V[int(80*period/timestep):]
    z_an_steady = z_an[int(80*period/timestep):]
    z_steady_RK4 = z_RK4[int(80*period/timestep):]
    time_steady = t[int(80*period/timestep):]

    z_values_V[n] = z_steady_V.copy()  # error for certain value of timestep
    z_values_RK4[n] = z_steady_RK4.copy() #error for certain value of timestep
    z_values_an[n] = z_an_steady.copy()  #error for certain value of timestep

def get_error(z, z_exact, dt):
    #Returns the error with respect to the analytical solution using L1 norm

    return dt * numpy.sum(numpy.abs(z-z_exact))

#NOW CALCULATE THE ERROR FOR EACH RESPECTIVE DELTA T
error_values_V = numpy.zeros_like(dt_values)
error_values_RK4 = numpy.zeros_like(dt_values)

for i, dt in enumerate(dt_values):
    ### call the function get_error() ###
    error_values_V[i] = get_error(z_values_V[i], z_values_an[i], dt)
    error_values_RK4[i] = get_error(z_values_RK4[i], z_values_an[i], dt)

plt.figure(1)
plt.title('Plot 5  Error analysis Verlet based on L1 norm', fontsize=20)
plt.tick_params(axis='both', labelsize=14)
plt.grid(True)                         #turn on grid lines
plt.xlabel('$\Delta t$ Verlet', fontsize=16)  #x label
plt.ylabel('Error Verlet', fontsize=16)       #y label
plt.loglog(dt_values, error_values_V, 'go-')  #log-log plot
plt.axis('equal')                      #make axes scale equally;

plt.figure(2)
plt.title('Plot 6  Error analysis RK4 based on L1 norm', fontsize=20)
plt.tick_params(axis='both', labelsize=14)
plt.grid(True)     #turn on grid lines
plt.xlabel('$\Delta t$ RK4', fontsize=16)  #x label
plt.ylabel('Error RK4', fontsize=16)       #y label
plt.loglog(dt_values, error_values_RK4,  'co-')  #log-log plot
plt.axis('equal')                      #make axes scale equally;
```


## Exercise 6
* Due: 17.2. at 14:00
* Max points: 8

## General rules
* **Submit your answers by using Moodle.** The link is https://moodle.jyu.fi/mod/assign/view.php?id=693972&forceview=1.
* Name your file as *lastname_ex6.xxx*. If you have multiple files, use *lastname_ex6_1.xxx, lastname_ex6_2.xxx* etc.
* **Remember to add comments to your answers and include also some testing part!**

## Exercises

For exercises 1-2, we study optimization problem
$$
\begin{align}
\min \qquad & x_1^2+x_2^2 + x_3^3+(1-x_4)^2\\
\text{s.t.}\qquad &x_1^2+x_2^2-1=0\\
    &x_1^2+x_3^2-1=0\\
    &x\in\mathbb R^4
\end{align}
$$

1. **(2 points)** Use the SQP method to solve the above problem. **Analyze carefully the result you got!** How does SQP work for this problem?

2. **(2 points)** Use the augmented Lagrangian method to solve the above problem. **Analyze carefully the result you got!** How does the method work for this problem?

3. **(2 points)** Study multiobjective optimization problem
$$
\begin{align}
\min & \{\|x-(1,0)\|^2,\|x-(0,1)\|^2\}\\
\text{s.t. }&x\in \mathbb R^2.
\end{align}
$$
Characterize algebraically  (i.e. give a mathematical formulation) the full set of Pareto optimal solutions.

4. **(2 points)** Calculate the ideal and nadir vectors for the above bi-objective problem. You can use any methods available.