# Quick Start: 単振り子の運動をシミュレーション

* Next >> None.
* Prev >> [1_modeling](https://github.com/yfur/basic-mechanics-python/blob/master/1_modeling/1_modeling.ipynb)

[単振り子](https://ja.wikipedia.org/wiki/%E6%8C%AF%E3%82%8A%E5%AD%90#.E5.8D.98.E6.8C.AF.E3.82.8A.E5.AD.90) の運動をシミュレーションする．

## 0. はじめに

一言に力学シミュレーションといっても，その方法は多様であり，いくつもの方法があるのだが，そのうちの一つを例としてあげる．シミュレーションを行うために必要な作業を

1. ** モデル化 **
2. ** 数値計算 **
3. ** 視覚化 **

の3つのプロセスに分けて考える．

まず，一つ目の「モデル化」について．これは，例えば考える物理モデルからニュートンの運動方程式を導くプロセスのことである．

次に，二つ目の「数値計算」について．これは，Python プログラムを用いてコンピュータ上で運動を計算させるプロセスのことである．このプロセスでコンピュータは振り子の運動を与えられたモデル情報に基づいて計算し，その結果を数値で示す．例えば

> シミュレーション開始0秒後，振り子は10 [rad]の位置にいる
>
> シミュレーション開始1秒後，振り子は0 [rad]の位置にいる
>
> シミュレーション開始2秒後，振り子は-10 [rad]の位置にいる
>
> シミュレーション開始3秒後，振り子は0 [rad]の位置にいる
>
> ...

という情報が，

| 時間 | 位置 |
|:------:|:---------:|
| 0     |  10      |
| 1     | 0     |
| 2     | -10      |
| 3     | 0     |

のような行列となって生成される．

最後に，「視覚化」について．「数値計算」によって得られた数値のデータは，あくまで数字の羅列であり，これだけでは物体の運動を理解しようとするのは難しい．そこで，運動を直感的に理解できるようにするために，この数値データからグラフやアニメーションを生成する．

## 1. モデル化

図のような，よくある [単振り子の力学モデル](https://ja.wikipedia.org/wiki/%E6%8C%AF%E3%82%8A%E5%AD%90#.E5.8D.98.E6.8C.AF.E3.82.8A.E5.AD.90) を考える．振り子の角度を$\theta$，振り子の長さを$l$，振り子の先端のおもりの質量を$m$，鉛直下向きにはたらく重力の重力加速度を$g$としている．

おもりの運動方向にはたらく力は
\begin{align}
F = -mg\sin\theta
\end{align}
である．おもりの運動方向の加速度は$l\ddot{\theta}$なので，ニュートンの運動方程式 ($ma=F$) は，
\begin{align}
m l\ddot{\theta}= -mg\sin\theta
\end{align}
したがって，
\begin{align}
\ddot{\theta} = -\frac{g}{l}\sin\theta
\end{align}
である．

さて，単振り子のモデルを考える上で，時間の経過とともに値が変化する　**変数** と，時間の経過に関わらず値が変化しない **定数** が存在した．$\theta$ が前者であり，$m$，$l$，$g$が後者である．** 1. モデル化 ** のプロセスで達成することは，運動方程式を**変数** について整理し，「（変数の時間の微分項）$=$（それ以外の項）」という式を得ることである．

## 2. 数値計算

**1. モデル化** では変数$\theta$についての運動方程式を導いたが， **2. 数値計算** では，これを用いて **常微分方程式** を構築する．

まず，$\theta$自身とその一階微分である$\dot{\theta}$を縦に並べた列ベクトルをつくり，これを$s$とする．（GiuHub上で行列が上手く表示されないのは仕方のないことなのかな？）
\begin{align}
  s =
  \begin{bmatrix}
    \theta \\ \dot{\theta}
  \end{bmatrix}
\end{align}
そして，$s$の時間の一階微分を式で表す．ここで，$\ddot{\theta}$は先ほど求めた運動方程式によって，$\ddot{\theta}$を含まない形で表すことができる．
\begin{align}
  \frac{d}{dt} s =
  \begin{bmatrix}
    \dot{\theta} \\ \ddot{\theta}
  \end{bmatrix} =
  \begin{bmatrix}
    \dot{\theta} \\ -\frac{g}{l}\sin\theta
  \end{bmatrix}
\end{align}
こうして得られた常微分方程式は，数値計算に適した形となっている．具体的には，$\theta = $ `s[0]` ，$\dot{\theta} = $ `s[1]` とすることで，

```
def odefunc(s, t):
    theta = s[0]
    dtheta = s[1]
    ddtheta = -g/l*sin(theta) # <- Equation of motion
    return np.r_[dtheta, ddtheta]
```
のようにして常微分方程式をPythonの関数として記述したうえで

```
s = odeint(odefunc, s_init, t)
```
を実行することで常微分方程式を数値的に解くことができる．ただし [`odeint`](https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.integrate.odeint.html) は [SciPy](https://www.scipy.org/) ライブラリの `integrate` クラスで定義される，常微分方程式を解く関数である．

```python
import numpy as np
from scipy.integrate import odeint
from math import sin

''' constants '''
m = 1 # mass of the pendulum [kg]
l = 1 # length of the pendulum [m]
g = 10 # Gravitational acceleration [m/s^2]

''' time setting '''
t_end = 10 # simulation time [s]
t_fps = 50 # frame per second. This value means smoothness of produced graph and animation
t_step = 1/t_fps
t = np.arange(0, t_end, t_step)

''' initial value '''
theta_init = 0 # initial value of theta [rad]
dtheta_init = 1 # initial value of dot theta [rad/s]
s_init = np.r_[theta_init, dtheta_init]

def odefunc(s, t):
    theta = s[0]
    dtheta = s[1]
    ddtheta = -g/l*sin(theta) # <- Equation of motion
    return np.r_[dtheta, ddtheta]

s = odeint(odefunc, s_init, t)
print('ODE calculation finished.')
```

    ODE calculation finished.

以上が数値計算を行うプログラムである．
```
print(np.c_[t, s])
```
を実行することで，

| 時間 | 振子の角度 | 振子の角速度 |
|:------:|:---------:|:---------:|
| 0.     |  0.      | 1. |
|... | ... | ... |
| 9.98   | -0.018 | 0.998 |

のように，それぞれの時間で振子の角度がどのようになっていて，また振子の角速度がどのようになっているのかを確認することができる．

```python
print(np.c_[t, s])
```

    [[ 0.          0.          1.        ]
     [ 0.02        0.01998668  0.99800074]
     [ 0.04        0.03989343  0.99201173]
     ...,
     [ 9.94       -0.05725082  0.98347963]
     [ 9.96       -0.03747992  0.99295215]
     [ 9.98       -0.01755919  0.9984571 ]]

## 3. 可視化

数値計算が完了したものの，数字の羅列では運動の様子が直感的には理解できない．そこで，振り子の運動の様子を表すアニメーション動画や，振子の角度の時間変化を表すグラフを作る．

以下に示す`animfunc` はアニメーション動画を生成する関数である．基本的な原理はパラパラ漫画であり，動画フレームごとに振り子の絵を描いては保存するの繰り返しを行っている．フレームごとの更新は`update_figure`関数が行っている．

```python
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from math import cos

def animfunc(s, t):
    ''' Create mp4 movie file of a pendulum '''
    plt.close()
    fig = plt.figure()
    plt.axis('scaled')
    plt.xlim(-1, 1)
    plt.ylim(-1.5, .5)
    plt.grid('on')
    draw_ceiling, = plt.plot([-2, 2], [0, 0], c='k', lw=2)
    draw_pendulum, = plt.plot([], [], lw=4, c='b')
    draw_mass, = plt.plot([], [], lw=2, marker='o', ms=20, mew=4, mec='b', mfc='c')
    indicate_time = plt.text(-0.3, 0.25, [], fontsize=12)

    def update_figure(i):
        ''' Set data of each movie frame '''
        mass_x = l*sin(s[i, 0])
        mass_y = - l*cos(s[i, 0])
        pendlum_x = [0, mass_x]
        pendlum_y = [0, mass_y]
        draw_pendulum.set_data(pendlum_x, pendlum_y)
        draw_mass.set_data(mass_x, mass_y)
        indicate_time.set_text('t = {0:4.2f} [s]'.format(t[i]))

    ''' Create a movie file '''
    line_ani = animation.FuncAnimation(fig, update_figure, frames=len(t))
    line_ani.save('./pendulum.mp4', fps=t_fps)
    print('pendulum.mp4 created')
```

`animfunc`を実行することで，`pendulum.mp4`のようなアニメーション動画が保存される．

```python
animfunc(s, t)
```

    pendulum.mp4 created

一方，振子の動きをグラフに示したい場合は，例えば次のようにしてプロットを行えば良い．次のプログラムを実行することで，`pendulum_graph.png`のようなイメージファイルが保存される．

```python
plt.figure()
plt.plot(t, s[:, 0])
plt.xlabel('t [s]')
plt.ylabel('theta [rad]')
plt.savefig('pendulum_graph.png')
```

## おわりに

以上が，Python で単振り子の運動をシミュレーションを行う例である．シミュレーションを行う上で最低限必要となるの要素のみを取り出してコードを書いたのだが，より高度な計算や表現を行うことはもちろん可能である．

## 参考文献

###### todo
* 単振り子の完成版プログラム
* 参考情報の追加

```python

```


```python
import sympy as sm
import matplotlib.pyplot as plt
import numpy as np
from chempy import ReactionSystem
from chempy.units import to_unitless, SI_base_registry as si, default_units as u, default_constants as const
from chempy.kinetics.ode import get_odesys
from chempy.kinetics.integrated import binary_rev
sm.init_printing()
%matplotlib inline
```

```python
R = 8.314472
T_K = 273.15 + 20  # 20 degree celsius
kB = 1.3806504e-23
h = 6.62606896e-34

dHf = 74e3
dSf = R*np.log(h/kB/T_K*1e16)

dHb = 79e3
dSb = dSf - 23

rsys1 = ReactionSystem.from_string("""
Fe+3 + SCN- -> FeSCN+2; EyringParam(dH={dHf}*J/mol, dS={dSf}*J/K/mol)
FeSCN+2 -> Fe+3 + SCN-; EyringParam(dH={dHb}*J/mol, dS={dSb}*J/K/mol)
""".format(dHf=dHf, dSf=dSf, dHb=dHb, dSb=dSb))
kf_ref = 20836643994.118652*T_K*np.exp(-(dHf - T_K*dSf)/(R*T_K))
kb_ref = 20836643994.118652*T_K*np.exp(-(dHb - T_K*dSb)/(R*T_K))
kf_ref, kb_ref
```

```python
Fe0 = 6e-3
SCN0 = 2e-3
init_cond = {
    'Fe+3': Fe0*u.M,
    'SCN-': SCN0*u.M,
    'FeSCN+2': 0*u.M
}
t = 3*u.second
```

```python
def integrate_and_plot(rsys, params):
    odes, extra = get_odesys(rsys, include_params=False, unit_registry=si, constants=const)
    fig, all_axes = plt.subplots(2, 3, figsize=(14, 6))
    for axes, odesys in zip(all_axes, [odes, odes.as_autonomous()]):
        res = odesys.integrate(t, init_cond, params, integrator='cvode')
        t_sec = to_unitless(res.xout, u.second)
        FeSCN_ref = binary_rev(t_sec, kf_ref, kb_ref, 0, Fe0, SCN0)
        cmp = to_unitless(res.yout, u.M)
        ref = np.empty_like(cmp)
        ref[:, odesys.names.index('FeSCN+2')] = FeSCN_ref
        ref[:, odesys.names.index('Fe+3')] = Fe0 - FeSCN_ref
        ref[:, odesys.names.index('SCN-')] = SCN0 - FeSCN_ref
        axes[0].plot(t_sec, cmp)
        axes[1].plot(t_sec, cmp - ref)
        res.plot_invariant_violations(ax=axes[2])
        assert np.allclose(cmp, ref)
        print({k: v for k, v in res.info.items() if not k.startswith('internal')})
```

```python
integrate_and_plot(rsys1, {'temperature': T_K*u.K})
```

```python
rsys2 = ReactionSystem.from_string("""
Fe+3 + SCN- -> FeSCN+2; MassAction(EyringHS([{dHf}*J/mol, {dSf}*J/K/mol]))
FeSCN+2 -> Fe+3 + SCN-; MassAction(EyringHS([{dHb}*J/mol, {dSb}*J/K/mol]))
""".format(dHf=dHf, dSf=dSf, dHb=dHb, dSb=dSb))
```

```python
integrate_and_plot(rsys2, {'temperature': T_K*u.K})
```

```python
rsys3 = ReactionSystem.from_string("""
Fe+3 + SCN- -> FeSCN+2; MassAction(EyringHS.fk('dHf', 'dSf'))
FeSCN+2 -> Fe+3 + SCN-; MassAction(EyringHS.fk('dHb', 'dSb'))
""")
```

```python
integrate_and_plot(rsys3, dict(temperature=T_K*u.K,
                               dHf=dHf*u.J/u.mol, dSf=dSf*u.J/u.mol/u.K,
                               dHb=dHb*u.J/u.mol, dSb=dSb*u.J/u.mol/u.K))
```


```python
import numpy as np                # Library used to perform scientific computation
import pandas as pd               # Library used to build data structures
import matplotlib.pyplot as plt   # Library used to plot data
import time                       # Library used to evaluate computation time
import copy                       # Library used to perform the deepcopy and avoid
from sympy import *               # Library used to manage visualize formulas and perform differentiation
init_printing()                   # Enable sympy LaTex print
plt.rcParams['figure.figsize'] = [10, 10] # Set plot size
ITERATIONS = 13                   # Constant number of iterations
```

## Tool setup
In this initial part we are going to develop and test some of the functions that will be useful for the computation of the Inexact Newton method

Initialization of the starting parameter vector $\omega$ and initialization of the conjugate gradient guess vector $d$.

```python
wVal=np.array([[4.] ,[2.]])
dVal=np.array([[4.] ,[3.]])
```

Creating symbol vectors to mantain the functions in verbose form and to differentiate  $F(\omega)$

```python
w=[]
d=[]
nVar=2
# Creating array symbols w and d in order to perform computation on functions using sympy
for i in range(len(wVal)):
    w.append(symbols('w%d'%i))
    d.append(symbols('d%d'%i))
w=np.array(w)
d=np.array(d)
fun = w[0]**2*w[1]**2  # F(w) Function Initianization
fun.evalf()            # Function Evaluation
```

Defining the gradient function, input: function to differentiate and symbols array containing the variables to differentiate on.

Computation of $\phi(\omega ;d)= \nabla F(\omega)^Td$ using the defined gradient function.

```python
def gradient(fun,w):
    res = []
    for i in range(len(w)):
        res.append([diff(fun,w[i])])   # Perform differentiation on the function fun in each variable of the array w
    return np.array(res)
```

```python
phi=np.dot(gradient(fun,w).T,d) # Performing dot product on the gradient of my starting function and
phi[0].evalf()
```

Computation of $\nabla_\omega \phi(\omega ;d)= \nabla^2 F(\omega)d$ to create the Hessian Vector

```python
gradphi=gradient(phi,w)                  #Computing the Hessian Vector product
g=gradient(fun,w)
gradphi=np.squeeze(gradphi, axis=1)      #Vertical squeeze to obtain  correct function formatting
print "Hessian vector of F(w) in function of w and d:",gradphi
```

    Hessian vector of F(w) in function of w and d:
    2.0*d0*w1**2 + 4.0*d1*w0*w1

Substitute the parameter vector values in the Hessian Vector and gradient so that the gradient will have numeric values and the Hessian Vector will be in function of the variable d

```python
#Substitute value of w in the function
for i in range(len(w)):
    for j in range(len(w)):
        gradphi[i]= gradphi[i][0].subs(w[j], wVal[j]) #Substituting parameter vector value in the Hessian vector
        g[i]=g[i][0].subs(w[j], wVal[j]) #Substituting parameter vector values in the gradient of the function
print "Hessian vector of F(w) in function of d:",gradphi
print "Gradient F(w):",g
```

    Hessian vector of F(w) in function of d: [[8.0*d0 + 32.0*d1]
     [32.0*d0 + 32.0*d1]]
    Gradient F(w): [[32.0000000000000]
     [64.0000000000000]]

Defining solveHessianVector(...) function that solve the Hessian-Vector substituting the values of the guess vector d

```python
def solveHessianVector(gradphi,dVal):
    res = copy.deepcopy(gradphi) # Using deepcopy to copy the variable by value and not reference
    d=[]
    for i in range(len(dVal)): #Filling symbols vector based on the number of parameters used
        d.append(symbols('d%d'%i))
    d=np.array(d)
    for i in range(len(dVal)):
        for j in range(len(dVal)):
            res[i][0]= res[i][0].subs(d[j], dVal[j])   #Substituting guess value vector value in the Hessian vector
    return res
solveHessianVector(gradphi,-g)   #Output example
```

    array([[-2304.00000000000],
           [-3072.00000000000]], dtype=object)

## Conjugate Gradient:

Implementation of the iterative conjugate gradient method to solve $Ax=b$ where $A$ is the matrix, $b$ the array and $x$ is the wanted array value.

```python
def conjugateGradient(A,b,x):
    r = b - np.dot(A, x)
    p=r
    rOld = np.dot(r.T,r)
    for i in range(len(b)):     #Stop iteration corresponding to the size of b
        Ap = np.dot(A, p)
        alpha = (rOld/np.dot(p.T,Ap))
        x = x+alpha*p
        r = r - alpha*Ap
        rNew = np.dot(r.T,r)
        beta = rNew/rOld
        p = r + beta*p
        rOld = rNew
    return x

# Testing Conjugate Gradient
A = np.array([[4.,1.],[1.,3.]])
b = np.array([[1.],[2.]])
x = np.array([[2.],[1.]])
t1 = time.time()
print 'start'
x=conjugateGradient(A,b,x)
t2 = time.time()
print "Computation time:",t2 - t1
print "x:",x
print "Ax=b:",np.dot(A,x) # Result validation
```

    start
    Computation time: 0.000399827957153
    Steps: [[0.09090909]
     [0.63636364]]
    Ax=b: [[1.]
     [2.]]

## Newton Conjugate Gradient

Defining Newton Conjugate Gradient similarly to the previous conjugate method bu substituting the Hessian vector and gradient and performing compututation on the Hessian-vector.

```python
def NewtonCG(gradphi,b,x):
    r = b - solveHessianVector(gradphi,x) #Substituting dot product with the Hessian vector using the guess values
    p=r
    rOld = np.dot(r.T,r)
    for i in range(len(b)):      #Stop iteration corresponding to the size of b, so of the gradient
        Ap = solveHessianVector(gradphi,p)
        alpha = (rOld/np.dot(p.T,Ap))
        x = x+alpha*p
        r = r - alpha*Ap
        rNew = np.dot(r.T,r)
        if sqrt(rNew) < 1e-10:
              break;
        beta = rNew/rOld
        p = r + beta*p
        rOld = rNew
    return x

# Testing Newton Conjugate Gradient
t1 = time.time()
print gradphi
res=NewtonCG(gradphi,-g,dVal)
t2 = time.time()
print "Computation time:",t2 - t1
print "Steps:",res
```

    [[8.0*d0 + 32.0*d1]
     [32.0*d0 + 32.0*d1]]
    Computation time: 0.0143001079559
    Steps: [[-1.33333333333333]
     [-0.666666666666665]]

Checking if the result of the Newton-CG satisfies $\nabla^2 F(\omega_k)s_k=-\nabla F(\omega_k)$

```python
print "-Gradient:",solveHessianVector(gradphi,res)
```

    -Gradient: [[-32.0000000000000]
     [-63.9999999999999]]

Defining generateFun(), it take as input the number of wanted parameter and return a sympy function that will be used to generate the dataset.

The function depend on w and x, but for visualization reasons we will generate the function using one variable x0 and multiple parameters. The generated function can be changed for mutate the nature of the problem.

```python
def generateFun(nw):
    w=[]
    x=symbols('x0')
    for i in range(nw): #Filling symbols vector based on the number of parameters used
        w.append(symbols('w%d'%i))
    w=np.array(w)
    fun=0
    #function generation
    for i in range(0,nw):
        fun += w[i]*np.power(x, i)
    return fun

#Testing created function
nw=4
fun = generateFun(nw)
fun
```

Defining generateDS(), it takes as inputs a sympy function, w values for generating the points, the size of the dataset and the wanted noise.

```python
def generateDS(fun,wVal,size,noise):
    elem=[]
    y=[]
    clmn =[]
    x=[]
    w=[]
    for i in range(len(wVal)):
        elem.append(np.linspace(-2, 2, size))
        clmn.append('x%d'%i)
        w.append(symbols('w%d'%i))
        x.append(symbols('x%d'%i))
    clmn.append('y')
    res=copy.deepcopy(fun)
    for i in range(size):
        for k in range(len(wVal)):
            res=res.subs(w[k], wVal[k])
            res=res.subs(x[k], elem[k][i])
        y.append(res +np.random.normal()*noise)
        res=copy.deepcopy(fun)
    y=np.squeeze(np.array(y))
    elem.append(y)
    elem=np.array(elem).T
    df= pd.DataFrame(elem, columns=clmn).astype(float)
    return df
```

Defining solveFun() that take as imput a sympy function, a sample of a dataset and the array of values of w.
This function aim is to solve the given function for each point of the dataset and w and avarage them

```python
def solveFun(fun,sample,wVal):
    sumf = 0
    w=[]
    x=[]
    for i in range(len(wVal)):
        w.append(symbols('w%d'%i))
        x.append(symbols('x%d'%i))
    y=symbols('y')
    fun1=copy.deepcopy(fun)
    for i in range(len(sample)):
            for j in range(len(fun1)):
                for k in range(len(w)):
                    fun1[j][0]=fun1[j][0].subs(w[k], wVal[k][0])   #substituting values in the function
                    fun1[j][0]=fun1[j][0].subs(x[k], sample.iloc[i][k])
                fun1[j][0]=fun1[j][0].subs(y, sample.iloc[i][len(w)])
            sumf+=fun1
            fun1=copy.deepcopy(fun)
    sumf=sumf/(2.*len(sample))
    return sumf
```

## Subsampled Hessian-Free Inexact Newton Method
Here we are going to develop the Subsampled Hessian-Free Inexact Newton Method, this function is built referring to the paragraph 6.1.1 of the paper https://arxiv.org/pdf/1606.04838.pdf.
The subSampledHessianFree() perform the main work, instead the selectA() function dinamically select the learning rate.

```python
def selectA(errfun,wVal,res,sampleFut,g):
    a=[0.0001,0.0005,0.001,0.005,0.01,0.05,0.1,0.5,1] # Possible learning rates
    n=0.01
    alpha=0.01
    sumF=0
    sumFP=0
    for i in range(len(a)):
        wVal1=wVal+a[i]*res
        sumF=solveFun(errfun,sampleFut,wVal)
        sumFP=solveFun(errfun,sampleFut,wVal1)
        if (sumFP<sumF+n*a[i]*np.dot(g.T,res)):
            alpha=a[i]
    return alpha
```

```python
def subSampledHessianFree(errfun,df,wVal,empErr):
    #sh<s
    s=40
    sh=20
    dVal=[]
    for i in range(nw):
        dVal.append([1.])
    dVal=np.array(dVal)
    sample=df.sample(s)
    sampleh=df.sample(sh)
    d=[]
    w=[]
    x=[]
    for i in range(nw): #Filling symbols vector based on the number of parameters used
        d.append(symbols('d%d'%i))
        w.append(symbols('w%d'%i))
        x.append(symbols('x%d'%i))
    for i in range(ITERATIONS):
        g=gradient(errfun,w)
        phi=np.dot(g.T,d)
        gradphi=gradient(phi,w)
        gradphi=np.squeeze(gradphi, axis=1) #Vertical squeeze used for good function formatting
        sumg=solveFun(g,sample,wVal)
        sumgradphi = solveFun(gradphi,sampleh,wVal)
        dVal=NewtonCG(sumgradphi,-sumg,dVal)   #compute the conjugate gradient
        sample=df.sample(s) # select a random sample of the dataset of size s
        sampleh=df.sample(sh)
        lr= selectA(np.array([[errfun]]),wVal,dVal,sample,sumg)   #select the best learning rate
        wVal=wVal+lr*dVal
        empErr.append(np.squeeze(solveFun(np.array([[errfun]]),sample,wVal)))
    return wVal
```

## Testing Phase
Perform computations for different number of paramethers and check the results. Here is defined the function print2D to print the data point, the model and the empirical risk.

```python
def print2D(wValInit,wValFin,empErr):
    df.plot.scatter("x0","y",grid=True)
    xdf = pd.DataFrame(np.linspace(-2, 2, 20))
    ax = plt.axes()
    yfin = []
    yinit =[]
    w=[]
    x=[]
    for i in range(nw): #Filling symbols vector based on the number of parameters used
        w.append(symbols('w%d'%i))
        x.append(symbols('x%d'%i))
    w=np.array(w)
    x=np.array(x)
    res1=copy.deepcopy(fun)
    res2=copy.deepcopy(fun)
    for i in range (20):
        res1=res1.subs(x[0], xdf[0][i])
        res2=res2.subs(x[0], xdf[0][i])
        for j in range(nw):
            res1=res1.subs(w[j], wValFin[j][0])
            res2=res2.subs(w[j], wValInit[j][0])
        yfin.append(res1)
        yinit.append(res2)
        res1=copy.deepcopy(fun)
        res2=copy.deepcopy(fun)

    ax.set_title('Model')
    ax.plot(xdf, yfin,color='r',linewidth=2.5)
    #ax.plot(xdf, yinit,color='b',linewidth=2.5)
    fig1, ax2 = plt.subplots()
    xln = np.linspace(0, ITERATIONS, ITERATIONS)
    ax2 = plt.axes()
    ax2.set_title('Empirical Risk')
    ax2.set_xlabel('Iterations')
    ax2.set_ylabel('Empirical Risk')
    plt.grid()
    ax2.plot(xln, empErr,linewidth=2.5,)
```

### 2 Parameter

```python
nw=2
size=400
wVal = np.random.randn(1,nw)[0]
wValInit = []
for i in range(nw):
        wValInit.append([40])
fun = generateFun(nw)
noise=1.5
df=generateDS(fun,wVal,size,noise)
y=symbols('y')
errfun=(fun -y)**2
print "Generated function:",fun
print "Used value of W for generate the Dataset:",wVal
print "Error function:",errfun
```

    Generated function: w0 + w1*x0
    Used value of W for generate the Dataset: [-1.37770485  0.47567684]
    Error function: (w0 + w1*x0 - y)**2

```python
empErr=[]
t1 = time.time()
wValFin=subSampledHessianFree(errfun,df,copy.deepcopy(wValInit),empErr)
t2 = time.time()
print "Computation time:",t2 - t1,"s"
print "Final Parameter Vector:",wValFin
print2D(wValInit,wValFin,empErr)
```

### 3 Parameter

```python
nw=3
size=400
wVal = np.random.randn(1,nw)[0]
wValInit = []
for i in range(nw):
        wValInit.append([40])
fun = generateFun(nw)
noise=2
df=generateDS(fun,wVal,size,noise)
y=symbols('y')
errfun=(fun -y)**2
print "Generated function:",fun
print "Used value of W for generate the Dataset:",wVal
print "Error function:",errfun
```

    Generated function: w0 + w1*x0 + w2*x0**2
    Used value of W for generate the Dataset: [ 0.51977161 -0.58817334 -1.08068713]
    Error function: (w0 + w1*x0 + w2*x0**2 - y)**2

```python
empErr=[]
t1 = time.time()
wValFin=subSampledHessianFree(errfun,df,copy.deepcopy(wValInit),empErr)
t2 = time.time()
print "Computation time:",t2 - t1,"s"
print "Final Parameter Vector:",wValFin
print2D(wValInit,wValFin,empErr)
```

### 4 Parameter

```python
nw=4
size=400
wVal = np.random.randn(1,nw)[0]
wValInit = []
for i in range(nw):
        wValInit.append([40])
fun = generateFun(nw)
noise=2
df=generateDS(fun,wVal,size,noise)
y=symbols('y')
errfun=(fun -y)**2
print "Generated function:",fun
print "Used value of W for generate the Dataset:",wVal
print "Error function:",errfun
```

    Generated function: w0 + w1*x0 + w2*x0**2 + w3*x0**3
    Used value of W for generate the Dataset: [-1.30907949 -2.40480645 -0.34589956 -0.39208418]
    Error function: (w0 + w1*x0 + w2*x0**2 + w3*x0**3 - y)**2

```python
empErr=[]
t1 = time.time()
wValFin=subSampledHessianFree(errfun,df,copy.deepcopy(wValInit),empErr)
t2 = time.time()
print "Computation time:",t2 - t1,"s"
print "Final Parameter Vector:",wValFin
print2D(wValInit,wValFin,empErr)
```

### 5 Parameters

```python
nw=5
size=400
wVal = np.random.randn(1,nw)[0]
wValInit = []
for i in range(nw):
        wValInit.append([40])
fun = generateFun(nw)
noise=2
df=generateDS(fun,wVal,size,noise)
y=symbols('y')
errfun=(fun -y)**2
print "Generated function:",fun
print "Used value of W for generate the Dataset:",wVal
print "Error function:",errfun
```

    Generated function: w0 + w1*x0 + w2*x0**2 + w3*x0**3 + w4*x0**4
    Used value of W for generate the Dataset: [-1.89752    -1.00070927  1.03739835 -0.3824817   1.39208397]
    Error function: (w0 + w1*x0 + w2*x0**2 + w3*x0**3 + w4*x0**4 - y)**2

```python
empErr=[]
t1 = time.time()
wValFin=subSampledHessianFree(errfun,df,copy.deepcopy(wValInit),empErr)
t2 = time.time()
print "Computation time:",t2 - t1,"s"
print "Final Parameter Vector:",wValFin
print2D(wValInit,wValFin,empErr)
```

```python

```

Subsampled Hessian-Free Inexact Newton Method.ipynb


# Black Scholes 模型

Black-Scholes模型是Fischer Black和Myron Scholes在1973的论文"The Pricing of Options and Corporate Liabilities"中提出的。自论文出版以来，这个模型成为了投资者广泛使用的工具。时至今日，它依然是确定期权公允价格的最好的方式之一。

## 假设

* 欧式期权之内在到期日行权
* 在期权的生命周期内，没有股息分派
* 市场走势不能被预测
* 无风险利率和波动率为常数
* 标的物的走势服从对数正态分布

## 无股息 Black-Scholes 公式

在Black-Schole公式中, 定义如下参数.

* $S$, 在时间$t$时的资产价格
* $T$, 期权到期时间. 期权的剩余时间定义为$T−t$
* $K$, 期权行权价
* $r$, 无风险收益率，假设在其在时间段$t$和$T$中为常量
* $\sigma$, 标的资产的波动率, 即标的资产回报的标准差

### $N(d)$是服从正态分布的随机变量$Z$的积累分布函数

$$N(d) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^d e^{-\frac{1}{2}x^2} dx$$

$C(S,t)$是在时间$t$的看涨期权的价值；$P(S,t)$是在时间$t$的看跌期权的价值。

Black-Scholes看涨期权公式是:

$$C(S,t) = SN(d_1) - Ke^{-r(T - t)} N(d_2)$$

看跌期权公式是：

$$P(S,t) = Ke^{-r(T - t)}N(-d_2) - SN(-d_1)$$

其中

$$d_1 = \frac{\ln \left(\frac{S}{K} \right) + \left(r + \frac{\sigma^2}{2} \right)(T - t)}{\sigma \sqrt{T - t}}$$

$$d_2 = d_1 - \sigma \sqrt{T - t} = \frac{\ln \left(\frac{S}{K} \right) + \left(r - \frac{\sigma^2}{2}\right)(T - t)}{\sigma \sqrt{T - t}}$$

## 无股息Black-Scholes公式的Python实现

```python
import numpy as np
import scipy.stats as si
import sympy as sy
import sympy.stats as systats
sy.init_printing()
```

```python
def euro_call(S, K, T, r, sigma):

    #S: spot price
    #K: strike price
    #T: time to maturity
    #r: interest rate
    #sigma: volatility of underlying asset

    N = systats.Normal("x", 0.0, 1.0)

    d1 = (sy.ln(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))
    d2 = (sy.ln(S / K) + (r - 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))

    call = (S * systats.cdf(N)(d1) - K * sy.exp(-r * T) * systats.cdf(N)(d2))

    return call
```

```python
def euro_put(S, K, T, r, sigma):

    #S: spot price
    #K: strike price
    #T: time to maturity
    #r: interest rate
    #sigma: volatility of underlying asset

    N = systats.Normal("x", 0.0, 1.0)

    d1 = (sy.ln(S / K) + (r + 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))
    d2 = (sy.ln(S / K) + (r - 0.5 * sigma ** 2) * T) / (sigma * sy.sqrt(T))

    put = (K * sy.exp(-r * T) * systats.cdf(N)(-d2) - S * systats.cdf(N)(-d1))

    return put
```

测试一下

```python
euro_call(50, 100, 1, 0.05, 0.25)
```

```python
euro_call(50, 100, 1, 0.05, 0.25).evalf()
```

### 符号计算扩展

```python
S, K, T, t, r, sigma = sy.symbols("S K T t r sigma")
call_price = euro_call(S, K, T-t, r, sigma)
```

#### Delta
$$\Delta = \frac{\partial V}{\partial S}$$

```python
delta = sy.diff(call_price, S)
```

#### Gamma
$$\Gamma = \frac{\partial^2 V}{\partial S^2}$$

```python
gamma = sy.diff(call_price, S, 2)
```

#### Theta
$$\Theta = \frac{\partial V}{\partial t}$$

```python
theta = sy.diff(call_price, t)
```

## 图像

```python
import matplotlib
import matplotlib.pyplot as plt
```

```python
# Data for plotting
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(call_price.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='C',
       title='Call Price Value of S')
ax.grid()

plt.show()
```

```python
## Delta Plot
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(delta.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='Delta',
       title='Call Delta Value of S')
ax.grid()

plt.show()
```

```python
## Gamma Plot
s = np.arange(40.0, 200.0, 1.0)
c = []
for s_i in s:
    c.append(gamma.evalf(subs={
        S: s_i,
        K: 100,
        T: 1,
        t: 0.5,
        r: 0.05,
        sigma: 0.25},n=2))

fig, ax = plt.subplots()
ax.plot(s, c)

ax.set(xlabel='S', ylabel='Gamma',
       title='Call Gamma Value of S')
ax.grid()

plt.show()
```

## 内容主要引用自
[Black-Scholes Formula and Python Implementation](https://aaronschlegel.me/black-scholes-formula-python.html)


Exercise material of the MSc-level course **Numerical Methods in Geotechnical Engineering**.
Held at Technische Universität Bergakademie Freiberg.

Comments to:

*Prof. Dr. Thomas Nagel
Chair of Soil Mechanics and Foundation Engineering
Geotechnical Institute
Technische Universität Bergakademie Freiberg.*

https://tu-freiberg.de/en/soilmechanics

# Exercise on Earth pressure and the Newton-Raphson scheme

## Geometry
Let the retaining wall have a vertical height $H$ and the origin be placed at the bottom of the slip surface which has length $L$. Then, the problem is characterized by the following points in the $x-z$ plane:

* Origin: $O = [0,0]$
* Top left: $A = [-H \tan \alpha, H]$
* Top right: $B = [L\cos \vartheta, L \sin \vartheta]$

The length of the slip surface can be found via the following trigonometric relation:

\begin{align}
    L \sin \vartheta &= (L \cos \vartheta + H \tan\alpha) \tan \beta
\end{align}

Thus we arrive at

\begin{align}
    L &= \frac{H(1 + \tan \alpha \tan \beta)}{\sin \vartheta - \cos \vartheta \tan \beta}
\end{align}

The area of the triangle (i.e. the volume of the slip body per meter) can also be calculated. We use Gauß' equation for that:

$$
    A = \frac{1}{2} \left| (x_2 y_3 - x_3 y_2) - (x_1 y_3 - x_3 y_1) + (x_1 y_2 - x_2 y_1) \right|
$$

Let Python do the math for us ...

```python
import numpy as np #numerical methods
#import sympy as sp #symbolic operations
import matplotlib.pyplot as plt #plotting
import sympy as sp
sp.init_printing(use_latex='mathjax') #makes sympy output look nice

from ipywidgets import widgets
from ipywidgets import interact

#Some plot settings
import plot_functions.plot_settings
%run plot_functions/earth_pressure_plots.ipynb
```

We first define our geometrical properties:

```python
H,alpha,beta,theta = sp.symbols('H alpha beta vartheta')
```

```python
L = H*(1+sp.tan(alpha)*sp.tan(beta))/(sp.sin(theta) - sp.cos(theta) * sp.tan(beta))
L
```

$\displaystyle \frac{H \left(\tan{\left(\alpha \right)} \tan{\left(\beta \right)} + 1\right)}{\sin{\left(\vartheta \right)} - \cos{\left(\vartheta \right)} \tan{\left(\beta \right)}}$

We now define the function that calculates the area of the triangle:

```python
def triangle_area_Gauss(A,B,C):
    term1 = (B[0]*C[1] - C[0] * B[1])
    term2 = (A[0]*C[1] - C[0] * A[1])
    term3 = (A[0]*B[1] - B[0] * A[1])
    return -(term1 - term2 + term3)
```

The points we defined above can now be inserted into this equation:

```python
PO = [0,0]
PA = [- H * sp.tan(alpha), H]
PB = [L * sp.cos(theta), L*sp.sin(theta)]
A = triangle_area_Gauss(PO,PA,PB).simplify()
A
```

$\displaystyle \frac{H^{2} \left(\sin{\left(\vartheta \right)} \tan{\left(\alpha \right)} + \cos{\left(\vartheta \right)}\right) \left(\tan{\left(\alpha \right)} \tan{\left(\beta \right)} + 1\right)}{\sin{\left(\vartheta \right)} - \cos{\left(\vartheta \right)} \tan{\left(\beta \right)}}$

Thus, the area of our triangle is given as:

$$
    A = \frac{H^2 (\sin \vartheta \tan \alpha + \cos \vartheta) (\tan \alpha \tan \beta + 1)}{\sin \vartheta - \cos \vartheta \tan \beta}
$$

Now we can write the force balance. For that purpose, we introduce the material's specific weight $\gamma$, friction angle $\varphi$ and the wall friction angle $\delta_\text{a}$ as well as the cohesion $c$.

```python
c, delta, phi, gamma = sp.symbols('c delta_a varphi gamma')
```

We can now calculate the downward force due to gravity as well as the cohesive force along the slip surface. The active earth pressure force $E_\text{a}$ and the reaction force $Q$ in the slip line are introduced as unknowns:

```python
Ea, Q, G, C = sp.symbols('E_a Q G C')
```

We now write the vertical and horizontal equilibrium conditions:

\begin{align}
    \text{horizontal:}\quad  0 &= C \cos\vartheta - Q \sin (\vartheta - \varphi) + E_\text{a} \cos (\alpha + \delta_\text{a})
    \\
    \text{vertikal:}\quad  0 &= C \sin\vartheta + Q \cos (\vartheta - \varphi) + E_\text{a} \sin (\alpha + \delta_\text{a}) - G
\end{align}

```python
Fh = sp.Eq(C * sp.cos(theta) - Q * sp.sin(theta - phi) + Ea * sp.cos(alpha + delta),0)
Fv = sp.Eq(C * sp.sin(theta) + Q * sp.cos(theta - phi) + Ea * sp.sin(alpha + delta) - G,0)
```

```python
Fh
```

$\displaystyle C \cos{\left(\vartheta \right)} + E_{a} \cos{\left(\alpha + \delta_{a} \right)} + Q \sin{\left(\varphi - \vartheta \right)} = 0$

```python
Fv
```

$\displaystyle C \sin{\left(\vartheta \right)} + E_{a} \sin{\left(\alpha + \delta_{a} \right)} - G + Q \cos{\left(\varphi - \vartheta \right)} = 0$

```python
result = sp.solve([Fh,Fv],(Ea,Q))
```

```python
result
```

$\displaystyle \left\{ E_{a} : - \frac{C \cos{\left(\varphi \right)} + G \sin{\left(\varphi - \vartheta \right)}}{\cos{\left(\alpha + \delta_{a} + \varphi - \vartheta \right)}}, \  Q : \frac{C \sin{\left(\alpha + \delta_{a} - \vartheta \right)} + G \cos{\left(\alpha + \delta_{a} \right)}}{\cos{\left(\alpha + \delta_{a} + \varphi - \vartheta \right)}}\right\}$

Gravitational and cohesive forces can be determined based on material properties and geometry:

```python
result.get(Ea).subs(G,A*gamma).simplify()
```

$\displaystyle - \frac{C \left(\sin{\left(\vartheta \right)} - \cos{\left(\vartheta \right)} \tan{\left(\beta \right)}\right) \cos{\left(\varphi \right)} + H^{2} \gamma \left(\sin{\left(\vartheta \right)} \tan{\left(\alpha \right)} + \cos{\left(\vartheta \right)}\right) \left(\tan{\left(\alpha \right)} \tan{\left(\beta \right)} + 1\right) \sin{\left(\varphi - \vartheta \right)}}{\left(\sin{\left(\vartheta \right)} - \cos{\left(\vartheta \right)} \tan{\left(\beta \right)}\right) \cos{\left(\alpha + \delta_{a} + \varphi - \vartheta \right)}}$

Now we would have to find the maximum with respect to $\vartheta$ in order to identify the active earth pressure and the critical slip angle $\vartheta_\text{a}$. The equations get very complex and we decide to go for a numerical solution.

First, we define the basic geometry of our problem:

```python
alpha = 10. * np.pi/180.
beta = 20. * np.pi/180.
H = 4. #m
```

```python
def slip_length(theta):
    L = H * (1 + np.tan(alpha) * np.tan(beta))/(np.sin(theta) - np.cos(theta)*np.tan(beta))
    return L
```

```python
def area(theta):
    term1 = (np.tan(alpha)*np.tan(beta) + 1) * (np.sin(theta)*np.tan(alpha) + np.cos(theta))
    term2 = np.sin(theta) - np.cos(theta) * np.tan(beta)
    return H*H * term1 / term2
```

Now we define the material properties and the interaction with the wall:

```python
gamma = 16.e3 #N/m³
phi = np.deg2rad(30.)
c = 30.e3 #N/m²
delta = np.deg2rad(10.)
```

```python
def cohesive_force(theta):
    return c * slip_length(theta)
```

```python
def weight(theta):
    return gamma*area(theta)
```

Now we have all the ingredients to calculate the active earth pressure given by the equation above:

```python
def Ea(theta):
    term1 = weight(theta) * np.sin(theta-phi) - cohesive_force(theta) * np.cos(phi)
    term2 = np.cos(alpha+delta+phi-theta)
    return term1/term2
```

Let's take a look at the result:

```python
thetas = np.linspace(np.pi/6,np.pi/2*0.9,100)
plt.plot(thetas*180/np.pi,Ea(thetas)/1e6)
plt.xlabel('$\\vartheta$ / °')
plt.ylabel('$E_\\mathrm{a}$ / MN m$^{-1}$');
```

The active earth pressure is found at the peak of this curve. In other words we need to identify

$$
    \frac{\partial E_\text{a}}{\partial \vartheta} = 0
$$

and

$$
    \vartheta_\text{a} = \underset{\vartheta \in [0,\pi/2]}{\text{arg max }}E_\text{a}
$$

As we saw above, this is cumbersome analytically. However, we can find the solution numerically in an iterative manner.

As we have

$$
    \left. \frac{\partial E_\text{a}}{\partial \vartheta} \right|_{\vartheta_\text{a}} = 0
$$

we find for other angles the following residual:

$$
    r(\vartheta) = \frac{\partial E_\text{a}}{\partial \vartheta} \neq 0
$$

to which we seek the solution

$$
    r(\vartheta_\text{a}) = 0
$$

Since we don't want to go through the trouble to determine the first derivative of $E_\text{a}$ with respect to $\vartheta$ analytically, we do so numerically by numerical differentiation using a central finite difference scheme:

$$
    \frac{\partial E_\text{a}}{\partial \vartheta} \approx \frac{E_\text{a}(\vartheta + \epsilon) - E_\text{a}(\vartheta - \epsilon)}{2\epsilon}
    \quad
    \text{with}
    \quad \epsilon \ll 1,\ \epsilon > 0
$$

```python
def dEa_dtheta(theta,eps=1.e-4):
    return (Ea(theta+eps) - Ea(theta-eps))/(2.*eps)
```

The former function defines the slope and equals our residual:

```python
def residual(theta):
    return dEa_dtheta(theta)
```

Now, we use Newton's method to find the solution to the non-linear equation $r(\vartheta) = 0$. We depart from a Taylor series expansion to first order around an initial guess $\vartheta_0$:

$$
    r(\vartheta) \approx r(\vartheta_0) + \left. \frac{\partial r}{\partial \vartheta} \right|_{\vartheta_0} \Delta \vartheta \overset{!}{=} 0
    \quad
    \text{with}
    \quad
    \Delta \vartheta = \vartheta - \vartheta_0
$$

We obtain the next guess for $\vartheta$ by rearranging:

$$
    \vartheta = \vartheta_0 - \left( \left. \frac{\partial r}{\partial \vartheta} \right|_{\vartheta_0} \right)^{-1} r(\vartheta_0)
$$

As we linearized the non-linear function, the new guess is likely still not the solution we seek. Hence, we iterate:

$$
    \vartheta_{i+1} = \vartheta_i - \left( \left. \frac{\partial r}{\partial \vartheta} \right|_{\vartheta_i} \right)^{-1} r(\vartheta_i)
$$

We terminate the solution once $\vartheta$ no longer changes, e.g. once

$$
    | \vartheta_{i+1} - \vartheta_i | < \epsilon_\text{tol}
$$

Another criterion, which is more direct, is to determine when the residual is close enough to zero:

$$
    | r(\vartheta_{i+1}) | < \epsilon_\text{tol}
$$

In order to code this algorithm, we need the first derivative of the residual with respect to $\vartheta$ (this is actually the second derivative of $E_a$):

```python
def dr_dtheta(theta,eps=1.e-4):
    return (residual(theta+eps) - residual(theta-eps))/(2.*eps)
```

We now code the algorithm and give it a starting guess $\vartheta_0$ and a termination tolerance $\epsilon_\text{tol}$:

```python
def find_theta(theta_0=np.pi/6, tol=1.e-6):
    theta = theta_0
    iteration_counter = 0
    res_vec = np.array([])
    theta_vec = np.array([theta])
    while (np.abs(residual(theta)) > tol and iteration_counter < 100):
        res_vec = np.append(res_vec,residual(theta))
        theta -= residual(theta) / dr_dtheta(theta)
        theta_vec = np.append(theta_vec,theta)
        iteration_counter += 1
    return theta, iteration_counter, res_vec, theta_vec
```

Running this algorithm will give us the solution we seek.

```python
theta_a, it, res, sols = find_theta()
```

```python
print("The slip angle of %.2f ° was found after %i iterations. The corresponding earth pressure is %.2f MN/m."
      %(np.rad2deg(theta_a),it,Ea(theta_a)/1.e6))
```

    The slip angle of 66.35 ° was found after 8 iterations. The corresponding earth pressure is -0.03 MN/m.

The result was found after several iterations. We may lower this number by improving our starting guess. For example, we could use the classical solution

$$
    \vartheta_0 = \frac{\pi}{4} + \frac{\varphi}{2}
$$

```python
theta_a, it, res, sols = find_theta(np.pi/4 + phi/2)
print("The slip angle of %.2f ° was found after %i iterations. The corresponding earth pressure is %.2f MN/m."
      %(np.rad2deg(theta_a),it,Ea(theta_a)/1.e6))
```

    The slip angle of 66.35 ° was found after 4 iterations. The corresponding earth pressure is -0.03 MN/m.

We arrive at the same result, albeit in fewer iterations. We can check this result graphically:

```python
plot_iterative_interactive()
```

    interactive(children=(BoundedFloatText(value=45.0, description='$\\vartheta_0 / °$', max=90.0, min=10.0), Outp…

We see that the Newton algorithm reaches quadratic convergence once we are in the proximity of the solution.

If we choose an initial guess that's outside of the convergence region for the Newton scheme, we may not converge at all or to the wrong solution.

We can also compare this result to the one from DIN4085, for example:

$$
    \vartheta_\text{ag} = \varphi + \text{arctan} \left[ \frac{\cos(\varphi - \alpha)}{\sin(\varphi - \alpha) + \sqrt{ \displaystyle \frac{\sin(\varphi + \delta_\text{a}) \cos (\alpha - \beta)}{\sin(\varphi - \beta) \cos (\alpha + \delta_\text{a})} } }  \right]
$$

```python
def theta_ag_norm(phi,alpha,delta,beta):
    t1 = np.cos(phi - alpha)
    t2 = np.sin(phi - alpha) + np.sqrt((np.sin(phi + delta)*np.cos(alpha-beta))/(np.sin(phi-beta)*np.cos(alpha+delta)))
    return (phi + np.arctan(t1/t2))*180/np.pi
```

```python
print("For the non-cohesive case the norm gives us %.2f °." %theta_ag_norm(phi,alpha,delta,beta))
```

    For the non-cohesive case the norm gives us 52.12 °.

## Tasks:

* Check whether you get the same result by setting $c=0$.
* Determine why some starting guesses don't work.
* Play with the parameters and learn something.

```python
plot_interactive_earth_pressure()
```

    interactive(children=(IntSlider(value=10, description='$c$ / kN m$^{-2}$:', max=50, step=5), IntSlider(value=3…


# Вариант 9

Рассматривается система линейных гиперболических уравнений:

\begin{gather*}
    \frac { \partial \mathbf { u } } { \partial t } + \mathbf { A } \frac { \partial \mathbf { u } } { \partial x } = \mathbf { b } ( x ) , ~ 0 \leqslant x \leqslant 1, ~ 0 \leqslant t \leqslant 1, ~~
    \mathbf { u } ( x , 0 ) = \left( \begin{array} { c } { 2 x - x ^ { 3 } } \\ { x ^ { 3 } } \\ { x ^ { 2 } + 1 } \end{array} \right) \\
    \mathbf { A } = \left( \begin{array} { c c c } { - 21 / 2 } & { 27 / 2 } & { 3 / 2 } \\ { - 7 } & { 9 } & { 1 } \\ { 13 / 2 } & { - 19 / 2 } & { - 3 / 2 } \end{array} \right), ~~
    \mathbf { b } ( x ) = \left( \begin{array} { l } { 0 } \\ { 0 } \\ { 0 } \end{array} \right)
\end{gather*}

Привести систему к характеристическому виду, предложить корректную постановку граничных условий.

Решить численно систему уравенний с использованием двух указанных схем.
Для каждой из схем выписать первое дифференциальное приближение (ПДП),
определить, диссипативная или дисперсионная ошибка преобладает.
Монотонна ли схема? Оценить апостериорно порядок сходимости каждой схемы.

### Разностные схемы
Схемы приводятся для модельного уравнения $ \frac { \partial u } { \partial t } + a \frac { \partial u } { \partial x } = 0 , ~ a = \text { const } > 0 $ на сетке $ x _ { m } = m h , ~ m = 0 \dots M , ~ M h = 1 ; ~ t ^ { n } = n \tau , ~ n = 0 \dots N, ~ N \tau = 1 $. При необходимости преобразовать схемы для случая $ a = \text{const} < 0 $.

1) "Неявный" правый уголок (схема бегущего счета)
\begin{equation}
    \frac { u _ { m } ^ { n + 1 } - u _ { m } ^ { n } } { \tau } + a \frac { u _ { m + 1 } ^ { n + 1 } - u _ { m } ^ { n + 1 } } { h } = 0
\end{equation}

2) Центральная схема второго порядка (Лакса-Вендроффа, МакКормака)
\begin{equation}
    \frac { u _ { m } ^ { n + 1 } - u _ { m } ^ { n } } { \tau } + a \frac { u _ { m + 1 } ^ { n } - u _ { m - 1 } ^ { n } } { 2 h } = \frac { a ^ { 2 } \tau } { 2 } \frac { u _ { m + 1 } ^ { n } - 2 u _ { m } ^ { n } + u _ { m - 1 } ^ { n } } { h ^ { 2 } }
\end{equation}

## Анализ системы

Приведем систему к характеристическому виду. Для этого найдем левые собственные вектора (см. код ниже), затем приведем систему к каноническому виду. Это реализовано ниже.

```python
# Подготовительный этап
from IPython.display import display, Math, Latex, Markdown
import pprint
import sympy as sp
from sympy import Matrix, Rational, latex, symbols
import numpy as np
from scipy.linalg import eig
import matplotlib.pyplot as plt
from matplotlib import animation
x, u, t = sp.symbols('x u t')
```

```python
# Вбиваем данные задачи
A = Matrix([
    [Rational(-21, 2), Rational(27, 2), Rational(3, 2)],
    [-7, 9, 1],
    [Rational(13, 2), Rational(-19, 2), Rational(-3, 2)]
])
#A = sp.diag(10, 5, 30)
b = Matrix([0, 0, 0])
u = Matrix([2*x - x**3, x**3, x**2 + 1])
```

```python
# Ищем собственные вектора и собственные значения
print('Собственные значения и собственные вектора:')
tmp = A.T.eigenvects()
# Выводим это на экран
for i, tup in enumerate(tmp):
    display(Math(
        '\lambda_{0} = {1}; ~ \omega_{0} = {2}'
        .format( str(i+1), tup[0], latex(tup[2][0]) )
    ))
```

    Собственные значения и собственные вектора:

$\displaystyle \lambda_1 = -2; ~ \omega_1 = \left[\begin{matrix}-5\\7\\1\end{matrix}\right]$

$\displaystyle \lambda_2 = -1; ~ \omega_2 = \left[\begin{matrix}-3\\5\\1\end{matrix}\right]$

$\displaystyle \lambda_3 = 0; ~ \omega_3 = \left[\begin{matrix}- \frac{2}{3}\\1\\0\end{matrix}\right]$

```python
# Считаем матрицы, которые нам потом пригодятся
Omega = Matrix([Matrix(i[2][0]).T for i in tmp])
lambdas = [i[0] for i in tmp]
Lambda = sp.diag(*lambdas)
```

```python
display(Latex(r'Запишем матрицу перехода: $ \Omega = {}$'.format(latex(Omega))),
        Latex(r'а также обратную ей: $ \Omega^{{-1}} = {}$'.format(latex(Omega**(-1)))),
        Math(r'A = \Omega^{-1} \Lambda \Omega')
       )
```

Запишем матрицу перехода: $ \Omega = \left[\begin{matrix}-5 & 7 & 1\\-3 & 5 & 1\\- \frac{2}{3} & 1 & 0\end{matrix}\right]$

а также обратную ей: $ \Omega^{-1} = \left[\begin{matrix}- \frac{3}{2} & \frac{3}{2} & 3\\-1 & 1 & 3\\\frac{1}{2} & \frac{1}{2} & -6\end{matrix}\right]$

$\displaystyle A = \Omega^{-1} \Lambda \Omega$

```python
u
```

$\displaystyle \left[\begin{matrix}- x^{3} + 2 x\\x^{3}\\x^{2} + 1\end{matrix}\right]$

```python
I = Omega*u
display(
    Math(r'\mathbf{{I}} (x, 0) = {}, ~~~~ \Lambda = {}'
         .format( latex(I), latex(Lambda) )
        )
)
```

$\displaystyle \mathbf{I} (x, 0) = \left[\begin{matrix}12 x^{3} + x^{2} - 10 x + 1\\8 x^{3} + x^{2} - 6 x + 1\\\frac{5 x^{3}}{3} - \frac{4 x}{3}\end{matrix}\right], ~~~~ \Lambda = \left[\begin{matrix}-2 & 0 & 0\\0 & -1 & 0\\0 & 0 & 0\end{matrix}\right]$

#### О постановке граничных условий:
В нашей системе есть 0 положительных собственных значений, 1 нулевое и 2
отрицательных.
Значит, на левой границе не нужно ставить граничных условий, на правой же границе нужно 2.
В нулевом значении достаточно задать только начальное условие.

Попробуем поставить граничные условия вида:

\begin{equation}
u|_{t = 0} = f(x), ~~ u|_{x = 1} = g(t)
\end{equation}

На них будет наложено необходимое условие разрешимости:
$ f(1) = g(0) $.
Потребуем также, чтобы производная не терпела разрыв: $ f'(1) = g'(0) $. Итого:

\begin{gather}
    \boxed{
    f(1) = g(0) \\
    f'(1) = g'(0)
    }
\end{gather}

```python
x_nepr_border = [I[i].subs(x, x-lambdas[i]*t).subs(x, 1) for i in range(len(lambdas))]
exact_sol = [I[i].subs(x, x-lambdas[i]*t) for i in range(len(lambdas))]
exact_sol_funcs = [sp.lambdify((t, x), exact_sol[i]) for i in range(len(lambdas))]
display(
    Markdown('Итак, мы получили {} уравнения:'.format(Lambda.shape[0])),
    *[
        Math(
            (r'\cfrac{{\partial I_{i} }} {{\partial t}} {sign}{koeff}\cdot'+
             r'\cfrac{{\partial I_{i} }} {{\partial x}} = {left_part}' +
             ', ~ I_{i} (x, 0) ='
            ).format(
                i=(k+1),
                sign='+' if lambdas[k] >= 0 else '-',
                koeff=abs(lambdas[k]),
                size=Lambda.shape[0],
                left_part=b[k]
            )
            + sp.latex( (Omega*u)[k] )
        ) for k in range(Lambda.shape[0])
    ],
    Markdown(
        'Эти уравнения имеют решения:',
    ),
    *[Math(
        'I_{} (t, x) = '.format(i+1) +
        latex(exact_sol[i])
    ) for i in range(len(lambdas))],
    Markdown(
        'из которых (полагая, что решение должно быть из класса $C^1$) имеем граничные условия:'
    ),
    *[
        Math(
            r'\left. I_{n} \right|_{{x={x_max}}} (t) = '.format(n=i+1, x_max=1) +
            latex(x_nepr_border[i])
        ) for i in range(len(lambdas))
    ]
)
```

Итак, мы получили 3 уравнения:

$\displaystyle \cfrac{\partial I_1 } {\partial t} -2\cdot\cfrac{\partial I_1 } {\partial x} = 0, ~ I_1 (x, 0) =12 x^{3} + x^{2} - 10 x + 1$

$\displaystyle \cfrac{\partial I_2 } {\partial t} -1\cdot\cfrac{\partial I_2 } {\partial x} = 0, ~ I_2 (x, 0) =8 x^{3} + x^{2} - 6 x + 1$

$\displaystyle \cfrac{\partial I_3 } {\partial t} +0\cdot\cfrac{\partial I_3 } {\partial x} = 0, ~ I_3 (x, 0) =\frac{5 x^{3}}{3} - \frac{4 x}{3}$

Эти уравнения имеют решения:

$\displaystyle I_1 (t, x) = - 20 t - 10 x + 12 \left(2 t + x\right)^{3} + \left(2 t + x\right)^{2} + 1$

$\displaystyle I_2 (t, x) = - 6 t - 6 x + 8 \left(t + x\right)^{3} + \left(t + x\right)^{2} + 1$

$\displaystyle I_3 (t, x) = \frac{5 x^{3}}{3} - \frac{4 x}{3}$

из которых (полагая, что решение должно быть из класса $C^1$) имеем граничные условия:

$\displaystyle \left. I_1 \right|_{x=1} (t) = - 20 t + 12 \left(2 t + 1\right)^{3} + \left(2 t + 1\right)^{2} - 9$

$\displaystyle \left. I_2 \right|_{x=1} (t) = - 6 t + 8 \left(t + 1\right)^{3} + \left(t + 1\right)^{2} - 5$

$\displaystyle \left. I_3 \right|_{x=1} (t) = \frac{1}{3}$

```python
from utils import make_grid, AnimationHelper
```

```python
X_MIN, X_MAX, T_MIN, T_MAX = 0.0, 1.0, 0.0, 1.0
```

```python
# pure function
def calculate_all(kernel, timezero_funcs, xzero_funcs,
                  a, h, tau,
                  x_min=0, x_max=1, t_min=0, t_max=1):
    """
    Делает подсчеты с заданным ядром метода для всех компонент
    """
    tmp = []
    # Создаем сетку
    T, x_grid, t_grid = make_grid(x_min, x_max, t_min,
                                   t_max, h, tau, fictive_x=True)
    # Для каждой компоненты
    for i in range(len(timezero_funcs)):
        # Считаем значения
        func_t, func_x, a_curr = timezero_funcs[i], xzero_funcs[i], np.float64(a[i])
        # Начальные условия
        T[:,0] = func_t(x_grid)
        # Граничные условия
        T[-1,:] = func_x(t_grid)
        # Приводим все к double
        T = T.astype('double')
        # Высчитываем sigma
        sigma = np.double(a_curr*tau/h)
        # Вызываем калькулятор
        tmp.append(kernel(T, sigma).T)
    return tmp, x_grid[1:-1], t_grid
```

```python
def plot_3d(x_grid, t_grid, u_i, position, label='u_1', filename='no_name'):
    from mpl_toolkits import mplot3d
    fig = plt.figure(figsize=(16, 10))
    ax = plt.axes(projection='3d')
    xx, yy = np.meshgrid(x_grid, t_grid)
    ax.plot_surface(xx, yy, u_i.T)
    ax.view_init(*position)
    ax.set_title(r'$ {} $'.format(label))
    ax.set_xlabel(r'$ x $')
    ax.set_ylabel(r'$ t $')
    fig.savefig(filename)
    fig.show()
```

# DASHA

```python
from numba import jit

@jit(nopython=True)
def dasha_kernel(T, sigma):
    """
    Предполагается, что граничные условия УЖЕ заданы
    """
    M, N = T.shape[0], T.shape[1]
    for n in range(0, N-1):
        # Решим систему и запишем результаты в следующую итерацию времени
        #count_current_x_layer(T, n, sigma)
        for m in range(M-2, -1, -1):
            T[m, n+1] = (0.5-sigma)*T[m+1, n] + (0.5+sigma)*T[m-1, n]
    # Выкинем фиктивные узлы в возвращаемом значении
    return T[1:-1, :]

h, tau = 10**-3, 10**-3/4

timezero_funcs = [sp.lambdify(x, I[i]) for i in range(Omega.shape[0])]
'''
xzero_funcs = [
    lambda t: np.sin(10*t)+timezero_funcs[0](1),
    lambda t: np.sin(10*t)+timezero_funcs[1](1),
    lambda t: np.sin(10*t)+timezero_funcs[2](1)
]
'''
#'''
xzero_funcs = [
    lambda t: timezero_funcs[0](1),
    lambda t: timezero_funcs[1](1),
    lambda t: timezero_funcs[2](1),
]
#'''

tmp = calculate_all(dasha_kernel, timezero_funcs, xzero_funcs,
                    lambdas, h, tau, t_max=1)
T, x_grid, t_grid = np.array(tmp[0]), tmp[1], tmp[2]
del tmp

# Вернемся обратно от инвариантов Римана
omega_rev = np.array(Omega**-1).astype(np.float64)
solution = T.T.dot(omega_rev)
del omega_rev

u_1_dasha = solution[:,:,0]
u_2_dasha = solution[:,:,1]
u_3_dasha = solution[:,:,2]

# Анимация первой координаты
anim1 = AnimationHelper(u_1_dasha, x_grid=x_grid, t_grid=t_grid, ylabel='u_1')
anim1.make_animation().save_as_file('dasha_u1.mp4')

# Анимация второй координаты
anim2 = AnimationHelper(u_2_dasha, x_grid=x_grid, t_grid=t_grid, ylabel='u_2')
anim2.make_animation().save_as_file('dasha_u2.mp4')

# Анимация третьей координаты
anim3 = AnimationHelper(u_3_dasha, x_grid=x_grid, t_grid=t_grid, ylabel='u_3')
anim3.make_animation().save_as_file('dasha_u3.mp4')

plot_3d(x_grid, t_grid, solution[:,:,0], (20, 130), label='u_1', filename='dasha_u1.pdf')

plot_3d(x_grid, t_grid, solution[:,:,1], (40, -20), label='u_2', filename='dasha_u2.pdf')

plot_3d(x_grid, t_grid, solution[:,:,2], (60, -30), label='u_3', filename='dasha_u3.pdf')

timezero_funcs = [sp.lambdify(x, I[i]) for i in range(Omega.shape[0])]
# Наглядные синусы
'''
xzero_funcs = [
    lambda t: np.sin(10*t)+timezero_funcs[0](1),
    lambda t: np.sin(10*t)+timezero_funcs[1](1),
    lambda t: np.sin(10*t)+timezero_funcs[2](1)
]
'''
# Не очень наглядно: просто константа
'''
xzero_funcs = [
    lambda t: timezero_funcs[0](1),
    lambda t: timezero_funcs[1](1),
    lambda t: timezero_funcs[2](1),
]
'''
# Продолжение по непрерывности. С ним и будем сравнивать
#xzero_funcs = [sp.lambdify(t, x_f) for x_f in x_nepr_border]
xzero_funcs = [sp.lambdify(t, func1.subs(x, X_MAX)) for func1 in exact_sol]

@jit(parallel=True)
def count_errors(exact_sol_func, kernel,
                         tau, a,
                         h_range, sigma=0.5,
                         x_min=0, x_max=1, t_min=0, t_max=1):
    #a, exact_sol_func, timezero_func, xzero_func, h_range = lambdas[0], exact_sol_funcs[0], timezero_funcs[0], xzero_funcs[0], np.linspace(10**(-4), 10**(-3), 5)
    sigma = 0.5
    tmp = []
    for h in h_range:
        # Пересчитаем tau через sigma
        # sigma = np.double(a_curr*tau/h)
        tau = sigma*h/abs(a)
        print('tau =', tau)
        # Создаем сетку для каждого h
        T, x_grid, t_grid = make_grid(x_min, x_max, t_min,
                                   t_max, h, tau, fictive_x=True)
        print('Created T...')
        # Создаем сетку для подсчета точного решения
        tt, xx = np.meshgrid(t_grid, x_grid)
        exact = exact_sol_func(tt, xx)
        print('Calculated EXACT...')
        # Граничные условия СПРАВА!!!
        # А вот начальные условия -- на t_min
        func_t, func_x = lambda t: exact_sol_func(t, x_max), lambda x: exact_sol_func(t_min, x)
        # Начальные условия
        T[:,0] = func_t(x_grid)
        # Граничные условия
        T[-1,:] = func_x(t_grid)
        print('Calculated inital values...')
        # Приводим все к double
        #T = T.astype('double')
        approx = kernel(T, sigma).T
        print('Calculated approx solution...')
        tmp.append(approx - exact[1:-1,:].T)
    return tmp

tau = 10**(-2)/2

# Все будем судить по 1-ой координате
N_coord = 0
h_range = np.linspace(10**(-2), 10**(-1), 30)
different_calcs = count_errors(jit(exact_sol_funcs[N_coord]), dasha_kernel,
                                       tau, lambdas[N_coord],
                                       h_range,
                                       sigma=0.5)

```

```python
plot_3d(x_grid, t_grid, solution[:,:,0], (20, 130), label='u_1', filename='dasha_u1.pdf')
```

```python
plot_3d(x_grid, t_grid, solution[:,:,1], (40, -20), label='u_2', filename='dasha_u2.pdf')
```

```python
plot_3d(x_grid, t_grid, solution[:,:,2], (60, -30), label='u_3', filename='dasha_u3.pdf')
```

```python
errors = np.array([np.linalg.norm(different_calcs[i].astype('float')) for i in range(len(different_calcs))])

y_plot = np.log(errors)
x_plot = np.log(h_range)

plt.scatter(x_plot, y_plot)
N_forfit_left, N_forfit_right = int(len(x_plot)*0), int(len(x_plot)*1)
x_forfit, y_forfit = x_plot[N_forfit_left:N_forfit_right], y_plot[N_forfit_left:N_forfit_right]
plt.scatter(x_forfit, y_forfit, label='Сырые данные')

from scipy.stats import linregress
slope, intercept, r_value, p_value, std_err = linregress(x_forfit, y_forfit)
x_plot_forfit = np.linspace(min(x_forfit), max(x_forfit), 10**3)
y_plot_forfit = slope*x_plot_forfit + intercept
plt.plot(x_plot_forfit, y_plot_forfit, label='МНК, $k={}$'.format(slope.round(2)))

plt.grid()
plt.legend()
plt.xlabel(r'$\log h$')
plt.ylabel(r'$\log e$')
plt.title(r'Зависимость $\log(error)$ от $\log(h)$')
plt.savefig('errors_dasha.pdf')
plt.show()

display(
    Markdown(
        'Апостериорный порядок сходмиости $k={}$ (коэфф. корреляции $r={}$)'.format(
            abs(slope.round(2)), abs(r_value.round(4))
        )
    )
)
```

## Решение центральной схемой второго порядка

Данная схема преобразуется к виду:
\begin{equation*}
    u _ { n } ^ { n + 1 } = u _ { m } ^ { n } - \frac { a \tau } { 2 h } \left( u _ { m + 1 } ^ { n } - u _ { m - 1 } ^ { n } \right) + \frac { a ^ { 2 } \tau ^ { 2 } } { 2 h ^ { 2 } } \left( u _ { m + 1 } ^ { n } - 2 u _ { m } ^ { n } + u _ { m - 1 } ^ { n } \right)
\end{equation*}
Мы введем два фиктивных узла по сетке $x$ (слева и справа). На каждой итерации по $n$ (время) значения в фиктивных узлах будем пересчитывать через линейную интерполяцию по двум предыдущим точкам (поскольку $h$ мало, это должно хватить).

### Первое дифференциальное приближение
Перепишем схему в виде:

\begin{gather*}
    \cfrac{u_{m+1}^{n+1} - u_m^n}{\tau} + a \cfrac{u_{m+1}^n - u_{m-1}^n}{2h} - \cfrac{a^2 \tau}{2} \cfrac{u_{m+1}^n - 2 u_m^n + u_{m-1}^n}{h^2} = 0 \\
    u_m^{n+1} = u_m^n + \tau u_t + \cfrac{1}{2} \tau^2 u_{tt} + \cfrac{1}{6} \tau^3 u_{ttt} \\
    u_{m-1}^n = u_m^n - h u_x + \cfrac{1}{2} h^2 u_{xx} - \cfrac{1}{6} h^3 u_{xxx} \\
    u_{m+1}^n = u_m^n + h u_x + \cfrac{1}{2} h^2 u_{xx} + \cfrac{1}{6} h^3 u_{xxx} \\
\end{gather*}
Невязка равна: $ \psi_n = (u_t + a u_x)_n - \left[ \cfrac{1}{\tau} ( u_m^{n+1} u_m^n) + \cfrac{a}{2h} (u_{m+1}^n - u_{m-1}^n) - \cfrac{a^2 \tau}{2h^2} (u_{m+1}^{n} - 2 u_{m}^{n} + u_{m-1}^{n}) \right] $

$ \phi_{n} = -\cfrac{\tau}{2} u_{tt} + \cfrac{a^2 \tau}{2} u_{xx} - \cfrac{\tau^2}{6} u_{ttt} - \cfrac{ah^2}{6} u_{xxx} $
С учетом того, что $ u_{tt}a^2 = u_{xx} $ и $ u_{ttt} = -a^3 u_{xxx} $ , имеем $ \psi_{n} = \cfrac{a}{6} (\tau^2 a^2 - h^2 ) u_{xxx} $.

**ПДП**:  $ \boxed{u_{t} = \cfrac{a}{6} (\tau^2 a^2 -h^2) u_{xxx} - a u_{x} }$ .

Для заданной схемы дисперсионное соотношение имеет вид:

\begin{equation}
    \lambda=\frac{1}{\tau} \ln \left(1-i \sigma \sin k h-2 \sigma^{2} \sin ^{2} \frac{k h}{2}\right)
\end{equation}
для длинноволновых гармоник с $ kh \ll 1 $ разложение вплоть до членов третьего порядка по этой величине дает

\begin{equation}
    \lambda(\tau, h, k) \approx-i k a+i k a \frac{k^{2} h^{2}}{6}\left(1-3 \sigma^{2}\right).
\end{equation}

Из полученных соотношений делаем вывод, что решения разностного уравнения второго порядка аппроксимации имеют вид:

$ y\left(t_{n}, x_{m}\right)=\exp \left(i k x_{m}+\lambda(\tau, h, k) t_{n}\right)=\exp \left(i k\left\{x_{m}-a\left[1-\frac{k^{2} h^{2}}{6}\left(1-3 \sigma^{2}\right)\right] t_{n}\right\}\right)= \exp \left(i k\left\{x_{m}-a\left[1+A_{\mathrm{k}}(k)\right] t_{n}\right\}\right) $

где введено обозначение $ A_{k}=-k^{2} h^{2}\left(1-3 \sigma^{2}\right) / 6 $.

Последнее выражение означает, что гармоники со временем не затухают, так как не появляется множителя с действительными отрицательным показателем экспоненты, численное решение не сглаживается со временем. Начальный волновой пакет и определяемый им начальный профиль лишь размываются с течением времени.

_Это означает, что в данной схеме преобладает дисперсионная ошибка_.
### Монотонность
Наша схема является линейной и имеет второй порядок аппроксимации. Согласно теореме Годунова, _схема не является монотонной_.

```python
from numba import jit
@jit(nopython=True, parallel=True)
def laks_kernel(T, sigma):
    """
    Основной калькулятор. Предполагается, что в `T` уже включены начальные данные.
    НАЧАЛЬНЫЕ УСЛОВИЯ ЗАДАНЫ СПРАВА!!!
    """
    M, N = T.shape[0], T.shape[1]
    for n in range(0, N-1):
        for m in range(1, M-1):
            T[m, n+1] = T[m, n] - sigma/2*(T[m+1, n] - T[m-1, n]) \
            + sigma**2/2* (T[m+1, n] - 2*T[m, n] + T[m-1, n])
        # Линейная экстраполяция слева
        T[0, n+1] = 2*T[1, n+1] - T[2, n+1]
        # А вот справа у нас уже есть начальные условия
        # Линейная экстраполяция справа
        #T[M-1, n+1] = 2*T[M-2, n+1] - T[M-3, n+1]
    return T[1:-1,:]
```

```python
h, tau = 10**-3, 10**-3/4
```

```python
timezero_funcs = [sp.lambdify(x, I[i]) for i in range(Omega.shape[0])]
# Наглядные синусы
'''
xzero_funcs = [
    lambda t: np.sin(10*t)+timezero_funcs[0](1),
    lambda t: np.sin(10*t)+timezero_funcs[1](1),
    lambda t: np.sin(10*t)+timezero_funcs[2](1)
]
'''
# Не очень наглядно: просто константа
#'''
xzero_funcs = [
    lambda t: timezero_funcs[0](1),
    lambda t: timezero_funcs[1](1),
    lambda t: timezero_funcs[2](1),
]
#'''
# Продолжение по непрерывности. С ним и будем сравнивать
#xzero_funcs = [sp.lambdify(t, x_f) for x_f in x_nepr_border]
```

```python
tmp = calculate_all(laks_kernel, timezero_funcs, xzero_funcs,
                    lambdas, h, tau,
                    x_min=X_MIN, x_max=X_MAX, t_min=T_MIN, t_max=T_MAX)
T, x_grid, t_grid = np.array(tmp[0]), tmp[1], tmp[2]
```

```python
# Вернемся обратно от инвариантов Римана
omega_rev = np.array(Omega**-1).astype(np.float64)
solution = T.T.dot(omega_rev)
```

```python
u_1_laks = solution[:,:,0]
u_2_laks = solution[:,:,1]
u_3_laks = solution[:,:,2]
```

```python
# Анимация первой координаты
anim1 = AnimationHelper(u_1_laks, x_grid=x_grid, t_grid=t_grid, ylabel='u_1')
anim1.make_animation()
anim1.save_as_file('laks_u1.mp4')

# Анимация второй координаты
anim2 = AnimationHelper(u_2_laks, x_grid=x_grid, t_grid=t_grid, ylabel='u_2')
anim2.make_animation().save_as_file('laks_u2.mp4')

# Анимация третьей координаты
anim3 = AnimationHelper(u_3_laks, x_grid=x_grid, t_grid=t_grid, ylabel='u_3')
anim3.make_animation().save_as_file('laks_u3.mp4')
```

Для наглядности построим график еще 3D график

```python
plot_3d(x_grid, t_grid, solution[:,:,0], (30, 110), label='u_1', filename='laks_u1.pdf')
#plot_3d(x_grid, t_grid, solution[:,:,0], (30, -10), label='u_1', filename='laks_u1.pdf')
```

```python
plot_3d(x_grid, t_grid, solution[:,:,1], (40, -20), label='u_2', filename='laks_u2.pdf')
```

```python
plot_3d(x_grid, t_grid, solution[:,:,2], (60, -30), label='u_3', filename='laks_u3.pdf')
```

Для определения порядка аппроксимации воспользуемся теми граничными условиями, которые определяют функцию из класса $C^1$. Они были выписаны выше, при теоретическом анализе схемы.

Мы будем считать ошибку по инвариантам Римана (как разность теоретического решения и численного).

```python
timezero_funcs = [sp.lambdify(x, I[i]) for i in range(Omega.shape[0])]
# Наглядные синусы
'''
xzero_funcs = [
    lambda t: np.sin(10*t)+timezero_funcs[0](1),
    lambda t: np.sin(10*t)+timezero_funcs[1](1),
    lambda t: np.sin(10*t)+timezero_funcs[2](1)
]
'''
# Не очень наглядно: просто константа
'''
xzero_funcs = [
    lambda t: timezero_funcs[0](1),
    lambda t: timezero_funcs[1](1),
    lambda t: timezero_funcs[2](1),
]
'''
# Продолжение по непрерывности. С ним и будем сравнивать
xzero_funcs = [sp.lambdify(t, x_f) for x_f in x_nepr_border]

tmp = calculate_all(laks_kernel, timezero_funcs, xzero_funcs,
                    lambdas, h, tau,
                    x_min=X_MIN, x_max=X_MAX, t_min=T_MIN, t_max=T_MAX)
T, x_grid, t_grid = np.array(tmp[0]), tmp[1], tmp[2]

solution_nepr = T.T
```

```python
@jit(parallel=True)
def count_errors(exact_sol_func, kernel,
                         tau, a,
                         h_range, sigma=0.5,
                         x_min=0, x_max=1, t_min=0, t_max=1):
    #a, exact_sol_func, timezero_func, xzero_func, h_range = lambdas[0], exact_sol_funcs[0], timezero_funcs[0], xzero_funcs[0], np.linspace(10**(-4), 10**(-3), 5)
    sigma = 0.5
    tmp = []
    for h in h_range:
        # Пересчитаем tau через sigma
        # sigma = np.double(a_curr*tau/h)
        tau = sigma*h/abs(a)
        print('tau =', tau)
        # Создаем сетку для каждого h
        T, x_grid, t_grid = make_grid(x_min, x_max, t_min,
                                   t_max, h, tau, fictive_x=True)
        print('Created T...')
        # Создаем сетку для подсчета точного решения
        tt, xx = np.meshgrid(t_grid, x_grid)
        exact = exact_sol_func(tt, xx)
        print('Calculated EXACT...')
        # Граничные условия СПРАВА!!!
        # А вот начальные условия -- на t_min
        func_t, func_x = lambda t: exact_sol_func(t, x_max), lambda x: exact_sol_func(t_min, x)
        # Начальные условия
        T[:,0] = func_t(x_grid)
        # Граничные условия
        T[-1,:] = func_x(t_grid)
        print('Calculated inital values...')
        # Приводим все к double
        #T = T.astype('double')
        approx = kernel(T, sigma).T
        print('Calculated approx solution...')
        tmp.append(approx - exact[1:-1,:].T)
    return tmp
```

```python
tau = 10**(-2)/2
```

```python
# Все будем судить по 1-ой координате
N_coord = 0
h_range = np.linspace(10**(-2), 10**(-1), 30)
different_calcs = count_errors(jit(exact_sol_funcs[N_coord]), laks_kernel,
                                       tau, lambdas[N_coord],
                                       h_range,
                                       sigma=0.5)
```

```python
errors = np.array([np.linalg.norm(different_calcs[i].astype('float')) for i in range(len(different_calcs))])
```

```python
y_plot = np.log(errors)
x_plot = np.log(h_range)

plt.scatter(x_plot, y_plot)
N_forfit=int(len(x_plot)*1)
x_forfit, y_forfit = x_plot[:N_forfit], y_plot[:N_forfit]
plt.scatter(x_forfit, y_forfit, label='Сырые данные')

# Нанесем прямую МНК
from scipy.stats import linregress
slope, intercept, r_value, p_value, std_err = linregress(x_forfit, y_forfit)
x_plot_forfit = np.linspace(min(x_forfit), max(x_forfit), 10**3)
y_plot_forfit = slope*x_plot_forfit + intercept
plt.plot(x_plot_forfit, y_plot_forfit, label=r'МНК, $k={}$'.format(slope.round(2)))

plt.grid()
plt.legend()
plt.xlabel(r'$\log h$')
plt.ylabel(r'$\log e$')
plt.title(r'Зависимость $\log(error)$ от $\log(h)$')
plt.savefig('errors_laks.pdf')
plt.show()
```

```python
display(
    Markdown(
        'Апостериорный порядок сходмиости $k={}$ (коэфф. корреляции $r={}$)'.format(
            abs(slope.round(2)), abs(r_value.round(4))
        )
    )
)
```


```python
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import sympy as sy
sy.init_printing()
```

#  Null Space

The **null space**, denoted as $\text{Nul}A$ is the solution set of a homogeneous linear system, i.e. $Ax=0$.

A null space is a always a subspace of $\mathbb{R}^n$, why? Because the solution can always be at origin $(0, 0, ...)$.

As an example, consider a linear system.

$$
2x_1-x_2+x_3 = 0\\
x_1+2x_2+3x_3= 0
$$

The augmented matrix is

$$
\left[
\begin{matrix}
2 & -1 & 1 & 0\\
1 & 2 & 3 & 0
\end{matrix}
\right]
$$

Before solving the system, we have already known there is no unique solution since a free variable presents, due to fact that two equation with three variables.

Solve for the reduced echelon form.

```python
Aug = sy.Matrix([[2,-1,1,0],[1,2,3,0]])
Aug.rref()
```

$x_3$ is a free variable, the solution set can be written as

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3
\end{matrix}
\right]=
\left[
\begin{matrix}
-x_3 \\ -x_3 \\ x_3
\end{matrix}
\right]=
x_3\left[
\begin{matrix}
-1 \\ -1 \\ 1
\end{matrix}
\right]
$$

which is a line passing both origin $(0, 0, 0)$ and $(-1, -1, 1)$, also a subspace of $\mathbb{R}^3$.

Consider another example, suppose we have an augmented matrix

```python
Aug = sy.Matrix([[-3,6,-1,1,-7,0],[1,-2,2,3,-1,0],[2,-4,5,8,-4,0]]);Aug
```

```python
Aug.rref()
```

The solution can be written as:

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3 \\x_4 \\ x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
2x_2+x_4-3x_5 \\ x_2 \\ -2x_4+2x_5 \\x_4 \\ x_5
\end{matrix}
\right]=
x_2\left[
\begin{matrix}
2 \\ 1 \\ 0 \\0 \\ 0
\end{matrix}
\right]
+
x_4\left[
\begin{matrix}
1 \\ 0 \\ -2 \\1 \\ 0
\end{matrix}
\right]
+x_5\left[
\begin{matrix}
-3 \\ 0 \\ 2 \\0 \\ 1
\end{matrix}
\right]
$$

The $\text{Nul}A$ is a subspace in $\mathbb{R}^5$ with $\text{dim}A=3$.

#  Null Space vs Col Space

Consider matrix $A$

```python
A = sy.Matrix([[2,4,-2,1],[-2,-5,7,3],[3,7,-8,6]]);A
```

Column space is a subspace in $\mathbb{R}^n$, what is $n$? It is the number of rows, $n=3$.

Null space is a subspace in $\mathbb{R}^m$, what is $m$? It is the number of columns, $m=4$.

How to find any nonzero vector in $\text{Col}A$ and in $\text{Nul}A$?

Any column in a matrix can be a nonzero vector in $\text{Col}A$, for instance first column: $(2, -2, 3)^T$.

But to find a nonzero vector in null space requires some effort, construct the augmented matrix then turn it into rref.

```python
Aug = sy.Matrix([[2,4,-2,1,0],[-2,-5,7,3,0],[3,7,-8,6,0]]);Aug.rref()
```

The solution set with a free variable $x_3$ (because column 3 has no pivot) is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4
\end{matrix}
\right]=
\left[
\begin{matrix}
-9x_3 \\ 5x_3 \\ x_3\\0
\end{matrix}
\right]
$$

If we pick $x_3 =1$, a nonzero vector in $\text{Nul}A$ is $(-9, 5, 1, 0)^T$

Now consider two vectors

$$
u = \left[
\begin{matrix}
3 \\ -2 \\ -1\\ 0
\end{matrix}
\right],\qquad
v = \left[
\begin{matrix}
3 \\ -1\\3
\end{matrix}
\right]\\
$$

Is $u$ in $\text{Nul}A$? It can be verified easily

```python
u = sy.Matrix([[3],[-2],[-1],[0]])
A*u
```

$Au\neq \mathbf{0}$, therefore $u$ is not in $\text{Nul}A$.

Is $v$ in $\text{Col}A$? Construct an augmented matrix with $v$, then solve it

```python
v = sy.Matrix([[3],[-1],[3]])
A.row_join(v).rref()
```

The augmented matrix show there are solutions, i.e. $v$ is a linear combination of its column space basis, so $v$ is in $\text{Col}A$.

#  Row Space

The **Row space** denoted as $\text{Row}A$, contains all linear combination of row vectors and subspace in $\mathbb{R}^n$.

If we perform row operations on $A$ to obtain $B$, both matrices have the same row space, because $B$'s rows are linear combinations of $A$'s. However, row operation will change the row dependence.

##  An Example

Find the row, column and null space of

```python
A = sy.Matrix([[-2, -5, 8, 0, -17],
               [1, 3, -5, 1, 5],
               [3, 11, -19, 7, 1],
               [1, 7, -13, 5, -3]]);A
```

```python
B = A.rref();B
```

The basis of the row space of $B$ is its first 3 rows: $(1,0,1,0,1), (0, 1, -2, 0, 3), (0, 0, 0, 1, -5)$ which are also the basis of the row space of $A$. However it does not necessarily mean that first 3 rows of $A$ forms the basis for row space, because the dependence among rows changed by row operation.

In constrast, the basis of col space of $A$ is $(-2, 1, 3, 1)^T, (-5, 3, 11, 7)^T, (0, 1, 7, 5)^T$.

```python
Aug = A.row_join(sy.zeros(4,1));Aug.rref()
```

The null space is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
-x_3-x_5 \\ 2x_3-3x_5 \\ x_3\\5x_5 \\x_5
\end{matrix}
\right]=
x_3\left[
\begin{matrix}
-1 \\ 2 \\ 1\\0 \\0
\end{matrix}
\right]+
x_5
\left[
\begin{matrix}
-1 \\ -3 \\ 0\\5 \\1
\end{matrix}
\right]
$$

#  Rank

Definition of rank:
The **rank** is the dimension of the column space of $A$. The **nullity** of $A$ is the dimension of the null space.

##  The Rank Theorem

The dimensions of the column space and the row space of an $m \times n$ matrix $A$ are equal that is why we only need to say rank is the dimension of the column space.

This common dimension, the rank of $A$, also equals the number of pivot positions in $A$ and satisfies the equation
$$
\operatorname{rank} A+\operatorname{dim} \mathrm{Nul} A=n
$$

The intuition is that when a matrix $A$ is converted into rref $B$, we can indirectly(matching the same column from $B$ to $A$) see the basis of column space, those columns in corresponding rref have pivots.

And in rref, we can also see the basis of row space directly, every row in the basis of row space must have a pivot. And those rows which does not have pivots are for free variables, which is the dimension of null space.

##  Example 1

If $A$ is $45 \times 50$ matrix with a $10$-dimension nullity, what is the rank of $A$?

$10$-$D$ nullity means 10 free variables, so the pivots are $50-10=40$, which is also the rank of $A$.

##  Example 2

The matrices below are row equivalent.
$$
A=\left[\begin{array}{rrrrr}
2 & -1 & 1 & -6 & 8 \\
1 & -2 & -4 & 3 & -2 \\
-7 & 8 & 10 & 3 & -10 \\
4 & -5 & -7 & 0 & 4
\end{array}\right], \quad B=\left[\begin{array}{rrrrr}
1 & -2 & -4 & 3 & -2 \\
0 & 3 & 9 & -12 & 12 \\
0 & 0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 & 0
\end{array}\right]
$$
1. Find rank $A$ and $\operatorname{dim}$ Nul $A$
2. Find bases for Col $A$ and Row $A$.
3. What is the next step to perform to find a basis for Nul $A$ ?
4. How many pivot columns are in a row echelon form of $A^{T} ?$

1. $rank(A)=2$, because $B$ has two pivots. And nullity is the number of free variables, there are 3, so $\text{dim Nul}A = 3$.

```python
A = sy.Matrix([[2,-1,1,-6,8,0],
               [1,-2,-4,3,-2,0],
               [-7,8,10,3,-10,0],
               [4,-5,-7,0,4,0]])
A.rref()
```

2. Bases for $\text{Col}A$ is $(2,1,-7,4)^T, (-1,-2,8,-5)^T$, and for $\text{Row}A$ is $(1,-2,-4,3,-2),(0,3,9,-12,12)$.

The $\text{Nul}A$ and basis is

$$
\left[
\begin{matrix}
x_1 \\ x_2 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
\left[
\begin{matrix}
-2x_3+5x_4-6x_5 \\ -3x_3+4x_4-4x_5 \\ x_3\\x_4 \\x_5
\end{matrix}
\right]=
x_3
\left[
\begin{matrix}
-2 \\ -3 \\ 1\\0 \\0
\end{matrix}
\right]+
x_4
\left[
\begin{matrix}
5 \\ 4 \\ 0\\1 \\0
\end{matrix}
\right]+
x_5
\left[
\begin{matrix}
-6 \\ -4 \\ 0\\0 \\1
\end{matrix}
\right]
$$

3. Perform rref on augmented $A$

4. Transpose $A$ then do rref.

```python
A.T.rref()
```

There are 2 pivot columns.

Actually, we don't need any calculation to know the rank of $A^T$, because

$$
rank(A)=rank(A^T)
$$

#  Orthogonality of $\text{Nul}A$ and $\text{Row}A$

##   $\text{Nul}A \perp \text{Row}A$

Here is the intersting connections of these subspaces we have discussed. Consider

```python
A = sy.Matrix([[5, 8, 2], [10, 16, 4], [3, 4, 1]]);A
```

```python
A.rref()
```

The basis of row space of $A$ is $(1, 0, 0)$ and $(0, 1, .25)$.And the $\text{Row}A$ is

$$
\text{Row}A=
s\left[
\begin{matrix}
1 \\ 0\\ 0
\end{matrix}
\right]+
t\left[
\begin{matrix}
0 \\ 1\\ 0.25
\end{matrix}
\right]
$$

The $\text{Nul}A$ is
$$
\left[
\begin{matrix}
x_1 \\ x_2\\ x_3
\end{matrix}
\right]=
x_3
\left[
\begin{matrix}
0 \\ -.25\\ 1
\end{matrix}
\right]
$$

Now we can visualize their relations geometrically. Again keep in mind that Matplotlib does not render 3D properly, so you need some imagination as well.

Here is what we observe.

The $\text{Row}A$ is a plane and $\text{Nul}A$ is a line which is perpendicular to the plane. It is easy to grasp the idea if you notice that in a homogeneous system $Ab = \mathbf{0}$, it breaks down into many dot products

$$
Ab =\left[
\begin{matrix}
A_{1i}\cdot b \\ A_{2i}\cdot b\\ A_{3i}\cdot b
\end{matrix}
\right]
$$

where $A_{1i}, A_{2i}, A_{3i}$ are the rows of $A$. In later chapters we will prove when the dot product of two vectors equals zero, which means geometrically they are perpendicular.

```python
%matplotlib inline
s = np.linspace(-1, 1, 10)
t = np.linspace(-1, 1, 10)
S, T = np.meshgrid(s, t)

X = S
Y = T
Z = T*.25

fig = plt.figure(figsize = (8,8))

ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(X, Y, Z, alpha = .9, cmap=plt.cm.coolwarm)

x3 = np.linspace(-1, 1, 10)
x1 = 0*x3
x2 = -.25*x3
ax.plot(x1,x2,x3, lw = 5)

ax.set_xlabel('x-axis', size = 18)
ax.set_ylabel('y-axis', size = 18)
ax.set_zlabel('z-axis', size = 18)

ax.axis([-1,1,-1,1])

ax.text(x = 1, y = -1, z = -.25, s = r'$Row\ A$', size = 17)
ax.text(0, -.25, 1, s = r'$Nul\ A$', size = 17)

ax.view_init(7, 20)
plt.show()
```

##   $\text{Nul}A^T \perp \text{Col}A$

The nullity of $A^T$ is

```python
A = sy.Matrix([[5, 8, 2], [10, 16, 4], [3, 4, 1]]);A.T.rref()
```

The $\text{Nul}A^T$ is

$$
\left[
\begin{matrix}
x_1 \\ x_2\\ x_3
\end{matrix}
\right]=
x_2
\left[
\begin{matrix}
-2 \\ 1\\ 0
\end{matrix}
\right]
$$

The  $\text{Col}A$ is

```python
A.rref()
```

$$
\text{Col}A=
s\left[
\begin{matrix}
5 \\ 10\\ 3
\end{matrix}
\right]+
t\left[
\begin{matrix}
8 \\ 16\\ 4
\end{matrix}
\right]
$$

$\text{Col}A$ is a plane and $\text{Nul}A^T$ is a line perpendicular to the plane. The intuition is similar to $\text{Nul}A \perp \text{Row}A$, here you can think of a system look like $b^TA = \mathbf{0}^T$.

```python
%matplotlib inline
s = np.linspace(-1, 1, 10)
t = np.linspace(-1, 1, 10)
S, T = np.meshgrid(s, t)

X = 5*S+8*T
Y = 10*S+16*T
Z = 3*S+4*T

fig = plt.figure(figsize = (8,8))

ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(X, Y, Z, cmap=plt.cm.coolwarm)

x2 = np.linspace(-1, 1, 10)
x3 = x2*0
x1 = -2*x2

ax.plot(x1,x2,x3, lw = 3)

ax.set_xlabel('x-axis', size = 18)
ax.set_ylabel('y-axis', size = 18)
ax.set_zlabel('z-axis', size = 18)

ax.axis([-1,1,-1,1])

ax.view_init(-67, 35)
```

#  Rank Decomposition

Consider a matrix $A$, the purpose is to decompose it into the multiplication of $C$, $R$, which are the  bases of column space and row space respectively.

$$
A = CR
$$

```python
A = sy.Matrix([[2, 4, 1, -1], [4, 2, -4, 2], [2, -2, -5, 3], [1, 9, -3, 2]]);A
```

```python
Arref = A.rref();Arref
```

Get the basis of $\text{Col}A$.

```python
ColA_basis = A[:,:3];ColA_basis
```

Then get the $\text{Row}A$.

```python
RowA_basis = Arref[0][0:3,:];RowA_basis
```

Multiply $CR$, we are getting back $A$.

```python
ColA_basis*RowA_basis
```

Verify if $CR$ equals $A$.

```python
ColA_basis*RowA_basis == A
```

Mathematics/Linear Algebra/0.0 MIT-18.06 - Jupyter/Strang-Book-Linear_Algebra_With_Python-Jupyter-Notebooks/Chapter 10 -Null Space vs Col Space, Row Space and Rank.ipynb