# Binary Digit Classifier Using QNN with GUI input

### Project Desciption
The Project first aims to briefly introduce Quantum Neural Networks and then build a Quantum Neural Network (QNN) to classify handwritten 0 and 1 (using MNIST handwritten data). And then, we'll make a Graphical User Interface (GUI) using which the user can draw a digit. And afterward, we'll integrate the GUI with the QNN above. And then, we'll classify whether the user has made 0 or 1.

### References

- https://arxiv.org/pdf/1802.06002.pdf
- https://www.tensorflow.org/quantum/tutorials/mnist
- https://docs.python.org/3/library/tk.html
- https://tkdocs.com/tutorial/index.html
- https://pennylane.ai/qml/glossary/quantum_neural_network.html
- https://en.wikipedia.org/wiki/Quantum_neural_network

### What is Quantum Neural Networks ?

A quantum neural network (QNN) is a machine learning model or algorithm that combines concepts from quantum computing and artifical neural networks.Quantum Neural Network extends the key features and structures of Neural Networks to quantum systems.
Most Quantum neural networks are developed as feed-forward networks. Similar to their classical counterparts, this structure intakes input from one layer of qubits, and passes that input onto another layer of qubits. This layer of qubits evaluates this information and passes on the output to the next layer. Eventually the path leads to the final layer of qubits.

Fig1: Illustration of QNN with the input |ψ>, the parameter θ and linear entanglement structure.[image_source](https://arxiv.org/pdf/2108.01468.pdf)

Now let's start building the QNN Model

### Libraries Used

- **cirq**
- **tensorflow**
- **tensorflow_quantum**
- **numpy**
- **sympy**
- **seaborn**
- **matplotlib**
- **tkinter**
- **opencv**

### Importing Libraries

```python
import tensorflow as tf
import tensorflow_quantum as tfq

import cirq
import sympy
import numpy as np
import seaborn as sns
import collections

%matplotlib inline
import matplotlib.pyplot as plt
from cirq.contrib.svg import SVGCircuit

```

### Flowchart

### Index

#### 1. Data Loading, Filtering and Encoding
#####     1.1 Data Loading
#####     1.2 Data Filtering
#####     1.3 Downscaling Images to 4x4
#####     1.4 Removing Contradictory Examples
#####     1.5 Encoding the data as quantum Circuits
#### 2. Building QNN (Quantum Neural Network)
#####     2.1 Building the model Circuit
#####     2.2 Wrapping the model_circuit in a tfq.keras model
#####     2.3 Training and Evaluating QNN
#### 3. Saving QNN Model
#### 4. Making GUI using tkinter
#### 5. Integrating GUI with QNN Model

#### 1. Data Loading, Filtering and Encoding

##### 1.1 Data Loading

```python
#Loading MNIST Dataset
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Rescaling the images to [0.0,1.0] Range.
x_train, x_test = x_train[..., np.newaxis]/255.0, x_test[..., np.newaxis]/255.0

print("Number of training examples before filtering:", len(x_train))
print("Number of testing examples before filtering:", len(x_test))
```

    Number of training examples before filtering: 60000
    Number of testing examples before filtering: 10000

##### 1.2 Data Filtering

```python

# Defining Function to filter dataset to keep just 0's and 1's.
def filter_01(x, y):
    keep = (y == 0) | (y == 1)
    x, y = x[keep], y[keep]
    y = y == 0
    return x,y

# Filtering using Above Function to keep 0's and 1's
x_train, y_train = filter_01(x_train, y_train)
x_test, y_test = filter_01(x_test, y_test)

print("Number of training examples after filtering:", len(x_train))
print("Number of testing examples after filtering:", len(x_test))
```

    Number of training examples after filtering: 12665
    Number of testing examples after filtering: 2115

##### 1.3 Downscaling Images to 4x4

```python
downscaled_x_train = tf.image.resize(x_train, (4,4)).numpy()
downscaled_x_test = tf.image.resize(x_test, (4,4)).numpy()

```

```python
# Displaying the first training image before and after downscaling
print("Before Downscaling:")
plt.imshow(x_train[0,:,:,0], vmin=0, vmax=1)
plt.colorbar()
```

```python
print("After Downscaling:")
plt.imshow(downscaled_x_train[0,:,:,0], vmin=0, vmax=1)
plt.colorbar()
```

##### 1.4 Removing Contradictory Examples

```python
# Defining Function to remove conradictory Examples.
def remove_contradicting(xs, ys):
    mapping = collections.defaultdict(set)
    orig_x = {}
    # Determine the set of labels for each unique image:
    for x,y in zip(xs,ys):
       orig_x[tuple(x.flatten())] = x
       mapping[tuple(x.flatten())].add(y)

    new_x = []
    new_y = []
    for flatten_x in mapping:
      x = orig_x[flatten_x]
      labels = mapping[flatten_x]
      if len(labels) == 1:
          new_x.append(x)
          new_y.append(next(iter(labels)))
      else:
          # Throw out images that match more than one label.
          pass

    num_uniq_0 = sum(1 for value in mapping.values() if len(value) == 1 and True in value)
    num_uniq_1 = sum(1 for value in mapping.values() if len(value) == 1 and False in value)
    num_uniq_both = sum(1 for value in mapping.values() if len(value) == 2)

    print("Number of unique images:", len(mapping.values()))
    print("Number of unique 0s: ", num_uniq_0)
    print("Number of unique 1s: ", num_uniq_1)
    print("Number of unique contradicting labels (both 0 and 1): ", num_uniq_both)
    print()
    print("Initial number of images: ", len(xs))
    print("Remaining non-contradicting unique images: ", len(new_x))

    return np.array(new_x), np.array(new_y)

x_train_nocon, y_train_nocon = remove_contradicting(downscaled_x_train, y_train)

```

    Number of unique images: 7100
    Number of unique 0s:  4924
    Number of unique 1s:  2068
    Number of unique contradicting labels (both 0 and 1):  108

    Initial number of images:  12665
    Remaining non-contradicting unique images:  6992

##### 1.5 Encoding the data as quantum Circuits

```python
THRESHOLD = 0.5
x_train_bin = np.array(x_train_nocon > THRESHOLD, dtype=np.float32)
x_test_bin = np.array(downscaled_x_test > THRESHOLD, dtype=np.float32)

_ = remove_contradicting(x_train_bin, y_train_nocon)

# Defining Function to convert images to circuit
def convert_to_circuit(image):
    """Encode truncated classical image into quantum datapoint."""
    values = np.ndarray.flatten(image)
    qubits = cirq.GridQubit.rect(4, 4)
    circuit = cirq.Circuit()
    for i, value in enumerate(values):
        if value:
            circuit.append(cirq.X(qubits[i]))
    return circuit

x_train_circ = [convert_to_circuit(x) for x in x_train_bin]
x_test_circ = [convert_to_circuit(x) for x in x_test_bin]

```

    Number of unique images: 126
    Number of unique 0s:  88
    Number of unique 1s:  5
    Number of unique contradicting labels (both 0 and 1):  33

    Initial number of images:  6992
    Remaining non-contradicting unique images:  93

```python
print("Circuit for the first train example")
SVGCircuit(x_train_circ[0])
```

    Circuit for the first train example

```python
# Converting Cirq circuits to tensors for TensorflowQuantum
x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)
x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)
```

#### 2. Building QNN (Quantum Neural Network)

##### 2.1 Building the model Circuit

```python
class CircuitLayerBuilder():
    def __init__(self, data_qubits, readout):
        self.data_qubits = data_qubits
        self.readout = readout

    def add_layer(self, circuit, gate, prefix):
        for i, qubit in enumerate(self.data_qubits):
            symbol = sympy.Symbol(prefix + '-' + str(i))
            circuit.append(gate(qubit, self.readout)**symbol)

def create_quantum_model():
    """Create a QNN model circuit and readout operation to go along with it."""
    data_qubits = cirq.GridQubit.rect(4, 4)  # a 4x4 grid.
    readout = cirq.GridQubit(-1, -1)         # a single qubit at [-1,-1]
    circuit = cirq.Circuit()

    # Prepare the readout qubit.
    circuit.append(cirq.X(readout))
    circuit.append(cirq.H(readout))

    builder = CircuitLayerBuilder(
        data_qubits = data_qubits,
        readout=readout)

    # Then add layers (experiment by adding more).
    builder.add_layer(circuit, cirq.XX, "xx1")
    builder.add_layer(circuit, cirq.ZZ, "zz1")

    # Finally, prepare the readout qubit.
    circuit.append(cirq.H(readout))

    return circuit, cirq.Z(readout)

model_circuit, model_readout = create_quantum_model()
```

##### 2.2 Wrapping the model_circuit in a tfq.keras model

```python
# Build the Keras model.
model = tf.keras.Sequential([
    # The input is the data-circuit, encoded as a tf.string
    tf.keras.layers.Input(shape=(), dtype=tf.string),
    # The PQC layer returns the expected value of the readout gate, range [-1,1].
    tfq.layers.PQC(model_circuit, model_readout),
])

y_train_hinge = 2.0*y_train_nocon-1.0
y_test_hinge = 2.0*y_test-1.0

def hinge_accuracy(y_true, y_pred):
    y_true = tf.squeeze(y_true) > 0.0
    y_pred = tf.squeeze(y_pred) > 0.0
    result = tf.cast(y_true == y_pred, tf.float32)

    return tf.reduce_mean(result)

model.compile(
    loss=tf.keras.losses.Hinge(),
    optimizer=tf.keras.optimizers.Adam(),
    metrics=[hinge_accuracy])
print(model.summary())
```

    Model: "sequential"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    pqc (PQC)                    (None, 1)                 32
    =================================================================
    Total params: 32
    Trainable params: 32
    Non-trainable params: 0
    _________________________________________________________________
    None

##### 2.3 Training and Evaluating QNN

```python
EPOCHS = 4
BATCH_SIZE = 32
NUM_EXAMPLES = len(x_train_tfcirc)
x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]
y_train_hinge_sub = y_train_hinge[:NUM_EXAMPLES]

qnn_history = model.fit(
      x_train_tfcirc_sub, y_train_hinge_sub,
      batch_size=32,
      epochs=EPOCHS,
      verbose=1,
      validation_data=(x_test_tfcirc, y_test_hinge))

qnn_results = model.evaluate(x_test_tfcirc, y_test)
```

    Train on 6992 samples, validate on 2115 samples
    Epoch 1/4
    6992/6992 [==============================] - 1199s 172ms/sample - loss: 0.8508 - hinge_accuracy: 0.6936 - val_loss: 0.5303 - val_hinge_accuracy: 0.8577
    Epoch 2/4
    6992/6992 [==============================] - 1206s 173ms/sample - loss: 0.5344 - hinge_accuracy: 0.7653 - val_loss: 0.5292 - val_hinge_accuracy: 0.8806
    Epoch 3/4
    6992/6992 [==============================] - 1205s 172ms/sample - loss: 0.4670 - hinge_accuracy: 0.8172 - val_loss: 0.5940 - val_hinge_accuracy: 0.6244
    Epoch 4/4
    6992/6992 [==============================] - 1522s 218ms/sample - loss: 0.4504 - hinge_accuracy: 0.8457 - val_loss: 0.5614 - val_hinge_accuracy: 0.9058
    2115/2115 [==============================] - 25s 12ms/sample - loss: 0.5614 - hinge_accuracy: 0.9058

#### 3. Saving QNN Model

```python
model.save_weights('01_MNIST.h5')
```

#### 4. Making GUI using tkinter

####  [Will be updated]

#### 5. Integrating GUI with QNN Model

####  [Will be updated]

```python

```

Pt-1-Binary Digit Classifier Using QNN with GUI input.ipynb


# Parameter drift : Indentifiability

# Purpose
If the matematical model is not correct or too little data is available this may lead to paramter drift, so that the parameters in the matematical model changes depending on how the fitted data has been sampled.

# Methodology
* Sample data of forces from a higher order model
* Fit a lower order model to a random sample of this data

# Setup

```python
# %load imports.py
## Local packages:

%matplotlib inline
%load_ext autoreload
%autoreload 2
%config Completer.use_jedi = False  ## (To fix autocomplete)

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)
import numpy as np
np.set_printoptions(linewidth=150)

import numpy as np
import os
import matplotlib.pyplot as plt
#if os.name == 'nt':
#    plt.style.use('presentation.mplstyle')  # Windows

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

from src.symbols import *
from src.parameters import *
import src.symbols as symbols
from src import prime_system
from src.models.regression import ForceRegression, results_summary_to_dataframe
from src.models.diff_eq_to_matrix import DiffEqToMatrix
from src.visualization.regression import show_pred, show_pred_captive
from src.visualization.plot import track_plot,captive_plot

## Load models:
# (Uncomment these for faster loading):
import src.models.vmm_abkowitz  as vmm
import src.models.vmm_martin as vmm_simpler

from src.models.vmm import ModelSimulator

from src.data.wpcc import ship_parameters, df_parameters, ps, ship_parameters_prime, ps_ship, scale_factor

```

```python
#format the book
import src.visualization.book_format as book_format
book_format.set_style()
```

## Load model

```python
model = ModelSimulator.load('../models/model_VCT_abkowitz.pkl')
```

```python
u0_=2
angle_deg = 20
result = model.zigzag(u0=u0_, angle=angle_deg)
```

```python
result.track_plot();
result.plot(compare=False);
```

```python
df_result = result.result.copy()
df_result_prime = model.prime_system.prime(df_result, U=df_result['U'])
```

```python

```

```python
def variate(df, variation_keys, N=10):

    variations = []
    for variation_key in variation_keys:
        variation = np.linspace(df[variation_key].min(),df[variation_key].max(), N)
        variations.append(variation)

    matrix = np.meshgrid(*variations)
    df_variation = pd.DataFrame()
    for variation_key,values in zip(variation_keys,matrix):
        df_variation[variation_key] = values.flatten()

    return df_variation
```

```python
variations = {
    'Rudder angle' : ['delta'],
    'Drift angle' : ['v'],
    'Circle' : ['r'],
    'resistance' : ['u'],
    "Rudder and drift angle" : ['delta','v'],
    "Circle + Drift" : ['r','v'],

}

N = 10
V_ = u0_

inputs_base = {}
inputs_base['u'] = df_result_prime['u'].mean()
inputs_base['v'] = 0
inputs_base['r'] = 0
inputs_base['delta'] = 0

df_inputs = pd.DataFrame()
for test_type, variation_keys in variations.items():

    inputs = variate(df=df_result_prime, variation_keys=variation_keys, N=N)

    for column in list(set(inputs_base.keys())-set(variation_keys)):
        inputs[column]=inputs_base[column]

    inputs['test type'] = test_type
    df_inputs = df_inputs.append(inputs, ignore_index=True)

df_outputs = model.forces(df_inputs)
df_captive = pd.concat([df_inputs,df_outputs], axis=1)

```

```python
captive_plot(df_captive=df_captive, suffixes=[],
             legends = ['VCT'], styles=['.', '-'])
```

## Fit a lower order model to this captive dataset

### Regression

```python
reg = ForceRegression(vmm=vmm, data=df_captive)
display(reg.show_pred_X())
display(reg.show_pred_Y())
display(reg.show_pred_N())
```

```python

```

```python
parameters = pd.DataFrame()
parameters['prime'] = model.parameters
model_vct = reg.create_model(df_parameters=parameters, ship_parameters=model.ship_parameters,
                             ps=model.prime_system, control_keys=['delta'])
```

```python
outputs = model_vct.forces(inputs = df_inputs)
df_captive_all = pd.merge(left=df_captive, right=outputs,
                      how='left',
                      left_index=True,
                      right_index=True,
                      suffixes = ('','_model'),
                      )

captive_plot(df_captive=df_captive_all, suffixes=['_model'],
             legends = ['VCT', 'model'], styles=['.', '-'])
```

```python
#result_vct = model_vct.zigzag(u0=u0_, angle=angle_deg)
result_vct = model_vct.simulate(df_result.loc[0:6])
```

```python
result_vct.track_plot(compare=True);
result_vct.plot(compare=True);
```

```python
variation_keys = ['u','v','r','delta']
df_inputs = variate(df=df_result_prime, variation_keys=variation_keys, N=8)
df_outputs = model.forces(df_inputs)
df_captive_all = pd.concat([df_inputs,df_outputs], axis=1)
```

```python
len(df_captive_all)
```

```python
reg_all = ForceRegression(vmm=model, data=df_captive_all)
display(reg_all.show_pred_X())
display(reg_all.show_pred_Y())
display(reg_all.show_pred_N())
```

```python
model_all = reg_all.create_model(df_parameters=parameters, ship_parameters=model.ship_parameters,
                             ps=model.prime_system, control_keys=['delta'])
```

```python
#result_all = model_all.simulate(df_result)
result_all = model.simulate(df_result)
```

```python
result_all.plot_compare();
```

```python
df_compare_parameters =pd.DataFrame()
df_compare_parameters['model'] = model.parameters
df_compare_parameters['model captive all'] = model_all.parameters
df_compare_parameters['model captive 1'] = model_vct.parameters

df_compare_parameters['model_abs'] = df_compare_parameters['model'].abs()
df_compare_parameters.sort_values(by='model_abs', ascending=False, inplace=True)
df_compare_parameters.drop(columns=['model_abs'], inplace=True)

df_compare_parameters = df_compare_parameters.divide(df_compare_parameters['model'], axis=0)

df_compare_parameters['dof'] = pd.Series(df_compare_parameters.index).apply(lambda x:x[0]).values

for dof, df_ in df_compare_parameters.groupby(by='dof', sort=False):
    fig,ax=plt.subplots()
    fig.set_size_inches(10,2)
    df_.plot(kind='bar', ax=ax)
    fig.suptitle(dof)
```

```python

```

```python

```

```python

```

```python

```

```python
df_captive = df_VCT_prime.copy()
df_captive['test type'] = df_VCT['test type']

N = len(df_captive)
N_sample = N - 5
df_captive_sample = df_captive.sample(n=N_sample, random_state=42)
```

```python
N
```

```python
model_names = [f'{i}' for i in range(20)]

df_captive_all = df_captive.copy()
np.random.seed(42)

models = {}

for model_name in model_names:

    df_captive_sample = df_captive.sample(n=N_sample)

    reg = ForceRegression(vmm=vmm, data=df_captive_sample)
    model_reg = reg.create_model(df_parameters=df_parameters, ship_parameters=ship_parameters, ps=ps)

    models[model_name] = model_vct = reg.create_model(df_parameters=df_parameters, ship_parameters=ship_parameters, ps=ps)

    outputs = model_reg.forces(inputs = df_captive)
    df_captive_all = pd.merge(left=df_captive_all, right=outputs,
                          how='left',
                          left_index=True,
                          right_index=True,
                          suffixes = ('',f'_{model_name}'),
                          )
```

```python
suffixes = [f'_{model_name}' for model_name in model_names]
styles = ['r.'] + ['b-' for model_name in model_names]

legends = ['VCT'] + model_names

captive_plot(df_captive=df_captive_all, suffixes=suffixes,
             legends = legends, styles=styles, alpha=0.2, lw=2, add_legend=False)
```

```python
df_results = pd.DataFrame()
result = model_vct.zigzag(u0=2, angle=30)

for model_name, model in models.items():
    result_ = model.simulate(result.result)
    df_ = result_.result
    df_['t'] = df_.index
    df_['model_name'] = model_name
    df_results = df_results.append(df_, ignore_index=True)

```

```python
from src.visualization.plot import track_plot

fig,ax=plt.subplots()
fig.set_size_inches(10,10)

for model_name, df_ in df_results.groupby(by='model_name'):

    df_.plot(x='x0', y='y0', ax=ax, alpha=0.2, lw=3, style='b-')

result.result.plot(x='x0', y='y0', style='k-', zorder=10, ax=ax)

ax.set_xlabel('x0 [m]')
ax.set_ylabel('y0 [m]')

ax.set_aspect("equal")
ax.set_title("Track plot")
ax.get_legend().set_visible(False)
ax.grid(True)
handles, labels = ax.get_legend_handles_labels()
ax.legend(handles[-2:],['simulations','model test'])
```

```python
fig,ax=plt.subplots()
fig.set_size_inches(14,3)

df_results['psi_deg'] = np.rad2deg(df_results['psi'])

df_results_ = result.result.copy()
df_results_['-delta_deg'] =-np.rad2deg(df_results_['delta'])
df_results_['psi_deg'] = np.rad2deg(df_results_['psi'])

for model_name, df_ in df_results.groupby(by='model_name'):

    df_.plot(x='t', y='psi_deg', ax=ax, alpha=0.2, lw=3, style='b-')

df_results_.plot(y='psi_deg', ax=ax, style='k-', zorder=10)
df_results_.plot(y='-delta_deg', ax=ax, style='m-', zorder=10)

ax.set_xlabel('time [s]')
ax.set_ylabel('Heading $\psi$ [deg]')

ax.set_title("ZigZag30/30")

handles, labels = ax.get_legend_handles_labels()
ax.legend(handles[-3:],['alternative models','model','rudder angle'])

ax.grid(True)
ax.set_ylim(-60,60)
```

```python
model_vct.parameters
```

```python

```

```python

```


```python
%matplotlib inline
from pyvista import set_plot_theme
set_plot_theme('document')
```

# 05 - Monte Carlo rejection and ensemble conditioning

The created geological models with gempy were exported as SHEMAT-Suite input files. `SHEMAT-Suite <https://git.rwth-aachen.de/SHEMAT-Suite/SHEMAT-Suite-open>`_ [1] is a code for
solving coupled heat transport in porous media. It is written in fortran and uses a finite differences scheme in a hexahedral grid.
In this example, we will load a heat transport simulation from the base POC model we created in "Geological model creation and gravity simulation". We will demonstrate methods contained
in OpenWF for loading the result file, displaying the parameters it contains and how to visualize these parameters. Finally, we will calculate the conductive heat flow and plot it.

```python
# import libraries
import warnings
warnings.filterwarnings("ignore")
import h5py
import numpy as np
import pandas as pd
import sys
sys.path.append('../../')

import OpenWF.postprocessing as pp
import random
import gempy as gp
from gempy.bayesian.fields import probability, information_entropy

import matplotlib.pyplot as plt

print(f"Run mit GemPy version {gp.__version__}")
```

## Rejection algorithm based on random walk
We created a tiny ensemble of 10 different SHEMAT-Suite models in the previous step and will use a rejection algorithm to get a posterior ensemble of models.
For this, we "borrow" the Metropolis acceptance probability which is defined as:

\begin{align}\alpha(x_{t-1},z) = \begin{cases} min\big(\frac{p(z)}{p(x_{t-1})},1\big), & \text{if } p(x_{t-1}) > 0\\
  1, & \text{if } p(x_{t-1}) = 0 \end{cases}\end{align}

A different approach would be to assess the missfit (as RMS error) of each realisation.

\begin{align}\alpha(x_{t-1},z) = \begin{cases} exp\big(-\frac{S(z) - S(x_{t-1}) }{u_T}\big), & \text{if } S(z) > S(x_{t-1})\\
  1, & \text{otherwise }  \end{cases}\end{align}

We will use the second approach for now. As discretization error, we take a value from Elison(2015), $u_{T-discr} = 0.7$ K, an estimate of error. This error should
be estimated to best knowledge.

Using Gauss error propagation, we assess a potential error for the realisations.

\begin{align}u_T = \sqrt{\big(\frac{\partial T}{\partial x_1}u_1 \big)^2 + ... + \big(\frac{\partial T}{\partial x_n}u_n \big)^2}\end{align}

## Literature sources for log-errors:
Multiple additional error-sources exist for borehole temperatures. Bottom hole temperatures, a standard measurement when it comes to
temperature estimates from boreholes, of ± 3 to 5 K, even when measured after a long enough time [1].
For logging devices, temperature logs can have effective accuracies between ± 0.25 and 0.5 °C [2]. In addition to the device specific
accuracies, errors can be introduced by the measurement procedure itself, e.g. by the logging speed [3].

## Model preparation
To see, where our data points are situated, we load the model topography and plot the position of gravity stations and temperature boreholes:

```python
# import DTM
dtm = np.load('../../models/Graben_base_model_topography.npy')

# load base model
model_path = '../../models/2021-06-04_POC_base_model/'

geo_model = gp.load_model('POC_PCT_model',
                          path=model_path, recompile=False)

# get delx and dely of the model, so cell sizes
delx = geo_model._grid.regular_grid.dx
dely = geo_model._grid.regular_grid.dy
delz = geo_model._grid.regular_grid.dz

# import gravity data and borehole locations
g_data = pd.read_csv('../../models/2021-06-16_grav_of_POC_base_model.csv')
bhole = np.array([[31, 14],
                 [78, 22],
                 [53, 34],
                 [49, 44]])

# plot the map
fig = plt.figure(figsize=[15,7])
cs = plt.contourf(dtm[:,:,0], dtm[:,:,1], dtm[:,:,2],20, cmap='gist_earth')
plt.contour(dtm[:,:,0], dtm[:,:,1], dtm[:,:,2],10, colors='gray', zorder=1)
plt.scatter(g_data['X'], g_data['Y'], marker='s', s=150, c='brown', edgecolor='k',
            label='gravity stations', zorder=2)
plt.scatter(bhole[:,0]*delx, bhole[:,1]*dely, marker='^', s=200, c='k', label='boreholes',
           zorder=3)
plt.colorbar(cs, label='elevation [m]')
plt.legend(frameon=True)
plt.xlabel('X [m]')
plt.ylabel('Y [m]');
```

## Load the Lithology Blocks
First let's load the lithology block of all 10 models, looking at the probabilities of the graben unit and at the model entropy.

```python
# load and calculate Probability and Entropy using GemPy bayesian field functions
ens = np.load('../../data/outputs/MCexample_10realizations.npy')
prior_prob = probability(ens)
prior_entr = information_entropy(prior_prob)

layer = 5
# upper filling
gp.plot_2d(geo_model,
                  show_lith=False, show_boundaries=False, show_data=False,
                  regular_grid=prior_prob[layer],
                  kwargs_regular_grid={'cmap': 'viridis',
                                        'norm': None})
# lower filling
gp.plot_2d(geo_model,
                  show_lith=False, show_boundaries=False, show_data=False,
                  regular_grid=prior_prob[layer-1],
                  kwargs_regular_grid={'cmap': 'viridis',
                                        'norm': None});

p2dp = gp.plot_2d(geo_model,
                  show_lith=False, show_boundaries=False, show_data=False,
                  regular_grid=prior_entr,
                  kwargs_regular_grid={'cmap': 'magma',
                                       'norm': None,
                                      'colorbar': True}
                  )
```

The Information entropy plot shows where the maximal Uncertainty is in our model, i.e. where the contacts are between the graben units and the basement.
A lot of uncertainty is visible in the right part of the model (between around 16000 and 20000), where the main graben unit may or may not be present.

## Gravity rejection
In a first stage, we take a look at the gravity signal of each realization. The gravity signal is "recorded" at each of the squares you see in the plot above.
Comparing the recorded gravity signals of each realization with the ones of the base model (which we regard as the "true" observations), we can differentiate between fitting and non-fitting ensemble
members.

```python
g_simu = pd.read_csv('../../data/outputs/MCexample_10grav.csv')
g_simu.head()
```

Remember the $u_T$ from previously? Here, we estimate it from an artificially superimposed noise on the data. As our "observed data" is actually just the simulated gravity from the base model,
it does not have noise. That's why we artificially add it. This would not be needed with real data.

```python
add_noise = True
if add_noise==True:
    np.random.seed(27)
    noise = np.random.normal(0, 1., size=15)
    g_data_noise = g_data.copy()
    g_data_noise['grav'] = g_data_noise['grav'] + noise
    print(np.mean(noise))
    u_g = np.mean(noise)
elif add_noise==False:
    u_g = 0.5

#calculate stdeviation and mean of the prior ensemble
g_simu_stdev = g_simu.std(axis=1)
g_simu_mean = g_simu.mean(axis=1)

fig = plt.figure(figsize=[15,7])
cs = plt.contourf(dtm[:,:,0], dtm[:,:,1], dtm[:,:,2],20, cmap='gist_earth')
plt.contour(dtm[:,:,0], dtm[:,:,1], dtm[:,:,2],10, colors='gray', zorder=1)
cs = plt.scatter(g_data['X'], g_data['Y'], c=g_simu_stdev, marker='s',
                 s=100, zorder=2, cmap='magma')
plt.xlabel('x (m)')
plt.ylabel('y (m)')
plt.colorbar(cs, label='standard deviation');
```

In the plot above, we see the distribution of the standard deviation of our gravity stations, so already where the most "sensitive" stations are. For a better performing rejection, it may be suitable
to remove redundant stations, i.e. once with a very low standard deviation.
Now, for the MonteCarlo rejection step, we use an implemented method `rejection`, which goes through the RMSE vector of our realizations and compares the RMSE of each realization. The ones with relatively
lower RMSE will get chosen:

```python
rmse = pp.c_rmse(g_simu, g_data['grav'])
accept_g, P = pp.rejection(rmse=rmse, rnseed=random.seed(7), u_g=u_g, median=False)
print(f"Accepted realizations are {accept_g}.")
```

The RMSE of our realizations is:

```python
rmse
```

Having accepted some of the initial 10 realizations, we can again calculate the probability field for different units
and the model entropy:

```python
accepted_reals = ens[accept_g, :]
grav_prob = probability(accepted_reals)
grav_entr = information_entropy(grav_prob)

p2dp = gp.plot_2d(geo_model,
                  show_lith=False, show_boundaries=False, show_data=False,
                  regular_grid=grav_entr,
                  kwargs_regular_grid={'cmap': 'magma',
                                       'norm': None}
                  )
```

Comparing this to the Entropy plot from above, we see that the overall entropy is reduced in many parts of the model. Also the "thickness" of the areas with increased entropy is reduced, hinting at a
sucessful reduction of depth uncertainty for, e.g. the graben units. We now go ahead and save the lithology blocks of the accepted realizations, as these could now be used for subsequent
heat tranpsort simulations.

```python
np.save('../../data/outputs/lith_blocks_accepted.npy', accepted_reals)
np.savetxt('../../data/outputs/accepted_realizations_ID.txt', accept_g, fmt='%d')
```

Remember how in a previous tutorial step ("Create SHEMAT-Suite models"), we created SHEMAT-Suite models for the whole 10 realizations, i.e. for the whole _apriori_ ensemble?
Following the workflow with sequentially constraining the model space, we wouldn't actually need to create a SHEMAT-Suite model for every ensemble member, but just for the accepted realizations.
Which means, in this case:

```python
print(f"Realizations accepted: {accept_g}.")
```

This means, we'd only need to run heat-transport simulations for the realizations accepted by the gravity rejection step.

## Temperature rejection
The black triangles in the Map plot are the locations from 4 different boreholes in the model. Temperature data from these boreholes is now used in a similar fashion to further reduce the model to
realizations, which now fit both the gravity and the temperature signal.

Similarly to the previous tutorial, where we saved the base model as a SHEMAT-Input file, we now do the same with the accepted realizations:

```python
f = h5py.File('../../models/SHEMAT-Suite_output/SHEMAT_PCT_base_model_temp_final.h5','r')

z,y,x = f['uindex'].shape
```

The openWF package hase one plotting function for displaying arbitary slices through the SHEMAT model, as presented in a previous tutorial step. Here, we have a look at the temperature field
of the base model:

```python
fig = plt.figure(figsize=[15,7])
pp.plot_slice('../../models/SHEMAT-Suite_output/SHEMAT_PCT_base_model_temp_final.h5',
              parameter='temp', cell_number=25, direction = 'y', model_depth=6500)
```

Similar to the previous step, where we estimated $u_g$, we now have to estimate $u_T$ for temperature. There we use some literature estimations for errors introduced in measurements of
borehole temperatures:

```python
# define uT
T_error = 0.25 # temperature error tool accuracy
s_error = pp.fahrenheit_to_celsius(1.25, difference=True) # sensor response time of 2 sec and 1 year after drilling
l_error = pp.fahrenheit_to_celsius(1.25, difference=True) # logging speed of 20/ft after 1 year
d_error = 1.0 # estimated temperature error by discretization

u_T = np.sqrt(T_error**2 + s_error**2 + l_error**2 + d_error**2)
print(u_T)
```

SHEMAT-Suite, in a specific operation mode, writes ASCII files, ending on `.dat`. In these files, SHEMAT-Suite provides information about simulated variables (in our case temperature)
compared to observed ones. OpenWF has a routine to read in these `.dat` files and display them as a pandas dataframe:

```python
pp.load_inv('../../models/SHEMAT-Suite_output/POC_base_model_final.dat')
```

Now let's load all these simulation files from our ensemble. As we already simulated all 10 realizations of the apriori ensemble, we load all 10 dat-files. However, in a sequential
conditioning workflow, we'd just have the simulations from realizations accepted by the gravity conditioning step.

```python
outp_path = '../../models/SHEMAT-Suite_output/'

diffs = np.loadtxt(outp_path+f'POC_MC_{accept_g[0]}_final.dat',skiprows=3,usecols=(8,),dtype=float)
for i in accept_g[1:]:
    n = np.loadtxt(outp_path+f'POC_MC_{i}_final.dat',skiprows=3,usecols=(8,),dtype=float)
    diffs=np.vstack([diffs,n])
```

The diffs array we now created consists of the stacked 9th columns of each `.dat` file in the accepted realizations. Which means, as we have in total 5 accepted realizations, that array has 5 rows and
128 columns (i.e. the number of measuring points).

As we already have the differences between observed and simulated values here (so difference between the columns `calc` and `obs` in the dataframe above), we do not need to use the
`calc_rmse` method from above. Instead, we calculate it directly drom the diffs array, by first calculating the Sum of Squared Residuals (SSR) and then the RMSE:

```python
# calculate the Sum of Squared Residuals
diffs_sq = diffs**2
ssr = diffs_sq.sum(axis=1)

# calculate RMSE of each realisation.
n = diffs.shape[1] # as we have 5 data sources / boreholes for temperature
rmse_T = np.sqrt((ssr/n))
```

We can now continue to work with the `rmse` array, but for having a complete information array, we stack it to the `diffs` array. This can be neat, e.g. for storing the diffs and RMSE in one file.
When we stack the calculated parameters, we'll end up with an array with 130 columns. The first 128 columns are the differences between observed and calculated values, the 129th the SSR, and the
130th column the RMSE.

To have information, which realizations (after constraining from gravity) these differences belong to, we finally add a 131st column, containing the realization number:

```python
total_diffs = np.column_stack((diffs,ssr,rmse_T))
# add index to the realizations
ind = np.array(range(total_diffs.shape[0]))
total_diffs = np.column_stack((total_diffs,accept_g))
print(total_diffs.shape)
```

Optionally, we can then save it in an ASCII file:

```python
np.savetxt('../../models/SHEMAT-Suite_output/differences_RMSE.txt', total_diffs, fmt='%.4f')
```

## Rejection sampling Temperature
Similar to the gravity rejection step before, we now work with the temperature RMSE to reject samples. However, this step might be somewhat ambigous:
Because we're only looking at conductive heat transport in this tutorial example, the sole difference between realizations will be based on the thickness of the geological unit with its thermal
conductivtiy. As thermal conductivity does not vary as much as, let's say permeabiliy, the effect on the overall temperature field will likely be small.

Considering the estimated error above, the error might as well be in the same region as the differences of the simulations, yielding an unsatisfactory rejection:

```python
accept_T, P_T = pp.rejection(rmse=rmse_T, rnseed=random.seed(1), u_g=u_T)

accept_T
```

To show, what the simple implemented rejection method does, we write it out in the following code segment. We chronologically go through the rmse array, as this is the result of a simple MC simulation.
This means, this start from 1 to N can be used here, if samples generated are already in a random order and not correlated. That is usually the case with GemPy exports to SHEMAT-Suite.

```python
# First we fix the random seed of this Jupyter cell to the same as the previous method
random.seed(1)
# The RMSE is in the 130th column. With Python indexing starting at 0, this means we choose column 129
col = 129

# We choose a reference. There are two options in the implemented method.
# One, to start from the median value, as shown here, one for starting at the first realization.
Ref = np.median(total_diffs[:,col])
accept = []
P = []
k=0
# then we iterate through the different RMSE values
for i in range(total_diffs.shape[0]):
    # if the current iteration has a smaller RMSE then the reference, we take it and mark it as the
    # new reference
    if total_diffs[i,col] < Ref:
        Ref = total_diffs[i,col]
        accept.append(i)

    # else we only accept it with a certain probability, defined by the exponential in the equation
    # at the beginning
    elif random.random() < np.exp(-(total_diffs[i,col] - Ref)/(u_T)):
        P.append(np.exp(-(total_diffs[i,col] - Ref)/(u_T)))
        Ref = total_diffs[i,col]
        accept.append(i)

print(f"Accepted realizations are: {accept}")
```

With the model space now reduced to three models, we can calculate for a final time the probability of the model units and entropy of the model:

```python
accepted_reals_T = accepted_reals[accept, :]
grav_T_prob = probability(accepted_reals_T)
grav_T_entr = information_entropy(grav_T_prob)

p2dp = gp.plot_2d(geo_model,
                  show_lith=False, show_boundaries=False, show_data=False,
                  regular_grid=grav_T_entr,
                  kwargs_regular_grid={'cmap': 'magma',
                                       'norm': None}
                  )
```

The plot above shows a strong "binary" entropy field. Entropy is maximum (bright color) especially with respect to the depth of the post-graben unit interface.
The area of interest, however, the depth of the graben is now significantly reduced. We see, that the two resulting models do not differ that much with respect to graben depth, but only depth of the
post-graben unit.

```python
print(f"So, the final realizations which remain from the original {[accept_g[real] for real in accept]}!")
```

## References
| [1] Schumacher, S., & Moeck, I. (2020). A new method for correcting temperature log profiles in low-enthalpy plays. Geothermal Energy, 8(1), 1-23.
| [2] Blackwell, D.D., and Spafford, R.E., 1987, Experimental methods in continental heat flow, chapter 14, in Sammis, C.G., and Henyey, T.L, eds., Geophysics, part B, field measurements: Academic Press, Inc., Orlando, Methods of Experimental Physics, v. 24, p. 189-226.
| [3] Sharma, J., Adnyana, G., Barnes, D., Mims, D., & Behrens, R. (2021). Temperature logging guidelines and factors that affect measurement accuracy in steamfloods. Journal of Petroleum Science and Engineering, 196, 107727.


The famous *diffusion equation*, also known as the *heat equation*,
reads

$$
\frac{\partial u}{\partial t} =
\dfc \frac{\partial^2 u}{\partial x^2},
$$

where $u(x,t)$ is the unknown function to be solved for, $x$ is a coordinate
in space, and $t$ is time. The coefficient $\dfc$ is the *diffusion
coefficient* and determines how fast $u$ changes in time. A quick
short form for the diffusion equation is $u_t = \dfc u_{xx}$.

Compared to the wave equation, $u_{tt}=c^2u_{xx}$, which looks very similar,
the diffusion equation features solutions that are very different from
those of the wave equation. Also, the diffusion equation
makes quite different demands to the numerical
methods.

Typical diffusion problems may experience rapid change in the very
beginning, but then the evolution of $u$ becomes slower and slower.
The solution is usually very smooth, and after some time, one cannot
recognize the initial shape of $u$. This is in sharp contrast to
solutions of the wave equation where the initial shape is preserved in
homogeneous media - the solution is then basically a moving initial
condition. The standard wave equation $u_{tt}=c^2u_{xx}$ has solutions
that propagate with speed $c$ forever, without changing shape, while
the diffusion equation converges to a *stationary solution* $\bar
u(x)$ as $t\rightarrow\infty$. In this limit, $u_t=0$, and $\bar u$ is
governed by $\bar u''(x)=0$.  This stationary limit of the diffusion
equation is called the *Laplace* equation and arises in a very wide
range of applications throughout the sciences.

mathcal{I}_t is possible to solve for $u(x,t)$ using an explicit scheme, as we
do in the section [An explicit method for the 1D diffusion equation](#diffu:pde1:FEsec), but the time step restrictions
soon become much less favorable than for an explicit scheme applied to
the wave equation. And of more importance, since the solution $u$ of
the diffusion equation is very smooth and changes slowly, small time
steps are not convenient and not required by accuracy as the diffusion
process converges to a stationary state. Therefore, implicit schemes
(as described in the section [Implicit methods for the 1D diffusion equation](#diffu:pde1:implicit)) are popular, but
these require solutions of systems of algebraic equations. We shall
use ready-made software for this purpose, but also program some simple
iterative methods.
The exposition is, as usual in this book, very basic and focuses on
the basic ideas and how to implement. More comprehensive mathematical
treatments and classical analysis
of the methods are found in lots of textbooks. A favorite
of ours in this respect is the one by LeVeque [[LeVeque_2007]](#LeVeque_2007).
The books by Strikwerda [[Strikwerda_2007]](#Strikwerda_2007) and by
Lapidus and Pinder [[Lapidus_Pinder_1982]](#Lapidus_Pinder_1982) are also highly recommended
as additional material on the topic.

# An explicit method for the 1D diffusion equation

Explicit finite difference methods for the wave equation $u_{tt}=c^2u_{xx}$
can be used, with small modifications, for solving $u_t = \dfc u_{xx}$
as well.
% if BOOK == "book":
The exposition below assumes that the reader is familiar with the
basic ideas of discretization and implementation of wave
equations from the chapter [ch:wave](#ch:wave). Readers not familiar with the
Forward Euler, Backward Euler, and Crank-Nicolson (or centered or
midpoint) discretization methods in time should consult, e.g., Section 1.1
in [[Langtangen_decay]](#Langtangen_decay).
% endif

## The initial-boundary value problem for 1D diffusion

To obtain a unique solution of the diffusion equation, or equivalently,
to apply numerical methods, we need initial and boundary conditions.
The diffusion equation goes with one initial condition $u(x,0)=I(x)$, where
$I$ is a prescribed function. One boundary condition is required at
each point on the boundary, which in 1D means that $u$ must be known,
$u_x$ must be known, or some combination of them.

We shall start with the simplest boundary condition: $u=0$. The
complete initial-boundary value diffusion problem in one space
dimension can then be specified as

$$
\begin{equation}
\frac{\partial u}{\partial t} =
\dfc \frac{\partial^2 u}{\partial x^2} + f, \quad x\in (0,L),\ t\in (0,T]
\label{diffu:pde1} \tag{1}
\end{equation}
$$

$$
\begin{equation}
u(x,0) = I(x), \quad  x\in [0,L]
\label{diffu:pde1:ic:u} \tag{2}
\end{equation}
$$

$$
\begin{equation}
u(0,t)  = 0, \quad  t>0,
\label{diffu:pde1:bc:0} \tag{3}
\end{equation}
$$

$$
\begin{equation}
u(L,t)  = 0, \quad  t>0\thinspace .
\label{diffu:pde1:bc:L} \tag{4}
\end{equation}
$$

With only a first-order derivative in time,
only one *initial condition* is needed, while the second-order
derivative in space leads to a demand for two *boundary conditions*.
We have added a source term $f=f(x,t)$, which is
convenient when testing implementations.

Diffusion equations like ([1](#diffu:pde1)) have a wide range of
applications throughout physical, biological, and financial sciences.
One of the most common applications is propagation of heat, where
$u(x,t)$ represents the temperature of some substance at point $x$ and
time $t$. Other applications are listed in the section [diffu:app](#diffu:app).

## Forward Euler scheme

The first step in the discretization procedure is to replace the
domain $[0,L]\times [0,T]$ by a set of mesh points. Here we apply
equally spaced mesh points

$$
x_i=i\Delta x,\quad i=0,\ldots,N_x,
$$

and

$$
t_n=n\Delta t,\quad n=0,\ldots,N_t \thinspace .
$$

Moreover, $u^n_i$ denotes the mesh function that
approximates $u(x_i,t_n)$ for $i=0,\ldots,N_x$ and $n=0,\ldots,N_t$.
Requiring the PDE ([1](#diffu:pde1)) to be fulfilled at a mesh point $(x_i,t_n)$
leads to the equation

$$
\begin{equation}
\frac{\partial}{\partial t} u(x_i, t_n) =
\dfc\frac{\partial^2}{\partial x^2} u(x_i, t_n) + f(x_i,t_n),
\label{diffu:pde1:step2} \tag{5}
\end{equation}
$$

The next step is to replace the derivatives by finite difference approximations.
The computationally simplest method arises from
using a forward difference in time and a central difference in
space:

$$
\begin{equation}
[D_t^+ u = \dfc D_xD_x u + f]^n_i \thinspace .
\label{diffu:pde1:step3a} \tag{6}
\end{equation}
$$

Written out,

$$
\begin{equation}
\frac{u^{n+1}_i-u^n_i}{\Delta t} = \dfc \frac{u^{n}_{i+1} - 2u^n_i + u^n_{i-1}}{\Delta x^2} + f_i^n\thinspace .
\label{diffu:pde1:step3b} \tag{7}
\end{equation}
$$

We have turned the PDE into algebraic equations, also often called
discrete equations. The key property of the equations is that they
are algebraic, which makes them easy to solve.
As usual, we anticipate that $u^n_i$ is already computed such that
$u^{n+1}_i$ is the only unknown in ([7](#diffu:pde1:step3b)).
Solving with respect to this unknown is easy:

$$
\begin{equation}
u^{n+1}_i = u^n_i + F\left(
u^{n}_{i+1} - 2u^n_i + u^n_{i-1}\right) + \Delta t f_i^n,
\label{diffu:pde1:step4} \tag{8}
\end{equation}
$$

where we have introduced the *mesh Fourier number*:

$$
\begin{equation}
F = \dfc\frac{\Delta t}{\Delta x^2}\thinspace .
\label{_auto1} \tag{9}
\end{equation}
$$

**$F$ is the key parameter in the discrete diffusion equation.**

Note that $F$ is a *dimensionless* number that lumps the key physical
parameter in the problem, $\dfc$, and the discretization parameters
$\Delta x$ and $\Delta t$ into a single parameter. Properties
of the numerical method are critically dependent upon the value of
$F$ (see the section [diffu:pde1:analysis](#diffu:pde1:analysis) for details).

The computational algorithm then becomes

1. compute $u^0_i=I(x_i)$ for $i=0,\ldots,N_x$

2. for $n=0,1,\ldots,N_t$:

a. apply ([8](#diffu:pde1:step4)) for all the internal
      spatial points $i=1,\ldots,N_x-1$

b. set the boundary values
      $u^{n+1}_i=0$ for $i=0$ and $i=N_x$

The algorithm is compactly and fully specified in Python:

```python
import numpy as np

I = lambda x: 1
Nx = 100000
a = 2.0
L = 2.0
dx = L/Nx
dt = dx**2/(2*a)
T = 100*dt
Nt = int(round(T/float(dt)))

x = np.linspace(0, L, Nx+1)    # mesh points in space
dx = x[1] - x[0]
t = np.linspace(0, T, Nt+1)    # mesh points in time
dt = t[1] - t[0]
F = a*dt/dx**2
u   = np.zeros(Nx+1)           # unknown u at new time level
u_n = np.zeros(Nx+1)           # u at the previous time level

# Set initial condition u(x,0) = I(x)
for i in range(0, Nx+1):
    u_n[i] = I(x[i])

for n in range(0, Nt):
    # Compute u at inner mesh points
    for i in range(1, Nx):
        u[i] = u_n[i] + F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) + \
	       dt*f(x[i], t[n])

    # Insert boundary conditions
    u[0] = 0;  u[Nx] = 0

    # Update u_n before next step
    u_n[:]= u
```

Note that we use `a` for $\dfc$ in the code, motivated by easy visual
mapping between the variable name and the mathematical symbol in formulas.

We need to state already now that the shown algorithm does not
produce meaningful results unless $F\leq 1/2$. Why is explained in
the section [diffu:pde1:analysis](#diffu:pde1:analysis).

## Implementation

The file [`diffu1D_u0.py`](${src_diffu}/diffu1D_u0.py)
contains a complete function `solver_FE_simple`
for solving the 1D diffusion equation with $u=0$ on the boundary
as specified in the algorithm above:

```python
import numpy as np

def solver_FE_simple(I, a, f, L, dt, F, T):
    """
    Simplest expression of the computational algorithm
    using the Forward Euler method and explicit Python loops.
    For this method F <= 0.5 for stability.
    """
    import time;  t0 = time.clock()  # For measuring the CPU time

    Nt = int(round(T/float(dt)))
    t = np.linspace(0, Nt*dt, Nt+1)   # Mesh points in time
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)       # Mesh points in space
    # Make sure dx and dt are compatible with x and t
    dx = x[1] - x[0]
    dt = t[1] - t[0]

    u   = np.zeros(Nx+1)
    u_n = np.zeros(Nx+1)

    # Set initial condition u(x,0) = I(x)
    for i in range(0, Nx+1):
        u_n[i] = I(x[i])

    for n in range(0, Nt):
        # Compute u at inner mesh points
        for i in range(1, Nx):
            u[i] = u_n[i] + F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) + \
                   dt*f(x[i], t[n])

        # Insert boundary conditions
        u[0] = 0;  u[Nx] = 0

        # Switch variables before next step
        #u_n[:] = u  # safe, but slow
        u_n, u = u, u_n

    t1 = time.clock()
    return u_n, x, t, t1-t0  # u_n holds latest u
```

A faster alternative is available in the function `solver_FE`, which
adds the possibility of solving the finite difference scheme by vectorization.
The vectorized version replaces the explicit loop

```python
for i in range(1, Nx):
    u[i] = u_n[i] + F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) \
           + dt*f(x[i], t[n])
```

by arithmetics on displaced slices of the `u` array:

```python
u[1:Nx] = u_n[1:Nx] + F*(u_n[0:Nx-1] - 2*u_n[1:Nx] + u_n[2:Nx+1]) \
          + dt*f(x[1:Nx], t[n])
# or
u[1:-1] = u_n[1:-1] + F*(u_n[0:-2] - 2*u_n[1:-1] + u_n[2:]) \
          + dt*f(x[1:-1], t[n])
```

For example,
the vectorized version runs 70 times faster than the scalar version
in a case with 100 time steps and a spatial mesh of $10^5$ cells.

The `solver_FE` function also features a callback function such that the
user can process the solution at each time level. The callback
function looks like `user_action(u, x, t, n)`, where `u` is the array
containing the solution at time level `n`, `x` holds all the
spatial mesh points, while `t` holds all the temporal mesh points.
The `solver_FE` function is very similar to `solver_FE_simple` above:

```python
def solver_FE(I, a, f, L, dt, F, T,
              user_action=None, version='scalar'):
    """
    Vectorized implementation of solver_FE_simple.
    """
    import time;  t0 = time.clock()  # for measuring the CPU time

    Nt = int(round(T/float(dt)))
    t = np.linspace(0, Nt*dt, Nt+1)   # Mesh points in time
    dx = np.sqrt(a*dt/F)
    Nx = int(round(L/dx))
    x = np.linspace(0, L, Nx+1)       # Mesh points in space
    # Make sure dx and dt are compatible with x and t
    dx = x[1] - x[0]
    dt = t[1] - t[0]

    u   = np.zeros(Nx+1)   # solution array
    u_n = np.zeros(Nx+1)   # solution at t-dt

    # Set initial condition
    for i in range(0,Nx+1):
        u_n[i] = I(x[i])

    if user_action is not None:
        user_action(u_n, x, t, 0)

    for n in range(0, Nt):
        # Update all inner points
        if version == 'scalar':
            for i in range(1, Nx):
                u[i] = u_n[i] +\
                       F*(u_n[i-1] - 2*u_n[i] + u_n[i+1]) +\
                       dt*f(x[i], t[n])

        elif version == 'vectorized':
            u[1:Nx] = u_n[1:Nx] +  \
                      F*(u_n[0:Nx-1] - 2*u_n[1:Nx] + u_n[2:Nx+1]) +\
                      dt*f(x[1:Nx], t[n])
        else:
            raise ValueError('version=%s' % version)

        # Insert boundary conditions
        u[0] = 0;  u[Nx] = 0
        if user_action is not None:
            user_action(u, x, t, n+1)

        # Switch variables before next step
        u_n, u = u, u_n

    t1 = time.clock()
    return t1-t0
```

## Verification

### Exact solution of discrete equations

Before thinking about running the functions in the previous section,
we need to construct a suitable test example for verification. mathcal{I}_t
appears that a manufactured solution that is linear in time and at
most quadratic in space fulfills the Forward Euler scheme
exactly. With the restriction that $u=0$ for $x=0,L$, we can try the
solution

$$
u(x,t) = 5tx(L-x)\thinspace .
$$

Inserted in the PDE, it requires a source term

$$
f(x,t) = 10\dfc t + 5x(L-x)\thinspace .
$$

% if BOOK == 'book':
With the formulas from  [sec:form:fdtn](#sec:form:fdtn) we can easily check
% else:
Let us check
% endif
that the manufactured `u` fulfills the scheme:

$$
\begin{align*}
\lbrack D_t^+ u = \dfc D_x D_x u + f\rbrack^n_i &=
\lbrack 5x(L-x)D_t^+ t  = 5 t\dfc D_x D_x (xL-x^2) +\\
&\quad\quad 10\dfc t + 5x(L-x)\rbrack^n_i\\
&=
\lbrack 5x(L-x)  = 5 t\dfc (-2) + 10\dfc t + 5x(L-x) \rbrack^n_i,
\end{align*}
$$

which is a 0=0 expression.
The computation of the source term, given any $u$,
is easily automated with `sympy`:

```python
import sympy as sym
x, t, a, L = sym.symbols('x t a L')
u = x*(L-x)*5*t

def pde(u):
    return sym.diff(u, t) - a*sym.diff(u, x, x)

f = sym.simplify(pde(u))
f
```

$\displaystyle 10 a t + 5 x \left(L - x\right)$

Now we can choose any expression for `u` and automatically
get the suitable source term `f`. However, the manufactured solution
`u` will in general
not be exactly reproduced by the scheme: only constant and linear
functions are differentiated correctly by a forward difference, while only
constant, linear, and quadratic functions are differentiated exactly by
a $[D_xD_x u]^n_i$ difference.

The numerical code will need to access the `u` and `f` above
as Python functions. The exact solution is wanted as a Python
function `u_exact(x, t)`, while the source term is wanted as
`f(x, t)`. The parameters `a` and `L` in `u` and `f` above
are symbols and must be replaced by `float` objects in a Python
function. This can be done by redefining `a` and `L` as
`float` objects and performing substitutions of symbols by
numbers in `u` and `f`. The appropriate code looks like this:

```python
a = 0.5
L = 1.5
u_exact = sym.lambdify(
    [x, t], u.subs('L', L).subs('a', a), modules='numpy')
f = sym.lambdify(
    [x, t], f.subs('L', L).subs('a', a), modules='numpy')
I = lambda x: u_exact(x, 0)
```

Here we also make a function `I` for the initial condition.

The idea now is that our manufactured solution should be
exactly reproduced by the code (to machine precision).
For this purpose we make a test function for comparing
the exact and numerical solutions at the end of the
time interval:

```python
def test_solver_FE():
    # Define u_exact, f, I as explained above

    dx = L/3  # 3 cells
    F = 0.5
    dt = F*dx**2

    u, x, t, cpu = solver_FE_simple(
        I=I, a=a, f=f, L=L, dt=dt, F=F, T=2)
    u_e = u_exact(x, t[-1])
    diff = abs(u_e - u).max()
    tol = 1E-14
    assert diff < tol, 'max diff solver_FE_simple: %g' % diff

    u, x, t, cpu = solver_FE(
        I=I, a=a, f=f, L=L, dt=dt, F=F, T=2,
        user_action=None, version='scalar')
    u_e = u_exact(x, t[-1])
    diff = abs(u_e - u).max()
    tol = 1E-14
    assert diff < tol, 'max diff solver_FE, scalar: %g' % diff

    u, x, t, cpu = solver_FE(
        I=I, a=a, f=f, L=L, dt=dt, F=F, T=2,
        user_action=None, version='vectorized')
    u_e = u_exact(x, t[-1])
    diff = abs(u_e - u).max()
    tol = 1E-14
    assert diff < tol, 'max diff solver_FE, vectorized: %g' % diff
```

**The critical value $F=0.5$.**

We emphasize that the value `F=0.5` is critical: the tests above
will fail if `F` has a larger value. This is because the Forward
Euler scheme is unstable for $F>1/2$.

The reader may wonder if
$F=1/2$ is safe or if $F<1/2$ should be required. Experiments show
that $F=1/2$ works fine for $u_t=\dfc u_{xx}$, so
there is no accumulation of rounding
errors in this case and hence no need to introduce any safety factor
to keep $F$ away from the limiting value 0.5.

### Checking convergence rates

If our chosen exact solution does not satisfy the discrete equations
exactly, we are left with checking the convergence rates, just as we did
previously for the wave equation. However, with the Euler scheme here,
we have different accuracies in time and space, since we use a second
order approximation to the spatial derivative and a first order approximation
to the time derivative. Thus, we must expect different convergence rates in
time and space. For the numerical error,

$$
E = C_t\Delta t^r + C_x\Delta x^p,
$$

we should get convergence rates $r=1$ and $p=2$ ($C_t$ and $C_x$ are unknown constants).
As previously,
in the section [wave:pde2:fd:MMS](#wave:pde2:fd:MMS),
we simplify matters by introducing a single discretization parameter $h$:

$$
h = \Delta t,\quad \Delta x = Kh^{r/p},
$$

where $K$ is any constant. This allows us to factor out only *one*
discretization parameter $h$ from the formula:

$$
E = C_t h + C_x (Kh^{r/p})^p = \tilde C h^r,\quad
\tilde C = C_t + C_sK^r\thinspace .
$$

The computed rate $r$ should approach 1 with increasing resolution.

mathcal{I}_t is tempting, for simplicity,
to choose $K=1$, which gives $\Delta x = h^{r/p}$, expected to be
$\sqrt{\Delta t}$. However,
we have to control the stability requirement: $F\leq\frac{1}{2}$,
which means

$$
\frac{\dfc\Delta t}{\Delta x^2}\leq\frac{1}{2}\quad\Rightarrow
\quad \Delta x \geq \sqrt{2\dfc}h^{1/2} ,
$$

implying that $K=\sqrt{2\dfc}$ is our choice in experiments where we
lie on the stability limit $F=1/2$.

## Numerical experiments

When a test function like the one above runs silently without errors,
we have some evidence for a correct implementation of the numerical
method.  The next step is to do some experiments with more interesting
solutions.

We target a scaled diffusion problem where $x/L$ is a new spatial
coordinate and $\dfc t/L^2$ is a new time coordinate. The source term
$f$ is omitted, and $u$ is scaled by $\max_{x\in [0,L]}|I(x)|$ (see Section 3.2 in
 [[Langtangen_scaling]](#Langtangen_scaling) for details).
The governing PDE is then

$$
\frac{\partial u}{\partial t} = \frac{\partial^2 u}{\partial x^2},
$$

in the spatial domain $[0,L]$, with boundary conditions $u(0)=u(1)=0$.
Two initial conditions will be tested: a discontinuous plug,

$$
I(x) = \left\lbrace\begin{array}{ll}
0, & |x-L/2| > 0.1\\
1, & \hbox{otherwise}
\end{array}\right.
$$

and a smooth Gaussian function,

$$
I(x) = e^{-\frac{1}{2\sigma^2}(x-L/2)^2}\thinspace .
$$

The functions `plug` and `gaussian` in [`diffu1D_u0.py`](${src_diffu}/diffu1D_u0.py) run the two cases,
respectively:

```python
def plug(scheme='FE', F=0.5, Nx=50):
    L = 1.
    a = 1.
    T = 0.1
    # Compute dt from Nx and F
    dx = L/Nx;  dt = F/a*dx**2

    def I(x):
        """Plug profile as initial condition."""
        if abs(x-L/2.0) > 0.1:
            return 0
        else:
            return 1

    cpu = viz(I, a, L, dt, F, T,
              umin=-0.1, umax=1.1,
              scheme=scheme, animate=True, framefiles=True)
    print('CPU time:', cpu)

def gaussian(scheme='FE', F=0.5, Nx=50, sigma=0.05):
    L = 1.
    a = 1.
    T = 0.1
    # Compute dt from Nx and F
    dx = L/Nx;  dt = F/a*dx**2

    def I(x):
        """Gaussian profile as initial condition."""
        return exp(-0.5*((x-L/2.0)**2)/sigma**2)

    u, cpu = viz(I, a, L, dt, F, T,
                 umin=-0.1, umax=1.1,
                 scheme=scheme, animate=True, framefiles=True)
    print('CPU time:', cpu)
```

These functions make use of the function `viz` for running the
solver and visualizing the solution using a callback function
with plotting:

```python
def viz(I, a, L, dt, F, T, umin, umax,
        scheme='FE', animate=True, framefiles=True):

    def plot_u(u, x, t, n):
        plt.plot(x, u, 'r-', axis=[0, L, umin, umax],
                 title='t=%f' % t[n])
        if framefiles:
            plt.savefig('tmp_frame%04d.png' % n)
        if t[n] == 0:
            time.sleep(2)
        elif not framefiles:
            # mathcal{I}_t takes time to write files so pause is needed
            # for screen only animation
            time.sleep(0.2)

    user_action = plot_u if animate else lambda u,x,t,n: None

    cpu = eval('solver_'+scheme)(I, a, L, dt, F, T,
                                 user_action=user_action)
    return cpu
```

Notice that this `viz` function stores all the solutions in a
list `solutions` in the callback function. Modern computers have
hardly any problem with storing a lot of such solutions for moderate
values of $N_x$ in 1D problems, but for 2D and 3D problems, this
technique cannot be used and solutions must be stored in files.

[hpl 1: Better to show the scalable file solution here?]

Our experiments employ a time step $\Delta t = 0.0002$ and
simulate for $t\in [0,0.1]$. First we try the highest value of
$F$: $F=0.5$. This resolution corresponds to
$N_x=50$. A possible terminal command is

        Terminal> python -c 'from diffu1D_u0 import gaussian
                  gaussian("solver_FE", F=0.5, dt=0.0002)'

The $u(x,t)$ curve as a function of $x$ is shown in [Figure](#diffu:pde1:FE:fig:F=0.5) at four time levels.

```python
from IPython.display import HTML
_s = """
<div>

</div>
<p><em></em></p>

<!-- Issue warning if in a Safari browser -->

"""
HTML(_s)
```

We see that the curves have saw-tooth waves in the beginning of the
simulation. This non-physical noise is smoothed out with time, but
solutions of the diffusion equations are known to be smooth, and
this numerical solution is definitely not smooth.
Lowering $F$ helps: $F\leq 0.25$ gives a smooth solution, see
% if FORMAT == "pdflatex":
[Figure](#diffu:pde1:FE:fig:F=0.25) (and a
[movie](${docraw}/mov-diffu/diffu1D_u0_FE_plug_F025/movie.ogg)).
% else:
[Figure](#diffu:pde1:FE:fig:F=0.25).

```python
_s = """
<div>

</div>
<p><em></em></p>

<!-- Issue warning if in a Safari browser -->

"""
HTML(_s)
```

% endif

Increasing $F$ slightly beyond the limit 0.5, to $F=0.51$,
gives growing, non-physical instabilities,
as seen in [Figure](#diffu:pde1:FE:fig:F=0.51).

Forward Euler scheme for $F=0.5$.

Forward Euler scheme for $F=0.25$.

Forward Euler scheme for $F=0.51$.

Instead of a discontinuous initial condition we now try the smooth
Gaussian function for $I(x)$. A simulation for $F=0.5$
is shown in [Figure](#diffu:pde1:FE:fig:gauss:F=0.5). Now the numerical solution
is smooth for all times, and this is true for any $F\leq 0.5$.

% if FORMAT != "pdflatex":

```python
_s = """
<div>

</div>
<p><em></em></p>

<!-- Issue warning if in a Safari browser -->

"""
HTML(_s)
```

% endif

Forward Euler scheme for $F=0.5$.

Experiments with these two choices of $I(x)$ reveal some
important observations:

 * The Forward Euler scheme leads to growing solutions if $F>\frac{1}{2}$.

 * $I(x)$ as a discontinuous plug leads to a saw tooth-like noise
   for $F=\frac{1}{2}$, which is absent for $F\leq\frac{1}{4}$.

 * The smooth Gaussian initial function leads to a smooth solution
   for all relevant $F$ values ($F\leq \frac{1}{2}$).

# Implicit methods for the 1D diffusion equation

Simulations with the Forward Euler scheme show that the time step
restriction, $F\leq\frac{1}{2}$, which means $\Delta t \leq \Delta x^2/(2\dfc)$,
may be relevant in the beginning of the diffusion process, when the
solution changes quite fast, but as time increases, the process slows
down, and a small $\Delta t$ may be inconvenient. With
*implicit schemes*, which lead to coupled systems of linear equations
to be solved at each time level, any size of $\Delta t$ is possible
(but the accuracy decreases with increasing $\Delta t$).
The Backward Euler scheme, derived and implemented below, is the
simplest implicit scheme for the diffusion equation.

## Backward Euler scheme

In ([5](#diffu:pde1:step2)), we now apply a backward difference in time,
but the same central difference in space:

$$
\begin{equation}
[D_t^- u = D_xD_x u + f]^n_i,
\label{diffu:pde1:step3aBE} \tag{10}
\end{equation}
$$

which written out reads

$$
\begin{equation}
\frac{u^{n}_i-u^{n-1}_i}{\Delta t} = \dfc\frac{u^{n}_{i+1} - 2u^n_i + u^n_{i-1}}{\Delta x^2} + f_i^n\thinspace .
\label{diffu:pde1:step3bBE} \tag{11}
\end{equation}
$$

Now we assume $u^{n-1}_i$ is already computed, but that all quantities at the "new"
time level $n$ are unknown. This time it is not possible to solve
with respect to $u_i^{n}$ because this value couples to its neighbors
in space, $u^n_{i-1}$ and $u^n_{i+1}$, which are also unknown.
Let us examine this fact for the case when $N_x=3$. Equation ([11](#diffu:pde1:step3bBE)) written for $i=1,\ldots,Nx-1= 1,2$ becomes

$$
\begin{equation}
\frac{u^{n}_1-u^{n-1}_1}{\Delta t} = \dfc\frac{u^{n}_{2} - 2u^n_1 + u^n_{0}}{\Delta x^2} + f_1^n
\label{_auto2} \tag{12}
\end{equation}
$$

$$
\begin{equation}
\frac{u^{n}_2-u^{n-1}_2}{\Delta t} = \dfc\frac{u^{n}_{3} - 2u^n_2 + u^n_{1}}{\Delta x^2} + f_2^n
\label{_auto3} \tag{13}
\end{equation}
$$

The boundary values $u^n_0$ and $u^n_3$ are known as zero. Collecting the
unknown new values $u^n_1$ and $u^n_2$ on the left-hand side and multiplying
by $\Delta t$ gives

$$
\begin{equation}
\left(1+  2F\right) u^{n}_1 - F u^{n}_{2}    = u^{n-1}_1 + \Delta t f_1^n,
\label{_auto4} \tag{14}
\end{equation}
$$

$$
\begin{equation}
- F u^{n}_{1} + \left(1+  2F\right) u^{n}_2  = u^{n-1}_2 + \Delta t f_2^n\thinspace .
\label{_auto5} \tag{15}
\end{equation}
$$

This is a coupled $2\times 2$ system of algebraic equations for
the unknowns $u^n_1$ and $u^n_2$. The equivalent matrix form is

$$
\left(\begin{array}{cc}
1+  2F &   - F\\
- F    & 1+  2F
\end{array}\right)
\left(\begin{array}{c}
u^{n}_1\\
u^{n}_2
\end{array}\right)
=
\left(\begin{array}{c}
u^{n-1}_1 + \Delta t f_1^n\\
u^{n-1}_2 + \Delta t f_2^n
\end{array}\right)
$$

**Terminology: implicit vs. explicit methods.**

Discretization methods that lead to a coupled system of equations
for the unknown function at a new time level are said to be
*implicit methods*.
The counterpart, *explicit methods*, refers to discretization
methods where there is a simple explicit formula for the values of
the unknown function at each of the spatial mesh points at the new
time level. From an implementational point of view, implicit methods
are more comprehensive to code since they require
the solution of coupled equations, i.e., a matrix system, at each time level.
With explicit methods we have a closed-form formula for the value of
the unknown at each mesh point.

Very often explicit schemes have a restriction on the size of the time
step that can be relaxed by using implicit schemes. In fact,
implicit schemes are frequently unconditionally stable, so the size of the
time step is governed by accuracy and not by stability. This is the great
advantage of implicit schemes.

In the general case, ([11](#diffu:pde1:step3bBE)) gives rise to
a coupled $(N_x-1)\times (N_x-1)$ system of algebraic equations for
all the unknown $u^n_i$ at the interior spatial points $i=1,\ldots,N_x-1$.
Collecting the unknowns on the left-hand side,
([11](#diffu:pde1:step3bBE)) can be written

$$
\begin{equation}
- F u^n_{i-1} + \left(1+  2F \right) u^{n}_i - F u^n_{i+1} =
u_{i-1}^{n-1},
\label{diffu:pde1:step4BE} \tag{16}
\end{equation}
$$

for $i=1,\ldots,N_x-1$.
One can either view these equations as a system where the
$u^{n}_i$ values at the internal mesh points, $i=1,\ldots,N_x-1$, are
unknown, or we may append the boundary values $u^n_0$ and $u^n_{N_x}$
to the system. In the latter case, all $u^n_i$ for $i=0,\ldots,N_x$
are considered unknown, and we must add the boundary equations to
the $N_x-1$ equations in ([16](#diffu:pde1:step4BE)):

$$
\begin{equation}
u_0^n = 0,\label{diffu:pde1:step4BE:BC:0} \tag{17}
\end{equation}
$$

$$
\begin{equation}
u_{N_x}^n = 0\thinspace .
\label{diffu:pde1:step4BE:BC:L} \tag{18}
\end{equation}
$$

A coupled system of algebraic equations can be written on matrix form,
and this is important if we want to call up ready-made software for
solving the system.  The equations ([16](#diffu:pde1:step4BE))
and ([17](#diffu:pde1:step4BE:BC:0))--([18](#diffu:pde1:step4BE:BC:L))
correspond to the matrix equation

$$
AU = b
$$

where $U=(u^n_0,\ldots,u^n_{N_x})$, and
the matrix $A$ has the following structure:

$$
\begin{equation}
A =
\left(
\begin{array}{cccccccccc}
A_{0,0} & A_{0,1} & 0
&\cdots &
\cdots & \cdots & \cdots &
\cdots & 0 \\
A_{1,0} & A_{1,1} & A_{1,2} & \ddots &   & &  & &  \vdots \\
0 & A_{2,1} & A_{2,2} & A_{2,3} &
\ddots & &  &  & \vdots \\
\vdots & \ddots &  & \ddots & \ddots & 0 &  & & \vdots \\
\vdots &  & \ddots & \ddots & \ddots & \ddots & \ddots & & \vdots \\
\vdots & &  & 0 & A_{i,i-1} & A_{i,i} & A_{i,i+1} & \ddots & \vdots \\
\vdots & & &  & \ddots & \ddots & \ddots &\ddots  & 0 \\
\vdots & & & &  &\ddots  & \ddots &\ddots  & A_{N_x-1,N_x} \\
0 &\cdots & \cdots &\cdots & \cdots & \cdots  & 0 & A_{N_x,N_x-1} & A_{N_x,N_x}
\end{array}
\right)
\label{diffu:pde1:matrix:sparsity} \tag{19}
\end{equation}
$$

The nonzero elements are given by

$$
\begin{equation}
A_{i,i-1} = -F
\label{_auto6} \tag{20}
\end{equation}
$$

$$
\begin{equation}
A_{i,i} = 1+ 2F
\label{_auto7} \tag{21}
\end{equation}
$$

$$
\begin{equation}
A_{i,i+1} = -F
\label{_auto8} \tag{22}
\end{equation}
$$

in the equations for internal points, $i=1,\ldots,N_x-1$. The first and last
equation correspond to the boundary condition, where we know the solution,
and therefore we must have

$$
\begin{equation}
A_{0,0} = 1,
\label{_auto9} \tag{23}
\end{equation}
$$

$$
\begin{equation}
A_{0,1} = 0,
\label{_auto10} \tag{24}
\end{equation}
$$

$$
\begin{equation}
A_{N_x,N_x-1} = 0,
\label{_auto11} \tag{25}
\end{equation}
$$

$$
\begin{equation}
A_{N_x,N_x} = 1\thinspace .
\label{_auto12} \tag{26}
\end{equation}
$$

The right-hand side $b$ is written as

$$
\begin{equation}
b = \left(\begin{array}{c}
b_0\\
b_1\\
\vdots\\
b_i\\
\vdots\\
b_{N_x}
\end{array}\right)
\label{_auto13} \tag{27}
\end{equation}
$$

with

$$
\begin{equation}
b_0 = 0,
\label{_auto14} \tag{28}
\end{equation}
$$

$$
\begin{equation}
b_i = u^{n-1}_i,\quad i=1,\ldots,N_x-1,
\label{_auto15} \tag{29}
\end{equation}
$$

$$
\begin{equation}
b_{N_x} = 0 \thinspace .   \label{_auto16} \tag{30}
\end{equation}
$$

We observe that the matrix $A$ contains quantities that do not change
in time. Therefore, $A$ can be formed once and for all before we enter
the recursive formulas for the time evolution.
The right-hand side $b$, however, must be updated at each time step.
This leads to the following computational algorithm, here sketched
with Python code:

```python
x = np.linspace(0, L, Nx+1)   # mesh points in space
dx = x[1] - x[0]
t = np.linspace(0, T, N+1)    # mesh points in time
u   = np.zeros(Nx+1)          # unknown u at new time level
u_n = np.zeros(Nx+1)          # u at the previous time level

# Data structures for the linear system
A = np.zeros((Nx+1, Nx+1))
b = np.zeros(Nx+1)

for i in range(1, Nx):
    A[i,i-1] = -F
    A[i,i+1] = -F
    A[i,i] = 1 + 2*F
A[0,0] = A[Nx,Nx] = 1

# Set initial condition u(x,0) = I(x)
for i in range(0, Nx+1):
    u_n[i] = I(x[i])

import scipy.linalg

for n in range(0, Nt):
    # Compute b and solve linear system
    for i in range(1, Nx):
        b[i] = -u_n[i]
    b[0] = b[Nx] = 0
    u[:] = scipy.linalg.solve(A, b)

    # Update u_n before next step
    u_n[:] = u
```

Regarding verification, the same considerations apply as for the
Forward Euler method (the section [Verification](#diffu:pde1:FE:verify)).

## Sparse matrix implementation

We have seen from ([19](#diffu:pde1:matrix:sparsity)) that the matrix
$A$ is tridiagonal. The code segment above used a full, dense matrix
representation of $A$, which stores a lot of values we know are zero
beforehand, and worse, the solution algorithm computes with all these
zeros.  With $N_x+1$ unknowns, the work by the solution algorithm is
$\frac{1}{3} (N_x+1)^3$ and the storage requirements $(N_x+1)^2$. By
utilizing the fact that $A$ is tridiagonal and employing corresponding
software tools that work with the three diagonals, the work and
storage demands can be proportional to $N_x$ only.  This leads to a
dramatic improvement: with $N_x=200$, which is a realistic resolution,
the code runs about 40,000 times faster and reduces the storage to
just 1.5%! mathcal{I}_t is no doubt that we should take advantage of the fact
that $A$ is tridiagonal.

The key idea is to apply a data structure for a tridiagonal or sparse
matrix. The `scipy.sparse` package has relevant utilities. For
example, we can store only the nonzero diagonals of a matrix. The
package also has linear system solvers that operate on sparse matrix
data structures. The code below illustrates how we can store only the
main diagonal and the upper and lower diagonals.

```python
# Representation of sparse matrix and right-hand side
main  = np.zeros(Nx+1)
lower = np.zeros(Nx)
upper = np.zeros(Nx)
b     = np.zeros(Nx+1)

# Precompute sparse matrix
main[:] = 1 + 2*F
lower[:] = -F
upper[:] = -F
# Insert boundary conditions
main[0] = 1
main[Nx] = 1

A = scipy.sparse.diags(
    diagonals=[main, lower, upper],
    offsets=[0, -1, 1], shape=(Nx+1, Nx+1),
    format='csr')
print A.todense()  # Check that A is correct

# Set initial condition
for i in range(0,Nx+1):
    u_n[i] = I(x[i])

for n in range(0, Nt):
    b = u_n
    b[0] = b[-1] = 0.0  # boundary conditions
    u[:] = scipy.sparse.linalg.spsolve(A, b)
    u_n[:] = u
```

The `scipy.sparse.linalg.spsolve` function utilizes the sparse storage
structure of `A` and performs, in this case, a very efficient Gaussian
elimination solve.

The program [`diffu1D_u0.py`](${src_diffu}/diffu1D_u0.py)
contains a function `solver_BE`, which implements the Backward Euler scheme
sketched above.
As mentioned in the section [Forward Euler scheme](#diffu:pde1:FE),
the functions `plug` and `gaussian`
run the case with $I(x)$ as a discontinuous plug or a smooth
Gaussian function. All experiments point to two characteristic
features of the Backward Euler scheme: 1) it is always stable, and
2) it always gives a smooth, decaying solution.

## Crank-Nicolson scheme

The idea in the Crank-Nicolson scheme is to apply centered
differences in space and time, combined with an average in time.
We demand the PDE to be fulfilled at the spatial mesh points, but
midway between the points in the time mesh:

$$
\frac{\partial}{\partial t} u(x_i, t_{n+\frac{1}{2}}) =
\dfc\frac{\partial^2}{\partial x^2}u(x_i, t_{n+\frac{1}{2}}) + f(x_i,t_{n+\frac{1}{2}}),
$$

for $i=1,\ldots,N_x-1$ and $n=0,\ldots, N_t-1$.

With centered differences in space and time, we get

$$
[D_t u = \dfc D_xD_x u + f]^{n+\frac{1}{2}}_i\thinspace .
$$

On the right-hand side we get an expression

$$
\frac{1}{\Delta x^2}\left(u^{n+\frac{1}{2}}_{i-1} - 2u^{n+\frac{1}{2}}_i + u^{n+\frac{1}{2}}_{i+1}\right) + f_i^{n+\frac{1}{2}}\thinspace .
$$

This expression is problematic since $u^{n+\frac{1}{2}}_i$ is not one of
the unknowns we compute. A possibility is to replace $u^{n+\frac{1}{2}}_i$
by an arithmetic average:

$$
u^{n+\frac{1}{2}}_i\approx
\frac{1}{2}\left(u^{n}_i +u^{n+1}_{i}\right)\thinspace .
$$

In the compact notation, we can use the arithmetic average
notation $\overline{u}^t$:

$$
[D_t u = \dfc D_xD_x \overline{u}^t + f]^{n+\frac{1}{2}}_i\thinspace .
$$

We can also use an average for $f_i^{n+\frac{1}{2}}$:

$$
[D_t u = \dfc D_xD_x \overline{u}^t + \overline{f}^t]^{n+\frac{1}{2}}_i\thinspace .
$$

After writing out the differences and average, multiplying by $\Delta t$,
and collecting all unknown terms on the left-hand side, we get

$$
u^{n+1}_i - \frac{1}{2} F(u^{n+1}_{i-1} - 2u^{n+1}_i + u^{n+1}_{i+1})
= u^{n}_i + \frac{1}{2} F(u^{n}_{i-1} - 2u^{n}_i + u^{n}_{i+1})\nonumber
$$

$$
\begin{equation}
\qquad + \frac{1}{2} f_i^{n+1} + \frac{1}{2} f_i^n\thinspace .
\label{_auto17} \tag{31}
\end{equation}
$$

Also here, as in the Backward Euler scheme, the new unknowns
$u^{n+1}_{i-1}$, $u^{n+1}_{i}$, and $u^{n+1}_{i+1}$ are coupled
in a linear system $AU=b$, where $A$ has the same structure
as in ([19](#diffu:pde1:matrix:sparsity)), but with slightly
different entries:

$$
\begin{equation}
A_{i,i-1} = -\frac{1}{2} F
\label{_auto18} \tag{32}
\end{equation}
$$

$$
\begin{equation}
A_{i,i} = 1 + F
\label{_auto19} \tag{33}
\end{equation}
$$

$$
\begin{equation}
A_{i,i+1} = -\frac{1}{2} F
\label{_auto20} \tag{34}
\end{equation}
$$

in the equations for internal points, $i=1,\ldots,N_x-1$. The equations
for the boundary points correspond to

$$
\begin{equation}
A_{0,0} = 1,
\label{_auto21} \tag{35}
\end{equation}
$$

$$
\begin{equation}
A_{0,1} = 0,
\label{_auto22} \tag{36}
\end{equation}
$$

$$
\begin{equation}
A_{N_x,N_x-1} = 0,
\label{_auto23} \tag{37}
\end{equation}
$$

$$
\begin{equation}
A_{N_x,N_x} = 1\thinspace .
\label{_auto24} \tag{38}
\end{equation}
$$

The right-hand side $b$ has entries

$$
\begin{equation}
b_0 = 0,
\label{_auto25} \tag{39}
\end{equation}
$$

$$
\begin{equation}
b_i = u^{n-1}_i + \frac{1}{2}(f_i^n + f_i^{n+1}),\quad i=1,\ldots,N_x-1,
\label{_auto26} \tag{40}
\end{equation}
$$

$$
\begin{equation}
b_{N_x} = 0 \thinspace .   \label{_auto27} \tag{41}
\end{equation}
$$

When verifying some implementation of the Crank-Nicolson scheme by convergence rate testing,
one should note that the scheme is second order accurate in both space and time. The numerical
error then reads

$$
E = C_t\Delta t^r + C_x\Delta x^r,
$$

where $r=2$ ($C_t$ and $C_x$ are unknown constants, as before).
When introducing a single discretization parameter, we may now simply choose

$$
h = \Delta x = \Delta t,
$$

which gives

$$
E = C_th^r + C_xh^r = (C_t + C_x)h^r,
$$

where $r$ should approach 2 as resolution is increased in the convergence rate computations.

## The unifying $\theta$ rule

For the equation

$$
\frac{\partial u}{\partial t} = G(u),
$$

where $G(u)$ is some
spatial differential operator, the $\theta$-rule
looks like

$$
\frac{u^{n+1}_i - u^n_i}{\Delta t} =
\theta G(u^{n+1}_i) + (1-\theta) G(u^{n}_i)\thinspace .
$$

The important feature of this time discretization scheme is that
we can implement one formula and then generate a family of
well-known and widely used schemes:

 * $\theta=0$ gives the Forward Euler scheme in time

 * $\theta=1$ gives the Backward Euler scheme in time

 * $\theta=\frac{1}{2}$ gives the Crank-Nicolson scheme in time

In the compact difference notation, we write the $\theta$ rule
as

$$
[\overline{D}_t u = \dfc D_xD_x u]^{n+\theta}\thinspace .
$$

We have that $t_{n+\theta} = \theta t_{n+1} + (1-\theta)t_n$.

Applied to the 1D diffusion problem, the $\theta$-rule gives

$$
\begin{align*}
\frac{u^{n+1}_i-u^n_i}{\Delta t} &=
\dfc\left( \theta \frac{u^{n+1}_{i+1} - 2u^{n+1}_i + u^{n+1}_{i-1}}{\Delta x^2}
+ (1-\theta) \frac{u^{n}_{i+1} - 2u^n_i + u^n_{i-1}}{\Delta x^2}\right)\\
&\qquad + \theta f_i^{n+1} + (1-\theta)f_i^n
\thinspace .
\end{align*}
$$

This scheme also leads to a matrix system with entries

$$
A_{i,i-1} = -F\theta,\quad A_{i,i} = 1+2F\theta\quad,
A_{i,i+1} = -F\theta,
$$

while right-hand side entry $b_i$ is

$$
b_i = u^n_{i} + F(1-\theta)
\frac{u^{n}_{i+1} - 2u^n_i + u^n_{i-1}}{\Delta x^2} +
\Delta t\theta f_i^{n+1} + \Delta t(1-\theta)f_i^n\thinspace .
$$

The corresponding entries for the boundary points are as in the Backward
Euler and Crank-Nicolson schemes listed earlier.

Note that convergence rate testing with implementations of the theta rule must
adjust the error expression according to which of the underlying schemes is actually being run.
That is, if $\theta=0$ (i.e., Forward Euler) or $\theta=1$ (i.e., Backward Euler), there should
be first order convergence, whereas with $\theta=0.5$ (i.e., Crank-Nicolson), one should get
second order convergence (as outlined in previous sections).

## Experiments

We can repeat the experiments from the section [Numerical experiments](#diffu:pde1:FE:experiments)
to see if the Backward Euler or Crank-Nicolson schemes have problems
with sawtooth-like noise when starting with a discontinuous initial
condition. We can also verify that we can have $F>\frac{1}{2}$,
which allows larger time steps than in the Forward Euler method.

Backward Euler scheme for $F=0.5$.

The Backward Euler scheme always produces smooth solutions for any $F$.
[Figure](#diffu:pde1:BE:fig:F=0.5) shows one example.
Note that the mathematical discontinuity at $t=0$ leads to a linear
variation on a mesh, but the approximation to a jump becomes better
as $N_x$ increases. In our simulation, we specify $\Delta t$ and $F$,
and set $N_x$ to $L/\sqrt{\dfc\Delta t/F}$. Since $N_x\sim\sqrt{F}$,
the discontinuity looks sharper in the Crank-Nicolson
simulations with larger $F$.

The Crank-Nicolson method produces smooth solutions for small $F$,
$F\leq\frac{1}{2}$, but small noise gets more and more evident as $F$
increases. Figures [diffu:pde1:CN:fig:F=3](#diffu:pde1:CN:fig:F=3) and [diffu:pde1:CN:fig:F=10](#diffu:pde1:CN:fig:F=10)
demonstrate the effect for $F=3$ and $F=10$, respectively.
The section [diffu:pde1:analysis](#diffu:pde1:analysis) explains why such noise occur.

Crank-Nicolson scheme for $F=3$.

Crank-Nicolson scheme for $F=10$.

## The Laplace and Poisson equation

The Laplace equation, $\nabla^2 u = 0$, and the Poisson equation,
$-\nabla^2 u = f$, occur in numerous applications throughout science and
engineering. In 1D these equations read
$u''(x)=0$ and $-u''(x)=f(x)$, respectively.
We can solve 1D variants of the Laplace equations with the listed
software, because we can interpret $u_{xx}=0$ as the limiting solution
of $u_t = \dfc u_{xx}$ when $u$ reaches a steady state limit where
$u_t\rightarrow 0$.
Similarly, Poisson's equation $-u_{xx}=f$ arises from solving
$u_t = u_{xx} + f$ and letting $t\rightarrow\infty$ so $u_t\rightarrow 0$.

Technically in a program, we can simulate $t\rightarrow\infty$
by just taking one large time step:
$\Delta t\rightarrow\infty$. In the limit, the Backward Euler
scheme gives

$$
-\frac{u^{n+1}_{i+1} - 2u^{n+1}_i + u^{n+1}_{i-1}}{\Delta x^2} = f^{n+1}_i,
$$

which is nothing but the discretization $[-D_xD_x u = f]^{n+1}_i=0$ of
$-u_{xx}=f$.

The result above means that
the Backward Euler scheme can solve the limit equation directly and
hence produce a solution of the 1D Laplace equation.
With the Forward Euler scheme we must do the time stepping since $\Delta t >
\Delta x^2/\dfc$
is illegal and leads to instability.
We may interpret this time stepping
as solving the equation system from $-u_{xx}=f$ by iterating on a
pseudo time variable.

[hpl 2: Better to say the last sentence when we treat iterative methods.]


# Lowpass filter instead of Kalman filter for the PIT

```python
# %load imports.py
## Local packages:

%matplotlib inline
%load_ext autoreload
%autoreload 2
%config Completer.use_jedi = False  ## (To fix autocomplete)

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt
#if os.name == 'nt':
#    plt.style.use('presentation.mplstyle')  # Windows

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

from src.symbols import *
from src.parameters import *
import src.symbols as symbols
from src import prime_system
from src.models import regression
from src.visualization.plot import track_plot
from src.equation import Equation

## Load models:
# (Uncomment these for faster loading):
#import src.models.vmm_abkowitz  as vmm
import src.models.vmm_martin  as vmm
```

```python
from src.data import kalman_filter
from src.data.lowpass_filter import lowpass_filter
import scipy.stats as stats
```

```python
id=22774
df, units, meta_data = mdl.load(id=id)
df_raw = df.copy()
df.index = df.index.total_seconds()
df.index-=df.index[0]
```

```python
def df_filter(df_gradient, cutoff=1, order=5):

    ts = np.mean(np.diff(df_gradient.index))
    fs = 1/ts

    df_lowpass = pd.DataFrame(columns=df_gradient.columns, index=df_gradient.index)
    for key,value in df_gradient.items():
        df_lowpass[key] = lowpass_filter(data=value, cutoff=cutoff, fs=fs, order=order)

    return df_lowpass

def lowpass(df_,cutoff=1, order=5):

    df_gradient = df_.copy()
    df_gradient.index=df.index

    df_gradient[['x0_filtered','y0_filtered','psi_filtered']] = df_filter(df_gradient[['x0','y0','psi']], cutoff=cutoff, order=order)
    df_gradient['x01d'] = np.gradient(df_gradient['x0_filtered'], df_gradient.index)
    df_gradient['y01d'] = np.gradient(df_gradient['y0_filtered'], df_gradient.index)
    df_gradient['z01d_gradient'] = np.gradient(df_gradient['z0'], df_gradient.index)

    df_gradient['r'] = df_gradient['psi1d'] = np.gradient(df_gradient['psi_filtered'], df_gradient.index)

    df_gradient[['x01d','y01d','r']] = df_filter(df_gradient[['x01d','y01d','r']], cutoff=cutoff, order=order)
    df_gradient['x02d'] = np.gradient(df_gradient['x01d'], df_gradient.index)
    df_gradient['y02d'] = np.gradient(df_gradient['y01d'], df_gradient.index)
    df_gradient['z02d_gradient'] = np.gradient(df_gradient['z01d_gradient'], df_gradient.index)
    df_gradient['r1d'] = df_gradient['psi2d'] = np.gradient(df_gradient['r'], df_gradient.index)

    df_gradient[['x02d','y02d','r1d']] = df_filter(df_gradient[['x02d','y02d','r1d']], cutoff=cutoff, order=order)

    return df_gradient
```

```python
df_lowpass = lowpass(df_=df, cutoff=1, order=1)

df_resample = df_raw.resample('0.5S').mean()
t_ = df_resample.index.total_seconds()
df_resample['x01d'] = np.gradient(df_resample['x0'], t_)
df_resample['y01d'] = np.gradient(df_resample['y0'], t_)
df_resample['r'] = np.gradient(df_resample['psi'], t_)
df_resample['x02d'] = np.gradient(df_resample['x01d'], t_)
df_resample['y02d'] = np.gradient(df_resample['y01d'], t_)
df_resample['r1d'] = np.gradient(df_resample['r'], t_)
df_resample.index=df_resample.index.total_seconds()
df_resample.index-=df_resample.index[0]

df_rolling = df.sort_index(ascending=False).rolling(100).mean().sort_index(ascending=True)
t_=df_rolling.index
df_rolling['x01d'] = np.gradient(df_rolling['x0'], t_)
df_rolling['y01d'] = np.gradient(df_rolling['y0'], t_)
df_rolling['r'] = np.gradient(df_rolling['psi'], t_)
df_rolling['x02d'] = np.gradient(df_rolling['x01d'], t_)
df_rolling['y02d'] = np.gradient(df_rolling['y01d'], t_)
df_rolling['r1d'] = np.gradient(df_rolling['r'], t_)

states = [
     ['x0','x01d','x02d'],
     ['y0','y01d','y02d'],
     ['psi','r','r1d']
]

for dof in states:
    fig,axes=plt.subplots(ncols=3)
    fig.set_size_inches(15,4)
    for state,ax in zip(dof,axes):
        ax.set_title(state)
        #df_lowpass_gradient.plot(y=state, ax=ax, label='lowpass gradient')
        df_lowpass.plot(y=state, ax=ax, label='lowpass')
        df_resample.plot(y=state, ax=ax, label='resample', alpha=0.5)
        df_rolling.plot(y=state, ax=ax, label='rolling', alpha=0.5)
```

```python
df = kalman_filter.transform_to_ship(df_lowpass, include_unfiltered=False)

df['thrust'] = df['Prop/PS/Thrust'] + df['Prop/SB/Thrust']
df['thrust'] = df_filter(df[['thrust']], cutoff=0.5, order=1)

df['U'] = np.sqrt(df['u']**2 + df['v']**2)

```

```python
df_cut = df.iloc[200:-200].copy()
```

```python
track_plot(df=df_cut, lpp=meta_data.lpp, x_dataset='x0', y_dataset='y0',  psi_dataset='psi',
           beam=meta_data.beam);

df_cut.plot(y='delta')
df_cut.plot(y='x02d')
df_cut.plot(y='thrust')

```

```python
_, _, meta_data = mdl.load(id=id, dir_path='../data/processed/kalman_cut')
meta_data['rho']=1000
meta_data['mass'] = meta_data['Volume']*meta_data['rho']
```

# Ship parameters

```python
T_ = (meta_data.TA + meta_data.TF)/2
L_ = meta_data.lpp
m_ = meta_data.mass
rho_ = meta_data.rho
B_ = meta_data.beam
CB_ = m_/(T_*B_*L_*rho_)
I_z_ = m_*meta_data.KZZ**2
#I_z_ = 900

ship_parameters = {
        'T' : T_,
        'L' : L_,
        'CB' :CB_,
        'B' : B_,
        'rho' : rho_,
        'x_G' : 0,  # motions are expressed at CG
        'm' : m_,
        'I_z': I_z_,
        'volume':meta_data.Volume,
    }

ps = prime_system.PrimeSystem(**ship_parameters)  # model

scale_factor = meta_data.scale_factor
ps_ship = prime_system.PrimeSystem(L=ship_parameters['L']*scale_factor, rho=meta_data['rho'])  # ship
```

# Brix parameters

```python
def calculate_prime(row, ship_parameters):
    return run(function=row['brix_lambda'], inputs=ship_parameters)

mask = df_parameters['brix_lambda'].notnull()
df_parameters.loc[mask,'brix_prime'] = df_parameters.loc[mask].apply(calculate_prime, ship_parameters=ship_parameters, axis=1)

df_parameters['brix_prime'].fillna(0, inplace=True)
#df_parameters['brix_SI'].fillna(0, inplace=True)

```

```python
df_parameters['prime'] = df_parameters['brix_prime']

def to_SI(row):

    prime = row['prime']
    denominator = row['denominator']
    lamda = lambdify(denominator)
    try:
        denominator_value = run(lamda, inputs=ship_parameters)
        SI = prime*denominator_value
    except:
        return None
    else:
        return SI

df_parameters['brix_SI'] = df_parameters.apply(to_SI, axis=1)

```

# Regression

```python
from statsmodels.sandbox.regression.predstd import wls_prediction_std
def show_pred(X,y,results, label):

    display(results.summary())

    X_ = X
    y_ = y
    y_pred = results.predict(X_)

    prstd, iv_l, iv_u = wls_prediction_std(results, exog=X_, alpha=0.05)
    #iv_l*=-1
    #iv_u*=-1

    fig,ax=plt.subplots()
    ax.plot(X_.index,y_, label='Numerical gradient from model test')
    ax.plot(X_.index,y_pred, '--', label='OLS')
    ax.set_ylabel(label)

    ax.fill_between(X_.index, y1=iv_l, y2=iv_u, zorder=-10, color='grey', alpha=0.5, label=r'5\% confidence')
    ax.legend();
```

## N

```python
N_ = sp.symbols('N_')

diff_eq_N = regression.DiffEqToMatrix(ode=vmm.N_qs_eq.subs(N_qs,N_),
                                      label=N_, base_features=[delta,u,v,r])
```

```python
Math(vlatex(diff_eq_N.acceleration_equation))
```

```python
X = diff_eq_N.calculate_features(data=df_cut)
y = diff_eq_N.calculate_label(y=df_cut['r1d'])

model_N = sm.OLS(y,X)
results_N = model_N.fit()

show_pred(X=X,y=y,results=results_N, label=r'$\dot{r}$')
```

## Y

```python
Y_ = sp.symbols('Y_')
diff_eq_Y = regression.DiffEqToMatrix(ode=vmm.Y_qs_eq.subs(Y_qs,Y_),
                                      label=Y_, base_features=[delta,u,v,r])
```

```python
Math(vlatex(diff_eq_Y.acceleration_equation))
```

```python
X = diff_eq_Y.calculate_features(data=df_cut)
y = diff_eq_Y.calculate_label(y=df_cut['v1d'])

model_Y = sm.OLS(y,X)
results_Y = model_Y.fit()

show_pred(X=X,y=y,results=results_Y, label=r'$\dot{v}$')
```

## X

```python
X_ = sp.symbols('X_')
diff_eq_X = regression.DiffEqToMatrix(ode=vmm.X_qs_eq.subs(X_qs,X_),
                                      label=X_, base_features=[delta,u,v,r,thrust])
```

```python
diff_eq_X.acceleration_equation
```

```python
X = diff_eq_X.calculate_features(data=df_cut)
y = diff_eq_X.calculate_label(y=df_cut['u1d'])

model_X = sm.OLS(y,X)
results_X = model_X.fit()

show_pred(X=X,y=y,results=results_X, label=r'$\dot{u}}$')
```

```python
results_summary_X = regression.results_summary_to_dataframe(results_X)
results_summary_Y = regression.results_summary_to_dataframe(results_Y)
results_summary_N = regression.results_summary_to_dataframe(results_N)
```

```python
subs = {value:key for key,value in p.items()}
A_ = vmm.simulator.A*sp.matrices.MutableDenseMatrix([A_coeff,B_coeff,C_coeff])
A_lambda=lambdify(A_.subs(subs))
```

```python
A_coeff_ = results_summary_X['coeff']
B_coeff_ = results_summary_Y['coeff']
C_coeff_ = results_summary_N['coeff']

coeffs = run(A_lambda,A_coeff=A_coeff_.values, B_coeff=B_coeff_.values, C_coeff=C_coeff_.values,
    **df_parameters['brix_SI'], **ship_parameters)
```

```python
results_summary_X['decoupled'] = coeffs[0][0]
results_summary_Y['decoupled'] = coeffs[1][0]
results_summary_N['decoupled'] = coeffs[2][0]

```

## Add the regressed parameters
Hydrodynamic derivatives that depend on acceleration cannot be obtained from the VCT regression. They are however essential if a time simulation should be conducted. These values have then been taken from Brix semi empirical formulas for the simulations below.

```python
df_parameters_all = df_parameters.copy()
for other in [results_summary_X, results_summary_Y, results_summary_N]:
    df_parameters_all = df_parameters_all.combine_first(other)

df_parameters_all.rename(columns={'decoupled':'regressed'}, inplace=True)
df_parameters_all.drop(columns=['brix_lambda'], inplace=True)

df_parameters_all['SI'] = df_parameters_all['regressed'].combine_first(df_parameters_all['brix_SI'])  # prefer regressed
df_parameters_all['SI'].fillna(0,inplace=True)
```

# Simulate forces

```python
df_cut['X_qs_pred'] = run(function=vmm.simulator.X_qs_lambda, inputs=df_cut, **df_parameters_all['SI'])
df_cut['Y_qs_pred'] = run(function=vmm.simulator.Y_qs_lambda, inputs=df_cut, **df_parameters_all['SI'])
df_cut['N_qs_pred'] = run(function=vmm.simulator.N_qs_lambda, inputs=df_cut, **df_parameters_all['SI'])

```

```python
X = diff_eq_X.calculate_features(data=df_cut)
X_forces = X*results_summary_X['coeff']
X_forces.index = df_cut.index

X = diff_eq_Y.calculate_features(data=df_cut)
Y_forces = X*results_summary_Y['coeff']
Y_forces.index = df_cut.index

X = diff_eq_N.calculate_features(data=df_cut)
N_forces = X*results_summary_N['coeff']
N_forces.index = df_cut.index

```

```python
display(px.line(X_forces, y=X_forces.columns, width=1000, height=400))
display(px.line(Y_forces, y=Y_forces.columns, width=1000, height=400))
display(px.line(N_forces, y=N_forces.columns, width=1000, height=400))
```

```python
df_captive = pd.DataFrame()

V_ = 2.0

betas = np.deg2rad(np.linspace(0,30,5))
deltas = np.deg2rad(np.linspace(-35,35,5))

Betas, Deltas = np.meshgrid(betas,deltas)

df_captive['beta'] = Betas.flatten()
df_captive['delta'] = Deltas.flatten()

df_captive['u'] = V_*np.cos(df_captive['beta'])
df_captive['v'] = -V_*np.sin(df_captive['beta'])
df_captive['thrust'] = 0
df_captive['r'] = 0

df_captive['X_qs_pred'] = run(function=vmm.simulator.X_qs_lambda, inputs=df_captive, **df_parameters_all['SI'])
df_captive['Y_qs_pred'] = run(function=vmm.simulator.Y_qs_lambda, inputs=df_captive, **df_parameters_all['SI'])
df_captive['N_qs_pred'] = run(function=vmm.simulator.N_qs_lambda, inputs=df_captive, **df_parameters_all['SI'])

keys = ['X_qs','Y_qs','N_qs']

x_keys = ['beta','delta']
for x_key in x_keys:

    fig,axes=plt.subplots(ncols=len(keys))
    fig.set_size_inches(15,4)

    others = list(x_keys)
    others.remove(x_key)

    for group_name, group in df_captive.groupby(others):

        if not isinstance(group_name,list):
            group_name = [group_name]
        label =''.join([f'{other}:{np.round(name,2)}, ' for other,name in zip(others,group_name)])

        for key,ax in zip(keys,axes):

            group.plot(x=x_key, y=f'{key}_pred',style='.-', ax=ax, label=label)
            #ax.get_legend().set_visible(False)
            ax.set_title(key)
            ax.grid(True)
```

# Simulation

The way that the regression is formulated, inertial forces, such as centrifugal force will be included into the derivatives (I think) which means that centrifugal force : $-m \cdot r \cdot u$ will be included into $Y_{ur}$ coefficient. This coefficient is therefore not pure hydrodynamic, and can potentially be counted twice..?
The coefficients are recalculated below to avooid this:

```python
parameters=df_parameters_all['SI'].copy()
x_G_ = ship_parameters['x_G']
#parameters['Xrr']+=(-m_*x_G_)
#parameters['Xvr']+=(-m_)
#parameters['Yur']+=m_
#parameters['Nur']+=m_*x_G_

```

```python
result_vmm = vmm.simulator.simulate(df_=df_cut, parameters=parameters, ship_parameters=ship_parameters)
```

```python
result_vmm.plot_compare()
```

```python
vmm.simulator.A
```

```python
S = sp.symbols('S')
S_eq=sp.Eq(S,(m-p.Yvdot)*(I_z-p.Nrdot) - (m*x_G-p.Yrdot)*(m*x_G-p.Nudot-p.Nvdot))
S_eq
```

```python
vmm.simulator.b
```

```python
A_inv = vmm.simulator.A.inv()
```

```python
inertia = sp.MutableDenseMatrix([-m*(-x_G*r**2-r*v), -m*r*u, -m*x_G*r*u])
accelerations = sp.MutableDenseMatrix([u1d, v1d, r1d])
sp.simplify(vmm.simulator.b - inertia)
```

```python
Math(vlatex(vmm.simulator.A*accelerations-inertia))
```

```python
Math(vlatex(eq_X_qs))
```

```python
Math(vlatex(vmm.simulator.A*accelerations-inertia))
```

```python
A_inv*(vmm.simulator.b-inertia) + A_inv*inertia
```

```python
A_inv*inertia
```

```python
solution = sp.simplify(A_inv*vmm.simulator.b)
solution
```

```python
solution[0]
```

```python
sp.fraction(solution[0])[0]
```

```python

```