# Bayesian Hierarchical Linear Regression
Author: [Carlos Souza](mailto:souza@gatech.edu)

Probabilistic Machine Learning models can not only make predictions about future data, but also **model uncertainty**. In areas such as **personalized medicine**, there might be a large amount of data, but there is still a relatively **small amount of data for each patient**. To customize predictions for each person it becomes necessary to **build a model for each person** — with its inherent **uncertainties** — and to couple these models together in a **hierarchy** so that information can be borrowed from other **similar people** [1].

The purpose of this tutorial is to demonstrate how to **implement a Bayesian Hierarchical Linear Regression model using NumPyro**. To motivate the tutorial, I will use [OSIC Pulmonary Fibrosis Progression](https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression) competition, hosted at Kaggle.

## 1. Understanding the task
Pulmonary fibrosis is a disorder with no known cause and no known cure, created by scarring of the lungs. In this competition, we were asked to predict a patient’s severity of decline in lung function. Lung function is assessed based on output from a spirometer, which measures the forced vital capacity (FVC), i.e. the volume of air exhaled.

In medical applications, it is useful to **evaluate a model's confidence in its decisions**. Accordingly, the metric used to rank the teams was designed to reflect **both the accuracy and certainty of each prediction**. It's a modified version of the Laplace Log Likelihood (more details on that later).

Let's explore the data and see what's that all about:

```python
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
```

```python
train = pd.read_csv('https://gist.githubusercontent.com/ucals/'
                    '2cf9d101992cb1b78c2cdd6e3bac6a4b/raw/'
                    '43034c39052dcf97d4b894d2ec1bc3f90f3623d9/'
                    'osic_pulmonary_fibrosis.csv')
train.head()
```

      Patient
      Weeks
      FVC
      Percent
      Age
      Sex
      SmokingStatus

      0
      ID00007637202177411956430
      -4
      2315
      58.253649
      79
      Male
      Ex-smoker

      1
      ID00007637202177411956430
      5
      2214
      55.712129
      79
      Male
      Ex-smoker

      2
      ID00007637202177411956430
      7
      2061
      51.862104
      79
      Male
      Ex-smoker

      3
      ID00007637202177411956430
      9
      2144
      53.950679
      79
      Male
      Ex-smoker

      4
      ID00007637202177411956430
      11
      2069
      52.063412
      79
      Male
      Ex-smoker

In the dataset, we were provided with a baseline chest CT scan and associated clinical information for a set of patients. A patient has an image acquired at time Week = 0 and has numerous follow up visits over the course of approximately 1-2 years, at which time their FVC is measured. For this tutorial, I will use only the Patient ID, the weeks and the FVC measurements, discarding all the rest. Using only these columns enabled our team to achieve a competitive score, which shows the power of Bayesian hierarchical linear regression models especially when gauging uncertainty is an important part of the problem.

Since this is real medical data, the relative timing of FVC measurements varies widely, as shown in the 3 sample patients below:

```python
def chart(patient_id, ax):
    data = train[train['Patient'] == patient_id]
    x = data['Weeks']
    y = data['FVC']
    ax.set_title(patient_id)
    ax = sns.regplot(x, y, ax=ax, ci=None, line_kws={'color':'red'})

f, axes = plt.subplots(1, 3, figsize=(15, 5))
chart('ID00007637202177411956430', axes[0])
chart('ID00009637202177434476278', axes[1])
chart('ID00010637202177584971671', axes[2])
```

On average, each of the 176 provided patients made 9 visits, when FVC was measured. The visits happened in specific weeks in the [-12, 133] interval. The decline in lung capacity is very clear. We see, though, they are very different from patient to patient.

We were are asked to predict every patient's FVC measurement for every possible week in the [-12, 133] interval, and the confidence for each prediction. In other words: we were asked fill a matrix like the one below, and provide a confidence score for each prediction:

The task was perfect to apply Bayesian inference. However, the vast majority of solutions shared by Kaggle community used discriminative machine learning models, disconsidering the fact that most discriminative methods are very poor at providing realistic uncertainty estimates. Because they are typically trained in a manner that optimizes the parameters to minimize some loss criterion (e.g. the predictive error), they do not, in general, encode any uncertainty in either their parameters or the subsequent predictions. Though many methods can produce uncertainty estimates either as a by-product or from a post-processing step, these are typically heuristic based, rather than stemming naturally from a statistically principled estimate of the target uncertainty distribution [2].

## 2. Modelling: Bayesian Hierarchical Linear Regression with Partial Pooling
The simplest possible linear regression, not hierarchical, would assume all FVC decline curves have the same $\alpha$ and $\beta$. That's the **pooled model**. In the other extreme, we could assume a model where each patient has a personalized FVC decline curve, and **these curves are completely unrelated**. That's the **unpooled model**, where each patient has completely separate regressions.

Here, I'll use the middle ground: **Partial pooling**. Specifically, I'll assume that while $\alpha$'s and $\beta$'s are different for each patient as in the unpooled case, **the coefficients all share similarity**. We can model this by assuming that each individual coefficient comes from a common group distribution. The image below represents this model graphically:

Mathematically, the model is described by the following equations:

\begin{align}
\mu_{\alpha} &\sim \mathcal{N}(0, 100) \\
\sigma_{\alpha} &\sim |\mathcal{N}(0, 100)| \\
\mu_{\beta} &\sim \mathcal{N}(0, 100) \\
\sigma_{\beta} &\sim |\mathcal{N}(0, 100)| \\
\alpha_i &\sim \mathcal{N}(\mu_{\alpha}, \sigma_{\alpha}) \\
\beta_i &\sim \mathcal{N}(\mu_{\beta}, \sigma_{\beta}) \\
\sigma &\sim \mathcal{N}(0, 100) \\
FVC_{ij} &\sim \mathcal{N}(\alpha_i + t \beta_i, \sigma)
\end{align}

where *t* is the time in weeks. Those are very uninformative priors, but that's ok: our model will converge!

Implementing this model in NumPyro is pretty straightforward:

```python
import numpyro
from numpyro.infer import MCMC, NUTS, Predictive
import numpyro.distributions as dist
from jax import random
```

```python
def model(PatientID, Weeks, FVC_obs=None):
    μ_α = numpyro.sample("μ_α", dist.Normal(0., 100.))
    σ_α = numpyro.sample("σ_α", dist.HalfNormal(100.))
    μ_β = numpyro.sample("μ_β", dist.Normal(0., 100.))
    σ_β = numpyro.sample("σ_β", dist.HalfNormal(100.))

    unique_patient_IDs = np.unique(PatientID)
    n_patients = len(unique_patient_IDs)

    with numpyro.plate("plate_i", n_patients):
        α = numpyro.sample("α", dist.Normal(μ_α, σ_α))
        β = numpyro.sample("β", dist.Normal(μ_β, σ_β))

    σ = numpyro.sample("σ", dist.HalfNormal(100.))
    FVC_est = α[PatientID] + β[PatientID] * Weeks

    with numpyro.plate("data", len(PatientID)):
        numpyro.sample("obs", dist.Normal(FVC_est, σ), obs=FVC_obs)
```

That's all for modelling!

## 3. Fitting the model
A great achievement of Probabilistic Programming Languages such as NumPyro is to decouple model specification and inference. After specifying my generative model, with priors, condition statements and data likelihood, I can leave the hard work to NumPyro's inference engine.

Calling it requires just a few lines. Before we do it, let's add a numerical Patient ID for each patient code. That can be easily done with scikit-learn's LabelEncoder:

```python
from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
train['PatientID'] = le.fit_transform(train['Patient'].values)

FVC_obs = train['FVC'].values
Weeks = train['Weeks'].values
PatientID = train['PatientID'].values
```

```python
numpyro.set_host_device_count(4)
```

Now, calling NumPyro's inference engine:

```python
nuts_kernel = NUTS(model)

mcmc = MCMC(nuts_kernel, num_samples=2000, num_warmup=2000)
rng_key = random.PRNGKey(0)
mcmc.run(rng_key, PatientID, Weeks, FVC_obs=FVC_obs)

posterior_samples = mcmc.get_samples()
```

    sample: 100%|██████████| 4000/4000 [00:27<00:00, 144.62it/s, 63 steps of size 1.09e-01. acc. prob=0.88]

## 4. Checking the model
### 4.1. Inspecting the learned parameters
First, let's inspect the parameters learned. To do that, I will use [ArviZ](https://arviz-devs.github.io/arviz/), which perfectly integrates with NumPyro:

```python
import arviz as az

data = az.from_numpyro(mcmc)
az.plot_trace(data, compact=True);
```

Looks like our model learned personalized alphas and betas for each patient!

### 4.2. Visualizing FVC decline curves for some patients
Now, let's visually inspect FVC decline curves predicted by our model. We will completely fill in the FVC table, predicting all missing values. The first step is to create a table to fill:

```python
pred_template = []
for i in range(train['Patient'].nunique()):
    df = pd.DataFrame(columns=['PatientID', 'Weeks'])
    df['Weeks'] = np.arange(-12, 134)
    df['PatientID'] = i
    pred_template.append(df)
pred_template = pd.concat(pred_template, ignore_index=True)
```

Predicting the missing values in the FVC table and confidence (sigma) for each value becomes really easy:

```python
PatientID = pred_template['PatientID'].values
Weeks = pred_template['Weeks'].values
predictive = Predictive(model, posterior_samples,
                        return_sites=['σ', 'obs'])
samples_predictive = predictive(random.PRNGKey(0),
                                PatientID, Weeks, None)
```

Let's now put the predictions together with the true values, to visualize them:

```python
df = pd.DataFrame(columns=['Patient', 'Weeks', 'FVC_pred', 'sigma'])
df['Patient'] = le.inverse_transform(pred_template['PatientID'])
df['Weeks'] = pred_template['Weeks']
df['FVC_pred'] = samples_predictive['obs'].T.mean(axis=1)
df['sigma'] = samples_predictive['obs'].T.std(axis=1)
df['FVC_inf'] = df['FVC_pred'] - df['sigma']
df['FVC_sup'] = df['FVC_pred'] + df['sigma']
df = pd.merge(df, train[['Patient', 'Weeks', 'FVC']],
              how='left', on=['Patient', 'Weeks'])
df = df.rename(columns={'FVC': 'FVC_true'})
df.head()
```

      Patient
      Weeks
      FVC_pred
      sigma
      FVC_inf
      FVC_sup
      FVC_true

      0
      ID00007637202177411956430
      -12
      2220.162598
      159.290878
      2060.871826
      2379.453369
      NaN

      1
      ID00007637202177411956430
      -11
      2210.083496
      157.518021
      2052.565430
      2367.601562
      NaN

      2
      ID00007637202177411956430
      -10
      2213.199951
      154.847916
      2058.352051
      2368.047852
      NaN

      3
      ID00007637202177411956430
      -9
      2209.025391
      153.300079
      2055.725342
      2362.325439
      NaN

      4
      ID00007637202177411956430
      -8
      2203.191895
      156.085449
      2047.106445
      2359.277344
      NaN

Finally, let's see our predictions for 3 patients:

```python
def chart(patient_id, ax):
    data = df[df['Patient'] == patient_id]
    x = data['Weeks']
    ax.set_title(patient_id)
    ax.plot(x, data['FVC_true'], 'o')
    ax.plot(x, data['FVC_pred'])
    ax = sns.regplot(x, data['FVC_true'], ax=ax, ci=None,
                     line_kws={'color':'red'})
    ax.fill_between(x, data["FVC_inf"], data["FVC_sup"],
                    alpha=0.5, color='#ffcd3c')
    ax.set_ylabel('FVC')

f, axes = plt.subplots(1, 3, figsize=(15, 5))
chart('ID00007637202177411956430', axes[0])
chart('ID00009637202177434476278', axes[1])
chart('ID00011637202177653955184', axes[2])
```

The results are exactly what we expected to see! Highlight observations:
- The model adequately learned Bayesian Linear Regressions! The orange line (learned predicted FVC mean) is very inline with the red line (deterministic linear regression). But most important: it learned to predict uncertainty, showed in the light orange region (one sigma above and below the mean FVC line)
- The model predicts a higher uncertainty where the data points are more disperse (1st and 3rd patients). Conversely, where the points are closely grouped together (2nd patient), the model predicts a higher confidence (narrower light orange region)
- Finally, in all patients, we can see that the uncertainty grows as the look more into the future: the light orange region widens as the # of weeks grow!

### 4.3. Computing the modified Laplace Log Likelihood and RMSE

As mentioned earlier, the competition was evaluated on a modified version of the Laplace Log Likelihood. In medical applications, it is useful to evaluate a model's confidence in its decisions. Accordingly, the metric is designed to reflect both the accuracy and certainty of each prediction.

For each true FVC measurement, we predicted both an FVC and a confidence measure (standard deviation $\sigma$). The metric was computed as:

\begin{align}
\sigma_{clipped} &= max(\sigma, 70) \\
\delta &= min(|FVC_{true} - FVC_{pred}|, 1000) \\
metric &= -\dfrac{\sqrt{2}\delta}{\sigma_{clipped}} - \ln(\sqrt{2} \sigma_{clipped})
\end{align}

The error was thresholded at 1000 ml to avoid large errors adversely penalizing results, while the confidence values were clipped at 70 ml to reflect the approximate measurement uncertainty in FVC. The final score was calculated by averaging the metric across all (Patient, Week) pairs. Note that metric values will be negative and higher is better.

Next, we calculate the metric and RMSE:

```python
y = df.dropna()
rmse = ((y['FVC_pred'] - y['FVC_true']) ** 2).mean() ** (1/2)
print(f'RMSE: {rmse:.1f} ml')

sigma_c = y['sigma'].values
sigma_c[sigma_c < 70] = 70
delta = (y['FVC_pred'] - y['FVC_true']).abs()
delta[delta > 1000] = 1000
lll = - np.sqrt(2) * delta / sigma_c - np.log(np.sqrt(2) * sigma_c)
print(f'Laplace Log Likelihood: {lll.mean():.4f}')
```

    RMSE: 122.1 ml
    Laplace Log Likelihood: -6.1375

What do these numbers mean? It means if you adopted this approach, you would **outperform most of the public solutions** in the competition. Curiously, the vast majority of public solutions adopt a standard deterministic Neural Network, modelling uncertainty through a quantile loss. **Most of the people still adopt a frequentist approach**.

**Uncertainty** for single predictions becomes more and more important in machine learning and is often a requirement. **Especially when the consequenses of a wrong prediction are high**, we need to know what the probability distribution of an individual prediction is. For perspective, Kaggle just launched a new competition sponsored by Lyft, to build motion prediction models for self-driving vehicles. "We ask that you predict a few trajectories for every agent **and provide a confidence score for each of them**."

Finally, I hope the great work done by Pyro/NumPyro developers help democratize Bayesian methods, empowering an ever growing community of researchers and practitioners to create models that can not only generate predictions, but also assess uncertainty in their predictions.

## References

1. Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015). https://doi.org/10.1038/nature14541

2. Rainforth, Thomas William Gamlen. Automating Inference, Learning, and Design Using Probabilistic Programming. University of Oxford, 2017.

```python

```


# Colour Temperature & Correlated Colour Temperature

The [colour temperature](http://en.wikipedia.org/wiki/Color_temperature) noted $T_c$ is the temperature of a Planckian radiator whose radiation has the same chromaticity as that of a given stimulus. [1]

The [correlated colour temperature](http://en.wikipedia.org/wiki/Color_temperature#Correlated_color_temperature) noted $T_{cp}$ and shortened to $CCT$ is the temperature of the Planckian radiator having the chromaticity nearest the chromaticity associated with the given spectral distribution on a diagram where the (CIE 1931 2° Standard Observer based) $u^\prime, \cfrac{2}{3}v^\prime$ coordinates of the Planckian locus and the test stimulus are depicted. [2]

The *CIE Standard Illuminant A*, *CIE Standard Illuminant D65* and *CIE Illuminant E* illuminants plotted in the *CIE 1960 UCS Chromaticity Diagram*:

```python
%matplotlib inline
```

```python
import colour
from colour.plotting import *

colour.utilities.filter_warnings(True, False)

colour_plotting_defaults()

planckian_locus_chromaticity_diagram_plot_CIE1960UCS(['A', 'D65', 'E'])
```

```python
# Zooming into the *Planckian Locus*.
planckian_locus_chromaticity_diagram_plot_CIE1960UCS(
    ['A', 'D65', 'E'], bounding_box=[0.15, 0.35, 0.25, 0.45])
```

The concept of correlated colour temperature should not be used if the chromaticity of the test source differs more than $\Delta C=5\cdot10^{-2}$ from the Planckian radiator with: [2]

$$
\Delta C= \Biggl[ \Bigl(u_t^\prime-u_p^\prime\Bigr)^2+\cfrac{4}{9}\Bigl(v_t^\prime-v_p^\prime\Bigr)^2\Biggr]^{1/2}
$$

where $u_t^\prime$, $u_p^\prime$ refer to the test source, $v_t^\prime$, $v_p^\prime$ to the Planckian radiator.

[Colour](https://github.com/colour-science/colour/) implements various methods for correlated colour temperature computation $T_{cp}$ from chromaticity coordinates $xy$ or $uv$ and chomaticity coordinates $xy$ or $uv$ computation from correlated colour temperature:

* Robertson (1968) correlated colour temperature $T_{cp}$ and $D_{uv}$ computation method by interpolation between isotemperature lines.
* Ohno (2013) correlated colour temperature $T_{cp}$ and $D_{uv}$ computation method by direct approach and combined triangular and parabolic solutions.
* McCamy (1992) correlated colour temperature $T_{cp}$ cubic approximation computation method.
* Hernandez-Andres, Lee and Romero (1999) correlated colour temperature $T_{cp}$ cubic approximation computation method.
* Krystek (1985) chomaticity coordinates $uv$ polynomial approximation computation method.
* Kang et al. (2002) chomaticity coordinates $xy$ cubic approximation computation method.
* *CIE Illuminant D Series* chomaticity coordinates $xy$ computation method.

## Robertson (1968) Method

Robertson (1968) method is based on $T_{cp}$ computation by linear interpolation between two adjacent members of a defined set of 31 isotemperature lines. [3]

In the *CIE 1960 UCS chromaticity diagram* the distance $d_i$ of the chromaticity point of given source ($u_s$, $v_s$) from each of the chromaticity point ($u_i$, $v_i$) through which the $i$th isotemperature line of slope $t_i$ passes is calculated as follows: [3]

$$
\begin{equation}
d_i=\cfrac{(v_s-v_i)-t_i(u_s-u_i)}{(1+t_i^2)^{1/2}}
\end{equation}
$$

The chromaticity point ($u_s$, $v_s$) is located between the adjacent isotemperature lines $j$ and $j + 1$ if $d_j/d_{j+1} < 0$

$$
\begin{equation}
T_c=\Biggl[\cfrac{1}{T_j}+\cfrac{\theta_1}{\theta_1+\theta_2}\biggl(\cfrac{1}{T_{j+1}}-\cfrac{1}{T_j}\biggr)\Biggr]^{-1}
\end{equation}
$$

where $\theta_1$ and $\theta_2$ are respectively the angles between the isotemperature lines $T_j$ and $T_{j+1}$ and the line joining ($u_s$, $v_s$) to their intersection. Since the isotemperature lines are narrow spaced $\theta_1$ and $\theta_2$ are small enough that one can set $\theta_1/\theta_2 = \sin\theta_1/\sin\theta_2$. The above equation can then be written:

$$
\begin{equation}
T_c=\Biggl[\cfrac{1}{T_j}+\cfrac{d_j}{d_j-d_{j+1}}\biggl(\cfrac{1}{T_{j+1}}-\cfrac{1}{T_j}\biggr)\Biggr]^{-1}
\end{equation}
$$

The *colour.uv_to_CCT_Robertson1968* definition is used to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ ($d_i$):

```python
colour.temperature.uv_to_CCT_Robertson1968((0.19783451566098664, 0.31221744678060825))
```

    array([  6.50303994e+03,   3.25561654e-03])

*colour.uv_to_CCT* definition is implemented as a wrapper for various correlated colour temperature computation methods:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'Robertson 1968')
```

    array([  6.50303994e+03,   3.25561654e-03])

> Note: *'robertson1968'* is defined as a convenient alias for *'Robertson 1968'*:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'robertson1968')
```

    array([  6.50303994e+03,   3.25561654e-03])

Converting from correlated colour temperature $T_{cp}$ and distance $D_{uv}$ to chomaticity coordinates $uv$:

```python
colour.CCT_to_uv(6503.03994225557, 'Robertson 1968', D_uv=0.0032556165414977167)
```

    array([ 0.19783447,  0.31221739])

```python
colour.CCT_to_uv(6503.03994225557, 'robertson1968', D_uv=0.0032556165414977167)
```

    array([ 0.19783447,  0.31221739])

## Ohno (2013) Method

Ohno (2013) presented new practical accurate methods to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ with an error of 1 $K$ in $T_{cp}$ range from 1000 to 20,000 and $\pm$0.03 in $D_{uv}$. [4]

### Triangular Solution

The correlated colour temperature is calculated by searching the closest point on the Planckian locus on the *CIE 1960 UCS chromaticity diagram* but without the complexity of Roberston (1968) method.

A table of coordinates ($U_i$, $V_i$) of Planckian locus (Planckian ($u$, $v$) table) in the estimated range of correlated colour temperature needed is generated and then the distance $d_i$ from the chromaticity coordinates ($U_x$, $V_x$) of a test light source is calculated.

The point $i = m$ is the point where $d_i$ is the smallest in the table ensuring that the correlated colour temperature to be obtained lies between $T_{m-1}$ and $T_{m+1}$.

The previous computation is repeated $n$ times through cascade expansion in order to reduce errors.

A triangle is then formed by the chromaticity point ($U_x$, $V_x$) of the test light soure and the chromaticity points on Planckian locus at $T_{m-1}$ and $T_{m+1}$. The blackbody temperature $T_x$ for the closest point to the line between $T_{m-1}$ and $T_{m+1}$ is calculated as follows: [4]

$$
\begin{equation}
T_x=T_{m-1}+(T_{m+1}-T_{m-1})\cdot\cfrac{x}{l}
\end{equation}
$$

with

$$
\begin{equation}
\begin{aligned}
x&=\cfrac{d_{m-1}^2-d_{m+1}^2+l^2}{2l}\\
l&=\sqrt{(u_{m+1}-u_{m-1})^2+(v_{m+1}-v_{m-1})^2}
\end{aligned}
\end{equation}
$$

$D_{uv}$ is then calculated as follows:

$$
\begin{equation}
D_{uv}=(d_{m-1}^2-x^2)^{1/2}\cdot sgn(v_x-v_{T_x})
\end{equation}
$$

with

$$
\begin{equation}
\begin{aligned}
v_{T_x}&=v_{m-1}+\{v_{m+1}-v_{m-1}\}\cdot x/l\\
SIGN(z)&=1\ for\ z \geq0\ and\ SIGN(z)=-1\ for\ z <0
\end{aligned}
\end{equation}
$$

Errors due to the non linearity of the correlated colour temperature scale on ($u$, $v$) coordinates are reduced by applying the following correction:

$$
\begin{equation}
T_{x,cor}=T_x\times 0.99991
\end{equation}
$$

This correction is not needed for Planckian ($u$, $v$) table with steps of 0.25% or smaller.

### Parabolic Solution

After finding $T_{m-1}$ and $T_{m+1}$ as described in the [triangular solution](#Triangular-Solution) method above, $d_{m−1}$, $d_m$, $d_{m+1}$ are fitted to a parabolic function. The polynomial is derived from $d_{m−1}$, $d_m$, $d_{m+1}$ and $T_{m−1}$, $T_m$, $T_{m+1}$ as: [4]

$$
\begin{equation}
d(T)=aT^2+bT+c
\end{equation}
$$

where

$$
\begin{equation}
\begin{aligned}
a&\ =[T_{m-1}(d_{m+1}-d_m)+T_m(d_{m-1}-d_{m+1})+T_{m+1}(d_m-d_{m-1})]\cdot X^{-1}\\
b&\ =-[T_{m-1}^2(d_{m+1}-d_m)+T_m^2(d_{m-1}-d_{m+1})+T_{m+1}^2(d_m-d_{m-1})]\cdot X^{-1}\\
c&\ =-[d_{m-1}(T_{m+1}-T_m)\cdot T_m\cdot T_{m+1}+d_m(T_{m-1}-T_{m+1})\cdot T_{m-1}\cdot T_{m+1}+d_{m+1}(T_m-T_{m-1})\cdot T_{m-1}\cdot T_m]\cdot X^{-1}
\end{aligned}
\end{equation}
$$

with

$$
\begin{equation}
X=(T_{m+1}-T_m)(T_{m-1}-T_{m+1})(T_m-T_{m-1})
\end{equation}
$$

The correlated colour temperature $T=T_x$ is then obtained as follows:

$$
\begin{equation}
T_X=-\cfrac{b}{2a}\qquad\because d^\prime(T)=2aT_x+b=0
\end{equation}
$$

The correction factor $T_{x,cor}$ for nonlinearity is applied as described in the [triangular solution](#Triangular-Solution) method.

$D_{uv}$ is then calculated as follows:

$$
\begin{equation}
D_{uv}=SIGN(v_x-v_{T_x})\cdot(aT_{x,cor}^2+bT_{x,cor}+c)
\end{equation}
$$

with

$$
\begin{equation}
SIGN(z)=1\ for\ z \geq0\ and\ SIGN(z)=-1\ for\ z <0
\end{equation}
$$

### Combined Solution

The parabolic solution works accurately except in on or near the Planckian locus. Taking triangular solution results for $|D_{uv}| < 0.002$ and the parabolic solution results for other regions solves that problem. [4]

The *colour.uv_to_CCT_Ohno2013* definition is used to calculate the correlated colour temperature $T_{cp}$ and distance $D_{uv}$ ($d_i$):

```python
colour.temperature.uv_to_CCT_Ohno2013((0.19783451566098664, 0.31221744678060825))
```

    array([  6.50364955e+03,   3.20563638e-03])

Precision can be changed by passing a value to the *iterations* argument:

```python
for i in range(10):
    print(colour.temperature.uv_to_CCT_Ohno2013(
        (0.19783451566098664, 0.31221744678060825), iterations=i + 1))
```

    [  1.68848187e+04   2.66789445e-03]
    [  6.79456522e+03   3.46904097e-03]
    [  6.55383806e+03   3.48674101e-03]
    [  6.50607833e+03   3.20681676e-03]
    [  6.50374738e+03   3.20565144e-03]
    [  6.50364955e+03   3.20563638e-03]
    [  6.50364731e+03   3.20563611e-03]
    [  6.50364722e+03   3.20563604e-03]
    [  6.50364721e+03   3.20563375e-03]
    [  6.50364721e+03   3.20561976e-03]

Using the *colour.uv_to_CCT* wrapper definition:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'Ohno 2013')
```

    array([  6.50364955e+03,   3.20563638e-03])

> Note: *'ohno2013'* is defined as a convenient alias for *'Ohno 2013'*:

```python
colour.uv_to_CCT((0.19783451566098664, 0.31221744678060825), 'ohno2013')
```

    array([  6.50364955e+03,   3.20563638e-03])

Converting from correlated colour temperature $T_{cp}$ and distance $D_{uv}$ to chomaticity coordinates $uv$:

```python
colour.CCT_to_uv(6503.03994225557, 'Ohno 2013', D_uv=0.0032556165414977167, )
```

    array([ 0.19779726,  0.31225121])

```python
colour.CCT_to_uv(6503.03994225557, 'Ohno 2013', D_uv=0.0032556165414977167)
```

    array([ 0.19779726,  0.31225121])

## McCamy (1992) Method

McCamy (1992) proposed an equation to compute correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ by using a chromaticity epicenter ($x_e$, $y_e$) where the isotemperature lines in some of the correlated colour temperature range converge and the inverse slope of the line $n$ that connects it to $x$, $y$. [5]

The cubic approximation equation is defined as follows: [5]

$$
\begin{equation}
T_{cp}=-449n^3+3525n^2-6823.3n+5520.33
\end{equation}
$$

where

$$
\begin{equation}
n=\cfrac{x-x_e}{y-ye}\\
x_e=0.3320\qquad y_e=0.1858
\end{equation}
$$

The *colour.xy_to_CCT_mccamy* definition is used to calculate the correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.temperature.xy_to_CCT_McCamy1992((0.31271, 0.32902))
```

    6504.3893830489724

The *colour.xy_to_CCT* definition is implemented as a wrapper for various correlated colour temperature computation methods from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'McCamy 1992')
```

    6504.3893830489724

> Note: *'mccamy1992'* is defined as a convenient alias for *'McCamy 1992'*:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'mccamy1992')
```

    6504.3893830489724

## Hernandez-Andres, Lee and Romero (1999) Method

Hernandez-Andres, Lee and Romero (1999) extended McCamy (1992) work by using a second epicenter to extend the accuracy over a wider correlated colour temperature and chromaticity coordinates range ($3000$–$10^6K$). [6]

The new extended equation to calculate the correlated colour temperature $T_{cp}$ is defined as follows: [6]

$$
\begin{equation}
T_{cp}=A_0+A_1exp(-n/t_1)+A_2exp(-n/t_2)+A_3exp(-n/t_3)
\end{equation}
$$

where

$$
\begin{equation}
n=\cfrac{x-x_e}{y-ye}\\
\end{equation}
$$

with

| Constants | $T_{cp}$ Range ($K$) $3000$-$50,000$ | $T_{cp}$ Range ($K$) $50,000$-$8\times10^5$ |
|:---------:|:------------------------------------:|:-------------------------------------------:|
| $A_0$     | $-949.86315$                         | $36284.48953$                               |
| $A_1$     | $6253.80338$                         | $0.00228$                                   |
| $t_1$     | $0.92159$                            | $0.07861$                                   |
| $A_2$     | $28.70599$                           | $5.4535\times10^{-36}$                      |
| $t_2$     | $0.20039$                            | $0.01543$                                   |
| $A_3$     | $0.00004$                            |                                             |
| $t_3$     | $0.07125$                            |                                             |
| $x_e$     | $0.3366$                             | $0.3356$                                    |
| $y_e$     | $0.1735$                             | $0.1691$                                    |

The *colour.xy_to_CCT_hernandez* definition is used to calculate the correlated colour temperature $T_{cp}$ from *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$:

```python
colour.temperature.xy_to_CCT_Hernandez1999((0.31271, 0.32902))
```

    6500.0421533365825

Using the *colour.xy_to_CCT* wrapper definition:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'Hernandez 1999')
```

    6500.0421533365825

> Note: *'hernandez1999'* is defined as a convenient alias for *'Hernandez 1999'*:

```python
colour.xy_to_CCT((0.31271, 0.32902), 'hernandez1999')
```

    6500.0421533365825

## Krystek (1985) Method

Krystek (1985) proposed a polynomial approximation valid from $1000K$ to $15000K$. [7]

The *CIE UCS* colourspace chromaticity coordinates $u$, $v$ are given by the following equations: [7]

$$
\begin{equation}
\begin{aligned}
u&\ =\cfrac{0.860117757 + 1.54118254 \times 10^{-4} T + 1.28641212 \times 10^{-7} T^2}{1 + 8.42420235 \times 10^{-4} T + 7.08145163 \times 10^{-7} T^2}\\
v&\ =\cfrac{0.317398726 + 4.22806245 \times 10^{-5} T + 4.20481691 \times 10^{-8} T^2}{1 - 2.89741816 \times 10^{-5} T + 1.61456053 \times 10^{-7} T^2}
\end{aligned}
\end{equation}
$$

The *colour.CCT_to_uv_Krystek1985* definition is used to calculate the *CIE UCS* colourspace chromaticity coordinates $u$, $v$ from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_uv_Krystek1985(6504.389383048972)
```

    array([ 0.18376696,  0.30934437])

Using the *colour.CCT_to_uv* wrapper definition:

```python
colour.CCT_to_uv(6504.389383048972, 'Krystek 1985')
```

    array([ 0.18376696,  0.30934437])

## Kang et al. (2002) Method

Kang et al. (2002) proposed an advanced colour-temperature control system for HDTV applications in the range from $1667K$ to $25000K$. [8]

The *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ are given by the following equations: [8]

$$
\begin{equation}
\begin{aligned}
x&\ =\begin{cases}-0.2661239\cfrac{10^9}{T_{cp}^3}-0.2343589\cfrac{10^6}{T_{cp}^2}+0.8776956\cfrac{10^3}{T_{cp}}+0.179910 & for\ 1667K\leq T_{cp}\leq4000k\\
-3.0258469\cfrac{10^9}{T_{cp}^3}+2.1070379\cfrac{10^6}{T_{cp}^2}+0.2226347\cfrac{10^3}{T_{cp}}+0.24039 & for\ 4000K\leq T_{cp}\leq25000k\end{cases}\\
y&\ =\begin{cases}-1.1063814x^3-1.34811020x^2+2.18555832x-0.20219683 & for\ 1667K\leq T_{cp}\leq2222k\\
-0.9549476x^3-1.37418593x^2+2.09137015x-0.16748867 & for\ 2222K\leq T_{cp}\leq4000k\\
3.0817580x^3-5.8733867x^2+3.75112997x-0.37001483 & for\ 4000K\leq T_{cp}\leq25000k\end{cases}
\end{aligned}
\end{equation}
$$

The *colour.CCT_to_xy_kang* definition is used to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_xy_Kang2002(6504.389383048972)
```

    array([ 0.313426  ,  0.32359597])

The *colour.CCT_to_xy* definition is implemented as a wrapper for various *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ computation from correlated colour temperature:

```python
colour.CCT_to_xy(6504.389383048972, 'Kang 2002')
```

    array([ 0.313426  ,  0.32359597])

> Note: *'kang2002'* is defined as a convenient alias for *'Kang 2002'*:

```python
colour.CCT_to_xy(6504.389383048972, 'kang2002')
```

    array([ 0.313426  ,  0.32359597])

## *CIE Illuminant D Series* Method

Judd et al. (1964) defined the following equations to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x_D$, $y_D$ of a *CIE Illuminant D Series*: [9]

$$
\begin{equation}
\begin{aligned}
x_D&\ =\begin{cases}-4,6070\cfrac{10^9}{T_{cp}^3}+2.9678\cfrac{10^6}{T_{cp}^2}+0.09911\cfrac{10^3}{T_{cp}}+0.244063 & for\ 4000K\leq T_{cp}\leq7000k\\
-2.0064\cfrac{10^9}{T_{cp}^3}+1.9018\cfrac{10^6}{T_{cp}^2}+0.24748\cfrac{10^3}{T_{cp}}+0.237040 & for\ 7000K\leq T_{cp}\leq25000k\end{cases}\\
y_D&\ =-3.000x_D^2+0.2.870x_D-0.275
\end{aligned}
\end{equation}
$$

The *colour.CCT_to_xy_CIE_D* definition is used to calculate the *CIE 1931 2° Standard Observer* chromaticity coordinates $x$, $y$ of a *CIE Illuminant D Series* from correlated colour temperature $T_{cp}$:

```python
colour.temperature.CCT_to_xy_CIE_D(6504.389383048972)
```

    array([ 0.31270775,  0.32911283])

Using the *colour.CCT_to_xy* wrapper definition:

```python
colour.CCT_to_xy(6504.389383048972, 'CIE Illuminant D Series')
```

    array([ 0.31270775,  0.32911283])

> Note: *'cie_d'* is defined as a convenient alias for *'CIE Illuminant D Series'*:

```python
colour.CCT_to_xy(6504.389383048972, 'cie_d')
```

    array([ 0.31270775,  0.32911283])

## Bibliography

1. ^ CIE. (n.d.). 17-231 colour temperature [Tc]. Retrieved from http://eilv.cie.co.at/term/231
2. ^ CIE. (n.d.). 17-258 correlated colour temperature [Tcp]. Retrieved from http://eilv.cie.co.at/term/258
3. ^ Wyszecki, G., & Stiles, W. S. (2000). DISTRIBUTION TEMPERATURE, COLOR TEMPERATURE, AND CORRELATED COLOR TEMPERATURE. In Color Science: Concepts and Methods, Quantitative Data and Formulae (pp. 224–229). Wiley. ISBN:978-0471399186
4. ^ Ohno, Y. (2014). Practical Use and Calculation of CCT and Duv. LEUKOS, 10(1), 47–55. doi:10.1080/15502724.2014.839020
5. ^ Wikipedia. (n.d.). Approximation. Retrieved June 28, 2014, from http://en.wikipedia.org/wiki/Color_temperature#Approximation
6. ^ Hernández-Andrés, J., Lee, R. L., & Romero, J. (1999). Calculating correlated color temperatures across the entire gamut of daylight and skylight chromaticities. Applied Optics, 38(27), 5703–5709. doi:10.1364/AO.38.005703
7. ^ Krystek, M. (1985). An algorithm to calculate correlated colour temperature. Color Research & Application, 10(1), 38–40. doi:10.1002/col.5080100109
8. ^ Kang, B., Moon, O., Hong, C., Lee, H., Cho, B., & Kim, Y. (2002). Design of advanced color: Temperature control system for HDTV applications. Journal of the Korean …, 41(6), 865–871. Retrieved from http://cat.inist.fr/?aModele=afficheN&cpsidt=14448733
9. ^ Wyszecki, G., & Stiles, W. S. (2000). CIE Method of Calculating D-Illuminants. In *Color Science: Concepts and Methods, Quantitative Data and Formulae* (pp. 145–146). Wiley. ISBN:978-0471399186


This notebook is part of https://github.com/AudioSceneDescriptionFormat/splines, see also http://splines.readthedocs.io/.

# Bézier Splines

See also https://pomax.github.io/bezierinfo/.

There are several ways to get to Bézier curves, one was already shown in
[the notebook about Hermite curves](hermite-uniform.ipynb#Relation-to-Bézier-Splines)
(but only for cubic curves).

TODO: first explain control polylines and then link to Hermite splines?

Another one is the so-called De Casteljau's algorithm. (TODO: link to De Casteljau)

One nice aspect of this is that the algorithm can be used for arbitrary polynomial degrees.

A Bézier spline is defined by a so-called *control polyline* (or *control polygon*), which comprises a sequence of *control points*.
Some of those control points are part of the final spline curve, others lie outside of it.
The degree of a spline segment determines how many "off-curve" control points are between two "on-curve" control points.

For example, in a cubic (degree = 3) Bézier spline there are two (= degree - 1) "off-curve" control points.

Two equally valid viewpoints for what a Bézier spline is:

* A sequence of curve segments, each defined by degree + 1 control points.
  The first control point of a segment is the same as the last control point of the previous one.

* A sequence of control points that can be used to shape the resulting curve.
  Every degree'th control point lies on the curve and the others define the shape of the curve segments.

TODO: most well-known: cubic Bézier splines (show screenshot from drawing program, e.g. Inkscape).
The two "off-curve" control points are shown as "handles".

TODO: typical set of constraints on continuity in drawing programs: C0, C1, G1

### Preparations

Before we continue, here are are few preparations for the following calculations:

```python
%matplotlib inline
%config InlineBackend.print_figure_kwargs = {'bbox_inches': None}
import matplotlib.pyplot as plt
import numpy as np
import sympy as sp
sp.init_printing()
```

We import stuff from the file [utility.py](utility.py):

```python
from utility import NamedExpression, NamedMatrix
```

Let's prepare a few symbols for later use:

```python
t, x0, x1, x2, x3, x4 = sp.symbols('t, xbm:5')
```

... and a helper function for plotting:

```python
def plot_curve(func, points, dots=30, ax=None):
    if ax is None:
        ax = plt.gca()
    times = np.linspace(0, 1, dots)
    ax.plot(*func(points, times).T, '.')
    ax.scatter(*np.asarray(points).T, marker='x', c='black')
    ax.set_title(func.__name__ + ' Bézier curve')
    ax.axis('equal')
```

We also need to prepare for the animations we will see below.
This is using code from the file [casteljau.py](casteljau.py):

```python
from casteljau import create_animation
from IPython.display import display, HTML

def show_casteljau_animation(points, frames=30, interval=200):
    ani = create_animation(points, frames=frames)
    display(HTML(ani.to_jshtml(default_mode='reflect')))
    plt.close()  # avoid spurious figure display
```

### Degree 1, a.k.a. linear

But let's start with the trivial case:
A Bézier spline of degree 1 is just a piecewise linear curve connecting all the control points.
There are no "off-curve" control points that could bend the curve segments.

Assume that we have two control points, $\boldsymbol{x}_0$ and $\boldsymbol{x}_1$ ...

... linear equation ...:

\begin{equation}
\boldsymbol{p}_{0,1}(t) = \boldsymbol{x}_0 + t (\boldsymbol{x}_1 - \boldsymbol{x}_0)
\end{equation}

... in other words ... this is called *affine combination*, but we don't really have to worry about it ...

\begin{equation}
\boldsymbol{p}_{0,1}(t) = (1 - t) \boldsymbol{x}_0 + t \boldsymbol{x}_1
\end{equation}

... with $t \in [0, 1]$ (which is called *uniform*)

TODO: show change of variables for *non-uniform* curve?

Since we will be needing quite a bunch of those affine combinations, let's create a helper function:

```python
def affine_combination(one, two):
    return (1 - t) * one + t * two
```

Now we can define the equation in SymPy:

```python
p01 = NamedExpression('pbm_0,1', affine_combination(x0, x1))
p01
```

```python
b1 = [p01.expr.expand().coeff(x.name).factor() for x in (x0, x1)]
b1
```

Doesn't look like much, but those are the Bernstein bases for degree 1 (<https://en.wikipedia.org/wiki/Bernstein_polynomial>).

It doesn't get much more interesting if we plot them:

```python
sp.plot(*b1, (t, 0, 1));
```

If you want to convert this to coefficients for the monomial basis $[t, 1]$ instead of the Bernstein basis functions, you can use this matrix:

```python
M_B1 = NamedMatrix(
    r'{M_\text{B}^{(1)}}',
    sp.Matrix([[c.coeff(x) for x in (x0, x1)]
               for c in p01.expr.as_poly(t).all_coeffs()]))
M_B1
```

Applying this matrix leads to the coefficients of the linear equation mentioned in the beginning of this section
($\boldsymbol{p}_{0,1}(t) = t (\boldsymbol{x}_1 - \boldsymbol{x}_0) + \boldsymbol{x}_0$):

```python
sp.MatMul(M_B1.expr, sp.Matrix([x0, x1]))
```

```python
_.doit()
```

If you ever need that, here's the inverse:

```python
M_B1.I
```

Anywho, let's calculate points on the curve by using the Bernstein basis functions:

```python
def linear(points, times):
    """Evaluate linear Bézier curve (given by two points) at given times."""
    return np.column_stack(sp.lambdify(t, b1)(times)) @ points
```

```python
points = [
    (0, 0),
    (1, 0.5),
]
```

```python
plot_curve(linear, points)
```

```python
show_casteljau_animation(points)
```

I know, not very exciting. But it gets better!

### Degree 2, a.k.a. quadratic

Consider three control points, $\boldsymbol{x}_0$, $\boldsymbol{x}_1$ and $\boldsymbol{x}_2$ ...

We use the affine combinations of the first two points from above ...

```python
p01
```

... and we do the same thing for the second and third point:

```python
p12 = NamedExpression('pbm_1,2', affine_combination(x1, x2))
p12
```

Finally, we make another affine combination of those two results:

```python
p02 = NamedExpression('pbm_0,2', affine_combination(p01.expr, p12.expr))
p02
```

Bernstein basis functions:

```python
b2 = [p02.expr.expand().coeff(x.name).factor() for x in (x0, x1, x2)]
b2
```

```python
sp.plot(*b2, (t, 0, 1));
```

```python
M_B2 = NamedMatrix(
    r'{M_\text{B}^{(2)}}',
    sp.Matrix([[c.coeff(x) for x in (x0, x1, x2)]
               for c in p02.expr.as_poly(t).all_coeffs()]))
M_B2
```

```python
M_B2.I
```

```python
def quadratic(points, times):
    """Evaluate quadratic Bézier curve (given by three points) at given times."""
    return np.column_stack(sp.lambdify(t, b2)(times)) @ points
```

```python
points = [
    (0, 0),
    (0.2, 0.5),
    (1, -0.3),
]
```

```python
plot_curve(quadratic, points)
```

```python
show_casteljau_animation(points)
```

For some more insight, let's look at the first derivative of the curve (i.e. the tangent vector):

```python
v02 = p02.expr.diff(t)
```

... at the beginning and the end of the curve:

```python
v02.subs(t, 0)
```

```python
v02.subs(t, 1)
```

This shows that the tangent vector at the beginning and end of the curve is parallel to the line
from $\boldsymbol{x}_0$ to $\boldsymbol{x}_1$ and
from $\boldsymbol{x}_1$ to $\boldsymbol{x}_2$, respectively.
The length of the tangent vectors is twice the length of those lines.

You might have already seen that coming, but it turns out that the last line in de Casteljau's algorithm ($\boldsymbol{p}_{1,2}(t) - \boldsymbol{p}_{0,1}(t)$ in our case) is exactly half of the tangent vector (at any given $t \in [0, 1]$).

```python
(v02 - 2 * (p12.expr - p01.expr)).simplify()
```

In case you are wondering, the factor 2 comes from the degree 2 of our quadratic curve.

### Degree 3, a.k.a. cubic

Consider four control points, $\boldsymbol{x}_0$, $\boldsymbol{x}_1$, $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$ ...

By now, the pattern should be clear: We take the result from the first three points from above and affine-combine it with the result for the three points $\boldsymbol{x}_1$, $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$.

Combination of $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$:

```python
p23 = NamedExpression('pbm_2,3', affine_combination(x2, x3))
p23
```

Combination of $\boldsymbol{x}_1$, $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$:

```python
p13 = NamedExpression('pbm_1,3', affine_combination(p12.expr, p23.expr))
p13
```

Combination of $\boldsymbol{x}_0$, $\boldsymbol{x}_1$, $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$:

```python
p03 = NamedExpression('pbm_0,3', affine_combination(p02.expr, p13.expr))
p03
```

Bernstein bases:

```python
b3 = [p03.expr.expand().coeff(x.name).factor() for x in (x0, x1, x2, x3)]
b3
```

TODO: show that those are the same Bernstein bases as in the notebook about Hermite splines

```python
sp.plot(*b3, (t, 0, 1));
```

```python
M_B3 = NamedMatrix(
    r'{M_\text{B}^{(3)}}',
    sp.Matrix([[c.coeff(x) for x in (x0, x1, x2, x3)]
               for c in p03.expr.as_poly(t).all_coeffs()]))
M_B3
```

```python
M_B3.I
```

```python
def cubic(points, times):
    """Evaluate cubic Bézier curve (given by four points) at given times."""
    return np.column_stack(sp.lambdify(t, b3)(times)) @ points
```

```python
points = [
    (0, 0.3),
    (0.2, 0.5),
    (0.1, 0),
    (1, 0.2),
]
```

```python
plot_curve(cubic, points)
```

```python
show_casteljau_animation(points)
```

As before, let's look at the derivative (i.e. the tangent vector) of the curve:

```python
v03 = p03.expr.diff(t)
```

... at the beginning and the end of the curve:

```python
v03.subs(t, 0)
```

```python
v03.subs(t, 1)
```

This shows that the tangent vector at the beginning and end of the curve is parallel to the line
from $\boldsymbol{x}_0$ to $\boldsymbol{x}_1$ and
from $\boldsymbol{x}_2$ to $\boldsymbol{x}_3$, respectively.
The length of the tangent vectors is three times the length of those lines.

We can now see that the last line in de Casteljau's algorithm ($\boldsymbol{p}_{1,3}(t) - \boldsymbol{p}_{0,2}(t)$ in this case) is exactly a third of the tangent vector (at any given $t \in [0, 1]$):

```python
(v03 - 3 * (p13.expr - p02.expr)).simplify()
```

Again, the factor 3 comes from the degree 3 of our curve.

We now know the tangent vectors at the beginning and the end of the curve, and obviously we know the values of the curve at the beginning and the end:

```python
p03.expr.subs(t, 0), p03.expr.subs(t, 1)
```

With these four pieces of information, we can find a transformation from the four Bézier control points to the two control points and two tangent vectors of Hermite splines:

```python
M_BtoH = NamedMatrix(
    r'{M_\text{B$\to$H}}',
    sp.Matrix([[expr.coeff(cv) for cv in [x0, x1, x2, x3]]
               for expr in [x0, x3, v03.subs(t, 0), v03.subs(t, 1)]]))
M_BtoH
```

And we can simply invert this if we want to go in the other direction, from Hermite to Bézier:

```python
M_BtoH.I.pull_out(sp.S.One / 3)
```

Of course, those are the same matrices as shown in the [notebook about uniform cubic Hermite splines](hermite-uniform.ipynb).

TODO: show tangent vectors for non-uniform case

### Degree 4, a.k.a. quartic

Consider five control points, $\boldsymbol{x}_0$, $\boldsymbol{x}_1$, $\boldsymbol{x}_2$, $\boldsymbol{x}_3$ and $\boldsymbol{x}_4$ ...

More combinations!

```python
p34 = NamedExpression('pbm_3,4', affine_combination(x3, x4))
p24 = NamedExpression('pbm_2,4', affine_combination(p23.expr, p34.expr))
p14 = NamedExpression('pbm_1,4', affine_combination(p13.expr, p24.expr))
p04 = NamedExpression('pbm_0,4', affine_combination(p03.expr, p14.expr))
p04
```

Kinda long, but anyway, let's try to extract the Bernstein bases:

```python
b4 = [p04.expr.expand().coeff(x.name).factor() for x in (x0, x1, x2, x3, x4)]
b4
```

```python
sp.plot(*b4, (t, 0, 1));
```

```python
M_B4 = NamedMatrix(
    '{M_B^{(4)}}',
    sp.Matrix([[c.coeff(x) for x in (x0, x1, x2, x3, x4)]
               for c in p04.expr.as_poly(t).all_coeffs()]))
M_B4
```

```python
M_B4.I
```

```python
def quartic(points, times):
    """Evaluate quartic Bézier curve (given by five points) at given times."""
    return np.column_stack(sp.lambdify(t, b4)(times)) @ points
```

```python
points = [
    (0, 0),
    (0.5, 0),
    (0.7, 1),
    (1, 1.5),
    (-1, 1),
]
```

```python
plot_curve(quartic, points)
```

```python
show_casteljau_animation(points)
```

For completeness' sake, let's look at the derivative (i.e. the tangent vector) of the curve:

```python
v04 = p04.expr.diff(t)
```

... at the beginning and the end of the curve:

```python
v04.subs(t, 0)
```

```python
v04.subs(t, 1)
```

By now it shouldn't be surprising that the tangent vector at the beginning and end of the curve is parallel to the line
from $\boldsymbol{x}_0$ to $\boldsymbol{x}_1$ and
from $\boldsymbol{x}_3$ to $\boldsymbol{x}_4$, respectively.
The length of the tangent vectors is four times the length of those lines.

The last line in de Casteljau's algorithm ($\boldsymbol{p}_{1,4}(t) - \boldsymbol{p}_{0,3}(t)$ in this case) is exactly a fourth of the tangent vector (at any given $t \in [0, 1]$):

```python
(v04 - 4 * (p14.expr - p03.expr)).simplify()
```

Again, the factor 4 comes from the degree 4 of our curve.

### Arbitrary Degree

We could go on doing this for higher and higher degrees, but this would get more and more annoying.

Luckily, there is a closed formula available to calculate Bernstein polynomials for an arbitrary degree $n$!

\begin{equation}
b_{i,n}(x) = {n \choose i} x^i \left( 1 - x \right)^{n - i}, \quad i = 0, \ldots, n.
\end{equation}

with the *binomial coefficient* ${n \choose i} = \frac{n!}{i!(n - i)!}$.

TODO: link to proof?

TODO: show Bernstein polynomials for "quintic" etc.?

```python
show_casteljau_animation([
    (0, 0),
    (-1, 1),
    (-0.5, 2),
    (1, 2.5),
    (2, 2),
    (2, 1.5),
    (0.5, 0.5),
    (1, -0.5),
])
```

```python

```


$$ \newcommand{\pd}[2]{ \frac{\partial #1}{\partial #2} }
\newcommand{\od}[2]{\frac{d #1}{d #2}}
\newcommand{\td}[2]{\frac{D #1}{D #2}}
\newcommand{\ab}[1]{\langle #1 \rangle}
\newcommand{\bss}[1]{\textsf{\textbf{#1}}}
\newcommand{\ol}{\overline}
\newcommand{\olx}[1]{\overline{#1}^x}
$$

# Equations of Motion

In this class, we will not undertake a formal derivation of the fluid equations of motion. That would take up too much time and require too much math. Interested students can consult Marshall and Plumb (sec. XX) for a more in-depth derivation. Instead, we will try to build an intuitive understanding of the equations of motion based on things we already all understand.

## Newton's Second Law for a Water Parcel

The starting point for understanding the equations of motion is the notion of a "water parcel". The water is hypothetical, infinitesimally small fluid element. We imagine the fluid to be composed of an infinite continuum of such parcels, each following its own unique path.

The equation of fluid motion is basically just [Newton's second law](https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Newton.27s_second_law) applied to this parcel. Newton's second law states that

$$m \mathbf{a} = \mathbf{F}$$

where $\mathbf{F}$ represents the sum of all forces acting on the parcel, $m$ is the parcel mass and $\mathbf{a}$ is the acceleration of the parcel. Both sides of the equations have the units of N (Newtons; 1 N = 1 kg m s$^{-2}$).

The expression above is valid for a finite-sized object, like a cannon ball. To move to an infintessimally small water parcel, we replace $m$, the mass, with $\rho$, the density, i.e.~the mass per unit volume:

$$ \rho \mathbf{a} = \mathbf{f} \ .$$

The force vector $\mathbf{f}$ now represents the force per unit volume and has units N m$^{-3}$.

In order to develop this equation into a useful form, we need to work on the following two problems:
1. What is the acceleration of a fluid parcel on a rotating, spherical planet?
1. What are the different forces that make up $\mathbf{f}$?

_Cartoon of a water parcel moving through space along a path (black line). The colored arrows represent the different forces on and acceleration of the parcel._

We will start with the forces, because they are easier. There are three forces we have to consider: the pressure gradient force ($\mathbf{f}_p$), the gravitational force ($\mathbf{f}_g$) and the viscous force ($\mathbf{f}_{visc}$). Together these sum to give the net force:

$$ \mathbf{f} = \mathbf{f}_p + \mathbf{f}_g + \mathbf{f}_{visc} $$

## Forces

### Pressure Gradient Force

We all have an intuitive understanding of pressure. [Pressure](https://en.wikipedia.org/wiki/Pressure) is the force the surrounding fluid exerts perpendicular to the water parcel surface. We denote pressure with the variable $p$: it has units of Pa (Pascals; 1 Pa = 1 N m$^{-2}$). Pressure is _isotropic_, i.e. it is the same in all directions.

If the pressure field is uniform, there will be no net force exerted on the water parcel, because the force is exactly the same on all sides. Only if there is a _pressure gradient_ does the parcel feel a force; the parcel is pushed away from the higher pressure towards the lower pressure. Mathematically, this becomes

$$\mathbf{f}_p = - \nabla p \ .$$

### Gravitational Force

The gravitational force per unit volume on the fluid parcel is given by

$$ \mathbf{f}_g = - g \rho \hat{\mathbf{k}} $$

where $g \simeq$ 9.8 m s$^{-2}$ is the gravitational acceleration and $\hat{\mathbf{k}}$ is the unit vector pointing "up", i.e. away from Earth's center of mass. A slightly more general form is to express this force in terms of the gravitational [geopotential field](https://en.wikipedia.org/wiki/Geopotential) $\Phi_g$ of earth:

$$ \mathbf{f}_g = - \rho \nabla \Phi_g \ .$$

The gravitational vector $g \hat{\mathbf{k}}$ is defined as the gradient of $\Phi_g$.

The gravitational force acts to pull all fluid parcels "down" towards the Earth's center of mass. However, because of the factor of $\rho$, it is less strong for less dense water. What matters is the relative difference of $\mathbf{f}_g$ between different fluid parcels. This leads to the concept of [buoyancy](https://en.wikipedia.org/wiki/Buoyancy) (discussed in more detail below), which can push water parcels both downward _and_ upward.

## Viscous Force

Viscosity represents the molecular friction that occurs within the fluid as is moves. The viscous force is given by the formula

$$ \mathbf{f}_{visc} = \mu \nabla^2 \mathbf{u} $$

where $\mathbf{u}$ is the fluid velocity and $\mu$ is the _dynamic viscosity_, a molecular property of the fluid. A representiative value of $\mu$ is 1.3 $\times 10^{-3}$ N s m$^{-2}$.

It can be easier to understand the viscous force if we consider flow in only one direction. Consider a zonal flow $u(y)$ that varies in the y-dimension, as shown in the cartoon below. In this case, the viscous vorce in the x-direction is given by

$$ \mu \frac{\partial^2 u}{\partial y^2} \ .$$

Which is proportional to the _curvature_ of the profile $u(y)$. The cartoon illustrates a typical jet, with enhanced velocity at the jet axis. The curvature of such a jet profile ensures that the viscous force is negatuve, such that it acts to _retard_ the jet, with a maximum magnitude right at the jet core. One interpretation of this situation is that viscosity diffuses momentum away from the jet core towards the jet flanks.

Viscosity is mathematically and physically analagous to diffusion.

## Acceleration

### Defining position, velocity and acceleration

A water parcel's position at a given time $t$ is given by a three dimensional vector $\mathbf{x}(x,y,z,t)$. The instantaneous rate of change of the position of such a fluid element defines the fluid velocity:

$$ \mathbf{u}  = (u, v, w)\equiv \od{\mathbf{x}}{t} \ . $$

The rate of change of velocity is acceleration:

$$ \mathbf{a}  = \od{\mathbf{u}}{t} \ . $$

If we planned to use the Lagrangian approach for describing the fluid motion (following each water parcel around), we could leave it at that.
But we intend to use the Eulerian approach (measuring quantities at fixed locations in space).
We need to recall the definition of the _material derivative_ from the advection / diffusion lecture.
The rate of change of any water parcel property in the Eulerian frame is given by the material derivative

$$ \td{}{t} = \pd{}{t} + \mathbf{u} \cdot \nabla . $$

Applying this to acceleration gives

$$ \mathbf{a} = \td{\mathbf{u}}{t} = \pd{\mathbf{u}}{t} + \mathbf{u} \cdot \nabla \mathbf{u} \ .$$

### Rotating Reference Frame

Newton's second law only applies with an [inertial frame of reference](https://en.wikipedia.org/wiki/Inertial_frame_of_reference).
Yet we make our observations of the ocean from on the rotating Earth, most definitely a non-inertial reference frame.
This complicates matters significantly.
In order to relate the velocity and acceleration we actually measure to the velocity in the inertial reference frame, we need to understand how to transform vectors from a rotating to a non-rotating reference frame.

_[Dennis Nilsson CC BY 3.0 license, via Wikimedia Commons](https://upload.wikimedia.org/wikipedia/commons/6/61/AxialTiltObliquity.png)_

The rotating reference frame rotates at rotation rate $\mathbf{\Omega}$ relative to the inertial frame.
$\mathbf{\Omega}$ is a vector which points along the axis of rotation, following the right-hand rule. The magnitude $|\mathbf{\Omega}|$, measured in radians / s, gives the rotation rate. For Earth $|\mathbf{\Omega}| \simeq 7.29 \times 10^{-5}$ s$^{-1}$.

Let $\mathbf{v}$ denote an arbirary vector and $( d \mathbf{v} / dt )_R$ its rate of change measured in the rotating reference frame. The rate of change of this vector in the inertial reference frame is given by

$$ \left( \od{\mathbf{v}}{t} \right)_I = \left( \od{\mathbf{v}}{t} \right)_R + \mathbf{\Omega} \times \mathbf{v} $$

The second term on the RHS, the cross product between $\mathbf{\Omega}$ and $\mathbf{v}$, is due to the motion of the rotating reference frame relative to the inertial reference frame.

We now apply this rule to velocity and acceleration. Our goal is to arrive an an expression for the acceleration in the inertial frame, but we start with the velocity. The velocity in the inertial frame is given by

$$ \begin{align}
\mathbf{u}_I = \left( \od{\mathbf{x}}{t} \right)_I &= \left( \od{\mathbf{x}}{t} \right)_R + \mathbf{\Omega} \times \mathbf{x} \\
&= \mathbf{u}_R + \mathbf{\Omega} \times \mathbf{x}
\end{align} $$

The acceleration in the inertial frame is given by the rate of change of velocity in the inertial frame:

$$ \begin{align}
\mathbf{a}_I = \left( \od{\mathbf{u}_I}{t} \right)_I &= \left( \od{\mathbf{u}_I}{t} \right)_R + \mathbf{\Omega} \times \mathbf{u}_I \\
&= \left( \od{\mathbf{u}_R}{t} \right)_R + \mathbf{\Omega} \times \mathbf{u}_R
   + \mathbf{\Omega} \times \mathbf{u}_R + \mathbf{\Omega} \times \mathbf{\Omega} \times \mathbf{x} \\
&= \mathbf{a}_R + 2 \mathbf{\Omega} \times \mathbf{u}_R + \mathbf{\Omega} \times \mathbf{\Omega} \times \mathbf{x}
\end{align} $$

The derivation above shows that, when measuring acceleration in the rotating frame, we need to add two additional terms (both proportional to $\mathbf{\Omega}$) in order to recover the acceleration in the inertial frame (which is what we need for Newton's second law).

### Centrifugal Acceleration / Force

The term $\mathbf{\Omega} \times \mathbf{\Omega} \times \mathbf{x}$ is called the centrifugal acceleration. It is a function of position only. Specifically, it is a function of the distance from the Earth's rotation axis $r$, and it points outward away from the rotation axis:

$$ \mathbf{\Omega} \times \mathbf{\Omega} \times \mathbf{x}
= |\mathbf{\Omega}|^2 r \hat{\mathbf{r}}
= |\mathbf{\Omega}|^2 a \cos(\varphi) $$

where $a$ is the Earth's radius and $\varphi$ is latitude. It is common pracise to move the centrifugual acceleration to the right hand side of equation of motion and call it the centrifugal force. It is a "fictious" force that is felt by observers in the rotating reference frame. We define this force as

$$ \mathbf{f}_{c} = - \mathbf{\Omega} \times \mathbf{\Omega} \times \mathbf{x} = - \nabla \Phi_{cent} $$

where

$$ \Phi_{c} = \frac{|\mathbf{\Omega}|^2 a^2 \cos^2( \varphi)}{2} \ .$$

We combine it with the gravitational potential to form a single "effective" geopotential:

$$ \Phi = \Phi_g + \Phi_{c} \ .$$

Because Earth's surface naturally conforms to this geopotential, we don't have to worry about it in practice. We simply treat $\Phi$ as the gravitational field.

### Coriolis Acceleration

The term $2 \mathbf{\Omega} \times \mathbf{u}_R$ is called the [Coriolis acceleration](https://en.wikipedia.org/wiki/Coriolis_force) (or, on the right hand side of the equation, the Coriolis force).
It can be interpreted as another fictious force that is felt in the non-rotating reference frame.

It is instructive to look at the equation of motion including only the acceleration and coriolis terms:

$$ \od{\mathbf{u}}{t} = - 2 \Omega \times \mathbf{u} \ .$$

(We have dropped the subscript $R$; everything is assumed to be measured in the rotating reference frame.) This equation tells us that the fluid will be accelerated in a direction perpendicular to it's velocity. A fluid parcel that obeys this equation would appear to move in a straight line in the inertial frame, but follows a curved path in the rotating frame. This sort of motion is called an _inertial oscillation_.

## Putting it all together

The full equation of motion, in the rotating reference frame, can now be written as

$$ \rho \left ( \td{\mathbf{u}}{t} + 2 \mathbf{\Omega} \times \mathbf{u} \right )
= - \nabla p - \rho \nabla \Phi + \mu \nabla^2 \mathbf{u} \ . $$

It is common to divide through by $\rho$ to obtain the form

$$ \td{\mathbf{u}}{t} + 2 \mathbf{\Omega} \times \mathbf{u}
= - \frac{1}{\rho} \nabla p - \nabla \Phi + \nu \nabla^2 \mathbf{u} $$

where $\nu = \mu / \rho$ is called the _kinematic viscosity_.

## Common Approximations

The full equations of motion are hard to solve and, in many cases, unnecessarily comlex for the study of large-scale ocean circulation. Below we review some of the common approximations used to simplify the equations.

### Hydrostatic Balance

The vertical component of the equation of motion is dominated by the effect of gravity. The dominant balance is between the gravitational force and the pressure gradient for;  this is called _hydrostatic balance_:

$$ \pd{p}{z} = -\rho g $$

where $g = | \nabla \Phi |$ is the combined gravitational and centrifugal acceleration coefficient.

### Boussinesq Approximation

We have already seen the Boussinesq approximation, which states that density variations $\delta \rho$ are very small compared to the background density $\rho_0$, i.e.

$$ \rho = \rho _0 + \delta \rho \ ; \ \ |\delta \rho| \ll \rho_0 \ . $$

The background density $\rho_0$ is in hydrostatic balance with a background pressure gradient, such that

$$ \pd{}{z} p_{ref}(z)= - \rho_0 \pd{\Phi}{z} = - \rho_0 g $$

What remains of the pressure is called the _dynamic pressure_ $\phi$; it is usually normalized by $\rho_0$, for reasons that will become clear:

$$ \delta p = p - p_{ref} \ ; \ \ \ \phi = \frac{\delta p}{\rho_0} $$

We now examine what happens when we apply the boussinesq approximation to the equations of motion. The left-hand side becomes

$$ (\rho_0 + \delta \rho) \left ( \td{\mathbf{u}}{t} + 2 \mathbf{\Omega} \times \mathbf{u} \right ) \ . $$

Because $\rho_0$ is so much greater than $\delta \rho$, we simply drop the $\delta \rho term to obtain

$$ \rho_0 \left ( \td{\mathbf{u}}{t} + 2 \mathbf{\Omega} \times \mathbf{u} \right ) \ .$$

We will divide the whole equation through by $\rho_0$ to remove it from the left-hand side.
Let's now consider together the pressure and gravity terms on the right-hand side:

$$ - \frac{1}{\rho_0} \nabla (p_{ref} + \delta p) - \frac{\rho_0 + \delta \rho}{\rho_0} \nabla \Phi \ . $$

Because of how we defined $p_{ref}$, it cancels with the gravitational term proportional to $\rho_0$, leaving just

$$ - \frac{1}{\rho_0} \nabla (\delta p) - \frac{\delta \rho}{\rho_0} \nabla \Phi \ . $$

Using our definition of $\phi = \delta p / \rho_0$, and defining the _buoyancy_ as

$$ b = - g\frac{\delta \rho}{\rho_0} \ , $$

we can rewrite these terms as

$$ - \nabla \phi + b \hat{\mathbf{z}} \ . $$

As for the viscous term, we simply approximate $\nu \simeq \mu / \rho_0$, leading to the final form of the Boussinesq equations of motion:

$$ \td{\mathbf{u}}{t} + 2 \mathbf{\Omega} \times \mathbf{u}
= - \nabla \phi + b \hat{\mathcal{z}} + \nu \nabla^2 \mathbf{u} \ . $$

This is the equation we will use for our study of dynamical physical oceanography.

### Coordinates and Approximations for the Sphere

For describing the global ocean circulation with the highest precision, one should work in spherical polar coordinates (longitude $\lambda$, latitude $\varphi$, and radial distace $r$). However, this turns out to be unnecessarily complicated for the things we will study in our class.

Instead, we will use a _local tangent plane approximation_, as illustrated in the figure below.

*By Mike1024 [Public domain], via [Wikimedia Commons](https://commons.wikimedia.org/wiki/File%3AECEF_ENU_Longitude_Latitude_relationships.svg)*

In this approximation, we assume we are looking at region small enough that the Earth appears locally flat, such that we can use cartesian geometry. This region is centered around position $\lambda_{ref}$, $\varphi_{ref}$.
The local cartesian coordinates are then defined as

| name | direction | velocity | local coord | relation to spherical coord |
|--|--|--|--|--|
| zonal | east | $u$ | $x$   | $(\lambda - \lambda_{ref}) \cos(\varphi) / a$ |
| meridional | north | $v$ | $y$ | $(\varphi - \varphi_{ref}) / a$ |
| vertical | up | $w$ | $z$     | $r - r_{ref} $ |

where $a$ is the Earth's radius.

As a further approximation, we only keep the components of the Coriolis force that act in the horizontal plane on the horizontal velocities. This turns out to be a very good approximation, due to the small aspect ratio of the ocean and the relative smallness of the vertical flow.

We can now write the Boussinesq equation of motion in component form:

$$ \begin{align}
\td{u}{t} - f v &= -\pd{\phi}{x} + \nu \nabla^2 u \\
\td{v}{t} + f u &= -\pd{\phi}{y} + \nu \nabla^2 v \\
\td{w}{t}       &= -\pd{\phi}{z} + b + \nu \nabla^2 w  \ .
\end{align} $$

We have introduced the _Coriolis parameter_ $f = 2 \Omega \sin(\varphi)$. The coriolis parameter varies from zero at the equator to its maximum value of $2 \Omega$ at the pole. This reflects the cross product in the full Coriolis acceleration ($2 \mathbf{\Omega} \times \mathbf{u} $). At the equator, the horizontal flow occurs in a plane parallel to the rotation vector and thus experiences no effective Coriolis acceleration. At the pole, the horizontal flow is completely perpendicular to the rotation vector.

```python

```


```python
import numpy
from matplotlib import pyplot
%matplotlib inline
pyplot.rcParams['font.family'] = 'serif'
pyplot.rcParams['font.size'] = 16
```

## Question 1.a)

```python
# set parameters
L = 1.0
pi = numpy.pi
dx = 0.01 # x resolution
N_x = int(L/dx) + 1 # number of time steps
x = numpy.linspace(0.0, L, num=N_x)
T_compute = numpy.zeros((5, N_x)) # store 5 different resolutions results
terms_grid = numpy.array([1, 2, 5, 100, 1000])
```

```python
i = 0 # index for (i+1)-th grid resolution
T_compute = numpy.zeros((5, N_x)) # clear
for j in terms_grid:
    for n in range(1,j+1): # calculate summation
        T_compute[i] = T_compute[i] + (-1) ** (n + 1) / (2 * n - 1) * \
                        numpy.cos((2 * n - 1) * pi * x / (2 * L))
    T_compute[i] = T_compute[i] * 4 / pi
    i = i + 1
```

```python
# plot
pyplot.figure(figsize=(10.0, 4.0))
pyplot.title('Initial condition value under \ndifferent series terms at \nspatial resolution: {}'.format(dx))
pyplot.xlabel('x')
pyplot.ylabel(r'$ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $')
pyplot.grid()
pyplot.plot(x, T_compute[0], label=r'$ \sum_{n=1}^{1}$',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[1], label=r'$ \sum_{n=1}^{2}$',
            color='C1', linestyle='--', linewidth=2)
pyplot.plot(x, T_compute[2], label=r'$ \sum_{n=1}^{5}$',
            color='C2', linestyle=':', linewidth=2)
pyplot.plot(x, T_compute[3], label=r'$ \sum_{n=1}^{100}$',
            color='C3', linestyle='-.', linewidth=2)
pyplot.plot(x, T_compute[4], label=r'$ \sum_{n=1}^{1000}$',
            color='C4', linestyle='-', linewidth=2)
pyplot.legend(loc='center left', bbox_to_anchor=(1, 0.5));
```

### Peformance of Different Number of Series Terms

As one should know from initial condition, the $T^{(0)}(x)$ is a consant, which means the value of $ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $ would be euqal to 1.

Therefore, by looking through all curves above, with the number of series terms increasing, the value is more closer to constant 1.

```python
# if expand to (-10 , 10)
# set parameters
x_begin = -10.0
x_end = 10.0
L = x_end - x_begin
pi = numpy.pi
dx = 0.01 # x resolution
N_x = int(L/dx) + 1 # number of time steps
x = numpy.linspace(x_begin, x_end, num=N_x)
T_compute = numpy.zeros((5, N_x)) # store 5 different resolutions results
terms_grid = numpy.array([1, 2, 5, 100, 1000])

i = 0 # index for (i+1)-th grid resolution
T_compute = numpy.zeros((5, N_x)) # clear
for j in terms_grid:
    for n in range(1,j+1): # calculate summation
        T_compute[i] = T_compute[i] + (-1) ** (n + 1) / (2 * n - 1) * \
                        numpy.cos((2 * n - 1) * pi * x / (2 * L))
    T_compute[i] = T_compute[i] * 4 / pi
    i = i + 1

# plot
pyplot.figure(figsize=(10.0, 4.0))
pyplot.title('Initial condition value under \n\
different series terms at \nspatial resolution: dx = {}, time t = 0.0'.format(dx))
pyplot.xlabel('x')
pyplot.ylabel(r'$ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $')
pyplot.grid()
pyplot.plot(x, T_compute[0], label=r'$ \sum_{n=1}^{1}$',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[1], label=r'$ \sum_{n=1}^{2}$',
            color='C1', linestyle='--', linewidth=2)
pyplot.plot(x, T_compute[2], label=r'$ \sum_{n=1}^{5}$',
            color='C2', linestyle=':', linewidth=2)
pyplot.plot(x, T_compute[3], label=r'$ \sum_{n=1}^{100}$',
            color='C3', linestyle='-.', linewidth=2)
pyplot.plot(x, T_compute[4], label=r'$ \sum_{n=1}^{1000}$',
            color='C4', linestyle='-', linewidth=2)
pyplot.legend(loc='center left', bbox_to_anchor=(1, 0.5));
```

The period is the suumation of different amplitude and period cosine function. With the number of series increasing, the value of each point is converging to the 1.0, which is the ideal value.

## Question 1.b)

```python
# set parameters
t_grid = numpy.array([0.1, 0.5, 1.0, 10.0])
D = 1.0
L = 1.0
terms_grid = 100
pi = numpy.pi
dx = 0.01
N_x = int(L/dx) + 1
x = numpy.linspace(0.0, L, N_x)
T_compute = numpy.zeros((4, N_x)) # store 4 different resolutions results
```

```python
i = 0
T_compute = numpy.zeros((4, N_x)) # clear
for t in t_grid:
    for n in range(1, terms_grid+1):
        eigen_value = (2 * n - 1) * pi / (2 * L)
        T_compute[i] = T_compute[i] + (-1) ** (n + 1) / (2 * n - 1) * \
                        numpy.cos(eigen_value * x) * \
                        numpy.exp(-D * t * (eigen_value ** 2))
    T_compute[i] = T_compute[i] * 4 / pi
    i = i + 1
```

```python
# plot
pyplot.figure(figsize=(10.0, 4.0))
pyplot.title('Difference between initial and wall temperature \
\n decrease with time alone with the L at \nspatial resolution: dx = {}'.format(dx))
pyplot.xlabel('x')
pyplot.ylabel(r'$ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $')
pyplot.grid()
pyplot.plot(x, T_compute[0], label=r'$t = 0.1$',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[1], label=r'$t = 0.5$',
            color='C1', linestyle='--', linewidth=2)
pyplot.plot(x, T_compute[2], label=r'$t = 1.0$',
            color='C2', linestyle=':', linewidth=2)
pyplot.plot(x, T_compute[3], label=r'$t = 10.0$',
            color='C3', linestyle='-.', linewidth=2)
pyplot.legend(loc='center left', bbox_to_anchor=(1, 0.5));
```

### The Curve Varies with Time

As one should know from initial condition, the $T^{(0)}(x) = T^{(0)} + \frac{T_L - T^{(0)}}{L}x $, which means the value of $ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $ would be simplified:

$$
\begin{equation}
\frac{T(x,0) - T_L}{T^{(0)} - T_L} = \frac{T^{(0)}}{T^{(0)} - T_L} - \frac{1}{L}x
\end{equation}
$$

which is a slope that is equal to $-\frac{1}{L}$ line, starting from $\frac{T^{(0)}}{T^{(0)} - T_L}$, which indicates the line at $t = 0.1$ is about correct.

Also, with time flows, the exponential terms will eventually goes close to zero, which indicates the temperature in the space is eventually equal to the wall temperature.

## Question 1.c)

```python
# set parameters
x_begin = 0.0
x_end = 1.0
L = x_end - x_begin
pi = numpy.pi
dx = 0.01 # x resolution
N_x = int(L/dx) + 1 # number of time steps
x = numpy.linspace(x_begin, x_end, num=N_x)
T_compute = numpy.zeros((6, N_x)) # store 6 different resolutions results
terms_grid = numpy.array([1, 2, 5, 10, 100, 1000])

i = 0 # index for (i+1)-th grid resolution
T_compute = numpy.zeros((6, N_x)) # clear
for j in terms_grid:
    for n in range(1,j+1): # calculate summation
        T_compute[i] = T_compute[i] + (1 / ((2 * n -1) ** 2)) * \
                        numpy.cos((2 * n - 1) * pi * x / (2 * L))
    T_compute[i] = T_compute[i] * 8 / (pi ** 2)
    i = i + 1

# plot
pyplot.figure(figsize=(7.0, 7.0))
pyplot.title('Linearly varying initial condition value under \n\
different series terms at \nspatial resolution: dx = {}, time t = 0.0'.format(dx))
pyplot.xlabel('x')
pyplot.ylabel(r'$ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $')
pyplot.grid()
pyplot.plot(x, T_compute[0], label=r'$ \sum_{n=1}^{1}$',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[1], label=r'$ \sum_{n=1}^{2}$',
            color='C1', linestyle='--', linewidth=2)
pyplot.plot(x, T_compute[2], label=r'$ \sum_{n=1}^{5}$',
            color='C2', linestyle=':', linewidth=2)
pyplot.plot(x, T_compute[3], label=r'$ \sum_{n=1}^{10}$',
            color='C3', linestyle='-.', linewidth=2)
pyplot.plot(x, T_compute[4], label=r'$ \sum_{n=1}^{100}$',
            color='C4', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[5], label=r'$ \sum_{n=1}^{1000}$',
            color='C5', linestyle='--', linewidth=2)
pyplot.legend(loc='center left', bbox_to_anchor=(1, 0.5));
```

### Peformance of Different Number of Series Terms

As one should know from initial condition, the $T^{(0)}(x) = T^{(0)} + \frac{T_L - T^{(0)}}{L}x $, which means the value of $ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $ would be simplified:

$$
\begin{equation}
\frac{T(x,0) - T_L}{T^{(0)} - T_L} = \frac{T^{(0)}}{T^{(0)} - T_L} - \frac{1}{L}x
\end{equation}
$$

which is a slope that is equal to $-\frac{1}{L}$ line, in this case L = 1, starting from $\frac{T^{(0)}}{T^{(0)} - T_L}$.

Therefore, if one could look through the curves, with the number of series terms increasing, the curve is closer to the ideal line.

## Question 1.d)

```python
# set parameters
t_grid = numpy.array([0.1, 0.5, 1.0, 10.0])
D = 1.0
L = 1.0
terms_grid = 100
pi = numpy.pi
dx = 0.01
N_x = int(L/dx) + 1
x = numpy.linspace(0.0, L, N_x)
T_compute = numpy.zeros((4, N_x)) # store 4 different resolutions results

# compute
i = 0
T_compute = numpy.zeros((4, N_x)) # clear
for t in t_grid:
    for n in range(1, terms_grid+1):
        eigen_value = (2 * n - 1) * pi / (2 * L)
        T_compute[i] = T_compute[i] + (1 / ((2 * n -1) ** 2)) * \
                        numpy.cos(eigen_value * x) * \
                        numpy.exp(-D * t * (eigen_value ** 2))
    T_compute[i] = T_compute[i] * 8 / (pi ** 2)
    i = i + 1

# plot
pyplot.figure(figsize=(6.0, 6.0))
pyplot.title('Difference between initial and wall temperature \
\n decrease with time alone with the L at \nspatial resolution: dx = {}'.format(dx))
pyplot.xlabel('x')
pyplot.ylabel(r'$ \frac{T(x,0) - T_L}{T^{(0)} - T_L} $')
pyplot.grid()
pyplot.plot(x, T_compute[0], label=r'$t = 0.1$',
            color='C0', linestyle='-', linewidth=2)
pyplot.plot(x, T_compute[1], label=r'$t = 0.5$',
            color='C1', linestyle='--', linewidth=2)
pyplot.plot(x, T_compute[2], label=r'$t = 1.0$',
            color='C2', linestyle=':', linewidth=2)
pyplot.plot(x, T_compute[3], label=r'$t = 10.0$',
            color='C3', linestyle='-.', linewidth=2)
pyplot.legend(loc='center left', bbox_to_anchor=(1, 0.5));
```

### The Curve Varies with Time

Similar with Q1.b, with time flows, the exponential terms will eventually goes close to zero, which indicates the temperature in the space is eventually equal to the wall temperature.


Contenido bajo licencia Creative Commons BY 4.0 y código bajo licencia MIT. © Juan Gómez y Nicolás Guarín-Zapata 2020. Este material es parte del curso Modelación Computacional en el programa de Ingeniería Civil de la Universidad EAFIT.

# Cerchas planas

## Introducción

En este Notebook se parte del programa de resortes simple discutido anteriormente y tras realizar cambios menores en algunos parámetros este se modifica para que resuelva estructuras correspondientes a cerchas planas. Además se introducen los conceptos de sistemas de referencia locales a los elementos y global para la estructura así como la relación entre ambos.

**Al completar este notebook usted debería estar en la capacidad de:**

* Entender el algoritmo de solución de estructuras como un proceso general de ensamblaje de rigideces en un sistema global de ecuaciones.

* Reconocer la diferencia entre los sistemas de referencia local (propio de cada elemento) y global (para toda la estructura) y la necesidad de expresar las rigideces en único sistema de referencia.

* Expresar relaciones fuerza-desplazamiento en los sistemas de referencia local y global.

* Reconocer las modificaciones necesarias para convertir un programa fundamental de ensamblaje de resortes en uno para estructuras mas complejas.

### Ensamblaje de elementos cercha

En lo que sigue se usarán las siguientes condiciones.

* Se asumirán estructuras correspondientes a ensamblajes de elementos barra conectados por articulaciones en sus uniones.

* Por sus condiciones geométricas se asume que los elementos solo tienen rigidez axial y que por ende estos pueden entenderse como equivalentes a resortes de rigidez $k=\frac{AE}{l}$.

La siguiente figura muestra un elemento típico. El eje de referencia $x$ dispuesto en la dirección longitudinal del elemento representa el sistema **local** de referencia. Nótese que si se estudia el elemento en sus sistema local este es completamente equivalente al resorte discutido anteriormente.

Usando $k = \frac{AE}{l}$ se tiene que la relación fuerza-desplazamiento en el sistema local de referencia es de la forma:

$$
\begin{Bmatrix}f_1\\f_2\end{Bmatrix} = \frac{AE}{l}
\begin{bmatrix}
1&-1\\
-1&1\end{bmatrix}
\begin{Bmatrix}u_1\\u_2\end{Bmatrix}\, .
$$

Considere ahora la siguiente estructura o ensamblaje conformado por 2 elementos tipo barra. Asumiendo que se conocen las relaciones fuerza-desplazamiento para cada una de las barras de la cercha se requiere determinar el desplazamiento del vértice superior de la cercha y las fuerzas internas en los elementos.

Nótese que además del sistema de referencia local de cada elemento ahora se tiene un sistema de referencia único denotado como $X-Y$ y denominado el sistema de referencia **Global**.

Para obtener la rigidez total de la estructura es necesario considerar la contribución de todos los elementos en el sistema de referencia de la cercha. Será necesario entonces transformar la matriz de rigidez de cada elemento del sistema local al sistema común o global. Para esto es conveniente definir:

* $U, F$ : Desplazamientos (o grados de libertad) y fuerzas en el sistema de referencia global.

* $u, f$ : Desplazamientos (o grados de libertad) y fuerzas en el sistema de referencia local.

Estas variables se relacionan mediante la matriz de rotación $\lambda$ de acuerdo con;

$$u=\lambda U\, .$$

Suponiendo que aplicamos ahora un desplazamiento $\delta u$ y determinamos el trabajo de las fuerzas a lo largo de este desplazamiento de acuerdo con:

$$
W = \delta u^T f\, .
$$

Ahora, dado que el trabajo es una cantidad escalar y por lo tanto independiente del sistema de referencia es válido escribir:

$$W = \delta U^T F=\delta u^T f\, .$$

Usando la ecuación de transformación bajo rotación en esta última se obtiene:

$$\delta U^T F=\delta U^T \lambda^T f$$

de donde se concluye que

$$F =\lambda^T f\, .$$

Si ahora se consideran las relaciones fuerza-desplazamiento

$$f = ku\, ,$$

donde $k$ es la matriz de rigidez local se tiene que podemos escribir:

\begin{align}
&\lambda^Tf = \lambda^T k u\\
&\lambda^Tf = \lambda^T k \lambda U\\
&F = KU
\end{align}

de donde se obtiene la ecuación de transformación bajo rotación de las relaciones fuerza-desplazamiento en un sistema de referencia local y un sistema global

$$K=\lambda^T k\lambda\, .$$

Es importante observar que en el sistema global el elemento parece tener 2 grados de libertad por nodo mientras que en el sistema local solo existe el desplazamiento axial.

La información requerida para calcular $K$ es entregada como parámetros de entrada a la función `uel` como se describe a continuación.

Encuentre la matriz de transformación bajo rotación $\lambda$ requerida para la formulación de la matriz de rigidez en el sistema de referencia global.

### Ejemplo

Las modificaciones que se deben aplicar al programa de resortes son aquellas relacionadas con el hecho de tener 2 grados de libertad por nodo.

(Los archivos de texto con los datos de entrada para las partículas, elementos, coeficientes de rigidez y cargas de este problema están almacenados en la carpeta `files` de este repositorio).

```python
import numpy as np
```

Lea los archivos de entrada de la carpeta `files`.

```python
def readin():
    nodes = np.loadtxt('files/Cnodes.txt', ndmin=2)
    mats = np.loadtxt('files/Cmater.txt', ndmin=2)
    elements = np.loadtxt('files/Celes.txt', ndmin=2)
    loads = np.loadtxt('files/Cloads.txt', ndmin=2)
    return nodes, mats, elements, loads
```

La subrutina `eqcounter` cuenta ecuaciones y genera el arreglo de condiciones de frontera.

```python
def eqcounter(nodes):
    nnodes = nodes.shape[0]
    IBC = np.zeros((nnodes, 2), dtype=np.integer)
    for node in range(nnodes):
        for dof in range(2):
            IBC[node, dof] = int(nodes[node, dof + 3])
    neq = 0
    for node in range(nnodes):
        for dof in range(2):
            if IBC[node, dof] == 0:
                IBC[node, dof] = neq
                neq = neq + 1
    return neq, IBC
```

Ahora la función `DME` calcula la matriz de ensamblaje de ecuaciones.

```python
def DME(nodes, elements):
    nels = elements.shape[0]
    IELCON = np.zeros((nels, 2), dtype=np.integer)
    DME = np.zeros((nels, 4), dtype=np.integer)
    neq, IBC = eqcounter(nodes)
    ndof = 4
    nnodes = 2
    for ele in range(nels):
        for node in range(nnodes):
            IELCON[ele, node] = elements[ele, node + 3]
            glob_num = IELCON[ele, node]
            for loc_num in range(2):
                DME[ele, 2*node + loc_num] = IBC[glob_num, loc_num]
    return DME, IBC, neq
```

La función `assembly` usa el modelo y la matriz `DME` para calcular la matriz de rigidez global.

```python
def assembly(elements, mats, nodes, neq, DME):
    IELCON = np.zeros((2), dtype=np.integer)
    KG = np.zeros((neq, neq))
    nels = elements.shape[0]
    nnodes = 2
    ndof = 4
    for el in range(nels):
        elcoor = np.zeros((nnodes, 2))
        im = np.int(elements[el , 2])
        par0 = mats[im, 0]
        par1 = mats[im, 1]
        for j in range(nnodes):
            IELCON[j] = elements[el , j+3]
            elcoor[j , 0] = nodes[IELCON[j], 1]
            elcoor[j , 1] = nodes[IELCON[j], 2]
        kloc = ueltruss2D(elcoor, par0, par1)
        dme = DME[el, :ndof]
        for row in range(ndof):
            glob_row = dme[row]
            if glob_row != -1:
                for col in range(ndof):
                    glob_col = dme[col]
                    if glob_col != -1:
                        KG[glob_row, glob_col] = KG[glob_row, glob_col] +\
                                                 kloc[row, col]
    return KG
```

La rutina `ueltruss2D` usa las coordenadas de los nudos y los parámetros de material para calcular la matriz de rigidez local ya transformada al sistema de referencia global.

Agregue un comentario a cada línea relevante de los códigos de las siguientes rutinas y úselos para escribir los pseudocódigos correspondientes. En particular identifique el calculo de la matriz de transformación bajo rotación $\lambda$.

```python
def ueltruss2D(coord, A, Emod):
    vec = coord[1, :] - coord[0, :]
    length = np.linalg.norm(vec)
    nx = vec[0]/length
    ny = vec[1]/length
    Q = np.array([
        [nx, ny , 0 , 0],
        [0,  0, nx , ny]])
    kl = (A*Emod/length) * np.array([
        [1, -1],
        [-1, 1]])
    kG = Q.T @ kl @ Q
    return kG
```

La rutina `loadassem` forma el vector de cargas en los nudos.

```python
def loadasem(loads, IBC, neq, nl):
    RHSG = np.zeros([neq])
    for cont in range(nl):
        il = int(loads[cont, 0])
        ilx = IBC[il , 0]
        ily = IBC[il , 1]
        if ilx != -1:
            RHSG[ilx] = loads[cont, 1]
        if ily != -1:
            RHSG[ily] = loads[cont, 2]
    return RHSG
```

El programa principal mantiene la misma estructura que el programa de resortes, es decir, se tienen siguientes pasos:

* Lee el modelo.

* Determina la matriz de ensamblaje `DME`.

* Ensambla el sistema global de ecuaciones.

* Determina los desplazamientos globales `UG` tras resolver el sistema global.

```python
nodes, mats, elements, loads = readin()
DME, IBC, neq = DME(nodes, elements)
KG = assembly(elements, mats, nodes, neq, DME)
RHSG = loadasem(loads, IBC, neq, 1)
UG = np.linalg.solve(KG, RHSG)
print(UG)
```

    [ 0.         -0.15018948]

### Problemas propuestos

#### Problema 1

Implemente una función que calcule las fuerzas nodales en cada elemento y que verifique el equilibrio del sistema.

#### Problema 2

Determine la rigidez lateral de la estructura usando la relación:

$$k = \frac{P}{\delta}\, .$$

#### Problema 3

Introduzca las modificaciones necesarias a los archivos de entrada para adicionar una barra adicional al modelo y que conecte los nodos $0$ y $1$. Luego use este nuevo modelo para determinar los desplazamientos. Comente los resultados.

#### Problema 4

Repare la cercha mostrada en la figura adicionando elementos o imponiendo restricciones apropiadas a los desplazamientos. (Cree un nuevo paquete de archivos de datos).

## Referencias

* Klaus-Jürgen Bathe (2006). Finite element procedures. Klaus-Jurgen Bathe. Prentice Hall International.

* Juan Gómez, Nicolás Guarín-Zapata (2018). SolidsPy: 2D-Finite Element Analysis with Python, <https://github.com/AppliedMechanics-EAFIT/SolidsPy>.

## Formato del notebook

La siguiente celda cambia el formato del Notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('./nb_style.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>

```python

```


# Heat Equation with pyGPs

Assumptions:

$\mathcal{L}_{\bar{x}}^{\phi}u(\bar{x}) = f(\bar{x})$

$u(\bar{x}) \sim \mathcal{GP}(0, k_{uu}(\bar{x}, \bar{x}', \theta))$

$f(\bar{x}) \sim \mathcal{GP}(0, k_{ff}(\bar{x}, \bar{x}', \theta, \phi))$

$\theta = \{\sigma, l\}$

Chosen operator:
$\mathcal{L}_{\bar{x}}^{\phi}u(\bar{x}) = \frac{\partial}{\partial t}u(\bar{x}) - \phi \frac{\partial^2}{\partial x^2}u(\bar{x})$, where $\bar{x} = (t, x) \in \mathbb{R}^2$

Problem at hand: Given $\{X_u, y_u\}$ and $\{X_f, y_f\}$, estimate $\phi$.

We employ a GP with a RBF kernel for u and f:

$k_{uu}(\bar{x}_i, \bar{x}_j; \theta_u) = \sigma_u^2 \exp(-\frac{1}{2l_u^2}\Vert \bar{x}_i - \bar{x}_j \Vert_2^2)$

$k_{ff}(\bar{x}_i, \bar{x}_j; \theta_f) = \sigma_f^2 \exp(-\frac{1}{2l_f^2}\Vert \bar{x}_i - \bar{x}_j\Vert_2^2) $

We use the known transformation behavior of Gaussian Processes:

$k_{ff}(\bar{x}_i, \bar{x}_j; \theta, \phi) = \mathcal{L}_{\bar{x}_i}^{\phi}\mathcal{L}_{\bar{x}_j}^{\phi}k_{uu}(\bar{x}_i, \bar{x}_j; \theta)$

Looking at an entry of the covariance matrix, where $\bar{x}_i = \bar{x}_j$ (a diagonal element), we can calculate that
$\phi = ((\sigma_f^2 - \frac{\sigma_u^2}{l_u})\frac{l_u^2}{3 \sigma_u^2})^{\frac{1}{2}}$.

```python
import numpy as np
import sympy as sp
import pyGPs

# Linear functional L = \frac{\partial}{\partial t}*u(t, x) - \phi \frac{\partial^2}{\partial x^2}*u(t, x) was chosen.
# Desired result: phi = 12.0.

# In general we want to arrange the t's and x's as: [[(t_0, x_0), (t_1, x_0), (t_2, x_0), ..., (t_n, x_0)][(t_0, x_1), ...], ...]
# We're setting u(t,x) = 2*x^2 + xt

# Generating data
x_u = np.linspace(0.2, 2*np.pi, 15)
t_u = np.linspace(0.2, 2*np.pi, 15)
y_u = (2.*x_u**2 + np.outer(t_u, x_u)).T             # Keeping it as simple as possible, with sin instead of sqrt the optimizer can't
                                                     # calculate the optimal hyperparameters, independent of the method
x_f = np.linspace(0.2, 2*np.pi, 15)
t_f = np.linspace(0.2, 2*np.pi, 15)
y_f = x_f - 48

# The function u is assumed to be a Gaussian Process.
# After a linear transformation, f has to be a Gaussian Process as well.

# Need to prepare data first
M_u = t_u.size
N_u = x_u.size
M_f = t_f.size
N_f = x_f.size

# Output data in an array
y_u.resize(M_u*N_u, 1)

# Input data in an array with two-dimensional entries
A_u = np.zeros((M_u*N_u, 2))
count = 0
for j in range(M_u):
    for i in range(N_u):
        A_u[i+j+count] = (t_u[j], x_u[i])
    count += M_u - 1

# Normally not needed
y_f = np.repeat(y_f, M_f)
y_f.resize((N_f, M_f))

# Output data in an array
y_f.resize(M_f*N_f, 1)

# Input data in an array with two-dimensional entries
A_f = np.zeros((M_f*N_f, 2))
count = 0
for i in range(N_f):
    for j in range(M_f):
        A_f[i+j+count] = (t_f[j], x_f[i])
    count += M_f - 1

model_u = pyGPs.GPR()
model_u.setData(A_u, y_u)
model_u.optimize(A_u, y_u)

model_f = pyGPs.GPR()
model_f.setData(A_f, y_f)
model_f.optimize(A_f, y_f)

# Note that in hyp only the logarithm of the hyperparameter is stored!
# Characteristic length-scale is equal to np.exp(hyp[0]) (Default: 1)
# Signal variance is equal to np.exp(hyp[1]) (Default: 1)

# Calculating k_ff using differentiation tools from sympy and inserting our optimal parameters.

# Declaring all the variables we need
x_i, x_j, t_i, t_j, sig_u, l_u, phi, sig_f = sp.symbols('x_i x_j t_i t_j sig_u l_u phi sig_f')
# Defining k_uu
k_uu = sig_u**2*sp.exp(-1/(2*l_u)*((x_j - x_i)**2 + (t_j - t_i)**2))
# Calculating k_ff by applying the linear transformation twice
k_ff = sp.diff(k_uu, t_j, t_i) - phi*sp.diff(k_uu, x_i, x_i, t_j) - phi*sp.diff(k_uu, t_i, x_j, x_j) + phi**2*sp.diff(k_uu, x_i, x_i, x_j, x_j)
# The square-root of the result is equal to sig_f. We substitute t_i for t_j and x_i for x_j (diagonal entry of the covariance matrix)
eqn = sp.Eq(sig_f, sp.sqrt(k_ff.subs({t_i:t_j, x_i:x_j})))
# We solve w.r.t. phi
res = sp.solve(eqn, phi)
# In our case in res[1] the positive solution of the square root is contained
phi_res = res[1].subs({l_u:np.exp(model_u.covfunc.hyp[0]), sig_u:np.exp(model_u.covfunc.hyp[1]), sig_f:np.exp(model_f.covfunc.hyp[1])})

print(phi_res.evalf())	# This should give 12 as output

# My Output: 9.49196403727793

```

```python

```