# Evolutionary Game Theory

In the previous chapter, we considered the case of fitness being independant of the distribution of the whole population (the rates of increase of 1 type just depended on the quantity of that type). That was a specific case of Evolutionary game theory which considers **frequency dependent selection**.

---

## Frequency dependent selection

[Video](https://youtu.be/PFtwwrcouXY?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

Consider. Let $x=(x_1, x_2)$ correspond to the population sizes of both types. The fitness functions are given by:

$$f_1(x)\qquad f_2(x)$$

As before we ensure a constant population size: $x_1 + x_2 = 1$. We have:

$$
\frac{dx_1}{dt}=x_1(f_1(x)-\phi) \qquad \frac{dx_2}{dt}=x_2(f_2(x)-\phi)
$$

we again have:

$$
\frac{dx_1}{dt} + \frac{dx_2}{dt}=x_1(f_1(x)-\phi) + x_2(f_2(x)-\phi)=0
$$

So $\phi=x_1f_1(x)+x_2f_2(x)$ (the average fitness).

We can substitute: $x_2=1-x_1$ to obtain:

$$
\frac{dx_1}{dt}=x_1(f_1(x)-x_1f_1(x)-x_2f_2(x))=x_1((1-x_1)f_1(x)-(1-x_1)f_2(x))
$$

$$
\frac{dx_1}{dt}=x_1(1-x_1)(f_1(x)-f_2(x))
$$

We see that we have 3 equilibria:

- $x_1=0$
- $x_1=1$
- Whatever distribution of $x$ that ensures: $f_1(x)=f_2(x)$

---

## Evolutionary Game Theory

Now we will consider potential differences of these equilibria. First we will return to considering Normal form games:

$$
A =
\begin{pmatrix}
a & b\\
c & d
\end{pmatrix}
$$

Evolutionary Game theory assigns strategies as types in a population, and indivividuals randomly encounter other individuals and play their corresponding strategy. The matrix $A$ correspods to the utility of a row player in a game where the row player is a given individual and the column player is the population.

This gives:

$$f_1=ax_1+bx_2\qquad f_2=cx_1+dx_2$$

or equivalently:

$$f=Ax\qquad \phi=fx$$

thus we have the same equation as before but in matrix notation:

$$\frac{dx}{dt}=x(f-\phi)$$

---

In this case, the 3 stable distributions correspond to:

- An entire population playing the first strategy;
- An entire population playing the second strategy;
- A population playing a mixture of first and second (such that there is indifference between the fitness).

---

We now consider the utility of a stable population in a **mutated** population.

---

## Mutated population

Given a strategy vector $x=(x_1, x_2)$, some $\epsilon>0$ and another strategy $y=(y_1, y_2)$, the post entry population $x_{\epsilon}$ is given by:

$$
x_{\epsilon} = (x_1 + \epsilon(y_1 - x_1), x_2 + \epsilon(y_2 - x_2))
$$

---

## Evolutionary Stable Strategies

[Video](https://youtu.be/lbzcToUM9ic?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

Given a stable population distribution, $x$ it represents an **Evolutionary Stable Strategy** (ESS) if and only if there exists $\bar\epsilon>0$:

$$u(x, x_{\epsilon})>u(y, x_{\epsilon})\text{ for all }0<\epsilon<\bar\epsilon, y$$

where $u(x, y)$ corresponds to the fitness of strategy $x$ in population $y$ which is given by:

$$xAy^T$$

---

For the first type to be an ESS this corresponds to:

$$a(1-\epsilon)+b\epsilon > c(1-\epsilon) + d\epsilon$$

For small values of $\epsilon$ this corresponds to:

$$a>c$$

However if $a=c$, this corresponds to:

$$b>d$$

Thus the first strategy is an ESS (ie resists invasion) iff one of the two hold:

1. $a > c$
2. $a=c$ and $b > d$

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

from scipy.integrate import odeint

t = np.linspace(0, 10, 100)  # Obtain 100 time points

def dx(x, t, A):
    """
    Define the derivate of x.
    """
    f = np.dot(A, x)
    phi = np.dot(f, x)
    return x * (f - phi)
```

The case of $a>c$:

```python
A = np.array([[4, 3], [2, 1]])
epsilon = 10 ** -1
xs = odeint(func=dx, y0=[1 - epsilon, epsilon], t=t, args=(A,))
plt.plot(xs);
```

The case of $a=c$ and $b>d$:

```python
A = np.array([[4, 3], [4, 1]])
epsilon = 10 ** -1
xs = odeint(func=dx, y0=[1 - epsilon, epsilon], t=t, args=(A,))
plt.plot(xs);
```

$a=c$ and $b < d$:

```python
A = np.array([[4, 3], [4, 5]])
epsilon = 10 ** -1
xs = odeint(func=dx, y0=[1 - epsilon, epsilon], t=t, args=(A,))
plt.plot(xs);
```

$a < c$:

```python
A = np.array([[1, 3], [4, 1]])
epsilon = 10 ** -1
xs = odeint(func=dx, y0=[1 - epsilon, epsilon], t=t, args=(A,))
plt.plot(xs);
```

We see in the above case that the population seems to stabilise at a mixed strategy. This leads to the general definition of the fitness of a mixed strategy: $x=(x_1, x_2)$:

$$u(x,x) = x_1f_1(x)+x_2f_2(x)$$

---

## General condition for ESS

[Video](https://youtu.be/zkhInay5xQc?list=PLnC5h3PY-znxMsG0TRYGOyrnEO-QhVwLb)

If $x$ is an ESS, then for all $y\ne x$, either:

1. $u(x,x)>u(y,x)$
2. $u(x,x)=u(y,x)$ and $u(x,y)>u(y,y)$
---

Conversely, if either (1) or (2) holds for all $y\ne x$ then $x$ is an ESS.

---

### Proof

---

If $x$ is an ESS, then by definition:

$$u(x,x_{\epsilon})>u(y,x_{\epsilon})$$

which corresponds to:

$$(1-\epsilon)u(x,x)+\epsilon u(x,y)>(1-\epsilon)u(y,x)+\epsilon u(y,y)$$

- If condition 1 of the theorem holds then the above inequality can be satisfied for \\(\epsilon\\) sufficiently small. If condition 2 holds then the inequality is satisfied.
- Conversely:

    - If $u(x,x) < u(y,x)$ then we can find $\epsilon$ sufficiently small such that the inequality is violated.

    - If $u(x, x) = u(y,x)$ and $u(x,y) \leq u(y,y)$ then the inequality is violated.

This result gives us an efficient way of computing ESS. The first condition is in fact almost a condition for Nash Equilibrium (with a strict inequality), the second is thus a stronger condition that removes certain Nash equilibria from consideration. This becomes particularly relevant when considering Nash equilibrium in mixed strategies.

To find ESS in a pairwise context population game we:

1. Write down the associated two-player game $(A, A^T)\in{\mathbb{R}^{m\times n}}^2$;
2. Identify all symmetric Nash equilibria of the game;
3. Test the Nash equilibrium against the two conditions of the above Theorem.

Let us apply it to the one example that seemed to stabilise at a mixed strategy:

$$
A =\begin{pmatrix}
1 & 3\\
4 & 1
\end{pmatrix}
$$

```python
import nashpy as nash
game = nash.Game(A, A.transpose())
list(game.support_enumeration())
```

    [(array([1., 0.]), array([0., 1.])),
     (array([0., 1.]), array([1., 0.])),
     (array([0.4, 0.6]), array([0.4, 0.6]))]

Looking at $x=(.4, .6)$ (which is the only symmetric nash equilibrium), we have

$$u(x, x)=u(y, x)$$

and (recall $y_1 + y_2 = 1$):

$$
u(x, y)=2.8y_1 + 1.8y_2=2.8y_1 + 1.8(1-y_1)=y_1+1.8
$$

\begin{align}
u(y, y)&=y_1^2+3y_1y_2+4y_1y_2+y_2^2\\
       &=y_1^2+7y_1-7y_1^2+1 - 2y_1 + y_1^2\\
       &=5y_1-5y_1^2+1
\end{align}

Thus:

$$u(x, y) - u(y, y) = -4y_1+5y_1^2+.8 = 5(y_1 - .4)^2$$

however $y_1\ne.4$ thus $x=(.4, .6)$ is an ESS.

Here is some code to verify the above calculations:

```python
import sympy as sym
sym.init_printing()
A = sym.Matrix(A)
y_1, y_2 = sym.symbols("y_1, y_2")
y = sym.Matrix([y_1, y_2])
A, y
```

```python
rhs = sym.expand((y.transpose() * A * y)[0].subs({y_2: 1 - y_1}))
rhs
```

```python
lhs = sym.expand((sym.Matrix([[.4, .6]]) * A * y)[0].subs({y_2: 1-y_1}))
lhs
```

```python
sym.factor(lhs - rhs)
```


```python
import control
from control.matlab import *
import numpy as np
import scipy as sc
import sympy as sy
import matplotlib.pyplot as plt
```

```python
A = [[-5, -1],[3, -1]]
B = [[2],[5]]
C = [[1, 2]]
D = [[0]]

ss2tf(A, B, C ,D)
```

$$\frac{12 s + 59}{s^2 + 6 s + 8}$$

```python
p1 = np.array([12, 59])
p2 = np.array([1, 6, 8])
tf2ss(p1, p2)
```

    A = [[-6. -8.]
     [ 1.  0.]]

    B = [[1.]
     [0.]]

    C = [[12. 59.]]

    D = [[0.]]

# Traditional way (symbolic)

$$
\phi(s) = (s\mathbf{I} - \mathbf{A})^{-1}
$$
$$
H(s) = \frac{Y(s)}{U(s)} = \mathbf{C}\phi(s)\mathbf{B} + \mathbf{D}
$$

```python
_A = sy.Matrix(A)
_B = sy.Matrix(B)
_C = sy.Matrix(C)
_D = sy.Matrix(D)

s, t = sy.symbols('s t')

phi = (s*sy.eye(2) - _A).inv()
print("State Transition Matrix")
phi
```

    State Transition Matrix

$\displaystyle \left[\begin{matrix}\frac{s + 1}{s^{2} + 6 s + 8} & - \frac{3}{3 s^{2} + 18 s + 24}\\- \frac{3}{- s^{2} - 6 s - 8} & \frac{s + 5}{s^{2} + 6 s + 8}\end{matrix}\right]$

```python
_H = sy.simplify(_C*phi*_B + _D)
_H[0]
```

$\displaystyle \frac{12 s + 59}{s^{2} + 6 s + 8}$

# Pregunta 3.b

```python
A = [[0, 1, 0],[0, 0, 1], [-65, -73, -9]]
B = [[0], [0], [1]]
C = [[8, 1, 0]]
D = [[0]]

ss2tf(A, B, C ,D)
```

$$\frac{1.421 \times 10^{-14} s^2 + s + 8}{s^3 + 9 s^2 + 73 s + 65}$$

```python
p1 = np.array([1, 8])
p2 = np.array([1, 9, 73, 65])
tf2ss(p1, p2)
```

    A = [[ -9. -73. -65.]
     [  1.   0.   0.]
     [  0.   1.   0.]]

    B = [[1.]
     [0.]
     [0.]]

    C = [[0. 1. 8.]]

    D = [[0.]]

```python
def f(t):
    aux = np.sqrt(6)*t
    ans = 2/35 + 3*np.exp(-7*t)/217
    ans -= (np.exp(-2*t)*(11*np.sqrt(6)*np.cos(aux)+7*np.sin(aux)))/(155*np.sqrt(6))
    return ans

l = 0; r = 4; n = 100;
_t = np.linspace(0,4,100)

plt.figure(figsize=(12,5))
plt.plot(_t, f(_t))
plt.grid()

maxi = np.max(f(_t))
posi = np.argmax(f(_t))

print(f(100), maxi, l+posi*(r-l)/(n-1))
```

```python
print(f(0.628))
```

    0.05140827065221175


# 論理回路をパーセプトロンで表現

パーセプトロン : 単層のネットワークであり、活性化関数にステップ関数を使用

- x : 入力
- w : 重み（各入力の重要度を表す）
- b : バイアス（発火のしやすさを表す）
- a : ニューロンへの入力の総和
- h : 活性化関数（今回はステップ関数）
- y : 出力

\begin{equation}
    a = (x_1 * w_1 + x_2 * w_2) + b
\end{equation}

\begin{equation}
    y = h(a)
\end{equation}

```python
import numpy as np
```

```python
# ステップ関数 h(a)
def step(a):
    if a <= 0:
        return 0
    elif a > 0:
        return 1
```

```python
# 1層
def AND(x1, x2):
    x = np.array([x1, x2])
    w = np.array([0.6, 0.6])
    b = -1.0
    a = np.sum(x*w) + b
    y = step(a)
    return y

# 1層
def NAND(x1, x2):
    x = np.array([x1, x2])
    w = np.array([-0.6, -0.6])
    b = 1.0
    a = np.sum(x*w) + b
    y = step(a)
    return y

# 1層
def OR(x1, x2):
    x = np.array([x1, x2])
    w = np.array([0.6, 0.6])
    b = -0.4
    a = np.sum(x*w) + b
    y = step(a)
    return y

# 2層
def XOR(x1, x2):
    s1 = NAND(x1, x2)
    s2 = OR(x1, x2)
    y = AND(s1, s2)
    return y
```

```python
# ANDの確認
print(AND(0, 0), end=" ")
print(AND(0, 1), end=" ")
print(AND(1, 0), end=" ")
print(AND(1, 1))
```

    0 0 0 1

```python
# NANDの確認
print(NAND(0, 0), end=" ")
print(NAND(0, 1), end=" ")
print(NAND(1, 0), end=" ")
print(NAND(1, 1))
```

    1 1 1 0

```python
# ORの確認
print(OR(0, 0), end=" ")
print(OR(0, 1), end=" ")
print(OR(1, 0), end=" ")
print(OR(1, 1))
```

    0 1 1 1

```python
# XORの確認
print(XOR(0, 0), end=" ")
print(XOR(0, 1), end=" ")
print(XOR(1, 0), end=" ")
print(XOR(1, 1))
```

    0 1 1 0

```python

```


```python
import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 10: Stochastic Time Series Processes

Most models of the business cycle are *stochastic* time series models. That is, the models incorporate randomness as a determinant of equilibrium. Randomness is necessary for two reasons. First, from a practical point of view, any model will be incomplete and will not fit the data perfectly and so randomness in a macroeconomic model is analagous to an error term in an OLS model. Second, from a philosophical perspective, randomness in a macroeconomic model reflects economists' consensus that fundamentally unpredictable forces cause the business cycle.

## Simulating Normal Random Variables with Numpy

Recall that the `numpy.random` module has functions for generating (pseudo) random variables. Learn more about the module by reading the documentation: https://docs.scipy.org/doc/numpy/reference/routines.random.html

We use the `numpy.random.normal()` function to crate arrays of random draws from the normal distribution. The function takes three arguments:
* `loc`: the mean of the distribution (default=0)
* `scale`: the standard deviation of the distribution (default=1)
* `size`: how many to numbers to draw (default = `None`)

The default is to draw numbers from the *standard normal* distribution.

### Example

Draw 500 values each from the $\mathcal{N}(0,1)$ and $\mathcal{N}(0,2^2)$ distributions. Plot.

```python
# Set the seed for the random number generator to 126

# Create two arrays:
#     x: 500 draws from the normal(0,1) distribution
#     y: 500 draws from the normal(0,4) distribution

# Plot x and y

```

## The White Noise Process

In the previous example, we created two variables that stored draws from normal distrbutions with means of zero but with different standard deviations. Both of the variables were simulations of *white noise processes*. A white noise process is a random variable $\epsilon_t$ with constant mean and constant variance. We are concerned only with the zero-mean white noise process and we'll denote that a variable is a zero-mean white noise process with the following shorthand notation:

\begin{align}
\epsilon_t & \sim \text{WN}(0,\sigma^2),
\end{align}

where $\sigma^2$ is the variance of the processes. Strictly speaking, a white noise process can follow any distribution as long as the mean and variance are constant, but we'll concentrate exclusively white noise process drawn from the normal distribution.

## The AR(1) Process

A random variable $y_t$ is an *autoregressive process of order 1*  or AR(1) process if it can be written in the following form:

\begin{align}
y_t & = \rho y_{t-1} + \epsilon_t,
\end{align}

where $\rho$ is a constant and $\epsilon \sim \text{WN}(0,\sigma^2)$. The AR(1) process is the stochastic analog of the first-order difference equation where the random variable $\epsilon_t$ replaces the exogenous variable $w_t$.

### Example

Simulate an AR(1) process for 101 periods ($t = 0,\ldots, 100$) using the following parameter values:

\begin{align}
\rho & = 0.5\\
\sigma &  = 1\\
y_0 & = 0
\end{align}

Plot the simulated values for $y$.

```python
# Set the seed for the random number generator to 126

# Initialize values for T, y0, rho, and sigma

# Initialize an array of zeros for y

# Set the first value of y equal to y0

# Create a variable called 'epsilon' equal to an array containing T draws from the normal(0,sigma^2) process

# Iterate over t in range(T-1) to compute y

# Plot y

```

The AR(1) process wtih $\rho = 0.5$ seems to fluctuate around the value $y=0$ and so the process appears to be stable

### Example

Simulate an AR(1) process for 101 periods ($t = 0,\ldots, 100$) using the following parameter values:

\begin{align}
\rho & = 1.5\\
\sigma &  = 1\\
y_0 & = 0
\end{align}

Plot the simulated values for $y$.

```python
# Set the seed for the random number generator to 126

# Initialize values for T, y0, rho, and sigma

# Initialize an array of zeros for y

# Set the first value of y equal to y0

# Create a variable called 'epsilon' equal to an array containing T draws from the normal(0,sigma^2) process

# Iterate over t in range(T-1) to compute y

# Plot y

```

The AR(1) process wtih $\rho = 1.5$ seems to be approaching infinity in magnitude and so the process appears to be explosive.

In general, like the first-order difference equation, if $|\rho| < 1$, the the AR(1) process is stable. If $|\rho|>1$ then the process is explosive. A special case is the *random walk process* which is an AR(1) process with $\rho = 1$. The random walk process has many important applications including asset pricing theory.

The function in the next cell

```python
# Define a function for simulating an AR(1) process. CELL PROVIDED
def ar1_sim(rho=0,sigma=1,y0=0,T=25):
    '''Funciton for simulating an AR(1) process for T periods

            Args:
            rho (float):   autoregressive parameter
            sigma (float): standard deviation of the white noise process
            y0 (float):    initial value of the process
            T (int):       number of periods to simulate

        Returns:
            numpy array
    '''

    # initialize y array
    y = np.zeros(T)
    y[0] = y0

    # draw random numbers for white noise process
    eps= np.random.normal(loc=0,scale=sigma,size=T-1)
    for t in range(T-1):
        y[t+1] = rho*y[t] + eps[t]

    return y
```

### Example:

Use the `ar1_sim()` function to simulate an AR(1) process for 201 periods ($t = 0,\ldots, 200$) using the following parameter values:

\begin{align}
\rho & = -0.99\\
\sigma &  = 0.5\\
y_0 & = 0
\end{align}

Set the seed for the NumPy random number generator to 126. Plot the simulated values for $y$.

```python
# Set the seed for the random number generator to 126

# Simulate y and plot

```

## Application: Estimate TFP as an AR(1) process

It is routine to model quarterly fulctuations in TFP as an AR(1) process and we will encounter this in our business cycle models. Here we will fit the following AR(1) model to US data:

\begin{align}
\log\left(A_t/A^{trend}_t\right) & = \rho \log\left(A_{t-1}/A^{trend}_{t-1}\right) + \epsilon_t
\end{align}

where $\log\left(A_t/A^{trend}_t\right) = \log A_t - \log A^{trend}$ is the log-deviation of TFP from its trend and $\epsilon_t$ is a white noize process with mean 0 and variance $\sigma^2$.

### The Data
The file `rbc_data_actual_trend.csv`, available at https://github.com/letsgoexploring/econ126/raw/master/Data/Csv/rbc_data_actual_trend.csv, contains actual and trend data for real GDP per capita, real consumption per capita, real investment per capita, real physical capital per capita, TFP, and hours per capita at quarterly frequency. The GDP, consumption, investment, and capital data are in terms of 2012 dollars. Hours is measured as an index with the value in October 2012 set to 100. All of the data are *real* quantities. That is, there are no *nominal* quantities like money or inflation or a nominal interest rate. The reason is that the first theory that we will encounter is called *real business cycle* or RBC theory and, in that theory, there is no place for nominal quantities. RBC theory seeks to explain fluctuations in real quantities as being primarily due to TFP shocks; i.e., shocks to the production function.

### Objectives

1. Import TFP data (trend and actual) and compute cyclical component as log-deviation from trend.
2. Construct a scatter plot of log-deviation of TFP from trend against *lagged* log-deviation of TFP from trend.
3. Estimate the AR(1) model of log-deviation of TFP from trend to obtain estimates of $\rho$ and $\sigma$.

```python
# Read business_cycle_data_actual_trend.csv into a Pandas DataFrame called 'df' with the first column set as the index and parse_dates=True

```

```python
# Construct a plot of TFP with its trend with:
#    1. Actual line: blue with lw=1, alpha=0.7, label = 'actual'
#    2. Trend line:  red with lw=3, alpha=0.7, label = 'trend'

```

Now we need to compute the log-deviation of TFP from its trend. Note that since $\log(a/b) = \log(a) - \log(b)$, we can write:

\begin{align}
\log A_t - \log A^{trend} & = \log\left(A_t/A^{trend}_t\right)
\end{align}

The term on the right will make the AR(1) model easier to read.

\begin{align}
\log\left(A_t/A^{trend}_t\right) & = \rho \log\left(A_{t-1}/A^{trend}_{t-1}\right) + \epsilon_t
\end{align}

```python
# Create a new column to df called tfp_cycle equal to the log difference between actual TFP and it's trend:

```

```python
# Plot the log deviation of TFP from its trend (times 100)

```

```python
# Create a new column to df called tfp_cycle_lag that, for each date, contains values
# in 'tfp_cycle' at the previous date

# Print the first five rows of only the 'tfp_cycle' and 'tfp_cycle_lag' columns. PROVIDED
print(df[['tfp_cycle','tfp_cycle_lag']].head())
```

Notice that there is a missing value inthe `tfp_cycle_lag` column for the first date in the index because there is no prior observation in the `tfp_cycle` column. To proceed with the estimation, we need to get rid of the row with missing values

```python
# Use dropna() method on df to remove the row iwth the missing value

# Print the first five rows of only the 'tfp_cycle' and 'tfp_cycle_lag' columns. PROVIDED
print(df[['tfp_cycle','tfp_cycle_lag']].head())
```

Next, we should make a scatter plot of log-deviation of TFP from trend against the one-period lag of log-deviation of TFP from trend to see if out AR(1) model is a good idea.

```python
# Construct a scatter plot of log-deviation of TFP from trend against the one-period lag of log-deviation
# of TFP from trend with:
#    1. scatter points sized at least 50, opacity (alpha) no greater than 0.25
#    2. x- and y-axis limits: [-0.04,0.04]

```

Now we use StatsModels to fit the model. First we estimate the AR(1) model to obtain an estimate of $\rho$. Then we find the standard deviation of the residuals of the regression to estimate $\sigma$.

```python
# Create variable 'X' to be the independent variable of the OLS model. Do not add a constant to X

# Create variable 'Y' to be the dependent variable of the OLS model.

# Create a variable called 'model' that initializes the OLS model

# Create a variable called 'results' that stores the results of the estimated OLS model

# Print output of summary2() mdethod of results

```

Estimated coefficients are stored as a Pandas `Series` in the `params` attribute of `results`. Index values correspond to names of columns in the dependent variable `X`

```python
# Print the contents of the params method of results

```

```python
# Create a variable called 'rho' that equals the value of the estimated coefficient on df['tfp_cycle_lag']

```

The residuals from the regression are stored in an attribute of `results` called `resid`

```python
# Create a variable called 'sigma' that equals the standard deviation of the residuals of the regression

# Print the value of sigma

```

And that's how you estimate the parameters of an AR(1) model of the cyclical component of TFP fr the US.

## Application: A Stochastic Solow Growth Model (Optional)

Consider the Solow growth model with written in "per worker" terms:

\begin{align}
y_t & = A_tk_t^{\alpha}\\
k_{t+1} & = i_t + (1-\delta)k_{t}\\
y_t & = c_t + i_t\\
c_t & = (1-s)y_t,
\end{align}

where $y_t$ is output per worker, $k_t$ is capital per worker, $c_t$ is consumption per worker, $i_t$ is investment per worker, and $A_t$ is time-varying TFP. Assume:

\begin{align}
\log A_{t+1} & = \rho \log A_{t} + \epsilon_{t+1}
\end{align}

where $\epsilon_t$ is a white noise process with mean 0 and variance $\sigma^2$. Since capital and TFP are the only two variables, that depend on past value of themselves, they are the two state variables of the model. Therefore, we can simulate simulate the model in two steps. First, simulate $k_t$ and $A_t$ using the following two laws of motion:

\begin{align}
k_{t+1} & = sAk_t^{\alpha} + (1-\delta)k_{t} \label{eqn:capital_solution}\\
\log A_{t+1} & = \rho \log A_{t} + \epsilon_{t+1}. \label{eqn:tfp_solution}
\end{align}

Second, compute simulated values for $y_t$, $c_t$, and $i_t$ using the following static relationships:

\begin{align}
y_t & = A_tk_t^{\alpha}\\
i_t & = sA_tk_t^{\alpha}\\
c_t & = (1-s)A_tk_t^{\alpha},
\end{align}

In the previous example, we estimated $\rho$ and $\sigma$ for the US so let's use those values for this simulation. For the other parameter, use the following values for the simulation:

| $A_0$ | $k_0$ | $s$ | $\alpha$ | $\delta $ | $T$  |
|-------|-------|-----|----------|-----------|------|
| 1     | 8.43  | 0.1 | 0.35     |  0.025    | 201  |

Where $T$ is the total number of simulation periods (i.e., $t$ will range from $0$ to $200$).

### Function

The function in the next cell simulates a stochastic Solow growth model. It returns a `DataFrame` with colums equal to the log-deviations of the simulated variables relative to the trend (i.e., nonstochastic steady state) implied by the model.

```python
# Define a function that returns a DataFrame of simulated value from the Solow model with exogenous labor and TFP growth. CELL PROVIDED
def solow_stochastic(s,alpha,delta,k0,A0,rho,sigma,T):

    '''Function for computing a simulation of the Solow growth model with exogenous labor and TFP growth.

            y[t]   = A[t]*K[t]^alpha
            k[t+1] = i[t] + (1-delta)*k[t]
            y[t]   = c[t] + i[t]
            c[t]   = (1-s)*y[t]
            A[t+1] = exp(rho*logA[t] + epsilon[t+1])

    Args:
        s (float):      Saving rate
        alpha (float):  Capital share in Cobb-Douglas production function
        delta (float):  Capital depreciation rate
        T (int):        Number of periods to simulate
        k0 (float):     Initial value of capital per worker
        A0 (float):     Initial TFP
        rho (float):    AR coeficient for log A[t]
        sigma (float):  Standard deviation of shock to log A[t]

     Returns:
         Pandas DataFrame
     '''
    # Create epsilon values
    epsilon = np.random.normal(scale=sigma,size=T)

    # Initialize capital values
    capital = np.zeros(T)

    # Set first value of capital equal to k0
    capital[0] = k0

    # Initialize TFP values
    tfp = np.zeros(T)

    # Set first value of TFP equal to A0
    tfp[0] = A0

    # Iterate over t in range(T-1) to update subsequent values in the capital and tfp arrays
    for t in range(T-1):
        capital[t+1] = s*tfp[t]*capital[t]**alpha + (1-delta)*capital[t]
        tfp[t+1] = np.exp(rho*np.log(tfp[t]) + epsilon[t])

    # Compute the values of the other aggregate variables
    output = tfp*capital**alpha
    consumption = (1-s)*output
    investment = s*output

    # Compute steady state (or trend) of endogenous vars
    capital_ss = (s/delta)**(1/(1-alpha))
    tfp_ss = 1
    output_ss = tfp_ss*capital_ss**alpha
    consumption_ss = (1-s)*output_ss
    investment_ss = s*output_ss

    # Put simulated data into a DataFrame
    df = pd.DataFrame({'output_log_dev':np.log(output/output_ss),
                       'consumption_log_dev':np.log(consumption/consumption_ss),
                       'investment_log_dev':np.log(investment/investment_ss),
                       'capital_log_dev':np.log(capital/capital_ss),
                       'tfp_log_dev':np.log(tfp/tfp_ss)})

    # Return the simulated data
    return df
```

### Simulate the Stochastic Solow Model

```python
# CELL PROVIDED
# Set parameters for simulation
alpha=0.35
s = 0.1
delta = 0.025
k0 = 8.43
A0=1
T = 201

np.random.seed(126)
# Simulate the model and store output in a variable called 'solow_df'
solow_df = solow_stochastic(s,alpha,delta,k0,A0,rho,sigma,T)

# Plot
fig = plt.figure()
ax = fig.add_subplot(1,1,1)
solow_df.plot(ax=ax,grid=True)
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))
```

### Simulation Results

Compute the standard deviation of the simulated data and the correlation coefficients of the simulated data

```python
# Standard deviations. CELL PROVIDED
solow_df.std()*100
```

```python
# Correlation coefficients. CELL PROVIDED
solow_df.corr()
```

It looks like the stochastic Solow model does a god job replicating the volatility (standard deviation) of ouput and consumption relative to the data. However the simulated investment is too small by about a factor of 6. The stochastic Solow model does captures the correlation between output, consumption, and investment, but it implies perfect correlation which is too much.

## The Random Walk Process (Optional)

The *random walk process* is an AR(1) process with $\rho=1$:

\begin{align}
y_t = y_{t-1} + \epsilon_t
\end{align}

The random walk process has an important place in finance since the evidence suggests that stock prices follow a random walk process.

### Example

Simulate 7 random walk processes for 501 periods. Set $\sigma = 1$. Plot all 7 simulated processes on the same ayes.

```python
# CELL PROVIDED
np.random.seed(126)
for i in range(7):
    plt.plot(ar1_sim(rho=1,T=501))

plt.title('Seven random walk processes')
plt.grid()
```


```python
import numpy as np
import scipy
import sympy as sym
import matplotlib
sym.init_printing()
```

## Integrate

```python
x = sym.symbols('x')
a = sym.Integral(sym.cos(x)*sym.exp(x), x)
sym.Eq(a, a.doit())
```

## Derivative

```python
y = x**2+4*x
y
```

```python
sym.plot(y, y.diff(x,1), y.diff(x, 2))
```

```python
a = sym.symbols('a')
x = sym.symbols('x')
y = sym.Function('y')(x)
z = sym.Function('z')(y)
sym.Eq(z.diff(y)*y.diff(x)*a, z.diff(x))
```

```python
x, y = sym.symbols('x y')
z = x**2+2*x+y**2+4*y+4
dzx = z.diff(x)
dzy = z.diff(y)
sym.plotting.plot3d(z, (x, -10, 10), (y, -10, 10))
sym.plotting.plot3d(sym.sqrt(dzx**2 +  dzy**2), (x, -10, 10), (y, -10, 10))
```

```python
x, y = sym.symbols('x y')
z = x**2+2*x+y**2+4*y+4
z
```

```python
(z.diff(x),z.diff(y))
```

$$lim_{x\rightarrow 0}{\frac{sin(x)}{x}}$$

```python
x, y = sym.symbols('x y')
sym.limit(sym.sin(x)/x,x,0)
```

## Series

```python
x, y = sym.symbols('x y')
z=sym.series((x-3)*(x+1)*(x+4)*(x-4)*x,x,x0=0,n=4)
z
```

```python
sym.plot(48*x,32*x**2,-19*x**3,(x,-4,4))
sym.plot(48*x+32*x**2-19*x**3,(x-3)*(x+1)*(x+4)*(x-4)*x,(x,-4,4))
```

## 三角函数与傅里叶变换

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl

mpl.style.use('seaborn-darkgrid')
x = np.linspace(-np.pi,np.pi,2000)
y = np.ones_like(x)
dx = np.mean(x[1:]-x[:-1])
y[:1000]=-1
plt.figure(1)
plt.plot(x, y, lw=2, alpha=.6)
plt.xlim([-1.2,1.2])

def par(n):
    an=1/np.pi*np.trapz(y*np.cos(x*n),dx=dx)
    bn=1/np.pi*np.trapz(y*np.sin(x*n),dx=dx)
    return an,bn
plt.figure(2)
for itr in range(20):
    an, bn = par(itr)
    plt.plot(x,an*np.cos(itr*x), lw=3, alpha=.5)
    plt.plot(x,bn*np.sin(itr*x), lw=3, alpha=.5)
plt.figure(3)
plt.plot(x, y, lw=2, alpha=.6)
sums = np.zeros_like(x)
for itr in range(20):
    an, bn = par(itr)
    sums[:] += an*np.cos(itr*x)+bn*np.sin(itr*x)
    plt.plot(x,sums, lw=3, alpha=.3)
plt.show()

```

## 曲线拟合

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import leastsq
import matplotlib as mpl
mpl.style.use('seaborn-darkgrid')
```

```python
#待拟合的函数，x是变量，p是参数
def func(x, p):
    a, b, c, d, e, f = p
    return a+b*x+c*x**2+d*x**3+e*x**4+f*x**5

#计算真实数据和拟合数据之间的误差，p是待拟合的参数，x和y分别是对应的真实数据
def error(p, x, y):
    return func(x, p) - y

#一组真实数据，在a=2, b=1的情况下得出
x1 = np.array([1, 2, 3, 4, 5, 6], dtype=float)
y1 = np.array([3, 2, 7, 9, 6, 13], dtype=float)

#调用拟合函数，第一个参数是需要拟合的差值函数，第二个是拟合初始值，第三个是传入函数的其他参数
r = leastsq(error, [1, 1, 1, 1, 1, 1], args=(x1, y1))

#打印结果，r[0]存储的是拟合的结果，r[1]、r[2]代表其他信息
x = np.linspace(1,6,1000)
y = func(x, r[0])
plt.plot(x,y, lw=2, alpha=0.4)
plt.scatter(x1, y1)
plt.show()
```

```python

```

math_slides/Chapter 3 - Function and derivative.ipynb


# Instrucciones

- El desafío es individual, por lo cual se considera que todo el código entregado es de elaboración propia y no ha sido compartido de forma alguna.
- Las consultas sobre el desafío se deben realizar por medio de la plataforma Aula. **No está permitido publicar código en consultas de Aula**.
- El desafío debe ser realizado en Jupyter Notebook (Python3) utilizando este archivo como base.
- Debe utilizar arreglos de ```NumPy``` junto a las funciones y bibliotecas que se ponen a disposición en este archivo.
- Se debe respetar las firmas de las funciones, sus parámetros y retornos. Para eso se incluye un detalle de cada función tanto de las entregadas como las que deberá implementar. En caso de no seguir estas instrucciones, el desafío no podrá evaluarse.
- Se evaluará la correcta utilización de librerias ```NumPy```, ```SciPy```, entre otras, así como la correcta implementación de algoritmos de forma vectorizada.
- Evitar la impresión de mensajes salvo que se solicite en el enunciado.
- El archivo de entrega debe denominarse **ROL-desafio-numero.ipynb**. **De no respetarse este formato existirá un descuento de 50 puntos**
- La fecha de entrega es el **miércoles 27 de Mayo a las 10:00 hrs**,  posteriormente existirá un descuento lineal hasta las 11:00 hrs del mismo día.

```python
# Libraries
import time # For time profiling
import numpy as np # Linear algebra and data structure
import matplotlib.pyplot as plt # Plots
from scipy import linalg as spla # Linear Algebra
```

# Introducción

En el siguiente desafio estudiaremos la complejidad temporal de resolver sistemas de ecuaciones lineales utilizando: la matriz inversa, PALU, Cholesky, y el _solver_ de NumPy.
Para poder obtener la complejidad temporal de forma experimental se deberá medir el tiempo que les toma a cada uno de los métodos en resolver un sistema de ecuaciones de tamaño $n \times n$.

## Recordatorio

### Sistema de ecuaciones lineales

\begin{equation}
  A \mathbf{x} = \mathbf{b}
\end{equation}

### Algoritmo de resolución utilizando la inversa

1. $B=A^{-1}$
2. $\mathbf{x} = B\,\mathbf{b}$
3. Retornar $\mathbf{x}$

### Resolución utilizando $PA=LU$

\begin{equation}
  \begin{split}
    A\, \mathbf{x} & = \mathbf{b} \\
    PA\, \mathbf{x} & = P\,\mathbf{b} \\
    LU\, \mathbf{x} & = P\, \mathbf{b} \\
    L\, \mathbf{c} & = P\, \mathbf{b}
  \end{split}
\end{equation}

Algoritmo,
1. Obtener descomposición PALU: $PA=LU$
2. Resolver para $\mathbf{c}$: $L\,\mathbf{c} = P\, \mathbf{b}$
3. Resolver para $\mathbf{x}$: $U\,\mathbf{x} = \mathbf{c}$
4. Retornar $\mathbf{x}$

### Resolución utilizando Cholesky

\begin{equation}
  \begin{split}
    A \, \mathbf{x} & = \mathbf{b} \\
    R^T\,R \,\mathbf{x} & = \mathbf{b}
  \end{split}
\end{equation}

Algoritmo,
1. Obtener descomposición de Cholesky: $A=R^T\,R$
2. Resolver para $\mathbf{c}$: $R^T\mathbf{c} = \mathbf{b}$
3. Resolver para $\mathbf{x}$: $R\,\mathbf{x} = \mathbf{c}$
4. Retornar $\mathbf{x}$

# Ejercicio

Midiendo el tiempo que demora la resolución de sistemas de ecuaciones lineales para distintos tamaños, se puede estimar la complejidad temporal en función de su tamaño, es decir $n$.

Para generar sistemas de ecuaciones lineales, usted dispone de la función  ```generateLinearSystem(n)``` que recibe como argumento un número entero $n$ y retorna una matriz $A\in\mathbb{R}^{n\times n}$ definida positiva y un vector $\mathbf{b}\in\mathbb{R}^n$ aleatorio.
A modo de referencia: ¿Qué propiedades tiene la matriz construida? ¿Cual sería la solución del sistema de ecuaciones lineales?

```python
def generateLinearSystem(n):
    """
    Parameters
    ----------
    n       : int
            System of linear equations size

    Returns
    -------
    A       : (n, n) array
            Positive definite matrix with random values
    b       : (n, ) array
            RHS vector
    """
    # Matrix generation
    A = np.random.rand(n, n)
    A = np.dot(A.T, A) # Symmetric matrix
    A += np.eye(n) # Diagonally dominant

    # RHS generation
    b = np.dot(A, np.ones(n))

    return A, b
```

A continuación se entregan las funciones que implementan los 4 algoritmos mencionados anteriormente:

```python
def solveInverse(A, b):
    """
    Parameters
    ----------
    A       : (n, n) array
            Positive definite matrix with random values
    b       : (n, ) array
            RHS vector

    Returns
    -------
    x       : (n, ) array
            System of linear equations solution
    """
    # Inverse matrix
    B = np.linalg.inv(A)

    # Get solution
    x = np.dot(B, b)

    return x
```

```python
def solvePALU(A, b):
    """
    Parameters
    ----------
    A       : (n, n) array
            Positive definite matrix with random values
    b       : (n, ) array
            RHS vector

    Returns
    -------
    x       : (n, ) array
            System of linear equations solution
    """
    # PA=LU factorization
    P, L, U = spla.lu(A)

    # Vector permutation
    Pb = np.dot(P.T, b)

    # Forward substitution
    c = spla.solve_triangular(L, Pb, lower=True)

    # Backward substituion
    x = spla.solve_triangular(U, c)

    return x
```

```python
def solveCholesky(A, b):
    """
    Parameters
    ----------
    A       : (n, n) array
            Positive definite matrix with random values
    b       : (n, ) array
            RHS vector

    Returns
    -------
    x       : (n, ) array
            System of linear equations solution
    """
    # Cholesky decomposition
    # Warning: R in Scipy is different from the classnotes, but it is the same idea.
    R = np.linalg.cholesky(A)

    # Forward substitution
    c = spla.solve_triangular(R, b, lower=True)

    # Backward substitution
    x = spla.solve_triangular(R.T, c)

    return x
```

```python
def solveDirect(A, b):
    """
    Parameters
    ----------
    A       : (n, n) array
            Positive definite matrix with random values
    b       : (n, ) array
            RHS vector

    Returns
    -------
    x       : (n, ) array
            System of linear equations solution
    """
    x = np.linalg.solve(A, b)
    return x
```

A continuación, se entrega la función ```obtainComputationTimes(solver, Ns, Ne)```, la cual obtiene el tiempo de computación utilizado en la ejecución por el algoritmo ```solver``` en resolver un sistema de ecuaciones lineales de tamaño $Ns[i]\times Ns[i]$, para todo $i$. Además se puede incluir un parámetro adicional, $N_e$, que define el número de veces que se repite el experimento para obtener una mejor aproximación y reducir los efectos de computaciones concurrentes que puedan afectar los tiempos de computación. Mientras mayor es $N_e$ mejor es la estimación pero toma mayor tiempo la ejecución.

```python
def obtainComputationTimes(solver, Ns, Ne=5):
    """
    Parameters
    ----------
    solver            : function
                      Function with linear system solver implementation
    N                 : (n, ) array
                      Array of integers with system size
    Ne                : int
                      Number of experiments

    Returns
    -------
    times_solver      : (n, ) array
                      Mean times using solver
    """
    Nn = Ns.shape[-1]
    times_solver = np.zeros(Nn)

    for i in range(Nn):
        # Generate system Ax=b
        A, b = generateLinearSystem(Ns[i])

        # Compute time
        start = time.time()
        # Repeating the computation "Ne" times to get more accurate results.
        for e in range(Ne):
            x_i = solver(A, b)
        end = time.time()

        # Get mean time
        times_solver[i] = (end - start) / Ne

    return times_solver
```

Recordando que para la factorización PALU se determinó que su complejidad computacional era $\sim \dfrac{2}{3}n^3$, donde $n$ correspondía al tamaño del sistema de ecuaciones lineales.
En este desafío se quiere estudiar la estimación experimental de tal complejidad, por ejemplo para un algoritmo denotado por ```solver``` podemos indicar que su complejidad computacional tiene la forma $\sim c\,n^p$, donde $c$ sería su coeficiente y $p$ su exponente, en el caso de PALU, $c=\dfrac{2}{3}$ y $p=3$.
En particular, podemos determinar el tiempo total de computación con la siguiente relación $\text{totalTime}\approx c\,n^p\,\text{TOp}$, donde ```TOp``` se puede obtener como se explica a continuación.

Mediante la función ```timeOperation()``` usted debe estimar el tiempo promedio de las $4$ operaciones elementales, el cual se denomina ```TOp```. **Debe considerar que el tiempo de ejecución de esta función no debe superar los $20$ segundos**.

```python
def timeOperation():
    """
    Returns
    -------
    TOp       : float
              Elementary operation estimation time
    """
    # Algorithm...
    TOp = 0
    return TOp
```

Además debe implementar la función ```findComplexity(Ns, times, TOp)``` donde ```Ns``` es el arreglo con el tamaño de los sistemas, ```times``` es el arreglo con los tiempos del algoritmo analizado y ```TOp``` es la estimación del tiempo que demora una operación elemental. Esta función debe retornar el coeficiente ```c``` y el exponente ```p``` indicados anteriormente.
Utilice su algoritmo para determinar la complejidad experimental de  ```solveInverse```, ```solvePALU```, ```solveCholesky```, y ```solveDirect``` a modo de ejemplo. Su tarea será evaluada con otros algoritmos luego.

```python
def findComplexity(Ns, times, Top):
    """
    Parameters
    ----------
    Ns                : (n, ) array
                      Array with system size
    times             : (n, ) array
                      Mean times using a solver
    TOp               : float
                      Elementary operation estimation time

    Returns
    -------
    c                 : float
                      Complexity constant
    p                 : float
                      Complexity exponent
    """
    # Algorithm
    c = 0
    p = 0
    return c, p
```


```python
# https://colab.research.google.com/github/kassbohm/tm-snippets/blob/master/ipynb/TM_A/TM_2/fem-rod-3-node_1.ipynb

from sympy.physics.units import *
from sympy import *
from numpy import linspace, array

l, E, A, q = var("l, E, A, q")

x = var("x")

S = q*(l-x)
sigma = S/A

pprint("\nu1:")
u1 = 1/E * integrate(sigma, (x, 0, l/2))
tmp = u1
pprint(tmp)

pprint("\nu2:")
u2 = 1/E * integrate(sigma, (x, 0, l))
tmp = u2
pprint(tmp)

# u1:
#    2
# 3⋅l ⋅q
# ──────
# 8⋅A⋅E
#
# u2:
#   2
#  l ⋅q
# ─────
# 2⋅A⋅E

```


```python
# Initialisation Cell
# You should always put imported modules here
from math import *
import numpy as np
import numpy.testing as nt
import numpy.linalg as LA
from matplotlib import pyplot as plt
import seaborn as sns
np.set_printoptions(suppress=True, precision=7)
```

# APPM4057 - November 2019

# Exam Office Info

| Examiner          | Name              |
|-------------------|-------------------|
| Internal Examiner | Prof B. A. Jacobs |
| External Examiner | Prof N. Hale (Stellenbosch) |

# CDEs Honours - Exam

## Instructions

* Read all the instructions carefully.
* The exam consists of **80 Marks**, with **three and a half** hours available.
* The written section is to be answered in the book provided.
* You must only access Moodle **TESTS** and NOT Moodle.
* The programming section is to be answered within this Jupyter notebook and resubmitted to Moodle **TESTS**.
* Do not rename the notebook, simply answer the questions and resubmit the file to Moodle.
* The moodle submission link will expire at exactly **12:30** and **NO** late submission will be accepted. Please make sure you submit timeously!
* The **Numpy** and **Matplotlib** documentation has been downloaded and is open on your current machine.
* **NB!!!** Anyone caught using Moodle (and its file storing), flash drives or external notes will be awarded zero and reported to the legal office.

## Written Section

* Answer the following questions in the answer book provided.

### Question 1 - 10 Marks

Discuss each of the below listed terms and their relation to one another (i.e. the theorem which describes this relationship):

* Consistency,
* Stability,
* Convergence,

Be sure to give potential methods of analysing consistency, stability and convergence of a  linear PDE.

### Question 2 - 10 Marks

Analyse the stability of the difference scheme given by:
$$
u^{m}_{n} = -\beta u^{m+1}_{n-1} + (1+2\beta) u^{m+1}_{n} - \beta u^{m+1}_{n+1},
$$
using the discrete Fourier Transform:
$$
\hat{u}^{m+1}(\xi)=\frac{1}{\sqrt{2 \pi}} \sum_{n=-\infty}^{\infty} e^{-i n \xi} u_{n}^{m+1}.
$$

### Question 3 - 10 Marks

Consider the Crank-Nicolson (CN) scheme applied to the heat equation:
\begin{equation}
u_t = \nu u_{xx}.\label{eq:heat}
\end{equation}

(a) [3 Marks] Write out the CN scheme for the heat equation given by above.

(b) [7 Marks] Investigate the consistency of the scheme derived in Question 3(a).

### Question 4 - 10 Marks

Consider the following two-dimensional wave equation:
$$
\dfrac{\partial^2 u}{\partial t^2} = \nu^2 \left(\dfrac{\partial^2 u}{\partial x^2} + \dfrac{\partial^2 u}{\partial y^2}\right).\label{eq:wave}
$$

(a) [5 Marks] Set up the system of coupled first-order conservative partial differential equations by structuring new dependent variables $r$, $s$ and $l$, which can be used to solve the wave equation given above via the Lax-Wendroff finite difference scheme.

(b) [5 Marks] Provide the system in vector notation where the system is represented as:
$$
\dfrac{\partial \bf{U}}{\partial t} + \nabla \bf{F}(\bf{U}) = 0,
$$
where $\bf{U} = [r,s,l]$.

## Programming Section

### Question 1 - 15 Marks

(a) [10 Marks] Consider the wave equation given by:
$$
u_{tt} = c^2u_{xx},
$$
$$
u(x, 0) = x(1 - x), \ \ \ u_t(x, 0) = 0, \ \ \ u(0, t) = u(1, t) = 0.
$$
Write a function that implements a centered space, centered time scheme. Specifically, your function should take as inputs, the stepsizes `dx` and `dt`, the wave speed `c` (not squared valued), the number of iterations to perform `N` and the left and right endpoints `a`, `b`.

Your function should return the solution matrix, containing the wave profile at each time iteration, i.e. `N + 1` rows.

```python
def wave_eq(dx, dt, c, N, ic, a, b):
    # YOUR CODE HERE
    raise NotImplementedError()

```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx = 0.1
c  = 0.2
tf = 8
N  = 20
dt = (tf - 0)/N
a  = 0
b  = 1
ic = lambda x: x*(1 - x)
tans = np.array([0.  , 0.09, 0.16, 0.21, 0.24, 0.25, 0.24, 0.21, 0.16, 0.09, 0.  ])
U = wave_eq(dx, dt, c, N, ic, a, b)
nt.assert_array_almost_equal(tans, U[0, :])
print('Test case passed!!!')
```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx = 0.1
c  = 0.2
tf = 8
N  = 20
dt = (tf - 0)/N
a  = 0
b  = 1
ic = lambda x: x*(1 - x)
tans = np.array([0.    , 0.0836, 0.1536, 0.2036, 0.2336, 0.2436, 0.2336, 0.2036,
       0.1536, 0.0836, 0.    ])
U = wave_eq(dx, dt, c, N, ic, a, b)
nt.assert_array_almost_equal(tans, U[1, :])
print('Test case passed!!!')
```

```python
# Hidden test
# No output will be produced
# 8 marks
```

(b) [5 Marks] Continuing with Question 1(a), given the input below, produce the 3D surface plot, illustrating the evolution of the wave profile over time.

```python
dx    = 0.001
c     = 0.2
tf    = 8
N     = 2000
dt    = (tf - 0)/N
a     = 0
b     = 1
ic    = lambda x: x*(1 - x)
xvals = np.linspace(a, b, int((b - a)/dx) + 1)
U     = wave_eq(dx, dt, c, N, ic, a, b)
```

```python
# 5 Marks
# YOUR CODE HERE
raise NotImplementedError()
```

### Question 2 - 10 Marks

(a) [8 Marks] Write a function implements a finite difference scheme to solve the partial differential equation below. The function should implement an explicit scheme which is forward difference in time and central difference in space, specifically first order in time and second order in space. It should take as inputs a time step `dt`, a spatial step `dx`, a number of time-steps `N` to perform, the coefficients `D` and `nu`, an initial function `ic` passed as a handle and boundary values `alpha` and `beta`. The function should output the solution space matrix `u`, as well as the `xvals` (i.e. vector of $x$ steps) and `tvals` (i.e the vector of time steps). The PDE is given below:

$$v_t + \nu v_x = D v_{xx},$$

$$\text{BC}: v(0, t) = \alpha = v(1, t) = \beta = 0;\quad\text{IC}: v(x, 0) = \sin(8\pi x); \quad t > 0, \ x \in [0, 1]$$

```python
def heatEqn(dt, dx, N, ic, D, alpha, beta, nu):
    # YOUR CODE HERE
    raise NotImplementedError()

```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx = 0.01
dt = 0.01
N  = 2
D  = 0.1
nu = 1
alpha = 0
beta  = 0
ic = lambda x: np.sin(8*np.pi*x)
u, xx, tt = heatEqn(dt, dx, N, ic, D, alpha, beta, nu)
tans = np.array(  [ 0.       ,  0.2486899,  0.4817537,  0.6845471,  0.8443279,
                    0.9510565,  0.9980267,  0.9822873,  0.9048271,  0.7705132,
                    0.5877853,  0.3681246,  0.1253332, -0.1253332, -0.3681246,
                   -0.5877853, -0.7705132, -0.9048271, -0.9822873, -0.9980267,
                   -0.9510565, -0.8443279, -0.6845471, -0.4817537, -0.2486899,
                   -0.       ,  0.2486899,  0.4817537,  0.6845471,  0.8443279,
                    0.9510565,  0.9980267,  0.9822873,  0.9048271,  0.7705132,
                    0.5877853,  0.3681246,  0.1253332, -0.1253332, -0.3681246,
                   -0.5877853, -0.7705132, -0.9048271, -0.9822873, -0.9980267,
                   -0.9510565, -0.8443279, -0.6845471, -0.4817537, -0.2486899,
                   -0.       ,  0.2486899,  0.4817537,  0.6845471,  0.8443279,
                    0.9510565,  0.9980267,  0.9822873,  0.9048271,  0.7705132,
                    0.5877853,  0.3681246,  0.1253332, -0.1253332, -0.3681246,
                   -0.5877853, -0.7705132, -0.9048271, -0.9822873, -0.9980267,
                   -0.9510565, -0.8443279, -0.6845471, -0.4817537, -0.2486899,
                   -0.       ,  0.2486899,  0.4817537,  0.6845471,  0.8443279,
                    0.9510565,  0.9980267,  0.9822873,  0.9048271,  0.7705132,
                    0.5877853,  0.3681246,  0.1253332, -0.1253332, -0.3681246,
                   -0.5877853, -0.7705132, -0.9048271, -0.9822873, -0.9980267,
                   -0.9510565, -0.8443279, -0.6845471, -0.4817537, -0.2486899,
                    0.       ])
nt.assert_array_almost_equal(tans, u[0])
print('Test case passed!!!')
```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx = 0.01
dt = 0.01
N  = 2
D  = 0.1
nu = 1
alpha = 0
beta  = 0
ic = lambda x: np.sin(8*np.pi*x)
u, xx, tt = heatEqn(dt, dx, N, ic, D, alpha, beta, nu)
tans = np.array(  [ 0.       ,  2.4511655, -0.1252406, -0.0825335, -0.0346406,
                    0.015429 ,  0.064529 ,  0.1095745,  0.1477351,  0.1766128,
                    0.1943934,  0.1999595,  0.1929614,  0.1738388,  0.1437933,
                    0.1047128,  0.0590527,  0.0096822, -0.0402967, -0.0877437,
                   -0.1296773, -0.1634629, -0.1869775, -0.1987436, -0.1980219,
                   -0.1848578, -0.1600783, -0.1252406, -0.0825335, -0.0346406,
                    0.015429 ,  0.064529 ,  0.1095745,  0.1477351,  0.1766128,
                    0.1943934,  0.1999595,  0.1929614,  0.1738388,  0.1437933,
                    0.1047128,  0.0590527,  0.0096822, -0.0402967, -0.0877437,
                   -0.1296773, -0.1634629, -0.1869775, -0.1987436, -0.1980219,
                   -0.1848578, -0.1600783, -0.1252406, -0.0825335, -0.0346406,
                    0.015429 ,  0.064529 ,  0.1095745,  0.1477351,  0.1766128,
                    0.1943934,  0.1999595,  0.1929614,  0.1738388,  0.1437933,
                    0.1047128,  0.0590527,  0.0096822, -0.0402967, -0.0877437,
                   -0.1296773, -0.1634629, -0.1869775, -0.1987436, -0.1980219,
                   -0.1848578, -0.1600783, -0.1252406, -0.0825335, -0.0346406,
                    0.015429 ,  0.064529 ,  0.1095745,  0.1477351,  0.1766128,
                    0.1943934,  0.1999595,  0.1929614,  0.1738388,  0.1437933,
                    0.1047128,  0.0590527,  0.0096822, -0.0402967, -0.0877437,
                   -0.1296773, -0.1634629, -0.1869775, -0.1987436,  2.164532 ,
                    0.       ])
nt.assert_array_almost_equal(tans, u[-1])
print('Test case passed!!!')
```

```python
# Hidden test
# No output will be produced
# 6 marks
```

(b) [2 Marks] Continuing on from Question 3(a). Plot the profile of each time step on the same set of axes, given the following inputs (be sure to use a legend indicating which iteration and curve is which):

```python
dx    = 0.01
dt    = 0.01
N     = 2
D     = 0.1
nu    = 1
alpha = 0
beta  = 0
ic    = lambda x: np.sin(8*np.pi*x)
```

```python
# 2 Marks
# YOUR CODE HERE
raise NotImplementedError()
```

### Question 3 - 15 Marks

(a) [10 Marks] Consider the PDE given by:
$$
v_t = D\left( v_{xx} + v_{yy}\right),
$$
$$
v(-1, y, t) = v(1, y, t) = 0,\ v(x, y, 0) = 0,
$$
\begin{equation}
v(x, -1, 0) = v(x, 1, 0) = 100, \ \ \ x \in (-1, 1)\ \ \text{i.e. endpoints excluded}
\end{equation}

Periodic boundary conditions should be imposed on $y = -1, y = 1$, i.e.$v(x,-1,t)=v(x,1,t)$ for all $t$ and $x$.

Write a function which implements an ADI scheme to solve the PDE given above. Specifically, the function should take as inputs, the step-sizes, `dx` and `dy`, the time-step `dt`, the constant `D`, and the number of iterations to perform `N`.

The function should return the 2D heatmap for each time-step, that is, a 3D array of `N + 1` time-steps.

```python
def heat2D_ADI(dx, dy, dt, D, N):
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx   = 0.2
dy   = 0.2
dt   = 0.1
D    = 0.1
N    = 5
tans = np.array([[  0., 100., 100., 100., 100., 100., 100., 100., 100., 100.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.,   0.],
       [  0., 100., 100., 100., 100., 100., 100., 100., 100., 100.,   0.]])
U    = heat2D_ADI(dx, dy, dt, D, N)
nt.assert_array_almost_equal(tans, U[:, :, 0])
print('Test case passed!!!')
```

```python
# Run this test cell to check your code
# Do not delete this cell
# 1 mark
# Unit test
dx   = 0.2
dy   = 0.2
dt   = 0.1
D    = 0.1
N    = 5
tans = np.array([[ 0.       , 65.3025091, 78.3317661, 79.6479746, 79.780803 ,
        79.7928783, 79.780803 , 79.6479746, 78.3317661, 65.3025091,
         0.       ],
       [ 0.       , 14.8641016, 17.82981  , 18.1294042, 18.1596385,
        18.1623871, 18.1596385, 18.1294042, 17.82981  , 14.8641016,
         0.       ],
       [ 0.       ,  1.5015808,  1.8011785,  1.8314437,  1.834498 ,
         1.8347756,  1.834498 ,  1.8314437,  1.8011785,  1.5015808,
         0.       ],
       [ 0.       ,  0.1517061,  0.1819747,  0.1850325,  0.185341 ,
         0.1853691,  0.185341 ,  0.1850325,  0.1819747,  0.1517061,
         0.       ],
       [ 0.       ,  0.0154802,  0.0185689,  0.0188809,  0.0189124,
         0.0189152,  0.0189124,  0.0188809,  0.0185689,  0.0154802,
         0.       ],
       [ 0.       ,  0.003096 ,  0.0037138,  0.0037762,  0.0037825,
         0.003783 ,  0.0037825,  0.0037762,  0.0037138,  0.003096 ,
         0.       ],
       [ 0.       ,  0.0154802,  0.0185689,  0.0188809,  0.0189124,
         0.0189152,  0.0189124,  0.0188809,  0.0185689,  0.0154802,
         0.       ],
       [ 0.       ,  0.1517061,  0.1819747,  0.1850325,  0.185341 ,
         0.1853691,  0.185341 ,  0.1850325,  0.1819747,  0.1517061,
         0.       ],
       [ 0.       ,  1.5015808,  1.8011785,  1.8314437,  1.834498 ,
         1.8347756,  1.834498 ,  1.8314437,  1.8011785,  1.5015808,
         0.       ],
       [ 0.       , 14.8641016, 17.82981  , 18.1294042, 18.1596385,
        18.1623871, 18.1596385, 18.1294042, 17.82981  , 14.8641016,
         0.       ],
       [ 0.       , 65.3025091, 78.3317661, 79.6479746, 79.780803 ,
        79.7928783, 79.780803 , 79.6479746, 78.3317661, 65.3025091,
         0.       ]])

U = heat2D_ADI(dx, dy, dt, D, N)
nt.assert_array_almost_equal(tans, U[:, :, 1])
print('Test case passed!!!')
```

```python
# Hidden test
# No output will be produced
# 8 marks
```

(b) [2 Marks] Plot a heatmap of the final time-step using the ADI function in Question 2(a), given the input below:

```python
dx = 0.1
dy = 0.1
dt = 0.1
D  = 0.2
N  = 5
U  = heat2D_ADI(dx, dy, dt, D, N)
```

```python
# 2 Marks
# YOUR CODE HERE
raise NotImplementedError()
```

(c) [3 Marks] Modify your function from Question 3(a) to plot the two tridiagonal ADI matrices $A$ and $B$.

```python
#3 Marks
# YOUR CODE HERE
raise NotImplementedError()
```


## Differentiation and Derivatives
So far in this course, you've learned how to evaluate limits for points on a line. Now you're going to build on that knowledge and look at a calculus technique called *differentiation*. In differentiation, we use our knowledge of limits to calculate the *derivative* of a function in order to determine the rate of change at an individual point on a line.

Let's remind ourselves of the problem we're trying to solve, here's a function:

\begin{equation}f(x) = x^{2} + x\end{equation}

We can visualize part of the line that this function defines using the folllowing Python code:

```python
%matplotlib inline

# Here's the function
def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

plt.show()
```

Now, we know that we can calculate the average rate of change for a given interval on the line by calculating the slope for a secant line that connects two points on the line. For example, we can calculate the average change for the interval between x=4 and x=6 by dividing the change (or *delta*, indicated as Δ) in the value of *f(x)* by the change in the value of *x*:

\begin{equation}m = \frac{\Delta{f(x)}}{\Delta{x}} \end{equation}

The delta for *f(x)* is calculated by subtracting the *f(x)* values of our points, and the delta for *x* is calculated by subtracting the *x* values of our points; like this:

\begin{equation}m = \frac{f(x)_{2} - f(x)_{1}}{x_{2} - x_{1}} \end{equation}

So for the interval between x=4 and x=6, that's:

\begin{equation}m = \frac{f(6) - f(4)}{6 - 4} \end{equation}

We can calculate and plot this using the following Python:

```python
%matplotlib inline

def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the a values
x1 = 4
x2 = 6

# Get the corresponding f(x) values
y1 = f(x1)
y2 = f(x2)

# Calculate the slope by dividing the deltas
a = (y2 - y1)/(x2 - x1)

# Create an array of x values for the secant line
sx = [x1,x2]

# Use the function to get the y values
sy = [f(i) for i in sx]

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the interval points
plt.scatter([x1,x2],[y1,y2], c='red')

# Plot the secant line
plt.plot(sx,sy, color='magenta')

# Display the calculated average rate of change
plt.annotate('Average change =' + str(a),(x2, (y2+y1)/2))

plt.show()
```

The average rate of change for the interval between x=4 and x=6 is 11/1 (or simply 11), meaning that for every **1** added to *x*, *f(x)* increases by **11**. Put another way, if x represents time in seconds and f(x) represents distance in meters, the average rate of change for distance over time (in other words, *velocity*) for the 4 to 6 second interval is 11 meters-per-second.

So far, this is just basic algebra; but what if instead of the average rate of change over an interval, we want to calculate the rate of change at a single point, say, where x = 4.5?

One approach we could take is to create a secant line between the point at which we want the slope and another point on the function line that is infintesimally close to it. So close in fact that the secant line is actually a tangent that goes through both points. We can then calculate the slope for the secant line as before. This would look something like the graph produced by the following code:

```python
%matplotlib inline

def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the x1 point, arbitrarily 5
x1 = 4.5
y1 = f(x1)

# Set the x2 point, very close to x1
x2 = 5.000000001
y2 = f(x2)

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the point
plt.scatter(x1,y1, c='red')
plt.annotate('x' + str(x1),(x1,y1), xytext=(x1-0.5, y1+3))

# Approximate the tangent slope and plot it
m = (y2-y1)/(x2-x1)
xMin = x1 - 3
yMin = y1 - (3*m)
xMax = x1 + 3
yMax = y1 + (3*m)
plt.plot([xMin,xMax],[yMin,yMax], color='magenta')

plt.show()
```

## Calculating a Derivative
In the Python code above, we created the (almost) tangential secant line by specifying a point that is very close to the point at which we want to calculate the rate of change. This is adequate to show the line conceptually in the graph, but it's not a particularly generalizable (or accurate) way to actually calculate the line so that we can get the rate of change at any given point.

If only we knew of a way to calculate a point on the line that is as close as possible to point with a given *x* value.

Oh wait, we do! It's a *limit*.

So how do we apply a limit in this scenario? Well, let's start by examining our general approach to calculating slope in a little more detail.Our tried and tested approach is to plot a secant line between two points at different values of x, so let's plot an arbitrary (*x,y*) point, and then add an arbitrary amount to *x*, which we'll call *h*. Then we know that we can plot a secant line between (*x,f(x)*) and (*x+h,f(x+h)*) and find its slope.

Run the cell below to see these points:

```python
%matplotlib inline

def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the x point
x1 = 3
y1 = f(x1)

# set the increment
h = 3

# set the x+h point
x2 = x1+h
y2 = f(x2)

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the x point
plt.scatter(x1,y1, c='red')
plt.annotate('(x,f(x))',(x1,y1), xytext=(x1-0.5, y1+3))

# Plot the x+h point
plt.scatter(x2,y2, c='red')
plt.annotate('(x+h, f(x+h))',(x2,y2), xytext=(x2+0.5, y2))

plt.show()
```

As we saw previously, our formula to calculate slope is:

\begin{equation}m = \frac{\Delta{f(x)}}{\Delta{x}} \end{equation}

The delta for *f(x)* is calculated by subtracting the *f(x + h)* and *f(x)* values of our points, and the delta for *x* is just the difference between *x* and *x + h*; in other words, *h*:

\begin{equation}m = \frac{f(x + h) - f(x)}{h} \end{equation}

What we actually need is the slope at the shortest possible distance between x and x+h, so we're looking for the smallest possible value of *h*. In other words, we need the limit as *h* approaches 0.

\begin{equation}\lim_{h \to 0} \frac{f(x + h) - f(x)}{h} \end{equation}

This equation is generalizable, and we can use it as the definition of a function to help us find the slope at any given value of *x* on the line, and it's what we call the *derivative* of our original function (which in this case is called *f*). This is generally indicated in *Lagrange* notation like this:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h} \end{equation}

You'll also sometimes see derivatives written in *Leibniz's* notation like this:

\begin{equation}\frac{d}{dx}f(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h} \end{equation}

***Note:*** *Some textbooks use **h** to symbolize the difference between **x0** and **x1**, while others use **Δx**. It makes no diffrerence which symbolic value you use.*

#### Alternate Form for a Derivative
The formula above shows the generalized form for a derivative. You can use the derivative function to get the slope at any given point, for example to get the slope at point *a* you could just plug the value for *a* into the generalized derivative function:

\begin{equation}f'(\textbf{a}) = \lim_{h \to 0} \frac{f(\textbf{a} + h) - f(\textbf{a})}{h} \end{equation}

Or you could use the alternate form, which is specific to point *a*:

\begin{equation}f'(a) = \lim_{x \to a} \frac{f(x) - f(a)}{x - a} \end{equation}

These are mathematically equivalent.

### Finding the Derivative for a Specific Point
It's easier to understand differentiation by seeing it in action, so let's use it to find the derivitive for a specific point in the function ***f***.

Here's the definition of function ***f***:

\begin{equation}f(x) = x^{2} + x\end{equation}

Let's say we want to find ***f'(2)*** (the derivative for ***f*** when ***x*** is 2); so we're trying to find the slope at the point shown by the following code:

```python
%matplotlib inline

def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the point
x1 = 2
y1 = f(x1)

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the point
plt.scatter(x1,y1, c='red')
plt.annotate('(x,f(x))',(x1,y1), xytext=(x1-0.5, y1+3))

plt.show()
```

Here's our generalized formula for finding a derivative at a specific point (*a*):

\begin{equation}f'(a) = \lim_{h \to 0} \frac{f(a + h) - f(a)}{h} \end{equation}

So let's just start by plugging our *a* value in:

\begin{equation}f'(\textbf{2}) = \lim_{h \to 0} \frac{f(\textbf{2} + h) - f(\textbf{2})}{h} \end{equation}

We know that ***f(x)*** encapsulates the equation ***x2 + x***, so we can rewrite our derivative equation as:

\begin{equation}f'(2) = \lim_{h \to 0} \frac{((2+h)^{2} + 2 + h) - (2^{2} + 2)}{h} \end{equation}

We can apply the distribution property to ***(2 + h)2*** using the rule that *(a + b)2 = a2 + b2 + 2ab*:

\begin{equation}f'(2) = \lim_{h \to 0} \frac{(4 + h^{2} + 4h + 2 + h) - (2^{2} + 2)}{h} \end{equation}

Then we can simplify 22 + 2 (22 is 4, plus 2 gives is 6):

\begin{equation}f'(2) = \lim_{h \to 0} \frac{(4 + h^{2} + 4h + 2 + h) - 6}{h} \end{equation}

We can combine like terms on the left side of the numerator to make things a little clearer:

\begin{equation}f'(2) = \lim_{h \to 0} \frac{(h^{2} + 5h + 6) - 6}{h} \end{equation}

Which combines even further to get rid of the *6*:

\begin{equation}f'(2) = \lim_{h \to 0} \frac{h^{2} + 5h}{h} \end{equation}

And finally, we can simplify the fraction:

\begin{equation}f'(2) = \lim_{h \to 0} h + 5 \end{equation}

To get the limit when *h* is approaching 0, we can use direct substitution for h:

\begin{equation}f'(2) = 0 + 5 \end{equation}

so:

\begin{equation}f'(2) = 5 \end{equation}

Let's draw a tangent line with that slope on our graph to see if it looks right:

```python
%matplotlib inline

def f(x):
    return x**2 + x

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the point
x1 = 2
y1 = f(x1)

# Specify the derivative we calculated above
m = 5

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the point
plt.scatter(x1,y1, c='red')
plt.annotate('(x,f(x))',(x1,y1), xytext=(x1-0.5, y1+3))

# Plot the tangent line using the derivative we calculated
xMin = x1 - 3
yMin = y1 - (3*m)
xMax = x1 + 3
yMax = y1 + (3*m)
plt.plot([xMin,xMax],[yMin,yMax], color='magenta')

plt.show()
```

### Finding a Derivative for Any Point
Now let's put it all together and define a function that we can use to find the derivative for any point in the ***f*** function:

Here's our general derivative function again:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h} \end{equation}

We know that ***f(x)*** encapsulates the equation ***x2 + x***, so we can rewrite our derivative equation as:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{((x+h)^{2} + x + h) - (x^{2} + x)}{h} \end{equation}

We can apply the distribution property to ***(x + h)2*** using the rule that *(a + b)2 = a2 + b2 + 2ab*:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{(x^{2} + h^{2} + 2xh + x + h) - (x^{2} + x)}{h} \end{equation}

Then we can use the distributive property to expand ***- (x2 + x)***, which is the same thing as *-1(x2 + x)*, to ***- x2 - x***:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{x^{2} + h^{2} + 2xh + x + h - x^{2} - x}{h} \end{equation}

We can combine like terms on the numerator to make things a little clearer:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{h^{2} + 2xh + h}{h} \end{equation}

And finally, we can simplify the fraction:

\begin{equation}f'(x) = \lim_{h \to 0} 2x + h + 1 \end{equation}

To get the limit when *h* is approaching 0, we can use direct substitution for h:

\begin{equation}f'(x) = 2x + 0 + 1 \end{equation}

so:

\begin{equation}f'(x) = 2x + 1 \end{equation}

Now we have a function for the derivative of ***f***, which we can apply to any *x* value to find the slope of the function at ***f(x***).

For example, let's find the derivative of ***f*** with an *x* value of 5:

\begin{equation}f'(5) = 2\cdot5 + 1 = 10 + 1 = 11\end{equation}

Let's use Python to define the ***f(x)*** and ***f'(x)*** functions, plot ***f(5)*** and show the tangent line for ***f'(5)***:

```python
%matplotlib inline

# Create function f
def f(x):
    return x**2 + x

# Create derivative function for f
def fd(x):
    return (2 * x) + 1

from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(0, 11))

# Use the function to get the y values
y = [f(i) for i in x]

# Set the point
x1 = 5
y1 = f(x1)

# Calculate the derivative using the derivative function
m = fd(x1)

# Set up the graph
plt.xlabel('x')
plt.ylabel('f(x)')
plt.grid()

# Plot the function
plt.plot(x,y, color='green')

# Plot the point
plt.scatter(x1,y1, c='red')
plt.annotate('(x,f(x))',(x1,y1), xytext=(x1-0.5, y1+3))

# Plot the tangent line using the derivative we calculated
xMin = x1 - 3
yMin = y1 - (3*m)
xMax = x1 + 3
yMax = y1 + (3*m)
plt.plot([xMin,xMax],[yMin,yMax], color='magenta')

plt.show()
```

## Differentiability
It's important to realize that a function may not be *differentiable* at every point; that is, you might not be able to calculate the derivative for every point on the function line.

To be differentiable at a given point:
- The function must be *continuous* at that point.
- The tangent line at that point cannot be vertical
- The line must be *smooth* at that point (that is, it cannot take on a sudden change of direction at the point)

For example, consider the following (somewhat bizarre) function:

\begin{equation}
q(x) = \begin{cases}
  \frac{40,000}{x^{2}}, & \text{if } x < -4, \\
  (x^{2} -2) \cdot (x - 1), & \text{if } x \ne 0 \text{ and } x \ge -4 \text{ and } x < 8, \\
  (x^{2} -2), & \text{if } x \ne 0 \text{ and } x \ge 8
\end{cases}
\end{equation}

```python
%matplotlib inline

# Define function q
def q(x):
    if x != 0:
        if x < -4:
            return 40000 / (x**2)
        elif x < 8:
            return (x**2 - 2) * x - 1
        else:
            return (x**2 - 2)

# Plot output from function g
from matplotlib import pyplot as plt

# Create an array of x values
x = list(range(-10, -5))
x.append(-4.01)
x2 = list(range(-4,8))
x2.append(7.9999)
x2 = x2 + list(range(8,11))

# Get the corresponding y values from the function
y = [q(i) for i in x]
y2 = [q(i) for i in x2]

# Set up the graph
plt.xlabel('x')
plt.ylabel('q(x)')
plt.grid()

# Plot x against q(x)
plt.plot(x,y, color='purple')
plt.plot(x2,y2, color='purple')

plt.scatter(-4,q(-4), c='red')
plt.annotate('A (x= -4)',(-5,q(-3.9)), xytext=(-7, q(-3.9)))

plt.scatter(0,0, c='red')
plt.annotate('B (x= 0)',(0,0), xytext=(-1, 40))

plt.scatter(8,q(8), c='red')
plt.annotate('C (x= 8)',(8,q(8)), xytext=(8, 100))

plt.show()
```

The points marked on this graph are non-differentiable:
- Point **A** is non-continuous - the limit from the negative side is infinity, but the limit from the positive side ≈ -57
- Point **B** is also non-continuous - the function is not defined at x = 0.
- Point **C** is defined and continuous, but the sharp change in direction makes it non-differentiable.

## Derivatives of Equations

We've been talking about derivatves of *functions*, but it's important to remember that functions are just named operations that return a value. We can apply what we know about calculating derivatives to any equation, for example:

\begin{equation}\frac{d}{dx}(2x + 6)\end{equation}

Note that we generally switch to *Leibniz's* notation when finding derivatives of equations that are not encapsulated as functions; but the approach for solving this example is exactly the same as if we had a hypothetical function with the definition *2x + 6*:

\begin{equation}\frac{d}{dx}(2x + 6) = \lim_{h \to 0} \frac{(2(x+h) + 6) - (2x + 6)}{h} \end{equation}

After factoring out the* 2(x+h)* on the left and the *-(2x - 6)* on the right, this is:

\begin{equation}\frac{d}{dx}(2x + 6) = \lim_{h \to 0} \frac{2x + 2h + 6 - 2x - 6}{h} \end{equation}

We can simplify this to:

\begin{equation}\frac{d}{dx}(2x + 6) = \lim_{h \to 0} \frac{2h}{h} \end{equation}

Now we can factor *h* out entirely, so at any point:

\begin{equation}\frac{d}{dx}(2x + 6) = 2 \end{equation}

If you run the Python code below to plot the line created by the equation, you'll see that it does indeed have a constant slope of 2:

```python
%matplotlib inline
from matplotlib import pyplot as plt

# Create an array of x values from 0 to 10 to plot
x = list(range(1, 11))

# Use the function to get the y values
y = [(2*i) + 6 for i in x]

# Set up the graph
plt.xlabel('x')
plt.xticks(range(1,11, 1))
plt.ylabel('y')
plt.yticks(range(8,27, 1))
plt.grid()

# Plot the function
plt.plot(x,y, color='purple')

plt.show()
```

## Derivative Rules and Operations
When working with derivatives, there are some rules, or shortcuts, that you can apply to make your life easier.

### Basic Derivative Rules
Let's start with some basic rules that it's useful to know.

- If *f(x)* = *C* (where *C* is a constant), then *f'(x)* = 0

    This makes sense if you think about it for a second. No matter what value you use for *x*, the function returns the same constant value; so the graph of the function will be a horizontal line. There's no rate of change in a horiziontal line, so its slope is 0 at all points. This is true of any constant, including symbolic constants like *π* (pi).

    So, for example:

\begin{equation}f(x) = 6 \;\; \therefore \;\; f'(x) = 0 \end{equation}

    Or:

\begin{equation}f(x) = \pi \;\; \therefore \;\; f'(x) = 0 \end{equation}

- If *f(x)* = *Cg(x)*, then *f'(x)* = *Cg'(x)*

    This rule tells us that if a function is equal to a second function multiplied by a constant, then the derivative of the first function will be equal to the derivative of the second function multiplied by the same constant. For example:

\begin{equation}f(x) = 2g(x) \;\; \therefore \;\; f'(x) = 2g'(x) \end{equation}

- If *f(x)* = *g(x)* + *h(x)*, then *f'(x)* = *g'(x)* + *h'(x)*

    In other words, if a function is the sum of two other functions, then the derivative of the first function is the sum of the derivatives of the other two functions. For example:

\begin{equation}f(x) = g(x) + h(x) \;\; \therefore \;\; f'(x) = g'(x) + h'(x) \end{equation}

    Of course, this also applies to subtraction:

\begin{equation}f(x) = k(x) - l(x) \;\; \therefore \;\; f'(x) = k'(x) - l'(x) \end{equation}

As discussed previously, functions are just equations encapsulated as a named entity that return a value; and the rules can be applied to any equation. For example:

\begin{equation}\frac{d}{dx}(2x + 6) = \frac{d}{dx} 2x +  \frac{d}{dx} 6\end{equation}

So we can take advantage of the rules to make the calculation a little easier:

\begin{equation}\frac{d}{dx}(2x) = \lim_{h \to 0} \frac{2(x+h) - 2x}{h} \end{equation}

After factoring out the* 2(x+h)* on the left, this is:

\begin{equation}\frac{d}{dx}(2x) = \lim_{h \to 0} \frac{2x + 2h - 2x}{h} \end{equation}

We can simplify this to:

\begin{equation}\frac{d}{dx}(2x) = \lim_{h \to 0} \frac{2h}{h} \end{equation}

Which gives us:

\begin{equation}\frac{d}{dx}(2x) = 2 \end{equation}

Now we can turn our attention to the derivative of the constant 6 with respect to *x*, and we know that the derivative of a constant is always 0, so:

\begin{equation}\frac{d}{dx}(6) = 0\end{equation}

We add the two derivatives we calculated:

\begin{equation}\frac{d}{dx}(2x + 6) = 2 + 0\end{equation}

Which gives us our result:

\begin{equation}\frac{d}{dx}(2x + 6) = 2\end{equation}

### The Power Rule
The *power rule* is one of the most useful shortcuts in the world of differential calculus. It can be stated like this:

\begin{equation}f(x) = x^{n} \;\; \therefore \;\; f'(x) = nx^{n-1}\end{equation}

So if our function for *x* returns *x* to the power of some constant (which we'll call *n*), then the derivative of the function for *x* is *n* times *x* to the power of *n* - 1.

It's probably helpful to look at a few examples to see how this works:

\begin{equation}f(x) = x^{3} \;\; \therefore \;\; f'(x) = 3x^{2}\end{equation}

\begin{equation}f(x) = x^{-2} \;\; \therefore \;\; f'(x) = -2x^{-3}\end{equation}

\begin{equation}f(x) = x^{2} \;\; \therefore \;\; f'(x) = 2x\end{equation}

In each of these examples, the exponential of *x* in the function definition becomes the coefficient for *x* in the derivative definition, with the exponential is decremented by 1.

Here's a worked example to find the derivative of the following function:

\begin{equation}f(x) = x^{2}\end{equation}

So we start with the general derivative function:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{f(x + h) - f(x)}{h} \end{equation}

We can plug in our definition for *f*:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{(x + h)^{2} - x^{2}}{h} \end{equation}

Now we can factor out the perfect square binomial on the left:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{x^{2} + h^{2} + 2xh - x^{2}}{h} \end{equation}

The x2 terms cancel each other out so we get to:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{h^{2} + 2xh}{h} \end{equation}

Which simplifies to:

\begin{equation}f'(x) = \lim_{h \to 0} h + 2x \end{equation}

With *h* approaching 0, this is:

\begin{equation}f'(x) = 0 + 2x \end{equation}

So our answer is:

\begin{equation}f'(x) = 2x \end{equation}

Note that we could have achieved the same result by simply applying the power rule and transforming x2 to 2x1 (which is the same as 2x).

### The Product Rule
The product rule can be stated as:

\begin{equation}\frac{d}{dx}[f(x)g(x)] = f'(x)g(x) + f(x)g'(x) \end{equation}

OK, let's break that down. What it's saying is that the derivative of *f(x)* multiplied by *g(x)* is equal to the derivative of *f(x)* multiplied by the value of *g(x)* added to the value of *f(x)* multiplied by the derivative of *g(x*).

Let's see an example based on the following two functions:

\begin{equation}f(x) = 2x^{2} \end{equation}

\begin{equation}g(x) = x + 1 \end{equation}

Let's start by calculating the derivative of *f(x)*:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{2(x + h)^{2} - 2x^{2}}{h} \end{equation}

This factors out to:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{2x^{2} + 2h^{2} + 4xh - 2x^{2}}{h} \end{equation}

Which when we cancel out the 2x2 and -2x2 is:

\begin{equation}f'(x) = \lim_{h \to 0} \frac{2h^{2} + 4xh}{h} \end{equation}

Which simplifies to:

\begin{equation}f'(x) = \lim_{h \to 0} 2h + 4x \end{equation}

With *h* approaching 0, this is:

\begin{equation}f'(x) = 4x \end{equation}

Now let's look at *g'(x)*:

\begin{equation}g'(x) = \lim_{h \to 0} \frac{(x + h) + 1 - (x + 1)}{h} \end{equation}

We can just remove the brackets on the left and factor out the *-(x + 1)* on the right:

\begin{equation}g'(x) = \lim_{h \to 0} \frac{x + h + 1 - x - 1}{h} \end{equation}

Which can be cleaned up to:

\begin{equation}g'(x) = \lim_{h \to 0} \frac{h}{h} \end{equation}

Enabling us to factor *h* out completely to give a constant derivative of *1*:

\begin{equation}g'(x) = 1 \end{equation}

So now we can calculate the derivative for the product of these functions by plugging the functions and the derivatives we've calculated for them into the product rule equation:

\begin{equation}\frac{d}{dx}[f(x)g(x)] = f'(x)g(x) + f(x)g'(x) \end{equation}

So:

\begin{equation}\frac{d}{dx}[f(x)g(x)] = (4x \cdot (x + 1)) + (2x^{2} \cdot 1) \end{equation}

Which can be simplified to:

\begin{equation}\frac{d}{dx}[f(x)g(x)] = (4x^{2} + 4x) + 2x^{2} \end{equation}

Which can be further simplified to:

\begin{equation}\frac{d}{dx}[f(x)g(x)] = 6x^{2} + 4x \end{equation}

### The Quotient Rule
The *quotient rule* applies to functions that are defined as a quotient of one expression divided by another; for example:

\begin{equation}r(x) = \frac{s(x)}{t(x)} \end{equation}

In this situation, you can apply the following quotient rule to find the derivative of *r(x)*:

\begin{equation}r'(x) = \frac{s'(x)t(x) - s(x)t'(x)}{[t(x)]^{2}} \end{equation}

Here are our definitions for *s(x)* and *t(x)*:

\begin{equation}s(x) = 3x^{2} \end{equation}

\begin{equation}t(x) = 2x\end{equation}

Let's start with *s'(x)*:

\begin{equation}s'(x) = \lim_{h \to 0} \frac{3(x + h)^{2} - 3x^{2}}{h} \end{equation}

This factors out to:

\begin{equation}s'(x) = \lim_{h \to 0} \frac{3x^{2} + 3h^{2} + 6xh - 3x^{2}}{h} \end{equation}

Which when we cancel out the 3x2 and -3x2 is:

\begin{equation}s'(x) = \lim_{h \to 0} \frac{3h^{2} + 6xh}{h} \end{equation}

Which simplifies to:

\begin{equation}s'(x) = \lim_{h \to 0} 3h + 6x \end{equation}

With *h* approaching 0, this is:

\begin{equation}s'(x) = 6x \end{equation}

Now let's look at *t'(x)*:

\begin{equation}t'(x) = \lim_{h \to 0} \frac{2(x + h) - 2x}{h} \end{equation}

We can just factor out the *2(x + h)* on the left:

\begin{equation}t'(x) = \lim_{h \to 0} \frac{2x + 2h - 2x}{h} \end{equation}

Which can be cleaned up to:

\begin{equation}t'(x) = \lim_{h \to 0} \frac{2h}{h} \end{equation}

Enabling us to factor *h* out completely to give a constant derivative of *2*:

\begin{equation}t'(x) = 2 \end{equation}

So now we can calculate the derivative for the quotient of these functions by plugging the function definitions and the derivatives we've calculated for them into the quotient rule equation:

\begin{equation}r'(x) = \frac{(6x \cdot 2x) - (3x^{2} \cdot 2)}{[2x]^{2}} \end{equation}

We can factor out the numerator terms like this:

\begin{equation}r'(x) = \frac{12x^{2} - 6x^{2}}{[2x]^{2}} \end{equation}

Which can then be combined:

\begin{equation}r'(x) = \frac{6x^{2}}{[2x]^{2}} \end{equation}

The denominator is [2x]2 (note that this is different from 2x2. [2x]2 is 2x • 2x, whereas  2x2 is 2 • x2):

\begin{equation}r'(x) = \frac{6x^{2}}{4x^{2}} \end{equation}

Which simplifies to:

\begin{equation}r'(x) = 1\frac{1}{2} \end{equation}

So the derivative of *r(x)* is 1.5.

### The Chain Rule

The *chain rule* takes advantage of the fact that equations can be encapsulated as functions, and since functions contain equations, it's possible to nest one function within another.

For example, consider the following function:

\begin{equation}u(x) = 2x^{2} \end{equation}

We could view the definition of *u(x)* as a composite of two functions,; an *inner* function that calculates x2, and an *outer* function that multiplies the result of the inner function by 2.

\begin{equation}u(x) = \widehat{\color{blue}2\color{blue}(\underline{\color{red}x^{\color{red}2}}\color{blue})} \end{equation}

To make things simpler, we can name these inner and outer functions:

\begin{equation}i(x) = x^{2} \end{equation}

\begin{equation}o(x) = 2x \end{equation}

Note that *x* indicates the input for each function. Function *i* takes its input and squares it, and function *o* takes its input and multiplies it by 2. When we use these as a composite function, the *x* value input into the outer function will be the output from the inner function.

Let's take a look at how we can apply these functions to get back to our original *u* function:

\begin{equation}u(x) = o(i(x)) \end{equation}

So first we need to find the output of the inner *i* function so we can use at as the input value for the outer *o* function. Well, that's easy, we know the definition of *i* (square the input), so we can just plug it in:

\begin{equation}u(x) = o(x^{2}) \end{equation}

We also know the definition for the outer *o* function (multiply the input by 2), so we can just apply that to the input:

\begin{equation}u(x) = 2x^{2} \end{equation}

OK, so now we know how to form a composite function. The *chain rule* can be stated like this:

\begin{equation}\frac{d}{dx}[o(i(x))] = o'(i(x)) \cdot i'(x)\end{equation}

Alright, let's start by plugging the output of the inner *i(x)* function in:

\begin{equation}\frac{d}{dx}[o(i(x))] = o'(x^{2}) \cdot i'(x)\end{equation}

Now let's use that to calculate the derivative of *o*, replacing each *x* in the equation with the output from the *i* function (*x2*):

\begin{equation}o'(x) = \lim_{h \to 0} \frac{2(x^{2} + h) - 2x^{2}}{h} \end{equation}

This factors out to:

\begin{equation}o'(x) = \lim_{h \to 0} \frac{2x^{2} + 2h - 2x^{2}}{h} \end{equation}

Which when we cancel out the 2x2 and -2x2 is:

\begin{equation}o'(x) = \lim_{h \to 0} \frac{2h}{h} \end{equation}

Which simplifies to:

\begin{equation}o'(x) = 2 \end{equation}

Now we can calculate *i'(x)*. We know that the definition of *i(x)* is x2, and we can use the power rule to determine that *i'(x)* is therefore 2x.

So our equation at this point is:

\begin{equation}\frac{d}{dx}[o(i(x))] = 2 \cdot 2x\end{equation}

Which is:

\begin{equation}\frac{d}{dx}[o(i(x))] = 4x\end{equation}

Commonly, the chain rule is stated using a slighly different notation that you may find easier to work with. In this case, we can take our equation:

\begin{equation}\frac{d}{dx}[o(i(x))] = o'(i(x)) \cdot i'(x)\end{equation}

and rewrite it as

\begin{equation}\frac{du}{dx} = \frac{do}{di}\frac{di}{dx}\end{equation}

We can then complete the calculations like this:

\begin{equation}\frac{du}{dx} = 2 \cdot 2x = 4x\end{equation}

```python

```

```python

```

Basics of Calculus by Hiren/02-03-Differentiation and Derivatives.ipynb


# Taylor approximations to color conversion

This notebook shows how to come up with all these magic constants that appear in the approximations to `LinearRgb` in my [go-colorful](https://github.com/lucasb-eyer/go-colorful) library in order to speed them up at almost no loss in accuracy.

The gist is to compute a Taylor expansion up to a degree that gives enough accuracy.
Taylor expansions work well for relatively linear-ish functions, as `LinearRgb` is.

Doing this is especially easy thanks to the SymPy library which has symbolic Taylor expansion built-in!

```python
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import matplotlib as mpl
import matplotlib.pyplot as plt
plt.style.use('ggplot')

import numpy as np

from sympy import *
init_printing()
```

The following is the conversion from RGB to linear RGB (aka. gamma-correction), where I'm dropping the conditional part for very small values of `x` as a first approximation.

```python
def linear_rgb(x):
    return ((x+0.055)/1.055)**2.4
```

Now, we can use SymPy to create a symbolic version of that equation, and compute a symbolic Taylor expansion around $0.5$ (the middle of our target range) up to the fourth degree:

```python
x = Symbol('x', real=True)
series(linear_rgb(x), x, x0=0.5, n=4)
```

In order to use it numerically, we will "drop the O", which means do the actual approximation, and "lambdify" the function, which turns a symbolic function into a NumPy function:

```python
fast_linear_rgb = lambdify([x], series(linear_rgb(x), x, x0=0.5, n=4).removeO())
```

As additional heuristic approximations, we'll include simply squaring the values, which should also be very fast, but quite wrong. Then, plot all these functions in order to see their behaviour, and compute errors:

```python
X = np.linspace(0,1,1001)

ref = linear_rgb(X)  # The (almost) correct implementation.
fast = fast_linear_rgb(X)  # The Taylor approximation
square = X*X  # The approximation by squaring.

fig, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize=(20,4))
ax1.plot(X, ref, label='linear')
ax1.plot(X, fast, label='fast (max/avg err: {:.4f} / {:.4f})'.format(np.max(np.abs(ref - fast)),
                                                                     np.mean(np.abs(ref - fast))))
ax1.plot(X, square, label='square (max/avg err: {:.4f} / {:.4f})'.format(np.max(np.abs(ref - square)),
                                                                         np.mean(np.abs(ref - square))))
ax2.plot(X, ref)
ax2.plot(X, fast)
ax2.plot(X, square)
ax2.set_xlim(0, 0.1)
ax2.set_ylim(0, 0.1)
ax2.set_title("Left end")
ax3.plot(X, ref)
ax3.plot(X, fast, ls=':')
ax3.plot(X, square)
ax3.set_xlim(0.45, 0.55)
ax3.set_ylim(0.15, 0.25)
ax3.set_title("Middle")
ax4.plot(X, ref)
ax4.plot(X, fast)
ax4.plot(X, square)
ax4.set_xlim(0.9, 1)
ax4.set_ylim(0.9, 1)
ax4.set_title("Right end")

ax1.legend();
```

# The inverse function (for Lab->RGB)

The inverse function is significantly more difficult, because its left part is highly non-linear and changes much faster than the rest of the function.

So what we do here, in order to keep reasonable accuracy, is split it into three parts with three different approximations. You will notice that the leftmost part has quite large coefficients, which hints to the approximation being worse/"harder".

```python
def delinear_rgb(x):
    return 1.055*(x**(1.0/2.4)) - 0.055

fast_delinear_rgb_part1 = lambdify([x], series(delinear_rgb(x), x, x0=0.015, n=6).removeO())
fast_delinear_rgb_part2 = lambdify([x], series(delinear_rgb(x), x, x0=0.03, n=6).removeO())
fast_delinear_rgb_part3 = lambdify([x], series(delinear_rgb(x), x, x0=0.6, n=6).removeO())
```

```python
ref = delinear_rgb(X)
fast1 = fast_delinear_rgb_part1(X)
fast2 = fast_delinear_rgb_part2(X)
fast3 = fast_delinear_rgb_part3(X)
sqrt = np.sqrt(X)

def plot(ax):
    ax.plot(X, ref, label='linear')
    l, = ax.plot(X, fast1, label='fast, part1', ls=':')
    ax.plot(X, fast2, label='fast, part2', c=l.get_color(), ls='--')
    ax.plot(X, fast3, label='fast, part3', c=l.get_color(), ls='-')
    ax.plot(X, sqrt, label='sqrt')

fig, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize=(20,4))
plot(ax1)
ax1.set_ylim(0, 1)
plot(ax2)
ax2.set_xlim(0, 0.05)
ax2.set_ylim(0, 0.25)
ax2.set_title("Left end")
plot(ax3)
ax3.set_xlim(0.45, 0.55)
ax3.set_ylim(0.65, 0.75)
ax3.set_title("Middle")
plot(ax4)
ax4.set_xlim(0.95, 1)
ax4.set_ylim(0.95, 1)
ax4.set_title("Right end")

ax1.legend()
```