# SARIMAX and ARIMA: Frequently Asked Questions (FAQ)

This notebook contains explanations for frequently asked questions.

* Comparing trends and exogenous variables in `SARIMAX`, `ARIMA` and `AutoReg`
* Reconstructing residuals, fitted values and forecasts in `SARIMAX` and `ARIMA`
* Initial residuals in `SARIMAX` and `ARIMA`

## Comparing trends and exogenous variables in `SARIMAX`, `ARIMA` and `AutoReg`

`ARIMA` are formally OLS with ARMA errors.  A basic AR(1) in the OLS with ARMA errors is described as

$$
\begin{align}
Y_t & = \delta + \epsilon_t \\
\epsilon_t & = \rho \epsilon_{t-1} + \eta_t \\
\eta_t & \sim WN(0,\sigma^2) \\
\end{align}
$$

In large samples, $\hat{\delta}\stackrel{p}{\rightarrow} E[Y]$.

`SARIMAX` uses a different representation, so that the model when estimated using `SARIMAX` is

$$
\begin{align}
Y_t & = \phi + \rho Y_{t-1} + \eta_t \\
\eta_t & \sim WN(0,\sigma^2) \\
\end{align}
$$

This is the same representation that is used when the model is estimated using OLS (`AutoReg`). In large samples, $\hat{\phi}\stackrel{p}{\rightarrow} E[Y](1-\rho)$.

In the next cell, we simulate a large sample and verify that these relationship hold in practice.

```python
%matplotlib inline
```

```python
import numpy as np
import pandas as pd

rng = np.random.default_rng(20210819)
eta = rng.standard_normal(5200)
rho = 0.8
beta = 10
epsilon = eta.copy()
for i in range(1, eta.shape[0]):
    epsilon[i] = rho * epsilon[i - 1] + eta[i]
y = beta + epsilon
y = y[200:]
```

```python
from statsmodels.tsa.api import SARIMAX, AutoReg
from statsmodels.tsa.arima.model import ARIMA
```

The three models are specified and estimated in the next cell.  An AR(0) is included as a reference. The AR(0) is identical using all three estimators.

```python
ar0_res = SARIMAX(y, order=(0, 0, 0), trend="c").fit()
sarimax_res = SARIMAX(y, order=(1, 0, 0), trend="c").fit()
arima_res = ARIMA(y, order=(1, 0, 0), trend="c").fit()
autoreg_res = AutoReg(y, 1, trend="c").fit()
```

The table below contains the estimated parameter in the model, the estimated AR(1) coefficient, and the long-run mean which is either equal to the estimated parameters (AR(0) or `ARIMA`), or depends on the ratio of the intercept to 1 minus the AR(1) parameter.

```python
intercept = [
    ar0_res.params[0],
    sarimax_res.params[0],
    arima_res.params[0],
    autoreg_res.params[0],
]
rho_hat = [0] + [r.params[1] for r in (sarimax_res, arima_res, autoreg_res)]
long_run = [
    ar0_res.params[0],
    sarimax_res.params[0] / (1 - sarimax_res.params[1]),
    arima_res.params[0],
    autoreg_res.params[0] / (1 - autoreg_res.params[1]),
]
cols = ["AR(0)", "SARIMAX", "ARIMA", "AutoReg"]
pd.DataFrame(
    [intercept, rho_hat, long_run],
    columns=cols,
    index=["delta-or-phi", "rho", "long-run mean"],
)
```

### Differences between trend and exog in `SARIMAX`

When `SARIMAX` includes `exog` variables, then the `exog` are treated as OLS regressors, so that the model estimated is

$$
\begin{align}
Y_t - X_t \beta & = \delta + \rho (Y_{t-1} - X_{t-1}\beta) + \eta_t \\
\eta_t & \sim WN(0,\sigma^2) \\
\end{align}
$$

In the next example, we omit the trend and instead include a column of 1, which produces a model that is equivalent, in large samples, to the case with no exogenous regressor and `trend="c"`. Here the estimated value of `const` matches the value estimated using `ARIMA`. This happens since both exog in `SARIMAX` and the trend in `ARIMA` are treated as linear regression models with ARMA errors.

```python
sarimax_exog_res = SARIMAX(y, exog=np.ones_like(y), order=(1, 0, 0), trend="n").fit()
print(sarimax_exog_res.summary())
```

### Using `exog` in `SARIMAX` and `ARIMA`

While `exog` are treated the same in both models, the intercept continues to differ.  Below we add an exogenous regressor to `y` and then fit the model using all three methods. The data generating process is now

$$
\begin{align}
Y_t & = \delta + X_t \beta + \epsilon_t \\
\epsilon_t & = \rho \epsilon_{t-1} + \eta_t \\
\eta_t & \sim WN(0,\sigma^2) \\
\end{align}
$$

```python
full_x = rng.standard_normal(eta.shape)
x = full_x[200:]
y += 3 * x
```

```python
sarimax_exog_res = SARIMAX(y, exog=x, order=(1, 0, 0), trend="c").fit()
arima_exog_res = ARIMA(y, exog=x, order=(1, 0, 0), trend="c").fit()
```

Examining the parameter tables, we see that the parameter estimates on `x1` are identical while the estimates of the `intercept` continue to differ due to the differences in the treatment of trends in these estimators.

#### `SARIMAX`

```python
def print_params(s):
    from io import StringIO

    return pd.read_csv(StringIO(s.tables[1].as_csv()), index_col=0)

print_params(sarimax_exog_res.summary())
```

#### `ARIMA`

```python
print_params(arima_exog_res.summary())
```

### `exog` in `AutoReg`

When using `AutoReg` to estimate a model using OLS, the model differs from both `SARIMAX` and `ARIMA`. The `AutoReg` specification with exogenous variables is

$$
\begin{align}
Y_t & = \phi + \rho Y_{t-1} + X_{t}\beta + \eta_t \\
\eta_t & \sim WN(0,\sigma^2) \\
\end{align}
$$

This specification is not equivalent to the specification estimated in `SARIMAX` and `ARIMA`. Here the difference is non-trivial, and naive estimation on the same time series results in different parameter values, even in large samples (and the limit). Estimating this model changes the parameter estimates on the AR(1) coefficient.

#### `AutoReg`

```python
autoreg_exog_res = AutoReg(y, 1, exog=x, trend="c").fit()
print_params(autoreg_exog_res.summary())
```

The key difference can be seen by writing the model in lag operator notation.

$$
\begin{align}
(1-\phi L ) Y_t & = X_{t}\beta + \eta_t \Rightarrow \\
Y_t & = (1-\phi L )^{-1}\left(X_{t}\beta + \eta_t\right) \\
Y_t & = \sum_{i=0}^{\infty} \phi^i \left(X_{t-i}\beta + \eta_{t-i}\right)
\end{align}
$$

where it is is assumed that $|\phi|<1$.  Here we see that $Y_t$ depends on all lagged values of $X_t$ and $\eta_t$.  This differs from the specification estimated by `SARIMAX` and `ARIMA`, which can be seen to be

$$
\begin{align}
Y_t - X_t \beta & = \delta + \rho (Y_{t-1} - X_{t-1}\beta) + \eta_t \\
\left(1-\rho L \right)\left(Y_t - X_t  \beta\right) & = \delta +  \eta_t \\
Y_t - X_t  \beta & = \frac{\delta}{1-\rho} +  \left(1-\rho L \right)^{-1}\eta_t \\
Y_t - X_t  \beta & = \frac{\delta}{1-\rho} +  \sum_{i=0}^\infty \rho^i \eta_{t-i} \\
Y_t  & = \frac{\delta}{1-\rho} + X_t  \beta +  \sum_{i=0}^\infty \rho^i \eta_{t-i} \\
\end{align}
$$

In this specification, $Y_t$ only depends on $X_t$ and no other lags.

### Using the correct DGP with `AutoReg`

Simulating the process that is estimated in `AutoReg` shows that the parameters are recovered from the true model.

```python
y = beta + eta
epsilon = eta.copy()
for i in range(1, eta.shape[0]):
    y[i] = beta * (1 - rho) + rho * y[i - 1] + 3 * full_x[i] + eta[i]
y = y[200:]
```

#### `AutoReg` with correct DGP

```python
autoreg_alt_exog_res = AutoReg(y, 1, exog=x, trend="c").fit()
print_params(autoreg_alt_exog_res.summary())
```

## Reconstructing residuals, fitted values and forecasts in `SARIMAX` and `ARIMA`

In models that contain only autoregressive terms, trends and exogenous variables, fitted values and forecasts can be easily reconstructed once the maximum lag length in the model has been reached.  In practice, this means after $(P+D)s+p+d$ periods. Earlier predictions and residuals are harder to reconstruct since the model builds the best prediction for $Y_t|Y_{t-1},Y_{t-2},...$.  When the number of lags of $Y$ is less than the autoregressive order, then the expression for the optimal prediction differs from the model.  For example, when predicting the very first value, $Y_1$, there is no information available from the history of $Y$, and so the best prediction is the unconditional mean. In the case of an AR(1), the second prediction will follow the model, so that when using `ARIMA`, the prediction is

$$
Y_2 = \hat{\delta} + \hat{\rho} \left(Y_1 - \hat{\delta}\right)
$$

since `ARIMA` treats both exogenous and trend terms as regression with ARMA errors.

This can be seen in the next set of cells.

```python
arima_res = ARIMA(y, order=(1, 0, 0), trend="c").fit()
print_params(arima_res.summary())
```

```python
arima_res.predict(0, 2)
```

```python
delta_hat, rho_hat = arima_res.params[:2]
delta_hat + rho_hat * (y[0] - delta_hat)
```

`SARIMAX` treats trend terms differently, and so the one-step forecast from a model estimated using `SARIMAX` is

$$
Y_2 = \hat\delta + \hat\rho Y_1
$$

```python
sarima_res = SARIMAX(y, order=(1, 0, 0), trend="c").fit()
print_params(sarima_res.summary())
```

```python
sarima_res.predict(0, 2)
```

```python
delta_hat, rho_hat = sarima_res.params[:2]
delta_hat + rho_hat * y[0]
```

### Prediction with MA components

When a model contains a MA component, the prediction is more complicated since errors are never directly observable.  The prediction is still $Y_t|Y_{t-1},Y_{t-2},...$, and when the MA component is invertible, then the optimal prediction can be represented as a $t$-lag AR process. When $t$ is large, this should be very close to the prediction as if the errors were observable. For short lags, this can differ markedly.

In the next cell we simulate an MA(1) process, and fit an MA model.

```python
rho = 0.8
beta = 10
epsilon = eta.copy()
for i in range(1, eta.shape[0]):
    epsilon[i] = rho * eta[i - 1] + eta[i]
y = beta + epsilon
y = y[200:]

ma_res = ARIMA(y, order=(0, 0, 1), trend="c").fit()
print_params(ma_res.summary())
```

We start by looking at predictions near the beginning of the sample corresponding `y[1]`, ..., `y[5]`.

```python
ma_res.predict(1, 5)
```

and the corresponding residuals that are needed to produce the "direct" forecasts

```python
ma_res.resid[:5]
```

Using the model parameters, we can produce the "direct" forecasts using the MA(1) specification

$$
\hat Y_t = \hat\delta + \hat\rho \hat\epsilon_{t-1}
$$

We see that these are not especially close to the actual model predictions for the initial forecasts, but that the gap quickly reduces.

```python
delta_hat, rho_hat = ma_res.params[:2]
direct = delta_hat + rho_hat * ma_res.resid[:5]
direct
```

The difference is nearly a standard deviation for the first but declines as the index increases.

```python
ma_res.predict(1, 5) - direct
```

We next look at the end of the sample and the final three predictions.

```python
t = y.shape[0]
ma_res.predict(t - 3, t - 1)
```

```python
ma_res.resid[-4:-1]
```

```python
direct = delta_hat + rho_hat * ma_res.resid[-4:-1]
direct
```

The "direct" forecasts are identical. This happens since the effect of the short sample has disappeared by the end of the sample (In practice it is negligible by observations 100 or so, and numerically absent by around observation 160).

```python
ma_res.predict(t - 3, t - 1) - direct
```

The same principle applies in more complicated model that include multiple lags or seasonal term - predictions in AR models are simple once the effective lag length has been reached, while predictions in models that contains MA components are only simple once the maximum root of the MA lag polynomial is sufficiently small so that the residuals are close to the true residuals.

### Prediction differences in `SARIMAX` and `ARIMA`

The formulas used to make predictions from `SARIMAX` and `ARIMA` models differ in one key aspect - `ARIMA` treats all trend terms, e.g, the intercept or time trend, as part of the exogenous regressors.  For example, an AR(1) model with an intercept and linear time trend estimated using `ARIMA` has the specification

$$
\begin{align*}
Y_t - \delta_0 - \delta_1 t & = \epsilon_t \\
\epsilon_t & = \rho \epsilon_{t-1} + \eta_t
\end{align*}
$$

When the same model is estimated using `SARIMAX`, the specification is

$$
\begin{align*}
Y_t & = \epsilon_t \\
\epsilon_t & =  \delta_0 + \delta_1 t  + \rho \epsilon_{t-1} + \eta_t
\end{align*}
$$

The differences are more apparent when the model contains exogenous regressors, $X_t$.  The `ARIMA` specification is

$$
\begin{align*}
Y_t - \delta_0 - \delta_1 t - X_t \beta & = \epsilon_t \\
\epsilon_t & = \rho \epsilon_{t-1} + \eta_t \\
           & = \rho \left(Y_{t-1} - \delta_0 - \delta_1 (t-1) - X_{t-1} \beta\right) + \eta_t
\end{align*}
$$

while the `SARIMAX` specification is

$$
\begin{align*}
Y_t & =  X_t \beta + \epsilon_t \\
\epsilon_t & =  \delta_0 + \delta_1 t  + \rho \epsilon_{t-1} + \eta_t \\
           & = \delta_0 + \delta_1 t  + \rho \left(Y_{t-1} - X_{t-1}\beta\right) + \eta_t
\end{align*}
$$

The key difference between these two is that the intercept and the trend are effectively equivalent to exogenous regressions in `ARIMA` while they are more like standard ARMA terms in `SARIMAX`.

The next cell simulates an ARX with a time trend using the specification in `ARIMA` and estimates the parameters using both estimators.

```python
rho = 0.8
beta = 2
delta0 = 10
delta1 = 0.5
epsilon = eta.copy()
for i in range(1, eta.shape[0]):
    epsilon[i] = rho * epsilon[i - 1] + eta[i]
t = np.arange(epsilon.shape[0])
y = delta0 + delta1 * t + beta * full_x + epsilon
y = y[200:]
```

```python
start = np.array([110, delta1, beta, rho, 1])
arx_res = ARIMA(y, exog=x, order=(1, 0, 0), trend="ct").fit()
mod = SARIMAX(y, exog=x, order=(1, 0, 0), trend="ct")
start[:2] *= 1 - rho
sarimax_res = mod.fit(start_params=start, method="bfgs")
```

The two estimators fit similarly, although there is a small difference in the log-likelihood.  This is a numerical issue and should not materially affect the predictions. Importantly the two trend parameters, `const` and `x1` (unfortunately named for the time trend), differ between the two.  The other parameters are effectively identical.

```python
print(arx_res.summary())
```

```python
print(sarimax_res.summary())
```

## Initial residuals `SARIMAX` and `ARIMA`

Residuals for observations before the maximal model order, which depends on the AR, MA, Seasonal AR, Seasonal MA and differencing parameters, are not reliable and should not be used for performance assessment. In general, in an ARIMA with orders $(p,d,q)\times(P,D,Q,s)$, the formula for residuals that are less well behaved is:

$$
\max((P+D)s+p+d,Qs+q)
$$

We can simulate some data from an ARIMA(1,0,0)(1,0,0,12) and examine the residuals.

```python
import numpy as np
import pandas as pd

rho = 0.8
psi = -0.6
beta = 20
epsilon = eta.copy()
for i in range(13, eta.shape[0]):
    epsilon[i] = (
        rho * epsilon[i - 1]
        + psi * epsilon[i - 12]
        - (rho * psi) * epsilon[i - 13]
        + eta[i]
    )
y = beta + epsilon
y = y[200:]
```

With a large sample, the parameter estimates are very close to the DGP parameters.

```python
res = ARIMA(y, order=(1, 0, 0), trend="c", seasonal_order=(1, 0, 0, 12)).fit()
print(res.summary())
```

We can first examine the initial 13 residuals by plotting against the actual shocks in the model.  While there is a correspondence, it is fairly weak and the correlation is much less than 1.

```python
import matplotlib.pyplot as plt

plt.rc("figure", figsize=(10, 10))
plt.rc("font", size=14)

_ = plt.scatter(res.resid[:13], eta[200 : 200 + 13])
```

Looking at the next 24 residuals and shocks, we see there is nearly perfect correlation. This is expected in large samples once the less accurate residuals are ignored.

```python
_ = plt.scatter(res.resid[13:37], eta[200 + 13 : 200 + 37])
```

Next, we simulate an ARIMA(1,1,0), and include a time trend.

```python
rng = np.random.default_rng(20210819)
eta = rng.standard_normal(5200)
rho = 0.8
beta = 20
epsilon = eta.copy()
for i in range(2, eta.shape[0]):
    epsilon[i] = (1 + rho) * epsilon[i - 1] - rho * epsilon[i - 2] + eta[i]
t = np.arange(epsilon.shape[0])
y = beta + 2 * t + epsilon
y = y[200:]
```

Again the parameter estimates are very close to the DGP parameters.

```python
res = ARIMA(y, order=(1, 1, 0), trend="t").fit()
print(res.summary())
```

The residuals are not accurate, and the first residual is approximately 500.  The others are closer, although in this model the first 2 should usually be ignored.

```python
res.resid[:5]
```

The reason why the first residual is so large is that the optimal prediction of this value is the mean of the difference, which is 1.77.  Once the first value is known, the second value makes use of the first value in its prediction and the prediction is substantially closer to the truth.

```python
res.predict(0, 5)
```

It is worth noting that the results class contains two parameters than can be helpful in understanding which residuals are problematic, `loglikelihood_burn` and `nobs_diffuse`.

```python
res.loglikelihood_burn, res.nobs_diffuse
```


```python
# Welcome to MoCChA - Mathematics on Computers for Chemistry Applications!
#
# This notebook calculates the change Delta_F, given a differential df = fx * dx + fy * dy.
# We approximate Delta_F by taking finite steps in x and y and summing up the changes df.
#
# This cell defines the differential df and the curve along which we're calculating the change.
# Feel free to experiment. You can change any line marked with "<---" below to change one of the parameters.
# For example, you can choose a different differential to calculate or a different path along which to sum up df.
# Or, you can change the numerical accuracy by varying the step size that determines the
# finite steps with which we approximate dx and dy.
#
#
# Hit Shift-Enter to execute each cell.
#

# You can ignore the following four lines. They are necessary to load some libraries and set up the calculation.
import sympy
from sympy import *
# tell the computer that x and y are variables
x, y = sympy.symbols('x y')

###### MODIFY BELOW ########

### Define the differential df ###
# df = fx * dx  +  fy * dy
fx = 0.5*y**2       # <--- modify here to change fx
fy = x*y       # <--- modify here to change fy

### Defines the pathway along which we sum up the change ###

# We specify the pathway by specifying a function y(x).
# We're going to sum up Delta_F along the curve defined by y(x).
y_of_x = x # <--- modify here to change path

# x_start is the x-value of the start point for the path
x_start = 0 # <--- modify here to change starting point
# x_end is the x-value where the path ends
x_end = 1 # <--- modify here to change end point

# Our program is going to calculate the change in f by summing up small changes df along the path defined by y(x).
# stepsize is the size of steps we're going to take
stepsize = 0.1 # <--- modify here to change finite step size
```

```python
# You can ignore the code in this section. Just look at the output below.
# The plot below shows the path along which we're going to sum up the change Delta_F.
#
# Note: Plotting may take 5-10 seconds, so have patience ;)

import matplotlib
import matplotlib.pyplot as plt
import numpy as np
from sympy.utilities.lambdify import lambdify

X = np.arange(x_start, x_end+stepsize, stepsize) #
func = lambdify(x, y_of_x,'numpy')
Y = func(X)
plt.plot(X,Y,'bo-')
plt.ylabel("y")
plt.xlabel("x")
print("Here's the path along which we're going to sum up df:")
plt.annotate('start', xy=(X[0], Y[0]), xytext=(X[0], Y[0]+0.2),
            arrowprops=dict(facecolor='black', shrink=0.1),
            )
plt.annotate('end', xy=(X[-1], Y[-1]), xytext=(X[-1], Y[-1]-0.2),
            arrowprops=dict(facecolor='black', shrink=0.1)
            )
plt.show()
```

```python
# Now we're going to calculate the change df along the path.

# makes steps along the pathway y(x), calculate fx and fy for each point,
Delta_f = 0
Npoints = len(X)
print("Pt.No.    x0    y0  fx(x0,y0)  fy(x0,y0)  Delta_x  Delta_y  Delta_f")
print("-------------------------------------------------------------------")
for i in range(1, Npoints):
    x0 = X[i]
    y0 = Y[i]
    fx0 = fx.subs({x: x0, y: y0}).evalf()
    fy0 = fy.evalf(subs={x:x0,y:y0})
    DeltaX = x0 - X[i-1]
    DeltaY = y0 - Y[i-1]
    Delta_f += fx0 * DeltaX + fy0 * DeltaY
    print("%6d %5.1f %5.1f %10.1f %10.1f %8.1f %8.1f %8.1f" % (i,x0,y0,fx0,fy0,DeltaX,DeltaY,Delta_f))

print("Change Delta_f approximated by summing over %d finite steps:   %16.3f" % (Npoints, Delta_f))
```

    Pt.No.    x0    y0  fx(x0,y0)  fy(x0,y0)  Delta_x  Delta_y  Delta_f
    -------------------------------------------------------------------
         1   0.1   0.1        0.0        0.0      0.1      0.1      0.0
         2   0.2   0.2        0.0        0.0      0.1      0.1      0.0
         3   0.3   0.3        0.0        0.1      0.1      0.1      0.0
         4   0.4   0.4        0.1        0.2      0.1      0.1      0.0
         5   0.5   0.5        0.1        0.2      0.1      0.1      0.1
         6   0.6   0.6        0.2        0.4      0.1      0.1      0.1
         7   0.7   0.7        0.2        0.5      0.1      0.1      0.2
         8   0.8   0.8        0.3        0.6      0.1      0.1      0.3
         9   0.9   0.9        0.4        0.8      0.1      0.1      0.4
        10   1.0   1.0        0.5        1.0      0.1      0.1      0.6
    Change Delta_f approximated by summing over 11 finite steps:              0.578

```python

```


```python
import sympy
from sympy.parsing.sympy_parser import parse_expr
from ipywidgets import interact, FloatRangeSlider
import plotnine as p9
import pandas as pd
from math import pi
```

```python
x = sympy.Symbol('x')
```

```python
def factorial(n):
    """
        Calculates factorial
    """
    result = n
    if n == 1 or n == 0:
        return 1
    while n > 1:
        result *= (n-1)
        n -= 1
    return result
```

```python
def n_derivative(expression, n: int):
    """
        Calculates n'th derivative
    """
    derivative = expression
    while n > 0:
        derivative = sympy.diff(derivative)
        n -= 1
    return derivative
```

```python
def taylor_series(expression, n: int, around: float):
    """
        Calculates Taylor series
        n - order / number of terms
        around - approximation around certain point; real or complex number
    """
    derr_sum = 0
    while n >= 0:
        derr_sum += (
            n_derivative(expression, n).evalf(subs={'x': around})
            * ((x - around) ** n)
            / factorial(n)
        )
        n -= 1
    return derr_sum

taylor_series(sympy.cos(x), 4, 0)
```

$\displaystyle 0.0416666666666667 x^{4} - 0.5 x^{2} + 1.0$

%pip install plotnine

```python
type(type(sympy.sin(x)))
```

    sympy.core.function.FunctionClass

```python
def create_dataset(expression, n, around):
    """

    """
    a = pd.Series(range(0, 1000+1, 1))/1000 * 4 * pi - 2 * pi
    expr = parse_expr(expression)
    def evaluate(x):
        return float(expr.evalf(subs={'x': x}))
    sin_data = pd.DataFrame(data={
        # create sample data
        'x': a,
        'f(x)': a.apply(evaluate),
        "f_approx": a.apply(
            sympy.lambdify(
                x,
                taylor_series(
                    expression=expr,
                    n=n,
                    around=around
                )
            )
        )
    })
    return sin_data
```

```python
@interact(
    expression='sin(x)',
    n=(0, 10, 1),
    around=(-4, 4, 0.1),
    y_display_range=FloatRangeSlider(
        value=[-1.5, 1.5],
        min=-10.0,
        max=10.0,
        step=0.5,
        description='Y axis range'
    )
)
def create_plot(expression, n, around, y_display_range):
    data = create_dataset(expression, n, around)
    y_values = data[['f_approx', 'f(x)']]
    data = data[
        (y_values.min(axis=1) > y_display_range[0])
        &
        (y_values.max(axis=1) < y_display_range[1])
    ]
    plot = (
        p9.ggplot(data=data)
        + p9.aes(x="x")
        + p9.geom_point(color="red", mapping=p9.aes(y="f(x)"))
        + p9.geom_point(color="blue", alpha=0.15, mapping=p9.aes(y="f_approx"))
        + p9.ylim(y_display_range)
    )
    plot.draw()
```

    interactive(children=(Text(value='sin(x)', description='expression'), IntSlider(value=5, description='n', max=…


# Numerical Solution of the Wave Equation using the Finite Element Method

This notebook illustrates the numerical time-domain solution of the wave equation using the [Finite Element Method](https://en.wikipedia.org/wiki/Finite_element_method) (FEM). The method aims at an approximate solution by subdividing the area of interest into smaller parts with simpler geometry, linking these parts together and applying methods from the calculus of variations to solve the problem numerically. The FEM is a well established method for the numerical approximation of the solution of partial differential equations (PDEs). The solutions of PDEs are often known analytically only for rather simple geometries. FEM based simulations allow to gain insights into other more complex cases.

## Problem Statement

The linear and lossless propagation of sound is governed by the inhomogeneous linear [wave equation](https://en.wikipedia.org/wiki/Wave_equation)

\begin{equation}
\Delta p(\mathbf{x}, t) - \frac{1}{c^2} \frac{\partial^2}{\partial t^2} p(\mathbf{x}, t) = - q(\mathbf{x}, t) ,
\end{equation}

where $p(\mathbf{x}, t)$ denotes the sound pressure at position $\mathbf{x}$, $c$ the speed of sound and $q(\mathbf{x}, t)$ the inhomogeneity.
We aim in the following for a numerical solution of the wave equation on the domain $V$ with respect to the homogeneous Dirichlet boundary condition

\begin{equation}
p(\mathbf{x}, t) = 0 \qquad \text{for } x \in \partial V ,
\end{equation}

or the homogeneous Neumann boundary condition

\begin{equation}
\frac{\partial}{\partial n} p(\mathbf{x}, t) = 0 \qquad \text{for } x \in \partial V ,
\end{equation}

where $\partial V $ denotes the boundary of $V$.

## Variational Formulation

The FEM is based on expressing the partial differential equation (PDE) to be solved in its [variational](https://en.wikipedia.org/wiki/Calculus_of_variations) or weak form.
The first step towards this is to approximate the second-order temporal derivative in the wave equation by its backward [finite difference](https://en.wikipedia.org/wiki/Finite_difference)

\begin{equation}
\frac{\partial^2}{\partial t^2} p(\mathbf{x}, t) \approx \frac{p(\mathbf{x}, t) - 2 p(\mathbf{x}, t - T) +
p(\mathbf{x}, t- 2 T)}{T^2} ,
\end{equation}

where $T$ denotes the temporal stepsize (i.e. the sampling interval).
Introducing this approximation into the wave equation and rearranging terms yields

\begin{equation}
c^2 T^2 \Delta p(\mathbf{x}, t) - p(\mathbf{x}, t) = - 2 p(\mathbf{x}, t - T) +
p(\mathbf{x}, t- 2 T) - c^2 T^2 q(\mathbf{x}, t) .
\end{equation}

In order to derive the variational formulation we follow the [procedure outlined for the Helmholz equation](FEM_Helmholtz_equation_2D.ipynb#Variational-Formulation).
Multiplication by the test function $v(\mathbf{x}, t)$, integration over the domain $V$ and application of Green's first identity yields

\begin{equation}
{-} \int_V \left( c^2 T^2  \nabla p(\mathbf{x}, t) \cdot \nabla v(\mathbf{x}, t) + p(\mathbf{x}, t) v(\mathbf{x}, t) \right) \mathrm{d}x = \int_V \left( - 2 p(\mathbf{x}, t - T) + p(\mathbf{x}, t- 2 T) - c^2 T^2 q(\mathbf{x}, t) \right) v(\mathbf{x}, t) \mathrm{d}x ,
\end{equation}

where  $v(\mathbf{x}, t) = 0$ on $\partial V$ where $p(\mathbf{x}, t)$ is known - for instance due to fixed boundary conditions - was exploited in case of a pure Dirichlet boundary condition or $\frac{\partial}{\partial n} p(\mathbf{x}, t) = 0$ on $\partial V$ in case of a pure Neumann boundary condition.
It is common to express the integral equation above in terms of the bilinear $a(P, V)$ and linear $L(V)$ forms

\begin{equation}
a(P, V) = \int_V \left( c^2 T^2  \nabla p(\mathbf{x}, t) \cdot \nabla v(\mathbf{x}, t) +
p(\mathbf{x}, t) v(\mathbf{x}, t) \right) \mathrm{d}x ,
\end{equation}

\begin{equation}
L(V) = \int_V \left( 2 p(\mathbf{x}, t - T) - p(\mathbf{x}, t- 2 T) + c^2 T^2 q(\mathbf{x}, t) \right) v(\mathbf{x}, t) \mathrm{d}x ,
\end{equation}

where

\begin{equation}
a(P, V) = L(V) .
\end{equation}

## Numerical Solution

The numerical solution of the variational problem is based on [FEniCS](https://fenicsproject.org/), an open-source framework for numerical solution of PDEs.
Its high-level Python interface `dolfin` is used in the following to define the problem and compute its solution.
The implementation is directly based on the variational formulation derived above.
It is common in the FEM to denote the solution of the problem by $u$ and the test function by $v$.
The definition of the problem in FEniCS is very close to the mathematical formulation of the problem.

```python
import dolfin
import mshr
import matplotlib.pyplot as plt
%matplotlib inline

T = 1/40000  # temporal sampling interval

def FEM_wave_equation(mesh, T, N, xs, neumann_bc=True, c=343):

    # define function space
    V = dolfin.FunctionSpace(mesh, "CG", 1)

    # define previous and second-last solution
    u1 = dolfin.interpolate(dolfin.Constant(0.0), V)
    u0 = dolfin.interpolate(dolfin.Constant(0.0), V)

    # define boundary conditions
    if neumann_bc:
        bcs = None
    else:
        bcs = dolfin.DirichletBC(V, dolfin.Constant(0.), "on_boundary")

    # define variational problem
    u = dolfin.TrialFunction(V)
    v = dolfin.TestFunction(V)

    a = dolfin.inner(u, v) * dolfin.dx + dolfin.Constant(T**2 * c**2) * dolfin.inner(dolfin.nabla_grad(u), dolfin.nabla_grad(v)) * dolfin.dx
    L = 2*u1*v * dolfin.dx - u0*v * dolfin.dx

    # compute solution for all time-steps
    u = dolfin.Function(V)

    for n in range(N):
        A, b = dolfin.assemble_system(a, L, bcs)
        # define inhomogenity
        if n==0:
            delta = dolfin.PointSource(V, xs, 1)
            delta.apply(b)
        # solve variational problem
        dolfin.solve(A, u.vector(), b)
        u0.assign(u1)
        u1.assign(u)

    return u

def plot_soundfield(u):
    '''plots solution of FEM-based simulation'''
    fig = plt.figure(figsize=(10,10))
    fig = dolfin.plot(u)
    plt.title(r'$p(\mathbf{x}, t)$')
    plt.xlabel(r'$x$ / m')
    plt.ylabel(r'$y$ / m')
    plt.colorbar(fig, fraction=0.038, pad=0.04);
```

### Sound Field in a Rectangular Room with Sound-Hard Boundaries

For a first validation of the numerical simulation by the FEM, the solution of the inhomogeneous two-dimensional wave equation for a point source $q(\mathbf{x}, t) = \delta(\mathbf{x}-\mathbf{x}_s) \cdot \delta(t)$ at position $\mathbf{x}_s = (2,2)$ m is considered for a Dirac shaped excitation in the time-domain. The simulated geometry is a two-dimensional rectangular room with size $4 \times 4$ meters and sound-hard boundaries (Neumann boundary condition). Note that the free-field solution of the two-dimensional wave equation for a spatio-temporally Dirac shaped excitation is given as

\begin{equation}
p(\mathbf{x}, t) = \frac{1}{2 \pi \sqrt{t^2 - (\frac{||\mathbf{x} - \mathbf{x}_s||}{c})^2}} \qquad \text{for } t > \frac{||\mathbf{x} - \mathbf{x}_s||}{c} .
\end{equation}

In order to validate the simulation for free-field propagation the timestep $N=150$ and sampling interval $T=\frac{1}{40000}$ s are chosen such that the incident wave is not reflected by one of the walls. The simulated time $t = N \cdot T = 3.75$ ms after excitation by the Dirac impulse.

```python
# define geometry and mesh
domain = mshr.Rectangle(dolfin.Point(0, 0), dolfin.Point(4,4))
mesh = mshr.generate_mesh(domain, 200)

# compute solution
u = FEM_wave_equation(mesh, T, 150, dolfin.Point(2,2), neumann_bc=True)

# plot sound field
plot_soundfield(u)
plt.grid()
```

The result of the numerical simulation is compared to the theoretical result given above for a line parallel to the y-axis for $y = 2$.

```python
import numpy as np

# extract simulation results on line
x = np.linspace(0 + 1E-14, 4 - 1E-14, 101)
points = [(x_, 2) for x_ in x]
ux = [u(point) for point in points]

# compute analytic result on line
px = np.zeros(len(x))
a = (150*T)**2 - (np.sqrt((x-2)**2 + (2-2)**2)/343)**2
px[a>0] = 0.34 * 1/(2*np.pi * np.sqrt(a[a>0]))

# plot comparison
plt.figure(figsize=(10,5))
plt.plot(x, px, 'k--' , label='analytic solution')
plt.plot(x, ux, label='numeric solution')
plt.xlabel(r'$x$ / m')
plt.ylabel(r'$p(\mathbf{x}, t)$')
plt.legend()
plt.ylim([0, 50]);
plt.grid()
```

### Sound Field in Two Coupled Rectangular Rooms#

In order to illustrate the procedure for a more complex geometry, the sound field in two coupled rectangular rooms with sound-hard boundaries is computed. First, the geometry of the problem is defined and the mesh is plotted with a low number of elements for ease of illustration.

```python
# define geometry and compute low resolution mesh for illustration
domain = mshr.Rectangle(dolfin.Point(0, 0), dolfin.Point(3,4)) + \
         mshr.Rectangle(dolfin.Point(3, 1.5), dolfin.Point(3.5, 2.5)) + \
         mshr.Rectangle(dolfin.Point(3.5, 0), dolfin.Point(6, 4))
mesh = mshr.generate_mesh(domain, 20)

dolfin.plot(mesh);
```

Now the problem is defined and solved with FEniCS with a high density mesh. The source position is chosen as $\mathbf{x}_s = (2,1)$ m and the total number of time-steps as $N=300$.

```python
# high resolution mesh for FEM simulation
mesh = mshr.generate_mesh(domain, 150)

# compute solution
u = FEM_wave_equation(mesh, T, 300, dolfin.Point(2,1), neumann_bc=True)

# plot sound field
plot_soundfield(u)
```

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT).


# Inaugural Project 2021: Housing Demand and Taxation

**Group:** SSTL

**Members:** Louise Otte Arildsen (srb330), Signe Kolind (hmk792), Stine Fürst (qvj635) & Thomas Nielsen (qzw392)

**Imports and set magics:**

```python
import numpy as np
from scipy import optimize

%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use("seaborn")

# autoreload modules when code is run
%load_ext autoreload
%autoreload 2
```

# Question 1

**We consider a model for a household looking to buy a home. The following applies for the household:**
* The household is endowed with an amount of cash, $m$, which can be spend on housing or consumption.
* The household derives utility based on the quality, $h$, of the house, which price is determined by $p_{h}$.
* The house is subject to progressive taxation and mortgage costs summarized by $\tau(\cdot)$.
* The objective of the household is to find the optimal division of money spend on housing and consumption in order to maximize utility.

**Model equations:**
$$
\begin{align}
c^{\star}, h^{\star} &=\arg \max _{c, h} c^{1-\phi} h^{\phi} \\
& \text { s.t. } \\
\widetilde{p}_{h} &=p_{h} \varepsilon \\
m &=\tau\left(p_{h}, \widetilde{p}_{h}\right)+c \\
\tau\left(p_{h}, \widetilde{p}_{h}\right) &=r p_{h}+\tau^{g} \widetilde{p}_{h}+\tau^{p} \max \left\{\widetilde{p}_{h}-\bar{p}, 0\right\}
\end{align}
$$

**Explanation:** The maximization problem of the household is to find the optimal allocation of money spend on consumption, $c$, and housing, given that the price of a house is equal to the house quality, $h$.
Houses are taxed based on a public assesment, $\tilde{p}_{h}$, which depends on the market value of the house $p_{h}$ and a factor $\varepsilon$, which is politically set below 1.
The household's cash endowment, $m$, is either spend on housing expenditures, $\tau\left(p_{h}, \widetilde{p}_{h}\right)$, or consumption, $c$. The housing expenditures are determined by a interest-only mortgage with interest rate, $r$, a base housing tax, $\tau^{g}$, and a progressive housing tax, $\tau^{p}$, levied on houses with a market value above the cutoff, $p$.

By combining equation (2)-(4) and applying that $p_{h}=h$, we are able to write $c$ as a function of $h$ and thus contruct a univariate mazimization problem for the household:

$ c=m-rh-\tau^gh\varepsilon-\tau^pmax\left\{ h\varepsilon-\bar{p},0\right\} $

$c^{\star}, h^{\star} =\arg \max _{h} (m-rh-\tau^gh\varepsilon-\tau^pmax\left\{ h\varepsilon-\bar{p},0\right\})^{1-\phi} h^{\phi} $

The optimizing function used to solve the maximization problem are defined in the following code.

```python
#a. setting parameter values:
m=0.5
phi=0.3
epsilon=0.5
r=0.03
tau_g=0.012
tau_p=0.004
p_bar=3

# utility function
def u_func(h, m, phi, epsilon, r, tau_g, tau_p, p_bar):

    """
    Utility function for a household making a housing and consumption decision subject to several          constraints.
    The optimisation problem only considers the variable housing as consumption can bew written as an      implicit function hereof.

    Input:
    h: money spent on housing
    m: cash-on-hand for the household
    phi: elasticity
    epsilon: public assessment factor, politically determined
    r: mortgage interest rate
    tau_g: base tax rate
    tau_p: progressive tax rate
    p_bar: cutoff value for homes regarding the progressive tax rate

    Output: Utility given housing choice.
    """

    c=m-r*h - tau_g*h*epsilon - tau_p*max((h*epsilon-p_bar),0)
    u = (c**(1-phi))*(h**phi)
    return u

# scalar optimiser function
def u_optimiser(m, phi, epsilon, r, tau_g, tau_p, p_bar):

    def objective(h, m, phi, epsilon, r, tau_g, tau_p, p_bar):
        return -u_func(h=h[0], m=m, phi=phi, epsilon=epsilon, r=r, tau_g=tau_g, tau_p=tau_p, p_bar=p_bar)

    guess = [0.5]
    Bc = lambda h: m - r*h[0] - tau_g*h[0]*epsilon - tau_p*max((h[0]*epsilon-p_bar),0)
    sol = optimize.minimize(objective,guess,bounds=[(0,np.inf)],args=(m, phi, epsilon, r, tau_g, tau_p, p_bar),constraints={'type':'ineq','fun':Bc})

    h_star = sol.x[0]
    c_star = m-r*h_star - tau_g*h_star*epsilon - tau_p*max(h_star*epsilon-p_bar,0)
    u_star = u_func(h=h_star,m=m, phi=phi, epsilon=epsilon, r=r, tau_g=tau_g, tau_p=tau_p, p_bar=p_bar)
    return h_star, c_star, u_star

#b. solve maximization problem
h_star, c_star, u_star = u_optimiser(m, phi, epsilon, r, tau_g, tau_p, p_bar)

# c. print solution for optimal choice of money spend on housing and consumption
print(f'The household will choose optimal housing = {h_star:.2f}, which implies optimal consumption = {c_star:.2f} and utility = {u_star:.2f}')
```

    The household will choose optimal housing = 4.16, which implies optimal consumption = 0.35 and utility = 0.74

# Question 2

We want to plot $c^{\star}$ and $h^{\star}$ as functions of $m$ in the range 0.4 to 1.5.

```python
# a. create array of m and container for values of h and c
N=100
m = np.linspace(0.5,1.5,num=N)
h_vals = np.empty(N)
c_vals = np.empty(N)

# b. loop the optimiser over the m array
for i,mi in enumerate(m):
    """
    Looping m[i] indexes, over optimizer in order to find optimal h and c values.
    """
    hc = u_optimiser(mi, phi, epsilon, r, tau_g, tau_p, p_bar)

    h_vals[i] = hc[0]
    c_vals[i] = hc[1]

def two_figures(x_left, y_left, title_left, xlabel_left, ylabel_left, x_right, y_right, title_right, xlabel_right, ylabel_right):
    """
    Plots two aligned figures.

    Inputs: axises, labels, titles of left/right figure.
    Output: Two figures in 2D
    """
    # a. initialise figure
    fig = plt.figure(figsize=(10,4))# figsize is in inches...

    # b. left plot
    ax_left = fig.add_subplot(1,2,1)
    ax_left.plot(x_left,y_left)

    ax_left.set_title(title_left)
    ax_left.set_xlabel(xlabel_left)
    ax_left.set_ylabel(ylabel_left)

    # c. right plot
    ax_right = fig.add_subplot(1,2,2)

    ax_right.plot(x_right, y_right)

    ax_right.set_title(title_right)
    ax_right.set_xlabel(xlabel_right)
    ax_right.set_ylabel(ylabel_right)

# c. plot the figure
two_figures(m, h_vals, "Housing", "$m$", "$h$", m, c_vals, "Consumption", "$m$", "$c$")
```

We see from the two graphs the optimal amount of consumption and housing for all values of cash in hand variable m between the interval of 0.5 to 1.4. The graphs is mostly linear except the interval 0.71 to 0.75, where housing costs doesn't change and consumption moves differently.

# Question 3

We now consider $N=10000$ household looking to buy houses, but each household differ wrt. their cash endowment. The distribution of cash-on-hand is given by

$$
m_{i} \sim \operatorname{Lognormal}(-0.4,0.35)
$$

We let the public assessment associated with the optimal housing choice of household $i$, $h_{i}^{\star}$
,be denoted $
\widetilde{p}_{h, i}^{\star}
$

$T=\sum_{i=1}^{N} \tau^{g} \widetilde{p}_{h, i}^{\star}+\tau^{p} \max \left\{\widetilde{p}_{h, i}^{\star}-\bar{p}, 0\right\}$

In the following module we calculate the average tax burden per household.

```python
#a. setting parameters
size=10000
seed=1
mu=-0.4
sigma=0.35

#b.
def tax_revenue(seed,size,mu,sigma,tau_g=0.012,epsilon=0.5,tau_p=0.004,p_bar=3):
    """
    Calculates the total tax revenue of N size agents, with utility u_func
    and log-normally distributed cash-on-hand mi.

    Inputs:
    seed: seed number
    size: random values of cash-on-hand mi (i.e. number of agents) to draw from distribution
    mu: distribution mean
    sigma: standard deviation
    tau_g: base tax rate
    epsilon: public assessment factor, politically determined
    tau_p: progressive tax rate
    p_bar: cutoff value for homes regarding the progressive tax rate

    Output: Total tax revenue from N size agents, derived from utility of housing choice
    depending on log-normally distributed values of cash-on-hand.
    """
    # a. set seed, draw random numbers
    np.random.seed(seed)
    mi = np.random.lognormal(mu,sigma,size)

    # b. define local parameter values
    phi=0.3
    r=0.03

    # c. solve each individual's optimisation problem
    tax_rev = 0

    for i, mi in enumerate (mi):
        """
        Looping mi, N times over optimizer in order to find tax revenue_i.
        Calculating total tax revenue as a sum of tax_revenue_i where i= 1,..,N
        """
        hc = u_optimiser(mi, phi, epsilon, r, tau_g, tau_p, p_bar)
        tax_i = tau_g*hc[0]*epsilon + tau_p*max(hc[0]*epsilon-p_bar,0)
        tax_rev += tax_i

    return tax_rev

# compute total tax revenue and average tax burden per household
total_tax_rev = tax_revenue(seed,size,mu,sigma)
average_tax=total_tax_rev/size

print(f'The total tax revenue is {total_tax_rev:.5f}')
print(f'The average tax burden per household is {average_tax:.5f}')

```

    The total tax revenue is 363.14180
    The average tax burden per household is 0.03631

# Question 4

We consider a reform of the tax system which changes the model parameters and we recalculate the average tax burden per household given these changes:

```python
# rerunning tax_revenue with new parameter values
total_tax_rev_2 = tax_revenue(seed,size,mu,sigma,tau_g=0.01,epsilon=0.8,tau_p=0.009,p_bar=8)
average_tax_2=total_tax_rev_2/size
print(f'The total tax revenue given the reform is {total_tax_rev_2:.2f}')
print(f'The average tax burden per household given the reform is {average_tax_2:.5f}')
```

    The total tax revenue given the reform is 450.19
    The average tax burden per household given the reform is 0.04502

The reform leads to an increase in the total tax revenue and an increase in the average tax burden per household, corresponding to a 24 pct. Increase.

# Question 5

Now the policy maker decide that instead of implementing the reform given in question 4, they would like a solution where the average home owner does not pay more in housing taxes. The policy maker therefore imposes that reform changes to $\varepsilon$, $\tau^{p}$ and $\bar{p}$, will be offset by lowering $\tau^{g}$, such that the average tax payments are unchaged from before the reform. In the next module we calculate the $\tau^{g}$ that meets this requirement.

**OBS:** Running the code with 10.000 households will take hours. We have reduced the number of households to 1.000 by assigning changing size=1000 in question 3 and thus the running time is reduced to around 5 minutes. If you want to lower the running time to less than one minute set size=100.

```python
# draws of random numbers
np.random.seed(seed)

#define the tax goal:
tax_goal=tax_revenue(seed,size,mu,sigma,tau_g=0.012,epsilon=0.5,tau_p=0.004,p_bar=3)

# excess tax revenue function
def excess_tax_rev(tau_g, epsilon, tau_p, p_bar):
    """
    Calculates difference between tax revenues before and after reform in housing taxes
    """
    excess_tax_revenue = tax_revenue(seed,size,mu,sigma,tau_g=tau_g,epsilon=0.8,tau_p=0.009,
                                     p_bar=8) - tax_goal
    #tau_g=tau_g so we loop over tau_g
    return excess_tax_revenue

# Equilibrium
def find_equilibrium(tau_g, epsilon, tau_p, p_bar, kappa=0.5, eps=1e-8, maxiter=500):
    """
    Iterating tau_g 500 times over excess_tax_return,
    until excess tax return is as close to zero as possible.
    Such that the average tax payments are unchanged from before the reform.

    Input:
    tau_g: base tax rate
    tau_p: progressive tax rate
    epsilon: public assessment factor, politically determined
    p_bar: cutoff value for homes regarding the progressive tax rate
    kappa: scalar
    eps: desired value of excess tax
    maxiter: maximum number of iterations

    Output:
    Prints first 5 iterations of tau_g with respective excess tax revenue.
    Returns last prints with a 25th iteration gap untill solution is found.

    """

    t = 0
    while True:

        # a. step 1: excess demand
        Z1 = excess_tax_rev(tau_g, epsilon, tau_p, p_bar)

        # b: step 2: stop iteration when excess tax is as close to zero as possible.
        if  np.abs(Z1) <= eps or t >= maxiter: #<= eps so we find a number 'close' to zero.
            print(f'{t:3d}: tau_g = {tau_g:12.8f} -> excess tax revenue -> {Z1:14.8f}')
            break

        # c. step 3: update tau_g with scaled error factor
        tau_g = tau_g - tau_g*kappa*Z1/total_tax_rev

        # d. step 4: return
        if t < 5 or t%25 == 0:
            print(f'{t:3d}: tau_g = {tau_g:12.8f} -> excess tax revenue -> {Z1:14.8f}')
        elif t == 5:
            print('   ...')

        t += 1

    return tau_g

# f. call find equilibrium function
kappa = 0.5
eps = 1e-8
tau_g = find_equilibrium(tau_g, epsilon, tau_p, p_bar,kappa=kappa,eps=eps)
```

      0: tau_g =   0.00941562 -> excess tax revenue ->    15.74696276
      1: tau_g =   0.00855005 -> excess tax revenue ->     6.72159202
      2: tau_g =   0.00814352 -> excess tax revenue ->     3.47656746
      3: tau_g =   0.00793056 -> excess tax revenue ->     1.91206370
      4: tau_g =   0.00781324 -> excess tax revenue ->     1.08163548
       ...
     25: tau_g =   0.00765603 -> excess tax revenue ->     0.00001393
     43: tau_g =   0.00765602 -> excess tax revenue ->    -0.00000000

**Task:** Comment on the optimal tau_g.

In conclusion we find that the household will choose optimal housing = 4.16, which implies optimal consumption = 0.35 and utility = 0.74. Before the tax reform, the total tax revenue is 363.14, and the average tax burden per household is 0.036. The tax reform leads to an increase in the total tax revenue and an increase in the average tax burden per household, corresponding to a 24 pct. increase. If the policymaker wants to ensure a constant average tax burden per household after the tax reform, they could offset $tau_g$ such that $tau_g =0.0077 $.


# Lake model

see also [this general introduction to the workbench](https://waterprogramming.wordpress.com/2017/11/01/using-the-exploratory-modelling-workbench/) as a source of inspiration for completing the assignment below

## the lake model
The exploratory modeling workbench includes an example folder. This folder contains a variety of examples that demonstrate the functionality of the workbench. Many of these examples have been drawn from published cases. Here, we use the Lake Problem as an example for demonstrating some of the key functionality of the workbench.

We demonstrate some of the key capabilities of the exploratory modeling workbench using the Lake problem. The lake problem is a stylized and hypothetical decision problem where the population of a city has to decide on the amount of annual pollution it will put into a lake. It the pollution in the lake passes a threshold, it will suffer irreversible eutrophication.

\begin{equation}
    X_{(t+1)}=X_t+a_t+\frac{(X_t^q)}{(1+X_t^q )}- bX_t+\epsilon_t
\end{equation}

where $X_t$ is the pollution at time $t$, $a_t$ is the rate of anthropogenic pollution at time $t$, $b$ is the lake’s natural removal rate, $q$ is the lake's natural recycling rate, $\epsilon_t$ is the rate of natural pollution at time $t$. The rate of anthropogenic pollution $a_t$ is the decision variable and is somewhere between 0, and 0.1. So $a_t \in [0,0.1]$. The natural pollution $\epsilon_t$ is modeled, following Singh et al. (2015), as a log normal distribution with mean $\mu$ and standard deviation $\sigma$.
There are four outcomes of interest. The first is the average concentration of phosphor in the lake.

\begin{equation}
    f_{phosphorus}=  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t\in{T}} X_t
\end{equation}

where $\left\vert{T}\right\vert$ is the cardinality of the set of points in time.
The second objective is the economic benefit derived from polluting the lake. Following Singh et al. (2015), this is defined as the discounted benefit of pollution mines the costs of having a polluted lake

\begin{equation}
    f_{economic} = \sum\limits_{t \in {T}}\alpha a_t \delta^t
\end{equation}

where $\alpha$ is the utility derived from polluting and $\delta$ is the discount rate. By default, $\alpha$ is 0.04.
The third objective is related to the year over year change in the anthropogenic pollution rate.

\begin{equation}
    f_{inertia} =\frac{1}{\left\vert{T}\right\vert-1} \sum\limits_{t=1}^{\left\vert{T}\right\vert} I(|a_{t}-a_{t-1} |>\tau)
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $\tau$ is the threshold that is deemed undesirable, and is for illustrative purposes et to 0.2. Effectively, f_{inertia} is the fraction of years where the absolute value of the change in anthropogenic pollution is larger then $\tau$.
The fourth objective is the fraction of years where the pollution in the lake is below the critical threshold.

\begin{equation}
    f_{reliability} =  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t \in T}I(X_{t}<X_{crit} )
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $X_{crit}$ is the critical threshold of pollution and is a function of both $b$ and $q$.

The lake problem is characterized by both stochastic uncertainty and deep uncertainty. The stochastic uncertainty arises from the natural inflow. To reduce this stochastic uncertainty, multiple replications are performed and the average over the replication is taken. Deep uncertainty is presented by uncertainty about the mean $\mu$ and standard deviation $sigma$ of the lognormal distribution characterizing the natural inflow, the natural removal rate of the lake $\beta$, the natural recycling rate of the lake $q$, and the discount rate $\delta$. The table below specifies the ranges for the deeply uncertain factors, as well as their best estimate or default values.

## Assignment
1. Given the Python implementation of the lake problem in lakemodel_function.py, adapt this code and connect it to the workbench

for the uncertainties, use the following table

|Parameter	|Range	        |Default value|
|-----------|--------------:|------------:|
|$\mu$    	|0.01 – 0.05	|0.02         |
|$\sigma$	|0.001 – 0.005 	|0.0017       |
|$b$      	|0.1 – 0.45	    |0.42         |
|$q$	    |2 – 4.5	    |2            |
|$\delta$	|0.93 – 0.99	|0.98         |

For now, assume that for each year a release decision is made. The release is between 0 and 0.1. Carefully look at line 23 in lake_model.py to identify the name to use for each lever.

2. Explore the behavior of the system in the absence of any release using 1000 scenarios, and the default sampling approach.
    * visualize the outcomes of interest, are there any apparent trade-offs?
    * can you visually identify the uncertainties that drive system behavior?

```python
import numpy as np
from lakemodel_function import lake_problem
from ema_workbench import (RealParameter, ScalarOutcome, Constant, Model)
import pandas as pd
import plotly_express as px
import seaborn as sns
from ema_workbench.analysis import pairs_plotting
import matplotlib.pyplot as plt
```

### 1. Defining a Model Without Levers

```python
model = Model('lakeproblem', function=lake_problem)

#specify uncertainties
model.uncertainties = [RealParameter('b', 0.1, 0.45),
                       RealParameter('q', 2.0, 4.5),
                       RealParameter('mean', 0.01, 0.05),
                       RealParameter('stdev', 0.001, 0.005),
                       RealParameter('delta', 0.93, 0.99)]

#specify outcomes
model.outcomes = [ScalarOutcome('max_P'),
                  ScalarOutcome('utility'),
                  ScalarOutcome('inertia'),
                  ScalarOutcome('reliability')]
```

* Run the model with a sequential evaluator.

```python
%%time
from ema_workbench import (SequentialEvaluator, ema_logging, perform_experiments)

ema_logging.log_to_stderr(ema_logging.INFO)

with SequentialEvaluator(model) as evaluator:
    experiments, outcomes = evaluator.perform_experiments(scenarios=1000, policies=0)
```

    [MainProcess/INFO] performing 1000 scenarios * 1 policies * 1 model(s) = 1000 experiments
    [MainProcess/INFO] performing experiments sequentially
    [MainProcess/INFO] 100 cases completed
    [MainProcess/INFO] 200 cases completed
    [MainProcess/INFO] 300 cases completed
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 500 cases completed
    [MainProcess/INFO] 600 cases completed
    [MainProcess/INFO] 700 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 900 cases completed
    [MainProcess/INFO] 1000 cases completed
    [MainProcess/INFO] experiments finished

    Wall time: 42.3 s

### 1.1 Visualizing results with pair plots

```python
fig, axes = pairs_plotting.pairs_scatter(experiments, outcomes, group_by="policy")

fig.set_size_inches(14,14)

plt.show()
```

### 1.2 Evaluating the Effects of Uncertainties
* We visualized the relation between parameters 'b' and 'q' in terms of the impact on outcome max_P.
* It appears that with low 'b' values the phosphorus levels are high.

```python
P = outcomes['max_P']

fig, ax = plt.subplots(figsize=(6,6))

sc = ax.scatter(experiments.b, experiments.q, c=P)
plt.colorbar(sc)
ax.set_xlabel('b')
ax.set_ylabel('q')
plt.show()
```

* No apparently discernible relation between 'delta' and 'mean' variables in terms of impact on outcome max_P.

```python
P = outcomes['max_P']

fig, ax = plt.subplots(figsize=(6,6))

sc = ax.scatter(experiments.delta, experiments["mean"], c=P)
plt.colorbar(sc)
ax.set_xlabel('delta')
ax.set_ylabel('mean')
plt.show()
```

### 2. Defining a Model with 100 Policy Levers (bounded between 0 and 0.1)

```python
model = Model('lakeproblem', function=lake_problem)

model.levers = [RealParameter('l' + str(i), 0, 0.1) for i in range(4)]

#specify uncertainties
model.uncertainties = [RealParameter('b', 0.1, 0.45),
                       RealParameter('q', 2.0, 4.5),
                       RealParameter('mean', 0.01, 0.05),
                       RealParameter('stdev', 0.001, 0.005),
                       RealParameter('delta', 0.93, 0.99)]

#specify outcomes
model.outcomes = [ScalarOutcome('max_P'),
                  ScalarOutcome('utility'),
                  ScalarOutcome('inertia'),
                  ScalarOutcome('reliability')]
```

* Run the model with a sequential evaluator.

```python
%%time
from ema_workbench import (SequentialEvaluator, ema_logging, perform_experiments)

ema_logging.log_to_stderr(ema_logging.INFO)

with SequentialEvaluator(model) as evaluator:
    experiments, outcomes = evaluator.perform_experiments(scenarios=1000, policies=4)
```

    [MainProcess/INFO] performing 1000 scenarios * 4 policies * 1 model(s) = 4000 experiments
    [MainProcess/INFO] performing experiments sequentially
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1600 cases completed
    [MainProcess/INFO] 2000 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 2800 cases completed
    [MainProcess/INFO] 3200 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4000 cases completed
    [MainProcess/INFO] experiments finished

    Wall time: 2min 45s

3. Explore the behavior of the system over 1000 scenarios for 4 randomly sampled candidate strategies.
    * visualize the outcomes of interest
    * what can you say about how the release decision influences the system?

### 2.1 Visualize Results

```python
outcomes_df = pd.DataFrame.from_dict(outcomes)
# melted = pd.melt(outcomes_df, value_vars = outcomes_df.columns)
```

```python
sns.jointplot(data=outcomes_df, y="max_P", x="utility", kind='kde')
```

* Visualize results with pair plots.

```python
fig, axes = pairs_plotting.pairs_scatter(experiments, outcomes, group_by="policy")

fig.set_size_inches(12,12)

plt.show()
```

### 3. Defining a Model with 100 Policy Levers (bounded between 0 and 0.5)

```python
model = Model('lakeproblem', function=lake_problem)

model.levers = [RealParameter('l' + str(i), 0, 0.5) for i in range(100)]

#specify uncertainties
model.uncertainties = [RealParameter('b', 0.1, 0.45),
                       RealParameter('q', 2.0, 4.5),
                       RealParameter('mean', 0.01, 0.05),
                       RealParameter('stdev', 0.001, 0.005),
                       RealParameter('delta', 0.93, 0.99)]
#specify outcomes
model.outcomes = [ScalarOutcome('max_P'),
                  ScalarOutcome('utility'),
                  ScalarOutcome('inertia'),
                  ScalarOutcome('reliability')]
```

* Running the model with a MultiprocessingEvaluator

```python
%%time
from ema_workbench import (MultiprocessingEvaluator, ema_logging,
                           perform_experiments)
ema_logging.log_to_stderr(ema_logging.INFO)

with MultiprocessingEvaluator(model) as evaluator:
    experiments, outcomes = evaluator.perform_experiments(scenarios=1000, policies=4)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] performing 1000 scenarios * 4 policies * 1 model(s) = 4000 experiments
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1600 cases completed
    [MainProcess/INFO] 2000 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 2800 cases completed
    [MainProcess/INFO] 3200 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4000 cases completed
    [MainProcess/INFO] experiments finished
    [MainProcess/INFO] terminating pool

    Wall time: 1min 14s

#### The multiprocessing evaluator was much faster than the sequential evaluator.

```python
fig, axes = pairs_plotting.pairs_scatter(experiments, outcomes, group_by="policy")

fig.set_size_inches(12,12)

plt.show()
```

*Interpretation of Results:*
* Inertia is obviously different per policy because the amount of release is different per policy; inertia is directly affected by release.
* Inertia does not affect Max_Phosphorus, reliability and utility when release is between 0 and 0.1. However, when release >0.1, it appears to affect utility and max phosphorus (see graph below for policy between 0 and 0.5).

4. If you have not used parallelization in the foregoing, try to adapt your code to use parallelization. The workbench comes with two evaluators for parallelization. The `MultiProcessingingEvaluator` and the `IpyparallelEvaluator`. When can you use each? Adapt your code from above and sue the `MultiProcessingingEvaluator`. Use the time library to check how much faster the computation for 1000 scenarios completes.

Using multiprocessing within a jupyter notebook is tricky. On linux it will work in general just fine. On mac it depends on the version of Mac OS and the version of Python. If you are on the latest version of Mac OS in combination with Python 3.8, it might work but no guarantees. On older versions of Python it should work fine. On Windows it is always a problem.

The underlying explanation is quite technical. It has to do with how your operating system creates the additional python processes. On windows, and the latest version of Mac OS in combination with Python 3.8. A completely new Python process is spawned. This new process does **not** inherit what is defined in memory of the parent process. The new child process will try to replicate what is in memory of the parent process by executing many of the import statements that have also been executed within the python process. Thus, if you define a model in the main process, it is not guaranteed to be known in the child processes. This is in particular true if you define the model within a jupyter notebook. Then the child processes will **never** know this function. Within jupyter notebooks, therefore, the best practice is to define your model within a .py file and import this .py file into the notebook. Now, each of the child processes will also execute this import statement and thus know the function.

epa1361_open/Week 1-2 - general intro to exploratory modelling/assignment 2 - lakemodel.ipynb


# Root Finder

```julia
using Plots

f(x) = x^4+0.4*x^3-0.5*x^2+1.2*x-1.5
range = -100:1:100
plot(range, f.(range), linewidth=4, label="p(x)")
```

## Vieta's formulas for $n=4$:

$
\begin{align}
x_1 + x_2 + x_3 + x_4 &= -0.4 \\
x_1 (x_2 + x_3 + x_4) + x_2 (x_3 + x_4) + x_3 x_4 &= -0.5 \\
x_1 x_2 ( x_3 + x_4) + x_2 x_3 x_4 + x_1 x_3 x_4   &= -1.2 \\
x_1 x_2 x_3 x_4 &= 1.5
\end{align}
$

## Jacobian Matrix

$
\begin{bmatrix}
  1 & 1 & 1 & 1 \\
  x_2 + x_3 + x_4 & x_1 + x_3 + x_4 & x_1 + x_2 + x_4 & x_1 + x_2 + x_3 \\
  x_2x_3 + x_2x_4 + x_3x_4 & x_1x_3 + x_1x_4 + x_3x_4 & x_1x_2 + x_2x_4 + x_1x_4 & x_1x_2 + x_2x_3 + x_1x_3 \\
  x_2x_3x_4 & x_1x_3x_4 & x_1x_2x_4 & x_1x_2x_3
\end{bmatrix}
$

# Solution with Roots

```julia
using Roots

p = find_zeros(f, -100, 100)
```

    2-element Array{Float64,1}:
     -1.5789900595892086
      0.8695222544496146

```julia
# test
f.(p)
```

    2-element Array{Float64,1}:
     -1.7763568394002505e-15
     -2.220446049250313e-16

# Solution with PolynomicalRoots

```julia
using PolynomialRoots

p = roots([-1.5, 1.2, -0.5, 0.4, 1])
```

    4-element Array{Complex{Float64},1}:
     -1.5789900595892088 + 0.0im
     0.15473390256979705 + 1.0337223322714935im
      0.8695222544496146 - 8.320880518027517e-17im
     0.15473390256979702 - 1.0337223322714935im

```julia
# test
f.(p)
```

    4-element Array{Complex{Float64},1}:
      2.220446049250313e-16 + 0.0im
     -4.440892098500626e-16 + 2.220446049250313e-16im
     -4.440892098500626e-16 - 3.218046955410157e-16im
     -2.220446049250313e-16 - 2.220446049250313e-16im

## Adjourn

```julia
using Dates
println("mahdiar")
Dates.format(now(), "Y/U/d HH:MM")
```

    mahdiar

    "2021/March/18 17:50"


# 数值积分

```python
from sympy import *
```

## Newton-Cotes求积公式

- trapezium & simpson

```python
def trapezium_integral(f, a, b):
    """梯形求积公式
    """
    return (b - a) * (f(a) + f(b)) / 2

def simpson_integral(f, a, b):
    """辛普森求积公式
    """
    return (b - a) * (f(a) + 4 * f((a + b) / 2) + f(b)) / 6
```

- Newton-Cotes

```python
def costes_coefficient(n, k):
    """求柯特斯系数
    """
    ckn = ((-1) ** (n - k))
    ckn /= n * factorial(k) * factorial(n - k)

    h = 1
    t = Symbol('t')
    for j in range(n+1):
        if j != k:
            h *= (t - j)

    ckn *= integrate(h, (t, 0, n))

    return ckn
```

打出一张表测试科特斯系数：

```python
for i in range(6):
    for j in range(i+2):
        print(costes_coefficient(i+1, j), end= "\t ")
    print()
```

    1/2	 1/2
    1/6	 2/3	 1/6
    1/8	 3/8	 3/8	 1/8
    7/90	 16/45	 2/15	 16/45	 7/90
    19/288	 25/96	 25/144	 25/144	 25/96	 19/288
    41/840	 9/35	 9/280	 34/105	 9/280	 9/35	 41/840

```python
def newton_cotes_integral(f, a, b, n):
    """牛顿-科特斯求积公式
    """
    step = (b - a) / n
    xs = [a + i * step for i in range(n+1)]
    return (b - a) * sum([
        costes_coefficient(n, k) * f(xs[k])
        for k in range(0, n+1)
    ])
```

牛顿-科特斯求积公式测试：

```python
r = newton_cotes_integral(sin, 0, pi, 10)

x = Symbol('x')
print(abs(r - integrate(sin(x), (x, 0, pi))) < 0.00001)
```

    True

## 复化求积公式

- 复化梯形公式

```python
def composite_trapezium_integral(f, a, b, epsilon, max_iter=10000):
    """ 复化梯形公式

    Args:
        f: 要求积的函数
        a, b: 求积区间
        epsilon: 目标精度，达到则停止，返回积分值
        max_iter: 最大迭代次数，超出这个次数迭代不到目标精度，则 raise 一个 RuntimeError

    Returns:
        i, iter
        i: 最终得到的积分值
        iter_times: 迭代次数

    Raises:
        RuntimeError: 无法在 max_iter 步内迭代到目标精度
    """
    m = 1
    h = b - a
    t = h * (f(a) + f(b)) / 2

    _iter_times = 0

    t_next = 0
    for _iter_times in range(int(max_iter)):
        h /= 2
        s = sum([f(a + (2 * k - 1) * h) for k in range(1, m+1)])
        t_next = t / 2 + h * s
        m <<= 1
        if abs(t_next - t) <= epsilon:
            break
        t = t_next
    else:
        raise RuntimeError('无法在 max_iter 步内迭代到目标精度')

    return t_next, _iter_times+1
```

测试复化梯形公式：

```python
def f(x):
    return x ** 2

x = Symbol('x')
a = integrate(f(x), (x, 0, 1))

t = composite_trapezium_integral(f, 0, 1, 0.0001)
print(f"result_composite_trapezium_integral = {t}, actual: {a}")
```

    result_composite_trapezium_integral = (0.333343505859375, 7), actual: 1/3

- 复化 Simpson 公式

```python
def composite_simpson_integral(f, a, b, epsilon, max_iter=1e6):
    """复化 Simpson 公式

    Args:
        f: 要求积的函数
        a, b: 求积区间
        epsilon: 目标精度，达到则停止，返回积分值
        max_iter: 最大迭代次数，超出这个次数迭代不到目标精度，则 raise 一个 RuntimeError

    Returns:
        i, iter
        i: 最终得到的积分值
        iter_times: 迭代次数

    Raises:
        RuntimeError: 无法在 max_iter 步内迭代到目标精度
    """
    m = 1
    h = (b - a) / 2
    i = h * (f(a) + 4 * f((a+b) / 2) + f(b)) / 3

    _iter_times = 0

    i_next = 0
    for _iter_times in range(int(max_iter)):
        h /= 2

        s0 = sum([f(a + (2 * k - 1) * h) for k in range(1, 2 * m + 1)])
        s1 = sum([f(a + (4 * k - 2) * h) for k in range(1, m + 1)])
        i_next = i / 2 + h * (4 * s0 - 2 * s1) / 3

        m <<= 1
        if abs(i_next - i) <= epsilon:
            break
        i = i_next
    else:
        raise RuntimeError('无法在 max_iter 步内迭代到目标精度')

    return i_next, _iter_times+1
```

```python
def f(x):
    return 3 * x ** 3 - 2 * x + sin(x)

x = Symbol('x')
a = integrate(f(x), (x, 0, 1))

t, times = composite_simpson_integral(f, 0, 1, 0.0001)
print(f"result_composite_simpson_integral = {t}, actual: {a}\ndiff < 0.0001: {abs(t-a) < 0.0001}")
```

    result_composite_simpson_integral = 0.0416666666666667*sin(1) + 0.174637027764799, actual: 3/4 - cos(1)
    diff < 0.0001: True

龙贝格算法

```python
def romberg_integral(f, a, b, epsilon, max_iter=1e6):
    """Romberg 积分

    Args:
        f: 要求积的函数
        a, b: 求积区间
        epsilon: 目标精度，达到则停止，返回积分值
        max_iter: 最大迭代次数，超出这个次数迭代不到目标精度，则 raise 一个 RuntimeError

    Returns:
        result, T, iter

        result: 最终得到的积分值
        T: 计算过程表
        iter_times: 迭代次数

    Raises:
        RuntimeError: 无法在 max_iter 步内迭代到目标精度
    """
    m = 0

    T = [[None] * 4]
    T[0][0] = (b - a) * (f(a) + f(b)) / 2

    for m in range(1, int(max_iter)):
        T.append([None] * 4)

        h = (b - a) / (2 ** m)

        _s = sum((f(a + (2 * k - 1) * h) for k in range(1, 1 + 2**(m-1))))
        T[m][0] = T[m-1][0] / 2 + h * _s

        _t = 4      # 4 ** 1
        T[m-1][1] = (_t * T[m][0] - T[m-1][0]) / (_t - 1)

        if m > 1:
            _t *= 4  # 4 ** 2
            T[m-2][2] = (_t * T[m-1][1] - T[m-2][1]) / (_t - 1)
        if m > 2:
            _t *= 4  # 4 ** 3
            T[m-3][3] = (_t * T[m-2][2] - T[m-3][2]) / (_t - 1)

        if (m > 3) and (abs(T[m-3][3] - T[m-4][3]) < epsilon):
            break
        # End for
    else:
        raise RuntimeError('无法在 max_iter 步内迭代到目标精度')

    return T[m-3][3], T, m
```

```python
import math

def f(x):
    return 3 * x ** 3 - 2 * x + sin(x)

x = Symbol('x')
a = integrate(f(x), (x, 2, 10))

t, T, iter_times = romberg_integral(f, 2, 10, 1e-10)
print(f"result_romberg_integral = {t}, actual: {a}\ndiff < 0.0001: {abs(t-a) < 0.0001}")
print(iter_times, len(str(T)))
print(T)
```

    result_romberg_integral = 31*sin(10)/1620 + 31*sin(2)/1620 + 7392.4159348371, actual: cos(2) - cos(10) + 7392
    diff < 0.0001: True
    7 1216
    [[4*sin(10) + 4*sin(2) + 12000, 4*sin(10)/3 + 4*sin(2)/3 + 7390.50978400961, 28*sin(10)/45 + 28*sin(2)/45 + 7392.36344871677, 124*sin(10)/405 + 124*sin(2)/405 + 7392.30668995305], [2*sin(10) + 2*sin(2) + 8542.8823380072, 2*sin(10)/3 + 2*sin(2)/3 + 7392.24759467258, 14*sin(10)/45 + 14*sin(2)/45 + 7392.30757680873, 62*sin(10)/405 + 62*sin(2)/405 + 7392.36701663887], [sin(10) + sin(2) + 7679.90628050623, sin(10)/3 + sin(2)/3 + 7392.30382792522, 7*sin(10)/45 + 7*sin(2)/45 + 7392.36608789152, 31*sin(10)/405 + 31*sin(2)/405 + 7392.39496529595], [sin(10)/2 + sin(2)/2 + 7464.20444107047, sin(10)/6 + sin(2)/6 + 7392.36219664363, 7*sin(10)/90 + 7*sin(2)/90 + 7392.39451408651, 31*sin(10)/810 + 31*sin(2)/810 + 7392.40894498176], [sin(10)/4 + sin(2)/4 + 7410.32275775034, sin(10)/12 + sin(2)/12 + 7392.39249424633, 7*sin(10)/180 + 7*sin(2)/180 + 7392.40871949902, 31*sin(10)/1620 + 31*sin(2)/1620 + 7392.4159348371], [sin(10)/8 + sin(2)/8 + 7396.87506012233, sin(10)/24 + sin(2)/24 + 7392.40770542073, 7*sin(10)/360 + 7*sin(2)/360 + 7392.41582209745, None], [sin(10)/16 + sin(2)/16 + 7393.52454409613, sin(10)/48 + sin(2)/48 + 7392.41531480515, None, None], [sin(10)/32 + sin(2)/32 + 7392.6926221279, None, None, None]]

```python
def romberg_integral_sw(f, a, b, epsilon, max_iter=1e6):
    """Romberg 积分

    用滑动窗口改写了 romberg_integral。
    在计算的过程中，不断删除不再需要的值，
    对于需要多次迭代的问题，可以有效节省内存。

    Args:
        f: 要求积的函数
        a, b: 求积区间
        epsilon: 目标精度，达到则停止，返回积分值
        max_iter: 最大迭代次数，超出这个次数迭代不到目标精度，则 raise 一个 RuntimeError

    Returns:
        result, T, iter

        result: 最终得到的积分值
        T: 计算过程表（只有最后的 m-4 到 m 步）
        iter_times: 迭代次数

    Raises:
        RuntimeError: 无法在 max_iter 步内迭代到目标精度
    """
    m = 0

    T = [[None] * 4]
    T[0][0] = (b - a) * (f(a) + f(b)) / 2

    for m in range(1, int(max_iter)):
        T.append([None] * 4)
        # 下面的索引 T[-1] 即 T_m, T[-2] 即 T_{m-1}, ...

        h = (b - a) / (2 ** m)

        _s = sum((f(a + (2 * k - 1) * h) for k in range(1, 1 + 2**(m-1))))
        T[-1][0] = T[-2][0] / 2 + h * _s

        _t = 4      # 4 ** 1
        T[-2][1] = (_t * T[-1][0] - T[-2][0]) / (_t - 1)

        if m > 1:
            _t *= 4  # 4 ** 2
            T[-3][2] = (_t * T[-2][1] - T[-3][1]) / (_t - 1)
        if m > 2:
            _t *= 4  # 4 ** 3
            T[-4][3] = (_t * T[-3][2] - T[-4][2]) / (_t - 1)

        if (m > 3) and (abs(T[-4][3] - T[-5][3]) < epsilon):
            break

        if (m > 4):
            T = T[1:] # 清除不需要的
        # End for
    else:
        raise RuntimeError('无法在 max_iter 步内迭代到目标精度')

    return T[-4][3], T, m
```

```python
import math

def f(x):
    return 3 * x ** 3 - 2 * x + sin(x)

x = Symbol('x')
a = integrate(f(x), (x, 2, 10))

t, T, iter_times = romberg_integral_sw(f, 2, 10, 1e-10)
print(f"result_romberg_integral_sw = {t}, actual: {a}\ndiff < 0.0001: {abs(t-a) < 0.0001}")
print(iter_times, len(str(T)))
print(T)
```

    result_romberg_integral_sw = 31*sin(10)/1620 + 31*sin(2)/1620 + 7392.4159348371, actual: cos(2) - cos(10) + 7392
    diff < 0.0001: True
    7 850
    [[sin(10) + sin(2) + 7679.90628050623, sin(10)/3 + sin(2)/3 + 7392.30382792522, 7*sin(10)/45 + 7*sin(2)/45 + 7392.36608789152, 31*sin(10)/405 + 31*sin(2)/405 + 7392.39496529595], [sin(10)/2 + sin(2)/2 + 7464.20444107047, sin(10)/6 + sin(2)/6 + 7392.36219664363, 7*sin(10)/90 + 7*sin(2)/90 + 7392.39451408651, 31*sin(10)/810 + 31*sin(2)/810 + 7392.40894498176], [sin(10)/4 + sin(2)/4 + 7410.32275775034, sin(10)/12 + sin(2)/12 + 7392.39249424633, 7*sin(10)/180 + 7*sin(2)/180 + 7392.40871949902, 31*sin(10)/1620 + 31*sin(2)/1620 + 7392.4159348371], [sin(10)/8 + sin(2)/8 + 7396.87506012233, sin(10)/24 + sin(2)/24 + 7392.40770542073, 7*sin(10)/360 + 7*sin(2)/360 + 7392.41582209745, None], [sin(10)/16 + sin(2)/16 + 7393.52454409613, sin(10)/48 + sin(2)/48 + 7392.41531480515, None, None], [sin(10)/32 + sin(2)/32 + 7392.6926221279, None, None, None]]

题目 4.4 实验内容和步骤

$$
I_1 = \int_0^1 e^{-x^2} dx
$$

```python
import math

def f1(x):
    if not isinstance(x, float):
        x = float(x)
    return math.exp(-x ** 2)

result_trapezium = trapezium_integral(f1, 0, 1)
result_simpson = simpson_integral(f1, 0, 1)
result_composite_simpson, times_composite_simpson = composite_simpson_integral(f1, 0, 1, 1e-6)
result_composite_trapezium, times_composite_trapezium = composite_trapezium_integral(f1, 0, 1, 1e-6)
result_romberg, _, times_romberg = romberg_integral(f1, 0, 1, 1e-6)

actual = integrate(exp(-Symbol('x') ** 2), (Symbol('x'), 0, 1))

print(f'''integral exp(-x ** 2) dx from 0 to 1

actual result (by sympy): {actual} = {actual.evalf()}

result_trapezium={result_trapezium}
result_simpson={result_simpson}
result_composite_trapezium={result_composite_trapezium}, times_composite_trapezium={times_composite_trapezium}
result_composite_simpson={result_composite_simpson}, times_composite_simpson={times_composite_simpson}
result_romberg={result_romberg}, times_romberg={times_romberg}
''')
```

    integral exp(-x ** 2) dx from 0 to 1

    actual result (by sympy): sqrt(pi)*erf(1)/2 = 0.746824132812427

    result_trapezium=0.6839397205857212
    result_simpson=0.7471804289095104
    result_composite_trapezium=0.7468238989209475, times_composite_trapezium=9
    result_composite_simpson=0.7468241406069852, times_composite_simpson=4
    result_romberg=0.7468241326473878, times_romberg=4

$$
I_2 = \int_0^1 \frac{\sin(x)}{x} dx
$$

```python
def f2(x):
    if not isinstance(x, float):
        x = float(x)
    return math.sin(x) / x

result_trapezium = trapezium_integral(f2, 1e-32, 1)
result_simpson = simpson_integral(f2, 1e-32, 1)
result_composite_simpson, times_composite_simpson = composite_simpson_integral(f2, 1e-32, 1, 1e-6)
result_composite_trapezium, times_composite_trapezium = composite_trapezium_integral(f2, 1e-32, 1, 1e-6)
result_romberg, _, times_romberg = romberg_integral(f2, 1e-32, 1, 1e-6)

actual = integrate(sin(Symbol('x')) / Symbol('x'), (Symbol('x'), 0, 1))

print(f'''integral sin(x) / x dx from 0 to 1

actual result (by sympy): {actual} = {actual.evalf()}

result_trapezium={result_trapezium}
result_simpson={result_simpson}
result_composite_trapezium={result_composite_trapezium}, times_composite_trapezium={times_composite_trapezium}
result_composite_simpson={result_composite_simpson}, times_composite_simpson={times_composite_simpson}
result_romberg={result_romberg}, times_romberg={times_romberg}
''')
```

    integral sin(x) / x dx from 0 to 1

    actual result (by sympy): Si(1) = 0.946083070367183

    result_trapezium=0.9207354924039483
    result_simpson=0.9461458822735868
    result_composite_trapezium=0.9460829746282349, times_composite_trapezium=9
    result_composite_simpson=0.9460830853849476, times_composite_simpson=3
    result_romberg=0.9460830703672595, times_romberg=4

```python
def adaptsim(f, a, b, eps=1e-8, max_iter=10000):
    """自适应 Simpson 求积

    P.S. 这个函数名来自 Gander, W. and W. Gautschi, “Adaptive
         Quadrature – Revisited,” BIT, Vol. 40, 2000, pp. 84-101.
         该文档可以在 https://people.inf.ethz.ch/gander/ 找到。
         但该函数的实现并没有使用此文中的递归方法。

    Args:
        f: 要求积的函数
        a, b: 求积区间
        eps: 目标精度，达到则停止，返回积分值
        max_iter: 最大迭代次数，超出这个次数迭代不到目标精度，则 raise 一个 Exception

    Returns: (I, m, p)
        I: 积分的近似值
        m: 分层数
        p: 分点

    Raises:
        Exception: 无法在 max_iter 步内迭代到目标精度
    """
    p = [a, b]  # 分点
    p0 = p
    ep = [eps]
    m = 0
    q = 0
    I = 0

    for _iter_times in range(int(max_iter)):
        n1 = len(ep)
        n = len(p0)

        if n <= 1:
            break

        h = p0[1] - p0[0]
        s0 = h /  6 * ( f(p0[0]) + 4 * f(p0[0] + h/2) + f(p0[0] + h  ) )
        s1 = h / 12 * ( f(p0[0]) + 4 * f(p0[0] + h/4) + f(p0[0] + h/2) )
        s2 = h / 12 * ( f(p0[0] + h/2) + 4 * f(p0[0] + 3*h/4) + f(p0[0] + h) )

        if abs(s0 - s1 - s2) <= 15 * ep[0]:
            I += s1 + s2
            p0 = p0[1:]

            if n1 >= 2:
                ep = ep[1:]

            q += 1
        else:
            m += 1
            p0 = [p0[0], p0[0] + h/2] + p0[1:]

            if n1 == 1:
                ep = [ep[0]/2, ep[0]/2]
            else:
                ep = [ep[0]/2, ep[1]/2] + ep[1:]

            if q == 0:
                p = p0
            else:
                p = p[:q] + p0

    else:
        raise Exception('无法在 max_iter 步内迭代到目标精度')

    return I, m, p
```

```python
import math
import numpy as np

def f(x):
    if not isinstance(x, float):
        x = float(x)
    return math.exp(-x ** 2)

# actual result (by sympy): sqrt(pi)*erf(1)/2 = 0.746824132812427

I, m, p = adaptsim(f, 0, 1, eps=1e-9)

# 画图
import matplotlib.pyplot as plt

xf = np.linspace(0, 1)
yf = [f(x) for x in xf]

yp = [f(x) for x in p]

plt.plot(xf, yf, 'r-', label='f=exp(-x ** 2)')
plt.plot(p, yp, 'bo', label='adaptive points')
plt.legend()
plt.xlim(0, 1)
plt.show()
```

```python
import math

def f(x):
    if not isinstance(x, float):
        x = float(x)
    return math.sin(x) / x

# actual result (by sympy): Si(1) = 0.946083070367183

I, m, p = adaptsim(f, 1e-64, 1)

# 画图
import matplotlib.pyplot as plt

xf = np.linspace(1e-64, 1)
yf = [f(x) for x in xf]

yp = [f(x) for x in p]

plt.plot(xf, yf, 'r-', label='f=sin(x) / x')
plt.plot(p, yp, 'bo', label='adaptive points')
plt.legend()
plt.xlim(0, 1)
plt.show()
```

```python
import math
import numpy as np
import matplotlib.pyplot as plt

def f(x):
    return (100 / x**2) * math.sin(10 / x)

I, m, p = adaptsim(f, 1, 3, eps=1e-4)
print(f'I={I}')

xf = np.linspace(1, 3)
yf = [f(x) for x in xf]

yp = [f(x) for x in p]

plt.plot(xf, yf, 'r-', label='f=100 / x**2 * sin(10 / x)')
plt.plot(p, yp, 'bo', label='adaptive points')
plt.legend()
plt.xlim(1, 3)
plt.show()
```

```python
import math
import matplotlib.pyplot as plt
import matplotlib.animation as animation

def f(x):
    return (100 / x**2) * math.sin(10 / x)

fig = plt.figure()

xf = np.linspace(1, 3)
yf = [f(x) for x in xf]
plt.plot(xf, yf, 'r-', label='f=100 / x**2 * sin(10 / x)')
plt.legend()
plt.xlim(1, 3)

text_pt = plt.text(2.05, 30, '', fontsize=12)

adaptsim_points, = plt.plot([1], [f(1)], 'bo', label='adaptsim points')

def update_adaptsim(i):
    eps = 10 ** (-int(i))

    I, m, p = adaptsim(f, 1, 3, eps=eps)
    yp = [f(x) for x in p]

    adaptsim_points.set_data(p, yp)
    text_pt.set_text(f"eps={eps}\npoints={m}\nI={I: .10f}")

    return adaptsim_points,

ani = animation.FuncAnimation(fig, update_adaptsim, np.arange(0, 8), interval=200, blit=True)

ani.save('adaptsim.gif', writer='imagemagick', fps=2)
# plt.show()
```

```python
def asr(f, a, b, eps=1e-8):
    """自适应 Simpson 求积的递归实现

    Args:
        f: 要求积的函数
        a, b: 求积区间
        eps: 目标精度，达到则停止，返回积分值

    Returns:
        I: 计算结果：积分的近似值
    """
    def simpson(f, a, b):
        return (b - a) * (f(a) + 4 * f((a + b) / 2) + f(b)) / 6

    def asrp(f, a, b, eps, sim):
        mid = (a + b) / 2
        L = simpson(f, a, mid)
        R = simpson(f, mid, b)

        if (abs(L + R - sim) <= eps):
            return L + R + (L + R - sim) / 15

        return asrp(f, a, mid, eps/2, L) + asrp(f, mid, b, eps/2, R)

    return asrp(f, a, b, eps, simpson(f, a, b))
```

```python
asr(lambda x: math.exp(-x ** 2), 0, 1)
# actual result (by sympy): sqrt(pi)*erf(1)/2 = 0.746824132812427
```

    0.7468241328125185

```python
asr(lambda x: math.sin(x) / x, 1e-64, 1)
# actual result (by sympy): Si(1) = 0.946083070367183
```

    0.9460830703671792

```python
asr(lambda x: (100 / x**2) * math.sin(10 / x), 1, 3)
```

    -1.42602475634631


```python
import ipywidgets as widgets
#https://towardsdatascience.com/bring-your-jupyter-notebook-to-life-with-interactive-widgets-bc12e03f0916
from IPython.display import display

#Have a table to compare when the best time to use radar vs ultrasound (cross reference on the slides)
speed_of_ultrasound_m_per_sec={ 'tissue':1540, #In most tissue
                                'water':1481,
                                'air': 343
                              }
speed_of_light_m_per_sec={ 'air': 299792458
                              }
```

Our goal is to understand the effect of the speed of a wave on how far it travels and how that influences how many computation

If we define the time the wave is traveling as:

\begin{equation}
\Delta t=t_{end}-t_{start}
\end{equation}

We can find the distance the wave travels given we know its speed (c)
\begin{equation}
	d=c\Delta t
\end{equation}

This means for a round trip time
\begin{equation}
	d=\frac{c}{2}\Delta t
\end{equation}

The two types of waves we will be examining are ultrasound and radio waves. In air ultrasound has a  speed of:
\begin{equation}
c_{ultrasound}=343 \frac{m}{s}
\end{equation}

while radio waves have a speed of:
\begin{equation}
c_{light}=299,792,458 \frac{m}{s}
\end{equation}

Below find the time it would take for an ultrasound wave to travel 10 meters (+/- 1m). Compare this distance with the distance radio waves would travel. What is the ratio of the difference?

```python
from ipywidgets import Layout
style = {'description_width': 'initial'}

time_traveled_output = widgets.Output()
time_traveled_slider = widgets.IntSlider(
    min=1,
    max=100,
    step=10,
    description='Time traveled (milliseconds):',
    value=3,
    layout=Layout(width='auto', height='80px'),
    style=style
)

def time_traveled_eventhandler(change):
    time_traveled_output.clear_output()
    with time_traveled_output:
        t_milliseconds = time_traveled_slider.value
        t_sec = t_milliseconds/1000
        print(f"For a time of {t_milliseconds} milliseconds the distance traveled for ultrasound is {(speed_of_ultrasound_m_per_sec['air']*t_sec):.1f} m")
        print(f"For a time of {t_milliseconds} milliseconds the distance traveled for light is {(speed_of_light_m_per_sec['air']*t_sec):.1f} m")

time_traveled_slider.observe(handler=time_traveled_eventhandler,type='change')
display(time_traveled_slider)

```

    IntSlider(value=3, description='Time traveled (milliseconds):', layout=Layout(height='80px', width='auto'), mi…

```python
display(time_traveled_output)
```

    Output()

Assume you have a car with both ultrasound and radar sensors. Your task is to design a system that can detect a car when it is 10 meters in front of your car. Change the variables `t_ultrasound_sec` and `t_radar_sec` in the cell below to the correct values for a 20 meter round trip time.

```python
t_ultrasound_sec=0.1
t_radar_sec=0.1

#Do not change anything below this line
d_m=10.0
d_ultrasound_m = t_ultrasound_sec*speed_of_ultrasound_m_per_sec['air']
d_radar_m = t_radar_sec*speed_of_light_m_per_sec['air']
print(f"An ultrasound signal completes a round trip distance of {d_ultrasound_m:.1f} meters in {t_ultrasound_sec} seconds.")
print(f"A radar signal completes a round trip distance of {d_radar_m:.1f} meters in {t_ultrasound_sec} seconds.")
```

    An ultrasound signal completes a round trip distance of 34.3 meters in 0.1 seconds.
    A radar signal completes a round trip distance of 29979245.8 meters in 0.1 seconds.

In this final section lets look at how many instructions a CPU that can complete 600 megaflops in a second can process. Which signal is the easiest to perform signal processing on and by what factor? Later we will investigate what level of processing can be completed by common algorihms such as the Fast Fourier Transform.

```python
CPU_flops=600e6
print(f"The number of flops a CPU can use to process the ultrasound signal is: {int(CPU_flops*t_ultrasound_sec)} flops")
print(f"The number of flops a CPU can use to process the radar signal is: {int(CPU_flops*t_radar_sec)} flops")
```

    The number of flops a CPU can use to process the ultrasound signal is: 60000000 flops
    The number of flops a CPU can use to process the radar signal is: 60000000 flops


# Example {-}
A slightly modified version of example 3.11 demonstrating various strategies on how to compute the transition matrix $\phi$ and the process noise matrix $Q$.

```python
from sympy import symbols, Matrix, sqrt, eye, inverse_laplace_transform, integrate

s, q = symbols('s q')
t = symbols('t', positive=True)

# Dynamic matrices
F1 = Matrix([[0, 1],
             [0, 0]])

F2 = Matrix([[0, 1],
             [-4, -2*sqrt(2)]])

# White noise coefficient vector
G = Matrix([[0],
            [sqrt(q)]])

# Compute transition matrix (from F1)
phi1 = eye(2) + F1*t
display(phi1)

phi1 = inverse_laplace_transform((s*eye(2) - F1).inv(), s, t)
display(phi1)
```

$\displaystyle \left[\begin{matrix}1 & t\\0 & 1\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}1 & t\\0 & 1\end{matrix}\right]$

```python
# Compute transition matrix (from F2)
phi2 = eye(2) + F2*t + (F2*t)**2/2 + (F2*t)**3/6 # + more terms -> i.e. only approximate
display(phi2)

phi2 = inverse_laplace_transform((s*eye(2) - F2).inv(), s, t)
display(phi2)
```

$\displaystyle \left[\begin{matrix}\frac{4 \sqrt{2} t^{3}}{3} - 2 t^{2} + 1 & \frac{2 t^{3}}{3} - \sqrt{2} t^{2} + t\\- \frac{8 t^{3}}{3} + 4 \sqrt{2} t^{2} - 4 t & 2 t^{2} - 2 \sqrt{2} t + 1\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}\left(\sin{\left(\sqrt{2} t \right)} + \cos{\left(\sqrt{2} t \right)}\right) e^{- \sqrt{2} t} & \frac{\sqrt{2} e^{- \sqrt{2} t} \sin{\left(\sqrt{2} t \right)}}{2}\\- 2 \sqrt{2} e^{- \sqrt{2} t} \sin{\left(\sqrt{2} t \right)} & \left(- \sin{\left(\sqrt{2} t \right)} + \cos{\left(\sqrt{2} t \right)}\right) e^{- \sqrt{2} t}\end{matrix}\right]$

```python
# Process noise (from F1)
Q1 = integrate(phi1@G@G.T@phi1.T, t)
display(Q1)
```

$\displaystyle \left[\begin{matrix}\frac{q t^{3}}{3} & \frac{q t^{2}}{2}\\\frac{q t^{2}}{2} & q t\end{matrix}\right]$

```python
from numpy import array, sqrt, eye
from lib.vanloan import numeval

# System parameters
t = 1  # [second]
q = 1  # [m^2/s^3]

# Dynamic matrix
F1 = array([[0, 1],
            [0, 0]])

F2 = array([[0, 1],
            [-4, -2*sqrt(2)]])

# White noise coefficient vector
G = array([[0],
           [sqrt(q)]])

# Numerical evaluation (from F1)
[phi1, Q1] = numeval(F1, G, t)
print(phi1)
print(Q1)
```

    [[1. 1.]
     [0. 1.]]
    [[0.33333333 0.5       ]
     [0.5        1.        ]]

```python
# Numerical evaluation (from F2)
[phi2, Q2] = numeval(F2, G, t)
print(phi2)
print(Q2)
```

    [[ 0.27805495  0.16980634]
     [-0.67922537 -0.20222991]]
    [[0.03568011 0.0144171 ]
     [0.0144171  0.14915822]]


```python
from sympy import *
```

```python
q = Symbol('Pr(z_t=0|z_{t-1}=0)')
p = Symbol('Pr(z_t=0|z_{t-1}=1)')
M = Matrix([[q,1-q],[p,1-p]])
print M
P,D = M.diagonalize()
print P
print D
```

    Matrix([[Pr(z_t=0|z_{t-1}=0), -Pr(z_t=0|z_{t-1}=0) + 1], [Pr(z_t=0|z_{t-1}=1), -Pr(z_t=0|z_{t-1}=1) + 1]])
    Matrix([[1, (Pr(z_t=0|z_{t-1}=0) - 1)/Pr(z_t=0|z_{t-1}=1)], [1, 1]])
    Matrix([[1, 0], [0, Pr(z_t=0|z_{t-1}=0) - Pr(z_t=0|z_{t-1}=1)]])

```python
simplify( P*D*(P**(-1)))
D_lim = Matrix([[1,0],[0,0]])
M_lim = simplify( P*D_lim*(P**(-1)))
#(p + 1-q)
```

    Matrix([[Pr(z_t=0|z_{t-1}=1), -Pr(z_t=0|z_{t-1}=0) + 1], [Pr(z_t=0|z_{t-1}=1), -Pr(z_t=0|z_{t-1}=0) + 1]])

```python
r= Symbol('Pr(z_{t-1}=0)')
init = Matrix([[r,1-r]])

print simplify( init* M_lim)
```

    Matrix([[Pr(z_t=0|z_{t-1}=1)/(-Pr(z_t=0|z_{t-1}=0) + Pr(z_t=0|z_{t-1}=1) + 1), (-Pr(z_t=0|z_{t-1}=0) + 1)/(-Pr(z_t=0|z_{t-1}=0) + Pr(z_t=0|z_{t-1}=1) + 1)]])

```python
D_lim2 = Matrix([[1,0],[0,1]])
print simplify( init.T * P*D_lim2*(P**(-1)) )[0]
```

    Pr(z_{t-1}=0)

```python
D_lim3 = Matrix([[1,0],[0,-1]])
print simplify( init.T * P*D_lim3*(P**(-1)) )[0]
```

    (Pr(z_t=0|z_{t-1}=1)*(-Pr(z_{t-1}=0) + 1) + Pr(z_t=0|z_{t-1}=1) + Pr(z_{t-1}=0)*(Pr(z_t=0|z_{t-1}=0) - 1))/(-Pr(z_t=0|z_{t-1}=0) + Pr(z_t=0|z_{t-1}=1) + 1)

```python

```


# 2D Biharmonic equation

We now consider the (inhomogeneous) biharmonic equation on the unit square $\Omega := (0,1) \times (0,1)$, with homogeneous essential boundary conditions on all sides.
Given a function $f \colon \Omega \to \mathbb{R}$, we search for a solution $\phi \colon \Omega \to \mathbb{R}$ that satisfies

\begin{equation}
\label{eq:BiharmonicEx_Dir_Strong}
\left\{
\begin{aligned}
        \nabla^4 \phi &= f  && \text{in $\Omega$}, \\
                    \phi &= 0  && \text{on $\partial\Omega$}, \\
     \nabla \phi \cdot \mathbf{n} &= 0  && \text{on $\partial\Omega$},
\end{aligned}
\right.
\end{equation}
where $\mathbf{n}$ is the unit vector normal to the boundary, pointing outwards.

If we let $f \in L^2(\Omega)$ and choose $H^2_0(\Omega)$ for the trial and test spaces, the weak formulation of reads:

\begin{equation}
\label{eq:BiharmonicEx_Dir_Weak}
        \text{Find $\phi \in H^2_0(\Omega)$ s.t.}
\quad
        \int_{\Omega} \nabla^2\phi\, \nabla^2\psi ~d\Omega = \int_{\Omega} f \psi ~d\Omega
\quad
        \forall \psi \in H^2_0(\Omega).
\end{equation}

For testing purposes we choose a manufactured solution $\phi$ that vanishes at the boundary, and whose gradient also vanishes:

\begin{equation}
\begin{split}
        \phi(x,y) &:= \sin^2(\pi x) \sin^2(\pi y), \\
           f(x,y) &:= \pi^{4} \left(64 \sin^{2}{\left (\pi x \right )} \sin^{2}{\left (\pi y \right )}
                     - 24 \sin^{2}{\left (\pi x \right )} - 24 \sin^{2}{\left (\pi y \right )} + 8\right).
\end{split}
\end{equation}

```python
# Exact solution
import numpy as np
u = lambda x,y: (np.sin(np.pi*x)*np.sin(np.pi*y))**2

ts = np.linspace(0., 1., 401)
x,y = np.meshgrid(ts,ts)

from matplotlib import pyplot as plt
plt.contourf(x,y, u(x,y)); plt.colorbar() ; plt.show()
```

### Abstract Model

```python
from sympy  import sin, pi

from sympde.calculus import laplace, dot, grad
from sympde.topology import Square, NormalVector
from sympde.topology import ScalarFunctionSpace, element_of
from sympde.expr     import integral, BilinearForm, LinearForm
from sympde.expr     import EssentialBC, find, Norm

from psydac.api.discretization import discretize
```

```python
# Define topological domain
Omega = Square()

# Method of manufactured solutions: define exact
# solution phi_e, then compute right-hand side f
x, y  = Omega.coordinates
phi_e = (sin(pi*x)*sin(pi*y))**2
f     = laplace(laplace(phi_e))

# Define abstract model
V = ScalarFunctionSpace('V', Omega)
v = element_of(V, name='v')
u = element_of(V, name='u')

a = BilinearForm((u,v), integral(Omega, laplace(v) * laplace(u)))
l =   LinearForm(   v , integral(Omega, f * v))

nn = NormalVector('nn')
bc = [EssentialBC(               u, 0, Omega.boundary),
      EssentialBC(dot(grad(u), nn), 0, Omega.boundary)]

equation = find(u, forall=v, lhs=a(u,v), rhs=l(v), bc=bc)

# Define (abstract) error norms
phi    = element_of(V, name='phi')
error  = phi - phi_e
l2norm = Norm(error, Omega, kind='l2')
h1norm = Norm(error, Omega, kind='h1')
h2norm = Norm(error, Omega, kind='h2')
```

### Discretization

```python
degree = [3,3]
ncells = [16,16]
```

```python
# Create computational domain from topological domain
Omega_h = discretize(Omega, ncells=ncells, comm=None)

# Create discrete spline space
Vh = discretize(V, Omega_h, degree=degree)

# Discretize equation
equation_h = discretize(equation, Omega_h, [Vh, Vh])

# Discretize norms
l2norm_h = discretize(l2norm, Omega_h, Vh)
h1norm_h = discretize(h1norm, Omega_h, Vh)
h2norm_h = discretize(h2norm, Omega_h, Vh)
```

### Solving the PDE

```python
# Solve discrete equation to obtain numerical solution
phi_h = equation_h.solve()

# Compute error norms from solution field
l2_error = l2norm_h.assemble(phi=phi_h)
h1_error = h1norm_h.assemble(phi=phi_h)
h2_error = h2norm_h.assemble(phi=phi_h)
```

```python
print('> L2-norm = ', l2_error)
print('> H2-norm = ', h1_error)
print('> H2-norm = ', h2_error)
```

    > L2-norm =  1.0838207325758795e-05
    > H2-norm =  0.0006993811311579355
    > H2-norm =  0.050092563231348974

### Plotting the solution

```python
from matplotlib.pyplot import plot, contourf, show, colorbar
%matplotlib inline
from utilities.plot import plot_field_2d
```

```python
nbasis = [W.nbasis for W in Vh.spaces]
x = phi_h.coeffs.toarray().reshape(nbasis)
plot_field_2d(Vh.knots, Vh.degree, x ); colorbar() ; show()
```

```python

```

```python

```

```python

```