# Bridging the epidemic models:
from exponential to SIR/SEIR/SIRX to Agent-based models

## An interactive notebook/lesson on nowcasting epidemic spreading
Christopher Monterola* and Erika Fille Legara
Aboitiz Chairs for Data Science

### General Notes

We provide in this Jupyter notebook the bridging mathematics needed to understand how the different classes of epidemic models are related to each other, and to what extent one model can reproduce the result of another. This is a notebook trilogy highlighting different facets of epidemic modeling:

**Notebook 1.** A case study on fitting and trend forecasting.
**Notebook 2.** A theoretical framework and numerical demonstration on how different models are derivable from one another. Covered in this notebook are algorithms and discussions on SIR, SEIR models. Also provided is an algorithm for changing reproduction number through Bayesian fitting. This is written as a live case on mathematical modeling and provides scenario modelling of epidemics based on real time data. Background on basic python programming and differential calculus necessary.
**Notebook 3.** From compartmental models to real-world granular model of epidemics. On the changing basic reproduction number.
**Notebook 4.** On Epidemic Indicators and Forecasting the New Normal: Is measuring $R_0$ sufficient?

-----
*Communications to:
cmonterola@aim.edu
Head, Aboitiz School of Innovation, Technology and Entrepreneurship (ASITE)
Head, Analytics, Computing and Complex Systems Laboratory (ACCeSs)
Professor, Asian Institute of Management (AIM)
Website: www.chrismonterola.net

## Introduction

A novel coronavirus (COVID-19) virus has crippled society, markets, and economies at an alarming rate globally. On 11 March 2020, the World Health Organization officially described the situation as a pandemic, which at that time had already resulted to 118,000 cases (detected) in 114 countries, and 4,291 deaths.  The emergence of this pandemic is not unexpected \cite{webby2003,mills2004}; with the 1918 pandemic the worst to date, killing an estimated 50-100 million persons. In fact, not only are pandemics expected to reemerge, they are expected to emerge in an increased frequency, which is mainly due to the of the increasing emergence of viral disease from animals~\cite{madhav2017}.

One of the challenges right now for most countries is to understand the pandemic potential of COVID-19 as government agencies, specially health organizations, struggle to acquire enough manpower and healthcare resources to track and detect the prevalence of the virus, with hopes of slowing it down. Aside from the socioeconomic impact of this resource scarcity, it has also resulted to limited and unreliable data for some countries including the Philippines. One obvious reason for under detection is limited testing kits that has gained a mass attention and call for immediate action in many developing nations like the Philippines, Iran, Myanmar, India, etc.

In this work, we develop a framework to bridge some of these data gaps, allowing us to nowcast the spread of epidemics using a simple exponential fitting that is optimize specifically for COVID19. This proposed approach also allows us to make estimates on when epidemic growths would peak and then eventually flatten. The work proposed by Fergusson *et al.* is most probably inaccurate as it underestimates the inherent stochasticity of the network and medium of spreading of the virus. In such a context, an adaptive empirical model, has more practical value especially on the impact of community quarantines and/or mass testing.

The work, therefore, provides an early assessment of effectiveness of various social distancing measures (and other measures) to potentially flatten the curve. With such a tool, assessment for different countries with different demographic make-up and geospatial signatures can be compared in real time.

## Section 1. Theoretical Framework

### 1.1 Exponential: a generic model for random epidemic spreading (at the onset)

Finding the peak of an epidemic in the real-world and in real-time is complicated mainly because:

1. the network of interaction is both complex and spatiotemporally dynamic. For example, it can evolve as result of intervention (normal vs. total lockdown) and/or has strong dependence on spatial setting (e.g. different spread dynamics in office, home, and transport points).

2. the extent it can spread varies both in terms of the emedium (airborne vs. plastic surface vs. metal surface) and different environmental parameters (temperature, humidity, etc).

A first approximation is to simplify the above by assuming that a generic mechanism can be derived to account for all of these factors. At the onset, the spreading can be transmitted randomly with a rate $r$ at time $t$ as:

\begin{equation}
y(t) = y_t = y_0 (1 + r)^t
\end{equation}

where $y(t)$ is the number of infected population at time $t$ and $y_0$ is the initial infected population. This result is especially true for COVID19 that has been shown to be asymptomatic for 11.5 days and hence it can infect the population with an effective rate $r$. Note that this immediately follows from the fact that at $t=1$ the number of infected is given by $y_{t=1}=y_0 + r y_0 = y_0 (1 + r)$, at $t=2$ this becomes $y_{t=2}= y_0 (1 + r) + r y_0 (1 + r) = y_0 (1 + r)^2$, and so on and so forth; and, by intuition, we can get the above expression.

The expression can be rewritten in a more compact form as exponential:

\begin{equation}
y = y_0 \exp^{bt}
\end{equation}

where the rate $r$ is derivable as:

\begin{equation}
r=\exp^{b}-1
\end{equation}

From the expression above, one can show that the infected population will double every

\begin{equation}
t_{\texttt{double}}= \frac{\ln(2)}{\ln(1+r)}.
\end{equation}

The above approximation holds as long as the number of people infected continue to grow unabated, which is true under the assumption that the system is still far from capacity ($y_t << y_{max}$); that is, there are still many susceptible individuals to infection.

### Some Exercises

#### Exercise 1. Prove that $y = y_0 (1 + r)^t$ can be written as $y = y_0 \exp^{bt}$ where $r=\exp^{b}-1$.

#### Solution:

Take the logarithm ($\ln$) of both sides.

\begin{equation}
\ln (y) =  \ln (y_0 (1 + r)^t)  \\
\ln (y) =  \ln (y_0) + \ln (1 + r)^t \\
\ln (y) =  \ln (y_0) + t \ln (1 + r)   \\
\ln (y) - \ln (y_0) = t \ln (1 + r)   \\
\ln (\frac{y}{y_0}) = t \ln (1 + r)
\end{equation}

Take the $\exp$ of both sides.

\begin{equation}
\bigg (\frac{y}{y_0} \bigg) = \exp (t\ln (1 + r)) \\
\bigg (\frac{y}{y_0} \bigg ) = \exp (bt)
\end{equation}

where $b = \ln (1 + r)$. Finally, by taking $\exp$ of both sides, we get $r=\exp^{b}-1$.

#### Exercise 2. Prove that the doubling time is given by $t_{double}= \frac{\ln(2)}{\ln(1+r)}$

#### Solution:

\begin{equation}
\bigg (\frac{y}{y_0} \bigg) = \bigg ( \frac{2y_0}{y_0} \bigg) = 2 = \exp \bigg (t_{\texttt{double}}\ln (1 + r) \bigg ) \\
\ln (2) = (t_{\texttt{double}}\ln (1 + r)) \\
t_{\texttt{double}}= \frac{\ln(2)}{\ln(1+r)}
\end{equation}

#### Exercise 3. Show that $y = 2\ln(2) y_0 \exp(t/ t_{double})$ and that when $t_{double}>5$, the results deviates from linear trend by about $2\%$

#### Solution:

\begin{equation}
y = 2 \ln2 y_0 \exp(\frac{t}{t_{double}}) \\
y \sim  2 \ln2 y_0  +  \frac{t}{t_{double}} + \frac{1}{2}\frac{t}{t_{double}}^2 + ... \\
y = 2 \ln2 y_0 \exp(\frac{t}{T_2}) = \sim  2 \ln2 y_0 (1 +  \frac{t}{T_2} + \frac{1}{2}\frac{t}{T_2}^2 + ...)
\end{equation}

### 1.2 From Exponential to Logistics: the Case of Finite Capacity Growth

The exponential expression aabove can also be written in differential form as:

\begin{equation}
\frac{dy}{dt} = r y
\end{equation}

If we assume that the system has a finite capacity, we can improve the model by adding another variable $y_{max}$ such that when $y = y_{max}$, the rate of change of $y$ is zero ($\frac{dy}{dt}=0$). We thus reformulate the rate equation as:

\begin{equation}
\frac{dy}{dt} = r y \bigg ( \frac{y_{max}-y}{y_{max}} \bigg ) = r y \bigg (1 - \frac{y}{y_{max}} \bigg )
\end{equation}

Notice that if $y_{max} >> y$, we recover the previous exponential expression. The solution to this finite capacity spreading (limit of growth is $y_{max}$; $\frac{dy}{dt} =0$) is a logistic function that can be solved in a straightforward manner by separation of variables, and is given by:

\begin{equation}
y = \frac{y_{max}}{1 + A \exp(-rt)}
\end{equation}

where $A = \frac{y_{max}-y0}{y0}$.

Again, we note that when $y_{max} >> y_0$, we recover the simpler exponential form.

### Exercise

#### Exercise 3.  Compute the Uncertainty and Certainty of forecast for both the logistic and exponential fits

To compute the propagation of the projected first order uncertainty (deviation) for the exponential, we have:

\begin{equation}
\Delta y \approx dy = y_0 b \exp^{bt};
\end{equation}

while for the logistic, it is given by:

\begin{equation}
\Delta y \approx dy = \frac{A r y_{max}}{A + \exp^{rt}}
\end{equation}

**Derive this using pen and paper.**

### 1.3 Illustrations: Fitting with Actual Data

In the succeeding code, we show how the equations in Sections 1.1 and 1.2 can be implemented using real data. We first look at the case of those countries exhibiting exponential growth during the first 18 days of the epidemic (per country) when significant number of cases are reported (i.e., $>10$ confirmed cases). Then, we look at cases when the curves seem to have flattened.

#### For MSDS Students: Code for exponential fitting. See for reference Notebook 1 (Mathematics for Data Science).

```python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy.optimize import curve_fit
from sklearn.metrics import mean_squared_error, r2_score
from sympy import Symbol, symbols, integrate
from msdshelper import *

%matplotlib inline
```

Let's track the following countries at the onset: Philippines, China, South Korea, Iran, Italy, USA, UK, France.

```python
df = pd.read_excel('epi-data-pp.xlsx')
countries = ['Philippines', 'China', 'South Korea', 'Iran', 'USA', 'Italy', 'UK', 'France']
```

```python
df.head(n=3)
```

      Days
      Country
      Cases
      Deaths
      Cases_Cumulative
      Deaths_Cumulative

      0
      2
      USA
      20
      1
      20
      1

      1
      3
      USA
      14
      4
      34
      5

      2
      4
      USA
      22
      3
      56
      8

#### 1.3.1  Illustration 1: Consider USA as an example of exponentially growing cases
In the figure below, we plot the number of confirmed cases in the US in a log-linear plot. The broken line is the exponential fit to the empirical data (open circle). The trend, at the onset, suggests that the number of confirmed cases in the US doubled every 2.32 days, with $R^2 \sim 1$.

```python
y = df[df.Country == 'USA']['Cases_Cumulative']
growth_cases_exp_countries(df, ['USA'])
```

#### 1.3.2  Illustration 2: Consider USA as an example of exponentially growing confirmed cases and recorded deaths

Exponential curve and its corresponding fit for COVID19 reported cases (hollow circle) and deaths (solid circle and dashed lines). In both cases, we obtained $R^2$ values  $> 0.99$.

```python
y = df[df.Country == 'USA']['Cases_Cumulative']
ydeaths = df[df.Country == 'USA']['Deaths_Cumulative']
growth_cases_deaths(y, ydeaths, 'USA')
```

The normalization indicates that for every 62.72 cases reported, about 1 death is recorded. Notice how similar the curves are. The same trend is observed for all the other countries mentioned (except for China and South Korea) at the onset.

#### 1.3.3  Illustration 3: Consider USA, Italy, Iran, and the Philippines

Figure 2 demonstrates the exponential nature of the first weeks when a significant portion of the population
($>10$) is reported as COVID19 positive in different countries. In all instances, the observed fit has an $R^2>>0.97$ indicative of exponential growth of cases.

The growth of an exponential curve is generally non-intuitive. Without the help of quantitatively trained individuals, the idea of 10 infected individuals growing to 20,000 within 2 weeks is not easily digestable. We note here, however, that in cases when testing kits are limited, the exponential fit obtained during the early stage is more reliable and less skewed in terms of under detection since they represent a larger number of reported cases. That is, for an exponentially growing curve, we need also an exponentially growing testings or undetected cases will also grow exponentially.

A daily progressio for countries used above are shown in appendix A.

```python
growth_cases_exp_countries(df, ['USA', 'Italy', 'Iran', 'Philippines'])
```

#### 1.3.4  From Exponential to Logistics: the Cases of China and South Korea

Figure 2 looks at cases when the growth of COVID cases have been contained and shows us that the time to flatten he curve can vary greatly. While South Korea was able to do it in 1 month with only 5000 cases, China extends the flattening to 2 months with 80,000 casualties.

Comparing with Figure 1, we note that Iran and Italy has not shown visually that it will peak anytime soon and a quantitative method to visually see this is the main contribution of this article that will be discussed in the next section.

#### 1.3.4.1: China

```python
y = df[df.Country == 'China']['Cases_Cumulative']
ydeaths = df[df.Country == 'China']['Deaths_Cumulative']
growth_cases_deaths_logi(y, ydeaths, 'China', ymax = 100000, lagdays = 7, semilog = False)
```

### Figure 2. Caption

Logistic growth of Covid19 in china (2A) and South korea (2B). The corresponding rates for both countries are for China $r=0.226$ while for South Korea $r=0.322$. South korea peaks after two weeks and "flattened the curve" in one month while China was able to accomplish such in twice the time, 4 weeks to peak and 2 months to flattened.

#### 1.3.4.2: South Korea

```python
y = df[df.Country == 'South Korea']['Cases_Cumulative']
ydeaths = df[df.Country == 'South Korea']['Deaths_Cumulative']
growth_cases_deaths_logi(y, ydeaths, 'South Korea', ymax = 10000, lagdays = 5, semilog = False)
```

#### 1.3.5. Declining Numbers: The Cases of China and South Korea

Another way of interpreting the asymptotic decay trends in China and South Korea is by looking at the how fast the number of cases has been declining over time; and, for a logistic function, the tail end is also an exponential as shown below.

```python
def exponential(x,a,b):
    return a * np.exp(b*x)

def logistic(x,ymax,A,r):
    return ymax/(1+ A*np.exp(-r*(x)))
```

```python
## tail end of the curve
df_march = pd.read_excel('COVID_perday.xlsx')
df_march.head()
```

      2020-03-01 00:00:00
      Philippines
      USA
      Italy
      South Korea
      China
      France
      Iran
      UK
      Malaysia

      0
      1
      0
      3
      240
      813
      574
      43
      205
      5
      0

      1
      2
      0
      20
      561
      595
      205
      30
      385
      13
      4

      2
      3
      0
      14
      146
      686
      127
      48
      523
      4
      0

      3
      4
      0
      22
      667
      600
      119
      34
      835
      11
      7

      4
      5
      0
      34
      587
      516
      117
      73
      586
      34
      14

```python
SK_perday = df_march['South Korea']
```

```python
#South Korea
plt.figure(figsize=(8,5))
plt.rcParams.update({'font.size': 13})

y = SK_perday
x = np.arange(0, len(y))
popt, pcov = curve_fit(exponential, x, y, p0 = [0, 0])

x_fit = np.arange(0, len(y))
y_exp_fit = exponential(x_fit, *popt)

x_projection = np.arange(len(y), len(y)+14)
y_projection=exponential(x_projection, *popt)

rate=(np.exp(tuple(popt)[1]))-1
double= np.log(2)/np.log(1+ rate)
R_squared=r2_score(y_exp_fit, y[-len(y_exp_fit):])

plt.semilogy(x,y,'o', ms=10, markerfacecolor='none', color = tab10[3],label='');
plt.semilogy(x_projection,y_projection,'o--', ms=10, color = tab10[3],
         label='South Korea: Halved every %.2f days with $R^2$ = %.2f' %(-double, R_squared))
plt.semilogy(x_fit, y_exp_fit,'-', lw=2, color = tab10[3])

plt.legend(frameon=False, fontsize=10)
plt.xlim(0, 35)
plt.ylim(10, 1000)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.xlabel('March 2020', size=12);
plt.ylabel('Number of Infected People per day', size=12);
plt.savefig('SouthKorea.png', dpi=150)

```

### Figure 3A Caption
Decay curve rate of South Korea as it approaches the asymptotic ("flattening") phase.

```python

```

### Appendix: Additional Exercises

#### Home Exercise 1. Exponential at the onset of all countries.

As discussed above, all countries will have exponential growth at the onset. Shown below are the first 14 days trend on various countries when significant number of cases are confirmed (> 5 cases) and sustained (no zero reported cases for 5 successive days).

Task 1: Replicate the figure we have generated below. Impose the following constraint when selecting the data: (a) No zero reported cases for 5 consecutive days; (b) should have at least 5 cases (infected $> 5$) at the starting point.

### Home Exercise 2 (Expert level):  Exponential Trajectory Fitting Using 7-11 consecutive days

#### Why do we run an exponential fit and not a mere running average? (Journal Submission in Preparation)

We now describe here a simple procedure that will allow us to track the growth of COVID19 from various countries and ascertain if the curve is being flattened.  Running average is a practical tool for cases when no functional trend is imminent like in stock market or commodity pricing. Here, we know that the fit is given by an exponential or a logistic function. The gradient of both trends varies proportional to exponential for both. For exponential it is obvious, for logistic for example, the slope of a running exponential fit is best fitted by:

\begin{equation}
y = \frac{y_{max} r A \exp(-rt)}{1 + A \exp(-rt)^2}
\end{equation}

when $t$ is significantly larger than $r$, which is true for COVID19 case where $r$ of China and South Korea are less than 0.40, $(1 + A \exp(-rt))^2 \sim 1$, hence, the leading term varies exponentially too.

### Why 7-11 days, proof of simulation?

COVID19 is asymptomatic within 7-11 days and as expected the dynamics are enveloped during this temporal segment.

```python
df = pd.read_excel('COVID_cumulative.xlsx')
df.head()
```

      2020-03-01 00:00:00
      Philippines
      USA
      Italy
      South Korea
      China
      France
      Iran
      UK
      Malaysia

      0
      1
      0
      3
      240
      813
      574
      43
      205
      5
      0

      1
      2
      0
      23
      801
      1408
      779
      73
      590
      18
      4

      2
      3
      0
      37
      947
      2094
      906
      121
      1113
      22
      4

      3
      4
      0
      59
      1614
      2694
      1025
      155
      1948
      33
      11

      4
      5
      0
      93
      2201
      3210
      1142
      228
      2534
      67
      25

### CREATE/INSERT THE CODE AND FIGURE for FIGURE 4 here.

```python
running_exponential(df, ['Philippines', 'USA', 'Italy', 'South Korea', \
                         'China', 'France', 'Iran', 'UK', 'Malaysia'], xmax = 22)
```

### Figure 5. Caption  (RUN ALL COUNTRIES to improve the FIGURE HERE!)
Trend of doubling derived from the exponential fit using the procedure described. Notice that all countries are confined to 2-4 doubling days except for South korea, China and Iran. The uncertainty represented as $t_{double}*(1-R^2)$.

```python
running_exponential(df, ['Philippines'], ymax = 5, xmax=22)
```

```python

```

notebooks/Part I - Bridging the Gaps (Epidemic Modeling) .ipynb


# Deep Quantile Regression

One area that Deep Learning has not explored extensively is the uncertainty in estimates. However, as far as decision making goes, most people actually require quantiles as opposed to true uncertainty in an estimate. eg. For a given age the weight of an individual will vary. What would be interesting is the (for arguments sake) the 10th and 90th percentile. The uncertainty of _the estimate_ of an individuals weight is less interesting.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, Activation
import keras.backend as K

%matplotlib inline

mcycle = pd.read_csv('mcycle',delimiter='\t')
```

    Using TensorFlow backend.

Standardise the inputs and outputs so that it is easier to train. I have't saved the mean and standard deviations, but you should.

```python
mcycle.times = (mcycle.times - mcycle.times.mean())/mcycle.times.std()
mcycle.accel = (mcycle.accel - mcycle.accel.mean())/mcycle.accel.std()
```

```python
model = Sequential()
model.add(Dense(units=10, input_dim=1,activation='relu'))
model.add(Dense(units=10, input_dim=1,activation='relu'))
model.add(Dense(1))
model.compile(loss='mae', optimizer='adadelta')
model.fit(mcycle.times.values, mcycle.accel.values, epochs=2000, batch_size=32, verbose=0)
model.evaluate(mcycle.times.values, mcycle.accel.values)
```

    32/94 [=========>....................] - ETA: 0s

    0.29437355094767631

```python
t_test = np.linspace(mcycle.times.min(),mcycle.times.max(),200)
y_test = model.predict(t_test)

plt.scatter(mcycle.times,mcycle.accel)
plt.plot(t_test, y_test,'r')
plt.show()
```

## Quantiles 0.1, 0.5, 0.9

The loss for an individual data point is defined as:
$$
\begin{align}
\mathcal{L}(\xi_i|\alpha)=\begin{cases}
\alpha \xi_i &\text{if }\xi_i\ge 0, \\
(\alpha-1) \xi_i &\text{if }\xi_i< 0.
\end{cases}
\end{align}
$$
where $\alpha$ is the required quantile and $\xi_i = y_i - f(\mathbf{x}_i)$ and, $f(\mathbf{x}_i)$ is the predicted (quantile) model. The final overall loss is defines as:
$$\mathcal{L}(\mathbf{y},\mathbf{f}|\alpha)=\frac{1}{N} \sum_{i=1}^N \mathcal{L}(y_i-f(\mathbf{x}_i)|\alpha)$$

The following function defines the loss function for a quantile model.

**Note**: The following 4 lines is ALL that you change in comparison to a normal Deep Learning method, i.e. The loss function is all that changes.

```python
def tilted_loss(q,y,f):
    e = (y-f)
    return K.mean(K.maximum(q*e, (q-1)*e), axis=-1)
```

```python
def mcycleModel():
    model = Sequential()
    model.add(Dense(units=10, input_dim=1,activation='relu'))
    model.add(Dense(units=10, input_dim=1,activation='relu'))
    model.add(Dense(1))

    return model
```

```python
qs = [0.1, 0.5, 0.9]

t_test = np.linspace(mcycle.times.min(),mcycle.times.max(),200)
plt.scatter(mcycle.times,mcycle.accel)

for q in qs:
    model = mcycleModel()
    model.compile(loss=lambda y,f: tilted_loss(q,y,f), optimizer='adadelta')
    model.fit(mcycle.times.values, mcycle.accel.values, epochs=2000, batch_size=32, verbose=0)

    # Predict the quantile
    y_test = model.predict(t_test)
    plt.plot(t_test, y_test, label=q) # plot out this quantile

plt.legend()
plt.show()
```

## Final Notes

1. Note that the quantile 0.5 is the same as median, which you can attain by minimising Mean Absolute Error, which you can attain in Keras regardless with `loss='mae'`.
2. Uncertainty and quantiles are **not** the same thing. But most of the time you care about quantiles and not uncertainty.
3. If you really do want uncertainty with deep nets checkout http://mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html

```python

```


# `GiRaFFE_NRPy`: Basic Equations

## Authors: Patrick Nelson &

In this tutorial, we will introduce the basic GRFFE equations that we will wish to solve.

In GiRaFFE, we choose the vector potential $A_i$, the densitized Poynting flux $\tilde{S}_i$, and $\psi^6 \Phi$ as our conservative, evolved variables. They evolve according to the following equations:
\begin{align}
\partial_t \tilde{S}_i + \partial_j \left( \alpha \sqrt{\gamma} T^j_{{\rm EM} i} \right) &= \frac{1}{2} \alpha \sqrt{\gamma} T^{\mu \nu}_{\rm EM} \partial_i g_{\mu \nu} \\
\partial_t [\sqrt{\gamma} \Phi] &= -\partial_j (\alpha\sqrt{\gamma}A^j - \beta^j [\sqrt{\gamma} \Phi]) - \xi \alpha [\sqrt{\gamma} \Phi] \\
\partial_t A_i &= \epsilon_{ijk} v^j B^k - \partial_i (\alpha \Phi - \beta^j A_j) \\
\end{align}

We also use the Valencia three-velocity $\bar{v}^i$ and the magnetic field $B^i$ as primitives, which can be defined in terms of the conservative variables:
\begin{align}
B^i &= \epsilon^{ijk} \partial_j A_k \\
\bar{v}^i &= 4 \pi \frac{\gamma^{ij} {\tilde S}_j}{\sqrt{\gamma} B^2} \\
\end{align}

```python

```


## Fitted regularized censored regression models for Weibull or Exponential distributions

$$
\newcommand{\bbeta}{\boldsymbol{\beta}}
\newcommand{\bX}{\boldsymbol{X}}
\newcommand{\bZ}{\boldsymbol{Z}}
\newcommand{\bx}{\boldsymbol{x}}
\newcommand{\bz}{\boldsymbol{z}}
\newcommand{\by}{\boldsymbol{y}}
\newcommand{\bt}{\boldsymbol{t}}
\newcommand{\biota}{\boldsymbol{\iota}}
\newcommand{\bdelta}{\boldsymbol{\delta}}
$$

There are many situations when the observed data is censored, meaning that its value is only *partially* known. For example, income data which is published by national statistics agencies will truncate the ganularity of the data to some upper bound (greater than \$500K dollars for example in order to preserve the privacy of high-income households). In the biomedical domain in which I work in, certain biological measurements are not measureable if there are outside some detectable range. In both cases, the data we record is partially known because we know the measurement was *at least* this large, also known as right-censoring. The canonical case of right-censoring is survival times in which a patient has lived for *at least* this long. When a measurment is *at most* some value, this is known as left censoring.

In this post I will show how to build a regularized linear model from scratch in `python` to be able to fit right-censored data with an explicit likelihood model. The data challenge I have in mind is the one of missing value imputation for censored data; namely we want to predict the expected value of a censored measurement observation.

### Exponential distribution

Recall that the likelihood of a sample of $n$ i.i.d. exponential distributions can be written: $f(t_i;\lambda)=\lambda \exp(-\lambda t_i)$, $S(t_i;\lambda)=\exp(-\lambda t_i)$, and $h(t_i; \lambda) = \lambda)$.

$$
\begin{align*}
L(\bbeta) &=  \prod_{i=1}^n \hspace{1mm} f(\bbeta;t_i)^{\delta_i} S(\bbeta;t_i)^{1-\delta_i}  \\
&= \prod_{i=1}^n \hspace{1mm} h(\bbeta; t_i)^{\delta_i} S(\bbeta;t_i) \\
\end{align*}
$$

Hence the log-likelihood is

$$
\begin{align*}
\ell(\bbeta) &= \sum_{i=1}^n \big[ \delta_i \log h(\bbeta; t_i) + \log S(\bbeta;t_i) \big]
\end{align*}
$$

As we have parameterized $h(\bbeta; t_i) = \lambda_i = \exp(\bx^T \bbeta)$, where $\bx^T \bbeta = \sum_{j=0}^p \beta_jx_j$, and $S(\bbeta; t_i) = \exp(- \lambda_i t_i)= \exp(-\exp(\bx^T \bbeta) t_i)$, we can defined the log-likelihood as:

$$
\begin{align*}
\ell(\bbeta) &= \sum_{i=1}^n \big[ \delta_i (\bx_i^T \bbeta ) - \exp(\bx_i^T \bbeta ) t_i \big] \\
\frac{\partial \ell(\bbeta)}{\partial \beta_j} &= \sum_{i=1}^n \big[ \delta_i x_{ij} - z_i x_{ij} \big] = \bX_j^T(\bdelta - \bz) , \hspace{3mm} z_i = \exp(\bx_i^T \bbeta) t_i \\
\frac{\partial ^2 \ell(\bbeta)}{\partial \beta_j \partial \beta_k} &= - \sum_{i=1}^n x_{ij}x_{ik} z_i
\end{align*}
$$

While the partial derivative were derived for the likelihood, as we will use function minimizers, the vectorized gradient adn Hessian below are derived for the negative log-likelihood (simply a sign change):

$$
\begin{align*}
\nabla_\bbeta &= - \bX^T (\bdelta - \bz) \\
H_\bbeta &=  \bX^T \bZ \bX, \hspace{2mm} \text{diag}(\bZ)_i = [\exp(\bx_i^T \bbeta) t_i ]
\end{align*}
$$

### Iteratively re-weighted least squares

The Newton-step update at the $t+1$ iteration can be written as:

$$
\begin{align*}
\bbeta^{(t+1)} &\gets \bbeta^{(t)} - (H_{\bbeta^{(t)}})^{-1} \nabla_{\bbeta^{(t)}} \\
&\gets \bbeta^{(t)} + (\bX^T \bZ^{(t)} \bX )^{-1} \bX^T (\bdelta - \bz^{(t)}) \\
&\gets (\bX^T \bZ^{(t)} \bX )^{-1} \bX^T \bZ^{(t)} \by^{(t)}, \hspace{2mm} \by^{(t)} = \bX \bbeta^{(t)} + (\bZ^{(t)})^{-1} (\bdelta - \bz^{(k)})
\end{align*}
$$

The last line reveals that the Newton update for the Exponential likelihood model is equivalent to a weighted least-squares problem.

$$
\begin{align*}
\bbeta^{(t+1)} &= \arg\min_{\bbeta} \hspace{2mm} (\by^{(t)} - \bX \bbeta)^T  \bZ^{(t)} (\by^{(t)} - \bX \bbeta)
\end{align*}
$$

Hence if our original likelihood model has some elastic-net regularization of the form:

$$
\begin{align*}
\ell_{\gamma}(\bbeta) &= \ell(\bbeta) - \gamma \|\bbeta_{-0}\|_1 ,
\end{align*}
$$

Where the L1-norm penalty is put on the non-intercept coefficient, then the Newton update will be equavilent to solving a least-squares Lasso problem at each iteration.

$$
\begin{align}
\bbeta^{(t+1)} &= \arg\min_{\bbeta} \hspace{2mm} (\by^{(t)} - \bX \bbeta)^T  \bZ^{(t)} (\by^{(t)} - \bX \bbeta) + \gamma \|\bbeta_{0}\|_1  \label{eq:irls}
\end{align}
$$

The `sklearn` package has very fast solvers for the type of optimization problem seen in $\eqref{eq:irls}$.

### Weibull model

In some cases the exponential distribution not be flexible enough to handle different types of censoring processes. For example the exponential distribution has a hazard rate which is constant across time (or whatever unit $t_i$ stands for). The Weibull distributions allows for a measurement-level dependent hazard with an additional shape parameter ($\alpha$) in addition to the rate parameter ($\lambda$). The case a simple rate/shape Weibull distribution has the following density, inverse CDF, and hazard function: $f(t_i;\lambda,\alpha) = \alpha\lambda t_i^{\alpha-1}\exp(-\lambda t_i^\alpha)$, $S(t_i;\lambda,\alpha)=\exp(-\lambda t_i^\alpha)$, and $h(t_i;\lambda, \alpha) = \alpha \lambda t_i^{\alpha-1}$. When $\alpha=1$, the Weibull distribution reduces to the Exponential making the latter a nested distribution.

In the case of $n$ data points with a $p$-dimensional covariate vector we will again parameterize the scale parameter as $\lambda_i = \exp(\bx_i^T \bbeta)$, and leave the shape parameter as a global term. Because $\alpha \in (0,\infty)$, it will be easier to optimize over $\alpha(\phi)=\exp(\phi)$ so that $\phi \in (-\infty,\infty)$. The log-likelihood funtion then becomes:

$$
\begin{align*}
\ell(\bbeta, \phi) &= \sum_{i=1}^n \big\{ \delta_i\big(\phi + (\bx_i^T\bbeta)+[\exp(\phi)-1]\cdot \log t_i \big) - \exp(\bx_i^T\bbeta)t_i^{\exp(\phi)} \big\} \\
\end{align*}
$$

The partial derivatives for both components can be calculated.

$$
\begin{align*}
\frac{\partial \ell(\phi;\bbeta)}{\partial \phi} &= \sum_{i=1}^n \delta_i + e^\phi \sum_{i=1}^n \log t_i \Big[1 - \exp(\bx_i^T \bbeta) t_i^{\exp(\phi)} \Big] \\
\frac{\partial \ell(\bbeta; \phi)}{\partial \bbeta} &= \bX^T( \bdelta - \bz_\phi), \hspace{2mm} z_{i,\phi} = \exp(\bx_i^T\bbeta) t_i^{\exp(\phi)} \\
\frac{\partial^2 \ell(\bbeta; \phi)}{\partial \bbeta \partial \bbeta^T } &= - \bX^T \bZ_\phi \bX
\end{align*}
$$

In order to optimize this likelihood, a block-coordinate descent will prove useful where the parameter $\phi$ is updated first through a root finding method, and then $\bbeta$ is solved for with the IRLS approach highlighted above.

$$
\begin{align*}
\phi^{(t+1)} &\gets \arg \min \sum_{i=1}^n \delta_i + e^\phi \sum_{i=1}^n \log t_i \Big[1 - \exp(\bx_i^T \bbeta) t_i^{\exp(\phi)} \Big] \\
\bbeta^{(t+1)} &\gets (\by^{(t)} - \bX \bbeta)^T  \bZ^{(t)}_\phi (\by^{(t)} - \bX \bbeta) + \gamma \|\bbeta_{-0}\|_1, \hspace{2mm} \by^{(t)} = \bX \bbeta^{(t)} + (\bZ^{(t)}_\phi)^{-1} (\bdelta - \bz^{(k)}_\phi)
\end{align*}
$$

```python

```

```python

```


# Spike and Slab GP-LVM for Factor Analysis

## Model Definition

GP-LVM (Lawrence 2005) with a linear kernel:
\begin{equation}
p(Y|X) = \prod_{d=1}^D p(y_{:,d}| X), \quad p(y_{:,d}|X) = \mathcal{N}(y_{:,d}; 0, XX^T+\beta^{-1}I)
\end{equation}
It is related to probabilistic PCA, while giving a prior distribution over the weight matrix $W$ instead of $X$ and marginalizing $W$:
\begin{equation}
p(Y|X) = \prod_{d=1}^D \mathcal{N}(y_{:,d};Xw_d, \beta^{-1}I), \quad
p(W) = \prod_{d=1}^D \mathcal{N}(w_d; 0, I).
\end{equation}

GP-LVM has been extended by giving a prior over $X$, known as Bayesian GP-LVM (Titsias & Lawrence 2010).
\begin{equation}
p(X) = \prod_{n=1}^N \mathcal{N}(x_n; 0, I)
\end{equation}

For factor analysis, it is often useful to assume $X$ to be sparse. Therefore, we extend Bayesian GP-LVM by assuming a Spike and Slab prior over $X$. Then, the GP model definition becomes:
\begin{align}
p(X)=\prod_{n=1}^N \mathcal{N}(x_n; 0, I), &\quad p(B) = \prod_{n=1}^N \prod_{q=1}^Q \pi^{b_{nq}} (1-\pi)^{1-b_{nq}},\\
p(Y|X) = \prod_{d=1}^D p(y_{:,d}| X), &\quad p(y_{:,d}|X) = \mathcal{N}(y_{:,d}; 0, K+\beta^{-1}I),
\end{align}
where $K$ is calculated according to the corresponding linear kernel function:
\begin{equation}
k(x,b,x',b') = \sum_{q=1}^Q \sigma^2_q x_q b_q x'_q b'_q
\end{equation}

## An Experiment with Synthetic Data

We generate some artifical data and mixing them with a weight matrix.

```python
import numpy as np
import GPy
%matplotlib inline
#%config InlineBackend.figure_format = 'svg'
from pylab import *
np.random.seed(113321)
```
We generated samples from 3 sine wave functions with random frequencies and offsets, and randomly set 80% of the samples to zeros (see the following figure).

```python
# Prepare the data
N,D,Q = 500, 100, 3
pi = 0.2
# sample from 3 random sine waves
X = np.sin(2*np.pi*(np.random.rand(Q)[None,:]+.5)*(np.linspace(0.,3.,N)[:,None]-np.random.rand(Q)[None,:]))
# set 80% of the values to zeros
X = X*((np.random.rand(N,Q)<pi)*1)
# plot the samples
_ = plot(X)
```

The obersved data $Y$ is generated by projecting the samples of the 3 sine waves onto a 100D space with a randomly generated weight matrix $W$. The first two dimensions of $Y$ is shown in the figure.

```python
noise_sigma = 0.1
# Generate a weight matrix
W = np.random.randn(D,Q)
# Project to the observed space plus some Gaussian noise
y = np.dot(X,W.T)+np.random.randn(N,D)*noise_sigma
y = (y-y.mean())/y.std()
# Plot the
_ = plot(y[:,:2])
```

## Applying Spike and Slab GP-LVM

We create a Spike and Slab GPLVM model object with a linear kernel:

```python
m = GPy.models.SSGPLVM(y, Q, kernel=GPy.kern.Linear(Q,ARD=True),num_inducing=Q)
# Show the model parameters
m.likelihood.variance = .01
m.X.variance = .1
print m
```

    Name : Spike_and_Slab GPLVM
    Objective : 1336353.57033
    Number of Parameters : 4516
    Number of Optimization Parameters : 4513
    Updates : True
    Parameters:
      Spike_and_Slab_GPLVM.     |     value  |     constraints      |  priors
      latent space.mean         |  (500, 3)  |                      |
      latent space.variance     |  (500, 3)  |         +ve          |
      latent space.binary_prob  |  (500, 3)  |  1e-10,0.9999999999  |
      inducing inputs           |    (3, 3)  |                      |
      linear.variances          |      (3,)  |         +ve          |
      Gaussian_noise.variance   |      0.01  |         +ve          |
      SpikeAndSlabPrior.Pi      |      (3,)  |        fixed         |

```python
# Optimize the model parameters
m.optimize('bfgs',messages=1,max_iters=10000)
```

```python
# Show the model parameters after optimization
print m
print m.variational_prior.pi
```

    Name : Spike_and_Slab GPLVM
    Objective : -6593.56992735
    Number of Parameters : 4516
    Number of Optimization Parameters : 4513
    Updates : True
    Parameters:
      Spike_and_Slab_GPLVM.     |            value  |     constraints      |  priors
      latent space.mean         |         (500, 3)  |                      |
      latent space.variance     |         (500, 3)  |         +ve          |
      latent space.binary_prob  |         (500, 3)  |  1e-10,0.9999999999  |
      inducing inputs           |           (3, 3)  |                      |
      linear.variances          |             (3,)  |         +ve          |
      Gaussian_noise.variance   |  0.0392378671791  |         +ve          |
      SpikeAndSlabPrior.Pi      |             (3,)  |        fixed         |
      index  |  Spike_and_Slab_GPLVM.SpikeAndSlabPrior.Pi  |  constraints  |  priors
      [0]    |                                 0.50000000  |     fixed     |
      [1]    |                                 0.50000000  |     fixed     |
      [2]    |                                 0.50000000  |     fixed     |

```python
# Plot the posterior distribution of X
# The left figure shows the mean and variance (the thickness) of
# the slab part (the gray color shows the other dimensions for comparison)
# The right figure shows the posterior probability of the binary variable for having the slab
a = m.X.plot()
```

```python
# In order to compare the recovered X, we flipped the sign of the 2nd, 3rd dimensions, and rescale the values.
# The figure shows the absolute difference between the recovered X and the original X
Xcom = m.X.mean.copy()*np.array([1.,-1.,-1.])
Xcom = Xcom/np.abs(Xcom).max(axis=0)
_ = plot(np.abs(Xcom-X))
```

```python
# Plot the distribution of the recovered X (only 1st and 3rd dimension)
# As shown in the figure, it follows a spike and slab distribution
_ = m.plot_latent()
```

## Apply Bayesian GP-LVM

As a comparison, we also apply the synthetic data to the original Bayesian GPLVM model.

```python
m = GPy.models.BayesianGPLVM(y, Q, kernel=GPy.kern.Linear(Q,ARD=True),num_inducing=Q)
# Show the model parameters
print m
```

    Name : bayesian gplvm
    Objective : 51624.7110985
    Number of Parameters : 3013
    Number of Optimization Parameters : 3013
    Updates : True
    Parameters:
      bayesian_gplvm.          |     value  |  constraints  |  priors
      latent space.mean        |  (500, 3)  |               |
      latent space.variance    |  (500, 3)  |      +ve      |
      inducing inputs          |    (3, 3)  |               |
      linear.variances         |      (3,)  |      +ve      |
      Gaussian_noise.variance  |       1.0  |      +ve      |

```python
# Optimize the model parameters
m.optimize('bfgs',messages=1,max_iters=10000)
```

```python
# Show the model parameters after optimization
print m
```

    Name : bayesian gplvm
    Objective : -3717.46060167
    Number of Parameters : 3013
    Number of Optimization Parameters : 3013
    Updates : True
    Parameters:
      bayesian_gplvm.          |            value  |  constraints  |  priors
      latent space.mean        |         (500, 3)  |               |
      latent space.variance    |         (500, 3)  |      +ve      |
      inducing inputs          |           (3, 3)  |               |
      linear.variances         |             (3,)  |      +ve      |
      Gaussian_noise.variance  |  0.0392832125766  |      +ve      |

```python
# Plot the posterior distribution of X
a = m.X.plot()
```

```python
# In order to compare the recovered X, we swapped the 2nd and 3rd dimension,
# flipped the sign of the 2nd dimension, and rescale the values.
# The figure shows the absolute difference between the recovered X and the original X
Xcom = np.dot(m.X.mean.copy(),np.array([[1.,0.,0.],[0.,0.,1.],[0.,-1.,0.]]))
Xcom = Xcom/np.abs(Xcom).max(axis=0)
_ = plot(np.abs(Xcom-X))
```

```python
# Plot the distribution of the recovered X (only 1st and 2nd dimension)
# As shown in the figure, it follows a spike and slab distribution
_ = m.plot_latent()
```

```python

```


### Prof. Dr. -Ing. Gerald Schuller
 Jupyter Notebook: Renato Profeta

# Quantization: Signal to Noise Ratio (SNR)

```python
%%html

```

Assume we have a A/D converter with a quantizer with a certain number of bits (say N bits), what is the resulting Signal to Noise Ratio (SNR) of this quantizer?

**The SNR is defined as the ratio of the expectation of the signal power to the expectation of the noise power.**

In our case, the expectation of the noise power is the expectation of the quantization error power. We already have the expectation of the quantization error power as $\large \dfrac{\Delta^2}{12}$.

So what we still need for the SNR is the **average or expectation of the signal power**. How do we obtain this?

Basically we can take the same approach as we did for the expectation of the power of the quantization error (which is basically the second moment of the distribution of the quantization error). So what we need to know from our signal is its **probability distribution**. For the quantization error it was a uniform distribution between $-\dfrac{\Delta}{2}$ and $+\dfrac{\Delta}{2}$.

A very **simple case** would be a **uniformly distributed signal** with amplitude $\dfrac{A}{2}$, which has values between $-\dfrac{A}{2}$ up to $+\dfrac{A}{2}$.

So we could again use our formula for the average power, but now for our signal x:

$$\large E(x^2)=\int_ {-A/2} ^ {A/2} x^2 \cdot p(x) dx$$

So here we have the same type of signal, and the resulting expectation of the power (its second moment, assumed we have a zero mean signal) is obtained by using our previous formula, and replace $\Delta$ by A. The resulting power is: $\frac{A^2}{12}$.

**Which signals have this property?** One example is uniformly distributed random values (basically like our quantization error).

**Observe: Speech or music has a non-uniform pdf**, it is usually modeled by a Laplacian distribution or a gaussian mixture model, so it doesn't apply to this case!

```python
%%html

```

An example for a uniform pdf: **a triangular wave**:

How do we obtain its pdf? One can imagine the vertical axis (the function value) covered by small intervals, and each interval is then passed in the same time-span. This means that the resulting pdf is also uniform!

A further example: **A sawtooth wave**:

Again we can make the same argument, each small interval of our function value is covered in the same time-span, hence we obtain a uniform distribution.

We now have seen a few examples which fulfil our assumption of a uniform distribution (realistic examples), and we know: their expectation of their power is $\dfrac{A^2}{12}$. So what does this then mean for the SNR? The **SNR** is just the ratio:

$$ \large SNR = \frac {\dfrac{A^2}{12}} {\dfrac{\Delta^2}{12}}= \frac{A^2} { \Delta^2} $$

If we assume our signal is full range, meaning the maximum values of our A/D converter is $-\dfrac{A}{2}$ and $+\dfrac{A}{2}$ (the signal goes to the maximum), we can compute the step size $\Delta$ if we know the **number of bits** of converter, and if we assume uniform quantization step sizes. Assume we have **N bits** in our converter. This means we have $2^N$ quantization intervals. We obtain $\Delta$ by dividing the full range by this number,

$$ \large
\Delta = \frac{A}{2^N}
$$

Plug this in the SNR equation, and we obtain:

$$
SNR=  \frac{A^2} { \Delta^2}= \frac{A^2} {\left( \dfrac {A}{2^N} \right)^2} = {2^{2N}}
$$

This is now quite a simple result! But usually, the SNR is given in dB (deciBel), so lets convert it into dB:

$$SNR_{dB} = 10 \cdot \log_{10} (2^{2N})=10 \cdot 2N \cdot \log_{10}(2) \approx $$
$$ \approx 10 \cdot 2N \cdot 0.301 dB =N \cdot 6.02 dB$$

This is now our famous **rule of thumb**, that **each bit** more gives you about **6 dB more SNR**. But observe that the above formula only holds for uniformly distributed full range signals! (the signal is between -A/2 and +A/2, using all possible values of our converter).

```python
%%html

```

What happens if the signal is not full range? What is the SNR if we have a signal with reduced range? Assume our signal has an amplitude of A/c, with a factor c>1.

We can then simply plug this into our equation:

$$ \large SNR=  \frac{\left(\frac{A}{c}\right)^2} {\Delta^2}= \frac{\left(\frac{A}{c}\right)^2 }{ \left(\frac{A}{2^N}\right)^2} = \frac{2^{2N}}{c^2}$$

in dB:

$$ \large SNR_{dB}=10 . \log_{10}( \frac {2^{2N}}  {c^2})=10\cdot2N.\log_{10}(2)-20.\log_{10}(c) \approx $$

$$ \large
 \approx 10 \cdot 2N \cdot 0.301 dB -20 \cdot \log_{10}(c) =
$$

$$ \large
 =N \cdot 6.02 dB -20 \cdot log_{10}(c)
$$

The last term, the $20 \cdot \log_{10}(c) $, is the number of dB which we are below our full range. This means we **reduce our SNR** by this number of **dB** which we are **below full range**!

**Example:** We have a 16 bit quantiser, then the SNR for uniformly distributed full range signals would be:

$$SNR = 6.02 \cdot 16 dB = 96.32 dB$$

Now assume we have the same signal, but 20dB below full range (meaning only 1/10th of the full range). Then the resulting SNR would be only:

$$SNR = 96.32-20 = 76.32 dB$$

This is considerably less. This also shows why it is important not to make the safety margin to full range too big! So for instance our sound engineer should keep the signal as big as possible, without ever reaching full range to avoid clipping the signal.

The other assumption we made concerned the type of signal we quantize.

**What if we don't have a uniformly distributed signal?**

As we saw, speech and audio signals are best modelled by a Laplacian distribution or a Gaussian mixture model, and similar for audio signals. Even a simple sine wave does not fulfil this assumption of a uniform distribution. What is the pdf of a simple sine wave?

**Observe:** If a sinusoid represents a full range signal, its values are from $-A/2$ to $+A/2$, as in the previous cases.

```python
%%html

```

What is our SNR if we have a sinusoidal signal? What is its pdf? Basically it is its normalized histogram, such that its integral becomes 1, to obtain a probability distribution.

If we look at the signal, and try to see how probable it is for the signal to be in a certain small interval on the y axis, we see that the signal stays longest around +1 and -1, because there the signal slowly turns around. Hence we would expect a pdf, which has peaks at +1 and -1.

If you calculate the pdf of a sine wave, x=sin(t), with t being continuous and with a range larger than 2pi, then the result is

$$
p(x)=\frac{1}  {\pi \cdot \sqrt{1-x^2}}
$$

This results from the derivative of the inverse sine function (arcsin). This derivation can be found for instance on Wikipedia. For our pdf we need to know how fast a signal x passes through a given bin in x. This is what we obtain if we compute the inverse function $x=f^{-1}(y)$, and then its derivative $df^{-1}(x)/dy$.

### PDF of Time Series

Given a signal a = f (t) which is sampled uniformly over a time period T , its PDF, p(a) can be calculated as follows. Because the signal is uniformly sampled we have $p(t) = \frac{1}{T}$ . The function f(t) acts to transform this density from one over *t* to one over *a*. Hence, using the method for transforming PDFs, we get:

$$\large
p(a)=\dfrac{p(t)}{\left|\frac {da}{dt} \right|} $$

where | | denotes the absolute value and the derivative is evaluated at $t=f^{-1}(x).$

From: https://www.fil.ion.ucl.ac.uk/~wpenny/course/appendixDE.pdf

```python
from sympy import symbols, pi, sqrt, Integral, Function, Eq, diff, sin, solve, simplify, Abs
x, t = symbols('x t', real=True)
A, w = symbols('A w', real=True, positive=True)
Eq_x=Eq(x, sin(t))
Eq_x
```

$\displaystyle x = \sin{\left(t \right)}$

```python
# Find the Inverse
y=solve(Eq_x,t)
Eq_y=Eq(t,y[1])
Eq_y
```

$\displaystyle t = \operatorname{asin}{\left(x \right)}$

The inverse sine is only defined for $-\frac{\pi}{2} \leq  t \leq +\frac{\pi}{2}$ and p(t) is uniform within this.
Hence, $ p(t) = \frac {1}{\pi} $.

```python
# Find dx\dt and evaluate at t=arsin(x)

dxdt = (diff(Eq_x.rhs,t))
dxdt = dxdt.subs(t,Eq_y.rhs)
dxdt
```

$\displaystyle \sqrt{1 - x^{2}}$

```python
# Calculate p(t)
from sympy.stats import Uniform, density
P_t = Function('p')(t)
p_t = Uniform('X',-pi/2,pi/2)
Eq_p_t=Eq(P_t,density(p_t)(t))
Eq_p_t
```

$\displaystyle p{\left(t \right)} = \begin{cases} \frac{1}{\pi} & \text{for}\: t \geq - \frac{\pi}{2} \wedge t \leq \frac{\pi}{2} \\0 & \text{otherwise} \end{cases}$

```python
# Calculate p(x)
p_x = Function('p')(x)
Eq_p_x= Eq(p_x,(1/pi)/dxdt)
Eq_p_x
```

$\displaystyle p{\left(x \right)} = \frac{1}{\pi \sqrt{1 - x^{2}}}$

```python
%%html

```

Here we can see that p(x) indeed becomes infinite at x=+/-1! We could now use the same approach as before to obtain the expectation of the power, multiplying it with $x^2$ and integrating it. But this seems to be somewhat tedious.  But since we now have a deterministic signal, we can also try an **alternative** solution, since the sine function is not a probabilistic function, but a deterministic function.

We can simply directly compute the power of our sine signal over t, and then take the average over at least one period of the sine function.

$$ \large
E(x^2)= \frac{1} {2\pi}\int _ {t=0} ^{2\pi} sin^2(t) dt = \frac{1}  {2\pi}\int _ {t=0} ^{2\pi} {\dfrac{\left(1-cos(2t)\right)}{2}}dt
$$

Trigonometric Identity: $cos(2x)=1-2sin^2(x)$

The cosine integrated over complete periods  becomes 0, hence we get:

$$ n\large
=\frac{1}  {2\pi} \int _{t=0}^{2\pi} {\dfrac{1}{2}} dt =\frac{1} {2\pi} \cdot \pi=\frac{1}{ 2}
$$

```python
# Calculate the Expctation of Power
E = Function('E')(x**2)
E_x2 = Eq(E,Integral(x**2*Eq_p_x.rhs,(x,-1,1)))
display(E_x2)
display(E_x2.doit())
```

$\displaystyle E{\left(x^{2} \right)} = \int\limits_{-1}^{1} \frac{x^{2}}{\pi \sqrt{1 - x^{2}}}\, dx$

$\displaystyle E{\left(x^{2} \right)} = \frac{1}{2}$

What do we get for a sinusoid with a different amplitude, say $A/2 \cdot sin(t)$?

The expected power is:

$$ \large
E(x^2)=\frac{A^2}{ 8}
$$

```python
# Calculate Expectation of Power of A/2 * sin(t)
E = Function('E')(x**2)
E_x2 = Eq(E,(1/(2*pi))*Integral(((A/2)*sin(t))**2,(t,0,2*pi)))
display(E_x2)
display(E_x2.doit())
```

$\displaystyle E{\left(x^{2} \right)} = \frac{\int\limits_{0}^{2 \pi} \frac{A^{2} \sin^{2}{\left(t \right)}}{4}\, dt}{2 \pi}$

$\displaystyle E{\left(x^{2} \right)} = \frac{A^{2}}{8}$

So this leads to an SNR of:

$$ \large
SNR=\frac{\frac{A^2}{8}} {\frac{\Delta^2}{12}}=\frac{3 \cdot A^2} {2 \cdot \Delta^2}
$$

Now assume again we have a A/D converter with N bits, and the sinusoid is at full range for this converter. Then
$A=2^N \cdot \Delta$

We can plug in this result into the above equation, and get

$$ \large
SNR=\frac{3 \cdot 2^{2N} \cdot \Delta^2}  {2 \cdot \Delta^2}={1.5 \cdot 2^{2N}} $$

In dB this will now be:

$$\large 10 \cdot \log_{10}(SNR)=10 \cdot \log_{10}(1.5) + N \cdot 20 \cdot \log_{10}(2)=
$$

$$\large = 1.76 dB +N \cdot 6.02 dB$$

Here we can see now, that using a sinusoidal signal instead of a uniformly distributed signal gives us a **boost of 1.76 dB** in SNR. This is because it is more likely to have larger values!
We see that our rule of 6dB more SNR for each bit still holds!

```python
# Stepsize as function of full range
delta, N = symbols('\Delta N')
Eq_delta = Eq(delta, A/(2**N) )
display(Eq_delta)
Eq_A = Eq(A,solve(Eq_delta,A)[0])
display(Eq_A)
```

$\displaystyle \Delta = 2^{- N} A$

$\displaystyle A = 2^{N} \Delta$

```python
# Calculate Signal to Noise Rate
SNR = E_x2.doit().rhs / (delta**2/12)
display(SNR)
display(SNR.subs(A,Eq_A.rhs))
```

$\displaystyle \frac{3 A^{2}}{2 \Delta^{2}}$

$\displaystyle \frac{3 \cdot 2^{2 N}}{2}$


# Equilibre énergétique de l'échangeur de séparation de réseau

On note :

- $eff$ l'efficacité de l'échangeur de séparation de réseau (on prendra une valeur de 0.8)

- $m_{dro}$ le débit massique du fluide dans le dromotherme en $\frac{kg}{s}$

- $Cp_{dro}$ la capacité thermique du fluide circulant dans le dromotherme en  $\frac{J}{kg K}$

- $m_{sto}$ le débit massique du fluide dans le système géothermique du stockage en $\frac{kg}{s}$

- $Cp_{sto}$ la capacité thermique du fluide circulant dans le système géothermique du stockage en $\frac{J}{kg K}$

On pose $coeff = \frac{m_{dro} * Cp_{dro}}{m_{sto} * Cp_{sto}}$

**On ne va s'intéresser qu'à un seul cas de figure : $coeff <= 1$**

On a

\begin{equation}
T_{inj\_dro} = T_{sor\_dro} - eff * (T_{sor\_dro} - T_{sor\_sto})
\end{equation}

Et :

\begin{equation}
T_{inj\_sto} = T_{sor\_sto} + coeff * (T_{sor\_dro} - T_{inj\_dro})
\end{equation}

On peut donc écrire :

\begin{equation}
T_{inj\_sto} = T_{sor\_sto} + coeff * (T_{sor\_dro} - T_{sor\_dro} + eff * (T_{sor\_dro} - T_{sor\_sto}))
\end{equation}

Ce qui laisse :

\begin{equation}
T_{inj\_sto} = T_{sor\_sto} + coeff * eff * (T_{sor\_dro} - T_{sor\_sto})
\end{equation}

# Echanges énergétiques au niveau du système géothermique équipant le système de stockage

Si k est le coefficient du système énergétique exprimé en W/K, et en notant y la température dans le massif de stockage, on a :

\begin{equation}
m_{sto} * Cp_{sto} * (T_{inj\_sto} - T_{sor\_sto}) = k * (\frac{T_{inj\_sto} + T_{sor\_sto}}{2} - y )
\end{equation}

Soit :

\begin{equation}
T_{inj\_sto}  (m_{sto} * Cp_{sto} - \frac{k}{2})  =  (m_{sto} * Cp_{sto} + \frac{k}{2})  T_{sor\_sto} - k y
\end{equation}

Ou encore :

\begin{equation}
T_{inj\_sto}  =  \frac{ (m_{sto} * Cp_{sto} + \frac{k}{2})  T_{sor\_sto} - k y }{m_{sto} * Cp_{sto} - \frac{k}{2}}
\end{equation}

# Bilan dromotherme/stockage

On obtient 2 expressions de $T_{inj\_sto}$, permettant ainsi de formuler $T_{sor\_sto}$ fonction de y et de $T_{sor\_dro}$ seulement :

\begin{equation}
T_{sor\_sto} + coeff * eff * (T_{sor\_dro} - T_{sor\_sto}) = \frac{(m_{sto} * Cp_{sto} + \frac{k}{2} )  T_{sor\_sto} - k y} {m_{sto} * Cp_{sto} - \frac{k}{2}}
\end{equation}

En posant

\begin{equation}
B = coeff * eff  * (m_{sto} * Cp_{sto} - \frac{k}{2})
\end{equation}

On obtient :

\begin{equation}
(m_{sto} * Cp_{sto} - \frac{k}{2})*T_{sor\_sto} + B * (T_{sor\_dro} - T_{sor\_sto}) =  (m_{sto} * Cp_{sto} + \frac{k}{2})  T_{sor\_sto} - k  y
\end{equation}

Soit :

\begin{equation}
(m_{sto} * Cp_{sto} - \frac{k}{2} - B - m_{sto} * Cp_{sto} - \frac{k}{2}) *  T_{sor\_sto} = - B * T_{sor\_dro} - k y
\end{equation}

Ou encore :

\begin{equation}
( k + B ) *  T_{sor\_sto} = ky + B * T_{sor\_dro}
\end{equation}

Il reste :

\begin{equation}
T_{sor\_sto} = \frac{ky + B * T_{sor\_dro}}{k+B}
\end{equation}

# Mise en oeuvre de la PAC

On note :

- $P_{geo}$ la puissance géothermique à extraire du stockage en  W

- $besoin$ le besoin énergétique du bâtiment à satisfaire en W

- $P_{elec}$ la puissance électrique de la PAC en W

- COP  le coefficient de performance de la PAC

- $m_{pac}$ le débit dans la PAC en $\frac{kg}{s}$

- $Cp_{pac}$ la capacité thermique du fluide dans la PAC en $\frac{J}{kg.K}$

On a : $besoin = P_{elec} + P_{geo}$

Et
\begin{equation}
COP=\frac{besoin}{P_{elec}}
\end{equation}

De ces deux expressions, on obtient la relation entre la puissance géothermique et le besoin du bâtiment:

\begin{equation}
P_{geo}=\frac{COP-1}{COP} * besoin
\end{equation}

La puissance géothermique s'exprime aussi : $P_{geo} = m_{pac} Cp_{pac} ( T_{inj\_pac}-T_{sor\_pac} )$

On en déduit la température de sortie du fluide dans la PAC:

\begin{equation}
T_{sor\_pac} = T_{inj\_pac} - \frac{P_{geo}}{m_{pac} * Cp_{pac}}
\end{equation}

En écrivant l'équilibre énergétique entre la PAC et le stockage, comme on a écrit celui entre le dromotherm et stockage, on a :

\begin{equation}
T_{sor\_pac}  =  \frac{ (m_{pac} * Cp_{pac} + \frac{k}{2})  T_{inj\_pac} - k y }{m_{pac} * Cp_{pac} - \frac{k}{2}}
\end{equation}

Les deux expressions de $T_{sor\_pac}$ nous permettent d'obtenir $T_{inj\_pac}$ :

\begin{equation}
T_{inj\_pac}  = y-\frac{C Pgeo}{k}
\end{equation}

avec

\begin{equation}
C  = 1-\frac{k}{2 m_{pac} Cp_{pac}}
\end{equation}

# Besoin du bâtiment

## cas du chauffage

Le bâtiment est assimilé à un modèle R3C.
On note :

- $Q_{chauffage}$ le besoin instantanné de chauffage en W

- $T_{consigne}$ : température de consigne en °C

- $T_{int}$ : température intérieure en °C

- $T_{ext}$ : température extérieure en °C

- $R_m$ : Résistance thermique des murs (K/W)

- $R_i$ : Résistance superficielle intérieure (K/W)

- $R_f$ : résistance de fuite (infiltrations+vitre+renouvellement d'air) K/W

Par analogie électrique, on assimile les températures à des tensions et les puissances à des intensités
en première approximation, on a donc :

\begin{equation}
    \frac {T_{int}-T_{ext}}{R_m+R_i}+ \frac {T_{int} -T_{ext}}{R_f} + C \frac{dT_{int}}{dt}=Q_{chauffage}
\end{equation}

Soit :

\begin{equation}
    C \frac{dT_{int}}{dt} = Q_{chauffage} - \frac {T_{int}-T_{ext}}{R_m+R_i}-\frac {T_{int} -T_{ext}}{R_f}
\end{equation}

Pour maintenir $T_{int}$ constante et égale à $T_{consigne}$, on doit donc développer :

\begin{equation}
   Q_{chauffage} = (T_{consigne}-T_{ext})*(\frac {1}{Rm+Ri}+ \frac {1}{Rf} )
\end{equation}

## cas de l'eau chaude sanitaire ECS

Notons :

 - $Q_{ECS}$ le besoin instantanné en ECS exprimé en W

 - $T_{eau}$ la température de l'eau du réseau, que l'on choisit de modéliser comme une fonction sinusoidale de période annuelle

 - $V$ la consommation d'eau moyenne journalière exprimée en litre, pour une personne seule

 - $N_{pers}$ le nombre de personnes occupant le bâtiment

 - $Cp_{eau}$ la capacité calorifique de l'eau exprimée en $\frac{J}{kg K}$

 - $T_{ballon}$, la température de l'eau chaude stockée dans le ballon (60°C)

\begin{equation}
   Q_{ECS}= \frac{V*N_{pers}*(T_{ballon}-T_{eau})*Cp_{eau}}{24*3600}
\end{equation}

## bilan sur le bâtiment

On a :

\begin{equation}
   besoin  = Q_{ECS} + Q_{chauffage}
\end{equation}

# Température de sortie de dromotherm

\begin{equation}
C_s h_s\frac{\partial T_s}{\partial t}=(1-\alpha)R_g(t)+ R_{atm}(t) -\epsilon\sigma T_s^{4}-H_v(T_s-T_{ext}(t))-r_{s/d}(T_s-T_f)
\end{equation}

\begin{equation}
((1-\phi) C_d h_d + \phi C_f h_d ) \frac {\partial T_f}{\partial t} + K_p C_f h_d \frac {\partial T_f}{\partial x}=r_{s/d}(T_s-T_f) - r_{d/b1}(T_f-T_{b1})
\end{equation}

\begin{equation}
C_b h_{b1}\frac{\partial T_{b1}}{\partial t}=r_{d/b1}(T_f-T_{b1})- r_{b1/b2}(T_{b1}-T_{b2})
\end{equation}

\begin{equation}
C_b h_{b2}\frac{\partial T_{b2}}{\partial t}= r_{b1/b2}(T_{b1}-T_{b2})
\end{equation}

Par descente de gradient, un modèle 1D a été fitté pour coller au modèle 2D développé par le Cerema en langage fortran. Une fonction python, appelée dromo.iterate dans l'algorithme, permet de calculer la température en sortie de dromotherm, avant que le fluide s'engage dans l'échangeur de séparation de réseau pour venir alimenter le stockage en chaleur.

Le calcul utilise :

- les données météo de l'instant en cours, à savoir la temperature d'air **exprimée en kelvin**, les rayonnement global et atmosphérique exprimés en $\frac{W}{m^2}$ ainsi que la vitesse du vent en $\frac{m}{s}$

- la température d'injection du fluide dans le dromotherme **exprimée en kelvin**

- le débit unitaire du fluide dans le dromotherme $qdro_u$ exprimé en $\frac{m^3}{s}$.

Par débit unitaire, on entend le débit pour un mètre linéaire de chaussée selon le profil en long.

Lorsque le dromotherme est à l'arrêt, la température de sortie de dromotherm est obtenue par le même calcul, avec un débit unitaire nul.

**Afin de ne pas refroidir le stockage, son alimentation ne se fera que la la température de sortie de dromotherme est supérieure à la température du stockage.**

# Equation du stockage

Notons :

- $m_{sable}$ la masse de sable humide (sable sec + eau ) contenue dans le stockage et exprimée en kg

- $Cp_{sable}$ la capacité calorifique massique du sable humide en J/Kg.K

Pour simplifier et faciliter la lecture, l'équation ci-dessous correspond au seul cas où le dromotherm et la PAC sont en fonctionnement simultané :

\begin{equation}
    m_{sable} Cp_{sable} \frac {dy}{dt}=m_{sto} Cp_{sto} ( T_{inj\_sto} - T_{sor\_sto} ) + m_{pac} Cp_{pac} (T_{sor\_pac}- T_{inj\_pac})
\end{equation}

Dans l'algorithme, il conviendra de traiter les autres cas de figure (dromotherme seul en fonctionnement, PAC seule en fonctionnement, aucun des 2 équipements en fonctionnement)

# Algorithme de résolution

Notons :

- i l'indice temporel

- DRO le booléen indiquant si le dromotherme est en marche

- PAC le booléen indiquant si la PAC est en marche

- step le pas de temps exprimé en secondes

Le schéma ci-dessous décrit une itération du système complet, pour passer de l'indice i-1 à i

Dans les parties précédantes du notebook, nous avons utilisé la notation y pour représenter la température du sable dans le stockage. Dans l'algorithme, y est remplacé par la notation Tsable et sa dérivée temporelle est notée diff.

En appliquant le schéma de discrétisation d'Euler explicite, on peut écrire :

\begin{equation}
 T_{sable}(i) = T_{sable}(i-1)+ step * \frac {dT_{sable}}{dt}(i) = T_{sable}(i-1)+ step * diff(i)
\end{equation}

A l'instant initial, la température d'injection du fluide dans le dromotherme est prise égale à 10°C

Cet algorithme a été codé sous la forme d'un module python appelé sencity (cf documentation en ligne : https://seviprince.github.io/dromotherm/index.html)

On pourrait poursuivre en faisant tourner le code en live, en important la bibilothèque

```python

```


##### Oumayma Mahfoudhi 2 DNI 1

## Objectif :
>L'objectif de ce pemier TP est de decouvrir et de se familiariser avec les trois methodes de résoulution f(x) = 0 présentés comme suit :
>
>* La méthode de la dichotomie (ou bissection)
>
>* La méthode de point fixe
>
>* La méthode de Newton

## Exercice 1:

>On considère le polynôme $p(x) = (x - 1)^6$.
>
>En développant on
>peut écrire ce même polynôme sous la forme
>$f(x) = x^6 - 6x^5 + 15x^4 - 20x^3 + 15x^2 - 6x + 1.$
>
>1) Avec Matlab évaluer $p(x)$ et $f(x)$ pour $x$ allant de $.998$ à
>$1.002$ en choisissant des pas de $.0001$.
>
>2) Faire un graphique de $f(x)$ et $p(x)$. Commenter.

#### 1)

### Résultat ( MATLAB/OCTAVE)

> Il existe plusieurs solutions pour $P(x)$ et $f(x)$ c'est a dire l'intersection avec la courbe d'abscisse

#### 2)

```python
f=lambda x: x**6 - 6*x**5 + 15*x**4 - 20*x**3 + 15*x**2 - 6*x + 1
p=lambda x:(x-1)**6
# le calcul de p(0.998)
print('p({})={} '.format(0.998,p(0.998)))
# le calcul de f(0.998)
print('f({})={} '.format(0.998,f(0.998)))
```

    p(0.998)=6.400000000000034e-17
    f(0.998)=1.7763568394002505e-15

### Courbe de la fonction f(x)

```python
import matplotlib.pyplot as plt
import numpy as np
T=np.linspace(.998,1.002)
plt.title("solution dans [0.998:1.002] ");
plt.xlabel("x")
plt.ylabel ("f(x)")
plt.grid(True)
plt.plot(T,f(T))
```

### Courbe de la fonction p(x)

```python
import matplotlib.pyplot as plt
import numpy as np
T=np.linspace(.998,1.002)
plt.title("solution dans [0.998:1.002] ");
plt.xlabel("x")
plt.ylabel ("p(x)")
plt.grid(True)
plt.plot(T,f(T))
```

# EXERCICE 2:

>On considère l'équation : $$ f(x)=x^3+x^2-3x-3=0$$
>1) Dessiner la courbe de $f$ dans l'intervalle $[-2;2]$, puis trouver des intervalles convenables pour appliquer la méthode de >bisection.
>
>2) Utiliser un programme `dicho.m` qui permettra de trouver les solutions de l'équation $f(x)=0$ avec $eps=0.001$.
>
>3) Modifier le programme `dicho.m` pour qu'il donne le nombre d'itération nécessaire pour avoir la solution à $eps$ près.

### la courbe de f(x) dans l'intervalle [-2;2]

```python
import matplotlib.pyplot as plt
from numpy import linspace
f=lambda x: x**3+x**2-3*x-3
T=linspace(-2,2)
plt.title("intervalle dans [-2:2] ");
plt.xlabel("x")
plt.ylabel ("f(x)")
plt.grid(True)
plt.plot(T,f(T))

```

## Daprés le graph de $f$ il existe 3 solutions :
- $\alpha_1 \in ]-2;-1.5[$
- $\alpha_2 \in ]-1.5;-0.5[$
- $\alpha_3 \in ]1.5;2[$

`[x]=dicho(f,-2,-1.5,0.001)`

`[x]=diho(f,-2,-1.5,0.001)`

`[x]=dicho(f,-2,-1.5,0.001)`

### La méthode de dichotomie

```python
f=lambda x: x**3+x**2-3*x-3
def dichotomie (f , a , b , epsilon ):
            g , d = min (a , b ) , max (a , b )
            valg , vald = f ( g ) , f ( d )
            n = 0
            while d - g > epsilon :
                n += 1
                m = ( g + d ) / 2
                valm = f ( m )
                if valg * valm < 0 :
                    d = m
                    vald = valm
                else :
                    g = m
                    valg = valm
            print (" nombre d'itérations nécessaires :", n )
            return ( g + d ) / 2
```

- $\alpha_1 \in ]-2;-1.5[$
- $\alpha_2 \in ]-1.5;-0.5[$
- $\alpha_3 \in ]1.5;2[$

```python
dichotomie(f,-2,-1.5,0.0001)
```

     nombre d'itérations nécessaires : 13

    -1.732025146484375

```python
import numpy as np
np.floor(2.3)
np.ceil(2.3)
```

    3.0

```python
import numpy as np
a=-1.5
b=-0.5
p=8 # précision à 10 ^-8 prés
dichotomie(f,a,b,10**(-p))
#à 10^-8

```

     nombre d'itérations nécessaires : 27

    -0.5000000037252903

```python

################################################
print(dichotomie(f,-2,-1.5,10**(-3)))
print(dichotomie(f,-1.5,-0.5,10**(-3)))
print(dichotomie(f,1.5,2,10**(-3)))

```

     nombre d'itérations nécessaires : 9
    -1.73193359375
     nombre d'itérations nécessaires : 10
    -0.50048828125
     nombre d'itérations nécessaires : 9
    1.73193359375

Le nombre d'itération N nécéssaire à la méthode de la dichotomie pour trouver $\alpha$ à $10^{-p}$ prés

$$N \geq E\left(\dfrac{\ln(b-a)+p\ln(10)}{\ln(2)}\right)+1$$

```python
a=-2
b=-1.5
ND(5)
```

    16.0

```python
#alpha 1
dichotomie(f,-2,-1.5,10**(-5))
#à 10^-8
```

     nombre d'itérations nécessaires : 16

    -1.7320518493652344

```python
#alpha 2
dichotomie(f,-1.51,-0.5,10**(-5))
#à 10^-8
```

     nombre d'itérations nécessaires : 17

    -1.0000028610229492

## Conclusion
> L’erreur dans la méthode de la dichotomie ne depend pas de f. Ceci est un avantage de cette méthode.
>
> La m´ethode de la dichotomie est lente.

## Méthode graphique pour la recherche de la solution

```python
%matplotlib inline
#import mpld3
#mpld3.enable_notebook()
import numpy as np
import matplotlib.pyplot as plt

f=lambda x:x**3+x**2-3*x-3

t=np.linspace(1.7,1.75,100)
plt.subplot(2,2,1)
plt.plot(t,f(t),'r')
plt.grid(True)
##########################################
t=np.linspace(1.2,1.4,100)
plt.subplot(2,2,2)
plt.plot(t,f(t),'r')
plt.grid(True)
##########################################
t=np.linspace(1.35,1.4,100)
plt.subplot(2,2,3)
plt.plot(t,f(t),'r')
plt.grid(True)
#########################################
t=np.linspace(1.365,1.367,100)
plt.subplot(2,2,4)
plt.plot(t,f(t),'r')
plt.grid(True)

```

# EXERCICE 3:
>On considère l'équation : $$ f(x)=x^3+4x^2-10=0$$
>1) Dessiner la courbe de $f$ dans l'intervalle $[1;2]$, puis trouver une valeur approché de la solution à 0.001 près (En >utilisant la fonction zoom du graphique).
>
>2) Vérifier que cette équation est équivalente aux 3 schéma du point fixe suivant:
>
>a) $g_1(x)=\frac{1}{2}\sqrt{10-x^3}=x$.
>
>b) $g_2(x)=\sqrt{\displaystyle\frac{10}{4+x}}=x$.
>
>c) $g_3(x)=x-x^3-4x^2+10=x$.
>
>3) Ecrire puis appliquer  une fonction Matlab que l'on appellera  `pointfixe.m` aux trois schémas précédents en prenant >`x0=1.5;eps=0.001`.
>
>4) Quelle est la fonction (parmi $g_1, g_2$ et $g_3$) qui donne la convergence la plus rapide?
>

### Correction

#### 1)

```python
%matplotlib inline
#import mpld3
#mpld3.enable_notebook()
import numpy as np
import matplotlib.pyplot as plt

f=lambda x:x**3+4*x**2-10

t=np.linspace(1,2,100)
plt.subplot(2,2,1)
plt.plot(t,f(t),'r')
plt.grid(True)
##########################################
t=np.linspace(1.2,1.4,100)
plt.subplot(2,2,2)
plt.plot(t,f(t),'r')
plt.grid(True)
##########################################
t=np.linspace(1.35,1.4,100)
plt.subplot(2,2,3)
plt.plot(t,f(t),'r')
plt.grid(True)
#########################################
t=np.linspace(1.365,1.367,100)
plt.subplot(2,2,4)
plt.plot(t,f(t),'r')
plt.grid(True)

```

#### 2) Justification théorique pour la fonction g1

On considère l'équation : $$ f(x)=x^3+4x^2-10=0$$

a) $g(x)=\frac{1}{2}\sqrt{10-x^3}=x$.

 $$x^3+4x^2-10=0$$

 $$4x^2=10-x^3$$

 $$2x=\sqrt{10-x^3}$$

 $$x=\frac{1}{2}\sqrt{10-x^3}$$

Donc $ f(x)=g(x)$

## Calcul symbolique de la dérivée

```python
from sympy import *
import numpy as np
x, y, z = symbols('x y z')
init_printing(use_unicode=True)
g1=lambda x: 1/2*sqrt(10-x**3)

```

```python
dg1=lambdify(x,diff(g1(x),x,1))
dg1(1.365)
```

### la convergences de g1

> g1 converge d'aprés la calcul symbolique de dérivé

## Justification théoriques pour g2 et g3

On considère l'équation : $$ f(x)=x^3+4x^2-10=0$$

b) $g(x)=\sqrt{\displaystyle\frac{10}{4+x}}=x$.

$$x^3+4x^2-10=0$$
$$x^3+4x^2=10$$
$$x^2(x+4)=10$$
$$x^2=\frac{10}{x+4}$$
$$x=\sqrt{\displaystyle\frac{10}{4+x}}$$
Donc $ f(x)=g(x)$

On considère l'équation : $$ f(x)=x^3+4x^2-10=0$$
c) $g(x)=x-x^3-4x^2+10=x$.

$$x^3+4x^2-10=0$$
$$x^3+4x^2-10+x-x=0$$
$$x^3-4x^2+x-x=10$$
$$x^3-4x^2+x=x+10$$
$$4x^2+x=x-x^3+10$$
$$x=x-x^3-4x^2+10$$
Donc $ f(x)=g(x)$

##  la convergences de g2 et g3

> g2 converge par contre g3 diverge

#### 4)

### la solution avec un codepython

```python
f=lambda x:  x**3+4*x**2-10 # la fonction f
g1=lambda x: 1/2*sqrt(10-x**3)
g2=lambda x: sqrt(10/(x+4))
g3=lambda x: x-x**3-4*x**2 + 10

from math import *

def point_fixe(g, x0, epsilon, n_itermax):
  """
    Retourne un tableau avec tous les points calculés ou une erreur
  """
  itr = 0
  xp = {}
  x = 0

  while (itr < n_itermax):
    try:
      x = g(x0)
    except OverflowError:
      break

    xp[itr] = x0
    if abs(x - x0) <= epsilon:
      return xp

    x0 = x
    itr += 1

  return "Aucun point fixe"

def print_table(xp):
  """
    Affiche un tableau avec toutes les valeurs ou l'erreur
  """
  s = ""
  if isinstance(xp, str):
    s = xp
  else:
    for key, value in xp.items():
      s += "n = {}\t\tx = {}\n".format(key, value)
  print(s)

```

```python
point_fixe(g1,1.5,0.0001,30)
```

```python
point_fixe(g2,1.5,0.0001,30)
```

```python
point_fixe(g3,1.5,0.0001,30)
```

    'Aucun point fixe'

### le nombre d'itération pour chaque fonction g(x)
+ $g_2$ donne le resultat en 5 itérations
>
+ $g_1$ donne le resultat en 13 itérations
>
+ $g_3$ par contre gne converge pas 'diverge '

### Conclusion
> la fonction qui donne la convergance la plus rapide est $g_2$
>
> l’ordre de convergence d’une méthode itérative depand du choix de la fonction g(x)

# EXERCICE 4:
>1) On considère la fonction $f(x)=e^{-x}-x$. Ecrire puis appliquer  une fonction Matlab que l'on appellera `newton.m` à la >fonction $f$, en prenant $x0=0$, $eps=0.001$ .
>
>2) Montrer que l'équation $f(x)=0$ admet une solution unique que l'on peut obtenir en utilisant la méthode de Newton à partir >de $x0=1$.

### Fonction MATLAB

#### 1)

```python
function sol=newtonn(a,eps)
x1=a;
x2=x1-f(x1)/fprime(x1);

while abs(x1-x2)>eps
     x1=x2;
     x2=x1-f(x1)/fprime(x1);
end
sol=x2;
```

```python
f=@(x) exp(-x)-x
fprime=@(x) -exp(-x)-1
t=linspace(0,1,100);
plot(t,f(t))
grid on
```

```python
sol=newton(0,0.001)
```

#### 2)

```python

```

### Solution Python

```python
def newton(fonction,derivee,a,e):
    delta = 1
    while delta > e:
        x = -fonction(a)/derivee(a) + a
        delta = abs(x - a)
        a = x

    return x , delta

print( newton(lambda x: x**2+2*x+1 , lambda x: 2*x+2 , 0 , 0.001) )
```

    (-0.9990234375, 0.0009765625)

> la méthode de Newton est un cas particulier de la m´ethode de point fixe avec la fixation de valeur de fonction de g(x) a partir de f(x)

## Conclusion générale
> la méthode de Newton est la plus rapide par rapport au autre methodes (point fixe , dichotomie)

```python

```


1) Как относятся друг к другу множество и последовательность? (в ответе использовать слова типа: часть, целое, общее, частное, родитель, дочерний субъект и т.д.)

Предположительно, множество отновится к последовательности как родитель к дочернему субъекту

```python
# Даны 4 последовательности. Необходимо:
# исследовать их на монотонность;
# исследовать на ограниченность;
# найти пятый по счету член.

n = 10
a = [0] * n
# ограничен снизу 1, сверху бесконечностью
for i in range(1, n):
    a[i] = 2 ** i - i

b = [0] * n
# ограничен снизу -1, сверху 0
for i in range(2, n):
    b[i] = 1 / (1 - i)

c = [0] * n
# ограничен снизу ~0 (точнее 0.4), сверху бесконечностью
for i in range(1, n):
    c[i] = (-1) ** i + (2 * i) ** .5

d = [0] * n
# ограничен снизу 2, сверху 1
for i in range(1, n):
    d[i] = (-1) ** (2 * i) + 1 / (i ** 2)

# обрезаем лишние(откуда начинаются последовательности)
my_dict = {
    'A': a[1:],
    'B': b[2:],
    'C': c[1:],
    'D': d[1:]
}
```

```python
def monotons(set):
    if all(set[i] < set[i+1] for i in range(len(set)-1)):
        return 'Монотонно возрастает'
    elif all(set[i] > set[i+1] for i in range(len(set)-1)):
        return 'Монотонно убывает'
    else:
        return 'Последовательность не возрастает и не убывает'

for k, i in my_dict.items():
    print(f'{k} = {i}')
    print(monotons(i))
    print(f'5-й по счету элемент: {k} = {i[4]}')
    print('=======================================================================================================')
```

    A = [1, 2, 5, 12, 27, 58, 121, 248, 503]
    Монотонно возрастает
    5-й по счету элемент: A = 27
    =======================================================================================================
    B = [-1.0, -0.5, -0.3333333333333333, -0.25, -0.2, -0.16666666666666666, -0.14285714285714285, -0.125]
    Монотонно возрастает
    5-й по счету элемент: B = -0.2
    =======================================================================================================
    C = [0.41421356237309515, 3.0, 1.4494897427831779, 3.8284271247461903, 2.1622776601683795, 4.464101615137754, 2.7416573867739413, 5.0, 3.2426406871192848]
    Последовательность не возрастает и не убывает
    5-й по счету элемент: C = 2.1622776601683795
    =======================================================================================================
    D = [2.0, 1.25, 1.1111111111111112, 1.0625, 1.04, 1.0277777777777777, 1.0204081632653061, 1.015625, 1.0123456790123457]
    Монотонно убывает
    5-й по счету элемент: D = 1.04
    =======================================================================================================

```python
# Найти 12-й член заданной неявно последовательности

a = 128
for i in range (2, 13):
    a = 6 + a

print(a)

print(f'Альтернатива = {128 + (12 - 1) * 6}')

```

    194
    Альтернатива = 194

```python
#*На языке Python предложить алгоритм вычисляющий численно предел с точностью
from sympy import factorial
eps = 10**-7
lim = 1
i = 1
n = 1
x0 = 1
while lim > eps:
    i += 1
    n += 1
    x1 = n / pow(factorial(n), 1 / n)
    lim = abs(x0 - x1)
    x0 = x1
print(f'n_iter: {i}')
print(f'f({n}) = {x0}')
```

    n_iter: 11779
    f(11779) = 2.71698842372422

```python
#*Предложить оптимизацию алгоритма, полученного в задании 3, ускоряющую его сходимость.
lim = 1
i = 1
n = 1
x0 = 1
while lim > eps:
    i += 1
    if lim > eps*100:
        n += 99
    else:
        n +=1
    x1 = n / pow(factorial(n), 1 / n)
    lim = abs(x0 - x1)
    x0 = x1
print(f'n_iter: {i}')
print(f'f({n}) = {x0}')
```

    n_iter: 121
    f(11783) = 2.71698882354986

```python

```


```python
# 그래프, 수학 기능 추가
# Add graph and math features
import pylab as py

```

# 수치 적분
Numerical Integration

여기서는 정적분의 근사값을 수열의 합으로 계산할 것이다.

Here, we would approximate the definite integral by a sum of a sequence.

면적이 2인 원의 반지름을 구해 보자.
Let's find the radius of a circle with area of 2.

$$
\begin{align}
    \pi r^2 &= 2 \\
    r^2 &= \frac{2}{\pi} \\
    r &= \sqrt{\frac{2}{\pi}}
\end{align}
$$

```python
r = py.sqrt(2.0 / py.pi)

```

```python
r

```

이러한 원의 중심이 원점에 위치하고 있다고 생각해 보자.
Let's assume that a circle of such radius has its center at the origin.

$$
x^2 + y^2 = r^2 \\
y^2 = r^2 - x^2 \\
y = \pm \sqrt{r^2 - x^2} \\
y_{plus} = + \sqrt{r^2 - x^2} \\
y_{minus} = - \sqrt{r^2 - x^2}
$$

```python
x_array = py.linspace(-r, r, 64)
y_plus = py.sqrt(r**2 - x_array ** 2)
py.plot(x_array, y_plus, '.-')

x_array2 = r * py.cos(py.deg2rad(py.linspace(180, 0, 16)))
y_minus = -py.sqrt(r**2 - x_array2 ** 2)
py.plot(x_array2, y_minus, '.-')

py.axis('equal')
py.grid(True)

```

$+$ 부분만 생각하기로 하자.
Let's just think about the $+$ side only.

$$
y = \sqrt{r^2 - x^2}
$$

```python
x_array = py.sort(r * py.cos(py.deg2rad(range(0, 180))))
y_plus = py.sqrt(r**2 - x_array ** 2)

py.fill_between(x_array, y_plus)

py.axis('equal')
py.grid(True)

```

이 반원의 면적을 수치적으로 구해보기로 하자.  (반원의 정확한 값은 얼마이겠는가?)

Let's try to numerically find the area of this half-circle. (What would be the exact value?)

## 0차 적분
0th Order Integration

우선 $x$를 일정 간격으로 나누어 보자.
Let's divide the $x$ coordinates in a constant interval.

```python
d = r * 2.0

n = 10

x_array_bar = py.linspace(-r, r, n+1)
y_array_bar = py.sqrt(abs(r**2 - x_array_bar ** 2))
delta_x = x_array_bar[1]-x_array_bar[0]

py.fill_between(x_array, y_plus)
# TODO : 막대그래프 직사각형 안을 칠하지 않으려면 어떻게 하면 좋겠는가?
# TODO : How can we remove the color inside the rectangle?
py.bar(x_array_bar, y_array_bar, width=delta_x, alpha=0.5, align='edge', edgecolor='k')

py.axis('equal')
py.grid(True)

```

아래 셀은 `x_array` 간격을 확인한다.

Following cell verifies increment of `x_array`.

```python
assert 1e-3 > abs(delta_x - (d/n)), (delta_x, d/n)

```

직사각형의 모양과 반원의 모양이 정확히 일치하지는 않는다는 점을 기억하자.

Let's remember that the areas of the rectangles and the half circle are not exactly the same.

각 직사각형의 면적을 구해서 더해 보자
Let's find the area of each rectangle and sum up.

$$
    Area = \sum_{k=0}^{n-1} F_k
$$

$$
    F_k = f(x_k)\cdot \Delta x
$$

$$
    Area = \sum_{k=0}^{n-1} f(x_k)\cdot \Delta x
$$

```python
summation = 0

for k in range(n):
    F_k = y_array_bar[k] * delta_x
    print('k = %2d, F_k = %g' % (k, F_k))
    summation += F_k

print('summation =', summation)

```

예상한 값 1에 더 비슷한 값을 얻기 위해 더 잘게 나누어 보자
To obtain the result closer to the expected value of 1, let's divide with a narrower interval.

```python
n = 100

x_array_bar = py.linspace(-r, r, n+1)

y_array_bar = py.sqrt(r**2 - x_array_bar ** 2)
delta_x = x_array_bar[1]-x_array_bar[0]

py.fill_between(x_array, y_plus)
py.bar(x_array_bar, y_array_bar, width=delta_x, alpha=0.5, align='edge', edgecolor='k')

py.axis('equal')
py.grid(True)

```

```python
summation = 0

for k in range(n):
    summation += delta_x * y_array_bar[k]

print('summation =', summation)

```

더 잘게 나눈 결과에 대한 의견은 어떠한가?

What is your opinion about using the narrower partitions?

### 함수로 구현
Implement in a Function

다양한 경우에 더 편리하게 적용하기 위해 함수 형태로 만들어 보자.
To make it more convenient to apply to various cases, let's implement in a function

```python
def half_circle(x):
    return py.sqrt(r**2 - x ** 2)

```

```python
def get_delta_x(xi, xe, n):
    return (xe - xi) / n

```

```python
def num_int_0(f, xi, xe, n, b_verbose=False):
    x_array = py.linspace(xi, xe, n+1)

    delta_x = x_array[1] - x_array[0]

    assert 1e-3 > (abs(delta_x - get_delta_x(xi, xe, n)) / get_delta_x(xi, xe, n)), f"delta_x = {delta_x}"

    integration_result = 0.0

    for k in range(n):
        x_k = x_array[k]
        F_k = f(x_k) * delta_x

        if b_verbose:
            print('k = %2d, F_k = %g' % (k, F_k))

        integration_result += F_k

    return integration_result

```

```python
n = 100
result = num_int_0(half_circle, -r, r, n)
print('result =', result)

```

```python
%timeit -n 100 result = num_int_0(half_circle, -r, r, n)

```

### $cos \theta$의 반 주기
Half period of $cos \theta$

```python
n = 10
result_cos = num_int_0(py.cos, 0, py.pi, n, b_verbose=True)
print('result =', result_cos)

```

```python
n = 100
result_cos = num_int_0(py.cos, 0, py.pi, n)
print('result =', result_cos)

```

### 1/4 원
A quarter circle

```python
n = 10
result_quarter = num_int_0(half_circle, -r, 0, n, b_verbose=True)
print('result =', result_quarter)

```

```python
n = 10
result_quarter = num_int_0(half_circle, 0, r, n, b_verbose=True)
print('result =', result_quarter)

```

```python
n = 100
result_quarter = num_int_0(half_circle, -r, 0, n)
print('result =', result_quarter)

```

## 연습문제
Exercises

도전 과제 1: $e^ x$ 를 $0 \le x \le 1$ 구간에서 0차 적분으로 적분하시오.  이론적 엄밀해와 비교하시오.
Try this 1: Integrate  $e^ x$ over $0 \le x \le 1$ interval using the 0th order integration.  Compare with the exact solution.

```python

```

도전 과제 2: $sin \left( cos x \right)$ 를 $0 \le x \le 1$ 구간에서 적분하시오.

Try this 2: Integrate $sin \left( cos x \right)$ over $0 \le x \le 1$ interval.

(ref : [Examples for
Numerical Integration](https://www.wolframalpha.com/examples/mathematics/applied-mathematics/numerical-analysis/numerical-integration/), Wolfram Alpha, Accessed Aug 28 2018)

```python

```

도전 과제 3: 이미 다루었던 이분법 함수와 0차 적분을 이용하여 면적이 2인 원의 반지름을 구하는 프로그램을 작성하고 실행해 보시오.

Try this 3: Using the bisection method function and 0th roder intergration, write a program finding radius of a circle with area of two, and run it.

```python

```

## 리만 합
Riemann Sum

이렇게 어떤 함수의 정적분을 유한한 합으로 바꾸어 계산하는 것을 리만 합이라고 부른다.

Riemanm Sum is a type of approximation of an integral by a finite sum. [[wikipedia](https://en.wikipedia.org/wiki/Riemann_sum)]

## `pylab.linspace()`

일정 간격의 배열을 생성한다.

This would generate an array of a constant interval.

```python
py.linspace(0, 10, 5)

```

아래 셀의 결과를 비교해 보시오.

Compare the results of the following cells.

```python
py.arange(0, 10+1, 1)

```

```python
py.linspace(0, 10, 10+1)

```

## 함수형 프로그래밍
Functional programming

간격이 일정하다면 면적의 근사값을 다음과 같이 바꾸어 쓸 수 있다.

If the interval $\Delta x$ is constant, we may rewrite the approximation of the area as follows.

$$
    Area = \sum_{k=0}^{n-1} f(x_k)\cdot \Delta x= \Delta x \sum_{k=0}^{n-1} f(x_k)
$$

할당문 없이 `sum()` 과 `map()` 함수로 구현해 보자.

Instead of assignments, let's implement using `sum()` and `map()` functions.

```python
def num_int_0_functional(f, xi, xe, n):
    return (
        get_delta_x(xi, xe, n) * sum(
            map(
                f,
                py.linspace(xi, xe, n+1)[:-1],
            )
        )
    )

```

```python
n = 100
result_func = num_int_0_functional(half_circle, -r, r, n)
print('result_func =', result_func)

```

```python
assert 1e-3 > abs(result - result_func), f"result = {result}, result_func = {result_func}"

```

```python
%timeit -n 100 result_func = num_int_0_functional(half_circle, -r, r, n)

```

이와 같이 할당문과 부가적 효과 없이 함수로만 구현하는 형태를 *함수형 프로그래밍* 이라고 한다.

*Functional programming* is to implement with functions only without assignments and side effects. [[Sahota](https://dev.to/navi/why-functional-programming-matters-2o95)]

## NumPy 벡터화
Vectorization of NumPy

```python
import pylab as py

```

```python
def num_int_0_vector(f, xi, xe, n):
    return f(
        py.linspace(xi, xe-get_delta_x(xi, xe, n), n)
    ).sum() * get_delta_x(xi, xe, n)

```

```python
n = 100
result_vect = num_int_0_vector(half_circle, -r, r, n)
print('result_vect =', result_vect)

```

```python
assert 1e-3 > abs(result - result_vect), f"result = {result}, result_vect = {result_vect}"

```

```python
%timeit -n 100 result_vect = num_int_0_vector(half_circle, -r, r, n)

```

## 시험
Test

아래는 함수가 맞게 작동하는지 확인함

Following cells verify whether the functions work correctly.

```python
import pylab as py
r = py.sqrt(1.0 / py.pi)
n = 10

def half_circle(x):
    return py.sqrt(r**2 - x ** 2)

assert 0.25 > num_int_0(half_circle, -r, 0, n), num_int_0(half_circle, -r, 0, n)
assert 0.25 < num_int_0(half_circle, 0, r, n), num_int_0(half_circle, 0, r, n)
assert 0.25 > num_int_0_functional(half_circle, -r, 0, n), num_int_0_functional(half_circle, -r, 0, n)
assert 0.25 < num_int_0_functional(half_circle, 0, r, n), num_int_0_functional(half_circle, 0, r, n)
assert 0.25 > num_int_0_vector(half_circle, -r, 0, n), num_int_0_vector(half_circle, -r, 0, n)
assert 0.25 < num_int_0_vector(half_circle, 0, r, n), num_int_0_vector(half_circle, 0, r, n)

```

```python
assert 0.1 > (abs(num_int_0(half_circle, -r, 0, n) - 0.25) * 4)
assert 0.1 > (abs(num_int_0(half_circle, 0, r, n) - 0.25) * 4)
assert 0.1 > (abs(num_int_0_functional(half_circle, -r, 0, n) - 0.25) * 4)
assert 0.1 > (abs(num_int_0_functional(half_circle, 0, r, n) - 0.25) * 4)
assert 0.1 > (abs(num_int_0_vector(half_circle, -r, 0, n) - 0.25) * 4)
assert 0.1 > (abs(num_int_0_vector(half_circle, 0, r, n) - 0.25) * 4)

```

## Final Bell
마지막 종

```python
# stackoverfow.com/a/24634221
import os
os.system("printf '\a'");

```

```python

```