# Value-at-Risk for Stocks: Delta-Normal Approach

### Lecture Notes by Jakov Ivan S. Dumbrique (jdumbrique@ateneo.edu)

MATH 100.2: Topics in Financial Mathematics II \
First Semester, S.Y. 2020-2021 \
Ateneo de Manila University

```python
import numpy as np # Numerical Computing
import pandas as pd # Data wrangling
import matplotlib.pyplot as plt # Plotting
from statistics import NormalDist # statistical analysis

%matplotlib inline
```

## Are stock returns normally distributed?
The Delta-Normal Approach assumes that the daily returns $R_t$ are normally distributed with mean 0 and variance $\sigma^2$. Is this a good assumption? Does the assumption of normality understate, properly state, or overstate the frequency of market disasters?

Let's look at the TEL data and plot it.

```python
data_path = "https://raw.githubusercontent.com/ateneomathdept/math100.2_2021Sem1/main/data/lectures/TEL_2018.csv"
TEL_df = pd.read_csv(data_path, index_col="dt")
TEL_df
```

      close

      dt

      2/23/18
      1488.74

      2/22/18
      1510.86

      2/21/18
      1513.72

      2/20/18
      1536.65

      2/16/18
      1476.37

      ...
      ...

      3/2/17
      1430.29

      3/1/17
      1408.94

      2/28/17
      1394.35

      2/27/17
      1374.93

      2/24/17
      1367.68

248 rows × 1 columns

```python
TEL_df["close"][::-1].plot() #pd.Series
plt.title("PLDT Historical Share Prices (Php)")
plt.show()
```

Let $S_t$ denote the value of the stock at time $t$. The one-day continuous return $R_t$ of the stock is given by
$$R_t = \ln\left(\frac{P_t}{P_{t-1}}\right).$$

```python
TEL_df["close"].shift(-1)
```

    dt
    2/23/18    1510.86
    2/22/18    1513.72
    2/21/18    1536.65
    2/20/18    1476.37
    2/16/18    1475.71
                ...
    3/2/17     1408.94
    3/1/17     1394.35
    2/28/17    1374.93
    2/27/17    1367.68
    2/24/17        NaN
    Name: close, Length: 248, dtype: float64

```python
# creates a new column that shifts the prices d values up (or -d values down)
TEL_df["previous"] = TEL_df["close"].shift(-1)
TEL_df
```

      close
      previous

      dt

      2/23/18
      1488.74
      1510.86

      2/22/18
      1510.86
      1513.72

      2/21/18
      1513.72
      1536.65

      2/20/18
      1536.65
      1476.37

      2/16/18
      1476.37
      1475.71

      ...
      ...
      ...

      3/2/17
      1430.29
      1408.94

      3/1/17
      1408.94
      1394.35

      2/28/17
      1394.35
      1374.93

      2/27/17
      1374.93
      1367.68

      2/24/17
      1367.68
      NaN

248 rows × 2 columns

```python
def get_return(df):
    return np.log(df["close"]/df["previous"])
```

```python
TEL_df["return"] = TEL_df.apply(get_return, axis=1)
TEL_df
```

      close
      previous
      return

      dt

      2/23/18
      1488.74
      1510.86
      -0.014749

      2/22/18
      1510.86
      1513.72
      -0.001891

      2/21/18
      1513.72
      1536.65
      -0.015035

      2/20/18
      1536.65
      1476.37
      0.040018

      2/16/18
      1476.37
      1475.71
      0.000447

      ...
      ...
      ...
      ...

      3/2/17
      1430.29
      1408.94
      0.015040

      3/1/17
      1408.94
      1394.35
      0.010409

      2/28/17
      1394.35
      1374.93
      0.014026

      2/27/17
      1374.93
      1367.68
      0.005287

      2/24/17
      1367.68
      NaN
      NaN

248 rows × 3 columns

Let's fit a normal distribution to the data, and plot the histogram of data against it.

```python
from scipy.stats import norm # statistical analysis
# Get data, drop the last row with nan value (not a number)
data = TEL_df["return"].dropna()

# Fit a normal distribution to the data:
mu, std = norm.fit(data) # provides MLE of the parameters for the Gaussian curve which has the highest likelihood of fitting our dataset

# Plot the histogram.
plt.hist(data, bins=40, density=True, alpha=0.6, color='g')

# Plot the PDF
xmin, xmax = plt.xlim() # gets the left and right limits of the x-axis of the current plot
x = np.linspace(xmin, xmax, 100) # array of 100 evenly-spaced numbers in the interval (xmin, xmax)
p = norm.pdf(x, mu, std) # array
plt.plot(x, p, 'k', linewidth=2) # k for black
title = "Fit results: mu = %.4f,  std = %.4f" % (mu, std)
plt.title(title)

plt.show()
```

The distribution looks approximately normal. Furthermore, the resulting MLE for the Gaussian mean is close to 0. Thus, by eyeballing, the normal distribution might be a good fit for the stock returns.

However, if we  look to the extreme left and right of the histogram distribution, we can see the famous fat tails. The fat tails mean that extreme events occur more frequently in reality than what a normal distribution would predict. More evidence of that is how the actual distribution of daily TEL returns is skinnier in its center than the normal distribution. The skinny middle and the fat tails imply that the normal distribution might not be the best distribution to describe the stock returns. This fat-tail distribution is common among stock returns calculated from real data. To more accurately model the stock returns, researchers consider 2 regimes — a calm regime where we spend most of the time that is normally distributed (but with a lower volatility) and a regime with high volatility and extreme (sometimes terrible) returns.

Nonethless, we cannot neglect that the normal distribution is still a good one to use. Practioners use the normal distribution as starting point for modelling stock returns as this is simple to use and implement. Any time we can model something with normal distributions, it makes life a lot easier. For example, the return of a portfolio consisting of many investments (each with normally distributed returns) is also normally distributed. And to describe an investment, we only need 2 values: the mean (a.k.a. the investment’s expected return) and the standard deviation (a.k.a. the investment’s risk). If we want to be thorough, we should also record the investment’s correlation with our overall portfolio.

There are other statistical ways to check normality of the daily returns, and not just to eyeball:
1. [QQ plot](https://towardsdatascience.com/what-in-the-world-are-qq-plots-20d0e41dece1)
2. [Jarque-Bera Normality test](https://www.statisticshowto.com/jarque-bera-test/)

Reference: https://towardsdatascience.com/are-stock-returns-normally-distributed-e0388d71267e

## VaR for Stocks: Single-Asset Portfolio

Consider a single-asset portfolio consisting of a long position on $N$ shares of a stock with current price $S_0$. Let $\Delta P$ denote the change in portfolio value.

The **1-day 99\% VaR** of the portfolio is the number $|V|$ such that
$$\mathbb{P}[\Delta P\geq V] = 0.99.$$
This is given by

\begin{equation}
\text{VaR}_{0.99} = |V| = NS_0\sigma\Phi^{-1}(0.99).
\end{equation}

**The 1-day 99\% VaR  is 1M pesos.**

There is 99% level of confidence that the loss of the portfolio will not exceed 1M pesos over the next trading day.

### Exercise 1:
A portfolio consists of Php 10,000,000 worth of shares of stock A. Assume that the daily volatility is 2%. What is the portfolio's one-day 99% VaR?

```python
portval = 10000000 # N*S_0
sigma = 0.02
alpha = 0.99

quantile = NormalDist().inv_cdf(alpha)
var = portval * sigma *quantile
round(var, 4)
```

    465269.5748

```python
quantile
```

    2.3263478740408408

### Exercise 2:
Today is February 23, 2018. You are a portfolio risk manager who is assigned to analyze the market risk of a portfolio of 700 PLDT (TEL) shares. What is the portfolio's one-day 99% VaR using the delta-normal approach?

```python
# read the data
TEL_df = pd.read_csv(data_path)

TEL_df
```

      dt
      close

      0
      2/23/18
      1488.74

      1
      2/22/18
      1510.86

      2
      2/21/18
      1513.72

      3
      2/20/18
      1536.65

      4
      2/16/18
      1476.37

      ...
      ...
      ...

      243
      3/2/17
      1430.29

      244
      3/1/17
      1408.94

      245
      2/28/17
      1394.35

      246
      2/27/17
      1374.93

      247
      2/24/17
      1367.68

248 rows × 2 columns

```python
alpha = 0.99
N = 700
S_0 = TEL_df.loc[0, "close"]
```

```python
S_0
```

    1488.74

```python
# creates a new column that shifts the prices d values up (or -d values down)
TEL_df["previous"] = TEL_df["close"].shift(-1)
TEL_df
```

      dt
      close
      previous

      0
      2/23/18
      1488.74
      1510.86

      1
      2/22/18
      1510.86
      1513.72

      2
      2/21/18
      1513.72
      1536.65

      3
      2/20/18
      1536.65
      1476.37

      4
      2/16/18
      1476.37
      1475.71

      ...
      ...
      ...
      ...

      243
      3/2/17
      1430.29
      1408.94

      244
      3/1/17
      1408.94
      1394.35

      245
      2/28/17
      1394.35
      1374.93

      246
      2/27/17
      1374.93
      1367.68

      247
      2/24/17
      1367.68
      NaN

248 rows × 3 columns

```python
TEL_df["return"] = TEL_df.apply(get_return, axis=1)
TEL_df
```

      dt
      close
      previous
      return

      0
      2/23/18
      1488.74
      1510.86
      -0.014749

      1
      2/22/18
      1510.86
      1513.72
      -0.001891

      2
      2/21/18
      1513.72
      1536.65
      -0.015035

      3
      2/20/18
      1536.65
      1476.37
      0.040018

      4
      2/16/18
      1476.37
      1475.71
      0.000447

      ...
      ...
      ...
      ...
      ...

      243
      3/2/17
      1430.29
      1408.94
      0.015040

      244
      3/1/17
      1408.94
      1394.35
      0.010409

      245
      2/28/17
      1394.35
      1374.93
      0.014026

      246
      2/27/17
      1374.93
      1367.68
      0.005287

      247
      2/24/17
      1367.68
      NaN
      NaN

248 rows × 4 columns

```python
# standard deviation of your stock returns
sigma = TEL_df["return"].std()
# returns the sample standard deviation (denom: N-1)
sigma
```

    0.019629260903431183

```python
quantile = NormalDist().inv_cdf(alpha)
var = N * S_0 * sigma * quantile
print(f"The one-day 99% VaR is Php {round(var, 2)}.")
```

    The one-day 99% VaR is Php 47587.79.

```python
N * S_0
```

    1042118.0

```python
def one_day_alpha_percent_VaR_single_stock_delta_normal(df, N, alpha):

    """Returns the one-day 100(alpha)% VaR of a single stock using Delta Normal Approach.

    Parameters
    ----------
    df : pandas.DataFrame
        has two columns: (1) dt [str] and (2) closing price [float]
        assumes the dates are arranged from newest to oldest, and the date today is the date on the first row
    N : int
        number of shares for the sole stock
    alpha : int
        the value to be used in calculting the 100(alpha)% VaR (e.g. 0.99, 0.95)

    Returns
    -------
    float (2 decimal places)
        one-day 100(alpha)% VaR of a single stock using Delta Normal Approach
    """

    S_0 = df.loc[0, "close"]

    # creates a new column that shifts the prices d values up (or -d values down)
    df["previous"] = df["close"].shift(-1)
    # get the daily returns
    df["return"] = df.apply(get_return, axis=1)

    # standard deviation of your stock returns
    sigma = df["return"].std()

    quantile = NormalDist().inv_cdf(alpha)
    var = N * S_0 * sigma * quantile

    return round(var, 2)

```

```python
one_day_alpha_percent_VaR_single_stock_delta_normal(TEL_df, N=700, alpha=0.99)
```

    47587.79

Today is February 23, 2018. You are a portfolio risk manager who is assigned to analyze the market risk of a portfolio of 1000 PLDT (TEL) shares. What is the portfolio's one-day 95% VaR using the delta-normal approach?

```python
# read the data
TEL_df = pd.read_csv(data_path)

one_day_alpha_percent_VaR_single_stock_delta_normal(TEL_df, N=1000, alpha=0.95)
```

    48067.34

```python

```


# ADG with a real variable and Chebyshev's inequality
**author:Alessio Benavoli**
 alessiobenavoli.com

We will learn how to build a PyRational **ADG (Almost Desirable Gambles)** belief model on the outcome of an experiment whose space of possibility is the real-line.

```python
%load_ext autoreload
%autoreload 2
from __future__ import absolute_import
from PyRational.models.ADG import  ADG
from sympy import symbols,  Interval, Piecewise, Eq, exp
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

*PyRational* uses *Sympy* for symbolic mathematics. We need to define in *Sympy*  a `symbol` associated to the real variable and its relative domain  (we use Sympy `Interval` for the latter).

```python
x=symbols('x', real=True)
domain_x=Interval(-50,50)
```

```python
model = ADG([x],[domain_x])
model
```

ADG model

  List of Symbols 	[x]

  Domain 	Ω=Interval(-50, 50)

  List of desirable gambles 	G=[]

  Avoiding sure loss? 	  to be verified

We assume that our agent, Alice, finds the following gambles desirable.

```python
G=[]
G.append( x)
G.append(-x)
G.append(x**2-1)
G.append(1-x**2)
```

We add all these gambles to `model` as follows:

```python
model.add_gambleList(G)
model
```

ADG model

  List of Symbols 	[x]

  Domain 	Ω=Interval(-50, 50)

  List of desirable gambles 	G=[x, -x, x**2 - 1, -x**2 + 1]

  Avoiding sure loss? 	  to be verified

```python
model.Gambles
```

    [x, -x, x**2 - 1, -x**2 + 1]

Note that $G$ is a list that includes all Alice's desirable gambles. We now `build` the belief model
and check if it avods sure loss:

```python
optimoptions={'method_LISP': 'Cutting_plane', #'Cutting_plane', 'discretise'
                       'SolverLP':'cplex', #'linprog', 'cplex'
                       'LP_acc_constraints':1e-30,
                       'SolverNLP':'differential_evolution',
                       'NLP_alpha_cut':-0.0001,
                       'num_support_points': 60,
                       'verbose':False}
model.buildModel(options=optimoptions)
model.check_avs(options=optimoptions)
model
```

    Belief Model avoids sure  loss

ADG model

  List of Symbols 	[x]

  Domain 	Ω=Interval(-50, 50)

  List of desirable gambles 	G=posi(ℒ(Ω)+      ∪    [x, x**2 - 1])

  Avoiding sure loss? 	 Yes

So Alice is **rational** or, equivalently, her set of desirable gambles is coherent.

## Inference

Assume Alice is interested in computing her maximum buying price for the gamble

$$
f=\left\{\begin{array}{ll}
1 & \text{if } |x|\leq k\\
0 & \text{if } |x|>k\\
\end{array}\right.
$$

for some $k>0$. We can do that as follows:

```python
LP=[]
f_range=(None,None)
kappas=np.linspace(1,20,100)
for k in kappas:
    f=Piecewise((1,abs(x)<=k),(0.0,True))
    lp=model.lower_prevision(f,f_range,options=optimoptions)
    LP.append(lp)
LP=np.array(LP)
```

```python
plt.plot(kappas,LP,label='Lower Prevision')
plt.plot(kappas,1-1/kappas**2,label="Chebyshev\'s bound")
plt.xlabel('k')
plt.legend()
```

Hence, the lower prevision of $f$ coincides with Chebyshev's inequality for a standardised variable (zero mean and variance one).

$$
P(|X|\leq k)\leq 1-\frac{1}{k^2}
$$

## Updating

In ADG we can define updating by either defining an indicator on an event or defining a weighting function.
This latter case is formally similar to Bayes'rule with proability density functions.
Let us consider for instance the following unnormalised Gaussian density

```python
h = exp(-(x-1)**2/0.5)
```

```python
LP=[]
f_range=(None,None)
kappas=np.linspace(1,20,100)
for k in kappas:
    f=Piecewise((1,abs(x)<=k),(0.0,True))
    lp=model.lower_prevision(f,f_range,h=h,options=optimoptions)
    LP.append(lp)
LP=np.array(LP)
```

```python
plt.plot(kappas,LP,label='Updated Lower Prevision')
plt.plot(kappas,1-1/kappas**2,label="Chebyshev\'s bound")
plt.xlabel('k')
plt.legend()
```

```python
newG = model.updating(options=optimoptions,h=h)
```

```python
newG
```

    [x + 0.332886488778677,
     -x + 1.60212040406126,
     x**2 - 9.47130647405148e-5,
     -x**2 + 3.12061775090938]


# Contact Integrators

## Herglotz variational principle
...

First we define the system's potential energy $V(x)$. In the case of the usual pendulum we have

\begin{equation}
    V(\theta) = \frac{1}{2} \big(1 - cos(\theta)\big)
\end{equation}

```python
import numpy as np
import matplotlib.pyplot as plt

def V(x):
    return 0.5*(1.-np.cos(x))
    #return 0.5*x**2.

def V_der(x):
    return 0.5*(np.sin(x))
    #return x
```

Given the above definition, the Lagrangian function will be given by

```python
def L(x_0,x_1,z_0,z_1,alpha):
    a = (x_1-x_0)/h
    b = V(x_0) + V(x_1)
    c = z_0**2 + z_1**2.
    return 0.5*a**2. - 0.5*b - 0.25*alpha*c
```

```python
h = 0.001
n_step = 50000
n_particles = 5

fig = plt.figure()
axs = fig.gca()

X = []
P = []
Z = []
alpha = 0.35
axs.set_title('alpha = ' + str(alpha))
Max = 0
Min = 0
for k in range(n_particles):
    x0 = 0.1
    p0 = (k+3.)*1.
    z0 = 0.1
    x = x0*np.ones(n_step)
    p = p0*np.ones(n_step)
    z = z0*np.ones(n_step)

    for i in range(1,n_step):
        x[i] = x[i-1] + h*(1. - 0.5*h*alpha*z[i-1])*p[i-1] - 0.5*h**2.*V_der(x[i-1])
        p[i] = ((1. - 0.5*h*alpha*z[i-1])*p[i-1] - 0.5*h*(V_der(x[i-1]) + V_der(x[i])))/(1. + 0.5*h*alpha*z[i])
        z[i] = z[i-1] + h*L(x[i-1],x[i],z[i-1],z[i],alpha)
    X.append(x)
    P.append(p)
    Z.append(z)

    plt.plot(X[k],P[k])
plt.show()
```

```python

```


# PRAKTIKUM 5
__Solusi Sistem Persamaan Linear (Iteratif)__

*Topik*
1. Metode Jacobi
2. Metode Gauss-Seidel
3. Rekonstruksi Aljabar
4. _Conjugate Gradient_

# 1 Metode Jacobi
Metode Jacobi merupakan salah satu contoh dari metode iteratif. Solusi yang dihasilkan oleh metode iteratif merupakan hampiran dari solusi eksak SPL. Tujuan utama dari metode Jacobi adalah melakukan serangkaian iterasi, sehingga hampiran solusi SPL memenuhi toleransi galat yang ditentukan terhadap solusi eksak SPL.

Dalam notasi matriks, matriks $ A $ pada SPL $AX=B$ dipisah menjadi seperti berikut.

\begin{align*}
A=
\underbrace{\begin{bmatrix}
a_{11}&	   0 &	  0 &	  0\\
	0 &a_{22}&    0 & 	  0\\
	0 &	   0 &a_{33}& 	  0\\
	0 &    0 &    0 &a_{44}\\
\end{bmatrix}}_{D}
+
\underbrace{\begin{bmatrix}
	0 &	   0 &	  0 &	 0 \\
a_{21}&	   0 &	  0 &	 0 \\
a_{31}&a_{32}&	  0 &	 0 \\
a_{41}&a_{42}&a_{43}&	 0 \\
\end{bmatrix}}_{L}
+
\underbrace{\begin{bmatrix}
	0 &a_{12}&a_{13}&a_{14}\\
	0 &    0 &a_{23}&a_{24}\\
	0 &    0 &    0 &a_{34}\\
	0 &    0 &    0 &    0 \\
\end{bmatrix}}_{U}
\end{align*}
Katakanlah $ R=L+U $, maka $ A=D+R $

Formula iterasi metode Jacobi adalah
\begin{equation}
X^{(k)} = D^{-1}\left( B-RX^{(k-1)}\right)
\end{equation}

__Pseudocode__:

```julia
using LinearAlgebra
function jacobi(A, B, Xawal::Array{Int64,1})
    # Definisikan nilai toleransi dan maksimum iterasi yang ditentukan.
    delta = 10^-7;
    maxi = 100;
    flag = 1;
    # Dekomposisi matriks A menjadi A=D+R
    D = Diagonal(diag(A))
    R = A - D;
    # Tentukan nilai tebakan awal dari metode Jacobi
    X = Xawal;
    M = [0 X' NaN];
    # Mulai proses iterasi metode jacobi
    for k = 1:maxi
        Xlama = X;
        X = inv(D)*(B - R*Xlama); # Isi dengan rumus iterasi jacobi
        # Hitung panjang vektor galat iterasi jacobi, jika Xn-Xn-1 atau B-AX telah memenuhi toleransi maka proses iterasi dapat dihentikan.
        err = norm(X-Xlama);
        M = [M; [k X' err] ];
        if err<delta || norm(B-A*X)<delta
            flag = 0;
            break
        end
    end
    err = M[end,end]
    return X, flag, err, M
end
```

### Contoh Soal 1
Diberikan sistem persamaan linear tiga peubah seperti berikut.
\begin{align}
 4x -  y + z &= 7\\
 4x - 8y + z &= -21\\
-2x + y +5z &= 15
\end{align}

Tunjukkan bahwa metode Jacobi menghasilkan solusi yang konvergen menuju solusi eksak SPL tersebut dengan nilai tebakan awal yaitu $ X_0=\begin{pmatrix}
1 & 2 & 2
\end{pmatrix}^T $.

**Solusi:**

```julia
A = [4 -1  1
     4 -8  1
    -2  1  5];
B = [7;-21;15];
Xa = [1,2,2];
X,flag,err,M = jacobi(A,B,Xa)
@show X
@show flag
@show err
M
```

Perambatan Panjang Vektor Galat

```julia
using Plots
```

```julia
# Plot panjang vektor galat.
iter = M[:,1]
err = M[:,end]
p1 = plot(iter, err, yaxis = :log, label = :none)
# Tambahkan title dan label
title!("Plot perambatan panjang vektor galat")
xlabel!("iterasi")
ylabel!("panjang vektor galat")
```

# 2 Metode Gauss-Seidel
Kelemahan dari metode Jacobi adalah proses komputasi membutuhkan jumlah iterasi yang cukup banyak. Dengan demikian, diperlukan modifikasi terhadap metode ini agar jumlah iterasi berkurang. Salah satu metode tersebut adalah metode Gauss-Seidel.

Dalam notasi matriks, matriks $ A $ dipisah menjadi seperti berikut.

\begin{align*}
A=
\underbrace{\begin{bmatrix}
	a_{11}&	   0 &	  0 &	  0\\
	0 &a_{22}&    0 & 	  0\\
	0 &	   0 &a_{33}& 	  0\\
	0 &    0 &    0 &a_{44}\\
	\end{bmatrix}}_{D}
+
\underbrace{\begin{bmatrix}
	0 &	   0 &	  0 &	 0 \\
	a_{21}&	   0 &	  0 &	 0 \\
	a_{31}&a_{32}&	  0 &	 0 \\
	a_{41}&a_{42}&a_{43}&	 0 \\
	\end{bmatrix}}_{L}
+
\underbrace{\begin{bmatrix}
	0 &a_{12}&a_{13}&a_{14}\\
	0 &    0 &a_{23}&a_{24}\\
	0 &    0 &    0 &a_{34}\\
	0 &    0 &    0 &    0 \\
	\end{bmatrix}}_{U}
\end{align*}
Katakanlah $ R=L+D $, maka $ A=R+U $.

Formula iterasi metode Gauss-Seidel, yaitu
\begin{equation}
X^{(k)} = R^{-1}\left( B-UX^{(k-1)}\right)
\end{equation}
__Pseudocode__:

```julia
function gaussSeidel(A,B,Xawal::Array{Int64,1})
    # Definisikan nilai toleransi dan maksimum iterasi yang ditentukan.
    delta = 10^-7;
    maxi = 100;
    flag = 1;
    # Dekomposisi matriks A menjadi A=L+U
    D = Diagonal(diag(A))
    R = tril(A);
    U = triu(A,1);
    # Tentukan nilai tebakan awal dari metode Gauss-Seidel
    X = Xawal;
    M = [0 X' NaN];
    # Mulai proses iterasi metode Gauss-Seidel
    for k = 1:maxi
        Xlama = X;
        X = inv(R)*(B - U*Xlama); # Isi dengan rumus iterasi Gauss-Seidel
        # Hitung panjang vektor galat iterasi jacobi, jika galat telah
        # memenuhi toleransi maka proses iterasi dapat dihentikan.
        err = norm(X-Xlama);
        M = [M; [k X' err] ];
        if err<delta
            flag = 0;
            break
        end
    end
    err = M[end,end]
    return X, flag, err, M
end
```

### Contoh Soal 2
Diberikan sistem persamaan linear tiga peubah seperti berikut.
\begin{align}
 4x -  y + z &= 7\\
 4x - 8y + z &= -21\\
-2x + y +5z &= 15
\end{align}

Tunjukkan bahwa metode Gausss-Seidel menghasilkan solusi yang konvergen menuju solusi eksak SPL tersebut dan lebih cepat dibandingkan metode Jacobi dengan nilai tebakan awal yaitu $ X_0=\begin{pmatrix}
1 & 2 & 2
\end{pmatrix}^T $.

**Solusi:**

```julia
A = [4 -1  1
     4 -8  1
    -2  1  5];
B = [7;-21;15];
Xa = [1,2,2];
X,flag,err,M = gaussSeidel(A,B,Xa)
@show X
@show flag
@show err
M
```

Perambatan Panjang Vektor Galat

```julia
# Plot panjang vektor galat.
iter = M[:,1]
err = M[:,end]
p2= plot(iter, err, yaxis = :log, label = :none)
# Tambahkan title dan label
title!("Plot perambatan panjang vektor galat")
xlabel!("iterasi")
ylabel!("panjang vektor galat")
```

# 3 Metode Rekonstruksi Aljabar

```julia
using LinearAlgebra
function rekons(A,B,Xawal::Array{Int64,1})
    delta = 10^-7;
    maxi = 100;
    flag = 1;
    X = Xawal;
    Xlama = X;
    M = [0 X' NaN];
    for k = 1:maxi
        for i = 1:length(B)
            Xlama = X;
            X = Xlama + A[i,:]*(B[i]-A[i,:]'Xlama)/(A[i,:]'A[i,:])
        end
        err = norm(X-Xlama)
        M = [M; [k X' err] ];
        if err<delta
            flag = 0;
            break
        end
    end
    err = M[end,end]
    return X, flag, err, M
end
```

### Contoh Soal 3
Diberikan sistem persamaan linear tiga peubah seperti berikut.
\begin{align}
 4x -  y + z &= 7\\
 4x - 8y + z &= -21\\
-2x + y +5z &= 15
\end{align}

Tunjukkan bahwa metode rekonstruksi aljabar menghasilkan solusi yang konvergen menuju solusi eksak SPL tersebut dan bandingkan dengan metode Jacobi serta Gauss-Seidel dengan nilai tebakan awal yaitu $ X_0=\begin{pmatrix}
1 & 2 & 2
\end{pmatrix}^T $.

**Solusi:**

```julia
A = [4 -1  1
     4 -8  1
    -2  1  5];
B = [7;-21;15];
Xa = [1,2,2];
X,flag,err,M = rekons(A,B,Xa)
@show X
@show flag
@show err
M
```

```julia
# Plot panjang vektor galat.
iter = M[:,1]
err = M[:,end]
p3 = plot(iter, err, yaxis = :log, label = :none)
# Tambahkan title dan label
title!("Plot perambatan panjang vektor galat")
xlabel!("iterasi")
ylabel!("panjang vektor galat")
```

# 4 Metode Conjugate Gradient

```julia
using LinearAlgebra
function conGrad(A,B,Xawal::Array{Int64,1})
    if ~(isposdef(A) && issymmetric(A))
        error("matriks A harus simetrik definit positif")
    end
    delta = 10^-7;
    maxi = 10000;
    flag = 1;
    X = Xawal;
    r = B-A*X
    d = r
    M = [0 X' NaN];
    for k = 1:maxi
        Xlama = X
        rlama = r
        dlama = d
        X = Xlama + ((rlama'rlama)/(dlama'A*dlama))*dlama
        r = rlama - ((rlama'rlama)/(dlama'A*dlama))*A*dlama
        d = r - ((r'r)/(rlama'rlama))*dlama
        err = norm(X-Xlama)
        M = [M; [k X' err] ];
        if err<delta
            flag = 0;
            break
        end
    end
    err = M[end,end]
    return X, flag, err, M
end
```

### Contoh Soal 4
Diberikan sistem persamaan linear tiga peubah seperti berikut.
\begin{align}
 4x -  y + z &= 7\\
 4x - 8y + z &= -21\\
-2x + y +5z &= 15
\end{align}

Apakah metode _conjugate gradient_ menghasilkan solusi yang konvergen menuju solusi eksak SPL tersebut? Jika iya, gunakan metode _conjugate gradient_ untuk menyelesaikan SPL dengan nilai tebakan awal yaitu $ X_0=\begin{pmatrix}
1 & 2 & 2
\end{pmatrix}^T $. Jika tidak, modifikasi terlebih dulu SPL sehingga dapat diselesaikan menggunakan metode _conjugate gradient_, kemudian selesaikan dengan nilai tebakan awal yaitu $ X_0=\begin{pmatrix}
1 & 2 & 2
\end{pmatrix}^T $.

**Solusi:**

```julia
A = [4 -1  1
     4 -8  1
    -2  1  5];
B = [7;-21;15];
Xa = [1,2,2];
X,flag,err,M = conGrad(A,B,Xa)
```

```julia
Amod = A'A
Bmod = A'B
Xa = [1,2,2];
X,flag,err,M = conGrad(Amod,Bmod,Xa)
@show X
@show flag
@show err
M
```

```julia
# Plot panjang vektor galat.
iter = M[:,1]
err = M[:,end]
p4 = plot(iter, err, yaxis = :log, label = :none)
# Tambahkan title dan label
title!("Plot perambatan panjang vektor galat")
xlabel!("iterasi")
ylabel!("panjang vektor galat")
```

# Soal Latihan
Kerjakan soal berikut pada saat kegiatan praktikum berlangsung.

`Nama: ________`

`NIM: ________`

### Soal 1
Diketahui sistem persamaan linear 4 peubah seperti berikut.

$$\begin{align*}
\begin{pmatrix}
  8 &  2 & -3 &  2 \\
 -2 & -7 & -1 &  3 \\
  1 & -2 &  6 & -2 \\
  2 & -1 & -4 &  9 \\
\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4\\
\end{pmatrix}=
\begin{pmatrix}
6\\-11\\15\\11
\end{pmatrix}
\end{align*}$$

Gunakan metode iterasi Jacobi untuk menyelesaikan SPL tersebut, kemudian gambarkan plot perambatan panjang vektor galatnya.

```julia

```

### Soal 2

Diketahui sistem persamaan linear 4 peubah seperti berikut.

$$\begin{align*}
\begin{pmatrix}
  8 &  2 & -3 &  2 \\
 -2 & -7 & -1 &  3 \\
  1 & -2 &  6 & -2 \\
  2 & -1 & -4 &  9 \\
\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4\\
\end{pmatrix}=
\begin{pmatrix}
6\\-11\\15\\11
\end{pmatrix}
\end{align*}$$

Gunakan metode iterasi Gauss-Seidel untuk menyelesaikan SPL tersebut, kemudian gambarkan plot perambatan panjang vektor galatnya.

```julia

```

### Soal 3

Diketahui sistem persamaan linear 4 peubah seperti berikut.

$$\begin{align*}
\begin{pmatrix}
  8 &  2 & -3 &  2 \\
 -2 & -7 & -1 &  3 \\
  1 & -2 &  6 & -2 \\
  2 & -1 & -4 &  9 \\
\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4\\
\end{pmatrix}=
\begin{pmatrix}
6\\-11\\15\\11
\end{pmatrix}
\end{align*}$$

Gunakan metode iterasi rekonstruksi aljabar untuk menyelesaikan SPL tersebut, kemudian gambarkan plot perambatan panjang vektor galatnya.

```julia

```

### Soal 4

Diketahui sistem persamaan linear 4 peubah seperti berikut.

$$\begin{align*}
\begin{pmatrix}
  8 &  2 & -3 &  2 \\
 -2 & -7 & -1 &  3 \\
  1 & -2 &  6 & -2 \\
  2 & -1 & -4 &  9 \\
\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4\\
\end{pmatrix}=
\begin{pmatrix}
6\\-11\\15\\11
\end{pmatrix}
\end{align*}$$

Gunakan metode _conjugate gradient_ untuk menyelesaikan SPL tersebut, kemudian gambarkan plot perambatan panjang vektor galatnya.

```julia

```


```python
import numpy as np
import pandas as pd
import linearsolve as ls
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 13: Introduction to Real Business Cycle Modeling

Real business cycle (RBC) models are extensions of the stochastic Solow model. RBC models replace the ad hoc assumption of a constant saving rate in the Solow model with the solution to an intertemporal utility maximization problem that gives rise to a variable saving rate. RBC models also often feature some sort of household labor-leisure tradeoff that produces endogenous labor varation.

In this notebook, we'll consider a baseline RBC model that does not have labor. We'll use the model to compute impulse responses to a one percent shock to TFP.

## The Baseline RBC Model without Labor

The equilibrium conditions for the RBC model without labor are:

\begin{align}
\frac{1}{C_t} & = \beta E_t \left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1} +1-\delta }{C_{t+1}}\right]\\
K_{t+1} & = I_t + (1-\delta) K_t\\
Y_t & = A_t K_t^{\alpha}\\
Y_t & = C_t + I_t\\
\log A_{t+1} & = \rho \log A_t + \epsilon_{t+1}
\end{align}

where $\epsilon_{t+1} \sim \mathcal{N}(0,\sigma^2)$.

The objective is use `linearsolve` to simulate impulse responses to a TFP shock using the following parameter values for the simulation:

| $$\rho$$ | $$\sigma$$ | $$\beta$$ | $$\alpha$$ | $$\delta $$ | $$T$$ |
|----------|------------|-----------|------------|-------------|-------|
| 0.75     | 0.006      | 0.99      | 0.35       |  0.025      | 26    |

## Model Preparation

Before proceding, let's recast the model in the form required for `linearsolve`. Write the model with all variables moved to the lefthand side of the equations and dropping the expecations operator $E_t$ and the exogenous shock $\epsilon_{t+1}$:

\begin{align}
0 & = \beta\left[\frac{\alpha A_{t+1}K_{t+1}^{\alpha-1} +1-\delta }{C_{t+1}}\right] - \frac{1}{C_t}\\
0 & = A_t K_t^{\alpha} - Y_t\\
0 & = I_t + (1-\delta) K_t - K_{t+1}\\
0 & = C_t + I_t - Y_t\\
0 & = \rho \log A_t - \log A_{t+1}
\end{align}

Remember, capital and TFP are called *state variables* because they're $t+1$ values are predetermined. Output, consumption, and investment are called a *costate* or *control* variables. Note that the model as 5 equations in 5 endogenous variables.

## Initialization, Approximation, and Solution

The next several cells initialize the model in `linearsolve` and then approximate and solve it.

```python
# Create a variable called 'parameters' that stores the model parameter values in a Pandas Series
parameters = pd.Series(dtype=float)
parameters['rho'] = .75
parameters['beta'] = 0.99
parameters['alpha'] = 0.35
parameters['delta'] = 0.025

# Print the model's parameters
print(parameters)
```

    rho      0.750
    beta     0.990
    alpha    0.350
    delta    0.025
    dtype: float64

```python
# Create a variable called 'sigma' that stores the value of sigma
sigma = 0.006
```

```python
# Create variable called 'var_names' that stores the variable names in a list with state variables ordered first
var_names = ['a','k','y','c','i']

# Create variable called 'shock_names' that stores an exogenous shock name for each state variable.
shock_names = ['e_a','e_k']
```

```python
# Define a function that evaluates the equilibrium conditions of the model solved for zero. PROVIDED
def equilibrium_equations(variables_forward,variables_current,parameters):

    # Parameters. PROVIDED
    p = parameters

    # Current variables. PROVIDED
    cur = variables_current

    # Forward variables. PROVIDED
    fwd = variables_forward

    # Euler equation
    euler_equation = p.beta*(p.alpha*fwd.a*fwd.k**(p.alpha-1)+1-p.delta)/fwd.c - 1/cur.c

    # Production function
    production_function = cur.a*cur.k**p.alpha - cur.y

    # Capital evolution
    capital_evolution = cur.i + (1 - p.delta)*cur.k - fwd.k

    # Market clearing
    market_clearing = cur.c+cur.i - cur.y

    # Exogenous tfp
    tfp_process = p.rho*np.log(cur.a) - np.log(fwd.a)

    # Stack equilibrium conditions into a numpy array
    return np.array([
        euler_equation,
        production_function,
        capital_evolution,
        market_clearing,
        tfp_process
        ])
```

Next, initialize the model using `ls.model` which takes the following required arguments:

* `equations`
* `n_states`
* `var_names`
* `shock_names`
* `parameters`

```python
# Initialize the model into a variable named 'rbc_model'
rbc_model = ls.model(equations = equilibrium_equations,
                 n_states=2,
                 var_names=var_names,
                 shock_names=shock_names,
                 parameters=parameters)
```

```python
# Compute the steady state numerically using .compute_ss() method of rbc_model
guess = [1,4,1,1,1]
rbc_model.compute_ss(guess)

# Print the computed steady state
print(rbc_model.ss)
```

    a     1.000000
    k    34.398226
    y     3.449750
    c     2.589794
    i     0.859956
    dtype: float64

```python
# Find the log-linear approximation around the non-stochastic steady state and solve using .approximate_and_solve() method of rbc_model
rbc_model.approximate_and_solve()
```

### Impulse Responses

Compute a 26 period impulse responses of the model's variables to a 0.01 unit shock to TFP in period 5.

```python
# Compute impulse responses
rbc_model.impulse(T=26,t0=5,shocks=[0.01,0])

# Print the first 10 rows of the computed impulse responses to the TFP shock
print(rbc_model.irs['e_a'].head(10))
```

        e_a         a         k         y         c         i
    0  0.00  0.000000  0.000000  0.000000  0.000000  0.000000
    1  0.00  0.000000  0.000000  0.000000  0.000000  0.000000
    2  0.00  0.000000  0.000000  0.000000  0.000000  0.000000
    3  0.00  0.000000  0.000000  0.000000  0.000000  0.000000
    4  0.00  0.000000  0.000000  0.000000  0.000000  0.000000
    5  0.01  0.010000  0.000000  0.010000  0.001253  0.036342
    6  0.00  0.007500  0.000909  0.007818  0.001493  0.026865
    7  0.00  0.005625  0.001557  0.006170  0.001654  0.019772
    8  0.00  0.004219  0.002013  0.004923  0.001755  0.014465
    9  0.00  0.003164  0.002324  0.003978  0.001812  0.010499

Construct a $2\times2$ grid of plots of simulated TFP, output, consumption, and investment. Be sure to multiply simulated values by 100 so that vertical axis units are in "percent deviation from steady state."

```python
# Create figure. PROVIDED
fig = plt.figure(figsize=(12,8))

# Create upper-left axis. PROVIDED
ax = fig.add_subplot(2,2,1)
ax.plot(rbc_model.irs['e_a']['a']*100,'b',lw=5,alpha=0.75)
ax.set_title('TFP')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.2,1.2])
ax.grid()

# Create upper-right axis. PROVIDED
ax = fig.add_subplot(2,2,2)
ax.plot(rbc_model.irs['e_a']['y']*100,'b',lw=5,alpha=0.75)
ax.set_title('Output')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.2,1.2])
ax.grid()

# Create lower-left axis. PROVIDED
ax = fig.add_subplot(2,2,3)
ax.plot(rbc_model.irs['e_a']['c']*100,'b',lw=5,alpha=0.75)
ax.set_title('Consumption')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-0.05,0.30])
ax.grid()

# Create lower-right axis. PROVIDED
ax = fig.add_subplot(2,2,4)
ax.plot(rbc_model.irs['e_a']['i']*100,'b',lw=5,alpha=0.75)
ax.set_title('Investment')
ax.set_ylabel('% dev from steady state')
ax.set_ylim([-1,6])
ax.grid()

fig.tight_layout()
```


Text provided under a Creative Commons Attribution license, CC-BY. All code is made available under the FSF-approved MIT license. (c) Yves Dubief, 2016. NSF for support via NSF-CBET award #1258697.
The following cell should always be the first coding cell of your python notebooks

```python
%matplotlib inline
# plots graphs within the notebook
%config InlineBackend.figure_format='svg' # not sure what this does, may be default images to svg format

from IPython.display import display,Image, Latex

from IPython.core.display import HTML
def header(text):
    raw_html = '<h4>' + str(text) + '</h4>'
    return raw_html

def box(text):
    raw_html = '<div style="border:1px dotted black;padding:2em;">'+str(text)+'</div>'
    return HTML(raw_html)

def nobox(text):
    raw_html = '<p>'+str(text)+'</p>'
    return HTML(raw_html)

def addContent(raw_html):
    global htmlContent
    htmlContent += raw_html

class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return ''.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in
        IPython Notebook. """

    def _repr_html_(self):
        html = ["<table>"]
        for row in self:
            html.append("<tr>")

            for col in row:
                html.append("<td>{0}</td>".format(col))

            html.append("</tr>")
        html.append("</table>")
        return ''.join(html)

font = {'family' : 'serif',
        'color'  : 'black',
        'weight' : 'normal',
        'size'   : 18,
        }

from IPython.display import clear_output

import SchemDraw as schem
import SchemDraw.elements as e

import matplotlib.pyplot as plt
import numpy as np

from scipy.constants.constants import C2K
from scipy.constants.constants import K2C
from scipy.constants.constants import F2K
from scipy.constants.constants import K2F
from scipy.constants.constants import C2F
from scipy.constants.constants import F2C
```

```python
student_id = raw_input('Please enter your NETID (e.g. ydubief)')
```

    Please enter your NETID (e.g. ydubief)ydubief

```python
print(student_id)
```

    ydubief

 Heat loss through a single-pane window

Consider a 0.8 m high and 1.5 m wide window with a thickness of 8 mm and a thermal conductivity of $k=0.78$ W/m.K. The room maintained at 20$^\circ$ C and the outside is at -10$^\circ$ C. The inner and outer convection coefficients are $h_1=10$ W/m$^2$.K and $h_2=40$ W/m$^2$.K

 Draw the thermal circuit of the problem.

 Compute the heat transfer through the window.

 Plot the temperature distribution across the window glass.

```python
PDF('figures/window-single-pane-fig.pdf',size=(600,500))
```

####Parameters

```python
H = 0.8 #m Height
W = 1.5 #m Width
t = 8e-3 #m window thickness

A = H*W #m^2 window surface area
k_glass = 0.78 #W/m.K thermal conductivity of glass

T_inf_in = 20 #C
T_inf_out = -10 #C
h_in = 10.
h_out = 40.
```

```python

from Libraries import HT_thermal_resistance as res
from Libraries import thermodynamics as thermo

Rth = []
Rth.append(res.Resistance('$R_{conv.0}$'))
Rth.append(res.Resistance('$R_{cond.0}$'))
Rth.append(res.Resistance('$R_{conv.1}$'))
d = schem.Drawing()
d.add( e.DOT, label='$T_{\infty.in}$')
d.add( e.RES, d='right', label=Rth[0].name )
d.add( e.DOT, label='$T_{in}$')
d.add( e.RES, d='right', label=Rth[1].name )
d.add( e.DOT, label='$T_{out}$')
d.add( e.RES, d='right', label=Rth[2].name )
d.add( e.DOT, label='$T_{\infty.out}$')
d.draw()

#d.save( 'testschematic.eps' )
```

The heat transfer across the window is

$$
q=\frac{1}{R_{tot}}(T_{\infty.in}-T_{\infty.out})
$$

where $R_tot$ is the sum of thermal resistance in the above thermal resistance circuit. Due to conservation of energy, the following equations also apply:
$$
q=\frac{1}{R_{conv.0}}(T_{\infty.in}-T_{in})\\
q=\frac{1}{R_{cond.0}}(T_{in}-T_{out})\\
q=\frac{1}{R_{conv.1}}(T_{out}-T_{\infty.out})\\
$$
which is useful to determine the temperature distribution.

```python
# definition of thermal resistances
Rth[0].convection(h_in,A)
Rth[1].conduction('plane',k_glass,t,0.,A)
Rth[2].convection(h_out,A)
# compute total resistance
Rtotal = res.serial_sum(Rth,0,2)
print('Total thermal resistance: %3.3f' %Rtotal)
q = (T_inf_in-T_inf_out)/Rtotal
print('Heat transfer: %4.2f W' %q)
T_in = T_inf_in-q*Rth[0].R
print('Surface temperature of the window (inside): %3.1f F' %C2F(T_in))
T_out = T_in-q*Rth[1].R
print('Surface temperature of the window (inside): %3.1f F' %C2F(T_out))

```

    Total thermal resistance: 0.113
    Heat transfer: 266.16 W
    Surface temperature of the window (inside): 28.1 F
    Surface temperature of the window (inside): 24.0 F

```python
from __future__ import division
import sympy as sym
from sympy import *
var = Rth[0].name
print(var[1:-1])
R = symbols(var[1:-1])
q = symbols('q')
#f, g, h = map(Function, 'fgh')
eq = Eq(q,1/R)
eq
```

    R_{conv.0}

    q == 1/R_{conv.0}

```python
import SchemDraw as schem
import SchemDraw.elements as e

from Libraries import HT_thermal_resistance as res
from Libraries import thermodynamics as thermo

L = 0.8
W = 1.5

A = L*W
h = 8.e-3
h_0 =10.
h_1 = 40.
k_glass = 0.78
T_inf_in = 20.
T_inf_out = -10.
Rth = []
Rth.append(res.Resistance('$R_{conv,0}$'))
Rth.append(res.Resistance('$R_{cond,0}$'))
Rth.append(res.Resistance('$R_{cond,1}$'))
Rth.append(res.Resistance('$R_{cond,2}$'))
Rth.append(res.Resistance('$R_{conv,1}$'))
d = schem.Drawing()
d.add( e.DOT, label='$T_{\infty,in}$')
d.add( e.RES, d='right', label=Rth[0].name )
d.add( e.DOT, label='$T_{in}$')
d.add( e.RES, d='right', label=Rth[1].name )
d.add( e.RES, d='right', label=Rth[2].name )
d.add( e.RES, d='right', label=Rth[3].name )
d.add( e.DOT, label='$T_{out}$')
d.add( e.RES, d='right', label=Rth[4].name )
d.add( e.DOT, label='$T_{\infty,out}$')
d.draw()
#d.save( 'testschematic.eps' )
```

```python
import numpy as np
from scipy.constants.constants import C2K
from scipy.constants.constants import K2C
from scipy.constants.constants import F2K
from scipy.constants.constants import K2F
gas = thermo.Fluid('krypton')
T_f = 0.5*(T_inf_in+T_inf_out)
print(T_f)
gas.get_properties(C2K(3.9))
k_gas = gas.k
h_gas = 8e-3
Rth[0].convection(h_0,A)
Rth[1].conduction('plane',k_glass,h,0.,A)
Rth[2].conduction('plane',k_gas,h_gas,0.,A)
Rth[3].conduction('plane',k_glass,h,0.,A)
Rth[4].convection(h_1,A)

Rtotal = res.serial_sum(Rth,0,4)
print(Rtotal)
q = (T_inf_in-T_inf_out)/Rtotal
T_in = T_inf_in-q*Rth[0].R
T_0 = T_in - q*Rth[1].R
T_1 = T_0 - q*Rth[2].R
print(T_in,T_0,T_1,0.5*(T_0+T_1))
print(q)
```

    5.0
    0.883004391282
    (17.168757002021742, 16.878373104793202, -9.0018053532768931, 3.9382838757581542)
    33.9749159757

```python

from Libraries import HT_natural_convection_enclosure as ncconvenc
t = 0.008
H = 0.05
T_f = 0.5*(T_inf_in+T_inf_out)
air = thermo.Fluid('air')
air.get_properties(C2K(T_f))
Ra_H = ncconvenc.Ra(9.81,air.beta,T_inf_in-T_inf_out,H,air.nu,air.alpha)
Nu = ncconvenc.Nu_vertical_enclosure(Ra_H,air.Pr,L,H)
print(Ra_H,Nu)
h = air.k*Nu/H
print(h)
Rth = []
Rth.append(res.Resistance('$R_{conv,0}$'))
Rth.append(res.Resistance('$R_{cond,0}$'))
Rth.append(res.Resistance('$R_{conv,0}$'))
Rth.append(res.Resistance('$R_{cond,1}$'))
Rth.append(res.Resistance('$R_{conv,1}$'))
d = schem.Drawing()
d.add( e.DOT, label='$T_{\infty,in}$')
d.add( e.RES, d='right', label=Rth[0].name )
d.add( e.DOT, label='$T_{in}$')
d.add( e.RES, d='right', label=Rth[1].name )
d.add( e.RES, d='right', label=Rth[2].name )
d.add( e.RES, d='right', label=Rth[3].name )
d.add( e.DOT, label='$T_{out}$')
d.add( e.RES, d='right', label=Rth[4].name )
d.add( e.DOT, label='$T_{\infty,out}$')
d.draw()
Rth[0].convection(h_0,A)
Rth[1].conduction('plane',k_glass,t,0.,A)
Rth[2].convection(h,A)
Rth[3].conduction('plane',k_glass,t,0.,A)
Rth[4].convection(h_1,A)

Rtotal = res.serial_sum(Rth,0,4)
print(Rtotal)
q = (T_inf_in-T_inf_out)/Rtotal
T_in = T_inf_in-q*Rth[0].R
T_0 = T_in - q*Rth[1].R
T_1 = T_0 - q*Rth[2].R
print(T_in,T_0,T_1,0.5*(T_0+T_1))
print(q)
```

    ok
    (497505.78966666822, 4.8354664449585139)
    2.38388495736
    0.470830125339
    (14.690229308927584, 14.145637443176568, -8.1279654614808798, 3.0088359908478441)
    63.7172482929

```python
array = np.genfromtxt('Libraries/Tables/Argon1atm.csv', delimiter=',', skip_header = 1, unpack=True, dtype=float)
```

```python

```

```python

```


# Neural Model - Binaural Integration

```python
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:60% !important; }</style>"))
```

<style>.container { width:60% !important; }</style>

In this notebook we present a first implementation of a neural model for sound source elevation estimation based on the computational HRTF model.
We use $binaural = \frac{Left}{Left + Right} $ an integration stage since it is more biologically plausible than the regular integration $\frac{Left}{Right}$.

To achieve similar responses we apply the same dynamical adaptation of the decay rate in the $r_{ipsi}$ neurons as demonstrated before (see Notebook *Neural Model - Gaussian Filtering*), where

\begin{equation}
 \tau \dot r^{ipsi}_\omega = - \alpha \cdot r_\omega + (\beta - r_\omega) \cdot {EX}_{\omega} - \kappa \cdot r_\omega \cdot {In}_{\omega}
\end{equation}

is extended to

\begin{equation}
 \tau \dot r^{ipsi}_\omega = - \alpha \cdot r_\omega \cdot {EX}_{\omega}  + (\beta - r_\omega) \cdot {EX}_{\omega} - \kappa \cdot r_\omega \cdot {In}_{\omega}
\end{equation}

```python
# Import necessary modules
from pathlib import Path
from src.data import generateData
from src.features import helpers as hp
from src.visualization import helpers as hpVis
import numpy as np
import matplotlib.pyplot as plt
import os
from IPython.display import clear_output
from scipy.ndimage import gaussian_filter1d,convolve1d

ROOT = Path(os.getcwd()).resolve().parents[0]

SOUND_FILES = ROOT / 'data/raw/sound_samples/'
# create a list of the sound files
SOUND_FILES = list(SOUND_FILES.glob('**/*.wav'))
```

```python
############################################################
#                  Simulation functions                    #
############################################################

tau = 0.005

# Defines the output transfer function of q_A_SC
def out_thres(q,threshold = 0.0,slope=1):
    return np.minimum(np.maximum((q-threshold)*slope,0),1)

# Defines the sigmoidal transfer function
def out_sigmoid(x,slope =5,operation_point=0.0):
#     return 1 / (1 + np.exp(-(x-working_point)*slope) )
    tmp = (x-operation_point)*slope
    return (2 / (1 + np.exp(-4*tmp**2)))-1

#define a gauss function
def gauss(x,mean,sigma):
    if sigma == 0.0:
        return np.zeros(x.shape)
    else:
        tmp = np.exp(-(x-mean)**2 /( 2 * sigma**2 ))
        return tmp/np.max(tmp)

# define the ODE for inhibitory input neurons
def ode_p_in(p ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1.0
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha=1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_p  = -alpha *p  + (beta -p )*excitatory_in

    return d_p /tau

# define the ODE for gaussian filter neurons
def ode_r_in(r,excitatory_in,inhibitory_in):
    # tau defines how fast the membrane potential builds up
#     tau = 1.0
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 200
    # gamma defines the subtractive influence of the inhibitory input
    gamma = 0.0
    # kappa defines the divisive influence of the inhibitory input
    kappa = 200

    # calculate the change of r_Alearn
    d_r  = -alpha  * r * excitatory_in  + (beta -r ) * excitatory_in  - (gamma  + kappa  * r ) * inhibitory_in

    return d_r / tau

# define the ODE for neuron p_sum
def ode_p_sum(p ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_p  = -alpha *p  + (beta -p )*excitatory_in

    return d_p /tau

# define the ODE for integration neurons
def ode_r(r,excitatory_in,inhibitory_in=0,gaba_ex=0.0,gaba_in=0.0):
    # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 2
    # gamma defines the subtractive influence of the inhibitory input
    gamma = 0
    # kappa defines the divisive influence of the inhibitory input
    kappa = 1

    excitatory_in *= (1-gaba_ex)
    inhibitory_in *= (1-gaba_in)

    # calculate the change of r_Alearn
    d_r  = -alpha  * r * excitatory_in   + (beta -r ) * excitatory_in  - (gamma  + kappa  * r ) * inhibitory_in

    return d_r /tau

# define the ODE for read out neurons
def ode_q_sum(q ,excitatory_in):
 # tau defines how fast the membrane potential builds up
#     tau = 1
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 1

    # calculate the change of r_Alearn
    d_q  = -alpha *q  + (beta -q )*excitatory_in

    return d_q /tau

# Defines the ODE for the p_gaba neuron which defines the strength of the modulation (inhibition) of the inputs of the r neuron
def ode_gaba(p_gaba, r_gbc):
    # get the globally defined parameters
    # alpha defines the decay rate of the membrane potential but also the value to which it saturates (implicitly)
    alpha= 1
    # beta defines the upper limit of the membrane potential
    beta= 1

    tau = 0.01

    d_p_pc =   -alpha*p_gaba +(beta - p_gaba)*r_gbc
    return  d_p_pc  / tau

```

### Parameters

```python
########################################################################
######################## Set parameters ################################
########################################################################
azimuth = 12
snr = 0.0
freq_bands = 128
participant_number = 1

normalize = False
time_window = 0.1  # time window in sec

# filtering parameters
normalization_type = 'sum_1'
sigma_smoothing = 0
sigma_gauss_norm = 1

# use the mean subtracted map as the learned map
mean_subtracted_map = True

ear = 'ipsi'

elevations = np.arange(0, 25, 1)

############################################################
#                  Simulation parameter                    #
############################################################

# Time step
dt = 0.0001 # ->

# Solve differential equation from time 0 to time T
T = 0.3#
# Descretize time into equal steps
ts = np.linspace (0 , T , int ( T / dt )+1)

```

### Simulation

```python
#generate inputs
psd_all_c, psd_all_i = generateData.create_data(freq_bands, participant_number, snr, normalize, azimuth, time_window)

# Take only given elevations
input_c = psd_all_c[:, elevations, :]
input_i = psd_all_i[:, elevations, :]

# normalize inputs over frequencies
input_c = input_c / input_c.sum(2)[:,:,np.newaxis]
input_i = input_i / input_i.sum(2)[:,:,np.newaxis]

# Define neurons
# sounds_types = np.array([0,4,6,9,10])
# sounds_types = np.array([0])
sounds_types = np.arange(0,20)

n_sounds = len(sounds_types)
# elevations_angles = np.array([0])
elevations_angles = np.arange(0,25,1)
n_elevations = len(elevations_angles)

sigma = 3

p_in_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
p_in_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

r_in_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
r_in_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

p_sum_i = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
p_sum_c = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

r_ipsi = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))
p_all = np.zeros((n_sounds,n_elevations,len(ts),freq_bands))

q = np.zeros((n_sounds,n_elevations,len(ts),len(elevations)))

x_kernel = np.arange(freq_bands)

gauss_kernel = gauss(np.arange(-4*sigma,4*sigma),0,sigma)

gauss_kernel_gaba = gauss(np.arange(-4*sigma,4*sigma),0,2)

for i_sound,sound in enumerate(sounds_types):
    for i_ele,ele in enumerate(elevations_angles):
        in_i = input_i[sound,ele]
        in_c = input_c[sound,ele]

        # since the input does not change over time. We can do this calculation ouside the loop
        excitatory_in_p_i = convolve1d(out_thres(in_i),weights=gauss_kernel,axis=0,mode='reflect')
        excitatory_in_p_c = convolve1d(out_thres(in_c),weights=gauss_kernel,axis=0,mode='reflect')

        for t in range(0,len(ts)-1):

            ## p_In_ipsi neuron
            # feed inputs ipsi inhibition
            p_in_i[i_sound,i_ele, t+1,:] = p_in_i[i_sound,i_ele, t,:] + dt* ode_p_in(p_in_i[i_sound,i_ele,t,:],excitatory_in_p_i)

            ## r_In_ipsi neuron
            excitatory_in = out_thres(in_i)
            inhibitory_in = out_thres(p_in_i[i_sound,i_ele, t,:])
            r_in_i[i_sound,i_ele, t+1,:] = r_in_i[i_sound,i_ele, t,:] + dt* ode_r_in(r_in_i[i_sound,i_ele,t,:],excitatory_in,inhibitory_in)

            ## p_In_contra neuron
            # feed inputs ipsi inhibition
            p_in_c[i_sound,i_ele, t+1,:] = p_in_c[i_sound,i_ele, t,:] + dt* ode_p_in(p_in_c[i_sound,i_ele,t,:],excitatory_in_p_c)

            ## r_In_contra neuron
            excitatory_in = out_thres(in_c)
            inhibitory_in = out_thres(p_in_c[i_sound,i_ele, t,:])
            r_in_c[i_sound,i_ele, t+1,:] = r_in_c[i_sound,i_ele, t,:] + dt* ode_r_in(r_in_c[i_sound,i_ele,t,:],excitatory_in,inhibitory_in)

            ## p_sum neurons
            excitatory_in = out_thres(r_in_i[i_sound,i_ele, t,:])
            p_sum_i[i_sound,i_ele, t+1,:] = p_sum_i[i_sound,i_ele, t,:] + dt* ode_p_sum( p_sum_i[i_sound,i_ele, t,:],excitatory_in)

            excitatory_in = out_thres(r_in_c[i_sound,i_ele, t,:])
            p_sum_c[i_sound,i_ele, t+1,:] = p_sum_c[i_sound,i_ele, t,:] + dt* ode_p_sum(p_sum_c[i_sound,i_ele, t,:],excitatory_in)

            ## r_ipsi neuron
#             gaba_ex = p_all[i_sound,i_ele, t,:]*1.5
#             gaba_in = p_all[i_sound,i_ele, t,:]*1.5

            gaba_ex = convolve1d(p_all[i_sound,i_ele, t,:],weights=gauss_kernel_gaba,axis=0,mode='reflect')
            gaba_in = convolve1d(p_all[i_sound,i_ele, t,:],weights=gauss_kernel_gaba,axis=0,mode='reflect')

            gaba_ex *= 0.7 # 0.9
            gaba_in *= 0.3

            excitatory_in = out_thres(r_in_i[i_sound,i_ele, t,:])
            inhibitory_in = out_thres(p_sum_c[i_sound,i_ele, t,:]) # + out_thres(p_sum_i[i_sound,i_ele, t,:])
            r_ipsi[i_sound,i_ele, t+1,:] = r_ipsi[i_sound,i_ele, t,:] + dt* ode_r(r_ipsi[i_sound,i_ele,t,:],excitatory_in,inhibitory_in,gaba_ex=gaba_ex,gaba_in=gaba_in)

            p_all[i_sound,i_ele, t+1,:] = p_all[i_sound,i_ele, t,:] + dt * ode_gaba(p_all[i_sound,i_ele, t,:],out_thres(r_ipsi[i_sound,i_ele, t+1,:]))

            if t%500 == 0:
                clear_output(wait=True)
                print('Sound No: '+str(i_sound+1)+' of '+str(n_sounds)+'.\n  -> Elevation : '+str(i_ele+1)+' of '+str(n_elevations)+'.\n  -> Time : '+str(t))
```

```python
plt.pcolormesh(p_all[0,10].T)
plt.colorbar()
```

```python
plt.plot(p_all[0,10,:,:])
plt.show()
```

```python
plt.pcolormesh(r_ipsi[0,10,:].T)
# plt.plot(r_ipsi[0,10,:,:])
plt.colorbar()
```

```python
plt.plot(r_ipsi[0,10,:,:])
plt.show()
```

## Single Sound Comparison of Different Model Stages

```python
fig = plt.figure(figsize=(10,10))

axs = fig.subplots(nrows=5,ncols=2,squeeze=False)

# sounds_types = np.arange(0,20)

sound = 0
filter_sigma = 1

in_i = np.squeeze(input_i[sounds_types[sound],:,:])
in_c = np.squeeze(input_c[sounds_types[sound],:,:])

ax = axs[0,0]
ax.set_title('Ipsi')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[0,1]
ax.set_title('Gaussian Model')
tmp1 = in_i
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Input Data')
fig.colorbar(a,ax=ax)

ax = axs[1,0]
tmp1 = np.squeeze(p_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('P Neuron')
fig.colorbar(a,ax=ax)

ax = axs[1,1]
tmp1 = in_i
tmp1 = gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
ax.set_ylabel('Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[2,0]
tmp1 = np.squeeze(r_in_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('R_in Neuron')
fig.colorbar(a,ax=ax)

ax = axs[2,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1)
ax.set_ylabel('S/Gauss(S)')
fig.colorbar(a,ax=ax)

ax = axs[3,0]
tmp1 = np.squeeze(p_sum_i[sound,:,-1:,:])
a = ax.pcolormesh(tmp1)
ax.set_ylabel('P_sum_i Neuron')
fig.colorbar(a,ax=ax)

ax = axs[3,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh((tmp1+tmp2))
ax.set_ylabel('SUM ')
fig.colorbar(a,ax=ax)

ax = axs[4,0]
tmp1 = (np.squeeze(r_ipsi[sound,:,-1:,:]))
a = ax.pcolormesh(tmp1)
ax.set_ylabel('R_ipsi Neuron')
fig.colorbar(a,ax=ax)

ax = axs[4,1]
tmp1 = in_i
tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=filter_sigma, mode='nearest', axis=1)

tmp2 = in_c
tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=filter_sigma, mode='nearest', axis=1)
a = ax.pcolormesh(tmp1/(tmp1+tmp2))
ax.set_ylabel('R_in Neuron')
fig.colorbar(a,ax=ax)

plt.show()

```

## Comparison of all Sounds

```python
fig = plt.figure(figsize=(10,10))

axs = fig.subplots(nrows=len(sounds_types),ncols=2,squeeze=False)

for i_sounds,sound in enumerate(sounds_types):

    in_i = np.squeeze(input_i[sound,:,:])
    in_c = np.squeeze(input_c[sound,:,:])

    ax = axs[i_sounds,0]
    tmp1 = np.squeeze(r_ipsi[i_sounds,:,-1:,:])
    a = ax.pcolormesh(tmp1)
    if i_sounds == 0:
        ax.set_title('R_ipsi Neuron')
    fig.colorbar(a,ax=ax)

    ax = axs[i_sounds,1]
    if i_sounds == 0:
        ax.set_title('Binaural Integration')
    tmp1 = in_i
    tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=1, mode='nearest', axis=1)

    tmp2 = in_c
    tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=1, mode='nearest', axis=1)
    a = ax.pcolormesh(tmp1/(tmp1+tmp2))
#     fig.colorbar(a,ax=ax)

plt.show()
```

```python

```

## Averaged Map

```python
all_maps = np.zeros((len(sounds_types),len(elevations),freq_bands))
all_maps_neural = np.zeros((len(sounds_types),len(elevations),freq_bands))

for i_sounds,sound in enumerate(sounds_types):

    in_i = np.squeeze(input_i[sound,:,:])
    in_c = np.squeeze(input_c[sound,:,:])

    # Neural Map
    tmp1 = np.squeeze(r_ipsi[i_sounds,:,-1:,:])
    all_maps_neural[i_sounds,:,:] = tmp1

    # Comp Map
    tmp1 = in_i
    tmp1 = tmp1/gaussian_filter1d(tmp1, sigma=1, mode='nearest', axis=1)

    tmp2 = in_c
    tmp2 = tmp2/gaussian_filter1d(tmp2, sigma=1, mode='nearest', axis=1)
    tmp3 = tmp1/(tmp1+tmp2)
    all_maps[i_sounds,:,:] = tmp3

fig = plt.figure(figsize=(10,5))

axs = fig.subplots(nrows=1,ncols=2,squeeze=False)

ax = axs[0,0]
ax.set_title('Neural Averaged Map')
tmp = np.mean(all_maps_neural,axis=0)
c = ax.pcolormesh(tmp)
plt.colorbar(c)

ax = axs[0,1]
ax.set_title('Comp. Averaged Map')
tmp = np.mean(all_maps,axis=0)
c = ax.pcolormesh(tmp)
plt.colorbar(c)

plt.show()
```

```python

```

```python

```

notebooks/Neural Model - GABA Adaptation.ipynb


```python
import sympy
from sympy import symbols , solve

x    = symbols('x')
expr = x -4 - 2
sol = solve(expr)
sol
```

    [6]

```python
num=sol[0]
num
```

$\displaystyle 6$

```python
from sympy import symbols , solve, Eq
y    = symbols('y')

eq1 = Eq(y + 3 + 8, 0)
sol = solve(eq1)
sol

```

    [-11]

```python
y    = symbols('x')
eq1  = Eq(3*x**2 - 5*x + 6, 0)
sol = solve(eq1,dict=True)
sol
```

    [{x: 5/6 - sqrt(47)*I/6}, {x: 5/6 + sqrt(47)*I/6}]

```python
from sympy import limit

for x in range(0,60):
    fx_nyc_stay_at_home  = limit(-39*x+5078,x,y)
    fx_nyc_face_covering = limit(-106*x+3304,x,y)
    fx_italy_normal      = limit(5006*x+4672,x,y)
    print(fx_nyc_stay_at_home,
          fx_nyc_face_covering,
          fx_italy_normal )

stay_at_home = 5078 / 4883
print(1/stay_at_home)
print(2774/3304)

stay_at_home  = 0.961599054745963
face_covering = 0.839588377723971
italy_normal = 6.35
italy_normal
```

    5078 3304 4672
    5039 3198 9678
    5000 3092 14684
    4961 2986 19690
    4922 2880 24696
    4883 2774 29702
    4844 2668 34708
    4805 2562 39714
    4766 2456 44720
    4727 2350 49726
    4688 2244 54732
    4649 2138 59738
    4610 2032 64744
    4571 1926 69750
    4532 1820 74756
    4493 1714 79762
    4454 1608 84768
    4415 1502 89774
    4376 1396 94780
    4337 1290 99786
    4298 1184 104792
    4259 1078 109798
    4220 972 114804
    4181 866 119810
    4142 760 124816
    4103 654 129822
    4064 548 134828
    4025 442 139834
    3986 336 144840
    3947 230 149846
    3908 124 154852
    3869 18 159858
    3830 -88 164864
    3791 -194 169870
    3752 -300 174876
    3713 -406 179882
    3674 -512 184888
    3635 -618 189894
    3596 -724 194900
    3557 -830 199906
    3518 -936 204912
    3479 -1042 209918
    3440 -1148 214924
    3401 -1254 219930
    3362 -1360 224936
    3323 -1466 229942
    3284 -1572 234948
    3245 -1678 239954
    3206 -1784 244960
    3167 -1890 249966
    3128 -1996 254972
    3089 -2102 259978
    3050 -2208 264984
    3011 -2314 269990
    2972 -2420 274996
    2933 -2526 280002
    2894 -2632 285008
    2855 -2738 290014
    2816 -2844 295020
    2777 -2950 300026
    0.961599054745963
    0.839588377723971

    6.35

```python

mascarillas_porcentaje = 1
normal_porcentaje = 1-  mascarillas_porcentaje

```

```python
import pandas as pd
import numpy as np
R0_face_covering = 0.839588377723971
#R0_face_covering = 0.446
R0_mascarillas = R0_face_covering

#R0_italy_normal  = 6.35
R0_italy_normal  = 3.590
R0_normal = R0_italy_normal

R0_mixto         = (face_covering + italy_normal / 2)

#4 posibilidades

#normal a mascarilla
#mascarilla a mascarilla
#mascarilla a normal
#mascarilla a mascarilla

array =[]
for porcentaje_uso_mascarillas in np.arange(0,1.1,.01) :
    mascarillas_porcentaje = porcentaje_uso_mascarillas
    normal_porcentaje      = 1-  mascarillas_porcentaje
    value = ( (normal_porcentaje*normal_porcentaje*R0_italy_normal) +
             (mascarillas_porcentaje * normal_porcentaje * (mascarillas_porcentaje )) +
             (mascarillas_porcentaje * normal_porcentaje * (mascarillas_porcentaje)) +
             (mascarillas_porcentaje*mascarillas_porcentaje*R0_face_covering) )

    value2 = (porcentaje_uso_mascarillas*R0_mascarillas) + ((1-porcentaje_uso_mascarillas)*R0_normal)
    dict_values = {'porcentaje_uso_mascarillas' : porcentaje_uso_mascarillas, 'calcul1' : value ,     'calcul2' : value2 }
    array.append(dict_values)
    print(str(porcentaje_uso_mascarillas)[:4],str(value)[:4],str(value2)[:4])

    #( (normal_porcentaje * R0_normal ) + ((1-normal_porcentaje) * R0_mascarillas))/2 +
df = pd.DataFrame(array)
df = df.set_index('porcentaje_uso_mascarillas')
df

```

    0.0 3.59 3.59
    0.01 3.51 3.56
    0.02 3.44 3.53
    0.03 3.38 3.50
    0.04 3.31 3.47
    0.05 3.24 3.45
    0.06 3.18 3.42
    0.07 3.11 3.39
    0.08 3.05 3.36
    0.09 2.99 3.34
    0.1 2.93 3.31
    0.11 2.87 3.28
    0.12 2.81 3.25
    0.13 2.76 3.23
    0.14 2.70 3.20
    0.15 2.65 3.17
    0.16 2.59 3.14
    0.17 2.54 3.12
    0.18 2.49 3.09
    0.19 2.44 3.06
    0.2 2.39 3.03
    0.21 2.34 3.01
    0.22 2.30 2.98
    0.23 2.25 2.95
    0.24 2.20 2.92
    0.25 2.16 2.90
    0.26 2.12 2.87
    0.27 2.08 2.84
    0.28 2.03 2.81
    0.29 1.99 2.79
    0.3 1.96 2.76
    0.31 1.92 2.73
    0.32 1.88 2.70
    0.33 1.84 2.68
    0.34 1.81 2.65
    0.35 1.77 2.62
    0.36 1.74 2.59
    0.37 1.71 2.57
    0.38 1.68 2.54
    0.39 1.64 2.51
    0.4 1.61 2.48
    0.41 1.58 2.46
    0.42 1.56 2.43
    0.43 1.53 2.40
    0.44 1.50 2.37
    0.45 1.47 2.35
    0.46 1.45 2.32
    0.47 1.42 2.29
    0.48 1.40 2.26
    0.49 1.38 2.24
    0.5 1.35 2.21
    0.51 1.33 2.18
    0.52 1.31 2.15
    0.53 1.29 2.13
    0.54 1.27 2.10
    0.55 1.25 2.07
    0.56 1.23 2.04
    0.57 1.21 2.02
    0.58 1.19 1.99
    0.59 1.18 1.96
    0.6 1.16 1.93
    0.61 1.14 1.91
    0.62 1.13 1.88
    0.63 1.11 1.85
    0.64 1.10 1.82
    0.65 1.09 1.80
    0.66 1.07 1.77
    0.67 1.06 1.74
    0.68 1.05 1.71
    0.69 1.03 1.69
    0.70 1.02 1.66
    0.71 1.01 1.63
    0.72 1.00 1.60
    0.73 0.99 1.58
    0.74 0.98 1.55
    0.75 0.97 1.52
    0.76 0.96 1.49
    0.77 0.96 1.47
    0.78 0.95 1.44
    0.79 0.94 1.41
    0.8 0.93 1.38
    0.81 0.92 1.36
    0.82 0.92 1.33
    0.83 0.91 1.30
    0.84 0.91 1.27
    0.85 0.90 1.25
    0.86 0.89 1.22
    0.87 0.89 1.19
    0.88 0.88 1.16
    0.89 0.88 1.14
    0.9 0.87 1.11
    0.91 0.87 1.08
    0.92 0.86 1.05
    0.93 0.86 1.03
    0.94 0.86 1.00
    0.95 0.85 0.97
    0.96 0.85 0.94
    0.97 0.84 0.92
    0.98 0.84 0.89
    0.99 0.84 0.86
    1.0 0.83 0.83
    1.01 0.83 0.81
    1.02 0.83 0.78
    1.03 0.83 0.75
    1.04 0.82 0.72
    1.05 0.82 0.70
    1.06 0.82 0.67
    1.07 0.81 0.64
    1.08 0.81 0.61
    1.09 0.81 0.59

      calcul1
      calcul2

      porcentaje_uso_mascarillas

      0.00
      3.590000
      3.590000

      0.01
      3.518841
      3.562496

      0.02
      3.448956
      3.534992

      0.03
      3.380333
      3.507488

      0.04
      3.312959
      3.479984

      ...
      ...
      ...

      1.05
      0.824371
      0.702068

      1.06
      0.821454
      0.674564

      1.07
      0.818550
      0.647060

      1.08
      0.815648
      0.619555

      1.09
      0.812736
      0.592051

110 rows × 2 columns

```python

distanciamiento s/n mascarilla s/n

cabrones (ni distanciamiento ni mascarilla)      : 6.35
vergonzosos (si distanciamiento, no mascarillas) : 3.590
por_cumplir (no distancimianto, si mascarilla)   :
responsable( si mascarillas, si responsable)     : 0.75

```

```python
from matplotlib import pyplot as plt
import pandas as pd
import numpy as np

def Get_Chart(df, title="default"):
    fig = plt.figure(figsize=(8, 6), dpi=80)

    for ca in df.columns:
        plt.plot(df[ca])
        plt.legend(df.columns)
        fig.suptitle(title, fontsize=20)
    plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%b-%d'))
    plt.gca().xaxis.set_major_locator(mdates.DayLocator(interval=7))
    plt.xticks(rotation=45)
    return plt
Get_Chart(df)
```

```python
"""
0.0 3.59
0.1 3.2826729539951573
0.2 2.989774140435836
0.30000000000000004 2.705303559322034
0.4 2.4232612106537528
0.5 2.1376470944309927
0.6000000000000001 1.8424612106537528
0.7000000000000001 1.5317035593220336
0.8 1.1993741404358353
0.9 0.8394729539951573
1.0 0.446
"""
```

```python

```


<p align="center">

</p>

## Interactive of the Central Limit Theorem Demonstration

### Summation or Averaging of Random Variables Tutorial

* demonstrate the practical impact of summation or averaging on distributions

* interactive plot demonstration with ipywidget package

#### Michael Pyrcz, Associate Professor, University of Texas at Austin

##### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1) | [GeostatsPy](https://github.com/GeostatsGuy/GeostatsPy)

#### Central Limit Theorem

The summation of independent random variables will approach Gaussian distributed as the number of random variables become large

* for any distribution shape for the individual random variables

* same for averaging since it is just the application of a scalar to the summation ($\frac{1}{m}$)

We can state this as:

\begin{equation}
Y \sim N\left[\right]
\end{equation}

where $Y$ is the summation of a large number of independent random variables, $X_i$, where $i = 1,\ldots,m$.

\begin{equation}
Y = \sum_{i=1}^{m \to \infty} X_i
\end{equation}

from statistical expectation we can calculate the central tedency as:

\begin{equation}
E[Y] = E \left[\sum_{i=1}^{m \to \infty} X_i \right] = \sum_{i=1}^{m \to \infty} E \left[X_i \right]
\end{equation}

and under the assumption of independence the dispersion as:

\begin{equation}
\sigma_Y^2 = \sum_{i=1}^{m \to \infty} \sigma_X^2
\end{equation}

therefore, we have our distribution as:

\begin{equation}
Y \sim N \left[ \sum_{i=1}^{m \to \infty} E[X_i], \sum_{i=1}^{m \to \infty} \sigma_X^2 \right]
\end{equation}

we can simplify by assuming the same central tendency and dispersion for all random variables, $X_i$, with central tendency and dispersion:

\begin{equation}
E[X_i] = \mu
\end{equation}

\begin{equation}
Var[X_i] = \sigma^2
\end{equation}

now we have:

\begin{equation}
Y \sim N \left[ m \mu, m \sigma_X^2 \right]
\end{equation}

for the case of the average instead of the summation of random variables, $X_i$:

\begin{equation}
Y = \frac{1}{m} \sum_{i=1}^{m \to \infty} X_i
\end{equation}

and all with the same central tendency and dispersion the distribution of $Y$ is given by:

\begin{equation}
Y \sim N \left[ \mu, \frac{\sigma^2}{n} \right]
\end{equation}

**Monte Carlo Simulation** is a method to sample from a single or set of random variables to assess the emergent distribution.

* also known as Monte Carlo methods and Monte Carlo experiments

* a method in statistics for generating draws from probability distributions empirically

* powerful due to it's broad applicability

We can apply Monte Carlo methods ot sample from the sum of our random variables.  We proceed as follows:

* for all $X_i$ random variables draw a random value (random realization), $x_i$

* calculate the average of the random realizations, $y_i = \frac{1}{n} \sum_{i=1}^{n} x_i$

* repeat over a large number of samples, $n$, and observe the resulting distribution

* compare the experimental CDF to the Gaussian distribution predicted from the Central Limit Theorem

assess the uncertainty in a sample statistic by repeated random sampling with replacement.

#### Objective

Provide an example and demonstration for:

1. interactive plotting in Jupyter Notebooks with Python packages matplotlib and ipywidgets
2. provide an intuitive hands-on demonstration of the central limit theorem

#### Getting Started

Here's the steps to get setup in Python with the GeostatsPy package:

1. Install Anaconda 3 on your machine (https://www.anaconda.com/download/).
2. Open Jupyter and in the top block get started by copy and pasting the code block below from this Jupyter Notebook to start using the geostatspy functionality.

#### Load the Required Libraries

The following code loads the required libraries.

```python
%matplotlib inline
from ipywidgets import interactive                        # widgets and interactivity
from ipywidgets import widgets
from ipywidgets import Layout
from ipywidgets import Label
from ipywidgets import VBox, HBox
import matplotlib.pyplot as plt                           # plotting
import numpy as np                                        # working with arrays
import pandas as pd                                       # working with DataFrames
from scipy.stats import triang                            # parametric distributions
from scipy.stats import binom
from scipy.stats import norm
from scipy.stats import uniform
from scipy.stats import triang
from scipy import stats                                   # statistical calculations
import random                                             # random drawing / bootstrap realizations of the data
import math                                               # square root operator
```

#### Specify the Distributions of $X_i$ and the Number of Random Variables, $m$, and Realizations, $L$

This is an interactive method to:

* select a parametric distribution

* select the distribution parameters

* select the number of random variables, $m$

* select the number of Monte Carlo realizations, $L$, to sample the distribution of $Y = \sum_{i=1}^m X_i$

```python
# parameters for the synthetic dataset
bins = np.linspace(0,100,50)

# interactive calculation of the sample set (control of source parametric distribution and number of samples)
l = widgets.Text(value='                              Central Limit Theorem Demonstration, Michael Pyrcz, Associate Professor, The University of Texas at Austin',layout=Layout(width='950px', height='30px'))
dist = widgets.Dropdown(
    options=['Triangular', 'Uniform', 'Gaussian'],
    value='Gaussian',
    description='Dataset Distribution:',
    disabled=False,
    layout=Layout(width='200px', height='30px')
)
a = widgets.FloatSlider(min=0.0, max = 100.0, value = 0.5, description = '$X_i$: Mean / Mode',orientation='vertical',layout=Layout(width='110px', height='200px'))
a.style.handle_color = 'blue'
d = widgets.FloatSlider(min=0.01, max = 30.0, value = 5.0, step = 1.0, description = '$X_i$: St.Dev.',orientation='vertical',layout=Layout(width='110px', height='200px'))
d.style.handle_color = 'green'
b = widgets.FloatSlider(min = 0, max = 100.0, value = 0.5, description = '$X_i$: Min.',orientation='vertical',layout=Layout(width='110px', height='200px'))
b.style.handle_color = 'red'
c = widgets.IntSlider(min = 0, max = 100, value = 100, description = '$X_i$: Max.',orientation='vertical',layout=Layout(width='110px', height='200px'))
c.style.handle_color = 'orange'
m = widgets.IntSlider(min = 1, max = 20, value = 1, description = '$m$ ',orientation='vertical',layout=Layout(width='110px', height='200px'))
m.style.handle_color = 'gray'
L = widgets.IntSlider(min = 1, max = 1000, value = 100, description = '$L$ ',orientation='vertical',layout=Layout(width='110px', height='200px'))
L.style.handle_color = 'gray'

ui = widgets.HBox([dist,a,d,b,c,m,L],)                   # basic widget formatting
ui2 = widgets.VBox([l,ui],)

def f_make(dist,a, b, c, d, m, L):                       # function to take parameters, make sample and plot
    dataset, average, stdev = make_average_data(dist,a, b, c, d, m, L)
    stdev = stdev / math.sqrt(m)

    plt.subplot(221)
    sample, null, null = make_average_data(dist,a, b, c, d, m=1, L=1000)
    plt.hist(sample,alpha=0.2,color="red",edgecolor="black",bins=bins)
    plt.xlim(0.0,100.0); plt.title('Distribution, $X_i$'); plt.ylabel('Frequency'); plt.xlabel('Values')

    plt.subplot(222)
    plt.hist(dataset,alpha=0.2,color="red",edgecolor="black",bins=bins)
    plt.xlim(0.0,100.0); plt.title('Distribution, $Y$'); plt.ylabel('Frequency'); plt.xlabel('Values')

    plt.subplot(223)
    plt.hist(dataset,cumulative = True, density = True, alpha=0.2,color="red",edgecolor="black", bins = bins, label = 'Bootstrap')
    plt.xlim(0.0,100.0); plt.title('Comparison to Gaussian CDF'); plt.xlabel('Values'); plt.ylabel('Cumulative Probability')

    cumul_prob = np.linspace(0.0,1.0,100)
    prop_values = norm.ppf(cumul_prob)
    prop_values = prop_values * stdev + average
    plt.plot(prop_values, cumul_prob, color = 'black', linewidth = 2, dashes = (5,2,1,2), label = 'Analytical')

    plt.legend()

    plt.subplots_adjust(left=0.0, bottom=0.0, right=2.0, top=2.6, wspace=0.2, hspace=0.2)
    plt.show()

def make_average_data(dist,a, b, c, d, m, L):            # function to check parameters and make samples
    average = 0.0; stdev = 0.0
    if dist == 'Uniform':
        if b >= c:
            print('Invalid uniform distribution parameters')
            return None
        dataset = np.mean(uniform.rvs(size=[m,L], loc = b, scale = c, random_state = 73073),axis = 0)
        average = uniform.mean(loc = b, scale = c)
        stdev = uniform.std(loc = b, scale = c)
        return dataset, average, stdev
    elif dist == 'Triangular':
        interval = c - b
        if b >= a or a >= c or interval <= 0:
            print('Invalid triangular distribution parameters')
            return None
        dataset = np.mean(triang.rvs(size=[m,L], loc = b, c = (a-b)/interval, scale = interval, random_state = 73073), axis = 0)
        average = triang.mean(loc = b, c = (a-b)/interval, scale = interval)
        stdev = triang.std(loc = b, c = (a-b)/interval, scale = interval)
        return dataset, average, stdev
    elif dist == 'Gaussian':
        dataset = np.mean(norm.rvs(size=[m,L], loc = a, scale = d, random_state = 73073), axis = 0)
        average = norm.mean(loc = a, scale = d)
        stdev = norm.std(loc = a, scale = d)
        return dataset, average, stdev

# connect the function to make the samples and plot to the widgets
interactive_plot = widgets.interactive_output(f_make, {'dist': dist,'a': a, 'd': d, 'b': b, 'c': c, 'm': m, 'L':L})
interactive_plot.clear_output(wait = True)               # reduce flickering by delaying plot updating
```

#### Display the GUI for Building the Synthetic Dataset

We display the GUI now.  Select the desired parametric distribution and associated parameters.

* if the parameters are invalid (e.g. traingular mode > max) an error message should display.

```python
display(ui2, interactive_plot)                            # display the interactive plot
```

    VBox(children=(Text(value='                              Central Limit Theorem Demonstration, Michael Pyrcz, A…

    Output()

#### Observations

Some observations:

* if $X_i$ are Gaussian, $Y$ is Gaussian distributed

* if $X_i$ are uniform then convergence occurs over 5 or more RVs averaged

* triangular distribution converges faster than uniform

#### Comments

This was a simple demonstration central limit theorem with interactive plots in Jupyter Notebook Python with the ipywidgets and matplotlib packages.

I have many other demonstrations on data analytics and machine learning, e.g. on the basics of working with DataFrames, ndarrays, univariate statistics, plotting data, declustering, data transformations, trend modeling and many other workflows available at https://github.com/GeostatsGuy/PythonNumericalDemos and https://github.com/GeostatsGuy/GeostatsPy.

I hope this was helpful,

*Michael*

#### The Author:

### Michael Pyrcz, Associate Professor, University of Texas at Austin
*Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions*

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers' and geoscientists' impact in subsurface resource development.

For more about Michael check out these links:

#### [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

#### Want to Work Together?

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

* Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I'd be happy to drop by and work with you!

* Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

* I can be reached at mpyrcz@austin.utexas.edu.

I'm always happy to discuss,

*Michael*

Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin

#### More Resources Available at: [Twitter](https://twitter.com/geostatsguy) | [GitHub](https://github.com/GeostatsGuy) | [Website](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446) | [YouTube](https://www.youtube.com/channel/UCLqEr-xV-ceHdXXXrTId5ig)  | [LinkedIn](https://www.linkedin.com/in/michael-pyrcz-61a648a1)

```python

```


# Examen Módulo 1 - Optimización.
- Lea cuidadosamente las siguientes **indicaciones** antes de comenzar el examen:
- Para resolver el examen edite este mismo archivo y renómbrelo de la siguiente manera: *Examen3_ApellidoNombre*, donde *ApellidoNombre* corresponde a su apellido paterno con la inicial en mayúscula, seguido de su primer nombre con la inicial en mayúscula **sin acentos**. Por ejemplo, en mi caso el nombre del archivo sería *Examen1_ArjonaAlberto*.
- Resuelva los puntos en el espacio provisto. Si requiere agregar más celdas para código o escritura, hágalo.
- Recuerde que también se está evaluando su capacidad de interpretar los resultados. Escriba sus interpretaciones/conclusiones en celdas utilizando *Markdown*.
- El formato de presentación de todo el examen en general debe ser adecuado. Use tamaños de letra, colores, etiquetas, etcétera.

## 1. Optimización de funciones escalares usando `sympy` (25 puntos).

**Enunciado.** Calcular **las dimensiones** del rectángulo de área máxima que puede inscribirse dentro de un semicírculo de radio 2.

Note que **las dimensiones** son base $b = 2x$ y altura $h = y$. Ver la siguiente figura.

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
x = np.linspace(-2, 2, 100)
y = np.sqrt(4 - x**2)

xr = [-1.5, -1.5, 1.5, 1.5, -1.5]
yr = [0, np.sqrt(4 - 1.5**2), np.sqrt(4 - 1.5**2), 0, 0]

xx = [0, 1.5]
yx = [0, 0]

xy = [1.5, 1.5]
yy = [0, np.sqrt(4 - 1.5**2)]

plt.figure(figsize = (12, 6))
plt.plot(x, y, 'b', label = 'semicírculo')
plt.plot(xr, yr, 'r', label = 'rectángulo')
plt.plot(xx, yx, 'k')
plt.plot(xy, yy, 'g')
plt.text(0.7, 0.1, '$x$', fontsize = 16)
plt.text(1.6, 0.6, '$y$', fontsize = 16, color = 'g')
plt.legend(loc = 'best')
plt.show()
```

**a) (10 puntos).** Encuentre el área del rectángulo en función de la variable $x$, $A_R(x)$. También provea el dominio de la función para que tenga sentido.

Realice este punto usando fórmulas en LaTeX, en celdas Markdown.

**b) (10 puntos).** Usando `sympy` maximice la función $A_R(x)$ en su dominio.

**c) (5 puntos).** ¿Cuáles son las dimensiones $b$ y $h$ del rectángulo resultante?

## 2. Programación lineal (30 puntos).

**Enunciado.** Considere un restaurante que abre los siete días de la semana. Con base en la experiencia, el mínimo número de trabajadores necesarios en cada día de la semana es:

```python
import pandas as pd
```

```python
df = pd.DataFrame(columns=['Lunes', 'Martes', 'Miercoles', 'Jueves', 'Viernes', 'Sábado', 'Domingo'], index = ['Num_trabajadores'])
df.loc['Num_trabajadores', :] = [14, 13, 15, 16, 19, 18, 11]
df
```

      Lunes
      Martes
      Miercoles
      Jueves
      Viernes
      Sábado
      Domingo

      Num_trabajadores
      14
      13
      15
      16
      19
      18
      11

Cada trabajador trabaja cinco días consecutivos, y luego descansa dos, repitiendo este patrón indefinidamente. Se debe minimizar la cantidad de trabajadores totales en el restaurante.

*Definición de variables*: enumeramos los días del uno al siete (Lunes - 1, Martes - 2, ..., Domingo - 7). Sea $x_i$ el número de trabajadores que empieza su turno de cinco días el día $i$.

**a) (20 puntos).** Escriba la función a minimizar junto con las restricciones, explicando detalladamente cada paso (usando fórmulas en LaTeX, en celdas Markdown). Además escriba el problema en la forma
\begin{equation}
\begin{array}{ll}
\min_{\boldsymbol{x}} & \boldsymbol{c}^T\boldsymbol{x} \\
\text{s. a. }        & \boldsymbol{A}_{eq}\boldsymbol{x}=\boldsymbol{b}_{eq} \\
                     & \boldsymbol{A}\boldsymbol{x}\leq\boldsymbol{b},
\end{array}
\end{equation}

proporcionando $\boldsymbol{c}$, $\boldsymbol{A}$ y $\boldsymbol{b}$ ($\boldsymbol{A}_{eq}$ y $\boldsymbol{b}_{eq}$, de ser necesario) como arreglos de `NumPy` (no es necesario escribir en LaTeX el problema en la forma indicada, solo proporcionar las matrices como arreglos de `NumPy`).

**b) (10 puntos).** Resuelva el problema utilizando la función `linprog`. ¿Cuántos trabajadores son necesarios en total?

## 3. Ajuste de curvas (30 puntos).

**Enunciado**. El archivo `population_data.csv` contiene información histórica anual de México relativa a la población, nacimientos y muertes desde el año 1950 hasta el año 2015. La primer columna corresponde a los años, la segunda a la población, la tercera a los nacimientos y la última a los decesos.

Tomado de: https://en.wikipedia.org/wiki/Demographics_of_Mexico.

**a) (10 puntos).** Importar los datos en un DataFrame de pandas.

**b) (15 puntos).** Usando los años como variable independiente $x$ y la población como variable dependiente $y$, ajustar polinomios de grado 1 hasta grado 3.

Mostrar en un solo gráfico los datos de población contra los años, y los polinomios ajustados.

Graficar el error cuadrático acumulado contra el número de términos. ¿Cuál es el polinomio que mejor se ajusta?

**c) (5 puntos).** Con los polinomios ajustados en el punto anterior, estime la población en el año 2016 y compare con el dato real 127,5 millones.

¿Cuál polinomio estima mejor este dato? ¿Es el polinomio que mejor se ajusta? Concluya.

## 4. Clasificador binario (25 puntos)

**Enunciado.** Se tienen datos de peso (kg) y de estatura (m) de 100 pacientes. Además, para cada uno de ellos se tiene una clasificación si está en sobrepeso (1) o no (0). Los datos son los siguientes:

```python
x1 = 1.5 + 0.5*np.random.random((100,1))
x2 = 45 + 70*np.random.random((100,1))
X = np.concatenate((x1, x2), axis = 1)
Y = (X[:, 1]/X[:, 0]**2 > 25*np.ones((100,)))*1
```

```python
plt.figure(figsize = (8,6))
plt.scatter(X[:,0], X[:,1], c=Y)
plt.show()
```

**a) (15 puntos).** Diseñe un clasificador binario por regresión logística lineal utilizando únicamente los datos de los primeros 80 pacientes. Muestre, además, el gráfico de la división del clasificador con los puntos de entrenamiento.

**b) (10 puntos).** Utilice el clasificador diseñado para clasificar a los 20 pacientes restantes. ¿Cuántos pacientes se clasifican bien? ¿Cuántos mal?

Created with Jupyter by Xiuhcoatl.


```python
# Biblioteka SymPy
import sympy as sp
'''
Dodatkowo importujemy osobno skok Heaviside'a poniewaz
funkcja jest zaimplementowana w sposob taki, aby
byc jak dopasowywana do uzytkownika tak wiec
musimy sami wybrac jej wartosc poczatkowa w tym wlasnie celu ja importujemy
'''
from sympy import Heaviside
# NumPy uzywanany do numerycznych operacji matematycznych
import numpy as np
# Matplotlib sluzacy do wizualizacji
import matplotlib.pyplot as plt
from sympy.printing.latex import print_latex
# from sympy import init_printing
# init_printing()
```

# a)

```python
# Definiujemy obiekty biblioteki SymPy
t, y, s, K= sp.symbols('t y s K', real=True)
```

```python
# Tworzymy rownanie
eq = 1/((s+1)*(s+3))
print_latex(eq)
eq
```

    \frac{1}{\left(s + 1\right) \left(s + 3\right)}

$\displaystyle \frac{1}{s \left(s + 1\right) \left(s + 3\right)}$

```python
sp.Eq(K, eq)
```

$\displaystyle K = \frac{1}{\left(s + 1\right) \left(s + 3\right)}$

```python
sp.Eq(K*((s+1)*(s+3)), eq*((s+1)*(s+3)))
```

$\displaystyle K \left(s + 1\right) \left(s + 3\right) = 1$

```python
a = sp.expand(sp.Eq(K*((s+1)*(s+3)), eq*((s+1)*(s+3))))
a
```

$\displaystyle K s^{2} + 4 K s + 3 K = 1$

```python
b = a.lhs.replace(K, 1)
b
```

$\displaystyle s^{2} + 4 s + 3$

```python
sp.inverse_laplace_transform(b, s, t)
```

$\displaystyle 4 \mathcal{L}^{-1}_{s}\left[s\right]\left(t\right) + \mathcal{L}^{-1}_{s}\left[s^{2}\right]\left(t\right) + 3 \delta\left(t\right)$

```python
b.subs(s, "y'")
```

$\displaystyle s^{2} + 4 s + 3$

```python

```

```python

```

```python

```

```python

```


# 机器学习练习 2 - 逻辑回归

## 逻辑回归

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
```

```python
path = 'ex2data1.txt'
data = pd.read_csv(path,header=None,names=['Exam1','Exam2','Admitted'])
data.head()
```

      Exam1
      Exam2
      Admitted

      0
      34.623660
      78.024693
      0

      1
      30.286711
      43.894998
      0

      2
      35.847409
      72.902198
      0

      3
      60.182599
      86.308552
      1

      4
      79.032736
      75.344376
      1

```python
positive = data[data['Admitted'].isin([1])]
negative = data[data['Admitted'].isin([0])]

fig,ax = plt.subplots(figsize=(12,8))
ax.scatter(positive['Exam1'],positive['Exam2'],s=50,c='b',marker='o',label='Admitted')
ax.scatter(negative['Exam1'],negative['Exam2'],s=50,c='r',marker='x',label='Not Admitted')
ax.legend()
ax.set_xlabel('Exam 1 Score')
ax.set_ylabel('Exam 2 Score')
plt.show()
```

## sigmoid 函数

```python
def sigmoid(z):
    return 1/(1+np.exp(-z))
```

```python
nums = np.arange(-10,10,step=1)

fig,ax = plt.subplots(figsize=(12,8))
ax.plot(nums,sigmoid(nums),'r')
plt.show()
```

代价函数：
$J\left( \theta  \right)=\frac{1}{m}\sum\limits_{i=1}^{m}{[-{{y}^{(i)}}\log \left( {{h}_{\theta }}\left( {{x}^{(i)}} \right) \right)-\left( 1-{{y}^{(i)}} \right)\log \left( 1-{{h}_{\theta }}\left( {{x}^{(i)}} \right) \right)]}$

```python
def cost(theta,X,y):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)
    first = np.multiply(-y,np.log(sigmoid(X.dot(theta.T))))
    second = np.multiply(1-y,np.log(1-sigmoid(X.dot(theta.T))))
    return np.sum(first - second) / len(X)
```

处理一下原数据

```python
data.insert(0,'Ones',1)
cols = data.shape[1]
X = data.iloc[:,0:cols-1]
y = data.iloc[:,cols-1:cols]

X = np.array(X.values)
y = np.array(y.values)
theta = np.zeros(3)
```

```python
print(theta,X.shape,theta.shape,y.shape)
```

    [0. 0. 0.] (100, 3) (3,) (100, 1)

```python
cost(theta,X,y)
```

    0.6931471805599453

gradient descent(梯度下降)
* 这是批量梯度下降（batch gradient descent）
* 转化为向量化计算： $\frac{1}{m} X^T( Sigmoid(X\theta) - y )$
$$\frac{\partial J\left( \theta  \right)}{\partial {{\theta }_{j}}}=\frac{1}{m}\sum\limits_{i=1}^{m}{({{h}_{\theta }}\left( {{x}^{(i)}} \right)-{{y}^{(i)}})x_{_{j}}^{(i)}}$$

```python
def gradient(theta,X,y):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)

    parameters = int(theta.ravel().shape[1])
    grad = np.zeros(parameters)

    error = sigmoid(X.dot(theta.T)) - y

    for i in range(parameters):
        term = np.multiply(error,X[:,i])
        grad[i] = np.sum(term) / len(X)
    return grad
```

```python
gradient(theta,X,y)
```

    array([ -0.1       , -12.00921659, -11.26284221])

现在可以用SciPy's truncated newton（TNC）实现寻找最优参数。

```python
import scipy.optimize as opt
result = opt.fmin_tnc(func=cost,x0=theta,fprime=gradient,args=(X,y))
# res 是 ex2_myAnswer 中的，这两个实际是一样的。
res = opt.minimize(fun=cost, x0=theta, args=(X, y), method='TNC', jac=gradient)
print(result,res,sep='\n')
```

    (array([-25.16131872,   0.20623159,   0.20147149]), 36, 0)
         fun: 0.20349770158947425
         jac: array([8.89622886e-09, 7.59647888e-08, 4.78664200e-07])
     message: 'Local minimum reached (|pg| ~= 0)'
        nfev: 36
         nit: 17
      status: 0
     success: True
           x: array([-25.16131872,   0.20623159,   0.20147149])

```python
cost(result[0],X,y)
```

    0.20349770158947425

接下来，我们需要编写一个函数，用我们所学的参数theta来为数据集X输出预测。然后，我们可以使用这个函数来给我们的分类器的训练精度打分。
逻辑回归模型的假设函数：
	\\[{{h}_{\theta }}\left( x \right)=\frac{1}{1+{{e}^{-{{\theta }^{T}}X}}}\\]
当${{h}_{\theta }}$大于等于0.5时，预测 y=1

当${{h}_{\theta }}$小于0.5时，预测 y=0 。

```python
def predict(theta,X):
    probability = sigmoid(X.dot(theta.T))
    return [1 if x >= 0.5 else 0 for x in probability]
```

```python
theta_min = np.matrix(result[0])
predictions = predict(theta_min,X)
correct = [1 if (a == 1 and b == 1) or (a == 0 and b == 0) else 0 for (a,b) in zip(predictions,y)]
accuracy = (sum(map(int,correct)) % len(correct))
print('accuracy = {0}%'.format(accuracy))
```

    accuracy = 89%

## 正则化逻辑回归

```python
path2 = 'ex2data2.txt'
data2 = pd.read_csv(path2,header=None,names=['Test1','Test2','Accepted'])
data2.head()
```

      Test1
      Test2
      Accepted

      0
      0.051267
      0.69956
      1

      1
      -0.092742
      0.68494
      1

      2
      -0.213710
      0.69225
      1

      3
      -0.375000
      0.50219
      1

      4
      -0.513250
      0.46564
      1

```python
positive = data2[data2['Accepted'].isin([1])]
negative = data2[data2['Accepted'].isin([0])]

fig,ax = plt.subplots(figsize=(12,8))
ax.scatter(positive['Test1'],positive['Test2'],s=50,c='b',marker='o',label='Accepted')
ax.scatter(negative['Test1'],negative['Test2'],s=50,c='r',marker='x',label='Rejected')
ax.legend()
ax.set_xlabel('Test 1 Score')
ax.set_ylabel('Test 2 Score')
plt.show()
```

```python
degree = 5
x1 = data2['Test1']
x2 = data2['Test2']

data2.insert(3,'ones',1)

for i in range(1,degree):
    for j in range(0,i):
        data2['F'+str(i)+str(j)] = np.power(x1,i-j) * np.power(x2,j)
data2.drop('Test1',axis=1,inplace=True)
data2.drop('Test2',axis=1,inplace=True)
data2.head()
```

      Accepted
      ones
      F10
      F20
      F21
      F30
      F31
      F32
      F40
      F41
      F42
      F43

      0
      1
      1
      0.051267
      0.002628
      0.035864
      0.000135
      0.001839
      0.025089
      0.000007
      0.000094
      0.001286
      0.017551

      1
      1
      1
      -0.092742
      0.008601
      -0.063523
      -0.000798
      0.005891
      -0.043509
      0.000074
      -0.000546
      0.004035
      -0.029801

      2
      1
      1
      -0.213710
      0.045672
      -0.147941
      -0.009761
      0.031616
      -0.102412
      0.002086
      -0.006757
      0.021886
      -0.070895

      3
      1
      1
      -0.375000
      0.140625
      -0.188321
      -0.052734
      0.070620
      -0.094573
      0.019775
      -0.026483
      0.035465
      -0.047494

      4
      1
      1
      -0.513250
      0.263426
      -0.238990
      -0.135203
      0.122661
      -0.111283
      0.069393
      -0.062956
      0.057116
      -0.051818

# regularized cost（正则化代价函数）
$$J\left( \theta  \right)=\frac{1}{m}\sum\limits_{i=1}^{m}{[-{{y}^{(i)}}\log \left( {{h}_{\theta }}\left( {{x}^{(i)}} \right) \right)-\left( 1-{{y}^{(i)}} \right)\log \left( 1-{{h}_{\theta }}\left( {{x}^{(i)}} \right) \right)]}+\frac{\lambda }{2m}\sum\limits_{j=1}^{n}{\theta _{j}^{2}}$$

```python
def costReg(theta,X,y,learningRate):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)
    first = np.multiply(-y,np.log(sigmoid(X.dot(theta.T))))
    second = np.multiply(1-y,np.log(1-sigmoid(X.dot(theta.T))))
    reg = (learningRate / (2 * len(X)) * np.sum(np.power(theta[:,1:theta.shape[1]],2)))
    return np.sum(first - second) / len(X) + reg
```

如果我们要使用梯度下降法令这个代价函数最小化，因为我们未对${{\theta }_{0}}$ 进行正则化，所以梯度下降算法将分两种情形：
\begin{align}
  & Repeat\text{ }until\text{ }convergence\text{ }\!\!\{\!\!\text{ } \\
 & \text{     }{{\theta }_{0}}:={{\theta }_{0}}-a\frac{1}{m}\sum\limits_{i=1}^{m}{[{{h}_{\theta }}\left( {{x}^{(i)}} \right)-{{y}^{(i)}}]x_{_{0}}^{(i)}} \\
 & \text{     }{{\theta }_{j}}:={{\theta }_{j}}-a\frac{1}{m}\sum\limits_{i=1}^{m}{[{{h}_{\theta }}\left( {{x}^{(i)}} \right)-{{y}^{(i)}}]x_{j}^{(i)}}+\frac{\lambda }{m}{{\theta }_{j}} \\
 & \text{          }\!\!\}\!\!\text{ } \\
 & Repeat \\
\end{align}

对上面的算法中 j=1,2,...,n 时的更新式子进行调整可得：
${{\theta }_{j}}:={{\theta }_{j}}(1-a\frac{\lambda }{m})-a\frac{1}{m}\sum\limits_{i=1}^{m}{({{h}_{\theta }}\left( {{x}^{(i)}} \right)-{{y}^{(i)}})x_{j}^{(i)}}$

```python
def gradientReg(theta,X,y,learningRate):
    theta = np.matrix(theta)
    X = np.matrix(X)
    y = np.matrix(y)

    parameters = int(theta.ravel().shape[1])
    grad = np.zeros(parameters)

    error = sigmoid(X * theta.T) - y

    for i in range(parameters):
        term = np.multiply(error, X[:,i])

        if (i == 0):
            grad[i] = np.sum(term) / len(X)
        else:
            grad[i] = (np.sum(term) / len(X)) + ((learningRate / len(X)) * theta[:,i])

    return grad
```

```python
# set X and y (remember from above that we moved the label to column 0)
cols = data2.shape[1]
X2 = data2.iloc[:,1:cols]
y2 = data2.iloc[:,0:1]

# convert to numpy arrays and initalize the parameter array theta
X2 = np.array(X2.values)
y2 = np.array(y2.values)
theta2 = np.zeros(11)
```

```python
learningRate = 1
```

```python
costReg(theta2, X2, y2, learningRate)
```

    0.6931471805599454

```python
gradientReg(theta2, X2, y2, learningRate)
```

    array([0.00847458, 0.01878809, 0.05034464, 0.01150133, 0.01835599,
           0.00732393, 0.00819244, 0.03934862, 0.00223924, 0.01286005,
           0.00309594])

```python
result2 = opt.fmin_tnc(func=costReg, x0=theta2, fprime=gradientReg, args=(X2, y2, learningRate))
result2
```

    (array([ 0.53010248,  0.29075567, -1.60725764, -0.58213819,  0.01781027,
            -0.21329508, -0.40024142, -1.37144139,  0.02264304, -0.9503358 ,
             0.0344085 ]), 22, 1)

```python
theta_min = np.matrix(result2[0])
predictions = predict(theta_min, X2)
correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y2)]
accuracy = (sum(map(int, correct)) % len(correct))
print ('accuracy = {0}%'.format(accuracy))
```

    accuracy = 78%

虽然我们实现了这些算法，值得注意的是，我们还可以使用高级Python库像scikit-learn来解决这个问题。

```python
from sklearn import linear_model
model = linear_model.LogisticRegression(penalty='l2',C=1.0)
model.fit(X2,y2.ravel())
```

    LogisticRegression()

```python
model.score(X2,y2)
```

    0.6610169491525424