# Bayesian Inference of COVID-19 Spreading Rates in South Africa

**Authors** :*Rendani Mbuvha [Github: [@rendanim](https://github.com/rendanim)] & Tshilidzi Marwala [Twitter: [@txm1971](https://twitter.com/@txm1971)]*

Follows methodology from:
* Bayesian Inference of COVID-19 Spreading Rates in South Africa
    Rendani Mbuvha, Tshilidzi Marwala
    medRxiv 2020.04.28.20083873; doi: https://doi.org/10.1101/2020.04.28.20083873 and

* Dehning, J., Zierenberg, J., Spitzner, F. P., Wibral, M., Neto, J. P., Wilczek, M., & Priesemann, V. (2020). Inferring COVID-19 spreading rates and potential change points for case number forecasts. arXiv preprint arXiv:2004.01105.

#### Only run the next block if you are running the notebook for the first time

```python
import warnings
warnings.filterwarnings('ignore')

!pip3 install pymc3
!pip3 install theano
!pip3 install git+https://github.com/rendanim/covid19_inference_forecast.git
```

#### ..Start here for reruns

```python
import sys
sys.path.append('..')
import datetime

import numpy as np
import pymc3 as pm
import matplotlib.pyplot as plt
import datetime
import time as time_module
import sys
import os
import scipy.stats
import theano
import matplotlib
import pandas as pd
try:
    import covid19_inference as cov19
except ModuleNotFoundError:
    sys.path.append('..')
    import covid19_inference as cov19
```

## Loading data from covid19za repo

https://github.com/dsfsi/covid19za

```python
url = 'https://raw.githubusercontent.com/dsfsi/covid19za/master/data/covid19za_provincial_cumulative_timeline_confirmed.csv'
df_cases = pd.read_csv(url,
                     parse_dates=['date'], dayfirst=True,
                     squeeze=True).sort_index()
df_cases.head()
```

```python
date_data_begin = datetime.datetime(2020,6,1)
date_data_end   = df_cases['date'].max()
df_filtered_cases = df_cases[df_cases['date'] >= date_data_begin]
cases_obs = cases_obs = df_filtered_cases['total'].values
```

```python
cases_obs
```

```python
num_days_data = (date_data_end-date_data_begin).days
diff_data_sim = 20
num_days_future = 30
date_begin_sim = date_data_begin - datetime.timedelta(days = diff_data_sim)
date_end_sim   = date_data_end   + datetime.timedelta(days = num_days_future)
num_days_sim = (date_end_sim-date_begin_sim).days
```

## Model Specification and Prior Distributions

Only starting from 15 june no change points considered here. Modelling is entirely in a level 3 context...

We use an SIR model defined by the following ODEs:

\begin{align}
\frac{dS}{dt} &= -\frac{\lambda S I}{N}\\
\frac{dI}{dt} &= \sigma S - \mu I\\
\frac{dR}{dt} &=  \mu I
\end{align}

We also include a delay $D$ in becoming infected ($I^{\text{new}}$) and being reported in the confirmed case statistics, such that the confirmed reported cases   $\text{CR}_t$ at some time $t$ are in the form  :
\begin{equation}
    \text{CR}_t = I^{\text{new}}_{t-D}
\end{equation}

We donot explicitly set inital conditions or fix parameters. Rather we put prior distributions on all parameters as follows:

|Parameter | Prior Distribution                      |
|----------:|-----------------------------------------:|
|Spreading rate $\lambda_0$ | LogNormal(log(0.4),0.7)|
|Recovery rate  $\mu$       |LogNormal(log(1/8),0.2) |
|Reporting Delay $D$        | LogNormal(log(8),0.2)  |
|Initial Infectious  $I_0$  |  Half-Cauchy(20)       |

```python
prior_date_level_3 =  datetime.datetime(2020,6,15)

change_points = [ dict(pr_mean_date_begin_transient = prior_date_level_3,
                      pr_sigma_date_begin_transient = 1,
                      pr_median_lambda = 0.4,
                      pr_sigma_lambda = 0.7)]

```

## SIR with priors specified below inference perfromed using No-U-Turn Sampler

Hoffman, Matthew D., and Andrew Gelman. "The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo." J. Mach. Learn. Res. 15.1 (2014): 1593-1623..

```python
sir_traces_NUTS = []
sir_models = []

sir_model = cov19.SIR_with_change_points(new_cases_obs = np.diff(cases_obs),
                                     change_points_list = change_points,
                                    date_begin_simulation =date_begin_sim,
                                    num_days_sim=num_days_sim,
                                    diff_data_sim=diff_data_sim,
                                   N=59e6,priors_dict=dict(pr_beta_I_begin = 20,
                                pr_median_lambda_0 =  0.4,
                                pr_sigma_lambda_0 = 0.7,
                                pr_median_mu = 1/8,
                                pr_sigma_mu = 0.2,
                                pr_median_delay = 8,
                                pr_sigma_delay = 0.2,
                                pr_beta_sigma_obs = 10))

sir_models.append(sir_model)

with sir_model:
    step_NUTS = pm.NUTS()
    sir_traces_NUTS.append(pm.sample(model=sir_model, init='advi',step=step_NUTS, draws=500,tune=10))
```

## Posterior Distributions of Parameters

```python
varnames = cov19.plotting.get_all_free_RVs_names(sir_models[0])
varnames = ['I_begin', 'lambda_0', 'mu', 'delay']
num_cols = 4
num_rows = int(np.ceil(len(varnames)/num_cols))
x_size = num_cols * 2.5
y_size = num_rows * 2.5

fig, axes = plt.subplots(num_rows, num_cols, figsize = (x_size, y_size),squeeze=False)
i_ax = 0
for i_row, axes_row in enumerate(axes):
    for i_col, ax in enumerate(axes_row):
        if i_ax >= len(varnames):
            ax.set_visible(False)
            continue
        else:
            cov19.plotting.plot_hist(sir_models[0], sir_traces_NUTS[0], ax, varnames[i_ax],colors = ('tab:cyan', 'tab:green'))
        if not i_col == 0:
            ax.set_ylabel('')
        if i_col == 0 and i_row == 0:
            ax.legend()
        i_ax += 1
fig.subplots_adjust(wspace=0.25, hspace=0.4)

```

## Predictions

```python
date_data_end_to_plot   = date_data_end
cases_obs_to_plot = cases_obs

def truncate_number(number, precision):
    return '{{:.{}f}}'.format(precision).format(number)

def print_median_CI(arr, prec = 2):
    f_trunc = lambda n: truncate_number(n, prec)
    med = f_trunc(np.mean(arr))
    perc1, perc2 = f_trunc(np.percentile(arr, q=2.5)), f_trunc(np.percentile(arr, q=97.5))
    return 'Median: {}\n95% CI: [{}, {}]'.format(med, perc1, perc2)
def conv_time_to_mpl_dates(arr):
    return matplotlib.dates.date2num([datetime.timedelta(days=float(date)) + date_begin_sim for date in arr])

print("Plots with data from {} to {}".format(date_data_begin.date().isoformat(), date_data_end.date().isoformat()))

trace = sir_traces_NUTS[0]
posterior = sir_traces_NUTS[:]

pos_letter = (-0.2, 1)

limit_lambda = (-0.1,0.5)
bins_lambda  = np.linspace(*limit_lambda, 30)

fig, axes = plt.subplots(2,1, figsize=(10,10), gridspec_kw={'height_ratios': [2,2]})

colors  = ['tab:green','tab:orange']
legends = ['Forecast: ', ' Predictions (9 July - Level 3) ']

ax = axes[0]

time1 = np.arange(-len(cases_obs)+2, 1)
mpl_dates = conv_time_to_mpl_dates(time1) + diff_data_sim + num_days_data
time_for_obs = np.arange(-len(cases_obs)+2, 1+len(cases_obs_to_plot)-len(cases_obs))
mpl_dates_for_obs = conv_time_to_mpl_dates(time_for_obs) + diff_data_sim + num_days_data

start_date = mpl_dates[0]
diff_cases = np.diff(cases_obs)
ax.plot(mpl_dates_for_obs, np.diff(cases_obs_to_plot), 'd', label='Confirmed new cases',
        markersize=4, color='tab:blue', zorder=5)

new_cases_past = trace.new_cases[:,:num_days_data]
percentiles = np.percentile(new_cases_past, q=2.5, axis=0), np.percentile(new_cases_past, q=97.5, axis=0)
ax.plot(mpl_dates, np.median(new_cases_past, axis=0),color='tab:green', linewidth=3, zorder=-1)
ax.fill_between(mpl_dates, percentiles[0], percentiles[1], alpha=0.3, color='tab:green')

ax.plot([],[], label=legends[0], alpha=0)
for trace_scen, color, legend in zip(posterior, colors, legends[1:]):
    new_cases_past = trace_scen.new_cases[:,:num_days_data]
    ax.plot(mpl_dates, np.median(new_cases_past, axis=0),'--',color=color, linewidth=1.5)

    time2 = np.arange(0, num_days_future+1)
    mpl_dates_fut = conv_time_to_mpl_dates(time2) + diff_data_sim + num_days_data
    end_date = mpl_dates_fut[-3]
    cases_future = trace_scen['new_cases'][:, num_days_data:].T

    median = np.median(cases_future, axis=-1)
    percentiles = (
    np.percentile(cases_future, q=2.5, axis=-1),
    np.percentile(cases_future, q=97.5, axis=-1),
    )

    ax.plot(mpl_dates_fut[1:], median, color=color, linewidth=3, label=legend)
    ax.fill_between(mpl_dates_fut[1:], percentiles[0], percentiles[1], alpha=0.15, color=color)

ax.set_xlabel('Date')
ax.set_ylabel('New confirmed cases\nin South Africa')
ax.legend(loc='upper left')
# ax.set_ylim(0, 450_000)

ax.locator_params(axis="y", nbins=4)

ax.xaxis.set_major_locator(matplotlib.dates.WeekdayLocator(interval=2, byweekday=matplotlib.dates.SU))
ax.xaxis.set_minor_locator(matplotlib.dates.DayLocator())
ax.xaxis.set_major_formatter(matplotlib.dates.DateFormatter('%m/%d'))
ax.set_xlim(start_date, end_date)
ax.xaxis.set_ticks_position('both')

ax.grid()
ax=axes[1]

time1 = np.arange(-len(cases_obs)+2, 1)
mpl_dates = conv_time_to_mpl_dates(time1) + diff_data_sim + num_days_data
ax.plot(mpl_dates_for_obs, cases_obs_to_plot[1:], 'd', label='Confirmed cases', markersize=4, color='tab:blue',
            zorder=5)

cum_cases = np.cumsum(new_cases_past, axis=1) + cases_obs[0]
percentiles = np.percentile(cum_cases, q=2.5, axis=0), np.percentile(cum_cases, q=97.5, axis=0)
ax.plot(mpl_dates, np.median(cum_cases, axis=0),color='tab:green', linewidth=3, zorder=-1)
ax.fill_between(mpl_dates, percentiles[0], percentiles[1], alpha=0.3, color='tab:green')

ax.plot([],[], label=legends[0], alpha=0)
for trace_scen, color, legend in zip(posterior, colors, legends[1:]):
    new_cases_past = trace_scen.new_cases[:,:num_days_data]
    cum_cases = np.cumsum(new_cases_past, axis=1) + cases_obs[0]
    ax.plot(mpl_dates, np.median(cum_cases, axis=0),'--',color=color, linewidth=1.5)

    time2 = np.arange(0, num_days_future+1)
    mpl_dates_fut = conv_time_to_mpl_dates(time2) + diff_data_sim + num_days_data
    cases_future = np.cumsum(trace_scen['new_cases'][:, num_days_data:].T, axis=0) + cases_obs[-1]

    median = np.median(cases_future, axis=-1)
    percentiles = (
        np.percentile(cases_future, q=2.5, axis=-1),
        np.percentile(cases_future, q=97.5, axis=-1),
    )

    ax.plot(mpl_dates_fut[1:], median, color=color, linewidth=3, label=legend)
    ax.fill_between(mpl_dates_fut[1:], percentiles[0], percentiles[1], alpha=0.15, color=color)

ax.set_xlabel('Date')
ax.set_ylabel('Total confirmed cases\nin South Africa')
ax.legend(loc='upper left')
# ax.set_ylim(0, 32_000_000)

ax.locator_params(axis="y", nbins=4)
func_format = lambda num, _: "${:.0f}\,$m".format(num/1_000_000)
ax.yaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(func_format))
ax.set_xlim(start_date, end_date)
ax.xaxis.set_major_locator(matplotlib.dates.WeekdayLocator(interval=2,byweekday=matplotlib.dates.SU))
ax.xaxis.set_minor_locator(matplotlib.dates.DayLocator())
ax.xaxis.set_major_formatter(matplotlib.dates.DateFormatter('%m/%d'))

ax.set_xlim(start_date, end_date)

ax.grid()
```

```python

```


# ***Introduction to Radar Using Python and MATLAB***
## Andy Harrison - Copyright (C) 2019 Artech House
<br/>

# Rectangular Plate Radar Cross Section
***

Referring to Section 7.4.1.4, consider the scattering from a three-dimensional object; specifically, a rectangular conducting plate, as shown in Figure 7.9.

For TEx polarization, the bistatic radar cross section can be written as (Equation 7.35)

$$
    \sigma = 4\pi\left( \frac{ab}{\lambda}\right)^2 (\cos^2\theta_o \sin^2\phi_o + \cos^2\phi_o)\left[ \frac{\sin(X)}{X}\right]^2\, \left[ \frac{\sin(Y)}{Y}\right]^2 \hspace{0.0in} \text{(m}^2\text{)},
$$

For the TMx polarization, the bistatic radar cross section can be written as (Equation 7.38)

\begin{align}
    \sigma = 4\pi\left( \frac{ab}{\lambda}\right)^2\big[\cos^2\theta_i (&\cos^2\theta_o \cos^2\phi_o + \sin^2\phi_o)\big]\, \nonumber \\ &\times\left[ \frac{\sin(X)}{X}\right]^2\, \left[ \frac{\sin(Y)}{Y}\right]^2 \hspace{0.5in} \text{(m}^2\text{)},
\end{align}

For the monostatic case, the expressions in (Equation 7.35) and (Equation 7.38) reduce to the same equation, given here as

$$
    \sigma = 4\pi\left( \frac{ab}{\lambda}\right)^2\cos^2\theta_i \left[ \frac{\sin(kb\sin\theta_i)}{kb\sin\theta_i}\right]^2 \hspace{0.5in} \text{(m}^2\text{)},
$$

For the TMx polarization, the maximum of scattered field always occurs at the specular angle.  However, this is not true for the TEx polarization, although it does approach the specular angle as the dimensions of the plate become large compared to the wavelength.  As well, the monostatic radar cross section for the flat plate is the same for both TMx and TEx polarizations.  This is also true for the two-dimensional plate.  Measurements have shown that these two values are slightly different.
***

Begin by getting the library path

```python
import lib_path
```

Set the operating frequency (Hz), the plate width (m), the plate length (m), the incident angle theta (degrees) and the observation angle (phi) (degrees)

```python
frequency = 1e9

width = 2.0

length = 5.0

incident_theta = 15.0

observation_phi = 270.0
```

Set up the observation angles theta (degrees) using the `linspace` routine from `scipy`

```python
from numpy import linspace

observation_theta = linspace(-90, 90, 1801)
```

Calculate the radar cross section (m^2) of the rectangular plate

```python
from Libs.rcs.rectangular_plate import radar_cross_section

rcs_tm, rcs_te = radar_cross_section(frequency, width, length, incident_theta, observation_theta, observation_phi)
```

Display the radar cross section (dBsm) of the rectangular plate

```python
from matplotlib import pyplot as plt

from numpy import log10

# Set the figure size

plt.rcParams["figure.figsize"] = (15, 10)

# Display the results

plt.plot(observation_theta, 10.0 * log10(rcs_te + 1e-10), '', label='TE$^x$')

plt.plot(observation_theta, 10.0 * log10(rcs_tm + 1e-10), '--', label='TM$^x$')

# Set the plot title and labels

plt.title('RCS vs Bistatic Angle', size=14)

plt.ylabel('RCS (dBsm)', size=12)

plt.xlabel('Observation Angle (deg)', size=12)

plt.ylim(min(10.0 * log10(rcs_te + 1e-4)) - 3, max(10.0 * log10(rcs_te + 1e-4)) + 3)

# Set the tick label size

plt.tick_params(labelsize=12)

# Turn on the grid

plt.grid(linestyle=':', linewidth=0.5)

# Set the legend

plt.legend(loc='upper right', prop={'size': 10})
```

```python

```


```
import math
import numpy as np
import matplotlib.pyplot as plt
from skimage import data
from skimage.transform import SimilarityTransform
from sympy import Matrix as M
from skimage import img_as_float
from chessocr import *
from fastai.data.all import *
from random import choice
from PIL import Image
```

```
img = Image.open(choice(get_image_files(boards_url))).convert("RGBA")
img.resize((100, 100))
```

```
img_as_float(img).shape
```

    (400, 400, 4)

```
tform = transform.SimilarityTransform(scale=1, rotation=0, translation=(-100,-100))
```

```
tf_img = transform.warp(img_as_float(img), tform.inverse)
fig, ax = plt.subplots()
ax.imshow(tf_img)
_ = ax.set_title('Euclidean transformation')
```

## Euclidean Transform

```
tf_img.shape
```

    (400, 400, 4)

```
img_as_float(img).dtype
```

    dtype('float64')

```

```


# Convective Velocity Profiles

Exploring convective velocity prediction from mixing length theory (MLT) as a function of different variables and at different evolutionary stages. The idea is to develop a phenomenological model for effects of dynamic pressure on stellar structure, particularly for RGB and AGB stars.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
```

If we define the dyanmic pressure as a turbulent pressure within the convection zone, then a first approximation is to use the mixing length theory velocity as a basis for the turbulent velocity. That is, we can define
\begin{equation}
    P_{\rm dyn} \approx P_{\rm turb} \approx C_{\rm turb}\rho u_{\rm mlt}^2,
\end{equation}
where $C_{\rm turb} = 1/3$ for isotropic turbulence. However, it's well known that mixing length theory is not valid in super-adiabatic regions. Mixing length theory is derived assuming adibatic convection. That is, assuming radiation does not carry a significnat fraction of energy away from convecting element and assuming that convective velocities are sub-sonic.

Let's load an envelope structure for a solar-mass star at 5 Gyr.

```python
env = np.genfromtxt('data/m1000_GS98_p000_p0_KS66_SolarCal.env')
```

A convective velocity profile for solar-mass star at 5 Gyr looks like this:

```python
fig, ax = plt.subplots(1, 1, figsize=(8., 5.))

ax.set_xlabel('log(P / [dyne/cm^2])', size=16.)
ax.set_ylabel('convective velocity', size=16.)
ax.set_xlim(7.0, 4.5)

ax.plot(env[:, 1], env[:, 14]/1.0e5, '-', lw=3, c='black')
```

This produces the following profile for the dynamical pressure contribution,

```python
fig, ax = plt.subplots(1, 1, figsize=(8., 5.))

ax.set_xlabel('log(P / [dyne/cm^2])', size=20.)
ax.set_ylabel('Pturb / Pgas', size=20.)
ax.set_xlim(7.0, 4.5)

ax.plot(env[:, 1], (10**env[:, 5]*env[:, 14]**2/3.0)/10**env[:, 1], '-', lw=3, c='black')
```

A critical feature of dynamical pressure that has been ignored, is that dynamical pressure extends beyond the nominal convective boundary. Waves and shocks can propagate across the boundary affecting higher atmospheric layers. This aspect of dynamical pressure is one we aim to introduce.

To do this, we can introduce a phenomenological model in the stellar atmospheric layers and superficial layers in the envelope. This model can be superposed with the dynamic pressure profile predicted from mixing length theory.

The phenomenological model can be controlled by a few parameters. First, the footpoint (in pressure or density space) that defines the zero-point in the envelope. Second, the gradient of the dynamic pressure as a function of pressure or density. Third, a peak value, likely related to the local sound speed. A fourth value may also be considered, which is a scaling factor to everywhere alter the dynamic pressure.

Above, we looked at dynamic pressure as a function of the gas pressure. But let's look instead as a function of the density. Since the dynamic pressure is a function of density already, this may be a suitable choice.

```python
fig, ax = plt.subplots(1, 1, figsize=(8., 5.))

ax.set_xlabel('log(rho / [g/cm^3])', size=20.)
ax.set_ylabel('P_dynamic / P_gas', size=20.)
ax.set_xlim(-5.0, -7.0)

ax.grid()

ax.plot(env[:, 5], (10**env[:, 5]*env[:, 14]**2/3.0)/10**env[:, 1], '-', lw=3, c='black')
fig.savefig('Pdyn_rho.pdf')
```

Now, let's load the atmospheric profile for this model.

```python
atm = np.genfromtxt('data/m1000_GS98_p000_p0_KS66_SolarCal.atm')
```

Define a function to compute the atmospheric dynamical pressure profile.

```python
def dynamicPressure(logD, logP, gamma_one, foot_point = -6.0, scale = 1.0e-3):
    dynamic_pressure_ratio = scale*(foot_point - logD)**3
    dynamic_pressure_ratio = np.array([max(0.0, x) for x in dynamic_pressure_ratio])
    return np.array([min(dynamic_pressure_ratio[i], gamma_one[i]) for i in
                     range(len(dynamic_pressure_ratio))])
```

```python
all_logd   = np.append(atm[:, 3], env[:, 5])
all_logP   = np.append(atm[:, 1], env[:, 1])
all_gamma  = np.append(atm[:,10], env[:,15])
all_dyn_pr = dynamicPressure(all_logd, all_logP, all_gamma)
```

Here's what that linear function looks like:

```python
fig, ax = plt.subplots(1, 1, figsize=(8., 5.))

ax.set_xlabel('log(rho / [g/cm^3])', size=20.)
ax.set_ylabel('Pturb / Pgas', size=20.)
ax.set_xlim(-5.0, -10.0)

ax.grid()

ax.plot(all_logd, all_dyn_pr, '-', lw=3, c='black')
```

Now linearly combine it with the profile generate from MLT.

```python
mlt_dyn_pr = np.append(np.zeros(len(atm)), (10**env[:, 5]*env[:, 14]**2/3.0)/10**env[:, 1])
tot_dyn_pr = all_dyn_pr + mlt_dyn_pr
```

```python
fig, ax = plt.subplots(1, 1, figsize=(8., 5.))

ax.set_xlabel('log(rho / [g/cm^3])', size=20.)
ax.set_ylabel('P_dynamic / P_gas', size=20.)
ax.set_xlim(-5.0, -9.0)

ax.grid()

ax.plot(all_logd, tot_dyn_pr, '-', lw=3, c='black')
fig.savefig('Pdyn_rho_ext.pdf')
```


We remind here the expression of the curvature for a given set of parametric equations $(x(t), y(t))$ :
$$ c(t) = \frac{x'(t) y''(t) - y'(t) x''(t) }{\left(x'(t)^2 + y'(t)^2 \right)^{3/2}}$$

Let :
$$ x(t) = t R_0 \cos\frac{\pi}{4}$$
$$ y(t) = 2 R_0 \mathrm{sech}\left( \frac{x(t)}{R_0} \right) = \frac{2 R_0}{\cosh\left( \frac{x(t)}{R_0} \right)}$$

```
from sympy import *
```

For a nice $\LaTeX$ printing :

```
init_printing()
```

Defining the symbolic notations :

```
t, r, a, b = symbols('t r a b')
```

```
x = -r * sin(t*pi/4)
y = a/cosh(sin(t*pi/4))+b
```

```
(diff(y,t)+1).subs(t,-1)
```

```
solve(_, a)
```
x,y
Calculating the differential expressions

```
xp = diff(x, t)
yp = diff(y, t).simplify()
xpp = diff(xp, t)
ypp = diff(yp, t).simplify()
```

```
xp, yp, xpp, ypp
```

$$\begin{pmatrix}- \frac{\pi r}{4} \cos{\left (\frac{\pi t}{4} \right )}, & - \frac{\pi a \cos{\left (\frac{\pi t}{4} \right )} \sinh{\left (\sin{\left (\frac{\pi t}{4} \right )} \right )}}{4 \cosh^{2}{\left (\sin{\left (\frac{\pi t}{4} \right )} \right )}}, & \frac{\pi^{2} r}{16} \sin{\left (\frac{\pi t}{4} \right )}, & \frac{\pi^{2} a}{16 \cosh{\left (\sin{\left (\frac{\pi t}{4} \right )} \right )}} \left(\sin{\left (\frac{\pi t}{4} \right )} \tanh{\left (\sin{\left (\frac{\pi t}{4} \right )} \right )} + \cos^{2}{\left (\frac{\pi t}{4} \right )} - \frac{2 \cos^{2}{\left (\frac{\pi t}{4} \right )}}{\cosh^{2}{\left (\sin{\left (\frac{\pi t}{4} \right )} \right )}}\right)\end{pmatrix}$$

```
curvature = (xp * ypp - yp * xpp) / (xp**2 + yp**2)**Rational(3,2)
```

```
curvature
```

```
curvature.simplify()
```

```
equa = r*cosh(a*t) - 1/curvature
```

```
equa.simplify()
```

```
u = symbols('u')
diff(1/cosh(a*u), u)
```

```
sinh(-u)
```

```
cosh(-u)
```

```
1/cosh(-sqrt(2)/2)
```

```
f = Function('f')(t)
diff(a/cosh(f/r)+b, t)
```

```
diff(-r*sin(t*pi/4),t)
```

```
cosh(a)**2/sinh(a)
```

```
expand(_)
```

```
cosh(sqrt(2)/2)
```

```

```

Reflectometry/E-plane bend with hyperbolic secant curvature.ipynb


**Note.** *The following notebook contains code in addition to text and figures. By default, the code has been hidden. You can click the icon that looks like an eye in the toolbar above to show the code. To run the code, click the cell menu, then "run all".*

```python
# Import packages, set preferences, etc.
%matplotlib inline
from brian2 import *
import ipywidgets as ipw
from numpy.random import poisson
import warnings
warnings.filterwarnings("ignore")

prefs.codegen.target = 'cython'
defaultclock.dt = 0.05*ms
```

```python
%%html
<!-- hack to improve styling of ipywidgets sliders -->
<style type="text/css">
.widget-label {
    min-width: 35ex;
    max-width: 35ex;
}
.widget-hslider {
    width: 100%;
}
</style>
```

# Basic model

We start from the following model of a chopper cell in the ventral cochlear nucleus. The cell is a leaky integrate and fire neuron, defined by the differential equation $$\tau\frac{\mathrm{d}v}{\mathrm{d}t}=-v.$$ If $v>1$ it fires a spike and instantly resets to $v\leftarrow 0$ and is held there for time $t_\mathrm{ref}$. It receives $N$ excitatory and $N$ inhibitory synapses from neurons that fire Poisson spikes at rates $\rho_E$ and $\rho_I$ respectively. The excitatory synapses have weight $w$ and the inhibitory $-w$, so an incoming spike causes an instantaneous change $v\leftarrow v\pm w$. We will sometimes write the inhibitory fraction $\alpha=\rho_I/\rho_E$ and reparametrise the pair $(\rho_E, \rho_I)$ as $(\rho, \alpha)$ where $\rho_E=\rho$ and $\rho_I=\alpha\rho$.

The following interactive figure allows you to visualise the behaviour of this model for different values of the parameters $\tau$, $N$, $w$, $\rho$, $\alpha$, $t_\mathrm{ref}$.

```python
def basic_model(N=50, tau_ms=10, w=0.02, rho_Hz=200, alpha=0.0, t_ref_ms=0.1):
    # Set parameters
    repeats = 100
    duration = 100*ms
    tau = tau_ms*ms
    rho = rho_Hz*Hz
    t_ref = t_ref_ms*ms
    # Define and run the model
    eqs = '''
    dv/dt = -v/tau : 1 (unless refractory)
    '''
    G = NeuronGroup(repeats, eqs, threshold='v>1', reset='v=0',
                    refractory=t_ref, method='euler')
    Pe = PoissonInput(G, 'v', N, rho, weight=w)
    Pi = PoissonInput(G, 'v', N, rho*alpha, weight=-w)
    spikemon = SpikeMonitor(G)
    statemon = StateMonitor(G, 'v', record=[0])
    run(duration)
    # Compute ISI histograms
    isi = []
    for train in spikemon.spike_trains().values():
        train.sort()
        isi.append(diff(train))
    isi = hstack(isi)
    cv = std(isi)/mean(isi)
    # Plot results
    figure(figsize=(10, 2.5))
    subplot(131)
    plot(spikemon.t/ms, spikemon.i, '.k')
    xlabel('Time (ms)')
    ylabel('Repeat number')
    title('Spike raster plot')
    xlim(0, duration/ms)
    ylim(0, repeats)

    subplot(132)
    plot(statemon.t/ms, statemon.v.T, '-k')
    xlabel('Time (ms)')
    ylabel('v')
    title('Membrane potential trace')
    xlim(0, duration/ms)
    ylim(-0.2, 1.2)
    axhline(0, ls=':', c='r')
    axhline(1, ls=':', c='g')

    subplot(133)
    hist(isi/ms, fc='k', bins=arange(60)*0.5)
    yticks([])
    ylabel('Frequency')
    xlabel('ISI (ms)')
    title('Interspike interval histogram')
    #title('CV = %.2f' % cv)
    text(0.95, 0.9, 'CV = %.2f' % cv, ha='right', va='top',
         bbox=dict(facecolor='white'),
         transform=gca().transAxes)
    tight_layout()

display(ipw.interact(basic_model,
             N=ipw.IntSlider(
                 min=1, max=100, step=1, value=50,
                 continuous_update=False,
                 description="Number of inputs $N$"),
             tau_ms=ipw.FloatSlider(
                 min=0.1, max=20.0, step=0.1, value=10.0,
                 continuous_update=False,
                 description=r"Membrane time constant $\tau$ (ms)"),
             rho_Hz=ipw.FloatSlider(
                 min=10, max=500, step=10, value=200.0,
                 continuous_update=False,
                 description=r"Excitatory input firing rate $\rho$ (sp/s)"),
             w=ipw.FloatSlider(
                 min=0, max=1, step=0.01, value=0.02,
                 continuous_update=False,
                 description=r"Synaptic weight $w$"),
             alpha=ipw.FloatSlider(
                 min=0, max=1.5, step=0.05, value=0,
                 continuous_update=False,
                 description=r"Inhibitory fraction $\alpha$"),
             t_ref_ms=ipw.FloatSlider(
                 min=0, max=5, step=0.05, value=0.1,
                 continuous_update=False,
                 description=r"Refractory period $t_\mathrm{ref}$ (ms)"),
             ));
```

## Diffusion approximation

We now simplify the model using the *diffusion approximation* of [Fourcaud and Brunel (2002)](http://www.ncbi.nlm.nih.gov/pubmed/12184844). The equations above can be approximated by a leaky integrate and fire neuron with the stochastic differential equation $$\tau\frac{\mathrm{d}v}{\mathrm{d}t}=-v+\mu+\sigma\sqrt{\tau}\xi,$$ where $\xi$ is a stochastic differential and the constants are computed as
$$\begin{align}
\mu &= wN\tau(\rho_E-\rho_I)=wN\tau\rho(1-\alpha)\\
\sigma^2 &= w^2N\tau(\rho_E+\rho_I)=w^2N\tau\rho(1+\alpha).
\end{align}$$

This approximation eliminates the need for the input neurons and synaptic connections. You can see in the interactive figure below how well this approximation holds. As long as the number of inputs $N$ and the firing rate $\rho$ are reasonably large, the two curves are statistically very similar (although of course since each trajectory is random, they are not close point-by-point and cannot be). Note that in the figure below, the spiking behaviour when $v>1$ has been removed to see the qualitative behaviour of the stochastic differential equation more clearly.

```python
def diffusion_panel(N=50, tau_ms=10, w=0.02, rho_Hz=200, alpha=0.0):
    # Parameters
    tau = tau_ms*ms
    rho = rho_Hz*Hz
    duration = 200*ms
    # diffusion approximation parameters
    mu = w*N*tau*rho*(1-alpha)
    sigma = w*sqrt(N*tau*rho*(1+alpha))
    # Define and run the model
    eqs = '''
    dv/dt = -v/tau : 1
    du/dt = (mu-u)/tau+sigma*xi*tau**-0.5 : 1
    '''
    G = NeuronGroup(1, eqs, method='euler')
    Pe = PoissonInput(G, 'v', N, rho, weight=w)
    Pi = PoissonInput(G, 'v', N, rho*alpha, weight=-w)
    M = StateMonitor(G, ['v', 'u'], record=[0])
    run(duration)
    # Plot
    plot(M.t/ms, M.v[0], label="Original")
    plot(M.t/ms, M.u[0], label="Diffusion approximation")
    yticks([])
    xlabel('Time (ms)')
    ylabel('Membrane potential')
    legend(loc='lower right')

display(ipw.interact(diffusion_panel,
             N=ipw.IntSlider(
                 min=1, max=100, step=1, value=50,
                 continuous_update=False,
                 description=r"Number of inputs $N$:"),
             tau_ms=ipw.FloatSlider(
                 min=0.1, max=20.0, step=0.1, value=10.0,
                 continuous_update=False,
                 description=r"Membrane time constant $\tau$ (ms):"),
             rho_Hz=ipw.FloatSlider(
                 min=10, max=500, step=10, value=200.0,
                 continuous_update=False,
                 description=r"Excitatory input firing rate $\rho$ (sp/s):"),
             w=ipw.FloatSlider(
                 min=0, max=1, step=0.01, value=0.02,
                 continuous_update=False,
                 description=r"Synaptic weight $w$:"),
             alpha=ipw.FloatSlider(
                 min=0, max=1.5, step=0.05, value=0,
                 continuous_update=False,
                 description=r"Inhibitory fraction $\alpha$:"),
            ));
```

## Reduced model

In the diffusion approximation, all the parameters $N$, $\rho_E$, $\rho_I$, $w$ reduce to just $\mu$ and $\sigma$ meaning that they are not independent. It is possible to directly set these values and this gives us a model with four independent parameters $\mu$, $\sigma$, $\tau$, and $t_\mathrm{ref}$. We call this the *reduced model* and you can explore its behaviour with the interactive figure below. Note that the larger you make $\sigma$ compared to $\mu$, the higher the CV (typically). We will explore this more in the next notebook, [Maps](maps.ipynb).

```python
def reduced_model(mu=1.5, sigma=0.5, tau_ms=10, t_ref_ms=0.1):
    # Set parameters
    repeats = 100
    duration = 100*ms
    tau = tau_ms*ms
    t_ref = t_ref_ms*ms
    # Define and run the model
    eqs = '''
    dv/dt = (mu-v)/tau+sigma*xi*tau**-0.5 : 1
    '''
    G = NeuronGroup(repeats, eqs, threshold='v>1', reset='v=0',
                    refractory=t_ref, method='euler')
    spikemon = SpikeMonitor(G)
    statemon = StateMonitor(G, 'v', record=[0])
    run(duration)
    # Compute ISI histograms
    isi = []
    for train in spikemon.spike_trains().values():
        train.sort()
        isi.append(diff(train))
    isi = hstack(isi)
    cv = std(isi)/mean(isi)
    # Plot results
    figure(figsize=(10, 2.5))
    subplot(131)
    plot(spikemon.t/ms, spikemon.i, '.k')
    xlabel('Time (ms)')
    ylabel('Repeat number')
    title('Spike raster plot')
    xlim(0, duration/ms)
    ylim(0, repeats)

    subplot(132)
    plot(statemon.t/ms, statemon.v.T, '-k')
    xlabel('Time (ms)')
    ylabel('v')
    title('Membrane potential trace')
    xlim(0, duration/ms)
    ylim(-0.2, 1.2)
    axhline(0, ls=':', c='r')
    axhline(1, ls=':', c='g')

    subplot(133)
    hist(isi/ms, fc='k', bins=arange(60)*0.5)
    yticks([])
    ylabel('Frequency')
    xlabel('ISI (ms)')
    title('Interspike interval histogram')
    #title('CV = %.2f' % cv)
    text(0.95, 0.9, 'CV = %.2f' % cv, ha='right', va='top',
         bbox=dict(facecolor='white'),
         transform=gca().transAxes)
    tight_layout()

display(ipw.interact(reduced_model,
             tau_ms=ipw.FloatSlider(
                 min=0.1, max=20.0, step=0.1, value=10.0,
                 continuous_update=False,
                 description=r"Membrane time constant $\tau$ (ms):"),
             t_ref_ms=ipw.FloatSlider(
                 min=0, max=5, step=0.05, value=0.1,
                 continuous_update=False,
                 description=r"Refractory period $t_\mathrm{ref}$ (ms):"),
             mu=ipw.FloatSlider(
                 min=0, max=5, step=0.05, value=1.5,
                 continuous_update=False,
                 description=r"Mean current $\mu$:"),
             sigma=ipw.FloatSlider(
                 min=0, max=5, step=0.05, value=0.5,
                 continuous_update=False,
                 description=r"Standard deviation of current $\sigma$:"),
             ));
```


 HW3 - Gaussians and linear regression

$$
\textbf{Team G} \\
\text{Evangelou Sotiris 2159}\\
\text{Kalais Konstantinos 2146}\\
\text{Chatziefremidis Leuteris 2209}\\
$$

1.Mathematical preliminaries

Consider the function :
$$f(x) = \frac{1}{z} exp(ax^2 + bx + c ) , a < 0 $$

a) Finding the maximum of log(f(x))

   Considering  $$ y = log(f(x)) $$ then :

    $$ y'(k) = 0 $$ and
    $$ y''(k) < 0 $$ then we got a maximum at x =k

```python
from sympy import *
import sympy as sy
import numpy as np
from sympy.solvers.inequalities import reduce_inequalities
#Define the function f(x)
x = Symbol('x')
z = Symbol('Z')
a = Symbol('a',positive=False)
b = Symbol('b')
c = Symbol('c')
y = log((1/z)*(exp(a*x**2 +b*x +c)))

#Derivative of log(f(x))
firstDerExp = diff(y,x)

#Second Derivative
secDerExp = diff(firstDerExp,x)

print("First Derivative: "+str(firstDerExp)+"\n")
print("Second Derivative: "+str(secDerExp)+"\n")

#Point where y'(k) = 0
derZero = solve(firstDerExp,x)

print("Point that the first derivative is zero:  k = "+str(derZero[0])+"\n")

print("y''(k) = "+str(secDerExp.subs(x, derZero[0]))+" and because a < 0 then y''(k) <0 ==> maximum at k."+"\n")

```

    First Derivative: (2*a*x + b)*exp(-a*x**2 - b*x - c)*exp(a*x**2 + b*x + c)

    Second Derivative: 2*a*exp(-a*x**2 - b*x - c)*exp(a*x**2 + b*x + c) + (-2*a*x - b)*(2*a*x + b)*exp(-a*x**2 - b*x - c)*exp(a*x**2 + b*x + c) + (2*a*x + b)**2*exp(-a*x**2 - b*x - c)*exp(a*x**2 + b*x + c)

    Point that the first derivative is zero:  k = -b/(2*a)

    y''(k) = 2*a*exp(-c + b**2/(4*a))*exp(c - b**2/(4*a)) and because a < 0 then y''(k) <0 ==> maximum at k.

b) Gauss Density

    $$ g(x) = \frac{1}{\sqrt(2\pi\sigma ^2)}exp(-\frac{1}{2}(\frac{x-\mu }{\sigma })^2) $$

Proving that f(x) is equivalent to g(x)

    $$ f(x) = g(x) $$
    $$ \frac{1}{z} exp(ax^2 + bx + c ) = \frac{1}{\sqrt(2\pi\sigma ^2)}exp(-\frac{1}{2}(\frac{x-\mu }{\sigma })^2)     $$

    First Part:
    $$ \frac{1}{z} = \frac{1}{\sqrt(2\pi\sigma ^2)} $$
    $$ z = \sqrt(2\pi\sigma^2) $$
    $$ \sigma =  \frac{z}{\sqrt(2\pi)} $$

    Second Part:
       $$ ax^2 + bx + c = -\frac{1}{2}(\frac{x-\mu }{\sigma })^2 $$
       $$ ax^2 + bx + c = -\frac{1}{2\sigma^2}(x^2 -2x\mu + \mu^2) $$
       $$ ax^2 + bx + c = -\frac{1}{2\sigma^2}x^2 + \frac{\mu}{\sigma^2}x - \frac{\mu^2}{2\sigma^2} $$
       $$  a = -\frac{1}{2\sigma^2} $$
       $$  b = \frac{\mu}{\sigma^2} $$
       $$  c = - \frac{\mu^2}{2\sigma^2} $$

# 2.Linear Regression

$\text{a)}$
$$\text{Cost function :}$$
$$\text{L(ω,D) = }\sum_{n=1}^{N}\text{(f(x}_{n}\text{,ω) - y}_{n}\text{)}^{2}\text{ + λ(ω}^{T}\text{ω)}$$
$$\text{ = }\sum_{n=1}^{N}\text{(ω}^{T}\text{x}_{n}\text{ - y}_{n}\text{)}^{2}\text{ + λ(ω}^{T}\text{ω)}$$
$$\text{ = }\sum_{n=1}^{N}\text{(x}_{n}^{T}\text{ω}\text{ - y}_{n}\text{)}^{2}\text{ + λ(ω}^{T}\text{ω)}$$
$$\text{ = (Xω - y)}^{T}\text{(Xω - y)}\text{ + λ(ω}^{T}\text{ω)}$$

$\text{where :} $
$$\text{x}_{n}\text{ = [x}_{n1}\text{,....,x}_{nM}\text{]}^{T}$$

$$\text{X = [x}_{1}^{T}\text{,....,x}_{n}^{T}\text{] , }\varepsilon\text{ }\mathbb{R}^{NXM}$$
$$\text{y}\text{ = [y}_{1}\text{,....,y}_{n}\text{]}^{T}$$
$$\text{w}\text{ = [w}_{1}\text{,....,w}_{m}\text{]}^{T}$$

$$\text{L(ω,D) = (y - Xω)}^{T}\text{(y - Xω) + ω}^{Τ}\text{λIω , I = identity matrix of size M}$$
$$\text{ = (y}^{T} \text{ - ω}^{T}\text{X}^{T}\text{)(y - Xω) +ω}^{T}\text{λIω}$$
$$\text{ = y}^{T}\text{y - y}^{T}\text{Xω -ω}^{Τ}\text{X}^{T}\text{y + ω}^{T}\text{X}^{T}\text{Xω + ω}^{T}\text{λIω}$$
$$\text{ = y}^{T}\text{y - y}^{T}\text{Xω - (y}^{T}\text{X)ω + ω}^{T}\text{(X}^{T}\text{X + λI)ω}$$
$$\text{ = y}^{T}\text{y -2y}^{T}\text{Xω + ω}^{T}\text{(X}^{T}\text{X + λI)ω}$$

$$ \frac{\partial L}{\partial ω}\text{ = -2y}^{T}\text{X + 2ω}^{Τ}\text{(Χ}^{T}\text{X + λI) ,**}$$
$$\frac{\partial L}{\partial ω}\text{ = 0}$$
$$\text{-2y}^{T}\text{X + 2ω}^{Τ}\text{(Χ}^{T}\text{X + λI) = 0 }$$
$$\text{ω}^{Τ}\text{(Χ}^{T}\text{X + λI) = y}^{T}\text{X}$$
$$\text{(Χ}^{T}\text{X + λI)ω = Χ}^{Τ}\text{y}$$
$$\text{ω = }\text{(Χ}^{T}\text{X + λI)}^{-1}\text{X}^{T}\text{y}$$
$$\text{(}\sum_{n=1}^{N}\text{x}_{n}\text{x}_{n}^{T}\text{ + λI}_{Μ}\text{)}^{-1}\text{(}\sum_{n=1}^{N}\text{x}_{n}\text{y}_{n}\text{)}$$

$$\text{** }$$
$$
\begin{Bmatrix}
 & f(x) = x^{T}Ax\\
 & y(x) = Ax
\end{Bmatrix}
$$

$$\text{f(x,y) = x}^{T}\text{y} $$
$$\text{f'(x,y) = }\frac{\partial f}{\partial x}\text{ + }\frac{\partial f}{\partial y}\frac{dy}{dx}$$
$$\text{ = }\frac{\partial (x^{T}y)}{\partial x} \text{ + }\frac{\partial (x^{T}y)}{\partial y}\frac{d(Ax)}{dx}$$
$$\text{ = }\frac{\partial (y^{T}x)}{\partial x}\text{ + x}^{T}\text{A }$$
$$\text{ = }y^{T}\text{ + x}^{T}\text{A }$$
$$\text{ = x}^{T}\text{A}^{T}\text{ + x}^{T}\text{A}$$
$$\text{ = x}^{T}\text{(A}^{T}\text{ + A)}$$

$$\text{Here: A}^{T}\text{ = (X}^{T}\text{X + λI)}^{T}\text{ = X}^{T}\text{X + λI = A}$$
$$\text{=> f'(x,y) = 2x}^{T}\text{A}$$

```python

```

```python
from sklearn.linear_model import LinearRegression
import numpy as np
import pandas as pd
from random import *
import matplotlib.pyplot as plt
from sklearn.metrics import r2_score

#############            (b)            #############

seed(2)

lambdas=0
valid_num=1000
omega_true=[2, 1,0,0,0,-.5,0,0,2,0,3,0,0,0,0,0,0]

xTrain=pd.read_csv('hw3xTrain.csv').values
xTest=pd.read_csv('hw3xTest.csv').values
xValidation=pd.read_csv('hw3xValidation.csv').values
yTrain=pd.read_csv('hw3yTrain.csv').values
yTest=pd.read_csv('hw3yTest.csv').values

lr = LinearRegression()
lr.fit(xTrain,yTrain)
omega=lr.coef_.T

yValidation = np.dot(xValidation,omega) + np.random.randn(valid_num-1,1)*10

xxtop=np.dot(xTrain.T,xTrain)
xxtop=xxtop + lambdas*np.eye(len(xxtop))
xtopy=np.dot(xTrain.T,yTrain)

omega_o= np.dot(np.linalg.inv(xxtop),xtopy)

plt.figure(1)
plt.plot(omega_o)
plt.plot(omega_true,'r')
plt.xlabel('Dimension')
plt.ylabel('Weight')
plt.legend(['estimated weights', 'true weights'])
plt.show()

#############            (c)            #############
lambdas=[1,5,10,25,50,75,100,250,500,750,1000]

omega=np.zeros((len(lambdas),len(omega_true)))
errorTrain=np.zeros((len(lambdas),1))
errorTest=np.zeros((len(lambdas),1))
errorValidation=np.zeros((len(lambdas),1))

for k in range(0,len(lambdas)):
    omega[k,:]=np.dot(np.linalg.inv(xxtop+lambdas[k]*np.eye(len(omega_true))),xtopy).T
    temp=omega[k,:]
    temp=np.reshape(temp,(1,len(temp)))

    ypredTrain=np.dot(temp,xTrain.T).T
    ypredTest=np.dot(temp,xTest.T).T
    ypredValidation=np.dot(temp,xValidation.T).T

    errorTrain[k]=np.mean(np.power((ypredTrain-yTrain),2)).T
    errorTest[k]=np.mean(np.power((ypredTest-yTest),2)).T
    errorValidation[k]=np.mean(np.power((ypredValidation-yValidation),2)).T

plt.semilogx(lambdas,errorTrain)
plt.semilogy(lambdas,errorTest,'r')
plt.semilogy(lambdas,errorValidation,'g')
plt.xlabel('lambda')
plt.ylabel('MSE')
plt.legend(['Train','Test','Validation','location','southeast'])
plt.show()

#############            (d)            #############

plt.xlabel('lambda')
plt.ylabel('MSE')
plt.plot(lambdas,errorTest)
plt.show()

bestMSE=np.amin(errorTest)
bestlambdaindex=np.argmin(errorTest)

best_omega=omega[bestlambdaindex,:]
best_lambda=lambdas[bestlambdaindex]

print("Best MSE = %.4f " %bestMSE + ", for λ = %d to give the best generalization performance" %best_lambda)

#############            (f)            #############
plt.plot(best_omega)
plt.plot(omega_true,'r');
plt.xlabel('Dimension')
plt.ylabel('Weight')
plt.legend(['best weights','true weights','location','southeast'])
plt.show()
```

$\bullet$ f) Important dimensions of x for predicting y are those for which the best(predicted) weights
are close to the real weights => D1=[7,9] and the rest are irrelevant


# Hazma Example Model

## Overview

In this notebook, we demonstrate how to construct a user-defined model in ``hazma``. The model we will consider is an effective field theory with a Dirac fermion DM particle which talks to neutral and charged pions through gauge-invariant dimension-5 operators. The Lagrangian for this model is:

\begin{align}
    \mathcal{L} &\supset \dfrac{c_1}{\Lambda}\overline{\chi}\chi\pi^{+}\pi^{-} + \dfrac{c_2}{\Lambda}\overline{\chi}\chi\pi^{0}\pi^{0}
\end{align}

where $c_{1}, c_{2}$ are dimensionless Wilson coefficients and $\Lambda$ is the cut-off scale of the theory. In order to implement this model in ``hazma``, we need to compute the annihilation cross sections and the FSR spectra. The annihilation channels for this model are simply $\bar{\chi}\chi\to\pi^{0}\pi^{0}$ and $\bar{\chi}\chi\to\pi^{+}\pi^{-}$. The computations for the cross sections are straight forward and yield:

\begin{align}
    \sigma(\bar{\chi}\chi\to\pi^{+}\pi^{-}) &= \frac{c_1^2 \sqrt{1-4 \mu _{\pi }^2} \sqrt{1-4 \mu _{\chi }^2}}{32 \pi \Lambda^2}\\
    \sigma(\bar{\chi}\chi\to\pi^{0}\pi^{0}) &= \frac{c_2^2 \sqrt{1-4 \mu_{\pi^{0}}^2} \sqrt{1-4 \mu_{\chi}^2}}{8 \pi \Lambda^2}
\end{align}

where \(Q\) is the center of mass energy, $\mu_{\chi} = m_{\chi}/Q$, $\mu_{\pi} = m_{\pi^{\pm}}/Q$ and $\mu_{\pi^{0}} = m_{\pi^{0}}/Q$. In addition to the cross sections, we need the FSR spectrum for $\overline{\chi}\chi\to\pi^{+}\pi^{-}\gamma$. This is:

\begin{align}
    \dfrac{dN(\bar{\chi}\chi\to\pi^{+}\pi^{-}\gamma)}{dE_{\gamma}} &= \frac{\alpha  \left(2 f(x)-2\left(1-x-2 \mu_{\pi} ^2\right)
   \log \left(\frac{1-x-f(x)}{1-x+f(x)}\right)\right)}{\pi  \sqrt{1-4 \mu_{\pi} ^2} x}
\end{align}

where

\begin{align}
    f(x) &= \sqrt{1-x} \sqrt{1-x-4 \mu_{\pi} ^2}
\end{align}

Below we will demonstrate how we think a user should implement a model.

## Imports

There are various packages that we will be using throughout the notebook:

```python
import numpy as np
import matplotlib.pyplot as plt

from hazma.parameters import neutral_pion_mass as mpi0
from hazma.parameters import charged_pion_mass as mpi
from hazma.parameters import electron_mass as me
from hazma.parameters import qe

from hazma.positron_spectra import charged_pion as pspec_charged_pion
from hazma.decay import neutral_pion, charged_pion

from hazma.theory import TheoryAnn
from hazma.gamma_ray_parameters import (
    energy_res_fermi,
    energy_res_e_astrogam,
    A_eff_e_astrogam,
    A_eff_comptel,
    egret_diffuse,
    fermi_diffuse,
    comptel_diffuse,
    gc_targets,
    gc_bg_model,
    solid_angle_cone,
    solid_angle_rect,
)

# We will ignore warnings since they aren't important for this example
import warnings

warnings.filterwarnings("ignore")
```

## Defining the model

As described in the paper, we will be be defined the model using mixin classes. For ``hazma`` to work properly, we will need to define the following five functions in our model:

1. ``annihilation_cross_section_funcs()``: A function returning a dictionary of the annihilation cross sections functions, each of which take a center of mass energy.
2. ``spectrum_funcs()``: A function returning a dictionary of functions which take photon energies and a center of mass energy and return the gamma-ray spectrum contribution from each final state.
3. ``gamma_ray_lines(e_cm)``: A function returning a dictionary of the gamma-ray lines for a given center of mass energy.
4. ``positron_spectrum_funcs()``: Like ``spectrum_funcs()``, but for positron spectra.
5. ``positron_lines(e_cm)``: A function returning a dictionary of the electron/positron lines for a center of mass energy.

The class representing the model will contain the following parameters:

1. $m_{\chi}$: the DM mass,
2. $c_{1}$: the Wilson coefficient of first effective operator,
3. $c_{2}$: the Wilson coefficient of second effective operator,
5. $\Lambda$: the cut-off scale of the effective theory.

First, we implement cross section mixin class:

```python
class HazmaExampleCrossSection:
    def sigma_xx_to_pipi(self, Q):
        mupi = mpi / Q
        mux = self.mx / Q

        if Q > 2 * self.mx and Q > 2 * mpi:
            sigma = (
                self.c1 ** 2
                * np.sqrt(1 - 4 * mupi ** 2)
                * np.sqrt(1 - 4 * mux ** 2) ** 2
                / (32.0 * self.lam ** 2 * np.pi)
            )
        else:
            sigma = 0.0

        return sigma

    def sigma_xx_to_pi0pi0(self, Q):
        mupi0 = mpi0 / Q
        mux = self.mx / Q

        if Q > 2 * self.mx and Q > 2 * mpi0:
            sigma = (
                self.c2 ** 2
                * np.sqrt(1 - 4 * mux ** 2)
                * np.sqrt(1 - 4 * mupi0 ** 2)
                / (8.0 * self.lam ** 2 * np.pi)
            )
        else:
            sigma = 0.0

        return sigma

    def annihilation_cross_section_funcs(self):
        return {"pi0 pi0": self.sigma_xx_to_pi0pi0, "pi pi": self.sigma_xx_to_pipi}
```

Next we define the total spectrum class. This class will implement the total gamma-ray spectrum from both $\bar{\chi}\chi\to\pi^{+}\pi^{-}$ and $\bar{\chi}\chi\to\pi^{0}\pi^{0}$, including FSR and decays:

```python
class HazmaExampleSpectra:
    def dnde_pi0pi0(self, e_gams, e_cm):
        return 2.0 * neutral_pion(e_gams, e_cm / 2.0)

    def __dnde_xx_to_pipig(self, e_gam, Q):
        # Unvectorized function for computing FSR spectrum
        mupi = mpi / Q
        mux = self.mx / Q
        x = 2.0 * e_gam / Q
        if 0.0 < x and x < 1.0 - 4.0 * mupi ** 2:
            dnde = (
                qe ** 2
                * (
                    2 * np.sqrt(1 - x) * np.sqrt(1 - 4 * mupi ** 2 - x)
                    + (-1 + 2 * mupi ** 2 + x)
                    * np.log(
                        (-1 + np.sqrt(1 - x) * np.sqrt(1 - 4 * mupi ** 2 - x) + x) ** 2
                        / (1 + np.sqrt(1 - x) * np.sqrt(1 - 4 * mupi ** 2 - x) - x) ** 2
                    )
                )
            ) / (Q * 2.0 * np.sqrt(1 - 4 * mupi ** 2) * np.pi ** 2 * x)
        else:
            dnde = 0

        return dnde

    def dnde_pipi(self, e_gams, e_cm):
        return np.vectorize(self.__dnde_xx_to_pipig)(e_gams, e_cm) + 2.0 * charged_pion(
            e_gams, e_cm / 2.0
        )

    def spectrum_funcs(self):
        return {"pi0 pi0": self.dnde_pi0pi0, "pi pi": self.dnde_pipi}

    def gamma_ray_lines(self, e_cm):
        return {}
```

Finally we define the positron/electron spectra from $\bar{\chi}\chi\to\pi^{+}\pi^{-}$ (note that $\bar{\chi}\chi\to\pi^{0}\pi^{0}$ does not contribute to the positron spectrum.)

```python
class HazmaExamplePositronSpectra:
    def dnde_pos_pipi(self, e_ps, e_cm):
        return pspec_charged_pion(e_ps, e_cm / 2.0)

    def positron_spectrum_funcs(self):
        return {"pi pi": self.dnde_pos_pipi}

    def positron_lines(self, e_cm):
        return {}
```

Lastly, we combine all the mixins into a master class. In the master class, we set the attributes of the model via and ``__init__``. Since this is a theory with annihilating dark matter, we subclass `TheoryAnn`. Theories where the dark matter instead decays should subclass `TheoryDec`.

```python
class HazmaExample(
    HazmaExampleCrossSection, HazmaExamplePositronSpectra, HazmaExampleSpectra, TheoryAnn
):
    def __init__(self, mx, c1, c2, lam):
        self.mx = mx
        self.c1 = c1
        self.c2 = c2
        self.lam = lam

    @staticmethod
    def list_annihilation_final_states():
        return ["pi pi", "pi0 pi0"]
```

## Computing Particle Physics Quantities

Now that we've defined our new model, we will demonstrate how to use it to compute various quantities. Let's define out model with the following parameters:

1. $m_{\chi} = 250 \ \mathrm{MeV}$
2. $c_{1} = 1.0$
3. $c_{2} = 1.0$
4. $\Lambda = 100 \ \mathrm{GeV}$

```python
model = HazmaExample(mpi, 1.0, 1.0, 100e3)
```

### Gamma-ray Spectra

Below, we demonstrate how to compute the total gamma ray spectrum (as well as the sub components):

```python
model.mx = 250.0

e_cm = 2.0 * model.mx * (1 + 0.5 * 1e-6)
photon_energies = np.logspace(-3.0, 3.0, num=150)
spectra = model.spectra(photon_energies, e_cm)

plt.figure(dpi=100)
plt.plot(photon_energies, spectra["pi pi"], label=r"$\pi^{+}\pi^{-}$", lw=1)
plt.plot(photon_energies, spectra["pi0 pi0"], label=r"$\pi^{0}\pi^{0}$", lw=1)
plt.plot(photon_energies, spectra["total"], label="Total", c="k")

plt.xlabel(r"$E_{\gamma} (\mathrm{MeV})$", fontsize=16)
plt.ylabel(r"$\frac{dN}{dE_{\gamma}} (\mathrm{MeV}^{-1})$", fontsize=16)
plt.xscale("log")
plt.yscale("log")
plt.xlim([np.min(photon_energies), np.max(photon_energies)])
plt.legend()
```

### Positron Spectrum

```python
model.mx = 250.0
e_cm = 2.0 * model.mx * (1 + 0.5 * 1e-6)

positron_energies = np.logspace(np.log10(me), 3.0, num=150)
spectra = model.positron_spectra(positron_energies, e_cm)

plt.figure(dpi=100)
plt.plot(positron_energies, spectra["total"], c="k")

plt.xlabel(r"$E_{e^{\pm}} (\mathrm{MeV})$", fontsize=16)
plt.ylabel(r"$\frac{dN}{dE_{e^{\pm}}} (\mathrm{MeV}^{-1})$", fontsize=16)
plt.xscale("log")
plt.yscale("log")
plt.xlim([np.min(positron_energies), np.max(positron_energies)])
```

### Annihilation Cross Sections and Branching Fractions

```python
mxs = np.linspace(mpi0, 250.0, num=100)
bfs = {"pi pi": np.zeros_like(mxs), "pi0 pi0": np.zeros_like(mxs)}
css = {
    "pi pi": np.zeros_like(mxs),
    "pi0 pi0": np.zeros_like(mxs),
    "total": np.zeros_like(mxs),
}

for i, mx in enumerate(mxs):
    model.mx = mx
    e_cm = 2.0 * mx + mx * 1e-6

    _bfs = model.annihilation_branching_fractions(e_cm)
    bfs["pi pi"][i] = _bfs["pi pi"]
    bfs["pi0 pi0"][i] = _bfs["pi0 pi0"]

    _css = model.annihilation_cross_sections(e_cm)
    css["pi pi"][i] = _css["pi pi"]
    css["pi0 pi0"][i] = _css["pi0 pi0"]
    css["total"][i] = _css["total"]
```

```python
plt.figure(dpi=100)
plt.plot(mxs, bfs["pi pi"], label=r"$\pi^{+}\pi^{-}$")
plt.plot(mxs, bfs["pi0 pi0"], label=r"$\pi^{0}\pi^{0}$")
plt.yscale("log")
plt.ylabel(r"Branching Fractions", fontsize=16)
plt.xlabel(r"$m_{\chi} \ (\mathrm{MeV})$", fontsize=16)
plt.xlim([np.min(mxs), np.max(mxs)])
plt.legend()
```

```python
plt.figure(dpi=100)
plt.plot(mxs, css["total"], label=r"Total")
plt.plot(mxs, css["pi pi"], "--", label=r"$\pi^{+}\pi^{-}$")
plt.plot(mxs, css["pi0 pi0"], "--", label=r"$\pi^{0}\pi^{0}$")
plt.yscale("log")
plt.ylabel(r"Cross Sections $(\mathrm{MeV}^{-2})$", fontsize=16)
plt.xlabel(r"$m_{\chi} \ (\mathrm{MeV})$", fontsize=16)
plt.xlim([np.min(mxs), np.max(mxs)])
plt.legend()
```

### Limits on $\langle\sigma v\rangle$

Below we compute the limits on $\langle\sigma v\rangle$ for different dark matter masses from MeV gamma-ray telescopes using EGRET data and projecting limits for e-ASTROGAM observations of the Galactic Center. This code should run in under one minute.

```python
T_obs = 3.14e7  # ~ 1 yr
mxs = np.linspace(mpi0, 250.0, num=100)
limits = {"e-ASTROGAM": np.zeros_like(mxs), "EGRET": np.zeros_like(mxs)}

for i, mx in enumerate(mxs):
    model.mx = mx
    limits["e-ASTROGAM"][i] = model.unbinned_limit(
        A_eff_e_astrogam,
        energy_res_e_astrogam,
        T_obs,
        gc_targets["nfw"]["1 arcmin cone"],
        gc_bg_model,
    )
    limits["EGRET"][i] = model.binned_limit(egret_diffuse)
```

```python
plt.figure(dpi=100)
plt.plot(mxs, limits["e-ASTROGAM"], label="e-ASTROGAM")
plt.plot(mxs, limits["EGRET"], label="EGRET")
plt.yscale("log")
plt.ylabel(r"$\langle\sigma v\rangle \ (\mathrm{cm}^3/\mathrm{s})$", fontsize=16)
plt.xlabel(r"$m_{\chi} \ (\mathrm{MeV})$", fontsize=16)
plt.xlim([np.min(mxs), np.max(mxs)])
plt.legend()
```

```python

```


# Aplicando Python para análisis de precios: simulación de escenarios futuros de precios

> En la clase anterior vimos como importar datos de activos de la base de datos de Yahoo Finance usando el paquete pandas-datareader. En esta clase, veremos como pronosticar escenarios de evolución de precios, suponiendo que los rendimientos diarios se distribuyen normalmente. Como esta evolución de precios es aleatoria, utilizaremos la simulación montecarlo (hacer muchas simulaciones de escenarios de evolución de precios) para obtener probabilidades de que los precios de cierre estén encima de un valor umbral y tomar decisiones con base en estas probabilidades.

**Referencias:**
- http://pandas.pydata.org/
- http://www.learndatasci.com/python-finance-part-yahoo-finance-api-pandas-matplotlib/

## 1. Recordemos como descargar datos...

Antes que nada, para poder hacer simular escenarios de predicción de precios, vamos a recordar lo que hicimos en la clase pasada de descargar los datos de Yahoo Finance, utilizando el paquete `data` de la librería `pandas_datareader`.

Esta vez, utilizaremos los datos de precios de cierre ajustados de activos de la compañía Apple en el año 2016 para nuestra aplicación.

```python
# Importamos librerías

```

```python
# Función para descargar precios de cierre ajustados de varios activos a la vez:
def get_closes(tickers, start_date=None, end_date=None):
    # Fecha inicio por defecto (start_date='2010-01-01') y fecha fin por defecto (end_date=today)
    # Frecuencia de muestreo por defecto (freq='d')
    closes = web.DataReader(tickers, start=start_date, end=end_date, data_source='yahoo')['Adj Close']
    closes = closes.sort_index()
    return closes
```

```python
# Descargamos datos...
# Instrumento: Apple

# Fechas de interés (inicio y fin): 2010-2016

# Función DataReader

```

```python
# Graficamos

```

## 2. Simulación de rendimientos diarios

Recordemos que los precios diarios de cierre ajustados no son un proceso estocástico estacionario, pero los rendimientos diarios si lo son. Por tanto calculamos los rendimientos a partir de los precios de cierre, obtenemos sus propiedades estadísticas muestrales y proyectamos los rendimientos. Luego, obtenemos la proyección de los precios.

Para una sucesión de precios $\{S_t\}_{t=0}^{n}$, el rendimiento simple $R_t$ se define como el el cambio porcentual

$$
R_t=\frac{S_t-S_{t-1}}{S_{t-1}}\approx \ln\left(\frac{S_t}{S_{t-1}}\right)=r_t.
$$
para $t=1,\ldots,n$.

Para el ejemplo en curso, ¿cómo calcular esto?

Además, supusimos que los rendimientos diarios eran una variable aleatoria con distribución normal (que se caracteriza con su media y varianza). Por tanto obtenemos la media y desviación estandar muestrales. Hagamos una función que retorne lo anterior.

```python
# Calcular rendimientos diarios y graficarlos

```

Entonces, suponemos que la diferencia logaritmica de los precios (rendimientos diarios) tiene una distribución normal.

¿Cómo se caracteriza una [distribución normal](https://es.wikipedia.org/wiki/Distribuci%C3%B3n_normal)?

```python
# Calculamos media y desviación estándar

```

Habiendo caracterizado los rendimientos diarios como una variable aleatoria normal con la media y la varianza muestral obtenida de los datos del 2016, podemos generar números aleatorios con estas características para simular el comportamiento de los precios de cierre de las acciones en el 2017 (hay un supuesto de que las cosas no cambiarán fundamentalmente).

Sin embargo, cada simulación que hagamos nos conducirá a distintos resultados (los precios siguen evolucionando aleatoriamente). Entonces, lo que haremos es simular varios escenarios para así ver alguna tendencia y tomar decisiones.

Hagamos una una función que simule varios escenarios de rendimientos diarios rendimientos diarios y que devuelva un dataframe con esta simulación.

```python
# Ayuda en la función np.random.randn

```

```python
# Función que simula varios escenarios de rendimientos diarios

```

```python
# Simulamos 100 escenarios para todo el 2017

```

```python
# Mostrar

```

## 3. Proyección de precios de cierre

Por tanto, para calcular los precios, tenemos:

$$\begin{align}
p_i&=p_{i-1}(R_i+1)\\
p_{i+1}&=p_i(R_{i+1}+1)=p_{i-1}(R_i+1)(R_{i+1}+1)\\
&\vdots\\
p_{i+k}&=p_{i-1}(R_i+1)\cdots(R_{i+k}+1).
\end{align}$$

Si hacemos $i=0$ en la última ecuación, tenemos que $p_{k}=p_{-1}\exp(r_0+\cdots+r_{k})$, donde $p_{-1}$ es el último precio reportado en el 2016.

Con los rendimientos, calculamos los precios de cierre...

```python
# Obtenemos los precios

```

Concatenamos y graficamos...

```python
# Concatenar y graficar

```

```python

```

## 4. Probabilidad Precio-Umbral

Ya que tenemos muchos escenarios de precios proyectados, podemos ver varias cosas. Por ejemplo, ¿cuál es la probabilidad de que el precio de cierre sobrepase algún valor umbral en algún momento?

```python
# Umbral de 110% del ultimo precio

# Fechas

# DataFrame de Strikes

# Concatenar y graficar

```

```python
# Comparar cada escenario en cada fecha

# Sumamos para cada fecha y dividimos entre el número de escenarios

# Gráfico de probabilidad

```

___
Entonces, ya aprendimos a bajar datos con pandas-datareader. En específico, a partir de los precios de cierre ajustados obtuvimos los rendimientos diarios.

Suponiendo que los rendimientos diarios son un proceso estocástico estacionario de distribución normal, pudimos caracaterizarlo y proyectar varios escenarios de evolución de los precios (montecarlo).

Con estas proyecciones pudimos calcular probabilidades de sobrepasar cierto precio umbral: toma de decisiones.

<footer id="attribution" style="float:right; color:#808080; background:#fff;">
Created with Jupyter by Esteban Jiménez Rodríguez.
</footer>


[Table of Contents](http://nbviewer.ipython.org/github/rlabbe/Kalman-and-Bayesian-Filters-in-Python/blob/master/table_of_contents.ipynb)

# Designing Nonlinear Kalman Filters

```python
#format the book
%matplotlib inline
from __future__ import division, print_function
from book_format import load_style
load_style()
```

<style>
@import url('http://fonts.googleapis.com/css?family=Source+Code+Pro');
@import url('http://fonts.googleapis.com/css?family=Lora');

//@import url('http://fonts.googleapis.com/css?family=Open+Sans');
//@import url('http://fonts.googleapis.com/css?family=Vollkorn');
//@import url('http://fonts.googleapis.com/css?family=Karla');
//@import url('http://fonts.googleapis.com/css?family=Poppins');
//@import url('http://fonts.googleapis.com/css?family=Arimo');
//@import url('http://fonts.googleapis.com/css?family=Roboto');
//@import url('http://fonts.googleapis.com/css?family=Lato');
//@import url('http://fonts.googleapis.com/css?family=Domine');
//@import url('http://fonts.googleapis.com/css?family=Chivo');
//@import url('http://fonts.googleapis.com/css?family=Cardo');
//@import url('http://fonts.googleapis.com/css?family=Arvo');
//@import url('http://fonts.googleapis.com/css?family=Crimson+Text');
//@import url('http://fonts.googleapis.com/css?family=Ubuntu');
//@import url('http://fonts.googleapis.com/css?family=Fontin');
//@import url('http://fonts.googleapis.com/css?family=Raleway');
//@import url('http://fonts.googleapis.com/css?family=Merriweather');

.CodeMirror pre {
    font-family: 'Source Code Pro', Consolas, monocco, monospace;
}
    div.cell{
        width: 850px;
        margin-left: 0% !important;
        margin-right: auto;
    }
    div.text_cell_render{
        font-family: 'Lora';
        //font-family: 'Open Sans';
        //font-family: 'Karla',verdana,arial,sans-serif;
        //font-family: 'Roboto',verdana,arial,sans-serif;
        //font-family: 'Lato',verdana,arial,sans-serif;
        //font-family: 'Domine',verdana,arial,sans-serif;
        //font-family: 'Chivo',verdana,arial,sans-serif;
        //font-family: 'Cardo',verdana,arial,sans-serif;
        //font-family: 'Arvo',verdana,arial,sans-serif;
        //font-family: 'Poppins',verdana,arial,sans-serif;
        //font-family: 'Ubuntu',verdana,arial,sans-serif;
        //font-family: 'Fontin',verdana,arial,sans-serif;
        //font-family: 'Raleway',verdana,arial,sans-serif;
        //font-family: 'Merriweather',verdana,arial,sans-serif;
        //font-family: 'Crimson Text', verdana,arial,sans-serif;
        //font-family: verdana,arial,sans-serif;
        //font-family: arial,sans-serif;
        line-height: 125%;
        font-size: 130%;
        text-align: justify;
        text-justify:inter-word;
    }
    div.text_cell code {
        background: transparent;
        color: #000000;
        font-weight: 400;
        font-size: 12pt;
        //font-style: bold;
        font-family:  'Source Code Pro', Consolas, monocco, monospace;
   }
    h1 {
        font-family: 'Open sans',verdana,arial,sans-serif;
	}

    div.input_area {
        background: #F6F6F9;
        border: 1px solid #586e75;
    }

    .text_cell_render h1 {
        font-weight: 200;
        font-size: 30pt;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h2 {
        font-family: 'Open sans',verdana,arial,sans-serif;
        text-align: left;
    }
    .text_cell_render h2 {
        font-weight: 200;
        font-size: 16pt;
        font-style: italic;
        line-height: 100%;
        color:#c76c0c;
        margin-bottom: 0.5em;
        margin-top: 1.5em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h3 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h3 {
        font-weight: 200;
        font-size: 14pt;
        line-height: 100%;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 2em;
        display: block;
        white-space: wrap;
        text-align: left;
    }
    h4 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }
    .text_cell_render h4 {
        font-weight: 100;
        font-size: 14pt;
        color:#d77c0c;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
        white-space: nowrap;
    }
    h5 {
        font-family: 'Open sans',verdana,arial,sans-serif;
    }

    .text_cell_render h5 {
        font-weight: 200;
        font-style: normal;
        color: #1d3b84;
        font-size: 16pt;
        margin-bottom: 0em;
        margin-top: 0.5em;
        display: block;
        white-space: nowrap;
    }
    div.output_subarea.output_text.output_pyout {
        overflow-x: auto;
        overflow-y: scroll;
        max-height: 50000px;
    }
    div.output_subarea.output_stream.output_stdout.output_text {
        overflow-x: auto;
        overflow-y: scroll;
        max-height: 50000px;
    }
    div.output_wrapper{
        margin-top:0.2em;
        margin-bottom:0.2em;
}

    code{
        font-size: 6pt;

    }
    .rendered_html code{
    background-color: transparent;
    }
    ul{
        margin: 2em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li li{
        padding-left: 0.2em;
        margin-bottom: 0.2em;
        margin-top: 0.2em;
    }
    ol{
        margin: 2em;
    }
    ol li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
    }
    ul li{
        padding-left: 0.5em;
        margin-bottom: 0.5em;
        margin-top: 0.2em;
    }
    a:link{
       font-weight: bold;
       color:#447adb;
    }
    a:visited{
       font-weight: bold;
       color: #1d3b84;
    }
    a:hover{
       font-weight: bold;
       color: #1d3b84;
    }
    a:focus{
       font-weight: bold;
       color:#447adb;
    }
    a:active{
       font-weight: bold;
       color:#447adb;
    }
    .rendered_html :link {
       text-decoration: underline;
    }
    .rendered_html :hover {
       text-decoration: none;
    }
    .rendered_html :visited {
      text-decoration: none;
    }
    .rendered_html :focus {
      text-decoration: none;
    }
    .rendered_html :active {
      text-decoration: none;
    }
    .warning{
        color: rgb( 240, 20, 20 )
    }
    hr {
      color: #f3f3f3;
      background-color: #f3f3f3;
      height: 1px;
    }
    blockquote{
      display:block;
      background: #fcfcfc;
      border-left: 5px solid #c76c0c;
      font-family: 'Open sans',verdana,arial,sans-serif;
      width:680px;
      padding: 10px 10px 10px 10px;
      text-align:justify;
      text-justify:inter-word;
      }
      blockquote p {
        margin-bottom: 0;
        line-height: 125%;
        font-size: 100%;
      }
</style>

## Introduction

** Author's note: I was initially planning to have a design nonlinear chapter that compares various approaches. This may or may not happen, but for now this chapter has no useful content and I suggest not reading it. **

We see that the Kalman filter reasonably tracks the ball. However, as already explained, this is a silly example; we can predict trajectories in a vacuum with arbitrary precision; using a Kalman filter in this example is a needless complication.

### Kalman Filter with Air Drag

I will dispense with the step 1, step 2, type approach and proceed in a more natural style that you would use in a non-toy engineering problem. We have already developed a Kalman filter that does excellently at tracking a ball in a vacuum, but that does not incorporate the effects of air drag into the model. We know that the process model is implemented with $\textbf{F}$, so we will turn our attention to that immediately.

Notionally, the computation that $\textbf{F}$ computes is

$$x' = Fx$$

With no air drag, we had

$$
\mathbf{F} = \begin{bmatrix}
1 & \Delta t & 0 & 0 & 0 \\
0 & 1 & 0 & 0 & 0 \\
0 & 0 & 1 & \Delta t & \frac{1}{2}{\Delta t}^2 \\
0 & 0 & 0 & 1 & \Delta t \\
0 & 0 & 0 & 0 & 1
\end{bmatrix}
$$

which corresponds to the equations

$$
\begin{aligned}
x &= x + v_x \Delta t \\
v_x &= v_x \\
\\
y &= y + v_y \Delta t + \frac{a_y}{2} {\Delta t}^2 \\
v_y &= v_y + a_y \Delta t \\
a_y &= a_y
\end{aligned}
$$

From the section above we know that our new Euler equations must be

$$
\begin{aligned}
x &= x + v_x \Delta t \\
v_x &= v_x \\
\\
y &= y + v_y \Delta t + \frac{a_y}{2} {\Delta t}^2 \\
v_y &= v_y + a_y \Delta t \\
a_y &= a_y
\end{aligned}
$$

## Realistic 2D Position Sensors

The position sensor in the last example are not very realistic. In general there is no 'raw' sensor that provides (x,y) coordinates. We have GPS, but GPS already uses a Kalman filter to create a filtered output; we should not be able to improve the signal by passing it through another Kalman filter unless we incorporate additional sensors to provide additional information. We will tackle that problem later.

Consider the following set up. In an open field we put two transmitters at a known location, each transmitting a signal that we can detect. We process the signal and determine  how far we are from that signal, with some noise. First, let's look at a visual depiction of that.

```python
import matplotlib.pyplot as plt

circle1=plt.Circle((-4, 0), 5, color='#004080',
                   fill=False, linewidth=20, alpha=.7)
circle2=plt.Circle((4, 0), 5, color='#E24A33',
                   fill=False, linewidth=5, alpha=.7)

fig = plt.gcf()
ax = fig.gca()

plt.axis('equal')
plt.xlim((-10, 10))
plt.ylim((-10, 10))

plt.plot ([-4, 0], [0, 3], c='#004080')
plt.plot ([4, 0], [0, 3], c='#E24A33')
plt.text(-4, -.5, "A", fontsize=16, horizontalalignment='center')
plt.text(4, -.5, "B", fontsize=16, horizontalalignment='center')

ax.add_artist(circle1)
ax.add_artist(circle2)
plt.show()
```

Here I have attempted to show transmitter A, drawn in red, at (-4,0) and a second one B, drawn in blue, at (4,0). The red and blue circles show the range from the transmitters to the robot, with the width illustrating the effect of the $1\sigma$ angular error for each transmitter. Here I have given the blue transmitter more error than the red one. The most probable position for the robot is where the two circles intersect, which I have depicted with the red and blue lines. You will object that we have two intersections, not one, but we will see how we deal with that when we design the measurement function.

This is a very common sensor set up. Aircraft still use this system to navigate, where it is called DME (Distance Measuring Equipment). Today GPS is a much more common navigation system, but I have worked on an aircraft where we integrated sensors like this into our filter along with the GPS, INS, altimeters, etc. We will tackle what is called *multi-sensor fusion* later; for now we will just address this simple configuration.

The first step is to design our state variables. We will assume that the robot is traveling in a straight direction with constant velocity. This is unlikely to be true for a long period of time, but is acceptable for short periods of time. This does not differ from the previous problem - we will want to track the values for the robot's position and velocity. Hence,

$$\mathbf{x} =
\begin{bmatrix}x\\v_x\\y\\v_y\end{bmatrix}$$

The next step is to design the state transition function. This also will be the same as the previous problem, so without further ado,

$$
\mathbf{x}' = \begin{bmatrix}1& \Delta t& 0& 0\\0& 1& 0& 0\\0& 0& 1& \Delta t\\ 0& 0& 0& 1\end{bmatrix}\mathbf{x}$$

The next step is to design the control inputs. We have none, so we set ${\mathbf{B}}=0$.

The next step is to design the measurement function $\mathbf{z} = \mathbf{Hx}$. We can model the measurement using the Pythagorean theorem.

$$
z_a = \sqrt{(x-x_A)^2 + (y-y_A)^2} + v_a\\[1em]
z_b = \sqrt{(x-x_B])^2 + (y-y_B)^2} + v_b
$$

where $v_a$ and $v_b$ are white noise.

We see an immediate problem. The Kalman filter is designed for linear equations, and this is obviously nonlinear. In the next chapters we will look at several ways to handle nonlinear problems in a robust way, but for now we will do something simpler. If we know the approximate position of the robot than we can linearize these equations around that point. I could develop the generalized mathematics for this technique now, but instead let me just present the worked example to give context to that development.

Instead of computing $\mathbf{H}$ we will compute the partial derivative of $\mathbf{H}$ with respect to the robot's position $\mathbf{x}$. You are probably familiar with the concept of partial derivative, but if not, it just means how $\mathbf{H}$ changes with respect to the robot's position. It is computed as the partial derivative of $\mathbf{H}$ as follows:

$$\frac{\partial \mathbf{h}}{\partial \mathbf{x}} =
\begin{bmatrix}
\frac{\partial h_1}{\partial x_1} & \frac{\partial h_1}{\partial x_2} &\dots \\
\frac{\partial h_2}{\partial x_1} & \frac{\partial h_2}{\partial x_2} &\dots \\
\vdots & \vdots
\end{bmatrix}
$$

Let's work the first partial derivative. We want to find

$$\frac{\partial }{\partial x} \sqrt{(x-x_A)^2 + (y-y_A)^2}
$$

Which we compute as

$$
\begin{aligned}
\frac{\partial h_1}{\partial x} &= ((x-x_A)^2 + (y-y_A)^2))^\frac{1}{2} \\
&= \frac{1}{2}\times 2(x-x_a)\times ((x-x_A)^2 + (y-y_A)^2))^{-\frac{1}{2}} \\
&= \frac{x_r - x_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}}
\end{aligned}
$$

We continue this computation for the partial derivatives of the two distance equations with respect to $x$, $y$, $dx$ and $dy$, yielding

$$\frac{\partial\mathbf{h}}{\partial\mathbf{x}}=
\begin{bmatrix}
\frac{x_r - x_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}} & 0 &
\frac{y_r - y_A}{\sqrt{(x_r-x_A)^2 + (y_r-y_A)^2}} & 0 \\
\frac{x_r - x_B}{\sqrt{(x_r-x_B)^2 + (y_r-y_B)^2}} & 0 &
\frac{y_r - y_B}{\sqrt{(x_r-x_B)^2 + (y_r-y_B)^2}} & 0 \\
\end{bmatrix}
$$

That is pretty painful, and these are very simple equations. Computing the Jacobian can be extremely difficult or even impossible for more complicated systems. However, there is an easy way to get Python to do the work for you by using the SymPy module [1]. SymPy is a Python library for symbolic mathematics. The full scope of its abilities are beyond this book, but it can perform algebra, integrate and differentiate equations, find solutions to differential equations, and much more. We will use it to compute our Jacobian!

First, a simple example. We will import SymPy, initialize its pretty print functionality (which will print equations using LaTeX). We will then declare a symbol for NumPy to use.

```python
import sympy
from sympy import init_printing
init_printing(use_latex='png')

phi, x = sympy.symbols('\phi, x')
phi
```

Notice how we use a latex expression for the symbol `phi`. This is not necessary, but if you do it will render as LaTeX when output. Now let's do some math. What is the derivative of $\sqrt{\phi}$?

```python
sympy.diff('sqrt(phi)')
```

We can factor equations.

```python
sympy.factor('phi**3 -phi**2 + phi - 1')
```

SymPy has a remarkable list of features, and as much as I enjoy exercising its features we cannot cover them all here. Instead, let's compute our Jacobian.

```python
from sympy import symbols, Matrix
phi = symbols('\phi')
phi

x, y, xa, xb, ya, yb, dx, dy = symbols('x y x_a x_b y_a y_b dx dy')

H = Matrix([[sympy.sqrt((x-xa)**2 + (y-ya)**2)],
            [sympy.sqrt((x-xb)**2 + (y-yb)**2)]])

state = Matrix([x, dx, y, dy])
H.jacobian(state)
```

In a nutshell, the entry (0,0) contains the difference between the x coordinate of the robot and transmitter A's x coordinate divided by the distance between the robot and A. (2,0) contains the same, except for the y coordinates of the robot and transmitters. The bottom row contains the same computations, except for transmitter B. The 0 entries account for the velocity components of the state variables; naturally the range does not provide us with velocity.

The values in this matrix change as the robot's position changes, so this is no longer a constant; we will have to recompute it for every time step of the filter.

If you look at this you may realize that this is just a computation of x/dist and y/dist, so we can switch this to a trigonometic form with no loss of generality:

$$\frac{\partial\mathbf{h}}{\partial\mathbf{x}}=
\begin{bmatrix}
-\cos{\theta_A} & 0 & -\sin{\theta_A} & 0 \\
-\cos{\theta_B} & 0 & -\sin{\theta_B} & 0
\end{bmatrix}
$$

However, this raises a huge problem. We are no longer computing $\mathbf{H}$, but $\Delta\mathbf{H}$, the change of $\mathbf{H}$. If we passed this into our Kalman filter without altering the rest of the design the output would be nonsense. Recall, for example, that we multiply $\mathbf{Hx}$ to generate the measurements that would result from the given estimate of $\mathbf{x}$ But now that $\mathbf{H}$ is linearized around our position it contains the *change* in the measurement function.

We are forced, therefore, to use the *change* in $\mathbf{x}$ for our state variables. So we have to go back and redesign our state variables.

>Please note this is a completely normal occurrence in designing Kalman filters. The textbooks present examples like this as *fait accompli*, as if it is trivially obvious that the state variables needed to be velocities, not positions. Perhaps once you do enough of these problems it would be trivially obvious, but at that point why are you reading a textbook? I find myself reading through a presentation multiple times, trying to figure out why they made a choice, finally to realize that it is because of the consequences of something on the next page. My presentation is longer, but it reflects what actually happens when you design a filter. You make what seem reasonable design choices, and as you move forward you discover properties that require you to recast your earlier steps. As a result, I am going to somewhat abandon my **step 1**, **step 2**, etc.,  approach, since so many real problems are not quite that straightforward.

If our state variables contain the velocities of the robot and not the position then how do we track where the robot is? We can't. Kalman filters that are linearized in this fashion use what is called a *nominal trajectory* - i.e. you assume a position and track direction, and then apply the changes in velocity and acceleration to compute the changes in that trajectory. How could it be otherwise? Recall the graphic showing the intersection of the two range circles - there are two areas of intersection. Think of what this would look like if the two transmitters were very close to each other - the intersections would be two very long crescent shapes. This Kalman filter, as designed, has no way of knowing your true position from only distance measurements to the transmitters. Perhaps your mind is already leaping to ways of working around this problem. If so, stay engaged, as later sections and chapters will provide you with these techniques. Presenting the full solution all at once leads to more confusion than insight, in my opinion.

So let's redesign our *state transition function*. We are assuming constant velocity and no acceleration, giving state equations of
$$
\dot{x}' = \dot{x} \\
\ddot{x}' = 0 \\
\dot{y}' = \dot{y} \\
\dot{y}' = 0$$

This gives us the the *state transition function* of

$$
\mathbf{F} = \begin{bmatrix}0 &1 & 0& 0\\0& 0& 0& 0\\0& 0& 0& 1\\ 0& 0& 0& 0\end{bmatrix}$$

A final complication comes from the measurements that we pass in. $\mathbf{Hx}$ is now computing the *change* in the measurement from our nominal position, so the measurement that we pass in needs to be not the range to A and B, but the *change* in range from our measured range to our nominal position.

There is a lot here to take in, so let's work through the code bit by bit. First we will define a function to compute $\frac{\partial\mathbf{h}}{\partial\mathbf{x}}$ for each time step.

```python
from math import sin, cos, atan2

def H_of(pos, pos_A, pos_B):
    """ Given the position of our object at 'pos' in 2D, and two
    transmitters A and B at positions 'pos_A' and 'pos_B', return
    the partial derivative of H
    """

    theta_a = atan2(pos_a[1] - pos[1], pos_a[0] - pos[0])
    theta_b = atan2(pos_b[1] - pos[1], pos_b[0] - pos[0])

    return np.array([[0, -cos(theta_a), 0, -sin(theta_a)],
                     [0, -cos(theta_b), 0, -sin(theta_b)]])
```

Now we need to create our simulated sensor.

```python
from numpy.random import randn

class DMESensor(object):
    def __init__(self, pos_a, pos_b, noise_factor=1.0):
        self.A = pos_a
        self.B = pos_b
        self.noise_factor = noise_factor

    def range_of(self, pos):
        """ returns tuple containing noisy range data to A and B
        given a position 'pos'
        """

        ra = math.sqrt((self.A[0] - pos[0])**2 + (self.A[1] - pos[1])**2)
        rb = math.sqrt((self.B[0] - pos[0])**2 + (self.B[1] - pos[1])**2)

        return (ra + randn()*self.noise_factor,
                rb + randn()*self.noise_factor)
```

Finally, we are ready for the Kalman filter code. I will position the transmitters at x=-100 and 100, both with y=-20. This gives me enough space to get good triangulation from both as the robot moves. I will start the robot at (0,0) and move by (1,1) each time step.

```python
import math
from filterpy.kalman import KalmanFilter
import numpy as np
import book_plots as bp

pos_a = (100, -20)
pos_b = (-100, -20)

f1 = KalmanFilter(dim_x=4, dim_z=2)

f1.F = np.array ([[0, 1, 0, 0],
                  [0, 0, 0, 0],
                  [0, 0, 0, 1],
                  [0, 0, 0, 0]], dtype=float)

f1.R *= 1.
f1.Q *= .1

f1.x = np.array([[1, 0, 1, 0]], dtype=float).T
f1.P = np.eye(4) * 5.

# initialize storage and other variables for the run
count = 30
xs, ys = [], []
pxs, pys = [], []

# create the simulated sensor
d = DMESensor(pos_a, pos_b, noise_factor=3.)

# pos will contain our nominal position since the filter does not
# maintain position.
pos = [0, 0]

for i in range(count):
    # move (1,1) each step, so just use i
    pos = [i, i]

    # compute the difference in range between the nominal track
    # and measured ranges
    ra,rb = d.range_of(pos)
    rx,ry = d.range_of((pos[0] + f1.x[0, 0], pos[1] + f1.x[2, 0]))
    z = np.array([[ra - rx], [rb - ry]])

    # compute linearized H for this time step
    f1.H = H_of (pos, pos_a, pos_b)

    # store stuff so we can plot it later
    xs.append(f1.x[0, 0]+i)
    ys.append(f1.x[2, 0]+i)
    pxs.append(pos[0])
    pys.append(pos[1])

    # perform the Kalman filter steps
    f1.predict()
    f1.update(z)

bp.plot_filter(xs, ys)
bp.plot_track(pxs, pys)
plt.legend(loc=2)
plt.show()
```

## Linearizing the Kalman Filter

Now that we have seen an example of linearizing the Kalman filter we are in a position to better understand the math.

We start by assuming some function $\mathbf f$

## Example: A falling Ball

**author's note: ignore this section for now. **

In the **Designing Kalman Filters** chapter I first considered tracking a ball in a vacuum, and then in the atmosphere. The Kalman filter performed very well for vacuum, but diverged from the ball's path in the atmosphere. Let us look at the output; to avoid littering this chapter with code from that chapter I have placed it all in the file `ekf_internal.py'.

```python
import ekf_internal
ekf_internal.plot_ball()
```

We can artificially force the Kalman filter to track the ball by making $Q$ large. That would cause the filter to mistrust its prediction, and scale the kalman gain $K$ to strongly favor the measurments. However, this is not a valid approach. If the Kalman filter is correctly predicting the process we should not 'lie' to the filter by telling it there are process errors that do not exist. We may get away with that for some problems, in some conditions, but in general the Kalman filter's performance will be substandard.

Recall from the **Designing Kalman Filters** chapter that the acceleration is

$$a_x = (0.0039 + \frac{0.0058}{1+\exp{[(v-35)/5]}})*v*v_x \\
a_y = (0.0039 + \frac{0.0058}{1+\exp{[(v-35)/5]}})*v*v_y- g
$$

These equations will be *very* unpleasant to work with while we develop this subject, so for now I will retreat to a simpler one dimensional problem using this simplified equation for acceleration that does not take the nonlinearity of the drag coefficient into account:

$$\begin{aligned}
\ddot{y} &= \frac{0.0034ge^{-y/20000}\dot{y}^2}{2\beta} - g \\
\ddot{x} &= \frac{0.0034ge^{-x/20000}\dot{x}^2}{2\beta}
\end{aligned}$$

Here $\beta$ is the ballistic coefficient, where a high number indicates a low drag.

This is still nonlinear, so we need to linearize this equation at the current state point. If our state is position and velocity, we need an equation for some arbitrarily small change in $\mathbf{x}$, like so:

$$ \begin{bmatrix}\Delta \dot{x} \\ \Delta \ddot{x} \\ \Delta \dot{y} \\ \Delta \ddot{y}\end{bmatrix} =
\large\begin{bmatrix}
\frac{\partial \dot{x}}{\partial x} &
\frac{\partial \dot{x}}{\partial \dot{x}} &
\frac{\partial \dot{x}}{\partial y} &
\frac{\partial \dot{x}}{\partial \dot{y}} \\
\frac{\partial \ddot{x}}{\partial x} &
\frac{\partial \ddot{x}}{\partial \dot{x}}&
\frac{\partial \ddot{x}}{\partial y}&
\frac{\partial \dot{x}}{\partial \dot{y}}\\
\frac{\partial \dot{y}}{\partial x} &
\frac{\partial \dot{y}}{\partial \dot{x}} &
\frac{\partial \dot{y}}{\partial y} &
\frac{\partial \dot{y}}{\partial \dot{y}} \\
\frac{\partial \ddot{y}}{\partial x} &
\frac{\partial \ddot{y}}{\partial \dot{x}}&
\frac{\partial \ddot{y}}{\partial y}&
\frac{\partial \dot{y}}{\partial \dot{y}}
\end{bmatrix}\normalsize
\begin{bmatrix}\Delta x \\ \Delta \dot{x} \\ \Delta \dot{y} \\ \Delta \ddot{y}\end{bmatrix}$$

The equations do not contain both an x and a y, so any partial derivative with both in it must be equal to zero. We also know that $\large\frac{\partial \dot{x}}{\partial x}\normalsize = 0$ and that  $\large\frac{\partial \dot{x}}{\partial \dot{x}}\normalsize = 1$, so our matrix ends up being

$$\mathbf{F} = \begin{bmatrix}0&1&0&0 \\
\frac{0.0034e^{-x/22000}\dot{x}^2g}{44000\beta}&0&0&0
\end{bmatrix}$$

$$\begin{aligned}\ddot{x} &= -\frac{1}{2}C_d\rho A \dot{x}\\
\ddot{y} &= -\frac{1}{2}C_d\rho A \dot{y}-g\end{aligned}$$

```python
from sympy.abc import *
from sympy import *

init_printing(pretty_print=True, use_latex='mathjax')

x1 = (0.0034*g*exp(-x/22000)*((x)**2))/(2*b) - g

x2 = (a*g*exp(-x/c)*(Derivative(x)**2))/(2*b) - g

#pprint(x1)
#pprint(Derivative(x)*Derivative(x,n=2))
#pprint(diff(x2, x))
```

** orphan text
This approach has many issues. First, of course, is the fact that the linearization does not produce an exact answer. More importantly, we are not linearizing the actual path, but our filter's estimation of the path. We linearize the estimation because it is statistically likely to be correct; but of course it is not required to be. So if the filter's output is bad that will cause us to linearize an incorrect estimate, which will almost certainly lead to an even worse estimate. In these cases the filter will quickly diverge. This is where the 'black art' of Kalman filter comes in. We are trying to linearize an estimate, and there is no guarantee that the filter will be stable. A vast amount of the literature on Kalman filters is devoted to this problem. Another issue is that we need to linearize the system using analytic methods. It may be difficult or impossible to find an analytic solution to some problems. In other cases we may be able to find the linearization, but the computation is very expensive. **

## References

[1] http://sympy.org


```python
import pandas as pd
import numpy as np
import altair as alt
```

### Problema 1 [20 puntos].
La oficina de admisión de una escuela de negocios ha usado el coeficiente de pregrado (GPA) y el test de aptitud para estudios de postgrado (GMAT) para decidir cuales postulantes deberían ser admitidos en dicha escuela. La imagen al final muestra los pares de variables para 85 estudiantes que han postulado a esta escuela. Definimos las variables $X_{1}=\mathrm{GPA}, X_{2}=$ GMAT. Las posibles categorizaciones son $\pi_{1}:$ Se admite; $\pi_{2}:$ No se admite; $\pi_{3}$ : Borderline. Usando estos datos determine a través del estimador Naive Bayes y usando regresión logística para respuesta categórica, en cuál categoría se debería clasificar a un estudiantes que obtuvo $X_{1}=3.21$ y $X_{2}=497$.

Solución: Pasamos los datos como csv.

```python
gsb = pd.read_csv('./data/adm_GSB.csv').drop('ap_n', axis=1)
gsb.head()
```

      gpa
      gmat
      pi

      0
      2.96
      596
      1

      1
      3.14
      473
      1

      2
      3.22
      482
      1

      3
      3.29
      527
      1

      4
      3.69
      505
      1

Usaremos `sklearn.naive_bayes.GaussianNB` y `sklearn.linear_mode.LogisticRegression` pare hacer el analisis

```python
from sklearn.naive_bayes import GaussianNB
X = gsb[['gpa','gmat']]
y = gsb.pi
gnb = GaussianNB()
gnb.fit(X, y)
gnb.predict(np.array([3.21, 497]).reshape(1, -1))
```

    array([1], dtype=int64)

```python
from sklearn.linear_model import LogisticRegression
X = gsb[['gpa','gmat']]
y = gsb.pi
clf  = LogisticRegression(random_state=0).fit(X, y)
clf.fit(X, y)
clf.predict(np.array([3.21, 497]).reshape(1, -1))
```

    array([1], dtype=int64)

Podemos ver que ambos metodos concuerdan en insertar al candidato en la el grupo de Admitidos.

### Problema $4.$
Sea $X=\left(X_{1}, X_{2}, X_{3}, X_{4}\right)^{\top}$ un vector aleatorio con media cero y matriz de covarianza $\Sigma$ dada por
$$
\boldsymbol{\Sigma}=\left(\begin{array}{cccc}
1 & \rho_{2} & \rho_{3} & \rho_{4} \\
\rho_{2} & 1 & \rho_{4} & \rho_{3} \\
\rho_{3} & \rho_{4} & 1 & \rho_{2} \\
\rho_{4} & \rho_{3} & \rho_{2} & 1
\end{array}\right)
$$
a) Demuestre que las cuatro componentes principales de $\boldsymbol{X}$ son $\frac{1}{2}\left(X_{1}+X_{2}+X_{3}+X_{4}\right)$, $\frac{1}{2}\left(X_{1}+X_{2}-X_{3}-X_{4}\right), \frac{1}{2}\left(X_{1}-X_{2}+X_{3}-X_{4}\right)$ y $\frac{1}{2}\left(X_{1}-X_{2}-X_{3}+X_{4}\right)$

Usamermos `sympy` para calcualr los valores y vectores propios de la matriz.

```python
import sympy as sp
```

```python
p2 = sp.symbols('p2')
p3 = sp.symbols('p3')
p4 = sp.symbols('p4')

sigma = sp.Matrix([[1,p2,p3,p4],[p2,1,p4,p3],[p3,p4,1,p2],[p4,p3,p2,1]]).T
sigma
```

$\displaystyle \left[\begin{matrix}1 & p_{2} & p_{3} & p_{4}\\p_{2} & 1 & p_{4} & p_{3}\\p_{3} & p_{4} & 1 & p_{2}\\p_{4} & p_{3} & p_{2} & 1\end{matrix}\right]$

```python
sigma.eigenvects()
```

    [(-p2 - p3 + p4 + 1,
      1,
      [Matrix([
       [ 1],
       [-1],
       [-1],
       [ 1]])]),
     (-p2 + p3 - p4 + 1,
      1,
      [Matrix([
       [-1],
       [ 1],
       [-1],
       [ 1]])]),
     (p2 - p3 - p4 + 1,
      1,
      [Matrix([
       [-1],
       [-1],
       [ 1],
       [ 1]])]),
     (p2 + p3 + p4 + 1,
      1,
      [Matrix([
       [1],
       [1],
       [1],
       [1]])])]

### Problema 5.
Consideremos los datos *Bone*, los cuales muestran las longitudes de cierto hueso en 20 niños medidos en 4 edades diferentes. Estos datos están descritos en *Methods of Multivariate Analysis*, escrito por Alvin C. Rencher. Los datos están disponibles en el aula. Realice un análisis de componentes principales usando el conjunto de datos Bone. ¿Es posible reducir la dimensión? ¿Es posible observar clusters en el gráfico de las dos primeras componentes principales usando el comando biplot ()?

Leemos los datos.

```python
bone = pd.read_csv('./data/BONE.csv', header=None, delimiter=r"\s+").drop([0], axis=1)
bone.columns= ['a', 'b', 'c', 'd']
bone.head()
```

      a
      b
      c
      d

      0
      47.8
      48.8
      49.0
      49.7

      1
      46.4
      47.3
      47.7
      48.4

      2
      46.3
      46.8
      47.8
      48.5

      3
      45.1
      45.3
      46.1
      47.2

      4
      47.6
      48.5
      48.9
      49.3

Usaremos un `Boxplot` para de forma rápida ver como distribuyen los datos.

```python
bone.boxplot()
```

Podemos ver que los datos se encuentran en rangos similares

Ahora usaremos `pca` para hacer análisis de componentes principales:

```python
from pca import pca

model = pca(n_components=0.95)
results = model.fit_transform(bone)
```

    [pca] >Processing dataframe..
    [pca] >The PCA reduction is performed to capture [95.0%] explained variance using the [4] columns of the input data.
    [pca] >Fitting using PCA..
    [pca] >Computing loadings and PCs..
    [pca] >Computing explained variance..
    [pca] >Number of components is [2] that covers the [95.00%] explained variance.
    [pca] >Outlier detection using Hotelling T2 test with alpha=[0.05] and n_components=[2]
    [pca] >Outlier detection using SPE/DmodX with n_std=[2]

```python
results['loadings']
```

      a
      b
      c
      d

      PC1
      0.474441
      0.492279
      0.515443
      0.516609

      PC2
      -0.591770
      -0.405545
      0.303532
      0.627065

      PC3
      0.547864
      -0.472574
      -0.514366
      0.460377

      PC4
      -0.352929
      0.608169
      -0.614503
      0.357712

Podemos ver que es posible hacer reducción de dimensión, es más, con tan solo dos dimensiones podemos cubrir un 95% de la varianza.

```python
fig, ax = model.plot()
```

Con esto podemos ver más aun que de hecho una gran cantidad de información se encuentra en la primera componente principal, pero necesitamos las dos primeras para superar eñ 95%.

```python
fig, ax = model.biplot(n_feat=4)
```

En base al `biplot` es posible ver 3 clusters claros. Quizas es posible separar el 10 y el 5 en su propio cluster, pero de forma clara yo apostaría por 3.